From patchwork Mon Oct 24 14:36:39 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Hunleth X-Patchwork-Id: 685894 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3t2f4s38cpz9snm for ; Tue, 25 Oct 2016 01:37:17 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=troodon-software-com.20150623.gappssmtp.com header.i=@troodon-software-com.20150623.gappssmtp.com header.b=Kdco69yl; dkim-atps=neutral Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id A3B628E2A8; Mon, 24 Oct 2016 14:37:15 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gmNWJzvVL4Ek; Mon, 24 Oct 2016 14:37:14 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 4B9F88D667; Mon, 24 Oct 2016 14:37:14 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 344431C16AB for ; Mon, 24 Oct 2016 14:37:13 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 2D780235BA for ; Mon, 24 Oct 2016 14:37:13 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Syi0YTgJS4A2 for ; Mon, 24 Oct 2016 14:37:12 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from mail-qk0-f194.google.com (mail-qk0-f194.google.com [209.85.220.194]) by silver.osuosl.org (Postfix) with ESMTPS id 37C372A0E7 for ; Mon, 24 Oct 2016 14:37:12 +0000 (UTC) Received: by mail-qk0-f194.google.com with SMTP id v138so13300071qka.2 for ; Mon, 24 Oct 2016 07:37:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=troodon-software-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id; bh=vV37S8mvb9/Y66jI/8cEAZT68JlJPuaCQ4hm+7Jvv/I=; b=Kdco69yl25vbpSS8f1yJdOx7DxSkJUBp88qFDblEbWPLCCBAD2LknrSgV/Rr6zlJzP OtUCncqjzNxR0ni3zvh39f6LpEQcjmV6YTlCYUrTosEkaJXCdGKJwvlFsvtVscfRhcjh dIfhEAZffQBOdm7QbehSUeyHuJEuot55MFdKVZE5bdg1Jh3JXeXSFaPzRKjl3p+Jz5Ip /7EIDwGRiiWxp1hcsaRic2vX/Md8M3rlYOFIshuOuQ6JW31t9SRfYFCVm3hN8N1d9/uB dZXZDIu6x/JbmMuxKGjag9ZdpOoP73hLGw0hH2ysUSTxDZ2ceL25RddYBuLtAHLX5nBG Fbqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=vV37S8mvb9/Y66jI/8cEAZT68JlJPuaCQ4hm+7Jvv/I=; b=Do+/k7jemXbdRjx+KbPHH2wB2ciqhg+w3cEgvLVPJfDCa9petfnHE/8P1BhC9zk/co laKFhgs7Y8hMDm/5aICsSdkmeIm51OdRX6eQSRoUMSkV84YnNjheQVuCusGtEJeZclJC U7kuHFX1eK3958ydlW6Ly0GLvH4ibm0c6UfaS9/sjriu0HvDegmFNTsnUH2wpWlNw22G 6YXaH91Vh+D/oUWLPtyLvE1Ru0G3JbgCsXwFmfpYI7dIu4V2/e4b5YfryiVSpsGID9hT PTNcPgX98IHWuOhgjLzSFJEYiOBnlm4R2M6G1HuLqDUzqKUybg0MLJFU9fzRz3bJtwEv PQfg== X-Gm-Message-State: ABUngvfm6xsrsJXwiYjBW0DGBDRt4+fYG6O1rZpPhN4aAQTPi/szTQrGAEjced/g3GgJ3g== X-Received: by 10.55.6.12 with SMTP id 12mr6478272qkg.182.1477319831131; Mon, 24 Oct 2016 07:37:11 -0700 (PDT) Received: from halfmarathon.home.hunleth.com (pool-96-255-73-68.washdc.fios.verizon.net. [96.255.73.68]) by smtp.gmail.com with ESMTPSA id g10sm8497182qkb.17.2016.10.24.07.37.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 24 Oct 2016 07:37:10 -0700 (PDT) From: Frank Hunleth To: buildroot@buildroot.org Date: Mon, 24 Oct 2016 10:36:39 -0400 Message-Id: <1477319799-13570-1-git-send-email-fhunleth@troodon-software.com> X-Mailer: git-send-email 2.7.4 Subject: [Buildroot] [PATCH v2] size-stats: don't count hard links X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" This change adds inode tracking to the size-stats script so that hard links don't cause files to be double counted. This has a significant effect on the size computation for some packages. For example, git has around a dozen hard links to a large file. Before this change, git would weigh in at about 170 MB with the total filesystem size reported as 175 MB. The actual rootfs.ext2 size was around 16 MB. With the change, the git package registers at 10.5 MB with a total filesystem size of 15.8 MB. Signed-off-by: Frank Hunleth Acked-by: Thomas De Schampheleire --- Changes v1 -> v2: - Moved hardlink check to build_package_size() so that hardlinks could also be detected in files belonging to no package - The v1 patch had the effect of listing all but one hardlinked files as having 0 size in file-size-stats.csv. This no longer happens anymore. All files have their listed size. support/scripts/size-stats | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) -- 2.7.4 diff --git a/support/scripts/size-stats b/support/scripts/size-stats index 0ddcc07..af45000 100755 --- a/support/scripts/size-stats +++ b/support/scripts/size-stats @@ -88,11 +88,20 @@ def build_package_dict(builddir): def build_package_size(filesdict, builddir): pkgsize = collections.defaultdict(int) + seeninodes = set() for root, _, files in os.walk(os.path.join(builddir, "target")): for f in files: fpath = os.path.join(root, f) if os.path.islink(fpath): continue + + st = os.stat(fpath) + if st.st_ino in seeninodes: + # hard link + continue + else: + seeninodes.add(st.st_ino) + frelpath = os.path.relpath(fpath, os.path.join(builddir, "target")) if not frelpath in filesdict: print("WARNING: %s is not part of any package" % frelpath) @@ -100,7 +109,7 @@ def build_package_size(filesdict, builddir): else: pkg = filesdict[frelpath][0] - pkgsize[pkg] += os.path.getsize(fpath) + pkgsize[pkg] += st.st_size return pkgsize