From patchwork Wed May 19 02:46:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Weber X-Patchwork-Id: 1480643 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FlHMF0DGQz9sWp for ; Wed, 19 May 2021 12:46:47 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 3918040692; Wed, 19 May 2021 02:46:44 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8uY2bcGynBFd; Wed, 19 May 2021 02:46:43 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp4.osuosl.org (Postfix) with ESMTP id 513A740694; Wed, 19 May 2021 02:46:42 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 4ED111BF966 for ; Wed, 19 May 2021 02:46:41 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 3DF3E83D35 for ; Wed, 19 May 2021 02:46:41 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5lm4lHd6VWJP for ; Wed, 19 May 2021 02:46:40 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from da1vs04.rockwellcollins.com (da1vs04.rockwellcollins.com [205.175.227.52]) by smtp1.osuosl.org (Postfix) with ESMTPS id 6757083D2E for ; Wed, 19 May 2021 02:46:40 +0000 (UTC) IronPort-SDR: lgJqLSUZxQfb7KuDMqTR/V3AMTTQ0TVe/1Gwq7OrNdapoU8qdTmbhM47Us75xlUiTAqeiHRX52 8mQIgPAlxDeG67GlHaCGIUkVlGTcSeRqvawIjNqL+zj9Yf1DR95klsIxm4qnnazdVjZCfHVo63 ps3K9XvXP60roFn2XFYlHeP9W7V9LovYgrJQ1uW3QrNFzjOPV1sPo/LnvYXQQlCMK4UIJVqzOs /XhghIMdKHoMSgYJENrlwGA1Htv/pYYPLXVcZRY2WIy9pkz/BWcDqQ5kuGciucG1uRQJtD1Qoz FZQ= Received: from ofwda1n02.rockwellcollins.com (HELO crulimr02.rockwellcollins.com) ([205.175.227.14]) by da1vs04.rockwellcollins.com with ESMTP; 18 May 2021 21:46:39 -0500 X-Received: from biscuits.rockwellcollins.com (biscuits.rockwellcollins.lab [10.148.119.137]) by crulimr02.rockwellcollins.com (Postfix) with ESMTP id 37CF36024C; Tue, 18 May 2021 21:46:39 -0500 (CDT) To: buildroot@buildroot.org Date: Tue, 18 May 2021 21:46:36 -0500 Message-Id: <20210519024638.63607-1-matthew.weber@collins.com> X-Mailer: git-send-email 2.17.1 Subject: [Buildroot] [PATCH v2 1/3] support/scripts/pkg-stats: verified CPE has a known id but not version X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Matthew Weber via buildroot Reply-To: Matthew Weber Cc: Matthew Weber , "Yann E . MORIN" MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Currently a verified CPE reports the following if versions are not found cpe:2.3:a:qemu:qemu:5.2.0:*:*:*:*:*:*:* CPE identifier unknown in CPE database (Search) This patch clarifies the report to state the 'version' is unknown instead of the 'identifier'. Cc: Yann E. MORIN Signed-off-by: Matthew Weber --- Changes v1 -> v2 - New --- support/scripts/pkg-stats | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats index 0cd3674c52..42c36f7f94 100755 --- a/support/scripts/pkg-stats +++ b/support/scripts/pkg-stats @@ -610,7 +610,7 @@ def check_package_cpes(nvd_path, packages): if cpedb.find(p.cpeid): p.status['cpe'] = ("ok", "verified CPE identifier") else: - p.status['cpe'] = ("error", "CPE identifier unknown in CPE database") + p.status['cpe'] = ("error", "CPE version unknown in CPE database") def calculate_stats(packages): From patchwork Wed May 19 02:46:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Weber X-Patchwork-Id: 1480644 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=busybox.net (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FlHMV0vbjz9sVt for ; Wed, 19 May 2021 12:47:01 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 4A4366066E; Wed, 19 May 2021 02:46:57 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id D7xRpZk869Ul; Wed, 19 May 2021 02:46:56 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp3.osuosl.org (Postfix) with ESMTP id 991EF6066C; Wed, 19 May 2021 02:46:55 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 625CE1C1178 for ; Wed, 19 May 2021 02:46:41 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 50AC9400ED for ; Wed, 19 May 2021 02:46:41 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q-jT4KzrLd0q for ; Wed, 19 May 2021 02:46:40 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from ch3vs03.rockwellcollins.com (ch3vs03.rockwellcollins.com [205.175.226.47]) by smtp2.osuosl.org (Postfix) with ESMTPS id 67849400A7 for ; Wed, 19 May 2021 02:46:40 +0000 (UTC) IronPort-SDR: E6ooHenmHQ1JHsoxSTD8oH0xHKP/DZ0iQR1FSl6jFdY+qPOPZX9td5DZysw5AAakeJ2871qG42 u6mnD0rVH88nwxa0hcC3cu4hNolroF0fNTAiHJlXrdzFTR5KBnvJuVdI0rHcM3epAX7lIIY86f B+tw3/icOnWSM808X+3eD3m1PytIhXBvnt/2fTqAWNsawK17d22Qnyn+JEWTYSMcWT9p3hyiu+ iq3VocLup/IEQjIfV0HihL+CcvzEzBD+4FHhOlgt3pigpLMY5ZapU0MZZty9Q/Ja3SWb8+Pjv2 NFA= Received: from ofwch3n02.rockwellcollins.com (HELO crulimr02.rockwellcollins.com) ([205.175.226.14]) by ch3vs03.rockwellcollins.com with ESMTP; 18 May 2021 21:46:39 -0500 X-Received: from biscuits.rockwellcollins.com (biscuits.rockwellcollins.lab [10.148.119.137]) by crulimr02.rockwellcollins.com (Postfix) with ESMTP id 445D360639; Tue, 18 May 2021 21:46:39 -0500 (CDT) To: buildroot@buildroot.org Date: Tue, 18 May 2021 21:46:37 -0500 Message-Id: <20210519024638.63607-2-matthew.weber@collins.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210519024638.63607-1-matthew.weber@collins.com> References: <20210519024638.63607-1-matthew.weber@collins.com> Subject: [Buildroot] [PATCH v2 2/3] support/scripts/pkg-stats: add is_actual_package() and rework has_valid_infra() X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Matthew Weber via buildroot Reply-To: Matthew Weber Cc: Matthew Weber , "Yann E . MORIN" MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" has_valid_infra() is incorrectly named; it probably should be named is_actual_package(), and has_valid_infra() would be changed to actually represent having an actual infra. This resolves packages reporting as having no valid package infra and cleans up reporting cases of CPE and CVEs where there isn't a valid version or package definition outside Buildroot Cc: Yann E. MORIN Signed-off-by: Matthew Weber --- Changes v1 -> v2 - None --- support/scripts/pkg-stats | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats index 42c36f7f94..0c34388066 100755 --- a/support/scripts/pkg-stats +++ b/support/scripts/pkg-stats @@ -131,7 +131,15 @@ class Package: @property def has_valid_infra(self): + if self.infras is None: + return False + return len(self.infras) > 0 + + @property + def is_actual_package(self): try: + if not self.has_valid_infra: + return False if self.infras[0][1] == 'virtual': return False except IndexError: @@ -159,7 +167,7 @@ class Package: """ Fills in the .status['license'] and .status['license-files'] fields """ - if not self.has_valid_infra: + if not self.is_actual_package: self.status['license'] = ("na", "no valid package infra") self.status['license-files'] = ("na", "no valid package infra") return @@ -177,7 +185,7 @@ class Package: """ Fills in the .status['hash'] field """ - if not self.has_valid_infra: + if not self.is_actual_package: self.status['hash'] = ("na", "no valid package infra") self.status['hash-license'] = ("na", "no valid package infra") return @@ -192,7 +200,7 @@ class Package: """ Fills in the .patch_count, .patch_files and .status['patches'] fields """ - if not self.has_valid_infra: + if not self.is_actual_package: self.status['patches'] = ("na", "no valid package infra") return @@ -220,7 +228,7 @@ class Package: Fills in the .cpeid field """ var = self.pkgvar() - if not self.has_valid_infra: + if not self.is_actual_package: self.status['cpe'] = ("na", "no valid package infra") return @@ -551,13 +559,13 @@ async def check_package_latest_version(packages): package, as known by release-monitoring.org """ - for pkg in [p for p in packages if not p.has_valid_infra]: + for pkg in [p for p in packages if not p.is_actual_package]: pkg.status['version'] = ("na", "no valid package infra") tasks = [] connector = aiohttp.TCPConnector(limit_per_host=5) async with aiohttp.ClientSession(connector=connector, trust_env=True) as sess: - packages = [p for p in packages if p.has_valid_infra] + packages = [p for p in packages if p.is_actual_package] for pkg in packages: tasks.append(asyncio.ensure_future(check_package_latest_version_get(sess, pkg, len(packages)))) await asyncio.wait(tasks) @@ -578,7 +586,7 @@ def check_package_cves(nvd_path, packages): cpe_product_pkgs = defaultdict(list) for pkg in packages: - if not pkg.has_valid_infra: + if not pkg.is_actual_package: pkg.status['cve'] = ("na", "no valid package infra") continue if not pkg.current_version: From patchwork Wed May 19 02:46:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Weber X-Patchwork-Id: 1480645 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FlHMg5Qpsz9sVt for ; Wed, 19 May 2021 12:47:11 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 1582640692; Wed, 19 May 2021 02:47:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Lwz8xbRwPLJg; Wed, 19 May 2021 02:47:09 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp4.osuosl.org (Postfix) with ESMTP id 27BF040E45; Wed, 19 May 2021 02:47:08 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 232CA1BF966 for ; Wed, 19 May 2021 02:46:43 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id CE0B4400A7 for ; Wed, 19 May 2021 02:46:41 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id C-x9E3zV_DAG for ; Wed, 19 May 2021 02:46:40 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from secvs04.rockwellcollins.com (secvs04.rockwellcollins.com [205.175.225.130]) by smtp2.osuosl.org (Postfix) with ESMTPS id AD81F400EB for ; Wed, 19 May 2021 02:46:40 +0000 (UTC) IronPort-SDR: xf9UEh1JvPiLhXB2utOkWB8EnXMK38CGqN/d1yRUPdsqgUla2rm1raoLADfHPXbLsOR0jl08yN bmcVIt4OOyz5tqjX8kxia6kAshPKfVr+9rJCvuOUjfGeYAd3WC5xRp3arNMrGH6T0udye1/OoN urj3zVNuTJ9PYwERPiijGo+hFC1GVd8UwphxAOJ/YYnPhKrTmxz9gvA5pS3VIBnVXn6d2IFqjD m4kSO6PLUcP2sfLihdXOX2ndxWaBLc5ncJSXAF340frr6QDfxd31gEFwSWO/Cp+oE9S7D0OLng vh0= Received: from ofwgwc03.rockwellcollins.com (HELO crulimr02.rockwellcollins.com) ([205.175.225.12]) by secvs04.rockwellcollins.com with ESMTP; 18 May 2021 21:46:39 -0500 X-Received: from biscuits.rockwellcollins.com (biscuits.rockwellcollins.lab [10.148.119.137]) by crulimr02.rockwellcollins.com (Postfix) with ESMTP id 481916064F; Tue, 18 May 2021 21:46:39 -0500 (CDT) To: buildroot@buildroot.org Date: Tue, 18 May 2021 21:46:38 -0500 Message-Id: <20210519024638.63607-3-matthew.weber@collins.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210519024638.63607-1-matthew.weber@collins.com> References: <20210519024638.63607-1-matthew.weber@collins.com> Subject: [Buildroot] [PATCH v2 3/3] support/scripts/pkg-stats: clarify when a CVE/CPE should report as N/A X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Matthew Weber via buildroot Reply-To: Matthew Weber Cc: Matthew Weber , "Yann E . MORIN" MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" - If a package doesn't have any versioning, ignore and state that - If a package is virtual, CVE=ignore and CPE state virtual - For any of these NA cases, don't provide search link and color box green Cc: Yann E. MORIN Signed-off-by: Matthew Weber --- Changes v1 -> v2 [Yann - Update coloring of case when package is virtual, color boxes green for N/A of CPE and CVEs --- support/scripts/pkg-stats | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats index 0c34388066..cc91d13167 100755 --- a/support/scripts/pkg-stats +++ b/support/scripts/pkg-stats @@ -229,7 +229,10 @@ class Package: """ var = self.pkgvar() if not self.is_actual_package: - self.status['cpe'] = ("na", "no valid package infra") + self.status['cpe'] = ("na", "N/A - virtual pkg") + return + if not self.current_version: + self.status['cpe'] = ("na", "no version information available") return if var in self.all_cpeids: @@ -587,7 +590,7 @@ def check_package_cves(nvd_path, packages): cpe_product_pkgs = defaultdict(list) for pkg in packages: if not pkg.is_actual_package: - pkg.status['cve'] = ("na", "no valid package infra") + pkg.status['cve'] = ("na", "N/A") continue if not pkg.current_version: pkg.status['cve'] = ("na", "no version information available") @@ -909,6 +912,8 @@ def dump_html_pkg(f, pkg): td_class.append("cve-ok") elif pkg.is_status_error("cve"): td_class.append("cve-nok") + elif pkg.is_status_na("cve") and not pkg.is_actual_package: + td_class.append("cve-ok") else: td_class.append("cve-unknown") f.write(" \n" % " ".join(td_class)) @@ -936,18 +941,23 @@ def dump_html_pkg(f, pkg): td_class.append("cpe-ok") elif pkg.is_status_error("cpe"): td_class.append("cpe-nok") + elif pkg.is_status_na("cpe") and not pkg.is_actual_package: + td_class.append("cpe-ok") else: td_class.append("cpe-unknown") f.write(" \n" % " ".join(td_class)) if pkg.cpeid: f.write(" %s\n" % pkg.cpeid) if not pkg.is_status_ok("cpe"): - if pkg.cpeid: - f.write("
%s (Search)\n" % # noqa: E501 - (pkg.status['cpe'][1], ":".join(pkg.cpeid.split(":")[0:5]))) + if pkg.is_actual_package and pkg.current_version: + if pkg.cpeid: + f.write("
%s (Search)\n" % # noqa: E501 + (pkg.status['cpe'][1], ":".join(pkg.cpeid.split(":")[0:5]))) + else: + f.write(" %s (Search)\n" % # noqa: E501 + (pkg.status['cpe'][1], pkg.name)) else: - f.write(" %s (Search)\n" % - (pkg.status['cpe'][1], pkg.name)) + f.write(" %s\n" % pkg.status['cpe'][1]) f.write(" \n")