From patchwork Mon Sep 21 10:15:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory CLEMENT X-Patchwork-Id: 1368201 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=busybox.net (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Bw0gh6431z9sTp for ; Mon, 21 Sep 2020 20:15:28 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 20E7787184; Mon, 21 Sep 2020 10:15:27 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VTtfPNdW2txT; Mon, 21 Sep 2020 10:15:26 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 6859E871BA; Mon, 21 Sep 2020 10:15:26 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 57CD11BF3F4 for ; Mon, 21 Sep 2020 10:15:23 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 545B0871BA for ; Mon, 21 Sep 2020 10:15:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id y0GpgGGf2cwd for ; Mon, 21 Sep 2020 10:15:21 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by hemlock.osuosl.org (Postfix) with ESMTPS id 70CE687184 for ; Mon, 21 Sep 2020 10:15:21 +0000 (UTC) X-Originating-IP: 90.65.92.90 Received: from localhost (lfbn-lyo-1-1913-90.w90-65.abo.wanadoo.fr [90.65.92.90]) (Authenticated sender: gregory.clement@bootlin.com) by relay2-d.mail.gandi.net (Postfix) with ESMTPSA id 6720540012; Mon, 21 Sep 2020 10:15:18 +0000 (UTC) From: Gregory CLEMENT To: buildroot@buildroot.org Date: Mon, 21 Sep 2020 12:15:13 +0200 Message-Id: <20200921101515.132359-2-gregory.clement@bootlin.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200921101515.132359-1-gregory.clement@bootlin.com> References: <20200921101515.132359-1-gregory.clement@bootlin.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH v5 1/3] support/script/pkg-stats: Manage the CVEs that need to be check X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Matt Weber , Thomas Petazzoni Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" When looking for if a package is affected, the version comparison can fail. This means that we don't know if the version of the package used is affected or not and we need to check manually the version. This patch exposes this new information in json and html format. Signed-off-by: Gregory CLEMENT --- support/scripts/pkg-stats | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats index 503cc45c16..69edeedec0 100755 --- a/support/scripts/pkg-stats +++ b/support/scripts/pkg-stats @@ -97,6 +97,7 @@ class Package: self.url = None self.url_worker = None self.cves = list() + self.cves_to_check = list() self.latest_version = {'status': RM_API_STATUS_ERROR, 'version': None, 'id': None} self.status = {} @@ -535,7 +536,10 @@ def check_package_cves(nvd_path, packages): for pkg_name in cve.pkg_names: if pkg_name in packages: pkg = packages[pkg_name] - if cve.affects(pkg.name, pkg.current_version, pkg.ignored_cves) == cve.CVE_AFFECTS: + affected = cve.affects(pkg.name, pkg.current_version, pkg.ignored_cves) + if affected == cve.CVE_UNKNOWN: + pkg.cves_to_check.append(cve.identifier) + if affected == cve.CVE_AFFECTS: pkg.cves.append(cve.identifier) @@ -576,8 +580,11 @@ def calculate_stats(packages): stats["version-not-uptodate"] += 1 stats["patches"] += pkg.patch_count stats["total-cves"] += len(pkg.cves) + stats["total-cves-to-check"] += len(pkg.cves_to_check) if len(pkg.cves) != 0: stats["pkg-cves"] += 1 + if len(pkg.cves_to_check) != 0: + stats["pkg-cves_to_check"] += 1 return stats @@ -800,6 +807,17 @@ def dump_html_pkg(f, pkg): f.write(" %s
\n" % (cve, cve)) f.write(" \n") + # CVEs to check + td_class = ["centered"] + if len(pkg.cves_to_check) == 0: + td_class.append("correct") + else: + td_class.append("wrong") + f.write(" \n" % " ".join(td_class)) + for cve in pkg.cves_to_check: + f.write(" %s
\n" % (cve, cve)) + f.write(" \n") + f.write(" \n") @@ -818,6 +836,7 @@ def dump_html_all_pkgs(f, packages): Warnings Upstream URL CVEs +CVEs to check """) for pkg in sorted(packages): @@ -856,6 +875,10 @@ def dump_html_stats(f, stats): stats["version-not-uptodate"]) f.write("Packages with no known upstream version%s\n" % stats["version-unknown"]) + f.write("Packages that might be affected by CVEs, where version needs to be checked%s\n" % + stats["pkg-cves_to_check"]) + f.write("Total number of CVEs that might affect all packages, where version needs to be checked%s\n" % + stats["total-cves_to_check"]) f.write("Packages affected by CVEs%s\n" % stats["pkg-cves"]) f.write("Total number of CVEs affecting all packages%s\n" % From patchwork Mon Sep 21 10:15:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory CLEMENT X-Patchwork-Id: 1368203 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Bw0gp0qQgz9sTW for ; Mon, 21 Sep 2020 20:15:34 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id A47B985FA8; Mon, 21 Sep 2020 10:15:32 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XisW9EnIAZSg; Mon, 21 Sep 2020 10:15:30 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id D3F7D85F18; Mon, 21 Sep 2020 10:15:30 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id F26E41BF3F4 for ; Mon, 21 Sep 2020 10:15:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id EE1ED20506 for ; Mon, 21 Sep 2020 10:15:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tPss+3k7uB1T for ; Mon, 21 Sep 2020 10:15:23 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [217.70.183.196]) by silver.osuosl.org (Postfix) with ESMTPS id C5BD920502 for ; Mon, 21 Sep 2020 10:15:22 +0000 (UTC) X-Originating-IP: 90.65.92.90 Received: from localhost (lfbn-lyo-1-1913-90.w90-65.abo.wanadoo.fr [90.65.92.90]) (Authenticated sender: gregory.clement@bootlin.com) by relay4-d.mail.gandi.net (Postfix) with ESMTPSA id 2D678E0005; Mon, 21 Sep 2020 10:15:18 +0000 (UTC) From: Gregory CLEMENT To: buildroot@buildroot.org Date: Mon, 21 Sep 2020 12:15:14 +0200 Message-Id: <20200921101515.132359-3-gregory.clement@bootlin.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200921101515.132359-1-gregory.clement@bootlin.com> References: <20200921101515.132359-1-gregory.clement@bootlin.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH v5 2/3] support/script/cve-checker: Manage the CVEs that need to be check X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Matt Weber , Thomas Petazzoni Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" When looking for if a package is affected, the version comparison can fail. This means that we don't know if the version of the package used is affected or not and we need to check manually the version. This patch exposes this new information in json and html format. Signed-off-by: Gregory CLEMENT --- support/scripts/cve-checker | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/support/scripts/cve-checker b/support/scripts/cve-checker index 998ea5b8af..b32e036d76 100755 --- a/support/scripts/cve-checker +++ b/support/scripts/cve-checker @@ -30,6 +30,7 @@ class Package: self.name = name self.version = version self.cves = list() + self.cves_to_check = list() self.ignored_cves = ignored_cves @@ -40,8 +41,12 @@ def check_package_cves(nvd_path, packages): for cve in cvecheck.CVE.read_nvd_dir(nvd_path): for pkg_name in cve.pkg_names: pkg = packages.get(pkg_name, '') - if pkg and cve.affects(pkg.name, pkg.version, pkg.ignored_cves) == cve.CVE_AFFECTS: - pkg.cves.append(cve.identifier) + if pkg: + affected = cve.affects(pkg.name, pkg.version, pkg.ignored_cves) + if (affected == cve.CVE_UNKNOWN): + pkg.cves_to_check.append(cve.identifier) + elif affected == cve.CVE_AFFECTS: + pkg.cves.append(cve.identifier) html_header = """ @@ -106,6 +111,17 @@ def dump_html_pkg(f, pkg): f.write(" %s
\n" % (cve, cve)) f.write(" \n") + # CVEs to check + td_class = ["centered"] + if len(pkg.cves_to_check) == 0: + td_class.append("correct") + else: + td_class.append("wrong") + f.write(" \n" % " ".join(td_class)) + for cve in pkg.cves_to_check: + f.write(" %s
\n" % (cve, cve)) + f.write(" \n") + f.write(" \n") @@ -116,6 +132,7 @@ def dump_html_all_pkgs(f, packages): Package Version CVEs +CVEs to check """) for pkg in packages: From patchwork Mon Sep 21 10:15:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory CLEMENT X-Patchwork-Id: 1368200 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Bw0gg4TTfz9sTW for ; Mon, 21 Sep 2020 20:15:26 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 220B4867FA; Mon, 21 Sep 2020 10:15:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Sl4k90BSBI3u; Mon, 21 Sep 2020 10:15:24 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 51DBD867F3; Mon, 21 Sep 2020 10:15:24 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 5B8011BF3F4 for ; Mon, 21 Sep 2020 10:15:22 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 58B3D85EE9 for ; Mon, 21 Sep 2020 10:15:22 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uQuZEvH6bWzh for ; Mon, 21 Sep 2020 10:15:21 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 9C9EB85EC2 for ; Mon, 21 Sep 2020 10:15:21 +0000 (UTC) X-Originating-IP: 90.65.92.90 Received: from localhost (lfbn-lyo-1-1913-90.w90-65.abo.wanadoo.fr [90.65.92.90]) (Authenticated sender: gregory.clement@bootlin.com) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id D565E1BF203; Mon, 21 Sep 2020 10:15:19 +0000 (UTC) From: Gregory CLEMENT To: buildroot@buildroot.org Date: Mon, 21 Sep 2020 12:15:15 +0200 Message-Id: <20200921101515.132359-4-gregory.clement@bootlin.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200921101515.132359-1-gregory.clement@bootlin.com> References: <20200921101515.132359-1-gregory.clement@bootlin.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH v5 3/3] package/pkg-utils/cve.py: Manage case when package version doesn't exist X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Matt Weber , Thomas Petazzoni Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Until now, when a package didn't report a version, then the CVE comparison was just skipped. It leads most of the time to declare the package not affected by the CVE. Instead of it, report the CVE_UNKNOWN status in order to be aware that the CVE related to this package has to be checked. Signed-off-by: Gregory CLEMENT --- support/scripts/cve.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/support/scripts/cve.py b/support/scripts/cve.py index 6396019e0e..3cc01248b2 100755 --- a/support/scripts/cve.py +++ b/support/scripts/cve.py @@ -198,6 +198,7 @@ class CVE: if self.identifier in cve_ignore_list: return self.CVE_DOESNT_AFFECT + unknown_pkg_version = False pkg_version = distutils.version.LooseVersion(version) if not hasattr(pkg_version, "version"): print("Cannot parse package '%s' version '%s'" % (name, version)) @@ -212,6 +213,7 @@ class CVE: print("No CVE affected version") continue if not pkg_version: + unknown_pkg_version = True continue if cpe['v_start']: @@ -241,4 +243,7 @@ class CVE: # We're in the version range affected by this CVE return self.CVE_AFFECTS - return self.CVE_DOESNT_AFFECT + if unknown_pkg_version: + return self.CVE_UNKNOWN + else: + return self.CVE_DOESNT_AFFECT