From patchwork Fri Dec 4 15:45:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 1411128 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CncWR1yNjz9sWQ for ; Sat, 5 Dec 2020 02:46:25 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id B0CF8204B1; Fri, 4 Dec 2020 15:46:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ue1-SGGv8Si3; Fri, 4 Dec 2020 15:46:19 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id E04EB204C6; Fri, 4 Dec 2020 15:46:18 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 731B41BF30D for ; Fri, 4 Dec 2020 15:46:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 7011C87AF8 for ; Fri, 4 Dec 2020 15:46:15 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4e9Y00g3M068 for ; Fri, 4 Dec 2020 15:46:13 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by whitealder.osuosl.org (Postfix) with ESMTPS id 0B60C87AEF for ; Fri, 4 Dec 2020 15:46:12 +0000 (UTC) X-Originating-IP: 90.2.82.147 Received: from localhost (aputeaux-654-1-223-147.w90-2.abo.wanadoo.fr [90.2.82.147]) (Authenticated sender: thomas.petazzoni@bootlin.com) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 29846240004; Fri, 4 Dec 2020 15:46:08 +0000 (UTC) From: Thomas Petazzoni To: Buildroot List Date: Fri, 4 Dec 2020 16:45:57 +0100 Message-Id: <20201204154601.125932-2-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201204154601.125932-1-thomas.petazzoni@bootlin.com> References: <20201204154601.125932-1-thomas.petazzoni@bootlin.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH v2 1/5] support/script/pkg-stats: show CPE ID in results X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Matt Weber , Heiko Thiery , Thomas Petazzoni Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: Gregory CLEMENT This commit improves the pkg-stats script to show the CPE ID of packages, if available. For now, it doesn't use CPE IDs to match CVEs. Signed-off-by: Gregory CLEMENT Signed-off-by: Thomas Petazzoni --- support/scripts/pkg-stats | 67 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 66 insertions(+), 1 deletion(-) diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats index d44f8241c1..b012e1437a 100755 --- a/support/scripts/pkg-stats +++ b/support/scripts/pkg-stats @@ -78,6 +78,7 @@ class Package: all_license_files = list() all_versions = dict() all_ignored_cves = dict() + all_cpeids = dict () # This is the list of all possible checks. Add new checks to this list so # a tool that post-processeds the json output knows the checks before # iterating over the packages. @@ -98,6 +99,7 @@ class Package: self.current_version = None self.url = None self.url_worker = None + self.cpeid = None self.cves = list() self.latest_version = {'status': RM_API_STATUS_ERROR, 'version': None, 'id': None} self.status = {} @@ -213,6 +215,21 @@ class Package: if var in self.all_versions: self.current_version = self.all_versions[var] + def set_cpeid(self): + """ + Fills in the .cpeid field + """ + var = self.pkgvar() + if not self.has_valid_infra: + self.status['cpe'] = ("na", "no valid package infra") + return + + if var in self.all_cpeids: + self.cpeid = self.all_cpeids[var] + self.status['cpe'] = ("ok", "verified CPE identifier") + else: + self.status['cpe'] = ("error", "no verified CPE identifier") + def set_check_package_warnings(self): """ Fills in the .warnings and .status['pkg-check'] fields @@ -342,7 +359,7 @@ def get_config_packages(): def package_init_make_info(): # Fetch all variables at once variables = subprocess.check_output(["make", "BR2_HAVE_DOT_CONFIG=y", "-s", "printvars", - "VARS=%_LICENSE %_LICENSE_FILES %_VERSION %_IGNORE_CVES"]) + "VARS=%_LICENSE %_LICENSE_FILES %_VERSION %_IGNORE_CVES %_CPE_ID"]) variable_list = variables.decode().splitlines() # We process first the host package VERSION, and then the target @@ -380,6 +397,9 @@ def package_init_make_info(): pkgvar = pkgvar[:-12] Package.all_ignored_cves[pkgvar] = value.split() + elif pkgvar.endswith("_CPE_ID"): + pkgvar = pkgvar[:-7] + Package.all_cpeids[pkgvar] = value check_url_count = 0 @@ -587,6 +607,10 @@ def calculate_stats(packages): stats["total-cves"] += len(pkg.cves) if len(pkg.cves) != 0: stats["pkg-cves"] += 1 + if pkg.cpeid: + stats["cpe-id"] += 1 + else: + stats["no-cpe-id"] += 1 return stats @@ -642,6 +666,18 @@ td.version-error { background: #ccc; } +td.cpe-ok { + background: #d2ffc4; +} + +td.cpe-nok { + background: #ff9a69; +} + +td.cpe-unknown { + background: #ffd870; +} + Statistics of Buildroot packages @@ -809,6 +845,23 @@ def dump_html_pkg(f, pkg): f.write(" %s
\n" % (cve, cve)) f.write(" \n") + # CPE ID + td_class = ["left"] + if pkg.status['cpe'][0] == "ok": + td_class.append("cpe-ok") + elif pkg.status['cpe'][0] == "error": + td_class.append("cpe-nok") + else: + td_class.append("cpe-unknown") + f.write(" \n" % " ".join(td_class)) + if pkg.status['cpe'][0] == "ok": + f.write(" %s\n" % pkg.cpeid) + elif pkg.status['cpe'][0] == "error": + f.write(" N/A\n") + else: + f.write(" %s\n" % pkg.status['cpe'][1]) + f.write(" \n") + f.write(" \n") @@ -827,6 +880,7 @@ def dump_html_all_pkgs(f, packages): Warnings Upstream URL CVEs +CPE ID """) for pkg in sorted(packages): @@ -869,6 +923,10 @@ def dump_html_stats(f, stats): stats["pkg-cves"]) f.write("Total number of CVEs affecting all packages%s\n" % stats["total-cves"]) + f.write("Packages with CPE ID%s\n" % + stats["cpe-id"]) + f.write("Packages without CPE ID%s\n" % + stats["no-cpe-id"]) f.write("\n") @@ -943,11 +1001,17 @@ def parse_args(): help='List of packages (comma separated)') parser.add_argument('--nvd-path', dest='nvd_path', help='Path to the local NVD database', type=resolvepath) + parser.add_argument("--cpeid", action='store_true') args = parser.parse_args() if not args.html and not args.json: parser.error('at least one of --html or --json (or both) is required') return args +def cpeid_name(pkg): + try: + return pkg.cpeid.split(':')[1] + except: + return '' def __main__(): args = parse_args() @@ -979,6 +1043,7 @@ def __main__(): pkg.set_patch_count() pkg.set_check_package_warnings() pkg.set_current_version() + pkg.set_cpeid() pkg.set_url() pkg.set_developers(developers) print("Checking URL status")