From patchwork Fri Mar 8 22:04:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 1053728 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44GM3t1kZMz9s7h for ; Sat, 9 Mar 2019 09:04:41 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id BC09F26D64; Fri, 8 Mar 2019 22:04:37 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CZFIUlEuZMOf; Fri, 8 Mar 2019 22:04:35 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 74271221C6; Fri, 8 Mar 2019 22:04:35 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id D210B1BF3A8 for ; Fri, 8 Mar 2019 22:04:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id CDEC985E03 for ; Fri, 8 Mar 2019 22:04:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c6U_AtlQCabj for ; Fri, 8 Mar 2019 22:04:29 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from da1vs02.rockwellcollins.com (da1vs02.rockwellcollins.com [205.175.227.29]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 5E0CA85DF8 for ; Fri, 8 Mar 2019 22:04:29 +0000 (UTC) Received: from ofwda1n02.rockwellcollins.com (HELO dtulimr01.rockwellcollins.com) ([205.175.227.14]) by da1vs02.rockwellcollins.com with ESMTP; 08 Mar 2019 16:04:28 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by dtulimr01.rockwellcollins.com (Postfix) with ESMTP id 54882604FB; Fri, 8 Mar 2019 16:04:28 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Fri, 8 Mar 2019 16:04:19 -0600 Message-Id: <1552082667-46877-2-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1552082667-46877-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1552082667-46877-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v8 1/9] cpe-info: new make target X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Similar to make legal-info, produce a csv delimited file containing all selected packages CPE identification. By default, support the pkg infra defining a set of CPE_ID_* defaults using the package name for the vendor and name as most CPE IDs seem to align with that assumption. Plus initially, use the pkg version as the CPE ID's version field. Signed-off-by: Matthew Weber --- Changes v8 - Removed CVE patched list as this isn't implmented. v4 -> v7 - No change v3 [Thomas P - Merged infra define CPE_ID_* into this patch - Report all packages vs restricting to just allowing based on if the VENDOR was set (v2). This now represents Thomas P's original idea to report everything. At first I felt I should restrict the reporting to those CPE IDs we had made sure were correct. Turns out we should have actually let the script handle fixing the CPEs and just make a complete design of this up front. [Matt - Moved to using the _project on all vendors instead of just name v2 [Thomas P - Moved comment on conditionals back to this patchset where the conditional is created vs later --- Makefile | 17 ++++++++++++++++- package/pkg-generic.mk | 13 +++++++++++++ package/pkg-utils.mk | 8 ++++++++ 3 files changed, 37 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index a64196f..cdf2cf9 100644 --- a/Makefile +++ b/Makefile @@ -135,7 +135,7 @@ nobuild_targets := source %-source \ clean distclean help show-targets graph-depends \ %-graph-depends %-show-depends %-show-version \ graph-build graph-size list-defconfigs \ - savedefconfig update-defconfig printvars + savedefconfig update-defconfig printvars cpe-info %-cpe-info ifeq ($(MAKECMDGOALS),) BR_BUILDING = y else ifneq ($(filter-out $(nobuild_targets),$(MAKECMDGOALS)),) @@ -219,6 +219,7 @@ LEGAL_MANIFEST_CSV_TARGET = $(LEGAL_INFO_DIR)/manifest.csv LEGAL_MANIFEST_CSV_HOST = $(LEGAL_INFO_DIR)/host-manifest.csv LEGAL_WARNINGS = $(LEGAL_INFO_DIR)/.warnings LEGAL_REPORT = $(LEGAL_INFO_DIR)/README +CPE_MANIFEST_CSV = $(BASE_DIR)/cpe-manifest.csv BR2_CONFIG = $(CONFIG_DIR)/.config @@ -852,6 +853,19 @@ legal-info: legal-info-clean legal-info-prepare $(foreach p,$(PACKAGES),$(p)-all mv .legal-info.sha256 legal-info.sha256) @echo "Legal info produced in $(LEGAL_INFO_DIR)" +.PHONY: cpe-info-clean +cpe-info-clean: + @rm -f $(CPE_MANIFEST_CSV) + +.PHONY: cpe-info-prepare +cpe-info-prepare: + @$(call MESSAGE,"Gathering CPE info") + @$(call cpe-manifest,CPE ID,CVE PATCHED,PACKAGE,VERSION,SOURCE SITE) + +.PHONY: cpe-info +cpe-info: cpe-info-clean cpe-info-prepare $(foreach p,$(PACKAGES),$(p)-cpe-info) + @echo "CPE info produced in $(CPE_MANIFEST_CSV)" + .PHONY: show-targets show-targets: @echo $(sort $(PACKAGES)) $(sort $(TARGETS_ROOTFS)) @@ -1124,6 +1138,7 @@ help: @echo ' source - download all sources needed for offline-build' @echo ' external-deps - list external packages used' @echo ' legal-info - generate info about license compliance' + @echo ' cpe-info - generate info about security CPE identification' @echo ' printvars - dump all the internal variables' @echo @echo ' make V=0|1 - 0 => quiet build (default), 1 => verbose build' diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index 4353bd3..644128d 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -911,6 +911,18 @@ else $(2)_KCONFIG_VAR = BR2_PACKAGE_$(2) endif +$(2)_CPE_ID_VENDOR ?= $$($(2)_NAME)_project +$(2)_CPE_ID_NAME ?= $$($(2)_NAME) +$(2)_CPE_ID_VERSION ?= $$($(2)_VERSION) +$(2)_CPE_ID ?= $$($(2)_CPE_ID_VENDOR):$$($(2)_CPE_ID_NAME):$$($(2)_CPE_ID_VERSION) + +$(1)-cpe-info: PKG=$(2) +$(1)-cpe-info: +ifneq ($$(call qstrip,$$($(2)_SOURCE)),) + @$$(call MESSAGE,"Collecting cpe info") + $(Q)$$(call cpe-manifest,$$($(2)_CPE_ID),$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_ACTUAL_SOURCE_SITE)) +endif # ifneq ($$(call qstrip,$$($(2)_SOURCE)),) + # legal-info: declare dependencies and set values used later for the manifest ifneq ($$($(2)_LICENSE_FILES),) $(2)_MANIFEST_LICENSE_FILES = $$($(2)_LICENSE_FILES) @@ -1052,6 +1064,7 @@ DL_TOOLS_DEPENDENCIES += $$(call extractor-dependency,$$($(2)_SOURCE)) $(1)-clean-for-reconfigure \ $(1)-clean-for-reinstall \ $(1)-configure \ + $(1)-cpe-info \ $(1)-depends \ $(1)-dirclean \ $(1)-external-deps \ diff --git a/package/pkg-utils.mk b/package/pkg-utils.mk index bffd79d..0201632 100644 --- a/package/pkg-utils.mk +++ b/package/pkg-utils.mk @@ -102,3 +102,11 @@ legal-deps = \ $(filter-out $(if $(1:host-%=),host-%),\ $(call non-virtual-deps,\ $($(call UPPERCASE,$(1))_FINAL_RECURSIVE_DEPENDENCIES))),$(p) [$($(call UPPERCASE,$(p))_LICENSE)]) + +# +# cpe-info helper functions +# + +define cpe-manifest # cpe, pkg name, version, url + echo '"$(1)","$(2)","$(3)","$(4)"' >>$(CPE_MANIFEST_CSV) +endef From patchwork Fri Mar 8 22:04:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 1053725 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44GM3n6RNFz9s9N for ; Sat, 9 Mar 2019 09:04:37 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 131E386879; Fri, 8 Mar 2019 22:04:34 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UIrVQt7NwDhD; Fri, 8 Mar 2019 22:04:32 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id AAB4B8683C; Fri, 8 Mar 2019 22:04:32 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 0E6921BF3A8 for ; Fri, 8 Mar 2019 22:04:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 0BE6F87702 for ; Fri, 8 Mar 2019 22:04:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RgposWyci-68 for ; Fri, 8 Mar 2019 22:04:29 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from secvs02.rockwellcollins.com (secvs02.rockwellcollins.com [205.175.225.241]) by hemlock.osuosl.org (Postfix) with ESMTPS id 9A8948767B for ; Fri, 8 Mar 2019 22:04:29 +0000 (UTC) Received: from ofwgwc03.rockwellcollins.com (HELO dtulimr01.rockwellcollins.com) ([205.175.225.12]) by secvs02.rockwellcollins.com with ESMTP; 08 Mar 2019 16:04:28 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by dtulimr01.rockwellcollins.com (Postfix) with ESMTP id 75EEF604FD; Fri, 8 Mar 2019 16:04:28 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Fri, 8 Mar 2019 16:04:20 -0600 Message-Id: <1552082667-46877-3-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1552082667-46877-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1552082667-46877-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v8 2/9] cpe-info: id prefix/suffix X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" There are two types of software CPE prefixes applicable for software, one for applications and one for operating systems. Note: The third type is for hardware. This patchset determines which should be used and stores that information with the package for later use when assembling the CPE report. Refs: https://nvlpubs.nist.gov/nistpubs/Legacy/IR/nistir7695.pdf https://cpe.mitre.org/specification/ Signed-off-by: Matthew Weber --- Changes v8 - Rebase v4 -> v7 - None v3 [Arnout - Moved CPE prefix and suffix defines to package/Makefile.in v1 -> v2 [Thomas P - Change to using a filter on pkg name value vs ifelse --- package/Makefile.in | 4 ++++ package/pkg-generic.mk | 8 +++++++- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/package/Makefile.in b/package/Makefile.in index dc818a2..644282f 100644 --- a/package/Makefile.in +++ b/package/Makefile.in @@ -389,6 +389,10 @@ TARGET_CONFIGURE_ARGS = \ ################################################################################ +CPE_PREFIX_OS = cpe:2.3:o +CPE_PREFIX_APP = cpe:2.3:a +CPE_SUFFIX = *:*:*:*:*:*:* + ifeq ($(BR2_SYSTEM_ENABLE_NLS),y) NLS_OPTS = --enable-nls TARGET_NLS_DEPENDENCIES = host-gettext diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index 644128d..a547c65 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -916,11 +916,17 @@ $(2)_CPE_ID_NAME ?= $$($(2)_NAME) $(2)_CPE_ID_VERSION ?= $$($(2)_VERSION) $(2)_CPE_ID ?= $$($(2)_CPE_ID_VENDOR):$$($(2)_CPE_ID_NAME):$$($(2)_CPE_ID_VERSION) +ifneq ($(filter linux linux-headers,$(1)),) +$(2)_CPE_PREFIX = $(CPE_PREFIX_OS) +else +$(2)_CPE_PREFIX = $(CPE_PREFIX_APP) +endif + $(1)-cpe-info: PKG=$(2) $(1)-cpe-info: ifneq ($$(call qstrip,$$($(2)_SOURCE)),) @$$(call MESSAGE,"Collecting cpe info") - $(Q)$$(call cpe-manifest,$$($(2)_CPE_ID),$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_ACTUAL_SOURCE_SITE)) + $(Q)$$(call cpe-manifest,$$($(2)_CPE_PREFIX):$$($(2)_CPE_ID):$(CPE_SUFFIX),$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_ACTUAL_SOURCE_SITE)) endif # ifneq ($$(call qstrip,$$($(2)_SOURCE)),) # legal-info: declare dependencies and set values used later for the manifest From patchwork Fri Mar 8 22:04:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 1053727 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44GM3t1kb0z9s9N for ; Sat, 9 Mar 2019 09:04:42 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 19139868E7; Fri, 8 Mar 2019 22:04:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id axE697aMLp5d; Fri, 8 Mar 2019 22:04:34 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 7FC138686A; Fri, 8 Mar 2019 22:04:33 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 4C7C31BF3A8 for ; Fri, 8 Mar 2019 22:04:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 49F8326A59 for ; Fri, 8 Mar 2019 22:04:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gdTEncnPT6w0 for ; Fri, 8 Mar 2019 22:04:30 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from secvs01.rockwellcollins.com (secvs01.rockwellcollins.com [205.175.225.240]) by silver.osuosl.org (Postfix) with ESMTPS id E342F221C6 for ; Fri, 8 Mar 2019 22:04:29 +0000 (UTC) Received: from ofwgwc03.rockwellcollins.com (HELO dtulimr01.rockwellcollins.com) ([205.175.225.12]) by secvs01.rockwellcollins.com with ESMTP; 08 Mar 2019 16:04:28 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by dtulimr01.rockwellcollins.com (Postfix) with ESMTP id 97549604FA; Fri, 8 Mar 2019 16:04:28 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Fri, 8 Mar 2019 16:04:21 -0600 Message-Id: <1552082667-46877-4-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1552082667-46877-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1552082667-46877-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v8 3/9] cpe-info: only report target pkgs X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" The reporting of host packages causes some duplication and complicates what is really in the targets configuration. For the purpose of the first version of this patchset, its assumed that host packages aren't relevant for the configuration and we only report the target's contents. Signed-off-by: Matthew Weber --- Changes v4 -> v8 - None v3 - Fixed host build error because cpe-info wasn't defined v1 -> v2 [Thomas P - select if target vs selecting not host --- package/pkg-generic.mk | 2 ++ 1 file changed, 2 insertions(+) diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index a547c65..8ff389e 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -924,10 +924,12 @@ endif $(1)-cpe-info: PKG=$(2) $(1)-cpe-info: +ifeq ($$($(2)_TYPE),target) ifneq ($$(call qstrip,$$($(2)_SOURCE)),) @$$(call MESSAGE,"Collecting cpe info") $(Q)$$(call cpe-manifest,$$($(2)_CPE_PREFIX):$$($(2)_CPE_ID):$(CPE_SUFFIX),$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_ACTUAL_SOURCE_SITE)) endif # ifneq ($$(call qstrip,$$($(2)_SOURCE)),) +endif # ifeq ($$($(2)_TYPE),target) # legal-info: declare dependencies and set values used later for the manifest ifneq ($$($(2)_LICENSE_FILES),) From patchwork Fri Mar 8 22:04:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 1053732 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44GM5h5ZvQz9s7h for ; Sat, 9 Mar 2019 09:06:16 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 2C10326CAB; Fri, 8 Mar 2019 22:06:15 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 03eK4udg-KZY; Fri, 8 Mar 2019 22:06:08 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 6345927FA1; Fri, 8 Mar 2019 22:05:53 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id D92441BF5E6 for ; Fri, 8 Mar 2019 22:05:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id D68D087702 for ; Fri, 8 Mar 2019 22:05:48 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xLf1I6kWMiVo for ; Fri, 8 Mar 2019 22:05:47 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from ch3vs01.rockwellcollins.com (ch3vs01.rockwellcollins.com [205.175.226.27]) by hemlock.osuosl.org (Postfix) with ESMTPS id 08F8387917 for ; Fri, 8 Mar 2019 22:05:46 +0000 (UTC) Received: from ofwch3n02.rockwellcollins.com (HELO dtulimr01.rockwellcollins.com) ([205.175.226.14]) by ch3vs01.rockwellcollins.com with ESMTP; 08 Mar 2019 16:05:45 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by dtulimr01.rockwellcollins.com (Postfix) with ESMTP id C73CF603EF; Fri, 8 Mar 2019 16:05:45 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Fri, 8 Mar 2019 16:04:23 -0600 Message-Id: <1552082667-46877-6-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1552082667-46877-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1552082667-46877-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v8 5/9] toolchain/toolchain-ext: glibc cpe-info support X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Shruthi Singh MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: Shruthi Singh This commit adds the correct CPE string for glibc, describing CPE ID, VERSION, PACKAGE NAME and URL. Signed-off-by: Shruthi Singh shruthi.singh@rockwellcollins.com Signed-off-by: Matthew Weber --- Changes v8 - No change v7 - New --- package/pkg-generic.mk | 14 ++++++++++++++ toolchain/toolchain-external/pkg-toolchain-external.mk | 7 +++++++ 2 files changed, 21 insertions(+) diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index 1b895d0..f9cedd2 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -924,8 +924,22 @@ $(2)_CPE_PREFIX = $(CPE_PREFIX_APP) endif $(1)-cpe-info: PKG=$(2) +ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y) +$(1)-cpe-info: toolchain +endif $(1)-cpe-info: ifeq ($$($(2)_TYPE),target) +ifneq ($$($(2)_NAME),toolchain-external) +ifneq ($(findstring TOOLCHAIN_EXTERNAL, $(2)),) +ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC),y) + $$(eval $(2)_VERSION = $$(shell $$(call TOOLCHAIN_CPE_INFO))) + $$(eval $(2)_CPE_ID_VENDOR = gnu) + $$(eval $(2)_CPE_ID_NAME = glibc) + $$(eval $(2)_ACTUAL_SOURCE_SITE = https://github.com/bminor/glibc/releases) + $$(eval $(2)_RAWNAME = glibc) +endif # ifeq ($(BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC),y) +endif # ifneq ($(findstring TOOLCHAIN_EXTERNAL, $(2)),) +endif # ifneq ($$($(2)_NAME),toolchain-external) ifneq ($$(call qstrip,$$($(2)_SOURCE)),) @$$(call MESSAGE,"Collecting cpe info") $(Q)$$(call cpe-manifest,$$($(2)_CPE_PREFIX):$$($(2)_CPE_ID):$(CPE_SUFFIX),$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_ACTUAL_SOURCE_SITE)) diff --git a/toolchain/toolchain-external/pkg-toolchain-external.mk b/toolchain/toolchain-external/pkg-toolchain-external.mk index db3570d..aed06c5 100644 --- a/toolchain/toolchain-external/pkg-toolchain-external.mk +++ b/toolchain/toolchain-external/pkg-toolchain-external.mk @@ -440,6 +440,13 @@ define TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS $(call copy_toolchain_sysroot,$${SYSROOT_DIR},$${ARCH_SYSROOT_DIR},$${ARCH_SUBDIR},$${ARCH_LIB_DIR},$${SUPPORT_LIB_DIR}) endef +define TOOLCHAIN_CPE_INFO + ARCH_SYSROOT_DIR="$(call toolchain_find_sysroot,$(TOOLCHAIN_EXTERNAL_CC) $(TOOLCHAIN_EXTERNAL_CFLAGS))" ; \ + MAJ=`awk '{ if ($$1 = /#define/ && ($$2= /__GLIBC__/)){printf $$3};}' $${ARCH_SYSROOT_DIR}/usr/include/features.h` ; \ + MIN=`awk '{ if ($$1 = /#define/ && ($$2 = /_GLIBC_MINOR/)){printf $$3};}' $${ARCH_SYSROOT_DIR}/usr/include/features.h` ; \ + echo $${MAJ}.$${MIN} +endef + # Create a symlink from (usr/)$(ARCH_LIB_DIR) to lib. # Note: the skeleton package additionally creates lib32->lib or lib64->lib # (as appropriate) From patchwork Fri Mar 8 22:04:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 1053731 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44GM5X5wRzz9s9N for ; Sat, 9 Mar 2019 09:06:08 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id DC4182A293; Fri, 8 Mar 2019 22:06:05 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JPSy8wscIsAY; Fri, 8 Mar 2019 22:05:53 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id E6B5726E60; Fri, 8 Mar 2019 22:05:49 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id C26AE1BF5E6 for ; Fri, 8 Mar 2019 22:05:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id C016E877DB for ; Fri, 8 Mar 2019 22:05:47 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m2B24Z5RLn7g for ; Fri, 8 Mar 2019 22:05:46 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from da1vs02.rockwellcollins.com (da1vs02.rockwellcollins.com [205.175.227.29]) by hemlock.osuosl.org (Postfix) with ESMTPS id ABC0987702 for ; Fri, 8 Mar 2019 22:05:46 +0000 (UTC) Received: from ofwda1n02.rockwellcollins.com (HELO dtulimr01.rockwellcollins.com) ([205.175.227.14]) by da1vs02.rockwellcollins.com with ESMTP; 08 Mar 2019 16:05:46 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by dtulimr01.rockwellcollins.com (Postfix) with ESMTP id E7F75604ED; Fri, 8 Mar 2019 16:05:45 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Fri, 8 Mar 2019 16:04:24 -0600 Message-Id: <1552082667-46877-7-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1552082667-46877-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1552082667-46877-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v8 6/9] cpe-info: update manual for new pkg vars X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Provide guidance on setting up the *_CPE_* and *_CVE_* variables. Signed-off-by: Matthew Weber Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
--- Changes v8 - Added note about minor version - Removed CVE patch listing as the backend has not be implemented (LIBFOO_CVE_PATCHED) v4 -> v7 - None v3 - Updated to make *_CPE_VENDOR optional - Changed wording around _CPE_ID as there is only one defined now v2 [Thomas P - Reworded LIBFOO_CVE_PATCHED description [Matt W - Added definition for new preset variables to auto-gen the CPE ID - Added example LIBFOO_CPE_ID_VENDOR to LIBFOO --- docs/manual/adding-packages-generic.txt | 111 +++++++++++++++++++------------- 1 file changed, 68 insertions(+), 43 deletions(-) diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt index 7be1754..ddf1b2e 100644 --- a/docs/manual/adding-packages-generic.txt +++ b/docs/manual/adding-packages-generic.txt @@ -24,57 +24,59 @@ system is based on hand-written Makefiles or shell scripts. 09: LIBFOO_SITE = http://www.foosoftware.org/download 10: LIBFOO_LICENSE = GPL-3.0+ 11: LIBFOO_LICENSE_FILES = COPYING -12: LIBFOO_INSTALL_STAGING = YES -13: LIBFOO_CONFIG_SCRIPTS = libfoo-config -14: LIBFOO_DEPENDENCIES = host-libaaa libbbb -15: -16: define LIBFOO_BUILD_CMDS -17: $(MAKE) $(TARGET_CONFIGURE_OPTS) -C $(@D) all -18: endef -19: -20: define LIBFOO_INSTALL_STAGING_CMDS -21: $(INSTALL) -D -m 0755 $(@D)/libfoo.a $(STAGING_DIR)/usr/lib/libfoo.a -22: $(INSTALL) -D -m 0644 $(@D)/foo.h $(STAGING_DIR)/usr/include/foo.h -23: $(INSTALL) -D -m 0755 $(@D)/libfoo.so* $(STAGING_DIR)/usr/lib -24: endef -25: -26: define LIBFOO_INSTALL_TARGET_CMDS -27: $(INSTALL) -D -m 0755 $(@D)/libfoo.so* $(TARGET_DIR)/usr/lib -28: $(INSTALL) -d -m 0755 $(TARGET_DIR)/etc/foo.d -29: endef -30: -31: define LIBFOO_USERS -32: foo -1 libfoo -1 * - - - LibFoo daemon -33: endef -34: -35: define LIBFOO_DEVICES -36: /dev/foo c 666 0 0 42 0 - - - -37: endef -38: -39: define LIBFOO_PERMISSIONS -40: /bin/foo f 4755 foo libfoo - - - - - -41: endef -42: -43: $(eval $(generic-package)) +12: LIBFOO_CPE_ID_VENDOR = foosoftware +13: LIBFOO_INSTALL_STAGING = YES +14: LIBFOO_CONFIG_SCRIPTS = libfoo-config +15: LIBFOO_DEPENDENCIES = host-libaaa libbbb +16: +17: define LIBFOO_BUILD_CMDS +18: $(MAKE) $(TARGET_CONFIGURE_OPTS) -C $(@D) all +19: endef +20: +21: define LIBFOO_INSTALL_STAGING_CMDS +22: $(INSTALL) -D -m 0755 $(@D)/libfoo.a $(STAGING_DIR)/usr/lib/libfoo.a +23: $(INSTALL) -D -m 0644 $(@D)/foo.h $(STAGING_DIR)/usr/include/foo.h +24: $(INSTALL) -D -m 0755 $(@D)/libfoo.so* $(STAGING_DIR)/usr/lib +25: endef +26: +27: define LIBFOO_INSTALL_TARGET_CMDS +28: $(INSTALL) -D -m 0755 $(@D)/libfoo.so* $(TARGET_DIR)/usr/lib +29: $(INSTALL) -d -m 0755 $(TARGET_DIR)/etc/foo.d +30: endef +31: +32: define LIBFOO_USERS +33: foo -1 libfoo -1 * - - - LibFoo daemon +34: endef +35: +36: define LIBFOO_DEVICES +37: /dev/foo c 666 0 0 42 0 - - - +38: endef +39: +40: define LIBFOO_PERMISSIONS +41: /bin/foo f 4755 foo libfoo - - - - - +42: endef +43: +44: $(eval $(generic-package)) -------------------------------- -The Makefile begins on line 7 to 11 with metadata information: the +The Makefile begins on line 7 to 12 with metadata information: the version of the package (+LIBFOO_VERSION+), the name of the tarball containing the package (+LIBFOO_SOURCE+) (xz-ed tarball recommended) the Internet location at which the tarball can be downloaded from -(+LIBFOO_SITE+), the license (+LIBFOO_LICENSE+) and file with the -license text (+LIBFOO_LICENSE_FILES+). All variables must start with +(+LIBFOO_SITE+), the license (+LIBFOO_LICENSE+), the file with the +license text (+LIBFOO_LICENSE_FILES+) and the vendor for vunerability +analysis (+LIBFOO_CPE_ID_VENDOR+). All variables must start with the same prefix, +LIBFOO_+ in this case. This prefix is always the uppercased version of the package name (see below to understand where the package name is defined). -On line 12, we specify that this package wants to install something to +On line 13, we specify that this package wants to install something to the staging space. This is often needed for libraries, since they must install header files and other development files in the staging space. This will ensure that the commands listed in the +LIBFOO_INSTALL_STAGING_CMDS+ variable will be executed. -On line 13, we specify that there is some fixing to be done to some +On line 14, we specify that there is some fixing to be done to some of the 'libfoo-config' files that were installed during +LIBFOO_INSTALL_STAGING_CMDS+ phase. These *-config files are executable shell script files that are @@ -122,14 +124,14 @@ IMAGEMAGICK_CONFIG_SCRIPTS = \ -------------------------------- ================================ -On line 14, we specify the list of dependencies this package relies +On line 15, we specify the list of dependencies this package relies on. These dependencies are listed in terms of lower-case package names, which can be packages for the target (without the +host-+ prefix) or packages for the host (with the +host-+) prefix). Buildroot will ensure that all these packages are built and installed 'before' the current package starts its configuration. -The rest of the Makefile, lines 16..29, defines what should be done +The rest of the Makefile, lines 17..29, defines what should be done at the different steps of the package configuration, compilation and installation. +LIBFOO_BUILD_CMDS+ tells what steps should be performed to @@ -142,16 +144,16 @@ All these steps rely on the +$(@D)+ variable, which contains the directory where the source code of the package has been extracted. -On lines 31..43, we define a user that is used by this package (e.g. +On lines 32..44, we define a user that is used by this package (e.g. to run a daemon as non-root) (+LIBFOO_USERS+). -On line 35..37, we define a device-node file used by this package +On line 36..38, we define a device-node file used by this package (+LIBFOO_DEVICES+). -On line 39..41, we define the permissions to set to specific files +On line 40..42, we define the permissions to set to specific files installed by this package (+LIBFOO_PERMISSIONS+). -Finally, on line 43, we call the +generic-package+ function, which +Finally, on line 44, we call the +generic-package+ function, which generates, according to the variables defined previously, all the Makefile code necessary to make your package working. @@ -482,6 +484,29 @@ not and can not work as people would expect it should: locations, `/lib/firmware`, `/usr/lib/firmware`, `/lib/modules`, `/usr/lib/modules`, and `/usr/share`, which are automatically excluded. +* +LIBFOO_CPE_ID_VENDOR+ + This variable is optional. It only must be defined if the package name + does not match what the CPE ID uses for the vendor. By default it's set + to _project. + +* +LIBFOO_CPE_ID_NAME+ + This variable is optional. It only must be defined if the package name + does not match what the CPE ID uses for the name. By default it's set + to . + +* +LIBFOO_CPE_ID_VERSION+ + This variable is optional. By default it's set to . + +* +LIBFOO_CPE_ID_VERSION_MINOR+ + This variable is optional. By default it's set to *. + +* +LIBFOO_CPE_ID+ is optional, as the package infrastructure hangles the + default case of a single package's Common Product Enumeration (CPE) + identification string. +make cpe-info+ copies all of these into a + +cpe-manifest.csv+ file. To identify a package's possible CPE, + the National Vunerability Database can be searched at + https://nvd.nist.gov/products/cpe/search. + The recommended way to define these variables is to use the following syntax: From patchwork Fri Mar 8 22:04:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 1053733 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44GM5r0w2mz9s9N for ; Sat, 9 Mar 2019 09:06:24 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 5DEB62C85C; Fri, 8 Mar 2019 22:06:19 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vKYYqjr1go6O; Fri, 8 Mar 2019 22:06:06 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 45C5827205; Fri, 8 Mar 2019 22:05:51 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 4E6371BF999 for ; Fri, 8 Mar 2019 22:05:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 49EFA87702 for ; Fri, 8 Mar 2019 22:05:48 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oq50bpstrWj3 for ; Fri, 8 Mar 2019 22:05:47 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from ch3vs01.rockwellcollins.com (ch3vs01.rockwellcollins.com [205.175.226.27]) by hemlock.osuosl.org (Postfix) with ESMTPS id 5FB5E87994 for ; Fri, 8 Mar 2019 22:05:47 +0000 (UTC) Received: from ofwch3n02.rockwellcollins.com (HELO dtulimr01.rockwellcollins.com) ([205.175.226.14]) by ch3vs01.rockwellcollins.com with ESMTP; 08 Mar 2019 16:05:46 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by dtulimr01.rockwellcollins.com (Postfix) with ESMTP id 14C9D604D2; Fri, 8 Mar 2019 16:05:46 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Fri, 8 Mar 2019 16:04:25 -0600 Message-Id: <1552082667-46877-8-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1552082667-46877-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1552082667-46877-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v8 7/9] support/scripts/cpedb.py: new CPE XML helper X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Python class which consumes a NIST CPE XML and provides helper functions to access and search the db's data. - Defines the CPE as a object with operations / formats - Processing of CPE dictionary Signed-off-by: Matthew Weber --- v8 - Added support for generation of update xml to maintain the NIST dictionary for any Buildroot package version bumps - Dropped searching of the Config.in files for URLs, instead assuming the first time a package is added to NIST, the xml is manually filled out with reference urls. Any updates to versions after that will use the proposed autogen xml that mines the URLS from the NIST dict file. - Caching support for a processed dictionary to speed up subsequent runs when testing, as a db doesn't update more then once a day v5 -> v7 - No change v5 [Ricardo - Fixed typo in join/split of cpe str without version - Removed extra prints as they aren't needed when we have the output reports/stdout - Updated v4 comments about general flake formatting cleanup - Incorporated parts of patch 1/2 suggestions for optimizations [Arnout - added pre-processing of cpe values into two sets, one with and one without version - Collectly with Ricardo, decided to move cpe class to this seperate script v1 -> v4 - No version --- support/scripts/cpedb.py | 185 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 185 insertions(+) create mode 100644 support/scripts/cpedb.py diff --git a/support/scripts/cpedb.py b/support/scripts/cpedb.py new file mode 100644 index 0000000..0369536 --- /dev/null +++ b/support/scripts/cpedb.py @@ -0,0 +1,185 @@ +import sys +import urllib2 +from collections import OrderedDict +import xmltodict +import gzip +from StringIO import StringIO +import os +import pickle + +VALID_REFS = ['VENDOR', 'VERSION', 'CHANGE_LOG', 'PRODUCT', 'PROJECT', 'ADVISORY'] + + +class CPE: + cpe_str = None + cpe_str_short = None + cpe_desc = None + cpe_cur_ver = None + titles = {} + references = {} + + def __init__(self, cpe_str, titles=None, refs=None): + self.cpe_str = cpe_str + self.cpe_str_short = ":".join(self.cpe_str.split(":")[:6]) + self.titles = titles + self.references = refs + self.cpe_cur_ver = "".join(self.cpe_str.split(":")[5:6]) + + def to_dict(self, cpe_str): + cpe_short_name = ":".join(cpe_str.split(":")[2:6]) + cpe_new_ver = "".join(cpe_str.split(":")[5:6]) + self.titles[0]['#text'] = self.titles[0]['#text'].replace(self.cpe_cur_ver, cpe_new_ver) + cpe_dict = OrderedDict([ + ('cpe-item', OrderedDict([ + ('@name', 'cpe:/' + cpe_short_name), + ('title', self.titles), + ('references', OrderedDict([('reference', self.references)])), + ('cpe-23:cpe23-item', OrderedDict([ + ('@name', cpe_str) + ])) + ])) + ]) + return cpe_dict + + +class CPEDB: + all_cpes = dict() + all_cpes_no_version = dict() + + def get_xml_dict(self, url): + print("CPE: Setting up NIST dictionary") + # Setup location to save dict and xmls, if it exists, assume we're + # reusing the previous dict + if not os.path.exists("cpe"): + os.makedirs("cpe") + self.get_new_xml_dict(url) + else: + print("CPE: Loading CACHED dictionary") + cpe_file = open('cpe/.all_cpes.pkl', 'rb') + self.all_cpes = pickle.load(cpe_file) + cpe_file.close() + cpe_file = open('cpe/.all_cpes_no_version.pkl', 'rb') + self.all_cpes_no_version = pickle.load(cpe_file) + cpe_file.close() + + def get_new_xml_dict(self, url): + print("CPE: Fetching xml manifest from [" + url + "]") + try: + compressed_cpe_file = urllib2.urlopen(url) + print("CPE: Unzipping xml manifest...") + nist_cpe_file = gzip.GzipFile(fileobj=StringIO(compressed_cpe_file.read())).read() + print("CPE: Converting xml manifest to dict...") + all_cpedb = xmltodict.parse(nist_cpe_file) + + # Cycle through the dict and build two dict to be used for custom + # lookups of partial and complete CPE objects + # The objects are then used to create new proposed XML updates if + # if is determined one is required + for cpe in all_cpedb['cpe-list']['cpe-item']: + cpe_titles = cpe['title'] + # There maybe multiple titles or one. Make sure this is + # always a list + if not isinstance(cpe_titles, (list,)): + cpe_titles = [cpe_titles] + # Out of the different language titles, select English + for title in cpe_titles: + if title['@xml:lang'] is 'en-US': + cpe_titles = [title] + # Some older CPE don't include references, if they do, make + # sure we handle the case of one ref needing to be packed + # in a list + if 'references' in cpe: + cpe_ref = cpe['references']['reference'] + if not isinstance(cpe_ref, (list,)): + cpe_ref = [cpe_ref] + # The reference text has not been consistantly upper case + # in the NIST dict but they now require it. So force upper + # and then check for compliance to a specific tagging + for ref_href in cpe_ref: + ref_href['#text'] = ref_href['#text'].upper() + if ref_href['#text'] not in VALID_REFS: + ref_href['#text'] = ref_href['#text'] + "-- UPDATE this entry, here are some exmaples and just one word should be used -- " + ' '.join(VALID_REFS) + cpe_str = cpe['cpe-23:cpe23-item']['@name'] + item = CPE(cpe_str, cpe_titles, cpe_ref) + cpe_str_no_version = self.get_cpe_no_version(cpe_str) + # This dict must have a unique key for every CPE version + # which allows matching to the specific obj data of that + # NIST dict entry + self.all_cpes.update({cpe_str: item}) + # This dict has one entry for every CPE (w/o version) to allow + # partial match (no valid version) check (the obj is saved and + # used as seed for suggested xml updates. By updating the same + # non-version'd entry, it assumes the last update here is the + # latest version in the NIST dict) + self.all_cpes_no_version.update({cpe_str_no_version: item}) + + except urllib2.HTTPError: + print("CPE: HTTP Error: %s" % url) + sys.exit(1) + except urllib2.URLError: + print("CPE: URL Error: %s" % url) + sys.exit(1) + + print("CPE: Caching dictionary") + cpes_file = open('cpe/.all_cpes.pkl', 'wb') + pickle.dump(self.all_cpes, cpes_file) + cpes_file.close() + cpes_file = open('cpe/.all_cpes_no_version.pkl', 'wb') + pickle.dump(self.all_cpes_no_version, cpes_file) + cpes_file.close() + + def find_partial(self, cpe_str): + cpe_str_no_version = self.get_cpe_no_version(cpe_str) + if cpe_str_no_version in self.all_cpes_no_version: + return cpe_str_no_version + + def find_partial_obj(self, cpe_str): + cpe_str_no_version = self.get_cpe_no_version(cpe_str) + if cpe_str_no_version in self.all_cpes_no_version: + return self.all_cpes_no_version[cpe_str_no_version] + + def find_partial_latest_version(self, cpe_str_partial): + cpe_obj = self.find_partial_obj(cpe_str_partial) + return cpe_obj.cpe_cur_ver + + def find(self, cpe_str): + if self.find_partial(cpe_str): + if cpe_str in self.all_cpes: + return cpe_str + + def update(self, cpe_str): + to_update = self.find_partial_obj(cpe_str) + xml = self.__gen_xml__(to_update.to_dict(cpe_str)) + fp = open(os.path.join('cpe', self.get_cpe_name(cpe_str) + '-' + self.get_cpe_version(cpe_str) + '.xml'), 'w+') + fp.write(xmltodict.unparse(xml, pretty=True)) + fp.close() + + def get_nvd_url(self, cpe_str): + return "https://nvd.nist.gov/products/cpe/search/results?keyword=" + \ + urllib2.quote(cpe_str) + \ + "&status=FINAL&orderBy=CPEURI&namingFormat=2.3" + + def get_cpe_no_version(self, cpe): + return ":".join(cpe.split(":")[:5]) + + def get_cpe_name(self, cpe_str): + return "".join(cpe_str.split(":")[4]) + + def get_cpe_version(self, cpe_str): + return "".join(cpe_str.split(":")[5]) + + def __gen_xml__(self, cpe_list): + list_header = { + "cpe-list": { + "@xmlns:config": "http://scap.nist.gov/schema/configuration/0.1", + "@xmlns": "http://cpe.mitre.org/dictionary/2.0", + "@xmlns:xsi": "http://www.w3.org/2001/XMLSchema-instance", + "@xmlnsscap-core": "http://scap.nist.gov/schema/scap-core/0.3", + "@xmlns:cpe-23": "http://scap.nist.gov/schema/cpe-extension/2.3", + "@xmlns:ns6": "http://scap.nist.gov/schema/scap-core/0.1", + "@xmlns:meta": "http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2", + "@xsi:schemaLocation": "http://scap.nist.gov/schema/cpe-extension/2.3 https://scap.nist.gov/schema/cpe/2.3/cpe-dictionary-extension_2.3.xsd http://cpe.mitre.org/dictionary/2.0 https://scap.nist.gov/schema/cpe/2.3/cpe-dictionary_2.3.xsd http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2 https://scap.nist.gov/schema/cpe/2.1/cpe-dictionary-metadata_0.2.xsd http://scap.nist.gov/schema/scap-core/0.3 https://scap.nist.gov/schema/nvd/scap-core_0.3.xsd http://scap.nist.gov/schema/configuration/0.1 https://scap.nist.gov/schema/nvd/configuration_0.1.xsd http://scap.nist.gov/schema/scap-core/0.1 https://scap.nist.gov/schema/nvd/scap-core_0.1.xsd" + } + } + list_header['cpe-list'].update(cpe_list) + return list_header From patchwork Fri Mar 8 22:04:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 1053729 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44GM5F2rKtz9s7h for ; Sat, 9 Mar 2019 09:05:53 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id A52921FF59; Fri, 8 Mar 2019 22:05:51 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 62WQ0WlKvBXE; Fri, 8 Mar 2019 22:05:49 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 3BD1C221C6; Fri, 8 Mar 2019 22:05:49 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 9C4221BF5E6 for ; Fri, 8 Mar 2019 22:05:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 99E42879A3 for ; Fri, 8 Mar 2019 22:05:47 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VPgHRE3rNfCb for ; Fri, 8 Mar 2019 22:05:47 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from da1vs02.rockwellcollins.com (da1vs02.rockwellcollins.com [205.175.227.29]) by hemlock.osuosl.org (Postfix) with ESMTPS id EFB54877DB for ; Fri, 8 Mar 2019 22:05:46 +0000 (UTC) Received: from ofwda1n02.rockwellcollins.com (HELO dtulimr01.rockwellcollins.com) ([205.175.227.14]) by da1vs02.rockwellcollins.com with ESMTP; 08 Mar 2019 16:05:46 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by dtulimr01.rockwellcollins.com (Postfix) with ESMTP id 313A9604EF; Fri, 8 Mar 2019 16:05:46 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Fri, 8 Mar 2019 16:04:26 -0600 Message-Id: <1552082667-46877-9-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1552082667-46877-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1552082667-46877-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v8 8/9] support/scripts/cpe-report: new script X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" The script supports looking up all the CPEs provided in a make cpe-info csv file export from a target Buildroot build. It checks the current version and suggests a CPE needs update or possibly an initial submission is required to NIST. Adds option to allow alternate locations for the dictionary URL and caching of a processed dictionary to speed up execution. Outputs a cpe/ folder with propsed xml generated from the dictionary contents to propose updated versions to NIST. For missing CPE matches, a cpe-report-missing.txt is created by the script that can be used later to manually create proposed new NIST dictionary entries. Ref: NIST has a group email (cpe_dictionary@nist.gov) used to recieve these version update and new entry xml files. They do process the XML and provide feedback. In some cases they will propose back something different where the vendor or version is slightly different. Limitations - Currently any use of non-number version identifiers isn't supported by NIST as they use ranges to determine impact of a CVE - Any Linux version from a non-upstream is also not supported without manually adjusting the information as the custom kernel will more then likely not match the upstream version used in the dictionary Signed-off-by: Matt Weber --- Changes v8 - Updated to just output missing and needs version update - Optional processed dictionary caching support - Optional dictionary URL - Creation of a missing status file (cpe-report-missing.txt) - Adjusted index used in CSV for removal of CVE patched item v5 -> v7 - No change v5 [Ricardo - Updated v4 comments about general flake formatting cleanup - Incorporated parts of patch 1/2 suggestions for optimizations [Ricardo/Arnout - Collectly, decided to move cpe report analysis to this script and use a seperate module cpedb class [Arnout - Rename cpe_dict to instead be cpedb v1 -> v4 - Patch did not exist and was part of pkg-stats file --- support/scripts/cpe-report | 70 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) create mode 100755 support/scripts/cpe-report diff --git a/support/scripts/cpe-report b/support/scripts/cpe-report new file mode 100755 index 0000000..7242a37 --- /dev/null +++ b/support/scripts/cpe-report @@ -0,0 +1,70 @@ +#!/usr/bin/env python + +import argparse +import sys +import csv +from cpedb import CPEDB + +CPE_XML_URL = "https://static.nvd.nist.gov/feeds/xml/cpe/dictionary/official-cpe-dictionary_v2.3.xml.gz" + + +def get_target_cpe_report(cpe_report_file, cpedb): + report_cpe_exact_match = "" + report_cpe_needing_update = "" + report_cpe_needing_update_list = "" + report_cpe_missing = "" + + print("CPE: Checking for matches...") + try: + with open(cpe_report_file) as cpe_file: + cpe_list = csv.reader(cpe_file) + next(cpe_list) # make cpe-info has a one line header + for cpe in cpe_list: + result = cpedb.find(cpe[0]) + if not result: + result = cpedb.find_partial(cpedb.get_cpe_no_version(cpe[0])) + if not result: + report_cpe_missing += cpe[0] + "," + cpe[1] + "," + cpe[3] + "\n" + else: + latest_version = cpedb.find_partial_latest_version(cpedb.get_cpe_no_version(cpe[0])) + report_cpe_needing_update += cpe[0] + ", Latest Version Guess from Dict[" + latest_version + "]\n" + report_cpe_needing_update_list += cpe[0] + "\n" + else: + report_cpe_exact_match += cpe[0] + "\n" + except (OSError, IOError) as e: + print("CPE: report csv file (%s): %s" % (e.errno, e.strerror)) + sys.exit(1) + + print("CPE: Found but may REQUIRE an UPDATE:\n" + report_cpe_needing_update) + print("CPE: Not found:\n" + report_cpe_missing) + + fp = open('cpe-report-missing.txt', 'w+') + fp.write(report_cpe_missing) + fp.close() + + for cpe in report_cpe_needing_update_list.splitlines(): + cpedb.update(cpe) + print("XML Generation Complete of NIST update files, see ./cpe/*") + + +def parse_args(): + parser = argparse.ArgumentParser() + parser.add_argument('-c', dest='cpe_report', action='store', required=True, + help='CPE Report generated by make cpe-info (csv format)') + parser.add_argument('-u', dest='url', action='store', required=False, + help='(optional)URL to the NIST dict (official-cpe-dictionary_v2.3.xml.gz)') + return parser.parse_args() + + +def __main__(): + args = parse_args() + cpedb = CPEDB() + url = CPE_XML_URL + if args.url: + url = args.url + cpedb.get_xml_dict(url) + print("Performing Target CPE Report Analysis...") + get_target_cpe_report(args.cpe_report, cpedb) + + +__main__() From patchwork Fri Mar 8 22:04:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 1053730 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44GM5X1NWrz9s7h for ; Sat, 9 Mar 2019 09:06:08 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 6F47F2A242; Fri, 8 Mar 2019 22:06:05 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zScw8UcaHqt9; Fri, 8 Mar 2019 22:05:55 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 9A10A26FB9; Fri, 8 Mar 2019 22:05:50 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 4B3C21BF5E6 for ; Fri, 8 Mar 2019 22:05:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 47F65221C6 for ; Fri, 8 Mar 2019 22:05:48 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AOnaLnP4tMAt for ; Fri, 8 Mar 2019 22:05:47 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from da1vs01.rockwellcollins.com (da1vs01.rockwellcollins.com [205.175.227.27]) by silver.osuosl.org (Postfix) with ESMTPS id 0EE601FF59 for ; Fri, 8 Mar 2019 22:05:46 +0000 (UTC) Received: from ofwda1n02.rockwellcollins.com (HELO dtulimr01.rockwellcollins.com) ([205.175.227.14]) by da1vs01.rockwellcollins.com with ESMTP; 08 Mar 2019 16:05:46 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by dtulimr01.rockwellcollins.com (Postfix) with ESMTP id 51FDC603EF; Fri, 8 Mar 2019 16:05:46 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Fri, 8 Mar 2019 16:04:27 -0600 Message-Id: <1552082667-46877-10-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1552082667-46877-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1552082667-46877-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v8 9/9] docs/manual: new security management section X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" This changeset captures an initial discussion on the use of CPE reporting within a target build. It notes the reporting limitations and provides actions a user could take to improve upon the current report information. There is also an example of how one might do CVE analysis using the CPE report information. Signed-off-by: Matthew Weber --- Changes v8 - Updated for cpe-report changes - Added notes on doing CVE searches and submissions v7 - New --- docs/manual/cpe-reporting.txt | 103 ++++++++++++++++++++++++++++++++++++++++++ docs/manual/manual.txt | 2 + 2 files changed, 105 insertions(+) create mode 100644 docs/manual/cpe-reporting.txt diff --git a/docs/manual/cpe-reporting.txt b/docs/manual/cpe-reporting.txt new file mode 100644 index 0000000..a6f32d5 --- /dev/null +++ b/docs/manual/cpe-reporting.txt @@ -0,0 +1,103 @@ +// -*- mode:doc; -*- +// vim: set syntax=asciidoc: + +[[cpe-info]] + +== Security Vulnerability Management + +There are many different vulnerability databases (open/paid). This +section documents the use of the National Vulnerability Database(NVD) +provided by the National Institute of Standards and Technology (NIST). + +Within Buildroot, the intent is to provide good reporting of the build +configuration's inventory of software. The vulnerability analysis is +assumed to occur outside of the Buildroot environment (at this time). + +=== Common Platform Enumeration (CPE) Reporting + +Buildroot consists of a series of upstream packages. Each of those +packages may have a CPE definition used to map vulnerabilities to Common +Vulnerabilities and Exposures (CVE). A single package CPE has many versions +and each version may have a suite of CVEs associated. + +To make the gathering of the software inventory of CPE easier, Buildroot can +collect for you all the CPE related to the configured defconfig. To produce +this material, after you have configured Buildroot with +make menuconfig+, ++make xconfig+ or +make gconfig+, run: + +-------------------- +make cpe-info +-------------------- + +Buildroot then collects and writes the +$(TOPDIR)/cpe-manifest.csv+. This file +can be used for manual inspection against a CVE database or provided to +external tools which perform CVE inventory/analysis. + +*CPE Maintenance* + +To maintain these CPE strings for version changes against the NIST dictionary, +the manifest can be further processed. First, navigate to your Buildroot +directory and execute the script below. The script has some optional arguments +for providing a alternate dictionary URL or caching a processed dictionary. + +-------------------- +support/scripts/cpe-report -c $(TOPDIR)/cpe-manifest.csv +-------------------- + +This script retrieves the NIST dictionary and classifies each CPE as either +matched, requires version update or missing. Based on this analysis, the script +automatically uses the NIST dictionary entries to produce a draft of XML which +can be submitted to NIST to update a version of an entry in the dictionary. It +is important to review the generated xml files in the cpe folder as they may +need refined reference tags and adjustments to how the version is represented +in the title. + +In the case of missing items, a +cpe-report-missing.txt+ report is output by +the script and can be used as a starting point to manually create a xml file +to submit. Note, some manual analysis using the NIST search engine (https://nvd.nist.gov/products/cpe/search) +is suggested for these missing item as the Buildroot +CPE_ID_+ variables maybe +slightly incorrect and cause the cpe-report script to catagorize the package +as missing. If that is the case, a change can be made by adjusting the default +CPE variables in the specific package's +.mk+. See xref:_infrastructure_for_packages_with_specific_build_systems[] +discussion on the use of +LIBFOO_CPE_*+. +If the package is truely missing, the package's Kconfig help material and .mk +should provide most of the information to construct a new NIST submission. + +To submit a new entry or updated entry to NIST, create an request email to the +cpe_dictionary@nist.gov recipient and attach a individual xml file per package +being added/updated. It is OK to have multiple version updates in a single +file as long as they are all for the same package. For reference the guidance +can be found on the NIST CPE site (https://nvd.nist.gov/products/cpe). + +*Limitations* + +Buildroot does not produce or accurately present some of the CPE material. Items +such as any versions which are non-number/hash are not compliant with the CPE +string specification and would require a manual analysis to update the CPE list +before any external CVE analysis should occur. This is a similar situation for +packages like the Linux kernel or U-Boot which may not have a version which +directly maps to a CPE. + +There is an assumed default CPE string for each package which is auto-generated +using existing package information. The output of +make cpe-info+ is based on +this default information and the packages which have been individually tailored +to match existing CPE strings. The Buildroot developers try to do their best to +keep those declarative statements as accurate as possible, to the best of their +knowledge. However, it is very well possible that those declarative statements +are not all fully accurate nor exhaustive. Similar to legal-info, it is your +responsibility to verify this information. + +=== Common Vulnerability and Exposures (CVE) Anaylsis +The Common Vulnerabilities and Exposures (CVE) system provides a +reference-method for publicly known information-security vulnerabilities and +exposures. (https://en.wikipedia.org/wiki/Common_Vulnerabilities_and_Exposures) + +Like previously stated, there are many tools and methods to perform this +analysis. The most basic example is to do a manual CVE analysis by navigating +to the NVD search engine (https://nvd.nist.gov/vuln/search) and using the CPE +string identified in the first field of the +$(TOPDIR)/cpe-manifest.csv+. +Here's an example for tcpdump (https://nvd.nist.gov/vuln/search/results?form_type=Basic&results_type=overview&query=cpe%3A2.3%3Aa%3Atcpdump%3Atcpdump%3A4.9.1%3A*%3A*%3A*%3A*%3A*%3A*%3A*&search_type=all). + +Beyond the manual search approach, the next step would be a more centralized +shared database with multi-feed support (NVD+). The cve-search project aims +to offer that type of solution (https://github.com/cve-search/cve-search). diff --git a/docs/manual/manual.txt b/docs/manual/manual.txt index 4eb4ba9..fad8bf2 100644 --- a/docs/manual/manual.txt +++ b/docs/manual/manual.txt @@ -46,6 +46,8 @@ include::legal-notice.txt[] include::beyond-buildroot.txt[] +include::cpe-reporting.txt[] + = Developer guide include::how-buildroot-works.txt[]