From patchwork Sun Mar 6 03:19:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1601654 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Uo7JOhww; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KB6Nh29L6z9sGP for ; Sun, 6 Mar 2022 14:22:59 +1100 (AEDT) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 44BA383D48; Sun, 6 Mar 2022 04:20:52 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Uo7JOhww"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id D83A183C29; Sun, 6 Mar 2022 04:20:09 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x135.google.com (mail-il1-x135.google.com [IPv6:2607:f8b0:4864:20::135]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 6558783D1C for ; Sun, 6 Mar 2022 04:19:47 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x135.google.com with SMTP id h3so3908008ila.3 for ; Sat, 05 Mar 2022 19:19:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8UGOw78q+vNiUEsFiffYWhG16XTMi3N5CJZaAstEl/0=; b=Uo7JOhwwTbV+qYZSrr6FXLBNoTmfxsmxBaY7FC//FPttFH4JsHkEBrJEMnneA3NmnI f+P/7Q2jlFa1HFBXz1nBJ7qnJNKmi+2KIkhucbik37aBkLCj1FO2BRZpzeSUS+5dJFnW eti5czQVjGdMrjhVKxmCbpc5DKllZRpjdTs6g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8UGOw78q+vNiUEsFiffYWhG16XTMi3N5CJZaAstEl/0=; b=Y8dgU2/OjgTz2TqkDkAsXbNj1raeq28vZ7jez/GdUlLzI+pC7pEbFIvxQ2tv862w52 IJVa9SUWhxtAcCNytKMx9b12QR4keR4MBZfLUPHEs29mKBdu7Y3TPvUG6YIJCNATdp9L cvajIr07UjOBasn5ypW7rgCfGnTkV9tcNu+4ZKsivPo0+alDPO1aDgvGcpkgbusLrlCo 51KdY69mbbhTQY2KBuKeMBPD1a0x6fqKz1ttWOtEz3jFjRlFStBQSwPC1cboKn7Z9SQ5 HGihjS7KNzHHmWnJneCxM5k+wRR9nSNT6j9wJ7sIFdm0cXwk2XkzFjF+6rDL2yCVP4Tf 8rDA== X-Gm-Message-State: AOAM531gCdMGluEvS8kmxJpL2tKFvr/wY5UQnkPNendm0IzXZKT+g10t a/cU+IsFkRbDI4NYSczgTRs1X3MQUKEPZA== X-Google-Smtp-Source: ABdhPJwtUFGDSGj9iVAdm8oMATGXN+mZ6f/EDTwZlYnG18hukgiwqKnZTBHSTjalsLjeCJOzTBGeMA== X-Received: by 2002:a05:6e02:f48:b0:2c6:107c:6100 with SMTP id y8-20020a056e020f4800b002c6107c6100mr5909019ilj.146.1646536785586; Sat, 05 Mar 2022 19:19:45 -0800 (PST) Received: from kiwi.bld.corp.google.com (c-67-190-101-114.hsd1.co.comcast.net. [67.190.101.114]) by smtp.gmail.com with ESMTPSA id t14-20020a056e02160e00b002c60907ec07sm8403358ilu.62.2022.03.05.19.19.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 05 Mar 2022 19:19:45 -0800 (PST) From: Simon Glass To: U-Boot Mailing List Cc: Roger Quadros , Alper Nebi Yasak , Philippe Reynes , Ivan Mikhaylov , Peter Geis , huang lin , Jeffy Chen , Simon Glass , Kever Yang , Philipp Tomsich , Tom Rini Subject: [PATCH v3 20/26] binman: Keep a separate list of entries for fit Date: Sat, 5 Mar 2022 20:19:11 -0700 Message-Id: <20220306031917.3005215-21-sjg@chromium.org> X-Mailer: git-send-email 2.35.1.616.g0bdcbb4464-goog In-Reply-To: <20220306031917.3005215-1-sjg@chromium.org> References: <20220306031917.3005215-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.5 at phobos.denx.de X-Virus-Status: Clean The current implementation sets up the FIT entries but then deletes the 'generator' ones so they don't appear in the final image. This is a bit clumsy. We cannot build the image more than once, since the generator entries are lost during the first build. Binman requires that calling BuildSectionData() multiple times returns a valid result each time. Keep a separate, private list which includes the generator nodes and use that where needed, to correct this problem. Ensure that the missing list includes removed generator entries too. Signed-off-by: Simon Glass Reviewed-by: Alper Nebi Yasak --- (no changes since v1) tools/binman/etype/fit.py | 33 ++++++++++++++++++++++++++++----- 1 file changed, 28 insertions(+), 5 deletions(-) diff --git a/tools/binman/etype/fit.py b/tools/binman/etype/fit.py index d2e3b4dea8..49f89851b0 100644 --- a/tools/binman/etype/fit.py +++ b/tools/binman/etype/fit.py @@ -163,12 +163,17 @@ class Entry_fit(Entry_section): key: relative path to entry Node (from the base of the FIT) value: Entry_section object comprising the contents of this node + _priv_entries: Internal copy of _entries which includes 'generator' + entries which are used to create the FIT, but should not be + processed as real entries. This is set up once we have the + entries """ super().__init__(section, etype, node) self._fit = None self._fit_props = {} self._fdts = None self.mkimage = None + self._priv_entries = {} def ReadNode(self): super().ReadNode() @@ -236,6 +241,10 @@ class Entry_fit(Entry_section): _add_entries(self._node, 0, self._node) + # Keep a copy of all entries, including generator entries, since these + # removed from self._entries later. + self._priv_entries = dict(self._entries) + def BuildSectionData(self, required): """Build FIT entry contents @@ -411,11 +420,12 @@ class Entry_fit(Entry_section): has_images = depth == 2 and in_images if has_images: - entry = self._entries[rel_path] + entry = self._priv_entries[rel_path] data = entry.GetData() fsw.property('data', bytes(data)) for subnode in node.subnodes: + subnode_path = f'{rel_path}/{subnode.name}' if has_images and not (subnode.name.startswith('hash') or subnode.name.startswith('signature')): # This subnode is a content node not meant to appear in @@ -423,11 +433,11 @@ class Entry_fit(Entry_section): # fsw.add_node() or _add_node() for it. pass elif self.GetImage().generate and subnode.name.startswith('@'): - subnode_path = f'{rel_path}/{subnode.name}' - entry = self._entries.get(subnode_path) _gen_node(base_node, subnode, depth, in_images) - if entry: - del self._entries[subnode_path] + # This is a generator (template) entry, so remove it from + # the list of entries used by PackEntries(), etc. Otherwise + # it will appear in the binman output + to_remove.append(subnode_path) else: with fsw.add_node(subnode.name): _add_node(base_node, depth + 1, subnode) @@ -436,10 +446,16 @@ class Entry_fit(Entry_section): # entry node fsw = libfdt.FdtSw() fsw.finish_reservemap() + to_remove = [] with fsw.add_node(''): _add_node(self._node, 0, self._node) fdt = fsw.as_fdt() + # Remove generator entries from the main list + for path in to_remove: + if path in self._entries: + del self._entries[path] + # Pack this new FDT and scan it so we can add the data later fdt.pack() data = fdt.as_bytearray() @@ -499,3 +515,10 @@ class Entry_fit(Entry_section): def AddBintools(self, btools): super().AddBintools(btools) self.mkimage = self.AddBintool(btools, 'mkimage') + + def CheckMissing(self, missing_list): + # We must use our private entry list for this since generator notes + # which are removed from self._entries will otherwise not show up as + # missing + for entry in self._priv_entries.values(): + entry.CheckMissing(missing_list)