From patchwork Tue Feb 8 18:50:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1590048 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=BekX5HAT; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JtXGJ2gGxz9s0B for ; Wed, 9 Feb 2022 05:53:28 +1100 (AEDT) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id BAF0A83E14; Tue, 8 Feb 2022 19:51:15 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="BekX5HAT"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 9D0DA83DB6; Tue, 8 Feb 2022 19:50:52 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x133.google.com (mail-il1-x133.google.com [IPv6:2607:f8b0:4864:20::133]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id AF2B183DC2 for ; Tue, 8 Feb 2022 19:50:41 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x133.google.com with SMTP id z18so14790160ilp.3 for ; Tue, 08 Feb 2022 10:50:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4S1CA/DOWEupIPb7IxT1RlGCxn/2US4T0YfOB445IMY=; b=BekX5HATUEO+D88vTQ14mywm2Z8v1SrEGScWf+UcShsciJu/YdNol4DnpU8L0HpnG5 +9Y6VFCy177FAqjuFRtE5fNDMxjKO8SzBtq+6EWSHV0GR2BIaySnYKrSruivc2hQv6ou YmKTjX9LH68x7C/MOoe+kBGUnlAnCc05iBcqg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4S1CA/DOWEupIPb7IxT1RlGCxn/2US4T0YfOB445IMY=; b=WHJQ6L45GCU90dp5d+MCL1xH/N8wFOMwAQJI1188tGnRDdvtF+9aRJCFLxdbo6Odn6 p2c33UXhXlI2U0fN4I9426h84mzxLueFwcfRRNFV78GDCUxRBj4SdgPUR0fvKVZ8zNcj OiY+utbSIs4tS6zU/+JPMGz28dT6r+D9c7ZIl9IaGSusCvlPNyc4qQgjO0KvVXgAIh6+ USy1+4u+1tw+mdGE4GRUMkjqECGb0fNKEbidpvTfkEq2KQX7POOV65+S5ZCGZD9EliJv wqMKkgVZ6eORCqfdHxIGVtHuB8o0i3xFAunWP5GZ8NXPEcW+X4TSR45tmlv0ZBFz/EnA e0XQ== X-Gm-Message-State: AOAM532SheKbwWFYMPEXhzkuYMhU/jmQK2NtjOhMV2Tb8StJbD8EvZ79 SS2yVnI+E9NIRlhWwMytqEVsab6IVYHS7w== X-Google-Smtp-Source: ABdhPJzJieSMUHQXKfS4oCICKeciANUneSWgDtjvWuz04OlWAcJ8pkxDPr02AAzH5bS67vhhkTnrtA== X-Received: by 2002:a92:ca4c:: with SMTP id q12mr2984376ilo.141.1644346240112; Tue, 08 Feb 2022 10:50:40 -0800 (PST) Received: from kiwi.bld.corp.google.com (c-67-190-101-114.hsd1.co.comcast.net. [67.190.101.114]) by smtp.gmail.com with ESMTPSA id q2sm2911844ilt.33.2022.02.08.10.50.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 10:50:39 -0800 (PST) From: Simon Glass To: U-Boot Mailing List Cc: huang lin , Jeffy Chen , Simon Glass , Kever Yang , Philipp Tomsich , Alper Nebi Yasak , Tom Rini , Roger Quadros , Philippe Reynes , Ivan Mikhaylov , Heiko Thiery Subject: [PATCH 16/24] binman: Move entry-data collection into a Entry method Date: Tue, 8 Feb 2022 11:50:00 -0700 Message-Id: <20220208114935.16.Ic1c42c06d1559ee11280381470b64b8400703627@changeid> X-Mailer: git-send-email 2.35.0.263.gb82422642f-goog In-Reply-To: <20220208185008.35843-1-sjg@chromium.org> References: <20220208185008.35843-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.5 at phobos.denx.de X-Virus-Status: Clean Collecting the data from a list of entries and putting it in a file is a useful operation that will be needed by other entry types. Put this into a method in the Entry class. Add some documentation about how to collect data for an entry type. Signed-off-by: Simon Glass --- tools/binman/binman.rst | 86 +++++++++++++++++++++++++++++++++++ tools/binman/entry.py | 28 ++++++++++++ tools/binman/etype/mkimage.py | 14 ++---- 3 files changed, 118 insertions(+), 10 deletions(-) diff --git a/tools/binman/binman.rst b/tools/binman/binman.rst index d6b95de1f5..771645380e 100644 --- a/tools/binman/binman.rst +++ b/tools/binman/binman.rst @@ -1358,6 +1358,92 @@ development, since dealing with exceptions and problems in threads is more difficult. This avoids any use of ThreadPoolExecutor. +Collecting data for an entry type +--------------------------------- + +Some entry types deal with data obtained from others. For example, +`Entry_mkimage` calls the `mkimage` tool with data from its subnodes:: + + mkimage { + args = "-n test -T script"; + + u-boot-spl { + }; + + u-boot { + }; + }; + +This shows mkimage being passed a file consisting of SPL and U-Boot proper. It +is create by calling `Entry.collect_contents_to_file()`. Note that in this case, +the data is passed to mkimage for processing but does not appear separately in +the image. It may not appear at all, depending on what mkimage does. The +contents of the `mkimage` entry are entirely dependent on the processing done +by the entry, with the provided subnodes (`u-boot-spl` and `u-boot`) simply +providing the input data for that processing. + +Note that `Entry.collect_contents_to_file()` simply concatenates the data from +the different entries together, with no control over alignment, etc. Another +approach is to subclass `Entry_section` so that those features become available, +such as `size` and `pad-byte`. Then the contents of the entry can be obtained by +calling `BuildSectionData()`. + +There are other ways to obtain data also, depending on the situation. If the +entry type is simply signing data which exists elsewhere in the image, then +you can use `Entry_collection` as a base class. It lets you use a property +called `content` which lists the entries containing data to be processed. This +is used by `Entry_vblock`, for example:: + + u_boot: u-boot { + }; + vblock { + content = <&u_boot &dtb>; + keyblock = "firmware.keyblock"; + signprivate = "firmware_data_key.vbprivk"; + version = <1>; + kernelkey = "kernel_subkey.vbpubk"; + preamble-flags = <1>; + }; + + dtb: u-boot-dtb { + }; + +which shows an image containing `u-boot` and `u-boot-dtb`, with the `vblock` +image collecting their contents to produce input for its signing process, +without affecting those entries, which still appear in the final image +untouched. + +Another example is where an entry type needs several independent pieces of input +to function. For example, `Entry_fip` allows a number of different binary blobs +to be placed in their own individual places in a custom data structure in the +output image. To make that work you can add subnodes for each of them and call +`Entry.Create()` on each subnode, as `Entry_fip` does. Then the data for each +blob can come from any suitable place, such as an `Entry_u_boot` or an +`Entry_blob` or anything else:: + + atf-fip { + fip-hdr-flags = /bits/ 64 <0x123>; + soc-fw { + fip-flags = /bits/ 64 <0x123456789abcdef>; + filename = "bl31.bin"; + }; + + u-boot { + fip-uuid = [fc 65 13 92 4a 5b 11 ec + 94 35 ff 2d 1c fc 79 9c]; + }; + }; + +The `soc-fw` node is a `blob-ext` (i.e. it reads in a named binary file) whereas +`u-boot` is a normal entry type. This works because `Entry_fip` selects the +`blob-ext` entry type if the node name (here `soc-fw`) is recognised as being +a known blob type. + +When adding new entry types you are encouraged to use subnodes to provide the +data for processing, unless the `content` approach is more suitable. Ad-hoc +properties and other methods of obtaining data are discouraged, since it adds to +confusion for users. + History / Credits ----------------- diff --git a/tools/binman/entry.py b/tools/binman/entry.py index 08770ec5f0..3eafa078ae 100644 --- a/tools/binman/entry.py +++ b/tools/binman/entry.py @@ -1108,3 +1108,31 @@ features to produce new behaviours. btool = bintool.Bintool.create(name) tools[name] = btool return btool + + def collect_contents_to_file(self, entries, prefix): + """Put the contents of a list of entries into a file + + Args: + entries (list of Entry): Entries to collect + prefix (str): Filename prefix of file to write to + + If any entry does not have contents yet, this function returns False + for the data. + + Returns: + Tuple: + bytes: Concatenated data from all the entries (or False) + str: Filename of file written (or False if no data) + str: Unique portion of filename (or False if no data) + """ + data = b'' + for entry in entries: + # First get the input data and put it in a file. If not available, + # try later. + if not entry.ObtainContents(): + return False, False, False + data += entry.GetData() + uniq = self.GetUniqueName() + fname = tools.GetOutputFilename(f'{prefix}.{uniq}') + tools.WriteFile(fname, data) + return data, fname, uniq diff --git a/tools/binman/etype/mkimage.py b/tools/binman/etype/mkimage.py index 9ecd1c2548..145a50f1aa 100644 --- a/tools/binman/etype/mkimage.py +++ b/tools/binman/etype/mkimage.py @@ -51,16 +51,10 @@ class Entry_mkimage(Entry): self.ReadEntries() def ObtainContents(self): - data = b'' - for entry in self._mkimage_entries.values(): - # First get the input data and put it in a file. If not available, - # try later. - if not entry.ObtainContents(): - return False - data += entry.GetData() - uniq = self.GetUniqueName() - input_fname = tools.GetOutputFilename('mkimage.%s' % uniq) - tools.WriteFile(input_fname, data) + data, input_fname, uniq = self.collect_contents_to_file( + self._mkimage_entries.values(), 'mkimage') + if data is False: + return False output_fname = tools.GetOutputFilename('mkimage-out.%s' % uniq) if self.mkimage.run_cmd('-d', input_fname, *self._args, output_fname) is not None: