From patchwork Mon Jul 8 19:18:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129228 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="JezAe753"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFh11fPxz9sNH for ; Tue, 9 Jul 2019 05:22:08 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id AEBE5C21E96; Mon, 8 Jul 2019 19:21:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 27A6CC21DB5; Mon, 8 Jul 2019 19:21:07 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id E5FD2C21C4A; Mon, 8 Jul 2019 19:21:03 +0000 (UTC) Received: from mail-io1-f67.google.com (mail-io1-f67.google.com [209.85.166.67]) by lists.denx.de (Postfix) with ESMTPS id 8E5C8C21C3F for ; Mon, 8 Jul 2019 19:21:02 +0000 (UTC) Received: by mail-io1-f67.google.com with SMTP id j6so37853718ioa.5 for ; Mon, 08 Jul 2019 12:21:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1A8Aul1Fj3uGQZ+rXKS5V8bUUHW9sp5/CZSfkSZcrac=; b=JezAe7537EwZanI41BVHtmYlmW0x9oFcvZLKGQ445+cKCkg7B9pRaxuYjj5XSMumbY VFN3szo617RpBE6wyFNJ0dARL2T2ZcqUNP39FlShMjoqCL2cXf4aKQySHOzbO6OFs9ge MvUVZ/86yIWLFHP+htIoBWY0+kYsZeJVdFtl0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1A8Aul1Fj3uGQZ+rXKS5V8bUUHW9sp5/CZSfkSZcrac=; b=FLR4pM0y0lrJKgcDXOuB5qBmLjcvu829nbB9nLgBoZC7UgvHPewWhB9PSrVHSkjEGi cRwJbt8+eVahrc/r9V3ZRItI15CNJwQqAbFfuDtyJXnMAnGsJ1yZPZXQPw5w5sHZCiAo k2D7hMCqR1bkR/RijtbpCIa5WOGKk5PO/3zmNdkrMrolcUurhkB9lpviPpUTyIY2IcqL e/p0pdbsf/hz8Cxc7lCUB+S/XwhRayj9e+tKWrdBvviyylmdOUToBLt4cwPfZlVImR7E yoZjxQhgWwnfvnx44OFTHNwdzMdmWmxw14tGRmLyja1y99sWbkOIM69pC9tbq9cTxxNr H2gQ== X-Gm-Message-State: APjAAAWYu6EivcNEnFjkiV2J/KzROOfxo/21nasNq2tKCsRYxfmnh/3P +ul4NNAMilvwa5aelhAYYtzDhOqYzZ8= X-Google-Smtp-Source: APXvYqwewNvn2TCnJvHNl/0sb6P1s8LckIQz90GIgat7KyeEkNvvJE1I+cAfOVsW8dVeTXCpBGsgTg== X-Received: by 2002:a5e:8210:: with SMTP id l16mr21222422iom.240.1562613660266; Mon, 08 Jul 2019 12:21:00 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.20.59 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:20:59 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:20 -0600 Message-Id: <20190708191856.138863-2-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 01/37] x86: Add ifwitool for Intel Integrated Firmware Image X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" Some Intel SoCs from about 2016 boot using an internal microcontroller via an 'IFWI' image. This is a special format which can hold firmware images. In U-Boot's case it holds u-boot-tpl.bin. Add this tool, taken from coreboot, so that we can build bootable images on apollolake SoCs. This tool itself has no tests. Some amount of coverage will be provided by the binman tests that use it, so enable building the tool on sandbox. Signed-off-by: Simon Glass --- Changes in v2: None tools/Makefile | 3 + tools/ifwitool.c | 2314 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 2317 insertions(+) create mode 100644 tools/ifwitool.c diff --git a/tools/Makefile b/tools/Makefile index 33e90a8025a..87d81a3d416 100644 --- a/tools/Makefile +++ b/tools/Makefile @@ -175,6 +175,9 @@ HOSTCFLAGS_mkexynosspl.o := -pedantic ifdtool-objs := $(LIBFDT_OBJS) ifdtool.o hostprogs-$(CONFIG_X86) += ifdtool +ifwitool-objs := ifwitool.o +hostprogs-$(CONFIG_X86)$(CONFIG_SANDBOX) += ifwitool + hostprogs-$(CONFIG_MX23) += mxsboot hostprogs-$(CONFIG_MX28) += mxsboot HOSTCFLAGS_mxsboot.o := -pedantic diff --git a/tools/ifwitool.c b/tools/ifwitool.c new file mode 100644 index 00000000000..476f0a3c6fc --- /dev/null +++ b/tools/ifwitool.c @@ -0,0 +1,2314 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * ifwitool, CLI utility for Integrated Firmware Image (IFWI) manipulation + * + * This is taken from the Coreboot project + */ + +#include +#include +#include +#include "os_support.h" + +#define __packed __attribute__((packed)) +#define KiB 1024 +#define ALIGN(x, a) __ALIGN_MASK((x), (typeof(x))(a) - 1) +#define __ALIGN_MASK(x, mask) (((x) + (mask)) & ~(mask)) +#define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0])) + +/* + * min()/max()/clamp() macros that also do + * strict type-checking.. See the + * "unnecessary" pointer comparison. + */ +#define min(x, y) ({ \ + typeof(x) _min1 = (x); \ + typeof(y) _min2 = (y); \ + (void)&_min1 == &_min2); \ + _min1 < _min2 ? _min1 : _min2; }) + +#define max(x, y) ({ \ + typeof(x) _max1 = (x); \ + typeof(y) _max2 = (y); \ + (void)(&_max1 == &_max2); \ + _max1 > _max2 ? _max1 : _max2; }) + +static int verbose = 1; + +/* Buffer and file I/O */ +struct buffer { + char *name; + char *data; + size_t offset; + size_t size; +}; + +#define ERROR(...) { fprintf(stderr, "E: " __VA_ARGS__); } +#define INFO(...) { if (verbose > 0) fprintf(stderr, "INFO: " __VA_ARGS__); } +#define DEBUG(...) { if (verbose > 1) fprintf(stderr, "DEBUG: " __VA_ARGS__); } + +/* + * BPDT is Boot Partition Descriptor Table. It is located at the start of a + * logical boot partition(LBP). It stores information about the critical + * sub-partitions present within the LBP. + * + * S-BPDT is Secondary Boot Partition Descriptor Table. It is located after the + * critical sub-partitions and contains information about the non-critical + * sub-partitions present within the LBP. + * + * Both tables are identified by BPDT_SIGNATURE stored at the start of the + * table. + */ +#define BPDT_SIGNATURE (0x000055AA) + +/* Parameters passed in by caller */ +static struct param { + const char *file_name; + const char *subpart_name; + const char *image_name; + bool dir_ops; + const char *dentry_name; +} param; + +struct bpdt_header { + /* + * This is used to identify start of BPDT. It should always be + * BPDT_SIGNATURE. + */ + uint32_t signature; + /* Count of BPDT entries present */ + uint16_t descriptor_count; + /* Version - Currently supported = 1 */ + uint16_t bpdt_version; + /* Unused - Should be 0 */ + uint32_t xor_redundant_block; + /* Version of IFWI build */ + uint32_t ifwi_version; + /* Version of FIT tool used to create IFWI */ + uint64_t fit_tool_version; +} __packed; +#define BPDT_HEADER_SIZE (sizeof(struct bpdt_header)) + +struct bpdt_entry { + /* Type of sub-partition */ + uint16_t type; + /* Attributes of sub-partition */ + uint16_t flags; + /* Offset of sub-partition from beginning of LBP */ + uint32_t offset; + /* Size in bytes of sub-partition */ + uint32_t size; +} __packed; +#define BPDT_ENTRY_SIZE (sizeof(struct bpdt_entry)) + +struct bpdt { + struct bpdt_header h; + /* In practice, this could be an array of 0 to n entries */ + struct bpdt_entry e[0]; +} __packed; + +static inline size_t get_bpdt_size(struct bpdt_header *h) +{ + return (sizeof(*h) + BPDT_ENTRY_SIZE * h->descriptor_count); +} + +/* Minimum size in bytes allocated to BPDT in IFWI */ +#define BPDT_MIN_SIZE ((size_t)512) + +/* Header to define directory header for sub-partition */ +struct subpart_dir_header { + /* Should be SUBPART_DIR_MARKER */ + uint32_t marker; + /* Number of directory entries in the sub-partition */ + uint32_t num_entries; + /* Currenty supported - 1 */ + uint8_t header_version; + /* Currenty supported - 1 */ + uint8_t entry_version; + /* Length of directory header in bytes */ + uint8_t header_length; + /* + * 2s complement of 8-bit sum from first byte of header to last byte of + * last directory entry. + */ + uint8_t checksum; + /* ASCII short name of sub-partition */ + uint8_t name[4]; +} __packed; +#define SUBPART_DIR_HEADER_SIZE \ + (sizeof(struct subpart_dir_header)) +#define SUBPART_DIR_MARKER 0x44504324 +#define SUBPART_DIR_HEADER_VERSION_SUPPORTED 1 +#define SUBPART_DIR_ENTRY_VERSION_SUPPORTED 1 + +/* Structure for each directory entry for sub-partition */ +struct subpart_dir_entry { + /* Name of directory entry - Not guaranteed to be NULL-terminated */ + uint8_t name[12]; + /* Offset of entry from beginning of sub-partition */ + uint32_t offset; + /* Length in bytes of sub-directory entry */ + uint32_t length; + /* Must be zero */ + uint32_t rsvd; +} __packed; +#define SUBPART_DIR_ENTRY_SIZE \ + (sizeof(struct subpart_dir_entry)) + +struct subpart_dir { + struct subpart_dir_header h; + /* In practice, this could be an array of 0 to n entries */ + struct subpart_dir_entry e[0]; +} __packed; + +static inline size_t subpart_dir_size(struct subpart_dir_header *h) +{ + return (sizeof(*h) + SUBPART_DIR_ENTRY_SIZE * h->num_entries); +} + +struct manifest_header { + uint32_t header_type; + uint32_t header_length; + uint32_t header_version; + uint32_t flags; + uint32_t vendor; + uint32_t date; + uint32_t size; + uint32_t id; + uint32_t rsvd; + uint64_t version; + uint32_t svn; + uint64_t rsvd1; + uint8_t rsvd2[64]; + uint32_t modulus_size; + uint32_t exponent_size; + uint8_t public_key[256]; + uint32_t exponent; + uint8_t signature[256]; +} __packed; + +#define DWORD_SIZE 4 +#define MANIFEST_HDR_SIZE (sizeof(struct manifest_header)) +#define MANIFEST_ID_MAGIC (0x324e4d24) + +struct module { + uint8_t name[12]; + uint8_t type; + uint8_t hash_alg; + uint16_t hash_size; + uint32_t metadata_size; + uint8_t metadata_hash[32]; +} __packed; + +#define MODULE_SIZE (sizeof(struct module)) + +struct signed_pkg_info_ext { + uint32_t ext_type; + uint32_t ext_length; + uint8_t name[4]; + uint32_t vcn; + uint8_t bitmap[16]; + uint32_t svn; + uint8_t rsvd[16]; +} __packed; + +#define SIGNED_PKG_INFO_EXT_TYPE 0x15 +#define SIGNED_PKG_INFO_EXT_SIZE \ + (sizeof(struct signed_pkg_info_ext)) + +/* + * Attributes for various IFWI sub-partitions. + * LIES_WITHIN_BPDT_4K = Sub-Partition should lie within the same 4K block as + * BPDT. + * NON_CRITICAL_SUBPART = Sub-Partition entry should be present in S-BPDT. + * CONTAINS_DIR = Sub-Partition contains directory. + * AUTO_GENERATED = Sub-Partition is generated by the tool. + * MANDATORY_BPDT_ENTRY = Even if sub-partition is deleted, BPDT should contain + * an entry for it with size 0 and offset 0. + */ +enum subpart_attributes { + LIES_WITHIN_BPDT_4K = (1 << 0), + NON_CRITICAL_SUBPART = (1 << 1), + CONTAINS_DIR = (1 << 2), + AUTO_GENERATED = (1 << 3), + MANDATORY_BPDT_ENTRY = (1 << 4), +}; + +/* Type value for various IFWI sub-partitions */ +enum bpdt_entry_type { + SMIP_TYPE = 0, + CSE_RBE_TYPE = 1, + CSE_BUP_TYPE = 2, + UCODE_TYPE = 3, + IBB_TYPE = 4, + S_BPDT_TYPE = 5, + OBB_TYPE = 6, + CSE_MAIN_TYPE = 7, + ISH_TYPE = 8, + CSE_IDLM_TYPE = 9, + IFP_OVERRIDE_TYPE = 10, + DEBUG_TOKENS_TYPE = 11, + UFS_PHY_TYPE = 12, + UFS_GPP_TYPE = 13, + PMC_TYPE = 14, + IUNIT_TYPE = 15, + NVM_CONFIG_TYPE = 16, + UEP_TYPE = 17, + UFS_RATE_B_TYPE = 18, + MAX_SUBPARTS = 19, +}; + +/* + * There are two order requirements for an IFWI image: + * 1. Order in which the sub-partitions lie within the BPDT entries. + * 2. Order in which the sub-partitions lie within the image. + * + * header_order defines #1 i.e. the order in which the sub-partitions should + * appear in the BPDT entries. pack_order defines #2 i.e. the order in which + * sub-partitions appear in the IFWI image. pack_order controls the offset and + * thus sub-partitions would have increasing offsets as we loop over pack_order. + */ +const enum bpdt_entry_type bpdt_header_order[MAX_SUBPARTS] = { + /* Order of the following entries is mandatory */ + CSE_IDLM_TYPE, + IFP_OVERRIDE_TYPE, + S_BPDT_TYPE, + CSE_RBE_TYPE, + UFS_PHY_TYPE, + UFS_GPP_TYPE, + /* Order of the following entries is recommended */ + UEP_TYPE, + NVM_CONFIG_TYPE, + UFS_RATE_B_TYPE, + IBB_TYPE, + SMIP_TYPE, + PMC_TYPE, + CSE_BUP_TYPE, + UCODE_TYPE, + DEBUG_TOKENS_TYPE, + IUNIT_TYPE, + CSE_MAIN_TYPE, + ISH_TYPE, + OBB_TYPE, +}; + +const enum bpdt_entry_type bpdt_pack_order[MAX_SUBPARTS] = { + /* Order of the following entries is mandatory */ + UFS_GPP_TYPE, + UFS_PHY_TYPE, + IFP_OVERRIDE_TYPE, + UEP_TYPE, + NVM_CONFIG_TYPE, + UFS_RATE_B_TYPE, + /* Order of the following entries is recommended */ + IBB_TYPE, + SMIP_TYPE, + CSE_RBE_TYPE, + PMC_TYPE, + CSE_BUP_TYPE, + UCODE_TYPE, + CSE_IDLM_TYPE, + DEBUG_TOKENS_TYPE, + S_BPDT_TYPE, + IUNIT_TYPE, + CSE_MAIN_TYPE, + ISH_TYPE, + OBB_TYPE, +}; + +/* Utility functions */ +enum ifwi_ret { + COMMAND_ERR = -1, + NO_ACTION_REQUIRED = 0, + REPACK_REQUIRED = 1, +}; + +struct dir_ops { + enum ifwi_ret (*dir_add)(int type); +}; + +static enum ifwi_ret ibbp_dir_add(int type); + +const struct subpart_info { + const char *name; + const char *readable_name; + uint32_t attr; + struct dir_ops dir_ops; +} subparts[MAX_SUBPARTS] = { + /* OEM SMIP */ + [SMIP_TYPE] = {"SMIP", "SMIP", CONTAINS_DIR, {NULL} }, + /* CSE RBE */ + [CSE_RBE_TYPE] = {"RBEP", "CSE_RBE", CONTAINS_DIR | + MANDATORY_BPDT_ENTRY, {NULL} }, + /* CSE BUP */ + [CSE_BUP_TYPE] = {"FTPR", "CSE_BUP", CONTAINS_DIR | + MANDATORY_BPDT_ENTRY, {NULL} }, + /* uCode */ + [UCODE_TYPE] = {"UCOD", "Microcode", CONTAINS_DIR, {NULL} }, + /* IBB */ + [IBB_TYPE] = {"IBBP", "Bootblock", CONTAINS_DIR, {ibbp_dir_add} }, + /* S-BPDT */ + [S_BPDT_TYPE] = {"S_BPDT", "S-BPDT", AUTO_GENERATED | + MANDATORY_BPDT_ENTRY, {NULL} }, + /* OBB */ + [OBB_TYPE] = {"OBBP", "OEM boot block", CONTAINS_DIR | + NON_CRITICAL_SUBPART, {NULL} }, + /* CSE Main */ + [CSE_MAIN_TYPE] = {"NFTP", "CSE_MAIN", CONTAINS_DIR | + NON_CRITICAL_SUBPART, {NULL} }, + /* ISH */ + [ISH_TYPE] = {"ISHP", "ISH", NON_CRITICAL_SUBPART, {NULL} }, + /* CSE IDLM */ + [CSE_IDLM_TYPE] = {"DLMP", "CSE_IDLM", CONTAINS_DIR | + MANDATORY_BPDT_ENTRY, {NULL} }, + /* IFP Override */ + [IFP_OVERRIDE_TYPE] = {"IFP_OVERRIDE", "IFP_OVERRIDE", + LIES_WITHIN_BPDT_4K | MANDATORY_BPDT_ENTRY, + {NULL} }, + /* Debug Tokens */ + [DEBUG_TOKENS_TYPE] = {"DEBUG_TOKENS", "Debug Tokens", 0, {NULL} }, + /* UFS Phy Configuration */ + [UFS_PHY_TYPE] = {"UFS_PHY", "UFS Phy", LIES_WITHIN_BPDT_4K | + MANDATORY_BPDT_ENTRY, {NULL} }, + /* UFS GPP LUN ID */ + [UFS_GPP_TYPE] = {"UFS_GPP", "UFS GPP", LIES_WITHIN_BPDT_4K | + MANDATORY_BPDT_ENTRY, {NULL} }, + /* PMC */ + [PMC_TYPE] = {"PMCP", "PMC firmware", CONTAINS_DIR, {NULL} }, + /* IUNIT */ + [IUNIT_TYPE] = {"IUNP", "IUNIT", NON_CRITICAL_SUBPART, {NULL} }, + /* NVM Config */ + [NVM_CONFIG_TYPE] = {"NVM_CONFIG", "NVM Config", 0, {NULL} }, + /* UEP */ + [UEP_TYPE] = {"UEP", "UEP", LIES_WITHIN_BPDT_4K | MANDATORY_BPDT_ENTRY, + {NULL} }, + /* UFS Rate B Config */ + [UFS_RATE_B_TYPE] = {"UFS_RATE_B", "UFS Rate B Config", 0, {NULL} }, +}; + +struct ifwi_image { + /* Data read from input file */ + struct buffer input_buff; + + /* BPDT header and entries */ + struct buffer bpdt; + size_t input_ifwi_start_offset; + size_t input_ifwi_end_offset; + + /* Subpartition content */ + struct buffer subpart_buf[MAX_SUBPARTS]; +} ifwi_image; + +/* Buffer and file I/O */ +static off_t get_file_size(FILE *f) +{ + off_t fsize; + + fseek(f, 0, SEEK_END); + fsize = ftell(f); + fseek(f, 0, SEEK_SET); + return fsize; +} + +static inline void *buffer_get(const struct buffer *b) +{ + return b->data; +} + +static inline size_t buffer_size(const struct buffer *b) +{ + return b->size; +} + +static inline size_t buffer_offset(const struct buffer *b) +{ + return b->offset; +} + +/* + * Shrink a buffer toward the beginning of its previous space. + * Afterward, buffer_delete() remains the means of cleaning it up + */ +static inline void buffer_set_size(struct buffer *b, size_t size) +{ + b->size = size; +} + +/* Splice a buffer into another buffer. Note that it's up to the caller to + * bounds check the offset and size. The resulting buffer is backed by the same + * storage as the original, so although it is valid to buffer_delete() either + * one of them, doing so releases both simultaneously + */ +static void buffer_splice(struct buffer *dest, const struct buffer *src, + size_t offset, size_t size) +{ + dest->name = src->name; + dest->data = src->data + offset; + dest->offset = src->offset + offset; + dest->size = size; +} + +/* + * Shrink a buffer toward the end of its previous space. + * Afterward, buffer_delete() remains the means of cleaning it up + */ +static inline void buffer_seek(struct buffer *b, size_t size) +{ + b->offset += size; + b->size -= size; + b->data += size; +} + +/* Returns the start of the underlying buffer, with the offset undone */ +static inline void *buffer_get_original_backing(const struct buffer *b) +{ + if (!b) + return NULL; + return buffer_get(b) - buffer_offset(b); +} + +int buffer_create(struct buffer *buffer, size_t size, const char *name) +{ + buffer->name = strdup(name); + buffer->offset = 0; + buffer->size = size; + buffer->data = (char *)malloc(buffer->size); + if (!buffer->data) { + fprintf(stderr, "%s: Insufficient memory (0x%zx).\n", __func__, + size); + } + + return !buffer->data; +} + +int buffer_write_file(struct buffer *buffer, const char *filename) +{ + FILE *fp = fopen(filename, "wb"); + + if (!fp) { + perror(filename); + return -1; + } + assert(buffer && buffer->data); + if (fwrite(buffer->data, 1, buffer->size, fp) != buffer->size) { + fprintf(stderr, "incomplete write: %s\n", filename); + fclose(fp); + return -1; + } + fclose(fp); + return 0; +} + +void buffer_delete(struct buffer *buffer) +{ + assert(buffer); + if (buffer->name) { + free(buffer->name); + buffer->name = NULL; + } + if (buffer->data) { + free(buffer_get_original_backing(buffer)); + buffer->data = NULL; + } + buffer->offset = 0; + buffer->size = 0; +} + +int buffer_from_file(struct buffer *buffer, const char *filename) +{ + FILE *fp = fopen(filename, "rb"); + + if (!fp) { + perror(filename); + return -1; + } + buffer->offset = 0; + off_t file_size = get_file_size(fp); + + if (file_size < 0) { + fprintf(stderr, "could not determine size of %s\n", filename); + fclose(fp); + return -1; + } + buffer->size = file_size; + buffer->name = strdup(filename); + buffer->data = (char *)malloc(buffer->size); + assert(buffer->data); + if (fread(buffer->data, 1, buffer->size, fp) != buffer->size) { + fprintf(stderr, "incomplete read: %s\n", filename); + fclose(fp); + buffer_delete(buffer); + return -1; + } + fclose(fp); + return 0; +} + +static void alloc_buffer(struct buffer *b, size_t s, const char *n) +{ + if (buffer_create(b, s, n) == 0) + return; + + ERROR("Buffer allocation failure for %s (size = %zx).\n", n, s); + exit(-1); +} + +/* Little-Endian functions */ +static inline uint8_t read_ble8(const void *src) +{ + const uint8_t *s = src; + return *s; +} + +static inline uint8_t read_at_ble8(const void *src, size_t offset) +{ + const uint8_t *s = src; + + s += offset; + return read_ble8(s); +} + +static inline void write_ble8(void *dest, uint8_t val) +{ + *(uint8_t *)dest = val; +} + +static inline void write_at_ble8(void *dest, uint8_t val, size_t offset) +{ + uint8_t *d = dest; + + d += offset; + write_ble8(d, val); +} + +static inline uint8_t read_le8(const void *src) +{ + return read_ble8(src); +} + +static inline uint8_t read_at_le8(const void *src, size_t offset) +{ + return read_at_ble8(src, offset); +} + +static inline void write_le8(void *dest, uint8_t val) +{ + write_ble8(dest, val); +} + +static inline void write_at_le8(void *dest, uint8_t val, size_t offset) +{ + write_at_ble8(dest, val, offset); +} + +static inline uint16_t read_le16(const void *src) +{ + const uint8_t *s = src; + + return (((uint16_t)s[1]) << 8) | (((uint16_t)s[0]) << 0); +} + +static inline uint16_t read_at_le16(const void *src, size_t offset) +{ + const uint8_t *s = src; + + s += offset; + return read_le16(s); +} + +static inline void write_le16(void *dest, uint16_t val) +{ + write_le8(dest, val >> 0); + write_at_le8(dest, val >> 8, sizeof(uint8_t)); +} + +static inline void write_at_le16(void *dest, uint16_t val, size_t offset) +{ + uint8_t *d = dest; + + d += offset; + write_le16(d, val); +} + +static inline uint32_t read_le32(const void *src) +{ + const uint8_t *s = src; + + return (((uint32_t)s[3]) << 24) | (((uint32_t)s[2]) << 16) | + (((uint32_t)s[1]) << 8) | (((uint32_t)s[0]) << 0); +} + +static inline uint32_t read_at_le32(const void *src, size_t offset) +{ + const uint8_t *s = src; + + s += offset; + return read_le32(s); +} + +static inline void write_le32(void *dest, uint32_t val) +{ + write_le16(dest, val >> 0); + write_at_le16(dest, val >> 16, sizeof(uint16_t)); +} + +static inline void write_at_le32(void *dest, uint32_t val, size_t offset) +{ + uint8_t *d = dest; + + d += offset; + write_le32(d, val); +} + +static inline uint64_t read_le64(const void *src) +{ + uint64_t val; + + val = read_at_le32(src, sizeof(uint32_t)); + val <<= 32; + val |= read_le32(src); + return val; +} + +static inline uint64_t read_at_le64(const void *src, size_t offset) +{ + const uint8_t *s = src; + + s += offset; + return read_le64(s); +} + +static inline void write_le64(void *dest, uint64_t val) +{ + write_le32(dest, val >> 0); + write_at_le32(dest, val >> 32, sizeof(uint32_t)); +} + +static inline void write_at_le64(void *dest, uint64_t val, size_t offset) +{ + uint8_t *d = dest; + + d += offset; + write_le64(d, val); +} + +static inline void zero_n(void *dest, size_t n) +{ + memset(dest, 0, n); +} + +/* + * Read header/entry members in little-endian format. + * Returns the offset upto which the read was performed. + */ +static size_t read_member(void *src, size_t offset, size_t size_bytes, + void *dst) +{ + switch (size_bytes) { + case 1: + *(uint8_t *)dst = read_at_le8(src, offset); + break; + case 2: + *(uint16_t *)dst = read_at_le16(src, offset); + break; + case 4: + *(uint32_t *)dst = read_at_le32(src, offset); + break; + case 8: + *(uint64_t *)dst = read_at_le64(src, offset); + break; + default: + ERROR("Read size not supported %zd\n", size_bytes); + exit(-1); + } + + return (offset + size_bytes); +} + +/* + * Convert to little endian format. + * Returns the offset upto which the fixup was performed. + */ +static size_t fix_member(void *data, size_t offset, size_t size_bytes) +{ + uint8_t *src = (uint8_t *)data + offset; + + switch (size_bytes) { + case 1: + write_at_le8(data, *(uint8_t *)src, offset); + break; + case 2: + write_at_le16(data, *(uint16_t *)src, offset); + break; + case 4: + write_at_le32(data, *(uint32_t *)src, offset); + break; + case 8: + write_at_le64(data, *(uint64_t *)src, offset); + break; + default: + ERROR("Write size not supported %zd\n", size_bytes); + exit(-1); + } + return (offset + size_bytes); +} + +static void print_subpart_dir(struct subpart_dir *s) +{ + if (verbose == 0) + return; + + size_t i; + + printf("%-25s 0x%-23.8x\n", "Marker", s->h.marker); + printf("%-25s %-25d\n", "Num entries", s->h.num_entries); + printf("%-25s %-25d\n", "Header Version", s->h.header_version); + printf("%-25s %-25d\n", "Entry Version", s->h.entry_version); + printf("%-25s 0x%-23x\n", "Header Length", s->h.header_length); + printf("%-25s 0x%-23x\n", "Checksum", s->h.checksum); + printf("%-25s ", "Name"); + for (i = 0; i < sizeof(s->h.name); i++) + printf("%c", s->h.name[i]); + + printf("\n"); + + printf("%-25s%-25s%-25s%-25s%-25s\n", "Entry #", "Name", "Offset", + "Length", "Rsvd"); + + printf("=========================================================================================================================\n"); + + for (i = 0; i < s->h.num_entries; i++) { + printf("%-25zd%-25.12s0x%-23x0x%-23x0x%-23x\n", i + 1, + s->e[i].name, s->e[i].offset, s->e[i].length, + s->e[i].rsvd); + } + + printf("=========================================================================================================================\n"); +} + +static void bpdt_print_header(struct bpdt_header *h, const char *name) +{ + if (verbose == 0) + return; + + printf("%-25s %-25s\n", "Header", name); + printf("%-25s 0x%-23.8x\n", "Signature", h->signature); + printf("%-25s %-25d\n", "Descriptor count", h->descriptor_count); + printf("%-25s %-25d\n", "BPDT Version", h->bpdt_version); + printf("%-25s 0x%-23x\n", "XOR checksum", h->xor_redundant_block); + printf("%-25s 0x%-23x\n", "IFWI Version", h->ifwi_version); + printf("%-25s 0x%-23llx\n", "FIT Tool Version", + (long long)h->fit_tool_version); +} + +static void bpdt_print_entries(struct bpdt_entry *e, size_t count, + const char *name) +{ + size_t i; + + if (verbose == 0) + return; + + printf("%s entries\n", name); + + printf("%-25s%-25s%-25s%-25s%-25s%-25s%-25s%-25s\n", "Entry #", + "Sub-Partition", "Name", "Type", "Flags", "Offset", "Size", + "File Offset"); + + printf("=========================================================================================================================================================================================================\n"); + + for (i = 0; i < count; i++) { + printf("%-25zd%-25s%-25s%-25d0x%-23.08x0x%-23x0x%-23x0x%-23zx\n", + i + 1, subparts[e[i].type].name, + subparts[e[i].type].readable_name, e[i].type, e[i].flags, + e[i].offset, e[i].size, + e[i].offset + ifwi_image.input_ifwi_start_offset); + } + + printf("=========================================================================================================================================================================================================\n"); +} + +static void bpdt_validate_header(struct bpdt_header *h, const char *name) +{ + assert(h->signature == BPDT_SIGNATURE); + + if (h->bpdt_version != 1) { + ERROR("Invalid header : %s\n", name); + exit(-1); + } + + DEBUG("Validated header : %s\n", name); +} + +static void bpdt_read_header(void *data, struct bpdt_header *h, + const char *name) +{ + size_t offset = 0; + + offset = read_member(data, offset, sizeof(h->signature), &h->signature); + offset = read_member(data, offset, sizeof(h->descriptor_count), + &h->descriptor_count); + offset = read_member(data, offset, sizeof(h->bpdt_version), + &h->bpdt_version); + offset = read_member(data, offset, sizeof(h->xor_redundant_block), + &h->xor_redundant_block); + offset = read_member(data, offset, sizeof(h->ifwi_version), + &h->ifwi_version); + read_member(data, offset, sizeof(h->fit_tool_version), + &h->fit_tool_version); + + bpdt_validate_header(h, name); + bpdt_print_header(h, name); +} + +static void bpdt_read_entries(void *data, struct bpdt *bpdt, const char *name) +{ + size_t i, offset = 0; + struct bpdt_entry *e = &bpdt->e[0]; + size_t count = bpdt->h.descriptor_count; + + for (i = 0; i < count; i++) { + offset = read_member(data, offset, sizeof(e[i].type), + &e[i].type); + offset = read_member(data, offset, sizeof(e[i].flags), + &e[i].flags); + offset = read_member(data, offset, sizeof(e[i].offset), + &e[i].offset); + offset = read_member(data, offset, sizeof(e[i].size), + &e[i].size); + } + + bpdt_print_entries(e, count, name); +} + +/* + * Given type of sub-partition, identify BPDT entry for it. + * Sub-Partition could lie either within BPDT or S-BPDT. + */ +static struct bpdt_entry *__find_entry_by_type(struct bpdt_entry *e, + size_t count, int type) +{ + size_t i; + + for (i = 0; i < count; i++) { + if (e[i].type == type) + break; + } + + if (i == count) + return NULL; + + return &e[i]; +} + +static struct bpdt_entry *find_entry_by_type(int type) +{ + struct bpdt *b = buffer_get(&ifwi_image.bpdt); + + if (!b) + return NULL; + + struct bpdt_entry *curr = __find_entry_by_type(&b->e[0], + b->h.descriptor_count, + type); + + if (curr) + return curr; + + b = buffer_get(&ifwi_image.subpart_buf[S_BPDT_TYPE]); + if (!b) + return NULL; + + return __find_entry_by_type(&b->e[0], b->h.descriptor_count, type); +} + +/* + * Find sub-partition type given its name. If the name does not exist, returns + * -1. + */ +static int find_type_by_name(const char *name) +{ + int i; + + for (i = 0; i < MAX_SUBPARTS; i++) { + if ((strlen(subparts[i].name) == strlen(name)) && + (!strcmp(subparts[i].name, name))) + break; + } + + if (i == MAX_SUBPARTS) { + ERROR("Invalid sub-partition name %s.\n", name); + return -1; + } + + return i; +} + +/* + * Read the content of a sub-partition from input file and store it in + * ifwi_image.subpart_buf[SUB-PARTITION_TYPE]. + * + * Returns the maximum offset occupied by the sub-partitions. + */ +static size_t read_subpart_buf(void *data, size_t size, struct bpdt_entry *e, + size_t count) +{ + size_t i, type; + struct buffer *buf; + size_t max_offset = 0; + + for (i = 0; i < count; i++) { + type = e[i].type; + + if (type >= MAX_SUBPARTS) { + ERROR("Invalid sub-partition type %zd.\n", type); + exit(-1); + } + + if (buffer_size(&ifwi_image.subpart_buf[type])) { + ERROR("Multiple sub-partitions of type %zd(%s).\n", + type, subparts[type].name); + exit(-1); + } + + if (e[i].size == 0) { + INFO("Dummy sub-partition %zd(%s). Skipping.\n", type, + subparts[type].name); + continue; + } + + assert((e[i].offset + e[i].size) <= size); + + /* + * Sub-partitions in IFWI image are not in the same order as + * in BPDT entries. BPDT entires are in header_order whereas + * sub-partition offsets in the image are in pack_order. + */ + if ((e[i].offset + e[i].size) > max_offset) + max_offset = e[i].offset + e[i].size; + + /* + * S-BPDT sub-partition contains information about all the + * non-critical sub-partitions. Thus, size of S-BPDT + * sub-partition equals size of S-BPDT plus size of all the + * non-critical sub-partitions. Thus, reading whole of S-BPDT + * here would be redundant as the non-critical partitions are + * read and allocated buffers separately. Also, S-BPDT requires + * special handling for reading header and entries. + */ + if (type == S_BPDT_TYPE) + continue; + + buf = &ifwi_image.subpart_buf[type]; + + alloc_buffer(buf, e[i].size, subparts[type].name); + memcpy(buffer_get(buf), (uint8_t *)data + e[i].offset, + e[i].size); + } + + assert(max_offset); + return max_offset; +} + +/* + * Allocate buffer for bpdt header, entries and all sub-partition content. + * Returns offset in data where BPDT ends. + */ +static size_t alloc_bpdt_buffer(void *data, size_t size, size_t offset, + struct buffer *b, const char *name) +{ + struct bpdt_header bpdt_header; + + assert((offset + BPDT_HEADER_SIZE) < size); + bpdt_read_header((uint8_t *)data + offset, &bpdt_header, name); + + /* Buffer to read BPDT header and entries */ + alloc_buffer(b, get_bpdt_size(&bpdt_header), name); + + struct bpdt *bpdt = buffer_get(b); + + memcpy(&bpdt->h, &bpdt_header, BPDT_HEADER_SIZE); + + /* + * If no entries are present, maximum offset occupied is (offset + + * BPDT_HEADER_SIZE). + */ + if (bpdt->h.descriptor_count == 0) + return (offset + BPDT_HEADER_SIZE); + + /* Read all entries */ + assert((offset + get_bpdt_size(&bpdt->h)) < size); + bpdt_read_entries((uint8_t *)data + offset + BPDT_HEADER_SIZE, bpdt, + name); + + /* Read all sub-partition content in subpart_buf */ + return read_subpart_buf(data, size, &bpdt->e[0], + bpdt->h.descriptor_count); +} + +static void parse_sbpdt(void *data, size_t size) +{ + struct bpdt_entry *s; + + s = find_entry_by_type(S_BPDT_TYPE); + if (!s) + return; + + assert(size > s->offset); + + alloc_bpdt_buffer(data, size, s->offset, + &ifwi_image.subpart_buf[S_BPDT_TYPE], + "S-BPDT"); +} + +static uint8_t calc_checksum(struct subpart_dir *s) +{ + size_t size = subpart_dir_size(&s->h); + uint8_t *data = (uint8_t *)s; + uint8_t checksum = 0; + size_t i; + uint8_t old_checksum = s->h.checksum; + + s->h.checksum = 0; + + for (i = 0; i < size; i++) + checksum += data[i]; + + s->h.checksum = old_checksum; + + /* 2s complement */ + return -checksum; +} + +static void validate_subpart_dir(struct subpart_dir *s, const char *name, + bool checksum_check) +{ + if (s->h.marker != SUBPART_DIR_MARKER || + s->h.header_version != SUBPART_DIR_HEADER_VERSION_SUPPORTED || + s->h.entry_version != SUBPART_DIR_ENTRY_VERSION_SUPPORTED || + s->h.header_length != SUBPART_DIR_HEADER_SIZE) { + ERROR("Invalid subpart_dir for %s.\n", name); + exit(-1); + } + + if (!checksum_check) + return; + + uint8_t checksum = calc_checksum(s); + + if (checksum != s->h.checksum) + ERROR("Invalid checksum for %s (Expected=0x%x, Actual=0x%x).\n", + name, checksum, s->h.checksum); +} + +static void validate_subpart_dir_without_checksum(struct subpart_dir *s, + const char *name) +{ + validate_subpart_dir(s, name, 0); +} + +static void validate_subpart_dir_with_checksum(struct subpart_dir *s, + const char *name) +{ + validate_subpart_dir(s, name, 1); +} + +static void parse_subpart_dir(struct buffer *subpart_dir_buf, + struct buffer *input_buf, const char *name) +{ + struct subpart_dir_header hdr; + size_t offset = 0; + uint8_t *data = buffer_get(input_buf); + size_t size = buffer_size(input_buf); + + /* Read Subpart_Dir header */ + assert(size >= SUBPART_DIR_HEADER_SIZE); + offset = read_member(data, offset, sizeof(hdr.marker), &hdr.marker); + offset = read_member(data, offset, sizeof(hdr.num_entries), + &hdr.num_entries); + offset = read_member(data, offset, sizeof(hdr.header_version), + &hdr.header_version); + offset = read_member(data, offset, sizeof(hdr.entry_version), + &hdr.entry_version); + offset = read_member(data, offset, sizeof(hdr.header_length), + &hdr.header_length); + offset = read_member(data, offset, sizeof(hdr.checksum), &hdr.checksum); + memcpy(hdr.name, data + offset, sizeof(hdr.name)); + offset += sizeof(hdr.name); + + validate_subpart_dir_without_checksum((struct subpart_dir *)&hdr, name); + + assert(size > subpart_dir_size(&hdr)); + alloc_buffer(subpart_dir_buf, subpart_dir_size(&hdr), "Subpart Dir"); + memcpy(buffer_get(subpart_dir_buf), &hdr, SUBPART_DIR_HEADER_SIZE); + + /* Read Subpart Dir entries */ + struct subpart_dir *subpart_dir = buffer_get(subpart_dir_buf); + struct subpart_dir_entry *e = &subpart_dir->e[0]; + uint32_t i; + + for (i = 0; i < hdr.num_entries; i++) { + memcpy(e[i].name, data + offset, sizeof(e[i].name)); + offset += sizeof(e[i].name); + offset = read_member(data, offset, sizeof(e[i].offset), + &e[i].offset); + offset = read_member(data, offset, sizeof(e[i].length), + &e[i].length); + offset = read_member(data, offset, sizeof(e[i].rsvd), + &e[i].rsvd); + } + + validate_subpart_dir_with_checksum(subpart_dir, name); + + print_subpart_dir(subpart_dir); +} + +/* Parse input image file to identify different sub-partitions */ +static int ifwi_parse(void) +{ + struct buffer *buff = &ifwi_image.input_buff; + const char *image_name = param.image_name; + + DEBUG("Parsing IFWI image...\n"); + + /* Read input file */ + if (buffer_from_file(buff, image_name)) { + ERROR("Failed to read input file %s.\n", image_name); + return -1; + } + + INFO("Buffer %p size 0x%zx\n", buff->data, buff->size); + + /* Look for BPDT signature at 4K intervals */ + size_t offset = 0; + void *data = buffer_get(buff); + + while (offset < buffer_size(buff)) { + if (read_at_le32(data, offset) == BPDT_SIGNATURE) + break; + offset += 4 * KiB; + } + + if (offset >= buffer_size(buff)) { + ERROR("Image does not contain BPDT!!\n"); + return -1; + } + + ifwi_image.input_ifwi_start_offset = offset; + INFO("BPDT starts at offset 0x%zx.\n", offset); + + data = (uint8_t *)data + offset; + size_t ifwi_size = buffer_size(buff) - offset; + + /* Read BPDT and sub-partitions */ + uintptr_t end_offset; + + end_offset = ifwi_image.input_ifwi_start_offset + + alloc_bpdt_buffer(data, ifwi_size, 0, &ifwi_image.bpdt, "BPDT"); + + /* Parse S-BPDT, if any */ + parse_sbpdt(data, ifwi_size); + + /* + * Store end offset of IFWI. Required for copying any trailing non-IFWI + * part of the image. + * ASSUMPTION: IFWI image always ends on a 4K boundary. + */ + ifwi_image.input_ifwi_end_offset = ALIGN(end_offset, 4 * KiB); + DEBUG("Parsing done.\n"); + + return 0; +} + +/* + * This function is used by repack to count the number of BPDT and S-BPDT + * entries that are present. It frees the current buffers used by the entries + * and allocates fresh buffers that can be used for repacking. Returns BPDT + * entries which are empty and need to be filled in. + */ +static void __bpdt_reset(struct buffer *b, size_t count, size_t size) +{ + size_t bpdt_size = BPDT_HEADER_SIZE + count * BPDT_ENTRY_SIZE; + + assert(size >= bpdt_size); + + /* + * If buffer does not have the required size, allocate a fresh buffer. + */ + if (buffer_size(b) != size) { + struct buffer temp; + + alloc_buffer(&temp, size, b->name); + memcpy(buffer_get(&temp), buffer_get(b), buffer_size(b)); + buffer_delete(b); + *b = temp; + } + + struct bpdt *bpdt = buffer_get(b); + uint8_t *ptr = (uint8_t *)&bpdt->e[0]; + size_t entries_size = BPDT_ENTRY_SIZE * count; + + /* Zero out BPDT entries */ + memset(ptr, 0, entries_size); + /* Fill any pad-space with FF */ + memset(ptr + entries_size, 0xFF, size - bpdt_size); + + bpdt->h.descriptor_count = count; +} + +static void bpdt_reset(void) +{ + size_t i; + size_t bpdt_count = 0, sbpdt_count = 0, dummy_bpdt_count = 0; + + /* Count number of BPDT and S-BPDT entries */ + for (i = 0; i < MAX_SUBPARTS; i++) { + if (buffer_size(&ifwi_image.subpart_buf[i]) == 0) { + if (subparts[i].attr & MANDATORY_BPDT_ENTRY) { + bpdt_count++; + dummy_bpdt_count++; + } + continue; + } + + if (subparts[i].attr & NON_CRITICAL_SUBPART) + sbpdt_count++; + else + bpdt_count++; + } + + DEBUG("Count: BPDT = %zd, Dummy BPDT = %zd, S-BPDT = %zd\n", bpdt_count, + dummy_bpdt_count, sbpdt_count); + + /* Update BPDT if required */ + size_t bpdt_size = max(BPDT_MIN_SIZE, + BPDT_HEADER_SIZE + bpdt_count * BPDT_ENTRY_SIZE); + __bpdt_reset(&ifwi_image.bpdt, bpdt_count, bpdt_size); + + /* Update S-BPDT if required */ + bpdt_size = ALIGN(BPDT_HEADER_SIZE + sbpdt_count * BPDT_ENTRY_SIZE, + 4 * KiB); + __bpdt_reset(&ifwi_image.subpart_buf[S_BPDT_TYPE], sbpdt_count, + bpdt_size); +} + +/* Initialize BPDT entries in header order */ +static void bpdt_entries_init_header_order(void) +{ + int i, type; + size_t size; + + struct bpdt *bpdt, *sbpdt, *curr; + size_t bpdt_curr = 0, sbpdt_curr = 0, *count_ptr; + + bpdt = buffer_get(&ifwi_image.bpdt); + sbpdt = buffer_get(&ifwi_image.subpart_buf[S_BPDT_TYPE]); + + for (i = 0; i < MAX_SUBPARTS; i++) { + type = bpdt_header_order[i]; + size = buffer_size(&ifwi_image.subpart_buf[type]); + + if (size == 0 && !(subparts[type].attr & MANDATORY_BPDT_ENTRY)) + continue; + + if (subparts[type].attr & NON_CRITICAL_SUBPART) { + curr = sbpdt; + count_ptr = &sbpdt_curr; + } else { + curr = bpdt; + count_ptr = &bpdt_curr; + } + + assert(*count_ptr < curr->h.descriptor_count); + curr->e[*count_ptr].type = type; + curr->e[*count_ptr].flags = 0; + curr->e[*count_ptr].offset = 0; + curr->e[*count_ptr].size = size; + + (*count_ptr)++; + } +} + +static void pad_buffer(struct buffer *b, size_t size) +{ + size_t buff_size = buffer_size(b); + + assert(buff_size <= size); + + if (buff_size == size) + return; + + struct buffer temp; + + alloc_buffer(&temp, size, b->name); + uint8_t *data = buffer_get(&temp); + + memcpy(data, buffer_get(b), buff_size); + memset(data + buff_size, 0xFF, size - buff_size); + + *b = temp; +} + +/* Initialize offsets of entries using pack order */ +static void bpdt_entries_init_pack_order(void) +{ + int i, type; + struct bpdt_entry *curr; + size_t curr_offset, curr_end; + + curr_offset = max(BPDT_MIN_SIZE, buffer_size(&ifwi_image.bpdt)); + + /* + * There are two types of sub-partitions that need to be handled here: + * 1. Sub-partitions that lie within the same 4K as BPDT + * 2. Sub-partitions that lie outside the 4K of BPDT + * + * For sub-partitions of type # 1, there is no requirement on the start + * or end of the sub-partition. They need to be packed in without any + * holes left in between. If there is any empty space left after the end + * of the last sub-partition in 4K of BPDT, then that space needs to be + * padded with FF bytes, but the size of the last sub-partition remains + * unchanged. + * + * For sub-partitions of type # 2, both the start and end should be a + * multiple of 4K. If not, then it needs to be padded with FF bytes and + * size adjusted such that the sub-partition ends on 4K boundary. + */ + + /* #1 Sub-partitions that lie within same 4K as BPDT */ + struct buffer *last_bpdt_buff = &ifwi_image.bpdt; + + for (i = 0; i < MAX_SUBPARTS; i++) { + type = bpdt_pack_order[i]; + curr = find_entry_by_type(type); + + if (!curr || curr->size == 0) + continue; + + if (!(subparts[type].attr & LIES_WITHIN_BPDT_4K)) + continue; + + curr->offset = curr_offset; + curr_offset = curr->offset + curr->size; + last_bpdt_buff = &ifwi_image.subpart_buf[type]; + DEBUG("type=%d, curr_offset=0x%zx, curr->offset=0x%x, curr->size=0x%x, buff_size=0x%zx\n", + type, curr_offset, curr->offset, curr->size, + buffer_size(&ifwi_image.subpart_buf[type])); + } + + /* Pad ff bytes if there is any empty space left in BPDT 4K */ + curr_end = ALIGN(curr_offset, 4 * KiB); + pad_buffer(last_bpdt_buff, + buffer_size(last_bpdt_buff) + (curr_end - curr_offset)); + curr_offset = curr_end; + + /* #2 Sub-partitions that lie outside of BPDT 4K */ + for (i = 0; i < MAX_SUBPARTS; i++) { + type = bpdt_pack_order[i]; + curr = find_entry_by_type(type); + + if (!curr || curr->size == 0) + continue; + + if (subparts[type].attr & LIES_WITHIN_BPDT_4K) + continue; + + assert(curr_offset == ALIGN(curr_offset, 4 * KiB)); + curr->offset = curr_offset; + curr_end = ALIGN(curr->offset + curr->size, 4 * KiB); + curr->size = curr_end - curr->offset; + + pad_buffer(&ifwi_image.subpart_buf[type], curr->size); + + curr_offset = curr_end; + DEBUG("type=%d, curr_offset=0x%zx, curr->offset=0x%x, curr->size=0x%x, buff_size=0x%zx\n", + type, curr_offset, curr->offset, curr->size, + buffer_size(&ifwi_image.subpart_buf[type])); + } + + /* + * Update size of S-BPDT to include size of all non-critical + * sub-partitions. + * + * Assumption: S-BPDT always lies at the end of IFWI image. + */ + curr = find_entry_by_type(S_BPDT_TYPE); + assert(curr); + + assert(curr_offset == ALIGN(curr_offset, 4 * KiB)); + curr->size = curr_offset - curr->offset; +} + +/* Convert all members of BPDT to little-endian format */ +static void bpdt_fixup_write_buffer(struct buffer *buf) +{ + struct bpdt *s = buffer_get(buf); + + struct bpdt_header *h = &s->h; + struct bpdt_entry *e = &s->e[0]; + + size_t count = h->descriptor_count; + + size_t offset = 0; + + offset = fix_member(&h->signature, offset, sizeof(h->signature)); + offset = fix_member(&h->descriptor_count, offset, + sizeof(h->descriptor_count)); + offset = fix_member(&h->bpdt_version, offset, sizeof(h->bpdt_version)); + offset = fix_member(&h->xor_redundant_block, offset, + sizeof(h->xor_redundant_block)); + offset = fix_member(&h->ifwi_version, offset, sizeof(h->ifwi_version)); + offset = fix_member(&h->fit_tool_version, offset, + sizeof(h->fit_tool_version)); + + uint32_t i; + + for (i = 0; i < count; i++) { + offset = fix_member(&e[i].type, offset, sizeof(e[i].type)); + offset = fix_member(&e[i].flags, offset, sizeof(e[i].flags)); + offset = fix_member(&e[i].offset, offset, sizeof(e[i].offset)); + offset = fix_member(&e[i].size, offset, sizeof(e[i].size)); + } +} + +/* Write BPDT to output buffer after fixup */ +static void bpdt_write(struct buffer *dst, size_t offset, struct buffer *src) +{ + bpdt_fixup_write_buffer(src); + memcpy(buffer_get(dst) + offset, buffer_get(src), buffer_size(src)); +} + +/* + * Follows these steps to re-create image: + * 1. Write any non-IFWI prefix. + * 2. Write out BPDT header and entries. + * 3. Write sub-partition buffers to respective offsets. + * 4. Write any non-IFWI suffix. + * + * While performing the above steps, make sure that any empty holes are filled + * with FF. + */ +static void ifwi_write(const char *image_name) +{ + struct bpdt_entry *s = find_entry_by_type(S_BPDT_TYPE); + + assert(s); + + size_t ifwi_start, ifwi_end, file_end; + + ifwi_start = ifwi_image.input_ifwi_start_offset; + ifwi_end = ifwi_start + ALIGN(s->offset + s->size, 4 * KiB); + file_end = ifwi_end + (buffer_size(&ifwi_image.input_buff) - + ifwi_image.input_ifwi_end_offset); + + struct buffer b; + + alloc_buffer(&b, file_end, "Final-IFWI"); + + uint8_t *input_data = buffer_get(&ifwi_image.input_buff); + uint8_t *output_data = buffer_get(&b); + + DEBUG("ifwi_start:0x%zx, ifwi_end:0x%zx, file_end:0x%zx\n", ifwi_start, + ifwi_end, file_end); + + /* Copy non-IFWI prefix, if any */ + memcpy(output_data, input_data, ifwi_start); + + DEBUG("Copied non-IFWI prefix (offset=0x0, size=0x%zx).\n", ifwi_start); + + struct buffer ifwi; + + buffer_splice(&ifwi, &b, ifwi_start, ifwi_end - ifwi_start); + uint8_t *ifwi_data = buffer_get(&ifwi); + + /* Copy sub-partitions using pack_order */ + struct bpdt_entry *curr; + struct buffer *subpart_buf; + int i, type; + + for (i = 0; i < MAX_SUBPARTS; i++) { + type = bpdt_pack_order[i]; + + if (type == S_BPDT_TYPE) + continue; + + curr = find_entry_by_type(type); + + if (!curr || !curr->size) + continue; + + subpart_buf = &ifwi_image.subpart_buf[type]; + + DEBUG("curr->offset=0x%x, curr->size=0x%x, type=%d, write_size=0x%zx\n", + curr->offset, curr->size, type, buffer_size(subpart_buf)); + + assert((curr->offset + buffer_size(subpart_buf)) <= + buffer_size(&ifwi)); + + memcpy(ifwi_data + curr->offset, buffer_get(subpart_buf), + buffer_size(subpart_buf)); + } + + /* Copy non-IFWI suffix, if any */ + if (ifwi_end != file_end) { + memcpy(output_data + ifwi_end, + input_data + ifwi_image.input_ifwi_end_offset, + file_end - ifwi_end); + DEBUG("Copied non-IFWI suffix (offset=0x%zx,size=0x%zx).\n", + ifwi_end, file_end - ifwi_end); + } + + /* + * Convert BPDT to little-endian format and write it to output buffer. + * S-BPDT is written first and then BPDT. + */ + bpdt_write(&ifwi, s->offset, &ifwi_image.subpart_buf[S_BPDT_TYPE]); + bpdt_write(&ifwi, 0, &ifwi_image.bpdt); + + if (buffer_write_file(&b, image_name)) { + ERROR("File write error\n"); + exit(-1); + } + + buffer_delete(&b); + printf("Image written successfully to %s.\n", image_name); +} + +/* + * Calculate size and offset of each sub-partition again since it might have + * changed because of add/delete operation. Also, re-create BPDT and S-BPDT + * entries and write back the new IFWI image to file. + */ +static void ifwi_repack(void) +{ + bpdt_reset(); + bpdt_entries_init_header_order(); + bpdt_entries_init_pack_order(); + + struct bpdt *b = buffer_get(&ifwi_image.bpdt); + + bpdt_print_entries(&b->e[0], b->h.descriptor_count, "BPDT"); + + b = buffer_get(&ifwi_image.subpart_buf[S_BPDT_TYPE]); + bpdt_print_entries(&b->e[0], b->h.descriptor_count, "S-BPDT"); + + DEBUG("Repack done.. writing image.\n"); + ifwi_write(param.image_name); +} + +static void init_subpart_dir_header(struct subpart_dir_header *hdr, + size_t count, const char *name) +{ + memset(hdr, 0, sizeof(*hdr)); + + hdr->marker = SUBPART_DIR_MARKER; + hdr->num_entries = count; + hdr->header_version = SUBPART_DIR_HEADER_VERSION_SUPPORTED; + hdr->entry_version = SUBPART_DIR_ENTRY_VERSION_SUPPORTED; + hdr->header_length = SUBPART_DIR_HEADER_SIZE; + memcpy(hdr->name, name, sizeof(hdr->name)); +} + +static size_t init_subpart_dir_entry(struct subpart_dir_entry *e, + struct buffer *b, size_t offset) +{ + memset(e, 0, sizeof(*e)); + + assert(strlen(b->name) <= sizeof(e->name)); + strncpy((char *)e->name, (char *)b->name, sizeof(e->name)); + e->offset = offset; + e->length = buffer_size(b); + + return (offset + buffer_size(b)); +} + +static void init_manifest_header(struct manifest_header *hdr, size_t size) +{ + memset(hdr, 0, sizeof(*hdr)); + + hdr->header_type = 0x4; + assert((MANIFEST_HDR_SIZE % DWORD_SIZE) == 0); + hdr->header_length = MANIFEST_HDR_SIZE / DWORD_SIZE; + hdr->header_version = 0x10000; + hdr->vendor = 0x8086; + + struct tm *local_time; + time_t curr_time; + char buffer[11]; + + curr_time = time(NULL); + local_time = localtime(&curr_time); + strftime(buffer, sizeof(buffer), "0x%Y%m%d", local_time); + hdr->date = strtoul(buffer, NULL, 16); + + assert((size % DWORD_SIZE) == 0); + hdr->size = size / DWORD_SIZE; + hdr->id = MANIFEST_ID_MAGIC; +} + +static void init_signed_pkg_info_ext(struct signed_pkg_info_ext *ext, + size_t count, const char *name) +{ + memset(ext, 0, sizeof(*ext)); + + ext->ext_type = SIGNED_PKG_INFO_EXT_TYPE; + ext->ext_length = SIGNED_PKG_INFO_EXT_SIZE + count * MODULE_SIZE; + memcpy(ext->name, name, sizeof(ext->name)); +} + +static void subpart_dir_fixup_write_buffer(struct buffer *buf) +{ + struct subpart_dir *s = buffer_get(buf); + struct subpart_dir_header *h = &s->h; + struct subpart_dir_entry *e = &s->e[0]; + + size_t count = h->num_entries; + size_t offset = 0; + + offset = fix_member(&h->marker, offset, sizeof(h->marker)); + offset = fix_member(&h->num_entries, offset, sizeof(h->num_entries)); + offset = fix_member(&h->header_version, offset, + sizeof(h->header_version)); + offset = fix_member(&h->entry_version, offset, + sizeof(h->entry_version)); + offset = fix_member(&h->header_length, offset, + sizeof(h->header_length)); + offset = fix_member(&h->checksum, offset, sizeof(h->checksum)); + offset += sizeof(h->name); + + uint32_t i; + + for (i = 0; i < count; i++) { + offset += sizeof(e[i].name); + offset = fix_member(&e[i].offset, offset, sizeof(e[i].offset)); + offset = fix_member(&e[i].length, offset, sizeof(e[i].length)); + offset = fix_member(&e[i].rsvd, offset, sizeof(e[i].rsvd)); + } +} + +static void create_subpart(struct buffer *dst, struct buffer *info[], + size_t count, const char *name) +{ + struct buffer subpart_dir_buff; + size_t size = SUBPART_DIR_HEADER_SIZE + count * SUBPART_DIR_ENTRY_SIZE; + + alloc_buffer(&subpart_dir_buff, size, "subpart-dir"); + + struct subpart_dir_header *h = buffer_get(&subpart_dir_buff); + struct subpart_dir_entry *e = (struct subpart_dir_entry *)(h + 1); + + init_subpart_dir_header(h, count, name); + + size_t curr_offset = size; + size_t i; + + for (i = 0; i < count; i++) { + curr_offset = init_subpart_dir_entry(&e[i], info[i], + curr_offset); + } + + alloc_buffer(dst, curr_offset, name); + uint8_t *data = buffer_get(dst); + + for (i = 0; i < count; i++) { + memcpy(data + e[i].offset, buffer_get(info[i]), + buffer_size(info[i])); + } + + h->checksum = calc_checksum(buffer_get(&subpart_dir_buff)); + + struct subpart_dir *dir = buffer_get(&subpart_dir_buff); + + print_subpart_dir(dir); + + subpart_dir_fixup_write_buffer(&subpart_dir_buff); + memcpy(data, dir, buffer_size(&subpart_dir_buff)); + + buffer_delete(&subpart_dir_buff); +} + +static enum ifwi_ret ibbp_dir_add(int type) +{ + struct buffer manifest; + struct signed_pkg_info_ext *ext; + struct buffer ibbl; + struct buffer ibb; + +#define DUMMY_IBB_SIZE (4 * KiB) + + assert(type == IBB_TYPE); + + /* + * Entry # 1 - IBBP.man + * Contains manifest header and signed pkg info extension. + */ + size_t size = MANIFEST_HDR_SIZE + SIGNED_PKG_INFO_EXT_SIZE; + + alloc_buffer(&manifest, size, "IBBP.man"); + + struct manifest_header *man_hdr = buffer_get(&manifest); + + init_manifest_header(man_hdr, size); + + ext = (struct signed_pkg_info_ext *)(man_hdr + 1); + + init_signed_pkg_info_ext(ext, 0, subparts[type].name); + + /* Entry # 2 - IBBL */ + if (buffer_from_file(&ibbl, param.file_name)) + return COMMAND_ERR; + + /* Entry # 3 - IBB */ + alloc_buffer(&ibb, DUMMY_IBB_SIZE, "IBB"); + memset(buffer_get(&ibb), 0xFF, DUMMY_IBB_SIZE); + + /* Create subpartition */ + struct buffer *info[] = { + &manifest, &ibbl, &ibb, + }; + create_subpart(&ifwi_image.subpart_buf[type], &info[0], + ARRAY_SIZE(info), subparts[type].name); + + return REPACK_REQUIRED; +} + +static enum ifwi_ret ifwi_raw_add(int type) +{ + if (buffer_from_file(&ifwi_image.subpart_buf[type], param.file_name)) + return COMMAND_ERR; + + printf("Sub-partition %s(%d) added from file %s.\n", param.subpart_name, + type, param.file_name); + return REPACK_REQUIRED; +} + +static enum ifwi_ret ifwi_dir_add(int type) +{ + if (!(subparts[type].attr & CONTAINS_DIR) || + !subparts[type].dir_ops.dir_add) { + ERROR("Sub-Partition %s(%d) does not support dir ops.\n", + subparts[type].name, type); + return COMMAND_ERR; + } + + if (!param.dentry_name) { + ERROR("%s: -e option required\n", __func__); + return COMMAND_ERR; + } + + enum ifwi_ret ret = subparts[type].dir_ops.dir_add(type); + + if (ret != COMMAND_ERR) + printf("Sub-partition %s(%d) entry %s added from file %s.\n", + param.subpart_name, type, param.dentry_name, + param.file_name); + else + ERROR("Sub-partition dir operation failed.\n"); + + return ret; +} + +static enum ifwi_ret ifwi_add(void) +{ + if (!param.file_name) { + ERROR("%s: -f option required\n", __func__); + return COMMAND_ERR; + } + + if (!param.subpart_name) { + ERROR("%s: -n option required\n", __func__); + return COMMAND_ERR; + } + + int type = find_type_by_name(param.subpart_name); + + if (type == -1) + return COMMAND_ERR; + + const struct subpart_info *curr_subpart = &subparts[type]; + + if (curr_subpart->attr & AUTO_GENERATED) { + ERROR("Cannot add auto-generated sub-partitions.\n"); + return COMMAND_ERR; + } + + if (buffer_size(&ifwi_image.subpart_buf[type])) { + ERROR("Image already contains sub-partition %s(%d).\n", + param.subpart_name, type); + return COMMAND_ERR; + } + + if (param.dir_ops) + return ifwi_dir_add(type); + + return ifwi_raw_add(type); +} + +static enum ifwi_ret ifwi_delete(void) +{ + if (!param.subpart_name) { + ERROR("%s: -n option required\n", __func__); + return COMMAND_ERR; + } + + int type = find_type_by_name(param.subpart_name); + + if (type == -1) + return COMMAND_ERR; + + const struct subpart_info *curr_subpart = &subparts[type]; + + if (curr_subpart->attr & AUTO_GENERATED) { + ERROR("Cannot delete auto-generated sub-partitions.\n"); + return COMMAND_ERR; + } + + if (buffer_size(&ifwi_image.subpart_buf[type]) == 0) { + printf("Image does not contain sub-partition %s(%d).\n", + param.subpart_name, type); + return NO_ACTION_REQUIRED; + } + + buffer_delete(&ifwi_image.subpart_buf[type]); + printf("Sub-Partition %s(%d) deleted.\n", subparts[type].name, type); + return REPACK_REQUIRED; +} + +static enum ifwi_ret ifwi_dir_extract(int type) +{ + if (!(subparts[type].attr & CONTAINS_DIR)) { + ERROR("Sub-Partition %s(%d) does not support dir ops.\n", + subparts[type].name, type); + return COMMAND_ERR; + } + + if (!param.dentry_name) { + ERROR("%s: -e option required.\n", __func__); + return COMMAND_ERR; + } + + struct buffer subpart_dir_buff; + + parse_subpart_dir(&subpart_dir_buff, &ifwi_image.subpart_buf[type], + subparts[type].name); + + uint32_t i; + struct subpart_dir *s = buffer_get(&subpart_dir_buff); + + for (i = 0; i < s->h.num_entries; i++) { + if (!strncmp((char *)s->e[i].name, param.dentry_name, + sizeof(s->e[i].name))) + break; + } + + if (i == s->h.num_entries) { + ERROR("Entry %s not found in subpartition for %s.\n", + param.dentry_name, param.subpart_name); + exit(-1); + } + + struct buffer dst; + + DEBUG("Splicing buffer at 0x%x size 0x%x\n", s->e[i].offset, + s->e[i].length); + buffer_splice(&dst, &ifwi_image.subpart_buf[type], s->e[i].offset, + s->e[i].length); + + if (buffer_write_file(&dst, param.file_name)) + return COMMAND_ERR; + + printf("Sub-Partition %s(%d), entry(%s) stored in %s.\n", + param.subpart_name, type, param.dentry_name, param.file_name); + + return NO_ACTION_REQUIRED; +} + +static enum ifwi_ret ifwi_raw_extract(int type) +{ + if (buffer_write_file(&ifwi_image.subpart_buf[type], param.file_name)) + return COMMAND_ERR; + + printf("Sub-Partition %s(%d) stored in %s.\n", param.subpart_name, type, + param.file_name); + + return NO_ACTION_REQUIRED; +} + +static enum ifwi_ret ifwi_extract(void) +{ + if (!param.file_name) { + ERROR("%s: -f option required\n", __func__); + return COMMAND_ERR; + } + + if (!param.subpart_name) { + ERROR("%s: -n option required\n", __func__); + return COMMAND_ERR; + } + + int type = find_type_by_name(param.subpart_name); + + if (type == -1) + return COMMAND_ERR; + + if (type == S_BPDT_TYPE) { + INFO("Tool does not support raw extract for %s\n", + param.subpart_name); + return NO_ACTION_REQUIRED; + } + + if (buffer_size(&ifwi_image.subpart_buf[type]) == 0) { + ERROR("Image does not contain sub-partition %s(%d).\n", + param.subpart_name, type); + return COMMAND_ERR; + } + + INFO("Extracting sub-partition %s(%d).\n", param.subpart_name, type); + if (param.dir_ops) + return ifwi_dir_extract(type); + + return ifwi_raw_extract(type); +} + +static enum ifwi_ret ifwi_print(void) +{ + verbose += 2; + + struct bpdt *b = buffer_get(&ifwi_image.bpdt); + + bpdt_print_header(&b->h, "BPDT"); + bpdt_print_entries(&b->e[0], b->h.descriptor_count, "BPDT"); + + b = buffer_get(&ifwi_image.subpart_buf[S_BPDT_TYPE]); + bpdt_print_header(&b->h, "S-BPDT"); + bpdt_print_entries(&b->e[0], b->h.descriptor_count, "S-BPDT"); + + if (param.dir_ops == 0) { + verbose -= 2; + return NO_ACTION_REQUIRED; + } + + int i; + struct buffer subpart_dir_buf; + + for (i = 0; i < MAX_SUBPARTS ; i++) { + if (!(subparts[i].attr & CONTAINS_DIR) || + (buffer_size(&ifwi_image.subpart_buf[i]) == 0)) + continue; + + parse_subpart_dir(&subpart_dir_buf, &ifwi_image.subpart_buf[i], + subparts[i].name); + buffer_delete(&subpart_dir_buf); + } + + verbose -= 2; + + return NO_ACTION_REQUIRED; +} + +static enum ifwi_ret ifwi_raw_replace(int type) +{ + buffer_delete(&ifwi_image.subpart_buf[type]); + return ifwi_raw_add(type); +} + +static enum ifwi_ret ifwi_dir_replace(int type) +{ + if (!(subparts[type].attr & CONTAINS_DIR)) { + ERROR("Sub-Partition %s(%d) does not support dir ops.\n", + subparts[type].name, type); + return COMMAND_ERR; + } + + if (!param.dentry_name) { + ERROR("%s: -e option required.\n", __func__); + return COMMAND_ERR; + } + + struct buffer subpart_dir_buf; + + parse_subpart_dir(&subpart_dir_buf, &ifwi_image.subpart_buf[type], + subparts[type].name); + + uint32_t i; + struct subpart_dir *s = buffer_get(&subpart_dir_buf); + + for (i = 0; i < s->h.num_entries; i++) { + if (!strcmp((char *)s->e[i].name, param.dentry_name)) + break; + } + + if (i == s->h.num_entries) { + ERROR("Entry %s not found in subpartition for %s.\n", + param.dentry_name, param.subpart_name); + exit(-1); + } + + struct buffer b; + + if (buffer_from_file(&b, param.file_name)) { + ERROR("Failed to read %s\n", param.file_name); + exit(-1); + } + + struct buffer dst; + size_t dst_size = buffer_size(&ifwi_image.subpart_buf[type]) + + buffer_size(&b) - s->e[i].length; + size_t subpart_start = s->e[i].offset; + size_t subpart_end = s->e[i].offset + s->e[i].length; + + alloc_buffer(&dst, dst_size, ifwi_image.subpart_buf[type].name); + + uint8_t *src_data = buffer_get(&ifwi_image.subpart_buf[type]); + uint8_t *dst_data = buffer_get(&dst); + size_t curr_offset = 0; + + /* Copy data before the sub-partition entry */ + memcpy(dst_data + curr_offset, src_data, subpart_start); + curr_offset += subpart_start; + + /* Copy sub-partition entry */ + memcpy(dst_data + curr_offset, buffer_get(&b), buffer_size(&b)); + curr_offset += buffer_size(&b); + + /* Copy remaining data */ + memcpy(dst_data + curr_offset, src_data + subpart_end, + buffer_size(&ifwi_image.subpart_buf[type]) - subpart_end); + + /* Update sub-partition buffer */ + int offset = s->e[i].offset; + + buffer_delete(&ifwi_image.subpart_buf[type]); + ifwi_image.subpart_buf[type] = dst; + + /* Update length of entry in the subpartition */ + s->e[i].length = buffer_size(&b); + buffer_delete(&b); + + /* Adjust offsets of affected entries in subpartition */ + offset = s->e[i].offset - offset; + for (; i < s->h.num_entries; i++) + s->e[i].offset += offset; + + /* Re-calculate checksum */ + s->h.checksum = calc_checksum(s); + + /* Convert members to litte-endian */ + subpart_dir_fixup_write_buffer(&subpart_dir_buf); + + memcpy(dst_data, buffer_get(&subpart_dir_buf), + buffer_size(&subpart_dir_buf)); + + buffer_delete(&subpart_dir_buf); + + printf("Sub-partition %s(%d) entry %s replaced from file %s.\n", + param.subpart_name, type, param.dentry_name, param.file_name); + + return REPACK_REQUIRED; +} + +static enum ifwi_ret ifwi_replace(void) +{ + if (!param.file_name) { + ERROR("%s: -f option required\n", __func__); + return COMMAND_ERR; + } + + if (!param.subpart_name) { + ERROR("%s: -n option required\n", __func__); + return COMMAND_ERR; + } + + int type = find_type_by_name(param.subpart_name); + + if (type == -1) + return COMMAND_ERR; + + const struct subpart_info *curr_subpart = &subparts[type]; + + if (curr_subpart->attr & AUTO_GENERATED) { + ERROR("Cannot replace auto-generated sub-partitions.\n"); + return COMMAND_ERR; + } + + if (buffer_size(&ifwi_image.subpart_buf[type]) == 0) { + ERROR("Image does not contain sub-partition %s(%d).\n", + param.subpart_name, type); + return COMMAND_ERR; + } + + if (param.dir_ops) + return ifwi_dir_replace(type); + + return ifwi_raw_replace(type); +} + +static enum ifwi_ret ifwi_create(void) +{ + /* + * Create peels off any non-IFWI content present in the input buffer and + * creates output file with only the IFWI present. + */ + + if (!param.file_name) { + ERROR("%s: -f option required\n", __func__); + return COMMAND_ERR; + } + + /* Peel off any non-IFWI prefix */ + buffer_seek(&ifwi_image.input_buff, + ifwi_image.input_ifwi_start_offset); + /* Peel off any non-IFWI suffix */ + buffer_set_size(&ifwi_image.input_buff, + ifwi_image.input_ifwi_end_offset - + ifwi_image.input_ifwi_start_offset); + + /* + * Adjust start and end offset of IFWI now that non-IFWI prefix is gone. + */ + ifwi_image.input_ifwi_end_offset -= ifwi_image.input_ifwi_start_offset; + ifwi_image.input_ifwi_start_offset = 0; + + param.image_name = param.file_name; + + return REPACK_REQUIRED; +} + +struct command { + const char *name; + const char *optstring; + enum ifwi_ret (*function)(void); +}; + +static const struct command commands[] = { + {"add", "f:n:e:dvh?", ifwi_add}, + {"create", "f:vh?", ifwi_create}, + {"delete", "f:n:vh?", ifwi_delete}, + {"extract", "f:n:e:dvh?", ifwi_extract}, + {"print", "dh?", ifwi_print}, + {"replace", "f:n:e:dvh?", ifwi_replace}, +}; + +static struct option long_options[] = { + {"subpart_dentry", required_argument, 0, 'e'}, + {"file", required_argument, 0, 'f'}, + {"help", required_argument, 0, 'h'}, + {"name", required_argument, 0, 'n'}, + {"dir_ops", no_argument, 0, 'd'}, + {"verbose", no_argument, 0, 'v'}, + {NULL, 0, 0, 0 } +}; + +static void usage(const char *name) +{ + printf("ifwitool: Utility for IFWI manipulation\n\n" + "USAGE:\n" + " %s [-h]\n" + " %s FILE COMMAND [PARAMETERS]\n\n" + "COMMANDs:\n" + " add -f FILE -n NAME [-d -e ENTRY]\n" + " create -f FILE\n" + " delete -n NAME\n" + " extract -f FILE -n NAME [-d -e ENTRY]\n" + " print [-d]\n" + " replace -f FILE -n NAME [-d -e ENTRY]\n" + "OPTIONs:\n" + " -f FILE : File to read/write/create/extract\n" + " -d : Perform directory operation\n" + " -e ENTRY: Name of directory entry to operate on\n" + " -v : Verbose level\n" + " -h : Help message\n" + " -n NAME : Name of sub-partition to operate on\n", + name, name + ); + + printf("\nNAME should be one of:\n"); + int i; + + for (i = 0; i < MAX_SUBPARTS; i++) + printf("%s(%s)\n", subparts[i].name, subparts[i].readable_name); + printf("\n"); +} + +int main(int argc, char **argv) +{ + if (argc < 3) { + usage(argv[0]); + return 1; + } + + param.image_name = argv[1]; + char *cmd = argv[2]; + + optind += 2; + + uint32_t i; + + for (i = 0; i < ARRAY_SIZE(commands); i++) { + if (strcmp(cmd, commands[i].name) != 0) + continue; + + int c; + + while (1) { + int option_index; + + c = getopt_long(argc, argv, commands[i].optstring, + long_options, &option_index); + + if (c == -1) + break; + + /* Filter out illegal long options */ + if (!strchr(commands[i].optstring, c)) { + ERROR("%s: invalid option -- '%c'\n", argv[0], + c); + c = '?'; + } + + switch (c) { + case 'n': + param.subpart_name = optarg; + break; + case 'f': + param.file_name = optarg; + break; + case 'd': + param.dir_ops = 1; + break; + case 'e': + param.dentry_name = optarg; + break; + case 'v': + verbose++; + break; + case 'h': + case '?': + usage(argv[0]); + return 1; + default: + break; + } + } + + if (ifwi_parse()) { + ERROR("%s: ifwi parsing failed\n", argv[0]); + return 1; + } + + enum ifwi_ret ret = commands[i].function(); + + if (ret == COMMAND_ERR) { + ERROR("%s: failed execution\n", argv[0]); + return 1; + } + + if (ret == REPACK_REQUIRED) + ifwi_repack(); + + return 0; + } + + ERROR("%s: invalid command\n", argv[0]); + return 1; +} From patchwork Mon Jul 8 19:18:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129222 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Bn0lYU7A"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFfr6Tw4z9sDB for ; Tue, 9 Jul 2019 05:21:08 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 57D4CC21DD9; Mon, 8 Jul 2019 19:21:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=RCVD_IN_MSPIKE_H2, T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 0777DC21C51; Mon, 8 Jul 2019 19:21:04 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 9F640C21C4A; Mon, 8 Jul 2019 19:21:02 +0000 (UTC) Received: from mail-io1-f68.google.com (mail-io1-f68.google.com [209.85.166.68]) by lists.denx.de (Postfix) with ESMTPS id 1BB5BC21BE5 for ; Mon, 8 Jul 2019 19:21:02 +0000 (UTC) Received: by mail-io1-f68.google.com with SMTP id u19so37798488ior.9 for ; Mon, 08 Jul 2019 12:21:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=64cJsR3lvwB6FMFI66yDV0/b/TpejNatThLhX1P7tvE=; b=Bn0lYU7A7hrT8UDweJ9nkLVPapsRudeiy1/rkaeXhSxMK3GDsWcrqAHSLzErnsHUXG b2Hlyi4XbI3gYbm7G+piBG89y9cAI31IsfjyiuTmq+XY6pCXL+tnyQVxMBTNTpQyzx6H nMJs1QkSkqDQugjVJKQPrgQJOb8GXOrJFfoW0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=64cJsR3lvwB6FMFI66yDV0/b/TpejNatThLhX1P7tvE=; b=PqFjnOu9W6gdO6sj82lwBCt2PflTEF2QMCybbvdLu2A+G3II6UJUJscnk1IzzEfjJz L6hZFTnNloe/rcENw9PSIT7I4ixn5AxJjpsuzHRrA2FbruE5hfoCPPFQvT1vwTWFX0fi twtW9Gwc58kh9XWHefR1buaQNGXS51sce3U9w8PcenxalAl7sQT4G/6WGxOzov0x0O/z D7R+yZkAFTdV5LInepGeLbJih5NKnfTHyuoha0jVhKxXWNNTsj9DmrK9LN6yVeZAqiMs T5IKP26ThrDpyr6TrRSWOv9BsqGOMvboFQraMXKXGOKgxEwB92oZ1P6qkQ2Z2RWHJ1J6 26ww== X-Gm-Message-State: APjAAAWevenSZat+hlOz1U2icexm2fx1NU2YRp4OwHWNuyT40Tfm+L1T R93Aev69BV0/pcuFQKQyKGJ0/lTGYd0= X-Google-Smtp-Source: APXvYqx3uWXGr6MrjiORwS3f2Z/iIHNOI33rgkehS8O1R4d2IyLEmQKCl6nzjzmSb6eooxp6o8QPew== X-Received: by 2002:a6b:7b01:: with SMTP id l1mr5533182iop.60.1562613660804; Mon, 08 Jul 2019 12:21:00 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.00 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:00 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:21 -0600 Message-Id: <20190708191856.138863-3-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Cc: Stefan Reinauer Subject: [U-Boot] [PATCH v2 02/37] cbfs: Add an enum and comment for the magic number X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" This field is not commented in the original file. Add a comment. Signed-off-by: Simon Glass --- Changes in v2: None include/cbfs.h | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/include/cbfs.h b/include/cbfs.h index bd1bf75bbfc..f2ede25f517 100644 --- a/include/cbfs.h +++ b/include/cbfs.h @@ -40,6 +40,17 @@ enum cbfs_filetype { CBFS_TYPE_CMOS_LAYOUT = 0x01aa }; +enum { + CBFS_HEADER_MAGIC = 0x4f524243, +}; + +/** + * struct cbfs_header - header at the start of a CBFS region + * + * All fields use big-endian format. + * + * @magic: Magic number (CBFS_HEADER_MAGIC) + */ struct cbfs_header { u32 magic; u32 version; From patchwork Mon Jul 8 19:18:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129226 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="i4hiIxEJ"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFgj5fnsz9sNH for ; Tue, 9 Jul 2019 05:21:53 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 82EBDC21EBF; Mon, 8 Jul 2019 19:21:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=RCVD_IN_MSPIKE_H2, T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 86F85C21D4A; Mon, 8 Jul 2019 19:21:04 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 2D000C21C4A; Mon, 8 Jul 2019 19:21:03 +0000 (UTC) Received: from mail-io1-f68.google.com (mail-io1-f68.google.com [209.85.166.68]) by lists.denx.de (Postfix) with ESMTPS id AB43EC21BE5 for ; Mon, 8 Jul 2019 19:21:02 +0000 (UTC) Received: by mail-io1-f68.google.com with SMTP id z3so22878044iog.0 for ; Mon, 08 Jul 2019 12:21:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JsskJFF5p1z+fp4pZLAuq4+Y0akOyedsvsKrGB6kOns=; b=i4hiIxEJD74/uRCjv0+d2tmCM9pF0zNV1fSC1RejYkROACIv1UNH7ZdtsqoupOioyf g+837h+6y29pOv/TjpDg+2GA3bQcScnpcaMvd9KVx9DkKzk2xXPL4vr/YXcog+DOiUC1 Su5mEdfm8tkmM66VnjQZwj/j8h0JShWGOzBpU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JsskJFF5p1z+fp4pZLAuq4+Y0akOyedsvsKrGB6kOns=; b=uJzCv9CxVIxYLQXlDmpGBowmWmj31uGrfvTIEzLgHFuWxEYxaynapohc0WZxXFF/iO TkgPNTCAmKa3tzlcOuivlEfYzmP9QTU6elP4nVnQgbaE6joIT6dXq3IWDrKTH+4XGaiD dhkJa+dayqRJD26F6b26ujzL5CGGwbGao9FOWX2fQvOVMOstPh/DeqOlpiYq/au3/plP wDrDMtOoZqpH76HrWw7WrNTan9E7ca/OoXpfGYIhWuFcNSxLco1Wn730zY/X9/riRess HDPNlqgb0bJT3MM5LQhVBNJL/zuEEcEseU+y3eOSjGiV4JuZFTIpQ8qP5Jof8nUsGEtI kqEQ== X-Gm-Message-State: APjAAAWoJyeJ4jxHebpzEWrSHA5WuHyjwI6pMavifMzVJE0td74RR4JK PDPRYoWbdk+NQn1oDxfxBwaNylsmcKs= X-Google-Smtp-Source: APXvYqzX9tC3Rn1jfPvLK8qBTg+uWgF/5JoTb9TPx8+9nZcmMZQjibZMDISNZ1qsUe/X2DxvBdWiEw== X-Received: by 2002:a5e:9319:: with SMTP id k25mr1525262iom.137.1562613661467; Mon, 08 Jul 2019 12:21:01 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.00 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:01 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:22 -0600 Message-Id: <20190708191856.138863-4-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Cc: Stefan Reinauer Subject: [U-Boot] [PATCH v2 03/37] cbfs: Rename checksum to attributes_offset X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" It seems that this field has been renamed in later version of coreboot. Update it. Signed-off-by: Simon Glass --- Changes in v2: None fs/cbfs/cbfs.c | 4 ++-- include/cbfs.h | 5 +++-- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/fs/cbfs/cbfs.c b/fs/cbfs/cbfs.c index 7b2513cb24b..af4d3c5e564 100644 --- a/fs/cbfs/cbfs.c +++ b/fs/cbfs/cbfs.c @@ -55,7 +55,7 @@ static void swap_file_header(struct cbfs_fileheader *dest, memcpy(&dest->magic, &src->magic, sizeof(dest->magic)); dest->len = be32_to_cpu(src->len); dest->type = be32_to_cpu(src->type); - dest->checksum = be32_to_cpu(src->checksum); + dest->attributes_offset = be32_to_cpu(src->attributes_offset); dest->offset = be32_to_cpu(src->offset); } @@ -108,7 +108,7 @@ static int file_cbfs_next_file(u8 *start, u32 size, u32 align, newNode->name = (char *)fileHeader + sizeof(struct cbfs_fileheader); newNode->name_length = name_len; - newNode->checksum = header.checksum; + newNode->attributes_offset = header.attributes_offset; step = header.len; if (step % align) diff --git a/include/cbfs.h b/include/cbfs.h index f2ede25f517..b8d1dabbf63 100644 --- a/include/cbfs.h +++ b/include/cbfs.h @@ -65,7 +65,8 @@ struct cbfs_fileheader { u8 magic[8]; u32 len; u32 type; - u32 checksum; + /* offset to struct cbfs_file_attribute or 0 */ + u32 attributes_offset; u32 offset; } __packed; @@ -76,7 +77,7 @@ struct cbfs_cachenode { u32 data_length; char *name; u32 name_length; - u32 checksum; + u32 attributes_offset; } __packed; extern enum cbfs_result file_cbfs_result; From patchwork Mon Jul 8 19:18:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129229 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="TTqHHoiq"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFjH0Z9Pz9sDB for ; Tue, 9 Jul 2019 05:23:15 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id A91B0C21CB1; Mon, 8 Jul 2019 19:23:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id B71F9C21CB1; Mon, 8 Jul 2019 19:21:07 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 2C585C21C4A; Mon, 8 Jul 2019 19:21:04 +0000 (UTC) Received: from mail-io1-f54.google.com (mail-io1-f54.google.com [209.85.166.54]) by lists.denx.de (Postfix) with ESMTPS id 9A4CAC21BE5 for ; Mon, 8 Jul 2019 19:21:03 +0000 (UTC) Received: by mail-io1-f54.google.com with SMTP id k20so37784119ios.10 for ; Mon, 08 Jul 2019 12:21:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CSwsbEgMEl5r5ivNLDazPY+NU57ZKLUMxMVM/bkfwUo=; b=TTqHHoiq01vYZwjZN8PVzNV6PX6k0CT8ETX6tLcbf7WB8bU2tVavtzhgN4vf5ciFwd CZwWsM3VLll4onKDBjFJqMRxZYP6fkWfz2cl7no16PvdGqyyEbMWuCWFywl3B5Hr4h1z XBB9q4mphM7GZ0Ly6f3UY5maSX1hELZPrO+PQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CSwsbEgMEl5r5ivNLDazPY+NU57ZKLUMxMVM/bkfwUo=; b=QvAtyq5nKqLif9Cwu8ItvtJhfq27PnGP97/07ISEw4pLGn2SKpDBOcL/06JDtAm/vn DNTKJlamoOVOze/osFRqmEzOJI8VMBFWuIndX8J3IBqrqJ95xIckW3fXOrK2EMC753+8 mYMTFE00sdZ5GnuwHQX1Pde0vA+Zsd291oNeHsktgMCyT9eFJS801JuzHICGFEoaFtky ppvVPTL9jLtUzGT5mq3Sv6+Lec+VZqFNVgFAkdppEMfRJU6YmiS1vie/QKdLUX7m1Ztx /revrdiO1FN8i6w4F8xNRTiQr8i6O4KTrZMjl4dKMvtdXh9SZP4ThMqkXWH6VLRExIoe jBow== X-Gm-Message-State: APjAAAVkOJjmrkEhcI4fZMag7iLvmKYRQ3/qb+fDuTSu/4MoJvegS/k9 XUW5FnJZ9DgSGy6k9n6V9J+NWrjdkSw= X-Google-Smtp-Source: APXvYqzs/cFWKRRi4rx0vh8XyvAvZ3ZVZqQW9CbPsZFiwVrZY5F4CcYCv6sNmxGP+nq2jkK0bouC+A== X-Received: by 2002:a02:5b05:: with SMTP id g5mr22380767jab.114.1562613662471; Mon, 08 Jul 2019 12:21:02 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.01 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:01 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:23 -0600 Message-Id: <20190708191856.138863-5-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 04/37] tools: Drop duplicate raise_on_error argument X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" If kwargs contains raise_on_error then this function generates an error due to a duplicate argument. Fix this. Signed-off-by: Simon Glass --- Changes in v2: None tools/patman/command.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/patman/command.py b/tools/patman/command.py index 14edcdaffd2..16299f3f5be 100644 --- a/tools/patman/command.py +++ b/tools/patman/command.py @@ -108,8 +108,8 @@ def RunPipe(pipe_list, infile=None, outfile=None, return result def Output(*cmd, **kwargs): - raise_on_error = kwargs.get('raise_on_error', True) - return RunPipe([cmd], capture=True, raise_on_error=raise_on_error).stdout + kwargs['raise_on_error'] = kwargs.get('raise_on_error', True) + return RunPipe([cmd], capture=True, **kwargs).stdout def OutputOneLine(*cmd, **kwargs): raise_on_error = kwargs.pop('raise_on_error', True) From patchwork Mon Jul 8 19:18:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129235 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="haPBs4W5"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFkX6xTwz9sDB for ; Tue, 9 Jul 2019 05:24:20 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 9ED4CC21E3B; Mon, 8 Jul 2019 19:23:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id B60A9C21DFB; Mon, 8 Jul 2019 19:21:14 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 9294BC21E08; Mon, 8 Jul 2019 19:21:07 +0000 (UTC) Received: from mail-io1-f44.google.com (mail-io1-f44.google.com [209.85.166.44]) by lists.denx.de (Postfix) with ESMTPS id 4D998C21C8B for ; Mon, 8 Jul 2019 19:21:04 +0000 (UTC) Received: by mail-io1-f44.google.com with SMTP id s7so37811911iob.11 for ; Mon, 08 Jul 2019 12:21:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=655WevlJXxzEk2s//Exs9RHiYaJiZiwG2MAP22WWtwY=; b=haPBs4W5ZiE8Izr4pQ+pVVYiIxQps5w/pdNg3FWYL9yQ8mZEDMKe1quV8qCFFKMzUS auOvRsfgrouUMkIH0CvxczNcWLib/3YcfcXlMIuzqbjLTnLyvwKjvBMMkLx3HYNviIG+ xv9v6ZqeoJd9Jv16jUlo614GNCbba23LRCxwI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=655WevlJXxzEk2s//Exs9RHiYaJiZiwG2MAP22WWtwY=; b=nisraSJxHHUDD7WaXQQm8K7Um5XF+0UamSH+wbefNlORFUVcc5FSinNUXlqAniROpS AG0M99+mREMbK4DrzajdXBd1rLFukjCfSMiZJYQ/7afLEA6YcYZwpo/S0t2hBp+P8+iQ t2xEmm4ORt1D+3XrhJe3dLq962HUgTA571GmSN9WPtOO4BCeISsgcRNfKH8mgL3zHJY6 ytrYhvlOG0/JIsKKnx2dTumOrHXiE389mp2Jx9lWHmDpJe4y+p/fe9r61UPBihM4ym6W ZySqXTDJ0fTosxqURg0k7T25HQdaAbYEoYa7nSpkRsLZCTNSqeUY7jyVVCIKyIlVEdho DLrA== X-Gm-Message-State: APjAAAVlTShkZJxzgOzY1SOOvOFu29Y2gCnE6/2FGq3J0WbcQsagtPEj /qkP3/DxmhZAuqdi2PaiVq+DRDOcM+A= X-Google-Smtp-Source: APXvYqwTSTUvGd+AgZ5Upv/9E7NWMiofMpjD+ATSrbgIFiNVbhn1m7w+tCoiStgLPnZt4Ywx9PkP+A== X-Received: by 2002:a02:5a89:: with SMTP id v131mr23826986jaa.130.1562613663221; Mon, 08 Jul 2019 12:21:03 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.02 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:02 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:24 -0600 Message-Id: <20190708191856.138863-6-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 05/37] binman: Fix comment in bsection.GetEntries() X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" This comment is out of date as it does not correctly describe the return value. Fix it. Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/bsection.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/binman/bsection.py b/tools/binman/bsection.py index 03dfa2f805c..a3026718794 100644 --- a/tools/binman/bsection.py +++ b/tools/binman/bsection.py @@ -399,10 +399,10 @@ class Section(object): raise ValueError("%s: No such property '%s'" % (msg, prop_name)) def GetEntries(self): - """Get the number of entries in a section + """Get the dict of entries in a section Returns: - Number of entries in a section + OrderedDict of entries in a section """ return self._entries From patchwork Mon Jul 8 19:18:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129234 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="ces9mPYL"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFkN2k2Pz9sDB for ; Tue, 9 Jul 2019 05:24:12 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id E111DC21E75; Mon, 8 Jul 2019 19:23:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 3C419C21DD3; Mon, 8 Jul 2019 19:21:12 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id E9858C21E02; Mon, 8 Jul 2019 19:21:08 +0000 (UTC) Received: from mail-io1-f45.google.com (mail-io1-f45.google.com [209.85.166.45]) by lists.denx.de (Postfix) with ESMTPS id EF5A5C21D8E for ; Mon, 8 Jul 2019 19:21:04 +0000 (UTC) Received: by mail-io1-f45.google.com with SMTP id z3so22878283iog.0 for ; Mon, 08 Jul 2019 12:21:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xcjDirazP+M5QNU7wsBpRlhG44T6bqzraC6c4QBMdg4=; b=ces9mPYLdZulVX8l/Tk1DXY8eMbhVWFnEc+uwDb0m9Na28KxFiLdO2LulqAmn3/2uQ 1nzg5v4z078dPHpK1TfeRw06a1ltuwiZjLZp+3S0H0RxwuFxLHljbb1NLNjPk3mI3yey XfNSy9jcfMf/tK6Q9BSTOavSEDsJaKlKUmQOw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xcjDirazP+M5QNU7wsBpRlhG44T6bqzraC6c4QBMdg4=; b=dy9ccgUWeXVtR7XtgAdTrmjQBNP7oUKX5pdshngAfpoWrGaxiKmPpIWa5nJgP8chRh kOEadCbTcn3VAi68IK7eLp1rUo87ka0Z5mHnpHodFgIawLU+MQma97gE0ox3Z9+1R4FH jHeTYcHXhIcR00gIlbYu01zuIJevdhX/CIj0OF6XAxx5cLfn7l/aHcw7nNbKr6pnc/Bv bgQvUhKn7GzfG5omje0Q7OoKN35RFTnw2BswqgYTdepl1SZ+9QALx+0RwNrlZbXPWMB1 ZSrHvix2j0vX9yYdt9Nyc09gMBckLRWiHTEvj1z6c4s9U4aXEtv/LBKl4ytDC4pnEDMU +uVg== X-Gm-Message-State: APjAAAVJFvYcSmk8yKy1pmdZ0r0DHsJJJI+0NljlQoE87wSEGgIere9x K7AzuCvPYnS+vxG3KcYlDNbtPR4O+DU= X-Google-Smtp-Source: APXvYqyKF2iCfCP6XAQ+MQ2Wnpm9jQsm/sALA0E8k9ERUuerFc1e6zVZagrsDwnBBpDJNnYkMmlXJA== X-Received: by 2002:a6b:7b07:: with SMTP id l7mr16479864iop.225.1562613663842; Mon, 08 Jul 2019 12:21:03 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.03 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:03 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:25 -0600 Message-Id: <20190708191856.138863-7-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 06/37] binman: Correct two typos in function names in ftest X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" Two functions have incorrect names. Fix them. Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/ftest.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index cc57ef3e04a..46f669e73b4 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -1758,7 +1758,7 @@ class TestFunctional(unittest.TestCase): TestFunctional._MakeInputFile('-boot', fd.read()) data = self._DoReadFile('096_elf.dts') - def testElfStripg(self): + def testElfStrip(self): """Basic test of ELF entries""" self._SetupSplElf() with open(self.TestFile('bss_data'), 'rb') as fd: @@ -1784,7 +1784,7 @@ class TestFunctional(unittest.TestCase): 00000003 00000004 u-boot-align ''', map_data) - def testPacRefCode(self): + def testPackRefCode(self): """Test that an image with an Intel Reference code binary works""" data = self._DoReadFile('100_intel_refcode.dts') self.assertEqual(REFCODE_DATA, data[:len(REFCODE_DATA)]) From patchwork Mon Jul 8 19:18:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129246 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="TXL+1g/1"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFm26QLGz9sDB for ; Tue, 9 Jul 2019 05:25:38 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 7B3DEC21E0B; Mon, 8 Jul 2019 19:24:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id BEE65C21DCA; Mon, 8 Jul 2019 19:21:17 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id C05A2C21C57; Mon, 8 Jul 2019 19:21:09 +0000 (UTC) Received: from mail-io1-f45.google.com (mail-io1-f45.google.com [209.85.166.45]) by lists.denx.de (Postfix) with ESMTPS id 95922C21C93 for ; Mon, 8 Jul 2019 19:21:05 +0000 (UTC) Received: by mail-io1-f45.google.com with SMTP id i10so37766287iol.13 for ; Mon, 08 Jul 2019 12:21:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=htEo0UTkduX+v1LoaSD6jFawDVNnirnzLTuEBT9a20o=; b=TXL+1g/1XUCrrww1fJ/2Lenn2fMvxoimykHcggU1Qvkluy8RyZdR8eiNxY7ZTBp157 4DuNMgN3tPZhNtslza+KTcf4aUZz5OIakwYye2r1UW1Nrn7w4OdU0Cv0a1bLthLqPHG0 DRGUTInvJmqbrT/1z4MXPuKp2gHJliK2MiSNM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=htEo0UTkduX+v1LoaSD6jFawDVNnirnzLTuEBT9a20o=; b=aJiYLtcVh7fKjxv7FjVR/BnVC/tBg5Ifoo9odcb72d1iHnt3PKlXeUPDefZlQ/Yen+ 16oM9G995EcHhLH0A4HwanUZZYY5D3c36Y6pTTwiAgLxJE+3s0KA6MHPmW0b3aiRVfsX XPal+46WxvQAELEIaLTeYlZE3pYN9N7k8B20aki+LGVdACb+MVF9jghlwTRxJfhNdO9d ZBBSMUibwUWeb2B+Z3z1lplvA3uw26VOOmklJQ62JQZMWMyAjrcG8WqrrSuTsWR+s6Uv CSGX4bhHnL7z4D7Kh+oWO6hi1U3T41FnSSZM4JAvSf/8jgk8syOBnYXrsduM0+bX/OdO 2G0g== X-Gm-Message-State: APjAAAV8iaz2JYgOeaXC3EBNbCMhjHcNTVa8JsShbMgGjWr1fYXsWMC0 TsQTJWol5Vuzx63Pp5rVqeNVofm7cgY= X-Google-Smtp-Source: APXvYqxSupk6kVAwR1aobtqvfuiRdzS9PO3Re0kWbXPIHqV8ghw7q2q/VUHzzQ7WGtUfmuct3Yg/Bg== X-Received: by 2002:a02:3093:: with SMTP id q141mr23429126jaq.128.1562613664498; Mon, 08 Jul 2019 12:21:04 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.03 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:04 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:26 -0600 Message-Id: <20190708191856.138863-8-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 07/37] binman: Add coverage tools info for Python 3 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" Test coverage with Python 3 requires a new package. Add details about this. Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/README | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/binman/README b/tools/binman/README index ac193f16cf7..decca47bbf3 100644 --- a/tools/binman/README +++ b/tools/binman/README @@ -699,7 +699,7 @@ implementations target 100% test coverage. Run 'binman -T' to check this. To enable Python test coverage on Debian-type distributions (e.g. Ubuntu): - $ sudo apt-get install python-coverage python-pytest + $ sudo apt-get install python-coverage python3-coverage python-pytest Concurrent tests From patchwork Mon Jul 8 19:18:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129282 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="HGsYczm8"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFtt44gbz9sDB for ; Tue, 9 Jul 2019 05:31:34 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 493F2C21DB5; Mon, 8 Jul 2019 19:25:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 6FA43C21E2C; Mon, 8 Jul 2019 19:21:21 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 38216C21DED; Mon, 8 Jul 2019 19:21:09 +0000 (UTC) Received: from mail-io1-f66.google.com (mail-io1-f66.google.com [209.85.166.66]) by lists.denx.de (Postfix) with ESMTPS id 410BFC21DA6 for ; Mon, 8 Jul 2019 19:21:06 +0000 (UTC) Received: by mail-io1-f66.google.com with SMTP id k20so37784435ios.10 for ; Mon, 08 Jul 2019 12:21:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jnfLe3c+FMyWtBqRG0sixjMkvhD/B9A1e2vp3GWvGc0=; b=HGsYczm8x/3a97YrrjaWOL7QmPc+3Bbsb78uDC14JEKVwL5V7pkwJVC/kFiIc5+yHb iG7Ie0k5OZffxlsNlFgyetgSsaTewiVsOoFhX57a0URrKUx+Fw0+LGKQzICcvgOb94e/ ywHN0H/M31efihU8Cs6AMmPWtXBwBOhUmqfCQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jnfLe3c+FMyWtBqRG0sixjMkvhD/B9A1e2vp3GWvGc0=; b=tS9KNlRVW3t7fk6hkGERGsvIcWzO1rCA5lp+57afYerL8gguiSo0yKpDxKcf2Hyt17 YoXwZeTyq+qk96Y7MMLn3cE+OBAiH+IpZF6Ubf2n3GKK98Zuu+rokPe33rE5GtXQHZoh /AQOK1a+EkDdTEPnvyoCXcjqN1/8Vt1JhIf9CoAiydEjOnD/pyEBumpYkz//0IH5Ihge jTVm1SGKK7BQcKTK8pZnlTPkhIlt2sNQftKldGncmmtUme06aBqixFhy3yhFKObj3k/g rOOnzlZl3LsJZYobKOMVUU1wsOp8yomwWCTS6s6AQAPtrnZhj5jCrEwODcrq78fh63fX yTJQ== X-Gm-Message-State: APjAAAW6WOrKx6yYaQVhn03UjcBd48OfXPMY+6RnfGdgr3PhjM7ZHg4t Ngw0bNson//GYwYNSU4fBT5zxYh1lPE= X-Google-Smtp-Source: APXvYqyJ8uUr3Np5lbyKRzLik15WttYBIKxwjai0WXym0rh72Vnc3ExagYf54Bui7CKtOrWVfOcyKA== X-Received: by 2002:a6b:7208:: with SMTP id n8mr20534821ioc.151.1562613665114; Mon, 08 Jul 2019 12:21:05 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.04 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:04 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:27 -0600 Message-Id: <20190708191856.138863-9-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 08/37] patman: Add a way to set the search path for tools X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" Sometimes tools can be located by looking in other locations. Add a way to direct the search. Signed-off-by: Simon Glass --- Changes in v2: None tools/patman/tools.py | 42 +++++++++++++++++++++++++++++++++++++----- 1 file changed, 37 insertions(+), 5 deletions(-) diff --git a/tools/patman/tools.py b/tools/patman/tools.py index 8e9f22afe8a..0b3049f91f4 100644 --- a/tools/patman/tools.py +++ b/tools/patman/tools.py @@ -24,6 +24,8 @@ chroot_path = None # Search paths to use for Filename(), used to find files search_paths = [] +tool_search_paths = [] + # Tools and the packages that contain them, on debian packages = { 'lz4': 'liblz4-tool', @@ -154,26 +156,56 @@ def Align(pos, align): def NotPowerOfTwo(num): return num and (num & (num - 1)) -def PathHasFile(fname): +def SetToolPaths(toolpaths): + """Set the path to search for tools + + Args: + toolpaths: List of paths to search for tools executed by Run() + """ + global tool_search_paths + + tool_search_paths = toolpaths + +def PathHasFile(path_spec, fname): """Check if a given filename is in the PATH Args: + path_spec: Value of PATH variable to check fname: Filename to check Returns: True if found, False if not """ - for dir in os.environ['PATH'].split(':'): + for dir in path_spec.split(':'): if os.path.exists(os.path.join(dir, fname)): return True return False def Run(name, *args, **kwargs): + """Run a tool with some arguments + + This runs a 'tool', which is a program used by binman to process files and + perhaps produce some output. Tools can be located on the PATH or in a + search path. + + Args: + name: Command name to run + args: Arguments to the tool + kwargs: Options to pass to command.run() + + Returns: + CommandResult object + """ try: - return command.Run(name, *args, cwd=outdir, capture=True, **kwargs) + env = None + if tool_search_paths: + env = dict(os.environ) + env['PATH'] = ':'.join(tool_search_paths) + ':' + env['PATH'] + return command.Run(name, *args, capture=True, + capture_stderr=True, env=env, **kwargs) except: - if not PathHasFile(name): - msg = "Plesae install tool '%s'" % name + if env and not PathHasFile(env['PATH'], name): + msg = "Please install tool '%s'" % name package = packages.get(name) if package: msg += " (e.g. from package '%s')" % package From patchwork Mon Jul 8 19:18:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129279 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="V2grdQEp"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFrk0THhz9sDB for ; Tue, 9 Jul 2019 05:29:41 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id A6106C21DEC; Mon, 8 Jul 2019 19:25:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=RCVD_IN_MSPIKE_H2, T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id BBEA9C21E13; Mon, 8 Jul 2019 19:21:20 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 8F901C21D74; Mon, 8 Jul 2019 19:21:10 +0000 (UTC) Received: from mail-io1-f68.google.com (mail-io1-f68.google.com [209.85.166.68]) by lists.denx.de (Postfix) with ESMTPS id D9017C21D9A for ; Mon, 8 Jul 2019 19:21:06 +0000 (UTC) Received: by mail-io1-f68.google.com with SMTP id u19so37799027ior.9 for ; Mon, 08 Jul 2019 12:21:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=l6T/z22h0Ghd14pPWpVHp4NvxI8qEKEzUQHsAjmgIbo=; b=V2grdQEpV4sKC7YO8WId86S/zuWmrhE2UnL3u7M+TQzd5dAb1gTMNCiD+otYEE+HJA f9T7q3aM9Vpj9ZhySwkzKW7TWRhs6+ZjE3lOnzx+zp4+twWW1Uu5aOp8Jqj83Ct5Y7LA kJx1gZh2gwQy5JgmoCF2CMhmDxo5Q5n56XOEY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=l6T/z22h0Ghd14pPWpVHp4NvxI8qEKEzUQHsAjmgIbo=; b=pPeph4IwbhUid0OLAvXRdMT7m8SYZJ9+boQHsjtxqJO/4d3VQTkz3ejU6QNptJv7Pn s3OV7Drvog8XYjKKKEYd2lA22KrZUDhpvgeQZF66b9Fbvk7eIQ1M1DJLwuOcG+Kk1ooi h+uNV/XBvn9sWj9K1yejwBndVZPSy+MN8fnK/V3vWfUKVEAwYh4m2OF5bAnDRcH3Gp+t s47SL9X4R9CAw3tm+2iI6Uug/fDO42Z0fjCh0Pz6ctt0FM0g7tsCsck6JdLUVlAdx3QJ ug9Jkm5vu1F4CWhtiEp0lcF9I+g1KzPdWc/B40LOUcUpzYbFgsyXF+q+qThafU895xy6 OM/Q== X-Gm-Message-State: APjAAAUKepZ4/f4MgNRb6KN0DIWW7LwTjYX/t9QS7pHCxyI1Pm8r4CWv 5GZOSXTh87CRhuSvu5bdEPGPTRa4IXs= X-Google-Smtp-Source: APXvYqyzaCiCbikoCIr49RKRaBebitdj+SRTm2+yh0YZCQDFAjf+6jk9boxcD/OMRRp8M6/Ne7XzLg== X-Received: by 2002:a02:ad15:: with SMTP id s21mr24068087jan.47.1562613665762; Mon, 08 Jul 2019 12:21:05 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.05 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:05 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:28 -0600 Message-Id: <20190708191856.138863-10-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 09/37] binman: Add a --toolpath option to set the tool search path X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" Sometimes tools used by binman may not be in the normal PATH search path, such as when the tool is built by the U-Boot build itself (e.g. mkimage). Provide a way to specify an additional search path for tools. The flag can be used multiple times. Update the help to describe this option. Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/README | 10 ++++++++++ tools/binman/cmdline.py | 2 ++ tools/binman/control.py | 1 + 3 files changed, 13 insertions(+) diff --git a/tools/binman/README b/tools/binman/README index decca47bbf3..28624fadb33 100644 --- a/tools/binman/README +++ b/tools/binman/README @@ -691,6 +691,16 @@ Not all properties can be provided this way. Only some entries support it, typically for filenames. +External tools +-------------- + +Binman can make use of external command-line tools to handle processing of +entry contents or to generate entry contents. These tools are executed using +the 'tools' module's Run() method. The tools generally must exist on the PATH, +but the --toolpath option can be used to specify additional search paths to +use. This option can be specified multiple times to add more than one path. + + Code coverage ------------- diff --git a/tools/binman/cmdline.py b/tools/binman/cmdline.py index 3886d52b3a0..ee19c5e33fe 100644 --- a/tools/binman/cmdline.py +++ b/tools/binman/cmdline.py @@ -52,6 +52,8 @@ def ParseArgs(argv): default=False, help='run tests') parser.add_option('-T', '--test-coverage', action='store_true', default=False, help='run tests and check for 100% coverage') + parser.add_option('--toolpath', type='string', action='append', + help='Add a path to the directories containing tools') parser.add_option('-u', '--update-fdt', action='store_true', default=False, help='Update the binman node with offset/size info') parser.add_option('-v', '--verbosity', default=1, diff --git a/tools/binman/control.py b/tools/binman/control.py index 20186ee1980..df78848e13d 100644 --- a/tools/binman/control.py +++ b/tools/binman/control.py @@ -112,6 +112,7 @@ def Binman(options, args): try: tools.SetInputDirs(options.indir) tools.PrepareOutputDir(options.outdir, options.preserve) + tools.SetToolPaths(options.toolpath) state.SetEntryArgs(options.entry_arg) # Get the device tree ready by compiling it and copying the compiled From patchwork Mon Jul 8 19:18:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129277 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="ENy/uKv7"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFqb4VBbz9sNx for ; Tue, 9 Jul 2019 05:28:43 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id E6A9DC21C93; Mon, 8 Jul 2019 19:25:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 17A2EC21E35; Mon, 8 Jul 2019 19:21:23 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 0A7DEC21D9A; Mon, 8 Jul 2019 19:21:12 +0000 (UTC) Received: from mail-io1-f46.google.com (mail-io1-f46.google.com [209.85.166.46]) by lists.denx.de (Postfix) with ESMTPS id 81FACC21DDC for ; Mon, 8 Jul 2019 19:21:07 +0000 (UTC) Received: by mail-io1-f46.google.com with SMTP id g20so16727752ioc.12 for ; Mon, 08 Jul 2019 12:21:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YmxX6KvI3+7fA8BBEPu3BXIykdtS+7zy65MkUxV9uiU=; b=ENy/uKv71whCdXSHHMnQGoq3VfY1c9UethoGjbsFXY1l6AH37LBr16g0jKjyBoza7/ tLC+LD3kw9FVqYyZvnHHppz0ucNyNUyZ37vw/82v2EjvpOdteW2xsi46YOht1aqCQXQi nLJebthVgGZd/vGhA26N4N+75MaWiaxc1jLtY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YmxX6KvI3+7fA8BBEPu3BXIykdtS+7zy65MkUxV9uiU=; b=ajyqD+f1Wn214uaJ8vjc9cFbOU4xyn1la0JXI9iqUAFymS9Rb5sMuCCsEAQOHilBJu AguGJnjP9DahrFtf0T5mSDd3uTisfL6pOMWZQcuDxesJWVbCBFCfPRh6b1MJaNQ4bSHP 5ERlEbMG7cSDp4JTrzAFtyuQLvXUC4dOaYYVksrTStrZY3jFzXWZg3arLgq73TT013E8 iQB1Q7poJE7SGFKIyR2MT7fJ0GjW4XH3tB0X4XW5VZoCUjhdmeN36Nwdz9DOtAqzYcIC y0/ubYC4QC2+qfGwLUMheUvUKDEHK4zkRo0E9AfVBBySlyd7kU4jNMgLi9bVWaUPJPB9 ICxg== X-Gm-Message-State: APjAAAVsPHFPcGEkS9ljjjJGujx5ggzjS8q4+U2eTxPzyBHUv63ehAoz HMUvHS65fDvegQNixGsx4Irtqr9RL8o= X-Google-Smtp-Source: APXvYqyMuP6eE7LlYJ1qGs0w/EtDfrgQAOAB5tpQloWEjkS1/sMMvqa3dlyDUS9HHMUQUsjT4Du9Bg== X-Received: by 2002:a02:6d24:: with SMTP id m36mr23423007jac.87.1562613666386; Mon, 08 Jul 2019 12:21:06 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.05 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:06 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:29 -0600 Message-Id: <20190708191856.138863-11-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 10/37] binman: Add missing comments to bsection X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" Some functions lack comments in this file. Add comments to cover this functionality. Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/bsection.py | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/tools/binman/bsection.py b/tools/binman/bsection.py index a3026718794..e0e3707db41 100644 --- a/tools/binman/bsection.py +++ b/tools/binman/bsection.py @@ -452,13 +452,36 @@ class Section(object): source_entry.Raise("Cannot find entry for node '%s'" % node.name) def ExpandSize(self, size): + """Change the size of an entry + + Args: + size: New size for entry + """ if size != self._size: self._size = size def GetRootSkipAtStart(self): + """Get the skip-at-start value for the top-level section + + This is used to find out the starting offset for root section that + contains this section. If this is a top-level section then it returns + the skip-at-start offset for this section. + + This is used to get the absolute position of section within the image. + + Returns: + Integer skip-at-start value for the root section containing this + section + """ if self._parent_section: return self._parent_section.GetRootSkipAtStart() return self._skip_at_start def GetImageSize(self): + """Get the size of the image containing this section + + Returns: + Image size as an integer number of bytes, which may be None if the + image size is dynamic and its sections have not yet been packed + """ return self._image._size From patchwork Mon Jul 8 19:18:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129273 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="VLjBY1S/"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFnT6lxKz9sDB for ; Tue, 9 Jul 2019 05:26:53 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 9454EC21D8A; Mon, 8 Jul 2019 19:24:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 384A7C21DED; Mon, 8 Jul 2019 19:21:20 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 2EB34C21E1E; Mon, 8 Jul 2019 19:21:13 +0000 (UTC) Received: from mail-io1-f66.google.com (mail-io1-f66.google.com [209.85.166.66]) by lists.denx.de (Postfix) with ESMTPS id 6E577C21DF9 for ; Mon, 8 Jul 2019 19:21:08 +0000 (UTC) Received: by mail-io1-f66.google.com with SMTP id s7so37812348iob.11 for ; Mon, 08 Jul 2019 12:21:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TUySJ0k5KIP9Gps2T22HwJ0ZxmvHxLtTtmHClKcccKQ=; b=VLjBY1S/yG3SqX4N0aJQtrBBPiU//3/c/HLEJfW4eN0os0ssohNJlO5rVPqn/FKkqn IWv9Wl8PV2GdpuRbJHRiVnXuHRE1qTh59pl6O61PsTbVC5q1zw74DSeltLzCxWDJPw1Z N8LAArcQf4baOlf0I7/65NO/TX+YqWJLYbubA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TUySJ0k5KIP9Gps2T22HwJ0ZxmvHxLtTtmHClKcccKQ=; b=KvlVDMp++ZS6hvOxWrwbjMwrjeO9uCb8Mp/NQ+28E4zKaOsrA4wigyGYsOky4hW/8l erkbimyNWqM2PaTtTHqM9A1RzlXy/vpv/GuIytbSQLj6uDqFE42TpQXtTAXvoz34Xv+g 5ci0U1l2tQGC99gOaH0690sx0zP/XjOtLUsKaCFoaU33JO97qYRbWQfjWyf0eyeRWaLe wACGHpPoxLHe3Kjuc4UkmVIQpD0MmPM9CEGMDjs0bwIKFuwxclEemUhCUYHFKuYudc2W /4osThPTNzy621fe4aA524bopHtWajP72PV/6svXeLZPlW2oqFX5zzUKKs2AETfScVbD RWzQ== X-Gm-Message-State: APjAAAUwpon3/6qUlC8YD5/jiK5UbNxdxQIe3iGnU0Wc6YyJRyBjjNAY FjH7pjqyo6JzCN1xqjE9nvWD3vz1H4s= X-Google-Smtp-Source: APXvYqxcmNDxIZLl72gYXkjAZnFsqs8teaaTJMLYceA/dAUD8RC8Os9j7AUGWBGPUWO1nqW+cOaodg== X-Received: by 2002:a02:54c1:: with SMTP id t184mr24313221jaa.10.1562613667252; Mon, 08 Jul 2019 12:21:07 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.06 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:06 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:30 -0600 Message-Id: <20190708191856.138863-12-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 11/37] binman: Add missing comments toentry X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" At present GetOffsets() lacks a function comment. Add one. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/entry.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/tools/binman/entry.py b/tools/binman/entry.py index d842d89dd66..e8d0adec1e9 100644 --- a/tools/binman/entry.py +++ b/tools/binman/entry.py @@ -355,6 +355,21 @@ class Entry(object): return self.data def GetOffsets(self): + """Get the offsets for siblings + + Some entry types can contain information about the position or size of + other entries. An example of this is the Intel Flash Descriptor, which + knows where the Intel Management Engine section should go. + + If this entry knows about the position of other entries, it can specify + this by returning values here + + Returns: + Dict: + key: Entry type + value: List containing position and size of the given entry + type. + """ return {} def SetOffsetSize(self, pos, size): From patchwork Mon Jul 8 19:18:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129280 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="BuwjpD0j"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFss66v0z9sNH for ; Tue, 9 Jul 2019 05:30:39 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 91D2FC21DA6; Mon, 8 Jul 2019 19:25:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 10F31C21E18; Mon, 8 Jul 2019 19:21:28 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id BBBF5C21DB6; Mon, 8 Jul 2019 19:21:13 +0000 (UTC) Received: from mail-io1-f66.google.com (mail-io1-f66.google.com [209.85.166.66]) by lists.denx.de (Postfix) with ESMTPS id 05A11C21DFD for ; Mon, 8 Jul 2019 19:21:09 +0000 (UTC) Received: by mail-io1-f66.google.com with SMTP id z3so22878719iog.0 for ; Mon, 08 Jul 2019 12:21:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AUzIgPGH/PJUm6gX49Yic/ksdM2+hKAbLQdk5RNmvbg=; b=BuwjpD0jw5TpUsFql6WN2xu18A2lplRbCCa09OiC7cH7Q1viGHKod1nFpWp4Mp4Do3 RvXENf6YysmSAzxN+QQdY6vT5TmIbJSGMtT+d+Vyp28mbBnzP2cfNEekXBB2/YTx4RJS THKfq2osh4DrM5bGaEQjHxZTiDvxf6N0q0Tq4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AUzIgPGH/PJUm6gX49Yic/ksdM2+hKAbLQdk5RNmvbg=; b=kTdDWf/qG/tEBvJIORqB8D1QZjkkYH0AJTxLsxK113LTBtWAZGg2N+deJ4ub8gE2om Bk+H18wS4QPfh6xWq2K096VXFMa0JKyFDp84CUufNKZ8rtxjQA+/R+Hfp+2PfuhfNLRy 2BjUJn6t4envupVJp74sbugZE1dCIpuyPemT0illoDOFDZmovjnBgnTTFuWbgAii7HDg 1vBAeBTXB9sT7DkBJarrcZfGPW4jWqscFzYnbQJmEdwSNsBIOmqsEfMEH0FrE2gqaFZF AAt69J9A4qx19oz/UtTmR+RxhaWNMf+bO9QVFdE5dsWwck6KSVBy3jM74yjpwGokvhwS 6S7Q== X-Gm-Message-State: APjAAAU0m6Z9GA0oVWEO6sdlA4TqUSL/VD1dNKmw9oQEdl8lOEtgGaAx gkILWXzn+7pVBzWWydq2O6Fx3UNpUgY= X-Google-Smtp-Source: APXvYqyk54l0drr/0x6E5VIMvIa8sEiBLi+ooZyhWz1D+GrRD2n+0mVd7k+qIejyNNpDcW9Ak8QzWg== X-Received: by 2002:a02:c65a:: with SMTP id k26mr16917969jan.18.1562613667930; Mon, 08 Jul 2019 12:21:07 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.07 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:07 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:31 -0600 Message-Id: <20190708191856.138863-13-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 12/37] binman: Tidy up help for --indir X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" The current help is confusing. Adjust it to indicate what the flag actually does. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/cmdline.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/binman/cmdline.py b/tools/binman/cmdline.py index ee19c5e33fe..39b835666ea 100644 --- a/tools/binman/cmdline.py +++ b/tools/binman/cmdline.py @@ -35,7 +35,7 @@ def ParseArgs(argv): parser.add_option('-i', '--image', type='string', action='append', help='Image filename to build (if not specified, build all)') parser.add_option('-I', '--indir', action='append', - help='Add a path to a directory to use for input files') + help='Add a path to the list of directories to use for input files') parser.add_option('-H', '--full-help', action='store_true', default=False, help='Display the README file') parser.add_option('-m', '--map', action='store_true', From patchwork Mon Jul 8 19:18:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129278 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="SdMgj7PL"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFrQ23jkz9sNH for ; Tue, 9 Jul 2019 05:29:26 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id E149BC21C51; Mon, 8 Jul 2019 19:26:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 81CBCC21E45; Mon, 8 Jul 2019 19:21:28 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 039BFC21DB6; Mon, 8 Jul 2019 19:21:14 +0000 (UTC) Received: from mail-io1-f52.google.com (mail-io1-f52.google.com [209.85.166.52]) by lists.denx.de (Postfix) with ESMTPS id AF209C21DF3 for ; Mon, 8 Jul 2019 19:21:09 +0000 (UTC) Received: by mail-io1-f52.google.com with SMTP id k20so37784819ios.10 for ; Mon, 08 Jul 2019 12:21:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+uVadY6LvgBxX3Z/2aDBBIO1caFYKwcaiJLTyhicidM=; b=SdMgj7PL+CC3Yk0BLMdEUcKZiOzHgn7MKgOPvO3vheIbicDw612PlZYNi3pERi42v9 vziTcEclRbDvKqTbdk2OyT8X4WSUA1FCPN614r4EP8iLfabTeR3VjCsSTLbHtNAlbDRI VG3E8xAZru/VQim/fQg9+XEEdOZaeXRAioQOY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+uVadY6LvgBxX3Z/2aDBBIO1caFYKwcaiJLTyhicidM=; b=ervhx/6JdaVhr/oP3jjb12ZyZQ5T8HwrxH0ezGRgg8IRzGrsNhRJTlMlnXkexDBHyq 1FInDtCmSnWZCIG+5OH6r3Odvo8PkBQHQwcQCAMKt8zl3cQTTZibPIZtijJpjHfUs+zJ mHs1xgwqPZkNKGbX0o0ErGGUhV6N8uHTR+e+lu7SEV3HGopRKuV4r1OKOQeC7Sw6Vlh7 BFHgVSeJD4gFWsBSUGALzBLqtgaDcf8LKOuoqQjAUYJWjqYuPj8+sQp6LQJuebsiaNQG 4SIrtE5+HycyPdlbVNgidxdzM6RsoB5bDArN15jwrqIuNqbRIzcf7lAMaXIA+pbhgYcL wjWQ== X-Gm-Message-State: APjAAAU84VKUcOt/R7fViFo21npMQL1Cj+2/9ZXeFtwpBB0rXbtASA4k URBgaAO8n5xy/HhhMpIqijGZ/ObFFLg= X-Google-Smtp-Source: APXvYqy03w8+6PUJvfOx8QBHxtTLj2G8YAQ0Awdq0vk+RDR+aBdQdi2TGSRtKklVKEdZhVO0Z04lug== X-Received: by 2002:a6b:6d07:: with SMTP id a7mr5027211iod.254.1562613668557; Mon, 08 Jul 2019 12:21:08 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.08 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:08 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:32 -0600 Message-Id: <20190708191856.138863-14-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 13/37] binman: Use a better error for missing Intel descriptor X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" FD is a bit confusing so write this out in full. Also avoid splitting the string so that people can grep for the error message more easily. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/etype/intel_descriptor.py | 2 +- tools/binman/ftest.py | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/binman/etype/intel_descriptor.py b/tools/binman/etype/intel_descriptor.py index 6acbbd8b7a5..9deb8dcf42c 100644 --- a/tools/binman/etype/intel_descriptor.py +++ b/tools/binman/etype/intel_descriptor.py @@ -51,7 +51,7 @@ class Entry_intel_descriptor(Entry_blob): def GetOffsets(self): offset = self.data.find(FD_SIGNATURE) if offset == -1: - self.Raise('Cannot find FD signature') + self.Raise('Cannot find Intel Flash Descriptor (FD) signature') flvalsig, flmap0, flmap1, flmap2 = struct.unpack('> 16) & 0xff) << 4 diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 46f669e73b4..8577adb5380 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -806,8 +806,8 @@ class TestFunctional(unittest.TestCase): TestFunctional._MakeInputFile('descriptor.bin', b'') with self.assertRaises(ValueError) as e: self._DoTestFile('031_x86-rom-me.dts') - self.assertIn("Node '/binman/intel-descriptor': Cannot find FD " - "signature", str(e.exception)) + self.assertIn("Node '/binman/intel-descriptor': Cannot find Intel Flash Descriptor (FD) signature", + str(e.exception)) def testPackX86RomBadDesc(self): """Test that the Intel requires a descriptor entry""" From patchwork Mon Jul 8 19:18:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129289 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="JsPZWtzC"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFyM2GfZz9sDB for ; Tue, 9 Jul 2019 05:34:35 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id E773FC21D4A; Mon, 8 Jul 2019 19:25:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 9E3E9C21E1B; Mon, 8 Jul 2019 19:21:25 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id B9FF7C21D4A; Mon, 8 Jul 2019 19:21:14 +0000 (UTC) Received: from mail-io1-f44.google.com (mail-io1-f44.google.com [209.85.166.44]) by lists.denx.de (Postfix) with ESMTPS id 4BAE1C21CB6 for ; Mon, 8 Jul 2019 19:21:10 +0000 (UTC) Received: by mail-io1-f44.google.com with SMTP id j6so37854637ioa.5 for ; Mon, 08 Jul 2019 12:21:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MEKkma6QQ0Npf+hqEz45BXVEFVv5ce5Oh6Jc14QT/hY=; b=JsPZWtzCfBwpmN4GA7l4ClgtWOsH6KSyqExds3c6oMPI/lPh4rvhBBW2RXpQhCb79n La5tLZs+C9i2EeIgLBvijjrA/r3AjUYxWyCSUPFnrg9X84hwp3qEboFncR6xepzSmw/L 1cDZeGxwsdeUPc7ZuQALesowknGBWEPRegWeI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MEKkma6QQ0Npf+hqEz45BXVEFVv5ce5Oh6Jc14QT/hY=; b=Etta2t0VOoKMT+WRdxGPUBY8Wbr16tVCZCw5DbgCuEnWHZbulSS6CJ4llGEjDANx/q CKqU2OG3NG0KRY+OI58/6qULexA44kH7YWt8aOSDxSrR24k5qOpY0H2uQXY5BbsWr3B+ 8POiNNaZJEKcsyZZr6O/SfA76yZiY30eCdppjpF6YNDX5kcaAnvt3Q+fSU6p3QgiD6g8 1vFRUx0osCtHwetrfjLBQZKRfJ4JAm7O/m4iZJfr2AltkB58nQUNL9CJCuaf+PJOxyj3 nfgbCJ9ly+8/GqgwB6wUo0t1jjCzP74FzCqe1iTndcd8YUAXnQeGQWANLSdpyUpn1y4F OJng== X-Gm-Message-State: APjAAAWOo38nKvxeRfSGaBvcDiVAjiKySgpGB88EpeqWlekEMO2qPkJm +O/T8JQaGKK/QLxp1x6wmszanBvboac= X-Google-Smtp-Source: APXvYqy8a4ZbhJsrV30Wm0RTQZKHChEdYriXxFY8RqDWi8sOOLHrFHftExkiZ9BES30l02MQlyKwlw== X-Received: by 2002:a5d:890d:: with SMTP id b13mr16293198ion.124.1562613669189; Mon, 08 Jul 2019 12:21:09 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.08 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:08 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:33 -0600 Message-Id: <20190708191856.138863-15-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 14/37] binman: Detect skipped tests X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" If tests are skipped we should ideally exit with an error, since there may be a missing dependency. However at present this is not desirable since it breaks travis tests. For now, just report the skips. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/binman.py | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/tools/binman/binman.py b/tools/binman/binman.py index aad2e9c8bc4..9f8c5c99b79 100755 --- a/tools/binman/binman.py +++ b/tools/binman/binman.py @@ -104,9 +104,14 @@ def RunTests(debug, processes, args): print(test.id(), err) for test, err in result.failures: print(err, result.failures) + if result.skipped: + print('%d binman test%s SKIPPED:' % + (len(result.skipped), 's' if len(result.skipped) > 1 else '')) + for skip_info in result.skipped: + print('%s: %s' % (skip_info[0], skip_info[1])) if result.errors or result.failures: - print('binman tests FAILED') - return 1 + print('binman tests FAILED') + return 1 return 0 def GetEntryModules(include_testing=True): From patchwork Mon Jul 8 19:18:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129243 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="f42OWMME"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFls1Hxmz9sNx for ; Tue, 9 Jul 2019 05:25:29 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id DB612C21D4A; Mon, 8 Jul 2019 19:24:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 31237C21E0B; Mon, 8 Jul 2019 19:21:17 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 9D6D6C21D9A; Mon, 8 Jul 2019 19:21:14 +0000 (UTC) Received: from mail-io1-f42.google.com (mail-io1-f42.google.com [209.85.166.42]) by lists.denx.de (Postfix) with ESMTPS id 0821EC21DB3 for ; Mon, 8 Jul 2019 19:21:11 +0000 (UTC) Received: by mail-io1-f42.google.com with SMTP id m24so28367501ioo.2 for ; Mon, 08 Jul 2019 12:21:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kgXYgctlTvxonU6v8HoSd7LsuglfONWuG3eeFIN7z/o=; b=f42OWMME/P6EXEohKp66Ve1wYxRF5GYqVyWNHxJNM4klwJeAAHu93GX2gX418zCsB6 9273Aba/xb+ewdpSwGBdorzDp31sHKXdIuiNm5j8mql3uZCDlsYFzmN7dhlNUVjVRnWe p1R/QF3eMUCEDuZJleznw7eHlEbwYuNMKuMpA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kgXYgctlTvxonU6v8HoSd7LsuglfONWuG3eeFIN7z/o=; b=YmsFUQTHxMXAKJrhHmPw+2DpTCPm1IHrxMT+u3BlU5AQH4Wbfsy4vB1v1rvpUIwCeD H9Aauc1A6amvp3kcxIdImPhyeoRMF1nuLRQTihWivzmbyTR1vaeMmVeEsH0sgrN1DerG 2NC4cNOWd0I+nF5BxZbfI3PychMhz5nfTnorA9DgDslvyX0LxPAD2/yem4uBH1BQszGT 8ivSO76E8mzsZH3EU8x9PwD1IwpLT5/vUcgbGxWdmkT75HbBApR4c8W/DAH1rPy001Uc SCRR3fBWKaeoWT7CT3iWyP2WVHym+f9MZf0VHyLLDPlxwY3YQVKdp+MZjfEVS+iwzLG3 nngQ== X-Gm-Message-State: APjAAAV4kPfQKWVw0i+goPLGdOXMFzIrlCvwvaISSppxPsJy9t5FqbkL 5JQU8wQ/3QPt3iYxwfFwm4HHKSARyFI= X-Google-Smtp-Source: APXvYqw7uwTNgodTN4WX8btdium3p12NVQTl27fHBlF4odihXmNRd7mNA3zM7EAAdVoyOuF6w952YQ== X-Received: by 2002:a5e:8210:: with SMTP id l16mr21223084iom.240.1562613669850; Mon, 08 Jul 2019 12:21:09 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.09 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:09 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:34 -0600 Message-Id: <20190708191856.138863-16-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 15/37] binman: Add a function to create a sample ELF file X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" It is useful to create an ELF file for testing purposes, with just the right attributes used by the test. Add a function to handle this, along with a test that it works correctly. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/elf.py | 98 ++++++++++++++++++++++++++++++++++++++++ tools/binman/elf_test.py | 20 ++++++++ 2 files changed, 118 insertions(+) diff --git a/tools/binman/elf.py b/tools/binman/elf.py index 828681d76d0..e6dc6ec1a9b 100644 --- a/tools/binman/elf.py +++ b/tools/binman/elf.py @@ -5,11 +5,15 @@ # Handle various things related to ELF images # +from __future__ import print_function + from collections import namedtuple, OrderedDict import command import os import re +import shutil import struct +import tempfile import tools @@ -128,3 +132,97 @@ def LookupAndWriteSymbols(elf_fname, entry, section): (msg, name, offset, value, len(value_bytes))) entry.data = (entry.data[:offset] + value_bytes + entry.data[offset + sym.size:]) + +def MakeElf(elf_fname, text, data): + """Make an elf file with the given data in a single section + + The output file has a several section including '.text' and '.data', + containing the info provided in arguments. + + Args: + elf_fname: Output filename + text: Text (code) to put in the file's .text section + data: Data to put in the file's .data section + """ + outdir = tempfile.mkdtemp(prefix='binman.elf.') + s_file = os.path.join(outdir, 'elf.S') + + # Spilt the text into two parts so that we can make the entry point two + # bytes after the start of the text section + text_bytes1 = ['\t.byte\t%#x' % tools.ToByte(byte) for byte in text[:2]] + text_bytes2 = ['\t.byte\t%#x' % tools.ToByte(byte) for byte in text[2:]] + data_bytes = ['\t.byte\t%#x' % tools.ToByte(byte) for byte in data] + with open(s_file, 'w') as fd: + print('''/* Auto-generated C program to produce an ELF file for testing */ + +.section .text +.code32 +.globl _start +.type _start, @function +%s +_start: +%s +.ident "comment" + +.comm fred,8,4 + +.section .empty +.globl _empty +_empty: +.byte 1 + +.globl ernie +.data +.type ernie, @object +.size ernie, 4 +ernie: +%s +''' % ('\n'.join(text_bytes1), '\n'.join(text_bytes2), '\n'.join(data_bytes)), + file=fd) + lds_file = os.path.join(outdir, 'elf.lds') + + # Use a linker script to set the alignment and text address. + with open(lds_file, 'w') as fd: + print('''/* Auto-generated linker script to produce an ELF file for testing */ + +PHDRS +{ + text PT_LOAD ; + data PT_LOAD ; + empty PT_LOAD FLAGS ( 6 ) ; + note PT_NOTE ; +} + +SECTIONS +{ + . = 0xfef20000; + ENTRY(_start) + .text . : SUBALIGN(0) + { + *(.text) + } :text + .data : { + *(.data) + } :data + _bss_start = .; + .empty : { + *(.empty) + } :empty + .note : { + *(.comment) + } :note + .bss _bss_start (OVERLAY) : { + *(.bss) + } +} +''', file=fd) + # -static: Avoid requiring any shared libraries + # -nostdlib: Don't link with C library + # -Wl,--build-id=none: Don't generate a build ID, so that we just get the + # text section at the start + # -m32: Build for 32-bit x86 + # -T...: Specifies the link script, which sets the start address + stdout = command.Output('cc', '-static', '-nostdlib', '-Wl,--build-id=none', + '-m32','-T', lds_file, '-o', elf_fname, s_file) + shutil.rmtree(outdir) + diff --git a/tools/binman/elf_test.py b/tools/binman/elf_test.py index 42d94cbbbe2..3172982427d 100644 --- a/tools/binman/elf_test.py +++ b/tools/binman/elf_test.py @@ -5,9 +5,12 @@ # Test for the elf module import os +import shutil import sys +import tempfile import unittest +import command import elf import test_util import tools @@ -136,6 +139,23 @@ class TestElf(unittest.TestCase): elf.debug = False self.assertTrue(len(stdout.getvalue()) > 0) + def testMakeElf(self): + """Test for the MakeElf function""" + outdir = tempfile.mkdtemp(prefix='elf.') + expected_text = b'1234' + expected_data = b'wxyz' + elf_fname = os.path.join(outdir, 'elf') + bin_fname = os.path.join(outdir, 'elf') + + # Make an Elf file and then convert it to a fkat binary file. This + # should produce the original data. + elf.MakeElf(elf_fname, expected_text, expected_data) + stdout = command.Output('objcopy', '-O', 'binary', elf_fname, bin_fname) + with open(bin_fname, 'rb') as fd: + data = fd.read() + self.assertEqual(expected_text + expected_data, data) + shutil.rmtree(outdir) + if __name__ == '__main__': unittest.main() From patchwork Mon Jul 8 19:18:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129292 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="IQSpRZKl"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jG0j1266z9sDB for ; Tue, 9 Jul 2019 05:36:37 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id B21D5C21DF3; Mon, 8 Jul 2019 19:28:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=RCVD_IN_MSPIKE_H2, T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 8FCA4C21E50; Mon, 8 Jul 2019 19:21:47 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id B3976C21E76; Mon, 8 Jul 2019 19:21:15 +0000 (UTC) Received: from mail-io1-f68.google.com (mail-io1-f68.google.com [209.85.166.68]) by lists.denx.de (Postfix) with ESMTPS id C8969C21E0B for ; Mon, 8 Jul 2019 19:21:11 +0000 (UTC) Received: by mail-io1-f68.google.com with SMTP id z3so22879048iog.0 for ; Mon, 08 Jul 2019 12:21:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=z9iCwtkMeX+t9aiy28IV0rQuJ8NEVHwXhb+W9umyo74=; b=IQSpRZKl7q5mkpCow55rK9hUEm9daLPKc/smvz1RF2PWY3jTmQ/aihSSbmPUnUTqxS Z0i/4KLJPJ7RAxitQ9xNKE9z/DqL5qRcgCBZmc7zxtEgIzOEl3+I0iv8/fBaFOzhGtLJ l1lpwAzd9ulUz9TSxL+iVr6Q03bC1W0FUyFyc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=z9iCwtkMeX+t9aiy28IV0rQuJ8NEVHwXhb+W9umyo74=; b=Y+kLrqT7ZtZsfm0UzSuMvsfni4enYdceaGkgSpuMLYi+XelojBwU5OF7OWgLF27a6W akFys72PPS7nWVnG0CyYZN1h5K0PWeyQh+KC+9fCe7b0FpW50TSJUiqN3j/MMJjBokYE Jsg9WTh1oB2sLqMTnbB6DtbDUiyGuk9wZ5i5LN0LmBmST0TWR7IiGukC+tTb8QnEv9Lu xwgWPUpbRDBWgfirnOrKrZlVauX2NtINt8AV0AUaZTJK8UDZGcj/R0log5H4vx4QqJkP OkrtS3thdBTg/En1wcY1wCI2nN81mP9F/jA0a9sFFh7Zo5+0+OjCRZxc13tCi8wInFj0 JTWA== X-Gm-Message-State: APjAAAXU2N4uxZnlc9Ko6zDkUemfiaoazOuYUp0WtN7hkvBShRh4SPZo IHlVF8gaE7QlQXjuxuE9e/SKfA+PlJM= X-Google-Smtp-Source: APXvYqwGr2DQqPCkWH5U8Qeplo8fh7CdoG/EDoVG9RkG2DGUOWhLI+K9Navz41JL7ct5MogyxeJfSQ== X-Received: by 2002:a6b:7b01:: with SMTP id l1mr5533785iop.60.1562613670476; Mon, 08 Jul 2019 12:21:10 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.09 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:10 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:35 -0600 Message-Id: <20190708191856.138863-17-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 16/37] binman: Add a function to decode an ELF file X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" Add a function which decodes an ELF file, working out where in memory each part of the data should be written. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/README | 4 +++ tools/binman/elf.py | 76 ++++++++++++++++++++++++++++++++++++++++ tools/binman/elf_test.py | 21 +++++++++++ 3 files changed, 101 insertions(+) diff --git a/tools/binman/README b/tools/binman/README index 28624fadb33..0ff30ef6fd9 100644 --- a/tools/binman/README +++ b/tools/binman/README @@ -181,6 +181,10 @@ the configuration of the Intel-format descriptor. Running binman -------------- +First install prerequisites, e.g. + + sudo apt-get install python-pyelftools python3-pyelftools + Type: binman -b diff --git a/tools/binman/elf.py b/tools/binman/elf.py index e6dc6ec1a9b..8147b3437dd 100644 --- a/tools/binman/elf.py +++ b/tools/binman/elf.py @@ -9,6 +9,7 @@ from __future__ import print_function from collections import namedtuple, OrderedDict import command +import io import os import re import shutil @@ -17,11 +18,26 @@ import tempfile import tools +ELF_TOOLS = True +try: + from elftools.elf.elffile import ELFFile + from elftools.elf.sections import SymbolTableSection +except: # pragma: no cover + ELF_TOOLS = False + # This is enabled from control.py debug = False Symbol = namedtuple('Symbol', ['section', 'address', 'size', 'weak']) +# Information about an ELF file: +# data: Extracted program contents of ELF file (this would be loaded by an +# ELF loader when reading this file +# load: Load address of code +# entry: Entry address of code +# memsize: Number of bytes in memory occupied by loading this ELF file +ElfInfo = namedtuple('ElfInfo', ['data', 'load', 'entry', 'memsize']) + def GetSymbols(fname, patterns): """Get the symbols from an ELF file @@ -226,3 +242,63 @@ SECTIONS '-m32','-T', lds_file, '-o', elf_fname, s_file) shutil.rmtree(outdir) +def DecodeElf(data, location): + """Decode an ELF file and return information about it + + Args: + data: Data from ELF file + location: Start address of data to return + + Returns: + ElfInfo object containing information about the decoded ELF file + """ + file_size = len(data) + with io.BytesIO(data) as fd: + elf = ELFFile(fd) + data_start = 0xffffffff; + data_end = 0; + mem_end = 0; + virt_to_phys = 0; + + for i in range(elf.num_segments()): + segment = elf.get_segment(i) + if segment['p_type'] != 'PT_LOAD' or not segment['p_memsz']: + skipped = 1 # To make code-coverage see this line + continue + start = segment['p_paddr'] + mend = start + segment['p_memsz'] + rend = start + segment['p_filesz'] + data_start = min(data_start, start) + data_end = max(data_end, rend) + mem_end = max(mem_end, mend) + if not virt_to_phys: + virt_to_phys = segment['p_paddr'] - segment['p_vaddr'] + + output = bytearray(data_end - data_start) + for i in range(elf.num_segments()): + segment = elf.get_segment(i) + if segment['p_type'] != 'PT_LOAD' or not segment['p_memsz']: + skipped = 1 # To make code-coverage see this line + continue + start = segment['p_paddr'] + offset = 0 + if start < location: + offset = location - start + start = location + # A legal ELF file can have a program header with non-zero length + # but zero-length file size and a non-zero offset which, added + # together, are greater than input->size (i.e. the total file size). + # So we need to not even test in the case that p_filesz is zero. + # Note: All of this code is commented out since we don't have a test + # case for it. + size = segment['p_filesz'] + #if not size: + #continue + #end = segment['p_offset'] + segment['p_filesz'] + #if end > file_size: + #raise ValueError('Underflow copying out the segment. File has %#x bytes left, segment end is %#x\n', + #file_size, end) + output[start - data_start:start - data_start + size] = ( + segment.data()[offset:]) + return ElfInfo(output, data_start, elf.header['e_entry'] + virt_to_phys, + mem_end - data_start) diff --git a/tools/binman/elf_test.py b/tools/binman/elf_test.py index 3172982427d..e2506377f26 100644 --- a/tools/binman/elf_test.py +++ b/tools/binman/elf_test.py @@ -156,6 +156,27 @@ class TestElf(unittest.TestCase): self.assertEqual(expected_text + expected_data, data) shutil.rmtree(outdir) + def testDecodeElf(self): + """Test for the MakeElf function""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') + outdir = tempfile.mkdtemp(prefix='elf.') + expected_text = b'1234' + expected_data = b'wxyz' + elf_fname = os.path.join(outdir, 'elf') + elf.MakeElf(elf_fname, expected_text, expected_data) + data = tools.ReadFile(elf_fname) + + load = 0xfef20000 + entry = load + 2 + expected = expected_text + expected_data + self.assertEqual(elf.ElfInfo(expected, load, entry, len(expected)), + elf.DecodeElf(data, 0)) + self.assertEqual(elf.ElfInfo(b'\0\0' + expected[2:], + load, entry, len(expected)), + elf.DecodeElf(data, load + 2)) + #shutil.rmtree(outdir) + if __name__ == '__main__': unittest.main() From patchwork Mon Jul 8 19:18:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129288 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="EnrTirXe"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFxl4xnFz9sDB for ; Tue, 9 Jul 2019 05:34:03 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 35B2AC21D65; Mon, 8 Jul 2019 19:26:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=RCVD_IN_MSPIKE_H2, T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 5E81EC21D83; Mon, 8 Jul 2019 19:21:36 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 5F3CFC21E3B; Mon, 8 Jul 2019 19:21:16 +0000 (UTC) Received: from mail-io1-f68.google.com (mail-io1-f68.google.com [209.85.166.68]) by lists.denx.de (Postfix) with ESMTPS id 8A9E9C21DF8 for ; Mon, 8 Jul 2019 19:21:12 +0000 (UTC) Received: by mail-io1-f68.google.com with SMTP id h6so29616969iom.7 for ; Mon, 08 Jul 2019 12:21:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4DtTwOzjL0LjFo6IzpJBbQ9P6f8R2zDl5+sFqnDc2Rg=; b=EnrTirXeGBr228QwOYKbA1ghAXg/1AB5+wWqt7PkwvzfolNjivUW7ZFedQTE7D5Sbp 6/lTWPG+Nb91wTxuFO+xqWrSxOsmm5jU6v7bbz604mpFgK+iZ9tSplfttw1Fz16aDwBU 6+0a6IuP14IU8FxwsZ2958vnxvF4VvAdR9J/g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4DtTwOzjL0LjFo6IzpJBbQ9P6f8R2zDl5+sFqnDc2Rg=; b=Q0WoSXhZzGfgbmghzYVTyLMnVASdy6Gnb3tMfhFcADF76PCWcPdGiEsZogBZKXmEw3 kg5zuME3gzMEStyaq4KW57aTC0EPUJLvh5Jonlla7MENIm6oUVH7/ByJrAwBzY2QP7mT p0qG/E3Wz+NzRolg+mkVtwg0YWn8pE8SPzfEccIpjba0klq9xIi7hSiofyW4F5Eb13NN f26bZq0HvKprdoUQBdpEepbhcCIuFGOCbo1Gnp1sxGnqepWQEQNO5TA/jWDD4dRjqGeU znNhBNwQIjAEXGTTAGf9Hg0Ex4HP9zP2CkyzVHCVx6LUKGl1Oh/dTfPxkT7Bdfjw5FH4 CnDg== X-Gm-Message-State: APjAAAUyDlw7loGyqy480xpfz6Z3dAlknYA9hWeQT2BxW0PD1QziwIGI M9uQi201T5Vk5lMm3Ap4hdaHXe2I9DY= X-Google-Smtp-Source: APXvYqyrRRcRLQzMERHP5MfwkSmpZcLiMLdQS/Dde08t/RQ+uS+4Q+OJXBpFe957Is6w1ZlF+g0khg== X-Received: by 2002:a02:b68f:: with SMTP id i15mr23171832jam.107.1562613671303; Mon, 08 Jul 2019 12:21:11 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.10 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:10 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:36 -0600 Message-Id: <20190708191856.138863-18-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 17/37] binman: Ensure that coverage has access to site packages X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" Code coverage tests fail on binman due to dist-packages being dropped from the python path on Ubuntu 16.04. Add them in so that we can find the elffile module, which is required by binman. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/binman.py | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tools/binman/binman.py b/tools/binman/binman.py index 9f8c5c99b79..05aeaecd8f3 100755 --- a/tools/binman/binman.py +++ b/tools/binman/binman.py @@ -11,9 +11,11 @@ from __future__ import print_function +from distutils.sysconfig import get_python_lib import glob import multiprocessing import os +import site import sys import traceback import unittest @@ -28,6 +30,12 @@ sys.path.insert(0, 'scripts/dtc/pylibfdt') sys.path.insert(0, os.path.join(our_path, '../../build-sandbox_spl/scripts/dtc/pylibfdt')) +# When running under python-coverage on Ubuntu 16.04, the dist-packages +# directories are dropped from the python path. Add them in so that we can find +# the elffile module. We could use site.getsitepackages() here but unfortunately +# that is not available in a virtualenv. +sys.path.append(get_python_lib()) + import cmdline import command use_concurrent = True From patchwork Mon Jul 8 19:18:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129299 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="dLc62KqM"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jG4Q3CPHz9sNH for ; Tue, 9 Jul 2019 05:39:50 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 2E36EC21DF3; Mon, 8 Jul 2019 19:28:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id C9702C21E26; Mon, 8 Jul 2019 19:21:46 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 6F6AFC21DB6; Mon, 8 Jul 2019 19:21:16 +0000 (UTC) Received: from mail-io1-f42.google.com (mail-io1-f42.google.com [209.85.166.42]) by lists.denx.de (Postfix) with ESMTPS id 417F2C21DA1 for ; Mon, 8 Jul 2019 19:21:13 +0000 (UTC) Received: by mail-io1-f42.google.com with SMTP id h6so29617043iom.7 for ; Mon, 08 Jul 2019 12:21:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TgJlnwSZEEnZKHg//2OEh2DC/gYivtTwc80YVGPi1wg=; b=dLc62KqMDggqiS42JwwCD6LuS2owl7TeCGDa+ryGeCI/JW0gJGo5/Z+sFUmxZjaIWT 5WLzUqG8RhuDIE4kv4fGz250m851d4LD+RZzhs0tQaa5x88ul4xltzUdCxRgUANBlxx4 Yx61vWFarDDNSMwCyMh22jU/zQnOJkJwV7gs4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TgJlnwSZEEnZKHg//2OEh2DC/gYivtTwc80YVGPi1wg=; b=hyeSYl4emBpYJ397TLZezQm3Tk5zrjNkCj4szu/nPAodknI63bGJweI92VTnXTNzJa w3ZtdS7pXhk0LJZBXkbGfR1oB+G2Rabqep60RoXciNCZV7BLl6P8wzwlXSEo0waqTf2H GdvAQRWqJvQF2osHuBoLR4c5kdzlpSocveChUScrOCyrFg4TGlgp4aOEA7gyKIcfyohb TKwnFT8+5JiCVzkYSB4OpXVjE3gjn95tcF2/g0q+lj5URjbGmPWudy8TYDXyVsnzCmBj wyyyou7zmTy5GGTyJ595wybkJDv7nD6Z4V33DkPYoJAGcML0oM9CiXaQq+Bmcd05T1l6 vI2w== X-Gm-Message-State: APjAAAUmoeOgrtb1l09wsYBBK5wmmxkFt+ldqa3mGBpNF1DffPnI22mA vn17W0WlM2sUxHKFxEB9k+JccSmYcms= X-Google-Smtp-Source: APXvYqzAv5xwD4mmqjUJicy+tp63XK0pahmkWpKnxxDmcFRka2+LR0fRLhsxxcjomzgymTm2OatwOg== X-Received: by 2002:a02:5b05:: with SMTP id g5mr22381432jab.114.1562613672100; Mon, 08 Jul 2019 12:21:12 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.11 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:11 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:37 -0600 Message-Id: <20190708191856.138863-19-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 18/37] binman: Assume Intel descriptor is at the start of the image X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" At present binman requires that the Intel descriptor has an explicit offset. Generally this is 0 since the descriptor is at the start of the image. Add a default to handle this, so users don't need to specify the offset. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/bsection.py | 8 ++++++++ tools/binman/etype/intel_descriptor.py | 2 ++ 2 files changed, 10 insertions(+) diff --git a/tools/binman/bsection.py b/tools/binman/bsection.py index e0e3707db41..49b8ef3e3e0 100644 --- a/tools/binman/bsection.py +++ b/tools/binman/bsection.py @@ -477,6 +477,14 @@ class Section(object): return self._parent_section.GetRootSkipAtStart() return self._skip_at_start + def GetStartOffset(self): + """Get the start offset for this section + + Returns: + The first available offset in this section (typically 0) + """ + return self._skip_at_start + def GetImageSize(self): """Get the size of the image containing this section diff --git a/tools/binman/etype/intel_descriptor.py b/tools/binman/etype/intel_descriptor.py index 9deb8dcf42c..661063457ed 100644 --- a/tools/binman/etype/intel_descriptor.py +++ b/tools/binman/etype/intel_descriptor.py @@ -47,6 +47,8 @@ class Entry_intel_descriptor(Entry_blob): def __init__(self, section, etype, node): Entry_blob.__init__(self, section, etype, node) self._regions = [] + if self.offset is None: + self.offset = self.section.GetStartOffset() def GetOffsets(self): offset = self.data.find(FD_SIGNATURE) From patchwork Mon Jul 8 19:18:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129301 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Mke9NBTw"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jG5p1QJkz9sDB for ; Tue, 9 Jul 2019 05:41:02 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id DE858C21E75; Mon, 8 Jul 2019 19:29:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 8FDF2C21E68; Mon, 8 Jul 2019 19:21:54 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 38771C21ECA; Mon, 8 Jul 2019 19:21:17 +0000 (UTC) Received: from mail-io1-f67.google.com (mail-io1-f67.google.com [209.85.166.67]) by lists.denx.de (Postfix) with ESMTPS id E104DC21DCA for ; Mon, 8 Jul 2019 19:21:13 +0000 (UTC) Received: by mail-io1-f67.google.com with SMTP id f4so22332438ioh.6 for ; Mon, 08 Jul 2019 12:21:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2Zw9WR+oiGH7tGP+dfdItFFAGuQq8ZOj0h+PASKm3Kw=; b=Mke9NBTwuMtDzF6c8S6xqZPBSzWsBi8zKzrOmNgoj2PFl0J3+gUdw4ycllW06EojXR agU1u1I85UoCdOsI7iB+mbPy/G5m6n0siNwf+llxBS/sWFPNSFgkIpyBCtaNWu3u0ocC KZUwNdU8CIwMsn9FctFv5Rnxl56HE3h9nx5xc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2Zw9WR+oiGH7tGP+dfdItFFAGuQq8ZOj0h+PASKm3Kw=; b=lz4ntmxOu7xBwNi0+ymqCAigUVorzT2kF9SqkzMR0s+33FUdwvnjo4yfHWwxWrflv5 LdsMALZUCB6LIXRROV4Ab1dTlwl2pdbN3wB69xwOYvV024crC01zmyfB0N3RbI8SBuSU MmuKyUKtnmUcxN5k4sZoTFU9LDzUJsD9h04nrRnsH/OPR5fwDMkt+h+SrIIYxklslyDA 73hytmq+ylTWEQXVpI57NLb9wE8S9JIek92gHugUK0mgolHaI9j0OiLPhA8K8VgsrrAr /wYM2QNzA4LN6qMiW5j1Jk/VAVmqKEasTKzWfgGGE9v2SCV+rYsAbFbfgH9YKYDBKKTT 7Nqg== X-Gm-Message-State: APjAAAWY0MOywqb38dbjvVuIFQ3GG/StIGiteTyEw5tHU94uK1nt9glF H9VYTMOY4dJvpfALU+zggYJBL7nokU0= X-Google-Smtp-Source: APXvYqyjfDY3I2CAcM+ev1H29dqsn0IFVCdL3dcLrRyh2yM6mDPfWBT47SkxVllDSMU0jDxQ2Hkh7g== X-Received: by 2002:a5d:8252:: with SMTP id n18mr15761211ioo.230.1562613672765; Mon, 08 Jul 2019 12:21:12 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.12 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:12 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:38 -0600 Message-Id: <20190708191856.138863-20-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 19/37] binman: Don't assume there is an ME region X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" At present having a descriptor means that there is an ME (Intel Management Engine) entry as well. The descriptor provides the ME location and assumes that it is present. For some SoCs this is not true. Before providing the location of a potentially non-existent entry, check if it is present. Update the comment in the ME entry also. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/README.entries | 2 ++ tools/binman/entry.py | 9 +++++++++ tools/binman/etype/intel_descriptor.py | 10 +++++++--- tools/binman/etype/intel_me.py | 2 ++ 4 files changed, 20 insertions(+), 3 deletions(-) diff --git a/tools/binman/README.entries b/tools/binman/README.entries index 357946d6305..702fc9fda08 100644 --- a/tools/binman/README.entries +++ b/tools/binman/README.entries @@ -206,6 +206,8 @@ does not directly execute code in the ME binary. A typical filename is 'me.bin'. +The position of this entry is generally set by the intel-descriptor entry. + See README.x86 for information about x86 binary blobs. diff --git a/tools/binman/entry.py b/tools/binman/entry.py index e8d0adec1e9..7ead997e0fd 100644 --- a/tools/binman/entry.py +++ b/tools/binman/entry.py @@ -544,3 +544,12 @@ features to produce new behaviours. # the data grows. This should not fail, but check it to be sure. if not self.ObtainContents(): self.Raise('Cannot obtain contents when expanding entry') + + def HasSibling(self, name): + """Check if there is a sibling of a given name + + Returns: + True if there is an entry with this name in the the same section, + else False + """ + return name in self.section.GetEntries() diff --git a/tools/binman/etype/intel_descriptor.py b/tools/binman/etype/intel_descriptor.py index 661063457ed..65ba2391e69 100644 --- a/tools/binman/etype/intel_descriptor.py +++ b/tools/binman/etype/intel_descriptor.py @@ -60,6 +60,10 @@ class Entry_intel_descriptor(Entry_blob): for i in range(MAX_REGIONS): self._regions.append(Region(self.data, frba, i)) - # Set the offset for ME only, for now, since the others are not used - return {'intel-me': [self._regions[REGION_ME].base, - self._regions[REGION_ME].size]} + # Set the offset for ME (Management Engine) only, for now, since the + # others are not used + info = {} + if self.HasSibling('intel-me'): + info['intel-me'] = [self._regions[REGION_ME].base, + self._regions[REGION_ME].size] + return info diff --git a/tools/binman/etype/intel_me.py b/tools/binman/etype/intel_me.py index 247c5b33866..c932ec52225 100644 --- a/tools/binman/etype/intel_me.py +++ b/tools/binman/etype/intel_me.py @@ -22,6 +22,8 @@ class Entry_intel_me(Entry_blob): A typical filename is 'me.bin'. + The position of this entry is generally set by the intel-descriptor entry. + See README.x86 for information about x86 binary blobs. """ def __init__(self, section, etype, node): From patchwork Mon Jul 8 19:18:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129293 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="VL90FEhA"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jG2W2b5dz9sDB for ; Tue, 9 Jul 2019 05:38:11 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 00837C21D8A; Mon, 8 Jul 2019 19:26:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 4DE32C21C3F; Mon, 8 Jul 2019 19:21:33 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 275D3C21DF9; Mon, 8 Jul 2019 19:21:18 +0000 (UTC) Received: from mail-io1-f65.google.com (mail-io1-f65.google.com [209.85.166.65]) by lists.denx.de (Postfix) with ESMTPS id 8F09BC21DED for ; Mon, 8 Jul 2019 19:21:14 +0000 (UTC) Received: by mail-io1-f65.google.com with SMTP id f4so22332503ioh.6 for ; Mon, 08 Jul 2019 12:21:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XB0yRLFuxPUjvsyYD8D++5UpEfMR/6wV4Jof14kZpqY=; b=VL90FEhA2T4XWEDDkIYzJEfE4cyr7XZf1OkmQsgfAgCz4mPxSndlQHa4imSFUFZH7l LMCt7dBXxmOR7HHbzHg7kSLMr05vsONilTFd7eNxYw/ig+cWXXQHGtLGHHMmLIujFmUW K7tl385U6yPdoacWsNuLRm9zjsiJHbi5bnUBw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XB0yRLFuxPUjvsyYD8D++5UpEfMR/6wV4Jof14kZpqY=; b=cUVcXwxKekpvOUd1P9A9CGzrwCTRsABqtqVV/MrIVrFSyRm1J0MNpWVqC6S4jpKhW3 dmIqGrd6dq5RWiKSHmpJN6KAOIG4ISXECs8VZRH7JShLuoOQuoU8AYfAtbV1seXcriAE LfBP8p+PF2nCyJV3bsgL+RnfmfGNgbAUK5y5zWpGtpd8CXOOk5XWb5ULAu4J8Q7LX4lj J+tbqXAXhawUQZTwvjbfkstRy9KczaOCyZT/8egxRCLrHuT4NODldlPIW/Of58aAdlpj JZOlwRgz/h8yVNTzX2OUQizJ2xl4EtlfLOXWOI4mgbwU+3eP2AhZcJoFhAU/L2bequwg DdGg== X-Gm-Message-State: APjAAAVgNRa4jbECs/NeFDIeHVnnGFc+C1Rw1zV8pGE35KTK8nMDpBw/ I4uj3maBo5e1XCkzZjjgKrFXo+el37g= X-Google-Smtp-Source: APXvYqxJP/GDF0MCs5CfQqVTSTzDHyEy89ItgLtMhtTTk8eQ+8PeYPLDGHHAZFkXXc8h+fg5+xbcBQ== X-Received: by 2002:a02:8a:: with SMTP id 132mr22548313jaa.89.1562613673403; Mon, 08 Jul 2019 12:21:13 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.12 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:12 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:39 -0600 Message-Id: <20190708191856.138863-21-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 20/37] binman: Update entry.SetOffsetSize to be optional X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" At present this function always sets both the offset and the size of entries. But in some cases we want to set only one or the other, for example with the forthcoming ifwi entry, where we only set the offset. Update the function to handle this. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/bsection.py | 7 ++++--- tools/binman/entry.py | 16 ++++++++++++---- 2 files changed, 16 insertions(+), 7 deletions(-) diff --git a/tools/binman/bsection.py b/tools/binman/bsection.py index 49b8ef3e3e0..3e3d369d5e4 100644 --- a/tools/binman/bsection.py +++ b/tools/binman/bsection.py @@ -236,14 +236,15 @@ class Section(object): Args: name: Entry name to update - offset: New offset - size: New size + offset: New offset, or None to leave alone + size: New size, or None to leave alone """ entry = self._entries.get(name) if not entry: self._Raise("Unable to set offset/size for unknown entry '%s'" % name) - entry.SetOffsetSize(self._skip_at_start + offset, size) + entry.SetOffsetSize(self._skip_at_start + offset if offset else None, + size) def GetEntryOffsets(self): """Handle entries that want to set the offset/size of other entries diff --git a/tools/binman/entry.py b/tools/binman/entry.py index 7ead997e0fd..7356c49c626 100644 --- a/tools/binman/entry.py +++ b/tools/binman/entry.py @@ -368,13 +368,21 @@ class Entry(object): Dict: key: Entry type value: List containing position and size of the given entry - type. + type. Either can be None if not known """ return {} - def SetOffsetSize(self, pos, size): - self.offset = pos - self.size = size + def SetOffsetSize(self, offset, size): + """Set the offset and/or size of an entry + + Args: + offset: New offset, or None to leave alone + size: New size, or None to leave alone + """ + if offset is not None: + self.offset = offset + if size is not None: + self.size = size def SetImagePos(self, image_pos): """Set the position in the image From patchwork Mon Jul 8 19:18:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129298 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="SJB4+uyM"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jG4720lkz9sNH for ; Tue, 9 Jul 2019 05:39:35 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id EA23CC21C51; Mon, 8 Jul 2019 19:28:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=RCVD_IN_MSPIKE_H2, T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id A7BCEC21E88; Mon, 8 Jul 2019 19:21:41 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 8D08AC21E90; Mon, 8 Jul 2019 19:21:19 +0000 (UTC) Received: from mail-io1-f68.google.com (mail-io1-f68.google.com [209.85.166.68]) by lists.denx.de (Postfix) with ESMTPS id 658C9C21E13 for ; Mon, 8 Jul 2019 19:21:15 +0000 (UTC) Received: by mail-io1-f68.google.com with SMTP id o9so22256335iom.3 for ; Mon, 08 Jul 2019 12:21:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RJmZt2VfJa6yJOEAMzBsz+ebiawTytJTDbbaXIeaFP4=; b=SJB4+uyMWXUHyNEIHm6bgT49OCIer2glDQHrlBA2aH5Cxj6Df7aLR+VClpTIHXVsvT CzIVjI55qTJHUOFAB+0G1l3HW+Iek3djsQ6WybsKfSeqzoaSzPcdB/kccWG8g/23WA7D WGf/MMUNrL/FGkH0/SwtnAVYPx5ReSRb7/3KM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RJmZt2VfJa6yJOEAMzBsz+ebiawTytJTDbbaXIeaFP4=; b=TT/Rk0dIefJ2zuDldNLDU01Q94ib2XTPSX2Gwa9sZbJoKxbjOvzpaXLrDKOD+LCWQg /T8B6dul016m/T6GacAHhdyVYqpalCdI9t0uIc1m4O91t2QYViDZS4e125FXbRGdhO8/ PibiC6BYFKKC7cYbIewkI+cQBAKS1eYXuwGZV9YWC1v+EjSSEbYy1f4PZxeo29+20VCW INvwAJhvOci2OR99E/XrqxRx5yQzZFnWzaUDWi0y8t9T1VslWQVgPsZkVsjRoC7ZA4NS CXYtrsz3s00+M4EzZtLdo/xIqCOZPMtdM4iZ4RzckztFkwepJ2Gc5HuM30i4oqGGZaJF 2yrQ== X-Gm-Message-State: APjAAAVZAbADTQhWaLGqExuUsxJJf1BYKasTE1q5DZJfBLws+KJn3zhA XRDARwhNRtSqg2z9QT21Yy4Ge9Yz+3s= X-Google-Smtp-Source: APXvYqysR5sTfZ+oeTWV+OoC2cy+aUjwAzIRvos0hNxv6XJqSU35eGbTwC5mHbTrfv8wkzi1/2hxTQ== X-Received: by 2002:a5d:97d8:: with SMTP id k24mr268587ios.84.1562613674255; Mon, 08 Jul 2019 12:21:14 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.13 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:13 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:40 -0600 Message-Id: <20190708191856.138863-22-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 21/37] binman: Allow text directly in the node X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" At present text entries use an indirect method to specify the text to use, with a label pointing to the text itself. Allow the text to be directly written into the node. This is more convenient in cases where the text is constant. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/README.entries | 9 +++++++++ tools/binman/etype/text.py | 23 +++++++++++++++++++---- tools/binman/ftest.py | 2 +- tools/binman/test/066_text.dts | 5 +++++ 4 files changed, 34 insertions(+), 5 deletions(-) diff --git a/tools/binman/README.entries b/tools/binman/README.entries index 702fc9fda08..9cbdbbaadef 100644 --- a/tools/binman/README.entries +++ b/tools/binman/README.entries @@ -312,6 +312,8 @@ Properties / Entry arguments: that contains the string to place in the entry (actual name is the value of text-label): contains the string to place in the entry. + : The text to place in the entry (overrides the above mechanism). + This is useful when the text is constant. Example node: @@ -334,6 +336,13 @@ It is also possible to put the string directly in the node: message = "a message directly in the node" }; +or just: + + text { + size = <8>; + text = "some text directly in the node" + }; + The text is not itself nul-terminated. This can be achieved, if required, by setting the size of the entry to something larger than the text. diff --git a/tools/binman/etype/text.py b/tools/binman/etype/text.py index 9ee04d7c9d8..da1813a638e 100644 --- a/tools/binman/etype/text.py +++ b/tools/binman/etype/text.py @@ -22,6 +22,8 @@ class Entry_text(Entry): that contains the string to place in the entry (actual name is the value of text-label): contains the string to place in the entry. + : The text to place in the entry (overrides the above mechanism). + This is useful when the text is constant. Example node: @@ -44,15 +46,28 @@ class Entry_text(Entry): message = "a message directly in the node" }; + or just: + + text { + size = <8>; + text = "some text directly in the node" + }; + The text is not itself nul-terminated. This can be achieved, if required, by setting the size of the entry to something larger than the text. """ def __init__(self, section, etype, node): Entry.__init__(self, section, etype, node) - label, = self.GetEntryArgsOrProps([EntryArg('text-label', str)]) - self.text_label = tools.ToStr(label) if type(label) != str else label - value, = self.GetEntryArgsOrProps([EntryArg(self.text_label, str)]) - value = tools.ToBytes(value) if value is not None else value + value = fdt_util.GetString(self._node, 'text') + if value: + value = tools.ToBytes(value) + else: + label, = self.GetEntryArgsOrProps([EntryArg('text-label', str)]) + self.text_label = label + if self.text_label: + value, = self.GetEntryArgsOrProps([EntryArg(self.text_label, + str)]) + value = tools.ToBytes(value) if value is not None else value self.value = value def ObtainContents(self): diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 8577adb5380..c74e12d13c8 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -1286,7 +1286,7 @@ class TestFunctional(unittest.TestCase): expected = (tools.ToBytes(TEXT_DATA) + tools.GetBytes(0, 8 - len(TEXT_DATA)) + tools.ToBytes(TEXT_DATA2) + tools.ToBytes(TEXT_DATA3) + - b'some text') + b'some text' + b'more text') self.assertEqual(expected, data) def testEntryDocs(self): diff --git a/tools/binman/test/066_text.dts b/tools/binman/test/066_text.dts index 59b1fed0ef8..f23a75ae929 100644 --- a/tools/binman/test/066_text.dts +++ b/tools/binman/test/066_text.dts @@ -24,5 +24,10 @@ text-label = "test-id4"; test-id4 = "some text"; }; + /* Put text directly in the node */ + text5 { + type = "text"; + text = "more text"; + }; }; }; From patchwork Mon Jul 8 19:18:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129296 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Xt8LZfEq"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jG3j3Nskz9sDB for ; Tue, 9 Jul 2019 05:39:13 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 2AC87C21DD7; Mon, 8 Jul 2019 19:29:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 1A2F6C21C4A; Mon, 8 Jul 2019 19:21:55 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id E802FC21DB5; Mon, 8 Jul 2019 19:21:19 +0000 (UTC) Received: from mail-io1-f67.google.com (mail-io1-f67.google.com [209.85.166.67]) by lists.denx.de (Postfix) with ESMTPS id 06B20C21DA2 for ; Mon, 8 Jul 2019 19:21:16 +0000 (UTC) Received: by mail-io1-f67.google.com with SMTP id o9so22256393iom.3 for ; Mon, 08 Jul 2019 12:21:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1QwmBlbnLMSlROy0CVKI64bsclTTy0fGW0z6/gq1mDE=; b=Xt8LZfEq/5dIEJvmH2AbSFW7v5xrxx7n7v65DbzZuWlNQkoAJFs+iSunf6KfURbS5V iF1ZfSk3W61YbvX/yrdxsEY56QUJwT1bscZPAzL4pXtJw5ULtKQpS1tanwPnigKWA5fr yWqglPp2RYeTgMtyy9zY1iumno4qbJ7Z5XZ/0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1QwmBlbnLMSlROy0CVKI64bsclTTy0fGW0z6/gq1mDE=; b=rS306t1d5Xzzi5sv74wEpB0+jgT3hWfKFd3zUAOJ3RnCYCfFAPlGMKrki2nfAmXBfR FVtq4EZZDzj8WiC/TyNJiQT7JSa32pg1eULhmZGTzwXjEC9FXFp4uGoUgZY+HRf/Dzno WOlm0pbplBzi8X4+w1rA3ZSqDW9MrIcpyNDWYalVuGUE7YKiRAMhh+K9Ub8LlfRRgnTa FfRS255ZcDWxri4VSBu+xEEYRXiVXqQ17U7WuP2g5mI5N8ljnF1bnuw8w8Uz4mjsAQDI oUvFdT53/bHLd4VVS+mv49dHsURw+FgMHs+qo297zsZjAaXdBZBnzqHqmp8gzxgqlfS7 Ljew== X-Gm-Message-State: APjAAAWVuP6WQqOojbs2KygTMLFGokuGs9rr7gbseSoteCyeMoNWpuBS BMz/Ae7EaWwYdmW94Jlmd3lRHLc3AZM= X-Google-Smtp-Source: APXvYqynoP+DDk2HhOlLNRN0dGiH/dVcQQPXFo7X2l8Fur2abV9AlAOKKtS+Q5uPwyC6N9CMIZQclw== X-Received: by 2002:a02:ac09:: with SMTP id a9mr9577423jao.48.1562613674821; Mon, 08 Jul 2019 12:21:14 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.14 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:14 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:41 -0600 Message-Id: <20190708191856.138863-23-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 22/37] patman: Add functions to compress and decompress data X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" Add utility functions to compress and decompress using lz4 and lzma algorithms. In the latter case these use the legacy lzma support favoured by coreboot's CBFS. No tests are provided as these functions will be tested by the CBFS tests in a separate patch. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/README | 3 +- tools/patman/tools.py | 66 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 68 insertions(+), 1 deletion(-) diff --git a/tools/binman/README b/tools/binman/README index 0ff30ef6fd9..7eda244bbe2 100644 --- a/tools/binman/README +++ b/tools/binman/README @@ -183,7 +183,8 @@ Running binman First install prerequisites, e.g. - sudo apt-get install python-pyelftools python3-pyelftools + sudo apt-get install python-pyelftools python3-pyelftools lzma-alone \ + liblz4-tool Type: diff --git a/tools/patman/tools.py b/tools/patman/tools.py index 0b3049f91f4..69d03d38608 100644 --- a/tools/patman/tools.py +++ b/tools/patman/tools.py @@ -374,3 +374,69 @@ def ToBytes(string): if sys.version_info[0] >= 3: return string.encode('utf-8') return string + +def Compress(indata, algo): + """Compress some data using a given algorithm + + Note that for lzma this uses an old version of the algorithm, not that + provided by xz. + + This requires 'lz4' and 'lzma_alone' tools. It also requires an output + directory to be previously set up, by calling PrepareOutputDir(). + + Args: + indata: Input data to compress + algo: Algorithm to use ('none', 'gzip', 'lz4' or 'lzma') + + Returns: + Compressed data + """ + if algo == 'none': + return indata + fname = GetOutputFilename('%s.comp.tmp' % algo) + WriteFile(fname, indata) + if algo == 'lz4': + data = Run('lz4', '--no-frame-crc', '-c', fname, binary=True) + # cbfstool uses a very old version of lzma + elif algo == 'lzma': + outfname = GetOutputFilename('%s.comp.otmp' % algo) + Run('lzma_alone', 'e', fname, outfname, '-lc1', '-lp0', '-pb0', '-d8') + data = ReadFile(outfname) + elif algo == 'gzip': + data = Run('gzip', '-c', fname, binary=True) + else: + raise ValueError("Unknown algorithm '%s'" % algo) + return data + +def Decompress(indata, algo): + """Decompress some data using a given algorithm + + Note that for lzma this uses an old version of the algorithm, not that + provided by xz. + + This requires 'lz4' and 'lzma_alone' tools. It also requires an output + directory to be previously set up, by calling PrepareOutputDir(). + + Args: + indata: Input data to decompress + algo: Algorithm to use ('none', 'gzip', 'lz4' or 'lzma') + + Returns: + Compressed data + """ + if algo == 'none': + return indata + fname = GetOutputFilename('%s.decomp.tmp' % algo) + with open(fname, 'wb') as fd: + fd.write(indata) + if algo == 'lz4': + data = Run('lz4', '-dc', fname, binary=True) + elif algo == 'lzma': + outfname = GetOutputFilename('%s.decomp.otmp' % algo) + Run('lzma_alone', 'd', fname, outfname) + data = ReadFile(outfname) + elif algo == 'gzip': + data = Run('gzip', '-cd', fname, binary=True) + else: + raise ValueError("Unknown algorithm '%s'" % algo) + return data From patchwork Mon Jul 8 19:18:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129297 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="n2HmnMHY"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jG3y0hC0z9sDB for ; Tue, 9 Jul 2019 05:39:26 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id CFC46C21D56; Mon, 8 Jul 2019 19:28:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 355C6C21E29; Mon, 8 Jul 2019 19:21:45 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 5B220C21E85; Mon, 8 Jul 2019 19:21:20 +0000 (UTC) Received: from mail-io1-f54.google.com (mail-io1-f54.google.com [209.85.166.54]) by lists.denx.de (Postfix) with ESMTPS id 986A3C21DB6 for ; Mon, 8 Jul 2019 19:21:16 +0000 (UTC) Received: by mail-io1-f54.google.com with SMTP id s7so37813223iob.11 for ; Mon, 08 Jul 2019 12:21:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=s1ig0QO3YezCnXj1lv2ah3WiSY75OKuhzaMokoWTgD8=; b=n2HmnMHYJev5QxZd3u7kMA6cqYn+Cg3sts9/MitVyLWbM2I/+27znGq4cHP39McNto fXUj52QsEP6UqW5+t/pkS8Mo9hK+1Mskj9nSqbwKwNPj3TJz0w/+78sArtrEaxeQmyID fyQCpXZHRoQ/124pNgrO/wLf3NG3FtK26BnUs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=s1ig0QO3YezCnXj1lv2ah3WiSY75OKuhzaMokoWTgD8=; b=VQIoonYipueRK/NOc4uHYG3IHjWSJjsWrbgtrfAxny7714WVGcM2e6tV1Wcm6/RGIs YLUMJgWohq71upZss8PFIo4xDFl1JwxdHjaxNMPlrxPiN8WVnZQPsrNIqX03RJ52LNWI ugHAyLa3YzByFGXKJ1qeKaiqL+Sfy1ufDzS2qrtOZl5Hj+koT+kFOS8PFCJVx5l+Ioiw vSuPZUl3q33zzBQ0pbtCqZwlFx0tIIjDwlUxza9ruIl2qxXqnaHmnt93ylDtbALan+na Q7BsCCEOOCTNVFoOd9EJp5K7jEWHt6NJ27qZsZt9GhEBXLiZEfLIWKTS5aUEJH7+p9uI 7Uww== X-Gm-Message-State: APjAAAWqYVa3L9M9E37RwmDJlVYM/ztarpuiKNIt2MYS4RW7SkOAYvgq J/euW+a2gfah2SFDaN4p3eMKN/Hhlnk= X-Google-Smtp-Source: APXvYqzlKRt/rsGKDopfQyGmCd14gOvHX+CXVRvGyiU3D2MXfWCmSH1jYHxFnWnKbrDV/qswCt8K0Q== X-Received: by 2002:a6b:b593:: with SMTP id e141mr21923010iof.203.1562613675515; Mon, 08 Jul 2019 12:21:15 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.14 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:15 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:42 -0600 Message-Id: <20190708191856.138863-24-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 23/37] binman: Use the tools.Decompress method X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" Update the compression test to use the tools module to decompress the output data. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/ftest.py | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index c74e12d13c8..6ff871b3c16 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -59,7 +59,7 @@ BMPBLK_DATA = b'bmp' VBLOCK_DATA = b'vblk' FILES_DATA = (b"sorry I'm late\nOh, don't bother apologising, I'm " + b"sorry you're alive\n") -COMPRESS_DATA = b'data to compress' +COMPRESS_DATA = b'compress xxxxxxxxxxxxxxxxxxxxxx data' REFCODE_DATA = b'refcode' @@ -1560,16 +1560,7 @@ class TestFunctional(unittest.TestCase): self._ResetDtbs() def _decompress(self, data): - out = os.path.join(self._indir, 'lz4.tmp') - with open(out, 'wb') as fd: - fd.write(data) - return tools.Run('lz4', '-dc', out, binary=True) - ''' - try: - orig = lz4.frame.decompress(data) - except AttributeError: - orig = lz4.decompress(data) - ''' + return tools.Decompress(data, 'lz4') def testCompress(self): """Test compression of blobs""" From patchwork Mon Jul 8 19:18:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129300 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="D+Erxv0d"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jG4f3PRgz9sDB for ; Tue, 9 Jul 2019 05:40:02 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id ED36CC21DD7; Mon, 8 Jul 2019 19:27:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 9CBE7C21DDC; Mon, 8 Jul 2019 19:21:37 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 6CE3EC21DDC; Mon, 8 Jul 2019 19:21:21 +0000 (UTC) Received: from mail-io1-f42.google.com (mail-io1-f42.google.com [209.85.166.42]) by lists.denx.de (Postfix) with ESMTPS id 56842C21E15 for ; Mon, 8 Jul 2019 19:21:17 +0000 (UTC) Received: by mail-io1-f42.google.com with SMTP id z3so22879572iog.0 for ; Mon, 08 Jul 2019 12:21:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZY/tjCOBLSktABTRs0V60v7iaNFQOwIUsFF5V/aXlss=; b=D+Erxv0duHs+BVz7f5w0JiWLfgqbtglmPYkS9TYOHBaVsLF9RPmXGQNr7l4tQhnxSy goPjuFA85YFcEbwrrQufwiKljqLbAOOoud9oQzn6PUSXb9yBV6zJO/LRtG7UialyL3AN Vu5zKr3Y1W8n1VQ1OtPIkzENCbmskagCDHfgo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZY/tjCOBLSktABTRs0V60v7iaNFQOwIUsFF5V/aXlss=; b=hKRFnRqhvSARUfFwBk9EiIPdC+yW83tplndV1XWhBN33QwAKZw6tLVbATTudJ5vj5X r5tNxYtrIca9jIx4t/cA78AdLi42rRUM0kOaYw7q1EuFpmObzxBnPKoiwOlry3nMyvFW vbXsrd0iWm1ts4qadyUrCI09OJoaoktserDTTCIcP3vysNaxDa6LzlX75Z+pxY8dqM05 ptguPSNU2dyk1KhDqY5KlDn74Ks+Kabzc0Odj1eWdqWcTn4C2MFjWwWchfVWTCnKYmX/ IM/uEfVuVma1KjlZ5by9BBiSd6ow0ltgHdFu6fRWoN/tO4AUJpZDfKWG4RiA6lu1QDYe wCxQ== X-Gm-Message-State: APjAAAXPMqCM3bLQqvsE54Z/1KKo7RD0kmvs85LP0Rz3ZDFgoSK6DSV8 MOWdSTXsmI++jZkCI/TsO3JJZRX4hXY= X-Google-Smtp-Source: APXvYqz1kOBS3/ppzUuST/i/qF/LN/6ubDxq4MhkqosVlhHDEXU1oCsrZe7AE75JxFU8d4JqN/DHkQ== X-Received: by 2002:a5d:9d58:: with SMTP id k24mr20385258iok.116.1562613676216; Mon, 08 Jul 2019 12:21:16 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.15 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:15 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:43 -0600 Message-Id: <20190708191856.138863-25-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 24/37] binman: Drop unnecessary debug handling X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" The -D option enables debug mode, but we only need to add -D to the command line once. Drop the duplicate code. Also drop the comment about enabling debugging since this can be done with -D. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/binman.py | 5 ----- 1 file changed, 5 deletions(-) diff --git a/tools/binman/binman.py b/tools/binman/binman.py index 05aeaecd8f3..bab98826dc6 100755 --- a/tools/binman/binman.py +++ b/tools/binman/binman.py @@ -71,8 +71,6 @@ def RunTests(debug, processes, args): sys.argv = [sys.argv[0]] if debug: sys.argv.append('-D') - if debug: - sys.argv.append('-D') # Run the entry tests first ,since these need to be the first to import the # 'entry' module. @@ -151,9 +149,6 @@ def RunBinman(options, args): """ ret_code = 0 - # For testing: This enables full exception traces. - #options.debug = True - if not options.debug: sys.tracebacklimit = 0 From patchwork Mon Jul 8 19:18:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129281 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="OEJvInwP"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFtF630fz9sNk for ; Tue, 9 Jul 2019 05:31:01 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 76906C21D56; Mon, 8 Jul 2019 19:29:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=RCVD_IN_MSPIKE_H2, T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 1AD5CC21EC8; Mon, 8 Jul 2019 19:21:53 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 5C7E0C21E1A; Mon, 8 Jul 2019 19:21:22 +0000 (UTC) Received: from mail-io1-f68.google.com (mail-io1-f68.google.com [209.85.166.68]) by lists.denx.de (Postfix) with ESMTPS id 6245BC21E0F for ; Mon, 8 Jul 2019 19:21:18 +0000 (UTC) Received: by mail-io1-f68.google.com with SMTP id f4so22332862ioh.6 for ; Mon, 08 Jul 2019 12:21:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=65N2gAuyIW9B3z61SDi9+Wg7t70RTTBH/fRuRc+JdLE=; b=OEJvInwPoD3vsrjf3G9Rz1HdC5QOEAVit9lVX2nkRfacE9q/Vl9jcfYMyEPSxAXc+s KirOlnh/cODPs2BladYduTmdJMAT9Q0MYFaLf/QaSOgATNbMHdDi6C2BmmN3vOYj9PcL Lcm/f5a75krpTOtM0Bjer+E7aiGdvdGKIRZ4c= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=65N2gAuyIW9B3z61SDi9+Wg7t70RTTBH/fRuRc+JdLE=; b=dcv00AHZYuBriKRJ9UnW5Kuya69DrRzK/jYy5vU6WrHNap/ZL9gXwOgVB4W1DJh5MH tsuBN6qv7Uhh1iHIXlbUEaAZUoFCPVjhK+dy/d92tyrjrwUrTcPqq9Vp76lAMIK/fzeS cnO8ikKaK7UYeCrV4CT8jnrl0sTGfunecko/XXCOjvaTkD/HPM/cisUEpmrqOXcPyLD8 Sm8POxPtih5v5dJ475CtM/rRO2aqsXQgSnetkK5MvUNV8rjqiPl3LRalmgXRNT1mKVwn 9HwYT4vrxfgFx6nndicipEMiWIdNJhseX1TDriBd78WFgMmjTsUZLdr4nVKLVUQke27h JmkQ== X-Gm-Message-State: APjAAAULUCehuo3/otkR+jC6pB90RLA57def5WIa5OrYYyUenHu1cYZS bQzcgs1vv7Fihi/FNEyvmOpPfmp6t68= X-Google-Smtp-Source: APXvYqwdzFptpMz6XlkdVd/nEdI2CcGyDyo2lGJhUqjnvWhgD5bnWFBKz8riVAInQrx0CTlEiH89+g== X-Received: by 2002:a05:6638:38a:: with SMTP id y10mr24418792jap.104.1562613677273; Mon, 08 Jul 2019 12:21:17 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.16 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:16 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:44 -0600 Message-Id: <20190708191856.138863-26-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 25/37] binman: Use tools compression function for blob handling X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" Avoid duplicate code here by using the new compression function in the tools module. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/etype/blob.py | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-) diff --git a/tools/binman/etype/blob.py b/tools/binman/etype/blob.py index f56a1f87688..a91e7847009 100644 --- a/tools/binman/etype/blob.py +++ b/tools/binman/etype/blob.py @@ -49,18 +49,10 @@ class Entry_blob(Entry): # new Entry method which can read in chunks. Then we could copy # the data in chunks and avoid reading it all at once. For now # this seems like an unnecessary complication. - data = tools.ReadFile(self._pathname) - if self._compress == 'lz4': - self._uncompressed_size = len(data) - ''' - import lz4 # Import this only if needed (python-lz4 dependency) - - try: - data = lz4.frame.compress(data) - except AttributeError: - data = lz4.compress(data) - ''' - data = tools.Run('lz4', '-c', self._pathname, binary=True) + indata = tools.ReadFile(self._pathname) + if self._compress != 'none': + self._uncompressed_size = len(indata) + data = tools.Compress(indata, self._compress) self.SetContents(data) return True From patchwork Mon Jul 8 19:18:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129283 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="HMN2b+2h"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFvP53rnz9sNH for ; Tue, 9 Jul 2019 05:32:01 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 50FC1C21E02; Mon, 8 Jul 2019 19:29:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id B9A85C21DD4; Mon, 8 Jul 2019 19:22:00 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id E4AB2C21EB4; Mon, 8 Jul 2019 19:21:23 +0000 (UTC) Received: from mail-io1-f65.google.com (mail-io1-f65.google.com [209.85.166.65]) by lists.denx.de (Postfix) with ESMTPS id 18670C21C93 for ; Mon, 8 Jul 2019 19:21:19 +0000 (UTC) Received: by mail-io1-f65.google.com with SMTP id k20so37785877ios.10 for ; Mon, 08 Jul 2019 12:21:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xQYbuOvXaV4/qh014rQ1KYra31L9GSNm+d9IMybz6Ok=; b=HMN2b+2hcmQ7nG6j2HmF7zpMovu0aPjNnU9FBeXJwhDgIk6EBaiLI4PFX7hotzlR1y yig9cUVZX6S5J0D79CErZPzaSkz53Bh5924waHViy5AQdGUf/xGOSMJ9r1LVnxrWomnq XEjJ3+qjo4bAkxd1H40NkC8NMTFJnCJWMEsGs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xQYbuOvXaV4/qh014rQ1KYra31L9GSNm+d9IMybz6Ok=; b=BFu9Ej56qKbLaXZ88G82IrXIS14KXMkx06OZO8fX9cTDP4USWKJK7AJpLznKZko5OF 20aSNz9rEXd6uHLwlN6Kz09XJtG0ImIVqjqmeCBG3A6I7GMJ2kkQoL1cTISXK+haN8PH T9RnFQqJHMppd/QTHzyOQlx1kF8Zvyr45NrOCJffkJ/ngtiEaMldUz/3wbWn3IuDfh3K wSobCu5s+xpFRq7ZY6v+iP6uh1a1vio7oohAxcfTmbRAHt8GxrHvlhvvNNsV1lqvU8s4 qaCL6hcCdQRAuCr/27E/AU65tul+y6xvmuuLvvlggxjloEYrBrVsKocCZ73/V+doS8dM UosA== X-Gm-Message-State: APjAAAWmSCF8+Eh5154TeMyECbKSsQ+inQuvcsTtIKsd39EGmQbBNfNc PxbmzIPZaD0hh4opxuSmo3GgZyLpe1I= X-Google-Smtp-Source: APXvYqyizD1eiJcFKVZ45qqm4IXJ9SKvJs05dNqs6VH0zP4ZLsD82u3vA5STuRClttxu4tYxz7dBmQ== X-Received: by 2002:a02:9f07:: with SMTP id z7mr22248138jal.29.1562613677988; Mon, 08 Jul 2019 12:21:17 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.17 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:17 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:45 -0600 Message-Id: <20190708191856.138863-27-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 26/37] binman: Correct comment in u_boot_spl_elf X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" This comment mentions the wrong default filename. Fix it. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/README.entries | 2 +- tools/binman/etype/u_boot_spl_elf.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/binman/README.entries b/tools/binman/README.entries index 9cbdbbaadef..c26addcfe64 100644 --- a/tools/binman/README.entries +++ b/tools/binman/README.entries @@ -496,7 +496,7 @@ Entry: u-boot-spl-elf: U-Boot SPL ELF image ------------------------------------------- Properties / Entry arguments: - - filename: Filename of SPL u-boot (default 'spl/u-boot') + - filename: Filename of SPL u-boot (default 'spl/u-boot-spl') This is the U-Boot SPL ELF image. It does not include a device tree but can be relocated to any address for execution. diff --git a/tools/binman/etype/u_boot_spl_elf.py b/tools/binman/etype/u_boot_spl_elf.py index da328ae15e1..24ee77237ed 100644 --- a/tools/binman/etype/u_boot_spl_elf.py +++ b/tools/binman/etype/u_boot_spl_elf.py @@ -12,7 +12,7 @@ class Entry_u_boot_spl_elf(Entry_blob): """U-Boot SPL ELF image Properties / Entry arguments: - - filename: Filename of SPL u-boot (default 'spl/u-boot') + - filename: Filename of SPL u-boot (default 'spl/u-boot-spl') This is the U-Boot SPL ELF image. It does not include a device tree but can be relocated to any address for execution. From patchwork Mon Jul 8 19:18:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129302 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="JMZ0VGD7"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jG5x5ZZrz9sDB for ; Tue, 9 Jul 2019 05:41:09 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 75520C21E38; Mon, 8 Jul 2019 19:29:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=RCVD_IN_MSPIKE_H2, T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 023A7C21E0F; Mon, 8 Jul 2019 19:21:57 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id B57ADC21E75; Mon, 8 Jul 2019 19:21:24 +0000 (UTC) Received: from mail-io1-f68.google.com (mail-io1-f68.google.com [209.85.166.68]) by lists.denx.de (Postfix) with ESMTPS id BE8B7C21D65 for ; Mon, 8 Jul 2019 19:21:19 +0000 (UTC) Received: by mail-io1-f68.google.com with SMTP id o9so22256810iom.3 for ; Mon, 08 Jul 2019 12:21:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OeQWxWEZ1N1OBg/HUFfXhQQYKnBZ9mvAl1QQhpzYay0=; b=JMZ0VGD7StJKTdtqaOFcGkWpoOs8UsucnMPSZp3Z8CtmIRYjEnM5YjcxfPGpgInUFx TMeqXbba9l5APk+ZITd4SmsQAFykFgsFJiW4QyiHtwmCEQ2G7c+05qhnJZ/hkK1qkbFr JeBlzCYHhGQvg/VcJ08PWxZERdoC85ZzhibX8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OeQWxWEZ1N1OBg/HUFfXhQQYKnBZ9mvAl1QQhpzYay0=; b=GGAd+tl87DrEJQ1nGgbypq2I5hI631SdulOsm9stT8rjE2SvoVaWo89W5LLq88nYnV pXeResx/KPdYza8t1EqlKkEXMSol3EccO4wWQQWHxhrTsJdaUYr2SNQfeyZhxl1c3va3 cNsb/aeCQCYH1L27pWxzPPlBC9Yc4SoWbKuGwvyCObLyJbl2RX3DtCzTMsJmZOvv2KWA 1xLi2ZtWmAAZidmzL41vtk7ZI0RIQF2MwM1BgPb9DHvir4hnWDR830G9n/0JT8LOOxHx IIvunQCgPJPX4q896S3mMgQy3nZA4Em3tcbkCQ2LQyo7bdh6ecZX/F+8dIIhQjMc+D8h 6LLw== X-Gm-Message-State: APjAAAUe/dyp21CcFyi6WTep7f8TE6fwkIu7Il0ygoXM5vniub1mD2or xF754JjGqLZS/XvE12w4wlhvskXA17Q= X-Google-Smtp-Source: APXvYqyChVv2SbBWKSkx1qor1vWZNmoaqoVp2wuEQ3uCyi5wWSuHYiPXrij5Wip/5qYA14SfXNk93Q== X-Received: by 2002:a02:3093:: with SMTP id q141mr23430174jaq.128.1562613678637; Mon, 08 Jul 2019 12:21:18 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.18 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:18 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:46 -0600 Message-Id: <20190708191856.138863-28-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 27/37] binman: Support ELF files for TPL X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" We currenty support using the ELF file in U-Boot proper and SPL, but not TPL. Add this as it is useful both with sandbox and for CBFS to allow adding TPL as a 'stage'. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/README.entries | 11 +++++++++++ tools/binman/etype/u_boot_tpl_elf.py | 24 ++++++++++++++++++++++++ tools/binman/ftest.py | 2 ++ tools/binman/test/096_elf.dts | 2 ++ 4 files changed, 39 insertions(+) create mode 100644 tools/binman/etype/u_boot_tpl_elf.py diff --git a/tools/binman/README.entries b/tools/binman/README.entries index c26addcfe64..9a316763ace 100644 --- a/tools/binman/README.entries +++ b/tools/binman/README.entries @@ -574,6 +574,17 @@ process. +Entry: u-boot-tpl-elf: U-Boot TPL ELF image +------------------------------------------- + +Properties / Entry arguments: + - filename: Filename of TPL u-boot (default 'tpl/u-boot-tpl') + +This is the U-Boot TPL ELF image. It does not include a device tree but can +be relocated to any address for execution. + + + Entry: u-boot-tpl-with-ucode-ptr: U-Boot TPL with embedded microcode pointer ---------------------------------------------------------------------------- diff --git a/tools/binman/etype/u_boot_tpl_elf.py b/tools/binman/etype/u_boot_tpl_elf.py new file mode 100644 index 00000000000..9cc1cc2c450 --- /dev/null +++ b/tools/binman/etype/u_boot_tpl_elf.py @@ -0,0 +1,24 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2018 Google, Inc +# Written by Simon Glass +# +# Entry-type module for U-Boot TPL ELF image +# + +from entry import Entry +from blob import Entry_blob + +class Entry_u_boot_tpl_elf(Entry_blob): + """U-Boot TPL ELF image + + Properties / Entry arguments: + - filename: Filename of TPL u-boot (default 'tpl/u-boot-tpl') + + This is the U-Boot TPL ELF image. It does not include a device tree but can + be relocated to any address for execution. + """ + def __init__(self, section, etype, node): + Entry_blob.__init__(self, section, etype, node) + + def GetDefaultFilename(self): + return 'tpl/u-boot-tpl' diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 6ff871b3c16..9cec5f42fa3 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -1745,6 +1745,8 @@ class TestFunctional(unittest.TestCase): def testElf(self): """Basic test of ELF entries""" self._SetupSplElf() + with open(self.TestFile('bss_data'), 'rb') as fd: + TestFunctional._MakeInputFile('tpl/u-boot-tpl', fd.read()) with open(self.TestFile('bss_data'), 'rb') as fd: TestFunctional._MakeInputFile('-boot', fd.read()) data = self._DoReadFile('096_elf.dts') diff --git a/tools/binman/test/096_elf.dts b/tools/binman/test/096_elf.dts index df3440c3194..8e3f3f15ef0 100644 --- a/tools/binman/test/096_elf.dts +++ b/tools/binman/test/096_elf.dts @@ -10,5 +10,7 @@ }; u-boot-spl-elf { }; + u-boot-tpl-elf { + }; }; }; From patchwork Mon Jul 8 19:18:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129287 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="cATR1Wez"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFwY1FPDz9sNH for ; Tue, 9 Jul 2019 05:33:01 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id D21DFC21E12; Mon, 8 Jul 2019 19:27:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id BC283C21E5B; Mon, 8 Jul 2019 19:21:39 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id C158EC21E5B; Mon, 8 Jul 2019 19:21:24 +0000 (UTC) Received: from mail-io1-f65.google.com (mail-io1-f65.google.com [209.85.166.65]) by lists.denx.de (Postfix) with ESMTPS id 59436C21D9A for ; Mon, 8 Jul 2019 19:21:20 +0000 (UTC) Received: by mail-io1-f65.google.com with SMTP id j5so18475164ioj.8 for ; Mon, 08 Jul 2019 12:21:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JAQK78BTlWcodS+S2rv5TP8xsF+ssKxkQQ20wAaEnq8=; b=cATR1Wezn0shyuRl0GsMgGRITsTZpL68Ttwi5eM0V/dyK86gRBgeclbWEF69LXzaJh Ww/uYGdc9UvVdcGbY0n1PwXLZBRfqgDy4pZodwDobcrQ0Y3xbmfAmxvdceD2FcrfAJVL GlzNN4jfjSTLytpdoqXBYgg8SimPPAyBcfGBo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JAQK78BTlWcodS+S2rv5TP8xsF+ssKxkQQ20wAaEnq8=; b=N6DK55U45ZB3AgogyUKpqglMbK8QXiNUFR+ErGD4KsWBX12MIEVtp14CBtnupIvT96 ucoTiAnz0TxHefUORmiJKC/BhKxDItHOSP1kARn9RlBt+BHBbLBHO/x4AQqvPAicg7SY zEz7MckQpluoEBwcxm8CTDJHO5ZAQVeyNI8sqNZzXkNQ57ukzVI2lIromuXFvknzKxB9 FDRPyzLyI6UV+IUBuypBWrZBip1RcBiOz29QKypsiuCBkF2f36EqAsW26YwdtwxWM4lf 5pAVgooMGyGJawttXyHr89t+Aji2qU0lHpffnekNwEzUVo5Mos0jef54l1KhvTWCO38c x7xw== X-Gm-Message-State: APjAAAU7uQGL7/lMLBMM/L1Q98ZJM+FPIFIb2KALGrBXjMKj4A5qR1Q/ 356Kxcjz2MlXxDmyJFhtPkaJLTtAieA= X-Google-Smtp-Source: APXvYqx9fZOha3Uwxz5p19eac6RQ8Uc+Oeg3MQJQHicnlcUN2eXou8ocPWIOrPqbGUy144u6bgaVnQ== X-Received: by 2002:a02:c519:: with SMTP id s25mr22286400jam.11.1562613679268; Mon, 08 Jul 2019 12:21:19 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.18 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:18 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:47 -0600 Message-Id: <20190708191856.138863-29-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 28/37] binman: Fix up the _DoTestFile() function -u argument X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" This should be -u, not -up, since we don't need to preserve the output directory in this case. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/ftest.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 9cec5f42fa3..b1780854cfe 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -208,7 +208,7 @@ class TestFunctional(unittest.TestCase): if map: args.append('-m') if update_dtb: - args.append('-up') + args.append('-u') if not use_real_dtb: args.append('--fake-dtb') if verbosity is not None: From patchwork Mon Jul 8 19:18:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129286 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="AZS9KGhw"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFwK5ng0z9sNH for ; Tue, 9 Jul 2019 05:32:49 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 7C69EC21C3F; Mon, 8 Jul 2019 19:26:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id C7E80C21DFA; Mon, 8 Jul 2019 19:21:33 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 076A6C21C93; Mon, 8 Jul 2019 19:21:24 +0000 (UTC) Received: from mail-io1-f45.google.com (mail-io1-f45.google.com [209.85.166.45]) by lists.denx.de (Postfix) with ESMTPS id 1DC90C21DF9 for ; Mon, 8 Jul 2019 19:21:21 +0000 (UTC) Received: by mail-io1-f45.google.com with SMTP id o9so22256953iom.3 for ; Mon, 08 Jul 2019 12:21:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bZ8VeAZ0T0YsD6VHmFS52eEfzIHOT7492V8vdKPhbcg=; b=AZS9KGhwOMMCHdI1NpHYcm6pw5w+CKVublNk+KKgjSYV65fVrIdEdUAuf3UTP6LPXS TCWTr6jB/ciwigRc4SRE9Xm+AmfLeDRt9iF6GYiBydHK07MBN+0D9chPNUeOBE+Z4GQv xbo+qzB4YlHKWvBSZV+GUpExuunfQFJwf7+4I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bZ8VeAZ0T0YsD6VHmFS52eEfzIHOT7492V8vdKPhbcg=; b=O854RMoJSeeb6bJTI9L/0TS8aX2iCO7Z0vH7BqBFNgefXCr70X37KUKlQkA+PfwnY6 T8SFMg9gtYBCjYdz2YLiH3sit9XVit8KDZoUBLJy+2TpTan4m/O1Ja8/8/su1KOPIGsu PYnQMVTvrsb5Z32ede7TqSaRZj5QPsSSwOKFJX0TLLsos4HZ/F6f8UFhO0dH3wsRDbbU YVpYjsXUU4np1XnUb6bzWd2EmoGTskPOzwYJgLX37ab1NgW5vT0RX2bdRLMqi87RKZRb w3tmx9MpkWEmYGF7gVKRe7su+/+gaYubnkPSftqzvCm/nlf6G5ztBEmst97w6kN3htGw JzEA== X-Gm-Message-State: APjAAAVp5pOeceOnRhsU4vHW9IYM/y4fLf9y9jtX3Fzp1qJPdEvfWFxB 7VyKB+OYV9fWAma7qKDSV5djaXWFTYA= X-Google-Smtp-Source: APXvYqxn7Ty5BJUs0O6m2yH3UGkn39Wk+a6q9e74mdJ0G2vcLA0ajUZLpnMccOwbwwaVrBepTqU/Gg== X-Received: by 2002:a02:ad15:: with SMTP id s21mr24069195jan.47.1562613679997; Mon, 08 Jul 2019 12:21:19 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.19 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:19 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:48 -0600 Message-Id: <20190708191856.138863-30-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 29/37] binman: Allow verbosity control when running tests X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" At present the -v flag is ignored with tests, so that (for example) -v2 does not have any effect. Update binman to pass this flag through to tests so that they work just like running binman normally, except in a few special cases where we are actually testing behaviour with different levels of verbosity. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/binman.py | 8 ++++++-- tools/binman/ftest.py | 17 ++++++++++++++++- 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/tools/binman/binman.py b/tools/binman/binman.py index bab98826dc6..7c1dcfb65fc 100755 --- a/tools/binman/binman.py +++ b/tools/binman/binman.py @@ -46,11 +46,12 @@ except: import control import test_util -def RunTests(debug, processes, args): +def RunTests(debug, verbosity, processes, args): """Run the functional tests and any embedded doctests Args: debug: True to enable debugging, which shows a full stack trace on error + verbosity: Verbosity level to use args: List of positional args provided to binman. This can hold a test name to execute (as in 'binman -t testSections', for example) processes: Number of processes to use to run tests (None=same as #CPUs) @@ -71,6 +72,8 @@ def RunTests(debug, processes, args): sys.argv = [sys.argv[0]] if debug: sys.argv.append('-D') + if verbosity: + sys.argv.append('-v%d' % verbosity) # Run the entry tests first ,since these need to be the first to import the # 'entry' module. @@ -153,7 +156,8 @@ def RunBinman(options, args): sys.tracebacklimit = 0 if options.test: - ret_code = RunTests(options.debug, options.processes, args[1:]) + ret_code = RunTests(options.debug, options.verbosity, options.processes, + args[1:]) elif options.test_coverage: RunTestCoverage() diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index b1780854cfe..f5e0b9b9742 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -153,6 +153,19 @@ class TestFunctional(unittest.TestCase): TestFunctional._MakeInputFile('spl/u-boot-spl.dtb', U_BOOT_SPL_DTB_DATA) TestFunctional._MakeInputFile('tpl/u-boot-tpl.dtb', U_BOOT_TPL_DTB_DATA) + def _GetVerbosity(self): + """Check if verbosity should be enabled + + Returns: + list containing either: + - Verbosity flag (e.g. '-v2') if it is present on the cmd line + - nothing if the flag is not present + """ + for arg in sys.argv[1:]: + if arg.startswith('-v'): + return [arg] + return [] + def _RunBinman(self, *args, **kwargs): """Run binman using the command line @@ -213,6 +226,8 @@ class TestFunctional(unittest.TestCase): args.append('--fake-dtb') if verbosity is not None: args.append('-v%d' % verbosity) + else: + args += self._GetVerbosity() if entry_args: for arg, value in entry_args.items(): args.append('-a%s=%s' % (arg, value)) @@ -1471,7 +1486,7 @@ class TestFunctional(unittest.TestCase): expected = 'Skipping images: image1' # We should only get the expected message in verbose mode - for verbosity in (None, 2): + for verbosity in (0, 2): with test_util.capture_sys_output() as (stdout, stderr): retcode = self._DoTestFile('006_dual_image.dts', verbosity=verbosity, From patchwork Mon Jul 8 19:18:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129284 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="BURK4cXN"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFvn5PSkz9sDB for ; Tue, 9 Jul 2019 05:32:21 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 661F6C21DDC; Mon, 8 Jul 2019 19:27:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 3A0B0C21C51; Mon, 8 Jul 2019 19:21:41 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 2D605C21E7F; Mon, 8 Jul 2019 19:21:26 +0000 (UTC) Received: from mail-io1-f50.google.com (mail-io1-f50.google.com [209.85.166.50]) by lists.denx.de (Postfix) with ESMTPS id D99C9C21DF8 for ; Mon, 8 Jul 2019 19:21:21 +0000 (UTC) Received: by mail-io1-f50.google.com with SMTP id q22so16029535iog.4 for ; Mon, 08 Jul 2019 12:21:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ycwjNOSW5hV5OgJnto3yKTM2jTr8VGafdLaoDtPQRgk=; b=BURK4cXNk0vrYJYQ/UCyL7JrbmHmdSaYp4k/SWOknDCmKc4neaHmBB8jlGwAAJLR2j 2rQzL5BfKrk0XdKQw0wapXE1c3YL3chsgIyg/oERCKHq/FCRLaMtAZwjT4TMGbdSvxiS dM36wMu5N1Ypqt+9bckHK7ktr34RIg3OTxSQg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ycwjNOSW5hV5OgJnto3yKTM2jTr8VGafdLaoDtPQRgk=; b=BTSI72PvqjH14aQnjCTknaKyzcaPcCe2tHdetNIQBSp8DKcs4ALxqDvfCnEXoqoVDi uDzuf1uplOo/gDzfDR+Dc/c8wFTt03zdmiM1yPDrCgkV2g2PZnSfSQeeA11m0woXyxg7 ZyZOmbyHfyD5Vx3QNwRvPG4DP16sZXYV2YA29fbw50x1wc93+v+mZsFLM+w4Bzeld0xN nB3SeYLF/SzQIojYyXJr1W0OKnp9fps+UwDLhK/roic4YLKyiLrVcHarx2+PIYTKJcG9 uvZ8lbnx1ppNVcyrI2caIXn5tAADBq9sEkiTPZy8OsaaT7/8rI7XKNRpDtrg3dHFEHkZ c40g== X-Gm-Message-State: APjAAAVnkjDKpi6vcRPWe+U8K0IjD2PwIQWSJydkO9MGcatbkh59S+gz XGq0vAiGvck12FSAvyUoEuAYtTENlVQ= X-Google-Smtp-Source: APXvYqxf9sC+Ym1BRnBgVdm6c1BW9VZBVPaMyKEvoy01Yf0/TNASLzNFpz0YimvNoJmncVgI2ufWUA== X-Received: by 2002:a02:c492:: with SMTP id t18mr23469610jam.67.1562613680658; Mon, 08 Jul 2019 12:21:20 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.20 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:20 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:49 -0600 Message-Id: <20190708191856.138863-31-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 30/37] binman: Allow preserving test directories X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" Sometimes when debugging tests it is useful to keep the input and output directories so they can be examined later. Add an option for this and update the binman tests to support it. This affects both the test class and the tearDown() function called after each test. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/README | 8 ++++++++ tools/binman/binman.py | 16 +++++++++++++--- tools/binman/cmdline.py | 4 ++++ tools/binman/ftest.py | 28 +++++++++++++++++++++++++--- 4 files changed, 50 insertions(+), 6 deletions(-) diff --git a/tools/binman/README b/tools/binman/README index 7eda244bbe2..2f4c7fec21e 100644 --- a/tools/binman/README +++ b/tools/binman/README @@ -731,6 +731,14 @@ Use '-P 1' to disable this. It is automatically disabled when code coverage is being used (-T) since they are incompatible. +Debugging tests +--------------- + +Sometimes when debugging tests it is useful to keep the input and output +directories so they can be examined later. Use -X or --test-preserve-dirs for +this. + + Advanced Features / Technical docs ---------------------------------- diff --git a/tools/binman/binman.py b/tools/binman/binman.py index 7c1dcfb65fc..9878eb86d4f 100755 --- a/tools/binman/binman.py +++ b/tools/binman/binman.py @@ -46,15 +46,20 @@ except: import control import test_util -def RunTests(debug, verbosity, processes, args): +def RunTests(debug, verbosity, processes, test_preserve_dirs, args): """Run the functional tests and any embedded doctests Args: debug: True to enable debugging, which shows a full stack trace on error verbosity: Verbosity level to use + test_preserve_dirs: True to preserve the input directory used by tests + so that it can be examined afterwards (only useful for debugging + tests). If a single test is selected (in args[0]) it also preserves + the output directory for this test. Both directories are displayed + on the command line. + processes: Number of processes to use to run tests (None=same as #CPUs) args: List of positional args provided to binman. This can hold a test name to execute (as in 'binman -t testSections', for example) - processes: Number of processes to use to run tests (None=same as #CPUs) """ import elf_test import entry_test @@ -82,6 +87,11 @@ def RunTests(debug, verbosity, processes, args): loader = unittest.TestLoader() for module in (entry_test.TestEntry, ftest.TestFunctional, fdt_test.TestFdt, elf_test.TestElf, image_test.TestImage): + # Test the test module about our arguments, if it is interested + if hasattr(module, 'setup_test_args'): + setup_test_args = getattr(module, 'setup_test_args') + setup_test_args(preserve_indir=test_preserve_dirs, + preserve_outdirs=test_preserve_dirs and test_name is not None) if test_name: try: suite.addTests(loader.loadTestsFromName(test_name, module)) @@ -157,7 +167,7 @@ def RunBinman(options, args): if options.test: ret_code = RunTests(options.debug, options.verbosity, options.processes, - args[1:]) + options.test_preserve_dirs, args[1:]) elif options.test_coverage: RunTestCoverage() diff --git a/tools/binman/cmdline.py b/tools/binman/cmdline.py index 39b835666ea..91e007e4e03 100644 --- a/tools/binman/cmdline.py +++ b/tools/binman/cmdline.py @@ -59,6 +59,10 @@ def ParseArgs(argv): parser.add_option('-v', '--verbosity', default=1, type='int', help='Control verbosity: 0=silent, 1=progress, 3=full, ' '4=debug') + parser.add_option('-X', '--test-preserve-dirs', action='store_true', + help='Preserve and display test-created input directories; also ' + 'preserve the output directory if a single test is run (pass test ' + 'name at the end of the command line') parser.usage += """ diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index f5e0b9b9742..256d4a1c5d8 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -6,6 +6,8 @@ # # python -m unittest func_test.TestFunctional.testHelp +from __future__ import print_function + import hashlib from optparse import OptionParser import os @@ -134,10 +136,27 @@ class TestFunctional(unittest.TestCase): @classmethod def tearDownClass(self): """Remove the temporary input directory and its contents""" - if self._indir: - shutil.rmtree(self._indir) + if self.preserve_indir: + print('Preserving input dir: %s' % self._indir) + else: + if self._indir: + shutil.rmtree(self._indir) self._indir = None + @classmethod + def setup_test_args(cls, preserve_indir=False, preserve_outdirs=False): + """Accept arguments controlling test execution + + Args: + preserve_indir: Preserve the shared input directory used by all + tests in this class. + preserve_outdir: Preserve the output directories used by tests. Each + test has its own, so this is normally only useful when running a + single test. + """ + cls.preserve_indir = preserve_indir + cls.preserve_outdirs = preserve_outdirs + def setUp(self): # Enable this to turn on debugging output # tout.Init(tout.DEBUG) @@ -145,7 +164,10 @@ class TestFunctional(unittest.TestCase): def tearDown(self): """Remove the temporary output directory""" - tools._FinaliseForTest() + if self.preserve_outdirs: + print('Preserving output dir: %s' % tools.outdir) + else: + tools._FinaliseForTest() @classmethod def _ResetDtbs(self): From patchwork Mon Jul 8 19:18:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129290 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="DXOowVJX"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFzW1Q6qz9sNH for ; Tue, 9 Jul 2019 05:35:35 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id DB225C21E0D; Mon, 8 Jul 2019 19:26:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 16BEDC21DF8; Mon, 8 Jul 2019 19:21:37 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 24CF4C21E3B; Mon, 8 Jul 2019 19:21:26 +0000 (UTC) Received: from mail-io1-f50.google.com (mail-io1-f50.google.com [209.85.166.50]) by lists.denx.de (Postfix) with ESMTPS id 8138EC21E31 for ; Mon, 8 Jul 2019 19:21:22 +0000 (UTC) Received: by mail-io1-f50.google.com with SMTP id o9so22257099iom.3 for ; Mon, 08 Jul 2019 12:21:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TospjvykXdj3eLPTCiu4/aV7JYgWQkWiWpk7x5asgl8=; b=DXOowVJXFoWE7P/gx5YvUHDI+znmP88O9cauz5ZjC79bz8sYc4wiZIgl3+GVhV6Q1u Ju/wCx9rkzDVshkhOO6TM24ZpxQvrnov5atu5PHb3OleIqC8On+aSh3BH9ierV3eiqtO LaBBQC8oT20FLLlXaxXQyc75HPySFTjFEN43Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TospjvykXdj3eLPTCiu4/aV7JYgWQkWiWpk7x5asgl8=; b=cYkCIA2obnlJ3j06s0gSClqxCZwashr1dTyuxOdcZ3AYANG+X+06xd8OJYMA2+zPGa 5tvjwins1BlFf1lJIxqOj0nvdxLN7p9SOYchU1N4UF6pL2MTwOFT+sgfcm0bfi5e1cIn JEpg9NNG0qMmxyIUNr90NhKO1Aaxtnqa7k0YkcgV+HLhxEiMl+L7UjcI4QzfNHuD2MRB eVpfBRUAU453wbb/XxhovBALWXAzHBnnsZNLPo5buaX7PLYXQiMfk7lb8AIFMCMy8bsJ 8abucZY4T2I/j3DA66QqlpwqiMlArYJ4kel1wWsBUQeUD77PqHpr4TicnQ15wJJnYL+q AU8g== X-Gm-Message-State: APjAAAU+9edQKXaJFzjPnQK8e/7U54E5dWp1Rr2q0PQOnvYyt710ugtB d863B3cSbsM9qaa77idVONchOTWZoPM= X-Google-Smtp-Source: APXvYqxDkDydUv9rpRsuaS0sgHqkk/35cps4hgdTRFb81jxVhHugufujM3SRbICfEILmSCDcB8wvxw== X-Received: by 2002:a02:6d24:: with SMTP id m36mr23424139jac.87.1562613681311; Mon, 08 Jul 2019 12:21:21 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.20 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:20 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:50 -0600 Message-Id: <20190708191856.138863-32-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 31/37] binman: Pass the toolpath to tests X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" Tools like ifwitool may not be available in the PATH, but are available in the build. These tools may be needed by tests, so allow tests to use the --toolpath flag. Also use this flag with travis. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None .travis.yml | 2 +- test/run | 9 ++++++--- tools/binman/binman.py | 12 +++++++++--- tools/binman/ftest.py | 8 +++++++- 4 files changed, 23 insertions(+), 8 deletions(-) diff --git a/.travis.yml b/.travis.yml index 6662ca126ab..c883198abbf 100644 --- a/.travis.yml +++ b/.travis.yml @@ -146,7 +146,7 @@ script: if [[ -n "${TEST_PY_TOOLS}" ]]; then PYTHONPATH="${UBOOT_TRAVIS_BUILD_DIR}/scripts/dtc/pylibfdt" PATH="${UBOOT_TRAVIS_BUILD_DIR}/scripts/dtc:${PATH}" - ./tools/binman/binman -t && + ./tools/binman/binman --toolpath ${UBOOT_TRAVIS_BUILD_DIR}/tools -t && ./tools/patman/patman --test && ./tools/buildman/buildman -t && PYTHONPATH="${UBOOT_TRAVIS_BUILD_DIR}/scripts/dtc/pylibfdt" diff --git a/test/run b/test/run index 55a6649a9c5..b97647eba6f 100755 --- a/test/run +++ b/test/run @@ -33,12 +33,14 @@ run_test "sandbox_flattree" ./test/py/test.py --bd sandbox_flattree --build \ -k test_ut # Set up a path to dtc (device-tree compiler) and libfdt.py, a library it -# provides and which is built by the sandbox_spl config. +# provides and which is built by the sandbox_spl config. Also set up the path +# to tools build by the build. DTC_DIR=build-sandbox_spl/scripts/dtc export PYTHONPATH=${DTC_DIR}/pylibfdt export DTC=${DTC_DIR}/dtc +TOOLS_DIR=build-sandbox_spl/tools -run_test "binman" ./tools/binman/binman -t +run_test "binman" ./tools/binman/binman -t --toolpath ${TOOLS_DIR} run_test "patman" ./tools/patman/patman --test [ "$1" == "quick" ] && skip=--skip-net-tests @@ -49,7 +51,8 @@ run_test "dtoc" ./tools/dtoc/dtoc -t # This needs you to set up Python test coverage tools. # To enable Python test coverage on Debian-type distributions (e.g. Ubuntu): # $ sudo apt-get install python-pytest python-coverage -run_test "binman code coverage" ./tools/binman/binman -T +export PATH=$PATH:${TOOLS_DIR} +run_test "binman code coverage" ./tools/binman/binman -T --toolpath ${TOOLS_DIR} run_test "dtoc code coverage" ./tools/dtoc/dtoc -T run_test "fdt code coverage" ./tools/dtoc/test_fdt -T diff --git a/tools/binman/binman.py b/tools/binman/binman.py index 9878eb86d4f..52c03f68c6d 100755 --- a/tools/binman/binman.py +++ b/tools/binman/binman.py @@ -46,7 +46,7 @@ except: import control import test_util -def RunTests(debug, verbosity, processes, test_preserve_dirs, args): +def RunTests(debug, verbosity, processes, test_preserve_dirs, args, toolpath): """Run the functional tests and any embedded doctests Args: @@ -60,6 +60,7 @@ def RunTests(debug, verbosity, processes, test_preserve_dirs, args): processes: Number of processes to use to run tests (None=same as #CPUs) args: List of positional args provided to binman. This can hold a test name to execute (as in 'binman -t testSections', for example) + toolpath: List of paths to use for tools """ import elf_test import entry_test @@ -79,6 +80,9 @@ def RunTests(debug, verbosity, processes, test_preserve_dirs, args): sys.argv.append('-D') if verbosity: sys.argv.append('-v%d' % verbosity) + if toolpath: + for path in toolpath: + sys.argv += ['--toolpath', path] # Run the entry tests first ,since these need to be the first to import the # 'entry' module. @@ -91,7 +95,8 @@ def RunTests(debug, verbosity, processes, test_preserve_dirs, args): if hasattr(module, 'setup_test_args'): setup_test_args = getattr(module, 'setup_test_args') setup_test_args(preserve_indir=test_preserve_dirs, - preserve_outdirs=test_preserve_dirs and test_name is not None) + preserve_outdirs=test_preserve_dirs and test_name is not None, + toolpath=toolpath) if test_name: try: suite.addTests(loader.loadTestsFromName(test_name, module)) @@ -167,7 +172,8 @@ def RunBinman(options, args): if options.test: ret_code = RunTests(options.debug, options.verbosity, options.processes, - options.test_preserve_dirs, args[1:]) + options.test_preserve_dirs, args[1:], + options.toolpath) elif options.test_coverage: RunTestCoverage() diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 256d4a1c5d8..3455b8ccebd 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -144,7 +144,8 @@ class TestFunctional(unittest.TestCase): self._indir = None @classmethod - def setup_test_args(cls, preserve_indir=False, preserve_outdirs=False): + def setup_test_args(cls, preserve_indir=False, preserve_outdirs=False, + toolpath=None): """Accept arguments controlling test execution Args: @@ -153,9 +154,11 @@ class TestFunctional(unittest.TestCase): preserve_outdir: Preserve the output directories used by tests. Each test has its own, so this is normally only useful when running a single test. + toolpath: ist of paths to use for tools """ cls.preserve_indir = preserve_indir cls.preserve_outdirs = preserve_outdirs + cls.toolpath = toolpath def setUp(self): # Enable this to turn on debugging output @@ -256,6 +259,9 @@ class TestFunctional(unittest.TestCase): if images: for image in images: args += ['-i', image] + if self.toolpath: + for path in self.toolpath: + args += ['--toolpath', path] return self._DoBinman(*args) def _SetupDtb(self, fname, outfile='u-boot.dtb'): From patchwork Mon Jul 8 19:18:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129285 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="j3i2hn7a"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jFvy63wrz9sDB for ; Tue, 9 Jul 2019 05:32:30 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 7E9DDC21E02; Mon, 8 Jul 2019 19:27:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 29AE2C21E53; Mon, 8 Jul 2019 19:21:39 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 22E0DC21C51; Mon, 8 Jul 2019 19:21:27 +0000 (UTC) Received: from mail-io1-f49.google.com (mail-io1-f49.google.com [209.85.166.49]) by lists.denx.de (Postfix) with ESMTPS id 2C4F3C21E38 for ; Mon, 8 Jul 2019 19:21:23 +0000 (UTC) Received: by mail-io1-f49.google.com with SMTP id m24so28368892ioo.2 for ; Mon, 08 Jul 2019 12:21:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qjFdlHlCeohCy1OBCS5TxhxSxryt/wsMo6V60jVFN0Q=; b=j3i2hn7aKn/9JLl9A3yKWHjWexpil8qtfGCKTIbbcVFbnLZfrF/5JNCJ/oYfus1hSv T0ylCAw6Y1aI6C8RoShM3nIs6gtP7njdJ4TJfOw4Fg+8hWoLMKROo9xV3pHEpfl/QG+l 1aUJbNX7z90Qhz4VoONn64/TKLu7Sgs4bJzFA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qjFdlHlCeohCy1OBCS5TxhxSxryt/wsMo6V60jVFN0Q=; b=CnCgk47cILAXxbDBXfTSQJ3i8XsAQEvr9ATm4CnSB35EfXcMaMU/Ju5aLlB8WhSFMc EJJO8XVyMDx3vIuiW6CKrHtHVzc+42F/cScnA8CNFoI/60CIKalL8Jbk9jUpKE24N4/q MruHopJuMcXU0W7mmjhlMJpyZqHbIFfe0kY8T57TaTAdskUDWjjo+0wkxFmm7mtgSHV5 Waz6fEDYN5RqzoCyby/4IoOGbd+AooDcqg4DX3UvqS7vD+XVOUD9AjRBhQmrkMM7qwys RotWfgzRp2qrVDtXsNusKwZyNLWtOGqt2XkLorCIjABNpzmVm/E6IIIkCAdPMxW9vm1e Nv1w== X-Gm-Message-State: APjAAAXL7nhzyQmbJAUTRHw27c8MNo4aqyPf75PrR/wHaClYD32B1q8s g+6kWaXVtT2mR2dAtW7IPLDg039n50o= X-Google-Smtp-Source: APXvYqyfqEU3jl3+ripko0Kz40mcpWAQTtnU5jZouOtRZDY6+31FkHgapnf99X/gZuFHELS9F6x1/A== X-Received: by 2002:a5d:890d:: with SMTP id b13mr16294062ion.124.1562613681996; Mon, 08 Jul 2019 12:21:21 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.21 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:21 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:51 -0600 Message-Id: <20190708191856.138863-33-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 32/37] patman: Add a function to write ifwitool X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" This tool has quite a few arguments and options, so put the functionality in a function so that we call it from one place and hopefully get it right. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/patman/tools.py | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/tools/patman/tools.py b/tools/patman/tools.py index 69d03d38608..e945b54fa28 100644 --- a/tools/patman/tools.py +++ b/tools/patman/tools.py @@ -3,6 +3,8 @@ # Copyright (c) 2016 Google, Inc # +from __future__ import print_function + import command import glob import os @@ -440,3 +442,34 @@ def Decompress(indata, algo): else: raise ValueError("Unknown algorithm '%s'" % algo) return data + +CMD_CREATE, CMD_DELETE, CMD_ADD, CMD_REPLACE, CMD_EXTRACT = range(5) + +IFWITOOL_CMDS = { + CMD_CREATE: 'create', + CMD_DELETE: 'delete', + CMD_ADD: 'add', + CMD_REPLACE: 'replace', + CMD_EXTRACT: 'extract', + } + +def RunIfwiTool(ifwi_file, cmd, fname=None, subpart=None, entry_name=None): + """Run ifwitool with the given arguments: + + Args: + ifwi_file: IFWI file to operation on + cmd: Command to execute (CMD_...) + fname: Filename of file to add/replace/extract/create (None for + CMD_DELETE) + subpart: Name of sub-partition to operation on (None for CMD_CREATE) + entry_name: Name of directory entry to operate on, or None if none + """ + args = ['ifwitool', ifwi_file] + args.append(IFWITOOL_CMDS[cmd]) + if fname: + args += ['-f', fname] + if subpart: + args += ['-n', subpart] + if entry_name: + args += ['-d', '-e', entry_name] + Run(*args) From patchwork Mon Jul 8 19:18:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129295 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="i1U3lBhz"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jG3L5CPzz9sDB for ; Tue, 9 Jul 2019 05:38:54 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 66191C21E31; Mon, 8 Jul 2019 19:30:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 3C73CC21EE7; Mon, 8 Jul 2019 19:22:08 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id B2CB2C21EDE; Mon, 8 Jul 2019 19:21:28 +0000 (UTC) Received: from mail-io1-f65.google.com (mail-io1-f65.google.com [209.85.166.65]) by lists.denx.de (Postfix) with ESMTPS id 64801C21D9A for ; Mon, 8 Jul 2019 19:21:24 +0000 (UTC) Received: by mail-io1-f65.google.com with SMTP id z3so22880323iog.0 for ; Mon, 08 Jul 2019 12:21:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yJ4H2usj9LxwPla4+VKkB5YP5C5W7Fw59XLPNR1U0dk=; b=i1U3lBhzeMEjlUZaA+3Kk8ivk7hRnkSYmEeKxPWiPft245bYCJ6anAzFeEU8IPzlUt 80YPCZmIenJmO0H77Kv+ADmFqAXnenjkuMd2vh/XSTl1UXsLj0Xz588pkVPIP8fuet1I eWCC1wmRmZNebiEzr57bEo/9ofA4OIGdyNt1E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yJ4H2usj9LxwPla4+VKkB5YP5C5W7Fw59XLPNR1U0dk=; b=LGZGON5uwE0wRG55BG/iy4UKifefYS4qDD8/F1M4kA8baS0SX66T7U07cqrYXRHZUD b7iBlN+3Q3VIQlSNMb2XiF0NNwprH3bZOxLpv7A5s+p1+OviEJtNVJiNffw+t6oynXYu uwCuuC22om+YaBi77KUrnCdFpA6bbbjX+oC1xT4m9Kqbol1PjFfWdK+WB/dwn65VD0BM +l1PQ4YaIo1LU3xtwsf5TMMS3d4j5OyNkJgzTxj2R0QHiK3wO6i3CrMaVM+N04IMDh4e eMJ4cFbrbMGRDh92OGrxd3jbmxDQ654cI5ubGD76n/MEgWH8bNygncEIwxWN46TsXXCH p9Xg== X-Gm-Message-State: APjAAAUM/pl5fiMS3aYpyX3a7vHRXy/9phfyvtKJqJGLaliR9Q2/qZHb 2eKEu8VKKz2SECxkv+OLR9/ZhqEkS8I= X-Google-Smtp-Source: APXvYqxWnXKcN9OZIYgLUnnBjTHnNTTdD7eBWcP1paRp1lZgI+6dMAoHShfCB/d+XI36walrtITBPA== X-Received: by 2002:a02:c95a:: with SMTP id u26mr21906717jao.15.1562613682763; Mon, 08 Jul 2019 12:21:22 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.22 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:22 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:52 -0600 Message-Id: <20190708191856.138863-34-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 33/37] binman: Add a utility library for coreboot CBFS X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" Coreboot uses a simple flash-based filesystem called Coreboot Filesystem (CBFS) to organise files used during boot. This allows files to be named and their position in the flash to be set. It has special features for dealing with x86 devices which typically memory-map their SPI flash to the top of 32-bit address space and need a 'boot block' ending there. Create a library to help create and read CBFS files. This includes a writer class, a reader class and associated other helpers. Only a subset of features are currently supported. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: - Deal with travis's old lz4 version by skipping tests as necessary - Install lzma tool in travis - Skip use of cbfstool in tests if it is not available .travis.yml | 1 + tools/binman/binman.py | 4 +- tools/binman/cbfs_util.py | 720 +++++++++++++++++++++++++++++++++ tools/binman/cbfs_util_test.py | 540 +++++++++++++++++++++++++ 4 files changed, 1264 insertions(+), 1 deletion(-) create mode 100644 tools/binman/cbfs_util.py create mode 100755 tools/binman/cbfs_util_test.py diff --git a/.travis.yml b/.travis.yml index c883198abbf..70d89d3e233 100644 --- a/.travis.yml +++ b/.travis.yml @@ -32,6 +32,7 @@ addons: - device-tree-compiler - lzop - liblz4-tool + - lzma-alone - libisl15 - clang-7 - srecord diff --git a/tools/binman/binman.py b/tools/binman/binman.py index 52c03f68c6d..613aad5c451 100755 --- a/tools/binman/binman.py +++ b/tools/binman/binman.py @@ -62,6 +62,7 @@ def RunTests(debug, verbosity, processes, test_preserve_dirs, args, toolpath): name to execute (as in 'binman -t testSections', for example) toolpath: List of paths to use for tools """ + import cbfs_util_test import elf_test import entry_test import fdt_test @@ -90,7 +91,8 @@ def RunTests(debug, verbosity, processes, test_preserve_dirs, args, toolpath): suite = unittest.TestSuite() loader = unittest.TestLoader() for module in (entry_test.TestEntry, ftest.TestFunctional, fdt_test.TestFdt, - elf_test.TestElf, image_test.TestImage): + elf_test.TestElf, image_test.TestImage, + cbfs_util_test.TestCbfs): # Test the test module about our arguments, if it is interested if hasattr(module, 'setup_test_args'): setup_test_args = getattr(module, 'setup_test_args') diff --git a/tools/binman/cbfs_util.py b/tools/binman/cbfs_util.py new file mode 100644 index 00000000000..197cff89509 --- /dev/null +++ b/tools/binman/cbfs_util.py @@ -0,0 +1,720 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright 2019 Google LLC +# Written by Simon Glass + +"""Support for coreboot's CBFS format + +CBFS supports a header followed by a number of files, generally targeted at SPI +flash. + +The format is somewhat defined by documentation in the coreboot tree although +it is necessary to rely on the C structures and source code (mostly cbfstool) +to fully understand it. + +Currently supported: raw and stage types with compression +""" + +from __future__ import print_function + +from collections import OrderedDict +import io +import struct +import sys + +import command +import elf +import tools + +# Set to True to enable printing output while working +DEBUG = False + +# Set to True to enable output from running cbfstool for debugging +VERBOSE = False + +# The master header, at the start of the CBFS +HEADER_FORMAT = '>IIIIIIII' +HEADER_LEN = 0x20 +HEADER_MAGIC = 0x4f524243 +HEADER_VERSION1 = 0x31313131 +HEADER_VERSION2 = 0x31313132 + +# The file header, at the start of each file in the CBFS +FILE_HEADER_FORMAT = b'>8sIIII' +FILE_HEADER_LEN = 0x18 +FILE_MAGIC = b'LARCHIVE' +FILENAME_ALIGN = 16 # Filename lengths are aligned to this + +# A stage header containing information about 'stage' files +# Yes this is correct: this header is in litte-endian format +STAGE_FORMAT = ' offset: + raise ValueError('No space for data before offset %#x (current offset %#x)' % + (offset, fd.tell())) + fd.write(tools.GetBytes(self._erase_byte, offset - fd.tell())) + + def _align_to(self, fd, align): + """Write out pad bytes until a given alignment is reached + + This only aligns if the resulting output would not reach the end of the + CBFS, since we want to leave the last 4 bytes for the master-header + pointer. + + Args: + fd: File objext to write to + align: Alignment to require (e.g. 4 means pad to next 4-byte + boundary) + """ + offset = align_int(fd.tell(), align) + if offset < self._size: + self._skip_to(fd, offset) + + def add_file_stage(self, name, data): + """Add a new stage file to the CBFS + + Args: + name: String file name to put in CBFS (does not need to correspond + to the name that the file originally came from) + data: Contents of file + + Returns: + CbfsFile object created + """ + cfile = CbfsFile.stage(self._base_address, name, data) + self._files[name] = cfile + return cfile + + def add_file_raw(self, name, data, compress=COMPRESS_NONE): + """Create a new raw file + + Args: + name: String file name to put in CBFS (does not need to correspond + to the name that the file originally came from) + data: Contents of file + compress: Compression algorithm to use (COMPRESS_...) + + Returns: + CbfsFile object created + """ + cfile = CbfsFile.raw(name, data, compress) + self._files[name] = cfile + return cfile + + def _write_header(self, fd, add_fileheader): + """Write out the master header to a CBFS + + Args: + fd: File object + add_fileheader: True to place the master header in a file header + record + """ + if fd.tell() > self._header_offset: + raise ValueError('No space for header at offset %#x (current offset %#x)' % + (self._header_offset, fd.tell())) + if not add_fileheader: + self._skip_to(fd, self._header_offset) + hdr = struct.pack(HEADER_FORMAT, HEADER_MAGIC, HEADER_VERSION2, + self._size, self._bootblock_size, self._align, + self._contents_offset, self._arch, 0xffffffff) + if add_fileheader: + name = _pack_string(self._master_name) + fd.write(struct.pack(FILE_HEADER_FORMAT, FILE_MAGIC, len(hdr), + TYPE_CBFSHEADER, 0, + FILE_HEADER_LEN + len(name))) + fd.write(name) + self._header_offset = fd.tell() + fd.write(hdr) + self._align_to(fd, self._align) + else: + fd.write(hdr) + + def get_data(self): + """Obtain the full contents of the CBFS + + Thhis builds the CBFS with headers and all required files. + + Returns: + 'bytes' type containing the data + """ + fd = io.BytesIO() + + # THe header can go at the start in some cases + if self._hdr_at_start: + self._write_header(fd, add_fileheader=self._add_fileheader) + self._skip_to(fd, self._contents_offset) + + # Write out each file + for cbf in self._files.values(): + fd.write(cbf.get_data()) + self._align_to(fd, self._align) + if not self._hdr_at_start: + self._write_header(fd, add_fileheader=self._add_fileheader) + + # Pad to the end and write a pointer to the CBFS master header + self._skip_to(fd, self._base_address or self._size - 4) + rel_offset = self._header_offset - self._size + fd.write(struct.pack('II", hdr) + data = hdr + fd.read(alen - 8) + if atag == FILE_ATTR_TAG_COMPRESSION: + # We don't currently use this information + atag, alen, compress, _decomp_size = struct.unpack( + ATTR_COMPRESSION_FORMAT, data) + else: + print('Unknown attribute tag %x' % atag) + attr_size -= len(data) + return compress + + def _read_header(self, fd): + """Read the master header + + Reads the header and stores the information obtained into the member + variables. + + Args: + fd: File to read from + + Returns: + True if header was read OK, False if it is truncated or has the + wrong magic or version + """ + pos = fd.tell() + data = fd.read(HEADER_LEN) + if len(data) < HEADER_LEN: + print('Header at %x ran out of data' % pos) + return False + (self.magic, self.version, self.rom_size, self.boot_block_size, + self.align, self.cbfs_offset, self.arch, _) = struct.unpack( + HEADER_FORMAT, data) + return self.magic == HEADER_MAGIC and ( + self.version == HEADER_VERSION1 or + self.version == HEADER_VERSION2) + + @classmethod + def _read_string(cls, fd): + """Read a string from a file + + This reads a string and aligns the data to the next alignment boundary + + Args: + fd: File to read from + + Returns: + string read ('str' type) encoded to UTF-8, or None if we ran out of + data + """ + val = b'' + while True: + data = fd.read(FILENAME_ALIGN) + if len(data) < FILENAME_ALIGN: + return None + pos = data.find(b'\0') + if pos == -1: + val += data + else: + val += data[:pos] + break + return val.decode('utf-8') + + +def cbfstool(fname, *cbfs_args): + """Run cbfstool with provided arguments + + If the tool fails then this function raises an exception and prints out the + output and stderr. + + Args: + fname: Filename of CBFS + *cbfs_args: List of arguments to pass to cbfstool + + Returns: + CommandResult object containing the results + """ + args = ('cbfstool', fname) + cbfs_args + result = command.RunPipe([args], capture=not VERBOSE, + capture_stderr=not VERBOSE, raise_on_error=False) + if result.return_code: + print(result.stderr, file=sys.stderr) + raise Exception("Failed to run (error %d): '%s'" % + (result.return_code, ' '.join(args))) diff --git a/tools/binman/cbfs_util_test.py b/tools/binman/cbfs_util_test.py new file mode 100755 index 00000000000..19086305af1 --- /dev/null +++ b/tools/binman/cbfs_util_test.py @@ -0,0 +1,540 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0+ +# Copyright 2019 Google LLC +# Written by Simon Glass + +"""Tests for cbfs_util + +These create and read various CBFSs and compare the results with expected +values and with cbfstool +""" + +from __future__ import print_function + +import io +import os +import shutil +import struct +import tempfile +import unittest + +import cbfs_util +from cbfs_util import CbfsWriter +import elf +import test_util +import tools + +U_BOOT_DATA = b'1234' +U_BOOT_DTB_DATA = b'udtb' +COMPRESS_DATA = b'compress xxxxxxxxxxxxxxxxxxxxxx data' + + +class TestCbfs(unittest.TestCase): + """Test of cbfs_util classes""" + #pylint: disable=W0212 + @classmethod + def setUpClass(cls): + # Create a temporary directory for test files + cls._indir = tempfile.mkdtemp(prefix='cbfs_util.') + tools.SetInputDirs([cls._indir]) + + # Set up some useful data files + TestCbfs._make_input_file('u-boot.bin', U_BOOT_DATA) + TestCbfs._make_input_file('u-boot.dtb', U_BOOT_DTB_DATA) + TestCbfs._make_input_file('compress', COMPRESS_DATA) + + # Set up a temporary output directory, used by the tools library when + # compressing files + tools.PrepareOutputDir(None) + + cls.have_cbfstool = True + try: + tools.Run('which', 'cbfstool') + except: + cls.have_cbfstool = False + + cls.have_lz4 = True + try: + tools.Run('lz4', '--no-frame-crc', '-c', + tools.GetInputFilename('u-boot.bin')) + except: + cls.have_lz4 = False + + @classmethod + def tearDownClass(cls): + """Remove the temporary input directory and its contents""" + if cls._indir: + shutil.rmtree(cls._indir) + cls._indir = None + tools.FinaliseOutputDir() + + @classmethod + def _make_input_file(cls, fname, contents): + """Create a new test input file, creating directories as needed + + Args: + fname: Filename to create + contents: File contents to write in to the file + Returns: + Full pathname of file created + """ + pathname = os.path.join(cls._indir, fname) + tools.WriteFile(pathname, contents) + return pathname + + def _check_hdr(self, data, size, offset=0, arch=cbfs_util.ARCHITECTURE_X86): + """Check that the CBFS has the expected header + + Args: + data: Data to check + size: Expected ROM size + offset: Expected offset to first CBFS file + arch: Expected architecture + + Returns: + CbfsReader object containing the CBFS + """ + cbfs = cbfs_util.CbfsReader(data) + self.assertEqual(cbfs_util.HEADER_MAGIC, cbfs.magic) + self.assertEqual(cbfs_util.HEADER_VERSION2, cbfs.version) + self.assertEqual(size, cbfs.rom_size) + self.assertEqual(0, cbfs.boot_block_size) + self.assertEqual(cbfs_util.ENTRY_ALIGN, cbfs.align) + self.assertEqual(offset, cbfs.cbfs_offset) + self.assertEqual(arch, cbfs.arch) + return cbfs + + def _check_uboot(self, cbfs, ftype=cbfs_util.TYPE_RAW, offset=0x38, + data=U_BOOT_DATA): + """Check that the U-Boot file is as expected + + Args: + cbfs: CbfsReader object to check + ftype: Expected file type + offset: Expected offset of file + data: Expected data in file + + Returns: + CbfsFile object containing the file + """ + self.assertIn('u-boot', cbfs.files) + cfile = cbfs.files['u-boot'] + self.assertEqual('u-boot', cfile.name) + self.assertEqual(offset, cfile.offset) + self.assertEqual(data, cfile.data) + self.assertEqual(ftype, cfile.ftype) + self.assertEqual(cbfs_util.COMPRESS_NONE, cfile.compress) + self.assertEqual(len(data), cfile.memlen) + return cfile + + def _check_dtb(self, cbfs, offset=0x38, data=U_BOOT_DTB_DATA): + """Check that the U-Boot dtb file is as expected + + Args: + cbfs: CbfsReader object to check + offset: Expected offset of file + data: Expected data in file + """ + self.assertIn('u-boot-dtb', cbfs.files) + cfile = cbfs.files['u-boot-dtb'] + self.assertEqual('u-boot-dtb', cfile.name) + self.assertEqual(offset, cfile.offset) + self.assertEqual(U_BOOT_DTB_DATA, cfile.data) + self.assertEqual(cbfs_util.TYPE_RAW, cfile.ftype) + self.assertEqual(cbfs_util.COMPRESS_NONE, cfile.compress) + self.assertEqual(len(U_BOOT_DTB_DATA), cfile.memlen) + + def _check_raw(self, data, size, offset=0, arch=cbfs_util.ARCHITECTURE_X86): + """Check that two raw files are added as expected + + Args: + data: Data to check + size: Expected ROM size + offset: Expected offset to first CBFS file + arch: Expected architecture + """ + cbfs = self._check_hdr(data, size, offset=offset, arch=arch) + self._check_uboot(cbfs) + self._check_dtb(cbfs) + + def _get_expected_cbfs(self, size, arch='x86', compress=None): + """Get the file created by cbfstool for a particular scenario + + Args: + size: Size of the CBFS in bytes + arch: Architecture of the CBFS, as a string + compress: Compression to use, e.g. cbfs_util.COMPRESS_LZMA + + Returns: + Resulting CBFS file, or None if cbfstool is not available + """ + if not self.have_cbfstool or not self.have_lz4: + return None + cbfs_fname = os.path.join(self._indir, 'test.cbfs') + cbfs_util.cbfstool(cbfs_fname, 'create', '-m', arch, '-s', '%#x' % size) + cbfs_util.cbfstool(cbfs_fname, 'add', '-n', 'u-boot', '-t', 'raw', + '-c', compress and compress[0] or 'none', + '-f', tools.GetInputFilename( + compress and 'compress' or 'u-boot.bin')) + cbfs_util.cbfstool(cbfs_fname, 'add', '-n', 'u-boot-dtb', '-t', 'raw', + '-c', compress and compress[1] or 'none', + '-f', tools.GetInputFilename( + compress and 'compress' or 'u-boot.dtb')) + return cbfs_fname + + def _compare_expected_cbfs(self, data, cbfstool_fname): + """Compare against what cbfstool creates + + This compares what binman creates with what cbfstool creates for what + is proportedly the same thing. + + Args: + data: CBFS created by binman + cbfstool_fname: CBFS created by cbfstool + """ + if not self.have_cbfstool or not self.have_lz4: + return + expect = tools.ReadFile(cbfstool_fname) + if expect != data: + tools.WriteFile('/tmp/expect', expect) + tools.WriteFile('/tmp/actual', data) + print('diff -y <(xxd -g1 /tmp/expect) <(xxd -g1 /tmp/actual) | colordiff') + self.fail('cbfstool produced a different result') + + def test_cbfs_functions(self): + """Test global functions of cbfs_util""" + self.assertEqual(cbfs_util.ARCHITECTURE_X86, cbfs_util.find_arch('x86')) + self.assertIsNone(cbfs_util.find_arch('bad-arch')) + + self.assertEqual(cbfs_util.COMPRESS_LZMA, cbfs_util.find_compress('lzma')) + self.assertIsNone(cbfs_util.find_compress('bad-comp')) + + def test_cbfstool_failure(self): + """Test failure to run cbfstool""" + if not self.have_cbfstool: + self.skipTest('No cbfstool available') + try: + # In verbose mode this test fails since stderr is not captured. Fix + # this by turning off verbosity. + old_verbose = cbfs_util.VERBOSE + cbfs_util.VERBOSE = False + with test_util.capture_sys_output() as (_stdout, stderr): + with self.assertRaises(Exception) as e: + cbfs_util.cbfstool('missing-file', 'bad-command') + finally: + cbfs_util.VERBOSE = old_verbose + self.assertIn('Unknown command', stderr.getvalue()) + self.assertIn('Failed to run', str(e.exception)) + + def test_cbfs_raw(self): + """Test base handling of a Coreboot Filesystem (CBFS)""" + size = 0xb0 + cbw = CbfsWriter(size) + cbw.add_file_raw('u-boot', U_BOOT_DATA) + cbw.add_file_raw('u-boot-dtb', U_BOOT_DTB_DATA) + data = cbw.get_data() + self._check_raw(data, size) + cbfs_fname = self._get_expected_cbfs(size=size) + self._compare_expected_cbfs(data, cbfs_fname) + + def test_cbfs_invalid_file_type(self): + """Check handling of an invalid file type when outputiing a CBFS""" + size = 0xb0 + cbw = CbfsWriter(size) + cfile = cbw.add_file_raw('u-boot', U_BOOT_DATA) + + # Change the type manually before generating the CBFS, and make sure + # that the generator complains + cfile.ftype = 0xff + with self.assertRaises(ValueError) as e: + cbw.get_data() + self.assertIn('Unknown type 0xff when writing', str(e.exception)) + + def test_cbfs_invalid_file_type_on_read(self): + """Check handling of an invalid file type when reading the CBFS""" + size = 0xb0 + cbw = CbfsWriter(size) + cbw.add_file_raw('u-boot', U_BOOT_DATA) + + data = cbw.get_data() + + # Read in the first file header + cbr = cbfs_util.CbfsReader(data, read=False) + with io.BytesIO(data) as fd: + self.assertTrue(cbr._find_and_read_header(fd, len(data))) + pos = fd.tell() + hdr_data = fd.read(cbfs_util.FILE_HEADER_LEN) + magic, size, ftype, attr, offset = struct.unpack( + cbfs_util.FILE_HEADER_FORMAT, hdr_data) + + # Create a new CBFS with a change to the file type + ftype = 0xff + newdata = data[:pos] + newdata += struct.pack(cbfs_util.FILE_HEADER_FORMAT, magic, size, ftype, + attr, offset) + newdata += data[pos + cbfs_util.FILE_HEADER_LEN:] + + # Read in this CBFS and make sure that the reader complains + with self.assertRaises(ValueError) as e: + cbfs_util.CbfsReader(newdata) + self.assertIn('Unknown type 0xff when reading', str(e.exception)) + + def test_cbfs_no_space(self): + """Check handling of running out of space in the CBFS""" + size = 0x60 + cbw = CbfsWriter(size) + cbw.add_file_raw('u-boot', U_BOOT_DATA) + with self.assertRaises(ValueError) as e: + cbw.get_data() + self.assertIn('No space for header', str(e.exception)) + + def test_cbfs_no_space_skip(self): + """Check handling of running out of space in CBFS with file header""" + size = 0x70 + cbw = CbfsWriter(size) + cbw._add_fileheader = True + cbw.add_file_raw('u-boot', U_BOOT_DATA) + with self.assertRaises(ValueError) as e: + cbw.get_data() + self.assertIn('No space for data before offset', str(e.exception)) + + def test_cbfs_bad_header_ptr(self): + """Check handling of a bad master-header pointer""" + size = 0x70 + cbw = CbfsWriter(size) + cbw.add_file_raw('u-boot', U_BOOT_DATA) + data = cbw.get_data() + + # Add one to the pointer to make it invalid + newdata = data[:-4] + struct.pack('I', fd.read(4)) + if tag == cbfs_util.FILE_ATTR_TAG_COMPRESSION: + break + + # Create a new CBFS with the tag changed to something invalid + newdata = data[:pos] + struct.pack('>I', 0x123) + data[pos + 4:] + with test_util.capture_sys_output() as (stdout, _stderr): + cbfs_util.CbfsReader(newdata) + self.assertEqual('Unknown attribute tag 123\n', stdout.getvalue()) + + def test_cbfs_missing_attribute(self): + """Check handling of an incomplete attribute tag""" + if not self.have_lz4: + self.skipTest('lz4 --no-frame-crc not available') + size = 0x140 + cbw = CbfsWriter(size) + cbw.add_file_raw('u-boot', COMPRESS_DATA, + compress=cbfs_util.COMPRESS_LZ4) + data = cbw.get_data() + + # Read in the CBFS master header (only), then stop + cbr = cbfs_util.CbfsReader(data, read=False) + with io.BytesIO(data) as fd: + self.assertTrue(cbr._find_and_read_header(fd, len(data))) + pos = fd.tell() + + # Create a new CBFS with only the first 4 bytes of the compression tag, + # then try to read the file + tag_pos = pos + cbfs_util.FILE_HEADER_LEN + cbfs_util.FILENAME_ALIGN + newdata = data[:tag_pos + 4] + with test_util.capture_sys_output() as (stdout, _stderr): + with io.BytesIO(newdata) as fd: + fd.seek(pos) + self.assertEqual(False, cbr._read_next_file(fd)) + self.assertIn('Attribute tag at %x ran out of data' % tag_pos, + stdout.getvalue()) + + def test_cbfs_file_master_header(self): + """Check handling of a file containing a master header""" + size = 0x100 + cbw = CbfsWriter(size) + cbw._add_fileheader = True + cbw.add_file_raw('u-boot', U_BOOT_DATA) + data = cbw.get_data() + + cbr = cbfs_util.CbfsReader(data) + self.assertIn('u-boot', cbr.files) + self.assertEqual(size, cbr.rom_size) + + def test_cbfs_arch(self): + """Test on non-x86 architecture""" + size = 0x100 + cbw = CbfsWriter(size, arch=cbfs_util.ARCHITECTURE_PPC64) + cbw.add_file_raw('u-boot', U_BOOT_DATA) + cbw.add_file_raw('u-boot-dtb', U_BOOT_DTB_DATA) + data = cbw.get_data() + self._check_raw(data, size, offset=0x40, + arch=cbfs_util.ARCHITECTURE_PPC64) + + # Compare against what cbfstool creates + cbfs_fname = self._get_expected_cbfs(size=size, arch='ppc64') + self._compare_expected_cbfs(data, cbfs_fname) + + def test_cbfs_stage(self): + """Tests handling of a Coreboot Filesystem (CBFS)""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') + elf_fname = os.path.join(self._indir, 'cbfs-stage.elf') + elf.MakeElf(elf_fname, U_BOOT_DATA, U_BOOT_DTB_DATA) + + size = 0xb0 + cbw = CbfsWriter(size) + cbw.add_file_stage('u-boot', tools.ReadFile(elf_fname)) + + data = cbw.get_data() + cbfs = self._check_hdr(data, size) + load = 0xfef20000 + entry = load + 2 + + cfile = self._check_uboot(cbfs, cbfs_util.TYPE_STAGE, offset=0x28, + data=U_BOOT_DATA + U_BOOT_DTB_DATA) + + self.assertEqual(entry, cfile.entry) + self.assertEqual(load, cfile.load) + self.assertEqual(len(U_BOOT_DATA) + len(U_BOOT_DTB_DATA), + cfile.data_len) + + # Compare against what cbfstool creates + if self.have_cbfstool: + cbfs_fname = os.path.join(self._indir, 'test.cbfs') + cbfs_util.cbfstool(cbfs_fname, 'create', '-m', 'x86', '-s', + '%#x' % size) + cbfs_util.cbfstool(cbfs_fname, 'add-stage', '-n', 'u-boot', + '-f', elf_fname) + self._compare_expected_cbfs(data, cbfs_fname) + + def test_cbfs_raw_compress(self): + """Test base handling of compressing raw files""" + if not self.have_lz4: + self.skipTest('lz4 --no-frame-crc not available') + size = 0x140 + cbw = CbfsWriter(size) + cbw.add_file_raw('u-boot', COMPRESS_DATA, + compress=cbfs_util.COMPRESS_LZ4) + cbw.add_file_raw('u-boot-dtb', COMPRESS_DATA, + compress=cbfs_util.COMPRESS_LZMA) + data = cbw.get_data() + + cbfs = self._check_hdr(data, size) + self.assertIn('u-boot', cbfs.files) + cfile = cbfs.files['u-boot'] + self.assertEqual(cfile.name, 'u-boot') + self.assertEqual(cfile.offset, 56) + self.assertEqual(cfile.data, COMPRESS_DATA) + self.assertEqual(cfile.ftype, cbfs_util.TYPE_RAW) + self.assertEqual(cfile.compress, cbfs_util.COMPRESS_LZ4) + self.assertEqual(cfile.memlen, len(COMPRESS_DATA)) + + self.assertIn('u-boot-dtb', cbfs.files) + cfile = cbfs.files['u-boot-dtb'] + self.assertEqual(cfile.name, 'u-boot-dtb') + self.assertEqual(cfile.offset, 56) + self.assertEqual(cfile.data, COMPRESS_DATA) + self.assertEqual(cfile.ftype, cbfs_util.TYPE_RAW) + self.assertEqual(cfile.compress, cbfs_util.COMPRESS_LZMA) + self.assertEqual(cfile.memlen, len(COMPRESS_DATA)) + + cbfs_fname = self._get_expected_cbfs(size=size, compress=['lz4', 'lzma']) + self._compare_expected_cbfs(data, cbfs_fname) + + +if __name__ == '__main__': + unittest.main() From patchwork Mon Jul 8 19:18:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129294 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="UT/fIX0R"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jG3M08zpz9sNH for ; Tue, 9 Jul 2019 05:38:55 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id C04A7C21E3A; Mon, 8 Jul 2019 19:35:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 61968C21EE8; Mon, 8 Jul 2019 19:22:21 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 8A59FC21EF2; Mon, 8 Jul 2019 19:21:29 +0000 (UTC) Received: from mail-io1-f47.google.com (mail-io1-f47.google.com [209.85.166.47]) by lists.denx.de (Postfix) with ESMTPS id 0F5CCC21D4A for ; Mon, 8 Jul 2019 19:21:25 +0000 (UTC) Received: by mail-io1-f47.google.com with SMTP id k20so37786550ios.10 for ; Mon, 08 Jul 2019 12:21:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VH6ckx6DKoH4LJ0L1V4yyx70QWuS8tiLDQJENVzanLU=; b=UT/fIX0RxYJ3RhhF95W3XmsuMWxAwuViHvh/ZygwLeZmcALUB9CdYe5TSuMSNeYXnU vaLzAvmGpDe1M4qWRZp5YywQO9vDksYKsllVdY9TB1lKWK4rginSh+gQsrJXWG7pp8VS 8DTIAzi7NmI2UE7XBc8LeFpkKfN2CPY4hfq7Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VH6ckx6DKoH4LJ0L1V4yyx70QWuS8tiLDQJENVzanLU=; b=pezE7IDw+Ld3xvgMeSf3jIR8smqVtW70WA6p9G1keOiwGkLuF1z9rRI1UAz3NZcHYX IiMGP4PqXCN9M6tKpLp84Of45Avp6fAR8NkX4F2XFku/UKT1IJHcfYZT5VW0OFvrIjIW CBbrCtGdMU1Of2uQi3k6NpMJHBc4JUAZ1bx8BgALgMGgpCvwtsSIT0jDCpeBw6pEFOpO 3HldVHRmftUuGE4JrXTVILN0ZLcw3CFVWHom8ZLwaxx4uwAGRaFqTf8l+tzUusuxODUZ MCsEVkq5rLXpHTQ6zc4XYcMcGk41xO/7GBINumRJ1Mj7t9D3huT+MXAbKWw6yBRAeFv4 EA3g== X-Gm-Message-State: APjAAAXCBVd/Tcg6MnxT9uW+xxba6jUOqFpi1BbFRh1SKCU52OQc1qmh EdLLxQFw8zSe/NYUf1cJd8tUgBjsiSw= X-Google-Smtp-Source: APXvYqyrLaFffy1crkUKCMCxafeYye4k/8qS+sYU3s+RyknpaTCbOFRD6B8BIAFHtAVi8Qh0mkAgjw== X-Received: by 2002:a02:54c1:: with SMTP id t184mr24314453jaa.10.1562613683501; Mon, 08 Jul 2019 12:21:23 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.22 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:23 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:53 -0600 Message-Id: <20190708191856.138863-35-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 34/37] binman: Add support for CBFS entries X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" Add support for putting CBFSs (Coreboot Filesystems) in an image. This allows binman to produce firmware images used by coreboot to boot. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: - Deal with travis's old lz4 version by skipping tests as necessary - Skip use of cbfstool in tests if it is not available tools/binman/README.entries | 142 ++++++++++++++ tools/binman/control.py | 2 + tools/binman/etype/cbfs.py | 193 ++++++++++++++++++++ tools/binman/ftest.py | 110 +++++++++++ tools/binman/test/102_cbfs_raw.dts | 20 ++ tools/binman/test/103_cbfs_raw_ppc.dts | 21 +++ tools/binman/test/104_cbfs_stage.dts | 19 ++ tools/binman/test/105_cbfs_raw_compress.dts | 26 +++ tools/binman/test/106_cbfs_bad_arch.dts | 15 ++ tools/binman/test/107_cbfs_no_size.dts | 13 ++ tools/binman/test/108_cbfs_no_contents.dts | 17 ++ tools/binman/test/109_cbfs_bad_compress.dts | 18 ++ tools/binman/test/110_cbfs_name.dts | 24 +++ 13 files changed, 620 insertions(+) create mode 100644 tools/binman/etype/cbfs.py create mode 100644 tools/binman/test/102_cbfs_raw.dts create mode 100644 tools/binman/test/103_cbfs_raw_ppc.dts create mode 100644 tools/binman/test/104_cbfs_stage.dts create mode 100644 tools/binman/test/105_cbfs_raw_compress.dts create mode 100644 tools/binman/test/106_cbfs_bad_arch.dts create mode 100644 tools/binman/test/107_cbfs_no_size.dts create mode 100644 tools/binman/test/108_cbfs_no_contents.dts create mode 100644 tools/binman/test/109_cbfs_bad_compress.dts create mode 100644 tools/binman/test/110_cbfs_name.dts diff --git a/tools/binman/README.entries b/tools/binman/README.entries index 9a316763ace..2e6aea1e84c 100644 --- a/tools/binman/README.entries +++ b/tools/binman/README.entries @@ -60,6 +60,148 @@ See cros_ec_rw for an example of this. +Entry: cbfs: Entry containing a Coreboot Filesystem (CBFS) +---------------------------------------------------------- + +A CBFS provides a way to group files into a group. It has a simple directory +structure and allows the position of individual files to be set, since it is +designed to support execute-in-place in an x86 SPI-flash device. Where XIP +is not used, it supports compression and storing ELF files. + +CBFS is used by coreboot as its way of orgnanising SPI-flash contents. + +The contents of the CBFS are defined by subnodes of the cbfs entry, e.g.: + + cbfs { + size = <0x100000>; + u-boot { + cbfs-type = "raw"; + }; + u-boot-dtb { + cbfs-type = "raw"; + }; + }; + +This creates a CBFS 1MB in size two files in it: u-boot.bin and u-boot.dtb. +Note that the size is required since binman does not support calculating it. +The contents of each entry is just what binman would normally provide if it +were not a CBFS node. A blob type can be used to import arbitrary files as +with the second subnode below: + + cbfs { + size = <0x100000>; + u-boot { + cbfs-name = "BOOT"; + cbfs-type = "raw"; + }; + + dtb { + type = "blob"; + filename = "u-boot.dtb"; + cbfs-type = "raw"; + cbfs-compress = "lz4"; + }; + }; + +This creates a CBFS 1MB in size with u-boot.bin (named "BOOT") and +u-boot.dtb (named "dtb") and compressed with the lz4 algorithm. + + +Properties supported in the top-level CBFS node: + +cbfs-arch: + Defaults to "x86", but you can specify the architecture if needed. + + +Properties supported in the CBFS entry subnodes: + +cbfs-name: + This is the name of the file created in CBFS. It defaults to the entry + name (which is the node name), but you can override it with this + property. + +cbfs-type: + This is the CBFS file type. The following are supported: + + raw: + This is a 'raw' file, although compression is supported. It can be + used to store any file in CBFS. + + stage: + This is an ELF file that has been loaded (i.e. mapped to memory), so + appears in the CBFS as a flat binary. The input file must be an ELF + image, for example this puts "u-boot" (the ELF image) into a 'stage' + entry: + + cbfs { + size = <0x100000>; + u-boot-elf { + cbfs-name = "BOOT"; + cbfs-type = "stage"; + }; + }; + + You can use your own ELF file with something like: + + cbfs { + size = <0x100000>; + something { + type = "blob"; + filename = "cbfs-stage.elf"; + cbfs-type = "stage"; + }; + }; + + As mentioned, the file is converted to a flat binary, so it is + equivalent to adding "u-boot.bin", for example, but with the load and + start addresses specified by the ELF. At present there is no option + to add a flat binary with a load/start address, similar to the + 'add-flat-binary' option in cbfstool. + + +The current implementation supports only a subset of CBFS features. It does +not support other file types (e.g. payload), adding multiple files (like the +'files' entry with a pattern supported by binman), putting files at a +particular offset in the CBFS and a few other things. + +Of course binman can create images containing multiple CBFSs, simply by +defining these in the binman config: + + + binman { + size = <0x800000>; + cbfs { + offset = <0x100000>; + size = <0x100000>; + u-boot { + cbfs-type = "raw"; + }; + u-boot-dtb { + cbfs-type = "raw"; + }; + }; + + cbfs2 { + offset = <0x700000>; + size = <0x100000>; + u-boot { + cbfs-type = "raw"; + }; + u-boot-dtb { + cbfs-type = "raw"; + }; + image { + type = "blob"; + filename = "image.jpg"; + }; + }; + }; + +This creates an 8MB image with two CBFSs, one at offset 1MB, one at 7MB, +both of size 1MB. + + + Entry: cros-ec-rw: A blob entry which contains a Chromium OS read-write EC image -------------------------------------------------------------------------------- diff --git a/tools/binman/control.py b/tools/binman/control.py index df78848e13d..4a94afc8640 100644 --- a/tools/binman/control.py +++ b/tools/binman/control.py @@ -12,6 +12,7 @@ import os import sys import tools +import cbfs_util import command import elf from image import Image @@ -108,6 +109,7 @@ def Binman(options, args): tout.Init(options.verbosity) elf.debug = options.debug + cbfs_util.VERBOSE = options.verbosity > 2 state.use_fake_dtb = options.fake_dtb try: tools.SetInputDirs(options.indir) diff --git a/tools/binman/etype/cbfs.py b/tools/binman/etype/cbfs.py new file mode 100644 index 00000000000..513df217bc1 --- /dev/null +++ b/tools/binman/etype/cbfs.py @@ -0,0 +1,193 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright 2019 Google LLC +# Written by Simon Glass +# +# Entry-type module for a Coreboot Filesystem (CBFS) +# + +from collections import OrderedDict + +import cbfs_util +from cbfs_util import CbfsWriter +from entry import Entry +import fdt_util + +class Entry_cbfs(Entry): + """Entry containing a Coreboot Filesystem (CBFS) + + A CBFS provides a way to group files into a group. It has a simple directory + structure and allows the position of individual files to be set, since it is + designed to support execute-in-place in an x86 SPI-flash device. Where XIP + is not used, it supports compression and storing ELF files. + + CBFS is used by coreboot as its way of orgnanising SPI-flash contents. + + The contents of the CBFS are defined by subnodes of the cbfs entry, e.g.: + + cbfs { + size = <0x100000>; + u-boot { + cbfs-type = "raw"; + }; + u-boot-dtb { + cbfs-type = "raw"; + }; + }; + + This creates a CBFS 1MB in size two files in it: u-boot.bin and u-boot.dtb. + Note that the size is required since binman does not support calculating it. + The contents of each entry is just what binman would normally provide if it + were not a CBFS node. A blob type can be used to import arbitrary files as + with the second subnode below: + + cbfs { + size = <0x100000>; + u-boot { + cbfs-name = "BOOT"; + cbfs-type = "raw"; + }; + + dtb { + type = "blob"; + filename = "u-boot.dtb"; + cbfs-type = "raw"; + cbfs-compress = "lz4"; + }; + }; + + This creates a CBFS 1MB in size with u-boot.bin (named "BOOT") and + u-boot.dtb (named "dtb") and compressed with the lz4 algorithm. + + + Properties supported in the top-level CBFS node: + + cbfs-arch: + Defaults to "x86", but you can specify the architecture if needed. + + + Properties supported in the CBFS entry subnodes: + + cbfs-name: + This is the name of the file created in CBFS. It defaults to the entry + name (which is the node name), but you can override it with this + property. + + cbfs-type: + This is the CBFS file type. The following are supported: + + raw: + This is a 'raw' file, although compression is supported. It can be + used to store any file in CBFS. + + stage: + This is an ELF file that has been loaded (i.e. mapped to memory), so + appears in the CBFS as a flat binary. The input file must be an ELF + image, for example this puts "u-boot" (the ELF image) into a 'stage' + entry: + + cbfs { + size = <0x100000>; + u-boot-elf { + cbfs-name = "BOOT"; + cbfs-type = "stage"; + }; + }; + + You can use your own ELF file with something like: + + cbfs { + size = <0x100000>; + something { + type = "blob"; + filename = "cbfs-stage.elf"; + cbfs-type = "stage"; + }; + }; + + As mentioned, the file is converted to a flat binary, so it is + equivalent to adding "u-boot.bin", for example, but with the load and + start addresses specified by the ELF. At present there is no option + to add a flat binary with a load/start address, similar to the + 'add-flat-binary' option in cbfstool. + + + The current implementation supports only a subset of CBFS features. It does + not support other file types (e.g. payload), adding multiple files (like the + 'files' entry with a pattern supported by binman), putting files at a + particular offset in the CBFS and a few other things. + + Of course binman can create images containing multiple CBFSs, simply by + defining these in the binman config: + + + binman { + size = <0x800000>; + cbfs { + offset = <0x100000>; + size = <0x100000>; + u-boot { + cbfs-type = "raw"; + }; + u-boot-dtb { + cbfs-type = "raw"; + }; + }; + + cbfs2 { + offset = <0x700000>; + size = <0x100000>; + u-boot { + cbfs-type = "raw"; + }; + u-boot-dtb { + cbfs-type = "raw"; + }; + image { + type = "blob"; + filename = "image.jpg"; + }; + }; + }; + + This creates an 8MB image with two CBFSs, one at offset 1MB, one at 7MB, + both of size 1MB. + """ + def __init__(self, section, etype, node): + Entry.__init__(self, section, etype, node) + self._cbfs_arg = fdt_util.GetString(node, 'cbfs-arch', 'x86') + self._cbfs_entries = OrderedDict() + self._ReadSubnodes() + + def ObtainContents(self): + arch = cbfs_util.find_arch(self._cbfs_arg) + if arch is None: + self.Raise("Invalid architecture '%s'" % self._cbfs_arg) + if self.size is None: + self.Raise("'cbfs' entry must have a size property") + cbfs = CbfsWriter(self.size, arch) + for entry in self._cbfs_entries.values(): + # First get the input data and put it in a file. If not available, + # try later. + if not entry.ObtainContents(): + return False + data = entry.GetData() + if entry._type == 'raw': + cbfs.add_file_raw(entry._cbfs_name, data, entry._cbfs_compress) + elif entry._type == 'stage': + cbfs.add_file_stage(entry._cbfs_name, data) + data = cbfs.get_data() + self.SetContents(data) + return True + + def _ReadSubnodes(self): + """Read the subnodes to find out what should go in this IFWI""" + for node in self._node.subnodes: + entry = Entry.Create(self.section, node) + entry._cbfs_name = fdt_util.GetString(node, 'cbfs-name', entry.name) + entry._type = fdt_util.GetString(node, 'cbfs-type') + compress = fdt_util.GetString(node, 'cbfs-compress', 'none') + entry._cbfs_compress = cbfs_util.find_compress(compress) + if entry._cbfs_compress is None: + self.Raise("Invalid compression in '%s': '%s'" % + (node.name, compress)) + self._cbfs_entries[entry._cbfs_name] = entry diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 3455b8ccebd..14abfbf774f 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -18,6 +18,7 @@ import tempfile import unittest import binman +import cbfs_util import cmdline import command import control @@ -133,6 +134,14 @@ class TestFunctional(unittest.TestCase): TestFunctional._MakeInputFile('compress', COMPRESS_DATA) + # Travis-CI may have an old lz4 + self.have_lz4 = True + try: + tools.Run('lz4', '--no-frame-crc', '-c', + os.path.join(self._indir, 'u-boot.bin')) + except: + self.have_lz4 = False + @classmethod def tearDownClass(self): """Remove the temporary input directory and its contents""" @@ -160,6 +169,10 @@ class TestFunctional(unittest.TestCase): cls.preserve_outdirs = preserve_outdirs cls.toolpath = toolpath + def _CheckLz4(self): + if not self.have_lz4: + self.skipTest('lz4 --no-frame-crc not available') + def setUp(self): # Enable this to turn on debugging output # tout.Init(tout.DEBUG) @@ -1607,6 +1620,7 @@ class TestFunctional(unittest.TestCase): def testCompress(self): """Test compression of blobs""" + self._CheckLz4() data, _, _, out_dtb_fname = self._DoReadFileDtb('083_compress.dts', use_real_dtb=True, update_dtb=True) dtb = fdt.Fdt(out_dtb_fname) @@ -1628,6 +1642,7 @@ class TestFunctional(unittest.TestCase): def testFilesCompress(self): """Test bringing in multiple files and compressing them""" + self._CheckLz4() data = self._DoReadFile('085_files_compress.dts') image = control.images['image'] @@ -1846,6 +1861,101 @@ class TestFunctional(unittest.TestCase): tools.GetBytes(0x26, 4) + U_BOOT_DATA + tools.GetBytes(0x26, 8)) + def testCbfsRaw(self): + """Test base handling of a Coreboot Filesystem (CBFS) + + The exact contents of the CBFS is verified by similar tests in + cbfs_util_test.py. The tests here merely check that the files added to + the CBFS can be found in the final image. + """ + data = self._DoReadFile('102_cbfs_raw.dts') + size = 0xb0 + + cbfs = cbfs_util.CbfsReader(data) + self.assertEqual(size, cbfs.rom_size) + + self.assertIn('u-boot-dtb', cbfs.files) + cfile = cbfs.files['u-boot-dtb'] + self.assertEqual(U_BOOT_DTB_DATA, cfile.data) + + def testCbfsArch(self): + """Test on non-x86 architecture""" + data = self._DoReadFile('103_cbfs_raw_ppc.dts') + size = 0x100 + + cbfs = cbfs_util.CbfsReader(data) + self.assertEqual(size, cbfs.rom_size) + + self.assertIn('u-boot-dtb', cbfs.files) + cfile = cbfs.files['u-boot-dtb'] + self.assertEqual(U_BOOT_DTB_DATA, cfile.data) + + def testCbfsStage(self): + """Tests handling of a Coreboot Filesystem (CBFS)""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') + elf_fname = os.path.join(self._indir, 'cbfs-stage.elf') + elf.MakeElf(elf_fname, U_BOOT_DATA, U_BOOT_DTB_DATA) + size = 0xb0 + + data = self._DoReadFile('104_cbfs_stage.dts') + cbfs = cbfs_util.CbfsReader(data) + self.assertEqual(size, cbfs.rom_size) + + self.assertIn('u-boot', cbfs.files) + cfile = cbfs.files['u-boot'] + self.assertEqual(U_BOOT_DATA + U_BOOT_DTB_DATA, cfile.data) + + def testCbfsRawCompress(self): + """Test handling of compressing raw files""" + self._CheckLz4() + data = self._DoReadFile('105_cbfs_raw_compress.dts') + size = 0x140 + + cbfs = cbfs_util.CbfsReader(data) + self.assertIn('u-boot', cbfs.files) + cfile = cbfs.files['u-boot'] + self.assertEqual(COMPRESS_DATA, cfile.data) + + def testCbfsBadArch(self): + """Test handling of a bad architecture""" + with self.assertRaises(ValueError) as e: + self._DoReadFile('106_cbfs_bad_arch.dts') + self.assertIn("Invalid architecture 'bad-arch'", str(e.exception)) + + def testCbfsNoSize(self): + """Test handling of a missing size property""" + with self.assertRaises(ValueError) as e: + self._DoReadFile('107_cbfs_no_size.dts') + self.assertIn('entry must have a size property', str(e.exception)) + + def testCbfsNoCOntents(self): + """Test handling of a CBFS entry which does not provide contentsy""" + with self.assertRaises(ValueError) as e: + self._DoReadFile('108_cbfs_no_contents.dts') + self.assertIn('Could not complete processing of contents', + str(e.exception)) + + def testCbfsBadCompress(self): + """Test handling of a bad architecture""" + with self.assertRaises(ValueError) as e: + self._DoReadFile('109_cbfs_bad_compress.dts') + self.assertIn("Invalid compression in 'u-boot': 'invalid-algo'", + str(e.exception)) + + def testCbfsNamedEntries(self): + """Test handling of named entries""" + data = self._DoReadFile('110_cbfs_name.dts') + + cbfs = cbfs_util.CbfsReader(data) + self.assertIn('FRED', cbfs.files) + cfile1 = cbfs.files['FRED'] + self.assertEqual(U_BOOT_DATA, cfile1.data) + + self.assertIn('hello', cbfs.files) + cfile2 = cbfs.files['hello'] + self.assertEqual(U_BOOT_DTB_DATA, cfile2.data) + if __name__ == "__main__": unittest.main() diff --git a/tools/binman/test/102_cbfs_raw.dts b/tools/binman/test/102_cbfs_raw.dts new file mode 100644 index 00000000000..779cbc121ad --- /dev/null +++ b/tools/binman/test/102_cbfs_raw.dts @@ -0,0 +1,20 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + cbfs { + size = <0xb0>; + u-boot { + cbfs-type = "raw"; + }; + u-boot-dtb { + cbfs-type = "raw"; + }; + }; + }; +}; diff --git a/tools/binman/test/103_cbfs_raw_ppc.dts b/tools/binman/test/103_cbfs_raw_ppc.dts new file mode 100644 index 00000000000..df1caf092f4 --- /dev/null +++ b/tools/binman/test/103_cbfs_raw_ppc.dts @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + cbfs { + size = <0x100>; + cbfs-arch = "ppc64"; + u-boot { + cbfs-type = "raw"; + }; + u-boot-dtb { + cbfs-type = "raw"; + }; + }; + }; +}; diff --git a/tools/binman/test/104_cbfs_stage.dts b/tools/binman/test/104_cbfs_stage.dts new file mode 100644 index 00000000000..215e2f287a4 --- /dev/null +++ b/tools/binman/test/104_cbfs_stage.dts @@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + cbfs { + size = <0xb0>; + u-boot { + type = "blob"; + filename = "cbfs-stage.elf"; + cbfs-type = "stage"; + }; + }; + }; +}; diff --git a/tools/binman/test/105_cbfs_raw_compress.dts b/tools/binman/test/105_cbfs_raw_compress.dts new file mode 100644 index 00000000000..646168d84b4 --- /dev/null +++ b/tools/binman/test/105_cbfs_raw_compress.dts @@ -0,0 +1,26 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + cbfs { + size = <0x140>; + u-boot { + type = "text"; + text = "compress xxxxxxxxxxxxxxxxxxxxxx data"; + cbfs-type = "raw"; + cbfs-compress = "lz4"; + }; + u-boot-dtb { + type = "text"; + text = "compress xxxxxxxxxxxxxxxxxxxxxx data"; + cbfs-type = "raw"; + cbfs-compress = "lzma"; + }; + }; + }; +}; diff --git a/tools/binman/test/106_cbfs_bad_arch.dts b/tools/binman/test/106_cbfs_bad_arch.dts new file mode 100644 index 00000000000..4318d45a7d4 --- /dev/null +++ b/tools/binman/test/106_cbfs_bad_arch.dts @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + cbfs { + size = <0x100>; + cbfs-arch = "bad-arch"; + }; + }; +}; diff --git a/tools/binman/test/107_cbfs_no_size.dts b/tools/binman/test/107_cbfs_no_size.dts new file mode 100644 index 00000000000..3592f62f7e6 --- /dev/null +++ b/tools/binman/test/107_cbfs_no_size.dts @@ -0,0 +1,13 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + cbfs { + }; + }; +}; diff --git a/tools/binman/test/108_cbfs_no_contents.dts b/tools/binman/test/108_cbfs_no_contents.dts new file mode 100644 index 00000000000..623346760d2 --- /dev/null +++ b/tools/binman/test/108_cbfs_no_contents.dts @@ -0,0 +1,17 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + cbfs { + size = <0x100>; + _testing { + return-unknown-contents; + }; + }; + }; +}; diff --git a/tools/binman/test/109_cbfs_bad_compress.dts b/tools/binman/test/109_cbfs_bad_compress.dts new file mode 100644 index 00000000000..9695024ee9b --- /dev/null +++ b/tools/binman/test/109_cbfs_bad_compress.dts @@ -0,0 +1,18 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + cbfs { + size = <0xb0>; + u-boot { + cbfs-type = "raw"; + cbfs-compress = "invalid-algo"; + }; + }; + }; +}; diff --git a/tools/binman/test/110_cbfs_name.dts b/tools/binman/test/110_cbfs_name.dts new file mode 100644 index 00000000000..98c16f30b41 --- /dev/null +++ b/tools/binman/test/110_cbfs_name.dts @@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + cbfs { + size = <0x100>; + u-boot { + cbfs-name = "FRED"; + cbfs-type = "raw"; + }; + + hello { + type = "blob"; + filename = "u-boot.dtb"; + cbfs-type = "raw"; + }; + }; + }; +}; From patchwork Mon Jul 8 19:18:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129291 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="dPrSxL6q"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jG0b0FPmz9sDB for ; Tue, 9 Jul 2019 05:36:30 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id AA6FEC21DF3; Mon, 8 Jul 2019 19:30:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 67FB3C21EC2; Mon, 8 Jul 2019 19:22:03 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 29549C21DD9; Mon, 8 Jul 2019 19:21:29 +0000 (UTC) Received: from mail-io1-f46.google.com (mail-io1-f46.google.com [209.85.166.46]) by lists.denx.de (Postfix) with ESMTPS id 6DC59C21DF9 for ; Mon, 8 Jul 2019 19:21:25 +0000 (UTC) Received: by mail-io1-f46.google.com with SMTP id f4so22333592ioh.6 for ; Mon, 08 Jul 2019 12:21:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=H7cV3QS02oQoN32NXEe9pzNmdA4JMaQPMIAhmyI6vW4=; b=dPrSxL6qqPGjZerplqVZIJ3rD+WOWVLFGMOz9xT931HYASqr8I7IaKwuq9NnWH2238 uXTZmoy2JzXb8nDkLs1BLp5G/ZBU7fo+lNRw8yqf3S4Ho9qbPDNtI5jspeeu0ZRs27ik 6jW5ZcatsYoA/uPwSZ48tbwxK1gDbdr1H4xto= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=H7cV3QS02oQoN32NXEe9pzNmdA4JMaQPMIAhmyI6vW4=; b=bvSqD9b0YDRCppgy3jCAwTYxA87oEhVc4bqNn3j9BWJAraAogmZze9UlKIo0qdeSE5 XQoUvNyrbWSYdPmKDedYijXvOr76lAGtUtCje+uDS4zhaPrwKQ1hicxa84VbM/07DIdm mSuGz3kFOtUi8DGe58lBguMDQGGM+MxBtuw6kYNa3y+CU5V5cBcv3kmheEqgVgasv6Rf gUnjlTUxPccTKhkAR1je74S3zp+2NqK819jM1KR3TinK08P4+vhuYx/H/w1hb0K2ME5h PRuMU3xELMhrT98tUa0H1HDxjFXkZKNN3/jocUdsghHGpzr6Kkvbzxzd1Q4JyTs+OUm7 2YLQ== X-Gm-Message-State: APjAAAVh06EpcibI8x+G5APHn4eYgj6OUPIiAhevAUyawGY7czHuvkMe H/9qpzeKWolOcRC1h8QnkHDSfzpa84A= X-Google-Smtp-Source: APXvYqwsEabzF1Uy848XAOfeMXHnogBXtPuGnLGOgXn3JlKLj0RXgJYDaZ6avURnOmbO7RaDb9/dog== X-Received: by 2002:a02:380c:: with SMTP id b12mr23557964jaa.85.1562613684100; Mon, 08 Jul 2019 12:21:24 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.23 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:23 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:54 -0600 Message-Id: <20190708191856.138863-36-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 35/37] binman: Add support for Intel IFWI entries X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" An Integrated Firmware Image is used to hold various binaries used for booting with Apollolake and some later devices. Add support for this. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/etype/intel_descriptor.py | 6 +- tools/binman/etype/intel_ifwi.py | 100 ++++++++++++++++++ tools/binman/ftest.py | 55 ++++++++++ tools/binman/test/111_x86-rom-ifwi.dts | 29 +++++ tools/binman/test/112_x86-rom-ifwi-nodesc.dts | 28 +++++ tools/binman/test/113_x86-rom-ifwi-nodata.dts | 29 +++++ tools/binman/test/fitimage.bin.gz | Bin 0 -> 8418 bytes tools/binman/test/ifwi.bin.gz | Bin 0 -> 1884 bytes 8 files changed, 245 insertions(+), 2 deletions(-) create mode 100644 tools/binman/etype/intel_ifwi.py create mode 100644 tools/binman/test/111_x86-rom-ifwi.dts create mode 100644 tools/binman/test/112_x86-rom-ifwi-nodesc.dts create mode 100644 tools/binman/test/113_x86-rom-ifwi-nodata.dts create mode 100644 tools/binman/test/fitimage.bin.gz create mode 100644 tools/binman/test/ifwi.bin.gz GIT binary patch literal 1884 zcmb2|=HQs3%ooeRoS9ahsh5ZAh64*sw_Dxbd0ww8D>=P* z^W@~OUw-I(Z)(?+JN{~)!PWBLwn@7MOY`Hd->*LY?P+wqeChi7+oxY`ZC%|RqCGwQ zUGTj-xATuq*m6Jp$^K}cjdFXdw_ILdTb}>t%TLSq^LW0RGXTNpXfuSFnfC2L-b8%e#XaJ3dz-S1J%n+#C OAO70lJukxo2?hX1K&cV{ literal 0 HcmV?d00001 diff --git a/tools/binman/etype/intel_descriptor.py b/tools/binman/etype/intel_descriptor.py index 65ba2391e69..adea578080c 100644 --- a/tools/binman/etype/intel_descriptor.py +++ b/tools/binman/etype/intel_descriptor.py @@ -60,10 +60,12 @@ class Entry_intel_descriptor(Entry_blob): for i in range(MAX_REGIONS): self._regions.append(Region(self.data, frba, i)) - # Set the offset for ME (Management Engine) only, for now, since the - # others are not used + # Set the offset for ME (Management Engine) and IFWI (Integrated + # Firmware Image), for now, since the others are not used. info = {} if self.HasSibling('intel-me'): info['intel-me'] = [self._regions[REGION_ME].base, self._regions[REGION_ME].size] + if self.HasSibling('intel-ifwi'): + info['intel-ifwi'] = [self._regions[REGION_BIOS].base, None] return info diff --git a/tools/binman/etype/intel_ifwi.py b/tools/binman/etype/intel_ifwi.py new file mode 100644 index 00000000000..8c79b2dd291 --- /dev/null +++ b/tools/binman/etype/intel_ifwi.py @@ -0,0 +1,100 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2016 Google, Inc +# Written by Simon Glass +# +# Entry-type module for Intel Management Engine binary blob +# + +from collections import OrderedDict + +from entry import Entry +from blob import Entry_blob +import fdt_util +import tools + +class Entry_intel_ifwi(Entry_blob): + """Entry containing an Intel Integrated Firmware Image (IFWI) file + + Properties / Entry arguments: + - filename: Filename of file to read into entry. This is either the + IFWI file itself, or a file that can be converted into one using a + tool + - convert-fit: If present this indicates that the ifwitool should be + used to convert the provided file into a IFWI. + + This file contains code and data used by the SoC that is required to make + it work. It includes U-Boot TPL, microcode, things related to the CSE + (Converged Security Engine, the microcontroller that loads all the firmware) + and other items beyond the wit of man. + + A typical filename is 'ifwi.bin' for an IFWI file, or 'fitimage.bin' for a + file that will be converted to an IFWI. + + The position of this entry is generally set by the intel-descriptor entry. + + The contents of the IFWI are specified by the subnodes of the IFWI node. + Each subnode describes an entry which is placed into the IFWFI with a given + sub-partition (and optional entry name). + + See README.x86 for information about x86 binary blobs. + """ + def __init__(self, section, etype, node): + Entry_blob.__init__(self, section, etype, node) + self._convert_fit = fdt_util.GetBool(self._node, 'convert-fit') + self._ifwi_entries = OrderedDict() + self._ReadSubnodes() + + def ObtainContents(self): + """Get the contects for the IFWI + + Unfortunately we cannot create anything from scratch here, as Intel has + tools which create precursor binaries with lots of data and settings, + and these are not incorporated into binman. + + The first step is to get a file in the IFWI format. This is either + supplied directly or is extracted from a fitimage using the 'create' + subcommand. + + After that we delete the OBBP sub-partition and add each of the files + that we want in the IFWI file, one for each sub-entry of the IWFI node. + """ + self._pathname = tools.GetInputFilename(self._filename) + + # Create the IFWI file if needed + if self._convert_fit: + inname = self._pathname + outname = tools.GetOutputFilename('ifwi.bin') + tools.RunIfwiTool(inname, tools.CMD_CREATE, outname) + self._filename = 'ifwi.bin' + self._pathname = outname + else: + # Provide a different code path here to ensure we have test coverage + inname = self._pathname + + # Delete OBBP if it is there, then add the required new items. + tools.RunIfwiTool(inname, tools.CMD_DELETE, subpart='OBBP') + + for entry in self._ifwi_entries.values(): + # First get the input data and put it in a file + if not entry.ObtainContents(): + return False + data = entry.GetData() + uniq = self.GetUniqueName() + input_fname = tools.GetOutputFilename('input.%s' % uniq) + tools.WriteFile(input_fname, data) + + tools.RunIfwiTool(inname, + tools.CMD_REPLACE if entry._ifwi_replace else tools.CMD_ADD, + input_fname, entry._ifwi_subpart, entry._ifwi_entry_name) + + self.ReadBlobContents() + return True + + def _ReadSubnodes(self): + """Read the subnodes to find out what should go in this IFWI""" + for node in self._node.subnodes: + entry = Entry.Create(self.section, node) + entry._ifwi_replace = fdt_util.GetBool(node, 'replace') + entry._ifwi_subpart = fdt_util.GetString(node, 'ifwi-subpart') + entry._ifwi_entry_name = fdt_util.GetString(node, 'ifwi-entry') + self._ifwi_entries[entry._ifwi_subpart] = entry diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 14abfbf774f..1355c4f55d3 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -27,6 +27,7 @@ import fdt import fdt_util import fmap_util import test_util +import gzip import state import tools import tout @@ -876,6 +877,9 @@ class TestFunctional(unittest.TestCase): def testPackX86RomMe(self): """Test that an x86 ROM with an ME region can be created""" data = self._DoReadFile('031_x86-rom-me.dts') + expected_desc = tools.ReadFile(self.TestFile('descriptor.bin')) + if data[:0x1000] != expected_desc: + self.fail('Expected descriptor binary at start of image') self.assertEqual(ME_DATA, data[0x1000:0x1000 + len(ME_DATA)]) def testPackVga(self): @@ -1956,6 +1960,57 @@ class TestFunctional(unittest.TestCase): cfile2 = cbfs.files['hello'] self.assertEqual(U_BOOT_DTB_DATA, cfile2.data) + def _SetupIfwi(self, fname): + """Set up to run an IFWI test + + Args: + fname: Filename of input file to provide (fitimage.bin or ifwi.bin) + """ + self._SetupSplElf() + + # Intel Integrated Firmware Image (IFWI) file + with gzip.open(self.TestFile('%s.gz' % fname), 'rb') as fd: + data = fd.read() + TestFunctional._MakeInputFile(fname,data) + + def _CheckIfwi(self, data): + """Check that an image with an IFWI contains the correct output + + Args: + data: Conents of output file + """ + expected_desc = tools.ReadFile(self.TestFile('descriptor.bin')) + if data[:0x1000] != expected_desc: + self.fail('Expected descriptor binary at start of image') + + # We expect to find the TPL wil in subpart IBBP entry IBBL + image_fname = tools.GetOutputFilename('image.bin') + tpl_fname = tools.GetOutputFilename('tpl.out') + tools.RunIfwiTool(image_fname, tools.CMD_EXTRACT, fname=tpl_fname, + subpart='IBBP', entry_name='IBBL') + + tpl_data = tools.ReadFile(tpl_fname) + self.assertEqual(tpl_data[:len(U_BOOT_TPL_DATA)], U_BOOT_TPL_DATA) + + def testPackX86RomIfwi(self): + """Test that an x86 ROM with Integrated Firmware Image can be created""" + self._SetupIfwi('fitimage.bin') + data = self._DoReadFile('111_x86-rom-ifwi.dts') + self._CheckIfwi(data) + + def testPackX86RomIfwiNoDesc(self): + """Test that an x86 ROM with IFWI can be created from an ifwi.bin file""" + self._SetupIfwi('ifwi.bin') + data = self._DoReadFile('112_x86-rom-ifwi-nodesc.dts') + self._CheckIfwi(data) + + def testPackX86RomIfwiNoData(self): + """Test that an x86 ROM with IFWI handles missing data""" + self._SetupIfwi('ifwi.bin') + with self.assertRaises(ValueError) as e: + data = self._DoReadFile('113_x86-rom-ifwi-nodata.dts') + self.assertIn('Could not complete processing of contents', + str(e.exception)) if __name__ == "__main__": unittest.main() diff --git a/tools/binman/test/111_x86-rom-ifwi.dts b/tools/binman/test/111_x86-rom-ifwi.dts new file mode 100644 index 00000000000..63b5972cc8e --- /dev/null +++ b/tools/binman/test/111_x86-rom-ifwi.dts @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + sort-by-offset; + end-at-4gb; + size = <0x800000>; + intel-descriptor { + filename = "descriptor.bin"; + }; + + intel-ifwi { + offset-unset; + filename = "fitimage.bin"; + convert-fit; + + u-boot-tpl { + replace; + ifwi-subpart = "IBBP"; + ifwi-entry = "IBBL"; + }; + }; + }; +}; diff --git a/tools/binman/test/112_x86-rom-ifwi-nodesc.dts b/tools/binman/test/112_x86-rom-ifwi-nodesc.dts new file mode 100644 index 00000000000..21ec4654ffe --- /dev/null +++ b/tools/binman/test/112_x86-rom-ifwi-nodesc.dts @@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + sort-by-offset; + end-at-4gb; + size = <0x800000>; + intel-descriptor { + filename = "descriptor.bin"; + }; + + intel-ifwi { + offset-unset; + filename = "ifwi.bin"; + + u-boot-tpl { + replace; + ifwi-subpart = "IBBP"; + ifwi-entry = "IBBL"; + }; + }; + }; +}; diff --git a/tools/binman/test/113_x86-rom-ifwi-nodata.dts b/tools/binman/test/113_x86-rom-ifwi-nodata.dts new file mode 100644 index 00000000000..62486fd990e --- /dev/null +++ b/tools/binman/test/113_x86-rom-ifwi-nodata.dts @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + sort-by-offset; + end-at-4gb; + size = <0x800000>; + intel-descriptor { + filename = "descriptor.bin"; + }; + + intel-ifwi { + offset-unset; + filename = "ifwi.bin"; + + _testing { + return-unknown-contents; + replace; + ifwi-subpart = "IBBP"; + ifwi-entry = "IBBL"; + }; + }; + }; +}; diff --git a/tools/binman/test/fitimage.bin.gz b/tools/binman/test/fitimage.bin.gz new file mode 100644 index 0000000000000000000000000000000000000000..0a9dcfc424845c89436f119eeda8983e12191364 GIT binary patch literal 8418 zcmb2|=HM`q;)`WqPRlIG%uP&B)l16EV|aVpI$I=Ag6%iwic{C_dF=b~^?`Hc!()y2E=_)}AHVn9-x@!`?a$ax zzr0fzJx_kkYyR8jE{kOgS41Cs@VMwo&FeqCWk3Jr%+r+;%-1{H_bktB+NF)6b33x4 z?USzmm^aJ#Y{$DD#{X{4Ib_Ss!0@N1-23+epq8>v6)m>F|_ozktf7elWHSAVcnPA}eC<;S6z{YtX#wvvQZDZ@j17wrsB+0|SF}?!$`zF6VB#UbHIP zbp7@7vLCOGW?OR4^M31Ud8#$@&lMHrGHJt8&lZ2&`sT)7<8$91@9(yk30u6kb@%qm z5nI>)HMz>Y{Cr8$a$i lx9Z%On{wd&ldm8bY};eMGaRUj;m X-Patchwork-Id: 1129303 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="RqZxQqeB"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jG676yZqz9sDB for ; Tue, 9 Jul 2019 05:41:19 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id A5C61C21E39; Mon, 8 Jul 2019 19:35:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 05AB8C21EF7; Mon, 8 Jul 2019 19:22:22 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id B3C4AC21E3B; Mon, 8 Jul 2019 19:21:30 +0000 (UTC) Received: from mail-io1-f65.google.com (mail-io1-f65.google.com [209.85.166.65]) by lists.denx.de (Postfix) with ESMTPS id 09C9FC21E42 for ; Mon, 8 Jul 2019 19:21:26 +0000 (UTC) Received: by mail-io1-f65.google.com with SMTP id f4so22333640ioh.6 for ; Mon, 08 Jul 2019 12:21:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OfPasFjPK9tdHOxSESZian5WbpU38sSxbBZzKA1EOKs=; b=RqZxQqeBTr76XitSlOjs5dFaUUSTdR12qhVkNAvdQKwq1PSoXxAub6iZbMaOv0OnuA r5PPssw2CPwt5Dr8IaK2zIvecCF6QhiR7pdVoxv8nbiFhsQTUTCo7tygV9qQvmpU5P9H oRK2HVNehoyuwJsqmkDeNSx56UT8z2DG+G5dY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OfPasFjPK9tdHOxSESZian5WbpU38sSxbBZzKA1EOKs=; b=RU58G6U/nQKrH81o6Zy/WCU34xduPUyuZZ2hIfoSGkVGnP6+z22O1HnKjDEWD4lJHr /qxCTw0R4GacIKmcuw1A7YI7r6EqlnT9AyAFI4A80/CM+TOe1XDmvtakZEJtcFUtfeyZ +inMrrYrKpsg1WTl2SoHN0s7ps4jTIti932KhZ4esum7S+4oObApkfLBQF7FarchT7GX DyZqT8vG+BxJLgISr926y+PnTNhV0ChpDKiFyTZg0/FD7usT15bkeoAORAvci1NakkBS k697Q07rFg1CVrZx+wWSRqfuhgF7/jjXxPqKc0Wrel46QXGfsRuFJVdVPkHW/5pn/jmb r2lg== X-Gm-Message-State: APjAAAU7R2OzV9BcqcNQxydSFUZD8G7EcAmp8KKMM7cYjz0yg8lzTSbC GICitQux51t+Ol+FGr2gI9raxp0Fmp4= X-Google-Smtp-Source: APXvYqyiJ9vP1o8haVr10njPsIqtt0z//4MQtKIQ01JmZJJHpKBXwvq+BEzQ3RdCAeFR6av+4YPHyw== X-Received: by 2002:a5e:c744:: with SMTP id g4mr19807779iop.187.1562613684632; Mon, 08 Jul 2019 12:21:24 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.24 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:24 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:55 -0600 Message-Id: <20190708191856.138863-37-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 36/37] binman: Pad empty areas of the CBFS with files X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" When there is lots of open space in a CBFS it is normally padded with 'empty' files so that sequentially scanning the CBFS can skip from one to the next without a break. Add support for this. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/binman/cbfs_util.py | 68 ++++++++++++++++++++++++++++++++-- tools/binman/cbfs_util_test.py | 23 +++++++++++- 2 files changed, 87 insertions(+), 4 deletions(-) diff --git a/tools/binman/cbfs_util.py b/tools/binman/cbfs_util.py index 197cff89509..ec4a2e5a8c6 100644 --- a/tools/binman/cbfs_util.py +++ b/tools/binman/cbfs_util.py @@ -11,7 +11,8 @@ The format is somewhat defined by documentation in the coreboot tree although it is necessary to rely on the C structures and source code (mostly cbfstool) to fully understand it. -Currently supported: raw and stage types with compression +Currently supported: raw and stage types with compression, padding empty areas + with empty files """ from __future__ import print_function @@ -102,6 +103,7 @@ ARCH_NAMES = { TYPE_CBFSHEADER = 0x02 # Master header, HEADER_FORMAT TYPE_STAGE = 0x10 # Stage, holding an executable, see STAGE_FORMAT TYPE_RAW = 0x50 # Raw file, possibly compressed +TYPE_EMPTY = 0xffffffff # Empty data # Compression types COMPRESS_NONE, COMPRESS_LZMA, COMPRESS_LZ4 = range(3) @@ -152,6 +154,19 @@ def align_int(val, align): """ return int((val + align - 1) / align) * align +def align_int_down(val, align): + """Align a value down to the given alignment + + Args: + val: Integer value to align + align: Integer alignment value (e.g. 4 to align to 4-byte boundary) + + Returns: + integer value aligned to the required boundary, rounding down if + necessary + """ + return int(val / align) * align + def _pack_string(instr): """Pack a string to the required aligned size by adding padding @@ -184,6 +199,9 @@ class CbfsFile(object): entry: Entry address in memory if known, else None. This is where execution starts after the file is loaded base_address: Base address to use for 'stage' files + erase_byte: Erase byte to use for padding between the file header and + contents (used for empty files) + size: Size of the file in bytes (used for empty files) """ def __init__(self, name, ftype, data, compress=COMPRESS_NONE): self.name = name @@ -196,6 +214,8 @@ class CbfsFile(object): self.entry = None self.base_address = None self.data_len = 0 + self.erase_byte = None + self.size = None def decompress(self): """Handle decompressing data if necessary""" @@ -242,6 +262,24 @@ class CbfsFile(object): """ return CbfsFile(name, TYPE_RAW, data, compress) + @classmethod + def empty(cls, space_to_use, erase_byte): + """Create a new empty file of a given size + + Args: + space_to_use:: Size of available space, which must be at least as + large as the alignment size for this CBFS + erase_byte: Byte to use for contents of file (repeated through the + whole file) + + Returns: + CbfsFile object containing the file information + """ + cfile = CbfsFile('', TYPE_EMPTY, b'') + cfile.size = space_to_use - FILE_HEADER_LEN - FILENAME_ALIGN + cfile.erase_byte = erase_byte + return cfile + def get_data(self): """Obtain the contents of the file, in CBFS format @@ -270,6 +308,8 @@ class CbfsFile(object): attr = struct.pack(ATTR_COMPRESSION_FORMAT, FILE_ATTR_TAG_COMPRESSION, ATTR_COMPRESSION_LEN, self.compress, len(orig_data)) + elif self.ftype == TYPE_EMPTY: + data = tools.GetBytes(self.erase_byte, self.size) else: raise ValueError('Unknown type %#x when writing\n' % self.ftype) if attr: @@ -357,6 +397,24 @@ class CbfsWriter(object): (offset, fd.tell())) fd.write(tools.GetBytes(self._erase_byte, offset - fd.tell())) + def _pad_to(self, fd, offset): + """Write out pad bytes and/or an empty file until a given offset + + Args: + fd: File objext to write to + offset: Offset to write to + """ + self._align_to(fd, self._align) + upto = fd.tell() + if upto > offset: + raise ValueError('No space for data before pad offset %#x (current offset %#x)' % + (offset, upto)) + todo = align_int_down(offset - upto, self._align) + if todo: + cbf = CbfsFile.empty(todo, self._erase_byte) + fd.write(cbf.get_data()) + self._skip_to(fd, offset) + def _align_to(self, fd, align): """Write out pad bytes until a given alignment is reached @@ -416,7 +474,7 @@ class CbfsWriter(object): raise ValueError('No space for header at offset %#x (current offset %#x)' % (self._header_offset, fd.tell())) if not add_fileheader: - self._skip_to(fd, self._header_offset) + self._pad_to(fd, self._header_offset) hdr = struct.pack(HEADER_FORMAT, HEADER_MAGIC, HEADER_VERSION2, self._size, self._bootblock_size, self._align, self._contents_offset, self._arch, 0xffffffff) @@ -455,7 +513,7 @@ class CbfsWriter(object): self._write_header(fd, add_fileheader=self._add_fileheader) # Pad to the end and write a pointer to the CBFS master header - self._skip_to(fd, self._base_address or self._size - 4) + self._pad_to(fd, self._base_address or self._size - 4) rel_offset = self._header_offset - self._size fd.write(struct.pack(' X-Patchwork-Id: 1129304 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="M0eob1Tf"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jG7D6P83z9sNH for ; Tue, 9 Jul 2019 05:42:16 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 8BFE3C21E74; Mon, 8 Jul 2019 19:34:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id BC64EC21F00; Mon, 8 Jul 2019 19:22:19 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 05183C21E3B; Mon, 8 Jul 2019 19:21:31 +0000 (UTC) Received: from mail-io1-f67.google.com (mail-io1-f67.google.com [209.85.166.67]) by lists.denx.de (Postfix) with ESMTPS id 13911C21E30 for ; Mon, 8 Jul 2019 19:21:27 +0000 (UTC) Received: by mail-io1-f67.google.com with SMTP id j5so18475881ioj.8 for ; Mon, 08 Jul 2019 12:21:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AL6lIAXGG4SW/fExettlcPLdaJE/xReWfsnaNuQ/u3I=; b=M0eob1TfF5+AMJ4/ZD45j90efYb4hr+1/k4ZqkTVRnuieUXvrG3jV6wCXQmnu9ogbv FyQzXoVUJ47rxv0+kw+GIf7I4Xdnj5cvl7bzl08xAhRRXpJgpcA97x/dHwFWKJdU4Hxg SL/e8MuSwaGoDxYs2i6t0M6EpQ9f6NoqxQGds= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AL6lIAXGG4SW/fExettlcPLdaJE/xReWfsnaNuQ/u3I=; b=QlQxAwVUZSk1hlHq0wsaR4gMT3O3VXoj7sGponcq5NVxQIyhu1ebT3fib1GhG78bf2 PRptB5KTD2MslZ1+2rhHm2pFoql35a3u9mBW8VKa/pCz0JCoLCJXOFyNjvDr91YsqmRZ 1kODt2TaI/iSlDiLXB/+8TYB7Cn7SPQk65rAihbzb73+qIU1rb2ICXNHiX8jE0qETdKW 08q1FBWXAgwb0DP5qng4c6onV3CgmYrrN4RASdo+ciIs9+Jr6iCoQZKraTMwHNWm9tJp E2Xs9l5Zk2T09PEabANnkxeXWYxniPa5PJejFM2Uj8N+bKwZh8bKa3jDfhsjJhgPbYhY bkyw== X-Gm-Message-State: APjAAAUzOlRHTxBWDGi0SLvns1TpP/MFkt4vY00mynNEygu5LvH/jazT EB/hDyrUBQNavBHBiO9YbqDyw0UOnGA= X-Google-Smtp-Source: APXvYqx8Ti+7e1LxUg8hbox7IGILIg6tJuXD6eU0isEcu2AhMi0kZjDvm0EoFhkeyy6QdVIKhMiBeA== X-Received: by 2002:a5e:8210:: with SMTP id l16mr21224207iom.240.1562613685508; Mon, 08 Jul 2019 12:21:25 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id j1sm15545094iop.14.2019.07.08.12.21.24 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 12:21:24 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 13:18:56 -0600 Message-Id: <20190708191856.138863-38-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708191856.138863-1-sjg@chromium.org> References: <20190708191856.138863-1-sjg@chromium.org> MIME-Version: 1.0 Subject: [U-Boot] [PATCH v2 37/37] binman: Add support for fixed-offset files in CBFS X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" A feature of CBFS is that it allows files to be positioned at particular offset (as with binman in general). This is useful to support execute-in-place (XIP) code, since this may not be relocatable. Add a new cbfs-offset property to control this. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: - Deal with travis's old lz4 version by skipping tests as necessary - Skip use of cbfstool in tests if it is not available tools/binman/README.entries | 38 ++++++++ tools/binman/cbfs_util.py | 125 +++++++++++++++++++++----- tools/binman/cbfs_util_test.py | 82 +++++++++++++++-- tools/binman/etype/cbfs.py | 16 +++- tools/binman/ftest.py | 23 +++++ tools/binman/test/114_cbfs_offset.dts | 26 ++++++ 6 files changed, 276 insertions(+), 34 deletions(-) create mode 100644 tools/binman/test/114_cbfs_offset.dts diff --git a/tools/binman/README.entries b/tools/binman/README.entries index 2e6aea1e84c..3241befc7f4 100644 --- a/tools/binman/README.entries +++ b/tools/binman/README.entries @@ -100,6 +100,7 @@ with the second subnode below: filename = "u-boot.dtb"; cbfs-type = "raw"; cbfs-compress = "lz4"; + cbfs-offset = <0x100000>; }; }; @@ -158,6 +159,15 @@ cbfs-type: to add a flat binary with a load/start address, similar to the 'add-flat-binary' option in cbfstool. +cbfs-offset: + This is the offset of the file's data within the CBFS. It is used to + specify where the file should be placed in cases where a fixed position + is needed. Typical uses are for code which is not relocatable and must + execute in-place from a particular address. This works because SPI flash + is generally mapped into memory on x86 devices. The file header is + placed before this offset so that the data start lines up exactly with + the chosen offset. If this property is not provided, then the file is + placed in the next available spot. The current implementation supports only a subset of CBFS features. It does not support other file types (e.g. payload), adding multiple files (like the @@ -334,6 +344,34 @@ See README.x86 for information about x86 binary blobs. +Entry: intel-ifwi: Entry containing an Intel Integrated Firmware Image (IFWI) file +---------------------------------------------------------------------------------- + +Properties / Entry arguments: + - filename: Filename of file to read into entry. This is either the + IFWI file itself, or a file that can be converted into one using a + tool + - convert-fit: If present this indicates that the ifwitool should be + used to convert the provided file into a IFWI. + +This file contains code and data used by the SoC that is required to make +it work. It includes U-Boot TPL, microcode, things related to the CSE +(Converged Security Engine, the microcontroller that loads all the firmware) +and other items beyond the wit of man. + +A typical filename is 'ifwi.bin' for an IFWI file, or 'fitimage.bin' for a +file that will be converted to an IFWI. + +The position of this entry is generally set by the intel-descriptor entry. + +The contents of the IFWI are specified by the subnodes of the IFWI node. +Each subnode describes an entry which is placed into the IFWFI with a given +sub-partition (and optional entry name). + +See README.x86 for information about x86 binary blobs. + + + Entry: intel-me: Entry containing an Intel Management Engine (ME) file ---------------------------------------------------------------------- diff --git a/tools/binman/cbfs_util.py b/tools/binman/cbfs_util.py index ec4a2e5a8c6..1cdbcb2339e 100644 --- a/tools/binman/cbfs_util.py +++ b/tools/binman/cbfs_util.py @@ -12,7 +12,7 @@ it is necessary to rely on the C structures and source code (mostly cbfstool) to fully understand it. Currently supported: raw and stage types with compression, padding empty areas - with empty files + with empty files, fixed-offset files """ from __future__ import print_function @@ -190,6 +190,8 @@ class CbfsFile(object): Properties: name: Name of file offset: Offset of file data from start of file header + cbfs_offset: Offset of file data in bytes from start of CBFS, or None to + place this file anyway data: Contents of file, uncompressed data_len: Length of (possibly compressed) data in bytes ftype: File type (TYPE_...) @@ -203,9 +205,10 @@ class CbfsFile(object): contents (used for empty files) size: Size of the file in bytes (used for empty files) """ - def __init__(self, name, ftype, data, compress=COMPRESS_NONE): + def __init__(self, name, ftype, data, cbfs_offset, compress=COMPRESS_NONE): self.name = name self.offset = None + self.cbfs_offset = cbfs_offset self.data = data self.ftype = ftype self.compress = compress @@ -231,7 +234,7 @@ class CbfsFile(object): self.data_len = len(indata) @classmethod - def stage(cls, base_address, name, data): + def stage(cls, base_address, name, data, cbfs_offset): """Create a new stage file Args: @@ -239,28 +242,32 @@ class CbfsFile(object): name: String file name to put in CBFS (does not need to correspond to the name that the file originally came from) data: Contents of file + cbfs_offset: Offset of file data in bytes from start of CBFS, or + None to place this file anyway Returns: CbfsFile object containing the file information """ - cfile = CbfsFile(name, TYPE_STAGE, data) + cfile = CbfsFile(name, TYPE_STAGE, data, cbfs_offset) cfile.base_address = base_address return cfile @classmethod - def raw(cls, name, data, compress): + def raw(cls, name, data, cbfs_offset, compress): """Create a new raw file Args: name: String file name to put in CBFS (does not need to correspond to the name that the file originally came from) data: Contents of file + cbfs_offset: Offset of file data in bytes from start of CBFS, or + None to place this file anyway compress: Compression algorithm to use (COMPRESS_...) Returns: CbfsFile object containing the file information """ - return CbfsFile(name, TYPE_RAW, data, compress) + return CbfsFile(name, TYPE_RAW, data, cbfs_offset, compress) @classmethod def empty(cls, space_to_use, erase_byte): @@ -275,12 +282,44 @@ class CbfsFile(object): Returns: CbfsFile object containing the file information """ - cfile = CbfsFile('', TYPE_EMPTY, b'') + cfile = CbfsFile('', TYPE_EMPTY, b'', None) cfile.size = space_to_use - FILE_HEADER_LEN - FILENAME_ALIGN cfile.erase_byte = erase_byte return cfile - def get_data(self): + def calc_start_offset(self): + """Check if this file needs to start at a particular offset in CBFS + + Returns: + None if the file can be placed anywhere, or + the largest offset where the file could start (integer) + """ + if self.cbfs_offset is None: + return None + return self.cbfs_offset - self.get_header_len() + + def get_header_len(self): + """Get the length of headers required for a file + + This is the minimum length required before the actual data for this file + could start. It might start later if there is padding. + + Returns: + Total length of all non-data fields, in bytes + """ + name = _pack_string(self.name) + hdr_len = len(name) + FILE_HEADER_LEN + if self.ftype == TYPE_STAGE: + pass + elif self.ftype == TYPE_RAW: + hdr_len += ATTR_COMPRESSION_LEN + elif self.ftype == TYPE_EMPTY: + pass + else: + raise ValueError('Unknown file type %#x\n' % self.ftype) + return hdr_len + + def get_data(self, offset=None, pad_byte=None): """Obtain the contents of the file, in CBFS format Returns: @@ -292,6 +331,7 @@ class CbfsFile(object): attr_pos = 0 content = b'' attr = b'' + pad = b'' data = self.data if self.ftype == TYPE_STAGE: elf_data = elf.DecodeElf(data, self.base_address) @@ -315,10 +355,33 @@ class CbfsFile(object): if attr: attr_pos = hdr_len hdr_len += len(attr) - hdr = struct.pack(FILE_HEADER_FORMAT, FILE_MAGIC, - len(content) + len(data), + if self.cbfs_offset is not None: + pad_len = self.cbfs_offset - offset - hdr_len + if pad_len < 0: # pragma: no cover + # Test coverage of this is not available since this should never + # happen. It indicates that get_header_len() provided an + # incorrect value (too small) so that we decided that we could + # put this file at the requested place, but in fact a previous + # file extends far enough into the CBFS that this is not + # possible. + raise ValueError("Internal error: CBFS file '%s': Requested offset %#x but current output position is %#x" % + (self.name, self.cbfs_offset, offset)) + pad = tools.GetBytes(pad_byte, pad_len) + hdr_len += pad_len + self.offset = len(content) + len(data) + hdr = struct.pack(FILE_HEADER_FORMAT, FILE_MAGIC, self.offset, self.ftype, attr_pos, hdr_len) - return hdr + name + attr + content + data + + # Do a sanity check of the get_header_len() function, to ensure that it + # stays in lockstep with this function + expected_len = self.get_header_len() + actual_len = len(hdr + name + attr) + if expected_len != actual_len: # pragma: no cover + # Test coverage of this is not available since this should never + # happen. It probably indicates that get_header_len() is broken. + raise ValueError("Internal error: CBFS file '%s': Expected headers of %#x bytes, got %#d" % + (self.name, expected_len, actual_len)) + return hdr + name + attr + pad + content + data class CbfsWriter(object): @@ -431,34 +494,39 @@ class CbfsWriter(object): if offset < self._size: self._skip_to(fd, offset) - def add_file_stage(self, name, data): + def add_file_stage(self, name, data, cbfs_offset=None): """Add a new stage file to the CBFS Args: name: String file name to put in CBFS (does not need to correspond to the name that the file originally came from) data: Contents of file + cbfs_offset: Offset of this file's data within the CBFS, in bytes, + or None to place this file anywhere Returns: CbfsFile object created """ - cfile = CbfsFile.stage(self._base_address, name, data) + cfile = CbfsFile.stage(self._base_address, name, data, cbfs_offset) self._files[name] = cfile return cfile - def add_file_raw(self, name, data, compress=COMPRESS_NONE): + def add_file_raw(self, name, data, cbfs_offset=None, + compress=COMPRESS_NONE): """Create a new raw file Args: name: String file name to put in CBFS (does not need to correspond to the name that the file originally came from) data: Contents of file + cbfs_offset: Offset of this file's data within the CBFS, in bytes, + or None to place this file anywhere compress: Compression algorithm to use (COMPRESS_...) Returns: CbfsFile object created """ - cfile = CbfsFile.raw(name, data, compress) + cfile = CbfsFile.raw(name, data, cbfs_offset, compress) self._files[name] = cfile return cfile @@ -507,7 +575,11 @@ class CbfsWriter(object): # Write out each file for cbf in self._files.values(): - fd.write(cbf.get_data()) + # Place the file at its requested place, if any + offset = cbf.calc_start_offset() + if offset is not None: + self._pad_to(fd, align_int_down(offset, self._align)) + fd.write(cbf.get_data(fd.tell(), self._erase_byte)) self._align_to(fd, self._align) if not self._hdr_at_start: self._write_header(fd, add_fileheader=self._add_fileheader) @@ -639,25 +711,27 @@ class CbfsReader(object): # Create the correct CbfsFile object depending on the type cfile = None - fd.seek(file_pos + offset, io.SEEK_SET) + cbfs_offset = file_pos + offset + fd.seek(cbfs_offset, io.SEEK_SET) if ftype == TYPE_CBFSHEADER: self._read_header(fd) elif ftype == TYPE_STAGE: data = fd.read(STAGE_LEN) - cfile = CbfsFile.stage(self.stage_base_address, name, b'') + cfile = CbfsFile.stage(self.stage_base_address, name, b'', + cbfs_offset) (cfile.compress, cfile.entry, cfile.load, cfile.data_len, cfile.memlen) = struct.unpack(STAGE_FORMAT, data) cfile.data = fd.read(cfile.data_len) elif ftype == TYPE_RAW: data = fd.read(size) - cfile = CbfsFile.raw(name, data, compress) + cfile = CbfsFile.raw(name, data, cbfs_offset, compress) cfile.decompress() if DEBUG: print('data', data) elif ftype == TYPE_EMPTY: # Just read the data and discard it, since it is only padding fd.read(size) - cfile = CbfsFile('', TYPE_EMPTY, b'') + cfile = CbfsFile('', TYPE_EMPTY, b'', cbfs_offset) else: raise ValueError('Unknown type %#x when reading\n' % ftype) if cfile: @@ -674,7 +748,8 @@ class CbfsReader(object): """Read attributes from the file CBFS files can have attributes which are things that cannot fit into the - header. The only attribute currently supported is compression. + header. The only attributes currently supported are compression and the + unused tag. Args: fd: File to read from @@ -703,6 +778,8 @@ class CbfsReader(object): # We don't currently use this information atag, alen, compress, _decomp_size = struct.unpack( ATTR_COMPRESSION_FORMAT, data) + elif atag == FILE_ATTR_TAG_UNUSED2: + break else: print('Unknown attribute tag %x' % atag) attr_size -= len(data) @@ -760,7 +837,7 @@ class CbfsReader(object): return val.decode('utf-8') -def cbfstool(fname, *cbfs_args): +def cbfstool(fname, *cbfs_args, **kwargs): """Run cbfstool with provided arguments If the tool fails then this function raises an exception and prints out the @@ -773,7 +850,9 @@ def cbfstool(fname, *cbfs_args): Returns: CommandResult object containing the results """ - args = ('cbfstool', fname) + cbfs_args + args = ['cbfstool', fname] + list(cbfs_args) + if kwargs.get('base') is not None: + args += ['-b', '%#x' % kwargs['base']] result = command.RunPipe([args], capture=not VERBOSE, capture_stderr=not VERBOSE, raise_on_error=False) if result.return_code: diff --git a/tools/binman/cbfs_util_test.py b/tools/binman/cbfs_util_test.py index 9bb6a298222..0fe4aa494ec 100755 --- a/tools/binman/cbfs_util_test.py +++ b/tools/binman/cbfs_util_test.py @@ -105,7 +105,7 @@ class TestCbfs(unittest.TestCase): return cbfs def _check_uboot(self, cbfs, ftype=cbfs_util.TYPE_RAW, offset=0x38, - data=U_BOOT_DATA): + data=U_BOOT_DATA, cbfs_offset=None): """Check that the U-Boot file is as expected Args: @@ -113,6 +113,7 @@ class TestCbfs(unittest.TestCase): ftype: Expected file type offset: Expected offset of file data: Expected data in file + cbfs_offset: Expected CBFS offset for file's data Returns: CbfsFile object containing the file @@ -121,24 +122,30 @@ class TestCbfs(unittest.TestCase): cfile = cbfs.files['u-boot'] self.assertEqual('u-boot', cfile.name) self.assertEqual(offset, cfile.offset) + if cbfs_offset is not None: + self.assertEqual(cbfs_offset, cfile.cbfs_offset) self.assertEqual(data, cfile.data) self.assertEqual(ftype, cfile.ftype) self.assertEqual(cbfs_util.COMPRESS_NONE, cfile.compress) self.assertEqual(len(data), cfile.memlen) return cfile - def _check_dtb(self, cbfs, offset=0x38, data=U_BOOT_DTB_DATA): + def _check_dtb(self, cbfs, offset=0x38, data=U_BOOT_DTB_DATA, + cbfs_offset=None): """Check that the U-Boot dtb file is as expected Args: cbfs: CbfsReader object to check offset: Expected offset of file data: Expected data in file + cbfs_offset: Expected CBFS offset for file's data """ self.assertIn('u-boot-dtb', cbfs.files) cfile = cbfs.files['u-boot-dtb'] self.assertEqual('u-boot-dtb', cfile.name) self.assertEqual(offset, cfile.offset) + if cbfs_offset is not None: + self.assertEqual(cbfs_offset, cfile.cbfs_offset) self.assertEqual(U_BOOT_DTB_DATA, cfile.data) self.assertEqual(cbfs_util.TYPE_RAW, cfile.ftype) self.assertEqual(cbfs_util.COMPRESS_NONE, cfile.compress) @@ -157,13 +164,14 @@ class TestCbfs(unittest.TestCase): self._check_uboot(cbfs) self._check_dtb(cbfs) - def _get_expected_cbfs(self, size, arch='x86', compress=None): + def _get_expected_cbfs(self, size, arch='x86', compress=None, base=None): """Get the file created by cbfstool for a particular scenario Args: size: Size of the CBFS in bytes arch: Architecture of the CBFS, as a string compress: Compression to use, e.g. cbfs_util.COMPRESS_LZMA + base: Base address of file, or None to put it anywhere Returns: Resulting CBFS file, or None if cbfstool is not available @@ -172,14 +180,18 @@ class TestCbfs(unittest.TestCase): return None cbfs_fname = os.path.join(self._indir, 'test.cbfs') cbfs_util.cbfstool(cbfs_fname, 'create', '-m', arch, '-s', '%#x' % size) + if base: + base = [(1 << 32) - size + b for b in base] cbfs_util.cbfstool(cbfs_fname, 'add', '-n', 'u-boot', '-t', 'raw', '-c', compress and compress[0] or 'none', '-f', tools.GetInputFilename( - compress and 'compress' or 'u-boot.bin')) + compress and 'compress' or 'u-boot.bin'), + base=base[0] if base else None) cbfs_util.cbfstool(cbfs_fname, 'add', '-n', 'u-boot-dtb', '-t', 'raw', '-c', compress and compress[1] or 'none', '-f', tools.GetInputFilename( - compress and 'compress' or 'u-boot.dtb')) + compress and 'compress' or 'u-boot.dtb'), + base=base[1] if base else None) return cbfs_fname def _compare_expected_cbfs(self, data, cbfstool_fname): @@ -407,7 +419,7 @@ class TestCbfs(unittest.TestCase): self.skipTest('lz4 --no-frame-crc not available') size = 0x140 cbw = CbfsWriter(size) - cbw.add_file_raw('u-boot', COMPRESS_DATA, + cbw.add_file_raw('u-boot', COMPRESS_DATA, None, compress=cbfs_util.COMPRESS_LZ4) data = cbw.get_data() @@ -431,7 +443,7 @@ class TestCbfs(unittest.TestCase): self.skipTest('lz4 --no-frame-crc not available') size = 0x140 cbw = CbfsWriter(size) - cbw.add_file_raw('u-boot', COMPRESS_DATA, + cbw.add_file_raw('u-boot', COMPRESS_DATA, None, compress=cbfs_util.COMPRESS_LZ4) data = cbw.get_data() @@ -517,9 +529,9 @@ class TestCbfs(unittest.TestCase): self.skipTest('lz4 --no-frame-crc not available') size = 0x140 cbw = CbfsWriter(size) - cbw.add_file_raw('u-boot', COMPRESS_DATA, + cbw.add_file_raw('u-boot', COMPRESS_DATA, None, compress=cbfs_util.COMPRESS_LZ4) - cbw.add_file_raw('u-boot-dtb', COMPRESS_DATA, + cbw.add_file_raw('u-boot-dtb', COMPRESS_DATA, None, compress=cbfs_util.COMPRESS_LZMA) data = cbw.get_data() @@ -556,6 +568,58 @@ class TestCbfs(unittest.TestCase): cbfs_fname = self._get_expected_cbfs(size=size) self._compare_expected_cbfs(data, cbfs_fname) + def test_cbfs_offset(self): + """Test a CBFS with files at particular offsets""" + size = 0x200 + cbw = CbfsWriter(size) + cbw.add_file_raw('u-boot', U_BOOT_DATA, 0x40) + cbw.add_file_raw('u-boot-dtb', U_BOOT_DTB_DATA, 0x140) + + data = cbw.get_data() + cbfs = self._check_hdr(data, size) + self._check_uboot(cbfs, ftype=cbfs_util.TYPE_RAW, offset=0x40, + cbfs_offset=0x40) + self._check_dtb(cbfs, offset=0x40, cbfs_offset=0x140) + + cbfs_fname = self._get_expected_cbfs(size=size, base=(0x40, 0x140)) + self._compare_expected_cbfs(data, cbfs_fname) + + def test_cbfs_invalid_file_type_header(self): + """Check handling of an invalid file type when outputting a header""" + size = 0xb0 + cbw = CbfsWriter(size) + cfile = cbw.add_file_raw('u-boot', U_BOOT_DATA, 0) + + # Change the type manually before generating the CBFS, and make sure + # that the generator complains + cfile.ftype = 0xff + with self.assertRaises(ValueError) as e: + cbw.get_data() + self.assertIn('Unknown file type 0xff', str(e.exception)) + + def test_cbfs_offset_conflict(self): + """Test a CBFS with files that want to overlap""" + size = 0x200 + cbw = CbfsWriter(size) + cbw.add_file_raw('u-boot', U_BOOT_DATA, 0x40) + cbw.add_file_raw('u-boot-dtb', U_BOOT_DTB_DATA, 0x80) + + with self.assertRaises(ValueError) as e: + cbw.get_data() + self.assertIn('No space for data before pad offset', str(e.exception)) + + def test_cbfs_check_offset(self): + """Test that we can discover the offset of a file after writing it""" + size = 0xb0 + cbw = CbfsWriter(size) + cbw.add_file_raw('u-boot', U_BOOT_DATA) + cbw.add_file_raw('u-boot-dtb', U_BOOT_DTB_DATA) + data = cbw.get_data() + + cbfs = cbfs_util.CbfsReader(data) + self.assertEqual(0x38, cbfs.files['u-boot'].cbfs_offset) + self.assertEqual(0x78, cbfs.files['u-boot-dtb'].cbfs_offset) + if __name__ == '__main__': unittest.main() diff --git a/tools/binman/etype/cbfs.py b/tools/binman/etype/cbfs.py index 513df217bc1..49baa6a4f63 100644 --- a/tools/binman/etype/cbfs.py +++ b/tools/binman/etype/cbfs.py @@ -52,6 +52,7 @@ class Entry_cbfs(Entry): filename = "u-boot.dtb"; cbfs-type = "raw"; cbfs-compress = "lz4"; + cbfs-offset = <0x100000>; }; }; @@ -110,6 +111,15 @@ class Entry_cbfs(Entry): to add a flat binary with a load/start address, similar to the 'add-flat-binary' option in cbfstool. + cbfs-offset: + This is the offset of the file's data within the CBFS. It is used to + specify where the file should be placed in cases where a fixed position + is needed. Typical uses are for code which is not relocatable and must + execute in-place from a particular address. This works because SPI flash + is generally mapped into memory on x86 devices. The file header is + placed before this offset so that the data start lines up exactly with + the chosen offset. If this property is not provided, then the file is + placed in the next available spot. The current implementation supports only a subset of CBFS features. It does not support other file types (e.g. payload), adding multiple files (like the @@ -172,9 +182,10 @@ class Entry_cbfs(Entry): return False data = entry.GetData() if entry._type == 'raw': - cbfs.add_file_raw(entry._cbfs_name, data, entry._cbfs_compress) + cbfs.add_file_raw(entry._cbfs_name, data, entry._cbfs_offset, + entry._cbfs_compress) elif entry._type == 'stage': - cbfs.add_file_stage(entry._cbfs_name, data) + cbfs.add_file_stage(entry._cbfs_name, data, entry._cbfs_offset) data = cbfs.get_data() self.SetContents(data) return True @@ -186,6 +197,7 @@ class Entry_cbfs(Entry): entry._cbfs_name = fdt_util.GetString(node, 'cbfs-name', entry.name) entry._type = fdt_util.GetString(node, 'cbfs-type') compress = fdt_util.GetString(node, 'cbfs-compress', 'none') + entry._cbfs_offset = fdt_util.GetInt(node, 'cbfs-offset') entry._cbfs_compress = cbfs_util.find_compress(compress) if entry._cbfs_compress is None: self.Raise("Invalid compression in '%s': '%s'" % diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 1355c4f55d3..5bde8aa30a1 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -2012,5 +2012,28 @@ class TestFunctional(unittest.TestCase): self.assertIn('Could not complete processing of contents', str(e.exception)) + def testCbfsOffset(self): + """Test a CBFS with files at particular offsets + + Like all CFBS tests, this is just checking the logic that calls + cbfs_util. See cbfs_util_test for fully tests (e.g. test_cbfs_offset()). + """ + data = self._DoReadFile('114_cbfs_offset.dts') + size = 0x200 + + cbfs = cbfs_util.CbfsReader(data) + self.assertEqual(size, cbfs.rom_size) + + self.assertIn('u-boot', cbfs.files) + cfile = cbfs.files['u-boot'] + self.assertEqual(U_BOOT_DATA, cfile.data) + self.assertEqual(0x40, cfile.cbfs_offset) + + self.assertIn('u-boot-dtb', cbfs.files) + cfile2 = cbfs.files['u-boot-dtb'] + self.assertEqual(U_BOOT_DTB_DATA, cfile2.data) + self.assertEqual(0x140, cfile2.cbfs_offset) + + if __name__ == "__main__": unittest.main() diff --git a/tools/binman/test/114_cbfs_offset.dts b/tools/binman/test/114_cbfs_offset.dts new file mode 100644 index 00000000000..7aa9d9d4bf3 --- /dev/null +++ b/tools/binman/test/114_cbfs_offset.dts @@ -0,0 +1,26 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + sort-by-offset; + end-at-4gb; + size = <0x200>; + cbfs { + size = <0x200>; + offset = <0xfffffe00>; + u-boot { + cbfs-offset = <0x40>; + cbfs-type = "raw"; + }; + u-boot-dtb { + cbfs-offset = <0x140>; + cbfs-type = "raw"; + }; + }; + }; +};