From patchwork Tue Feb 23 05:55:53 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 586683 X-Patchwork-Delegate: trini@ti.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from theia.denx.de (theia.denx.de [85.214.87.163]) by ozlabs.org (Postfix) with ESMTP id B07E114076A for ; Tue, 23 Feb 2016 16:59:40 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.b=ZHWBh4W+; dkim-atps=neutral Received: from localhost (localhost [127.0.0.1]) by theia.denx.de (Postfix) with ESMTP id 8D43BA76A4; Tue, 23 Feb 2016 06:58:20 +0100 (CET) Received: from theia.denx.de ([127.0.0.1]) by localhost (theia.denx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Tm1KJKSy7TCG; Tue, 23 Feb 2016 06:58:20 +0100 (CET) Received: from theia.denx.de (localhost [127.0.0.1]) by theia.denx.de (Postfix) with ESMTP id 25203A7676; Tue, 23 Feb 2016 06:58:16 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by theia.denx.de (Postfix) with ESMTP id 27087A7647 for ; Tue, 23 Feb 2016 06:56:43 +0100 (CET) Received: from theia.denx.de ([127.0.0.1]) by localhost (theia.denx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id o6XYp0zsiRKY for ; Tue, 23 Feb 2016 06:56:43 +0100 (CET) X-policyd-weight: NOT_IN_SBL_XBL_SPAMHAUS=-1.5 NOT_IN_SPAMCOP=-1.5 NOT_IN_BL_NJABL=-1.5 (only DNSBL check requested) Received: from mail-ob0-f170.google.com (mail-ob0-f170.google.com [209.85.214.170]) by theia.denx.de (Postfix) with ESMTPS id AA4D9A7640 for ; Tue, 23 Feb 2016 06:56:27 +0100 (CET) Received: by mail-ob0-f170.google.com with SMTP id gc3so185459269obb.3 for ; Mon, 22 Feb 2016 21:56:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=2SaJE1SoNF7ZoLGuxWcvxpdDbkmQFWMR1hmYJVA0Xes=; b=ZHWBh4W+1HXXkNSga+QvQb0+QiJ1FfzP1r+Fc6HIN9i5CiwQi/Xn4scPqwj54Km2Oe ou73ZXYSNioflPRUJRoUb1uH8hmYUdtkBF433XeWdZAoNoJ4Dx8PGmU/aeev6FmdJ0Of rYlbXuFFgmxTFFq6FLCiQJjQyvjhqq2sTbK5FVSkYtai4gYdXy4pTv4kUzzh4XrFInYi nVi9IVThokp9E/Q5r+wkOt/FcqwMegHHooO8Nbnps+Lk2J/6irOyZhJXNyoPtR7kuGrx zCGgnxh96iCAKkp6xFiEMXNY9q1kXkWBCUHx3YZ3jAnYQNqIp+8U1Sbfl+nbX9q1Wdnq PhIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=2SaJE1SoNF7ZoLGuxWcvxpdDbkmQFWMR1hmYJVA0Xes=; b=P3BA3sbaKhEUFH572B+myFQp+ufWO3DICIQF1cAhDkyoc9OyeiFWaTnoRedtfAAlIY /aLf8yDnPKYnbQtuHatgZOY4Lzmq5z7T9MhdGYAV64l1v5vO9domzp1+yHygA0LtNM0K Ki69z63Hv/7iYGn8+ksrhKlhVLj0BOIgTxFqivYOGCTL18yPX0jCn3Hu4ZTm9W19vQ8e VLm8mydVo4W6XyWKRaP/HrLgNbOokfaDujfsu0y4jF+CtFdweMT1Q7ccYEvCdd6pix5G ghQ5FbFC//t3pUDXC4yu6fWPc1wGQ4iojblvsVhMAvX3xW8ZbXZDKHRl/gCLHmlyJQAQ fi7Q== X-Gm-Message-State: AG10YOT8rsB4gOYVapKd6iud71uWcDd3hOmufRrqPdGGbhvfH+l6hZ2y06djgRskl1g2Xzde X-Received: by 10.60.232.99 with SMTP id tn3mr25299416oec.8.1456206985958; Mon, 22 Feb 2016 21:56:25 -0800 (PST) Received: from kaki.bld.corp.google.com ([172.29.216.32]) by smtp.gmail.com with ESMTPSA id v6sm19254037oer.10.2016.02.22.21.56.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Feb 2016 21:56:22 -0800 (PST) Received: by kaki.bld.corp.google.com (Postfix, from userid 121222) id ECAB1223116; Mon, 22 Feb 2016 22:56:15 -0700 (MST) From: Simon Glass To: U-Boot Mailing List Date: Mon, 22 Feb 2016 22:55:53 -0700 Message-Id: <1456206959-29115-22-git-send-email-sjg@chromium.org> X-Mailer: git-send-email 2.7.0.rc3.207.g0ac5344 In-Reply-To: <1456206959-29115-1-git-send-email-sjg@chromium.org> References: <1456206959-29115-1-git-send-email-sjg@chromium.org> Cc: Joe Hershberger , Tom Rini Subject: [U-Boot] [PATCH v2 21/27] mkimage: Support placing data outside the FIT X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.15 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" One limitation of FIT is that all the data is 'inline' within it, using a 'data' property in each image node. This means that to find out what is in the FIT it is necessary to scan the entire file. Once loaded it can be scanned and then the images can be copied to the correct place in memory. In SPL it can take a significant amount of time to copy images around in memory. Also loading data that does not end up being used is wasteful. It would be useful if the FIT were small, acting as a directory, with the actual data stored elsewhere. This allows SPL to load the entire FIT, without the images, then load the images it wants later. Add a -E option to mkimage to request that it output an 'external' FIT. Signed-off-by: Simon Glass --- Changes in v2: None doc/mkimage.1 | 9 +++ doc/uImage.FIT/source_file_format.txt | 20 ++++++- tools/fit_image.c | 109 ++++++++++++++++++++++++++++++++++ tools/imagetool.h | 1 + tools/mkimage.c | 5 +- 5 files changed, 142 insertions(+), 2 deletions(-) diff --git a/doc/mkimage.1 b/doc/mkimage.1 index 036b095..e0f210a 100644 --- a/doc/mkimage.1 +++ b/doc/mkimage.1 @@ -112,6 +112,15 @@ Provide special options to the device tree compiler that is used to create the image. .TP +.BI "\-E +After processing, move the image data outside the FIT and store a data offset +in the FIT. Images will be placed one after the other immediately after the +FIT, with each one aligned to a 4-byte boundary. The existing 'data' property +in each image will be replaced with 'data-offset' and 'data-size' properties. +A 'data-offset' of 0 indicates that it starts in the first (4-byte aligned) +byte after the FIT. + +.TP .BI "\-f [" "image tree source file" " | " "auto" "]" Image tree source file that describes the structure and contents of the FIT image. diff --git a/doc/uImage.FIT/source_file_format.txt b/doc/uImage.FIT/source_file_format.txt index 3175c9f..3db068d 100644 --- a/doc/uImage.FIT/source_file_format.txt +++ b/doc/uImage.FIT/source_file_format.txt @@ -2,6 +2,7 @@ U-Boot new uImage source file format (bindings definition) ========================================================== Author: Marian Balakowicz +External data additions, 25/1/16 Simon Glass 1) Introduction --------------- @@ -262,7 +263,24 @@ Older, 2.4 kernel and 2.6 non-FDT kernel do not use FDT blob, in such cases not* be specified in a configuration node. -8) Examples +8) External data +---------------- + +The above format shows a 'data' property which holds the data for each image. +It is also possible for this data to reside outside the FIT itself. This +allows the FIT to be quite small, so that it can be loaded and scanned +without loading a large amount of data. Then when an image is needed it can +be loaded from an external source. + +In this case the 'data' property is omitted. Instead you can use: + + - data-offset : offset of the data in a separate image store. The image + store is placed immediately after the last byte of the device tree binary, + aligned to a 4-byte boundary. + - data-size : size of the data in bytes + + +9) Examples ----------- Please see doc/uImage.FIT/*.its for actual image source files. diff --git a/tools/fit_image.c b/tools/fit_image.c index 765ff31..06b561d 100644 --- a/tools/fit_image.c +++ b/tools/fit_image.c @@ -354,6 +354,108 @@ err: } /** + * fit_extract_data() - Move all data outside the FIT + * + * This takes a normal FIT file and removes all the 'data' properties from it. + * The data is placed in an area after the FIT so that it can be accessed + * using an offset into that area. The 'data' properties turn into + * 'data-offset' properties. + * + * This function cannot cope with FITs with 'data-offset' properties. All + * data must be in 'data' properties on entry. + */ +static int fit_extract_data(struct image_tool_params *params, const char *fname) +{ + void *buf; + int buf_ptr; + int fit_size, new_size; + int fd; + struct stat sbuf; + void *fdt; + int ret; + int images; + int node; + + fd = mmap_fdt(params->cmdname, fname, 0, &fdt, &sbuf, false); + if (fd < 0) + return -EIO; + fit_size = fdt_totalsize(fdt); + + /* Allocate space to hold the image data we will extract */ + buf = malloc(fit_size); + if (!buf) { + ret = -ENOMEM; + goto err; + } + buf_ptr = 0; + + images = fdt_path_offset(fdt, FIT_IMAGES_PATH); + if (images < 0) { + debug("%s: Cannot find /images node: %d\n", __func__, images); + ret = -EINVAL; + goto err; + } + + for (node = fdt_first_subnode(fdt, images); + node >= 0; + node = fdt_next_subnode(fdt, node)) { + const char *data; + int len; + + data = fdt_getprop(fdt, node, "data", &len); + if (!data) + continue; + memcpy(buf + buf_ptr, data, len); + debug("Extracting data size %x\n", len); + + ret = fdt_delprop(fdt, node, "data"); + if (ret) { + ret = -EPERM; + goto err; + } + fdt_setprop_u32(fdt, node, "data-offset", buf_ptr); + fdt_setprop_u32(fdt, node, "data-size", len); + + buf_ptr += (len + 3) & ~3; + } + + /* Pack the FDT and place the data after it */ + fdt_pack(fdt); + + debug("Size reduced from %x to %x\n", fit_size, fdt_totalsize(fdt)); + debug("External data size %x\n", buf_ptr); + new_size = fdt_totalsize(fdt); + new_size = (new_size + 3) & ~3; + munmap(fdt, sbuf.st_size); + + if (ftruncate(fd, new_size)) { + debug("%s: Failed to truncate file: %s\n", __func__, + strerror(errno)); + ret = -EIO; + goto err; + } + if (lseek(fd, new_size, SEEK_SET) < 0) { + debug("%s: Failed to seek to end of file: %s\n", __func__, + strerror(errno)); + ret = -EIO; + goto err; + } + if (write(fd, buf, buf_ptr) != buf_ptr) { + debug("%s: Failed to write external data to file %s\n", + __func__, strerror(errno)); + ret = -EIO; + goto err; + } + close(fd); + + ret = 0; + +err: + close(fd); + return ret; +} + +/** * fit_handle_file - main FIT file processing function * * fit_handle_file() runs dtc to convert .its to .itb, includes @@ -430,6 +532,13 @@ static int fit_handle_file(struct image_tool_params *params) goto err_system; } + /* Move the data so it is external to the FIT, if requested */ + if (params->external_data) { + ret = fit_extract_data(params, tmpfile); + if (ret) + goto err_system; + } + if (rename (tmpfile, params->imagefile) == -1) { fprintf (stderr, "%s: Can't rename %s to %s: %s\n", params->cmdname, tmpfile, params->imagefile, diff --git a/tools/imagetool.h b/tools/imagetool.h index 3d30fbe..24f8f4b 100644 --- a/tools/imagetool.h +++ b/tools/imagetool.h @@ -72,6 +72,7 @@ struct image_tool_params { int fit_image_type; /* Image type to put into the FIT */ struct content_info *content_head; /* List of files to include */ struct content_info *content_tail; + bool external_data; /* Store data outside the FIT */ }; /* diff --git a/tools/mkimage.c b/tools/mkimage.c index 63ba4a1..2446cab 100644 --- a/tools/mkimage.c +++ b/tools/mkimage.c @@ -138,7 +138,7 @@ static void process_args(int argc, char **argv) expecting = IH_TYPE_COUNT; /* Unknown */ while ((opt = getopt(argc, argv, - "-a:A:bcC:d:D:e:f:Fk:K:ln:O:rR:sT:vVx")) != -1) { + "-a:A:bcC:d:D:e:Ef:Fk:K:ln:O:rR:sT:vVx")) != -1) { switch (opt) { case 'a': params.addr = strtoul(optarg, &ptr, 16); @@ -180,6 +180,9 @@ static void process_args(int argc, char **argv) } params.eflag = 1; break; + case 'E': + params.external_data = true; + break; case 'f': datafile = optarg; params.auto_its = !strcmp(datafile, "auto");