From patchwork Mon Oct 19 02:41:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1383949 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=X4O75u9i; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CF1HT2tMFz9sSG for ; Mon, 19 Oct 2020 13:41:57 +1100 (AEDT) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 4BFBA81B79; Mon, 19 Oct 2020 04:41:50 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="X4O75u9i"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 136EC801D8; Mon, 19 Oct 2020 04:41:49 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,SPF_HELO_NONE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd42.google.com (mail-io1-xd42.google.com [IPv6:2607:f8b0:4864:20::d42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 3C1D4801D8 for ; Mon, 19 Oct 2020 04:41:46 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd42.google.com with SMTP id k25so11279128ioh.7 for ; Sun, 18 Oct 2020 19:41:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=5JQSxnzkd+Nh/yJr1R77gnO2YXkvwDeR4tTmgjq3Y9M=; b=X4O75u9iugoxs4mIoyJQYWC1j2EyJkT1xaxoPcL+eBhKukbS/w1FtwaACWvtXJ20a9 mUE3fkJ/XTHNiTJOeFHxbhFdFjriOeucYk/9m2hrrhSkGn+hmeQV4uoaARMzqKpIg/mq ZO8IdInUD9oYLqZS/8ENjj+uTGq/RN8bbT0nI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=5JQSxnzkd+Nh/yJr1R77gnO2YXkvwDeR4tTmgjq3Y9M=; b=pZxmNoOkJY9X4ZxzAV11mbzvxRMVxRq3ow3GMhXx1IIS0S1hxhut+KCZJ1rywk2/+L 3EPm4wG+nZM4EZNgIiJAD9k+lbeHw4QzqmX5HBbMkgxkvglBqWcs+mhwg3LdN4sue4BU sKQXcB46Y342o/KCaIewLkct/2S0vKASlbK6yUGVmIfLUj8t9/llPWYzpnW3xBv7jRqi 5IfdUsEDe/IJZVosSk9o9dIPBB7EYYgOXdaHmt5iSBAQ5NJ3bkeNO+sZZnwEZO9D6GSS MC53RVivH0LQXqh1fWjXYhtIPbAlFbUAbNRMSqrDK8B+4IsLoyuhHP4ONLETiF9auNxP uVaw== X-Gm-Message-State: AOAM5316Tltz/lJ+uJ/BfSvFizxgS7SK/O+mCRpK05Lb7dlwe9AQtuh5 Oxcmk5jeR8JMgNHohEgVYvvri+VwL6w6VamG X-Google-Smtp-Source: ABdhPJyO8DcWfEijZ08PYs2+lBIPI9BEg6i4JSRStfZy795w+Tsn4aAKqqKJR02ZPQnaylxBajmmog== X-Received: by 2002:a6b:9187:: with SMTP id t129mr9371679iod.30.1603075304659; Sun, 18 Oct 2020 19:41:44 -0700 (PDT) Received: from localhost.localdomain (c-73-14-175-90.hsd1.co.comcast.net. [73.14.175.90]) by smtp.gmail.com with ESMTPSA id g185sm9750871ilh.35.2020.10.18.19.41.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 18 Oct 2020 19:41:43 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass , Alper Nebi Yasak , Andy Shevchenko , Bin Meng Subject: [PATCH 00/25] binman: Support compression of sections Date: Sun, 18 Oct 2020 20:41:13 -0600 Message-Id: <20201019024138.3804540-1-sjg@chromium.org> X-Mailer: git-send-email 2.29.0.rc1.297.gfa9743e501-goog MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.34 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.102.3 at phobos.denx.de X-Virus-Status: Clean This series includes a number of improvements and refactors to support compressing entire sections. This is sometimes useful when a section contains a number of entries which are accessed as a whole and are best compressed together. Most of the effort here is clarifying what is in a section and what is added by its parent (i.e. padding). But the opportunity is taken to tidy up and somewhat simplify some of the section-handling code. Also some new tests are added to reduce the amount of undefined behaviour. This series is available at u-boot-dm/binman-working Simon Glass (25): binman: Give a sensible error if no command is given binman: Fix return from u-boot-ucode if there is no DT binman: Remove references to 'image' in entry_Section binman: Expand the error message for breaching a section binman: Move CompressData() into Entry base class binman: Use 'files-compress' to set compression for files binman: Update testPackExtra with more checks binman: Expand docs and test for padding binman: Expand docs and test for alignment binman: Move section-building code into a function binman: Refactor _BuildSectionData() binman: Move section padding to the parent binman: Make section padding consistent with other entries binman: Store the original data before compression binman: Set section contents in GetData() binman: Avoid reporting image-pos with compression binman: Drop Entry.CheckOffset() binman: Move sort and expand to the main Pack() function binman: Drop the Entry.CheckSize() method binman: Call CheckSize() from the section's Pack() method binman: Drop CheckEntries() binman: Update CheckEntries() for compressed sections binman: Use the actual contents in CheckSize() binman: Support compression of sections binman: Avoid calculated section data repeatedly tools/binman/README | 72 ++-- tools/binman/README.entries | 15 +- tools/binman/cmdline.py | 1 + tools/binman/control.py | 4 +- tools/binman/entry.py | 65 +++- tools/binman/etype/blob.py | 7 - tools/binman/etype/cbfs.py | 6 +- tools/binman/etype/files.py | 7 +- tools/binman/etype/section.py | 167 +++++++-- tools/binman/etype/u_boot_ucode.py | 1 + tools/binman/ftest.py | 347 +++++++++++++++++- tools/binman/image.py | 2 +- tools/binman/test/009_pack_extra.dts | 2 +- tools/binman/test/085_files_compress.dts | 2 +- tools/binman/test/177_section_pad.dts | 27 ++ tools/binman/test/178_section_align.dts | 34 ++ tools/binman/test/179_compress_image.dts | 14 + tools/binman/test/180_compress_image_less.dts | 14 + .../binman/test/181_compress_section_size.dts | 17 + tools/binman/test/182_compress_section.dts | 16 + tools/binman/test/183_compress_extra.dts | 37 ++ 21 files changed, 753 insertions(+), 104 deletions(-) create mode 100644 tools/binman/test/177_section_pad.dts create mode 100644 tools/binman/test/178_section_align.dts create mode 100644 tools/binman/test/179_compress_image.dts create mode 100644 tools/binman/test/180_compress_image_less.dts create mode 100644 tools/binman/test/181_compress_section_size.dts create mode 100644 tools/binman/test/182_compress_section.dts create mode 100644 tools/binman/test/183_compress_extra.dts