From patchwork Tue Aug 16 06:41:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 1666682 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-weidmueller-onmicrosoft-com header.b=F2k1kjbe; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M6M5t0Xhsz1yft for ; Tue, 16 Aug 2022 16:42:40 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id A78F2848D2; Tue, 16 Aug 2022 08:42:31 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.b="F2k1kjbe"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 0B77A84228; Tue, 16 Aug 2022 08:42:30 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50057.outbound.protection.outlook.com [40.107.5.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id A5060841F4 for ; Tue, 16 Aug 2022 08:42:26 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=stefan.herbrechtsmeier-oss@weidmueller.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OzvDa4iuLVpKtMOgvTqsxAaSatqLZl3v/10LrsFLgdVpBOcpW3Y1/V/PFqcS3g/E0843pHsLaPkj+XjcdOaELReq3CbWxYpjwE/IebM+7Nc6oJgqIsL3Z8O7p0P6OcMBwx9okpWCT3gUWq05miLw7zwqDXmVqUBIWVC9CzOvl1jG3YwM4QBsnSuUDESQSEBFCB8NOZa5cHFnA8Zr3A1NB/Qgcgp/WIo+B2uhW3axID3+gBl4vzf39qQJ7u4YgeobBwBDyklC0z1vT27k9xPjjhpU8t8/GKBWg0nY2ubh6J7HnaHswQaejHHzJwi1KMNmg85dL1KbYon9kVn82Zaa7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=w+3uOVo2qqUJunkfu0BCNOoXns5+5awb6U7JSeaaE2o=; b=A8d/eMwFcOrVnvqGKdM5LXa03eJ9NNpb6engpoORn31LUEYVZRMpd4Y3+tn84pp82BMJYvFLZHhqXCjajqc6QHWM2YWdM27zweMVEyWfw23IL+aknCVVN0Sa1pVOLKvJVAk5kWw7VizmaAqAYjK9le5BTeVERT2J0WHCu5IZ5U1JJ65ZLfs64H7eM4Xzk8PA+FqTo2P/Bd/hcGR3jSEWeZv+6loyZ4YKDX072gjvQ11+aWCsCF0Pp+hzHhahYoHFO8eWFGTA+bMUQcIdE7lVIp8sZrdIRMzDgUGWaBhB+BHqbvXEV8dvQy0kf3zWpH90AR8C173UY0aM7BbtuoMlzQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.onmicrosoft.com; s=selector1-weidmueller-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=w+3uOVo2qqUJunkfu0BCNOoXns5+5awb6U7JSeaaE2o=; b=F2k1kjbedBuS9IPluBdQuKb6KzUeMJMj9BknZIn9/LmXIE4Qhe/JaoLLzLaKHQS6YgG24brVMes0ZKSZsVEXhhogbCcow7cfDa0QThFK1xsKwkkAbK6Ky6lfTRh2nOIuPkHQ1ZDuFGYCCpMJaETpzrvG1IkDpyAdMptJ30UiAJM= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) by VE1PR08MB4685.eurprd08.prod.outlook.com (2603:10a6:802:ae::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.25; Tue, 16 Aug 2022 06:42:24 +0000 Received: from PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44]) by PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44%3]) with mapi id 15.20.5525.011; Tue, 16 Aug 2022 06:42:24 +0000 From: Stefan Herbrechtsmeier To: u-boot@lists.denx.de Cc: Stefan Herbrechtsmeier , Simon Glass , Alper Nebi Yasak Subject: [PATCH v3 01/13] binman: Skip elf tests if python elftools is not available Date: Tue, 16 Aug 2022 08:41:39 +0200 Message-Id: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.30.2 X-ClientProxiedBy: AS9PR06CA0744.eurprd06.prod.outlook.com (2603:10a6:20b:487::19) To PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 94d365fa-0b21-4ccb-6734-08da7f527a2d X-MS-TrafficTypeDiagnostic: VE1PR08MB4685:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yDQoT+5GdYAXRsaBxiYIll6a/GsszSBbKNvNv1lEvNH/OO4q/GNSB65liGceR0fbKr0oeBSdro90Bt8v1vakyHNPEn1je7lEKRYLyoHTbXMJ0jvasTGUrLKvcn4LIdwk1Xml08KJ6l4NHZBKCfPB0Qnx23rHEOqNwOqi0+jdk/ibmO3QHUAs6z5sSBgV3ESHVnrk5o3+muWCDj3cIZ9RQBg8vBt1MfKyz5/CgQ9nZuEs8g1dEpY8YqXpE/9yEcxxiOkzjiT8sFslaMh7RX7GHIhN1KwYbirA7Os9hiotcpDGoMUKHcCe2VY5Ku8kDHQoOKwDduSHmrtcIroAPsQ0uL78Ggku3PQB2UBphtOOzd/3J8m7TmujsZjTmJDcbZ+O55l79JHhuNKVddqWc9pKZzObNYGAGIsymg+wvdru1paRXmUlJq95o6tXdjCiIAXvBpIi93BPSvTkG7czcZtf40/Anqyl4mR2ElYTCZiBp9XcuEH//Ef5BOMHIJbtrKC1/tn+3hIL+Lg3zLNMh/wdYwVxvm1kG5z06aDfhqJt4K2tNbuZVu+UP9gZP5HN6iAFW/6VJwsHR5WXq/swod3ZGlGnri4MEjLeeaM3Em7YKI/qJu31rg8GQ6ceBXEq98y6XbL6Dyqxz3fXzQ0dqUY/vJqtIBkZ/yHJ052d8LuoPE6WV+Xh3WFWmvPXBiRpT75DbTKe7JodZdo7uP0cnYIb589TutOv8HVl+z2Von0tjTdq457rbu6OHCW0xZ+kzeShSBfaCJBz5f6KQo5T+SnteAt048JeLycXic1JSxcXyg3sYsThPGnYUi00u6is77gc X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR08MB6969.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(136003)(366004)(396003)(346002)(376002)(39860400002)(86362001)(38350700002)(38100700002)(36756003)(316002)(8936002)(41300700001)(4326008)(66946007)(5660300002)(1076003)(66476007)(478600001)(8676002)(66556008)(6486002)(6512007)(26005)(52116002)(83380400001)(6506007)(2906002)(6666004)(186003)(2616005)(6916009)(54906003)(21314003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: BTfjASLQl9fRmrtsR84lfTV8Xbq9bQj2068/KwUeD6NhMRD1aYRBjlYxuRbZTgOJnsrmCR5LjUsPLSEO2EoYLCXpFLx+qjtO9SI0FkPzL/utGD3m4Cobj+a8UwhbuhCRbZAwb8mFqJYdY8Lops/BxIz6m/uyT4JAyiVTD1tlq144Hzpx3AfC/G5jV+bnsZddCFuhOsIJwlV9B2akF5wzPRMTnkixq8SEZefIo5p9dUEhTWyZYBJ7HaNMwgmYiCxkBbfa46aZLo5LK5kMvcLBrx3ZZjX6uWJpbILrs1GQknbxBZNk/LmIbHTQICA0iC6Mg+4/x3kOdirXsswZ56NNXD1RThV4wQp1PtvQ+YDoC98rbCkn6Ns6texgdgLdtFR7I3HxlSFTibq7veTIjFrF4VQPAEI57dBF/I0hpLcQvlIEapKW7x92sVZXbIt7pyEPEW81JXJm+luMdbeE9YjiOH0onxfTOF6pL5LGspjrSSRaoD+1GdV2GPrukOZBKFx2nS6oUibHTN9nCLSnmffwP64cXerw3Qnpw6auM0fD+DFaDGBED/ALku/qJkrhjuLOF9IHfYEPllaUKhLSUaBhxHaFDoKTdOyZsKOMgzPXzdHntiogKSZfBxy49d96bd5CqiWJBlITYF6OQtzo0Zm1htxgzl70O5Tn1/cOoaT70NGxYmMZNGIR2VZYxmPBtBJIRSPvD572nfOHQzd5gp9Flx3/jTRZ8s2MiyX+ZtJcT1biLaCnE/2Qp/khrZ86079QEIz6geRLmj+t+nr8iZ5eBWjoF8wkhdFLTC/RQUqHpo30kIEKjxHB3yJuBXo0/S7XAg7fUdT3a4bQ9ONDuaUSaEJQsHi3WTr21pQjS4AvFh38uJhI41Bch8zcvA/nxwF20Br+BRyzr+4LFkDVpLrIamDuzwLEEnWaAKSCTGRNtEdpG/8uIw6OQAIeCLZxOwYvtu+ObrfjQaon9uABiHsqExjMnqPJHZu0Desryf5VlltqOOmVeDcStx2ChSd8NfQczWS9Kw37mCofX3cAfoa1kocbMvreUCGlQunPKBZxbt2YSkYH+8IUvFPBhvY9RA7h2ervypcEHQfhmlNU9n9kIuCy0iFAOFZbYQyEV5O01iEKwk2m4b3MN7XgmCvHWnlVKbN3W5zr2Me1G2GwAK8hytEcT4Cufb4eFbcWnEgApmniYg/giDHz9bYMYFmmENU4hbYDiAihVWrqGp0cmjBLUeZMNSHcjqxHFW1wKADRRF7Y2AOrG1jPCgSjTK/kS9xEonaGzCL7PhzicoJN480WPRhKhcTKnJ41/lmGvTUXnHEQdOa9K5JXq11TDckEP1BQQFGFr3u8KnyIYxHSe/sbeScp8/LkZZPQo3H8976iKC9UJmj1D+6c+IhmWZ9FZmNVTiHAtCD9d9Ef2Z9GiyoUbO9IyPHEZ7rhCr3ik3REP+M5KDvhBMMjVBURmWbZshXSusYhEqFNPDl9yrqEIyoTytLykEmEl8MN66bQpCSQoyL2nD/eRn+5UwEv7xMRkHMBlyQ/Fz6WuGAKpAmPr5ATAgWL31bFywgs1lO/Mb3tvWrXy/PDvGs8YKGBWVojF0qZrraDYnWF2zqtcxn50/IE+w== X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: 94d365fa-0b21-4ccb-6734-08da7f527a2d X-MS-Exchange-CrossTenant-AuthSource: PAXPR08MB6969.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2022 06:42:24.1606 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7Y0DB4mjeOtzl6H9IJUDjMKokKo0VFhYlvoTg2TN3Q+PoLhdfnHvshpqkKB2N0sX64nv8p1yiUl1FGbCxJpe9g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4685 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean From: Stefan Herbrechtsmeier Skip tests which requires python elftools if the tool is not available. Signed-off-by: Stefan Herbrechtsmeier Reviewed-by: Simon Glass --- (no changes since v2) Changes in v2: - Added tools/binman/elf_test.py | 14 ++++++++++++++ tools/binman/ftest.py | 18 ++++++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/tools/binman/elf_test.py b/tools/binman/elf_test.py index 5a51c64cfe..75b867c2be 100644 --- a/tools/binman/elf_test.py +++ b/tools/binman/elf_test.py @@ -122,6 +122,8 @@ class TestElf(unittest.TestCase): def testOutsideFile(self): """Test a symbol which extends outside the entry area is detected""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') entry = FakeEntry(10) section = FakeSection() elf_fname = self.ElfTestFile('u_boot_binman_syms') @@ -147,6 +149,8 @@ class TestElf(unittest.TestCase): Only 32 and 64 bits are supported, since we need to store an offset into the image. """ + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') entry = FakeEntry(10) section = FakeSection() elf_fname =self.ElfTestFile('u_boot_binman_syms_size') @@ -161,6 +165,8 @@ class TestElf(unittest.TestCase): This should produce -1 values for all thress symbols, taking up the first 16 bytes of the image. """ + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') entry = FakeEntry(28) section = FakeSection(sym_value=None) elf_fname = self.ElfTestFile('u_boot_binman_syms') @@ -172,6 +178,8 @@ class TestElf(unittest.TestCase): def testDebug(self): """Check that enabling debug in the elf module produced debug output""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') try: tout.init(tout.DEBUG) entry = FakeEntry(24) @@ -281,6 +289,8 @@ class TestElf(unittest.TestCase): def test_read_segments_bad_data(self): """Test for read_loadable_segments() with an invalid ELF file""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') fname = self.ElfTestFile('embed_data') with self.assertRaises(ValueError) as e: elf.read_loadable_segments(tools.get_bytes(100, 100)) @@ -288,6 +298,8 @@ class TestElf(unittest.TestCase): def test_get_file_offset(self): """Test GetFileOffset() gives the correct file offset for a symbol""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') fname = self.ElfTestFile('embed_data') syms = elf.GetSymbols(fname, ['embed']) addr = syms['embed'].address @@ -314,6 +326,8 @@ class TestElf(unittest.TestCase): def test_get_symbol_from_address(self): """Test GetSymbolFromAddress()""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') fname = self.ElfTestFile('elf_sections') sym_name = 'calculate' syms = elf.GetSymbols(fname, [sym_name]) diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index fa1f421c05..da9aa9e679 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -4807,6 +4807,8 @@ class TestFunctional(unittest.TestCase): def testUpdateFdtInElf(self): """Test that we can update the devicetree in an ELF file""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') infile = elf_fname = self.ElfTestFile('u_boot_binman_embed') outfile = os.path.join(self._indir, 'u-boot.out') begin_sym = 'dtb_embed_begin' @@ -4858,6 +4860,8 @@ class TestFunctional(unittest.TestCase): def testUpdateFdtInElfNoSyms(self): """Test that missing symbols are detected with --update-fdt-in-elf""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') infile = elf_fname = self.ElfTestFile('u_boot_binman_embed') outfile = '' begin_sym = 'wrong_begin' @@ -4871,6 +4875,8 @@ class TestFunctional(unittest.TestCase): def testUpdateFdtInElfTooSmall(self): """Test that an over-large dtb is detected with --update-fdt-in-elf""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') infile = elf_fname = self.ElfTestFile('u_boot_binman_embed_sm') outfile = os.path.join(self._indir, 'u-boot.out') begin_sym = 'dtb_embed_begin' @@ -5344,6 +5350,8 @@ fdt fdtmap Extract the devicetree blob from the fdtmap def testFitSplitElf(self): """Test an image with an FIT with an split-elf operation""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') entry_args = { 'of-list': 'test-fdt1 test-fdt2', 'default-dt': 'test-fdt2', @@ -5421,6 +5429,8 @@ fdt fdtmap Extract the devicetree blob from the fdtmap def testFitSplitElfBadElf(self): """Test a FIT split-elf operation with an invalid ELF file""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') TestFunctional._MakeInputFile('bad.elf', tools.get_bytes(100, 100)) entry_args = { 'of-list': 'test-fdt1 test-fdt2', @@ -5440,6 +5450,8 @@ fdt fdtmap Extract the devicetree blob from the fdtmap def testFitSplitElfBadDirective(self): """Test a FIT split-elf invalid fit,xxx directive in an image node""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') err = self._check_bad_fit('227_fit_bad_dir.dts') self.assertIn( "Node '/binman/fit': subnode 'images/@atf-SEQ': Unknown directive 'fit,something'", @@ -5447,6 +5459,8 @@ fdt fdtmap Extract the devicetree blob from the fdtmap def testFitSplitElfBadDirectiveConfig(self): """Test a FIT split-elf with invalid fit,xxx directive in config""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') err = self._check_bad_fit('228_fit_bad_dir_config.dts') self.assertEqual( "Node '/binman/fit': subnode 'configurations/@config-SEQ': Unknown directive 'fit,config'", @@ -5470,6 +5484,8 @@ fdt fdtmap Extract the devicetree blob from the fdtmap def testFitSplitElfMissing(self): """Test an split-elf FIT with a missing ELF file""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') err = self.checkFitSplitElf(allow_missing=True) self.assertRegex( err, @@ -5477,6 +5493,8 @@ fdt fdtmap Extract the devicetree blob from the fdtmap def testFitSplitElfFaked(self): """Test an split-elf FIT with faked ELF file""" + if not elf.ELF_TOOLS: + self.skipTest('Python elftools not available') err = self.checkFitSplitElf(allow_missing=True, allow_fake_blobs=True) self.assertRegex( err, From patchwork Tue Aug 16 06:41:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 1666683 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-weidmueller-onmicrosoft-com header.b=mCXX2JDa; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M6M673xnDz1ygQ for ; Tue, 16 Aug 2022 16:42:55 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 1295B848E1; Tue, 16 Aug 2022 08:42:34 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.b="mCXX2JDa"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id C6D81848BF; Tue, 16 Aug 2022 08:42:29 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-cy1gcc01bn2023.outbound.protection.outlook.com [52.100.19.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 720948419D for ; Tue, 16 Aug 2022 08:42:26 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=stefan.herbrechtsmeier-oss@weidmueller.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dWRbgmX3/uUZKU22lVP362SldWQmExrzzSPR15rZFlOn/dICd/RbEuGyGz0wl/mtguQHRhExY4GUWtfN1nFdceWAsCSQbpNnjtH8aVeLP3s87L5mQ+HYOGqUv3GxcIOAbMFJZgYyRiUs5B1RNTmAiIFS//jTVO7PbflL61ZuTl9KFTYp6u+7ug2M5/QSoMGXz0DEK3Ucb3zRdOxUZPmqL/VqN+iyVTerEo9Qf4Qh3hJhHHIjBPBn4E8UZVbKyJPYM5PQ1Kh7ZuI2FXEhabiDHN1DV6ZNuJolFdtBD+jFOx7V3CR4qFgL4qK1xZzpbe5Rm9rJmn3Dc1MjXzvvs1QLFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fPV5zrKdp6JQZKz5HhI7TgDWQttsT5xLyvkmCadFWog=; b=L6Xfoo3Z1kKWqFDoR5juRpnZLHSr6UbMEk9LW5iqp2KP1IaB0pTLssWTOgGF8VIZE7CA4DPGCbXyC+a7adYbw5fCE+FcR5R9sNHrDVWN8ojjswju5YKmW/pUrq5+VA5FlWRSeRAlSL7S2oh+WzwRD97XbZvqxctYPsrE+QLtqwFtfgv6CXF5WiBrqBlqMFbikNsrbiQLgfUswokVBD7T27XrXGFtA3uduoy1ffb/FfsCncWaF7QrYQPIqIvBknVIIFDQZcn4/38V6NMmUgMy0FDC0f+oUtxAJgrbR1WcbR/shQxeaO8czw+TwuMHbaumhmXxRT2Z2BV2FKTq4uWAPA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.onmicrosoft.com; s=selector1-weidmueller-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fPV5zrKdp6JQZKz5HhI7TgDWQttsT5xLyvkmCadFWog=; b=mCXX2JDajOMopNeZbRXmqxjqhG+2txlWuH+spbayPDM5vUhhzXVjHM4RkOjapE2+nZ1PTlknR9SeqNNqNVuactyTgATr4HhtlNuQeL0yCoNlGv+2DZA6Ip9/lNDBYQT8AeW/bs6Bas0hb97RKV1pNtRIVoFCipmmLCNViDHaMRI= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) by VE1PR08MB4685.eurprd08.prod.outlook.com (2603:10a6:802:ae::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.25; Tue, 16 Aug 2022 06:42:24 +0000 Received: from PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44]) by PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44%3]) with mapi id 15.20.5525.011; Tue, 16 Aug 2022 06:42:24 +0000 From: Stefan Herbrechtsmeier To: u-boot@lists.denx.de Cc: Stefan Herbrechtsmeier , Alper Nebi Yasak , Simon Glass Subject: [PATCH v3 02/13] binman: Add length header attribute to dtb entry Date: Tue, 16 Aug 2022 08:41:40 +0200 Message-Id: <20220816064151.9848-2-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: AS9PR06CA0744.eurprd06.prod.outlook.com (2603:10a6:20b:487::19) To PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8bb1b29f-92d9-4d39-07f6-08da7f527a81 X-MS-TrafficTypeDiagnostic: VE1PR08MB4685:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TgwNb7Eb4XO/v/PSR+l0XuHjKliq4ZmLl21QvoPYBR6hE+0ouc9z7P0SykRHgZq//YK3P8/AzdG6as16aEPZDcFBRAab8CvJLctVv5TAh52mpPvMRmTRGcbbUeVQoF1OwsJZQ3a74jAoWpnt2hZpQ89BMCMrE7lsaBPUJzDUG+mE7q7hI8BmFwH5dRLUquNb9oc0sYemTNY02lRNXQGyPEAC6E4asmGljAukXLwAF6qvUX0A0Ukezs/1QNELwUg8aV/qq8uu6KfjLjqKv6XPXpagFn3Bakj1gPqNFAccD3ZsbRkaywwKKPHIN2v+KHXhjA+Oxe94j//q034PUhdzEWM4PA3QRRc2sgpX4xGKBCUAuHu9dMrOeVR0ijE1Jtgpe5KO7SAr5mj4gJsnpL/D6IzpGs03tHOoLBK6YXMr2gpdipqVev+mTThaEAuX6escNRaezxC5HDiX/+qAGGF2Jp9iMHYaDDKpmxqUnRC/fgfsFTrHzimowRKzbRehv0CINeHt5rncZ33EJG2RCp538tXszyPg/z6NXsgalaQjKMvAG/DoRkrvQkq+CYay/7nOKxhx1gtgdRJzPDrX+JXmqZ8prurf/HHJsqzF0w97uwOtOoIXod+tJ9FLsurVQ+DL9RdUkjesqXzBrpKot9ceap7PirF/z2TXZSKEtQCD/WRTVyLpJHrWI35I+n0EIcdpDy+r0Hvz2B42eJ4yE8S4Wd4mcYoi4S6VY5pwwKvbroEcZ2pwRiEC3nxlcoS+ok1/Oad2ZN2IEEK735Xhd9Rg6akIYYcuKunYg+QPkiqzwH9l/U3fTifMKD+49F/BB9Ilw9SF7kxaI3qb3nAU5AEx4W/9nSedEJTDFQ1ADIcgp/N6kNAREXpOmQF7aSLU41BGJoOVospfxIqxoX6qS7kYW8PresaHWBO1Q3FvXPKl43aDTqcq5T+PVZPHVOgjUqDXd1XBYUjdVS5GHYr4MaWpkn06P8kv58Tt2HgvfciqsVUps0X91IbL0Mhn8sFY7rDW2CY5Q4wktx0GDLho6c4jd0wMkAsM2iTMANNRf2QsuDXOYo3Q96S6/hWqvQF74NRgbIAPIohfRwcNn+H13Zmj0t6n0Hg36RuRmU/EmbwtY8U4A/4tDw9mmQBe8teaDYIIonctVamgvxuFaZJK6kZo8CnOZv1bzqKRF1uiEsZwDi1ytZBxJOEeQF3hKyeXRe/LJ1aAdo/qPijK54Y0dQc9pQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:5; SRV:; IPV:NLI; SFV:SPM; H:PAXPR08MB6969.eurprd08.prod.outlook.com; PTR:; CAT:OSPM; SFS:(13230016)(4636009)(136003)(366004)(396003)(346002)(376002)(39860400002)(86362001)(38350700002)(38100700002)(36756003)(316002)(8936002)(41300700001)(4326008)(66946007)(5660300002)(1076003)(66476007)(478600001)(8676002)(66556008)(6486002)(6512007)(26005)(52116002)(83380400001)(6506007)(2906002)(6666004)(186003)(2616005)(6916009)(54906003)(59833002); DIR:OUT; SFP:1501; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: q8hsuRGmg8l0p2GG3XiWkixeaLV4lSQp6nWsEllG3hJ5qhJd52Kvc72yJDyjPpiNU1hbtMu67mDlrKGbslRXw2PUdol6fAgMOnmiSo9dYz7kJyiawUeiWbBCuKSdO/BX/g+iEgH+LRAMyfvo0CBN9lM9VUboYgN/zNwLQ3Enbw90Os6Dy5Rid275Y/5wvfruixsKhMwNkDaFKrnxMvWpV00P/yM1Ee75FHn+VGBFBAXUOgDPEWv7AvVOy/kYd5jQ1aStyc0CZM/sqpQSh67LjgY7abr2p71CreJqk6hCiKW6B3DqQF+NZ+rXPiTqqSe7EezUkM06vCP+/C5/K/hbX+BBY96l05DJlL3BglxF2hP4ezuMaRa+9NQnJ1zPTeDddImi0hQiEHegbJ37vyRCWr7JWfDF1szaFM6LX5j58hL/0d6gGZX0D8FqYm8r/LELpAfA7dN3FcplRxeAI9Fvwqeqp5/sS7IqIgX4CiqRtGlb3Prh5p91cv5qMTvY31cv57uYanspsbvwgf/CLZkMS2fdNXS5p2CDvMi3v1Ec2eU/Ys7sQ7ZcYzduj/xdZljzjSZJkhFq9bNp75GpQV1crc2l+I1ih+o7xKKUPwqpdxXhfXxQPsfuGQPwCCHgH/UD8lrg9Avk8hn7h8K6P8l1ybiq8KKN+f053+/tIASsu8Lu3MPZ9Z3gv3BRSGUij8Zw+9X/x4XMVo8hmrAzZxxS6Y9XIEqcxQ0O+XnJhu6JuvFcOgmWHCWqB/x75gxDRHWBydM9OxP5olR6183SL23Euod/vLf974117FYh+l6jIF+digng48h56oTweiFMmHrfIzfvm0uxK8u6dnUorZJ1QVYhXaSG9Fl0rZxlhx0/pygblu15RG3P2oT11ONJ6R81UWWtNOUtfn4kASYtoONoFhz3zzBZu5GuCJP4DgsRhZkbj+b24ppLZdX2C7wkJQfhmFLrOfVrVuA7XSZCQl2Qn8DLdgYcmgSWiO4JiGLCNsQbYmjjOuNLdDnMijq/kqNiHp+NpJ3nyWB3pVXI5CIdzITX8YN4R/X7gQMCUiA1WFiTkHSsmOvHb+2hts6yItWDKBpOyqTObgQ+58Uzs32DHtDoiZU1idpS2L8J+OH0VbBdWn/WecvMchsQTXmzfWJAyG0RosaVoJSYwtYzCz1da1AXNnxmVOLOZ79UsXKeFuXjx/tOLKuWpC8RtL9f+lxHSTaNZAENIgBqjpGvNLEA0diA14ZflBNho6nu4EaTO9P2wVljkrrAFpXOzp4hJ3OOqmlYSyewq1qB7XV/5UXK52Kh1KMDLN3Af2rTKHUbi5svz4rxuA7MrShe3q91Sf0l3UaK0vGjy7nanqXBJPc+A3QnklfZYOMt72ItOgxtVJDcH84c7jPUcH27ENPiY1jbpBeARdJjqC3O+YV2iXMP/wfU+SMrlKmMGofb6Uzq9LSpIkc4g2+Xpkmru/6lpxA5MeAoyfWBpO6YKjayYchrCWqzrBzNIG+iZR94y2XT5NIzXDRj6OOjqRei1vSLMKWPQcjVxJr4PMujPa9s3fQ552Dj3wgBq7SBPF7wwQfEMzC3PoFxuN1abbUNkU8bWM26MZcYZf3PuWga6wJPVVrA5Q== X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8bb1b29f-92d9-4d39-07f6-08da7f527a81 X-MS-Exchange-CrossTenant-AuthSource: PAXPR08MB6969.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2022 06:42:24.7074 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: gkptW6zr+wcnL8LjNKoHzs0MxRVlb24OpNnKxrxDge1MYiuJXwTtwt2hrbbI3MSWXy3vMHLWwITdwdRE0NFkKw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4685 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean From: Stefan Herbrechtsmeier Add an optional length header attribute to the device tree blob entry class based on the compressed data header from the utilities to compress and decompress data. If needed the header could be enabled with the following attribute beside the compress attribute: prepend = "length"; The header was introduced as part of commit eb0f4a4cb402 ("binman: Support replacing data in a cbfs") to allow device tree entries to be larger than the compressed contents. Regarding the commit "this is necessary to cope with a compressed device tree being updated in such a way that it shrinks after the entry size is already set (an obscure case)". This case need to be fixed without influence any compressed data by itself. Signed-off-by: Stefan Herbrechtsmeier --- Changes in v3: - Move comp_util.py changes (drop with_header) into separate commits. Changes in v2: - Reworked to make the feature optional. tools/binman/entries.rst | 3 +++ tools/binman/etype/blob_dtb.py | 24 +++++++++++++++++++ tools/binman/ftest.py | 22 +++++++++++++++++ .../test/235_compress_prepend_length_dtb.dts | 17 +++++++++++++ 4 files changed, 66 insertions(+) create mode 100644 tools/binman/test/235_compress_prepend_length_dtb.dts diff --git a/tools/binman/entries.rst b/tools/binman/entries.rst index 782bff1f8d..c9336d1bb4 100644 --- a/tools/binman/entries.rst +++ b/tools/binman/entries.rst @@ -216,6 +216,9 @@ This is a blob containing a device tree. The contents of the blob are obtained from the list of available device-tree files, managed by the 'state' module. +Additional Properties / Entry arguments: + - prepend: Header type to use: + length: 32-bit length header .. _etype_blob_ext: diff --git a/tools/binman/etype/blob_dtb.py b/tools/binman/etype/blob_dtb.py index 4159e3032a..4fd2ecda83 100644 --- a/tools/binman/etype/blob_dtb.py +++ b/tools/binman/etype/blob_dtb.py @@ -7,6 +7,8 @@ from binman.entry import Entry from binman.etype.blob import Entry_blob +from dtoc import fdt_util +import struct # This is imported if needed state = None @@ -17,6 +19,9 @@ class Entry_blob_dtb(Entry_blob): This is a blob containing a device tree. The contents of the blob are obtained from the list of available device-tree files, managed by the 'state' module. + + Additional attributes: + prepend: Header used (e.g. 'length') """ def __init__(self, section, etype, node): # Put this here to allow entry-docs and help to work without libfdt @@ -25,6 +30,15 @@ class Entry_blob_dtb(Entry_blob): super().__init__(section, etype, node) + self.prepend = None + + def ReadNode(self): + super().ReadNode() + self.prepend = fdt_util.GetString(self._node, 'prepend') + if self.prepend and self.prepend not in ['length']: + self.Raise("Invalid prepend in '%s': '%s'" % + (self._node.name, self.prepend)) + def ObtainContents(self): """Get the device-tree from the list held by the 'state' module""" self._filename = self.GetDefaultFilename() @@ -35,6 +49,9 @@ class Entry_blob_dtb(Entry_blob): """Re-read the DTB contents so that we get any calculated properties""" _, indata = state.GetFdtContents(self.GetFdtEtype()) data = self.CompressData(indata) + if self.prepend == 'length': + hdr = struct.pack('; + #size-cells = <1>; + + binman { + u-boot { + }; + u-boot-dtb { + compress = "lz4"; + prepend = "length"; + }; + }; +}; From patchwork Tue Aug 16 06:41:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 1666685 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-weidmueller-onmicrosoft-com header.b=tWmcNCti; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M6M6G2FwWz1yft for ; Tue, 16 Aug 2022 16:43:02 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 27FF984228; Tue, 16 Aug 2022 08:42:39 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.b="tWmcNCti"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id B746E848E1; Tue, 16 Aug 2022 08:42:32 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50057.outbound.protection.outlook.com [40.107.5.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 0338684512 for ; Tue, 16 Aug 2022 08:42:27 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=stefan.herbrechtsmeier-oss@weidmueller.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TyAVWodrZXfrvS7iGh13dCjYpQ27VKeUHt6E/hgijACQe0XQdhO+KPsCsLs/BxuRiq/WfL5cCy+c8wvUCzt5d9TTgzPdGKVH0h3tdX15gzD6SW0TZo/olv16DIeq16ChFX/otc6CumZXrHZ0x+TZYyIw3a4vZ2zhWglzC72Gk9BWeYm+YlCl2lxplW/9bEef/0srpN6poHO1N70B374j4M4t+mWa01i/aFi0G+bzopPUiabDWosb1ksf2uq+wMQuyn9gJiH1iXBzms8lDrG6jbelrvgb1eEkV38J80HFK/47/HKGcvjxx6q4E1NkZD2oNww5dlbIqkkXa/OA90Em/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UNPQf20GG0Fu0o+pUfUf5FyTWEC6vY7fpKp4q387xsU=; b=XOQt6BHhh2jOR615tddS/ucNRyhMg5PFZrmGkiHU5wTVHkd5ynTL1OMZwIW5eZnj16GAigQffaoNmCooiuLJ4jPYjzdVo3wrq1xIfeb6aqXTgGovcbC6mytRhlwdrUJ2F2EJadplm/q6gpCTwefvwf+GJ4g/xIMtOoGWdUzWUjU6G2k+O03e5SUBq9JvNu/hZehdiSJOYoJ09rXG0dKrW4VDCyl03YjMWD0Lkii45mD3u8RvZzjF9XFc14hhBKu6pKO++6gWFTMOGNg1ZllP/3OSvEMfFjEXuklLhctxSxJa4fwm+kaLmsLY7HawMYS49J54qdfc+kIgAnSFFZiJng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.onmicrosoft.com; s=selector1-weidmueller-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UNPQf20GG0Fu0o+pUfUf5FyTWEC6vY7fpKp4q387xsU=; b=tWmcNCtiQdcr0GRlBy+2rW7Kof2g1VXGo8X971sbZJ/HdkbC+27ErdYnVx/ReKHZ0EuGA0NqFr6vpkb2DHNa0/4tdYMYEFg1kWkNSFSAlq+IT2/wv7F5gmpLYIY8fg3uyAbLTxJi1e8z52Du4yJ/TBWX3xlr55myMG5ReOaq/10= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) by VE1PR08MB4685.eurprd08.prod.outlook.com (2603:10a6:802:ae::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.25; Tue, 16 Aug 2022 06:42:25 +0000 Received: from PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44]) by PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44%3]) with mapi id 15.20.5525.011; Tue, 16 Aug 2022 06:42:25 +0000 From: Stefan Herbrechtsmeier To: u-boot@lists.denx.de Cc: Stefan Herbrechtsmeier , Alper Nebi Yasak , Heiko Thiery , Simon Glass Subject: [PATCH v3 03/13] binman: Disable compressed data header Date: Tue, 16 Aug 2022 08:41:41 +0200 Message-Id: <20220816064151.9848-3-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: AS9PR06CA0744.eurprd06.prod.outlook.com (2603:10a6:20b:487::19) To PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 89a52549-bb19-4a60-4e1a-08da7f527acd X-MS-TrafficTypeDiagnostic: VE1PR08MB4685:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: v1b43/TsoVfsMMbwFECTOU8fk/EvMNYbaX7Z7EnatazhhCV4vuSFWboCBLUb9l0TWmjzoOy9S1qt5cb6R0PpMSl27l7OW93YcGtBnpYVoQfW5UQczJWfgxLgIGbltT6Dam4OitdYM7elV3yZNzvb74yMvvLuCgUM4y66DHZQS4h3lji9gZlcji5iMu5cHUbuksb5/H/gXDJWwrAg7PxbU107GEtBNOZl7O24/DcafL6G+xhwa77ILfRbHRu1Uay1tuxgAZAEMMSsyyIZS7HAXg2zDVCkXKgr2kxrlON0cyDssCdP9aexzccc4kfrTlERfkLsAidJK4+IDMtMJ6mlbNG3JbQdTYM0ns4fkmGqV896LF8pTz717lmTDZl6NZ8Yl+mokslpqZcOuSymAcVYLUSsB5y7MV7a1u/WSHonC+tHCDHAKt3U3Vl+n1oX+d1W/ggus4AZ8h0AdFmMMt577wBCGGi77bn+Dl5fPliJAGccOWxrDvtgdpY3XtmBLu9sCLR1Dxl5bYCT1Bk0XhBaNm4RxjES/NYLOWxvu7P/ldsOymTFHCh8u5M0YiDj6Olceh7s4lJ48UuMYsAExaQKLUk3tKf0ir2QgaO2mXGiZJrjweOQ2lDllYZUlThxA3MEAjHF8YuUIq0eN+s5NbyafGm2faAQNtQmOJnU+PpeDO7t9hlpICGXWu50u52lu7upkGXkGc1pWLyz8TxvilL67gW7QwHdxVPoxFZyfITU0uO5krjfk4WUoI1A6m8lVRiQ+Ih4C06elk+2AJA8u7Xs3reaHRv4S2uPjjrdb/kHqvo= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR08MB6969.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(136003)(366004)(396003)(346002)(376002)(39860400002)(86362001)(38350700002)(38100700002)(36756003)(316002)(8936002)(41300700001)(4326008)(66946007)(5660300002)(1076003)(66476007)(478600001)(8676002)(66556008)(6486002)(6512007)(26005)(52116002)(83380400001)(6506007)(2906002)(6666004)(186003)(2616005)(6916009)(54906003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: LWrq50pNXgRbIxPGz1RB03E2RnoHm01yzOrzArLYm7dX4yvQSza3gH9S3C7fIm2lQuQYAq/kMyMfYIiPNRQW1UzxgEVP2zq/AaQ9lsqHaCFCgUgwdVtbRCnesheIlD9tNGvhf/gEztPMku2i49n+bQApKdqSffKxfPQZvWgxgXNO7CNG/18vD908SPRWB386GsxcljukW7ze6MSqc7oKJeejNhGXPRy4+ONobFY0e69i04lqxGFfxTymyvGTGXi6o2a7eSipRQl5QSVxtPxsMWMpwhmPEXzaRkb480jl1oIOdeqCHNlWkoFWliMAlskFt/+vx+vG3fLNR4VBjrzMGJGHRtJdaKkD25+jCaSLurnQOhukMnRIfHWHrJC7xqB1n1AgLPxmthAb0Qm293CUSTPE6+dFVP99ai3BwLRl09ea+eOgy3wQHfmVkiInRu/v2mo3YoM3UGSckdcJL4IKJQolbEKmDsTusJyqTXFZTmyijJT/ubi4UMaEAk8u8oilJ53tJQ2PtQNM5RL9TEXrzR1Y8In1gkBggFEGNQSWkS8FPvKGZpqjup8adtMxnnhx4qz0Kc4yrEP9J9S6ti2EBre5IhuZntWhZtGN6PV7bVmd7+yQ1WLM3KjjNREXXI6bn11lm/Em9hyfWQK/DcxsyDMeooH4quN1LgQB+UecEFobv1kBx835WIYCfIkEPx4/HLuFkZZE9Bi6kuE7IUxlkGFk/23SoLWPhfx3a5eUAYZflcLNx9yR6DK2LP/jMGiX3XziDYkOkDkpmpAhrEEi3DJ8eIWy2gmTZgPjcJCupwN3/wNyq8OQlNoYrs96Ge1ZCKkNk7r+1XQsmwNBxVlha9ooPLG/LJzLhzZurJ8RMuE4CDVqXHBShvBgGsFYvnPrdPjKjSTQK35lJnsbcPI1/u8DglV1X2agb+mmUfObppqPC9LctAUhCnPuQ7qIRwQf0/mkPtCHSZDbbqrcQQ08nLbPCNzsYAa2CP4nLkSh2ELA0gZrHlj3jHygspHW6aI5SLreGonB7FLVvGeTtVol3/C1yLbkQ5zMAYZOEGZQkzW5DWwEZHR6+GKK3RAyBHJPAuOVy1Ou1i3e55jHeMkcD390u69frIHpi0gdJhMej0FLHdOWukgW7VfX0ALy86vihU7eqs2b1ZNPmmaDLKHQ8eRO3p8awn3TAMmwanENmVR8VTcR/gr4lQzED06vJBegrDtka02B1t9WDxeHQW5Iz2Zq+Ttn8+wH7iuzX4WBKpit8tO2Ty07jq0APKeT3rw/ABUxGIsPWjLuBQ8fNW3woiyCIasm1DUjKEBFyzJATqik+PDKMr4cZiqn+IbO+ijOh2mHncMZOEOVPvGAqUaa+ZMqRT13aG9lcrvAQa7LVfgil5CQyPOBPKgGh1fftFhImD8Lmf4ltX7dY+LdkXjg4yr7cqJw8/pZmhAxn8rIjH3mUrPDzThqd/qStmhMd9nNcIM7inbWhgGE4NEjnJ8NNRm7QdOUF+9T2g7JbbxG3UL9WIYCpBsa8VdLEMghXS5wZIAVQ8cw0tFg+ODZHmuSGSWRSemq+Dvsbq/HYTxONhYYDI33G5suwnDPTpw84Hwao9JU8GxO67ygaSWpnUuJSg== X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: 89a52549-bb19-4a60-4e1a-08da7f527acd X-MS-Exchange-CrossTenant-AuthSource: PAXPR08MB6969.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2022 06:42:25.1761 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: n8s9noWG4MarQS15BlVKovl91GBFGStZDhNe720upZbfDh86lp5W9xxidA6/v5tg3Yy1wYQXQd7iVWEZ0yJeWw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4685 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean From: Stefan Herbrechtsmeier Disable the compressed data header of the utilities to compress and decompress data. The header is uncommon, not supported by U-Boot and incompatible with external compressed artifacts. The header was introduced as part of commit eb0f4a4cb402 ("binman: Support replacing data in a cbfs") to allow device tree entries to be larger than the compressed contents. Signed-off-by: Stefan Herbrechtsmeier --- Changes in v3: - Added tools/binman/entry.py | 2 +- tools/binman/etype/section.py | 3 ++- tools/binman/ftest.py | 18 +++++++++++------- 3 files changed, 14 insertions(+), 9 deletions(-) diff --git a/tools/binman/entry.py b/tools/binman/entry.py index e3767aefa7..a45aeeaa59 100644 --- a/tools/binman/entry.py +++ b/tools/binman/entry.py @@ -1079,7 +1079,7 @@ features to produce new behaviours. self.uncomp_data = indata if self.compress != 'none': self.uncomp_size = len(indata) - data = comp_util.compress(indata, self.compress) + data = comp_util.compress(indata, self.compress, with_header=False) return data @classmethod diff --git a/tools/binman/etype/section.py b/tools/binman/etype/section.py index bd67238b91..0a92bf2386 100644 --- a/tools/binman/etype/section.py +++ b/tools/binman/etype/section.py @@ -777,7 +777,8 @@ class Entry_section(Entry): data = parent_data[offset:offset + child.size] if decomp: indata = data - data = comp_util.decompress(indata, child.compress) + data = comp_util.decompress(indata, child.compress, + with_header=False) if child.uncomp_size: tout.info("%s: Decompressing data size %#x with algo '%s' to data size %#x" % (child.GetPath(), len(indata), child.compress, diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 1b468d8e6d..d082442bec 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -1967,7 +1967,7 @@ class TestFunctional(unittest.TestCase): self._ResetDtbs() def _decompress(self, data): - return comp_util.decompress(data, 'lz4') + return comp_util.decompress(data, 'lz4', with_header=False) def testCompress(self): """Test compression of blobs""" @@ -4449,15 +4449,19 @@ class TestFunctional(unittest.TestCase): rest = base[len(U_BOOT_DATA):] # Check compressed data - section1 = self._decompress(rest) - expect1 = comp_util.compress(COMPRESS_DATA + U_BOOT_DATA, 'lz4') - self.assertEquals(expect1, rest[:len(expect1)]) + expect1 = comp_util.compress(COMPRESS_DATA + U_BOOT_DATA, 'lz4', + with_header=False) + data1 = rest[:len(expect1)] + section1 = self._decompress(data1) + self.assertEquals(expect1, data1) self.assertEquals(COMPRESS_DATA + U_BOOT_DATA, section1) rest1 = rest[len(expect1):] - section2 = self._decompress(rest1) - expect2 = comp_util.compress(COMPRESS_DATA + COMPRESS_DATA, 'lz4') - self.assertEquals(expect2, rest1[:len(expect2)]) + expect2 = comp_util.compress(COMPRESS_DATA + COMPRESS_DATA, 'lz4', + with_header=False) + data2 = rest1[:len(expect2)] + section2 = self._decompress(data2) + self.assertEquals(expect2, data2) self.assertEquals(COMPRESS_DATA + COMPRESS_DATA, section2) rest2 = rest1[len(expect2):] From patchwork Tue Aug 16 06:41:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 1666686 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-weidmueller-onmicrosoft-com header.b=qtc+JB55; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M6M6Z5QDHz1yft for ; Tue, 16 Aug 2022 16:43:18 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 2330E848A6; Tue, 16 Aug 2022 08:42:43 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.b="qtc+JB55"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id C39C3848AC; Tue, 16 Aug 2022 08:42:33 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50057.outbound.protection.outlook.com [40.107.5.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 2670C845F2 for ; Tue, 16 Aug 2022 08:42:27 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=stefan.herbrechtsmeier-oss@weidmueller.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FD6Z5KPkQPO/SOqICY8dBJmL6MiGLwpNpG/5I7PRhBIn3WIM4C8GU2CVIDsehTxvLmeV/pPF5Ga/5MNRpnpuaAFXNfAG9MTbMStCcgu0wuZi5sRz0LF3uBapMC/qNWvHxx9h0CQmGcYvrVEz/iKePonD57zvO9+C/i1Y3PVAsVcvwcyk1k2fDANZ+mrQbUuAX+/TcHprNuhETn0tNHLvTwuBvucInY5LApCAz9Pw4Pj6L1bBP3pgEzoP/m2TAZiYvAJ0Tq308bmAy0tj93hnGOuBIVagomILuCkYelJWxeBXicf9i+zav0fGTGIurC2NinOyRvGoSz0Kpzia6KN6tg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QniTEMk3Kc+/CXs6z7boArcNhGcWq217d0B6tyEKUnY=; b=dLZvK0YHsBLH0gz3eNZVj+MrLL1jtYZkViUJiMNM0Ux2NwvGIXPWlRh/3EYa7TMuUqEZ3Sv3vXsnq55VhRHaAdKi6w7hJlWH7qrybqx0YPJYg1aHyc9BMcPMcSpkwRLTV4B1JYQ8DfiORSICHzCW/3GoqGSKzpVnzrLIKF0p9FjXm07kslqpjoZyAD0FCaGsK4yBIGmS9y8eg07c1hjd0IioGBrQcO5kaWbezGJtu8/KdHLPGM+E7oNIgI0shg353N4kq3XBBuzlzSR/Hzn7TBa9Wq/h/eIaUL4bYtxqNcRegATegSOPFAe8ALh55Wh6HfH9ykqsjzuniU2C4v9tWw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.onmicrosoft.com; s=selector1-weidmueller-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QniTEMk3Kc+/CXs6z7boArcNhGcWq217d0B6tyEKUnY=; b=qtc+JB55kPaPZvIALd6uCI+/NOpWc2qn72nv0lodphBogoZUCL0xcI2Isq0rwECLENzj1GH05E2sd8h3SLxcP2pTkun2ixDfneKkkGOQJkm31lEoY22Gn6QIvhjs+VJdeWJMDiT690RJFd0hP/Xf+ShcUuDdyCzfXGTyZj7DuBo= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) by VE1PR08MB4685.eurprd08.prod.outlook.com (2603:10a6:802:ae::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.25; Tue, 16 Aug 2022 06:42:25 +0000 Received: from PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44]) by PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44%3]) with mapi id 15.20.5525.011; Tue, 16 Aug 2022 06:42:25 +0000 From: Stefan Herbrechtsmeier To: u-boot@lists.denx.de Cc: Stefan Herbrechtsmeier , Alper Nebi Yasak , Heiko Thiery , Simon Glass Subject: [PATCH v3 04/13] binman: Remove obsolete compressed data header handling Date: Tue, 16 Aug 2022 08:41:42 +0200 Message-Id: <20220816064151.9848-4-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: AS9PR06CA0744.eurprd06.prod.outlook.com (2603:10a6:20b:487::19) To PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bf5f0874-754e-4c4a-6f25-08da7f527b12 X-MS-TrafficTypeDiagnostic: VE1PR08MB4685:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cvLYOQGyfN4e+Js4DvCh3J58ICTUufwvgjvAQG+j8cfXJS1RPPbaCe7Cl93T+G5l62IXtF7RJrHWurNhkeVPsFcjIpasCYV39pEaRIucA1UvM5wnJ13vxGVXwghG1sCY7p6+zcc8jHsbGVG+aofr84mM7IWSid6On7S/v/baNnoW/Dvt8FxhnAxwBEqcma1kAsxof92rvtfJ9xVVY1zKPehB4kGYvhuDwe7/kW4HFv+3KFPJ8hNxh65fvdnl3OnHopt6Lo113w2iemDhcnKVDzSnbGLeTKexVPNec/J9C5EmrCbnt9ZtDF6G7quoXTdl2UNCC00EghidYKtBkq0OtBpop4YBzAzbrl/H+a6n9i+nXMCrObtaHKYmxQQ7MC3HdswEs78cVYt+61N6dOhxX0EyiakV2n9PQXGUHOyHPuw44zwY10OQSJPvw0ZywIsjN9GxSdTa0g96PYAxHvptgO1DcjdheFh4NU02VCg1vXh+FXbqaWvQawWpN7OSmo/wNW+IpyTo4EE/v5pfBhMM33BfSEslKAlZbMYtPUQjCrsDmFmgNI9bBeParFRwO8OBs5cF1UU9BKOtkTgAaq7jffgG3oBg/MWKusYxha2Nny9sD0bxYe4xdOEYT9t+MPzEgKpDFnrihhXYlm/zq6W4VQ57CoRt04MoK7/mvB1eqbxbb6rfJGxrh6cRdQOD4I0ds0AGj5FpLOV6K6Qu5UIklEOcj4MY+OIPPLKeMF4wc6Hxz43wHmEcHKMX9h0m3SF3wGNycxeNAooctS1dNLrlLtro+/8xEHQYcJs/D/vAH5Y= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR08MB6969.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(136003)(366004)(396003)(346002)(376002)(39860400002)(86362001)(38350700002)(38100700002)(36756003)(316002)(8936002)(41300700001)(4326008)(66946007)(5660300002)(1076003)(66476007)(478600001)(8676002)(66556008)(6486002)(6512007)(26005)(52116002)(83380400001)(6506007)(2906002)(6666004)(186003)(2616005)(6916009)(54906003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: s2GvnMKJpDaNtCtEjaZFyrQYzXuXAQdpmO8993XagEKMQJNtnB5oVgIKfR8kYPfMFn+YmlidXPRkOYqlZf5GAeeKgjA6CatPYaVw5JPwWcbfvD8o4YrzPAijAtGHeHAzYZumVPZ2+p3gb3C92b/Vq9GYTiOBTql5PQt6Y5g5hxMiLoMtdm3g08k7oO0+dbah2Q/iosMMPyubO28lkKiS5V0ixBskEUHvAUP3EoQezcv2xCvB2L/ZfymvSDj7lQ0DYVwnTcbOLy5CsZmyBxN+KzLEqc41xZTkwMGnbNBRAk65LSL9SmL4ljCpmJDSQTUJlirb7EklNTgkNr3zsUd/QjoRrgvSonTJGeaDdihQEGGn719FXBJ57zxAY6Fv+S+ZiEGAXgBA10LhDWdGW76oA74aXI5zCzuJRYpVQVA4I4YVOJ0/w+R7T3wFK7IvaUh4K1kmVJfdVKPu8tXmYtLhh81CBWBT1VIvs0wHZjQr/kFlLxDqD81IpUDzUJKeHPxtJWfsoa2ZNz//ZxRZx01h629GVIVzmJiNp/YisTuRvXHnUepUc7J0Y8RjOx+U5/XB4V0Vasj8I1V2e2ZGGmOBwdg7tBP7ApkCdbP3OLNFaoyZFj1vBr3TraD85BOuEe8k7DeJTbH0TblEgVeJRWzKDWv6eN5v8zqOAW1n8wWjwLdlucU3V/hdr6AsCtFUimmlUg+CLHCbY361ghzPD5RAzBH8q4lkDlJkNFJkLmf+ZOkAR0b/Key5w6C/7dcLd+guHkBCk9XZXGZ8X/fiuWDiE4oWr6/2/UlfYUEwKjvstxVbESBW84Su1HSLT3iaSk6LjIjWza1GRxtwxJ2uhbzJEHwfKaU3OLJs49CpS9ZXbHMDzuuCUtfcX+rOvFuHCr9BzghaHVaT0CGIFHfIDGutr+9woItrevmwVTzW+HvHpoNGaAAVTLJ+x3jd5a9taiXQr09RAzMC9OYe4nLy319/GI2IUhTTqVzezGKazEFEWJK9N5CR7xgajX17hfrL5QH40lrinaEaIlAf9p8iK/GvsqBc2Gh23gwIXpq+Evipn6kcDwuGGE/lmaEHp+97aXBWSlVPxWAMIEr16ZqpZ+zzYHC3wt+QCxBkXtqHiDHzrB8H2OcEOc/MgSizIFmSxnPPOjVCUNGhKDOlYTsufMIQx41FydUf40i6eq5DHIEyB4TndJtMZgUPIaqp9ps/1SopBrj3UcSLey7VltvGs7S3lD9R7Ni4FbT1MEeVWS3AFGZO1xyXVm48oXWu8DLtZiWr/w+y8Z3cv9EtZpk2DVo69chUr2BVAPXOt92bXneDVL1+r3JXzOppxy/B7Ob+blBCT7F6t744Mfltzz/cdF7+1XHZBXA8hK1x5Yxqw2JiIB4o7w8WjTGaFbBqMuSO0Ws3KrrSdwL1vpNwFLjyfODV3gzBCBg2To5h322YjZPwcKxzfu1h2AIOi+Ge0Gyr0xCzZsXQsyLWzifWdbGa3Ojs1GxT1gzL87jTARREPhngUarCE2rgSvdshF8gkwH3/5gZ+JrBSE1B/T5f2i6TinJ+Wu8u8qVG0fQgd0z9Nyzcqnjmr8eopHZ7H8XmYTSs2gwGjVqfUFPStkmWyfTsRsO8NQ== X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: bf5f0874-754e-4c4a-6f25-08da7f527b12 X-MS-Exchange-CrossTenant-AuthSource: PAXPR08MB6969.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2022 06:42:25.6283 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: acgHa8XtbvkdVidGP3LRZdl4V7gzA3DBAIXhba4nbwpSlrM4JspOzOGq3mTMxJrfTwLhegMvMzHRfSKZB7kAUg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4685 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean From: Stefan Herbrechtsmeier Remove the obsolete compressed data header handling from the utilities to compress and decompress data. The header is uncommon, not supported by U-Boot and incompatible with external compressed artifacts. Signed-off-by: Stefan Herbrechtsmeier --- Changes in v3: - Added tools/binman/cbfs_util.py | 8 ++++---- tools/binman/comp_util.py | 11 ++--------- tools/binman/entry.py | 4 ++-- tools/binman/etype/section.py | 3 +-- tools/binman/ftest.py | 10 ++++------ 5 files changed, 13 insertions(+), 23 deletions(-) diff --git a/tools/binman/cbfs_util.py b/tools/binman/cbfs_util.py index 9cad03886f..a1836f4ad3 100644 --- a/tools/binman/cbfs_util.py +++ b/tools/binman/cbfs_util.py @@ -241,9 +241,9 @@ class CbfsFile(object): """Handle decompressing data if necessary""" indata = self.data if self.compress == COMPRESS_LZ4: - data = comp_util.decompress(indata, 'lz4', with_header=False) + data = comp_util.decompress(indata, 'lz4') elif self.compress == COMPRESS_LZMA: - data = comp_util.decompress(indata, 'lzma', with_header=False) + data = comp_util.decompress(indata, 'lzma') else: data = indata self.memlen = len(data) @@ -362,9 +362,9 @@ class CbfsFile(object): elif self.ftype == TYPE_RAW: orig_data = data if self.compress == COMPRESS_LZ4: - data = comp_util.compress(orig_data, 'lz4', with_header=False) + data = comp_util.compress(orig_data, 'lz4') elif self.compress == COMPRESS_LZMA: - data = comp_util.compress(orig_data, 'lzma', with_header=False) + data = comp_util.compress(orig_data, 'lzma') self.memlen = len(orig_data) self.data_len = len(data) attr = struct.pack(ATTR_COMPRESSION_FORMAT, diff --git a/tools/binman/comp_util.py b/tools/binman/comp_util.py index dc76adab35..269bbf7975 100644 --- a/tools/binman/comp_util.py +++ b/tools/binman/comp_util.py @@ -3,7 +3,6 @@ # """Utilities to compress and decompress data""" -import struct import tempfile from binman import bintool @@ -16,7 +15,7 @@ LZMA_ALONE = bintool.Bintool.create('lzma_alone') HAVE_LZMA_ALONE = LZMA_ALONE.is_present() -def compress(indata, algo, with_header=True): +def compress(indata, algo): """Compress some data using a given algorithm Note that for lzma this uses an old version of the algorithm, not that @@ -41,12 +40,9 @@ def compress(indata, algo, with_header=True): data = LZMA_ALONE.compress(indata) else: raise ValueError("Unknown algorithm '%s'" % algo) - if with_header: - hdr = struct.pack(' X-Patchwork-Id: 1666687 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-weidmueller-onmicrosoft-com header.b=Z/xQmiIB; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M6M6q1tmtz1yft for ; Tue, 16 Aug 2022 16:43:31 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 5E0B8848F0; Tue, 16 Aug 2022 08:42:47 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.b="Z/xQmiIB"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 6B752848EB; Tue, 16 Aug 2022 08:42:33 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50057.outbound.protection.outlook.com [40.107.5.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 4848D848A6 for ; Tue, 16 Aug 2022 08:42:27 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=stefan.herbrechtsmeier-oss@weidmueller.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lNlNcDcNO53tg7F5ZXo03c5M+glL/7UULwE5kNQWx9A6nXPyLTOj8R0PoFL2VgAzKtrNS6vNiMXrlNtwtXeonCy+otsoTd56JJg6ULLlFCyL6Ecis4F6EwX6kjUQ3ZY1y9Qh9GpoZFStung80TFmTtbRZPCsNDcLMoQKtT7kQDvKJ7gCdj0foXc0TqEhXoB19GMTfToLAcbAJFEB/j2E8Ag7PSOAdOOLUgSMWobcrmbsQBvoFvdETuMLAjeiV7AVEb4nzmMSt6HArEOxDnbgzvuWTRnovwyFqbBv81UMxqG9MIepeGmRbXvO1ZWgxSCluDHusUL2XEZGJSha1I1M8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/aDKAbshxZqz/YTDq6Q/OC+XD1ehoqXzGpt7T154zMk=; b=iP4MmB9uYwBgIPy6t6svTRfyXShIdIVXZLugZ9u9vDJPA9K68CYL2+0jq5vuA/sWHNa5fTiuVUHwKRSjJ5vhK3KntIj9jUb5Uqdtko+y7Om7JvGTyv3Ff8lSXSymC4cz/q5x/P0QQURN10pZzEeWjK1ofqQRpOc7rDKmgZlplVCuOE0ydYsYJ6Ho4ZWZVM9Wfc5FXCqMfcbL0tpymPNSocXkH1jZ1uJ132v18ZBCwApmqhLJRVVjsTos9CjCXrzjZkfmtxIQbMb0/5RK394XMbXCVlpjxwyGhFr19OFcuMMTKvWXiMBnwxlt6jKDxyctBsGGps4N7QmwHRfkJ/VvTw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.onmicrosoft.com; s=selector1-weidmueller-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/aDKAbshxZqz/YTDq6Q/OC+XD1ehoqXzGpt7T154zMk=; b=Z/xQmiIBoHqec9qm+UpUkALvzjzWJiu+sTmrqCF8QlyL+MVl9Utd1vVaiCJLZMh1GXloNolcAKAcSqwR58xRjUJJbf4PD00DfdyNF3xEzZYZbKsWGGFdpwy7qoGmu7C/j63GLYQLrS8lEOwKLYmg50rgPWKZ6TEnDzmumL1r+0E= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) by VE1PR08MB4685.eurprd08.prod.outlook.com (2603:10a6:802:ae::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.25; Tue, 16 Aug 2022 06:42:26 +0000 Received: from PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44]) by PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44%3]) with mapi id 15.20.5525.011; Tue, 16 Aug 2022 06:42:26 +0000 From: Stefan Herbrechtsmeier To: u-boot@lists.denx.de Cc: Stefan Herbrechtsmeier , Alper Nebi Yasak , Simon Glass Subject: [PATCH v3 05/13] binman: Simplify comp_util class Date: Tue, 16 Aug 2022 08:41:43 +0200 Message-Id: <20220816064151.9848-5-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: AS9PR06CA0744.eurprd06.prod.outlook.com (2603:10a6:20b:487::19) To PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ffe740bb-cfd3-46d1-0dfd-08da7f527b55 X-MS-TrafficTypeDiagnostic: VE1PR08MB4685:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lLspUvFpzQm64cZ6+dvJiPKn6h1MZW6co0McNghWXWMAxJ3i/VZfUnmjttXfwdOvcWL3wqLCFcsvMnKFIF/3sVL/V+wvyUUuKJz6nnEbaXVRNd9DAe87gXfihh6UcVJu82Y32ML5gAQbdT2gtbyNX6xRxYUBEYCJ0UvPGXAB7RXXxY+6Dk7897TuDaKXpimy1+GemWfUqDV8Ej6Kmsm3MzUeMGufs/MvxO6feROAyM/aezbL3DLG3aOQCzbYG/xENJuWcKBuaryM1EWo8Qn3tf4ytQGH0E089H8/Oafw/aUTQQaISCV0J/cLvi6K0GWSz711iHtKey2IGrhojTPiOXCf7Tp1nBWA8ZE1P6mMUrY0n+lRFv7HhyVLEi/i0H94HZWB/NnQQmCHuhXC+HgdichCAIW4FxzEjxA0DkBryzzJWKfxjBiOWes6gs2w4AxPRG7/W7JqXWBUi8kZhN8sP2AKoxRgUwQLjv9l10BO7sc6E2EIAz+tLjnyvYFDrGbCNscEq40VnhHDtKU/N1QwVUKj6a0KLsPv6gWXeMMm0uDSFaEbsfT3muSV0SS+gAVugyaB5JFhWAqjAgFzeSWoFGwOwSwuhwKY7670KKFhyuaK9hvqwO6Z4RAJBfjtCcb/uIh27mzXqzgd85r95OhiGDeMvB0c07kQJ3FTyi9hHtOikspBj8iZeC8CZcgKZ0CrqxfhzzJrL9yrDNWvLX9zsMTVk4d+I0kzZdZkK6B9NeBt0BFWinqjgIMQRzdJX5vU1AntQQprK7qGng6dxe7bwoFUWYqpHxsKoA4TVBOAtPU= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR08MB6969.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(136003)(366004)(396003)(346002)(376002)(39860400002)(86362001)(38350700002)(38100700002)(36756003)(316002)(8936002)(41300700001)(4326008)(66946007)(5660300002)(1076003)(66476007)(478600001)(8676002)(66556008)(6486002)(6512007)(26005)(52116002)(83380400001)(6506007)(2906002)(6666004)(186003)(2616005)(6916009)(54906003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?hObMlWxhc5fFx0kiwd4RUuCHKFHv?= =?utf-8?q?PGx4sBMpK4LaF1Qt+JFmE6JbyOpZVup0FSXFVjcPznccoqzZBzZWGzTprqznOaRa0?= =?utf-8?q?GfHjyIC5z7qo/fkmh5RWvWzuTI23JIxi5umuX+Vv9oNtmIAvzbdwMQ7+BgxBv4QqR?= =?utf-8?q?UEUWxaRlbq4TqZglz/phmpljmj0y6AULUKP34sJNLnJLD8IHGK/Mwwa9+m/5xxfZY?= =?utf-8?q?9BCTz5ed3TG80hSQi+dJEAGrTSc2th8NLlLszL1aTTgEZwCPkiHk4reHWk/15hRxH?= =?utf-8?q?2uvuGsRDsjnYmyO76fg+rMD2w8AGGwD04n64Xt/1vdcVpey1J4//2/ILKFaAaaJ5C?= =?utf-8?q?1BsJseC5WRFaYORnbjdl6Ri1VPT/lzKuW0mjMx18sQeagrGQFjAIiO7tLH1+hQQaw?= =?utf-8?q?slWSTB9tdZyaFd33c7vmjVxJ7YvXESNPyv6SWD2Kk6TibDFnsaLcS3WrRZ+b/e+HU?= =?utf-8?q?T2apD0mwvGIkuxuzcn9LUeWVgvYKqiLXPGGA7a3WpyEup7XCka7N6V+axLRjuIXw+?= =?utf-8?q?jIT61UHYZfrYj++3IH3dmzha9dpm7gR2HAYpkkDGpc0v/7nNWchbZZbPlt48WsF48?= =?utf-8?q?boyJKEPxzjnYmfxDVfvE1DYSyDKCytuGiM/8h/gNTdVcPzodi+UBXwx0hanJqoxGK?= =?utf-8?q?CL7k4HRj9Ai4K5aC7e6jgBCN4Vtc2fmnOmqdTbDvMcrkNY2ESdBnyPUmM2OUDjmdx?= =?utf-8?q?vMNC/4zLwHjxH0dAePk2z6FJzn4iqU81JKce1gAmIGW3DqoQ5QOqp86+s0o4QzSmM?= =?utf-8?q?6TG1/90Ohc8HQX/jAhIktgjFjQoQn8Xih8ty0ktrEjAAuc7U6RTtcPw1UJUhgzKuR?= =?utf-8?q?phHRMN1lbb5Kk2bKqbAZHgLIEQE7+25T/OkOE/FC4dfhG8vrfE1Xs6NGBt5YfXxpb?= =?utf-8?q?qkLssUZ6iWQl+0LEzgE978cA7lt6zsK64zfI0ASCF4kBhGxsB9TN/3kvHLdMZWNJf?= =?utf-8?q?p4sipIPR/fwzvGpY6+IwCp98awIqg6DPL5ieXYvrQXwNXBtk0Gb7fyHrd1cG8P+cV?= =?utf-8?q?m1b5VID355PCw76xWjixWfZ7q6vmWv1bwg4HjRhFj99/qIGHnbkNNMi1EjFnOcKVN?= =?utf-8?q?0lag3k4OqziwnOJx144X9rAoOIippW9zUGzo0hzGlt5IwXphMtokW9xcBVLlX8WYU?= =?utf-8?q?ain6TSzQaTSyHmWkCxQEl+Vv7kntzeU/ZZDrcvtwymQX6Eyc7y2+XYDF0y2FYJU6/?= =?utf-8?q?5pExmdJ3YR4QIADZ8bVXO8pc6PcKLtR3lX8T/PucHNOfV5laNP0ky1W1k61N6g627?= =?utf-8?q?yVWvYm1/OBnAIWWvZKyee7AZSrg3lqHw6qoKc6UMT9l1bHL7NF+3cUWW8xs0oX0xr?= =?utf-8?q?Q9gHc54pRzukkc8lXJeYgjLN3+/WY8oZlVk/0c/Qu1XJUkU7k1k32pIJTGPQKWokN?= =?utf-8?q?1pCnteHyMbpU4lmYleMAE4OqOyMH/TyQdimEKh2Dc2csyfY9O5XLRgCeeUPuEuMA9?= =?utf-8?q?dg07p924uq9Bk+cdku7QqZ6nzGp/0HmWpaqzceehjtyCRW3TqW+bZbMgg2g8RbcYk?= =?utf-8?q?+OWKLSDkownf41oI6iE+YTZ2htdtBDynjA=3D=3D?= X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: ffe740bb-cfd3-46d1-0dfd-08da7f527b55 X-MS-Exchange-CrossTenant-AuthSource: PAXPR08MB6969.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2022 06:42:26.0827 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3aTrRGtZyvLWvbdibVpAbwn47Dwke46VbOE0tstIkwdg7eL9tLf42E8mTeqIL89p9hCjPYPLViu+Xexun6/MFg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4685 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean From: Stefan Herbrechtsmeier Simplify the comp_util class by remove duplicate code as preparation for additional compress algorithms. Signed-off-by: Stefan Herbrechtsmeier --- Changes in v3: - Rename COMPRESSIONS to ALGORITHMS - Rework existing comments - Add _get_tool_name function comment - Add _get_tool function comment tools/binman/cbfs_util_test.py | 2 +- tools/binman/comp_util.py | 101 +++++++++++++++++++++++---------- tools/binman/ftest.py | 2 +- 3 files changed, 72 insertions(+), 33 deletions(-) diff --git a/tools/binman/cbfs_util_test.py b/tools/binman/cbfs_util_test.py index f86b295149..44ebd04278 100755 --- a/tools/binman/cbfs_util_test.py +++ b/tools/binman/cbfs_util_test.py @@ -50,7 +50,7 @@ class TestCbfs(unittest.TestCase): cls.cbfstool = bintool.Bintool.create('cbfstool') cls.have_cbfstool = cls.cbfstool.is_present() - cls.have_lz4 = comp_util.HAVE_LZ4 + cls.have_lz4 = comp_util.is_present('lz4') @classmethod def tearDownClass(cls): diff --git a/tools/binman/comp_util.py b/tools/binman/comp_util.py index 269bbf7975..00761d44cc 100644 --- a/tools/binman/comp_util.py +++ b/tools/binman/comp_util.py @@ -1,69 +1,108 @@ # SPDX-License-Identifier: GPL-2.0+ # Copyright 2022 Google LLC +# Copyright (C) 2022 Weidmüller Interface GmbH & Co. KG +# Stefan Herbrechtsmeier # -"""Utilities to compress and decompress data""" +"""Utilities to compress and decompress data + +This supports the following compression algorithm: + none + lz4 + lzma + +Note that for lzma this uses an old version of the algorithm, not that +provided by xz. + +This requires the following tools: + lz4 + lzma_alone + +It also requires an output directory to be previously set up, by calling +PrepareOutputDir(). +""" import tempfile from binman import bintool from patman import tools -LZ4 = bintool.Bintool.create('lz4') -HAVE_LZ4 = LZ4.is_present() +# Supported compression algorithms +ALGORITHMS = ['lz4', 'lzma'] -LZMA_ALONE = bintool.Bintool.create('lzma_alone') -HAVE_LZMA_ALONE = LZMA_ALONE.is_present() +bintools = {} +def _get_tool_name(algo): + """Get the tool name of a compression algorithm -def compress(indata, algo): - """Compress some data using a given algorithm + Args: + algo (str): Algorithm to use - Note that for lzma this uses an old version of the algorithm, not that - provided by xz. + Returns: + str: Tool name + """ + names = {'lzma': 'lzma_alone'} + return names.get(algo, algo) + +def _get_tool(algo): + """Get the bintool object of a compression algorithm - This requires 'lz4' and 'lzma_alone' tools. It also requires an output - directory to be previously set up, by calling PrepareOutputDir(). + The function creates new bintool object on demand per compression algorithm + and save it in a global bintools dictionary. + + Args: + algo (str): Algorithm to use + + Returns: + A bintool object for the compression algorithm + """ + global bintools + name = _get_tool_name(algo) + tool = bintools.get(name) + if not tool: + tool = bintool.Bintool.create(name) + bintools[name] = tool + return tool + +def compress(indata, algo): + """Compress some data using a given algorithm Args: indata (bytes): Input data to compress - algo (str): Algorithm to use ('none', 'lz4' or 'lzma') + algo (str): Algorithm to use Returns: bytes: Compressed data """ if algo == 'none': return indata - if algo == 'lz4': - data = LZ4.compress(indata) - # cbfstool uses a very old version of lzma - elif algo == 'lzma': - data = LZMA_ALONE.compress(indata) - else: + if algo not in ALGORITHMS: raise ValueError("Unknown algorithm '%s'" % algo) + + tool = _get_tool(algo) + data = tool.compress(indata) + return data def decompress(indata, algo): """Decompress some data using a given algorithm - Note that for lzma this uses an old version of the algorithm, not that - provided by xz. - - This requires 'lz4' and 'lzma_alone' tools. It also requires an output - directory to be previously set up, by calling PrepareOutputDir(). - Args: indata (bytes): Input data to decompress - algo (str): Algorithm to use ('none', 'lz4' or 'lzma') + algo (str): Algorithm to use Returns: - (bytes) Compressed data + bytes: Decompressed data """ if algo == 'none': return indata - if algo == 'lz4': - data = LZ4.decompress(indata) - elif algo == 'lzma': - data = LZMA_ALONE.decompress(indata) - else: + if algo not in ALGORITHMS: raise ValueError("Unknown algorithm '%s'" % algo) + + tool = _get_tool(algo) + data = tool.decompress(indata) + return data + +def is_present(algo): + tool = _get_tool(algo) + return tool.is_present() diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 057b4e28b7..96c15cff77 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -212,7 +212,7 @@ class TestFunctional(unittest.TestCase): TestFunctional._MakeInputFile('tee.elf', tools.read_file(cls.ElfTestFile('elf_sections'))) - cls.have_lz4 = comp_util.HAVE_LZ4 + cls.have_lz4 = comp_util.is_present('lz4') @classmethod def tearDownClass(cls): From patchwork Tue Aug 16 06:41:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 1666688 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-weidmueller-onmicrosoft-com header.b=dik0a/N2; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M6M712d1lz1yft for ; Tue, 16 Aug 2022 16:43:41 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 237FA848E5; Tue, 16 Aug 2022 08:42:50 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.b="dik0a/N2"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id ABD8D848AC; Tue, 16 Aug 2022 08:42:34 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50054.outbound.protection.outlook.com [40.107.5.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id AB3B5848B0 for ; Tue, 16 Aug 2022 08:42:27 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=stefan.herbrechtsmeier-oss@weidmueller.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PdDFt3XEls9ke5gcmgG9CmSPIVe8B+P66DE41qV7jKSTcsT5xBeWpM5uEVkQrTg95AMac6H1P9kuqNpI+bBONj4F5i8DZwg1xLo/Bqz0uMbHFT3Axx8Sdu3iOezXrlKOIVsUVd3xGp+927+d8sM7rw/fuRbThdDmAVry5ejLOUK93K8/ZLXtNr5aoLN88IC3m4jPxl7tZrIYFGoNmVLS59eUawI1SQYUZ/Fx5GRNTSB8udEPYIk/evJEjx4Ij7PPRNCJnCeQJPsFNEKGewdWvfssbSJHTKOqlJNxdBGKVDzcfdvv/p+otm2EaP24ldXjj8PPfnImzIFh8rMO9ah8Hw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=agyVQJIVBeMMoWUGWS8luN99Qn14dtfTNjeNYOYUNhU=; b=JmxIgJk6LZKCcIxoq9iVbS9Tc9i8FOKaTR8SjV4wevpbTd1NT5U8gU4dxf1P5x1NSNTffqLAQMGAR95Z0LYQzKCAXloXzzJattKvIQllCVaZ675xLBlUPJny4vSv4qGQ+KaDXRP8lKiXnpGAPVEOiNon2+AnliGaVhIcQWTfyICZWpRAVmCg6ZTG/gSVhu/dGAaRIqF2lZMBb4LQt/tj2sNPUGsR4WQZNNyPw6enXC09GvNsaRUiIcpcHShBziCmp/TwipZV8obTIQxnMGDATh/LSpyq444iXi+/hXmCdKCiZg5YPMqZv5I2xrUpUZtwRAiwKMu8u74WjGQPlz+6Nw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.onmicrosoft.com; s=selector1-weidmueller-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=agyVQJIVBeMMoWUGWS8luN99Qn14dtfTNjeNYOYUNhU=; b=dik0a/N2ayNFSz8BvHE0qnWLgDxldw3wlZWiCj78IX4YaD9qXMDycvOJpXL6FISzWuyRVyDX9Ggc8hP3l5O4Rcf1ZNNWNMB1APo4Lns/yU0I5IUqy544NQIFfzkKITr5MHXvDiaGQJkRygzH7nKYalZZy976zYJJqxp4zVyGEKE= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) by VE1PR08MB4685.eurprd08.prod.outlook.com (2603:10a6:802:ae::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.25; Tue, 16 Aug 2022 06:42:26 +0000 Received: from PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44]) by PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44%3]) with mapi id 15.20.5525.011; Tue, 16 Aug 2022 06:42:26 +0000 From: Stefan Herbrechtsmeier To: u-boot@lists.denx.de Cc: Stefan Herbrechtsmeier , Simon Glass , Alper Nebi Yasak Subject: [PATCH v3 06/13] binman: Add compression tests Date: Tue, 16 Aug 2022 08:41:44 +0200 Message-Id: <20220816064151.9848-6-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: AS9PR06CA0744.eurprd06.prod.outlook.com (2603:10a6:20b:487::19) To PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2880719e-48d5-4364-62f2-08da7f527b9a X-MS-TrafficTypeDiagnostic: VE1PR08MB4685:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CSXIPdlnA04E8sXDZqN1yxtk878OhgWKet8lV4sYGATOxyazbhEqiHXlwFmbJKDb1n0NrBLiRAbS/DffdB58RWiYaYjizBUlTgd1OQN3MqQ5uvcYABvQqxyuGAmO0OtOv0wd7z2vPEOh0yIUcvv8rCxvLy98dzMwZwJ2o7KmF6nADpjbkqE40fx7SvRFpJtcr6HzmSKXCeWXSpCGej9QtVM226JBJI7RidR5qYO/UDKMUiXgonYfhuNuugzfmBx0ntaCeJalekNCWk0BDDGG9wTWRvLUDTiZki9PRfVkgZoZJ5+GDpdYiIMAFsuXRMpQNz8ZWlyHjuzckteNPKn0Hi8LChlAvQJ2w+UDlPS7B/9+9dDdOhKmP7h73Li+Cyed47H4K0UmtHGSj05sz2LAtGfKNPZuKQ91irdjDsQpPlvmU+SHLX3u7K3+nu+kdEW3ljTnSV5E+k+/6Fgz58kWsRePoSxkbyjMbcel+FupT+13tRyEcnUIdE0z/KLYdIE64i0nGZPHFVW80vVWwmC+0f3fpGn8AwOvSmWQDn0SXWpofeLdWbSynQsfdZWFs8Zs1gbQ9GecpYOKdUwecVUI3xo/3I3RvmC2S8YPqiD/NUZ8MLSmA+8G+E2xozLbSTYJIk9ea7+4d+UzHnoKAQjPjQNK1YDcts5bmp3yzOob8m/gFbhM/p9btkIi6mFQ+12BF0f1brPeHBuZGuF/JLLeuif+YlBeA5UmQKD958hIrx6xifD5doJEDrZNDIg6x8NqGnZq+emRsxvICWGntdV0oBVqz2HrWs8jAi02DgMV4wVUP4p8ekJ0FD8V+LSH1L1bT8N6PTQBiG8SSzQLUArnfA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR08MB6969.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(136003)(366004)(396003)(346002)(376002)(39860400002)(86362001)(38350700002)(38100700002)(36756003)(316002)(8936002)(41300700001)(4326008)(66946007)(5660300002)(1076003)(66476007)(966005)(478600001)(8676002)(66556008)(6486002)(6512007)(26005)(52116002)(6506007)(2906002)(6666004)(186003)(2616005)(6916009)(54906003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: NyBxq9JvFbye5cQv1fSSKJW6SwlydXMJo5aoQZwiAvQM6wuD1+R/WV2pBF0BcSVxPQNDSY2J44FgDNk71iV07+XWCILVPr3hfIFGWSpqiAo0QLRlOdmX5zXt3DlpzfDRf6jmbm1lOpBGj/5prXzAZIUh1uRdaIL6WRrWX9bWWjkMOxmvyJIzILphCbEjeJ0ZODon+I8loXRWAnlFaYGFXY5r0gith5D4NAuS0blu4BWkmoaefsyqUEpQQFYNRLIQN0Tm3er43uDHGGyRpvGyyHYhnnL2IRLQ90fDgNqWDfwWOU1tL5E7KrM+wJG6/OrHH/5hq7wCgQwkAyX9JZNpQBTtDR+cFn0FmW6dTZ1+Sn2L6dt83b93GbXNisKjrWPNg3wSRsOscyiBhHdATwOAuQme6Wqq/jdgRwvyGQhAunWuVdAf6HLizygf5Z/IUUg7G1I6JYlnZ1Bam8wUpspot/e26xJplu4E+S3zsIgA2GaOTrkYVmqrHmSeRD5p/b52N/TKboj7LHgVKrgI0T+uUfG/CjFrs02RRo7ocIqDWOuB+J0qorxBzup1ESJ1M/dRjdgAnQtqXLsTAA4wjtrXUpLq1QZ3usuKrGd8ypHz7O7ei9WBEWhFkCYPLhUYz5QI8FthLpaIdv2CLiAY2gEGLut5FGVk+pUt+yDp0jTtRxOX34OmVbRws6x0SX/B2otzX5HEQ6xcMyJ56qdHj0siX2dApKpMpcCDWsH07k4tHw6BOpln31rV3BrSHkeawnnb+asKJnN0CDgiVrgTEp9rL6eUFxsedrazGNjs7hqNlr/0rzrylNkcx7pTORQqlv6JDXeqgCwcO7C4wrnbxiY1Hu5MQyYmEfAcpG0qPJ1Bac+9fSRrIDQ6UWShtYg+emvu0A0NFWgxm9AIh6qAP9y7aA54bvKmTC71NKGNfdPa9iAO5DaBFO5UlDOhtQuWwTKsrXpwUTIvM1lGJKH6E+z2qQ/DNMznzUzCARX6Z0lLuNrmWTc5zNjEo1WgtVruN5UkY77gatEmk4j4N15eSrnvOQh2onokKlTXcymkAXZpm/yL3Szn1egPiIxsQCTkpgWjH1RR5xNcL4DhfBQf5PzJc39X6tCUPV1McBLpjqnMXm7WqyULIy0G+/ie7xu/giR2UnEb8Mbf7XiHNp/NTI46l0HN208GcrZFEOHB4lJ+hs7cXrVnL/4EmBSZteTb8iIeYyzeAlyT2jM7RaNuyTBu7ewIz6ZggIP1jSH243rCXVo2i6hYyMsvTYOCIBV24SmODnkNLSmWGMPueQ6iZ7ghCfGVIObhBI0LhVIjJR3pOjPchhXgCirW+4jgib0HYOGYiYo+w63USvfwqSuuxKU2KAugS7dsQ52LdHNstNN7nyWcvUR7SOo3dyTXSP4vSemFkJdrBKsyoPtt983So3n5mcpHv9flgQUwsullnqgPyfLtOVWpTjmj9tRwLRzw35mVez1Pb5gOs89GCcbpHUM5U8k948DMJ2cMhms56rm69IXJh4CEE3oKsEudEIe7Z8d5r1cynyTU4Qw/XVpOkBx7lCsfOXnQgVl4aF/gc6sNLsUwkhQqwXAWNjJuREb7uSP0g2p0pASfPjHHG3OTyYqa3A== X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2880719e-48d5-4364-62f2-08da7f527b9a X-MS-Exchange-CrossTenant-AuthSource: PAXPR08MB6969.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2022 06:42:26.5201 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: J5vpi8dck8wiQN5nePvg3U9RJB5zH4Eyny5pOQYuRF8sgWQ4GfqG+yqI6TIAUGYcIPmPKExtRV+H7o/lKEIigQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4685 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean From: Stefan Herbrechtsmeier Add common test functions to test all supported compressions. Signed-off-by: Stefan Herbrechtsmeier Reviewed-by: Simon Glass --- Instead of the for loop it is possible to use Parameterized [1] testing. [1] https://github.com/wolever/parameterized Changes in v3: - Use 'tools.get_bytes(0, 64)' instead of 'bytes([0]) * 64' - Check if tool is present - Rename tests Changes in v2: - Added tools/binman/ftest.py | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 96c15cff77..bc17107735 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -5248,6 +5248,37 @@ fdt fdtmap Extract the devicetree blob from the fdtmap comp_util.decompress(b'1234', 'invalid') self.assertIn("Unknown algorithm 'invalid'", str(e.exception)) + def _checkCompUtil(self, algo): + if not comp_util.is_present(algo): + self.skipTest('%s not available' % comp_util._get_tool_name(algo)) + + def testCompUtilCompressions(self): + """Test compression algorithms""" + for algo in comp_util.ALGORITHMS: + self._checkCompUtil(algo) + data = comp_util.compress(COMPRESS_DATA, algo) + self.assertNotEqual(COMPRESS_DATA, data) + orig = comp_util.decompress(data, algo) + self.assertEquals(COMPRESS_DATA, orig) + + def testCompUtilVersions(self): + """Test tool version of compression algorithms""" + for algo in comp_util.ALGORITHMS: + self._checkCompUtil(algo) + tool = comp_util._get_tool(algo) + version = tool.version() + print('%s - %s' % (algo, version)) + self.assertRegex(version, '^v?[0-9]+[0-9.]*') + + def testCompUtilPadding(self): + """Test padding of compression algorithms""" + for algo in comp_util.ALGORITHMS: + self._checkCompUtil(algo) + data = comp_util.compress(COMPRESS_DATA, algo) + data = data + tools.get_bytes(0, 64) + orig = comp_util.decompress(data, algo) + self.assertEquals(COMPRESS_DATA, orig) + def testBintoolDocs(self): """Test for creation of bintool documentation""" with test_util.capture_sys_output() as (stdout, stderr): From patchwork Tue Aug 16 06:41:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 1666689 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-weidmueller-onmicrosoft-com header.b=e6xlNBZx; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M6M7D2zgqz1yft for ; Tue, 16 Aug 2022 16:43:52 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 5195183FCD; Tue, 16 Aug 2022 08:42:54 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.b="e6xlNBZx"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 596E2848A5; Tue, 16 Aug 2022 08:42:34 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50054.outbound.protection.outlook.com [40.107.5.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id CDE3A848B7 for ; Tue, 16 Aug 2022 08:42:27 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=stefan.herbrechtsmeier-oss@weidmueller.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=C+AfnSaSlWouyNAtc5CDuXq/d2uSXziN+M3Rzv8hJJpxYV2t5YhBkLvrOf/nQTvD1LrbCBSMdcWFYBVv5mJuQ7PnKMgj++bMIQEhHJF8zL5Qcm5VzQpSz4BV0lt7Hg0nYPHNflRYx/pYRa9Ct/nLXUc1bsBM66Of1I0Bnp08+p8qHBSH0oAenffxUl/r2wEE39u6wMOxsb+OgwuBNb65QsYZGqSsKBBm4mPWV/c9qD/sdyaApE9TtxSuI3p0c5b/IjYNBxjm3MqCkPTqr4XcjdGMLyD/JzqlWQaSftBA99FYaopDfc5inOc7pxEjHGT/e35lSdOUlmqmZHCH6vwGNA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8/RREc3jXBEUmWurnYo53hXoNMt7KDnMS+ritmwA2FI=; b=FdTxOkYH90BdrUGj+n65ZqwP681SoK1rJagZ+TnD7Km+T1Iq6LXZb7aZZzVkazxSFB9XmgpOeg/n9ZGwjCfkO3euG7Va0yPspMdJHnw6htHU8wxCzor8d70rbROJlbTFvteuNIVkRizBzIoCGpm6MbgDE9ArPUdT3WZE7r0IswP1tFXv70u50LGxPI5ict4xv4/n5uMbQm14qAJau+NZOUiKXc+y7ssg6GSmbtb0yza5n6PGrb6XlKNluzh1Kr8BACVq4yxYPjUjs3Lsxm0ObiA5runC44mRHNgQdUiKqBfCUBa38ZbWlJiLv/FTh0vBbZKLySb367vc7gzjjmHYFA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.onmicrosoft.com; s=selector1-weidmueller-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8/RREc3jXBEUmWurnYo53hXoNMt7KDnMS+ritmwA2FI=; b=e6xlNBZxCOWGOliGdKAGgmBspJlPugLcU6k8noXKAbc4nUQyeuFNwQAty9ZhcyHBNx7Wc+cjSv16U93Txxx7AjP8KQyvP/Cz3QHJxBHosluwOHTbNsUU2nWNfvx1kIYeCOgad9aDUBVd2jy5Blphcagd62lksl+2jvqyVaBdUx8= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) by VE1PR08MB4685.eurprd08.prod.outlook.com (2603:10a6:802:ae::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.25; Tue, 16 Aug 2022 06:42:27 +0000 Received: from PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44]) by PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44%3]) with mapi id 15.20.5525.011; Tue, 16 Aug 2022 06:42:27 +0000 From: Stefan Herbrechtsmeier To: u-boot@lists.denx.de Cc: Stefan Herbrechtsmeier , Alper Nebi Yasak , Simon Glass Subject: [PATCH v3 07/13] binman: Add BintoolPacker class to bintool Date: Tue, 16 Aug 2022 08:41:45 +0200 Message-Id: <20220816064151.9848-7-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: AS9PR06CA0744.eurprd06.prod.outlook.com (2603:10a6:20b:487::19) To PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f212cf58-b863-4b8b-bf82-08da7f527bda X-MS-TrafficTypeDiagnostic: VE1PR08MB4685:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Njh8ji50uh8MnMh1ksLXBX2LWgFPok3g7LzxEapl+PDWEu+p1EYIx9zswMXVHKfqQm/gYduKAiEGwvs9ANUWeMCXJ7clM/1jOLE4GhCGtd2I8FUw8RRi2eew1CzW7RaahM3H6GdfNGDrTAoKLNiHuPoTLRUUOAL0gJ5b2yq9EaxETCAJMjn0hGve6anuh4rQuU52FH0J5WIsU3Dj7ghEN74Tnfq4cBCa2H4v1aaGUwFN5gnms8Htcgo0n0UNSx5Q10oOTANCSubnP5BFDg5qAE6amKQoRwSTJHN9iZfVHpoz5Bw9gvQAxft+E8VpcaM6P39N+SlIBBswefRiMo/gDqwmsqOnYpl/q3nUu6mq8XL3YsBSgwTC/wJqM0wozS1ZRDTrXoFimCs6AqR6233naj4tMfFNIiAM6pLydC1/NIPkGmGMKIJ1cRz/MZaX9Ck72vHpLHUUg9nWN1jvwIXrLO0UA10SRSPuoIb/7ohDkn8LuY7hxMQy52GpD5dgkVmlXftSAPUDGHeOQu0iUZtsdcZNTT5RwH5IoQrnTg+DaFz7x8WtcrmSYgvOQ/o7SGuOD/Ylp6D6XO/v+rGTP2uDLKIsWvWfdCQqy185872iUd12HYsDpgh7Xxgg5roNraAjMqck3KmSAkRtDL5oYDK0Z+G6z25YwXOrEHLxKigyfcpeStts+Psd6Uupbdne1RCpcq+OOKICcNvDd6xY0lfmWIT645hQcqUEIr7D+qoicjjBJbDQ7ee2qkRxT3KgRZNifsKLcrcvDEguX72wz03CKHnJ2mxeUZvHgFkgLZ9h7YQUsFaDT71TlctNigYUmZNr X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR08MB6969.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(136003)(366004)(396003)(346002)(376002)(39860400002)(86362001)(38350700002)(38100700002)(36756003)(316002)(8936002)(41300700001)(4326008)(66946007)(5660300002)(1076003)(66476007)(478600001)(8676002)(66556008)(6486002)(6512007)(26005)(52116002)(83380400001)(6506007)(2906002)(66574015)(6666004)(186003)(2616005)(6916009)(54906003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?uQsic90jKTf9eSQ2ceqhhGbEDNO1?= =?utf-8?q?5v/bduE4DH8DMpmip6bpqS5P75r/lBRe5dhR3bVyj6MVX6LzdEgX68pI8uuvKOxfM?= =?utf-8?q?L+eubTRw/8oWrw6PXrDNaJmMpaL9R5WRH+nxBpj8Llchah9RDSxxExo370Q8nJ8nw?= =?utf-8?q?8ESM6F7OFsL/V0jZJPU2DVc8N82ZptCq4b2Aekrw6Aol7/peyw0uVZWBlMUptHouS?= =?utf-8?q?mucbQ2hfblDWM/l1zOmvuZk5AynrpX0jju2QXRXcg7b8i9H1MmVPuzd1w/amTMUVU?= =?utf-8?q?+cqBjqSevzNM9zYLUvNLYs2DIbZU486G220DfZOL1Ce1pYy1fvROjzkQrSPuC1WrP?= =?utf-8?q?TnjdUh8zfVcJUDa20FIuoi8bx4R79K61QJ8ToxhrWTdscO9FtQV4RMS2GqNTq8bn3?= =?utf-8?q?W9DyBvJQfivgUNBkaYqCBj1zEgxf4l6zmOWvwlxLC4hm3tid5m+LlOahKnJ4Cv8jG?= =?utf-8?q?2gpQmuun/dR2NJkE56IP4XUjIE1SGP4Et1FyQLHkgEw5hjq3gWyLPt1xB/yyFFQHJ?= =?utf-8?q?4JvjGA2cP1lnEImk+Dmm4fKYl1djWz0i9i6Wnccp5AwtucZvb2KAwXd9b8g29geWB?= =?utf-8?q?YqYDxdvcstdNmeCeFQaYUOMhjGs5siog1AI0dCcChSI0mDNwHU/5nFu/uG3zvhmIH?= =?utf-8?q?xIBNFWGiB7ix1J1nDeeWO7RCllOmcsdDL5ssAfaNHb5o/q3althYV25x9mZEM5WSX?= =?utf-8?q?Lat2OtvVycGrxSgz/NfsRCrB7+5/0Wc5uzNnpCVGOszwITkJDSQHlqeQerT9aEFqC?= =?utf-8?q?qulFlAj5MvImAt/qTawIYIJt1h9XfchKo9ljGj6aZkPJ6Ld0RbUyoUxKVgmCNH5g7?= =?utf-8?q?Nj+klkuoQ+3Nqt2au6Peh9yvE3S5g+iMKGbl5y/XBMl6IGEls8s0diLFsG146BUKJ?= =?utf-8?q?B9ygy9lLv5tVtENtL4rwvY3UqKLxag1OEIOCGeclg6wxkMq+mEw2KHFBdUs6qmo9b?= =?utf-8?q?WXc+JmgPR7oxKBhGWo3gaEqZovfuzlwqPx5A5AWEVZZx3Bnp6zqQtKa2aXXikZrMF?= =?utf-8?q?v7D+DZ2N2vbYp839BTi/U0aEDabarJ94dAeEw4Vmj5Wid3B3d0Gkbq0pLn7nyEmSn?= =?utf-8?q?iJeY4Xzo+/miYmww6OSJEopssJc/FIoFSgIA3G0wXcFY0VeKW8ihX7HWEm1n20mJ7?= =?utf-8?q?KkMGBPc/+CDwEGVRRUmLpFA9C+NHqfyXW0oDxoS1dTYdPy/PZXjLrCoEutKtpjzwB?= =?utf-8?q?cxINORKnF8/FVXQhezyoJzQ+YuOL5W+SuHrnP3qS+BNe76BICEDV9GZ5whVENdBWs?= =?utf-8?q?ZXpe9HhLaIDqWx61i12N8chTMZS/rKtnDEJao5UJGNWXr/HkQcuT9AYqsC6prUHG/?= =?utf-8?q?u7I2RINNj64GbSTyqGlQhTyImEoidtWM6F8pw/A40MJWsgGaR5I1AMFhOy0MWqAsQ?= =?utf-8?q?ANWboYfRe9qI+73G082LT6n+NuMGeD63OnvyVQVzxNh+Nodoh6a2S8aw5lONw/lhI?= =?utf-8?q?JtXJovsVDViP258m+HgJ/TN4xT3k1x7dzyKPSmtfZwaf6bekHxN3vybzIOj66f6tC?= =?utf-8?q?WI/NJwaj9SBohNj3aal8grFg2+C8g25zKQ=3D=3D?= X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: f212cf58-b863-4b8b-bf82-08da7f527bda X-MS-Exchange-CrossTenant-AuthSource: PAXPR08MB6969.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2022 06:42:26.9731 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: CtOUXvHbcBE+iI0d58q4hZoo+vBJnbgJub1wwNvGZAVolZrfFg8lfMELkEacKzLc409kut4SRS+Zq5mN41QQmQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4685 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean From: Stefan Herbrechtsmeier Add a bintools base class for packers which compression / decompression entry contents. Signed-off-by: Stefan Herbrechtsmeier --- Changes in v3: - Document class properties Changes in v2: - Added tools/binman/bintool.py | 107 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 107 insertions(+) diff --git a/tools/binman/bintool.py b/tools/binman/bintool.py index 8435b29749..2eaad418f2 100644 --- a/tools/binman/bintool.py +++ b/tools/binman/bintool.py @@ -1,5 +1,7 @@ # SPDX-License-Identifier: GPL-2.0+ # Copyright 2022 Google LLC +# Copyright (C) 2022 Weidmüller Interface GmbH & Co. KG +# Stefan Herbrechtsmeier # """Base class for all bintools @@ -464,3 +466,108 @@ binaries. It is fairly easy to create new bintools. Just add a new file to the str: Version string for this bintool """ return 'unknown' + +class BintoolPacker(Bintool): + """Tool which compression / decompression entry contents + + This is a bintools base class for compression / decompression packer + + Properties: + name: Name of packer tool + compression: Compression type (COMPRESS_...), value of 'name' property + if none + compress_args: List of positional args provided to tool for compress, + ['--compress'] if none + decompress_args: List of positional args provided to tool for + decompress, ['--decompress'] if none + fetch_package: Name of the tool installed using the apt, value of 'name' + property if none + version_regex: Regular expressions to extract the version from tool + version output, '(v[0-9.]+)' if none + """ + def __init__(self, name, compression=None, compress_args=None, + decompress_args=None, fetch_package=None, + version_regex=r'(v[0-9.]+)'): + desc = '%s compression' % (compression if compression else name) + super().__init__(name, desc) + if compress_args is None: + compress_args = ['--compress'] + self.compress_args = compress_args + if decompress_args is None: + decompress_args = ['--decompress'] + self.decompress_args = decompress_args + if fetch_package is None: + fetch_package = name + self.fetch_package = fetch_package + self.version_regex = version_regex + + def compress(self, indata): + """Compress data + + Args: + indata (bytes): Data to compress + + Returns: + bytes: Compressed data + """ + with tempfile.NamedTemporaryFile(prefix='comp.tmp', + dir=tools.get_output_dir()) as tmp: + tools.write_file(tmp.name, indata) + args = self.compress_args + ['--stdout', tmp.name] + return self.run_cmd(*args, binary=True) + + def decompress(self, indata): + """Decompress data + + Args: + indata (bytes): Data to decompress + + Returns: + bytes: Decompressed data + """ + with tempfile.NamedTemporaryFile(prefix='decomp.tmp', + dir=tools.get_output_dir()) as inf: + tools.write_file(inf.name, indata) + args = self.decompress_args + ['--stdout', inf.name] + return self.run_cmd(*args, binary=True) + + def fetch(self, method): + """Fetch handler + + This installs the gzip package using the apt utility. + + Args: + method (FETCH_...): Method to use + + Returns: + True if the file was fetched and now installed, None if a method + other than FETCH_BIN was requested + + Raises: + Valuerror: Fetching could not be completed + """ + if method != FETCH_BIN: + return None + pkg = self.fetch_package if self.fetch_package else self.name + return self.apt_install(pkg) + + def version(self): + """Version handler + + Returns: + str: Version number + """ + import re + + result = self.run_cmd_result('-V') + if not result: + return super().version() + + out = result.stdout.strip() + if not out: + out = result.stderr.strip() + if not out: + return super().version() + + m_version = re.search(self.version_regex, out) + return m_version.group(1) if m_version else out From patchwork Tue Aug 16 06:41:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 1666692 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-weidmueller-onmicrosoft-com header.b=LKubp393; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M6M7g43Nxz1yft for ; Tue, 16 Aug 2022 16:44:15 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 71A1B849BA; Tue, 16 Aug 2022 08:42:59 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.b="LKubp393"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id ED192848B8; Tue, 16 Aug 2022 08:42:36 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60046.outbound.protection.outlook.com [40.107.6.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id A157A848BE for ; Tue, 16 Aug 2022 08:42:29 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=stefan.herbrechtsmeier-oss@weidmueller.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ds3mHEbm1ATUE69b0cT3yC/vJgbYdCad+EWhAaEb9b2o2H0awXkuOkXJcpG1WtNvVvtD2MJk3gwF0yjpYslSsyoU1ELGYSQbKzfOe4m5A6l66cp6ISwhqQ+sfipq73xUfmhgXjdMH3j8vAANWYuURXBGO781dKEXJwS9dRTF2amKQ5405hthzbpS70IPE2bAr8LsNV/qgdP4rFMS/iHpvhkgxW1jz57Aqcwjelsv03KqvCfumjbum9EJSj+AHIw+zN2Ba00k7No7PAKfEdPpwEgeM26Mm+gwfsAWAOQSqSUZEWWrLASnIfDaH/eXN5VYHkpk4epuapTkQM2bdbSsTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YXGHiqj8fFIYd25fpTuI0nVxxCNTocW+MnE6BoTF5no=; b=lDVwuRv5Iwj3NBTYIrRqjCA4LX0yi4sz48fAk5Z7j6Txvq+uBNuTRcjxN8+id5fXjPU3hWY1H2gAGeoJeudQwfS9pQEl60kCZ8PgNbtxY2w+iqs4CORaVBtFf3hMBkotSnAWfLzZVjEsPDpUJHoNS1TzXebIozOeduRTia+gnj5BXonO9XJn68xRRHQNkmasUdRWsSbDqEjJgtmhFS1vZz3qp9lBnhj2PkyWqwwKbu8g05tj94ElZysyu+8d2r3+xY0rfJbZG68bQoz1kLZo3vO+pWp+cWl+ZTWZF8yNon8FtbQulsN8dVlAq87H7h6b9yF+S5BfEB1ExXw32vpCvA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.onmicrosoft.com; s=selector1-weidmueller-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YXGHiqj8fFIYd25fpTuI0nVxxCNTocW+MnE6BoTF5no=; b=LKubp393osrLLfebaUKaOvzmWqiThNBOYEkhy36ZNWf3HQGoPW3+9NDUoOUr18z8nR/+Ev3c2+HCWfNzXjk+mf0hbFlJJZOXCvZuM95o/ETLZR10jQVEV/0GsG/QYmWXV/kxhFVX6K7KaZrnh37TcJnSeyb8aojWHE709I2VHmA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) by DU0PR08MB8662.eurprd08.prod.outlook.com (2603:10a6:10:402::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5525.11; Tue, 16 Aug 2022 06:42:27 +0000 Received: from PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44]) by PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44%3]) with mapi id 15.20.5525.011; Tue, 16 Aug 2022 06:42:27 +0000 From: Stefan Herbrechtsmeier To: u-boot@lists.denx.de Cc: Stefan Herbrechtsmeier , Alper Nebi Yasak , Simon Glass Subject: [PATCH v3 08/13] binman: Add bzip2 bintool Date: Tue, 16 Aug 2022 08:41:46 +0200 Message-Id: <20220816064151.9848-8-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: AS9PR06CA0744.eurprd06.prod.outlook.com (2603:10a6:20b:487::19) To PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 49f3ce91-768a-46e5-e08d-08da7f527c1f X-MS-TrafficTypeDiagnostic: DU0PR08MB8662:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +euKl3yDNyBofjdvC+CDzslD7x++2ibXZiH7YbRhVADCPnBEJYm8wo+2ELP2rZ3pMh8gIi6eEXfrIkt9RIxgdbIZO6XLopREgLwxSwaYWEcyJLxRGQLHVAVkhQfByoDOxcRnrGHo7wK3T2Rf6Qch8dAYkt5GKK4ArVcThmk7aSONQJdZapLwKc0kkx5YuhyytgzQJ8+KVDAcMgJvqZmE+VhC8Q8JnMxdlF2kLWprk6OhYnwarSmzR97E7dkFPvX2xwCLPgzDWx5yrj894VEO6Jnc0dCjKc3Uo7KD06RUvUmX3MzZVZkf42macB2u/mAfIWK6leEZ5x4OvIsnGQu6iZ2PZfe97fhrLLQZUyr3czpRYqBu9LMEWg99VupNQIeNA6pP2p0ZLDc5BTOTWNx5YmOT2sX0genPytDWwh0spfkl/1AQP67BdEKKCwSivVyIQ56Hc/jjbjrBjctkFatHnshELPiN/eWunWtgarFyFaGOVUZY2un6I4jq6+jdHLzmF+6wz0VNbPbsYut/De6cTqrE1Jwa/jAuLz+y1hUuj2kANeC5UUgiQDDugRzj8msdNV3ETU+kabU1jyjxlvHSUgDwl6e0VhZWqdUDcucLL9+38hcK1YJ1AiOZTMNpK5D4QwD8IBJ6Rt6RnHNDcbVfRdRiaK3gCp9Vf/LldqhJ2DaIYp0jrdJOJehgZEdVTyaaCHH3eK7IfQcr2Cu4Oq7iFihHPLvDPmedo5IQx75Fuo0lpVxZJF2iQw+Jgqs1InXsXY6xStazEmxMuc77nCJ3OvjAoC4AXD+UJ9SIbo6qfngAVH7Hi1dgJ2kk5c7mMj5L X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR08MB6969.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(346002)(376002)(39860400002)(136003)(396003)(366004)(316002)(6666004)(41300700001)(52116002)(6506007)(6916009)(54906003)(86362001)(478600001)(6486002)(38100700002)(38350700002)(83380400001)(2616005)(186003)(1076003)(26005)(6512007)(66574015)(66476007)(66556008)(4326008)(8676002)(66946007)(36756003)(2906002)(8936002)(5660300002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?LeuS7BTEl0X7ONsi31ZI/mgqJ698?= =?utf-8?q?Br6kPB9fVJ+/0DQBB5Qrx0BDu/A6lzdSZ/N2j4KCUk/aOVQ8HShyc2KHv9mTzvrws?= =?utf-8?q?U7hYGp96nGp7Gpvec+DTfcRi8axfmv+jKXMkIx/dCsXmQwckmWEqoyORul6OTC4Vx?= =?utf-8?q?XEBnlkiO4/vF3XiNcj0yyYqp/hoU0s44ZQkRTX4feCaefJI6thyWCfZLCxf78TdZ8?= =?utf-8?q?xH2DpvEzWJfrf/5pZLTYGy18GOB66hifAgHMMRwSTEzE7cf/bCM9Pe5uy2UPma7k4?= =?utf-8?q?HfPCNDfogwf5l3UkYvQSmWHiPyUPJLbe2G4I7vRgJ8295m20482TtzZ0ROI61UkBN?= =?utf-8?q?9/hqASg4LrBgeDbrsS9oPUShW7bCxPju5LUruD6G/BEey8HairtFO3FXtdx6oZBid?= =?utf-8?q?0oUtOXKFypCuPpmrodRCMUp3PhRdaEs2j3lyD6+v9QGoQBxC3jL/PK/g10UfQUMpp?= =?utf-8?q?nloHtqJ00NFrVeWXZXPOFL2tlIDpM0jscOKxnRN8rf4nC6y9bovi0+RL/lGxjzMLX?= =?utf-8?q?HT6hQfVDULogl8+FDzx6PeS5WHZxLBcm2C6xDzvEmDBbSEwq3IJNQJOR+pL2gHweB?= =?utf-8?q?tGwdgYDNRcDnxsU2b9I+uDuOLDCqKxsftaLuueKwSlMnyfq+7EV3m0S6ZFxDoIu9p?= =?utf-8?q?ZehPMVnrYiS5d3SQDlKqToI8wIiTJyTfJLW/uvp/cFlYvEMrmX4bv5jb1wgxaAw9e?= =?utf-8?q?8Z2Z5u88qseiWSFi6eJukri9cwLifxZOMx7btkFNOs4fS9bdDCK+Iijxxf/dP0D3l?= =?utf-8?q?lEqp7ui0K+ZcDNI+az3MtaW9tC6UsPXQtZz4RcqlEICN+46lqS5E8hNRDMSutyZC1?= =?utf-8?q?7wHjzZ0U7YW63Q//nVrhOfb7fN272IpOYTPQyWBCjdwidqvWNdChOsg4Bj2F66iFJ?= =?utf-8?q?5O0TzX8XkPfJ5KxFJDHZZwu7JZ0iqpFS92cWeBiIiGjBklXdBTKOIHDrx9xqabQQT?= =?utf-8?q?BNiGOTIAt1PNWcYpuwt1umgogTpvJUYVFJK58OmTPY8ydrP4bQVQHoanPX+GFsAg6?= =?utf-8?q?CnzqKkmEtnw/wp78CAhQyW89CG9/rVFEjrfVtF8zfNcTsh9Z0rrqyyi9KamyR4TnT?= =?utf-8?q?8ffXhH45QRbw7oQpLq0B3KLLT+xdsfaXCUAQcql0VpGZgxo9cBCDcSB4VzU1NbZG1?= =?utf-8?q?vLuZvAgLQgxIJ54voG8Q7kHmpUfYEpCIQHDo65exZsoK/vxpEg0uQcoVQAAQ3S3nh?= =?utf-8?q?xmnS68QmPTX+svZ+ZDkJokc1fOQUJ4LPe0FdXKUrj1vQ+Z2DnIxBqKlu1ZEsOuyIs?= =?utf-8?q?IW+jpYloCZMS9RcO+qTcWjba3uSwf2rCJkEM/8EnlPTtOusifqcHn+TIB77MnYFzm?= =?utf-8?q?IVz4/gjFRgnA9pEnpd50LFKVUpmlf3euCaelZn5ltUlIUu54rt53SMOuNbnpdpPjw?= =?utf-8?q?IV2oXMykfCMNwxQxtmIaQssO1bcgZ45ffHOepuE6GReZxpitu/KP4Dmap0P5u8kLl?= =?utf-8?q?V02dTKlSQdpsf9IOwpynVWg+YMA9qybnjDa7TWoK0gjMka2I1wORLm5vfYlSfV0gd?= =?utf-8?q?RT93nufBSX3qxfh6QCNQRZEWwNTq8qZt5g=3D=3D?= X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: 49f3ce91-768a-46e5-e08d-08da7f527c1f X-MS-Exchange-CrossTenant-AuthSource: PAXPR08MB6969.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2022 06:42:27.4119 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: I/d2KN6cN8VVv3gELfDyJAoRdDk6WFbeFpE4jdjclv3KkMIWb6MuVy7XxRcmsDNQw6RJecnTBD95Q7TXxtkNJQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB8662 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean From: Stefan Herbrechtsmeier Add bzip2 bintool to binman to support on-the-fly compression. Signed-off-by: Stefan Herbrechtsmeier --- Changes in v3: - Rebase Changes in v2: - Added tools/binman/btool/bzip2.py | 30 ++++++++++++++++++++++++++++++ tools/binman/comp_util.py | 4 +++- 2 files changed, 33 insertions(+), 1 deletion(-) create mode 100644 tools/binman/btool/bzip2.py diff --git a/tools/binman/btool/bzip2.py b/tools/binman/btool/bzip2.py new file mode 100644 index 0000000000..9be87a621f --- /dev/null +++ b/tools/binman/btool/bzip2.py @@ -0,0 +1,30 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (C) 2022 Weidmüller Interface GmbH & Co. KG +# Stefan Herbrechtsmeier +# +"""Bintool implementation for bzip2 + +bzip2 allows compression and decompression of files. + +Documentation is available via:: + + man bzip2 +""" + +from binman import bintool + +# pylint: disable=C0103 +class Bintoolbzip2(bintool.BintoolPacker): + """Compression/decompression using the bzip2 algorithm + + This bintool supports running `bzip2` to compress and decompress data, as + used by binman. + + It is also possible to fetch the tool, which uses `apt` to install it. + + Documentation is available via:: + + man bzip2 + """ + def __init__(self, name): + super().__init__(name, version_regex=r'bzip2.*Version ([0-9.]+)') diff --git a/tools/binman/comp_util.py b/tools/binman/comp_util.py index 00761d44cc..6ec371b145 100644 --- a/tools/binman/comp_util.py +++ b/tools/binman/comp_util.py @@ -7,6 +7,7 @@ This supports the following compression algorithm: none + bzip2 lz4 lzma @@ -14,6 +15,7 @@ Note that for lzma this uses an old version of the algorithm, not that provided by xz. This requires the following tools: + bzip2 lz4 lzma_alone @@ -27,7 +29,7 @@ from binman import bintool from patman import tools # Supported compression algorithms -ALGORITHMS = ['lz4', 'lzma'] +ALGORITHMS = ['bzip2', 'lz4', 'lzma'] bintools = {} From patchwork Tue Aug 16 06:41:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 1666690 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-weidmueller-onmicrosoft-com header.b=I1GweqSW; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M6M7S1brHz1ygQ for ; Tue, 16 Aug 2022 16:44:04 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 2DF61849AA; Tue, 16 Aug 2022 08:42:57 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.b="I1GweqSW"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id CAA55848A5; Tue, 16 Aug 2022 08:42:35 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60046.outbound.protection.outlook.com [40.107.6.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 82503848B8 for ; Tue, 16 Aug 2022 08:42:29 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=stefan.herbrechtsmeier-oss@weidmueller.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Njd64a44W/Gme0CL94SSSC+kgj3XXfFswAFkMafF6qBAD1WRFVC54EOrTYtPMV0bdzlLxg+Z4wtp2Pla1QVuEVJxhtDiNg95KhcDiQP7tETYTgymLsxCIRP+kkGdEe59TGtZb6iLBsMMjssrBDTNRsKG5w3S6E87nvu4PH8Zb2KYLYH7TP3PG/EAwN+f9VJwUChRU1k9pwv3z5/BlbQTJqkQL5ZVd7KT/cs9raakgqMr+BrzEJwuYZci6WaB9i+7y0LQb5SZ6roHNfDk1Rn4wLGW0OLEyCfRgN9dJ24jPy1lsGOA2DM7UckvwVeT0OxmcAy3aanV351RnL8wdZSfVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kPY/da9qEt3AvLeBQej4HnmKH056C89Si+NTOtNsXJw=; b=dAsPp5JD7/K3IIvrlcJaEe2+EnxgQXhi63Begjt9X3kokthqN9inNxFiX+4PrKott6i/lqUypBsSGlVpk3/C1U7+cliEnGmxQf5wFWrnj4zNjz9QlOxkQ+nk75ER/dq6PzpaB7cr5mK989wmirBzMIJs1rigjmYVbB7K9HlNt+LiO90xBKi3Z0NZGxb4J6MaWegcleJGDhgRovka2ALDltfa8pwyOMdMM65Lf2ltMPBFjVSPSzX+TBxickbV39WmnHXulJae7uiwNIQ3wZgeX8UD5o83bexw4utZhreClNOiQPly2HYlctG5Y2Hj3sUwUWLsMTzv1JKYPUbWPwjLrg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.onmicrosoft.com; s=selector1-weidmueller-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kPY/da9qEt3AvLeBQej4HnmKH056C89Si+NTOtNsXJw=; b=I1GweqSWjsNkPT/fj3+Soz2DTUKrMqQ4lCHjjIK+RGE7zrw2ixykMIPMDvGgUHWXblt1Ta/XmkybtX4QtRjlP+/mXP2U+dFiy9gKCE7Snflqx0w7VcE6ihg3SG9ptllWPAb3+irQRkUFjh2YTxpq+VrFroYAR0ywh8/J5Hy6TUI= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) by DU0PR08MB8662.eurprd08.prod.outlook.com (2603:10a6:10:402::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5525.11; Tue, 16 Aug 2022 06:42:27 +0000 Received: from PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44]) by PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44%3]) with mapi id 15.20.5525.011; Tue, 16 Aug 2022 06:42:27 +0000 From: Stefan Herbrechtsmeier To: u-boot@lists.denx.de Cc: Stefan Herbrechtsmeier , Alper Nebi Yasak , Simon Glass Subject: [PATCH v3 09/13] binman: Add gzip bintool Date: Tue, 16 Aug 2022 08:41:47 +0200 Message-Id: <20220816064151.9848-9-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: AS9PR06CA0744.eurprd06.prod.outlook.com (2603:10a6:20b:487::19) To PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bae0a9d5-64c4-47b6-3572-08da7f527c65 X-MS-TrafficTypeDiagnostic: DU0PR08MB8662:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: IozEaqPUF7rXELJRZ7yHExDuaeG+8L3wIle/emWaJRUo3X/ugJLXQ3OFJ/CM79D6+XxbSQ8gEH79Ub0rP+utE0grGRxIvSPtMfTkbqYOV2N6wVt6Em9doV1LXuP9roHzJ5KxKPQ2zWg0y1ZiebR/QNAxWY/vw+ECXP29qOViFPRJe9ttLCLb4zrI19l2zogk6xz9cLq+hPCYSJYxVuJ/h6+DNVIrW/vofGAKhKay1pJnCGABENBju6Pz8x/datk7UyHWXWZqV40OiVkrGB4soM176/AB+XeqHMj4+ELmj638IS0DB9fw3zKPTGAEFrIiw55512+Bh2DRc+dpO12B1NlGvxLgwVbbUmdQocMwdo3TRBNIipu/cSFaESw9fa8+W3W2Gl5NJUKzVOZM5dS+RuJmFNxOjlYqI/IztlGDQngu8fiBrfz2dgZ0Tj5cqkVQcK+J7GMAyCV17UxOYOTKcOG1LmDdMgO/BJsqf9hy85H2MY6U4xdds/87oyLGWfYgOP6ghkvYLO8omcDYTOkoHrbX64n9gDihAlwdprRX2MGkM/IO4q+eXPRZ+fo8nYL1GJcnvvmjFyiQ0wFHzQlOfskQ55y/aNHNAJwvUI4sZlEcxLAuB7bTRUTAJMVd1KmHRDTrx/thYT9INEUVhBEDK+tANpFK8v/3NCpIjZWWdAvEPJQ5LoRN1gBQKFpojCy7xfjl2MVgNHWXYZKVHqNxGX8593zlctmCUUcVlmrDXl+wM6WcEvqbdPZLPPJtnsMajbqlPCpjm4RI/TghLxjR87KQTOTEqYfABdsYTIoKl93l5uOCj2C0L5KohYtXZfJS X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR08MB6969.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(346002)(376002)(39860400002)(136003)(396003)(366004)(316002)(6666004)(41300700001)(52116002)(6506007)(6916009)(54906003)(86362001)(478600001)(6486002)(38100700002)(38350700002)(83380400001)(2616005)(186003)(1076003)(26005)(6512007)(66574015)(66476007)(66556008)(4326008)(8676002)(66946007)(36756003)(2906002)(8936002)(5660300002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?15hlkJPD2fuiCuh1rZm7gqPhwrXq?= =?utf-8?q?rKu1gA7pe9fAhO3JfYympfZZYdmtgXEyYSliv/U+ogV7QzFu1v7gjak9QVYop/bOq?= =?utf-8?q?WoQsa7lAnf+qCx3NmrL5mNAn/zD42/vuHCHWn46RNdOZ5NQLNMIFtDM16xBs8ZPnv?= =?utf-8?q?Hh0MNGk9y49xLG1Lo+9NFNcfbzJHWPa/fn7+1TwPb+P3a2syu+Db7UJ/k2RyIb6yJ?= =?utf-8?q?wv9zPw5bNoz3fDuSFOcIY5J3+Gk5BMuoEwUspx20f3Xg2BPCXKTFoHGQM9BXJRZIu?= =?utf-8?q?mnNWaKpdpRmBqI/msj/xYsofVnozO505LP5Z9DOT8mlL3FKHbCTbl2JdHH3SLhln3?= =?utf-8?q?H9/Zx65UfhEbKis0yM6OO2gAzlPlEDJDw4XFVxNQjC6OUpnnXopW+fw5btMlpsFev?= =?utf-8?q?hppsUf1aCgugXc6CGwx9wJriPQbTFf6YQ9JfnU2GMRBchPi2zDr7dJKCLScySFech?= =?utf-8?q?Usz96Dp890GGy3C6YGDCRAVCrY6f3hnm5IVjeV5s+Yh1011n4R96xs3UBg4p+oCog?= =?utf-8?q?1ARpo+d9JXrAyob9DjBDLWzrdTmz6vK+TBufqQd2JMjSGT5ySO9mnRctbom3yvgjX?= =?utf-8?q?gU+9n0yecGRlopi8khFVIcMIy3kIKj5Y+kuhgBBkLPFgszurFogWpe+v06n9tQfbx?= =?utf-8?q?f0Eo3BvGW1Fx5+n4K9Lab6/hldcpDF9Y5z/bmNGnoOUSC8cdlJZoV9l/vMVQoO+GH?= =?utf-8?q?vS6kT64+mGKBtutfhbK6W6kK7gE2V1iYYhM2hlnZmlfrfDVbzpj6Cu3+wcY6tY6Vs?= =?utf-8?q?hkdoFYojWrQnU/e3tHG381WSs+Q4II6ICHPc2zixPgelMgpZZho7zH+wJCkmRg5mN?= =?utf-8?q?CjBqfoDwcKftlJjGkKiNZesILJO5d7qjdSl7oEkj3x2Vqk2QcKizwtnHq+iXfvm0Z?= =?utf-8?q?wkmjmC3Hjfr7Z6TAYNg4ZE79O4KZBOjZM5pBgeQsQ3LDXOy95HZ67VN7FW8Bpy3/S?= =?utf-8?q?LS8X8+k3nWxHMLtD+42k/iAWEmBPz1IsA57QToZ4OT8GiPgSsdlVF7VWTvjaSZGo1?= =?utf-8?q?FVw2zOKqrOzZhQ4dtYz5YX/5qALnBncZAr8gYN4EXmAcRbYYY9sUEvy6nr4H+hmlD?= =?utf-8?q?uDVo2QdKdS+BdxHc16REdjdcr/Fr9U0JTSpidNMBnbNOj5ybS+fxFJ8GZDwMgDE8I?= =?utf-8?q?moXsLJoR6EG5o79e+jOs/kNoFoKzZeINzl0a18HEO35rjBDXTZLooE80oRha8TJFh?= =?utf-8?q?cDeMuO3cDuFG9kMMK1dO3rdxdBwq7L7tmkNIE6jFm8d+7bbzTrUAOmj3L8usVr4Tz?= =?utf-8?q?seEX4frEKDp6rrwy7mhkRg7OVC3S1O0HRO3ViL+Bnc6hzXulXdklgIaJEpVDm11Vs?= =?utf-8?q?zf7GQ5GiTa4GbLnNyFImRqlOFdaYYw4ZwuRcgeVGYah7p8pdNE+d8EqwN/TgNBn4o?= =?utf-8?q?mOl6xO7fV7kS9WiMign6bgtJnOIRUu0fc7LJdxh87wIyP4fukus9OcBM0QbnOdPcc?= =?utf-8?q?enr8ISYfZPBtJwVCBmdzi2jij2U6MYJsvABqvHuGwaOcl2LbfPJ/hJmreOy41A1Fu?= =?utf-8?q?pRSYgxm+ZXlWPYUFnSxH+Kml9tLjsOcXjA=3D=3D?= X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: bae0a9d5-64c4-47b6-3572-08da7f527c65 X-MS-Exchange-CrossTenant-AuthSource: PAXPR08MB6969.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2022 06:42:27.8805 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: W6nxgUvSwbsE/pGxnV69LhgDZdjlWSadvG4d31iXBJZ4On6tCgF2yjxLvLk0qWn5sLWb3V7uaFuFWJqoBSrM2w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB8662 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean From: Stefan Herbrechtsmeier Add gzip bintool to binman to support on-the-fly compression of Linux kernel images and FPGA bitstreams. The SPL basic fitImage implementation supports only gzip decompression. Signed-off-by: Stefan Herbrechtsmeier --- Changes in v3: - Rebase Changes in v2: - Added tools/binman/btool/gzip.py | 31 +++++++++++++++++++++++++++++++ tools/binman/comp_util.py | 4 +++- 2 files changed, 34 insertions(+), 1 deletion(-) create mode 100644 tools/binman/btool/gzip.py diff --git a/tools/binman/btool/gzip.py b/tools/binman/btool/gzip.py new file mode 100644 index 0000000000..0d75028120 --- /dev/null +++ b/tools/binman/btool/gzip.py @@ -0,0 +1,31 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (C) 2022 Weidmüller Interface GmbH & Co. KG +# Stefan Herbrechtsmeier +# +"""Bintool implementation for gzip + +gzip allows compression and decompression of files. + +Documentation is available via:: + + man gzip +""" + +from binman import bintool + +# pylint: disable=C0103 +class Bintoolgzip(bintool.BintoolPacker): + """Compression/decompression using the gzip algorithm + + This bintool supports running `gzip` to compress and decompress data, as + used by binman. + + It is also possible to fetch the tool, which uses `apt` to install it. + + Documentation is available via:: + + man gzip + """ + def __init__(self, name): + super().__init__(name, compress_args=[], + version_regex=r'gzip ([0-9.]+)') diff --git a/tools/binman/comp_util.py b/tools/binman/comp_util.py index 6ec371b145..19627e490c 100644 --- a/tools/binman/comp_util.py +++ b/tools/binman/comp_util.py @@ -8,6 +8,7 @@ This supports the following compression algorithm: none bzip2 + gzip lz4 lzma @@ -16,6 +17,7 @@ provided by xz. This requires the following tools: bzip2 + gzip lz4 lzma_alone @@ -29,7 +31,7 @@ from binman import bintool from patman import tools # Supported compression algorithms -ALGORITHMS = ['bzip2', 'lz4', 'lzma'] +ALGORITHMS = ['bzip2', 'gzip', 'lz4', 'lzma'] bintools = {} From patchwork Tue Aug 16 06:41:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 1666695 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-weidmueller-onmicrosoft-com header.b=0wXfcYx0; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M6M8S62xpz1yft for ; Tue, 16 Aug 2022 16:44:56 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 7A58A848D2; Tue, 16 Aug 2022 08:43:12 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.b="0wXfcYx0"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 28D778484F; Tue, 16 Aug 2022 08:42:39 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60046.outbound.protection.outlook.com [40.107.6.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id D10C98419D for ; Tue, 16 Aug 2022 08:42:29 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=stefan.herbrechtsmeier-oss@weidmueller.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fdE563SvaxsuUQu4pPCuJOCEq1wMSY6E5G+vkPujHTyrP0H4SOftkiJINXwQqRFqyiX8GKCV5KS8svblykAVcn3Sp7kAY7BDvJ7ZfBjVjEdYBGVD8XS/1ErvJ4mOmWE/m8eDKFxqYGcvnC/gpJ+trojSjsrc0he212U79YEmncSehg8MB/MY9hQEF4J4Z3K0IMMW2NUGuXXpjJtK3Rh4MzcmV6d1fZH8axYdai5Oc1rTYWDy4pYeNOmNiDwC+2TIIbAADmZWSTRU6pX5zT1/kOEW6nMHGEWxfy5zPCnUXfXXYRl65nGZh/tDrujRwZqowkRVleeEO7oqInztV51+wQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5Ggd4FCuPhfzg/fi5XvJNMokzEPqyTZgAJSLP4JHGcU=; b=YNIGxPB9AdBEcM8fVOgZIPIpYx8yh4oopnugO0D2gDNjM14VQPhvacOOYQ2IOJRqBBEgBfyyM+PFZamBKssM6tkVo5mo05Q0bB3d+e/IEw9ORdox3Ln2Ofuh+qc1p+Iyi20luTyacr1/9zXPiDdl3vGBT50jEB9+WvNJnyfncs3+JuiBQHMAMwg6dcLFbOEem0hWbaWM3OZdmwAqPwKsd0WTfVXJVvtUywxJkrsjWW0CzkgY5/biYAU+Fbb7nioLdB9fneIsXNRVQOud6lNiY2SqeoL58P8863BXKJ4IDmGmRavNGGImomuxD3PHB5LQvUrCaS/fTIIUaKgMvh3OGg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.onmicrosoft.com; s=selector1-weidmueller-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5Ggd4FCuPhfzg/fi5XvJNMokzEPqyTZgAJSLP4JHGcU=; b=0wXfcYx0JxD3EBe9Fw8reSycRChLZviTQtwmvQLHO0eNSQRLDX+cQ6PUfhRxt5gMxqiJ2b6THcTPE9E0JRab0WOMdOk6M1Y9HFQCA2cj7NNQ6SnJMazJ+gAeOgS70948z9oD8tIh6qOx9F3XJvFOZPy6lrfGJO6ICWQBUlq5ums= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) by DU0PR08MB8662.eurprd08.prod.outlook.com (2603:10a6:10:402::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5525.11; Tue, 16 Aug 2022 06:42:28 +0000 Received: from PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44]) by PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44%3]) with mapi id 15.20.5525.011; Tue, 16 Aug 2022 06:42:28 +0000 From: Stefan Herbrechtsmeier To: u-boot@lists.denx.de Cc: Stefan Herbrechtsmeier , Alper Nebi Yasak , Simon Glass Subject: [PATCH v3 10/13] binman: Add lzop bintool Date: Tue, 16 Aug 2022 08:41:48 +0200 Message-Id: <20220816064151.9848-10-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: AS9PR06CA0744.eurprd06.prod.outlook.com (2603:10a6:20b:487::19) To PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 04ddec94-850a-4d20-b2b4-08da7f527cb1 X-MS-TrafficTypeDiagnostic: DU0PR08MB8662:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WnX5x1CdVtjTufmlgTQ4NvXQmyX/UvsyUcbQy6fZfKMQn1rtZPZoMSjod2+SCWn9YeKy/5joGYa/Nu6SeHCvwYUMYRANs+z5z8fyPHmdczN2YT/lKp/q4duo2hR3EidS5f8QyB1BOlV9w2D71j9LjJogdhfuW0oIMcCE5mhCWAuC5W4jejQhw8OGAtsPKqgZqlnRaAzTNcWZk4FiGKF4jZHeGnpaJtN8cQqJVZhycDmU2ScLO0ru5T1GQfzON1nNU1C/CAr3X0SePIDJ5xr5AxX3twhG9pc0wNtzCW3EhsWH4JKo0SJJpBRjY0ZDzCj5P1ROJYy5vqfjnU4ww7Kl8nML/+5RG1b3SYkCiRCBMeXpVC0KpJORQvwA5uDRQbVL6TbVRkfAnNUff+V5n2XtbcwLzadure5QHPsiz3du2hT/PnKMJZPvdTTkLcRmfhvUagRE+lx1zTbb5QM0lDgNudwdn266d9wR4mdZNc7gR8ywx1gCgECGpWxgUarqDPlbRb27jYPIoAAsNQA8+ymzhWGfpkib7wSMgduEWmIGm1El+Rxhx/KTuuKrollJVwvbsO5/cFct0usDgbAO6ZY9fhnVxuCDlHNPL3J6J4lAv4LlOMjomeCfzfJ7D35RYsRlocgLbE7CIBKT38WMtRpnxUaHZtaD/L0ZINJWTuLqG+TnfbIdu3qrXNOEZWJr+ZPuwmpJUOyzxM3Lg/Jrqq2XE/KwEbs+fnQXT6QcbXIt94mgepA1COmXeFwH/NEQgV5vKbvnpmwAzPLsV8ZfQPpVg0UHs4x4+ioKpviAj7oa9Xg9ORT76TVUuicMiA8TcIXf X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR08MB6969.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(346002)(376002)(39860400002)(136003)(396003)(366004)(316002)(6666004)(41300700001)(52116002)(6506007)(6916009)(54906003)(86362001)(478600001)(6486002)(38100700002)(38350700002)(83380400001)(2616005)(186003)(1076003)(26005)(6512007)(66476007)(66556008)(4326008)(8676002)(66946007)(36756003)(2906002)(8936002)(5660300002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?RejVCCqjyhF9V07vrhw/55nI3DU+?= =?utf-8?q?Unah5koKLv7BJ2jtxEf6By4kWDvj41eODzBcHq0PKT1W+UGjLV4cbws4OehiPL0NO?= =?utf-8?q?n7sQZfLw4I8pSXKzA23or+MS2bEuHS00oSeD2gSWac9tA7c7pcVW8/u387Da+jkSx?= =?utf-8?q?wq63X6HP9eXSs7Jve5BNyOOEiz6TRk31H3U+mgeveZMETxRDF3394IYOhDcnqeZbJ?= =?utf-8?q?z+gvFgyiNA+IYU07wB7w1/cpVA+X8hZBbdLHPYt1k/NYuf801ytcZ2ScoEoduc763?= =?utf-8?q?pQxpYTO7zhGCPI54S6j4hPpTgcwpNbMcYNZhVxPhC36GeuAQi18HWIDhp43HLI4e8?= =?utf-8?q?uckJW3XB9A2KCwtIsOhREd1nu4gVu+0ZF0cl8fIb4bCtQ/3wLZrUsYuzJf59CZb3y?= =?utf-8?q?+cSV309ryTepT+sBLzw9y4O0fcdKiE8QQZYCrAPPQPIwyipDDpNx48OWWZvD/D2i5?= =?utf-8?q?8mtkVh6rBlTd6eTVdmF/FwVam0IEd9dDWhyB5twC9W91+n7e3QpFvOeEowhtDOPac?= =?utf-8?q?YepRYAkjRnHnPyUfmnQi+PzellsytV3/gv67jjKaQZ3opSARdJEZqpln38nUNMmWT?= =?utf-8?q?nv5dusuGQV6uHSPnYVSXgspm1TMgpvu29SWjobspRku0I2JAIJ9g+RQSBcZ7M961f?= =?utf-8?q?Wrs8FccW35SnHg4LecoVpEOqbOjXw2qhedZBKLY+fyGW/KEJNmPrbUDkyuui50jsL?= =?utf-8?q?ZvZvpIxpN3SrsnysNLggcNzMTKTOY6Zgbq6AYh8Rr0+lT6RJnQHJCt4Z01wFceDpn?= =?utf-8?q?BqmLjowYzW/xM8C8QktNSgHOVJaW8hu3kBRMHt4QDgAhl3XNYqKg4QmVD+oQWQOyY?= =?utf-8?q?UQjy4YsfokmgW2sNj8U7asRf+SlQwy9W8i/fTbpISlptIlKrikakAgAPwjDTnfkJp?= =?utf-8?q?26Qi5baaY12QpK65VR4HZ8EXERxQyAPTLthZeRmbA+eJsRwrDKe8sIDCOQYhh6d2Q?= =?utf-8?q?4PC2HoA3ft+47QktyUwrHqUZ0Y/L+6Yqzp0k5mMnwNYUzGPJTjtj7PP73XMCx7DGQ?= =?utf-8?q?7UYif8qqRjnpO8Qmbzj+85RLh0hbwRtcDBspzoR2GtxFtl7sLvf4TgIm6xyu6UUi9?= =?utf-8?q?Ug9b0wN+qLYgbbwyFWBqjgRj0PPSL7DM0JsbOzOPKa/XtzdjoGeSJ3MQw1m0BooL3?= =?utf-8?q?+d9xLCCAvUY8fQjt4rzCtTxvn0ddkaHi7IaMT7EFCXVztttUx/HO1Kydl3581MlHq?= =?utf-8?q?Lh1G6UPgxpn2VOKZ5y4F+OZJGZBNeTJGFm5CFGxe99iDCYOXv7e60FCAySidzJ8d2?= =?utf-8?q?G263WMqPXpgrtpwjsi7qk5pwbwdQenzr9N7AHhuaPzqnk+s9iWJHbwOuBp9ie3c78?= =?utf-8?q?4RD5dsvvhef2TGRbVOIbjVBY/A660LIDg5TOKO9uL1cjAzcFj7rdqcHUwo82QGOIG?= =?utf-8?q?H2dPdi5t+uWsk06cGKqIJTgYxu8g8mi7WAH2OsAK1LSxmPwHdbJ04yImiHBlK+0gi?= =?utf-8?q?G+4y4Nih8IjfgEpx9Kqa+6CkZ2A3eX5ovuJ3kO+LXt0BA/ph6GbVnUnUKhg8LSm1J?= =?utf-8?q?QqM34xFJb25S+Zruu6++V38/+6ZjjePqgg=3D=3D?= X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: 04ddec94-850a-4d20-b2b4-08da7f527cb1 X-MS-Exchange-CrossTenant-AuthSource: PAXPR08MB6969.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2022 06:42:28.3818 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fV8ZrD2YIm9Q9Do6fbxExi+Hah+PfQEj01lUWcMfG0hKi1Ux8BpHMyjU2uMsOFZoiHzrk6bpLd+J23K4Ba19Zg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB8662 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean From: Stefan Herbrechtsmeier Add lzop bintool to binman to support on-the-fly compression. Signed-off-by: Stefan Herbrechtsmeier --- Changes in v3: - Rebase Changes in v2: - Added tools/binman/btool/lzop.py | 30 ++++++++++++++++++++++++++++++ tools/binman/comp_util.py | 6 ++++-- 2 files changed, 34 insertions(+), 2 deletions(-) create mode 100644 tools/binman/btool/lzop.py diff --git a/tools/binman/btool/lzop.py b/tools/binman/btool/lzop.py new file mode 100644 index 0000000000..f6903b4db7 --- /dev/null +++ b/tools/binman/btool/lzop.py @@ -0,0 +1,30 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (C) 2022 Weidmüller Interface GmbH & Co. KG +# Stefan Herbrechtsmeier +# +"""Bintool implementation for lzop + +lzop allows compression and decompression of files. + +Documentation is available via:: + + man lzop +""" + +from binman import bintool + +# pylint: disable=C0103 +class Bintoollzop(bintool.BintoolPacker): + """Compression/decompression using the lzop algorithm + + This bintool supports running `lzop` to compress and decompress data, as + used by binman. + + It is also possible to fetch the tool, which uses `apt` to install it. + + Documentation is available via:: + + man lzop + """ + def __init__(self, name): + super().__init__(name, 'lzo', compress_args=[]) diff --git a/tools/binman/comp_util.py b/tools/binman/comp_util.py index 19627e490c..d8fdd5c7a2 100644 --- a/tools/binman/comp_util.py +++ b/tools/binman/comp_util.py @@ -11,6 +11,7 @@ This supports the following compression algorithm: gzip lz4 lzma + lzo Note that for lzma this uses an old version of the algorithm, not that provided by xz. @@ -20,6 +21,7 @@ This requires the following tools: gzip lz4 lzma_alone + lzop It also requires an output directory to be previously set up, by calling PrepareOutputDir(). @@ -31,7 +33,7 @@ from binman import bintool from patman import tools # Supported compression algorithms -ALGORITHMS = ['bzip2', 'gzip', 'lz4', 'lzma'] +ALGORITHMS = ['bzip2', 'gzip', 'lz4', 'lzma', 'lzo'] bintools = {} @@ -44,7 +46,7 @@ def _get_tool_name(algo): Returns: str: Tool name """ - names = {'lzma': 'lzma_alone'} + names = {'lzma': 'lzma_alone', 'lzo': 'lzop'} return names.get(algo, algo) def _get_tool(algo): From patchwork Tue Aug 16 06:41:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 1666693 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-weidmueller-onmicrosoft-com header.b=KEWtcPQV; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M6M7v04hkz1yft for ; Tue, 16 Aug 2022 16:44:26 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 42AD1849CA; Tue, 16 Aug 2022 08:43:02 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.b="KEWtcPQV"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 79311848BE; Tue, 16 Aug 2022 08:42:37 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60046.outbound.protection.outlook.com [40.107.6.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 0EF30841F4 for ; Tue, 16 Aug 2022 08:42:30 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=stefan.herbrechtsmeier-oss@weidmueller.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MAGerjdWJ29y42tHC9QzvaHxweTLGPjqltGQKmIDKIHRuxf+2IuTgohln590DP4aL3LFIX+ogPk4SQew5sL1gVai8EdNvAyoOa1zKwTSzIv3UHLXQjWtT7D1V7945evh55YGuR7NGZ4OvJVKuN+gEJM1eI+fTo9ddQrH2XTx2i2jr3EOFFIodRBOEUY1Wi0lP0D4i9bxZ3IIsAWOvXgaMFtD4jaynKQ2sFZ5jih4hMJdBs43b0j0Wqn83QuhEkR6Qm6zG/NlEUat6Qww0zWEJv9PcNB9iFe1TNL/qhX/dM8YuUqg+3eVLpgj+MkxTNPnorsH1xLPXqcl2kaMlz/i+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QiWMO8r3LVCwwxk4EblTVE3VOt7qPwKZ5njzJbRMsYU=; b=ceaj8v0UEjMlR8hU9WFxTx+hMoPFK5hQOU9Mm8bWYBWZWwI91fgQ1lOlBe2qNgwJ02St1N2i3KS9fipO3QAyg3rB5zob7F72MG2+1oOIvZflb3K7lGjU0s1z7LTLLGHwIni2PcUsV8fW5ZzWgtCL55AuxiFwDVKDBUEoCEJUKCn5+7U9W9IqOLUOW2awxOLataCVqPA8JsF/TXAaiFlwo1XR4ovQeru4+OWHwGSA35jxgFdYIbbPsMAz6Aw6ganE8rjX6h2r/tNZ63LqSrFh4NM1vg+36Ic4q9ZgWpfW/2wLIWtwSvDHXq5ceGGpgAZJL8ZkteuKMFNRQU0AIzQ1dA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.onmicrosoft.com; s=selector1-weidmueller-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QiWMO8r3LVCwwxk4EblTVE3VOt7qPwKZ5njzJbRMsYU=; b=KEWtcPQVLBcDoZdWqdx65BJTBR+GLdV+GZjuHiU8xlr7/mQCeRJ2O/jjZB0SNA5XUuYE8vIqOibfH6ybZXYH4nCGnczk7TPI4YJJ5/G7PRaTgejFWh3w6uSlt/mKl8wnXXBvvZNce58fPzpUtsxRikjBMh9bjeb8ke7UXlc9+Bw= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) by DU0PR08MB8662.eurprd08.prod.outlook.com (2603:10a6:10:402::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5525.11; Tue, 16 Aug 2022 06:42:28 +0000 Received: from PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44]) by PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44%3]) with mapi id 15.20.5525.011; Tue, 16 Aug 2022 06:42:28 +0000 From: Stefan Herbrechtsmeier To: u-boot@lists.denx.de Cc: Stefan Herbrechtsmeier , Alper Nebi Yasak , Simon Glass Subject: [PATCH v3 11/13] binman: Add xz bintool Date: Tue, 16 Aug 2022 08:41:49 +0200 Message-Id: <20220816064151.9848-11-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: AS9PR06CA0744.eurprd06.prod.outlook.com (2603:10a6:20b:487::19) To PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3341bc48-4828-4d0a-5cb2-08da7f527d00 X-MS-TrafficTypeDiagnostic: DU0PR08MB8662:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cIML6im5CPyzmNiXwxaXAbd+Fu2cebJaBG6H94+CcwPhj03a87HHCyUYY2WIw97ziyHOQk9C3aEZjI4NfjNQ6kkyRTBy5W9UroJb7KhnEWWE/eH+TVlZnKfBCxdeSTjm2pPJ8UWIWqxsKPO/scnUBHGQAA4/pSb87L+wCOtjipPiNlfrHKphETsjW8VYd1AxA+HQfZkpq/tHeD77/IcNdPykdUKd4yudFYzJ4eOCkpUkbpS5EOByBwmK4WKdD4GZ7bDa6ziomWUcpvokqlaRKPYk9m6fLn8/beSTGpj0Gx1KCGmYPMt0IgYHifSPTg6UxQ6438cPeh8s7/SezLI1161xsNNtm7XHmUd4lp4V2/II5nXgVNI4JSS5nTup2HUIh+JXLfzYeYl0bTjXOutADKVQpJy07PawvNlWQoU9o4VtxOai/IGN6h+DNIIkaek8rgaKV31ASSksxUrERhybYel1Quk1lEDFM6u090VL5iEOhLxdjB81NuVEuIHJqO18ouN7soDD6eWRUVNkhOux44T+W8RZu2wTt5T4kj9ETcVFiZm+0FbGrxEd6MQWU+FMW3aCaQJ/yBwJCpGf8XZ6SRrCCsFPqZFmZDlYxQsmX1AM/94zDBmq3HAdH8FnL+9W0SGZVQcLcBkl5RACVTdutq5oGRt634sBA22jpbxNSTbWiw1eKWo5VWD55OA9r03wT84A6vBlhDk1BQvZrqO2VfbqBU9dmFGOyq7AfP9lVfbbpCdgii7aMKbNp2rtiQfqhgIvg+1c7hyOfkLt0GNjU7X4tGGRRw5NSUXpH4tSMTXkk/s7t8YfECgOWLOeSsqn X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR08MB6969.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(346002)(376002)(39860400002)(136003)(396003)(366004)(316002)(6666004)(41300700001)(52116002)(6506007)(6916009)(54906003)(86362001)(478600001)(6486002)(38100700002)(38350700002)(83380400001)(2616005)(186003)(1076003)(26005)(6512007)(66574015)(66476007)(66556008)(4326008)(8676002)(66946007)(36756003)(2906002)(8936002)(5660300002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?IgEr1BagQLtiCg7uRHPv3x9/ITgE?= =?utf-8?q?WR9+Cjq7kbnxzoZ/vbc9F7MPDUsDFqAgk4tnlawnBXcbjC71XuD29mMvLLwOnij/R?= =?utf-8?q?5YrEFjcJNfY3Q0mQu3yVIVawskHd0NO8hJEddA3u5bjiJf97JPgClPXEEwRhzYhRq?= =?utf-8?q?J4mgJ4DF6rFqMUYK+EbYePyiA30hiM8nOBZNknSIWO0g7ZAFEvuIApSIcgjqwT5i/?= =?utf-8?q?vyhQbqLJj6yiiNVJXjcejkPIzLvXobG/HGipw2TYDdfGTIijiAJxJoFQBxJYgaCet?= =?utf-8?q?93iFyZxYkekQ7KnJycs5nyeCrA+vFqosDb1EZwBmZMSOsUk5472kiKgL3GS+KzGIr?= =?utf-8?q?v8KqNkW/xze9o9no9hKDNW71sUS/YlZqM0tBieIzGzu+ED7aUVPZ3K5W1nNV3oopf?= =?utf-8?q?OY9m7MxIvmWFDh9h78w+9AwpzBiLp41H+RAeTe18JqtG4vR2JjTIlqbOdoMS4phLn?= =?utf-8?q?7hqxQu5wEB9nHqwNGwSUzrMsRiKHUatlJae6YfC2uo9pswL6pxvzK36tqFZA4UhG9?= =?utf-8?q?012KJEpw1ffFGTG2i3XYdY8Q7Etv1LZkY7VkHyJVMXCK4bnL3f27U31/O/vjb+Y6g?= =?utf-8?q?9DLJnu+CMDp1WoX9fNaihy5rVLX9Pqc3IdVEgrRKJincInfi+IL9+3t3DqoHk8CVz?= =?utf-8?q?vzkjGEnHKmwoJ/EsuOQF947zw+0LvFxpydKqe6EuZhd7DUd8o9lOJZFuOIRQmTMDe?= =?utf-8?q?KcQFQsoLHO6wR6chmL4r0w/c/zdkkwm92vKzQd8It3vE2kkcJ23wgRBrBUuErck0m?= =?utf-8?q?EphxFS0V64VQddz2NVwpumv0A/7pjFQhMol4devKtO7sv369OcMvdW9rH9/pKQ0mZ?= =?utf-8?q?a9ZpNwFz3LTdh3xl73LCB/ZN25nHCAzpmpBu1xT3qGVNiHuaYZ1u7Rv/ROp21LE8H?= =?utf-8?q?3SVy17oUGe9kXkCp0GjMXo77PypKL8HLoRouTuegbv7Eo0H1ixd1xSFt0DTss0HfT?= =?utf-8?q?/QvzHGVSEOmjH8hE+RN74zhdAS7C/vVq9+cS3s7W8Vp+B70RunEs2m+LO0/Vyo1GQ?= =?utf-8?q?LH2lU0RvEIO6hApKHatilLK73XadvNSCuKwyrlhGFebUPr+AK0yllaYwoNciIO26s?= =?utf-8?q?glNVQQncwsvmk5jHZm2NsVKFKDdGrfFny4kTRR/USBrcueAxseBf+678Tanc5+Rh2?= =?utf-8?q?3lpOVeF0KmfX1TJV3a7YsjcGsKDJLIuLcBzd4kXBzrdC5a1CMCWWx15DiN+gzFSJ0?= =?utf-8?q?Zup3JymSzFPUDbbD5o1g5j+geReU/SV3RgCm6gXtRo5OfOumZ0GXLuss2mdi/rQCK?= =?utf-8?q?D+yWI2LptUaXskCCM/Ku0j3DkXGszcM7w9dMPP9ogSb++NOn3ltbzV+IhNSCd+5Ij?= =?utf-8?q?iEuIz6nT6nWYM0aWrobmCtH7kyHRLbwJaP2Ez+Viw9vc/fh2s1DxrQtVUv7UiWoVG?= =?utf-8?q?agDtSL4vcoMCWL75vh0bBI9DxxxLWrhEjUkUqvKkK4Cunily8fkq4BV/lWBjGJHoV?= =?utf-8?q?0Idx3ue83Fg+0cg5ksBq7dKGvfqRt0PL3ucoW76mCJiKBCbVvWGjnQmz68X0RZmp4?= =?utf-8?q?sgmam386XasGIveQGHfpGsTwle4Q8gijaQ=3D=3D?= X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3341bc48-4828-4d0a-5cb2-08da7f527d00 X-MS-Exchange-CrossTenant-AuthSource: PAXPR08MB6969.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2022 06:42:28.8661 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hDEzmJOwLNFu/IAmvsplkbWdcwN36TBv3a1UmOCHEwfDb1l8I54WKZSTiOSbq01X2kFBWq5vC1+5Tht8DCwqMg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB8662 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean From: Stefan Herbrechtsmeier Add xz bintool to binman to support on-the-fly compression. Signed-off-by: Stefan Herbrechtsmeier --- Changes in v3: - Rebase Changes in v2: - Added tools/binman/btool/xz.py | 31 +++++++++++++++++++++++++++++++ tools/binman/comp_util.py | 4 +++- 2 files changed, 34 insertions(+), 1 deletion(-) create mode 100644 tools/binman/btool/xz.py diff --git a/tools/binman/btool/xz.py b/tools/binman/btool/xz.py new file mode 100644 index 0000000000..e2b413d18b --- /dev/null +++ b/tools/binman/btool/xz.py @@ -0,0 +1,31 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (C) 2022 Weidmüller Interface GmbH & Co. KG +# Stefan Herbrechtsmeier +# +"""Bintool implementation for xz + +xz allows compression and decompression of files. + +Documentation is available via:: + + man xz +""" + +from binman import bintool + +# pylint: disable=C0103 +class Bintoolxz(bintool.BintoolPacker): + """Compression/decompression using the xz algorithm + + This bintool supports running `xz` to compress and decompress data, as + used by binman. + + It is also possible to fetch the tool, which uses `apt` to install it. + + Documentation is available via:: + + man xz + """ + def __init__(self, name): + super().__init__(name, fetch_package='xz-utils', + version_regex=r'xz \(XZ Utils\) ([0-9.]+)') diff --git a/tools/binman/comp_util.py b/tools/binman/comp_util.py index d8fdd5c7a2..8c0fe5078f 100644 --- a/tools/binman/comp_util.py +++ b/tools/binman/comp_util.py @@ -12,6 +12,7 @@ This supports the following compression algorithm: lz4 lzma lzo + xz Note that for lzma this uses an old version of the algorithm, not that provided by xz. @@ -22,6 +23,7 @@ This requires the following tools: lz4 lzma_alone lzop + xz It also requires an output directory to be previously set up, by calling PrepareOutputDir(). @@ -33,7 +35,7 @@ from binman import bintool from patman import tools # Supported compression algorithms -ALGORITHMS = ['bzip2', 'gzip', 'lz4', 'lzma', 'lzo'] +ALGORITHMS = ['bzip2', 'gzip', 'lz4', 'lzma', 'lzo', 'xz'] bintools = {} From patchwork Tue Aug 16 06:41:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 1666696 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-weidmueller-onmicrosoft-com header.b=QPK2iQZz; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M6M8h64t4z1yfd for ; Tue, 16 Aug 2022 16:45:08 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 961D384A0F; Tue, 16 Aug 2022 08:43:14 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.b="QPK2iQZz"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 07D4D848C4; Tue, 16 Aug 2022 08:42:40 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60046.outbound.protection.outlook.com [40.107.6.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 4896F8437C for ; Tue, 16 Aug 2022 08:42:30 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=stefan.herbrechtsmeier-oss@weidmueller.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iJ8cFFGxGO30dpPdqUJeOsrDL5sCCp++YM6NKEnYIvY/ODiPamTbA8uV9ybwGh2WCP7RMaaCvUd53wFXDwEC5ED8XL9cobkskIg7lTQ+vaQ+/HPs+jeZhNqb3dxJk2qPmfp8U7bEinPsBtdNAK0/lQF/uhzBv/oRfLIaD6lGvpUzmANoACgUTu1dJQOSMHS42KDD0YVKdF2QwSNJdrkSccxsoI2c9CQikMUVSJvyWfN+FwTAhaIfcg1C2LPmFbqUmSgr5zKMiIFel1VQcWNzjTO23jOdoIrH3s4CFS8zcMhwywHjFExYwdtFnBHuADCPHf7PAL1Y65G/1zA72X4xfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BXK15Q2QksGyAbySRrBo8NEtWAYTTN3HjwQNCS5Jvvo=; b=W33AG14Y6fcf0hjMJwH94iIe9rK/yuwsgjFxrglhDe++5lHyYouQ3+aSfdQBm7+/PLmJuz0aiH+roPO0y06uGMmYqHDeQLlk7+p6xclqO6VEqS+38+CWQ881apv2tmD7pfPj1SNS11MPNsEdTDpal1jyM63d3zKbW5bjtAnjgewSjUIYkhlK1WXTluY4AqtCUlWJYtKHiqW/AT+VZmCvt90KFQr7K4duCrfuADpP3hS89YyxgcT07iNjGVVGE/agHksKuHc2NZPheD3Xoa6IOep9QL7zdAfKduieeE8bLAyLqwty/s5aewQ3qFHi0Ct1ZPAqMAreL8jfXdXnpbxO6A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.onmicrosoft.com; s=selector1-weidmueller-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BXK15Q2QksGyAbySRrBo8NEtWAYTTN3HjwQNCS5Jvvo=; b=QPK2iQZz+4RvkM2JveA9tmWzcYxc7DURVjz3WOA3xO0ZpJZqerxf5vVkIiPmTvvKvbDhGUnFAK1SCEx2NVGAuwa0uqGif7P4LtzGct/E+SeX8eO2pIUtY9iQOeUgEu1RZG42oXZoJfNNX1dMyterhhcTp7Xkb+4qSs7gNh/wkew= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) by DU0PR08MB8662.eurprd08.prod.outlook.com (2603:10a6:10:402::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5525.11; Tue, 16 Aug 2022 06:42:29 +0000 Received: from PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44]) by PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44%3]) with mapi id 15.20.5525.011; Tue, 16 Aug 2022 06:42:29 +0000 From: Stefan Herbrechtsmeier To: u-boot@lists.denx.de Cc: Stefan Herbrechtsmeier , Alper Nebi Yasak , Simon Glass Subject: [PATCH v3 12/13] binman: Add zstd bintool Date: Tue, 16 Aug 2022 08:41:50 +0200 Message-Id: <20220816064151.9848-12-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: AS9PR06CA0744.eurprd06.prod.outlook.com (2603:10a6:20b:487::19) To PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 419ddcda-da69-4c66-1dd9-08da7f527d43 X-MS-TrafficTypeDiagnostic: DU0PR08MB8662:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HJ90hnRdoVmAGN8ovQTfZ7ovHJDdFLYmXy+tvQ7Yt/CYbkcyBuhLnFHSGf4ZeR/PkpmpQZBjNi7SxMHycz26m07ocfQomNlJx/CGaJ8RrKVkr9Lr5MU9SukOMuh6blmCiZh04suz/DWsc2PWRM1q7NSleCqj7261qSivcCI8caeeE7P3H3LQWTjbbrXC1T5lMJdyBFp1e7YsVWggZpPjPK3GhPCmjaBdYCMvzioTzH4e8DTunRg8OwdNza7R01TSUR/6l7KEOj77Tp0xdQJL5r31bdhl0zrqOmIyH7ss5n3nOZr1KNsFeWDYA/zPZIaBjqzqMIP3r3Ajsfy/wLJlf8JyfMeBd7MX+xiXkqgpF3MHK72H1RFcLvP/9wIPWGE1QG+cOTu3qnUC0JXldF7iHXTrtS6IQGlXeCzkkMm2mD83nhT7VKNuzQAGE49+LrlQuztUguazedyZvOf1qARVMkjF92UpENXjQyo1/jRaIC1bQrhYpAy6VoTS0LtK5XpgnvbnH8WMsD9fI19lh1934qkVmZCXepAEoFiVebkTSyAp1uZfFIzVTU5FFHBT2Nu2+aydlnSWILkMk+vy2U+lnW2oljNYgU3IhUVTePbP6WbpRae4iUFISXh4zDUUs/tDL5/8HXghIebSgjFm3NtMUprVvjVNnSH1d351hUmDZ1yoEJhQ9DRMMazwZ2H9VAhZka0AlBe95QjctUx/akQ9t39q0UpUGbMYBfLtxNd3X0bTZfW3GF4+n0dQ7pfHmSFsRcQH4FYBPtPDxjDOOuwewGSSuoPtmbVQO6NRADOXm/IdIWd575jwEMC41mHeRwKA X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR08MB6969.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(346002)(376002)(39860400002)(136003)(396003)(366004)(316002)(6666004)(41300700001)(52116002)(6506007)(6916009)(54906003)(86362001)(478600001)(6486002)(38100700002)(38350700002)(83380400001)(2616005)(186003)(1076003)(26005)(6512007)(66476007)(66556008)(4326008)(8676002)(66946007)(36756003)(2906002)(8936002)(5660300002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?FRUB8dB+1Z8fZ5kcjOxjAf1OBKcG?= =?utf-8?q?XjXs+aSPkIOOKm2UbcbiJlWVZdfel3Xf9zsAcAa7tiM50MZamng7I+kyfcl6uRqte?= =?utf-8?q?b4idk3KUyrnPLNgOV6ln+EnbBk0FXAz/F+W1qQEeAo/3Ch3VfSR0U2/gO9nMkiAZS?= =?utf-8?q?pWi8YFJV/2yoM2p0TAr5AWha7h7ge4s6M4/e1Hrz2R8qVba2V2rAiFlZFIr0fVOE9?= =?utf-8?q?ECrdNzZNvbQdxgNTnIkRC1C4tQDNcObEFOl20nCSK7KiBnjhSorL6Vp7UXBwBV/kl?= =?utf-8?q?dKTY3Xbo5vrlzi78kEGRnejfAAQybZx6CBiaZIYJ9lJVN2fmhW3BFG+QddxG6sRrN?= =?utf-8?q?/IOYzFC5qPPKkNH0mUUiSTQjs1figbUNnpI6dAJ2uQY3tMChrmMX200oJPNVVIph8?= =?utf-8?q?Tn+xFnauTuPIU90+CjPezXxvQgz9GGAxKGPRU/DOLBilTDHlLW6blc9IHXR5O/i4L?= =?utf-8?q?Isdl4CyEYOhHTc3+XQNFiEuOqC2Kz7kkopWnNOWbUevjUvpmM3KE9N63PDbfj+eBO?= =?utf-8?q?pJj3wnhauQVjnnMahWCm5SNXPrCG98xzPM4foFl4pvRFH8opeJCOklNcWyt9F+aPM?= =?utf-8?q?GxcYD1jr4KBWBem1wQvQJ0CY/ZHltAfrfW++S0aUF3g6fE4zNt8fyiiX5IS/KKxGR?= =?utf-8?q?KMbW6yOsGh3zOruq8dIyy+Y4gBaBOE7CLn4KQaWfxj469LNhXM8U4w61oCH8TBJeK?= =?utf-8?q?RixQ49qj0YRBQ+MEo9gw2X8aX8mdoGcMYfdLGDYkShk8pA5TPApCeM89Oxkw6NeNY?= =?utf-8?q?r7llgP+5WHsfbxPc2xlfGZR1EbDL4HQlfRQqopO7CWT/3g2WRpPpt+1lnT2FKHQaB?= =?utf-8?q?n0fJcLRfRZEV5qV3Ivq/QcoHJs+zWMqPgHXUF+w9IUn59ODyByKDGJYCv1p/8CIvX?= =?utf-8?q?iAUyS7z57fXFMojf9YgUsEoKUkeEMzcOW1PhgrE37gxFGHxtPCy6V37QBpZfDOAnu?= =?utf-8?q?95dOzI8Zct7ofJin2q9lRdD8MqH5O6VhmLcwnZTjNz8unEg8UspLNo5icgGhdGL0Z?= =?utf-8?q?IKzss1zS1m3fvY3GLRSJAwKsMRz6BNE80rPuWGkfoVWjDy3v+lP3D0MkxvsVC/eAV?= =?utf-8?q?TuIRpxVw6ngpk5rCWql0DhxWzNIeS/0LmGKVUUM3Lm+q5fqxEqpSkJGeefnrpnXho?= =?utf-8?q?hhOd2oXoj+6rUh/phcHSQXhA/lvR4LQRP30RdeeFs5yoXAROfZjPNwsjeyaWEj4Aj?= =?utf-8?q?PwZYV6r9z0zangFaSjcJsLRlyX3hUtR3sEJTA1/wig8rQQKhTTX+wwNsUKv4wDfEN?= =?utf-8?q?2/I4BhZIWMYZ8OIXl5OMaEZrYaAJWqRu+C8oyt5dL2z3IsmsDXP/43JeMn7ohnwid?= =?utf-8?q?aWblvw1GFSruz7D/5qbJYoP06bN82dYzTYXZdWOYVE4JTNiZ4EEo/QSndWdeo0a2m?= =?utf-8?q?Br/fEMtwwplcZVtjQfLlec0hOx5d0fmVXi0rHFg/MB2DP0ydpNI3n9jrBq+J09KGm?= =?utf-8?q?SRClGTumlwBj5MefOr9TzyTRONClRLHNCO7UaQSgK3Pgc70/5HqE6S7Nn6XreJn1j?= =?utf-8?q?iUqjtkerxPBDKpsK3Qua/AVWawyvFRf+Kg=3D=3D?= X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: 419ddcda-da69-4c66-1dd9-08da7f527d43 X-MS-Exchange-CrossTenant-AuthSource: PAXPR08MB6969.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2022 06:42:29.3661 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QvH5COZZ2YSC2PS0HBxaELDjYlpUBAm47+6uCXsnQmKyaRTLt4IJXoab+O80kgt8G9i4NEc7XXGQIuIWx92dag== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB8662 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean From: Stefan Herbrechtsmeier Add zstd bintool to binman to support on-the-fly compression. Signed-off-by: Stefan Herbrechtsmeier --- Changes in v3: - Rebase Changes in v2: - Added tools/binman/btool/zstd.py | 30 ++++++++++++++++++++++++++++++ tools/binman/comp_util.py | 4 +++- tools/binman/etype/blob_dtb.py | 4 ++++ tools/binman/ftest.py | 3 ++- 4 files changed, 39 insertions(+), 2 deletions(-) create mode 100644 tools/binman/btool/zstd.py diff --git a/tools/binman/btool/zstd.py b/tools/binman/btool/zstd.py new file mode 100644 index 0000000000..299bd37126 --- /dev/null +++ b/tools/binman/btool/zstd.py @@ -0,0 +1,30 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (C) 2022 Weidmüller Interface GmbH & Co. KG +# Stefan Herbrechtsmeier +# +"""Bintool implementation for zstd + +zstd allows compression and decompression of files. + +Documentation is available via:: + + man zstd +""" + +from binman import bintool + +# pylint: disable=C0103 +class Bintoolzstd(bintool.BintoolPacker): + """Compression/decompression using the zstd algorithm + + This bintool supports running `zstd` to compress and decompress data, as + used by binman. + + It is also possible to fetch the tool, which uses `apt` to install it. + + Documentation is available via:: + + man zstd + """ + def __init__(self, name): + super().__init__(name) diff --git a/tools/binman/comp_util.py b/tools/binman/comp_util.py index 8c0fe5078f..6b4ab646e0 100644 --- a/tools/binman/comp_util.py +++ b/tools/binman/comp_util.py @@ -13,6 +13,7 @@ This supports the following compression algorithm: lzma lzo xz + zstd Note that for lzma this uses an old version of the algorithm, not that provided by xz. @@ -24,6 +25,7 @@ This requires the following tools: lzma_alone lzop xz + zstd It also requires an output directory to be previously set up, by calling PrepareOutputDir(). @@ -35,7 +37,7 @@ from binman import bintool from patman import tools # Supported compression algorithms -ALGORITHMS = ['bzip2', 'gzip', 'lz4', 'lzma', 'lzo', 'xz'] +ALGORITHMS = ['bzip2', 'gzip', 'lz4', 'lzma', 'lzo', 'xz', 'zstd'] bintools = {} diff --git a/tools/binman/etype/blob_dtb.py b/tools/binman/etype/blob_dtb.py index 4fd2ecda83..fab79e43cc 100644 --- a/tools/binman/etype/blob_dtb.py +++ b/tools/binman/etype/blob_dtb.py @@ -48,6 +48,10 @@ class Entry_blob_dtb(Entry_blob): def ProcessContents(self): """Re-read the DTB contents so that we get any calculated properties""" _, indata = state.GetFdtContents(self.GetFdtEtype()) + + if self.compress == 'zstd' and self.prepend != 'length': + self.Raise('The zstd compression requires a length header') + data = self.CompressData(indata) if self.prepend == 'length': hdr = struct.pack(' X-Patchwork-Id: 1666694 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-weidmueller-onmicrosoft-com header.b=mNhNi45a; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M6M86072Yz1yft for ; Tue, 16 Aug 2022 16:44:38 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id C7C7C849E3; Tue, 16 Aug 2022 08:43:06 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.b="mNhNi45a"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 58D3A841F4; Tue, 16 Aug 2022 08:42:38 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60053.outbound.protection.outlook.com [40.107.6.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 0164A84228 for ; Tue, 16 Aug 2022 08:42:30 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=stefan.herbrechtsmeier-oss@weidmueller.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XzIbvViIModoW39LpxxT5eHQ6l8HsH7B6J9d00cz2enpUD7+nGQtvDHcmZoRdaiRLXWkW7L/0Pe+e0q5EgGlm++z8KBiznkIeeVK4KzwcVURmiervvizO8qCVLjzYqp0Q1sqUBSmS07Ad1+WrEaIseZy1cEu1fG43R/V83LBwtZAlxloMJQhewcEzm/jv8vYDcbfXfDzfVvWa3jDm0+RdiiJMER6gRMYubdiHNGt2JcgwtwTeBiTNzet18JBdjqF822L3uwhX+7DaWqb8O2IOkvW50xGOErFJdycRoHcHGAszGlyFYaVhovla3/gxWF7st8Or5SilrIA/GyMNzULaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2Cx1bC+Wc2IcXvm8KTfJM/DAhke0WZgO9IQDipi4Ntc=; b=UoxuO4FSrtUPVNhk2cO5tCBQ10MCZhQQursj7ociP0HmvgTbodsCGNKxTI7Z7uujVlEOLFPjNwh6avy+YDHAHKoss4TaqvgZXa8vxey4i1+ICeo4iN+6UjmTneIwpG+zC6XXH7YOm6oWJE3+TfCUwoyRDISK1T8Nis2KlTWZ9bo+j3tH61KC0oScJHmQw1pYWdbKuExZA9d+fOFcImIchIOjqzGmggTSdp9zWK5Uqs3+VSmnjXeoZHxRW5MAPRJWj5Vu0D1SzchEcEJrOdNquMmJFIoT+4MlqgYVbjDLF/AZyg9/O+WDRe5LzG+Y4jAbV7NwOP6g4I2tio4iAkZZuA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.onmicrosoft.com; s=selector1-weidmueller-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2Cx1bC+Wc2IcXvm8KTfJM/DAhke0WZgO9IQDipi4Ntc=; b=mNhNi45afcbAZepxjrz462WX6UulnGfb9anNJc6k/Y54HVFG688R6DD4Qf832magPSs6IAgdWtn5CWOd9gSr70WyjPY6timJmGTlMIn7VHD2zczqA95qvDjz5u2D+0xAInqNG/FAY8J30rHrhX9O1dYdeQBdPOwjDFSQJNqNuzU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) by DU0PR08MB8662.eurprd08.prod.outlook.com (2603:10a6:10:402::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5525.11; Tue, 16 Aug 2022 06:42:30 +0000 Received: from PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44]) by PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44%3]) with mapi id 15.20.5525.011; Tue, 16 Aug 2022 06:42:30 +0000 From: Stefan Herbrechtsmeier To: u-boot@lists.denx.de Cc: Stefan Herbrechtsmeier , Alper Nebi Yasak , Simon Glass Subject: [PATCH v3 13/13] binman: Support missing compression tools Date: Tue, 16 Aug 2022 08:41:51 +0200 Message-Id: <20220816064151.9848-13-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: AS9PR06CA0744.eurprd06.prod.outlook.com (2603:10a6:20b:487::19) To PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 130e6435-e951-4ee3-9950-08da7f527db5 X-MS-TrafficTypeDiagnostic: DU0PR08MB8662:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: eYmImg1vbzz/KGPUTaBBmp61aGWRnozi377ZSrQxphBDuoY6yNflKW+imKi+9OK3gOyx1kWAJFh89X5Piwzm7o45w3R/oJvWD/+A9nohGWjhf1R7S7/z2so/ev7c3ITQD/OI+sIMBRlEFWtV+Rat4VPDsnF3XqoDSNtwLOMWirUsOBTSSYi9KXZc3qXVKDkROU1HiVO2f2urtRY1/7Mw50vGomAlKT4i+DWP8n4uK3sv422RbkRc+1thvbdyFIn4KCW5JMaRQLl4iEQKcLnkieGpOReEIrc3EVBJQfJfxNYDI/Zaf0G78RVQOIqYU+ICTynwbnUFgT0zfyMznGakO6lcqynH33lBCBa6QBt/xUEOlXCS7bLcUaArQLhfISEoO1HN3lfsaE+10cvBVWfbRTHYgAcxw0LmaQu4DQpv1dDIkGnLwGKkTJISGCxhJ8BItAJHsVcBlQd00L3ANyAvo03XhPntnwQokyCwGRdPa0PrPZ6i/MbAJ9xtnRqXuhVaErwpkG+d8C7I2UrVq+VIbdBYa+XFGFvjqSlgI7gOWjJY4OEPAZuAw69gKAFzwVPuX118kX4Qxgz6LjVSrhqkZ1O3EPFEnonwlvxWEySxLfLR2senL7zst/zs+/989ay6xmgjlkezkxob5B27WWZaJuZr1yRMmyoqootG8UP5NdiQ1D5IBvaU1AjAv9i+c2/F3pvp4tTzis5/yEbi75/c3/L4riADhpY58CHAdUm/2cBB+fwSZ6Jd2qoEMySPUPRR6royq0W2chpUsadpVqXmpQtGaF6PeOtwG9cAX/SHQnXyrgtEgkZnD4voYm9fM2eO X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR08MB6969.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(346002)(376002)(39860400002)(136003)(396003)(366004)(316002)(6666004)(41300700001)(52116002)(6506007)(6916009)(54906003)(86362001)(478600001)(6486002)(38100700002)(38350700002)(83380400001)(2616005)(186003)(1076003)(26005)(6512007)(66476007)(66556008)(4326008)(8676002)(66946007)(36756003)(2906002)(8936002)(5660300002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: xW7Gv6+W3UnQNyQcLCV3lkl1A93332gxQGGx8dLgz2YGTysh5aHXwhPwGfOqYWP0miEaUWJK+opz0VjYH1zdnskCtXJyQHhAG+zVUk2Buez2krTezXtZTCh4nA+bAYaFYjSE+oVXIExRKjMc2J495FKlPIa1CL88ceumQ2PHg2ocH6MXB+6AbRNhtx3i8qkIBCNGgLdxpFRuCBVZWcKbW3K3RR1BrBc168FQV9LZXnRnMuK24BF3bMGGi3e0cjxWzcKsylZlONnleriA6e+k+ndS/pF33Mlqpl2wL1XnqXPJ+v0eYA9tSk5PGxGear5hp6l6omlq7Mf0lcxmqEQGemNAWxh1fC3zUHPZMscBNU31NBhYDnheUdM8K1NSgcO5O0SHem/nXPBj4b0AUPyPvDlH2YpPznCjVRfy19zXL2ORm+8dVIH508eHC2QUB7zidWIMaIhZrMjFDxs0DaSNCxZZIuR51c+NiDYg6Lj1wX+d9OHJm1i9nll3uS0l/GOMvvQgidklSy9kZY7eCacc1AwmHt8K5Pr1hEOqVxJZFEuVO/Sbn4RTaLWWzbIIJfBPmCPFYXBMia7rDELzsY8qqfXpeVpICFKVw3Gm+iZBcNADT6/jLyB4hsk3ia6Nhg6LYH4pVdbJFl4rXwQKb/zylES18qNEHlpOcqTpiKcEy6lD1fR9LnmuKzu8YKYUoZsq/VxGgDl0o9dhGLSe08sj4ETs5gJAw9T0YVKWNvkSz1ib9kuLrNI2TBHmnNlawGtLWKv4/tBVelu/Z/lV1vNhOPxRt0E0ip1Jy9gxF2TdKCP89/2K1Z5Ld15rdDRK2VeP9IgsbxDoiqBHEEEjaDbvI88YSUa0KAbS6nKjPWbMInU8WYh5XPZjGgnsXPWcSLZjvJBcyvFiFek08ecv5e1HVC9u2IUyxdqB8Z/T0yJ2nD6l8e1MkySEGSsxsYSOOlmq05qjJwkhMTSy1SHidQu3oh2xCSYqiG6GBgP3RMfJ24KKi5WOiIDIr6yTgbgz2ilWfPdddSpDf/KGWCyswHmNvbUrbHSsr0dn94/dQKHB4H7lhxpF0GNKw0nvULjRK5yTV4pOK7q1havEHIVxmd2u7CNFifH6FKIpAIS0KBZszUizXeE3773+NU3uq2YgOkgKY1dLkVnJUWVuvM9k4zbYUuOmXuUIQ2QY73imdidmSVXlGrLYzIs6Cel66jQ7dInSlZHTZuloLkp2Q5LXDnyjGnrIR2BwlNRRehY8NWVp96WIQEQy1EVIYI5B10ti/nV8xPMd2gy1dzvlYeRii02kpLt9HY6WKSZkwVDB4mSijl9w6F83sAZhVWY71wlZN4938LWqF5Lm/VahRokTlUpuGsTCsRPSmzF1nLVT+EZ2+W4uhvd05mEtnExyg7kBTF2QoT20NFnpv27xYtHRNUbpmJr4vpar+O0p+/FKC2OU3ryGGPIcwMsHeXNuKsb3eoo2YqXPPvavUnERqmRGvLmCjVCT/X59vHTREZTPXPVrwmNmaPHhYvS3deWxrRC7raUvj/mXBSnssbuvxk7lg6VLdH+nvDIoUqwoJE+CXCkMU4WlK7f4gzO1ghv5wpxtttcse03BRlDeDnw8/kEXbPOuVA== X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: 130e6435-e951-4ee3-9950-08da7f527db5 X-MS-Exchange-CrossTenant-AuthSource: PAXPR08MB6969.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2022 06:42:30.0226 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +9TeWE7WCgsHKhwaInwDWnExLU4ED4jdkoezm2TEuimHTDEbzfMwXRlYje+Z5eaDORV4ipDTkEJgoh8JoLkEyw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB8662 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean From: Stefan Herbrechtsmeier Handle missing compression tools by returning empty data and marking the entry as 'missing'. Signed-off-by: Stefan Herbrechtsmeier --- Changes in v3: - Added tools/binman/entry.py | 4 ++++ tools/binman/ftest.py | 8 ++++++++ 2 files changed, 12 insertions(+) diff --git a/tools/binman/entry.py b/tools/binman/entry.py index 9ec5811b46..c86b757a4e 100644 --- a/tools/binman/entry.py +++ b/tools/binman/entry.py @@ -1078,7 +1078,11 @@ features to produce new behaviours. """ self.uncomp_data = indata if self.compress != 'none': + if not comp_util.is_present(self.compress): + self.missing = True + return b'' self.uncomp_size = len(indata) + data = comp_util.compress(indata, self.compress) return data diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index a360ebeef5..eac7ccb087 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -2557,6 +2557,14 @@ class TestFunctional(unittest.TestCase): } self.assertEqual(expected, props) + def testCompressMissingBintool(self): + """Test that compress of device-tree files with missing bintool is + supported + """ + data = self.data = self._DoReadFileRealDtb('236_compress_dtb_missing_bintool.dts') + self.assertEqual(U_BOOT_DATA, data[:len(U_BOOT_DATA)]) + dtb_data = data[len(U_BOOT_DATA):] + self.assertEqual(0, len(dtb_data)) def testCbfsUpdateFdt(self): """Test that we can update the device tree with CBFS offset/size info"""