From patchwork Mon Jul 8 20:25:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1129338 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="IKYsZqJ4"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 45jHSn2qcbz9sNH for ; Tue, 9 Jul 2019 06:42:33 +1000 (AEST) Received: by lists.denx.de (Postfix, from userid 105) id 08231C21DA2; Mon, 8 Jul 2019 20:36:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id 2EC51C21DFD; Mon, 8 Jul 2019 20:28:48 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 0D1D8C21E53; Mon, 8 Jul 2019 20:27:53 +0000 (UTC) Received: from mail-io1-f67.google.com (mail-io1-f67.google.com [209.85.166.67]) by lists.denx.de (Postfix) with ESMTPS id A0948C21DF9 for ; Mon, 8 Jul 2019 20:27:49 +0000 (UTC) Received: by mail-io1-f67.google.com with SMTP id m24so28798131ioo.2 for ; Mon, 08 Jul 2019 13:27:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fiNNSK9ckz9N+ZxU7yD2N/VonSotBf+awfyCLc07OPc=; b=IKYsZqJ47OKJXbTccFMqCAk2B+VbYd9girRQ02Ol0ekyR1YLdHw3LSVvqHKfC3HWCN 0NznATWtLeqshyZGkRy2e9nNetaIRt7B39dSMrd2KEbnnjkIjn2MZmFKScn+beOV9ztm 4yj9zg4YT0MZsFqTGY0DYWphKNOWftWSWQntI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fiNNSK9ckz9N+ZxU7yD2N/VonSotBf+awfyCLc07OPc=; b=W3EejH+LyW97ynNDLgZbGYWQI+OfG9w2s7l0VVDNaLsK1QmfDUtxwHszH4OMrCttBV GKjk+WhPryO31CDobch0+xRr1xSa4T7Ko+eD4jvkHcvcfYlTSEDPFEuisX4Ggw9qMs6M 0dg09vm/FaC4OZp99agBQEh0GCydmCl5/fpNQmLDV/WukoIA9JVh+5PoyxQENX6Jyb0y Nm85uiuFB53UVLFHs7B3MzL1Zn8YYOXIRNuweJpYAV4zIGjslIwIZQLpEJtkADadBb9f TSTitFB7Xu9poQ9phSLrPVZ538aPkkxeeXHOcIoB9mWFfVvFjwHasskpvhTdRLRuhFOt RTPQ== X-Gm-Message-State: APjAAAX5WQ5JWhSE0mFZl7vXHN6jb2qS3ZW7yec0A9ZYU/k38AUhyCoo 93kAei9layt2P8EVqWQyd47jLxFfR8g= X-Google-Smtp-Source: APXvYqwkwQD735RRXlVRR6rBIntdlV+D0gmtZVpNHGr33cY6I+SSVCc7S/onO2MA4rOks4OlmZ5tvw== X-Received: by 2002:a5d:885a:: with SMTP id t26mr20212916ios.218.1562617668486; Mon, 08 Jul 2019 13:27:48 -0700 (PDT) Received: from kiwi.bld.corp.google.com ([2620:15c:183:0:8223:87c:a681:66aa]) by smtp.gmail.com with ESMTPSA id v13sm16692359ioq.13.2019.07.08.13.27.47 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 08 Jul 2019 13:27:48 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Date: Mon, 8 Jul 2019 14:25:38 -0600 Message-Id: <20190708202553.225715-17-sjg@chromium.org> X-Mailer: git-send-email 2.22.0.410.gd8fdbe21b5-goog In-Reply-To: <20190708202553.225715-1-sjg@chromium.org> References: <20190708202553.225715-1-sjg@chromium.org> MIME-Version: 1.0 Cc: Tom Rini Subject: [U-Boot] [PATCH v2 16/31] binman: Allow device-tree entries to be compressed X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" At present the logic skips the blob class' handling of compression, so this is not supported with device tree entries. Fix this. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: - Update test to use _DoReadFileRealDtb() helper tools/binman/etype/blob.py | 25 +++++++++++++++++-------- tools/binman/etype/blob_dtb.py | 8 ++++---- tools/binman/ftest.py | 18 ++++++++++++++++++ tools/binman/test/124_compress_dtb.dts | 14 ++++++++++++++ 4 files changed, 53 insertions(+), 12 deletions(-) create mode 100644 tools/binman/test/124_compress_dtb.dts diff --git a/tools/binman/etype/blob.py b/tools/binman/etype/blob.py index ec94568ac0a..a4ff0efcebc 100644 --- a/tools/binman/etype/blob.py +++ b/tools/binman/etype/blob.py @@ -41,17 +41,26 @@ class Entry_blob(Entry): self.ReadBlobContents() return True - def ReadBlobContents(self): - # We assume the data is small enough to fit into memory. If this - # is used for large filesystem image that might not be true. - # In that case, Image.BuildImage() could be adjusted to use a - # new Entry method which can read in chunks. Then we could copy - # the data in chunks and avoid reading it all at once. For now - # this seems like an unnecessary complication. - indata = tools.ReadFile(self._pathname) + def CompressData(self, indata): if self.compress != 'none': self.uncomp_size = len(indata) data = tools.Compress(indata, self.compress) + return data + + def ReadBlobContents(self): + """Read blob contents into memory + + This function compresses the data before storing if needed. + + We assume the data is small enough to fit into memory. If this + is used for large filesystem image that might not be true. + In that case, Image.BuildImage() could be adjusted to use a + new Entry method which can read in chunks. Then we could copy + the data in chunks and avoid reading it all at once. For now + this seems like an unnecessary complication. + """ + indata = tools.ReadFile(self._pathname) + data = self.CompressData(indata) self.SetContents(data) return True diff --git a/tools/binman/etype/blob_dtb.py b/tools/binman/etype/blob_dtb.py index 09d5d727138..88ed55d8865 100644 --- a/tools/binman/etype/blob_dtb.py +++ b/tools/binman/etype/blob_dtb.py @@ -23,11 +23,11 @@ class Entry_blob_dtb(Entry_blob): def ObtainContents(self): """Get the device-tree from the list held by the 'state' module""" self._filename = self.GetDefaultFilename() - self._pathname, data = state.GetFdtContents(self._filename) - self.SetContents(data) - return True + self._pathname, _ = state.GetFdtContents(self._filename) + return Entry_blob.ReadBlobContents(self) def ProcessContents(self): """Re-read the DTB contents so that we get any calculated properties""" - _, data = state.GetFdtContents(self._filename) + _, indata = state.GetFdtContents(self._filename) + data = self.CompressData(indata) return self.ProcessContentsUpdate(data) diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index aae8dbc1b33..89e732fca31 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -2142,6 +2142,24 @@ class TestFunctional(unittest.TestCase): self.assertEqual(U_BOOT_DATA, data[2:2 + len(U_BOOT_DATA)]) self.assertEqual(b'aa', data[-2:]) + def testCompressDtb(self): + """Test that compress of device-tree files is supported""" + self._CheckLz4() + data = self.data = self._DoReadFileRealDtb('124_compress_dtb.dts') + self.assertEqual(U_BOOT_DATA, data[:len(U_BOOT_DATA)]) + comp_data = data[len(U_BOOT_DATA):] + orig = self._decompress(comp_data) + dtb = fdt.Fdt.FromData(orig) + dtb.Scan() + props = self._GetPropTree(dtb, ['size', 'uncomp-size']) + expected = { + 'u-boot:size': len(U_BOOT_DATA), + 'u-boot-dtb:uncomp-size': len(orig), + 'u-boot-dtb:size': len(comp_data), + 'size': len(data), + } + self.assertEqual(expected, props) + if __name__ == "__main__": unittest.main() diff --git a/tools/binman/test/124_compress_dtb.dts b/tools/binman/test/124_compress_dtb.dts new file mode 100644 index 00000000000..46bfd8b265f --- /dev/null +++ b/tools/binman/test/124_compress_dtb.dts @@ -0,0 +1,14 @@ +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + u-boot { + }; + u-boot-dtb { + compress = "lz4"; + }; + }; +};