From patchwork Tue Nov 14 01:55:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 837663 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.denx.de (client-ip=81.169.180.215; helo=lists.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.b="v/9r1tRA"; dkim-atps=neutral Received: from lists.denx.de (dione.denx.de [81.169.180.215]) by ozlabs.org (Postfix) with ESMTP id 3ybW6y75Lkz9s7M for ; Tue, 14 Nov 2017 13:05:14 +1100 (AEDT) Received: by lists.denx.de (Postfix, from userid 105) id 7BA9CC21E52; Tue, 14 Nov 2017 01:59:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lists.denx.de X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.denx.de (localhost [IPv6:::1]) by lists.denx.de (Postfix) with ESMTP id D794DC21E48; Tue, 14 Nov 2017 01:56:09 +0000 (UTC) Received: by lists.denx.de (Postfix, from userid 105) id 4B4E2C21F16; Tue, 14 Nov 2017 01:55:27 +0000 (UTC) Received: from mail-ot0-f172.google.com (mail-ot0-f172.google.com [74.125.82.172]) by lists.denx.de (Postfix) with ESMTPS id A60D8C21E0C for ; Tue, 14 Nov 2017 01:55:21 +0000 (UTC) Received: by mail-ot0-f172.google.com with SMTP id 18so15368615oty.9 for ; Mon, 13 Nov 2017 17:55:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=qnKpwYfOPVB2iAookiAvQrnw6ieE16g4N8wW2wSKOSc=; b=v/9r1tRAYW9QpiRAgZgWBOx3Jfgub3cMAU+U8euAFt3JGmZJ+clr9xYqEeFQwN0M/b DXntM86OPHD1r7/hm0p4L8Rxym99wb9t2Fzogul5nA4IxinuQLJG9LVNUFT5VhZfo56D Uhw8K30BdwDkb6VONj2Oru6gIpXh9V9YaihTsnWKh1eCFVS2agZBAwY+LaPYRjz3NBjf Ag95cH+rE8WI7QViyDQEgzjSHNS4g+Mc/GqVbvvpqcpirrF8tTUEfy3n7xvIjhCWd/zY u88y3VTsqXby1z6nEBajAR2uewXTbpmvsjCbt6XxtgDO4cev8U8mW6xevfKW3dH7qnhH X+RA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=qnKpwYfOPVB2iAookiAvQrnw6ieE16g4N8wW2wSKOSc=; b=MZ1d/t8TRp/qT4J6yZAsZR/9I0/nMscO5YyTjahwj3kK0L2fDPJu9O2izXu2sGv+Bo nOTlwJMai6w9NTg4qOgcQYrjaLyUaxUuFog9/BttGBjr+uUf1mj3lFaKo0lI/8t92xgd pp8byE3PZviufSuNUjarBh3slwlzukWkmqSCNU6gxZ4ne9wmSAfcg+VMt1/JeJi3hZDh tRogxMh5VzChZnwIcrk0UUdlYUfu/+ARl+CWugVQpHdIglx6iGvhueP4fpZjRce/qn1q sOtQMugfC1xkrUHlwqdj1oCpMe8CX2ZzejNjjFxwwytWeKb3+Ld+nxWp4ZH0/97kPgsx kLjw== X-Gm-Message-State: AJaThX6dFhMxBIcOtbcYGr6yr1JAEoHO3S9BP/3QJGYMzDabwlIuwdLq T3+vvJAwLEEfaQPAZUfDckQtPg== X-Google-Smtp-Source: AGs4zMZBM0RJ9VRGhT3Sz/sylRQEjRB1YrRJv5LW99WCAWfDhlcgotPn7uLvMnkJIbb5ePXh2ofnIw== X-Received: by 10.157.9.66 with SMTP id 60mr6689908otp.56.1510624519917; Mon, 13 Nov 2017 17:55:19 -0800 (PST) Received: from kiwi.bld.corp.google.com ([100.101.164.89]) by smtp.gmail.com with ESMTPSA id r184sm7765043oia.40.2017.11.13.17.55.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Nov 2017 17:55:19 -0800 (PST) Received: by kiwi.bld.corp.google.com (Postfix, from userid 121222) id 06D15141AE7; Mon, 13 Nov 2017 18:55:18 -0700 (MST) From: Simon Glass To: U-Boot Mailing List Date: Mon, 13 Nov 2017 18:55:01 -0700 Message-Id: <20171114015506.25673-9-sjg@chromium.org> X-Mailer: git-send-email 2.15.0.448.gf294e3d99a-goog In-Reply-To: <20171114015506.25673-1-sjg@chromium.org> References: <20171114015506.25673-1-sjg@chromium.org> Cc: Tom Rini Subject: [U-Boot] [PATCH 08/13] binman: Support accessing binman tables at run time X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.18 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" Binman construct images consisting of multiple binary files. These files sometimes need to know (at run timme) where their peers are located. For example, SPL may want to know where U-Boot is located in the image, so that it can jump to U-Boot correctly on boot. In general the positions where the binaries end up after binman has finished packing them cannot be known at compile time. One reason for this is that binman does not know the size of the binaries until everything is compiled, linked and converted to binaries with objcopy. To make this work, we add a feature to binman which checks each binary for symbol names starting with '_binman'. These are then decoded to figure out which entry and property they refer to. Then binman writes the value of this symbol into the appropriate binary. With this, the symbol will have the correct value at run time. Macros are used to make this easier to use. As an example, this declares a symbol that will access the 'u-boot-spl' entry to find the 'pos' value (i.e. the position of SPL in the image): binman_sym_declare(unsigned long, u_boot_spl, pos); This converts to a symbol called '_binman_u_boot_spl_prop_pos' in any binary that includes it. Binman then updates the value in that binary, ensuring that it can be accessed at runtime with: ulong u_boot_pos = binman_sym(ulong, u_boot_spl, pos); This assigns the variable u_boot_pos to the position of SPL in the image. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- include/binman_sym.h | 80 ++++++++++++++++++++++++++++++++++ tools/binman/binman.py | 4 +- tools/binman/control.py | 4 +- tools/binman/elf.py | 55 ++++++++++++++++++++++-- tools/binman/elf_test.py | 92 +++++++++++++++++++++++++++++++++++++++- tools/binman/etype/entry.py | 8 ++++ tools/binman/etype/u_boot_spl.py | 6 +++ tools/binman/ftest.py | 19 +++++++++ tools/binman/image.py | 79 ++++++++++++++++++++++++++++++++-- tools/binman/image_test.py | 46 ++++++++++++++++++++ tools/binman/test/53_symbols.dts | 20 +++++++++ 11 files changed, 403 insertions(+), 10 deletions(-) create mode 100644 include/binman_sym.h create mode 100644 tools/binman/image_test.py create mode 100644 tools/binman/test/53_symbols.dts diff --git a/include/binman_sym.h b/include/binman_sym.h new file mode 100644 index 00000000000..3999b26d8d4 --- /dev/null +++ b/include/binman_sym.h @@ -0,0 +1,80 @@ +/* + * Symbol access for symbols set up by binman as part of the build. + * + * This allows C code to access the position of a particular part of the image + * assembled by binman. + * + * Copyright (c) 2017 Google, Inc + * + * SPDX-License-Identifier: GPL-2.0+ + */ + +#ifndef __BINMAN_SYM_H +#define __BINMAN_SYM_H + +#define BINMAN_SYM_MISSING (-1UL) + +#ifdef CONFIG_BINMAN + +/** + * binman_symname() - Internal fnuction to get a binman symbol name + * + * @entry_name: Name of the entry to look for (e.g. 'u_boot_spl') + * @_prop_name: Property value to get from that entry (e.g. 'pos') + * @returns name of the symbol for that entry and property + */ +#define binman_symname(_entry_name, _prop_name) \ + _binman_ ## _entry_name ## _prop_ ## _prop_name + +/** + * binman_sym_declare() - Declare a symbol that will be used at run-time + * + * @_type: Type f the symbol (e.g. unsigned long) + * @entry_name: Name of the entry to look for (e.g. 'u_boot_spl') + * @_prop_name: Property value to get from that entry (e.g. 'pos') + */ +#define binman_sym_declare(_type, _entry_name, _prop_name) \ + _type binman_symname(_entry_name, _prop_name) \ + __attribute__((aligned(4), unused, section(".binman_sym"))) + +/** + * binman_sym_declare_optional() - Declare an optional symbol + * + * If this symbol cannot be provided by binman, an error will not be generated. + * Instead the image will be assigned the value BINMAN_SYM_MISSING. + * + * @_type: Type f the symbol (e.g. unsigned long) + * @entry_name: Name of the entry to look for (e.g. 'u_boot_spl') + * @_prop_name: Property value to get from that entry (e.g. 'pos') + */ +#define binman_sym_declare_optional(_type, _entry_name, _prop_name) \ + _type binman_symname(_entry_name, _prop_name) \ + __attribute__((aligned(4), weak, unused, \ + section(".binman_sym"))) + +/** + * binman_sym() - Access a previously declared symbol + * + * This is used to get the value of a symbol. E.g.: + * + * ulong address = binman_sym(ulong, u_boot_spl, pos); + * + * @_type: Type f the symbol (e.g. unsigned long) + * @entry_name: Name of the entry to look for (e.g. 'u_boot_spl') + * @_prop_name: Property value to get from that entry (e.g. 'pos') + * @returns value of that property (filled in by binman) + */ +#define binman_sym(_type, _entry_name, _prop_name) \ + (*(_type *)&binman_symname(_entry_name, _prop_name)) + +#else /* !BINMAN */ + +#define binman_sym_declare(_type, _entry_name, _prop_name) + +#define binman_sym_declare_optional(_type, _entry_name, _prop_name) + +#define binman_sym(_type, _entry_name, _prop_name) BINMAN_SYM_MISSING + +#endif /* BINMAN */ + +#endif diff --git a/tools/binman/binman.py b/tools/binman/binman.py index aa513962663..1c8e8dbff65 100755 --- a/tools/binman/binman.py +++ b/tools/binman/binman.py @@ -37,6 +37,7 @@ def RunTests(debug): import entry_test import fdt_test import ftest + import image_test import test import doctest @@ -53,7 +54,8 @@ def RunTests(debug): # 'entry' module. suite = unittest.TestLoader().loadTestsFromTestCase(entry_test.TestEntry) suite.run(result) - for module in (ftest.TestFunctional, fdt_test.TestFdt, elf_test.TestElf): + for module in (ftest.TestFunctional, fdt_test.TestFdt, elf_test.TestElf, + image_test.TestImage): suite = unittest.TestLoader().loadTestsFromTestCase(module) suite.run(result) diff --git a/tools/binman/control.py b/tools/binman/control.py index e175e8d41bd..ffa2bbd80f6 100644 --- a/tools/binman/control.py +++ b/tools/binman/control.py @@ -90,8 +90,7 @@ def Binman(options, args): try: tout.Init(options.verbosity) - if options.debug: - elf.debug = True + elf.debug = options.debug try: tools.SetInputDirs(options.indir) tools.PrepareOutputDir(options.outdir, options.preserve) @@ -112,6 +111,7 @@ def Binman(options, args): image.CheckSize() image.CheckEntries() image.ProcessEntryContents() + image.WriteSymbols() image.BuildImage() finally: tools.FinaliseOutputDir() diff --git a/tools/binman/elf.py b/tools/binman/elf.py index 0fb5a4a8ed6..80ff2253f03 100644 --- a/tools/binman/elf.py +++ b/tools/binman/elf.py @@ -19,9 +19,6 @@ debug = False Symbol = namedtuple('Symbol', ['section', 'address', 'size', 'weak']) -# Used for tests which don't have an ELF file to read -ignore_missing_files = False - def GetSymbols(fname, patterns): """Get the symbols from an ELF file @@ -78,3 +75,55 @@ def GetSymbolAddress(fname, sym_name): if not sym: return None return sym.address + +def LookupAndWriteSymbols(elf_fname, entry, image): + """Replace all symbols in an entry with their correct values + + The entry contents is updated so that values for referenced symbols will be + visible at run time. This is done by finding out the symbols positions in + the entry (using the ELF file) and replacing them with values from binman's + data structures. + + Args: + elf_fname: Filename of ELF image containing the symbol information for + entry + entry: Entry to process + image: Image which can be used to lookup symbol values + """ + fname = tools.GetInputFilename(elf_fname) + syms = GetSymbols(fname, ['image', 'binman']) + if not syms: + return + base = syms.get('__image_copy_start') + if not base: + return + for name, sym in syms.iteritems(): + if name.startswith('_binman'): + msg = ("Image '%s': Symbol '%s'\n in entry '%s'" % + (image.GetPath(), name, entry.GetPath())) + offset = sym.address - base.address + if offset < 0 or offset + sym.size > entry.contents_size: + raise ValueError('%s has offset %x (size %x) but the contents ' + 'size is %x' % (entry.GetPath(), offset, + sym.size, entry.contents_size)) + if sym.size == 4: + pack_string = ' 0) + + if __name__ == '__main__': unittest.main() diff --git a/tools/binman/etype/entry.py b/tools/binman/etype/entry.py index 67c57341ca2..5541887d475 100644 --- a/tools/binman/etype/entry.py +++ b/tools/binman/etype/entry.py @@ -198,3 +198,11 @@ class Entry(object): def ProcessContents(self): pass + + def WriteSymbols(self, image): + """Write symbol values into binary files for access at run time + + Args: + image: Image containing the entry + """ + pass diff --git a/tools/binman/etype/u_boot_spl.py b/tools/binman/etype/u_boot_spl.py index 68b0148427d..3720b47fef6 100644 --- a/tools/binman/etype/u_boot_spl.py +++ b/tools/binman/etype/u_boot_spl.py @@ -6,12 +6,18 @@ # Entry-type module for spl/u-boot-spl.bin # +import elf + from entry import Entry from blob import Entry_blob class Entry_u_boot_spl(Entry_blob): def __init__(self, image, etype, node): Entry_blob.__init__(self, image, etype, node) + self.elf_fname = 'spl/u-boot-spl' def GetDefaultFilename(self): return 'spl/u-boot-spl.bin' + + def WriteSymbols(self, image): + elf.LookupAndWriteSymbols(self.elf_fname, self, image) diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 2bee6a168f3..5812ab397cf 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -20,6 +20,7 @@ import binman import cmdline import command import control +import elf import fdt import fdt_util import tools @@ -573,6 +574,8 @@ class TestFunctional(unittest.TestCase): def testImagePadByte(self): """Test that the image pad byte can be specified""" + with open(self.TestFile('bss_data')) as fd: + TestFunctional._MakeInputFile('spl/u-boot-spl', fd.read()) data = self._DoReadFile('21_image_pad.dts') self.assertEqual(U_BOOT_SPL_DATA + (chr(0xff) * 1) + U_BOOT_DATA, data) @@ -889,6 +892,22 @@ class TestFunctional(unittest.TestCase): data = self._DoReadFile('52_u_boot_spl_nodtb.dts') self.assertEqual(U_BOOT_SPL_NODTB_DATA, data[:len(U_BOOT_SPL_NODTB_DATA)]) + def testSymbols(self): + """Test binman can assign symbols embedded in U-Boot""" + elf_fname = self.TestFile('u_boot_binman_syms') + syms = elf.GetSymbols(elf_fname, ['binman', 'image']) + addr = elf.GetSymbolAddress(elf_fname, '__image_copy_start') + self.assertEqual(syms['_binman_u_boot_spl_prop_pos'].address, addr) + + with open(self.TestFile('u_boot_binman_syms')) as fd: + TestFunctional._MakeInputFile('spl/u-boot-spl', fd.read()) + data = self._DoReadFile('53_symbols.dts') + sym_values = struct.pack('_prop_ where is the name of + the entry and is the property to find (e.g. + _binman_u_boot_prop_pos). As a special case, you can append + _any to to have it search for any matching entry. E.g. + _binman_u_boot_any_prop_pos will match entries called u-boot, + u-boot-img and u-boot-nodtb) + optional: True if the symbol is optional. If False this function + will raise if the symbol is not found + msg: Message to display if an error occurs + + Returns: + Value that should be assigned to that symbol, or None if it was + optional and not found + + Raises: + ValueError if the symbol is invalid or not found, or references a + property which is not supported + """ + m = re.match(r'^_binman_(\w+)_prop_(\w+)$', sym_name) + if not m: + raise ValueError("%s: Symbol '%s' has invalid format" % + (msg, sym_name)) + entry_name, prop_name = m.groups() + entry_name = entry_name.replace('_', '-') + entry = self._entries.get(entry_name) + if not entry: + if entry_name.endswith('-any'): + root = entry_name[:-4] + for name in self._entries: + if name.startswith(root): + rest = name[len(root):] + if rest in ['', '-img', '-nodtb']: + entry = self._entries[name] + if not entry: + err = ("%s: Entry '%s' not found in list (%s)" % + (msg, entry_name, ','.join(self._entries.keys()))) + if optional: + print('Warning: %s' % err, file=sys.stderr) + return None + raise ValueError(err) + if prop_name == 'pos': + return entry.pos + else: + raise ValueError("%s: No such property '%s'" % (msg, prop_name)) diff --git a/tools/binman/image_test.py b/tools/binman/image_test.py new file mode 100644 index 00000000000..1b50dda4dca --- /dev/null +++ b/tools/binman/image_test.py @@ -0,0 +1,46 @@ +# +# Copyright (c) 2017 Google, Inc +# Written by Simon Glass +# +# SPDX-License-Identifier: GPL-2.0+ +# +# Test for the image module + +import unittest + +from image import Image +from elf_test import capture_sys_output + +class TestImage(unittest.TestCase): + def testInvalidFormat(self): + image = Image('name', 'node', test=True) + with self.assertRaises(ValueError) as e: + image.LookupSymbol('_binman_something_prop_', False, 'msg') + self.assertIn( + "msg: Symbol '_binman_something_prop_' has invalid format", + str(e.exception)) + + def testMissingSymbol(self): + image = Image('name', 'node', test=True) + image._entries = {} + with self.assertRaises(ValueError) as e: + image.LookupSymbol('_binman_type_prop_pname', False, 'msg') + self.assertIn("msg: Entry 'type' not found in list ()", + str(e.exception)) + + def testMissingSymbolOptional(self): + image = Image('name', 'node', test=True) + image._entries = {} + with capture_sys_output() as (stdout, stderr): + val = image.LookupSymbol('_binman_type_prop_pname', True, 'msg') + self.assertEqual(val, None) + self.assertEqual("Warning: msg: Entry 'type' not found in list ()\n", + stderr.getvalue()) + self.assertEqual('', stdout.getvalue()) + + def testBadProperty(self): + image = Image('name', 'node', test=True) + image._entries = {'u-boot': 1} + with self.assertRaises(ValueError) as e: + image.LookupSymbol('_binman_u_boot_prop_bad', False, 'msg') + self.assertIn("msg: No such property 'bad", str(e.exception)) diff --git a/tools/binman/test/53_symbols.dts b/tools/binman/test/53_symbols.dts new file mode 100644 index 00000000000..980b066eb21 --- /dev/null +++ b/tools/binman/test/53_symbols.dts @@ -0,0 +1,20 @@ +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + pad-byte = <0xff>; + u-boot-spl { + }; + + u-boot { + pos = <20>; + }; + + u-boot-spl2 { + type = "u-boot-spl"; + }; + }; +};