| Message ID | 20230206162617.1907911-1-john@metanate.com |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | package/gcc/gcc-final: map debug prefix when reproducible | expand |
John, All, On 2023-02-06 16:26 +0000, John Keeping spake thusly: > When building with BR2_REPRODUCIBLE the toolchain wrapper passes > -fdebug-prefix-map for all packages that are built. But this doesn't > affect the target libraries (like libgcc) built by GCC's build system. > > GCC 4.3 added a configure option to set the debug prefix map for these > libraries, which is used here to avoid encoding potentially > non-reproducible build paths into the debug data. Thanks, this is indeed very interesting. We (briefly) discussed this at the developers day minutes ago, and the question that was brought up was: why is not needed for the first-stage compiler? Indeed, the first-stage compiler builds objects files that are then used by the C library, we would expect the debug symbols for those to also need prefix mapping. Could you look into that, and either pass the option to both the initial and final gcc if needed, or explain in the commit log why it is not needed for the initial gcc, please? In the meantime, I've marked this patch as changes-requested in patchwork. Regards, Yann E. MORIN. > Signed-off-by: John Keeping <john@metanate.com> > --- > package/gcc/gcc-final/gcc-final.mk | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/package/gcc/gcc-final/gcc-final.mk b/package/gcc/gcc-final/gcc-final.mk > index aa883beb7b..5f222a324c 100644 > --- a/package/gcc/gcc-final/gcc-final.mk > +++ b/package/gcc/gcc-final/gcc-final.mk > @@ -108,6 +108,10 @@ else > HOST_GCC_FINAL_CONF_OPTS += --disable-libgomp > endif > > +ifeq ($(BR2_REPRODUCIBLE),y) > +HOST_GCC_FINAL_CONF_OPTS += --with-debug-prefix-map=$(BASE_DIR)=buildroot > +endif > + > # End with user-provided options, so that they can override previously > # defined options. > HOST_GCC_FINAL_CONF_OPTS += \ > -- > 2.39.1 > > _______________________________________________ > buildroot mailing list > buildroot@buildroot.org > https://lists.buildroot.org/mailman/listinfo/buildroot
Hi Yann, On Tue, Feb 07, 2023 at 02:55:03PM +0100, Yann E. MORIN wrote: > On 2023-02-06 16:26 +0000, John Keeping spake thusly: > > When building with BR2_REPRODUCIBLE the toolchain wrapper passes > > -fdebug-prefix-map for all packages that are built. But this doesn't > > affect the target libraries (like libgcc) built by GCC's build system. > > > > GCC 4.3 added a configure option to set the debug prefix map for these > > libraries, which is used here to avoid encoding potentially > > non-reproducible build paths into the debug data. > > Thanks, this is indeed very interesting. We (briefly) discussed this at > the developers day minutes ago, and the question that was brought up > was: why is not needed for the first-stage compiler? > > Indeed, the first-stage compiler builds objects files that are then used > by the C library, we would expect the debug symbols for those to also > need prefix mapping. > > Could you look into that, and either pass the option to both the initial > and final gcc if needed, or explain in the commit log why it is not > needed for the initial gcc, please? I've just sent v2 which moves this to HOST_GCC_COMMON_CONF_OPTS. You're right that some files are affected by the initial compiler - I missed it by focusing on executables (nearly all of which link libgcc.a) when the initial compiler only affects some of glibc's shared objects. I've been testing this with --enable-linker-build-id in the configure options as well to see whether it's possible to enable build IDs for reproducible builds and this change gets a lot closer - the differences are now a handful of files that embed an RPATH pointing to .libs during the build. John
On 07/02/2023 21:05, John Keeping wrote: [snip] > I've been testing this with --enable-linker-build-id in the configure > options as well to see whether it's possible to enable build IDs for > reproducible builds and this change gets a lot closer - the differences > are now a handful of files that embed an RPATH pointing to .libs during > the build. Just so I understand things correctly: those .libs RPATH entries get removed by Buildroot's patchelf run at the end of the build, but they still end up affecting the build-id, correct? And these RPATH entries are of course not affected by file-prefix-map, because that would break them entirely... Regards, Arnout
On Wed, Feb 08, 2023 at 09:05:01AM +0100, Arnout Vandecappelle wrote: > On 07/02/2023 21:05, John Keeping wrote: > [snip] > > I've been testing this with --enable-linker-build-id in the configure > > options as well to see whether it's possible to enable build IDs for > > reproducible builds and this change gets a lot closer - the differences > > are now a handful of files that embed an RPATH pointing to .libs during > > the build. > > Just so I understand things correctly: those .libs RPATH entries get > removed by Buildroot's patchelf run at the end of the build, but they still > end up affecting the build-id, correct? And these RPATH entries are of > course not affected by file-prefix-map, because that would break them > entirely... Yes, that's largely correct - the RPATH entries are removed and are not present in the target. The files are still slightly different as a follow-on effect since the path is included in the string table. (Note that the two build directories I'm using for testing have different *length* names.) $ diff -u <(readelf -hlS build-1/target/bin/mount) <(readelf -hlS build-1abc/target/bin/mount) --- /dev/fd/63 2023-02-08 09:26:16.879386009 +0000 +++ /dev/fd/62 2023-02-08 09:26:16.882719365 +0000 @@ -8,7 +8,7 @@ Type: DYN (Position-Independent Executable file) Machine: ARM Version: 0x1 - Entry point address: 0x44ac + Entry point address: 0x44b0 Start of program headers: 52 (bytes into file) Start of section headers: 41268 (bytes into file) Flags: 0x5000200, Version5 EABI, soft-float ABI @@ -28,18 +28,18 @@ [ 4] .hash HASH 000001ac 0001ac 0004a0 04 A 6 0 4 [ 5] .gnu.hash GNU_HASH 0000064c 00064c 000018 04 A 6 0 4 [ 6] .dynsym DYNSYM 00000664 000664 000a30 10 A 7 3 4 - [ 7] .dynstr STRTAB 00001094 001094 000a69 00 A 0 0 1 - [ 8] .gnu.version VERSYM 00001afe 001afe 000146 02 A 6 0 2 - [ 9] .gnu.version_r VERNEED 00001c44 001c44 000130 00 A 7 3 4 - [10] .rel.dyn REL 00001d74 001d74 000260 08 A 6 0 4 - [11] .rel.plt REL 00001fd4 001fd4 0004a8 08 AI 6 23 4 - [12] .init PROGBITS 0000247c 00247c 00000c 00 AX 0 0 4 - [13] .plt PROGBITS 00002488 002488 000710 04 AX 0 0 4 - [14] .text PROGBITS 00002b98 002b98 0057d4 00 AX 0 0 4 - [15] .fini PROGBITS 0000836c 00836c 000008 00 AX 0 0 4 - [16] .rodata PROGBITS 00008374 008374 00151c 00 A 0 0 4 - [17] .ARM.exidx ARM_EXIDX 00009890 009890 000008 00 AL 14 0 4 - [18] .eh_frame PROGBITS 00009898 009898 000004 00 A 0 0 4 + [ 7] .dynstr STRTAB 00001094 001094 000a6c 00 A 0 0 1 + [ 8] .gnu.version VERSYM 00001b00 001b00 000146 02 A 6 0 2 + [ 9] .gnu.version_r VERNEED 00001c48 001c48 000130 00 A 7 3 4 + [10] .rel.dyn REL 00001d78 001d78 000260 08 A 6 0 4 + [11] .rel.plt REL 00001fd8 001fd8 0004a8 08 AI 6 23 4 + [12] .init PROGBITS 00002480 002480 00000c 00 AX 0 0 4 + [13] .plt PROGBITS 0000248c 00248c 000710 04 AX 0 0 4 + [14] .text PROGBITS 00002b9c 002b9c 0057d4 00 AX 0 0 4 + [15] .fini PROGBITS 00008370 008370 000008 00 AX 0 0 4 + [16] .rodata PROGBITS 00008378 008378 00151c 00 A 0 0 4 + [17] .ARM.exidx ARM_EXIDX 00009894 009894 000008 00 AL 14 0 4 + [18] .eh_frame PROGBITS 0000989c 00989c 000004 00 A 0 0 4 [19] .init_array INIT_ARRAY 0001994c 00994c 000004 04 WA 0 0 4 [20] .fini_array FINI_ARRAY 00019950 009950 000004 04 WA 0 0 4 [21] .data.rel.ro PROGBITS 00019954 009954 0002fc 00 WA 0 0 4 @@ -57,11 +57,11 @@ Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align - EXIDX 0x009890 0x00009890 0x00009890 0x00008 0x00008 R 0x4 + EXIDX 0x009894 0x00009894 0x00009894 0x00008 0x00008 R 0x4 PHDR 0x000034 0x00000034 0x00000034 0x00120 0x00120 R 0x4 INTERP 0x000154 0x00000154 0x00000154 0x00013 0x00013 R 0x1 [Requesting program interpreter: /lib/ld-linux.so.3] - LOAD 0x000000 0x00000000 0x00000000 0x0989c 0x0989c R E 0x10000 + LOAD 0x000000 0x00000000 0x00000000 0x098a0 0x098a0 R E 0x10000 LOAD 0x00994c 0x0001994c 0x0001994c 0x006c0 0x006cc RW 0x10000 DYNAMIC 0x009c50 0x00019c50 0x00019c50 0x00120 0x00120 RW 0x4 NOTE 0x000168 0x00000168 0x00000168 0x00044 0x00044 R 0x4 $ diff -u <(readelf -p 7 build-1/target/bin/mount) <(readelf -p 7 build-1abc/target/bin/mount) --- /dev/fd/63 2023-02-08 09:26:28.439469928 +0000 +++ /dev/fd/62 2023-02-08 09:26:28.439469928 +0000 @@ -171,5 +171,5 @@ [ a0b] MOUNT_2.21 [ a16] MOUNT_2.24 [ a21] MOUNT_2.19 - [ a2d] XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX + [ a2d] XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
diff --git a/package/gcc/gcc-final/gcc-final.mk b/package/gcc/gcc-final/gcc-final.mk index aa883beb7b..5f222a324c 100644 --- a/package/gcc/gcc-final/gcc-final.mk +++ b/package/gcc/gcc-final/gcc-final.mk @@ -108,6 +108,10 @@ else HOST_GCC_FINAL_CONF_OPTS += --disable-libgomp endif +ifeq ($(BR2_REPRODUCIBLE),y) +HOST_GCC_FINAL_CONF_OPTS += --with-debug-prefix-map=$(BASE_DIR)=buildroot +endif + # End with user-provided options, so that they can override previously # defined options. HOST_GCC_FINAL_CONF_OPTS += \
When building with BR2_REPRODUCIBLE the toolchain wrapper passes -fdebug-prefix-map for all packages that are built. But this doesn't affect the target libraries (like libgcc) built by GCC's build system. GCC 4.3 added a configure option to set the debug prefix map for these libraries, which is used here to avoid encoding potentially non-reproducible build paths into the debug data. Signed-off-by: John Keeping <john@metanate.com> --- package/gcc/gcc-final/gcc-final.mk | 4 ++++ 1 file changed, 4 insertions(+)