powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

On Thu, 04 Aug 2016 12:37:41 +0200
Arnd Bergmann <arnd@arndb.de> wrote:

> On Thursday, August 4, 2016 11:00:49 AM CEST Arnd Bergmann wrote:
> > I tried this
> > 
> > diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> > index b5e40ed86e60..89bca1a25916 100755
> > --- a/scripts/link-vmlinux.sh
> > +++ b/scripts/link-vmlinux.sh
> > @@ -44,7 +44,7 @@ modpost_link()
> >         local objects
> >  
> >         if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
> > -               objects="--whole-archive ${KBUILD_VMLINUX_INIT} ${KBUILD_VMLINUX_MAIN} --no-whole-archive"
> > +               objects="${KBUILD_VMLINUX_INIT} ${KBUILD_VMLINUX_MAIN}"
> >         else
> >                 objects="${KBUILD_VMLINUX_INIT} --start-group ${KBUILD_VMLINUX_MAIN} --end-group"
> >         fi
> > 
> > but that did not seem to change anything, the extra symbols are
> > still there. I have not tried to understand what that actually
> > does, so maybe I misunderstood your suggestion.
> >   
> 
> On a second attempt, I did the same change for vmlinux instead of the
> module (d'oh), and got a link failure instead:
> 
> 
> arch/arm/mm/proc-xscale.o: In function `cpu_xscale_do_resume':
> (.text+0x3d4): undefined reference to `cpu_resume_mmu'
> arch/arm/kernel/setup.o: In function `setup_arch':
> setup.c:(.init.text+0x910): undefined reference to `init_uts_ns'
> kernel/nsproxy.o:(.data+0x4): undefined reference to `init_uts_ns'
> kernel/sched/core.o: In function `update_rq_clock':
> core.c:(.text+0x6d8): undefined reference to `paravirt_steal_rq_enabled'
> core.c:(.text+0x6dc): undefined reference to `pv_time_ops'
> kernel/sched/cputime.o: In function `account_process_tick':
> cputime.c:(.text+0x794): undefined reference to `paravirt_steal_enabled'
> cputime.c:(.text+0x7a0): undefined reference to `pv_time_ops'
> kernel/locking/lockdep.o: In function `save_trace':
> lockdep.c:(.text+0xfe8): undefined reference to `save_stack_trace'
> kernel/module.o: In function `load_module':
> module.c:(.text+0x1b54): undefined reference to `elf_check_arch'
> module.c:(.text+0x2024): undefined reference to `apply_relocate'
> kernel/debug/debug_core.o: In function `kgdb_unregister_io_module':
> debug_core.c:(.text+0x2e4): undefined reference to `kgdb_arch_exit'
> kernel/debug/debug_core.o: In function `kgdb_arch_set_breakpoint':
> debug_core.c:(.text+0x3bc): undefined reference to `arch_kgdb_ops'
> kernel/debug/debug_core.o: In function `dbg_remove_all_break':
> debug_core.c:(.text+0x6d0): undefined reference to `arch_kgdb_ops'
> ...
> 
> However, I also see a link failure in some rare configurations
> with just your patch:
> 
> arch/arm/lib/lib.a(io-acorn.o): In function `outsl':
> (.text+0x38): undefined reference to `printk'
> 
> The problem being a file in a library object that is not referenced,
> but that references another symbol that is not defined
> (CONFIG_PRINTK=n).

The first problem is the existing link system is buggy. I think an
unconditional switch to --whole-archive (at least for modular kernels)
should probably be done anyway. For example, on powerpc when building
with --whole-archive, I have:

+dma_noop_alloc
+dma_noop_free
+dma_noop_map_page
+dma_noop_mapping_error
+dma_noop_map_sg
+dma_noop_ops
+dma_noop_supported
+fdt_add_reservemap_entry
+fdt_begin_node
+fdt_create
+fdt_create_empty_tree
+fdt_end_node
+fdt_errtable
+find_cpio_data
+ioremap_page_range

find_cpio_data is unnecessary and it's a codesize regression to link it.
But dma_noop_ops and ioremap_page_range are exported symbols. If I
reference dma_noop_ops from some random module with otherwise unpatched
kernel:

ERROR: "dma_noop_ops" [drivers/char/bsr.ko] undefined!

The real problem is that our linkage requirements are like a shared
library when we build modular.

We could build a list of exports and make it link objects with those
symbols, to solve this, but IMO that's just wasting lipstick on a pig.
But I will to propose a patch to always use --whole-archive, thin
archives or not, and transition all archs over to it in a few release
cycles. It just works by luck right now.

Why is it a pig? Because having the linker to notice no external
references and just skipping the .o completely is trying to use a hammer
as a scalpel. It's just not a very effective way to eliminate dead code
--  I pulled in only a handful of unneeded functions by switching it.

I mean it is a quick simple feature that probably works well enough with
simple build systems. But not an advanced one that builds almost
everything on demand and also has loadable modules and must act like a
shared library.

Real linker DCE is a valid optimisation that can't be replaced by the
build system of course, but we need to do it properly. Here's what I'm
working on.

It applies on top of the previous patch I sent, plus some powerpc stuff
I'm working on that you should be able to just ignore for another arch.
it's a WIP, but if you can see if it works for arm that would be cool.

It doesn't actually build allyesconfig after this,
ld: .tmp_vmlinux1: Too many sections: 220655 (>= 65280)

But on a more reasonable configuration (ppc64le)
    text      data   bss            dec   filename
11191672   1183536   1923820   14299028   vmlinux
10625528    861895   1919707   13407130	  vmlinux.thin+gc

10M-552K   1M-314K         ~   13M-870K

And it actually boots too, which is fairly astounding considering that
it lost half a meg of code and 1/3 of its data. I'm not completely sure
I've not done something wrong...

Thanks,
Nick

powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

Commit Message

Comments

Patch