Message ID | CACRpkdaTv4NBrRXeJE6dVi4W88+LZdsTCSzTHi0vdcKMW3Qfmw@mail.gmail.com |
---|---|
State | New |
Headers | show |
Series | [GIT,PULL] pin control bulk changes for v4.16 | expand |
On Fri, Feb 2, 2018 at 5:28 AM, Linus Walleij <linus.walleij@linaro.org> wrote: > > here is the big slew of changes in pin control for the v4.16 cycle. So I pulled this, and then was surprised by how *everything* got rebuilt even though it only touched pinctl files. The reason for that seems to be that the pinctl pull got me this: include/linux/pinctrl/devinfo.h | 2 + and in include/linux/device.h we have #include <linux/pinctrl/devinfo.h> so pretty much *every* driver ends up depending on that silly two-line change. I *think* that the only reason that happens is because of this: #ifdef CONFIG_PINCTRL struct dev_pin_info *pins; #endif and honestly, that could be trivially done by just having a forward declaration, replacing the pinctrl/devinfo.h header include entirely. Ironically, that two-line change to pinctrl/devinfo.h was a forward declaration in the exact reverse direction: +struct device; + so I would really prefer to speed up recompiles and just generally try to avoid horrible header file inclusion by doing the same thing in <linux/device.h>, adding just that struct dev_pin_info; declaration, and removing the <linux/pinctrl/devinfo.h> include. Hmm? I also wonder if there are any automated tools that try to find these kinds of crazy things. I suspect a lot of our build times is the poor compiler just reading and parsing header files over and over again, and a lot of them are probably not needed. A year ago, Ingo did patches limit some of the header file issues for the core headers (<linux/sched.h> in particular). Maybe he had tooling? Ingo? Linus -- To unsubscribe from this list: send the line "unsubscribe linux-gpio" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Feb 2, 2018 at 2:56 PM, Linus Torvalds <torvalds@linux-foundation.org> wrote: > > so I would really prefer to speed up recompiles and just generally try > to avoid horrible header file inclusion by doing the same thing in > <linux/device.h>, adding just that > > struct dev_pin_info; > > declaration, and removing the <linux/pinctrl/devinfo.h> include. It turns out that some pinctl users seem to depend on this broken situation., with at least drivers/pinctrl/core.c drivers/media/platform/sti/c8sectpfe/c8sectpfe-core.c drivers/pinctrl/pinctrl-ocelot.c drivers/pinctrl/bcm/pinctrl-iproc-gpio.c expecting to magically get some of the pinctrl function declarations not through some pinctrl header file, but just from <linux/device.h>. Adding that include to <linux/pinctrl/pinctrl.h> would seem to make those happy and make 'allmodconfig' build for me. But I'm only testing x86-64. Can somebody test at least arm too? Stupid patch attached. I don't know how much this helps the insane dependency hell for <linux/pinctrl/devinfo.h>, but it's bound to help _some_. Comments? Linus include/linux/device.h | 2 +- include/linux/pinctrl/pinctrl.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/include/linux/device.h b/include/linux/device.h index f649fc0c2571..b093405ed525 100644 --- a/include/linux/device.h +++ b/include/linux/device.h @@ -20,7 +20,6 @@ #include <linux/compiler.h> #include <linux/types.h> #include <linux/mutex.h> -#include <linux/pinctrl/devinfo.h> #include <linux/pm.h> #include <linux/atomic.h> #include <linux/ratelimit.h> @@ -41,6 +40,7 @@ struct fwnode_handle; struct iommu_ops; struct iommu_group; struct iommu_fwspec; +struct dev_pin_info; struct bus_attribute { struct attribute attr; diff --git a/include/linux/pinctrl/pinctrl.h b/include/linux/pinctrl/pinctrl.h index 5e45385c5bdc..8f5dbb84547a 100644 --- a/include/linux/pinctrl/pinctrl.h +++ b/include/linux/pinctrl/pinctrl.h @@ -18,6 +18,7 @@ #include <linux/list.h> #include <linux/seq_file.h> #include <linux/pinctrl/pinctrl-state.h> +#include <linux/pinctrl/devinfo.h> struct device; struct pinctrl_dev;
On Fri, Feb 2, 2018 at 4:44 PM, Linus Torvalds <torvalds@linux-foundation.org> wrote: > > Stupid patch attached. I don't know how much this helps the insane > dependency hell for <linux/pinctrl/devinfo.h>, but it's bound to help > _some_. Testing it, that patch definitely cuts down on recompiles after touch include/linux/pinctrl/devinfo.h a lot. It still ends up rebuilding a fair amount of odd drivers, but now the files it rebuilds at least make _some_ sense. It used to really rebuild just about everything (because pretty much everything includes <linux/device.h>). Now it rebuilds various snd/soc files,gpio stuff and mmc/mfc stuff. I'm sure it could be improved upon still, but I think this is already a fairly noticeable improvement. One odd header include down. Ten million to go. Linus -- To unsubscribe from this list: send the line "unsubscribe linux-gpio" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
* Linus Torvalds <torvalds@linux-foundation.org> wrote: > I also wonder if there are any automated tools that try to find these > kinds of crazy things. I suspect a lot of our build times is the poor > compiler just reading and parsing header files over and over again, > and a lot of them are probably not needed. Yes. I'd guesstimate that in a typical defconfig kernel build the compiler is building at least 10x as much as it should with a cleaner header file layout, based on preprocessed source code file sizes. Interestingly the .i file linecount difference isn't all that large between allmodconfig and allnoconfig kernels - which I think is further proof of our 'header spaghetti bloat' problems. While central files like fork.c or sched/*.c are expected to have a lot of dependencies, we also have a lot of bloat if we build much more isolated, standalone core kernel functionality: # allmodconfig: triton:~/tip> wc -l kernel/task_work.i 43522 kernel/task_work.i # allnoconfig: triton:~/tip> wc -l kernel/task_work.i 37123 kernel/task_work.i # source code size: triton:~/tip> wc -l kernel/task_work.c 118 kernel/task_work.c We are bringing in 37 KLOC of headers to build a 0.1 KLOC .c file ... > A year ago, Ingo did patches limit some of the header file issues for > the core headers (<linux/sched.h> in particular). Maybe he had > tooling? Ingo? No, unfortunately I didn't use much tooling: I only used simple manual tools like 'grep' and small ad-hoc shell scripts to discover some of the deeper dependencies (long lost - nor was there any real value in them). What I relied on mostly was randconfig build coverage. In that sched.h split-up effort a year ago I literally removed/moved the prototypes and header files one by one and tried to see what breaks. If the breakage was too widespread I tried to grep. But based on the sched.h experiment I do think our kernel build times could be significantly improved by organizing the headers better. Splitting up sched.h also improved readability and maintainability, so it was a win-win all around. With a more aggressive reorganization of our header architecture I believe we could achieve a more than 5x improvement in kernel build times (!) - but that would involve some trade-offs for header maintainability: a finer grained hierarchy is somewhat harder to maintain. With extreme measures that would involve runtime performance trade-offs as well (to get rid of excessive inlining cross-dependencies) we could possibly achieve a 30x improvement in kernel compilation times: the build would be link time and build parallelism limited on most systems. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-gpio" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sat, Feb 3, 2018 at 1:51 AM, Linus Torvalds <torvalds@linux-foundation.org> wrote: > On Fri, Feb 2, 2018 at 4:44 PM, Linus Torvalds > <torvalds@linux-foundation.org> wrote: >> >> Stupid patch attached. I don't know how much this helps the insane >> dependency hell for <linux/pinctrl/devinfo.h>, but it's bound to help >> _some_. > > Testing it, that patch definitely cuts down on recompiles after > > touch include/linux/pinctrl/devinfo.h > > a lot. Hey very nice. Sorry I was offline this weekend and didn't provide much feedback. Indeed it is smarter to forward-declare struct dev_pin_info. I rebuilt my platforms with the mainline and all is working just fine of course. > It still ends up rebuilding a fair amount of odd drivers, but now the > files it rebuilds at least make _some_ sense. Yeah :/ I guess the lesson learned is that when I push stuff into device core like this, it needs to be done as exquisitely as cache-aligned structs because of the overall impact on the build systems. > One odd header include down. Ten million to go. Sorry about contributing to that :( Another thing that comes to mind was Paul Gortmaker's tedious work to remove #include <linux/module.h> from drivers that cannot be built as modules that happened in the last few months. My subsystems had a few of those and it visibly impacted build time. As usual clean and consistent code is code that compiles quickly... We definitely need some better tooling to find these things, using Ingo's head and your occasional frustration is not going to scale. Julia: do you have ideas on tooling that can loosen #include deps and advise on when to replace #includes with forward declarations of structs (etc) to bring down rebuild-triggering dependencies? Yours, Linus Walleij -- To unsubscribe from this list: send the line "unsubscribe linux-gpio" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, 5 Feb 2018, Linus Walleij wrote: > On Sat, Feb 3, 2018 at 1:51 AM, Linus Torvalds > <torvalds@linux-foundation.org> wrote: > > On Fri, Feb 2, 2018 at 4:44 PM, Linus Torvalds > > <torvalds@linux-foundation.org> wrote: > >> > >> Stupid patch attached. I don't know how much this helps the insane > >> dependency hell for <linux/pinctrl/devinfo.h>, but it's bound to help > >> _some_. > > > > Testing it, that patch definitely cuts down on recompiles after > > > > touch include/linux/pinctrl/devinfo.h > > > > a lot. > > Hey very nice. Sorry I was offline this weekend and didn't provide > much feedback. > > Indeed it is smarter to forward-declare struct dev_pin_info. > > I rebuilt my platforms with the mainline and all is working just fine > of course. > > > It still ends up rebuilding a fair amount of odd drivers, but now the > > files it rebuilds at least make _some_ sense. > > Yeah :/ > > I guess the lesson learned is that when I push stuff into device > core like this, it needs to be done as exquisitely as cache-aligned > structs because of the overall impact on the build systems. > > > One odd header include down. Ten million to go. > > Sorry about contributing to that :( > > Another thing that comes to mind was Paul Gortmaker's tedious > work to remove #include <linux/module.h> from drivers that cannot > be built as modules that happened in the last few months. My > subsystems had a few of those and it visibly impacted build > time. As usual clean and consistent code is code that compiles > quickly... > > We definitely need some better tooling to find these things, > using Ingo's head and your occasional frustration is not going to > scale. > > Julia: do you have ideas on tooling that can loosen #include > deps and advise on when to replace #includes with forward > declarations of structs (etc) to bring down rebuild-triggering > dependencies? Could you explain more? Is the point that you want to remove an include but it has one declaration that you need, and so you want to bring it down into the .c file? Would the need for that actually indicate that the include file is designed incorrectly? Can one assume that each include is self contained, ie it includes the things that it needs and does not rely on the .c file having included other things beforehand? julia -- To unsubscribe from this list: send the line "unsubscribe linux-gpio" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Feb 5, 2018 at 10:27 AM, Julia Lawall <julia.lawall@lip6.fr> wrote: > On Mon, 5 Feb 2018, Linus Walleij wrote: >> We definitely need some better tooling to find these things, >> using Ingo's head and your occasional frustration is not going to >> scale. >> >> Julia: do you have ideas on tooling that can loosen #include >> deps and advise on when to replace #includes with forward >> declarations of structs (etc) to bring down rebuild-triggering >> dependencies? > > Could you explain more? Is the point that you want to remove an include > but it has one declaration that you need, and so you want to bring it down > into the .c file? Would the need for that actually indicate that the > include file is designed incorrectly? > > Can one assume that each include is self contained, ie it includes the > things that it needs and does not rely on the .c file having included > other things beforehand? Usually (in my limited experience, le's see what Ingo and Torvalds say) the problem manifests mainly in include files including other include files. So say <linux/foo.h>: #include <linux/bar.h> struct foo { struct bar *barp; }; Since this is only putting a pointer inside its struct and doesn't need the information on the whole structs, as the size of a pointer is well known we can reduce it to: struct bar; struct foo { struct bar *barp; }; And thus as <linux/bar.h> is not even included, it can change all it wants, our foo.h include file is not affected, neither will any driver just casually #including <linux/foo.h> need to be rebuilt. This type of case (and variations on this theme) is the reason we put a bunch of forward-declarations in kernel .h-files just to break dependencies to stuff just referenced by pointer. There is a counter-pattern saying "files should #include the headers prototypes, structs (etc) it uses" that drives a truck through this approach. But IMO when done properly, this forward-declaring approach is quite readable. I have very limited idea of where, whether in the preprocessor or the compiler itself, the decision to treat struct bar *barp as "just some pointer" happens, but it is a real neat trick, the dependency chain is broken in CPP AFAICT anyways, and cuts down the rebuilds. Yours, Linus Walleij -- To unsubscribe from this list: send the line "unsubscribe linux-gpio" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, 5 Feb 2018, Linus Walleij wrote: > On Mon, Feb 5, 2018 at 10:27 AM, Julia Lawall <julia.lawall@lip6.fr> wrote: > > On Mon, 5 Feb 2018, Linus Walleij wrote: > > >> We definitely need some better tooling to find these things, > >> using Ingo's head and your occasional frustration is not going to > >> scale. > >> > >> Julia: do you have ideas on tooling that can loosen #include > >> deps and advise on when to replace #includes with forward > >> declarations of structs (etc) to bring down rebuild-triggering > >> dependencies? > > > > Could you explain more? Is the point that you want to remove an include > > but it has one declaration that you need, and so you want to bring it down > > into the .c file? Would the need for that actually indicate that the > > include file is designed incorrectly? > > > > Can one assume that each include is self contained, ie it includes the > > things that it needs and does not rely on the .c file having included > > other things beforehand? > > Usually (in my limited experience, le's see what Ingo and Torvalds > say) the problem manifests mainly in include files including other > include files. > > So say <linux/foo.h>: > > #include <linux/bar.h> > > struct foo { > struct bar *barp; > }; > > Since this is only putting a pointer inside its struct and doesn't > need the information on the whole structs, as the size of a pointer > is well known we can reduce it to: > > struct bar; > > struct foo { > struct bar *barp; > }; > > And thus as <linux/bar.h> is not even included, it can change > all it wants, our foo.h include file is not affected, neither will > any driver just casually #including <linux/foo.h> need to be > rebuilt. > > This type of case (and variations on this theme) is the reason > we put a bunch of forward-declarations in kernel .h-files > just to break dependencies to stuff just referenced by pointer. > > There is a counter-pattern saying "files should #include the > headers prototypes, structs (etc) it uses" that drives a truck > through this approach. But IMO when done properly, this > forward-declaring approach is quite readable. > > I have very limited idea of where, whether in the preprocessor > or the compiler itself, the decision to treat struct bar *barp > as "just some pointer" happens, but it is a real neat trick, the > dependency chain is broken in CPP AFAICT anyways, and cuts > down the rebuilds. OK, thanks for the explanation. It seems like a very interesting problem. I will think about it and see if something can be done. It seems like it may need careful checking by a human, due to macros, ifdefs, etc, but perhaps it can at least be heplful to narrow down the opportunities. julia -- To unsubscribe from this list: send the line "unsubscribe linux-gpio" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Linus Torvalds > Sent: 02 February 2018 22:57 ... > I also wonder if there are any automated tools that try to find these > kinds of crazy things. I suspect a lot of our build times is the poor > compiler just reading and parsing header files over and over again, > and a lot of them are probably not needed. I've counted system calls during a NetBSD kernel build, I imagine Linux is much the same. Most of the calls were open(), and most of those failing opens. I suspected that most came from searching the -I path to find headers. Build over NFS and the cost is even more significant (every directory name in the path (used to) require an NFS message exchange). David
On Mon, Feb 5, 2018 at 2:09 AM, David Laight <David.Laight@aculab.com> wrote: > From: Linus Torvalds >> Sent: 02 February 2018 22:57 > ... >> I also wonder if there are any automated tools that try to find these >> kinds of crazy things. I suspect a lot of our build times is the poor >> compiler just reading and parsing header files over and over again, >> and a lot of them are probably not needed. > > I've counted system calls during a NetBSD kernel build, I imagine Linux is > much the same. > Most of the calls were open(), and most of those failing opens. > I suspected that most came from searching the -I path to find headers. > Build over NFS and the cost is even more significant (every directory > name in the path (used to) require an NFS message exchange). I suspect that Linux is actually very _different_ in this area, because Linux has a very extensive name cache for pathname lookup that also captures negative path component entries. End result: opening a file - whether it exists or not - doesn't actually go down to the filesystem at all when things are cached. My kernel profiles also show that very clearly, there's absolutely no filesystem component to the build at all (but there is a noticeable VFS component to it, and __d_lookup_rcu is generally one of the hottest kernel functions along with the system call entry/exit code). Now, on NFS the caching will be a bit less effective because we have a timeout that forces re-validation of the caches, but even there at least the "look up the same header pathname" will be captured very well. So the most common case - of doing the include path lookup for all those really core header files that _everbody_ ends up including - ends up being captured pretty much 100% in the caches even on NFS. On a local filesystem, as long as you have enough RAM, those lookups will be cached even across multiple kernel compiles. At least if you are like me, and all you do all day long is build the kernel ;) Linus -- To unsubscribe from this list: send the line "unsubscribe linux-gpio" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Feb 5, 2018 at 8:55 AM, Linus Torvalds <torvalds@linux-foundation.org> wrote: > > End result: opening a file - whether it exists or not - doesn't > actually go down to the filesystem at all when things are cached. My > kernel profiles also show that very clearly, there's absolutely no > filesystem component to the build at all (but there is a noticeable > VFS component to it, and __d_lookup_rcu is generally one of the > hottest kernel functions along with the system call entry/exit code). Note that when I do kernel profiles of kernel builds, I do it mostly for the "everything is already built" case, so the real footprint for much of my profiles is actually mostly "make" doing millions of open/stat calls. Because once you actually build things, the kernel is almost not noticeable any more. It's all gcc. And people always say that it's optimizations that are expensive, but from the profiling I've done of user space, a _lot_ of time is spent in just parsing and reading the data. In fact, having just re-done this, the top function in profiling is "_cpp_lex_token()" at 3.4% of overall time for my test kernel build. That matches my experience from sparse: the real overhead in a compiler is just the stupid lexing/parsing. Cache misses galore, and there's nothing really smart you can do about it. Once you get to optimization, you can do smart things like hash the SSA representation to do CSE cheaply etc clever data structures. But lexing and parsing the tree is reading text and allocating and generating the internal representation, and it's just "work". Lots of it. And that is why trying to avoid unnecessary header includes matters so much. Because the front-end really does matter for compiler performance. (And it's at least partly why C++ is such a pain to compile, and why C++ people want pre-compiled headers etc. You can't just do a forward declaration of a struct type, and you get header inclusion from hell when you have "clever" classes and inheritance etc. C++ build times tend to be really nasty as a result). Linus -- To unsubscribe from this list: send the line "unsubscribe linux-gpio" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Re: [GIT PULL] pin control bulk changes for v4.16] On 05/02/2018 (Mon 10:27) Julia Lawall wrote: > > > On Mon, 5 Feb 2018, Linus Walleij wrote: > [...] > > > > Another thing that comes to mind was Paul Gortmaker's tedious > > work to remove #include <linux/module.h> from drivers that cannot > > be built as modules that happened in the last few months. I also took aim at headers including giant headers ; see below. [...] > Could you explain more? Is the point that you want to remove an include > but it has one declaration that you need, and so you want to bring it down > into the .c file? Would the need for that actually indicate that the > include file is designed incorrectly? I put a bit of an explanation in this commit: commit d47529b2e9fe0ec2eb1f072afad8849f52e385c4 Author: Paul Gortmaker <paul.gortmaker@windriver.com> Date: Mon Sep 12 18:16:31 2016 -0400 gpio: don't include module.h in shared driver header Most shared headers in include/linux don't need to know what the internals of a struct module are; all they care about is that it is a struct and hence they may require a pointer to one. The advantage in this is that module.h is including a lot of stuff itself, and an otherwise empty C file that just contains module.h will result in ~750kB from CPP (compared to say 12kB from init.h) So we have approximately 50 instances of "struct module;" in the various include/linux headers already that help us keep module.h out of other headers; here we do the same for gpio. Cc: Linus Walleij <linus.walleij@linaro.org> Cc: Alexandre Courbot <gnurou@gmail.com> Cc: linux-gpio@vger.kernel.org Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com> Signed-off-by: Linus Walleij <linus.walleij@linaro.org> I'm sure I've got more of these type of commits sitting around locally, unfortunately there have been other "interesting" things going on in the linux world in the last month that have prevented me from touching any of them. :-/ Paul. -- > > Can one assume that each include is self contained, ie it includes the > things that it needs and does not rely on the .c file having included > other things beforehand? > > julia -- To unsubscribe from this list: send the line "unsubscribe linux-gpio" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html