"make check" times
diff mbox series

Message ID xnh87hllo7.fsf@greed.delorie.com
State New
Headers show
Series
  • "make check" times
Related show

Commit Message

DJ Delorie July 19, 2019, 10:11 p.m. UTC
While digging for some low-hanging fruit in "make" times, I did this:


That one instance of "head" is called over 6000 times per "make
check", and as it's the only non-builtin in that script, it adds about
11 seconds of overhead compared to just reading that one line with
builtins.

My question here is: how much of a time savings is worth the
complexity of said savings?

Comments

Carlos O'Donell July 20, 2019, 12:05 a.m. UTC | #1
On 7/19/19 6:11 PM, DJ Delorie wrote:
> While digging for some low-hanging fruit in "make" times, I did this:
> 
> diff --git a/scripts/merge-test-results.sh b/scripts/merge-test-results.sh
> index 919bbae253..7088ef6996 100755
> --- a/scripts/merge-test-results.sh
> +++ b/scripts/merge-test-results.sh
> @@ -35,7 +35,11 @@ case $type in
>       subdir=${subdir:+$subdir/}
>       for t in "$@"; do
>         if [ -s "$objpfx$t.test-result" ]; then
> -	head -n1 "$objpfx$t.test-result"
> +	  #head -n1 "$objpfx$t.test-result"
> +	  exec 6<"$objpfx$t.test-result"
> +	  read line <&6
> +	  echo $line
> +	  exec 6<&-
>         else
>   	echo "UNRESOLVED: $subdir$t"
>         fi
> 
> That one instance of "head" is called over 6000 times per "make
> check", and as it's the only non-builtin in that script, it adds about
> 11 seconds of overhead compared to just reading that one line with
> builtins.
> 
> My question here is: how much of a time savings is worth the
> complexity of said savings?

One needs to tackle the problem the other way around.

* First determine those things we cannot change and remove them
   form the accounting.
   - We cannot avoid compiling each file.

* Whatever is left over after removing the things we can't
   avoid doing is our "overhead".

* Sort the "overhead" by the amount of time spent, and find ways
   to reduce it.

If the "overhead" is small, then there isn't much value in adding
complexity.

If the "overhead" is large, we may want to rewrite the whole build
system to get that time back.

Does that make sense?

I can't answer your question without knowing how much is 11 seconds
out of the total time taken.

If the build took 20 seconds to complete, then yes this is a valuable
change ;-)

Patch
diff mbox series

diff --git a/scripts/merge-test-results.sh b/scripts/merge-test-results.sh
index 919bbae253..7088ef6996 100755
--- a/scripts/merge-test-results.sh
+++ b/scripts/merge-test-results.sh
@@ -35,7 +35,11 @@  case $type in
     subdir=${subdir:+$subdir/}
     for t in "$@"; do
       if [ -s "$objpfx$t.test-result" ]; then
-	head -n1 "$objpfx$t.test-result"
+	  #head -n1 "$objpfx$t.test-result"
+	  exec 6<"$objpfx$t.test-result"
+	  read line <&6
+	  echo $line
+	  exec 6<&-
       else
 	echo "UNRESOLVED: $subdir$t"
       fi