mbox series

[v2,0/1] core: Use of percent_defconfig seems to impact performance

Message ID 20230105015759.63582-1-nhed+buildroot@starry.com
Headers show
Series core: Use of percent_defconfig seems to impact performance | expand

Message

Nevo Hed Jan. 5, 2023, 1:57 a.m. UTC 
Noticing serious performance issues on `make X_defconfig` with growth of

number of external trees.  At 7 trees it is somewhat tolerable but at 8
trees we have seen it take 80-110 seconds.

Numbers for before and after the change in commit itself.

I do not know for sure what the underlying issue is but looking at some
`strace` and `make -d` output it feels like implosion of implicit rules
being evaluated.

Note that after I submitted v1 of this patch it was pointed out to me that
a different patch is already in the works.  But since that neither that
patch nor mine had any seen any movement in months I decided to take a
stab at addressing the feedback I received from Yann on my patch:

> ... it defers the test that the defconfig exists into the shell
> rather than the Makefile

Adding the resets as suggested here
https://lists.gnu.org/archive/html/help-make/2002-11/msg00062.html seem to
help a lot but not as much as the attached patch.

--8<------------
%: %,v
%: RCS/%,v
%: RCS/%
%: s.%
%: SCCS/s.%
--8<------------

Inlined below a script to repro this issue, not sure if this functionality
should be integrated anywhere or just left in the mailing list.  By
default the script runs the scenario from 1 tree to 8 trees.  Running with
param (1-10) will allow a single run with that many trees.


--8<------------
#!/usr/bin/env bash

declare treedirs=()

declare -r cfg="pc_x86_64_bios_defconfig"

function setup_dir {
    for tree in tree{1..10}; do
        local treedir="defconf_test/${tree}"

        mkdir -p "${treedir}/configs"
        touch "${treedir}/Config.in"
        touch "${treedir}/external.mk"
        echo -e "name: ${tree^^}\ndesc: ${tree^}" > "${treedir}/external.desc"

        cp "./configs/${cfg}" "${treedir}/configs/${tree}_${cfg}"

        treedirs+=( "${treedir}" )
    done
}

function run_one {
    local count="${1}"
    local br2_external=$(IFS=':'; echo "${treedirs[*]:0:count}")
    echo "Defconfig with ${count} trees (BR2_EXTERNAL=\"${br2_external}\")"
    /usr/bin/time -f "${count},%e" /bin/make BR2_EXTERNAL="${br2_external}" "${cfg}"
}

# Main
setup_dir
if [ -n "${1}" ]; then
    run_one "${1}"
else
    for i in {1..8}; do
        run_one $i
    done
fi
--8<------------

Changes v1 -> v2:
  - Testing for defconfig existence and error when failing to find it is
    now done at the Makefile level rather than the shell (utilizing `$(or
    ...` and `$(error ...`).

Nevo Hed (1):
  core: Use of percent_defconfig seems to impact performance

 Makefile | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)