Message ID | 20210218103256.3895561-1-i.maximets@ovn.org |
---|---|
State | Accepted |
Headers | show |
Series | [ovs-dev] ci: Use parrallel build for distcheck. | expand |
On Thu, Feb 18, 2021 at 6:33 PM Ilya Maximets <i.maximets@ovn.org> wrote: > > 'distcheck' target builds the whole OVS, but for some reason we're > running it in a single-threaded mode. Running in parallel saves > a couple of minutes. > > Signed-off-by: Ilya Maximets <i.maximets@ovn.org> > --- > .ci/linux-build.sh | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh > index 581a8888b..977449350 100755 > --- a/.ci/linux-build.sh > +++ b/.ci/linux-build.sh > @@ -244,7 +244,7 @@ if [ "$TESTSUITE" ]; then > configure_ovs > > export DISTCHECK_CONFIGURE_FLAGS="$OPTS" > - if ! make distcheck CFLAGS="${CFLAGS_FOR_OVS}" \ > + if ! make distcheck -j4 CFLAGS="${CFLAGS_FOR_OVS}" \ why not use $(nproc)? > TESTSUITEFLAGS=-j4 RECHECK=yes; then and -j4 --> $(nproc) ? > # testsuite.log is necessary for debugging. > cat */_build/sub/tests/testsuite.log > -- > 2.26.2 > > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
On 2/18/21 1:20 PM, Tonghao Zhang wrote: > On Thu, Feb 18, 2021 at 6:33 PM Ilya Maximets <i.maximets@ovn.org> wrote: >> >> 'distcheck' target builds the whole OVS, but for some reason we're >> running it in a single-threaded mode. Running in parallel saves >> a couple of minutes. >> >> Signed-off-by: Ilya Maximets <i.maximets@ovn.org> >> --- >> .ci/linux-build.sh | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh >> index 581a8888b..977449350 100755 >> --- a/.ci/linux-build.sh >> +++ b/.ci/linux-build.sh >> @@ -244,7 +244,7 @@ if [ "$TESTSUITE" ]; then >> configure_ovs >> >> export DISTCHECK_CONFIGURE_FLAGS="$OPTS" >> - if ! make distcheck CFLAGS="${CFLAGS_FOR_OVS}" \ >> + if ! make distcheck -j4 CFLAGS="${CFLAGS_FOR_OVS}" \ > why not use $(nproc)? >> TESTSUITEFLAGS=-j4 RECHECK=yes; then > and -j4 --> $(nproc) ? CI systems provides 2 CPUs, so nproc is equal to 2. Both build and tests are IO intensive and not able to fully utilize CPU resources while waiting on IO, so we're creating more threads than CPUs. From my experience 2x is enough, higher parallelism doesn't usually give any improvement. We can replace 4 with 'nproc * 2', but I do not expect changes in number of available CPU cores in CI, so it's not very important. >> # testsuite.log is necessary for debugging. >> cat */_build/sub/tests/testsuite.log >> -- >> 2.26.2
Ilya Maximets <i.maximets@ovn.org> writes: > 'distcheck' target builds the whole OVS, but for some reason we're > running it in a single-threaded mode. Running in parallel saves > a couple of minutes. > > Signed-off-by: Ilya Maximets <i.maximets@ovn.org> > --- Cool - from what I can tell of the jobs, it shaves a couple minutes off the build time (looking at the 'linux gcc test' builds that run the distcheck). Acked-by: Aaron Conole <aconole@redhat.com>
On 2/18/21 2:51 PM, Aaron Conole wrote: > Ilya Maximets <i.maximets@ovn.org> writes: > >> 'distcheck' target builds the whole OVS, but for some reason we're >> running it in a single-threaded mode. Running in parallel saves >> a couple of minutes. >> >> Signed-off-by: Ilya Maximets <i.maximets@ovn.org> >> --- > > Cool - from what I can tell of the jobs, it shaves a couple minutes off > the build time (looking at the 'linux gcc test' builds that run the > distcheck). > > Acked-by: Aaron Conole <aconole@redhat.com> > Thanks! Applied to master and a couple of older branches. Best regards, Ilya Maximets.
diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index 581a8888b..977449350 100755 --- a/.ci/linux-build.sh +++ b/.ci/linux-build.sh @@ -244,7 +244,7 @@ if [ "$TESTSUITE" ]; then configure_ovs export DISTCHECK_CONFIGURE_FLAGS="$OPTS" - if ! make distcheck CFLAGS="${CFLAGS_FOR_OVS}" \ + if ! make distcheck -j4 CFLAGS="${CFLAGS_FOR_OVS}" \ TESTSUITEFLAGS=-j4 RECHECK=yes; then # testsuite.log is necessary for debugging. cat */_build/sub/tests/testsuite.log
'distcheck' target builds the whole OVS, but for some reason we're running it in a single-threaded mode. Running in parallel saves a couple of minutes. Signed-off-by: Ilya Maximets <i.maximets@ovn.org> --- .ci/linux-build.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)