| Message ID | 1453991009-32736-3-git-send-email-alex.bennee@linaro.org |
|---|---|
| State | New |
| Headers | show |
On Thu, Jan 28, 2016 at 02:23:28PM +0000, Alex Bennée wrote: > We only ran make check once before it used to be an unreliable target. > It was only a stop gap measure and we should be able to revert it now. > This also stops us needing a large all-MMU build. > > We disable "make check" for a couple of the extra config targets which > are currently broken. > > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> So, in general I like the idea of running make check more widely. However.. I was wondering - what's the rationale for having separate matrix builds for each target (or small group) rather than just doing one build with all the targets? I can't see any obvious benefit to splitting the build that way, but it does increase the total build time significantly - and will do so rather more so with make check added. > --- > .travis.yml | 15 ++++++++------- > 1 file changed, 8 insertions(+), 7 deletions(-) > > diff --git a/.travis.yml b/.travis.yml > index 4a0c23a..16be23f 100644 > --- a/.travis.yml > +++ b/.travis.yml > @@ -40,7 +40,7 @@ notifications: > on_failure: always > env: > global: > - - TEST_CMD="" > + - TEST_CMD="make check" > - EXTRA_CONFIG="" > matrix: > # Group major targets together with their linux-user counterparts > @@ -73,17 +73,14 @@ script: > matrix: > # We manually include a number of additional build for non-standard bits > include: > - # Make check target (we only do this once) > - - env: > - - TARGETS=alpha-softmmu,arm-softmmu,aarch64-softmmu,cris-softmmu,i386-softmmu,x86_64-softmmu,m68k-softmmu,microblaze-softmmu,microblazeel-softmmu,mips-softmmu,mips64-softmmu,mips64el-softmmu,mipsel-softmmu,or32-softmmu,ppc-softmmu,ppc64-softmmu,ppcemb-softmmu,s390x-softmmu,sh4-softmmu,sh4eb-softmmu,sparc-softmmu,sparc64-softmmu,unicore32-softmmu,lm32-softmmu,moxie-softmmu,tricore-softmmu,xtensa-softmmu,xtensaeb-softmmu > - TEST_CMD="make check" > - compiler: gcc > # Debug related options > - env: TARGETS=i386-softmmu,x86_64-softmmu > EXTRA_CONFIG="--enable-debug" > compiler: gcc > + # We currently disable "make check" > - env: TARGETS=i386-softmmu,x86_64-softmmu > EXTRA_CONFIG="--enable-debug --enable-tcg-interpreter" > + TEST_CMD="" > compiler: gcc > # Disable a few of the optional features > - env: TARGETS=i386-softmmu,x86_64-softmmu > @@ -104,11 +101,15 @@ matrix: > - env: TARGETS=i386-softmmu,x86_64-softmmu > EXTRA_CONFIG="--enable-trace-backends=simple" > compiler: gcc > + # We currently disable "make check" > - env: TARGETS=i386-softmmu,x86_64-softmmu > EXTRA_CONFIG="--enable-trace-backends=ftrace" > + TEST_CMD="" > compiler: gcc > + # We currently disable "make check" > - env: TARGETS=i386-softmmu,x86_64-softmmu > - EXTRA_CONFIG="--enable-trace-backends=ust" > + EXTRA_CONFIG="--enable-trace-backends=ust" > + TEST_CMD="" > compiler: gcc > - env: TARGETS=i386-softmmu,x86_64-softmmu > EXTRA_CONFIG="--enable-modules"
David Gibson <david@gibson.dropbear.id.au> writes: > On Thu, Jan 28, 2016 at 02:23:28PM +0000, Alex Bennée wrote: >> We only ran make check once before it used to be an unreliable target. >> It was only a stop gap measure and we should be able to revert it now. >> This also stops us needing a large all-MMU build. >> >> We disable "make check" for a couple of the extra config targets which >> are currently broken. >> >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > > So, in general I like the idea of running make check more widely. > > However.. I was wondering - what's the rationale for having separate > matrix builds for each target (or small group) rather than just doing > one build with all the targets? Each individual part of the matrix can be run in parallel with the others so it makes sense to keep the build component small (as each softmmu target rebuilds a significant chunk of the build). Having said that there is a fair amount of repetition as we are repeating all the generic qtests each time just so we can run the extra ${TARGET}-qtest binaries. Travis does has an option for using ccache so it might be worth experimenting with that to see if things are improved. > I can't see any obvious benefit to splitting the build that way, but > it does increase the total build time significantly - and will do so > rather more so with make check added. Elapsed and total are the ones to look at: https://travis-ci.org/stsquad/qemu/builds/105401126 vs https://travis-ci.org/qemu/qemu/builds/105711606 However it looks like Travis are having scaling growing pains because there "old style" VM approach is running a lot faster than it used to. > >> --- >> .travis.yml | 15 ++++++++------- >> 1 file changed, 8 insertions(+), 7 deletions(-) >> >> diff --git a/.travis.yml b/.travis.yml >> index 4a0c23a..16be23f 100644 >> --- a/.travis.yml >> +++ b/.travis.yml >> @@ -40,7 +40,7 @@ notifications: >> on_failure: always >> env: >> global: >> - - TEST_CMD="" >> + - TEST_CMD="make check" >> - EXTRA_CONFIG="" >> matrix: >> # Group major targets together with their linux-user counterparts >> @@ -73,17 +73,14 @@ script: >> matrix: >> # We manually include a number of additional build for non-standard bits >> include: >> - # Make check target (we only do this once) >> - - env: >> - - TARGETS=alpha-softmmu,arm-softmmu,aarch64-softmmu,cris-softmmu,i386-softmmu,x86_64-softmmu,m68k-softmmu,microblaze-softmmu,microblazeel-softmmu,mips-softmmu,mips64-softmmu,mips64el-softmmu,mipsel-softmmu,or32-softmmu,ppc-softmmu,ppc64-softmmu,ppcemb-softmmu,s390x-softmmu,sh4-softmmu,sh4eb-softmmu,sparc-softmmu,sparc64-softmmu,unicore32-softmmu,lm32-softmmu,moxie-softmmu,tricore-softmmu,xtensa-softmmu,xtensaeb-softmmu >> - TEST_CMD="make check" >> - compiler: gcc >> # Debug related options >> - env: TARGETS=i386-softmmu,x86_64-softmmu >> EXTRA_CONFIG="--enable-debug" >> compiler: gcc >> + # We currently disable "make check" >> - env: TARGETS=i386-softmmu,x86_64-softmmu >> EXTRA_CONFIG="--enable-debug --enable-tcg-interpreter" >> + TEST_CMD="" >> compiler: gcc >> # Disable a few of the optional features >> - env: TARGETS=i386-softmmu,x86_64-softmmu >> @@ -104,11 +101,15 @@ matrix: >> - env: TARGETS=i386-softmmu,x86_64-softmmu >> EXTRA_CONFIG="--enable-trace-backends=simple" >> compiler: gcc >> + # We currently disable "make check" >> - env: TARGETS=i386-softmmu,x86_64-softmmu >> EXTRA_CONFIG="--enable-trace-backends=ftrace" >> + TEST_CMD="" >> compiler: gcc >> + # We currently disable "make check" >> - env: TARGETS=i386-softmmu,x86_64-softmmu >> - EXTRA_CONFIG="--enable-trace-backends=ust" >> + EXTRA_CONFIG="--enable-trace-backends=ust" >> + TEST_CMD="" >> compiler: gcc >> - env: TARGETS=i386-softmmu,x86_64-softmmu >> EXTRA_CONFIG="--enable-modules" -- Alex Bennée
On Sun, Jan 31, 2016 at 08:37:49AM +0000, Alex Bennée wrote: > > David Gibson <david@gibson.dropbear.id.au> writes: > > > On Thu, Jan 28, 2016 at 02:23:28PM +0000, Alex Bennée wrote: > >> We only ran make check once before it used to be an unreliable target. > >> It was only a stop gap measure and we should be able to revert it now. > >> This also stops us needing a large all-MMU build. > >> > >> We disable "make check" for a couple of the extra config targets which > >> are currently broken. > >> > >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > > > > So, in general I like the idea of running make check more widely. > > > > However.. I was wondering - what's the rationale for having separate > > matrix builds for each target (or small group) rather than just doing > > one build with all the targets? > > Each individual part of the matrix can be run in parallel with the > others so it makes sense to keep the build component small (as each > softmmu target rebuilds a significant chunk of the build). It does rebuild a significant chunk, but there's a significant chunk that isn't rebuilt as well. When I tried this I found a recombined build marginally decreases the elapsed time and significantly (maybe 30-40%) reduced the total time. Given the load the travis system is under, it seems to me that we should try to keep our total demand on its resources down when it doens't significantly lower our coverage. > Having said that there is a fair amount of repetition as we are > repeating all the generic qtests each time just so we can run the extra > ${TARGET}-qtest binaries. That too. > Travis does has an option for using ccache so it might be worth > experimenting with that to see if things are improved. That does sound like something worth looking at. One thing that does annoy me about travis is that it will do a full rebuild if you have two branches on exactly the same commit, or if you revert a branch to an earlier commit which was built previously. > > I can't see any obvious benefit to splitting the build that way, but > > it does increase the total build time significantly - and will do so > > rather more so with make check added. > > Elapsed and total are the ones to look at: > > https://travis-ci.org/stsquad/qemu/builds/105401126 > > vs > > https://travis-ci.org/qemu/qemu/builds/105711606 > > However it looks like Travis are having scaling growing pains because > there "old style" VM approach is running a lot faster than it used > to. Not terribly surprising TBH.
diff --git a/.travis.yml b/.travis.yml index 4a0c23a..16be23f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -40,7 +40,7 @@ notifications: on_failure: always env: global: - - TEST_CMD="" + - TEST_CMD="make check" - EXTRA_CONFIG="" matrix: # Group major targets together with their linux-user counterparts @@ -73,17 +73,14 @@ script: matrix: # We manually include a number of additional build for non-standard bits include: - # Make check target (we only do this once) - - env: - - TARGETS=alpha-softmmu,arm-softmmu,aarch64-softmmu,cris-softmmu,i386-softmmu,x86_64-softmmu,m68k-softmmu,microblaze-softmmu,microblazeel-softmmu,mips-softmmu,mips64-softmmu,mips64el-softmmu,mipsel-softmmu,or32-softmmu,ppc-softmmu,ppc64-softmmu,ppcemb-softmmu,s390x-softmmu,sh4-softmmu,sh4eb-softmmu,sparc-softmmu,sparc64-softmmu,unicore32-softmmu,lm32-softmmu,moxie-softmmu,tricore-softmmu,xtensa-softmmu,xtensaeb-softmmu - TEST_CMD="make check" - compiler: gcc # Debug related options - env: TARGETS=i386-softmmu,x86_64-softmmu EXTRA_CONFIG="--enable-debug" compiler: gcc + # We currently disable "make check" - env: TARGETS=i386-softmmu,x86_64-softmmu EXTRA_CONFIG="--enable-debug --enable-tcg-interpreter" + TEST_CMD="" compiler: gcc # Disable a few of the optional features - env: TARGETS=i386-softmmu,x86_64-softmmu @@ -104,11 +101,15 @@ matrix: - env: TARGETS=i386-softmmu,x86_64-softmmu EXTRA_CONFIG="--enable-trace-backends=simple" compiler: gcc + # We currently disable "make check" - env: TARGETS=i386-softmmu,x86_64-softmmu EXTRA_CONFIG="--enable-trace-backends=ftrace" + TEST_CMD="" compiler: gcc + # We currently disable "make check" - env: TARGETS=i386-softmmu,x86_64-softmmu - EXTRA_CONFIG="--enable-trace-backends=ust" + EXTRA_CONFIG="--enable-trace-backends=ust" + TEST_CMD="" compiler: gcc - env: TARGETS=i386-softmmu,x86_64-softmmu EXTRA_CONFIG="--enable-modules"
We only ran make check once before it used to be an unreliable target. It was only a stop gap measure and we should be able to revert it now. This also stops us needing a large all-MMU build. We disable "make check" for a couple of the extra config targets which are currently broken. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> --- .travis.yml | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)