| Message ID | 20200122012753.9846-5-wainersm@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | Acceptance tests: boot Linux with KVM test | expand |
On 22/01/2020 02.27, Wainer dos Santos Moschetta wrote: > Some acceptance tests require KVM or they are skipped. Travis > enables nested virtualization by default with Ubuntu > 18.04 (Bionic) on x86_64. So in order to run the kvm tests, this > changed the acceptance builder to run in a Bionic VM. Also > it was needed to ensure the current user has rw permission > to /dev/kvm. > > Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> > --- > .travis.yml | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/.travis.yml b/.travis.yml > index 6c1038a0f1..c3edd0a907 100644 > --- a/.travis.yml > +++ b/.travis.yml > @@ -2,6 +2,7 @@ > # Additional builds with specific requirements for a full VM need to > # be added as additional matrix: entries later on > dist: xenial > +sudo: true > language: c > compiler: > - gcc > @@ -83,6 +84,9 @@ git: > > before_script: > - if command -v ccache ; then ccache --zero-stats ; fi > + - if [[ -e /dev/kvm ]] && ! [[ -r /dev/kvm && -w /dev/kvm ]]; then > + sudo chmod o+rw /dev/kvm ; > + fi > - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR} > - ${SRC_DIR}/configure ${BASE_CONFIG} ${CONFIG} || { cat config.log && exit 1; } > script: > @@ -272,12 +276,13 @@ matrix: > - TEST_CMD="make check-acceptance" > after_script: > - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat > + dist: bionic > addons: > apt: > packages: > - python3-pil > - python3-pip > - - python3.5-venv > + - python3.6-venv > - tesseract-ocr > - tesseract-ocr-eng I'm surprised that the chmod is sufficient enough here (since I was having trouble with that in the kvm-unit-tests), but it seems to work, indeed: https://travis-ci.com/huth/qemu/jobs/278226646#L3762 So: Tested-by: Thomas Huth <thuth@redhat.com>
On 22/01/2020 02.27, Wainer dos Santos Moschetta wrote: > Some acceptance tests require KVM or they are skipped. Travis > enables nested virtualization by default with Ubuntu > 18.04 (Bionic) on x86_64. So in order to run the kvm tests, this > changed the acceptance builder to run in a Bionic VM. Also > it was needed to ensure the current user has rw permission > to /dev/kvm. > > Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> > --- > .travis.yml | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/.travis.yml b/.travis.yml > index 6c1038a0f1..c3edd0a907 100644 > --- a/.travis.yml > +++ b/.travis.yml > @@ -2,6 +2,7 @@ > # Additional builds with specific requirements for a full VM need to > # be added as additional matrix: entries later on > dist: xenial > +sudo: true > language: c > compiler: > - gcc > @@ -83,6 +84,9 @@ git: > > before_script: > - if command -v ccache ; then ccache --zero-stats ; fi > + - if [[ -e /dev/kvm ]] && ! [[ -r /dev/kvm && -w /dev/kvm ]]; then By the way, in case you respin, could you please use singel "[" instead of "[[" ... since that's what we use in almost all other shell scripts, too. Thomas
On 1/22/20 2:27 AM, Wainer dos Santos Moschetta wrote: > Some acceptance tests require KVM or they are skipped. Travis > enables nested virtualization by default with Ubuntu > 18.04 (Bionic) on x86_64. So in order to run the kvm tests, this > changed the acceptance builder to run in a Bionic VM. Also > it was needed to ensure the current user has rw permission > to /dev/kvm. > > Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> > --- > .travis.yml | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/.travis.yml b/.travis.yml > index 6c1038a0f1..c3edd0a907 100644 > --- a/.travis.yml > +++ b/.travis.yml > @@ -2,6 +2,7 @@ > # Additional builds with specific requirements for a full VM need to > # be added as additional matrix: entries later on > dist: xenial > +sudo: true > language: c > compiler: > - gcc > @@ -83,6 +84,9 @@ git: > > before_script: > - if command -v ccache ; then ccache --zero-stats ; fi > + - if [[ -e /dev/kvm ]] && ! [[ -r /dev/kvm && -w /dev/kvm ]]; then > + sudo chmod o+rw /dev/kvm ; > + fi > - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR} > - ${SRC_DIR}/configure ${BASE_CONFIG} ${CONFIG} || { cat config.log && exit 1; } > script: > @@ -272,12 +276,13 @@ matrix: > - TEST_CMD="make check-acceptance" > after_script: > - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat > + dist: bionic > addons: > apt: > packages: > - python3-pil > - python3-pip > - - python3.5-venv > + - python3.6-venv This line doesn't seem related to the patch. > - tesseract-ocr > - tesseract-ocr-eng > >
On 24/01/2020 10.38, Philippe Mathieu-Daudé wrote: > On 1/22/20 2:27 AM, Wainer dos Santos Moschetta wrote: >> Some acceptance tests require KVM or they are skipped. Travis >> enables nested virtualization by default with Ubuntu >> 18.04 (Bionic) on x86_64. So in order to run the kvm tests, this >> changed the acceptance builder to run in a Bionic VM. Also >> it was needed to ensure the current user has rw permission >> to /dev/kvm. >> >> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> >> --- >> .travis.yml | 7 ++++++- >> 1 file changed, 6 insertions(+), 1 deletion(-) >> >> diff --git a/.travis.yml b/.travis.yml >> index 6c1038a0f1..c3edd0a907 100644 >> --- a/.travis.yml >> +++ b/.travis.yml >> @@ -2,6 +2,7 @@ >> # Additional builds with specific requirements for a full VM need to >> # be added as additional matrix: entries later on >> dist: xenial >> +sudo: true >> language: c >> compiler: >> - gcc >> @@ -83,6 +84,9 @@ git: >> before_script: >> - if command -v ccache ; then ccache --zero-stats ; fi >> + - if [[ -e /dev/kvm ]] && ! [[ -r /dev/kvm && -w /dev/kvm ]]; then >> + sudo chmod o+rw /dev/kvm ; >> + fi >> - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR} >> - ${SRC_DIR}/configure ${BASE_CONFIG} ${CONFIG} || { cat >> config.log && exit 1; } >> script: >> @@ -272,12 +276,13 @@ matrix: >> - TEST_CMD="make check-acceptance" >> after_script: >> - python3 -c 'import json; r = >> json.load(open("tests/results/latest/results.json")); >> [print(t["logfile"]) for t in r["tests"] if t["status"] not in >> ("PASS", "SKIP")]' | xargs cat >> + dist: bionic >> addons: >> apt: >> packages: >> - python3-pil >> - python3-pip >> - - python3.5-venv >> + - python3.6-venv > > This line doesn't seem related to the patch. "dist:" has been switched from xenial to bionic, so I think it is required to update to python3.6 here, too? Thomas
On 1/24/20 10:44 AM, Thomas Huth wrote: > On 24/01/2020 10.38, Philippe Mathieu-Daudé wrote: >> On 1/22/20 2:27 AM, Wainer dos Santos Moschetta wrote: >>> Some acceptance tests require KVM or they are skipped. Travis >>> enables nested virtualization by default with Ubuntu >>> 18.04 (Bionic) on x86_64. So in order to run the kvm tests, this >>> changed the acceptance builder to run in a Bionic VM. Also >>> it was needed to ensure the current user has rw permission >>> to /dev/kvm. >>> >>> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> >>> --- >>> .travis.yml | 7 ++++++- >>> 1 file changed, 6 insertions(+), 1 deletion(-) >>> >>> diff --git a/.travis.yml b/.travis.yml >>> index 6c1038a0f1..c3edd0a907 100644 >>> --- a/.travis.yml >>> +++ b/.travis.yml >>> @@ -2,6 +2,7 @@ >>> # Additional builds with specific requirements for a full VM need to >>> # be added as additional matrix: entries later on >>> dist: xenial >>> +sudo: true >>> language: c >>> compiler: >>> - gcc >>> @@ -83,6 +84,9 @@ git: >>> before_script: >>> - if command -v ccache ; then ccache --zero-stats ; fi >>> + - if [[ -e /dev/kvm ]] && ! [[ -r /dev/kvm && -w /dev/kvm ]]; then >>> + sudo chmod o+rw /dev/kvm ; >>> + fi >>> - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR} >>> - ${SRC_DIR}/configure ${BASE_CONFIG} ${CONFIG} || { cat >>> config.log && exit 1; } >>> script: >>> @@ -272,12 +276,13 @@ matrix: >>> - TEST_CMD="make check-acceptance" >>> after_script: >>> - python3 -c 'import json; r = >>> json.load(open("tests/results/latest/results.json")); >>> [print(t["logfile"]) for t in r["tests"] if t["status"] not in >>> ("PASS", "SKIP")]' | xargs cat >>> + dist: bionic >>> addons: >>> apt: >>> packages: >>> - python3-pil >>> - python3-pip >>> - - python3.5-venv >>> + - python3.6-venv >> >> This line doesn't seem related to the patch. > > "dist:" has been switched from xenial to bionic, so I think it is > required to update to python3.6 here, too? OK, I got confused because line 4 is still "dist: xenial". Wainer can you add a comment about this in the commit description? I'm still not convinced we should enable "sudo: true" on all our jobs.
On 1/24/20 7:54 AM, Philippe Mathieu-Daudé wrote: > On 1/24/20 10:44 AM, Thomas Huth wrote: >> On 24/01/2020 10.38, Philippe Mathieu-Daudé wrote: >>> On 1/22/20 2:27 AM, Wainer dos Santos Moschetta wrote: >>>> Some acceptance tests require KVM or they are skipped. Travis >>>> enables nested virtualization by default with Ubuntu >>>> 18.04 (Bionic) on x86_64. So in order to run the kvm tests, this >>>> changed the acceptance builder to run in a Bionic VM. Also >>>> it was needed to ensure the current user has rw permission >>>> to /dev/kvm. >>>> >>>> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> >>>> --- >>>> .travis.yml | 7 ++++++- >>>> 1 file changed, 6 insertions(+), 1 deletion(-) >>>> >>>> diff --git a/.travis.yml b/.travis.yml >>>> index 6c1038a0f1..c3edd0a907 100644 >>>> --- a/.travis.yml >>>> +++ b/.travis.yml >>>> @@ -2,6 +2,7 @@ >>>> # Additional builds with specific requirements for a full VM >>>> need to >>>> # be added as additional matrix: entries later on >>>> dist: xenial >>>> +sudo: true >>>> language: c >>>> compiler: >>>> - gcc >>>> @@ -83,6 +84,9 @@ git: >>>> before_script: >>>> - if command -v ccache ; then ccache --zero-stats ; fi >>>> + - if [[ -e /dev/kvm ]] && ! [[ -r /dev/kvm && -w /dev/kvm ]]; then >>>> + sudo chmod o+rw /dev/kvm ; >>>> + fi Philippe, anwsering here your question about 'sudo'. The above statement runs on before_script for all the builders. As far as I know only on Bionic-based builders 'chmod' (that needs sudo) will be executed, so technically 'sudo' should be enabled only on those builders. But I thought that would be error-prone not enable it globally since the code requiring it is globally declared too. All in all, I don't have a strong option for this. >>>> >>>> - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR} >>>> - ${SRC_DIR}/configure ${BASE_CONFIG} ${CONFIG} || { cat >>>> config.log && exit 1; } >>>> script: >>>> @@ -272,12 +276,13 @@ matrix: >>>> - TEST_CMD="make check-acceptance" >>>> after_script: >>>> - python3 -c 'import json; r = >>>> json.load(open("tests/results/latest/results.json")); >>>> [print(t["logfile"]) for t in r["tests"] if t["status"] not in >>>> ("PASS", "SKIP")]' | xargs cat >>>> + dist: bionic >>>> addons: >>>> apt: >>>> packages: >>>> - python3-pil >>>> - python3-pip >>>> - - python3.5-venv >>>> + - python3.6-venv >>> >>> This line doesn't seem related to the patch. >> >> "dist:" has been switched from xenial to bionic, so I think it is >> required to update to python3.6 here, too? Thomas is right, python3.5-venv isn't available on Ubuntu Bionic. >> > > OK, I got confused because line 4 is still "dist: xenial". I'm about to send a proposal to bump dist to bionic. There are some non-acceptance tests being skipped because of the lack of nested kvm on Travis's xenial VMs, so that would be beneficial to them as well. Thomas mentioned in another email thread that there is a build problem with the libssh version of Bionic (I hope that can be worked out). Other than that, do you see any impediment to switch all builders completely? > > > Wainer can you add a comment about this in the commit description? Sure, actually I should have done it. Thanks for raising that point too. - Wainer > > > I'm still not convinced we should enable "sudo: true" on all our jobs. >
On 1/22/20 7:22 AM, Thomas Huth wrote: > On 22/01/2020 02.27, Wainer dos Santos Moschetta wrote: >> Some acceptance tests require KVM or they are skipped. Travis >> enables nested virtualization by default with Ubuntu >> 18.04 (Bionic) on x86_64. So in order to run the kvm tests, this >> changed the acceptance builder to run in a Bionic VM. Also >> it was needed to ensure the current user has rw permission >> to /dev/kvm. >> >> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> >> --- >> .travis.yml | 7 ++++++- >> 1 file changed, 6 insertions(+), 1 deletion(-) >> >> diff --git a/.travis.yml b/.travis.yml >> index 6c1038a0f1..c3edd0a907 100644 >> --- a/.travis.yml >> +++ b/.travis.yml >> @@ -2,6 +2,7 @@ >> # Additional builds with specific requirements for a full VM need to >> # be added as additional matrix: entries later on >> dist: xenial >> +sudo: true >> language: c >> compiler: >> - gcc >> @@ -83,6 +84,9 @@ git: >> >> before_script: >> - if command -v ccache ; then ccache --zero-stats ; fi >> + - if [[ -e /dev/kvm ]] && ! [[ -r /dev/kvm && -w /dev/kvm ]]; then > By the way, in case you respin, could you please use singel "[" instead > of "[[" ... since that's what we use in almost all other shell scripts, too. Like this? -> if [ -e /dev/kvm ] && [ ! -r /dev/kvm ] || [ ! -w /dev/kvm ]; then Thanks, Wainer > > Thomas
On 24/01/2020 22.15, Wainer dos Santos Moschetta wrote: > > On 1/22/20 7:22 AM, Thomas Huth wrote: >> On 22/01/2020 02.27, Wainer dos Santos Moschetta wrote: >>> Some acceptance tests require KVM or they are skipped. Travis >>> enables nested virtualization by default with Ubuntu >>> 18.04 (Bionic) on x86_64. So in order to run the kvm tests, this >>> changed the acceptance builder to run in a Bionic VM. Also >>> it was needed to ensure the current user has rw permission >>> to /dev/kvm. >>> >>> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> >>> --- >>> .travis.yml | 7 ++++++- >>> 1 file changed, 6 insertions(+), 1 deletion(-) >>> >>> diff --git a/.travis.yml b/.travis.yml >>> index 6c1038a0f1..c3edd0a907 100644 >>> --- a/.travis.yml >>> +++ b/.travis.yml >>> @@ -2,6 +2,7 @@ >>> # Additional builds with specific requirements for a full VM need to >>> # be added as additional matrix: entries later on >>> dist: xenial >>> +sudo: true >>> language: c >>> compiler: >>> - gcc >>> @@ -83,6 +84,9 @@ git: >>> before_script: >>> - if command -v ccache ; then ccache --zero-stats ; fi >>> + - if [[ -e /dev/kvm ]] && ! [[ -r /dev/kvm && -w /dev/kvm ]]; then >> By the way, in case you respin, could you please use singel "[" instead >> of "[[" ... since that's what we use in almost all other shell >> scripts, too. > > Like this? -> > > if [ -e /dev/kvm ] && [ ! -r /dev/kvm ] || [ ! -w /dev/kvm ]; then If I get the man-page of bash right, && and || have equal precedence ... so I'd maybe rather write it as: if [ -e /dev/kvm ]; then if [ ! -r /dev/kvm ] || [ ! -w /dev/kvm ]; ... ... ok, this is getting uglier ...maybe it's better to rather stick with your original code...? Thomas PS: You could also use -c instead -e in the first test.
On 1/24/20 8:55 PM, Wainer dos Santos Moschetta wrote: > On 1/24/20 7:54 AM, Philippe Mathieu-Daudé wrote: >> On 1/24/20 10:44 AM, Thomas Huth wrote: >>> On 24/01/2020 10.38, Philippe Mathieu-Daudé wrote: >>>> On 1/22/20 2:27 AM, Wainer dos Santos Moschetta wrote: >>>>> Some acceptance tests require KVM or they are skipped. Travis >>>>> enables nested virtualization by default with Ubuntu >>>>> 18.04 (Bionic) on x86_64. So in order to run the kvm tests, this >>>>> changed the acceptance builder to run in a Bionic VM. Also >>>>> it was needed to ensure the current user has rw permission >>>>> to /dev/kvm. >>>>> >>>>> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> >>>>> --- >>>>> .travis.yml | 7 ++++++- >>>>> 1 file changed, 6 insertions(+), 1 deletion(-) >>>>> >>>>> diff --git a/.travis.yml b/.travis.yml >>>>> index 6c1038a0f1..c3edd0a907 100644 >>>>> --- a/.travis.yml >>>>> +++ b/.travis.yml >>>>> @@ -2,6 +2,7 @@ >>>>> # Additional builds with specific requirements for a full VM >>>>> need to >>>>> # be added as additional matrix: entries later on >>>>> dist: xenial >>>>> +sudo: true >>>>> language: c >>>>> compiler: >>>>> - gcc >>>>> @@ -83,6 +84,9 @@ git: >>>>> before_script: >>>>> - if command -v ccache ; then ccache --zero-stats ; fi >>>>> + - if [[ -e /dev/kvm ]] && ! [[ -r /dev/kvm && -w /dev/kvm ]]; then >>>>> + sudo chmod o+rw /dev/kvm ; >>>>> + fi > > > Philippe, anwsering here your question about 'sudo'. > > The above statement runs on before_script for all the builders. As far > as I know only on Bionic-based builders 'chmod' (that needs sudo) will > be executed, so technically 'sudo' should be enabled only on those > builders. But I thought that would be error-prone not enable it globally > since the code requiring it is globally declared too. All in all, I > don't have a strong option for this. > > >>>>> >>>>> - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR} >>>>> - ${SRC_DIR}/configure ${BASE_CONFIG} ${CONFIG} || { cat >>>>> config.log && exit 1; } >>>>> script: >>>>> @@ -272,12 +276,13 @@ matrix: >>>>> - TEST_CMD="make check-acceptance" >>>>> after_script: >>>>> - python3 -c 'import json; r = >>>>> json.load(open("tests/results/latest/results.json")); >>>>> [print(t["logfile"]) for t in r["tests"] if t["status"] not in >>>>> ("PASS", "SKIP")]' | xargs cat >>>>> + dist: bionic >>>>> addons: >>>>> apt: >>>>> packages: >>>>> - python3-pil >>>>> - python3-pip >>>>> - - python3.5-venv >>>>> + - python3.6-venv >>>> >>>> This line doesn't seem related to the patch. >>> >>> "dist:" has been switched from xenial to bionic, so I think it is >>> required to update to python3.6 here, too? > > > Thomas is right, python3.5-venv isn't available on Ubuntu Bionic. > > >>> >> >> OK, I got confused because line 4 is still "dist: xenial". > > > I'm about to send a proposal to bump dist to bionic. There are some > non-acceptance tests being skipped because of the lack of nested kvm on > Travis's xenial VMs, so that would be beneficial to them as well. > > Thomas mentioned in another email thread that there is a build problem > with the libssh version of Bionic (I hope that can be worked out). Other > than that, do you see any impediment to switch all builders completely? No, this is a distribution bug, we can use "--disable-libssh" in the Bionic jobs. >> >> >> Wainer can you add a comment about this in the commit description? > > > Sure, actually I should have done it. Thanks for raising that point too. > > - Wainer > >> >> >> I'm still not convinced we should enable "sudo: true" on all our jobs. >> >
diff --git a/.travis.yml b/.travis.yml index 6c1038a0f1..c3edd0a907 100644 --- a/.travis.yml +++ b/.travis.yml @@ -2,6 +2,7 @@ # Additional builds with specific requirements for a full VM need to # be added as additional matrix: entries later on dist: xenial +sudo: true language: c compiler: - gcc @@ -83,6 +84,9 @@ git: before_script: - if command -v ccache ; then ccache --zero-stats ; fi + - if [[ -e /dev/kvm ]] && ! [[ -r /dev/kvm && -w /dev/kvm ]]; then + sudo chmod o+rw /dev/kvm ; + fi - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR} - ${SRC_DIR}/configure ${BASE_CONFIG} ${CONFIG} || { cat config.log && exit 1; } script: @@ -272,12 +276,13 @@ matrix: - TEST_CMD="make check-acceptance" after_script: - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat + dist: bionic addons: apt: packages: - python3-pil - python3-pip - - python3.5-venv + - python3.6-venv - tesseract-ocr - tesseract-ocr-eng
Some acceptance tests require KVM or they are skipped. Travis enables nested virtualization by default with Ubuntu 18.04 (Bionic) on x86_64. So in order to run the kvm tests, this changed the acceptance builder to run in a Bionic VM. Also it was needed to ensure the current user has rw permission to /dev/kvm. Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> --- .travis.yml | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)