| Message ID | 20211124130150.268230-6-berrange@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | tests/docker: start using libvirt-ci's "lcitool" for dockerfiles | expand |
On 24/11/2021 14.01, Daniel P. Berrangé wrote: > The block I/O tests don't work on Alpine because their alternative libc > impl emits different strings for errnos, which breaks the expected > output matching. e.g. > > === IO: pattern 102 > wrote 512/512 bytes at offset 512 > 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > -qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: Input/output error > +qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: I/O error > 4 > Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 > Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=0 > > Currently the I/O tests are skipped as a side effect of the Alpine image > containing a minimal busybox 'sed' binary, rather than GNU Sed. This is > a fragile assumption that will be invalidated when the dockerfile is > changed to be autogenerated from a standardized package list that > includes GNU Sed. > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> > --- > .gitlab-ci.d/buildtest.yml | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml > index 71d0f407ad..e1fe37e563 100644 > --- a/.gitlab-ci.d/buildtest.yml > +++ b/.gitlab-ci.d/buildtest.yml > @@ -24,7 +24,7 @@ check-system-alpine: > artifacts: true > variables: > IMAGE: alpine > - MAKE_CHECK_ARGS: check > + MAKE_CHECK_ARGS: check-unit check-qtest Hmm, that's just a work-around ... what if some user wants to run "make check" on an Alpine installation that has the real GNU sed installed? ... I think this rather requires some fixing in the iotests instead - or maybe the related tests should simply not be in the "auto" group anymore? Thomas
On 11/24/21 14:36, Thomas Huth wrote: > On 24/11/2021 14.01, Daniel P. Berrangé wrote: >> The block I/O tests don't work on Alpine because their alternative libc >> impl emits different strings for errnos, which breaks the expected >> output matching. e.g. >> >> === IO: pattern 102 >> wrote 512/512 bytes at offset 512 >> 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> -qemu-img: Error while reading offset 0 of >> blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: Input/output error >> +qemu-img: Error while reading offset 0 of >> blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: I/O error >> 4 >> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 >> Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=0 >> >> Currently the I/O tests are skipped as a side effect of the Alpine image >> containing a minimal busybox 'sed' binary, rather than GNU Sed. This is >> a fragile assumption that will be invalidated when the dockerfile is >> changed to be autogenerated from a standardized package list that >> includes GNU Sed. >> >> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> >> --- >> .gitlab-ci.d/buildtest.yml | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml >> index 71d0f407ad..e1fe37e563 100644 >> --- a/.gitlab-ci.d/buildtest.yml >> +++ b/.gitlab-ci.d/buildtest.yml >> @@ -24,7 +24,7 @@ check-system-alpine: >> artifacts: true >> variables: >> IMAGE: alpine >> - MAKE_CHECK_ARGS: check >> + MAKE_CHECK_ARGS: check-unit check-qtest > > Hmm, that's just a work-around ... what if some user wants to run "make > check" on an Alpine installation that has the real GNU sed installed? > ... I think this rather requires some fixing in the iotests instead - or > maybe the related tests should simply not be in the "auto" group anymore? Or kludge EIO in error_setg_errno_internal()?
On Wed, Nov 24, 2021 at 02:36:59PM +0100, Thomas Huth wrote: > On 24/11/2021 14.01, Daniel P. Berrangé wrote: > > The block I/O tests don't work on Alpine because their alternative libc > > impl emits different strings for errnos, which breaks the expected > > output matching. e.g. > > > > === IO: pattern 102 > > wrote 512/512 bytes at offset 512 > > 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > > -qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: Input/output error > > +qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: I/O error > > 4 > > Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 > > Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=0 > > > > Currently the I/O tests are skipped as a side effect of the Alpine image > > containing a minimal busybox 'sed' binary, rather than GNU Sed. This is > > a fragile assumption that will be invalidated when the dockerfile is > > changed to be autogenerated from a standardized package list that > > includes GNU Sed. > > > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> > > --- > > .gitlab-ci.d/buildtest.yml | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml > > index 71d0f407ad..e1fe37e563 100644 > > --- a/.gitlab-ci.d/buildtest.yml > > +++ b/.gitlab-ci.d/buildtest.yml > > @@ -24,7 +24,7 @@ check-system-alpine: > > artifacts: true > > variables: > > IMAGE: alpine > > - MAKE_CHECK_ARGS: check > > + MAKE_CHECK_ARGS: check-unit check-qtest > > Hmm, that's just a work-around ... what if some user wants to run "make > check" on an Alpine installation that has the real GNU sed installed? ... I > think this rather requires some fixing in the iotests instead - or maybe the > related tests should simply not be in the "auto" group anymore? Of course, the I/o tests should be fixed, but that's a big job that on one has volunteered for. Taking the tests in question out of the "auto" group would be very detrimental for test coverage on other platforms. There are many affected tests so that is not desirable. As explained above, the CI job is already skipping the I/O tests today as a side effect of only having Busybox sed. This change just makes that explicit in the CI config so it is clear that we're missing this coverage and won't accidentally break when GNU Sed appears in the dockerfile. Regards, Daniel
On 11/24/21 14:46, Daniel P. Berrangé wrote: > On Wed, Nov 24, 2021 at 02:36:59PM +0100, Thomas Huth wrote: >> On 24/11/2021 14.01, Daniel P. Berrangé wrote: >>> The block I/O tests don't work on Alpine because their alternative libc >>> impl emits different strings for errnos, which breaks the expected >>> output matching. e.g. >>> >>> === IO: pattern 102 >>> wrote 512/512 bytes at offset 512 >>> 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >>> -qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: Input/output error >>> +qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: I/O error >>> 4 >>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 >>> Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=0 >>> >>> Currently the I/O tests are skipped as a side effect of the Alpine image >>> containing a minimal busybox 'sed' binary, rather than GNU Sed. This is >>> a fragile assumption that will be invalidated when the dockerfile is >>> changed to be autogenerated from a standardized package list that >>> includes GNU Sed. "GNU sed" in lowercase? ("stream editor", 2 occurrences in description). >>> >>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> >>> --- >>> .gitlab-ci.d/buildtest.yml | 2 +- >>> 1 file changed, 1 insertion(+), 1 deletion(-) >>> >>> diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml >>> index 71d0f407ad..e1fe37e563 100644 >>> --- a/.gitlab-ci.d/buildtest.yml >>> +++ b/.gitlab-ci.d/buildtest.yml >>> @@ -24,7 +24,7 @@ check-system-alpine: >>> artifacts: true >>> variables: >>> IMAGE: alpine >>> - MAKE_CHECK_ARGS: check >>> + MAKE_CHECK_ARGS: check-unit check-qtest >> >> Hmm, that's just a work-around ... what if some user wants to run "make >> check" on an Alpine installation that has the real GNU sed installed? ... I >> think this rather requires some fixing in the iotests instead - or maybe the >> related tests should simply not be in the "auto" group anymore? > > Of course, the I/o tests should be fixed, but that's a big job that on > one has volunteered for. > > Taking the tests in question out of the "auto" group would be very > detrimental for test coverage on other platforms. There are many > affected tests so that is not desirable. > > As explained above, the CI job is already skipping the I/O tests today > as a side effect of only having Busybox sed. This change just makes > that explicit in the CI config so it is clear that we're missing this > coverage and won't accidentally break when GNU Sed appears in the > dockerfile. This sounds reasonable. Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml index 71d0f407ad..e1fe37e563 100644 --- a/.gitlab-ci.d/buildtest.yml +++ b/.gitlab-ci.d/buildtest.yml @@ -24,7 +24,7 @@ check-system-alpine: artifacts: true variables: IMAGE: alpine - MAKE_CHECK_ARGS: check + MAKE_CHECK_ARGS: check-unit check-qtest avocado-system-alpine: extends: .avocado_test_job_template
The block I/O tests don't work on Alpine because their alternative libc impl emits different strings for errnos, which breaks the expected output matching. e.g. === IO: pattern 102 wrote 512/512 bytes at offset 512 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: Input/output error +qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: I/O error 4 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=0 Currently the I/O tests are skipped as a side effect of the Alpine image containing a minimal busybox 'sed' binary, rather than GNU Sed. This is a fragile assumption that will be invalidated when the dockerfile is changed to be autogenerated from a standardized package list that includes GNU Sed. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- .gitlab-ci.d/buildtest.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)