| Message ID | 20200325131632.311034-4-dgilbert@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | [PULL,1/9] hmp-cmd: fix a missing_break warning | expand |
On Wed, Mar 25, 2020 at 01:16:26PM +0000, Dr. David Alan Gilbert (git) wrote: > From: Philippe Mathieu-Daudé <philmd@redhat.com> > > When using max-bandwidth=~100Mb/s, this test fails on Travis-CI > s390x when configured with --disable-tcg: > > $ make check-qtest > TEST check-qtest-s390x: tests/qtest/boot-serial-test > qemu-system-s390x: -accel tcg: invalid accelerator tcg > qemu-system-s390x: falling back to KVM > TEST check-qtest-s390x: tests/qtest/pxe-test > TEST check-qtest-s390x: tests/qtest/test-netfilter > TEST check-qtest-s390x: tests/qtest/test-filter-mirror > TEST check-qtest-s390x: tests/qtest/test-filter-redirector > TEST check-qtest-s390x: tests/qtest/drive_del-test > TEST check-qtest-s390x: tests/qtest/device-plug-test > TEST check-qtest-s390x: tests/qtest/virtio-ccw-test > TEST check-qtest-s390x: tests/qtest/cpu-plug-test > TEST check-qtest-s390x: tests/qtest/migration-test > ** > ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE > ERROR - Bail out! ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE > make: *** [tests/Makefile.include:633: check-qtest-s390x] Error 1 > > Per David Gilbert, "it could just be the writing is slow on s390 > and the migration thread fast; in which case the autocomplete > wouldn't be needed. Perhaps we just need to reduce the bandwidth > limit." > > Tuning the threshold by reducing the initial bandwidth makes the > autoconverge test pass. > > Suggested-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> > Message-Id: <20200323184015.11565-1-philmd@redhat.com> > Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > Tested-by: Alex Bennée <alex.bennee@linaro.org> > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> This slows make check down significantly for me, it stays at the migration test for minutes. I'm carrying a revert at top of my tree for now but I'd rather not need that. This seems like a fragile way to test things anyway. What happens if someone slows writing even more e.g. because it's running in a container or a VM? How about detecting that migration finished too early and slowing it down until autocomplete triggers? > --- > tests/qtest/migration-test.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c > index 3d6cc83b88..2568c9529c 100644 > --- a/tests/qtest/migration-test.c > +++ b/tests/qtest/migration-test.c > @@ -1211,7 +1211,7 @@ static void test_migrate_auto_converge(void) > * without throttling. > */ > migrate_set_parameter_int(from, "downtime-limit", 1); > - migrate_set_parameter_int(from, "max-bandwidth", 100000000); /* ~100Mb/s */ > + migrate_set_parameter_int(from, "max-bandwidth", 1000000); /* ~1Mb/s */ > > /* To check remaining size after precopy */ > migrate_set_capability(from, "pause-before-switchover", true); > -- > 2.25.1 > >
On 6/9/20 6:36 PM, Michael S. Tsirkin wrote: > On Wed, Mar 25, 2020 at 01:16:26PM +0000, Dr. David Alan Gilbert (git) wrote: >> From: Philippe Mathieu-Daudé <philmd@redhat.com> >> >> When using max-bandwidth=~100Mb/s, this test fails on Travis-CI >> s390x when configured with --disable-tcg: >> >> $ make check-qtest >> TEST check-qtest-s390x: tests/qtest/boot-serial-test >> qemu-system-s390x: -accel tcg: invalid accelerator tcg >> qemu-system-s390x: falling back to KVM >> TEST check-qtest-s390x: tests/qtest/pxe-test >> TEST check-qtest-s390x: tests/qtest/test-netfilter >> TEST check-qtest-s390x: tests/qtest/test-filter-mirror >> TEST check-qtest-s390x: tests/qtest/test-filter-redirector >> TEST check-qtest-s390x: tests/qtest/drive_del-test >> TEST check-qtest-s390x: tests/qtest/device-plug-test >> TEST check-qtest-s390x: tests/qtest/virtio-ccw-test >> TEST check-qtest-s390x: tests/qtest/cpu-plug-test >> TEST check-qtest-s390x: tests/qtest/migration-test >> ** >> ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE >> ERROR - Bail out! ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE >> make: *** [tests/Makefile.include:633: check-qtest-s390x] Error 1 >> >> Per David Gilbert, "it could just be the writing is slow on s390 >> and the migration thread fast; in which case the autocomplete >> wouldn't be needed. Perhaps we just need to reduce the bandwidth >> limit." >> >> Tuning the threshold by reducing the initial bandwidth makes the >> autoconverge test pass. >> >> Suggested-by: Dr. David Alan Gilbert <dgilbert@redhat.com> >> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> >> Message-Id: <20200323184015.11565-1-philmd@redhat.com> >> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> >> Tested-by: Alex Bennée <alex.bennee@linaro.org> >> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > > > This slows make check down significantly for me, it stays > at the migration test for minutes. Alex Bennée reported the same problem, I don't know what is the best way to fix this. > > I'm carrying a revert at top of my tree for now but I'd rather > not need that. > > > This seems like a fragile way to test things anyway. > What happens if someone slows writing even more > e.g. because it's running in a container or a VM? This seems to be the problem I noticed and tried to fix. > > How about detecting that migration finished too early > and slowing it down until autocomplete triggers? David, I am a bit clueless with this code, do you mind having a look? We can revert this test and disable the s390x runner, but it is our only big-endian host, which recently show to be useful. > >> --- >> tests/qtest/migration-test.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c >> index 3d6cc83b88..2568c9529c 100644 >> --- a/tests/qtest/migration-test.c >> +++ b/tests/qtest/migration-test.c >> @@ -1211,7 +1211,7 @@ static void test_migrate_auto_converge(void) >> * without throttling. >> */ >> migrate_set_parameter_int(from, "downtime-limit", 1); >> - migrate_set_parameter_int(from, "max-bandwidth", 100000000); /* ~100Mb/s */ >> + migrate_set_parameter_int(from, "max-bandwidth", 1000000); /* ~1Mb/s */ >> >> /* To check remaining size after precopy */ >> migrate_set_capability(from, "pause-before-switchover", true); >> -- >> 2.25.1 >> >> >
* Michael S. Tsirkin (mst@redhat.com) wrote: > On Wed, Mar 25, 2020 at 01:16:26PM +0000, Dr. David Alan Gilbert (git) wrote: > > From: Philippe Mathieu-Daudé <philmd@redhat.com> > > > > When using max-bandwidth=~100Mb/s, this test fails on Travis-CI > > s390x when configured with --disable-tcg: > > > > $ make check-qtest > > TEST check-qtest-s390x: tests/qtest/boot-serial-test > > qemu-system-s390x: -accel tcg: invalid accelerator tcg > > qemu-system-s390x: falling back to KVM > > TEST check-qtest-s390x: tests/qtest/pxe-test > > TEST check-qtest-s390x: tests/qtest/test-netfilter > > TEST check-qtest-s390x: tests/qtest/test-filter-mirror > > TEST check-qtest-s390x: tests/qtest/test-filter-redirector > > TEST check-qtest-s390x: tests/qtest/drive_del-test > > TEST check-qtest-s390x: tests/qtest/device-plug-test > > TEST check-qtest-s390x: tests/qtest/virtio-ccw-test > > TEST check-qtest-s390x: tests/qtest/cpu-plug-test > > TEST check-qtest-s390x: tests/qtest/migration-test > > ** > > ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE > > ERROR - Bail out! ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE > > make: *** [tests/Makefile.include:633: check-qtest-s390x] Error 1 > > > > Per David Gilbert, "it could just be the writing is slow on s390 > > and the migration thread fast; in which case the autocomplete > > wouldn't be needed. Perhaps we just need to reduce the bandwidth > > limit." > > > > Tuning the threshold by reducing the initial bandwidth makes the > > autoconverge test pass. > > > > Suggested-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > > Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> > > Message-Id: <20200323184015.11565-1-philmd@redhat.com> > > Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > > Tested-by: Alex Bennée <alex.bennee@linaro.org> > > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > > > This slows make check down significantly for me, it stays > at the migration test for minutes. Is this on the s390x version or all of them? > I'm carrying a revert at top of my tree for now but I'd rather > not need that. > > > This seems like a fragile way to test things anyway. > What happens if someone slows writing even more > e.g. because it's running in a container or a VM? > > How about detecting that migration finished too early > and slowing it down until autocomplete triggers? THe problem is you can't rely on any form of consistency in a heavily contended container, so the fact that it took n-seconds to migrate at this attempt means very little about what the next attempt will take. Dave > > > > --- > > tests/qtest/migration-test.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c > > index 3d6cc83b88..2568c9529c 100644 > > --- a/tests/qtest/migration-test.c > > +++ b/tests/qtest/migration-test.c > > @@ -1211,7 +1211,7 @@ static void test_migrate_auto_converge(void) > > * without throttling. > > */ > > migrate_set_parameter_int(from, "downtime-limit", 1); > > - migrate_set_parameter_int(from, "max-bandwidth", 100000000); /* ~100Mb/s */ > > + migrate_set_parameter_int(from, "max-bandwidth", 1000000); /* ~1Mb/s */ > > > > /* To check remaining size after precopy */ > > migrate_set_capability(from, "pause-before-switchover", true); > > -- > > 2.25.1 > > > > > -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
On 6/9/20 7:00 PM, Dr. David Alan Gilbert wrote: > * Michael S. Tsirkin (mst@redhat.com) wrote: >> On Wed, Mar 25, 2020 at 01:16:26PM +0000, Dr. David Alan Gilbert (git) wrote: >>> From: Philippe Mathieu-Daudé <philmd@redhat.com> >>> >>> When using max-bandwidth=~100Mb/s, this test fails on Travis-CI >>> s390x when configured with --disable-tcg: >>> >>> $ make check-qtest >>> TEST check-qtest-s390x: tests/qtest/boot-serial-test >>> qemu-system-s390x: -accel tcg: invalid accelerator tcg >>> qemu-system-s390x: falling back to KVM >>> TEST check-qtest-s390x: tests/qtest/pxe-test >>> TEST check-qtest-s390x: tests/qtest/test-netfilter >>> TEST check-qtest-s390x: tests/qtest/test-filter-mirror >>> TEST check-qtest-s390x: tests/qtest/test-filter-redirector >>> TEST check-qtest-s390x: tests/qtest/drive_del-test >>> TEST check-qtest-s390x: tests/qtest/device-plug-test >>> TEST check-qtest-s390x: tests/qtest/virtio-ccw-test >>> TEST check-qtest-s390x: tests/qtest/cpu-plug-test >>> TEST check-qtest-s390x: tests/qtest/migration-test >>> ** >>> ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE >>> ERROR - Bail out! ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE >>> make: *** [tests/Makefile.include:633: check-qtest-s390x] Error 1 >>> >>> Per David Gilbert, "it could just be the writing is slow on s390 >>> and the migration thread fast; in which case the autocomplete >>> wouldn't be needed. Perhaps we just need to reduce the bandwidth >>> limit." >>> >>> Tuning the threshold by reducing the initial bandwidth makes the >>> autoconverge test pass. >>> >>> Suggested-by: Dr. David Alan Gilbert <dgilbert@redhat.com> >>> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> >>> Message-Id: <20200323184015.11565-1-philmd@redhat.com> >>> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> >>> Tested-by: Alex Bennée <alex.bennee@linaro.org> >>> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> >> >> >> This slows make check down significantly for me, it stays >> at the migration test for minutes. > > Is this on the s390x version or all of them? Personally I only noticed it on s390x. Alex have you seen another arch? > >> I'm carrying a revert at top of my tree for now but I'd rather >> not need that. >> >> >> This seems like a fragile way to test things anyway. >> What happens if someone slows writing even more >> e.g. because it's running in a container or a VM? >> >> How about detecting that migration finished too early >> and slowing it down until autocomplete triggers? > > THe problem is you can't rely on any form of consistency in a heavily > contended container, so the fact that it took n-seconds to migrate at > this attempt means very little about what the next attempt will take. > > Dave > >> >> >>> --- >>> tests/qtest/migration-test.c | 2 +- >>> 1 file changed, 1 insertion(+), 1 deletion(-) >>> >>> diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c >>> index 3d6cc83b88..2568c9529c 100644 >>> --- a/tests/qtest/migration-test.c >>> +++ b/tests/qtest/migration-test.c >>> @@ -1211,7 +1211,7 @@ static void test_migrate_auto_converge(void) >>> * without throttling. >>> */ >>> migrate_set_parameter_int(from, "downtime-limit", 1); >>> - migrate_set_parameter_int(from, "max-bandwidth", 100000000); /* ~100Mb/s */ >>> + migrate_set_parameter_int(from, "max-bandwidth", 1000000); /* ~1Mb/s */ >>> >>> /* To check remaining size after precopy */ >>> migrate_set_capability(from, "pause-before-switchover", true); >>> -- >>> 2.25.1 >>> >>> >> > -- > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK >
On Tue, Jun 09, 2020 at 06:00:17PM +0100, Dr. David Alan Gilbert wrote: > * Michael S. Tsirkin (mst@redhat.com) wrote: > > On Wed, Mar 25, 2020 at 01:16:26PM +0000, Dr. David Alan Gilbert (git) wrote: > > > From: Philippe Mathieu-Daudé <philmd@redhat.com> > > > > > > When using max-bandwidth=~100Mb/s, this test fails on Travis-CI > > > s390x when configured with --disable-tcg: > > > > > > $ make check-qtest > > > TEST check-qtest-s390x: tests/qtest/boot-serial-test > > > qemu-system-s390x: -accel tcg: invalid accelerator tcg > > > qemu-system-s390x: falling back to KVM > > > TEST check-qtest-s390x: tests/qtest/pxe-test > > > TEST check-qtest-s390x: tests/qtest/test-netfilter > > > TEST check-qtest-s390x: tests/qtest/test-filter-mirror > > > TEST check-qtest-s390x: tests/qtest/test-filter-redirector > > > TEST check-qtest-s390x: tests/qtest/drive_del-test > > > TEST check-qtest-s390x: tests/qtest/device-plug-test > > > TEST check-qtest-s390x: tests/qtest/virtio-ccw-test > > > TEST check-qtest-s390x: tests/qtest/cpu-plug-test > > > TEST check-qtest-s390x: tests/qtest/migration-test > > > ** > > > ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE > > > ERROR - Bail out! ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE > > > make: *** [tests/Makefile.include:633: check-qtest-s390x] Error 1 > > > > > > Per David Gilbert, "it could just be the writing is slow on s390 > > > and the migration thread fast; in which case the autocomplete > > > wouldn't be needed. Perhaps we just need to reduce the bandwidth > > > limit." > > > > > > Tuning the threshold by reducing the initial bandwidth makes the > > > autoconverge test pass. > > > > > > Suggested-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > > > Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> > > > Message-Id: <20200323184015.11565-1-philmd@redhat.com> > > > Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > > > Tested-by: Alex Bennée <alex.bennee@linaro.org> > > > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > > > > > > This slows make check down significantly for me, it stays > > at the migration test for minutes. > > Is this on the s390x version or all of them? x86 slows down for me. > > I'm carrying a revert at top of my tree for now but I'd rather > > not need that. > > > > > > This seems like a fragile way to test things anyway. > > What happens if someone slows writing even more > > e.g. because it's running in a container or a VM? > > > > How about detecting that migration finished too early > > and slowing it down until autocomplete triggers? > > THe problem is you can't rely on any form of consistency in a heavily > contended container, so the fact that it took n-seconds to migrate at > this attempt means very little about what the next attempt will take. > > Dave What I'm saying is try migrating quickly. If it's too quick try again ... > > > > > > > --- > > > tests/qtest/migration-test.c | 2 +- > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c > > > index 3d6cc83b88..2568c9529c 100644 > > > --- a/tests/qtest/migration-test.c > > > +++ b/tests/qtest/migration-test.c > > > @@ -1211,7 +1211,7 @@ static void test_migrate_auto_converge(void) > > > * without throttling. > > > */ > > > migrate_set_parameter_int(from, "downtime-limit", 1); > > > - migrate_set_parameter_int(from, "max-bandwidth", 100000000); /* ~100Mb/s */ > > > + migrate_set_parameter_int(from, "max-bandwidth", 1000000); /* ~1Mb/s */ > > > > > > /* To check remaining size after precopy */ > > > migrate_set_capability(from, "pause-before-switchover", true); > > > -- > > > 2.25.1 > > > > > > > > > -- > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 3d6cc83b88..2568c9529c 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -1211,7 +1211,7 @@ static void test_migrate_auto_converge(void) * without throttling. */ migrate_set_parameter_int(from, "downtime-limit", 1); - migrate_set_parameter_int(from, "max-bandwidth", 100000000); /* ~100Mb/s */ + migrate_set_parameter_int(from, "max-bandwidth", 1000000); /* ~1Mb/s */ /* To check remaining size after precopy */ migrate_set_capability(from, "pause-before-switchover", true);