| Message ID | 20230209233426.37811-1-quintela@redhat.com |
|---|---|
| State | New |
| Headers | show |
On Thu, 9 Feb 2023 at 23:35, Juan Quintela <quintela@redhat.com> wrote: > > The following changes since commit 417296c8d8588f782018d01a317f88957e9786d6: > > tests/qtest/netdev-socket: Raise connection timeout to 60 seconds (2023-02-09 11:23:53 +0000) > > are available in the Git repository at: > > https://gitlab.com/juan.quintela/qemu.git tags/migration-20230209-pull-request > > for you to fetch changes up to 858191aebda251a4d1e3bc77b238096673241cdd: > > migration: Postpone postcopy preempt channel to be after main (2023-02-09 21:26:02 +0100) > > ---------------------------------------------------------------- > Migration Pull request > > Hi > > This are all the reviewed patches for migration: > - AVX512 support for xbzrle (Ling Xu) > - /dev/userfaultd support (Peter Xu) > - Improve ordering of channels (Peter Xu) > - multifd cleanups (Li Zhang) > - Remove spurious files from last merge (me) > Rebase makes that to you > - Fix mixup between state_pending_{exact,estimate} (me) > - Cache RAM size during migration (me) > - cleanup several functions (me) > > Please apply. > > ---------------------------------------------------------------- Fails to build the user-mode emulators: https://gitlab.com/qemu-project/qemu/-/jobs/3749435025 In file included from ../authz/base.c:24: ../authz/trace.h:1:10: fatal error: trace/trace-authz.h: No such file or directory 1 | #include "trace/trace-authz.h" https://gitlab.com/qemu-project/qemu/-/jobs/3749435094 In file included from ../authz/simple.c:23: ../authz/trace.h:1:10: fatal error: trace/trace-authz.h: No such file or directory 1 | #include "trace/trace-authz.h" https://gitlab.com/qemu-project/qemu/-/jobs/3749434963 In file included from ../authz/listfile.c:23: ../authz/trace.h:1:10: fatal error: trace/trace-authz.h: No such file or directory 1 | #include "trace/trace-authz.h" etc thanks -- PMM
Peter Maydell <peter.maydell@linaro.org> wrote: > On Thu, 9 Feb 2023 at 23:35, Juan Quintela <quintela@redhat.com> wrote: >> >> The following changes since commit 417296c8d8588f782018d01a317f88957e9786d6: >> >> tests/qtest/netdev-socket: Raise connection timeout to 60 seconds (2023-02-09 11:23:53 +0000) >> >> are available in the Git repository at: >> >> https://gitlab.com/juan.quintela/qemu.git tags/migration-20230209-pull-request >> >> for you to fetch changes up to 858191aebda251a4d1e3bc77b238096673241cdd: >> >> migration: Postpone postcopy preempt channel to be after main (2023-02-09 21:26:02 +0100) >> >> ---------------------------------------------------------------- >> Migration Pull request >> >> Hi >> >> This are all the reviewed patches for migration: >> - AVX512 support for xbzrle (Ling Xu) >> - /dev/userfaultd support (Peter Xu) >> - Improve ordering of channels (Peter Xu) >> - multifd cleanups (Li Zhang) >> - Remove spurious files from last merge (me) >> Rebase makes that to you >> - Fix mixup between state_pending_{exact,estimate} (me) >> - Cache RAM size during migration (me) >> - cleanup several functions (me) >> >> Please apply. >> >> ---------------------------------------------------------------- > > Fails to build the user-mode emulators: This is weird. $ ls *user -d aarch64_be-linux-user m68k-linux-user ppc-linux-user aarch64-linux-user microblazeel-linux-user riscv32-linux-user alpha-linux-user microblaze-linux-user riscv64-linux-user armeb-linux-user mips64el-linux-user s390x-linux-user arm-linux-user mips64-linux-user sh4eb-linux-user bsd-user mipsel-linux-user sh4-linux-user common-user mips-linux-user sparc32plus-linux-user cris-linux-user mipsn32el-linux-user sparc64-linux-user hexagon-linux-user mipsn32-linux-user sparc-linux-user hppa-linux-user nios2-linux-user x86_64-linux-user i386-linux-user or1k-linux-user xtensaeb-linux-user linux-user ppc64le-linux-user xtensa-linux-user loongarch64-linux-user ppc64-linux-user I think I build all user emulators locally. > https://gitlab.com/qemu-project/qemu/-/jobs/3749435025 > > In file included from ../authz/base.c:24: > ../authz/trace.h:1:10: fatal error: trace/trace-authz.h: No such file > or directory > 1 | #include "trace/trace-authz.h" This series only have one change for traces: diff --git a/util/trace-events b/util/trace-events index c8f53d7d9f..16f78d8fe5 100644 --- a/util/trace-events +++ b/util/trace-events @@ -93,6 +93,7 @@ qemu_vfio_region_info(const char *desc, uint64_t region_ofs, uint64_t region_siz qemu_vfio_pci_map_bar(int index, uint64_t region_ofs, uint64_t region_size, int ofs, void *host) "map region bar#%d addr 0x%"PRIx64" size 0x%"PRIx64" ofs 0x%x host %p" #userfaultfd.c +uffd_detect_open_mode(int mode) "%d" uffd_query_features_nosys(int err) "errno: %i" uffd_query_features_api_failed(int err) "errno: %i" uffd_create_fd_nosys(int err) "errno: %i" Rest of trace mentions are for the removal of migration.multifd.c.orig And I don't play with authentication at all. This is Fedora 37. > https://gitlab.com/qemu-project/qemu/-/jobs/3749435094 > In file included from ../authz/simple.c:23: > ../authz/trace.h:1:10: fatal error: trace/trace-authz.h: No such file > or directory Problem is that this trace file is not generated, but I can think how any change that I did can influence this. > 1 | #include "trace/trace-authz.h" > > > https://gitlab.com/qemu-project/qemu/-/jobs/3749434963 > In file included from ../authz/listfile.c:23: > ../authz/trace.h:1:10: fatal error: trace/trace-authz.h: No such file > or directory > 1 | #include "trace/trace-authz.h" Looking at the ouptut of these, they are not informatives at all. I am going to try to compile linux-user without system, and see if that brings a clue. But I don't know what is going on here. Later, Juan.
On Fri, 10 Feb 2023 at 14:21, Juan Quintela <quintela@redhat.com> wrote: > > Peter Maydell <peter.maydell@linaro.org> wrote: > > Fails to build the user-mode emulators: > > This is weird. > > https://gitlab.com/qemu-project/qemu/-/jobs/3749435025 > > > > In file included from ../authz/base.c:24: > > ../authz/trace.h:1:10: fatal error: trace/trace-authz.h: No such file > > or directory > > 1 | #include "trace/trace-authz.h" > > This series only have one change for traces: > > diff --git a/util/trace-events b/util/trace-events > index c8f53d7d9f..16f78d8fe5 100644 > --- a/util/trace-events > +++ b/util/trace-events > @@ -93,6 +93,7 @@ qemu_vfio_region_info(const char *desc, uint64_t region_ofs, uint64_t region_siz > qemu_vfio_pci_map_bar(int index, uint64_t region_ofs, uint64_t region_size, int ofs, void *host) "map region bar#%d addr 0x%"PRIx64" size 0x%"PRIx64" ofs 0x%x host %p" > > #userfaultfd.c > +uffd_detect_open_mode(int mode) "%d" > uffd_query_features_nosys(int err) "errno: %i" > uffd_query_features_api_failed(int err) "errno: %i" > uffd_create_fd_nosys(int err) "errno: %i" > > Rest of trace mentions are for the removal of migration.multifd.c.orig > > And I don't play with authentication at all. > > This is Fedora 37. > > > https://gitlab.com/qemu-project/qemu/-/jobs/3749435094 > > In file included from ../authz/simple.c:23: > > ../authz/trace.h:1:10: fatal error: trace/trace-authz.h: No such file > > or directory > > Problem is that this trace file is not generated, but I can think how > any change that I did can influence this. > > > 1 | #include "trace/trace-authz.h" > > > > > > https://gitlab.com/qemu-project/qemu/-/jobs/3749434963 > > In file included from ../authz/listfile.c:23: > > ../authz/trace.h:1:10: fatal error: trace/trace-authz.h: No such file > > or directory > > 1 | #include "trace/trace-authz.h" > > Looking at the ouptut of these, they are not informatives at all. > > I am going to try to compile linux-user without system, and see if that > brings a clue. Yes, I suspect this is a "user-mode only build" specific failure (you may need --disable-system --disable-tools to see it). meson.build only puts authz into trace_events_subdirs "if have_block" (which is to say "if have_system or have_tools"). However the bit of meson.build that says "subdir('authz') does not have the same condition on it -- it's just been put in the list without any condition on it. So I think that in a build-only-user-emulators config meson will not generate trace events for the subdirectory but will try to build it, which falls over. Contrast 'block', 'nbd', 'scsi', which are all guarded by 'if have_block' for their subdir() lines, to match the guard on the trace_events_subdirs. OTOH 'io' is also mismatched-guards... Why this only shows up with your pullreq I have no idea. thanks -- PMM
Peter Maydell <peter.maydell@linaro.org> wrote: > On Fri, 10 Feb 2023 at 14:21, Juan Quintela <quintela@redhat.com> wrote: >> >> Peter Maydell <peter.maydell@linaro.org> wrote: >> > Fails to build the user-mode emulators: >> >> This is weird. > >> > https://gitlab.com/qemu-project/qemu/-/jobs/3749435025 >> > >> > In file included from ../authz/base.c:24: >> > ../authz/trace.h:1:10: fatal error: trace/trace-authz.h: No such file >> > or directory >> > 1 | #include "trace/trace-authz.h" >> >> This series only have one change for traces: >> >> diff --git a/util/trace-events b/util/trace-events >> index c8f53d7d9f..16f78d8fe5 100644 >> --- a/util/trace-events >> +++ b/util/trace-events >> @@ -93,6 +93,7 @@ qemu_vfio_region_info(const char *desc, uint64_t region_ofs, uint64_t region_siz >> qemu_vfio_pci_map_bar(int index, uint64_t region_ofs, uint64_t >> region_size, int ofs, void *host) "map region bar#%d addr >> 0x%"PRIx64" size 0x%"PRIx64" ofs 0x%x host %p" >> >> #userfaultfd.c >> +uffd_detect_open_mode(int mode) "%d" >> uffd_query_features_nosys(int err) "errno: %i" >> uffd_query_features_api_failed(int err) "errno: %i" >> uffd_create_fd_nosys(int err) "errno: %i" >> >> Rest of trace mentions are for the removal of migration.multifd.c.orig >> >> And I don't play with authentication at all. >> >> This is Fedora 37. >> >> > https://gitlab.com/qemu-project/qemu/-/jobs/3749435094 >> > In file included from ../authz/simple.c:23: >> > ../authz/trace.h:1:10: fatal error: trace/trace-authz.h: No such file >> > or directory >> >> Problem is that this trace file is not generated, but I can think how >> any change that I did can influence this. >> >> > 1 | #include "trace/trace-authz.h" >> > >> > >> > https://gitlab.com/qemu-project/qemu/-/jobs/3749434963 >> > In file included from ../authz/listfile.c:23: >> > ../authz/trace.h:1:10: fatal error: trace/trace-authz.h: No such file >> > or directory >> > 1 | #include "trace/trace-authz.h" >> >> Looking at the ouptut of these, they are not informatives at all. >> >> I am going to try to compile linux-user without system, and see if that >> brings a clue. > > Yes, I suspect this is a "user-mode only build" specific failure > (you may need --disable-system --disable-tools to see it). git-bisect is my friend O:-) And yes, the problem was in my PULL request. Again, I don't know why it fails. diff --git a/tests/bench/meson.build b/tests/bench/meson.build index daefead58d..7477a1f401 100644 --- a/tests/bench/meson.build +++ b/tests/bench/meson.build @@ -3,9 +3,11 @@ qht_bench = executable('qht-bench', sources: 'qht-bench.c', dependencies: [qemuutil]) +if have_system xbzrle_bench = executable('xbzrle-bench', sources: 'xbzrle-bench.c', dependencies: [qemuutil,migration]) +endif executable('atomic_add-bench', sources: files('atomic_add-bench.c'), This make it works. And no, I still not have a single clue how creating a new executable in tests/bench/ can make trace files not to be generated somewhere else. > meson.build only puts authz into trace_events_subdirs "if have_block" > (which is to say "if have_system or have_tools"). However the > bit of meson.build that says "subdir('authz') does not have > the same condition on it -- it's just been put in the list without > any condition on it. So I think that in a build-only-user-emulators > config meson will not generate trace events for the subdirectory > but will try to build it, which falls over. > > Contrast 'block', 'nbd', 'scsi', which are all guarded by > 'if have_block' for their subdir() lines, to match the guard on > the trace_events_subdirs. OTOH 'io' is also mismatched-guards... > > Why this only shows up with your pullreq I have no idea. xbzrle_bench. Notice that dependency on migration. As said, I compile every user and system targets that compile in fedora before I submit a PULL request. But it is getting clear than that is not enough anymore. Sorry about the noise. Will resubmit the PULL requset. Later, Juan.
On Fri, 10 Feb 2023 at 16:13, Juan Quintela <quintela@redhat.com> wrote: > Again, I don't know why it fails. > > diff --git a/tests/bench/meson.build b/tests/bench/meson.build > index daefead58d..7477a1f401 100644 > --- a/tests/bench/meson.build > +++ b/tests/bench/meson.build > @@ -3,9 +3,11 @@ qht_bench = executable('qht-bench', > sources: 'qht-bench.c', > dependencies: [qemuutil]) > > +if have_system > xbzrle_bench = executable('xbzrle-bench', > sources: 'xbzrle-bench.c', > dependencies: [qemuutil,migration]) > +endif > > executable('atomic_add-bench', > sources: files('atomic_add-bench.c'), > > This make it works. Before you added your test, meson had no need to compile any of the object files in 'migration', so it didn't. Now you tell meson to build a new executable, and it says "OK, I must build these object files". Only it turns out that they won't actually compile in this config, so you get an error. The same issue can happen in good old Make :-) thanks -- PMM
The following changes since commit 417296c8d8588f782018d01a317f88957e9786d6: tests/qtest/netdev-socket: Raise connection timeout to 60 seconds (2023-02-09 11:23:53 +0000) are available in the Git repository at: https://gitlab.com/juan.quintela/qemu.git tags/migration-20230209-pull-request for you to fetch changes up to 858191aebda251a4d1e3bc77b238096673241cdd: migration: Postpone postcopy preempt channel to be after main (2023-02-09 21:26:02 +0100) ---------------------------------------------------------------- Migration Pull request Hi This are all the reviewed patches for migration: - AVX512 support for xbzrle (Ling Xu) - /dev/userfaultd support (Peter Xu) - Improve ordering of channels (Peter Xu) - multifd cleanups (Li Zhang) - Remove spurious files from last merge (me) Rebase makes that to you - Fix mixup between state_pending_{exact,estimate} (me) - Cache RAM size during migration (me) - cleanup several functions (me) Please apply. ---------------------------------------------------------------- Juan Quintela (7): migration: Remove spurious files migration: Simplify ram_find_and_save_block() migration: Make find_dirty_block() return a single parameter migration: Split ram_bytes_total_common() in two functions migration: Calculate ram size once migration: Make ram_save_target_page() a pointer migration: I messed state_pending_exact/estimate Li Zhang (2): multifd: cleanup the function multifd_channel_connect multifd: Remove some redundant code Peter Xu (6): linux-headers: Update to v6.1 util/userfaultfd: Support /dev/userfaultfd migration: Rework multi-channel checks on URI migration: Cleanup postcopy_preempt_setup() migration: Add a semaphore to count PONGs migration: Postpone postcopy preempt channel to be after main ling xu (2): AVX512 support for xbzrle_encode_buffer Update bench-code for addressing CI problem MAINTAINERS | 1 + .../x86_64-quintela-devices.mak | 7 - .../x86_64-quintela2-devices.mak | 6 - meson.build | 17 + include/standard-headers/drm/drm_fourcc.h | 34 +- include/standard-headers/linux/ethtool.h | 63 +- include/standard-headers/linux/fuse.h | 6 +- .../linux/input-event-codes.h | 1 + include/standard-headers/linux/virtio_blk.h | 19 + linux-headers/asm-generic/hugetlb_encode.h | 26 +- linux-headers/asm-generic/mman-common.h | 2 + linux-headers/asm-mips/mman.h | 2 + linux-headers/asm-riscv/kvm.h | 4 + linux-headers/linux/kvm.h | 1 + linux-headers/linux/psci.h | 14 + linux-headers/linux/userfaultfd.h | 4 + linux-headers/linux/vfio.h | 142 ++ migration/migration.h | 15 +- migration/postcopy-ram.h | 4 +- migration/xbzrle.h | 4 + migration/migration.c | 122 +- migration/multifd.c | 70 +- migration/postcopy-ram.c | 31 +- migration/ram.c | 136 +- migration/savevm.c | 56 +- migration/xbzrle.c | 124 ++ tests/bench/xbzrle-bench.c | 473 ++++++ tests/unit/test-xbzrle.c | 42 +- util/userfaultfd.c | 32 + meson_options.txt | 2 + migration/multifd.c.orig | 1274 ----------------- scripts/meson-buildoptions.sh | 3 + tests/bench/meson.build | 4 + util/trace-events | 1 + 34 files changed, 1257 insertions(+), 1485 deletions(-) delete mode 100644 configs/devices/x86_64-softmmu/x86_64-quintela-devices.mak delete mode 100644 configs/devices/x86_64-softmmu/x86_64-quintela2-devices.mak create mode 100644 tests/bench/xbzrle-bench.c delete mode 100644 migration/multifd.c.orig