mbox

[PULL,00/26] migration and friends queue

Message ID 20200925120655.295142-1-dgilbert@redhat.com
State New
Headers show

Pull-request

git://github.com/dagrh/qemu.git tags/pull-migration-20200925a

Message

Dr. David Alan Gilbert Sept. 25, 2020, 12:06 p.m. UTC
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>

The following changes since commit 8c1c07929feae876202ba26f07a540c5115c18cd:

  Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging (2020-09-24 18:48:45 +0100)

are available in the Git repository at:

  git://github.com/dagrh/qemu.git tags/pull-migration-20200925a

for you to fetch changes up to e12a0edafeb5019aac74114b62a4703f79c5c693:

  virtiofsd: Add -o allow_direct_io|no_allow_direct_io options (2020-09-25 12:45:58 +0100)

----------------------------------------------------------------
Migration and virtiofsd pull

Chuan Zheng's Dirtyrate and TLS changes, with small fixes from Dov and
Luarent and Peter.
Small virtiofs changes from Harry, Stefan, Vivek and Jiachen.
One HMP/monitor rework from me.

----------------------------------------------------------------
Chuan Zheng (18):
      migration/dirtyrate: setup up query-dirtyrate framwork
      migration/dirtyrate: add DirtyRateStatus to denote calculation status
      migration/dirtyrate: Add RamblockDirtyInfo to store sampled page info
      migration/dirtyrate: Add dirtyrate statistics series functions
      migration/dirtyrate: move RAMBLOCK_FOREACH_MIGRATABLE into ram.h
      migration/dirtyrate: Record hash results for each sampled page
      migration/dirtyrate: Compare page hash results for recorded sampled page
      migration/dirtyrate: skip sampling ramblock with size below MIN_RAMBLOCK_SIZE
      migration/dirtyrate: Implement set_sample_page_period() and is_sample_period_valid()
      migration/dirtyrate: Implement calculate_dirtyrate() function
      migration/dirtyrate: Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function
      migration/dirtyrate: Add trace_calls to make it easier to debug
      migration/tls: save hostname into MigrationState
      migration/tls: extract migration_tls_client_create for common-use
      migration/tls: add tls_hostname into MultiFDSendParams
      migration/tls: extract cleanup function for common-use
      migration/tls: add support for multifd tls-handshake
      migration/tls: add trace points for multifd-tls

Dov Murik (1):
      migration: Truncate state file in xen-save-devices-state

Dr. David Alan Gilbert (1):
      monitor: Use LOCK_GUARD macros

Harry G. Coin (1):
      virtiofsd: document cache=auto default

Jiachen Zhang (1):
      virtiofsd: Add -o allow_direct_io|no_allow_direct_io options

Laurent Vivier (1):
      migration: increase max-bandwidth to 128 MiB/s (1 Gib/s)

Peter Xu (2):
      migration: Properly destroy variables on incoming side
      migration: Rework migrate_send_rp_req_pages() function

Vivek Goyal (1):
      virtiofsd: Used glib "shared" thread pool

 docs/tools/virtiofsd.rst              |   1 +
 migration/channel.c                   |   1 +
 migration/dirtyrate.c                 | 426 ++++++++++++++++++++++++++++++++++
 migration/dirtyrate.h                 |  69 ++++++
 migration/meson.build                 |   2 +-
 migration/migration.c                 |  36 ++-
 migration/migration.h                 |   9 +-
 migration/multifd.c                   | 124 ++++++++--
 migration/multifd.h                   |   2 +
 migration/postcopy-ram.c              |  24 +-
 migration/ram.c                       |  11 +-
 migration/ram.h                       |  10 +
 migration/savevm.c                    |   3 +-
 migration/tls.c                       |  28 ++-
 migration/tls.h                       |   6 +
 migration/trace-events                |  12 +
 monitor/misc.c                        |  44 ++--
 qapi/migration.json                   |  67 ++++++
 tools/virtiofsd/fuse_virtio.c         |   2 +-
 tools/virtiofsd/helper.c              |   4 +
 tools/virtiofsd/passthrough_ll.c      |  20 +-
 tools/virtiofsd/passthrough_seccomp.c |   2 +
 22 files changed, 797 insertions(+), 106 deletions(-)
 create mode 100644 migration/dirtyrate.c
 create mode 100644 migration/dirtyrate.h

Comments

no-reply@patchew.org Sept. 25, 2020, 12:35 p.m. UTC | #1
Patchew URL: https://patchew.org/QEMU/20200925120655.295142-1-dgilbert@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20200925120655.295142-1-dgilbert@redhat.com
Subject: [PULL 00/26] migration and friends queue

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag]         patchew/20200925120655.295142-1-dgilbert@redhat.com -> patchew/20200925120655.295142-1-dgilbert@redhat.com
Switched to a new branch 'test'
786afbf virtiofsd: Add -o allow_direct_io|no_allow_direct_io options
b4e3d2f virtiofsd: Used glib "shared" thread pool
5c6cf53 virtiofsd: document cache=auto default
b0a5d54 monitor: Use LOCK_GUARD macros
88b31e8 migration/tls: add trace points for multifd-tls
42dd844 migration/tls: add support for multifd tls-handshake
0dd99f4 migration/tls: extract cleanup function for common-use
e951359 migration/tls: add tls_hostname into MultiFDSendParams
652dea8 migration/tls: extract migration_tls_client_create for common-use
d70fc72 migration/tls: save hostname into MigrationState
cb0c3a3 migration: increase max-bandwidth to 128 MiB/s (1 Gib/s)
b4f2d50 migration: Truncate state file in xen-save-devices-state
7448415 migration/dirtyrate: Add trace_calls to make it easier to debug
aa44e3a migration/dirtyrate: Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function
9a1f005 migration/dirtyrate: Implement calculate_dirtyrate() function
f1bfba7 migration/dirtyrate: Implement set_sample_page_period() and is_sample_period_valid()
cb3b7ff migration/dirtyrate: skip sampling ramblock with size below MIN_RAMBLOCK_SIZE
a82b646 migration/dirtyrate: Compare page hash results for recorded sampled page
7d0f4d2 migration/dirtyrate: Record hash results for each sampled page
8a90389 migration/dirtyrate: move RAMBLOCK_FOREACH_MIGRATABLE into ram.h
539c0d7 migration/dirtyrate: Add dirtyrate statistics series functions
421892b migration/dirtyrate: Add RamblockDirtyInfo to store sampled page info
cab1614 migration/dirtyrate: add DirtyRateStatus to denote calculation status
83826ff migration/dirtyrate: setup up query-dirtyrate framwork
c58f028 migration: Rework migrate_send_rp_req_pages() function
012226f migration: Properly destroy variables on incoming side

=== OUTPUT BEGIN ===
1/26 Checking commit 012226fcacb7 (migration: Properly destroy variables on incoming side)
2/26 Checking commit c58f0286fdde (migration: Rework migrate_send_rp_req_pages() function)
WARNING: Block comments use a leading /* on a separate line
#32: FILE: migration/migration.c:317:
+/* Request one page from the source VM at the given start address.

total: 0 errors, 1 warnings, 89 lines checked

Patch 2/26 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
3/26 Checking commit 83826ffcaf74 (migration/dirtyrate: setup up query-dirtyrate framwork)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#20: 
new file mode 100644

total: 0 errors, 1 warnings, 71 lines checked

Patch 3/26 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
4/26 Checking commit cab161480dd1 (migration/dirtyrate: add DirtyRateStatus to denote calculation status)
5/26 Checking commit 421892b2ce34 (migration/dirtyrate: Add RamblockDirtyInfo to store sampled page info)
6/26 Checking commit 539c0d7abcf0 (migration/dirtyrate: Add dirtyrate statistics series functions)
7/26 Checking commit 8a9038992bc0 (migration/dirtyrate: move RAMBLOCK_FOREACH_MIGRATABLE into ram.h)
ERROR: Macros with multiple statements should be enclosed in a do - while loop
#67: FILE: migration/ram.h:42:
+#define RAMBLOCK_FOREACH_NOT_IGNORED(block)            \
+    INTERNAL_RAMBLOCK_FOREACH(block)                   \
+        if (ramblock_is_ignored(block)) {} else

ERROR: trailing statements should be on next line
#69: FILE: migration/ram.h:44:
+        if (ramblock_is_ignored(block)) {} else

ERROR: Macros with multiple statements should be enclosed in a do - while loop
#71: FILE: migration/ram.h:46:
+#define RAMBLOCK_FOREACH_MIGRATABLE(block)             \
+    INTERNAL_RAMBLOCK_FOREACH(block)                   \
+        if (!qemu_ram_is_migratable(block)) {} else

ERROR: trailing statements should be on next line
#73: FILE: migration/ram.h:48:
+        if (!qemu_ram_is_migratable(block)) {} else

ERROR: braces {} are necessary for all arms of this statement
#73: FILE: migration/ram.h:48:
+        if (!qemu_ram_is_migratable(block)) {} else
[...]
+        if (!qemu_ram_is_migratable(block)) {} else
[...]

total: 5 errors, 0 warnings, 45 lines checked

Patch 7/26 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

8/26 Checking commit 7d0f4d22333b (migration/dirtyrate: Record hash results for each sampled page)
9/26 Checking commit a82b6464e8aa (migration/dirtyrate: Compare page hash results for recorded sampled page)
10/26 Checking commit cb3b7ff83ca3 (migration/dirtyrate: skip sampling ramblock with size below MIN_RAMBLOCK_SIZE)
11/26 Checking commit f1bfba70ecdd (migration/dirtyrate: Implement set_sample_page_period() and is_sample_period_valid())
12/26 Checking commit 9a1f0050a876 (migration/dirtyrate: Implement calculate_dirtyrate() function)
13/26 Checking commit aa44e3a85c97 (migration/dirtyrate: Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function)
14/26 Checking commit 74484157cf26 (migration/dirtyrate: Add trace_calls to make it easier to debug)
15/26 Checking commit b4f2d50c15a8 (migration: Truncate state file in xen-save-devices-state)
16/26 Checking commit cb0c3a374335 (migration: increase max-bandwidth to 128 MiB/s (1 Gib/s))
17/26 Checking commit d70fc7217cc7 (migration/tls: save hostname into MigrationState)
18/26 Checking commit 652dea809a4b (migration/tls: extract migration_tls_client_create for common-use)
19/26 Checking commit e951359067b9 (migration/tls: add tls_hostname into MultiFDSendParams)
20/26 Checking commit 0dd99f4db197 (migration/tls: extract cleanup function for common-use)
21/26 Checking commit 42dd8447efd6 (migration/tls: add support for multifd tls-handshake)
22/26 Checking commit 88b31e8e1583 (migration/tls: add trace points for multifd-tls)
23/26 Checking commit b0a5d54e6e3e (monitor: Use LOCK_GUARD macros)
24/26 Checking commit 5c6cf5345c0f (virtiofsd: document cache=auto default)
25/26 Checking commit b4e3d2f9afca (virtiofsd: Used glib "shared" thread pool)
26/26 Checking commit 786afbf48ae9 (virtiofsd: Add -o allow_direct_io|no_allow_direct_io options)
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20200925120655.295142-1-dgilbert@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Dr. David Alan Gilbert Sept. 25, 2020, 2:31 p.m. UTC | #2
* no-reply@patchew.org (no-reply@patchew.org) wrote:
> Patchew URL: https://patchew.org/QEMU/20200925120655.295142-1-dgilbert@redhat.com/
> 
> 
> 
> Hi,
> 
> This series seems to have some coding style problems. See output below for
> more information:
> 
> Type: series
> Message-id: 20200925120655.295142-1-dgilbert@redhat.com
> Subject: [PULL 00/26] migration and friends queue
> 
> === TEST SCRIPT BEGIN ===
> #!/bin/bash
> git rev-parse base > /dev/null || exit 0
> git config --local diff.renamelimit 0
> git config --local diff.renames True
> git config --local diff.algorithm histogram
> ./scripts/checkpatch.pl --mailback base..
> === TEST SCRIPT END ===
> 
> Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
> From https://github.com/patchew-project/qemu
>  * [new tag]         patchew/20200925120655.295142-1-dgilbert@redhat.com -> patchew/20200925120655.295142-1-dgilbert@redhat.com
> Switched to a new branch 'test'
> 786afbf virtiofsd: Add -o allow_direct_io|no_allow_direct_io options
> b4e3d2f virtiofsd: Used glib "shared" thread pool
> 5c6cf53 virtiofsd: document cache=auto default
> b0a5d54 monitor: Use LOCK_GUARD macros
> 88b31e8 migration/tls: add trace points for multifd-tls
> 42dd844 migration/tls: add support for multifd tls-handshake
> 0dd99f4 migration/tls: extract cleanup function for common-use
> e951359 migration/tls: add tls_hostname into MultiFDSendParams
> 652dea8 migration/tls: extract migration_tls_client_create for common-use
> d70fc72 migration/tls: save hostname into MigrationState
> cb0c3a3 migration: increase max-bandwidth to 128 MiB/s (1 Gib/s)
> b4f2d50 migration: Truncate state file in xen-save-devices-state
> 7448415 migration/dirtyrate: Add trace_calls to make it easier to debug
> aa44e3a migration/dirtyrate: Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function
> 9a1f005 migration/dirtyrate: Implement calculate_dirtyrate() function
> f1bfba7 migration/dirtyrate: Implement set_sample_page_period() and is_sample_period_valid()
> cb3b7ff migration/dirtyrate: skip sampling ramblock with size below MIN_RAMBLOCK_SIZE
> a82b646 migration/dirtyrate: Compare page hash results for recorded sampled page
> 7d0f4d2 migration/dirtyrate: Record hash results for each sampled page
> 8a90389 migration/dirtyrate: move RAMBLOCK_FOREACH_MIGRATABLE into ram.h
> 539c0d7 migration/dirtyrate: Add dirtyrate statistics series functions
> 421892b migration/dirtyrate: Add RamblockDirtyInfo to store sampled page info
> cab1614 migration/dirtyrate: add DirtyRateStatus to denote calculation status
> 83826ff migration/dirtyrate: setup up query-dirtyrate framwork
> c58f028 migration: Rework migrate_send_rp_req_pages() function
> 012226f migration: Properly destroy variables on incoming side
> 
> === OUTPUT BEGIN ===
> 1/26 Checking commit 012226fcacb7 (migration: Properly destroy variables on incoming side)
> 2/26 Checking commit c58f0286fdde (migration: Rework migrate_send_rp_req_pages() function)
> WARNING: Block comments use a leading /* on a separate line
> #32: FILE: migration/migration.c:317:
> +/* Request one page from the source VM at the given start address.
> 
> total: 0 errors, 1 warnings, 89 lines checked

This is just changing the text of an existing comment that's of slightly
the wrong format.


> 
> Patch 2/26 has style problems, please review.  If any of these errors
> are false positives report them to the maintainer, see
> CHECKPATCH in MAINTAINERS.
> 3/26 Checking commit 83826ffcaf74 (migration/dirtyrate: setup up query-dirtyrate framwork)
> WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
> #20: 
> new file mode 100644
> 
> total: 0 errors, 1 warnings, 71 lines checked
> 
> Patch 3/26 has style problems, please review.  If any of these errors
> are false positives report them to the maintainer, see
> CHECKPATCH in MAINTAINERS.
> 4/26 Checking commit cab161480dd1 (migration/dirtyrate: add DirtyRateStatus to denote calculation status)
> 5/26 Checking commit 421892b2ce34 (migration/dirtyrate: Add RamblockDirtyInfo to store sampled page info)
> 6/26 Checking commit 539c0d7abcf0 (migration/dirtyrate: Add dirtyrate statistics series functions)
> 7/26 Checking commit 8a9038992bc0 (migration/dirtyrate: move RAMBLOCK_FOREACH_MIGRATABLE into ram.h)
> ERROR: Macros with multiple statements should be enclosed in a do - while loop
> #67: FILE: migration/ram.h:42:
> +#define RAMBLOCK_FOREACH_NOT_IGNORED(block)            \
> +    INTERNAL_RAMBLOCK_FOREACH(block)                   \
> +        if (ramblock_is_ignored(block)) {} else

This is just moving an existing macro and not changing it.

> ERROR: trailing statements should be on next line
> #69: FILE: migration/ram.h:44:
> +        if (ramblock_is_ignored(block)) {} else
> 
> ERROR: Macros with multiple statements should be enclosed in a do - while loop
> #71: FILE: migration/ram.h:46:
> +#define RAMBLOCK_FOREACH_MIGRATABLE(block)             \
> +    INTERNAL_RAMBLOCK_FOREACH(block)                   \
> +        if (!qemu_ram_is_migratable(block)) {} else
> 
> ERROR: trailing statements should be on next line
> #73: FILE: migration/ram.h:48:
> +        if (!qemu_ram_is_migratable(block)) {} else
> 
> ERROR: braces {} are necessary for all arms of this statement
> #73: FILE: migration/ram.h:48:
> +        if (!qemu_ram_is_migratable(block)) {} else
> [...]
> +        if (!qemu_ram_is_migratable(block)) {} else
> [...]
> 
> total: 5 errors, 0 warnings, 45 lines checked
> 
> Patch 7/26 has style problems, please review.  If any of these errors
> are false positives report them to the maintainer, see
> CHECKPATCH in MAINTAINERS.
> 
> 8/26 Checking commit 7d0f4d22333b (migration/dirtyrate: Record hash results for each sampled page)
> 9/26 Checking commit a82b6464e8aa (migration/dirtyrate: Compare page hash results for recorded sampled page)
> 10/26 Checking commit cb3b7ff83ca3 (migration/dirtyrate: skip sampling ramblock with size below MIN_RAMBLOCK_SIZE)
> 11/26 Checking commit f1bfba70ecdd (migration/dirtyrate: Implement set_sample_page_period() and is_sample_period_valid())
> 12/26 Checking commit 9a1f0050a876 (migration/dirtyrate: Implement calculate_dirtyrate() function)
> 13/26 Checking commit aa44e3a85c97 (migration/dirtyrate: Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function)
> 14/26 Checking commit 74484157cf26 (migration/dirtyrate: Add trace_calls to make it easier to debug)
> 15/26 Checking commit b4f2d50c15a8 (migration: Truncate state file in xen-save-devices-state)
> 16/26 Checking commit cb0c3a374335 (migration: increase max-bandwidth to 128 MiB/s (1 Gib/s))
> 17/26 Checking commit d70fc7217cc7 (migration/tls: save hostname into MigrationState)
> 18/26 Checking commit 652dea809a4b (migration/tls: extract migration_tls_client_create for common-use)
> 19/26 Checking commit e951359067b9 (migration/tls: add tls_hostname into MultiFDSendParams)
> 20/26 Checking commit 0dd99f4db197 (migration/tls: extract cleanup function for common-use)
> 21/26 Checking commit 42dd8447efd6 (migration/tls: add support for multifd tls-handshake)
> 22/26 Checking commit 88b31e8e1583 (migration/tls: add trace points for multifd-tls)
> 23/26 Checking commit b0a5d54e6e3e (monitor: Use LOCK_GUARD macros)
> 24/26 Checking commit 5c6cf5345c0f (virtiofsd: document cache=auto default)
> 25/26 Checking commit b4e3d2f9afca (virtiofsd: Used glib "shared" thread pool)
> 26/26 Checking commit 786afbf48ae9 (virtiofsd: Add -o allow_direct_io|no_allow_direct_io options)
> === OUTPUT END ===
> 
> Test command exited with code: 1
> 
> 
> The full log is available at
> http://patchew.org/logs/20200925120655.295142-1-dgilbert@redhat.com/testing.checkpatch/?type=message.
> ---
> Email generated automatically by Patchew [https://patchew.org/].
> Please send your feedback to patchew-devel@redhat.com
Peter Maydell Sept. 25, 2020, 4:22 p.m. UTC | #3
On Fri, 25 Sep 2020 at 13:09, Dr. David Alan Gilbert (git)
<dgilbert@redhat.com> wrote:
>
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>
> The following changes since commit 8c1c07929feae876202ba26f07a540c5115c18cd:
>
>   Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging (2020-09-24 18:48:45 +0100)
>
> are available in the Git repository at:
>
>   git://github.com/dagrh/qemu.git tags/pull-migration-20200925a
>
> for you to fetch changes up to e12a0edafeb5019aac74114b62a4703f79c5c693:
>
>   virtiofsd: Add -o allow_direct_io|no_allow_direct_io options (2020-09-25 12:45:58 +0100)
>
> ----------------------------------------------------------------
> Migration and virtiofsd pull
>
> Chuan Zheng's Dirtyrate and TLS changes, with small fixes from Dov and
> Luarent and Peter.
> Small virtiofs changes from Harry, Stefan, Vivek and Jiachen.
> One HMP/monitor rework from me.
>


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.2
for any user-visible changes.

-- PMM