mbox

[PULL,00/15] Migration

Message ID 1475675109-8105-1-git-send-email-quintela@redhat.com
State New
Headers show

Pull-request

git://github.com/juanquintela/qemu.git tags/migration/20161005-1

Message

Juan Quintela Oct. 5, 2016, 1:44 p.m. UTC
From: Juan Quintela <quintela@trasno.org>

The following changes since commit bbc4c3f4f3c624e2de64fdcb79f4dd8c1a508e9d:

  Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging (2016-10-04 14:25:08 +0100)

are available in the git repository at:

  git://github.com/juanquintela/qemu.git tags/migration/20161005-1

for you to fetch changes up to c62da143b6792cc32dbd5db15b936d7f58cc36ee:

  docs/xbzrle: correction (2016-10-05 14:28:41 +0200)

----------------------------------------------------------------
migration/next for 20161005

Hi

This get:
- documentation fix: Cao
- improve max-bandwidth and downtime-limit (Ashijeet)
- move migration commands to "boxed" (Eric)
- rdma fixes (David)
- postcopy fixes (David)
- better errors (David)

Please apply

Thanks, Juan
----------------------------------------------------------------
Ashijeet Acharya (2):
      migrate: Fix bounds check for migration parameters in migration.c
      migrate: move max-bandwidth and downtime-limit to migrate_set_parameter

Cao jin (1):
      docs/xbzrle: correction

Dr. David Alan Gilbert (9):
      migration: report an error giving the failed field
      migration: Report values for comparisons
      migration/rdma: Pass qemu_file errors across link
      migration: Make failed migration load set file error
      migration/rdma: Don't flag an error when we've been told about one
      Postcopy vs xbzrle: Don't send xbzrle pages once in postcopy [for 2.8]
      RAMBlocks: Store page size
      migration/postcopy: Explicitly disallow huge pages
      migration: Fix seg with missing port

Eric Blake (3):
      migrate: Fix cpu-throttle-increment regression in HMP
      migrate: Share common MigrationParameters struct
      migrate: Use boxed qapi for migrate-set-parameters

 docs/qmp-commands.txt         |  13 +++-
 docs/xbzrle.txt               |   4 +-
 exec.c                        |  19 +++--
 hmp.c                         |  77 ++++++++++++++-------
 include/exec/cpu-common.h     |   1 +
 include/exec/ram_addr.h       |   1 +
 include/migration/migration.h |   1 -
 migration/migration.c         | 157 +++++++++++++++++++++++-------------------
 migration/postcopy-ram.c      |  24 +++++++
 migration/ram.c               |   4 +-
 migration/rdma.c              |   9 ++-
 migration/savevm.c            |  19 +++--
 migration/socket.c            |  16 +++--
 migration/vmstate.c           |  10 +++
 qapi-schema.json              | 109 +++++++++++++----------------
 15 files changed, 282 insertions(+), 182 deletions(-)

Comments

Peter Maydell Oct. 6, 2016, 10:59 a.m. UTC | #1
On 5 October 2016 at 14:44, Juan Quintela <quintela@redhat.com> wrote:
> From: Juan Quintela <quintela@trasno.org>
>
> The following changes since commit bbc4c3f4f3c624e2de64fdcb79f4dd8c1a508e9d:
>
>   Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging (2016-10-04 14:25:08 +0100)
>
> are available in the git repository at:
>
>   git://github.com/juanquintela/qemu.git tags/migration/20161005-1
>
> for you to fetch changes up to c62da143b6792cc32dbd5db15b936d7f58cc36ee:
>
>   docs/xbzrle: correction (2016-10-05 14:28:41 +0200)
>
> ----------------------------------------------------------------
> migration/next for 20161005
>
> Hi
>
> This get:
> - documentation fix: Cao
> - improve max-bandwidth and downtime-limit (Ashijeet)
> - move migration commands to "boxed" (Eric)
> - rdma fixes (David)
> - postcopy fixes (David)
> - better errors (David)
>
> Please apply

This runs into the "OSX %zu vs SIZE_MAX" issue again:

/Users/pm215/src/qemu-for-merges/migration/migration.c:810:61:
warning: format specifies type 'size_t' (aka 'unsigned long') but the
argument has type 'unsigned long long' [-Wformat]
                         " range of 0 to %zu bytes/second", SIZE_MAX);
                                         ~~~                ^~~~~~~~
                                         %llu
/Users/pm215/src/qemu-for-merges/include/qapi/error.h:163:35: note:
expanded from macro 'error_setg'
                        (fmt), ## __VA_ARGS__)
                                  ^
/usr/include/stdint.h:153:20: note: expanded from macro 'SIZE_MAX'
#define SIZE_MAX          UINT64_MAX
                          ^~~~~~~~~~
/usr/include/stdint.h:87:27: note: expanded from macro 'UINT64_MAX'
#define UINT64_MAX        18446744073709551615ULL
                          ^~~~~~~~~~~~~~~~~~~~~~~

In the absence of a suitable configure test and override of SIZE_MAX,
we've been adding explicit (uint64_t) casts in the printf as the
simplest workaround.

thanks
-- PMM
Juan Quintela Oct. 6, 2016, 4:09 p.m. UTC | #2
Peter Maydell <peter.maydell@linaro.org> wrote:
> On 5 October 2016 at 14:44, Juan Quintela <quintela@redhat.com> wrote:
>> From: Juan Quintela <quintela@trasno.org>
>>
>> The following changes since commit bbc4c3f4f3c624e2de64fdcb79f4dd8c1a508e9d:
>>
>>   Merge remote-tracking branch 'remotes/kevin/tags/for-upstream'
>> into staging (2016-10-04 14:25:08 +0100)
>>
>> are available in the git repository at:
>>
>>   git://github.com/juanquintela/qemu.git tags/migration/20161005-1
>>
>> for you to fetch changes up to c62da143b6792cc32dbd5db15b936d7f58cc36ee:
>>
>>   docs/xbzrle: correction (2016-10-05 14:28:41 +0200)
>>
>> ----------------------------------------------------------------
>> migration/next for 20161005
>>
>> Hi
>>
>> This get:
>> - documentation fix: Cao
>> - improve max-bandwidth and downtime-limit (Ashijeet)
>> - move migration commands to "boxed" (Eric)
>> - rdma fixes (David)
>> - postcopy fixes (David)
>> - better errors (David)
>>
>> Please apply
>
> This runs into the "OSX %zu vs SIZE_MAX" issue again:

Grrrr.
I already have trouble with %PR vs %zu for i386 vs anything else.

Anyone has a linux crosscompiler for osx that I can use?  Or there is a
way to get darwin or whatever virtual machine to test compile this?

It gets frustrating, before submissing I do:
- make check
- avocado virt test
- full compile on linux x86_64 (64 bits)
- full compile on linux i386 (32 bits)
- full cross-compile for windows 64bit
- full cross-compile for windows 32bit

(ok, I only test linux 64 bit), and I still don't get all the errors.

> /Users/pm215/src/qemu-for-merges/migration/migration.c:810:61:
> warning: format specifies type 'size_t' (aka 'unsigned long') but the
> argument has type 'unsigned long long' [-Wformat]
>                          " range of 0 to %zu bytes/second", SIZE_MAX);
>                                          ~~~                ^~~~~~~~
>                                          %llu
> /Users/pm215/src/qemu-for-merges/include/qapi/error.h:163:35: note:
> expanded from macro 'error_setg'
>                         (fmt), ## __VA_ARGS__)
>                                   ^
> /usr/include/stdint.h:153:20: note: expanded from macro 'SIZE_MAX'
> #define SIZE_MAX          UINT64_MAX
>                           ^~~~~~~~~~
> /usr/include/stdint.h:87:27: note: expanded from macro 'UINT64_MAX'
> #define UINT64_MAX        18446744073709551615ULL
>                           ^~~~~~~~~~~~~~~~~~~~~~~
>
> In the absence of a suitable configure test and override of SIZE_MAX,
> we've been adding explicit (uint64_t) casts in the printf as the
> simplest workaround.

Will do that, but as I have no way to compile test this, it will
continue hitting me :-(

Thanks for telling me.

Later, Juan.

PD.  Sorry if it sounded like a rant, just frustration O:-)
Peter Maydell Oct. 6, 2016, 4:19 p.m. UTC | #3
On 6 October 2016 at 17:09, Juan Quintela <quintela@redhat.com> wrote:
> Peter Maydell <peter.maydell@linaro.org> wrote:
>> On 5 October 2016 at 14:44, Juan Quintela <quintela@redhat.com> wrote:
>>> From: Juan Quintela <quintela@trasno.org>
>>>
>>> The following changes since commit bbc4c3f4f3c624e2de64fdcb79f4dd8c1a508e9d:
>>>
>>>   Merge remote-tracking branch 'remotes/kevin/tags/for-upstream'
>>> into staging (2016-10-04 14:25:08 +0100)
>>>
>>> are available in the git repository at:
>>>
>>>   git://github.com/juanquintela/qemu.git tags/migration/20161005-1
>>>
>>> for you to fetch changes up to c62da143b6792cc32dbd5db15b936d7f58cc36ee:
>>>
>>>   docs/xbzrle: correction (2016-10-05 14:28:41 +0200)
>>>
>>> ----------------------------------------------------------------
>>> migration/next for 20161005
>>>
>>> Hi
>>>
>>> This get:
>>> - documentation fix: Cao
>>> - improve max-bandwidth and downtime-limit (Ashijeet)
>>> - move migration commands to "boxed" (Eric)
>>> - rdma fixes (David)
>>> - postcopy fixes (David)
>>> - better errors (David)
>>>
>>> Please apply
>>
>> This runs into the "OSX %zu vs SIZE_MAX" issue again:
>
> Grrrr.
> I already have trouble with %PR vs %zu for i386 vs anything else.
>
> Anyone has a linux crosscompiler for osx that I can use?  Or there is a
> way to get darwin or whatever virtual machine to test compile this?

Travis builds include OSX, if you're prepared to wait around
for them to complete before submitting (they're usually about 10
hours wall-clock time to complete a build).

> It gets frustrating, before submissing I do:
> - make check
> - avocado virt test
> - full compile on linux x86_64 (64 bits)
> - full compile on linux i386 (32 bits)
> - full cross-compile for windows 64bit
> - full cross-compile for windows 32bit
>
> (ok, I only test linux 64 bit), and I still don't get all the errors.

This is why I do OSX builds in my merge tests, they do pick
up some stuff other builds don't.

thanks
-- PMM
Juan Quintela Oct. 6, 2016, 4:23 p.m. UTC | #4
Peter Maydell <peter.maydell@linaro.org> wrote:
> On 6 October 2016 at 17:09, Juan Quintela <quintela@redhat.com> wrote:
>> Peter Maydell <peter.maydell@linaro.org> wrote:
>>> On 5 October 2016 at 14:44, Juan Quintela <quintela@redhat.com> wrote:
>>>> From: Juan Quintela <quintela@trasno.org>
>>>>
>>>> The following changes since commit bbc4c3f4f3c624e2de64fdcb79f4dd8c1a508e9d:
>>>>
>>>>   Merge remote-tracking branch 'remotes/kevin/tags/for-upstream'
>>>> into staging (2016-10-04 14:25:08 +0100)
>>>>
>>>> are available in the git repository at:
>>>>
>>>>   git://github.com/juanquintela/qemu.git tags/migration/20161005-1
>>>>
>>>> for you to fetch changes up to c62da143b6792cc32dbd5db15b936d7f58cc36ee:
>>>>
>>>>   docs/xbzrle: correction (2016-10-05 14:28:41 +0200)
>>>>
>>>> ----------------------------------------------------------------
>>>> migration/next for 20161005
>>>>
>>>> Hi
>>>>
>>>> This get:
>>>> - documentation fix: Cao
>>>> - improve max-bandwidth and downtime-limit (Ashijeet)
>>>> - move migration commands to "boxed" (Eric)
>>>> - rdma fixes (David)
>>>> - postcopy fixes (David)
>>>> - better errors (David)
>>>>
>>>> Please apply
>>>
>>> This runs into the "OSX %zu vs SIZE_MAX" issue again:
>>
>> Grrrr.
>> I already have trouble with %PR vs %zu for i386 vs anything else.
>>
>> Anyone has a linux crosscompiler for osx that I can use?  Or there is a
>> way to get darwin or whatever virtual machine to test compile this?
>
> Travis builds include OSX, if you're prepared to wait around
> for them to complete before submitting (they're usually about 10
> hours wall-clock time to complete a build).

And I was complaining that each of my full builds after a rebase can
take around 45mins ..... O:-)  They are all local.

>
>> It gets frustrating, before submissing I do:
>> - make check
>> - avocado virt test
>> - full compile on linux x86_64 (64 bits)
>> - full compile on linux i386 (32 bits)
>> - full cross-compile for windows 64bit
>> - full cross-compile for windows 32bit
>>
>> (ok, I only test linux 64 bit), and I still don't get all the errors.
>
> This is why I do OSX builds in my merge tests, they do pick
> up some stuff other builds don't.
>
> thanks
> -- PMM

Later, Juan.
Peter Maydell Oct. 6, 2016, 4:30 p.m. UTC | #5
On 6 October 2016 at 17:23, Juan Quintela <quintela@redhat.com> wrote:
> Peter Maydell <peter.maydell@linaro.org> wrote:
>> Travis builds include OSX, if you're prepared to wait around
>> for them to complete before submitting (they're usually about 10
>> hours wall-clock time to complete a build).
>
> And I was complaining that each of my full builds after a rebase can
> take around 45mins ..... O:-)  They are all local.

None of the individual travis jobs take very long, I think it
just throttles our access to infrastructure and refuses to
parallelize us very much.

thanks
-- PMM
Peter Maydell Oct. 12, 2016, 10:04 a.m. UTC | #6
On 6 October 2016 at 11:59, Peter Maydell <peter.maydell@linaro.org> wrote:
> On 5 October 2016 at 14:44, Juan Quintela <quintela@redhat.com> wrote:
>> From: Juan Quintela <quintela@trasno.org>
>>
>> The following changes since commit bbc4c3f4f3c624e2de64fdcb79f4dd8c1a508e9d:
>>
>>   Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging (2016-10-04 14:25:08 +0100)
>>
>> are available in the git repository at:
>>
>>   git://github.com/juanquintela/qemu.git tags/migration/20161005-1
>>
>> for you to fetch changes up to c62da143b6792cc32dbd5db15b936d7f58cc36ee:
>>
>>   docs/xbzrle: correction (2016-10-05 14:28:41 +0200)
>>
>> ----------------------------------------------------------------
>> migration/next for 20161005
>>
>> Hi
>>
>> This get:
>> - documentation fix: Cao
>> - improve max-bandwidth and downtime-limit (Ashijeet)
>> - move migration commands to "boxed" (Eric)
>> - rdma fixes (David)
>> - postcopy fixes (David)
>> - better errors (David)
>>
>> Please apply
>
> This runs into the "OSX %zu vs SIZE_MAX" issue again

Since Eric has now kindly provided a proper configure test and
generic workaround for the SIZE_MAX bug and it is in master,
I tried just re-applying this pull request, but unfortunately
it has a merge conflict in qapi-schema.json that I don't
understand well enough to resolve. So if you could produce
a new pull request that would be best, but you don't need
to worry about the OSX failure any more.

thanks
-- PMM