| Message ID | 168853615963.17240.15832775267134683267-4@git.sr.ht |
|---|---|
| State | New |
| Headers | show |
| Series | migration: introduce dirtylimit capability | expand |
~hyman <hyman@git.sr.ht> writes: > From: Hyman Huang(黄勇) <yong.huang@smartx.com> > > Introduce migration dirty-limit capability, which can > be turned on before live migration and limit dirty > page rate durty live migration. > > Introduce migrate_dirty_limit function to help check > if dirty-limit capability enabled during live migration. > > Meanwhile, refactor vcpu_dirty_rate_stat_collect > so that period can be configured instead of hardcoded. > > dirty-limit capability is kind of like auto-converge > but using dirty limit instead of traditional cpu-throttle > to throttle guest down. To enable this feature, turn on > the dirty-limit capability before live migration using > migrate-set-capabilities, and set the parameters > "x-vcpu-dirty-limit-period", "vcpu-dirty-limit" suitably > to speed up convergence. > > Signed-off-by: Hyman Huang(黄勇) <yong.huang@smartx.com> > Acked-by: Peter Xu <peterx@redhat.com> > Reviewed-by: Juan Quintela <quintela@redhat.com> [...] > diff --git a/migration/options.h b/migration/options.h > index 9aaf363322..b5a950d4e4 100644 > --- a/migration/options.h > +++ b/migration/options.h > @@ -24,6 +24,7 @@ extern Property migration_properties[]; > /* capabilities */ > > bool migrate_auto_converge(void); > +bool migrate_dirty_limit(void); > bool migrate_background_snapshot(void); > bool migrate_block(void); > bool migrate_colo(void); > diff --git a/qapi/migration.json b/qapi/migration.json > index aa590dbf0e..cc51835cdd 100644 > --- a/qapi/migration.json > +++ b/qapi/migration.json > @@ -497,6 +497,16 @@ > # are present. 'return-path' capability must be enabled to use > # it. (since 8.1) > # > +# @dirty-limit: If enabled, migration will use the dirty-limit algo to > +# throttle down guest instead of auto-converge algo. > +# Throttle algo only works when vCPU's dirtyrate greater > +# than 'vcpu-dirty-limit', read processes in guest os > +# aren't penalized any more, so this algo can improve > +# performance of vCPU during live migration. This is an > +# optional performance feature and should not affect the > +# correctness of the existing auto-converge algo. > +# (since 8.1) > +# Please format like # @dirty-limit: If enabled, migration will use the dirty-limit algo to # throttle down guest instead of auto-converge algo. Throttle # algo only works when vCPU's dirtyrate greater than # 'vcpu-dirty-limit', read processes in guest os aren't penalized # any more, so this algo can improve performance of vCPU during # live migration. This is an optional performance feature and # should not affect the correctness of the existing auto-converge # algo. (since 8.1) to blend in with recent commit a937b6aa739 (qapi: Reformat doc comments to conform to current conventions). "Dirty rate" with a space, because that's how we spell it elsewhere. Moreover, "algo" is not a word, "algorithm" is :) Is "the dirty-limit algorithm" defined anywhere? More word-smithing is needed, but I'd like to get the reference to "the dirty-limit algorithm" clarified first. > # Features: > # > # @unstable: Members @x-colo and @x-ignore-shared are experimental. > @@ -512,7 +522,8 @@ > 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', > { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, > 'validate-uuid', 'background-snapshot', > - 'zero-copy-send', 'postcopy-preempt', 'switchover-ack'] } > + 'zero-copy-send', 'postcopy-preempt', 'switchover-ack', > + 'dirty-limit'] } > > ## > # @MigrationCapabilityStatus: [...]
On Thu, Jul 6, 2023 at 10:59 PM Markus Armbruster <armbru@redhat.com> wrote: > ~hyman <hyman@git.sr.ht> writes: > > > From: Hyman Huang(黄勇) <yong.huang@smartx.com> > > > > Introduce migration dirty-limit capability, which can > > be turned on before live migration and limit dirty > > page rate durty live migration. > > > > Introduce migrate_dirty_limit function to help check > > if dirty-limit capability enabled during live migration. > > > > Meanwhile, refactor vcpu_dirty_rate_stat_collect > > so that period can be configured instead of hardcoded. > > > > dirty-limit capability is kind of like auto-converge > > but using dirty limit instead of traditional cpu-throttle > > to throttle guest down. To enable this feature, turn on > > the dirty-limit capability before live migration using > > migrate-set-capabilities, and set the parameters > > "x-vcpu-dirty-limit-period", "vcpu-dirty-limit" suitably > > to speed up convergence. > > > > Signed-off-by: Hyman Huang(黄勇) <yong.huang@smartx.com> > > Acked-by: Peter Xu <peterx@redhat.com> > > Reviewed-by: Juan Quintela <quintela@redhat.com> > > [...] > > > diff --git a/migration/options.h b/migration/options.h > > index 9aaf363322..b5a950d4e4 100644 > > --- a/migration/options.h > > +++ b/migration/options.h > > @@ -24,6 +24,7 @@ extern Property migration_properties[]; > > /* capabilities */ > > > > bool migrate_auto_converge(void); > > +bool migrate_dirty_limit(void); > > bool migrate_background_snapshot(void); > > bool migrate_block(void); > > bool migrate_colo(void); > > diff --git a/qapi/migration.json b/qapi/migration.json > > index aa590dbf0e..cc51835cdd 100644 > > --- a/qapi/migration.json > > +++ b/qapi/migration.json > > @@ -497,6 +497,16 @@ > > # are present. 'return-path' capability must be enabled to use > > # it. (since 8.1) > > # > > +# @dirty-limit: If enabled, migration will use the dirty-limit algo to > > +# throttle down guest instead of auto-converge algo. > > +# Throttle algo only works when vCPU's dirtyrate greater > > +# than 'vcpu-dirty-limit', read processes in guest os > > +# aren't penalized any more, so this algo can improve > > +# performance of vCPU during live migration. This is an > > +# optional performance feature and should not affect the > > +# correctness of the existing auto-converge algo. > > +# (since 8.1) > > +# > > Please format like > > # @dirty-limit: If enabled, migration will use the dirty-limit algo to > # throttle down guest instead of auto-converge algo. Throttle > # algo only works when vCPU's dirtyrate greater than > # 'vcpu-dirty-limit', read processes in guest os aren't penalized > # any more, so this algo can improve performance of vCPU during > # live migration. This is an optional performance feature and > # should not affect the correctness of the existing auto-converge > # algo. (since 8.1) > > to blend in with recent commit a937b6aa739 (qapi: Reformat doc comments > to conform to current conventions). > > "Dirty rate" with a space, because that's how we spell it elsewhere. > > Moreover, "algo" is not a word, "algorithm" is :) > > Is "the dirty-limit algorithm" defined anywhere? > > More word-smithing is needed, but I'd like to get the reference to "the > dirty-limit algorithm" clarified first. > Dirty limit algorithm just uses the existing internal implementation of qmp api "set-vcpu-dirty-limit" to implement the throttle algorithm of live migration. So the capability is named "dirty-limit" :) More details about dirty-limit algorithm can be referenced as following two commits: https://lore.kernel.org/qemu-devel/20220719170221.576190-8-dgilbert@redhat.com/ https://lore.kernel.org/qemu-devel/20220719170221.576190-7-dgilbert@redhat.com/ > > > # Features: > > # > > # @unstable: Members @x-colo and @x-ignore-shared are experimental. > > @@ -512,7 +522,8 @@ > > 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', > > { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, > > 'validate-uuid', 'background-snapshot', > > - 'zero-copy-send', 'postcopy-preempt', 'switchover-ack'] } > > + 'zero-copy-send', 'postcopy-preempt', 'switchover-ack', > > + 'dirty-limit'] } > > > > ## > > # @MigrationCapabilityStatus: > > [...] > >
diff --git a/migration/options.c b/migration/options.c index 7d2d98830e..8b4eb8c519 100644 --- a/migration/options.c +++ b/migration/options.c @@ -27,6 +27,7 @@ #include "qemu-file.h" #include "ram.h" #include "options.h" +#include "sysemu/kvm.h" /* Maximum migrate downtime set to 2000 seconds */ #define MAX_MIGRATE_DOWNTIME_SECONDS 2000 @@ -196,6 +197,8 @@ Property migration_properties[] = { #endif DEFINE_PROP_MIG_CAP("x-switchover-ack", MIGRATION_CAPABILITY_SWITCHOVER_ACK), + DEFINE_PROP_MIG_CAP("x-dirty-limit", + MIGRATION_CAPABILITY_DIRTY_LIMIT), DEFINE_PROP_END_OF_LIST(), }; @@ -242,6 +245,13 @@ bool migrate_dirty_bitmaps(void) return s->capabilities[MIGRATION_CAPABILITY_DIRTY_BITMAPS]; } +bool migrate_dirty_limit(void) +{ + MigrationState *s = migrate_get_current(); + + return s->capabilities[MIGRATION_CAPABILITY_DIRTY_LIMIT]; +} + bool migrate_events(void) { MigrationState *s = migrate_get_current(); @@ -573,6 +583,20 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) } } + if (new_caps[MIGRATION_CAPABILITY_DIRTY_LIMIT]) { + if (new_caps[MIGRATION_CAPABILITY_AUTO_CONVERGE]) { + error_setg(errp, "dirty-limit conflicts with auto-converge" + " either of then available currently"); + return false; + } + + if (!kvm_enabled() || !kvm_dirty_ring_enabled()) { + error_setg(errp, "dirty-limit requires KVM with accelerator" + " property 'dirty-ring-size' set"); + return false; + } + } + return true; } diff --git a/migration/options.h b/migration/options.h index 9aaf363322..b5a950d4e4 100644 --- a/migration/options.h +++ b/migration/options.h @@ -24,6 +24,7 @@ extern Property migration_properties[]; /* capabilities */ bool migrate_auto_converge(void); +bool migrate_dirty_limit(void); bool migrate_background_snapshot(void); bool migrate_block(void); bool migrate_colo(void); diff --git a/qapi/migration.json b/qapi/migration.json index aa590dbf0e..cc51835cdd 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -497,6 +497,16 @@ # are present. 'return-path' capability must be enabled to use # it. (since 8.1) # +# @dirty-limit: If enabled, migration will use the dirty-limit algo to +# throttle down guest instead of auto-converge algo. +# Throttle algo only works when vCPU's dirtyrate greater +# than 'vcpu-dirty-limit', read processes in guest os +# aren't penalized any more, so this algo can improve +# performance of vCPU during live migration. This is an +# optional performance feature and should not affect the +# correctness of the existing auto-converge algo. +# (since 8.1) +# # Features: # # @unstable: Members @x-colo and @x-ignore-shared are experimental. @@ -512,7 +522,8 @@ 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, 'validate-uuid', 'background-snapshot', - 'zero-copy-send', 'postcopy-preempt', 'switchover-ack'] } + 'zero-copy-send', 'postcopy-preempt', 'switchover-ack', + 'dirty-limit'] } ## # @MigrationCapabilityStatus: diff --git a/softmmu/dirtylimit.c b/softmmu/dirtylimit.c index 5c12d26d49..953ef934bc 100644 --- a/softmmu/dirtylimit.c +++ b/softmmu/dirtylimit.c @@ -24,6 +24,9 @@ #include "hw/boards.h" #include "sysemu/kvm.h" #include "trace.h" +#include "migration/misc.h" +#include "migration/migration.h" +#include "migration/options.h" /* * Dirtylimit stop working if dirty page rate error @@ -75,11 +78,18 @@ static bool dirtylimit_quit; static void vcpu_dirty_rate_stat_collect(void) { + MigrationState *s = migrate_get_current(); VcpuStat stat; int i = 0; + int64_t period = DIRTYLIMIT_CALC_TIME_MS; + + if (migrate_dirty_limit() && + migration_is_active(s)) { + period = s->parameters.x_vcpu_dirty_limit_period; + } /* calculate vcpu dirtyrate */ - vcpu_calculate_dirtyrate(DIRTYLIMIT_CALC_TIME_MS, + vcpu_calculate_dirtyrate(period, &stat, GLOBAL_DIRTY_LIMIT, false);