| Message ID | 20240109064628.595453-8-peterx@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | docs/migration: Reorganize migration documentations | expand |
On 1/9/24 07:46, peterx@redhat.com wrote: > From: Peter Xu <peterx@redhat.com> > > Split that into a separate file, put under "features". > > Cc: Yong Huang <yong.huang@smartx.com> > Signed-off-by: Peter Xu <peterx@redhat.com> Reviewed-by: Cédric Le Goater <clg@redhat.com> Thanks, C. > --- > docs/devel/migration/dirty-limit.rst | 71 ++++++++++++++++++++++++++++ > docs/devel/migration/features.rst | 1 + > docs/devel/migration/main.rst | 71 ---------------------------- > 3 files changed, 72 insertions(+), 71 deletions(-) > create mode 100644 docs/devel/migration/dirty-limit.rst > > diff --git a/docs/devel/migration/dirty-limit.rst b/docs/devel/migration/dirty-limit.rst > new file mode 100644 > index 0000000000..8f32329d5f > --- /dev/null > +++ b/docs/devel/migration/dirty-limit.rst > @@ -0,0 +1,71 @@ > +Dirty limit > +=========== > + > +The dirty limit, short for dirty page rate upper limit, is a new capability > +introduced in the 8.1 QEMU release that uses a new algorithm based on the KVM > +dirty ring to throttle down the guest during live migration. > + > +The algorithm framework is as follows: > + > +:: > + > + ------------------------------------------------------------------------------ > + main --------------> throttle thread ------------> PREPARE(1) <-------- > + thread \ | | > + \ | | > + \ V | > + -\ CALCULATE(2) | > + \ | | > + \ | | > + \ V | > + \ SET PENALTY(3) ----- > + -\ | > + \ | > + \ V > + -> virtual CPU thread -------> ACCEPT PENALTY(4) > + ------------------------------------------------------------------------------ > + > +When the qmp command qmp_set_vcpu_dirty_limit is called for the first time, > +the QEMU main thread starts the throttle thread. The throttle thread, once > +launched, executes the loop, which consists of three steps: > + > + - PREPARE (1) > + > + The entire work of PREPARE (1) is preparation for the second stage, > + CALCULATE(2), as the name implies. It involves preparing the dirty > + page rate value and the corresponding upper limit of the VM: > + The dirty page rate is calculated via the KVM dirty ring mechanism, > + which tells QEMU how many dirty pages a virtual CPU has had since the > + last KVM_EXIT_DIRTY_RING_FULL exception; The dirty page rate upper > + limit is specified by caller, therefore fetch it directly. > + > + - CALCULATE (2) > + > + Calculate a suitable sleep period for each virtual CPU, which will be > + used to determine the penalty for the target virtual CPU. The > + computation must be done carefully in order to reduce the dirty page > + rate progressively down to the upper limit without oscillation. To > + achieve this, two strategies are provided: the first is to add or > + subtract sleep time based on the ratio of the current dirty page rate > + to the limit, which is used when the current dirty page rate is far > + from the limit; the second is to add or subtract a fixed time when > + the current dirty page rate is close to the limit. > + > + - SET PENALTY (3) > + > + Set the sleep time for each virtual CPU that should be penalized based > + on the results of the calculation supplied by step CALCULATE (2). > + > +After completing the three above stages, the throttle thread loops back > +to step PREPARE (1) until the dirty limit is reached. > + > +On the other hand, each virtual CPU thread reads the sleep duration and > +sleeps in the path of the KVM_EXIT_DIRTY_RING_FULL exception handler, that > +is ACCEPT PENALTY (4). Virtual CPUs tied with writing processes will > +obviously exit to the path and get penalized, whereas virtual CPUs involved > +with read processes will not. > + > +In summary, thanks to the KVM dirty ring technology, the dirty limit > +algorithm will restrict virtual CPUs as needed to keep their dirty page > +rate inside the limit. This leads to more steady reading performance during > +live migration and can aid in improving large guest responsiveness. > diff --git a/docs/devel/migration/features.rst b/docs/devel/migration/features.rst > index 0054e0c900..e257d0d100 100644 > --- a/docs/devel/migration/features.rst > +++ b/docs/devel/migration/features.rst > @@ -7,3 +7,4 @@ Migration has plenty of features to support different use cases. > :maxdepth: 2 > > postcopy > + dirty-limit > diff --git a/docs/devel/migration/main.rst b/docs/devel/migration/main.rst > index 051ea43f0e..00b9c3d32f 100644 > --- a/docs/devel/migration/main.rst > +++ b/docs/devel/migration/main.rst > @@ -573,74 +573,3 @@ path. > Return path - opened by main thread, written by main thread AND postcopy > thread (protected by rp_mutex) > > -Dirty limit > -===================== > -The dirty limit, short for dirty page rate upper limit, is a new capability > -introduced in the 8.1 QEMU release that uses a new algorithm based on the KVM > -dirty ring to throttle down the guest during live migration. > - > -The algorithm framework is as follows: > - > -:: > - > - ------------------------------------------------------------------------------ > - main --------------> throttle thread ------------> PREPARE(1) <-------- > - thread \ | | > - \ | | > - \ V | > - -\ CALCULATE(2) | > - \ | | > - \ | | > - \ V | > - \ SET PENALTY(3) ----- > - -\ | > - \ | > - \ V > - -> virtual CPU thread -------> ACCEPT PENALTY(4) > - ------------------------------------------------------------------------------ > - > -When the qmp command qmp_set_vcpu_dirty_limit is called for the first time, > -the QEMU main thread starts the throttle thread. The throttle thread, once > -launched, executes the loop, which consists of three steps: > - > - - PREPARE (1) > - > - The entire work of PREPARE (1) is preparation for the second stage, > - CALCULATE(2), as the name implies. It involves preparing the dirty > - page rate value and the corresponding upper limit of the VM: > - The dirty page rate is calculated via the KVM dirty ring mechanism, > - which tells QEMU how many dirty pages a virtual CPU has had since the > - last KVM_EXIT_DIRTY_RING_FULL exception; The dirty page rate upper > - limit is specified by caller, therefore fetch it directly. > - > - - CALCULATE (2) > - > - Calculate a suitable sleep period for each virtual CPU, which will be > - used to determine the penalty for the target virtual CPU. The > - computation must be done carefully in order to reduce the dirty page > - rate progressively down to the upper limit without oscillation. To > - achieve this, two strategies are provided: the first is to add or > - subtract sleep time based on the ratio of the current dirty page rate > - to the limit, which is used when the current dirty page rate is far > - from the limit; the second is to add or subtract a fixed time when > - the current dirty page rate is close to the limit. > - > - - SET PENALTY (3) > - > - Set the sleep time for each virtual CPU that should be penalized based > - on the results of the calculation supplied by step CALCULATE (2). > - > -After completing the three above stages, the throttle thread loops back > -to step PREPARE (1) until the dirty limit is reached. > - > -On the other hand, each virtual CPU thread reads the sleep duration and > -sleeps in the path of the KVM_EXIT_DIRTY_RING_FULL exception handler, that > -is ACCEPT PENALTY (4). Virtual CPUs tied with writing processes will > -obviously exit to the path and get penalized, whereas virtual CPUs involved > -with read processes will not. > - > -In summary, thanks to the KVM dirty ring technology, the dirty limit > -algorithm will restrict virtual CPUs as needed to keep their dirty page > -rate inside the limit. This leads to more steady reading performance during > -live migration and can aid in improving large guest responsiveness. > -
diff --git a/docs/devel/migration/dirty-limit.rst b/docs/devel/migration/dirty-limit.rst new file mode 100644 index 0000000000..8f32329d5f --- /dev/null +++ b/docs/devel/migration/dirty-limit.rst @@ -0,0 +1,71 @@ +Dirty limit +=========== + +The dirty limit, short for dirty page rate upper limit, is a new capability +introduced in the 8.1 QEMU release that uses a new algorithm based on the KVM +dirty ring to throttle down the guest during live migration. + +The algorithm framework is as follows: + +:: + + ------------------------------------------------------------------------------ + main --------------> throttle thread ------------> PREPARE(1) <-------- + thread \ | | + \ | | + \ V | + -\ CALCULATE(2) | + \ | | + \ | | + \ V | + \ SET PENALTY(3) ----- + -\ | + \ | + \ V + -> virtual CPU thread -------> ACCEPT PENALTY(4) + ------------------------------------------------------------------------------ + +When the qmp command qmp_set_vcpu_dirty_limit is called for the first time, +the QEMU main thread starts the throttle thread. The throttle thread, once +launched, executes the loop, which consists of three steps: + + - PREPARE (1) + + The entire work of PREPARE (1) is preparation for the second stage, + CALCULATE(2), as the name implies. It involves preparing the dirty + page rate value and the corresponding upper limit of the VM: + The dirty page rate is calculated via the KVM dirty ring mechanism, + which tells QEMU how many dirty pages a virtual CPU has had since the + last KVM_EXIT_DIRTY_RING_FULL exception; The dirty page rate upper + limit is specified by caller, therefore fetch it directly. + + - CALCULATE (2) + + Calculate a suitable sleep period for each virtual CPU, which will be + used to determine the penalty for the target virtual CPU. The + computation must be done carefully in order to reduce the dirty page + rate progressively down to the upper limit without oscillation. To + achieve this, two strategies are provided: the first is to add or + subtract sleep time based on the ratio of the current dirty page rate + to the limit, which is used when the current dirty page rate is far + from the limit; the second is to add or subtract a fixed time when + the current dirty page rate is close to the limit. + + - SET PENALTY (3) + + Set the sleep time for each virtual CPU that should be penalized based + on the results of the calculation supplied by step CALCULATE (2). + +After completing the three above stages, the throttle thread loops back +to step PREPARE (1) until the dirty limit is reached. + +On the other hand, each virtual CPU thread reads the sleep duration and +sleeps in the path of the KVM_EXIT_DIRTY_RING_FULL exception handler, that +is ACCEPT PENALTY (4). Virtual CPUs tied with writing processes will +obviously exit to the path and get penalized, whereas virtual CPUs involved +with read processes will not. + +In summary, thanks to the KVM dirty ring technology, the dirty limit +algorithm will restrict virtual CPUs as needed to keep their dirty page +rate inside the limit. This leads to more steady reading performance during +live migration and can aid in improving large guest responsiveness. diff --git a/docs/devel/migration/features.rst b/docs/devel/migration/features.rst index 0054e0c900..e257d0d100 100644 --- a/docs/devel/migration/features.rst +++ b/docs/devel/migration/features.rst @@ -7,3 +7,4 @@ Migration has plenty of features to support different use cases. :maxdepth: 2 postcopy + dirty-limit diff --git a/docs/devel/migration/main.rst b/docs/devel/migration/main.rst index 051ea43f0e..00b9c3d32f 100644 --- a/docs/devel/migration/main.rst +++ b/docs/devel/migration/main.rst @@ -573,74 +573,3 @@ path. Return path - opened by main thread, written by main thread AND postcopy thread (protected by rp_mutex) -Dirty limit -===================== -The dirty limit, short for dirty page rate upper limit, is a new capability -introduced in the 8.1 QEMU release that uses a new algorithm based on the KVM -dirty ring to throttle down the guest during live migration. - -The algorithm framework is as follows: - -:: - - ------------------------------------------------------------------------------ - main --------------> throttle thread ------------> PREPARE(1) <-------- - thread \ | | - \ | | - \ V | - -\ CALCULATE(2) | - \ | | - \ | | - \ V | - \ SET PENALTY(3) ----- - -\ | - \ | - \ V - -> virtual CPU thread -------> ACCEPT PENALTY(4) - ------------------------------------------------------------------------------ - -When the qmp command qmp_set_vcpu_dirty_limit is called for the first time, -the QEMU main thread starts the throttle thread. The throttle thread, once -launched, executes the loop, which consists of three steps: - - - PREPARE (1) - - The entire work of PREPARE (1) is preparation for the second stage, - CALCULATE(2), as the name implies. It involves preparing the dirty - page rate value and the corresponding upper limit of the VM: - The dirty page rate is calculated via the KVM dirty ring mechanism, - which tells QEMU how many dirty pages a virtual CPU has had since the - last KVM_EXIT_DIRTY_RING_FULL exception; The dirty page rate upper - limit is specified by caller, therefore fetch it directly. - - - CALCULATE (2) - - Calculate a suitable sleep period for each virtual CPU, which will be - used to determine the penalty for the target virtual CPU. The - computation must be done carefully in order to reduce the dirty page - rate progressively down to the upper limit without oscillation. To - achieve this, two strategies are provided: the first is to add or - subtract sleep time based on the ratio of the current dirty page rate - to the limit, which is used when the current dirty page rate is far - from the limit; the second is to add or subtract a fixed time when - the current dirty page rate is close to the limit. - - - SET PENALTY (3) - - Set the sleep time for each virtual CPU that should be penalized based - on the results of the calculation supplied by step CALCULATE (2). - -After completing the three above stages, the throttle thread loops back -to step PREPARE (1) until the dirty limit is reached. - -On the other hand, each virtual CPU thread reads the sleep duration and -sleeps in the path of the KVM_EXIT_DIRTY_RING_FULL exception handler, that -is ACCEPT PENALTY (4). Virtual CPUs tied with writing processes will -obviously exit to the path and get penalized, whereas virtual CPUs involved -with read processes will not. - -In summary, thanks to the KVM dirty ring technology, the dirty limit -algorithm will restrict virtual CPUs as needed to keep their dirty page -rate inside the limit. This leads to more steady reading performance during -live migration and can aid in improving large guest responsiveness. -