| Message ID | 20220518133935.3878954-4-npiggin@gmail.com (mailing list archive) |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | [1/4] KVM: PPC: Book3S HV P9: Restore stolen time logging in dtl | expand |
| Context | Check | Description |
|---|---|---|
| snowpatch_ozlabs/github-powerpc_ppctests | success | Successfully ran 10 jobs. |
| snowpatch_ozlabs/github-powerpc_selftests | success | Successfully ran 10 jobs. |
| snowpatch_ozlabs/github-powerpc_kernel_qemu | success | Successfully ran 24 jobs. |
| snowpatch_ozlabs/github-powerpc_sparse | success | Successfully ran 4 jobs. |
| snowpatch_ozlabs/github-powerpc_clang | success | Successfully ran 7 jobs. |
Nicholas Piggin <npiggin@gmail.com> writes: > CONFIG_VIRT_CPU_ACCOUNTING_GEN under pseries does not implement > stolen time accounting. Implement it with the paravirt time > accounting option. > > Signed-off-by: Nicholas Piggin <npiggin@gmail.com> > --- > .../admin-guide/kernel-parameters.txt | 6 +++--- > arch/powerpc/include/asm/paravirt.h | 12 ++++++++++++ > arch/powerpc/platforms/pseries/Kconfig | 8 ++++++++ > arch/powerpc/platforms/pseries/lpar.c | 11 +++++++++++ > arch/powerpc/platforms/pseries/setup.c | 19 +++++++++++++++++++ > 5 files changed, 53 insertions(+), 3 deletions(-) > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt > index 3f1cc5e317ed..855fc7b02261 100644 > --- a/Documentation/admin-guide/kernel-parameters.txt > +++ b/Documentation/admin-guide/kernel-parameters.txt > @@ -3604,9 +3604,9 @@ > [X86,PV_OPS] Disable paravirtualized VMware scheduler > clock and use the default one. > > - no-steal-acc [X86,PV_OPS,ARM64] Disable paravirtualized steal time > - accounting. steal time is computed, but won't > - influence scheduler behaviour > + no-steal-acc [X86,PV_OPS,ARM64,PPC/PSERIES] Disable paravirtualized > + steal time accounting. steal time is computed, but > + won't influence scheduler behaviour > > nolapic [X86-32,APIC] Do not enable or use the local APIC. > > diff --git a/arch/powerpc/include/asm/paravirt.h b/arch/powerpc/include/asm/paravirt.h > index eb7df559ae74..f5ba1a3c41f8 100644 > --- a/arch/powerpc/include/asm/paravirt.h > +++ b/arch/powerpc/include/asm/paravirt.h > @@ -21,6 +21,18 @@ static inline bool is_shared_processor(void) > return static_branch_unlikely(&shared_processor); > } > > +#ifdef CONFIG_PARAVIRT_TIME_ACCOUNTING > +extern struct static_key paravirt_steal_enabled; > +extern struct static_key paravirt_steal_rq_enabled; > + > +u64 pseries_paravirt_steal_clock(int cpu); > + > +static inline u64 paravirt_steal_clock(int cpu) > +{ > + return pseries_paravirt_steal_clock(cpu); > +} > +#endif > + > /* If bit 0 is set, the cpu has been ceded, conferred, or preempted */ > static inline u32 yield_count_of(int cpu) > { > diff --git a/arch/powerpc/platforms/pseries/Kconfig b/arch/powerpc/platforms/pseries/Kconfig > index f7fd91d153a4..d4306ebdca5e 100644 > --- a/arch/powerpc/platforms/pseries/Kconfig > +++ b/arch/powerpc/platforms/pseries/Kconfig > @@ -24,13 +24,21 @@ config PPC_PSERIES > select SWIOTLB > default y > > +config PARAVIRT > + bool > + In file included from ../kernel/sched/build_utility.c:53: ../kernel/sched/sched.h:87:11: fatal error: asm/paravirt_api_clock.h: No such file or directory 87 | # include <asm/paravirt_api_clock.h> $ find . -name paravirt_api_clock.h ./arch/arm64/include/asm/paravirt_api_clock.h ./arch/x86/include/asm/paravirt_api_clock.h ./arch/arm/include/asm/paravirt_api_clock.h > config PARAVIRT_SPINLOCKS > bool > > +config PARAVIRT_TIME_ACCOUNTING > + select PARAVIRT > + bool > + > config PPC_SPLPAR > bool "Support for shared-processor logical partitions" > depends on PPC_PSERIES > select PARAVIRT_SPINLOCKS if PPC_QUEUED_SPINLOCKS > + select PARAVIRT_TIME_ACCOUNTING if VIRT_CPU_ACCOUNTING_GEN > default y > help > Enabling this option will make the kernel run more efficiently
On 5/18/22 7:09 PM, Nicholas Piggin wrote: > CONFIG_VIRT_CPU_ACCOUNTING_GEN under pseries does not implement > stolen time accounting. Implement it with the paravirt time > accounting option. > > Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Tested-by: Shrikanth Hegde <sshegde@linux.ibm.com> Patch fails to compile with CONFIG_PARAVIRT=y with below error. In file included from kernel/sched/core.c:81: kernel/sched/sched.h:87:11: fatal error: asm/paravirt_api_clock.h: No such file or directory 87 | # include <asm/paravirt_api_clock.h> compilation terminated. after adding the file, it compiled. Please add the file as well. patch i did. diff --git a/arch/powerpc/include/asm/paravirt_api_clock.h b/arch/powerpc/include/asm/paravirt_api_clock.h new file mode 100644 index 000000000000..65ac7cee0dad --- /dev/null +++ b/arch/powerpc/include/asm/paravirt_api_clock.h @@ -0,0 +1 @@ +#include <asm/paravirt.h> After successful compilation, it was tested on Power10 Shared LPAR. system has two LPAR. we will call first one LPAR1 and second one as LPAR2. Test was carried out in SMT=1. Similar observation was seen in SMT=8 as well. LPAR config header from each LPAR is below. LPAR1 is twice as big as LPAR2. Since Both are sharing the same underlying hardware, work stealing will happen when both the LPAR's are contending for the same resource. LPAR1: type=Shared mode=Uncapped smt=Off lcpu=40 mem=2094637056 kB cpus=40 ent=20.00 LPAR2: type=Shared mode=Uncapped smt=Off lcpu=20 mem=2083908608 kB cpus=40 ent=10.00 mpstat was used to check for the utilization. stress-ng has been used as the workload. Few cases are tested. when the both LPAR are idle there is no steal time. when LPAR1 starts running at 100% which consumes all of the physical resource, steal time starts to get accounted. With LPAR1 running at 100% and LPAR2 starts running, steal time starts increasing. This is as expected. When the LPAR2 Load is increased further, steal time increases further. Case 1: 0% LPAR1; 0% LPAR2 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle all 0.00 0.00 0.05 0.00 0.00 0.00 0.00 0.00 0.00 99.95 Case 2: 100% LPAR1; 0% LPAR2 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle all 97.68 0.00 0.00 0.00 0.00 0.00 2.32 0.00 0.00 0.00 Case 3: 100% LPAR1; 50% LPAR2 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle all 86.34 0.00 0.10 0.00 0.00 0.03 13.54 0.00 0.00 0.00 Case 4: 100% LPAR1; 100% LPAR2 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle all 78.54 0.00 0.07 0.00 0.00 0.02 21.36 0.00 0.00 0.00 Case 5: 50% LPAR1; 100% LPAR2 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle all 49.37 0.00 0.00 0.00 0.00 0.00 1.17 0.00 0.00 49.47 Patch is accounting for the steal time and basic tests are holding good. -- Shrikanth Hegde
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 3f1cc5e317ed..855fc7b02261 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3604,9 +3604,9 @@ [X86,PV_OPS] Disable paravirtualized VMware scheduler clock and use the default one. - no-steal-acc [X86,PV_OPS,ARM64] Disable paravirtualized steal time - accounting. steal time is computed, but won't - influence scheduler behaviour + no-steal-acc [X86,PV_OPS,ARM64,PPC/PSERIES] Disable paravirtualized + steal time accounting. steal time is computed, but + won't influence scheduler behaviour nolapic [X86-32,APIC] Do not enable or use the local APIC. diff --git a/arch/powerpc/include/asm/paravirt.h b/arch/powerpc/include/asm/paravirt.h index eb7df559ae74..f5ba1a3c41f8 100644 --- a/arch/powerpc/include/asm/paravirt.h +++ b/arch/powerpc/include/asm/paravirt.h @@ -21,6 +21,18 @@ static inline bool is_shared_processor(void) return static_branch_unlikely(&shared_processor); } +#ifdef CONFIG_PARAVIRT_TIME_ACCOUNTING +extern struct static_key paravirt_steal_enabled; +extern struct static_key paravirt_steal_rq_enabled; + +u64 pseries_paravirt_steal_clock(int cpu); + +static inline u64 paravirt_steal_clock(int cpu) +{ + return pseries_paravirt_steal_clock(cpu); +} +#endif + /* If bit 0 is set, the cpu has been ceded, conferred, or preempted */ static inline u32 yield_count_of(int cpu) { diff --git a/arch/powerpc/platforms/pseries/Kconfig b/arch/powerpc/platforms/pseries/Kconfig index f7fd91d153a4..d4306ebdca5e 100644 --- a/arch/powerpc/platforms/pseries/Kconfig +++ b/arch/powerpc/platforms/pseries/Kconfig @@ -24,13 +24,21 @@ config PPC_PSERIES select SWIOTLB default y +config PARAVIRT + bool + config PARAVIRT_SPINLOCKS bool +config PARAVIRT_TIME_ACCOUNTING + select PARAVIRT + bool + config PPC_SPLPAR bool "Support for shared-processor logical partitions" depends on PPC_PSERIES select PARAVIRT_SPINLOCKS if PPC_QUEUED_SPINLOCKS + select PARAVIRT_TIME_ACCOUNTING if VIRT_CPU_ACCOUNTING_GEN default y help Enabling this option will make the kernel run more efficiently diff --git a/arch/powerpc/platforms/pseries/lpar.c b/arch/powerpc/platforms/pseries/lpar.c index 760581c5752f..1965b7d7d8f1 100644 --- a/arch/powerpc/platforms/pseries/lpar.c +++ b/arch/powerpc/platforms/pseries/lpar.c @@ -661,6 +661,17 @@ static int __init vcpudispatch_stats_procfs_init(void) } machine_device_initcall(pseries, vcpudispatch_stats_procfs_init); + +#ifdef CONFIG_PARAVIRT_TIME_ACCOUNTING +u64 pseries_paravirt_steal_clock(int cpu) +{ + struct lppaca *lppaca = &lppaca_of(cpu); + + return be64_to_cpu(READ_ONCE(lppaca->enqueue_dispatch_tb)) + + be64_to_cpu(READ_ONCE(lppaca->ready_enqueue_tb)); +} +#endif + #endif /* CONFIG_PPC_SPLPAR */ void vpa_init(int cpu) diff --git a/arch/powerpc/platforms/pseries/setup.c b/arch/powerpc/platforms/pseries/setup.c index 955ff8aa1644..691c9add4a5a 100644 --- a/arch/powerpc/platforms/pseries/setup.c +++ b/arch/powerpc/platforms/pseries/setup.c @@ -78,6 +78,20 @@ DEFINE_STATIC_KEY_FALSE(shared_processor); EXPORT_SYMBOL(shared_processor); +#ifdef CONFIG_PARAVIRT_TIME_ACCOUNTING +struct static_key paravirt_steal_enabled; +struct static_key paravirt_steal_rq_enabled; + +static bool steal_acc = true; +static int __init parse_no_stealacc(char *arg) +{ + steal_acc = false; + return 0; +} + +early_param("no-steal-acc", parse_no_stealacc); +#endif + int CMO_PrPSP = -1; int CMO_SecPSP = -1; unsigned long CMO_PageSize = (ASM_CONST(1) << IOMMU_PAGE_SHIFT_4K); @@ -831,6 +845,11 @@ static void __init pSeries_setup_arch(void) if (lppaca_shared_proc(get_lppaca())) { static_branch_enable(&shared_processor); pv_spinlocks_init(); +#ifdef CONFIG_PARAVIRT_TIME_ACCOUNTING + static_key_slow_inc(¶virt_steal_enabled); + if (steal_acc) + static_key_slow_inc(¶virt_steal_rq_enabled); +#endif } ppc_md.power_save = pseries_lpar_idle;
CONFIG_VIRT_CPU_ACCOUNTING_GEN under pseries does not implement stolen time accounting. Implement it with the paravirt time accounting option. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> --- .../admin-guide/kernel-parameters.txt | 6 +++--- arch/powerpc/include/asm/paravirt.h | 12 ++++++++++++ arch/powerpc/platforms/pseries/Kconfig | 8 ++++++++ arch/powerpc/platforms/pseries/lpar.c | 11 +++++++++++ arch/powerpc/platforms/pseries/setup.c | 19 +++++++++++++++++++ 5 files changed, 53 insertions(+), 3 deletions(-)