| Message ID | 20190416173216.9028-3-mathieu.desnoyers@efficios.com |
|---|---|
| State | New |
| Headers | show |
| Series | Restartable Sequences support for glibc 2.30 | expand |
On 16/04/2019 18:32, Mathieu Desnoyers wrote: > --- a/sysdeps/unix/sysv/linux/sched_getcpu.c > +++ b/sysdeps/unix/sysv/linux/sched_getcpu.c > @@ -37,3 +37,26 @@ sched_getcpu (void) > return -1; > #endif > } > + > +#ifdef __NR_rseq > +#include <sys/rseq.h> > +#endif > + > +#if defined __NR_rseq && defined RSEQ_SIG > +extern __attribute__ ((tls_model ("initial-exec"))) > +__thread volatile struct rseq __rseq_abi; i'd expect sys/rseq.h to provide this declaration. > + > +int > +sched_getcpu (void) > +{ > + int cpu_id = __rseq_abi.cpu_id; > + > + return cpu_id >= 0 ? cpu_id : vsyscall_sched_getcpu (); > +} > +#else > +int > +sched_getcpu (void) > +{ > + return vsyscall_sched_getcpu (); > +} > +#endif > -- 2.17.1 >
----- On Apr 18, 2019, at 11:33 AM, Szabolcs Nagy Szabolcs.Nagy@arm.com wrote: > On 16/04/2019 18:32, Mathieu Desnoyers wrote: >> --- a/sysdeps/unix/sysv/linux/sched_getcpu.c >> +++ b/sysdeps/unix/sysv/linux/sched_getcpu.c >> @@ -37,3 +37,26 @@ sched_getcpu (void) >> return -1; >> #endif >> } >> + >> +#ifdef __NR_rseq >> +#include <sys/rseq.h> >> +#endif >> + >> +#if defined __NR_rseq && defined RSEQ_SIG >> +extern __attribute__ ((tls_model ("initial-exec"))) >> +__thread volatile struct rseq __rseq_abi; > > i'd expect sys/rseq.h to provide this declaration. And it actually does! Will remove this duplicate. Thanks, Mathieu > >> + >> +int >> +sched_getcpu (void) >> +{ >> + int cpu_id = __rseq_abi.cpu_id; >> + >> + return cpu_id >= 0 ? cpu_id : vsyscall_sched_getcpu (); >> +} >> +#else >> +int >> +sched_getcpu (void) >> +{ >> + return vsyscall_sched_getcpu (); >> +} >> +#endif >> -- 2.17.1
diff --git a/sysdeps/unix/sysv/linux/sched_getcpu.c b/sysdeps/unix/sysv/linux/sched_getcpu.c index fb0d317f83..f9466c3b22 100644 --- a/sysdeps/unix/sysv/linux/sched_getcpu.c +++ b/sysdeps/unix/sysv/linux/sched_getcpu.c @@ -24,8 +24,8 @@ #endif #include <sysdep-vdso.h> -int -sched_getcpu (void) +static int +vsyscall_sched_getcpu (void) { #ifdef __NR_getcpu unsigned int cpu; @@ -37,3 +37,26 @@ sched_getcpu (void) return -1; #endif } + +#ifdef __NR_rseq +#include <sys/rseq.h> +#endif + +#if defined __NR_rseq && defined RSEQ_SIG +extern __attribute__ ((tls_model ("initial-exec"))) +__thread volatile struct rseq __rseq_abi; + +int +sched_getcpu (void) +{ + int cpu_id = __rseq_abi.cpu_id; + + return cpu_id >= 0 ? cpu_id : vsyscall_sched_getcpu (); +} +#else +int +sched_getcpu (void) +{ + return vsyscall_sched_getcpu (); +} +#endif
When available, use the cpu_id field from __rseq_abi on Linux to implement sched_getcpu(). Fall-back on the vgetcpu vDSO if unavailable. Benchmarks: x86-64: Intel E5-2630 v3@2.40GHz, 16-core, hyperthreading glibc sched_getcpu(): 13.7 ns (baseline) glibc sched_getcpu() using rseq: 2.5 ns (speedup: 5.5x) inline load cpuid from __rseq_abi TLS: 0.8 ns (speedup: 17.1x) Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Carlos O'Donell <carlos@redhat.com> CC: Florian Weimer <fweimer@redhat.com> CC: Joseph Myers <joseph@codesourcery.com> CC: Szabolcs Nagy <szabolcs.nagy@arm.com> CC: Thomas Gleixner <tglx@linutronix.de> CC: Ben Maurer <bmaurer@fb.com> CC: Peter Zijlstra <peterz@infradead.org> CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Will Deacon <will.deacon@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Paul Turner <pjt@google.com> CC: libc-alpha@sourceware.org CC: linux-kernel@vger.kernel.org CC: linux-api@vger.kernel.org --- Changes since v1: - rseq is only used if both __NR_rseq and RSEQ_SIG are defined. --- sysdeps/unix/sysv/linux/sched_getcpu.c | 27 ++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-)