[v2] arch/powerpc: Rework local_paca to avoid LTO warnings

From: Alastair D'Silva <alastair@d-silva.org>

When building an LTO kernel, the existing code generates warnings:
    ./arch/powerpc/include/asm/paca.h:37:30: warning: register of
        ‘local_paca’ used for multiple global register variables
     register struct paca_struct *local_paca asm("r13");
                                  ^
    ./arch/powerpc/include/asm/paca.h:37:30: note: conflicts with
        ‘local_paca’

This patch reworks local_paca into an inline getter & setter function,
which addresses the warning.

Changelog:
V2
  - Address whitespace issues
  - keep new implementation close to where the old implementation was

Signed-off-by: Alastair D'Silva <alastair@d-silva.org>
---
 arch/powerpc/include/asm/paca.h | 37 +++++++++++++++++++++++++--------
 arch/powerpc/kernel/paca.c      |  2 +-
 2 files changed, 29 insertions(+), 10 deletions(-)

Le 14/03/2019 à 03:31, Alastair D'Silva a écrit :
> From: Alastair D'Silva <alastair@d-silva.org>
> 
> When building an LTO kernel, the existing code generates warnings:
>      ./arch/powerpc/include/asm/paca.h:37:30: warning: register of
>          ‘local_paca’ used for multiple global register variables
>       register struct paca_struct *local_paca asm("r13");
>                                    ^
>      ./arch/powerpc/include/asm/paca.h:37:30: note: conflicts with
>          ‘local_paca’
> 
> This patch reworks local_paca into an inline getter & setter function,
> which addresses the warning.
> 
> Changelog:
> V2
>    - Address whitespace issues
>    - keep new implementation close to where the old implementation was
> 
> Signed-off-by: Alastair D'Silva <alastair@d-silva.org>

Reviewed-by: Christophe Leroy <christophe.leroy@c-s.fr>

> ---
>   arch/powerpc/include/asm/paca.h | 37 +++++++++++++++++++++++++--------
>   arch/powerpc/kernel/paca.c      |  2 +-
>   2 files changed, 29 insertions(+), 10 deletions(-)
> 
> diff --git a/arch/powerpc/include/asm/paca.h b/arch/powerpc/include/asm/paca.h
> index e843bc5d1a0f..2fa0b43357c9 100644
> --- a/arch/powerpc/include/asm/paca.h
> +++ b/arch/powerpc/include/asm/paca.h
> @@ -34,19 +34,38 @@
>   #include <asm/cpuidle.h>
>   #include <asm/atomic.h>
>   
> -register struct paca_struct *local_paca asm("r13");
> -
>   #if defined(CONFIG_DEBUG_PREEMPT) && defined(CONFIG_SMP)
>   extern unsigned int debug_smp_processor_id(void); /* from linux/smp.h */
> -/*
> - * Add standard checks that preemption cannot occur when using get_paca():
> - * otherwise the paca_struct it points to may be the wrong one just after.
> - */
> -#define get_paca()	((void) debug_smp_processor_id(), local_paca)
> -#else
> -#define get_paca()	local_paca
>   #endif
>   
> +static inline struct paca_struct *get_paca_no_preempt_check(void)
> +{
> +	register struct paca_struct *paca asm("r13");
> +
> +	return paca;
> +}
> +
> +static inline struct paca_struct *get_paca(void)
> +{
> +#if defined(CONFIG_DEBUG_PREEMPT) && defined(CONFIG_SMP)
> +	/*
> +	 * Add standard checks that preemption cannot occur when using get_paca():
> +	 * otherwise the paca_struct it points to may be the wrong one just after.
> +	 */
> +	debug_smp_processor_id();
> +#endif
> +	return get_paca_no_preempt_check();
> +}
> +
> +#define local_paca	get_paca_no_preempt_check()
> +
> +static inline void set_paca(struct paca_struct *new)
> +{
> +	register struct paca_struct *paca asm("r13");
> +
> +	paca = new;
> +}
> +
>   #ifdef CONFIG_PPC_PSERIES
>   #define get_lppaca()	(get_paca()->lppaca_ptr)
>   #endif
> diff --git a/arch/powerpc/kernel/paca.c b/arch/powerpc/kernel/paca.c
> index 913bfca09c4f..ae5c243f9d5a 100644
> --- a/arch/powerpc/kernel/paca.c
> +++ b/arch/powerpc/kernel/paca.c
> @@ -172,7 +172,7 @@ void __init initialise_paca(struct paca_struct *new_paca, int cpu)
>   void setup_paca(struct paca_struct *new_paca)
>   {
>   	/* Setup r13 */
> -	local_paca = new_paca;
> +	set_paca(new_paca);
>   
>   #ifdef CONFIG_PPC_BOOK3E
>   	/* On Book3E, initialize the TLB miss exception frames */
>

Alastair D'Silva's on March 14, 2019 12:31 pm:
> From: Alastair D'Silva <alastair@d-silva.org>
> 
> When building an LTO kernel, the existing code generates warnings:
>     ./arch/powerpc/include/asm/paca.h:37:30: warning: register of
>         ‘local_paca’ used for multiple global register variables
>      register struct paca_struct *local_paca asm("r13");
>                                   ^
>     ./arch/powerpc/include/asm/paca.h:37:30: note: conflicts with
>         ‘local_paca’

Isn't this a bogus warning? It doesn't look like there's a way to 
define it any other way.

> 
> This patch reworks local_paca into an inline getter & setter function,
> which addresses the warning.
> 
> Changelog:
> V2
>   - Address whitespace issues
>   - keep new implementation close to where the old implementation was
> 
> Signed-off-by: Alastair D'Silva <alastair@d-silva.org>
> ---
>  arch/powerpc/include/asm/paca.h | 37 +++++++++++++++++++++++++--------
>  arch/powerpc/kernel/paca.c      |  2 +-
>  2 files changed, 29 insertions(+), 10 deletions(-)
> 
> diff --git a/arch/powerpc/include/asm/paca.h b/arch/powerpc/include/asm/paca.h
> index e843bc5d1a0f..2fa0b43357c9 100644
> --- a/arch/powerpc/include/asm/paca.h
> +++ b/arch/powerpc/include/asm/paca.h
> @@ -34,19 +34,38 @@
>  #include <asm/cpuidle.h>
>  #include <asm/atomic.h>
>  
> -register struct paca_struct *local_paca asm("r13");
> -
>  #if defined(CONFIG_DEBUG_PREEMPT) && defined(CONFIG_SMP)
>  extern unsigned int debug_smp_processor_id(void); /* from linux/smp.h */
> -/*
> - * Add standard checks that preemption cannot occur when using get_paca():
> - * otherwise the paca_struct it points to may be the wrong one just after.
> - */
> -#define get_paca()	((void) debug_smp_processor_id(), local_paca)
> -#else
> -#define get_paca()	local_paca
>  #endif
>  
> +static inline struct paca_struct *get_paca_no_preempt_check(void)
> +{
> +	register struct paca_struct *paca asm("r13");
> +
> +	return paca;
> +}

Problem is it now changes the global register variable to a local 
register variable. The compiler would presumably be within its rights
to "cache" that return value or use another register for it, which
is not really what we want.

Thanks,
Nick

On Tue, 2019-03-26 at 15:58 +1000, Nicholas Piggin wrote:
> Alastair D'Silva's on March 14, 2019 12:31 pm:
> > From: Alastair D'Silva <alastair@d-silva.org>
> > 
> > When building an LTO kernel, the existing code generates warnings:
> >     ./arch/powerpc/include/asm/paca.h:37:30: warning: register of
> >         ‘local_paca’ used for multiple global register variables
> >      register struct paca_struct *local_paca asm("r13");
> >                                   ^
> >     ./arch/powerpc/include/asm/paca.h:37:30: note: conflicts with
> >         ‘local_paca’
> 
> Isn't this a bogus warning? It doesn't look like there's a way to 
> define it any other way.

There isn't any other way to define it as a global. However, the
warning is legitimate.

The compiler sees that there are multiple global register variables,
all pointing at the same register.

The compiler can only determine this when LTO is used, as otherwise it
only sees the one in the current compilation unit, whicd disappears by
the time the kernel is linked.

> 
> > This patch reworks local_paca into an inline getter & setter
> > function,
> > which addresses the warning.
> > 
> > Changelog:
> > V2
> >   - Address whitespace issues
> >   - keep new implementation close to where the old implementation
> > was
> > 
> > Signed-off-by: Alastair D'Silva <alastair@d-silva.org>
> > ---
> >  arch/powerpc/include/asm/paca.h | 37 +++++++++++++++++++++++++--
> > ------
> >  arch/powerpc/kernel/paca.c      |  2 +-
> >  2 files changed, 29 insertions(+), 10 deletions(-)
> > 
> > diff --git a/arch/powerpc/include/asm/paca.h
> > b/arch/powerpc/include/asm/paca.h
> > index e843bc5d1a0f..2fa0b43357c9 100644
> > --- a/arch/powerpc/include/asm/paca.h
> > +++ b/arch/powerpc/include/asm/paca.h
> > @@ -34,19 +34,38 @@
> >  #include <asm/cpuidle.h>
> >  #include <asm/atomic.h>
> >  
> > -register struct paca_struct *local_paca asm("r13");
> > -
> >  #if defined(CONFIG_DEBUG_PREEMPT) && defined(CONFIG_SMP)
> >  extern unsigned int debug_smp_processor_id(void); /* from
> > linux/smp.h */
> > -/*
> > - * Add standard checks that preemption cannot occur when using
> > get_paca():
> > - * otherwise the paca_struct it points to may be the wrong one
> > just after.
> > - */
> > -#define get_paca()	((void) debug_smp_processor_id(), local_paca)
> > -#else
> > -#define get_paca()	local_paca
> >  #endif
> >  
> > +static inline struct paca_struct *get_paca_no_preempt_check(void)
> > +{
> > +	register struct paca_struct *paca asm("r13");
> > +
> > +	return paca;
> > +}
> 
> Problem is it now changes the global register variable to a local 
> register variable. The compiler would presumably be within its rights
> to "cache" that return value or use another register for it, which
> is not really what we want.
> 
> 

I've confirmed that at least with GCC 8.2.0, the generated assembler is
similar, but yes, the compiler may be free to take a copy into another
register (although that would be a terrible optimisation), and then
operate on that value.

Subsequent uses would still have to call the function (ie. fetch the
data from r13) regardless, so I believe this scenario is safe.

Can you think of a scenario where this is a problem?

Alastair D'Silva's on March 27, 2019 2:37 pm:
> On Tue, 2019-03-26 at 15:58 +1000, Nicholas Piggin wrote:
>> Alastair D'Silva's on March 14, 2019 12:31 pm:
>> > From: Alastair D'Silva <alastair@d-silva.org>
>> > 
>> > When building an LTO kernel, the existing code generates warnings:
>> >     ./arch/powerpc/include/asm/paca.h:37:30: warning: register of
>> >         ‘local_paca’ used for multiple global register variables
>> >      register struct paca_struct *local_paca asm("r13");
>> >                                   ^
>> >     ./arch/powerpc/include/asm/paca.h:37:30: note: conflicts with
>> >         ‘local_paca’
>> 
>> Isn't this a bogus warning? It doesn't look like there's a way to 
>> define it any other way.
> 
> There isn't any other way to define it as a global. However, the
> warning is legitimate.
> 
> The compiler sees that there are multiple global register variables,
> all pointing at the same register.

It's one variable though, so it's not a legitimate warning (unless
there is a way to declare a "reference" to it that we are not doing).

> 
> The compiler can only determine this when LTO is used, as otherwise it
> only sees the one in the current compilation unit, whicd disappears by
> the time the kernel is linked.
> 
>> 
>> > This patch reworks local_paca into an inline getter & setter
>> > function,
>> > which addresses the warning.
>> > 
>> > Changelog:
>> > V2
>> >   - Address whitespace issues
>> >   - keep new implementation close to where the old implementation
>> > was
>> > 
>> > Signed-off-by: Alastair D'Silva <alastair@d-silva.org>
>> > ---
>> >  arch/powerpc/include/asm/paca.h | 37 +++++++++++++++++++++++++--
>> > ------
>> >  arch/powerpc/kernel/paca.c      |  2 +-
>> >  2 files changed, 29 insertions(+), 10 deletions(-)
>> > 
>> > diff --git a/arch/powerpc/include/asm/paca.h
>> > b/arch/powerpc/include/asm/paca.h
>> > index e843bc5d1a0f..2fa0b43357c9 100644
>> > --- a/arch/powerpc/include/asm/paca.h
>> > +++ b/arch/powerpc/include/asm/paca.h
>> > @@ -34,19 +34,38 @@
>> >  #include <asm/cpuidle.h>
>> >  #include <asm/atomic.h>
>> >  
>> > -register struct paca_struct *local_paca asm("r13");
>> > -
>> >  #if defined(CONFIG_DEBUG_PREEMPT) && defined(CONFIG_SMP)
>> >  extern unsigned int debug_smp_processor_id(void); /* from
>> > linux/smp.h */
>> > -/*
>> > - * Add standard checks that preemption cannot occur when using
>> > get_paca():
>> > - * otherwise the paca_struct it points to may be the wrong one
>> > just after.
>> > - */
>> > -#define get_paca()	((void) debug_smp_processor_id(), local_paca)
>> > -#else
>> > -#define get_paca()	local_paca
>> >  #endif
>> >  
>> > +static inline struct paca_struct *get_paca_no_preempt_check(void)
>> > +{
>> > +	register struct paca_struct *paca asm("r13");
>> > +
>> > +	return paca;
>> > +}
>> 
>> Problem is it now changes the global register variable to a local 
>> register variable. The compiler would presumably be within its rights
>> to "cache" that return value or use another register for it, which
>> is not really what we want.
>> 
>> 
> 
> I've confirmed that at least with GCC 8.2.0, the generated assembler is
> similar, but yes, the compiler may be free to take a copy into another
> register (although that would be a terrible optimisation), and then
> operate on that value.

Yep.

> 
> Subsequent uses would still have to call the function (ie. fetch the
> data from r13) regardless, so I believe this scenario is safe.
> 
> Can you think of a scenario where this is a problem?

It wouldn't be the subsequent use but the one use. If you're preempted
then you can't be using a stale r13 value.

Quite possibly any bug like that would already be buggy now, the cases
where it matters tend to need asm to access it. And it's something we
need to really audit and have proper accessors and preempt warnings
that Ben always harped on about.

But to just work around this warning it seems pretty dangerous to
change this and hope.

Thanks,
Nick