[PATCHv4,02/11] pci: use weak functions for MSI arch-specific functions

Until now, the MSI architecture-specific functions could be overloaded
using a fairly complex set of #define and compile-time
conditionals. In order to prepare for the introduction of the msi_chip
infrastructure, it is desirable to switch all those functions to use
the 'weak' mechanism. This commit converts all the architectures that
were overidding those MSI functions to use the new strategy.

Note that we keep a separate, non-weak, function
default_teardown_msi_irqs() for the default behavior of the
arch_teardown_msi_irqs(), as the default behavior is needed by the Xen
x86 PCI code.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: linuxppc-dev@lists.ozlabs.org
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: linux390@de.ibm.com
Cc: linux-s390@vger.kernel.org
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: x86@kernel.org
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: linux-ia64@vger.kernel.org
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: David S. Miller <davem@davemloft.net>
Cc: sparclinux@vger.kernel.org
Cc: Chris Metcalf <cmetcalf@tilera.com>
---
 arch/mips/include/asm/pci.h    |  5 -----
 arch/powerpc/include/asm/pci.h |  5 -----
 arch/s390/include/asm/pci.h    |  4 ----
 arch/x86/include/asm/pci.h     | 28 --------------------------
 arch/x86/kernel/x86_init.c     | 21 ++++++++++++++++++++
 drivers/pci/msi.c              | 45 +++++++++++++++++++-----------------------
 include/linux/msi.h            |  7 ++++++-
 7 files changed, 47 insertions(+), 68 deletions(-)

On Mon, 2013-07-01 at 15:42 +0200, Thomas Petazzoni wrote:
> Until now, the MSI architecture-specific functions could be overloaded
> using a fairly complex set of #define and compile-time
> conditionals. In order to prepare for the introduction of the msi_chip
> infrastructure, it is desirable to switch all those functions to use
> the 'weak' mechanism. This commit converts all the architectures that
> were overidding those MSI functions to use the new strategy.

The MSI code used to use weak functions, until we discovered they were
being miscompiled on some toolchains (11df1f0). I assume these days
we're confident they work correctly.

cheers

Dear Michael Ellerman,

On Tue, 02 Jul 2013 10:53:16 +1000, Michael Ellerman wrote:
> On Mon, 2013-07-01 at 15:42 +0200, Thomas Petazzoni wrote:
> > Until now, the MSI architecture-specific functions could be overloaded
> > using a fairly complex set of #define and compile-time
> > conditionals. In order to prepare for the introduction of the msi_chip
> > infrastructure, it is desirable to switch all those functions to use
> > the 'weak' mechanism. This commit converts all the architectures that
> > were overidding those MSI functions to use the new strategy.
> 
> The MSI code used to use weak functions, until we discovered they were
> being miscompiled on some toolchains (11df1f0). I assume these days
> we're confident they work correctly.

Hum, interesting. I see from your commit that gcc 4.3.2 was apparently
affected, and gcc 4.3.x is not /that/ old. Bjorn, what's your point of
view on this?

Another option would be to have architecture register some msi_arch_ops
structure, with a set of operations, which I believe is a pattern that
is more widespread in the kernel than weak functions.

Thoughts?

Thomas

On Tue, Jul 02, 2013 at 07:30:37AM +0200, Thomas Petazzoni wrote:
> Dear Michael Ellerman,
> 
> On Tue, 02 Jul 2013 10:53:16 +1000, Michael Ellerman wrote:
> > On Mon, 2013-07-01 at 15:42 +0200, Thomas Petazzoni wrote:
> > > Until now, the MSI architecture-specific functions could be overloaded
> > > using a fairly complex set of #define and compile-time
> > > conditionals. In order to prepare for the introduction of the msi_chip
> > > infrastructure, it is desirable to switch all those functions to use
> > > the 'weak' mechanism. This commit converts all the architectures that
> > > were overidding those MSI functions to use the new strategy.
> > 
> > The MSI code used to use weak functions, until we discovered they were
> > being miscompiled on some toolchains (11df1f0). I assume these days
> > we're confident they work correctly.
> 
> Hum, interesting. I see from your commit that gcc 4.3.2 was apparently
> affected, and gcc 4.3.x is not /that/ old. Bjorn, what's your point of
> view on this?

Stop press.

I went back and found the old threads on this, it's been a while. It
looks like it was only gcc 4.1.[01] that miscompiled. The reference to
gcc 4.3.2 was WRT ellision of the unused code, which is a separate
issue.

The kernel blacklists gcc 4.1.[01] (see f9d1425), so weak should be
safe to use.

We merged the change to the PCI code anyway because we thought it was
nicer and it also avoided any problems with weak.

So pretend I never said anything :)

cheers

On Mon, Jul 1, 2013 at 11:30 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Michael Ellerman,
>
> On Tue, 02 Jul 2013 10:53:16 +1000, Michael Ellerman wrote:
>> On Mon, 2013-07-01 at 15:42 +0200, Thomas Petazzoni wrote:
>> > Until now, the MSI architecture-specific functions could be overloaded
>> > using a fairly complex set of #define and compile-time
>> > conditionals. In order to prepare for the introduction of the msi_chip
>> > infrastructure, it is desirable to switch all those functions to use
>> > the 'weak' mechanism. This commit converts all the architectures that
>> > were overidding those MSI functions to use the new strategy.
>>
>> The MSI code used to use weak functions, until we discovered they were
>> being miscompiled on some toolchains (11df1f0). I assume these days
>> we're confident they work correctly.
>
> Hum, interesting. I see from your commit that gcc 4.3.2 was apparently
> affected, and gcc 4.3.x is not /that/ old. Bjorn, what's your point of
> view on this?

There *were* compilation issues with weak functions, but AFAIK they've
been resolved and there's no reason to avoid them.  Commit f9d142500
addressed this.

Bjorn

On Mon, Jul 1, 2013 at 7:42 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Until now, the MSI architecture-specific functions could be overloaded
> using a fairly complex set of #define and compile-time
> conditionals. In order to prepare for the introduction of the msi_chip
> infrastructure, it is desirable to switch all those functions to use
> the 'weak' mechanism. This commit converts all the architectures that
> were overidding those MSI functions to use the new strategy.
>
> Note that we keep a separate, non-weak, function
> default_teardown_msi_irqs() for the default behavior of the
> arch_teardown_msi_irqs(), as the default behavior is needed by the Xen
> x86 PCI code.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> Cc: Paul Mackerras <paulus@samba.org>
> Cc: linuxppc-dev@lists.ozlabs.org
> Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
> Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
> Cc: linux390@de.ibm.com
> Cc: linux-s390@vger.kernel.org
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: H. Peter Anvin <hpa@zytor.com>
> Cc: x86@kernel.org
> Cc: Russell King <linux@arm.linux.org.uk>
> Cc: Tony Luck <tony.luck@intel.com>
> Cc: Fenghua Yu <fenghua.yu@intel.com>
> Cc: linux-ia64@vger.kernel.org
> Cc: Ralf Baechle <ralf@linux-mips.org>
> Cc: linux-mips@linux-mips.org
> Cc: David S. Miller <davem@davemloft.net>
> Cc: sparclinux@vger.kernel.org
> Cc: Chris Metcalf <cmetcalf@tilera.com>

Acked-by: Bjorn Helgaas <bhelgaas@google.com>

> ---
>  arch/mips/include/asm/pci.h    |  5 -----
>  arch/powerpc/include/asm/pci.h |  5 -----
>  arch/s390/include/asm/pci.h    |  4 ----
>  arch/x86/include/asm/pci.h     | 28 --------------------------
>  arch/x86/kernel/x86_init.c     | 21 ++++++++++++++++++++
>  drivers/pci/msi.c              | 45 +++++++++++++++++++-----------------------
>  include/linux/msi.h            |  7 ++++++-
>  7 files changed, 47 insertions(+), 68 deletions(-)
>
> diff --git a/arch/mips/include/asm/pci.h b/arch/mips/include/asm/pci.h
> index b8e24fd..031f4c1 100644
> --- a/arch/mips/include/asm/pci.h
> +++ b/arch/mips/include/asm/pci.h
> @@ -137,11 +137,6 @@ static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
>         return channel ? 15 : 14;
>  }
>
> -#ifdef CONFIG_CPU_CAVIUM_OCTEON
> -/* MSI arch hook for OCTEON */
> -#define arch_setup_msi_irqs arch_setup_msi_irqs
> -#endif
> -
>  extern char * (*pcibios_plat_setup)(char *str);
>
>  #ifdef CONFIG_OF
> diff --git a/arch/powerpc/include/asm/pci.h b/arch/powerpc/include/asm/pci.h
> index 6653f27..95145a1 100644
> --- a/arch/powerpc/include/asm/pci.h
> +++ b/arch/powerpc/include/asm/pci.h
> @@ -113,11 +113,6 @@ extern int pci_domain_nr(struct pci_bus *bus);
>  /* Decide whether to display the domain number in /proc */
>  extern int pci_proc_domain(struct pci_bus *bus);
>
> -/* MSI arch hooks */
> -#define arch_setup_msi_irqs arch_setup_msi_irqs
> -#define arch_teardown_msi_irqs arch_teardown_msi_irqs
> -#define arch_msi_check_device arch_msi_check_device
> -
>  struct vm_area_struct;
>  /* Map a range of PCI memory or I/O space for a device into user space */
>  int pci_mmap_page_range(struct pci_dev *pdev, struct vm_area_struct *vma,
> diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h
> index 6c18012..8641e8d 100644
> --- a/arch/s390/include/asm/pci.h
> +++ b/arch/s390/include/asm/pci.h
> @@ -21,10 +21,6 @@ void pci_iounmap(struct pci_dev *, void __iomem *);
>  int pci_domain_nr(struct pci_bus *);
>  int pci_proc_domain(struct pci_bus *);
>
> -/* MSI arch hooks */
> -#define arch_setup_msi_irqs    arch_setup_msi_irqs
> -#define arch_teardown_msi_irqs arch_teardown_msi_irqs
> -
>  #define ZPCI_BUS_NR                    0       /* default bus number */
>  #define ZPCI_DEVFN                     0       /* default device number */
>
> diff --git a/arch/x86/include/asm/pci.h b/arch/x86/include/asm/pci.h
> index d9e9e6c..8c61de0 100644
> --- a/arch/x86/include/asm/pci.h
> +++ b/arch/x86/include/asm/pci.h
> @@ -100,29 +100,6 @@ static inline void early_quirks(void) { }
>  extern void pci_iommu_alloc(void);
>
>  #ifdef CONFIG_PCI_MSI
> -/* MSI arch specific hooks */
> -static inline int x86_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
> -{
> -       return x86_msi.setup_msi_irqs(dev, nvec, type);
> -}
> -
> -static inline void x86_teardown_msi_irqs(struct pci_dev *dev)
> -{
> -       x86_msi.teardown_msi_irqs(dev);
> -}
> -
> -static inline void x86_teardown_msi_irq(unsigned int irq)
> -{
> -       x86_msi.teardown_msi_irq(irq);
> -}
> -static inline void x86_restore_msi_irqs(struct pci_dev *dev, int irq)
> -{
> -       x86_msi.restore_msi_irqs(dev, irq);
> -}
> -#define arch_setup_msi_irqs x86_setup_msi_irqs
> -#define arch_teardown_msi_irqs x86_teardown_msi_irqs
> -#define arch_teardown_msi_irq x86_teardown_msi_irq
> -#define arch_restore_msi_irqs x86_restore_msi_irqs
>  /* implemented in arch/x86/kernel/apic/io_apic. */
>  struct msi_desc;
>  int native_setup_msi_irqs(struct pci_dev *dev, int nvec, int type);
> @@ -130,11 +107,6 @@ void native_teardown_msi_irq(unsigned int irq);
>  void native_restore_msi_irqs(struct pci_dev *dev, int irq);
>  int setup_msi_irq(struct pci_dev *dev, struct msi_desc *msidesc,
>                   unsigned int irq_base, unsigned int irq_offset);
> -/* default to the implementation in drivers/lib/msi.c */
> -#define HAVE_DEFAULT_MSI_TEARDOWN_IRQS
> -#define HAVE_DEFAULT_MSI_RESTORE_IRQS
> -void default_teardown_msi_irqs(struct pci_dev *dev);
> -void default_restore_msi_irqs(struct pci_dev *dev, int irq);
>  #else
>  #define native_setup_msi_irqs          NULL
>  #define native_teardown_msi_irq                NULL
> diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c
> index 45a14db..a2b189c 100644
> --- a/arch/x86/kernel/x86_init.c
> +++ b/arch/x86/kernel/x86_init.c
> @@ -116,6 +116,27 @@ struct x86_msi_ops x86_msi = {
>         .setup_hpet_msi         = default_setup_hpet_msi,
>  };
>
> +/* MSI arch specific hooks */
> +int arch_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
> +{
> +       return x86_msi.setup_msi_irqs(dev, nvec, type);
> +}
> +
> +void arch_teardown_msi_irqs(struct pci_dev *dev)
> +{
> +       x86_msi.teardown_msi_irqs(dev);
> +}
> +
> +void arch_teardown_msi_irq(unsigned int irq)
> +{
> +       x86_msi.teardown_msi_irq(irq);
> +}
> +
> +void arch_restore_msi_irqs(struct pci_dev *dev, int irq)
> +{
> +       x86_msi.restore_msi_irqs(dev, irq);
> +}
> +
>  struct x86_io_apic_ops x86_io_apic_ops = {
>         .init                   = native_io_apic_init_mappings,
>         .read                   = native_io_apic_read,
> diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c
> index 2c10752..289fbfd 100644
> --- a/drivers/pci/msi.c
> +++ b/drivers/pci/msi.c
> @@ -30,20 +30,21 @@ static int pci_msi_enable = 1;
>
>  /* Arch hooks */
>
> -#ifndef arch_msi_check_device
> -int arch_msi_check_device(struct pci_dev *dev, int nvec, int type)
> +int __weak arch_setup_msi_irq(struct pci_dev *dev, struct msi_desc *desc)
>  {
> -       return 0;
> +       return -EINVAL;
>  }
> -#endif
>
> -#ifndef arch_setup_msi_irqs
> -# define arch_setup_msi_irqs default_setup_msi_irqs
> -# define HAVE_DEFAULT_MSI_SETUP_IRQS
> -#endif
> +void __weak arch_teardown_msi_irq(unsigned int irq)
> +{
> +}
>
> -#ifdef HAVE_DEFAULT_MSI_SETUP_IRQS
> -int default_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
> +int __weak arch_msi_check_device(struct pci_dev *dev, int nvec, int type)
> +{
> +       return 0;
> +}
> +
> +int __weak arch_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
>  {
>         struct msi_desc *entry;
>         int ret;
> @@ -65,14 +66,11 @@ int default_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
>
>         return 0;
>  }
> -#endif
>
> -#ifndef arch_teardown_msi_irqs
> -# define arch_teardown_msi_irqs default_teardown_msi_irqs
> -# define HAVE_DEFAULT_MSI_TEARDOWN_IRQS
> -#endif
> -
> -#ifdef HAVE_DEFAULT_MSI_TEARDOWN_IRQS
> +/*
> + * We have a default implementation available as a separate non-weak
> + * function, as it is used by the Xen x86 PCI code
> + */
>  void default_teardown_msi_irqs(struct pci_dev *dev)
>  {
>         struct msi_desc *entry;
> @@ -86,15 +84,13 @@ void default_teardown_msi_irqs(struct pci_dev *dev)
>                         arch_teardown_msi_irq(entry->irq + i);
>         }
>  }
> -#endif
>
> -#ifndef arch_restore_msi_irqs
> -# define arch_restore_msi_irqs default_restore_msi_irqs
> -# define HAVE_DEFAULT_MSI_RESTORE_IRQS
> -#endif
> +void __weak arch_teardown_msi_irqs(struct pci_dev *dev)
> +{
> +       return default_teardown_msi_irqs(dev);
> +}
>
> -#ifdef HAVE_DEFAULT_MSI_RESTORE_IRQS
> -void default_restore_msi_irqs(struct pci_dev *dev, int irq)
> +void __weak arch_restore_msi_irqs(struct pci_dev *dev, int irq)
>  {
>         struct msi_desc *entry;
>
> @@ -111,7 +107,6 @@ void default_restore_msi_irqs(struct pci_dev *dev, int irq)
>         if (entry)
>                 write_msi_msg(irq, &entry->msg);
>  }
> -#endif
>
>  static void msi_set_enable(struct pci_dev *dev, int enable)
>  {
> diff --git a/include/linux/msi.h b/include/linux/msi.h
> index 20c2d6d..c82ff8d 100644
> --- a/include/linux/msi.h
> +++ b/include/linux/msi.h
> @@ -50,12 +50,17 @@ struct msi_desc {
>  };
>
>  /*
> - * The arch hook for setup up msi irqs
> + * The arch hooks to setup up msi irqs. Those functions are
> + * implemented as weak symbols so that they /can/ be overriden by
> + * architecture specific code if needed.
>   */
>  int arch_setup_msi_irq(struct pci_dev *dev, struct msi_desc *desc);
>  void arch_teardown_msi_irq(unsigned int irq);
>  int arch_setup_msi_irqs(struct pci_dev *dev, int nvec, int type);
>  void arch_teardown_msi_irqs(struct pci_dev *dev);
>  int arch_msi_check_device(struct pci_dev* dev, int nvec, int type);
> +void arch_restore_msi_irqs(struct pci_dev *dev, int irq);
> +
> +void default_teardown_msi_irqs(struct pci_dev *dev);
>
>  #endif /* LINUX_MSI_H */
> --
> 1.8.1.2
>

On Fri, Jul 5, 2013 at 3:32 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:

> Acked-by: Bjorn Helgaas <bhelgaas@google.com>

But please update your subject line to use consistent capitalization, e.g.,

PCI: Use weak ...

Dear Bjorn Helgaas,

On Fri, 5 Jul 2013 15:34:10 -0600, Bjorn Helgaas wrote:
> On Fri, Jul 5, 2013 at 3:32 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> 
> > Acked-by: Bjorn Helgaas <bhelgaas@google.com>
> 
> But please update your subject line to use consistent capitalization, e.g.,
> 
> PCI: Use weak ...

Sure, will do.

Would it be possible to get Tested-by and/or Acked-by from the
different architecture maintainers affected by PATCH 02/11 and PATCH
03/11 ?

Thanks,

Thomas