[v3,2/2] asm-generic: Add new pci.h and use it

The asm/pci.h used for many newer architectures share similar
definitions.  Move the common parts to asm-generic/pci.h to allow for
sharing code.

Two things to note are:

 - isa_dma_bridge_buggy, traditionally this is defined in asm/dma.h but
   these architectures avoid creating that file and add the definition
   to asm/pci.h.
 - ARCH_GENERIC_PCI_MMAP_RESOURCE, csky does not define this so we
   undefine it after including asm-generic/pci.h.  Why doesn't csky
   define it?
 - pci_get_legacy_ide_irq, This function is only used on architectures
   that support PNP.  It is only maintained for arm64, in other
   architectures it is removed.

Suggested-by: Arnd Bergmann <arnd@arndb.de>
Link: https://lore.kernel.org/lkml/CAK8P3a0JmPeczfmMBE__vn=Jbvf=nkbpVaZCycyv40pZNCJJXQ@mail.gmail.com/
Signed-off-by: Stafford Horne <shorne@gmail.com>
---
Second note on isa_dma_bridge_buggy, this is set on x86 but it it also set in
pci/quirks.c.  We discussed limiting it only to x86 though as its a general
quick triggered by pci ids I think it will be more tricky than we thought so I
will leave as is.  It might be nice to move it out of asm/dma.h and into
asm/pci.h though.

Since v2:
 - Nothing
Since v1:
 - Remove definition of pci_get_legacy_ide_irq

 arch/arm64/include/asm/pci.h | 12 +++---------
 arch/csky/include/asm/pci.h  | 24 ++++--------------------
 arch/riscv/include/asm/pci.h | 25 +++----------------------
 arch/um/include/asm/pci.h    | 24 ++----------------------
 include/asm-generic/pci.h    | 36 ++++++++++++++++++++++++++++++++++++
 5 files changed, 48 insertions(+), 73 deletions(-)
 create mode 100644 include/asm-generic/pci.h

On Sun, 17 Jul 2022 17:41:14 PDT (-0700), shorne@gmail.com wrote:
> The asm/pci.h used for many newer architectures share similar
> definitions.  Move the common parts to asm-generic/pci.h to allow for
> sharing code.
>
> Two things to note are:
>
>  - isa_dma_bridge_buggy, traditionally this is defined in asm/dma.h but
>    these architectures avoid creating that file and add the definition
>    to asm/pci.h.
>  - ARCH_GENERIC_PCI_MMAP_RESOURCE, csky does not define this so we
>    undefine it after including asm-generic/pci.h.  Why doesn't csky
>    define it?
>  - pci_get_legacy_ide_irq, This function is only used on architectures
>    that support PNP.  It is only maintained for arm64, in other
>    architectures it is removed.
>
> Suggested-by: Arnd Bergmann <arnd@arndb.de>
> Link: https://lore.kernel.org/lkml/CAK8P3a0JmPeczfmMBE__vn=Jbvf=nkbpVaZCycyv40pZNCJJXQ@mail.gmail.com/
> Signed-off-by: Stafford Horne <shorne@gmail.com>
> ---
> Second note on isa_dma_bridge_buggy, this is set on x86 but it it also set in
> pci/quirks.c.  We discussed limiting it only to x86 though as its a general
> quick triggered by pci ids I think it will be more tricky than we thought so I
> will leave as is.  It might be nice to move it out of asm/dma.h and into
> asm/pci.h though.
>
> Since v2:
>  - Nothing
> Since v1:
>  - Remove definition of pci_get_legacy_ide_irq
>
>  arch/arm64/include/asm/pci.h | 12 +++---------
>  arch/csky/include/asm/pci.h  | 24 ++++--------------------
>  arch/riscv/include/asm/pci.h | 25 +++----------------------
>  arch/um/include/asm/pci.h    | 24 ++----------------------
>  include/asm-generic/pci.h    | 36 ++++++++++++++++++++++++++++++++++++
>  5 files changed, 48 insertions(+), 73 deletions(-)
>  create mode 100644 include/asm-generic/pci.h
>
> diff --git a/arch/arm64/include/asm/pci.h b/arch/arm64/include/asm/pci.h
> index b33ca260e3c9..1180e83712f5 100644
> --- a/arch/arm64/include/asm/pci.h
> +++ b/arch/arm64/include/asm/pci.h
> @@ -9,7 +9,6 @@
>  #include <asm/io.h>
>
>  #define PCIBIOS_MIN_IO		0x1000
> -#define PCIBIOS_MIN_MEM		0
>
>  /*
>   * Set to 1 if the kernel should re-assign all PCI bus numbers
> @@ -18,9 +17,6 @@
>  	(pci_has_flag(PCI_REASSIGN_ALL_BUS))
>
>  #define arch_can_pci_mmap_wc() 1
> -#define ARCH_GENERIC_PCI_MMAP_RESOURCE	1
> -
> -extern int isa_dma_bridge_buggy;
>
>  #ifdef CONFIG_PCI
>  static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
> @@ -28,11 +24,9 @@ static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
>  	/* no legacy IRQ on arm64 */
>  	return -ENODEV;
>  }
> -
> -static inline int pci_proc_domain(struct pci_bus *bus)
> -{
> -	return 1;
> -}
>  #endif  /* CONFIG_PCI */
>
> +/* Generic PCI */
> +#include <asm-generic/pci.h>
> +
>  #endif  /* __ASM_PCI_H */
> diff --git a/arch/csky/include/asm/pci.h b/arch/csky/include/asm/pci.h
> index ebc765b1f78b..44866c1ad461 100644
> --- a/arch/csky/include/asm/pci.h
> +++ b/arch/csky/include/asm/pci.h
> @@ -9,26 +9,10 @@
>
>  #include <asm/io.h>
>
> -#define PCIBIOS_MIN_IO		0
> -#define PCIBIOS_MIN_MEM		0
> +/* Generic PCI */
> +#include <asm-generic/pci.h>
>
> -/* C-SKY shim does not initialize PCI bus */
> -#define pcibios_assign_all_busses() 1
> -
> -extern int isa_dma_bridge_buggy;
> -
> -#ifdef CONFIG_PCI
> -static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
> -{
> -	/* no legacy IRQ on csky */
> -	return -ENODEV;
> -}
> -
> -static inline int pci_proc_domain(struct pci_bus *bus)
> -{
> -	/* always show the domain in /proc */
> -	return 1;
> -}
> -#endif  /* CONFIG_PCI */
> +/* csky doesn't use generic pci resource mapping */
> +#undef ARCH_GENERIC_PCI_MMAP_RESOURCE
>
>  #endif  /* __ASM_CSKY_PCI_H */
> diff --git a/arch/riscv/include/asm/pci.h b/arch/riscv/include/asm/pci.h
> index 7fd52a30e605..12ce8150cfb0 100644
> --- a/arch/riscv/include/asm/pci.h
> +++ b/arch/riscv/include/asm/pci.h
> @@ -12,29 +12,7 @@
>
>  #include <asm/io.h>
>
> -#define PCIBIOS_MIN_IO		0
> -#define PCIBIOS_MIN_MEM		0

My for-next changes these in bb356ddb78b2 ("RISC-V: PCI: Avoid handing 
out address 0 to devices").  Do you mind either splitting out the 
arch/riscv bits or having this in via some sort of shared tag?

> -
> -/* RISC-V shim does not initialize PCI bus */
> -#define pcibios_assign_all_busses() 1
> -
> -#define ARCH_GENERIC_PCI_MMAP_RESOURCE 1
> -
> -extern int isa_dma_bridge_buggy;
> -
>  #ifdef CONFIG_PCI
> -static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
> -{
> -	/* no legacy IRQ on risc-v */
> -	return -ENODEV;
> -}
> -
> -static inline int pci_proc_domain(struct pci_bus *bus)
> -{
> -	/* always show the domain in /proc */
> -	return 1;
> -}
> -
>  #ifdef	CONFIG_NUMA
>
>  static inline int pcibus_to_node(struct pci_bus *bus)
> @@ -50,4 +28,7 @@ static inline int pcibus_to_node(struct pci_bus *bus)
>
>  #endif  /* CONFIG_PCI */
>
> +/* Generic PCI */
> +#include <asm-generic/pci.h>
> +
>  #endif  /* _ASM_RISCV_PCI_H */
> diff --git a/arch/um/include/asm/pci.h b/arch/um/include/asm/pci.h
> index da13fd5519ef..34fe4921b5fa 100644
> --- a/arch/um/include/asm/pci.h
> +++ b/arch/um/include/asm/pci.h
> @@ -4,28 +4,8 @@
>  #include <linux/types.h>
>  #include <asm/io.h>
>
> -#define PCIBIOS_MIN_IO		0
> -#define PCIBIOS_MIN_MEM		0
> -
> -#define pcibios_assign_all_busses() 1
> -
> -extern int isa_dma_bridge_buggy;
> -
> -#ifdef CONFIG_PCI
> -static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
> -{
> -	/* no legacy IRQs */
> -	return -ENODEV;
> -}
> -#endif
> -
> -#ifdef CONFIG_PCI_DOMAINS
> -static inline int pci_proc_domain(struct pci_bus *bus)
> -{
> -	/* always show the domain in /proc */
> -	return 1;
> -}
> -#endif  /* CONFIG_PCI */
> +/* Generic PCI */
> +#include <asm-generic/pci.h>
>
>  #ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
>  /*
> diff --git a/include/asm-generic/pci.h b/include/asm-generic/pci.h
> new file mode 100644
> index 000000000000..fbc25741696a
> --- /dev/null
> +++ b/include/asm-generic/pci.h
> @@ -0,0 +1,36 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +
> +#ifndef __ASM_GENERIC_PCI_H
> +#define __ASM_GENERIC_PCI_H
> +
> +#include <linux/types.h>
> +
> +#ifndef PCIBIOS_MIN_IO
> +#define PCIBIOS_MIN_IO		0
> +#endif
> +
> +#ifndef PCIBIOS_MIN_MEM
> +#define PCIBIOS_MIN_MEM		0
> +#endif
> +
> +#ifndef pcibios_assign_all_busses
> +/* For bootloaders that do not initialize the PCI bus */
> +#define pcibios_assign_all_busses() 1
> +#endif
> +
> +extern int isa_dma_bridge_buggy;
> +
> +/* Enable generic resource mapping code in drivers/pci/ */
> +#define ARCH_GENERIC_PCI_MMAP_RESOURCE
> +
> +#ifdef CONFIG_PCI
> +
> +static inline int pci_proc_domain(struct pci_bus *bus)
> +{
> +	/* always show the domain in /proc */
> +	return 1;
> +}
> +
> +#endif /* CONFIG_PCI */
> +
> +#endif /* __ASM_GENERIC_PCI_H */

On Tue, Jul 19, 2022 at 08:58:39AM -0700, Palmer Dabbelt wrote:
> On Sun, 17 Jul 2022 17:41:14 PDT (-0700), shorne@gmail.com wrote:
> > The asm/pci.h used for many newer architectures share similar
> > definitions.  Move the common parts to asm-generic/pci.h to allow for
> > sharing code.
> > 
> > Two things to note are:
> > 
> >  - isa_dma_bridge_buggy, traditionally this is defined in asm/dma.h but
> >    these architectures avoid creating that file and add the definition
> >    to asm/pci.h.
> >  - ARCH_GENERIC_PCI_MMAP_RESOURCE, csky does not define this so we
> >    undefine it after including asm-generic/pci.h.  Why doesn't csky
> >    define it?
> >  - pci_get_legacy_ide_irq, This function is only used on architectures
> >    that support PNP.  It is only maintained for arm64, in other
> >    architectures it is removed.
> > 
> > Suggested-by: Arnd Bergmann <arnd@arndb.de>
> > Link: https://lore.kernel.org/lkml/CAK8P3a0JmPeczfmMBE__vn=Jbvf=nkbpVaZCycyv40pZNCJJXQ@mail.gmail.com/
> > Signed-off-by: Stafford Horne <shorne@gmail.com>
> > ---
> > Second note on isa_dma_bridge_buggy, this is set on x86 but it it also set in
> > pci/quirks.c.  We discussed limiting it only to x86 though as its a general
> > quick triggered by pci ids I think it will be more tricky than we thought so I
> > will leave as is.  It might be nice to move it out of asm/dma.h and into
> > asm/pci.h though.
> > 
> > Since v2:
> >  - Nothing
> > Since v1:
> >  - Remove definition of pci_get_legacy_ide_irq
> > 
> >  arch/arm64/include/asm/pci.h | 12 +++---------
> >  arch/csky/include/asm/pci.h  | 24 ++++--------------------
> >  arch/riscv/include/asm/pci.h | 25 +++----------------------
> >  arch/um/include/asm/pci.h    | 24 ++----------------------
> >  include/asm-generic/pci.h    | 36 ++++++++++++++++++++++++++++++++++++
> >  5 files changed, 48 insertions(+), 73 deletions(-)
> >  create mode 100644 include/asm-generic/pci.h
> > 
> > diff --git a/arch/arm64/include/asm/pci.h b/arch/arm64/include/asm/pci.h
> > index b33ca260e3c9..1180e83712f5 100644
> > --- a/arch/arm64/include/asm/pci.h
> > +++ b/arch/arm64/include/asm/pci.h
> > @@ -9,7 +9,6 @@
> >  #include <asm/io.h>
> > 
> >  #define PCIBIOS_MIN_IO		0x1000
> > -#define PCIBIOS_MIN_MEM		0
> > 
> >  /*
> >   * Set to 1 if the kernel should re-assign all PCI bus numbers
> > @@ -18,9 +17,6 @@
> >  	(pci_has_flag(PCI_REASSIGN_ALL_BUS))
> > 
> >  #define arch_can_pci_mmap_wc() 1
> > -#define ARCH_GENERIC_PCI_MMAP_RESOURCE	1
> > -
> > -extern int isa_dma_bridge_buggy;
> > 
> >  #ifdef CONFIG_PCI
> >  static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
> > @@ -28,11 +24,9 @@ static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
> >  	/* no legacy IRQ on arm64 */
> >  	return -ENODEV;
> >  }
> > -
> > -static inline int pci_proc_domain(struct pci_bus *bus)
> > -{
> > -	return 1;
> > -}
> >  #endif  /* CONFIG_PCI */
> > 
> > +/* Generic PCI */
> > +#include <asm-generic/pci.h>
> > +
> >  #endif  /* __ASM_PCI_H */
> > diff --git a/arch/csky/include/asm/pci.h b/arch/csky/include/asm/pci.h
> > index ebc765b1f78b..44866c1ad461 100644
> > --- a/arch/csky/include/asm/pci.h
> > +++ b/arch/csky/include/asm/pci.h
> > @@ -9,26 +9,10 @@
> > 
> >  #include <asm/io.h>
> > 
> > -#define PCIBIOS_MIN_IO		0
> > -#define PCIBIOS_MIN_MEM		0
> > +/* Generic PCI */
> > +#include <asm-generic/pci.h>
> > 
> > -/* C-SKY shim does not initialize PCI bus */
> > -#define pcibios_assign_all_busses() 1
> > -
> > -extern int isa_dma_bridge_buggy;
> > -
> > -#ifdef CONFIG_PCI
> > -static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
> > -{
> > -	/* no legacy IRQ on csky */
> > -	return -ENODEV;
> > -}
> > -
> > -static inline int pci_proc_domain(struct pci_bus *bus)
> > -{
> > -	/* always show the domain in /proc */
> > -	return 1;
> > -}
> > -#endif  /* CONFIG_PCI */
> > +/* csky doesn't use generic pci resource mapping */
> > +#undef ARCH_GENERIC_PCI_MMAP_RESOURCE
> > 
> >  #endif  /* __ASM_CSKY_PCI_H */
> > diff --git a/arch/riscv/include/asm/pci.h b/arch/riscv/include/asm/pci.h
> > index 7fd52a30e605..12ce8150cfb0 100644
> > --- a/arch/riscv/include/asm/pci.h
> > +++ b/arch/riscv/include/asm/pci.h
> > @@ -12,29 +12,7 @@
> > 
> >  #include <asm/io.h>
> > 
> > -#define PCIBIOS_MIN_IO		0
> > -#define PCIBIOS_MIN_MEM		0
> 
> My for-next changes these in bb356ddb78b2 ("RISC-V: PCI: Avoid handing out
> address 0 to devices").  Do you mind either splitting out the arch/riscv
> bits or having this in via some sort of shared tag?

Hi Palmer,

I replied last on my phone but since it produces HTML multi-part email it got
rejected from a few places and I am not sure if you saw my reply.

It might be a bit hard to separate out the architecture specific bits as it
requires a bit of coordination with the asm-generic/pci.h move.

Some options:
  1 I can produce a tag for you to merge into your for-next.
  2 I can skip touching riscv at all in this patch and you can take care of it
    in a future patch.
  3 I could cherry pick your change bb356ddb78b2 ("RISC-V: PCI: Avoid handing out
    address 0 to devices") to my for-next branch.  It should help avoid conflict.

Seeing that I am still doing small updates here and there I am not sure when I
will be ablt to create a stable tag.  So maybe 3 would work best?

-Stafford

On Thu, 21 Jul 2022 15:05:34 PDT (-0700), shorne@gmail.com wrote:
> On Tue, Jul 19, 2022 at 08:58:39AM -0700, Palmer Dabbelt wrote:
>> On Sun, 17 Jul 2022 17:41:14 PDT (-0700), shorne@gmail.com wrote:
>> > The asm/pci.h used for many newer architectures share similar
>> > definitions.  Move the common parts to asm-generic/pci.h to allow for
>> > sharing code.
>> >
>> > Two things to note are:
>> >
>> >  - isa_dma_bridge_buggy, traditionally this is defined in asm/dma.h but
>> >    these architectures avoid creating that file and add the definition
>> >    to asm/pci.h.
>> >  - ARCH_GENERIC_PCI_MMAP_RESOURCE, csky does not define this so we
>> >    undefine it after including asm-generic/pci.h.  Why doesn't csky
>> >    define it?
>> >  - pci_get_legacy_ide_irq, This function is only used on architectures
>> >    that support PNP.  It is only maintained for arm64, in other
>> >    architectures it is removed.
>> >
>> > Suggested-by: Arnd Bergmann <arnd@arndb.de>
>> > Link: https://lore.kernel.org/lkml/CAK8P3a0JmPeczfmMBE__vn=Jbvf=nkbpVaZCycyv40pZNCJJXQ@mail.gmail.com/
>> > Signed-off-by: Stafford Horne <shorne@gmail.com>
>> > ---
>> > Second note on isa_dma_bridge_buggy, this is set on x86 but it it also set in
>> > pci/quirks.c.  We discussed limiting it only to x86 though as its a general
>> > quick triggered by pci ids I think it will be more tricky than we thought so I
>> > will leave as is.  It might be nice to move it out of asm/dma.h and into
>> > asm/pci.h though.
>> >
>> > Since v2:
>> >  - Nothing
>> > Since v1:
>> >  - Remove definition of pci_get_legacy_ide_irq
>> >
>> >  arch/arm64/include/asm/pci.h | 12 +++---------
>> >  arch/csky/include/asm/pci.h  | 24 ++++--------------------
>> >  arch/riscv/include/asm/pci.h | 25 +++----------------------
>> >  arch/um/include/asm/pci.h    | 24 ++----------------------
>> >  include/asm-generic/pci.h    | 36 ++++++++++++++++++++++++++++++++++++
>> >  5 files changed, 48 insertions(+), 73 deletions(-)
>> >  create mode 100644 include/asm-generic/pci.h
>> >
>> > diff --git a/arch/arm64/include/asm/pci.h b/arch/arm64/include/asm/pci.h
>> > index b33ca260e3c9..1180e83712f5 100644
>> > --- a/arch/arm64/include/asm/pci.h
>> > +++ b/arch/arm64/include/asm/pci.h
>> > @@ -9,7 +9,6 @@
>> >  #include <asm/io.h>
>> >
>> >  #define PCIBIOS_MIN_IO		0x1000
>> > -#define PCIBIOS_MIN_MEM		0
>> >
>> >  /*
>> >   * Set to 1 if the kernel should re-assign all PCI bus numbers
>> > @@ -18,9 +17,6 @@
>> >  	(pci_has_flag(PCI_REASSIGN_ALL_BUS))
>> >
>> >  #define arch_can_pci_mmap_wc() 1
>> > -#define ARCH_GENERIC_PCI_MMAP_RESOURCE	1
>> > -
>> > -extern int isa_dma_bridge_buggy;
>> >
>> >  #ifdef CONFIG_PCI
>> >  static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
>> > @@ -28,11 +24,9 @@ static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
>> >  	/* no legacy IRQ on arm64 */
>> >  	return -ENODEV;
>> >  }
>> > -
>> > -static inline int pci_proc_domain(struct pci_bus *bus)
>> > -{
>> > -	return 1;
>> > -}
>> >  #endif  /* CONFIG_PCI */
>> >
>> > +/* Generic PCI */
>> > +#include <asm-generic/pci.h>
>> > +
>> >  #endif  /* __ASM_PCI_H */
>> > diff --git a/arch/csky/include/asm/pci.h b/arch/csky/include/asm/pci.h
>> > index ebc765b1f78b..44866c1ad461 100644
>> > --- a/arch/csky/include/asm/pci.h
>> > +++ b/arch/csky/include/asm/pci.h
>> > @@ -9,26 +9,10 @@
>> >
>> >  #include <asm/io.h>
>> >
>> > -#define PCIBIOS_MIN_IO		0
>> > -#define PCIBIOS_MIN_MEM		0
>> > +/* Generic PCI */
>> > +#include <asm-generic/pci.h>
>> >
>> > -/* C-SKY shim does not initialize PCI bus */
>> > -#define pcibios_assign_all_busses() 1
>> > -
>> > -extern int isa_dma_bridge_buggy;
>> > -
>> > -#ifdef CONFIG_PCI
>> > -static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
>> > -{
>> > -	/* no legacy IRQ on csky */
>> > -	return -ENODEV;
>> > -}
>> > -
>> > -static inline int pci_proc_domain(struct pci_bus *bus)
>> > -{
>> > -	/* always show the domain in /proc */
>> > -	return 1;
>> > -}
>> > -#endif  /* CONFIG_PCI */
>> > +/* csky doesn't use generic pci resource mapping */
>> > +#undef ARCH_GENERIC_PCI_MMAP_RESOURCE
>> >
>> >  #endif  /* __ASM_CSKY_PCI_H */
>> > diff --git a/arch/riscv/include/asm/pci.h b/arch/riscv/include/asm/pci.h
>> > index 7fd52a30e605..12ce8150cfb0 100644
>> > --- a/arch/riscv/include/asm/pci.h
>> > +++ b/arch/riscv/include/asm/pci.h
>> > @@ -12,29 +12,7 @@
>> >
>> >  #include <asm/io.h>
>> >
>> > -#define PCIBIOS_MIN_IO		0
>> > -#define PCIBIOS_MIN_MEM		0
>>
>> My for-next changes these in bb356ddb78b2 ("RISC-V: PCI: Avoid handing out
>> address 0 to devices").  Do you mind either splitting out the arch/riscv
>> bits or having this in via some sort of shared tag?
>
> Hi Palmer,
>
> I replied last on my phone but since it produces HTML multi-part email it got
> rejected from a few places and I am not sure if you saw my reply.
>
> It might be a bit hard to separate out the architecture specific bits as it
> requires a bit of coordination with the asm-generic/pci.h move.
>
> Some options:
>   1 I can produce a tag for you to merge into your for-next.
>   2 I can skip touching riscv at all in this patch and you can take care of it
>     in a future patch.
>   3 I could cherry pick your change bb356ddb78b2 ("RISC-V: PCI: Avoid handing out
>     address 0 to devices") to my for-next branch.  It should help avoid conflict.
>
> Seeing that I am still doing small updates here and there I am not sure when I
> will be ablt to create a stable tag.  So maybe 3 would work best?

If you cherry pick it then we're going to both end up with the patch, 
which IMO is worse than the conflict (and the patch is pretty far back, 
so I don't really want to rebase everything this late).

Since this isn't in yet, would it be OK just landing it as a single 
patch on top of 5.19-rc1 that gets merged into whatever tree it ends up 
in (ie, yours or Arnd's)?  Then as long as the commit is stable when it 
lands in for-next I can just merge it down to my tree and sort out the 
conflict from there.

If that's not possible then I think the best bet is to just drop the 
RISC-V diff, I can deal with it when the rest lands in Linus' tree.

> -Stafford

On Tue, Jul 19, 2022 at 9:59 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>
> On Sun, 17 Jul 2022 17:41:14 PDT (-0700), shorne@gmail.com wrote:
> > The asm/pci.h used for many newer architectures share similar
> > definitions.  Move the common parts to asm-generic/pci.h to allow for
> > sharing code.
> >
> > Two things to note are:
> >
> >  - isa_dma_bridge_buggy, traditionally this is defined in asm/dma.h but
> >    these architectures avoid creating that file and add the definition
> >    to asm/pci.h.
> >  - ARCH_GENERIC_PCI_MMAP_RESOURCE, csky does not define this so we
> >    undefine it after including asm-generic/pci.h.  Why doesn't csky
> >    define it?
> >  - pci_get_legacy_ide_irq, This function is only used on architectures
> >    that support PNP.  It is only maintained for arm64, in other
> >    architectures it is removed.
> >
> > Suggested-by: Arnd Bergmann <arnd@arndb.de>
> > Link: https://lore.kernel.org/lkml/CAK8P3a0JmPeczfmMBE__vn=Jbvf=nkbpVaZCycyv40pZNCJJXQ@mail.gmail.com/
> > Signed-off-by: Stafford Horne <shorne@gmail.com>
> > ---
> > Second note on isa_dma_bridge_buggy, this is set on x86 but it it also set in
> > pci/quirks.c.  We discussed limiting it only to x86 though as its a general
> > quick triggered by pci ids I think it will be more tricky than we thought so I
> > will leave as is.  It might be nice to move it out of asm/dma.h and into
> > asm/pci.h though.
> >
> > Since v2:
> >  - Nothing
> > Since v1:
> >  - Remove definition of pci_get_legacy_ide_irq
> >
> >  arch/arm64/include/asm/pci.h | 12 +++---------
> >  arch/csky/include/asm/pci.h  | 24 ++++--------------------
> >  arch/riscv/include/asm/pci.h | 25 +++----------------------
> >  arch/um/include/asm/pci.h    | 24 ++----------------------
> >  include/asm-generic/pci.h    | 36 ++++++++++++++++++++++++++++++++++++
> >  5 files changed, 48 insertions(+), 73 deletions(-)
> >  create mode 100644 include/asm-generic/pci.h
> >
> > diff --git a/arch/arm64/include/asm/pci.h b/arch/arm64/include/asm/pci.h
> > index b33ca260e3c9..1180e83712f5 100644
> > --- a/arch/arm64/include/asm/pci.h
> > +++ b/arch/arm64/include/asm/pci.h
> > @@ -9,7 +9,6 @@
> >  #include <asm/io.h>
> >
> >  #define PCIBIOS_MIN_IO               0x1000
> > -#define PCIBIOS_MIN_MEM              0
> >
> >  /*
> >   * Set to 1 if the kernel should re-assign all PCI bus numbers
> > @@ -18,9 +17,6 @@
> >       (pci_has_flag(PCI_REASSIGN_ALL_BUS))
> >
> >  #define arch_can_pci_mmap_wc() 1
> > -#define ARCH_GENERIC_PCI_MMAP_RESOURCE       1
> > -
> > -extern int isa_dma_bridge_buggy;
> >
> >  #ifdef CONFIG_PCI
> >  static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
> > @@ -28,11 +24,9 @@ static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
> >       /* no legacy IRQ on arm64 */
> >       return -ENODEV;
> >  }
> > -
> > -static inline int pci_proc_domain(struct pci_bus *bus)
> > -{
> > -     return 1;
> > -}
> >  #endif  /* CONFIG_PCI */
> >
> > +/* Generic PCI */
> > +#include <asm-generic/pci.h>
> > +
> >  #endif  /* __ASM_PCI_H */
> > diff --git a/arch/csky/include/asm/pci.h b/arch/csky/include/asm/pci.h
> > index ebc765b1f78b..44866c1ad461 100644
> > --- a/arch/csky/include/asm/pci.h
> > +++ b/arch/csky/include/asm/pci.h
> > @@ -9,26 +9,10 @@
> >
> >  #include <asm/io.h>
> >
> > -#define PCIBIOS_MIN_IO               0
> > -#define PCIBIOS_MIN_MEM              0
> > +/* Generic PCI */
> > +#include <asm-generic/pci.h>
> >
> > -/* C-SKY shim does not initialize PCI bus */
> > -#define pcibios_assign_all_busses() 1
> > -
> > -extern int isa_dma_bridge_buggy;
> > -
> > -#ifdef CONFIG_PCI
> > -static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
> > -{
> > -     /* no legacy IRQ on csky */
> > -     return -ENODEV;
> > -}
> > -
> > -static inline int pci_proc_domain(struct pci_bus *bus)
> > -{
> > -     /* always show the domain in /proc */
> > -     return 1;
> > -}
> > -#endif  /* CONFIG_PCI */
> > +/* csky doesn't use generic pci resource mapping */
> > +#undef ARCH_GENERIC_PCI_MMAP_RESOURCE
> >
> >  #endif  /* __ASM_CSKY_PCI_H */
> > diff --git a/arch/riscv/include/asm/pci.h b/arch/riscv/include/asm/pci.h
> > index 7fd52a30e605..12ce8150cfb0 100644
> > --- a/arch/riscv/include/asm/pci.h
> > +++ b/arch/riscv/include/asm/pci.h
> > @@ -12,29 +12,7 @@
> >
> >  #include <asm/io.h>
> >
> > -#define PCIBIOS_MIN_IO               0
> > -#define PCIBIOS_MIN_MEM              0
>
> My for-next changes these in bb356ddb78b2 ("RISC-V: PCI: Avoid handing
> out address 0 to devices").  Do you mind either splitting out the
> arch/riscv bits or having this in via some sort of shared tag?

Shouldn't the values not matter here if the IO and mem resources are
described in the DT (and don't use 0)? The values of 4 and 16 look
odd.

Rob

On Fri, Jul 22, 2022 at 1:06 AM Rob Herring <robh@kernel.org> wrote:
> On Tue, Jul 19, 2022 at 9:59 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
> > On Sun, 17 Jul 2022 17:41:14 PDT (-0700), shorne@gmail.com wrote:

> > > diff --git a/arch/riscv/include/asm/pci.h b/arch/riscv/include/asm/pci.h
> > > index 7fd52a30e605..12ce8150cfb0 100644
> > > --- a/arch/riscv/include/asm/pci.h
> > > +++ b/arch/riscv/include/asm/pci.h
> > > @@ -12,29 +12,7 @@
> > >
> > >  #include <asm/io.h>
> > >
> > > -#define PCIBIOS_MIN_IO               0
> > > -#define PCIBIOS_MIN_MEM              0
> >
> > My for-next changes these in bb356ddb78b2 ("RISC-V: PCI: Avoid handing
> > out address 0 to devices").  Do you mind either splitting out the
> > arch/riscv bits or having this in via some sort of shared tag?
>
> Shouldn't the values not matter here if the IO and mem resources are
> described in the DT (and don't use 0)? The values of 4 and 16 look
> odd.

I think it's different for the two types: For memory resources, this only
matters if the bus actually contains MMIO address zero. In most cases
the MMIO addresses are the same as the address seen by the CPU
and already nonzero based on the SoC design.

For the I/O port numbers, the port numbers tend to be more dynamic,
but you'd normally have addresses 0 through 0xffff on each PCI host
bridge with memory mapped I/O ports, so this can clearly happen.

Still, it seems better to not address the port zero issue in architecture
specific code but instead do it in the PCI core code. Ideally
we'd just use the 0x1000 minimum, which also helps stay out of
the ISA port numbers that may be used by things like
VGA or SATA adapters in legacy mode. The only reason I can
see for allowed smaller port numbers is for machines that have
a very limited I/O port window and do not have ports over
0x1000 at all.

        Arnd

On Thu, 21 Jul 2022 16:06:52 PDT (-0700), Rob Herring wrote:
> On Tue, Jul 19, 2022 at 9:59 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>>
>> On Sun, 17 Jul 2022 17:41:14 PDT (-0700), shorne@gmail.com wrote:
>> > The asm/pci.h used for many newer architectures share similar
>> > definitions.  Move the common parts to asm-generic/pci.h to allow for
>> > sharing code.
>> >
>> > Two things to note are:
>> >
>> >  - isa_dma_bridge_buggy, traditionally this is defined in asm/dma.h but
>> >    these architectures avoid creating that file and add the definition
>> >    to asm/pci.h.
>> >  - ARCH_GENERIC_PCI_MMAP_RESOURCE, csky does not define this so we
>> >    undefine it after including asm-generic/pci.h.  Why doesn't csky
>> >    define it?
>> >  - pci_get_legacy_ide_irq, This function is only used on architectures
>> >    that support PNP.  It is only maintained for arm64, in other
>> >    architectures it is removed.
>> >
>> > Suggested-by: Arnd Bergmann <arnd@arndb.de>
>> > Link: https://lore.kernel.org/lkml/CAK8P3a0JmPeczfmMBE__vn=Jbvf=nkbpVaZCycyv40pZNCJJXQ@mail.gmail.com/
>> > Signed-off-by: Stafford Horne <shorne@gmail.com>
>> > ---
>> > Second note on isa_dma_bridge_buggy, this is set on x86 but it it also set in
>> > pci/quirks.c.  We discussed limiting it only to x86 though as its a general
>> > quick triggered by pci ids I think it will be more tricky than we thought so I
>> > will leave as is.  It might be nice to move it out of asm/dma.h and into
>> > asm/pci.h though.
>> >
>> > Since v2:
>> >  - Nothing
>> > Since v1:
>> >  - Remove definition of pci_get_legacy_ide_irq
>> >
>> >  arch/arm64/include/asm/pci.h | 12 +++---------
>> >  arch/csky/include/asm/pci.h  | 24 ++++--------------------
>> >  arch/riscv/include/asm/pci.h | 25 +++----------------------
>> >  arch/um/include/asm/pci.h    | 24 ++----------------------
>> >  include/asm-generic/pci.h    | 36 ++++++++++++++++++++++++++++++++++++
>> >  5 files changed, 48 insertions(+), 73 deletions(-)
>> >  create mode 100644 include/asm-generic/pci.h
>> >
>> > diff --git a/arch/arm64/include/asm/pci.h b/arch/arm64/include/asm/pci.h
>> > index b33ca260e3c9..1180e83712f5 100644
>> > --- a/arch/arm64/include/asm/pci.h
>> > +++ b/arch/arm64/include/asm/pci.h
>> > @@ -9,7 +9,6 @@
>> >  #include <asm/io.h>
>> >
>> >  #define PCIBIOS_MIN_IO               0x1000
>> > -#define PCIBIOS_MIN_MEM              0
>> >
>> >  /*
>> >   * Set to 1 if the kernel should re-assign all PCI bus numbers
>> > @@ -18,9 +17,6 @@
>> >       (pci_has_flag(PCI_REASSIGN_ALL_BUS))
>> >
>> >  #define arch_can_pci_mmap_wc() 1
>> > -#define ARCH_GENERIC_PCI_MMAP_RESOURCE       1
>> > -
>> > -extern int isa_dma_bridge_buggy;
>> >
>> >  #ifdef CONFIG_PCI
>> >  static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
>> > @@ -28,11 +24,9 @@ static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
>> >       /* no legacy IRQ on arm64 */
>> >       return -ENODEV;
>> >  }
>> > -
>> > -static inline int pci_proc_domain(struct pci_bus *bus)
>> > -{
>> > -     return 1;
>> > -}
>> >  #endif  /* CONFIG_PCI */
>> >
>> > +/* Generic PCI */
>> > +#include <asm-generic/pci.h>
>> > +
>> >  #endif  /* __ASM_PCI_H */
>> > diff --git a/arch/csky/include/asm/pci.h b/arch/csky/include/asm/pci.h
>> > index ebc765b1f78b..44866c1ad461 100644
>> > --- a/arch/csky/include/asm/pci.h
>> > +++ b/arch/csky/include/asm/pci.h
>> > @@ -9,26 +9,10 @@
>> >
>> >  #include <asm/io.h>
>> >
>> > -#define PCIBIOS_MIN_IO               0
>> > -#define PCIBIOS_MIN_MEM              0
>> > +/* Generic PCI */
>> > +#include <asm-generic/pci.h>
>> >
>> > -/* C-SKY shim does not initialize PCI bus */
>> > -#define pcibios_assign_all_busses() 1
>> > -
>> > -extern int isa_dma_bridge_buggy;
>> > -
>> > -#ifdef CONFIG_PCI
>> > -static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
>> > -{
>> > -     /* no legacy IRQ on csky */
>> > -     return -ENODEV;
>> > -}
>> > -
>> > -static inline int pci_proc_domain(struct pci_bus *bus)
>> > -{
>> > -     /* always show the domain in /proc */
>> > -     return 1;
>> > -}
>> > -#endif  /* CONFIG_PCI */
>> > +/* csky doesn't use generic pci resource mapping */
>> > +#undef ARCH_GENERIC_PCI_MMAP_RESOURCE
>> >
>> >  #endif  /* __ASM_CSKY_PCI_H */
>> > diff --git a/arch/riscv/include/asm/pci.h b/arch/riscv/include/asm/pci.h
>> > index 7fd52a30e605..12ce8150cfb0 100644
>> > --- a/arch/riscv/include/asm/pci.h
>> > +++ b/arch/riscv/include/asm/pci.h
>> > @@ -12,29 +12,7 @@
>> >
>> >  #include <asm/io.h>
>> >
>> > -#define PCIBIOS_MIN_IO               0
>> > -#define PCIBIOS_MIN_MEM              0
>>
>> My for-next changes these in bb356ddb78b2 ("RISC-V: PCI: Avoid handing
>> out address 0 to devices").  Do you mind either splitting out the
>> arch/riscv bits or having this in via some sort of shared tag?
>
> Shouldn't the values not matter here if the IO and mem resources are
> described in the DT (and don't use 0)? The values of 4 and 16 look
> odd.

The linked thread has a fairly long discussion 
<https://lore.kernel.org/all/alpine.DEB.2.21.2202260044180.25061@angie.orcam.me.uk/>.  
I agree it's odd to have this in arch code: "don't hand out address 0" 
isn't really a RISC-V constraint (ie, we don't have architecture-defined 
limitations on these address spaces) but a constraint that comes from 
the generic port I/O functions and some other related PCI/resource code 
where the value 0 is a sentinel.

Maybe the right thing to do here is actually to make the default 
definitions of these macros non-zero, or to add some sort of ARCH_ 
flavor of them and move that non-zero requirement closer to where it 
comes from?  From the look of it any port that uses the generic port I/O 
functions and has 0 for these will be broken in the same way.

That said, I'm not really a PCI guy so maybe Bjorn or Maciej has a 
better idea?

On Fri, Jul 22, 2022 at 9:27 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>
> On Thu, 21 Jul 2022 16:06:52 PDT (-0700), Rob Herring wrote:
> > On Tue, Jul 19, 2022 at 9:59 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
> >>
> >> On Sun, 17 Jul 2022 17:41:14 PDT (-0700), shorne@gmail.com wrote:
> >> > The asm/pci.h used for many newer architectures share similar
> >> > definitions.  Move the common parts to asm-generic/pci.h to allow for
> >> > sharing code.
> >> >
> >> > Two things to note are:
> >> >
> >> >  - isa_dma_bridge_buggy, traditionally this is defined in asm/dma.h but
> >> >    these architectures avoid creating that file and add the definition
> >> >    to asm/pci.h.
> >> >  - ARCH_GENERIC_PCI_MMAP_RESOURCE, csky does not define this so we
> >> >    undefine it after including asm-generic/pci.h.  Why doesn't csky
> >> >    define it?
> >> >  - pci_get_legacy_ide_irq, This function is only used on architectures
> >> >    that support PNP.  It is only maintained for arm64, in other
> >> >    architectures it is removed.
> >> >
> >> > Suggested-by: Arnd Bergmann <arnd@arndb.de>
> >> > Link: https://lore.kernel.org/lkml/CAK8P3a0JmPeczfmMBE__vn=Jbvf=nkbpVaZCycyv40pZNCJJXQ@mail.gmail.com/
> >> > Signed-off-by: Stafford Horne <shorne@gmail.com>
> >> > ---
> >> > Second note on isa_dma_bridge_buggy, this is set on x86 but it it also set in
> >> > pci/quirks.c.  We discussed limiting it only to x86 though as its a general
> >> > quick triggered by pci ids I think it will be more tricky than we thought so I
> >> > will leave as is.  It might be nice to move it out of asm/dma.h and into
> >> > asm/pci.h though.
> >> >
> >> > Since v2:
> >> >  - Nothing
> >> > Since v1:
> >> >  - Remove definition of pci_get_legacy_ide_irq
> >> >
> >> >  arch/arm64/include/asm/pci.h | 12 +++---------
> >> >  arch/csky/include/asm/pci.h  | 24 ++++--------------------
> >> >  arch/riscv/include/asm/pci.h | 25 +++----------------------
> >> >  arch/um/include/asm/pci.h    | 24 ++----------------------
> >> >  include/asm-generic/pci.h    | 36 ++++++++++++++++++++++++++++++++++++
> >> >  5 files changed, 48 insertions(+), 73 deletions(-)
> >> >  create mode 100644 include/asm-generic/pci.h
> >> >
> >> > diff --git a/arch/arm64/include/asm/pci.h b/arch/arm64/include/asm/pci.h
> >> > index b33ca260e3c9..1180e83712f5 100644
> >> > --- a/arch/arm64/include/asm/pci.h
> >> > +++ b/arch/arm64/include/asm/pci.h
> >> > @@ -9,7 +9,6 @@
> >> >  #include <asm/io.h>
> >> >
> >> >  #define PCIBIOS_MIN_IO               0x1000
> >> > -#define PCIBIOS_MIN_MEM              0
> >> >
> >> >  /*
> >> >   * Set to 1 if the kernel should re-assign all PCI bus numbers
> >> > @@ -18,9 +17,6 @@
> >> >       (pci_has_flag(PCI_REASSIGN_ALL_BUS))
> >> >
> >> >  #define arch_can_pci_mmap_wc() 1
> >> > -#define ARCH_GENERIC_PCI_MMAP_RESOURCE       1
> >> > -
> >> > -extern int isa_dma_bridge_buggy;
> >> >
> >> >  #ifdef CONFIG_PCI
> >> >  static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
> >> > @@ -28,11 +24,9 @@ static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
> >> >       /* no legacy IRQ on arm64 */
> >> >       return -ENODEV;
> >> >  }
> >> > -
> >> > -static inline int pci_proc_domain(struct pci_bus *bus)
> >> > -{
> >> > -     return 1;
> >> > -}
> >> >  #endif  /* CONFIG_PCI */
> >> >
> >> > +/* Generic PCI */
> >> > +#include <asm-generic/pci.h>
> >> > +
> >> >  #endif  /* __ASM_PCI_H */
> >> > diff --git a/arch/csky/include/asm/pci.h b/arch/csky/include/asm/pci.h
> >> > index ebc765b1f78b..44866c1ad461 100644
> >> > --- a/arch/csky/include/asm/pci.h
> >> > +++ b/arch/csky/include/asm/pci.h
> >> > @@ -9,26 +9,10 @@
> >> >
> >> >  #include <asm/io.h>
> >> >
> >> > -#define PCIBIOS_MIN_IO               0
> >> > -#define PCIBIOS_MIN_MEM              0
> >> > +/* Generic PCI */
> >> > +#include <asm-generic/pci.h>
> >> >
> >> > -/* C-SKY shim does not initialize PCI bus */
> >> > -#define pcibios_assign_all_busses() 1
> >> > -
> >> > -extern int isa_dma_bridge_buggy;
> >> > -
> >> > -#ifdef CONFIG_PCI
> >> > -static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
> >> > -{
> >> > -     /* no legacy IRQ on csky */
> >> > -     return -ENODEV;
> >> > -}
> >> > -
> >> > -static inline int pci_proc_domain(struct pci_bus *bus)
> >> > -{
> >> > -     /* always show the domain in /proc */
> >> > -     return 1;
> >> > -}
> >> > -#endif  /* CONFIG_PCI */
> >> > +/* csky doesn't use generic pci resource mapping */
> >> > +#undef ARCH_GENERIC_PCI_MMAP_RESOURCE
> >> >
> >> >  #endif  /* __ASM_CSKY_PCI_H */
> >> > diff --git a/arch/riscv/include/asm/pci.h b/arch/riscv/include/asm/pci.h
> >> > index 7fd52a30e605..12ce8150cfb0 100644
> >> > --- a/arch/riscv/include/asm/pci.h
> >> > +++ b/arch/riscv/include/asm/pci.h
> >> > @@ -12,29 +12,7 @@
> >> >
> >> >  #include <asm/io.h>
> >> >
> >> > -#define PCIBIOS_MIN_IO               0
> >> > -#define PCIBIOS_MIN_MEM              0
> >>
> >> My for-next changes these in bb356ddb78b2 ("RISC-V: PCI: Avoid handing
> >> out address 0 to devices").  Do you mind either splitting out the
> >> arch/riscv bits or having this in via some sort of shared tag?
> >
> > Shouldn't the values not matter here if the IO and mem resources are
> > described in the DT (and don't use 0)? The values of 4 and 16 look
> > odd.
>
> The linked thread has a fairly long discussion
> <https://lore.kernel.org/all/alpine.DEB.2.21.2202260044180.25061@angie.orcam.me.uk/>.
> I agree it's odd to have this in arch code: "don't hand out address 0"
> isn't really a RISC-V constraint (ie, we don't have architecture-defined
> limitations on these address spaces) but a constraint that comes from
> the generic port I/O functions and some other related PCI/resource code
> where the value 0 is a sentinel.

If you look at arm32, we have a variable for PCIBIOS_MIN_MEM because
pre-DT what platforms required was all over the place. Nothing using
DT needs to set that variable. And arm64 uses 0 without problems. In
all those platforms, none of them have the same restrictions? So it is
still curious to me how PCIBIOS_MIN_MEM matters for Risc-V.

I/O is different as Arnd said, but I'd imagine we could just set the
min to 4 in the generic header and be done with it.

> Maybe the right thing to do here is actually to make the default
> definitions of these macros non-zero, or to add some sort of ARCH_
> flavor of them and move that non-zero requirement closer to where it
> comes from?  From the look of it any port that uses the generic port I/O
> functions and has 0 for these will be broken in the same way.
>
> That said, I'm not really a PCI guy so maybe Bjorn or Maciej has a
> better idea?

From fu740:
                       ranges = <0x81000000  0x0 0x60080000  0x0
0x60080000 0x0 0x10000>,      /* I/O */
                                 <0x82000000  0x0 0x60090000  0x0
0x60090000 0x0 0xff70000>,    /* mem */
                                 <0x82000000  0x0 0x70000000  0x0
0x70000000 0x0 0x1000000>,    /* mem */
                                 <0xc3000000 0x20 0x00000000 0x20
0x00000000 0x20 0x00000000>;  /* mem prefetchable */

So again, how does one get a 0 address handed out when that's not even
a valid region according to DT? Is there some legacy stuff that
ignores the bridge windows?

Rob

On Fri, 22 Jul 2022, Rob Herring wrote:

> > Maybe the right thing to do here is actually to make the default
> > definitions of these macros non-zero, or to add some sort of ARCH_
> > flavor of them and move that non-zero requirement closer to where it
> > comes from?  From the look of it any port that uses the generic port I/O
> > functions and has 0 for these will be broken in the same way.
> >
> > That said, I'm not really a PCI guy so maybe Bjorn or Maciej has a
> > better idea?
> 
> >From fu740:
>                        ranges = <0x81000000  0x0 0x60080000  0x0
> 0x60080000 0x0 0x10000>,      /* I/O */
>                                  <0x82000000  0x0 0x60090000  0x0
> 0x60090000 0x0 0xff70000>,    /* mem */
>                                  <0x82000000  0x0 0x70000000  0x0
> 0x70000000 0x0 0x1000000>,    /* mem */
>                                  <0xc3000000 0x20 0x00000000 0x20
> 0x00000000 0x20 0x00000000>;  /* mem prefetchable */
> 
> So again, how does one get a 0 address handed out when that's not even
> a valid region according to DT? Is there some legacy stuff that
> ignores the bridge windows?

 It doesn't matter as <asm/pci.h> just sets it as a generic parameter for 
the platform, reflecting the limitation of PCI core, which in the course 
of the discussion referred was found rather infeasible to remove.  The 
FU740 does not decode to PCI at 0, but another RISC-V device could.  And I 
think that DT should faithfully describe hardware and not our software 
limitations.

 Mind that PCI has originated from the x86 world where decoding low 24-bit 
memory space to ISA has been implied (implicitly decoded on PCI systems by 
the southbridge) for areas not decoded to DRAM by the memory controller. 
So the inability of our PCI core to handle MMIO at 0 did not matter at the 
time it was introduced as the value of 0 would never be used for a memory 
BAR.

  Maciej

On Fri, Jul 22, 2022 at 1:23 PM Maciej W. Rozycki <macro@orcam.me.uk> wrote:
>
> On Fri, 22 Jul 2022, Rob Herring wrote:
>
> > > Maybe the right thing to do here is actually to make the default
> > > definitions of these macros non-zero, or to add some sort of ARCH_
> > > flavor of them and move that non-zero requirement closer to where it
> > > comes from?  From the look of it any port that uses the generic port I/O
> > > functions and has 0 for these will be broken in the same way.
> > >
> > > That said, I'm not really a PCI guy so maybe Bjorn or Maciej has a
> > > better idea?
> >
> > >From fu740:
> >                        ranges = <0x81000000  0x0 0x60080000  0x0
> > 0x60080000 0x0 0x10000>,      /* I/O */
> >                                  <0x82000000  0x0 0x60090000  0x0
> > 0x60090000 0x0 0xff70000>,    /* mem */
> >                                  <0x82000000  0x0 0x70000000  0x0
> > 0x70000000 0x0 0x1000000>,    /* mem */
> >                                  <0xc3000000 0x20 0x00000000 0x20
> > 0x00000000 0x20 0x00000000>;  /* mem prefetchable */
> >
> > So again, how does one get a 0 address handed out when that's not even
> > a valid region according to DT? Is there some legacy stuff that
> > ignores the bridge windows?
>
>  It doesn't matter as <asm/pci.h> just sets it as a generic parameter for
> the platform, reflecting the limitation of PCI core, which in the course
> of the discussion referred was found rather infeasible to remove.  The
> FU740 does not decode to PCI at 0, but another RISC-V device could.  And I
> think that DT should faithfully describe hardware and not our software
> limitations.

Let me ask this another way. When would a 0 memory or i/o address ever
work? It doesn't seem this s/w limitation has anything specific to
Risc-V. Given pci_iomap_range() rejects 0, I can't see how it could
ever work. Maybe only for legacy ISA? So should the generic defaults
just be what Risc-V is using instead of 0?

Rob

On Fri, Jul 22, 2022 at 6:36 PM Rob Herring <robh@kernel.org> wrote:
> On Fri, Jul 22, 2022 at 9:27 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>
> From fu740:
>                        ranges = <0x81000000  0x0 0x60080000  0x0
> 0x60080000 0x0 0x10000>,      /* I/O */
...
> So again, how does one get a 0 address handed out when that's not even
> a valid region according to DT? Is there some legacy stuff that
> ignores the bridge windows?

The PCI-side port number 0x60080000 gets turned into Linux I/O resource 0,
which I think is what __pci_assign_resource operates on.

The other question is why the platform would want to configure the
PCI bus to have a PCI I/O space window of size 0x10000 at the address
it's mapped into, rather than putting it at address zero. Is this a hardware
bug, a bootloader bug, or just badly set up in the DT?

Putting the PCI address of the I/O space window at port 0 is usually
better because it works with PCI devices and drivers that assume that
port numbers are below 0xfffff, and makes the PCI port number match
the Linux port number.

         Arnd

On 22 Jul 2022, at 20:55, Arnd Bergmann <arnd@arndb.de> wrote:
> 
> On Fri, Jul 22, 2022 at 6:36 PM Rob Herring <robh@kernel.org> wrote:
>> On Fri, Jul 22, 2022 at 9:27 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>> 
>> From fu740:
>>                       ranges = <0x81000000  0x0 0x60080000  0x0
>> 0x60080000 0x0 0x10000>,      /* I/O */
> ...
>> So again, how does one get a 0 address handed out when that's not even
>> a valid region according to DT? Is there some legacy stuff that
>> ignores the bridge windows?
> 
> The PCI-side port number 0x60080000 gets turned into Linux I/O resource 0,
> which I think is what __pci_assign_resource operates on.
> 
> The other question is why the platform would want to configure the
> PCI bus to have a PCI I/O space window of size 0x10000 at the address
> it's mapped into, rather than putting it at address zero. Is this a hardware
> bug, a bootloader bug, or just badly set up in the DT?
> 
> Putting the PCI address of the I/O space window at port 0 is usually
> better because it works with PCI devices and drivers that assume that
> port numbers are below 0xfffff, and makes the PCI port number match
> the Linux port number.

Possibly related is the (harmless) warning spew seen during boot on
FreeBSD. The bridge, and all downstream ones, reset to having I/O
window 0x0-0xfff and memory/prefetch window 0x0-0xfffff (which FreeBSD
tries to allocate, fails because that’s outside the valid ranges above,
so it then just paves over with a sane fresh config). Linux resets the
controller on probe so this is the initial state when enumerating, and
perhaps that’s not handled so gracefully as on FreeBSD (which also
resets in device_attach, the second half of what Linux calls probe)?

Jess

On Fri, Jul 22, 2022 at 1:55 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Fri, Jul 22, 2022 at 6:36 PM Rob Herring <robh@kernel.org> wrote:
> > On Fri, Jul 22, 2022 at 9:27 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
> >
> > From fu740:
> >                        ranges = <0x81000000  0x0 0x60080000  0x0
> > 0x60080000 0x0 0x10000>,      /* I/O */
> ...
> > So again, how does one get a 0 address handed out when that's not even
> > a valid region according to DT? Is there some legacy stuff that
> > ignores the bridge windows?
>
> The PCI-side port number 0x60080000 gets turned into Linux I/O resource 0,
> which I think is what __pci_assign_resource operates on.
>
> The other question is why the platform would want to configure the
> PCI bus to have a PCI I/O space window of size 0x10000 at the address
> it's mapped into, rather than putting it at address zero. Is this a hardware
> bug, a bootloader bug, or just badly set up in the DT?

...putting it at *PCI* address zero, right? Yeah, that looks
suspicious. The core code seems to not use the PCI address, but
various drivers do. Maybe they are miscalculating things and still end
up with 0. If so, we're stuck with that ABI though we could fix it up
in the ranges parsing code and make driver behavior consistent.

In any case, that seems to be a somewhat common occurrence. A somewhat
accurate search (ignore the MBUS_ID ones):

$ git grep -A4 '\sranges =' -- arch/ | grep '0x81000000' | grep -v -E
'0x81000000\s[0x]+\s[0x]+\s'
arch/arm/boot/dts/armada-370.dtsi-
0x81000000 0x1 0     MBUS_ID(0x04, 0xe0) 0       1 0 /* Port 0.0 IO
*/
arch/arm/boot/dts/armada-375.dtsi-
0x81000000 0x1 0       MBUS_ID(0x04, 0xe0) 0 1 0 /* Port 0 IO  */
arch/arm/boot/dts/armada-xp-98dx3236.dtsi-
 0x81000000 0x1 0       MBUS_ID(0x04, 0xe0) 0 1 0 /* Port 0.0 IO  */>;
arch/arm/boot/dts/bcm47622.dtsi:                ranges = <0 0x81000000
0x818000>;
arch/arm/boot/dts/dove.dtsi-                              0x81000000
0x1 0x0 MBUS_ID(0x04, 0xe0) 0 1 0   /* Port 0.0 I/O */
arch/arm/boot/dts/kirkwood-6192.dtsi-
0x81000000 0x1 0     MBUS_ID(0x04, 0xe0) 0       1 0 /* Port 0.0 IO
*/>;
arch/arm/boot/dts/kirkwood-6281.dtsi-
0x81000000 0x1 0     MBUS_ID(0x04, 0xe0) 0       1 0 /* Port 0.0 IO
*/>;
arch/arm/boot/dts/kirkwood-98dx4122.dtsi-
 0x81000000 0x1 0     MBUS_ID(0x04, 0xe0) 0       1 0 /* Port 0.0 IO
*/>;
arch/arm/boot/dts/mt7623.dtsi:          ranges = <0x81000000 0
0x1a160000 0 0x1a160000 0 0x00010000
arch/arm/boot/dts/qcom-ipq4019.dtsi:                    ranges =
<0x81000000 0 0x40200000 0x40200000 0 0x00100000>,
arch/arm/boot/dts/qcom-ipq8064.dtsi:                    ranges =
<0x81000000 0 0x0fe00000 0x0fe00000 0 0x00100000   /* downstream I/O
*/
arch/arm/boot/dts/qcom-ipq8064.dtsi:                    ranges =
<0x81000000 0 0x31e00000 0x31e00000 0 0x00100000   /* downstream I/O
*/
arch/arm/boot/dts/qcom-ipq8064.dtsi:                    ranges =
<0x81000000 0 0x35e00000 0x35e00000 0 0x00100000   /* downstream I/O
*/
arch/arm64/boot/dts/broadcom/bcm4908/bcm4908.dtsi:              ranges
= <0x00 0x00 0x81000000 0x4000>;
arch/arm64/boot/dts/marvell/armada-3720-turris-mox.dts: ranges =
<0x81000000 0 0xe8000000   0 0xe8000000   0 0x01000000   /* Port 0 IO
*/
arch/arm64/boot/dts/mediatek/mt8192.dtsi-
  <0x81000000 0 0x12800000 0x0 0x12800000 0 0x0800000>;
arch/arm64/boot/dts/qcom/ipq6018.dtsi:                  ranges =
<0x81000000 0 0x20200000 0 0x20200000
arch/arm64/boot/dts/qcom/ipq8074.dtsi:                  ranges =
<0x81000000 0 0x10200000 0x10200000
arch/arm64/boot/dts/qcom/ipq8074.dtsi:                  ranges =
<0x81000000 0 0x20200000 0x20200000
arch/arm64/boot/dts/rockchip/rk3399.dtsi-
<0x81000000 0x0 0xfbe00000 0x0 0xfbe00000 0x0 0x100000>;
arch/arm64/boot/dts/toshiba/tmpv7708.dtsi:                      ranges
= <0x81000000 0 0x40000000 0 0x40000000 0 0x00010000
arch/riscv/boot/dts/sifive/fu740-c000.dtsi:                     ranges
= <0x81000000  0x0 0x60080000  0x0 0x60080000 0x0 0x10000>,      /*
I/O */


> Putting the PCI address of the I/O space window at port 0 is usually
> better because it works with PCI devices and drivers that assume that
> port numbers are below 0xfffff, and makes the PCI port number match
> the Linux port number.

I could make the PCI schema check for this, but I guess technically
any 32-bit PCI I/O address is valid even if 64K is the practical
limit.

Rob

P.S. I really wish I/O space would disappear completely.

On Fri, 22 Jul 2022, Rob Herring wrote:

> > > So again, how does one get a 0 address handed out when that's not even
> > > a valid region according to DT? Is there some legacy stuff that
> > > ignores the bridge windows?
> >
> >  It doesn't matter as <asm/pci.h> just sets it as a generic parameter for
> > the platform, reflecting the limitation of PCI core, which in the course
> > of the discussion referred was found rather infeasible to remove.  The
> > FU740 does not decode to PCI at 0, but another RISC-V device could.  And I
> > think that DT should faithfully describe hardware and not our software
> > limitations.
> 
> Let me ask this another way. When would a 0 memory or i/o address ever
> work? It doesn't seem this s/w limitation has anything specific to
> Risc-V. Given pci_iomap_range() rejects 0, I can't see how it could
> ever work. Maybe only for legacy ISA? So should the generic defaults
> just be what Risc-V is using instead of 0?

 Absolutely, cf.: 
<https://lore.kernel.org/lkml/alpine.DEB.2.21.2202260044180.25061@angie.orcam.me.uk/>.

  Maciej

On Fri, 22 Jul 2022, Rob Herring wrote:

> P.S. I really wish I/O space would disappear completely.

 Some systems make it happen already, such as the POWER9 platform and its 
PHB4 host bridge (which doesn't handle PCIe TLP I/O read or write commands 
at all), however some PCI/e devices do require I/O space, such as IEEE 
1284 parallel port adapters.  Some PCI devices dating back as far as to 
1990s provide dual mapping of their resources via an I/O and a memory BAR 
both at a time, so we can choose which to use depending on circumstances.

 FWIW,

  Maciej