diff mbox series

PCI: dwc: Program outbound ATU upper limit register

Message ID 1585785493-23210-1-git-send-email-alan.mikhak@sifive.com
State New
Headers show
Series PCI: dwc: Program outbound ATU upper limit register | expand

Commit Message

Alan Mikhak April 1, 2020, 11:58 p.m. UTC 
From: Alan Mikhak <alan.mikhak@sifive.com>

Function dw_pcie_prog_outbound_atu_unroll() does not program the upper
32-bit ATU limit register. Since ATU programming functions limit the
size of the translated region to 4GB by using a u32 size parameter,
these issues may combine into undefined behavior for resource sizes
with non-zero upper 32-bits.

For example, a 128GB address space starting at physical CPU address of
0x2000000000 with size of 0x2000000000 needs the following values
programmed into the lower and upper 32-bit limit registers:
 0x3fffffff in the upper 32-bit limit register
 0xffffffff in the lower 32-bit limit register

Currently, only the lower 32-bit limit register is programmed with a
value of 0xffffffff but the upper 32-bit limit register is not being
programmed. As a result, the upper 32-bit limit register remains at its
default value after reset of 0x0.

These issues may combine to produce undefined behavior since the ATU
limit address may be lower than the ATU base address. Programming the
upper ATU limit address register prevents such undefined behavior despite
the region size getting truncated due to the 32-bit size limit.

Signed-off-by: Alan Mikhak <alan.mikhak@sifive.com>
---
 drivers/pci/controller/dwc/pcie-designware.c | 7 +++++--
 drivers/pci/controller/dwc/pcie-designware.h | 3 ++-
 2 files changed, 7 insertions(+), 3 deletions(-)

Comments

Lorenzo Pieralisi May 5, 2020, 10:29 a.m. UTC | #1 
On Wed, Apr 01, 2020 at 04:58:13PM -0700, Alan Mikhak wrote:
> From: Alan Mikhak <alan.mikhak@sifive.com>
> 
> Function dw_pcie_prog_outbound_atu_unroll() does not program the upper
> 32-bit ATU limit register. Since ATU programming functions limit the
> size of the translated region to 4GB by using a u32 size parameter,
> these issues may combine into undefined behavior for resource sizes
> with non-zero upper 32-bits.
> 
> For example, a 128GB address space starting at physical CPU address of
> 0x2000000000 with size of 0x2000000000 needs the following values
> programmed into the lower and upper 32-bit limit registers:
>  0x3fffffff in the upper 32-bit limit register
>  0xffffffff in the lower 32-bit limit register
> 
> Currently, only the lower 32-bit limit register is programmed with a
> value of 0xffffffff but the upper 32-bit limit register is not being
> programmed. As a result, the upper 32-bit limit register remains at its
> default value after reset of 0x0.
> 
> These issues may combine to produce undefined behavior since the ATU
> limit address may be lower than the ATU base address. Programming the
> upper ATU limit address register prevents such undefined behavior despite
> the region size getting truncated due to the 32-bit size limit.
> 
> Signed-off-by: Alan Mikhak <alan.mikhak@sifive.com>
> ---
>  drivers/pci/controller/dwc/pcie-designware.c | 7 +++++--
>  drivers/pci/controller/dwc/pcie-designware.h | 3 ++-
>  2 files changed, 7 insertions(+), 3 deletions(-)

I would appreciate some feedback and possibly and ACK from DWC
maintainers. Should this go to stable kernels ? It seems so,
let me know if we want to add a stable tag.

I will merge it, along with:

https://patchwork.kernel.org/patch/11468465/

Lorenzo

> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> index 681548c88282..c92496e36fd5 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.c
> +++ b/drivers/pci/controller/dwc/pcie-designware.c
> @@ -244,13 +244,16 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, int index,
>  					     u64 pci_addr, u32 size)
>  {
>  	u32 retries, val;
> +	u64 limit_addr = cpu_addr + size - 1;
>  
>  	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_BASE,
>  				 lower_32_bits(cpu_addr));
>  	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_BASE,
>  				 upper_32_bits(cpu_addr));
> -	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LIMIT,
> -				 lower_32_bits(cpu_addr + size - 1));
> +	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_LIMIT,
> +				 lower_32_bits(limit_addr));
> +	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_LIMIT,
> +				 upper_32_bits(limit_addr));
>  	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_TARGET,
>  				 lower_32_bits(pci_addr));
>  	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
> diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> index a22ea5982817..5ce1aef706c5 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.h
> +++ b/drivers/pci/controller/dwc/pcie-designware.h
> @@ -112,9 +112,10 @@
>  #define PCIE_ATU_UNR_REGION_CTRL2	0x04
>  #define PCIE_ATU_UNR_LOWER_BASE		0x08
>  #define PCIE_ATU_UNR_UPPER_BASE		0x0C
> -#define PCIE_ATU_UNR_LIMIT		0x10
> +#define PCIE_ATU_UNR_LOWER_LIMIT	0x10
>  #define PCIE_ATU_UNR_LOWER_TARGET	0x14
>  #define PCIE_ATU_UNR_UPPER_TARGET	0x18
> +#define PCIE_ATU_UNR_UPPER_LIMIT	0x20
>  
>  /*
>   * The default address offset between dbi_base and atu_base. Root controller
> -- 
> 2.7.4
>
Gustavo Pimentel May 5, 2020, 11:16 a.m. UTC | #2 
On Thu, Apr 2, 2020 at 0:58:13, Alan Mikhak <alan.mikhak@sifive.com> 
wrote:

> From: Alan Mikhak <alan.mikhak@sifive.com>
> 
> Function dw_pcie_prog_outbound_atu_unroll() does not program the upper
> 32-bit ATU limit register. Since ATU programming functions limit the
> size of the translated region to 4GB by using a u32 size parameter,
> these issues may combine into undefined behavior for resource sizes
> with non-zero upper 32-bits.
> 
> For example, a 128GB address space starting at physical CPU address of
> 0x2000000000 with size of 0x2000000000 needs the following values
> programmed into the lower and upper 32-bit limit registers:
>  0x3fffffff in the upper 32-bit limit register
>  0xffffffff in the lower 32-bit limit register
> 
> Currently, only the lower 32-bit limit register is programmed with a
> value of 0xffffffff but the upper 32-bit limit register is not being
> programmed. As a result, the upper 32-bit limit register remains at its
> default value after reset of 0x0.
> 
> These issues may combine to produce undefined behavior since the ATU
> limit address may be lower than the ATU base address. Programming the
> upper ATU limit address register prevents such undefined behavior despite
> the region size getting truncated due to the 32-bit size limit.
> 
> Signed-off-by: Alan Mikhak <alan.mikhak@sifive.com>
> ---
>  drivers/pci/controller/dwc/pcie-designware.c | 7 +++++--
>  drivers/pci/controller/dwc/pcie-designware.h | 3 ++-
>  2 files changed, 7 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> index 681548c88282..c92496e36fd5 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.c
> +++ b/drivers/pci/controller/dwc/pcie-designware.c
> @@ -244,13 +244,16 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, int index,
>  					     u64 pci_addr, u32 size)
>  {
>  	u32 retries, val;
> +	u64 limit_addr = cpu_addr + size - 1;
>  
>  	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_BASE,
>  				 lower_32_bits(cpu_addr));
>  	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_BASE,
>  				 upper_32_bits(cpu_addr));
> -	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LIMIT,
> -				 lower_32_bits(cpu_addr + size - 1));
> +	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_LIMIT,
> +				 lower_32_bits(limit_addr));
> +	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_LIMIT,
> +				 upper_32_bits(limit_addr));
>  	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_TARGET,
>  				 lower_32_bits(pci_addr));
>  	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
> diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> index a22ea5982817..5ce1aef706c5 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.h
> +++ b/drivers/pci/controller/dwc/pcie-designware.h
> @@ -112,9 +112,10 @@
>  #define PCIE_ATU_UNR_REGION_CTRL2	0x04
>  #define PCIE_ATU_UNR_LOWER_BASE		0x08
>  #define PCIE_ATU_UNR_UPPER_BASE		0x0C
> -#define PCIE_ATU_UNR_LIMIT		0x10
> +#define PCIE_ATU_UNR_LOWER_LIMIT	0x10
>  #define PCIE_ATU_UNR_LOWER_TARGET	0x14
>  #define PCIE_ATU_UNR_UPPER_TARGET	0x18
> +#define PCIE_ATU_UNR_UPPER_LIMIT	0x20
>  
>  /*
>   * The default address offset between dbi_base and atu_base. Root controller
> -- 
> 2.7.4


Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Gustavo Pimentel May 5, 2020, 11:24 a.m. UTC | #3 
Hi Lorenzo,

On Tue, May 5, 2020 at 11:29:33, Lorenzo Pieralisi 
<lorenzo.pieralisi@arm.com> wrote:

> On Wed, Apr 01, 2020 at 04:58:13PM -0700, Alan Mikhak wrote:
> > From: Alan Mikhak <alan.mikhak@sifive.com>
> > 
> > Function dw_pcie_prog_outbound_atu_unroll() does not program the upper
> > 32-bit ATU limit register. Since ATU programming functions limit the
> > size of the translated region to 4GB by using a u32 size parameter,
> > these issues may combine into undefined behavior for resource sizes
> > with non-zero upper 32-bits.
> > 
> > For example, a 128GB address space starting at physical CPU address of
> > 0x2000000000 with size of 0x2000000000 needs the following values
> > programmed into the lower and upper 32-bit limit registers:
> >  0x3fffffff in the upper 32-bit limit register
> >  0xffffffff in the lower 32-bit limit register
> > 
> > Currently, only the lower 32-bit limit register is programmed with a
> > value of 0xffffffff but the upper 32-bit limit register is not being
> > programmed. As a result, the upper 32-bit limit register remains at its
> > default value after reset of 0x0.
> > 
> > These issues may combine to produce undefined behavior since the ATU
> > limit address may be lower than the ATU base address. Programming the
> > upper ATU limit address register prevents such undefined behavior despite
> > the region size getting truncated due to the 32-bit size limit.
> > 
> > Signed-off-by: Alan Mikhak <alan.mikhak@sifive.com>
> > ---
> >  drivers/pci/controller/dwc/pcie-designware.c | 7 +++++--
> >  drivers/pci/controller/dwc/pcie-designware.h | 3 ++-
> >  2 files changed, 7 insertions(+), 3 deletions(-)
> 
> I would appreciate some feedback and possibly and ACK from DWC
> maintainers. Should this go to stable kernels ? It seems so,
> let me know if we want to add a stable tag.
> 
> I will merge it, along with:
> 
> https://urldefense.com/v3/__https://patchwork.kernel.org/patch/11468465/__;!!A4F2R9G_pg!NoymSJCWmOx51jB7LdQQAbXFin14nfuVIQNQxROnskLmmGkzFeNOrf8nFWX_-KgsgO87N9M$ 
> 
> Lorenzo

Sorry for the delay. I just gave the ACK to that patch. For me, it makes 
sense to me to send it along with the patch that you just referred to the 
stable kernels.

-Gustavo
Lorenzo Pieralisi May 12, 2020, 10:52 a.m. UTC | #4 
On Wed, Apr 01, 2020 at 04:58:13PM -0700, Alan Mikhak wrote:
> From: Alan Mikhak <alan.mikhak@sifive.com>
> 
> Function dw_pcie_prog_outbound_atu_unroll() does not program the upper
> 32-bit ATU limit register. Since ATU programming functions limit the
> size of the translated region to 4GB by using a u32 size parameter,
> these issues may combine into undefined behavior for resource sizes
> with non-zero upper 32-bits.
> 
> For example, a 128GB address space starting at physical CPU address of
> 0x2000000000 with size of 0x2000000000 needs the following values
> programmed into the lower and upper 32-bit limit registers:
>  0x3fffffff in the upper 32-bit limit register
>  0xffffffff in the lower 32-bit limit register
> 
> Currently, only the lower 32-bit limit register is programmed with a
> value of 0xffffffff but the upper 32-bit limit register is not being
> programmed. As a result, the upper 32-bit limit register remains at its
> default value after reset of 0x0.
> 
> These issues may combine to produce undefined behavior since the ATU
> limit address may be lower than the ATU base address. Programming the
> upper ATU limit address register prevents such undefined behavior despite
> the region size getting truncated due to the 32-bit size limit.
> 
> Signed-off-by: Alan Mikhak <alan.mikhak@sifive.com>
> ---
>  drivers/pci/controller/dwc/pcie-designware.c | 7 +++++--
>  drivers/pci/controller/dwc/pcie-designware.h | 3 ++-
>  2 files changed, 7 insertions(+), 3 deletions(-)

Applied to pci/dwc, thanks.

Lorenzo

> diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> index 681548c88282..c92496e36fd5 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.c
> +++ b/drivers/pci/controller/dwc/pcie-designware.c
> @@ -244,13 +244,16 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, int index,
>  					     u64 pci_addr, u32 size)
>  {
>  	u32 retries, val;
> +	u64 limit_addr = cpu_addr + size - 1;
>  
>  	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_BASE,
>  				 lower_32_bits(cpu_addr));
>  	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_BASE,
>  				 upper_32_bits(cpu_addr));
> -	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LIMIT,
> -				 lower_32_bits(cpu_addr + size - 1));
> +	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_LIMIT,
> +				 lower_32_bits(limit_addr));
> +	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_LIMIT,
> +				 upper_32_bits(limit_addr));
>  	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_TARGET,
>  				 lower_32_bits(pci_addr));
>  	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
> diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> index a22ea5982817..5ce1aef706c5 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.h
> +++ b/drivers/pci/controller/dwc/pcie-designware.h
> @@ -112,9 +112,10 @@
>  #define PCIE_ATU_UNR_REGION_CTRL2	0x04
>  #define PCIE_ATU_UNR_LOWER_BASE		0x08
>  #define PCIE_ATU_UNR_UPPER_BASE		0x0C
> -#define PCIE_ATU_UNR_LIMIT		0x10
> +#define PCIE_ATU_UNR_LOWER_LIMIT	0x10
>  #define PCIE_ATU_UNR_LOWER_TARGET	0x14
>  #define PCIE_ATU_UNR_UPPER_TARGET	0x18
> +#define PCIE_ATU_UNR_UPPER_LIMIT	0x20
>  
>  /*
>   * The default address offset between dbi_base and atu_base. Root controller
> -- 
> 2.7.4
>
diff mbox series

Patch

diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index 681548c88282..c92496e36fd5 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -244,13 +244,16 @@  static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, int index,
 					     u64 pci_addr, u32 size)
 {
 	u32 retries, val;
+	u64 limit_addr = cpu_addr + size - 1;
 
 	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_BASE,
 				 lower_32_bits(cpu_addr));
 	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_BASE,
 				 upper_32_bits(cpu_addr));
-	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LIMIT,
-				 lower_32_bits(cpu_addr + size - 1));
+	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_LIMIT,
+				 lower_32_bits(limit_addr));
+	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_LIMIT,
+				 upper_32_bits(limit_addr));
 	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_TARGET,
 				 lower_32_bits(pci_addr));
 	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index a22ea5982817..5ce1aef706c5 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -112,9 +112,10 @@ 
 #define PCIE_ATU_UNR_REGION_CTRL2	0x04
 #define PCIE_ATU_UNR_LOWER_BASE		0x08
 #define PCIE_ATU_UNR_UPPER_BASE		0x0C
-#define PCIE_ATU_UNR_LIMIT		0x10
+#define PCIE_ATU_UNR_LOWER_LIMIT	0x10
 #define PCIE_ATU_UNR_LOWER_TARGET	0x14
 #define PCIE_ATU_UNR_UPPER_TARGET	0x18
+#define PCIE_ATU_UNR_UPPER_LIMIT	0x20
 
 /*
  * The default address offset between dbi_base and atu_base. Root controller