mbox series

[0/2,SRU,B] i40e: xps management broken when > 64 queues/cpus

Message ID 20190326083049.6273-1-nivedita.singhvi@canonical.com
Headers show
Series i40e: xps management broken when > 64 queues/cpus | expand

Message

Nivedita Singhvi March 26, 2019, 8:30 a.m. UTC
BugLink: https://bugs.launchpad.net/bugs/1820948

[Impact]
Transmit packet steering (xps) settings don't work when
the number of queues (cpus) is higher than 64. This is
currently still an issue on the 4.15 kernel (Xenial -hwe
and Bionic kernels).

It was fixed in Intel's i40e driver version 2.7.11 and
in 4.16-rc1 mainline Linux (i.e. Cosmic, Disco have fix).

Fix
-----
The following commit fixes this issue (as identified
by Lihong Yang in discussion with Intel i40e team):

"i40e: Fix the number of queues available to be mapped for use"
Commit: bc6d33c8d93f5999920e97a8c6330b8910053d4f

It requires the following commit as well: 

i40e: Do not allow use more TC queue pairs than MSI-X vectors exist
Commit:	1563f2d2e01242f05dd523ffd56fe104bc1afd58


[Test Case]
1. Kernel version: Bionic/Xenial -hwe: any 4.15 kernel
   i40e driver version: 2.1.14-k
   Any system with > 64 CPUs

2. For any queue 0 - 63, you can read/set tx xps:

echo ffffffff > /sys/class/net/eth2/queues/tx-63/xps_cpus
echo $?
0
cat /sys/class/net/eth2/queues/tx-63/xps_cpus
00,00000000,ffffffff

  But for any queue number > 63, we see this error:

echo ffffffff > /sys/class/net/eth2/queues/tx-64/xps_cpus
echo: write error: Invalid argument

cat /sys/class/net/eth2/queues/tx-64/xps_cpus
cat: /sys/class/net/eth2/queues/tx-64/xps_cpus: Invalid argument


Amritha Nambiar (1):
  i40e: Fix the number of queues available to be mapped for use

Paweł Jabłoński (1):
  i40e: Do not allow use more TC queue pairs than MSI-X vectors exist

 drivers/net/ethernet/intel/i40e/i40e_main.c | 29 ++++++++++++---------
 1 file changed, 17 insertions(+), 12 deletions(-)

Comments

Juerg Haefliger March 26, 2019, 10:31 a.m. UTC | #1
On Tue, 26 Mar 2019 14:00:46 +0530
Nivedita Singhvi <nivedita.singhvi@canonical.com> wrote:

> BugLink: https://bugs.launchpad.net/bugs/1820948
> 
> [Impact]
> Transmit packet steering (xps) settings don't work when
> the number of queues (cpus) is higher than 64. This is
> currently still an issue on the 4.15 kernel (Xenial -hwe
> and Bionic kernels).

Is this a problem in Xenial 4.4 as well?

...Juerg
Nivedita Singhvi March 26, 2019, 11:26 a.m. UTC | #2
On 3/26/19 4:01 PM, Juerg Haefliger wrote:
> On Tue, 26 Mar 2019 14:00:46 +0530
> Nivedita Singhvi <nivedita.singhvi@canonical.com> wrote:
> 
>> BugLink: https://bugs.launchpad.net/bugs/1820948
>>
>> [Impact]
>> Transmit packet steering (xps) settings don't work when
>> the number of queues (cpus) is higher than 64. This is
>> currently still an issue on the 4.15 kernel (Xenial -hwe
>> and Bionic kernels).
> 
> Is this a problem in Xenial 4.4 as well?
> 

From what I could tell to this point, it should not be (introduced
after), but I'm double-checking with the maintainer just in
case I have it wrong.

Nivedita
Kleber Sacilotto de Souza March 26, 2019, 12:10 p.m. UTC | #3
On 3/26/19 9:30 AM, Nivedita Singhvi wrote:
> BugLink: https://bugs.launchpad.net/bugs/1820948
> 
> [Impact]
> Transmit packet steering (xps) settings don't work when
> the number of queues (cpus) is higher than 64. This is
> currently still an issue on the 4.15 kernel (Xenial -hwe
> and Bionic kernels).
> 
> It was fixed in Intel's i40e driver version 2.7.11 and
> in 4.16-rc1 mainline Linux (i.e. Cosmic, Disco have fix).
> 
> Fix
> -----
> The following commit fixes this issue (as identified
> by Lihong Yang in discussion with Intel i40e team):
> 
> "i40e: Fix the number of queues available to be mapped for use"
> Commit: bc6d33c8d93f5999920e97a8c6330b8910053d4f
> 
> It requires the following commit as well: 
> 
> i40e: Do not allow use more TC queue pairs than MSI-X vectors exist
> Commit:	1563f2d2e01242f05dd523ffd56fe104bc1afd58
> 
> 
> [Test Case]
> 1. Kernel version: Bionic/Xenial -hwe: any 4.15 kernel
>    i40e driver version: 2.1.14-k
>    Any system with > 64 CPUs
> 
> 2. For any queue 0 - 63, you can read/set tx xps:
> 
> echo ffffffff > /sys/class/net/eth2/queues/tx-63/xps_cpus
> echo $?
> 0
> cat /sys/class/net/eth2/queues/tx-63/xps_cpus
> 00,00000000,ffffffff
> 
>   But for any queue number > 63, we see this error:
> 
> echo ffffffff > /sys/class/net/eth2/queues/tx-64/xps_cpus
> echo: write error: Invalid argument
> 
> cat /sys/class/net/eth2/queues/tx-64/xps_cpus
> cat: /sys/class/net/eth2/queues/tx-64/xps_cpus: Invalid argument
> 
> 
> Amritha Nambiar (1):
>   i40e: Fix the number of queues available to be mapped for use
> 
> Paweł Jabłoński (1):
>   i40e: Do not allow use more TC queue pairs than MSI-X vectors exist
> 
>  drivers/net/ethernet/intel/i40e/i40e_main.c | 29 ++++++++++++---------
>  1 file changed, 17 insertions(+), 12 deletions(-)
> 

With the "cherry picked" line fixed as pointed out by Juerg:

Acked-by: Kleber Sacilotto de Souza <kleber.souza@canonical.com>
Khalid Elmously March 27, 2019, 5:33 a.m. UTC | #4
On 2019-03-26 14:00:46 , Nivedita Singhvi wrote:
> BugLink: https://bugs.launchpad.net/bugs/1820948
> 
> [Impact]
> Transmit packet steering (xps) settings don't work when
> the number of queues (cpus) is higher than 64. This is
> currently still an issue on the 4.15 kernel (Xenial -hwe
> and Bionic kernels).
> 
> It was fixed in Intel's i40e driver version 2.7.11 and
> in 4.16-rc1 mainline Linux (i.e. Cosmic, Disco have fix).
> 
> Fix
> -----
> The following commit fixes this issue (as identified
> by Lihong Yang in discussion with Intel i40e team):
> 
> "i40e: Fix the number of queues available to be mapped for use"
> Commit: bc6d33c8d93f5999920e97a8c6330b8910053d4f
> 
> It requires the following commit as well: 
> 
> i40e: Do not allow use more TC queue pairs than MSI-X vectors exist
> Commit:	1563f2d2e01242f05dd523ffd56fe104bc1afd58
> 
> 
> [Test Case]
> 1. Kernel version: Bionic/Xenial -hwe: any 4.15 kernel
>    i40e driver version: 2.1.14-k
>    Any system with > 64 CPUs
> 
> 2. For any queue 0 - 63, you can read/set tx xps:
> 
> echo ffffffff > /sys/class/net/eth2/queues/tx-63/xps_cpus
> echo $?
> 0
> cat /sys/class/net/eth2/queues/tx-63/xps_cpus
> 00,00000000,ffffffff
> 
>   But for any queue number > 63, we see this error:
> 
> echo ffffffff > /sys/class/net/eth2/queues/tx-64/xps_cpus
> echo: write error: Invalid argument
> 
> cat /sys/class/net/eth2/queues/tx-64/xps_cpus
> cat: /sys/class/net/eth2/queues/tx-64/xps_cpus: Invalid argument
> 
> 
> Amritha Nambiar (1):
>   i40e: Fix the number of queues available to be mapped for use
> 
> Paweł Jabłoński (1):
>   i40e: Do not allow use more TC queue pairs than MSI-X vectors exist
> 
>  drivers/net/ethernet/intel/i40e/i40e_main.c | 29 ++++++++++++---------
>  1 file changed, 17 insertions(+), 12 deletions(-)
> 
Acked-by: Khalid Elmously <khalid.elmously@canonical.com>
Khalid Elmously March 27, 2019, 5:36 a.m. UTC | #5
Changed cherry-picked -> cherry picked


On 2019-03-26 14:00:46 , Nivedita Singhvi wrote:
> BugLink: https://bugs.launchpad.net/bugs/1820948
> 
> [Impact]
> Transmit packet steering (xps) settings don't work when
> the number of queues (cpus) is higher than 64. This is
> currently still an issue on the 4.15 kernel (Xenial -hwe
> and Bionic kernels).
> 
> It was fixed in Intel's i40e driver version 2.7.11 and
> in 4.16-rc1 mainline Linux (i.e. Cosmic, Disco have fix).
> 
> Fix
> -----
> The following commit fixes this issue (as identified
> by Lihong Yang in discussion with Intel i40e team):
> 
> "i40e: Fix the number of queues available to be mapped for use"
> Commit: bc6d33c8d93f5999920e97a8c6330b8910053d4f
> 
> It requires the following commit as well: 
> 
> i40e: Do not allow use more TC queue pairs than MSI-X vectors exist
> Commit:	1563f2d2e01242f05dd523ffd56fe104bc1afd58
> 
> 
> [Test Case]
> 1. Kernel version: Bionic/Xenial -hwe: any 4.15 kernel
>    i40e driver version: 2.1.14-k
>    Any system with > 64 CPUs
> 
> 2. For any queue 0 - 63, you can read/set tx xps:
> 
> echo ffffffff > /sys/class/net/eth2/queues/tx-63/xps_cpus
> echo $?
> 0
> cat /sys/class/net/eth2/queues/tx-63/xps_cpus
> 00,00000000,ffffffff
> 
>   But for any queue number > 63, we see this error:
> 
> echo ffffffff > /sys/class/net/eth2/queues/tx-64/xps_cpus
> echo: write error: Invalid argument
> 
> cat /sys/class/net/eth2/queues/tx-64/xps_cpus
> cat: /sys/class/net/eth2/queues/tx-64/xps_cpus: Invalid argument
> 
> 
> Amritha Nambiar (1):
>   i40e: Fix the number of queues available to be mapped for use
> 
> Paweł Jabłoński (1):
>   i40e: Do not allow use more TC queue pairs than MSI-X vectors exist
> 
>  drivers/net/ethernet/intel/i40e/i40e_main.c | 29 ++++++++++++---------
>  1 file changed, 17 insertions(+), 12 deletions(-)
> 
> -- 
> 2.17.1
> 
> 
> -- 
> kernel-team mailing list
> kernel-team@lists.ubuntu.com
> https://lists.ubuntu.com/mailman/listinfo/kernel-team