[{"id":1764832,"web_url":"http://patchwork.ozlabs.org/comment/1764832/","msgid":"","list_archive_url":null,"date":"2017-09-07T16:45:33","subject":"Re: [Intel-wired-lan] [RFC PATCH v3 0/6] Configuring traffic classes\n\tvia new hardware offload mechanism in tc/mqprio","submitter":{"id":70766,"url":"http://patchwork.ozlabs.org/api/people/70766/","name":"Shannon Nelson","email":"shannon.nelson@oracle.com"},"content":"On 9/7/2017 4:00 AM, Amritha Nambiar wrote:\n> The following series introduces a new hardware offload mode in\n> tc/mqprio where the TCs, the queue configurations and\n> bandwidth rate limits are offloaded to the hardware. The existing\n> mqprio framework is extended to configure the queue counts and\n> layout and also added support for rate limiting. This is achieved\n> through new netlink attributes for the 'mode' option which takes\n> values such as 'dcb' (default) and 'channel' and a 'shaper' option\n> for QoS attributes such as bandwidth rate limits in hw mode 1.\n> Legacy devices can fall back to the existing setup supporting hw mode\n> 1 without these additional options where only the TCs are offloaded\n> and then the 'mode' and 'shaper' options defaults to DCB support.\n> The i40e driver enables the new mqprio hardware offload mechanism\n> factoring the TCs, queue configuration and bandwidth rates by\n> creating HW channel VSIs.\n> \n> In this new mode, the priority to traffic class mapping and the\n> user specified queue ranges are used to configure the traffic\n> class when the 'mode' option is set to 'channel'. This is achieved by\n> creating HW channels(VSI). A new channel is created for each of the\n> traffic class configuration offloaded via mqprio framework except for\n> the first TC (TC0) which is for the main VSI. TC0 for the main VSI is\n> also reconfigured as per user provided queue parameters. Finally,\n> bandwidth rate limits are set on these traffic classes through the\n> shaper attribute by sending these rates in addition to the number of\n> TCs and the queue configurations.\n> \n> Example:\n> # tc qdisc add dev eth0 root mqprio num_tc 2 map 0 0 0 0 1 1 1 1\\\n> queues 4@0 4@4 hw 1 mode channel shaper bw_rlimit\\\n> min_rate 1Gbit 2Gbit max_rate 4Gbit 5Gbit\n> \n> To dump the bandwidth rates:\n> \n> # tc qdisc show dev eth0\n> \n> qdisc mqprio 804a: root tc 2 map 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0\n> queues:(0:3) (4:7)\n> mode:channel\n> shaper:bw_rlimit min_rate:1Gbit 2Gbit max_rate:4Gbit 5Gbit\n> \n> ---\n> \n> Amritha Nambiar (6):\n> mqprio: Introduce new hardware offload mode and shaper in mqprio\n> i40e: Add macro for PF reset bit\n> i40e: Add infrastructure for queue channel support\n> i40e: Enable 'channel' mode in mqprio for TC configs\n> i40e: Refactor VF BW rate limiting\n> i40e: Add support setting TC max bandwidth rates\n> \n\nIt would be nice to know what has changed since the last review, either \nsummarized here or in the individual patch files. This helps in knowing \nhow much attention should be given to this new set of patches, and \nencourages further review. I don't remember seeing any responses to my \nprevious comments, and it looks like not all of them were acted upon.\n\nsln\n\n> \n> drivers/net/ethernet/intel/i40e/i40e.h | 44 +\n> drivers/net/ethernet/intel/i40e/i40e_debugfs.c | 3\n> drivers/net/ethernet/intel/i40e/i40e_ethtool.c | 8\n> drivers/net/ethernet/intel/i40e/i40e_main.c | 1463 +++++++++++++++++---\n> drivers/net/ethernet/intel/i40e/i40e_txrx.h | 2\n> drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c | 50 -\n> include/net/pkt_cls.h | 9\n> include/uapi/linux/pkt_sched.h | 32\n> net/sched/sch_mqprio.c | 183 ++-\n> 9 files changed, 1551 insertions(+), 243 deletions(-)\n> \n> --\n> _______________________________________________\n> Intel-wired-lan mailing list\n> Intel-wired-lan@osuosl.org\n> https://lists.osuosl.org/mailman/listinfo/intel-wired-lan\n>","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xp5sT5Sjdz9s8J\n\tfor ;\n\tFri, 8 Sep 2017 02:45:01 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1753399AbdIGQo7 (ORCPT );\n\tThu, 7 Sep 2017 12:44:59 -0400","from aserp1040.oracle.com ([141.146.126.69]:29443 \"EHLO\n\taserp1040.oracle.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1751417AbdIGQo6 (ORCPT\n\t); Thu, 7 Sep 2017 12:44:58 -0400","from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233])\n\tby aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2)\n\twith ESMTP id v87GitaV010419\n\t(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256\n\tverify=OK); Thu, 7 Sep 2017 16:44:56 GMT","from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235])\n\tby aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id\n\tv87GitiU015299\n\t(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256\n\tverify=OK); Thu, 7 Sep 2017 16:44:55 GMT","from abhmp0001.oracle.com (abhmp0001.oracle.com [141.146.116.7])\n\tby aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id v87GitHM000391; \n\tThu, 7 Sep 2017 16:44:55 GMT","from [10.159.133.68] (/10.159.133.68)\n\tby default (Oracle Beehive Gateway v4.0)\n\twith ESMTP ; Thu, 07 Sep 2017 09:44:55 -0700"],"Subject":"Re: [Intel-wired-lan] [RFC PATCH v3 0/6] Configuring traffic classes\n\tvia new hardware offload mechanism in tc/mqprio","To":"Amritha Nambiar ,\n\tintel-wired-lan@lists.osuosl.org, jeffrey.t.kirsher@intel.com","Cc":"netdev@vger.kernel.org","References":"<150478158684.24662.17975701233699487888.stgit@anamdev.jf.intel.com>","From":"Shannon Nelson ","Organization":"Oracle Corporation","Message-ID":"","Date":"Thu, 7 Sep 2017 09:45:33 -0700","User-Agent":"Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<150478158684.24662.17975701233699487888.stgit@anamdev.jf.intel.com>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","X-Source-IP":"aserv0021.oracle.com [141.146.126.233]","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1764846,"web_url":"http://patchwork.ozlabs.org/comment/1764846/","msgid":"","list_archive_url":null,"date":"2017-09-07T17:22:50","subject":"Re: [Intel-wired-lan] [RFC PATCH v3 0/6] Configuring traffic classes\n\tvia new hardware offload mechanism in tc/mqprio","submitter":{"id":68504,"url":"http://patchwork.ozlabs.org/api/people/68504/","name":"Nambiar, Amritha","email":"amritha.nambiar@intel.com"},"content":"On 9/7/2017 9:45 AM, Shannon Nelson wrote:\n> On 9/7/2017 4:00 AM, Amritha Nambiar wrote:\n>> The following series introduces a new hardware offload mode in\n>> tc/mqprio where the TCs, the queue configurations and\n>> bandwidth rate limits are offloaded to the hardware. The existing\n>> mqprio framework is extended to configure the queue counts and\n>> layout and also added support for rate limiting. This is achieved\n>> through new netlink attributes for the 'mode' option which takes\n>> values such as 'dcb' (default) and 'channel' and a 'shaper' option\n>> for QoS attributes such as bandwidth rate limits in hw mode 1.\n>> Legacy devices can fall back to the existing setup supporting hw mode\n>> 1 without these additional options where only the TCs are offloaded\n>> and then the 'mode' and 'shaper' options defaults to DCB support.\n>> The i40e driver enables the new mqprio hardware offload mechanism\n>> factoring the TCs, queue configuration and bandwidth rates by\n>> creating HW channel VSIs.\n>>\n>> In this new mode, the priority to traffic class mapping and the\n>> user specified queue ranges are used to configure the traffic\n>> class when the 'mode' option is set to 'channel'. This is achieved by\n>> creating HW channels(VSI). A new channel is created for each of the\n>> traffic class configuration offloaded via mqprio framework except for\n>> the first TC (TC0) which is for the main VSI. TC0 for the main VSI is\n>> also reconfigured as per user provided queue parameters. Finally,\n>> bandwidth rate limits are set on these traffic classes through the\n>> shaper attribute by sending these rates in addition to the number of\n>> TCs and the queue configurations.\n>>\n>> Example:\n>> # tc qdisc add dev eth0 root mqprio num_tc 2 map 0 0 0 0 1 1 1 1\\\n>> queues 4@0 4@4 hw 1 mode channel shaper bw_rlimit\\\n>> min_rate 1Gbit 2Gbit max_rate 4Gbit 5Gbit\n>>\n>> To dump the bandwidth rates:\n>>\n>> # tc qdisc show dev eth0\n>>\n>> qdisc mqprio 804a: root tc 2 map 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0\n>> queues:(0:3) (4:7)\n>> mode:channel\n>> shaper:bw_rlimit min_rate:1Gbit 2Gbit max_rate:4Gbit 5Gbit\n>>\n>> ---\n>>\n>> Amritha Nambiar (6):\n>> mqprio: Introduce new hardware offload mode and shaper in mqprio\n>> i40e: Add macro for PF reset bit\n>> i40e: Add infrastructure for queue channel support\n>> i40e: Enable 'channel' mode in mqprio for TC configs\n>> i40e: Refactor VF BW rate limiting\n>> i40e: Add support setting TC max bandwidth rates\n>>\n> \n> It would be nice to know what has changed since the last review, either \n> summarized here or in the individual patch files. This helps in knowing \n> how much attention should be given to this new set of patches, and \n> encourages further review. I don't remember seeing any responses to my \n> previous comments, and it looks like not all of them were acted upon.\n\nFor all those patch files that have changed since the last revision, I\nhave captured all the new changes in the section titled \"v3: \" of each\npatch file. Also, I had replied to most of your comments that they'll be\nfixed in v3 and the one in the mqprio patch that error handling was\nalready being done and these responses can be verified in patchwork. I\nmissed to respond to your comment regarding supporting macvlan offloads\nthrough these channels, that will not be a part of this series and will\nbe looked upon at a later time in a subsequent patch series.\n\n> \n> sln\n> \n>>\n>> drivers/net/ethernet/intel/i40e/i40e.h | 44 +\n>> drivers/net/ethernet/intel/i40e/i40e_debugfs.c | 3\n>> drivers/net/ethernet/intel/i40e/i40e_ethtool.c | 8\n>> drivers/net/ethernet/intel/i40e/i40e_main.c | 1463 +++++++++++++++++---\n>> drivers/net/ethernet/intel/i40e/i40e_txrx.h | 2\n>> drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c | 50 -\n>> include/net/pkt_cls.h | 9\n>> include/uapi/linux/pkt_sched.h | 32\n>> net/sched/sch_mqprio.c | 183 ++-\n>> 9 files changed, 1551 insertions(+), 243 deletions(-)\n>>\n>> --\n>> _______________________________________________\n>> Intel-wired-lan mailing list\n>> Intel-wired-lan@osuosl.org\n>> https://lists.osuosl.org/mailman/listinfo/intel-wired-lan\n>>","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xp6ml2N41z9t2M\n\tfor ;\n\tFri, 8 Sep 2017 03:25:59 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1753853AbdIGRZz (ORCPT );\n\tThu, 7 Sep 2017 13:25:55 -0400","from mga09.intel.com ([134.134.136.24]:32956 \"EHLO mga09.intel.com\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1752784AbdIGRZy (ORCPT );\n\tThu, 7 Sep 2017 13:25:54 -0400","from fmsmga003.fm.intel.com ([10.253.24.29])\n\tby orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t07 Sep 2017 10:22:51 -0700","from anambiar-mobl.amr.corp.intel.com (HELO [10.254.117.80])\n\t([10.254.117.80])\n\tby FMSMGA003.fm.intel.com with ESMTP; 07 Sep 2017 10:22:50 -0700"],"X-ExtLoop1":"1","X-IronPort-AV":"E=Sophos;i=\"5.42,359,1500966000\"; d=\"scan'208\";a=\"898103650\"","Subject":"Re: [Intel-wired-lan] [RFC PATCH v3 0/6] Configuring traffic classes\n\tvia new hardware offload mechanism in tc/mqprio","To":"Shannon Nelson ,\n\tintel-wired-lan@lists.osuosl.org, jeffrey.t.kirsher@intel.com","References":"<150478158684.24662.17975701233699487888.stgit@anamdev.jf.intel.com>\n\t","Cc":"netdev@vger.kernel.org","From":"\"Nambiar, Amritha\" ","Message-ID":"","Date":"Thu, 7 Sep 2017 10:22:50 -0700","User-Agent":"Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101\n\tThunderbird/45.8.0","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"7bit","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1764851,"web_url":"http://patchwork.ozlabs.org/comment/1764851/","msgid":"","list_archive_url":null,"date":"2017-09-07T17:38:32","subject":"Re: [Intel-wired-lan] [RFC PATCH v3 0/6] Configuring traffic classes\n\tvia new hardware offload mechanism in tc/mqprio","submitter":{"id":70766,"url":"http://patchwork.ozlabs.org/api/people/70766/","name":"Shannon Nelson","email":"shannon.nelson@oracle.com"},"content":"On 9/7/2017 10:22 AM, Nambiar, Amritha wrote:\n> On 9/7/2017 9:45 AM, Shannon Nelson wrote:\n[...]\n>>\n>> It would be nice to know what has changed since the last review, either\n>> summarized here or in the individual patch files. \n[...]\n> \n> For all those patch files that have changed since the last revision, I\n> have captured all the new changes in the section titled \"v3: \" of each\n> patch file. \n\nMea culpa.\n\nugh.\nsln","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xp72v0bC1z9t2M\n\tfor ;\n\tFri, 8 Sep 2017 03:38:15 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1755741AbdIGRh6 (ORCPT );\n\tThu, 7 Sep 2017 13:37:58 -0400","from userp1040.oracle.com ([156.151.31.81]:35616 \"EHLO\n\tuserp1040.oracle.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1755728AbdIGRh5 (ORCPT\n\t); Thu, 7 Sep 2017 13:37:57 -0400","from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74])\n\tby userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with\n\tESMTP id v87HbsdX009428\n\t(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256\n\tverify=OK); Thu, 7 Sep 2017 17:37:55 GMT","from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236])\n\tby userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id\n\tv87HbsVa010233\n\t(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256\n\tverify=OK); Thu, 7 Sep 2017 17:37:54 GMT","from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11])\n\tby aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id\n\tv87HbrVj005860; Thu, 7 Sep 2017 17:37:53 GMT","from [10.159.133.68] (/10.159.133.68)\n\tby default (Oracle Beehive Gateway v4.0)\n\twith ESMTP ; Thu, 07 Sep 2017 10:37:53 -0700"],"Subject":"Re: [Intel-wired-lan] [RFC PATCH v3 0/6] Configuring traffic classes\n\tvia new hardware offload mechanism in tc/mqprio","To":"\"Nambiar, Amritha\" ,\n\tintel-wired-lan@lists.osuosl.org, jeffrey.t.kirsher@intel.com","Cc":"netdev@vger.kernel.org","References":"<150478158684.24662.17975701233699487888.stgit@anamdev.jf.intel.com>\n\t\n\t","From":"Shannon Nelson ","Organization":"Oracle Corporation","Message-ID":"","Date":"Thu, 7 Sep 2017 10:38:32 -0700","User-Agent":"Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","X-Source-IP":"userv0022.oracle.com [156.151.31.74]","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1764883,"web_url":"http://patchwork.ozlabs.org/comment/1764883/","msgid":"<3ecacf3d-1f02-fbc6-0e90-8e84dcd15a4e@gmail.com>","list_archive_url":null,"date":"2017-09-07T18:34:45","subject":"Re: [RFC PATCH v3 0/6] Configuring traffic classes via new hardware\n\toffload mechanism in tc/mqprio","submitter":{"id":2800,"url":"http://patchwork.ozlabs.org/api/people/2800/","name":"Florian Fainelli","email":"f.fainelli@gmail.com"},"content":"On 09/07/2017 04:00 AM, Amritha Nambiar wrote:\n> The following series introduces a new hardware offload mode in\n> tc/mqprio where the TCs, the queue configurations and\n> bandwidth rate limits are offloaded to the hardware. The existing\n> mqprio framework is extended to configure the queue counts and\n> layout and also added support for rate limiting. This is achieved\n> through new netlink attributes for the 'mode' option which takes\n> values such as 'dcb' (default) and 'channel' and a 'shaper' option\n> for QoS attributes such as bandwidth rate limits in hw mode 1.\n\nSo \"dcb\" defines a default priorities to queue mapping?\n\n> Legacy devices can fall back to the existing setup supporting hw mode\n> 1 without these additional options where only the TCs are offloaded\n> and then the 'mode' and 'shaper' options defaults to DCB support.\n\nThat's the last part that confuses me, see below.\n\n> The i40e driver enables the new mqprio hardware offload mechanism\n> factoring the TCs, queue configuration and bandwidth rates by\n> creating HW channel VSIs.\n\nI am really confused by what you call hw_mode 1, as I understand it\nthere are really 3 different modes:\n\n- legacy: you don't define any traffic class mapping, but you can still\nchain this scheduler with a match + action (like what\nDocumentation/networking/multiqueue.txt) you can optionally also add\n\"shaper\" arguments, but there should not be any default DCB queue\nmapping either?\n\n- dcb: a default mapping for traffic classes to queues is defined,\noptional \"shaper\" arguments\n\n- channel: (maybe calling that \"custom_tc_map\" would be clearer?) where\nyou express the exact traffic classes to queue mapping and optional\n\"shaper\" arguments\n\nI think that's what you are doing, but I just got confused by the cover\nletter.\n\n> \n> In this new mode, the priority to traffic class mapping and the\n> user specified queue ranges are used to configure the traffic\n> class when the 'mode' option is set to 'channel'. This is achieved by\n> creating HW channels(VSI). A new channel is created for each of the\n> traffic class configuration offloaded via mqprio framework except for\n> the first TC (TC0) which is for the main VSI. TC0 for the main VSI is\n> also reconfigured as per user provided queue parameters. Finally,\n> bandwidth rate limits are set on these traffic classes through the\n> shaper attribute by sending these rates in addition to the number of\n> TCs and the queue configurations.\n> \n> Example:\n> # tc qdisc add dev eth0 root mqprio num_tc 2 map 0 0 0 0 1 1 1 1\\\n> queues 4@0 4@4 hw 1 mode channel shaper bw_rlimit\\\n\nDo you see a case where you can declare a different number of traffic\nclasses say 4 and map them onto just 2 hardware queues? If not, it seems\na tiny bit redundant to have to specify both the map and the queue\nmapping should be sufficient, right?\n\n> min_rate 1Gbit 2Gbit max_rate 4Gbit 5Gbit\n> \n> To dump the bandwidth rates:\n> \n> # tc qdisc show dev eth0\n> \n> qdisc mqprio 804a: root tc 2 map 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0\n> queues:(0:3) (4:7)\n> mode:channel\n> shaper:bw_rlimit min_rate:1Gbit 2Gbit max_rate:4Gbit 5Gbit\n\nI am not well versed into tc, but being able to specify \"shaper\"\narguments has actually value outside of just the multiq scheduler and it\ncould probably be an action on its own?\n\n> \n> ---\n> \n> Amritha Nambiar (6):\n> mqprio: Introduce new hardware offload mode and shaper in mqprio\n> i40e: Add macro for PF reset bit\n> i40e: Add infrastructure for queue channel support\n> i40e: Enable 'channel' mode in mqprio for TC configs\n> i40e: Refactor VF BW rate limiting\n> i40e: Add support setting TC max bandwidth rates\n> \n> \n> drivers/net/ethernet/intel/i40e/i40e.h | 44 +\n> drivers/net/ethernet/intel/i40e/i40e_debugfs.c | 3 \n> drivers/net/ethernet/intel/i40e/i40e_ethtool.c | 8 \n> drivers/net/ethernet/intel/i40e/i40e_main.c | 1463 +++++++++++++++++---\n> drivers/net/ethernet/intel/i40e/i40e_txrx.h | 2 \n> drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c | 50 -\n> include/net/pkt_cls.h | 9 \n> include/uapi/linux/pkt_sched.h | 32 \n> net/sched/sch_mqprio.c | 183 ++-\n> 9 files changed, 1551 insertions(+), 243 deletions(-)\n> \n> --\n>","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"VuKCJwKm\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xp8JP2bNmz9s7C\n\tfor ;\n\tFri, 8 Sep 2017 04:35:01 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1753640AbdIGSey (ORCPT );\n\tThu, 7 Sep 2017 14:34:54 -0400","from mail-wr0-f182.google.com ([209.85.128.182]:33098 \"EHLO\n\tmail-wr0-f182.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1751424AbdIGSex (ORCPT\n\t); Thu, 7 Sep 2017 14:34:53 -0400","by mail-wr0-f182.google.com with SMTP id a43so973728wrc.0\n\tfor ; Thu, 07 Sep 2017 11:34:52 -0700 (PDT)","from [10.112.156.244] ([192.19.255.250])\n\tby smtp.googlemail.com with ESMTPSA id\n\td5sm321207wma.22.2017.09.07.11.34.47\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tThu, 07 Sep 2017 11:34:50 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=gmail.com; s=20161025;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=WjYUEI8TLi+rLzj2W01gp3EvX9lMzL1WjpJYQisleTw=;\n\tb=VuKCJwKmlCOPN7JF/zVRZNQ1PKlmNu2agcuo59v/BpwGPoKL5n+bIoyY6bJHNLyp9w\n\tzq/klSzfJjVQ/2rlWF54AeIxvav2ZWcUieHzo3qBUc6zIzWEb79VICrxYHxgLD2m64xC\n\t7jnrmvawroiyslP32YuwUkgT11re96zzh3CIiendOKgOyO3bmi7FcxDVInI69R9RURLg\n\twhwanTk4U0SnmrEQ5YXXhD3mGtJ834pJ/FDrG+p2M1s5x8Y5JpJdyH2UdUc5Un+/3jIh\n\te99H2lhdHjozcbWwGmL9SbGND3Sxz6/PUfKbsbC/I73tbM5BlwuNBUh1WNO8T1QAHQhv\n\thXRg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=WjYUEI8TLi+rLzj2W01gp3EvX9lMzL1WjpJYQisleTw=;\n\tb=CaIWbLjlE2qiE5gQX+AHhLqEeFrWbj4Xtc6slVRvpbh6KiKd0+ufmlJP8nSBVIOc1g\n\tPXs873zihqGy+zGXFdx6tAOxULmkREFq5NW7k1bbB2o1nmqGQalyjTqbySHjTZVfYWxo\n\tnA6IBR2LrB8HyPBvi5IHAT3+Oxpzw4CE5IlV5qvJTXe7ytHWqX3YNu/lGAp3/M18a4Td\n\tdWt6BU6blWZnv3yqkfqC2FI0326K1i1wFKOSMUMzLwNiK6Vk7ZFVC/whZQBC5ByLor9p\n\t7gxPTeoqiEi1+O/if3xahGNc+q2VrfZ3bxH4b93ufDKJmgQivib27OmiCVwVmnZ6U29k\n\tHuHA==","X-Gm-Message-State":"AHPjjUirCzhPkGHGXk6y6YBobW429duOImbA67MiyB3prBgYZ5wab3EP\n\tYGiiSPxvo93a10/gHGY=","X-Google-Smtp-Source":"ADKCNb7NOSfcuD7lc4dqy4X77ULNYnPpOCRjXGfSA9JY1LeKVTsFzRB9o5alZsk67t/VxnY0CQc10A==","X-Received":"by 10.223.173.242 with SMTP id w105mr174258wrc.111.1504809291756;\n\tThu, 07 Sep 2017 11:34:51 -0700 (PDT)","Subject":"Re: [RFC PATCH v3 0/6] Configuring traffic classes via new hardware\n\toffload mechanism in tc/mqprio","To":"Amritha Nambiar ,\n\tintel-wired-lan@lists.osuosl.org, jeffrey.t.kirsher@intel.com","Cc":"alexander.h.duyck@intel.com, netdev@vger.kernel.org","References":"<150478158684.24662.17975701233699487888.stgit@anamdev.jf.intel.com>","From":"Florian Fainelli ","Message-ID":"<3ecacf3d-1f02-fbc6-0e90-8e84dcd15a4e@gmail.com>","Date":"Thu, 7 Sep 2017 11:34:45 -0700","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<150478158684.24662.17975701233699487888.stgit@anamdev.jf.intel.com>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-US","Content-Transfer-Encoding":"8bit","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1764928,"web_url":"http://patchwork.ozlabs.org/comment/1764928/","msgid":"<14376038-c365-9637-3517-aae8e99a4883@intel.com>","list_archive_url":null,"date":"2017-09-07T20:09:27","subject":"Re: [RFC PATCH v3 0/6] Configuring traffic classes via new hardware\n\toffload mechanism in tc/mqprio","submitter":{"id":68504,"url":"http://patchwork.ozlabs.org/api/people/68504/","name":"Nambiar, Amritha","email":"amritha.nambiar@intel.com"},"content":"On 9/7/2017 11:34 AM, Florian Fainelli wrote:\n> On 09/07/2017 04:00 AM, Amritha Nambiar wrote:\n>> The following series introduces a new hardware offload mode in \n>> tc/mqprio where the TCs, the queue configurations and bandwidth \n>> rate limits are offloaded to the hardware. The existing mqprio \n>> framework is extended to configure the queue counts and layout and\n>> also added support for rate limiting. This is achieved through new\n>> netlink attributes for the 'mode' option which takes values such\n>> as 'dcb' (default) and 'channel' and a 'shaper' option for QoS \n>> attributes such as bandwidth rate limits in hw mode 1.\n> \n> So \"dcb\" defines a default priorities to queue mapping?\n\nIn the default offload implementation, only the basic hw offload was\nsupported factoring only the 'number of TCs' values. The rest of the\nqueue configuration was being ignored. This is the legacy behavior with\nhw mode set to 1.\nExample:\n# tc qdisc add dev eth0 root mqprio num_tc 4 map 0 0 0 0 1 1 1 1 queues\n4@0 4@4 hw 1\nI just named this default behavior to 'dcb' while introducing a new\noffload mechanism.\n\n> \n>> Legacy devices can fall back to the existing setup supporting hw \n>> mode 1 without these additional options where only the TCs are \n>> offloaded and then the 'mode' and 'shaper' options defaults to DCB\n>> support.\n> \n> That's the last part that confuses me, see below.\n\nAs I introduced new options for 'mode' and 'shaper', I set the defaults\nfor these options to 'dcb' so existing offloaders can continue to work\nwithout supporting these new options. Patch 1 has a detailed description\non how this is done.\n\n> \n>> The i40e driver enables the new mqprio hardware offload mechanism \n>> factoring the TCs, queue configuration and bandwidth rates by \n>> creating HW channel VSIs.\n> \n> I am really confused by what you call hw_mode 1, as I understand it \n> there are really 3 different modes:\n\nThere are actually 2 modes now with 'hw' option set to 1, legacy/dcb and\nchannel.\n> \n> - legacy: you don't define any traffic class mapping, but you can \n> still chain this scheduler with a match + action (like what \n> Documentation/networking/multiqueue.txt) you can optionally also add \n> \"shaper\" arguments, but there should not be any default DCB queue \n> mapping either?\n> \n> - dcb: a default mapping for traffic classes to queues is defined, \n> optional \"shaper\" arguments\n\nThe legacy mode now becomes the dcb mode. In this mode, although the TC\nvalues, the queue configurations, prio-tc-mapping are all offloaded to\nthe device, the existing implementation in current drivers support only\na basic hw offload factoring only the TC values.\n\nExamples:\n# ... num_tc 2 map 0 0 0 0 1 1 1 1 queues 4@0 4@4 hw 1\n# ... num_tc 2 map 0 0 0 0 1 1 1 1 queues 4@0 4@4 hw 1 mode dcb\n# ... num_tc 2 map 0 0 0 0 1 1 1 1 queues 4@0 4@4 hw 1 mode dcb\\\n shaper dcb\n\n> \n> - channel: (maybe calling that \"custom_tc_map\" would be clearer?) \n> where you express the exact traffic classes to queue mapping and \n> optional \"shaper\" arguments\n\nIn the channel mode, a full hw offload is supported, the TC values, the\nqueue configurations and additionally QoS attributes (optional) are all\nused in the new implementation.\n\nExamples:\n# ... num_tc 2 map 0 0 0 0 1 1 1 1 queues 4@0 4@4 hw 1 mode channel\n# ... num_tc 2 map 0 0 0 0 1 1 1 1 queues 4@0 4@4 hw 1 mode channel\\\nshaper bw_rlimit max_rate 4Gbit 5Gbit\n\n> \n> I think that's what you are doing, but I just got confused by the \n> cover letter.\n> \n>> \n>> In this new mode, the priority to traffic class mapping and the \n>> user specified queue ranges are used to configure the traffic class\n>> when the 'mode' option is set to 'channel'. This is achieved by\n>> creating HW channels(VSI). A new channel is created for each of the\n>> traffic class configuration offloaded via mqprio framework except\n>> for the first TC (TC0) which is for the main VSI. TC0 for the main\n>> VSI is also reconfigured as per user provided queue parameters.\n>> Finally, bandwidth rate limits are set on these traffic classes\n>> through the shaper attribute by sending these rates in addition to\n>> the number of TCs and the queue configurations.\n>> \n>> Example: # tc qdisc add dev eth0 root mqprio num_tc 2 map 0 0 0 0 1\n>> 1 1 1\\ queues 4@0 4@4 hw 1 mode channel shaper bw_rlimit\\\n> \n> Do you see a case where you can declare a different number of\n> traffic classes say 4 and map them onto just 2 hardware queues? If\n> not, it seems a tiny bit redundant to have to specify both the map\n> and the queue mapping should be sufficient, right?\n\nThis will be subjected to validation of the user input and will be\ntreated as invalid configuration. The 'map' specifies the mapping\nbetween user priorities and traffic classes while the queue mapping is\nthe queue layout specifying the queue count and offsets.\n\n> \n>> min_rate 1Gbit 2Gbit max_rate 4Gbit 5Gbit\n>> \n>> To dump the bandwidth rates:\n>> \n>> # tc qdisc show dev eth0\n>> \n>> qdisc mqprio 804a: root tc 2 map 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 \n>> queues:(0:3) (4:7) mode:channel shaper:bw_rlimit min_rate:1Gbit \n>> 2Gbit max_rate:4Gbit 5Gbit\n> \n> I am not well versed into tc, but being able to specify \"shaper\" \n> arguments has actually value outside of just the multiq scheduler and\n> it could probably be an action on its own?\n\nThe mqprio scheduler already supports configuring the traffic classes\nand enabling support for HW shapers was just a matter to extending it\nwith new netlink based attributes.\n\n> \n>> \n>> ---\n>> \n>> Amritha Nambiar (6): mqprio: Introduce new hardware offload mode \n>> and shaper in mqprio i40e: Add macro for PF reset bit i40e: Add \n>> infrastructure for queue channel support i40e: Enable 'channel' \n>> mode in mqprio for TC configs i40e: Refactor VF BW rate limiting \n>> i40e: Add support setting TC max bandwidth rates\n>> \n>> \n>> drivers/net/ethernet/intel/i40e/i40e.h | 44 + \n>> drivers/net/ethernet/intel/i40e/i40e_debugfs.c | 3 \n>> drivers/net/ethernet/intel/i40e/i40e_ethtool.c | 8 \n>> drivers/net/ethernet/intel/i40e/i40e_main.c | 1463 \n>> +++++++++++++++++--- drivers/net/ethernet/intel/i40e/i40e_txrx.h | \n>> 2 drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c | 50 - \n>> include/net/pkt_cls.h | 9 \n>> include/uapi/linux/pkt_sched.h | 32 \n>> net/sched/sch_mqprio.c | 183 ++- 9 \n>> files changed, 1551 insertions(+), 243 deletions(-)\n>> \n>> --\n>> \n> \n>","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xpBPl4Jgsz9s83\n\tfor ;\n\tFri, 8 Sep 2017 06:09:47 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S932689AbdIGUJp (ORCPT );\n\tThu, 7 Sep 2017 16:09:45 -0400","from mga14.intel.com ([192.55.52.115]:51808 \"EHLO mga14.intel.com\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S932378AbdIGUJo (ORCPT );\n\tThu, 7 Sep 2017 16:09:44 -0400","from orsmga003.jf.intel.com ([10.7.209.27])\n\tby fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t07 Sep 2017 13:09:30 -0700","from anambiar-mobl.amr.corp.intel.com (HELO [10.254.117.80])\n\t([10.254.117.80])\n\tby orsmga003.jf.intel.com with ESMTP; 07 Sep 2017 13:09:27 -0700"],"X-ExtLoop1":"1","X-IronPort-AV":"E=Sophos;i=\"5.42,360,1500966000\"; d=\"scan'208\";a=\"1012139241\"","Subject":"Re: [RFC PATCH v3 0/6] Configuring traffic classes via new hardware\n\toffload mechanism in tc/mqprio","To":"Florian Fainelli ,\n\tintel-wired-lan@lists.osuosl.org, jeffrey.t.kirsher@intel.com","References":"<150478158684.24662.17975701233699487888.stgit@anamdev.jf.intel.com>\n\t<3ecacf3d-1f02-fbc6-0e90-8e84dcd15a4e@gmail.com>","Cc":"alexander.h.duyck@intel.com, netdev@vger.kernel.org","From":"\"Nambiar, Amritha\" ","Message-ID":"<14376038-c365-9637-3517-aae8e99a4883@intel.com>","Date":"Thu, 7 Sep 2017 13:09:27 -0700","User-Agent":"Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101\n\tThunderbird/45.8.0","MIME-Version":"1.0","In-Reply-To":"<3ecacf3d-1f02-fbc6-0e90-8e84dcd15a4e@gmail.com>","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"7bit","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}}]