From patchwork Thu Sep 7 11:00:32 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nambiar, Amritha" X-Patchwork-Id: 810972 X-Patchwork-Delegate: jeffrey.t.kirsher@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=osuosl.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3xnyFd0w0mz9s7C for ; Thu, 7 Sep 2017 21:01:57 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id ACDF389A61; Thu, 7 Sep 2017 11:01:55 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AvTfbIo8iozd; Thu, 7 Sep 2017 11:01:54 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id D41CF89BEA; Thu, 7 Sep 2017 11:01:54 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id C2CAB1C2121 for ; Thu, 7 Sep 2017 11:01:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id BB3E889612 for ; Thu, 7 Sep 2017 11:01:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x9t4N7zBQCUZ for ; Thu, 7 Sep 2017 11:01:51 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by hemlock.osuosl.org (Postfix) with ESMTPS id 9455589514 for ; Thu, 7 Sep 2017 11:01:51 +0000 (UTC) Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 Sep 2017 04:01:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.42,357,1500966000"; d="scan'208";a="126453236" Received: from anamdev.jf.intel.com ([10.166.29.110]) by orsmga004.jf.intel.com with ESMTP; 07 Sep 2017 04:01:37 -0700 From: Amritha Nambiar To: intel-wired-lan@lists.osuosl.org, jeffrey.t.kirsher@intel.com Date: Thu, 07 Sep 2017 04:00:32 -0700 Message-ID: <150478203291.24662.3361290802097939640.stgit@anamdev.jf.intel.com> In-Reply-To: <150478158684.24662.17975701233699487888.stgit@anamdev.jf.intel.com> References: <150478158684.24662.17975701233699487888.stgit@anamdev.jf.intel.com> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Cc: netdev@vger.kernel.org Subject: [Intel-wired-lan] [RFC PATCH v3 6/6] i40e: Add support setting TC max bandwidth rates X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" This patch enables setting up maximum Tx rates for the traffic classes in i40e. The maximum rate is offloaded to the hardware through the mqprio framework by specifying the mode option as 'channel' and shaper option as 'bw_rlimit' and is configured for the VSI. Configuring minimum Tx rate limit is not supported in the device. The minimum usable value for Tx rate is 50Mbps. Example: # tc qdisc add dev eth0 root mqprio num_tc 2 map 0 0 0 0 1 1 1 1\ queues 4@0 4@4 hw 1 mode channel shaper bw_rlimit\ max_rate 4Gbit 5Gbit To dump the bandwidth rates: # tc qdisc show dev eth0 qdisc mqprio 804a: root tc 2 map 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 queues:(0:3) (4:7) mode:channel shaper:bw_rlimit max_rate:4Gbit 5Gbit Signed-off-by: Amritha Nambiar Tested-by: Andrew Bowers --- drivers/net/ethernet/intel/i40e/i40e.h | 2 + drivers/net/ethernet/intel/i40e/i40e_main.c | 101 +++++++++++++++++++++++++-- 2 files changed, 94 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/intel/i40e/i40e.h b/drivers/net/ethernet/intel/i40e/i40e.h index 2ee0197..02cae34 100644 --- a/drivers/net/ethernet/intel/i40e/i40e.h +++ b/drivers/net/ethernet/intel/i40e/i40e.h @@ -359,6 +359,8 @@ struct i40e_channel { u8 enabled_tc; struct i40e_aqc_vsi_properties_data info; + u64 max_tx_rate; + /* track this channel belongs to which VSI */ struct i40e_vsi *parent_vsi; }; diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c index 7f13bc7..14a177b 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_main.c +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c @@ -5202,9 +5202,16 @@ static int i40e_vsi_configure_bw_alloc(struct i40e_vsi *vsi, u8 enabled_tc, i40e_status ret; int i; - if ((vsi->back->flags & I40E_FLAG_TC_MQPRIO) || - !vsi->mqprio_qopt.qopt.hw) + if (vsi->back->flags & I40E_FLAG_TC_MQPRIO) return 0; + if (!vsi->mqprio_qopt.qopt.hw) { + ret = i40e_set_bw_limit(vsi, vsi->seid, 0); + if (ret) + dev_info(&vsi->back->pdev->dev, + "Failed to reset tx rate for vsi->seid %u\n", + vsi->seid); + return ret; + } bw_data.tc_valid_bits = enabled_tc; for (i = 0; i < I40E_MAX_TRAFFIC_CLASS; i++) bw_data.tc_bw_credits[i] = bw_share[i]; @@ -5511,6 +5518,13 @@ static void i40e_remove_queue_channels(struct i40e_vsi *vsi) rx_ring->ch = NULL; } + /* Reset BW configured for this VSI via mqprio */ + ret = i40e_set_bw_limit(vsi, ch->seid, 0); + if (ret) + dev_info(&vsi->back->pdev->dev, + "Failed to reset tx rate for ch->seid %u\n", + ch->seid); + /* delete VSI from FW */ ret = i40e_aq_delete_element(&vsi->back->hw, ch->seid, NULL); @@ -6053,6 +6067,17 @@ int i40e_create_queue_channel(struct i40e_vsi *vsi, "Setup channel (id:%u) utilizing num_queues %d\n", ch->seid, ch->num_queue_pairs); + /* configure VSI for BW limit */ + if (ch->max_tx_rate) { + if (i40e_set_bw_limit(vsi, ch->seid, ch->max_tx_rate)) + return -EINVAL; + + dev_dbg(&pf->pdev->dev, + "Set tx rate of %llu Mbps (count of 50Mbps %llu) for vsi->seid %u\n", + ch->max_tx_rate, + ch->max_tx_rate / I40E_BW_CREDIT_DIVISOR, ch->seid); + } + /* in case of VF, this will be main SRIOV VSI */ ch->parent_vsi = vsi; @@ -6088,6 +6113,12 @@ static int i40e_configure_queue_channels(struct i40e_vsi *vsi) ch->base_queue = vsi->tc_config.tc_info[i].qoffset; + /* Bandwidth limit through tc interface is in bytes/s, + * change to Mbit/s + */ + ch->max_tx_rate = + vsi->mqprio_qopt.max_rate[i] / (1000000 / 8); + list_add_tail(&ch->list, &vsi->ch_list); ret = i40e_create_queue_channel(vsi, ch); @@ -6514,6 +6545,7 @@ void i40e_down(struct i40e_vsi *vsi) static int i40e_validate_mqprio_qopt(struct i40e_vsi *vsi, struct tc_mqprio_qopt_offload *mqprio_qopt) { + u64 sum_max_rate = 0; int i; if ((mqprio_qopt->qopt.offset[0] != 0) || @@ -6523,8 +6555,13 @@ static int i40e_validate_mqprio_qopt(struct i40e_vsi *vsi, for (i = 0; ; i++) { if (!mqprio_qopt->qopt.count[i]) return -EINVAL; - if (mqprio_qopt->min_rate[i] || mqprio_qopt->max_rate[i]) + if (mqprio_qopt->min_rate[i]) { + dev_err(&vsi->back->pdev->dev, + "Invalid min tx rate (greater than 0) specified\n"); return -EINVAL; + } + sum_max_rate += (mqprio_qopt->max_rate[i] / (1000000 / 8)); + if (i >= mqprio_qopt->qopt.num_tc - 1) break; if (mqprio_qopt->qopt.offset[i + 1] != @@ -6535,6 +6572,11 @@ static int i40e_validate_mqprio_qopt(struct i40e_vsi *vsi, (mqprio_qopt->qopt.offset[i] + mqprio_qopt->qopt.count[i])) { return -EINVAL; } + if (sum_max_rate > i40e_get_link_speed(vsi)) { + dev_err(&vsi->back->pdev->dev, + "Invalid max tx rate specified\n"); + return -EINVAL; + } return 0; } @@ -6661,6 +6703,21 @@ static int i40e_setup_tc(struct net_device *netdev, void *type_data) } if (pf->flags & I40E_FLAG_TC_MQPRIO) { + if (vsi->mqprio_qopt.max_rate[0]) { + u64 max_tx_rate = vsi->mqprio_qopt.max_rate[0] / + (1000000 / 8); + ret = i40e_set_bw_limit(vsi, vsi->seid, max_tx_rate); + if (!ret) { + dev_dbg(&vsi->back->pdev->dev, + "Set tx rate of %llu Mbps (count of 50Mbps %llu) for vsi->seid %u\n", + max_tx_rate, + max_tx_rate / I40E_BW_CREDIT_DIVISOR, + vsi->seid); + } else { + need_reset = true; + goto exit; + } + } ret = i40e_configure_queue_channels(vsi); if (ret) { netdev_info(netdev, @@ -8114,6 +8171,18 @@ static int i40e_rebuild_channels(struct i40e_vsi *vsi) vsi->uplink_seid); return ret; } + if (ch->max_tx_rate) { + if (i40e_set_bw_limit(vsi, ch->seid, + ch->max_tx_rate)) + return -EINVAL; + + dev_dbg(&vsi->back->pdev->dev, + "Set tx rate of %llu Mbps (count of 50Mbps %llu) for vsi->seid %u\n", + ch->max_tx_rate, + ch->max_tx_rate / + I40E_BW_CREDIT_DIVISOR, + ch->seid); + } } return 0; } @@ -8254,6 +8323,7 @@ static int i40e_reset(struct i40e_pf *pf) **/ static void i40e_rebuild(struct i40e_pf *pf, bool reinit, bool lock_acquired) { + struct i40e_vsi *vsi = pf->vsi[pf->lan_vsi]; struct i40e_hw *hw = &pf->hw; u8 set_fc_aq_fail = 0; i40e_status ret; @@ -8336,7 +8406,7 @@ static void i40e_rebuild(struct i40e_pf *pf, bool reinit, bool lock_acquired) * If there were VEBs but the reconstitution failed, we'll try * try to recover minimal use by getting the basic PF VSI working. */ - if (pf->vsi[pf->lan_vsi]->uplink_seid != pf->mac_seid) { + if (vsi->uplink_seid != pf->mac_seid) { dev_dbg(&pf->pdev->dev, "attempting to rebuild switch\n"); /* find the one VEB connected to the MAC, and find orphans */ for (v = 0; v < I40E_MAX_VEB; v++) { @@ -8360,8 +8430,7 @@ static void i40e_rebuild(struct i40e_pf *pf, bool reinit, bool lock_acquired) dev_info(&pf->pdev->dev, "rebuild of switch failed: %d, will try to set up simple PF connection\n", ret); - pf->vsi[pf->lan_vsi]->uplink_seid - = pf->mac_seid; + vsi->uplink_seid = pf->mac_seid; break; } else if (pf->veb[v]->uplink_seid == 0) { dev_info(&pf->pdev->dev, @@ -8372,10 +8441,10 @@ static void i40e_rebuild(struct i40e_pf *pf, bool reinit, bool lock_acquired) } } - if (pf->vsi[pf->lan_vsi]->uplink_seid == pf->mac_seid) { + if (vsi->uplink_seid == pf->mac_seid) { dev_dbg(&pf->pdev->dev, "attempting to rebuild PF VSI\n"); /* no VEB, so rebuild only the Main VSI */ - ret = i40e_add_vsi(pf->vsi[pf->lan_vsi]); + ret = i40e_add_vsi(vsi); if (ret) { dev_info(&pf->pdev->dev, "rebuild of Main VSI failed: %d\n", ret); @@ -8383,10 +8452,24 @@ static void i40e_rebuild(struct i40e_pf *pf, bool reinit, bool lock_acquired) } } + if (vsi->mqprio_qopt.max_rate[0]) { + u64 max_tx_rate = vsi->mqprio_qopt.max_rate[0] / (1000000 / 8); + + ret = i40e_set_bw_limit(vsi, vsi->seid, max_tx_rate); + if (!ret) + dev_dbg(&vsi->back->pdev->dev, + "Set tx rate of %llu Mbps (count of 50Mbps %llu) for vsi->seid %u\n", + max_tx_rate, + max_tx_rate / I40E_BW_CREDIT_DIVISOR, + vsi->seid); + else + goto end_unlock; + } + /* PF Main VSI is rebuild by now, go ahead and rebuild channel VSIs * for this main VSI if they exist */ - ret = i40e_rebuild_channels(pf->vsi[pf->lan_vsi]); + ret = i40e_rebuild_channels(vsi); if (ret) goto end_unlock;