From patchwork Sat Apr 28 06:32:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Manlunas, Felix" X-Patchwork-Id: 906052 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=cavium.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=CAVIUMNETWORKS.onmicrosoft.com header.i=@CAVIUMNETWORKS.onmicrosoft.com header.b="Fmnnep1w"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40Y1G41Rxgz9s06 for ; Sat, 28 Apr 2018 16:33:16 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933442AbeD1GdN (ORCPT ); Sat, 28 Apr 2018 02:33:13 -0400 Received: from mail-by2nam01on0086.outbound.protection.outlook.com ([104.47.34.86]:18048 "EHLO NAM01-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S932841AbeD1GdE (ORCPT ); Sat, 28 Apr 2018 02:33:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=r/s0fsSkQYqyFTRLUOwXdPA6f3VDm7tS86Kz/fSM+YU=; b=Fmnnep1wA4dv/7+BED9+GlrGEe8iMBi8F2RxegA93cKayVQWvDAOal8s+5K9EV+EQaDX5TgZZbzSriPcsav7U20F4WBFr/oR8hxdhAh+BXx3EWeO/ZkQrgHMHi+n3ntMcu7ISVFf2QeTzaLo5sCgiC/rnxh/6+imEVj8XbfpGIg= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Felix.Manlunas@cavium.com; Received: from localhost (50.233.148.156) by BN7PR07MB4211.namprd07.prod.outlook.com (2603:10b6:406:b2::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.715.18; Sat, 28 Apr 2018 06:33:01 +0000 Date: Fri, 27 Apr 2018 23:32:57 -0700 From: Felix Manlunas To: davem@davemloft.net Cc: netdev@vger.kernel.org, raghu.vatsavayi@cavium.com, derek.chickles@cavium.com, satananda.burla@cavium.com, felix.manlunas@cavium.com, intiyaz.basha@cavium.com Subject: [PATCH V2 net-next 6/6] liquidio: enhanced ethtool --set-channels feature Message-ID: <20180428063257.GA3302@felix-thinkpad.cavium.com> References: <20180428063204.GA3229@felix-thinkpad.cavium.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20180428063204.GA3229@felix-thinkpad.cavium.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Originating-IP: [50.233.148.156] X-ClientProxiedBy: CO1PR15CA0110.namprd15.prod.outlook.com (2603:10b6:101:21::30) To BN7PR07MB4211.namprd07.prod.outlook.com (2603:10b6:406:b2::15) X-MS-PublicTrafficType: Email X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(4534165)(4627221)(201703031133081)(201702281549075)(5600026)(2017052603328)(7153060)(7193020); SRVR:BN7PR07MB4211; X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4211; 3:Jk97qvq50F7/FqZLKoVBGoWlSfRa1NqjQci4CEPHCLOxBZ14XonWG3usu6V+X/WLOWGxP2HqXXs7ilF7hCo8Vn5HjXNreLWX5BaxeseQgOWs/DU8bd8d3g0iMTmQ+oHkBUJK+3jRhKuU6GWDfsWeNgUeaQw5u0Rvo36Tl8okMOzfz1Y692OHz0s4pei7YY3HsLTrE6QlVPBUcrORSSQKUXONh7sBm9TzWOoysn+o0Z4f4Pwo/nJFaUbR4DriOuCW; 25:CUz/WbMd5TzG5c+dG4o/toIa7g+x+zQ3ULUaDT6wPn7/Kl1a+PIvV8lkOiU14T4heV19+ao2SvkJL7ZUyLrQ/nCMGW60G0ctH8Pf6JGdm2LqnIHD9oW+7t5YeOPD1SEBWUJ0evlh4dVuKaH76lSU5alXKL6CM6/3I6+Ry6ISepTsU+qpmWXZLPe5WtNbsglIOOjp+bkmSnrnAaqa0B1y/5pnnbCYQ/NJB8rB9GiXEQPyNDbBmgOdk5uzJrDxOlZ/5HX+d2jExonHCGmRI82uI80mh6XZ47yMH/1hrvDUpXjRp6v2ynmpf3YTdGOGkSL0VmqwYhTVCr4vZGZKd73a3w==; 31:nsQ1Y014sJnrax1gAP2d+o+JkOaRPnBq28e+7HAUlWKbVT/xy7LD4Ty1Us7/XyPK2PNVufrRcIZ9/VJ5N+AAZGLGlpsuFJwC/z6FWoyzJlvKeRYshI8EpRwrCJtd18sboabI8gvbk/e7MS5nxKFWnAoynwtB/K9t5YQrJXfyK0SMt78FyGf7slaycRsKV+W45Ifivugrb9u9C054w54DJUHSvg/t6AicMO9zSo4Z3PM= X-MS-TrafficTypeDiagnostic: BN7PR07MB4211: X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4211; 20:jTn7k8co/9zGS6cBJTUpe3QCaWqv7mNT7H9hfDNRXUCepJWFN/rneSbGdUhJ1JiGEDfpJXx3dyS2k1VgT0lh0iZBM3c9kpOZX8WkXGia3jb2oNYwG7JkWvnCs5D3qf+6dprAOG2xahb2bhkEJtcBYv1VtugWhBsfZB1UD5xGFr5zBjuDdUEfF0yKZSvaa66Bsnj6BP6z4Twc+DX0ihtjLwvBZbLB8fVUH/y5OVyY9qAmDSebChSUJFS/xjMoTYeGx50BmU5TgK6LtnkC5Ew5t8OoRRx4zppsVYWy5m3MLNigjQHn/GFKOs0c28H+EFi/I1BD8Dp4ZQL6OwH0M5IzGe0gsUSQPkU1XCch8zfW0U6dVhTXgiAu1hYhOrAloQLpGmQ9GQT9J+ZS5dfQR0a2nOSrJ9ubRqQAxoZHq+nYyP7V4wrBpEdl0y5Zm9mYowraDC32mAKyqxvBhru28985oBqOLkiIBKYgj9YYq5lbYKPq15f8PiaNgJ4Ooerr6XiZ; 4:4PcQCAkeV5UnWR0ylM7PVxuo5Gz74X9wuFNWhRhoKNNXbX+DbKLQ5Fckzx3iMzkhYgysa6rHcZlnOLqfHmMCjRIy/m2FxKFnOmqMCPNGDAu5CuwOCK2KTpOoLYLHHgRZwgCU77VcbGE9Y7nEmBhSdQTYzbzmZFi1GQhdYdf0We5q1anEqB2yzMtIg2TlcZtqX4RVe192b5e1ow4WChV2TokVAufHnEYv2Tmbnhr/YPYg82e1CKVoHvIY4eOQpqZkjADi9U/udUC8EzSS5RjY3A== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(3231254)(944501410)(52105095)(93006095)(93001095)(3002001)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123564045)(20161123558120)(20161123562045)(6072148)(201708071742011); SRVR:BN7PR07MB4211; BCL:0; PCL:0; RULEID:; SRVR:BN7PR07MB4211; X-Forefront-PRVS: 0656A4403B X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6069001)(39380400002)(376002)(366004)(39850400004)(396003)(346002)(199004)(189003)(105586002)(59450400001)(76506005)(106356001)(6496006)(956004)(486006)(575784001)(44832011)(386003)(476003)(50466002)(2361001)(2351001)(86362001)(26005)(16526019)(186003)(68736007)(58126008)(316002)(52116002)(478600001)(76176011)(446003)(16586007)(11346002)(72206003)(8676002)(1076002)(6486002)(7736002)(6116002)(81166006)(3846002)(6916009)(4326008)(53936002)(97736004)(6666003)(25786009)(23726003)(47776003)(5660300001)(107886003)(2906002)(81156014)(8936002)(66066001)(33656002)(305945005)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:BN7PR07MB4211; H:localhost; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4211; 23:Dr4XM1x9VKCV+WzeqwB7wMWdl+EOLplke+A90dFEGWHCLOXyemZ2F+LHWzIQAm6XVCkRZP/ewQiv7v3+nWD9h1LGSkw3eVAYM9LS2RjDSUViGVrAY0faXts3gwC5ZAyxxtvek+6A9hYJDd1hHRlIC/mW/qkj22Hfq2Y+Xbo0sysPftLtW13gX/sYl7ejZVXLtColSm/S6h6jSJhL0D9DhLp8LuF2T4zRWPy4Scl/+UMTJDtLrJnqMJ19Ubdte9Ofc0OeHzn6XzyR9h0lPYFQutdeNC+wlFda/KyVycMeL07lOTsaPhbHLaNMqejHaW6CAeAq2qaEP9KIXUy++7EVvWEy1i1M1LBRULOHD2SguQmjMLIao8HVjkHdrNOo2kcKb1IrX7KPnFDEPlomjS00QGqUjmpe76hCWEoKWgHamBfkF3mjqIO9tayvDI39CK2BARm5P0hn/P4dpI7iUKcOIwd86PIJmI8iPx6XUa8GPskE/KUkjxna2vbiPRkiMT85sQQ6ux1e/0CVyuZJRYsSh4gDqexxZwQQ7gCSvTDQJll+vzeyrMT16kpq6sl6hqnGVe35Czmtr5Db8D5rVEmcjwArohAOoskeaOB+OErI8GWCLB/Fea6um5Q+uUejJP3yBmoUD62Wfwkagwqv/TMaDGep7nrEteyRngWoa9P5lRWgbnGhBq1oqgyOmbpFQnKr0bsnZIbePl0MUjchdNUxFO1pDh2piCXTkNpNxfD5CGHWH6OUNzwm9eEK8CwJTEMrbvd0LdVnF0LjkZi96JdiW5kmNx6axaQ+jq0cPnUDFbZCYvg8RJWfdR+k/rgPMAxd74dh1XpKBdwhmpIAI85lpZ3KeVC5Xc2F3+UcS/FQhXon8XRcVn+vA6XrBG96Sew3+z6tW1HJWPrFcFlYcaJ+gv9BVrtfVtoB14+lZ8JIDByE/E/2ZJ/PslOLkW9NNlrqTK7avAcBBk1Qs92bE17XVjkFxgCBvMjeOgWdM8e2A/sWfpHAayf0meNSozNRdpWA6zjABUkfmceX0tTzvzrDamfLUXwKp1ALaY7ldVmeWcLKcYThQOTfTOA2HRo31pkIWkUfsnMxnSsuzQGXAAp2VIFOMqCorrGZGxW7TqxdSuruowDVuoTTG5TMHqtXtpT8LMzBp0HR3O9XmkmZAOwScpFWjlhVUoCIK46Q5grRLg3HBLBQRjo3hiSinvMY6RFLfDTXd9dmvYkuDd1Cdjfar+Z6T9/fmN/gVdtkJWsgXOxsBg/38Nr1qGqsZHaql7X0uJlbJMaioX3gi/g7U0vpeu8qOoGNO9NUEbVfwwF8LwbwQ8zloshmLzJ9TcH0MQ/7CoMIcAC+RHrQk2vrjsooYQ== X-Microsoft-Antispam-Message-Info: 3byf0ArbHAj1ZE/2o7rklRxecYIbSw5511+v0FnM/tRdKrWh+SDC31jXNCrGPi/wl5IkmutQY5mGzDdvWgyXQ2hSQvmiU8ADGjyvoW5ORLnde9Bsv0Agn9sFL+01uYvHmJs9tpLyxh/j1k1Qkfj2GP36HiYdeHlOmsz0OPuro0ErJbqCDQfM2PH43caz95qw X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4211; 6:Ypq6mBiTDPIMYsjPoXUSR/iL4m+RlhyOKnXDlA4pYzNwqhYa5eNiVo0f4kRwRA5KPBDXhGJiAy01J0qgMq36pHhnvwaAHLaRvZGLuVwkHcjVXjNyOcw6Yc2BPQdoGnYw4+tBMvjzPQxICSqj4/Qbs/Gls4TNV3ZLN9mB4qzJ9sz9SzNUziitLf8TCCFMeuuDU86kC6nZf0xGq/g7DPcyx/6ovaQcgPPLu8wU/7aKDsbFnnrFK0QvEQaM1Kv+Hoqspz0wdD2g91tlzlYZeOTSejQHNr0q51qsKVubQv9aGzHlm2c00GB+07sKgYOy0i6edRKwYlvt34xiCp6rR6eojydj287HTYV4j0CJmOHuXevS4tem23w+XiPr8sPI0AymiwmsYHo4qdbXL66fwUPhyaYoiJqcGdLD1o9HH2VQXkg81NYMGUWutJwS9BWFhYhE+u/3bTRevHkfF/96pUMBMg==; 5:VwC6IIuFPygmfml57VVEJJXBalDJrZVZIhnLta3WHqHlApyBP84lJVdzf+RAecmuoeOzQ1GXbq1ccS09MKxugstkp7WAyFpFXdLFRqT3SNe/LgylIsykTsGt/+z+W9ClALaSDER4eAKokhuUL+shmX3a7uHHJTiAzW2gaPFPZy4=; 24:48Bg3pffTzARTt0GmlVc47K1X8UfGVvjkl03RA1ToQeJr++Z9Zu7+FCJ0lAvbi56XLy9E141BG+Wv+ORXp0Dln4Inqw4awDZfBeRNWQpkMg= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4211; 7:22gtGNJHzbp9NqjvdVeayNw6YVRa2ARibbqBiDKIQRudxsOdtWI23U7Xdq44e22K48lNlkczo4g0/hgpLsJAx1viKposnjo6fg1e5EjHhu2OPO7eP2Nb6Ccef2sE8HCmFm05YevmbE2Rfr9aiBqQBbl1dPSLCeZq/RK7WDExaC8EhGAGsvk9037BuGN0v+5N9ddL2U9l3CTvr8xL23PbYhwQ0alnnaBIPAS3u2Y861+6a9eTL8gOb+iXxHHCasii X-MS-Office365-Filtering-Correlation-Id: b89b44db-b910-4e0d-fdeb-08d5acd1e3dd X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2018 06:33:01.1724 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b89b44db-b910-4e0d-fdeb-08d5acd1e3dd X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR07MB4211 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Intiyaz Basha Enhancing driver to accept max supported queues for ethtool --set-channels Signed-off-by: Intiyaz Basha Acked-by: Derek Chickles Signed-off-by: Felix Manlunas --- .../ethernet/cavium/liquidio/cn23xx_pf_device.c | 6 +- .../ethernet/cavium/liquidio/cn23xx_pf_device.h | 2 + drivers/net/ethernet/cavium/liquidio/lio_core.c | 4 +- drivers/net/ethernet/cavium/liquidio/lio_ethtool.c | 263 +++++++++++++++++++-- drivers/net/ethernet/cavium/liquidio/lio_main.c | 64 +++-- drivers/net/ethernet/cavium/liquidio/lio_vf_main.c | 8 +- .../net/ethernet/cavium/liquidio/liquidio_common.h | 1 + .../net/ethernet/cavium/liquidio/octeon_device.c | 12 +- .../net/ethernet/cavium/liquidio/octeon_device.h | 2 +- .../net/ethernet/cavium/liquidio/octeon_network.h | 12 +- 10 files changed, 316 insertions(+), 58 deletions(-) diff --git a/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.c b/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.c index bc9861c..929d485 100644 --- a/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.c +++ b/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.c @@ -1245,7 +1245,7 @@ static void cn23xx_setup_reg_address(struct octeon_device *oct) CN23XX_SLI_MAC_PF_INT_ENB64(oct->pcie_port, oct->pf_num); } -static int cn23xx_sriov_config(struct octeon_device *oct) +int cn23xx_sriov_config(struct octeon_device *oct) { struct octeon_cn23xx_pf *cn23xx = (struct octeon_cn23xx_pf *)oct->chip; u32 max_rings, total_rings, max_vfs, rings_per_vf; @@ -1269,8 +1269,8 @@ static int cn23xx_sriov_config(struct octeon_device *oct) break; } - if (max_rings <= num_present_cpus()) - num_pf_rings = 1; + if (oct->sriov_info.num_pf_rings) + num_pf_rings = oct->sriov_info.num_pf_rings; else num_pf_rings = num_present_cpus(); diff --git a/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.h b/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.h index 63b3de4..e6f31d0 100644 --- a/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.h +++ b/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.h @@ -61,6 +61,8 @@ u32 cn23xx_pf_get_oq_ticks(struct octeon_device *oct, u32 time_intr_in_us); void cn23xx_dump_pf_initialized_regs(struct octeon_device *oct); +int cn23xx_sriov_config(struct octeon_device *oct); + int cn23xx_fw_loaded(struct octeon_device *oct); void cn23xx_tell_vf_its_macaddr_changed(struct octeon_device *oct, int vfidx, diff --git a/drivers/net/ethernet/cavium/liquidio/lio_core.c b/drivers/net/ethernet/cavium/liquidio/lio_core.c index b4f9275..7e24b51 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_core.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_core.c @@ -78,7 +78,7 @@ void lio_delete_glists(struct lio *lio) if (!lio->glist) return; - for (i = 0; i < lio->linfo.num_txpciq; i++) { + for (i = 0; i < lio->oct_dev->num_iqs; i++) { do { g = (struct octnic_gather *) lio_list_delete_head(&lio->glist[i]); @@ -1036,8 +1036,8 @@ int octeon_setup_interrupt(struct octeon_device *oct, u32 num_ioqs) int num_ioq_vectors; int irqret, err; - oct->num_msix_irqs = num_ioqs; if (oct->msix_on) { + oct->num_msix_irqs = num_ioqs; if (OCTEON_CN23XX_PF(oct)) { num_interrupts = MAX_IOQ_INTERRUPTS_PER_PF + 1; diff --git a/drivers/net/ethernet/cavium/liquidio/lio_ethtool.c b/drivers/net/ethernet/cavium/liquidio/lio_ethtool.c index 9926a12..7ca246e 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_ethtool.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_ethtool.c @@ -353,7 +353,14 @@ lio_ethtool_get_channels(struct net_device *dev, rx_count = CFG_GET_NUM_RXQS_NIC_IF(conf6x, lio->ifidx); tx_count = CFG_GET_NUM_TXQS_NIC_IF(conf6x, lio->ifidx); } else if (OCTEON_CN23XX_PF(oct)) { - max_combined = lio->linfo.num_txpciq; + if (oct->sriov_info.sriov_enabled) { + max_combined = lio->linfo.num_txpciq; + } else { + struct octeon_config *conf23_pf = + CHIP_CONF(oct, cn23xx_pf); + + max_combined = CFG_GET_IQ_MAX_Q(conf23_pf); + } combined_count = oct->num_iqs; } else if (OCTEON_CN23XX_VF(oct)) { u64 reg_val = 0ULL; @@ -417,9 +424,15 @@ lio_irq_reallocate_irqs(struct octeon_device *oct, uint32_t num_ioqs) kfree(oct->irq_name_storage); oct->irq_name_storage = NULL; + + if (octeon_allocate_ioq_vector(oct, num_ioqs)) { + dev_err(&oct->pci_dev->dev, "OCTEON: ioq vector allocation failed\n"); + return -1; + } + if (octeon_setup_interrupt(oct, num_ioqs)) { dev_info(&oct->pci_dev->dev, "Setup interrupt failed\n"); - return 1; + return -1; } /* Enable Octeon device interrupts */ @@ -449,7 +462,16 @@ lio_ethtool_set_channels(struct net_device *dev, combined_count = channel->combined_count; if (OCTEON_CN23XX_PF(oct)) { - max_combined = channel->max_combined; + if (oct->sriov_info.sriov_enabled) { + max_combined = lio->linfo.num_txpciq; + } else { + struct octeon_config *conf23_pf = + CHIP_CONF(oct, + cn23xx_pf); + + max_combined = + CFG_GET_IQ_MAX_Q(conf23_pf); + } } else if (OCTEON_CN23XX_VF(oct)) { u64 reg_val = 0ULL; u64 ctrl = CN23XX_VF_SLI_IQ_PKT_CONTROL64(0); @@ -477,7 +499,6 @@ lio_ethtool_set_channels(struct net_device *dev, if (lio_reset_queues(dev, combined_count)) return -EINVAL; - lio_irq_reallocate_irqs(oct, combined_count); if (stopped) dev->netdev_ops->ndo_open(dev); @@ -816,12 +837,120 @@ lio_ethtool_get_ringparam(struct net_device *netdev, ering->rx_jumbo_max_pending = 0; } +static int lio_23xx_reconfigure_queue_count(struct lio *lio) +{ + struct octeon_device *oct = lio->oct_dev; + struct liquidio_if_cfg_context *ctx; + u32 resp_size, ctx_size, data_size; + struct liquidio_if_cfg_resp *resp; + struct octeon_soft_command *sc; + union oct_nic_if_cfg if_cfg; + struct lio_version *vdata; + u32 ifidx_or_pfnum; + int retval; + int j; + + resp_size = sizeof(struct liquidio_if_cfg_resp); + ctx_size = sizeof(struct liquidio_if_cfg_context); + data_size = sizeof(struct lio_version); + sc = (struct octeon_soft_command *) + octeon_alloc_soft_command(oct, data_size, + resp_size, ctx_size); + if (!sc) { + dev_err(&oct->pci_dev->dev, "%s: Failed to allocate soft command\n", + __func__); + return -1; + } + + resp = (struct liquidio_if_cfg_resp *)sc->virtrptr; + ctx = (struct liquidio_if_cfg_context *)sc->ctxptr; + vdata = (struct lio_version *)sc->virtdptr; + + vdata->major = (__force u16)cpu_to_be16(LIQUIDIO_BASE_MAJOR_VERSION); + vdata->minor = (__force u16)cpu_to_be16(LIQUIDIO_BASE_MINOR_VERSION); + vdata->micro = (__force u16)cpu_to_be16(LIQUIDIO_BASE_MICRO_VERSION); + + ifidx_or_pfnum = oct->pf_num; + WRITE_ONCE(ctx->cond, 0); + ctx->octeon_id = lio_get_device_id(oct); + init_waitqueue_head(&ctx->wc); + + if_cfg.u64 = 0; + if_cfg.s.num_iqueues = oct->sriov_info.num_pf_rings; + if_cfg.s.num_oqueues = oct->sriov_info.num_pf_rings; + if_cfg.s.base_queue = oct->sriov_info.pf_srn; + if_cfg.s.gmx_port_id = oct->pf_num; + + sc->iq_no = 0; + octeon_prepare_soft_command(oct, sc, OPCODE_NIC, + OPCODE_NIC_QCOUNT_UPDATE, 0, + if_cfg.u64, 0); + sc->callback = lio_if_cfg_callback; + sc->callback_arg = sc; + sc->wait_time = LIO_IFCFG_WAIT_TIME; + + retval = octeon_send_soft_command(oct, sc); + if (retval == IQ_SEND_FAILED) { + dev_err(&oct->pci_dev->dev, + "iq/oq config failed status: %x\n", + retval); + goto qcount_update_fail; + } + + if (sleep_cond(&ctx->wc, &ctx->cond) == -EINTR) { + dev_err(&oct->pci_dev->dev, "Wait interrupted\n"); + return -1; + } + + retval = resp->status; + if (retval) { + dev_err(&oct->pci_dev->dev, "iq/oq config failed\n"); + goto qcount_update_fail; + } + + octeon_swap_8B_data((u64 *)(&resp->cfg_info), + (sizeof(struct liquidio_if_cfg_info)) >> 3); + + lio->ifidx = ifidx_or_pfnum; + lio->linfo.num_rxpciq = hweight64(resp->cfg_info.iqmask); + lio->linfo.num_txpciq = hweight64(resp->cfg_info.iqmask); + for (j = 0; j < lio->linfo.num_rxpciq; j++) { + lio->linfo.rxpciq[j].u64 = + resp->cfg_info.linfo.rxpciq[j].u64; + } + + for (j = 0; j < lio->linfo.num_txpciq; j++) { + lio->linfo.txpciq[j].u64 = + resp->cfg_info.linfo.txpciq[j].u64; + } + + lio->linfo.hw_addr = resp->cfg_info.linfo.hw_addr; + lio->linfo.gmxport = resp->cfg_info.linfo.gmxport; + lio->linfo.link.u64 = resp->cfg_info.linfo.link.u64; + lio->txq = lio->linfo.txpciq[0].s.q_no; + lio->rxq = lio->linfo.rxpciq[0].s.q_no; + + octeon_free_soft_command(oct, sc); + dev_info(&oct->pci_dev->dev, "Queue count updated to %d\n", + lio->linfo.num_rxpciq); + + return 0; + +qcount_update_fail: + octeon_free_soft_command(oct, sc); + + return -1; +} + static int lio_reset_queues(struct net_device *netdev, uint32_t num_qs) { struct lio *lio = GET_LIO(netdev); struct octeon_device *oct = lio->oct_dev; + int i, queue_count_update = 0; struct napi_struct *napi, *n; - int i, update = 0; + int ret; + + schedule_timeout_uninterruptible(msecs_to_jiffies(100)); if (wait_for_pending_requests(oct)) dev_err(&oct->pci_dev->dev, "There were pending requests\n"); @@ -830,7 +959,7 @@ static int lio_reset_queues(struct net_device *netdev, uint32_t num_qs) dev_err(&oct->pci_dev->dev, "IQ had pending instructions\n"); if (octeon_set_io_queues_off(oct)) { - dev_err(&oct->pci_dev->dev, "setting io queues off failed\n"); + dev_err(&oct->pci_dev->dev, "Setting io queues off failed\n"); return -1; } @@ -843,9 +972,40 @@ static int lio_reset_queues(struct net_device *netdev, uint32_t num_qs) netif_napi_del(napi); if (num_qs != oct->num_iqs) { - netif_set_real_num_rx_queues(netdev, num_qs); - netif_set_real_num_tx_queues(netdev, num_qs); - update = 1; + ret = netif_set_real_num_rx_queues(netdev, num_qs); + if (ret) { + dev_err(&oct->pci_dev->dev, + "Setting real number rx failed\n"); + return ret; + } + + ret = netif_set_real_num_tx_queues(netdev, num_qs); + if (ret) { + dev_err(&oct->pci_dev->dev, + "Setting real number tx failed\n"); + return ret; + } + + /* The value of queue_count_update decides whether it is the + * queue count or the descriptor count that is being + * re-configured. + */ + queue_count_update = 1; + } + + /* Re-configuration of queues can happen in two scenarios, SRIOV enabled + * and SRIOV disabled. Few things like recreating queue zero, resetting + * glists and IRQs are required for both. For the latter, some more + * steps like updating sriov_info for the octeon device need to be done. + */ + if (queue_count_update) { + lio_delete_glists(lio); + + /* Delete mbox for PF which is SRIOV disabled because sriov_info + * will be now changed. + */ + if ((OCTEON_CN23XX_PF(oct)) && !oct->sriov_info.sriov_enabled) + oct->fn_list.free_mbox(oct); } for (i = 0; i < MAX_OCTEON_OUTPUT_QUEUES(oct); i++) { @@ -860,24 +1020,91 @@ static int lio_reset_queues(struct net_device *netdev, uint32_t num_qs) octeon_delete_instr_queue(oct, i); } + if (queue_count_update) { + /* For PF re-configure sriov related information */ + if ((OCTEON_CN23XX_PF(oct)) && + !oct->sriov_info.sriov_enabled) { + oct->sriov_info.num_pf_rings = num_qs; + if (cn23xx_sriov_config(oct)) { + dev_err(&oct->pci_dev->dev, + "Queue reset aborted: SRIOV config failed\n"); + return -1; + } + + num_qs = oct->sriov_info.num_pf_rings; + } + } + if (oct->fn_list.setup_device_regs(oct)) { dev_err(&oct->pci_dev->dev, "Failed to configure device registers\n"); return -1; } - if (liquidio_setup_io_queues(oct, 0, num_qs, num_qs)) { - dev_err(&oct->pci_dev->dev, "IO queues initialization failed\n"); - return -1; + /* The following are needed in case of queue count re-configuration and + * not for descriptor count re-configuration. + */ + if (queue_count_update) { + if (octeon_setup_instr_queues(oct)) + return -1; + + if (octeon_setup_output_queues(oct)) + return -1; + + /* Recreating mbox for PF that is SRIOV disabled */ + if (OCTEON_CN23XX_PF(oct) && !oct->sriov_info.sriov_enabled) { + if (oct->fn_list.setup_mbox(oct)) { + dev_err(&oct->pci_dev->dev, "Mailbox setup failed\n"); + return -1; + } + } + + /* Deleting and recreating IRQs whether the interface is SRIOV + * enabled or disabled. + */ + if (lio_irq_reallocate_irqs(oct, num_qs)) { + dev_err(&oct->pci_dev->dev, "IRQs could not be allocated\n"); + return -1; + } + + /* Enable the input and output queues for this Octeon device */ + if (oct->fn_list.enable_io_queues(oct)) { + dev_err(&oct->pci_dev->dev, "Failed to enable input/output queues\n"); + return -1; + } + + for (i = 0; i < oct->num_oqs; i++) + writel(oct->droq[i]->max_count, + oct->droq[i]->pkts_credit_reg); + + /* Informing firmware about the new queue count. It is required + * for firmware to allocate more number of queues than those at + * load time. + */ + if (OCTEON_CN23XX_PF(oct) && !oct->sriov_info.sriov_enabled) { + if (lio_23xx_reconfigure_queue_count(lio)) + return -1; + } } - /* Enable the input and output queues for this Octeon device */ - if (oct->fn_list.enable_io_queues(oct)) { - dev_err(&oct->pci_dev->dev, "Failed to enable input/output queues"); + /* Once firmware is aware of the new value, queues can be recreated */ + if (liquidio_setup_io_queues(oct, 0, num_qs, num_qs)) { + dev_err(&oct->pci_dev->dev, "I/O queues creation failed\n"); return -1; } - if (update && lio_send_queue_count_update(netdev, num_qs)) - return -1; + if (queue_count_update) { + if (lio_setup_glists(oct, lio, num_qs)) { + dev_err(&oct->pci_dev->dev, "Gather list allocation failed\n"); + return -1; + } + + /* Send firmware the information about new number of queues + * if the interface is a VF or a PF that is SRIOV enabled. + */ + if (oct->sriov_info.sriov_enabled || OCTEON_CN23XX_VF(oct)) + if (lio_send_queue_count_update(netdev, num_qs)) + return -1; + } return 0; } @@ -922,7 +1149,7 @@ static int lio_ethtool_set_ringparam(struct net_device *netdev, CFG_SET_NUM_RX_DESCS_NIC_IF(octeon_get_conf(oct), lio->ifidx, rx_count); - if (lio_reset_queues(netdev, lio->linfo.num_txpciq)) + if (lio_reset_queues(netdev, oct->num_iqs)) goto err_lio_reset_queues; if (stopped) diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c index f414cd7..dc801b1 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_main.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c @@ -497,7 +497,7 @@ static void liquidio_deinit_pci(void) */ static inline int check_txq_status(struct lio *lio) { - int numqs = lio->netdev->num_tx_queues; + int numqs = lio->netdev->real_num_tx_queues; int ret_val = 0; int q, iq; @@ -1521,7 +1521,7 @@ static void free_netsgbuf(void *buf) i++; } - iq = skb_iq(lio, skb); + iq = skb_iq(lio->oct_dev, skb); spin_lock(&lio->glist_lock[iq]); list_add_tail(&g->list, &lio->glist[iq]); spin_unlock(&lio->glist_lock[iq]); @@ -1564,7 +1564,7 @@ static void free_netsgbuf_with_resp(void *buf) i++; } - iq = skb_iq(lio, skb); + iq = skb_iq(lio->oct_dev, skb); spin_lock(&lio->glist_lock[iq]); list_add_tail(&g->list, &lio->glist[iq]); @@ -1851,11 +1851,6 @@ static int liquidio_open(struct net_device *netdev) ifstate_set(lio, LIO_IFSTATE_RUNNING); - /* Ready for link status updates */ - lio->intf_open = 1; - - netif_info(lio, ifup, lio->netdev, "Interface Open, ready for traffic\n"); - if (OCTEON_CN23XX_PF(oct)) { if (!oct->msix_on) if (setup_tx_poll_fn(netdev)) @@ -1865,7 +1860,12 @@ static int liquidio_open(struct net_device *netdev) return -1; } - start_txqs(netdev); + netif_tx_start_all_queues(netdev); + + /* Ready for link status updates */ + lio->intf_open = 1; + + netif_info(lio, ifup, lio->netdev, "Interface Open, ready for traffic\n"); /* tell Octeon to start forwarding packets to host */ send_rx_ctrl_cmd(lio, 1); @@ -1888,11 +1888,15 @@ static int liquidio_stop(struct net_device *netdev) ifstate_reset(lio, LIO_IFSTATE_RUNNING); - netif_tx_disable(netdev); + /* Stop any link updates */ + lio->intf_open = 0; + + stop_txqs(netdev); /* Inform that netif carrier is down */ netif_carrier_off(netdev); - lio->intf_open = 0; + netif_tx_disable(netdev); + lio->linfo.link.s.link_up = 0; lio->link_changes++; @@ -2312,7 +2316,7 @@ static int liquidio_xmit(struct sk_buff *skb, struct net_device *netdev) lio = GET_LIO(netdev); oct = lio->oct_dev; - q_idx = skb_iq(lio, skb); + q_idx = skb_iq(oct, skb); tag = q_idx; iq_no = lio->linfo.txpciq[q_idx].s.q_no; @@ -3278,6 +3282,7 @@ static int setup_nic_devices(struct octeon_device *octeon_dev) struct liquidio_if_cfg_resp *resp; struct octdev_props *props; int retval, num_iqueues, num_oqueues; + int max_num_queues = 0; union oct_nic_if_cfg if_cfg; unsigned int base_queue; unsigned int gmx_port_id; @@ -3360,7 +3365,7 @@ static int setup_nic_devices(struct octeon_device *octeon_dev) sc->callback = lio_if_cfg_callback; sc->callback_arg = sc; - sc->wait_time = 3000; + sc->wait_time = LIO_IFCFG_WAIT_TIME; retval = octeon_send_soft_command(octeon_dev, sc); if (retval == IQ_SEND_FAILED) { @@ -3414,11 +3419,20 @@ static int setup_nic_devices(struct octeon_device *octeon_dev) resp->cfg_info.oqmask); goto setup_nic_dev_fail; } + + if (OCTEON_CN6XXX(octeon_dev)) { + max_num_queues = CFG_GET_IQ_MAX_Q(CHIP_CONF(octeon_dev, + cn6xxx)); + } else if (OCTEON_CN23XX_PF(octeon_dev)) { + max_num_queues = CFG_GET_IQ_MAX_Q(CHIP_CONF(octeon_dev, + cn23xx_pf)); + } + dev_dbg(&octeon_dev->pci_dev->dev, - "interface %d, iqmask %016llx, oqmask %016llx, numiqueues %d, numoqueues %d\n", + "interface %d, iqmask %016llx, oqmask %016llx, numiqueues %d, numoqueues %d max_num_queues: %d\n", i, resp->cfg_info.iqmask, resp->cfg_info.oqmask, - num_iqueues, num_oqueues); - netdev = alloc_etherdev_mq(LIO_SIZE, num_iqueues); + num_iqueues, num_oqueues, max_num_queues); + netdev = alloc_etherdev_mq(LIO_SIZE, max_num_queues); if (!netdev) { dev_err(&octeon_dev->pci_dev->dev, "Device allocation failed\n"); @@ -3433,6 +3447,20 @@ static int setup_nic_devices(struct octeon_device *octeon_dev) netdev->netdev_ops = &lionetdevops; SWITCHDEV_SET_OPS(netdev, &lio_pf_switchdev_ops); + retval = netif_set_real_num_rx_queues(netdev, num_oqueues); + if (retval) { + dev_err(&octeon_dev->pci_dev->dev, + "setting real number rx failed\n"); + goto setup_nic_dev_fail; + } + + retval = netif_set_real_num_tx_queues(netdev, num_iqueues); + if (retval) { + dev_err(&octeon_dev->pci_dev->dev, + "setting real number tx failed\n"); + goto setup_nic_dev_fail; + } + lio = GET_LIO(netdev); memset(lio, 0, sizeof(struct lio)); @@ -4053,7 +4081,9 @@ static int octeon_device_init(struct octeon_device *octeon_dev) } atomic_set(&octeon_dev->status, OCT_DEV_MBOX_SETUP_DONE); - if (octeon_allocate_ioq_vector(octeon_dev)) { + if (octeon_allocate_ioq_vector + (octeon_dev, + octeon_dev->sriov_info.num_pf_rings)) { dev_err(&octeon_dev->pci_dev->dev, "OCTEON: ioq vector allocation failed\n"); return 1; } diff --git a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c index 246752a..4b5ba02 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c @@ -849,7 +849,7 @@ static void free_netsgbuf(void *buf) i++; } - iq = skb_iq(lio, skb); + iq = skb_iq(lio->oct_dev, skb); spin_lock(&lio->glist_lock[iq]); list_add_tail(&g->list, &lio->glist[iq]); @@ -893,7 +893,7 @@ static void free_netsgbuf_with_resp(void *buf) i++; } - iq = skb_iq(lio, skb); + iq = skb_iq(lio->oct_dev, skb); spin_lock(&lio->glist_lock[iq]); list_add_tail(&g->list, &lio->glist[iq]); @@ -1392,7 +1392,7 @@ static int liquidio_xmit(struct sk_buff *skb, struct net_device *netdev) lio = GET_LIO(netdev); oct = lio->oct_dev; - q_idx = skb_iq(lio, skb); + q_idx = skb_iq(lio->oct_dev, skb); tag = q_idx; iq_no = lio->linfo.txpciq[q_idx].s.q_no; @@ -2324,7 +2324,7 @@ static int octeon_device_init(struct octeon_device *oct) } atomic_set(&oct->status, OCT_DEV_MBOX_SETUP_DONE); - if (octeon_allocate_ioq_vector(oct)) { + if (octeon_allocate_ioq_vector(oct, oct->sriov_info.rings_per_vf)) { dev_err(&oct->pci_dev->dev, "ioq vector allocation failed\n"); return 1; } diff --git a/drivers/net/ethernet/cavium/liquidio/liquidio_common.h b/drivers/net/ethernet/cavium/liquidio/liquidio_common.h index 34a94da..ba854f1 100644 --- a/drivers/net/ethernet/cavium/liquidio/liquidio_common.h +++ b/drivers/net/ethernet/cavium/liquidio/liquidio_common.h @@ -84,6 +84,7 @@ enum octeon_tag_type { #define OPCODE_NIC_IF_CFG 0x09 #define OPCODE_NIC_VF_DRV_NOTICE 0x0A #define OPCODE_NIC_INTRMOD_PARAMS 0x0B +#define OPCODE_NIC_QCOUNT_UPDATE 0x12 #define OPCODE_NIC_SET_TRUSTED_VF 0x13 #define OPCODE_NIC_SYNC_OCTEON_TIME 0x14 #define VF_DRV_LOADED 1 diff --git a/drivers/net/ethernet/cavium/liquidio/octeon_device.c b/drivers/net/ethernet/cavium/liquidio/octeon_device.c index f38abf6..f878a55 100644 --- a/drivers/net/ethernet/cavium/liquidio/octeon_device.c +++ b/drivers/net/ethernet/cavium/liquidio/octeon_device.c @@ -824,23 +824,18 @@ int octeon_deregister_device(struct octeon_device *oct) } int -octeon_allocate_ioq_vector(struct octeon_device *oct) +octeon_allocate_ioq_vector(struct octeon_device *oct, u32 num_ioqs) { - int i, num_ioqs = 0; struct octeon_ioq_vector *ioq_vector; int cpu_num; int size; - - if (OCTEON_CN23XX_PF(oct)) - num_ioqs = oct->sriov_info.num_pf_rings; - else if (OCTEON_CN23XX_VF(oct)) - num_ioqs = oct->sriov_info.rings_per_vf; + int i; size = sizeof(struct octeon_ioq_vector) * num_ioqs; oct->ioq_vector = vzalloc(size); if (!oct->ioq_vector) - return 1; + return -1; for (i = 0; i < num_ioqs; i++) { ioq_vector = &oct->ioq_vector[i]; ioq_vector->oct_dev = oct; @@ -856,6 +851,7 @@ octeon_allocate_ioq_vector(struct octeon_device *oct) else ioq_vector->ioq_num = i; } + return 0; } diff --git a/drivers/net/ethernet/cavium/liquidio/octeon_device.h b/drivers/net/ethernet/cavium/liquidio/octeon_device.h index 91937cc..9430c0a 100644 --- a/drivers/net/ethernet/cavium/liquidio/octeon_device.h +++ b/drivers/net/ethernet/cavium/liquidio/octeon_device.h @@ -867,7 +867,7 @@ void *oct_get_config_info(struct octeon_device *oct, u16 card_type); struct octeon_config *octeon_get_conf(struct octeon_device *oct); void octeon_free_ioq_vector(struct octeon_device *oct); -int octeon_allocate_ioq_vector(struct octeon_device *oct); +int octeon_allocate_ioq_vector(struct octeon_device *oct, u32 num_ioqs); void lio_enable_irq(struct octeon_droq *droq, struct octeon_instr_queue *iq); /* LiquidIO driver pivate flags */ diff --git a/drivers/net/ethernet/cavium/liquidio/octeon_network.h b/drivers/net/ethernet/cavium/liquidio/octeon_network.h index 8894889..f3bf635 100644 --- a/drivers/net/ethernet/cavium/liquidio/octeon_network.h +++ b/drivers/net/ethernet/cavium/liquidio/octeon_network.h @@ -47,6 +47,8 @@ struct liquidio_if_cfg_resp { u64 status; }; +#define LIO_IFCFG_WAIT_TIME 3000 /* In milli seconds */ + /* Structure of a node in list of gather components maintained by * NIC driver for each network device. */ @@ -544,7 +546,7 @@ static inline void stop_txqs(struct net_device *netdev) { int i; - for (i = 0; i < netdev->num_tx_queues; i++) + for (i = 0; i < netdev->real_num_tx_queues; i++) netif_stop_subqueue(netdev, i); } @@ -557,7 +559,7 @@ static inline void wake_txqs(struct net_device *netdev) struct lio *lio = GET_LIO(netdev); int i, qno; - for (i = 0; i < netdev->num_tx_queues; i++) { + for (i = 0; i < netdev->real_num_tx_queues; i++) { qno = lio->linfo.txpciq[i % lio->oct_dev->num_iqs].s.q_no; if (__netif_subqueue_stopped(netdev, i)) { @@ -578,14 +580,14 @@ static inline void start_txqs(struct net_device *netdev) int i; if (lio->linfo.link.s.link_up) { - for (i = 0; i < netdev->num_tx_queues; i++) + for (i = 0; i < netdev->real_num_tx_queues; i++) netif_start_subqueue(netdev, i); } } -static inline int skb_iq(struct lio *lio, struct sk_buff *skb) +static inline int skb_iq(struct octeon_device *oct, struct sk_buff *skb) { - return skb->queue_mapping % lio->linfo.num_txpciq; + return skb->queue_mapping % oct->num_iqs; } /**