From patchwork Wed Nov 20 17:48:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sunil Kovvuri X-Patchwork-Id: 1198402 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="E7q8OuSK"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 47J9Fm1NGpz9sPn for ; Thu, 21 Nov 2019 04:50:20 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728024AbfKTRuT (ORCPT ); Wed, 20 Nov 2019 12:50:19 -0500 Received: from mail-pg1-f175.google.com ([209.85.215.175]:41515 "EHLO mail-pg1-f175.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726907AbfKTRuR (ORCPT ); Wed, 20 Nov 2019 12:50:17 -0500 Received: by mail-pg1-f175.google.com with SMTP id 207so78412pge.8 for ; Wed, 20 Nov 2019 09:50:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=cEGkrqnTiounr1guH0iyywyvbWV4X5eDmEqM8UfYYwM=; b=E7q8OuSKv4chX1r4XHfUF/3eJt1TH/Eerrmc+ESg1Q8fnT0CIry8zDEUQM6U8AxSgN wNSe9BfC5GboDm/EgoN923KB/D3kODVXow/5Hya6UHtAUR8bNDVTSSIrpam2ofqgmwHV 6jQNrDgGegm2629kPFnaq5mTrwVOebtaQ9FsxP6xmCmzPE7OuBUTOmFLPAbE9RJRx4QF nJojNCNnw6rVoyyr9cTPDQUUeUDJKZ/bWQo1TuYRNizWJjH9QcLbWo6lA4Gvz/srzC15 njnQtKcIofNXfcw7xSWjYRqqrzZ9TUuZl9iHyMWID+w1Ur/v1mw2NUVSpO1xGC5GtD/P ks4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=cEGkrqnTiounr1guH0iyywyvbWV4X5eDmEqM8UfYYwM=; b=MBpwFp8LQ8puLwE98aNNHjdrqkXp7UDWND0YFuQ1Mdq3qcUMGRUP4Zr5ezJ64W0jL/ X9LUP1zAc9FzroMesR1H2/MtvL5l+v5W0M9JgJdpTmhKz7kRPujltY1x1Ln7kiOpEuzV E5G46atLcaJWIi2HcwP3vKewLvSiUiVXjZLJ8oIacWpyOP7Cj+XdccSkyjQeWeobHaZP S/Xp4PeT7U+P6Cw0dTZ+mOg/Hq4qzxzi7MGR4LpEg3US0dopKlpSVWKZa2d32muKS2hG +w3O67Q31UiQaOPOscnZPwPOTtfn9F2YSKWjpkKdbQngiJ+9l+roqI9lursfZrMU1aHg /ruQ== X-Gm-Message-State: APjAAAV8FR8uVYMlnv1Qy0L3vefWCUYHIFX49/ZYumZYZGi0WG1mQXMs 0lyFlk3B4uAWRknUbV6cJa3qSZgY X-Google-Smtp-Source: APXvYqw0msB1OWY/KoiNa2tf4y9cNAy1Gu1QznzzsMgdvvmmRa5DiyDIJrUDZQDEjcZXphFnrFQvgQ== X-Received: by 2002:a65:5542:: with SMTP id t2mr4596123pgr.74.1574272209378; Wed, 20 Nov 2019 09:50:09 -0800 (PST) Received: from machine421.caveonetworks.com ([115.113.156.2]) by smtp.googlemail.com with ESMTPSA id y24sm32230522pfr.116.2019.11.20.09.50.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 20 Nov 2019 09:50:07 -0800 (PST) From: sunil.kovvuri@gmail.com To: netdev@vger.kernel.org Cc: davem@davemloft.net, jakub.kicinski@netronome.com, Sunil Goutham Subject: [PATCH v3 16/16] Documentation: net: octeontx2: Add RVU HW and drivers overview. Date: Wed, 20 Nov 2019 23:18:06 +0530 Message-Id: <1574272086-21055-17-git-send-email-sunil.kovvuri@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1574272086-21055-1-git-send-email-sunil.kovvuri@gmail.com> References: <1574272086-21055-1-git-send-email-sunil.kovvuri@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Sunil Goutham Added high level overview of OcteonTx2 RVU HW and functionality of various drivers which will be upstreamed. Signed-off-by: Sunil Goutham --- Documentation/networking/device_drivers/index.rst | 1 + .../device_drivers/marvell/octeontx2.rst | 162 + .../marvell/resource_virtualization_unit.svg | 3297 ++++++++++++++++++++ MAINTAINERS | 1 + 4 files changed, 3461 insertions(+) create mode 100644 Documentation/networking/device_drivers/marvell/octeontx2.rst create mode 100644 Documentation/networking/device_drivers/marvell/resource_virtualization_unit.svg diff --git a/Documentation/networking/device_drivers/index.rst b/Documentation/networking/device_drivers/index.rst index c1f7f75..8aba64b 100644 --- a/Documentation/networking/device_drivers/index.rst +++ b/Documentation/networking/device_drivers/index.rst @@ -22,6 +22,7 @@ Contents: intel/iavf intel/ice google/gve + marvell/octeontx2 mellanox/mlx5 netronome/nfp pensando/ionic diff --git a/Documentation/networking/device_drivers/marvell/octeontx2.rst b/Documentation/networking/device_drivers/marvell/octeontx2.rst new file mode 100644 index 0000000..c8a5150 --- /dev/null +++ b/Documentation/networking/device_drivers/marvell/octeontx2.rst @@ -0,0 +1,162 @@ +.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +============================================= +Marvell OcteonTx2 RVU Kernel Drivers +============================================= + +Copyright (c) 2019 Marvell International Ltd. + +Contents +======== + +- `Overview`_ +- `Drivers`_ +- `Basic packet flow`_ + +Overview +======== +Resource virtualization unit (RVU) on Marvell's OcteonTX2 SOC maps HW +resources from the network, crypto and other functional blocks into +PCI-compatible physical and virtual functions. Each functional block +again has multiple local functions (LFs) for provisioning to PCI devices. +RVU supports multiple PCIe SRIOV physical functions (PFs) and virtual +functions (VFs). PF0 is called the administrative / admin function (AF) +and has privileges to provision RVU functional block's LFs to each of the +PF/VF. + +RVU managed networking functional blocks + - Network pool or buffer allocator (NPA) + - Network interface controller (NIX) + - Network parser CAM (NPC) + - Schedule/Synchronize/Order unit (SSO) + - Loopback interface (LBK) + +RVU managed non-networking functional blocks + - Crypto accelerator (CPT) + - Scheduled timers unit (TIM) + - Schedule/Synchronize/Order unit (SSO) + Used for both networking and non networking usecases + +Resource provisioning examples + - A PF/VF with NIX-LF & NPA-LF resources works as a pure network device + - A PF/VF with CPT-LF resource works as a pure cyrpto offload device. + +.. kernel-figure:: resource_virtualization_unit.svg + :alt: RVU + :align: center + :figwidth: 60em + + RVU HW block connectivity + +RVU functional blocks are highly configurable as per software requirements. + +Firmware setups following stuff before kernel boots + - Enables required number of RVU PFs based on number of physical links. + - Number of VFs per PF are either static or configurable at compile time. + Based on config, firmware assigns VFs to each of the PFs. + - Also assigns MSIX vectors to each of PF and VFs. + - These are not changed after kernel boot. + +Drivers +======= + +Linux kernel will have multiple drivers registering to different PF and VFs +of RVU. Wrt networking there will be 3 flavours of drivers. + +Admin Function driver +--------------------- + +As mentioned above RVU PF0 is called the admin function (AF), this driver +supports resource provisioning and configuration of functional blocks. +Doesn't handle any I/O. It sets up few basic stuff but most of the +funcionality is achieved via configuration requests from PFs and VFs. + +PF/VFs communicates with AF via a shared memory region (mailbox). Upon +receiving requests AF does resource provisioning and other HW configuration. +AF is always attached to host kernel, but PFs and their VFs may be used by host +kernel itself, or attached to VMs or to userspace applications like +DPDK etc. So AF has to handle provisioning/configuration requests sent +by any device from any domain. + +AF driver also interacts with underlying firmware to + - Manage physical ethernet links ie CGX LMACs. + - Retrieve information like speed, duplex, autoneg etc + - Retrieve PHY EEPROM and stats. + - Configure FEC, PAM modes + - etc + +From pure networking side AF driver supports following functionality. + - Map a physical link to a RVU PF to which a netdev is registered. + - Attach NIX and NPA block LFs to RVU PF/VF which provide buffer pools, RQs, SQs + for regular networking functionality. + - Flow control (pause frames) enable/disable/config. + - HW PTP timestamping related config. + - NPC parser profile config, basically how to parse pkt and what info to extract. + - NPC extract profile config, what to extract from the pkt to match data in MCAM entries. + - Manage NPC MCAM entries, upon request can frame and install requested packet forwarding rules. + - Defines receive side scaling (RSS) algorithms. + - Defines segmentation offload algorithms (eg TSO) + - VLAN stripping, capture and insertion config. + - SSO and TIM blocks config which provide packet scheduling support. + - Debugfs support, to check current resource provising, current status of + NPA pools, NIX RQ, SQ and CQs, various stats etc which helps in debugging issues. + - And many more. + +Physical Function driver +------------------------ + +This RVU PF handles IO, is mapped to a physical ethernet link and this +driver registers a netdev. This supports SR-IOV. As said above this driver +communicates with AF with a mailbox. To retrieve information from physical +links this driver talks to AF and AF gets that info from firmware and responds +back ie cannot talk to firmware directly. + +Supports ethtool for configuring links, RSS, queue count, queue size, +flow control, ntuple filters, dump PHY EEPROM, config FEC etc. + +Virtual Function driver +----------------------- + +There are two types VFs, VFs that share the physical link with their parent +SR-IOV PF and the VFs which work in pairs using internal HW loopback channels (LBK). + +Type1: + - These VFs and their parent PF share a physical link and used for outside communication. + - VFs cannot communicate with AF directly, they send mbox message to PF and PF + forwards that to AF. AF after processing, responds back to PF and PF forwards + the reply to VF. + - From functionality point of view there is no difference between PF and VF as same type + HW resources are attached to both. But user would be able to configure few stuff only + from PF as PF is treated as owner/admin of the link. + +Type2: + - RVU PF0 ie admin function creates these VFs and maps them to loopback block's channels. + - A set of two VFs (VF0 & VF1, VF2 & VF3 .. so on) works as a pair ie pkts sent out of + VF0 will be received by VF1 and viceversa. + - These VFs can be used by applications or virtual machines to communicate between them + without sending traffic outside. There is no switch present in HW, hence the support + for loopback VFs. + - These communicate directly with AF (PF0) via mbox. + +Except for the IO channels or links used for packet reception and transmission there is +no other difference between these VF types. AF driver takes care of IO channel mapping, +hence same VF driver works for both types of devices. + +Basic packet flow +=========== +Ingress +------- +1. CGX LMAC receives packet. +2. Forwards the packet to the NIX block. +3. Then submitted to NPC block for parsing and then MCAM lookup to get the destination RVU device. +4. NIX LF attached to the destination RVU device allocates a buffer from RQ mapped buffer pool of NPA block LF. +5. RQ may be selected by RSS or by configuring MCAM rule with a RQ number. +6. Packet is DMA'ed and driver is notified. + +Egress +------ +1. Driver prepares a send descriptor and submits to SQ for transmission. +2. The SQ is already configured (by AF) to transmit on a specific link/channel. +3. The SQ descriptor ring is maintained in buffers allocated from SQ mapped pool of NPA block LF. +4. NIX block transmits the pkt on the designated channel. +5. NPC MCAM entries can be installed to divert pkt onto a different channel. diff --git a/Documentation/networking/device_drivers/marvell/resource_virtualization_unit.svg b/Documentation/networking/device_drivers/marvell/resource_virtualization_unit.svg new file mode 100644 index 0000000..3ff9839 --- /dev/null +++ b/Documentation/networking/device_drivers/marvell/resource_virtualization_unit.svg @@ -0,0 +1,3297 @@ + + + + diff --git a/MAINTAINERS b/MAINTAINERS index 7600494..bcce963 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -9860,6 +9860,7 @@ M: Jerin Jacob L: netdev@vger.kernel.org S: Supported F: drivers/net/ethernet/marvell/octeontx2/af/ +F: Documentation/networking/device_drivers/marvell/octeontx2.rst MATROX FRAMEBUFFER DRIVER L: linux-fbdev@vger.kernel.org