From patchwork Thu Feb 27 20:57:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: William Tu X-Patchwork-Id: 1246107 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=vU3FPACT; dkim-atps=neutral Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 48T4ks0WVxz9sPK for ; Fri, 28 Feb 2020 07:58:12 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 2E6F722DB0; Thu, 27 Feb 2020 20:58:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eEzGTGWWEdi7; Thu, 27 Feb 2020 20:58:08 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id B30B51FEAE; Thu, 27 Feb 2020 20:58:08 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 961EFC08A0; Thu, 27 Feb 2020 20:58:08 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 99B0CC0177 for ; Thu, 27 Feb 2020 20:58:07 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 890AF87E6D for ; Thu, 27 Feb 2020 20:58:07 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PfSQHy1UUKoS for ; Thu, 27 Feb 2020 20:58:07 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by hemlock.osuosl.org (Postfix) with ESMTPS id EE32287D09 for ; Thu, 27 Feb 2020 20:58:06 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id d6so310129pgn.5 for ; Thu, 27 Feb 2020 12:58:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=osKS4jvQ0qJ0rbksDoVspyJjwi89uVj7lwPuZCFF4oM=; b=vU3FPACTwWAIIdCnCMxqpeOq2+Z2QNxpobopbMSK3EW00U8Uj1/1oJMNqxMcGLzwGD 1Fk0ffyTKduG9foeVFSmWb605kPNsvwtxhlvOTWhbwJcFQo+8RKp4k8PcNPW6mjFD5b5 La+PPZ3QP4EEJRE04tT6ricOIKsQyRoQvlT3B9azNPxrDuFy7kF3BI2O9vtBu5m7/LJ9 l3x1q4Eo2hPSNBJC025WzVGuGqUMho9sdQMOMxyuG1zrQzFgswdQ9IlWnqRnCyOE6x/I 81RLF8nJdBnFz6ubQ3NHWoDhlp0B20PUa8MjmGklSpbyddyuxccA3rnPLc8nKuCMf4w2 OpCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=osKS4jvQ0qJ0rbksDoVspyJjwi89uVj7lwPuZCFF4oM=; b=pOv9Yst2MhenAGjxKnia+MwwHRAX6lWXsDgCmZ2zXR8xenf/vhrOADW4C//x+Ecgv8 IKP9oYQh1NopB/EE9nPWSqW0rmGZkxXbJX4rywwqpPSMDhWZqZNv1qOxG18/PdasPWrG Nb8EFFxim6YbYsnQqlVuk4cCT7IV4csALVP8p1dzA3tPkNhS5Lej7VK7a5gK+arrG/xU s7XV9tfpY3sPFoYg6Cg/jV7GdmV7Db6W1ure5X0pR2HMPq6Ad1pqYvyIIw/42G8tBdtm CFDU5vU9NfpkhDurVujMvmYU8TrQW2aSWrwqKyiM6lH7DmZVYXQvE+797W5v1FBF4ucs bgTA== X-Gm-Message-State: APjAAAWH/ilHRXgwSYWyPEM0CNidMPHv0clirmI1dIIX1xhqpbX171jg MaotgtaOYBfLIWGB9m4kJIxZHBKm X-Google-Smtp-Source: APXvYqzlQG+ocNgRakFgkMeXkBi+jLLrWLBCZErY6QZKf3J43fi1Q2gOOeASbQ1/hmuDezKbbHTEIg== X-Received: by 2002:a63:1044:: with SMTP id 4mr1198756pgq.412.1582837086017; Thu, 27 Feb 2020 12:58:06 -0800 (PST) Received: from sc9-mailhost2.vmware.com (c-76-21-95-192.hsd1.ca.comcast.net. [76.21.95.192]) by smtp.gmail.com with ESMTPSA id o9sm8463366pfg.130.2020.02.27.12.58.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 27 Feb 2020 12:58:05 -0800 (PST) From: William Tu To: dev@openvswitch.org Date: Thu, 27 Feb 2020 12:57:18 -0800 Message-Id: <1582837038-31955-1-git-send-email-u9012063@gmail.com> X-Mailer: git-send-email 2.7.4 Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCHv3] netdev-afxdp: Add interrupt mode netdev class. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The patch adds a new netdev class 'afxdp-nonpmd' to enable afxdp interrupt mode. This is similar to 'type=afxdp', except that the is_pmd field is set to false. As a result, the packet processing is handled by main thread, not pmd thread. This avoids burning the CPU to always 100% when there is no traffic. Tested-at: https://travis-ci.org/williamtu/ovs-travis/builds/655885506 Signed-off-by: William Tu --- NEWS | 3 +++ lib/netdev-afxdp.c | 2 +- lib/netdev-linux.c | 20 ++++++++++++++++++++ lib/netdev-provider.h | 1 + lib/netdev.c | 1 + tests/system-afxdp.at | 23 +++++++++++++++++++++++ 6 files changed, 49 insertions(+), 1 deletion(-) diff --git a/NEWS b/NEWS index f62ef1f47ea8..594c55dc11d6 100644 --- a/NEWS +++ b/NEWS @@ -4,6 +4,9 @@ Post-v2.13.0 - OpenFlow: * The OpenFlow ofp_desc/serial_num may now be configured by setting the value of other-config:dp-sn in the Bridge table. + - AF_XDP: + * New netdev class 'afxdp-nonpmd' for netdev-afxdp to save CPU cycles + by enabling interrupt mode. v2.13.0 - 14 Feb 2020 diff --git a/lib/netdev-afxdp.c b/lib/netdev-afxdp.c index 482400d8d135..b806a6344e1e 100644 --- a/lib/netdev-afxdp.c +++ b/lib/netdev-afxdp.c @@ -1115,9 +1115,9 @@ netdev_afxdp_batch_send(struct netdev *netdev, int qid, struct netdev_linux *dev; int ret; + qid = qid % netdev_n_txq(netdev); if (concurrent_txq) { dev = netdev_linux_cast(netdev); - qid = qid % netdev_n_txq(netdev); ovs_spin_lock(&dev->tx_locks[qid].lock); ret = __netdev_afxdp_batch_send(netdev, qid, batch); diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c index c6f3d27409b6..75b3b55d3cad 100644 --- a/lib/netdev-linux.c +++ b/lib/netdev-linux.c @@ -3605,6 +3605,26 @@ const struct netdev_class netdev_afxdp_class = { .rxq_destruct = netdev_afxdp_rxq_destruct, .rxq_recv = netdev_afxdp_rxq_recv, }; + +const struct netdev_class netdev_afxdp_nonpmd_class = { + NETDEV_LINUX_CLASS_COMMON, + .type = "afxdp-nonpmd", + .is_pmd = false, + .init = netdev_afxdp_init, + .construct = netdev_afxdp_construct, + .destruct = netdev_afxdp_destruct, + .get_stats = netdev_afxdp_get_stats, + .get_custom_stats = netdev_afxdp_get_custom_stats, + .get_status = netdev_linux_get_status, + .set_config = netdev_afxdp_set_config, + .get_config = netdev_afxdp_get_config, + .reconfigure = netdev_afxdp_reconfigure, + .get_numa_id = netdev_linux_get_numa_id, + .send = netdev_afxdp_batch_send, + .rxq_construct = netdev_afxdp_rxq_construct, + .rxq_destruct = netdev_afxdp_rxq_destruct, + .rxq_recv = netdev_afxdp_rxq_recv, +}; #endif diff --git a/lib/netdev-provider.h b/lib/netdev-provider.h index 22f4cde3337a..06c1d98a0b07 100644 --- a/lib/netdev-provider.h +++ b/lib/netdev-provider.h @@ -848,6 +848,7 @@ extern const struct netdev_class netdev_tap_class; #ifdef HAVE_AF_XDP extern const struct netdev_class netdev_afxdp_class; +extern const struct netdev_class netdev_afxdp_nonpmd_class; #endif #ifdef __cplusplus } diff --git a/lib/netdev.c b/lib/netdev.c index f95b19af4da0..6d9723ebc3c1 100644 --- a/lib/netdev.c +++ b/lib/netdev.c @@ -154,6 +154,7 @@ netdev_initialize(void) netdev_register_flow_api_provider(&netdev_offload_tc); #ifdef HAVE_AF_XDP netdev_register_provider(&netdev_afxdp_class); + netdev_register_provider(&netdev_afxdp_nonpmd_class); #endif #endif #if defined(__FreeBSD__) || defined(__NetBSD__) diff --git a/tests/system-afxdp.at b/tests/system-afxdp.at index e4451624f882..0d09906fb6c8 100644 --- a/tests/system-afxdp.at +++ b/tests/system-afxdp.at @@ -22,3 +22,26 @@ AT_CHECK([grep "ovs-p0: could not set configuration" ovs-vswitchd.log | wc -l], OVS_TRAFFIC_VSWITCHD_STOP(["/ovs-p0: Too big 'n_rxq'/d /ovs-p0: could not set configuration/d"]) AT_CLEANUP + + +AT_SETUP([AF_XDP - ping between pmd and non-pmd ports]) +AT_KEYWORDS([afxdp nonpmd]) +OVS_TRAFFIC_VSWITCHD_START() + +AT_CHECK([ovs-ofctl add-flow br0 "actions=normal"]) + +ADD_NAMESPACES(at_ns0, at_ns1) +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24") +ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24") + +AT_CHECK([ovs-vsctl del-port ovs-p0]) +AT_CHECK([ovs-vsctl add-port br0 ovs-p0 -- \ + set interface ovs-p0 type=afxdp-nonpmd options:n_rxq=1], + [0], [], [stderr]) + +NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -w 2 10.1.1.2 | FORMAT_PING], [0], [dnl +3 packets transmitted, 3 received, 0% packet loss, time 0ms +]) + +OVS_TRAFFIC_VSWITCHD_STOP +AT_CLEANUP