From patchwork Mon Dec 13 07:37:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sriharsha Basavapatna X-Patchwork-Id: 1567171 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=broadcom.com header.i=@broadcom.com header.a=rsa-sha256 header.s=google header.b=PC8Gf6WN; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JCCyX3FHzz9s1l for ; Mon, 13 Dec 2021 18:37:22 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id E75FC60F0B; Mon, 13 Dec 2021 07:37:19 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8ZKRUUppopjh; Mon, 13 Dec 2021 07:37:19 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 0C13F60EFA; Mon, 13 Dec 2021 07:37:18 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id BF3B5C001E; Mon, 13 Dec 2021 07:37:17 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id CA7CAC0012 for ; Mon, 13 Dec 2021 07:37:16 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id A272385A1B for ; Mon, 13 Dec 2021 07:37:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=broadcom.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hbZLFyzKhlsM for ; Mon, 13 Dec 2021 07:37:15 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by smtp1.osuosl.org (Postfix) with ESMTPS id 216ED85A1A for ; Mon, 13 Dec 2021 07:37:14 +0000 (UTC) Received: by mail-pj1-x102b.google.com with SMTP id p18-20020a17090ad31200b001a78bb52876so14005803pju.3 for ; Sun, 12 Dec 2021 23:37:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:mime-version; bh=g3+D2xTouAQYOzcgQERcUcScK4raADOFE5PjSm5LEe8=; b=PC8Gf6WNio29+1NSLqQOWqbCSqKzbDHrgQUdP8AklPrPqv2IRKvuBcRM0JHLonxRbA H+fCtyhSKrKH5wBfIsGhPdd67qvQMkNAUvXlYgPLpybhqbkpFLlXQ6jwC4FlDnl9ao6i 5Al1pFtNx2DLjf8Zthy+3oZdokmG337JpDkP0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version; bh=g3+D2xTouAQYOzcgQERcUcScK4raADOFE5PjSm5LEe8=; b=4Aoh4exGTyb0n9/Ehz3Pxod7NeCFC1cev1Jb8F2LhGEXbNPJ7aHLg3Jltxcs9hkPA9 60ED3kfE4j2ZN6zlCyzgpZwkYRytnx2MdOjb7qZUbswLhrcJgkoegQmaPAaxBq6bmy1Y MvaNI4UB7f4LYx83rxdHzYheArETRzDlDRBqITvhHz9G+CHC/Bi/sMKZR+1Ozsyn989P txkR8o7Izt29Rv/7siv1G9LvpsN8B44LClxcXzxXrPh4kg7IGlIJUVfvqeLgBrb0jl6d pnCFf7QCAflG0G5LEnLG3OkVx/qUMVwho6V6mLEFlMDoca7ZGXnzjXuKtnVbKV+UZ11L dvSQ== X-Gm-Message-State: AOAM531H0gtKVTAgxcJHDtNhl7ajC8RRcKOBr/N4xUCYCPhgn68LLp3a meiqBWNboBzhhh3g5Xpqnirj6CmnaXH1vuHtlAWK4jYIzlXSrxVmCXa2qeKybmsuH83z/Otxj/B Eu+GfhGTpX4kjL3dD1wpDyQF5+CwC6lCDzQ6V9w9+ZWFLnvzTQJsDaGzPpvSDcYvb0O2s4BQdVB MSZDynIWA= X-Google-Smtp-Source: ABdhPJyRNWdyBzGyDMn9gw2pm/xCtCnta6DFGjVbWTVjk7P88IEtkdanCVK12i+v8gZpuorYltPgwg== X-Received: by 2002:a17:902:9303:b0:143:d6c7:babc with SMTP id bc3-20020a170902930300b00143d6c7babcmr94308283plb.58.1639381033815; Sun, 12 Dec 2021 23:37:13 -0800 (PST) Received: from dhcp-10-123-154-23.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id f7sm11358489pfv.89.2021.12.12.23.37.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Dec 2021 23:37:13 -0800 (PST) To: dev@openvswitch.org Date: Sun, 12 Dec 2021 23:37:06 -0800 Message-Id: <20211213073706.225685-1-sriharsha.basavapatna@broadcom.com> X-Mailer: git-send-email 2.30.0.349.g30b29f044a MIME-Version: 1.0 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [ovs-dev] [PATCH v2] dpif-netdev: avoid hw_miss_packet_recover() for devices with no support X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Sriharsha Basavapatna via dev From: Sriharsha Basavapatna Reply-To: Sriharsha Basavapatna Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The hw_miss_packet_recover() API results in performance degradation, for ports that are either not offload capable or do not support this specific offload API. For example, in the test configuration shown below, the vhost-user port does not support offloads and the VF port doesn't support hw_miss offload API. But because tunnel offload needs to be configured in other bridges (br-vxlan and br-phy), OVS has been built with -DALLOW_EXPERIMENTAL_API. br-vhost br-vxlan br-phy vhost-user<-->VF VF-Rep<-->VxLAN uplink-port For every packet between the VF and the vhost-user ports, hw_miss API is called even though it is not supported by the ports involved. This leads to significant performance drop (~3x in some cases; both cycles and pps). Return EOPNOTSUPP when this API fails for a device that doesn't support it and avoid this API on that port for subsequent packets. Signed-off-by: Sriharsha Basavapatna Acked-by: Ilya Maximets --- v2: Rebased changes to commit: 20a4f546f7db; updated the logic to save 'hw_miss_api_supported' variable in struct dp_netdev_rxq. --- lib/dpif-netdev.c | 18 +++++++++++++----- lib/netdev-offload-dpdk.c | 8 ++++++-- 2 files changed, 19 insertions(+), 7 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index a790df5fd..dbe3f427b 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -377,6 +377,7 @@ struct dp_netdev_rxq { unsigned intrvl_idx; /* Write index for 'cycles_intrvl'. */ struct dp_netdev_pmd_thread *pmd; /* pmd thread that polls this queue. */ bool is_vhost; /* Is rxq of a vhost port. */ + bool hw_miss_api_supported; /* hw_miss_packet_recover() supported */ /* Counters of cycles spent successfully polling and processing pkts. */ atomic_ullong cycles[RXQ_N_CYCLES]; @@ -4790,6 +4791,7 @@ port_reconfigure(struct dp_netdev_port *port) port->rxqs[i].port = port; port->rxqs[i].is_vhost = !strncmp(port->type, "dpdkvhost", 9); + port->rxqs[i].hw_miss_api_supported = true; err = netdev_rxq_open(netdev, &port->rxqs[i].rx, i); if (err) { @@ -7332,12 +7334,18 @@ dp_netdev_hw_flow(const struct dp_netdev_pmd_thread *pmd, #ifdef ALLOW_EXPERIMENTAL_API /* Packet restoration API required. */ /* Restore the packet if HW processing was terminated before completion. */ struct dp_netdev_rxq *rxq = pmd->ctx.last_rxq; - int err; - err = netdev_hw_miss_packet_recover(rxq->port->netdev, packet); - if (err && err != EOPNOTSUPP) { - COVERAGE_INC(datapath_drop_hw_miss_recover); - return -1; + if (rxq->hw_miss_api_supported) { + int err = netdev_hw_miss_packet_recover(rxq->port->netdev, packet); + if (err) { + if (err != EOPNOTSUPP) { + COVERAGE_INC(datapath_drop_hw_miss_recover); + return -1; + } else { + /* API unsupported by the port; avoid subsequent calls. */ + rxq->hw_miss_api_supported = false; + } + } } #endif diff --git a/lib/netdev-offload-dpdk.c b/lib/netdev-offload-dpdk.c index 9fee7570a..b335572cd 100644 --- a/lib/netdev-offload-dpdk.c +++ b/lib/netdev-offload-dpdk.c @@ -2292,11 +2292,15 @@ netdev_offload_dpdk_hw_miss_packet_recover(struct netdev *netdev, odp_port_t vport_odp; int ret = 0; - if (netdev_dpdk_rte_flow_get_restore_info(netdev, packet, - &rte_restore_info, NULL)) { + ret = netdev_dpdk_rte_flow_get_restore_info(netdev, packet, + &rte_restore_info, NULL); + if (ret) { /* This function is called for every packet, and in most cases there * will be no restore info from the HW, thus error is expected. */ + if (ret == -EOPNOTSUPP) { + return -ret; + } return 0; }