{"id":818462,"url":"http://patchwork.ozlabs.org/api/patches/818462/?format=json","web_url":"http://patchwork.ozlabs.org/project/openvswitch/patch/1506404199-23579-7-git-send-email-yliu@fridaylinux.org/","project":{"id":47,"url":"http://patchwork.ozlabs.org/api/projects/47/?format=json","name":"Open vSwitch","link_name":"openvswitch","list_id":"ovs-dev.openvswitch.org","list_email":"ovs-dev@openvswitch.org","web_url":"http://openvswitch.org/","scm_url":"git@github.com:openvswitch/ovs.git","webscm_url":"https://github.com/openvswitch/ovs","list_archive_url":"","list_archive_url_format":"","commit_url_format":""},"msgid":"<1506404199-23579-7-git-send-email-yliu@fridaylinux.org>","list_archive_url":null,"date":"2017-09-26T05:36:36","name":"[ovs-dev,v3,6/9] netdev-dpdk: retry with queue action","commit_ref":null,"pull_url":null,"state":"changes-requested","archived":false,"hash":"4cef99b9c55078773c36c52bfb59dca79adf0a34","submitter":{"id":72215,"url":"http://patchwork.ozlabs.org/api/people/72215/?format=json","name":"Yuanhan Liu","email":"yliu@fridaylinux.org"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/openvswitch/patch/1506404199-23579-7-git-send-email-yliu@fridaylinux.org/mbox/","series":[{"id":5061,"url":"http://patchwork.ozlabs.org/api/series/5061/?format=json","web_url":"http://patchwork.ozlabs.org/project/openvswitch/list/?series=5061","date":"2017-09-26T05:36:30","name":"OVS-DPDK flow offload with rte_flow","version":3,"mbox":"http://patchwork.ozlabs.org/series/5061/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/818462/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/818462/checks/","tags":{},"related":[],"headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=fridaylinux-org.20150623.gappssmtp.com\n\theader.i=@fridaylinux-org.20150623.gappssmtp.com\n\theader.b=\"kDG+XUMK\"; dkim-atps=neutral"],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y1VGQ2QFGz9t30\n\tfor ;\n\tTue, 26 Sep 2017 15:42:38 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id 9E979BFA;\n\tTue, 26 Sep 2017 05:40:21 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id B6E47BD7\n\tfor ; Tue, 26 Sep 2017 05:40:20 +0000 (UTC)","from mail-pf0-f170.google.com (mail-pf0-f170.google.com\n\t[209.85.192.170])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id 46B94D3\n\tfor ; Tue, 26 Sep 2017 05:40:20 +0000 (UTC)","by mail-pf0-f170.google.com with SMTP id d187so5031819pfg.11\n\tfor ; Mon, 25 Sep 2017 22:40:20 -0700 (PDT)","from localhost.localdomain ([101.228.205.132])\n\tby smtp.gmail.com with ESMTPSA id\n\to79sm13180077pfi.108.2017.09.25.22.40.07\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128);\n\tMon, 25 Sep 2017 22:40:17 -0700 (PDT)"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=fridaylinux-org.20150623.gappssmtp.com; s=20150623;\n\th=from:to:cc:subject:date:message-id:in-reply-to:references;\n\tbh=rZzys5P5Qb9qUuQqIYkm3GUlaA9bioDy/ONdg5vGU6M=;\n\tb=kDG+XUMK0fzuVysNSFClYQFVJnCwM1DFgkN3k4gq4pTZYLE6A4hMob2EK3Bq5WFypS\n\t35EXBwiG4GUjBhx3UKkwjgVJjPrtG9TagayeZJ9OeNo0Ikti8HcSAyYsc1qG4ok4v3mq\n\tduegiHu8vAx/qVU4si/qvdqqDevr9hyzSNo3CgNAss5wPW+fao6SUxnh7FRArSfAjXlI\n\tbXHZGFdu2mU7JI1YZ5WxD3bQZTbNK0RK332iZ+ZIkk+zLwQh6zT65GGtmokD1eI9QOz3\n\tN37o35WL4IRuNiBRBkTWypqRrO9eC/YfW2kPj8AvTL3yVBhzqteSVJ0Q3hpHv/vgAV/S\n\tb98Q==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references;\n\tbh=rZzys5P5Qb9qUuQqIYkm3GUlaA9bioDy/ONdg5vGU6M=;\n\tb=S8oVqubYCkwIJF6cqQyoFxR5XprSaz0D+8YVUposkkeh1sPE3uNXqSAribctBZ2uYZ\n\tXwOwvDBQpHo6yUd1Fzq3zHgf12ZrirQhTxeSiNE6DRgkmc61Z8liZWdi35FPuSsYfHWG\n\tATBYEzl/nRfNpJichaHez1gr1c3j9G0Ih0GpRmk685hLOAhVwSFg3FjDtYYBDcedoooo\n\t0//k+HWZ1s+uvNYeDzKTuBxtnr4UlTUz+VGS7ykbpPbCoNlWuTikxQ8Qe2jihGKefqau\n\trW22lvO4g6RH+s4ho6aSbsy6DLFV0vNaXRYECIT62Al6vwIYmHUlT3llyFb7M81kWaFv\n\tlkkw==","X-Gm-Message-State":"AHPjjUhhwSo8r9bt7H12PnAnHjOBW72CWZEALSQ/4YFO+xv9gjnPbL8l\n\tfsUzHQGY1juxnvQtcHBn5IWNaVYonUiTxA==","X-Google-Smtp-Source":"AOwi7QCFl2fhyLvbZr3uZ5pUpHTu6x6gsoIccnGm/9JBl/CxwSP/ACmTdeEAOAbkU0nna0W26zTCvg==","X-Received":"by 10.99.3.208 with SMTP id 199mr10116809pgd.111.1506404419338; \n\tMon, 25 Sep 2017 22:40:19 -0700 (PDT)","From":"Yuanhan Liu ","To":"dev@openvswitch.org","Date":"Tue, 26 Sep 2017 13:36:36 +0800","Message-Id":"<1506404199-23579-7-git-send-email-yliu@fridaylinux.org>","X-Mailer":"git-send-email 2.7.4","In-Reply-To":"<1506404199-23579-1-git-send-email-yliu@fridaylinux.org>","References":"<1506404199-23579-1-git-send-email-yliu@fridaylinux.org>","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"Simon Horman ","Subject":"[ovs-dev] [PATCH v3 6/9] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"},"content":"From: Finn Christensen \n\nAFAIK, most (if not all) NICs (including Mellanox and Intel) do not\nsupport a pure MARK action. It's required to be used together with\nsome other actions, like QUEUE.\n\nTo workaround it, retry with a queue action when first try failed.\n\nMoreover, some Intel's NIC (say XL710) needs the QUEUE action set\nbefore the MARK action.\n\nCo-authored-by: Yuanhan Liu \nSigned-off-by: Finn Christensen \nSigned-off-by: Yuanhan Liu \n---\n\nv3: - retry when 2 specific errors are met\n\nv2: - workarounded the queue action issue, which sets the queue index\n to 0 blindly.\n - added some comments regarding to the retry with queue action\n---\n lib/netdev-dpdk.c | 63 ++++++++++++++++++++++++++++++++++++++++++++++++-------\n 1 file changed, 56 insertions(+), 7 deletions(-)","diff":"diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c\nindex 525536a..3e2b96b 100644\n--- a/lib/netdev-dpdk.c\n+++ b/lib/netdev-dpdk.c\n@@ -3324,6 +3324,7 @@ static struct ovs_mutex ufid_lock = OVS_MUTEX_INITIALIZER;\n struct ufid_dpdk_flow_data {\n struct hmap_node node;\n ovs_u128 ufid;\n+ int rxq;\n struct rte_flow *rte_flow;\n };\n \n@@ -3369,7 +3370,8 @@ del_ufid_dpdk_flow_mapping(const ovs_u128 *ufid)\n \n /* Add ufid to dpdk_flow mapping */\n static inline void\n-add_ufid_dpdk_flow_mapping(const ovs_u128 *ufid, struct rte_flow *rte_flow)\n+add_ufid_dpdk_flow_mapping(const ovs_u128 *ufid, struct rte_flow *rte_flow,\n+ int rxq)\n {\n size_t hash = hash_bytes(ufid, sizeof(*ufid), 0);\n struct ufid_dpdk_flow_data *data = xzalloc(sizeof(*data));\n@@ -3384,6 +3386,7 @@ add_ufid_dpdk_flow_mapping(const ovs_u128 *ufid, struct rte_flow *rte_flow)\n \n data->ufid = *ufid;\n data->rte_flow = rte_flow;\n+ data->rxq = rxq;\n \n ovs_mutex_lock(&ufid_lock);\n hmap_insert(&ufid_dpdk_flow, &data->node, hash);\n@@ -3665,15 +3668,55 @@ netdev_dpdk_add_rte_flow_offload(struct netdev *netdev,\n add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_MARK, &mark);\n add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_END, NULL);\n \n+ bool tried = 0;\n+ struct rte_flow_action_queue queue;\n+again:\n flow = rte_flow_create(dev->port_id, &flow_attr, patterns.items,\n actions.actions, &error);\n- if (!flow) {\n+ if (!flow && !tried &&\n+ (error.type == RTE_FLOW_ERROR_TYPE_ACTION ||\n+ error.type == RTE_FLOW_ERROR_TYPE_HANDLE)) {\n+ /*\n+ * Seems most (if not all) NICs do not support a pure MARK\n+ * action. It's required to be used together with some other\n+ * actions, such as QUEUE.\n+ *\n+ * Thus, if flow creation fails (more precisely, if above 2\n+ * errors are met), here we try again with QUEUE action.\n+ *\n+ * The reason why there are 2 error codes is DPDK drivers are\n+ * not quite consistent on error report for this case.\n+ */\n+ if (info->rxq < 0 || info->rxq >= netdev->n_rxq) {\n+ VLOG_ERR(\"invalid queue index: %d\\n\", info->rxq);\n+ ret = -1;\n+ goto out;\n+ }\n+ queue.index = info->rxq;\n+\n+ /* re-build the action */\n+ actions.cnt = 0;\n+ /*\n+ * NOTE: Intel PMD driver needs the QUEUE action set before\n+ * the MARK action.\n+ */\n+ add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_QUEUE, &queue);\n+ add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_MARK, &mark);\n+ add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_END, NULL);\n+\n+ VLOG_INFO(\"failed to create rte flow, \"\n+ \"try again with QUEUE action (queue index = %d)\\n\",\n+ info->rxq);\n+ tried = true;\n+\n+ goto again;\n+ } else if (!flow) {\n VLOG_ERR(\"rte flow creat error: %u : message : %s\\n\",\n error.type, error.message);\n ret = -1;\n goto out;\n }\n- add_ufid_dpdk_flow_mapping(ufid, flow);\n+ add_ufid_dpdk_flow_mapping(ufid, flow, info->rxq);\n VLOG_INFO(\"installed flow %p by ufid \"UUID_FMT\"\\n\",\n flow, UUID_ARGS((struct uuid *)ufid));\n \n@@ -3807,17 +3850,23 @@ netdev_dpdk_flow_put(struct netdev *netdev, struct match *match,\n const ovs_u128 *ufid, struct offload_info *info,\n struct dpif_flow_stats *stats OVS_UNUSED)\n {\n- struct rte_flow *rte_flow;\n+ struct ufid_dpdk_flow_data *flow_data;\n int ret;\n \n /*\n * If an old rte_flow exists, it means it's a flow modification.\n * Here destroy the old rte flow first before adding a new one.\n */\n- rte_flow = get_rte_flow_by_ufid(ufid);\n- if (rte_flow) {\n+ flow_data = find_ufid_dpdk_flow_mapping(ufid);\n+ if (flow_data && flow_data->rte_flow) {\n+ /*\n+ * there is no rxq given for flow modification. Instead, we\n+ * retrieve it from the rxq firstly registered here (by flow\n+ * add operation).\n+ */\n+ info->rxq = flow_data->rxq;\n ret = netdev_dpdk_destroy_rte_flow(netdev_dpdk_cast(netdev),\n- ufid, rte_flow);\n+ ufid, flow_data->rte_flow);\n if (ret < 0) {\n return ret;\n }\n","prefixes":["ovs-dev","v3","6/9"]}