[{"id":1765515,"web_url":"http://patchwork.ozlabs.org/comment/1765515/","msgid":"<20170908164849.GH7356@vergenet.net>","list_archive_url":null,"date":"2017-09-08T16:48:50","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":64714,"url":"http://patchwork.ozlabs.org/api/people/64714/","name":"Simon Horman","email":"simon.horman@netronome.com"},"content":"On Tue, Sep 05, 2017 at 05:22:59PM +0800, Yuanhan Liu wrote:\n> From: Finn Christensen \n> \n> AFAIK, most (if not all) NICs (including Mellanox and Intel) do not\n> support a pure MARK action. It's required to be used together with\n> some other actions, like QUEUE.\n> \n> To workaround it, retry with a queue action when first try failed.\n> \n> Moreover, some Intel's NIC (say XL710) needs the QUEUE action set\n> before the MARK action.\n> \n> Co-authored-by: Yuanhan Liu \n> Signed-off-by: Finn Christensen \n> Signed-off-by: Yuanhan Liu \n\nThis feels a bit like the tail wagging the dog.\nIs this the lowest level at which it makes sense to implement\nthis logic?\n\nIf so then I wonder if some sort of probing would be in order\nto avoid the cost of trying to add the flow twice to hardware\nwhere the queue is required.\n\n...","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=netronome-com.20150623.gappssmtp.com\n\theader.i=@netronome-com.20150623.gappssmtp.com\n\theader.b=\"FpPcqU9v\"; dkim-atps=neutral"],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xpjxz0V7Rz9s7h\n\tfor ;\n\tSat, 9 Sep 2017 02:51:03 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id 41619BEA;\n\tFri, 8 Sep 2017 16:48:53 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 9DB45BE9\n\tfor ; Fri, 8 Sep 2017 16:48:51 +0000 (UTC)","from mail-wm0-f41.google.com (mail-wm0-f41.google.com\n\t[74.125.82.41])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id 02B0446C\n\tfor ; Fri, 8 Sep 2017 16:48:50 +0000 (UTC)","by mail-wm0-f41.google.com with SMTP id f145so7352352wme.0\n\tfor ; Fri, 08 Sep 2017 09:48:50 -0700 (PDT)","from vergenet.net (53.red-80-24-208.staticip.rima-tde.net.\n\t[80.24.208.53]) by smtp.gmail.com with ESMTPSA id\n\tw6sm1638071wra.45.2017.09.08.09.48.48\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tFri, 08 Sep 2017 09:48:49 -0700 (PDT)"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=netronome-com.20150623.gappssmtp.com; s=20150623;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=KWH7oLhyrtgzYNDFjUeefr1fgADA9xHcJUF0tkBCgO4=;\n\tb=FpPcqU9vY258oxSTII6x9A5KixgZC5ZHirqG5MhE8Cj7Zgk/Lu8eQdQhBzxpGQMo8j\n\tt1dIAtK7+dK1okp2q33fFDSQ/I4sfUAbfKZSI6aDtiVZ32vScqmq4GT0JxeVrwM5apby\n\tq17oSVpnqjJfaMcWcc+hlrvKm8xaO7yz7dli7I0CMuE6dzj8o6PiLMhMO05pkNDSLGVs\n\t12vc7Z5v9XpzkY2jMaXXm2ioMqEe1jF773164l979NKBB3SAkl66kd8mLGd30qLly8Kn\n\t1vzrZIdbt5lFj/W6fJgQLmZ1zt+7shKnrbZ95G3qM9MyM1kQC2JwQw+h6rh68s15/Uba\n\thfzA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=KWH7oLhyrtgzYNDFjUeefr1fgADA9xHcJUF0tkBCgO4=;\n\tb=MsqBjBtUNWc2R/yUKEmNQK0nACGj74v7+VLfCADwjnpBg63HbOB4rec65jNczExfh9\n\tBN8pSjE2wLAS44IQb0aoRoatUDY1zpzUitkt68oBx0wQaGwR+2tXGwox0vBn9X92P2q6\n\tZAaw/s7r0PAKDiJEtJAlpCGueSoU4sr0cU41qb14RtxrO4WnHSXuC5MF4Gnq2ANAINTp\n\t69+uer0mnEhH1hcNx2Gqk4wuFjW0rd5mfAUoGMjtqVTawe9bVdvE97PHvKf/y/RuRP/n\n\tKkW8ZkjpTFLa3OV77FIznk1kELlmsMIEvgGiJlI54IAn9QZGrIJo5GQ6ItaUaTfwoskl\n\tIGXw==","X-Gm-Message-State":"AHPjjUi8bu8gLGwfiAldyfTskZeUqxzjCHso6YY5BrLHZ6ptkEc1XUQG\n\tHt9pf2Gj9vHHFe1v","X-Google-Smtp-Source":"AOwi7QBN+ToZuQVmRD0KUc8sFZ5KHBxm9itgGHNd1ELjuOQEHFKaMBYyEZvrsZB/rVhZusDGxIJeFw==","X-Received":"by 10.28.19.76 with SMTP id 73mr2192413wmt.122.1504889329660;\n\tFri, 08 Sep 2017 09:48:49 -0700 (PDT)","Date":"Fri, 8 Sep 2017 18:48:50 +0200","From":"Simon Horman ","To":"Yuanhan Liu ","Message-ID":"<20170908164849.GH7356@vergenet.net>","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>","User-Agent":"Mutt/1.5.23 (2014-03-12)","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"dev@openvswitch.org","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1766054,"web_url":"http://patchwork.ozlabs.org/comment/1766054/","msgid":"<20170911061425.GN9736@yliu-home>","list_archive_url":null,"date":"2017-09-11T06:14:25","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":72215,"url":"http://patchwork.ozlabs.org/api/people/72215/","name":"Yuanhan Liu","email":"yliu@fridaylinux.org"},"content":"On Fri, Sep 08, 2017 at 06:48:50PM +0200, Simon Horman wrote:\n> On Tue, Sep 05, 2017 at 05:22:59PM +0800, Yuanhan Liu wrote:\n> > From: Finn Christensen \n> > \n> > AFAIK, most (if not all) NICs (including Mellanox and Intel) do not\n> > support a pure MARK action. It's required to be used together with\n> > some other actions, like QUEUE.\n> > \n> > To workaround it, retry with a queue action when first try failed.\n> > \n> > Moreover, some Intel's NIC (say XL710) needs the QUEUE action set\n> > before the MARK action.\n> > \n> > Co-authored-by: Yuanhan Liu \n> > Signed-off-by: Finn Christensen \n> > Signed-off-by: Yuanhan Liu \n> \n> This feels a bit like the tail wagging the dog.\n> Is this the lowest level at which it makes sense to implement\n> this logic?\n> \n> If so then I wonder if some sort of probing would be in order\n> to avoid the cost of trying to add the flow twice to hardware\n> where the queue is required.\n\nDo you mean something like rte_flow capability query, like whether\na queue action is needed for a mark action? If so, yes, I do think\nwe miss an interface like this.\n\nNote that even in this solution, the flow won't be created twice\nto the hardware, because the first try would be failed.\n\n\t--yliu","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=fridaylinux-org.20150623.gappssmtp.com\n\theader.i=@fridaylinux-org.20150623.gappssmtp.com\n\theader.b=\"HTl0B8+n\"; dkim-atps=neutral"],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xrHhK5HZ0z9s76\n\tfor ;\n\tMon, 11 Sep 2017 16:14:41 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id 824A6AB9;\n\tMon, 11 Sep 2017 06:14:38 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 5523EA92\n\tfor ; Mon, 11 Sep 2017 06:14:37 +0000 (UTC)","from mail-pg0-f48.google.com (mail-pg0-f48.google.com\n\t[74.125.83.48])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id F1E66E7\n\tfor ; Mon, 11 Sep 2017 06:14:36 +0000 (UTC)","by mail-pg0-f48.google.com with SMTP id v66so13670491pgb.5\n\tfor ; Sun, 10 Sep 2017 23:14:36 -0700 (PDT)","from yliu-home ([45.63.61.64]) by smtp.gmail.com with ESMTPSA id\n\tt81sm14191824pfg.154.2017.09.10.23.14.33\n\t(version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128);\n\tSun, 10 Sep 2017 23:14:35 -0700 (PDT)"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=fridaylinux-org.20150623.gappssmtp.com; s=20150623;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=bDmvu7Cq5OFIGY894p876CUqPKqzcZfVtKjThRw50wk=;\n\tb=HTl0B8+n1+VrkkkLvCwkV3YTGBC9D/W2Fnotafyqo5ynna1QTEw4HHJvDY9pirY05u\n\tdqXWsa3Q8DugbvFEiTLf0AwS1vTk2VHb5HFZFLI7kPgaZoV0NMfvx0R4d0pW+rUSOU2v\n\tUqkmcbOO0xrBVfng3iQBEBacRhXNC/LyL79bfvLeHcIU0mMH/zafaCB2OahjPsbpt0VD\n\tQASBmRi7tSjGAjQZosNAffrp0DM7lRVIKIpztS9zv+QgETY/ToRHylfSoNaS8HZJkFMq\n\tqeBCJKbCdA/3Qbv+E+250Nw8uvNdaH360PKuI6r1ttqrd0kn0J8ipoRsAl0ULhDxkFi4\n\t7o7w==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=bDmvu7Cq5OFIGY894p876CUqPKqzcZfVtKjThRw50wk=;\n\tb=H/jFQ5eZ9V+dEh9rm2J0WlP8rh7/Do+46ncaGaLY5n734X0vMcVjK0zu50Rm50gmE/\n\t9HZgbSJY+THvkUtyZh5b4zZ2G9zdWWlF6LL0dyxaiS1Hzc5pWXHqZR6a+XpZXFMJ9NKE\n\t17Fc2AKBXiL0mbrQo/oENSBFwMig4mA2uP8YpYamZ+NUB76pz99D9zr97FfvM9+ELVA9\n\tlaL58q1mLRmeoJZGqJJPiha7EvJvWzTEm1w5PXfpPzx3dQHN4WBhYnhmVQcNayw+HTw6\n\tvR4YYL2HomKyVVapsd1A7W9B0f/B1lC6szLVKku9DdcklFZPZB3cAlj1zQYEAlDM+l32\n\t3c0g==","X-Gm-Message-State":"AHPjjUg/d3D/hguFzobSucp6sXvqDa0DTb3wBIsYGNxbYHBz4l+Ax0MC\n\txekNxHshDQxP/zo/","X-Google-Smtp-Source":"ADKCNb5SNjwFCwO6KR3qLZK2vGPxnw1VNTYgUJV/iicegMrSTEXra9d2NIY+xEq5LcwpRQxpyPynMQ==","X-Received":"by 10.98.93.25 with SMTP id r25mr10774539pfb.252.1505110476581; \n\tSun, 10 Sep 2017 23:14:36 -0700 (PDT)","Date":"Mon, 11 Sep 2017 14:14:25 +0800","From":"Yuanhan Liu ","To":"Simon Horman ","Message-ID":"<20170911061425.GN9736@yliu-home>","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<20170908164849.GH7356@vergenet.net>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20170908164849.GH7356@vergenet.net>","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"dev@openvswitch.org","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1766102,"web_url":"http://patchwork.ozlabs.org/comment/1766102/","msgid":"<2EF2F5C0CC56984AA024D0B180335FCB4221DE59@IRSMSX102.ger.corp.intel.com>","list_archive_url":null,"date":"2017-09-11T07:42:19","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":67440,"url":"http://patchwork.ozlabs.org/api/people/67440/","name":"Chandran, Sugesh","email":"sugesh.chandran@intel.com"},"content":"Regards\n_Sugesh\n\n\n> -----Original Message-----\n> From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev-\n> bounces@openvswitch.org] On Behalf Of Yuanhan Liu\n> Sent: Tuesday, September 5, 2017 10:23 AM\n> To: dev@openvswitch.org\n> Subject: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action\n> \n> From: Finn Christensen \n> \n> AFAIK, most (if not all) NICs (including Mellanox and Intel) do not support a\n> pure MARK action. It's required to be used together with some other\n> actions, like QUEUE.\n> \n> To workaround it, retry with a queue action when first try failed.\n[Sugesh] This means a NIC that doesn't support pure MARK, will\ndo every flow install twice no matter how many times the flow install failed before for the same reason. \nThe retry can be avoided if we know what a port/NIC can support. It leads to a need of feature discovery of a NIC\nat the init time.\n> \n> Moreover, some Intel's NIC (say XL710) needs the QUEUE action set before\n> the MARK action.\n> \n> Co-authored-by: Yuanhan Liu \n> Signed-off-by: Finn Christensen \n> Signed-off-by: Yuanhan Liu \n> ---\n> \n> v2: - workarounded the queue action issue, which sets the queue index\n> to 0 blindly.\n> - added some comments regarding to the retry with queue action\n> ---\n> lib/netdev-dpdk.c | 59\n> ++++++++++++++++++++++++++++++++++++++++++++++++-------\n> 1 file changed, 52 insertions(+), 7 deletions(-)\n> \n> diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 37b0f99..320fe80\n> 100644\n> --- a/lib/netdev-dpdk.c\n> +++ b/lib/netdev-dpdk.c\n> @@ -3324,6 +3324,7 @@ static struct ovs_mutex ufid_lock =\n> OVS_MUTEX_INITIALIZER; struct ufid_dpdk_flow_data {\n> struct hmap_node node;\n> ovs_u128 ufid;\n> + int rxq;\n> struct rte_flow *rte_flow;\n> };\n> \n> @@ -3366,7 +3367,8 @@ del_ufid_dpdk_flow_mapping(const ovs_u128\n> *ufid)\n> \n> /* Add ufid to dpdk_flow mapping */\n> static inline void\n> -add_ufid_dpdk_flow_mapping(const ovs_u128 *ufid, struct rte_flow\n> *rte_flow)\n> +add_ufid_dpdk_flow_mapping(const ovs_u128 *ufid, struct rte_flow\n> *rte_flow,\n> + int rxq)\n> {\n> size_t hash = hash_bytes(ufid, sizeof(*ufid), 0);\n> struct ufid_dpdk_flow_data *data = xzalloc(sizeof(*data)); @@ -3381,6\n> +3383,7 @@ add_ufid_dpdk_flow_mapping(const ovs_u128 *ufid, struct\n> rte_flow *rte_flow)\n> \n> data->ufid = *ufid;\n> data->rte_flow = rte_flow;\n> + data->rxq = rxq;\n> \n> ovs_mutex_lock(&ufid_lock);\n> hmap_insert(&ufid_dpdk_flow, &data->node, hash); @@ -3664,15\n> +3667,51 @@ netdev_dpdk_add_rte_flow_offload(struct netdev *netdev,\n> }\n> add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_END, NULL);\n> \n> + bool tried = 0;\n> + struct rte_flow_action_queue queue;\n> +again:\n> flow = rte_flow_create(dev->port_id, &flow_attr, patterns.items,\n> actions.actions, &error);\n> - if (!flow) {\n> + if (!flow && !tried && actions_len) {\n> + /*\n> + * Seems most (if not all) NICs do not support a pure MARK\n> + * action. It's required to be used together with some other\n> + * actions, such as QUEUE.\n> + *\n> + * Thus, if flow creation fails, here we try again with\n> + * QUEUE action.\n> + *\n> + */\n> + if (info->rxq < 0 || info->rxq >= netdev->n_rxq) {\n> + VLOG_ERR(\"invalid queue index: %d\\n\", info->rxq);\n> + ret = -1;\n> + goto out;\n> + }\n> + queue.index = info->rxq;\n> +\n> + /* re-build the action */\n> + actions.cnt = 0;\n> + /*\n> + * NOTE: Intel PMD driver needs the QUEUE action set before\n> + * the MARK action.\n> + */\n> + add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_QUEUE,\n> &queue);\n> + add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_MARK, &mark);\n> + add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_END, NULL);\n> +\n> + VLOG_INFO(\"failed to create rte flow, \"\n> + \"try again with QUEUE action (queue index = %d)\\n\",\n> + info->rxq);\n> + tried = true;\n> +\n> + goto again;\n> + } else if (!flow) {\n> VLOG_ERR(\"rte flow creat error: %u : message : %s\\n\",\n> error.type, error.message);\n> ret = -1;\n> goto out;\n> }\n> - add_ufid_dpdk_flow_mapping(ufid, flow);\n> + add_ufid_dpdk_flow_mapping(ufid, flow, info->rxq);\n> VLOG_INFO(\"installed flow %p by ufid \"UUID_FMT\"\\n\",\n> flow, UUID_ARGS((struct uuid *)ufid));\n> \n> @@ -3784,17 +3823,23 @@ netdev_dpdk_flow_put(struct netdev *netdev,\n> struct match *match,\n> const ovs_u128 *ufid, struct offload_info *info,\n> struct dpif_flow_stats *stats OVS_UNUSED) {\n> - struct rte_flow *rte_flow;\n> + struct ufid_dpdk_flow_data *flow_data;\n> int ret;\n> \n> /*\n> * If an old rte_flow exists, it means it's a flow modification.\n> * Here destroy the old rte flow first before adding a new one.\n> */\n> - rte_flow = get_rte_flow_by_ufid(ufid);\n> - if (rte_flow) {\n> + flow_data = find_ufid_dpdk_flow_mapping(ufid);\n> + if (flow_data && flow_data->rte_flow) {\n> + /*\n> + * there is no rxq given for flow modification. Instead, we\n> + * retrieve it from the rxq firstly registered here (by flow\n> + * add operation).\n> + */\n> + info->rxq = flow_data->rxq;\n> ret = netdev_dpdk_destroy_rte_flow(netdev_dpdk_cast(netdev),\n> - ufid, rte_flow);\n> + ufid, flow_data->rte_flow);\n> if (ret < 0)\n> return ret;\n> }\n> --\n> 2.7.4\n> \n> _______________________________________________\n> dev mailing list\n> dev@openvswitch.org\n> https://mail.openvswitch.org/mailman/listinfo/ovs-dev","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xrKdf4djqz9s7F\n\tfor ;\n\tMon, 11 Sep 2017 17:42:28 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id 482DE87A;\n\tMon, 11 Sep 2017 07:42:24 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 586405A7\n\tfor ; Mon, 11 Sep 2017 07:42:23 +0000 (UTC)","from mga07.intel.com (mga07.intel.com [134.134.136.100])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id ACFFE159\n\tfor ; Mon, 11 Sep 2017 07:42:22 +0000 (UTC)","from fmsmga001.fm.intel.com ([10.253.24.23])\n\tby orsmga105.jf.intel.com with ESMTP; 11 Sep 2017 00:42:21 -0700","from irsmsx109.ger.corp.intel.com ([163.33.3.23])\n\tby fmsmga001.fm.intel.com with ESMTP; 11 Sep 2017 00:42:20 -0700","from irsmsx102.ger.corp.intel.com ([169.254.2.59]) by\n\tIRSMSX109.ger.corp.intel.com ([169.254.13.28]) with mapi id\n\t14.03.0319.002; Mon, 11 Sep 2017 08:42:19 +0100"],"X-Greylist":"domain auto-whitelisted by SQLgrey-1.7.6","X-ExtLoop1":"1","X-IronPort-AV":"E=Sophos; i=\"5.42,376,1500966000\"; d=\"scan'208\";\n\ta=\"1193684034\"","From":"\"Chandran, Sugesh\" ","To":"'Yuanhan Liu' , \"dev@openvswitch.org\"\n\t","Thread-Topic":"[ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","Thread-Index":"AQHTJilLVRUwf8GQ/0SDIXf1r0WzAKKuKMYg","Date":"Mon, 11 Sep 2017 07:42:19 +0000","Message-ID":"<2EF2F5C0CC56984AA024D0B180335FCB4221DE59@IRSMSX102.ger.corp.intel.com>","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>","In-Reply-To":"<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>","Accept-Language":"en-US","Content-Language":"en-US","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","x-titus-metadata-40":"eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiYTMwYzExMDQtMWJiMy00ODkwLWFiZWUtY2Q5Y2JjN2RlM2Q5IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6ImFMT1lycnpDM2VwRHBJSUdjZWRxMVZDQjVKY3M4XC9kYk1ZMTVMUzJKOTY0PSJ9","x-ctpclassification":"CTP_IC","dlp-product":"dlpe-windows","dlp-version":"11.0.0.116","dlp-reaction":"no-action","x-originating-ip":"[163.33.239.181]","MIME-Version":"1.0","X-Spam-Status":"No, score=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED,\n\tRP_MATCHES_RCVD autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1766115,"web_url":"http://patchwork.ozlabs.org/comment/1766115/","msgid":"<20170911075552.GS9736@yliu-home>","list_archive_url":null,"date":"2017-09-11T07:55:52","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":72215,"url":"http://patchwork.ozlabs.org/api/people/72215/","name":"Yuanhan Liu","email":"yliu@fridaylinux.org"},"content":"On Mon, Sep 11, 2017 at 07:42:19AM +0000, Chandran, Sugesh wrote:\n> \n> \n> Regards\n> _Sugesh\n> \n> \n> > -----Original Message-----\n> > From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev-\n> > bounces@openvswitch.org] On Behalf Of Yuanhan Liu\n> > Sent: Tuesday, September 5, 2017 10:23 AM\n> > To: dev@openvswitch.org\n> > Subject: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action\n> > \n> > From: Finn Christensen \n> > \n> > AFAIK, most (if not all) NICs (including Mellanox and Intel) do not support a\n> > pure MARK action. It's required to be used together with some other\n> > actions, like QUEUE.\n> > \n> > To workaround it, retry with a queue action when first try failed.\n> [Sugesh] This means a NIC that doesn't support pure MARK, will\n> do every flow install twice no matter how many times the flow install failed before for the same reason. \n> The retry can be avoided if we know what a port/NIC can support. It leads to a need of feature discovery of a NIC\n> at the init time.\n\nI believe the reply I made to Simon answered all questions you asked above.\n\n\t--yliu","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=fridaylinux-org.20150623.gappssmtp.com\n\theader.i=@fridaylinux-org.20150623.gappssmtp.com\n\theader.b=\"nKClC80+\"; dkim-atps=neutral"],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xrKxf1Bjgz9s81\n\tfor ;\n\tMon, 11 Sep 2017 17:56:22 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id A1354B35;\n\tMon, 11 Sep 2017 07:56:03 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 32F31B1F\n\tfor ; Mon, 11 Sep 2017 07:56:02 +0000 (UTC)","from mail-pg0-f51.google.com (mail-pg0-f51.google.com\n\t[74.125.83.51])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id C6A661BB\n\tfor ; Mon, 11 Sep 2017 07:56:01 +0000 (UTC)","by mail-pg0-f51.google.com with SMTP id d8so14022558pgt.4\n\tfor ; Mon, 11 Sep 2017 00:56:01 -0700 (PDT)","from yliu-home ([45.63.61.64]) by smtp.gmail.com with ESMTPSA id\n\th192sm11942280pgc.36.2017.09.11.00.55.58\n\t(version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128);\n\tMon, 11 Sep 2017 00:56:00 -0700 (PDT)"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=fridaylinux-org.20150623.gappssmtp.com; s=20150623;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=BNgkpj3W8uvJYR9ZcK11d/JOGYXuViVYSaoK9LL6IMU=;\n\tb=nKClC80+I5wmKniO0yyLKmOc5heZY0U25mQiS7Uhdot35szT+3iXZHmxp+LUM24iaP\n\t7jQ51sepuFTS/eEQwwL5x/OMlqsPb3zjSBnf8cV//GpjS+PzW/3XoBcObAVwdHEqIEvY\n\toiCWsNVLYVVoYwup7nFA85v9pDYp/JKjBvaD7IRjQY2ajc2vSnBH6rDuqtB65dcR2MfY\n\tV3RYJbSlQc0uQ/G8P00lX/WVSVkHutHsVOMQXbUZNTxJnYJrrf4k9upgTWwH7PqGEQOY\n\tKSgAqB2lJVG/s2CuxkuHK+hwK82fxgBCbRv8YufFbMP3zpY0jHh3Fcb+aFiU5lwQ7DGI\n\tryoQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=BNgkpj3W8uvJYR9ZcK11d/JOGYXuViVYSaoK9LL6IMU=;\n\tb=fT8PCNbHnox/Aot2VSAFQAwh7Jq5q7YeCrT4Va74LFCgGewcxVCTaZU2Dh8J++WDzE\n\tnJOjrFrCWVlYdvnofj2gcXSVDKOu76nOYu+WfL9iX4lqPUYCmTTikitYZeCxMHCf1mme\n\tjswZACHBw/aXMv3XCzO+wxpI51NBFHO0NfjURmuYiE/wF5WO+MPmzFTlHk/IOdHfgBFf\n\tqPkr+C1rl0ohFSm6cOjt/V6H1+YaYvf4WN8MRZQkfq9oulEZoiHyvO+3QWEzSoYv7IUL\n\tyDdPhg6qrQTiPlmoKcRWeHkc7RspoJwgXqkplalY4hL/Loa7FgGWNYESlJEkwF0UlWhO\n\tW0fw==","X-Gm-Message-State":"AHPjjUgfQKrkCeAgyPx95Ad8zAxBNf0VA0/jxK1/5FlsYIz/P8w1xjPv\n\tZHSFFDiCPKqY7wXZjsAzLg==","X-Google-Smtp-Source":"ADKCNb5kRPUi/JFMIBfmO246EYuUhp7cf0BzqDoklT9FPEwt1xwS/JdZ+D8rPEFip5JYwZVh2j4inQ==","X-Received":"by 10.84.233.10 with SMTP id j10mr12761069plk.134.1505116561364; \n\tMon, 11 Sep 2017 00:56:01 -0700 (PDT)","Date":"Mon, 11 Sep 2017 15:55:52 +0800","From":"Yuanhan Liu ","To":"\"Chandran, Sugesh\" ","Message-ID":"<20170911075552.GS9736@yliu-home>","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<2EF2F5C0CC56984AA024D0B180335FCB4221DE59@IRSMSX102.ger.corp.intel.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<2EF2F5C0CC56984AA024D0B180335FCB4221DE59@IRSMSX102.ger.corp.intel.com>","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-Spam-Status":"No, score=0.5 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE,\n\tRCVD_IN_SORBS_SPAM autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1766136,"web_url":"http://patchwork.ozlabs.org/comment/1766136/","msgid":"<2EF2F5C0CC56984AA024D0B180335FCB4221DF80@IRSMSX102.ger.corp.intel.com>","list_archive_url":null,"date":"2017-09-11T08:55:19","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":67440,"url":"http://patchwork.ozlabs.org/api/people/67440/","name":"Chandran, Sugesh","email":"sugesh.chandran@intel.com"},"content":"Regards\n_Sugesh\n\n\n> -----Original Message-----\n> From: Yuanhan Liu [mailto:yliu@fridaylinux.org]\n> Sent: Monday, September 11, 2017 8:56 AM\n> To: Chandran, Sugesh \n> Cc: dev@openvswitch.org\n> Subject: Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action\n> \n> On Mon, Sep 11, 2017 at 07:42:19AM +0000, Chandran, Sugesh wrote:\n> >\n> >\n> > Regards\n> > _Sugesh\n> >\n> >\n> > > -----Original Message-----\n> > > From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev-\n> > > bounces@openvswitch.org] On Behalf Of Yuanhan Liu\n> > > Sent: Tuesday, September 5, 2017 10:23 AM\n> > > To: dev@openvswitch.org\n> > > Subject: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue\n> > > action\n> > >\n> > > From: Finn Christensen \n> > >\n> > > AFAIK, most (if not all) NICs (including Mellanox and Intel) do not\n> > > support a pure MARK action. It's required to be used together with\n> > > some other actions, like QUEUE.\n> > >\n> > > To workaround it, retry with a queue action when first try failed.\n> > [Sugesh] This means a NIC that doesn't support pure MARK, will do\n> > every flow install twice no matter how many times the flow install failed\n> before for the same reason.\n> > The retry can be avoided if we know what a port/NIC can support. It\n> > leads to a need of feature discovery of a NIC at the init time.\n> \n> I believe the reply I made to Simon answered all questions you asked above.\n[Sugesh] Apologies , as I didn't see some of comments and its replies.\nThank you Yuanhan. Hope you will be working on some probing mechanism.\nI am interested to know how its going to looks like\n> \n> \t--yliu","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xrMGQ72Xxz9s7g\n\tfor ;\n\tMon, 11 Sep 2017 18:55:58 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id 7F5FAB55;\n\tMon, 11 Sep 2017 08:55:57 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 57133B30\n\tfor ; Mon, 11 Sep 2017 08:55:56 +0000 (UTC)","from mga04.intel.com (mga04.intel.com [192.55.52.120])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id CC3D21CE\n\tfor ; Mon, 11 Sep 2017 08:55:55 +0000 (UTC)","from fmsmga004.fm.intel.com ([10.253.24.48])\n\tby fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t11 Sep 2017 01:55:55 -0700","from irsmsx103.ger.corp.intel.com ([163.33.3.157])\n\tby fmsmga004.fm.intel.com with ESMTP; 11 Sep 2017 01:55:53 -0700","from irsmsx111.ger.corp.intel.com (10.108.20.4) by\n\tIRSMSX103.ger.corp.intel.com (163.33.3.157) with Microsoft SMTP\n\tServer (TLS) id 14.3.319.2; Mon, 11 Sep 2017 09:55:20 +0100","from irsmsx102.ger.corp.intel.com ([169.254.2.59]) by\n\tirsmsx111.ger.corp.intel.com ([169.254.2.30]) with mapi id\n\t14.03.0319.002; Mon, 11 Sep 2017 09:55:20 +0100"],"X-Greylist":"domain auto-whitelisted by SQLgrey-1.7.6","X-ExtLoop1":"1","X-IronPort-AV":"E=Sophos;i=\"5.42,376,1500966000\"; d=\"scan'208\";a=\"310237267\"","From":"\"Chandran, Sugesh\" ","To":"Yuanhan Liu ","Thread-Topic":"[ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","Thread-Index":"AQHTJilLVRUwf8GQ/0SDIXf1r0WzAKKuKMYggAEg6gCAACDZoA==","Date":"Mon, 11 Sep 2017 08:55:19 +0000","Message-ID":"<2EF2F5C0CC56984AA024D0B180335FCB4221DF80@IRSMSX102.ger.corp.intel.com>","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<2EF2F5C0CC56984AA024D0B180335FCB4221DE59@IRSMSX102.ger.corp.intel.com>\n\t<20170911075552.GS9736@yliu-home>","In-Reply-To":"<20170911075552.GS9736@yliu-home>","Accept-Language":"en-US","Content-Language":"en-US","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","x-titus-metadata-40":"eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMmEyZDg3NzQtY2ZjYS00YjQ0LWJmNTAtMTE1OGNiZTQ0ZGNjIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6IkUzdG9TVWtWSkpsTFc0XC9GNVZHY25MeUhZUWNTNW5VNmV2bzdUSm1BcGZRPSJ9","x-ctpclassification":"CTP_IC","dlp-product":"dlpe-windows","dlp-version":"11.0.0.116","dlp-reaction":"no-action","x-originating-ip":"[163.33.239.181]","MIME-Version":"1.0","X-Spam-Status":"No, score=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED,\n\tRP_MATCHES_RCVD autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1766140,"web_url":"http://patchwork.ozlabs.org/comment/1766140/","msgid":"<2EF2F5C0CC56984AA024D0B180335FCB4221DFA4@IRSMSX102.ger.corp.intel.com>","list_archive_url":null,"date":"2017-09-11T08:56:54","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":67440,"url":"http://patchwork.ozlabs.org/api/people/67440/","name":"Chandran, Sugesh","email":"sugesh.chandran@intel.com"},"content":"Regards\n_Sugesh\n\n\n> -----Original Message-----\n> From: Yuanhan Liu [mailto:yliu@fridaylinux.org]\n> Sent: Monday, September 11, 2017 8:56 AM\n> To: Chandran, Sugesh \n> Cc: dev@openvswitch.org\n> Subject: Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action\n> \n> On Mon, Sep 11, 2017 at 07:42:19AM +0000, Chandran, Sugesh wrote:\n> >\n> >\n> > Regards\n> > _Sugesh\n> >\n> >\n> > > -----Original Message-----\n> > > From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev-\n> > > bounces@openvswitch.org] On Behalf Of Yuanhan Liu\n> > > Sent: Tuesday, September 5, 2017 10:23 AM\n> > > To: dev@openvswitch.org\n> > > Subject: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue\n> > > action\n> > >\n> > > From: Finn Christensen \n> > >\n> > > AFAIK, most (if not all) NICs (including Mellanox and Intel) do not\n> > > support a pure MARK action. It's required to be used together with\n> > > some other actions, like QUEUE.\n> > >\n> > > To workaround it, retry with a queue action when first try failed.\n> > [Sugesh] This means a NIC that doesn't support pure MARK, will do\n> > every flow install twice no matter how many times the flow install failed\n> before for the same reason.\n> > The retry can be avoided if we know what a port/NIC can support. It\n> > leads to a need of feature discovery of a NIC at the init time.\n> \n> I believe the reply I made to Simon answered all questions you asked above.\n[Sugesh] Thanks!\n\n> \n> \t--yliu","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xrMJ61VHQz9s8J\n\tfor ;\n\tMon, 11 Sep 2017 18:57:26 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id C8FADB77;\n\tMon, 11 Sep 2017 08:57:08 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 499F2B6C\n\tfor ; Mon, 11 Sep 2017 08:57:07 +0000 (UTC)","from mga02.intel.com (mga02.intel.com [134.134.136.20])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id E78871D7\n\tfor ; Mon, 11 Sep 2017 08:57:06 +0000 (UTC)","from orsmga003.jf.intel.com ([10.7.209.27])\n\tby orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t11 Sep 2017 01:57:06 -0700","from irsmsx104.ger.corp.intel.com ([163.33.3.159])\n\tby orsmga003.jf.intel.com with ESMTP; 11 Sep 2017 01:56:58 -0700","from irsmsx102.ger.corp.intel.com ([169.254.2.59]) by\n\tIRSMSX104.ger.corp.intel.com ([163.33.3.159]) with mapi id\n\t14.03.0319.002; Mon, 11 Sep 2017 09:56:54 +0100"],"X-Greylist":"domain auto-whitelisted by SQLgrey-1.7.6","X-ExtLoop1":"1","X-IronPort-AV":"E=Sophos; i=\"5.42,376,1500966000\"; d=\"scan'208\";\n\ta=\"1013099708\"","From":"\"Chandran, Sugesh\" ","To":"Yuanhan Liu ","Thread-Topic":"[ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","Thread-Index":"AQHTJilLVRUwf8GQ/0SDIXf1r0WzAKKuKMYggAEg6gCAACHEkA==","Date":"Mon, 11 Sep 2017 08:56:54 +0000","Message-ID":"<2EF2F5C0CC56984AA024D0B180335FCB4221DFA4@IRSMSX102.ger.corp.intel.com>","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<2EF2F5C0CC56984AA024D0B180335FCB4221DE59@IRSMSX102.ger.corp.intel.com>\n\t<20170911075552.GS9736@yliu-home>","In-Reply-To":"<20170911075552.GS9736@yliu-home>","Accept-Language":"en-US","Content-Language":"en-US","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","x-titus-metadata-40":"eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMzczNGMyNGYtNjJjMC00YWQ3LTk2MmItMzI4NzNjZTZmNjQ3IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6IitsTXZWdDBQV0t4OVlub3FPVWRiNDdmTng4bDBjcVladnpBSUhjcFRraXM9In0=","x-ctpclassification":"CTP_IC","dlp-product":"dlpe-windows","dlp-version":"11.0.0.116","dlp-reaction":"no-action","x-originating-ip":"[163.33.239.181]","MIME-Version":"1.0","X-Spam-Status":"No, score=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED,\n\tRP_MATCHES_RCVD autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1766756,"web_url":"http://patchwork.ozlabs.org/comment/1766756/","msgid":"<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>","list_archive_url":null,"date":"2017-09-12T08:36:19","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":68212,"url":"http://patchwork.ozlabs.org/api/people/68212/","name":"Darrell Ball","email":"dball@vmware.com"},"content":"On 9/10/17, 11:14 PM, \"ovs-dev-bounces@openvswitch.org on behalf of Yuanhan Liu\" wrote:\r\n\r\n On Fri, Sep 08, 2017 at 06:48:50PM +0200, Simon Horman wrote:\r\n > On Tue, Sep 05, 2017 at 05:22:59PM +0800, Yuanhan Liu wrote:\r\n > > From: Finn Christensen \r\n > > \r\n > > AFAIK, most (if not all) NICs (including Mellanox and Intel) do not\r\n > > support a pure MARK action. It's required to be used together with\r\n > > some other actions, like QUEUE.\r\n > > \r\n > > To workaround it, retry with a queue action when first try failed.\r\n > > \r\n > > Moreover, some Intel's NIC (say XL710) needs the QUEUE action set\r\n > > before the MARK action.\r\n > > \r\n > > Co-authored-by: Yuanhan Liu \r\n > > Signed-off-by: Finn Christensen \r\n > > Signed-off-by: Yuanhan Liu \r\n > \r\n > This feels a bit like the tail wagging the dog.\r\n > Is this the lowest level at which it makes sense to implement\r\n > this logic?\r\n > \r\n > If so then I wonder if some sort of probing would be in order\r\n > to avoid the cost of trying to add the flow twice to hardware\r\n > where the queue is required.\r\n \r\n Do you mean something like rte_flow capability query, like whether\r\n a queue action is needed for a mark action? If so, yes, I do think\r\n we miss an interface like this.\r\n \r\n Note that even in this solution, the flow won't be created twice\r\n to the hardware, because the first try would be failed.\r\n\r\n[Darrell]\r\n\r\n Having an api to quey capability and avoid the first try to HW would be nice, but there are dependencies\r\n on RTE, drivers etc and I don’t know definitive the api would be.\r\n\r\n Also, as nics are added this capability needs to be done and state needs to be kept in all cases.\r\n\r\n It is an enhancement and if done should be reliable.\r\n\r\n A separate comment is we need to document which nics need the queue action.\r\n\r\n Also, I think we should check errno in the present code.\r\n\r\n\r\n \r\n \t--yliu\r\n _______________________________________________\r\n dev mailing list\r\n dev@openvswitch.org\r\n https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=71eTQnWPnndmPoDf64Z0cHyxIDvF0etl3eUy8H8xPFU&s=-54ktRZZw64tHzc56bL7yylfvkPgEOtmTS_89Cho02M&e=","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=onevmw.onmicrosoft.com\n\theader.i=@onevmw.onmicrosoft.com header.b=\"Nq3QKlSW\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=dball@vmware.com; "],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xrynP5dTmz9s7g\n\tfor ;\n\tTue, 12 Sep 2017 18:36:25 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id 93EE2AB6;\n\tTue, 12 Sep 2017 08:36:22 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 747AAAB2\n\tfor ; Tue, 12 Sep 2017 08:36:21 +0000 (UTC)","from NAM03-CO1-obe.outbound.protection.outlook.com\n\t(mail-co1nam03on0085.outbound.protection.outlook.com [104.47.40.85])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id C90ECA4\n\tfor ; Tue, 12 Sep 2017 08:36:20 +0000 (UTC)","from BLUPR05MB611.namprd05.prod.outlook.com (10.141.204.27) by\n\tBLUPR05MB306.namprd05.prod.outlook.com (10.141.23.148) with Microsoft\n\tSMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id\n\t15.20.56.4; Tue, 12 Sep 2017 08:36:19 +0000","from BLUPR05MB611.namprd05.prod.outlook.com ([10.141.204.27]) by\n\tBLUPR05MB611.namprd05.prod.outlook.com ([10.141.204.27]) with mapi id\n\t15.20.0056.010; Tue, 12 Sep 2017 08:36:19 +0000"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=onevmw.onmicrosoft.com; s=selector1-vmware-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=H4do46hMzeZgvY8DkmPqoD6kGYxFjJmCMte9Difewfw=;\n\tb=Nq3QKlSWVYvjUuHm/mmpL18EEsLW6P/b8R65cc2QvhcH1k6revwm1+6Mf7pbCksn4N0FavymuvL6smlRUsr+HKc39plu28tDb0hBYqicz0OwwyZZKH2HW+1SVMv4+BKS56sz8e/tOu0vJwVbrhveeSBKM7wTn0R2QoeoFRfLgg4=","From":"Darrell Ball ","To":"Yuanhan Liu , Simon Horman\n\t","Thread-Topic":"[ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","Thread-Index":"AQHTJiizNzCJ8+cj4EuMMlhNVpa+HKKrOF8AgAQFvoCAAUSgAA==","Date":"Tue, 12 Sep 2017 08:36:19 +0000","Message-ID":"<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<20170908164849.GH7356@vergenet.net>\n\t<20170911061425.GN9736@yliu-home>","In-Reply-To":"<20170911061425.GN9736@yliu-home>","Accept-Language":"en-US","Content-Language":"en-US","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","user-agent":"Microsoft-MacOutlook/f.25.0.170815","authentication-results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=onevmw.onmicrosoft.com\n\theader.i=@onevmw.onmicrosoft.com header.b=\"Nq3QKlSW\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=dball@vmware.com; "],"x-originating-ip":"[73.162.236.45]","x-ms-publictraffictype":"Email","x-microsoft-exchange-diagnostics":"1; BLUPR05MB306;\n\t20:KLRdD5OpoJ8+oe5dRtK0FQNRjULBVWhZfyF0r62IQZ/MRn9XI6sL+B9Rdt5tkUEsNjukM3qcVGMa/MxOrzAdnWQL6Q1svyW0PrNpPBQo59GBTy2zi9AdsHNt/P+UqbX6SvxMpACvyhClY+hjmf0LHlHiNcHP/F0Kbdq2kauiTOs=","x-ms-exchange-antispam-srfa-diagnostics":"SSOS;","x-ms-office365-filtering-correlation-id":"5d82235e-739a-4c14-f93a-08d4f9b9571b","x-microsoft-antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:BLUPR05MB306; ","x-ms-traffictypediagnostic":"BLUPR05MB306:","x-exchange-antispam-report-test":"UriScan:(10436049006162)(216315784871565);","x-microsoft-antispam-prvs":"","x-exchange-antispam-report-cfa-test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(93006095)(93001095)(10201501046)(100000703101)(100105400095)(3002001)(6041248)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123564025)(20161123558100)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:BLUPR05MB306; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:BLUPR05MB306; ","x-forefront-prvs":"042857DBB5","x-forefront-antispam-report":"SFV:NSPM;\n\tSFS:(10009020)(6009001)(54094003)(24454002)(377454003)(189002)(199003)(8676002)(3660700001)(81166006)(3280700002)(6436002)(33656002)(68736007)(97736004)(105586002)(39060400002)(6506006)(14454004)(966005)(25786009)(101416001)(189998001)(4326008)(316002)(106356001)(2900100001)(81156014)(6306002)(93886005)(8936002)(6512007)(82746002)(53936002)(7736002)(2906002)(6246003)(77096006)(6116002)(2950100002)(5660300001)(102836003)(3846002)(229853002)(76176999)(478600001)(99286003)(50986999)(53546010)(575784001)(83716003)(6486002)(83506001)(305945005)(66066001)(54356999)(4001350100001)(86362001)(36756003);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR05MB306;\n\tH:BLUPR05MB611.namprd05.prod.outlook.com; FPR:; SPF:None;\n\tPTR:InfoNoRecords; MX:1; A:1; LANG:en; ","received-spf":"None (protection.outlook.com: vmware.com does not designate\n\tpermitted sender hosts)","spamdiagnosticoutput":"1:99","spamdiagnosticmetadata":"NSPM","Content-ID":"","MIME-Version":"1.0","X-OriginatorOrg":"vmware.com","X-MS-Exchange-CrossTenant-originalarrivaltime":"12 Sep 2017 08:36:19.1018\n\t(UTC)","X-MS-Exchange-CrossTenant-fromentityheader":"Hosted","X-MS-Exchange-CrossTenant-id":"b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"BLUPR05MB306","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1767746,"web_url":"http://patchwork.ozlabs.org/comment/1767746/","msgid":"<20170913095730.GE27555@vergenet.net>","list_archive_url":null,"date":"2017-09-13T09:57:31","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":64714,"url":"http://patchwork.ozlabs.org/api/people/64714/","name":"Simon Horman","email":"simon.horman@netronome.com"},"content":"On Tue, Sep 12, 2017 at 08:36:19AM +0000, Darrell Ball wrote:\n> \n> On 9/10/17, 11:14 PM, \"ovs-dev-bounces@openvswitch.org on behalf of Yuanhan Liu\" wrote:\n> \n> On Fri, Sep 08, 2017 at 06:48:50PM +0200, Simon Horman wrote:\n> > On Tue, Sep 05, 2017 at 05:22:59PM +0800, Yuanhan Liu wrote:\n> > > From: Finn Christensen \n> > > \n> > > AFAIK, most (if not all) NICs (including Mellanox and Intel) do not\n> > > support a pure MARK action. It's required to be used together with\n> > > some other actions, like QUEUE.\n> > > \n> > > To workaround it, retry with a queue action when first try failed.\n> > > \n> > > Moreover, some Intel's NIC (say XL710) needs the QUEUE action set\n> > > before the MARK action.\n> > > \n> > > Co-authored-by: Yuanhan Liu \n> > > Signed-off-by: Finn Christensen \n> > > Signed-off-by: Yuanhan Liu \n> > \n> > This feels a bit like the tail wagging the dog.\n> > Is this the lowest level at which it makes sense to implement\n> > this logic?\n> > \n> > If so then I wonder if some sort of probing would be in order\n> > to avoid the cost of trying to add the flow twice to hardware\n> > where the queue is required.\n> \n> Do you mean something like rte_flow capability query, like whether\n> a queue action is needed for a mark action? If so, yes, I do think\n> we miss an interface like this.\n> \n> Note that even in this solution, the flow won't be created twice\n> to the hardware, because the first try would be failed.\n> \n> [Darrell]\n> \n> Having an api to quey capability and avoid the first try to HW would be nice, but there are dependencies\n> on RTE, drivers etc and I don’t know definitive the api would be.\n> \n> Also, as nics are added this capability needs to be done and state needs to be kept in all cases.\n> \n> It is an enhancement and if done should be reliable.\n\nAgreed. Though I was more thinking of probing the hardware rather than\nhaving a capability API - I expect this would remove several of the\ndependencies you describe above.\n\nAssuming no such enhancement is appropriate at this time I would\nstill like to ask if this is the best place for this hardware-specific code?\n\n> A separate comment is we need to document which nics need the queue action.\n> \n> Also, I think we should check errno in the present code.","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=netronome-com.20150623.gappssmtp.com\n\theader.i=@netronome-com.20150623.gappssmtp.com\n\theader.b=\"wZnbrgXS\"; dkim-atps=neutral"],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xscXf35ytz9s76\n\tfor ;\n\tWed, 13 Sep 2017 19:57:38 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id DF8D6BC2;\n\tWed, 13 Sep 2017 09:57:34 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 37876B80\n\tfor ; Wed, 13 Sep 2017 09:57:34 +0000 (UTC)","from mail-wm0-f49.google.com (mail-wm0-f49.google.com\n\t[74.125.82.49])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id 6E12AA7\n\tfor ; Wed, 13 Sep 2017 09:57:33 +0000 (UTC)","by mail-wm0-f49.google.com with SMTP id i189so2411879wmf.1\n\tfor ; Wed, 13 Sep 2017 02:57:33 -0700 (PDT)","from vergenet.net (reginn.horms.nl.\n\t[2001:470:7eb3:403:d63d:7eff:fe99:ac9d])\n\tby smtp.gmail.com with ESMTPSA id\n\te12sm6717843edj.88.2017.09.13.02.57.31\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tWed, 13 Sep 2017 02:57:31 -0700 (PDT)"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=netronome-com.20150623.gappssmtp.com; s=20150623;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:content-transfer-encoding:in-reply-to\n\t:user-agent; bh=QLM1DhTIzyiYoUApNimRkuEDG8sel/CPzNaIdqCO2+4=;\n\tb=wZnbrgXSXlYcUtvu8Bc0o2qgNAmr0qNMTLD5AVIYOxAjkEJeUsg6Aj4Qln3ksT+a44\n\tVJDy0CimMr9HVoNah1T1RODWVfLTzX71d8stOYp46+Sdr0Zw+yzH+nDExJjkI114u5jJ\n\tFGXQMgO+AAUgQCudHrf27dpY7hHh/uc03KZzjDQkTgK05li2M7dwEDpZCqaZRldZKCf+\n\tkMSICzxUbxFyW0dBFJnTvADsbB8tAL6bZw5SOgn5GY1pmSjWf4ToNfg9nwuBkixA4UVJ\n\tUQe02kis/HYNs7G8bqxSgbx5Mo54fL6Y5Yk/+JXTShU1DBBh3W4E7d5zNgByawHeX2b2\n\t7eJw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:content-transfer-encoding\n\t:in-reply-to:user-agent;\n\tbh=QLM1DhTIzyiYoUApNimRkuEDG8sel/CPzNaIdqCO2+4=;\n\tb=d3gI+bMYeMlWVHg/xzDTF8PTmdhyNHGNfahfjRQE9tyRw1O8zXavw+O6vjn6uD2qxE\n\txhFaDIfH8EQC7wkF5wLkr9cd6sd09j36GP+L5IFsCB+6kczKSTtaCHboW1WfuxQsrCN6\n\tuVhZRGktwxXKUGy0fS6AV7ERVX0nrXnzIxzOQmbzbmqkCXfMQC8Jis8fWGLnQlDjLztB\n\t6lJfJOuYsaP2dVmddlpqAOABl1T/P8OJUEnuBS6mS+JbDO/yYuYi/jFcW3Uz2DXUNE2R\n\txdYo5mEz2jDy60DW/umdJP/08jUevIW1h4ekjE7If9OYjCrjs327q1MqJttRZTLFyChH\n\tPSOw==","X-Gm-Message-State":"AHPjjUiIT6nMARK6zy6fgUN9H7c/it6vcIxrk+JtNUb/kZpb+S93DqSJ\n\tFFQXQPKrbjlvBOLl","X-Google-Smtp-Source":"ADKCNb5P4L7qYD8Sn60/GsDwVfFOuIRYnQ95f5sUN+2OUpYwcZFvH4Pw/A7nDY/0MGucuxMm+5DOYg==","X-Received":"by 10.80.212.150 with SMTP id s22mr10532840edi.286.1505296652080;\n\tWed, 13 Sep 2017 02:57:32 -0700 (PDT)","Date":"Wed, 13 Sep 2017 11:57:31 +0200","From":"Simon Horman ","To":"Darrell Ball ","Message-ID":"<20170913095730.GE27555@vergenet.net>","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<20170908164849.GH7356@vergenet.net>\n\t<20170911061425.GN9736@yliu-home>\n\t<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>","User-Agent":"Mutt/1.5.23 (2014-03-12)","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1768001,"web_url":"http://patchwork.ozlabs.org/comment/1768001/","msgid":"","list_archive_url":null,"date":"2017-09-13T16:17:37","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":68212,"url":"http://patchwork.ozlabs.org/api/people/68212/","name":"Darrell Ball","email":"dball@vmware.com"},"content":"On 9/13/17, 2:57 AM, \"Simon Horman\" wrote:\r\n\r\n On Tue, Sep 12, 2017 at 08:36:19AM +0000, Darrell Ball wrote:\r\n > \r\n > On 9/10/17, 11:14 PM, \"ovs-dev-bounces@openvswitch.org on behalf of Yuanhan Liu\" wrote:\r\n > \r\n > On Fri, Sep 08, 2017 at 06:48:50PM +0200, Simon Horman wrote:\r\n > > On Tue, Sep 05, 2017 at 05:22:59PM +0800, Yuanhan Liu wrote:\r\n > > > From: Finn Christensen \r\n > > > \r\n > > > AFAIK, most (if not all) NICs (including Mellanox and Intel) do not\r\n > > > support a pure MARK action. It's required to be used together with\r\n > > > some other actions, like QUEUE.\r\n > > > \r\n > > > To workaround it, retry with a queue action when first try failed.\r\n > > > \r\n > > > Moreover, some Intel's NIC (say XL710) needs the QUEUE action set\r\n > > > before the MARK action.\r\n > > > \r\n > > > Co-authored-by: Yuanhan Liu \r\n > > > Signed-off-by: Finn Christensen \r\n > > > Signed-off-by: Yuanhan Liu \r\n > > \r\n > > This feels a bit like the tail wagging the dog.\r\n > > Is this the lowest level at which it makes sense to implement\r\n > > this logic?\r\n > > \r\n > > If so then I wonder if some sort of probing would be in order\r\n > > to avoid the cost of trying to add the flow twice to hardware\r\n > > where the queue is required.\r\n > \r\n > Do you mean something like rte_flow capability query, like whether\r\n > a queue action is needed for a mark action? If so, yes, I do think\r\n > we miss an interface like this.\r\n > \r\n > Note that even in this solution, the flow won't be created twice\r\n > to the hardware, because the first try would be failed.\r\n > \r\n > [Darrell]\r\n > \r\n > Having an api to quey capability and avoid the first try to HW would be nice, but there are dependencies\r\n > on RTE, drivers etc and I don’t know definitive the api would be.\r\n > \r\n > Also, as nics are added this capability needs to be done and state needs to be kept in all cases.\r\n > \r\n > It is an enhancement and if done should be reliable.\r\n \r\n Agreed. Though I was more thinking of probing the hardware rather than\r\n having a capability API - I expect this would remove several of the\r\n dependencies you describe above.\r\n\r\n\r\n[Darrell] I have been pondering the probing option as well. It is certainly a valid option; \r\nwe use it in other cases such as datapath probing. One of the aspects that worries me here is\r\nmaintaining the correct per interface (essentially; although the attribute is per nic) state\r\nacross various events such as new ports being added, vswitchd restarts, races with flow\r\ncreation. It would be non-trivial I guess and probably appropriate for the next patch series, if done.\r\n\r\nIn this case, we have what seems like a clear distinction b/w Napatech which does not need the\r\nqueue action workaround and everything else, which does.\r\nBesides the non-Napatech behavior, which is worrisome, maintaining the difference for flow handling\r\nunder the covers is concerning.\r\n\r\nI wonder if we should be upfront as possible here and just have a dpdk interface configuration – maybe\r\nsomething like “supports native HWOL mark action” since the better behavior is the exception?\r\nThe interface config would be more robust than probing.\r\nThis would need documentation, of course.\r\n\r\nI think anyways we need documentation describing the difference b/w nics in the dpdk documentation (howto part).\r\n\r\n \r\n Assuming no such enhancement is appropriate at this time I would\r\n still like to ask if this is the best place for this hardware-specific code?\r\n\r\n[Darrell]\r\n\r\nFor OVS, the netdev-dpdk layer is the lowest layer.\r\nThis kind of workaround is hard to hide, since we are messing with the rxq, so I think OVS needs to know\r\nthat it is in effect anyways. An alternative is to supply a mark and an ‘optional queue’ and let the driver decide if the queue is\r\nneeded and report back whether it was. This would be hard to do across various drivers. Supporting in the rte layer would require both\r\nrte and driver support, so even more support.\r\n\r\n\r\n \r\n > A separate comment is we need to document which nics need the queue action.\r\n > \r\n > Also, I think we should check errno in the present code.","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=onevmw.onmicrosoft.com\n\theader.i=@onevmw.onmicrosoft.com header.b=\"EQWNr/Co\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=dball@vmware.com; "],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xsmzD54Pxz9sPm\n\tfor ;\n\tThu, 14 Sep 2017 02:17:44 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id 98AA098A;\n\tWed, 13 Sep 2017 16:17:41 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 98D5989E\n\tfor ; Wed, 13 Sep 2017 16:17:40 +0000 (UTC)","from NAM02-BL2-obe.outbound.protection.outlook.com\n\t(mail-bl2nam02on0072.outbound.protection.outlook.com [104.47.38.72])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id 930871DF\n\tfor ; Wed, 13 Sep 2017 16:17:39 +0000 (UTC)","from BLUPR05MB611.namprd05.prod.outlook.com (10.141.204.27) by\n\tBLUPR05MB401.namprd05.prod.outlook.com (10.141.26.139) with Microsoft\n\tSMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id\n\t15.20.77.5; Wed, 13 Sep 2017 16:17:37 +0000","from BLUPR05MB611.namprd05.prod.outlook.com ([10.141.204.27]) by\n\tBLUPR05MB611.namprd05.prod.outlook.com ([10.141.204.27]) with mapi id\n\t15.20.0056.010; Wed, 13 Sep 2017 16:17:37 +0000"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=onevmw.onmicrosoft.com; s=selector1-vmware-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=zrhB2Y2noTYJQZVq5mXrdAZR6c3gF1wcpk2dJue0Dpg=;\n\tb=EQWNr/CoUjWbWQRin1/wwuXzp3x0M6FK6v8l7IqRfpe51Hap3VaOBlgVGBzCULdpoY90yvrAuFmhucTaF18tauMW1bM6Za9wLBmeoxRwK83yfLMnOfL6BjZuaq8hGkQ2SinmiCFLx+zN/bF3PmqdKeZDic8KiHDsF+IQhb9/V6U=","From":"Darrell Ball ","To":"Simon Horman ","Thread-Topic":"[ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","Thread-Index":"AQHTJiizNzCJ8+cj4EuMMlhNVpa+HKKrOF8AgAQFvoCAAUSgAIACHl+A///02QA=","Date":"Wed, 13 Sep 2017 16:17:37 +0000","Message-ID":"","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<20170908164849.GH7356@vergenet.net>\n\t<20170911061425.GN9736@yliu-home>\n\t<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>\n\t<20170913095730.GE27555@vergenet.net>","In-Reply-To":"<20170913095730.GE27555@vergenet.net>","Accept-Language":"en-US","Content-Language":"en-US","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","user-agent":"Microsoft-MacOutlook/f.25.0.170815","authentication-results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=onevmw.onmicrosoft.com\n\theader.i=@onevmw.onmicrosoft.com header.b=\"EQWNr/Co\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=dball@vmware.com; "],"x-originating-ip":"[73.162.236.45]","x-ms-publictraffictype":"Email","x-microsoft-exchange-diagnostics":"1; BLUPR05MB401;\n\t20:1KdUtLXeppyHmYGuDns7oxN02ELP6QWFSSZAK9N6RxiTUFvdr12o21OLaOUfmOj5gHjTUzkpvvKP5FkhK6l8337OhZnZxkw/fAS4t2s4jnEW+iOQXh8gO4W3Pcv9J+qK62O8KedlpzjO313Ve8T5xdpXxo544fIzHbYPDLNOfY0=","x-ms-exchange-antispam-srfa-diagnostics":"SSOS;","x-ms-office365-filtering-correlation-id":"8517d519-0fda-4670-71b6-08d4fac2f328","x-microsoft-antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:BLUPR05MB401; ","x-ms-traffictypediagnostic":"BLUPR05MB401:","x-exchange-antispam-report-test":"UriScan:(216315784871565);","x-microsoft-antispam-prvs":"","x-exchange-antispam-report-cfa-test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(93006095)(93001095)(100000703101)(100105400095)(3002001)(10201501046)(6041248)(20161123564025)(20161123558100)(20161123560025)(20161123555025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:BLUPR05MB401; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:BLUPR05MB401; ","x-forefront-prvs":"042957ACD7","x-forefront-antispam-report":"SFV:NSPM;\n\tSFS:(10009020)(6009001)(39860400002)(366002)(346002)(376002)(377454003)(199003)(24454002)(54094003)(189002)(81166006)(2906002)(8936002)(93886005)(50986999)(68736007)(2900100001)(81156014)(54356999)(76176999)(14454004)(101416001)(6486002)(77096006)(25786009)(5660300001)(6506006)(53546010)(36756003)(3846002)(6116002)(102836003)(305945005)(229853002)(6436002)(110136004)(39060400002)(53936002)(8676002)(86362001)(6246003)(82746002)(105586002)(106356001)(316002)(4326008)(6916009)(7736002)(66066001)(189998001)(99286003)(478600001)(54906002)(3280700002)(3660700001)(97736004)(4001350100001)(83506001)(2950100002)(83716003)(6512007)(33656002);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR05MB401;\n\tH:BLUPR05MB611.namprd05.prod.outlook.com; FPR:; SPF:None;\n\tPTR:InfoNoRecords; A:1; MX:1; LANG:en; ","received-spf":"None (protection.outlook.com: vmware.com does not designate\n\tpermitted sender hosts)","spamdiagnosticoutput":"1:99","spamdiagnosticmetadata":"NSPM","Content-ID":"","MIME-Version":"1.0","X-OriginatorOrg":"vmware.com","X-MS-Exchange-CrossTenant-originalarrivaltime":"13 Sep 2017 16:17:37.4334\n\t(UTC)","X-MS-Exchange-CrossTenant-fromentityheader":"Hosted","X-MS-Exchange-CrossTenant-id":"b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"BLUPR05MB401","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1768408,"web_url":"http://patchwork.ozlabs.org/comment/1768408/","msgid":"<3edc7d93b8c94c3c8e316b299981b984@napatech.com>","list_archive_url":null,"date":"2017-09-14T08:14:25","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":72305,"url":"http://patchwork.ozlabs.org/api/people/72305/","name":"Finn Christensen","email":"fc@napatech.com"},"content":"-----Original Message-----\r\nFrom: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev-bounces@openvswitch.org] On Behalf Of Darrell Ball\r\nSent: 13. september 2017 18:18\r\nTo: Simon Horman \r\nCc: dev@openvswitch.org\r\nSubject: Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action\r\n\r\n\r\n\r\nOn 9/13/17, 2:57 AM, \"Simon Horman\" wrote:\r\n\r\n On Tue, Sep 12, 2017 at 08:36:19AM +0000, Darrell Ball wrote:\r\n > \r\n > On 9/10/17, 11:14 PM, \"ovs-dev-bounces@openvswitch.org on behalf of Yuanhan Liu\" wrote:\r\n > \r\n > On Fri, Sep 08, 2017 at 06:48:50PM +0200, Simon Horman wrote:\r\n > > On Tue, Sep 05, 2017 at 05:22:59PM +0800, Yuanhan Liu wrote:\r\n > > > From: Finn Christensen \r\n > > > \r\n > > > AFAIK, most (if not all) NICs (including Mellanox and Intel) do not\r\n > > > support a pure MARK action. It's required to be used together with\r\n > > > some other actions, like QUEUE.\r\n > > > \r\n > > > To workaround it, retry with a queue action when first try failed.\r\n > > > \r\n > > > Moreover, some Intel's NIC (say XL710) needs the QUEUE action set\r\n > > > before the MARK action.\r\n > > > \r\n > > > Co-authored-by: Yuanhan Liu \r\n > > > Signed-off-by: Finn Christensen \r\n > > > Signed-off-by: Yuanhan Liu \r\n > > \r\n > > This feels a bit like the tail wagging the dog.\r\n > > Is this the lowest level at which it makes sense to implement\r\n > > this logic?\r\n > > \r\n > > If so then I wonder if some sort of probing would be in order\r\n > > to avoid the cost of trying to add the flow twice to hardware\r\n > > where the queue is required.\r\n > \r\n > Do you mean something like rte_flow capability query, like whether\r\n > a queue action is needed for a mark action? If so, yes, I do think\r\n > we miss an interface like this.\r\n > \r\n > Note that even in this solution, the flow won't be created twice\r\n > to the hardware, because the first try would be failed.\r\n > \r\n > [Darrell]\r\n > \r\n > Having an api to quey capability and avoid the first try to HW would be nice, but there are dependencies\r\n > on RTE, drivers etc and I don’t know definitive the api would be.\r\n > \r\n > Also, as nics are added this capability needs to be done and state needs to be kept in all cases.\r\n > \r\n > It is an enhancement and if done should be reliable.\r\n \r\n Agreed. Though I was more thinking of probing the hardware rather than\r\n having a capability API - I expect this would remove several of the\r\n dependencies you describe above.\r\n\r\n\r\n[Darrell] I have been pondering the probing option as well. It is certainly a valid option; we use it in other cases such as datapath probing. One of the aspects that worries me here is maintaining the correct per interface (essentially; although the attribute is per nic) state across various events such as new ports being added, vswitchd restarts, races with flow creation. It would be non-trivial I guess and probably appropriate for the next patch series, if done.\r\n\r\nIn this case, we have what seems like a clear distinction b/w Napatech which does not need the queue action workaround and everything else, which does.\r\nBesides the non-Napatech behavior, which is worrisome, maintaining the difference for flow handling under the covers is concerning.\r\n\r\nI wonder if we should be upfront as possible here and just have a dpdk interface configuration – maybe something like “supports native HWOL mark action” since the better behavior is the exception?\r\nThe interface config would be more robust than probing.\r\nThis would need documentation, of course.\r\n\r\n[Finn]\r\nI think the rte queue action should never be used here when using partial HWOL. Not the way OVS handles multi queues today. \r\nMaybe a \"default queue\" could be used in the dpdk PMDs when no queue is specified in rte flow?\r\nEssentially, this is a mismatch between the rte flow impl functionality in PMD and the needs in OVS for flow classification, in a partial HWOL setup.\r\nHowever, we would not mind Darells proposal, it makes sense since most nic PMDs unfortunately needs this and it is only relevant for partial HWOL. We are mostly concerned with full HWOL and therefore see partial HWOL as a failover when full HWOL could not be handled in nic, if enabled.\r\n\r\n\r\nI think anyways we need documentation describing the difference b/w nics in the dpdk documentation (howto part).\r\n\r\n \r\n Assuming no such enhancement is appropriate at this time I would\r\n still like to ask if this is the best place for this hardware-specific code?\r\n\r\n[Darrell]\r\n\r\nFor OVS, the netdev-dpdk layer is the lowest layer.\r\nThis kind of workaround is hard to hide, since we are messing with the rxq, so I think OVS needs to know that it is in effect anyways. An alternative is to supply a mark and an ‘optional queue’ and let the driver decide if the queue is needed and report back whether it was. This would be hard to do across various drivers. Supporting in the rte layer would require both rte and driver support, so even more support.\r\n\r\n\r\n \r\n > A separate comment is we need to document which nics need the queue action.\r\n > \r\n > Also, I think we should check errno in the present code.\r\n \r\n\r\n_______________________________________________\r\ndev mailing list\r\ndev@openvswitch.org\r\nhttps://mail.openvswitch.org/mailman/listinfo/ovs-dev","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=napatech.com header.i=@napatech.com\n\theader.b=\"RpCJfiTl\"; dkim-atps=neutral"],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtBCQ0CL8z9sP1\n\tfor ;\n\tThu, 14 Sep 2017 18:14:39 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id 3FFC29EE;\n\tThu, 14 Sep 2017 08:14:35 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 8F94099F\n\tfor ; Thu, 14 Sep 2017 08:14:33 +0000 (UTC)","from mail01.napatech.com (mail01.napatech.com [188.120.77.121])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id EB5D3A7\n\tfor ; Thu, 14 Sep 2017 08:14:31 +0000 (UTC)","from cph-gen-exch02.napatech.com (10.240.1.84) by\n\tcph-gen-exch02.napatech.com (10.240.1.84) with Microsoft SMTP Server\n\t(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id\n\t15.1.1034.26; Thu, 14 Sep 2017 10:14:25 +0200","from cph-gen-exch02.napatech.com ([fe80::581:51a1:ac3f:84e]) by\n\tcph-gen-exch02.napatech.com ([fe80::581:51a1:ac3f:84e%12]) with mapi\n\tid 15.01.1034.026; Thu, 14 Sep 2017 10:14:25 +0200"],"X-Greylist":"domain auto-whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=napatech.com; i=@napatech.com; q=dns/txt; s=mar2017;\n\tt=1505376872; x=1536912872;\n\th=from:to:cc:subject:date:message-id:references:\n\tin-reply-to:content-transfer-encoding:mime-version;\n\tbh=jg9Nbowjp72P4X4jzRYXcSAjRq1ZOPhU0HngENTj3Hk=;\n\tb=RpCJfiTlCyMznxrnootE2bhUva+mzscf470lkrYb5VkfryTXfa6gZT8P\n\ton2qdMCGzXQLxGwU2W9t7p7JM6QgXrjERgnZnVb/+2I4w4Tn8URHdk5Z9\n\tHttlluclfgZifv+PBnV25+o3YSY2zt3gQbUU/VvR26gNHjGQGEcD2w1CJ\n\t7DCdFYqfLbUr8OXuDDgsXqBoGArml9lCgGaKNh2TF0w7a5hwsXP444hyZ\n\ttLQutRgAPy6LNsccuo7zI1NMKoh5w8dRpt6nKLXS/OV8kjA0CDcvoiS1U\n\tdl2c7VXvDAB5PRt2Z3fFMaILfglZ19v4MHrBqB6reodRPMe5yWGX0+P9o A==;","IronPort-PHdr":"9a23:wLZcxR1/ULwWmujksmDT+DRfVm0co7zxezQtwd8ZseIRKPad9pjvdHbS+e9qxAeQG96KurQY1qGG7+jJYi8p2d65qncMcZhBBVcuqP49uEgeOvODElDxN/XwbiY3T4xoXV5h+GynYwAOQJ6tL1LdrWev4jEMBx7xKRR6JvjvGo7Vks+7y/2+94fdbghMgDexe7x/IRW5oQjSucQdnJdvJLs2xhbVuHVDZv5YxXlvJVKdnhb84tm/8Zt++ClOuPwv6tBNX7zic6s3UbJXAjImM3so5MLwrhnMURGP5noHXWoIlBdDHhXI4wv7Xpf1tSv6q/Z91SyHNsD4Ubw4RTKv5LpwRRT2lCkIKSI28GDPisxxkq1bpg6hpwdiyILQeY2ZKeZycr/Ycd4cWGFPXNteVzZZD428cYUBEvYBM+hboYnzpVQOrAexCwajC+701j9HnX320bEm3+g9EwzL2hErEdIUsHTTqdX4LKccUeGzzKnO0D7OcfNW2S386IjTbhAuv/eMXalufsrX1EIiEA3FgUmLpIzjJTyVzv4Cs3SF4OV8VeKjkXIoqwZ0ojW2wMonl4fHhoUQyl/e9CV5xp44JdqgSEFlZ96kDoBQti+bN4tqXswiQ3tkuCEgyr0Jv5OwYSsEyIw/yhLCd/CLaZaE7x3/WOqLPDt0nnFodb2nixqv7USs0PPwW8ao3FpQsyZJjsPAum4D2hHX8sSHROVy80S91TuK0g3e6uJJLV0xmKfbNpIu2KM8m58JvkvfECL2lkD7gayZe0o6/OWj9v7pba/8ppCGMo95kgT+MqMzlcOhGek4KQ0OX3SD+eS7yb3j4VX1QLVUgf0ylanUqJXaKt4apq69GQNV1Jws6w6lADe6ztsXgXkHIEhZdxKAiojlI1DOIPbmAvejm1mgjStny+rYMrDuHpnBNGXPnK3icLty80JczRA8zdFb55JaELEBJ/fzV1f/tNPEFRI5NRa7w/79B9VhyIwRRWKPDrWFP6PVtF+E/vgvLPWUZI8JpDb9LOAo6OPwgn8nglIderGp0oURaHCmBfRnLUSZYWbwjdcBC2sKuRA+TOO5wGGFBBJafWy/W6Z0zDg/DMryAY3KQoSFnrme1T22WJZRYzYVJEqLFCLGdoOCE9wMciOJPsJniTECHeyvQKcn3AmnqALxy/xsKe+CqX5Qjo7qyNUgv76brho17zEhSp3Fi2w=","X-IronPort-Anti-Spam-Filtered":"true","X-IronPort-Anti-Spam-Result":"A2FwAAD2ObpZ/1QB8ApcGQEBAQEBAQEBAQEBBwEBAQEBFAEBAQEBAQEBAQEBBwEBAQEBhBOBFQeDcIogkhuWJ4ILBwoYC4UZAhqFAhgBAQEBAQEBAQEBAQKBEIIzJAENcgEBAQEBAQEBAUwCDV0BAQEBAgEBASEROgsMBAIBCA4DBAEBAQICIwMCAgIlCxQBCAgCBA4FCBOKDxise4InizYBAQEBAQEBAQIBAQEBAQEBAQEaBYEOgh2DUoFjghuBDYRYAQGDMYJgBYl/iRKNaIdag1qJFZJ/hxONcAICAgIJAhqBOR+BRncVSoccdoZQgSOBDwEBAQ","X-IPAS-Result":"A2FwAAD2ObpZ/1QB8ApcGQEBAQEBAQEBAQEBBwEBAQEBFAEBAQEBAQEBAQEBBwEBAQEBhBOBFQeDcIogkhuWJ4ILBwoYC4UZAhqFAhgBAQEBAQEBAQEBAQKBEIIzJAENcgEBAQEBAQEBAUwCDV0BAQEBAgEBASEROgsMBAIBCA4DBAEBAQICIwMCAgIlCxQBCAgCBA4FCBOKDxise4InizYBAQEBAQEBAQIBAQEBAQEBAQEaBYEOgh2DUoFjghuBDYRYAQGDMYJgBYl/iRKNaIdag1qJFZJ/hxONcAICAgIJAhqBOR+BRncVSoccdoZQgSOBDwEBAQ","From":"Finn Christensen ","To":"Darrell Ball ","Thread-Topic":"[ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","Thread-Index":"AQHTJiizfJsWL3b3Rkev+6Ia92jE1aKrFtgAgAQFvoCAAbn6gIABqQWAgABqM4CAAD/isA==","Date":"Thu, 14 Sep 2017 08:14:25 +0000","Message-ID":"<3edc7d93b8c94c3c8e316b299981b984@napatech.com>","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<20170908164849.GH7356@vergenet.net>\n\t<20170911061425.GN9736@yliu-home>\n\t<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>\n\t<20170913095730.GE27555@vergenet.net>\n\t","In-Reply-To":"","Accept-Language":"da-DK, en-US","Content-Language":"en-US","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","x-originating-ip":"[10.240.10.239]","MIME-Version":"1.0","X-Spam-Status":"No, score=-2.4 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tDKIM_VALID_AU,RCVD_IN_DNSWL_MED,RP_MATCHES_RCVD autolearn=disabled\n\tversion=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1768796,"web_url":"http://patchwork.ozlabs.org/comment/1768796/","msgid":"","list_archive_url":null,"date":"2017-09-14T19:35:18","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":68212,"url":"http://patchwork.ozlabs.org/api/people/68212/","name":"Darrell Ball","email":"dball@vmware.com"},"content":"On 9/14/17, 1:14 AM, \"Finn Christensen\" wrote:\r\n\r\n \r\n \r\n \r\n \r\n -----Original Message-----\r\n \r\n From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev-bounces@openvswitch.org] On Behalf Of Darrell Ball\r\n \r\n Sent: 13. september 2017 18:18\r\n \r\n To: Simon Horman \r\n \r\n Cc: dev@openvswitch.org\r\n \r\n Subject: Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n On 9/13/17, 2:57 AM, \"Simon Horman\" wrote:\r\n \r\n \r\n \r\n On Tue, Sep 12, 2017 at 08:36:19AM +0000, Darrell Ball wrote:\r\n \r\n >\r\n \r\n > On 9/10/17, 11:14 PM, \"ovs-dev-bounces@openvswitch.org on behalf of Yuanhan Liu\" wrote:\r\n \r\n >\r\n \r\n > On Fri, Sep 08, 2017 at 06:48:50PM +0200, Simon Horman wrote:\r\n \r\n > > On Tue, Sep 05, 2017 at 05:22:59PM +0800, Yuanhan Liu wrote:\r\n \r\n > > > From: Finn Christensen \r\n \r\n > > >\r\n \r\n > > > AFAIK, most (if not all) NICs (including Mellanox and Intel) do not\r\n \r\n > > > support a pure MARK action. It's required to be used together with\r\n \r\n > > > some other actions, like QUEUE.\r\n \r\n > > >\r\n \r\n > > > To workaround it, retry with a queue action when first try failed.\r\n \r\n > > >\r\n \r\n > > > Moreover, some Intel's NIC (say XL710) needs the QUEUE action set\r\n \r\n > > > before the MARK action.\r\n \r\n > > >\r\n \r\n > > > Co-authored-by: Yuanhan Liu \r\n \r\n > > > Signed-off-by: Finn Christensen \r\n \r\n > > > Signed-off-by: Yuanhan Liu \r\n \r\n > >\r\n \r\n > > This feels a bit like the tail wagging the dog.\r\n \r\n > > Is this the lowest level at which it makes sense to implement\r\n \r\n > > this logic?\r\n \r\n > >\r\n \r\n > > If so then I wonder if some sort of probing would be in order\r\n \r\n > > to avoid the cost of trying to add the flow twice to hardware\r\n \r\n > > where the queue is required.\r\n \r\n >\r\n \r\n > Do you mean something like rte_flow capability query, like whether\r\n \r\n > a queue action is needed for a mark action? If so, yes, I do think\r\n \r\n > we miss an interface like this.\r\n \r\n >\r\n \r\n > Note that even in this solution, the flow won't be created twice\r\n \r\n > to the hardware, because the first try would be failed.\r\n \r\n >\r\n \r\n > [Darrell]\r\n \r\n >\r\n \r\n > Having an api to quey capability and avoid the first try to HW would be nice, but there are dependencies\r\n \r\n > on RTE, drivers etc and I don’t know definitive the api would be.\r\n \r\n >\r\n \r\n > Also, as nics are added this capability needs to be done and state needs to be kept in all cases.\r\n \r\n >\r\n \r\n > It is an enhancement and if done should be reliable.\r\n \r\n \r\n \r\n Agreed. Though I was more thinking of probing the hardware rather than\r\n \r\n having a capability API - I expect this would remove several of the\r\n \r\n dependencies you describe above.\r\n \r\n \r\n \r\n \r\n \r\n [Darrell] I have been pondering the probing option as well. It is certainly a valid option; we use it in other cases such as datapath probing. One of the aspects that worries me here is maintaining the correct per interface (essentially; although the attribute is per nic) state across various events such as new ports being added, vswitchd restarts, races with flow creation. It would be non-trivial I guess and probably appropriate for the next patch series, if done.\r\n \r\n \r\n \r\n In this case, we have what seems like a clear distinction b/w Napatech which does not need the queue action workaround and everything else, which does.\r\n \r\n Besides the non-Napatech behavior, which is worrisome, maintaining the difference for flow handling under the covers is concerning.\r\n \r\n \r\n \r\n I wonder if we should be upfront as possible here and just have a dpdk interface configuration – maybe something like “supports native HWOL mark action” since the better behavior is the exception?\r\n \r\n The interface config would be more robust than probing.\r\n \r\n This would need documentation, of course.\r\n \r\n \r\n \r\n [Finn]\r\n \r\n I think the rte queue action should never be used here when using partial HWOL. Not the way OVS handles multi queues today.\r\n \r\n Maybe a \"default queue\" could be used in the dpdk PMDs when no queue is specified in rte flow?\r\n\r\n[Darrell] This is the Napatech case, where no queue action is needed; you are suggesting programming a default queue in this case.\r\nI don’t follow how this would be helpful/desired?\r\n\r\n Essentially, this is a mismatch between the rte flow impl functionality in PMD and the needs in OVS for flow classification, in a partial HWOL setup.\r\n\r\n\r\n[Darrell] Maybe this has been explored, but were additional workaround actions, like RTE_FLOW_ACTION_TYPE_RSS considered ?\r\n I guess we could make this essentially a NOP, but if mark action could ride along with it, then that would be good.\r\n \r\n However, we would not mind Darells proposal, it makes sense since most nic PMDs unfortunately needs this and it is only relevant for partial HWOL. We are mostly concerned with full HWOL and therefore see partial HWOL as a failover when full HWOL could not be handled in nic, if enabled.\r\n \r\n \r\n \r\n \r\n \r\n I think anyways we need documentation describing the difference b/w nics in the dpdk documentation (howto part).\r\n \r\n \r\n \r\n \r\n \r\n Assuming no such enhancement is appropriate at this time I would\r\n \r\n still like to ask if this is the best place for this hardware-specific code?\r\n \r\n \r\n \r\n [Darrell]\r\n \r\n \r\n \r\n For OVS, the netdev-dpdk layer is the lowest layer.\r\n \r\n This kind of workaround is hard to hide, since we are messing with the rxq, so I think OVS needs to know that it is in effect anyways. An alternative is to supply a mark and an ‘optional queue’ and let the driver decide if the queue is needed and report back whether it was. This would be hard to do across various drivers. Supporting in the rte layer would require both rte and driver support, so even more support.\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n > A separate comment is we need to document which nics need the queue action.\r\n \r\n >\r\n \r\n > Also, I think we should check errno in the present code.\r\n \r\n \r\n \r\n \r\n \r\n _______________________________________________\r\n \r\n dev mailing list\r\n \r\n dev@openvswitch.org\r\n \r\n https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwIGaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=IHypHavCy0AKjNxqOMyc4w3ILyC-BuwkB8fuVvQUA3k&s=deJQWP9KI22Xp46tEoZ6o6Emitr3Bhfd7iSMxNpudeg&e= \r\n \r\n Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system.","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=onevmw.onmicrosoft.com\n\theader.i=@onevmw.onmicrosoft.com header.b=\"ksTmU1EC\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=dball@vmware.com; "],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtTJv6jnpz9s7M\n\tfor ;\n\tFri, 15 Sep 2017 05:35:27 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id 5636FB50;\n\tThu, 14 Sep 2017 19:35:24 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 4E023A6E\n\tfor ; Thu, 14 Sep 2017 19:35:22 +0000 (UTC)","from NAM01-BY2-obe.outbound.protection.outlook.com\n\t(mail-by2nam01on0053.outbound.protection.outlook.com [104.47.34.53])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id 62DFE499\n\tfor ; Thu, 14 Sep 2017 19:35:21 +0000 (UTC)","from BLUPR05MB611.namprd05.prod.outlook.com (10.141.204.27) by\n\tBLUPR05MB627.namprd05.prod.outlook.com (10.141.204.150) with\n\tMicrosoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id\n\t15.20.77.5; Thu, 14 Sep 2017 19:35:18 +0000","from BLUPR05MB611.namprd05.prod.outlook.com ([10.141.204.27]) by\n\tBLUPR05MB611.namprd05.prod.outlook.com ([10.141.204.27]) with mapi id\n\t15.20.0077.005; Thu, 14 Sep 2017 19:35:18 +0000"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=onevmw.onmicrosoft.com; s=selector1-vmware-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=9PhsKp2r/rnqcRhoecJjakS3a/tnwWki/hUhcknu6x0=;\n\tb=ksTmU1ECqqeTO0RkFMIn91+4Z6d67gZm/ekBXG9/xQljnTdzAArmGS3cOTFsP9VX0qz3pHC/rxkwaq/d6r3eX3N1bPld6PAheZgrAweHjCBwouUNUUHcU1HqxLNbumdx9HKAQ7l+P25vir7ATF5bMstLjoYv95LrO5PQvz0HOL4=","From":"Darrell Ball ","To":"Finn Christensen ","Thread-Topic":"[ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","Thread-Index":"AQHTJiizNzCJ8+cj4EuMMlhNVpa+HKKrOF8AgAQFvoCAAUSgAIACHl+A///02QCAAYCtgIAASOOA","Date":"Thu, 14 Sep 2017 19:35:18 +0000","Message-ID":"","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<20170908164849.GH7356@vergenet.net>\n\t<20170911061425.GN9736@yliu-home>\n\t<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>\n\t<20170913095730.GE27555@vergenet.net>\n\t\n\t<3edc7d93b8c94c3c8e316b299981b984@napatech.com>","In-Reply-To":"<3edc7d93b8c94c3c8e316b299981b984@napatech.com>","Accept-Language":"en-US","Content-Language":"en-US","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","user-agent":"Microsoft-MacOutlook/f.25.0.170815","authentication-results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=onevmw.onmicrosoft.com\n\theader.i=@onevmw.onmicrosoft.com header.b=\"ksTmU1EC\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=dball@vmware.com; "],"x-originating-ip":"[73.162.236.45]","x-ms-publictraffictype":"Email","x-microsoft-exchange-diagnostics":"1; BLUPR05MB627;\n\t20:ikXmfVfsf8C4HFXeSByIBsA1NkbOzMZJwp4RJU56wpTEidDQDTj2QwwduGIrEWxOPXzb93EOyHAALvaYTiuUW6Q2he+dVtmL0Y9rL7qGOg7CSODclmblS+/G+wrBfPbkAgHoonlqRWjdbns1rZ5zTrvp3FpAIVxSFzFOJO7uygI=","x-ms-exchange-antispam-srfa-diagnostics":"SSOS;","x-ms-office365-filtering-correlation-id":"a93e55e3-11c7-47d4-19e2-08d4fba7bb74","x-microsoft-antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(300000502095)(300135100095)(22001)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:BLUPR05MB627; ","x-ms-traffictypediagnostic":"BLUPR05MB627:","x-exchange-antispam-report-test":"UriScan:(10436049006162)(216315784871565);","x-microsoft-antispam-prvs":"","x-exchange-antispam-report-cfa-test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(10201501046)(3002001)(100000703101)(100105400095)(93006095)(93001095)(6041248)(20161123555025)(20161123562025)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:BLUPR05MB627; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:BLUPR05MB627; ","x-forefront-prvs":"0430FA5CB7","x-forefront-antispam-report":"SFV:NSPM;\n\tSFS:(10009020)(6009001)(376002)(346002)(39860400002)(24454002)(13464003)(189002)(377454003)(54094003)(199003)(6116002)(2906002)(3846002)(66066001)(102836003)(5660300001)(99286003)(97736004)(68736007)(2900100001)(7736002)(3660700001)(6486002)(305945005)(6436002)(77096006)(189998001)(25786009)(6916009)(83506001)(2950100002)(83716003)(53546010)(58126008)(50986999)(101416001)(110136004)(561944003)(316002)(6306002)(6506006)(6512007)(966005)(82746002)(53936002)(105586002)(8676002)(575784001)(86362001)(4326008)(3280700002)(106356001)(54356999)(8936002)(76176999)(93886005)(478600001)(33656002)(81166006)(36756003)(6246003)(81156014)(229853002)(14454004);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR05MB627;\n\tH:BLUPR05MB611.namprd05.prod.outlook.com; FPR:; SPF:None;\n\tPTR:InfoNoRecords; MX:1; A:1; LANG:en; ","received-spf":"None (protection.outlook.com: vmware.com does not designate\n\tpermitted sender hosts)","spamdiagnosticoutput":"1:99","spamdiagnosticmetadata":"NSPM","Content-ID":"","MIME-Version":"1.0","X-OriginatorOrg":"vmware.com","X-MS-Exchange-CrossTenant-originalarrivaltime":"14 Sep 2017 19:35:18.7112\n\t(UTC)","X-MS-Exchange-CrossTenant-fromentityheader":"Hosted","X-MS-Exchange-CrossTenant-id":"b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"BLUPR05MB627","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1769027,"web_url":"http://patchwork.ozlabs.org/comment/1769027/","msgid":"<20170915081842.GH2050@yliu-home>","list_archive_url":null,"date":"2017-09-15T08:18:42","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":72215,"url":"http://patchwork.ozlabs.org/api/people/72215/","name":"Yuanhan Liu","email":"yliu@fridaylinux.org"},"content":"On Wed, Sep 13, 2017 at 04:17:37PM +0000, Darrell Ball wrote:\n> \n> \n> On 9/13/17, 2:57 AM, \"Simon Horman\" wrote:\n> \n> On Tue, Sep 12, 2017 at 08:36:19AM +0000, Darrell Ball wrote:\n> > \n> > On 9/10/17, 11:14 PM, \"ovs-dev-bounces@openvswitch.org on behalf of Yuanhan Liu\" wrote:\n> > \n> > On Fri, Sep 08, 2017 at 06:48:50PM +0200, Simon Horman wrote:\n> > > On Tue, Sep 05, 2017 at 05:22:59PM +0800, Yuanhan Liu wrote:\n> > > > From: Finn Christensen \n> > > > \n> > > > AFAIK, most (if not all) NICs (including Mellanox and Intel) do not\n> > > > support a pure MARK action. It's required to be used together with\n> > > > some other actions, like QUEUE.\n> > > > \n> > > > To workaround it, retry with a queue action when first try failed.\n> > > > \n> > > > Moreover, some Intel's NIC (say XL710) needs the QUEUE action set\n> > > > before the MARK action.\n> > > > \n> > > > Co-authored-by: Yuanhan Liu \n> > > > Signed-off-by: Finn Christensen \n> > > > Signed-off-by: Yuanhan Liu \n> > > \n> > > This feels a bit like the tail wagging the dog.\n> > > Is this the lowest level at which it makes sense to implement\n> > > this logic?\n> > > \n> > > If so then I wonder if some sort of probing would be in order\n> > > to avoid the cost of trying to add the flow twice to hardware\n> > > where the queue is required.\n> > \n> > Do you mean something like rte_flow capability query, like whether\n> > a queue action is needed for a mark action? If so, yes, I do think\n> > we miss an interface like this.\n> > \n> > Note that even in this solution, the flow won't be created twice\n> > to the hardware, because the first try would be failed.\n> > \n> > [Darrell]\n> > \n> > Having an api to quey capability and avoid the first try to HW would be nice, but there are dependencies\n> > on RTE, drivers etc and I don’t know definitive the api would be.\n> > \n> > Also, as nics are added this capability needs to be done and state needs to be kept in all cases.\n> > \n> > It is an enhancement and if done should be reliable.\n> \n> Agreed. Though I was more thinking of probing the hardware rather than\n> having a capability API - I expect this would remove several of the\n> dependencies you describe above.\n> \n> \n> [Darrell] I have been pondering the probing option as well. It is certainly a valid option; \n\nAfter thinking twice, I do think the probing might be better than capability\nfeedback in some cases. For example, it makes more sense to probe the \"mark\nand queue\" actions than asking DPDK to provide such ability. It's more like\na combination, which is hard to give from DPDK side.\n\nBut for some cases, like do the underlaying NIC support a (few) specifics\nactions, or a (few) protocols, it's more clean and easier to provide cap\nfeedback API from DPDK, IMO.\n\n> we use it in other cases such as datapath probing. One of the aspects that worries me here is\n> maintaining the correct per interface (essentially; although the attribute is per nic) state\n> across various events such as new ports being added, vswitchd restarts, races with flow\n> creation. It would be non-trivial I guess and probably appropriate for the next patch series, if done.\n\nI would think so. I will think about it after this patchset, to see which one\nsuits better for our case. Or maybe, we could have both: I do think it makes\nsense to let DPDK to provide some basic capability feedbacks, for example,\nthe supported actions, protocols, etc.\n\n> In this case, we have what seems like a clear distinction b/w Napatech which does not need the\n> queue action workaround and everything else, which does.\n> Besides the non-Napatech behavior, which is worrisome, maintaining the difference for flow handling\n> under the covers is concerning.\n> \n> I wonder if we should be upfront as possible here and just have a dpdk interface configuration – maybe\n> something like “supports native HWOL mark action” since the better behavior is the exception?\n> The interface config would be more robust than probing.\n> This would need documentation, of course.\n\nThe thing is that the option is fixed, while the NIC driver may change.\nSuch option may apply to old versions may not apply to newer versions.\n\n> I think anyways we need documentation describing the difference b/w nics in the dpdk documentation (howto part).\n\nIIRC, there was already an ask from the DPDK mailing list.\n\n\t--yliu\n\n> Assuming no such enhancement is appropriate at this time I would\n> still like to ask if this is the best place for this hardware-specific code?\n> \n> [Darrell]\n> \n> For OVS, the netdev-dpdk layer is the lowest layer.\n> This kind of workaround is hard to hide, since we are messing with the rxq, so I think OVS needs to know\n> that it is in effect anyways. An alternative is to supply a mark and an ‘optional queue’ and let the driver decide if the queue is\n> needed and report back whether it was. This would be hard to do across various drivers. Supporting in the rte layer would require both\n> rte and driver support, so even more support.\n> \n> \n> \n> > A separate comment is we need to document which nics need the queue action.\n> > \n> > Also, I think we should check errno in the present code.\n> \n>","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=fridaylinux-org.20150623.gappssmtp.com\n\theader.i=@fridaylinux-org.20150623.gappssmtp.com\n\theader.b=\"aVE0MOCW\"; dkim-atps=neutral"],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtpFv2MXFz9sPr\n\tfor ;\n\tFri, 15 Sep 2017 18:18:57 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id 175ED481;\n\tFri, 15 Sep 2017 08:18:52 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id A5C6C82\n\tfor ; Fri, 15 Sep 2017 08:18:50 +0000 (UTC)","from mail-pg0-f53.google.com (mail-pg0-f53.google.com\n\t[74.125.83.53])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id 1FB961CE\n\tfor ; Fri, 15 Sep 2017 08:18:49 +0000 (UTC)","by mail-pg0-f53.google.com with SMTP id 188so1128604pgb.2\n\tfor ; Fri, 15 Sep 2017 01:18:49 -0700 (PDT)","from yliu-home ([45.63.61.64]) by smtp.gmail.com with ESMTPSA id\n\tv2sm1036407pgo.38.2017.09.15.01.18.46\n\t(version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128);\n\tFri, 15 Sep 2017 01:18:47 -0700 (PDT)"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=fridaylinux-org.20150623.gappssmtp.com; s=20150623;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:content-transfer-encoding:in-reply-to\n\t:user-agent; bh=xq3JZ/Ldmn5NHcCElKFQEndqvGmCeHngKN+jlSaUwUU=;\n\tb=aVE0MOCW/PCfPnDcW1bzN97BudP5L8G3vrJHDLaFYVtVH2+SCxshwwxiewzGfw69Iw\n\ta0i5/nH/gI535AZOUuOL2WSWrf0Hvoo55Sdi65qLhRvKZhqgUe+EXEyTGOZ+V6BaVKlz\n\tUnoeKbmfdgoJ6jeFRndubvS2bijv4/hIpRwPZ00BJ4rmzS8SiRfRAmV8EoE9fgFIAhFO\n\t8PGH2FDWHVY1ohAiSXHviKaOBLXX6/uGm4z8kC9B/xdMwpW8KUkdtw/pYXIfS5oTW1Ca\n\tEmB2/Ufqy9mK/6b2siwJ7ls3KUxdi4Qn35izv3540CEQ/TYqhjdThpMGeuPwjSCspQTk\n\tWTxg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:content-transfer-encoding\n\t:in-reply-to:user-agent;\n\tbh=xq3JZ/Ldmn5NHcCElKFQEndqvGmCeHngKN+jlSaUwUU=;\n\tb=I6SbkBuklX7KAtIIJEne935hIlADgFDYh2oG6JRcs3ShdfPRYpK21F0qgxM8v1Rq3n\n\tNk1tL/v87/phkArE2lUPhO0QjQrSvdI4xWyGEF6ySF/MYjtbU6YW1O2SgTJixJw5eQZn\n\tBxPabP0gZs1zdOFi5/pPEjJA9ZGjsO7BZiS0f4OkXeaCeCN6x+4WftwP5BHCw2EF0Gsw\n\tikRomu7e6/GovZbW7yPjBtwc13o+4eQUybX3nNbEpPpO0OttB3y0bhCp/QZdbgnLBMkD\n\tc/J2bA7zGUv8ZQc/FO8A8bYnEdgFg690VnWkrFX8LKNEllh1O4FEMh3hBGbvM5gH1lv/\n\tV3YQ==","X-Gm-Message-State":"AHPjjUiCJN9BNb3mlEC/RAtbj6NXSet+drU2SnbfPGr8N9rcLCmckY1S\n\t5KCDloBlkZwAPxPa","X-Google-Smtp-Source":"ADKCNb7OtmAKuQ91UMh6DjjeciGUA+2fmA+61I3T3KQM4FflmPqjWTAr4CwpG9wjVCawvf06wdksyQ==","X-Received":"by 10.99.186.3 with SMTP id k3mr23601474pgf.149.1505463529493;\n\tFri, 15 Sep 2017 01:18:49 -0700 (PDT)","Date":"Fri, 15 Sep 2017 16:18:42 +0800","From":"Yuanhan Liu ","To":"Darrell Ball ","Message-ID":"<20170915081842.GH2050@yliu-home>","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<20170908164849.GH7356@vergenet.net>\n\t<20170911061425.GN9736@yliu-home>\n\t<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>\n\t<20170913095730.GE27555@vergenet.net>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ,\n\tSimon Horman ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1769040,"web_url":"http://patchwork.ozlabs.org/comment/1769040/","msgid":"<20170915083419.GI2050@yliu-home>","list_archive_url":null,"date":"2017-09-15T08:34:19","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":72215,"url":"http://patchwork.ozlabs.org/api/people/72215/","name":"Yuanhan Liu","email":"yliu@fridaylinux.org"},"content":"On Tue, Sep 12, 2017 at 08:36:19AM +0000, Darrell Ball wrote:\n> \n> On 9/10/17, 11:14 PM, \"ovs-dev-bounces@openvswitch.org on behalf of Yuanhan Liu\" wrote:\n> \n> On Fri, Sep 08, 2017 at 06:48:50PM +0200, Simon Horman wrote:\n> > On Tue, Sep 05, 2017 at 05:22:59PM +0800, Yuanhan Liu wrote:\n> > > From: Finn Christensen \n> > > \n> > > AFAIK, most (if not all) NICs (including Mellanox and Intel) do not\n> > > support a pure MARK action. It's required to be used together with\n> > > some other actions, like QUEUE.\n> > > \n> > > To workaround it, retry with a queue action when first try failed.\n> > > \n> > > Moreover, some Intel's NIC (say XL710) needs the QUEUE action set\n> > > before the MARK action.\n> > > \n> > > Co-authored-by: Yuanhan Liu \n> > > Signed-off-by: Finn Christensen \n> > > Signed-off-by: Yuanhan Liu \n> > \n> > This feels a bit like the tail wagging the dog.\n> > Is this the lowest level at which it makes sense to implement\n> > this logic?\n> > \n> > If so then I wonder if some sort of probing would be in order\n> > to avoid the cost of trying to add the flow twice to hardware\n> > where the queue is required.\n> \n> Do you mean something like rte_flow capability query, like whether\n> a queue action is needed for a mark action? If so, yes, I do think\n> we miss an interface like this.\n> \n> Note that even in this solution, the flow won't be created twice\n> to the hardware, because the first try would be failed.\n> \n> [Darrell]\n> \n> Having an api to quey capability and avoid the first try to HW would be nice, but there are dependencies\n> on RTE, drivers etc and I don’t know definitive the api would be.\n> \n> Also, as nics are added this capability needs to be done and state needs to be kept in all cases.\n> \n> It is an enhancement and if done should be reliable.\n> \n> A separate comment is we need to document which nics need the queue action.\n> \n> Also, I think we should check errno in the present code.\n\n\nThat was my first reaction to remove such blind re-try. It could have\nbeen a good option if all the PMD driver report the error consistenly.\n\nAnd unfortunately, it's not true. For example, for MARK without QUEUE\naction, i40e reports RTE_FLOW_ERROR_TYPE_ACTION, which, IMO, is nothing\nwrong. While for mlx5, it reports RTE_FLOW_ERROR_TYPE_HANDLE. I don't\nknow why it was set like this, and we may could fix this. But my point\nwas, it's not that reliable to use rte_errno, at least it's true for now.\n\n\t--yliu","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=fridaylinux-org.20150623.gappssmtp.com\n\theader.i=@fridaylinux-org.20150623.gappssmtp.com\n\theader.b=\"UDV2jEzj\"; dkim-atps=neutral"],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtpbr12Dcz9t2f\n\tfor ;\n\tFri, 15 Sep 2017 18:34:31 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id BF0DA978;\n\tFri, 15 Sep 2017 08:34:28 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id CD452970\n\tfor ; Fri, 15 Sep 2017 08:34:27 +0000 (UTC)","from mail-pg0-f44.google.com (mail-pg0-f44.google.com\n\t[74.125.83.44])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id 443FD18A\n\tfor ; Fri, 15 Sep 2017 08:34:27 +0000 (UTC)","by mail-pg0-f44.google.com with SMTP id c137so1134450pga.11\n\tfor ; Fri, 15 Sep 2017 01:34:27 -0700 (PDT)","from yliu-home ([45.63.61.64]) by smtp.gmail.com with ESMTPSA id\n\to30sm1060445pgc.83.2017.09.15.01.34.24\n\t(version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128);\n\tFri, 15 Sep 2017 01:34:25 -0700 (PDT)"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=fridaylinux-org.20150623.gappssmtp.com; s=20150623;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:content-transfer-encoding:in-reply-to\n\t:user-agent; bh=vQiDjJMnQNlcerLd/KmPC8dfxhQDii5OaIhw/CM1uMI=;\n\tb=UDV2jEzjjHBw+kGcmzPFtx+lfWkHyOcC8jatDwVNlMMFzBWv6pE+qqszNp7Wf/qVXO\n\tLrUtEvJwFNYpCpeptpK9saamJOoujB2YD33fz7/IAloBQZmQe7Sv0wxokbi7mUhJn6yd\n\t+LM+Q4XjBzFwGtBjcLlvFUmWZK8FqTnn7j/SzW3Qv88nfCTQjOLic51LA7g5JECJXgyu\n\toXvNummw+E8eWJHV48gHG7LYdWqLTLl+FJUvR1viAsg4gfe5IPSXlUNpkoKI19B3wZET\n\tRs2aScRkI2TbCbOncf2WDKJbKgAmfJgu4/lnwQ5kFVX/DFt/pZtzkf62wMiCRFV9HCaX\n\tktkA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:content-transfer-encoding\n\t:in-reply-to:user-agent;\n\tbh=vQiDjJMnQNlcerLd/KmPC8dfxhQDii5OaIhw/CM1uMI=;\n\tb=qqS1uV15utC/nK8X9soLzHU11TrTZdgJJigou4mZAGwW5Lu8VnUlcFAgksQ9Ms8SIB\n\tLoTSZw7emPYbOFJMvxcgldrQ3o23LQ8+vne/B5xfaoLT/fU1Xn4Drq20GSd5Y+JIQ7pO\n\tHk+6tAc1FuLlf3Jrhtp+mOhVPRJWs6MFpnFCLWwVOsWi9KfytOdoC5rqVf9b+OW2td7q\n\tI1UCPen1vbqbSpkWqu5CedIE766lrCHacVpZjk96I2t/SaVLCokspnTB8Q7qD9YbHeXZ\n\tKhruTDvgVpCabDsolhqRo6bBNxSYsscR219MxA7syf8sj8BE75jAigzhqYory7chFuCM\n\tpuvw==","X-Gm-Message-State":"AHPjjUgeAfP9RjrOw0wOuCaPL+z3k5KJyTJZ+ipXTJ6nCNQIhhCZXnl8\n\tLbaLWZtNr5U6TL6D","X-Google-Smtp-Source":"ADKCNb5Q7f9fht2saMcYjqaSbfP+csDpmzWPGSL3D8vV/qtoHzH7/+r5yW5RlAxLa+akb0ICqA3RBQ==","X-Received":"by 10.84.129.98 with SMTP id 89mr27217502plb.383.1505464466891; \n\tFri, 15 Sep 2017 01:34:26 -0700 (PDT)","Date":"Fri, 15 Sep 2017 16:34:19 +0800","From":"Yuanhan Liu ","To":"Darrell Ball ","Message-ID":"<20170915083419.GI2050@yliu-home>","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<20170908164849.GH7356@vergenet.net>\n\t<20170911061425.GN9736@yliu-home>\n\t<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-Spam-Status":"No, score=0.5 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE,\n\tRCVD_IN_SORBS_SPAM autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ,\n\tSimon Horman ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1769067,"web_url":"http://patchwork.ozlabs.org/comment/1769067/","msgid":"","list_archive_url":null,"date":"2017-09-15T09:15:52","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":72305,"url":"http://patchwork.ozlabs.org/api/people/72305/","name":"Finn Christensen","email":"fc@napatech.com"},"content":"-----Original Message-----\r\n From: Darrell Ball [mailto:dball@vmware.com]\r\n Sent: 14. september 2017 21:35\r\n To: Finn Christensen \r\n Cc: dev@openvswitch.org\r\n Subject: Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue\r\n action\r\n \r\n \r\n \r\n On 9/14/17, 1:14 AM, \"Finn Christensen\" wrote:\r\n \r\n \r\n \r\n \r\n \r\n -----Original Message-----\r\n \r\n From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev-\r\n bounces@openvswitch.org] On Behalf Of Darrell Ball\r\n \r\n Sent: 13. september 2017 18:18\r\n \r\n To: Simon Horman \r\n \r\n Cc: dev@openvswitch.org\r\n \r\n Subject: Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue\r\n action\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n On 9/13/17, 2:57 AM, \"Simon Horman\"\r\n wrote:\r\n \r\n \r\n \r\n On Tue, Sep 12, 2017 at 08:36:19AM +0000, Darrell Ball wrote:\r\n \r\n >\r\n \r\n > On 9/10/17, 11:14 PM, \"ovs-dev-bounces@openvswitch.org on\r\n behalf of Yuanhan Liu\" wrote:\r\n \r\n >\r\n \r\n > On Fri, Sep 08, 2017 at 06:48:50PM +0200, Simon Horman wrote:\r\n \r\n > > On Tue, Sep 05, 2017 at 05:22:59PM +0800, Yuanhan Liu wrote:\r\n \r\n > > > From: Finn Christensen \r\n \r\n > > >\r\n \r\n > > > AFAIK, most (if not all) NICs (including Mellanox and Intel) do\r\n not\r\n \r\n > > > support a pure MARK action. It's required to be used together\r\n with\r\n \r\n > > > some other actions, like QUEUE.\r\n \r\n > > >\r\n \r\n > > > To workaround it, retry with a queue action when first try\r\n failed.\r\n \r\n > > >\r\n \r\n > > > Moreover, some Intel's NIC (say XL710) needs the QUEUE\r\n action set\r\n \r\n > > > before the MARK action.\r\n \r\n > > >\r\n \r\n > > > Co-authored-by: Yuanhan Liu \r\n \r\n > > > Signed-off-by: Finn Christensen \r\n \r\n > > > Signed-off-by: Yuanhan Liu \r\n \r\n > >\r\n \r\n > > This feels a bit like the tail wagging the dog.\r\n \r\n > > Is this the lowest level at which it makes sense to implement\r\n \r\n > > this logic?\r\n \r\n > >\r\n \r\n > > If so then I wonder if some sort of probing would be in order\r\n \r\n > > to avoid the cost of trying to add the flow twice to hardware\r\n \r\n > > where the queue is required.\r\n \r\n >\r\n \r\n > Do you mean something like rte_flow capability query, like\r\n whether\r\n \r\n > a queue action is needed for a mark action? If so, yes, I do think\r\n \r\n > we miss an interface like this.\r\n \r\n >\r\n \r\n > Note that even in this solution, the flow won't be created twice\r\n \r\n > to the hardware, because the first try would be failed.\r\n \r\n >\r\n \r\n > [Darrell]\r\n \r\n >\r\n \r\n > Having an api to quey capability and avoid the first try to HW\r\n would be nice, but there are dependencies\r\n \r\n > on RTE, drivers etc and I don’t know definitive the api would\r\n be.\r\n \r\n >\r\n \r\n > Also, as nics are added this capability needs to be done and\r\n state needs to be kept in all cases.\r\n \r\n >\r\n \r\n > It is an enhancement and if done should be reliable.\r\n \r\n \r\n \r\n Agreed. Though I was more thinking of probing the hardware rather\r\n than\r\n \r\n having a capability API - I expect this would remove several of the\r\n \r\n dependencies you describe above.\r\n \r\n \r\n \r\n \r\n \r\n [Darrell] I have been pondering the probing option as well. It is certainly a\r\n valid option; we use it in other cases such as datapath probing. One of the\r\n aspects that worries me here is maintaining the correct per interface\r\n (essentially; although the attribute is per nic) state across various events\r\n such as new ports being added, vswitchd restarts, races with flow creation.\r\n It would be non-trivial I guess and probably appropriate for the next patch\r\n series, if done.\r\n \r\n \r\n \r\n In this case, we have what seems like a clear distinction b/w Napatech\r\n which does not need the queue action workaround and everything else,\r\n which does.\r\n \r\n Besides the non-Napatech behavior, which is worrisome, maintaining the\r\n difference for flow handling under the covers is concerning.\r\n \r\n \r\n \r\n I wonder if we should be upfront as possible here and just have a dpdk\r\n interface configuration – maybe something like “supports native HWOL\r\n mark action” since the better behavior is the exception?\r\n \r\n The interface config would be more robust than probing.\r\n \r\n This would need documentation, of course.\r\n \r\n \r\n \r\n [Finn]\r\n \r\n I think the rte queue action should never be used here when using\r\n partial HWOL. Not the way OVS handles multi queues today.\r\n \r\n Maybe a \"default queue\" could be used in the dpdk PMDs when no\r\n queue is specified in rte flow?\r\n \r\n [Darrell] This is the Napatech case, where no queue action is needed; you\r\n are suggesting programming a default queue in this case.\r\n I don’t follow how this would be helpful/desired?\r\n\r\n[Finn]\r\nI was trying to make my view on this, not particularly arguing for the Napatech \r\nCase.\r\nHere is what I was thinking:\r\nTaking the case of this partial HWOL, then we are trying to offload the flow\r\nclassification to HW, like \"pre-classify and mark\". Then this mark is used to \r\naccelerate OVS while finding the actions to execute. Since we do leave all \r\nprocessing of the actions to OVS, there is no way for the partial HWOL to \r\nknow, at rte flow creation time, where to send the pre-classified packets \r\n(which is strictly needed when seen from a DPDK rte flow point of view).\r\nWhen multiple queues are specified in OVS, a hash splitting mechanism \r\nis used in the nic to support RSS. Then the nic is responsible for selecting the\r\nright queue according to the configured algorithm for RSS.\r\nOVS only needs to know how many queues to service per port. No knowledge\r\nabout the association b/w flow <-> rxq is used in OVS today.\r\nWhen looking at full HWOL, all flow actions will have to be interpreted, supported \r\nand programmed to NIC using rte flow - send directly to target port and we \r\ndo not have this issue.\r\nHave I missed something?\r\n \r\n Essentially, this is a mismatch between the rte flow impl functionality in\r\n PMD and the needs in OVS for flow classification, in a partial HWOL setup.\r\n \r\n \r\n [Darrell] Maybe this has been explored, but were additional workaround\r\n actions, like RTE_FLOW_ACTION_TYPE_RSS considered ?\r\n I guess we could make this essentially a NOP, but if mark action\r\n could ride along with it, then that would be good.\r\n\r\n[Finn]\r\nNo. We have not explored that.\r\nI'm not sure I understand how you would use RTE_FLOW_ACTION_TYPE_RSS\r\nIn this situation.\r\n\r\n \r\n However, we would not mind Darells proposal, it makes sense since\r\n most nic PMDs unfortunately needs this and it is only relevant for partial\r\n HWOL. We are mostly concerned with full HWOL and therefore see partial\r\n HWOL as a failover when full HWOL could not be handled in nic, if enabled.\r\n \r\n \r\n \r\n \r\n \r\n I think anyways we need documentation describing the difference b/w\r\n nics in the dpdk documentation (howto part).\r\n \r\n \r\n \r\n \r\n \r\n Assuming no such enhancement is appropriate at this time I would\r\n \r\n still like to ask if this is the best place for this hardware-specific code?\r\n \r\n \r\n \r\n [Darrell]\r\n \r\n \r\n \r\n For OVS, the netdev-dpdk layer is the lowest layer.\r\n \r\n This kind of workaround is hard to hide, since we are messing with the\r\n rxq, so I think OVS needs to know that it is in effect anyways. An\r\n alternative is to supply a mark and an ‘optional queue’ and let the driver\r\n decide if the queue is needed and report back whether it was. This would\r\n be hard to do across various drivers. Supporting in the rte layer would\r\n require both rte and driver support, so even more support.\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n > A separate comment is we need to document which nics need\r\n the queue action.\r\n \r\n >\r\n \r\n > Also, I think we should check errno in the present code.\r\n \r\n \r\n \r\n \r\n \r\n _______________________________________________\r\n \r\n dev mailing list\r\n \r\n dev@openvswitch.org\r\n \r\n https://urldefense.proofpoint.com/v2/url?u=https-\r\n 3A__mail.openvswitch.org_mailman_listinfo_ovs-\r\n 2Ddev&d=DwIGaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-\r\n uZnsw&m=IHypHavCy0AKjNxqOMyc4w3ILyC-\r\n BuwkB8fuVvQUA3k&s=deJQWP9KI22Xp46tEoZ6o6Emitr3Bhfd7iSMxNpude\r\n g&e=\r\n \r\n Disclaimer: This email and any files transmitted with it may contain\r\n confidential information intended for the addressee(s) only. The\r\n information is not to be surrendered or copied to unauthorized persons. If\r\n you have received this communication in error, please notify the sender\r\n immediately and delete this e-mail from your system.","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=napatech.com header.i=@napatech.com\n\theader.b=\"DjWS+q50\"; dkim-atps=neutral"],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtqWk2KTpz9s4q\n\tfor ;\n\tFri, 15 Sep 2017 19:16:02 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id 49EB498C;\n\tFri, 15 Sep 2017 09:16:00 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 96442970\n\tfor ; Fri, 15 Sep 2017 09:15:58 +0000 (UTC)","from mail02.napatech.com (mail02.napatech.com [188.120.77.119])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id 010FC1E8\n\tfor ; Fri, 15 Sep 2017 09:15:56 +0000 (UTC)","from cph-gen-exch02.napatech.com (10.240.1.84) by\n\tcph-gen-exch02.napatech.com (10.240.1.84) with Microsoft SMTP Server\n\t(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id\n\t15.1.1034.26; Fri, 15 Sep 2017 11:15:52 +0200","from cph-gen-exch02.napatech.com ([fe80::581:51a1:ac3f:84e]) by\n\tcph-gen-exch02.napatech.com ([fe80::581:51a1:ac3f:84e%12]) with mapi\n\tid 15.01.1034.026; Fri, 15 Sep 2017 11:15:52 +0200"],"X-Greylist":"domain auto-whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=napatech.com; i=@napatech.com; q=dns/txt; s=mar2017;\n\tt=1505466957; x=1537002957;\n\th=from:to:cc:subject:date:message-id:references:\n\tin-reply-to:content-transfer-encoding:mime-version;\n\tbh=PbKQOu21ZgnUJ9xqKUpSb4TmCqbG8Z++f6dp01+la6U=;\n\tb=DjWS+q50wcL6Geg+jRnzWqFolvEFZxl4DzJySbWsx025ec+qzTB4DzT6\n\tgjNGTS36XNLIHkz4ZP7FWBM7t3btPN8+ZBajYC2nrksZLWR+wQhK8eGo7\n\tcPWUOhIeKKdtss6sTHC+dYXPmR3pB1YO0KgjEewe/AdTLpEuxmM8kt52a\n\tkWVmgEmdOONS4QBS47pz9/RLNodYc/DAqId8GGJzph9AfX4d+qnApmSXX\n\t2PkW7f8iMRgv/ha9SGUVaVIQgaYW3/3ijssLv5vvYmR1WQOzJR5QbGUGS\n\t+5SdqdqkSyr243yGnunYYWDfmN6K3J9UTsU3pTHokXRzST8Lq4yf61ofK A==;","IronPort-PHdr":"9a23:7kMsYBT1OM8wVQZw6QEbEt6CONpsv+yvbD5Q0YIujvd0So/mwa6zZBaN2/xhgRfzUJnB7Loc0qyN4vGmBTxLuM/JmUtBWaQEbwUCh8QSkl5oK+++Imq/EsTXaTcnFt9JTl5v8iLzG0FUHMHjew+a+SXqvnYdFRrlKAV6OPn+FJLMgMSrzeCy/IDYbxlViDanb75/KBG7oR/eu8QVjoduN7s9wQbVr3VVfOhb2XlmLk+JkRbm4cew8p9j8yBOtP8k6sVNT6b0cbkmQLJBFDgpPHw768PttRnYUAuA/WAcXXkMkhpJGAfK8hf3VYrsvyTgt+p93C6aPdDqTb0xRD+v4btnRAPuhSwaLDMy7n3ZhdJsg6JauBKhpgJww4jIYIGOKfFyerrRcc4GSWZdW8pcUTFKDIGhYIsVF+cPPfhWoZThp1UArhW+CwujBOLzxTFHiXD7xrE63P8jEQ3awAAsA9ADvXLJp9v1LqcSVuW1wbHIwzXCafNW3yr25ZbIchA7oPGMRq5wftTXyUk0CQzFiEibpIvrPzyJzekNtXKU7/J6WuKzlWEotwFxriKzyccrj4nEn4QYwU3K+yV+xYY6P9y4SEhjbN6jCJtfqSeaN5VtQsIsQmFopDo1yr0ctZ68ZigKx4wrxwbFa/yAdIiI7ArjVOGQITd+mHJpYq6whxG38UWm1+byVdG03U5XoiZZiNXAqH8A2wDJ5sSaUPdw/Uis1S6S2wzP8O1IP085mbbBJ5I83rI8jIQfvErHEyPulkX5kqybelkh9+Wt6+nqYajqq5qcOoNpkA7yL6EjldajDuk2PAgDWmuW9Oui27Dl4Eb3Wq9FjucsnancqJ3aIMMbqbOnDAJNyYYj7gq/Dy+h0NQFgXkLNFJFdwyDj4juI1zOJer3Dfa7g1i2ljdk3ejGMaf9AprTMnfDkK3tcqp6605Z0AYzzNZf6IxICrwZPf7/RlX9uMLXAxMlKQC43vzrBdZy248GXGKAGK6ZMKfcsV+S4eIvJvGBZIEJtzvmLfgq/ebugmUlmVADYaap3YEbZ2y/HvRjO0mZe2bjgs8dEWcWuQozVPHlh0OcUTNIYHayR7wz5jclCIK9A4bDR5ytj6CB3CuhGZ1WfG9GWRiwFiLEfp+eVvMIIAibJsspxjAOXLylY5Ekyhi0uUnxzL8xfcTO/ShNm5Pl0pBe5/fSjg0/8yd5CYzJ3WqlSWhsl38FTD9w16d69x8ugmyf2LR11qQLXedY4OlEB0JjbcbR","X-IronPort-Anti-Spam-Filtered":"true","X-IronPort-Anti-Spam-Result":"A2ExAgDembtZ/1QB8ApdGwEBAQMBAQEJAQEBFQEBAQECAQEBAQgBAQEBhBOBFQeDboogkWV5klUIglEOgUFDCiOFGQIahE4YAQEBAQEBAQEBAQECgRBCEgGBXiQBDS8IAwwhCAMBAQEBAQEBAQEBAQEBAQFGAg9BAQEYAQEBAQIBIwQNOgsMBAIBCA4DBAEBAwIjAwICAjAUAQgIAgQOBQgTihAYq0qBbTqLMQEBAQEBAQEBAgEBAQEBAQEBARoFgQ6CHYExgiGBY4IbWDWDKIEUCQESAYMygmAFigOJFI1th1uDXIkVghyFaop7hxOKPYM1AgICAgkCGoE5H4E7C3cVSYUYHYFndoRFghqBI4EPAQEB","X-IPAS-Result":"A2ExAgDembtZ/1QB8ApdGwEBAQMBAQEJAQEBFQEBAQECAQEBAQgBAQEBhBOBFQeDboogkWV5klUIglEOgUFDCiOFGQIahE4YAQEBAQEBAQEBAQECgRBCEgGBXiQBDS8IAwwhCAMBAQEBAQEBAQEBAQEBAQFGAg9BAQEYAQEBAQIBIwQNOgsMBAIBCA4DBAEBAwIjAwICAjAUAQgIAgQOBQgTihAYq0qBbTqLMQEBAQEBAQEBAgEBAQEBAQEBARoFgQ6CHYExgiGBY4IbWDWDKIEUCQESAYMygmAFigOJFI1th1uDXIkVghyFaop7hxOKPYM1AgICAgkCGoE5H4E7C3cVSYUYHYFndoRFghqBI4EPAQEB","From":"Finn Christensen ","To":"Darrell Ball ","Thread-Topic":"[ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","Thread-Index":"AQHTJiizfJsWL3b3Rkev+6Ia92jE1aKrFtgAgAQFvoCAAbn6gIABqQWAgABqM4CAAD/isIABia4AgADnWpA=","Date":"Fri, 15 Sep 2017 09:15:52 +0000","Message-ID":"","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<20170908164849.GH7356@vergenet.net>\n\t<20170911061425.GN9736@yliu-home>\n\t<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>\n\t<20170913095730.GE27555@vergenet.net>\n\t\n\t<3edc7d93b8c94c3c8e316b299981b984@napatech.com>\n\t","In-Reply-To":"","Accept-Language":"da-DK, en-US","Content-Language":"en-US","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","x-originating-ip":"[10.240.10.239]","MIME-Version":"1.0","X-Spam-Status":"No, score=-0.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tDKIM_VALID_AU,RP_MATCHES_RCVD autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1769346,"web_url":"http://patchwork.ozlabs.org/comment/1769346/","msgid":"","list_archive_url":null,"date":"2017-09-15T18:09:24","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":68212,"url":"http://patchwork.ozlabs.org/api/people/68212/","name":"Darrell Ball","email":"dball@vmware.com"},"content":"On 9/15/17, 2:16 AM, \"Finn Christensen\" wrote:\r\n\r\n \r\n \r\n -----Original Message-----\r\n From: Darrell Ball [mailto:dball@vmware.com]\r\n Sent: 14. september 2017 21:35\r\n To: Finn Christensen \r\n Cc: dev@openvswitch.org\r\n Subject: Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue\r\n action\r\n \r\n \r\n \r\n On 9/14/17, 1:14 AM, \"Finn Christensen\" wrote:\r\n \r\n \r\n \r\n \r\n \r\n -----Original Message-----\r\n \r\n From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev-\r\n bounces@openvswitch.org] On Behalf Of Darrell Ball\r\n \r\n Sent: 13. september 2017 18:18\r\n \r\n To: Simon Horman \r\n \r\n Cc: dev@openvswitch.org\r\n \r\n Subject: Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue\r\n action\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n On 9/13/17, 2:57 AM, \"Simon Horman\"\r\n wrote:\r\n \r\n \r\n \r\n On Tue, Sep 12, 2017 at 08:36:19AM +0000, Darrell Ball wrote:\r\n \r\n >\r\n \r\n > On 9/10/17, 11:14 PM, \"ovs-dev-bounces@openvswitch.org on\r\n behalf of Yuanhan Liu\" wrote:\r\n \r\n >\r\n \r\n > On Fri, Sep 08, 2017 at 06:48:50PM +0200, Simon Horman wrote:\r\n \r\n > > On Tue, Sep 05, 2017 at 05:22:59PM +0800, Yuanhan Liu wrote:\r\n \r\n > > > From: Finn Christensen \r\n \r\n > > >\r\n \r\n > > > AFAIK, most (if not all) NICs (including Mellanox and Intel) do\r\n not\r\n \r\n > > > support a pure MARK action. It's required to be used together\r\n with\r\n \r\n > > > some other actions, like QUEUE.\r\n \r\n > > >\r\n \r\n > > > To workaround it, retry with a queue action when first try\r\n failed.\r\n \r\n > > >\r\n \r\n > > > Moreover, some Intel's NIC (say XL710) needs the QUEUE\r\n action set\r\n \r\n > > > before the MARK action.\r\n \r\n > > >\r\n \r\n > > > Co-authored-by: Yuanhan Liu \r\n \r\n > > > Signed-off-by: Finn Christensen \r\n \r\n > > > Signed-off-by: Yuanhan Liu \r\n \r\n > >\r\n \r\n > > This feels a bit like the tail wagging the dog.\r\n \r\n > > Is this the lowest level at which it makes sense to implement\r\n \r\n > > this logic?\r\n \r\n > >\r\n \r\n > > If so then I wonder if some sort of probing would be in order\r\n \r\n > > to avoid the cost of trying to add the flow twice to hardware\r\n \r\n > > where the queue is required.\r\n \r\n >\r\n \r\n > Do you mean something like rte_flow capability query, like\r\n whether\r\n \r\n > a queue action is needed for a mark action? If so, yes, I do think\r\n \r\n > we miss an interface like this.\r\n \r\n >\r\n \r\n > Note that even in this solution, the flow won't be created twice\r\n \r\n > to the hardware, because the first try would be failed.\r\n \r\n >\r\n \r\n > [Darrell]\r\n \r\n >\r\n \r\n > Having an api to quey capability and avoid the first try to HW\r\n would be nice, but there are dependencies\r\n \r\n > on RTE, drivers etc and I don’t know definitive the api would\r\n be.\r\n \r\n >\r\n \r\n > Also, as nics are added this capability needs to be done and\r\n state needs to be kept in all cases.\r\n \r\n >\r\n \r\n > It is an enhancement and if done should be reliable.\r\n \r\n \r\n \r\n Agreed. Though I was more thinking of probing the hardware rather\r\n than\r\n \r\n having a capability API - I expect this would remove several of the\r\n \r\n dependencies you describe above.\r\n \r\n \r\n \r\n \r\n \r\n [Darrell] I have been pondering the probing option as well. It is certainly a\r\n valid option; we use it in other cases such as datapath probing. One of the\r\n aspects that worries me here is maintaining the correct per interface\r\n (essentially; although the attribute is per nic) state across various events\r\n such as new ports being added, vswitchd restarts, races with flow creation.\r\n It would be non-trivial I guess and probably appropriate for the next patch\r\n series, if done.\r\n \r\n \r\n \r\n In this case, we have what seems like a clear distinction b/w Napatech\r\n which does not need the queue action workaround and everything else,\r\n which does.\r\n \r\n Besides the non-Napatech behavior, which is worrisome, maintaining the\r\n difference for flow handling under the covers is concerning.\r\n \r\n \r\n \r\n I wonder if we should be upfront as possible here and just have a dpdk\r\n interface configuration – maybe something like “supports native HWOL\r\n mark action” since the better behavior is the exception?\r\n \r\n The interface config would be more robust than probing.\r\n \r\n This would need documentation, of course.\r\n \r\n \r\n \r\n [Finn]\r\n \r\n I think the rte queue action should never be used here when using\r\n partial HWOL. Not the way OVS handles multi queues today.\r\n \r\n Maybe a \"default queue\" could be used in the dpdk PMDs when no\r\n queue is specified in rte flow?\r\n \r\n [Darrell] This is the Napatech case, where no queue action is needed; you\r\n are suggesting programming a default queue in this case.\r\n I don’t follow how this would be helpful/desired?\r\n \r\n [Finn]\r\n I was trying to make my view on this, not particularly arguing for the Napatech\r\n Case.\r\n Here is what I was thinking:\r\n Taking the case of this partial HWOL, then we are trying to offload the flow\r\n classification to HW, like \"pre-classify and mark\". Then this mark is used to\r\n accelerate OVS while finding the actions to execute. Since we do leave all\r\n processing of the actions to OVS, there is no way for the partial HWOL to\r\n know, at rte flow creation time, where to send the pre-classified packets\r\n (which is strictly needed when seen from a DPDK rte flow point of view).\r\n When multiple queues are specified in OVS, a hash splitting mechanism\r\n is used in the nic to support RSS. Then the nic is responsible for selecting the\r\n right queue according to the configured algorithm for RSS.\r\n OVS only needs to know how many queues to service per port. No knowledge\r\n about the association b/w flow <-> rxq is used in OVS today.\r\n When looking at full HWOL, all flow actions will have to be interpreted, supported\r\n and programmed to NIC using rte flow - send directly to target port and we\r\n do not have this issue.\r\n Have I missed something?\r\n\r\n[Darrell] \r\nI understand you points about the full offload you are working on.\r\nI just did not follow this one comment:\r\n\r\n“Maybe a \"default queue\" could be used in the dpdk PMDs when no\r\n queue is specified in rte flow?”\r\n\r\nwhich I thought is related to partial offloads used here and the receive queue\r\naction we are discussing?; I am concerned that I missed your point about this \r\nspecific comment ?\r\n\r\n \r\n Essentially, this is a mismatch between the rte flow impl functionality in\r\n PMD and the needs in OVS for flow classification, in a partial HWOL setup.\r\n \r\n \r\n [Darrell] Maybe this has been explored, but were additional workaround\r\n actions, like RTE_FLOW_ACTION_TYPE_RSS considered ?\r\n I guess we could make this essentially a NOP, but if mark action\r\n could ride along with it, then that would be good.\r\n \r\n [Finn]\r\n No. We have not explored that.\r\n I'm not sure I understand how you would use RTE_FLOW_ACTION_TYPE_RSS\r\n In this situation.\r\n\r\n\r\n[Darrell]\r\nSo there are two points here:\r\n1/ ‘IF’ RTE_FLOW_ACTION_TYPE_RSS can used in lieu of RTE_FLOW_ACTION_TYPE_QUEUE\r\ncombined with RTE_FLOW_ACTION_TYPE_MARK, as need for the workaround for most nics, then\r\nthis would be better for distributing packets across PMDs for the masked flows relevant here. This\r\nwould help mitigate one of the concerns. We could use the RSS we use globally, essentially making\r\nthe distribution a NOP compared with what we normally do today with RSS….\r\n\r\n2/ Assuming ‘1’, we might even use this in all cases as a first try for all nics, which if it could be\r\ndone solves the queue action workaround capability/probing thing?\r\n \r\n \r\n However, we would not mind Darells proposal, it makes sense since\r\n most nic PMDs unfortunately needs this and it is only relevant for partial\r\n HWOL. We are mostly concerned with full HWOL and therefore see partial\r\n HWOL as a failover when full HWOL could not be handled in nic, if enabled.\r\n \r\n \r\n \r\n \r\n \r\n I think anyways we need documentation describing the difference b/w\r\n nics in the dpdk documentation (howto part).\r\n \r\n \r\n \r\n \r\n \r\n Assuming no such enhancement is appropriate at this time I would\r\n \r\n still like to ask if this is the best place for this hardware-specific code?\r\n \r\n \r\n \r\n [Darrell]\r\n \r\n \r\n \r\n For OVS, the netdev-dpdk layer is the lowest layer.\r\n \r\n This kind of workaround is hard to hide, since we are messing with the\r\n rxq, so I think OVS needs to know that it is in effect anyways. An\r\n alternative is to supply a mark and an ‘optional queue’ and let the driver\r\n decide if the queue is needed and report back whether it was. This would\r\n be hard to do across various drivers. Supporting in the rte layer would\r\n require both rte and driver support, so even more support.\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n > A separate comment is we need to document which nics need\r\n the queue action.\r\n \r\n >\r\n \r\n > Also, I think we should check errno in the present code.\r\n \r\n \r\n \r\n \r\n \r\n _______________________________________________\r\n \r\n dev mailing list\r\n \r\n dev@openvswitch.org\r\n \r\n https://urldefense.proofpoint.com/v2/url?u=https-\r\n 3A__mail.openvswitch.org_mailman_listinfo_ovs-\r\n 2Ddev&d=DwIGaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-\r\n uZnsw&m=IHypHavCy0AKjNxqOMyc4w3ILyC-\r\n BuwkB8fuVvQUA3k&s=deJQWP9KI22Xp46tEoZ6o6Emitr3Bhfd7iSMxNpude\r\n g&e=\r\n \r\n Disclaimer: This email and any files transmitted with it may contain\r\n confidential information intended for the addressee(s) only. The\r\n information is not to be surrendered or copied to unauthorized persons. If\r\n you have received this communication in error, please notify the sender\r\n immediately and delete this e-mail from your system.\r\n \r\n \r\n \r\n Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system.","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=onevmw.onmicrosoft.com\n\theader.i=@onevmw.onmicrosoft.com header.b=\"SZFrnGL1\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=dball@vmware.com; "],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xv3ML4V4Mz9s0Z\n\tfor ;\n\tSat, 16 Sep 2017 04:09:32 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id DA988B1D;\n\tFri, 15 Sep 2017 18:09:29 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 2E091ABA\n\tfor ; Fri, 15 Sep 2017 18:09:28 +0000 (UTC)","from NAM03-BY2-obe.outbound.protection.outlook.com\n\t(mail-by2nam03on0063.outbound.protection.outlook.com [104.47.42.63])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id 2CE68442\n\tfor ; Fri, 15 Sep 2017 18:09:27 +0000 (UTC)","from BLUPR05MB611.namprd05.prod.outlook.com (10.141.204.27) by\n\tBLUPR05MB418.namprd05.prod.outlook.com (10.141.27.16) with Microsoft\n\tSMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id\n\t15.20.77.5; Fri, 15 Sep 2017 18:09:25 +0000","from BLUPR05MB611.namprd05.prod.outlook.com ([10.141.204.27]) by\n\tBLUPR05MB611.namprd05.prod.outlook.com ([10.141.204.27]) with mapi id\n\t15.20.0077.006; Fri, 15 Sep 2017 18:09:25 +0000"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=onevmw.onmicrosoft.com; s=selector1-vmware-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=EwPWw+LoXjL8ormHfFrqe5kkljS+IvnVNuIX2GbEQDI=;\n\tb=SZFrnGL16XI9jKo3v2ryOKEKaaYvLH+fRMacBjS1dzbs4/eoiG3xWVSK4kwz+uRiJluveoteK/ymRIPVaNZGGXxg+2fVxX0+FFvU2vEhVLxHTlmSqLvJJDCLZbCVILhz5TNMi+/fdesp0EyJ5QT6dMUvNSWJiu249AlaNjFKxTo=","From":"Darrell Ball ","To":"Finn Christensen ","Thread-Topic":"[ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","Thread-Index":"AQHTJiizNzCJ8+cj4EuMMlhNVpa+HKKrOF8AgAQFvoCAAUSgAIACHl+A///02QCAAYCtgIAASOOAgAFanQCAAB+5AA==","Date":"Fri, 15 Sep 2017 18:09:24 +0000","Message-ID":"","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<20170908164849.GH7356@vergenet.net>\n\t<20170911061425.GN9736@yliu-home>\n\t<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>\n\t<20170913095730.GE27555@vergenet.net>\n\t\n\t<3edc7d93b8c94c3c8e316b299981b984@napatech.com>\n\t\n\t","In-Reply-To":"","Accept-Language":"en-US","Content-Language":"en-US","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","user-agent":"Microsoft-MacOutlook/f.25.0.170815","authentication-results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=onevmw.onmicrosoft.com\n\theader.i=@onevmw.onmicrosoft.com header.b=\"SZFrnGL1\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=dball@vmware.com; "],"x-originating-ip":"[73.162.236.45]","x-ms-publictraffictype":"Email","x-microsoft-exchange-diagnostics":"1; BLUPR05MB418;\n\t20:mc807zXghIFpH7/bKw5L2SkBU65hqMDND7G236ZQV17jq8GkXOmGK0DjpuDUJ3bOxkLlv2yLeY+Ogoq2o3JTlMbj3xwCuSC/QNAuM6zGuM8gF+fQKJuCplVJ+XenPZxHAn2XJ10jZmGpSxD8t5XpeXdSAJeHELEN/oA3J43hd2M=","x-ms-exchange-antispam-srfa-diagnostics":"SSOS;","x-ms-office365-filtering-correlation-id":"78d82c94-2ac3-442e-a227-08d4fc64e5fa","x-microsoft-antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:BLUPR05MB418; ","x-ms-traffictypediagnostic":"BLUPR05MB418:","x-exchange-antispam-report-test":"UriScan:(61668805478150)(10436049006162)(216315784871565); ","x-microsoft-antispam-prvs":"","x-exchange-antispam-report-cfa-test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(10201501046)(3002001)(93006095)(93001095)(100000703101)(100105400095)(6041248)(20161123558100)(20161123560025)(20161123564025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:BLUPR05MB418; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:BLUPR05MB418; ","x-forefront-prvs":"0431F981D8","x-forefront-antispam-report":"SFV:NSPM;\n\tSFS:(10009020)(6009001)(376002)(346002)(199003)(54094003)(13464003)(189002)(24454002)(377454003)(229853002)(2906002)(33656002)(6436002)(77096006)(6486002)(6506006)(966005)(83716003)(53546010)(6916009)(2950100002)(5660300001)(4326008)(82746002)(101416001)(8936002)(8676002)(6246003)(6116002)(110136004)(106356001)(83506001)(3846002)(105586002)(102836003)(81156014)(81166006)(3660700001)(93886005)(97736004)(561944003)(3280700002)(2900100001)(68736007)(316002)(58126008)(14454004)(99286003)(25786009)(189998001)(575784001)(6512007)(7736002)(36756003)(305945005)(6306002)(86362001)(50986999)(66066001)(54356999)(53936002)(76176999)(478600001)(19627235001);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR05MB418;\n\tH:BLUPR05MB611.namprd05.prod.outlook.com; FPR:; SPF:None;\n\tPTR:InfoNoRecords; A:1; MX:1; LANG:en; ","received-spf":"None (protection.outlook.com: vmware.com does not designate\n\tpermitted sender hosts)","spamdiagnosticoutput":"1:99","spamdiagnosticmetadata":"NSPM","Content-ID":"","MIME-Version":"1.0","X-OriginatorOrg":"vmware.com","X-MS-Exchange-CrossTenant-originalarrivaltime":"15 Sep 2017 18:09:25.0013\n\t(UTC)","X-MS-Exchange-CrossTenant-fromentityheader":"Hosted","X-MS-Exchange-CrossTenant-id":"b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"BLUPR05MB418","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1769355,"web_url":"http://patchwork.ozlabs.org/comment/1769355/","msgid":"<64837D85-3898-403C-9F40-A0EDBB1448B1@vmware.com>","list_archive_url":null,"date":"2017-09-15T18:30:06","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":68212,"url":"http://patchwork.ozlabs.org/api/people/68212/","name":"Darrell Ball","email":"dball@vmware.com"},"content":"On 9/15/17, 1:34 AM, \"Yuanhan Liu\" wrote:\r\n\r\n On Tue, Sep 12, 2017 at 08:36:19AM +0000, Darrell Ball wrote:\r\n > \r\n > On 9/10/17, 11:14 PM, \"ovs-dev-bounces@openvswitch.org on behalf of Yuanhan Liu\" wrote:\r\n > \r\n > On Fri, Sep 08, 2017 at 06:48:50PM +0200, Simon Horman wrote:\r\n > > On Tue, Sep 05, 2017 at 05:22:59PM +0800, Yuanhan Liu wrote:\r\n > > > From: Finn Christensen \r\n > > > \r\n > > > AFAIK, most (if not all) NICs (including Mellanox and Intel) do not\r\n > > > support a pure MARK action. It's required to be used together with\r\n > > > some other actions, like QUEUE.\r\n > > > \r\n > > > To workaround it, retry with a queue action when first try failed.\r\n > > > \r\n > > > Moreover, some Intel's NIC (say XL710) needs the QUEUE action set\r\n > > > before the MARK action.\r\n > > > \r\n > > > Co-authored-by: Yuanhan Liu \r\n > > > Signed-off-by: Finn Christensen \r\n > > > Signed-off-by: Yuanhan Liu \r\n > > \r\n > > This feels a bit like the tail wagging the dog.\r\n > > Is this the lowest level at which it makes sense to implement\r\n > > this logic?\r\n > > \r\n > > If so then I wonder if some sort of probing would be in order\r\n > > to avoid the cost of trying to add the flow twice to hardware\r\n > > where the queue is required.\r\n > \r\n > Do you mean something like rte_flow capability query, like whether\r\n > a queue action is needed for a mark action? If so, yes, I do think\r\n > we miss an interface like this.\r\n > \r\n > Note that even in this solution, the flow won't be created twice\r\n > to the hardware, because the first try would be failed.\r\n > \r\n > [Darrell]\r\n > \r\n > Having an api to quey capability and avoid the first try to HW would be nice, but there are dependencies\r\n > on RTE, drivers etc and I don’t know definitive the api would be.\r\n > \r\n > Also, as nics are added this capability needs to be done and state needs to be kept in all cases.\r\n > \r\n > It is an enhancement and if done should be reliable.\r\n > \r\n > A separate comment is we need to document which nics need the queue action.\r\n > \r\n > Also, I think we should check errno in the present code.\r\n \r\n \r\n That was my first reaction to remove such blind re-try. It could have\r\n been a good option if all the PMD driver report the error consistenly.\r\n \r\n And unfortunately, it's not true. For example, for MARK without QUEUE\r\n action, i40e reports RTE_FLOW_ERROR_TYPE_ACTION, which, IMO, is nothing\r\n wrong. While for mlx5, it reports RTE_FLOW_ERROR_TYPE_HANDLE. I don't\r\n know why it was set like this, and we may could fix this. But my point\r\n was, it's not that reliable to use rte_errno, at least it's true for now.\r\n\r\n[Darrell] I see what you mean looking at the drivers code and I imagine it makes you work difficult.\r\n It looks like only either RTE_FLOW_ERROR_TYPE_ACTION or RTE_FLOW_ERROR_TYPE_HANDLE\r\n is used though in these cases from what I saw; would it make sense to check for those specifically?\r\n\r\n \r\n \t--yliu","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=onevmw.onmicrosoft.com\n\theader.i=@onevmw.onmicrosoft.com header.b=\"qsI6SnfQ\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=dball@vmware.com; "],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xv3q96tSBz9s7h\n\tfor ;\n\tSat, 16 Sep 2017 04:30:13 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id 4F6BAB0B;\n\tFri, 15 Sep 2017 18:30:10 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id B5DB9A81\n\tfor ; Fri, 15 Sep 2017 18:30:09 +0000 (UTC)","from NAM03-BY2-obe.outbound.protection.outlook.com\n\t(mail-by2nam03on0046.outbound.protection.outlook.com [104.47.42.46])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id DF4BF48B\n\tfor ; Fri, 15 Sep 2017 18:30:08 +0000 (UTC)","from BLUPR05MB611.namprd05.prod.outlook.com (10.141.204.27) by\n\tBLUPR05MB579.namprd05.prod.outlook.com (10.141.203.18) with Microsoft\n\tSMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id\n\t15.20.77.5; Fri, 15 Sep 2017 18:30:06 +0000","from BLUPR05MB611.namprd05.prod.outlook.com ([10.141.204.27]) by\n\tBLUPR05MB611.namprd05.prod.outlook.com ([10.141.204.27]) with mapi id\n\t15.20.0077.006; Fri, 15 Sep 2017 18:30:06 +0000"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=onevmw.onmicrosoft.com; s=selector1-vmware-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=ibO084N6F64w9oIoQYaVMVaqODsg/pZi7bDpFj/YfbY=;\n\tb=qsI6SnfQTTuMv5EuLvPQOS1ECJvEchN9jYKh73XAy9xSlQZVf8y75wZPdBuynyp4c4n6aVin1bqbpHt5SmkWMEDh4XX7Dnhrh1JGv74Eckv9BGbI899e/B6SQyGnLL0vAIbS3TnORg7yLsiH3s8/6cw7jPzcuP42+9B6LR8Kl1Q=","From":"Darrell Ball ","To":"Yuanhan Liu ","Thread-Topic":"[ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","Thread-Index":"AQHTJiizNzCJ8+cj4EuMMlhNVpa+HKKrOF8AgAQFvoCAAUSgAIAFK8qAgAAxHQA=","Date":"Fri, 15 Sep 2017 18:30:06 +0000","Message-ID":"<64837D85-3898-403C-9F40-A0EDBB1448B1@vmware.com>","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<20170908164849.GH7356@vergenet.net>\n\t<20170911061425.GN9736@yliu-home>\n\t<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>\n\t<20170915083419.GI2050@yliu-home>","In-Reply-To":"<20170915083419.GI2050@yliu-home>","Accept-Language":"en-US","Content-Language":"en-US","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","user-agent":"Microsoft-MacOutlook/f.25.0.170815","x-originating-ip":"[73.162.236.45]","x-ms-publictraffictype":"Email","x-microsoft-exchange-diagnostics":"1; BLUPR05MB579;\n\t20:j6KDfYbA55+KDUvq9oVjPR+wnjOl8t8A2ND3go3zAXA9nSjmJ6IWcMTRM/zovtnNopamy8C5RhRvjfX2zh7RGqB42v6VLKyTN7cCqQZNAOvCfJNTk55grndLnkgQmUzN9HbY9yZeLYnaeuIjoPpbVLCE7MDyrACPhbgwLECApmc=","x-ms-exchange-antispam-srfa-diagnostics":"SSOS;","x-ms-office365-filtering-correlation-id":"f8b769d0-0d27-48b7-ab3f-08d4fc67ca07","x-microsoft-antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(300000502095)(300135100095)(22001)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:BLUPR05MB579; ","x-ms-traffictypediagnostic":"BLUPR05MB579:","authentication-results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=onevmw.onmicrosoft.com\n\theader.i=@onevmw.onmicrosoft.com header.b=\"qsI6SnfQ\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=dball@vmware.com; "],"x-exchange-antispam-report-test":"UriScan:(20558992708506)(216315784871565);","x-microsoft-antispam-prvs":"","x-exchange-antispam-report-cfa-test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(93006095)(93001095)(100000703101)(100105400095)(3002001)(10201501046)(6041248)(20161123560025)(20161123558100)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123562025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:BLUPR05MB579; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:BLUPR05MB579; ","x-forefront-prvs":"0431F981D8","x-forefront-antispam-report":"SFV:NSPM;\n\tSFS:(10009020)(6009001)(376002)(346002)(377454003)(199003)(24454002)(189002)(54094003)(110136004)(25786009)(478600001)(66066001)(2906002)(105586002)(101416001)(54356999)(50986999)(58126008)(2900100001)(106356001)(33656002)(76176999)(189998001)(3280700002)(102836003)(3660700001)(82746002)(3846002)(6116002)(53936002)(81156014)(6512007)(99286003)(6486002)(8676002)(86362001)(93886005)(97736004)(7736002)(14454004)(8936002)(36756003)(6436002)(81166006)(305945005)(316002)(77096006)(229853002)(4326008)(39060400002)(5660300001)(6506006)(6246003)(54906002)(53546010)(2950100002)(68736007)(83506001)(83716003)(6916009);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR05MB579;\n\tH:BLUPR05MB611.namprd05.prod.outlook.com; FPR:; SPF:None;\n\tPTR:InfoNoRecords; A:1; MX:1; LANG:en; ","received-spf":"None (protection.outlook.com: vmware.com does not designate\n\tpermitted sender hosts)","spamdiagnosticoutput":"1:99","spamdiagnosticmetadata":"NSPM","Content-ID":"<2891E1656B76BC42A431EA7FA2DBC1FE@namprd05.prod.outlook.com>","MIME-Version":"1.0","X-OriginatorOrg":"vmware.com","X-MS-Exchange-CrossTenant-originalarrivaltime":"15 Sep 2017 18:30:06.6476\n\t(UTC)","X-MS-Exchange-CrossTenant-fromentityheader":"Hosted","X-MS-Exchange-CrossTenant-id":"b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"BLUPR05MB579","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ,\n\tSimon Horman ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1769367,"web_url":"http://patchwork.ozlabs.org/comment/1769367/","msgid":"<609B44E2-D29A-4E72-BAD2-505369F3328F@vmware.com>","list_archive_url":null,"date":"2017-09-15T19:00:40","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":68212,"url":"http://patchwork.ozlabs.org/api/people/68212/","name":"Darrell Ball","email":"dball@vmware.com"},"content":"On 9/15/17, 1:18 AM, \"Yuanhan Liu\" wrote:\r\n\r\n On Wed, Sep 13, 2017 at 04:17:37PM +0000, Darrell Ball wrote:\r\n > \r\n > \r\n > On 9/13/17, 2:57 AM, \"Simon Horman\" wrote:\r\n > \r\n > On Tue, Sep 12, 2017 at 08:36:19AM +0000, Darrell Ball wrote:\r\n > > \r\n > > On 9/10/17, 11:14 PM, \"ovs-dev-bounces@openvswitch.org on behalf of Yuanhan Liu\" wrote:\r\n > > \r\n > > On Fri, Sep 08, 2017 at 06:48:50PM +0200, Simon Horman wrote:\r\n > > > On Tue, Sep 05, 2017 at 05:22:59PM +0800, Yuanhan Liu wrote:\r\n > > > > From: Finn Christensen \r\n > > > > \r\n > > > > AFAIK, most (if not all) NICs (including Mellanox and Intel) do not\r\n > > > > support a pure MARK action. It's required to be used together with\r\n > > > > some other actions, like QUEUE.\r\n > > > > \r\n > > > > To workaround it, retry with a queue action when first try failed.\r\n > > > > \r\n > > > > Moreover, some Intel's NIC (say XL710) needs the QUEUE action set\r\n > > > > before the MARK action.\r\n > > > > \r\n > > > > Co-authored-by: Yuanhan Liu \r\n > > > > Signed-off-by: Finn Christensen \r\n > > > > Signed-off-by: Yuanhan Liu \r\n > > > \r\n > > > This feels a bit like the tail wagging the dog.\r\n > > > Is this the lowest level at which it makes sense to implement\r\n > > > this logic?\r\n > > > \r\n > > > If so then I wonder if some sort of probing would be in order\r\n > > > to avoid the cost of trying to add the flow twice to hardware\r\n > > > where the queue is required.\r\n > > \r\n > > Do you mean something like rte_flow capability query, like whether\r\n > > a queue action is needed for a mark action? If so, yes, I do think\r\n > > we miss an interface like this.\r\n > > \r\n > > Note that even in this solution, the flow won't be created twice\r\n > > to the hardware, because the first try would be failed.\r\n > > \r\n > > [Darrell]\r\n > > \r\n > > Having an api to quey capability and avoid the first try to HW would be nice, but there are dependencies\r\n > > on RTE, drivers etc and I don’t know definitive the api would be.\r\n > > \r\n > > Also, as nics are added this capability needs to be done and state needs to be kept in all cases.\r\n > > \r\n > > It is an enhancement and if done should be reliable.\r\n > \r\n > Agreed. Though I was more thinking of probing the hardware rather than\r\n > having a capability API - I expect this would remove several of the\r\n > dependencies you describe above.\r\n > \r\n > \r\n > [Darrell] I have been pondering the probing option as well. It is certainly a valid option; \r\n \r\n After thinking twice, I do think the probing might be better than capability\r\n feedback in some cases. For example, it makes more sense to probe the \"mark\r\n and queue\" actions than asking DPDK to provide such ability. It's more like\r\n a combination, which is hard to give from DPDK side.\r\n\r\n[Darrell] Agreed, for this case, probing does seem better vs capability query in\r\n the comparison of these 2 options.\r\n \r\n But for some cases, like do the underlaying NIC support a (few) specifics\r\n actions, or a (few) protocols, it's more clean and easier to provide cap\r\n feedback API from DPDK, IMO.\r\n\r\n[Darrell] For sure.\r\n \r\n > we use it in other cases such as datapath probing. One of the aspects that worries me here is\r\n > maintaining the correct per interface (essentially; although the attribute is per nic) state\r\n > across various events such as new ports being added, vswitchd restarts, races with flow\r\n > creation. It would be non-trivial I guess and probably appropriate for the next patch series, if done.\r\n \r\n I would think so. I will think about it after this patchset, to see which one\r\n suits better for our case. Or maybe, we could have both: I do think it makes\r\n sense to let DPDK to provide some basic capability feedbacks, for example,\r\n the supported actions, protocols, etc.\r\n\r\n[Darrell] Sounds good. If/when we have access to a capability query, we would use it in these cases.\r\n\r\n \r\n > In this case, we have what seems like a clear distinction b/w Napatech which does not need the\r\n > queue action workaround and everything else, which does.\r\n > Besides the non-Napatech behavior, which is worrisome, maintaining the difference for flow handling\r\n > under the covers is concerning.\r\n > \r\n > I wonder if we should be upfront as possible here and just have a dpdk interface configuration – maybe\r\n > something like “supports native HWOL mark action” since the better behavior is the exception?\r\n > The interface config would be more robust than probing.\r\n > This would need documentation, of course.\r\n \r\n The thing is that the option is fixed, while the NIC driver may change.\r\n Such option may apply to old versions may not apply to newer versions.\r\n\r\n[Darrell] If the limitation is only at driver layer, then that would be good and we should identify\r\n that in discussion with Intel and Mellanox. In the case of a ‘good driver version’, the user would consult the \r\n documentation and use the new driver version if possible and configure as\r\n “supports native HWOL mark action”.\r\n At least in one case, my understanding was that there was no plan to fix this, although that plan may\r\n change of course.\r\n Anyways, if you want to support probing in the next patchset, then that is fine.\r\n But, I think we still need verbose documentation about the nics differences in OVS docs, including the impact\r\n on receive queue, as used by OVS-DPDK.\r\n\r\n \r\n > I think anyways we need documentation describing the difference b/w nics in the dpdk documentation (howto part).\r\n \r\n IIRC, there was already an ask from the DPDK mailing list.\r\n\r\n[Darrell] I meant we need to document this in OVS since we have OVS code that is exposed to these differences.\r\n\r\n \r\n \t--yliu\r\n \r\n > Assuming no such enhancement is appropriate at this time I would\r\n > still like to ask if this is the best place for this hardware-specific code?\r\n > \r\n > [Darrell]\r\n > \r\n > For OVS, the netdev-dpdk layer is the lowest layer.\r\n > This kind of workaround is hard to hide, since we are messing with the rxq, so I think OVS needs to know\r\n > that it is in effect anyways. An alternative is to supply a mark and an ‘optional queue’ and let the driver decide if the queue is\r\n > needed and report back whether it was. This would be hard to do across various drivers. Supporting in the rte layer would require both\r\n > rte and driver support, so even more support.\r\n > \r\n > \r\n > \r\n > > A separate comment is we need to document which nics need the queue action.\r\n > > \r\n > > Also, I think we should check errno in the present code.\r\n > \r\n >","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=onevmw.onmicrosoft.com\n\theader.i=@onevmw.onmicrosoft.com header.b=\"bcPw55fh\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=dball@vmware.com; "],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xv4VR0zGcz9sBZ\n\tfor ;\n\tSat, 16 Sep 2017 05:00:46 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id 3D7A3B4B;\n\tFri, 15 Sep 2017 19:00:44 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 361247AA\n\tfor ; Fri, 15 Sep 2017 19:00:43 +0000 (UTC)","from NAM02-BL2-obe.outbound.protection.outlook.com\n\t(mail-bl2nam02on0049.outbound.protection.outlook.com [104.47.38.49])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id 4F850E0\n\tfor ; Fri, 15 Sep 2017 19:00:42 +0000 (UTC)","from BLUPR05MB611.namprd05.prod.outlook.com (10.141.204.27) by\n\tBLUPR05MB1970.namprd05.prod.outlook.com (10.162.224.24) with\n\tMicrosoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id\n\t15.20.77.5; Fri, 15 Sep 2017 19:00:40 +0000","from BLUPR05MB611.namprd05.prod.outlook.com ([10.141.204.27]) by\n\tBLUPR05MB611.namprd05.prod.outlook.com ([10.141.204.27]) with mapi id\n\t15.20.0077.006; Fri, 15 Sep 2017 19:00:40 +0000"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=onevmw.onmicrosoft.com; s=selector1-vmware-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=SnzABxweCdZKaPRpyR2AqCBOh/yrjIIklKNAREJS5tI=;\n\tb=bcPw55fh/Sxe31Dg5EYfPtEUrkZLuuYmus6WbDgEnJ0BQ0fxv7C8QK44wYIVvojE2B4jEI+5Ij4N6I9au+vC3A6lfp2Vu9+etBh2iD7MvTyh6PUh3S3iAauotY91U551h23vMYdqcSeSYmv1gZPbYNqrCCMLk/snJLF47+XCyHs=","From":"Darrell Ball ","To":"Yuanhan Liu ","Thread-Topic":"[ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","Thread-Index":"AQHTJiizNzCJ8+cj4EuMMlhNVpa+HKKrOF8AgAQFvoCAAUSgAIACHl+A///02QCAAxQ1AIAAPgOA","Date":"Fri, 15 Sep 2017 19:00:40 +0000","Message-ID":"<609B44E2-D29A-4E72-BAD2-505369F3328F@vmware.com>","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<20170908164849.GH7356@vergenet.net>\n\t<20170911061425.GN9736@yliu-home>\n\t<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>\n\t<20170913095730.GE27555@vergenet.net>\n\t\n\t<20170915081842.GH2050@yliu-home>","In-Reply-To":"<20170915081842.GH2050@yliu-home>","Accept-Language":"en-US","Content-Language":"en-US","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","user-agent":"Microsoft-MacOutlook/f.25.0.170815","x-originating-ip":"[73.162.236.45]","x-ms-publictraffictype":"Email","x-microsoft-exchange-diagnostics":"1; BLUPR05MB1970;\n\t20:vHQfWysijpbDaZJ+RlXd+98zZMfx8Cvc2yyN18CLk1RiRFKGk6LOosBUld2NG6UuQMry0Plbjl495SWA3Vi2usEODNt6mE08VoNsItM9vGVh3pMsJx67/miLCn25LpB6pHJYxEnt20Fa5LmpTGzy7YYidippTIXbEw3kq+9fuRc=","x-ms-exchange-antispam-srfa-diagnostics":"SSOS;","x-ms-office365-filtering-correlation-id":"b7d4ec80-e654-4868-be6f-08d4fc6c0efa","x-microsoft-antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:BLUPR05MB1970; ","x-ms-traffictypediagnostic":"BLUPR05MB1970:","authentication-results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=onevmw.onmicrosoft.com\n\theader.i=@onevmw.onmicrosoft.com header.b=\"bcPw55fh\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=dball@vmware.com; "],"x-exchange-antispam-report-test":"UriScan:(216315784871565);","x-microsoft-antispam-prvs":"","x-exchange-antispam-report-cfa-test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(10201501046)(3002001)(93006095)(93001095)(100000703101)(100105400095)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123564025)(20161123555025)(20161123562025)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:BLUPR05MB1970; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:BLUPR05MB1970; ","x-forefront-prvs":"0431F981D8","x-forefront-antispam-report":"SFV:NSPM;\n\tSFS:(10009020)(6009001)(346002)(376002)(377454003)(189002)(199003)(54094003)(24454002)(102836003)(3846002)(36756003)(6246003)(105586002)(316002)(110136004)(83506001)(5660300001)(3280700002)(25786009)(6916009)(86362001)(99286003)(14454004)(68736007)(6116002)(82746002)(97736004)(6506006)(2950100002)(305945005)(33656002)(58126008)(189998001)(54356999)(2906002)(50986999)(76176999)(81166006)(3660700001)(6436002)(101416001)(7736002)(229853002)(83716003)(77096006)(53546010)(8936002)(6512007)(2900100001)(478600001)(66066001)(53936002)(39060400002)(6486002)(106356001)(8676002)(4326008)(93886005)(81156014)(54906002);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR05MB1970;\n\tH:BLUPR05MB611.namprd05.prod.outlook.com; FPR:; SPF:None;\n\tPTR:InfoNoRecords; MX:1; A:1; LANG:en; ","received-spf":"None (protection.outlook.com: vmware.com does not designate\n\tpermitted sender hosts)","spamdiagnosticoutput":"1:99","spamdiagnosticmetadata":"NSPM","Content-ID":"","MIME-Version":"1.0","X-OriginatorOrg":"vmware.com","X-MS-Exchange-CrossTenant-originalarrivaltime":"15 Sep 2017 19:00:40.3438\n\t(UTC)","X-MS-Exchange-CrossTenant-fromentityheader":"Hosted","X-MS-Exchange-CrossTenant-id":"b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"BLUPR05MB1970","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ,\n\tSimon Horman ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1769414,"web_url":"http://patchwork.ozlabs.org/comment/1769414/","msgid":"","list_archive_url":null,"date":"2017-09-15T19:55:27","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":72305,"url":"http://patchwork.ozlabs.org/api/people/72305/","name":"Finn Christensen","email":"fc@napatech.com"},"content":"-----Original Message-----\r\n From: Darrell Ball [mailto:dball@vmware.com]\r\n Sent: 15. september 2017 20:09\r\n To: Finn Christensen \r\n Cc: dev@openvswitch.org\r\n Subject: Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action\r\n \r\n \r\n \r\n On 9/15/17, 2:16 AM, \"Finn Christensen\" wrote:\r\n \r\n \r\n \r\n -----Original Message-----\r\n From: Darrell Ball [mailto:dball@vmware.com]\r\n Sent: 14. september 2017 21:35\r\n To: Finn Christensen \r\n Cc: dev@openvswitch.org\r\n Subject: Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue\r\n action\r\n \r\n \r\n \r\n On 9/14/17, 1:14 AM, \"Finn Christensen\" wrote:\r\n \r\n \r\n \r\n \r\n \r\n -----Original Message-----\r\n \r\n From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev-\r\n bounces@openvswitch.org] On Behalf Of Darrell Ball\r\n \r\n Sent: 13. september 2017 18:18\r\n \r\n To: Simon Horman \r\n \r\n Cc: dev@openvswitch.org\r\n \r\n Subject: Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue\r\n action\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n On 9/13/17, 2:57 AM, \"Simon Horman\"\r\n wrote:\r\n \r\n \r\n \r\n On Tue, Sep 12, 2017 at 08:36:19AM +0000, Darrell Ball wrote:\r\n \r\n >\r\n \r\n > On 9/10/17, 11:14 PM, \"ovs-dev-bounces@openvswitch.org on\r\n behalf of Yuanhan Liu\" wrote:\r\n \r\n >\r\n \r\n > On Fri, Sep 08, 2017 at 06:48:50PM +0200, Simon Horman\r\n wrote:\r\n \r\n > > On Tue, Sep 05, 2017 at 05:22:59PM +0800, Yuanhan Liu\r\n wrote:\r\n \r\n > > > From: Finn Christensen \r\n \r\n > > >\r\n \r\n > > > AFAIK, most (if not all) NICs (including Mellanox and Intel)\r\n do\r\n not\r\n \r\n > > > support a pure MARK action. It's required to be used\r\n together\r\n with\r\n \r\n > > > some other actions, like QUEUE.\r\n \r\n > > >\r\n \r\n > > > To workaround it, retry with a queue action when first try\r\n failed.\r\n \r\n > > >\r\n \r\n > > > Moreover, some Intel's NIC (say XL710) needs the QUEUE\r\n action set\r\n \r\n > > > before the MARK action.\r\n \r\n > > >\r\n \r\n > > > Co-authored-by: Yuanhan Liu \r\n \r\n > > > Signed-off-by: Finn Christensen \r\n \r\n > > > Signed-off-by: Yuanhan Liu \r\n \r\n > >\r\n \r\n > > This feels a bit like the tail wagging the dog.\r\n \r\n > > Is this the lowest level at which it makes sense to implement\r\n \r\n > > this logic?\r\n \r\n > >\r\n \r\n > > If so then I wonder if some sort of probing would be in order\r\n \r\n > > to avoid the cost of trying to add the flow twice to hardware\r\n \r\n > > where the queue is required.\r\n \r\n >\r\n \r\n > Do you mean something like rte_flow capability query, like\r\n whether\r\n \r\n > a queue action is needed for a mark action? If so, yes, I do\r\n think\r\n \r\n > we miss an interface like this.\r\n \r\n >\r\n \r\n > Note that even in this solution, the flow won't be created\r\n twice\r\n \r\n > to the hardware, because the first try would be failed.\r\n \r\n >\r\n \r\n > [Darrell]\r\n \r\n >\r\n \r\n > Having an api to quey capability and avoid the first try to\r\n HW\r\n would be nice, but there are dependencies\r\n \r\n > on RTE, drivers etc and I don’t know definitive the api\r\n would\r\n be.\r\n \r\n >\r\n \r\n > Also, as nics are added this capability needs to be done\r\n and\r\n state needs to be kept in all cases.\r\n \r\n >\r\n \r\n > It is an enhancement and if done should be reliable.\r\n \r\n \r\n \r\n Agreed. Though I was more thinking of probing the hardware\r\n rather\r\n than\r\n \r\n having a capability API - I expect this would remove several of the\r\n \r\n dependencies you describe above.\r\n \r\n \r\n \r\n \r\n \r\n [Darrell] I have been pondering the probing option as well. It is\r\n certainly a\r\n valid option; we use it in other cases such as datapath probing. One of\r\n the\r\n aspects that worries me here is maintaining the correct per interface\r\n (essentially; although the attribute is per nic) state across various\r\n events\r\n such as new ports being added, vswitchd restarts, races with flow\r\n creation.\r\n It would be non-trivial I guess and probably appropriate for the next\r\n patch\r\n series, if done.\r\n \r\n \r\n \r\n In this case, we have what seems like a clear distinction b/w\r\n Napatech\r\n which does not need the queue action workaround and everything\r\n else,\r\n which does.\r\n \r\n Besides the non-Napatech behavior, which is worrisome,\r\n maintaining the\r\n difference for flow handling under the covers is concerning.\r\n \r\n \r\n \r\n I wonder if we should be upfront as possible here and just have a\r\n dpdk\r\n interface configuration – maybe something like “supports native\r\n HWOL\r\n mark action” since the better behavior is the exception?\r\n \r\n The interface config would be more robust than probing.\r\n \r\n This would need documentation, of course.\r\n \r\n \r\n \r\n [Finn]\r\n \r\n I think the rte queue action should never be used here when using\r\n partial HWOL. Not the way OVS handles multi queues today.\r\n \r\n Maybe a \"default queue\" could be used in the dpdk PMDs when no\r\n queue is specified in rte flow?\r\n \r\n [Darrell] This is the Napatech case, where no queue action is needed;\r\n you\r\n are suggesting programming a default queue in this case.\r\n I don’t follow how this would be helpful/desired?\r\n \r\n [Finn]\r\n I was trying to make my view on this, not particularly arguing for the\r\n Napatech\r\n Case.\r\n Here is what I was thinking:\r\n Taking the case of this partial HWOL, then we are trying to offload the\r\n flow\r\n classification to HW, like \"pre-classify and mark\". Then this mark is used\r\n to\r\n accelerate OVS while finding the actions to execute. Since we do leave all\r\n processing of the actions to OVS, there is no way for the partial HWOL to\r\n know, at rte flow creation time, where to send the pre-classified packets\r\n (which is strictly needed when seen from a DPDK rte flow point of view).\r\n When multiple queues are specified in OVS, a hash splitting mechanism\r\n is used in the nic to support RSS. Then the nic is responsible for selecting\r\n the\r\n right queue according to the configured algorithm for RSS.\r\n OVS only needs to know how many queues to service per port. No\r\n knowledge\r\n about the association b/w flow <-> rxq is used in OVS today.\r\n When looking at full HWOL, all flow actions will have to be interpreted,\r\n supported\r\n and programmed to NIC using rte flow - send directly to target port and\r\n we\r\n do not have this issue.\r\n Have I missed something?\r\n \r\n [Darrell]\r\n I understand you points about the full offload you are working on.\r\n I just did not follow this one comment:\r\n \r\n “Maybe a \"default queue\" could be used in the dpdk PMDs when no\r\n queue is specified in rte flow?”\r\n \r\n which I thought is related to partial offloads used here and the receive\r\n queue action we are discussing?; I am concerned that I missed your point\r\n about this specific comment ?\r\n \r\n\r\n[Finn]\r\nSorry for my messy descriptions, and yes the \"default queue\" idea was for the \r\npartial HWOL case. I just think if the queue could be set to 0 if 1 rxq used and let \r\nthe hash algorithm set the queue if RSS is used, then we do not have to add QUEUE \r\nto the rte flow create. \r\nHowever, it was just an idea and I don't know if it can be done reliably in all DPDK PMDs.\r\n\r\n \r\n Essentially, this is a mismatch between the rte flow impl\r\n functionality in\r\n PMD and the needs in OVS for flow classification, in a partial HWOL\r\n setup.\r\n \r\n \r\n [Darrell] Maybe this has been explored, but were additional\r\n workaround\r\n actions, like RTE_FLOW_ACTION_TYPE_RSS considered ?\r\n I guess we could make this essentially a NOP, but if mark\r\n action\r\n could ride along with it, then that would be good.\r\n \r\n [Finn]\r\n No. We have not explored that.\r\n I'm not sure I understand how you would use\r\n RTE_FLOW_ACTION_TYPE_RSS\r\n In this situation.\r\n \r\n \r\n [Darrell]\r\n So there are two points here:\r\n 1/ ‘IF’ RTE_FLOW_ACTION_TYPE_RSS can used in lieu of\r\n RTE_FLOW_ACTION_TYPE_QUEUE combined with\r\n RTE_FLOW_ACTION_TYPE_MARK, as need for the workaround for most\r\n nics, then this would be better for distributing packets across PMDs for the\r\n masked flows relevant here. This would help mitigate one of the concerns.\r\n We could use the RSS we use globally, essentially making the distribution a\r\n NOP compared with what we normally do today with RSS….\r\n \r\n 2/ Assuming ‘1’, we might even use this in all cases as a first try for all nics,\r\n which if it could be done solves the queue action workaround\r\n capability/probing thing?\r\n \r\n[Finn] Thanks, I see what you mean. It makes very good sense if it is supported\r\non the nic DPDK PMDs.\r\n\r\n \r\n However, we would not mind Darells proposal, it makes sense since\r\n most nic PMDs unfortunately needs this and it is only relevant for\r\n partial\r\n HWOL. We are mostly concerned with full HWOL and therefore see\r\n partial\r\n HWOL as a failover when full HWOL could not be handled in nic, if\r\n enabled.\r\n \r\n \r\n \r\n \r\n \r\n I think anyways we need documentation describing the difference\r\n b/w\r\n nics in the dpdk documentation (howto part).\r\n \r\n \r\n \r\n \r\n \r\n Assuming no such enhancement is appropriate at this time I\r\n would\r\n \r\n still like to ask if this is the best place for this hardware-specific\r\n code?\r\n \r\n \r\n \r\n [Darrell]\r\n \r\n \r\n \r\n For OVS, the netdev-dpdk layer is the lowest layer.\r\n \r\n This kind of workaround is hard to hide, since we are messing with\r\n the\r\n rxq, so I think OVS needs to know that it is in effect anyways. An\r\n alternative is to supply a mark and an ‘optional queue’ and let the\r\n driver\r\n decide if the queue is needed and report back whether it was. This\r\n would\r\n be hard to do across various drivers. Supporting in the rte layer would\r\n require both rte and driver support, so even more support.\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n > A separate comment is we need to document which nics\r\n need\r\n the queue action.\r\n \r\n >\r\n \r\n > Also, I think we should check errno in the present code.\r\n \r\n \r\n \r\n \r\n \r\n _______________________________________________\r\n \r\n dev mailing list\r\n \r\n dev@openvswitch.org\r\n \r\n https://urldefense.proofpoint.com/v2/url?u=https-\r\n 3A__mail.openvswitch.org_mailman_listinfo_ovs-\r\n \r\n 2Ddev&d=DwIGaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-\r\n uZnsw&m=IHypHavCy0AKjNxqOMyc4w3ILyC-\r\n \r\n BuwkB8fuVvQUA3k&s=deJQWP9KI22Xp46tEoZ6o6Emitr3Bhfd7iSMxNpude\r\n g&e=\r\n \r\n Disclaimer: This email and any files transmitted with it may contain\r\n confidential information intended for the addressee(s) only. The\r\n information is not to be surrendered or copied to unauthorized\r\n persons. If\r\n you have received this communication in error, please notify the\r\n sender\r\n immediately and delete this e-mail from your system.\r\n \r\n \r\n \r\n Disclaimer: This email and any files transmitted with it may contain\r\n confidential information intended for the addressee(s) only. The\r\n information is not to be surrendered or copied to unauthorized persons. If\r\n you have received this communication in error, please notify the sender\r\n immediately and delete this e-mail from your system.","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=napatech.com header.i=@napatech.com\n\theader.b=\"O4sh7cj3\"; dkim-atps=neutral"],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xv5jm20gpz9s7g\n\tfor ;\n\tSat, 16 Sep 2017 05:55:39 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id CF037B69;\n\tFri, 15 Sep 2017 19:55:35 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 48ECAB50\n\tfor ; Fri, 15 Sep 2017 19:55:34 +0000 (UTC)","from mail01.napatech.com (mail01.napatech.com [188.120.77.121])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id 6DB0D1E8\n\tfor ; Fri, 15 Sep 2017 19:55:32 +0000 (UTC)","from cph-gen-exch02.napatech.com (10.240.1.84) by\n\tcph-gen-exch02.napatech.com (10.240.1.84) with Microsoft SMTP Server\n\t(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id\n\t15.1.1034.26; Fri, 15 Sep 2017 21:55:28 +0200","from cph-gen-exch02.napatech.com ([fe80::581:51a1:ac3f:84e]) by\n\tcph-gen-exch02.napatech.com ([fe80::581:51a1:ac3f:84e%12]) with mapi\n\tid 15.01.1034.026; Fri, 15 Sep 2017 21:55:28 +0200"],"X-Greylist":"domain auto-whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=napatech.com; i=@napatech.com; q=dns/txt; s=mar2017;\n\tt=1505505332; x=1537041332;\n\th=from:to:cc:subject:date:message-id:references:\n\tin-reply-to:content-transfer-encoding:mime-version;\n\tbh=SztLrK4mrpAoKVQjYF/AOScRqs/J5dVmVQMBZHk8FpY=;\n\tb=O4sh7cj3iz+3vjZJxHcIYEAkcrBcuUk0BWHvnIIhmf241ybDmIR15OWF\n\tGZk9K2aJZYYNI6taZCq5pku81He3Mo7OdLuW4luFwhfJWfqkFYIS/NsGB\n\tfhPDsDlk/iP2w7MIYcXBqnqZlECY4cnA3z5oIsvYKt+hERzFLaUxNVZeP\n\tKgwddiBrvuZpmiPjftG4FA6r4ZXSDfu7520Q2Pu1AIqmUz37CGFDRRTFK\n\tPQGHNe32gkdEGtZILofuUhuqF3pkXZ9TUQT7YjaF8RzDymTRZIG536hYn\n\tr4Hc8fllAN1sOcyaqF3RMmD7j3lmToHXooDPruv+pHdRkFHWKt31lVbh+ g==;","IronPort-PHdr":"9a23:UcOC5RJrH6TlEyNG1dmcpTZWNBhigK39O0sv0rFitYgeI/XxwZ3uMQTl6Ol3ixeRBMOAuqMC17qd6vu+EUU7or+5+EgYd5JNUxJXwe43pCcHRPC/NEvgMfTxZDY7FskRHHVs/nW8LFQHUJ2mPw6arXK99yMdFQviPgRpOOv1BpTSj8Oq3Oyu5pHfeQtFiT65bL9oIxi7rgrdutQYjIZjN6081gbHrnxUdupM2GhmP0iTnxHy5sex+J5s7SFdsO8/+sBDTKv3Yb02QaRXAzo6PW814tbrtQTYQguU+nQcSGQWnQFWDAXD8Rr3Q43+sir+tup6xSmaIcj7Rq06VDi+86tmTgLjhTwZPDAl7m7Yls1wjLpaoB2/oRx/35XUa5yROPZnY6/RYc8WSW9HU81MVSJOH5m8YpMPAeQfIOhYs4fzqVgArRS8BAmjGOzhxTBTi3/qxqI60fgtHR3a0AEiGd8FrXTarM/yNKcXSe27zKjIzTPFb/hLxzr96JLIchE6ofGQQ71wd9HRxlcpFwjYiViQp5DqMiiT1usXq2iU9fZgWvyzhG4nsQ1+vj+vxsI1h4TPm4kbxFfE9SBjz4Y0I921UEl7Yca6H5tWqSGaLIV3QsI+Q250uCY20LoGuYS0fCQS0JQn3Rnfa/uJc4iQ5RLjVeCRIStiiH15f7K/ghC/+lWjxO3kTsS4zUpGojBbntTDqnwBzQHf5tKER/Zy5kus2jiC2xrX5+xKO0w5lLDXJp4lz7IomJocr0fOEjPzlUjzjKKbdVgo9+2o5uj6eLrqu5qROJVohQzwPKQjn8+yDOsmPQUIQmOV4/6z1Kf58k38WLhKi/o2nbTHv53CPsQbo7K5AxdS0oY+9xazFzem38ocnXkANF9FfQiIj4ntO13UJvD3F++/jE6wkDh12//GPqftDYnKLnjGiLvhfLB95FBAyAcr0NxT+4hYBq8OLf7vQEP9qcbUAxw2PgCsxuboEtR91ocQWWKVBa+ZNbvfsVGU6e80JemDfpcVtyzhK/c7+/HujWU1lkMafamsxZcXcmy3Hux6I0WFZnrhmsoOHnkUvgclS+zqkEONUThNZ3apUaM85y07B56mDYvZQYCtmrOBj2+HGch6b3pcB1SIWV3hc4HMD/sGYSaWCtFkjTUeWP6qTIp3hj+0swqv5bthKKL/+jcZro7u0sN44aWHmxoa8zVsBtiQ2GHLRGZxyDBbDwQq1bxy9BQugmyI1rJ11rkBTYRe","X-IronPort-Anti-Spam-Filtered":"true","X-IronPort-Anti-Spam-Result":"A2HCBAA8L7xZ/1QB8ApdGgEBAQECAQEBAQgBAQEBFQEBAQECAQEBAQgBAQEBgy9kbicHg26cB3mSVQiCUQ6BQUMKI4UZAhqEEEAXAQEBAQEBAQEBAQECaChCEgGBXiQBDS8IAwwhCAMBAQEBAQEBAQECAQEBAUYCD0EBARgBAQEBAgEjBA06CwwEAgEFAQIOAwQBAQMCIwMCAgIwFAEICAIEDgUIE4oQDAyOBZ1mgW06iywBAQEBAQEEAQEBAQEBAQEBGgWBDoIdgTGCIYFjghtYNYMogRQJARIBgzKCYAWKA4kUjW6HW4NciRWCHIVqinuHE4o/gzUCAgICCQIagTkgATeBAgt3FUmFGAEcgWdANoREgh+BIwGBDgEBAQ","X-IPAS-Result":"A2HCBAA8L7xZ/1QB8ApdGgEBAQECAQEBAQgBAQEBFQEBAQECAQEBAQgBAQEBgy9kbicHg26cB3mSVQiCUQ6BQUMKI4UZAhqEEEAXAQEBAQEBAQEBAQECaChCEgGBXiQBDS8IAwwhCAMBAQEBAQEBAQECAQEBAUYCD0EBARgBAQEBAgEjBA06CwwEAgEFAQIOAwQBAQMCIwMCAgIwFAEICAIEDgUIE4oQDAyOBZ1mgW06iywBAQEBAQEEAQEBAQEBAQEBGgWBDoIdgTGCIYFjghtYNYMogRQJARIBgzKCYAWKA4kUjW6HW4NciRWCHIVqinuHE4o/gzUCAgICCQIagTkgATeBAgt3FUmFGAEcgWdANoREgh+BIwGBDgEBAQ","From":"Finn Christensen ","To":"Darrell Ball ","Thread-Topic":"[ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","Thread-Index":"AQHTJiizfJsWL3b3Rkev+6Ia92jE1aKrFtgAgAQFvoCAAbn6gIABqQWAgABqM4CAAD/isIABia4AgADnWpCAAJL7AIAANUDw","Date":"Fri, 15 Sep 2017 19:55:27 +0000","Message-ID":"","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<20170908164849.GH7356@vergenet.net>\n\t<20170911061425.GN9736@yliu-home>\n\t<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>\n\t<20170913095730.GE27555@vergenet.net>\n\t\n\t<3edc7d93b8c94c3c8e316b299981b984@napatech.com>\n\t\n\t\n\t","In-Reply-To":"","Accept-Language":"da-DK, en-US","Content-Language":"en-US","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","x-originating-ip":"[83.93.6.59]","MIME-Version":"1.0","X-Spam-Status":"No, score=-2.4 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tDKIM_VALID_AU,RCVD_IN_DNSWL_MED,RP_MATCHES_RCVD autolearn=disabled\n\tversion=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1769838,"web_url":"http://patchwork.ozlabs.org/comment/1769838/","msgid":"<20170918021139.GO2050@yliu-home>","list_archive_url":null,"date":"2017-09-18T02:11:39","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":72215,"url":"http://patchwork.ozlabs.org/api/people/72215/","name":"Yuanhan Liu","email":"yliu@fridaylinux.org"},"content":"On Fri, Sep 15, 2017 at 07:00:40PM +0000, Darrell Ball wrote:\n> > In this case, we have what seems like a clear distinction b/w Napatech which does not need the\n> > queue action workaround and everything else, which does.\n> > Besides the non-Napatech behavior, which is worrisome, maintaining the difference for flow handling\n> > under the covers is concerning.\n> > \n> > I wonder if we should be upfront as possible here and just have a dpdk interface configuration – maybe\n> > something like “supports native HWOL mark action” since the better behavior is the exception?\n> > The interface config would be more robust than probing.\n> > This would need documentation, of course.\n> \n> The thing is that the option is fixed, while the NIC driver may change.\n> Such option may apply to old versions may not apply to newer versions.\n> \n> [Darrell] If the limitation is only at driver layer, then that would be good and we should identify\n> that in discussion with Intel and Mellanox. In the case of a ‘good driver version’, the user would consult the \n> documentation and use the new driver version if possible and configure as\n> “supports native HWOL mark action”.\n> At least in one case, my understanding was that there was no plan to fix this, although that plan may\n> change of course.\n\nFrom what I know, it's unfortunately true.\n\n> Anyways, if you want to support probing in the next patchset, then that is fine.\n\nYes, I will think about it.\n\n> But, I think we still need verbose documentation about the nics differences in OVS docs, including the impact\n> on receive queue, as used by OVS-DPDK.\n\nNo objections.\n\n> > I think anyways we need documentation describing the difference b/w nics in the dpdk documentation (howto part).\n> \n> IIRC, there was already an ask from the DPDK mailing list.\n> \n> [Darrell] I meant we need to document this in OVS since we have OVS code that is exposed to these differences.\n\nOkay, makes sense. I will start a doc and let other venders to fill it.\n\n\t--yliu","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=fridaylinux-org.20150623.gappssmtp.com\n\theader.i=@fridaylinux-org.20150623.gappssmtp.com\n\theader.b=\"14maVlAs\"; dkim-atps=neutral"],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xwTz04F6xz9rxm\n\tfor ;\n\tMon, 18 Sep 2017 12:11:55 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id C6D0487A;\n\tMon, 18 Sep 2017 02:11:51 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 88CB45A7\n\tfor ; Mon, 18 Sep 2017 02:11:50 +0000 (UTC)","from mail-pg0-f45.google.com (mail-pg0-f45.google.com\n\t[74.125.83.45])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id 2BD50174\n\tfor ; Mon, 18 Sep 2017 02:11:50 +0000 (UTC)","by mail-pg0-f45.google.com with SMTP id u18so4226057pgo.0\n\tfor ; Sun, 17 Sep 2017 19:11:50 -0700 (PDT)","from yliu-home ([45.63.61.64]) by smtp.gmail.com with ESMTPSA id\n\ty128sm10817664pfy.125.2017.09.17.19.11.45\n\t(version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128);\n\tSun, 17 Sep 2017 19:11:47 -0700 (PDT)"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=fridaylinux-org.20150623.gappssmtp.com; s=20150623;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:content-transfer-encoding:in-reply-to\n\t:user-agent; bh=KYCdQEb8llEufQNfMQ7OBkTOs5zrdLL2ZPd3oIGU8Gs=;\n\tb=14maVlAs/jIMUImk/5SgO/AiAGjoRV+lZXGBRaK0ij7Mw1DmlT4ZChV7mqcP/vrnWQ\n\tX4t/4AbvLPQsq4fDitHmfdCeNnstsdrLhLTmkjhOdybXXYZWIDMVejWEH3p971DJN+/S\n\twxCg+tCUInVDcEL1l/BVpwHOCB74EOH8Bepo0vWNxPxW+vlB/+3J7qb0V4kYbkADvCTu\n\tyM4vN6plb8Zr0XjdKvfVZykKkgi+atx40TezYpblE5iLH8ygDFmwtrj5R9j3sdhIdYUc\n\tmI+bIrolZ1y8Ja7icYy9UUNIIVTOANHBbLQMyB0GRpRBs92WgpugLz4aH3jrA5GAvOFG\n\t7J9w==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:content-transfer-encoding\n\t:in-reply-to:user-agent;\n\tbh=KYCdQEb8llEufQNfMQ7OBkTOs5zrdLL2ZPd3oIGU8Gs=;\n\tb=c7xvZ9ez+ip6JRJ7E098BEI2s8vJgrMoZsa97kivilLM6sRjm9zQjgnfxrKx5MmL/L\n\tJQg8ajL6NGBQ1WMB/bcKcZt0TScCxVQ9GnDtFPnFE0VNiTp6kOk++35osCJTmqC5BXcQ\n\tfIep3U2gJ24fyH1W5vro4u0G5dXGMzbeQb1oi0WS/a1M4MwRVGgV6CPtFF/68eA+OMTA\n\t2xFIBkrPh/iJJMsN5I0j8xXwicu6Oxub3XmxZg9jO+IN5FruoF3/HrVNmFo6++O9MTjm\n\tx5gGMs77EXwmB7+l6rXHlqiYe/vcWOGCEDVzApTDe4KmvHOsosq/GjMoPUMdQW8/dXmH\n\tfwaw==","X-Gm-Message-State":"AHPjjUipOxcRk3pSu4PrLMvs4CjmyEPq+LWhcDFCEI9Wd7Gn6MSlc2Qy\n\t++38dXojjsdtpcQF","X-Google-Smtp-Source":"ADKCNb7yD7PghCJkagm+klbwBD1zC5qdcxfvXXvMROmwZXYPRFwof5yUrwZXHHmKRv0SdseoRW2xxw==","X-Received":"by 10.98.159.76 with SMTP id g73mr30831914pfe.293.1505700709668; \n\tSun, 17 Sep 2017 19:11:49 -0700 (PDT)","Date":"Mon, 18 Sep 2017 10:11:39 +0800","From":"Yuanhan Liu ","To":"Darrell Ball ","Message-ID":"<20170918021139.GO2050@yliu-home>","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<20170908164849.GH7356@vergenet.net>\n\t<20170911061425.GN9736@yliu-home>\n\t<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>\n\t<20170913095730.GE27555@vergenet.net>\n\t\n\t<20170915081842.GH2050@yliu-home>\n\t<609B44E2-D29A-4E72-BAD2-505369F3328F@vmware.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<609B44E2-D29A-4E72-BAD2-505369F3328F@vmware.com>","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ,\n\tSimon Horman ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1769841,"web_url":"http://patchwork.ozlabs.org/comment/1769841/","msgid":"<20170918022140.GP2050@yliu-home>","list_archive_url":null,"date":"2017-09-18T02:21:40","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":72215,"url":"http://patchwork.ozlabs.org/api/people/72215/","name":"Yuanhan Liu","email":"yliu@fridaylinux.org"},"content":"On Fri, Sep 15, 2017 at 06:30:06PM +0000, Darrell Ball wrote:\n> That was my first reaction to remove such blind re-try. It could have\n> been a good option if all the PMD driver report the error consistenly.\n> \n> And unfortunately, it's not true. For example, for MARK without QUEUE\n> action, i40e reports RTE_FLOW_ERROR_TYPE_ACTION, which, IMO, is nothing\n> wrong. While for mlx5, it reports RTE_FLOW_ERROR_TYPE_HANDLE. I don't\n> know why it was set like this, and we may could fix this. But my point\n> was, it's not that reliable to use rte_errno, at least it's true for now.\n> \n> [Darrell] I see what you mean looking at the drivers code and I imagine it makes you work difficult.\n> It looks like only either RTE_FLOW_ERROR_TYPE_ACTION or RTE_FLOW_ERROR_TYPE_HANDLE\n> is used though in these cases from what I saw; would it make sense to check for those specifically?\n\n\nYes, I think it's better than a blind retry (assuming it's failed due\nto an unsupported protocol). Meanwhile, I will double check the DPDK\ndriver code.\n\n\t--yliu","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=fridaylinux-org.20150623.gappssmtp.com\n\theader.i=@fridaylinux-org.20150623.gappssmtp.com\n\theader.b=\"uoRiGR7E\"; dkim-atps=neutral"],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xwVBS6ZBFz9sBZ\n\tfor ;\n\tMon, 18 Sep 2017 12:21:52 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id C58E58D7;\n\tMon, 18 Sep 2017 02:21:49 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id B6E855A7\n\tfor ; Mon, 18 Sep 2017 02:21:48 +0000 (UTC)","from mail-pg0-f45.google.com (mail-pg0-f45.google.com\n\t[74.125.83.45])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id 54C2F174\n\tfor ; Mon, 18 Sep 2017 02:21:48 +0000 (UTC)","by mail-pg0-f45.google.com with SMTP id i130so4233302pgc.3\n\tfor ; Sun, 17 Sep 2017 19:21:48 -0700 (PDT)","from yliu-home ([45.63.61.64]) by smtp.gmail.com with ESMTPSA id\n\t76sm12206199pfp.158.2017.09.17.19.21.45\n\t(version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128);\n\tSun, 17 Sep 2017 19:21:46 -0700 (PDT)"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=fridaylinux-org.20150623.gappssmtp.com; s=20150623;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=9hhuiBolZ7wClGvze5UYo9ZIBCuRQRN7wF3bu3CuwhE=;\n\tb=uoRiGR7EBQeHYQVzQz0UcOcb+x2W5GGYcR+PqcndhB7LbM+EjdHLT30PZDe2+vrCL7\n\tcN9TBSEfpyfuGC2tboQesisYQ76LGpX/Qm1ouknoJpBezZpC5U0gUkzzQVEgjdS3pA08\n\tvE6k39soXZSyi+cr3QskPhMu2pCraqnJl2Q4HiDdnxOkPt9SYjJmILizKzKjaQkdXquC\n\tfsnx8fHuRqucYBsKLEGB0GPrz3vGuTqmhchEh+wSoMk+8/3l4SXkR/w6CjHiV3f/i3gO\n\tSdc9LFGdBWcM69k0/CTDvm/zFJMH0fG+1tQ8fkhfCnwNIgR+3mnRmHx/vseqrXag5jWG\n\tW++g==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=9hhuiBolZ7wClGvze5UYo9ZIBCuRQRN7wF3bu3CuwhE=;\n\tb=VsDmRLl7kwcCDahyjOjRqOaVumfRFHIZ07PCmpJUPqrp81NJZxqa7T76N4oGcAHvba\n\ts52AuwUU9SIpN19MZpfUXbpwvDV8Erq8C93ov2EgnRGvLLaHr4fJA93gqRh139B/TlPb\n\tlxuGdAYBeD3hI1YjbJcYlp+LixI279usWjEGZ4Jl6urjR0vXBvf/HLaTaXklAr0kW2E6\n\t8Eg10WlDZ7cWxzuEmBeiy8k3/UeTYCSkHGeTvhexiwrlVPkhdEMG3TJWa/r/+TB10zfh\n\tbHhwLYi7uDlDkD55gE8wBaVt1EdZV8SpR1n/8yZuhIEhgmoV6JXSywqq0MmOdJj6ZJkK\n\tWrAw==","X-Gm-Message-State":"AHPjjUiaNG9hlkZQGleiesWe+Gw3GZZ2TjqmqiJEWTfneXialjTHnIY3\n\tCbEO76UGJyp4bmL4","X-Google-Smtp-Source":"AOwi7QBHhc/i06t6Cssa4mx3qYKQhHiO8XATxsWaDlyMHP4a8bW49ZIjpHlVEQFZMCfIcUbXYI0rqA==","X-Received":"by 10.98.2.1 with SMTP id 1mr1674536pfc.93.1505701307981;\n\tSun, 17 Sep 2017 19:21:47 -0700 (PDT)","Date":"Mon, 18 Sep 2017 10:21:40 +0800","From":"Yuanhan Liu ","To":"Darrell Ball ","Message-ID":"<20170918022140.GP2050@yliu-home>","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<20170908164849.GH7356@vergenet.net>\n\t<20170911061425.GN9736@yliu-home>\n\t<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>\n\t<20170915083419.GI2050@yliu-home>\n\t<64837D85-3898-403C-9F40-A0EDBB1448B1@vmware.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<64837D85-3898-403C-9F40-A0EDBB1448B1@vmware.com>","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ,\n\tSimon Horman ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1772508,"web_url":"http://patchwork.ozlabs.org/comment/1772508/","msgid":"<09481AFC-6423-46A4-958B-82A3008C55E3@vmware.com>","list_archive_url":null,"date":"2017-09-21T07:55:53","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":68212,"url":"http://patchwork.ozlabs.org/api/people/68212/","name":"Darrell Ball","email":"dball@vmware.com"},"content":"Hi Yuanhan\r\n\r\nWe discussed the RTE_FLOW_ACTION_TYPE_RSS action, in the dpdk meeting.\r\nThere is related discussion and details in this thread.\r\nIt seems the RTE_FLOW_ACTION_TYPE_RSS action can be supported by the drivers.\r\nIf this is the case, then:\r\n1/ We could program only one try with this action.\r\n2/ Masked flows would be distributed to different PMDs as is otherwise true today\r\n3/ Since multiple PMDs could see the same flow, the mark would be global across PMDs.\r\n\r\nCan you check if this can work ?\r\n\r\nThanks Darrell\r\n\r\nOn 9/15/17, 12:55 PM, \"Finn Christensen\" wrote:\r\n\r\n \r\n \r\n -----Original Message-----\r\n From: Darrell Ball [mailto:dball@vmware.com]\r\n Sent: 15. september 2017 20:09\r\n To: Finn Christensen \r\n Cc: dev@openvswitch.org\r\n Subject: Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action\r\n \r\n \r\n \r\n On 9/15/17, 2:16 AM, \"Finn Christensen\" wrote:\r\n \r\n \r\n \r\n -----Original Message-----\r\n From: Darrell Ball [mailto:dball@vmware.com]\r\n Sent: 14. september 2017 21:35\r\n To: Finn Christensen \r\n Cc: dev@openvswitch.org\r\n Subject: Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue\r\n action\r\n \r\n \r\n \r\n On 9/14/17, 1:14 AM, \"Finn Christensen\" wrote:\r\n \r\n \r\n \r\n \r\n \r\n -----Original Message-----\r\n \r\n From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev-\r\n bounces@openvswitch.org] On Behalf Of Darrell Ball\r\n \r\n Sent: 13. september 2017 18:18\r\n \r\n To: Simon Horman \r\n \r\n Cc: dev@openvswitch.org\r\n \r\n Subject: Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue\r\n action\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n On 9/13/17, 2:57 AM, \"Simon Horman\"\r\n wrote:\r\n \r\n \r\n \r\n On Tue, Sep 12, 2017 at 08:36:19AM +0000, Darrell Ball wrote:\r\n \r\n >\r\n \r\n > On 9/10/17, 11:14 PM, \"ovs-dev-bounces@openvswitch.org on\r\n behalf of Yuanhan Liu\" wrote:\r\n \r\n >\r\n \r\n > On Fri, Sep 08, 2017 at 06:48:50PM +0200, Simon Horman\r\n wrote:\r\n \r\n > > On Tue, Sep 05, 2017 at 05:22:59PM +0800, Yuanhan Liu\r\n wrote:\r\n \r\n > > > From: Finn Christensen \r\n \r\n > > >\r\n \r\n > > > AFAIK, most (if not all) NICs (including Mellanox and Intel)\r\n do\r\n not\r\n \r\n > > > support a pure MARK action. It's required to be used\r\n together\r\n with\r\n \r\n > > > some other actions, like QUEUE.\r\n \r\n > > >\r\n \r\n > > > To workaround it, retry with a queue action when first try\r\n failed.\r\n \r\n > > >\r\n \r\n > > > Moreover, some Intel's NIC (say XL710) needs the QUEUE\r\n action set\r\n \r\n > > > before the MARK action.\r\n \r\n > > >\r\n \r\n > > > Co-authored-by: Yuanhan Liu \r\n \r\n > > > Signed-off-by: Finn Christensen \r\n \r\n > > > Signed-off-by: Yuanhan Liu \r\n \r\n > >\r\n \r\n > > This feels a bit like the tail wagging the dog.\r\n \r\n > > Is this the lowest level at which it makes sense to implement\r\n \r\n > > this logic?\r\n \r\n > >\r\n \r\n > > If so then I wonder if some sort of probing would be in order\r\n \r\n > > to avoid the cost of trying to add the flow twice to hardware\r\n \r\n > > where the queue is required.\r\n \r\n >\r\n \r\n > Do you mean something like rte_flow capability query, like\r\n whether\r\n \r\n > a queue action is needed for a mark action? If so, yes, I do\r\n think\r\n \r\n > we miss an interface like this.\r\n \r\n >\r\n \r\n > Note that even in this solution, the flow won't be created\r\n twice\r\n \r\n > to the hardware, because the first try would be failed.\r\n \r\n >\r\n \r\n > [Darrell]\r\n \r\n >\r\n \r\n > Having an api to quey capability and avoid the first try to\r\n HW\r\n would be nice, but there are dependencies\r\n \r\n > on RTE, drivers etc and I don’t know definitive the api\r\n would\r\n be.\r\n \r\n >\r\n \r\n > Also, as nics are added this capability needs to be done\r\n and\r\n state needs to be kept in all cases.\r\n \r\n >\r\n \r\n > It is an enhancement and if done should be reliable.\r\n \r\n \r\n \r\n Agreed. Though I was more thinking of probing the hardware\r\n rather\r\n than\r\n \r\n having a capability API - I expect this would remove several of the\r\n \r\n dependencies you describe above.\r\n \r\n \r\n \r\n \r\n \r\n [Darrell] I have been pondering the probing option as well. It is\r\n certainly a\r\n valid option; we use it in other cases such as datapath probing. One of\r\n the\r\n aspects that worries me here is maintaining the correct per interface\r\n (essentially; although the attribute is per nic) state across various\r\n events\r\n such as new ports being added, vswitchd restarts, races with flow\r\n creation.\r\n It would be non-trivial I guess and probably appropriate for the next\r\n patch\r\n series, if done.\r\n \r\n \r\n \r\n In this case, we have what seems like a clear distinction b/w\r\n Napatech\r\n which does not need the queue action workaround and everything\r\n else,\r\n which does.\r\n \r\n Besides the non-Napatech behavior, which is worrisome,\r\n maintaining the\r\n difference for flow handling under the covers is concerning.\r\n \r\n \r\n \r\n I wonder if we should be upfront as possible here and just have a\r\n dpdk\r\n interface configuration – maybe something like “supports native\r\n HWOL\r\n mark action” since the better behavior is the exception?\r\n \r\n The interface config would be more robust than probing.\r\n \r\n This would need documentation, of course.\r\n \r\n \r\n \r\n [Finn]\r\n \r\n I think the rte queue action should never be used here when using\r\n partial HWOL. Not the way OVS handles multi queues today.\r\n \r\n Maybe a \"default queue\" could be used in the dpdk PMDs when no\r\n queue is specified in rte flow?\r\n \r\n [Darrell] This is the Napatech case, where no queue action is needed;\r\n you\r\n are suggesting programming a default queue in this case.\r\n I don’t follow how this would be helpful/desired?\r\n \r\n [Finn]\r\n I was trying to make my view on this, not particularly arguing for the\r\n Napatech\r\n Case.\r\n Here is what I was thinking:\r\n Taking the case of this partial HWOL, then we are trying to offload the\r\n flow\r\n classification to HW, like \"pre-classify and mark\". Then this mark is used\r\n to\r\n accelerate OVS while finding the actions to execute. Since we do leave all\r\n processing of the actions to OVS, there is no way for the partial HWOL to\r\n know, at rte flow creation time, where to send the pre-classified packets\r\n (which is strictly needed when seen from a DPDK rte flow point of view).\r\n When multiple queues are specified in OVS, a hash splitting mechanism\r\n is used in the nic to support RSS. Then the nic is responsible for selecting\r\n the\r\n right queue according to the configured algorithm for RSS.\r\n OVS only needs to know how many queues to service per port. No\r\n knowledge\r\n about the association b/w flow <-> rxq is used in OVS today.\r\n When looking at full HWOL, all flow actions will have to be interpreted,\r\n supported\r\n and programmed to NIC using rte flow - send directly to target port and\r\n we\r\n do not have this issue.\r\n Have I missed something?\r\n \r\n [Darrell]\r\n I understand you points about the full offload you are working on.\r\n I just did not follow this one comment:\r\n \r\n “Maybe a \"default queue\" could be used in the dpdk PMDs when no\r\n queue is specified in rte flow?”\r\n \r\n which I thought is related to partial offloads used here and the receive\r\n queue action we are discussing?; I am concerned that I missed your point\r\n about this specific comment ?\r\n \r\n \r\n [Finn]\r\n Sorry for my messy descriptions, and yes the \"default queue\" idea was for the\r\n partial HWOL case. I just think if the queue could be set to 0 if 1 rxq used and let\r\n the hash algorithm set the queue if RSS is used, then we do not have to add QUEUE\r\n to the rte flow create.\r\n However, it was just an idea and I don't know if it can be done reliably in all DPDK PMDs.\r\n \r\n \r\n Essentially, this is a mismatch between the rte flow impl\r\n functionality in\r\n PMD and the needs in OVS for flow classification, in a partial HWOL\r\n setup.\r\n \r\n \r\n [Darrell] Maybe this has been explored, but were additional\r\n workaround\r\n actions, like RTE_FLOW_ACTION_TYPE_RSS considered ?\r\n I guess we could make this essentially a NOP, but if mark\r\n action\r\n could ride along with it, then that would be good.\r\n \r\n [Finn]\r\n No. We have not explored that.\r\n I'm not sure I understand how you would use\r\n RTE_FLOW_ACTION_TYPE_RSS\r\n In this situation.\r\n \r\n \r\n [Darrell]\r\n So there are two points here:\r\n 1/ ‘IF’ RTE_FLOW_ACTION_TYPE_RSS can used in lieu of\r\n RTE_FLOW_ACTION_TYPE_QUEUE combined with\r\n RTE_FLOW_ACTION_TYPE_MARK, as need for the workaround for most\r\n nics, then this would be better for distributing packets across PMDs for the\r\n masked flows relevant here. This would help mitigate one of the concerns.\r\n We could use the RSS we use globally, essentially making the distribution a\r\n NOP compared with what we normally do today with RSS….\r\n \r\n 2/ Assuming ‘1’, we might even use this in all cases as a first try for all nics,\r\n which if it could be done solves the queue action workaround\r\n capability/probing thing?\r\n \r\n [Finn] Thanks, I see what you mean. It makes very good sense if it is supported\r\n on the nic DPDK PMDs.\r\n\r\n \r\n \r\n However, we would not mind Darells proposal, it makes sense since\r\n most nic PMDs unfortunately needs this and it is only relevant for\r\n partial\r\n HWOL. We are mostly concerned with full HWOL and therefore see\r\n partial\r\n HWOL as a failover when full HWOL could not be handled in nic, if\r\n enabled.\r\n \r\n \r\n \r\n \r\n \r\n I think anyways we need documentation describing the difference\r\n b/w\r\n nics in the dpdk documentation (howto part).\r\n \r\n \r\n \r\n \r\n \r\n Assuming no such enhancement is appropriate at this time I\r\n would\r\n \r\n still like to ask if this is the best place for this hardware-specific\r\n code?\r\n \r\n \r\n \r\n [Darrell]\r\n \r\n \r\n \r\n For OVS, the netdev-dpdk layer is the lowest layer.\r\n \r\n This kind of workaround is hard to hide, since we are messing with\r\n the\r\n rxq, so I think OVS needs to know that it is in effect anyways. An\r\n alternative is to supply a mark and an ‘optional queue’ and let the\r\n driver\r\n decide if the queue is needed and report back whether it was. This\r\n would\r\n be hard to do across various drivers. Supporting in the rte layer would\r\n require both rte and driver support, so even more support.\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n > A separate comment is we need to document which nics\r\n need\r\n the queue action.\r\n \r\n >\r\n \r\n > Also, I think we should check errno in the present code.\r\n \r\n \r\n \r\n \r\n \r\n _______________________________________________\r\n \r\n dev mailing list\r\n \r\n dev@openvswitch.org\r\n \r\n https://urldefense.proofpoint.com/v2/url?u=https-\r\n 3A__mail.openvswitch.org_mailman_listinfo_ovs-\r\n \r\n 2Ddev&d=DwIGaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-\r\n uZnsw&m=IHypHavCy0AKjNxqOMyc4w3ILyC-\r\n \r\n BuwkB8fuVvQUA3k&s=deJQWP9KI22Xp46tEoZ6o6Emitr3Bhfd7iSMxNpude\r\n g&e=\r\n \r\n Disclaimer: This email and any files transmitted with it may contain\r\n confidential information intended for the addressee(s) only. The\r\n information is not to be surrendered or copied to unauthorized\r\n persons. If\r\n you have received this communication in error, please notify the\r\n sender\r\n immediately and delete this e-mail from your system.\r\n \r\n \r\n \r\n Disclaimer: This email and any files transmitted with it may contain\r\n confidential information intended for the addressee(s) only. The\r\n information is not to be surrendered or copied to unauthorized persons. If\r\n you have received this communication in error, please notify the sender\r\n immediately and delete this e-mail from your system.\r\n \r\n \r\n Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system.","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=onevmw.onmicrosoft.com\n\theader.i=@onevmw.onmicrosoft.com header.b=\"MWaRUQXZ\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=dball@vmware.com; "],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xyTSf4YGQz9t3m\n\tfor ;\n\tThu, 21 Sep 2017 17:56:02 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id B18AEABA;\n\tThu, 21 Sep 2017 07:55:58 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id A702FA7F\n\tfor ; Thu, 21 Sep 2017 07:55:57 +0000 (UTC)","from NAM01-BN3-obe.outbound.protection.outlook.com\n\t(mail-bn3nam01on0060.outbound.protection.outlook.com [104.47.33.60])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id 6361217E\n\tfor ; Thu, 21 Sep 2017 07:55:56 +0000 (UTC)","from MWHPR05MB3406.namprd05.prod.outlook.com (10.174.175.155) by\n\tMWHPR05MB3407.namprd05.prod.outlook.com (10.174.175.156) with\n\tMicrosoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id\n\t15.20.77.5; Thu, 21 Sep 2017 07:55:53 +0000","from MWHPR05MB3406.namprd05.prod.outlook.com ([10.174.175.155]) by\n\tMWHPR05MB3406.namprd05.prod.outlook.com ([10.174.175.155]) with\n\tmapi id 15.20.0077.007; Thu, 21 Sep 2017 07:55:53 +0000"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=onevmw.onmicrosoft.com; s=selector1-vmware-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=zNjwnWeVuc/SQO+HSkLI942j8T0c9VQNpfa/xs/vluY=;\n\tb=MWaRUQXZb08WVg7ndA8MmMzokk1y2601P5H4LlPkCw1AVj7aEFpWtvNOm3kOorqvrItRoOogGFBMaa6MfwPuPKD4TQLAlisqc540frqN9Q/UJRWVdHZxkmNvCCOpzXCmMmuknOn6ZKy1KbfPgGYo1f9eI+xCs20aBC1xudaI/EE=","From":"Darrell Ball ","To":"Finn Christensen , Yuanhan Liu ","Thread-Topic":"[ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","Thread-Index":"AQHTJiizNzCJ8+cj4EuMMlhNVpa+HKKrOF8AgAQFvoCAAUSgAIACHl+A///02QCAAYCtgIAASOOAgAFanQCAAB+5AIAAkvqAgAik8oA=","Date":"Thu, 21 Sep 2017 07:55:53 +0000","Message-ID":"<09481AFC-6423-46A4-958B-82A3008C55E3@vmware.com>","References":"<1504603381-30071-1-git-send-email-yliu@fridaylinux.org>\n\t<1504603381-30071-7-git-send-email-yliu@fridaylinux.org>\n\t<20170908164849.GH7356@vergenet.net>\n\t<20170911061425.GN9736@yliu-home>\n\t<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>\n\t<20170913095730.GE27555@vergenet.net>\n\t\n\t<3edc7d93b8c94c3c8e316b299981b984@napatech.com>\n\t\n\t\n\t\n\t","In-Reply-To":"","Accept-Language":"en-US","Content-Language":"en-US","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","user-agent":"Microsoft-MacOutlook/f.26.0.170902","authentication-results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=onevmw.onmicrosoft.com\n\theader.i=@onevmw.onmicrosoft.com header.b=\"MWaRUQXZ\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=dball@vmware.com; "],"x-originating-ip":"[73.162.236.45]","x-ms-publictraffictype":"Email","x-microsoft-exchange-diagnostics":"1; MWHPR05MB3407;\n\t20:nWaOc5mocHFAj6PF/+l/veLRNpzT3qKivHFKweAxzMul2ugKUKDEgWGPSJdur+7Slh0Hn5Qsm2H9AkeAlu/qqHWp7d+5VqJ84C5vJRUh5jPlH3zHPJQbmnK6ooVwWHZUu8mnNqC5pz2uXB4B8+khb44BZpyDOtW+Wta0tXaJvWc=","x-ms-exchange-antispam-srfa-diagnostics":"SSOS;","x-ms-office365-filtering-correlation-id":"6753523b-f827-4ef9-797b-08d500c62f37","x-microsoft-antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:MWHPR05MB3407; ","x-ms-traffictypediagnostic":"MWHPR05MB3407:","x-exchange-antispam-report-test":"UriScan:(61668805478150)(10436049006162)(216315784871565); ","x-microsoft-antispam-prvs":"","x-exchange-antispam-report-cfa-test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(10201501046)(93006095)(93001095)(100000703101)(100105400095)(3002001)(6041248)(20161123564025)(20161123558100)(20161123555025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:MWHPR05MB3407; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:MWHPR05MB3407; ","x-forefront-prvs":"04371797A5","x-forefront-antispam-report":"SFV:NSPM;\n\tSFS:(10009020)(6009001)(376002)(346002)(13464003)(24454002)(377454003)(189002)(199003)(54094003)(36756003)(93886005)(25786009)(76176999)(77096006)(5660300001)(6116002)(102836003)(3846002)(478600001)(966005)(101416001)(110136005)(54356999)(50986999)(8676002)(105586002)(561944003)(3280700002)(6486002)(3660700001)(97736004)(33656002)(2906002)(81156014)(8936002)(189998001)(6436002)(82746002)(2950100002)(53946003)(99286003)(7736002)(58126008)(6246003)(305945005)(66066001)(2900100001)(6512007)(53546010)(14454004)(6306002)(53936002)(68736007)(4326008)(83716003)(83506001)(106356001)(316002)(86362001)(575784001)(6506006)(81166006)(229853002)(19627235001);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR05MB3407;\n\tH:MWHPR05MB3406.namprd05.prod.outlook.com; FPR:; SPF:None;\n\tPTR:InfoNoRecords; MX:1; A:1; LANG:en; ","received-spf":"None (protection.outlook.com: vmware.com does not designate\n\tpermitted sender hosts)","spamdiagnosticoutput":"1:99","spamdiagnosticmetadata":"NSPM","Content-ID":"","MIME-Version":"1.0","X-OriginatorOrg":"vmware.com","X-MS-Exchange-CrossTenant-originalarrivaltime":"21 Sep 2017 07:55:53.6920\n\t(UTC)","X-MS-Exchange-CrossTenant-fromentityheader":"Hosted","X-MS-Exchange-CrossTenant-id":"b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"MWHPR05MB3407","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1772568,"web_url":"http://patchwork.ozlabs.org/comment/1772568/","msgid":"<20170921085133.GC2251@yliu-home>","list_archive_url":null,"date":"2017-09-21T08:51:33","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":72215,"url":"http://patchwork.ozlabs.org/api/people/72215/","name":"Yuanhan Liu","email":"yliu@fridaylinux.org"},"content":"On Thu, Sep 21, 2017 at 07:55:53AM +0000, Darrell Ball wrote:\n> Hi Yuanhan\n> \n> We discussed the RTE_FLOW_ACTION_TYPE_RSS action, in the dpdk meeting.\n> There is related discussion and details in this thread.\n> It seems the RTE_FLOW_ACTION_TYPE_RSS action can be supported by the drivers.\n> If this is the case, then:\n> 1/ We could program only one try with this action.\n> 2/ Masked flows would be distributed to different PMDs as is otherwise true today\n> 3/ Since multiple PMDs could see the same flow, the mark would be global across PMDs.\n\nI have already made it to global in v3. Hopefully I could send it out by\nthe end of tomorrow.\n\n> \n> Can you check if this can work ?\n\nSure. Just want to make it clearn, if it works, should we still keep the\nqueue index workaround? Or, just remove it?\n\nThanks.\n\n\t--yliu","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=fridaylinux-org.20150623.gappssmtp.com\n\theader.i=@fridaylinux-org.20150623.gappssmtp.com\n\theader.b=\"cUd62Iul\"; dkim-atps=neutral"],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xyVhz05F3z9s7g\n\tfor ;\n\tThu, 21 Sep 2017 18:51:46 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id 6A644B6E;\n\tThu, 21 Sep 2017 08:51:43 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id C1E13B0C\n\tfor ; Thu, 21 Sep 2017 08:51:42 +0000 (UTC)","from mail-pg0-f51.google.com (mail-pg0-f51.google.com\n\t[74.125.83.51])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id D064E204\n\tfor ; Thu, 21 Sep 2017 08:51:41 +0000 (UTC)","by mail-pg0-f51.google.com with SMTP id i130so3186398pgc.3\n\tfor ; Thu, 21 Sep 2017 01:51:41 -0700 (PDT)","from yliu-home ([45.63.61.64]) by smtp.gmail.com with ESMTPSA id\n\tp70sm1744030pfk.130.2017.09.21.01.51.38\n\t(version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128);\n\tThu, 21 Sep 2017 01:51:40 -0700 (PDT)"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=fridaylinux-org.20150623.gappssmtp.com; s=20150623;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=iOBuFX1+G9OZv7H1vNt2GFNUfxYuOlF8YQJFsDV/b3s=;\n\tb=cUd62Iulev+v4fynCbqZDnWgR7p2f+dehYJctOwCOWMrlSrbB7+f6KjpW6tieRzVDg\n\tI0z2e4QigAmVHIhlV15Bm0OA+OIhRriCSQjl7rny4+7S2Q40p/4pUndogwTc44GHc3fF\n\tIEX3u2fNzzbsv8NNv5cHKwNNJnbiFaz5cjLINTS2Znsi98W6iHdrJ7gXHcjTonl54Heo\n\tCl6YkvSn43Jvdm7hi9r2pRa77p5BbT1eIEZFiFpOdaeV6Kci8yL1boGjsjz5zZlqei6m\n\tq8Pr9yABqytOcrjUUjn6sA/uf/6KdNIIyXusXiNOD9LcBg1Ylge2wIzES0J7JfS6Sqxs\n\txqfw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=iOBuFX1+G9OZv7H1vNt2GFNUfxYuOlF8YQJFsDV/b3s=;\n\tb=euw9z/X27uX8tMbeB68drklArTnEd8cJIfEIlqcG5Xyg6nmOBI5ilhdeW3Cz5Payrr\n\tOmblPisvDXq/MIr91lgjzJ9MPSkIC7K02UGlQfCf6RDwjNaoejhkhbNNVwVaHQc28+Ac\n\tbEfC+klqwrpw2aNYlkPLx1F2iIl1b4JrxjBjtN4KVRsfBKBGBUOmmTzthxAvnUqF4TbT\n\thIgOJ/r6P9XDJ8FEdKLPllSbojJm4+dsroKMjM4MaZb/kwsdeNQcVEwvaMwe8Eg5lzHb\n\trQqCOSEehrQr8VmW6wBmecYTdzkQqcOdq+x+l7SoS7DiWFqf5PxwSjtp4BdClQ4hJvYw\n\t/L+g==","X-Gm-Message-State":"AHPjjUifj9/MfAXlSWPv6Xcdvq/Ayy+L+3diHe+5fszGR8E503mH1God\n\to4Zk2t199c9ogs2Bm0rnHcJ90Z26pYs=","X-Google-Smtp-Source":"AOwi7QBxihoeYYQWYhoixKs0nj3WtbK1amuKsw34l4aclBkqc9YeEBRdjf72hp1MsOD7HvX49Zk8Ig==","X-Received":"by 10.98.76.134 with SMTP id e6mr5001992pfj.180.1505983901419;\n\tThu, 21 Sep 2017 01:51:41 -0700 (PDT)","Date":"Thu, 21 Sep 2017 16:51:33 +0800","From":"Yuanhan Liu ","To":"Darrell Ball ","Message-ID":"<20170921085133.GC2251@yliu-home>","References":"<20170911061425.GN9736@yliu-home>\n\t<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>\n\t<20170913095730.GE27555@vergenet.net>\n\t\n\t<3edc7d93b8c94c3c8e316b299981b984@napatech.com>\n\t\n\t\n\t\n\t\n\t<09481AFC-6423-46A4-958B-82A3008C55E3@vmware.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<09481AFC-6423-46A4-958B-82A3008C55E3@vmware.com>","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}},{"id":1772588,"web_url":"http://patchwork.ozlabs.org/comment/1772588/","msgid":"","list_archive_url":null,"date":"2017-09-21T09:07:19","subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","submitter":{"id":68212,"url":"http://patchwork.ozlabs.org/api/people/68212/","name":"Darrell Ball","email":"dball@vmware.com"},"content":"On 9/21/17, 1:51 AM, \"Yuanhan Liu\" wrote:\n\n On Thu, Sep 21, 2017 at 07:55:53AM +0000, Darrell Ball wrote:\n > Hi Yuanhan\n > \n > We discussed the RTE_FLOW_ACTION_TYPE_RSS action, in the dpdk meeting.\n > There is related discussion and details in this thread.\n > It seems the RTE_FLOW_ACTION_TYPE_RSS action can be supported by the drivers.\n > If this is the case, then:\n > 1/ We could program only one try with this action.\n > 2/ Masked flows would be distributed to different PMDs as is otherwise true today\n > 3/ Since multiple PMDs could see the same flow, the mark would be global across PMDs.\n \n I have already made it to global in v3. Hopefully I could send it out by\n the end of tomorrow.\n \n > \n > Can you check if this can work ?\n \n Sure. Just want to make it clearn, if it works, should we still keep the\n queue index workaround? Or, just remove it?\n\nThe second try with a single queue would not be needed.\nThe first try should always (hopefully) work for RTE_FLOW_ACTION_TYPE_RSS\nThis would need to include all the queues used for the port in question\nand the same RSS config we use, which is in netdev-dpdk.\nIt is a bit more work overall.\n\nDarrell\n\n \n Thanks.\n \n \t--yliu","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=onevmw.onmicrosoft.com\n\theader.i=@onevmw.onmicrosoft.com header.b=\"otHNeKV5\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=dball@vmware.com; "],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xyW3k2DPLz9sBZ\n\tfor ;\n\tThu, 21 Sep 2017 19:08:02 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id 56A52BD6;\n\tThu, 21 Sep 2017 09:07:24 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id E16D9BC0\n\tfor ; Thu, 21 Sep 2017 09:07:22 +0000 (UTC)","from NAM02-CY1-obe.outbound.protection.outlook.com\n\t(mail-cys01nam02on0063.outbound.protection.outlook.com\n\t[104.47.37.63])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id 5DFB81AE\n\tfor ; Thu, 21 Sep 2017 09:07:22 +0000 (UTC)","from MWHPR05MB3406.namprd05.prod.outlook.com (10.174.175.155) by\n\tMWHPR05MB3407.namprd05.prod.outlook.com (10.174.175.156) with\n\tMicrosoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id\n\t15.20.77.5; Thu, 21 Sep 2017 09:07:20 +0000","from MWHPR05MB3406.namprd05.prod.outlook.com ([10.174.175.155]) by\n\tMWHPR05MB3406.namprd05.prod.outlook.com ([10.174.175.155]) with\n\tmapi id 15.20.0077.007; Thu, 21 Sep 2017 09:07:20 +0000"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=onevmw.onmicrosoft.com; s=selector1-vmware-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=Kfgzp4njYrkJK2umVvcPgGHt+WISOOqX5u0lA6Ag2pw=;\n\tb=otHNeKV5QCl1SezAoRgrXXKXnKqb9qtEA+cRIkIQSz6Bt58OviKCJkb7B2nEhMQRjuUpvBYSyTFpf8dgxWQam92G+OH1qWCpUm+1r4Nfuu57uyR0D7GibTO7f6TQpxnYPsm++ZJ5Kx1C7NBAp4qfCcUjjaFr35vdFuuQUSypHx8=","From":"Darrell Ball ","To":"Yuanhan Liu ","Thread-Topic":"[ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","Thread-Index":"AQHTJiizNzCJ8+cj4EuMMlhNVpa+HKKrOF8AgAQFvoCAAUSgAIACHl+A///02QCAAYCtgIAASOOAgAFanQCAAB+5AIAAkvqAgAik8oCAAA+NgIAABGiA","Date":"Thu, 21 Sep 2017 09:07:19 +0000","Message-ID":"","References":"<20170911061425.GN9736@yliu-home>\n\t<882899CE-6F26-472D-85B9-6A8BB80B9B11@vmware.com>\n\t<20170913095730.GE27555@vergenet.net>\n\t\n\t<3edc7d93b8c94c3c8e316b299981b984@napatech.com>\n\t\n\t\n\t\n\t\n\t<09481AFC-6423-46A4-958B-82A3008C55E3@vmware.com>\n\t<20170921085133.GC2251@yliu-home>","In-Reply-To":"<20170921085133.GC2251@yliu-home>","Accept-Language":"en-US","Content-Language":"en-US","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","user-agent":"Microsoft-MacOutlook/f.26.0.170902","authentication-results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=onevmw.onmicrosoft.com\n\theader.i=@onevmw.onmicrosoft.com header.b=\"otHNeKV5\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=dball@vmware.com; "],"x-originating-ip":"[73.162.236.45]","x-ms-publictraffictype":"Email","x-microsoft-exchange-diagnostics":"1; MWHPR05MB3407;\n\t20:xuiim58iJGPTTc71kRks8YW8cnyWaR75t8neQPfdqNxh43NLQuWcChMlsLrWc2D6WsjClRM8buymMGnHWV/Ow7IsFmnxY+85GZ0RtbIZx8d+RoMdDjOLc/CdNZuWRO5vcAQ+ql0lliuqt4HihTT4a8edEtBwUJ4TonNn+XrItcY=","x-ms-exchange-antispam-srfa-diagnostics":"SSOS;","x-ms-office365-filtering-correlation-id":"0efc7426-9879-4f03-e5ed-08d500d02a07","x-microsoft-antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:MWHPR05MB3407; ","x-ms-traffictypediagnostic":"MWHPR05MB3407:","x-exchange-antispam-report-test":"UriScan:;","x-microsoft-antispam-prvs":"","x-exchange-antispam-report-cfa-test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(10201501046)(93006095)(93001095)(100000703101)(100105400095)(3002001)(6041248)(20161123564025)(20161123558100)(20161123555025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:MWHPR05MB3407; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:MWHPR05MB3407; ","x-forefront-prvs":"04371797A5","x-forefront-antispam-report":"SFV:NSPM;\n\tSFS:(10009020)(6009001)(376002)(346002)(189002)(199003)(24454002)(377454003)(6916009)(6512007)(2900100001)(53936002)(99286003)(6246003)(305945005)(66066001)(58126008)(7736002)(53546010)(14454004)(81156014)(2906002)(2950100002)(189998001)(8936002)(82746002)(6436002)(6506006)(54906003)(229853002)(81166006)(68736007)(83716003)(4326008)(316002)(106356001)(86362001)(83506001)(478600001)(3846002)(25786009)(36756003)(93886005)(102836003)(77096006)(6116002)(5660300001)(76176999)(3280700002)(33656002)(97736004)(6486002)(3660700001)(50986999)(54356999)(101416001)(105586002)(8676002);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR05MB3407;\n\tH:MWHPR05MB3406.namprd05.prod.outlook.com; FPR:; SPF:None;\n\tPTR:InfoNoRecords; MX:1; A:1; LANG:en; ","received-spf":"None (protection.outlook.com: vmware.com does not designate\n\tpermitted sender hosts)","spamdiagnosticoutput":"1:99","spamdiagnosticmetadata":"NSPM","Content-ID":"<9A4207A63CE257439FA94B26FD7DB123@namprd05.prod.outlook.com>","MIME-Version":"1.0","X-OriginatorOrg":"vmware.com","X-MS-Exchange-CrossTenant-originalarrivaltime":"21 Sep 2017 09:07:20.0057\n\t(UTC)","X-MS-Exchange-CrossTenant-fromentityheader":"Hosted","X-MS-Exchange-CrossTenant-id":"b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"MWHPR05MB3407","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"\"dev@openvswitch.org\" ","Subject":"Re: [ovs-dev] [PATCH v2 6/8] netdev-dpdk: retry with queue action","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"}}]