{"id":818455,"url":"http://patchwork.ozlabs.org/api/covers/818455/?format=json","web_url":"http://patchwork.ozlabs.org/project/openvswitch/cover/1506404199-23579-1-git-send-email-yliu@fridaylinux.org/","project":{"id":47,"url":"http://patchwork.ozlabs.org/api/projects/47/?format=json","name":"Open vSwitch","link_name":"openvswitch","list_id":"ovs-dev.openvswitch.org","list_email":"ovs-dev@openvswitch.org","web_url":"http://openvswitch.org/","scm_url":"git@github.com:openvswitch/ovs.git","webscm_url":"https://github.com/openvswitch/ovs","list_archive_url":"","list_archive_url_format":"","commit_url_format":""},"msgid":"<1506404199-23579-1-git-send-email-yliu@fridaylinux.org>","list_archive_url":null,"date":"2017-09-26T05:36:30","name":"[ovs-dev,v3,0/9] OVS-DPDK flow offload with rte_flow","submitter":{"id":72215,"url":"http://patchwork.ozlabs.org/api/people/72215/?format=json","name":"Yuanhan Liu","email":"yliu@fridaylinux.org"},"mbox":"http://patchwork.ozlabs.org/project/openvswitch/cover/1506404199-23579-1-git-send-email-yliu@fridaylinux.org/mbox/","series":[{"id":5061,"url":"http://patchwork.ozlabs.org/api/series/5061/?format=json","web_url":"http://patchwork.ozlabs.org/project/openvswitch/list/?series=5061","date":"2017-09-26T05:36:30","name":"OVS-DPDK flow offload with rte_flow","version":3,"mbox":"http://patchwork.ozlabs.org/series/5061/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/covers/818455/comments/","headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","ovs-dev@mail.linuxfoundation.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=openvswitch.org\n\t(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;\n\tenvelope-from=ovs-dev-bounces@openvswitch.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=fridaylinux-org.20150623.gappssmtp.com\n\theader.i=@fridaylinux-org.20150623.gappssmtp.com\n\theader.b=\"Zji2g+ZF\"; dkim-atps=neutral"],"Received":["from mail.linuxfoundation.org (mail.linuxfoundation.org\n\t[140.211.169.12])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y1V8V49ybz9t30\n\tfor ;\n\tTue, 26 Sep 2017 15:37:30 +1000 (AEST)","from mail.linux-foundation.org (localhost [127.0.0.1])\n\tby mail.linuxfoundation.org (Postfix) with ESMTP id 890F0ABC;\n\tTue, 26 Sep 2017 05:37:28 +0000 (UTC)","from smtp1.linuxfoundation.org (smtp1.linux-foundation.org\n\t[172.17.192.35])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPS id 7A77BAB9\n\tfor ; Tue, 26 Sep 2017 05:37:27 +0000 (UTC)","from mail-pg0-f48.google.com (mail-pg0-f48.google.com\n\t[74.125.83.48])\n\tby smtp1.linuxfoundation.org (Postfix) with ESMTPS id EFAA4D3\n\tfor ; Tue, 26 Sep 2017 05:37:26 +0000 (UTC)","by mail-pg0-f48.google.com with SMTP id i195so5358776pgd.9\n\tfor ; Mon, 25 Sep 2017 22:37:26 -0700 (PDT)","from localhost.localdomain ([101.228.205.132])\n\tby smtp.gmail.com with ESMTPSA id\n\to79sm13180077pfi.108.2017.09.25.22.37.15\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128);\n\tMon, 25 Sep 2017 22:37:24 -0700 (PDT)"],"X-Greylist":"whitelisted by SQLgrey-1.7.6","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=fridaylinux-org.20150623.gappssmtp.com; s=20150623;\n\th=from:to:cc:subject:date:message-id;\n\tbh=L32x2K1oDI1fc/W8EfWl4m48JADmcomBaiadG8IxrVE=;\n\tb=Zji2g+ZFqlusUb5hdk4p12H9Jfl2e0gYS1Y7hzksmMV411FGonwqp6RPjDWz2UJsxO\n\tgWhQN1bpc1Y+c6HlnM9HkUbTjwXS1HAb3uS8XXihfwIZEK80qqOBTBDWSccfVSPDArD0\n\tSeYkf+bN6wMEQN/Qo0jDDxucoIVMSlrKav3IOyKaZC7K9U1Z9OTaNwzQxce7M4mJb4Xm\n\tnuRhJ5G4YDoyiVDyKQogiVTTLUPev6Q+3PWmHSNrqc+wsOylnG1Jiy1SHuxVNXuc52Pv\n\tcety+dn53QFAv+vGc15vujknLp+iMmfQ3Ei/LIeDznXsvz/Ly+Phq5VIe7c2I2fOTtER\n\tJ/tw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id;\n\tbh=L32x2K1oDI1fc/W8EfWl4m48JADmcomBaiadG8IxrVE=;\n\tb=KgNVZAUE2EOkB8LmVjMkvwSN3ZcJvAKxS69nMFHQzz/aRNhh9pYGVWXsIualXBtYYo\n\tyVrj2u5u9UfPI0jwfLom+tpY1ur0aPBXCOh4t/bwa5W8/Pc3mZI0JVRP2jpaEIE8FR/2\n\t6r3eoGlDq/KrX8ZxcY8dP9Gxu5eNQ41gxTfMI6MlU3QDRpVaSiZS/0cvb4/K/z3vsV2C\n\tOMbPVUeRnTIzXR0xAXCziEl8mS6nXE0i5PJP4cFp+ZtWOtJihfvZwtTcWgQLY3j06Pcg\n\t2RnhpsLtYL8IgV1Knrzq/VXVnVTCYrc+jT3/5/0/Pstb2B1IYv4W9X7GnTajGiLv6efV\n\tAPxw==","X-Gm-Message-State":"AHPjjUjqe9CK1timiXjY22uirgSCseeMGMIoCErTz6CSmHs0JyuqrWEQ\n\tnaI6T9ZOY+Q0w3Na+KN4vCi+6K30Yuo=","X-Google-Smtp-Source":"AOwi7QBSuDekKwOaRpuVyKsK06HuO1ezyMPOXyxlIKlX7JBpb7L7mJmXeiBR0Nz1U3uF76THZgoRYg==","X-Received":"by 10.84.217.129 with SMTP id p1mr9623917pli.87.1506404246056;\n\tMon, 25 Sep 2017 22:37:26 -0700 (PDT)","From":"Yuanhan Liu ","To":"dev@openvswitch.org","Date":"Tue, 26 Sep 2017 13:36:30 +0800","Message-Id":"<1506404199-23579-1-git-send-email-yliu@fridaylinux.org>","X-Mailer":"git-send-email 2.7.4","X-Spam-Status":"No, score=0.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,\n\tRCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1","X-Spam-Checker-Version":"SpamAssassin 3.3.1 (2010-03-16) on\n\tsmtp1.linux-foundation.org","Cc":"Simon Horman ","Subject":"[ovs-dev] [PATCH v3 0/9] OVS-DPDK flow offload with rte_flow","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.12","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"ovs-dev-bounces@openvswitch.org","Errors-To":"ovs-dev-bounces@openvswitch.org"},"content":"Some hightlights in v3\n======================\n\nHere is the v3, with 2 major changes and further testings (including\nmany flows). This took more effort than I thought, thus v3 publication\nhas been delayed for a while.\n\nThe first major change is the mark and id association is done with array\ninstead of CMAP now. This gives us further performance gain: it could\nbe up to 70% now (please see the exact number below).\n\nThis change also make the code a bit more complex though, due to the\nlock issue. RCU is used (not quite sure it's been used rightly though).\nFor now, RCU only protects the array base address update (due to\nreallocate), it doesn't protect the array item (array[i] = xx]) change.\nI think it's buggy and I need rethink about it.\n\nThe second major change is there is a thread introduced to do the real\nflow offload. This is for diminishing the overhead by hw flow offload\ninstallation/deletion at data path. See patch 9 for more detailed info.\n\nIn the last discussion, RSS action was suggested to use to get rid of\nthe QUEUE action workaround. Unfortunately, it didn't work. The flow\ncreation failed with MLX5 PMD driver, and that's the only driver in\nDPDK that supports RSS action so far.\n\nI also tested many flows this time. The result is more exciting: it\ncould be up to 267% boost, with 512 mega flows (with each has another\n512 exact matching flows, thus it's 512*512=256K flows in total), one\ncore and one queue doing PHY-PHY forwarding. For the offload case, the\nperformance keeps the same as with one flow only: because the cost of\nthe mark to flow translation is constant, no matter how many flows\nare inserted (as far as they are all offloaded). However, for the\nvanilla ovs-dpdk, the more flows, the worse the performance is. In\nanother word, the more flows, the bigger difference we will see.\n\nThere were too many discussions in last version. I'm sorry if I missed\nsome comments and didn't do the corresponding changes in v3. Please let\nme know if I made such mistakes.\n\nAnd below are the formal cover letter introduction, for someone who\nis the first time to see this patchset.\n\n---\nHi,\n\nHere is a joint work from Mellanox and Napatech, to enable the flow hw\noffload with the DPDK generic flow interface (rte_flow).\n\nThe basic idea is to associate the flow with a mark id (a unit32_t number).\nLater, we then get the flow directly from the mark id, bypassing the heavy\nemc processing, including miniflow_extract.\n\nThe association is done with array in patch 1. It also reuses the flow\nAPIs introduced while adding the tc offloads. The emc bypassing is done\nin patch 2. The flow offload is done in patch 4, which mainly does two\nthings:\n\n- translate the ovs match to DPDK rte flow patterns\n- bind those patterns with a MARK action.\n\nAfterwards, the NIC will set the mark id in every pkt's mbuf when it\nmatches the flow. That's basically how we could get the flow directly\nfrom the received mbuf.\n\nWhile testing with PHY-PHY forwarding with one core, one queue and one\nflow, I got about 70% performance boost. For PHY-vhost forwarding, I got\nabout 50% performance boost. It's basically the performance I got with v1,\nwhen the tcp_flags is the ignored. In summary, the CMPA to array change\ngives up yet another 16% performance boost.\n\nThe major issue mentioned in v1 is also workarounded: the queue index\nis never set to 0 blindly anymore, but set to the rxq that first\nreceives the upcall pkt.\n\nNote that it's disabled by default, which can be enabled by:\n\n $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true\n\nv3: - The mark and id association is done with array instead of CMAP.\n - Added a thread to do hw offload operations\n - Removed macros completely\n - dropped the patch to set FDIR_CONF, which is a workround some\n Intel NICs.\n - Added a debug patch to show all flow patterns we have created.\n - Misc fixes\n\nv2: - workaround the queue action issue\n - fixed the tcp_flags being skipped issue, which also fixed the\n build warnings\n - fixed l2 patterns for Intel nic\n - Converted some macros to functions\n - did not hardcode the max number of flow/action\n - rebased on top of the latest code\n\nThanks.\n\n --yliu\n\n---\nFinn Christensen (2):\n netdev-dpdk: implement flow put with rte flow\n netdev-dpdk: retry with queue action\n\nShachar Beiser (1):\n dpif-netdev: record rx queue id for the upcall\n\nYuanhan Liu (6):\n dpif-netdev: associate flow with a mark id\n dpif-netdev: retrieve flow directly from the flow mark\n netdev-dpdk: convert ufid to dpdk flow\n netdev-dpdk: remove offloaded flow on deletion\n netdev-dpdk: add debug for rte flow patterns\n dpif-netdev: do hw flow offload in another thread\n\n lib/dp-packet.h | 14 +\n lib/dpif-netdev.c | 421 ++++++++++++++++++++++++++++-\n lib/flow.c | 155 ++++++++---\n lib/flow.h | 1 +\n lib/netdev-dpdk.c | 776 +++++++++++++++++++++++++++++++++++++++++++++++++++++-\n lib/netdev.c | 1 +\n lib/netdev.h | 7 +\n 7 files changed, 1331 insertions(+), 44 deletions(-)"}