[{"id":1799027,"web_url":"http://patchwork.ozlabs.org/comment/1799027/","msgid":"<6b79dc0a-1b41-84b5-34a1-05a66e4d0694@gmail.com>","list_archive_url":null,"date":"2017-11-04T04:49:31","subject":"Re: [PATCH RFC,WIP 0/5] Flow offload infrastructure","submitter":{"id":2800,"url":"http://patchwork.ozlabs.org/api/people/2800/","name":"Florian Fainelli","email":"f.fainelli@gmail.com"},"content":"Hi Pablo,\n\nOn 11/03/2017 08:26 AM, Pablo Neira Ayuso wrote:\n> Hi,\n> \n> This patch adds the flow offload infrastructure for Netfilter. This adds\n> a new 'nf_flow_offload' module that registers a hook at ingress. Every\n> packet that hits the flow table is forwarded to where the flow table\n> entry specifies in terms of destination/gateway and netdevice. In case\n> of flow table miss, the packet follows the classic forward path.\n> \n> This flow table is populated via the new nftables VM action\n> 'flow_offload', so the user can selectively specify what flows are\n> placed into the flow table, an example ruleset would look like this:\n> \n> table inet x {\n> chain y {\n> type filter hook forward priority 0; policy accept;\n> ip protocol tcp flow offload counter\n> counter\n> }\n> }\n> \n> The 'flow offload' action adds the flow entry once the flow is in\n> established state, according to the connection tracking definition, ie.\n> we have seen traffic in both directions. Therefore, only initial packets\n> of the flow follow the classic forwarding path.\n> \n> * Patch 1/5 is nothing really interesting, just a little preparation change.\n> \n> * Patch 2/5 adds a software flow table representation. It uses the\n> rhashtable and an API to operate with it, it also introduces the\n> 'struct flow_offload' that represents a flow table entry. There's a\n> garbage collector kernel thread that cleans up entries for which we\n> have not seen any packet for a while.\n> \n> * Patch 3/5 Just adds the missing bits to integrate the software flow\n> table with conntrack. The software flow table owns the conntrack\n> object, so it is basically responsible for releasing it. Conntrack\n> entries that have been offloaded in the conntrack table will look like\n> this:\n> \n> ipv4 2 tcp 6 src=10.141.10.2 dst=147.75.205.195 sport=36392 dport=443 src=147.75.205.195 dst=192.168.2.195 sport=443 dport=36392 [OFFLOAD] use=2\n> \n> * Patch 4/5 adds the extension for nf_tables that can be used to select\n> what flows are offloaded through policy.\n> \n> * Patch 5/5 Switches and NICs come with built-in flow table, I've been\n> observing out of tree patches in OpenWRT/LEDE to integrate this into\n> Netfilter for a little while. This patch adds the ndo hooks to\n> populate hardware flow table. This patchs a workqueue to configure\n> from user context - we need to hold the mdio mutex for this. There\n> will be a little time until packets will follow the hardware path.\n> So packets will be following the software flow table path for a little\n> while until the start going through hardware.\n> \n> I'm measuring here that the software flow table forwarding path is 2.5\n> faster than the classic forwarding path in my testbed.\n> \n> TODO, still many things:\n> \n> * Only IPv4 at this time.\n> * Only IPv4 SNAT is supported.\n> * No netns support yet.\n> * Missing netlink interface to operate with the flow table, to force the\n> handover of flow to the software path.\n> * Higher configurability, instead of registering the flow table\n> inconditionally, add an interface to specify software flow table\n> properties.\n> * No flow counters at this time.\n> \n> This should serve a number of usecases where we can rely on this kernel\n> bypass. Packets that need fragmentation / PMTU / IP option handling /\n> ... and any specific handling, then we should pass them up to the\n> forwarding classic path.\n> \n> Comments welcome,\n\nA lot of us have been waiting for this for some time, so thanks a lot\nfor posting the patches. At first glance this seems to cover most of the\nHW that I know about out there and it does so without that much code\nadded which is great. Did you have a particular platform you did\nexperiment this with and if so, should we expect patches to be posted to\nsee how it integrates with real hardware?\n\nThanks!\n\n> Thanks.\n> \n> Pablo Neira Ayuso (5):\n> netfilter: nf_conntrack: move nf_ct_netns_{get,put}() to core\n> netfilter: add software flow offload infrastructure\n> netfilter: nf_flow_offload: integration with conntrack\n> netfilter: nf_tables: flow offload expression\n> netfilter: nft_flow_offload: add ndo hooks for hardware offload\n> \n> include/linux/netdevice.h | 4 +\n> include/net/flow_offload.h | 67 ++++\n> include/net/netfilter/nf_conntrack.h | 3 +-\n> include/uapi/linux/netfilter/nf_conntrack_common.h | 4 +\n> include/uapi/linux/netfilter/nf_tables.h | 9 +\n> net/netfilter/Kconfig | 14 +\n> net/netfilter/Makefile | 4 +\n> net/netfilter/nf_conntrack_core.c | 7 +-\n> net/netfilter/nf_conntrack_netlink.c | 15 +-\n> net/netfilter/nf_conntrack_proto.c | 37 +-\n> net/netfilter/nf_conntrack_proto_tcp.c | 3 +\n> net/netfilter/nf_conntrack_standalone.c | 12 +-\n> net/netfilter/nf_flow_offload.c | 421 ++++++++++++++++++++\n> net/netfilter/nft_ct.c | 39 +-\n> net/netfilter/nft_flow_offload.c | 430 +++++++++++++++++++++\n> 15 files changed, 1024 insertions(+), 45 deletions(-)\n> create mode 100644 include/net/flow_offload.h\n> create mode 100644 net/netfilter/nf_flow_offload.c\n> create mode 100644 net/netfilter/nft_flow_offload.c\n>","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"iWmfNIik\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yTRFP6ZRMz9sNc\n\tfor ;\n\tSat, 4 Nov 2017 15:49:45 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751845AbdKDEth (ORCPT );\n\tSat, 4 Nov 2017 00:49:37 -0400","from mail-ot0-f173.google.com ([74.125.82.173]:51877 \"EHLO\n\tmail-ot0-f173.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1751447AbdKDEtf (ORCPT\n\t); Sat, 4 Nov 2017 00:49:35 -0400","by mail-ot0-f173.google.com with SMTP id n74so4323159ota.8;\n\tFri, 03 Nov 2017 21:49:34 -0700 (PDT)","from ?IPv6:2001:470:d:73f:4539:f27d:e411:23c6?\n\t([2001:470:d:73f:4539:f27d:e411:23c6])\n\tby smtp.gmail.com with ESMTPSA id\n\tb46sm313198otd.52.2017.11.03.21.49.32\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tFri, 03 Nov 2017 21:49:32 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=gmail.com; s=20161025;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=hCFsw4CEUNh4aA7TeGMXoBn1LuJQYxTS+dfrOwJGw5w=;\n\tb=iWmfNIik/uta+nsjodYNTmGcYXiaLE+Zbegz52pQ45iUyNtLh5x8YfxS3XEW7L0qkP\n\tVY+nB2n0khEzKJs+ZLhXvqPLAbKsYgIq+OOVy1LnR7jgkBM8rEKJkQy0CT0FUZ2k0w3e\n\tXZRruRFriWGhiJjrv80z2//M7La4c/+0c9GINHYBYE+2sP5xjomoiJ4f0zQr3IAmktv9\n\tpcrhvsS9TTALcrFyhSewLCoZDjHZ7zlp/NmoaIdE1j7TAQ64nFiIf4usLGC/ok/wn8GD\n\tir1ueKmiv1aHds7OA2ox8l+rA2BxY8MJMGEa4PE+rvtXYshwJ+3i4AgZVGz91AKpw+D9\n\t3ziw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=hCFsw4CEUNh4aA7TeGMXoBn1LuJQYxTS+dfrOwJGw5w=;\n\tb=MIctK0RaC7z3hDlNqnrgZkQ/Q/4Ztu/3oX+fzuhKOpEkMHnZwf1XdmMC20V24+/p82\n\tNScwDTgSDfS3k1z1DelRQYHdJhOWrQ75Ip/umGjq3wTDWHCKXBJ3vgaeKVn4OjcAWsFz\n\tNTzOEtMPg8yRP9vTkVtTwtMOwe6UxOgY07kchaVre8EnLbJt9URGl+r543n1/2QxnEVP\n\tO0EhnacsWXoRyskWg+IrJgsp9XbJcOB0zbocype9KmajYTIni506ickDGpcEH5kJMPgY\n\t20+kLLr4iMEOBZZtEI/CfBvOM+MZJM2YY/ZtMeaDQNxVw9CfjvqTrc0C6DP16fRKXMTV\n\tfWdA==","X-Gm-Message-State":"AJaThX5g9Si/wRWCsoDe1jyNivcyASNp+Jy1YB0mFRFoQ/b5Ku6HR94I\n\tXOIGsAHxqUpbBK0smc0bb7t2TF1U","X-Google-Smtp-Source":"ABhQp+SwsIxkrR+HEHkqaVdPNf2tEjKIYD1fTbBoECeUsHAoNOMc1ZBZAI/kKdsemxn3ZuTfWxPJHw==","X-Received":"by 10.157.5.133 with SMTP id 5mr5332455otd.208.1509770973744;\n\tFri, 03 Nov 2017 21:49:33 -0700 (PDT)","Subject":"Re: [PATCH RFC,WIP 0/5] Flow offload infrastructure","To":"Pablo Neira Ayuso , netfilter-devel@vger.kernel.org","Cc":"netdev@vger.kernel.org","References":"<20171103152636.9967-1-pablo@netfilter.org>","From":"Florian Fainelli ","Message-ID":"<6b79dc0a-1b41-84b5-34a1-05a66e4d0694@gmail.com>","Date":"Fri, 3 Nov 2017 21:49:31 -0700","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.4.0","MIME-Version":"1.0","In-Reply-To":"<20171103152636.9967-1-pablo@netfilter.org>","Content-Type":"text/plain; charset=windows-1252","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1804031,"web_url":"http://patchwork.ozlabs.org/comment/1804031/","msgid":"<20171113165244.153915ef@cakuba>","list_archive_url":null,"date":"2017-11-14T00:52:44","subject":"Re: [PATCH RFC,WIP 0/5] Flow offload infrastructure","submitter":{"id":17220,"url":"http://patchwork.ozlabs.org/api/people/17220/","name":"Jakub Kicinski","email":"kubakici@wp.pl"},"content":"On Fri, 3 Nov 2017 16:26:31 +0100, Pablo Neira Ayuso wrote:\n> I'm measuring here that the software flow table forwarding path is 2.5\n> faster than the classic forwarding path in my testbed.\n> \n> TODO, still many things:\n> \n> * Only IPv4 at this time.\n> * Only IPv4 SNAT is supported.\n> * No netns support yet.\n> * Missing netlink interface to operate with the flow table, to force the\n> handover of flow to the software path.\n> * Higher configurability, instead of registering the flow table\n> inconditionally, add an interface to specify software flow table\n> properties.\n> * No flow counters at this time.\n> \n> This should serve a number of usecases where we can rely on this kernel\n> bypass. Packets that need fragmentation / PMTU / IP option handling /\n> ... and any specific handling, then we should pass them up to the\n> forwarding classic path.\n\nI didn't realize it from this patch set, but it was mentioned at the\nconference that this patch set is completely stateless. I.e. things\nlike TCP window tracking are not included here. IMHO that's a big\nconcern, because offloading flows is trivial when compared to state\nsync. IMHO state sync is *the* challenge in implementing connection\ntacking offload...","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (1024-bit key;\n\tunprotected) header.d=wp.pl header.i=@wp.pl header.b=\"HKDdb+Wa\";\n\tdkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3ybTWg611hz9s7c\n\tfor ;\n\tTue, 14 Nov 2017 11:53:03 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751925AbdKNAwx (ORCPT );\n\tMon, 13 Nov 2017 19:52:53 -0500","from mx3.wp.pl ([212.77.101.10]:41990 \"EHLO mx3.wp.pl\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1751247AbdKNAww (ORCPT );\n\tMon, 13 Nov 2017 19:52:52 -0500","(wp-smtpd smtp.wp.pl 27959 invoked from network);\n\t14 Nov 2017 01:52:49 +0100","from unknown (HELO cakuba) (kubakici@wp.pl@[75.53.12.129])\n\t(envelope-sender )\n\tby smtp.wp.pl (WP-SMTPD) with ECDHE-RSA-AES256-GCM-SHA384 encrypted\n\tSMTP for ; 14 Nov 2017 01:52:49 +0100"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=wp.pl; s=1024a;\n\tt=1510620769; bh=DyiJcM4GHRnsV9UsU0fq59nvjBVDqNUliqvYHAUgZWU=;\n\th=From:To:Cc:Subject;\n\tb=HKDdb+WafWxAideVIrI5ffcWBPx5hJeWsRwErC31Ka3qlUzkez5j3yAuqfvD/x9r6\n\t6ZCtKL3uQPj0xKgmeWB1W3xFYBwpLQYXNxoiGLsAwY7Renc2kEzJLCJ683ENA1U8NY\n\tWzYgZWlRNDYL20pebz8yRzUNOS8cscgrmHohKmJY=","Date":"Mon, 13 Nov 2017 16:52:44 -0800","From":"Jakub Kicinski ","To":"Pablo Neira Ayuso ","Cc":"netfilter-devel@vger.kernel.org, netdev@vger.kernel.org","Subject":"Re: [PATCH RFC,WIP 0/5] Flow offload infrastructure","Message-ID":"<20171113165244.153915ef@cakuba>","In-Reply-To":"<20171103152636.9967-1-pablo@netfilter.org>","References":"<20171103152636.9967-1-pablo@netfilter.org>","MIME-Version":"1.0","Content-Type":"text/plain; charset=US-ASCII","Content-Transfer-Encoding":"7bit","X-WP-MailID":"fb95e63868ad08e6d6fa6aa9f003d996","X-WP-AV":"skaner antywirusowy Poczty Wirtualnej Polski","X-WP-SPAM":"NO 000000A [EeNE] ","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}}]