From patchwork Thu Sep 7 12:33:13 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesper Dangaard Brouer X-Patchwork-Id: 811000 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3xp0H40wMMz9s81 for ; Thu, 7 Sep 2017 22:33:20 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932095AbdIGMdS (ORCPT ); Thu, 7 Sep 2017 08:33:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57922 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755228AbdIGMdR (ORCPT ); Thu, 7 Sep 2017 08:33:17 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1D45BC04B31B; Thu, 7 Sep 2017 12:33:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 1D45BC04B31B Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=brouer@redhat.com Received: from firesoul.localdomain (ovpn-200-42.brq.redhat.com [10.40.200.42]) by smtp.corp.redhat.com (Postfix) with ESMTP id 08CDA17D59; Thu, 7 Sep 2017 12:33:14 +0000 (UTC) Received: from [192.168.5.1] (localhost [IPv6:::1]) by firesoul.localdomain (Postfix) with ESMTP id 2FF2A3073EC87; Thu, 7 Sep 2017 14:33:13 +0200 (CEST) Subject: [V2 PATCH net-next 1/2] xdp: implement xdp_redirect_map for generic XDP From: Jesper Dangaard Brouer To: netdev@vger.kernel.org, "David S. Miller" Cc: Daniel Borkmann , John Fastabend , Andy Gospodarek , Jesper Dangaard Brouer Date: Thu, 07 Sep 2017 14:33:13 +0200 Message-ID: <150478759310.28665.17184783248584070473.stgit@firesoul> In-Reply-To: <150478756604.28665.6915020425359475729.stgit@firesoul> References: <150478756604.28665.6915020425359475729.stgit@firesoul> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Thu, 07 Sep 2017 12:33:17 +0000 (UTC) Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Using bpf_redirect_map is allowed for generic XDP programs, but the appropriate map lookup was never performed in xdp_do_generic_redirect(). Instead the map-index is directly used as the ifindex. For the xdp_redirect_map sample in SKB-mode '-S', this resulted in trying sending on ifindex 0 which isn't valid, resulting in getting SKB packets dropped. Thus, the reported performance numbers are wrong in commit 24251c264798 ("samples/bpf: add option for native and skb mode for redirect apps") for the 'xdp_redirect_map -S' case. It might seem innocent this was lacking, but it can actually crash the kernel. The potential crash is caused by not consuming redirect_info->map. The bpf_redirect_map helper will set this_cpu_ptr(&redirect_info)->map pointer, which will survive even after unloading the xdp bpf_prog and deallocating the devmap data-structure. This leaves a dead map pointer around. The kernel will crash when loading the xdp_redirect sample (in native XDP mode) as it doesn't reset map (via bpf_redirect) and returns XDP_REDIRECT, which will cause it to dereference the map pointer. Fixes: 6103aa96ec07 ("net: implement XDP_REDIRECT for xdp generic") Fixes: 24251c264798 ("samples/bpf: add option for native and skb mode for redirect apps") Signed-off-by: Jesper Dangaard Brouer Acked-by: Daniel Borkmann --- include/trace/events/xdp.h | 4 ++-- net/core/filter.c | 14 +++++++++++--- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/include/trace/events/xdp.h b/include/trace/events/xdp.h index 862575ac8da9..4e16c43fba10 100644 --- a/include/trace/events/xdp.h +++ b/include/trace/events/xdp.h @@ -138,11 +138,11 @@ DEFINE_EVENT_PRINT(xdp_redirect_template, xdp_redirect_map_err, #define _trace_xdp_redirect_map(dev, xdp, fwd, map, idx) \ trace_xdp_redirect_map(dev, xdp, fwd ? fwd->ifindex : 0, \ - 0, map, idx); + 0, map, idx) #define _trace_xdp_redirect_map_err(dev, xdp, fwd, map, idx, err) \ trace_xdp_redirect_map_err(dev, xdp, fwd ? fwd->ifindex : 0, \ - err, map, idx); + err, map, idx) #endif /* _TRACE_XDP_H */ diff --git a/net/core/filter.c b/net/core/filter.c index 5912c738a7b2..3767470cab6c 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -2566,13 +2566,19 @@ int xdp_do_generic_redirect(struct net_device *dev, struct sk_buff *skb, struct bpf_prog *xdp_prog) { struct redirect_info *ri = this_cpu_ptr(&redirect_info); + struct bpf_map *map = ri->map; u32 index = ri->ifindex; struct net_device *fwd; unsigned int len; int err = 0; - fwd = dev_get_by_index_rcu(dev_net(dev), index); ri->ifindex = 0; + ri->map = NULL; + + if (map) + fwd = __dev_map_lookup_elem(map, index); + else + fwd = dev_get_by_index_rcu(dev_net(dev), index); if (unlikely(!fwd)) { err = -EINVAL; goto err; @@ -2590,10 +2596,12 @@ int xdp_do_generic_redirect(struct net_device *dev, struct sk_buff *skb, } skb->dev = fwd; - _trace_xdp_redirect(dev, xdp_prog, index); + map ? _trace_xdp_redirect_map(dev, xdp_prog, fwd, map, index) + : _trace_xdp_redirect(dev, xdp_prog, index); return 0; err: - _trace_xdp_redirect_err(dev, xdp_prog, index, err); + map ? _trace_xdp_redirect_map_err(dev, xdp_prog, fwd, map, index, err) + : _trace_xdp_redirect_err(dev, xdp_prog, index, err); return err; } EXPORT_SYMBOL_GPL(xdp_do_generic_redirect); From patchwork Thu Sep 7 12:33:18 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesper Dangaard Brouer X-Patchwork-Id: 811001 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3xp0H95ZYYz9s81 for ; Thu, 7 Sep 2017 22:33:25 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932122AbdIGMdX (ORCPT ); Thu, 7 Sep 2017 08:33:23 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55862 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932105AbdIGMdV (ORCPT ); Thu, 7 Sep 2017 08:33:21 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8A2EBC0587D6; Thu, 7 Sep 2017 12:33:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 8A2EBC0587D6 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=brouer@redhat.com Received: from firesoul.localdomain (ovpn-200-42.brq.redhat.com [10.40.200.42]) by smtp.corp.redhat.com (Postfix) with ESMTP id 24F7818B11; Thu, 7 Sep 2017 12:33:19 +0000 (UTC) Received: from [192.168.5.1] (localhost [IPv6:::1]) by firesoul.localdomain (Postfix) with ESMTP id 49ECE3073EC87; Thu, 7 Sep 2017 14:33:18 +0200 (CEST) Subject: [V2 PATCH net-next 2/2] xdp: catch invalid XDP_REDIRECT API usage From: Jesper Dangaard Brouer To: netdev@vger.kernel.org, "David S. Miller" Cc: Daniel Borkmann , John Fastabend , Andy Gospodarek , Jesper Dangaard Brouer Date: Thu, 07 Sep 2017 14:33:18 +0200 Message-ID: <150478759820.28665.14031878598812204399.stgit@firesoul> In-Reply-To: <150478756604.28665.6915020425359475729.stgit@firesoul> References: <150478756604.28665.6915020425359475729.stgit@firesoul> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Thu, 07 Sep 2017 12:33:21 +0000 (UTC) Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Catch different invalid XDP_REDIRECT and bpf_redirect_map API usage. It is fairly easy to create a dangling redirect_info->map pointer, which (until John or Daniel fix this) can crash the kernel. The intended usage of the BPF helper bpf_redirect_map(), is to return XDP_REDIRECT action after invoking it, but there is nothing stopping the bpf_prog to return anything else. When XDP_REDIRECT isn't returned, then a dangling ->map pointer is left behind, as xdp_do_redirect() isn't called. This also happens for drivers not implementing XDP_REDIRECT, as they are not aware of this new XDP_REDIRECT return code, they leave the map pointer dangling. The simply solution to check for a dangling ->map pointer after each driver napi->poll() invocation, see xdp_do_map_check(). This patch also add a check for a dangling ->map_to_flush pointer. This should be considered a driver bug, as the driver contract is that a pair of xdp_do_redirect and xdp_do_flush_map MUST be called in the same cpu context. Note, we need to check after each drivers napi->poll call, as: 1. DevA poll call bpf_redirect_map() but not xdp_do_redirect() 2. DevB bpf_prog uses bpf_redirect() and call xdp_do_redirect() which now use map from DevA Signed-off-by: Jesper Dangaard Brouer Reported-by: Jesper Dangaard Brouer Signed-off-by: Daniel Borkmann Signed-off-by: John Fastabend --- include/linux/filter.h | 1 + net/core/dev.c | 3 +++ net/core/filter.c | 25 +++++++++++++++++++++++++ 3 files changed, 29 insertions(+) diff --git a/include/linux/filter.h b/include/linux/filter.h index d29e58fde364..0c48941e0022 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -724,6 +724,7 @@ int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp, struct bpf_prog *prog); void xdp_do_flush_map(void); +void xdp_do_map_check(struct napi_struct *napi); void bpf_warn_invalid_xdp_action(u32 act); void bpf_warn_invalid_xdp_redirect(u32 ifindex); diff --git a/net/core/dev.c b/net/core/dev.c index 6f845e4fec17..7eac642b469f 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -5320,6 +5320,7 @@ static void busy_poll_stop(struct napi_struct *napi, void *have_poll_lock) */ rc = napi->poll(napi, BUSY_POLL_BUDGET); trace_napi_poll(napi, rc, BUSY_POLL_BUDGET); + xdp_do_map_check(napi); netpoll_poll_unlock(have_poll_lock); if (rc == BUSY_POLL_BUDGET) __napi_schedule(napi); @@ -5367,6 +5368,7 @@ void napi_busy_loop(unsigned int napi_id, } work = napi_poll(napi, BUSY_POLL_BUDGET); trace_napi_poll(napi, work, BUSY_POLL_BUDGET); + xdp_do_map_check(napi); count: if (work > 0) __NET_ADD_STATS(dev_net(napi->dev), @@ -5529,6 +5531,7 @@ static int napi_poll(struct napi_struct *n, struct list_head *repoll) if (test_bit(NAPI_STATE_SCHED, &n->state)) { work = n->poll(n, weight); trace_napi_poll(n, work, weight); + xdp_do_map_check(n); } WARN_ON_ONCE(work > weight); diff --git a/net/core/filter.c b/net/core/filter.c index 3767470cab6c..f0e1135eeb9d 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -2500,6 +2500,31 @@ void xdp_do_flush_map(void) } EXPORT_SYMBOL_GPL(xdp_do_flush_map); +void xdp_do_map_check(struct napi_struct *napi) +{ + struct redirect_info *ri = this_cpu_ptr(&redirect_info); + + /* XDP drivers (and XDP-generic) must invoke xdp_do_redirect() + * when bpf_prog use helper bpf_redirect_map(), else the map + * pointer can be left dangling. Catch this invalid API + * usage, instead of potentially crashing. + */ + if (ri->map) { + ri->map = NULL; + net_err_ratelimited("%s: caught invalid XDP bpf_redirect_map\n", + napi->dev->name); + trace_xdp_exception(napi->dev, NULL, XDP_REDIRECT); + } + if (ri->map_to_flush) { /* Driver bug */ + net_err_ratelimited("%s: XDP driver miss xdp_do_flush_map\n", + napi->dev->name); + trace_xdp_exception(napi->dev, NULL, XDP_REDIRECT); + /* Flush map, else pkts can be stuck on XDP TXq */ + xdp_do_flush_map(); + } +} +EXPORT_SYMBOL_GPL(xdp_do_map_check); + static int xdp_do_redirect_map(struct net_device *dev, struct xdp_buff *xdp, struct bpf_prog *xdp_prog) {