From patchwork Mon Oct 9 17:30:14 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 823381 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="vJvgMucj"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3y9nMN2nqVz9tXl for ; Tue, 10 Oct 2017 04:30:40 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755059AbdJIRai (ORCPT ); Mon, 9 Oct 2017 13:30:38 -0400 Received: from mail-pf0-f178.google.com ([209.85.192.178]:51886 "EHLO mail-pf0-f178.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755005AbdJIRac (ORCPT ); Mon, 9 Oct 2017 13:30:32 -0400 Received: by mail-pf0-f178.google.com with SMTP id n14so12538397pfh.8 for ; Mon, 09 Oct 2017 10:30:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=YuJV7DZdqfghLdgCpHY1qZTP7PriMZbk0htiri3DQUs=; b=vJvgMucj9r7e7Eb3xYGPgVq7JP5SZdrlxa/LAJznZLs4xEFf8aYjPGIhYdKKVU5ZjB ZrMl+I4ZFnilXph2DEGnFKTYOHpejMdy6Av1pIKmDhQ6D8E49+X+TW/cO9LRuXIXzdPM n5fF+ViOubuMi268tFvao7FUllpN3+AgRG+/7chrCQJOGHidxBUEGMsL9m2V6JJyvACJ s0iYhp+ZSJ5OR71vfPquazu079Tu5mVUTYKI5aIUwP1oM1APh/k0iSs8Xbz/pNIraay6 FB6phcTbF3RqdThvqCFY+EMLboL3fY1jcMTBqkg0op1MgqqUjS6oDtKkQ05voj5lmOuC 74Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=YuJV7DZdqfghLdgCpHY1qZTP7PriMZbk0htiri3DQUs=; b=Z0JdZ2JUgEaIP0rrME+pXhBYPBh7AOqjywsSFh4MFnvVNuLM8ZjpAeCvzyMbeLGY1i ESJLtEUOdvIUgfxR7njV+H4n0UqSY0CbEWpMRAnviQr25r2kFgpMkSM2vlXxlgSR7uus etDJilcs7ZkfPnzrWufHCIMhYw8R4wnkwRqwLlW8+X4TT2NDq9OLLgaPztbk7cmkOuqN SaXdyvCGeaxs5vhnRdz+ob+IUss0U6X00TbVYt31WUWtjkUG8qz5MMCGI3hKwVz3NvdW xhpw0JJ9Xg/FoH0sH3dJoIaauiq4KbKkXtTzEv04SmNTr88NmG7yLmYnajmKiNPBro0X cdoA== X-Gm-Message-State: AMCzsaVycDYozAFjGoTGGhaQyGJw3PIESk21netW7lO2Je9/ZxyuT7BF TwVWdDcrz7YGl3nqANTT9KvGFNzu X-Google-Smtp-Source: AOwi7QAJmTvlCWxIVpDKiZgjfeSyW/J2WkFE4Q7UTdm0vdpFYxnSeM0K5Pcz1i/T9Qr49xxRv3fdmg== X-Received: by 10.99.119.206 with SMTP id s197mr9508744pgc.83.1507570232106; Mon, 09 Oct 2017 10:30:32 -0700 (PDT) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id c185sm17342537pfb.57.2017.10.09.10.30.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 09 Oct 2017 10:30:31 -0700 (PDT) From: Jakub Kicinski To: netdev@vger.kernel.org Cc: oss-drivers@netronome.com, alexei.starovoitov@gmail.com, daniel@iogearbox.net, Jakub Kicinski Subject: [PATCH net-next v2 6/7] bpf: don't rely on the verifier lock for metadata_dst allocation Date: Mon, 9 Oct 2017 10:30:14 -0700 Message-Id: <20171009173015.23520-7-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20171009173015.23520-1-jakub.kicinski@netronome.com> References: <20171009173015.23520-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org bpf_skb_set_tunnel_*() functions require allocation of per-cpu metadata_dst. The allocation happens upon verification of the first program using those helpers. In preparation for removing the verifier lock, use cmpxchg() to make sure we only allocate the metadata_dsts once. Signed-off-by: Jakub Kicinski Reviewed-by: Simon Horman Acked-by: Alexei Starovoitov Acked-by: Daniel Borkmann --- include/net/dst_metadata.h | 1 + net/core/dst.c | 16 ++++++++++++++++ net/core/filter.c | 16 +++++++++------- 3 files changed, 26 insertions(+), 7 deletions(-) diff --git a/include/net/dst_metadata.h b/include/net/dst_metadata.h index 9fba2ebf6dda..87a0bb8d449f 100644 --- a/include/net/dst_metadata.h +++ b/include/net/dst_metadata.h @@ -87,6 +87,7 @@ static inline int skb_metadata_dst_cmp(const struct sk_buff *skb_a, void metadata_dst_free(struct metadata_dst *); struct metadata_dst *metadata_dst_alloc(u8 optslen, enum metadata_type type, gfp_t flags); +void metadata_dst_free_percpu(struct metadata_dst __percpu *md_dst); struct metadata_dst __percpu * metadata_dst_alloc_percpu(u8 optslen, enum metadata_type type, gfp_t flags); diff --git a/net/core/dst.c b/net/core/dst.c index a6c47da7d0f8..8b2eafac984d 100644 --- a/net/core/dst.c +++ b/net/core/dst.c @@ -322,3 +322,19 @@ metadata_dst_alloc_percpu(u8 optslen, enum metadata_type type, gfp_t flags) return md_dst; } EXPORT_SYMBOL_GPL(metadata_dst_alloc_percpu); + +void metadata_dst_free_percpu(struct metadata_dst __percpu *md_dst) +{ + int cpu; + +#ifdef CONFIG_DST_CACHE + for_each_possible_cpu(cpu) { + struct metadata_dst *one_md_dst = per_cpu_ptr(md_dst, cpu); + + if (one_md_dst->type == METADATA_IP_TUNNEL) + dst_cache_destroy(&one_md_dst->u.tun_info.dst_cache); + } +#endif + free_percpu(md_dst); +} +EXPORT_SYMBOL_GPL(metadata_dst_free_percpu); diff --git a/net/core/filter.c b/net/core/filter.c index b7e8caa1e790..140fa9f9c0f4 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -43,6 +43,7 @@ #include #include #include +#include #include #include #include @@ -2987,14 +2988,15 @@ static const struct bpf_func_proto * bpf_get_skb_set_tunnel_proto(enum bpf_func_id which) { if (!md_dst) { - /* Race is not possible, since it's called from verifier - * that is holding verifier mutex. - */ - md_dst = metadata_dst_alloc_percpu(IP_TUNNEL_OPTS_MAX, - METADATA_IP_TUNNEL, - GFP_KERNEL); - if (!md_dst) + struct metadata_dst __percpu *tmp; + + tmp = metadata_dst_alloc_percpu(IP_TUNNEL_OPTS_MAX, + METADATA_IP_TUNNEL, + GFP_KERNEL); + if (!tmp) return NULL; + if (cmpxchg(&md_dst, NULL, tmp)) + metadata_dst_free_percpu(tmp); } switch (which) {