From patchwork Wed Aug 14 17:37:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanislav Fomichev X-Patchwork-Id: 1147147 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.b="gWD4LQo0"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 467xck61lGz9s7T for ; Thu, 15 Aug 2019 03:37:58 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728546AbfHNRh6 (ORCPT ); Wed, 14 Aug 2019 13:37:58 -0400 Received: from mail-qk1-f201.google.com ([209.85.222.201]:38685 "EHLO mail-qk1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728219AbfHNRh6 (ORCPT ); Wed, 14 Aug 2019 13:37:58 -0400 Received: by mail-qk1-f201.google.com with SMTP id n190so100295717qkd.5 for ; Wed, 14 Aug 2019 10:37:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=AJSXkIdAVH1oaTxBLhQrKqbmftfzMmTC+dK8Oqetgno=; b=gWD4LQo02Rl1oO95HM3VIMxCmviZpwWy2/z7MX+XEJbBpGlneWo4vAmHB/d5S2Eoxn ueEusHgNpYPcSc6BvpuXDL36hDpPZxA4Op2hDVt0RjKuLbPTuvo5XJnHDJvIqGuPjF7J cTmP6L+Kmr7OthlmlMfcqZogyduQvvMStEUahLxl70sCRwJH3fxrjJCxmqSXYeewRCHB eSlYT3I8wB/Am3RWsI3aNCo995kfjlQyCe/VO/kYJqFFQeYI4ICQDS5u++8CJPTxHc3v mwnk0BCYqBYsx0gRTkJUPPFKZglRcyuDZGolgxq674x9g6qhEF9frlm8ztIg2GC2dtfT rpLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=AJSXkIdAVH1oaTxBLhQrKqbmftfzMmTC+dK8Oqetgno=; b=eVa0CbwWO/8efPnL8FV/lCf15RNg2jHC5+Ce1uL8z7qeRN7UQqq8nNI7smeVKHWaTx e86wPN7Ibg1bwtQGoMQoAAlss0tM5wlmYp51egVaI3Dt/kMIUpJEzESIIIuhjY3I2yRm ggh7dY3t/u5mXDVTTLV5qPFXhEXQ86psf6cMsxGewxT4dR5G2sp+f8NryuvewXLF8VXk dV23o4F5fglHXfnUWeej7FaNPf5AWNHnGEpmowf1qOe1BlvwwDUtX+nS5S/pNFimWj5A NJs2Yov2Uy6f5xWU4CYre1LlbInGX5QCUsK+lc0UzSDtkODqY2L3V0RLwOWEgUpqaIVc KXUg== X-Gm-Message-State: APjAAAV1o6IJkb01kVp/pO/W/MP5dsJ+2fBGwevDJP/fL6SIpjQPb+DU cJQ4qzsEY0uy5WSOJCcsDQPQmrc= X-Google-Smtp-Source: APXvYqzI633zXBTyAMNZ8L6CuGXCE7Cfa6sHI4mPchxTrzKnXdkKMvhTHcr7OTkgZadJR7J65DkHHyw= X-Received: by 2002:ae9:f804:: with SMTP id x4mr584188qkh.178.1565804277231; Wed, 14 Aug 2019 10:37:57 -0700 (PDT) Date: Wed, 14 Aug 2019 10:37:48 -0700 In-Reply-To: <20190814173751.31806-1-sdf@google.com> Message-Id: <20190814173751.31806-2-sdf@google.com> Mime-Version: 1.0 References: <20190814173751.31806-1-sdf@google.com> X-Mailer: git-send-email 2.23.0.rc1.153.gdeed80330f-goog Subject: [PATCH bpf-next v4 1/4] bpf: export bpf_map_inc_not_zero From: Stanislav Fomichev To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: davem@davemloft.net, ast@kernel.org, daniel@iogearbox.net, Stanislav Fomichev , Martin KaFai Lau , Yonghong Song Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Rename existing bpf_map_inc_not_zero to __bpf_map_inc_not_zero to indicate that it's caller's responsibility to do proper locking. Create and export bpf_map_inc_not_zero wrapper that properly locks map_idr_lock. Will be used in the next commit to hold a map while cloning a socket. Cc: Martin KaFai Lau Cc: Yonghong Song Acked-by: Martin KaFai Lau Acked-by: Yonghong Song Signed-off-by: Stanislav Fomichev --- include/linux/bpf.h | 2 ++ kernel/bpf/syscall.c | 16 +++++++++++++--- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index f9a506147c8a..15ae49862b82 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -647,6 +647,8 @@ void bpf_map_free_id(struct bpf_map *map, bool do_idr_lock); struct bpf_map *bpf_map_get_with_uref(u32 ufd); struct bpf_map *__bpf_map_get(struct fd f); struct bpf_map * __must_check bpf_map_inc(struct bpf_map *map, bool uref); +struct bpf_map * __must_check bpf_map_inc_not_zero(struct bpf_map *map, + bool uref); void bpf_map_put_with_uref(struct bpf_map *map); void bpf_map_put(struct bpf_map *map); int bpf_map_charge_memlock(struct bpf_map *map, u32 pages); diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 5d141f16f6fa..cf8052b016e7 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -683,8 +683,8 @@ struct bpf_map *bpf_map_get_with_uref(u32 ufd) } /* map_idr_lock should have been held */ -static struct bpf_map *bpf_map_inc_not_zero(struct bpf_map *map, - bool uref) +static struct bpf_map *__bpf_map_inc_not_zero(struct bpf_map *map, + bool uref) { int refold; @@ -704,6 +704,16 @@ static struct bpf_map *bpf_map_inc_not_zero(struct bpf_map *map, return map; } +struct bpf_map *bpf_map_inc_not_zero(struct bpf_map *map, bool uref) +{ + spin_lock_bh(&map_idr_lock); + map = __bpf_map_inc_not_zero(map, uref); + spin_unlock_bh(&map_idr_lock); + + return map; +} +EXPORT_SYMBOL_GPL(bpf_map_inc_not_zero); + int __weak bpf_stackmap_copy(struct bpf_map *map, void *key, void *value) { return -ENOTSUPP; @@ -2177,7 +2187,7 @@ static int bpf_map_get_fd_by_id(const union bpf_attr *attr) spin_lock_bh(&map_idr_lock); map = idr_find(&map_idr, id); if (map) - map = bpf_map_inc_not_zero(map, true); + map = __bpf_map_inc_not_zero(map, true); else map = ERR_PTR(-ENOENT); spin_unlock_bh(&map_idr_lock); From patchwork Wed Aug 14 17:37:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanislav Fomichev X-Patchwork-Id: 1147149 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.b="odigD1yi"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 467xcn5F05z9s7T for ; Thu, 15 Aug 2019 03:38:01 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728533AbfHNRiB (ORCPT ); Wed, 14 Aug 2019 13:38:01 -0400 Received: from mail-yb1-f201.google.com ([209.85.219.201]:41118 "EHLO mail-yb1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728511AbfHNRiB (ORCPT ); Wed, 14 Aug 2019 13:38:01 -0400 Received: by mail-yb1-f201.google.com with SMTP id 137so7918458ybd.8 for ; Wed, 14 Aug 2019 10:38:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=a7H16UQmJw6VgKiMqqu0R0QEkK8DCgpC0fmnX/y6MpI=; b=odigD1yiSpXcvenlUb3RoKMjOjsUPHNirq43Q9GruOcDZeVGkqQxi88qiRC48KbB/o FZRLsa6mZuygL1e7LhWPeyq+ykzbiUV9k2xlKMRIHoAukPx34lBYHWugZYpfD6UnSwAo Nb/QGJY0e93Ij3RNZKEknIQWQJJ1Fee8vkJWWp9Hoi0FyJq2qcSPzLBIuw5ne3FW/jqT susA0L+wnxF6V0AaFdUSRoP4Abb8kppjeFYTf9010Efs+MsBf2P/reaHsguI/oOVs/x4 akz28JIrM/jmgpzZvT8mQrsNhZNROFKLYuaDhfDiITKPuLqf3z/edbjKb41eJhPliZI9 eglQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=a7H16UQmJw6VgKiMqqu0R0QEkK8DCgpC0fmnX/y6MpI=; b=lvZHEC6zUvxleN++C2kUQVRd7yZMYmrouBt2vx0MHM9iPsqd4JR17YmLtoYWIysfev UZ9tWsZ9r2880lElRE8izA6omjUbbaKpMNrJDjDn4UzodpciYcZ3bHfXK1KXiZgL51at rqyigDaQic8VDUlUYyVZjvRwH2MBvsvCQEMN1nqB0hlUw8BrlEeDbuNsW3R9J3PWkl+W WvshzpGHA2XC9Ui51/Ztjl0HeO0rkMZ2WwQZaiIHT+i4Q/cQc0mXv2PoJi1RW3P/JjYz iM0HY7bl0C4lwGLhNKnc0c3uo24RUFaCLLdDqbCXOyW/Kl/zPHTcmNje3uT25Df49rd6 8F/w== X-Gm-Message-State: APjAAAXfWVzDRuY+MH3uq4xs6lPyh4nUp6TdyELCc3m545uzNRdBV6Vv jAoyScQPaa7BoEIIbG00rXhVXbo= X-Google-Smtp-Source: APXvYqxxkcCMAbUMycc6zxfAX9dGuoLco/71gWMwv1KmFCtZf3F+SQMcptqLLomHQOBpUl7VtY5pLPw= X-Received: by 2002:a25:9202:: with SMTP id b2mr27850ybo.65.1565804280099; Wed, 14 Aug 2019 10:38:00 -0700 (PDT) Date: Wed, 14 Aug 2019 10:37:49 -0700 In-Reply-To: <20190814173751.31806-1-sdf@google.com> Message-Id: <20190814173751.31806-3-sdf@google.com> Mime-Version: 1.0 References: <20190814173751.31806-1-sdf@google.com> X-Mailer: git-send-email 2.23.0.rc1.153.gdeed80330f-goog Subject: [PATCH bpf-next v4 2/4] bpf: support cloning sk storage on accept() From: Stanislav Fomichev To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: davem@davemloft.net, ast@kernel.org, daniel@iogearbox.net, Stanislav Fomichev , Martin KaFai Lau , Yonghong Song Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Add new helper bpf_sk_storage_clone which optionally clones sk storage and call it from sk_clone_lock. Cc: Martin KaFai Lau Cc: Yonghong Song Acked-by: Martin KaFai Lau Acked-by: Yonghong Song Signed-off-by: Stanislav Fomichev --- include/net/bpf_sk_storage.h | 10 ++++ include/uapi/linux/bpf.h | 3 + net/core/bpf_sk_storage.c | 104 ++++++++++++++++++++++++++++++++++- net/core/sock.c | 9 ++- 4 files changed, 120 insertions(+), 6 deletions(-) diff --git a/include/net/bpf_sk_storage.h b/include/net/bpf_sk_storage.h index b9dcb02e756b..8e4f831d2e52 100644 --- a/include/net/bpf_sk_storage.h +++ b/include/net/bpf_sk_storage.h @@ -10,4 +10,14 @@ void bpf_sk_storage_free(struct sock *sk); extern const struct bpf_func_proto bpf_sk_storage_get_proto; extern const struct bpf_func_proto bpf_sk_storage_delete_proto; +#ifdef CONFIG_BPF_SYSCALL +int bpf_sk_storage_clone(const struct sock *sk, struct sock *newsk); +#else +static inline int bpf_sk_storage_clone(const struct sock *sk, + struct sock *newsk) +{ + return 0; +} +#endif + #endif /* _BPF_SK_STORAGE_H */ diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 4393bd4b2419..0ef594ac3899 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -337,6 +337,9 @@ enum bpf_attach_type { #define BPF_F_RDONLY_PROG (1U << 7) #define BPF_F_WRONLY_PROG (1U << 8) +/* Clone map from listener for newly accepted socket */ +#define BPF_F_CLONE (1U << 9) + /* flags for BPF_PROG_QUERY */ #define BPF_F_QUERY_EFFECTIVE (1U << 0) diff --git a/net/core/bpf_sk_storage.c b/net/core/bpf_sk_storage.c index 94c7f77ecb6b..da5639a5bd3b 100644 --- a/net/core/bpf_sk_storage.c +++ b/net/core/bpf_sk_storage.c @@ -12,6 +12,9 @@ static atomic_t cache_idx; +#define SK_STORAGE_CREATE_FLAG_MASK \ + (BPF_F_NO_PREALLOC | BPF_F_CLONE) + struct bucket { struct hlist_head list; raw_spinlock_t lock; @@ -209,7 +212,6 @@ static void selem_unlink_sk(struct bpf_sk_storage_elem *selem) kfree_rcu(sk_storage, rcu); } -/* sk_storage->lock must be held and sk_storage->list cannot be empty */ static void __selem_link_sk(struct bpf_sk_storage *sk_storage, struct bpf_sk_storage_elem *selem) { @@ -509,7 +511,7 @@ static int sk_storage_delete(struct sock *sk, struct bpf_map *map) return 0; } -/* Called by __sk_destruct() */ +/* Called by __sk_destruct() & bpf_sk_storage_clone() */ void bpf_sk_storage_free(struct sock *sk) { struct bpf_sk_storage_elem *selem; @@ -557,6 +559,11 @@ static void bpf_sk_storage_map_free(struct bpf_map *map) smap = (struct bpf_sk_storage_map *)map; + /* Note that this map might be concurrently cloned from + * bpf_sk_storage_clone. Wait for any existing bpf_sk_storage_clone + * RCU read section to finish before proceeding. New RCU + * read sections should be prevented via bpf_map_inc_not_zero. + */ synchronize_rcu(); /* bpf prog and the userspace can no longer access this map @@ -601,7 +608,9 @@ static void bpf_sk_storage_map_free(struct bpf_map *map) static int bpf_sk_storage_map_alloc_check(union bpf_attr *attr) { - if (attr->map_flags != BPF_F_NO_PREALLOC || attr->max_entries || + if (attr->map_flags & ~SK_STORAGE_CREATE_FLAG_MASK || + !(attr->map_flags & BPF_F_NO_PREALLOC) || + attr->max_entries || attr->key_size != sizeof(int) || !attr->value_size || /* Enforce BTF for userspace sk dumping */ !attr->btf_key_type_id || !attr->btf_value_type_id) @@ -739,6 +748,95 @@ static int bpf_fd_sk_storage_delete_elem(struct bpf_map *map, void *key) return err; } +static struct bpf_sk_storage_elem * +bpf_sk_storage_clone_elem(struct sock *newsk, + struct bpf_sk_storage_map *smap, + struct bpf_sk_storage_elem *selem) +{ + struct bpf_sk_storage_elem *copy_selem; + + copy_selem = selem_alloc(smap, newsk, NULL, true); + if (!copy_selem) + return NULL; + + if (map_value_has_spin_lock(&smap->map)) + copy_map_value_locked(&smap->map, SDATA(copy_selem)->data, + SDATA(selem)->data, true); + else + copy_map_value(&smap->map, SDATA(copy_selem)->data, + SDATA(selem)->data); + + return copy_selem; +} + +int bpf_sk_storage_clone(const struct sock *sk, struct sock *newsk) +{ + struct bpf_sk_storage *new_sk_storage = NULL; + struct bpf_sk_storage *sk_storage; + struct bpf_sk_storage_elem *selem; + int ret = 0; + + RCU_INIT_POINTER(newsk->sk_bpf_storage, NULL); + + rcu_read_lock(); + sk_storage = rcu_dereference(sk->sk_bpf_storage); + + if (!sk_storage || hlist_empty(&sk_storage->list)) + goto out; + + hlist_for_each_entry_rcu(selem, &sk_storage->list, snode) { + struct bpf_sk_storage_elem *copy_selem; + struct bpf_sk_storage_map *smap; + struct bpf_map *map; + + smap = rcu_dereference(SDATA(selem)->smap); + if (!(smap->map.map_flags & BPF_F_CLONE)) + continue; + + /* Note that for lockless listeners adding new element + * here can race with cleanup in bpf_sk_storage_map_free. + * Try to grab map refcnt to make sure that it's still + * alive and prevent concurrent removal. + */ + map = bpf_map_inc_not_zero(&smap->map, false); + if (IS_ERR(map)) + continue; + + copy_selem = bpf_sk_storage_clone_elem(newsk, smap, selem); + if (!copy_selem) { + ret = -ENOMEM; + bpf_map_put(map); + goto out; + } + + if (new_sk_storage) { + selem_link_map(smap, copy_selem); + __selem_link_sk(new_sk_storage, copy_selem); + } else { + ret = sk_storage_alloc(newsk, smap, copy_selem); + if (ret) { + kfree(copy_selem); + atomic_sub(smap->elem_size, + &newsk->sk_omem_alloc); + bpf_map_put(map); + goto out; + } + + new_sk_storage = rcu_dereference(copy_selem->sk_storage); + } + bpf_map_put(map); + } + +out: + rcu_read_unlock(); + + /* In case of an error, don't free anything explicitly here, the + * caller is responsible to call bpf_sk_storage_free. + */ + + return ret; +} + BPF_CALL_4(bpf_sk_storage_get, struct bpf_map *, map, struct sock *, sk, void *, value, u64, flags) { diff --git a/net/core/sock.c b/net/core/sock.c index d57b0cc995a0..f5e801a9cea4 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -1851,9 +1851,12 @@ struct sock *sk_clone_lock(const struct sock *sk, const gfp_t priority) goto out; } RCU_INIT_POINTER(newsk->sk_reuseport_cb, NULL); -#ifdef CONFIG_BPF_SYSCALL - RCU_INIT_POINTER(newsk->sk_bpf_storage, NULL); -#endif + + if (bpf_sk_storage_clone(sk, newsk)) { + sk_free_unlock_clone(newsk); + newsk = NULL; + goto out; + } newsk->sk_err = 0; newsk->sk_err_soft = 0; From patchwork Wed Aug 14 17:37:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanislav Fomichev X-Patchwork-Id: 1147151 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.b="Q1bioI/n"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 467xcr269jz9s7T for ; Thu, 15 Aug 2019 03:38:04 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728657AbfHNRiE (ORCPT ); Wed, 14 Aug 2019 13:38:04 -0400 Received: from mail-qt1-f202.google.com ([209.85.160.202]:38055 "EHLO mail-qt1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728520AbfHNRiD (ORCPT ); Wed, 14 Aug 2019 13:38:03 -0400 Received: by mail-qt1-f202.google.com with SMTP id i13so8225271qtq.5 for ; Wed, 14 Aug 2019 10:38:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=80gVs8IzWGoGeL5TQc1siGuhLRsZ2VKQxE10IPj1Irs=; b=Q1bioI/n+qQcRZew1C8g0VJqGvV5n9pgtuUy6trAUYjnHmvSrOtv1BC0gRwtv3cIYN qXy3mNfP95c6J2AAmospsrxq6/YW+sZwiWl9rUqgdoxAITU7hknQQVPj4YFyLfSDDB8U x3kQhiAa8FnI33t9USybZgIsKakb7nP3SblL54Aieh53ARev4PKL1STrgK89hAbrxumo g6HCpN67+bDmvrgrEYokOsbcXIMvg5zMYUxscI6MCSfnJdUJPJXDUbRg7ovVpCZgRhZO PrpEl0heiaThkGfdvC8vekhSgu3wHH0lZxnaw7WybIVUervtdn0g3UVfxxyekEyydTgp l8fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=80gVs8IzWGoGeL5TQc1siGuhLRsZ2VKQxE10IPj1Irs=; b=sWhzhSuIDDpiCETndVtjrrWStuCbtYASOzXf3H7Kvee+muw5Nb+YuUTIer2C3nEFAm wGmwNWhJB+RJ6grEGuouqTa7qSUr03XhygfTvxfNmSjP1PsTlxNODg6uQ5ZFDvV8Rd3y Udtjswo+RZCDm+cwlBmKRpmMCe+ARCMXWyItj2xfCm4v/lF4+j1QIM55vuzVxgqvY7I4 +tBNYgPAtn78SRemm/l05DscKW/M1OKhLBOai+oNgxbQ2T2fdazA/EDZ2NGRaVN9A3bI 9phxTVpjv1wZhLvGQedsAsYceg+ZDoNb5WEhgQUuamayai7ndpPjWU94BtvsVqOt8SAu SQAg== X-Gm-Message-State: APjAAAXdl4n8eMbP5M7QwJshw+vYfjwZv0fPlHJK9V899dSIMBcxEjfF tgO3Iccate7wNSiovL1JK/stvFg= X-Google-Smtp-Source: APXvYqzM7C2qcyJEskTtqWiuXOTMK3bdPyK7ataW8vGHhJkDbLNYXvf5f+x6KUGryz2W6YLKW13zzo4= X-Received: by 2002:a37:ac19:: with SMTP id e25mr551488qkm.155.1565804282786; Wed, 14 Aug 2019 10:38:02 -0700 (PDT) Date: Wed, 14 Aug 2019 10:37:50 -0700 In-Reply-To: <20190814173751.31806-1-sdf@google.com> Message-Id: <20190814173751.31806-4-sdf@google.com> Mime-Version: 1.0 References: <20190814173751.31806-1-sdf@google.com> X-Mailer: git-send-email 2.23.0.rc1.153.gdeed80330f-goog Subject: [PATCH bpf-next v4 3/4] bpf: sync bpf.h to tools/ From: Stanislav Fomichev To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: davem@davemloft.net, ast@kernel.org, daniel@iogearbox.net, Stanislav Fomichev , Martin KaFai Lau , Yonghong Song Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Sync new sk storage clone flag. Cc: Martin KaFai Lau Cc: Yonghong Song Acked-by: Martin KaFai Lau Acked-by: Yonghong Song Signed-off-by: Stanislav Fomichev --- tools/include/uapi/linux/bpf.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 4393bd4b2419..0ef594ac3899 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -337,6 +337,9 @@ enum bpf_attach_type { #define BPF_F_RDONLY_PROG (1U << 7) #define BPF_F_WRONLY_PROG (1U << 8) +/* Clone map from listener for newly accepted socket */ +#define BPF_F_CLONE (1U << 9) + /* flags for BPF_PROG_QUERY */ #define BPF_F_QUERY_EFFECTIVE (1U << 0) From patchwork Wed Aug 14 17:37:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanislav Fomichev X-Patchwork-Id: 1147153 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.b="sFTePvWQ"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 467xcv3hzZz9sBF for ; Thu, 15 Aug 2019 03:38:07 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728687AbfHNRiH (ORCPT ); Wed, 14 Aug 2019 13:38:07 -0400 Received: from mail-pl1-f201.google.com ([209.85.214.201]:48057 "EHLO mail-pl1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728510AbfHNRiG (ORCPT ); Wed, 14 Aug 2019 13:38:06 -0400 Received: by mail-pl1-f201.google.com with SMTP id j12so64979479pll.14 for ; Wed, 14 Aug 2019 10:38:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=BQVa+qbBH4bh5+nlrxKSZKO3bsSmEmE4aUyMPpPTRxg=; b=sFTePvWQb0J804ila02EjgwdHduQLYeyCkeLwx7aIGst+//hTL0Ou9V+kkBrm1vVGN dd1+D+DY5LfT45XBLM+wyE8FJQ1x/+TBld+4usE6fLutRMrgmLvdcP/zB0M7QfgeyYq+ y/cO6r1H8coqN1YRiMwIHPl+sVLMDYnqms2KmM0T32YqsanVPZz4W0SCy0H1+rqsh3nz cqNsDDOcV+bRbSeamBWNtPiv+mJLLK+T8WK25wMKjzUyqwBEYpyoUH81jmTTnyAcBODD w0Xv1T865ltL+MeI4Zy9C5MeokOxmix0xHlIbqU90mm6mDwj2mFxRDkn7s8ZJ5ganUXC lcWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=BQVa+qbBH4bh5+nlrxKSZKO3bsSmEmE4aUyMPpPTRxg=; b=HH41i+c8v4Jd1Tf146gThpuKLHfwencRamOxnTXbndIAsQn9dRhbLI8to2kwGFjw9L RZf2FaVobu+vTolPyyi3KZ/32Z4VbdaoBjr5YgKCDId3iPQlsgfnnsJ6lriJq6meu2uv c857zbGLobxh80dvKuKDAoRUYPCUHRm8gZ0g/mlaqEaFejRXz4nNZgWnSmfP8twnffdR Eb8+tmbg7FmtG2QqfKTo6xMarDNBmo2RfGNvnTD/ylvkDiVrJ+7j/5CxmEnruEQAbPtm NY7onxVFN7bBi04n2NoF1BdcrMAHAytYUAG2QBOo7bOayt6ZGAkoX8w3OavNkDZW2QkZ WMLw== X-Gm-Message-State: APjAAAULJWNs1d5xcQ2jv43fx48hapgB/0TxNvjbayGwNV039sDAfpCL SYszInhxrYT14wpZbypIheGfSmA= X-Google-Smtp-Source: APXvYqxNtM/rj2Olw+ynAAiavzbzhwjKgrPntJSb9fqWXTYsl6zIz/ymSe+fb1o3ybb13C+lLMemMAI= X-Received: by 2002:a63:124a:: with SMTP id 10mr297753pgs.254.1565804285896; Wed, 14 Aug 2019 10:38:05 -0700 (PDT) Date: Wed, 14 Aug 2019 10:37:51 -0700 In-Reply-To: <20190814173751.31806-1-sdf@google.com> Message-Id: <20190814173751.31806-5-sdf@google.com> Mime-Version: 1.0 References: <20190814173751.31806-1-sdf@google.com> X-Mailer: git-send-email 2.23.0.rc1.153.gdeed80330f-goog Subject: [PATCH bpf-next v4 4/4] selftests/bpf: add sockopt clone/inheritance test From: Stanislav Fomichev To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: davem@davemloft.net, ast@kernel.org, daniel@iogearbox.net, Stanislav Fomichev , Martin KaFai Lau , Yonghong Song Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Add a test that calls setsockopt on the listener socket which triggers BPF program. This BPF program writes to the sk storage and sets clone flag. Make sure that sk storage is cloned for a newly accepted connection. We have two cloned maps in the tests to make sure we hit both cases in bpf_sk_storage_clone: first element (sk_storage_alloc) and non-first element(s) (selem_link_map). Cc: Martin KaFai Lau Cc: Yonghong Song Acked-by: Martin KaFai Lau Acked-by: Yonghong Song Signed-off-by: Stanislav Fomichev --- tools/testing/selftests/bpf/.gitignore | 1 + tools/testing/selftests/bpf/Makefile | 3 +- .../selftests/bpf/progs/sockopt_inherit.c | 97 +++++++ .../selftests/bpf/test_sockopt_inherit.c | 253 ++++++++++++++++++ 4 files changed, 353 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/progs/sockopt_inherit.c create mode 100644 tools/testing/selftests/bpf/test_sockopt_inherit.c diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore index 90f70d2c7c22..60c9338cd9b4 100644 --- a/tools/testing/selftests/bpf/.gitignore +++ b/tools/testing/selftests/bpf/.gitignore @@ -42,4 +42,5 @@ xdping test_sockopt test_sockopt_sk test_sockopt_multi +test_sockopt_inherit test_tcp_rtt diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 29001f944db7..1faad0c3c3c9 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -29,7 +29,7 @@ TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test test_cgroup_storage test_select_reuseport test_section_names \ test_netcnt test_tcpnotify_user test_sock_fields test_sysctl test_hashmap \ test_btf_dump test_cgroup_attach xdping test_sockopt test_sockopt_sk \ - test_sockopt_multi test_tcp_rtt + test_sockopt_multi test_sockopt_inherit test_tcp_rtt BPF_OBJ_FILES = $(patsubst %.c,%.o, $(notdir $(wildcard progs/*.c))) TEST_GEN_FILES = $(BPF_OBJ_FILES) @@ -111,6 +111,7 @@ $(OUTPUT)/test_cgroup_attach: cgroup_helpers.c $(OUTPUT)/test_sockopt: cgroup_helpers.c $(OUTPUT)/test_sockopt_sk: cgroup_helpers.c $(OUTPUT)/test_sockopt_multi: cgroup_helpers.c +$(OUTPUT)/test_sockopt_inherit: cgroup_helpers.c $(OUTPUT)/test_tcp_rtt: cgroup_helpers.c .PHONY: force diff --git a/tools/testing/selftests/bpf/progs/sockopt_inherit.c b/tools/testing/selftests/bpf/progs/sockopt_inherit.c new file mode 100644 index 000000000000..dede0fcd6102 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/sockopt_inherit.c @@ -0,0 +1,97 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include "bpf_helpers.h" + +char _license[] SEC("license") = "GPL"; +__u32 _version SEC("version") = 1; + +#define SOL_CUSTOM 0xdeadbeef +#define CUSTOM_INHERIT1 0 +#define CUSTOM_INHERIT2 1 +#define CUSTOM_LISTENER 2 + +struct sockopt_inherit { + __u8 val; +}; + +struct { + __uint(type, BPF_MAP_TYPE_SK_STORAGE); + __uint(map_flags, BPF_F_NO_PREALLOC | BPF_F_CLONE); + __type(key, int); + __type(value, struct sockopt_inherit); +} cloned1_map SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_SK_STORAGE); + __uint(map_flags, BPF_F_NO_PREALLOC | BPF_F_CLONE); + __type(key, int); + __type(value, struct sockopt_inherit); +} cloned2_map SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_SK_STORAGE); + __uint(map_flags, BPF_F_NO_PREALLOC); + __type(key, int); + __type(value, struct sockopt_inherit); +} listener_only_map SEC(".maps"); + +static __inline struct sockopt_inherit *get_storage(struct bpf_sockopt *ctx) +{ + if (ctx->optname == CUSTOM_INHERIT1) + return bpf_sk_storage_get(&cloned1_map, ctx->sk, 0, + BPF_SK_STORAGE_GET_F_CREATE); + else if (ctx->optname == CUSTOM_INHERIT2) + return bpf_sk_storage_get(&cloned2_map, ctx->sk, 0, + BPF_SK_STORAGE_GET_F_CREATE); + else + return bpf_sk_storage_get(&listener_only_map, ctx->sk, 0, + BPF_SK_STORAGE_GET_F_CREATE); +} + +SEC("cgroup/getsockopt") +int _getsockopt(struct bpf_sockopt *ctx) +{ + __u8 *optval_end = ctx->optval_end; + struct sockopt_inherit *storage; + __u8 *optval = ctx->optval; + + if (ctx->level != SOL_CUSTOM) + return 1; /* only interested in SOL_CUSTOM */ + + if (optval + 1 > optval_end) + return 0; /* EPERM, bounds check */ + + storage = get_storage(ctx); + if (!storage) + return 0; /* EPERM, couldn't get sk storage */ + + ctx->retval = 0; /* Reset system call return value to zero */ + + optval[0] = storage->val; + ctx->optlen = 1; + + return 1; +} + +SEC("cgroup/setsockopt") +int _setsockopt(struct bpf_sockopt *ctx) +{ + __u8 *optval_end = ctx->optval_end; + struct sockopt_inherit *storage; + __u8 *optval = ctx->optval; + + if (ctx->level != SOL_CUSTOM) + return 1; /* only interested in SOL_CUSTOM */ + + if (optval + 1 > optval_end) + return 0; /* EPERM, bounds check */ + + storage = get_storage(ctx); + if (!storage) + return 0; /* EPERM, couldn't get sk storage */ + + storage->val = optval[0]; + ctx->optlen = -1; + + return 1; +} diff --git a/tools/testing/selftests/bpf/test_sockopt_inherit.c b/tools/testing/selftests/bpf/test_sockopt_inherit.c new file mode 100644 index 000000000000..1bf699815b9b --- /dev/null +++ b/tools/testing/selftests/bpf/test_sockopt_inherit.c @@ -0,0 +1,253 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include "bpf_rlimit.h" +#include "bpf_util.h" +#include "cgroup_helpers.h" + +#define CG_PATH "/sockopt_inherit" +#define SOL_CUSTOM 0xdeadbeef +#define CUSTOM_INHERIT1 0 +#define CUSTOM_INHERIT2 1 +#define CUSTOM_LISTENER 2 + +static int connect_to_server(int server_fd) +{ + struct sockaddr_storage addr; + socklen_t len = sizeof(addr); + int fd; + + fd = socket(AF_INET, SOCK_STREAM, 0); + if (fd < 0) { + log_err("Failed to create client socket"); + return -1; + } + + if (getsockname(server_fd, (struct sockaddr *)&addr, &len)) { + log_err("Failed to get server addr"); + goto out; + } + + if (connect(fd, (const struct sockaddr *)&addr, len) < 0) { + log_err("Fail to connect to server"); + goto out; + } + + return fd; + +out: + close(fd); + return -1; +} + +static int verify_sockopt(int fd, int optname, const char *msg, char expected) +{ + socklen_t optlen = 1; + char buf = 0; + int err; + + err = getsockopt(fd, SOL_CUSTOM, optname, &buf, &optlen); + if (err) { + log_err("%s: failed to call getsockopt", msg); + return 1; + } + + printf("%s %d: got=0x%x ? expected=0x%x\n", msg, optname, buf, expected); + + if (buf != expected) { + log_err("%s: unexpected getsockopt value %d != %d", msg, + buf, expected); + return 1; + } + + return 0; +} + +static void *server_thread(void *arg) +{ + struct sockaddr_storage addr; + socklen_t len = sizeof(addr); + int fd = *(int *)arg; + int client_fd; + int err = 0; + + if (listen(fd, 1) < 0) + error(1, errno, "Failed to listed on socket"); + + err += verify_sockopt(fd, CUSTOM_INHERIT1, "listen", 1); + err += verify_sockopt(fd, CUSTOM_INHERIT2, "listen", 1); + err += verify_sockopt(fd, CUSTOM_LISTENER, "listen", 1); + + client_fd = accept(fd, (struct sockaddr *)&addr, &len); + if (client_fd < 0) + error(1, errno, "Failed to accept client"); + + err += verify_sockopt(client_fd, CUSTOM_INHERIT1, "accept", 1); + err += verify_sockopt(client_fd, CUSTOM_INHERIT2, "accept", 1); + err += verify_sockopt(client_fd, CUSTOM_LISTENER, "accept", 0); + + close(client_fd); + + return (void *)(long)err; +} + +static int start_server(void) +{ + struct sockaddr_in addr = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + }; + char buf; + int err; + int fd; + int i; + + fd = socket(AF_INET, SOCK_STREAM, 0); + if (fd < 0) { + log_err("Failed to create server socket"); + return -1; + } + + for (i = CUSTOM_INHERIT1; i <= CUSTOM_LISTENER; i++) { + buf = 0x01; + err = setsockopt(fd, SOL_CUSTOM, i, &buf, 1); + if (err) { + log_err("Failed to call setsockopt(%d)", i); + close(fd); + return -1; + } + } + + if (bind(fd, (const struct sockaddr *)&addr, sizeof(addr)) < 0) { + log_err("Failed to bind socket"); + close(fd); + return -1; + } + + return fd; +} + +static int prog_attach(struct bpf_object *obj, int cgroup_fd, const char *title) +{ + enum bpf_attach_type attach_type; + enum bpf_prog_type prog_type; + struct bpf_program *prog; + int err; + + err = libbpf_prog_type_by_name(title, &prog_type, &attach_type); + if (err) { + log_err("Failed to deduct types for %s BPF program", title); + return -1; + } + + prog = bpf_object__find_program_by_title(obj, title); + if (!prog) { + log_err("Failed to find %s BPF program", title); + return -1; + } + + err = bpf_prog_attach(bpf_program__fd(prog), cgroup_fd, + attach_type, 0); + if (err) { + log_err("Failed to attach %s BPF program", title); + return -1; + } + + return 0; +} + +static int run_test(int cgroup_fd) +{ + struct bpf_prog_load_attr attr = { + .file = "./sockopt_inherit.o", + }; + int server_fd = -1, client_fd; + struct bpf_object *obj; + void *server_err; + pthread_t tid; + int ignored; + int err; + + err = bpf_prog_load_xattr(&attr, &obj, &ignored); + if (err) { + log_err("Failed to load BPF object"); + return -1; + } + + err = prog_attach(obj, cgroup_fd, "cgroup/getsockopt"); + if (err) + goto close_bpf_object; + + err = prog_attach(obj, cgroup_fd, "cgroup/setsockopt"); + if (err) + goto close_bpf_object; + + server_fd = start_server(); + if (server_fd < 0) { + err = -1; + goto close_bpf_object; + } + + pthread_create(&tid, NULL, server_thread, (void *)&server_fd); + + client_fd = connect_to_server(server_fd); + if (client_fd < 0) { + err = -1; + goto close_server_fd; + } + + err += verify_sockopt(client_fd, CUSTOM_INHERIT1, "connect", 0); + err += verify_sockopt(client_fd, CUSTOM_INHERIT2, "connect", 0); + err += verify_sockopt(client_fd, CUSTOM_LISTENER, "connect", 0); + + pthread_join(tid, &server_err); + + err += (int)(long)server_err; + + close(client_fd); + +close_server_fd: + close(server_fd); +close_bpf_object: + bpf_object__close(obj); + return err; +} + +int main(int args, char **argv) +{ + int cgroup_fd; + int err = EXIT_SUCCESS; + + if (setup_cgroup_environment()) + return err; + + cgroup_fd = create_and_get_cgroup(CG_PATH); + if (cgroup_fd < 0) + goto cleanup_cgroup_env; + + if (join_cgroup(CG_PATH)) + goto cleanup_cgroup; + + if (run_test(cgroup_fd)) + err = EXIT_FAILURE; + + printf("test_sockopt_inherit: %s\n", + err == EXIT_SUCCESS ? "PASSED" : "FAILED"); + +cleanup_cgroup: + close(cgroup_fd); +cleanup_cgroup_env: + cleanup_cgroup_environment(); + return err; +}