From patchwork Wed May 8 17:18:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanislav Fomichev X-Patchwork-Id: 1097148 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.b="sClWGi7u"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 44zjqw6GkKz9s7h for ; Thu, 9 May 2019 03:18:52 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728921AbfEHRSw (ORCPT ); Wed, 8 May 2019 13:18:52 -0400 Received: from mail-vs1-f74.google.com ([209.85.217.74]:39122 "EHLO mail-vs1-f74.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728676AbfEHRSw (ORCPT ); Wed, 8 May 2019 13:18:52 -0400 Received: by mail-vs1-f74.google.com with SMTP id i9so4048201vsm.6 for ; Wed, 08 May 2019 10:18:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=YsTefcGNkf2Q/mZpauJsyMPBMEq1hDK7EmnhSqbxuI0=; b=sClWGi7ujpn87u3owuoyPatrLqrKe+FTZdsQzsYSMxV5GuSZTuH8BYL/C2yvEffDyY oalmfgqVtv+4X20AMQvFrA9oyRuRwBfL6N0upurbJVP9gK1w+O6ZPcRPMU/BbkJiT6DL Sp5YRlbx97ohczv6bKsEHo7ycwOPjANOgh2sZaarU2YGiYBVFB9o/hJjSrkH1pA+jMLV yKN+BJPUzSdCttpln7RAD5hPLWYJH2JalICt0odlRuWy1nVM0duxML6TI+qE2I8uCIuV 7QBYN8Ij1U5/VZue/1nr+NrnF8PobvqpcuBf7h9jggOJs/Z3tn2WvFSy+UR8S4UNf1BP oCDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=YsTefcGNkf2Q/mZpauJsyMPBMEq1hDK7EmnhSqbxuI0=; b=T4HVF/LJozyA6UAoNkDHXKTdCjqZGUWf4ZYErb+8Mn9rRyMNAL1KjbfXzDHa+ItOiv tW75uNNURMbXpQxJG9LKlVd+ECRao5xB0pkEHDbZxsh8zJ5/f0En3q2heoOg6gR5d9s3 fMX2Z2H9Mh1cydElVhhWP/4IXkjWDj0myF+mPSUoCdvCoOduds2cflbwmEHE0lstOIVd X5QxjjSKE7OCueVtxga/0YsIu8yZZipcCPWrh4ygm/9sk7GwT+2r215HS+GrVPUUPPWC uA1soQnkYS3fXvvZu/5hxbxxGbLJtyVfRwzk4O5rSdCkBBxCBzSChSorWmA0SRripnCz Mkcg== X-Gm-Message-State: APjAAAUHDrMI/KY68K6R1Bb+mWI0CRPJOAhcOLDKCV81YmaiRVxrtPd9 KxCcPWZhm/p4JBM62YVJsG8RBdo= X-Google-Smtp-Source: APXvYqyf7nPoRvIo5uDirN7fq+seYuLhBE/Tg4GvR/S9VmBqkEK+iF95ILtrtDPiU+1A+MbJlEmH7U0= X-Received: by 2002:ab0:c12:: with SMTP id a18mr7746194uak.69.1557335930641; Wed, 08 May 2019 10:18:50 -0700 (PDT) Date: Wed, 8 May 2019 10:18:42 -0700 In-Reply-To: <20190508171845.201303-1-sdf@google.com> Message-Id: <20190508171845.201303-2-sdf@google.com> Mime-Version: 1.0 References: <20190508171845.201303-1-sdf@google.com> X-Mailer: git-send-email 2.21.0.1020.gf2820cf01a-goog Subject: [PATCH bpf 1/4] bpf: remove __rcu annotations from bpf_prog_array From: Stanislav Fomichev To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: davem@davemloft.net, ast@kernel.org, daniel@iogearbox.net, Stanislav Fomichev Sender: bpf-owner@vger.kernel.org Precedence: bulk List-Id: netdev.vger.kernel.org Drop __rcu annotations and rcu read sections. That's not needed since all existing callers call those helpers from the rcu update side and under a mutex. This guarantees that use-after-free could not happen. In the next patches I'll fix the callers with missing rcu_dereference_protected to make sparse/lockdep happy. Signed-off-by: Stanislav Fomichev --- include/linux/bpf.h | 12 ++++++------ kernel/bpf/core.c | 31 ++++++++++++------------------- 2 files changed, 18 insertions(+), 25 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 944ccc310201..b90d2859bc60 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -476,17 +476,17 @@ struct bpf_prog_array { }; struct bpf_prog_array *bpf_prog_array_alloc(u32 prog_cnt, gfp_t flags); -void bpf_prog_array_free(struct bpf_prog_array __rcu *progs); -int bpf_prog_array_length(struct bpf_prog_array __rcu *progs); -int bpf_prog_array_copy_to_user(struct bpf_prog_array __rcu *progs, +void bpf_prog_array_free(struct bpf_prog_array *progs); +int bpf_prog_array_length(struct bpf_prog_array *progs); +int bpf_prog_array_copy_to_user(struct bpf_prog_array *progs, __u32 __user *prog_ids, u32 cnt); -void bpf_prog_array_delete_safe(struct bpf_prog_array __rcu *progs, +void bpf_prog_array_delete_safe(struct bpf_prog_array *progs, struct bpf_prog *old_prog); -int bpf_prog_array_copy_info(struct bpf_prog_array __rcu *array, +int bpf_prog_array_copy_info(struct bpf_prog_array *array, u32 *prog_ids, u32 request_cnt, u32 *prog_cnt); -int bpf_prog_array_copy(struct bpf_prog_array __rcu *old_array, +int bpf_prog_array_copy(struct bpf_prog_array *old_array, struct bpf_prog *exclude_prog, struct bpf_prog *include_prog, struct bpf_prog_array **new_array); diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index ff09d32a8a1b..da03fbc811fd 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -1794,38 +1794,33 @@ struct bpf_prog_array *bpf_prog_array_alloc(u32 prog_cnt, gfp_t flags) return &empty_prog_array.hdr; } -void bpf_prog_array_free(struct bpf_prog_array __rcu *progs) +void bpf_prog_array_free(struct bpf_prog_array *progs) { - if (!progs || - progs == (struct bpf_prog_array __rcu *)&empty_prog_array.hdr) + if (!progs || progs == &empty_prog_array.hdr) return; kfree_rcu(progs, rcu); } -int bpf_prog_array_length(struct bpf_prog_array __rcu *array) +int bpf_prog_array_length(struct bpf_prog_array *array) { struct bpf_prog_array_item *item; u32 cnt = 0; - rcu_read_lock(); - item = rcu_dereference(array)->items; - for (; item->prog; item++) + for (item = array->items; item->prog; item++) if (item->prog != &dummy_bpf_prog.prog) cnt++; - rcu_read_unlock(); return cnt; } -static bool bpf_prog_array_copy_core(struct bpf_prog_array __rcu *array, +static bool bpf_prog_array_copy_core(struct bpf_prog_array *array, u32 *prog_ids, u32 request_cnt) { struct bpf_prog_array_item *item; int i = 0; - item = rcu_dereference_check(array, 1)->items; - for (; item->prog; item++) { + for (item = array->items; item->prog; item++) { if (item->prog == &dummy_bpf_prog.prog) continue; prog_ids[i] = item->prog->aux->id; @@ -1838,7 +1833,7 @@ static bool bpf_prog_array_copy_core(struct bpf_prog_array __rcu *array, return !!(item->prog); } -int bpf_prog_array_copy_to_user(struct bpf_prog_array __rcu *array, +int bpf_prog_array_copy_to_user(struct bpf_prog_array *array, __u32 __user *prog_ids, u32 cnt) { unsigned long err = 0; @@ -1858,9 +1853,7 @@ int bpf_prog_array_copy_to_user(struct bpf_prog_array __rcu *array, ids = kcalloc(cnt, sizeof(u32), GFP_USER | __GFP_NOWARN); if (!ids) return -ENOMEM; - rcu_read_lock(); nospc = bpf_prog_array_copy_core(array, ids, cnt); - rcu_read_unlock(); err = copy_to_user(prog_ids, ids, cnt * sizeof(u32)); kfree(ids); if (err) @@ -1870,19 +1863,19 @@ int bpf_prog_array_copy_to_user(struct bpf_prog_array __rcu *array, return 0; } -void bpf_prog_array_delete_safe(struct bpf_prog_array __rcu *array, +void bpf_prog_array_delete_safe(struct bpf_prog_array *array, struct bpf_prog *old_prog) { - struct bpf_prog_array_item *item = array->items; + struct bpf_prog_array_item *item; - for (; item->prog; item++) + for (item = array->items; item->prog; item++) if (item->prog == old_prog) { WRITE_ONCE(item->prog, &dummy_bpf_prog.prog); break; } } -int bpf_prog_array_copy(struct bpf_prog_array __rcu *old_array, +int bpf_prog_array_copy(struct bpf_prog_array *old_array, struct bpf_prog *exclude_prog, struct bpf_prog *include_prog, struct bpf_prog_array **new_array) @@ -1946,7 +1939,7 @@ int bpf_prog_array_copy(struct bpf_prog_array __rcu *old_array, return 0; } -int bpf_prog_array_copy_info(struct bpf_prog_array __rcu *array, +int bpf_prog_array_copy_info(struct bpf_prog_array *array, u32 *prog_ids, u32 request_cnt, u32 *prog_cnt) { From patchwork Wed May 8 17:18:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanislav Fomichev X-Patchwork-Id: 1097150 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.b="pANr7Ap0"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 44zjqz6vwGz9s55 for ; Thu, 9 May 2019 03:18:55 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728953AbfEHRSz (ORCPT ); Wed, 8 May 2019 13:18:55 -0400 Received: from mail-qt1-f202.google.com ([209.85.160.202]:56636 "EHLO mail-qt1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728943AbfEHRSy (ORCPT ); Wed, 8 May 2019 13:18:54 -0400 Received: by mail-qt1-f202.google.com with SMTP id v3so5139100qtp.23 for ; Wed, 08 May 2019 10:18:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=dBVF9tqLXphyriby2Sali7IJHS+SaCg6nJ2FY2lFxPo=; b=pANr7Ap0bi82ywW+IsBk6pG6XmeRA3aCL68H2DnzN+AKtyjvd7LsrJs7hy8JdHGnhU Bl8NMnwZoWlVOigv4u3XWznJ1JmeWFf39CuMEVJQbMhL94Gfh8nXH3B3pZB8b9wCd5u3 m8MdHX9C0varGlzBwzKIRstmEz6nr1XrWnkzd8D275SKuhQiGwRZur1kq6BLlluG/Egf Uxvwd9tS0JTuKxpNAiPg9CE/tf//Pwjegj2zI+QWlwu/jtnTVGByoqOgGL6RQ28DV9pl LDRmQKLs6GUcVSaWcmv1Ia39qEq4Lrbj92sqFZ+5euvQ0q3Qa+jh8NggfKFfdVa0diTs MwRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=dBVF9tqLXphyriby2Sali7IJHS+SaCg6nJ2FY2lFxPo=; b=p8wW21VHt6G+QbEsIoJ1UudWEgVTddMHWE6WwqrzsLpk04dcIY6hdb4UlzL/II0F1M rNlhkfw+A/VZHOnVwif0vsGrlgwzXYumFCnWsueVM7JKPodQo39Jh/ezozlpp2drXfUQ OcE1gIs4+8XAGlSe1efV33rpRbr8In8ux7/HgTI/8U38zXQ/zOZJ6TOz1uZX2mOQ4MPT VzDz0hW2rlcD34dMv/T85TNWhbOE1vBrJUBikwE78M8tq/PoWmiQerynEc/V1MQyQavP Fw4VoY2kkHg1gPcuc0O2VcRII7LkfGZ+5nznbNx+6kh+iXcwT+nnPUnhW1CU7uKB34Od uevA== X-Gm-Message-State: APjAAAWgmu2u3b4qya+zfJ48xfRz50VY2KpeF7khuYHy6eTuo7nB8y5Q XyHTNPzrQGbAHcSMzRYTttLfzpw= X-Google-Smtp-Source: APXvYqxR4fl9jIla2OhmLIzaZ6+WE2iEANBaN6rxOlayRJlxxKof7suMNC0Z0moEAiBEdZucFt7h+zI= X-Received: by 2002:a37:884:: with SMTP id 126mr1997936qki.67.1557335933118; Wed, 08 May 2019 10:18:53 -0700 (PDT) Date: Wed, 8 May 2019 10:18:43 -0700 In-Reply-To: <20190508171845.201303-1-sdf@google.com> Message-Id: <20190508171845.201303-3-sdf@google.com> Mime-Version: 1.0 References: <20190508171845.201303-1-sdf@google.com> X-Mailer: git-send-email 2.21.0.1020.gf2820cf01a-goog Subject: [PATCH bpf 2/4] bpf: media: properly use bpf_prog_array api From: Stanislav Fomichev To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: davem@davemloft.net, ast@kernel.org, daniel@iogearbox.net, Stanislav Fomichev , linux-media@vger.kernel.org, Sean Young Sender: bpf-owner@vger.kernel.org Precedence: bulk List-Id: netdev.vger.kernel.org Now that we don't have __rcu markers on the bpf_prog_array helpers, let's use proper rcu_dereference_protected to obtain array pointer under mutex. Cc: linux-media@vger.kernel.org Cc: Sean Young Signed-off-by: Stanislav Fomichev --- drivers/media/rc/bpf-lirc.c | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/drivers/media/rc/bpf-lirc.c b/drivers/media/rc/bpf-lirc.c index 390a722e6211..38d4c01174fc 100644 --- a/drivers/media/rc/bpf-lirc.c +++ b/drivers/media/rc/bpf-lirc.c @@ -8,6 +8,9 @@ #include #include "rc-core-priv.h" +#define lirc_dereference(p) \ + rcu_dereference_protected(p, lockdep_is_held(&ir_raw_handler_lock)) + /* * BPF interface for raw IR */ @@ -130,7 +133,7 @@ const struct bpf_verifier_ops lirc_mode2_verifier_ops = { static int lirc_bpf_attach(struct rc_dev *rcdev, struct bpf_prog *prog) { - struct bpf_prog_array __rcu *old_array; + struct bpf_prog_array *old_array; struct bpf_prog_array *new_array; struct ir_raw_event_ctrl *raw; int ret; @@ -148,12 +151,12 @@ static int lirc_bpf_attach(struct rc_dev *rcdev, struct bpf_prog *prog) goto unlock; } - if (raw->progs && bpf_prog_array_length(raw->progs) >= BPF_MAX_PROGS) { + old_array = lirc_dereference(raw->progs); + if (old_array && bpf_prog_array_length(old_array) >= BPF_MAX_PROGS) { ret = -E2BIG; goto unlock; } - old_array = raw->progs; ret = bpf_prog_array_copy(old_array, NULL, prog, &new_array); if (ret < 0) goto unlock; @@ -168,7 +171,7 @@ static int lirc_bpf_attach(struct rc_dev *rcdev, struct bpf_prog *prog) static int lirc_bpf_detach(struct rc_dev *rcdev, struct bpf_prog *prog) { - struct bpf_prog_array __rcu *old_array; + struct bpf_prog_array *old_array; struct bpf_prog_array *new_array; struct ir_raw_event_ctrl *raw; int ret; @@ -186,7 +189,7 @@ static int lirc_bpf_detach(struct rc_dev *rcdev, struct bpf_prog *prog) goto unlock; } - old_array = raw->progs; + old_array = lirc_dereference(raw->progs); ret = bpf_prog_array_copy(old_array, prog, NULL, &new_array); /* * Do not use bpf_prog_array_delete_safe() as we would end up @@ -217,21 +220,25 @@ void lirc_bpf_run(struct rc_dev *rcdev, u32 sample) /* * This should be called once the rc thread has been stopped, so there can be * no concurrent bpf execution. + * + * Should be called with the ir_raw_handler_lock held. */ void lirc_bpf_free(struct rc_dev *rcdev) { struct bpf_prog_array_item *item; + struct bpf_prog_array *array; - if (!rcdev->raw->progs) + array = lirc_dereference(rcdev->raw->progs); + if (!array) return; - item = rcu_dereference(rcdev->raw->progs)->items; + item = array->items; while (item->prog) { bpf_prog_put(item->prog); item++; } - bpf_prog_array_free(rcdev->raw->progs); + bpf_prog_array_free(array); } int lirc_prog_attach(const union bpf_attr *attr, struct bpf_prog *prog) @@ -284,7 +291,7 @@ int lirc_prog_detach(const union bpf_attr *attr) int lirc_prog_query(const union bpf_attr *attr, union bpf_attr __user *uattr) { __u32 __user *prog_ids = u64_to_user_ptr(attr->query.prog_ids); - struct bpf_prog_array __rcu *progs; + struct bpf_prog_array *progs; struct rc_dev *rcdev; u32 cnt, flags = 0; int ret; @@ -305,7 +312,7 @@ int lirc_prog_query(const union bpf_attr *attr, union bpf_attr __user *uattr) if (ret) goto put; - progs = rcdev->raw->progs; + progs = lirc_dereference(rcdev->raw->progs); cnt = progs ? bpf_prog_array_length(progs) : 0; if (copy_to_user(&uattr->query.prog_cnt, &cnt, sizeof(cnt))) { From patchwork Wed May 8 17:18:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanislav Fomichev X-Patchwork-Id: 1097152 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.b="hwvwjjRP"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 44zjr36Jt4z9s7h for ; Thu, 9 May 2019 03:18:59 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728984AbfEHRS6 (ORCPT ); Wed, 8 May 2019 13:18:58 -0400 Received: from mail-pg1-f202.google.com ([209.85.215.202]:52035 "EHLO mail-pg1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728966AbfEHRS4 (ORCPT ); Wed, 8 May 2019 13:18:56 -0400 Received: by mail-pg1-f202.google.com with SMTP id 63so13046908pga.18 for ; Wed, 08 May 2019 10:18:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=74cU3TEY3NZS3RPv5+orUOI/SPoOJAznACo1y6F/NDw=; b=hwvwjjRPOZrrfGNXRMOer5WLDSi7Y31SUnrZrATb0qUDouxcLan1DCSuDi3qUjUzH6 3WV0/MMdatcSUiAVrs13UjXPL6+DFPNQ+WcSsDIAdNaCIk46vEcjFGYuVNIj9m2GlJVU 9CTjMRfAWED5I9vTjp/Nd/1NVzWIuFE1rmr9//6meu+OHNOHjzdyN7B4AfoCgF1BbUOM U8U3WN3ORG0Ci2vISxdz0ZftbOBNmjDgwQDmzGosjEcpo5kv1X5qkA04CJ6oC4Cpwpa9 zsm56D0Ookgf8CwgnnlSz4grIZ3A1ZGJZ4mQcjLjh5/IbyLe5uoRu/vsKLOPK5fQTxkT 13qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=74cU3TEY3NZS3RPv5+orUOI/SPoOJAznACo1y6F/NDw=; b=qqBE4WAglGEF4sAtgPklst5UZyYvXOyP62kQ5JEZkP0BVh5IJkz2UtF6aKxdd02Vp1 EDMiOGuoVvoff22D5SC0+ztDun5BAg4v+qoSWIzJ68lrYc2S7eqTXd6j7D057nNNNMmB 3R/BKihxdtW0ziG6uIa4VHoBOumqgRzGu/dbRMLOyiDqLnJ/ifqPBaW+eqFAAIOqWTDs b/eTp4FfuSTKZInBj2ZRd7sGJTmfosyLknhopOJsZ+hafYQbJbnRNuLNy/jsDWZOKmVO UZowsQTIwD7Pa0+Cwz2KVdvdmoI8lm4gSR3RqcztMpbZme6ZPSTlFPcbDI+j05H3eCwZ Vbdg== X-Gm-Message-State: APjAAAXGbKrwEYFHQZFok8ZmHcmqcP79FIFuK8pGgjdM2yLt6FF2wgpi agrIEKR964fRTLv6ruy5T3nJV4xVfR5PtCwOnv2gZqTASsZTp7aRrOpDXCaisU9gyP6VQFk2l4x jlUiEDFaUxvihMljQuGkbZxoOwriobjinwqaCiE/ZmYPT69aijvEaKw== X-Google-Smtp-Source: APXvYqwmRD762PKVo674Po7dw8yw8DkjXrl2ffkrOz3cS/0rd9MGQlICco2ivsWVFmV77sSOay+zvyk= X-Received: by 2002:a63:931c:: with SMTP id b28mr17083813pge.182.1557335935040; Wed, 08 May 2019 10:18:55 -0700 (PDT) Date: Wed, 8 May 2019 10:18:44 -0700 In-Reply-To: <20190508171845.201303-1-sdf@google.com> Message-Id: <20190508171845.201303-4-sdf@google.com> Mime-Version: 1.0 References: <20190508171845.201303-1-sdf@google.com> X-Mailer: git-send-email 2.21.0.1020.gf2820cf01a-goog Subject: [PATCH bpf 3/4] bpf: cgroup: properly use bpf_prog_array api From: Stanislav Fomichev To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: davem@davemloft.net, ast@kernel.org, daniel@iogearbox.net, Stanislav Fomichev Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Now that we don't have __rcu markers on the bpf_prog_array helpers, let's use proper rcu_dereference_protected to obtain array pointer under mutex. We also don't need __rcu annotations on cgroup_bpf.inactive since it's not read/updated concurrently. Signed-off-by: Stanislav Fomichev --- include/linux/bpf-cgroup.h | 2 +- kernel/bpf/cgroup.c | 27 +++++++++++++++++---------- 2 files changed, 18 insertions(+), 11 deletions(-) diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h index a4c644c1c091..5e515b72ff55 100644 --- a/include/linux/bpf-cgroup.h +++ b/include/linux/bpf-cgroup.h @@ -69,7 +69,7 @@ struct cgroup_bpf { u32 flags[MAX_BPF_ATTACH_TYPE]; /* temp storage for effective prog array used by prog_attach/detach */ - struct bpf_prog_array __rcu *inactive; + struct bpf_prog_array *inactive; }; void cgroup_bpf_put(struct cgroup *cgrp); diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c index 4e807973aa80..d59826add5ef 100644 --- a/kernel/bpf/cgroup.c +++ b/kernel/bpf/cgroup.c @@ -19,6 +19,9 @@ DEFINE_STATIC_KEY_FALSE(cgroup_bpf_enabled_key); EXPORT_SYMBOL(cgroup_bpf_enabled_key); +#define cgroup_dereference(p) \ + rcu_dereference_protected(p, lockdep_is_held(&cgroup_mutex)) + /** * cgroup_bpf_put() - put references of all bpf programs * @cgrp: the cgroup to modify @@ -26,6 +29,7 @@ EXPORT_SYMBOL(cgroup_bpf_enabled_key); void cgroup_bpf_put(struct cgroup *cgrp) { enum bpf_cgroup_storage_type stype; + struct bpf_prog_array *old_array; unsigned int type; for (type = 0; type < ARRAY_SIZE(cgrp->bpf.progs); type++) { @@ -42,7 +46,8 @@ void cgroup_bpf_put(struct cgroup *cgrp) kfree(pl); static_branch_dec(&cgroup_bpf_enabled_key); } - bpf_prog_array_free(cgrp->bpf.effective[type]); + old_array = cgroup_dereference(cgrp->bpf.effective[type]); + bpf_prog_array_free(old_array); } } @@ -98,7 +103,7 @@ static bool hierarchy_allows_attach(struct cgroup *cgrp, */ static int compute_effective_progs(struct cgroup *cgrp, enum bpf_attach_type type, - struct bpf_prog_array __rcu **array) + struct bpf_prog_array **array) { enum bpf_cgroup_storage_type stype; struct bpf_prog_array *progs; @@ -136,17 +141,17 @@ static int compute_effective_progs(struct cgroup *cgrp, } } while ((p = cgroup_parent(p))); - rcu_assign_pointer(*array, progs); + *array = progs; return 0; } static void activate_effective_progs(struct cgroup *cgrp, enum bpf_attach_type type, - struct bpf_prog_array __rcu *array) + struct bpf_prog_array *array) { - struct bpf_prog_array __rcu *old_array; + struct bpf_prog_array *old_array; - old_array = xchg(&cgrp->bpf.effective[type], array); + old_array = xchg((__force struct bpf_prog_array **)&cgrp->bpf.effective[type], array); /* free prog array after grace period, since __cgroup_bpf_run_*() * might be still walking the array */ @@ -163,7 +168,7 @@ int cgroup_bpf_inherit(struct cgroup *cgrp) * that array below is variable length */ #define NR ARRAY_SIZE(cgrp->bpf.effective) - struct bpf_prog_array __rcu *arrays[NR] = {}; + struct bpf_prog_array *arrays[NR] = {}; int i; for (i = 0; i < NR; i++) @@ -441,10 +446,13 @@ int __cgroup_bpf_query(struct cgroup *cgrp, const union bpf_attr *attr, enum bpf_attach_type type = attr->query.attach_type; struct list_head *progs = &cgrp->bpf.progs[type]; u32 flags = cgrp->bpf.flags[type]; + struct bpf_prog_array *effective; int cnt, ret = 0, i; + effective = cgroup_dereference(cgrp->bpf.effective[type]); + if (attr->query.query_flags & BPF_F_QUERY_EFFECTIVE) - cnt = bpf_prog_array_length(cgrp->bpf.effective[type]); + cnt = bpf_prog_array_length(effective); else cnt = prog_list_length(progs); @@ -461,8 +469,7 @@ int __cgroup_bpf_query(struct cgroup *cgrp, const union bpf_attr *attr, } if (attr->query.query_flags & BPF_F_QUERY_EFFECTIVE) { - return bpf_prog_array_copy_to_user(cgrp->bpf.effective[type], - prog_ids, cnt); + return bpf_prog_array_copy_to_user(effective, prog_ids, cnt); } else { struct bpf_prog_list *pl; u32 id; From patchwork Wed May 8 17:18:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanislav Fomichev X-Patchwork-Id: 1097154 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.b="sCcoidYM"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 44zjr52vznz9s55 for ; Thu, 9 May 2019 03:19:01 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728995AbfEHRTA (ORCPT ); Wed, 8 May 2019 13:19:00 -0400 Received: from mail-pl1-f202.google.com ([209.85.214.202]:34938 "EHLO mail-pl1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728982AbfEHRS6 (ORCPT ); Wed, 8 May 2019 13:18:58 -0400 Received: by mail-pl1-f202.google.com with SMTP id x5so11921055pll.2 for ; Wed, 08 May 2019 10:18:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Oc7kGTxNNeGPMnF+md75kkPiq5ynkJFXLqMqAf1RaYg=; b=sCcoidYMTgO8wDHVNyo0zWkArtuSb7+lrPRlpqwpffe0Qu3asnZmSU3fYLKVcgq4OG cvdqIziG4VFAUaXw5hlGpmPAxzyN2fGyhXIxCqwCd8PMTgb62S7MPB/6SvM0fYeaPRC5 q5l24qgY2+rD3nsgx2QYGgw0ohAfXfww3oU0BAJYO8NOJCMFCl57QvdEDJVw1K0FgmHe WIuGspqMa+feZ7XQbBsTFd+qPatPsh++1NPW4i7A+9AnoLE19Df3PcrZBikTobBqcYvl jNaFIyOo+H3IwNWC4JxrOZMKaUXa4dyZ7nuQEVCN/ubWWz/j4q1Z8xwepbS6J6GmdR2C K8vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Oc7kGTxNNeGPMnF+md75kkPiq5ynkJFXLqMqAf1RaYg=; b=hLko2Qcc9D4LhELCdgz7uM/2+qq9LN3icnQT88jQ3dJAZ2x5g3dDXsGMHrIs4WIYmF 1GIqTRrABwBBqva3Czjps5OGPxBo170GF9DGdF/POK5DGxwEIUsb7oT+ojLzyr3DPR4w g2q/ltWdAlFlzB7RrLcmEJRaGlULAe0JQRQNZC/gwTwCO9uxRI2VbQxldv+aYf5rp9Nz YpOmhamNRmjFp9vSJZ2DvBHFZ2Fo0w4eOSaMleLknGwlIkCmb1I8oTyNvpGzHgN209BH uhMoCSu0ULWRvjh+OsRwcXjUgkojMCyAIUQQbrmqbQbTk4OEPbt6OdOMDALU3PsmSHZH VqVw== X-Gm-Message-State: APjAAAUhyKQLa9PrRCI+iTLx9KHdhf6WLiiAXGWUPbSP840bX9vl4KjZ nFlFWv313EwViO6sWfXuO+Dg+fTPkc78a4lWsNQNpegRn0Wy8yiJRZBGvtcvqekuJ+qyvU1JHli vsEIuXSfHwq0Yl/eWff3xbfm3dKmWijs6kp++xFuznDcgNOJ2je9XFw== X-Google-Smtp-Source: APXvYqzuMXoR4vtaeQnTo9Th5W6+ExYtT4hBDJZP8CCSEYSCPHu3PrFjoogcKMz4CIiw8wE10YkgZkI= X-Received: by 2002:a63:191b:: with SMTP id z27mr48733425pgl.327.1557335937408; Wed, 08 May 2019 10:18:57 -0700 (PDT) Date: Wed, 8 May 2019 10:18:45 -0700 In-Reply-To: <20190508171845.201303-1-sdf@google.com> Message-Id: <20190508171845.201303-5-sdf@google.com> Mime-Version: 1.0 References: <20190508171845.201303-1-sdf@google.com> X-Mailer: git-send-email 2.21.0.1020.gf2820cf01a-goog Subject: [PATCH bpf 4/4] bpf: tracing: properly use bpf_prog_array api From: Stanislav Fomichev To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: davem@davemloft.net, ast@kernel.org, daniel@iogearbox.net, Stanislav Fomichev Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Now that we don't have __rcu markers on the bpf_prog_array helpers, let's use proper rcu_dereference_protected to obtain array pointer under mutex. Signed-off-by: Stanislav Fomichev --- kernel/trace/bpf_trace.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index d64c00afceb5..5f8f7fdbe27c 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -17,6 +17,9 @@ #include "trace_probe.h" #include "trace.h" +#define bpf_event_dereference(p) \ + rcu_dereference_protected(p, lockdep_is_held(&bpf_event_mutex)) + #ifdef CONFIG_MODULES struct bpf_trace_module { struct module *module; @@ -999,7 +1002,7 @@ static DEFINE_MUTEX(bpf_event_mutex); int perf_event_attach_bpf_prog(struct perf_event *event, struct bpf_prog *prog) { - struct bpf_prog_array __rcu *old_array; + struct bpf_prog_array *old_array; struct bpf_prog_array *new_array; int ret = -EEXIST; @@ -1017,7 +1020,7 @@ int perf_event_attach_bpf_prog(struct perf_event *event, if (event->prog) goto unlock; - old_array = event->tp_event->prog_array; + old_array = bpf_event_dereference(event->tp_event->prog_array); if (old_array && bpf_prog_array_length(old_array) >= BPF_TRACE_MAX_PROGS) { ret = -E2BIG; @@ -1040,7 +1043,7 @@ int perf_event_attach_bpf_prog(struct perf_event *event, void perf_event_detach_bpf_prog(struct perf_event *event) { - struct bpf_prog_array __rcu *old_array; + struct bpf_prog_array *old_array; struct bpf_prog_array *new_array; int ret; @@ -1049,7 +1052,7 @@ void perf_event_detach_bpf_prog(struct perf_event *event) if (!event->prog) goto unlock; - old_array = event->tp_event->prog_array; + old_array = bpf_event_dereference(event->tp_event->prog_array); ret = bpf_prog_array_copy(old_array, event->prog, NULL, &new_array); if (ret == -ENOENT) goto unlock; @@ -1071,6 +1074,7 @@ int perf_event_query_prog_array(struct perf_event *event, void __user *info) { struct perf_event_query_bpf __user *uquery = info; struct perf_event_query_bpf query = {}; + struct bpf_prog_array *progs; u32 *ids, prog_cnt, ids_len; int ret; @@ -1095,10 +1099,8 @@ int perf_event_query_prog_array(struct perf_event *event, void __user *info) */ mutex_lock(&bpf_event_mutex); - ret = bpf_prog_array_copy_info(event->tp_event->prog_array, - ids, - ids_len, - &prog_cnt); + progs = bpf_event_dereference(event->tp_event->prog_array); + ret = bpf_prog_array_copy_info(progs, ids, ids_len, &prog_cnt); mutex_unlock(&bpf_event_mutex); if (copy_to_user(&uquery->prog_cnt, &prog_cnt, sizeof(prog_cnt)) ||