From patchwork Fri Jan 5 06:09:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 855929 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="EB1W0e+3"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zCZ7819ryz9s75 for ; Fri, 5 Jan 2018 17:11:32 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751330AbeAEGKL (ORCPT ); Fri, 5 Jan 2018 01:10:11 -0500 Received: from mail-pf0-f193.google.com ([209.85.192.193]:42381 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751202AbeAEGKG (ORCPT ); Fri, 5 Jan 2018 01:10:06 -0500 Received: by mail-pf0-f193.google.com with SMTP id d23so1827494pfe.9 for ; Thu, 04 Jan 2018 22:10:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=QNLEL+oeHtuiYJvYRrdmrZBDA5Ci/ZpE8usqRRM8G+o=; b=EB1W0e+3y/HWQbaUmfPtuxblAM6URfQ9J7nTu7LGLBqdz1hwV3+QuXKuybpXIwd9Bf pl76FnA0IDmxk09dveTAZHACwPRDI+X8Cj/e1PafZAJwavNhHgn8cFSG1W5PDczpw76f nSuzbyubKq+HfLHiR1iC98V6oPLj9boykfqHT0Vj+C46EYqHbUlOGHPyKL51W5GD19A3 MBMd+zlIkOO8i602Nn2mRAszhcZ45hmfmwve/QnrV2VedbRAhBZQkTW2M12I6jC5lPmw hldyItClkUxLaj/96ZX+b/Xq9QvFWsBNMG+1EJHrI/EV3Xq3ilWOiAvcFb6EnfkPMEJJ Xrkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=QNLEL+oeHtuiYJvYRrdmrZBDA5Ci/ZpE8usqRRM8G+o=; b=Xc1NaXl5xX1aFyZbpW1Aeg243lokNfhgfK7ajHhghsVCfSBe2ntWKc/yoE8dh+eHiP VnUKMZiGlb1NAcDKzK4x1Yj6MAAo7cjM5VJlmii8x7j1SEueZKwfgxbxOOpC/PPFgwqA VswMiVwDlCwppvtXDY4T0cgVnNgRa2sFIza9CBAg79dBF52QO4ItuD37uaoYvTZK+dly AjWTrPNRcHH2QKErnoT2IlIZA5eRxuMzb5udkvfPRIL2MaVtio4wKdxVRYfxakk78T6+ fkeaZbsX9EAnamACGA0ED2jUY2UFQgR1EC3hlEq+/fYYQtLRVpCiGFEPLhZwXvpUU1f5 sXvw== X-Gm-Message-State: AKGB3mKhmaU5rF0Er6csNM1u08jzfXIdE1JOrYEB39ytIIl23ikD7mWN 5HKs05UNyG6AYzws+EY1rdnlzUD4 X-Google-Smtp-Source: ACJfBovHh7eJ62LQRkVatVIhd71OBuQMxco3q6tSorZB5xJW19MrFuQU9IZM4LrkrDQJK80ZK7ndHA== X-Received: by 10.99.122.77 with SMTP id j13mr1655676pgn.239.1515132605514; Thu, 04 Jan 2018 22:10:05 -0800 (PST) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o64sm2444086pga.1.2018.01.04.22.10.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Jan 2018 22:10:05 -0800 (PST) From: Jakub Kicinski To: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, daniel@iogearbox.net, davem@davemloft.net Cc: oss-drivers@netronome.com, tehnerd@fb.com, Jakub Kicinski Subject: [PATCH bpf-next 01/16] bpf: add map_alloc_check callback Date: Thu, 4 Jan 2018 22:09:16 -0800 Message-Id: <20180105060931.30815-2-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180105060931.30815-1-jakub.kicinski@netronome.com> References: <20180105060931.30815-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org .map_alloc callbacks contain a number of checks validating user- -provided map attributes against constraints of a particular map type. For offloaded maps we will need to check map attributes without actually allocating any memory on the host. Add a new callback for validating attributes before any memory is allocated. This callback can be selectively implemented by map types for sharing code with offloads, or simply to separate the logical steps of validation and allocation. Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet --- include/linux/bpf.h | 1 + kernel/bpf/syscall.c | 17 +++++++++++++---- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 7810ae57b357..ab8988d5e6c5 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -25,6 +25,7 @@ struct bpf_map; /* map is generic key/value storage optionally accesible by eBPF programs */ struct bpf_map_ops { /* funcs callable from userspace (via syscall) */ + int (*map_alloc_check)(union bpf_attr *attr); struct bpf_map *(*map_alloc)(union bpf_attr *attr); void (*map_release)(struct bpf_map *map, struct file *map_file); void (*map_free)(struct bpf_map *map); diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index ebf0fb23e237..297677c27095 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -96,16 +96,25 @@ static int check_uarg_tail_zero(void __user *uaddr, static struct bpf_map *find_and_alloc_map(union bpf_attr *attr) { + const struct bpf_map_ops *ops; struct bpf_map *map; + int err; - if (attr->map_type >= ARRAY_SIZE(bpf_map_types) || - !bpf_map_types[attr->map_type]) + if (attr->map_type >= ARRAY_SIZE(bpf_map_types)) + return ERR_PTR(-EINVAL); + ops = bpf_map_types[attr->map_type]; + if (!ops) return ERR_PTR(-EINVAL); - map = bpf_map_types[attr->map_type]->map_alloc(attr); + if (ops->map_alloc_check) { + err = ops->map_alloc_check(attr); + if (err) + return ERR_PTR(err); + } + map = ops->map_alloc(attr); if (IS_ERR(map)) return map; - map->ops = bpf_map_types[attr->map_type]; + map->ops = ops; map->map_type = attr->map_type; return map; } From patchwork Fri Jan 5 06:09:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 855915 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="qYIGnGUN"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zCZ5k6tYNz9s4q for ; Fri, 5 Jan 2018 17:10:18 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751346AbeAEGKN (ORCPT ); Fri, 5 Jan 2018 01:10:13 -0500 Received: from mail-pg0-f68.google.com ([74.125.83.68]:39879 "EHLO mail-pg0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751194AbeAEGKH (ORCPT ); Fri, 5 Jan 2018 01:10:07 -0500 Received: by mail-pg0-f68.google.com with SMTP id 81so1665812pgf.6 for ; Thu, 04 Jan 2018 22:10:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=c+rZLVMW6KMaTfkcX/OkvFCITTk7EJMikG/F4QUsPuI=; b=qYIGnGUNn5z6dTTug3nv7WZUTWEonVEhRvmuQhY8xfVA3EK91YI5n9Xu6li2oiWgN2 qcNoyMVTsy0U/9FnPARU+kI6gRlcrLfAir8IxAYW7m66Wg3Cyxp+TaFkGlsjQR66M3/W Lltj6YfVisFrzCnTph4LOTiibBVUot3c5P9iWJmV4Dwkp62Q7R+rpah7/c6IR2m3gX2a VrsJlAdwd1BXI5ULL18fhoJpMFS8iYPm+rtgA47o5G0vcgYRpJP2gNkh5qF7DdZrX8rq Fc4pXUnGIWhCmKBRh7DGqnkErf2xw9rA0nVOvPgZXSHiePzPgt9WNU4JsUXqGzDI2O7r ioNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=c+rZLVMW6KMaTfkcX/OkvFCITTk7EJMikG/F4QUsPuI=; b=X5uDbUPOpRZoiFjIz4GdZPk2BPpV81jQEPttRVPsTJZvNqYxRHhh2MEOXmijrkzBEO Av8elE9zBt4YTWJIrsDNlpWwLjQ0WGYBPI0YneVAsrVFIcb8PuwB4aC07eJAt7EwnBbU rkvFADFgf0Uc18nY9UKxKNt1f4xSrU4dnVvl4PPIAtQ+YTESYLZsN8qnxroDMMEjjkiX AyYVznwwXSrim9y/8g+ULWpT5M1izdVXh9N0FZrepW0eI7PWN4C9bllDzSf659QVhokS LD03A8i5usZ+GI/pQK+8yNxv3xA/kJR/XhAfAZl8F0x20jFaemOO2HCM6m6DNWvB6ZfU ouiA== X-Gm-Message-State: AKGB3mJQhJ+a5wo5gvVxc2sKoHL+1P9ua7Lij8a0+2VKS9oFYkmbNrq8 Ja0QTTgWpqYO/IDhs0ao6eRwGUJD X-Google-Smtp-Source: ACJfBov7mj97jFz+dnQqaGF0pTrJMQh57m1FCCBlnvXc8Eii/5c3pi66oGrpjh4UuO31qfnv19DmOg== X-Received: by 10.99.65.70 with SMTP id o67mr1665234pga.348.1515132606430; Thu, 04 Jan 2018 22:10:06 -0800 (PST) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o64sm2444086pga.1.2018.01.04.22.10.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Jan 2018 22:10:05 -0800 (PST) From: Jakub Kicinski To: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, daniel@iogearbox.net, davem@davemloft.net Cc: oss-drivers@netronome.com, tehnerd@fb.com, Jakub Kicinski Subject: [PATCH bpf-next 02/16] bpf: array: move checks out of alloc function Date: Thu, 4 Jan 2018 22:09:17 -0800 Message-Id: <20180105060931.30815-3-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180105060931.30815-1-jakub.kicinski@netronome.com> References: <20180105060931.30815-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Use the new callback to perform allocation checks for array maps. The fd maps don't need a special allocation callback, they only need a special check callback. Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet --- kernel/bpf/arraymap.c | 40 +++++++++++++++++++++++++++------------- 1 file changed, 27 insertions(+), 13 deletions(-) diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index 7c25426d3cf5..7f9a684e4947 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -49,26 +49,34 @@ static int bpf_array_alloc_percpu(struct bpf_array *array) } /* Called from syscall */ -static struct bpf_map *array_map_alloc(union bpf_attr *attr) +static int array_map_alloc_check(union bpf_attr *attr) { bool percpu = attr->map_type == BPF_MAP_TYPE_PERCPU_ARRAY; int numa_node = bpf_map_attr_numa_node(attr); - struct bpf_array *array; - u64 array_size; - u32 elem_size; /* check sanity of attributes */ if (attr->max_entries == 0 || attr->key_size != 4 || attr->value_size == 0 || attr->map_flags & ~ARRAY_CREATE_FLAG_MASK || (percpu && numa_node != NUMA_NO_NODE)) - return ERR_PTR(-EINVAL); + return -EINVAL; if (attr->value_size > KMALLOC_MAX_SIZE) /* if value_size is bigger, the user space won't be able to * access the elements. */ - return ERR_PTR(-E2BIG); + return -E2BIG; + + return 0; +} + +static struct bpf_map *array_map_alloc(union bpf_attr *attr) +{ + bool percpu = attr->map_type == BPF_MAP_TYPE_PERCPU_ARRAY; + int numa_node = bpf_map_attr_numa_node(attr); + struct bpf_array *array; + u64 array_size; + u32 elem_size; elem_size = round_up(attr->value_size, 8); @@ -296,6 +304,7 @@ static void array_map_free(struct bpf_map *map) } const struct bpf_map_ops array_map_ops = { + .map_alloc_check = array_map_alloc_check, .map_alloc = array_map_alloc, .map_free = array_map_free, .map_get_next_key = array_map_get_next_key, @@ -306,6 +315,7 @@ const struct bpf_map_ops array_map_ops = { }; const struct bpf_map_ops percpu_array_map_ops = { + .map_alloc_check = array_map_alloc_check, .map_alloc = array_map_alloc, .map_free = array_map_free, .map_get_next_key = array_map_get_next_key, @@ -314,12 +324,12 @@ const struct bpf_map_ops percpu_array_map_ops = { .map_delete_elem = array_map_delete_elem, }; -static struct bpf_map *fd_array_map_alloc(union bpf_attr *attr) +static int fd_array_map_alloc_check(union bpf_attr *attr) { /* only file descriptors can be stored in this type of map */ if (attr->value_size != sizeof(u32)) - return ERR_PTR(-EINVAL); - return array_map_alloc(attr); + return -EINVAL; + return array_map_alloc_check(attr); } static void fd_array_map_free(struct bpf_map *map) @@ -443,7 +453,8 @@ void bpf_fd_array_map_clear(struct bpf_map *map) } const struct bpf_map_ops prog_array_map_ops = { - .map_alloc = fd_array_map_alloc, + .map_alloc_check = fd_array_map_alloc_check, + .map_alloc = array_map_alloc, .map_free = fd_array_map_free, .map_get_next_key = array_map_get_next_key, .map_lookup_elem = fd_array_map_lookup_elem, @@ -530,7 +541,8 @@ static void perf_event_fd_array_release(struct bpf_map *map, } const struct bpf_map_ops perf_event_array_map_ops = { - .map_alloc = fd_array_map_alloc, + .map_alloc_check = fd_array_map_alloc_check, + .map_alloc = array_map_alloc, .map_free = fd_array_map_free, .map_get_next_key = array_map_get_next_key, .map_lookup_elem = fd_array_map_lookup_elem, @@ -561,7 +573,8 @@ static void cgroup_fd_array_free(struct bpf_map *map) } const struct bpf_map_ops cgroup_array_map_ops = { - .map_alloc = fd_array_map_alloc, + .map_alloc_check = fd_array_map_alloc_check, + .map_alloc = array_map_alloc, .map_free = cgroup_fd_array_free, .map_get_next_key = array_map_get_next_key, .map_lookup_elem = fd_array_map_lookup_elem, @@ -579,7 +592,7 @@ static struct bpf_map *array_of_map_alloc(union bpf_attr *attr) if (IS_ERR(inner_map_meta)) return inner_map_meta; - map = fd_array_map_alloc(attr); + map = array_map_alloc(attr); if (IS_ERR(map)) { bpf_map_meta_free(inner_map_meta); return map; @@ -636,6 +649,7 @@ static u32 array_of_map_gen_lookup(struct bpf_map *map, } const struct bpf_map_ops array_of_maps_map_ops = { + .map_alloc_check = fd_array_map_alloc_check, .map_alloc = array_of_map_alloc, .map_free = array_of_map_free, .map_get_next_key = array_map_get_next_key, From patchwork Fri Jan 5 06:09:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 855916 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="lHW3eBs8"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zCZ5m3pJsz9s4q for ; Fri, 5 Jan 2018 17:10:20 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751357AbeAEGKO (ORCPT ); Fri, 5 Jan 2018 01:10:14 -0500 Received: from mail-pf0-f196.google.com ([209.85.192.196]:45471 "EHLO mail-pf0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751286AbeAEGKI (ORCPT ); Fri, 5 Jan 2018 01:10:08 -0500 Received: by mail-pf0-f196.google.com with SMTP id u19so1820923pfa.12 for ; Thu, 04 Jan 2018 22:10:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=nxWGW1apzp6+ziYmDn0+IJ35G6QjXgYQNZWsIHy4Fow=; b=lHW3eBs80kSI1/k3MMgjsAfnB2AcS7rhO4CdoDLzonYg9JoQQU0GRTvDZVTYTHAnKu T1t/ubQfQWvFlhDEusS14mPTD52lFQcHEj2GrKjQGDY8H9HMWj85FI9YHG1xgwO+JjtE a96Y9wz+8FzT04BafF1JCt0DG1j12IC429bP8aboj/fP3DVoBXIOuhQTLTP4olhXdCbi QG28f5O9IX/LJvPucSkv2kVFViQy6OyKLVQGlYaeY8RGojRTcaC+4izoDbThY2DkDYqd UluHAoJPCin1BmOx/CTy8HqVwcf6kbkRQaroOMwNEPaaabwCr44QCt+aHdYJ8hO5r9iC viQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=nxWGW1apzp6+ziYmDn0+IJ35G6QjXgYQNZWsIHy4Fow=; b=cCN8dnCigh5TiUfw+ujTXt/YivWF4HPctQvWCAHn1zNg8x30Blx0mJdwXheCVZqPdK 1UAKAeCvgbxFw4S6HHCFf3a7rSurPkipU2beeDrsfGPFR0+907eOCIHHkeiEVrKoXvMZ jpX4zCUgELoH3+JP4T81gTNpKOISo3flF7QNCjr38k1ORxqNh1pGWP7HOswxvF69M3v+ x55HxhEKtq+7zkVO3hspXqvQG1CbSfaLhDxCCjOPAm9P3L2x1XjzlpxvFtQKBw6LD5nU nz/DoVxmGPi/QJcP1PLE3IiMXNkliVlk05jTIzfXjzWUoSaoP+ynwhml4nd+Buf1lOpB QP7g== X-Gm-Message-State: AKGB3mKvWsY43DGi1a8J9qvIXJ0aqgn2mLz5ma7F6A1XId0oDtEamiA3 7hYwnPDsDHCMZMh8Lfw6FCkwzvEy X-Google-Smtp-Source: ACJfBov7i7SLLKXja9sRboMec32c3ko1rdjgN7hL/d3QZ2Ep+AS+YTteFmmx0NbW56Av6muBmXJLxg== X-Received: by 10.99.94.193 with SMTP id s184mr1680165pgb.325.1515132607472; Thu, 04 Jan 2018 22:10:07 -0800 (PST) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o64sm2444086pga.1.2018.01.04.22.10.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Jan 2018 22:10:06 -0800 (PST) From: Jakub Kicinski To: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, daniel@iogearbox.net, davem@davemloft.net Cc: oss-drivers@netronome.com, tehnerd@fb.com, Jakub Kicinski Subject: [PATCH bpf-next 03/16] bpf: hashtab: move attribute validation before allocation Date: Thu, 4 Jan 2018 22:09:18 -0800 Message-Id: <20180105060931.30815-4-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180105060931.30815-1-jakub.kicinski@netronome.com> References: <20180105060931.30815-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Number of attribute checks are currently performed after hashtab is already allocated. Move them to be able to split them out to the check function later on. Checks have to now be performed on the attr union directly instead of the members of bpf_map, since bpf_map will be allocated later. No functional changes. Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet --- kernel/bpf/hashtab.c | 47 +++++++++++++++++++++++------------------------ 1 file changed, 23 insertions(+), 24 deletions(-) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 3905d4bc5b80..b80f42adf068 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -269,6 +269,28 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) if (numa_node != NUMA_NO_NODE && (percpu || percpu_lru)) return ERR_PTR(-EINVAL); + /* check sanity of attributes. + * value_size == 0 may be allowed in the future to use map as a set + */ + if (attr->max_entries == 0 || attr->key_size == 0 || + attr->value_size == 0) + return ERR_PTR(-EINVAL); + + if (attr->key_size > MAX_BPF_STACK) + /* eBPF programs initialize keys on stack, so they cannot be + * larger than max stack size + */ + return ERR_PTR(-E2BIG); + + if (attr->value_size >= KMALLOC_MAX_SIZE - + MAX_BPF_STACK - sizeof(struct htab_elem)) + /* if value_size is bigger, the user space won't be able to + * access the elements via bpf syscall. This check also makes + * sure that the elem_size doesn't overflow and it's + * kmalloc-able later in htab_map_update_elem() + */ + return ERR_PTR(-E2BIG); + htab = kzalloc(sizeof(*htab), GFP_USER); if (!htab) return ERR_PTR(-ENOMEM); @@ -281,14 +303,6 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) htab->map.map_flags = attr->map_flags; htab->map.numa_node = numa_node; - /* check sanity of attributes. - * value_size == 0 may be allowed in the future to use map as a set - */ - err = -EINVAL; - if (htab->map.max_entries == 0 || htab->map.key_size == 0 || - htab->map.value_size == 0) - goto free_htab; - if (percpu_lru) { /* ensure each CPU's lru list has >=1 elements. * since we are at it, make each lru list has the same @@ -304,22 +318,6 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) /* hash table size must be power of 2 */ htab->n_buckets = roundup_pow_of_two(htab->map.max_entries); - err = -E2BIG; - if (htab->map.key_size > MAX_BPF_STACK) - /* eBPF programs initialize keys on stack, so they cannot be - * larger than max stack size - */ - goto free_htab; - - if (htab->map.value_size >= KMALLOC_MAX_SIZE - - MAX_BPF_STACK - sizeof(struct htab_elem)) - /* if value_size is bigger, the user space won't be able to - * access the elements via bpf syscall. This check also makes - * sure that the elem_size doesn't overflow and it's - * kmalloc-able later in htab_map_update_elem() - */ - goto free_htab; - htab->elem_size = sizeof(struct htab_elem) + round_up(htab->map.key_size, 8); if (percpu) @@ -327,6 +325,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) else htab->elem_size += round_up(htab->map.value_size, 8); + err = -E2BIG; /* prevent zero size kmalloc and check for u32 overflow */ if (htab->n_buckets == 0 || htab->n_buckets > U32_MAX / sizeof(struct bucket)) From patchwork Fri Jan 5 06:09:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 855925 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="Bh0MkBN2"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zCZ6h2wGmz9s4q for ; Fri, 5 Jan 2018 17:11:08 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751368AbeAEGKQ (ORCPT ); Fri, 5 Jan 2018 01:10:16 -0500 Received: from mail-pf0-f195.google.com ([209.85.192.195]:41176 "EHLO mail-pf0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751289AbeAEGKJ (ORCPT ); Fri, 5 Jan 2018 01:10:09 -0500 Received: by mail-pf0-f195.google.com with SMTP id j28so1827644pfk.8 for ; Thu, 04 Jan 2018 22:10:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=K15AjhoT4E+QWWyW+Bv/emEQvDUxMSlX4I59mYR70bA=; b=Bh0MkBN2bgJfGzKpsRgItenCCm81pL0chnntcH01F5NH/laeDNy0KcQG+FSEbIAuI9 VMFHMk+7x8v2ZU74xhVzmQL4JN7t9d27lzm62rH/b19vW95XK50/oM2OQrtcCM5FgeZD 7jLoEM1dkSQtEyLxlDQaSbXjkqhsjahCBB0IJ4Bh6/IJlqxObphJ8ySkJtFK87nOzex2 7BGn7cVNKwiBNFKmkM4+EP70SDU/hvM5K24GeAEOs9bi7W7Zm1XP9Lqx0A4uV5M5OkkL 6O5pmNWiCWC1pWTdd4WFtppMW3r0rAcSD5Lvx1CpHgIxTxCvF/sSvzJLy2AB/az0dIxc mAww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=K15AjhoT4E+QWWyW+Bv/emEQvDUxMSlX4I59mYR70bA=; b=IIuUCgpMoidqiQ39dtQUPcanMUQKebZTksZyhgsi/es5hBVpKTddjJuCGaBeEHrokR FAzC/wVW0Ify7t9ryGpbVVojWuLjjd/dFsD9K7Ozq8bnd3Lj7+MEpYPIQ4Fw8dFfGlIx 9oNhv0Sfe++5yq7Rkphinj4hL65RlRDNQ0XuJ7Pn6O+nNM2k27VB7BOnKj48LjCYPxqn WEkKblNiYQT7Ln0Jw0AhWLYIlAxX8NBUjK6rOXWe+fIQYfQ8q92Aer4C7mZJwD/C0Vak TJlCcblyJS9qmB1lzSl9lAwTyH/1K6jDlUf9rn+sXBgMaAJ+41ybiRJysxEJ4T7LOQLs ugJQ== X-Gm-Message-State: AKGB3mLxfsI5aiEeHnaeSiOM2k/U9Adk/BzpBSJj2RAmD7k7utlaZd2U XE8RUuPJCGrvKRs7sg8jcvei9Tn3 X-Google-Smtp-Source: ACJfBot2DsmnkJo57pVv0gWStFQr0YTE8uvVkYGa5Kt39OfWem0sKxHKsfMviKbkNrePO3oQwYigVg== X-Received: by 10.101.77.133 with SMTP id p5mr1678287pgq.106.1515132608305; Thu, 04 Jan 2018 22:10:08 -0800 (PST) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o64sm2444086pga.1.2018.01.04.22.10.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Jan 2018 22:10:07 -0800 (PST) From: Jakub Kicinski To: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, daniel@iogearbox.net, davem@davemloft.net Cc: oss-drivers@netronome.com, tehnerd@fb.com, Jakub Kicinski Subject: [PATCH bpf-next 04/16] bpf: hashtab: move checks out of alloc function Date: Thu, 4 Jan 2018 22:09:19 -0800 Message-Id: <20180105060931.30815-5-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180105060931.30815-1-jakub.kicinski@netronome.com> References: <20180105060931.30815-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Use the new callback to perform allocation checks for hash maps. Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet --- kernel/bpf/hashtab.c | 55 +++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 39 insertions(+), 16 deletions(-) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index b80f42adf068..7fd6519444d3 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -227,7 +227,7 @@ static int alloc_extra_elems(struct bpf_htab *htab) } /* Called from syscall */ -static struct bpf_map *htab_map_alloc(union bpf_attr *attr) +static int htab_map_alloc_check(union bpf_attr *attr) { bool percpu = (attr->map_type == BPF_MAP_TYPE_PERCPU_HASH || attr->map_type == BPF_MAP_TYPE_LRU_PERCPU_HASH); @@ -241,9 +241,6 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) bool percpu_lru = (attr->map_flags & BPF_F_NO_COMMON_LRU); bool prealloc = !(attr->map_flags & BPF_F_NO_PREALLOC); int numa_node = bpf_map_attr_numa_node(attr); - struct bpf_htab *htab; - int err, i; - u64 cost; BUILD_BUG_ON(offsetof(struct htab_elem, htab) != offsetof(struct htab_elem, hash_node.pprev)); @@ -254,33 +251,33 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) /* LRU implementation is much complicated than other * maps. Hence, limit to CAP_SYS_ADMIN for now. */ - return ERR_PTR(-EPERM); + return -EPERM; if (attr->map_flags & ~HTAB_CREATE_FLAG_MASK) /* reserved bits should not be used */ - return ERR_PTR(-EINVAL); + return -EINVAL; if (!lru && percpu_lru) - return ERR_PTR(-EINVAL); + return -EINVAL; if (lru && !prealloc) - return ERR_PTR(-ENOTSUPP); + return -ENOTSUPP; if (numa_node != NUMA_NO_NODE && (percpu || percpu_lru)) - return ERR_PTR(-EINVAL); + return -EINVAL; /* check sanity of attributes. * value_size == 0 may be allowed in the future to use map as a set */ if (attr->max_entries == 0 || attr->key_size == 0 || attr->value_size == 0) - return ERR_PTR(-EINVAL); + return -EINVAL; if (attr->key_size > MAX_BPF_STACK) /* eBPF programs initialize keys on stack, so they cannot be * larger than max stack size */ - return ERR_PTR(-E2BIG); + return -E2BIG; if (attr->value_size >= KMALLOC_MAX_SIZE - MAX_BPF_STACK - sizeof(struct htab_elem)) @@ -289,7 +286,28 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) * sure that the elem_size doesn't overflow and it's * kmalloc-able later in htab_map_update_elem() */ - return ERR_PTR(-E2BIG); + return -E2BIG; + + return 0; +} + +static struct bpf_map *htab_map_alloc(union bpf_attr *attr) +{ + bool percpu = (attr->map_type == BPF_MAP_TYPE_PERCPU_HASH || + attr->map_type == BPF_MAP_TYPE_LRU_PERCPU_HASH); + bool lru = (attr->map_type == BPF_MAP_TYPE_LRU_HASH || + attr->map_type == BPF_MAP_TYPE_LRU_PERCPU_HASH); + /* percpu_lru means each cpu has its own LRU list. + * it is different from BPF_MAP_TYPE_PERCPU_HASH where + * the map's value itself is percpu. percpu_lru has + * nothing to do with the map's value. + */ + bool percpu_lru = (attr->map_flags & BPF_F_NO_COMMON_LRU); + bool prealloc = !(attr->map_flags & BPF_F_NO_PREALLOC); + int numa_node = bpf_map_attr_numa_node(attr); + struct bpf_htab *htab; + int err, i; + u64 cost; htab = kzalloc(sizeof(*htab), GFP_USER); if (!htab) @@ -1142,6 +1160,7 @@ static void htab_map_free(struct bpf_map *map) } const struct bpf_map_ops htab_map_ops = { + .map_alloc_check = htab_map_alloc_check, .map_alloc = htab_map_alloc, .map_free = htab_map_free, .map_get_next_key = htab_map_get_next_key, @@ -1152,6 +1171,7 @@ const struct bpf_map_ops htab_map_ops = { }; const struct bpf_map_ops htab_lru_map_ops = { + .map_alloc_check = htab_map_alloc_check, .map_alloc = htab_map_alloc, .map_free = htab_map_free, .map_get_next_key = htab_map_get_next_key, @@ -1235,6 +1255,7 @@ int bpf_percpu_hash_update(struct bpf_map *map, void *key, void *value, } const struct bpf_map_ops htab_percpu_map_ops = { + .map_alloc_check = htab_map_alloc_check, .map_alloc = htab_map_alloc, .map_free = htab_map_free, .map_get_next_key = htab_map_get_next_key, @@ -1244,6 +1265,7 @@ const struct bpf_map_ops htab_percpu_map_ops = { }; const struct bpf_map_ops htab_lru_percpu_map_ops = { + .map_alloc_check = htab_map_alloc_check, .map_alloc = htab_map_alloc, .map_free = htab_map_free, .map_get_next_key = htab_map_get_next_key, @@ -1252,11 +1274,11 @@ const struct bpf_map_ops htab_lru_percpu_map_ops = { .map_delete_elem = htab_lru_map_delete_elem, }; -static struct bpf_map *fd_htab_map_alloc(union bpf_attr *attr) +static int fd_htab_map_alloc_check(union bpf_attr *attr) { if (attr->value_size != sizeof(u32)) - return ERR_PTR(-EINVAL); - return htab_map_alloc(attr); + return -EINVAL; + return htab_map_alloc_check(attr); } static void fd_htab_map_free(struct bpf_map *map) @@ -1327,7 +1349,7 @@ static struct bpf_map *htab_of_map_alloc(union bpf_attr *attr) if (IS_ERR(inner_map_meta)) return inner_map_meta; - map = fd_htab_map_alloc(attr); + map = htab_map_alloc(attr); if (IS_ERR(map)) { bpf_map_meta_free(inner_map_meta); return map; @@ -1371,6 +1393,7 @@ static void htab_of_map_free(struct bpf_map *map) } const struct bpf_map_ops htab_of_maps_map_ops = { + .map_alloc_check = fd_htab_map_alloc_check, .map_alloc = htab_of_map_alloc, .map_free = htab_of_map_free, .map_get_next_key = htab_map_get_next_key, From patchwork Fri Jan 5 06:09:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 855917 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="kxM9DHiK"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zCZ5n3wm0z9t2Q for ; Fri, 5 Jan 2018 17:10:21 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751289AbeAEGKR (ORCPT ); Fri, 5 Jan 2018 01:10:17 -0500 Received: from mail-pf0-f196.google.com ([209.85.192.196]:44110 "EHLO mail-pf0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751298AbeAEGKK (ORCPT ); Fri, 5 Jan 2018 01:10:10 -0500 Received: by mail-pf0-f196.google.com with SMTP id m26so1821839pfj.11 for ; Thu, 04 Jan 2018 22:10:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=wY9y38HleA7ZrsONkdblBE7o/Q09FNcam8jVAHShCqw=; b=kxM9DHiKFDZoCWvijZWf/3OHaPK8juJd0tRE/bPehdBzv1fgRilWVNCI/qQJ7pdPtl mMxFmHjrdY2YwDEccRKvgh+j40u1B48aG9DiSSm00SUoDKfE8N65h+FdUhDWD73V7RSE 7JsVHIY/vDHrGikfYfT73r1GlVS+Zu8NCdrj0i4Dk7XklBlXha5b+Nbtv1QTb4SbIXsv cvPPxRpIYPQBxcC7QTpyCLxQ1ak+/GfnPJE2KS6I+BvtUsV9dDhV76OW7xYWcxQRb7Sd iIPV7R964ZxXoomCmBCjG45c4zJ9ST6UcCv/jmBiGpuzWBjMzB3kKc9zhvdhwomdyYtn jhSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=wY9y38HleA7ZrsONkdblBE7o/Q09FNcam8jVAHShCqw=; b=oBT6x1NnhqlNHh/vachQVa2WVXq1M3IWisjWsjs2UoDktoWXp7Xphw+dcojelozp9e KC8nvlUiVVnF0lBjM+nOfRqTxJEgEs+KLq+RTZO4kfhwFgwlRXuoAj4tkyGx6V+oPKmp +aR+pV37rbGNTX40jqsqXsROh0VEPrAAArcrw9Y8tW3UdALnm1GN3G/P1yGOV/SY5B1u ShsDR4kwcH6Fb3l3YZ5Se/X98GQvMuZ47U8ZDQK67jju04cnXXbh6/Nv7X7PVcE1A6Xj 0JeV7vseA0Ju59WxbjlNkky8v1R8/TXkpgJXJTzQkvfyPbrA3zGnjGNxS2xFFbrl2vIO TBfw== X-Gm-Message-State: AKGB3mK3YWR5WmdZyNeJM2ssSyG0t6Dt++1FQBfxLdOISN+DfIjpvE9R IKItaCHl/pN3ZOxkVFsax7rM5F7c X-Google-Smtp-Source: ACJfBovrqNv7I277QL/n6D+NKiCFxbo4ZTsBWLyRQ8BTvkbILzCUlCGr/DRE2VbVzCkC/j0bxlNvZg== X-Received: by 10.99.94.69 with SMTP id s66mr1644346pgb.190.1515132609274; Thu, 04 Jan 2018 22:10:09 -0800 (PST) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o64sm2444086pga.1.2018.01.04.22.10.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Jan 2018 22:10:08 -0800 (PST) From: Jakub Kicinski To: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, daniel@iogearbox.net, davem@davemloft.net Cc: oss-drivers@netronome.com, tehnerd@fb.com, Jakub Kicinski Subject: [PATCH bpf-next 05/16] bpf: add helper for copying attrs to struct bpf_map Date: Thu, 4 Jan 2018 22:09:20 -0800 Message-Id: <20180105060931.30815-6-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180105060931.30815-1-jakub.kicinski@netronome.com> References: <20180105060931.30815-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org All map types reimplement the field-by-field copy of union bpf_attr members into struct bpf_map. Add a helper to perform this operation. Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet Acked-by: Alexei Starovoitov --- include/linux/bpf.h | 1 + kernel/bpf/arraymap.c | 7 +------ kernel/bpf/cpumap.c | 8 +------- kernel/bpf/devmap.c | 8 +------- kernel/bpf/hashtab.c | 9 +-------- kernel/bpf/lpm_trie.c | 7 +------ kernel/bpf/sockmap.c | 8 +------- kernel/bpf/stackmap.c | 6 +----- kernel/bpf/syscall.c | 10 ++++++++++ 9 files changed, 18 insertions(+), 46 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index ab8988d5e6c5..2146520e5214 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -368,6 +368,7 @@ void bpf_map_put(struct bpf_map *map); int bpf_map_precharge_memlock(u32 pages); void *bpf_map_area_alloc(size_t size, int numa_node); void bpf_map_area_free(void *base); +void bpf_map_init_from_attr(struct bpf_map *map, union bpf_attr *attr); extern int sysctl_unprivileged_bpf_disabled; diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index 7f9a684e4947..e25984885d27 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -96,12 +96,7 @@ static struct bpf_map *array_map_alloc(union bpf_attr *attr) return ERR_PTR(-ENOMEM); /* copy mandatory map attributes */ - array->map.map_type = attr->map_type; - array->map.key_size = attr->key_size; - array->map.value_size = attr->value_size; - array->map.max_entries = attr->max_entries; - array->map.map_flags = attr->map_flags; - array->map.numa_node = numa_node; + bpf_map_init_from_attr(&array->map, attr); array->elem_size = elem_size; if (!percpu) diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index ce5b669003b2..192151ec9d12 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -94,13 +94,7 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) if (!cmap) return ERR_PTR(-ENOMEM); - /* mandatory map attributes */ - cmap->map.map_type = attr->map_type; - cmap->map.key_size = attr->key_size; - cmap->map.value_size = attr->value_size; - cmap->map.max_entries = attr->max_entries; - cmap->map.map_flags = attr->map_flags; - cmap->map.numa_node = bpf_map_attr_numa_node(attr); + bpf_map_init_from_attr(&cmap->map, attr); /* Pre-limit array size based on NR_CPUS, not final CPU check */ if (cmap->map.max_entries > NR_CPUS) { diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index ebdef54bf7df..565f9ece9115 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -93,13 +93,7 @@ static struct bpf_map *dev_map_alloc(union bpf_attr *attr) if (!dtab) return ERR_PTR(-ENOMEM); - /* mandatory map attributes */ - dtab->map.map_type = attr->map_type; - dtab->map.key_size = attr->key_size; - dtab->map.value_size = attr->value_size; - dtab->map.max_entries = attr->max_entries; - dtab->map.map_flags = attr->map_flags; - dtab->map.numa_node = bpf_map_attr_numa_node(attr); + bpf_map_init_from_attr(&dtab->map, attr); /* make sure page count doesn't overflow */ cost = (u64) dtab->map.max_entries * sizeof(struct bpf_dtab_netdev *); diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 7fd6519444d3..b76828f23b49 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -304,7 +304,6 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) */ bool percpu_lru = (attr->map_flags & BPF_F_NO_COMMON_LRU); bool prealloc = !(attr->map_flags & BPF_F_NO_PREALLOC); - int numa_node = bpf_map_attr_numa_node(attr); struct bpf_htab *htab; int err, i; u64 cost; @@ -313,13 +312,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) if (!htab) return ERR_PTR(-ENOMEM); - /* mandatory map attributes */ - htab->map.map_type = attr->map_type; - htab->map.key_size = attr->key_size; - htab->map.value_size = attr->value_size; - htab->map.max_entries = attr->max_entries; - htab->map.map_flags = attr->map_flags; - htab->map.numa_node = numa_node; + bpf_map_init_from_attr(&htab->map, attr); if (percpu_lru) { /* ensure each CPU's lru list has >=1 elements. diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c index 885e45479680..584e02227671 100644 --- a/kernel/bpf/lpm_trie.c +++ b/kernel/bpf/lpm_trie.c @@ -522,12 +522,7 @@ static struct bpf_map *trie_alloc(union bpf_attr *attr) return ERR_PTR(-ENOMEM); /* copy mandatory map attributes */ - trie->map.map_type = attr->map_type; - trie->map.key_size = attr->key_size; - trie->map.value_size = attr->value_size; - trie->map.max_entries = attr->max_entries; - trie->map.map_flags = attr->map_flags; - trie->map.numa_node = bpf_map_attr_numa_node(attr); + bpf_map_init_from_attr(&trie->map, attr); trie->data_size = attr->key_size - offsetof(struct bpf_lpm_trie_key, data); trie->max_prefixlen = trie->data_size * 8; diff --git a/kernel/bpf/sockmap.c b/kernel/bpf/sockmap.c index 5ee2e41893d9..2ccd4b1a8683 100644 --- a/kernel/bpf/sockmap.c +++ b/kernel/bpf/sockmap.c @@ -521,13 +521,7 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr) if (!stab) return ERR_PTR(-ENOMEM); - /* mandatory map attributes */ - stab->map.map_type = attr->map_type; - stab->map.key_size = attr->key_size; - stab->map.value_size = attr->value_size; - stab->map.max_entries = attr->max_entries; - stab->map.map_flags = attr->map_flags; - stab->map.numa_node = bpf_map_attr_numa_node(attr); + bpf_map_init_from_attr(&stab->map, attr); /* make sure page count doesn't overflow */ cost = (u64) stab->map.max_entries * sizeof(struct sock *); diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c index a15bc636cc98..6039bf1aa945 100644 --- a/kernel/bpf/stackmap.c +++ b/kernel/bpf/stackmap.c @@ -88,14 +88,10 @@ static struct bpf_map *stack_map_alloc(union bpf_attr *attr) if (cost >= U32_MAX - PAGE_SIZE) goto free_smap; - smap->map.map_type = attr->map_type; - smap->map.key_size = attr->key_size; + bpf_map_init_from_attr(&smap->map, attr); smap->map.value_size = value_size; - smap->map.max_entries = attr->max_entries; - smap->map.map_flags = attr->map_flags; smap->n_buckets = n_buckets; smap->map.pages = round_up(cost, PAGE_SIZE) >> PAGE_SHIFT; - smap->map.numa_node = bpf_map_attr_numa_node(attr); err = bpf_map_precharge_memlock(smap->map.pages); if (err) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 297677c27095..14adf29fc0f2 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -143,6 +143,16 @@ void bpf_map_area_free(void *area) kvfree(area); } +void bpf_map_init_from_attr(struct bpf_map *map, union bpf_attr *attr) +{ + map->map_type = attr->map_type; + map->key_size = attr->key_size; + map->value_size = attr->value_size; + map->max_entries = attr->max_entries; + map->map_flags = attr->map_flags; + map->numa_node = bpf_map_attr_numa_node(attr); +} + int bpf_map_precharge_memlock(u32 pages) { struct user_struct *user = get_current_user(); From patchwork Fri Jan 5 06:09:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 855918 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="IylR9U7F"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zCZ5p60Z7z9s4q for ; Fri, 5 Jan 2018 17:10:22 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751397AbeAEGKT (ORCPT ); Fri, 5 Jan 2018 01:10:19 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:41351 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751302AbeAEGKK (ORCPT ); Fri, 5 Jan 2018 01:10:10 -0500 Received: by mail-pl0-f68.google.com with SMTP id g2so2543153pli.8 for ; Thu, 04 Jan 2018 22:10:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=qXzVmLGnc24RnAhpuIUfutOXfOaFhxwwMfgjR1IckwE=; b=IylR9U7Fawu3LrFnLOZZPS5sSBTBZI+MTfzKmECoe3HZqhBzj7L/GvXKZs2Wkz9ipB sxawnsMR4es6FIxbzm1Vu27vPqHKdZAmN9hSDK6q+Lc5w5z7fCBZLfYRrOy5Loi4rdSu W61fbQUTWPy1kBSjk9XGFEsm4Oq7tThNi81GQ18nL93lFT5pWN9Tc7n7/ewiUUJBE8+g jXRUVfDt+DOEJz0J/VRaBSzP5oid6n5Toj/WsKAY6STgYQ+boZqeJlm/laJhyZcq8db1 axZ8cc7Ty39WPV9w/t68BiajIBYY8ykd0CNLlpoOE8t8EutBfl3cwNbXvjHyWXe3cpTw 85tQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=qXzVmLGnc24RnAhpuIUfutOXfOaFhxwwMfgjR1IckwE=; b=OcW4z/PBn/IZV0SaJ+Cg6DoisiTIzGDiK2bP1MyFhVz51u6mpA1HcQj7RdeNV0p/tQ Qi9VZWFVdzYKuPW6v0ow5ztu1cuJizupLA/KRHSDYtcc9hBvKkcEQyR4sR3y8viWMFk+ 7xu2JdfRM1xrg9ozJ/nJTYOREg2aWGgVR3gYg+OJ+7YZTUD75eKLYw47/vO9tDOQtyal ZwO5P4dRukSllijKNh0cXAfZaXZZBfeixAQGqzkh8LnIiEmkLM+Xco8pGTiUquiVQPaq acuD0l5boHkiPc9bJgkGsFY8WvovqLAqn74T0mH3JPsXK5hnz8m9iHk22v1bHSLTaf5o rKxQ== X-Gm-Message-State: AKGB3mIfHjZZHPWrZK4QoPz6XKyWhNmqJk2LI0EQCryc7QHW+8UAe15O DNntFsjmmDKeHk48GKHO5wsrM7ml X-Google-Smtp-Source: ACJfBovqCz/8BX2IGvy5JDPHqC+qcSVhGifNQZp7dOPv44E6jiphulsir6WvSfHWVeFGk2kjj0NK9Q== X-Received: by 10.84.253.144 with SMTP id a16mr2063270plm.100.1515132610120; Thu, 04 Jan 2018 22:10:10 -0800 (PST) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o64sm2444086pga.1.2018.01.04.22.10.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Jan 2018 22:10:09 -0800 (PST) From: Jakub Kicinski To: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, daniel@iogearbox.net, davem@davemloft.net Cc: oss-drivers@netronome.com, tehnerd@fb.com, Jakub Kicinski Subject: [PATCH bpf-next 06/16] bpf: rename bpf_dev_offload -> bpf_prog_offload Date: Thu, 4 Jan 2018 22:09:21 -0800 Message-Id: <20180105060931.30815-7-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180105060931.30815-1-jakub.kicinski@netronome.com> References: <20180105060931.30815-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org With map offload coming, we need to call program offload structure something less ambiguous. Pure rename, no functional changes. Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet --- drivers/net/ethernet/netronome/nfp/bpf/offload.c | 2 +- include/linux/bpf.h | 4 ++-- kernel/bpf/offload.c | 10 +++++----- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/bpf/offload.c b/drivers/net/ethernet/netronome/nfp/bpf/offload.c index fa2905e67b07..1a67673261b7 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/offload.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/offload.c @@ -234,7 +234,7 @@ int nfp_net_bpf_offload(struct nfp_net *nn, struct bpf_prog *prog, int err; if (prog) { - struct bpf_dev_offload *offload = prog->aux->offload; + struct bpf_prog_offload *offload = prog->aux->offload; if (!offload) return -EINVAL; diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 2146520e5214..425ca3cd1e32 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -191,7 +191,7 @@ struct bpf_prog_offload_ops { int insn_idx, int prev_insn_idx); }; -struct bpf_dev_offload { +struct bpf_prog_offload { struct bpf_prog *prog; struct net_device *netdev; void *dev_priv; @@ -221,7 +221,7 @@ struct bpf_prog_aux { #ifdef CONFIG_SECURITY void *security; #endif - struct bpf_dev_offload *offload; + struct bpf_prog_offload *offload; union { struct work_struct work; struct rcu_head rcu; diff --git a/kernel/bpf/offload.c b/kernel/bpf/offload.c index 040d4e0edf3f..001ddfde7874 100644 --- a/kernel/bpf/offload.c +++ b/kernel/bpf/offload.c @@ -32,7 +32,7 @@ static LIST_HEAD(bpf_prog_offload_devs); int bpf_prog_offload_init(struct bpf_prog *prog, union bpf_attr *attr) { - struct bpf_dev_offload *offload; + struct bpf_prog_offload *offload; if (attr->prog_type != BPF_PROG_TYPE_SCHED_CLS && attr->prog_type != BPF_PROG_TYPE_XDP) @@ -72,7 +72,7 @@ int bpf_prog_offload_init(struct bpf_prog *prog, union bpf_attr *attr) static int __bpf_offload_ndo(struct bpf_prog *prog, enum bpf_netdev_command cmd, struct netdev_bpf *data) { - struct bpf_dev_offload *offload = prog->aux->offload; + struct bpf_prog_offload *offload = prog->aux->offload; struct net_device *netdev; ASSERT_RTNL(); @@ -110,7 +110,7 @@ int bpf_prog_offload_verifier_prep(struct bpf_verifier_env *env) int bpf_prog_offload_verify_insn(struct bpf_verifier_env *env, int insn_idx, int prev_insn_idx) { - struct bpf_dev_offload *offload; + struct bpf_prog_offload *offload; int ret = -ENODEV; down_read(&bpf_devs_lock); @@ -124,7 +124,7 @@ int bpf_prog_offload_verify_insn(struct bpf_verifier_env *env, static void __bpf_prog_offload_destroy(struct bpf_prog *prog) { - struct bpf_dev_offload *offload = prog->aux->offload; + struct bpf_prog_offload *offload = prog->aux->offload; struct netdev_bpf data = {}; data.offload.prog = prog; @@ -242,7 +242,7 @@ static int bpf_offload_notification(struct notifier_block *notifier, ulong event, void *ptr) { struct net_device *netdev = netdev_notifier_info_to_dev(ptr); - struct bpf_dev_offload *offload, *tmp; + struct bpf_prog_offload *offload, *tmp; ASSERT_RTNL(); From patchwork Fri Jan 5 06:09:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 855930 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="wudDfGri"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zCZ7B3Fp9z9s75 for ; Fri, 5 Jan 2018 17:11:34 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751503AbeAEGLb (ORCPT ); Fri, 5 Jan 2018 01:11:31 -0500 Received: from mail-pl0-f67.google.com ([209.85.160.67]:38406 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751323AbeAEGKL (ORCPT ); Fri, 5 Jan 2018 01:10:11 -0500 Received: by mail-pl0-f67.google.com with SMTP id s10so2545440plj.5 for ; Thu, 04 Jan 2018 22:10:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=zGHHDDAVegvngvePbSX4ZbG1R7HCKtnFykW5EKppWrQ=; b=wudDfGriHt9X//GF5RtNNtyiXuwdrvIoapD2aX7PQb7+Uv7xATVw2Hs4h/9a1lKKNG XQEhXVn0a2p54TY03l2u8JGITmnwche/wOQ7zeOEDW7lY3+jYE3uGSAq2XgQOQDbz5qd Q1+zLvkscsqJXGK7wx2kAaIXxYF0BCRAGvuC+WsQ00oVS08MAcL//CrICUw/WEj86o+G v7CkvXfI3fjL/SFnFmw2i2s8BP1Yog0B1xfWa87sAvVFwjCULLgKrU7wZIfPgMAh0/W4 kQRp5FmERGRdBy6z5ydVeZX0GLvyzcVSOj+pKoevRxqsU//t6Vg9sRuGorezCnoVj3Ls O+Bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=zGHHDDAVegvngvePbSX4ZbG1R7HCKtnFykW5EKppWrQ=; b=dVQS+P6gay5Bc21ubRknHoRiXje2KXYBSb6WRZEXgTpLCeb6nCG3plej0aD3WdY2gX xXROiaphKRNk6dZA4SDWbbtIEOuiWpPtemXvlIFKOzbW5R+Z92dlgQN00hkBdG9io8NC vuaqDDyM6uOpJjkeCeiV8B9bwCE8NSRoAvF5DHVdtAQ4hzjmySJ5IVIL/lXilknk44Ck 1XrS+OuHe/io0IMEJCZRx7mR3VxneJQFh8VmWzNgBNvsfWVRnolOMo3QFQsQrdGFbfzL KhTfUL2I0dEsZJfj949d/LX+eBsd4troywtAgTshLiaSljZ0QfM6+x7M2NLlWSsZysYe xVYw== X-Gm-Message-State: AKGB3mIF9cV60D0L2fbWT2JINIFSDQchh3ShR55dMpWFkHlQtwz9z131 6T77UZUOc33USsk9wF8P0Lm/9H6b X-Google-Smtp-Source: ACJfBov12CqEB3aWkE1tbqlOYc07h50ZaueKUNBWPRTjCRVHYs8OH7aCA9asJu01G8TptaZFKRx+aw== X-Received: by 10.84.132.100 with SMTP id 91mr2001058ple.88.1515132610988; Thu, 04 Jan 2018 22:10:10 -0800 (PST) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o64sm2444086pga.1.2018.01.04.22.10.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Jan 2018 22:10:10 -0800 (PST) From: Jakub Kicinski To: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, daniel@iogearbox.net, davem@davemloft.net Cc: oss-drivers@netronome.com, tehnerd@fb.com, Jakub Kicinski Subject: [PATCH bpf-next 07/16] bpf: offload: factor out netdev checking at allocation time Date: Thu, 4 Jan 2018 22:09:22 -0800 Message-Id: <20180105060931.30815-8-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180105060931.30815-1-jakub.kicinski@netronome.com> References: <20180105060931.30815-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add a helper to check if netdev could be found and whether it has .ndo_bpf callback. There is no need to check the callback every time it's invoked, ndos can't reasonably be swapped for a set without .ndp_bpf while program is loaded. bpf_dev_offload_check() will also be used by map offload. Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet --- kernel/bpf/offload.c | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/kernel/bpf/offload.c b/kernel/bpf/offload.c index 001ddfde7874..cdd1e19a668b 100644 --- a/kernel/bpf/offload.c +++ b/kernel/bpf/offload.c @@ -30,9 +30,19 @@ static DECLARE_RWSEM(bpf_devs_lock); static LIST_HEAD(bpf_prog_offload_devs); +static int bpf_dev_offload_check(struct net_device *netdev) +{ + if (!netdev) + return -EINVAL; + if (!netdev->netdev_ops->ndo_bpf) + return -EOPNOTSUPP; + return 0; +} + int bpf_prog_offload_init(struct bpf_prog *prog, union bpf_attr *attr) { struct bpf_prog_offload *offload; + int err; if (attr->prog_type != BPF_PROG_TYPE_SCHED_CLS && attr->prog_type != BPF_PROG_TYPE_XDP) @@ -49,12 +59,15 @@ int bpf_prog_offload_init(struct bpf_prog *prog, union bpf_attr *attr) offload->netdev = dev_get_by_index(current->nsproxy->net_ns, attr->prog_ifindex); - if (!offload->netdev) - goto err_free; + err = bpf_dev_offload_check(offload->netdev); + if (err) + goto err_maybe_put; down_write(&bpf_devs_lock); - if (offload->netdev->reg_state != NETREG_REGISTERED) + if (offload->netdev->reg_state != NETREG_REGISTERED) { + err = -EINVAL; goto err_unlock; + } prog->aux->offload = offload; list_add_tail(&offload->offloads, &bpf_prog_offload_devs); dev_put(offload->netdev); @@ -63,10 +76,11 @@ int bpf_prog_offload_init(struct bpf_prog *prog, union bpf_attr *attr) return 0; err_unlock: up_write(&bpf_devs_lock); - dev_put(offload->netdev); -err_free: +err_maybe_put: + if (offload->netdev) + dev_put(offload->netdev); kfree(offload); - return -EINVAL; + return err; } static int __bpf_offload_ndo(struct bpf_prog *prog, enum bpf_netdev_command cmd, @@ -80,8 +94,6 @@ static int __bpf_offload_ndo(struct bpf_prog *prog, enum bpf_netdev_command cmd, if (!offload) return -ENODEV; netdev = offload->netdev; - if (!netdev->netdev_ops->ndo_bpf) - return -EOPNOTSUPP; data->command = cmd; From patchwork Fri Jan 5 06:09:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 855927 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="ld/Ceho/"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zCZ6x2jfCz9s75 for ; Fri, 5 Jan 2018 17:11:21 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751498AbeAEGLT (ORCPT ); Fri, 5 Jan 2018 01:11:19 -0500 Received: from mail-pl0-f67.google.com ([209.85.160.67]:40123 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751336AbeAEGKN (ORCPT ); Fri, 5 Jan 2018 01:10:13 -0500 Received: by mail-pl0-f67.google.com with SMTP id 62so2545311pld.7 for ; Thu, 04 Jan 2018 22:10:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=qU0K5dynrVZzYJhVARytVGNqWH9GT9yrYWc/It/JJZE=; b=ld/Ceho/99QvU6MHwUwt2mQtf3n0/Ya8oDVIYtXMfGJ87qVa75NlXPnkboul78yNcp zgO3F8kl9lPXuUikha2Kx8dGZuIdFqQ7n1ua7XAKx4it3mxOpEvcJ35TWJCRM2qs+nB1 tMrAp9wXKzo5k/+UF1rAm7lKL2+hkHLrtHBE8rNHslN/YmOBRwwg7l5QpnIlLBdyzE1I G8t4+zOSywmM7x0xvjZe+1jaoobdUw/usiuwsirNZ6EDcOH3LofnXn12W+TsYCnkXPj+ 8m0KXtz0toG8MUxbkoyXu0sG9y37pwJCJ42gTHtwyRu1sp6ki08xGaXCUidyZ6GfiT7D VmzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=qU0K5dynrVZzYJhVARytVGNqWH9GT9yrYWc/It/JJZE=; b=Bk8s+/IQNcdMzkHm8MNSnxSP7epc04AHCLLegh9X9RgXLG56/6TT41aqK9pChFX/wn 2UmoP+T8eXi6dp+VvtoWfwQmtrbVN0QaOjzyilbrKXfXxkzHK2T2ZwYshAeEh6fsRU+R ApsJS9h/ta45tifHGeOKaNqMGGGptIUmPRX0vujRanHSFxiq8LZGU1sjpGMtcBDr1U/2 MeJlKAJ9rMeAJivtZzEEm2RuRRwtBhAiP1yT9KthyCsV6wdf1dAmxCRx4JVKpcJEkWCa KUPchq5cORMY11af6Z3aQTE3rnEk+TWnbAxyne9VV4nyRNAOyqgXbQu9xyxnBqFJY+O8 YEVw== X-Gm-Message-State: AKGB3mLDmYq5DehcAE3Fuz/CJCfHVKiP/FIkr+trBmNA3pMEqT5arwqq ctRgJFFHnB+LoccZgLPVm/MWdBNG X-Google-Smtp-Source: ACJfBov9l9AbwPRZnQRczvIJEyBtqCIsbfxmcU/G1JE/hKHSzYKAUjhmAd0q1jWD/MDwb+7uvCR1HQ== X-Received: by 10.84.160.197 with SMTP id v5mr2014202plg.268.1515132612186; Thu, 04 Jan 2018 22:10:12 -0800 (PST) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o64sm2444086pga.1.2018.01.04.22.10.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Jan 2018 22:10:11 -0800 (PST) From: Jakub Kicinski To: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, daniel@iogearbox.net, davem@davemloft.net Cc: oss-drivers@netronome.com, tehnerd@fb.com, Jakub Kicinski Subject: [PATCH bpf-next 08/16] bpf: offload: add map offload infrastructure Date: Thu, 4 Jan 2018 22:09:23 -0800 Message-Id: <20180105060931.30815-9-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180105060931.30815-1-jakub.kicinski@netronome.com> References: <20180105060931.30815-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org BPF map offload follow similar path to program offload. At creation time users may specify ifindex of the device on which they want to create the map. Map will be validated by the kernel's .map_alloc_check callback and device driver will be called for the actual allocation. Map will have an empty set of operations associated with it (save for alloc and free callbacks). The real device callbacks are kept in map->offload->dev_ops because they have slightly different signatures. Map operations are called in process context so the driver may communicate with HW freely, msleep(), wait() etc. Map alloc and free callbacks are muxed via existing .ndo_bpf, and are always called with rtnl lock held. Maps and programs are guaranteed to be destroyed before .ndo_uninit (i.e. before unregister_netdev() returns). Map callbacks are invoked with bpf_devs_lock *read* locked, drivers must take care of exclusive locking if necessary. All offload-specific branches are marked with unlikely() (through bpf_map_is_dev_bound()), given that branch penalty will be negligible compared to IO anyway, and we don't want to penalize SW path unnecessarily. Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet --- include/linux/bpf.h | 59 +++++++++++++ include/linux/netdevice.h | 6 ++ include/uapi/linux/bpf.h | 1 + kernel/bpf/offload.c | 189 +++++++++++++++++++++++++++++++++++++++-- kernel/bpf/syscall.c | 42 +++++++-- kernel/bpf/verifier.c | 7 ++ tools/include/uapi/linux/bpf.h | 1 + 7 files changed, 293 insertions(+), 12 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 425ca3cd1e32..d9a0beb78825 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -65,6 +65,33 @@ struct bpf_map { #endif }; +struct bpf_offloaded_map; + +struct bpf_map_dev_ops { + int (*map_get_next_key)(struct bpf_offloaded_map *map, + void *key, void *next_key); + int (*map_lookup_elem)(struct bpf_offloaded_map *map, + void *key, void *value); + int (*map_update_elem)(struct bpf_offloaded_map *map, + void *key, void *value, u64 flags); + int (*map_delete_elem)(struct bpf_offloaded_map *map, void *key); +}; + +struct bpf_offloaded_map { + struct bpf_map map; + struct net_device *netdev; + const struct bpf_map_dev_ops *dev_ops; + void *dev_priv; + struct list_head offloads; +}; + +static inline struct bpf_offloaded_map *map_to_offmap(struct bpf_map *map) +{ + return container_of(map, struct bpf_offloaded_map, map); +} + +extern const struct bpf_map_ops bpf_map_offload_ops; + /* function argument constraints */ enum bpf_arg_type { ARG_DONTCARE = 0, /* unused argument in helper function */ @@ -359,6 +386,7 @@ int __bpf_prog_charge(struct user_struct *user, u32 pages); void __bpf_prog_uncharge(struct user_struct *user, u32 pages); void bpf_prog_free_id(struct bpf_prog *prog, bool do_idr_lock); +void bpf_map_free_id(struct bpf_map *map, bool do_idr_lock); struct bpf_map *bpf_map_get_with_uref(u32 ufd); struct bpf_map *__bpf_map_get(struct fd f); @@ -536,6 +564,15 @@ void bpf_prog_offload_destroy(struct bpf_prog *prog); int bpf_prog_offload_info_fill(struct bpf_prog_info *info, struct bpf_prog *prog); +int bpf_map_offload_lookup_elem(struct bpf_map *map, void *key, void *value); +int bpf_map_offload_update_elem(struct bpf_map *map, + void *key, void *value, u64 flags); +int bpf_map_offload_delete_elem(struct bpf_map *map, void *key); +int bpf_map_offload_get_next_key(struct bpf_map *map, + void *key, void *next_key); + +bool bpf_offload_dev_match(struct bpf_prog *prog, struct bpf_map *map); + #if defined(CONFIG_NET) && defined(CONFIG_BPF_SYSCALL) int bpf_prog_offload_init(struct bpf_prog *prog, union bpf_attr *attr); @@ -543,6 +580,14 @@ static inline bool bpf_prog_is_dev_bound(struct bpf_prog_aux *aux) { return aux->offload_requested; } + +static inline bool bpf_map_is_dev_bound(struct bpf_map *map) +{ + return unlikely(map->ops == &bpf_map_offload_ops); +} + +struct bpf_map *bpf_map_offload_map_alloc(union bpf_attr *attr); +void bpf_map_offload_map_free(struct bpf_map *map); #else static inline int bpf_prog_offload_init(struct bpf_prog *prog, union bpf_attr *attr) @@ -554,6 +599,20 @@ static inline bool bpf_prog_is_dev_bound(struct bpf_prog_aux *aux) { return false; } + +static inline bool bpf_map_is_dev_bound(struct bpf_map *map) +{ + return false; +} + +static inline struct bpf_map *bpf_map_offload_map_alloc(union bpf_attr *attr) +{ + return ERR_PTR(-EOPNOTSUPP); +} + +static inline void bpf_map_offload_map_free(struct bpf_map *map) +{ +} #endif /* CONFIG_NET && CONFIG_BPF_SYSCALL */ #if defined(CONFIG_STREAM_PARSER) && defined(CONFIG_BPF_SYSCALL) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 49bfc6eec74c..ecbdb5d519d6 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -802,6 +802,8 @@ enum bpf_netdev_command { BPF_OFFLOAD_VERIFIER_PREP, BPF_OFFLOAD_TRANSLATE, BPF_OFFLOAD_DESTROY, + BPF_OFFLOAD_MAP_ALLOC, + BPF_OFFLOAD_MAP_FREE, }; struct bpf_prog_offload_ops; @@ -832,6 +834,10 @@ struct netdev_bpf { struct { struct bpf_prog *prog; } offload; + /* BPF_OFFLOAD_MAP_ALLOC, BPF_OFFLOAD_MAP_FREE */ + struct { + struct bpf_offloaded_map *offmap; + }; }; }; diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index f2f8b36e2ad4..9910ab632652 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -245,6 +245,7 @@ union bpf_attr { * BPF_F_NUMA_NODE is set). */ char map_name[BPF_OBJ_NAME_LEN]; + __u32 map_ifindex; /* ifindex of netdev to create on */ }; struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */ diff --git a/kernel/bpf/offload.c b/kernel/bpf/offload.c index cdd1e19a668b..dddc172c8818 100644 --- a/kernel/bpf/offload.c +++ b/kernel/bpf/offload.c @@ -24,11 +24,13 @@ #include #include -/* Protects bpf_prog_offload_devs and offload members of all progs. +/* Protects bpf_prog_offload_devs, bpf_map_offload_devs and offload members + * of all progs. * RTNL lock cannot be taken when holding this lock. */ static DECLARE_RWSEM(bpf_devs_lock); static LIST_HEAD(bpf_prog_offload_devs); +static LIST_HEAD(bpf_map_offload_devs); static int bpf_dev_offload_check(struct net_device *netdev) { @@ -250,11 +252,187 @@ int bpf_prog_offload_info_fill(struct bpf_prog_info *info, const struct bpf_prog_ops bpf_offload_prog_ops = { }; +static int bpf_map_offload_ndo(struct bpf_offloaded_map *offmap, + enum bpf_netdev_command cmd) +{ + struct netdev_bpf data = {}; + struct net_device *netdev; + + ASSERT_RTNL(); + + data.command = cmd; + data.offmap = offmap; + /* Caller must make sure netdev is valid */ + netdev = offmap->netdev; + + return netdev->netdev_ops->ndo_bpf(netdev, &data); +} + +struct bpf_map *bpf_map_offload_map_alloc(union bpf_attr *attr) +{ + struct net *net = current->nsproxy->net_ns; + struct bpf_offloaded_map *offmap; + int err; + + if (!capable(CAP_SYS_ADMIN)) + return ERR_PTR(-EPERM); + if (attr->map_type != BPF_MAP_TYPE_ARRAY && + attr->map_type != BPF_MAP_TYPE_HASH) + return ERR_PTR(-EINVAL); + + offmap = kzalloc(sizeof(*offmap), GFP_USER); + if (!offmap) + return ERR_PTR(-ENOMEM); + + bpf_map_init_from_attr(&offmap->map, attr); + + rtnl_lock(); + down_write(&bpf_devs_lock); + offmap->netdev = __dev_get_by_index(net, attr->map_ifindex); + err = bpf_dev_offload_check(offmap->netdev); + if (err) + goto err_unlock; + + err = bpf_map_offload_ndo(offmap, BPF_OFFLOAD_MAP_ALLOC); + if (err) + goto err_unlock; + + list_add_tail(&offmap->offloads, &bpf_map_offload_devs); + up_write(&bpf_devs_lock); + rtnl_unlock(); + + return &offmap->map; + +err_unlock: + up_write(&bpf_devs_lock); + rtnl_unlock(); + kfree(offmap); + return ERR_PTR(err); +} + +static void __bpf_map_offload_destroy(struct bpf_offloaded_map *offmap) +{ + WARN_ON(bpf_map_offload_ndo(offmap, BPF_OFFLOAD_MAP_FREE)); + /* Make sure BPF_MAP_GET_NEXT_ID can't find this dead map */ + bpf_map_free_id(&offmap->map, true); + list_del_init(&offmap->offloads); + offmap->netdev = NULL; +} + +void bpf_map_offload_map_free(struct bpf_map *map) +{ + struct bpf_offloaded_map *offmap = map_to_offmap(map); + + rtnl_lock(); + down_write(&bpf_devs_lock); + if (offmap->netdev) + __bpf_map_offload_destroy(offmap); + up_write(&bpf_devs_lock); + rtnl_unlock(); + + kfree(offmap); +} + +int bpf_map_offload_lookup_elem(struct bpf_map *map, void *key, void *value) +{ + struct bpf_offloaded_map *offmap = map_to_offmap(map); + int ret = -ENODEV; + + down_read(&bpf_devs_lock); + if (offmap->netdev) + ret = offmap->dev_ops->map_lookup_elem(offmap, key, value); + up_read(&bpf_devs_lock); + + return ret; +} + +int bpf_map_offload_update_elem(struct bpf_map *map, + void *key, void *value, u64 flags) +{ + struct bpf_offloaded_map *offmap = map_to_offmap(map); + int ret = -ENODEV; + + if (unlikely(flags > BPF_EXIST)) + return -EINVAL; + + down_read(&bpf_devs_lock); + if (offmap->netdev) + ret = offmap->dev_ops->map_update_elem(offmap, key, value, + flags); + up_read(&bpf_devs_lock); + + return ret; +} + +int bpf_map_offload_delete_elem(struct bpf_map *map, void *key) +{ + struct bpf_offloaded_map *offmap = map_to_offmap(map); + int ret = -ENODEV; + + down_read(&bpf_devs_lock); + if (offmap->netdev) + ret = offmap->dev_ops->map_delete_elem(offmap, key); + up_read(&bpf_devs_lock); + + return ret; +} + +int bpf_map_offload_get_next_key(struct bpf_map *map, void *key, void *next_key) +{ + struct bpf_offloaded_map *offmap = map_to_offmap(map); + int ret = -ENODEV; + + down_read(&bpf_devs_lock); + if (offmap->netdev) + ret = offmap->dev_ops->map_get_next_key(offmap, key, next_key); + up_read(&bpf_devs_lock); + + return ret; +} + +bool bpf_offload_dev_match(struct bpf_prog *prog, struct bpf_map *map) +{ + struct bpf_offloaded_map *offmap; + struct bpf_prog_offload *offload; + bool ret; + + if (!!bpf_prog_is_dev_bound(prog->aux) != !!bpf_map_is_dev_bound(map)) + return false; + if (!bpf_prog_is_dev_bound(prog->aux)) + return true; + + down_read(&bpf_devs_lock); + offload = prog->aux->offload; + offmap = map_to_offmap(map); + + ret = offload && offload->netdev == offmap->netdev; + up_read(&bpf_devs_lock); + + return ret; +} + +static void bpf_offload_orphan_all_progs(struct net_device *netdev) +{ + struct bpf_prog_offload *offload, *tmp; + + list_for_each_entry_safe(offload, tmp, &bpf_prog_offload_devs, offloads) + if (offload->netdev == netdev) + __bpf_prog_offload_destroy(offload->prog); +} + +static void bpf_offload_orphan_all_maps(struct net_device *netdev) +{ + struct bpf_offloaded_map *offmap, *tmp; + + list_for_each_entry_safe(offmap, tmp, &bpf_map_offload_devs, offloads) + if (offmap->netdev == netdev) + __bpf_map_offload_destroy(offmap); +} + static int bpf_offload_notification(struct notifier_block *notifier, ulong event, void *ptr) { struct net_device *netdev = netdev_notifier_info_to_dev(ptr); - struct bpf_prog_offload *offload, *tmp; ASSERT_RTNL(); @@ -265,11 +443,8 @@ static int bpf_offload_notification(struct notifier_block *notifier, break; down_write(&bpf_devs_lock); - list_for_each_entry_safe(offload, tmp, &bpf_prog_offload_devs, - offloads) { - if (offload->netdev == netdev) - __bpf_prog_offload_destroy(offload->prog); - } + bpf_offload_orphan_all_progs(netdev); + bpf_offload_orphan_all_maps(netdev); up_write(&bpf_devs_lock); break; default: diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 14adf29fc0f2..d5b773f04da1 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -94,6 +94,11 @@ static int check_uarg_tail_zero(void __user *uaddr, return 0; } +const struct bpf_map_ops bpf_map_offload_ops = { + .map_alloc = bpf_map_offload_map_alloc, + .map_free = bpf_map_offload_map_free, +}; + static struct bpf_map *find_and_alloc_map(union bpf_attr *attr) { const struct bpf_map_ops *ops; @@ -111,6 +116,8 @@ static struct bpf_map *find_and_alloc_map(union bpf_attr *attr) if (err) return ERR_PTR(err); } + if (attr->map_ifindex) + ops = &bpf_map_offload_ops; map = ops->map_alloc(attr); if (IS_ERR(map)) return map; @@ -208,16 +215,25 @@ static int bpf_map_alloc_id(struct bpf_map *map) return id > 0 ? 0 : id; } -static void bpf_map_free_id(struct bpf_map *map, bool do_idr_lock) +void bpf_map_free_id(struct bpf_map *map, bool do_idr_lock) { unsigned long flags; + /* Offloaded maps are removed from the IDR store when their device + * disappears - even if someone holds an fd to them they are unusable, + * the memory is gone, all ops will fail; they are simply waiting for + * refcnt to drop to be freed. + */ + if (!map->id) + return; + if (do_idr_lock) spin_lock_irqsave(&map_idr_lock, flags); else __acquire(&map_idr_lock); idr_remove(&map_idr, map->id); + map->id = 0; if (do_idr_lock) spin_unlock_irqrestore(&map_idr_lock, flags); @@ -397,7 +413,7 @@ static int bpf_obj_name_cpy(char *dst, const char *src) return 0; } -#define BPF_MAP_CREATE_LAST_FIELD map_name +#define BPF_MAP_CREATE_LAST_FIELD map_ifindex /* called via syscall */ static int map_create(union bpf_attr *attr) { @@ -585,8 +601,10 @@ static int map_lookup_elem(union bpf_attr *attr) if (!value) goto free_key; - if (map->map_type == BPF_MAP_TYPE_PERCPU_HASH || - map->map_type == BPF_MAP_TYPE_LRU_PERCPU_HASH) { + if (bpf_map_is_dev_bound(map)) { + err = bpf_map_offload_lookup_elem(map, key, value); + } else if (map->map_type == BPF_MAP_TYPE_PERCPU_HASH || + map->map_type == BPF_MAP_TYPE_LRU_PERCPU_HASH) { err = bpf_percpu_hash_copy(map, key, value); } else if (map->map_type == BPF_MAP_TYPE_PERCPU_ARRAY) { err = bpf_percpu_array_copy(map, key, value); @@ -676,6 +694,9 @@ static int map_update_elem(union bpf_attr *attr) if (map->map_type == BPF_MAP_TYPE_CPUMAP) { err = map->ops->map_update_elem(map, key, value, attr->flags); goto out; + } else if (bpf_map_is_dev_bound(map)) { + err = bpf_map_offload_update_elem(map, key, value, attr->flags); + goto out; } /* must increment bpf_prog_active to avoid kprobe+bpf triggering from @@ -750,6 +771,11 @@ static int map_delete_elem(union bpf_attr *attr) goto err_put; } + if (bpf_map_is_dev_bound(map)) { + err = bpf_map_offload_delete_elem(map, key); + goto out; + } + preempt_disable(); __this_cpu_inc(bpf_prog_active); rcu_read_lock(); @@ -757,7 +783,7 @@ static int map_delete_elem(union bpf_attr *attr) rcu_read_unlock(); __this_cpu_dec(bpf_prog_active); preempt_enable(); - +out: if (!err) trace_bpf_map_delete_elem(map, ufd, key); kfree(key); @@ -807,9 +833,15 @@ static int map_get_next_key(union bpf_attr *attr) if (!next_key) goto free_key; + if (bpf_map_is_dev_bound(map)) { + err = bpf_map_offload_get_next_key(map, key, next_key); + goto out; + } + rcu_read_lock(); err = map->ops->map_get_next_key(map, key, next_key); rcu_read_unlock(); +out: if (err) goto free_next_key; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index a2b211262c25..187f245a3338 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -4795,6 +4795,13 @@ static int check_map_prog_compatibility(struct bpf_verifier_env *env, return -EINVAL; } } + + if ((bpf_prog_is_dev_bound(prog->aux) || bpf_map_is_dev_bound(map)) && + !bpf_offload_dev_match(prog, map)) { + verbose(env, "offload device mismatch between prog and map\n"); + return -EINVAL; + } + return 0; } diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 4e8c60acfa32..69f96af4a569 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -245,6 +245,7 @@ union bpf_attr { * BPF_F_NUMA_NODE is set). */ char map_name[BPF_OBJ_NAME_LEN]; + __u32 map_ifindex; /* ifindex of netdev to create on */ }; struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */ From patchwork Fri Jan 5 06:09:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 855928 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="qqUqW+RQ"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zCZ716rLlz9s4q for ; Fri, 5 Jan 2018 17:11:25 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751490AbeAEGLS (ORCPT ); Fri, 5 Jan 2018 01:11:18 -0500 Received: from mail-pl0-f66.google.com ([209.85.160.66]:33879 "EHLO mail-pl0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751347AbeAEGKN (ORCPT ); Fri, 5 Jan 2018 01:10:13 -0500 Received: by mail-pl0-f66.google.com with SMTP id d21so2548465pll.1 for ; Thu, 04 Jan 2018 22:10:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=eumvFGClPewJEju+kJqNzLJeD1jsXIV05ha8QsIjt/s=; b=qqUqW+RQr12wQyeJa6OKjO2EUDjeSj+KsGY3giv/10+FtPN0otjhD1MmFxoiXsApti l9IYAI66iy9E2ZBNPKIFVwq5HnXMmzz+Yn5LgXOHikeuqjCk5/Qv4SoxBJzd8JQLWrOA g4M1IfftqUioEN7CmeNoIBaQi3fOqlL7wO0H5gtZ97ke3BHXINUyDxqj4aYQBy/uS27/ Xq1y1hEJ0bp7UZPiXAIyqzegm6DnipvzEtplKndeL1pop3SvnUvfVPbQb/VI0W42ere2 V7QGB6fTiVyTQJaFzUgb46yi7/P9l9XKXAy6PsMjFLR+vwngkLfZL0B3hw+X4WXRZ4lS QrCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=eumvFGClPewJEju+kJqNzLJeD1jsXIV05ha8QsIjt/s=; b=P1Yd61c7xXPbj/FBvb4qZGuXAMthSzXh+GjxuHxi9zZd2EorQs3iETsNY6UrdoIyyT mqpS9bEvrzunz1V11XbDU2EhQDfNGd12OrUxvGs1nFuO0pgVOCZwcAtxhWgCFo1F4hYh DdrnOwbywya0MFjsTbqIGn2gH7hgL3kx5ufg7+6iA3UtAa/ZHXfEssMK2HqGMLq4po93 lADtizHS177nTrAvVYurnNnezOxat/cffnPJrO5TB7NpbTjGeuU0AtTU/8hM1gftTrRp 3dBDrYGzDTwS1DJnqojkQSYIXk2S2zwDq5IShx2e0RjLs15+p1u8khPa/QGFq8BNnht5 eG5Q== X-Gm-Message-State: AKGB3mIp3HlSkuCT1CUw0e24DAmM0ykXLTq3mZcw2Ob0jdM/9LYBBE2v fcKqYdUF9goXXCQOpmsPE0VlAApw X-Google-Smtp-Source: ACJfBovD8z8SjxGS+zYFVUE0gpLAgNvgWB2Ko69bpNjF89OVEVBL3vYaWPnzgbhNtF2NOhTn32bp1Q== X-Received: by 10.159.244.3 with SMTP id x3mr2062411plr.192.1515132613126; Thu, 04 Jan 2018 22:10:13 -0800 (PST) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o64sm2444086pga.1.2018.01.04.22.10.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Jan 2018 22:10:12 -0800 (PST) From: Jakub Kicinski To: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, daniel@iogearbox.net, davem@davemloft.net Cc: oss-drivers@netronome.com, tehnerd@fb.com, Jakub Kicinski Subject: [PATCH bpf-next 09/16] nfp: hand over to BPF offload app at coarser granularity Date: Thu, 4 Jan 2018 22:09:24 -0800 Message-Id: <20180105060931.30815-10-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180105060931.30815-1-jakub.kicinski@netronome.com> References: <20180105060931.30815-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Instead of having an app callback per message type hand off all offload-related handling to apps with one "rest of ndo_bpf" callback. Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet --- drivers/net/ethernet/netronome/nfp/bpf/main.c | 5 +-- drivers/net/ethernet/netronome/nfp/bpf/main.h | 8 +--- drivers/net/ethernet/netronome/nfp/bpf/offload.c | 25 +++++++++--- drivers/net/ethernet/netronome/nfp/nfp_app.h | 47 +++++----------------- .../net/ethernet/netronome/nfp/nfp_net_common.c | 10 +---- 5 files changed, 34 insertions(+), 61 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/bpf/main.c b/drivers/net/ethernet/netronome/nfp/bpf/main.c index 4b63167906ca..684b16cb6a20 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/main.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/main.c @@ -318,9 +318,6 @@ const struct nfp_app_type app_bpf = { .setup_tc = nfp_bpf_setup_tc, .tc_busy = nfp_bpf_tc_busy, + .bpf = nfp_ndo_bpf, .xdp_offload = nfp_bpf_xdp_offload, - - .bpf_verifier_prep = nfp_bpf_verifier_prep, - .bpf_translate = nfp_bpf_translate, - .bpf_destroy = nfp_bpf_destroy, }; diff --git a/drivers/net/ethernet/netronome/nfp/bpf/main.h b/drivers/net/ethernet/netronome/nfp/bpf/main.h index 89a9b6393882..c7d815455cb0 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/main.h +++ b/drivers/net/ethernet/netronome/nfp/bpf/main.h @@ -244,15 +244,11 @@ struct netdev_bpf; struct nfp_app; struct nfp_net; +int nfp_ndo_bpf(struct nfp_app *app, struct nfp_net *nn, + struct netdev_bpf *bpf); int nfp_net_bpf_offload(struct nfp_net *nn, struct bpf_prog *prog, bool old_prog); -int nfp_bpf_verifier_prep(struct nfp_app *app, struct nfp_net *nn, - struct netdev_bpf *bpf); -int nfp_bpf_translate(struct nfp_app *app, struct nfp_net *nn, - struct bpf_prog *prog); -int nfp_bpf_destroy(struct nfp_app *app, struct nfp_net *nn, - struct bpf_prog *prog); struct nfp_insn_meta * nfp_bpf_goto_meta(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, unsigned int insn_idx, unsigned int n_insns); diff --git a/drivers/net/ethernet/netronome/nfp/bpf/offload.c b/drivers/net/ethernet/netronome/nfp/bpf/offload.c index 1a67673261b7..12a48b7583a3 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/offload.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/offload.c @@ -102,8 +102,9 @@ static void nfp_prog_free(struct nfp_prog *nfp_prog) kfree(nfp_prog); } -int nfp_bpf_verifier_prep(struct nfp_app *app, struct nfp_net *nn, - struct netdev_bpf *bpf) +static int +nfp_bpf_verifier_prep(struct nfp_app *app, struct nfp_net *nn, + struct netdev_bpf *bpf) { struct bpf_prog *prog = bpf->verifier.prog; struct nfp_prog *nfp_prog; @@ -133,8 +134,7 @@ int nfp_bpf_verifier_prep(struct nfp_app *app, struct nfp_net *nn, return ret; } -int nfp_bpf_translate(struct nfp_app *app, struct nfp_net *nn, - struct bpf_prog *prog) +static int nfp_bpf_translate(struct nfp_net *nn, struct bpf_prog *prog) { struct nfp_prog *nfp_prog = prog->aux->offload->dev_priv; unsigned int stack_size; @@ -161,8 +161,7 @@ int nfp_bpf_translate(struct nfp_app *app, struct nfp_net *nn, return nfp_bpf_jit(nfp_prog); } -int nfp_bpf_destroy(struct nfp_app *app, struct nfp_net *nn, - struct bpf_prog *prog) +static int nfp_bpf_destroy(struct nfp_net *nn, struct bpf_prog *prog) { struct nfp_prog *nfp_prog = prog->aux->offload->dev_priv; @@ -172,6 +171,20 @@ int nfp_bpf_destroy(struct nfp_app *app, struct nfp_net *nn, return 0; } +int nfp_ndo_bpf(struct nfp_app *app, struct nfp_net *nn, struct netdev_bpf *bpf) +{ + switch (bpf->command) { + case BPF_OFFLOAD_VERIFIER_PREP: + return nfp_bpf_verifier_prep(app, nn, bpf); + case BPF_OFFLOAD_TRANSLATE: + return nfp_bpf_translate(nn, bpf->offload.prog); + case BPF_OFFLOAD_DESTROY: + return nfp_bpf_destroy(nn, bpf->offload.prog); + default: + return -EINVAL; + } +} + static int nfp_net_bpf_load(struct nfp_net *nn, struct bpf_prog *prog) { struct nfp_prog *nfp_prog = prog->aux->offload->dev_priv; diff --git a/drivers/net/ethernet/netronome/nfp/nfp_app.h b/drivers/net/ethernet/netronome/nfp/nfp_app.h index 3af1943a8521..ee67a7202819 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_app.h +++ b/drivers/net/ethernet/netronome/nfp/nfp_app.h @@ -87,10 +87,8 @@ extern const struct nfp_app_type app_flower; * @ctrl_msg_rx: control message handler * @setup_tc: setup TC ndo * @tc_busy: TC HW offload busy (rules loaded) + * @bpf: BPF ndo offload-related calls * @xdp_offload: offload an XDP program - * @bpf_verifier_prep: verifier prep for dev-specific BPF programs - * @bpf_translate: translate call for dev-specific BPF programs - * @bpf_destroy: destroy for dev-specific BPF programs * @eswitch_mode_get: get SR-IOV eswitch mode * @sriov_enable: app-specific sriov initialisation * @sriov_disable: app-specific sriov clean-up @@ -128,14 +126,10 @@ struct nfp_app_type { int (*setup_tc)(struct nfp_app *app, struct net_device *netdev, enum tc_setup_type type, void *type_data); bool (*tc_busy)(struct nfp_app *app, struct nfp_net *nn); + int (*bpf)(struct nfp_app *app, struct nfp_net *nn, + struct netdev_bpf *xdp); int (*xdp_offload)(struct nfp_app *app, struct nfp_net *nn, struct bpf_prog *prog); - int (*bpf_verifier_prep)(struct nfp_app *app, struct nfp_net *nn, - struct netdev_bpf *bpf); - int (*bpf_translate)(struct nfp_app *app, struct nfp_net *nn, - struct bpf_prog *prog); - int (*bpf_destroy)(struct nfp_app *app, struct nfp_net *nn, - struct bpf_prog *prog); int (*sriov_enable)(struct nfp_app *app, int num_vfs); void (*sriov_disable)(struct nfp_app *app); @@ -303,6 +297,14 @@ static inline int nfp_app_setup_tc(struct nfp_app *app, return app->type->setup_tc(app, netdev, type, type_data); } +static inline int nfp_app_bpf(struct nfp_app *app, struct nfp_net *nn, + struct netdev_bpf *bpf) +{ + if (!app || !app->type->bpf) + return -EINVAL; + return app->type->bpf(app, nn, bpf); +} + static inline int nfp_app_xdp_offload(struct nfp_app *app, struct nfp_net *nn, struct bpf_prog *prog) { @@ -311,33 +313,6 @@ static inline int nfp_app_xdp_offload(struct nfp_app *app, struct nfp_net *nn, return app->type->xdp_offload(app, nn, prog); } -static inline int -nfp_app_bpf_verifier_prep(struct nfp_app *app, struct nfp_net *nn, - struct netdev_bpf *bpf) -{ - if (!app || !app->type->bpf_verifier_prep) - return -EOPNOTSUPP; - return app->type->bpf_verifier_prep(app, nn, bpf); -} - -static inline int -nfp_app_bpf_translate(struct nfp_app *app, struct nfp_net *nn, - struct bpf_prog *prog) -{ - if (!app || !app->type->bpf_translate) - return -EOPNOTSUPP; - return app->type->bpf_translate(app, nn, prog); -} - -static inline int -nfp_app_bpf_destroy(struct nfp_app *app, struct nfp_net *nn, - struct bpf_prog *prog) -{ - if (!app || !app->type->bpf_destroy) - return -EOPNOTSUPP; - return app->type->bpf_destroy(app, nn, prog); -} - static inline bool nfp_app_ctrl_tx(struct nfp_app *app, struct sk_buff *skb) { trace_devlink_hwmsg(priv_to_devlink(app->pf), false, 0, diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c index 0add4870ce2e..773089442b64 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c @@ -3394,16 +3394,8 @@ static int nfp_net_xdp(struct net_device *netdev, struct netdev_bpf *xdp) xdp->prog_id = nn->xdp_prog ? nn->xdp_prog->aux->id : 0; xdp->prog_flags = nn->xdp_prog ? nn->xdp_flags : 0; return 0; - case BPF_OFFLOAD_VERIFIER_PREP: - return nfp_app_bpf_verifier_prep(nn->app, nn, xdp); - case BPF_OFFLOAD_TRANSLATE: - return nfp_app_bpf_translate(nn->app, nn, - xdp->offload.prog); - case BPF_OFFLOAD_DESTROY: - return nfp_app_bpf_destroy(nn->app, nn, - xdp->offload.prog); default: - return -EINVAL; + return nfp_app_bpf(nn->app, nn, xdp); } } From patchwork Fri Jan 5 06:09:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 855919 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="KbtZ4CFO"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zCZ5s28gHz9s4q for ; Fri, 5 Jan 2018 17:10:25 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751416AbeAEGKV (ORCPT ); Fri, 5 Jan 2018 01:10:21 -0500 Received: from mail-pl0-f67.google.com ([209.85.160.67]:38410 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751286AbeAEGKO (ORCPT ); Fri, 5 Jan 2018 01:10:14 -0500 Received: by mail-pl0-f67.google.com with SMTP id s10so2545494plj.5 for ; Thu, 04 Jan 2018 22:10:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=adEl4IsZPo8j2jXTsJFEDvgpVxWFJP3Ii0A8oQnKUJQ=; b=KbtZ4CFOBliTKwDzVw7ucQYHmfERSt0EbcVcU77r7vP1LUnbwMA8GMtLLkxCaXyMK6 ovpVGJPRXPhR9DviniBCmr1NLXjbBwjSlB4HO/VRFqE0vREVXlHpY9OWhokAuW/lfXKw gEsSVmekMZcpLHJZ8I0IV1ZjxYjzTcygcjAD9tf0koWM549Yx/NTdKJIluYQX+0C0u0Y JBfYpWC825I+QVEGCoG0+YRfJuphEw9EPDPd1fllp/LxbGdwddZcD5c1hCXylDmQ9MEe UZEsYsiTBjCAUp6S9tCfScn1DRtAlo7bX4AtaEmHkgX5M9oG8YkqUS3gkr9rf6KWqVEA 738w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=adEl4IsZPo8j2jXTsJFEDvgpVxWFJP3Ii0A8oQnKUJQ=; b=nYjvbHHd06P/yHiYEEYZi9a8HkjiBAyin9c2HLJpNeJyvruyEZFsQRurdOVZDOlruK TObvHg8cJLZRBSWmFGyKW5L+t6h9A+tou4McpaPoBAsiz4uNNc44eB48vf4RfQiyHiX0 RufYFrPnOqL8rW2wZzRqChC/FnWevdcWd1EZEeEMCTEpNgG2Zhb+9bKgPUjWvyKNFSa9 EGArL5rSafnXwWmBcte6L/TDC/RoPDm1zWyImL7UyXST9PsQMMTOcLeeuMqX/rU8MbJ5 iiGo/Y/XlOXeBY4xZxZrSGKe5ZDqFtfuuY1zzQ6su10x9jyfR3s3GmZeDhApDUKHQ50W +2rw== X-Gm-Message-State: AKGB3mJ7GSfxvp2dDYjmxXdpQPLm4+npDTpXvI59IFAxIU+Fpqn9A6qT BdeDwtvHQ+nRDeKiMCSHf0YcnRNM X-Google-Smtp-Source: ACJfBovp0stmh7mIh9ERzWpzF4fxIQHzRXgP5vNEQm9SfAUv/M2m+pbCxc8J1POwtcLPKzNHmaAJ6w== X-Received: by 10.84.225.129 with SMTP id u1mr2069028plj.329.1515132614232; Thu, 04 Jan 2018 22:10:14 -0800 (PST) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o64sm2444086pga.1.2018.01.04.22.10.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Jan 2018 22:10:13 -0800 (PST) From: Jakub Kicinski To: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, daniel@iogearbox.net, davem@davemloft.net Cc: oss-drivers@netronome.com, tehnerd@fb.com, Jakub Kicinski Subject: [PATCH bpf-next 10/16] nfp: bpf: add map data structure Date: Thu, 4 Jan 2018 22:09:25 -0800 Message-Id: <20180105060931.30815-11-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180105060931.30815-1-jakub.kicinski@netronome.com> References: <20180105060931.30815-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org To be able to split code into reasonable chunks we need to add the map data structures already. Later patches will add code piece by piece. Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet --- drivers/net/ethernet/netronome/nfp/bpf/main.c | 7 ++++++- drivers/net/ethernet/netronome/nfp/bpf/main.h | 18 ++++++++++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/netronome/nfp/bpf/main.c b/drivers/net/ethernet/netronome/nfp/bpf/main.c index 684b16cb6a20..2506d1139fa9 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/main.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/main.c @@ -288,6 +288,8 @@ static int nfp_bpf_init(struct nfp_app *app) bpf->app = app; app->priv = bpf; + INIT_LIST_HEAD(&bpf->map_list); + err = nfp_bpf_parse_capabilities(app); if (err) goto err_free_bpf; @@ -301,7 +303,10 @@ static int nfp_bpf_init(struct nfp_app *app) static void nfp_bpf_clean(struct nfp_app *app) { - kfree(app->priv); + struct nfp_app_bpf *bpf = app->priv; + + WARN_ON(!list_empty(&bpf->map_list)); + kfree(bpf); } const struct nfp_app_type app_bpf = { diff --git a/drivers/net/ethernet/netronome/nfp/bpf/main.h b/drivers/net/ethernet/netronome/nfp/bpf/main.h index c7d815455cb0..71c58d25858b 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/main.h +++ b/drivers/net/ethernet/netronome/nfp/bpf/main.h @@ -82,6 +82,8 @@ enum pkt_vec { * struct nfp_app_bpf - bpf app priv structure * @app: backpointer to the app * + * @map_list: list of offloaded maps + * * @adjust_head: adjust head capability * @flags: extra flags for adjust head * @off_min: minimal packet offset within buffer required @@ -92,6 +94,8 @@ enum pkt_vec { struct nfp_app_bpf { struct nfp_app *app; + struct list_head map_list; + struct nfp_bpf_cap_adjust_head { u32 flags; int off_min; @@ -101,6 +105,20 @@ struct nfp_app_bpf { } adjust_head; }; +/** + * struct nfp_bpf_map - private per-map data attached to BPF maps for offload + * @offmap: pointer to the offloaded BPF map + * @bpf: back pointer to bpf app private structure + * @tid: table id identifying map on datapath + * @l: link on the nfp_app_bpf->map_list list + */ +struct nfp_bpf_map { + struct bpf_offloaded_map *offmap; + struct nfp_app_bpf *bpf; + u32 tid; + struct list_head l; +}; + struct nfp_prog; struct nfp_insn_meta; typedef int (*instr_cb_t)(struct nfp_prog *, struct nfp_insn_meta *); From patchwork Fri Jan 5 06:09:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 855926 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="Nmq45NK/"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zCZ6j1sRKz9s75 for ; Fri, 5 Jan 2018 17:11:09 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751478AbeAEGLH (ORCPT ); Fri, 5 Jan 2018 01:11:07 -0500 Received: from mail-pg0-f65.google.com ([74.125.83.65]:45406 "EHLO mail-pg0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751365AbeAEGKQ (ORCPT ); Fri, 5 Jan 2018 01:10:16 -0500 Received: by mail-pg0-f65.google.com with SMTP id c194so687654pga.12 for ; Thu, 04 Jan 2018 22:10:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Df787wkGpGvzrVNnsN49cy4IrDUPEZX7Zu9g6YU3Ubo=; b=Nmq45NK/ji6r4pONh8XEyymur5p7wN7b6tnF/F6DyOkoQhNc2ZYnLeyvoYd0KZ+Gl1 e+9rRYNITVLYMoPp/ShDyS1QnRbAhlYmQsxj5cCdvzN5I2ob+uO7C2jzVsORQXRym80F G+jSoTieNOqaXSnJAlD91OMsTsi7gHtwzY6JCoAePuuriggmymLEQ/3p/JfG+ZSbK8JB ldatslUU/CqW2m+z+PBf+TCCUA+VoVhQiwnxrzwZq/CwW5lLF0vLvaBas+i974IKodmM CZyj0AijBmocVDqsihssRhPdVqrXce5cB/nPlgPN8qibj9aeZoND/+Sz1FmtJx7tg++U h3ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Df787wkGpGvzrVNnsN49cy4IrDUPEZX7Zu9g6YU3Ubo=; b=tnKXXplo7tTohPfjiLmAMSufl4UMrZo7fL8D6MmSKO+QILyEZmuOJsjO6I1IqSTgCq bs7GiIaTmNURePQ8UJAOD293/PWVTYZK3ZF961r9Www97395vc4VAp7PyRj3i3iR5/79 42NLcnlwibhoE1Tyv95WGRipHGaKnXwmul8eQJrB6m3y61RStXjO4KGlHIJEyKqBPQvZ BzkNgQO2n+9NablmK+higK8dsFcDAFGQ7m9hbwYC1bwaCduRYMNBcIZm5dZnsYHW1WaB +J766hT6OrQ2/MTxgokhpN7hELmLFO1x/fJu6ORjbc2UKF4j14IaD7fEa6FHBePiookc VkUA== X-Gm-Message-State: AKGB3mJeQbQ66jpfDG2hnET3IL/FtbxGhmJ+dRaDzYt81BhAHMFg+oat VQPx+qE6abZrWEy6ofuR29XuuVte X-Google-Smtp-Source: ACJfBovUjfrTeZgT4JHm3roUftuA377EB+yJQzXLVvNZm2f3polyrsdPpMpBI18TBxOlvwi7W653xg== X-Received: by 10.101.82.68 with SMTP id q4mr1657580pgp.70.1515132615188; Thu, 04 Jan 2018 22:10:15 -0800 (PST) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o64sm2444086pga.1.2018.01.04.22.10.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Jan 2018 22:10:14 -0800 (PST) From: Jakub Kicinski To: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, daniel@iogearbox.net, davem@davemloft.net Cc: oss-drivers@netronome.com, tehnerd@fb.com, Jakub Kicinski Subject: [PATCH bpf-next 11/16] nfp: bpf: add basic control channel communication Date: Thu, 4 Jan 2018 22:09:26 -0800 Message-Id: <20180105060931.30815-12-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180105060931.30815-1-jakub.kicinski@netronome.com> References: <20180105060931.30815-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org For map support we will need to send and receive control messages. Add basic support for sending a message to FW, and waiting for a reply. Control messages are tagged with a 16 bit ID. Add a simple ID allocator and make sure we don't allow too many messages in flight, to avoid request <> reply mismatches. Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet --- drivers/net/ethernet/netronome/nfp/Makefile | 1 + drivers/net/ethernet/netronome/nfp/bpf/cmsg.c | 238 +++++++++++++++++++++ drivers/net/ethernet/netronome/nfp/bpf/fw.h | 22 ++ drivers/net/ethernet/netronome/nfp/bpf/main.c | 5 + drivers/net/ethernet/netronome/nfp/bpf/main.h | 23 ++ drivers/net/ethernet/netronome/nfp/nfp_app.h | 9 + drivers/net/ethernet/netronome/nfp/nfp_net.h | 12 ++ .../net/ethernet/netronome/nfp/nfp_net_common.c | 7 + 8 files changed, 317 insertions(+) create mode 100644 drivers/net/ethernet/netronome/nfp/bpf/cmsg.c diff --git a/drivers/net/ethernet/netronome/nfp/Makefile b/drivers/net/ethernet/netronome/nfp/Makefile index 6e5ef984398b..064f00e23a19 100644 --- a/drivers/net/ethernet/netronome/nfp/Makefile +++ b/drivers/net/ethernet/netronome/nfp/Makefile @@ -44,6 +44,7 @@ endif ifeq ($(CONFIG_BPF_SYSCALL),y) nfp-objs += \ + bpf/cmsg.o \ bpf/main.o \ bpf/offload.o \ bpf/verifier.o \ diff --git a/drivers/net/ethernet/netronome/nfp/bpf/cmsg.c b/drivers/net/ethernet/netronome/nfp/bpf/cmsg.c new file mode 100644 index 000000000000..46753ee9f7c5 --- /dev/null +++ b/drivers/net/ethernet/netronome/nfp/bpf/cmsg.c @@ -0,0 +1,238 @@ +/* + * Copyright (C) 2017 Netronome Systems, Inc. + * + * This software is dual licensed under the GNU General License Version 2, + * June 1991 as shown in the file COPYING in the top-level directory of this + * source tree or the BSD 2-Clause License provided below. You have the + * option to license this software under the complete terms of either license. + * + * The BSD 2-Clause License: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * 1. Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * 2. Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +#include +#include +#include +#include +#include + +#include "../nfp_app.h" +#include "../nfp_net.h" +#include "fw.h" +#include "main.h" + +#define cmsg_warn(bpf, msg...) nn_dp_warn(&(bpf)->app->ctrl->dp, msg) + +#define NFP_BPF_TAG_ALLOC_SPAN (U16_MAX / 4) + +static bool nfp_bpf_all_tags_busy(struct nfp_app_bpf *bpf) +{ + u16 used_tags; + + used_tags = bpf->tag_alloc_next - bpf->tag_alloc_last; + + return used_tags > NFP_BPF_TAG_ALLOC_SPAN; +} + +static int nfp_bpf_alloc_tag(struct nfp_app_bpf *bpf) +{ + /* All FW communication for BPF is request-reply. To make sure we + * don't reuse the message ID too early after timeout - limit the + * number of requests in flight. + */ + if (nfp_bpf_all_tags_busy(bpf)) { + cmsg_warn(bpf, "all FW request contexts busy!\n"); + return -EAGAIN; + } + + WARN_ON(__test_and_set_bit(bpf->tag_alloc_next, bpf->tag_allocator)); + return bpf->tag_alloc_next++; +} + +static void nfp_bpf_free_tag(struct nfp_app_bpf *bpf, u16 tag) +{ + WARN_ON(!__test_and_clear_bit(tag, bpf->tag_allocator)); + + while (!test_bit(bpf->tag_alloc_last, bpf->tag_allocator) && + bpf->tag_alloc_last != bpf->tag_alloc_next) + bpf->tag_alloc_last++; +} + +static unsigned int nfp_bpf_cmsg_get_tag(struct sk_buff *skb) +{ + struct cmsg_hdr *hdr; + + hdr = (struct cmsg_hdr *)skb->data; + + return be16_to_cpu(hdr->tag); +} + +static struct sk_buff *__nfp_bpf_reply(struct nfp_app_bpf *bpf, u16 tag) +{ + unsigned int msg_tag; + struct sk_buff *skb; + + skb_queue_walk(&bpf->cmsg_replies, skb) { + msg_tag = nfp_bpf_cmsg_get_tag(skb); + if (msg_tag == tag) { + nfp_bpf_free_tag(bpf, tag); + __skb_unlink(skb, &bpf->cmsg_replies); + return skb; + } + } + + return NULL; +} + +static struct sk_buff *nfp_bpf_reply(struct nfp_app_bpf *bpf, u16 tag) +{ + struct sk_buff *skb; + + nfp_ctrl_lock(bpf->app->ctrl); + skb = __nfp_bpf_reply(bpf, tag); + nfp_ctrl_unlock(bpf->app->ctrl); + + return skb; +} + +static struct sk_buff *nfp_bpf_reply_drop_tag(struct nfp_app_bpf *bpf, u16 tag) +{ + struct sk_buff *skb; + + nfp_ctrl_lock(bpf->app->ctrl); + skb = __nfp_bpf_reply(bpf, tag); + if (!skb) + nfp_bpf_free_tag(bpf, tag); + nfp_ctrl_unlock(bpf->app->ctrl); + + return skb; +} + +static struct sk_buff * +nfp_bpf_cmsg_wait_reply(struct nfp_app_bpf *bpf, enum nfp_bpf_cmsg_type type, + int tag) +{ + struct sk_buff *skb; + int err; + + err = wait_event_interruptible_timeout(bpf->cmsg_wq, + skb = nfp_bpf_reply(bpf, tag), + msecs_to_jiffies(5000)); + /* We didn't get a response - try last time and atomically drop + * the tag even if no response is matched. + */ + if (!skb) + skb = nfp_bpf_reply_drop_tag(bpf, tag); + if (err < 0) { + cmsg_warn(bpf, "%s waiting for response to 0x%02x: %d\n", + err == ERESTARTSYS ? "interrupted" : "error", + type, err); + return ERR_PTR(err); + } + if (!skb) { + cmsg_warn(bpf, "timeout waiting for response to 0x%02x\n", + type); + return ERR_PTR(-ETIMEDOUT); + } + + return skb; +} + +struct sk_buff * +nfp_bpf_cmsg_communicate(struct nfp_app_bpf *bpf, struct sk_buff *skb, + enum nfp_bpf_cmsg_type type, unsigned int reply_size) +{ + struct cmsg_hdr *hdr; + int tag; + + nfp_ctrl_lock(bpf->app->ctrl); + tag = nfp_bpf_alloc_tag(bpf); + if (tag < 0) { + nfp_ctrl_unlock(bpf->app->ctrl); + dev_kfree_skb_any(skb); + return ERR_PTR(tag); + } + + hdr = (void *)skb->data; + hdr->ver = CMSG_MAP_ABI_VERSION; + hdr->type = type; + hdr->tag = cpu_to_be16(tag); + + __nfp_app_ctrl_tx(bpf->app, skb); + + nfp_ctrl_unlock(bpf->app->ctrl); + + skb = nfp_bpf_cmsg_wait_reply(bpf, type, tag); + if (IS_ERR(skb)) + return skb; + + hdr = (struct cmsg_hdr *)skb->data; + /* 0 reply_size means caller will do the validation */ + if (reply_size && skb->len != reply_size) { + cmsg_warn(bpf, "cmsg drop - wrong size %d != %d!\n", + skb->len, reply_size); + goto err_free; + } + if (hdr->type != __CMSG_REPLY(type)) { + cmsg_warn(bpf, "cmsg drop - wrong type 0x%02x != 0x%02lx!\n", + hdr->type, __CMSG_REPLY(type)); + goto err_free; + } + + return skb; +err_free: + dev_kfree_skb_any(skb); + return ERR_PTR(-EIO); +} + +void nfp_bpf_ctrl_msg_rx(struct nfp_app *app, struct sk_buff *skb) +{ + struct nfp_app_bpf *bpf = app->priv; + unsigned int tag; + + if (unlikely(skb->len < sizeof(struct cmsg_reply_map_simple))) { + cmsg_warn(bpf, "cmsg drop - too short %d!\n", skb->len); + goto err_free; + } + + nfp_ctrl_lock(bpf->app->ctrl); + + tag = nfp_bpf_cmsg_get_tag(skb); + if (unlikely(!test_bit(tag, bpf->tag_allocator))) { + cmsg_warn(bpf, "cmsg drop - no one is waiting for tag %u!\n", + tag); + goto err_unlock; + } + + __skb_queue_tail(&bpf->cmsg_replies, skb); + wake_up_interruptible_all(&bpf->cmsg_wq); + + nfp_ctrl_unlock(bpf->app->ctrl); + + return; +err_unlock: + nfp_ctrl_unlock(bpf->app->ctrl); +err_free: + dev_kfree_skb_any(skb); +} diff --git a/drivers/net/ethernet/netronome/nfp/bpf/fw.h b/drivers/net/ethernet/netronome/nfp/bpf/fw.h index 7206aa1522db..107676b34760 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/fw.h +++ b/drivers/net/ethernet/netronome/nfp/bpf/fw.h @@ -51,4 +51,26 @@ struct nfp_bpf_cap_tlv_adjust_head { #define NFP_BPF_ADJUST_HEAD_NO_META BIT(0) +/* + * Types defined for map related control messages + */ +#define CMSG_MAP_ABI_VERSION 1 + +enum nfp_bpf_cmsg_type { + __CMSG_TYPE_MAP_MAX, +}; + +#define CMSG_TYPE_MAP_REPLY_BIT 7 +#define __CMSG_REPLY(req) (BIT(CMSG_TYPE_MAP_REPLY_BIT) | (req)) + +struct cmsg_hdr { + u8 type; + u8 ver; + __be16 tag; +}; + +struct cmsg_reply_map_simple { + struct cmsg_hdr hdr; + __be32 rc; +}; #endif diff --git a/drivers/net/ethernet/netronome/nfp/bpf/main.c b/drivers/net/ethernet/netronome/nfp/bpf/main.c index 2506d1139fa9..74d42bdcb905 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/main.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/main.c @@ -288,6 +288,8 @@ static int nfp_bpf_init(struct nfp_app *app) bpf->app = app; app->priv = bpf; + skb_queue_head_init(&bpf->cmsg_replies); + init_waitqueue_head(&bpf->cmsg_wq); INIT_LIST_HEAD(&bpf->map_list); err = nfp_bpf_parse_capabilities(app); @@ -305,6 +307,7 @@ static void nfp_bpf_clean(struct nfp_app *app) { struct nfp_app_bpf *bpf = app->priv; + WARN_ON(!skb_queue_empty(&bpf->cmsg_replies)); WARN_ON(!list_empty(&bpf->map_list)); kfree(bpf); } @@ -321,6 +324,8 @@ const struct nfp_app_type app_bpf = { .vnic_alloc = nfp_bpf_vnic_alloc, .vnic_free = nfp_bpf_vnic_free, + .ctrl_msg_rx = nfp_bpf_ctrl_msg_rx, + .setup_tc = nfp_bpf_setup_tc, .tc_busy = nfp_bpf_tc_busy, .bpf = nfp_ndo_bpf, diff --git a/drivers/net/ethernet/netronome/nfp/bpf/main.h b/drivers/net/ethernet/netronome/nfp/bpf/main.h index 71c58d25858b..fd9987d6a374 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/main.h +++ b/drivers/net/ethernet/netronome/nfp/bpf/main.h @@ -37,10 +37,14 @@ #include #include #include +#include #include +#include #include +#include #include "../nfp_asm.h" +#include "fw.h" /* For branch fixup logic use up-most byte of branch instruction as scratch * area. Remember to clear this before sending instructions to HW! @@ -82,6 +86,13 @@ enum pkt_vec { * struct nfp_app_bpf - bpf app priv structure * @app: backpointer to the app * + * @tag_allocator: bitmap of control message tags in use + * @tag_alloc_next: next tag bit to allocate + * @tag_alloc_last: next tag bit to be freed + * + * @cmsg_replies: received cmsg replies waiting to be consumed + * @cmsg_wq: work queue for waiting for cmsg replies + * * @map_list: list of offloaded maps * * @adjust_head: adjust head capability @@ -94,6 +105,13 @@ enum pkt_vec { struct nfp_app_bpf { struct nfp_app *app; + DECLARE_BITMAP(tag_allocator, U16_MAX + 1); + u16 tag_alloc_next; + u16 tag_alloc_last; + + struct sk_buff_head cmsg_replies; + struct wait_queue_head cmsg_wq; + struct list_head map_list; struct nfp_bpf_cap_adjust_head { @@ -270,4 +288,9 @@ int nfp_net_bpf_offload(struct nfp_net *nn, struct bpf_prog *prog, struct nfp_insn_meta * nfp_bpf_goto_meta(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, unsigned int insn_idx, unsigned int n_insns); + +struct sk_buff * +nfp_bpf_cmsg_communicate(struct nfp_app_bpf *bpf, struct sk_buff *skb, + enum nfp_bpf_cmsg_type type, unsigned int reply_size); +void nfp_bpf_ctrl_msg_rx(struct nfp_app *app, struct sk_buff *skb); #endif diff --git a/drivers/net/ethernet/netronome/nfp/nfp_app.h b/drivers/net/ethernet/netronome/nfp/nfp_app.h index ee67a7202819..9dd4158bf89e 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_app.h +++ b/drivers/net/ethernet/netronome/nfp/nfp_app.h @@ -160,6 +160,7 @@ struct nfp_app { void *priv; }; +bool __nfp_ctrl_tx(struct nfp_net *nn, struct sk_buff *skb); bool nfp_ctrl_tx(struct nfp_net *nn, struct sk_buff *skb); static inline int nfp_app_init(struct nfp_app *app) @@ -313,6 +314,14 @@ static inline int nfp_app_xdp_offload(struct nfp_app *app, struct nfp_net *nn, return app->type->xdp_offload(app, nn, prog); } +static inline bool __nfp_app_ctrl_tx(struct nfp_app *app, struct sk_buff *skb) +{ + trace_devlink_hwmsg(priv_to_devlink(app->pf), false, 0, + skb->data, skb->len); + + return __nfp_ctrl_tx(app->ctrl, skb); +} + static inline bool nfp_app_ctrl_tx(struct nfp_app *app, struct sk_buff *skb) { trace_devlink_hwmsg(priv_to_devlink(app->pf), false, 0, diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net.h b/drivers/net/ethernet/netronome/nfp/nfp_net.h index 3801c52098d5..3d61bab0ff2f 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net.h +++ b/drivers/net/ethernet/netronome/nfp/nfp_net.h @@ -836,6 +836,18 @@ static inline const char *nfp_net_name(struct nfp_net *nn) return nn->dp.netdev ? nn->dp.netdev->name : "ctrl"; } +static inline void nfp_ctrl_lock(struct nfp_net *nn) + __acquires(&nn->r_vecs[0].lock) +{ + spin_lock_bh(&nn->r_vecs[0].lock); +} + +static inline void nfp_ctrl_unlock(struct nfp_net *nn) + __releases(&nn->r_vecs[0].lock) +{ + spin_unlock_bh(&nn->r_vecs[0].lock); +} + /* Globals */ extern const char nfp_driver_version[]; diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c index 773089442b64..7972e6dd9547 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c @@ -1917,6 +1917,13 @@ nfp_ctrl_tx_one(struct nfp_net *nn, struct nfp_net_r_vector *r_vec, return false; } +bool __nfp_ctrl_tx(struct nfp_net *nn, struct sk_buff *skb) +{ + struct nfp_net_r_vector *r_vec = &nn->r_vecs[0]; + + return nfp_ctrl_tx_one(nn, r_vec, skb, false); +} + bool nfp_ctrl_tx(struct nfp_net *nn, struct sk_buff *skb) { struct nfp_net_r_vector *r_vec = &nn->r_vecs[0]; From patchwork Fri Jan 5 06:09:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 855922 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="BJ6QaGW7"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zCZ6R2pFNz9s4q for ; Fri, 5 Jan 2018 17:10:55 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751476AbeAEGKw (ORCPT ); Fri, 5 Jan 2018 01:10:52 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:37129 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751376AbeAEGKR (ORCPT ); Fri, 5 Jan 2018 01:10:17 -0500 Received: by mail-pl0-f68.google.com with SMTP id s3so2543315plp.4 for ; Thu, 04 Jan 2018 22:10:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=V3SHtQ/1N7O4C7o6pbzyoMGY1VG0ROUJRh4erlouK5U=; b=BJ6QaGW70OFVQ8kTVwuVZ7vCndHyTv9Wdeey7yMAJr/x3K6HEy2PJvjSluy08zQzF2 ykvbF0qQ0YGj3Vam1Vtk6J+Dl1hYPSH9xwnsVs2PJUuS0Qo9oq++fvFW1FrweTzo6AJS 4p5uHut9cti/udMtWqeUgSvbeJ1QeqoUAZmpuu/YgTAubD2fp2tXtSeNmyYFQAfbE/6K VxejkVTcE9xybjpuX8k3xhFI2CenVX/LMUlqhmi2s3r5o+A7DcvxYDESIuR9zfDQP//2 NP8c4tO4bC0x/3D8zH8Fzaaf6JZIQpBGTqwaIISvC4HUAhWDbUYcwK5alNXjB4amBVvw fPqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=V3SHtQ/1N7O4C7o6pbzyoMGY1VG0ROUJRh4erlouK5U=; b=uXI/vOQr4UUf0CtvzDT+eJeeyuPMi6N7cfk4Qhm712B9n1c5aNNObBzaKJIw7qVdAa TXSL5TbPX96svzn9RQPJ/cqQ8WUZSHMjHsSFsYgskGVuKI1C0Sotm9gEEoF/yJa4XriO f7XSDMHDgijX0vC8bn34ekoiv2AeRmAsn7SKeWZ7hT2fX1X7F8UkLpNUcWeE5v+SlVkG +58a8WHXqxLC/2oumGU+nxmiLtmLDebhPwsfaLc2JQYH7h2FwWOP4PS3QVxuU3akqd/f sz6K4uSwssJsZkOiAyXDanxBBYpynJnYJQxaKFHE4fSqC2RVoqRD5evgHN9fKJ5goh6T sDCw== X-Gm-Message-State: AKGB3mJUNj9j4ABvvoPO2l95zt/0adc6xGUyFPIPVwbdKnz9ULQjJpsV cirC3LitmH1v+fqG+AKTUcswbiOr X-Google-Smtp-Source: ACJfBotACmHz+6dvHs0O89hQyax3ig6hY+/NwDXcJK1sjUx7DemZdX/yCAl3WIAs1qEPlS5e9zt7hw== X-Received: by 10.84.248.68 with SMTP id e4mr2036949pln.296.1515132616163; Thu, 04 Jan 2018 22:10:16 -0800 (PST) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o64sm2444086pga.1.2018.01.04.22.10.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Jan 2018 22:10:15 -0800 (PST) From: Jakub Kicinski To: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, daniel@iogearbox.net, davem@davemloft.net Cc: oss-drivers@netronome.com, tehnerd@fb.com, Jakub Kicinski Subject: [PATCH bpf-next 12/16] nfp: bpf: implement helpers for FW map ops Date: Thu, 4 Jan 2018 22:09:27 -0800 Message-Id: <20180105060931.30815-13-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180105060931.30815-1-jakub.kicinski@netronome.com> References: <20180105060931.30815-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Implement calls for FW map communication. Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet --- drivers/net/ethernet/netronome/nfp/bpf/cmsg.c | 210 +++++++++++++++++++++++++- drivers/net/ethernet/netronome/nfp/bpf/fw.h | 65 ++++++++ drivers/net/ethernet/netronome/nfp/bpf/main.h | 17 ++- 3 files changed, 288 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/bpf/cmsg.c b/drivers/net/ethernet/netronome/nfp/bpf/cmsg.c index 46753ee9f7c5..71e6586acc36 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/cmsg.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/cmsg.c @@ -31,6 +31,7 @@ * SOFTWARE. */ +#include #include #include #include @@ -79,6 +80,28 @@ static void nfp_bpf_free_tag(struct nfp_app_bpf *bpf, u16 tag) bpf->tag_alloc_last++; } +static struct sk_buff * +nfp_bpf_cmsg_alloc(struct nfp_app_bpf *bpf, unsigned int size) +{ + struct sk_buff *skb; + + skb = nfp_app_ctrl_msg_alloc(bpf->app, size, GFP_KERNEL); + skb_put(skb, size); + + return skb; +} + +static struct sk_buff * +nfp_bpf_cmsg_map_req_alloc(struct nfp_app_bpf *bpf, unsigned int n) +{ + unsigned int size; + + size = sizeof(struct cmsg_req_map_op); + size += sizeof(struct cmsg_key_value_pair) * n; + + return nfp_bpf_cmsg_alloc(bpf, size); +} + static unsigned int nfp_bpf_cmsg_get_tag(struct sk_buff *skb) { struct cmsg_hdr *hdr; @@ -159,7 +182,7 @@ nfp_bpf_cmsg_wait_reply(struct nfp_app_bpf *bpf, enum nfp_bpf_cmsg_type type, return skb; } -struct sk_buff * +static struct sk_buff * nfp_bpf_cmsg_communicate(struct nfp_app_bpf *bpf, struct sk_buff *skb, enum nfp_bpf_cmsg_type type, unsigned int reply_size) { @@ -206,6 +229,191 @@ nfp_bpf_cmsg_communicate(struct nfp_app_bpf *bpf, struct sk_buff *skb, return ERR_PTR(-EIO); } +static int +nfp_bpf_ctrl_rc_to_errno(struct nfp_app_bpf *bpf, + struct cmsg_reply_map_simple *reply) +{ + static const int res_table[] = { + [CMSG_RC_SUCCESS] = 0, + [CMSG_RC_ERR_MAP_FD] = -EBADFD, + [CMSG_RC_ERR_MAP_NOENT] = -ENOENT, + [CMSG_RC_ERR_MAP_ERR] = -EINVAL, + [CMSG_RC_ERR_MAP_PARSE] = -EIO, + [CMSG_RC_ERR_MAP_EXIST] = -EEXIST, + [CMSG_RC_ERR_MAP_NOMEM] = -ENOMEM, + [CMSG_RC_ERR_MAP_E2BIG] = -E2BIG, + }; + u32 rc; + + rc = be32_to_cpu(reply->rc); + if (rc >= ARRAY_SIZE(res_table)) { + cmsg_warn(bpf, "FW responded with invalid status: %u\n", rc); + return -EIO; + } + + return res_table[rc]; +} + +long long int +nfp_bpf_ctrl_alloc_map(struct nfp_app_bpf *bpf, struct bpf_map *map) +{ + struct cmsg_reply_map_alloc_tbl *reply; + struct cmsg_req_map_alloc_tbl *req; + struct sk_buff *skb; + u32 tid; + int err; + + skb = nfp_bpf_cmsg_alloc(bpf, sizeof(*req)); + if (!skb) + return -ENOMEM; + + req = (void *)skb->data; + req->key_size = cpu_to_be32(map->key_size); + req->value_size = cpu_to_be32(map->value_size); + req->max_entries = cpu_to_be32(map->max_entries); + req->map_type = cpu_to_be32(map->map_type); + req->map_flags = 0; + + skb = nfp_bpf_cmsg_communicate(bpf, skb, CMSG_TYPE_MAP_ALLOC, + sizeof(*reply)); + if (IS_ERR(skb)) + return PTR_ERR(skb); + + reply = (void *)skb->data; + err = nfp_bpf_ctrl_rc_to_errno(bpf, &reply->reply_hdr); + if (err) + goto err_free; + + tid = be32_to_cpu(reply->tid); + dev_consume_skb_any(skb); + + return tid; +err_free: + dev_kfree_skb_any(skb); + return err; +} + +void nfp_bpf_ctrl_free_map(struct nfp_app_bpf *bpf, struct nfp_bpf_map *nfp_map) +{ + struct cmsg_reply_map_free_tbl *reply; + struct cmsg_req_map_free_tbl *req; + struct sk_buff *skb; + int err; + + skb = nfp_bpf_cmsg_alloc(bpf, sizeof(*req)); + if (!skb) { + cmsg_warn(bpf, "leaking map - failed to allocate msg\n"); + return; + } + + req = (void *)skb->data; + req->tid = cpu_to_be32(nfp_map->tid); + + skb = nfp_bpf_cmsg_communicate(bpf, skb, CMSG_TYPE_MAP_FREE, + sizeof(*reply)); + if (IS_ERR(skb)) { + cmsg_warn(bpf, "leaking map - I/O error\n"); + return; + } + + reply = (void *)skb->data; + err = nfp_bpf_ctrl_rc_to_errno(bpf, &reply->reply_hdr); + if (err) + cmsg_warn(bpf, "leaking map - FW responded with: %d\n", err); + + dev_consume_skb_any(skb); +} + +static int +nfp_bpf_ctrl_entry_op(struct bpf_offloaded_map *offmap, + enum nfp_bpf_cmsg_type op, + u8 *key, u8 *value, u64 flags, u8 *out_key, u8 *out_value) +{ + struct nfp_bpf_map *nfp_map = offmap->dev_priv; + struct nfp_app_bpf *bpf = nfp_map->bpf; + struct bpf_map *map = &offmap->map; + struct cmsg_reply_map_op *reply; + struct cmsg_req_map_op *req; + struct sk_buff *skb; + int err; + + /* FW messages have no space for more than 32 bits of flags */ + if (flags >> 32) + return -EOPNOTSUPP; + + skb = nfp_bpf_cmsg_map_req_alloc(bpf, 1); + if (!skb) + return -ENOMEM; + + req = (void *)skb->data; + req->tid = cpu_to_be32(nfp_map->tid); + req->count = cpu_to_be32(1); + req->flags = cpu_to_be32(flags); + + /* Copy inputs */ + if (key) + memcpy(&req->elem[0].key, key, map->key_size); + if (value) + memcpy(&req->elem[0].value, value, map->value_size); + + skb = nfp_bpf_cmsg_communicate(bpf, skb, op, + sizeof(*reply) + sizeof(*reply->elem)); + if (IS_ERR(skb)) + return PTR_ERR(skb); + + reply = (void *)skb->data; + err = nfp_bpf_ctrl_rc_to_errno(bpf, &reply->reply_hdr); + if (err) + goto err_free; + + /* Copy outputs */ + if (out_key) + memcpy(out_key, &reply->elem[0].key, map->key_size); + if (out_value) + memcpy(out_value, &reply->elem[0].value, map->value_size); + + dev_consume_skb_any(skb); + + return 0; +err_free: + dev_kfree_skb_any(skb); + return err; +} + +int nfp_bpf_ctrl_update_entry(struct bpf_offloaded_map *offmap, + void *key, void *value, u64 flags) +{ + return nfp_bpf_ctrl_entry_op(offmap, CMSG_TYPE_MAP_UPDATE, + key, value, flags, NULL, NULL); +} + +int nfp_bpf_ctrl_del_entry(struct bpf_offloaded_map *offmap, void *key) +{ + return nfp_bpf_ctrl_entry_op(offmap, CMSG_TYPE_MAP_DELETE, + key, NULL, 0, NULL, NULL); +} + +int nfp_bpf_ctrl_lookup_entry(struct bpf_offloaded_map *offmap, + void *key, void *value) +{ + return nfp_bpf_ctrl_entry_op(offmap, CMSG_TYPE_MAP_LOOKUP, + key, NULL, 0, NULL, value); +} + +int nfp_bpf_ctrl_getfirst_entry(struct bpf_offloaded_map *offmap, + void *next_key) +{ + return nfp_bpf_ctrl_entry_op(offmap, CMSG_TYPE_MAP_GETFIRST, + NULL, NULL, 0, next_key, NULL); +} + +int nfp_bpf_ctrl_getnext_entry(struct bpf_offloaded_map *offmap, + void *key, void *next_key) +{ + return nfp_bpf_ctrl_entry_op(offmap, CMSG_TYPE_MAP_GETNEXT, + key, NULL, 0, next_key, NULL); +} + void nfp_bpf_ctrl_msg_rx(struct nfp_app *app, struct sk_buff *skb) { struct nfp_app_bpf *bpf = app->priv; diff --git a/drivers/net/ethernet/netronome/nfp/bpf/fw.h b/drivers/net/ethernet/netronome/nfp/bpf/fw.h index 107676b34760..e0ff68fc9562 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/fw.h +++ b/drivers/net/ethernet/netronome/nfp/bpf/fw.h @@ -57,12 +57,33 @@ struct nfp_bpf_cap_tlv_adjust_head { #define CMSG_MAP_ABI_VERSION 1 enum nfp_bpf_cmsg_type { + CMSG_TYPE_MAP_ALLOC = 1, + CMSG_TYPE_MAP_FREE = 2, + CMSG_TYPE_MAP_LOOKUP = 3, + CMSG_TYPE_MAP_UPDATE = 4, + CMSG_TYPE_MAP_DELETE = 5, + CMSG_TYPE_MAP_GETNEXT = 6, + CMSG_TYPE_MAP_GETFIRST = 7, __CMSG_TYPE_MAP_MAX, }; #define CMSG_TYPE_MAP_REPLY_BIT 7 #define __CMSG_REPLY(req) (BIT(CMSG_TYPE_MAP_REPLY_BIT) | (req)) +#define CMSG_MAP_KEY_LW 16 +#define CMSG_MAP_VALUE_LW 16 + +enum nfp_bpf_cmsg_status { + CMSG_RC_SUCCESS = 0, + CMSG_RC_ERR_MAP_FD = 1, + CMSG_RC_ERR_MAP_NOENT = 2, + CMSG_RC_ERR_MAP_ERR = 3, + CMSG_RC_ERR_MAP_PARSE = 4, + CMSG_RC_ERR_MAP_EXIST = 5, + CMSG_RC_ERR_MAP_NOMEM = 6, + CMSG_RC_ERR_MAP_E2BIG = 7, +}; + struct cmsg_hdr { u8 type; u8 ver; @@ -73,4 +94,48 @@ struct cmsg_reply_map_simple { struct cmsg_hdr hdr; __be32 rc; }; + +struct cmsg_req_map_alloc_tbl { + struct cmsg_hdr hdr; + __be32 key_size; /* in bytes */ + __be32 value_size; /* in bytes */ + __be32 max_entries; + __be32 map_type; + __be32 map_flags; /* reserved */ +}; + +struct cmsg_reply_map_alloc_tbl { + struct cmsg_reply_map_simple reply_hdr; + __be32 tid; +}; + +struct cmsg_req_map_free_tbl { + struct cmsg_hdr hdr; + __be32 tid; +}; + +struct cmsg_reply_map_free_tbl { + struct cmsg_reply_map_simple reply_hdr; + __be32 count; +}; + +struct cmsg_key_value_pair { + __be32 key[CMSG_MAP_KEY_LW]; + __be32 value[CMSG_MAP_VALUE_LW]; +}; + +struct cmsg_req_map_op { + struct cmsg_hdr hdr; + __be32 tid; + __be32 count; + __be32 flags; + struct cmsg_key_value_pair elem[0]; +}; + +struct cmsg_reply_map_op { + struct cmsg_reply_map_simple reply_hdr; + __be32 count; + __be32 resv; + struct cmsg_key_value_pair elem[0]; +}; #endif diff --git a/drivers/net/ethernet/netronome/nfp/bpf/main.h b/drivers/net/ethernet/netronome/nfp/bpf/main.h index fd9987d6a374..51b8c30ab5db 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/main.h +++ b/drivers/net/ethernet/netronome/nfp/bpf/main.h @@ -289,8 +289,19 @@ struct nfp_insn_meta * nfp_bpf_goto_meta(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, unsigned int insn_idx, unsigned int n_insns); -struct sk_buff * -nfp_bpf_cmsg_communicate(struct nfp_app_bpf *bpf, struct sk_buff *skb, - enum nfp_bpf_cmsg_type type, unsigned int reply_size); +long long int +nfp_bpf_ctrl_alloc_map(struct nfp_app_bpf *bpf, struct bpf_map *map); +void +nfp_bpf_ctrl_free_map(struct nfp_app_bpf *bpf, struct nfp_bpf_map *nfp_map); +int nfp_bpf_ctrl_getfirst_entry(struct bpf_offloaded_map *offmap, + void *next_key); +int nfp_bpf_ctrl_update_entry(struct bpf_offloaded_map *offmap, + void *key, void *value, u64 flags); +int nfp_bpf_ctrl_del_entry(struct bpf_offloaded_map *offmap, void *key); +int nfp_bpf_ctrl_lookup_entry(struct bpf_offloaded_map *offmap, + void *key, void *value); +int nfp_bpf_ctrl_getnext_entry(struct bpf_offloaded_map *offmap, + void *key, void *next_key); + void nfp_bpf_ctrl_msg_rx(struct nfp_app *app, struct sk_buff *skb); #endif From patchwork Fri Jan 5 06:09:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 855923 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="JKGjVxGV"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zCZ6S5D7fz9s75 for ; Fri, 5 Jan 2018 17:10:56 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751469AbeAEGKv (ORCPT ); Fri, 5 Jan 2018 01:10:51 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:42929 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751387AbeAEGKR (ORCPT ); Fri, 5 Jan 2018 01:10:17 -0500 Received: by mail-pg0-f67.google.com with SMTP id q67so1661428pga.9 for ; Thu, 04 Jan 2018 22:10:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=rSYCmdSKDwUW1UV+WiZz+oRHeOR9kaBeheuCi+F/Obg=; b=JKGjVxGV92OMQG+yfSa1xbuDRyGZ0tmoVOzkcex+4f0dhz9NdwC1JCk/uNRV7Zocsb BmOYEWXASqhl8hKWTgUgmrohwy+Y7sXT6YC0Kr+lYlUbTmmG47bO4SeS6reG+gGRUuHP wQrYr2GwrIy6yWaeaeQbhx33W6Pz+JapPoG6myTNc51NFyKcBFdYYFSnOlWZaSC1jw6X 5SzeEaY7L3PWXnSPwT9xOU1Zq7aNGI0v9fUHscfdUhuoeDOCQA4HLbwH2Rch9m28J36d A9WeL1Wgjw8hL21OPes8RPBORYDyQpRp6ybEv1txoK4jNt/FHvM+N8UrzmlCrHmXOymb 54ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=rSYCmdSKDwUW1UV+WiZz+oRHeOR9kaBeheuCi+F/Obg=; b=cSdVyJuyq4aRFCnN26o9n5+NQ25lP7fSFSgA7uzOWfgjC3vru15+bFfJmnhg+4xSSN EdaCmwJqeQ5dkZzS5A+teJxLOdHvHdMuy69OcBDL0CEkRfj+hrHKCsavuC8UPYWF1bvK XxgYMYj0Jq9ESjLv9dXi3tUS/Mm++fm2GXNBxieJcS8anmdHxdj1SXcEL/7Fk4W21m4L /tgh3tJ+/ia6KCAbzLYeERYgY2MoUmIQ40KyL7OwJqhuYKaz8e/cT8alumgSTgMNcFn7 G730SCD8RYejU80nESsglhwnzhIfCNcZnu8218ulsT+f+eqVeRrNKvjo7QGnWaR2iPOj 9XAA== X-Gm-Message-State: AKGB3mKCCy5BBSEr3waUKQxa3Kc4eGuD6HEOh4HzlGFwwIJ2TGzonhzZ gmkuocpXPtky/VWyQYIiXxhuIjBs X-Google-Smtp-Source: ACJfBotQtwRiSYDpuW1AU3y106hDLhrP6C20HRlF5D4KyGWBhUZgsLGGbXc76Ny3k765ga5Pdik8fA== X-Received: by 10.99.44.66 with SMTP id s63mr1628454pgs.257.1515132617013; Thu, 04 Jan 2018 22:10:17 -0800 (PST) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o64sm2444086pga.1.2018.01.04.22.10.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Jan 2018 22:10:16 -0800 (PST) From: Jakub Kicinski To: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, daniel@iogearbox.net, davem@davemloft.net Cc: oss-drivers@netronome.com, tehnerd@fb.com, Jakub Kicinski Subject: [PATCH bpf-next 13/16] nfp: bpf: parse function call and map capabilities Date: Thu, 4 Jan 2018 22:09:28 -0800 Message-Id: <20180105060931.30815-14-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180105060931.30815-1-jakub.kicinski@netronome.com> References: <20180105060931.30815-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Parse helper function and supported map FW TLV capabilities. Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet --- drivers/net/ethernet/netronome/nfp/bpf/fw.h | 16 +++++++++ drivers/net/ethernet/netronome/nfp/bpf/main.c | 47 +++++++++++++++++++++++++++ drivers/net/ethernet/netronome/nfp/bpf/main.h | 24 ++++++++++++++ 3 files changed, 87 insertions(+) diff --git a/drivers/net/ethernet/netronome/nfp/bpf/fw.h b/drivers/net/ethernet/netronome/nfp/bpf/fw.h index e0ff68fc9562..cfcc7bcb2c67 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/fw.h +++ b/drivers/net/ethernet/netronome/nfp/bpf/fw.h @@ -38,7 +38,14 @@ #include enum bpf_cap_tlv_type { + NFP_BPF_CAP_TYPE_FUNC = 1, NFP_BPF_CAP_TYPE_ADJUST_HEAD = 2, + NFP_BPF_CAP_TYPE_MAPS = 3, +}; + +struct nfp_bpf_cap_tlv_func { + __le32 func_id; + __le32 func_addr; }; struct nfp_bpf_cap_tlv_adjust_head { @@ -51,6 +58,15 @@ struct nfp_bpf_cap_tlv_adjust_head { #define NFP_BPF_ADJUST_HEAD_NO_META BIT(0) +struct nfp_bpf_cap_tlv_maps { + __le32 types; + __le32 max_maps; + __le32 max_elems; + __le32 max_key_sz; + __le32 max_val_sz; + __le32 max_elem_sz; +}; + /* * Types defined for map related control messages */ diff --git a/drivers/net/ethernet/netronome/nfp/bpf/main.c b/drivers/net/ethernet/netronome/nfp/bpf/main.c index 74d42bdcb905..251cd198e710 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/main.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/main.c @@ -226,6 +226,45 @@ nfp_bpf_parse_cap_adjust_head(struct nfp_app_bpf *bpf, void __iomem *value, return 0; } +static int +nfp_bpf_parse_cap_func(struct nfp_app_bpf *bpf, void __iomem *value, u32 length) +{ + struct nfp_bpf_cap_tlv_func __iomem *cap = value; + + if (length < sizeof(*cap)) { + nfp_err(bpf->app->cpp, "truncated function TLV: %d\n", length); + return -EINVAL; + } + + switch (readl(&cap->func_id)) { + case BPF_FUNC_map_lookup_elem: + bpf->helpers.map_lookup = readl(&cap->func_addr); + break; + } + + return 0; +} + +static int +nfp_bpf_parse_cap_maps(struct nfp_app_bpf *bpf, void __iomem *value, u32 length) +{ + struct nfp_bpf_cap_tlv_maps __iomem *cap = value; + + if (length < sizeof(*cap)) { + nfp_err(bpf->app->cpp, "truncated maps TLV: %d\n", length); + return -EINVAL; + } + + bpf->maps.types = readl(&cap->types); + bpf->maps.max_maps = readl(&cap->max_maps); + bpf->maps.max_elems = readl(&cap->max_elems); + bpf->maps.max_key_sz = readl(&cap->max_key_sz); + bpf->maps.max_val_sz = readl(&cap->max_val_sz); + bpf->maps.max_elem_sz = readl(&cap->max_elem_sz); + + return 0; +} + static int nfp_bpf_parse_capabilities(struct nfp_app *app) { struct nfp_cpp *cpp = app->pf->cpp; @@ -251,11 +290,19 @@ static int nfp_bpf_parse_capabilities(struct nfp_app *app) goto err_release_free; switch (type) { + case NFP_BPF_CAP_TYPE_FUNC: + if (nfp_bpf_parse_cap_func(app->priv, value, length)) + goto err_release_free; + break; case NFP_BPF_CAP_TYPE_ADJUST_HEAD: if (nfp_bpf_parse_cap_adjust_head(app->priv, value, length)) goto err_release_free; break; + case NFP_BPF_CAP_TYPE_MAPS: + if (nfp_bpf_parse_cap_maps(app->priv, value, length)) + goto err_release_free; + break; default: nfp_dbg(cpp, "unknown BPF capability: %d\n", type); break; diff --git a/drivers/net/ethernet/netronome/nfp/bpf/main.h b/drivers/net/ethernet/netronome/nfp/bpf/main.h index 51b8c30ab5db..9dd76bf248b1 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/main.h +++ b/drivers/net/ethernet/netronome/nfp/bpf/main.h @@ -101,6 +101,17 @@ enum pkt_vec { * @off_max: maximum packet offset within buffer required * @guaranteed_sub: amount of negative adjustment guaranteed possible * @guaranteed_add: amount of positive adjustment guaranteed possible + * + * @maps: map capability + * @types: supported map types + * @max_maps: max number of maps supported + * @max_elems: max number of entries in each map + * @max_key_sz: max size of map key + * @max_val_sz: max size of map value + * @max_elem_sz: max size of map entry (key + value) + * + * @helpers: helper addressess for various calls + * @map_lookup: map lookup helper address */ struct nfp_app_bpf { struct nfp_app *app; @@ -121,6 +132,19 @@ struct nfp_app_bpf { int guaranteed_sub; int guaranteed_add; } adjust_head; + + struct { + u32 types; + u32 max_maps; + u32 max_elems; + u32 max_key_sz; + u32 max_val_sz; + u32 max_elem_sz; + } maps; + + struct { + u32 map_lookup; + } helpers; }; /** From patchwork Fri Jan 5 06:09:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 855924 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="qLctmiQx"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zCZ6Y4mKgz9s4q for ; Fri, 5 Jan 2018 17:11:01 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751458AbeAEGKv (ORCPT ); Fri, 5 Jan 2018 01:10:51 -0500 Received: from mail-pf0-f196.google.com ([209.85.192.196]:36771 "EHLO mail-pf0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751298AbeAEGKS (ORCPT ); Fri, 5 Jan 2018 01:10:18 -0500 Received: by mail-pf0-f196.google.com with SMTP id p84so1833432pfd.3 for ; Thu, 04 Jan 2018 22:10:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Qcyh4Y1Zm1RsJt01l69ABUtr7MolBhKq3gQtNV+wn7M=; b=qLctmiQxAEjWrJsH9ghQ4VFZZZvjO6aY4TkTs7LtExec3Md3jhKWrjg3JjOFa6E/JY jesUkz94MiUY5FEz5C2crRUziP0YRpPkplRxbt+k/Bs8xs7gen60xUeiBUJA8lHImMcc K5KLysOE/F82FbIIZooBstCa7aooZzBivQemgbEtfAZ9NKt9YhQKsv1qF4Y5NftLVRCu GtEjH481xOP9obYa3SHPUcMIwE1hEDLWgo2x0qDHuyb2Zr68g9hvrJISW2OOXEp48ENp GSakbIFtnh4Wj7pM/UTdP9PDfr3oBmjnEHdzNFiTXYC8c1UU7U/HxURX+hItSWT+thiV 2zBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Qcyh4Y1Zm1RsJt01l69ABUtr7MolBhKq3gQtNV+wn7M=; b=CtaLGB021wLsZTFBrbWSCS1MXNeBSSWH7S68fZO9BSE62qNTVFvBA4yu2/70TikXfI eLCDx/y9fMTD6c99RuXCxLjop5tleM1Dj66GhTgiAmdz5chuue+AqWLIemEHIt1ME12o eKwcSXw+VHwt0ymtDxezB0rg9MyJU8/eW6N67HoTeTGAH9qAxQLoABOHs9A2E5mqTnbb N77OPKHPmtoK26y8BFq08/lAvoI7eNagM9XaI4GPaQgjk23xlU+ksGErJ1atiVIKYwHB oPXDUmOZl4RfvFFiQb5XxvOtNmHATTXKq4WpzIdTY4eonXVPCmes08s8iCGsXgMGSa7m XZFg== X-Gm-Message-State: AKGB3mINZSXdeyL0YQTs5XMDcv3HdSOizv1QxHQtbreEcHOlWNXTZXT/ PlYjkV/6VINB80rNFr6rfc2RTJdC X-Google-Smtp-Source: ACJfBovknrr4S7C5WwpKe1wVwfErrcq3OCsEPnpdQCKBpmDdcs/ArFPjZdNV5/Y7GFtZK+nwQp1x3A== X-Received: by 10.101.98.9 with SMTP id d9mr1669389pgv.416.1515132617959; Thu, 04 Jan 2018 22:10:17 -0800 (PST) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o64sm2444086pga.1.2018.01.04.22.10.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Jan 2018 22:10:17 -0800 (PST) From: Jakub Kicinski To: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, daniel@iogearbox.net, davem@davemloft.net Cc: oss-drivers@netronome.com, tehnerd@fb.com, Jakub Kicinski Subject: [PATCH bpf-next 14/16] nfp: bpf: add verification and codegen for map lookups Date: Thu, 4 Jan 2018 22:09:29 -0800 Message-Id: <20180105060931.30815-15-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180105060931.30815-1-jakub.kicinski@netronome.com> References: <20180105060931.30815-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Verify our current constraints on the location of the key are met and generate the code for calling map lookup on the datapath. Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet --- drivers/net/ethernet/netronome/nfp/bpf/jit.c | 51 +++++++++++++++++++++++ drivers/net/ethernet/netronome/nfp/bpf/main.h | 12 +++++- drivers/net/ethernet/netronome/nfp/bpf/verifier.c | 39 +++++++++++++++++ 3 files changed, 100 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/bpf/jit.c b/drivers/net/ethernet/netronome/nfp/bpf/jit.c index 0de59f04da84..a037c0bb4185 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/jit.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/jit.c @@ -1289,6 +1289,55 @@ static int adjust_head(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) return 0; } +static int +map_lookup_stack(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) +{ + struct bpf_offloaded_map *offmap; + struct nfp_bpf_map *nfp_map; + bool load_lm_ptr; + u32 ret_tgt; + s64 lm_off; + swreg tid; + + offmap = (struct bpf_offloaded_map *)meta->arg1.map_ptr; + nfp_map = offmap->dev_priv; + + /* We only have to reload LM0 if the key is not at start of stack */ + lm_off = nfp_prog->stack_depth; + lm_off += meta->arg2.var_off.value + meta->arg2.off; + load_lm_ptr = meta->arg2_var_off || lm_off; + + /* Set LM0 to start of key */ + if (load_lm_ptr) + emit_csr_wr(nfp_prog, reg_b(2 * 2), NFP_CSR_ACT_LM_ADDR0); + + /* Load map ID into a register, it should actually fit as an immediate + * but in case it doesn't deal with it here, not in the delay slots. + */ + tid = ur_load_imm_any(nfp_prog, nfp_map->tid, imm_a(nfp_prog)); + + emit_br(nfp_prog, BR_UNC, nfp_prog->bpf->helpers.map_lookup, 2); + ret_tgt = nfp_prog_current_offset(nfp_prog) + 2; + + /* Load map ID into A0 */ + wrp_mov(nfp_prog, reg_a(0), tid); + + /* Load the return address into B0 */ + wrp_immed(nfp_prog, reg_b(0), nfp_prog_current_offset(nfp_prog) + 1); + + if (!nfp_prog_confirm_current_offset(nfp_prog, ret_tgt)) + return -EINVAL; + + /* Reset the LM0 pointer */ + if (!load_lm_ptr) + return 0; + + emit_csr_wr(nfp_prog, stack_reg(nfp_prog), NFP_CSR_ACT_LM_ADDR0); + wrp_nops(nfp_prog, 3); + + return 0; +} + /* --- Callbacks --- */ static int mov_reg64(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) { @@ -2028,6 +2077,8 @@ static int call(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) switch (meta->insn.imm) { case BPF_FUNC_xdp_adjust_head: return adjust_head(nfp_prog, meta); + case BPF_FUNC_map_lookup_elem: + return map_lookup_stack(nfp_prog, meta); default: WARN_ONCE(1, "verifier allowed unsupported function\n"); return -EOPNOTSUPP; diff --git a/drivers/net/ethernet/netronome/nfp/bpf/main.h b/drivers/net/ethernet/netronome/nfp/bpf/main.h index 9dd76bf248b1..a3b428e1181d 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/main.h +++ b/drivers/net/ethernet/netronome/nfp/bpf/main.h @@ -180,9 +180,12 @@ typedef int (*instr_cb_t)(struct nfp_prog *, struct nfp_insn_meta *); * @ptr: pointer type for memory operations * @ldst_gather_len: memcpy length gathered from load/store sequence * @paired_st: the paired store insn at the head of the sequence - * @arg2: arg2 for call instructions * @ptr_not_const: pointer is not always constant * @jmp_dst: destination info for jump instructions + * @func_id: function id for call instructions + * @arg1: arg1 for call instructions + * @arg2: arg2 for call instructions + * @arg2_var_off: arg2 changes stack offset on different paths * @off: index of first generated machine instruction (in nfp_prog.prog) * @n: eBPF instruction number * @flags: eBPF instruction extra optimization flags @@ -200,7 +203,12 @@ struct nfp_insn_meta { bool ptr_not_const; }; struct nfp_insn_meta *jmp_dst; - struct bpf_reg_state arg2; + struct { + u32 func_id; + struct bpf_reg_state arg1; + struct bpf_reg_state arg2; + bool arg2_var_off; + }; }; unsigned int off; unsigned short n; diff --git a/drivers/net/ethernet/netronome/nfp/bpf/verifier.c b/drivers/net/ethernet/netronome/nfp/bpf/verifier.c index d8870c2f11f3..589f56f9ea96 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/verifier.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/verifier.c @@ -109,9 +109,11 @@ static int nfp_bpf_check_call(struct nfp_prog *nfp_prog, struct bpf_verifier_env *env, struct nfp_insn_meta *meta) { + const struct bpf_reg_state *reg1 = cur_regs(env) + BPF_REG_1; const struct bpf_reg_state *reg2 = cur_regs(env) + BPF_REG_2; struct nfp_app_bpf *bpf = nfp_prog->bpf; u32 func_id = meta->insn.imm; + s64 off, old_off; switch (func_id) { case BPF_FUNC_xdp_adjust_head: @@ -126,11 +128,48 @@ nfp_bpf_check_call(struct nfp_prog *nfp_prog, struct bpf_verifier_env *env, nfp_record_adjust_head(bpf, nfp_prog, meta, reg2); break; + + case BPF_FUNC_map_lookup_elem: + if (!bpf->helpers.map_lookup) { + pr_info("map_lookup: not supported by FW\n"); + return -EOPNOTSUPP; + } + if (reg2->type != PTR_TO_STACK) { + pr_info("map_lookup: unsupported key ptr type %d\n", + reg2->type); + return -EOPNOTSUPP; + } + if (!tnum_is_const(reg2->var_off)) { + pr_info("map_lookup: variable key pointer\n"); + return -EOPNOTSUPP; + } + + off = reg2->var_off.value + reg2->off; + if (-off % 4) { + pr_info("map_lookup: unaligned stack pointer %lld\n", + -off); + return -EOPNOTSUPP; + } + + /* Rest of the checks is only if we re-parse the same insn */ + if (!meta->func_id) + break; + + old_off = meta->arg2.var_off.value + meta->arg2.off; + meta->arg2_var_off |= off != old_off; + + if (meta->arg1.map_ptr != reg1->map_ptr) { + pr_info("map_lookup: called for different map\n"); + return -EOPNOTSUPP; + } + break; default: pr_warn("unsupported function id: %d\n", func_id); return -EOPNOTSUPP; } + meta->func_id = func_id; + meta->arg1 = *reg1; meta->arg2 = *reg2; return 0; From patchwork Fri Jan 5 06:09:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 855920 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="gbOFWfll"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zCZ5x017zz9s4q for ; Fri, 5 Jan 2018 17:10:29 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751429AbeAEGK1 (ORCPT ); Fri, 5 Jan 2018 01:10:27 -0500 Received: from mail-pl0-f67.google.com ([209.85.160.67]:40133 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751400AbeAEGKT (ORCPT ); Fri, 5 Jan 2018 01:10:19 -0500 Received: by mail-pl0-f67.google.com with SMTP id 62so2545444pld.7 for ; Thu, 04 Jan 2018 22:10:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=aCZLSnhTUegMBRbqrahHYWJThg9imLb10RtGQ1x4RSo=; b=gbOFWfll6Rus7nwNGSWDwe/1VD2PiOhQhfaKzIfoUPR/RCiG9WJMN8dw5o5Ssbfq7P OfmpdQx7Vz8E1QVR3Qse02K9XCgRLfSKliojjEeDmyYjJMGqxyjEhgOEGion4mXMSjaL Sdqtuy1qmouwIUC7pfI3p0uPgs/38WPXgwi4EizummhXSPY8dqgyHNsLBQIV/eyduoqr Ky1UWhiTr6XErx5luiPVKC9a4GqZDY3Xw9nYNPuFXYQzOg4rHgvrQcmI4pQRK/TMueXc fecP8N1hasWsN+7ftCJblH6oPJTRxDLegR2YniEwlwZkuM0mdtU8vBtlFq8JpTBrOtUq 8LbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=aCZLSnhTUegMBRbqrahHYWJThg9imLb10RtGQ1x4RSo=; b=i4xh07HO2eBx251YfzOwpW5vAPdQ33madh3zzcS4t60268gRvrxkaFBvaBdMCwcq2y C862YhCNJjN8UL9FeZ+cBlYZVLzIiPG08/PxT+dI46960nlMRWnhFEWJFSFT2sxXWh/A Sf0dvYbe6l/KpZ5b8U3brfPJdoeUGSASwawNR6VJwpjmZXnu0qm3p+Zngui3MvYbIq35 QFzKGrSYd7BAo61QeT03F51k2ra/1cVPy4SgVSRCtoSMrPBUOoQthXm921u6+rdH01xB Xoc8MLbHUou9fH91uMCeOoWVRGLUNkBVguuSYDjcDCINJCJqi23/K4AzvJK182m27iQF PoEw== X-Gm-Message-State: AKGB3mJhqCGawc4DxdR1z5pFjbvh3tAaV2gcU3hJ83/vhyf3aHy2F/xf idn68OzBB7wHdO0EFUNIu5i+FBAo X-Google-Smtp-Source: ACJfBotXDC/mSEs4WwnmzFmkXirUvmSZlHd7MjEjagxEY5aFYO8RZSB7xbVA6f95Z8gaHrz9uPaRzg== X-Received: by 10.159.194.129 with SMTP id y1mr2044001pln.391.1515132619069; Thu, 04 Jan 2018 22:10:19 -0800 (PST) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o64sm2444086pga.1.2018.01.04.22.10.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Jan 2018 22:10:18 -0800 (PST) From: Jakub Kicinski To: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, daniel@iogearbox.net, davem@davemloft.net Cc: oss-drivers@netronome.com, tehnerd@fb.com, Jakub Kicinski Subject: [PATCH bpf-next 15/16] nfp: bpf: add support for reading map memory Date: Thu, 4 Jan 2018 22:09:30 -0800 Message-Id: <20180105060931.30815-16-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180105060931.30815-1-jakub.kicinski@netronome.com> References: <20180105060931.30815-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Map memory needs to use 40 bit addressing. Add handling of such accesses. Since 40 bit addresses are formed by using both 32 bit operands we need to pre-calculate the actual address instead of adding in the offset inside the instruction, like we did in 32 bit mode. Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet --- drivers/net/ethernet/netronome/nfp/bpf/jit.c | 77 ++++++++++++++++++++--- drivers/net/ethernet/netronome/nfp/bpf/verifier.c | 8 +++ 2 files changed, 76 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/bpf/jit.c b/drivers/net/ethernet/netronome/nfp/bpf/jit.c index a037c0bb4185..f1632c80eb3e 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/jit.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/jit.c @@ -548,27 +548,51 @@ wrp_reg_subpart(struct nfp_prog *nfp_prog, swreg dst, swreg src, u8 field_len, emit_ld_field_any(nfp_prog, dst, mask, src, sc, offset * 8, true); } +static void +addr40_offset(struct nfp_prog *nfp_prog, u8 src_gpr, swreg offset, + swreg *rega, swreg *regb) +{ + if (offset == reg_imm(0)) { + *rega = reg_a(src_gpr); + *regb = reg_b(src_gpr + 1); + return; + } + + emit_alu(nfp_prog, imm_a(nfp_prog), reg_a(src_gpr), ALU_OP_ADD, offset); + emit_alu(nfp_prog, imm_b(nfp_prog), reg_b(src_gpr + 1), ALU_OP_ADD_C, + reg_imm(0)); + *rega = imm_a(nfp_prog); + *regb = imm_b(nfp_prog); +} + /* NFP has Command Push Pull bus which supports bluk memory operations. */ static int nfp_cpp_memcpy(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) { bool descending_seq = meta->ldst_gather_len < 0; s16 len = abs(meta->ldst_gather_len); swreg src_base, off; + bool src_40bit_addr; unsigned int i; u8 xfer_num; off = re_load_imm_any(nfp_prog, meta->insn.off, imm_b(nfp_prog)); + src_40bit_addr = meta->ptr.type == PTR_TO_MAP_VALUE; src_base = reg_a(meta->insn.src_reg * 2); xfer_num = round_up(len, 4) / 4; + if (src_40bit_addr) + addr40_offset(nfp_prog, meta->insn.src_reg, off, &src_base, + &off); + /* Setup PREV_ALU fields to override memory read length. */ if (len > 32) wrp_immed(nfp_prog, reg_none(), CMD_OVE_LEN | FIELD_PREP(CMD_OV_LEN, xfer_num - 1)); /* Memory read from source addr into transfer-in registers. */ - emit_cmd_any(nfp_prog, CMD_TGT_READ32_SWAP, CMD_MODE_32b, 0, src_base, - off, xfer_num - 1, true, len > 32); + emit_cmd_any(nfp_prog, CMD_TGT_READ32_SWAP, + src_40bit_addr ? CMD_MODE_40b_BA : CMD_MODE_32b, 0, + src_base, off, xfer_num - 1, true, len > 32); /* Move from transfer-in to transfer-out. */ for (i = 0; i < xfer_num; i++) @@ -706,20 +730,20 @@ data_ld(struct nfp_prog *nfp_prog, swreg offset, u8 dst_gpr, int size) } static int -data_ld_host_order(struct nfp_prog *nfp_prog, u8 src_gpr, swreg offset, - u8 dst_gpr, int size) +data_ld_host_order(struct nfp_prog *nfp_prog, u8 dst_gpr, + swreg lreg, swreg rreg, int size, enum cmd_mode mode) { unsigned int i; u8 mask, sz; - /* We load the value from the address indicated in @offset and then + /* We load the value from the address indicated in rreg + lreg and then * mask out the data we don't need. Note: this is little endian! */ sz = max(size, 4); mask = size < 4 ? GENMASK(size - 1, 0) : 0; - emit_cmd(nfp_prog, CMD_TGT_READ32_SWAP, CMD_MODE_32b, 0, - reg_a(src_gpr), offset, sz / 4 - 1, true); + emit_cmd(nfp_prog, CMD_TGT_READ32_SWAP, mode, 0, + lreg, rreg, sz / 4 - 1, true); i = 0; if (mask) @@ -735,6 +759,26 @@ data_ld_host_order(struct nfp_prog *nfp_prog, u8 src_gpr, swreg offset, return 0; } +static int +data_ld_host_order_addr32(struct nfp_prog *nfp_prog, u8 src_gpr, swreg offset, + u8 dst_gpr, u8 size) +{ + return data_ld_host_order(nfp_prog, dst_gpr, reg_a(src_gpr), offset, + size, CMD_MODE_32b); +} + +static int +data_ld_host_order_addr40(struct nfp_prog *nfp_prog, u8 src_gpr, swreg offset, + u8 dst_gpr, u8 size) +{ + swreg rega, regb; + + addr40_offset(nfp_prog, src_gpr, offset, ®a, ®b); + + return data_ld_host_order(nfp_prog, dst_gpr, rega, regb, + size, CMD_MODE_40b_BA); +} + static int construct_data_ind_ld(struct nfp_prog *nfp_prog, u16 offset, u16 src, u8 size) { @@ -1772,8 +1816,20 @@ mem_ldx_data(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, tmp_reg = re_load_imm_any(nfp_prog, meta->insn.off, imm_b(nfp_prog)); - return data_ld_host_order(nfp_prog, meta->insn.src_reg * 2, tmp_reg, - meta->insn.dst_reg * 2, size); + return data_ld_host_order_addr32(nfp_prog, meta->insn.src_reg * 2, + tmp_reg, meta->insn.dst_reg * 2, size); +} + +static int +mem_ldx_emem(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, + unsigned int size) +{ + swreg tmp_reg; + + tmp_reg = re_load_imm_any(nfp_prog, meta->insn.off, imm_b(nfp_prog)); + + return data_ld_host_order_addr40(nfp_prog, meta->insn.src_reg * 2, + tmp_reg, meta->insn.dst_reg * 2, size); } static int @@ -1797,6 +1853,9 @@ mem_ldx(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, return mem_ldx_stack(nfp_prog, meta, size, meta->ptr.off + meta->ptr.var_off.value); + if (meta->ptr.type == PTR_TO_MAP_VALUE) + return mem_ldx_emem(nfp_prog, meta, size); + return -EOPNOTSUPP; } diff --git a/drivers/net/ethernet/netronome/nfp/bpf/verifier.c b/drivers/net/ethernet/netronome/nfp/bpf/verifier.c index 589f56f9ea96..3ae0add8e4d2 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/verifier.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/verifier.c @@ -247,6 +247,7 @@ nfp_bpf_check_ptr(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, if (reg->type != PTR_TO_CTX && reg->type != PTR_TO_STACK && + reg->type != PTR_TO_MAP_VALUE && reg->type != PTR_TO_PACKET) { pr_info("unsupported ptr type: %d\n", reg->type); return -EINVAL; @@ -258,6 +259,13 @@ nfp_bpf_check_ptr(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, return err; } + if (reg->type == PTR_TO_MAP_VALUE) { + if (is_mbpf_store(meta)) { + pr_info("map writes not supported\n"); + return -EOPNOTSUPP; + } + } + if (meta->ptr.type != NOT_INIT && meta->ptr.type != reg->type) { pr_info("ptr type changed for instruction %d -> %d\n", meta->ptr.type, reg->type); From patchwork Fri Jan 5 06:09:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 855921 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="ia5dFS0z"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zCZ611GrMz9s4q for ; Fri, 5 Jan 2018 17:10:33 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751422AbeAEGK1 (ORCPT ); Fri, 5 Jan 2018 01:10:27 -0500 Received: from mail-pg0-f66.google.com ([74.125.83.66]:44322 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751302AbeAEGKU (ORCPT ); Fri, 5 Jan 2018 01:10:20 -0500 Received: by mail-pg0-f66.google.com with SMTP id i5so1662191pgq.11 for ; Thu, 04 Jan 2018 22:10:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=CVobDfJjGtbpxKgkisGFzgVeq9o4kv5TJMWnfvhqiVg=; b=ia5dFS0zwRpmplI9hHn5vJcYE8AAPmNHvAawGjda7hYGlTKetpSe7dHxOz4lIzgkd8 zoj3dRhwnVgIqxM1Z4/NwlocHmxuqN0NyRL28AWbhSIqpBEP5dmtSad3402haOSgxUcb LfVhkkaBCpvDY1Rd54k6XLRlbtx7BD9DI/xz7XVW1Z1G3PO99TR4j4sLwGLtNXTBXBpo Eurg0io85i+bveIvnM/YZ6iw0tgx8BiUPS2YsleGshg8brJlCP/tTrp6aaMzcRD695SU lZdVuFQdpo2vpOpTDXjbCoERpS+FODer70h9cRqWkOtvN1JPZCRXE5y2c73yLT0b8Vsf ABMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=CVobDfJjGtbpxKgkisGFzgVeq9o4kv5TJMWnfvhqiVg=; b=NkJfseDyO6cEIGnL64QDg1Mb6S2RDCElbu+wyYoN31tXGDRIHWo18E18l9CI5KvPb+ Lnl0LCIycwUxjAzdMZZbLBWr0cDIG5Y9J0A4418tXpmGENFikKuJA2xnYryZjw+fVVqs UUuJyj5h/g6AnyfVfNs0uZhvXr7Y4YeGhcbtlq1yY1rX94q4tYmcP4V+sZhLdC+2QqAw Hz2/spcuGqjR7T+7Pm4/Vmy1ip/11GgJ4cioeuUDdZd/PUGYvGBwp67DxtEvn4hsybWG AHj6P4cscqlt6YVaxMOznx68zKMKz+Pi4tStK69wzRBSiEYzepSgt+Sq7MjYy6QYQEHB zKhw== X-Gm-Message-State: AKGB3mIgUwvcH0cVIHBEPPP/ojbefUpsb4be5B93ChgEwttDTm83he1o 3qG6/qwBvxZW+Vr1iZJtqO7IV63n X-Google-Smtp-Source: ACJfBoup5NoGnWJdjq59IbqxZpXRKrO6h3Kzg8sRU6k00N0W/Bi9UNn9XzgJ9lvTzp5xshVm3t7JCQ== X-Received: by 10.98.67.78 with SMTP id q75mr1900025pfa.55.1515132620194; Thu, 04 Jan 2018 22:10:20 -0800 (PST) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o64sm2444086pga.1.2018.01.04.22.10.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Jan 2018 22:10:19 -0800 (PST) From: Jakub Kicinski To: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, daniel@iogearbox.net, davem@davemloft.net Cc: oss-drivers@netronome.com, tehnerd@fb.com, Jakub Kicinski Subject: [PATCH bpf-next 16/16] nfp: bpf: implement bpf map offload Date: Thu, 4 Jan 2018 22:09:31 -0800 Message-Id: <20180105060931.30815-17-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180105060931.30815-1-jakub.kicinski@netronome.com> References: <20180105060931.30815-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Plug in to the stack's map offload callbacks for BPF map offload. Get next call needs some special handling on the FW side, since we can't send a NULL pointer to the FW there is a get first entry FW command. Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet --- drivers/net/ethernet/netronome/nfp/bpf/main.c | 1 + drivers/net/ethernet/netronome/nfp/bpf/main.h | 4 + drivers/net/ethernet/netronome/nfp/bpf/offload.c | 106 +++++++++++++++++++++++ 3 files changed, 111 insertions(+) diff --git a/drivers/net/ethernet/netronome/nfp/bpf/main.c b/drivers/net/ethernet/netronome/nfp/bpf/main.c index 251cd198e710..b23075021bba 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/main.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/main.c @@ -356,6 +356,7 @@ static void nfp_bpf_clean(struct nfp_app *app) WARN_ON(!skb_queue_empty(&bpf->cmsg_replies)); WARN_ON(!list_empty(&bpf->map_list)); + WARN_ON(bpf->maps_in_use || bpf->map_elems_in_use); kfree(bpf); } diff --git a/drivers/net/ethernet/netronome/nfp/bpf/main.h b/drivers/net/ethernet/netronome/nfp/bpf/main.h index a3b428e1181d..7d74b44209a9 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/main.h +++ b/drivers/net/ethernet/netronome/nfp/bpf/main.h @@ -94,6 +94,8 @@ enum pkt_vec { * @cmsg_wq: work queue for waiting for cmsg replies * * @map_list: list of offloaded maps + * @maps_in_use: number of currently offloaded maps + * @map_elems_in_use: number of elements allocated to offloaded maps * * @adjust_head: adjust head capability * @flags: extra flags for adjust head @@ -124,6 +126,8 @@ struct nfp_app_bpf { struct wait_queue_head cmsg_wq; struct list_head map_list; + unsigned int maps_in_use; + unsigned int map_elems_in_use; struct nfp_bpf_cap_adjust_head { u32 flags; diff --git a/drivers/net/ethernet/netronome/nfp/bpf/offload.c b/drivers/net/ethernet/netronome/nfp/bpf/offload.c index 12a48b7583a3..e94ef02c3b0b 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/offload.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/offload.c @@ -36,6 +36,9 @@ * Netronome network device driver: TC offload functions for PF and VF */ +#define pr_fmt(fmt) "NFP net bpf: " fmt + +#include #include #include #include @@ -171,6 +174,105 @@ static int nfp_bpf_destroy(struct nfp_net *nn, struct bpf_prog *prog) return 0; } +static int +nfp_bpf_map_get_next_key(struct bpf_offloaded_map *offmap, + void *key, void *next_key) +{ + if (!key) + return nfp_bpf_ctrl_getfirst_entry(offmap, next_key); + return nfp_bpf_ctrl_getnext_entry(offmap, key, next_key); +} + +static int +nfp_bpf_map_delete_elem(struct bpf_offloaded_map *offmap, void *key) +{ + if (offmap->map.map_type == BPF_MAP_TYPE_ARRAY) + return -EINVAL; + return nfp_bpf_ctrl_del_entry(offmap, key); +} + +static const struct bpf_map_dev_ops nfp_bpf_map_ops = { + .map_get_next_key = nfp_bpf_map_get_next_key, + .map_lookup_elem = nfp_bpf_ctrl_lookup_entry, + .map_update_elem = nfp_bpf_ctrl_update_entry, + .map_delete_elem = nfp_bpf_map_delete_elem, +}; + +static int +nfp_bpf_map_alloc(struct nfp_app_bpf *bpf, struct bpf_offloaded_map *offmap) +{ + struct nfp_bpf_map *nfp_map; + long long int res; + + if (!bpf->maps.types) + return -EOPNOTSUPP; + + if (offmap->map.map_flags || + offmap->map.numa_node != NUMA_NO_NODE) { + pr_info("map flags are not supported\n"); + return -EINVAL; + } + + if (!(bpf->maps.types & 1 << offmap->map.map_type)) { + pr_info("map type not supported\n"); + return -EOPNOTSUPP; + } + if (bpf->maps.max_maps == bpf->maps_in_use) { + pr_info("too many maps for a device\n"); + return -ENOMEM; + } + if (bpf->maps.max_elems - bpf->map_elems_in_use < + offmap->map.max_entries) { + pr_info("map with too many elements: %u, left: %u\n", + offmap->map.max_entries, + bpf->maps.max_elems - bpf->map_elems_in_use); + return -ENOMEM; + } + if (offmap->map.key_size > bpf->maps.max_key_sz || + offmap->map.value_size > bpf->maps.max_val_sz || + round_up(offmap->map.key_size, 8) + + round_up(offmap->map.value_size, 8) > bpf->maps.max_elem_sz) { + pr_info("elements don't fit in device constraints\n"); + return -ENOMEM; + } + + nfp_map = kzalloc(sizeof(*nfp_map), GFP_USER); + if (!nfp_map) + return -ENOMEM; + + offmap->dev_priv = nfp_map; + nfp_map->offmap = offmap; + nfp_map->bpf = bpf; + + res = nfp_bpf_ctrl_alloc_map(bpf, &offmap->map); + if (res < 0) { + kfree(nfp_map); + return res; + } + + nfp_map->tid = res; + offmap->dev_ops = &nfp_bpf_map_ops; + bpf->maps_in_use++; + bpf->map_elems_in_use += offmap->map.max_entries; + list_add_tail(&nfp_map->l, &bpf->map_list); + + return 0; +} + +static int +nfp_bpf_map_free(struct nfp_app_bpf *bpf, struct bpf_offloaded_map *offmap) +{ + struct nfp_bpf_map *nfp_map = offmap->dev_priv; + + nfp_bpf_ctrl_free_map(bpf, nfp_map); + list_del_init(&nfp_map->l); + bpf->map_elems_in_use -= offmap->map.max_entries; + bpf->maps_in_use--; + kfree(nfp_map); + + return 0; +} + int nfp_ndo_bpf(struct nfp_app *app, struct nfp_net *nn, struct netdev_bpf *bpf) { switch (bpf->command) { @@ -180,6 +282,10 @@ int nfp_ndo_bpf(struct nfp_app *app, struct nfp_net *nn, struct netdev_bpf *bpf) return nfp_bpf_translate(nn, bpf->offload.prog); case BPF_OFFLOAD_DESTROY: return nfp_bpf_destroy(nn, bpf->offload.prog); + case BPF_OFFLOAD_MAP_ALLOC: + return nfp_bpf_map_alloc(app->priv, bpf->offmap); + case BPF_OFFLOAD_MAP_FREE: + return nfp_bpf_map_free(app->priv, bpf->offmap); default: return -EINVAL; }