From patchwork Fri Nov 3 20:56:28 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 834091 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="kUVMFu1C"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3yTDlx4Prwz9s7p for ; Sat, 4 Nov 2017 07:57:01 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755830AbdKCU5A (ORCPT ); Fri, 3 Nov 2017 16:57:00 -0400 Received: from mail-pf0-f194.google.com ([209.85.192.194]:43566 "EHLO mail-pf0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752454AbdKCU46 (ORCPT ); Fri, 3 Nov 2017 16:56:58 -0400 Received: by mail-pf0-f194.google.com with SMTP id a8so3043465pfc.0 for ; Fri, 03 Nov 2017 13:56:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=NEIeUBzkMmQb5EPd5GQJ2pCK91bHz364jSJ5uUlf0Lc=; b=kUVMFu1CkoxsYn3wy7kKk9M3Pxr2tKHgrWUse+ftPFerwZA5gVQHMibGRrmGYrcFkJ Im0+HuIrxTqIQRh2/TmVIVZb9K+Tkj7TY0u7QjSJsSWLFzRn0oA2cDy9LgW2uu81JWBf nCwajMs0ehRGjguWGD2awK820mm0A1hqVh3tUwOhUicvd8t3IOtneVrhlDBlOXOAz9qg 8+1EYkeL00kTuPZAGx5BYz93ClP0sDtoRBaUGEjWz/FIRDtcdesdbvbXpK5W2fJBvIDD gcawcUVwNntNm63hYGmRBcJjQ0sWGh8zpFIRuhC07b8dEnaua5UMMA+sDk5tY5SI2Bxo SCWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=NEIeUBzkMmQb5EPd5GQJ2pCK91bHz364jSJ5uUlf0Lc=; b=UUnP7DgIn9+Sib2GplAUOgC8e3ADbuHBG4FACllFE4B5wBMKgtWUd/H0ePTZVBdw2J lQNppv1tthiXpr/shfoFL7IS0eM3JDI3Ujapmiz8NiWT+S++ZTzSor9t+5iqPea1MjHk lIt54k8q13WEetw0Ib30VrPpfCNqY90BByAe6Zr3yBSMKcNOdHvbX4eXH5HpO7VQTZvM z+FCScGAnNKpmOBETdHa1X2IPZRK8iZMQ0NvZwLTg59cYbQU1Jb6fuGt+afxJbiInXeu LqEm67aERGYKWF78y4Yoi+2LfgmB+Xl4gAByGEfrz/rrAuqpSX9CLga3Kk9SFRJwRU4B QAZA== X-Gm-Message-State: AMCzsaUN989zHWOmRPhLkCP2ITRU9NNEvy8p0RmRwgfNQmVzizw5HcSA LLRLPSSuNZ59j9cbQfhrtLjyxIxH X-Google-Smtp-Source: ABhQp+QVJ65cxY2Ne5WNCJka5ca+14uI1hmieun7Wo7CGG9dlCrKJnkuNDHjE27nBVD6knrkmCcErw== X-Received: by 10.84.248.10 with SMTP id p10mr7847135pll.447.1509742617289; Fri, 03 Nov 2017 13:56:57 -0700 (PDT) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id o10sm9989029pgq.69.2017.11.03.13.56.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 03 Nov 2017 13:56:56 -0700 (PDT) From: Jakub Kicinski To: netdev@vger.kernel.org Cc: oss-drivers@netronome.com, alexei.starovoitov@gmail.com, daniel@iogearbox.net, Jakub Kicinski Subject: [PATCH net-next v2 13/15] nfp: bpf: move translation prepare to offload.c Date: Fri, 3 Nov 2017 13:56:28 -0700 Message-Id: <20171103205630.1083-14-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20171103205630.1083-1-jakub.kicinski@netronome.com> References: <20171103205630.1083-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org struct nfp_prog is currently only used internally by the translator. This means there is a lot of parameter passing going on, between the translator and different stages of offload. Simplify things by allocating nfp_prog in offload.c already. We will now use kmalloc() to allocate the program area and only DMA map it for the time of loading (instead of allocating DMA coherent memory upfront). Signed-off-by: Jakub Kicinski Reviewed-by: Quentin Monnet --- drivers/net/ethernet/netronome/nfp/bpf/jit.c | 43 ++------ drivers/net/ethernet/netronome/nfp/bpf/main.h | 14 +-- drivers/net/ethernet/netronome/nfp/bpf/offload.c | 128 +++++++++++++++-------- 3 files changed, 94 insertions(+), 91 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/bpf/jit.c b/drivers/net/ethernet/netronome/nfp/bpf/jit.c index 2eddbb45fd60..eae7a137a7a8 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/jit.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/jit.c @@ -2245,58 +2245,27 @@ static int nfp_bpf_ustore_calc(struct nfp_prog *nfp_prog, __le64 *ustore) /** * nfp_bpf_jit() - translate BPF code into NFP assembly + * @nfp_prog: nfp_prog prepared based on @filter * @filter: kernel BPF filter struct - * @prog_mem: memory to store assembler instructions - * @prog_start: offset of the first instruction when loaded - * @prog_done: where to jump on exit - * @prog_sz: size of @prog_mem in instructions - * @res: achieved parameters of translation results */ -int -nfp_bpf_jit(struct bpf_prog *filter, void *prog_mem, - unsigned int prog_start, unsigned int prog_done, - unsigned int prog_sz, struct nfp_bpf_result *res) +int nfp_bpf_jit(struct nfp_prog *nfp_prog, struct bpf_prog *filter) { - struct nfp_prog *nfp_prog; int ret; - nfp_prog = kzalloc(sizeof(*nfp_prog), GFP_KERNEL); - if (!nfp_prog) - return -ENOMEM; - - INIT_LIST_HEAD(&nfp_prog->insns); - nfp_prog->type = filter->type; - nfp_prog->start_off = prog_start; - nfp_prog->tgt_done = prog_done; - - ret = nfp_prog_prepare(nfp_prog, filter->insnsi, filter->len); - if (ret) - goto out; - ret = nfp_prog_verify(nfp_prog, filter); if (ret) - goto out; + return ret; ret = nfp_bpf_optimize(nfp_prog); if (ret) - goto out; - - nfp_prog->prog = prog_mem; - nfp_prog->__prog_alloc_len = prog_sz; + return ret; ret = nfp_translate(nfp_prog); if (ret) { pr_err("Translation failed with error %d (translated: %u)\n", ret, nfp_prog->n_translated); - ret = -EINVAL; - goto out; + return -EINVAL; } - ret = nfp_bpf_ustore_calc(nfp_prog, (__force __le64 *)prog_mem); - - res->n_instr = nfp_prog->prog_len; -out: - nfp_prog_free(nfp_prog); - - return ret; + return nfp_bpf_ustore_calc(nfp_prog, (__force __le64 *)nfp_prog->prog); } diff --git a/drivers/net/ethernet/netronome/nfp/bpf/main.h b/drivers/net/ethernet/netronome/nfp/bpf/main.h index b77231a134b9..36b4eda2d3f8 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/main.h +++ b/drivers/net/ethernet/netronome/nfp/bpf/main.h @@ -169,19 +169,7 @@ struct nfp_prog { struct list_head insns; }; -struct nfp_bpf_result { - unsigned int n_instr; -}; - -int -nfp_prog_prepare(struct nfp_prog *nfp_prog, const struct bpf_insn *prog, - unsigned int cnt); -void nfp_prog_free(struct nfp_prog *nfp_prog); - -int -nfp_bpf_jit(struct bpf_prog *filter, void *prog, - unsigned int prog_start, unsigned int prog_done, - unsigned int prog_sz, struct nfp_bpf_result *res); +int nfp_bpf_jit(struct nfp_prog *nfp_prog, struct bpf_prog *filter); int nfp_prog_verify(struct nfp_prog *nfp_prog, struct bpf_prog *prog); diff --git a/drivers/net/ethernet/netronome/nfp/bpf/offload.c b/drivers/net/ethernet/netronome/nfp/bpf/offload.c index 3eeee200051e..c5546c0e87d8 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/offload.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/offload.c @@ -51,7 +51,7 @@ #include "../nfp_net_ctrl.h" #include "../nfp_net.h" -int +static int nfp_prog_prepare(struct nfp_prog *nfp_prog, const struct bpf_insn *prog, unsigned int cnt) { @@ -73,7 +73,7 @@ nfp_prog_prepare(struct nfp_prog *nfp_prog, const struct bpf_insn *prog, return 0; } -void nfp_prog_free(struct nfp_prog *nfp_prog) +static void nfp_prog_free(struct nfp_prog *nfp_prog) { struct nfp_insn_meta *meta, *tmp; @@ -84,25 +84,36 @@ void nfp_prog_free(struct nfp_prog *nfp_prog) kfree(nfp_prog); } -static int -nfp_net_bpf_offload_prepare(struct nfp_net *nn, struct bpf_prog *prog, - struct nfp_bpf_result *res, - void **code, dma_addr_t *dma_addr, u16 max_instr) +static struct nfp_prog *nfp_bpf_verifier_prep(struct bpf_prog *prog) { - unsigned int code_sz = max_instr * sizeof(u64); - unsigned int stack_size; - u16 start_off, done_off; - unsigned int max_mtu; + struct nfp_prog *nfp_prog; int ret; - max_mtu = nn_readb(nn, NFP_NET_CFG_BPF_INL_MTU) * 64 - 32; - if (max_mtu < nn->dp.netdev->mtu) { - nn_info(nn, "BPF offload not supported with MTU larger than HW packet split boundary\n"); - return -EOPNOTSUPP; - } + nfp_prog = kzalloc(sizeof(*nfp_prog), GFP_KERNEL); + if (!nfp_prog) + return NULL; + + INIT_LIST_HEAD(&nfp_prog->insns); + nfp_prog->type = prog->type; - start_off = nn_readw(nn, NFP_NET_CFG_BPF_START); - done_off = nn_readw(nn, NFP_NET_CFG_BPF_DONE); + ret = nfp_prog_prepare(nfp_prog, prog->insnsi, prog->len); + if (ret) + goto err_free; + + return nfp_prog; + +err_free: + nfp_prog_free(nfp_prog); + + return NULL; +} + +static int +nfp_bpf_translate(struct nfp_net *nn, struct nfp_prog *nfp_prog, + struct bpf_prog *prog) +{ + unsigned int stack_size; + unsigned int max_instr; stack_size = nn_readb(nn, NFP_NET_CFG_BPF_STACK_SZ) * 64; if (prog->aux->stack_depth > stack_size) { @@ -111,28 +122,68 @@ nfp_net_bpf_offload_prepare(struct nfp_net *nn, struct bpf_prog *prog, return -EOPNOTSUPP; } - *code = dma_zalloc_coherent(nn->dp.dev, code_sz, dma_addr, GFP_KERNEL); - if (!*code) + nfp_prog->stack_depth = prog->aux->stack_depth; + nfp_prog->start_off = nn_readw(nn, NFP_NET_CFG_BPF_START); + nfp_prog->tgt_done = nn_readw(nn, NFP_NET_CFG_BPF_DONE); + + max_instr = nn_readw(nn, NFP_NET_CFG_BPF_MAX_LEN); + nfp_prog->__prog_alloc_len = max_instr * sizeof(u64); + + nfp_prog->prog = kmalloc(nfp_prog->__prog_alloc_len, GFP_KERNEL); + if (!nfp_prog->prog) return -ENOMEM; - ret = nfp_bpf_jit(prog, *code, start_off, done_off, max_instr, res); - if (ret) - goto out; + return nfp_bpf_jit(nfp_prog, prog); +} + +static void nfp_bpf_destroy(struct nfp_prog *nfp_prog) +{ + kfree(nfp_prog->prog); + nfp_prog_free(nfp_prog); +} + +static struct nfp_prog * +nfp_net_bpf_offload_prepare(struct nfp_net *nn, struct bpf_prog *prog, + dma_addr_t *dma_addr) +{ + struct nfp_prog *nfp_prog; + unsigned int max_mtu; + int err; + + max_mtu = nn_readb(nn, NFP_NET_CFG_BPF_INL_MTU) * 64 - 32; + if (max_mtu < nn->dp.netdev->mtu) { + nn_info(nn, "BPF offload not supported with MTU larger than HW packet split boundary\n"); + return NULL; + } + + nfp_prog = nfp_bpf_verifier_prep(prog); + if (!nfp_prog) + return NULL; + + err = nfp_bpf_translate(nn, nfp_prog, prog); + if (err) + goto err_destroy_prog; + + *dma_addr = dma_map_single(nn->dp.dev, nfp_prog->prog, + nfp_prog->prog_len * sizeof(u64), + DMA_TO_DEVICE); + if (dma_mapping_error(nn->dp.dev, *dma_addr)) + goto err_destroy_prog; return 0; -out: - dma_free_coherent(nn->dp.dev, code_sz, *code, *dma_addr); - return ret; +err_destroy_prog: + nfp_bpf_destroy(nfp_prog); + return NULL; } static void -nfp_net_bpf_load(struct nfp_net *nn, void *code, dma_addr_t dma_addr, - unsigned int code_sz, unsigned int n_instr) +nfp_net_bpf_load(struct nfp_net *nn, struct nfp_prog *nfp_prog, + dma_addr_t dma_addr) { int err; - nn_writew(nn, NFP_NET_CFG_BPF_SIZE, n_instr); + nn_writew(nn, NFP_NET_CFG_BPF_SIZE, nfp_prog->prog_len); nn_writeq(nn, NFP_NET_CFG_BPF_ADDR, dma_addr); /* Load up the JITed code */ @@ -140,7 +191,9 @@ nfp_net_bpf_load(struct nfp_net *nn, void *code, dma_addr_t dma_addr, if (err) nn_err(nn, "FW command error while loading BPF: %d\n", err); - dma_free_coherent(nn->dp.dev, code_sz, code, dma_addr); + dma_unmap_single(nn->dp.dev, dma_addr, nfp_prog->prog_len * sizeof(u64), + DMA_TO_DEVICE); + nfp_bpf_destroy(nfp_prog); } static void nfp_net_bpf_start(struct nfp_net *nn) @@ -169,11 +222,8 @@ static int nfp_net_bpf_stop(struct nfp_net *nn) int nfp_net_bpf_offload(struct nfp_net *nn, struct bpf_prog *prog, bool old_prog) { - struct nfp_bpf_result res; + struct nfp_prog *nfp_prog; dma_addr_t dma_addr; - u16 max_instr; - void *code; - int err; if (prog && old_prog) { u8 cap; @@ -192,15 +242,11 @@ int nfp_net_bpf_offload(struct nfp_net *nn, struct bpf_prog *prog, if (old_prog && !prog) return nfp_net_bpf_stop(nn); - max_instr = nn_readw(nn, NFP_NET_CFG_BPF_MAX_LEN); - - err = nfp_net_bpf_offload_prepare(nn, prog, &res, &code, &dma_addr, - max_instr); - if (err) - return err; + nfp_prog = nfp_net_bpf_offload_prepare(nn, prog, &dma_addr); + if (!nfp_prog) + return -EINVAL; - nfp_net_bpf_load(nn, code, dma_addr, max_instr * sizeof(u64), - res.n_instr); + nfp_net_bpf_load(nn, nfp_prog, dma_addr); if (!old_prog) nfp_net_bpf_start(nn);