From patchwork Wed Apr 25 04:22:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 903950 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=netronome.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="No+E9rMy"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40W6W74fcMz9ryk for ; Wed, 25 Apr 2018 14:22:59 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751314AbeDYEW4 (ORCPT ); Wed, 25 Apr 2018 00:22:56 -0400 Received: from mail-pg0-f66.google.com ([74.125.83.66]:35932 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751181AbeDYEWw (ORCPT ); Wed, 25 Apr 2018 00:22:52 -0400 Received: by mail-pg0-f66.google.com with SMTP id i6so12416515pgv.3 for ; Tue, 24 Apr 2018 21:22:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ETAxSbNmPrOm8OT/7U7Sqk9O1NbcUBCKjcsmy5wa8y4=; b=No+E9rMyAdxKo+dpic0DBwzrEAN7IkYnvJC3WbwTFANt9IAQDp6okL2kbRAlOuqO6Q i6YYGxijf8Wy0VjkUNh1LoVS1cUvCFLfrTs3isoe8qswNH4fP0HsGTmOUhwW7nr/mpL2 pZPKRs2UxsBlGqbRYlKLO3YbLppyIMt7J6p2csHoqoe3jFC/2ci7lJJbkZVLBQNlE6Ab G1DUlvych2ConRNtPunNBWmGCz8w5QJN8iHnJgrZYXxNx+ejpIMQfLPTV5fUczM+Dw6G lzYpT3/FfsJIOiTTyGfv44XSiCLWZrE1vLOHMbFycFATHfAD5CCYwegNcUeYgzv7+9Ll ztdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ETAxSbNmPrOm8OT/7U7Sqk9O1NbcUBCKjcsmy5wa8y4=; b=g9Zr4u2FMf0Z7KzMxjpTBHycksh/zOf3+JzP3Dl/9NBh2xZCitN6gcwE00pd/4+RVp uJK5KvKEBNmIGvQX9855+rrZ952aoTZFXHHWRYbsYKASkQaGcKfdZGoRq5dtJJErlJPW SPmkC64wuKr0efXf2b/LjTMhkvYkSYHn0luKsHg+inNAabgXNLZYntkHQs/0Q2NqMSiM Gv0klHI+eWcInikO7BnYWiDrcFcii1wwGeD1YqMJ7hXkMHiIFbxpfQnzCCkIAxw6M6I7 utRI0YnqSxAyeXzbzdQ401NNSVjY5qrI2DcZKk6Y2LcncqM3EwKpsrtNGR8OZGVJxHJX TMUw== X-Gm-Message-State: ALQs6tA4hOe6D1VTGuDI8E02AaDoFzK+cAfsgDQM6sr7wUP4hov9Uqky W1xexofyLwayjaZ7BytU2q5hJQ== X-Google-Smtp-Source: AIpwx4/RUNbKE5XKb2N1P7FzzXWn3S0uNbvEZugetRyfl0IJHTGUTuFA/Lb9RE1rH4Vb5JKQ+mkppA== X-Received: by 10.98.131.69 with SMTP id h66mr16603019pfe.0.1524630171610; Tue, 24 Apr 2018 21:22:51 -0700 (PDT) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id k14sm22449002pgt.27.2018.04.24.21.22.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Apr 2018 21:22:51 -0700 (PDT) From: Jakub Kicinski To: alexei.starovoitov@gmail.com, daniel@iogearbox.net Cc: oss-drivers@netronome.com, netdev@vger.kernel.org, Jakub Kicinski Subject: [PATCH bpf-next 1/4] nfp: bpf: remove double space Date: Tue, 24 Apr 2018 21:22:36 -0700 Message-Id: <20180425042239.27869-2-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180425042239.27869-1-jakub.kicinski@netronome.com> References: <20180425042239.27869-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Whitespace cleanup - remove double space. Signed-off-by: Jakub Kicinski --- drivers/net/ethernet/netronome/nfp/bpf/jit.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/ethernet/netronome/nfp/bpf/jit.c b/drivers/net/ethernet/netronome/nfp/bpf/jit.c index 29b4e5f8c102..9cc638718272 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/jit.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/jit.c @@ -1400,7 +1400,7 @@ map_call_stack_common(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) if (!load_lm_ptr) return 0; - emit_csr_wr(nfp_prog, stack_reg(nfp_prog), NFP_CSR_ACT_LM_ADDR0); + emit_csr_wr(nfp_prog, stack_reg(nfp_prog), NFP_CSR_ACT_LM_ADDR0); wrp_nops(nfp_prog, 3); return 0; From patchwork Wed Apr 25 04:22:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 903951 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=netronome.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="zR25WX7z"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40W6WF0wVgz9ryk for ; Wed, 25 Apr 2018 14:23:05 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751359AbeDYEXC (ORCPT ); Wed, 25 Apr 2018 00:23:02 -0400 Received: from mail-pg0-f68.google.com ([74.125.83.68]:45032 "EHLO mail-pg0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750962AbeDYEWx (ORCPT ); Wed, 25 Apr 2018 00:22:53 -0400 Received: by mail-pg0-f68.google.com with SMTP id 82so2567899pge.11 for ; Tue, 24 Apr 2018 21:22:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=aQZZ6OCPkWtzHyWJIj9Lfi/cYvQk2FWTBC341gR1M6M=; b=zR25WX7zs1soCQnhui261TqMuwP0EWdtZjgVmaMpGVy9z8PUeZr3n1WBl9SpJKR0fj KX0eSNYqkwgqjeoiTiv4vEGdrDmtAR0/X28l9E4v6hCOESiERS030el/f6HBnTGH5OTE NbQ4Qu2axzwEpRUtv4xAgXu6FyOPwVp2/oVwcT51uEFuqBqj6gOWhITrnyKelpEBBjeg C+1fF++YtX2NzSqUFMZK1cTjLbJGQj+BDbVI1YTMAAhPMTzfTDNhWB/NVPnNBrGkdlo3 biveZzah8JnTBGGE8a+x1T6WrUYPOrWCQlOieoCStT4SL628zzwQ6ZUckjn/OvklFHsO pKFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=aQZZ6OCPkWtzHyWJIj9Lfi/cYvQk2FWTBC341gR1M6M=; b=qnRx/fbeOu8a1P/pj25ES+WbiY6nwxij3we5HzXDC3hq4alnM7gW/RWuucw4qN7Aim /O+mp0GhX3h5YpMbOIa2paIPiMaJ7Hs+o9D/QfKdw/1kxqQeQp0v9RSsR9CWTWuCKf7e 4CsgvYS63h/1H4G+sQux/i+iq5iqcFAixz+phPssopkTWCze6akNE4zkan0F+I0vexOE j3Dpp/ZysYGZ3yyw4qO+U2liUMboSfBcaDeesQn6C4i7QtiknjrJi+Ch2DpcMlYTbyuY OfEBu9eBJwwKuWIQbewImj1EdBJ7ZA5dkBrV33K56ELDA/jgXyx86oIx66WX5p9gZiIG SN9g== X-Gm-Message-State: ALQs6tAF2WdSVgEP49or5bExRV4t04B9GE5HzzsBaviSejvD1JLmm7B1 QSy1u2JURYNi59RCPxFGm6YQUw== X-Google-Smtp-Source: AIpwx49D5Tz7efvgl3q/m2OCuU3Yinl5S3N//nZ7hOUbtbdUpvmLaWg5Fba5oY+9KMT14XXmTvyHkw== X-Received: by 10.98.155.141 with SMTP id e13mr17079355pfk.157.1524630172469; Tue, 24 Apr 2018 21:22:52 -0700 (PDT) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id k14sm22449002pgt.27.2018.04.24.21.22.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Apr 2018 21:22:51 -0700 (PDT) From: Jakub Kicinski To: alexei.starovoitov@gmail.com, daniel@iogearbox.net Cc: oss-drivers@netronome.com, netdev@vger.kernel.org, Jakub Kicinski Subject: [PATCH bpf-next 2/4] nfp: bpf: optimize add/sub of a negative constant Date: Tue, 24 Apr 2018 21:22:37 -0700 Message-Id: <20180425042239.27869-3-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180425042239.27869-1-jakub.kicinski@netronome.com> References: <20180425042239.27869-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org NFP instruction set can fit small immediates into the instruction. Negative integers, however, will never fit because they will have highest bit set. If we swap the ALU op between ADD and SUB and negate the constant we have a better chance of fitting small negative integers into the instruction itself and saving one or two cycles. immed[gprB_21, 0xfffffffc] alu[gprA_4, gprA_4, +, gprB_21], gpr_wrboth immed[gprB_21, 0xffffffff] alu[gprA_5, gprA_5, +carry, gprB_21], gpr_wrboth now becomes: alu[gprA_4, gprA_4, -, 4], gpr_wrboth alu[gprA_5, gprA_5, -carry, 0], gpr_wrboth Signed-off-by: Jakub Kicinski --- drivers/net/ethernet/netronome/nfp/bpf/jit.c | 35 ++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/drivers/net/ethernet/netronome/nfp/bpf/jit.c b/drivers/net/ethernet/netronome/nfp/bpf/jit.c index 9cc638718272..a5590988fc69 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/jit.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/jit.c @@ -2777,6 +2777,40 @@ static void nfp_bpf_opt_reg_init(struct nfp_prog *nfp_prog) } } +/* abs(insn.imm) will fit better into unrestricted reg immediate - + * convert add/sub of a negative number into a sub/add of a positive one. + */ +static void nfp_bpf_opt_neg_add_sub(struct nfp_prog *nfp_prog) +{ + struct nfp_insn_meta *meta; + + list_for_each_entry(meta, &nfp_prog->insns, l) { + struct bpf_insn insn = meta->insn; + + if (meta->skip) + continue; + + if (BPF_CLASS(insn.code) != BPF_ALU && + BPF_CLASS(insn.code) != BPF_ALU64) + continue; + if (BPF_SRC(insn.code) != BPF_K) + continue; + if (insn.imm >= 0) + continue; + + if (BPF_OP(insn.code) == BPF_ADD) + insn.code = BPF_CLASS(insn.code) | BPF_SUB; + else if (BPF_OP(insn.code) == BPF_SUB) + insn.code = BPF_CLASS(insn.code) | BPF_ADD; + else + continue; + + meta->insn.code = insn.code | BPF_K; + + meta->insn.imm = -insn.imm; + } +} + /* Remove masking after load since our load guarantees this is not needed */ static void nfp_bpf_opt_ld_mask(struct nfp_prog *nfp_prog) { @@ -3212,6 +3246,7 @@ static int nfp_bpf_optimize(struct nfp_prog *nfp_prog) { nfp_bpf_opt_reg_init(nfp_prog); + nfp_bpf_opt_neg_add_sub(nfp_prog); nfp_bpf_opt_ld_mask(nfp_prog); nfp_bpf_opt_ld_shift(nfp_prog); nfp_bpf_opt_ldst_gather(nfp_prog); From patchwork Wed Apr 25 04:22:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 903952 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=netronome.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="0IMglJpA"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40W6WG45xdz9ryk for ; Wed, 25 Apr 2018 14:23:06 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751389AbeDYEXE (ORCPT ); Wed, 25 Apr 2018 00:23:04 -0400 Received: from mail-pf0-f196.google.com ([209.85.192.196]:42621 "EHLO mail-pf0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750991AbeDYEWy (ORCPT ); Wed, 25 Apr 2018 00:22:54 -0400 Received: by mail-pf0-f196.google.com with SMTP id o16so14078654pfk.9 for ; Tue, 24 Apr 2018 21:22:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=CR7khFd1A3qqnKx3WrUrIA98f38kyOgIDvNGAgAXPAs=; b=0IMglJpAoN2IiY6O6S4xlnYcI1O205jipB1ThgJTfLF5l6ccAo/ke5eY+dX+9fhLnh MW2QvXUvO+xNxkn3/7/8g0gjMCUPylWWRuTo3MGwDECxOwgp0GdyJv5Z7KBHwKfDZIls h1BLNvxAm2ymlu/xIt1geGMujiV661KdewzvchMPkGwPQ2iWBklZWxUabNvQqenO3s81 aSkncly0Oy+alUi/Yf0i3hVj20qxjQ2NZwKyJpEzxhWbj4e7b+ilpAia2nLPC1P/s2E2 WHNpkAu2LUgWZlurK2y2K0K1sixX8S4roYnoWlLjmZqTYbh8xEZW0Xkxjt6WSZ2GEUmc M9tQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=CR7khFd1A3qqnKx3WrUrIA98f38kyOgIDvNGAgAXPAs=; b=bDPQ2MymIa6S1AByfsGEcWR3EUo9zx7lhhKeHskHGdUCU6oANOiXHm/5XF41Ahxviq /24T/432Dl2sppwmA8v6giOkZtIRkcB0kMgcv1YmM48b7XrmU/8IM//wFrWAT87TaSrp RJXamNuM8jamqR/sOv/cw9zWELJvaQs+OkWgNeC46x43iCRwsqi51Nfc209bnO8FKVG4 CazUFIjPB5rzTWaFFp997j98kIvY+O2Xa2ZLFqbI/6FriWoc6EPkabNvqlJH3cw4VxuX 12Y37VANdFnLXIbLLNuZWvAf+i3XncrP/BcaWMvDcMMcKKkBVffAN+NdWgRs9Z79i7jt ofAA== X-Gm-Message-State: ALQs6tAvtJF4oO9k/dUHhG06/uyvakX4jIc6W0adsrapCrggViSyjIbR /Mhp4I4j7OjAXii7S2gXYLv79g== X-Google-Smtp-Source: AIpwx4+X9syP3ddsLN4qSssEfxwKmkvbt1koAwLzknQ4DBnTEEVNX3LVAUwCvRd1J67l2TeUmsMo4w== X-Received: by 10.98.15.23 with SMTP id x23mr25104472pfi.3.1524630173342; Tue, 24 Apr 2018 21:22:53 -0700 (PDT) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id k14sm22449002pgt.27.2018.04.24.21.22.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Apr 2018 21:22:52 -0700 (PDT) From: Jakub Kicinski To: alexei.starovoitov@gmail.com, daniel@iogearbox.net Cc: oss-drivers@netronome.com, netdev@vger.kernel.org, Jakub Kicinski Subject: [PATCH bpf-next 3/4] nfp: bpf: tabularize generations of compare operations Date: Tue, 24 Apr 2018 21:22:38 -0700 Message-Id: <20180425042239.27869-4-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180425042239.27869-1-jakub.kicinski@netronome.com> References: <20180425042239.27869-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org There are quite a few compare instructions now, use a table to translate BPF instruction code to NFP instruction parameters instead of parameterizing helpers. This saves LOC and makes future extensions easier. Signed-off-by: Jakub Kicinski --- drivers/net/ethernet/netronome/nfp/bpf/jit.c | 168 ++++++++++----------------- 1 file changed, 61 insertions(+), 107 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/bpf/jit.c b/drivers/net/ethernet/netronome/nfp/bpf/jit.c index a5590988fc69..5b8da7a67df4 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/jit.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/jit.c @@ -1214,45 +1214,79 @@ wrp_test_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, return 0; } -static int -wrp_cmp_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, - enum br_mask br_mask, bool swap) +static const struct jmp_code_map { + enum br_mask br_mask; + bool swap; +} jmp_code_map[] = { + [BPF_JGT >> 4] = { BR_BLO, true }, + [BPF_JGE >> 4] = { BR_BHS, false }, + [BPF_JLT >> 4] = { BR_BLO, false }, + [BPF_JLE >> 4] = { BR_BHS, true }, + [BPF_JSGT >> 4] = { BR_BLT, true }, + [BPF_JSGE >> 4] = { BR_BGE, false }, + [BPF_JSLT >> 4] = { BR_BLT, false }, + [BPF_JSLE >> 4] = { BR_BGE, true }, +}; + +static const struct jmp_code_map *nfp_jmp_code_get(struct nfp_insn_meta *meta) +{ + unsigned int op; + + op = BPF_OP(meta->insn.code) >> 4; + /* br_mask of 0 is BR_BEQ which we don't use in jump code table */ + if (WARN_ONCE(op >= ARRAY_SIZE(jmp_code_map) || + !jmp_code_map[op].br_mask, + "no code found for jump instruction")) + return NULL; + + return &jmp_code_map[op]; +} + +static int cmp_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) { const struct bpf_insn *insn = &meta->insn; u64 imm = insn->imm; /* sign extend */ + const struct jmp_code_map *code; u8 reg = insn->dst_reg * 2; swreg tmp_reg; + code = nfp_jmp_code_get(meta); + if (!code) + return -EINVAL; + tmp_reg = ur_load_imm_any(nfp_prog, imm & ~0U, imm_b(nfp_prog)); - if (!swap) + if (!code->swap) emit_alu(nfp_prog, reg_none(), reg_a(reg), ALU_OP_SUB, tmp_reg); else emit_alu(nfp_prog, reg_none(), tmp_reg, ALU_OP_SUB, reg_a(reg)); tmp_reg = ur_load_imm_any(nfp_prog, imm >> 32, imm_b(nfp_prog)); - if (!swap) + if (!code->swap) emit_alu(nfp_prog, reg_none(), reg_a(reg + 1), ALU_OP_SUB_C, tmp_reg); else emit_alu(nfp_prog, reg_none(), tmp_reg, ALU_OP_SUB_C, reg_a(reg + 1)); - emit_br(nfp_prog, br_mask, insn->off, 0); + emit_br(nfp_prog, code->br_mask, insn->off, 0); return 0; } -static int -wrp_cmp_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, - enum br_mask br_mask, bool swap) +static int cmp_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) { const struct bpf_insn *insn = &meta->insn; + const struct jmp_code_map *code; u8 areg, breg; + code = nfp_jmp_code_get(meta); + if (!code) + return -EINVAL; + areg = insn->dst_reg * 2; breg = insn->src_reg * 2; - if (swap) { + if (code->swap) { areg ^= breg; breg ^= areg; areg ^= breg; @@ -1261,7 +1295,7 @@ wrp_cmp_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, emit_alu(nfp_prog, reg_none(), reg_a(areg), ALU_OP_SUB, reg_b(breg)); emit_alu(nfp_prog, reg_none(), reg_a(areg + 1), ALU_OP_SUB_C, reg_b(breg + 1)); - emit_br(nfp_prog, br_mask, insn->off, 0); + emit_br(nfp_prog, code->br_mask, insn->off, 0); return 0; } @@ -2283,46 +2317,6 @@ static int jeq_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) return 0; } -static int jgt_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) -{ - return wrp_cmp_imm(nfp_prog, meta, BR_BLO, true); -} - -static int jge_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) -{ - return wrp_cmp_imm(nfp_prog, meta, BR_BHS, false); -} - -static int jlt_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) -{ - return wrp_cmp_imm(nfp_prog, meta, BR_BLO, false); -} - -static int jle_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) -{ - return wrp_cmp_imm(nfp_prog, meta, BR_BHS, true); -} - -static int jsgt_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) -{ - return wrp_cmp_imm(nfp_prog, meta, BR_BLT, true); -} - -static int jsge_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) -{ - return wrp_cmp_imm(nfp_prog, meta, BR_BGE, false); -} - -static int jslt_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) -{ - return wrp_cmp_imm(nfp_prog, meta, BR_BLT, false); -} - -static int jsle_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) -{ - return wrp_cmp_imm(nfp_prog, meta, BR_BGE, true); -} - static int jset_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) { const struct bpf_insn *insn = &meta->insn; @@ -2392,46 +2386,6 @@ static int jeq_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) return 0; } -static int jgt_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) -{ - return wrp_cmp_reg(nfp_prog, meta, BR_BLO, true); -} - -static int jge_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) -{ - return wrp_cmp_reg(nfp_prog, meta, BR_BHS, false); -} - -static int jlt_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) -{ - return wrp_cmp_reg(nfp_prog, meta, BR_BLO, false); -} - -static int jle_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) -{ - return wrp_cmp_reg(nfp_prog, meta, BR_BHS, true); -} - -static int jsgt_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) -{ - return wrp_cmp_reg(nfp_prog, meta, BR_BLT, true); -} - -static int jsge_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) -{ - return wrp_cmp_reg(nfp_prog, meta, BR_BGE, false); -} - -static int jslt_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) -{ - return wrp_cmp_reg(nfp_prog, meta, BR_BLT, false); -} - -static int jsle_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) -{ - return wrp_cmp_reg(nfp_prog, meta, BR_BGE, true); -} - static int jset_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) { return wrp_test_reg(nfp_prog, meta, ALU_OP_AND, BR_BNE); @@ -2520,25 +2474,25 @@ static const instr_cb_t instr_cb[256] = { [BPF_ST | BPF_MEM | BPF_DW] = mem_st8, [BPF_JMP | BPF_JA | BPF_K] = jump, [BPF_JMP | BPF_JEQ | BPF_K] = jeq_imm, - [BPF_JMP | BPF_JGT | BPF_K] = jgt_imm, - [BPF_JMP | BPF_JGE | BPF_K] = jge_imm, - [BPF_JMP | BPF_JLT | BPF_K] = jlt_imm, - [BPF_JMP | BPF_JLE | BPF_K] = jle_imm, - [BPF_JMP | BPF_JSGT | BPF_K] = jsgt_imm, - [BPF_JMP | BPF_JSGE | BPF_K] = jsge_imm, - [BPF_JMP | BPF_JSLT | BPF_K] = jslt_imm, - [BPF_JMP | BPF_JSLE | BPF_K] = jsle_imm, + [BPF_JMP | BPF_JGT | BPF_K] = cmp_imm, + [BPF_JMP | BPF_JGE | BPF_K] = cmp_imm, + [BPF_JMP | BPF_JLT | BPF_K] = cmp_imm, + [BPF_JMP | BPF_JLE | BPF_K] = cmp_imm, + [BPF_JMP | BPF_JSGT | BPF_K] = cmp_imm, + [BPF_JMP | BPF_JSGE | BPF_K] = cmp_imm, + [BPF_JMP | BPF_JSLT | BPF_K] = cmp_imm, + [BPF_JMP | BPF_JSLE | BPF_K] = cmp_imm, [BPF_JMP | BPF_JSET | BPF_K] = jset_imm, [BPF_JMP | BPF_JNE | BPF_K] = jne_imm, [BPF_JMP | BPF_JEQ | BPF_X] = jeq_reg, - [BPF_JMP | BPF_JGT | BPF_X] = jgt_reg, - [BPF_JMP | BPF_JGE | BPF_X] = jge_reg, - [BPF_JMP | BPF_JLT | BPF_X] = jlt_reg, - [BPF_JMP | BPF_JLE | BPF_X] = jle_reg, - [BPF_JMP | BPF_JSGT | BPF_X] = jsgt_reg, - [BPF_JMP | BPF_JSGE | BPF_X] = jsge_reg, - [BPF_JMP | BPF_JSLT | BPF_X] = jslt_reg, - [BPF_JMP | BPF_JSLE | BPF_X] = jsle_reg, + [BPF_JMP | BPF_JGT | BPF_X] = cmp_reg, + [BPF_JMP | BPF_JGE | BPF_X] = cmp_reg, + [BPF_JMP | BPF_JLT | BPF_X] = cmp_reg, + [BPF_JMP | BPF_JLE | BPF_X] = cmp_reg, + [BPF_JMP | BPF_JSGT | BPF_X] = cmp_reg, + [BPF_JMP | BPF_JSGE | BPF_X] = cmp_reg, + [BPF_JMP | BPF_JSLT | BPF_X] = cmp_reg, + [BPF_JMP | BPF_JSLE | BPF_X] = cmp_reg, [BPF_JMP | BPF_JSET | BPF_X] = jset_reg, [BPF_JMP | BPF_JNE | BPF_X] = jne_reg, [BPF_JMP | BPF_CALL] = call, From patchwork Wed Apr 25 04:22:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 903953 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=netronome.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="vA5Ce1+2"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40W6WS2z6vz9ryk for ; Wed, 25 Apr 2018 14:23:16 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751351AbeDYEXO (ORCPT ); Wed, 25 Apr 2018 00:23:14 -0400 Received: from mail-pf0-f193.google.com ([209.85.192.193]:41648 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751277AbeDYEWz (ORCPT ); Wed, 25 Apr 2018 00:22:55 -0400 Received: by mail-pf0-f193.google.com with SMTP id v63so4504320pfk.8 for ; Tue, 24 Apr 2018 21:22:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vcd+/EMZjcxRp59iCdeJMUVpmEEOtU8yCgmZj1Fyk0M=; b=vA5Ce1+29FHq5+D5VYPRnqMFDoaxTo21/VDSZt0eAs7uAnnbcKKdru3Ix/2fAX/olc ff2PBFmd6Y3n5RFVfMhL4I052lq+JAeyXhuZv6pkLs2Luqlm7Lt6ads9DRbsJCfbCSgR OAJj6Ce28nFN/FnRIefWnXkrn7/yU/Q1QxjA4nD3pTVpQuORDukGIKOTZWe8sZIfnXnJ AqD0PYFy+Wwq1EU/LjEKuKnQvvZUNmTyiPWxiHayY3a506hM8jQkmOrgVZblqidxzrlR JcNHWmzp9OZO0mp1ZJyPO2waou3fyCyFWoF2nP5mlcioqpX5rhkW1+kfdlqs4TvLwTfr v+Dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vcd+/EMZjcxRp59iCdeJMUVpmEEOtU8yCgmZj1Fyk0M=; b=OwgJIiDMzwp76CU24N0fPOb7UdNNvELj4oaIqw4kfzF6ReESEtNCd67WpB94gaEdrk Sd5hdQMpoV4tzglnd7O9DQ6mfyMonuPM6z5WAsu9wosMLB1quT4FlxMQcnDQtLo2ryYq d558VGQxxF+ZYLWBPLS7Fom6ECNQ4uviJysWhwxCYrLt57wNlfeGdANLssSbNgCAkg8e W4yhhbmlrrWcogmJWMwpot2aX0wwVPCvmNqyffpvK0a6uR645/S6r2iuP1N2+B9Th6Jg Hgfjfi0gkQoBEsLeyEq9ysN0OAUU+OFf6dDhH9U9FEa7cONO0HtdEDmhuHXQFIBm42Hl qdtg== X-Gm-Message-State: ALQs6tD/QW1B90y8DlT2/UJ1r1VdZwpr/vuk0TJzqrTwN4Xa/ZlqxY0R jvDHVARRlf6MQvYYRmDJ657SSw== X-Google-Smtp-Source: AIpwx4+1zy0p+X7/lAcva43d7Lj3LK0qGO7YNPyxongX7CmImjpubqq+Qgb0AT44gBNndULAyn9blg== X-Received: by 10.99.110.199 with SMTP id j190mr22023143pgc.71.1524630174443; Tue, 24 Apr 2018 21:22:54 -0700 (PDT) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id k14sm22449002pgt.27.2018.04.24.21.22.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Apr 2018 21:22:53 -0700 (PDT) From: Jakub Kicinski To: alexei.starovoitov@gmail.com, daniel@iogearbox.net Cc: oss-drivers@netronome.com, netdev@vger.kernel.org, Jakub Kicinski Subject: [PATCH bpf-next 4/4] nfp: bpf: optimize comparisons to negative constants Date: Tue, 24 Apr 2018 21:22:39 -0700 Message-Id: <20180425042239.27869-5-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180425042239.27869-1-jakub.kicinski@netronome.com> References: <20180425042239.27869-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Comparison instruction requires a subtraction. If the constant is negative we are more likely to fit it into a NFP instruction directly if we change the sign and use addition. Signed-off-by: Jakub Kicinski --- drivers/net/ethernet/netronome/nfp/bpf/jit.c | 42 +++++++++++++++++++-------- drivers/net/ethernet/netronome/nfp/bpf/main.h | 6 +++- 2 files changed, 35 insertions(+), 13 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/bpf/jit.c b/drivers/net/ethernet/netronome/nfp/bpf/jit.c index 5b8da7a67df4..65f0791cae0c 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/jit.c +++ b/drivers/net/ethernet/netronome/nfp/bpf/jit.c @@ -1247,6 +1247,7 @@ static int cmp_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) const struct bpf_insn *insn = &meta->insn; u64 imm = insn->imm; /* sign extend */ const struct jmp_code_map *code; + enum alu_op alu_op, carry_op; u8 reg = insn->dst_reg * 2; swreg tmp_reg; @@ -1254,19 +1255,22 @@ static int cmp_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) if (!code) return -EINVAL; + alu_op = meta->jump_neg_op ? ALU_OP_ADD : ALU_OP_SUB; + carry_op = meta->jump_neg_op ? ALU_OP_ADD_C : ALU_OP_SUB_C; + tmp_reg = ur_load_imm_any(nfp_prog, imm & ~0U, imm_b(nfp_prog)); if (!code->swap) - emit_alu(nfp_prog, reg_none(), reg_a(reg), ALU_OP_SUB, tmp_reg); + emit_alu(nfp_prog, reg_none(), reg_a(reg), alu_op, tmp_reg); else - emit_alu(nfp_prog, reg_none(), tmp_reg, ALU_OP_SUB, reg_a(reg)); + emit_alu(nfp_prog, reg_none(), tmp_reg, alu_op, reg_a(reg)); tmp_reg = ur_load_imm_any(nfp_prog, imm >> 32, imm_b(nfp_prog)); if (!code->swap) emit_alu(nfp_prog, reg_none(), - reg_a(reg + 1), ALU_OP_SUB_C, tmp_reg); + reg_a(reg + 1), carry_op, tmp_reg); else emit_alu(nfp_prog, reg_none(), - tmp_reg, ALU_OP_SUB_C, reg_a(reg + 1)); + tmp_reg, carry_op, reg_a(reg + 1)); emit_br(nfp_prog, code->br_mask, insn->off, 0); @@ -2745,21 +2749,35 @@ static void nfp_bpf_opt_neg_add_sub(struct nfp_prog *nfp_prog) continue; if (BPF_CLASS(insn.code) != BPF_ALU && - BPF_CLASS(insn.code) != BPF_ALU64) + BPF_CLASS(insn.code) != BPF_ALU64 && + BPF_CLASS(insn.code) != BPF_JMP) continue; if (BPF_SRC(insn.code) != BPF_K) continue; if (insn.imm >= 0) continue; - if (BPF_OP(insn.code) == BPF_ADD) - insn.code = BPF_CLASS(insn.code) | BPF_SUB; - else if (BPF_OP(insn.code) == BPF_SUB) - insn.code = BPF_CLASS(insn.code) | BPF_ADD; - else - continue; + if (BPF_CLASS(insn.code) == BPF_JMP) { + switch (BPF_OP(insn.code)) { + case BPF_JGE: + case BPF_JSGE: + case BPF_JLT: + case BPF_JSLT: + meta->jump_neg_op = true; + break; + default: + continue; + } + } else { + if (BPF_OP(insn.code) == BPF_ADD) + insn.code = BPF_CLASS(insn.code) | BPF_SUB; + else if (BPF_OP(insn.code) == BPF_SUB) + insn.code = BPF_CLASS(insn.code) | BPF_ADD; + else + continue; - meta->insn.code = insn.code | BPF_K; + meta->insn.code = insn.code | BPF_K; + } meta->insn.imm = -insn.imm; } diff --git a/drivers/net/ethernet/netronome/nfp/bpf/main.h b/drivers/net/ethernet/netronome/nfp/bpf/main.h index 4981c8944ca3..68b5d326483d 100644 --- a/drivers/net/ethernet/netronome/nfp/bpf/main.h +++ b/drivers/net/ethernet/netronome/nfp/bpf/main.h @@ -236,6 +236,7 @@ struct nfp_bpf_reg_state { * @xadd_over_16bit: 16bit immediate is not guaranteed * @xadd_maybe_16bit: 16bit immediate is possible * @jmp_dst: destination info for jump instructions + * @jump_neg_op: jump instruction has inverted immediate, use ADD instead of SUB * @func_id: function id for call instructions * @arg1: arg1 for call instructions * @arg2: arg2 for call instructions @@ -264,7 +265,10 @@ struct nfp_insn_meta { bool xadd_maybe_16bit; }; /* jump */ - struct nfp_insn_meta *jmp_dst; + struct { + struct nfp_insn_meta *jmp_dst; + bool jump_neg_op; + }; /* function calls */ struct { u32 func_id;