From patchwork Fri Sep 21 17:10:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Stringer X-Patchwork-Id: 973330 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=wand.net.nz Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="BGbV3pjp"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 42H0VX2KNMz9sC7 for ; Sat, 22 Sep 2018 03:11:00 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390885AbeIUXAq (ORCPT ); Fri, 21 Sep 2018 19:00:46 -0400 Received: from mail-yb1-f193.google.com ([209.85.219.193]:43057 "EHLO mail-yb1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390788AbeIUXAp (ORCPT ); Fri, 21 Sep 2018 19:00:45 -0400 Received: by mail-yb1-f193.google.com with SMTP id w80-v6so5712634ybe.10 for ; Fri, 21 Sep 2018 10:10:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=PdlLq3uDBvNGnXuIYJtgHg9yTfg9mfTeaHBXcUxlKGc=; b=BGbV3pjpEKHG4+G4YlF/CSQmeXonD/NfXBPd/XTRiM4o1XrFrQHI/E7ALg3DXlN2pj 7SHWaEp642QPKKJBIYanXYpcZhMXeLXnfFTkr8kJrZwV6AGIj3mZKP2rplN+JAPaeaEl zUebdAMEmN7k0xKF+XC6E+vOvP0/8HFGzgzPfjdFk35+pdcYK8Vfw2Gs7d89TD9uoV6t 1U4hVZQgTBzSNh0fiVGktzOwPm5TqdK5njZr+Y4uWcHLDTDeXIll0D5D2rV8mAT8QJwW O0DY9IejQhpvAqH63Mn1qNsL9+LD2qzVJ1YeyQut1Ql69NPprdNezq8uyZeWhtQ+94qp D7Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=PdlLq3uDBvNGnXuIYJtgHg9yTfg9mfTeaHBXcUxlKGc=; b=QVHTA6P8jLqoaLnvT1uzXJjl/TNYbRwXz22LZIr34BdauxlUlgOv5MkeQWFoizjTeU lWcyFTLCmlA77mqq3wIpE7pIcSGKod265tbmT9LhtUen7XX6THuPSJRmEcRq0lnjznHm j3JlCXh5gksY02OAejKFjebOqKvoZ9BLiQ2k1GLydJEM9XjK4/rYJpurpDNsst8Ce5WW x+NFaYV4eR+ZHCMUr6AqYsKerb35CAgGSqfnKF41/RqDHKelpVEKeWzia8IDDV7NHS/R AP6+FwqQpdRl1xTeBjzezlX1xK6VMIQK2rFBVsAosGfI3CDOmMAhmE21rdkex8V63rAR H8EQ== X-Gm-Message-State: APzg51C7U/vI11j3vya+CvddXX7JhzLY8v3v6FS2sZUUuHLj653Q1n4Z SDkUrcQlS7MPk0oMCCfdpKU= X-Google-Smtp-Source: ANB0VdY4rn2QtL0aJ0o/pTSczREo8xFkqnRIN2hGY9SAHMawiZ2gGMZ0wzsqsgt8caCAtG0Fvg6eEw== X-Received: by 2002:a5b:44b:: with SMTP id s11-v6mr20793896ybp.505.1537549856328; Fri, 21 Sep 2018 10:10:56 -0700 (PDT) Received: from localhost.localdomain ([99.0.85.34]) by smtp.gmail.com with ESMTPSA id h10-v6sm9930972ywa.35.2018.09.21.10.10.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Sep 2018 10:10:55 -0700 (PDT) From: Joe Stringer To: ast@kernel.org Cc: netdev@vger.kernel.org, daniel@iogearbox.net, john.fastabend@gmail.com, tgraf@suug.ch, kafai@fb.com, nitin.hande@gmail.com, mauricio.vasquez@polito.it Subject: [PATCHv2 bpf-next 05/11] bpf: Macrofy stack state copy Date: Fri, 21 Sep 2018 10:10:37 -0700 Message-Id: <20180921171043.20823-6-joe@wand.net.nz> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180921171043.20823-1-joe@wand.net.nz> References: <20180921171043.20823-1-joe@wand.net.nz> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org An upcoming commit will need very similar copy/realloc boilerplate, so refactor the existing stack copy/realloc functions into macros to simplify it. Signed-off-by: Joe Stringer Acked-by: Alexei Starovoitov --- kernel/bpf/verifier.c | 106 ++++++++++++++++++++++++------------------ 1 file changed, 60 insertions(+), 46 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 1fee63d82290..311340360aa3 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -388,60 +388,74 @@ static void print_verifier_state(struct bpf_verifier_env *env, verbose(env, "\n"); } -static int copy_stack_state(struct bpf_func_state *dst, - const struct bpf_func_state *src) -{ - if (!src->stack) - return 0; - if (WARN_ON_ONCE(dst->allocated_stack < src->allocated_stack)) { - /* internal bug, make state invalid to reject the program */ - memset(dst, 0, sizeof(*dst)); - return -EFAULT; - } - memcpy(dst->stack, src->stack, - sizeof(*src->stack) * (src->allocated_stack / BPF_REG_SIZE)); - return 0; -} +#define COPY_STATE_FN(NAME, COUNT, FIELD, SIZE) \ +static int copy_##NAME##_state(struct bpf_func_state *dst, \ + const struct bpf_func_state *src) \ +{ \ + if (!src->FIELD) \ + return 0; \ + if (WARN_ON_ONCE(dst->COUNT < src->COUNT)) { \ + /* internal bug, make state invalid to reject the program */ \ + memset(dst, 0, sizeof(*dst)); \ + return -EFAULT; \ + } \ + memcpy(dst->FIELD, src->FIELD, \ + sizeof(*src->FIELD) * (src->COUNT / SIZE)); \ + return 0; \ +} +/* copy_stack_state() */ +COPY_STATE_FN(stack, allocated_stack, stack, BPF_REG_SIZE) +#undef COPY_STATE_FN + +#define REALLOC_STATE_FN(NAME, COUNT, FIELD, SIZE) \ +static int realloc_##NAME##_state(struct bpf_func_state *state, int size, \ + bool copy_old) \ +{ \ + u32 old_size = state->COUNT; \ + struct bpf_##NAME##_state *new_##FIELD; \ + int slot = size / SIZE; \ + \ + if (size <= old_size || !size) { \ + if (copy_old) \ + return 0; \ + state->COUNT = slot * SIZE; \ + if (!size && old_size) { \ + kfree(state->FIELD); \ + state->FIELD = NULL; \ + } \ + return 0; \ + } \ + new_##FIELD = kmalloc_array(slot, sizeof(struct bpf_##NAME##_state), \ + GFP_KERNEL); \ + if (!new_##FIELD) \ + return -ENOMEM; \ + if (copy_old) { \ + if (state->FIELD) \ + memcpy(new_##FIELD, state->FIELD, \ + sizeof(*new_##FIELD) * (old_size / SIZE)); \ + memset(new_##FIELD + old_size / SIZE, 0, \ + sizeof(*new_##FIELD) * (size - old_size) / SIZE); \ + } \ + state->COUNT = slot * SIZE; \ + kfree(state->FIELD); \ + state->FIELD = new_##FIELD; \ + return 0; \ +} +/* realloc_stack_state() */ +REALLOC_STATE_FN(stack, allocated_stack, stack, BPF_REG_SIZE) +#undef REALLOC_STATE_FN /* do_check() starts with zero-sized stack in struct bpf_verifier_state to * make it consume minimal amount of memory. check_stack_write() access from * the program calls into realloc_func_state() to grow the stack size. - * Note there is a non-zero parent pointer inside each reg of bpf_verifier_state - * which this function copies over. It points to corresponding reg in previous - * bpf_verifier_state which is never reallocated + * Note there is a non-zero 'parent' pointer inside bpf_verifier_state + * which realloc_stack_state() copies over. It points to previous + * bpf_verifier_state which is never reallocated. */ static int realloc_func_state(struct bpf_func_state *state, int size, bool copy_old) { - u32 old_size = state->allocated_stack; - struct bpf_stack_state *new_stack; - int slot = size / BPF_REG_SIZE; - - if (size <= old_size || !size) { - if (copy_old) - return 0; - state->allocated_stack = slot * BPF_REG_SIZE; - if (!size && old_size) { - kfree(state->stack); - state->stack = NULL; - } - return 0; - } - new_stack = kmalloc_array(slot, sizeof(struct bpf_stack_state), - GFP_KERNEL); - if (!new_stack) - return -ENOMEM; - if (copy_old) { - if (state->stack) - memcpy(new_stack, state->stack, - sizeof(*new_stack) * (old_size / BPF_REG_SIZE)); - memset(new_stack + old_size / BPF_REG_SIZE, 0, - sizeof(*new_stack) * (size - old_size) / BPF_REG_SIZE); - } - state->allocated_stack = slot * BPF_REG_SIZE; - kfree(state->stack); - state->stack = new_stack; - return 0; + return realloc_stack_state(state, size, copy_old); } static void free_func_state(struct bpf_func_state *state)