| Message ID | 20180517010948.21249-1-daniel@iogearbox.net |
|---|---|
| State | Accepted, archived |
| Delegated to: | David Miller |
| Headers | show |
| Series | pull-request: bpf-next 2018-05-17 | expand |
From: Daniel Borkmann <daniel@iogearbox.net> Date: Thu, 17 May 2018 03:09:48 +0200 > The following pull-request contains BPF updates for your *net-next* > tree. Looks good, pulled, thanks Daniel.
Hi David, The following pull-request contains BPF updates for your *net-next* tree. The main changes are: 1) Provide a new BPF helper for doing a FIB and neighbor lookup in the kernel tables from an XDP or tc BPF program. The helper provides a fast-path for forwarding packets. The API supports IPv4, IPv6 and MPLS protocols, but currently IPv4 and IPv6 are implemented in this initial work, from David (Ahern). 2) Just a tiny diff but huge feature enabled for nfp driver by extending the BPF offload beyond a pure host processing offload. Offloaded XDP programs are allowed to set the RX queue index and thus opening the door for defining a fully programmable RSS/n-tuple filter replacement. Once BPF decided on a queue already, the device data-path will skip the conventional RSS processing completely, from Jakub. 3) The original sockmap implementation was array based similar to devmap. However unlike devmap where an ifindex has a 1:1 mapping into the map there are use cases with sockets that need to be referenced using longer keys. Hence, sockhash map is added reusing as much of the sockmap code as possible, from John. 4) Introduce BTF ID. The ID is allocatd through an IDR similar as with BPF maps and progs. It also makes BTF accessible to user space via BPF_BTF_GET_FD_BY_ID and adds exposure of the BTF data through BPF_OBJ_GET_INFO_BY_FD, from Martin. 5) Enable BPF stackmap with build_id also in NMI context. Due to the up_read() of current->mm->mmap_sem build_id cannot be parsed. This work defers the up_read() via a per-cpu irq_work so that at least limited support can be enabled, from Song. 6) Various BPF JIT follow-up cleanups and fixups after the LD_ABS/LD_IND JIT conversion as well as implementation of an optimized 32/64 bit immediate load in the arm64 JIT that allows to reduce the number of emitted instructions; in case of tested real-world programs they were shrinking by three percent, from Daniel. 7) Add ifindex parameter to the libbpf loader in order to enable BPF offload support. Right now only iproute2 can load offloaded BPF and this will also enable libbpf for direct integration into other applications, from David (Beckett). 8) Convert the plain text documentation under Documentation/bpf/ into RST format since this is the appropriate standard the kernel is moving to for all documentation. Also add an overview README.rst, from Jesper. 9) Add __printf verification attribute to the bpf_verifier_vlog() helper. Though it uses va_list we can still allow gcc to check the format string, from Mathieu. 10) Fix a bash reference in the BPF selftest's Makefile. The '|& ...' is a bash 4.0+ feature which is not guaranteed to be available when calling out to shell, therefore use a more portable variant, from Joe. 11) Fix a 64 bit division in xdp_umem_reg() by using div_u64() instead of relying on the gcc built-in, from Björn. 12) Fix a sock hashmap kmalloc warning reported by syzbot when an overly large key size is used in hashmap then causing overflows in htab->elem_size. Reject bogus attr->key_size early in the sock_hash_alloc(), from Yonghong. 13) Ensure in BPF selftests when urandom_read is being linked that --build-id is always enabled so that test_stacktrace_build_id[_nmi] won't be failing, from Alexei. 14) Add bitsperlong.h as well as errno.h uapi headers into the tools header infrastructure which point to one of the arch specific uapi headers. This was needed in order to fix a build error on some systems for the BPF selftests, from Sirio. 15) Allow for short options to be used in the xdp_monitor BPF sample code. And also a bpf.h tools uapi header sync in order to fix a selftest build failure. Both from Prashant. 16) More formally clarify the meaning of ID in the direct packet access section of the BPF documentation, from Wang. Please consider pulling these changes from: git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git Thanks a lot! ---------------------------------------------------------------- The following changes since commit 53a7bdfb2a2756cce8003b90817f8a6fb4d830d9: dt-bindings: dsa: Remove unnecessary #address/#size-cells (2018-05-08 20:28:44 -0400) are available in the git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git for you to fetch changes up to e23afe5e7cba89cd0744c5218eda1b3553455c17: bpf: sockmap, on update propagate errors back to userspace (2018-05-17 01:48:22 +0200) ---------------------------------------------------------------- Alexei Starovoitov (4): Merge branch 'bpf-jit-cleanups' Merge branch 'fix-samples' Merge branch 'convert-doc-to-rst' selftests/bpf: make sure build-id is on Björn Töpel (1): xsk: fix 64-bit division Daniel Borkmann (14): Merge branch 'bpf-btf-id' Merge branch 'bpf-nfp-programmable-rss' Merge branch 'bpf-fib-lookup-helper' Merge branch 'bpf-perf-rb-libbpf' Merge branch 'bpf-stackmap-nmi' bpf, mips: remove unused function bpf, sparc: remove unused variable bpf, x64: clean up retpoline emission slightly bpf, arm32: save 4 bytes of unneeded stack space bpf, arm64: save 4 bytes of unneeded stack space bpf, arm64: optimize 32/64 immediate emission bpf, arm64: save 4 bytes in prologue when ebpf insns came from cbpf bpf: add ld64 imm test cases Merge branch 'bpf-sock-hashmap' David Ahern (10): net/ipv6: Rename fib6_lookup to fib6_node_lookup net/ipv6: Rename rt6_multipath_select net/ipv6: Extract table lookup from ip6_pol_route net/ipv6: Refactor fib6_rule_action net/ipv6: Add fib6_lookup net/ipv6: Update fib6 tracepoint to take fib6_info net/ipv6: Add fib lookup stubs for use in bpf helper bpf: Provide helper to do forwarding lookups in kernel FIB table samples/bpf: Add example of ipv4 and ipv6 forwarding in XDP samples/bpf: Decrement ttl in fib forwarding example David Beckett (1): libbpf: add ifindex to enable offload support Jakub Kicinski (14): bpf: xdp: allow offloads to store into rx_queue_index nfp: bpf: support setting the RX queue index tools: bpftool: use PERF_SAMPLE_TIME instead of reading the clock samples: bpf: rename struct bpf_map_def to avoid conflict with libbpf samples: bpf: compile and link against full libbpf tools: bpf: move the event reading loop to libbpf tools: bpf: improve comments in libbpf.h tools: bpf: don't complain about no kernel version for networking code samples: bpf: convert some XDP samples from bpf_load to libbpf samples: bpf: include bpf/bpf.h instead of local libbpf.h samples: bpf: rename libbpf.h to bpf_insn.h samples: bpf: fix build after move to compiling full libbpf.a samples: bpf: move libbpf from object dependencies to libs samples: bpf: make the build less noisy Jesper Dangaard Brouer (5): bpf, doc: add basic README.rst file bpf, doc: rename txt files to rst files bpf, doc: convert bpf_design_QA.rst to use RST formatting bpf, doc: convert bpf_devel_QA.rst to use RST formatting bpf, doc: howto use/run the BPF selftests Joe Stringer (1): selftests/bpf: Fix bash reference in Makefile John Fastabend (5): bpf: sockmap, refactor sockmap routines to work with hashmap bpf: sockmap, add hash map support bpf: selftest additions for SOCKHASH bpf: bpftool, support for sockhash bpf: sockmap, on update propagate errors back to userspace Martin KaFai Lau (6): bpf: btf: Avoid WARN_ON when CONFIG_REFCOUNT_FULL=y bpf: btf: Introduce BTF ID bpf: btf: Add struct bpf_btf_info bpf: btf: Some test_btf clean up bpf: btf: Update tools/include/uapi/linux/btf.h with BTF ID bpf: btf: Tests for BPF_OBJ_GET_INFO_BY_FD and BPF_BTF_GET_FD_BY_ID Mathieu Malaterre (1): bpf: add __printf verification to bpf_verifier_vlog Prashant Bhole (2): bpf: sync tools bpf.h uapi header samples/bpf: xdp_monitor, accept short options Sirio Balmelli (2): selftests/bpf: add architecture-agnostic headers selftests/bpf: ignore build products Song Liu (2): bpf: enable stackmap with build_id in nmi context bpf: add selftest for stackmap with build_id in NMI context Wang YanQing (1): bpf, doc: clarification for the meaning of 'id' Yonghong Song (1): bpf: fix sock hashmap kmalloc warning Documentation/bpf/README.rst | 36 ++ Documentation/bpf/bpf_design_QA.rst | 221 ++++++++ Documentation/bpf/bpf_design_QA.txt | 156 ------ Documentation/bpf/bpf_devel_QA.rst | 640 +++++++++++++++++++++ Documentation/bpf/bpf_devel_QA.txt | 570 ------------------- Documentation/networking/filter.txt | 15 +- arch/arm/net/bpf_jit_32.c | 13 +- arch/arm64/net/bpf_jit_comp.c | 115 ++-- arch/mips/net/ebpf_jit.c | 26 - arch/sparc/net/bpf_jit_comp_64.c | 1 - arch/x86/include/asm/nospec-branch.h | 29 +- drivers/net/ethernet/netronome/nfp/bpf/fw.h | 1 + drivers/net/ethernet/netronome/nfp/bpf/jit.c | 47 ++ drivers/net/ethernet/netronome/nfp/bpf/main.c | 11 + drivers/net/ethernet/netronome/nfp/bpf/main.h | 8 + drivers/net/ethernet/netronome/nfp/bpf/verifier.c | 28 +- drivers/net/ethernet/netronome/nfp/nfp_asm.h | 22 +- include/linux/bpf.h | 10 +- include/linux/bpf_types.h | 1 + include/linux/bpf_verifier.h | 4 +- include/linux/btf.h | 2 + include/linux/filter.h | 3 +- include/net/addrconf.h | 14 + include/net/ip6_fib.h | 21 +- include/net/tcp.h | 3 +- include/trace/events/fib6.h | 14 +- include/uapi/linux/bpf.h | 142 ++++- init/Kconfig | 1 + kernel/bpf/btf.c | 136 ++++- kernel/bpf/core.c | 1 + kernel/bpf/sockmap.c | 644 +++++++++++++++++++--- kernel/bpf/stackmap.c | 59 +- kernel/bpf/syscall.c | 41 +- kernel/bpf/verifier.c | 16 +- net/core/filter.c | 365 +++++++++++- net/ipv6/addrconf_core.c | 33 +- net/ipv6/af_inet6.c | 6 +- net/ipv6/fib6_rules.c | 138 ++++- net/ipv6/ip6_fib.c | 21 +- net/ipv6/route.c | 76 +-- net/xdp/xdp_umem.c | 2 +- samples/bpf/Makefile | 166 +++--- samples/bpf/{libbpf.h => bpf_insn.h} | 8 +- samples/bpf/bpf_load.c | 12 +- samples/bpf/bpf_load.h | 6 +- samples/bpf/cookie_uid_helper_example.c | 2 +- samples/bpf/cpustat_user.c | 2 +- samples/bpf/fds_example.c | 4 +- samples/bpf/lathist_user.c | 2 +- samples/bpf/load_sock_ops.c | 2 +- samples/bpf/lwt_len_hist_user.c | 2 +- samples/bpf/map_perf_test_user.c | 2 +- samples/bpf/sock_example.c | 3 +- samples/bpf/sock_example.h | 1 - samples/bpf/sockex1_user.c | 2 +- samples/bpf/sockex2_user.c | 2 +- samples/bpf/sockex3_user.c | 2 +- samples/bpf/syscall_tp_user.c | 2 +- samples/bpf/tc_l2_redirect_user.c | 2 +- samples/bpf/test_cgrp2_array_pin.c | 2 +- samples/bpf/test_cgrp2_attach.c | 3 +- samples/bpf/test_cgrp2_attach2.c | 3 +- samples/bpf/test_cgrp2_sock.c | 3 +- samples/bpf/test_cgrp2_sock2.c | 3 +- samples/bpf/test_current_task_under_cgroup_user.c | 2 +- samples/bpf/test_lru_dist.c | 2 +- samples/bpf/test_map_in_map_user.c | 2 +- samples/bpf/test_overhead_user.c | 2 +- samples/bpf/test_probe_write_user_user.c | 2 +- samples/bpf/trace_output_user.c | 8 +- samples/bpf/tracex1_user.c | 2 +- samples/bpf/tracex2_user.c | 2 +- samples/bpf/tracex3_user.c | 2 +- samples/bpf/tracex4_user.c | 2 +- samples/bpf/tracex5_user.c | 2 +- samples/bpf/tracex6_user.c | 2 +- samples/bpf/tracex7_user.c | 2 +- samples/bpf/xdp1_user.c | 31 +- samples/bpf/xdp_adjust_tail_user.c | 36 +- samples/bpf/xdp_fwd_kern.c | 138 +++++ samples/bpf/xdp_fwd_user.c | 136 +++++ samples/bpf/xdp_monitor_user.c | 6 +- samples/bpf/xdp_redirect_cpu_user.c | 2 +- samples/bpf/xdp_redirect_map_user.c | 2 +- samples/bpf/xdp_redirect_user.c | 2 +- samples/bpf/xdp_router_ipv4_user.c | 2 +- samples/bpf/xdp_rxq_info_user.c | 46 +- samples/bpf/xdp_tx_iptunnel_user.c | 2 +- samples/bpf/xdpsock_user.c | 2 +- tools/bpf/bpftool/.gitignore | 3 + tools/bpf/bpftool/map.c | 1 + tools/bpf/bpftool/map_perf_ring.c | 83 +-- tools/include/uapi/asm/bitsperlong.h | 18 + tools/include/uapi/asm/errno.h | 18 + tools/include/uapi/linux/bpf.h | 143 ++++- tools/lib/bpf/Makefile | 2 +- tools/lib/bpf/bpf.c | 12 + tools/lib/bpf/bpf.h | 3 + tools/lib/bpf/libbpf.c | 125 ++++- tools/lib/bpf/libbpf.h | 62 ++- tools/testing/selftests/bpf/.gitignore | 1 + tools/testing/selftests/bpf/Makefile | 12 +- tools/testing/selftests/bpf/bpf_helpers.h | 11 + tools/testing/selftests/bpf/bpf_rand.h | 80 +++ tools/testing/selftests/bpf/test_btf.c | 478 ++++++++++++---- tools/testing/selftests/bpf/test_progs.c | 140 ++++- tools/testing/selftests/bpf/test_sockhash_kern.c | 5 + tools/testing/selftests/bpf/test_sockmap.c | 27 +- tools/testing/selftests/bpf/test_sockmap_kern.c | 343 +----------- tools/testing/selftests/bpf/test_sockmap_kern.h | 363 ++++++++++++ tools/testing/selftests/bpf/test_verifier.c | 62 +++ tools/testing/selftests/bpf/trace_helpers.c | 87 +-- tools/testing/selftests/bpf/trace_helpers.h | 11 +- tools/testing/selftests/bpf/urandom_read.c | 10 +- 114 files changed, 4600 insertions(+), 1865 deletions(-) create mode 100644 Documentation/bpf/README.rst create mode 100644 Documentation/bpf/bpf_design_QA.rst delete mode 100644 Documentation/bpf/bpf_design_QA.txt create mode 100644 Documentation/bpf/bpf_devel_QA.rst delete mode 100644 Documentation/bpf/bpf_devel_QA.txt rename samples/bpf/{libbpf.h => bpf_insn.h} (98%) create mode 100644 samples/bpf/xdp_fwd_kern.c create mode 100644 samples/bpf/xdp_fwd_user.c create mode 100644 tools/bpf/bpftool/.gitignore create mode 100644 tools/include/uapi/asm/bitsperlong.h create mode 100644 tools/include/uapi/asm/errno.h create mode 100644 tools/testing/selftests/bpf/bpf_rand.h create mode 100644 tools/testing/selftests/bpf/test_sockhash_kern.c create mode 100644 tools/testing/selftests/bpf/test_sockmap_kern.h