pull-request: bpf-next 2018-05-17

Message ID 20180517010948.21249-1-daniel@iogearbox.net
State Accepted
Delegated to: David Miller
Headers show
Series
  • pull-request: bpf-next 2018-05-17
Related show

Pull-request

git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git

Message

Daniel Borkmann May 17, 2018, 1:09 a.m.
Hi David,

The following pull-request contains BPF updates for your *net-next* tree.

The main changes are:

1) Provide a new BPF helper for doing a FIB and neighbor lookup
   in the kernel tables from an XDP or tc BPF program. The helper
   provides a fast-path for forwarding packets. The API supports
   IPv4, IPv6 and MPLS protocols, but currently IPv4 and IPv6 are
   implemented in this initial work, from David (Ahern).

2) Just a tiny diff but huge feature enabled for nfp driver by
   extending the BPF offload beyond a pure host processing offload.
   Offloaded XDP programs are allowed to set the RX queue index and
   thus opening the door for defining a fully programmable RSS/n-tuple
   filter replacement. Once BPF decided on a queue already, the device
   data-path will skip the conventional RSS processing completely,
   from Jakub.

3) The original sockmap implementation was array based similar to
   devmap. However unlike devmap where an ifindex has a 1:1 mapping
   into the map there are use cases with sockets that need to be
   referenced using longer keys. Hence, sockhash map is added reusing
   as much of the sockmap code as possible, from John.

4) Introduce BTF ID. The ID is allocatd through an IDR similar as
   with BPF maps and progs. It also makes BTF accessible to user
   space via BPF_BTF_GET_FD_BY_ID and adds exposure of the BTF data
   through BPF_OBJ_GET_INFO_BY_FD, from Martin.

5) Enable BPF stackmap with build_id also in NMI context. Due to the
   up_read() of current->mm->mmap_sem build_id cannot be parsed.
   This work defers the up_read() via a per-cpu irq_work so that
   at least limited support can be enabled, from Song.

6) Various BPF JIT follow-up cleanups and fixups after the LD_ABS/LD_IND
   JIT conversion as well as implementation of an optimized 32/64 bit
   immediate load in the arm64 JIT that allows to reduce the number of
   emitted instructions; in case of tested real-world programs they
   were shrinking by three percent, from Daniel.

7) Add ifindex parameter to the libbpf loader in order to enable
   BPF offload support. Right now only iproute2 can load offloaded
   BPF and this will also enable libbpf for direct integration into
   other applications, from David (Beckett).

8) Convert the plain text documentation under Documentation/bpf/ into
   RST format since this is the appropriate standard the kernel is
   moving to for all documentation. Also add an overview README.rst,
   from Jesper.

9) Add __printf verification attribute to the bpf_verifier_vlog()
   helper. Though it uses va_list we can still allow gcc to check
   the format string, from Mathieu.

10) Fix a bash reference in the BPF selftest's Makefile. The '|& ...'
    is a bash 4.0+ feature which is not guaranteed to be available
    when calling out to shell, therefore use a more portable variant,
    from Joe.

11) Fix a 64 bit division in xdp_umem_reg() by using div_u64()
    instead of relying on the gcc built-in, from Björn.

12) Fix a sock hashmap kmalloc warning reported by syzbot when an
    overly large key size is used in hashmap then causing overflows
    in htab->elem_size. Reject bogus attr->key_size early in the
    sock_hash_alloc(), from Yonghong.

13) Ensure in BPF selftests when urandom_read is being linked that
    --build-id is always enabled so that test_stacktrace_build_id[_nmi]
    won't be failing, from Alexei.

14) Add bitsperlong.h as well as errno.h uapi headers into the tools
    header infrastructure which point to one of the arch specific
    uapi headers. This was needed in order to fix a build error on
    some systems for the BPF selftests, from Sirio.

15) Allow for short options to be used in the xdp_monitor BPF sample
    code. And also a bpf.h tools uapi header sync in order to fix a
    selftest build failure. Both from Prashant.

16) More formally clarify the meaning of ID in the direct packet access
    section of the BPF documentation, from Wang.

Please consider pulling these changes from:

  git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git

Thanks a lot!

----------------------------------------------------------------

The following changes since commit 53a7bdfb2a2756cce8003b90817f8a6fb4d830d9:

  dt-bindings: dsa: Remove unnecessary #address/#size-cells (2018-05-08 20:28:44 -0400)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git 

for you to fetch changes up to e23afe5e7cba89cd0744c5218eda1b3553455c17:

  bpf: sockmap, on update propagate errors back to userspace (2018-05-17 01:48:22 +0200)

----------------------------------------------------------------
Alexei Starovoitov (4):
      Merge branch 'bpf-jit-cleanups'
      Merge branch 'fix-samples'
      Merge branch 'convert-doc-to-rst'
      selftests/bpf: make sure build-id is on

Björn Töpel (1):
      xsk: fix 64-bit division

Daniel Borkmann (14):
      Merge branch 'bpf-btf-id'
      Merge branch 'bpf-nfp-programmable-rss'
      Merge branch 'bpf-fib-lookup-helper'
      Merge branch 'bpf-perf-rb-libbpf'
      Merge branch 'bpf-stackmap-nmi'
      bpf, mips: remove unused function
      bpf, sparc: remove unused variable
      bpf, x64: clean up retpoline emission slightly
      bpf, arm32: save 4 bytes of unneeded stack space
      bpf, arm64: save 4 bytes of unneeded stack space
      bpf, arm64: optimize 32/64 immediate emission
      bpf, arm64: save 4 bytes in prologue when ebpf insns came from cbpf
      bpf: add ld64 imm test cases
      Merge branch 'bpf-sock-hashmap'

David Ahern (10):
      net/ipv6: Rename fib6_lookup to fib6_node_lookup
      net/ipv6: Rename rt6_multipath_select
      net/ipv6: Extract table lookup from ip6_pol_route
      net/ipv6: Refactor fib6_rule_action
      net/ipv6: Add fib6_lookup
      net/ipv6: Update fib6 tracepoint to take fib6_info
      net/ipv6: Add fib lookup stubs for use in bpf helper
      bpf: Provide helper to do forwarding lookups in kernel FIB table
      samples/bpf: Add example of ipv4 and ipv6 forwarding in XDP
      samples/bpf: Decrement ttl in fib forwarding example

David Beckett (1):
      libbpf: add ifindex to enable offload support

Jakub Kicinski (14):
      bpf: xdp: allow offloads to store into rx_queue_index
      nfp: bpf: support setting the RX queue index
      tools: bpftool: use PERF_SAMPLE_TIME instead of reading the clock
      samples: bpf: rename struct bpf_map_def to avoid conflict with libbpf
      samples: bpf: compile and link against full libbpf
      tools: bpf: move the event reading loop to libbpf
      tools: bpf: improve comments in libbpf.h
      tools: bpf: don't complain about no kernel version for networking code
      samples: bpf: convert some XDP samples from bpf_load to libbpf
      samples: bpf: include bpf/bpf.h instead of local libbpf.h
      samples: bpf: rename libbpf.h to bpf_insn.h
      samples: bpf: fix build after move to compiling full libbpf.a
      samples: bpf: move libbpf from object dependencies to libs
      samples: bpf: make the build less noisy

Jesper Dangaard Brouer (5):
      bpf, doc: add basic README.rst file
      bpf, doc: rename txt files to rst files
      bpf, doc: convert bpf_design_QA.rst to use RST formatting
      bpf, doc: convert bpf_devel_QA.rst to use RST formatting
      bpf, doc: howto use/run the BPF selftests

Joe Stringer (1):
      selftests/bpf: Fix bash reference in Makefile

John Fastabend (5):
      bpf: sockmap, refactor sockmap routines to work with hashmap
      bpf: sockmap, add hash map support
      bpf: selftest additions for SOCKHASH
      bpf: bpftool, support for sockhash
      bpf: sockmap, on update propagate errors back to userspace

Martin KaFai Lau (6):
      bpf: btf: Avoid WARN_ON when CONFIG_REFCOUNT_FULL=y
      bpf: btf: Introduce BTF ID
      bpf: btf: Add struct bpf_btf_info
      bpf: btf: Some test_btf clean up
      bpf: btf: Update tools/include/uapi/linux/btf.h with BTF ID
      bpf: btf: Tests for BPF_OBJ_GET_INFO_BY_FD and BPF_BTF_GET_FD_BY_ID

Mathieu Malaterre (1):
      bpf: add __printf verification to bpf_verifier_vlog

Prashant Bhole (2):
      bpf: sync tools bpf.h uapi header
      samples/bpf: xdp_monitor, accept short options

Sirio Balmelli (2):
      selftests/bpf: add architecture-agnostic headers
      selftests/bpf: ignore build products

Song Liu (2):
      bpf: enable stackmap with build_id in nmi context
      bpf: add selftest for stackmap with build_id in NMI context

Wang YanQing (1):
      bpf, doc: clarification for the meaning of 'id'

Yonghong Song (1):
      bpf: fix sock hashmap kmalloc warning

 Documentation/bpf/README.rst                      |  36 ++
 Documentation/bpf/bpf_design_QA.rst               | 221 ++++++++
 Documentation/bpf/bpf_design_QA.txt               | 156 ------
 Documentation/bpf/bpf_devel_QA.rst                | 640 +++++++++++++++++++++
 Documentation/bpf/bpf_devel_QA.txt                | 570 -------------------
 Documentation/networking/filter.txt               |  15 +-
 arch/arm/net/bpf_jit_32.c                         |  13 +-
 arch/arm64/net/bpf_jit_comp.c                     | 115 ++--
 arch/mips/net/ebpf_jit.c                          |  26 -
 arch/sparc/net/bpf_jit_comp_64.c                  |   1 -
 arch/x86/include/asm/nospec-branch.h              |  29 +-
 drivers/net/ethernet/netronome/nfp/bpf/fw.h       |   1 +
 drivers/net/ethernet/netronome/nfp/bpf/jit.c      |  47 ++
 drivers/net/ethernet/netronome/nfp/bpf/main.c     |  11 +
 drivers/net/ethernet/netronome/nfp/bpf/main.h     |   8 +
 drivers/net/ethernet/netronome/nfp/bpf/verifier.c |  28 +-
 drivers/net/ethernet/netronome/nfp/nfp_asm.h      |  22 +-
 include/linux/bpf.h                               |  10 +-
 include/linux/bpf_types.h                         |   1 +
 include/linux/bpf_verifier.h                      |   4 +-
 include/linux/btf.h                               |   2 +
 include/linux/filter.h                            |   3 +-
 include/net/addrconf.h                            |  14 +
 include/net/ip6_fib.h                             |  21 +-
 include/net/tcp.h                                 |   3 +-
 include/trace/events/fib6.h                       |  14 +-
 include/uapi/linux/bpf.h                          | 142 ++++-
 init/Kconfig                                      |   1 +
 kernel/bpf/btf.c                                  | 136 ++++-
 kernel/bpf/core.c                                 |   1 +
 kernel/bpf/sockmap.c                              | 644 +++++++++++++++++++---
 kernel/bpf/stackmap.c                             |  59 +-
 kernel/bpf/syscall.c                              |  41 +-
 kernel/bpf/verifier.c                             |  16 +-
 net/core/filter.c                                 | 365 +++++++++++-
 net/ipv6/addrconf_core.c                          |  33 +-
 net/ipv6/af_inet6.c                               |   6 +-
 net/ipv6/fib6_rules.c                             | 138 ++++-
 net/ipv6/ip6_fib.c                                |  21 +-
 net/ipv6/route.c                                  |  76 +--
 net/xdp/xdp_umem.c                                |   2 +-
 samples/bpf/Makefile                              | 166 +++---
 samples/bpf/{libbpf.h => bpf_insn.h}              |   8 +-
 samples/bpf/bpf_load.c                            |  12 +-
 samples/bpf/bpf_load.h                            |   6 +-
 samples/bpf/cookie_uid_helper_example.c           |   2 +-
 samples/bpf/cpustat_user.c                        |   2 +-
 samples/bpf/fds_example.c                         |   4 +-
 samples/bpf/lathist_user.c                        |   2 +-
 samples/bpf/load_sock_ops.c                       |   2 +-
 samples/bpf/lwt_len_hist_user.c                   |   2 +-
 samples/bpf/map_perf_test_user.c                  |   2 +-
 samples/bpf/sock_example.c                        |   3 +-
 samples/bpf/sock_example.h                        |   1 -
 samples/bpf/sockex1_user.c                        |   2 +-
 samples/bpf/sockex2_user.c                        |   2 +-
 samples/bpf/sockex3_user.c                        |   2 +-
 samples/bpf/syscall_tp_user.c                     |   2 +-
 samples/bpf/tc_l2_redirect_user.c                 |   2 +-
 samples/bpf/test_cgrp2_array_pin.c                |   2 +-
 samples/bpf/test_cgrp2_attach.c                   |   3 +-
 samples/bpf/test_cgrp2_attach2.c                  |   3 +-
 samples/bpf/test_cgrp2_sock.c                     |   3 +-
 samples/bpf/test_cgrp2_sock2.c                    |   3 +-
 samples/bpf/test_current_task_under_cgroup_user.c |   2 +-
 samples/bpf/test_lru_dist.c                       |   2 +-
 samples/bpf/test_map_in_map_user.c                |   2 +-
 samples/bpf/test_overhead_user.c                  |   2 +-
 samples/bpf/test_probe_write_user_user.c          |   2 +-
 samples/bpf/trace_output_user.c                   |   8 +-
 samples/bpf/tracex1_user.c                        |   2 +-
 samples/bpf/tracex2_user.c                        |   2 +-
 samples/bpf/tracex3_user.c                        |   2 +-
 samples/bpf/tracex4_user.c                        |   2 +-
 samples/bpf/tracex5_user.c                        |   2 +-
 samples/bpf/tracex6_user.c                        |   2 +-
 samples/bpf/tracex7_user.c                        |   2 +-
 samples/bpf/xdp1_user.c                           |  31 +-
 samples/bpf/xdp_adjust_tail_user.c                |  36 +-
 samples/bpf/xdp_fwd_kern.c                        | 138 +++++
 samples/bpf/xdp_fwd_user.c                        | 136 +++++
 samples/bpf/xdp_monitor_user.c                    |   6 +-
 samples/bpf/xdp_redirect_cpu_user.c               |   2 +-
 samples/bpf/xdp_redirect_map_user.c               |   2 +-
 samples/bpf/xdp_redirect_user.c                   |   2 +-
 samples/bpf/xdp_router_ipv4_user.c                |   2 +-
 samples/bpf/xdp_rxq_info_user.c                   |  46 +-
 samples/bpf/xdp_tx_iptunnel_user.c                |   2 +-
 samples/bpf/xdpsock_user.c                        |   2 +-
 tools/bpf/bpftool/.gitignore                      |   3 +
 tools/bpf/bpftool/map.c                           |   1 +
 tools/bpf/bpftool/map_perf_ring.c                 |  83 +--
 tools/include/uapi/asm/bitsperlong.h              |  18 +
 tools/include/uapi/asm/errno.h                    |  18 +
 tools/include/uapi/linux/bpf.h                    | 143 ++++-
 tools/lib/bpf/Makefile                            |   2 +-
 tools/lib/bpf/bpf.c                               |  12 +
 tools/lib/bpf/bpf.h                               |   3 +
 tools/lib/bpf/libbpf.c                            | 125 ++++-
 tools/lib/bpf/libbpf.h                            |  62 ++-
 tools/testing/selftests/bpf/.gitignore            |   1 +
 tools/testing/selftests/bpf/Makefile              |  12 +-
 tools/testing/selftests/bpf/bpf_helpers.h         |  11 +
 tools/testing/selftests/bpf/bpf_rand.h            |  80 +++
 tools/testing/selftests/bpf/test_btf.c            | 478 ++++++++++++----
 tools/testing/selftests/bpf/test_progs.c          | 140 ++++-
 tools/testing/selftests/bpf/test_sockhash_kern.c  |   5 +
 tools/testing/selftests/bpf/test_sockmap.c        |  27 +-
 tools/testing/selftests/bpf/test_sockmap_kern.c   | 343 +-----------
 tools/testing/selftests/bpf/test_sockmap_kern.h   | 363 ++++++++++++
 tools/testing/selftests/bpf/test_verifier.c       |  62 +++
 tools/testing/selftests/bpf/trace_helpers.c       |  87 +--
 tools/testing/selftests/bpf/trace_helpers.h       |  11 +-
 tools/testing/selftests/bpf/urandom_read.c        |  10 +-
 114 files changed, 4600 insertions(+), 1865 deletions(-)
 create mode 100644 Documentation/bpf/README.rst
 create mode 100644 Documentation/bpf/bpf_design_QA.rst
 delete mode 100644 Documentation/bpf/bpf_design_QA.txt
 create mode 100644 Documentation/bpf/bpf_devel_QA.rst
 delete mode 100644 Documentation/bpf/bpf_devel_QA.txt
 rename samples/bpf/{libbpf.h => bpf_insn.h} (98%)
 create mode 100644 samples/bpf/xdp_fwd_kern.c
 create mode 100644 samples/bpf/xdp_fwd_user.c
 create mode 100644 tools/bpf/bpftool/.gitignore
 create mode 100644 tools/include/uapi/asm/bitsperlong.h
 create mode 100644 tools/include/uapi/asm/errno.h
 create mode 100644 tools/testing/selftests/bpf/bpf_rand.h
 create mode 100644 tools/testing/selftests/bpf/test_sockhash_kern.c
 create mode 100644 tools/testing/selftests/bpf/test_sockmap_kern.h

Comments

David Miller May 17, 2018, 2:47 a.m. | #1
From: Daniel Borkmann <daniel@iogearbox.net>
Date: Thu, 17 May 2018 03:09:48 +0200

> The following pull-request contains BPF updates for your *net-next*
> tree.

Looks good, pulled, thanks Daniel.