mbox series

[v5,bpf-next,0/5] Add support for memory-mapping BPF array maps

Message ID 20191117065035.202462-1-andriin@fb.com
Headers show
Series Add support for memory-mapping BPF array maps | expand

Message

Andrii Nakryiko Nov. 17, 2019, 6:50 a.m. UTC 
This patch set adds ability to memory-map BPF array maps (single- and
multi-element). The primary use case is memory-mapping BPF array maps, created
to back global data variables, created by libbpf implicitly. This allows for
much better usability, along with avoiding syscalls to read or update data
completely.

Due to memory-mapping requirements, BPF array map that is supposed to be
memory-mapped, has to be created with special BPF_F_MMAPABLE attribute, which
triggers slightly different memory allocation strategy internally. See
patch 1 for details.

Libbpf is extended to detect kernel support for this flag, and if supported,
will specify it for all global data maps automatically.

Patch #1 refactors bpf_map_inc() and converts bpf_map's refcnt to atomic64_t
to make refcounting never fail. Patch #2 does similar refactoring for
bpf_prog_add()/bpf_prog_inc().

v4->v5:
- change bpf_prog's refcnt to atomic64_t (Daniel);

v3->v4:
- add mmap's open() callback to fix refcounting (Johannes);
- switch to remap_vmalloc_pages() instead of custom fault handler (Johannes);
- converted bpf_map's refcnt/usercnt into atomic64_t;
- provide default bpf_map_default_vmops handling open/close properly;

v2->v3:
- change allocation strategy to avoid extra pointer dereference (Jakub);

v1->v2:
- fix map lookup code generation for BPF_F_MMAPABLE case;
- prevent BPF_F_MMAPABLE flag for all but plain array map type;
- centralize ref-counting in generic bpf_map_mmap();
- don't use uref counting (Alexei);
- use vfree() directly;
- print flags with %x (Song);
- extend tests to verify bpf_map_{lookup,update}_elem() logic as well.

Andrii Nakryiko (5):
  bpf: switch bpf_map ref counter to atomic64_t so bpf_map_inc() never
    fails
  bpf: convert bpf_prog refcnt to atomic64_t
  bpf: add mmap() support for BPF_MAP_TYPE_ARRAY
  libbpf: make global data internal arrays mmap()-able, if possible
  selftests/bpf: add BPF_TYPE_MAP_ARRAY mmap() tests

 drivers/net/ethernet/broadcom/bnxt/bnxt.c     |   9 +-
 .../net/ethernet/cavium/thunder/nicvf_main.c  |   9 +-
 .../net/ethernet/freescale/dpaa2/dpaa2-eth.c  |   7 +-
 .../net/ethernet/mellanox/mlx4/en_netdev.c    |  24 +-
 .../net/ethernet/mellanox/mlx5/core/en_main.c |  18 +-
 .../net/ethernet/netronome/nfp/bpf/offload.c  |   4 +-
 drivers/net/ethernet/qlogic/qede/qede_main.c  |   8 +-
 drivers/net/virtio_net.c                      |   7 +-
 include/linux/bpf.h                           |  34 +--
 include/linux/vmalloc.h                       |   1 +
 include/uapi/linux/bpf.h                      |   3 +
 kernel/bpf/arraymap.c                         |  58 ++++-
 kernel/bpf/inode.c                            |   7 +-
 kernel/bpf/map_in_map.c                       |   2 +-
 kernel/bpf/syscall.c                          | 174 ++++++++++----
 kernel/bpf/verifier.c                         |   6 +-
 kernel/bpf/xskmap.c                           |   6 +-
 kernel/events/core.c                          |   7 +-
 mm/vmalloc.c                                  |  20 ++
 net/core/bpf_sk_storage.c                     |   2 +-
 tools/include/uapi/linux/bpf.h                |   3 +
 tools/lib/bpf/libbpf.c                        |  32 ++-
 .../selftests/bpf/prog_tests/core_reloc.c     |  45 ++--
 tools/testing/selftests/bpf/prog_tests/mmap.c | 220 ++++++++++++++++++
 tools/testing/selftests/bpf/progs/test_mmap.c |  45 ++++
 25 files changed, 576 insertions(+), 175 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/mmap.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_mmap.c