[v2,bpf-next,0/5] Bitfield and size relocations support in libbpf

This patch set adds support for reading bitfields in a relocatable manner
through a set of relocations emitted by Clang, corresponding libbpf support
for those relocations, as well as abstracting details into
BPF_CORE_READ_BITFIELD/BPF_CORE_READ_BITFIELD_PROBED macro.

We also add support for capturing relocatable field size, so that BPF program
code can adjust its logic to actual amount of data it needs to operate on,
even if it changes between kernels. New convenience macro is added to
bpf_core_read.h (bpf_core_field_size(), in the same family of macro as
bpf_core_read() and bpf_core_field_exists()). Corresponding set of selftests
are added to excercise this logic and validate correctness in a variety of
scenarios.

Some of the overly strict logic of matching fields is relaxed to support wider
variety of scenarios. See patch #1 for that.

Patch #1 removes few overly strict test cases.
Patch #2 adds support for bitfield-related relocations.
Patch #3 adds some further adjustments to support generic field size
relocations and introduces bpf_core_field_size() macro.
Patch #4 tests bitfield reading.
Patch #5 tests field size relocations.

v1->v2:
- added direct memory read-based macro and tests for bitfield reads.

Andrii Nakryiko (5):
  selftests/bpf: remove too strict field offset relo test cases
  libbpf: add support for relocatable bitfields
  libbpf: add support for field size relocations
  selftest/bpf: add relocatable bitfield reading tests
  selftests/bpf: add field size relocation tests

 tools/lib/bpf/bpf_core_read.h                 |  79 ++++++
 tools/lib/bpf/libbpf.c                        | 243 +++++++++++++-----
 tools/lib/bpf/libbpf_internal.h               |   4 +
 .../selftests/bpf/prog_tests/core_reloc.c     | 123 ++++++++-
 ...__core_reloc_arrays___err_wrong_val_type.c |   3 +
 ..._core_reloc_arrays___err_wrong_val_type1.c |   3 -
 ..._core_reloc_arrays___err_wrong_val_type2.c |   3 -
 .../bpf/progs/btf__core_reloc_bitfields.c     |   3 +
 ...tf__core_reloc_bitfields___bit_sz_change.c |   3 +
 ...__core_reloc_bitfields___bitfield_vs_int.c |   3 +
 ...e_reloc_bitfields___err_too_big_bitfield.c |   3 +
 ...__core_reloc_bitfields___just_big_enough.c |   3 +
 .../btf__core_reloc_ints___err_bitfield.c     |   3 -
 .../btf__core_reloc_ints___err_wrong_sz_16.c  |   3 -
 .../btf__core_reloc_ints___err_wrong_sz_32.c  |   3 -
 .../btf__core_reloc_ints___err_wrong_sz_64.c  |   3 -
 .../btf__core_reloc_ints___err_wrong_sz_8.c   |   3 -
 .../bpf/progs/btf__core_reloc_size.c          |   3 +
 .../progs/btf__core_reloc_size___diff_sz.c    |   3 +
 .../selftests/bpf/progs/core_reloc_types.h    | 173 ++++++++-----
 .../progs/test_core_reloc_bitfields_direct.c  |  63 +++++
 .../progs/test_core_reloc_bitfields_probed.c  |  62 +++++
 .../bpf/progs/test_core_reloc_size.c          |  51 ++++
 23 files changed, 682 insertions(+), 161 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_arrays___err_wrong_val_type.c
 delete mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_arrays___err_wrong_val_type1.c
 delete mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_arrays___err_wrong_val_type2.c
 create mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_bitfields.c
 create mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_bitfields___bit_sz_change.c
 create mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_bitfields___bitfield_vs_int.c
 create mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_bitfields___err_too_big_bitfield.c
 create mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_bitfields___just_big_enough.c
 delete mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_ints___err_bitfield.c
 delete mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_ints___err_wrong_sz_16.c
 delete mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_ints___err_wrong_sz_32.c
 delete mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_ints___err_wrong_sz_64.c
 delete mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_ints___err_wrong_sz_8.c
 create mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_size.c
 create mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_size___diff_sz.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_core_reloc_bitfields_direct.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_core_reloc_bitfields_probed.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_core_reloc_size.c

On 11/1/19 11:28 PM, Andrii Nakryiko wrote:
> This patch set adds support for reading bitfields in a relocatable manner
> through a set of relocations emitted by Clang, corresponding libbpf support
> for those relocations, as well as abstracting details into
> BPF_CORE_READ_BITFIELD/BPF_CORE_READ_BITFIELD_PROBED macro.
> 
> We also add support for capturing relocatable field size, so that BPF program
> code can adjust its logic to actual amount of data it needs to operate on,
> even if it changes between kernels. New convenience macro is added to
> bpf_core_read.h (bpf_core_field_size(), in the same family of macro as
> bpf_core_read() and bpf_core_field_exists()). Corresponding set of selftests
> are added to excercise this logic and validate correctness in a variety of
> scenarios.
> 
> Some of the overly strict logic of matching fields is relaxed to support wider
> variety of scenarios. See patch #1 for that.
> 
> Patch #1 removes few overly strict test cases.
> Patch #2 adds support for bitfield-related relocations.
> Patch #3 adds some further adjustments to support generic field size
> relocations and introduces bpf_core_field_size() macro.
> Patch #4 tests bitfield reading.
> Patch #5 tests field size relocations.
> 
> v1->v2:
> - added direct memory read-based macro and tests for bitfield reads.

Applied, thanks!