mbox series

[bpf-next,v3,00/10] BTF: BPF Type Format

Message ID 20180416193327.477239-1-kafai@fb.com
Headers show
Series BTF: BPF Type Format | expand

Message

Martin KaFai Lau April 16, 2018, 7:33 p.m. UTC 
This patch introduces BPF Type Format (BTF).

BTF (BPF Type Format) is the meta data format which describes
the data types of BPF program/map.  Hence, it basically focus
on the C programming language which the modern BPF is primary
using.  The first use case is to provide a generic pretty print
capability for a BPF map.

A modified pahole (Cc: Arnaldo) that can convert dwarf to BTF is here:
https://github.com/iamkafai/pahole/tree/btf

Please see individual patch for details.

v3:
- Rebase to bpf-next
- Fix sparse warning (by adding static)
- Add BTF header logging: btf_verifier_log_hdr()
- Fix the alignment test on btf->type_off
- Add tests for the BTF header
- Lower the max BTF size to 16MB.  It should be enough
  for some time.  We could raise it later if it would
  be needed.

v2:
- Use kvfree where needed in patch 1 and 2
- Also consider BTF_INT_OFFSET() in the btf_int_check_meta()
  in patch 1
- Fix an incorrect goto target in map_create() during
  the btf-error-path in patch 7
- re-org some local vars to keep the rev xmas tree in btf.c

Martin KaFai Lau (10):
  bpf: btf: Introduce BPF Type Format (BTF)
  bpf: btf: Validate type reference
  bpf: btf: Check members of struct/union
  bpf: btf: Add pretty print capability for data with BTF type info
  bpf: btf: Add BPF_BTF_LOAD command
  bpf: btf: Add BPF_OBJ_GET_INFO_BY_FD support to BTF fd
  bpf: btf: Add pretty print support to the basic arraymap
  bpf: btf: Sync bpf.h and btf.h to tools/
  bpf: btf: Add BTF support to libbpf
  bpf: btf: Add BTF tests

 include/linux/bpf.h                          |   20 +-
 include/linux/btf.h                          |   48 +
 include/uapi/linux/bpf.h                     |   12 +
 include/uapi/linux/btf.h                     |  132 ++
 kernel/bpf/Makefile                          |    1 +
 kernel/bpf/arraymap.c                        |   50 +
 kernel/bpf/btf.c                             | 2093 ++++++++++++++++++++++++++
 kernel/bpf/inode.c                           |  146 +-
 kernel/bpf/syscall.c                         |   51 +-
 tools/include/uapi/linux/bpf.h               |   13 +
 tools/include/uapi/linux/btf.h               |  132 ++
 tools/lib/bpf/Build                          |    2 +-
 tools/lib/bpf/bpf.c                          |   92 +-
 tools/lib/bpf/bpf.h                          |   16 +
 tools/lib/bpf/btf.c                          |  377 +++++
 tools/lib/bpf/btf.h                          |   22 +
 tools/lib/bpf/libbpf.c                       |  148 +-
 tools/lib/bpf/libbpf.h                       |    3 +
 tools/testing/selftests/bpf/Makefile         |   26 +-
 tools/testing/selftests/bpf/test_btf.c       | 1669 ++++++++++++++++++++
 tools/testing/selftests/bpf/test_btf_haskv.c |   48 +
 tools/testing/selftests/bpf/test_btf_nokv.c  |   43 +
 22 files changed, 5103 insertions(+), 41 deletions(-)
 create mode 100644 include/linux/btf.h
 create mode 100644 include/uapi/linux/btf.h
 create mode 100644 kernel/bpf/btf.c
 create mode 100644 tools/include/uapi/linux/btf.h
 create mode 100644 tools/lib/bpf/btf.c
 create mode 100644 tools/lib/bpf/btf.h
 create mode 100644 tools/testing/selftests/bpf/test_btf.c
 create mode 100644 tools/testing/selftests/bpf/test_btf_haskv.c
 create mode 100644 tools/testing/selftests/bpf/test_btf_nokv.c

Comments

Arnaldo Carvalho de Melo April 16, 2018, 8:22 p.m. UTC | #1 
Em Mon, Apr 16, 2018 at 12:33:17PM -0700, Martin KaFai Lau escreveu:
> This patch introduces BPF Type Format (BTF).
> 
> BTF (BPF Type Format) is the meta data format which describes
> the data types of BPF program/map.  Hence, it basically focus
> on the C programming language which the modern BPF is primary
> using.  The first use case is to provide a generic pretty print
> capability for a BPF map.
> 
> A modified pahole (Cc: Arnaldo) that can convert dwarf to BTF is here:
> https://github.com/iamkafai/pahole/tree/btf

Thanks for CCing me, no changes since when you posted the pahole
patches, I gave it a quick look, seems sane, will try to merge and push
a new pahole version out so that distros can pick it, at least fedora
will 8-)

- Arnaldo
 
> Please see individual patch for details.
> 
> v3:
> - Rebase to bpf-next
> - Fix sparse warning (by adding static)
> - Add BTF header logging: btf_verifier_log_hdr()
> - Fix the alignment test on btf->type_off
> - Add tests for the BTF header
> - Lower the max BTF size to 16MB.  It should be enough
>   for some time.  We could raise it later if it would
>   be needed.
> 
> v2:
> - Use kvfree where needed in patch 1 and 2
> - Also consider BTF_INT_OFFSET() in the btf_int_check_meta()
>   in patch 1
> - Fix an incorrect goto target in map_create() during
>   the btf-error-path in patch 7
> - re-org some local vars to keep the rev xmas tree in btf.c
> 
> Martin KaFai Lau (10):
>   bpf: btf: Introduce BPF Type Format (BTF)
>   bpf: btf: Validate type reference
>   bpf: btf: Check members of struct/union
>   bpf: btf: Add pretty print capability for data with BTF type info
>   bpf: btf: Add BPF_BTF_LOAD command
>   bpf: btf: Add BPF_OBJ_GET_INFO_BY_FD support to BTF fd
>   bpf: btf: Add pretty print support to the basic arraymap
>   bpf: btf: Sync bpf.h and btf.h to tools/
>   bpf: btf: Add BTF support to libbpf
>   bpf: btf: Add BTF tests
> 
>  include/linux/bpf.h                          |   20 +-
>  include/linux/btf.h                          |   48 +
>  include/uapi/linux/bpf.h                     |   12 +
>  include/uapi/linux/btf.h                     |  132 ++
>  kernel/bpf/Makefile                          |    1 +
>  kernel/bpf/arraymap.c                        |   50 +
>  kernel/bpf/btf.c                             | 2093 ++++++++++++++++++++++++++
>  kernel/bpf/inode.c                           |  146 +-
>  kernel/bpf/syscall.c                         |   51 +-
>  tools/include/uapi/linux/bpf.h               |   13 +
>  tools/include/uapi/linux/btf.h               |  132 ++
>  tools/lib/bpf/Build                          |    2 +-
>  tools/lib/bpf/bpf.c                          |   92 +-
>  tools/lib/bpf/bpf.h                          |   16 +
>  tools/lib/bpf/btf.c                          |  377 +++++
>  tools/lib/bpf/btf.h                          |   22 +
>  tools/lib/bpf/libbpf.c                       |  148 +-
>  tools/lib/bpf/libbpf.h                       |    3 +
>  tools/testing/selftests/bpf/Makefile         |   26 +-
>  tools/testing/selftests/bpf/test_btf.c       | 1669 ++++++++++++++++++++
>  tools/testing/selftests/bpf/test_btf_haskv.c |   48 +
>  tools/testing/selftests/bpf/test_btf_nokv.c  |   43 +
>  22 files changed, 5103 insertions(+), 41 deletions(-)
>  create mode 100644 include/linux/btf.h
>  create mode 100644 include/uapi/linux/btf.h
>  create mode 100644 kernel/bpf/btf.c
>  create mode 100644 tools/include/uapi/linux/btf.h
>  create mode 100644 tools/lib/bpf/btf.c
>  create mode 100644 tools/lib/bpf/btf.h
>  create mode 100644 tools/testing/selftests/bpf/test_btf.c
>  create mode 100644 tools/testing/selftests/bpf/test_btf_haskv.c
>  create mode 100644 tools/testing/selftests/bpf/test_btf_nokv.c
> 
> -- 
> 2.9.5
Martin KaFai Lau April 17, 2018, 6:19 p.m. UTC | #2 
On Mon, Apr 16, 2018 at 05:22:00PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Mon, Apr 16, 2018 at 12:33:17PM -0700, Martin KaFai Lau escreveu:
> > This patch introduces BPF Type Format (BTF).
> > 
> > BTF (BPF Type Format) is the meta data format which describes
> > the data types of BPF program/map.  Hence, it basically focus
> > on the C programming language which the modern BPF is primary
> > using.  The first use case is to provide a generic pretty print
> > capability for a BPF map.
> > 
> > A modified pahole (Cc: Arnaldo) that can convert dwarf to BTF is here:
> > https://github.com/iamkafai/pahole/tree/btf
> 
> Thanks for CCing me, no changes since when you posted the pahole
> patches, I gave it a quick look, seems sane, will try to merge and push
> a new pahole version out so that distros can pick it, at least fedora
> will 8-)
Thanks!  Right, the last commit should be on April 3rd.

> 
> - Arnaldo
>  
> > Please see individual patch for details.
> > 
> > v3:
> > - Rebase to bpf-next
> > - Fix sparse warning (by adding static)
> > - Add BTF header logging: btf_verifier_log_hdr()
> > - Fix the alignment test on btf->type_off
> > - Add tests for the BTF header
> > - Lower the max BTF size to 16MB.  It should be enough
> >   for some time.  We could raise it later if it would
> >   be needed.
> > 
> > v2:
> > - Use kvfree where needed in patch 1 and 2
> > - Also consider BTF_INT_OFFSET() in the btf_int_check_meta()
> >   in patch 1
> > - Fix an incorrect goto target in map_create() during
> >   the btf-error-path in patch 7
> > - re-org some local vars to keep the rev xmas tree in btf.c
> > 
> > Martin KaFai Lau (10):
> >   bpf: btf: Introduce BPF Type Format (BTF)
> >   bpf: btf: Validate type reference
> >   bpf: btf: Check members of struct/union
> >   bpf: btf: Add pretty print capability for data with BTF type info
> >   bpf: btf: Add BPF_BTF_LOAD command
> >   bpf: btf: Add BPF_OBJ_GET_INFO_BY_FD support to BTF fd
> >   bpf: btf: Add pretty print support to the basic arraymap
> >   bpf: btf: Sync bpf.h and btf.h to tools/
> >   bpf: btf: Add BTF support to libbpf
> >   bpf: btf: Add BTF tests
> > 
> >  include/linux/bpf.h                          |   20 +-
> >  include/linux/btf.h                          |   48 +
> >  include/uapi/linux/bpf.h                     |   12 +
> >  include/uapi/linux/btf.h                     |  132 ++
> >  kernel/bpf/Makefile                          |    1 +
> >  kernel/bpf/arraymap.c                        |   50 +
> >  kernel/bpf/btf.c                             | 2093 ++++++++++++++++++++++++++
> >  kernel/bpf/inode.c                           |  146 +-
> >  kernel/bpf/syscall.c                         |   51 +-
> >  tools/include/uapi/linux/bpf.h               |   13 +
> >  tools/include/uapi/linux/btf.h               |  132 ++
> >  tools/lib/bpf/Build                          |    2 +-
> >  tools/lib/bpf/bpf.c                          |   92 +-
> >  tools/lib/bpf/bpf.h                          |   16 +
> >  tools/lib/bpf/btf.c                          |  377 +++++
> >  tools/lib/bpf/btf.h                          |   22 +
> >  tools/lib/bpf/libbpf.c                       |  148 +-
> >  tools/lib/bpf/libbpf.h                       |    3 +
> >  tools/testing/selftests/bpf/Makefile         |   26 +-
> >  tools/testing/selftests/bpf/test_btf.c       | 1669 ++++++++++++++++++++
> >  tools/testing/selftests/bpf/test_btf_haskv.c |   48 +
> >  tools/testing/selftests/bpf/test_btf_nokv.c  |   43 +
> >  22 files changed, 5103 insertions(+), 41 deletions(-)
> >  create mode 100644 include/linux/btf.h
> >  create mode 100644 include/uapi/linux/btf.h
> >  create mode 100644 kernel/bpf/btf.c
> >  create mode 100644 tools/include/uapi/linux/btf.h
> >  create mode 100644 tools/lib/bpf/btf.c
> >  create mode 100644 tools/lib/bpf/btf.h
> >  create mode 100644 tools/testing/selftests/bpf/test_btf.c
> >  create mode 100644 tools/testing/selftests/bpf/test_btf_haskv.c
> >  create mode 100644 tools/testing/selftests/bpf/test_btf_nokv.c
> > 
> > -- 
> > 2.9.5