| Message ID | 20181017072315.2766920-1-yhs@fb.com |
|---|---|
| Headers | show |
| Series | bpf: add btf func info support | expand |
I think the BTF work needs to be better documented; at the moment the only way to determine how BTF sections are structured is to read through the headers, and cross-reference with the DWARF spec to guess at the semantics of various fields. I've been working on adding BTF support to ebpf_asm, and finding very frustrating the amount of guesswork required. Therefore please make sure that each patch extending the BTF format includes documentation patches describing both the layout and the semantics of the new extensions. For example in patch #9 there is no explanation of btf_ext_header.line_info_off and btf_ext_header.line_info_len (they're not even used by the code, so one cannot reverse-engineer it); while it's fairly clear that they indicate the bounds of the line_info subsection, there is no specification of what this subsection contains. -Ed
On 10/17/18 4:02 AM, Edward Cree wrote: > I think the BTF work needs to be better documented; at the moment the only way > to determine how BTF sections are structured is to read through the headers, > and cross-reference with the DWARF spec to guess at the semantics of various > fields. I've been working on adding BTF support to ebpf_asm, and finding > very frustrating the amount of guesswork required. > Therefore please make sure that each patch extending the BTF format includes > documentation patches describing both the layout and the semantics of the new Make sense. I will add some comments to describe the layout in patch #9. > extensions. For example in patch #9 there is no explanation of > btf_ext_header.line_info_off and btf_ext_header.line_info_len (they're not > even used by the code, so one cannot reverse-engineer it); while it's fairly > clear that they indicate the bounds of the line_info subsection, there is no The line_info field is added because it is implemented in llvm. I imported it to kernel tools directory to be compatible with what llvm generates although we did not process it yet. I will add a comment on this. In the long term, I guess we should add description of format etc. in Documentation/bpf directory like BTF.rst. > specification of what this subsection contains. > > -Ed >
The BTF support was added to kernel by Commit 69b693f0aefa ("bpf: btf: Introduce BPF Type Format (BTF)"), which introduced .BTF section into ELF file and is primarily used for map pretty print. pahole is used to convert dwarf to BTF for ELF files. This patch added func info support to the kernel so we can get better ksym's for bpf function calls. Basically, pairs of bpf function calls and their corresponding types are passed to the kernel. Extracting function names from the types, the kernel is able to construct a ksym for each function call with embedded function name. LLVM patch https://reviews.llvm.org/D53261 can generate func info, encoded in .BTF.ext ELF section. In addition, it also added support for FUNC and FUNC_PROTO types, which are also supported through this patch set. The following is an example to show FUNC and FUNC_PROTO difference, compiled with the above LLVM patch with Debug mode. -bash-4.2$ cat test.c int foo(int (*bar)(int)) { return bar(5); } -bash-4.2$ clang -target bpf -g -O2 -mllvm -debug-only=btf -c test.c Type Table: [1] FUNC name_off=1 info=0x0c000001 size/type=2 param_type=3 [2] INT name_off=11 info=0x01000000 size/type=4 desc=0x01000020 [3] PTR name_off=0 info=0x02000000 size/type=4 [4] FUNC_PROTO name_off=0 info=0x0d000001 size/type=2 param_type=2 String Table: 0 : 1 : foo 5 : .text 11 : int 15 : test.c 22 : int foo(int (*bar)(int)) { return bar(5); } FuncInfo Table: sec_name_off=5 insn_offset=<Omitted> type_id=1 ... In the above, type and string tables are in .BTF section and the func info in .BTF.ext. The "<Omitted>" is the insn offset which is not available during the dump time but resolved during later compilation process. Following the format specification at Patch #9 and examine the raw data in .BTF.ext section, we have FuncInfo Table: sec_name_off=5 insn_offset=0 type_id=1 The (insn_offset, type_id) can be passed to the kernel so the kernel can find the func name and use it in the ksym. Below is a demonstration from Patch #13. $ bpftool prog dump jited id 1 int _dummy_tracepoint(struct dummy_tracepoint_args * ): bpf_prog_b07ccb89267cf242__dummy_tracepoint: 0: push %rbp 1: mov %rsp,%rbp ...... 3c: add $0x28,%rbp 40: leaveq 41: retq int test_long_fname_1(struct dummy_tracepoint_args * ): bpf_prog_2dcecc18072623fc_test_long_fname_1: 0: push %rbp 1: mov %rsp,%rbp ...... 3a: add $0x28,%rbp 3e: leaveq 3f: retq int test_long_fname_2(struct dummy_tracepoint_args * ): bpf_prog_89d64e4abf0f0126_test_long_fname_2: 0: push %rbp 1: mov %rsp,%rbp ...... 80: add $0x28,%rbp 84: leaveq 85: retq For the patchset, Patch #1 refactors the code to break up btf_type_is_void(). Patch #2 introduces new BTF types BTF_KIND_FUNC and BTF_KIND_FUNC_PROTO. Patch #3 syncs btf.h header to tools directory. Patch #4 adds btf func/func_proto self tests in test_btf. Patch #5 adds kernel interface to load func_info to kernel and pass func_info to userspace. Patch #6 syncs bpf.h header to tools directory. Patch #7 adds news btf/func_info related fields in libbpf program load function. Patch #8 extends selftest test_btf to test load/retrieve func_type info. Patch #9 adds .BTF.ext func info support. Patch #10 changes Makefile to avoid using pahole if llvm is capable of generating BTF sections. Patch #11 refactors to have btf_get_from_id() in libbpf for reuse. Patch #12 enhance test_btf file testing to test func info. Patch #13 adds bpftool support for func signature dump. Yonghong Song (13): bpf: btf: Break up btf_type_is_void() bpf: btf: Add BTF_KIND_FUNC and BTF_KIND_FUNC_PROTO tools/bpf: sync kernel btf.h header tools/bpf: add btf func/func_proto unit tests in selftest test_btf bpf: get better bpf_prog ksyms based on btf func type_id tools/bpf: sync kernel uapi bpf.h header to tools directory tools/bpf: add new fields for program load in lib/bpf tools/bpf: extends test_btf to test load/retrieve func_type info tools/bpf: add support to read .BTF.ext sections tools/bpf: do not use pahole if clang/llvm can generate BTF sections tools/bpf: refactor to implement btf_get_from_id() in lib/bpf tools/bpf: enhance test_btf file testing to test func info tools/bpf: bpftool: add support for jited func types Changelogs: v1 -> v2: . Added missing sign-off . Limited the func_name/struct_member_name length for validity test . Removed/changed several verifier messages . Modified several commit messages to remove line_off reference include/linux/bpf.h | 2 + include/linux/bpf_verifier.h | 1 + include/linux/btf.h | 2 + include/uapi/linux/bpf.h | 11 + include/uapi/linux/btf.h | 9 +- kernel/bpf/btf.c | 325 +++++++++-- kernel/bpf/core.c | 10 + kernel/bpf/syscall.c | 83 ++- kernel/bpf/verifier.c | 46 ++ samples/bpf/Makefile | 8 + tools/bpf/bpftool/btf_dumper.c | 96 ++++ tools/bpf/bpftool/main.h | 2 + tools/bpf/bpftool/map.c | 68 +-- tools/bpf/bpftool/prog.c | 54 ++ tools/include/uapi/linux/bpf.h | 11 + tools/include/uapi/linux/btf.h | 9 +- tools/lib/bpf/bpf.c | 3 + tools/lib/bpf/bpf.h | 3 + tools/lib/bpf/btf.c | 305 ++++++++++ tools/lib/bpf/btf.h | 33 ++ tools/lib/bpf/libbpf.c | 53 +- tools/testing/selftests/bpf/Makefile | 8 + tools/testing/selftests/bpf/test_btf.c | 566 ++++++++++++++++++- tools/testing/selftests/bpf/test_btf_haskv.c | 16 +- tools/testing/selftests/bpf/test_btf_nokv.c | 16 +- 25 files changed, 1611 insertions(+), 129 deletions(-)