| Message ID | 20200527015704.2294223-1-dxu@dxuuu.xyz |
|---|---|
| State | Changes Requested |
| Delegated to: | BPF Maintainers |
| Headers | show |
| Series | [bpf-next] libbpf: Export bpf_object__load_vmlinux_btf | expand |
On Tue, May 26, 2020 at 7:09 PM Daniel Xu <dxu@dxuuu.xyz> wrote: > > Right now the libbpf model encourages loading the entire object at once. > In this model, libbpf handles loading BTF from vmlinux for us. However, > it can be useful to selectively load certain maps and programs inside an > object without loading everything else. There is no way to selectively load or not load a map. All maps are created, unless they are reusing map FD or pinned instances. See below, I'd like to understand the use case better. > > In the latter model, there was perviously no way to load BTF on-demand. > This commit exports the bpf_object__load_vmlinux_btf such that we are > able to load BTF on demand. > Let's start with the real problem, not a solution. Do you have specific use case where you need bpf_object__load_vmlinux_btf()? It might not do anything if none of BPF programs in the object requires BTF, because it's very much tightly coupled with loading bpf_object as a whole model. I'd like to understand what you are after with this, before exposing internal implementation details as an API. > Signed-off-by: Daniel Xu <dxu@dxuuu.xyz> > --- > tools/lib/bpf/libbpf.c | 2 +- > tools/lib/bpf/libbpf.h | 1 + > tools/lib/bpf/libbpf.map | 1 + > 3 files changed, 3 insertions(+), 1 deletion(-) > [...]
Hi Andrii, On Tue May 26, 2020 at 3:09 PM PST, Andrii Nakryiko wrote: > On Tue, May 26, 2020 at 7:09 PM Daniel Xu <dxu@dxuuu.xyz> wrote: > > > > Right now the libbpf model encourages loading the entire object at once. > > In this model, libbpf handles loading BTF from vmlinux for us. However, > > it can be useful to selectively load certain maps and programs inside an > > object without loading everything else. > > There is no way to selectively load or not load a map. All maps are > created, unless they are reusing map FD or pinned instances. See > below, I'd like to understand the use case better. > > > > > In the latter model, there was perviously no way to load BTF on-demand. > > This commit exports the bpf_object__load_vmlinux_btf such that we are > > able to load BTF on demand. > > > > Let's start with the real problem, not a solution. Do you have > specific use case where you need bpf_object__load_vmlinux_btf()? It > might not do anything if none of BPF programs in the object requires > BTF, because it's very much tightly coupled with loading bpf_object as > a whole model. I'd like to understand what you are after with this, > before exposing internal implementation details as an API. If I try loading a program through the following sequence: bpf_object__open_file() bpf_object__find_program_by_name() bpf_program__load() And the program require BTF (tp_btf), I get an unavoidable (to the best of my knowledge) segfault in the following code path: bpf_program__load() libbpf_find_attach_btf_id() <-- [0] __find_vmlinx_btf_id() find_btf_by_prefix_kind() btf__find_by_name_kind() <-- boom (btf->nr_types) because [0] passes prog->obj->btf_vmlinux which is still null. So the solution I'm proposing is exporting bpf_object__load_vmlinux_btf() and calling that on struct bpf_object before performing prog loads. [...] Thanks, Daniel
On Wed, May 27, 2020 at 10:12 AM Daniel Xu <dxu@dxuuu.xyz> wrote: > > Hi Andrii, > > On Tue May 26, 2020 at 3:09 PM PST, Andrii Nakryiko wrote: > > On Tue, May 26, 2020 at 7:09 PM Daniel Xu <dxu@dxuuu.xyz> wrote: > > > > > > Right now the libbpf model encourages loading the entire object at once. > > > In this model, libbpf handles loading BTF from vmlinux for us. However, > > > it can be useful to selectively load certain maps and programs inside an > > > object without loading everything else. > > > > There is no way to selectively load or not load a map. All maps are > > created, unless they are reusing map FD or pinned instances. See > > below, I'd like to understand the use case better. > > > > > > > > In the latter model, there was perviously no way to load BTF on-demand. > > > This commit exports the bpf_object__load_vmlinux_btf such that we are > > > able to load BTF on demand. > > > > > > > Let's start with the real problem, not a solution. Do you have > > specific use case where you need bpf_object__load_vmlinux_btf()? It > > might not do anything if none of BPF programs in the object requires > > BTF, because it's very much tightly coupled with loading bpf_object as > > a whole model. I'd like to understand what you are after with this, > > before exposing internal implementation details as an API. > > If I try loading a program through the following sequence: > > bpf_object__open_file() > bpf_object__find_program_by_name() > bpf_program__load() > bpf_program__load() is just broken and shouldn't have been ever exposed. It **might** work for trivial BPF programs not using maps, Kconfig and global variables, etc, but more by accident. I think the right fix for your use-case is to allow more control of which programs are auto-loaded. There was a patch by Eric Sage previously adding bpf_program__set_autoload(), but it never landed. We should actually do that approach instead. > And the program require BTF (tp_btf), I get an unavoidable (to the best > of my knowledge) segfault in the following code path: > > bpf_program__load() > libbpf_find_attach_btf_id() <-- [0] > __find_vmlinx_btf_id() > find_btf_by_prefix_kind() > btf__find_by_name_kind() <-- boom (btf->nr_types) > > because [0] passes prog->obj->btf_vmlinux which is still null. So the > solution I'm proposing is exporting bpf_object__load_vmlinux_btf() and > calling that on struct bpf_object before performing prog loads. > > [...] > > Thanks, > Daniel
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 5d60de6fd818..399094b1f580 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -2477,7 +2477,7 @@ static inline bool libbpf_prog_needs_vmlinux_btf(struct bpf_program *prog) return false; } -static int bpf_object__load_vmlinux_btf(struct bpf_object *obj) +int bpf_object__load_vmlinux_btf(struct bpf_object *obj) { struct bpf_program *prog; int err; diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index 1e2e399a5f2c..6cbd678eb124 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -147,6 +147,7 @@ LIBBPF_API unsigned int bpf_object__kversion(const struct bpf_object *obj); struct btf; LIBBPF_API struct btf *bpf_object__btf(const struct bpf_object *obj); LIBBPF_API int bpf_object__btf_fd(const struct bpf_object *obj); +LIBBPF_API int bpf_object__load_vmlinux_btf(struct bpf_object *obj); LIBBPF_API struct bpf_program * bpf_object__find_program_by_title(const struct bpf_object *obj, diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index 381a7342ecfc..56415e671c70 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -261,6 +261,7 @@ LIBBPF_0.0.9 { bpf_iter_create; bpf_link_get_fd_by_id; bpf_link_get_next_id; + bpf_object__load_vmlinux_btf; bpf_program__attach_iter; perf_buffer__consume; } LIBBPF_0.0.8;
Right now the libbpf model encourages loading the entire object at once. In this model, libbpf handles loading BTF from vmlinux for us. However, it can be useful to selectively load certain maps and programs inside an object without loading everything else. In the latter model, there was perviously no way to load BTF on-demand. This commit exports the bpf_object__load_vmlinux_btf such that we are able to load BTF on demand. Signed-off-by: Daniel Xu <dxu@dxuuu.xyz> --- tools/lib/bpf/libbpf.c | 2 +- tools/lib/bpf/libbpf.h | 1 + tools/lib/bpf/libbpf.map | 1 + 3 files changed, 3 insertions(+), 1 deletion(-)