diff mbox series

[02/11] bpf: Compile btfid tool at kernel compilation start

Message ID 20200616100512.2168860-3-jolsa@kernel.org
State Changes Requested
Delegated to: BPF Maintainers
Headers show
Series bpf: Add d_path helper | expand

Commit Message

Jiri Olsa June 16, 2020, 10:05 a.m. UTC
The btfid tool will be used during the vmlinux linking,
so it's necessary it's ready for it.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 Makefile           | 22 ++++++++++++++++++----
 tools/Makefile     |  3 +++
 tools/bpf/Makefile |  5 ++++-
 3 files changed, 25 insertions(+), 5 deletions(-)

Comments

John Fastabend June 18, 2020, 8:40 p.m. UTC | #1
Jiri Olsa wrote:
> The btfid tool will be used during the vmlinux linking,
> so it's necessary it's ready for it.
> 
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  Makefile           | 22 ++++++++++++++++++----
>  tools/Makefile     |  3 +++
>  tools/bpf/Makefile |  5 ++++-
>  3 files changed, 25 insertions(+), 5 deletions(-)

This breaks the build for me. I fixed it with this but then I get warnings,

diff --git a/tools/bpf/btfid/btfid.c b/tools/bpf/btfid/btfid.c
index 7cdf39bfb150..3697e8ae9efa 100644
--- a/tools/bpf/btfid/btfid.c
+++ b/tools/bpf/btfid/btfid.c
@@ -48,7 +48,7 @@
 #include <errno.h>
 #include <linux/rbtree.h>
 #include <linux/zalloc.h>
-#include <btf.h>
+#include <linux/btf.h>
 #include <libbpf.h>
 #include <parse-options.h>

Here is the error. Is it something about my setup? bpftool/btf.c uses
<btf.h>. Because this in top-level Makefile we probably don't want to
push extra setup onto folks.

In file included from btfid.c:51:
/home/john/git/bpf-next/tools/lib/bpf/btf.h: In function ‘btf_is_var’:
/home/john/git/bpf-next/tools/lib/bpf/btf.h:254:24: error: ‘BTF_KIND_VAR’ undeclared (first use in this function); did you mean ‘BTF_KIND_PTR’?
  return btf_kind(t) == BTF_KIND_VAR;
                        ^~~~~~~~~~~~
                        BTF_KIND_PTR
/home/john/git/bpf-next/tools/lib/bpf/btf.h:254:24: note: each undeclared identifier is reported only once for each function it appears in
/home/john/git/bpf-next/tools/lib/bpf/btf.h: In function ‘btf_is_datasec’:
/home/john/git/bpf-next/tools/lib/bpf/btf.h:259:24: error: ‘BTF_KIND_DATASEC’ undeclared (first use in this function); did you mean ‘BTF_KIND_PTR’?
  return btf_kind(t) == BTF_KIND_DATASEC;
                        ^~~~~~~~~~~~~~~~
                        BTF_KIND_PTR
mv: cannot stat '/home/john/git/bpf-next/tools/bpf/btfid/.btfid.o.tmp': No such file or directory
make[3]: *** [/home/john/git/bpf-next/tools/build/Makefile.build:97: /home/john/git/bpf-next/tools/bpf/btfid/btfid.o] Error 1
make[2]: *** [Makefile:59: /home/john/git/bpf-next/tools/bpf/btfid/btfid-in.o] Error 2
make[1]: *** [Makefile:71: bpf/btfid] Error 2
make: *** [Makefile:1894: tools/bpf/btfid] Error 2
John Fastabend June 18, 2020, 9:17 p.m. UTC | #2
John Fastabend wrote:
> Jiri Olsa wrote:
> > The btfid tool will be used during the vmlinux linking,
> > so it's necessary it's ready for it.
> > 
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > ---
> >  Makefile           | 22 ++++++++++++++++++----
> >  tools/Makefile     |  3 +++
> >  tools/bpf/Makefile |  5 ++++-
> >  3 files changed, 25 insertions(+), 5 deletions(-)
> 
> This breaks the build for me. I fixed it with this but then I get warnings,

Also maybe fix below is not good because now I get a segfault in btfid.

> 
> diff --git a/tools/bpf/btfid/btfid.c b/tools/bpf/btfid/btfid.c
> index 7cdf39bfb150..3697e8ae9efa 100644
> --- a/tools/bpf/btfid/btfid.c
> +++ b/tools/bpf/btfid/btfid.c
> @@ -48,7 +48,7 @@
>  #include <errno.h>
>  #include <linux/rbtree.h>
>  #include <linux/zalloc.h>
> -#include <btf.h>
> +#include <linux/btf.h>
>  #include <libbpf.h>
>  #include <parse-options.h>
> 
> Here is the error. Is it something about my setup? bpftool/btf.c uses
> <btf.h>. Because this in top-level Makefile we probably don't want to
> push extra setup onto folks.
> 
> In file included from btfid.c:51:
> /home/john/git/bpf-next/tools/lib/bpf/btf.h: In function ‘btf_is_var’:
> /home/john/git/bpf-next/tools/lib/bpf/btf.h:254:24: error: ‘BTF_KIND_VAR’ undeclared (first use in this function); did you mean ‘BTF_KIND_PTR’?
>   return btf_kind(t) == BTF_KIND_VAR;
>                         ^~~~~~~~~~~~
>                         BTF_KIND_PTR
> /home/john/git/bpf-next/tools/lib/bpf/btf.h:254:24: note: each undeclared identifier is reported only once for each function it appears in
> /home/john/git/bpf-next/tools/lib/bpf/btf.h: In function ‘btf_is_datasec’:
> /home/john/git/bpf-next/tools/lib/bpf/btf.h:259:24: error: ‘BTF_KIND_DATASEC’ undeclared (first use in this function); did you mean ‘BTF_KIND_PTR’?
>   return btf_kind(t) == BTF_KIND_DATASEC;
>                         ^~~~~~~~~~~~~~~~
>                         BTF_KIND_PTR
> mv: cannot stat '/home/john/git/bpf-next/tools/bpf/btfid/.btfid.o.tmp': No such file or directory
> make[3]: *** [/home/john/git/bpf-next/tools/build/Makefile.build:97: /home/john/git/bpf-next/tools/bpf/btfid/btfid.o] Error 1
> make[2]: *** [Makefile:59: /home/john/git/bpf-next/tools/bpf/btfid/btfid-in.o] Error 2
> make[1]: *** [Makefile:71: bpf/btfid] Error 2
> make: *** [Makefile:1894: tools/bpf/btfid] Error 2
Andrii Nakryiko June 19, 2020, 12:40 a.m. UTC | #3
On Tue, Jun 16, 2020 at 3:06 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> The btfid tool will be used during the vmlinux linking,
> so it's necessary it's ready for it.
>

Seeing troubles John runs into, I wonder if it maybe would be better
to add it to pahole instead? It's already a dependency for anything
BTF-related in the kernel. It has libelf, libbpf linked and set up.
WDYT? I've cc'ed Arnaldo as well for an opinion.

> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  Makefile           | 22 ++++++++++++++++++----
>  tools/Makefile     |  3 +++
>  tools/bpf/Makefile |  5 ++++-
>  3 files changed, 25 insertions(+), 5 deletions(-)
>

[...]
Arnaldo June 19, 2020, 12:47 a.m. UTC | #4
On June 18, 2020 9:40:32 PM GMT-03:00, Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote:
>On Tue, Jun 16, 2020 at 3:06 AM Jiri Olsa <jolsa@kernel.org> wrote:
>>
>> The btfid tool will be used during the vmlinux linking,
>> so it's necessary it's ready for it.
>>
>
>Seeing troubles John runs into, I wonder if it maybe would be better
>to add it to pahole instead? It's already a dependency for anything
>BTF-related in the kernel. It has libelf, libbpf linked and set up.
>WDYT? I've cc'ed Arnaldo as well for an opinion.

I was reading this thread with a low prio, but my gut feeling was that yeah, since pahole is already there, why not have it do this?

I'll try to look at this tomorrow and see if this is more than just a hunch.

- Arnaldo
Alexei Starovoitov June 19, 2020, 2:08 a.m. UTC | #5
On Thu, Jun 18, 2020 at 5:47 PM Arnaldo Carvalho de Melo
<arnaldo.melo@gmail.com> wrote:
>
>
>
> On June 18, 2020 9:40:32 PM GMT-03:00, Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote:
> >On Tue, Jun 16, 2020 at 3:06 AM Jiri Olsa <jolsa@kernel.org> wrote:
> >>
> >> The btfid tool will be used during the vmlinux linking,
> >> so it's necessary it's ready for it.
> >>
> >
> >Seeing troubles John runs into, I wonder if it maybe would be better
> >to add it to pahole instead? It's already a dependency for anything
> >BTF-related in the kernel. It has libelf, libbpf linked and set up.
> >WDYT? I've cc'ed Arnaldo as well for an opinion.
>
> I was reading this thread with a low prio, but my gut feeling was that yeah, since pahole is already there, why not have it do this?
>
> I'll try to look at this tomorrow and see if this is more than just a hunch.

I think it's better to keep it separate like Jiri did.
It is really vmlinux specific as far as I can see and can change in the future.
Andrii Nakryiko June 19, 2020, 3:51 a.m. UTC | #6
On Thu, Jun 18, 2020 at 7:08 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Thu, Jun 18, 2020 at 5:47 PM Arnaldo Carvalho de Melo
> <arnaldo.melo@gmail.com> wrote:
> >
> >
> >
> > On June 18, 2020 9:40:32 PM GMT-03:00, Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote:
> > >On Tue, Jun 16, 2020 at 3:06 AM Jiri Olsa <jolsa@kernel.org> wrote:
> > >>
> > >> The btfid tool will be used during the vmlinux linking,
> > >> so it's necessary it's ready for it.
> > >>
> > >
> > >Seeing troubles John runs into, I wonder if it maybe would be better
> > >to add it to pahole instead? It's already a dependency for anything
> > >BTF-related in the kernel. It has libelf, libbpf linked and set up.
> > >WDYT? I've cc'ed Arnaldo as well for an opinion.
> >
> > I was reading this thread with a low prio, but my gut feeling was that yeah, since pahole is already there, why not have it do this?
> >
> > I'll try to look at this tomorrow and see if this is more than just a hunch.
>
> I think it's better to keep it separate like Jiri did.
> It is really vmlinux specific as far as I can see and can change in the future.

Yeah, I actually agree, on the second though, no real reason to put it
in pahole.
Jiri Olsa June 19, 2020, 1:23 p.m. UTC | #7
On Thu, Jun 18, 2020 at 01:40:40PM -0700, John Fastabend wrote:
> Jiri Olsa wrote:
> > The btfid tool will be used during the vmlinux linking,
> > so it's necessary it's ready for it.
> > 
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > ---
> >  Makefile           | 22 ++++++++++++++++++----
> >  tools/Makefile     |  3 +++
> >  tools/bpf/Makefile |  5 ++++-
> >  3 files changed, 25 insertions(+), 5 deletions(-)
> 
> This breaks the build for me. I fixed it with this but then I get warnings,
> 
> diff --git a/tools/bpf/btfid/btfid.c b/tools/bpf/btfid/btfid.c
> index 7cdf39bfb150..3697e8ae9efa 100644
> --- a/tools/bpf/btfid/btfid.c
> +++ b/tools/bpf/btfid/btfid.c
> @@ -48,7 +48,7 @@
>  #include <errno.h>
>  #include <linux/rbtree.h>
>  #include <linux/zalloc.h>
> -#include <btf.h>
> +#include <linux/btf.h>
>  #include <libbpf.h>
>  #include <parse-options.h>
> 
> Here is the error. Is it something about my setup? bpftool/btf.c uses
> <btf.h>. Because this in top-level Makefile we probably don't want to
> push extra setup onto folks.

ouch, I wonder it's because I have libbpf installed and the
setup got mixed up.. I'll erase and try to reproduce

thanks,
jirka

> 
> In file included from btfid.c:51:
> /home/john/git/bpf-next/tools/lib/bpf/btf.h: In function ‘btf_is_var’:
> /home/john/git/bpf-next/tools/lib/bpf/btf.h:254:24: error: ‘BTF_KIND_VAR’ undeclared (first use in this function); did you mean ‘BTF_KIND_PTR’?
>   return btf_kind(t) == BTF_KIND_VAR;
>                         ^~~~~~~~~~~~
>                         BTF_KIND_PTR
> /home/john/git/bpf-next/tools/lib/bpf/btf.h:254:24: note: each undeclared identifier is reported only once for each function it appears in
> /home/john/git/bpf-next/tools/lib/bpf/btf.h: In function ‘btf_is_datasec’:
> /home/john/git/bpf-next/tools/lib/bpf/btf.h:259:24: error: ‘BTF_KIND_DATASEC’ undeclared (first use in this function); did you mean ‘BTF_KIND_PTR’?
>   return btf_kind(t) == BTF_KIND_DATASEC;
>                         ^~~~~~~~~~~~~~~~
>                         BTF_KIND_PTR
> mv: cannot stat '/home/john/git/bpf-next/tools/bpf/btfid/.btfid.o.tmp': No such file or directory
> make[3]: *** [/home/john/git/bpf-next/tools/build/Makefile.build:97: /home/john/git/bpf-next/tools/bpf/btfid/btfid.o] Error 1
> make[2]: *** [Makefile:59: /home/john/git/bpf-next/tools/bpf/btfid/btfid-in.o] Error 2
> make[1]: *** [Makefile:71: bpf/btfid] Error 2
> make: *** [Makefile:1894: tools/bpf/btfid] Error 2
>
diff mbox series

Patch

diff --git a/Makefile b/Makefile
index 839f9fee22cb..b190d502d7d7 100644
--- a/Makefile
+++ b/Makefile
@@ -1066,9 +1066,10 @@  export mod_sign_cmd
 
 HOST_LIBELF_LIBS = $(shell pkg-config libelf --libs 2>/dev/null || echo -lelf)
 
+has_libelf = $(call try-run,\
+               echo "int main() {}" | $(HOSTCC) -xc -o /dev/null $(HOST_LIBELF_LIBS) -,1,0)
+
 ifdef CONFIG_STACK_VALIDATION
-  has_libelf := $(call try-run,\
-		echo "int main() {}" | $(HOSTCC) -xc -o /dev/null $(HOST_LIBELF_LIBS) -,1,0)
   ifeq ($(has_libelf),1)
     objtool_target := tools/objtool FORCE
   else
@@ -1077,6 +1078,14 @@  ifdef CONFIG_STACK_VALIDATION
   endif
 endif
 
+ifdef CONFIG_DEBUG_INFO_BTF
+  ifeq ($(has_libelf),1)
+    btfid_target := tools/bpf/btfid FORCE
+  else
+    ERROR_BTF_IDS_RESOLVE := 1
+  endif
+endif
+
 PHONY += prepare0
 
 export MODORDER := $(extmod-prefix)modules.order
@@ -1188,7 +1197,7 @@  prepare0: archprepare
 	$(Q)$(MAKE) $(build)=.
 
 # All the preparing..
-prepare: prepare0 prepare-objtool
+prepare: prepare0 prepare-objtool prepare-btfid
 
 # Support for using generic headers in asm-generic
 asm-generic := -f $(srctree)/scripts/Makefile.asm-generic obj
@@ -1201,7 +1210,7 @@  uapi-asm-generic:
 	$(Q)$(MAKE) $(asm-generic)=arch/$(SRCARCH)/include/generated/uapi/asm \
 	generic=include/uapi/asm-generic
 
-PHONY += prepare-objtool
+PHONY += prepare-objtool prepare-btfid
 prepare-objtool: $(objtool_target)
 ifeq ($(SKIP_STACK_VALIDATION),1)
 ifdef CONFIG_UNWINDER_ORC
@@ -1212,6 +1221,11 @@  else
 endif
 endif
 
+prepare-btfid: $(btfid_target)
+ifeq ($(ERROR_BTF_IDS_RESOLVE),1)
+	@echo "error: Cannot resolve BTF IDs for CONFIG_DEBUG_INFO_BTF, please install libelf-dev, libelf-devel or elfutils-libelf-devel" >&2
+	@false
+endif
 # Generate some files
 # ---------------------------------------------------------------------------
 
diff --git a/tools/Makefile b/tools/Makefile
index bd778812e915..85af6ebbce91 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -67,6 +67,9 @@  cpupower: FORCE
 cgroup firewire hv guest bootconfig spi usb virtio vm bpf iio gpio objtool leds wmi pci firmware debugging: FORCE
 	$(call descend,$@)
 
+bpf/%: FORCE
+	$(call descend,$@)
+
 liblockdep: FORCE
 	$(call descend,lib/lockdep)
 
diff --git a/tools/bpf/Makefile b/tools/bpf/Makefile
index 77472e28c8fd..d8bbe7ef264f 100644
--- a/tools/bpf/Makefile
+++ b/tools/bpf/Makefile
@@ -124,5 +124,8 @@  runqslower_install:
 runqslower_clean:
 	$(call descend,runqslower,clean)
 
+btfid:
+	$(call descend,btfid)
+
 .PHONY: all install clean bpftool bpftool_install bpftool_clean \
-	runqslower runqslower_install runqslower_clean
+	runqslower runqslower_install runqslower_clean btfid