| Message ID | 20191202131847.30837-1-jolsa@kernel.org |
|---|---|
| Headers | show |
| Series | perf/bpftool: Allow to link libbpf dynamically | expand |
On Mon, Dec 2, 2019 at 5:19 AM Jiri Olsa <jolsa@kernel.org> wrote: > > hi, > adding support to link bpftool with libbpf dynamically, > and config change for perf. > > It's now possible to use: > $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 > > which will detect libbpf devel package and if found, link it with bpftool. > > It's possible to use arbitrary installed libbpf: > $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/libbpf/ > > I based this change on top of Arnaldo's perf/core, because > it contains libbpf feature detection code as dependency. > > Also available in: > git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git > libbpf/dyn > > v4 changes: > - based on Toke's v3 post, there's no need for additional API exports: > > Since bpftool uses bits of libbpf that are not exported as public API in > the .so version, we also pass in libbpf.a to the linker, which allows it to > pick up the private functions from the static library without having to > expose them as ABI. Whoever understands how this is supposed to work, can you please explain? From reading this, I think what we **want** is: - all LIBBPF_API-exposed APIs should be dynamically linked against libbpf.so; - everything else used from libbpf (e.g., netlink APIs), should come from libbpf.a. Am I getting the idea right? If yes, are we sure it actually works like that in practice? I've compiled with LIBBPF_DYNAMIC=1, and what I see is that libelf, libc, zlib, etc functions do have relocations against them in ".rela.plt" section. None of libbpf exposed APIs, though, have any of such relocations. Which to me suggests that they are just statically linked against libbpf.a and libbpf.so is just recorded in ELF as a dynamic library dependency because of this extra -lbpf flag. Which kind of defeats the purpose of this whole endeavor, no? I'm no linker expert, though, so I apologize if I got it completely wrong, would really appreciate someone to detail this a bit more. Thanks! > > - changing some Makefile variable names > - documenting LIBBPF_DYNAMIC and LIBBPF_DIR in the Makefile comment > - extending test_bpftool_build.sh with libbpf dynamic link > > thanks, > jirka > > > --- > Jiri Olsa (6): > perf tools: Allow to specify libbpf install directory > bpftool: Allow to link libbpf dynamically > bpftool: Rename BPF_DIR Makefile variable to LIBBPF_SRC_DIR > bpftool: Rename LIBBPF_OUTPUT Makefile variable to LIBBPF_BUILD_OUTPUT > bpftool: Rename LIBBPF_PATH Makefile variable to LIBBPF_BUILD_PATH > selftests, bpftool: Add build test for libbpf dynamic linking > > tools/bpf/bpftool/Makefile | 54 ++++++++++++++++++++++++++++++++++++++++++++++-------- > tools/perf/Makefile.config | 27 ++++++++++++++++++++------- > tools/testing/selftests/bpf/test_bpftool_build.sh | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++ > 3 files changed, 119 insertions(+), 15 deletions(-) >
Andrii Nakryiko <andrii.nakryiko@gmail.com> writes: > On Mon, Dec 2, 2019 at 5:19 AM Jiri Olsa <jolsa@kernel.org> wrote: >> >> hi, >> adding support to link bpftool with libbpf dynamically, >> and config change for perf. >> >> It's now possible to use: >> $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 >> >> which will detect libbpf devel package and if found, link it with bpftool. >> >> It's possible to use arbitrary installed libbpf: >> $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/libbpf/ >> >> I based this change on top of Arnaldo's perf/core, because >> it contains libbpf feature detection code as dependency. >> >> Also available in: >> git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git >> libbpf/dyn >> >> v4 changes: >> - based on Toke's v3 post, there's no need for additional API exports: >> >> Since bpftool uses bits of libbpf that are not exported as public API in >> the .so version, we also pass in libbpf.a to the linker, which allows it to >> pick up the private functions from the static library without having to >> expose them as ABI. > > Whoever understands how this is supposed to work, can you please > explain? From reading this, I think what we **want** is: > > - all LIBBPF_API-exposed APIs should be dynamically linked against libbpf.so; > - everything else used from libbpf (e.g., netlink APIs), should come > from libbpf.a. > > Am I getting the idea right? > > If yes, are we sure it actually works like that in practice? I've > compiled with LIBBPF_DYNAMIC=1, and what I see is that libelf, libc, > zlib, etc functions do have relocations against them in ".rela.plt" > section. None of libbpf exposed APIs, though, have any of such > relocations. Which to me suggests that they are just statically linked > against libbpf.a and libbpf.so is just recorded in ELF as a dynamic > library dependency because of this extra -lbpf flag. Which kind of > defeats the purpose of this whole endeavor, no? > > I'm no linker expert, though, so I apologize if I got it completely > wrong, would really appreciate someone to detail this a bit more. > Thanks! Ah, that is my mistake: I was getting dynamic libbpf symbols with this approach, but that was because I had the version of libbpf.so in my $LIBDIR that had the patch to expose the netlink APIs as versioned symbols; so it was just pulling in everything from the shared library. So what I was going for was exactly what you described above; but it seems that doesn't actually work. Too bad, and sorry for wasting your time on this :/ -Toke
On Mon, Dec 2, 2019 at 1:15 PM Toke Høiland-Jørgensen <toke@redhat.com> wrote: > > Ah, that is my mistake: I was getting dynamic libbpf symbols with this > approach, but that was because I had the version of libbpf.so in my > $LIBDIR that had the patch to expose the netlink APIs as versioned > symbols; so it was just pulling in everything from the shared library. > > So what I was going for was exactly what you described above; but it > seems that doesn't actually work. Too bad, and sorry for wasting your > time on this :/ bpftool is currently tightly coupled with libbpf and very likely in the future the dependency will be even tighter. In that sense bpftool is an extension of libbpf and libbpf is an extension of bpftool. Andrii is working on set of patches to generate user space .c code from bpf program. bpftool will be generating the code that is specific for the version bpftool and for the version of libbpf. There will be compatibility layers as usual. But in general the situation where a bug in libbpf is so criticial that bpftool needs to repackaged is imo less likely than a bug in bpftool that will require re-packaging of libbpf. bpftool is quite special. It's not a typical user of libbpf. The other way around is more correct. libbpf is a user of the code that bpftool generates and both depend on each other. perf on the other side is what typical user space app that uses libbpf will look like. I think keeping bpftool in the kernel while packaging libbpf out of github was an oversight. I think we need to mirror bpftool into github/libbpf as well and make sure they stay together. The version of libbpf == version of bpftool. Both should come from the same package and so on. May be they can be two different packages but upgrading one should trigger upgrade of another and vice versa. I think one package would be easier though. Thoughts?
On Tue, Dec 03, 2019 at 09:52:11PM -0800, Alexei Starovoitov wrote: > On Mon, Dec 2, 2019 at 1:15 PM Toke Høiland-Jørgensen <toke@redhat.com> wrote: > > > > Ah, that is my mistake: I was getting dynamic libbpf symbols with this > > approach, but that was because I had the version of libbpf.so in my > > $LIBDIR that had the patch to expose the netlink APIs as versioned > > symbols; so it was just pulling in everything from the shared library. > > > > So what I was going for was exactly what you described above; but it > > seems that doesn't actually work. Too bad, and sorry for wasting your > > time on this :/ > > bpftool is currently tightly coupled with libbpf and very likely > in the future the dependency will be even tighter. > In that sense bpftool is an extension of libbpf and libbpf is an extension > of bpftool. > Andrii is working on set of patches to generate user space .c code > from bpf program. > bpftool will be generating the code that is specific for the version > bpftool and for > the version of libbpf. There will be compatibility layers as usual. > But in general the situation where a bug in libbpf is so criticial > that bpftool needs to repackaged is imo less likely than a bug in > bpftool that will require re-packaging of libbpf. > bpftool is quite special. It's not a typical user of libbpf. > The other way around is more correct. libbpf is a user of the code > that bpftool generates and both depend on each other. > perf on the other side is what typical user space app that uses > libbpf will look like. > I think keeping bpftool in the kernel while packaging libbpf > out of github was an oversight. > I think we need to mirror bpftool into github/libbpf as well > and make sure they stay together. The version of libbpf == version of bpftool. > Both should come from the same package and so on. > May be they can be two different packages but > upgrading one should trigger upgrade of another and vice versa. > I think one package would be easier though. > Thoughts? great, makes sense.. Toke already mentioned we might need to move bpftool to libbpf package to solve our issues, now when you guys have already plans for that, it works for us ;-) thanks for sharing the plan, jirka
Alexei Starovoitov <alexei.starovoitov@gmail.com> writes: > On Mon, Dec 2, 2019 at 1:15 PM Toke Høiland-Jørgensen <toke@redhat.com> wrote: >> >> Ah, that is my mistake: I was getting dynamic libbpf symbols with this >> approach, but that was because I had the version of libbpf.so in my >> $LIBDIR that had the patch to expose the netlink APIs as versioned >> symbols; so it was just pulling in everything from the shared library. >> >> So what I was going for was exactly what you described above; but it >> seems that doesn't actually work. Too bad, and sorry for wasting your >> time on this :/ > > bpftool is currently tightly coupled with libbpf and very likely > in the future the dependency will be even tighter. > In that sense bpftool is an extension of libbpf and libbpf is an extension > of bpftool. > Andrii is working on set of patches to generate user space .c code > from bpf program. > bpftool will be generating the code that is specific for the version > bpftool and for > the version of libbpf. There will be compatibility layers as usual. > But in general the situation where a bug in libbpf is so criticial > that bpftool needs to repackaged is imo less likely than a bug in > bpftool that will require re-packaging of libbpf. > bpftool is quite special. It's not a typical user of libbpf. > The other way around is more correct. libbpf is a user of the code > that bpftool generates and both depend on each other. > perf on the other side is what typical user space app that uses > libbpf will look like. > I think keeping bpftool in the kernel while packaging libbpf > out of github was an oversight. > I think we need to mirror bpftool into github/libbpf as well > and make sure they stay together. The version of libbpf == version of bpftool. > Both should come from the same package and so on. > May be they can be two different packages but > upgrading one should trigger upgrade of another and vice versa. > I think one package would be easier though. > Thoughts? Yup, making bpftool explicitly the "libbpf command line interface" makes sense and would help clarify the relationship between the two. As Jiri said, we are already moving in that direction packaging-wise... -Toke
On Wed, Dec 4, 2019 at 2:58 AM Toke Høiland-Jørgensen <toke@redhat.com> wrote: > > Alexei Starovoitov <alexei.starovoitov@gmail.com> writes: > > > On Mon, Dec 2, 2019 at 1:15 PM Toke Høiland-Jørgensen <toke@redhat.com> wrote: > >> > >> Ah, that is my mistake: I was getting dynamic libbpf symbols with this > >> approach, but that was because I had the version of libbpf.so in my > >> $LIBDIR that had the patch to expose the netlink APIs as versioned > >> symbols; so it was just pulling in everything from the shared library. > >> > >> So what I was going for was exactly what you described above; but it > >> seems that doesn't actually work. Too bad, and sorry for wasting your > >> time on this :/ > > > > bpftool is currently tightly coupled with libbpf and very likely > > in the future the dependency will be even tighter. > > In that sense bpftool is an extension of libbpf and libbpf is an extension > > of bpftool. > > Andrii is working on set of patches to generate user space .c code > > from bpf program. > > bpftool will be generating the code that is specific for the version > > bpftool and for > > the version of libbpf. There will be compatibility layers as usual. > > But in general the situation where a bug in libbpf is so criticial > > that bpftool needs to repackaged is imo less likely than a bug in > > bpftool that will require re-packaging of libbpf. > > bpftool is quite special. It's not a typical user of libbpf. > > The other way around is more correct. libbpf is a user of the code > > that bpftool generates and both depend on each other. > > perf on the other side is what typical user space app that uses > > libbpf will look like. > > I think keeping bpftool in the kernel while packaging libbpf > > out of github was an oversight. > > I think we need to mirror bpftool into github/libbpf as well > > and make sure they stay together. The version of libbpf == version of bpftool. > > Both should come from the same package and so on. > > May be they can be two different packages but > > upgrading one should trigger upgrade of another and vice versa. > > I think one package would be easier though. > > Thoughts? > > Yup, making bpftool explicitly the "libbpf command line interface" makes > sense and would help clarify the relationship between the two. As Jiri > said, we are already moving in that direction packaging-wise... Awesome. Let's figure out the logistics. Should we do: git mv tools/bpf/bpftool/ tools/lib/bpf/ and appropriate adjustment to Makefiles ? or keep it where it is and only add to https://github.com/libbpf/libbpf/blob/master/scripts/sync-kernel.sh ?
On Wed, Dec 04, 2019 at 09:39:59AM -0800, Alexei Starovoitov wrote: > On Wed, Dec 4, 2019 at 2:58 AM Toke Høiland-Jørgensen <toke@redhat.com> wrote: > > Alexei Starovoitov <alexei.starovoitov@gmail.com> writes: > > > On Mon, Dec 2, 2019 at 1:15 PM Toke Høiland-Jørgensen <toke@redhat.com> wrote: > > >> > > >> Ah, that is my mistake: I was getting dynamic libbpf symbols with this > > >> approach, but that was because I had the version of libbpf.so in my > > >> $LIBDIR that had the patch to expose the netlink APIs as versioned > > >> symbols; so it was just pulling in everything from the shared library. > > >> > > >> So what I was going for was exactly what you described above; but it > > >> seems that doesn't actually work. Too bad, and sorry for wasting your > > >> time on this :/ > > > > > > bpftool is currently tightly coupled with libbpf and very likely > > > in the future the dependency will be even tighter. > > > In that sense bpftool is an extension of libbpf and libbpf is an extension > > > of bpftool. > > > Andrii is working on set of patches to generate user space .c code > > > from bpf program. > > > bpftool will be generating the code that is specific for the version > > > bpftool and for > > > the version of libbpf. There will be compatibility layers as usual. > > > But in general the situation where a bug in libbpf is so criticial > > > that bpftool needs to repackaged is imo less likely than a bug in > > > bpftool that will require re-packaging of libbpf. > > > bpftool is quite special. It's not a typical user of libbpf. > > > The other way around is more correct. libbpf is a user of the code > > > that bpftool generates and both depend on each other. > > > perf on the other side is what typical user space app that uses > > > libbpf will look like. > > > I think keeping bpftool in the kernel while packaging libbpf > > > out of github was an oversight. > > > I think we need to mirror bpftool into github/libbpf as well > > > and make sure they stay together. The version of libbpf == version of bpftool. > > > Both should come from the same package and so on. > > > May be they can be two different packages but > > > upgrading one should trigger upgrade of another and vice versa. > > > I think one package would be easier though. > > > Thoughts? > > > > Yup, making bpftool explicitly the "libbpf command line interface" makes > > sense and would help clarify the relationship between the two. As Jiri > > said, we are already moving in that direction packaging-wise... > > Awesome. Let's figure out the logistics. > Should we do: > git mv tools/bpf/bpftool/ tools/lib/bpf/ > and appropriate adjustment to Makefiles ? > or keep it where it is and only add to > https://github.com/libbpf/libbpf/blob/master/scripts/sync-kernel.sh ? I'd be in preference of the latter aka keeping where it is. Thanks, Daniel
Daniel Borkmann <daniel@iogearbox.net> writes: > On Wed, Dec 04, 2019 at 09:39:59AM -0800, Alexei Starovoitov wrote: >> On Wed, Dec 4, 2019 at 2:58 AM Toke Høiland-Jørgensen <toke@redhat.com> wrote: >> > Alexei Starovoitov <alexei.starovoitov@gmail.com> writes: >> > > On Mon, Dec 2, 2019 at 1:15 PM Toke Høiland-Jørgensen <toke@redhat.com> wrote: >> > >> >> > >> Ah, that is my mistake: I was getting dynamic libbpf symbols with this >> > >> approach, but that was because I had the version of libbpf.so in my >> > >> $LIBDIR that had the patch to expose the netlink APIs as versioned >> > >> symbols; so it was just pulling in everything from the shared library. >> > >> >> > >> So what I was going for was exactly what you described above; but it >> > >> seems that doesn't actually work. Too bad, and sorry for wasting your >> > >> time on this :/ >> > > >> > > bpftool is currently tightly coupled with libbpf and very likely >> > > in the future the dependency will be even tighter. >> > > In that sense bpftool is an extension of libbpf and libbpf is an extension >> > > of bpftool. >> > > Andrii is working on set of patches to generate user space .c code >> > > from bpf program. >> > > bpftool will be generating the code that is specific for the version >> > > bpftool and for >> > > the version of libbpf. There will be compatibility layers as usual. >> > > But in general the situation where a bug in libbpf is so criticial >> > > that bpftool needs to repackaged is imo less likely than a bug in >> > > bpftool that will require re-packaging of libbpf. >> > > bpftool is quite special. It's not a typical user of libbpf. >> > > The other way around is more correct. libbpf is a user of the code >> > > that bpftool generates and both depend on each other. >> > > perf on the other side is what typical user space app that uses >> > > libbpf will look like. >> > > I think keeping bpftool in the kernel while packaging libbpf >> > > out of github was an oversight. >> > > I think we need to mirror bpftool into github/libbpf as well >> > > and make sure they stay together. The version of libbpf == version of bpftool. >> > > Both should come from the same package and so on. >> > > May be they can be two different packages but >> > > upgrading one should trigger upgrade of another and vice versa. >> > > I think one package would be easier though. >> > > Thoughts? >> > >> > Yup, making bpftool explicitly the "libbpf command line interface" makes >> > sense and would help clarify the relationship between the two. As Jiri >> > said, we are already moving in that direction packaging-wise... >> >> Awesome. Let's figure out the logistics. >> Should we do: >> git mv tools/bpf/bpftool/ tools/lib/bpf/ >> and appropriate adjustment to Makefiles ? >> or keep it where it is and only add to >> https://github.com/libbpf/libbpf/blob/master/scripts/sync-kernel.sh ? > > I'd be in preference of the latter aka keeping where it is. I don't have any strong preference either way. It would make sense to move it to make clear the interdependency (and that bpftool is really the "libbpf cli interface"); but it could also just be kept separate and just document this in the existing bpftool dir. The github repository may need some surgery, though. So maybe let the changes in the kernel tree depend on what's easiest for that? IDK? -Toke
On Tue, Dec 3, 2019 at 9:52 PM Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote: > > On Mon, Dec 2, 2019 at 1:15 PM Toke Høiland-Jørgensen <toke@redhat.com> wrote: > > > > Ah, that is my mistake: I was getting dynamic libbpf symbols with this > > approach, but that was because I had the version of libbpf.so in my > > $LIBDIR that had the patch to expose the netlink APIs as versioned > > symbols; so it was just pulling in everything from the shared library. > > > > So what I was going for was exactly what you described above; but it > > seems that doesn't actually work. Too bad, and sorry for wasting your > > time on this :/ > > bpftool is currently tightly coupled with libbpf and very likely > in the future the dependency will be even tighter. > In that sense bpftool is an extension of libbpf and libbpf is an extension > of bpftool. > Andrii is working on set of patches to generate user space .c code > from bpf program. > bpftool will be generating the code that is specific for the version > bpftool and for > the version of libbpf. There will be compatibility layers as usual. > But in general the situation where a bug in libbpf is so criticial > that bpftool needs to repackaged is imo less likely than a bug in > bpftool that will require re-packaging of libbpf. > bpftool is quite special. It's not a typical user of libbpf. > The other way around is more correct. libbpf is a user of the code > that bpftool generates and both depend on each other. > perf on the other side is what typical user space app that uses > libbpf will look like. > I think keeping bpftool in the kernel while packaging libbpf > out of github was an oversight. I wonder what big advantage having bpftool in libbpf's Github repo brings, actually? The reason we need libbpf on github is to allow other projects like pahole to be able to use libbpf from submodule. There is no such need for bpftool. I agree about preference to release them in sync, but that could be easily done by releasing based on corresponding commits in github's libbpf repo and kernel repo. bpftool doesn't have to physically live next to libbpf on Github, does it? Calling github repo a "mirror" is incorrect. It's not a 1:1 copy of files. We have a completely separate Makefile for libbpf, and we have a bunch of stuff we had to re-implement to detach libbpf code from kernel's non-UAPI headers. Doing this for bpftool as well seems like just more maintenance. Keeping github's Makefile in sync with kernel's Makefile (for libbpf) is PITA, I'd rather avoid similar pains for bpftool without a really good reason. > I think we need to mirror bpftool into github/libbpf as well > and make sure they stay together. The version of libbpf == version of bpftool. > Both should come from the same package and so on. > May be they can be two different packages but > upgrading one should trigger upgrade of another and vice versa. > I think one package would be easier though. > Thoughts?
On Wed, 4 Dec 2019 13:16:13 -0800, Andrii Nakryiko wrote: > I wonder what big advantage having bpftool in libbpf's Github repo > brings, actually? The reason we need libbpf on github is to allow > other projects like pahole to be able to use libbpf from submodule. > There is no such need for bpftool. > > I agree about preference to release them in sync, but that could be > easily done by releasing based on corresponding commits in github's > libbpf repo and kernel repo. bpftool doesn't have to physically live > next to libbpf on Github, does it? +1 > Calling github repo a "mirror" is incorrect. It's not a 1:1 copy of > files. We have a completely separate Makefile for libbpf, and we have > a bunch of stuff we had to re-implement to detach libbpf code from > kernel's non-UAPI headers. Doing this for bpftool as well seems like > just more maintenance. Keeping github's Makefile in sync with kernel's > Makefile (for libbpf) is PITA, I'd rather avoid similar pains for > bpftool without a really good reason. Agreed. Having libbpf on GH is definitely useful today, but one can hope a day will come when distroes will get up to speed on packaging libbpf, and perhaps we can retire it? Maybe 2, 3 years from now? Putting bpftool in the same boat is just more baggage.
On Wed, Dec 04, 2019 at 01:54:05PM -0800, Jakub Kicinski wrote: > On Wed, 4 Dec 2019 13:16:13 -0800, Andrii Nakryiko wrote: > > I wonder what big advantage having bpftool in libbpf's Github repo > > brings, actually? The reason we need libbpf on github is to allow > > other projects like pahole to be able to use libbpf from submodule. > > There is no such need for bpftool. > > > > I agree about preference to release them in sync, but that could be > > easily done by releasing based on corresponding commits in github's > > libbpf repo and kernel repo. bpftool doesn't have to physically live > > next to libbpf on Github, does it? > > +1 > > > Calling github repo a "mirror" is incorrect. It's not a 1:1 copy of > > files. We have a completely separate Makefile for libbpf, and we have > > a bunch of stuff we had to re-implement to detach libbpf code from > > kernel's non-UAPI headers. Doing this for bpftool as well seems like > > just more maintenance. Keeping github's Makefile in sync with kernel's > > Makefile (for libbpf) is PITA, I'd rather avoid similar pains for > > bpftool without a really good reason. > > Agreed. Having libbpf on GH is definitely useful today, but one can hope > a day will come when distroes will get up to speed on packaging libbpf, > and perhaps we can retire it? Maybe 2, 3 years from now? Putting > bpftool in the same boat is just more baggage. Distros should be packaging libbpf and bpftool from single repo on github. Kernel tree is for packaging kernel.
On Wed, 4 Dec 2019 15:39:49 -0800, Alexei Starovoitov wrote: > > Agreed. Having libbpf on GH is definitely useful today, but one can hope > > a day will come when distroes will get up to speed on packaging libbpf, > > and perhaps we can retire it? Maybe 2, 3 years from now? Putting > > bpftool in the same boat is just more baggage. > > Distros should be packaging libbpf and bpftool from single repo on github. > Kernel tree is for packaging kernel. Okay, single repo on GitHub: https://github.com/torvalds/linux we are in agreement 😝 Jokes aside, you may need to provide some reasoning on this one.. The recommendation for packaging libbpf from GitHub never had any clear justification either AFAICR. I honestly don't see why location matters. bpftool started out on GitHub but we moved it into the tree for... ease of packaging/distribution(?!) Now it's handy to have it in the tree to reuse the uapi headers. As much as I don't care if we move it (back) out of the tree - having two copies makes no sense to me. As does having it in the libbpf repo. The sync effort is not warranted. User confusion is not warranted. The distroes already package bpftool from the kernel sources, people had put in time to get to this stage and there aren't any complaints. In fact all the BPF projects and test suites we are involved in at Netronome are entirely happy the packaged versions of LLVM and libbpf in Fedora _today_, IOW the GH libbpf is irrelevant to us already. As for the problem which sparked this discussion - I disagree that bpftool should have "special relationship" with the library. In fact bpftool uses the widest range of libbpf's interfaces of all known projects so it's invaluable for making sure that those interfaces are usable, consistent and complete. You also said a few times you don't want to merge fixes into bpf/net. That divergence from kernel development process is worrying. None of this makes very much sense to me. We're diverging from well established development practices without as much as a justification. Perhaps I'm not clever enough to follow. But if I'm allowed to make an uneducated guess it would be that it's some Facebook internal reason, like it's hard to do backports? :/
From: Jakub Kicinski <jakub.kicinski@netronome.com> Date: Wed, 4 Dec 2019 16:23:48 -0800 > Jokes aside, you may need to provide some reasoning on this one.. > The recommendation for packaging libbpf from GitHub never had any > clear justification either AFAICR. > > I honestly don't see why location matters. bpftool started out on GitHub > but we moved it into the tree for... ease of packaging/distribution(?!) > Now it's handy to have it in the tree to reuse the uapi headers. > > As much as I don't care if we move it (back) out of the tree - having > two copies makes no sense to me. As does having it in the libbpf repo. > The sync effort is not warranted. User confusion is not warranted. Part of this story has to do with how bug fixes propagate via bpf-next instead of the bpf tree, as I understand it. But yeah it would be nice to have a clear documentation on all of the reasoning. On the distro side, people seem to not want to use the separate repo. If you're supporting enterprise customers you don't just sync with upstream, you cherry pick. When cherry picking gets too painful, you sync with upstream possibly eliding upstream new features you don't want to appear in your supported product yet. I agree with tying bpftool and libbpf into the _resulting_ binary distro package, but I'm not totally convinced about separating them out of the kernel source tree.
On Wed, Dec 04, 2019 at 04:23:48PM -0800, Jakub Kicinski wrote: > On Wed, 4 Dec 2019 15:39:49 -0800, Alexei Starovoitov wrote: > > > Agreed. Having libbpf on GH is definitely useful today, but one can hope > > > a day will come when distroes will get up to speed on packaging libbpf, > > > and perhaps we can retire it? Maybe 2, 3 years from now? Putting > > > bpftool in the same boat is just more baggage. > > > > Distros should be packaging libbpf and bpftool from single repo on github. > > Kernel tree is for packaging kernel. > > Okay, single repo on GitHub: > > https://github.com/torvalds/linux and how will you git submodule only libbpf part of kernel github into bcc and other projects? > You also said a few times you don't want to merge fixes into bpf/net. > That divergence from kernel development process is worrying. worrying - why? what exactly the concern you see? Tying user space release into kernel release and user space process into kernel process makes little sense to me. Packaging is different. Compatibility requirements are different. CI is different. Integration with other projects is different. libbpf source code is in the kernel tree only because kernel changes plus libbpf changes plus selftests changes come as single patchset. That is really the only reason. Packaging scripts, CI scripts, etc should be kept out of kernel tree. All that stuff belongs at github/libbpf. > None of this makes very much sense to me. We're diverging from well > established development practices without as much as a justification. The kernel development process was never used for libbpf. Even coding style is different. I'm puzzled why you think user space should be tied to kernel. Everything is so vastly different. Some people say that 8 weeks to bump libbpf version is too long. Other people say that it's too often. libbpf version numbers != kernel version numbers. There is no definition of LTS for libbpf. One day it will be and the version of libbpf picked for LTS will likely have nothing to do with kernel LTS choices. libbpf has to run on all kernels. Newer and older. How do you support that if libbpf is tied with the kernel? > Perhaps I'm not clever enough to follow. But if I'm allowed to make an > uneducated guess it would be that it's some Facebook internal reason, > like it's hard to do backports? :/ hard to do backports? of what?
On Wed, Dec 04, 2019 at 04:29:29PM -0800, David Miller wrote: > From: Jakub Kicinski <jakub.kicinski@netronome.com> > Date: Wed, 4 Dec 2019 16:23:48 -0800 > > > Jokes aside, you may need to provide some reasoning on this one.. > > The recommendation for packaging libbpf from GitHub never had any > > clear justification either AFAICR. > > > > I honestly don't see why location matters. bpftool started out on GitHub > > but we moved it into the tree for... ease of packaging/distribution(?!) > > Now it's handy to have it in the tree to reuse the uapi headers. > > > > As much as I don't care if we move it (back) out of the tree - having > > two copies makes no sense to me. As does having it in the libbpf repo. > > The sync effort is not warranted. User confusion is not warranted. > > Part of this story has to do with how bug fixes propagate via bpf-next > instead of the bpf tree, as I understand it. > > But yeah it would be nice to have a clear documentation on all of the > reasoning. > > On the distro side, people seem to not want to use the separate repo. > If you're supporting enterprise customers you don't just sync with > upstream, you cherry pick. When cherry picking gets too painful, you > sync with upstream possibly eliding upstream new features you don't > want to appear in your supported product yet. > > I agree with tying bpftool and libbpf into the _resulting_ binary > distro package, but I'm not totally convinced about separating them > out of the kernel source tree. Looks like there is a confusion here. I'm not proposing to move bpftool out of kernel tree. The kernel+libbpf+bpftool+selftests already come as single patch set. bpftool has to stay in the kernel tree otherwise things like skeleton patchset won't be possible to accomplish without a lot of coordination between different trees and propagation delays. I'm proposing to tweak github/libbpf sync script to sync bpftool sources from kernel into github, so both libbpf and bpftool can be tested and packaged together. People are working on adding proper CI to github/libbpf. bpftool testing will automatically get more mileage out of that effort. github/libbpf is self contained. It should be built and tested on many different kernels and build environments (like any user space package should be). That's an important goal of CI. When bpftool is part of github/libbpf it will get the same treatment. I see only advantages and not a single disadvantage of building, testing, packaging bpftool out of github/libbpf. To support stable libbpf+bpftool releases we can branch in github and push fixes into branches. Same CI can test master and stable branches.
On Wed, 4 Dec 2019 17:09:32 -0800, Alexei Starovoitov wrote: > On Wed, Dec 04, 2019 at 04:23:48PM -0800, Jakub Kicinski wrote: > > On Wed, 4 Dec 2019 15:39:49 -0800, Alexei Starovoitov wrote: > > > > Agreed. Having libbpf on GH is definitely useful today, but one can hope > > > > a day will come when distroes will get up to speed on packaging libbpf, > > > > and perhaps we can retire it? Maybe 2, 3 years from now? Putting > > > > bpftool in the same boat is just more baggage. > > > > > > Distros should be packaging libbpf and bpftool from single repo on github. > > > Kernel tree is for packaging kernel. > > > > Okay, single repo on GitHub: > > > > https://github.com/torvalds/linux > > and how will you git submodule only libbpf part of kernel github into bcc > and other projects? Why does bcc have to submodule libbpf? Is it in a "special relationship" with libbpf as well? dnf/apt install libbpf Or rather: dnf/apt install bcc since BCC's user doesn't care about dependencies. The day distroes started packaging libbpf and bpftool the game has changed. > > You also said a few times you don't want to merge fixes into bpf/net. > > That divergence from kernel development process is worrying. > > worrying - why? what exactly the concern you see? First it's still in the tree so it just feels wrong to have the process fundamentally different for sections of the tree. Secondly it's best to stick to the tried and tested processes unless there's a good and stated reason to diverge. Now we're neither doing standard user library process nor kernel process. Both of which must account for stable fixes (Dave explained the enterprise distro needs in his reply). > Tying user space release into kernel release and user space process into > kernel process makes little sense to me. I'm not saying we can't do user space, but we should pick one. I have slight preference for kernel process, because it's familiar and it automatically takes of policy questions, which otherwise consume developer's time. Please accept iproute2 as an example of a user space toolset closely related to the kernel. If kernel release model and process made no sense in user space, why do iproute2s developers continue to follow it for years? Let's learn from experience :/ > Packaging is different. There are mostly disadvantages, but the process should be well known. perf has been packaged for years. iproute2 sometimes lags behind the upstream, more than perf in my experience. Also there's rarely a need to update tools closely related to the kernel without the kernel.. > Compatibility requirements are different. True. > CI is different. Well, hard to argue about things which don't exist :/ > Integration with other projects is different. IMHO the submodule use case is fading. dnf/apt install, it's just a normal C library, we've done this for decades. And there is no submodule need for bpftool, which we are discussing. > libbpf source code is in the kernel tree only because kernel changes plus > libbpf changes plus selftests changes come as single patchset. That is really > the only reason. Packaging scripts, CI scripts, etc should be kept out of > kernel tree. All that stuff belongs at github/libbpf. > > > None of this makes very much sense to me. We're diverging from well > > established development practices without as much as a justification. > > The kernel development process was never used for libbpf. What do you mean? I've sure as hell sent patches to net with Fixes tags, S-o-B and all that jazz for libbpf and bpftool. > Even coding style is different. Is it? You mean the damn underscores people are making fun of? :/ > I'm puzzled why you think user space > should be tied to kernel. Everything is so vastly different. Not what I'm saying, I'm saying either run it as user space project in separate repos or kernel project. You're proposing we do both with some syncs and no clear process. > Some people say that 8 weeks to bump libbpf version is too long. > Other people say that it's too often. Those are two conflicting requirements, Alexei, how is breaking apart from the kernel going to make any difference? At least kernel gives us this somewhat reasonable 8 week cadence and we don't have to ponder whether it should be shorter or longer until someone presents us with a compelling reason. > libbpf version numbers != kernel version numbers. Well, libbpf's numbers are arbitrary, and meaningless. They could as well be kernel numbers, like they are for bpftool. Or dates. libbpf doesn't have a roadmap either, it's not really a full-on project on its own. What's 0.1.0 gonna be? Besides stuff lands in libbpf before it hits a major kernel release. So how are you gonna make libbpf releases independently from kernel ones? What if a feature gets a last minute revert in the kernel and it's in libbpf's ABI? Eh, you're just convincing me it should just stay in the kernel now. > There is no definition of LTS for libbpf. One day it will be > and the version of libbpf picked for LTS will likely have > nothing to do with kernel LTS choices. Again, libbpf inherits the kernel process so there is LTS process, the kernel one, and it's taken care of for us by all the existing stable automation. By merging everything into -next you're loosing that, with all but a vague mirage of how there may one day be multiple stable branches.. > libbpf has to run on all kernels. Newer and older. How do you support > that if libbpf is tied with the kernel? Say I have built N kernels UM or for a VM, and we have some test suite: I pull libbpf, build it, run its tests. The only difference between in tree and out of tree is that "pull libbpf" means pulling smaller or larger repo. Doesn't matter that match, it's a low --depth local clone. Sorry for the long response.
On Wed, Dec 04, 2019 at 06:10:28PM -0800, Jakub Kicinski wrote: > On Wed, 4 Dec 2019 17:09:32 -0800, Alexei Starovoitov wrote: > > On Wed, Dec 04, 2019 at 04:23:48PM -0800, Jakub Kicinski wrote: > > > On Wed, 4 Dec 2019 15:39:49 -0800, Alexei Starovoitov wrote: > > > > > Agreed. Having libbpf on GH is definitely useful today, but one can hope > > > > > a day will come when distroes will get up to speed on packaging libbpf, > > > > > and perhaps we can retire it? Maybe 2, 3 years from now? Putting > > > > > bpftool in the same boat is just more baggage. > > > > > > > > Distros should be packaging libbpf and bpftool from single repo on github. > > > > Kernel tree is for packaging kernel. > > > > > > Okay, single repo on GitHub: > > > > > > https://github.com/torvalds/linux > > > > and how will you git submodule only libbpf part of kernel github into bcc > > and other projects? > > Why does bcc have to submodule libbpf? Is it in a "special > relationship" with libbpf as well? > > dnf/apt install libbpf > > Or rather: > > dnf/apt install bcc > > since BCC's user doesn't care about dependencies. The day distroes > started packaging libbpf and bpftool the game has changed. have you ever built bcc ? or bpftrace? I'm not sure how to answer such 'suggestion'. > Please accept iproute2 as an example of a user space toolset closely > related to the kernel. If kernel release model and process made no > sense in user space, why do iproute2s developers continue to follow it > for years? imo iproute2 is an example how things should not be run. But that's a very different topic. > > Packaging is different. > > There are mostly disadvantages, but the process should be well known. > perf has been packaged for years. perf was initially seen as something that should match kernel one to one. yet it diverged over years. I think it's a counter example. > What do you mean? I've sure as hell sent patches to net with Fixes tags which was complete waste of time for people who were sending these patches, for maintainers who applied them and for all stables folks who carried them into kernel stable releases. Not a single libbpf build was made out of those sources. > S-o-B and all that jazz for libbpf and bpftool. Many open source projects use SOB. It's not kernel specific. > > > Even coding style is different. > > Is it? You mean the damn underscores people are making fun of? :/ Are you trolling? Do you understand why __ is there? > libbpf doesn't have a roadmap either, I think you're contrasting that with kernel and saying that kernel has a roadmap ? What is kernel roadmap? > it's not really a full-on project > on its own. What's 0.1.0 gonna be? whenever this bpf community decides to call it 0.1.0. > Besides stuff lands in libbpf before it hits a major kernel release. > So how are you gonna make libbpf releases independently from kernel > ones? What if a feature gets a last minute revert in the kernel and it's > in libbpf's ABI? You mean when kernel gets new feature, then libbpf gets new feature, then libbpf is released, but then kernel feature is reverted? Obviously we should avoid making a libbpf release that relies on kernel features that didn't reach the mainline. Yet there could be plenty of reasons why making libbpf release in the middle of kernel development cycle makes perfect sense. Also reaching Linus's tree in rc1 is also not a guarantee of non-revert. Yet we release libbpf around rc1 because everyone expects bug-fixes after rc1. So it's an exception that solidifies the rule. > > libbpf has to run on all kernels. Newer and older. How do you support > > that if libbpf is tied with the kernel? > > Say I have built N kernels UM or for a VM, and we have some test > suite: I pull libbpf, build it, run its tests. The only difference > between in tree and out of tree is that "pull libbpf" means pulling > smaller or larger repo. Doesn't matter that match, it's a low --depth > local clone. The expected CI is: 1. pull-req proposed. 2. CI picks it up, builds, run tests. 3. humans see results and land or reject pull-req. Now try to think through how CI on top of full kernel tree will be able to pick just the right commits to start build/test cycle. Is it going to cherry-pick from patchworks? That would be awesome. Yet intel 0bot results show that it's easier said than done. I'm not saying it's not possible. Just complex. If you have cycles to integrate *-next into kernelci.org, please go ahead.
On Wed, 4 Dec 2019 19:17:20 -0800, Alexei Starovoitov wrote: > On Wed, Dec 04, 2019 at 06:10:28PM -0800, Jakub Kicinski wrote: > > On Wed, 4 Dec 2019 17:09:32 -0800, Alexei Starovoitov wrote: > > > On Wed, Dec 04, 2019 at 04:23:48PM -0800, Jakub Kicinski wrote: > > > > On Wed, 4 Dec 2019 15:39:49 -0800, Alexei Starovoitov wrote: > > > > > > Agreed. Having libbpf on GH is definitely useful today, but one can hope > > > > > > a day will come when distroes will get up to speed on packaging libbpf, > > > > > > and perhaps we can retire it? Maybe 2, 3 years from now? Putting > > > > > > bpftool in the same boat is just more baggage. > > > > > > > > > > Distros should be packaging libbpf and bpftool from single repo on github. > > > > > Kernel tree is for packaging kernel. > > > > > > > > Okay, single repo on GitHub: > > > > > > > > https://github.com/torvalds/linux > > > > > > and how will you git submodule only libbpf part of kernel github into bcc > > > and other projects? > > > > Why does bcc have to submodule libbpf? Is it in a "special > > relationship" with libbpf as well? > > > > dnf/apt install libbpf > > > > Or rather: > > > > dnf/apt install bcc > > > > since BCC's user doesn't care about dependencies. The day distroes > > started packaging libbpf and bpftool the game has changed. > > have you ever built bcc ? or bpftrace? > I'm not sure how to answer such 'suggestion'. Perhaps someone else has more patience to explain it - why bcc can't just use binary libbpf distribution (static lib + headers) and link against it like it links against other libraries? > > Please accept iproute2 as an example of a user space toolset closely > > related to the kernel. If kernel release model and process made no > > sense in user space, why do iproute2s developers continue to follow it > > for years? > > imo iproute2 is an example how things should not be run. > But that's a very different topic. Please explain, the topic is how to maintain user space closely related to the kernel. Share with us what you dislike about iproute2 so we can fix it. Instead of adding parts of it to bpftool and then pretending that the API added to libbpf to facilitate that duplication is some internal bpftool-only magic which then prevents us from dynamic linking..... 😠 > > > Packaging is different. > > > > There are mostly disadvantages, but the process should be well known. > > perf has been packaged for years. > > perf was initially seen as something that should match kernel one to one. > yet it diverged over years. I think it's a counter example. > > > What do you mean? I've sure as hell sent patches to net with Fixes tags > > which was complete waste of time for people who were sending these > patches, for maintainers who applied them and for all stables folks > who carried them into kernel stable releases. > Not a single libbpf build was made out of those sources. Because libbpf just now entered the distroes, and you suggested the distroes use the GH repo, so sure now it's wasted work. IIRC there were bpftool crash fixes which landed in Fedora via stable. > > > Even coding style is different. > > > > Is it? You mean the damn underscores people are making fun of? :/ > > Are you trolling? Do you understand why __ is there? Not the point. Tell me how the coding style is different. The underscores is the only thing I could think of that's not common in the kernel. > > libbpf doesn't have a roadmap either, > > I think you're contrasting that with kernel and saying > that kernel has a roadmap ? What is kernel roadmap? Kernel road map is the same as libbpf's road map. > > it's not really a full-on project > > on its own. What's 0.1.0 gonna be? > > whenever this bpf community decides to call it 0.1.0. > > > Besides stuff lands in libbpf before it hits a major kernel release. > > So how are you gonna make libbpf releases independently from kernel > > ones? What if a feature gets a last minute revert in the kernel and it's > > in libbpf's ABI? > > You mean when kernel gets new feature, then libbpf gets new feature, then > libbpf is released, but then kernel feature is reverted? Obviously we should > avoid making a libbpf release that relies on kernel features that didn't reach > the mainline. Yet there could be plenty of reasons why making libbpf release in > the middle of kernel development cycle makes perfect sense. But master of libbpf must have all features to test the kernel with, right? So how do we branch of a release in the middle? That's only possible if kernel cycle happens to not have had any features that required libbpf yet? Or are you thinking 3 tier branching where we'd branch off libbpf release, say 2.6.0 that corresponds to kernel X, but it wouldn't be a stable-only release, and we can still backport features added in kernel X + 1 cycle, features which don't require kernel support, and release libbpf 2.7.0? Could work but it'd get tricky, cause if we want to break ABI we'd actually need 4 tiers. ABI compat, kernel version, feature version, stable version. > Also reaching Linus's tree in rc1 is also not a guarantee of non-revert. Yet we > release libbpf around rc1 because everyone expects bug-fixes after rc1. I consider current process to be broken. Hopefully we can improve it. > So it's an exception that solidifies the rule. > > > > libbpf has to run on all kernels. Newer and older. How do you support > > > that if libbpf is tied with the kernel? > > > > Say I have built N kernels UM or for a VM, and we have some test > > suite: I pull libbpf, build it, run its tests. The only difference > > between in tree and out of tree is that "pull libbpf" means pulling > > smaller or larger repo. Doesn't matter that match, it's a low --depth > > local clone. > > The expected CI is: > 1. pull-req proposed. > 2. CI picks it up, builds, run tests. > 3. humans see results and land or reject pull-req. > Now try to think through how CI on top of full kernel tree will > be able to pick just the right commits to start build/test cycle. > Is it going to cherry-pick from patchworks? That would be awesome. > Yet intel 0bot results show that it's easier said than done. > I'm not saying it's not possible. Just complex. > If you have cycles to integrate *-next into kernelci.org, please go ahead. Yes, it is very complex, I know. I've been hacking on something along those lines for the last few weeks. Hopefully I'll have results at some point.. First stab is just doing build testing, checkpatch, verify tags etc. Uploading to patchwork, and sending an email if there were failures. Even that's not easy as a weekend/evening task :( And it requires a lot of manual inspection upfront before it's unleashed on the ML, because it will catch a lot of stupid little stuff and a lot of people will get grumpy. We need to modernize the process across the board. I don't think having zombie read-only repos on GitHub will give contributors confidence so it's not a step in right direction. We should start from the hard problem, that is the CI itself. The problem of correlating user space and kernel patches will have to be solved for netdev, because netdev tests depend on iproute2.
On Wed, Dec 04, 2019 at 08:26:38PM -0800, Jakub Kicinski wrote: > On Wed, 4 Dec 2019 19:17:20 -0800, Alexei Starovoitov wrote: > > On Wed, Dec 04, 2019 at 06:10:28PM -0800, Jakub Kicinski wrote: > > > On Wed, 4 Dec 2019 17:09:32 -0800, Alexei Starovoitov wrote: > > > > On Wed, Dec 04, 2019 at 04:23:48PM -0800, Jakub Kicinski wrote: > > > > > On Wed, 4 Dec 2019 15:39:49 -0800, Alexei Starovoitov wrote: > > > > > > > Agreed. Having libbpf on GH is definitely useful today, but one can hope > > > > > > > a day will come when distroes will get up to speed on packaging libbpf, > > > > > > > and perhaps we can retire it? Maybe 2, 3 years from now? Putting > > > > > > > bpftool in the same boat is just more baggage. > > > > > > > > > > > > Distros should be packaging libbpf and bpftool from single repo on github. > > > > > > Kernel tree is for packaging kernel. > > > > > > > > > > Okay, single repo on GitHub: > > > > > > > > > > https://github.com/torvalds/linux > > > > > > > > and how will you git submodule only libbpf part of kernel github into bcc > > > > and other projects? > > > > > > Why does bcc have to submodule libbpf? Is it in a "special > > > relationship" with libbpf as well? > > > > > > dnf/apt install libbpf > > > > > > Or rather: > > > > > > dnf/apt install bcc > > > > > > since BCC's user doesn't care about dependencies. The day distroes > > > started packaging libbpf and bpftool the game has changed. > > > > have you ever built bcc ? or bpftrace? > > I'm not sure how to answer such 'suggestion'. > > Perhaps someone else has more patience to explain it - why bcc can't > just use binary libbpf distribution (static lib + headers) and link > against it like it links against other libraries? why systemd considered using libbpf as submodule ? When project like bcc needs to run on different distros and different versions of the same distro such project cannot force users to upgrade core libraries to get features that project might need. libbpf is far from stability of libc, libmnl, libelf. There are mature libraries and actively developed libraries. libbpf is at its infancy stage. We're trying hard to be stable, but accumulated baggage is already huge and it is slowing us down. systemd is considering to use libbpf without being submodule too, but it's mainly driven by systemd's CI limitations and nothing else. > Share with us what you dislike about iproute2 so we can fix it. let's start with 24-hr review cycle that we're trying hard to keep in bpf/net trees. Can you help with 24-hr review in iproute2 ? > Because libbpf just now entered the distroes, and you suggested the > distroes use the GH repo, so sure now it's wasted work. You got the timeline wrong. GH repo was done before some distros started packaging libbpf. libbpf is still not available in centos/rhel afaik. But bcc and bpftrace need to work there. Answer? submodules... > > Are you trolling? Do you understand why __ is there? > > Not the point. Tell me how the coding style is different. The > underscores is the only thing I could think of that's not common > in the kernel. we don't actively enforce xmas tree :) > But master of libbpf must have all features to test the kernel with, > right? So how do we branch of a release in the middle? That's only > possible if kernel cycle happens to not have had any features that > required libbpf yet? libbpf rate of changes is higher than bpf bits of the kernel. Amount of patches per week is higher as well. Something like skeleton work might warrant independent release just to get into the hands of people faster. Why wait 8 weeks for kernel release? No reason. > Or are you thinking 3 tier branching where we'd branch off libbpf > release, say 2.6.0 that corresponds to kernel X, libbpf does not correspond to kernel. All libbpf releases must work with any kernel. > but it wouldn't be a > stable-only release, and we can still backport features added in kernel > X + 1 cycle, features which don't require kernel support, and release > libbpf 2.7.0? Say libbpf 0.1 was developed during kernel 5.4. We can 'backport' all libbpf features from kernel tree 5.5 into libbpf 0.1+ and that libbpf 0.1+ must work with kernels 5.4 and 5.5. That is what existing github/libbpf CI is testing.
On Wed, 4 Dec 2019 15:39:49 -0800 Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote: > On Wed, Dec 04, 2019 at 01:54:05PM -0800, Jakub Kicinski wrote: > > On Wed, 4 Dec 2019 13:16:13 -0800, Andrii Nakryiko wrote: > > > I wonder what big advantage having bpftool in libbpf's Github repo > > > brings, actually? The reason we need libbpf on github is to allow > > > other projects like pahole to be able to use libbpf from submodule. > > > There is no such need for bpftool. > > > > > > I agree about preference to release them in sync, but that could be > > > easily done by releasing based on corresponding commits in github's > > > libbpf repo and kernel repo. bpftool doesn't have to physically live > > > next to libbpf on Github, does it? > > > > +1 I don't see any advantage of having bpftool in libbpf's GitHub repo. As Jakub mention we have seen bpftool crash fixes, which would be painful/annoying to maintain fixes for in the libbpf GitHub repo. As Andrii also points out, it requires more work, as GitHub libbpf have to maintain a separate Makefile for bpftool. > > > Calling github repo a "mirror" is incorrect. It's not a 1:1 copy of > > > files. We have a completely separate Makefile for libbpf, and we have > > > a bunch of stuff we had to re-implement to detach libbpf code from > > > kernel's non-UAPI headers. Doing this for bpftool as well seems like > > > just more maintenance. Keeping github's Makefile in sync with kernel's > > > Makefile (for libbpf) is PITA, I'd rather avoid similar pains for > > > bpftool without a really good reason. > > > > Agreed. Having libbpf on GH is definitely useful today, but one can hope > > a day will come when distroes will get up to speed on packaging libbpf, > > and perhaps we can retire it? Maybe 2, 3 years from now? Putting > > bpftool in the same boat is just more baggage. > > Distros should be packaging libbpf and bpftool from single repo on github. > Kernel tree is for packaging kernel. I don't think that is a good idea. You are creating double work and wasting distro developers time. Let me explain: 1. First of all, GitHub libbpf does not have a stable branches (which makes sense, given this is a read-only clone of kernel tree). Thus, distro developers have to maintain that themselves, in their internal package tree (that is based on GitHub libbpf). 2. Kernel BPF changes usually require updates to libbpf, as selftests uses libbpf. Thus, the distro kernel backporter is already required to backport libbpf parts. This is double work, the code changes to libbpf are now maintained in two places for the distro. The disadvantage for distros to package libbpf (+ bpftool and perf) off their distro kernel tree is that a fix to libbpf, requires rolling a new kernel minor release. The solution to that is simply that distro package for libbpf have a separate (RPM) spec file, with own versioning, which sources points to distro kernel tree.
On Thu, Dec 05, 2019 at 09:35:48AM +0100, Jesper Dangaard Brouer wrote: > I don't think that is a good idea. You are creating double work and > wasting distro developers time. Let me explain: > > 1. First of all, GitHub libbpf does not have a stable branches (which > makes sense, given this is a read-only clone of kernel tree). Thus, > distro developers have to maintain that themselves, in their internal > package tree (that is based on GitHub libbpf). > > 2. Kernel BPF changes usually require updates to libbpf, as selftests > uses libbpf. Thus, the distro kernel backporter is already required to > backport libbpf parts. > > This is double work, the code changes to libbpf are now maintained in > two places for the distro. I totally agree with Jesper here. I don't know how the situatiom with packaging libbpf and bpftool looks like in Fedora/Centos/RHEL now, but in openSUSE we would like to build both of them from the kernel source - use kernel-source package as a requirement and use the kernel tree from /usr/src/linux to build those. We do that for bpftool and perf currently. So far we are building bpftool and perf without libbpf being dynamically linked, so there is no dependency between those packages, although we would like to change it as soon as we find a consensus on this series of patches. > The disadvantage for distros to package libbpf (+ bpftool and perf) off > their distro kernel tree is that a fix to libbpf, requires rolling a > new kernel minor release. The solution to that is simply that distro > package for libbpf have a separate (RPM) spec file, with own > versioning, which sources points to distro kernel tree. That's a great idea. So far, we are using the same version for kernel, bfptool and perf, but all of these have separate RPM specs, so we can do that. Michal
hi, adding support to link bpftool with libbpf dynamically, and config change for perf. It's now possible to use: $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 which will detect libbpf devel package and if found, link it with bpftool. It's possible to use arbitrary installed libbpf: $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/libbpf/ I based this change on top of Arnaldo's perf/core, because it contains libbpf feature detection code as dependency. Also available in: git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git libbpf/dyn v4 changes: - based on Toke's v3 post, there's no need for additional API exports: Since bpftool uses bits of libbpf that are not exported as public API in the .so version, we also pass in libbpf.a to the linker, which allows it to pick up the private functions from the static library without having to expose them as ABI. - changing some Makefile variable names - documenting LIBBPF_DYNAMIC and LIBBPF_DIR in the Makefile comment - extending test_bpftool_build.sh with libbpf dynamic link thanks, jirka --- Jiri Olsa (6): perf tools: Allow to specify libbpf install directory bpftool: Allow to link libbpf dynamically bpftool: Rename BPF_DIR Makefile variable to LIBBPF_SRC_DIR bpftool: Rename LIBBPF_OUTPUT Makefile variable to LIBBPF_BUILD_OUTPUT bpftool: Rename LIBBPF_PATH Makefile variable to LIBBPF_BUILD_PATH selftests, bpftool: Add build test for libbpf dynamic linking tools/bpf/bpftool/Makefile | 54 ++++++++++++++++++++++++++++++++++++++++++++++-------- tools/perf/Makefile.config | 27 ++++++++++++++++++++------- tools/testing/selftests/bpf/test_bpftool_build.sh | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 119 insertions(+), 15 deletions(-)