mbox series

[0/3] perf/bpftool: Allow to link libbpf dynamically

Message ID 20191127094837.4045-1-jolsa@kernel.org
Headers show
Series perf/bpftool: Allow to link libbpf dynamically | expand

Message

Jiri Olsa Nov. 27, 2019, 9:48 a.m. UTC
hi,
adding support to link bpftool with libbpf dynamically,
and config change for perf.

It's now possible to use:
  $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1

which will detect libbpf devel package with needed version,
and if found, link it with bpftool.

It's possible to use arbitrary installed libbpf:
  $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/libbpf/

I based this change on top of Arnaldo's perf/core, because
it contains libbpf feature detection code as dependency.
It's now also synced with latest bpf-next, so Toke's change
applies correctly.

Also available in:
  git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
  libbpf/dyn

thanks,
jirka


---
Jiri Olsa (2):
      perf tools: Allow to specify libbpf install directory
      bpftool: Allow to link libbpf dynamically

Toke Høiland-Jørgensen (1):
      libbpf: Export netlink functions used by bpftool

 tools/bpf/bpftool/Makefile        | 40 +++++++++++++++++++++++++++++++++++++++-
 tools/build/feature/test-libbpf.c |  9 +++++++++
 tools/lib/bpf/libbpf.h            | 22 +++++++++++++---------
 tools/lib/bpf/libbpf.map          |  7 +++++++
 tools/lib/bpf/nlattr.h            | 15 ++++++++++-----
 tools/perf/Makefile.config        | 27 ++++++++++++++++++++-------
 6 files changed, 98 insertions(+), 22 deletions(-)

Comments

Alexei Starovoitov Nov. 27, 2019, 4:37 p.m. UTC | #1
On Wed, Nov 27, 2019 at 1:48 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> hi,
> adding support to link bpftool with libbpf dynamically,
> and config change for perf.
>
> It's now possible to use:
>   $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1
>
> which will detect libbpf devel package with needed version,
> and if found, link it with bpftool.
>
> It's possible to use arbitrary installed libbpf:
>   $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/libbpf/
>
> I based this change on top of Arnaldo's perf/core, because
> it contains libbpf feature detection code as dependency.
> It's now also synced with latest bpf-next, so Toke's change
> applies correctly.

I don't like it.
Especially Toke's patch to expose netlink as public and stable libbpf api.
bpftools needs to stay tightly coupled with libbpf (and statically
linked for that reason).
Otherwise libbpf will grow a ton of public api that would have to be stable
and will quickly become a burden.
Arnaldo Nov. 27, 2019, 6:44 p.m. UTC | #2
Em Wed, Nov 27, 2019 at 08:37:59AM -0800, Alexei Starovoitov escreveu:
> On Wed, Nov 27, 2019 at 1:48 AM Jiri Olsa <jolsa@kernel.org> wrote:
> > adding support to link bpftool with libbpf dynamically,
> > and config change for perf.

> > It's now possible to use:
> >   $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1

> > which will detect libbpf devel package with needed version,
> > and if found, link it with bpftool.

> > It's possible to use arbitrary installed libbpf:
> >   $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/libbpf/

> > I based this change on top of Arnaldo's perf/core, because
> > it contains libbpf feature detection code as dependency.
> > It's now also synced with latest bpf-next, so Toke's change
> > applies correctly.
 
> I don't like it.
> Especially Toke's patch to expose netlink as public and stable libbpf api.
> bpftools needs to stay tightly coupled with libbpf (and statically
> linked for that reason).
> Otherwise libbpf will grow a ton of public api that would have to be stable
> and will quickly become a burden.

I can relate to that, tools/lib/perf/ is only now getting put in place
because of these fears, hopefully the evsel/evlist/cpumap/etc
abstractions there have been in use for a long time and will be not be a
burden... :-)

- Arnaldo
Andrii Nakryiko Nov. 27, 2019, 8:24 p.m. UTC | #3
On Wed, Nov 27, 2019 at 8:38 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Wed, Nov 27, 2019 at 1:48 AM Jiri Olsa <jolsa@kernel.org> wrote:
> >
> > hi,
> > adding support to link bpftool with libbpf dynamically,
> > and config change for perf.
> >
> > It's now possible to use:
> >   $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1
> >
> > which will detect libbpf devel package with needed version,
> > and if found, link it with bpftool.
> >
> > It's possible to use arbitrary installed libbpf:
> >   $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/libbpf/
> >
> > I based this change on top of Arnaldo's perf/core, because
> > it contains libbpf feature detection code as dependency.
> > It's now also synced with latest bpf-next, so Toke's change
> > applies correctly.
>
> I don't like it.
> Especially Toke's patch to expose netlink as public and stable libbpf api.
> bpftools needs to stay tightly coupled with libbpf (and statically
> linked for that reason).
> Otherwise libbpf will grow a ton of public api that would have to be stable
> and will quickly become a burden.

I second that. I'm currently working on adding few more APIs that I'd
like to keep unstable for a while, until we have enough real-world
usage (and feedback) accumulated, before we stabilize them. With
LIBBPF_API and a promise of stable API, we are going to over-stress
and over-design APIs, potentially making them either too generic and
bloated, or too limited (and thus become deprecated almost at
inception time). I'd like to take that pressure off for a super-new
and in flux APIs and not hamper the progress.

I'm thinking of splitting off those non-stable, sort-of-internal APIs
into separate libbpf-experimental.h (or whatever name makes sense),
and let those be used only by tools like bpftool, which are only ever
statically link against libbpf and are ok with occasional changes to
those APIs (which we'll obviously fix in bpftool as well). Pahole
seems like another candidate that fits this bill and we might expose
some stuff early on to it, if it provides tangible benefits (e.g., BTF
dedup speeds ups, etc).

Then as APIs mature, we might decide to move them into libbpf.h with
LIBBPF_API slapped onto them. Any objections?
Daniel Borkmann Nov. 27, 2019, 9:22 p.m. UTC | #4
On 11/27/19 9:24 PM, Andrii Nakryiko wrote:
> On Wed, Nov 27, 2019 at 8:38 AM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
>> On Wed, Nov 27, 2019 at 1:48 AM Jiri Olsa <jolsa@kernel.org> wrote:
>>>
>>> hi,
>>> adding support to link bpftool with libbpf dynamically,
>>> and config change for perf.
>>>
>>> It's now possible to use:
>>>    $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1
>>>
>>> which will detect libbpf devel package with needed version,
>>> and if found, link it with bpftool.
>>>
>>> It's possible to use arbitrary installed libbpf:
>>>    $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/libbpf/
>>>
>>> I based this change on top of Arnaldo's perf/core, because
>>> it contains libbpf feature detection code as dependency.
>>> It's now also synced with latest bpf-next, so Toke's change
>>> applies correctly.
>>
>> I don't like it.
>> Especially Toke's patch to expose netlink as public and stable libbpf api.
>> bpftools needs to stay tightly coupled with libbpf (and statically
>> linked for that reason).
>> Otherwise libbpf will grow a ton of public api that would have to be stable
>> and will quickly become a burden.

+1, and would also be out of scope from a BPF library point of view.

> I second that. I'm currently working on adding few more APIs that I'd
> like to keep unstable for a while, until we have enough real-world
> usage (and feedback) accumulated, before we stabilize them. With
> LIBBPF_API and a promise of stable API, we are going to over-stress
> and over-design APIs, potentially making them either too generic and
> bloated, or too limited (and thus become deprecated almost at
> inception time). I'd like to take that pressure off for a super-new
> and in flux APIs and not hamper the progress.
> 
> I'm thinking of splitting off those non-stable, sort-of-internal APIs
> into separate libbpf-experimental.h (or whatever name makes sense),
> and let those be used only by tools like bpftool, which are only ever
> statically link against libbpf and are ok with occasional changes to
> those APIs (which we'll obviously fix in bpftool as well). Pahole
> seems like another candidate that fits this bill and we might expose
> some stuff early on to it, if it provides tangible benefits (e.g., BTF
> dedup speeds ups, etc).
> 
> Then as APIs mature, we might decide to move them into libbpf.h with
> LIBBPF_API slapped onto them. Any objections?

I don't think adding yet another libbpf_experimental.h makes sense, it feels
too much of an invitation to add all sort of random stuff in there. We already
do have libbpf.h and libbpf_internal.h, so everything that does not relate to
the /stable and public/ API should be moved from libbpf.h into libbpf_internal.h
such as the netlink helpers, as one example, and bpftool can use these since
in-tree changes also cover the latter just fine. So overall, same page, just
reuse/improve libbpf_internal.h instead of a new libbpf_experimental.h.

Thanks,
Daniel
Jiri Olsa Nov. 27, 2019, 10:47 p.m. UTC | #5
On Wed, Nov 27, 2019 at 10:22:31PM +0100, Daniel Borkmann wrote:
> On 11/27/19 9:24 PM, Andrii Nakryiko wrote:
> > On Wed, Nov 27, 2019 at 8:38 AM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > > On Wed, Nov 27, 2019 at 1:48 AM Jiri Olsa <jolsa@kernel.org> wrote:
> > > > 
> > > > hi,
> > > > adding support to link bpftool with libbpf dynamically,
> > > > and config change for perf.
> > > > 
> > > > It's now possible to use:
> > > >    $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1
> > > > 
> > > > which will detect libbpf devel package with needed version,
> > > > and if found, link it with bpftool.
> > > > 
> > > > It's possible to use arbitrary installed libbpf:
> > > >    $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/libbpf/
> > > > 
> > > > I based this change on top of Arnaldo's perf/core, because
> > > > it contains libbpf feature detection code as dependency.
> > > > It's now also synced with latest bpf-next, so Toke's change
> > > > applies correctly.
> > > 
> > > I don't like it.
> > > Especially Toke's patch to expose netlink as public and stable libbpf api.
> > > bpftools needs to stay tightly coupled with libbpf (and statically
> > > linked for that reason).
> > > Otherwise libbpf will grow a ton of public api that would have to be stable
> > > and will quickly become a burden.
> 
> +1, and would also be out of scope from a BPF library point of view.

ok, static it is.. ;-) thanks for the feedback,

jirka


> 
> > I second that. I'm currently working on adding few more APIs that I'd
> > like to keep unstable for a while, until we have enough real-world
> > usage (and feedback) accumulated, before we stabilize them. With
> > LIBBPF_API and a promise of stable API, we are going to over-stress
> > and over-design APIs, potentially making them either too generic and
> > bloated, or too limited (and thus become deprecated almost at
> > inception time). I'd like to take that pressure off for a super-new
> > and in flux APIs and not hamper the progress.
> > 
> > I'm thinking of splitting off those non-stable, sort-of-internal APIs
> > into separate libbpf-experimental.h (or whatever name makes sense),
> > and let those be used only by tools like bpftool, which are only ever
> > statically link against libbpf and are ok with occasional changes to
> > those APIs (which we'll obviously fix in bpftool as well). Pahole
> > seems like another candidate that fits this bill and we might expose
> > some stuff early on to it, if it provides tangible benefits (e.g., BTF
> > dedup speeds ups, etc).
> > 
> > Then as APIs mature, we might decide to move them into libbpf.h with
> > LIBBPF_API slapped onto them. Any objections?
> 
> I don't think adding yet another libbpf_experimental.h makes sense, it feels
> too much of an invitation to add all sort of random stuff in there. We already
> do have libbpf.h and libbpf_internal.h, so everything that does not relate to
> the /stable and public/ API should be moved from libbpf.h into libbpf_internal.h
> such as the netlink helpers, as one example, and bpftool can use these since
> in-tree changes also cover the latter just fine. So overall, same page, just
> reuse/improve libbpf_internal.h instead of a new libbpf_experimental.h.
> 
> Thanks,
> Daniel
>
Toke Høiland-Jørgensen Nov. 28, 2019, 9:06 a.m. UTC | #6
Alexei Starovoitov <alexei.starovoitov@gmail.com> writes:

> On Wed, Nov 27, 2019 at 1:48 AM Jiri Olsa <jolsa@kernel.org> wrote:
>>
>> hi,
>> adding support to link bpftool with libbpf dynamically,
>> and config change for perf.
>>
>> It's now possible to use:
>>   $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1
>>
>> which will detect libbpf devel package with needed version,
>> and if found, link it with bpftool.
>>
>> It's possible to use arbitrary installed libbpf:
>>   $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/libbpf/
>>
>> I based this change on top of Arnaldo's perf/core, because
>> it contains libbpf feature detection code as dependency.
>> It's now also synced with latest bpf-next, so Toke's change
>> applies correctly.
>
> I don't like it.
> Especially Toke's patch to expose netlink as public and stable libbpf
> api.

Figured you might say that :)

> bpftools needs to stay tightly coupled with libbpf (and statically
> linked for that reason).
> Otherwise libbpf will grow a ton of public api that would have to be stable
> and will quickly become a burden.

I can see why you don't want to expose the "internal" functions as
LIBBPF_API. Doesn't *have* to mean we can't link bpftool dynamically
against the .so version of libbpf, though; will see if I can figure out
a clean way to do that...

-Toke
Andrii Nakryiko Dec. 2, 2019, 5:09 p.m. UTC | #7
On Wed, Nov 27, 2019 at 1:49 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> hi,
> adding support to link bpftool with libbpf dynamically,
> and config change for perf.
>
> It's now possible to use:
>   $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1

I wonder what's the motivation behind these changes, though? Why is
linking bpftool dynamically with libbpf is necessary and important?
They are both developed tightly within kernel repo, so I fail to see
what are the huge advantages one can get from linking them
dynamically.

>
> which will detect libbpf devel package with needed version,
> and if found, link it with bpftool.
>
> It's possible to use arbitrary installed libbpf:
>   $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/libbpf/
>
> I based this change on top of Arnaldo's perf/core, because
> it contains libbpf feature detection code as dependency.
> It's now also synced with latest bpf-next, so Toke's change
> applies correctly.
>
> Also available in:
>   git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
>   libbpf/dyn
>
> thanks,
> jirka
>
>
> ---
> Jiri Olsa (2):
>       perf tools: Allow to specify libbpf install directory
>       bpftool: Allow to link libbpf dynamically
>
> Toke Høiland-Jørgensen (1):
>       libbpf: Export netlink functions used by bpftool
>
>  tools/bpf/bpftool/Makefile        | 40 +++++++++++++++++++++++++++++++++++++++-
>  tools/build/feature/test-libbpf.c |  9 +++++++++
>  tools/lib/bpf/libbpf.h            | 22 +++++++++++++---------
>  tools/lib/bpf/libbpf.map          |  7 +++++++
>  tools/lib/bpf/nlattr.h            | 15 ++++++++++-----
>  tools/perf/Makefile.config        | 27 ++++++++++++++++++++-------
>  6 files changed, 98 insertions(+), 22 deletions(-)
>
Toke Høiland-Jørgensen Dec. 2, 2019, 6:08 p.m. UTC | #8
Andrii Nakryiko <andrii.nakryiko@gmail.com> writes:

> On Wed, Nov 27, 2019 at 1:49 AM Jiri Olsa <jolsa@kernel.org> wrote:
>>
>> hi,
>> adding support to link bpftool with libbpf dynamically,
>> and config change for perf.
>>
>> It's now possible to use:
>>   $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1
>
> I wonder what's the motivation behind these changes, though? Why is
> linking bpftool dynamically with libbpf is necessary and important?
> They are both developed tightly within kernel repo, so I fail to see
> what are the huge advantages one can get from linking them
> dynamically.

Well, all the regular reasons for using dynamic linking (memory usage,
binary size, etc). But in particular, the ability to update the libbpf
package if there's a serious bug, and have that be picked up by all
utilities making use of it. No reason why bpftool should be special in
that respect.

-Toke
Andrii Nakryiko Dec. 2, 2019, 6:42 p.m. UTC | #9
On Mon, Dec 2, 2019 at 10:09 AM Toke Høiland-Jørgensen <toke@redhat.com> wrote:
>
> Andrii Nakryiko <andrii.nakryiko@gmail.com> writes:
>
> > On Wed, Nov 27, 2019 at 1:49 AM Jiri Olsa <jolsa@kernel.org> wrote:
> >>
> >> hi,
> >> adding support to link bpftool with libbpf dynamically,
> >> and config change for perf.
> >>
> >> It's now possible to use:
> >>   $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1
> >
> > I wonder what's the motivation behind these changes, though? Why is
> > linking bpftool dynamically with libbpf is necessary and important?
> > They are both developed tightly within kernel repo, so I fail to see
> > what are the huge advantages one can get from linking them
> > dynamically.
>
> Well, all the regular reasons for using dynamic linking (memory usage,
> binary size, etc).

bpftool is 327KB with statically linked libbpf. Hardly a huge problem
for either binary size or memory usage. CPU instruction cache usage is
also hardly a concern for bpftool specifically.

> But in particular, the ability to update the libbpf
> package if there's a serious bug, and have that be picked up by all
> utilities making use of it.

I agree, and that works only for utilities linking with libbpf
dynamically. For tools that build statically, you'd have to update
tools anyways. And if you can update libbpf, you can as well update
bpftool at the same time, so I don't think linking bpftool statically
with libbpf causes any new problems.

> No reason why bpftool should be special in that respect.

But I think bpftool is special and we actually want it to be special
and tightly coupled to libbpf with sometimes very intimate knowledge
of libbpf and access to "hidden" APIs. That allows us to experiment
with new stuff that requires use of bpftool (e.g., code generation for
BPF programs), without having to expose and seal public APIs. And I
don't think it's a problem from the point of code maintenance, because
both live in the same repository and are updated "atomically" when new
features are added or changed.

Beyond superficial binary size worries, I don't see any good reason
why we should add more complexity and variables to libbpf and bpftool
build processes just to have a "nice to have" option of linking
bpftool dynamically with libbpf.


>
> -Toke
>
Arnaldo Dec. 2, 2019, 6:54 p.m. UTC | #10
Em Mon, Dec 02, 2019 at 10:42:53AM -0800, Andrii Nakryiko escreveu:
> On Mon, Dec 2, 2019 at 10:09 AM Toke Høiland-Jørgensen <toke@redhat.com> wrote:
> > Andrii Nakryiko <andrii.nakryiko@gmail.com> writes:
> > > On Wed, Nov 27, 2019 at 1:49 AM Jiri Olsa <jolsa@kernel.org> wrote:
> > >> adding support to link bpftool with libbpf dynamically,
> > >> and config change for perf.

> > >> It's now possible to use:
> > >>   $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1

> > > I wonder what's the motivation behind these changes, though? Why is
> > > linking bpftool dynamically with libbpf is necessary and important?
> > > They are both developed tightly within kernel repo, so I fail to see
> > > what are the huge advantages one can get from linking them
> > > dynamically.

> > Well, all the regular reasons for using dynamic linking (memory usage,
> > binary size, etc).

> bpftool is 327KB with statically linked libbpf. Hardly a huge problem
> for either binary size or memory usage. CPU instruction cache usage is
> also hardly a concern for bpftool specifically.

> > But in particular, the ability to update the libbpf
> > package if there's a serious bug, and have that be picked up by all
> > utilities making use of it.

> I agree, and that works only for utilities linking with libbpf
> dynamically. For tools that build statically, you'd have to update
> tools anyways. And if you can update libbpf, you can as well update
> bpftool at the same time, so I don't think linking bpftool statically
> with libbpf causes any new problems.

> > No reason why bpftool should be special in that respect.

> But I think bpftool is special and we actually want it to be special
> and tightly coupled to libbpf with sometimes very intimate knowledge
> of libbpf and access to "hidden" APIs. That allows us to experiment
> with new stuff that requires use of bpftool (e.g., code generation for
> BPF programs), without having to expose and seal public APIs. And I
> don't think it's a problem from the point of code maintenance, because
> both live in the same repository and are updated "atomically" when new
> features are added or changed.

> Beyond superficial binary size worries, I don't see any good reason
> why we should add more complexity and variables to libbpf and bpftool
> build processes just to have a "nice to have" option of linking
> bpftool dynamically with libbpf.

s/bpftool/perf/g
s/libbpf/libperf/g

And I would also agree 8-)

- Arnaldo
Jiri Olsa Dec. 2, 2019, 7:21 p.m. UTC | #11
On Mon, Dec 02, 2019 at 10:42:53AM -0800, Andrii Nakryiko wrote:
> On Mon, Dec 2, 2019 at 10:09 AM Toke Høiland-Jørgensen <toke@redhat.com> wrote:
> >
> > Andrii Nakryiko <andrii.nakryiko@gmail.com> writes:
> >
> > > On Wed, Nov 27, 2019 at 1:49 AM Jiri Olsa <jolsa@kernel.org> wrote:
> > >>
> > >> hi,
> > >> adding support to link bpftool with libbpf dynamically,
> > >> and config change for perf.
> > >>
> > >> It's now possible to use:
> > >>   $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1
> > >
> > > I wonder what's the motivation behind these changes, though? Why is
> > > linking bpftool dynamically with libbpf is necessary and important?
> > > They are both developed tightly within kernel repo, so I fail to see
> > > what are the huge advantages one can get from linking them
> > > dynamically.
> >
> > Well, all the regular reasons for using dynamic linking (memory usage,
> > binary size, etc).
> 
> bpftool is 327KB with statically linked libbpf. Hardly a huge problem
> for either binary size or memory usage. CPU instruction cache usage is
> also hardly a concern for bpftool specifically.
> 
> > But in particular, the ability to update the libbpf
> > package if there's a serious bug, and have that be picked up by all
> > utilities making use of it.
> 
> I agree, and that works only for utilities linking with libbpf
> dynamically. For tools that build statically, you'd have to update
> tools anyways. And if you can update libbpf, you can as well update
> bpftool at the same time, so I don't think linking bpftool statically
> with libbpf causes any new problems.

it makes difference for us if we need to respin just one library
instead of several applications (bpftool and perf at the moment),
because of the bug in the library

with the Toke's approach we compile some bits of libbpf statically into
bpftool, but there's still the official API in the dynamic libbpf that
we care about and that could carry on the fix without bpftool respin

> > No reason why bpftool should be special in that respect.
> 
> But I think bpftool is special and we actually want it to be special
> and tightly coupled to libbpf with sometimes very intimate knowledge
> of libbpf and access to "hidden" APIs. That allows us to experiment
> with new stuff that requires use of bpftool (e.g., code generation for
> BPF programs), without having to expose and seal public APIs. And I
> don't think it's a problem from the point of code maintenance, because
> both live in the same repository and are updated "atomically" when new
> features are added or changed.

I thought we solved this by Toke's approach, so there' no need
to expose any new/experimental API .. also you guys will probably
continue using static linking I guess

jirka

> 
> Beyond superficial binary size worries, I don't see any good reason
> why we should add more complexity and variables to libbpf and bpftool
> build processes just to have a "nice to have" option of linking
> bpftool dynamically with libbpf.
Andrii Nakryiko Dec. 2, 2019, 7:54 p.m. UTC | #12
On Mon, Dec 2, 2019 at 11:21 AM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Mon, Dec 02, 2019 at 10:42:53AM -0800, Andrii Nakryiko wrote:
> > On Mon, Dec 2, 2019 at 10:09 AM Toke Høiland-Jørgensen <toke@redhat.com> wrote:
> > >
> > > Andrii Nakryiko <andrii.nakryiko@gmail.com> writes:
> > >
> > > > On Wed, Nov 27, 2019 at 1:49 AM Jiri Olsa <jolsa@kernel.org> wrote:
> > > >>
> > > >> hi,
> > > >> adding support to link bpftool with libbpf dynamically,
> > > >> and config change for perf.
> > > >>
> > > >> It's now possible to use:
> > > >>   $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1
> > > >
> > > > I wonder what's the motivation behind these changes, though? Why is
> > > > linking bpftool dynamically with libbpf is necessary and important?
> > > > They are both developed tightly within kernel repo, so I fail to see
> > > > what are the huge advantages one can get from linking them
> > > > dynamically.
> > >
> > > Well, all the regular reasons for using dynamic linking (memory usage,
> > > binary size, etc).
> >
> > bpftool is 327KB with statically linked libbpf. Hardly a huge problem
> > for either binary size or memory usage. CPU instruction cache usage is
> > also hardly a concern for bpftool specifically.
> >
> > > But in particular, the ability to update the libbpf
> > > package if there's a serious bug, and have that be picked up by all
> > > utilities making use of it.
> >
> > I agree, and that works only for utilities linking with libbpf
> > dynamically. For tools that build statically, you'd have to update
> > tools anyways. And if you can update libbpf, you can as well update
> > bpftool at the same time, so I don't think linking bpftool statically
> > with libbpf causes any new problems.
>
> it makes difference for us if we need to respin just one library
> instead of several applications (bpftool and perf at the moment),
> because of the bug in the library
>
> with the Toke's approach we compile some bits of libbpf statically into
> bpftool, but there's still the official API in the dynamic libbpf that
> we care about and that could carry on the fix without bpftool respin

See my replies on v4 of your patchset. I have doubts this actually
works as we hope it works.

I also don't see how that is going to work in general. Imagine
something like this:

static int some_state = 123;

LIBBPF_API void set_state(int x) { some_state = x; }

int get_state() { return some_state; }

If bpftool does:

set_state(42);
printf("%d\n", get_state());


How is this supposed to work with set_state() coming from libbpf.so,
while get_state() being statically linked? Who "owns" memory of `int
some_state` -- bpftool or libbpf.so? Can they magically share it? Or
rather maybe some_state will be actually two different variables in
two different memory regions? And set_state() would be setting one of
them, while get_state() would be reading another one?

It would be good to test this out. Do you mind checking?

>
> > > No reason why bpftool should be special in that respect.
> >
> > But I think bpftool is special and we actually want it to be special
> > and tightly coupled to libbpf with sometimes very intimate knowledge
> > of libbpf and access to "hidden" APIs. That allows us to experiment
> > with new stuff that requires use of bpftool (e.g., code generation for
> > BPF programs), without having to expose and seal public APIs. And I
> > don't think it's a problem from the point of code maintenance, because
> > both live in the same repository and are updated "atomically" when new
> > features are added or changed.
>
> I thought we solved this by Toke's approach, so there' no need
> to expose any new/experimental API .. also you guys will probably
> continue using static linking I guess
>
> jirka
>
> >
> > Beyond superficial binary size worries, I don't see any good reason
> > why we should add more complexity and variables to libbpf and bpftool
> > build processes just to have a "nice to have" option of linking
> > bpftool dynamically with libbpf.
>
Jiri Olsa Dec. 2, 2019, 8:02 p.m. UTC | #13
On Mon, Dec 02, 2019 at 11:54:27AM -0800, Andrii Nakryiko wrote:
> On Mon, Dec 2, 2019 at 11:21 AM Jiri Olsa <jolsa@redhat.com> wrote:
> >
> > On Mon, Dec 02, 2019 at 10:42:53AM -0800, Andrii Nakryiko wrote:
> > > On Mon, Dec 2, 2019 at 10:09 AM Toke Høiland-Jørgensen <toke@redhat.com> wrote:
> > > >
> > > > Andrii Nakryiko <andrii.nakryiko@gmail.com> writes:
> > > >
> > > > > On Wed, Nov 27, 2019 at 1:49 AM Jiri Olsa <jolsa@kernel.org> wrote:
> > > > >>
> > > > >> hi,
> > > > >> adding support to link bpftool with libbpf dynamically,
> > > > >> and config change for perf.
> > > > >>
> > > > >> It's now possible to use:
> > > > >>   $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1
> > > > >
> > > > > I wonder what's the motivation behind these changes, though? Why is
> > > > > linking bpftool dynamically with libbpf is necessary and important?
> > > > > They are both developed tightly within kernel repo, so I fail to see
> > > > > what are the huge advantages one can get from linking them
> > > > > dynamically.
> > > >
> > > > Well, all the regular reasons for using dynamic linking (memory usage,
> > > > binary size, etc).
> > >
> > > bpftool is 327KB with statically linked libbpf. Hardly a huge problem
> > > for either binary size or memory usage. CPU instruction cache usage is
> > > also hardly a concern for bpftool specifically.
> > >
> > > > But in particular, the ability to update the libbpf
> > > > package if there's a serious bug, and have that be picked up by all
> > > > utilities making use of it.
> > >
> > > I agree, and that works only for utilities linking with libbpf
> > > dynamically. For tools that build statically, you'd have to update
> > > tools anyways. And if you can update libbpf, you can as well update
> > > bpftool at the same time, so I don't think linking bpftool statically
> > > with libbpf causes any new problems.
> >
> > it makes difference for us if we need to respin just one library
> > instead of several applications (bpftool and perf at the moment),
> > because of the bug in the library
> >
> > with the Toke's approach we compile some bits of libbpf statically into
> > bpftool, but there's still the official API in the dynamic libbpf that
> > we care about and that could carry on the fix without bpftool respin
> 
> See my replies on v4 of your patchset. I have doubts this actually
> works as we hope it works.
> 
> I also don't see how that is going to work in general. Imagine
> something like this:
> 
> static int some_state = 123;
> 
> LIBBPF_API void set_state(int x) { some_state = x; }
> 
> int get_state() { return some_state; }
> 
> If bpftool does:
> 
> set_state(42);
> printf("%d\n", get_state());
> 
> 
> How is this supposed to work with set_state() coming from libbpf.so,
> while get_state() being statically linked? Who "owns" memory of `int
> some_state` -- bpftool or libbpf.so? Can they magically share it? Or
> rather maybe some_state will be actually two different variables in
> two different memory regions? And set_state() would be setting one of
> them, while get_state() would be reading another one?
> 
> It would be good to test this out. Do you mind checking?

I think you're right.. sry, I should have checked on this more,
there are no relocations for libbpf.so, so it's all statically
linked and the libbpf is just in 'needed' libs record.. ugh :-\

jirka