[bpf-next,v5,0/9] tools: bpftool: add probes for system and device

Hi,
This set adds a new command to bpftool in order to dump a list of
eBPF-related parameters for the system (or for a specific network
device) to the console. Once again, this is based on a suggestion from
Daniel.

At this time, output includes:

    - Availability of bpf() system call
    - Availability of bpf() system call for unprivileged users
    - JIT status (enabled or not, with or without debugging traces)
    - JIT hardening status
    - JIT kallsyms exports status
    - Global memory limit for JIT compiler for unprivileged users
    - Status of kernel compilation options related to BPF features
    - Availability of known eBPF program types
    - Availability of known eBPF map types
    - Availability of known eBPF helper functions

There are three different ways to dump this information at this time:

    - Plain output dumps probe results in plain text. It is the most
      flexible options for providing descriptive output to the user, but
      should not be relied upon for parsing the output.
    - JSON output is supported.
    - A third mode, available through the "macros" keyword appended to the
      command line, dumps some of those parameters (not all) as a series of
      "#define" directives, that can be included into a C header file for
      example.

Probes for supported program and map types, and supported helpers, are
directly added to libbpf, so that other applications (or selftests) can
reuse them as necessary.

If the user does not have root privileges (or more precisely, the
CAP_SYS_ADMIN capability) detection will be erroneous for most
parameters. Therefore, forbid non-root users to run the command.

v5:
- Move exported symbols to a new LIBBPF_0.0.2 section in libbpf.map
  (patches 4 to 6).
- Minor fixes on patches 3 and 4.

v4:
- Probe bpf_jit_limit parameter (patch 2).
- Probe some additional kernel config options (patch 3).
- Minor fixes on patch 6.

v3:
- Do not probe kernel version in bpftool (just retrieve it to probe support
  for kprobes in libbpf).
- Change the way results for helper support is displayed: now one list of
  compatible helpers for each program type (and C-style output gets a
  HAVE_PROG_TYPE_HELPER(prog_type, helper) macro to help with tests. See
  patches 6, 7.
- Address other comments from feedback from v2 (please refer to individual
  patches' history).

v2 (please also refer to individual patches' history):
- Move probes for prog/map types, helpers, from bpftool to libbpf.
- Move C-style output as a separate patch, and restrict it to a subset of
  collected information (bpf() availability, prog/map types, helpers).
- Now probe helpers with all supported program types, and display a list of
  compatible program types (as supported on the system) for each helper.
- NOT addressed: grouping compilation options for kernel into subsections
  (patch 3) (I don't see an easy way of grouping them at the moment, please
  see also the discussion on v1 thread).

Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Jesper Dangaard Brouer <brouer@redhat.com>
Cc: Stanislav Fomichev <sdf@google.com>

Quentin Monnet (9):
  tools: bpftool: add basic probe capability, probe syscall availability
  tools: bpftool: add probes for /proc/ eBPF parameters
  tools: bpftool: add probes for kernel configuration options
  tools: bpftool: add probes for eBPF program types
  tools: bpftool: add probes for eBPF map types
  tools: bpftool: add probes for eBPF helper functions
  tools: bpftool: add C-style "#define" output for probes
  tools: bpftool: add probes for a network device
  tools: bpftool: add bash completion for bpftool probes

 .../bpftool/Documentation/bpftool-cgroup.rst  |   1 +
 .../bpftool/Documentation/bpftool-feature.rst |  85 ++
 .../bpf/bpftool/Documentation/bpftool-map.rst |   1 +
 .../bpf/bpftool/Documentation/bpftool-net.rst |   1 +
 .../bpftool/Documentation/bpftool-perf.rst    |   1 +
 .../bpftool/Documentation/bpftool-prog.rst    |   1 +
 tools/bpf/bpftool/Documentation/bpftool.rst   |   1 +
 tools/bpf/bpftool/bash-completion/bpftool     |  19 +
 tools/bpf/bpftool/feature.c                   | 764 ++++++++++++++++++
 tools/bpf/bpftool/main.c                      |   3 +-
 tools/bpf/bpftool/main.h                      |   4 +
 tools/bpf/bpftool/map.c                       |   4 +-
 tools/lib/bpf/Build                           |   2 +-
 tools/lib/bpf/libbpf.h                        |  14 +
 tools/lib/bpf/libbpf.map                      |   9 +
 tools/lib/bpf/libbpf_probes.c                 | 242 ++++++
 16 files changed, 1149 insertions(+), 3 deletions(-)
 create mode 100644 tools/bpf/bpftool/Documentation/bpftool-feature.rst
 create mode 100644 tools/bpf/bpftool/feature.c
 create mode 100644 tools/lib/bpf/libbpf_probes.c

On Thu, Jan 17, 2019 at 7:28 AM Quentin Monnet
<quentin.monnet@netronome.com> wrote:
>
>
> If the user does not have root privileges (or more precisely, the
> CAP_SYS_ADMIN capability) detection will be erroneous for most
> parameters. Therefore, forbid non-root users to run the command.
>
> v5:
> - Move exported symbols to a new LIBBPF_0.0.2 section in libbpf.map
>   (patches 4 to 6).
> - Minor fixes on patches 3 and 4.

I was about to apply the set,
but while testing on older kernels bpftool crashed the box.
It's a kernel bug, no doubt, but bpftool shouldn't be doing
dangerous operations if it can potentially crash it.
I'm debugging what went wrong.
It crashed after this message:
CONFIG_XFRM is set to y
dmesg was:
[1757577.380114] BUG: unable to handle kernel
[1757577.388476] NULL pointer dereference
[1757577.395957]  at           (null)
[1757577.402752] IP:           (null)
[1757577.421265] Call Trace:
[1757577.421269]  ? SyS_bpf+0x18b/0x16b0
[1757577.421272]  ? __audit_syscall_entry+0xb5/0x100
[1757577.421277]  do_syscall_64+0x53/0x150
Not sure how that was possible.
I'm still debugging.

On Tue, Jan 22, 2019 at 10:39 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Thu, Jan 17, 2019 at 7:28 AM Quentin Monnet
> <quentin.monnet@netronome.com> wrote:
> >
> >
> > If the user does not have root privileges (or more precisely, the
> > CAP_SYS_ADMIN capability) detection will be erroneous for most
> > parameters. Therefore, forbid non-root users to run the command.
> >
> > v5:
> > - Move exported symbols to a new LIBBPF_0.0.2 section in libbpf.map
> >   (patches 4 to 6).
> > - Minor fixes on patches 3 and 4.
>
> I was about to apply the set,
> but while testing on older kernels bpftool crashed the box.
> It's a kernel bug, no doubt, but bpftool shouldn't be doing
> dangerous operations if it can potentially crash it.
> I'm debugging what went wrong.
> It crashed after this message:
> CONFIG_XFRM is set to y
> dmesg was:
> [1757577.380114] BUG: unable to handle kernel
> [1757577.388476] NULL pointer dereference
> [1757577.395957]  at           (null)
> [1757577.402752] IP:           (null)
> [1757577.421265] Call Trace:
> [1757577.421269]  ? SyS_bpf+0x18b/0x16b0
> [1757577.421272]  ? __audit_syscall_entry+0xb5/0x100
> [1757577.421277]  do_syscall_64+0x53/0x150
> Not sure how that was possible.
> I'm still debugging.

Changed my mind and applied to bpf-next.
Turned out that one of the map types was defined in ops,
but body was not backported due to large dependencies
not suitable for backport.
That caused map_create syscall to crash.
Will send a separate fix to improve backport resilience.