[v4,bpf-next,00/13] bpf: Enable bpf_skc_to_* sock casting helper to networking prog type

This set allows networking prog type to directly read fields from
the in-kernel socket type, e.g. "struct tcp_sock".

Patch 2 has the details on the use case.

v3:
- Pass arg_btf_id instead of fn into check_reg_type() in Patch 1 (Lorenz)
- Move arg_btf_id from func_proto to struct bpf_reg_types in Patch 2 (Lorenz)
- Remove test_sock_fields from .gitignore in Patch 8 (Andrii)
- Add tests to have better coverage on the modified helpers (Alexei)
  Patch 13 is added.
- Use "void *sk" as the helper argument in UAPI bpf.h
  
v3:
- ARG_PTR_TO_SOCK_COMMON_OR_NULL was attempted in v2.  The _OR_NULL was
  needed because the PTR_TO_BTF_ID could be NULL but note that a could be NULL
  PTR_TO_BTF_ID is not a scalar NULL to the verifier.  "_OR_NULL" implicitly
  gives an expectation that the helper can take a scalar NULL which does
  not make sense in most (except one) helpers.  Passing scalar NULL
  should be rejected at the verification time.

  Thus, this patch uses ARG_PTR_TO_BTF_ID_SOCK_COMMON to specify that the
  helper can take both the btf-id ptr or the legacy PTR_TO_SOCK_COMMON but
  not scalar NULL.  It requires the func_proto to explicitly specify the
  arg_btf_id such that there is a very clear expectation that the helper
  can handle a NULL PTR_TO_BTF_ID.

v2:
- Add ARG_PTR_TO_SOCK_COMMON_OR_NULL (Lorenz)

Martin KaFai Lau (13):
  bpf: Move the PTR_TO_BTF_ID check to check_reg_type()
  bpf: Enable bpf_skc_to_* sock casting helper to networking prog type
  bpf: Change bpf_sk_release and bpf_sk_*cgroup_id to accept
    ARG_PTR_TO_BTF_ID_SOCK_COMMON
  bpf: Change bpf_sk_storage_*() to accept ARG_PTR_TO_BTF_ID_SOCK_COMMON
  bpf: Change bpf_tcp_*_syncookie to accept
    ARG_PTR_TO_BTF_ID_SOCK_COMMON
  bpf: Change bpf_sk_assign to accept ARG_PTR_TO_BTF_ID_SOCK_COMMON
  bpf: selftest: Add ref_tracking verifier test for bpf_skc casting
  bpf: selftest: Move sock_fields test into test_progs
  bpf: selftest: Adapt sock_fields test to use skel and global variables
  bpf: selftest: Use network_helpers in the sock_fields test
  bpf: selftest: Use bpf_skc_to_tcp_sock() in the sock_fields test
  bpf: selftest: Remove enum tcp_ca_state from bpf_tcp_helpers.h
  bpf: selftest: Add test_btf_skc_cls_ingress

 include/linux/bpf.h                           |   1 +
 include/net/bpf_sk_storage.h                  |   2 -
 include/uapi/linux/bpf.h                      |  15 +-
 kernel/bpf/bpf_lsm.c                          |   4 +-
 kernel/bpf/verifier.c                         |  94 ++--
 net/core/bpf_sk_storage.c                     |  29 +-
 net/core/filter.c                             | 111 ++--
 net/ipv4/bpf_tcp_ca.c                         |  23 +-
 tools/include/uapi/linux/bpf.h                |  15 +-
 tools/testing/selftests/bpf/.gitignore        |   1 -
 tools/testing/selftests/bpf/Makefile          |   2 +-
 tools/testing/selftests/bpf/bpf_tcp_helpers.h |  13 +-
 .../bpf/prog_tests/btf_skc_cls_ingress.c      | 234 +++++++++
 .../selftests/bpf/prog_tests/sock_fields.c    | 382 ++++++++++++++
 tools/testing/selftests/bpf/progs/bpf_cubic.c |   2 +
 tools/testing/selftests/bpf/progs/bpf_dctcp.c |   2 +
 .../bpf/progs/test_btf_skc_cls_ingress.c      | 174 +++++++
 ..._sock_fields_kern.c => test_sock_fields.c} | 176 ++++---
 .../testing/selftests/bpf/test_sock_fields.c  | 482 ------------------
 .../selftests/bpf/verifier/ref_tracking.c     |  47 ++
 20 files changed, 1093 insertions(+), 716 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_skc_cls_ingress.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/sock_fields.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_btf_skc_cls_ingress.c
 rename tools/testing/selftests/bpf/progs/{test_sock_fields_kern.c => test_sock_fields.c} (61%)
 delete mode 100644 tools/testing/selftests/bpf/test_sock_fields.c

On Fri, 25 Sep 2020 at 01:03, Martin KaFai Lau <kafai@fb.com> wrote:
>
> This set allows networking prog type to directly read fields from
> the in-kernel socket type, e.g. "struct tcp_sock".
>
> Patch 2 has the details on the use case.
>
> v3:
> - Pass arg_btf_id instead of fn into check_reg_type() in Patch 1 (Lorenz)
> - Move arg_btf_id from func_proto to struct bpf_reg_types in Patch 2 (Lorenz)
> - Remove test_sock_fields from .gitignore in Patch 8 (Andrii)
> - Add tests to have better coverage on the modified helpers (Alexei)
>   Patch 13 is added.
> - Use "void *sk" as the helper argument in UAPI bpf.h
>
> v3:
> - ARG_PTR_TO_SOCK_COMMON_OR_NULL was attempted in v2.  The _OR_NULL was
>   needed because the PTR_TO_BTF_ID could be NULL but note that a could be NULL
>   PTR_TO_BTF_ID is not a scalar NULL to the verifier.  "_OR_NULL" implicitly
>   gives an expectation that the helper can take a scalar NULL which does
>   not make sense in most (except one) helpers.  Passing scalar NULL
>   should be rejected at the verification time.
>
>   Thus, this patch uses ARG_PTR_TO_BTF_ID_SOCK_COMMON to specify that the
>   helper can take both the btf-id ptr or the legacy PTR_TO_SOCK_COMMON but
>   not scalar NULL.  It requires the func_proto to explicitly specify the
>   arg_btf_id such that there is a very clear expectation that the helper
>   can handle a NULL PTR_TO_BTF_ID.
>
> v2:
> - Add ARG_PTR_TO_SOCK_COMMON_OR_NULL (Lorenz)
>
> Martin KaFai Lau (13):
>   bpf: Move the PTR_TO_BTF_ID check to check_reg_type()
>   bpf: Enable bpf_skc_to_* sock casting helper to networking prog type
>   bpf: Change bpf_sk_release and bpf_sk_*cgroup_id to accept
>     ARG_PTR_TO_BTF_ID_SOCK_COMMON
>   bpf: Change bpf_sk_storage_*() to accept ARG_PTR_TO_BTF_ID_SOCK_COMMON
>   bpf: Change bpf_tcp_*_syncookie to accept
>     ARG_PTR_TO_BTF_ID_SOCK_COMMON
>   bpf: Change bpf_sk_assign to accept ARG_PTR_TO_BTF_ID_SOCK_COMMON
>   bpf: selftest: Add ref_tracking verifier test for bpf_skc casting
>   bpf: selftest: Move sock_fields test into test_progs
>   bpf: selftest: Adapt sock_fields test to use skel and global variables
>   bpf: selftest: Use network_helpers in the sock_fields test
>   bpf: selftest: Use bpf_skc_to_tcp_sock() in the sock_fields test
>   bpf: selftest: Remove enum tcp_ca_state from bpf_tcp_helpers.h
>   bpf: selftest: Add test_btf_skc_cls_ingress


LGTM, thanks!

On Thu, Sep 24, 2020 at 5:03 PM Martin KaFai Lau <kafai@fb.com> wrote:
>
> This set allows networking prog type to directly read fields from
> the in-kernel socket type, e.g. "struct tcp_sock".

Applied. Thanks