[PATCHv4,bpf-next,00/13] Add socket lookup support

This series proposes a new helper for the BPF API which allows BPF programs to
perform lookups for sockets in a network namespace. This would allow programs
to determine early on in processing whether the stack is expecting to receive
the packet, and perform some action (eg drop, forward somewhere) based on this
information.

The series is structured roughly into:
* Misc refactor
* Add the socket pointer type
* Add reference tracking to ensure that socket references are freed
* Extend the BPF API to add sk_lookup_xxx() / sk_release() functions
* Add tests/documentation

The helper proposed in this series includes a parameter for a tuple which must
be filled in by the caller to determine the socket to look up. The simplest
case would be filling with the contents of the packet, ie mapping the packet's
5-tuple into the parameter. In common cases, it may alternatively be useful to
reverse the direction of the tuple and perform a lookup, to find the socket
that initiates this connection; and if the BPF program ever performs a form of
IP address translation, it may further be useful to be able to look up
arbitrary tuples that are not based upon the packet, but instead based on state
held in BPF maps or hardcoded in the BPF program.

Currently, access into the socket's fields are limited to those which are
otherwise already accessible, and are restricted to read-only access.

Changes since v3:
* New patch: "bpf: Reuse canonical string formatter for ctx errs"
* Add PTR_TO_SOCKET to is_ctx_reg().
* Add a few new checks to prevent mixing of socket/non-socket pointers.
* Swap order of checks in sock_filter_is_valid_access().
* Prefix register spill macros with "bpf_".
* Add acks from previous round
* Rebase

Changes since v2:
* New patch: "selftests/bpf: Generalize dummy program types".
  This enables adding verifier tests for socket lookup with tail calls.
* Define the semantics of the new helpers more clearly in uAPI header.
* Fix release of caller_net when netns is not specified.
* Use skb->sk to find caller net when skb->dev is unavailable.
* Fix build with !CONFIG_NET.
* Replace ptr_id defensive coding when releasing reference state with an
  internal error (-EFAULT).
* Remove flags argument to sk_release().
* Add several new assembly tests suggested by Daniel.
* Add a few new C tests.
* Fix typo in verifier error message.

Changes since v1:
* Limit netns_id field to 32 bits
* Reuse reg_type_mismatch() in more places
* Reduce the number of passes at convert_ctx_access()
* Replace ptr_id defensive coding when releasing reference state with an
  internal error (-EFAULT)
* Rework 'struct bpf_sock_tuple' to allow passing a packet pointer
* Allow direct packet access from helper
* Fix compile error with CONFIG_IPV6 enabled
* Improve commit messages

Changes since RFC:
* Split up sk_lookup() into sk_lookup_tcp(), sk_lookup_udp().
* Only take references on the socket when necessary.
  * Make sk_release() only free the socket reference in this case.
* Fix some runtime reference leaks:
  * Disallow BPF_LD_[ABS|IND] instructions while holding a reference.
  * Disallow bpf_tail_call() while holding a reference.
* Prevent the same instruction being used for reference and other
  pointer type.
* Simplify locating copies of a reference during helper calls by caching
  the pointer id from the caller.
* Fix kbuild compilation warnings with particular configs.
* Improve code comments describing the new verifier pieces.
* Tested by Nitin

This tree is also available at:
https://github.com/joestringer/linux/commits/submit/sk-lookup-v4

Joe Stringer (13):
  bpf: Add iterator for spilled registers
  bpf: Simplify ptr_min_max_vals adjustment
  bpf: Reuse canonical string formatter for ctx errs
  bpf: Generalize ptr_or_null regs check
  bpf: Add PTR_TO_SOCKET verifier type
  bpf: Macrofy stack state copy
  bpf: Add reference tracking to verifier
  bpf: Add helper to retrieve socket in BPF
  selftests/bpf: Generalize dummy program types
  selftests/bpf: Add tests for reference tracking
  libbpf: Support loading individual progs
  selftests/bpf: Add C tests for reference tracking
  Documentation: Describe bpf reference tracking

 Documentation/networking/filter.txt           |  64 ++
 include/linux/bpf.h                           |  34 +
 include/linux/bpf_verifier.h                  |  37 +-
 include/uapi/linux/bpf.h                      |  93 +-
 kernel/bpf/verifier.c                         | 604 ++++++++++---
 net/core/filter.c                             | 181 +++-
 tools/include/uapi/linux/bpf.h                |  93 +-
 tools/lib/bpf/libbpf.c                        |   4 +-
 tools/lib/bpf/libbpf.h                        |   3 +
 tools/testing/selftests/bpf/Makefile          |   2 +-
 tools/testing/selftests/bpf/bpf_helpers.h     |  12 +
 tools/testing/selftests/bpf/test_progs.c      |  38 +
 .../selftests/bpf/test_sk_lookup_kern.c       | 180 ++++
 tools/testing/selftests/bpf/test_verifier.c   | 814 +++++++++++++++++-
 14 files changed, 2002 insertions(+), 157 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/test_sk_lookup_kern.c

On 10/02/2018 10:35 PM, Joe Stringer wrote:
> This series proposes a new helper for the BPF API which allows BPF programs to
> perform lookups for sockets in a network namespace. This would allow programs
> to determine early on in processing whether the stack is expecting to receive
> the packet, and perform some action (eg drop, forward somewhere) based on this
> information.
> 
> The series is structured roughly into:
> * Misc refactor
> * Add the socket pointer type
> * Add reference tracking to ensure that socket references are freed
> * Extend the BPF API to add sk_lookup_xxx() / sk_release() functions
> * Add tests/documentation
> 
> The helper proposed in this series includes a parameter for a tuple which must
> be filled in by the caller to determine the socket to look up. The simplest
> case would be filling with the contents of the packet, ie mapping the packet's
> 5-tuple into the parameter. In common cases, it may alternatively be useful to
> reverse the direction of the tuple and perform a lookup, to find the socket
> that initiates this connection; and if the BPF program ever performs a form of
> IP address translation, it may further be useful to be able to look up
> arbitrary tuples that are not based upon the packet, but instead based on state
> held in BPF maps or hardcoded in the BPF program.
> 
> Currently, access into the socket's fields are limited to those which are
> otherwise already accessible, and are restricted to read-only access.
> 
> Changes since v3:
> * New patch: "bpf: Reuse canonical string formatter for ctx errs"
> * Add PTR_TO_SOCKET to is_ctx_reg().
> * Add a few new checks to prevent mixing of socket/non-socket pointers.
> * Swap order of checks in sock_filter_is_valid_access().
> * Prefix register spill macros with "bpf_".
> * Add acks from previous round
> * Rebase
> 
> Changes since v2:
> * New patch: "selftests/bpf: Generalize dummy program types".
>   This enables adding verifier tests for socket lookup with tail calls.
> * Define the semantics of the new helpers more clearly in uAPI header.
> * Fix release of caller_net when netns is not specified.
> * Use skb->sk to find caller net when skb->dev is unavailable.
> * Fix build with !CONFIG_NET.
> * Replace ptr_id defensive coding when releasing reference state with an
>   internal error (-EFAULT).
> * Remove flags argument to sk_release().
> * Add several new assembly tests suggested by Daniel.
> * Add a few new C tests.
> * Fix typo in verifier error message.
> 
> Changes since v1:
> * Limit netns_id field to 32 bits
> * Reuse reg_type_mismatch() in more places
> * Reduce the number of passes at convert_ctx_access()
> * Replace ptr_id defensive coding when releasing reference state with an
>   internal error (-EFAULT)
> * Rework 'struct bpf_sock_tuple' to allow passing a packet pointer
> * Allow direct packet access from helper
> * Fix compile error with CONFIG_IPV6 enabled
> * Improve commit messages
> 
> Changes since RFC:
> * Split up sk_lookup() into sk_lookup_tcp(), sk_lookup_udp().
> * Only take references on the socket when necessary.
>   * Make sk_release() only free the socket reference in this case.
> * Fix some runtime reference leaks:
>   * Disallow BPF_LD_[ABS|IND] instructions while holding a reference.
>   * Disallow bpf_tail_call() while holding a reference.
> * Prevent the same instruction being used for reference and other
>   pointer type.
> * Simplify locating copies of a reference during helper calls by caching
>   the pointer id from the caller.
> * Fix kbuild compilation warnings with particular configs.
> * Improve code comments describing the new verifier pieces.
> * Tested by Nitin
> 
> This tree is also available at:
> https://github.com/joestringer/linux/commits/submit/sk-lookup-v4
> 
> Joe Stringer (13):
>   bpf: Add iterator for spilled registers
>   bpf: Simplify ptr_min_max_vals adjustment
>   bpf: Reuse canonical string formatter for ctx errs
>   bpf: Generalize ptr_or_null regs check
>   bpf: Add PTR_TO_SOCKET verifier type
>   bpf: Macrofy stack state copy
>   bpf: Add reference tracking to verifier
>   bpf: Add helper to retrieve socket in BPF
>   selftests/bpf: Generalize dummy program types
>   selftests/bpf: Add tests for reference tracking
>   libbpf: Support loading individual progs
>   selftests/bpf: Add C tests for reference tracking
>   Documentation: Describe bpf reference tracking

Applied to bpf-next, thanks Joe!