| Message ID | 20190223010703.678070-3-brakmo@fb.com |
|---|---|
| State | Changes Requested |
| Delegated to: | BPF Maintainers |
| Headers | show |
| Series | bpf: Network Resource Manager (NRM) | expand |
On 02/22/2019 05:06 PM, brakmo wrote: > From: Martin KaFai Lau <kafai@fb.com> > > This patch adds a new bpf helper BPF_FUNC_tcp_enter_cwr > "int bpf_tcp_enter_cwr(struct bpf_tcp_sock *tp)". > It is added to BPF_PROG_TYPE_CGROUP_SKB which can be attached > to the egress path where the bpf prog is called by > ip_finish_output() or ip6_finish_output(). The verifier > ensures that the parameter must be a tcp_sock. > > This helper makes a tcp_sock enter CWR state. It can be used > by a bpf_prog to manage egress network bandwidth limit per > cgroupv2. A later patch will have a sample program to > show how it can be used to limit bandwidth usage per cgroupv2. > > To ensure it is only called from BPF_CGROUP_INET_EGRESS, the > attr->expected_attach_type must be specified as BPF_CGROUP_INET_EGRESS > during load time if the prog uses this new helper. > The newly added prog->enforce_expected_attach_type bit will also be set > if this new helper is used. This bit is for backward compatibility reason > because currently prog->expected_attach_type has been ignored in > BPF_PROG_TYPE_CGROUP_SKB. During attach time, > prog->expected_attach_type is only enforced if the > prog->enforce_expected_attach_type bit is set. > i.e. prog->expected_attach_type is only enforced if this new helper > is used by the prog. > BTW, it seems to me that BPF_CGROUP_INET_EGRESS can be used while the socket lock is not held. Maybe we should fix :/
On Sat, Feb 23, 2019 at 05:32:14PM -0800, Eric Dumazet wrote: > > > On 02/22/2019 05:06 PM, brakmo wrote: > > From: Martin KaFai Lau <kafai@fb.com> > > > > This patch adds a new bpf helper BPF_FUNC_tcp_enter_cwr > > "int bpf_tcp_enter_cwr(struct bpf_tcp_sock *tp)". > > It is added to BPF_PROG_TYPE_CGROUP_SKB which can be attached > > to the egress path where the bpf prog is called by > > ip_finish_output() or ip6_finish_output(). The verifier > > ensures that the parameter must be a tcp_sock. > > > > This helper makes a tcp_sock enter CWR state. It can be used > > by a bpf_prog to manage egress network bandwidth limit per > > cgroupv2. A later patch will have a sample program to > > show how it can be used to limit bandwidth usage per cgroupv2. > > > > To ensure it is only called from BPF_CGROUP_INET_EGRESS, the > > attr->expected_attach_type must be specified as BPF_CGROUP_INET_EGRESS > > during load time if the prog uses this new helper. > > The newly added prog->enforce_expected_attach_type bit will also be set > > if this new helper is used. This bit is for backward compatibility reason > > because currently prog->expected_attach_type has been ignored in > > BPF_PROG_TYPE_CGROUP_SKB. During attach time, > > prog->expected_attach_type is only enforced if the > > prog->enforce_expected_attach_type bit is set. > > i.e. prog->expected_attach_type is only enforced if this new helper > > is used by the prog. > > > > BTW, it seems to me that BPF_CGROUP_INET_EGRESS can be used while the socket lock is not held. Thanks for pointing it out. ic. I just noticed the comments at ip6_xmit(): /* * xmit an sk_buff (used by TCP, SCTP and DCCP) * Note : socket lock is not held for SYNACK packets, but might be modified * by calls to skb_set_owner_w() and ipv6_local_error(), * which are using proper atomic operations or spinlocks. */ Is there other cases other than SYNACK? Thanks, Martin
On Sun, Feb 24, 2019 at 03:08:48AM +0000, Martin Lau wrote: > On Sat, Feb 23, 2019 at 05:32:14PM -0800, Eric Dumazet wrote: > > > > > > On 02/22/2019 05:06 PM, brakmo wrote: > > > From: Martin KaFai Lau <kafai@fb.com> > > > > > > This patch adds a new bpf helper BPF_FUNC_tcp_enter_cwr > > > "int bpf_tcp_enter_cwr(struct bpf_tcp_sock *tp)". > > > It is added to BPF_PROG_TYPE_CGROUP_SKB which can be attached > > > to the egress path where the bpf prog is called by > > > ip_finish_output() or ip6_finish_output(). The verifier > > > ensures that the parameter must be a tcp_sock. > > > > > > This helper makes a tcp_sock enter CWR state. It can be used > > > by a bpf_prog to manage egress network bandwidth limit per > > > cgroupv2. A later patch will have a sample program to > > > show how it can be used to limit bandwidth usage per cgroupv2. > > > > > > To ensure it is only called from BPF_CGROUP_INET_EGRESS, the > > > attr->expected_attach_type must be specified as BPF_CGROUP_INET_EGRESS > > > during load time if the prog uses this new helper. > > > The newly added prog->enforce_expected_attach_type bit will also be set > > > if this new helper is used. This bit is for backward compatibility reason > > > because currently prog->expected_attach_type has been ignored in > > > BPF_PROG_TYPE_CGROUP_SKB. During attach time, > > > prog->expected_attach_type is only enforced if the > > > prog->enforce_expected_attach_type bit is set. > > > i.e. prog->expected_attach_type is only enforced if this new helper > > > is used by the prog. > > > > > > > BTW, it seems to me that BPF_CGROUP_INET_EGRESS can be used while the socket lock is not held. > Thanks for pointing it out. > > ic. I just noticed the comments at ip6_xmit(): > /* > * xmit an sk_buff (used by TCP, SCTP and DCCP) > * Note : socket lock is not held for SYNACK packets, but might be modified > * by calls to skb_set_owner_w() and ipv6_local_error(), > * which are using proper atomic operations or spinlocks. > */ > Is there other cases other than SYNACK? I don't think it's a problem. the helper does: BPF_CALL_1(bpf_tcp_enter_cwr, struct tcp_sock *, tp) +{ + struct sock *sk = (struct sock *)tp; + + if (sk->sk_state == TCP_ESTABLISHED) { + tcp_enter_cwr(sk); I believe at the time ip_finish_output is called on established socket it's safe to call tcp_enter_cwr. I don't see how this is different from normal __tcp_transmit_skb path. Eric, what issue do you see?
On 02/23/2019 07:08 PM, Martin Lau wrote: > On Sat, Feb 23, 2019 at 05:32:14PM -0800, Eric Dumazet wrote: >> >> >> On 02/22/2019 05:06 PM, brakmo wrote: >>> From: Martin KaFai Lau <kafai@fb.com> >>> >>> This patch adds a new bpf helper BPF_FUNC_tcp_enter_cwr >>> "int bpf_tcp_enter_cwr(struct bpf_tcp_sock *tp)". >>> It is added to BPF_PROG_TYPE_CGROUP_SKB which can be attached >>> to the egress path where the bpf prog is called by >>> ip_finish_output() or ip6_finish_output(). The verifier >>> ensures that the parameter must be a tcp_sock. >>> >>> This helper makes a tcp_sock enter CWR state. It can be used >>> by a bpf_prog to manage egress network bandwidth limit per >>> cgroupv2. A later patch will have a sample program to >>> show how it can be used to limit bandwidth usage per cgroupv2. >>> >>> To ensure it is only called from BPF_CGROUP_INET_EGRESS, the >>> attr->expected_attach_type must be specified as BPF_CGROUP_INET_EGRESS >>> during load time if the prog uses this new helper. >>> The newly added prog->enforce_expected_attach_type bit will also be set >>> if this new helper is used. This bit is for backward compatibility reason >>> because currently prog->expected_attach_type has been ignored in >>> BPF_PROG_TYPE_CGROUP_SKB. During attach time, >>> prog->expected_attach_type is only enforced if the >>> prog->enforce_expected_attach_type bit is set. >>> i.e. prog->expected_attach_type is only enforced if this new helper >>> is used by the prog. >>> >> >> BTW, it seems to me that BPF_CGROUP_INET_EGRESS can be used while the socket lock is not held. > Thanks for pointing it out. > > ic. I just noticed the comments at ip6_xmit(): > /* > * xmit an sk_buff (used by TCP, SCTP and DCCP) > * Note : socket lock is not held for SYNACK packets, but might be modified > * by calls to skb_set_owner_w() and ipv6_local_error(), > * which are using proper atomic operations or spinlocks. > */ > Is there other cases other than SYNACK? Well, I was referring to various virtual devices, re-entering ip stack. Since we can have a qdisc on any netdev, there is no way we can guarantee the socket is locked by the current thread. Random example : ipvlan_process_v4_outbound() ... err = ip_local_out(net, skb->sk, skb); ...
On 02/22, brakmo wrote: > From: Martin KaFai Lau <kafai@fb.com> > > This patch adds a new bpf helper BPF_FUNC_tcp_enter_cwr > "int bpf_tcp_enter_cwr(struct bpf_tcp_sock *tp)". > It is added to BPF_PROG_TYPE_CGROUP_SKB which can be attached > to the egress path where the bpf prog is called by > ip_finish_output() or ip6_finish_output(). The verifier > ensures that the parameter must be a tcp_sock. > > This helper makes a tcp_sock enter CWR state. It can be used > by a bpf_prog to manage egress network bandwidth limit per > cgroupv2. A later patch will have a sample program to > show how it can be used to limit bandwidth usage per cgroupv2. > > To ensure it is only called from BPF_CGROUP_INET_EGRESS, the > attr->expected_attach_type must be specified as BPF_CGROUP_INET_EGRESS > during load time if the prog uses this new helper. > The newly added prog->enforce_expected_attach_type bit will also be set > if this new helper is used. This bit is for backward compatibility reason > because currently prog->expected_attach_type has been ignored in > BPF_PROG_TYPE_CGROUP_SKB. During attach time, > prog->expected_attach_type is only enforced if the > prog->enforce_expected_attach_type bit is set. > i.e. prog->expected_attach_type is only enforced if this new helper > is used by the prog. > > Signed-off-by: Lawrence Brakmo <brakmo@fb.com> > Signed-off-by: Martin KaFai Lau <kafai@fb.com> > --- > include/linux/bpf.h | 1 + > include/linux/filter.h | 3 ++- > include/uapi/linux/bpf.h | 9 ++++++++- > kernel/bpf/syscall.c | 12 ++++++++++++ > kernel/bpf/verifier.c | 4 ++++ > net/core/filter.c | 25 +++++++++++++++++++++++++ > 6 files changed, 52 insertions(+), 2 deletions(-) > > diff --git a/include/linux/bpf.h b/include/linux/bpf.h > index d5ba2fc01af3..2d54ba7cf9dd 100644 > --- a/include/linux/bpf.h > +++ b/include/linux/bpf.h > @@ -195,6 +195,7 @@ enum bpf_arg_type { > ARG_PTR_TO_SOCKET, /* pointer to bpf_sock */ > ARG_PTR_TO_SPIN_LOCK, /* pointer to bpf_spin_lock */ > ARG_PTR_TO_SOCK_COMMON, /* pointer to sock_common */ > + ARG_PTR_TO_TCP_SOCK, /* pointer to tcp_sock */ > }; > > /* type of values returned from helper functions */ > diff --git a/include/linux/filter.h b/include/linux/filter.h > index f32b3eca5a04..c6e878bdc5a6 100644 > --- a/include/linux/filter.h > +++ b/include/linux/filter.h > @@ -510,7 +510,8 @@ struct bpf_prog { > blinded:1, /* Was blinded */ > is_func:1, /* program is a bpf function */ > kprobe_override:1, /* Do we override a kprobe? */ > - has_callchain_buf:1; /* callchain buffer allocated? */ > + has_callchain_buf:1, /* callchain buffer allocated? */ > + enforce_expected_attach_type:1; /* Enforce expected_attach_type checking at attach time */ > enum bpf_prog_type type; /* Type of BPF program */ > enum bpf_attach_type expected_attach_type; /* For some prog types */ > u32 len; /* Number of filter blocks */ > diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h > index bcdd2474eee7..95b5058fa945 100644 > --- a/include/uapi/linux/bpf.h > +++ b/include/uapi/linux/bpf.h > @@ -2359,6 +2359,12 @@ union bpf_attr { > * Return > * A **struct bpf_tcp_sock** pointer on success, or NULL in > * case of failure. > + * > + * int bpf_tcp_enter_cwr(struct bpf_tcp_sock *tp) > + * Description > + * Make a tcp_sock enter CWR state. > + * Return > + * 0 on success, or a negative error in case of failure. > */ > #define __BPF_FUNC_MAPPER(FN) \ > FN(unspec), \ > @@ -2457,7 +2463,8 @@ union bpf_attr { > FN(spin_lock), \ > FN(spin_unlock), \ > FN(sk_fullsock), \ > - FN(tcp_sock), > + FN(tcp_sock), \ > + FN(tcp_enter_cwr), > > /* integer value in 'imm' field of BPF_CALL instruction selects which helper > * function eBPF program intends to call > diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c > index ec7c552af76b..9a478f2875cd 100644 > --- a/kernel/bpf/syscall.c > +++ b/kernel/bpf/syscall.c > @@ -1482,6 +1482,14 @@ bpf_prog_load_check_attach_type(enum bpf_prog_type prog_type, > default: > return -EINVAL; > } > + case BPF_PROG_TYPE_CGROUP_SKB: > + switch (expected_attach_type) { > + case BPF_CGROUP_INET_INGRESS: > + case BPF_CGROUP_INET_EGRESS: > + return 0; > + default: > + return -EINVAL; > + } > default: > return 0; > } > @@ -1725,6 +1733,10 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog, > case BPF_PROG_TYPE_CGROUP_SOCK: > case BPF_PROG_TYPE_CGROUP_SOCK_ADDR: > return attach_type == prog->expected_attach_type ? 0 : -EINVAL; > + case BPF_PROG_TYPE_CGROUP_SKB: > + return prog->enforce_expected_attach_type && > + prog->expected_attach_type != attach_type ? > + -EINVAL : 0; > default: > return 0; > } > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c > index 1b9496c41383..95fb385c6f3c 100644 > --- a/kernel/bpf/verifier.c > +++ b/kernel/bpf/verifier.c > @@ -2424,6 +2424,10 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 regno, > return -EFAULT; > } > meta->ptr_id = reg->id; > + } else if (arg_type == ARG_PTR_TO_TCP_SOCK) { > + expected_type = PTR_TO_TCP_SOCK; > + if (type != expected_type) > + goto err_type; > } else if (arg_type == ARG_PTR_TO_SPIN_LOCK) { > if (meta->func_id == BPF_FUNC_spin_lock) { > if (process_spin_lock(env, regno, true)) > diff --git a/net/core/filter.c b/net/core/filter.c > index 97916eedfe69..ca57ef25279c 100644 > --- a/net/core/filter.c > +++ b/net/core/filter.c > @@ -5426,6 +5426,24 @@ static const struct bpf_func_proto bpf_tcp_sock_proto = { > .arg1_type = ARG_PTR_TO_SOCK_COMMON, > }; > > +BPF_CALL_1(bpf_tcp_enter_cwr, struct tcp_sock *, tp) > +{ > + struct sock *sk = (struct sock *)tp; > + > + if (sk->sk_state == TCP_ESTABLISHED) { > + tcp_enter_cwr(sk); > + return 0; > + } > + > + return -EINVAL; > +} > + > +static const struct bpf_func_proto bpf_tcp_enter_cwr_proto = { > + .func = bpf_tcp_enter_cwr, > + .gpl_only = false, > + .ret_type = RET_INTEGER, > + .arg1_type = ARG_PTR_TO_TCP_SOCK, > +}; > #endif /* CONFIG_INET */ > > bool bpf_helper_changes_pkt_data(void *func) > @@ -5585,6 +5603,13 @@ cg_skb_func_proto(enum bpf_func_id func_id, struct bpf_prog *prog) > #ifdef CONFIG_INET > case BPF_FUNC_tcp_sock: > return &bpf_tcp_sock_proto; [...] > + case BPF_FUNC_tcp_enter_cwr: > + if (prog->expected_attach_type == BPF_CGROUP_INET_EGRESS) { > + prog->enforce_expected_attach_type = 1; > + return &bpf_tcp_enter_cwr_proto; Instead of this back and forth with enforce_expected_attach_type, can we just do here: if (prog->expected_attach_type == BPF_CGROUP_INET_EGRESS) return &bpf_tcp_enter_cwr_proto; else return null; Wouldn't it have the same effect? > + } else { > + return NULL; > + } > #endif > default: > return sk_filter_func_proto(func_id, prog); > -- > 2.17.1 >
On Mon, Feb 25, 2019 at 03:14:38PM -0800, Stanislav Fomichev wrote: [ ... ] > > > > To ensure it is only called from BPF_CGROUP_INET_EGRESS, the > > attr->expected_attach_type must be specified as BPF_CGROUP_INET_EGRESS > > during load time if the prog uses this new helper. > > The newly added prog->enforce_expected_attach_type bit will also be set > > if this new helper is used. This bit is for backward compatibility reason > > because currently prog->expected_attach_type has been ignored in > > BPF_PROG_TYPE_CGROUP_SKB. During attach time, > > prog->expected_attach_type is only enforced if the > > prog->enforce_expected_attach_type bit is set. > > i.e. prog->expected_attach_type is only enforced if this new helper > > is used by the prog. > > [ ... ] > > @@ -1725,6 +1733,10 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog, > > case BPF_PROG_TYPE_CGROUP_SOCK: > > case BPF_PROG_TYPE_CGROUP_SOCK_ADDR: > > return attach_type == prog->expected_attach_type ? 0 : -EINVAL; > > + case BPF_PROG_TYPE_CGROUP_SKB: > > + return prog->enforce_expected_attach_type && > > + prog->expected_attach_type != attach_type ? > > + -EINVAL : 0; > > default: > > return 0; > > } [ ... ] > > diff --git a/net/core/filter.c b/net/core/filter.c > > index 97916eedfe69..ca57ef25279c 100644 > > --- a/net/core/filter.c > > +++ b/net/core/filter.c > > @@ -5426,6 +5426,24 @@ static const struct bpf_func_proto bpf_tcp_sock_proto = { > > .arg1_type = ARG_PTR_TO_SOCK_COMMON, > > }; > > > > +BPF_CALL_1(bpf_tcp_enter_cwr, struct tcp_sock *, tp) > > +{ > > + struct sock *sk = (struct sock *)tp; > > + > > + if (sk->sk_state == TCP_ESTABLISHED) { > > + tcp_enter_cwr(sk); > > + return 0; > > + } > > + > > + return -EINVAL; > > +} > > + > > +static const struct bpf_func_proto bpf_tcp_enter_cwr_proto = { > > + .func = bpf_tcp_enter_cwr, > > + .gpl_only = false, > > + .ret_type = RET_INTEGER, > > + .arg1_type = ARG_PTR_TO_TCP_SOCK, > > +}; > > #endif /* CONFIG_INET */ > > > > bool bpf_helper_changes_pkt_data(void *func) > > @@ -5585,6 +5603,13 @@ cg_skb_func_proto(enum bpf_func_id func_id, struct bpf_prog *prog) > > #ifdef CONFIG_INET > > case BPF_FUNC_tcp_sock: > > return &bpf_tcp_sock_proto; > > [...] > > + case BPF_FUNC_tcp_enter_cwr: > > + if (prog->expected_attach_type == BPF_CGROUP_INET_EGRESS) { > > + prog->enforce_expected_attach_type = 1; > > + return &bpf_tcp_enter_cwr_proto; > Instead of this back and forth with enforce_expected_attach_type, can we > just do here: > > if (prog->expected_attach_type == BPF_CGROUP_INET_EGRESS) > return &bpf_tcp_enter_cwr_proto; > else > return null; > > Wouldn't it have the same effect? The attr->expected_attach_type is currently ignored (i.e. not checked) during the bpf load time. How to avoid breaking backward compatibility without selectively enforcing prog->expected_attach_type during attach time?
On 02/26, Martin Lau wrote: > On Mon, Feb 25, 2019 at 03:14:38PM -0800, Stanislav Fomichev wrote: > [ ... ] > > > > > > > To ensure it is only called from BPF_CGROUP_INET_EGRESS, the > > > attr->expected_attach_type must be specified as BPF_CGROUP_INET_EGRESS > > > during load time if the prog uses this new helper. > > > The newly added prog->enforce_expected_attach_type bit will also be set > > > if this new helper is used. This bit is for backward compatibility reason > > > because currently prog->expected_attach_type has been ignored in > > > BPF_PROG_TYPE_CGROUP_SKB. During attach time, > > > prog->expected_attach_type is only enforced if the > > > prog->enforce_expected_attach_type bit is set. > > > i.e. prog->expected_attach_type is only enforced if this new helper > > > is used by the prog. > > > > [ ... ] > > > > @@ -1725,6 +1733,10 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog, > > > case BPF_PROG_TYPE_CGROUP_SOCK: > > > case BPF_PROG_TYPE_CGROUP_SOCK_ADDR: > > > return attach_type == prog->expected_attach_type ? 0 : -EINVAL; > > > + case BPF_PROG_TYPE_CGROUP_SKB: > > > + return prog->enforce_expected_attach_type && > > > + prog->expected_attach_type != attach_type ? > > > + -EINVAL : 0; > > > default: > > > return 0; > > > } > [ ... ] > > > > diff --git a/net/core/filter.c b/net/core/filter.c > > > index 97916eedfe69..ca57ef25279c 100644 > > > --- a/net/core/filter.c > > > +++ b/net/core/filter.c > > > @@ -5426,6 +5426,24 @@ static const struct bpf_func_proto bpf_tcp_sock_proto = { > > > .arg1_type = ARG_PTR_TO_SOCK_COMMON, > > > }; > > > > > > +BPF_CALL_1(bpf_tcp_enter_cwr, struct tcp_sock *, tp) > > > +{ > > > + struct sock *sk = (struct sock *)tp; > > > + > > > + if (sk->sk_state == TCP_ESTABLISHED) { > > > + tcp_enter_cwr(sk); > > > + return 0; > > > + } > > > + > > > + return -EINVAL; > > > +} > > > + > > > +static const struct bpf_func_proto bpf_tcp_enter_cwr_proto = { > > > + .func = bpf_tcp_enter_cwr, > > > + .gpl_only = false, > > > + .ret_type = RET_INTEGER, > > > + .arg1_type = ARG_PTR_TO_TCP_SOCK, > > > +}; > > > #endif /* CONFIG_INET */ > > > > > > bool bpf_helper_changes_pkt_data(void *func) > > > @@ -5585,6 +5603,13 @@ cg_skb_func_proto(enum bpf_func_id func_id, struct bpf_prog *prog) > > > #ifdef CONFIG_INET > > > case BPF_FUNC_tcp_sock: > > > return &bpf_tcp_sock_proto; > > > > [...] > > > + case BPF_FUNC_tcp_enter_cwr: > > > + if (prog->expected_attach_type == BPF_CGROUP_INET_EGRESS) { > > > + prog->enforce_expected_attach_type = 1; > > > + return &bpf_tcp_enter_cwr_proto; > > Instead of this back and forth with enforce_expected_attach_type, can we > > just do here: > > > > if (prog->expected_attach_type == BPF_CGROUP_INET_EGRESS) > > return &bpf_tcp_enter_cwr_proto; > > else > > return null; > > > > Wouldn't it have the same effect? > The attr->expected_attach_type is currently ignored (i.e. not checked) > during the bpf load time. But nothing stops you form checking prog->expected_attach_type in the cg_skb_func_proto, right? That is done at the time of loading. So depending on prog->expected_attach_type just return null or non-null and the verifier will take care of the rest. Then, at attach time just make sure we are attaching it to the expected_attach_type. We also should not have any existing use cases for BPF_FUNC_tcp_enter_cwr I suppose. In other words, why something like below won't work? Am I missing something? --- diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index b155cd17c1bd..86dc7cd00f34 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -1678,6 +1678,10 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog, case BPF_PROG_TYPE_CGROUP_SOCK: case BPF_PROG_TYPE_CGROUP_SOCK_ADDR: return attach_type == prog->expected_attach_type ? 0 : -EINVAL; + case BPF_PROG_TYPE_CGROUP_SKB: + if (prog->expected_attach_type) + return attach_type == prog->expected_attach_type ? 0 : -EINVAL; + return 0; default: return 0; } diff --git a/net/core/filter.c b/net/core/filter.c index 7559d6835ecb..56f70468fc7a 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -5396,6 +5396,11 @@ cg_skb_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) switch (func_id) { case BPF_FUNC_get_local_storage: return &bpf_get_local_storage_proto; + case BPF_FUNC_tcp_enter_cwr: + if (prog->expected_attach_type == BPF_CGROUP_INET_EGRESS) + return &bpf_tcp_enter_cwr_proto; + else + return NULL; default: return sk_filter_func_proto(func_id, prog); } > > How to avoid breaking backward compatibility without selectively > enforcing prog->expected_attach_type during attach time?
diff --git a/include/linux/bpf.h b/include/linux/bpf.h index d5ba2fc01af3..2d54ba7cf9dd 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -195,6 +195,7 @@ enum bpf_arg_type { ARG_PTR_TO_SOCKET, /* pointer to bpf_sock */ ARG_PTR_TO_SPIN_LOCK, /* pointer to bpf_spin_lock */ ARG_PTR_TO_SOCK_COMMON, /* pointer to sock_common */ + ARG_PTR_TO_TCP_SOCK, /* pointer to tcp_sock */ }; /* type of values returned from helper functions */ diff --git a/include/linux/filter.h b/include/linux/filter.h index f32b3eca5a04..c6e878bdc5a6 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -510,7 +510,8 @@ struct bpf_prog { blinded:1, /* Was blinded */ is_func:1, /* program is a bpf function */ kprobe_override:1, /* Do we override a kprobe? */ - has_callchain_buf:1; /* callchain buffer allocated? */ + has_callchain_buf:1, /* callchain buffer allocated? */ + enforce_expected_attach_type:1; /* Enforce expected_attach_type checking at attach time */ enum bpf_prog_type type; /* Type of BPF program */ enum bpf_attach_type expected_attach_type; /* For some prog types */ u32 len; /* Number of filter blocks */ diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index bcdd2474eee7..95b5058fa945 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -2359,6 +2359,12 @@ union bpf_attr { * Return * A **struct bpf_tcp_sock** pointer on success, or NULL in * case of failure. + * + * int bpf_tcp_enter_cwr(struct bpf_tcp_sock *tp) + * Description + * Make a tcp_sock enter CWR state. + * Return + * 0 on success, or a negative error in case of failure. */ #define __BPF_FUNC_MAPPER(FN) \ FN(unspec), \ @@ -2457,7 +2463,8 @@ union bpf_attr { FN(spin_lock), \ FN(spin_unlock), \ FN(sk_fullsock), \ - FN(tcp_sock), + FN(tcp_sock), \ + FN(tcp_enter_cwr), /* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index ec7c552af76b..9a478f2875cd 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -1482,6 +1482,14 @@ bpf_prog_load_check_attach_type(enum bpf_prog_type prog_type, default: return -EINVAL; } + case BPF_PROG_TYPE_CGROUP_SKB: + switch (expected_attach_type) { + case BPF_CGROUP_INET_INGRESS: + case BPF_CGROUP_INET_EGRESS: + return 0; + default: + return -EINVAL; + } default: return 0; } @@ -1725,6 +1733,10 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog, case BPF_PROG_TYPE_CGROUP_SOCK: case BPF_PROG_TYPE_CGROUP_SOCK_ADDR: return attach_type == prog->expected_attach_type ? 0 : -EINVAL; + case BPF_PROG_TYPE_CGROUP_SKB: + return prog->enforce_expected_attach_type && + prog->expected_attach_type != attach_type ? + -EINVAL : 0; default: return 0; } diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 1b9496c41383..95fb385c6f3c 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -2424,6 +2424,10 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 regno, return -EFAULT; } meta->ptr_id = reg->id; + } else if (arg_type == ARG_PTR_TO_TCP_SOCK) { + expected_type = PTR_TO_TCP_SOCK; + if (type != expected_type) + goto err_type; } else if (arg_type == ARG_PTR_TO_SPIN_LOCK) { if (meta->func_id == BPF_FUNC_spin_lock) { if (process_spin_lock(env, regno, true)) diff --git a/net/core/filter.c b/net/core/filter.c index 97916eedfe69..ca57ef25279c 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -5426,6 +5426,24 @@ static const struct bpf_func_proto bpf_tcp_sock_proto = { .arg1_type = ARG_PTR_TO_SOCK_COMMON, }; +BPF_CALL_1(bpf_tcp_enter_cwr, struct tcp_sock *, tp) +{ + struct sock *sk = (struct sock *)tp; + + if (sk->sk_state == TCP_ESTABLISHED) { + tcp_enter_cwr(sk); + return 0; + } + + return -EINVAL; +} + +static const struct bpf_func_proto bpf_tcp_enter_cwr_proto = { + .func = bpf_tcp_enter_cwr, + .gpl_only = false, + .ret_type = RET_INTEGER, + .arg1_type = ARG_PTR_TO_TCP_SOCK, +}; #endif /* CONFIG_INET */ bool bpf_helper_changes_pkt_data(void *func) @@ -5585,6 +5603,13 @@ cg_skb_func_proto(enum bpf_func_id func_id, struct bpf_prog *prog) #ifdef CONFIG_INET case BPF_FUNC_tcp_sock: return &bpf_tcp_sock_proto; + case BPF_FUNC_tcp_enter_cwr: + if (prog->expected_attach_type == BPF_CGROUP_INET_EGRESS) { + prog->enforce_expected_attach_type = 1; + return &bpf_tcp_enter_cwr_proto; + } else { + return NULL; + } #endif default: return sk_filter_func_proto(func_id, prog);