| Message ID | 158824576377.2172139.12065840702900641458.stgit@firesoul |
|---|---|
| State | Changes Requested |
| Delegated to: | BPF Maintainers |
| Headers | show |
| Series | [net-next,v2,01/33] xdp: add frame size to xdp_buff | expand |
On 4/30/2020 2:22 PM, Jesper Dangaard Brouer wrote: > The mlx5 driver have multiple memory models, which are also changed > according to whether a XDP bpf_prog is attached. > > The 'rx_striding_rq' setting is adjusted via ethtool priv-flags e.g.: > # ethtool --set-priv-flags mlx5p2 rx_striding_rq off > > On the general case with 4K page_size and regular MTU packet, then > the frame_sz is 2048 and 4096 when XDP is enabled, in both modes. > > The info on the given frame size is stored differently depending on the > RQ-mode and encoded in a union in struct mlx5e_rq union wqe/mpwqe. > In rx striding mode rq->mpwqe.log_stride_sz is either 11 or 12, which > corresponds to 2048 or 4096 (MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ). > In non-striding mode (MLX5_WQ_TYPE_CYCLIC) the frag_stride is stored > in rq->wqe.info.arr[0].frag_stride, for the first fragment, which is > what the XDP case cares about. > > To reduce effect on fast-path, this patch determine the frame_sz at > setup time, to avoid determining the memory model runtime. Variable > is named first_frame_sz to make it clear that this is only the frame > size of the first fragment. > > This mlx5 driver does a DMA-sync on XDP_TX action, but grow is safe > as it have done a DMA-map on the entire PAGE_SIZE. The driver also > already does a XDP length check against sq->hw_mtu on the possible > XDP xmit paths mlx5e_xmit_xdp_frame() + mlx5e_xmit_xdp_frame_mpwqe(). > > V2: Fix that frag_size need to be recalc before creating SKB. > > Cc: Tariq Toukan <tariqt@mellanox.com> > Cc: Saeed Mahameed <saeedm@mellanox.com> > Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com> > --- > drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 + > drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c | 1 + > drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 6 ++++++ > drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 2 ++ > 4 files changed, 10 insertions(+) > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h > index 23701c0e36ec..ba6a0ee297c6 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h > @@ -652,6 +652,7 @@ struct mlx5e_rq { > struct { > u16 umem_headroom; > u16 headroom; > + u32 first_frame_sz; > u8 map_dir; /* dma map direction */ > } buff; > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c > index f049e0ac308a..b63abaf51253 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c > @@ -137,6 +137,7 @@ bool mlx5e_xdp_handle(struct mlx5e_rq *rq, struct mlx5e_dma_info *di, > if (xsk) > xdp.handle = di->xsk.handle; > xdp.rxq = &rq->xdp_rxq; > + xdp.frame_sz = rq->buff.first_frame_sz; > > act = bpf_prog_run_xdp(prog, &xdp); > if (xsk) { > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > index 47396f1b02f4..1d04ed3feead 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > @@ -462,6 +462,8 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c, > rq->mpwqe.num_strides = > BIT(mlx5e_mpwqe_get_log_num_strides(mdev, params, xsk)); > > + rq->buff.first_frame_sz = (1 << rq->mpwqe.log_stride_sz); > + > err = mlx5e_create_rq_umr_mkey(mdev, rq); > if (err) > goto err_rq_wq_destroy; > @@ -485,6 +487,8 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c, > num_xsk_frames = wq_sz << rq->wqe.info.log_num_frags; > > rq->wqe.info = rqp->frags_info; > + rq->buff.first_frame_sz = rq->wqe.info.arr[0].frag_stride; > + > rq->wqe.frags = > kvzalloc_node(array_size(sizeof(*rq->wqe.frags), > (wq_sz << rq->wqe.info.log_num_frags)), > @@ -522,6 +526,8 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c, > } > > if (xsk) { > + rq->buff.first_frame_sz = xsk_umem_xdp_frame_sz(umem); > + > err = mlx5e_xsk_resize_reuseq(umem, num_xsk_frames); > if (unlikely(err)) { > mlx5_core_err(mdev, "Unable to allocate the Reuse Ring for %u frames\n", > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c > index e2beb89c1832..04671ed977a5 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c > @@ -1084,6 +1084,7 @@ mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe, > if (consumed) > return NULL; /* page/packet was consumed by XDP */ > > + frag_size = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt); This is a re-calculation of frag_size, using the exact same command used earlier in this function, but with a newer value of rx_headroom. This wasn't part of the previous patchset. I understand the need. However, this code repetition looks weird and non-optimal to me. I think we can come up with something better. > skb = mlx5e_build_linear_skb(rq, va, frag_size, rx_headroom, cqe_bcnt); > if (unlikely(!skb)) > return NULL; > @@ -1385,6 +1386,7 @@ mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, > return NULL; /* page/packet was consumed by XDP */ > } > > + frag_size = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt32); Same here. > skb = mlx5e_build_linear_skb(rq, va, frag_size, rx_headroom, cqe_bcnt32); > if (unlikely(!skb)) > return NULL; > > My suggetion is: Pass &frag_size to mlx5e_xdp_handle(), and update it within it, just next to the update of rx_headroom. All the needed information is there: the new rx_headroom, and cqe_bcnt. Thanks, Tariq
On 4/30/2020 8:07 PM, Tariq Toukan wrote: > > > On 4/30/2020 2:22 PM, Jesper Dangaard Brouer wrote: >> The mlx5 driver have multiple memory models, which are also changed >> according to whether a XDP bpf_prog is attached. >> >> The 'rx_striding_rq' setting is adjusted via ethtool priv-flags e.g.: >> # ethtool --set-priv-flags mlx5p2 rx_striding_rq off >> >> On the general case with 4K page_size and regular MTU packet, then >> the frame_sz is 2048 and 4096 when XDP is enabled, in both modes. >> >> The info on the given frame size is stored differently depending on the >> RQ-mode and encoded in a union in struct mlx5e_rq union wqe/mpwqe. >> In rx striding mode rq->mpwqe.log_stride_sz is either 11 or 12, which >> corresponds to 2048 or 4096 (MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ). >> In non-striding mode (MLX5_WQ_TYPE_CYCLIC) the frag_stride is stored >> in rq->wqe.info.arr[0].frag_stride, for the first fragment, which is >> what the XDP case cares about. >> >> To reduce effect on fast-path, this patch determine the frame_sz at >> setup time, to avoid determining the memory model runtime. Variable >> is named first_frame_sz to make it clear that this is only the frame >> size of the first fragment. >> >> This mlx5 driver does a DMA-sync on XDP_TX action, but grow is safe >> as it have done a DMA-map on the entire PAGE_SIZE. The driver also >> already does a XDP length check against sq->hw_mtu on the possible >> XDP xmit paths mlx5e_xmit_xdp_frame() + mlx5e_xmit_xdp_frame_mpwqe(). >> >> V2: Fix that frag_size need to be recalc before creating SKB. >> >> Cc: Tariq Toukan <tariqt@mellanox.com> >> Cc: Saeed Mahameed <saeedm@mellanox.com> >> Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com> >> --- >> drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 + >> drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c | 1 + >> drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 6 ++++++ >> drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 2 ++ >> 4 files changed, 10 insertions(+) >> >> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h >> b/drivers/net/ethernet/mellanox/mlx5/core/en.h >> index 23701c0e36ec..ba6a0ee297c6 100644 >> --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h >> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h >> @@ -652,6 +652,7 @@ struct mlx5e_rq { >> struct { >> u16 umem_headroom; >> u16 headroom; >> + u32 first_frame_sz; I also think that a better name would be: frame0_sz, or frag0_sz. Thanks.
On Thu, 30 Apr 2020 20:12:11 +0300 Tariq Toukan <tariqt@mellanox.com> wrote: > >> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h > >> b/drivers/net/ethernet/mellanox/mlx5/core/en.h > >> index 23701c0e36ec..ba6a0ee297c6 100644 > >> --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h > >> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h > >> @@ -652,6 +652,7 @@ struct mlx5e_rq { > >> struct { > >> u16 umem_headroom; > >> u16 headroom; > >> + u32 first_frame_sz; > > I also think that a better name would be: frame0_sz, or frag0_sz. You do realize that the name "first_frame_sz" was your suggestion last time... Now you give me two options, can please select one of them so I can update the patch for a V3 with that?
On Thu, 30 Apr 2020 20:07:43 +0300 Tariq Toukan <tariqt@mellanox.com> wrote: > On 4/30/2020 2:22 PM, Jesper Dangaard Brouer wrote: > > The mlx5 driver have multiple memory models, which are also changed > > according to whether a XDP bpf_prog is attached. > > > > The 'rx_striding_rq' setting is adjusted via ethtool priv-flags e.g.: > > # ethtool --set-priv-flags mlx5p2 rx_striding_rq off > > > > On the general case with 4K page_size and regular MTU packet, then > > the frame_sz is 2048 and 4096 when XDP is enabled, in both modes. > > > > The info on the given frame size is stored differently depending on the > > RQ-mode and encoded in a union in struct mlx5e_rq union wqe/mpwqe. > > In rx striding mode rq->mpwqe.log_stride_sz is either 11 or 12, which > > corresponds to 2048 or 4096 (MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ). > > In non-striding mode (MLX5_WQ_TYPE_CYCLIC) the frag_stride is stored > > in rq->wqe.info.arr[0].frag_stride, for the first fragment, which is > > what the XDP case cares about. > > > > To reduce effect on fast-path, this patch determine the frame_sz at > > setup time, to avoid determining the memory model runtime. Variable > > is named first_frame_sz to make it clear that this is only the frame > > size of the first fragment. > > > > This mlx5 driver does a DMA-sync on XDP_TX action, but grow is safe > > as it have done a DMA-map on the entire PAGE_SIZE. The driver also > > already does a XDP length check against sq->hw_mtu on the possible > > XDP xmit paths mlx5e_xmit_xdp_frame() + mlx5e_xmit_xdp_frame_mpwqe(). > > > > V2: Fix that frag_size need to be recalc before creating SKB. > > > > Cc: Tariq Toukan <tariqt@mellanox.com> > > Cc: Saeed Mahameed <saeedm@mellanox.com> > > Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com> > > --- > > drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 + > > drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c | 1 + > > drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 6 ++++++ > > drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 2 ++ > > 4 files changed, 10 insertions(+) > > [... cut ...] > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c > > index e2beb89c1832..04671ed977a5 100644 > > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c > > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c > > @@ -1084,6 +1084,7 @@ mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe, > > if (consumed) > > return NULL; /* page/packet was consumed by XDP */ > > > > + frag_size = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt); > > This is a re-calculation of frag_size, using the exact same command used > earlier in this function, but with a newer value of rx_headroom. > This wasn't part of the previous patchset. I understand the need. Yes, kernel will crash without this change. > However, this code repetition looks weird and non-optimal to me. I think > we can come up with something better. > > > skb = mlx5e_build_linear_skb(rq, va, frag_size, rx_headroom, cqe_bcnt); > > if (unlikely(!skb)) > > return NULL; > > @@ -1385,6 +1386,7 @@ mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, > > return NULL; /* page/packet was consumed by XDP */ > > } > > > > + frag_size = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt32); > > Same here. > > > skb = mlx5e_build_linear_skb(rq, va, frag_size, rx_headroom, cqe_bcnt32); > > if (unlikely(!skb)) > > return NULL; > > > > > > My suggetion is: > Pass &frag_size to mlx5e_xdp_handle(), and update it within it, just > next to the update of rx_headroom. > All the needed information is there: the new rx_headroom, and cqe_bcnt. First of all, passing yet-another argument to mlx5e_xdp_handle(), also looks weird, and is on the brink of becoming a performance issue, as on x86_64 you can pass max 6 arguments in registers before they get pushed on the stack. Adding this would be the 7th argument. Second the MLX5_SKB_FRAG_SZ() calculation is also weird, because it does not provide any tailroom in the packet, I guess it is for supporting another memory mode, as in case XDP is activated there are plenty of tailroom. I though about increasing the frag_size, but I choose not to, because then this patch would change the driver behavior beyond adding frame_sz for XDP.
On Fri, 1 May 2020 14:32:32 +0200 Jesper Dangaard Brouer <brouer@redhat.com> wrote: > On Thu, 30 Apr 2020 20:12:11 +0300 > Tariq Toukan <tariqt@mellanox.com> wrote: > > > >> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h > > >> b/drivers/net/ethernet/mellanox/mlx5/core/en.h > > >> index 23701c0e36ec..ba6a0ee297c6 100644 > > >> --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h > > >> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h > > >> @@ -652,6 +652,7 @@ struct mlx5e_rq { > > >> struct { > > >> u16 umem_headroom; > > >> u16 headroom; > > >> + u32 first_frame_sz; > > > > I also think that a better name would be: frame0_sz, or frag0_sz. > > You do realize that the name "first_frame_sz" was your suggestion last > time... Now you give me two options, can please select one of them so I > can update the patch for a V3 with that? As I've not gotten any feedback from you, I'm going to choose your first suggestion "frame0_sz" and update the patch with that...
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h index 23701c0e36ec..ba6a0ee297c6 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -652,6 +652,7 @@ struct mlx5e_rq { struct { u16 umem_headroom; u16 headroom; + u32 first_frame_sz; u8 map_dir; /* dma map direction */ } buff; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c index f049e0ac308a..b63abaf51253 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c @@ -137,6 +137,7 @@ bool mlx5e_xdp_handle(struct mlx5e_rq *rq, struct mlx5e_dma_info *di, if (xsk) xdp.handle = di->xsk.handle; xdp.rxq = &rq->xdp_rxq; + xdp.frame_sz = rq->buff.first_frame_sz; act = bpf_prog_run_xdp(prog, &xdp); if (xsk) { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 47396f1b02f4..1d04ed3feead 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -462,6 +462,8 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c, rq->mpwqe.num_strides = BIT(mlx5e_mpwqe_get_log_num_strides(mdev, params, xsk)); + rq->buff.first_frame_sz = (1 << rq->mpwqe.log_stride_sz); + err = mlx5e_create_rq_umr_mkey(mdev, rq); if (err) goto err_rq_wq_destroy; @@ -485,6 +487,8 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c, num_xsk_frames = wq_sz << rq->wqe.info.log_num_frags; rq->wqe.info = rqp->frags_info; + rq->buff.first_frame_sz = rq->wqe.info.arr[0].frag_stride; + rq->wqe.frags = kvzalloc_node(array_size(sizeof(*rq->wqe.frags), (wq_sz << rq->wqe.info.log_num_frags)), @@ -522,6 +526,8 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c, } if (xsk) { + rq->buff.first_frame_sz = xsk_umem_xdp_frame_sz(umem); + err = mlx5e_xsk_resize_reuseq(umem, num_xsk_frames); if (unlikely(err)) { mlx5_core_err(mdev, "Unable to allocate the Reuse Ring for %u frames\n", diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c index e2beb89c1832..04671ed977a5 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -1084,6 +1084,7 @@ mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe, if (consumed) return NULL; /* page/packet was consumed by XDP */ + frag_size = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt); skb = mlx5e_build_linear_skb(rq, va, frag_size, rx_headroom, cqe_bcnt); if (unlikely(!skb)) return NULL; @@ -1385,6 +1386,7 @@ mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, return NULL; /* page/packet was consumed by XDP */ } + frag_size = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt32); skb = mlx5e_build_linear_skb(rq, va, frag_size, rx_headroom, cqe_bcnt32); if (unlikely(!skb)) return NULL;
The mlx5 driver have multiple memory models, which are also changed according to whether a XDP bpf_prog is attached. The 'rx_striding_rq' setting is adjusted via ethtool priv-flags e.g.: # ethtool --set-priv-flags mlx5p2 rx_striding_rq off On the general case with 4K page_size and regular MTU packet, then the frame_sz is 2048 and 4096 when XDP is enabled, in both modes. The info on the given frame size is stored differently depending on the RQ-mode and encoded in a union in struct mlx5e_rq union wqe/mpwqe. In rx striding mode rq->mpwqe.log_stride_sz is either 11 or 12, which corresponds to 2048 or 4096 (MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ). In non-striding mode (MLX5_WQ_TYPE_CYCLIC) the frag_stride is stored in rq->wqe.info.arr[0].frag_stride, for the first fragment, which is what the XDP case cares about. To reduce effect on fast-path, this patch determine the frame_sz at setup time, to avoid determining the memory model runtime. Variable is named first_frame_sz to make it clear that this is only the frame size of the first fragment. This mlx5 driver does a DMA-sync on XDP_TX action, but grow is safe as it have done a DMA-map on the entire PAGE_SIZE. The driver also already does a XDP length check against sq->hw_mtu on the possible XDP xmit paths mlx5e_xmit_xdp_frame() + mlx5e_xmit_xdp_frame_mpwqe(). V2: Fix that frag_size need to be recalc before creating SKB. Cc: Tariq Toukan <tariqt@mellanox.com> Cc: Saeed Mahameed <saeedm@mellanox.com> Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com> --- drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 + drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c | 1 + drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 6 ++++++ drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 2 ++ 4 files changed, 10 insertions(+)