| Message ID | 4F464063.1010001@mellanox.co.il |
|---|---|
| State | Rejected, archived |
| Delegated to: | David Miller |
| Headers | show |
From: Yevgeny Petrilin <yevgenyp@mellanox.co.il> Date: Thu, 23 Feb 2012 15:34:27 +0200 > No need to ask for completion for every packet being sent. > So the method is to ask for a completion every 16 packets, > or when the queue is about to be full. > > Signed-off-by: Yevgeny Petrilin <yevgenyp@mellanox.co.il> You absolutely cannot do this, you must signal completion and free up TX queue packets in a finite amount of time. This means that if you suddenly stop getting new packets to send you must still free up all the pending TX SKBs even if no more packets are given to the driver. Does your hardware unconditionally give a TX completion interrupt when the TX queue empties completely? If not, then you cannot make the change contained in this patch. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, 2012-02-23 at 14:44 -0500, David Miller wrote: > From: Yevgeny Petrilin <yevgenyp@mellanox.co.il> > Date: Thu, 23 Feb 2012 15:34:27 +0200 > > > No need to ask for completion for every packet being sent. > > So the method is to ask for a completion every 16 packets, > > or when the queue is about to be full. > > > > Signed-off-by: Yevgeny Petrilin <yevgenyp@mellanox.co.il> > > You absolutely cannot do this, you must signal completion and free up > TX queue packets in a finite amount of time. Really, sfc has been doing this forever. Ben. > This means that if you suddenly stop getting new packets to send > you must still free up all the pending TX SKBs even if no more > packets are given to the driver. > > Does your hardware unconditionally give a TX completion interrupt when > the TX queue empties completely? If not, then you cannot make the > change contained in this patch. > -- > To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, 2012-02-23 at 21:13 +0000, Ben Hutchings wrote: > On Thu, 2012-02-23 at 14:44 -0500, David Miller wrote: > > From: Yevgeny Petrilin <yevgenyp@mellanox.co.il> > > Date: Thu, 23 Feb 2012 15:34:27 +0200 > > > > > No need to ask for completion for every packet being sent. > > > So the method is to ask for a completion every 16 packets, > > > or when the queue is about to be full. > > > > > > Signed-off-by: Yevgeny Petrilin <yevgenyp@mellanox.co.il> > > > > You absolutely cannot do this, you must signal completion and free up > > TX queue packets in a finite amount of time. > > Really, sfc has been doing this forever. > > Ben. > > > This means that if you suddenly stop getting new packets to send > > you must still free up all the pending TX SKBs even if no more > > packets are given to the driver. > > > > Does your hardware unconditionally give a TX completion interrupt when > > the TX queue empties completely? If not, then you cannot make the > > change contained in this patch. And it's pretty obvious the hardware has to also send a completion when the queue is empty. I hope. Ben.
From: Ben Hutchings <bhutchings@solarflare.com> Date: Thu, 23 Feb 2012 21:13:56 +0000 > On Thu, 2012-02-23 at 14:44 -0500, David Miller wrote: >> From: Yevgeny Petrilin <yevgenyp@mellanox.co.il> >> Date: Thu, 23 Feb 2012 15:34:27 +0200 >> >> > No need to ask for completion for every packet being sent. >> > So the method is to ask for a completion every 16 packets, >> > or when the queue is about to be full. >> > >> > Signed-off-by: Yevgeny Petrilin <yevgenyp@mellanox.co.il> >> >> You absolutely cannot do this, you must signal completion and free up >> TX queue packets in a finite amount of time. > > Really, sfc has been doing this forever. You have to stop, because otherwise sockets can hang since those packets can hold a reference to the socket so they must be released in a finite amount of time regardless of network traffic. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Ben Hutchings <bhutchings@solarflare.com> Date: Thu, 23 Feb 2012 21:26:20 +0000 > And it's pretty obvious the hardware has to also send a completion when > the queue is empty. I hope. Right and we know of several chips that don't. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_tx.c b/drivers/net/ethernet/mellanox/mlx4/en_tx.c index ff32505..e452bba 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_tx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_tx.c @@ -615,6 +615,7 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev) int lso_header_size; void *fragptr; bool bounce = false; + int ring_busy; if (!priv->port_up) goto tx_drop; @@ -638,8 +639,8 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev) vlan_tag = vlan_tx_tag_get(skb); /* Check available TXBBs And 2K spare for prefetch */ - if (unlikely(((int)(ring->prod - ring->cons)) > - ring->size - HEADROOM - MAX_DESC_TXBBS)) { + ring_busy = (int)(ring->prod - ring->cons) - (ring->size - HEADROOM - MAX_DESC_TXBBS); + if (unlikely(ring_busy > 0)) { /* every full Tx ring stops queue */ netif_tx_stop_queue(netdev_get_tx_queue(dev, tx_ind)); ring->blocked = 1; @@ -680,6 +681,8 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev) !!vlan_tx_tag_present(skb); tx_desc->ctrl.fence_size = (real_size / 16) & 0x3f; tx_desc->ctrl.srcrb_flags = priv->ctrl_flags; + if (!(!(index & 0xf) || vlan_tag || (ring_busy > -MAX_DESC_TXBBS))) + tx_desc->ctrl.srcrb_flags &= cpu_to_be32(~MLX4_WQE_CTRL_CQ_UPDATE); if (likely(skb->ip_summed == CHECKSUM_PARTIAL)) { tx_desc->ctrl.srcrb_flags |= cpu_to_be32(MLX4_WQE_CTRL_IP_CSUM | MLX4_WQE_CTRL_TCP_UDP_CSUM);
No need to ask for completion for every packet being sent. So the method is to ask for a completion every 16 packets, or when the queue is about to be full. Signed-off-by: Yevgeny Petrilin <yevgenyp@mellanox.co.il> --- drivers/net/ethernet/mellanox/mlx4/en_tx.c | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-)