diff mbox

[net-next] vhost: fix condition check for # of outstanding dma buffers

Message ID 1311100678.8573.16.camel@localhost.localdomain
State Changes Requested, archived
Delegated to: David Miller
Headers show

Commit Message

Shirley Ma July 19, 2011, 6:37 p.m. UTC 
Signed-off-by: Shirley Ma <xma@us.ibm.com>
---

 drivers/vhost/net.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)



--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Comments

Michael S. Tsirkin July 19, 2011, 7:09 p.m. UTC | #1 
On Tue, Jul 19, 2011 at 11:37:58AM -0700, Shirley Ma wrote:
> Signed-off-by: Shirley Ma <xma@us.ibm.com>
> ---
> 
>  drivers/vhost/net.c |    6 ++++--
>  1 files changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
> index 70ac604..83cb738 100644
> --- a/drivers/vhost/net.c
> +++ b/drivers/vhost/net.c
> @@ -189,8 +189,10 @@ static void handle_tx(struct vhost_net *net)
>  				break;
>  			}
>  			/* If more outstanding DMAs, queue the work */
> -			if (unlikely(vq->upend_idx - vq->done_idx >
> -				     VHOST_MAX_PEND)) {
> +			if (unlikely((vq->upend_idx - vq->done_idx >
> +					VHOST_MAX_PEND) ||
> +				     (vq->upend_idx - vq->done_idx >
> +					 VHOST_MAX_PEND - UIO_MAXIOV))) {

Could you please explain why this makes sense please?
VHOST_MAX_PEND is 128 UIO_MAXIOV is 1024 so
the result is negative?

I thought upend_idx - done_idx is exactly the number
of buffers, so once we get too many we stop until
one gets freed?


>  				tx_poll_start(net, sock);
>  				set_bit(SOCK_ASYNC_NOSPACE, &sock->flags);
>  				break;
> 
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Shirley Ma July 19, 2011, 8:56 p.m. UTC | #2 
On Tue, 2011-07-19 at 22:09 +0300, Michael S. Tsirkin wrote:
> On Tue, Jul 19, 2011 at 11:37:58AM -0700, Shirley Ma wrote:
> > Signed-off-by: Shirley Ma <xma@us.ibm.com>
> > ---
> > 
> >  drivers/vhost/net.c |    6 ++++--
> >  1 files changed, 4 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
> > index 70ac604..83cb738 100644
> > --- a/drivers/vhost/net.c
> > +++ b/drivers/vhost/net.c
> > @@ -189,8 +189,10 @@ static void handle_tx(struct vhost_net *net)
> >                               break;
> >                       }
> >                       /* If more outstanding DMAs, queue the work */
> > -                     if (unlikely(vq->upend_idx - vq->done_idx >
> > -                                  VHOST_MAX_PEND)) {
> > +                     if (unlikely((vq->upend_idx - vq->done_idx >
> > +                                     VHOST_MAX_PEND) ||
> > +                                  (vq->upend_idx - vq->done_idx >
> > +                                      VHOST_MAX_PEND -
> UIO_MAXIOV))) {
> 
> Could you please explain why this makes sense please?
> VHOST_MAX_PEND is 128 UIO_MAXIOV is 1024 so
> the result is negative?

I thought it is equal to:

if (vq->upend_idx > vq->done_idx) 
	check vq->upend_idx - vq->done_idx > VHOST_MAX_PEND
if (vq->upend_idx < vq->done_idx)
	check vq->upend_idx + UIO_MAXIOV - vq->done_idx > VHOST_MAX_PEND
	

> I thought upend_idx - done_idx is exactly the number
> of buffers, so once we get too many we stop until
> one gets freed?

They are index, so in vhost zerocopy callback, we can get the idx right
away.

> 
> >                               tx_poll_start(net, sock);
> >                               set_bit(SOCK_ASYNC_NOSPACE,
> &sock->flags);
> >                               break;
> > 

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Michael S. Tsirkin July 20, 2011, 10:28 a.m. UTC | #3 
On Tue, Jul 19, 2011 at 01:56:25PM -0700, Shirley Ma wrote:
> On Tue, 2011-07-19 at 22:09 +0300, Michael S. Tsirkin wrote:
> > On Tue, Jul 19, 2011 at 11:37:58AM -0700, Shirley Ma wrote:
> > > Signed-off-by: Shirley Ma <xma@us.ibm.com>
> > > ---
> > > 
> > >  drivers/vhost/net.c |    6 ++++--
> > >  1 files changed, 4 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
> > > index 70ac604..83cb738 100644
> > > --- a/drivers/vhost/net.c
> > > +++ b/drivers/vhost/net.c
> > > @@ -189,8 +189,10 @@ static void handle_tx(struct vhost_net *net)
> > >                               break;
> > >                       }
> > >                       /* If more outstanding DMAs, queue the work */
> > > -                     if (unlikely(vq->upend_idx - vq->done_idx >
> > > -                                  VHOST_MAX_PEND)) {
> > > +                     if (unlikely((vq->upend_idx - vq->done_idx >
> > > +                                     VHOST_MAX_PEND) ||
> > > +                                  (vq->upend_idx - vq->done_idx >
> > > +                                      VHOST_MAX_PEND -
> > UIO_MAXIOV))) {
> > 
> > Could you please explain why this makes sense please?
> > VHOST_MAX_PEND is 128 UIO_MAXIOV is 1024 so
> > the result is negative?
> 
> I thought it is equal to:
> 
> if (vq->upend_idx > vq->done_idx) 
> 	check vq->upend_idx - vq->done_idx > VHOST_MAX_PEND
> if (vq->upend_idx < vq->done_idx)
> 	check vq->upend_idx + UIO_MAXIOV - vq->done_idx > VHOST_MAX_PEND
> 	

Check it out: upend_idx == done_idx == 0 does not satisfy the
above conditions but does trigger in your code, right?

Better keep it simple. Maybe:

	if (unlikely(vq->upend_idx - vq->done_idx > VHOST_MAX_PEND) ||
		(unlikely(vq->upend_idx < vq->done_idx) &&
	 	unlikely(vq->upend_idx + UIO_MAXIOV - vq->done_idx >
			 VHOST_MAX_PEND)))

?

Also, please add commit log documenting what does the patch
fix: something like:
	'the test for # of outstanding buffers returned
	 incorrect results when due to wrap around,
	 upend_idx < done_idx'?

> > I thought upend_idx - done_idx is exactly the number
> > of buffers, so once we get too many we stop until
> > one gets freed?
> 
> They are index, so in vhost zerocopy callback, we can get the idx right
> away.
> 
> > 
> > >                               tx_poll_start(net, sock);
> > >                               set_bit(SOCK_ASYNC_NOSPACE,
> > &sock->flags);
> > >                               break;
> > > 
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Shirley Ma July 20, 2011, 3:43 p.m. UTC | #4 
On Wed, 2011-07-20 at 13:28 +0300, Michael S. Tsirkin wrote:
> On Tue, Jul 19, 2011 at 01:56:25PM -0700, Shirley Ma wrote:
> > On Tue, 2011-07-19 at 22:09 +0300, Michael S. Tsirkin wrote:
> > > On Tue, Jul 19, 2011 at 11:37:58AM -0700, Shirley Ma wrote:
> > > > Signed-off-by: Shirley Ma <xma@us.ibm.com>
> > > > ---
> > > > 
> > > >  drivers/vhost/net.c |    6 ++++--
> > > >  1 files changed, 4 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
> > > > index 70ac604..83cb738 100644
> > > > --- a/drivers/vhost/net.c
> > > > +++ b/drivers/vhost/net.c
> > > > @@ -189,8 +189,10 @@ static void handle_tx(struct vhost_net
> *net)
> > > >                               break;
> > > >                       }
> > > >                       /* If more outstanding DMAs, queue the
> work */
> > > > -                     if (unlikely(vq->upend_idx - vq->done_idx
> >
> > > > -                                  VHOST_MAX_PEND)) {
> > > > +                     if (unlikely((vq->upend_idx - vq->done_idx
> >
> > > > +                                     VHOST_MAX_PEND) ||
> > > > +                                  (vq->upend_idx - vq->done_idx
> >
> > > > +                                      VHOST_MAX_PEND -
> > > UIO_MAXIOV))) {
> > > 
> > > Could you please explain why this makes sense please?
> > > VHOST_MAX_PEND is 128 UIO_MAXIOV is 1024 so
> > > the result is negative?
> > 
> > I thought it is equal to:
> > 
> > if (vq->upend_idx > vq->done_idx) 
> >       check vq->upend_idx - vq->done_idx > VHOST_MAX_PEND
> > if (vq->upend_idx < vq->done_idx)
> >       check vq->upend_idx + UIO_MAXIOV - vq->done_idx >
> VHOST_MAX_PEND
> >       
> 
> Check it out: upend_idx == done_idx == 0 does not satisfy the
> above conditions but does trigger in your code, right?

We don't hit upend_idx == done_idx == 0. Only upend_idx == done_idx ==
UIO_MAXIOV could happen if the lower device has issue and never DMA any
packets out.

> Better keep it simple. Maybe:
> 
>         if (unlikely(vq->upend_idx - vq->done_idx > VHOST_MAX_PEND) ||
>                 (unlikely(vq->upend_idx < vq->done_idx) &&
>                 unlikely(vq->upend_idx + UIO_MAXIOV - vq->done_idx >
>                          VHOST_MAX_PEND)))
> 
> ?
> 
> Also, please add commit log documenting what does the patch
> fix: something like:
>         'the test for # of outstanding buffers returned
>          incorrect results when due to wrap around,
>          upend_idx < done_idx'?

Sure, will modify it and resubmit.

> > > I thought upend_idx - done_idx is exactly the number
> > > of buffers, so once we get too many we stop until
> > > one gets freed?
> > 
> > They are index, so in vhost zerocopy callback, we can get the idx
> right
> > away.
> > 
> > > 
> > > >                               tx_poll_start(net, sock);
> > > >                               set_bit(SOCK_ASYNC_NOSPACE,
> > > &sock->flags);
> > > >                               break;
> > > > 

Thanks
Shirley

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Michael S. Tsirkin July 20, 2011, 4:17 p.m. UTC | #5 
On Wed, Jul 20, 2011 at 08:43:09AM -0700, Shirley Ma wrote:
> On Wed, 2011-07-20 at 13:28 +0300, Michael S. Tsirkin wrote:
> > On Tue, Jul 19, 2011 at 01:56:25PM -0700, Shirley Ma wrote:
> > > On Tue, 2011-07-19 at 22:09 +0300, Michael S. Tsirkin wrote:
> > > > On Tue, Jul 19, 2011 at 11:37:58AM -0700, Shirley Ma wrote:
> > > > > Signed-off-by: Shirley Ma <xma@us.ibm.com>
> > > > > ---
> > > > > 
> > > > >  drivers/vhost/net.c |    6 ++++--
> > > > >  1 files changed, 4 insertions(+), 2 deletions(-)
> > > > > 
> > > > > diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
> > > > > index 70ac604..83cb738 100644
> > > > > --- a/drivers/vhost/net.c
> > > > > +++ b/drivers/vhost/net.c
> > > > > @@ -189,8 +189,10 @@ static void handle_tx(struct vhost_net
> > *net)
> > > > >                               break;
> > > > >                       }
> > > > >                       /* If more outstanding DMAs, queue the
> > work */
> > > > > -                     if (unlikely(vq->upend_idx - vq->done_idx
> > >
> > > > > -                                  VHOST_MAX_PEND)) {
> > > > > +                     if (unlikely((vq->upend_idx - vq->done_idx
> > >
> > > > > +                                     VHOST_MAX_PEND) ||
> > > > > +                                  (vq->upend_idx - vq->done_idx
> > >
> > > > > +                                      VHOST_MAX_PEND -
> > > > UIO_MAXIOV))) {
> > > > 
> > > > Could you please explain why this makes sense please?
> > > > VHOST_MAX_PEND is 128 UIO_MAXIOV is 1024 so
> > > > the result is negative?
> > > 
> > > I thought it is equal to:
> > > 
> > > if (vq->upend_idx > vq->done_idx) 
> > >       check vq->upend_idx - vq->done_idx > VHOST_MAX_PEND
> > > if (vq->upend_idx < vq->done_idx)
> > >       check vq->upend_idx + UIO_MAXIOV - vq->done_idx >
> > VHOST_MAX_PEND
> > >       
> > 
> > Check it out: upend_idx == done_idx == 0 does not satisfy the
> > above conditions but does trigger in your code, right?
> 
> We don't hit upend_idx == done_idx == 0. Only upend_idx == done_idx ==
> UIO_MAXIOV could happen if the lower device has issue and never DMA any
> packets out.

My point was that the logic isn't the same, even though
you said 'it is equal to'.
Same applies to upend_idx == 1, done_idx == 0.

> > Better keep it simple. Maybe:
> > 
> >         if (unlikely(vq->upend_idx - vq->done_idx > VHOST_MAX_PEND) ||
> >                 (unlikely(vq->upend_idx < vq->done_idx) &&
> >                 unlikely(vq->upend_idx + UIO_MAXIOV - vq->done_idx >
> >                          VHOST_MAX_PEND)))
> > 
> > ?
> > 
> > Also, please add commit log documenting what does the patch
> > fix: something like:
> >         'the test for # of outstanding buffers returned
> >          incorrect results when due to wrap around,
> >          upend_idx < done_idx'?
> 
> Sure, will modify it and resubmit.
> 
> > > > I thought upend_idx - done_idx is exactly the number
> > > > of buffers, so once we get too many we stop until
> > > > one gets freed?
> > > 
> > > They are index, so in vhost zerocopy callback, we can get the idx
> > right
> > > away.
> > > 
> > > > 
> > > > >                               tx_poll_start(net, sock);
> > > > >                               set_bit(SOCK_ASYNC_NOSPACE,
> > > > &sock->flags);
> > > > >                               break;
> > > > > 
> 
> Thanks
> Shirley
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Shirley Ma July 20, 2011, 4:49 p.m. UTC | #6 
On Wed, 2011-07-20 at 19:17 +0300, Michael S. Tsirkin wrote:
> My point was that the logic isn't the same, even though
> you said 'it is equal to'.
> Same applies to upend_idx == 1, done_idx == 0.

:) not sure what I was thinking.

I will resubmit the patch.

Thanks
Shirley


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index 70ac604..83cb738 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -189,8 +189,10 @@  static void handle_tx(struct vhost_net *net)
 				break;
 			}
 			/* If more outstanding DMAs, queue the work */
-			if (unlikely(vq->upend_idx - vq->done_idx >
-				     VHOST_MAX_PEND)) {
+			if (unlikely((vq->upend_idx - vq->done_idx >
+					VHOST_MAX_PEND) ||
+				     (vq->upend_idx - vq->done_idx >
+					 VHOST_MAX_PEND - UIO_MAXIOV))) {
 				tx_poll_start(net, sock);
 				set_bit(SOCK_ASYNC_NOSPACE, &sock->flags);
 				break;