| Message ID | 20121101160721.GA9176@redhat.com |
|---|---|
| State | New |
| Headers | show |
On (Thu) 01 Nov 2012 [18:07:21], Michael S. Tsirkin wrote: > Commit 0d8d7690850eb0cf2b2b60933cf47669a6b6f18f introduced > a regression in virtio-net performance because it looks > into the ring aggressively while we really only care > about a single packet worth of buffers. > To fix, add parameters limiting lookahead, and > use in virtqueue_avail_bytes. > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com> > Reported-by: Edivaldo de Araujo Pereira <edivaldoapereira@yahoo.com.br> Acked-by: Amit Shah <amit.shah@redhat.com> Amit
On Thu, Nov 1, 2012 at 5:07 PM, Michael S. Tsirkin <mst@redhat.com> wrote: > Commit 0d8d7690850eb0cf2b2b60933cf47669a6b6f18f introduced > a regression in virtio-net performance because it looks > into the ring aggressively while we really only care > about a single packet worth of buffers. > To fix, add parameters limiting lookahead, and > use in virtqueue_avail_bytes. > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com> > Reported-by: Edivaldo de Araujo Pereira <edivaldoapereira@yahoo.com.br> Nice, much simpler than the ideas I had. Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
On Fri, Nov 02, 2012 at 11:18:18AM +0100, Stefan Hajnoczi wrote: > On Thu, Nov 1, 2012 at 5:07 PM, Michael S. Tsirkin <mst@redhat.com> wrote: > > Commit 0d8d7690850eb0cf2b2b60933cf47669a6b6f18f introduced > > a regression in virtio-net performance because it looks > > into the ring aggressively while we really only care > > about a single packet worth of buffers. > > To fix, add parameters limiting lookahead, and > > use in virtqueue_avail_bytes. > > > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com> > > Reported-by: Edivaldo de Araujo Pereira <edivaldoapereira@yahoo.com.br> > > Nice, much simpler than the ideas I had. > > Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Anthony could you apply this out of band please so this stops biting people? Thanks, MST
On Fri, Nov 2, 2012 at 3:48 PM, Michael S. Tsirkin <mst@redhat.com> wrote: > On Fri, Nov 02, 2012 at 11:18:18AM +0100, Stefan Hajnoczi wrote: >> On Thu, Nov 1, 2012 at 5:07 PM, Michael S. Tsirkin <mst@redhat.com> wrote: >> > Commit 0d8d7690850eb0cf2b2b60933cf47669a6b6f18f introduced >> > a regression in virtio-net performance because it looks >> > into the ring aggressively while we really only care >> > about a single packet worth of buffers. >> > To fix, add parameters limiting lookahead, and >> > use in virtqueue_avail_bytes. >> > >> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com> >> > Reported-by: Edivaldo de Araujo Pereira <edivaldoapereira@yahoo.com.br> >> >> Nice, much simpler than the ideas I had. >> >> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> > > Anthony could you apply this out of band please so this stops > biting people? Especially for the 1.3 release so that we don't have a virtio performance regression. Stefan
On Thu, Nov 01, 2012 at 06:07:21PM +0200, Michael S. Tsirkin wrote: > Commit 0d8d7690850eb0cf2b2b60933cf47669a6b6f18f introduced > a regression in virtio-net performance because it looks > into the ring aggressively while we really only care > about a single packet worth of buffers. > To fix, add parameters limiting lookahead, and > use in virtqueue_avail_bytes. > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com> > Reported-by: Edivaldo de Araujo Pereira <edivaldoapereira@yahoo.com.br> Ping. Anthony - going to apply this? > --- > > Edivaldo could you please confirm this fixes regression? > > diff --git a/hw/virtio-serial-bus.c b/hw/virtio-serial-bus.c > index d20bd8b..a761cdc 100644 > --- a/hw/virtio-serial-bus.c > +++ b/hw/virtio-serial-bus.c > @@ -297,7 +297,7 @@ size_t virtio_serial_guest_ready(VirtIOSerialPort *port) > if (use_multiport(port->vser) && !port->guest_connected) { > return 0; > } > - virtqueue_get_avail_bytes(vq, &bytes, NULL); > + virtqueue_get_avail_bytes(vq, &bytes, NULL, UINT_MAX, 0); > return bytes; > } > > diff --git a/hw/virtio.c b/hw/virtio.c > index ec8b7d8..f40a8c5 100644 > --- a/hw/virtio.c > +++ b/hw/virtio.c > @@ -336,7 +336,8 @@ static unsigned virtqueue_next_desc(hwaddr desc_pa, > } > > void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes, > - unsigned int *out_bytes) > + unsigned int *out_bytes, > + unsigned max_in_bytes, unsigned max_out_bytes) > { > unsigned int idx; > unsigned int total_bufs, in_total, out_total; > @@ -385,6 +386,9 @@ void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes, > } else { > out_total += vring_desc_len(desc_pa, i); > } > + if (in_total >= max_in_bytes && out_total >= max_out_bytes) { > + goto done; > + } > } while ((i = virtqueue_next_desc(desc_pa, i, max)) != max); > > if (!indirect) > @@ -392,6 +396,7 @@ void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes, > else > total_bufs++; > } > +done: > if (in_bytes) { > *in_bytes = in_total; > } > @@ -405,12 +410,8 @@ int virtqueue_avail_bytes(VirtQueue *vq, unsigned int in_bytes, > { > unsigned int in_total, out_total; > > - virtqueue_get_avail_bytes(vq, &in_total, &out_total); > - if ((in_bytes && in_bytes < in_total) > - || (out_bytes && out_bytes < out_total)) { > - return 1; > - } > - return 0; > + virtqueue_get_avail_bytes(vq, &in_total, &out_total, in_bytes, out_bytes); > + return in_bytes <= in_total && out_bytes <= out_total; > } > > void virtqueue_map_sg(struct iovec *sg, hwaddr *addr, > diff --git a/hw/virtio.h b/hw/virtio.h > index ac482be..1278328 100644 > --- a/hw/virtio.h > +++ b/hw/virtio.h > @@ -150,7 +150,8 @@ int virtqueue_pop(VirtQueue *vq, VirtQueueElement *elem); > int virtqueue_avail_bytes(VirtQueue *vq, unsigned int in_bytes, > unsigned int out_bytes); > void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes, > - unsigned int *out_bytes); > + unsigned int *out_bytes, > + unsigned max_in_bytes, unsigned max_out_bytes); > > void virtio_notify(VirtIODevice *vdev, VirtQueue *vq); >
Dear friends, Please excuse-me for not reporting earlier... I confirm that the patch by Michael really fixes the problem I've reported. The regression has gone away when I used it, so I think it is good to be applied. Thanks, Edivaldo de Araújo Pereira --- Em ter, 27/11/12, Michael S. Tsirkin <mst@redhat.com> escreveu: > De: Michael S. Tsirkin <mst@redhat.com> > Assunto: Re: [PATCH] virtio: limit avail bytes lookahead > Para: "Amit Shah" <amit.shah@redhat.com> > Cc: "Stefan Hajnoczi" <stefanha@gmail.com>, "Edivaldo de Araujo Pereira" <edivaldoapereira@yahoo.com.br>, qemu-devel@nongnu.org, "Anthony Liguori" <anthony@codemonkey.ws>, "Bug 1066055" <1066055@bugs.launchpad.net> > Data: Terça-feira, 27 de Novembro de 2012, 8:25 > On Thu, Nov 01, 2012 at 06:07:21PM > +0200, Michael S. Tsirkin wrote: > > Commit 0d8d7690850eb0cf2b2b60933cf47669a6b6f18f > introduced > > a regression in virtio-net performance because it > looks > > into the ring aggressively while we really only care > > about a single packet worth of buffers. > > To fix, add parameters limiting lookahead, and > > use in virtqueue_avail_bytes. > > > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com> > > Reported-by: Edivaldo de Araujo Pereira <edivaldoapereira@yahoo.com.br> > > Ping. > Anthony - going to apply this? > > > > --- > > > > Edivaldo could you please confirm this fixes > regression? > > > > diff --git a/hw/virtio-serial-bus.c > b/hw/virtio-serial-bus.c > > index d20bd8b..a761cdc 100644 > > --- a/hw/virtio-serial-bus.c > > +++ b/hw/virtio-serial-bus.c > > @@ -297,7 +297,7 @@ size_t > virtio_serial_guest_ready(VirtIOSerialPort *port) > > if (use_multiport(port->vser) > && !port->guest_connected) { > > return 0; > > } > > - virtqueue_get_avail_bytes(vq, > &bytes, NULL); > > + virtqueue_get_avail_bytes(vq, > &bytes, NULL, UINT_MAX, 0); > > return bytes; > > } > > > > diff --git a/hw/virtio.c b/hw/virtio.c > > index ec8b7d8..f40a8c5 100644 > > --- a/hw/virtio.c > > +++ b/hw/virtio.c > > @@ -336,7 +336,8 @@ static unsigned > virtqueue_next_desc(hwaddr desc_pa, > > } > > > > void virtqueue_get_avail_bytes(VirtQueue *vq, > unsigned int *in_bytes, > > - > > unsigned int *out_bytes) > > + > > unsigned int *out_bytes, > > + > > unsigned max_in_bytes, unsigned > max_out_bytes) > > { > > unsigned int idx; > > unsigned int total_bufs, in_total, > out_total; > > @@ -385,6 +386,9 @@ void > virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int > *in_bytes, > > } else > { > > > out_total += vring_desc_len(desc_pa, i); > > } > > + if (in_total > >= max_in_bytes && out_total >= max_out_bytes) > { > > + > goto done; > > + } > > } while ((i = > virtqueue_next_desc(desc_pa, i, max)) != max); > > > > if (!indirect) > > @@ -392,6 +396,7 @@ void > virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int > *in_bytes, > > else > > > total_bufs++; > > } > > +done: > > if (in_bytes) { > > *in_bytes = > in_total; > > } > > @@ -405,12 +410,8 @@ int > virtqueue_avail_bytes(VirtQueue *vq, unsigned int in_bytes, > > { > > unsigned int in_total, out_total; > > > > - virtqueue_get_avail_bytes(vq, > &in_total, &out_total); > > - if ((in_bytes && in_bytes < > in_total) > > - || (out_bytes && > out_bytes < out_total)) { > > - return 1; > > - } > > - return 0; > > + virtqueue_get_avail_bytes(vq, > &in_total, &out_total, in_bytes, out_bytes); > > + return in_bytes <= in_total > && out_bytes <= out_total; > > } > > > > void virtqueue_map_sg(struct iovec *sg, hwaddr > *addr, > > diff --git a/hw/virtio.h b/hw/virtio.h > > index ac482be..1278328 100644 > > --- a/hw/virtio.h > > +++ b/hw/virtio.h > > @@ -150,7 +150,8 @@ int virtqueue_pop(VirtQueue *vq, > VirtQueueElement *elem); > > int virtqueue_avail_bytes(VirtQueue *vq, unsigned > int in_bytes, > > > unsigned int > out_bytes); > > void virtqueue_get_avail_bytes(VirtQueue *vq, > unsigned int *in_bytes, > > - > > unsigned int *out_bytes); > > + > > unsigned int *out_bytes, > > + > > unsigned max_in_bytes, unsigned > max_out_bytes); > > > > void virtio_notify(VirtIODevice *vdev, VirtQueue > *vq); > > >
"Michael S. Tsirkin" <mst@redhat.com> writes: > On Thu, Nov 01, 2012 at 06:07:21PM +0200, Michael S. Tsirkin wrote: >> Commit 0d8d7690850eb0cf2b2b60933cf47669a6b6f18f introduced >> a regression in virtio-net performance because it looks >> into the ring aggressively while we really only care >> about a single packet worth of buffers. >> To fix, add parameters limiting lookahead, and >> use in virtqueue_avail_bytes. >> >> Signed-off-by: Michael S. Tsirkin <mst@redhat.com> >> Reported-by: Edivaldo de Araujo Pereira <edivaldoapereira@yahoo.com.br> > > Ping. > Anthony - going to apply this? Yes, I've got it queued now. In the future, please top post patches. Regards, Anthony Liguori > > >> --- >> >> Edivaldo could you please confirm this fixes regression? >> >> diff --git a/hw/virtio-serial-bus.c b/hw/virtio-serial-bus.c >> index d20bd8b..a761cdc 100644 >> --- a/hw/virtio-serial-bus.c >> +++ b/hw/virtio-serial-bus.c >> @@ -297,7 +297,7 @@ size_t virtio_serial_guest_ready(VirtIOSerialPort *port) >> if (use_multiport(port->vser) && !port->guest_connected) { >> return 0; >> } >> - virtqueue_get_avail_bytes(vq, &bytes, NULL); >> + virtqueue_get_avail_bytes(vq, &bytes, NULL, UINT_MAX, 0); >> return bytes; >> } >> >> diff --git a/hw/virtio.c b/hw/virtio.c >> index ec8b7d8..f40a8c5 100644 >> --- a/hw/virtio.c >> +++ b/hw/virtio.c >> @@ -336,7 +336,8 @@ static unsigned virtqueue_next_desc(hwaddr desc_pa, >> } >> >> void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes, >> - unsigned int *out_bytes) >> + unsigned int *out_bytes, >> + unsigned max_in_bytes, unsigned max_out_bytes) >> { >> unsigned int idx; >> unsigned int total_bufs, in_total, out_total; >> @@ -385,6 +386,9 @@ void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes, >> } else { >> out_total += vring_desc_len(desc_pa, i); >> } >> + if (in_total >= max_in_bytes && out_total >= max_out_bytes) { >> + goto done; >> + } >> } while ((i = virtqueue_next_desc(desc_pa, i, max)) != max); >> >> if (!indirect) >> @@ -392,6 +396,7 @@ void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes, >> else >> total_bufs++; >> } >> +done: >> if (in_bytes) { >> *in_bytes = in_total; >> } >> @@ -405,12 +410,8 @@ int virtqueue_avail_bytes(VirtQueue *vq, unsigned int in_bytes, >> { >> unsigned int in_total, out_total; >> >> - virtqueue_get_avail_bytes(vq, &in_total, &out_total); >> - if ((in_bytes && in_bytes < in_total) >> - || (out_bytes && out_bytes < out_total)) { >> - return 1; >> - } >> - return 0; >> + virtqueue_get_avail_bytes(vq, &in_total, &out_total, in_bytes, out_bytes); >> + return in_bytes <= in_total && out_bytes <= out_total; >> } >> >> void virtqueue_map_sg(struct iovec *sg, hwaddr *addr, >> diff --git a/hw/virtio.h b/hw/virtio.h >> index ac482be..1278328 100644 >> --- a/hw/virtio.h >> +++ b/hw/virtio.h >> @@ -150,7 +150,8 @@ int virtqueue_pop(VirtQueue *vq, VirtQueueElement *elem); >> int virtqueue_avail_bytes(VirtQueue *vq, unsigned int in_bytes, >> unsigned int out_bytes); >> void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes, >> - unsigned int *out_bytes); >> + unsigned int *out_bytes, >> + unsigned max_in_bytes, unsigned max_out_bytes); >> >> void virtio_notify(VirtIODevice *vdev, VirtQueue *vq); >>
diff --git a/hw/virtio-serial-bus.c b/hw/virtio-serial-bus.c index d20bd8b..a761cdc 100644 --- a/hw/virtio-serial-bus.c +++ b/hw/virtio-serial-bus.c @@ -297,7 +297,7 @@ size_t virtio_serial_guest_ready(VirtIOSerialPort *port) if (use_multiport(port->vser) && !port->guest_connected) { return 0; } - virtqueue_get_avail_bytes(vq, &bytes, NULL); + virtqueue_get_avail_bytes(vq, &bytes, NULL, UINT_MAX, 0); return bytes; } diff --git a/hw/virtio.c b/hw/virtio.c index ec8b7d8..f40a8c5 100644 --- a/hw/virtio.c +++ b/hw/virtio.c @@ -336,7 +336,8 @@ static unsigned virtqueue_next_desc(hwaddr desc_pa, } void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes, - unsigned int *out_bytes) + unsigned int *out_bytes, + unsigned max_in_bytes, unsigned max_out_bytes) { unsigned int idx; unsigned int total_bufs, in_total, out_total; @@ -385,6 +386,9 @@ void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes, } else { out_total += vring_desc_len(desc_pa, i); } + if (in_total >= max_in_bytes && out_total >= max_out_bytes) { + goto done; + } } while ((i = virtqueue_next_desc(desc_pa, i, max)) != max); if (!indirect) @@ -392,6 +396,7 @@ void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes, else total_bufs++; } +done: if (in_bytes) { *in_bytes = in_total; } @@ -405,12 +410,8 @@ int virtqueue_avail_bytes(VirtQueue *vq, unsigned int in_bytes, { unsigned int in_total, out_total; - virtqueue_get_avail_bytes(vq, &in_total, &out_total); - if ((in_bytes && in_bytes < in_total) - || (out_bytes && out_bytes < out_total)) { - return 1; - } - return 0; + virtqueue_get_avail_bytes(vq, &in_total, &out_total, in_bytes, out_bytes); + return in_bytes <= in_total && out_bytes <= out_total; } void virtqueue_map_sg(struct iovec *sg, hwaddr *addr, diff --git a/hw/virtio.h b/hw/virtio.h index ac482be..1278328 100644 --- a/hw/virtio.h +++ b/hw/virtio.h @@ -150,7 +150,8 @@ int virtqueue_pop(VirtQueue *vq, VirtQueueElement *elem); int virtqueue_avail_bytes(VirtQueue *vq, unsigned int in_bytes, unsigned int out_bytes); void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes, - unsigned int *out_bytes); + unsigned int *out_bytes, + unsigned max_in_bytes, unsigned max_out_bytes); void virtio_notify(VirtIODevice *vdev, VirtQueue *vq);
Commit 0d8d7690850eb0cf2b2b60933cf47669a6b6f18f introduced a regression in virtio-net performance because it looks into the ring aggressively while we really only care about a single packet worth of buffers. To fix, add parameters limiting lookahead, and use in virtqueue_avail_bytes. Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Reported-by: Edivaldo de Araujo Pereira <edivaldoapereira@yahoo.com.br> --- Edivaldo could you please confirm this fixes regression?