| Message ID | 20200129140702.5411-3-dplotnikov@virtuozzo.com |
|---|---|
| State | New |
| Headers | show |
| Series | Increase default virtqueue size to improve performance | expand |
On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote: > The goal is to reduce the amount of requests issued by a guest on > 1M reads/writes. This rises the performance up to 4% on that kind of > disk access pattern. > > The maximum chunk size to be used for the guest disk accessing is > limited with seg_max parameter, which represents the max amount of > pices in the scatter-geather list in one guest disk request. > > Since seg_max is virqueue_size dependent, increasing the virtqueue > size increases seg_max, which, in turn, increases the maximum size > of data to be read/write from guest disk. > > More details in the original problem statment: > https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html > > Suggested-by: Denis V. Lunev <den@openvz.org> > Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> looks good but let's just patch each device separately. > --- > hw/core/machine.c | 3 +++ > include/hw/virtio/virtio.h | 2 +- > 2 files changed, 4 insertions(+), 1 deletion(-) > > diff --git a/hw/core/machine.c b/hw/core/machine.c > index 3e288bfceb..8bc401d8b7 100644 > --- a/hw/core/machine.c > +++ b/hw/core/machine.c > @@ -28,6 +28,9 @@ > #include "hw/mem/nvdimm.h" > > GlobalProperty hw_compat_4_2[] = { > + { "virtio-blk-device", "queue-size", "128"}, > + { "virtio-scsi-device", "virtqueue_size", "128"}, > + { "vhost-blk-device", "virtqueue_size", "128"}, > { "virtio-blk-device", "x-enable-wce-if-config-wce", "off" }, > { "virtio-blk-device", "seg-max-adjust", "off"}, > { "virtio-scsi-device", "seg_max_adjust", "off"}, > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h > index a66ea2368b..16d540e390 100644 > --- a/include/hw/virtio/virtio.h > +++ b/include/hw/virtio/virtio.h > @@ -48,7 +48,7 @@ size_t virtio_feature_get_config_size(VirtIOFeature *features, > typedef struct VirtQueue VirtQueue; > > #define VIRTQUEUE_MAX_SIZE 1024 > -#define VIRTQUEUE_DEFAULT_SIZE 128 > +#define VIRTQUEUE_DEFAULT_SIZE 256 > > typedef struct VirtQueueElement > { > -- > 2.17.0
On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote: > The goal is to reduce the amount of requests issued by a guest on > 1M reads/writes. This rises the performance up to 4% on that kind of > disk access pattern. > > The maximum chunk size to be used for the guest disk accessing is > limited with seg_max parameter, which represents the max amount of > pices in the scatter-geather list in one guest disk request. > > Since seg_max is virqueue_size dependent, increasing the virtqueue > size increases seg_max, which, in turn, increases the maximum size > of data to be read/write from guest disk. > > More details in the original problem statment: > https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html > > Suggested-by: Denis V. Lunev <den@openvz.org> > Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> > --- > hw/core/machine.c | 3 +++ > include/hw/virtio/virtio.h | 2 +- > 2 files changed, 4 insertions(+), 1 deletion(-) > > diff --git a/hw/core/machine.c b/hw/core/machine.c > index 3e288bfceb..8bc401d8b7 100644 > --- a/hw/core/machine.c > +++ b/hw/core/machine.c > @@ -28,6 +28,9 @@ > #include "hw/mem/nvdimm.h" > > GlobalProperty hw_compat_4_2[] = { > + { "virtio-blk-device", "queue-size", "128"}, > + { "virtio-scsi-device", "virtqueue_size", "128"}, > + { "vhost-blk-device", "virtqueue_size", "128"}, vhost-blk-device?! Who has this? It's not in qemu.git so please omit this line. ;-) On the other hand, do you want to do this for the vhost-user-blk, vhost-user-scsi, and vhost-scsi devices that exist in qemu.git? Those devices would benefit from better performance too.
On 30.01.2020 16:40, Michael S. Tsirkin wrote: > On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote: >> The goal is to reduce the amount of requests issued by a guest on >> 1M reads/writes. This rises the performance up to 4% on that kind of >> disk access pattern. >> >> The maximum chunk size to be used for the guest disk accessing is >> limited with seg_max parameter, which represents the max amount of >> pices in the scatter-geather list in one guest disk request. >> >> Since seg_max is virqueue_size dependent, increasing the virtqueue >> size increases seg_max, which, in turn, increases the maximum size >> of data to be read/write from guest disk. >> >> More details in the original problem statment: >> https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html >> >> Suggested-by: Denis V. Lunev <den@openvz.org> >> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> > > looks good but let's just patch each device separately. Ok Denis > >> --- >> hw/core/machine.c | 3 +++ >> include/hw/virtio/virtio.h | 2 +- >> 2 files changed, 4 insertions(+), 1 deletion(-) >> >> diff --git a/hw/core/machine.c b/hw/core/machine.c >> index 3e288bfceb..8bc401d8b7 100644 >> --- a/hw/core/machine.c >> +++ b/hw/core/machine.c >> @@ -28,6 +28,9 @@ >> #include "hw/mem/nvdimm.h" >> >> GlobalProperty hw_compat_4_2[] = { >> + { "virtio-blk-device", "queue-size", "128"}, >> + { "virtio-scsi-device", "virtqueue_size", "128"}, >> + { "vhost-blk-device", "virtqueue_size", "128"}, >> { "virtio-blk-device", "x-enable-wce-if-config-wce", "off" }, >> { "virtio-blk-device", "seg-max-adjust", "off"}, >> { "virtio-scsi-device", "seg_max_adjust", "off"}, >> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h >> index a66ea2368b..16d540e390 100644 >> --- a/include/hw/virtio/virtio.h >> +++ b/include/hw/virtio/virtio.h >> @@ -48,7 +48,7 @@ size_t virtio_feature_get_config_size(VirtIOFeature *features, >> typedef struct VirtQueue VirtQueue; >> >> #define VIRTQUEUE_MAX_SIZE 1024 >> -#define VIRTQUEUE_DEFAULT_SIZE 128 >> +#define VIRTQUEUE_DEFAULT_SIZE 256 >> >> typedef struct VirtQueueElement >> { >> -- >> 2.17.0
On 30.01.2020 17:58, Stefan Hajnoczi wrote: > On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote: >> The goal is to reduce the amount of requests issued by a guest on >> 1M reads/writes. This rises the performance up to 4% on that kind of >> disk access pattern. >> >> The maximum chunk size to be used for the guest disk accessing is >> limited with seg_max parameter, which represents the max amount of >> pices in the scatter-geather list in one guest disk request. >> >> Since seg_max is virqueue_size dependent, increasing the virtqueue >> size increases seg_max, which, in turn, increases the maximum size >> of data to be read/write from guest disk. >> >> More details in the original problem statment: >> https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html >> >> Suggested-by: Denis V. Lunev <den@openvz.org> >> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> >> --- >> hw/core/machine.c | 3 +++ >> include/hw/virtio/virtio.h | 2 +- >> 2 files changed, 4 insertions(+), 1 deletion(-) >> >> diff --git a/hw/core/machine.c b/hw/core/machine.c >> index 3e288bfceb..8bc401d8b7 100644 >> --- a/hw/core/machine.c >> +++ b/hw/core/machine.c >> @@ -28,6 +28,9 @@ >> #include "hw/mem/nvdimm.h" >> >> GlobalProperty hw_compat_4_2[] = { >> + { "virtio-blk-device", "queue-size", "128"}, >> + { "virtio-scsi-device", "virtqueue_size", "128"}, >> + { "vhost-blk-device", "virtqueue_size", "128"}, > vhost-blk-device?! Who has this? It's not in qemu.git so please omit > this line. ;-) So in this case the line: { "vhost-blk-device", "seg_max_adjust", "off"}, introduced by my patch: commit 1bf8a989a566b2ba41c197004ec2a02562a766a4 Author: Denis Plotnikov <dplotnikov@virtuozzo.com> Date: Fri Dec 20 17:09:04 2019 +0300 virtio: make seg_max virtqueue size dependent is also wrong. It should be: { "vhost-scsi-device", "seg_max_adjust", "off"}, Am I right? > > On the other hand, do you want to do this for the vhost-user-blk, > vhost-user-scsi, and vhost-scsi devices that exist in qemu.git? Those > devices would benefit from better performance too. It seems to be so. We also have the test checking those settings: tests/acceptance/virtio_seg_max_adjust.py For now it checks virtio-scsi-pci and virtio-blk-pci. I'm going to extend it for the virtqueue size checking. If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth to check those devices too. But I don't know how to form a command line for that 3 devices since they should involve some third party components as backends (kernel modules, DPDK, etc.) and they seems to be not available in the qemu git. Is there any way to do it with some qit.qemu available stubs or something else? If so, could you please point out the proper way to do it? Thanks! Denis
On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote: > > > On 30.01.2020 17:58, Stefan Hajnoczi wrote: > > On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote: > > > The goal is to reduce the amount of requests issued by a guest on > > > 1M reads/writes. This rises the performance up to 4% on that kind of > > > disk access pattern. > > > > > > The maximum chunk size to be used for the guest disk accessing is > > > limited with seg_max parameter, which represents the max amount of > > > pices in the scatter-geather list in one guest disk request. > > > > > > Since seg_max is virqueue_size dependent, increasing the virtqueue > > > size increases seg_max, which, in turn, increases the maximum size > > > of data to be read/write from guest disk. > > > > > > More details in the original problem statment: > > > https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html > > > > > > Suggested-by: Denis V. Lunev <den@openvz.org> > > > Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> > > > --- > > > hw/core/machine.c | 3 +++ > > > include/hw/virtio/virtio.h | 2 +- > > > 2 files changed, 4 insertions(+), 1 deletion(-) > > > > > > diff --git a/hw/core/machine.c b/hw/core/machine.c > > > index 3e288bfceb..8bc401d8b7 100644 > > > --- a/hw/core/machine.c > > > +++ b/hw/core/machine.c > > > @@ -28,6 +28,9 @@ > > > #include "hw/mem/nvdimm.h" > > > GlobalProperty hw_compat_4_2[] = { > > > + { "virtio-blk-device", "queue-size", "128"}, > > > + { "virtio-scsi-device", "virtqueue_size", "128"}, > > > + { "vhost-blk-device", "virtqueue_size", "128"}, > > vhost-blk-device?! Who has this? It's not in qemu.git so please omit > > this line. ;-) > So in this case the line: > > { "vhost-blk-device", "seg_max_adjust", "off"}, > > introduced by my patch: > > commit 1bf8a989a566b2ba41c197004ec2a02562a766a4 > Author: Denis Plotnikov <dplotnikov@virtuozzo.com> > Date: Fri Dec 20 17:09:04 2019 +0300 > > virtio: make seg_max virtqueue size dependent > > is also wrong. It should be: > > { "vhost-scsi-device", "seg_max_adjust", "off"}, > > Am I right? It's just called "vhost-scsi": include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi" > > > > On the other hand, do you want to do this for the vhost-user-blk, > > vhost-user-scsi, and vhost-scsi devices that exist in qemu.git? Those > > devices would benefit from better performance too. > It seems to be so. We also have the test checking those settings: > tests/acceptance/virtio_seg_max_adjust.py > For now it checks virtio-scsi-pci and virtio-blk-pci. > I'm going to extend it for the virtqueue size checking. > If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth > to check those devices too. But I don't know how to form a command line > for that 3 devices since they should involve some third party components as > backends (kernel modules, DPDK, etc.) and they seems to be not available in > the > qemu git. > Is there any way to do it with some qit.qemu available stubs or something > else? > If so, could you please point out the proper way to do it? qemu.git has contrib/vhost-user-blk/ and contrib/vhost-user-scsi/ if you need to test those vhost-user devices without external dependencies. Stefan
On 05.02.2020 14:19, Stefan Hajnoczi wrote: > On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote: >> >> On 30.01.2020 17:58, Stefan Hajnoczi wrote: >>> On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote: >>>> The goal is to reduce the amount of requests issued by a guest on >>>> 1M reads/writes. This rises the performance up to 4% on that kind of >>>> disk access pattern. >>>> >>>> The maximum chunk size to be used for the guest disk accessing is >>>> limited with seg_max parameter, which represents the max amount of >>>> pices in the scatter-geather list in one guest disk request. >>>> >>>> Since seg_max is virqueue_size dependent, increasing the virtqueue >>>> size increases seg_max, which, in turn, increases the maximum size >>>> of data to be read/write from guest disk. >>>> >>>> More details in the original problem statment: >>>> https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html >>>> >>>> Suggested-by: Denis V. Lunev <den@openvz.org> >>>> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> >>>> --- >>>> hw/core/machine.c | 3 +++ >>>> include/hw/virtio/virtio.h | 2 +- >>>> 2 files changed, 4 insertions(+), 1 deletion(-) >>>> >>>> diff --git a/hw/core/machine.c b/hw/core/machine.c >>>> index 3e288bfceb..8bc401d8b7 100644 >>>> --- a/hw/core/machine.c >>>> +++ b/hw/core/machine.c >>>> @@ -28,6 +28,9 @@ >>>> #include "hw/mem/nvdimm.h" >>>> GlobalProperty hw_compat_4_2[] = { >>>> + { "virtio-blk-device", "queue-size", "128"}, >>>> + { "virtio-scsi-device", "virtqueue_size", "128"}, >>>> + { "vhost-blk-device", "virtqueue_size", "128"}, >>> vhost-blk-device?! Who has this? It's not in qemu.git so please omit >>> this line. ;-) >> So in this case the line: >> >> { "vhost-blk-device", "seg_max_adjust", "off"}, >> >> introduced by my patch: >> >> commit 1bf8a989a566b2ba41c197004ec2a02562a766a4 >> Author: Denis Plotnikov <dplotnikov@virtuozzo.com> >> Date: Fri Dec 20 17:09:04 2019 +0300 >> >> virtio: make seg_max virtqueue size dependent >> >> is also wrong. It should be: >> >> { "vhost-scsi-device", "seg_max_adjust", "off"}, >> >> Am I right? > It's just called "vhost-scsi": > > include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi" > >>> On the other hand, do you want to do this for the vhost-user-blk, >>> vhost-user-scsi, and vhost-scsi devices that exist in qemu.git? Those >>> devices would benefit from better performance too. After thinking about that for a while, I think we shouldn't extend queue sizes for vhost-user-blk, vhost-user-scsi and vhost-scsi. This is because increasing the queue sizes seems to be just useless for them: the all thing is about increasing the queue sizes for increasing seg_max (it limits the max block query size from the guest). For virtio-blk-device and virtio-scsi-device it makes sense, since they have seg-max-adjust property which, if true, sets seg_max to virtqueue_size-2. vhost-scsi also have this property but it seems the property just doesn't affect anything (remove it?). Also vhost-user-blk, vhost-user-scsi and vhost-scsi don't do any seg_max settings. If I understand correctly, their backends are ment to be responsible for doing that. So, what about changing the queue sizes just for virtio-blk-device and virtio-scsi-device? Denis >> It seems to be so. We also have the test checking those settings: >> tests/acceptance/virtio_seg_max_adjust.py >> For now it checks virtio-scsi-pci and virtio-blk-pci. >> I'm going to extend it for the virtqueue size checking. >> If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth >> to check those devices too. But I don't know how to form a command line >> for that 3 devices since they should involve some third party components as >> backends (kernel modules, DPDK, etc.) and they seems to be not available in >> the >> qemu git. >> Is there any way to do it with some qit.qemu available stubs or something >> else? >> If so, could you please point out the proper way to do it? > qemu.git has contrib/vhost-user-blk/ and contrib/vhost-user-scsi/ if > you need to test those vhost-user devices without external dependencies. > > Stefan
On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote: > > > On 05.02.2020 14:19, Stefan Hajnoczi wrote: > > On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote: > > > > > > On 30.01.2020 17:58, Stefan Hajnoczi wrote: > > > > On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote: > > > > > The goal is to reduce the amount of requests issued by a guest on > > > > > 1M reads/writes. This rises the performance up to 4% on that kind of > > > > > disk access pattern. > > > > > > > > > > The maximum chunk size to be used for the guest disk accessing is > > > > > limited with seg_max parameter, which represents the max amount of > > > > > pices in the scatter-geather list in one guest disk request. > > > > > > > > > > Since seg_max is virqueue_size dependent, increasing the virtqueue > > > > > size increases seg_max, which, in turn, increases the maximum size > > > > > of data to be read/write from guest disk. > > > > > > > > > > More details in the original problem statment: > > > > > https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html > > > > > > > > > > Suggested-by: Denis V. Lunev <den@openvz.org> > > > > > Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> > > > > > --- > > > > > hw/core/machine.c | 3 +++ > > > > > include/hw/virtio/virtio.h | 2 +- > > > > > 2 files changed, 4 insertions(+), 1 deletion(-) > > > > > > > > > > diff --git a/hw/core/machine.c b/hw/core/machine.c > > > > > index 3e288bfceb..8bc401d8b7 100644 > > > > > --- a/hw/core/machine.c > > > > > +++ b/hw/core/machine.c > > > > > @@ -28,6 +28,9 @@ > > > > > #include "hw/mem/nvdimm.h" > > > > > GlobalProperty hw_compat_4_2[] = { > > > > > + { "virtio-blk-device", "queue-size", "128"}, > > > > > + { "virtio-scsi-device", "virtqueue_size", "128"}, > > > > > + { "vhost-blk-device", "virtqueue_size", "128"}, > > > > vhost-blk-device?! Who has this? It's not in qemu.git so please omit > > > > this line. ;-) > > > So in this case the line: > > > > > > { "vhost-blk-device", "seg_max_adjust", "off"}, > > > > > > introduced by my patch: > > > > > > commit 1bf8a989a566b2ba41c197004ec2a02562a766a4 > > > Author: Denis Plotnikov <dplotnikov@virtuozzo.com> > > > Date: Fri Dec 20 17:09:04 2019 +0300 > > > > > > virtio: make seg_max virtqueue size dependent > > > > > > is also wrong. It should be: > > > > > > { "vhost-scsi-device", "seg_max_adjust", "off"}, > > > > > > Am I right? > > It's just called "vhost-scsi": > > > > include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi" > > > > > > On the other hand, do you want to do this for the vhost-user-blk, > > > > vhost-user-scsi, and vhost-scsi devices that exist in qemu.git? Those > > > > devices would benefit from better performance too. > After thinking about that for a while, I think we shouldn't extend queue > sizes for vhost-user-blk, vhost-user-scsi and vhost-scsi. > This is because increasing the queue sizes seems to be just useless for > them: the all thing is about increasing the queue sizes for increasing > seg_max (it limits the max block query size from the guest). For > virtio-blk-device and virtio-scsi-device it makes sense, since they have > seg-max-adjust property which, if true, sets seg_max to virtqueue_size-2. > vhost-scsi also have this property but it seems the property just doesn't > affect anything (remove it?). > Also vhost-user-blk, vhost-user-scsi and vhost-scsi don't do any seg_max > settings. If I understand correctly, their backends are ment to be > responsible for doing that. > So, what about changing the queue sizes just for virtio-blk-device and > virtio-scsi-device? That's fine. If it's beneficial to extend it to the vhost devices then it can be done later. I didn't look into it (and the way that responsibility for these parameters is shared between QEMU and the vhost-user device backend is a little complicated). Stefan
On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote: > > > On 05.02.2020 14:19, Stefan Hajnoczi wrote: > > On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote: > > > > > > On 30.01.2020 17:58, Stefan Hajnoczi wrote: > > > > On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote: > > > > > The goal is to reduce the amount of requests issued by a guest on > > > > > 1M reads/writes. This rises the performance up to 4% on that kind of > > > > > disk access pattern. > > > > > > > > > > The maximum chunk size to be used for the guest disk accessing is > > > > > limited with seg_max parameter, which represents the max amount of > > > > > pices in the scatter-geather list in one guest disk request. > > > > > > > > > > Since seg_max is virqueue_size dependent, increasing the virtqueue > > > > > size increases seg_max, which, in turn, increases the maximum size > > > > > of data to be read/write from guest disk. > > > > > > > > > > More details in the original problem statment: > > > > > https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html > > > > > > > > > > Suggested-by: Denis V. Lunev <den@openvz.org> > > > > > Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> > > > > > --- > > > > > hw/core/machine.c | 3 +++ > > > > > include/hw/virtio/virtio.h | 2 +- > > > > > 2 files changed, 4 insertions(+), 1 deletion(-) > > > > > > > > > > diff --git a/hw/core/machine.c b/hw/core/machine.c > > > > > index 3e288bfceb..8bc401d8b7 100644 > > > > > --- a/hw/core/machine.c > > > > > +++ b/hw/core/machine.c > > > > > @@ -28,6 +28,9 @@ > > > > > #include "hw/mem/nvdimm.h" > > > > > GlobalProperty hw_compat_4_2[] = { > > > > > + { "virtio-blk-device", "queue-size", "128"}, > > > > > + { "virtio-scsi-device", "virtqueue_size", "128"}, > > > > > + { "vhost-blk-device", "virtqueue_size", "128"}, > > > > vhost-blk-device?! Who has this? It's not in qemu.git so please omit > > > > this line. ;-) > > > So in this case the line: > > > > > > { "vhost-blk-device", "seg_max_adjust", "off"}, > > > > > > introduced by my patch: > > > > > > commit 1bf8a989a566b2ba41c197004ec2a02562a766a4 > > > Author: Denis Plotnikov <dplotnikov@virtuozzo.com> > > > Date: Fri Dec 20 17:09:04 2019 +0300 > > > > > > virtio: make seg_max virtqueue size dependent > > > > > > is also wrong. It should be: > > > > > > { "vhost-scsi-device", "seg_max_adjust", "off"}, > > > > > > Am I right? > > It's just called "vhost-scsi": > > > > include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi" > > > > > > On the other hand, do you want to do this for the vhost-user-blk, > > > > vhost-user-scsi, and vhost-scsi devices that exist in qemu.git? Those > > > > devices would benefit from better performance too. > After thinking about that for a while, I think we shouldn't extend queue > sizes for vhost-user-blk, vhost-user-scsi and vhost-scsi. > This is because increasing the queue sizes seems to be just useless for > them: the all thing is about increasing the queue sizes for increasing > seg_max (it limits the max block query size from the guest). For > virtio-blk-device and virtio-scsi-device it makes sense, since they have > seg-max-adjust property which, if true, sets seg_max to virtqueue_size-2. > vhost-scsi also have this property but it seems the property just doesn't > affect anything (remove it?). > Also vhost-user-blk, vhost-user-scsi and vhost-scsi don't do any seg_max > settings. If I understand correctly, their backends are ment to be > responsible for doing that. The queue size is set by qemu IIRC. > So, what about changing the queue sizes just for virtio-blk-device and > virtio-scsi-device? Hmm that would break ability to migrate between userspace and vhost backends, would it not? > Denis > > > > It seems to be so. We also have the test checking those settings: > > > tests/acceptance/virtio_seg_max_adjust.py > > > For now it checks virtio-scsi-pci and virtio-blk-pci. > > > I'm going to extend it for the virtqueue size checking. > > > If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth > > > to check those devices too. But I don't know how to form a command line > > > for that 3 devices since they should involve some third party components as > > > backends (kernel modules, DPDK, etc.) and they seems to be not available in > > > the > > > qemu git. > > > Is there any way to do it with some qit.qemu available stubs or something > > > else? > > > If so, could you please point out the proper way to do it? > > qemu.git has contrib/vhost-user-blk/ and contrib/vhost-user-scsi/ if > > you need to test those vhost-user devices without external dependencies. > > > > Stefan
On 09.02.2020 10:49, Michael S. Tsirkin wrote: > On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote: >> >> On 05.02.2020 14:19, Stefan Hajnoczi wrote: >>> On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote: >>>> On 30.01.2020 17:58, Stefan Hajnoczi wrote: >>>>> On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote: >>>>>> The goal is to reduce the amount of requests issued by a guest on >>>>>> 1M reads/writes. This rises the performance up to 4% on that kind of >>>>>> disk access pattern. >>>>>> >>>>>> The maximum chunk size to be used for the guest disk accessing is >>>>>> limited with seg_max parameter, which represents the max amount of >>>>>> pices in the scatter-geather list in one guest disk request. >>>>>> >>>>>> Since seg_max is virqueue_size dependent, increasing the virtqueue >>>>>> size increases seg_max, which, in turn, increases the maximum size >>>>>> of data to be read/write from guest disk. >>>>>> >>>>>> More details in the original problem statment: >>>>>> https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html >>>>>> >>>>>> Suggested-by: Denis V. Lunev <den@openvz.org> >>>>>> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> >>>>>> --- >>>>>> hw/core/machine.c | 3 +++ >>>>>> include/hw/virtio/virtio.h | 2 +- >>>>>> 2 files changed, 4 insertions(+), 1 deletion(-) >>>>>> >>>>>> diff --git a/hw/core/machine.c b/hw/core/machine.c >>>>>> index 3e288bfceb..8bc401d8b7 100644 >>>>>> --- a/hw/core/machine.c >>>>>> +++ b/hw/core/machine.c >>>>>> @@ -28,6 +28,9 @@ >>>>>> #include "hw/mem/nvdimm.h" >>>>>> GlobalProperty hw_compat_4_2[] = { >>>>>> + { "virtio-blk-device", "queue-size", "128"}, >>>>>> + { "virtio-scsi-device", "virtqueue_size", "128"}, >>>>>> + { "vhost-blk-device", "virtqueue_size", "128"}, >>>>> vhost-blk-device?! Who has this? It's not in qemu.git so please omit >>>>> this line. ;-) >>>> So in this case the line: >>>> >>>> { "vhost-blk-device", "seg_max_adjust", "off"}, >>>> >>>> introduced by my patch: >>>> >>>> commit 1bf8a989a566b2ba41c197004ec2a02562a766a4 >>>> Author: Denis Plotnikov <dplotnikov@virtuozzo.com> >>>> Date: Fri Dec 20 17:09:04 2019 +0300 >>>> >>>> virtio: make seg_max virtqueue size dependent >>>> >>>> is also wrong. It should be: >>>> >>>> { "vhost-scsi-device", "seg_max_adjust", "off"}, >>>> >>>> Am I right? >>> It's just called "vhost-scsi": >>> >>> include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi" >>> >>>>> On the other hand, do you want to do this for the vhost-user-blk, >>>>> vhost-user-scsi, and vhost-scsi devices that exist in qemu.git? Those >>>>> devices would benefit from better performance too. >> After thinking about that for a while, I think we shouldn't extend queue >> sizes for vhost-user-blk, vhost-user-scsi and vhost-scsi. >> This is because increasing the queue sizes seems to be just useless for >> them: the all thing is about increasing the queue sizes for increasing >> seg_max (it limits the max block query size from the guest). For >> virtio-blk-device and virtio-scsi-device it makes sense, since they have >> seg-max-adjust property which, if true, sets seg_max to virtqueue_size-2. >> vhost-scsi also have this property but it seems the property just doesn't >> affect anything (remove it?). >> Also vhost-user-blk, vhost-user-scsi and vhost-scsi don't do any seg_max >> settings. If I understand correctly, their backends are ment to be >> responsible for doing that. > The queue size is set by qemu IIRC. > >> So, what about changing the queue sizes just for virtio-blk-device and >> virtio-scsi-device? > > Hmm that would break ability to migrate between userspace and vhost > backends, would it not? I'm not sure I've understood what you meant. Just for the record, I was going to change virtqueue-size for virtio-blk-device and virtio-scsi-device since they can adjust seg_max to the specified queue size and I don't want to touch vhost-s and vhost-user-s since they don't have adjustable seg_max for now. Denis > > >> Denis >> >>>> It seems to be so. We also have the test checking those settings: >>>> tests/acceptance/virtio_seg_max_adjust.py >>>> For now it checks virtio-scsi-pci and virtio-blk-pci. >>>> I'm going to extend it for the virtqueue size checking. >>>> If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth >>>> to check those devices too. But I don't know how to form a command line >>>> for that 3 devices since they should involve some third party components as >>>> backends (kernel modules, DPDK, etc.) and they seems to be not available in >>>> the >>>> qemu git. >>>> Is there any way to do it with some qit.qemu available stubs or something >>>> else? >>>> If so, could you please point out the proper way to do it? >>> qemu.git has contrib/vhost-user-blk/ and contrib/vhost-user-scsi/ if >>> you need to test those vhost-user devices without external dependencies. >>> >>> Stefan
On Sun, Feb 09, 2020 at 02:49:09AM -0500, Michael S. Tsirkin wrote: > On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote: > > On 05.02.2020 14:19, Stefan Hajnoczi wrote: > > > On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote: > > > > On 30.01.2020 17:58, Stefan Hajnoczi wrote: > > > > > On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote: > > So, what about changing the queue sizes just for virtio-blk-device and > > virtio-scsi-device? > > Hmm that would break ability to migrate between userspace and vhost > backends, would it not? I think live and offline migration between them would still work as long as the queue size is manually specified. Stefan
On Mon, Feb 10, 2020 at 06:34:15PM +0300, Denis Plotnikov wrote: > > > On 09.02.2020 10:49, Michael S. Tsirkin wrote: > > On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote: > > > > > > On 05.02.2020 14:19, Stefan Hajnoczi wrote: > > > > On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote: > > > > > On 30.01.2020 17:58, Stefan Hajnoczi wrote: > > > > > > On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote: > > > > > > > The goal is to reduce the amount of requests issued by a guest on > > > > > > > 1M reads/writes. This rises the performance up to 4% on that kind of > > > > > > > disk access pattern. > > > > > > > > > > > > > > The maximum chunk size to be used for the guest disk accessing is > > > > > > > limited with seg_max parameter, which represents the max amount of > > > > > > > pices in the scatter-geather list in one guest disk request. > > > > > > > > > > > > > > Since seg_max is virqueue_size dependent, increasing the virtqueue > > > > > > > size increases seg_max, which, in turn, increases the maximum size > > > > > > > of data to be read/write from guest disk. > > > > > > > > > > > > > > More details in the original problem statment: > > > > > > > https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html > > > > > > > > > > > > > > Suggested-by: Denis V. Lunev <den@openvz.org> > > > > > > > Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> > > > > > > > --- > > > > > > > hw/core/machine.c | 3 +++ > > > > > > > include/hw/virtio/virtio.h | 2 +- > > > > > > > 2 files changed, 4 insertions(+), 1 deletion(-) > > > > > > > > > > > > > > diff --git a/hw/core/machine.c b/hw/core/machine.c > > > > > > > index 3e288bfceb..8bc401d8b7 100644 > > > > > > > --- a/hw/core/machine.c > > > > > > > +++ b/hw/core/machine.c > > > > > > > @@ -28,6 +28,9 @@ > > > > > > > #include "hw/mem/nvdimm.h" > > > > > > > GlobalProperty hw_compat_4_2[] = { > > > > > > > + { "virtio-blk-device", "queue-size", "128"}, > > > > > > > + { "virtio-scsi-device", "virtqueue_size", "128"}, > > > > > > > + { "vhost-blk-device", "virtqueue_size", "128"}, > > > > > > vhost-blk-device?! Who has this? It's not in qemu.git so please omit > > > > > > this line. ;-) > > > > > So in this case the line: > > > > > > > > > > { "vhost-blk-device", "seg_max_adjust", "off"}, > > > > > > > > > > introduced by my patch: > > > > > > > > > > commit 1bf8a989a566b2ba41c197004ec2a02562a766a4 > > > > > Author: Denis Plotnikov <dplotnikov@virtuozzo.com> > > > > > Date: Fri Dec 20 17:09:04 2019 +0300 > > > > > > > > > > virtio: make seg_max virtqueue size dependent > > > > > > > > > > is also wrong. It should be: > > > > > > > > > > { "vhost-scsi-device", "seg_max_adjust", "off"}, > > > > > > > > > > Am I right? > > > > It's just called "vhost-scsi": > > > > > > > > include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi" > > > > > > > > > > On the other hand, do you want to do this for the vhost-user-blk, > > > > > > vhost-user-scsi, and vhost-scsi devices that exist in qemu.git? Those > > > > > > devices would benefit from better performance too. > > > After thinking about that for a while, I think we shouldn't extend queue > > > sizes for vhost-user-blk, vhost-user-scsi and vhost-scsi. > > > This is because increasing the queue sizes seems to be just useless for > > > them: the all thing is about increasing the queue sizes for increasing > > > seg_max (it limits the max block query size from the guest). For > > > virtio-blk-device and virtio-scsi-device it makes sense, since they have > > > seg-max-adjust property which, if true, sets seg_max to virtqueue_size-2. > > > vhost-scsi also have this property but it seems the property just doesn't > > > affect anything (remove it?). > > > Also vhost-user-blk, vhost-user-scsi and vhost-scsi don't do any seg_max > > > settings. If I understand correctly, their backends are ment to be > > > responsible for doing that. > > The queue size is set by qemu IIRC. > > > > > So, what about changing the queue sizes just for virtio-blk-device and > > > virtio-scsi-device? > > > > Hmm that would break ability to migrate between userspace and vhost > > backends, would it not? > I'm not sure I've understood what you meant. > Just for the record, I was going to change virtqueue-size for > virtio-blk-device and virtio-scsi-device If virtqueue size is different between virtio-blk-device and vhost-user-blk then one can not migrate the former to the later. > since they can adjust seg_max to > the specified queue size and I don't want to touch vhost-s and vhost-user-s > since they don't have adjustable seg_max for now. > > Denis If I just grep seg_max_adjust I see them for vhost devices too. Is that code ineffective somehow? What's it doing then? > > > > > > > Denis > > > > > > > > It seems to be so. We also have the test checking those settings: > > > > > tests/acceptance/virtio_seg_max_adjust.py > > > > > For now it checks virtio-scsi-pci and virtio-blk-pci. > > > > > I'm going to extend it for the virtqueue size checking. > > > > > If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth > > > > > to check those devices too. But I don't know how to form a command line > > > > > for that 3 devices since they should involve some third party components as > > > > > backends (kernel modules, DPDK, etc.) and they seems to be not available in > > > > > the > > > > > qemu git. > > > > > Is there any way to do it with some qit.qemu available stubs or something > > > > > else? > > > > > If so, could you please point out the proper way to do it? > > > > qemu.git has contrib/vhost-user-blk/ and contrib/vhost-user-scsi/ if > > > > you need to test those vhost-user devices without external dependencies. > > > > > > > > Stefan
Hi Denis,
I just found this discussion since we still have the following line in
hw/core/machine.c:
{ "vhost-blk-device", "seg_max_adjust", "off"}
IIUC it was a typo, and I think we should fix it since in the future we
can have `vhost-blk-device`.
So, I think we have 2 options:
1. remove that line since for now is useless
2. replace with "vhost-scsi"
I'm not sure which is the best, what do you suggest?
Thanks,
Stefano
On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote:
>On 05.02.2020 14:19, Stefan Hajnoczi wrote:
>>On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote:
>>>On 30.01.2020 17:58, Stefan Hajnoczi wrote:
>>>>On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote:
>>>>>The goal is to reduce the amount of requests issued by a guest on
>>>>>1M reads/writes. This rises the performance up to 4% on that kind of
>>>>>disk access pattern.
>>>>>
>>>>>The maximum chunk size to be used for the guest disk accessing is
>>>>>limited with seg_max parameter, which represents the max amount of
>>>>>pices in the scatter-geather list in one guest disk request.
>>>>>
>>>>>Since seg_max is virqueue_size dependent, increasing the virtqueue
>>>>>size increases seg_max, which, in turn, increases the maximum size
>>>>>of data to be read/write from guest disk.
>>>>>
>>>>>More details in the original problem statment:
>>>>>https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html
>>>>>
>>>>>Suggested-by: Denis V. Lunev <den@openvz.org>
>>>>>Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
>>>>>---
>>>>> hw/core/machine.c | 3 +++
>>>>> include/hw/virtio/virtio.h | 2 +-
>>>>> 2 files changed, 4 insertions(+), 1 deletion(-)
>>>>>
>>>>>diff --git a/hw/core/machine.c b/hw/core/machine.c
>>>>>index 3e288bfceb..8bc401d8b7 100644
>>>>>--- a/hw/core/machine.c
>>>>>+++ b/hw/core/machine.c
>>>>>@@ -28,6 +28,9 @@
>>>>> #include "hw/mem/nvdimm.h"
>>>>> GlobalProperty hw_compat_4_2[] = {
>>>>>+ { "virtio-blk-device", "queue-size", "128"},
>>>>>+ { "virtio-scsi-device", "virtqueue_size", "128"},
>>>>>+ { "vhost-blk-device", "virtqueue_size", "128"},
>>>>vhost-blk-device?! Who has this? It's not in qemu.git so please omit
>>>>this line. ;-)
>>>So in this case the line:
>>>
>>>{ "vhost-blk-device", "seg_max_adjust", "off"},
>>>
>>>introduced by my patch:
>>>
>>>commit 1bf8a989a566b2ba41c197004ec2a02562a766a4
>>>Author: Denis Plotnikov <dplotnikov@virtuozzo.com>
>>>Date: Fri Dec 20 17:09:04 2019 +0300
>>>
>>> virtio: make seg_max virtqueue size dependent
>>>
>>>is also wrong. It should be:
>>>
>>>{ "vhost-scsi-device", "seg_max_adjust", "off"},
>>>
>>>Am I right?
>>It's just called "vhost-scsi":
>>
>>include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi"
>>
>>>>On the other hand, do you want to do this for the vhost-user-blk,
>>>>vhost-user-scsi, and vhost-scsi devices that exist in qemu.git? Those
>>>>devices would benefit from better performance too.
>After thinking about that for a while, I think we shouldn't extend
>queue sizes for vhost-user-blk, vhost-user-scsi and vhost-scsi.
>This is because increasing the queue sizes seems to be just useless
>for them: the all thing is about increasing the queue sizes for
>increasing seg_max (it limits the max block query size from the
>guest). For virtio-blk-device and virtio-scsi-device it makes sense,
>since they have seg-max-adjust property which, if true, sets seg_max
>to virtqueue_size-2. vhost-scsi also have this property but it seems
>the property just doesn't affect anything (remove it?).
>Also vhost-user-blk, vhost-user-scsi and vhost-scsi don't do any
>seg_max settings. If I understand correctly, their backends are ment
>to be responsible for doing that.
>So, what about changing the queue sizes just for virtio-blk-device and
>virtio-scsi-device?
>
>Denis
>
>>>It seems to be so. We also have the test checking those settings:
>>>tests/acceptance/virtio_seg_max_adjust.py
>>>For now it checks virtio-scsi-pci and virtio-blk-pci.
>>>I'm going to extend it for the virtqueue size checking.
>>>If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth
>>>to check those devices too. But I don't know how to form a command line
>>>for that 3 devices since they should involve some third party components as
>>>backends (kernel modules, DPDK, etc.) and they seems to be not available in
>>>the
>>>qemu git.
>>>Is there any way to do it with some qit.qemu available stubs or something
>>>else?
>>>If so, could you please point out the proper way to do it?
>>qemu.git has contrib/vhost-user-blk/ and contrib/vhost-user-scsi/ if
>>you need to test those vhost-user devices without external dependencies.
>>
>>Stefan
>
>
Message bounced, I use new Denis's email address. On Wed, Sep 08, 2021 at 03:17:16PM +0200, Stefano Garzarella wrote: >Hi Denis, >I just found this discussion since we still have the following line in >hw/core/machine.c: > { "vhost-blk-device", "seg_max_adjust", "off"} > >IIUC it was a typo, and I think we should fix it since in the future >we can have `vhost-blk-device`. > >So, I think we have 2 options: >1. remove that line since for now is useless >2. replace with "vhost-scsi" > >I'm not sure which is the best, what do you suggest? > >Thanks, >Stefano > >On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote: >>On 05.02.2020 14:19, Stefan Hajnoczi wrote: >>>On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote: >>>>On 30.01.2020 17:58, Stefan Hajnoczi wrote: >>>>>On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote: >>>>>>The goal is to reduce the amount of requests issued by a guest on >>>>>>1M reads/writes. This rises the performance up to 4% on that kind of >>>>>>disk access pattern. >>>>>> >>>>>>The maximum chunk size to be used for the guest disk accessing is >>>>>>limited with seg_max parameter, which represents the max amount of >>>>>>pices in the scatter-geather list in one guest disk request. >>>>>> >>>>>>Since seg_max is virqueue_size dependent, increasing the virtqueue >>>>>>size increases seg_max, which, in turn, increases the maximum size >>>>>>of data to be read/write from guest disk. >>>>>> >>>>>>More details in the original problem statment: >>>>>>https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html >>>>>> >>>>>>Suggested-by: Denis V. Lunev <den@openvz.org> >>>>>>Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> >>>>>>--- >>>>>> hw/core/machine.c | 3 +++ >>>>>> include/hw/virtio/virtio.h | 2 +- >>>>>> 2 files changed, 4 insertions(+), 1 deletion(-) >>>>>> >>>>>>diff --git a/hw/core/machine.c b/hw/core/machine.c >>>>>>index 3e288bfceb..8bc401d8b7 100644 >>>>>>--- a/hw/core/machine.c >>>>>>+++ b/hw/core/machine.c >>>>>>@@ -28,6 +28,9 @@ >>>>>> #include "hw/mem/nvdimm.h" >>>>>> GlobalProperty hw_compat_4_2[] = { >>>>>>+ { "virtio-blk-device", "queue-size", "128"}, >>>>>>+ { "virtio-scsi-device", "virtqueue_size", "128"}, >>>>>>+ { "vhost-blk-device", "virtqueue_size", "128"}, >>>>>vhost-blk-device?! Who has this? It's not in qemu.git so please omit >>>>>this line. ;-) >>>>So in this case the line: >>>> >>>>{ "vhost-blk-device", "seg_max_adjust", "off"}, >>>> >>>>introduced by my patch: >>>> >>>>commit 1bf8a989a566b2ba41c197004ec2a02562a766a4 >>>>Author: Denis Plotnikov <dplotnikov@virtuozzo.com> >>>>Date: Fri Dec 20 17:09:04 2019 +0300 >>>> >>>> virtio: make seg_max virtqueue size dependent >>>> >>>>is also wrong. It should be: >>>> >>>>{ "vhost-scsi-device", "seg_max_adjust", "off"}, >>>> >>>>Am I right? >>>It's just called "vhost-scsi": >>> >>>include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi" >>> >>>>>On the other hand, do you want to do this for the vhost-user-blk, >>>>>vhost-user-scsi, and vhost-scsi devices that exist in qemu.git? Those >>>>>devices would benefit from better performance too. >>After thinking about that for a while, I think we shouldn't extend >>queue sizes for vhost-user-blk, vhost-user-scsi and vhost-scsi. >>This is because increasing the queue sizes seems to be just useless >>for them: the all thing is about increasing the queue sizes for >>increasing seg_max (it limits the max block query size from the >>guest). For virtio-blk-device and virtio-scsi-device it makes sense, >>since they have seg-max-adjust property which, if true, sets seg_max >>to virtqueue_size-2. vhost-scsi also have this property but it seems >>the property just doesn't affect anything (remove it?). >>Also vhost-user-blk, vhost-user-scsi and vhost-scsi don't do any >>seg_max settings. If I understand correctly, their backends are ment >>to be responsible for doing that. >>So, what about changing the queue sizes just for virtio-blk-device >>and virtio-scsi-device? >> >>Denis >> >>>>It seems to be so. We also have the test checking those settings: >>>>tests/acceptance/virtio_seg_max_adjust.py >>>>For now it checks virtio-scsi-pci and virtio-blk-pci. >>>>I'm going to extend it for the virtqueue size checking. >>>>If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth >>>>to check those devices too. But I don't know how to form a command line >>>>for that 3 devices since they should involve some third party components as >>>>backends (kernel modules, DPDK, etc.) and they seems to be not available in >>>>the >>>>qemu git. >>>>Is there any way to do it with some qit.qemu available stubs or something >>>>else? >>>>If so, could you please point out the proper way to do it? >>>qemu.git has contrib/vhost-user-blk/ and contrib/vhost-user-scsi/ if >>>you need to test those vhost-user devices without external dependencies. >>> >>>Stefan >> >>
On 08.09.2021 16:22, Stefano Garzarella wrote: > Message bounced, I use new Denis's email address. > > On Wed, Sep 08, 2021 at 03:17:16PM +0200, Stefano Garzarella wrote: >> Hi Denis, >> I just found this discussion since we still have the following line >> in hw/core/machine.c: >> { "vhost-blk-device", "seg_max_adjust", "off"} >> >> IIUC it was a typo, and I think we should fix it since in the future >> we can have `vhost-blk-device`. >> >> So, I think we have 2 options: >> 1. remove that line since for now is useless >> 2. replace with "vhost-scsi" >> >> I'm not sure which is the best, what do you suggest? >> >> Thanks, >> Stefano Hi Stefano I prefer to just remove the line without replacing. This will keep things exactly like it is now. Replacing with "vhost-scsi" will affect seg_max (limit to 126) on newly created VMs with machine types using "hw_compat_4_2" and older. Now, because of the typo (error), their seg-max is queue-size dependent. I'm not sure, if replacing the line may cause any problems, for example on migration: source (queue-size 256, seg max 254) -> destination (queue-size 256, seg max 126). But it will definitely introduce two different behaviors for VMs with "hw_compat_4_2" and older. So, I'd choose the lesser of two evils and keep the things like it's now. Thanks! Denis >> >> On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote: >>> On 05.02.2020 14:19, Stefan Hajnoczi wrote: >>>> On Tue, Feb 04, 2020 at 12:59:04PM +0300, Denis Plotnikov wrote: >>>>> On 30.01.2020 17:58, Stefan Hajnoczi wrote: >>>>>> On Wed, Jan 29, 2020 at 05:07:00PM +0300, Denis Plotnikov wrote: >>>>>>> The goal is to reduce the amount of requests issued by a guest on >>>>>>> 1M reads/writes. This rises the performance up to 4% on that >>>>>>> kind of >>>>>>> disk access pattern. >>>>>>> >>>>>>> The maximum chunk size to be used for the guest disk accessing is >>>>>>> limited with seg_max parameter, which represents the max amount of >>>>>>> pices in the scatter-geather list in one guest disk request. >>>>>>> >>>>>>> Since seg_max is virqueue_size dependent, increasing the virtqueue >>>>>>> size increases seg_max, which, in turn, increases the maximum size >>>>>>> of data to be read/write from guest disk. >>>>>>> >>>>>>> More details in the original problem statment: >>>>>>> https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html >>>>>>> >>>>>>> Suggested-by: Denis V. Lunev <den@openvz.org> >>>>>>> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> >>>>>>> --- >>>>>>> hw/core/machine.c | 3 +++ >>>>>>> include/hw/virtio/virtio.h | 2 +- >>>>>>> 2 files changed, 4 insertions(+), 1 deletion(-) >>>>>>> >>>>>>> diff --git a/hw/core/machine.c b/hw/core/machine.c >>>>>>> index 3e288bfceb..8bc401d8b7 100644 >>>>>>> --- a/hw/core/machine.c >>>>>>> +++ b/hw/core/machine.c >>>>>>> @@ -28,6 +28,9 @@ >>>>>>> #include "hw/mem/nvdimm.h" >>>>>>> GlobalProperty hw_compat_4_2[] = { >>>>>>> + { "virtio-blk-device", "queue-size", "128"}, >>>>>>> + { "virtio-scsi-device", "virtqueue_size", "128"}, >>>>>>> + { "vhost-blk-device", "virtqueue_size", "128"}, >>>>>> vhost-blk-device?! Who has this? It's not in qemu.git so please >>>>>> omit >>>>>> this line. ;-) >>>>> So in this case the line: >>>>> >>>>> { "vhost-blk-device", "seg_max_adjust", "off"}, >>>>> >>>>> introduced by my patch: >>>>> >>>>> commit 1bf8a989a566b2ba41c197004ec2a02562a766a4 >>>>> Author: Denis Plotnikov <dplotnikov@virtuozzo.com> >>>>> Date: Fri Dec 20 17:09:04 2019 +0300 >>>>> >>>>> virtio: make seg_max virtqueue size dependent >>>>> >>>>> is also wrong. It should be: >>>>> >>>>> { "vhost-scsi-device", "seg_max_adjust", "off"}, >>>>> >>>>> Am I right? >>>> It's just called "vhost-scsi": >>>> >>>> include/hw/virtio/vhost-scsi.h:#define TYPE_VHOST_SCSI "vhost-scsi" >>>> >>>>>> On the other hand, do you want to do this for the vhost-user-blk, >>>>>> vhost-user-scsi, and vhost-scsi devices that exist in qemu.git? >>>>>> Those >>>>>> devices would benefit from better performance too. >>> After thinking about that for a while, I think we shouldn't extend >>> queue sizes for vhost-user-blk, vhost-user-scsi and vhost-scsi. >>> This is because increasing the queue sizes seems to be just useless >>> for them: the all thing is about increasing the queue sizes for >>> increasing seg_max (it limits the max block query size from the >>> guest). For virtio-blk-device and virtio-scsi-device it makes sense, >>> since they have seg-max-adjust property which, if true, sets seg_max >>> to virtqueue_size-2. vhost-scsi also have this property but it seems >>> the property just doesn't affect anything (remove it?). >>> Also vhost-user-blk, vhost-user-scsi and vhost-scsi don't do any >>> seg_max settings. If I understand correctly, their backends are ment >>> to be responsible for doing that. >>> So, what about changing the queue sizes just for virtio-blk-device >>> and virtio-scsi-device? >>> >>> Denis >>> >>>>> It seems to be so. We also have the test checking those settings: >>>>> tests/acceptance/virtio_seg_max_adjust.py >>>>> For now it checks virtio-scsi-pci and virtio-blk-pci. >>>>> I'm going to extend it for the virtqueue size checking. >>>>> If I change vhost-user-blk, vhost-user-scsi and vhost-scsi it's worth >>>>> to check those devices too. But I don't know how to form a command >>>>> line >>>>> for that 3 devices since they should involve some third party >>>>> components as >>>>> backends (kernel modules, DPDK, etc.) and they seems to be not >>>>> available in >>>>> the >>>>> qemu git. >>>>> Is there any way to do it with some qit.qemu available stubs or >>>>> something >>>>> else? >>>>> If so, could you please point out the proper way to do it? >>>> qemu.git has contrib/vhost-user-blk/ and contrib/vhost-user-scsi/ if >>>> you need to test those vhost-user devices without external >>>> dependencies. >>>> >>>> Stefan >>> >>> >
On Wed, Sep 08, 2021 at 06:20:49PM +0300, Denis Plotnikov wrote: > >On 08.09.2021 16:22, Stefano Garzarella wrote: >>Message bounced, I use new Denis's email address. >> >>On Wed, Sep 08, 2021 at 03:17:16PM +0200, Stefano Garzarella wrote: >>>Hi Denis, >>>I just found this discussion since we still have the following >>>line in hw/core/machine.c: >>> { "vhost-blk-device", "seg_max_adjust", "off"} >>> >>>IIUC it was a typo, and I think we should fix it since in the >>>future we can have `vhost-blk-device`. >>> >>>So, I think we have 2 options: >>>1. remove that line since for now is useless >>>2. replace with "vhost-scsi" >>> >>>I'm not sure which is the best, what do you suggest? >>> >>>Thanks, >>>Stefano > >Hi Stefano > >I prefer to just remove the line without replacing. This will keep >things exactly like it is now. > >Replacing with "vhost-scsi" will affect seg_max (limit to 126) on >newly created VMs with machine types using "hw_compat_4_2" and older. > >Now, because of the typo (error), their seg-max is queue-size >dependent. I'm not sure, if replacing the line may cause any problems, >for example on migration: source (queue-size 256, seg max 254) -> >destination (queue-size 256, seg max 126). But it will definitely >introduce two different behaviors for VMs with "hw_compat_4_2" and >older. So, I'd choose the lesser of two evils and keep the things like >it's now. > Yep, make sense. It was the same concern I had. Do you want to send a patch to remove it with this explanation? Thanks for the clarification, Stefano
On 09.09.2021 11:28, Stefano Garzarella wrote: > On Wed, Sep 08, 2021 at 06:20:49PM +0300, Denis Plotnikov wrote: >> >> On 08.09.2021 16:22, Stefano Garzarella wrote: >>> Message bounced, I use new Denis's email address. >>> >>> On Wed, Sep 08, 2021 at 03:17:16PM +0200, Stefano Garzarella wrote: >>>> Hi Denis, >>>> I just found this discussion since we still have the following line >>>> in hw/core/machine.c: >>>> { "vhost-blk-device", "seg_max_adjust", "off"} >>>> >>>> IIUC it was a typo, and I think we should fix it since in the >>>> future we can have `vhost-blk-device`. >>>> >>>> So, I think we have 2 options: >>>> 1. remove that line since for now is useless >>>> 2. replace with "vhost-scsi" >>>> >>>> I'm not sure which is the best, what do you suggest? >>>> >>>> Thanks, >>>> Stefano >> >> Hi Stefano >> >> I prefer to just remove the line without replacing. This will keep >> things exactly like it is now. >> >> Replacing with "vhost-scsi" will affect seg_max (limit to 126) on >> newly created VMs with machine types using "hw_compat_4_2" and older. >> >> Now, because of the typo (error), their seg-max is queue-size >> dependent. I'm not sure, if replacing the line may cause any >> problems, for example on migration: source (queue-size 256, seg max >> 254) -> destination (queue-size 256, seg max 126). But it will >> definitely introduce two different behaviors for VMs with >> "hw_compat_4_2" and older. So, I'd choose the lesser of two evils and >> keep the things like it's now. >> > > Yep, make sense. It was the same concern I had. > > Do you want to send a patch to remove it with this explanation? Yes, sure. I'll do it today. Denis > > Thanks for the clarification, > Stefano >
diff --git a/hw/core/machine.c b/hw/core/machine.c index 3e288bfceb..8bc401d8b7 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -28,6 +28,9 @@ #include "hw/mem/nvdimm.h" GlobalProperty hw_compat_4_2[] = { + { "virtio-blk-device", "queue-size", "128"}, + { "virtio-scsi-device", "virtqueue_size", "128"}, + { "vhost-blk-device", "virtqueue_size", "128"}, { "virtio-blk-device", "x-enable-wce-if-config-wce", "off" }, { "virtio-blk-device", "seg-max-adjust", "off"}, { "virtio-scsi-device", "seg_max_adjust", "off"}, diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index a66ea2368b..16d540e390 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -48,7 +48,7 @@ size_t virtio_feature_get_config_size(VirtIOFeature *features, typedef struct VirtQueue VirtQueue; #define VIRTQUEUE_MAX_SIZE 1024 -#define VIRTQUEUE_DEFAULT_SIZE 128 +#define VIRTQUEUE_DEFAULT_SIZE 256 typedef struct VirtQueueElement {
The goal is to reduce the amount of requests issued by a guest on 1M reads/writes. This rises the performance up to 4% on that kind of disk access pattern. The maximum chunk size to be used for the guest disk accessing is limited with seg_max parameter, which represents the max amount of pices in the scatter-geather list in one guest disk request. Since seg_max is virqueue_size dependent, increasing the virtqueue size increases seg_max, which, in turn, increases the maximum size of data to be read/write from guest disk. More details in the original problem statment: https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html Suggested-by: Denis V. Lunev <den@openvz.org> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> --- hw/core/machine.c | 3 +++ include/hw/virtio/virtio.h | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-)