| Message ID | 20210713153758.323614-6-andrew@daynix.com |
|---|---|
| State | New |
| Headers | show |
| Series | ebpf: Added ebpf helper for libvirtd. | expand |
Andrew Melnychenko <andrew@daynix.com> writes: > New qmp command to query ebpf helper. > It's crucial that qemu and helper are in sync and in touch. > Technically helper should pass eBPF fds that qemu may accept. > And different qemu's builds may have different eBPF programs and helpers. > Qemu returns helper that should "fit" to virtio-net. > > Signed-off-by: Andrew Melnychenko <andrew@daynix.com> [...] > diff --git a/qapi/misc.json b/qapi/misc.json > index 156f98203e..9aaf8fbcca 100644 > --- a/qapi/misc.json > +++ b/qapi/misc.json > @@ -519,3 +519,36 @@ > 'data': { '*option': 'str' }, > 'returns': ['CommandLineOptionInfo'], > 'allow-preconfig': true } > + > +## > +# @HelperPath: > +# > +# Name of the helper and binary location. > +## > +{ 'struct': 'HelperPath', > + 'data': {'name': 'str', 'path': 'str'} } > + > +## > +# @query-helper-paths: > +# > +# Query helper paths. Initially, this command was added for > +# qemu-ebpf-rss-helper. The qemu would check "the stamp" and > +# returns proper helper. > +# > +# Returns: list of object that contains name and path for helper. > +# > +# Since: 6.1 > +# > +# Example: > +# > +# -> { "execute": "query-helper-paths" } > +# <- { "return": [ > +# { > +# "name": "qemu-ebpf-rss-helper", > +# "path": "/usr/local/libexec/qemu-ebpf-rss-helper" > +# } > +# ] > +# } > +# > +## > +{ 'command': 'query-helper-paths', 'returns': ['HelperPath'] } Hmm. I understand the desire to help management applications to use the right helper. But I'm not sure this command is actually useful. The helper may or may not be installed at the path compiled into QEMU. What happens when you use the wrong helper? Even if we conclude this is the right approach for this helper, we still need to review the other helpers to see which of them we should have query-helper-paths cover.
Hi, > The helper may or may not be installed at the path compiled into QEMU. > Yes, so the helper will not be called - QEMU will try to initiate eBPF RSS or use "in-qemu" RSS. What happens when you use the wrong helper? > UB - in most cases, eBPF program will work with wrong configurations. That's why the stamp was added. query-helper-paths checks the stamp only for RSS helper. On Sat, Aug 7, 2021 at 3:54 PM Markus Armbruster <armbru@redhat.com> wrote: > Andrew Melnychenko <andrew@daynix.com> writes: > > > New qmp command to query ebpf helper. > > It's crucial that qemu and helper are in sync and in touch. > > Technically helper should pass eBPF fds that qemu may accept. > > And different qemu's builds may have different eBPF programs and helpers. > > Qemu returns helper that should "fit" to virtio-net. > > > > Signed-off-by: Andrew Melnychenko <andrew@daynix.com> > > [...] > > > diff --git a/qapi/misc.json b/qapi/misc.json > > index 156f98203e..9aaf8fbcca 100644 > > --- a/qapi/misc.json > > +++ b/qapi/misc.json > > @@ -519,3 +519,36 @@ > > 'data': { '*option': 'str' }, > > 'returns': ['CommandLineOptionInfo'], > > 'allow-preconfig': true } > > + > > +## > > +# @HelperPath: > > +# > > +# Name of the helper and binary location. > > +## > > +{ 'struct': 'HelperPath', > > + 'data': {'name': 'str', 'path': 'str'} } > > + > > +## > > +# @query-helper-paths: > > +# > > +# Query helper paths. Initially, this command was added for > > +# qemu-ebpf-rss-helper. The qemu would check "the stamp" and > > +# returns proper helper. > > +# > > +# Returns: list of object that contains name and path for helper. > > +# > > +# Since: 6.1 > > +# > > +# Example: > > +# > > +# -> { "execute": "query-helper-paths" } > > +# <- { "return": [ > > +# { > > +# "name": "qemu-ebpf-rss-helper", > > +# "path": "/usr/local/libexec/qemu-ebpf-rss-helper" > > +# } > > +# ] > > +# } > > +# > > +## > > +{ 'command': 'query-helper-paths', 'returns': ['HelperPath'] } > > > Hmm. > > I understand the desire to help management applications to use the right > helper. But I'm not sure this command is actually useful. The helper > may or may not be installed at the path compiled into QEMU. > > What happens when you use the wrong helper? > > Even if we conclude this is the right approach for this helper, we still > need to review the other helpers to see which of them we should have > query-helper-paths cover. > >
Andrew Melnichenko <andrew@daynix.com> writes: > Hi, > >> The helper may or may not be installed at the path compiled into QEMU. >> > Yes, so the helper will not be called - QEMU will try to initiate eBPF RSS > or use "in-qemu" RSS. My point is: the proposed command's mission is to help the management application run the right helper. However, its advice is *unreliable*. It may point to the wrong helper, or to nothing at all. The right helper may still exist elsewhere. I suspect you're trying to address the problem at the wrong level. Similar versioning issues exist with other helpers. We've been doing fine without QEMU providing unreliable advice on where they might sit in the file system. What makes this one different? >> What happens when you use the wrong helper? >> > UB - in most cases, eBPF program will work with wrong configurations. > That's why the stamp was added. > > query-helper-paths checks the stamp only for RSS helper. I have no idea what you're talking about :) My best guess is that you're trying to tell me that attempting to work with the wrong helper will fail cleanly due to some stamp check. That would be nice.
Hi, Yes, the stamp check was added. So the qemu emulator should return a suitable RSS BPF helper or nothing. Each qemu emulator may have a different helper that suits it. So, the idea is to ask for the helper from qemu. On Tue, Aug 24, 2021 at 9:41 AM Markus Armbruster <armbru@redhat.com> wrote: > Andrew Melnichenko <andrew@daynix.com> writes: > > > Hi, > > > >> The helper may or may not be installed at the path compiled into QEMU. > >> > > Yes, so the helper will not be called - QEMU will try to initiate eBPF > RSS > > or use "in-qemu" RSS. > > My point is: the proposed command's mission is to help the management > application run the right helper. However, its advice is *unreliable*. > It may point to the wrong helper, or to nothing at all. The right > helper may still exist elsewhere. > > I suspect you're trying to address the problem at the wrong level. > > Similar versioning issues exist with other helpers. We've been doing > fine without QEMU providing unreliable advice on where they might sit in > the file system. What makes this one different? > > >> What happens when you use the wrong helper? > >> > > UB - in most cases, eBPF program will work with wrong configurations. > > That's why the stamp was added. > > > > query-helper-paths checks the stamp only for RSS helper. > > I have no idea what you're talking about :) > > My best guess is that you're trying to tell me that attempting to work > with the wrong helper will fail cleanly due to some stamp check. That > would be nice. > >
Andrew Melnichenko <andrew@daynix.com> writes: > Hi, > Yes, the stamp check was added. > So the qemu emulator should return a suitable RSS BPF helper or nothing. > Each qemu emulator may have a different helper that suits it. > So, the idea is to ask for the helper from qemu. Please answer the question below. > > On Tue, Aug 24, 2021 at 9:41 AM Markus Armbruster <armbru@redhat.com> wrote: > >> Andrew Melnichenko <andrew@daynix.com> writes: >> >> > Hi, >> > >> >> The helper may or may not be installed at the path compiled into QEMU. >> >> >> > Yes, so the helper will not be called - QEMU will try to initiate eBPF >> RSS >> > or use "in-qemu" RSS. >> >> My point is: the proposed command's mission is to help the management >> application run the right helper. However, its advice is *unreliable*. >> It may point to the wrong helper, or to nothing at all. The right >> helper may still exist elsewhere. >> >> I suspect you're trying to address the problem at the wrong level. >> >> Similar versioning issues exist with other helpers. We've been doing >> fine without QEMU providing unreliable advice on where they might sit in >> the file system. What makes this one different? This one. >> >> What happens when you use the wrong helper? >> >> >> > UB - in most cases, eBPF program will work with wrong configurations. >> > That's why the stamp was added. >> > >> > query-helper-paths checks the stamp only for RSS helper. >> >> I have no idea what you're talking about :) >> >> My best guess is that you're trying to tell me that attempting to work >> with the wrong helper will fail cleanly due to some stamp check. That >> would be nice. >> >>
On Tue, Aug 24, 2021 at 9:41 AM Markus Armbruster <armbru@redhat.com> wrote: > > Andrew Melnichenko <andrew@daynix.com> writes: > > > Hi, > > > >> The helper may or may not be installed at the path compiled into QEMU. > >> > > Yes, so the helper will not be called - QEMU will try to initiate eBPF RSS > > or use "in-qemu" RSS. > > My point is: the proposed command's mission is to help the management > application run the right helper. However, its advice is *unreliable*. > It may point to the wrong helper, or to nothing at all. The right > helper may still exist elsewhere. Hi Markus, Indeed the intention of this command is to return the proper helper. Especially in the case of RSS helper this is *reliable* advice and it points to the helper that was built together with QEMU, i.e. with the same headers. This was discussed earlier, for example in https://lists.nongnu.org/archive/html/qemu-devel/2021-06/msg02248.html > > I suspect you're trying to address the problem at the wrong level. What is the proper solution for the problem from your point of view? > > Similar versioning issues exist with other helpers. We've been doing > fine without QEMU providing unreliable advice on where they might sit in > the file system. What makes this one different? This one is required to be *fully synchronized* with the existing build of QEMU. Other helpers are probably less restrictive and do not have common structures definitions with the QEMU, otherwise they would face the same problem. > > >> What happens when you use the wrong helper? Our intention is that libvirt should never use the wrong RSS helper. But it does not have any ability to check which helper is compatible with the QEMU. QEMU can easily recognize the correct one. > >> > > UB - in most cases, eBPF program will work with wrong configurations. > > That's why the stamp was added. > > > > query-helper-paths checks the stamp only for RSS helper. > > I have no idea what you're talking about :) > > My best guess is that you're trying to tell me that attempting to work > with the wrong helper will fail cleanly due to some stamp check. That > would be nice. >
Yuri Benditovich <yuri.benditovich@daynix.com> writes: > On Tue, Aug 24, 2021 at 9:41 AM Markus Armbruster <armbru@redhat.com> wrote: >> >> Andrew Melnichenko <andrew@daynix.com> writes: >> >> > Hi, >> > >> >> The helper may or may not be installed at the path compiled into QEMU. >> >> >> > Yes, so the helper will not be called - QEMU will try to initiate eBPF RSS >> > or use "in-qemu" RSS. >> >> My point is: the proposed command's mission is to help the management >> application run the right helper. However, its advice is *unreliable*. >> It may point to the wrong helper, or to nothing at all. The right >> helper may still exist elsewhere. > > Hi Markus, > Indeed the intention of this command is to return the proper helper. > Especially in the case of RSS helper this is *reliable* advice and it > points to the helper that was built together with QEMU, i.e. with the > same headers. > This was discussed earlier, for example in > https://lists.nongnu.org/archive/html/qemu-devel/2021-06/msg02248.html > >> >> I suspect you're trying to address the problem at the wrong level. > > What is the proper solution for the problem from your point of view? I'll explain in more detail, but first I'd like you to answer my question below. >> Similar versioning issues exist with other helpers. We've been doing >> fine without QEMU providing unreliable advice on where they might sit in >> the file system. What makes this one different? > > This one is required to be *fully synchronized* with the existing build of QEMU. > Other helpers are probably less restrictive and do not have common > structures definitions with the QEMU, otherwise they would face the > same problem. > >> >> >> What happens when you use the wrong helper? > > Our intention is that libvirt should never use the wrong RSS helper. > But it does not have any ability to check which helper is compatible > with the QEMU. > QEMU can easily recognize the correct one. You did not actually answer my question :) So let's try again: if libvirt does use the wrong RSS helper, how does the system behave? >> >> >> > UB - in most cases, eBPF program will work with wrong configurations. >> > That's why the stamp was added. >> > >> > query-helper-paths checks the stamp only for RSS helper. >> >> I have no idea what you're talking about :) >> >> My best guess is that you're trying to tell me that attempting to work >> with the wrong helper will fail cleanly due to some stamp check. That >> would be nice. >>
On Mon, Aug 30, 2021 at 9:10 AM Markus Armbruster <armbru@redhat.com> wrote: > > Yuri Benditovich <yuri.benditovich@daynix.com> writes: > > > On Tue, Aug 24, 2021 at 9:41 AM Markus Armbruster <armbru@redhat.com> wrote: > >> > >> Andrew Melnichenko <andrew@daynix.com> writes: > >> > >> > Hi, > >> > > >> >> The helper may or may not be installed at the path compiled into QEMU. > >> >> > >> > Yes, so the helper will not be called - QEMU will try to initiate eBPF RSS > >> > or use "in-qemu" RSS. > >> > >> My point is: the proposed command's mission is to help the management > >> application run the right helper. However, its advice is *unreliable*. > >> It may point to the wrong helper, or to nothing at all. The right > >> helper may still exist elsewhere. > > > > Hi Markus, > > Indeed the intention of this command is to return the proper helper. > > Especially in the case of RSS helper this is *reliable* advice and it > > points to the helper that was built together with QEMU, i.e. with the > > same headers. > > This was discussed earlier, for example in > > https://lists.nongnu.org/archive/html/qemu-devel/2021-06/msg02248.html > > > >> > >> I suspect you're trying to address the problem at the wrong level. > > > > What is the proper solution for the problem from your point of view? > > I'll explain in more detail, but first I'd like you to answer my > question below. > > >> Similar versioning issues exist with other helpers. We've been doing > >> fine without QEMU providing unreliable advice on where they might sit in > >> the file system. What makes this one different? > > > > This one is required to be *fully synchronized* with the existing build of QEMU. > > Other helpers are probably less restrictive and do not have common > > structures definitions with the QEMU, otherwise they would face the > > same problem. > > > >> > >> >> What happens when you use the wrong helper? > > > > Our intention is that libvirt should never use the wrong RSS helper. > > But it does not have any ability to check which helper is compatible > > with the QEMU. > > QEMU can easily recognize the correct one. > > You did not actually answer my question :) > > So let's try again: if libvirt does use the wrong RSS helper, how does > the system behave? The receive-side scaling may work incorrectly, i.e. finally may move incoming packets to a virtqueue different than expected one. > > >> >> > >> > UB - in most cases, eBPF program will work with wrong configurations. > >> > That's why the stamp was added. > >> > > >> > query-helper-paths checks the stamp only for RSS helper. > >> > >> I have no idea what you're talking about :) > >> > >> My best guess is that you're trying to tell me that attempting to work > >> with the wrong helper will fail cleanly due to some stamp check. That > >> would be nice. > >> >
Yuri Benditovich <yuri.benditovich@daynix.com> writes: > On Mon, Aug 30, 2021 at 9:10 AM Markus Armbruster <armbru@redhat.com> wrote: >> >> Yuri Benditovich <yuri.benditovich@daynix.com> writes: >> >> > On Tue, Aug 24, 2021 at 9:41 AM Markus Armbruster <armbru@redhat.com> wrote: >> >> >> >> Andrew Melnichenko <andrew@daynix.com> writes: >> >> >> >> > Hi, >> >> > >> >> >> The helper may or may not be installed at the path compiled into QEMU. >> >> >> >> >> > Yes, so the helper will not be called - QEMU will try to initiate eBPF RSS >> >> > or use "in-qemu" RSS. >> >> >> >> My point is: the proposed command's mission is to help the management >> >> application run the right helper. However, its advice is *unreliable*. >> >> It may point to the wrong helper, or to nothing at all. The right >> >> helper may still exist elsewhere. >> > >> > Hi Markus, >> > Indeed the intention of this command is to return the proper helper. >> > Especially in the case of RSS helper this is *reliable* advice and it >> > points to the helper that was built together with QEMU, i.e. with the >> > same headers. >> > This was discussed earlier, for example in >> > https://lists.nongnu.org/archive/html/qemu-devel/2021-06/msg02248.html >> > >> >> >> >> I suspect you're trying to address the problem at the wrong level. >> > >> > What is the proper solution for the problem from your point of view? >> >> I'll explain in more detail, but first I'd like you to answer my >> question below. >> >> >> Similar versioning issues exist with other helpers. We've been doing >> >> fine without QEMU providing unreliable advice on where they might sit in >> >> the file system. What makes this one different? >> > >> > This one is required to be *fully synchronized* with the existing build of QEMU. >> > Other helpers are probably less restrictive and do not have common >> > structures definitions with the QEMU, otherwise they would face the >> > same problem. >> > >> >> >> >> >> What happens when you use the wrong helper? >> > >> > Our intention is that libvirt should never use the wrong RSS helper. >> > But it does not have any ability to check which helper is compatible >> > with the QEMU. >> > QEMU can easily recognize the correct one. >> >> You did not actually answer my question :) >> >> So let's try again: if libvirt does use the wrong RSS helper, how does >> the system behave? > > The receive-side scaling may work incorrectly, i.e. finally may move > incoming packets to a virtqueue different than expected one. Then I'm confused about the purpose of "the stamp" mentioned below. Can you enlighten me? > >> >> >> >> >> >> > UB - in most cases, eBPF program will work with wrong configurations. >> >> > That's why the stamp was added. >> >> > >> >> > query-helper-paths checks the stamp only for RSS helper. >> >> >> >> I have no idea what you're talking about :) >> >> >> >> My best guess is that you're trying to tell me that attempting to work >> >> with the wrong helper will fail cleanly due to some stamp check. That >> >> would be nice.
On Mon, Aug 30, 2021 at 11:14 AM Markus Armbruster <armbru@redhat.com> wrote: > > Yuri Benditovich <yuri.benditovich@daynix.com> writes: > > > On Mon, Aug 30, 2021 at 9:10 AM Markus Armbruster <armbru@redhat.com> wrote: > >> > >> Yuri Benditovich <yuri.benditovich@daynix.com> writes: > >> > >> > On Tue, Aug 24, 2021 at 9:41 AM Markus Armbruster <armbru@redhat.com> wrote: > >> >> > >> >> Andrew Melnichenko <andrew@daynix.com> writes: > >> >> > >> >> > Hi, > >> >> > > >> >> >> The helper may or may not be installed at the path compiled into QEMU. > >> >> >> > >> >> > Yes, so the helper will not be called - QEMU will try to initiate eBPF RSS > >> >> > or use "in-qemu" RSS. > >> >> > >> >> My point is: the proposed command's mission is to help the management > >> >> application run the right helper. However, its advice is *unreliable*. > >> >> It may point to the wrong helper, or to nothing at all. The right > >> >> helper may still exist elsewhere. > >> > > >> > Hi Markus, > >> > Indeed the intention of this command is to return the proper helper. > >> > Especially in the case of RSS helper this is *reliable* advice and it > >> > points to the helper that was built together with QEMU, i.e. with the > >> > same headers. > >> > This was discussed earlier, for example in > >> > https://lists.nongnu.org/archive/html/qemu-devel/2021-06/msg02248.html > >> > > >> >> > >> >> I suspect you're trying to address the problem at the wrong level. > >> > > >> > What is the proper solution for the problem from your point of view? > >> > >> I'll explain in more detail, but first I'd like you to answer my > >> question below. > >> > >> >> Similar versioning issues exist with other helpers. We've been doing > >> >> fine without QEMU providing unreliable advice on where they might sit in > >> >> the file system. What makes this one different? > >> > > >> > This one is required to be *fully synchronized* with the existing build of QEMU. > >> > Other helpers are probably less restrictive and do not have common > >> > structures definitions with the QEMU, otherwise they would face the > >> > same problem. > >> > > >> >> > >> >> >> What happens when you use the wrong helper? > >> > > >> > Our intention is that libvirt should never use the wrong RSS helper. > >> > But it does not have any ability to check which helper is compatible > >> > with the QEMU. > >> > QEMU can easily recognize the correct one. > >> > >> You did not actually answer my question :) > >> > >> So let's try again: if libvirt does use the wrong RSS helper, how does > >> the system behave? > > > > The receive-side scaling may work incorrectly, i.e. finally may move > > incoming packets to a virtqueue different than expected one. > > Then I'm confused about the purpose of "the stamp" mentioned below. Can > you enlighten me? The stamp is a string (common for qemu executable and RSS helper executable during build) that qemu can later retrieve from the helper in run-time and ensure this helper is fully compatible with this build of qemu (in terms of eBPF operation). The helper is built with the same C headers (related to ebpf operation) as the qemu, the qemu is able to receive file descriptors created by the helper (of ebpf program and ebpf data structure's maps) from libvirt and deal with them as if it has created them. > > > > >> > >> >> >> > >> >> > UB - in most cases, eBPF program will work with wrong configurations. > >> >> > That's why the stamp was added. > >> >> > > >> >> > query-helper-paths checks the stamp only for RSS helper. > >> >> > >> >> I have no idea what you're talking about :) > >> >> > >> >> My best guess is that you're trying to tell me that attempting to work > >> >> with the wrong helper will fail cleanly due to some stamp check. That > >> >> would be nice. >
Yuri Benditovich <yuri.benditovich@daynix.com> writes: > On Mon, Aug 30, 2021 at 11:14 AM Markus Armbruster <armbru@redhat.com> wrote: >> >> Yuri Benditovich <yuri.benditovich@daynix.com> writes: >> >> > On Mon, Aug 30, 2021 at 9:10 AM Markus Armbruster <armbru@redhat.com> wrote: [...] >> >> So let's try again: if libvirt does use the wrong RSS helper, how does >> >> the system behave? >> > >> > The receive-side scaling may work incorrectly, i.e. finally may move >> > incoming packets to a virtqueue different than expected one. >> >> Then I'm confused about the purpose of "the stamp" mentioned below. Can >> you enlighten me? > > The stamp is a string (common for qemu executable and RSS helper > executable during build) that qemu can later retrieve from the helper > in run-time and ensure this helper is fully compatible with this build > of qemu (in terms of eBPF operation). The helper is built with the > same C headers (related to ebpf operation) as the qemu, the qemu is > able to receive file descriptors created by the helper (of ebpf > program and ebpf data structure's maps) from libvirt and deal with > them as if it has created them. Thanks. I tried to apply your series to check a few of my assumptions, but I'm getting conflicts. Got something I could git-pull? [...]
Hi, > Got something I could git-pull? > I can share some links for tests: https://github.com/daynix/qemu/tree/HelperEBPFv3 - qemu with helper https://github.com/daynix/libvirt/tree/RSS_RFC_v1 - libvirt with RSS On Tue, Aug 31, 2021 at 6:00 PM Markus Armbruster <armbru@redhat.com> wrote: > Yuri Benditovich <yuri.benditovich@daynix.com> writes: > > > On Mon, Aug 30, 2021 at 11:14 AM Markus Armbruster <armbru@redhat.com> > wrote: > >> > >> Yuri Benditovich <yuri.benditovich@daynix.com> writes: > >> > >> > On Mon, Aug 30, 2021 at 9:10 AM Markus Armbruster <armbru@redhat.com> > wrote: > > [...] > > >> >> So let's try again: if libvirt does use the wrong RSS helper, how > does > >> >> the system behave? > >> > > >> > The receive-side scaling may work incorrectly, i.e. finally may move > >> > incoming packets to a virtqueue different than expected one. > >> > >> Then I'm confused about the purpose of "the stamp" mentioned below. Can > >> you enlighten me? > > > > The stamp is a string (common for qemu executable and RSS helper > > executable during build) that qemu can later retrieve from the helper > > in run-time and ensure this helper is fully compatible with this build > > of qemu (in terms of eBPF operation). The helper is built with the > > same C headers (related to ebpf operation) as the qemu, the qemu is > > able to receive file descriptors created by the helper (of ebpf > > program and ebpf data structure's maps) from libvirt and deal with > > them as if it has created them. > > Thanks. > > I tried to apply your series to check a few of my assumptions, but I'm > getting conflicts. Got something I could git-pull? > > [...] > >
Andrew Melnichenko <andrew@daynix.com> writes: > Hi, > >> Got something I could git-pull? >> > I can share some links for tests: > https://github.com/daynix/qemu/tree/HelperEBPFv3 - qemu with helper > https://github.com/daynix/libvirt/tree/RSS_RFC_v1 - libvirt with RSS Thanks! I get a few warnings. I'm copying the ones from Clang: ../hw/net/virtio-net.c:1274:63: warning: array index 2 is past the end of the array (which contains 2 elements) [-Warray-bounds] ret = ebpf_rss_load_fds(&n->ebpf_rss, fds[0], fds[1], fds[2], fds[3]); ^ ~ ../hw/net/virtio-net.c:1257:5: note: array 'fds' declared here int fds[EBPF_RSS_MAX_FDS]; ^ ../hw/net/virtio-net.c:1274:71: warning: array index 3 is past the end of the array (which contains 2 elements) [-Warray-bounds] ret = ebpf_rss_load_fds(&n->ebpf_rss, fds[0], fds[1], fds[2], fds[3]); ^ ~ ../hw/net/virtio-net.c:1257:5: note: array 'fds' declared here int fds[EBPF_RSS_MAX_FDS]; ^ ../ebpf/qemu-ebpf-rss-helper.c:30:6: warning: no previous prototype for function 'qemuHelperStamp_a59ff8818a819adc85c28504b7961bdf89f0d9c0' [-Wmissing-prototypes] void QEMU_HELPER_STAMP(void) {} ^ ./qemu-helper-stamp.h:1:27: note: expanded from macro 'QEMU_HELPER_STAMP' #define QEMU_HELPER_STAMP qemuHelperStamp_a59ff8818a819adc85c28504b7961bdf89f0d9c0 ^ ../ebpf/qemu-ebpf-rss-helper.c:30:1: note: declare 'static' if the function is not intended to be used outside of this translation unit void QEMU_HELPER_STAMP(void) {} ^ static
Yuri Benditovich <yuri.benditovich@daynix.com> writes: > On Mon, Aug 30, 2021 at 11:14 AM Markus Armbruster <armbru@redhat.com> wrote: >> >> Yuri Benditovich <yuri.benditovich@daynix.com> writes: >> >> > On Mon, Aug 30, 2021 at 9:10 AM Markus Armbruster <armbru@redhat.com> wrote: >> >> >> >> Yuri Benditovich <yuri.benditovich@daynix.com> writes: >> >> >> >> > On Tue, Aug 24, 2021 at 9:41 AM Markus Armbruster <armbru@redhat.com> wrote: >> >> >> >> >> >> Andrew Melnichenko <andrew@daynix.com> writes: >> >> >> >> >> >> > Hi, >> >> >> > >> >> >> >> The helper may or may not be installed at the path compiled into QEMU. >> >> >> >> >> >> >> > Yes, so the helper will not be called - QEMU will try to initiate eBPF RSS >> >> >> > or use "in-qemu" RSS. >> >> >> >> >> >> My point is: the proposed command's mission is to help the management >> >> >> application run the right helper. However, its advice is *unreliable*. >> >> >> It may point to the wrong helper, or to nothing at all. The right >> >> >> helper may still exist elsewhere. >> >> > >> >> > Hi Markus, >> >> > Indeed the intention of this command is to return the proper helper. >> >> > Especially in the case of RSS helper this is *reliable* advice and it >> >> > points to the helper that was built together with QEMU, i.e. with the >> >> > same headers. In my testing, the patch fails at providing reliable advice. I did a "git fetch https://github.com/daynix/qemu HelperEBPFv3", rebased the result to master, build the thing, ran qemu-system-x86_64 right from the build tree, tried query-helper-paths and got {"return": [ {"name": "qemu-bridge-helper", "path": "/work/armbru/tmp/inst-qemu/libexec/qemu-bridge-helper"}, {"name": "qemu-pr-helper", "path": "/work/armbru/tmp/inst-qemu/libexec/qemu-pr-helper"}, {"name": "qemu-ebpf-rss-helper", "path": "/work/armbru/qemu/bld-x86/qemu-ebpf-rss-helper"}]} Ther first two are bogus. /work/armbru/tmp/inst-qemu/... is where "make install" would put things. I last ran "make install" almost three months ago. The last one is accurate in this particular scenario. More on why below. It won't be in a deployment where the actual installation paths do not match the ones we told configure at build time. Yes, people do that. Please read my entire reply before you comment on this paragraph. >> >> > This was discussed earlier, for example in >> >> > https://lists.nongnu.org/archive/html/qemu-devel/2021-06/msg02248.html >> >> > >> >> >> >> >> >> I suspect you're trying to address the problem at the wrong level. >> >> > >> >> > What is the proper solution for the problem from your point of view? >> >> >> >> I'll explain in more detail, but first I'd like you to answer my >> >> question below. >> >> >> >> >> Similar versioning issues exist with other helpers. We've been doing >> >> >> fine without QEMU providing unreliable advice on where they might sit in >> >> >> the file system. What makes this one different? >> >> > >> >> > This one is required to be *fully synchronized* with the existing build of QEMU. >> >> > Other helpers are probably less restrictive and do not have common >> >> > structures definitions with the QEMU, otherwise they would face the >> >> > same problem. >> >> > >> >> >> >> >> >> >> What happens when you use the wrong helper? >> >> > >> >> > Our intention is that libvirt should never use the wrong RSS helper. >> >> > But it does not have any ability to check which helper is compatible >> >> > with the QEMU. >> >> > QEMU can easily recognize the correct one. >> >> >> >> You did not actually answer my question :) >> >> >> >> So let's try again: if libvirt does use the wrong RSS helper, how does >> >> the system behave? >> > >> > The receive-side scaling may work incorrectly, i.e. finally may move >> > incoming packets to a virtqueue different than expected one. >> >> Then I'm confused about the purpose of "the stamp" mentioned below. Can >> you enlighten me? > > The stamp is a string (common for qemu executable and RSS helper > executable during build) that qemu can later retrieve from the helper > in run-time and ensure this helper is fully compatible with this build > of qemu (in terms of eBPF operation). The helper is built with the > same C headers (related to ebpf operation) as the qemu, the qemu is > able to receive file descriptors created by the helper (of ebpf > program and ebpf data structure's maps) from libvirt and deal with > them as if it has created them. query-helper-paths looks for the helper a number of locations, and reports the first one. This is fundamentally unrealiable. For qemu-ebpf-rss-helper, it additionally searches for a stamp symbol in the ELF symbol table. That's what makes it reliable in the sense of "won't report crap". It's still unreliable in the sense of "may not find the helper", see above. Searching the ELF symbol table requires ELF. I suspect your meson.build doesn't reflect that. It also requires the symbol table to be present. Statically linked programs don't have one, if I remember correctly. >> >> >> >> >> >> >> > UB - in most cases, eBPF program will work with wrong configurations. >> >> >> > That's why the stamp was added. >> >> >> > >> >> >> > query-helper-paths checks the stamp only for RSS helper. >> >> >> >> >> >> I have no idea what you're talking about :) >> >> >> >> >> >> My best guess is that you're trying to tell me that attempting to work >> >> >> with the wrong helper will fail cleanly due to some stamp check. That >> >> >> would be nice. Looks like my best guess was correct. Let's take a step back. Management applications run qemu-system-FOO and helpers. They know where to find qemu-system-FOO. It stands to reason that they also know where to find the matching helpers. I fail to see why they need help from qemu-system-FOO via QMP. Even if they need help, qemu-system-FOO can't give it, because it cannot know for sure. It is wherever the system administrator / distro put it. When the system administrator / distro put stuff in unusual places, they get to configure the programs that use it. In this case, that's the management application. How QEMU may be able to help is refuse to run with the wrong helper. If qemu-system-FOO talks directly to the helper, you can have the helper present the stamp for qemu-system-FOO to accept or reject. Or vice versa, doesn't matter. Or you can provide means for the management application to retrieve a stamp from all programs involved, so it can fail unless they match. There are much easier ways for that than searching through ELF symbol tables. Have a command line option to print it. For qemu-system-FOO, you can also have a QMP command to query it.
Hi, > Ther first two are bogus. /work/armbru/tmp/inst-qemu/... is where "make > install" would put things. I last ran "make install" almost three > months ago. > The stamp check is implemented only for the RSS helper. Qemu looks for a helper first in HelpDir, then next to the binary. Bridge/PR helpers found in "inst-qemu" according to build configuration. To be more strict, for the next patch we may return only ebpf helper in the list. Other helpers may be added later with stamp check. It's still unreliable in the sense of "may not > find the helper" > Ok, I see the problem. Here I can propose possible solutions: 1. Add the argument to QMP request - the path(s) where the qemu should look for helper by priority: argument path, installation dir, qemu's dir. And return an array of all possible helpers binaries. 2. Retrieve the stamp from QMP request, so Libvirt may check the stamp by itself. 3. Set suid bit to helper and pass the helper's path to the qemu, so qemu may run it by itself. Searching the ELF symbol table requires ELF. I suspect your meson.build > doesn't reflect that. > > It also requires the symbol table to be present. Statically linked > programs don't have one, if I remember correctly. > Well, qemu-ebpf-rss-helper is only for Linux and expectations are to have ELF binaries. The helper builds dynamically linked with libbpf.so at least. Management applications run qemu-system-FOO and helpers. They know > where to find qemu-system-FOO. It stands to reason that they also know > where to find the matching helpers. I fail to see why they need help > from qemu-system-FOO via QMP. Even if they need help, qemu-system-FOO > can't give it, because it cannot know for sure. It is wherever the > system administrator / distro put it. > We may have several different qemu builds which may require different rss helpers. Domain with a custom emulator: > > * <devices> > <emulator>/path_to_my_awesome_qemu_with_custom_bpf/qemu-system-FOO</emulator> > * > Libvirt would gather properties of that qemu and we need to provide the proper helper for it. The "default" helper from libvirt configure may not suit the qemu. So the idea is to ask the qemu itself where is a possible helper. So Libvirt may use only <emulator> without additional changes in XML document. So we need to solve next cases: - System admin or distributor places the qemu and helpers in InstallPath. Libvirt uses installed qemu and helpers. No hints from qemu are required. - Libvirt uses custom qemu <emulator>. Libvirt doesn't know where is the helper that was built with custom qemu. Qemu if finds the helper - verifies the stamp, to make sure. - Custom-built qemu is installed in the configured path(proper prefix etc.). Libvirt may use that emulator but don't know where is the helper - qemu knows and may provide a hint. There are much easier ways for that than searching through ELF symbol > tables. Have a command line option to print it. For qemu-system-FOO, > you can also have a QMP command to query it. > The stamp was added in a similar way as qemu modules(special symbol, checked during load). Also, checking the stamp without running is a good security measure. I get a few warnings. I'm copying the ones from Clang: > Thank you, I'll fix them in v2 patches with rebased tree.
diff --git a/monitor/qmp-cmds.c b/monitor/qmp-cmds.c index f7d64a6457..c042ab5466 100644 --- a/monitor/qmp-cmds.c +++ b/monitor/qmp-cmds.c @@ -40,6 +40,7 @@ #include "qapi/qmp/qerror.h" #include "hw/mem/memory-device.h" #include "hw/acpi/acpi_dev_interface.h" +#include "qemu-helper-stamp-utils.h" NameInfo *qmp_query_name(Error **errp) { @@ -351,3 +352,34 @@ void qmp_display_reload(DisplayReloadOptions *arg, Error **errp) abort(); } } + +HelperPathList *qmp_query_helper_paths(Error **errp) +{ + HelperPathList *ret = NULL; + struct { + const char *helper; + bool check_stamp; + } helpers_list[] = { +#ifdef CONFIG_EBPF + { "qemu-ebpf-rss-helper", true }, +#endif + { "qemu-pr-helper", false }, + { "qemu-bridge-helper", false }, + { NULL, false }, + }, *helper_iter; + helper_iter = helpers_list; + + for (; helper_iter->helper != NULL; ++helper_iter) { + char *path = qemu_find_helper(helper_iter->helper, + helper_iter->check_stamp); + if (path) { + HelperPath *helper = g_new0(HelperPath, 1); + helper->name = g_strdup(helper_iter->helper); + helper->path = path; + + QAPI_LIST_PREPEND(ret, helper); + } + } + + return ret; +} diff --git a/qapi/misc.json b/qapi/misc.json index 156f98203e..9aaf8fbcca 100644 --- a/qapi/misc.json +++ b/qapi/misc.json @@ -519,3 +519,36 @@ 'data': { '*option': 'str' }, 'returns': ['CommandLineOptionInfo'], 'allow-preconfig': true } + +## +# @HelperPath: +# +# Name of the helper and binary location. +## +{ 'struct': 'HelperPath', + 'data': {'name': 'str', 'path': 'str'} } + +## +# @query-helper-paths: +# +# Query helper paths. Initially, this command was added for +# qemu-ebpf-rss-helper. The qemu would check "the stamp" and +# returns proper helper. +# +# Returns: list of object that contains name and path for helper. +# +# Since: 6.1 +# +# Example: +# +# -> { "execute": "query-helper-paths" } +# <- { "return": [ +# { +# "name": "qemu-ebpf-rss-helper", +# "path": "/usr/local/libexec/qemu-ebpf-rss-helper" +# } +# ] +# } +# +## +{ 'command': 'query-helper-paths', 'returns': ['HelperPath'] }
New qmp command to query ebpf helper. It's crucial that qemu and helper are in sync and in touch. Technically helper should pass eBPF fds that qemu may accept. And different qemu's builds may have different eBPF programs and helpers. Qemu returns helper that should "fit" to virtio-net. Signed-off-by: Andrew Melnychenko <andrew@daynix.com> --- monitor/qmp-cmds.c | 32 ++++++++++++++++++++++++++++++++ qapi/misc.json | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+)