| Message ID | 20201002010633.3706122-1-andriin@fb.com |
|---|---|
| Headers | show |
| Series | libbpf: auto-resize relocatable LOAD/STORE instructions | expand |
Hello Andrii, On Fri, Oct 2, 2020 at 3:09 AM Andrii Nakryiko <andriin@fb.com> wrote: > Patch set implements logic in libbpf to auto-adjust memory size (1-, 2-, 4-, > 8-bytes) of load/store (LD/ST/STX) instructions which have BPF CO-RE field > offset relocation associated with it. In practice this means transparent > handling of 32-bit kernels, both pointer and unsigned integers. Signed > integers are not relocatable with zero-extending loads/stores, so libbpf > poisons them and generates a warning. If/when BPF gets support for sign-extending > loads/stores, it would be possible to automatically relocate them as well. > > All the details are contained in patch #1 comments and commit message. > Patch #2 is a simple change in libbpf to make advanced testing with custom BTF > easier. Patch #3 validates correct uses of auto-resizable loads, as well as > check that libbpf fails invalid uses. > > I'd really appreciate folks that use BPF on 32-bit architectures to test this > out with their BPF programs and report if there are any problems with the > approach. > > Cc: Luka Perkov <luka.perkov@sartura.hr> First, thank you for the support and sending this series. It took us a bit longer to run the tests as our target hardware still did not fully get complete mainline support and we had to rebase our patches. These are not related to BPF. Related to this patch, we have tested various BPF programs with this patch, and can confirm that it fixed previous issues with pointer offsets that we had and reported at: https://lore.kernel.org/r/CA+XBgLU=8PFkP8S32e4gpst0=R4MFv8rZA5KaO+cEPYSnTRYYw@mail.gmail.com/. Most of our programs now work and we are currently debugging other programs that still aren't working. We are still not sure if the remaining issues are related to this or not, but will let you know sometime this week after further and more detailed investigation. Thanks, Luka
On Wed, Oct 7, 2020 at 10:56 AM Luka Perkov <luka.perkov@sartura.hr> wrote: > > Hello Andrii, > > On Fri, Oct 2, 2020 at 3:09 AM Andrii Nakryiko <andriin@fb.com> wrote: > > Patch set implements logic in libbpf to auto-adjust memory size (1-, 2-, 4-, > > 8-bytes) of load/store (LD/ST/STX) instructions which have BPF CO-RE field > > offset relocation associated with it. In practice this means transparent > > handling of 32-bit kernels, both pointer and unsigned integers. Signed > > integers are not relocatable with zero-extending loads/stores, so libbpf > > poisons them and generates a warning. If/when BPF gets support for sign-extending > > loads/stores, it would be possible to automatically relocate them as well. > > > > All the details are contained in patch #1 comments and commit message. > > Patch #2 is a simple change in libbpf to make advanced testing with custom BTF > > easier. Patch #3 validates correct uses of auto-resizable loads, as well as > > check that libbpf fails invalid uses. > > > > I'd really appreciate folks that use BPF on 32-bit architectures to test this > > out with their BPF programs and report if there are any problems with the > > approach. > > > > Cc: Luka Perkov <luka.perkov@sartura.hr> > > First, thank you for the support and sending this series. It took us a > bit longer to run the tests as our target hardware still did not fully > get complete mainline support and we had to rebase our patches. These > are not related to BPF. > > Related to this patch, we have tested various BPF programs with this > patch, and can confirm that it fixed previous issues with pointer > offsets that we had and reported at: > > https://lore.kernel.org/r/CA+XBgLU=8PFkP8S32e4gpst0=R4MFv8rZA5KaO+cEPYSnTRYYw@mail.gmail.com/. > > Most of our programs now work and we are currently debugging other > programs that still aren't working. We are still not sure if the > remaining issues are related to this or not, but will let you know > sometime this week after further and more detailed investigation. > Ok, great, thanks for the update. > Thanks, > Luka
Hello Andrii, On Wed, Oct 7, 2020 at 8:01 PM Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote: > > On Wed, Oct 7, 2020 at 10:56 AM Luka Perkov <luka.perkov@sartura.hr> wrote: > > > > Hello Andrii, > > > > On Fri, Oct 2, 2020 at 3:09 AM Andrii Nakryiko <andriin@fb.com> wrote: > > > Patch set implements logic in libbpf to auto-adjust memory size (1-, 2-, 4-, > > > 8-bytes) of load/store (LD/ST/STX) instructions which have BPF CO-RE field > > > offset relocation associated with it. In practice this means transparent > > > handling of 32-bit kernels, both pointer and unsigned integers. Signed > > > integers are not relocatable with zero-extending loads/stores, so libbpf > > > poisons them and generates a warning. If/when BPF gets support for sign-extending > > > loads/stores, it would be possible to automatically relocate them as well. > > > > > > All the details are contained in patch #1 comments and commit message. > > > Patch #2 is a simple change in libbpf to make advanced testing with custom BTF > > > easier. Patch #3 validates correct uses of auto-resizable loads, as well as > > > check that libbpf fails invalid uses. > > > > > > I'd really appreciate folks that use BPF on 32-bit architectures to test this > > > out with their BPF programs and report if there are any problems with the > > > approach. > > > > > > Cc: Luka Perkov <luka.perkov@sartura.hr> > > > > First, thank you for the support and sending this series. It took us a > > bit longer to run the tests as our target hardware still did not fully > > get complete mainline support and we had to rebase our patches. These > > are not related to BPF. > > > > Related to this patch, we have tested various BPF programs with this > > patch, and can confirm that it fixed previous issues with pointer > > offsets that we had and reported at: > > > > https://lore.kernel.org/r/CA+XBgLU=8PFkP8S32e4gpst0=R4MFv8rZA5KaO+cEPYSnTRYYw@mail.gmail.com/. > > > > Most of our programs now work and we are currently debugging other > > programs that still aren't working. We are still not sure if the > > remaining issues are related to this or not, but will let you know > > sometime this week after further and more detailed investigation. > > > > Ok, great, thanks for the update. Just to update you that we have identified that the problem was a known issue with JIT as we had enabled the BPF_JIT_ALWAYS_ON. That said, it would be great to see this series included in 5.10 :) Thanks, Luka
On Thu, Oct 8, 2020 at 3:34 AM Luka Perkov <luka.perkov@sartura.hr> wrote: > > Hello Andrii, > > On Wed, Oct 7, 2020 at 8:01 PM Andrii Nakryiko > <andrii.nakryiko@gmail.com> wrote: > > > > On Wed, Oct 7, 2020 at 10:56 AM Luka Perkov <luka.perkov@sartura.hr> wrote: > > > > > > Hello Andrii, > > > > > > On Fri, Oct 2, 2020 at 3:09 AM Andrii Nakryiko <andriin@fb.com> wrote: > > > > Patch set implements logic in libbpf to auto-adjust memory size (1-, 2-, 4-, > > > > 8-bytes) of load/store (LD/ST/STX) instructions which have BPF CO-RE field > > > > offset relocation associated with it. In practice this means transparent > > > > handling of 32-bit kernels, both pointer and unsigned integers. Signed > > > > integers are not relocatable with zero-extending loads/stores, so libbpf > > > > poisons them and generates a warning. If/when BPF gets support for sign-extending > > > > loads/stores, it would be possible to automatically relocate them as well. > > > > > > > > All the details are contained in patch #1 comments and commit message. > > > > Patch #2 is a simple change in libbpf to make advanced testing with custom BTF > > > > easier. Patch #3 validates correct uses of auto-resizable loads, as well as > > > > check that libbpf fails invalid uses. > > > > > > > > I'd really appreciate folks that use BPF on 32-bit architectures to test this > > > > out with their BPF programs and report if there are any problems with the > > > > approach. > > > > > > > > Cc: Luka Perkov <luka.perkov@sartura.hr> > > > > > > First, thank you for the support and sending this series. It took us a > > > bit longer to run the tests as our target hardware still did not fully > > > get complete mainline support and we had to rebase our patches. These > > > are not related to BPF. > > > > > > Related to this patch, we have tested various BPF programs with this > > > patch, and can confirm that it fixed previous issues with pointer > > > offsets that we had and reported at: > > > > > > https://lore.kernel.org/r/CA+XBgLU=8PFkP8S32e4gpst0=R4MFv8rZA5KaO+cEPYSnTRYYw@mail.gmail.com/. > > > > > > Most of our programs now work and we are currently debugging other > > > programs that still aren't working. We are still not sure if the > > > remaining issues are related to this or not, but will let you know > > > sometime this week after further and more detailed investigation. > > > > > > > Ok, great, thanks for the update. > > Just to update you that we have identified that the problem was a > known issue with JIT as we had enabled the BPF_JIT_ALWAYS_ON. > > That said, it would be great to see this series included in 5.10 :) This is purely a libbpf feature, completely agnostic to kernel versions. So you'll get this with upcoming libbpf 0.2.0 release. > > Thanks, > Luka
Patch set implements logic in libbpf to auto-adjust memory size (1-, 2-, 4-, 8-bytes) of load/store (LD/ST/STX) instructions which have BPF CO-RE field offset relocation associated with it. In practice this means transparent handling of 32-bit kernels, both pointer and unsigned integers. Signed integers are not relocatable with zero-extending loads/stores, so libbpf poisons them and generates a warning. If/when BPF gets support for sign-extending loads/stores, it would be possible to automatically relocate them as well. All the details are contained in patch #1 comments and commit message. Patch #2 is a simple change in libbpf to make advanced testing with custom BTF easier. Patch #3 validates correct uses of auto-resizable loads, as well as check that libbpf fails invalid uses. I'd really appreciate folks that use BPF on 32-bit architectures to test this out with their BPF programs and report if there are any problems with the approach. Cc: Luka Perkov <luka.perkov@sartura.hr> Cc: Tony Ambardar <tony.ambardar@gmail.com> Andrii Nakryiko (3): libbpf: support safe subset of load/store instruction resizing with CO-RE libbpf: allow specifying both ELF and raw BTF for CO-RE BTF override selftests/bpf: validate libbpf's auto-sizing of LD/ST/STX instructions tools/lib/bpf/libbpf.c | 146 ++++++++++++- .../selftests/bpf/prog_tests/core_autosize.c | 199 ++++++++++++++++++ .../selftests/bpf/progs/test_core_autosize.c | 148 +++++++++++++ 3 files changed, 483 insertions(+), 10 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/core_autosize.c create mode 100644 tools/testing/selftests/bpf/progs/test_core_autosize.c