| Message ID | 20230723092314.129079-4-thomas@devoogdt.com |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | [v3,1/4] package/fluent-bit: bump to version 2.1.7 | expand |
Hello Thomas, On Sun, 23 Jul 2023 11:23:14 +0200 Thomas Devoogdt <thomas@devoogdt.com> wrote: > -if BR2_PACKAGE_FLUENT_BIT > - > -config BR2_PACKAGE_FLUENT_BIT_WASM_ARCH_SUPPORTS > - bool > - # see lib/wasm-micro-runtime-WAMR-1.1.1/README.md#supported-architectures-and-platforms > - default y if BR2_arc > - default y if BR2_arm || BR2_armeb > - default y if BR2_aarch64 || BR2_aarch64_be > - default y if BR2_i386 || BR2_x86_64 > - default y if BR2_RISCV_64 > - default y if BR2_xtensa > - > -config BR2_PACKAGE_FLUENT_BIT_WASM > - bool "wasm runtime support" > - depends on BR2_INSTALL_LIBSTDCPP > - depends on BR2_PACKAGE_FLUENT_BIT_WASM_ARCH_SUPPORTS > - help > - Support for WASM-based plugins. Something that isn't explained in your commit log is why you're making the wasm feature automatically enabled, rather than optional as it was? What is the size impact? Would someone possibly want fluent-bit without wasm support? > Two failures: > > - sourcery-mips64: > > core/iwasm/common/arch/invokeNative_mips.s: Assembler messages: > core/iwasm/common/arch/invokeNative_mips.s:58: Error: opcode not > supported on this processor: mips64r2 (mips64r2) `ldc1 $f12,0($sp)' > core/iwasm/common/arch/invokeNative_mips.s:59: Error: opcode not > supported on this processor: mips64r2 (mips64r2) `ldc1 $f14,8($sp)' > make[4]: *** [src/wasm/CMakeFiles/vmlib-static.dir/build.make:495: > src/wasm/CMakeFiles/vmlib-static.dir/__/__/lib/wasm-micro-runtime-WAMR-1.1.1/core/iwasm/common/arch/invokeNative_mips.s.o] > Error 1 make[4]: *** Waiting for unfinished jobs.... > > - sourcery-mips: > > core/iwasm/common/arch/invokeNative_mips.s: Assembler messages: > core/iwasm/common/arch/invokeNative_mips.s:58: Error: opcode not > supported on this processor: mips32r2 (mips32r2) `ldc1 $f12,0($sp)' > core/iwasm/common/arch/invokeNative_mips.s:59: Error: opcode not > supported on this processor: mips32r2 (mips32r2) `ldc1 $f14,8($sp)' > make[4]: *** [src/wasm/CMakeFiles/vmlib-static.dir/build.make:495: > src/wasm/CMakeFiles/vmlib-static.dir/__/__/lib/wasm-micro-runtime-WAMR-1.1.1/core/iwasm/common/arch/invokeNative_mips.s.o] > Error 1 make[4]: *** Waiting for unfinished jobs.... > > Fix see: > https://github.com/bytecodealliance/wasm-micro-runtime/issues/625 This part of your commit log is really not clear: we don't understand if the issue still exists, or is fixed in your commit. You're passing -DWAMR_BUILD_INVOKE_NATIVE_GENERAL=1 for MIPS, but your commit log contains a test-pkg run showing that it still fails to build for MIPS. So? Thanks! Thomas
Hi Thomas, Op zo 23 jul 2023 om 12:06 schreef Thomas Petazzoni <thomas.petazzoni@bootlin.com>: > > Hello Thomas, > > On Sun, 23 Jul 2023 11:23:14 +0200 > Thomas Devoogdt <thomas@devoogdt.com> wrote: > > > -if BR2_PACKAGE_FLUENT_BIT > > - > > -config BR2_PACKAGE_FLUENT_BIT_WASM_ARCH_SUPPORTS > > - bool > > - # see lib/wasm-micro-runtime-WAMR-1.1.1/README.md#supported-architectures-and-platforms > > - default y if BR2_arc > > - default y if BR2_arm || BR2_armeb > > - default y if BR2_aarch64 || BR2_aarch64_be > > - default y if BR2_i386 || BR2_x86_64 > > - default y if BR2_RISCV_64 > > - default y if BR2_xtensa > > - > > -config BR2_PACKAGE_FLUENT_BIT_WASM > > - bool "wasm runtime support" > > - depends on BR2_INSTALL_LIBSTDCPP > > - depends on BR2_PACKAGE_FLUENT_BIT_WASM_ARCH_SUPPORTS > > - help > > - Support for WASM-based plugins. > > Something that isn't explained in your commit log is why you're making > the wasm feature automatically enabled, rather than optional as it was? I had to re-add the logic to the make file in order to select the right target. I dropped the Config.In logic to avoid double maintenance of the supported architectures. There is also otherwise no specific reason to not always compile it. > What is the size impact? Would someone possibly want fluent-bit without > wasm support? FLB_WASM=No : -rwxr-xr-x 1 thomas thomas 31769156 Jul 24 09:10 output/target/usr/lib/fluent-bit/libfluent-bit.so FLB_WASM=Yes: -rwxr-xr-x 1 thomas thomas 33801056 Jul 24 09:07 output/target/usr/lib/fluent-bit/libfluent-bit.so So the binary goes from 31.8MB to 33.8MB. > > Two failures: > > > > - sourcery-mips64: > > > > core/iwasm/common/arch/invokeNative_mips.s: Assembler messages: > > core/iwasm/common/arch/invokeNative_mips.s:58: Error: opcode not > > supported on this processor: mips64r2 (mips64r2) `ldc1 $f12,0($sp)' > > core/iwasm/common/arch/invokeNative_mips.s:59: Error: opcode not > > supported on this processor: mips64r2 (mips64r2) `ldc1 $f14,8($sp)' > > make[4]: *** [src/wasm/CMakeFiles/vmlib-static.dir/build.make:495: > > src/wasm/CMakeFiles/vmlib-static.dir/__/__/lib/wasm-micro-runtime-WAMR-1.1.1/core/iwasm/common/arch/invokeNative_mips.s.o] > > Error 1 make[4]: *** Waiting for unfinished jobs.... > > > > - sourcery-mips: > > > > core/iwasm/common/arch/invokeNative_mips.s: Assembler messages: > > core/iwasm/common/arch/invokeNative_mips.s:58: Error: opcode not > > supported on this processor: mips32r2 (mips32r2) `ldc1 $f12,0($sp)' > > core/iwasm/common/arch/invokeNative_mips.s:59: Error: opcode not > > supported on this processor: mips32r2 (mips32r2) `ldc1 $f14,8($sp)' > > make[4]: *** [src/wasm/CMakeFiles/vmlib-static.dir/build.make:495: > > src/wasm/CMakeFiles/vmlib-static.dir/__/__/lib/wasm-micro-runtime-WAMR-1.1.1/core/iwasm/common/arch/invokeNative_mips.s.o] > > Error 1 make[4]: *** Waiting for unfinished jobs.... > > > > Fix see: > > https://github.com/bytecodealliance/wasm-micro-runtime/issues/625 > > This part of your commit log is really not clear: we don't understand > if the issue still exists, or is fixed in your commit. You're passing > -DWAMR_BUILD_INVOKE_NATIVE_GENERAL=1 for MIPS, but your commit log > contains a test-pkg run showing that it still fails to build for MIPS. So? The log indeed shows that there are two failures, but those are fixed by setting the obscure WAMR_BUILD_INVOKE_NATIVE_GENERAL flag. > Thanks! > > Thomas > -- > Thomas Petazzoni, co-owner and CEO, Bootlin > Embedded Linux and Kernel engineering and training > https://bootlin.com > This git commit message is perhaps better:
Hello Thomas, On Mon, 24 Jul 2023 10:29:45 +0200 Thomas Devoogdt <thomas@devoogdt.com> wrote: > > Something that isn't explained in your commit log is why you're making > > the wasm feature automatically enabled, rather than optional as it was? > > I had to re-add the logic to the make file in order to select the right target. > I dropped the Config.In logic to avoid double maintenance of the > supported architectures. Well, in fact the logic for the architecture would be cleaner in the Config.in file. if BR2_PACKAGE_FLUENT_BIT config BR2_PACKAGE_FLUENT_BIT_WASM_ARCH string default "AARCH64" if BR2_aarch64 || BR2_aarch64be default "ARC" if BR2_arc || BR2_arceb default "ARM" if BR2_arm || BR2_armeb ... config BR2_PACKAGE_FLUENT_BIT_WASM_ARCH_SUPPORTS bool default y if BR2_PACKAGE_FLUENT_BIT_WASM_ARCH != "" config BR2_PACKAGE_FLUENT_BIT_WASM bool "wasm runtime support" depends on BR2_INSTALL_LIBSTDCPP depends on BR2_PACKAGE_FLUENT_BIT_WASM_ARCH_SUPPORTS help Support for WASM-based plugins. ... endif And then in the Makefile: FLUENT_BIT_WASM_ARCH = $(call qstrip,$(BR2_PACKAGE_FLUENT_BIT_WASM_ARCH)) and voilà. > > What is the size impact? Would someone possibly want fluent-bit without > > wasm support? > > FLB_WASM=No : -rwxr-xr-x 1 thomas thomas 31769156 Jul 24 09:10 > output/target/usr/lib/fluent-bit/libfluent-bit.so > FLB_WASM=Yes: -rwxr-xr-x 1 thomas thomas 33801056 Jul 24 09:07 > output/target/usr/lib/fluent-bit/libfluent-bit.so > > So the binary goes from 31.8MB to 33.8MB. Admittedly, it's already a huge thing, so another 2 MB more... But in general in Buildroot our policy is to ensure things that are optional in upstream are indeed optional in Buildroot. > package/fluent-bit: rework wasm handling > > Fluent-bit isn't very great at supporting the various architectures, > so help a little bit by defining the WAMR_BUILD_TARGET ourselves. > > The BR2_PACKAGE_FLUENT_BIT_WASM option is removed to avoid > double architecture maintenance in both the KConfig and make file. This See above: no double maintenance is needed. > means that WAMR will always get compiled resulting in a slightly binary > size increase from 31.8MB to 33.8MB. > > Two failures while executing ./utils/test-pkg: > > - sourcery-mips64: > core/iwasm/common/arch/invokeNative_mips.s: Assembler messages: > core/iwasm/common/arch/invokeNative_mips.s:58: Error: opcode not > supported on this processor: mips64r2 (mips64r2) `ldc1 $f12,0($sp)' > core/iwasm/common/arch/invokeNative_mips.s:59: Error: opcode not > supported on this processor: mips64r2 (mips64r2) `ldc1 $f14,8($sp)' > make[4]: *** [src/wasm/CMakeFiles/vmlib-static.dir/build.make:495: > src/wasm/CMakeFiles/vmlib-static.dir/__/__/lib/wasm-micro-runtime-WAMR-1.1.1/core/iwasm/common/arch/invokeNative_mips.s.o] > Error 1 > make[4]: *** Waiting for unfinished jobs.... > > - sourcery-mips: > core/iwasm/common/arch/invokeNative_mips.s: Assembler messages: > core/iwasm/common/arch/invokeNative_mips.s:58: Error: opcode not > supported on this processor: mips32r2 (mips32r2) `ldc1 $f12,0($sp)' > core/iwasm/common/arch/invokeNative_mips.s:59: Error: opcode not > supported on this processor: mips32r2 (mips32r2) `ldc1 $f14,8($sp)' > make[4]: *** [src/wasm/CMakeFiles/vmlib-static.dir/build.make:495: > src/wasm/CMakeFiles/vmlib-static.dir/__/__/lib/wasm-micro-runtime-WAMR-1.1.1/core/iwasm/common/arch/invokeNative_mips.s.o] > Error 1 > make[4]: *** Waiting for unfinished jobs.... > > Which are fixed by setting the obscure flag "WAMR_BUILD_INVOKE_NATIVE_GENERAL". > > See https://github.com/bytecodealliance/wasm-micro-runtime/issues/625 > for more info: > “Hi, building the invokeNative_general.c explicitly is supported now, > please pull the latest code and use > "cmake -DWAMR_BUILD_INVOKE_NATIVE_GENERAL=1 ..." to build it.” Thanks, this explanation looks good. Thomas
diff --git a/package/fluent-bit/Config.in b/package/fluent-bit/Config.in index bfdea895b0..dc477e9f39 100644 --- a/package/fluent-bit/Config.in +++ b/package/fluent-bit/Config.in @@ -11,31 +11,6 @@ config BR2_PACKAGE_FLUENT_BIT https://github.com/fluent/fluent-bit -if BR2_PACKAGE_FLUENT_BIT - -config BR2_PACKAGE_FLUENT_BIT_WASM_ARCH_SUPPORTS - bool - # see lib/wasm-micro-runtime-WAMR-1.1.1/README.md#supported-architectures-and-platforms - default y if BR2_arc - default y if BR2_arm || BR2_armeb - default y if BR2_aarch64 || BR2_aarch64_be - default y if BR2_i386 || BR2_x86_64 - default y if BR2_RISCV_64 - default y if BR2_xtensa - -config BR2_PACKAGE_FLUENT_BIT_WASM - bool "wasm runtime support" - depends on BR2_INSTALL_LIBSTDCPP - depends on BR2_PACKAGE_FLUENT_BIT_WASM_ARCH_SUPPORTS - help - Support for WASM-based plugins. - -comment "wasm runtime support needs a toolchain w/ C++" - depends on BR2_PACKAGE_FLUENT_BIT_WASM_ARCH_SUPPORTS - depends on !BR2_INSTALL_LIBSTDCPP - -endif - comment "fluent-bit needs a toolchain w/ threads, dynamic library" depends on BR2_TOOLCHAIN_HAS_SYNC_4 depends on !BR2_TOOLCHAIN_HAS_THREADS || BR2_STATIC_LIBS diff --git a/package/fluent-bit/fluent-bit.mk b/package/fluent-bit/fluent-bit.mk index cd8f347768..c863e62fdb 100644 --- a/package/fluent-bit/fluent-bit.mk +++ b/package/fluent-bit/fluent-bit.mk @@ -22,8 +22,36 @@ FLUENT_BIT_CONF_OPTS += \ -DFLB_PREFER_SYSTEM_LIBS=Yes \ -DFLB_BACKTRACE=No -ifeq ($(BR2_PACKAGE_FLUENT_BIT_WASM),y) -FLUENT_BIT_CONF_OPTS += -DFLB_WASM=Yes +# WASM runtime support needs a toolchain w/ C++ +ifeq ($(BR2_INSTALL_LIBSTDCPP),y) +ifeq ($(BR2_aarch64)$(BR2_aarch64_be),y) +FLUENT_BIT_WAMR_BUILD_TARGET = AARCH64 +else ifeq ($(BR2_arcle)$(BR2_arceb),y) +FLUENT_BIT_WAMR_BUILD_TARGET = ARC +else ifeq ($(BR2_arm)$(BR2_armeb),y) +FLUENT_BIT_WAMR_BUILD_TARGET = ARM +else ifeq ($(BR2_mips)$(BR2_mipsel)$(BR2_mips64)$(BR2_mips64el),y) +FLUENT_BIT_WAMR_BUILD_TARGET = MIPS +# https://github.com/bytecodealliance/wasm-micro-runtime/issues/625 +# Fix unknown opcode 'ldc1', seen on mips32r2 and mips64r2. +FLUENT_BIT_CONF_OPTS += \ + -DWAMR_BUILD_INVOKE_NATIVE_GENERAL=1 +else ifeq ($(BR2_riscv)$(BR2_RISCV_32),yy) +FLUENT_BIT_WAMR_BUILD_TARGET = RISCV32 +else ifeq ($(BR2_riscv)$(BR2_RISCV_64),yy) +FLUENT_BIT_WAMR_BUILD_TARGET = RISCV64 +else ifeq ($(BR2_i386),y) +FLUENT_BIT_WAMR_BUILD_TARGET = X86_32 +else ifeq ($(BR2_x86_64),y) +FLUENT_BIT_WAMR_BUILD_TARGET = X86_64 +else ifeq ($(BR2_xtensa),y) +FLUENT_BIT_WAMR_BUILD_TARGET = XTENSA +endif +endif + +ifneq ($(FLUENT_BIT_WAMR_BUILD_TARGET),) +FLUENT_BIT_CONF_OPTS += -DFLB_WASM=Yes \ + -DWAMR_BUILD_TARGET=$(FLUENT_BIT_WAMR_BUILD_TARGET) else FLUENT_BIT_CONF_OPTS += -DFLB_WASM=No endif
Fluent-bit isn't very great at supporting the various architectures, so help a little bit by defining the WAMR_BUILD_TARGET ourselves. ./utils/test-pkg -c fluent-bit.config -p fluent-bit -a arm-aarch64 [ 1/45]: OK bootlin-aarch64-glibc [ 2/45]: OK bootlin-arcle-hs38-uclibc [ 3/45]: OK bootlin-armv5-uclibc [ 4/45]: OK bootlin-armv7-glibc [ 5/45]: OK bootlin-armv7m-uclibc [ 6/45]: SKIPPED bootlin-armv7-musl [ 7/45]: OK bootlin-m68k-5208-uclibc [ 8/45]: SKIPPED bootlin-m68k-68040-uclibc [ 9/45]: OK bootlin-microblazeel-uclibc [10/45]: OK bootlin-mipsel32r6-glibc [11/45]: OK bootlin-mipsel-uclibc [12/45]: OK bootlin-nios2-glibc [13/45]: OK bootlin-openrisc-uclibc [14/45]: OK bootlin-powerpc64le-power8-glibc [15/45]: OK bootlin-powerpc-e500mc-uclibc [16/45]: OK bootlin-riscv32-glibc [17/45]: OK bootlin-riscv64-glibc [18/45]: OK bootlin-riscv64-musl [19/45]: OK bootlin-s390x-z13-glibc [20/45]: OK bootlin-sh4-uclibc [21/45]: OK bootlin-sparc64-glibc [22/45]: OK bootlin-sparc-uclibc [23/45]: SKIPPED bootlin-x86-64-glibc [24/45]: OK bootlin-x86-64-musl [25/45]: OK bootlin-x86-64-uclibc [26/45]: OK bootlin-xtensa-uclibc [27/45]: OK br-arm-basic [28/45]: OK br-arm-full-nothread [29/45]: SKIPPED br-arm-full-static [30/45]: SKIPPED br-i386-pentium4-full [31/45]: OK br-i386-pentium-mmx-musl [32/45]: OK br-mips64-n64-full [33/45]: OK br-mips64r6-el-hf-glibc [34/45]: OK br-powerpc-603e-basic-cpp [35/45]: OK br-powerpc64-power7-glibc [36/45]: OK linaro-aarch64-be [37/45]: OK linaro-aarch64 [38/45]: OK linaro-arm [39/45]: OK sourcery-arm-armv4t [40/45]: OK sourcery-arm [41/45]: OK sourcery-arm-thumb2 [42/45]: OK sourcery-mips64 [43/45]: FAILED sourcery-mips [44/45]: FAILED sourcery-nios2 [45/45]: OK Two failures: - sourcery-mips64: core/iwasm/common/arch/invokeNative_mips.s: Assembler messages: core/iwasm/common/arch/invokeNative_mips.s:58: Error: opcode not supported on this processor: mips64r2 (mips64r2) `ldc1 $f12,0($sp)' core/iwasm/common/arch/invokeNative_mips.s:59: Error: opcode not supported on this processor: mips64r2 (mips64r2) `ldc1 $f14,8($sp)' make[4]: *** [src/wasm/CMakeFiles/vmlib-static.dir/build.make:495: src/wasm/CMakeFiles/vmlib-static.dir/__/__/lib/wasm-micro-runtime-WAMR-1.1.1/core/iwasm/common/arch/invokeNative_mips.s.o] Error 1 make[4]: *** Waiting for unfinished jobs.... - sourcery-mips: core/iwasm/common/arch/invokeNative_mips.s: Assembler messages: core/iwasm/common/arch/invokeNative_mips.s:58: Error: opcode not supported on this processor: mips32r2 (mips32r2) `ldc1 $f12,0($sp)' core/iwasm/common/arch/invokeNative_mips.s:59: Error: opcode not supported on this processor: mips32r2 (mips32r2) `ldc1 $f14,8($sp)' make[4]: *** [src/wasm/CMakeFiles/vmlib-static.dir/build.make:495: src/wasm/CMakeFiles/vmlib-static.dir/__/__/lib/wasm-micro-runtime-WAMR-1.1.1/core/iwasm/common/arch/invokeNative_mips.s.o] Error 1 make[4]: *** Waiting for unfinished jobs.... Fix see: https://github.com/bytecodealliance/wasm-micro-runtime/issues/625 >> Hi, building the invokeNative_general.c explicitly is supported now, please pull the latest code and use >> "cmake -DWAMR_BUILD_INVOKE_NATIVE_GENERAL=1 ..." to build it. Signed-off-by: Thomas Devoogdt <thomas@devoogdt.com> --- v2: No change. v3: No change. --- package/fluent-bit/Config.in | 25 ------------------------- package/fluent-bit/fluent-bit.mk | 32 ++++++++++++++++++++++++++++++-- 2 files changed, 30 insertions(+), 27 deletions(-)