| Message ID | 20230123135253.1733981-1-thomas.devoogdt@barco.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | [v2] package/fluent-bit: add new package | expand |
>>>>> "Thomas" == Thomas Devoogdt <thomas@devoogdt.com> writes: > Fluent Bit is a super fast, lightweight, and highly > scalable logging and metrics processor and forwarder. > Signed-off-by: Thomas Devoogdt <thomas.devoogdt@barco.com> Now we just miss the change summary ;) > +++ b/package/fluent-bit/S99fluent-bit > @@ -0,0 +1,55 @@ > +#!/bin/sh > +# > +# Starts fluent-bit. > +# > +NAME=fluent-bit > +DAEMON=/usr/bin/$NAME check-package complains about this: http://nightly.buildroot.org/#adding-packages-start-script There are also some shellcheck warnings. > +PID_FILE="/var/run/$NAME.pid" > +CONF_FILE="/etc/$NAME/$NAME.conf" > + > +[ -r /etc/default/$NAME ] && . /etc/default/$NAME > + > +start() { > + printf "Starting fluent-bit: " > + start-stop-daemon -S -q -b -m -p $PID_FILE --exec $DAEMON -- -c $CONF_FILE > + [ $? = 0 ] && echo "OK" || echo "FAIL" > +} > + > +stop() { > + printf "Stopping fluent-bit: " > + start-stop-daemon -K -q -p $PID_FILE > + # https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/yaml/configuration-file#config_section > + # The default grace time is set to 5 seconds, so use 6 seconds to have some margin. > + TIMEOUT=6 > + PID=$(cat $PID_FILE 2>/dev/null) > + while kill -0 $PID 2>/dev/null; do > + [ $TIMEOUT -eq 0 ] && echo "FAIL" && return 1 > + TIMEOUT=$((TIMEOUT - 1)) > + sleep 1 > + done > + rm -f $PID_FILE So the pidfile is not removed on timeouts. Is that on purpose? > +++ b/package/fluent-bit/fluent-bit.mk > @@ -0,0 +1,43 @@ > +################################################################################ > +# > +# fluent-bit > +# > +################################################################################ > + > +FLUENT_BIT_VERSION = 2.0.8 > +FLUENT_BIT_SITE = $(call github,fluent,fluent-bit,v$(FLUENT_BIT_VERSION)) > +FLUENT_BIT_LICENSE = Apache-2.0 > +FLUENT_BIT_LICENSE_FILES = LICENSE > +FLUENT_BIT_DEPENDENCIES = host-bison host-flex libyaml libopenssl > + > +FLUENT_BIT_CONF_OPTS += \ > + -DFLB_DEBUG=No \ > + -DFLB_RELEASE=Yes \ > + -DFLB_EXAMPLES=No \ > + -DFLB_CHUNK_TRACE=No \ > + -DFLB_BACKTRACE=No \ > + -DFLB_LUAJIT=No > + > +# Undefining _FILE_OFFSET_BITS here because of a "bug" with glibc fts.h > +# large file support. > +# See https://bugzilla.redhat.com/show_bug.cgi?id=574992 for more information. > +FLUENT_BIT_CONF_OPTS += \ > + -DCMAKE_C_FLAGS="$(TARGET_CFLAGS) -U_FILE_OFFSET_BITS" \ > + -DCMAKE_CXX_FLAGS="$(TARGET_CXXFLAGS) -U_FILE_OFFSET_BITS" > + > +# Move the config files from /usr/etc/ to /etc/. > +FLUENT_BIT_CONF_OPTS += \ > + -DCMAKE_INSTALL_SYSCONFDIR="/etc/" Where do you see that it gets set to /usr/etc/? I just dropped this and ran the configure script and I ended up with the normal: grep SYSCONFDIR build/fluent-bit-2.0.8/CMakeCache.txt CMAKE_INSTALL_SYSCONFDIR:PATH=etc //ADVANCED property for variable: CMAKE_INSTALL_SYSCONFDIR CMAKE_INSTALL_SYSCONFDIR-ADVANCED:INTERNAL=1 And indeed, building it I don't see any references to /usr/etc: strings build/fluent-bit-2.0.8/bin/fluent-bit | grep /etc /etc/machine-id /etc/ssl/certs/ca-bundle.crt /etc/ssl/certs/ /etc/hosts /etc/resolv.conf /etc/nsswitch.conf /etc/host.conf /etc/svc.conf I do see that it gets confused and ends up installing the default config files in usr/etc/fluent-bit. I don't right away see why this is so, but indeed explicitly setting it to /etc fixes that. Notice that it does lead to an invalid service file: https://github.com/fluent/fluent-bit/issues/6619 So maybe a better fix is to just move the files in a post-install hook? > +define FLUENT_BIT_POST_INSTALL_TARGET_HOOK > + cp -dpf $(@D)/lib/libminiz*.so* $(TARGET_DIR)/usr/lib/ > +endef > + > +FLUENT_BIT_POST_INSTALL_TARGET_HOOKS += FLUENT_BIT_POST_INSTALL_TARGET_HOOK Again, why not just build with -DBUILD_SHARED_LIBS=OFF? I see that the package installs a shared library in a custom location: target/usr/lib/fluent-bit/libfluent-bit.so What is that used for / how does it find it? I see there is a FLB_SHARED_LIB option to disable it.
Op ma 23 jan. 2023 om 17:11 schreef Peter Korsgaard <peter@korsgaard.com>: > > >>>>> "Thomas" == Thomas Devoogdt <thomas@devoogdt.com> writes: > > > Fluent Bit is a super fast, lightweight, and highly > > scalable logging and metrics processor and forwarder. > > > Signed-off-by: Thomas Devoogdt <thomas.devoogdt@barco.com> > > Now we just miss the change summary ;) Sorry about that. I'm not comfortable using git-mail. I at least won't be sad if buildroot goes to e.g. GitHub or GitLab. But that is no excuse. > > +++ b/package/fluent-bit/S99fluent-bit > > @@ -0,0 +1,55 @@ > > +#!/bin/sh > > +# > > +# Starts fluent-bit. > > +# > > +NAME=fluent-bit > > +DAEMON=/usr/bin/$NAME > > check-package complains about this: > > http://nightly.buildroot.org/#adding-packages-start-script > > There are also some shellcheck warnings. Will check. > > +PID_FILE="/var/run/$NAME.pid" > > +CONF_FILE="/etc/$NAME/$NAME.conf" > > + > > +[ -r /etc/default/$NAME ] && . /etc/default/$NAME > > + > > +start() { > > + printf "Starting fluent-bit: " > > + start-stop-daemon -S -q -b -m -p $PID_FILE --exec $DAEMON -- -c $CONF_FILE > > + [ $? = 0 ] && echo "OK" || echo "FAIL" > > +} > > + > > +stop() { > > + printf "Stopping fluent-bit: " > > + start-stop-daemon -K -q -p $PID_FILE > > + # https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/yaml/configuration-file#config_section > > + # The default grace time is set to 5 seconds, so use 6 seconds to have some margin. > > + TIMEOUT=6 > > + PID=$(cat $PID_FILE 2>/dev/null) > > + while kill -0 $PID 2>/dev/null; do > > + [ $TIMEOUT -eq 0 ] && echo "FAIL" && return 1 > > + TIMEOUT=$((TIMEOUT - 1)) > > + sleep 1 > > + done > > + rm -f $PID_FILE > > So the pidfile is not removed on timeouts. Is that on purpose? No idea what is preferably, I see at least that it is also not removed in https://nightly.buildroot.org/#adding-packages-start-script. > > +++ b/package/fluent-bit/fluent-bit.mk > > @@ -0,0 +1,43 @@ > > +################################################################################ > > +# > > +# fluent-bit > > +# > > +################################################################################ > > + > > +FLUENT_BIT_VERSION = 2.0.8 > > +FLUENT_BIT_SITE = $(call github,fluent,fluent-bit,v$(FLUENT_BIT_VERSION)) > > +FLUENT_BIT_LICENSE = Apache-2.0 > > +FLUENT_BIT_LICENSE_FILES = LICENSE > > +FLUENT_BIT_DEPENDENCIES = host-bison host-flex libyaml libopenssl > > + > > +FLUENT_BIT_CONF_OPTS += \ > > + -DFLB_DEBUG=No \ > > + -DFLB_RELEASE=Yes \ > > + -DFLB_EXAMPLES=No \ > > + -DFLB_CHUNK_TRACE=No \ > > + -DFLB_BACKTRACE=No \ > > + -DFLB_LUAJIT=No > > + > > +# Undefining _FILE_OFFSET_BITS here because of a "bug" with glibc fts.h > > +# large file support. > > +# See https://bugzilla.redhat.com/show_bug.cgi?id=574992 for more information. > > +FLUENT_BIT_CONF_OPTS += \ > > + -DCMAKE_C_FLAGS="$(TARGET_CFLAGS) -U_FILE_OFFSET_BITS" \ > > + -DCMAKE_CXX_FLAGS="$(TARGET_CXXFLAGS) -U_FILE_OFFSET_BITS" > > + > > +# Move the config files from /usr/etc/ to /etc/. > > +FLUENT_BIT_CONF_OPTS += \ > > + -DCMAKE_INSTALL_SYSCONFDIR="/etc/" > > Where do you see that it gets set to /usr/etc/? I just dropped this and > ran the configure script and I ended up with the normal: > > grep SYSCONFDIR build/fluent-bit-2.0.8/CMakeCache.txt > CMAKE_INSTALL_SYSCONFDIR:PATH=etc > //ADVANCED property for variable: CMAKE_INSTALL_SYSCONFDIR > CMAKE_INSTALL_SYSCONFDIR-ADVANCED:INTERNAL=1 > > And indeed, building it I don't see any references to /usr/etc: > > strings build/fluent-bit-2.0.8/bin/fluent-bit | grep /etc > /etc/machine-id > /etc/ssl/certs/ca-bundle.crt > /etc/ssl/certs/ > /etc/hosts > /etc/resolv.conf > /etc/nsswitch.conf > /etc/host.conf > /etc/svc.conf > > I do see that it gets confused and ends up installing the default config > files in usr/etc/fluent-bit. I don't right away see why this is so, but > indeed explicitly setting it to /etc fixes that. Notice that it does > lead to an invalid service file: > > https://github.com/fluent/fluent-bit/issues/6619 > > So maybe a better fix is to just move the files in a post-install hook? I do prefer to get it done by setting CMAKE_INSTALL_SYSCONFDIR. But perhaps, we could wait the response on that thread? > > +define FLUENT_BIT_POST_INSTALL_TARGET_HOOK > > + cp -dpf $(@D)/lib/libminiz*.so* $(TARGET_DIR)/usr/lib/ > > +endef > > + > > +FLUENT_BIT_POST_INSTALL_TARGET_HOOKS += FLUENT_BIT_POST_INSTALL_TARGET_HOOK > > Again, why not just build with -DBUILD_SHARED_LIBS=OFF? > > I see that the package installs a shared library in a custom > location: > > target/usr/lib/fluent-bit/libfluent-bit.so > > What is that used for / how does it find it? I see there is a > FLB_SHARED_LIB option to disable it. Already changed that in V3. https://patchwork.ozlabs.org/project/buildroot/patch/20230123151459.1899697-1-thomas.devoogdt@barco.com/ > -- > Bye, Peter Korsgaard > Kr, Thomas
>>>>> "Thomas" == Thomas Devoogdt <thomas@devoogdt.com> writes: Hello, >> Now we just miss the change summary ;) > Sorry about that. I'm not comfortable using git-mail. I at least won't > be sad if buildroot goes to e.g. GitHub or GitLab. > But that is no excuse. ;) We do have some documentation on https://buildroot.org/downloads/manual/manual.html#submitting-patches, but the summary is simply just to run git format-patch to generate the patch file, edit it to add the information about the change and then run git send-email on the patch file instead of directly running git send-email to generate patch + send it in one go. >> > +stop() { >> > + printf "Stopping fluent-bit: " >> > + start-stop-daemon -K -q -p $PID_FILE >> > + # https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/yaml/configuration-file#config_section >> > + # The default grace time is set to 5 seconds, so use 6 seconds to have some margin. >> > + TIMEOUT=6 >> > + PID=$(cat $PID_FILE 2>/dev/null) >> > + while kill -0 $PID 2>/dev/null; do >> > + [ $TIMEOUT -eq 0 ] && echo "FAIL" && return 1 >> > + TIMEOUT=$((TIMEOUT - 1)) >> > + sleep 1 >> > + done >> > + rm -f $PID_FILE >> >> So the pidfile is not removed on timeouts. Is that on purpose? > No idea what is preferably, I see at least that it is also not removed in > https://nightly.buildroot.org/#adding-packages-start-script. My point was that you were explictly removing the pid file in the happy case (which is fine), but not in the unhappy one. Perhaps just move the rm line just above the while? >> I do see that it gets confused and ends up installing the default config >> files in usr/etc/fluent-bit. I don't right away see why this is so, but >> indeed explicitly setting it to /etc fixes that. Notice that it does >> lead to an invalid service file: >> >> https://github.com/fluent/fluent-bit/issues/6619 >> >> So maybe a better fix is to just move the files in a post-install hook? > I do prefer to get it done by setting CMAKE_INSTALL_SYSCONFDIR. > But perhaps, we could wait the response on that thread? Ok. Perhaps add a reference to that bugreport in a comment? >> > +define FLUENT_BIT_POST_INSTALL_TARGET_HOOK >> > + cp -dpf $(@D)/lib/libminiz*.so* $(TARGET_DIR)/usr/lib/ >> > +endef >> > + >> > +FLUENT_BIT_POST_INSTALL_TARGET_HOOKS += FLUENT_BIT_POST_INSTALL_TARGET_HOOK >> >> Again, why not just build with -DBUILD_SHARED_LIBS=OFF? >> >> I see that the package installs a shared library in a custom >> location: >> >> target/usr/lib/fluent-bit/libfluent-bit.so >> >> What is that used for / how does it find it? I see there is a >> FLB_SHARED_LIB option to disable it. > Already changed that in V3. > https://patchwork.ozlabs.org/project/buildroot/patch/20230123151459.1899697-1-thomas.devoogdt@barco.com/ I guess that is a reference to the miniz issue rather than the FLB_SHARED_LIB one? Why do you need to patch the sources for that instead of just passing -DBUILD_SHARED_LIBS=OFF in FLUENT_BIT_CONF_OPTS?
diff --git a/package/Config.in b/package/Config.in index 995dae2c57..b281a67157 100644 --- a/package/Config.in +++ b/package/Config.in @@ -2610,6 +2610,7 @@ menu "System tools" source "package/efivar/Config.in" source "package/embiggen-disk/Config.in" source "package/emlog/Config.in" + source "package/fluent-bit/Config.in" source "package/ftop/Config.in" source "package/getent/Config.in" source "package/gkrellm/Config.in" diff --git a/package/fluent-bit/Config.in b/package/fluent-bit/Config.in new file mode 100644 index 0000000000..c8f9d4e489 --- /dev/null +++ b/package/fluent-bit/Config.in @@ -0,0 +1,13 @@ +config BR2_PACKAGE_FLUENT_BIT + bool "fluent-bit" + depends on BR2_TOOLCHAIN_USES_GLIBC + depends on !BR2_STATIC_LIBS + select BR2_PACKAGE_OPENSSL + select BR2_PACKAGE_LIBYAML + help + Fast and Lightweight Logs and Metrics processor. + + https://github.com/fluent/fluent-bit + +comment "fluent-bit needs a glibc toolchain, dynamic library" + depends on !BR2_TOOLCHAIN_USES_GLIBC || BR2_STATIC_LIBS diff --git a/package/fluent-bit/S99fluent-bit b/package/fluent-bit/S99fluent-bit new file mode 100644 index 0000000000..1d5a40e020 --- /dev/null +++ b/package/fluent-bit/S99fluent-bit @@ -0,0 +1,55 @@ +#!/bin/sh +# +# Starts fluent-bit. +# +NAME=fluent-bit +DAEMON=/usr/bin/$NAME +PID_FILE="/var/run/$NAME.pid" +CONF_FILE="/etc/$NAME/$NAME.conf" + +[ -r /etc/default/$NAME ] && . /etc/default/$NAME + +start() { + printf "Starting fluent-bit: " + start-stop-daemon -S -q -b -m -p $PID_FILE --exec $DAEMON -- -c $CONF_FILE + [ $? = 0 ] && echo "OK" || echo "FAIL" +} + +stop() { + printf "Stopping fluent-bit: " + start-stop-daemon -K -q -p $PID_FILE + # https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/yaml/configuration-file#config_section + # The default grace time is set to 5 seconds, so use 6 seconds to have some margin. + TIMEOUT=6 + PID=$(cat $PID_FILE 2>/dev/null) + while kill -0 $PID 2>/dev/null; do + [ $TIMEOUT -eq 0 ] && echo "FAIL" && return 1 + TIMEOUT=$((TIMEOUT - 1)) + sleep 1 + done + rm -f $PID_FILE + echo "OK" +} + +restart() { + stop + start +} + +case "$1" in +start) + start + ;; +stop) + stop + ;; +restart | reload) + restart + ;; +*) + echo "Usage: $0 {start|stop|restart}" + exit 1 + ;; +esac + +exit $? diff --git a/package/fluent-bit/fluent-bit.hash b/package/fluent-bit/fluent-bit.hash new file mode 100644 index 0000000000..ce7cea7b59 --- /dev/null +++ b/package/fluent-bit/fluent-bit.hash @@ -0,0 +1,3 @@ +# Locally computed +sha256 8ff5566389033669feabc9c69a5c6f417dad5c8b066454388e6a706507262acf fluent-bit-2.0.8.tar.gz +sha256 0d542e0c8804e39aa7f37eb00da5a762149dc682d7829451287e11b938e94594 LICENSE diff --git a/package/fluent-bit/fluent-bit.mk b/package/fluent-bit/fluent-bit.mk new file mode 100644 index 0000000000..c213ca2239 --- /dev/null +++ b/package/fluent-bit/fluent-bit.mk @@ -0,0 +1,43 @@ +################################################################################ +# +# fluent-bit +# +################################################################################ + +FLUENT_BIT_VERSION = 2.0.8 +FLUENT_BIT_SITE = $(call github,fluent,fluent-bit,v$(FLUENT_BIT_VERSION)) +FLUENT_BIT_LICENSE = Apache-2.0 +FLUENT_BIT_LICENSE_FILES = LICENSE +FLUENT_BIT_DEPENDENCIES = host-bison host-flex libyaml libopenssl + +FLUENT_BIT_CONF_OPTS += \ + -DFLB_DEBUG=No \ + -DFLB_RELEASE=Yes \ + -DFLB_EXAMPLES=No \ + -DFLB_CHUNK_TRACE=No \ + -DFLB_BACKTRACE=No \ + -DFLB_LUAJIT=No + +# Undefining _FILE_OFFSET_BITS here because of a "bug" with glibc fts.h +# large file support. +# See https://bugzilla.redhat.com/show_bug.cgi?id=574992 for more information. +FLUENT_BIT_CONF_OPTS += \ + -DCMAKE_C_FLAGS="$(TARGET_CFLAGS) -U_FILE_OFFSET_BITS" \ + -DCMAKE_CXX_FLAGS="$(TARGET_CXXFLAGS) -U_FILE_OFFSET_BITS" + +# Move the config files from /usr/etc/ to /etc/. +FLUENT_BIT_CONF_OPTS += \ + -DCMAKE_INSTALL_SYSCONFDIR="/etc/" + +define FLUENT_BIT_INSTALL_INIT_SYSV + $(INSTALL) -D -m 0755 package/fluent-bit/S99fluent-bit \ + $(TARGET_DIR)/etc/init.d/S99fluent-bit +endef + +define FLUENT_BIT_POST_INSTALL_TARGET_HOOK + cp -dpf $(@D)/lib/libminiz*.so* $(TARGET_DIR)/usr/lib/ +endef + +FLUENT_BIT_POST_INSTALL_TARGET_HOOKS += FLUENT_BIT_POST_INSTALL_TARGET_HOOK + +$(eval $(cmake-package))
Fluent Bit is a super fast, lightweight, and highly scalable logging and metrics processor and forwarder. Signed-off-by: Thomas Devoogdt <thomas.devoogdt@barco.com> --- package/Config.in | 1 + package/fluent-bit/Config.in | 13 +++++++ package/fluent-bit/S99fluent-bit | 55 ++++++++++++++++++++++++++++++ package/fluent-bit/fluent-bit.hash | 3 ++ package/fluent-bit/fluent-bit.mk | 43 +++++++++++++++++++++++ 5 files changed, 115 insertions(+) create mode 100644 package/fluent-bit/Config.in create mode 100644 package/fluent-bit/S99fluent-bit create mode 100644 package/fluent-bit/fluent-bit.hash create mode 100644 package/fluent-bit/fluent-bit.mk