[v3,2/8] dumb-init: new package

Message ID 20180304004346.331-3-christian@paral.in
State Changes Requested
Headers show
Series
  • Untitled series #31749
Related show

Commit Message

Christian Stewart March 4, 2018, 12:43 a.m.
dumb-init is a minimal init process to act as PID 1 for containers.

Static compilation is used, as dumb-init will typically be executed inside a
container.

dumb-init can satisfy docker-engine's docker-init requirement.

Signed-off-by: Christian Stewart <christian@paral.in>

---

Changes since v1:

 - Upgraded to v1.2.1
 - Added license hash
---
 package/Config.in                |  1 +
 package/dumb-init/Config.in      |  7 +++++++
 package/dumb-init/dumb-init.hash |  3 +++
 package/dumb-init/dumb-init.mk   | 24 ++++++++++++++++++++++++
 4 files changed, 35 insertions(+)
 create mode 100644 package/dumb-init/Config.in
 create mode 100644 package/dumb-init/dumb-init.hash
 create mode 100644 package/dumb-init/dumb-init.mk

Comments

Romain Naour March 31, 2018, 3:42 p.m. | #1
Hi Christian,

Le 04/03/2018 à 01:43, Christian Stewart a écrit :
> dumb-init is a minimal init process to act as PID 1 for containers.
> 
> Static compilation is used, as dumb-init will typically be executed inside a
> container.
> 
> dumb-init can satisfy docker-engine's docker-init requirement.
> 
> Signed-off-by: Christian Stewart <christian@paral.in>
> 
> ---
> 
> Changes since v1:
> 
>  - Upgraded to v1.2.1
>  - Added license hash
> ---
>  package/Config.in                |  1 +
>  package/dumb-init/Config.in      |  7 +++++++
>  package/dumb-init/dumb-init.hash |  3 +++
>  package/dumb-init/dumb-init.mk   | 24 ++++++++++++++++++++++++
>  4 files changed, 35 insertions(+)
>  create mode 100644 package/dumb-init/Config.in
>  create mode 100644 package/dumb-init/dumb-init.hash
>  create mode 100644 package/dumb-init/dumb-init.mk
> 
> diff --git a/package/Config.in b/package/Config.in
> index 5bdaa24340..dd42db79ec 100644
> --- a/package/Config.in
> +++ b/package/Config.in
> @@ -1906,6 +1906,7 @@ comment "Utilities"
>  	source "package/crudini/Config.in"
>  	source "package/dialog/Config.in"
>  	source "package/dtach/Config.in"
> +	source "package/dumb-init/Config.in"
>  	source "package/easy-rsa/Config.in"
>  	source "package/file/Config.in"
>  	source "package/gnupg/Config.in"
> diff --git a/package/dumb-init/Config.in b/package/dumb-init/Config.in
> new file mode 100644
> index 0000000000..e817493605
> --- /dev/null
> +++ b/package/dumb-init/Config.in
> @@ -0,0 +1,7 @@
> +config BR2_PACKAGE_DUMB_INIT

The code use fork(), so a depends on BR2_USE_MMU must be added here.

> +	bool "dumb-init"
> +	help
> +	  dumb-init is a simple but valid init binary to
> +	  act as PID 1 for containers.
> +
> +	  https://github.com/Yelp/dumb-init
> diff --git a/package/dumb-init/dumb-init.hash b/package/dumb-init/dumb-init.hash
> new file mode 100644
> index 0000000000..9e74ff5580
> --- /dev/null
> +++ b/package/dumb-init/dumb-init.hash
> @@ -0,0 +1,3 @@
> +# Locally computed
> +sha256	56858088d9d6c911c227d196fcbfca06aff31fbe83e1519f9cabe5cf6f11b242	dumb-init-v1.2.1.tar.gz
> +sha256	4d74123f166ad8f147e9939bfb451578db84823f22b48fd64b1086d3654a0149  LICENSE
> diff --git a/package/dumb-init/dumb-init.mk b/package/dumb-init/dumb-init.mk
> new file mode 100644
> index 0000000000..4ebbff1930
> --- /dev/null
> +++ b/package/dumb-init/dumb-init.mk
> @@ -0,0 +1,24 @@
> +################################################################################
> +#
> +# dumb-init
> +#
> +################################################################################
> +
> +DUMB_INIT_VERSION = v1.2.1
> +DUMB_INIT_SITE = $(call github,Yelp,dumb-init,$(DUMB_INIT_VERSION))
> +
> +DUMB_INIT_LICENSE = MIT
> +DUMB_INIT_LICENSE_FILES = LICENSE
> +DUMB_INIT_CFLAGS += -static

Why you need to add -static ?
Also $(TARGET_CFLAGS) may be used here.

Best regards,
Romain

> +
> +define DUMB_INIT_BUILD_CMDS
> +	$(TARGET_CC) \
> +		$(DUMB_INIT_CFLAGS) \
> +		-o $(@D)/dumb-init $(@D)/dumb-init.c
> +endef
> +
> +define DUMB_INIT_INSTALL_TARGET_CMDS
> +	$(INSTALL) -D -m 0755 $(@D)/dumb-init $(TARGET_DIR)/usr/bin/dumb-init
> +endef
> +
> +$(eval $(generic-package))
>
Christian Stewart March 31, 2018, 5:42 p.m. | #2
Hi Romain,

On Sat, Mar 31, 2018, 11:42 AM Romain Naour <romain.naour@smile.fr> wrote:

> Le 04/03/2018 à 01:43, Christian Stewart a écrit :
> > Static compilation is used, as dumb-init will typically be executed
> inside a
> > container.
>

See above.

> +config BR2_PACKAGE_DUMB_INIT
>
> The code use fork(), so a depends on BR2_USE_MMU must be added here.
>

Noted.

Also $(TARGET_CFLAGS) may be used here.
>

Also noted :)

Best,
Christian
Hi Romain,<br><br><div class="gmail_quote"><div dir="ltr">On Sat, Mar 31, 2018, 11:42 AM Romain Naour &lt;<a href="mailto:romain.naour@smile.fr">romain.naour@smile.fr</a>&gt; wrote:</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Le 04/03/2018 à 01:43, Christian Stewart a écrit :<br>
&gt; Static compilation is used, as dumb-init will typically be executed inside a<br>
&gt; container.<br></blockquote></div><div><br></div><div>See above. </div><div><br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
&gt; +config BR2_PACKAGE_DUMB_INIT<br>
<br>
The code use fork(), so a depends on BR2_USE_MMU must be added here.<br></blockquote></div><div><br></div><div>Noted. </div><div><br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Also $(TARGET_CFLAGS) may be used here.<br></blockquote></div><div><br></div><div>Also noted :)</div><div><br></div><div>Best,</div><div>Christian</div>
Thomas Petazzoni April 1, 2018, 12:43 p.m. | #3
Hello,

On Sat,  3 Mar 2018 19:43:40 -0500, Christian Stewart wrote:
> dumb-init is a minimal init process to act as PID 1 for containers.
> 
> Static compilation is used, as dumb-init will typically be executed inside a
> container.

For the poor souls that are not Docker gurus, it is not immediately
clear why static linking is needed. dumb-init runs inside the
container, but the container is going to contain (!) other programs.
Will all those programs be statically linked ? Probably not.

So maybe what's happening is that dumb-init is *executed* inside the
container but is actually provided by the host environment outside the
container, and you want to be able to use the same dumb-init binary
regardless of what the container contains. Is that the use case ?

If so, extending the commit log would be nice.

The next issue is that your package assumes that static linking is
possible. But Buildroot defaults to BR2_SHARED_LIBS, which means that
the toolchain may not necessarily provide libc.a and allow static
linking.

So probably dumb-init should "depends on !BR2_SHARED_LIBS", which means
it wouldn't be visible by default. A Config.in comment would be needed,
like:

comment "dumb-init needs a toolchain w/ static library"
	depends on BR2_SHARED_LIBS

Alternatively, you could decide to build dumb-init statically only
if !BR2_SHARED_LIBS, but that means that in the default Buildroot
configurations, dumb-init would be linked dynamically.

Best regards,

Thomas
Christian Stewart April 6, 2018, 9:50 p.m. | #4
Hi Thomas,

Revisiting this now.

Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:
> On Sat,  3 Mar 2018 19:43:40 -0500, Christian Stewart wrote:
>> dumb-init is a minimal init process to act as PID 1 for containers.
>> 
>> Static compilation is used, as dumb-init will typically be executed inside a
>> container.

> So maybe what's happening is that dumb-init is *executed* inside the
> container but is actually provided by the host environment outside the
> container, and you want to be able to use the same dumb-init binary
> regardless of what the container contains. Is that the use case ?

The dumb-init process is executed inside the container and has no access
to the host environment. If it is compiled dynamically, the container
environment must match the target environment in terms of dynamically
linked parts.

> The next issue is that your package assumes that static linking is
> possible. But Buildroot defaults to BR2_SHARED_LIBS, which means that
> the toolchain may not necessarily provide libc.a and allow static
> linking.

I'm able to compile with BR2_SHARED_LIBS and still produce a static
output with -static. Why is this possible? I remember seeing an email
about this at some point, where it was stated that glibc.a or so was not
produced with BR2_SHARED_LIBS. Why can I then compile this package in
this way?

> Alternatively, you could decide to build dumb-init statically only
> if !BR2_SHARED_LIBS, but that means that in the default Buildroot
> configurations, dumb-init would be linked dynamically.

This wouldn't work unfortunately in 95% of the intended cases where
dumb-init would be used.

Best,
Christian Stewart
Thomas Petazzoni April 7, 2018, 6:20 a.m. | #5
Hello,

On Fri, 06 Apr 2018 17:50:30 -0400, Christian Stewart wrote:

> > So maybe what's happening is that dumb-init is *executed* inside the
> > container but is actually provided by the host environment outside the
> > container, and you want to be able to use the same dumb-init binary
> > regardless of what the container contains. Is that the use case ?  
> 
> The dumb-init process is executed inside the container and has no access
> to the host environment. If it is compiled dynamically, the container
> environment must match the target environment in terms of dynamically
> linked parts.

What is important here is that even though dumb-init is executed
*inside* the container, it is provided by the system *outside* the
container.

> > The next issue is that your package assumes that static linking is
> > possible. But Buildroot defaults to BR2_SHARED_LIBS, which means that
> > the toolchain may not necessarily provide libc.a and allow static
> > linking.  
> 
> I'm able to compile with BR2_SHARED_LIBS and still produce a static
> output with -static. Why is this possible? I remember seeing an email
> about this at some point, where it was stated that glibc.a or so was not
> produced with BR2_SHARED_LIBS. Why can I then compile this package in
> this way?

Could you try a completely dynamic build with uClibc and musl ?

Something like this:

BR2_arm=y
BR2_cortex_a8=y
BR2_SHARED_LIBS=y

> > Alternatively, you could decide to build dumb-init statically only
> > if !BR2_SHARED_LIBS, but that means that in the default Buildroot
> > configurations, dumb-init would be linked dynamically.  
> 
> This wouldn't work unfortunately in 95% of the intended cases where
> dumb-init would be used.

Let's see what the above test does. If it's able to build a static
binary even though BR2_SHARED_LIBS=y, then OK, we're good. Otherwise,
we'll have to find a different solution.

Best regards,

Thomas
Christian Stewart April 10, 2018, 4:24 p.m. | #6
Hi Thomas,

Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:
> Could you try a completely dynamic build with uClibc and musl ?
>
> Something like this:
>
> BR2_arm=y
> BR2_cortex_a8=y
> BR2_SHARED_LIBS=y

I completed the test with this defconfig:

BR2_arm=y
BR2_cortex_a8=y
BR2_SHARED_LIBS=y
BR2_PACKAGE_DUMB_INIT=y

The build passed. 

> Let's see what the above test does. If it's able to build a static
> binary even though BR2_SHARED_LIBS=y, then OK, we're good. Otherwise,
> we'll have to find a different solution.

Looks OK to me.

Best,
Christian

Patch

diff --git a/package/Config.in b/package/Config.in
index 5bdaa24340..dd42db79ec 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1906,6 +1906,7 @@  comment "Utilities"
 	source "package/crudini/Config.in"
 	source "package/dialog/Config.in"
 	source "package/dtach/Config.in"
+	source "package/dumb-init/Config.in"
 	source "package/easy-rsa/Config.in"
 	source "package/file/Config.in"
 	source "package/gnupg/Config.in"
diff --git a/package/dumb-init/Config.in b/package/dumb-init/Config.in
new file mode 100644
index 0000000000..e817493605
--- /dev/null
+++ b/package/dumb-init/Config.in
@@ -0,0 +1,7 @@ 
+config BR2_PACKAGE_DUMB_INIT
+	bool "dumb-init"
+	help
+	  dumb-init is a simple but valid init binary to
+	  act as PID 1 for containers.
+
+	  https://github.com/Yelp/dumb-init
diff --git a/package/dumb-init/dumb-init.hash b/package/dumb-init/dumb-init.hash
new file mode 100644
index 0000000000..9e74ff5580
--- /dev/null
+++ b/package/dumb-init/dumb-init.hash
@@ -0,0 +1,3 @@ 
+# Locally computed
+sha256	56858088d9d6c911c227d196fcbfca06aff31fbe83e1519f9cabe5cf6f11b242	dumb-init-v1.2.1.tar.gz
+sha256	4d74123f166ad8f147e9939bfb451578db84823f22b48fd64b1086d3654a0149  LICENSE
diff --git a/package/dumb-init/dumb-init.mk b/package/dumb-init/dumb-init.mk
new file mode 100644
index 0000000000..4ebbff1930
--- /dev/null
+++ b/package/dumb-init/dumb-init.mk
@@ -0,0 +1,24 @@ 
+################################################################################
+#
+# dumb-init
+#
+################################################################################
+
+DUMB_INIT_VERSION = v1.2.1
+DUMB_INIT_SITE = $(call github,Yelp,dumb-init,$(DUMB_INIT_VERSION))
+
+DUMB_INIT_LICENSE = MIT
+DUMB_INIT_LICENSE_FILES = LICENSE
+DUMB_INIT_CFLAGS += -static
+
+define DUMB_INIT_BUILD_CMDS
+	$(TARGET_CC) \
+		$(DUMB_INIT_CFLAGS) \
+		-o $(@D)/dumb-init $(@D)/dumb-init.c
+endef
+
+define DUMB_INIT_INSTALL_TARGET_CMDS
+	$(INSTALL) -D -m 0755 $(@D)/dumb-init $(TARGET_DIR)/usr/bin/dumb-init
+endef
+
+$(eval $(generic-package))