| Message ID | 20181108032458.1028-1-christian@paral.in |
|---|---|
| State | Superseded |
| Headers | show |
| Series | [v8,1/1] docker-engine: split docker-{cli, engine}, bump to v18.06.1-ce | expand |
Christian, On Wed, Nov 7, 2018 at 9:25 PM Christian Stewart <christian@paral.in> wrote: > > Docker upstream has split the Docker daemon and CLI into separate codebases: > > - github.com/docker/engine: daemon, "dockerd" binary > - github.com/docker/cli: "docker" command line interface > > This commit splits the docker-engine package into docker-engine and docker-cli. > Conveniently, the Docker project has begun maintaining two separate > release-tagged repositories for the CLI and daemon as of v18.06-ce-rc1. Previous > versions were tagged in a common "docker-ce" repository which makes compilation > awkward for Buildroot, especially due to some limitations in the new Go package > infrastructure. > > Docker repositories "engine" and "cli" recently started tagging releases as > recent as the latest stable v18.06.1. > > The CLI is no longer automatically included with the engine. Users will need to > select BR2_PACKAGE_DOCKER_CLI to produce a both docker and dockerd target > binaries. > > Docker CLI can be statically compiled. This enables usage of the system docker > client binary to access the parent daemon API from within containers, where > shared libraries are not available. I've started to test this series and wondered what the most minimal BR config would look like for the host /parent build? Just systemd and docker-engine? Or are there dependencies on full core-utils / util-linux items? I did look at your SkiffOS project and when I merge a config together their were a good amount of packages and a ~150MB rootfs. Just curious what a minimal build might look like. > > Signed-off-by: Christian Stewart <christian@paral.in> > > --- > > Changes since v4: > > - Use latest stable release v18.06.1 > > Changes since v5: > > - Clarify commit message > - Move legacy option to 2018.11 segment and clarify helptext > - Daemon no longer automatically selects CLI > - Remove duplicate journald build tag merge artifact in docker-engine > - Remove unnecessary IAMSTATIC declaration > - Remove unnecessary containerd -> docker-containerd link > > Changes since v6: > > - Remove static docker client option, causes SIGILL on thumb2 arm > > Changes since v7: > > - Re-introduce static compilation option for client by disabling cgo > - Without cgo, there are no shared-linked dependencies for the docker cli > --- > Config.in.legacy | 9 +++++++ > package/Config.in | 1 + > package/docker-cli/Config.in | 25 +++++++++++++++++++ > package/docker-cli/docker-cli.hash | 2 ++ > package/docker-cli/docker-cli.mk | 31 ++++++++++++++++++++++++ > package/docker-engine/Config.in | 6 ----- > package/docker-engine/docker-engine.hash | 2 +- > package/docker-engine/docker-engine.mk | 25 ++++--------------- > 8 files changed, 74 insertions(+), 27 deletions(-) > create mode 100644 package/docker-cli/Config.in > create mode 100644 package/docker-cli/docker-cli.hash > create mode 100644 package/docker-cli/docker-cli.mk > > diff --git a/Config.in.legacy b/Config.in.legacy > index c3df37ddd9..3ceefc98b1 100644 > --- a/Config.in.legacy > +++ b/Config.in.legacy > @@ -145,6 +145,15 @@ endif > ############################################################################### > comment "Legacy options removed in 2018.11" > > +config BR2_PACKAGE_DOCKER_ENGINE_STATIC_CLIENT > + bool "docker-engine static client option renamed" > + select BR2_LEGACY > + select BR2_PACKAGE_DOCKER_CLI_STATIC > + help > + BR2_PACKAGE_DOCKER_ENGINE_STATIC_CLIENT has been renamed to > + BR2_PACKAGE_DOCKER_CLI_STATIC, following the package split of > + docker-engine and docker-cli. > + > config BR2_TARGET_XLOADER > bool "xloader has been removed" > select BR2_LEGACY > diff --git a/package/Config.in b/package/Config.in > index 712e360402..8ca88e7d28 100644 > --- a/package/Config.in > +++ b/package/Config.in > @@ -2105,6 +2105,7 @@ menu "System tools" > source "package/dcron/Config.in" > source "package/ddrescue/Config.in" > source "package/debianutils/Config.in" > + source "package/docker-cli/Config.in" > source "package/docker-compose/Config.in" > source "package/docker-containerd/Config.in" > source "package/docker-engine/Config.in" > diff --git a/package/docker-cli/Config.in b/package/docker-cli/Config.in > new file mode 100644 > index 0000000000..82c35c453d > --- /dev/null > +++ b/package/docker-cli/Config.in > @@ -0,0 +1,25 @@ > +config BR2_PACKAGE_DOCKER_CLI > + bool "docker-cli" > + depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS > + depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS > + depends on BR2_TOOLCHAIN_HAS_THREADS > + help > + Docker is a platform to build, ship, > + and run applications as lightweight containers. > + > + https://github.com/docker/cli > + > +if BR2_PACKAGE_DOCKER_CLI > + > +config BR2_PACKAGE_DOCKER_CLI_STATIC > + bool "build static client" > + depends on !BR2_STATIC_LIBS > + help > + Build a static docker client. > + > +endif > + > +comment "docker-cli needs a toolchain w/ threads" > + depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS > + depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS > + depends on !BR2_TOOLCHAIN_HAS_THREADS > diff --git a/package/docker-cli/docker-cli.hash b/package/docker-cli/docker-cli.hash > new file mode 100644 > index 0000000000..50a81792d0 > --- /dev/null > +++ b/package/docker-cli/docker-cli.hash > @@ -0,0 +1,2 @@ > +# Locally calculated > +sha256 47eda354afc199693aa338f5b2bb87e4511354c02561ddc6520a06d169921bdf docker-cli-v18.06.1-ce.tar.gz Please add license file hashes for both packages. > diff --git a/package/docker-cli/docker-cli.mk b/package/docker-cli/docker-cli.mk > new file mode 100644 > index 0000000000..46449fa058 > --- /dev/null > +++ b/package/docker-cli/docker-cli.mk > @@ -0,0 +1,31 @@ > +############################################################################### > +# > +# docker-cli > +# > +################################################################################ > + > +DOCKER_CLI_VERSION = v18.06.1-ce > +DOCKER_CLI_SITE = $(call github,docker,cli,$(DOCKER_CLI_VERSION)) > +DOCKER_CLI_WORKSPACE = gopath > + > +DOCKER_CLI_LICENSE = Apache-2.0 > +DOCKER_CLI_LICENSE_FILES = LICENSE > + > +DOCKER_CLI_DEPENDENCIES = host-go host-pkgconf > + > +DOCKER_CLI_TAGS = cgo autogen > +DOCKER_CLI_BUILD_TARGETS = cmd/docker > + > +DOCKER_CLI_LDFLAGS = \ > + -X github.com/docker/cli/cli.GitCommit=$(DOCKER_CLI_VERSION) \ > + -X github.com/docker/cli/cli.Version=$(DOCKER_CLI_VERSION) > + > +ifeq ($(BR2_PACKAGE_DOCKER_CLI_STATIC),y) > +DOCKER_CLI_LDFLAGS += -extldflags '-static' > +DOCKER_CLI_TAGS += osusergo netgo > +DOCKER_CLI_GO_ENV = CGO_ENABLED=no > +endif > + > +DOCKER_CLI_INSTALL_BINS = $(notdir $(DOCKER_CLI_BUILD_TARGETS)) > + > +$(eval $(golang-package)) > diff --git a/package/docker-engine/Config.in b/package/docker-engine/Config.in > index 8feb11b48c..11f17820a7 100644 > --- a/package/docker-engine/Config.in > +++ b/package/docker-engine/Config.in > @@ -26,12 +26,6 @@ config BR2_PACKAGE_DOCKER_ENGINE_DAEMON > config BR2_PACKAGE_DOCKER_ENGINE_EXPERIMENTAL > bool "build experimental features" > > -config BR2_PACKAGE_DOCKER_ENGINE_STATIC_CLIENT > - bool "build static client" > - depends on !BR2_STATIC_LIBS > - help > - Build a static docker client. > - > if BR2_PACKAGE_DOCKER_ENGINE_DAEMON > > config BR2_PACKAGE_DOCKER_ENGINE_DRIVER_BTRFS > diff --git a/package/docker-engine/docker-engine.hash b/package/docker-engine/docker-engine.hash > index 6c0ed338a5..d5167a41a0 100644 > --- a/package/docker-engine/docker-engine.hash > +++ b/package/docker-engine/docker-engine.hash > @@ -1,2 +1,2 @@ > # Locally calculated > -sha256 4716df117d867b82ddab2e82395cd40aa3d0925a689eedcec8919729e4c9f121 docker-engine-v17.05.0-ce.tar.gz > +sha256 df4a3d2100d9b55c36288e721277dc61e4ceded5738423b09fe0d5ed42369f54 docker-engine-v18.06.1-ce.tar.gz > diff --git a/package/docker-engine/docker-engine.mk b/package/docker-engine/docker-engine.mk > index ba84b4cf0a..7023d40f58 100644 > --- a/package/docker-engine/docker-engine.mk > +++ b/package/docker-engine/docker-engine.mk > @@ -4,25 +4,21 @@ > # > ################################################################################ > > -DOCKER_ENGINE_VERSION = v17.05.0-ce > -DOCKER_ENGINE_COMMIT = 89658bed64c2a8fe05a978e5b87dbec409d57a0f > -DOCKER_ENGINE_SITE = $(call github,docker,docker,$(DOCKER_ENGINE_VERSION)) > +DOCKER_ENGINE_VERSION = v18.06.1-ce > +DOCKER_ENGINE_SITE = $(call github,docker,engine,$(DOCKER_ENGINE_VERSION)) > > DOCKER_ENGINE_LICENSE = Apache-2.0 > DOCKER_ENGINE_LICENSE_FILES = LICENSE > > DOCKER_ENGINE_DEPENDENCIES = host-go host-pkgconf > +DOCKER_ENGINE_SRC_SUBDIR = github.com/docker/docker > > DOCKER_ENGINE_LDFLAGS = \ > -X main.GitCommit=$(DOCKER_ENGINE_VERSION) \ > -X main.Version=$(DOCKER_ENGINE_VERSION) > > -ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_STATIC_CLIENT),y) > -DOCKER_ENGINE_LDFLAGS += -extldflags '-static' > -endif > - > DOCKER_ENGINE_TAGS = cgo exclude_graphdriver_zfs autogen > -DOCKER_ENGINE_BUILD_TARGETS = cmd/docker > +DOCKER_ENGINE_BUILD_TARGETS = cmd/dockerd > > ifeq ($(BR2_PACKAGE_LIBSECCOMP),y) > DOCKER_ENGINE_TAGS += seccomp > @@ -30,15 +26,9 @@ DOCKER_ENGINE_DEPENDENCIES += libseccomp > endif > > ifeq ($(BR2_INIT_SYSTEMD),y) > -DOCKER_ENGINE_TAGS += journald > DOCKER_ENGINE_DEPENDENCIES += systemd > +DOCKER_ENGINE_TAGS += systemd journald > endif > - > -ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_DAEMON),y) > -DOCKER_ENGINE_TAGS += daemon > -DOCKER_ENGINE_BUILD_TARGETS += cmd/dockerd > -endif > - > ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_EXPERIMENTAL),y) > DOCKER_ENGINE_TAGS += experimental > endif > @@ -65,7 +55,6 @@ DOCKER_ENGINE_INSTALL_BINS = $(notdir $(DOCKER_ENGINE_BUILD_TARGETS)) > > define DOCKER_ENGINE_RUN_AUTOGEN > cd $(@D) && \ > - GITCOMMIT="$$(echo $(DOCKER_ENGINE_COMMIT) | head -c7)" \ > BUILDTIME="$$(date)" \ > VERSION="$(patsubst v%,%,$(DOCKER_ENGINE_VERSION))" \ > PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" $(TARGET_MAKE_ENV) \ > @@ -74,8 +63,6 @@ endef > > DOCKER_ENGINE_POST_CONFIGURE_HOOKS += DOCKER_ENGINE_RUN_AUTOGEN > > -ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_DAEMON),y) > - > define DOCKER_ENGINE_INSTALL_INIT_SYSTEMD > $(INSTALL) -D -m 0644 $(@D)/contrib/init/systemd/docker.service \ > $(TARGET_DIR)/usr/lib/systemd/system/docker.service > @@ -90,6 +77,4 @@ define DOCKER_ENGINE_USERS > - - docker -1 * - - - Docker Application Container Framework > endef > > -endif > - > $(eval $(golang-package)) > -- > 2.18.1 > > _______________________________________________ > buildroot mailing list > buildroot@busybox.net > http://lists.busybox.net/mailman/listinfo/buildroot
Matthew, Matthew Weber <matthew.weber@rockwellcollins.com> writes: > I've started to test this series and wondered what the most minimal BR > config would look like for the host /parent build? Just systemd and > docker-engine? Or are there dependencies on full core-utils / > util-linux items? I did look at your SkiffOS project and when I merge > a config together their were a good amount of packages and a ~150MB > rootfs. Just curious what a minimal build might look like. This is an interesting question. I had a quick scan of a defconfig Skiff produces, and took some quick notes on what might be immediately removable without consequence: - util-linux: fallocate, kill, line, login, more, newgrp, rfkill, setpriv, su - editors: nano and vim (can be removed) - wpa-supplicant: cli, hotspot, wps support likely can be removed - other utils: file, ssh server (dropbear could be used), mosh, compression tools - libv4l: unsure if necessary The majority of these can be disabled from the base config fragments here: https://github.com/paralin/SkiffOS/tree/master/configs/skiff/standard/buildroot They were originally included due to operational requirements / ease of use in the early days of the project, but are probably not necessary in a "production" image. They can certainly be removed from the base Skiff packages, and left to developers to enable in other packages if needed. I'll begin doing some tests of minimal images this week. Something else to note: it's likely unnecessary to include Docker at all, and these days Docker brings a lot of binary bloat with it. It contains a full cluster management system (swarm) and a kubernetes client, for example. Some alternatives: - http://cri-o.io/ - container runtime interface, will add to BR - runc - can be used independently of Docker - docker-containerd - separate daemon for managing container basics - rkt - coreos container manager - libpod/podman: https://github.com/containers/libpod I hope to add at least cri-o, rkt, and podman support to Buildroot in the near future. >> +sha256 47eda354afc199693aa338f5b2bb87e4511354c02561ddc6520a06d169921bdf docker-cli-v18.06.1-ce.tar.gz > > Please add license file hashes for both packages. Will include in the next revision of the series: # docker-cli.hash sha256 2d81ea060825006fc8f3fe28aa5dc0ffeb80faf325b612c955229157b8c10dc0 LICENSE # docker-engine.hash sha256 2d81ea060825006fc8f3fe28aa5dc0ffeb80faf325b612c955229157b8c10dc0 LICENSE Thanks, Christian
Christian, On Tue, Nov 13, 2018 at 11:53 PM Christian Stewart <christian@paral.in> wrote: > > Matthew, > > Matthew Weber <matthew.weber@rockwellcollins.com> writes: > > I've started to test this series and wondered what the most minimal BR > > config would look like for the host /parent build? Just systemd and > > docker-engine? Or are there dependencies on full core-utils / > > util-linux items? I did look at your SkiffOS project and when I merge > > a config together their were a good amount of packages and a ~150MB > > rootfs. Just curious what a minimal build might look like. > > This is an interesting question. I had a quick scan of a defconfig Skiff > produces, and took some quick notes on what might be immediately > removable without consequence: > > - util-linux: fallocate, kill, line, login, more, newgrp, rfkill, setpriv, su > - editors: nano and vim (can be removed) > - wpa-supplicant: cli, hotspot, wps support likely can be removed > - other utils: file, ssh server (dropbear could be used), mosh, compression tools > - libv4l: unsure if necessary > > The majority of these can be disabled from the base config fragments > here: > > https://github.com/paralin/SkiffOS/tree/master/configs/skiff/standard/buildroot > > They were originally included due to operational requirements / ease of > use in the early days of the project, but are probably not necessary in > a "production" image. They can certainly be removed from the base Skiff > packages, and left to developers to enable in other packages if needed. > > I'll begin doing some tests of minimal images this week. > I did find that the dockercli just hangs when I call it, however I check and the daemon is running. I'll see if I can get more debug (strace, etc). > Something else to note: it's likely unnecessary to include Docker at > all, and these days Docker brings a lot of binary bloat with it. It > contains a full cluster management system (swarm) and a kubernetes > client, for example. > > Some alternatives: > > - http://cri-o.io/ - container runtime interface, will add to BR > - runc - can be used independently of Docker > - docker-containerd - separate daemon for managing container basics > - rkt - coreos container manager > - libpod/podman: https://github.com/containers/libpod > > I hope to add at least cri-o, rkt, and podman support to Buildroot in > the near future. Are any of those OCI compliant and not written in GO? :-) > > >> +sha256 47eda354afc199693aa338f5b2bb87e4511354c02561ddc6520a06d169921bdf docker-cli-v18.06.1-ce.tar.gz > > > > Please add license file hashes for both packages. > > Will include in the next revision of the series: > > # docker-cli.hash > sha256 2d81ea060825006fc8f3fe28aa5dc0ffeb80faf325b612c955229157b8c10dc0 LICENSE > # docker-engine.hash > sha256 2d81ea060825006fc8f3fe28aa5dc0ffeb80faf325b612c955229157b8c10dc0 LICENSE > Thanks for the response and details! Matt
Hi Matthew, Matthew Weber <matthew.weber@rockwellcollins.com> writes: > I did find that the dockercli just hangs when I call it, however I > check and the daemon is running. I'll see if I can get more debug > (strace, etc). The CLI itself is fine, but the daemon sometimes hangs on startup. I have noticed this problem on my laptop as well (x86) so it's not specific to this build or environment, it seems. The daemon will be called, but will not log anything, and basically hang for sometimes minutes. I try restarting it a few times, but that doesn't seem to work. If I wait, it usually starts up eventually. Most of the time the daemon starts immediately, but something seems to trigger this hang. Once the daemon is running and logging messages, the CLI is responsive. It will probably require a strace to figure out why Docker is hanging on startup like this. >> Some alternatives: >> >> - http://cri-o.io/ - container runtime interface, will add to BR >> - runc - can be used independently of Docker >> - docker-containerd - separate daemon for managing container basics >> - rkt - coreos container manager >> - libpod/podman: https://github.com/containers/libpod >> >> I hope to add at least cri-o, rkt, and podman support to Buildroot in >> the near future. > > Are any of those OCI compliant and not written in GO? :-) I haven't seen a OCI compliant runtime that is not Go, yet. Best, Christian
Christian, On Thu, Nov 15, 2018 at 9:22 PM Christian Stewart <christian@paral.in> wrote: > > Hi Matthew, > > Matthew Weber <matthew.weber@rockwellcollins.com> writes: > > I did find that the dockercli just hangs when I call it, however I > > check and the daemon is running. I'll see if I can get more debug > > (strace, etc). > > The CLI itself is fine, but the daemon sometimes hangs on startup. I > have noticed this problem on my laptop as well (x86) so it's not > specific to this build or environment, it seems. > > The daemon will be called, but will not log anything, and basically hang > for sometimes minutes. I try restarting it a few times, but that doesn't > seem to work. If I wait, it usually starts up eventually. > > Most of the time the daemon starts immediately, but something seems to > trigger this hang. > > Once the daemon is running and logging messages, the CLI is responsive. > > It will probably require a strace to figure out why Docker is hanging on > startup like this. I haven't got back to doing more evaluation of docker, but when I stopped working on things, I was able to narrow the hang down to be related to the initialization of random. > > >> Some alternatives: > >> > >> - http://cri-o.io/ - container runtime interface, will add to BR > >> - runc - can be used independently of Docker > >> - docker-containerd - separate daemon for managing container basics > >> - rkt - coreos container manager > >> - libpod/podman: https://github.com/containers/libpod > >> > >> I hope to add at least cri-o, rkt, and podman support to Buildroot in > >> the near future. Cool. The one use case I have been thinking about for new projects is building the most basic possible hosting environment to manage the security footprint. Maybe a LXC system container hosting environment or a non-systemd dockerd. Then inside of that run a systemd based docker deployment which would be easier to swap out and do a true shared filesystem approach vs a system container...... > > > > Are any of those OCI compliant and not written in GO? :-) > > I haven't seen a OCI compliant runtime that is not Go, yet. > > Best, > Christian > Matt
Hi Matthew, Matthew Weber <Matthew.Weber@collins.com> writes: > I haven't got back to doing more evaluation of docker, but when I > stopped working on things, I was able to narrow the hang down to be > related to the initialization of random. I have found the same. My laptop boot up would be blocked for minutes, and it was quite annoying / concerning, but I found that when I pressed some keys / moved the mouse the hang would go away. That would match with a starvation of entropy in the kernel. > The one use case I have been thinking about for new projects is > building the most basic possible hosting environment to manage the > security footprint. Maybe a LXC system container hosting environment > or a non-systemd dockerd. Then inside of that run a systemd based > docker deployment which would be easier to swap out and do a true > shared filesystem approach vs a system container...... I think the most minimum environment would be base Buildroot with either the docker-containerd / runc stack, or something like cri-o (a Kubernetes minimal container runtime I am examining for use with Buildroot). One thing to consider is that the Go dependencies could be compiled together into a single binary, which would reduce binary footprint quite a bit I think. Much of Docker / dockerd features are unnecessary and can be removed as well / mapped to Buildroot options. Docker 2018.09 has been released, and I'm testing it now / will update the next docker-engine build series submission to target v18.09.0. https://github.com/paralin/buildroot/commit/8ac79e55d9495627c891b50dadf8fbde8352c282 Best, Christian
Christian, On Mon, Nov 26, 2018 at 9:30 PM Christian Stewart <christian@paral.in> wrote: > > Hi Matthew, > > Matthew Weber <Matthew.Weber@collins.com> writes: > > I haven't got back to doing more evaluation of docker, but when I > > stopped working on things, I was able to narrow the hang down to be > > related to the initialization of random. > > I have found the same. My laptop boot up would be blocked for minutes, > and it was quite annoying / concerning, but I found that when I pressed > some keys / moved the mouse the hang would go away. That would match > with a starvation of entropy in the kernel. > > > The one use case I have been thinking about for new projects is > > building the most basic possible hosting environment to manage the > > security footprint. Maybe a LXC system container hosting environment > > or a non-systemd dockerd. Then inside of that run a systemd based > > docker deployment which would be easier to swap out and do a true > > shared filesystem approach vs a system container...... > > I think the most minimum environment would be base Buildroot with either > the docker-containerd / runc stack, or something like cri-o (a > Kubernetes minimal container runtime I am examining for use with > Buildroot). One thing to consider is that the Go dependencies could be > compiled together into a single binary, which would reduce binary > footprint quite a bit I think. Much of Docker / dockerd features are > unnecessary and can be removed as well / mapped to Buildroot options. > I noticed that flatpak now has OCI format support. That could be an interesting way to go light weight without a GO dependency..... Matt
Hi Matthew, On Wed, Nov 28, 2018 at 2:35 PM Matthew Weber <Matthew.Weber@collins.com> wrote: > > > The one use case I have been thinking about for new projects is > > > building the most basic possible hosting environment to manage the > > > security footprint. Maybe a LXC system container hosting environment > > > or a non-systemd dockerd. Then inside of that run a systemd based > > > docker deployment which would be easier to swap out and do a true > > > shared filesystem approach vs a system container...... > > > > I think the most minimum environment would be base Buildroot with either > > the docker-containerd / runc stack, or something like cri-o (a > > Kubernetes minimal container runtime I am examining for use with > > Buildroot). One thing to consider is that the Go dependencies could be > > compiled together into a single binary, which would reduce binary > > footprint quite a bit I think. Much of Docker / dockerd features are > > unnecessary and can be removed as well / mapped to Buildroot options. > > > > I noticed that flatpak now has OCI format support. That could be an > interesting way to go light weight without a GO dependency..... I still think either the cri-o (your aversion to Go is noted) or Flatpak is the right approach. To that end I've implemented Flatpak and related dependencies in Buildroot and have a working patchset here: https://github.com/paralin/buildroot/commits/add-flatpak Will begin testing over the next week. Best regards, Christian
>>>>> "Christian" == Christian Stewart <christian@paral.in> writes: > Docker upstream has split the Docker daemon and CLI into separate codebases: > - github.com/docker/engine: daemon, "dockerd" binary > - github.com/docker/cli: "docker" command line interface > This commit splits the docker-engine package into docker-engine and docker-cli. > Conveniently, the Docker project has begun maintaining two separate > release-tagged repositories for the CLI and daemon as of v18.06-ce-rc1. Previous > versions were tagged in a common "docker-ce" repository which makes compilation > awkward for Buildroot, especially due to some limitations in the new Go package > infrastructure. > Docker repositories "engine" and "cli" recently started tagging releases as > recent as the latest stable v18.06.1. > The CLI is no longer automatically included with the engine. Users will need to > select BR2_PACKAGE_DOCKER_CLI to produce a both docker and dockerd target > binaries. > Docker CLI can be statically compiled. This enables usage of the system docker > client binary to access the parent daemon API from within containers, where > shared libraries are not available. > Signed-off-by: Christian Stewart <christian@paral.in> Committed to 2018.11.x, thanks.
diff --git a/Config.in.legacy b/Config.in.legacy index c3df37ddd9..3ceefc98b1 100644 --- a/Config.in.legacy +++ b/Config.in.legacy @@ -145,6 +145,15 @@ endif ############################################################################### comment "Legacy options removed in 2018.11" +config BR2_PACKAGE_DOCKER_ENGINE_STATIC_CLIENT + bool "docker-engine static client option renamed" + select BR2_LEGACY + select BR2_PACKAGE_DOCKER_CLI_STATIC + help + BR2_PACKAGE_DOCKER_ENGINE_STATIC_CLIENT has been renamed to + BR2_PACKAGE_DOCKER_CLI_STATIC, following the package split of + docker-engine and docker-cli. + config BR2_TARGET_XLOADER bool "xloader has been removed" select BR2_LEGACY diff --git a/package/Config.in b/package/Config.in index 712e360402..8ca88e7d28 100644 --- a/package/Config.in +++ b/package/Config.in @@ -2105,6 +2105,7 @@ menu "System tools" source "package/dcron/Config.in" source "package/ddrescue/Config.in" source "package/debianutils/Config.in" + source "package/docker-cli/Config.in" source "package/docker-compose/Config.in" source "package/docker-containerd/Config.in" source "package/docker-engine/Config.in" diff --git a/package/docker-cli/Config.in b/package/docker-cli/Config.in new file mode 100644 index 0000000000..82c35c453d --- /dev/null +++ b/package/docker-cli/Config.in @@ -0,0 +1,25 @@ +config BR2_PACKAGE_DOCKER_CLI + bool "docker-cli" + depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS + depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS + depends on BR2_TOOLCHAIN_HAS_THREADS + help + Docker is a platform to build, ship, + and run applications as lightweight containers. + + https://github.com/docker/cli + +if BR2_PACKAGE_DOCKER_CLI + +config BR2_PACKAGE_DOCKER_CLI_STATIC + bool "build static client" + depends on !BR2_STATIC_LIBS + help + Build a static docker client. + +endif + +comment "docker-cli needs a toolchain w/ threads" + depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS + depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS + depends on !BR2_TOOLCHAIN_HAS_THREADS diff --git a/package/docker-cli/docker-cli.hash b/package/docker-cli/docker-cli.hash new file mode 100644 index 0000000000..50a81792d0 --- /dev/null +++ b/package/docker-cli/docker-cli.hash @@ -0,0 +1,2 @@ +# Locally calculated +sha256 47eda354afc199693aa338f5b2bb87e4511354c02561ddc6520a06d169921bdf docker-cli-v18.06.1-ce.tar.gz diff --git a/package/docker-cli/docker-cli.mk b/package/docker-cli/docker-cli.mk new file mode 100644 index 0000000000..46449fa058 --- /dev/null +++ b/package/docker-cli/docker-cli.mk @@ -0,0 +1,31 @@ +############################################################################### +# +# docker-cli +# +################################################################################ + +DOCKER_CLI_VERSION = v18.06.1-ce +DOCKER_CLI_SITE = $(call github,docker,cli,$(DOCKER_CLI_VERSION)) +DOCKER_CLI_WORKSPACE = gopath + +DOCKER_CLI_LICENSE = Apache-2.0 +DOCKER_CLI_LICENSE_FILES = LICENSE + +DOCKER_CLI_DEPENDENCIES = host-go host-pkgconf + +DOCKER_CLI_TAGS = cgo autogen +DOCKER_CLI_BUILD_TARGETS = cmd/docker + +DOCKER_CLI_LDFLAGS = \ + -X github.com/docker/cli/cli.GitCommit=$(DOCKER_CLI_VERSION) \ + -X github.com/docker/cli/cli.Version=$(DOCKER_CLI_VERSION) + +ifeq ($(BR2_PACKAGE_DOCKER_CLI_STATIC),y) +DOCKER_CLI_LDFLAGS += -extldflags '-static' +DOCKER_CLI_TAGS += osusergo netgo +DOCKER_CLI_GO_ENV = CGO_ENABLED=no +endif + +DOCKER_CLI_INSTALL_BINS = $(notdir $(DOCKER_CLI_BUILD_TARGETS)) + +$(eval $(golang-package)) diff --git a/package/docker-engine/Config.in b/package/docker-engine/Config.in index 8feb11b48c..11f17820a7 100644 --- a/package/docker-engine/Config.in +++ b/package/docker-engine/Config.in @@ -26,12 +26,6 @@ config BR2_PACKAGE_DOCKER_ENGINE_DAEMON config BR2_PACKAGE_DOCKER_ENGINE_EXPERIMENTAL bool "build experimental features" -config BR2_PACKAGE_DOCKER_ENGINE_STATIC_CLIENT - bool "build static client" - depends on !BR2_STATIC_LIBS - help - Build a static docker client. - if BR2_PACKAGE_DOCKER_ENGINE_DAEMON config BR2_PACKAGE_DOCKER_ENGINE_DRIVER_BTRFS diff --git a/package/docker-engine/docker-engine.hash b/package/docker-engine/docker-engine.hash index 6c0ed338a5..d5167a41a0 100644 --- a/package/docker-engine/docker-engine.hash +++ b/package/docker-engine/docker-engine.hash @@ -1,2 +1,2 @@ # Locally calculated -sha256 4716df117d867b82ddab2e82395cd40aa3d0925a689eedcec8919729e4c9f121 docker-engine-v17.05.0-ce.tar.gz +sha256 df4a3d2100d9b55c36288e721277dc61e4ceded5738423b09fe0d5ed42369f54 docker-engine-v18.06.1-ce.tar.gz diff --git a/package/docker-engine/docker-engine.mk b/package/docker-engine/docker-engine.mk index ba84b4cf0a..7023d40f58 100644 --- a/package/docker-engine/docker-engine.mk +++ b/package/docker-engine/docker-engine.mk @@ -4,25 +4,21 @@ # ################################################################################ -DOCKER_ENGINE_VERSION = v17.05.0-ce -DOCKER_ENGINE_COMMIT = 89658bed64c2a8fe05a978e5b87dbec409d57a0f -DOCKER_ENGINE_SITE = $(call github,docker,docker,$(DOCKER_ENGINE_VERSION)) +DOCKER_ENGINE_VERSION = v18.06.1-ce +DOCKER_ENGINE_SITE = $(call github,docker,engine,$(DOCKER_ENGINE_VERSION)) DOCKER_ENGINE_LICENSE = Apache-2.0 DOCKER_ENGINE_LICENSE_FILES = LICENSE DOCKER_ENGINE_DEPENDENCIES = host-go host-pkgconf +DOCKER_ENGINE_SRC_SUBDIR = github.com/docker/docker DOCKER_ENGINE_LDFLAGS = \ -X main.GitCommit=$(DOCKER_ENGINE_VERSION) \ -X main.Version=$(DOCKER_ENGINE_VERSION) -ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_STATIC_CLIENT),y) -DOCKER_ENGINE_LDFLAGS += -extldflags '-static' -endif - DOCKER_ENGINE_TAGS = cgo exclude_graphdriver_zfs autogen -DOCKER_ENGINE_BUILD_TARGETS = cmd/docker +DOCKER_ENGINE_BUILD_TARGETS = cmd/dockerd ifeq ($(BR2_PACKAGE_LIBSECCOMP),y) DOCKER_ENGINE_TAGS += seccomp @@ -30,15 +26,9 @@ DOCKER_ENGINE_DEPENDENCIES += libseccomp endif ifeq ($(BR2_INIT_SYSTEMD),y) -DOCKER_ENGINE_TAGS += journald DOCKER_ENGINE_DEPENDENCIES += systemd +DOCKER_ENGINE_TAGS += systemd journald endif - -ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_DAEMON),y) -DOCKER_ENGINE_TAGS += daemon -DOCKER_ENGINE_BUILD_TARGETS += cmd/dockerd -endif - ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_EXPERIMENTAL),y) DOCKER_ENGINE_TAGS += experimental endif @@ -65,7 +55,6 @@ DOCKER_ENGINE_INSTALL_BINS = $(notdir $(DOCKER_ENGINE_BUILD_TARGETS)) define DOCKER_ENGINE_RUN_AUTOGEN cd $(@D) && \ - GITCOMMIT="$$(echo $(DOCKER_ENGINE_COMMIT) | head -c7)" \ BUILDTIME="$$(date)" \ VERSION="$(patsubst v%,%,$(DOCKER_ENGINE_VERSION))" \ PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" $(TARGET_MAKE_ENV) \ @@ -74,8 +63,6 @@ endef DOCKER_ENGINE_POST_CONFIGURE_HOOKS += DOCKER_ENGINE_RUN_AUTOGEN -ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_DAEMON),y) - define DOCKER_ENGINE_INSTALL_INIT_SYSTEMD $(INSTALL) -D -m 0644 $(@D)/contrib/init/systemd/docker.service \ $(TARGET_DIR)/usr/lib/systemd/system/docker.service @@ -90,6 +77,4 @@ define DOCKER_ENGINE_USERS - - docker -1 * - - - Docker Application Container Framework endef -endif - $(eval $(golang-package))
Docker upstream has split the Docker daemon and CLI into separate codebases: - github.com/docker/engine: daemon, "dockerd" binary - github.com/docker/cli: "docker" command line interface This commit splits the docker-engine package into docker-engine and docker-cli. Conveniently, the Docker project has begun maintaining two separate release-tagged repositories for the CLI and daemon as of v18.06-ce-rc1. Previous versions were tagged in a common "docker-ce" repository which makes compilation awkward for Buildroot, especially due to some limitations in the new Go package infrastructure. Docker repositories "engine" and "cli" recently started tagging releases as recent as the latest stable v18.06.1. The CLI is no longer automatically included with the engine. Users will need to select BR2_PACKAGE_DOCKER_CLI to produce a both docker and dockerd target binaries. Docker CLI can be statically compiled. This enables usage of the system docker client binary to access the parent daemon API from within containers, where shared libraries are not available. Signed-off-by: Christian Stewart <christian@paral.in> --- Changes since v4: - Use latest stable release v18.06.1 Changes since v5: - Clarify commit message - Move legacy option to 2018.11 segment and clarify helptext - Daemon no longer automatically selects CLI - Remove duplicate journald build tag merge artifact in docker-engine - Remove unnecessary IAMSTATIC declaration - Remove unnecessary containerd -> docker-containerd link Changes since v6: - Remove static docker client option, causes SIGILL on thumb2 arm Changes since v7: - Re-introduce static compilation option for client by disabling cgo - Without cgo, there are no shared-linked dependencies for the docker cli --- Config.in.legacy | 9 +++++++ package/Config.in | 1 + package/docker-cli/Config.in | 25 +++++++++++++++++++ package/docker-cli/docker-cli.hash | 2 ++ package/docker-cli/docker-cli.mk | 31 ++++++++++++++++++++++++ package/docker-engine/Config.in | 6 ----- package/docker-engine/docker-engine.hash | 2 +- package/docker-engine/docker-engine.mk | 25 ++++--------------- 8 files changed, 74 insertions(+), 27 deletions(-) create mode 100644 package/docker-cli/Config.in create mode 100644 package/docker-cli/docker-cli.hash create mode 100644 package/docker-cli/docker-cli.mk