| Message ID | 1468916208-18668-3-git-send-email-famz@redhat.com |
|---|---|
| State | New |
| Headers | show |
On Tue, Jul 19, 2016 at 04:16:42PM +0800, Fam Zheng wrote: > From: Alex Bennée <alex.bennee@linaro.org> > > When passed the path to a binary we copy it and any linked libraries (if > it is dynamically linked) into the docker build context. These can then > be included by a dockerfile with the line: > > # Copy all of context into container > ADD . / > > This is mainly intended for setting up foreign architecture docker > images which use qemu-$arch to do cross-architecture linux-user > execution. It also relies on the host and guest file-system following > reasonable multi-arch layouts so the copied libraries don't clash with > the guest ones. So that's going to fail on anything other than Debian derivatives, since they'll be using /usr/lib or /usr/lib64 for all arches. IMHO it'd be better to simply reject use of qemu-$arch if it is not statically linked, rather than trying to deal with fact that libraries between host FS and foreign guest arch FS may clash. All distros except Fedora have long provided static qemu-$arch builds and I've recently improved Fedora to also provide static qemu$arch builds in F24 & rawhide. So I don't see much compelling reason to support dynamically linked qemu-$arch binaries - it'll just cause pain & suffering with clashing libs on many distros. Regards, Daniel
Daniel P. Berrange <berrange@redhat.com> writes: > On Tue, Jul 19, 2016 at 04:16:42PM +0800, Fam Zheng wrote: >> From: Alex Bennée <alex.bennee@linaro.org> >> >> When passed the path to a binary we copy it and any linked libraries (if >> it is dynamically linked) into the docker build context. These can then >> be included by a dockerfile with the line: >> >> # Copy all of context into container >> ADD . / >> >> This is mainly intended for setting up foreign architecture docker >> images which use qemu-$arch to do cross-architecture linux-user >> execution. It also relies on the host and guest file-system following >> reasonable multi-arch layouts so the copied libraries don't clash with >> the guest ones. > > So that's going to fail on anything other than Debian derivatives, > since they'll be using /usr/lib or /usr/lib64 for all arches. Well currently we only have a bootstrap for Debian anyway. However it doesn't preclude you from using -static builds if you want. Is Debian really the only multi-arch supporting distro out there? For example I tested this in Arch as well. I'm fairly sure Gentoo is doing it right too. > IMHO it'd be better to simply reject use of qemu-$arch if it is > not statically linked, rather than trying to deal with fact that > libraries between host FS and foreign guest arch FS may clash. > > All distros except Fedora have long provided static qemu-$arch > builds and I've recently improved Fedora to also provide static > qemu$arch builds in F24 & rawhide. That's great but we also want to test our built qemus. For example the hot-path stuff has all been tested inside docker containers because it makes the setting up of test cases a lot simpler and reproducible. > > So I don't see much compelling reason to support dynamically > linked qemu-$arch binaries - it'll just cause pain & suffering > with clashing libs on many distros. I do, because I built with dynamic libs all the time for system emulation. This way it is simple to build test other arches when I'm making system emulation changes without having to constantly switch from one build configuration to another. I'll look into if there is any way we can do better with warnings on non-multiarch systems? -- Alex Bennée
diff --git a/tests/docker/docker.py b/tests/docker/docker.py index ae40bb3..96d906e 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -20,6 +20,7 @@ import atexit import uuid import argparse import tempfile +import re from shutil import copy, rmtree def _text_checksum(text): @@ -38,6 +39,54 @@ def _guess_docker_command(): raise Exception("Cannot find working docker command. Tried:\n%s" % \ commands_txt) +def _copy_with_mkdir(src, root_dir, sub_path): + """Copy src into root_dir, creating sub_path as needed.""" + dest_dir = os.path.normpath("%s/%s" % (root_dir, sub_path)) + try: + os.makedirs(dest_dir) + except OSError: + # we can safely ignore already created directories + pass + + dest_file = "%s/%s" % (dest_dir, os.path.basename(src)) + copy(src, dest_file) + + +def _get_so_libs(executable): + """Return a list of libraries associated with an executable. + + The paths may be symbolic links which would need to be resolved to + ensure theright data is copied.""" + + libs = [] + ldd_re = re.compile(r"(/.*/)(\S*)") + try: + ldd_output = subprocess.check_output(["ldd", executable]) + for line in ldd_output.split("\n"): + search = ldd_re.search(line) + if search and len(search.groups()) == 2: + so_path = search.groups()[0] + so_lib = search.groups()[1] + libs.append("%s/%s" % (so_path, so_lib)) + except subprocess.CalledProcessError: + print "%s had no associated libraries (static build?)" % (executable) + + return libs + +def _copy_binary_with_libs(src, dest_dir): + """Copy a binary executable and all its dependant libraries. + + This does rely on the host file-system being fairly multi-arch + aware so the file don't clash with the guests layout.""" + + _copy_with_mkdir(src, dest_dir, "/usr/bin") + + libs = _get_so_libs(src) + if libs: + for l in libs: + so_path = os.path.dirname(l) + _copy_with_mkdir(l , dest_dir, so_path) + class Docker(object): """ Running Docker commands """ def __init__(self): @@ -151,6 +200,10 @@ class BuildCommand(SubCommand): """ Build docker image out of a dockerfile. Arguments: <tag> <dockerfile>""" name = "build" def args(self, parser): + parser.add_argument("--include-executable", "-e", + help="""Specify a binary that will be copied to the + container together with all its dependent + libraries""") parser.add_argument("tag", help="Image Tag") parser.add_argument("dockerfile", @@ -168,6 +221,11 @@ class BuildCommand(SubCommand): # Create a docker context directory for the build docker_dir = tempfile.mkdtemp(prefix="docker_build") + # Do we include a extra binary? + if args.include_executable: + _copy_binary_with_libs(args.include_executable, + docker_dir) + dkr.build_image(tag, docker_dir, dockerfile, quiet=args.quiet, argv=argv)