From patchwork Thu Feb 7 18:37:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 1038280 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="GdBriDtv"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43wSN671nVz9s9G for ; Fri, 8 Feb 2019 06:01:42 +1100 (AEDT) Received: from localhost ([127.0.0.1]:45462 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1grovZ-00026b-Rg for incoming@patchwork.ozlabs.org; Thu, 07 Feb 2019 14:01:37 -0500 Received: from eggs.gnu.org ([209.51.188.92]:41183) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1groYt-0000RQ-VS for qemu-devel@nongnu.org; Thu, 07 Feb 2019 13:38:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1groYr-0007kl-Lh for qemu-devel@nongnu.org; Thu, 07 Feb 2019 13:38:11 -0500 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]:44436) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1groYp-0007TC-Ii for qemu-devel@nongnu.org; Thu, 07 Feb 2019 13:38:09 -0500 Received: by mail-wr1-x443.google.com with SMTP id v16so911298wrn.11 for ; Thu, 07 Feb 2019 10:37:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HG1vTAmvanN6jqH+SWmOxUZc43Hn8ObxxcUu8+Vvrbc=; b=GdBriDtvTdfm+SGg2mPhuVcHL8ffPOEBmRU1Jr/teIr/o/Sdf/6m43wW53cCfkXARt ZxjwCar/EcIrSZo0KimQB/DDuBEPazNfUH8bGovEifDEgbjwy1NsLiRCx+xizrFevu1v t+A1RxjoCrq4UYHiDtt8v1ob95uKGt6HMiQ/Q6s6LzLJGqxEWZGn+IBnpgLr+fAjK4Bu h8mPa81IpoiDqkZGIjcxdP93tkL23L3IUpwBeL+3UbFZ5F9yhoNlIHX3AsN8SMTl1jb1 56hjKS7uflmOaZGwN9kUah0Q/Mnb7g20G+VVsi7HLmJTvwwDWdR8+jyn28DXmJS0WcxI 7rEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HG1vTAmvanN6jqH+SWmOxUZc43Hn8ObxxcUu8+Vvrbc=; b=S2q6Cm0+izUdZsGGWN9rNK0Wc1br7RcVqWrzV8VyW87imvLWLnqtY5Staf9zdwif6v 8i3Wn48qZLZtACg/NqyiB+3Nbc7DzE7PfU3dINIWV+kvM/52qyV/Fvg5lCjyCcqbodop D8pvj/6P/mDYvAut8XfWGsu2l1CTXuaugYXJdEBP6fh0sl2EusgB96le5Gk6Pw5x6LNL OLnCV7rULkT/zVRj9s4vfHfe9Z1XTJv+iUW74ZYC1cwWJvfDeHwJYIdFT3CMK9enu8Re /+h9xhXGHULbkPEPxw5eNnmMcDmk9pgQ5vtnZ/rouZfs6SunxLH2FTivGAJIYLw7TggR Wq6g== X-Gm-Message-State: AHQUAua544zJWw6EgJIdJkFoID/ztmQpdkog1vRckcadiyp84GKKEJmi vtdzuwUTUfOynSBo8NI8XnQq7A== X-Google-Smtp-Source: AHgI3IYS4vMO4RK6VFbz4QxxOm4qT94NBNV662Dn8Y0Gnd8HYaTnMRPpVpUzxNeP7oeGFtOPS/zGfQ== X-Received: by 2002:adf:9b11:: with SMTP id b17mr13666471wrc.168.1549564676358; Thu, 07 Feb 2019 10:37:56 -0800 (PST) Received: from zen.linaroharston ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id u3sm90545wrq.21.2019.02.07.10.37.48 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 07 Feb 2019 10:37:53 -0800 (PST) Received: from zen.linaroharston. (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 16C1D1FF8A; Thu, 7 Feb 2019 18:37:45 +0000 (UTC) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Date: Thu, 7 Feb 2019 18:37:38 +0000 Message-Id: <20190207183744.5054-11-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190207183744.5054-1-alex.bennee@linaro.org> References: <20190207183744.5054-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::443 Subject: [Qemu-devel] [PATCH v2 10/16] tests: docker.py be even smarter with persistent binfmt_misc X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, =?utf-8?q?Alex_Benn=C3=A9e?= Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" If we have a persistent mapping we don't need the QEMU binary copied into the container as the kernel has already opened the file and will pass the fd in. However the support libraries will still need to be there. Signed-off-by: Alex Bennée --- tests/docker/docker.py | 56 ++++++++++++++++++++++++++++-------------- 1 file changed, 38 insertions(+), 18 deletions(-) diff --git a/tests/docker/docker.py b/tests/docker/docker.py index 768728785f..a74338cb61 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -96,13 +96,22 @@ def _get_so_libs(executable): return libs -def _copy_binary_with_libs(src, dest_dir): - """Copy a binary executable and all its dependent libraries. +def _copy_binary_with_libs(src, bin_dest, dest_dir): + """Maybe copy a binary and all its dependent libraries. + + If bin_dest isn't set we only copy the support libraries because + we don't need qemu in the docker path to run (due to persistent + mapping). Indeed users may get confused if we aren't running what + is in the image. This does rely on the host file-system being fairly multi-arch - aware so the file don't clash with the guests layout.""" + aware so the file don't clash with the guests layout. + """ - _copy_with_mkdir(src, dest_dir, "/usr/bin") + if bin_dest: + _copy_with_mkdir(src, dest_dir, os.path.dirname(bin_dest)) + else: + print("only copying support libraries for %s" % (src)) libs = _get_so_libs(src) if libs: @@ -116,21 +125,26 @@ def _check_binfmt_misc(executable): The details of setting up binfmt_misc are outside the scope of this script but we should at least fail early with a useful - message if it won't work.""" + message if it won't work. + + Returns the configured binfmt path and a valid flag. For + persistent configurations we will still want to copy and dependent + libraries. + """ binary = os.path.basename(executable) binfmt_entry = "/proc/sys/fs/binfmt_misc/%s" % (binary) if not os.path.exists(binfmt_entry): print ("No binfmt_misc entry for %s" % (binary)) - return None + return None, False with open(binfmt_entry) as x: entry = x.read() if re.search("flags:.*F.*\n", entry): print("binfmt_misc for %s uses persistent(F) mapping to host binary\n" % (binary)) - return None + return None, True m = re.search("interpreter (\S+)\n", entry) interp = m.group(1) @@ -138,7 +152,8 @@ def _check_binfmt_misc(executable): print("binfmt_misc for %s does not point to %s, using %s" % (binary, executable, interp)) - return interp + return interp, True + def _read_qemu_dockerfile(img_name): # special case for Debian linux-user images @@ -345,7 +360,8 @@ class BuildCommand(SubCommand): # Validate binfmt_misc will work if args.include_executable: - if not _check_binfmt_misc(args.include_executable): + qpath, enabled = _check_binfmt_misc(args.include_executable) + if not enabled: return 1 # Is there a .pre file to run in the build context? @@ -368,7 +384,9 @@ class BuildCommand(SubCommand): # FIXME: there is no checksum of this executable and the linked # libraries, once the image built any change of this executable # or any library won't trigger another build. - _copy_binary_with_libs(args.include_executable, docker_dir) + _copy_binary_with_libs(args.include_executable, + qpath, docker_dir) + for filename in args.extra_files or []: _copy_with_mkdir(filename, docker_dir) cksum += [(filename, _file_checksum(filename))] @@ -400,14 +418,16 @@ class UpdateCommand(SubCommand): tmp_tar = TarFile(fileobj=tmp, mode='w') # Add the executable to the tarball, using the current - # configured binfmt_misc path. - ff = _check_binfmt_misc(args.executable) - if not ff: - bn = os.path.basename(args.executable) - ff = "/usr/bin/%s" % bn - print ("No binfmt_misc configured: copied to %s" % (ff)) - - tmp_tar.add(args.executable, arcname=ff) + # configured binfmt_misc path. If we don't get a path then we + # only need the support libraries copied + ff, enabled = _check_binfmt_misc(args.executable) + + if not enabled: + print("binfmt_misc not enabled, update disabled") + return 1 + + if ff: + tmp_tar.add(args.executable, arcname=ff) # Add any associated libraries libs = _get_so_libs(args.executable)