From patchwork Mon Jul 3 15:28:57 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas De Schampheleire X-Patchwork-Id: 783534 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3x1WJf41sjz9s1h for ; Tue, 4 Jul 2017 01:29:22 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="QBf38t4M"; dkim-atps=neutral Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 1465F887D1; Mon, 3 Jul 2017 15:29:21 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GmZ8STjqDopz; Mon, 3 Jul 2017 15:29:20 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 4DCE0887EC; Mon, 3 Jul 2017 15:29:20 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 8E9881C26AA for ; Mon, 3 Jul 2017 15:29:19 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 7B056866F5 for ; Mon, 3 Jul 2017 15:29:19 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sP-wJO0fwHr9 for ; Mon, 3 Jul 2017 15:29:18 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-qk0-f182.google.com (mail-qk0-f182.google.com [209.85.220.182]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 827F5866F0 for ; Mon, 3 Jul 2017 15:29:18 +0000 (UTC) Received: by mail-qk0-f182.google.com with SMTP id p21so148749408qke.3 for ; Mon, 03 Jul 2017 08:29:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=xhR+CeXpzi68lufOjxPQUAvys6SYP1pVFMbBOlbKxGo=; b=QBf38t4MtKHJMj5xBY3/AmZ22OT3zTcxA+iSIqLk1eOVBNheGX29ldkckeDM4EyuXe oRdSEH4I9iQd5Z16a49gHR5YR/d2WfM7vJdIxM+sbG0GqYIBtczkG/y3N4ldAq2TYUfh TRT+6EoWlgfmf9iim8noQ8Im01oOaTzrdk6+xDlO9P/NdxOnZHAOGi5KUtwqC89hgF+w IvQrtw7PntY04W7eTMQw3Sti8sJXUlqIPTd+rYWCN12ILl3QdXWJsnOHLfns78tn5gBX KOKiljmuEoX1o+q2IKIu2uSeQRTTr6O4fB9III9W3PO4Px2XAxku2XnVVsycqdO+pmkc xRWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=xhR+CeXpzi68lufOjxPQUAvys6SYP1pVFMbBOlbKxGo=; b=MeqS/6EiCk/irOnwbVj8OaWzasgl3/WuEhxKlL3h1txD+3gwPM5PTtjhj+i6+qx2tZ V6hmCFHbMazSuhiwUd1BB4vq0+pyVYZB8AvO7WsqIQIbluSzgoH3hOoRuK7Dv+Yl76Kz srhd3DNt0Rp4Fr4tXFxd893+p0jOgBL16tUTLWeX3wsuYfaSoFNwz6z5b/S7YMK2kph3 9ca5GLZpV8lnlGjvHnQSQabsYFJCJJKyegnieSLy+hnTphxTSppvvOVc5azYgqS+NISq PLyKwTyTyyaPcZxMXL+o77cm8DNr3UwqTCXHfznVpKKMCXvC5to+YCpEGx29dEZJlLbf AkUQ== X-Gm-Message-State: AIVw111Uks+gP998icXYFhIEwTxxi2X+LtlyQJcjf5gqrJG//13IZNJB PLHZDxpK4goOLO+cyo5rQ3Ymk/yqoA== X-Received: by 10.55.215.4 with SMTP id m4mr28377896qki.117.1499095757681; Mon, 03 Jul 2017 08:29:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.200.34.10 with HTTP; Mon, 3 Jul 2017 08:28:57 -0700 (PDT) In-Reply-To: References: <20170702103030.2210-1-yann.morin.1998@free.fr> <20170702123621.67df4ae7@windsurf> From: Thomas De Schampheleire Date: Mon, 3 Jul 2017 17:28:57 +0200 Message-ID: To: Arnout Vandecappelle Cc: Thomas Petazzoni , "Yann E. MORIN" , buildroot Subject: Re: [Buildroot] [PATCH] core: postpone evaluation on decompressor dependencies X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" 2017-07-02 13:18 GMT+02:00 Arnout Vandecappelle : > > > On 02-07-17 12:36, Thomas Petazzoni wrote: >>> DL_TOOLS_DEPENDENCIES += $$(call extractor-dependency,$$($(2)_SOURCE)) >>> +UNCOMP_TOOLS_DEPENDENCIES += $$(UNCOMP_TOOL_DEPENDENCY$$(suffix $$($(2)_SOURCE))) >> I know we looked at it yesterday, but I again don't remember what >> extractor-dependency is for. When looking at this code, it seems a bit >> weird that extractor dependencies are added to DL_TOOLS_DEPENDENCIES >> and "uncompressor dependencies" are added to UNCOMP_TOOLS_DEPENDENCIES. >> >> Can we clarify this ? Should the output of extractor-dependency be added >> to UNCOMP_TOOLS_DEPENDENCIES ? Do we need both extractor-dependency and >> UNCOMP_TOOL_DEPENDENCY ? > > It also took me a bit of browsing to find out what this was about. IIUC, > DL_TOOLS_DEPENDENCIES is not actually a list of dependencies, but rather a list > of tools that should be checked by dependencies.mk. In addition, this list mixes > download tools (e.g. git) with extractor/uncompressor tools - and it could in > fact contain anything else. So it would be better to call it something like > DEPENDENCIES_TO_CHECK. > > Also, this patch only looks at the uncompressor tools, but the same approach > could be applied to the download tools. In fact, I'd like those two to be > refactored a little. > > Ideally, we'd even have a distinct name for dependencies.mk "things" (i.e. > tools that may or may not be available on the host and, for some of them, that > we build on demand) and FOO_DEPENDENCIES "things" (i.e. packages that we need to > build before package foo). > > So perhaps: > DL_TOOLS_DEPENDENCIES -> TOOLS_TO_CHECK > UNCOMP_TOOLS_DEPENDENCIES -> TOOLS_DEPENDENCIES > > I have been looking at this in the past too. Here is some input from my side: - host-lzip requires tar to extract. If the tar on the host is not good enough (RHEL4/5), then host-tar is built, but host-lzip does not express a dependency on it. As a quick fix we have currently following patch: - regarding the naming: I agree with Thomas that EXTRACT_TOOLS is better than UNCOMP_TOOLS; - but further, like Arnout, I also think that a better split may be needed. In my rough ideas I had written down: TOOLS_DEPENDENCIES_EXPECTED_ON_HOST versus TOOLS_DEPENDENCIES_BUILT_IF_NEEDED, because that is actually the split we need, regardless of whether it is a tool used for downloading, extracting or something else. Best regards, Thomas diff --git a/support/dependencies/check-host-lzip.mk b/support/dependencies/check-host-lzip.mk --- a/support/dependencies/check-host-lzip.mk +++ b/support/dependencies/check-host-lzip.mk @@ -1,4 +1,7 @@ ifeq (,$(call suitable-host-package,lzip,$(LZCAT))) +ifeq (,$(call suitable-host-package,tar,tar)) +HOST_LZIP_DEPENDENCIES = host-tar +endif DEPENDENCIES_HOST_PREREQ += host-lzip EXTRACTOR_DEPENDENCY_PRECHECKED_EXTENSIONS += .lz LZCAT = $(HOST_DIR)/usr/bin/lzip -d -c - when ccache is enabled, we need to make sure that ccache is not used for any of the dependencies, also for those in DEPENDENCIES_HOST_PREREQ, as ccache is not yet necessarily built. Today, this is happening based on the 'dependencies' target, but if any of them are built via another path, this is not working. One such case had to do with the above lzip/tar problem, but I don't recall all details. Another case is that 'make clean host-ccache' does not work. Following patch fixes that: (we didn't get around to sending that properly yet) diff --git a/support/dependencies/dependencies.mk b/support/dependencies/dependencies.mk --- a/support/dependencies/dependencies.mk +++ b/support/dependencies/dependencies.mk @@ -25,8 +25,8 @@ core-dependencies: DL_TOOLS="$(sort $(DL_TOOLS_DEPENDENCIES))" \ $(TOPDIR)/support/dependencies/dependencies.sh -dependencies: HOSTCC=$(HOSTCC_NOCCACHE) -dependencies: HOSTCXX=$(HOSTCXX_NOCCACHE) +core-dependencies $(DEPENDENCIES_HOST_PREREQ): HOSTCC=$(HOSTCC_NOCCACHE) +core-dependencies $(DEPENDENCIES_HOST_PREREQ): HOSTCXX=$(HOSTCXX_NOCCACHE) dependencies: core-dependencies $(DEPENDENCIES_HOST_PREREQ) ################################################################################