From patchwork Tue Feb 3 03:20:26 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Olivari X-Patchwork-Id: 435689 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from arrakis.dune.hu (arrakis.dune.hu [78.24.191.176]) (using TLSv1.1 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 34FAB1401B1 for ; Tue, 3 Feb 2015 14:21:10 +1100 (AEDT) Received: from arrakis.dune.hu (localhost [127.0.0.1]) by arrakis.dune.hu (Postfix) with ESMTP id 040DA28BFDC; Tue, 3 Feb 2015 04:18:26 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on arrakis.dune.hu X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,T_DKIM_INVALID, T_RP_MATCHES_RCVD autolearn=unavailable version=3.3.2 Received: from arrakis.dune.hu (localhost [127.0.0.1]) by arrakis.dune.hu (Postfix) with ESMTP id 09BF828ADB8 for ; Tue, 3 Feb 2015 04:18:20 +0100 (CET) X-policyd-weight: using cached result; rate: -8.5 Received: from sabertooth01.qualcomm.com (sabertooth01.qualcomm.com [65.197.215.72]) by arrakis.dune.hu (Postfix) with ESMTPS for ; Tue, 3 Feb 2015 04:18:19 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=qca.qualcomm.com; i=@qca.qualcomm.com; q=dns/txt; s=qcdkim; t=1422933651; x=1454469651; h=from:to:cc:subject:date:message-id:mime-version; bh=02j4rhZlY9tPChmYSIvBiYoJgzWJDC+XCK+Q8vDxlLY=; b=FzEJCKUVaq1oKaUsqGUi4nBCk5z1pUPPDeLP6K+OeS9538VU6n9ry7Pm lpFa5X27S8wi+9Korj6F1hiSdA3tlZZQn7DrZLlAPyQ1GySRqv+dH4ku7 wweKGkxu/qZsnmkW03LOU3WuxPEfdlgRC/mexTeTxcuvYHvX+kjed3cXC M=; X-IronPort-AV: E=McAfee;i="5600,1067,7700"; a="82588195" Received: from ironmsg06-lv.qualcomm.com ([10.47.202.185]) by sabertooth01.qualcomm.com with ESMTP; 02 Feb 2015 19:20:48 -0800 X-IronPort-AV: E=Sophos;i="5.09,510,1418112000"; d="scan'208";a="16908964" Received: from nalasexr01g.na.qualcomm.com ([10.49.56.53]) by Ironmsg06-lv.qualcomm.com with ESMTP/TLS/RC4-SHA; 02 Feb 2015 19:20:49 -0800 Received: from mathieu-linux.qualcomm.com (10.80.80.8) by NALASEXR01G.na.qualcomm.com (10.49.56.53) with Microsoft SMTP Server (TLS) id 15.0.995.29; Mon, 2 Feb 2015 19:20:47 -0800 From: Mathieu Olivari To: Date: Mon, 2 Feb 2015 19:20:26 -0800 Message-ID: <1422933629-25793-1-git-send-email-mathieu@qca.qualcomm.com> X-Mailer: git-send-email 1.9.1 MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: NASANEXM01H.na.qualcomm.com (10.85.0.34) To NALASEXR01G.na.qualcomm.com (10.49.56.53) Cc: openwrt-devel@lists.openwrt.org Subject: [OpenWrt-Devel] [Patch v3 1/4] build: allow openwrt.git packages to be replaced by feeds X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openwrt-devel-bounces@lists.openwrt.org Sender: "openwrt-devel" Currently, replacing a package available in openwrt.git requires modifications in openwrt.git, or requires duplicating the package in a feed but with a different name, which causes all kind of problems related to dependencies (all packages selecting it would have to be modified accordingly to select the new package). With this change, if a package with the same name is present both in feeds/ and package/ folders, the one in feeds/ can override the one in package/, both in the menuconfig and during the build, by passing the "-f" option to "./scripts/feeds install" This mechanism is particularly useful for vendor tree, or in general for application which needs to replace one particular package which exists within openwrt.git by a custom/newer version. Signed-off-by: Mathieu Olivari --- Notes: v2: *fix an issue when trying to override packages which include a dependency in the core packages. (ex: lua depends on liblua, which exists in the core, which ended-up in the package not being overridden properly). *address John's concerns about core OpenWrt packages being overriden without notice and control. Override is now disable by default, and "-f" needs to be passed to "./scripts/feeds install" to allow it. v3: *fix the errors reported by ./scripts/feeds while installing regular packages: Use of uninitialized value $override in numeric eq (==) at ./scripts/feeds line 400. include/scan.awk | 17 +++++++++++++++++ include/scan.mk | 2 +- scripts/feeds | 30 ++++++++++++++++++++++-------- 3 files changed, 40 insertions(+), 9 deletions(-) create mode 100644 include/scan.awk diff --git a/include/scan.awk b/include/scan.awk new file mode 100644 index 0000000..39b2977 --- /dev/null +++ b/include/scan.awk @@ -0,0 +1,17 @@ +BEGIN { FS="/" } +$1 ~ /^feeds/ { FEEDS[$NF]=$0 } +$1 !~ /^feeds/ { PKGS[$NF]=$0 } +END { + # Filter-out OpenWrt packages which have a feeds equivalent + for (pkg in PKGS) + if (pkg in FEEDS) + delete PKGS[pkg] + n = asort(PKGS) + for (i=1; i <= n; i++) { + print PKGS[i] + } + n = asort(FEEDS) + for (i=1; i <= n; i++){ + print FEEDS[i] + } +} diff --git a/include/scan.mk b/include/scan.mk index 0998333..138707d 100644 --- a/include/scan.mk +++ b/include/scan.mk @@ -43,7 +43,7 @@ endef $(FILELIST): rm -f $(TMP_DIR)/info/.files-$(SCAN_TARGET)-* - $(call FIND_L, $(SCAN_DIR)) $(SCAN_EXTRA) -mindepth 1 $(if $(SCAN_DEPTH),-maxdepth $(SCAN_DEPTH)) -name Makefile | xargs grep -HE 'call (Build/DefaultTargets|Build(Package|Target)|.+Package)' | sed -e 's#^$(SCAN_DIR)/##' -e 's#/Makefile:.*##' | uniq > $@ + $(call FIND_L, $(SCAN_DIR)) $(SCAN_EXTRA) -mindepth 1 $(if $(SCAN_DEPTH),-maxdepth $(SCAN_DEPTH)) -name Makefile | xargs grep -HE 'call (Build/DefaultTargets|Build(Package|Target)|.+Package)' | sed -e 's#^$(SCAN_DIR)/##' -e 's#/Makefile:.*##' | uniq | awk -f include/scan.awk > $@ $(TMP_DIR)/info/.files-$(SCAN_TARGET).mk: $(FILELIST) ( \ diff --git a/scripts/feeds b/scripts/feeds index 31ad544..4b98d3e 100755 --- a/scripts/feeds +++ b/scripts/feeds @@ -358,6 +358,7 @@ sub is_core_package($) { sub install_package { my $feed = shift; my $name = shift; + my $force = shift; my $ret = 0; $feed = lookup_package($feed, $name); @@ -382,14 +383,26 @@ sub install_package { my $type = $feed->[0]; $src or $src = $name; + # If it's a core package and we don't want to override, just return + !$force and is_core_package($src) and return 0; + # previously installed packages set the runtime package - # newly installed packages set the source package - $installed{$src} and return 0; + # newly installed packages set the source package to 1 + $installed{$src} and $installed{$src} == 1 and return 0; + + # we'll trigger the override only with the 3 conditions below: + # - override is allowed by command line (-f) + # - a package with the same src exists in the core packages list + # - the package previously installed is not from a feed + my $override = 1 if ($force and is_core_package($src) and !$installed{$name}->{feed}); # check previously installed packages - $installed{$name} and return 0; + $installed{$name} and !$override and return 0; $installed{$src} = 1; - warn "Installing package '$src'\n"; + + defined($override) and $override == 1 + and warn "Overriding package '$src'\n" + or warn "Installing package '$src'\n"; $install_method{$type} or do { warn "Unknown installation method: '$type'\n"; @@ -409,7 +422,7 @@ sub install_package { $dep =~ s/^.+://; $dep =~ s/\/.+$//; next unless $dep; - install_package($feed, $dep) == 0 or $ret = 1; + install_package($feed, $dep, 0) == 0 or $ret = 1; } } @@ -436,7 +449,7 @@ sub install { my $feed; my $ret = 0; - getopts('ap:d:h', \%opts); + getopts('ap:d:fh', \%opts); if ($opts{h}) { usage(); @@ -462,7 +475,7 @@ sub install { my $p = $feed_package->{$name}; next if $p->{vdepends}; if( $p->{name} ) { - install_package($feed, $p->{name}) == 0 or $ret = 1; + install_package($feed, $p->{name}, exists($opts{f})) == 0 or $ret = 1; get_feed($f->[1]); } } @@ -470,7 +483,7 @@ sub install { } } else { while ($name = shift @ARGV) { - install_package($feed, $name) == 0 or $ret = 1; + install_package($feed, $name, exists($opts{f})) == 0 or $ret = 1; } } @@ -638,6 +651,7 @@ Commands: -a : Install all packages from all feeds or from the specified feed using the -p option. -p : Prefer this feed when installing packages. -d : Set default for newly installed packages. + -f : Install will be forced even if the package exists in core OpenWrt (override) search [options] : Search for a package Options: