From patchwork Fri Jul 10 10:47:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Schmutzler X-Patchwork-Id: 1326668 X-Patchwork-Delegate: freifunk@adrianschmutzler.de Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.openwrt.org (client-ip=2001:8b0:10b:1231::1; helo=merlin.infradead.org; envelope-from=openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=adrianschmutzler.de Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=merlin.20170209 header.b=niyKXVdY; dkim-atps=neutral Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4B38x348bYz9sRf for ; Fri, 10 Jul 2020 20:51:34 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Owner; bh=rD7Z5VzVMGa3mmix1lW3jM/T+Upx3GlJt0VuZREP/Bc=; b=niyKXVdYFbiwkT6XmfxHtH9Mzl SjXNXZcp9Oy/90dDBB75Qap4FS7OYmLbs1ZzkZaovMono6uNbxl6nUbrfyKG6G6snnXnhnfA6SZiU kCMqXEMS47WOPkG2gCnI3EhIG1XoleW/6LN+8oVLBzjUv04DMDh0Jqy7pM5jRdj4fNYJXXO8Pqj2l BOj+Ylm0Mc8tfShnVsZlJ5SLq4cB6hECHQnuhKtCmTE9o3t1AbJrlEDz7UUPU4J9f/O38GxLrmlt8 YZsCbuVf2u/AEQJao8oaFVHyE5fBz4SjJGk/n5ldyteVl8M+cmd33Hi5vZrASNST6xM2CSds8DB2Z NNVtaKqQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jtqah-0001ZZ-Ef; Fri, 10 Jul 2020 10:49:15 +0000 Received: from mout.kundenserver.de ([212.227.126.187]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jtqae-0001YE-LL for openwrt-devel@lists.openwrt.org; Fri, 10 Jul 2020 10:49:13 +0000 Received: from buildfff.adridolf.com ([188.192.134.246]) by mrelayeu.kundenserver.de (mreue011 [212.227.15.167]) with ESMTPSA (Nemesis) id 1Mw9Dg-1klo9Y2Chw-00s6Dd for ; Fri, 10 Jul 2020 12:49:08 +0200 From: Adrian Schmutzler To: openwrt-devel@lists.openwrt.org Subject: [RFC PATCH 1/2] base-files: sysupgrade: add function for conffiles retrieval Date: Fri, 10 Jul 2020 12:47:49 +0200 Message-Id: <20200710104750.1212-1-freifunk@adrianschmutzler.de> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-Provags-ID: V03:K1:sijDHkmUcX3tDkwYWLMpEDqUtjlHiIPYeoV/IYyhJhijFV+RlMj KpJ++wrOfwpmUNU4f1OQaqWo6kaoXCFc2+gcWcfXeeoJ7O5l9nOCrGRIogleTWNK47W5Hko GLKXNpmCWK/cHUXvSqSkEUiBI/wI3yV0MJbOf1rn5iJ4DGmr08PsoWgZ7gEbcS6FCIRNox0 qQVYYJ8a9yZUF3CdQ3EpA== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:wAwP28inEnM=:s3JPE95Dkp/X8DCWuWDlwm 4V7CA0r/b4CzMpwFjdOSt/2/tPdt1ka3O2LhodoIdDJTK9k2MwjCDgWA+NcPqjgY0/trwxn73 505ZWkhcApMBpFwpt8Pxkt6Ov0nN8mvPCyJ/dZT+gBkH0DLTn2CJD7c5FChG+vE+TbIRnhHDN BZeAd34+SSMtjxFQBt27ahRrmePr0jo81/MwQvgHz5ueRjP0NaYWM36wmla5D5QKGswqW1VJS GTcRiIgb2wmAV3G8EnOYSX0tAkSENeTXWu9KZgELFk/8tpZpwWZf/ZwZoiPmyF8o6CVXvWm3k Q3U4x2/Ev4pq/HX26DjD8neWW9SubVbse4XlE/+UsflFd3wKOxjMBgxxnVHyorDpGvj24B8jT Okv6MYRpQ+6Lx2KnjY7Y2tOyj0XL6EaiKk9QIgFo33CA4uC1evzouE9ORIOEbqVfkoDc05WJx I2k3xWYFYNomRQSfVlA3ysYwPAwMGh38Ahm03GF4i+pSTS7OvpX4GhhlJEHozIeRxxIqU9t/S 4gioiEJXYbp9cQpWlAyDUgC3PGY7Jhp9Wm3iKBIZKO34YVNKbM+4R/vHYLTFvh0iwGY4Kvuip DAaehxmTv1qUsZm9DptJSmg1krBcJqJAgbkrO1YmTdWLVubwQou1+PVvlN22jcQ3oPatpo+yH jGNfKI2AZj2chCkVgYK6I6eIRwRZoht+0oS//R/Bv+hkaQlr1/pyHSsfWVV9UW4O8wapyF40n nGiPZ+kKp2uUg5/36MYKoIpN8piRtvAQeBrhNOAZIq84gIHpsUKmtp0MtKaMpywMGARpRUUYC O5iD1xU+U1zGxPk8DdwB7j2HB4V6MsXP3swbAth6b/ThqzIlz32V6aGjYPDiijeZtq6kg90De QGQKgvPflrF9IM8mAYhA== X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200710_064912_923080_30637138 X-CRM114-Status: UNSURE ( 7.30 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -0.0 (/) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-0.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [212.227.126.187 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [212.227.126.187 listed in wl.mailspike.net] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "openwrt-devel" Errors-To: openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org The find command to retrieve files from /etc/sysupgrade.conf and /lib/upgrade/keep.d/* is used twice in almost similar way. Move it into a function to consolidate, enhance readability and make future adjustments easier. Signed-off-by: Adrian Schmutzler Acked-by: Paul Spooren Reviewed-by: Philip Prindeville --- package/base-files/files/sbin/sysupgrade | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/package/base-files/files/sbin/sysupgrade b/package/base-files/files/sbin/sysupgrade index ca2eb298dc..4ecea5a303 100755 --- a/package/base-files/files/sbin/sysupgrade +++ b/package/base-files/files/sbin/sysupgrade @@ -131,12 +131,18 @@ list_changed_conffiles() { done } +list_static_conffiles() { + local filter=$1 + + find $(sed -ne '/^[[:space:]]*$/d; /^#/d; p' \ + /etc/sysupgrade.conf /lib/upgrade/keep.d/* 2>/dev/null) \ + \( -type f -o -type l \) $filter 2>/dev/null +} + add_conffiles() { local file="$1" - ( find $(sed -ne '/^[[:space:]]*$/d; /^#/d; p' \ - /etc/sysupgrade.conf /lib/upgrade/keep.d/* 2>/dev/null) \ - \( -type f -o -type l \) $find_filter 2>/dev/null; - list_changed_conffiles ) | sort -u > "$file" + + ( list_static_conffiles "$find_filter"; list_changed_conffiles ) | sort -u > "$file" return 0 } @@ -154,9 +160,7 @@ add_overlayfiles() { # backup files from /etc/sysupgrade.conf and /lib/upgrade/keep.d, but # ignore those aready controlled by opkg conffiles - find $(sed -ne '/^[[:space:]]*$/d; /^#/d; p' \ - /etc/sysupgrade.conf /lib/upgrade/keep.d/* 2>/dev/null) \ - \( -type f -o -type l \) 2>/dev/null | sort -u | + list_static_conffiles | sort -u | grep -h -v -x -F -f $conffiles > "$keepfiles" # backup conffiles, but only those changed if '-u' From patchwork Fri Jul 10 10:47:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Schmutzler X-Patchwork-Id: 1326669 X-Patchwork-Delegate: freifunk@adrianschmutzler.de Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.openwrt.org (client-ip=2001:8b0:10b:1231::1; helo=merlin.infradead.org; envelope-from=openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=adrianschmutzler.de Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=merlin.20170209 header.b=1B9uCrX7; dkim-atps=neutral Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4B38x34LxJz9sSn for ; Fri, 10 Jul 2020 20:51:34 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=cLtApvyEdd8eP3wd+hfUkA4vMMypi/oZhBnhYNH73I8=; b=1B9uCrX7am3YTNWsjQgvuhbz2Z C66htzFMmyTTaDiM1kcAHXd27j9MG5FhnijIEjjzoFBPQIC8SGaEOx6CigNxEjgTPOM78EzLN7J+x v9XpokEQfXJVx2nqBY+gc4ShmTWNYLZ8IOmcQhQuMcqWgmuplSb4P7qcIuCr0agcdtdpunZE8PoMU Vd8IXZxYxiuAZIFx7fY5NqKINlSF0kwLb1L+n+VWxVyS7HibSqSwhxHGl8JKJD/PNi9SLwXDbgnyp /W7pBxMsca89TU0rtEM5gOE6VZoQmP6QjFQoMXV2JIAxL0YV6wJGlZGn26N2oIMLp5wfJvalusiEA eOWbmUMA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jtqaj-0001Zt-IR; Fri, 10 Jul 2020 10:49:17 +0000 Received: from mout.kundenserver.de ([212.227.126.135]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jtqaf-0001YU-A6 for openwrt-devel@lists.openwrt.org; Fri, 10 Jul 2020 10:49:14 +0000 Received: from buildfff.adridolf.com ([188.192.134.246]) by mrelayeu.kundenserver.de (mreue011 [212.227.15.167]) with ESMTPSA (Nemesis) id 1M89XH-1jz5OB2gxn-005L0s for ; Fri, 10 Jul 2020 12:49:08 +0200 From: Adrian Schmutzler To: openwrt-devel@lists.openwrt.org Subject: [RFC PATCH 2/2] base-files: sysupgrade: fine-tune preserved files during upgrade Date: Fri, 10 Jul 2020 12:47:50 +0200 Message-Id: <20200710104750.1212-2-freifunk@adrianschmutzler.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200710104750.1212-1-freifunk@adrianschmutzler.de> References: <20200710104750.1212-1-freifunk@adrianschmutzler.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:hIr3Gd/3sMDnF+Dv82PdEd3Wtmfl6Gc/Yt17SJF4QbWvmC/e8kX 28jDaOPQS5F0LLJVeT+8rCBGHQaTyzv9laI6ltBb8Pykw/S7pd5qmLtLyfarhw8ic6bufL6 n2V/8FeNEgpFZ1opmCnjQ6XoQp47RCgeFiKF7fSBTrsLuppV9J8dluEOlrPmL2YIXdrbI+y v0ZVnZASS7yWWXCelwIlA== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:f2IvFt6xMYE=:57x4uBf0/EMtyEG+vC5M3u XgH7XV1cbrWKig4dy3QWUTldRNcxh70p6p2cGac6HaBlhIwZtJktwLcA0GT2jbO0iVd95hYN1 7MFSqgIj0YY3qTVCGRR1AJpqJ3CYDKtf0qN38Li8RojLuV5eI6M/d/DtignIIcn9H8lquM27i 8prouz0+cgnLC4o6t0D4KiMQxTADiFAvqDrrOTXzUR/bs3182KsRTD+FT1HA0xddRqHPpjqOi oyhJW0rkvpoBhA7fQtDaMKg13UZhIyqB4a5fEs3qU9Y00Sx+9Ab7f8bFaaIjCIDQ7R2/m/FWE yXueZMYfJxOL5G53WFIsRfnrT081uVqfnN3O/bRPI+d+3BxWkZ3TNV+oJJMyB5SeAeb5DkPbw lMkxXJgiRer9m7UlvLtF85K9ET1d1p5JqWVQQQCuuUEgAUrrW06DMKfXSejpAPfa3LfrHvLHR aB+Ym6zru0lZ4zmQ8VPTSfJq2MszlDby4l28utJ8BqIOb5SmelMHZzxZURWuiWQqmv7216tJY cATMv6HbX7g7w9Jw2wk8ORray0OBZSaOI2duBfFyEXwBJ2OgDC50cm7f3pqT/KL2d69h/Kbqr gbQoVQqFDCwh3Vbwq3SrunNbhSGlUtrtZrtK+wySRLnVI2Vl2bL+HdGSBEPJ8uEv/JCNA3LEE zFFwHJh5Q4l/cwLVA+f9p8t4YgoDeBWp0RiVnzTNBW8QZ2NIqiihh50jdb+M252X7OJBof3MK RXcUQVGsah4cCHzAKHY8PH/9ifIIT3DqgAH3vtzWd2cs1K8TRs+/K2hnW+LxwGmAa8Yc5IF1d rJz2LxnWa68dyiek4UZqpVduu8JnUXYAxYgxkwgIzx+US9xHF2BqGxTMoXffGf+VGdeo7unUQ F4eVtgm8yqiB8joA8w+A== X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200710_064913_604273_2256DC6F X-CRM114-Status: GOOD ( 10.50 ) X-Spam-Score: -0.0 (/) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-0.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [212.227.126.135 listed in list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [212.227.126.135 listed in wl.mailspike.net] X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "openwrt-devel" Errors-To: openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org For some downstream projects it is desirable not to keep the whole set of configuration files, but to have most of them reset during upgrade (and reconfigured afterwards) and just keep a small subset in /etc/sysupgrade.conf. With this patch, the subset of files kept during upgrade is made configurable, while default setup remains untouched: By setting the appropriate variables or providing command line options, one can now selectively disable keeping ... ... the files in /lib/upgrade/keep.d/* by setting SAVE_CONFIG_KEEPD=0 or providing --no-keepd ... the files managed by opkg accessible via /usr/lib/opkg/status by setting SAVE_CONFIG_OPKG=0 or providing --no-opkg If a file is listed in multiple locations, it is kept until it is "disabled" everywhere. Signed-off-by: Adrian Schmutzler Acked-by: Paul Spooren --- package/base-files/files/sbin/sysupgrade | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/package/base-files/files/sbin/sysupgrade b/package/base-files/files/sbin/sysupgrade index 4ecea5a303..e48870497e 100755 --- a/package/base-files/files/sbin/sysupgrade +++ b/package/base-files/files/sbin/sysupgrade @@ -10,6 +10,8 @@ export MTD_CONFIG_ARGS="" export INTERACTIVE=0 export VERBOSE=1 export SAVE_CONFIG=1 +export SAVE_CONFIG_KEEPD=1 +export SAVE_CONFIG_OPKG=1 export SAVE_OVERLAY=0 export SAVE_OVERLAY_PATH= export SAVE_PARTITIONS=1 @@ -32,6 +34,8 @@ while [ -n "$1" ]; do -v) export VERBOSE="$(($VERBOSE + 1))";; -q) export VERBOSE="$(($VERBOSE - 1))";; -n) export SAVE_CONFIG=0;; + --no-keepd) export SAVE_CONFIG_KEEPD=0;; + --no-opkg) export SAVE_CONFIG_OPKG=0;; -c) export SAVE_OVERLAY=1 SAVE_OVERLAY_PATH=/etc;; -o) export SAVE_OVERLAY=1 SAVE_OVERLAY_PATH=/;; -p) export SAVE_PARTITIONS=0;; @@ -73,6 +77,8 @@ upgrade-option: from packages but including changed confs. -u skip from backup files that are equal to those in /rom -n do not save configuration over reflash + --no-keepd do not save (exclude) keep.d configuration files over reflash + --no-opkg do not save (exclude) opkg-managed configuration files over reflash -p do not attempt to restore the partition table after flash. -k include in backup a list of current installed packages at $INSTALLED_PACKAGES @@ -123,6 +129,8 @@ list_conffiles() { } list_changed_conffiles() { + [ $SAVE_CONFIG_OPKG = 1 ] || return + # Cannot handle spaces in filenames - but opkg cannot either... list_conffiles | while read file csum; do [ -r "$file" ] || continue @@ -133,9 +141,11 @@ list_changed_conffiles() { list_static_conffiles() { local filter=$1 + local filelist=/etc/sysupgrade.conf + [ $SAVE_CONFIG_KEEPD = 1 ] && filelist="$filelist /lib/upgrade/keep.d/*" find $(sed -ne '/^[[:space:]]*$/d; /^#/d; p' \ - /etc/sysupgrade.conf /lib/upgrade/keep.d/* 2>/dev/null) \ + $filelist 2>/dev/null) \ \( -type f -o -type l \) $filter 2>/dev/null }