From patchwork Wed Apr 4 23:28:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luiz Angelo Daros de Luca X-Patchwork-Id: 895209 X-Patchwork-Delegate: blogic@openwrt.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (helo) smtp.helo=arrakis.dune.hu (client-ip=78.24.191.176; helo=arrakis.dune.hu; envelope-from=openwrt-devel-bounces@lists.openwrt.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="tpTxxjaR"; dkim-atps=neutral Received: from arrakis.dune.hu (arrakis.dune.hu [78.24.191.176]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40GhxB5Zkhz9ry1 for ; Thu, 5 Apr 2018 09:29:02 +1000 (AEST) Received: from arrakis.dune.hu (localhost [127.0.0.1]) by arrakis.dune.hu (Postfix) with ESMTP id 3DE51B91481; Thu, 5 Apr 2018 01:28:43 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on arrakis.dune.hu X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.1 Received: from arrakis.dune.hu (localhost [127.0.0.1]) by arrakis.dune.hu (Postfix) with ESMTP; Thu, 5 Apr 2018 01:28:43 +0200 (CEST) Received: from arrakis.dune.hu (localhost [127.0.0.1]) by arrakis.dune.hu (Postfix) with ESMTP id 797B1B91460 for ; Thu, 5 Apr 2018 01:28:40 +0200 (CEST) X-policyd-weight: using cached result; rate: -7 Received: from mail-qk0-f195.google.com (mail-qk0-f195.google.com [209.85.220.195]) by arrakis.dune.hu (Postfix) with ESMTPS for ; Thu, 5 Apr 2018 01:28:39 +0200 (CEST) Received: by mail-qk0-f195.google.com with SMTP id w6so24477227qkb.4 for ; Wed, 04 Apr 2018 16:28:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=z9gDxAlYU7Sc9Rusi6QrGB+G155SjgcroQRG7/pFYqo=; b=tpTxxjaR1wDfsd9IxLYprcCBCZ7x6CfddlK+M56GolTK2wgf07XxRDpczsrURwfTwg mnIs6TPiVomnNi5n26C5n93k2cdQa470wKlwHuvPZGSkz9KYh0+uHUFfOT9VCuDe40xs TlvMsdS0Xj8hc+/KURKeOVBiQyzAtA2fSZ7twlrgyKXNFlWZeI+TGaPpvjYFC9xpYzM5 Bhs0eSPtViso5kh/h0ayUslGvPcCV4CgAur+CuRTGVlExvf7q3Cv8YDoM3GDD6Ymdm05 DiuU9zEPi88QUsHsG6nZ1aNt77qxTh26LYbfFoI8gwwGOHuHKUkVifCngsmAzhHrFLmX ZnDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=z9gDxAlYU7Sc9Rusi6QrGB+G155SjgcroQRG7/pFYqo=; b=m8avwU1Ai/L+sdRe9pB+tpUzsEOkAw3Clt28PmBRfeIVVh2DGXgVoKGVFtmcKMonQ/ s2JO5Z6dA13KnLaFpPG3krProFemWSrZhfTZZ2tAy38cXNT4QCdIwXJMRMPdQdq3eAvj Nc2ztUqsqQSn83cffaTP7qh6Kw36DqmOjUVApsqqbZ27WNwo0ee6aTj4ZZmU9HxhGVTj W8h72Rtxut4Smr5QndQg7d3/sATbFAs8cJ5xBTdKdk6HassuU1tnodPAKjXCveToJlK+ cbNp/ISMkf1eBhhI046cmcNBSIq9/aycKPYqWmQBRUSM3le2RVCTVMDtuLe4v1bCbYAf DdCA== X-Gm-Message-State: ALQs6tCKRgjCofZhX9UQ5OIYnoEtWI9p88XD7uhV/k86xY49a0g7vMtP 8J9fpDkqck9NPGvVzFkwhrDxMA== X-Google-Smtp-Source: AIpwx4+trSBEpUUrqFqbiuKhObDrGyfeLaVblvFR97w9MkvGwhBU8Ll95xuWD92nYuYsoE+FybXb9A== X-Received: by 10.55.168.212 with SMTP id r203mr26083548qke.154.1522884518037; Wed, 04 Apr 2018 16:28:38 -0700 (PDT) Received: from tresc043793.redetresc.tre-sc.gov.br ([201.55.123.206]) by smtp.gmail.com with ESMTPSA id z7sm5149132qke.19.2018.04.04.16.28.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Apr 2018 16:28:37 -0700 (PDT) From: Luiz Angelo Daros de Luca To: openwrt-devel@lists.openwrt.org, lede-dev@lists.infradead.org Date: Wed, 4 Apr 2018 20:28:23 -0300 Message-Id: <20180404232826.10815-2-luizluca@gmail.com> X-Mailer: git-send-email 2.16.3 In-Reply-To: <20180404232826.10815-1-luizluca@gmail.com> References: <20171117004110.23143-1-luizluca@gmail.com> <20180404232826.10815-1-luizluca@gmail.com> Subject: [OpenWrt-Devel] [PATCH v2 1/4] base-files: minor cleanups on sysupgrade X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: openwrt-devel-bounces@lists.openwrt.org Sender: "openwrt-devel" Renamed add_uci_conffiles to add_conffiles as it includes any conffiles listed, not only UCI ones. Make do_save_conffiles arg mandatory Allow other options after -l (like -c) Signed-off-by: Luiz Angelo Daros de Luca --- package/base-files/files/sbin/sysupgrade | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/package/base-files/files/sbin/sysupgrade b/package/base-files/files/sbin/sysupgrade index bf5428af25..46e46c3342 100755 --- a/package/base-files/files/sbin/sysupgrade +++ b/package/base-files/files/sbin/sysupgrade @@ -30,7 +30,7 @@ while [ -n "$1" ]; do -p) export SAVE_PARTITIONS=0;; -b|--create-backup) export CONF_BACKUP="$2" NEED_IMAGE=1; shift;; -r|--restore-backup) export CONF_RESTORE="$2" NEED_IMAGE=1; shift;; - -l|--list-backup) export CONF_BACKUP_LIST=1; break;; + -l|--list-backup) export CONF_BACKUP_LIST=1;; -f) export CONF_IMAGE="$2"; shift;; -F|--force) export FORCE=1;; -T|--test) export TEST=1;; @@ -49,10 +49,10 @@ export CONF_TAR=/tmp/sysupgrade.tgz IMAGE="$1" -[ -z "$IMAGE" -a -z "$NEED_IMAGE" -o $HELP -gt 0 ] && { +[ -z "$IMAGE" -a -z "$NEED_IMAGE" -a $CONF_BACKUP_LIST -eq 0 -o $HELP -gt 0 ] && { cat <...] - $0 [-q] [-i] + $0 [-q] [-i] [-c] upgrade-option: -f restore configuration from .tar.gz (file or url) @@ -115,7 +115,7 @@ list_changed_conffiles() { done } -add_uci_conffiles() { +add_conffiles() { local file="$1" ( find $(sed -ne '/^[[:space:]]*$/d; /^#/d; p' \ /etc/sysupgrade.conf /lib/upgrade/keep.d/* 2>/dev/null) \ @@ -131,6 +131,7 @@ add_overlayfiles() { -e '\,/META_[a-zA-Z0-9]*$,d' \ -e '\,/functions.sh$,d' \ -e '\,/[^/]*-opkg$,d' \ + -e '\,/etc/urandom.seed$,d' \ > "$file" return 0 } @@ -146,13 +147,13 @@ if [ $SAVE_OVERLAY = 1 ]; then } sysupgrade_init_conffiles="add_overlayfiles" else - sysupgrade_init_conffiles="add_uci_conffiles" + sysupgrade_init_conffiles="add_conffiles" fi include /lib/upgrade do_save_conffiles() { - local conf_tar="${1:-$CONF_TAR}" + local conf_tar="$1" [ -z "$(rootfs_type)" ] && { echo "Cannot save config while running from ramdisk." @@ -247,7 +248,7 @@ if [ -n "$CONF_IMAGE" ]; then get_image "$CONF_IMAGE" "cat" > "$CONF_TAR" export SAVE_CONFIG=1 elif ask_bool $SAVE_CONFIG "Keep config files over reflash"; then - [ $TEST -eq 1 ] || do_save_conffiles + [ $TEST -eq 1 ] || do_save_conffiles "$CONF_TAR" export SAVE_CONFIG=1 else [ $TEST -eq 1 ] || rm -f "$CONF_TAR" From patchwork Wed Apr 4 23:28:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luiz Angelo Daros de Luca X-Patchwork-Id: 895211 X-Patchwork-Delegate: blogic@openwrt.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (helo) smtp.helo=arrakis.dune.hu (client-ip=78.24.191.176; helo=arrakis.dune.hu; envelope-from=openwrt-devel-bounces@lists.openwrt.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="tSx0Ntq9"; dkim-atps=neutral Received: from arrakis.dune.hu (arrakis.dune.hu [78.24.191.176]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40GhxW6v6qz9ry1 for ; Thu, 5 Apr 2018 09:29:19 +1000 (AEST) Received: from arrakis.dune.hu (localhost [127.0.0.1]) by arrakis.dune.hu (Postfix) with ESMTP id 8F459B91446; Thu, 5 Apr 2018 01:28:46 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on arrakis.dune.hu X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.1 Received: from arrakis.dune.hu (localhost [127.0.0.1]) by arrakis.dune.hu (Postfix) with ESMTP; Thu, 5 Apr 2018 01:28:46 +0200 (CEST) Received: from arrakis.dune.hu (localhost [127.0.0.1]) by arrakis.dune.hu (Postfix) with ESMTP id 1F891B91460 for ; Thu, 5 Apr 2018 01:28:41 +0200 (CEST) X-policyd-weight: using cached result; rate:hard: -7 Received: from mail-qk0-f195.google.com (mail-qk0-f195.google.com [209.85.220.195]) by arrakis.dune.hu (Postfix) with ESMTPS for ; Thu, 5 Apr 2018 01:28:41 +0200 (CEST) Received: by mail-qk0-f195.google.com with SMTP id o205so24461541qke.3 for ; Wed, 04 Apr 2018 16:28:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=rFusgNfTIEUMj2zlZB8qk0i1/WBV8GPdMFtd71dtU9Q=; b=tSx0Ntq9HzyPMxwnj0RrTUni3I/rcBGAF5H3jdM7rEHdK+zpsAImTM5HGxsMsNl7Ab sc84Tt/DjBmdbuS0t7PwdOxiR16r+vn+oVaj4wHpjtoNTuPD+Blpa4352NIdqJlZprFg X54v2B5jEqKWiabpfxLwQWBDlXUgAGzf4MsnXfP8R+FGsQc6C2nG2C8WX/EL7+tLOf/N eo8g8ss6WUwZfsz9CBB45B72YaTnmc7giuqo6nAg7+Wmd+ovLVgZ3yA20QgcDlJDrt48 oEGKhj7tH2APQ2ImeQqwkMIG6KgWMDzJ6Zk0NHClzpmuZwHbh7iYbhmh+reyDdANm40d QW3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=rFusgNfTIEUMj2zlZB8qk0i1/WBV8GPdMFtd71dtU9Q=; b=YIDNJNFQFPtgAS5Ifq5VJ3Q7gQVA8PtPMJgH65HS7KvrLlckEoavytNv3FICy866Wx LSncSWLcnnUjR5kg/73bP4Tq+z101GBqBRe2D6y1rn/yAT4mqzNoos3jH6AJWw6JjbyQ AtVNyQSIVO/jZXmxJfhdHW/3f1tRSfx2JQd2HBUdx194UWxd3srPN3oA0hhkv/TEe204 FCoBW1yYa83wUWBq0dTBLKyRse9O3FNtCds9+5czK8Jwe5kTNPvNYUrLmXebi+XwTf9e ln1GKuyWdV9fymZfuFBVhRj01c5MyvNASndZWP9/6aCFa+znmCqbr6C4bX6MLKupfvMN Ut+A== X-Gm-Message-State: ALQs6tDe4oo5/eoaPMaP7B90gl7CUkiIOyR5ayS8PurzC2bZ9tIK/rvI aQzMwCI2WF57caOS2TEUZU2i5g== X-Google-Smtp-Source: AIpwx4+J+YebuBEzR/uschas8nHMEV/zwGuQ98n39m1tgkN3u8Nmpe3kzKTMLzpQILeOtdox/OBTxQ== X-Received: by 10.55.217.77 with SMTP id u74mr26971054qki.148.1522884519791; Wed, 04 Apr 2018 16:28:39 -0700 (PDT) Received: from tresc043793.redetresc.tre-sc.gov.br ([201.55.123.206]) by smtp.gmail.com with ESMTPSA id z7sm5149132qke.19.2018.04.04.16.28.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Apr 2018 16:28:39 -0700 (PDT) From: Luiz Angelo Daros de Luca To: openwrt-devel@lists.openwrt.org, lede-dev@lists.infradead.org Date: Wed, 4 Apr 2018 20:28:24 -0300 Message-Id: <20180404232826.10815-3-luizluca@gmail.com> X-Mailer: git-send-email 2.16.3 In-Reply-To: <20180404232826.10815-1-luizluca@gmail.com> References: <20171117004110.23143-1-luizluca@gmail.com> <20180404232826.10815-1-luizluca@gmail.com> Subject: [OpenWrt-Devel] [PATCH v2 2/4] base-files: add sysupgrade -u to skip unchanged files X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: openwrt-devel-bounces@lists.openwrt.org Sender: "openwrt-devel" With '-u', for a file /aaa/bbb/ccc enlisted for backup, it will only get into backup if /rom/aaa/bbb/ccc does not exist or /aaa/bbb/ccc is different from /rom/aaa/bbb/ccc. It also works with '-c', but only effective for files touched but not modified. Signed-off-by: Luiz Angelo Daros de Luca --- package/base-files/files/sbin/sysupgrade | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/package/base-files/files/sbin/sysupgrade b/package/base-files/files/sbin/sysupgrade index 46e46c3342..4d221ef5d6 100755 --- a/package/base-files/files/sbin/sysupgrade +++ b/package/base-files/files/sbin/sysupgrade @@ -10,6 +10,7 @@ export VERBOSE=1 export SAVE_CONFIG=1 export SAVE_OVERLAY=0 export SAVE_PARTITIONS=1 +export SKIP_UNCHANGED=0 export CONF_IMAGE= export CONF_BACKUP_LIST=0 export CONF_BACKUP= @@ -28,6 +29,7 @@ while [ -n "$1" ]; do -n) export SAVE_CONFIG=0;; -c) export SAVE_OVERLAY=1;; -p) export SAVE_PARTITIONS=0;; + -u) export SKIP_UNCHANGED=1;; -b|--create-backup) export CONF_BACKUP="$2" NEED_IMAGE=1; shift;; -r|--restore-backup) export CONF_RESTORE="$2" NEED_IMAGE=1; shift;; -l|--list-backup) export CONF_BACKUP_LIST=1;; @@ -52,12 +54,13 @@ IMAGE="$1" [ -z "$IMAGE" -a -z "$NEED_IMAGE" -a $CONF_BACKUP_LIST -eq 0 -o $HELP -gt 0 ] && { cat <...] - $0 [-q] [-i] [-c] + $0 [-q] [-i] [-c] [-u] upgrade-option: -f restore configuration from .tar.gz (file or url) -i interactive mode -c attempt to preserve all changed files in /etc/ + -u skip from backup files that are equals to those in /rom -n do not save configuration over reflash -p do not attempt to restore the partition table after flash. -T | --test @@ -119,20 +122,19 @@ add_conffiles() { local file="$1" ( find $(sed -ne '/^[[:space:]]*$/d; /^#/d; p' \ /etc/sysupgrade.conf /lib/upgrade/keep.d/* 2>/dev/null) \ - -type f -o -type l 2>/dev/null; + \( -type f -o -type l \) $find_filter 2>/dev/null; list_changed_conffiles ) | sort -u > "$file" return 0 } add_overlayfiles() { local file="$1" - find /overlay/upper/etc/ -type f -o -type l | sed \ - -e 's,^/overlay\/upper/,/,' \ - -e '\,/META_[a-zA-Z0-9]*$,d' \ - -e '\,/functions.sh$,d' \ + ( cd /overlay/upper/; find ./etc \( -type f -o -type l \) $find_filter | sed \ + -e 's,^\.,,' \ + -e '\,^/etc/board.json$,d' \ -e '\,/[^/]*-opkg$,d' \ - -e '\,/etc/urandom.seed$,d' \ - > "$file" + -e '\,^/etc/urandom.seed$,d' \ + )> "$file" return 0 } @@ -150,6 +152,15 @@ else sysupgrade_init_conffiles="add_conffiles" fi +find_filter="" +if [ $SKIP_UNCHANGED = 1 ]; then + [ ! -d /rom/ ] && { + echo "'/rom/' is required by '-u'" + exit 1 + } + find_filter='( ( -exec test -e /rom/{} ; -exec cmp -s /{} /rom/{} ; ) -o -print )' +fi + include /lib/upgrade do_save_conffiles() { From patchwork Wed Apr 4 23:28:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luiz Angelo Daros de Luca X-Patchwork-Id: 895213 X-Patchwork-Delegate: blogic@openwrt.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (helo) smtp.helo=arrakis.dune.hu (client-ip=78.24.191.176; helo=arrakis.dune.hu; envelope-from=openwrt-devel-bounces@lists.openwrt.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="PMkW2WeH"; dkim-atps=neutral Received: from arrakis.dune.hu (arrakis.dune.hu [78.24.191.176]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40Ghxn1fbrz9ry1 for ; Thu, 5 Apr 2018 09:29:33 +1000 (AEST) Received: from arrakis.dune.hu (localhost [127.0.0.1]) by arrakis.dune.hu (Postfix) with ESMTP id 1F17CB914A3; Thu, 5 Apr 2018 01:28:50 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on arrakis.dune.hu X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.1 Received: from arrakis.dune.hu (localhost [127.0.0.1]) by arrakis.dune.hu (Postfix) with ESMTP; Thu, 5 Apr 2018 01:28:50 +0200 (CEST) Received: from arrakis.dune.hu (localhost [127.0.0.1]) by arrakis.dune.hu (Postfix) with ESMTP id 2DA36B91480 for ; Thu, 5 Apr 2018 01:28:43 +0200 (CEST) X-policyd-weight: NOT_IN_SBL_XBL_SPAMHAUS=-1.5 NOT_IN_SPAMCOP=-1.5 CL_IP_EQ_HELO_IP=-2 (check from: .gmail. - helo: .mail-qk0-f169.google. - helo-domain: .google.) FROM/MX_MATCHES_HELO(DOMAIN)=-2; rate: -7 Received: from mail-qk0-f169.google.com (mail-qk0-f169.google.com [209.85.220.169]) by arrakis.dune.hu (Postfix) with ESMTPS for ; Thu, 5 Apr 2018 01:28:42 +0200 (CEST) Received: by mail-qk0-f169.google.com with SMTP id c188so24454056qkg.2 for ; Wed, 04 Apr 2018 16:28:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=G7SCLG/RJFm8Yacaz4EUfxl1TM8orL4JWJaNFdrlztc=; b=PMkW2WeHVCmsBwBr9wH7qj52zoUacOy/yNbcAtjo8NpUaBK/GBRe23dPOc36/qs3aY cSQDrIp9EKEIbmiFR42wtWSXBTdQQKA4/WFZyhTuS7FMElP6KICu76+ACzJck3HitN4E /XCHpLMnmh5AqjrMbVpeh9fdzR4LYYTzTjOI39iV6KI7xyWyoqZqn878uJfw/FWl733Q DQcM3SBagrz2YP6JW6/y9gqEa3mxaIG5turXpYphsn0GUcejFk43pL5ksWb/3zPaXshr V9+JIzoi4yAg9AgncjZlX4Lco+KbFUKM8jvV/YOYlgguwNW0rX//Ik5f7bMGaTj59t4A kpxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=G7SCLG/RJFm8Yacaz4EUfxl1TM8orL4JWJaNFdrlztc=; b=C/NgOztECqtTdoQfHqZLepHr2p4dUfwQ8HTdGRJsf1bhIEEeHzJxSvDis8ZaMcxUWX cKziCGSaPHLI8AF1ZspG0SLJe63sjDIGoIN+Lk3XU5grkJhCxoiB9Wz2NKyCd1n4VEJk C7mIwiX6PEmCpDnZHYLH3PHUpOWnwxYkVeBLhzhmF7gSi/7ZMIkTJN6Fdq//2j7wz5bF spimpVTpXkjOK4kKUV4BFGgexdcQctLXcS72hG1nMHglAyraxise+6E3GpzQHMdCcCcf E9LNzmxBovhTPBOD2q60HciHwcXbkSCqie1g8/zq8cSo4dVBmFVrnIPORyVJ3eZlb7cw fppQ== X-Gm-Message-State: ALQs6tBpVAZXyMg2c6qq5CefNOH5gbzBuEkxqV0HRivbmccAsi2oYuAU RcLi+z6xpwfUmvnY4OSvKDT8TQ== X-Google-Smtp-Source: AIpwx485SAFBe7Af2RtHufum4bU3f/+YZPmKQWvv5JepiWOqrAhs/dX1tAEh0mAkux8e3ZeXmoYpXg== X-Received: by 10.55.7.18 with SMTP id 18mr27503426qkh.249.1522884521558; Wed, 04 Apr 2018 16:28:41 -0700 (PDT) Received: from tresc043793.redetresc.tre-sc.gov.br ([201.55.123.206]) by smtp.gmail.com with ESMTPSA id z7sm5149132qke.19.2018.04.04.16.28.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Apr 2018 16:28:41 -0700 (PDT) From: Luiz Angelo Daros de Luca To: openwrt-devel@lists.openwrt.org, lede-dev@lists.infradead.org Date: Wed, 4 Apr 2018 20:28:25 -0300 Message-Id: <20180404232826.10815-4-luizluca@gmail.com> X-Mailer: git-send-email 2.16.3 In-Reply-To: <20180404232826.10815-1-luizluca@gmail.com> References: <20171117004110.23143-1-luizluca@gmail.com> <20180404232826.10815-1-luizluca@gmail.com> Subject: [OpenWrt-Devel] [PATCH v2 3/4] base-files: add sysupgrade -o to save all overlay files X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: openwrt-devel-bounces@lists.openwrt.org Sender: "openwrt-devel" Add sysupgrade '-o' option in order to include all overlay files in backup, except for those that are from packages but including files listed in conffiles, sysupgrade.conf or /lib/upgrade/keep.d. With '-u' option, it will skip files equals to /rom and conffiles that were not changed. Signed-off-by: Luiz Angelo Daros de Luca --- package/base-files/files/sbin/sysupgrade | 46 +++++++++++++++++++++++++++++--- 1 file changed, 43 insertions(+), 3 deletions(-) diff --git a/package/base-files/files/sbin/sysupgrade b/package/base-files/files/sbin/sysupgrade index 4d221ef5d6..01a942bac6 100755 --- a/package/base-files/files/sbin/sysupgrade +++ b/package/base-files/files/sbin/sysupgrade @@ -9,6 +9,7 @@ export INTERACTIVE=0 export VERBOSE=1 export SAVE_CONFIG=1 export SAVE_OVERLAY=0 +export SAVE_OVERLAY_PATH= export SAVE_PARTITIONS=1 export SKIP_UNCHANGED=0 export CONF_IMAGE= @@ -27,7 +28,8 @@ while [ -n "$1" ]; do -v) export VERBOSE="$(($VERBOSE + 1))";; -q) export VERBOSE="$(($VERBOSE - 1))";; -n) export SAVE_CONFIG=0;; - -c) export SAVE_OVERLAY=1;; + -c) export SAVE_OVERLAY=1 SAVE_OVERLAY_PATH=/etc;; + -o) export SAVE_OVERLAY=1 SAVE_OVERLAY_PATH=/;; -p) export SAVE_PARTITIONS=0;; -u) export SKIP_UNCHANGED=1;; -b|--create-backup) export CONF_BACKUP="$2" NEED_IMAGE=1; shift;; @@ -60,6 +62,8 @@ upgrade-option: -f restore configuration from .tar.gz (file or url) -i interactive mode -c attempt to preserve all changed files in /etc/ + -o attempt to preserve all changed files in /, exept those + from packages but including changed confs. -u skip from backup files that are equals to those in /rom -n do not save configuration over reflash -p do not attempt to restore the partition table after flash. @@ -129,12 +133,48 @@ add_conffiles() { add_overlayfiles() { local file="$1" - ( cd /overlay/upper/; find ./etc \( -type f -o -type l \) $find_filter | sed \ + + local packagesfiles=$1.packagesfiles + touch "$packagesfiles" + + if [ "$SAVE_OVERLAY_PATH" = / ]; then + local conffiles=$1.conffiles + local keepfiles=$1.keepfiles + + list_conffiles | cut -f2 -d ' ' | sort -u > "$conffiles" + + # backup files from /etc/sysupgrade.conf and /lib/upgrade/keep.d, but + # ignore those aready controlled by opkg conffiles + find $(sed -ne '/^[[:space:]]*$/d; /^#/d; p' \ + /etc/sysupgrade.conf /lib/upgrade/keep.d/* 2>/dev/null) \ + \( -type f -o -type l \) 2>/dev/null | sort -u | + grep -h -v -x -F -f $conffiles > "$keepfiles" + + # backup conffiles, but only those changed if '-u' + [ $SKIP_UNCHANGED = 1 ] && + list_changed_conffiles | sort -u > "$conffiles" + + # do not backup files from packages, except those listed + # in conffiles and keep.d + find /usr/lib/opkg/info -type f -name "*.list" | sort -u | + xargs -r grep -h -v -x -F -f $conffiles | + grep -v -x -F -f $keepfiles > "$packagesfiles" + rm -f "$keepfiles" "$conffiles" + fi + + # busybox grep bug when file is empty + [ -s "$packagesfiles" ] || echo > $packagesfiles + + ( cd /overlay/upper/; find .$SAVE_OVERLAY_PATH \( -type f -o -type l \) $find_filter | sed \ -e 's,^\.,,' \ -e '\,^/etc/board.json$,d' \ -e '\,/[^/]*-opkg$,d' \ -e '\,^/etc/urandom.seed$,d' \ - )> "$file" + -e '\,^/usr/lib/opkg/.*,d' \ + ) | grep -v -x -F -f $packagesfiles > "$file" + + rm -f "$packagesfiles" + return 0 } From patchwork Wed Apr 4 23:28:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luiz Angelo Daros de Luca X-Patchwork-Id: 895214 X-Patchwork-Delegate: blogic@openwrt.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (helo) smtp.helo=arrakis.dune.hu (client-ip=78.24.191.176; helo=arrakis.dune.hu; envelope-from=openwrt-devel-bounces@lists.openwrt.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="CDq1Kw4s"; dkim-atps=neutral Received: from arrakis.dune.hu (arrakis.dune.hu [78.24.191.176]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40Ghy40Dr5z9s0W for ; Thu, 5 Apr 2018 09:29:47 +1000 (AEST) Received: from arrakis.dune.hu (localhost [127.0.0.1]) by arrakis.dune.hu (Postfix) with ESMTP id A9B8FB914AE; Thu, 5 Apr 2018 01:28:53 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on arrakis.dune.hu X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.1 Received: from arrakis.dune.hu (localhost [127.0.0.1]) by arrakis.dune.hu (Postfix) with ESMTP; Thu, 5 Apr 2018 01:28:53 +0200 (CEST) Received: from arrakis.dune.hu (localhost [127.0.0.1]) by arrakis.dune.hu (Postfix) with ESMTP id 2BB71B91487 for ; Thu, 5 Apr 2018 01:28:45 +0200 (CEST) X-policyd-weight: NOT_IN_SBL_XBL_SPAMHAUS=-1.5 NOT_IN_SPAMCOP=-1.5 CL_IP_EQ_HELO_IP=-2 (check from: .gmail. - helo: .mail-qk0-f196.google. - helo-domain: .google.) FROM/MX_MATCHES_HELO(DOMAIN)=-2; rate: -7 Received: from mail-qk0-f196.google.com (mail-qk0-f196.google.com [209.85.220.196]) by arrakis.dune.hu (Postfix) with ESMTPS for ; Thu, 5 Apr 2018 01:28:44 +0200 (CEST) Received: by mail-qk0-f196.google.com with SMTP id p67so15941781qke.13 for ; Wed, 04 Apr 2018 16:28:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=X/4mz6LYQyCvAH5sw8Chht8yYTZ/eRasKScCnrU5P3Q=; b=CDq1Kw4sDJL0Tac3lCCSrnxYNySsriHrkzpa/NRgaFmyv6XaL8tTM8I6M+XWWaoHBV +JeOv1j2VBHANrRZ+ezE6pFgw6oNfAXIKIliXYvKrul7LdITna1fInTZ4rMUgoyWSSmp xr+6AClMVjg6tnBiHaLtDaAOXfWLBxbelan5o+lHFMj/Jext9j921Jnd0x6LivMjMmGg 2KPXGieE2vbpV0PtR1/mI+XHydmDZNVTTq/iEc1Xt8mYxAfMlvTt0k++vxgsAevZ4v8J Q79QEm5S9Wrpfku9pjvJRt9fd3Tau7bmKdQCobgzghPNZ+DeXwkk78yrixaZpM86VjCJ BsJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=X/4mz6LYQyCvAH5sw8Chht8yYTZ/eRasKScCnrU5P3Q=; b=Biq5bSWTkmDzuvAVokWjmMP6kXjh4P4UbYxKQI+9U0swXvJMxy640DJ3CL6DLw9pqh HwIhZAj5CjEJ64oOWhzUlA3DpqmAD8KuoMAs9/+Uqi4/2hmPNpCD1y0DViJwjI1WE7Qd ljHkmM9MPaSvxE/MtEYTS3jx/Z4XRrRAUCOp2DL5w3srHMOsIKGgxe/tgCWNg7QAtM7H e637yOOWTOmtIIq26CnzzYUA3zN/+DcmTxHC3iLVDNsf4uKxol1NHrZieTdoPH09DeON g5Jesds0NP42wpRV5gwV5YzY51+ZkOxfe+t+Jva4RDUygadmYpT3GMqHsOJHwB8iIYuw BlWw== X-Gm-Message-State: ALQs6tA3t3d2JrEYBnlVfTm5GPn6S94XBAVv3H7Xp7xK40vicYpXe4fc KrDwQeYdq/r5xriRpRNNb2qC2w== X-Google-Smtp-Source: AIpwx4+Ez+adhjzh4KvD840370oVTl243nMiWwaEzdxFRLEggIa7fneozHi5agdxMjDtjF7Kep0Pdg== X-Received: by 10.55.78.86 with SMTP id c83mr27231078qkb.264.1522884523366; Wed, 04 Apr 2018 16:28:43 -0700 (PDT) Received: from tresc043793.redetresc.tre-sc.gov.br ([201.55.123.206]) by smtp.gmail.com with ESMTPSA id z7sm5149132qke.19.2018.04.04.16.28.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Apr 2018 16:28:42 -0700 (PDT) From: Luiz Angelo Daros de Luca To: openwrt-devel@lists.openwrt.org, lede-dev@lists.infradead.org Date: Wed, 4 Apr 2018 20:28:26 -0300 Message-Id: <20180404232826.10815-5-luizluca@gmail.com> X-Mailer: git-send-email 2.16.3 In-Reply-To: <20180404232826.10815-1-luizluca@gmail.com> References: <20171117004110.23143-1-luizluca@gmail.com> <20180404232826.10815-1-luizluca@gmail.com> Subject: [OpenWrt-Devel] [PATCH v2 4/4] base-files: add sysupgrade -k to save list of pkgs X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: openwrt-devel-bounces@lists.openwrt.org Sender: "openwrt-devel" When '-k' is used, sysupgrade inserts into backup a new file /etc/sysupgrade.installed which contains pkgname and origin (rom, overlay, unknown). It's maily used to reinstall all extra packages: # opkg update # grep "\toverlay" etc/sysupgrade.installed | cut -f1 | xargs opkg install # rm etc/sysupgrade.installed Signed-off-by: Luiz Angelo Daros de Luca --- package/base-files/files/sbin/sysupgrade | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/package/base-files/files/sbin/sysupgrade b/package/base-files/files/sbin/sysupgrade index 01a942bac6..c5067a757b 100755 --- a/package/base-files/files/sbin/sysupgrade +++ b/package/base-files/files/sbin/sysupgrade @@ -11,6 +11,7 @@ export SAVE_CONFIG=1 export SAVE_OVERLAY=0 export SAVE_OVERLAY_PATH= export SAVE_PARTITIONS=1 +export SAVE_INSTALLED_PKGS=0 export SKIP_UNCHANGED=0 export CONF_IMAGE= export CONF_BACKUP_LIST=0 @@ -31,6 +32,7 @@ while [ -n "$1" ]; do -c) export SAVE_OVERLAY=1 SAVE_OVERLAY_PATH=/etc;; -o) export SAVE_OVERLAY=1 SAVE_OVERLAY_PATH=/;; -p) export SAVE_PARTITIONS=0;; + -k) export SAVE_INSTALLED_PKGS=1;; -u) export SKIP_UNCHANGED=1;; -b|--create-backup) export CONF_BACKUP="$2" NEED_IMAGE=1; shift;; -r|--restore-backup) export CONF_RESTORE="$2" NEED_IMAGE=1; shift;; @@ -50,6 +52,7 @@ done export CONFFILES=/tmp/sysupgrade.conffiles export CONF_TAR=/tmp/sysupgrade.tgz +export INSTALLED_PACKAGES=/etc/sysupgrade.installed IMAGE="$1" @@ -67,6 +70,8 @@ upgrade-option: -u skip from backup files that are equals to those in /rom -n do not save configuration over reflash -p do not attempt to restore the partition table after flash. + -k include in backup a list of current installed packages at + $INSTALLED_PACKAGES -T | --test Verify image and config .tar.gz but do not actually flash. -F | --force @@ -203,6 +208,15 @@ fi include /lib/upgrade +targz_append() { + local tar="$1" + local append_tar="$2" + ( gunzip -c "$tar" | head -c -1024; + gunzip -c $append_tar + ) | gzip > $tar.new + mv -f $tar.new $tar +} + do_save_conffiles() { local conf_tar="$1" @@ -219,6 +233,22 @@ do_save_conffiles() { [ "$VERBOSE" -gt 1 ] && TAR_V="v" || TAR_V="" tar c${TAR_V}zf "$conf_tar" -T "$CONFFILES" 2>/dev/null + if [ "$SAVE_INSTALLED_PKGS" -eq 1 ]; then + mkdir -p /tmp/etc + cd /tmp/ + + # Format: pkg-name{rom,overlay,unkown} + # rom is used for pkgs in /rom, even if updated later + find /usr/lib/opkg/info -name "*.control" \( \ + \( -exec test -f /rom/{} \; -exec echo {} rom \; \) -o \ + \( -exec test -f /overlay/upper/{} \; -exec echo {} overlay \; \) -o \ + \( -exec echo {} unknown \; \) \ + \) | sed -e 's,.*/,,;s/\.control /\t/' > ${INSTALLED_PACKAGES:1} + + tar c${TAR_V}zf sysupgrade.installed.packages.tgz ${INSTALLED_PACKAGES:1} + targz_append "$conf_tar" sysupgrade.installed.packages.tgz + rm -f sysupgrade.installed.packages.tgz ${INSTALLED_PACKAGES:1} + fi rm -f "$CONFFILES" }