From patchwork Sat May 13 19:56:37 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthias Schiffer X-Patchwork-Id: 762076 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3wQHkG0kgvz9sCX for ; Sun, 14 May 2017 05:59:50 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="MTZR+Xt/"; dkim-atps=neutral DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Subject:References: In-Reply-To:Message-Id:Date:To:From:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=iIxES056p6mitmn6prfVTkND+K4sWXXrat8i4QMHaz8=; b=MTZR+Xt/NdN7VMjwdrroqqms7E gwxdw+P0oBJ9kzEYvnPw+QsnO3NnKaNJLGsystUjtY3oQ8/xz+poavUET9GgN1O1i9mVULDrNhae4 P3YtNOSTlYMBHMb5WYatACKm4Grndz8FynJfovwsQ9tveMca6d+a/aRBn1k0lb2NjRSSHRphdlUKq /MYKeqwJcsYlevpulGnJSF7zo44TdaqaSrG7OcNY1FipjqSTWgP3NvtcJAVhWDZU6N4QZhAd4jhCd 5i0H8gXx+Rwlm1ZGOO6igMH8LfUrUhSaa51E+NAc/V+pn+g2hWxUlpnOTZiU0soiB2e7ZgYGboelA FnBU/T6w==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1d9dCW-0006lc-Qk; Sat, 13 May 2017 19:59:40 +0000 Received: from chaos.universe-factory.net ([37.72.148.22]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1d9dAd-0003qd-Sw for lede-dev@lists.infradead.org; Sat, 13 May 2017 19:58:03 +0000 Received: from localhost.localdomain (unknown [IPv6:fd1b:c28a:2fd6::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by chaos.universe-factory.net (Postfix) with ESMTPSA id A69D4186C96 for ; Sat, 13 May 2017 21:56:55 +0200 (CEST) From: Matthias Schiffer To: lede-dev@lists.infradead.org Date: Sat, 13 May 2017 21:56:37 +0200 Message-Id: <5c0bcf4136ac2bebb45c14050882dbf08bf8937b.1494704975.git.mschiffer@universe-factory.net> X-Mailer: git-send-email 2.13.0 In-Reply-To: References: In-Reply-To: References: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170513_125745_281415_6C6A5BAC X-CRM114-Status: GOOD ( 15.11 ) X-Spam-Score: -1.9 (-) X-Spam-Report: SpamAssassin version 3.4.1 on bombadil.infradead.org summary: Content analysis details: (-1.9 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] Subject: [LEDE-DEV] [PATCH v2 07/19] base-files: always use staged sysupgrade X-BeenThere: lede-dev@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "Lede-dev" Errors-To: lede-dev-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org Support for the -d and -p options is dropped; it may be added again at some point by adding these flags to the ubus sysupgrade call. A downside of this is that we get a lot less information about the progress of the upgrade: as soon as the actual upgrade starts, all shell sessions are killed to allow unmounting the root filesystem. Signed-off-by: Matthias Schiffer --- package/base-files/files/lib/upgrade/common.sh | 123 +++++++++++-------------- package/base-files/files/lib/upgrade/nand.sh | 60 ++---------- package/base-files/files/lib/upgrade/stage2 | 50 ++++++++++ package/base-files/files/sbin/sysupgrade | 77 +++++++++------- 4 files changed, 162 insertions(+), 148 deletions(-) create mode 100755 package/base-files/files/lib/upgrade/stage2 diff --git a/package/base-files/files/lib/upgrade/common.sh b/package/base-files/files/lib/upgrade/common.sh index e3519ca2cb..17248c2b1d 100644 --- a/package/base-files/files/lib/upgrade/common.sh +++ b/package/base-files/files/lib/upgrade/common.sh @@ -56,7 +56,6 @@ run_ramfs() { # [...] /bin/rm /usr/bin/basename /bin/kill /bin/chmod /usr/bin/find \ /bin/mknod - install_bin /bin/uclient-fetch /bin/wget install_bin /sbin/mtd install_bin /sbin/mount_root install_bin /sbin/snapshot @@ -96,51 +95,37 @@ run_ramfs() { # [...] exec /bin/busybox ash -c "$*" } -kill_remaining() { # [ ] +kill_remaining() { # [ [ ] ] local sig="${1:-TERM}" + local loop="${2:-0}" + local run=true + local stat + echo -n "Sending $sig to remaining processes ... " - local my_pid=$$ - local my_ppid=$(cut -d' ' -f4 /proc/$my_pid/stat) - local my_ppisupgraded= - grep -q upgraded /proc/$my_ppid/cmdline >/dev/null && { - local my_ppisupgraded=1 - } - - local stat - for stat in /proc/[0-9]*/stat; do - [ -f "$stat" ] || continue - - local pid name state ppid rest - read pid name state ppid rest < $stat - name="${name#(}"; name="${name%)}" - - local cmdline - read cmdline < /proc/$pid/cmdline - - # Skip kernel threads - [ -n "$cmdline" ] || continue - - if [ $$ -eq 1 ] || [ $my_ppid -eq 1 ] && [ -n "$my_ppisupgraded" ]; then - # Running as init process, kill everything except me - if [ $pid -ne $$ ] && [ $pid -ne $my_ppid ]; then - echo -n "$name " - kill -$sig $pid 2>/dev/null - fi - else - case "$name" in - # Skip essential services - *procd*|*ash*|*init*|*watchdog*|*ssh*|*dropbear*|*telnet*|*login*|*hostapd*|*wpa_supplicant*|*nas*|*relayd*) : ;; - - # Killable process - *) - if [ $pid -ne $$ ] && [ $ppid -ne $$ ]; then - echo -n "$name " - kill -$sig $pid 2>/dev/null - fi - ;; - esac - fi + while $run; do + run=false + for stat in /proc/[0-9]*/stat; do + [ -f "$stat" ] || continue + + local pid name state ppid rest + read pid name state ppid rest < $stat + name="${name#(}"; name="${name%)}" + + # Skip PID1, ourself and our children + [ $pid -ne 1 -a $pid -ne $$ -a $ppid -ne $$ ] || continue + + local cmdline + read cmdline < /proc/$pid/cmdline + + # Skip kernel threads + [ -n "$cmdline" ] || continue + + echo -n "$name " + kill -$sig $pid 2>/dev/null + + [ $loop -eq 1 ] && run=true + done done echo "" } @@ -175,28 +160,31 @@ v() { [ "$VERBOSE" -ge 1 ] && echo "$@" } +json_string() { + local v="$1" + v="${v//\\/\\\\}" + v="${v//\"/\\\"}" + echo "\"$v\"" +} + rootfs_type() { /bin/mount | awk '($3 ~ /^\/$/) && ($5 !~ /rootfs/) { print $5 }' } get_image() { # [ ] local from="$1" - local conc="$2" - local cmd - - case "$from" in - http://*|ftp://*) cmd="wget -O- -q";; - *) cmd="cat";; - esac - if [ -z "$conc" ]; then - local magic="$(eval $cmd \"$from\" 2>/dev/null | dd bs=2 count=1 2>/dev/null | hexdump -n 2 -e '1/1 "%02x"')" + local cat="$2" + + if [ -z "$cat" ]; then + local magic="$(dd if="$from" bs=2 count=1 2>/dev/null | hexdump -n 2 -e '1/1 "%02x"')" case "$magic" in - 1f8b) conc="zcat";; - 425a) conc="bzcat";; + 1f8b) cat="zcat";; + 425a) cat="bzcat";; + *) cat="cat";; esac fi - eval "$cmd \"$from\" 2>/dev/null ${conc:+| $conc}" + $cat "$from" 2>/dev/null } get_magic_word() { @@ -320,12 +308,14 @@ default_do_upgrade() { fi } -do_upgrade() { +do_upgrade_stage2() { v "Performing system upgrade..." - if type 'platform_do_upgrade' >/dev/null 2>/dev/null; then - platform_do_upgrade "$ARGV" + if [ -n "$do_upgrade" ]; then + $do_upgrade "$IMAGE" + elif type 'platform_do_upgrade' >/dev/null 2>/dev/null; then + platform_do_upgrade "$IMAGE" else - default_do_upgrade "$ARGV" + default_do_upgrade "$IMAGE" fi if [ "$SAVE_CONFIG" -eq 1 ] && type 'platform_copy_config' >/dev/null 2>/dev/null; then @@ -333,12 +323,11 @@ do_upgrade() { fi v "Upgrade completed" - [ -n "$DELAY" ] && sleep "$DELAY" - ask_bool 1 "Reboot" && { - v "Rebooting system..." - umount -a - reboot -f - sleep 5 - echo b 2>/dev/null >/proc/sysrq-trigger - } + sleep 1 + + v "Rebooting system..." + umount -a + reboot -f + sleep 5 + echo b 2>/dev/null >/proc/sysrq-trigger } diff --git a/package/base-files/files/lib/upgrade/nand.sh b/package/base-files/files/lib/upgrade/nand.sh index 9c831df3b4..894964e178 100644 --- a/package/base-files/files/lib/upgrade/nand.sh +++ b/package/base-files/files/lib/upgrade/nand.sh @@ -283,7 +283,16 @@ nand_upgrade_tar() { } # Recognize type of passed file and start the upgrade process -nand_do_upgrade_stage2() { +nand_do_upgrade() { + if [ -n "$IS_PRE_UPGRADE" ]; then + # Previously, nand_do_upgrade was called from the platform_pre_upgrade + # hook; this piece of code handles scripts that haven't been + # updated. All scripts should gradually move to call nand_do_upgrade + # from platform_do_upgrade instead. + export do_upgrade=nand_do_upgrade + return + fi + local file_type=$(identify $1) if type 'platform_nand_pre_upgrade' >/dev/null 2>/dev/null; then @@ -299,45 +308,6 @@ nand_do_upgrade_stage2() { esac } -nand_upgrade_stage2() { - [ $1 = "nand" ] && { - [ -f "$2" ] && { - touch /tmp/sysupgrade - - killall -9 telnetd - killall -9 dropbear - killall -9 ash - - kill_remaining TERM - sleep 3 - kill_remaining KILL - - sleep 1 - - if [ -n "$(rootfs_type)" ]; then - v "Switching to ramdisk..." - run_ramfs ". /lib/functions.sh; include /lib/upgrade; nand_do_upgrade_stage2 $2" - else - nand_do_upgrade_stage2 $2 - fi - return 0 - } - echo "Nand upgrade failed" - exit 1 - } -} - -nand_upgrade_stage1() { - [ -f /tmp/sysupgrade-nand-path ] && { - path="$(cat /tmp/sysupgrade-nand-path)" - [ "$SAVE_CONFIG" != 1 -a -f "$CONF_TAR" ] && - rm $CONF_TAR - - ubus call system nandupgrade "{\"prefix\": \"$RAM_ROOT\", \"path\": \"$path\" }" - exit 0 - } -} - # Check if passed file is a valid one for NAND sysupgrade. Currently it accepts # 3 types of files: # 1) UBI - should contain an ubinized image, header is checked for the proper @@ -364,13 +334,3 @@ nand_do_platform_check() { return 0 } - -# Start NAND upgrade process -# -# $(1): file to be used for upgrade -nand_do_upgrade() { - echo -n $1 > /tmp/sysupgrade-nand-path - install_bin /sbin/upgraded - ln -s "$RAM_ROOT"/sbin/upgraded /tmp/upgraded - nand_upgrade_stage1 -} diff --git a/package/base-files/files/lib/upgrade/stage2 b/package/base-files/files/lib/upgrade/stage2 new file mode 100755 index 0000000000..4e2aa3a23c --- /dev/null +++ b/package/base-files/files/lib/upgrade/stage2 @@ -0,0 +1,50 @@ +#!/bin/sh + +. /lib/functions.sh +. /lib/functions/system.sh + +export IMAGE="$1" +COMMAND="$2" + +export ARGV="$IMAGE" +export ARGC=1 + +export SAVE_CONFIG=1 +export SAVE_PARTITIONS=1 + +export INTERACTIVE=0 +export VERBOSE=1 +export CONFFILES=/tmp/sysupgrade.conffiles +export CONF_TAR=/tmp/sysupgrade.tgz + + +[ -f "$CONF_TAR" ] || export SAVE_CONFIG=0 +[ -f /tmp/sysupgrade.always.overwrite.bootdisk.partmap ] && export SAVE_PARTITIONS=0 + +include /lib/upgrade + + +killall -9 telnetd +killall -9 dropbear +killall -9 ash + +kill_remaining TERM +sleep 3 +kill_remaining KILL 1 + +sleep 1 + + +if [ -n "$IMAGE" ] && type 'platform_pre_upgrade' >/dev/null 2>/dev/null; then + IS_PRE_UPGRADE=1 platform_pre_upgrade "$IMAGE" + + # Needs to be unset again because of busybox weirdness ... + IS_PRE_UPGRADE= +fi + +if [ -n "$(rootfs_type)" ]; then + echo "Switching to ramdisk..." + run_ramfs "$COMMAND" +else + exec /bin/busybox ash -c "$COMMAND" +fi diff --git a/package/base-files/files/sbin/sysupgrade b/package/base-files/files/sbin/sysupgrade index c095ca81c5..2d67371ef7 100755 --- a/package/base-files/files/sbin/sysupgrade +++ b/package/base-files/files/sbin/sysupgrade @@ -1,4 +1,7 @@ #!/bin/sh + +[ "$1" = "nand" ] && exec /lib/upgrade/stage2 "$2" "$3" + . /lib/functions.sh . /lib/functions/system.sh @@ -11,7 +14,6 @@ export VERBOSE=1 export SAVE_CONFIG=1 export SAVE_OVERLAY=0 export SAVE_PARTITIONS=1 -export DELAY= export CONF_IMAGE= export CONF_BACKUP_LIST=0 export CONF_BACKUP= @@ -25,7 +27,6 @@ export TEST=0 while [ -n "$1" ]; do case "$1" in -i) export INTERACTIVE=1;; - -d) export DELAY="$2"; shift;; -v) export VERBOSE="$(($VERBOSE + 1))";; -q) export VERBOSE="$(($VERBOSE - 1))";; -n) export SAVE_CONFIG=0;; @@ -50,10 +51,9 @@ done export CONFFILES=/tmp/sysupgrade.conffiles export CONF_TAR=/tmp/sysupgrade.tgz -export ARGV="$*" -export ARGC="$#" +IMAGE="$1" -[ -z "$ARGV" -a -z "$NEED_IMAGE" -o $HELP -gt 0 ] && { +[ -z "$IMAGE" -a -z "$NEED_IMAGE" -o $HELP -gt 0 ] && { cat <...] $0 [-q] [-i] @@ -90,7 +90,7 @@ EOF exit 1 } -[ -n "$ARGV" -a -n "$NEED_IMAGE" ] && { +[ -n "$IMAGE" -a -n "$NEED_IMAGE" ] && { cat <<-EOF -b|--create-backup and -r|--restore-backup do not perform a firmware upgrade. Do not specify both -b|-r and a firmware image. @@ -136,14 +136,13 @@ sysupgrade_pre_upgrade="fwtool_pre_upgrade" include /lib/upgrade -[ "$1" = "nand" ] && nand_upgrade_stage2 $@ - do_save_conffiles() { local conf_tar="${1:-$CONF_TAR}" [ -z "$(rootfs_type)" ] && { echo "Cannot save config while running from ramdisk." ask_bool 0 "Abort" && exit + rm -f "$conf_tar" return 0 } run_hooks "$CONFFILES" $sysupgrade_init_conffiles @@ -184,8 +183,33 @@ type platform_check_image >/dev/null 2>/dev/null || { exit 1 } +case "$IMAGE" in + http://*) + wget -O/tmp/sysupgrade.img "$IMAGE" + IMAGE=/tmp/sysupgrade.img + ;; +esac + +IMAGE="$(readlink -f "$IMAGE")" + +case "$IMAGE" in + '') + echo "Image file not found." + exit 1 + ;; + /tmp/*) ;; + *) + v "Image not in /tmp, copying..." + cp -f "$IMAGE" /tmp/sysupgrade.img + IMAGE=/tmp/sysupgrade.img + ;; +esac + +export ARGV="$IMAGE" +export ARGC=1 + for check in $sysupgrade_image_check; do - ( eval "$check \"\$ARGV\"" ) || { + ( $check "$IMAGE" ) || { if [ $FORCE -eq 1 ]; then echo "Image check '$check' failed but --force given - will update anyway!" break @@ -211,6 +235,7 @@ elif ask_bool $SAVE_CONFIG "Keep config files over reflash"; then [ $TEST -eq 1 ] || do_save_conffiles export SAVE_CONFIG=1 else + [ $TEST -eq 1 ] || rm -f "$CONF_TAR" export SAVE_CONFIG=0 fi @@ -218,28 +243,18 @@ if [ $TEST -eq 1 ]; then exit 0 fi -run_hooks "" $sysupgrade_pre_upgrade - -# Some platforms/devices may want different sysupgrade process, e.g. without -# killing processes yet or calling ubus system upgrade method. -# This is needed e.g. on NAND devices where we just want to trigger stage1 at -# this point. -if type 'platform_pre_upgrade' >/dev/null 2>/dev/null; then - platform_pre_upgrade "$ARGV" +if [ $SAVE_PARTITIONS -eq 0 ]; then + touch /tmp/sysupgrade.always.overwrite.bootdisk.partmap +else + rm -f /tmp/sysupgrade.always.overwrite.bootdisk.partmap fi -ubus call system upgrade -touch /tmp/sysupgrade - -if [ ! -f /tmp/failsafe ] ; then - kill_remaining TERM - sleep 3 - kill_remaining KILL -fi +run_hooks "" $sysupgrade_pre_upgrade -if [ -n "$(rootfs_type)" ]; then - v "Switching to ramdisk..." - run_ramfs '. /lib/functions.sh; include /lib/upgrade; do_upgrade' -else - do_upgrade -fi +install_bin /sbin/upgraded +v "Commencing upgrade. All shell sessions will be closed now." +ubus call system sysupgrade "{ + \"prefix\": \"$RAM_ROOT\", + \"path\": $(json_string "$IMAGE"), + \"command\": \". /lib/functions.sh; include /lib/upgrade; do_upgrade_stage2\" +}"