| Message ID | 20200605114736.63933-1-freifunk@adrianschmutzler.de |
|---|---|
| State | RFC |
| Headers | show |
| Series | [OpenWrt-Devel,RFC] sysupgrade: introduce compatibility version for devices | expand |
I haven't even bother to try to write any code to see if this is feasible, but anyway... I wonder if there might be more flexible and user-friendly ways to handle upgrade incompatibilities if we are allowed to use code/metadata from the new image in the sysupgrade process? Instead of just providing a version number with some simple semantics like you describe, the new image could provide a script snippet or similar which codifies a more precise description of the incompatibility. And even a solution, if there is one. For the DSA example, such a script could (optionally?) move an incompatible config/network out of the way, while leaving all other settings untouched. Preserving e.g. wireless config has a lot of value. With the possibily to cancel the upgrade, it would even attempt an incomplete conversion. IMHO, the main reason we can't do automatic DSA conversion is because there will be failures. But this isn't critical if it is detected prior to upgrading. Just a few random thoughts. Since you are about to fundamentally improve sysupgrade anyway ;-) Bjørn
On 05/06/2020 09:27, Bjørn Mork wrote: > I wonder if there might be more flexible and user-friendly ways to > handle upgrade incompatibilities if we are allowed to use code/metadata > from the new image in the sysupgrade process? Instead of just providing > a version number with some simple semantics like you describe, the new > image could provide a script snippet or similar which codifies a more > precise description of the incompatibility. And even a solution, if > there is one. A message (or URL?) might be nice, yes. That's not something we have right now... > For the DSA example, such a script could (optionally?) move an > incompatible config/network out of the way, while leaving all other That's typically a job offloaded to /etc/uci-defaults/* from the new image, isn't it? There's a lot of ar71xx -> ath79 handling done that way already... Downgrades are, of course, unsupported. They could be, but it would waste precious flash, it makes more sense to warn users to backup beforehand in the LuCI interface (and not mess with sysupgrade itself, which needs to be able to **safely** work unattended as well).
On 5 June 2020 14:27:24 CEST, "Bjørn Mork" <bjorn@mork.no> wrote: >I haven't even bother to try to write any code to see if this is >feasible, but anyway... > >I wonder if there might be more flexible and user-friendly ways to >handle upgrade incompatibilities if we are allowed to use code/metadata I see the appeal of this idea, but executing _code_ from an image before it's properly verified sounds like a security issue to me. >from the new image in the sysupgrade process? Instead of just >providing >a version number with some simple semantics like you describe, the new >image could provide a script snippet or similar which codifies a more >precise description of the incompatibility. And even a solution, if >there is one. Sounds a little too much for my taste at first look. > >For the DSA example, such a script could (optionally?) move an >incompatible config/network out of the way, while leaving all other >settings untouched. Preserving e.g. wireless config has a lot of >value. We don't even have a general conversion script for DSA, and I don't think there will be much desire to provide and maintain individual solutions. Partial config preservation will be even more work, particularly since packages will add them in an unforeseeable way. So, providing a warning seems achievable to me, and everything beyond could be provided via other resources (e.g. in the Wiki), as the educated (and warned) user could always force-flash and copy an external script just aiming at e.g. DSA migration... But thanks for your input, I'm actually still widening my view on this. Best Adrian >With the possibily to cancel the upgrade, it would even attempt an >incomplete conversion. IMHO, the main reason we can't do automatic DSA >conversion is because there will be failures. But this isn't critical >if it is detected prior to upgrading. > >Just a few random thoughts. Since you are about to fundamentally >improve sysupgrade anyway ;-) > > >Bjørn > >_______________________________________________ >openwrt-devel mailing list >openwrt-devel@lists.openwrt.org >https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Hi, > -----Original Message----- > From: openwrt-devel [mailto:openwrt-devel-bounces@lists.openwrt.org] > On Behalf Of Henrique de Moraes Holschuh > Sent: Freitag, 5. Juni 2020 15:50 > To: openwrt-devel@lists.openwrt.org > Subject: Re: [OpenWrt-Devel] [RFC PATCH] sysupgrade: introduce > compatibility version for devices > > On 05/06/2020 09:27, Bjørn Mork wrote: > > I wonder if there might be more flexible and user-friendly ways to > > handle upgrade incompatibilities if we are allowed to use > > code/metadata from the new image in the sysupgrade process? Instead > > of just providing a version number with some simple semantics like you > > describe, the new image could provide a script snippet or similar > > which codifies a more precise description of the incompatibility. And > > even a solution, if there is one. > > A message (or URL?) might be nice, yes. That's not something we have right > now... > > > For the DSA example, such a script could (optionally?) move an > > incompatible config/network out of the way, while leaving all other > > That's typically a job offloaded to /etc/uci-defaults/* from the new > image, isn't it? There's a lot of ar71xx -> ath79 handling done that > way already... Writing an uci-defaults script is not the problem, but how to get the data from the Make variable. Of course it would be easy to just have a uci-defaults with the deviating versions, but I try to not define the same value twice, as we will need it in image/Makefile for the SUPPORTED_DEVICES. > > Downgrades are, of course, unsupported. They could be, but it would waste Downgrade will follow the same rules as upgrades, I don't think they will need special consideration at all here. Best Adrian > precious flash, it makes more sense to warn users to backup beforehand in > the LuCI interface (and not mess with sysupgrade itself, which needs to be > able to **safely** work unattended as well). > > -- > Henrique de Moraes Holschuh > > _______________________________________________ > openwrt-devel mailing list > openwrt-devel@lists.openwrt.org > https://lists.openwrt.org/mailman/listinfo/openwrt-devel
diff --git a/include/image-commands.mk b/include/image-commands.mk index e7db7128b4..53ee24aa36 100644 --- a/include/image-commands.mk +++ b/include/image-commands.mk @@ -390,7 +390,8 @@ metadata_devices=$(if $(1),$(subst "$(space)","$(comma)",$(strip $(foreach v,$(1 metadata_json = \ '{ $(if $(IMAGE_METADATA),$(IMAGE_METADATA)$(comma)) \ "metadata_version": "1.0", \ - "supported_devices":[$(call metadata_devices,$(1))], \ + "compat_version": "$(call json_quote,$(if $(DEVICE_COMPAT_VERSION),$(DEVICE_COMPAT_VERSION),1.0))", \ + "supported_devices":[$(call metadata_devices,$(SUPPORTED_DEVICES))], \ "version": { \ "dist": "$(call json_quote,$(VERSION_DIST))", \ "version": "$(call json_quote,$(VERSION_NUMBER))", \ @@ -401,7 +402,7 @@ metadata_json = \ }' define Build/append-metadata - $(if $(SUPPORTED_DEVICES),-echo $(call metadata_json,$(SUPPORTED_DEVICES)) | fwtool -I - $@) + $(if $(SUPPORTED_DEVICES),-echo $(call metadata_json) | fwtool -I - $@) [ ! -s "$(BUILD_KEY)" -o ! -s "$(BUILD_KEY).ucert" -o ! -s "$@" ] || { \ cp "$(BUILD_KEY).ucert" "$@.ucert" ;\ usign -S -m "$@" -s "$(BUILD_KEY)" -x "$@.sig" ;\ diff --git a/include/image.mk b/include/image.mk index 984b64fb9c..346ab0d5d3 100644 --- a/include/image.mk +++ b/include/image.mk @@ -419,6 +419,7 @@ define Device/Init BOARD_NAME := UIMAGE_NAME := + DEVICE_COMPAT_VERSION := 1.0 SUPPORTED_DEVICES := IMAGE_METADATA := @@ -435,7 +436,7 @@ DEFAULT_DEVICE_VARS := \ VID_HDR_OFFSET UBINIZE_OPTS UBINIZE_PARTS MKUBIFS_OPTS DEVICE_DTS \ DEVICE_DTS_CONFIG DEVICE_DTS_DIR SOC BOARD_NAME UIMAGE_NAME SUPPORTED_DEVICES \ IMAGE_METADATA KERNEL_ENTRY KERNEL_LOADADDR UBOOT_PATH IMAGE_SIZE \ - DEVICE_VENDOR DEVICE_MODEL DEVICE_VARIANT \ + DEVICE_COMPAT_VERSION DEVICE_VENDOR DEVICE_MODEL DEVICE_VARIANT \ DEVICE_ALT0_VENDOR DEVICE_ALT0_MODEL DEVICE_ALT0_VARIANT \ DEVICE_ALT1_VENDOR DEVICE_ALT1_MODEL DEVICE_ALT1_VARIANT \ DEVICE_ALT2_VENDOR DEVICE_ALT2_MODEL DEVICE_ALT2_VARIANT diff --git a/package/base-files/files/lib/upgrade/fwtool.sh b/package/base-files/files/lib/upgrade/fwtool.sh index a0b3fb0a04..b887408f7e 100644 --- a/package/base-files/files/lib/upgrade/fwtool.sh +++ b/package/base-files/files/lib/upgrade/fwtool.sh @@ -44,6 +44,23 @@ fwtool_check_image() { } device="$(cat /tmp/sysinfo/board_name)" + compat="$(cat somewherelocal)" + [ -n "$compat" ] || compat="1.0" + + json_get_var imagecompat compat_version + [ -n "$imagecompat" ] || imagecompat="1.0" + + # major compat version -> no sysupgrade + if [ "${compat%.*}" != "${imagecompat%.*}" ]; then + echo "This image is incompatible for sysupgrading based on the image version." + return 1 + fi + + # minor compat version -> sysupgrade with -n required + if [ "${compat#.*}" != "${imagecompat#.*}" ] && [ "$SAVE_CONFIG" = "1" ]; then + echo "This config is incompatible to the new image. Please upgrade with -n" + return 1 + fi json_select supported_devices || return 1
We regularly encounter the situation that devices are subject to changes that will make them incompatible to previous versions. Removing SUPPORTED_DEVICES will not really be helpful in most of these cases, as this only helps after a rename. To solve this situation, this patch introduces a compatibility version for devices. It will be implemented via a per-device Make variable DEVICE_COMPAT_VERSION, which will be set to 1.0 globally by default and then can be overwritten as needed. The value for "old" devices/images without it is assumed to be 1.0 as well. If an incompatible change is introduced, one can now increase either the minor version (1.0->1.1) or the major version (1.0->2.0). The former will still allow sysupgrade and only require to have -n set to wipe existing config. The latter is meant for cases where sysupgrade will break entirely, and a new installation should be performed. Both can still be forced as usual (-F). Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de> --- ** HELP NEEDED ** This patch is completely untested and won't work right now due to the following problem: The major obstacle (not implemented at all) currently is how to get the compat version (on)to the running device. Including it into image metadata is easy, but how to properly transform the Make variable into a variable on the device? (Of course, I can just create a separate file and write to it, but maybe there are better ways I don't see right now.) Of course, the current draft will only work after the second upgrade, i.e. when upgrading from the new firmware to an even newer one. I'm currently trying to work around that and provide compatibility with older images as well, e.g. by renaming the supported_devices in metadata based on the compat_version. --- include/image-commands.mk | 5 +++-- include/image.mk | 3 ++- package/base-files/files/lib/upgrade/fwtool.sh | 17 +++++++++++++++++ 3 files changed, 22 insertions(+), 3 deletions(-)