| Message ID | 20210624210408.19248-1-bjorn@mork.no |
|---|---|
| State | Accepted, archived |
| Delegated to: | Christian Lamparter |
| Headers | show |
| Series | realtek: fix ZyXEL initramfs image generation | expand |
ping? Installation from OEM on ZyXEL realtek devices is currently broken. And I'm getting requests like this: https://forum.openwrt.org/t/openwrt-installation-on-a-zyxel-gs1900-8/97771/39 But I have absolutely no intention of starting up my own forked distribution... How about just applying this patch? This is also required for 21.02.xx. Bjørn Bjørn Mork <bjorn@mork.no> writes: > The current rule produces empty trailers, causing the OEM firmware > update application to reject our images. > > The double expansion of a makefile variable does not work inside > shell code. The second round is interpreted as a shell expansion, > attempting to run the command ZYXEL_VERS instead of expanding the > $(ZYXEL_VERS) makefile variable. > > Fix by removing one level of variable indirection. > > Fixes: c6c8d597e183 ("realtek: Add generic zyxel_gs1900 image definition") > Signed-off-by: Bjørn Mork <bjorn@mork.no> > --- > I got myself another brand new GS1900-10HP and used the opportunity to > verify the console-less installation procedure. And of course, it didn't > work.... > > The reason is that we currently build images with a bogus trailer, > lacking the crucial hardware version info the OEM firmware looks > for. > > This needs to be backported yto 21.02 as well. I dowloaded the > initramfs images from downloads.openwrt.org and verified that they have > the same issue: > > $ hexdump -C openwrt-21.02.0-rc3-realtek-generic-zyxel_gs1900-10hp-initramfs-kernel.bin |tail -4 > 005b7240 ca 0f 86 61 cc 1b 7d 0a 0b 09 45 88 fc 06 fd ac |...a..}...E.....| > 005b7250 f6 82 7e 5d 7d 13 5a 56 8c 14 fe 7f 55 bf 19 d4 |..~]}.ZV....U...| > 005b7260 ea 2d d7 00 56 45 52 53 0a |.-..VERS.| > 005b7269 > > > A proper image should have a trailer similar to this: > > $ hexdump -C openwrt-initramfs.bin |tail -4 > 005746d0 10 e1 d2 0c 73 0a ff 07 eb 12 c6 f1 0a eb ca 00 |....s...........| > 005746e0 56 45 52 53 0a 56 39 2e 39 39 28 41 41 5a 49 2e |VERS.V9.99(AAZI.| > 005746f0 30 29 20 7c 20 30 36 2f 32 34 2f 32 30 32 31 0a |0) | 06/24/2021.| > 00574700 > > The last one is the actual image I used to my initial install from OEM. > It worked. > > > Bjørn > > target/linux/realtek/image/Makefile | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/target/linux/realtek/image/Makefile b/target/linux/realtek/image/Makefile > index ebc0c0a78480..38b4d5e441cc 100644 > --- a/target/linux/realtek/image/Makefile > +++ b/target/linux/realtek/image/Makefile > @@ -10,7 +10,7 @@ DEVICE_VARS += ZYXEL_VERS > > define Build/zyxel-vers > ( echo VERS;\ > - for hw in $(1); do\ > + for hw in $(ZYXEL_VERS); do\ > echo -n "V9.99($$hw.0) | ";\ > date -d @$(SOURCE_DATE_EPOCH) +%m/%d/%Y;\ > done ) >> $@ > @@ -117,7 +117,7 @@ define Device/zyxel_gs1900 > IMAGE_SIZE := 6976k > DEVICE_VENDOR := ZyXEL > UIMAGE_MAGIC := 0x83800000 > - KERNEL_INITRAMFS := kernel-bin | append-dtb | gzip | zyxel-vers $$$$(ZYXEL_VERS) | \ > + KERNEL_INITRAMFS := kernel-bin | append-dtb | gzip | zyxel-vers | \ > uImage gzip > endef
Hi Bjørn, On Thu, 2021-06-24 at 23:04 +0200, Bjørn Mork wrote: > The current rule produces empty trailers, causing the OEM firmware > update application to reject our images. > > The double expansion of a makefile variable does not work inside > shell code. The second round is interpreted as a shell expansion, > attempting to run the command ZYXEL_VERS instead of expanding the > $(ZYXEL_VERS) makefile variable. > > Fix by removing one level of variable indirection. > > Fixes: c6c8d597e183 ("realtek: Add generic zyxel_gs1900 image definition") > Signed-off-by: Bjørn Mork <bjorn@mork.no> > --- > I got myself another brand new GS1900-10HP and used the opportunity to > verify the console-less installation procedure. And of course, it didn't > work.... I wanted to verify that your patch fixes OpenWrt flashing, but V2.60(AAHH.4) on my GS1900-8 actually accepts images with bogus trailers. That being said, an image produced with your patch applied, thus a correct trailer, can also be flashed and the trailer is recognized by the stock firmware. So I guess I can already provide this: Tested-by: Sander Vanheule <sander@svanheule.net> I also have one small question below. > The reason is that we currently build images with a bogus trailer, > lacking the crucial hardware version info the OEM firmware looks > for. > > This needs to be backported yto 21.02 as well. I dowloaded the > initramfs images from downloads.openwrt.org and verified that they have > the same issue: > > $ hexdump -C openwrt-21.02.0-rc3-realtek-generic-zyxel_gs1900-10hp-initramfs- > kernel.bin |tail -4 > 005b7240 ca 0f 86 61 cc 1b 7d 0a 0b 09 45 88 fc 06 fd ac |...a..}...E.....| > 005b7250 f6 82 7e 5d 7d 13 5a 56 8c 14 fe 7f 55 bf 19 d4 |..~]}.ZV....U...| > 005b7260 ea 2d d7 00 56 45 52 53 0a |.-..VERS.| > 005b7269 > > > A proper image should have a trailer similar to this: > > $ hexdump -C openwrt-initramfs.bin |tail -4 > 005746d0 10 e1 d2 0c 73 0a ff 07 eb 12 c6 f1 0a eb ca 00 |....s...........| > 005746e0 56 45 52 53 0a 56 39 2e 39 39 28 41 41 5a 49 2e |VERS.V9.99(AAZI.| > 005746f0 30 29 20 7c 20 30 36 2f 32 34 2f 32 30 32 31 0a |0) | 06/24/2021.| > 00574700 > > The last one is the actual image I used to my initial install from OEM. > It worked. > > > Bjørn > > target/linux/realtek/image/Makefile | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/target/linux/realtek/image/Makefile > b/target/linux/realtek/image/Makefile > index ebc0c0a78480..38b4d5e441cc 100644 > --- a/target/linux/realtek/image/Makefile > +++ b/target/linux/realtek/image/Makefile > @@ -10,7 +10,7 @@ DEVICE_VARS += ZYXEL_VERS > > define Build/zyxel-vers > ( echo VERS;\ > - for hw in $(1); do\ > + for hw in $(ZYXEL_VERS); do\ > echo -n "V9.99($$hw.0) | ";\ > date -d @$(SOURCE_DATE_EPOCH) +%m/%d/%Y;\ > done ) >> $@ Would it be worthwile to make zyxel-vers fail if $(ZYXEL_VERS) is not defined/empty? A user could still define a wrong value of course, but at least they would know to provide something. Best, Sander > @@ -117,7 +117,7 @@ define Device/zyxel_gs1900 > IMAGE_SIZE := 6976k > DEVICE_VENDOR := ZyXEL > UIMAGE_MAGIC := 0x83800000 > - KERNEL_INITRAMFS := kernel-bin | append-dtb | gzip | zyxel-vers $$$$(ZYXEL_VERS) > | \ > + KERNEL_INITRAMFS := kernel-bin | append-dtb | gzip | zyxel-vers | \ > uImage gzip > endef >
Sander Vanheule <sander@svanheule.net> writes: > I wanted to verify that your patch fixes OpenWrt flashing, but V2.60(AAHH.4) on my > GS1900-8 actually accepts images with bogus trailers. This is so strange. I just went through my notes from the experiments on my first GS1900-10HP, and they confirmed what I suspected: I did upgrade the OEM firmware on it to V2.60(AAZI.2) before trying to load OpenWrt via the web GUI. My first attempt installing an OpenWrt image from the V2.60(AAZI.2) GUI failed with the message "Device only can support firmware from V2.10(AAZI.0) and later version" I guess ZyXEL could have intentionally relaxed this between V2.60(*.2) and V2.60(*.4), but I really don't see any good reason for them to do that... Note in particular the minimum OEM version requirement. This is hardware specific value coded into their turnkey/install/board_conf.ko module. AFAICS, this is required to prevent anyone from trying to install an older image on new hardware, since they use unified images for this family and have added some models later. Another more likely explanation could be that the OEM GUI doesn't care about the version trailer on hardware where the minimum version is V1.00(*.0). To ZyXEL, this means that any firmware is acceptable on that hardware. So it would make some sense to just allow anything regardless of trailer. This would explain why you see that on the GS1900-8 since board_conf.ko lists it as (simply using strings to "analyze" the hardware table in the binary): GS1900-8 RTL8380M V1.00(AAHH.0) V2.60(AAHH.2) (The last version number is 2.60.2 because this is a binary from that release). If this theory is correct, then any of the following hardware might work without a valid trailer: model hwver mim OEM ver GS1900-8 V1.0 V1.00(AAHH.0) GS1900-8HP V1.0 V1.00(AAHI.0) GS1900-16 V1.0 V1.00(AAHJ.0) GS1900-24E V1.0 V1.00(AAHK.0) GS1900-24 V1.0 V1.00(AAHL.0) GS1900-24HP V1.0 V1.00(AAHM.0) GS1900-48 V1.0 V1.00(AAHN.0) GS1900-48HP V1.0 V1.00(AAHO.0) But these, and any newer models, would require a matching trailer: model hwver mim OEM ver GS1900-8HP V2.0 V2.10(AAHI.0) GS1900-10HP V1.0 V2.10(AAZI.0) GS1900-24EP V1.0 V2.60(ABTO.0) GS1900-24HP V2.0 V2.60(ABTP.0) GS1900-48HP V2.0 V2.60(ABTQ.0) One very interesting candidate for testing is the GS1900-8HP, which uses the same four-letter code for both hardware revisions but with different minimum firmware versions. A final question: Did you try to install an image *without* any trailer at all? I was hoping we could use the trailer to prevent users from trying to flash a sysupgrade image from OEM. That would still work if the OEM firmware requires some trailer. > That being said, an image > produced with your patch applied, thus a correct trailer, can also be flashed and the > trailer is recognized by the stock firmware. > > So I guess I can already provide this: > > Tested-by: Sander Vanheule <sander@svanheule.net> Thanks. >> target/linux/realtek/image/Makefile | 4 ++-- >> 1 file changed, 2 insertions(+), 2 deletions(-) >> >> diff --git a/target/linux/realtek/image/Makefile >> b/target/linux/realtek/image/Makefile >> index ebc0c0a78480..38b4d5e441cc 100644 >> --- a/target/linux/realtek/image/Makefile >> +++ b/target/linux/realtek/image/Makefile >> @@ -10,7 +10,7 @@ DEVICE_VARS += ZYXEL_VERS >> >> define Build/zyxel-vers >> ( echo VERS;\ >> - for hw in $(1); do\ >> + for hw in $(ZYXEL_VERS); do\ >> echo -n "V9.99($$hw.0) | ";\ >> date -d @$(SOURCE_DATE_EPOCH) +%m/%d/%Y;\ >> done ) >> $@ > > Would it be worthwile to make zyxel-vers fail if $(ZYXEL_VERS) is not defined/empty? > A user could still define a wrong value of course, but at least they would know to > provide something. We could do that, but I don't think that sort of failsafe coding is common in our image make rules. They are not intended for end users anyway. If you add a new device, then you're supposed to know what you do. Let's see of we can get this fix in there first so that OpenWrt becomes installable again on the supposedly supported devices. Bjørn
On Sat, 2021-10-30 at 10:38 +0200, Bjørn Mork wrote: > Sander Vanheule <sander@svanheule.net> writes: > > > I wanted to verify that your patch fixes OpenWrt flashing, but V2.60(AAHH.4) on > > my > > GS1900-8 actually accepts images with bogus trailers. > > This is so strange. I just went through my notes from the experiments > on my first GS1900-10HP, and they confirmed what I suspected: I did > upgrade the OEM firmware on it to V2.60(AAZI.2) before trying to load > OpenWrt via the web GUI. > > My first attempt installing an OpenWrt image from the V2.60(AAZI.2) GUI > failed with the message > > "Device only can support firmware from V2.10(AAZI.0) and later version" > > I guess ZyXEL could have intentionally relaxed this between V2.60(*.2) > and V2.60(*.4), but I really don't see any good reason for them to do > that... > > Note in particular the minimum OEM version requirement. This is > hardware specific value coded into their turnkey/install/board_conf.ko > module. AFAICS, this is required to prevent anyone from trying to > install an older image on new hardware, since they use unified images > for this family and have added some models later. > > Another more likely explanation could be that the OEM GUI doesn't care > about the version trailer on hardware where the minimum version is > V1.00(*.0). To ZyXEL, this means that any firmware is acceptable on > that hardware. So it would make some sense to just allow anything > regardless of trailer. > > This would explain why you see that on the GS1900-8 since board_conf.ko > lists it as (simply using strings to "analyze" the hardware table in the > binary): > > GS1900-8 > RTL8380M > V1.00(AAHH.0) > V2.60(AAHH.2) > > (The last version number is 2.60.2 because this is a binary from that > release). > > If this theory is correct, then any of the following hardware might work > without a valid trailer: > > model hwver mim OEM ver > GS1900-8 V1.0 V1.00(AAHH.0) > GS1900-8HP V1.0 V1.00(AAHI.0) > GS1900-16 V1.0 V1.00(AAHJ.0) > GS1900-24E V1.0 V1.00(AAHK.0) > GS1900-24 V1.0 V1.00(AAHL.0) > GS1900-24HP V1.0 V1.00(AAHM.0) > GS1900-48 V1.0 V1.00(AAHN.0) > GS1900-48HP V1.0 V1.00(AAHO.0) > > But these, and any newer models, would require a matching trailer: > > model hwver mim OEM ver > GS1900-8HP V2.0 V2.10(AAHI.0) > GS1900-10HP V1.0 V2.10(AAZI.0) > GS1900-24EP V1.0 V2.60(ABTO.0) > GS1900-24HP V2.0 V2.60(ABTP.0) > GS1900-48HP V2.0 V2.60(ABTQ.0) > > One very interesting candidate for testing is the GS1900-8HP, which uses > the same four-letter code for both hardware revisions but with different > minimum firmware versions. > > A final question: Did you try to install an image *without* any trailer > at all? I was hoping we could use the trailer to prevent users from > trying to flash a sysupgrade image from OEM. That would still work if > the OEM firmware requires some trailer. Thanks for the in-depth analysis! I've now also tested without any trailer, and this image is also accepted in the vendor's web interface. I don't have any devices in the former list, so I'm afraid I can't check whether a trailer-less firmware is rejected on these devices. > > > That being said, an image > > produced with your patch applied, thus a correct trailer, can also be flashed and > > the > > trailer is recognized by the stock firmware. > > > > So I guess I can already provide this: > > > > Tested-by: Sander Vanheule <sander@svanheule.net> > > Thanks. > > > > target/linux/realtek/image/Makefile | 4 ++-- > > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > > > diff --git a/target/linux/realtek/image/Makefile > > > b/target/linux/realtek/image/Makefile > > > index ebc0c0a78480..38b4d5e441cc 100644 > > > --- a/target/linux/realtek/image/Makefile > > > +++ b/target/linux/realtek/image/Makefile > > > @@ -10,7 +10,7 @@ DEVICE_VARS += ZYXEL_VERS > > > > > > define Build/zyxel-vers > > > ( echo VERS;\ > > > - for hw in $(1); do\ > > > + for hw in $(ZYXEL_VERS); do\ > > > echo -n "V9.99($$hw.0) | ";\ > > > date -d @$(SOURCE_DATE_EPOCH) +%m/%d/%Y;\ > > > done ) >> $@ > > > > Would it be worthwile to make zyxel-vers fail if $(ZYXEL_VERS) is not > > defined/empty? > > A user could still define a wrong value of course, but at least they would know > > to > > provide something. > > We could do that, but I don't think that sort of failsafe coding is > common in our image make rules. They are not intended for end users > anyway. If you add a new device, then you're supposed to know what you > do. > > Let's see of we can get this fix in there first so that OpenWrt becomes > installable again on the supposedly supported devices. I agree, adding the check should be for another patch. It was more just something that crossed my mind as I was reading the patch :) Best, Sander
diff --git a/target/linux/realtek/image/Makefile b/target/linux/realtek/image/Makefile index ebc0c0a78480..38b4d5e441cc 100644 --- a/target/linux/realtek/image/Makefile +++ b/target/linux/realtek/image/Makefile @@ -10,7 +10,7 @@ DEVICE_VARS += ZYXEL_VERS define Build/zyxel-vers ( echo VERS;\ - for hw in $(1); do\ + for hw in $(ZYXEL_VERS); do\ echo -n "V9.99($$hw.0) | ";\ date -d @$(SOURCE_DATE_EPOCH) +%m/%d/%Y;\ done ) >> $@ @@ -117,7 +117,7 @@ define Device/zyxel_gs1900 IMAGE_SIZE := 6976k DEVICE_VENDOR := ZyXEL UIMAGE_MAGIC := 0x83800000 - KERNEL_INITRAMFS := kernel-bin | append-dtb | gzip | zyxel-vers $$$$(ZYXEL_VERS) | \ + KERNEL_INITRAMFS := kernel-bin | append-dtb | gzip | zyxel-vers | \ uImage gzip endef
The current rule produces empty trailers, causing the OEM firmware update application to reject our images. The double expansion of a makefile variable does not work inside shell code. The second round is interpreted as a shell expansion, attempting to run the command ZYXEL_VERS instead of expanding the $(ZYXEL_VERS) makefile variable. Fix by removing one level of variable indirection. Fixes: c6c8d597e183 ("realtek: Add generic zyxel_gs1900 image definition") Signed-off-by: Bjørn Mork <bjorn@mork.no> --- I got myself another brand new GS1900-10HP and used the opportunity to verify the console-less installation procedure. And of course, it didn't work.... The reason is that we currently build images with a bogus trailer, lacking the crucial hardware version info the OEM firmware looks for. This needs to be backported yto 21.02 as well. I dowloaded the initramfs images from downloads.openwrt.org and verified that they have the same issue: $ hexdump -C openwrt-21.02.0-rc3-realtek-generic-zyxel_gs1900-10hp-initramfs-kernel.bin |tail -4 005b7240 ca 0f 86 61 cc 1b 7d 0a 0b 09 45 88 fc 06 fd ac |...a..}...E.....| 005b7250 f6 82 7e 5d 7d 13 5a 56 8c 14 fe 7f 55 bf 19 d4 |..~]}.ZV....U...| 005b7260 ea 2d d7 00 56 45 52 53 0a |.-..VERS.| 005b7269 A proper image should have a trailer similar to this: $ hexdump -C openwrt-initramfs.bin |tail -4 005746d0 10 e1 d2 0c 73 0a ff 07 eb 12 c6 f1 0a eb ca 00 |....s...........| 005746e0 56 45 52 53 0a 56 39 2e 39 39 28 41 41 5a 49 2e |VERS.V9.99(AAZI.| 005746f0 30 29 20 7c 20 30 36 2f 32 34 2f 32 30 32 31 0a |0) | 06/24/2021.| 00574700 The last one is the actual image I used to my initial install from OEM. It worked. Bjørn target/linux/realtek/image/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)