| Message ID | 20180926223806.12704-1-nunes.erico@gmail.com |
|---|---|
| State | Accepted |
| Commit | e14102c6f2f6eb31844ebbe844fb64baa3a85c11 |
| Headers | show |
| Series | [1/1] linux-firmware: bump version and fix hash | expand |
>>>>> "Erico" == Erico Nunes <nunes.erico@gmail.com> writes: > Bump the package to the most up to date version and fix the sha256 hash. > linux-firmware was failing due to an incorrect sha256 hash, as follows: > Fetching all references > warning: redirecting to https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/ > remote: Counting objects: 6972, done. > remote: Total 6972 (delta 0), reused 0 (delta 0) > Receiving objects: 100% (6972/6972), 196.63 MiB | 4.22 MiB/s, done. > Resolving deltas: 100% (4516/4516), done. > From http://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware > * [new branch] master -> origin/master > warning: redirecting to https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/ > warning: refname '8d69bab7a3da1913113ea98cefb73d5fa6988286' is ambiguous. > Git normally never creates a ref that ends with 40 hex characters > because it will be ignored when you just specify 40-hex. These refs > may be created by mistake. For example, > git checkout -b $br $(git rev-parse ...) > where "$br" is somehow empty and a 40-hex ref is created. Please > examine these refs and maybe delete them. Turn this message off by > running "git config advice.objectNameWarning false" > ERROR: linux-firmware-8d69bab7a3da1913113ea98cefb73d5fa6988286.tar.gz has wrong sha256 hash: > ERROR: expected: 905be20e4e2d7628dea4e2e99195520fc0cce8b247faabdc52fc44a3ff2ceb04 > ERROR: got : b9fce72a7b0b55eb311701dfd47914bc9e037134fa401d33e6e73ab9ebc9d116 > ERROR: Incomplete download, or man-in-the-middle (MITM) attack Hmm, odd? > -LINUX_FIRMWARE_VERSION = 8d69bab7a3da1913113ea98cefb73d5fa6988286 > +LINUX_FIRMWARE_VERSION = 44d4fca9922a252a0bd81f6307bcc072a78da54a Did you verify if we need to adjust any of sub options logic? There are quite some changes, but from a quick look I didn't see any new files that need to be explicitly handled. Committed, thanks.
On Thu, Sep 27, 2018 at 8:50 PM Peter Korsgaard <peter@korsgaard.com> wrote: > > ERROR: linux-firmware-8d69bab7a3da1913113ea98cefb73d5fa6988286.tar.gz has wrong sha256 hash: > > ERROR: expected: 905be20e4e2d7628dea4e2e99195520fc0cce8b247faabdc52fc44a3ff2ceb04 > > ERROR: got : b9fce72a7b0b55eb311701dfd47914bc9e037134fa401d33e6e73ab9ebc9d116 > > ERROR: Incomplete download, or man-in-the-middle (MITM) attack > > Hmm, odd? > > > -LINUX_FIRMWARE_VERSION = 8d69bab7a3da1913113ea98cefb73d5fa6988286 > > +LINUX_FIRMWARE_VERSION = 44d4fca9922a252a0bd81f6307bcc072a78da54a > > Did you verify if we need to adjust any of sub options logic? There are > quite some changes, but from a quick look I didn't see any new files > that need to be explicitly handled. I see that the hash got updated again after this commit, and now it's broken again for me. I re-tested on more machines running Fedora 28 or Centos 7, including one where I had never used Buildroot before, and apparently I even get different hashes on each for linux-firmware: ERROR: linux-firmware-44d4fca9922a252a0bd81f6307bcc072a78da54a.tar.gz has wrong sha256 hash: ERROR: expected: b279ca4d086887c2efab13e28a7ca36e409410d3df38a62d7c7b5799ee3de916 ERROR: got : 12d025328deab2bd2bec489c5f51181db6530cc9eb91d10ef66a55c18c2da8bf ERROR: linux-firmware-44d4fca9922a252a0bd81f6307bcc072a78da54a.tar.gz has wrong sha256 hash: ERROR: expected: b279ca4d086887c2efab13e28a7ca36e409410d3df38a62d7c7b5799ee3de916 ERROR: got : a20e4d92274bceb400f85cae873f05fa24575f69bcee5741c56affdb426b017a ERROR: linux-firmware-44d4fca9922a252a0bd81f6307bcc072a78da54a.tar.gz has wrong sha256 hash: ERROR: expected: b279ca4d086887c2efab13e28a7ca36e409410d3df38a62d7c7b5799ee3de916 ERROR: got : f8047ad77dd71fe05569bdaf29cb9ea7b23ce216f0d8df21b364ba68398de29e Is there some known cause for this? Erico
Erico, All, On 2018-10-03 21:11 +0200, Erico Nunes spake thusly: > On Thu, Sep 27, 2018 at 8:50 PM Peter Korsgaard <peter@korsgaard.com> wrote: > > > ERROR: linux-firmware-8d69bab7a3da1913113ea98cefb73d5fa6988286.tar.gz has wrong sha256 hash: > > > ERROR: expected: 905be20e4e2d7628dea4e2e99195520fc0cce8b247faabdc52fc44a3ff2ceb04 > > > ERROR: got : b9fce72a7b0b55eb311701dfd47914bc9e037134fa401d33e6e73ab9ebc9d116 > > > ERROR: Incomplete download, or man-in-the-middle (MITM) attack > > Hmm, odd? [--SNIP--] > I see that the hash got updated again after this commit, and now it's > broken again for me. > I re-tested on more machines running Fedora 28 or Centos 7, including > one where I had never used Buildroot before, and apparently I even get > different hashes on each for linux-firmware: > > ERROR: linux-firmware-44d4fca9922a252a0bd81f6307bcc072a78da54a.tar.gz has wrong sha256 hash: > ERROR: expected: b279ca4d086887c2efab13e28a7ca36e409410d3df38a62d7c7b5799ee3de916 > ERROR: got : 12d025328deab2bd2bec489c5f51181db6530cc9eb91d10ef66a55c18c2da8bf [--SNIP--] Weird; I do get the correct hash: linux-firmware-44d4fca9922a252a0bd81f6307bcc072a78da54a.tar.gz: OK (sha256: b279ca4d086887c2efab13e28a7ca36e409410d3df38a62d7c7b5799ee3de916) > Is there some known cause for this? In the past, this could have been caused by a too-olld or too-recent tar version. But nowadays, we do build our own tar when we need it. Can you check what tar Fedora 28 has, and check whether Buildroot built host-tar or not? Regards, Yann E. MORIN.
Am Wed, 03 Oct 2018 21:40:23 +0200 schrieb Yann E. MORIN: > Weird; I do get the correct hash: > linux-firmware-44d4fca9922a252a0bd81f6307bcc072a78da54a.tar.gz: OK > (sha256: > b279ca4d086887c2efab13e28a7ca36e409410d3df38a62d7c7b5799ee3de916) Hi, same here on Debian buster: linux-firmware-44d4fca9922a252a0bd81f6307bcc072a78da54a.tar.gz: OK (sha256: b279ca4d086887c2efab13e28a7ca36e409410d3df38a62d7c7b5799ee3de916) with a private tar 1.29 binary to avoid building host-tar $ which tar /home/bernd/bin/tar $ /home/bernd/bin/tar --version tar (GNU tar) 1.29 Regards, Bernd
On Wed, Oct 3, 2018 at 9:40 PM Yann E. MORIN <yann.morin.1998@free.fr> wrote: > > Erico, All, > > On 2018-10-03 21:11 +0200, Erico Nunes spake thusly: > > On Thu, Sep 27, 2018 at 8:50 PM Peter Korsgaard <peter@korsgaard.com> wrote: > > > > ERROR: linux-firmware-8d69bab7a3da1913113ea98cefb73d5fa6988286.tar.gz has wrong sha256 hash: > > > > ERROR: expected: 905be20e4e2d7628dea4e2e99195520fc0cce8b247faabdc52fc44a3ff2ceb04 > > > > ERROR: got : b9fce72a7b0b55eb311701dfd47914bc9e037134fa401d33e6e73ab9ebc9d116 > > > > ERROR: Incomplete download, or man-in-the-middle (MITM) attack > > > Hmm, odd? > [--SNIP--] > > I see that the hash got updated again after this commit, and now it's > > broken again for me. > > I re-tested on more machines running Fedora 28 or Centos 7, including > > one where I had never used Buildroot before, and apparently I even get > > different hashes on each for linux-firmware: > > > > ERROR: linux-firmware-44d4fca9922a252a0bd81f6307bcc072a78da54a.tar.gz has wrong sha256 hash: > > ERROR: expected: b279ca4d086887c2efab13e28a7ca36e409410d3df38a62d7c7b5799ee3de916 > > ERROR: got : 12d025328deab2bd2bec489c5f51181db6530cc9eb91d10ef66a55c18c2da8bf > [--SNIP--] > > Weird; I do get the correct hash: > linux-firmware-44d4fca9922a252a0bd81f6307bcc072a78da54a.tar.gz: OK (sha256: b279ca4d086887c2efab13e28a7ca36e409410d3df38a62d7c7b5799ee3de916) > > > Is there some known cause for this? > > In the past, this could have been caused by a too-olld or too-recent tar > version. But nowadays, we do build our own tar when we need it. Can you > check what tar Fedora 28 has, and check whether Buildroot built host-tar > or not? Ok so I finally tested this out and it seems to be due to two reasons: 1) The tar version in Fedora is 1.30, Buildroot host-tar is 1.29. I can reproduce it using 1.30, if I force a host-tar build before getting the source, it works. 2) This is a bit unsupported and I had already tried disabling it before reporting, but I was experimenting with system-side pigz (something like https://askubuntu.com/a/62608), this seems to affect it too. Ignoring (2) for now, tar 1.30 seems to be an issue. Locally bumping the Buildroot host-tar to 1.30 reproduces the issue to me. Also to confirm, downgrading my Fedora tar version to 1.29 no longer reproduces the issue. I see that support/dependencies/check-host-tar.sh has some handling for tar 1.30 already, but it doesn't seem to be triggering for me. Maybe it is because I switched to testing with 'make source' rather than a full build to test it quickly? Erico
Erico, All, Adding Thomas in Cc, for he may need to act on existing patches ;-) On 2018-10-28 08:59 +0100, Erico Nunes spake thusly: [--SNIP--] > Ok so I finally tested this out and it seems to be due to two reasons: > > 1) The tar version in Fedora is 1.30, Buildroot host-tar is 1.29. I > can reproduce it using 1.30, if I force a host-tar build before > getting the source, it works. I was pretty sure we had added support for download dependencies recently, and added host-tar as needed, but it seems this series of mine is still pending: https://patchwork.ozlabs.org/project/buildroot/list/?series=43275 > 2) This is a bit unsupported and I had already tried disabling it > before reporting, but I was experimenting with system-side pigz > (something like https://askubuntu.com/a/62608), this seems to affect > it too. So, we should now detect if gzip is acceptable for use by Buildroot, and if it is not, build our own host-gzip. Does pigz isdentifies itself as pigz, or does it really impersonate gzip? If the former, then it will be easy to detect, but if the latter, it becomes more complex (we'd have to compress a wellknown blob and see if the compressed stream matches what we expect.) > Maybe it is because I switched to testing with 'make source' rather > than a full build to test it quickly? Yes, see the series I referenced above, as well as its cover letter: http://lists.busybox.net/pipermail/buildroot/2018-May/221176.html Regards, Yann E. MORIN.
Erico, All, On 2018-11-01 15:14 +0100, Yann E. MORIN spake thusly: > On 2018-10-28 08:59 +0100, Erico Nunes spake thusly: > [--SNIP--] > > Ok so I finally tested this out and it seems to be due to two reasons: > > > > 1) The tar version in Fedora is 1.30, Buildroot host-tar is 1.29. I > > can reproduce it using 1.30, if I force a host-tar build before > > getting the source, it works. > > I was pretty sure we had added support for download dependencies > recently, and added host-tar as needed, but it seems this series > of mine is still pending: > > https://patchwork.ozlabs.org/project/buildroot/list/?series=43275 The part of that series that was dealing with download dependencies has now been applied, so you should now be able to use 'make clean; make source' and get the expected host-tar when needed. Can you confirm that on your side with your fancy Fedora 28+ ? ;-) Regards, Yann E. MORIN. > > 2) This is a bit unsupported and I had already tried disabling it > > before reporting, but I was experimenting with system-side pigz > > (something like https://askubuntu.com/a/62608), this seems to affect > > it too. > > So, we should now detect if gzip is acceptable for use by Buildroot, and > if it is not, build our own host-gzip. > > Does pigz isdentifies itself as pigz, or does it really impersonate > gzip? If the former, then it will be easy to detect, but if the latter, > it becomes more complex (we'd have to compress a wellknown blob and see > if the compressed stream matches what we expect.) > > > Maybe it is because I switched to testing with 'make source' rather > > than a full build to test it quickly? > > Yes, see the series I referenced above, as well as its cover letter: > http://lists.busybox.net/pipermail/buildroot/2018-May/221176.html > > Regards, > Yann E. MORIN. > > -- > .-----------------.--------------------.------------------.--------------------. > | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | > | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | > | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | > | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | > '------------------------------^-------^------------------^--------------------' > _______________________________________________ > buildroot mailing list > buildroot@busybox.net > http://lists.busybox.net/mailman/listinfo/buildroot
On Thu, Nov 1, 2018 at 10:06 PM Yann E. MORIN <yann.morin.1998@free.fr> wrote: > > https://patchwork.ozlabs.org/project/buildroot/list/?series=43275 > The part of that series that was dealing with download dependencies has > now been applied, so you should now be able to use 'make clean; make > source' and get the expected host-tar when needed. > > Can you confirm that on your side with your fancy Fedora 28+ ? ;-) Looks like you predicted this issue some time ago :) And the new applied patches do fix it, a host-tar build gets triggered even on 'make source' now and therefore the problem is gone. > > Does pigz isdentifies itself as pigz, or does it really impersonate > > gzip? If the former, then it will be easy to detect, but if the latter, > > it becomes more complex (we'd have to compress a wellknown blob and see > > if the compressed stream matches what we expect.) So with the symlink setup, all of the gzip related commands point to /usr/bin/pigz, which detects the called command name and then act accordingly. With the parallelism I was able to get significant performance improvements on e.g. compressing large Buildroot built images with this setup. Too bad it seems to affect the reproducibility right now. Looks like it is still detectable: $ gzip --version pigz 2.4 I think it is a much minor issue though, more likely my fault than an issue with Buildroot. Would be better to be able to keep some of the parallelism benefits while maintaining reproducibility than forcing a host-gzip. I guess I'll just try some flags with it or maybe file an issue on the project to figure that out better. Thanks Yann for the support and the fix. Erico
diff --git a/package/linux-firmware/linux-firmware.hash b/package/linux-firmware/linux-firmware.hash index b1f46b97e9..d8fdf77016 100644 --- a/package/linux-firmware/linux-firmware.hash +++ b/package/linux-firmware/linux-firmware.hash @@ -1,5 +1,5 @@ # Locally calculated -sha256 905be20e4e2d7628dea4e2e99195520fc0cce8b247faabdc52fc44a3ff2ceb04 linux-firmware-8d69bab7a3da1913113ea98cefb73d5fa6988286.tar.gz +sha256 8f419b5d0c65d0f861a2cfc634f30e7952135de71a3ebe6b7afbda091f5a94ba linux-firmware-44d4fca9922a252a0bd81f6307bcc072a78da54a.tar.gz sha256 8116433f4004fc0c24d72b3d9e497808b724aa0e5e1cd63fc1bf66b715b1e2e9 LICENCE.Abilis sha256 4b3ea5d5a03c0db81bee0bcb14b30d75b30ef568597bb5be7d4dee57f434265f LICENSE.amdgpu sha256 38f2037aa14631b4b29826d7a99379613c41a97064d1defdee30a7a022138b20 LICENCE.Marvell diff --git a/package/linux-firmware/linux-firmware.mk b/package/linux-firmware/linux-firmware.mk index c9866fd1bb..4eba57434b 100644 --- a/package/linux-firmware/linux-firmware.mk +++ b/package/linux-firmware/linux-firmware.mk @@ -4,7 +4,7 @@ # ################################################################################ -LINUX_FIRMWARE_VERSION = 8d69bab7a3da1913113ea98cefb73d5fa6988286 +LINUX_FIRMWARE_VERSION = 44d4fca9922a252a0bd81f6307bcc072a78da54a LINUX_FIRMWARE_SITE = http://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git LINUX_FIRMWARE_SITE_METHOD = git
Bump the package to the most up to date version and fix the sha256 hash. linux-firmware was failing due to an incorrect sha256 hash, as follows: Fetching all references warning: redirecting to https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/ remote: Counting objects: 6972, done. remote: Total 6972 (delta 0), reused 0 (delta 0) Receiving objects: 100% (6972/6972), 196.63 MiB | 4.22 MiB/s, done. Resolving deltas: 100% (4516/4516), done. From http://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware * [new branch] master -> origin/master warning: redirecting to https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/ warning: refname '8d69bab7a3da1913113ea98cefb73d5fa6988286' is ambiguous. Git normally never creates a ref that ends with 40 hex characters because it will be ignored when you just specify 40-hex. These refs may be created by mistake. For example, git checkout -b $br $(git rev-parse ...) where "$br" is somehow empty and a 40-hex ref is created. Please examine these refs and maybe delete them. Turn this message off by running "git config advice.objectNameWarning false" ERROR: linux-firmware-8d69bab7a3da1913113ea98cefb73d5fa6988286.tar.gz has wrong sha256 hash: ERROR: expected: 905be20e4e2d7628dea4e2e99195520fc0cce8b247faabdc52fc44a3ff2ceb04 ERROR: got : b9fce72a7b0b55eb311701dfd47914bc9e037134fa401d33e6e73ab9ebc9d116 ERROR: Incomplete download, or man-in-the-middle (MITM) attack Signed-off-by: Erico Nunes <nunes.erico@gmail.com> --- package/linux-firmware/linux-firmware.hash | 2 +- package/linux-firmware/linux-firmware.mk | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)