| Message ID | 20180616220601.4629-2-martin@barkynet.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | package/ca-certificates improvements | expand |
Hello, On Sat, 16 Jun 2018 23:05:59 +0100, Martin Bark wrote: > Currently c_rehash mistakenly hashes the certificates bundle > certificates.crt resulting in ${TAGET_DIR}/etc/ssl/certs/128805a3.0 > incorrectly linking to ca-certificates.crt when it should be linked to > EE_Certification_Centre_Root_CA_2.pem I can't reproduce this issue here: output/target$ ls -l etc/ssl/certs/128805a3.0 lrwxrwxrwx 1 thomas thomas 35 Jun 17 20:58 etc/ssl/certs/128805a3.0 -> EE_Certification_Centre_Root_CA.pem Also, during the review, you said that you noticed this problem when comparing the /etc/ssl/certs generated by Buildroot with the one available in Ubuntu. On Fedora, it looks just like this: $ ls -l /etc/ssl/certs/ total 4 lrwxrwxrwx 1 root root 49 May 18 13:21 ca-bundle.crt -> /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem lrwxrwxrwx 1 root root 55 May 18 13:21 ca-bundle.trust.crt -> /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt -rw-r--r-- 1 root root 2516 Apr 3 13:18 Makefile So it really doesn't look like what Buildroot produces. But I'm not familiar at all with those certificates. Peter, since you've reviewed the previous iteration of this patch series, perhaps you can take care of merging it ? Thanks! Thomas Petazzoni
Thomas, On 17 June 2018 at 20:25, Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote: > Hello, > > On Sat, 16 Jun 2018 23:05:59 +0100, Martin Bark wrote: >> Currently c_rehash mistakenly hashes the certificates bundle >> certificates.crt resulting in ${TAGET_DIR}/etc/ssl/certs/128805a3.0 >> incorrectly linking to ca-certificates.crt when it should be linked to >> EE_Certification_Centre_Root_CA_2.pem > > I can't reproduce this issue here: > > output/target$ ls -l etc/ssl/certs/128805a3.0 > lrwxrwxrwx 1 thomas thomas 35 Jun 17 20:58 etc/ssl/certs/128805a3.0 -> EE_Certification_Centre_Root_CA.pem Did you check other hashes under etc/ssl/certs/ ? Check for any hashes that link to ca-certificates.crt. I suspect you have a different hash pointing to ca-certificates. > > Also, during the review, you said that you noticed this problem when > comparing the /etc/ssl/certs generated by Buildroot with the one > available in Ubuntu. The ca-certificates package in Buildroot is based on the Debian package. Buildroot currently uses version 20180409 which is the same as currently used by Ubuntu 18.04 and Debian sid hence the contents of etc/ssl/certs should match. > > On Fedora, it looks just like this: > > $ ls -l /etc/ssl/certs/ > total 4 > lrwxrwxrwx 1 root root 49 May 18 13:21 ca-bundle.crt -> /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem > lrwxrwxrwx 1 root root 55 May 18 13:21 ca-bundle.trust.crt -> /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt > -rw-r--r-- 1 root root 2516 Apr 3 13:18 Makefile > > So it really doesn't look like what Buildroot produces. But I'm not > familiar at all with those certificates. The Red Hat based distros and Arch Linux organise their certificates slightly differently. You will not be able to do an exact comparison but in the end they achieve the same results as Debian, Ubuntu and Buildroot. Thanks Martin > > Peter, since you've reviewed the previous iteration of this patch > series, perhaps you can take care of merging it ? > > Thanks! > > Thomas Petazzoni > -- > Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons) > Embedded Linux and Kernel engineering > https://bootlin.com
Hello, On Mon, 18 Jun 2018 10:51:34 +0100, Martin Bark wrote: > > output/target$ ls -l etc/ssl/certs/128805a3.0 > > lrwxrwxrwx 1 thomas thomas 35 Jun 17 20:58 etc/ssl/certs/128805a3.0 -> EE_Certification_Centre_Root_CA.pem > > Did you check other hashes under etc/ssl/certs/ ? Check for any hashes > that link to ca-certificates.crt. I suspect you have a different hash > pointing to ca-certificates. There are no files in /etc/ssl/certs that are symlinks to ca-certificates.crt: output/target$ ls -l etc/ssl/certs/| grep ca-certificates.crt -rw-r--r-- 1 thomas thomas 207436 Jun 18 15:30 ca-certificates.crt Best regards, Thomas
Thomas, On 18 June 2018 at 15:39, Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote: > Hello, > > On Mon, 18 Jun 2018 10:51:34 +0100, Martin Bark wrote: > >> > output/target$ ls -l etc/ssl/certs/128805a3.0 >> > lrwxrwxrwx 1 thomas thomas 35 Jun 17 20:58 etc/ssl/certs/128805a3.0 -> EE_Certification_Centre_Root_CA.pem >> >> Did you check other hashes under etc/ssl/certs/ ? Check for any hashes >> that link to ca-certificates.crt. I suspect you have a different hash >> pointing to ca-certificates. > > There are no files in /etc/ssl/certs that are symlinks to > ca-certificates.crt: > > output/target$ ls -l etc/ssl/certs/| grep ca-certificates.crt > -rw-r--r-- 1 thomas thomas 207436 Jun 18 15:30 ca-certificates.crt I did some more testing and found sometimes by chance it does work, however, i was able to reproduce the issue as follows docker run -it --rm buildroot/base:20180318.1724 bash git clone git://git.busybox.net/buildroot cd buildroot make olddefconfig make ca-certificates you will see an output like this at the end of the build # Create symlinks to the certificates by their hash values /home/br-user/buildroot/output/host/bin/c_rehash /home/br-user/buildroot/output/target/etc/ssl/certs Doing /home/br-user/buildroot/output/target/etc/ssl/certs WARNING: Skipping duplicate certificate OpenTrust_Root_CA_G1.pem and one the the hashes will be wrong $ ls -l output/target/etc/ssl/certs/| grep ca-certificates.crt lrwxrwxrwx 1 br-user br-user 19 Jun 21 21:52 87229d21.0 -> ca-certificates.crt the incorrect hash is the one mentioned in the warning $ ls -l /etc/ssl/certs/87229d21.0 lrwxrwxrwx 1 root root 24 Mar 18 16:30 /etc/ssl/certs/87229d21.0 -> OpenTrust_Root_CA_G1.pem The key issue is the WARNING: Skipping duplicate certificate xxxxx where xxxx is the name of one of the files under etc/ssl/certs. Sometimes it's ca-certificates.crt and hence you don't see any issue (as you found). Sometime it's one of the CA files which is the bug i'm trying to fix. You can run c_rehash directly ./output/host/bin/c_rehash ./output/target/etc/ssl/certs and you will see the warning. If you rm output/target/etc/ssl/certs/ca-certificates.crt and try again then the warning will go. I had not released quite how random the bug. It certainly is not always EE_Certification_Centre_Root_CA.pem, it can be any CA that goes wrong. The commit message could be changed to make it clearer the bug does not allows happen to EE_Certification_Centre_Root_CA.pem but i'm 100% sure this is a bug that needs fixing. Thanks Martin > > Best regards, > > Thomas > -- > Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons) > Embedded Linux and Kernel engineering > https://bootlin.com
Hi Martin, Sorry for the late reply, On 21/06/2018 23:04, Martin Bark wrote: > Thomas, > > On 18 June 2018 at 15:39, Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote: >> Hello, >> >> On Mon, 18 Jun 2018 10:51:34 +0100, Martin Bark wrote: >> >>>> output/target$ ls -l etc/ssl/certs/128805a3.0 >>>> lrwxrwxrwx 1 thomas thomas 35 Jun 17 20:58 etc/ssl/certs/128805a3.0 -> EE_Certification_Centre_Root_CA.pem >>> >>> Did you check other hashes under etc/ssl/certs/ ? Check for any hashes >>> that link to ca-certificates.crt. I suspect you have a different hash >>> pointing to ca-certificates. >> >> There are no files in /etc/ssl/certs that are symlinks to >> ca-certificates.crt: >> >> output/target$ ls -l etc/ssl/certs/| grep ca-certificates.crt >> -rw-r--r-- 1 thomas thomas 207436 Jun 18 15:30 ca-certificates.crt > > I did some more testing and found sometimes by chance it does work, > however, i was able to reproduce the issue as follows > > docker run -it --rm buildroot/base:20180318.1724 bash > git clone git://git.busybox.net/buildroot > cd buildroot > make olddefconfig > make ca-certificates > > you will see an output like this at the end of the build > > # Create symlinks to the certificates by their hash values > /home/br-user/buildroot/output/host/bin/c_rehash > /home/br-user/buildroot/output/target/etc/ssl/certs > Doing /home/br-user/buildroot/output/target/etc/ssl/certs > WARNING: Skipping duplicate certificate OpenTrust_Root_CA_G1.pem > > and one the the hashes will be wrong > > $ ls -l output/target/etc/ssl/certs/| grep ca-certificates.crt > lrwxrwxrwx 1 br-user br-user 19 Jun 21 21:52 87229d21.0 -> > ca-certificates.crt > > the incorrect hash is the one mentioned in the warning So, if I understand correctly, what happens is this: 1. certificates get installed in /etc/ssl/certs. 2. All the certificates are bundled into a ca-certificates.crt file. 3. c_rehash is run. It looks at each certificate, calculates the hash, and creates a symlink from that hash to the certificate. The problem is that if ca-certificates.crt exists already, c_rehash will take some random certificate from it and create a symlink to ca-certificates.crt instead of to the real certificate file. But depending on the order of evaluation of the different certificate files, it may actually make the symlink point to the real certificate. What you propose looks like the good solution for it. Except in case of rebuild, so you should remove /etc/ssl/ca-certificates.crt before running c_rehash. That would also remove the need for patch 2 I think. Could you check if I'm correct, and if so, resubmit the series with: - the additional rm -f; - a commit message that includes the explanation above; - drops patch 2. If patch 2 really is needed, it needs a better explanation. Regards, Arnout > > $ ls -l /etc/ssl/certs/87229d21.0 > lrwxrwxrwx 1 root root 24 Mar 18 16:30 /etc/ssl/certs/87229d21.0 -> > OpenTrust_Root_CA_G1.pem > > The key issue is the > > WARNING: Skipping duplicate certificate xxxxx > > where xxxx is the name of one of the files under etc/ssl/certs. > Sometimes it's ca-certificates.crt and hence you don't see any issue > (as you found). Sometime it's one of the CA files which is the bug > i'm trying to fix. You can run c_rehash directly > > ./output/host/bin/c_rehash ./output/target/etc/ssl/certs > > and you will see the warning. If you rm > output/target/etc/ssl/certs/ca-certificates.crt and try again then the > warning will go. > > I had not released quite how random the bug. It certainly is not > always EE_Certification_Centre_Root_CA.pem, it can be any CA that goes > wrong. The commit message could be changed to make it clearer the bug > does not allows happen to EE_Certification_Centre_Root_CA.pem but i'm > 100% sure this is a bug that needs fixing. > > Thanks > > Martin > >> >> Best regards, >> >> Thomas >> -- >> Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons) >> Embedded Linux and Kernel engineering >> https://bootlin.com > _______________________________________________ > buildroot mailing list > buildroot@busybox.net > http://lists.busybox.net/mailman/listinfo/buildroot >
On 21/10/2018 14:15, Arnout Vandecappelle wrote: > Hi Martin, > > Sorry for the late reply, > > On 21/06/2018 23:04, Martin Bark wrote: >> Thomas, >> >> On 18 June 2018 at 15:39, Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote: >>> Hello, >>> >>> On Mon, 18 Jun 2018 10:51:34 +0100, Martin Bark wrote: >>> >>>>> output/target$ ls -l etc/ssl/certs/128805a3.0 >>>>> lrwxrwxrwx 1 thomas thomas 35 Jun 17 20:58 etc/ssl/certs/128805a3.0 -> EE_Certification_Centre_Root_CA.pem >>>> Did you check other hashes under etc/ssl/certs/ ? Check for any hashes >>>> that link to ca-certificates.crt. I suspect you have a different hash >>>> pointing to ca-certificates. >>> There are no files in /etc/ssl/certs that are symlinks to >>> ca-certificates.crt: >>> >>> output/target$ ls -l etc/ssl/certs/| grep ca-certificates.crt >>> -rw-r--r-- 1 thomas thomas 207436 Jun 18 15:30 ca-certificates.crt >> I did some more testing and found sometimes by chance it does work, >> however, i was able to reproduce the issue as follows >> >> docker run -it --rm buildroot/base:20180318.1724 bash >> git clone git://git.busybox.net/buildroot >> cd buildroot >> make olddefconfig >> make ca-certificates >> >> you will see an output like this at the end of the build >> >> # Create symlinks to the certificates by their hash values >> /home/br-user/buildroot/output/host/bin/c_rehash >> /home/br-user/buildroot/output/target/etc/ssl/certs >> Doing /home/br-user/buildroot/output/target/etc/ssl/certs >> WARNING: Skipping duplicate certificate OpenTrust_Root_CA_G1.pem >> >> and one the the hashes will be wrong >> >> $ ls -l output/target/etc/ssl/certs/| grep ca-certificates.crt >> lrwxrwxrwx 1 br-user br-user 19 Jun 21 21:52 87229d21.0 -> >> ca-certificates.crt >> >> the incorrect hash is the one mentioned in the warning > So, if I understand correctly, what happens is this: > > 1. certificates get installed in /etc/ssl/certs. > > 2. All the certificates are bundled into a ca-certificates.crt file. > > 3. c_rehash is run. It looks at each certificate, calculates the hash, and > creates a symlink from that hash to the certificate. > > The problem is that if ca-certificates.crt exists already, c_rehash will take > some random certificate from it and create a symlink to ca-certificates.crt > instead of to the real certificate file. But depending on the order of > evaluation of the different certificate files, it may actually make the symlink > point to the real certificate. > > What you propose looks like the good solution for it. Except in case of > rebuild, so you should remove /etc/ssl/ca-certificates.crt before running > c_rehash. That would also remove the need for patch 2 I think. > > Could you check if I'm correct, and if so, resubmit the series with: > > - the additional rm -f; > - a commit message that includes the explanation above; > - drops patch 2. > > If patch 2 really is needed, it needs a better explanation. Investigating a bit more turns out that I was rather wrong. The rm -f is already there, as correctly noted in your commit message. And patch 2 really is needed, it fixes a completely different set of duplicates. So I've extended the commit message a little and applied to master, thanks. Regards, Arnout > > > Regards, > Arnout > >> $ ls -l /etc/ssl/certs/87229d21.0 >> lrwxrwxrwx 1 root root 24 Mar 18 16:30 /etc/ssl/certs/87229d21.0 -> >> OpenTrust_Root_CA_G1.pem >> >> The key issue is the >> >> WARNING: Skipping duplicate certificate xxxxx >> >> where xxxx is the name of one of the files under etc/ssl/certs. >> Sometimes it's ca-certificates.crt and hence you don't see any issue >> (as you found). Sometime it's one of the CA files which is the bug >> i'm trying to fix. You can run c_rehash directly >> >> ./output/host/bin/c_rehash ./output/target/etc/ssl/certs >> >> and you will see the warning. If you rm >> output/target/etc/ssl/certs/ca-certificates.crt and try again then the >> warning will go. >> >> I had not released quite how random the bug. It certainly is not >> always EE_Certification_Centre_Root_CA.pem, it can be any CA that goes >> wrong. The commit message could be changed to make it clearer the bug >> does not allows happen to EE_Certification_Centre_Root_CA.pem but i'm >> 100% sure this is a bug that needs fixing. >> >> Thanks >> >> Martin >> >>> Best regards, >>> >>> Thomas >>> -- >>> Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons) >>> Embedded Linux and Kernel engineering >>> https://bootlin.com >> _______________________________________________ >> buildroot mailing list >> buildroot@busybox.net >> http://lists.busybox.net/mailman/listinfo/buildroot >>
Arnout, On Sun, 21 Oct 2018 at 14:38, Arnout Vandecappelle <arnout@mind.be> wrote: > > > On 21/10/2018 14:15, Arnout Vandecappelle wrote: > > Hi Martin, > > > > Sorry for the late reply, > > > > On 21/06/2018 23:04, Martin Bark wrote: > >> Thomas, > >> > >> On 18 June 2018 at 15:39, Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote: > >>> Hello, > >>> > >>> On Mon, 18 Jun 2018 10:51:34 +0100, Martin Bark wrote: > >>> > >>>>> output/target$ ls -l etc/ssl/certs/128805a3.0 > >>>>> lrwxrwxrwx 1 thomas thomas 35 Jun 17 20:58 etc/ssl/certs/128805a3.0 -> EE_Certification_Centre_Root_CA.pem > >>>> Did you check other hashes under etc/ssl/certs/ ? Check for any hashes > >>>> that link to ca-certificates.crt. I suspect you have a different hash > >>>> pointing to ca-certificates. > >>> There are no files in /etc/ssl/certs that are symlinks to > >>> ca-certificates.crt: > >>> > >>> output/target$ ls -l etc/ssl/certs/| grep ca-certificates.crt > >>> -rw-r--r-- 1 thomas thomas 207436 Jun 18 15:30 ca-certificates.crt > >> I did some more testing and found sometimes by chance it does work, > >> however, i was able to reproduce the issue as follows > >> > >> docker run -it --rm buildroot/base:20180318.1724 bash > >> git clone git://git.busybox.net/buildroot > >> cd buildroot > >> make olddefconfig > >> make ca-certificates > >> > >> you will see an output like this at the end of the build > >> > >> # Create symlinks to the certificates by their hash values > >> /home/br-user/buildroot/output/host/bin/c_rehash > >> /home/br-user/buildroot/output/target/etc/ssl/certs > >> Doing /home/br-user/buildroot/output/target/etc/ssl/certs > >> WARNING: Skipping duplicate certificate OpenTrust_Root_CA_G1.pem > >> > >> and one the the hashes will be wrong > >> > >> $ ls -l output/target/etc/ssl/certs/| grep ca-certificates.crt > >> lrwxrwxrwx 1 br-user br-user 19 Jun 21 21:52 87229d21.0 -> > >> ca-certificates.crt > >> > >> the incorrect hash is the one mentioned in the warning > > So, if I understand correctly, what happens is this: > > > > 1. certificates get installed in /etc/ssl/certs. > > > > 2. All the certificates are bundled into a ca-certificates.crt file. > > > > 3. c_rehash is run. It looks at each certificate, calculates the hash, and > > creates a symlink from that hash to the certificate. > > > > The problem is that if ca-certificates.crt exists already, c_rehash will take > > some random certificate from it and create a symlink to ca-certificates.crt > > instead of to the real certificate file. But depending on the order of > > evaluation of the different certificate files, it may actually make the symlink > > point to the real certificate. > > > > What you propose looks like the good solution for it. Except in case of > > rebuild, so you should remove /etc/ssl/ca-certificates.crt before running > > c_rehash. That would also remove the need for patch 2 I think. > > > > Could you check if I'm correct, and if so, resubmit the series with: > > > > - the additional rm -f; > > - a commit message that includes the explanation above; > > - drops patch 2. > > > > If patch 2 really is needed, it needs a better explanation. > > Investigating a bit more turns out that I was rather wrong. The rm -f is > already there, as correctly noted in your commit message. And patch 2 really is > needed, it fixes a completely different set of duplicates. > > > So I've extended the commit message a little and applied to master, thanks. Thanks for committing the patches. Martin > > > Regards, > Arnout > > > > > > > > Regards, > > Arnout > > > >> $ ls -l /etc/ssl/certs/87229d21.0 > >> lrwxrwxrwx 1 root root 24 Mar 18 16:30 /etc/ssl/certs/87229d21.0 -> > >> OpenTrust_Root_CA_G1.pem > >> > >> The key issue is the > >> > >> WARNING: Skipping duplicate certificate xxxxx > >> > >> where xxxx is the name of one of the files under etc/ssl/certs. > >> Sometimes it's ca-certificates.crt and hence you don't see any issue > >> (as you found). Sometime it's one of the CA files which is the bug > >> i'm trying to fix. You can run c_rehash directly > >> > >> ./output/host/bin/c_rehash ./output/target/etc/ssl/certs > >> > >> and you will see the warning. If you rm > >> output/target/etc/ssl/certs/ca-certificates.crt and try again then the > >> warning will go. > >> > >> I had not released quite how random the bug. It certainly is not > >> always EE_Certification_Centre_Root_CA.pem, it can be any CA that goes > >> wrong. The commit message could be changed to make it clearer the bug > >> does not allows happen to EE_Certification_Centre_Root_CA.pem but i'm > >> 100% sure this is a bug that needs fixing. > >> > >> Thanks > >> > >> Martin > >> > >>> Best regards, > >>> > >>> Thomas > >>> -- > >>> Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons) > >>> Embedded Linux and Kernel engineering > >>> https://bootlin.com > >> _______________________________________________ > >> buildroot mailing list > >> buildroot@busybox.net > >> http://lists.busybox.net/mailman/listinfo/buildroot > >>
>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes: Hi, >> Could you check if I'm correct, and if so, resubmit the series with: >> >> - the additional rm -f; >> - a commit message that includes the explanation above; >> - drops patch 2. >> >> If patch 2 really is needed, it needs a better explanation. > Investigating a bit more turns out that I was rather wrong. The rm -f is > already there, as correctly noted in your commit message. And patch 2 really is > needed, it fixes a completely different set of duplicates. > So I've extended the commit message a little and applied to master, thanks. Committed to 2018.02.x and 2018.08.x with the extended commit message, thanks.
diff --git a/package/ca-certificates/ca-certificates.mk b/package/ca-certificates/ca-certificates.mk index c19d37788b..9685d0e6f0 100644 --- a/package/ca-certificates/ca-certificates.mk +++ b/package/ca-certificates/ca-certificates.mk @@ -33,11 +33,15 @@ define CA_CERTIFICATES_INSTALL_TARGET_CMDS cd $(TARGET_DIR) ;\ for i in `find usr/share/ca-certificates -name "*.crt"` ; do \ ln -sf ../../../$$i etc/ssl/certs/`basename $${i} .crt`.pem ;\ - cat $$i >>etc/ssl/certs/ca-certificates.crt ;\ - done + cat $$i ;\ + done >$(@D)/ca-certificates.crt # Create symlinks to the certificates by their hash values $(HOST_DIR)/bin/c_rehash $(TARGET_DIR)/etc/ssl/certs + + # Install the certificates bundle + $(INSTALL) -D -m 644 $(@D)/ca-certificates.crt \ + $(TARGET_DIR)/etc/ssl/certs/ca-certificates.crt endef $(eval $(generic-package))
Currently c_rehash mistakenly hashes the certificates bundle certificates.crt resulting in ${TAGET_DIR}/etc/ssl/certs/128805a3.0 incorrectly linking to ca-certificates.crt when it should be linked to EE_Certification_Centre_Root_CA_2.pem To fix this install certificates.crt to /etc/ssl/certs after we run c_rehash to prevent it getting hashed by mistake. Note: $(TARGET_DIR)/etc/ssl/certs/ is already removed during install so this fix also works for rebuilds. Signed-off-by: Martin Bark <martin@barkynet.com> --- Changes v1 -> v2: - reword commit message and improve creation of ca-certificates.crt (Suggested By Peter Korsgaard) --- package/ca-certificates/ca-certificates.mk | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)