[5/6] pkg-infra: add possiblity to check downloaded files against known hashes

From: "Yann E. MORIN" <yann.morin.1998@free.fr>

Some of the packages that Buildroot might build are sensitive pacakges,
related to security: openssl, dropbear, ca-certificates...

Some of those packages are download over plain http, because there is
no way to get them over a secure channel, such as https.

In these dark times of pervasive surveillance, the potential for harm
that a tampered package could generate, we may want to check the integrity
of those sensitive packages.

So, each package may now provide a list of hashes for all files that
needs to be downloaded, and Buildroot will just fail if any download file
does not match its known hash.

The choosen hash function is SHA1 since it is widely available, though
theoretical attacks have been devised (but no known practical attack is
known).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
Note: this is not a bullet-proof solution, since Buildroot may itself be
compromised. But if we eventually sign our releases, then we secure the
list of hashes at the same time. Only random snapshots from the repository
may be at risk of tampering, although this is highly doubtfull, given how
git stores its data.

Also, before we commit a list of hashes to the tree, we may want to
setup a chain-of-trust to validate that thos hashes are correct.
We may want to discuss this during our next developpers' day in
Brussels in February.

Note-2: The laternative to sha1 would be sha2 (256- or 512-bit), but
oldish "enterprise-class" distributions  may be missing them entirely.
sha256sum and sha512sum were added to coreutils in 2005-10-23, and RHEL5
seems to have them. But better be safe than sorry. If sha2 should be
considered instead of sha1, then it is very easy to switch now. Switching
later would require that we revalidate all packages that have hashes,
which could prove to be quite time-demanding if we have lots of
packages using hashes.
---
 docs/manual/adding-packages-directory.txt | 35 ++++++++++++++++++++++++++++
 package/pkg-download.mk                   | 15 ++++++++++--
 support/download/check-hash               | 38 +++++++++++++++++++++++++++++++
 3 files changed, 86 insertions(+), 2 deletions(-)
 create mode 100755 support/download/check-hash

Hi Yann,

On Mon, Jan 13, 2014 at 12:44:48AM +0100, Yann E. MORIN wrote:

[...]

> diff --git a/support/download/check-hash b/support/download/check-hash
> new file mode 100755
> index 0000000..5cf708f
> --- /dev/null
> +++ b/support/download/check-hash
> @@ -0,0 +1,38 @@
> +#!/bin/sh
> +set -e
> +
> +# Helper to check a file matches its known hash
> +# Call it with:
> +#   $1: the basename of the package's tarball
> +#   $2: the full path to the file to check
> +#   $3: the path of the file containing all the the expected hashes
> +
> +tarball="${1}"
> +file="${2}"
> +h_file="${3}"
> +
> +# Does the hash-file exist?
> +if [ ! -f "${h_file}" ]; then
> +    exit 0
> +fi
> +
> +# Do we know a hash for that tarball?
> +known=$( grep -E '^[[:xdigit:]]+[[:space:]]{2}'"${tarball}"'$$' "${h_file}" \
> +         |cut -d ' ' -f 1
> +       )
> +if [ -z "${known}" ]; then
> +    exit 0
> +fi
> +
> +# Do the hashes match?
> +hash=$( sha1sum "${file}" |cut -d ' ' -f 1 )
> +if [ "${hash}" = "${known}" ]; then
> +    exit 0
> +fi
> +
> +printf "ERROR: %s has wrong SHA256\n" "${tarball}"

That's SHA1.

> +printf "ERROR: expected: %s\n" "${known}"
> +printf "ERROR: got     : %s\n" "${hash}"
> +printf "ERROR: Incomplete download, or MITM attack\n"
> +
> +exit 1

baruch

Baruch, All,

On 2014-01-13 06:53 +0200, Baruch Siach spake thusly:
> On Mon, Jan 13, 2014 at 12:44:48AM +0100, Yann E. MORIN wrote:
[--SNIP--]
> > diff --git a/support/download/check-hash b/support/download/check-hash
> > new file mode 100755
> > index 0000000..5cf708f
> > --- /dev/null
> > +++ b/support/download/check-hash
> > @@ -0,0 +1,38 @@
> > +#!/bin/sh
> > +set -e
> > +
> > +# Helper to check a file matches its known hash
> > +# Call it with:
> > +#   $1: the basename of the package's tarball
> > +#   $2: the full path to the file to check
> > +#   $3: the path of the file containing all the the expected hashes
> > +
> > +tarball="${1}"
> > +file="${2}"
> > +h_file="${3}"
> > +
> > +# Does the hash-file exist?
> > +if [ ! -f "${h_file}" ]; then
> > +    exit 0
> > +fi
> > +
> > +# Do we know a hash for that tarball?
> > +known=$( grep -E '^[[:xdigit:]]+[[:space:]]{2}'"${tarball}"'$$' "${h_file}" \
> > +         |cut -d ' ' -f 1
> > +       )
> > +if [ -z "${known}" ]; then
> > +    exit 0
> > +fi
> > +
> > +# Do the hashes match?
> > +hash=$( sha1sum "${file}" |cut -d ' ' -f 1 )
> > +if [ "${hash}" = "${known}" ]; then
> > +    exit 0
> > +fi
> > +
> > +printf "ERROR: %s has wrong SHA256\n" "${tarball}"
> 
> That's SHA1.

Doh, good catch!

Thank you!

Regards,
Yann E. MORIN.

On 13/01/14 00:44, Yann E. MORIN wrote:
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
>
> Some of the packages that Buildroot might build are sensitive pacakges,
> related to security: openssl, dropbear, ca-certificates...
>
> Some of those packages are download over plain http, because there is
> no way to get them over a secure channel, such as https.
>
> In these dark times of pervasive surveillance, the potential for harm
> that a tampered package could generate, we may want to check the integrity
> of those sensitive packages.
>
> So, each package may now provide a list of hashes for all files that
> needs to be downloaded, and Buildroot will just fail if any download file
> does not match its known hash.
>
> The choosen hash function is SHA1 since it is widely available, though
> theoretical attacks have been devised (but no known practical attack is
> known).
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> ---
> Note: this is not a bullet-proof solution, since Buildroot may itself be
> compromised. But if we eventually sign our releases, then we secure the
> list of hashes at the same time. Only random snapshots from the repository
> may be at risk of tampering, although this is highly doubtfull, given how
> git stores its data.
>
> Also, before we commit a list of hashes to the tree, we may want to
> setup a chain-of-trust to validate that thos hashes are correct.
> We may want to discuss this during our next developpers' day in
> Brussels in February.

  I think the risk is small, because the package will be downloaded by 
multiple users and autobuilders, so an incorrect hash in the buildroot 
sources will lead to download failure reports.

>
> Note-2: The laternative to sha1 would be sha2 (256- or 512-bit), but
> oldish "enterprise-class" distributions  may be missing them entirely.
> sha256sum and sha512sum were added to coreutils in 2005-10-23, and RHEL5
> seems to have them. But better be safe than sorry. If sha2 should be
> considered instead of sha1, then it is very easy to switch now. Switching
> later would require that we revalidate all packages that have hashes,
> which could prove to be quite time-demanding if we have lots of
> packages using hashes.

  We can be more future-safe by storing the hash that is used in the 
.hash file itself.

[snip]
> diff --git a/package/pkg-download.mk b/package/pkg-download.mk
> index f3354d1..5627850 100644
> --- a/package/pkg-download.mk
> +++ b/package/pkg-download.mk
> @@ -58,6 +58,14 @@ domainseparator=$(if $(1),$(1),/)
>   # github(user,package,version): returns site of github repository
>   github = https://github.com/$(1)/$(2)/tarball/$(3)
>
> +# Helper for checking a tarball's checksum
> +# $(1): the basename of the tarball to check
> +# $(2): the full path to the file to check
> +define VERIFY_SHA256

  VERIFY_HASH would be better.

  Regards,
  Arnout

> +	support/download/check-hash $(1) $(2) \
> +		$($(PKG)_DIR_PREFIX)/$($(PKG)_NAME)/$($(PKG)_NAME).hash
> +endef
> +
>   ################################################################################
>   # The DOWNLOAD_* helpers are in charge of getting a working copy
>   # of the source repository for their corresponding SCM,

[snip]

Arnout, All,

On 2014-01-14 22:37 +0100, Arnout Vandecappelle spake thusly:
> On 13/01/14 00:44, Yann E. MORIN wrote:
> >From: "Yann E. MORIN" <yann.morin.1998@free.fr>
> >Some of the packages that Buildroot might build are sensitive pacakges,
> >related to security: openssl, dropbear, ca-certificates...
[--SNIP--]
> >So, each package may now provide a list of hashes for all files that
> >needs to be downloaded, and Buildroot will just fail if any download file
> >does not match its known hash.
[--SNIP--]
> >Also, before we commit a list of hashes to the tree, we may want to
> >setup a chain-of-trust to validate that thos hashes are correct.
> >We may want to discuss this during our next developpers' day in
> >Brussels in February.
> 
>  I think the risk is small, because the package will be downloaded by
> multiple users and autobuilders, so an incorrect hash in the buildroot
> sources will lead to download failure reports.

Yes, anyone may run "make allyespackageconfig source" (although that
will leave out a few packages).

> >Note-2: The laternative to sha1 would be sha2 (256- or 512-bit), but
> >oldish "enterprise-class" distributions  may be missing them entirely.
> >sha256sum and sha512sum were added to coreutils in 2005-10-23, and RHEL5
> >seems to have them. But better be safe than sorry. If sha2 should be
> >considered instead of sha1, then it is very easy to switch now. Switching
> >later would require that we revalidate all packages that have hashes,
> >which could prove to be quite time-demanding if we have lots of
> >packages using hashes.
> 
>  We can be more future-safe by storing the hash that is used in the .hash
> file itself.

Hu?

> >diff --git a/package/pkg-download.mk b/package/pkg-download.mk
> >index f3354d1..5627850 100644
> >--- a/package/pkg-download.mk
> >+++ b/package/pkg-download.mk
> >@@ -58,6 +58,14 @@ domainseparator=$(if $(1),$(1),/)
> >  # github(user,package,version): returns site of github repository
> >  github = https://github.com/$(1)/$(2)/tarball/$(3)
> >
> >+# Helper for checking a tarball's checksum
> >+# $(1): the basename of the tarball to check
> >+# $(2): the full path to the file to check
> >+define VERIFY_SHA256
> 
>  VERIFY_HASH would be better.

Doh. Right.

Thank you!

Regards,
Yann E. MORIN.

On 01/12/2014 08:44 PM, Yann E. MORIN wrote:

> The choosen hash function is SHA1 since it is widely available, though
> theoretical attacks have been devised (but no known practical attack is
> known).

Hi.
Normally you want whichever upstream uses for their announcement, you
shouldn't hardcode the method and you definitely shouldn't compute when
you download.
That's because you may bump some time later after compromise and if you
compute it when bumping then you're just giving your word on a maybe
compromised one.
Just my $.02
Regards.

On 15/01/14 00:34, Yann E. MORIN wrote:
> Arnout, All,
>
> On 2014-01-14 22:37 +0100, Arnout Vandecappelle spake thusly:
>> On 13/01/14 00:44, Yann E. MORIN wrote:
[snip]
>>> Note-2: The laternative to sha1 would be sha2 (256- or 512-bit), but
>>> oldish "enterprise-class" distributions  may be missing them entirely.
>>> sha256sum and sha512sum were added to coreutils in 2005-10-23, and RHEL5
>>> seems to have them. But better be safe than sorry. If sha2 should be
>>> considered instead of sha1, then it is very easy to switch now. Switching
>>> later would require that we revalidate all packages that have hashes,
>>> which could prove to be quite time-demanding if we have lots of
>>> packages using hashes.
>>
>>   We can be more future-safe by storing the hash that is used in the .hash
>> file itself.
>
> Hu?

  If the hash file contains the following:

486fb55c3efa71148fe07895fd713ea3a5ae343a  sha1  libfoo-1.2.3.tar.bz2

then you can now let the script check that the second field is sha1, and 
later you can support different hash methods. In that case, it is not 
necessary to update all the files when we want to switch to a new hash 
method.

  (Incidentally, it also enables Gustavo's suggestion to use whatever 
upstream provides.)

[snip]

On 01/15/2014 05:22 AM, Arnout Vandecappelle wrote:

>  If the hash file contains the following:
> 
> 486fb55c3efa71148fe07895fd713ea3a5ae343a  sha1  libfoo-1.2.3.tar.bz2
> 
> then you can now let the script check that the second field is sha1, and
> later you can support different hash methods. In that case, it is not
> necessary to update all the files when we want to switch to a new hash
> method.
> 
>  (Incidentally, it also enables Gustavo's suggestion to use whatever
> upstream provides.)

Yes.
A little explanation on why upstream hashes should be used (my mail last
night was a bit rushed out, busy busy).
When upstream releases a tarball normally it'll fire off an announcement
mail to some mailing list with (hopefully) the hash for the tarball(s).
Usually this hash(es) will also live in some project website hosting.
If the website is compromised then the hash there can also be
compromised and you are computing your hash on a bad tarball (if done
when bumping version) or an altered hash in the website.
Sent mail (mailing list archives, personal mail) is much harder to
compromise all at once hence the original hash will still be true if all
is dandy.
Of course there's the possibility that the developer machine has been
compromised or some code has been "sneaked under the carpet" with peer
review failing to notice, but then there's no hash to save you there and
it's not the intention of this either.
Regards.

Arnout, All,

On 2014-01-15 09:22 +0100, Arnout Vandecappelle spake thusly:
> On 15/01/14 00:34, Yann E. MORIN wrote:
> >Arnout, All,
> >
> >On 2014-01-14 22:37 +0100, Arnout Vandecappelle spake thusly:
> >>On 13/01/14 00:44, Yann E. MORIN wrote:
> [snip]
> >>>Note-2: The laternative to sha1 would be sha2 (256- or 512-bit), but
> >>>oldish "enterprise-class" distributions  may be missing them entirely.
> >>>sha256sum and sha512sum were added to coreutils in 2005-10-23, and RHEL5
> >>>seems to have them. But better be safe than sorry. If sha2 should be
> >>>considered instead of sha1, then it is very easy to switch now. Switching
> >>>later would require that we revalidate all packages that have hashes,
> >>>which could prove to be quite time-demanding if we have lots of
> >>>packages using hashes.
> >>
> >>  We can be more future-safe by storing the hash that is used in the .hash
> >>file itself.
> >
> >Hu?
> 
>  If the hash file contains the following:
> 
> 486fb55c3efa71148fe07895fd713ea3a5ae343a  sha1  libfoo-1.2.3.tar.bz2

OK, I see what you meant, now.

> then you can now let the script check that the second field is sha1, and
> later you can support different hash methods. In that case, it is not
> necessary to update all the files when we want to switch to a new hash
> method.

However, that means the file is no longer the output of:
    sha1sum files-to-check* >package.hash

or of any other hash utility: sha*sum all generates similarly-formatted
outputs:
    hash <two spaces> filename

Which was the reason I choose that format.

If we'd use your suggestion, we'd need a simple way to generate that
file, or it'd be error prone.

Regards,
Yann E. MORIN.

Gustavo, All,

On 2014-01-15 10:22 -0300, Gustavo Zacarias spake thusly:
> On 01/15/2014 05:22 AM, Arnout Vandecappelle wrote:
> 
> >  If the hash file contains the following:
> > 
> > 486fb55c3efa71148fe07895fd713ea3a5ae343a  sha1  libfoo-1.2.3.tar.bz2
> > 
> > then you can now let the script check that the second field is sha1, and
> > later you can support different hash methods. In that case, it is not
> > necessary to update all the files when we want to switch to a new hash
> > method.
> > 
> >  (Incidentally, it also enables Gustavo's suggestion to use whatever
> > upstream provides.)
> 
> Yes.
> A little explanation on why upstream hashes should be used (my mail last
> night was a bit rushed out, busy busy).
> When upstream releases a tarball normally it'll fire off an announcement
> mail to some mailing list with (hopefully) the hash for the tarball(s).
> Usually this hash(es) will also live in some project website hosting.
> If the website is compromised then the hash there can also be
> compromised and you are computing your hash on a bad tarball (if done
> when bumping version) or an altered hash in the website.
> Sent mail (mailing list archives, personal mail) is much harder to
> compromise all at once hence the original hash will still be true if all
> is dandy.
> Of course there's the possibility that the developer machine has been
> compromised or some code has been "sneaked under the carpet" with peer
> review failing to notice, but then there's no hash to save you there and
> it's not the intention of this either.

I understand your concerns.

But, whether we do compute the hashes ourselves, or retrieve them from
an annoucment mail (which by your own saying is not systematic), we
can't blindly accept a patch that contains hashes without verifying
them.

So, whenever Peter would be about to apply a package, he'd have to check
for himself that the hashes are correct.

For that, he'd have to go the package's website, dig up the announcement
mail from the list, and compare the hashes.

So, whether we compute them, or get them from the annoucement mail,
Peter would still have to check them in the end, prior to applying the
patch.

And we still do not have solved the problem of packages for which no
hash has been publicly posted and archived.

So, all that comes to mind now is that we need signatures, not hashes.
But not all packages have signed releases either. So we're back to
square-one.

In the end, I wonder how much we do want to protect the downloads.
I firmly believe that a set of hashes are enough for what we want to do.
If a user is even most concerned about security, then he should (and
would anyway) audit the download for integrity before shipping a
product.

And since we do sign our releases (peter does it), a user can verify our
releases easily, and assess that the hashes are legit.

Regards,
Yann E. MORIN.

On 01/17/2014 08:02 PM, Yann E. MORIN wrote:

> I understand your concerns.
> 
> But, whether we do compute the hashes ourselves, or retrieve them from
> an annoucment mail (which by your own saying is not systematic), we
> can't blindly accept a patch that contains hashes without verifying
> them.
> 
> So, whenever Peter would be about to apply a package, he'd have to check
> for himself that the hashes are correct.
> 
> For that, he'd have to go the package's website, dig up the announcement
> mail from the list, and compare the hashes.
> 
> So, whether we compute them, or get them from the annoucement mail,
> Peter would still have to check them in the end, prior to applying the
> patch.
> 
> And we still do not have solved the problem of packages for which no
> hash has been publicly posted and archived.
> 
> So, all that comes to mind now is that we need signatures, not hashes.
> But not all packages have signed releases either. So we're back to
> square-one.
> 
> In the end, I wonder how much we do want to protect the downloads.
> I firmly believe that a set of hashes are enough for what we want to do.
> If a user is even most concerned about security, then he should (and
> would anyway) audit the download for integrity before shipping a
> product.
> 
> And since we do sign our releases (peter does it), a user can verify our
> releases easily, and assess that the hashes are legit.

The hashes are first and foremost a consistency feature, they are not in
any way a security feature and anyone who believes that is fooling himself.
My comment was to just give an extra degree of consistency since the
developer's box can be compromised without his knowledge and would emit
a perfectly valid hashed tarball with a backdoor, weakness or whatnot.
This would need guidelines as to the preferred origin of the hashes, and
yes not everyone does hashes and/or announcements so it's only that,
guidelines :)
I don't know if i'd go as far as a chain of trust, the burden would be
too much at the moment i think, if we add file size combined with the
hash the collisions would be reduced and you get a certain degree of
extra protection.
Peter or some other committer shouldn't need to get too involved in
this, basically make the check against the submitted package and it will
match or not.
If at some point it's discovered there was a compromise upstream you can
compare hashes and know if you're affected (depending on the package
being submitted/updated after the compromise or not).
But remember, short of doing source code audit there's never any
guarantee, so i wouldn't want to sign that everything is fine in an
irresponsible way when many of the packages are just built tested.
Regards.

Hi Yann, Arnout,

Yann E. MORIN wrote:
> Arnout, All,
>
> On 2014-01-15 09:22 +0100, Arnout Vandecappelle spake thusly:
>> On 15/01/14 00:34, Yann E. MORIN wrote:
>>> Arnout, All,
>>>
>>> On 2014-01-14 22:37 +0100, Arnout Vandecappelle spake thusly:
>>>> On 13/01/14 00:44, Yann E. MORIN wrote:
>> [snip]
>>>>> Note-2: The laternative to sha1 would be sha2 (256- or 512-bit), but
>>>>> oldish "enterprise-class" distributions  may be missing them entirely.
>>>>> sha256sum and sha512sum were added to coreutils in 2005-10-23, and RHEL5
>>>>> seems to have them. But better be safe than sorry. If sha2 should be
>>>>> considered instead of sha1, then it is very easy to switch now. Switching
>>>>> later would require that we revalidate all packages that have hashes,
>>>>> which could prove to be quite time-demanding if we have lots of
>>>>> packages using hashes.
>>>>
>>>>   We can be more future-safe by storing the hash that is used in the .hash
>>>> file itself.
>>>
>>> Hu?
>>
>>   If the hash file contains the following:
>>
>> 486fb55c3efa71148fe07895fd713ea3a5ae343a  sha1  libfoo-1.2.3.tar.bz2
>
> OK, I see what you meant, now.
>
>> then you can now let the script check that the second field is sha1, and
>> later you can support different hash methods. In that case, it is not
>> necessary to update all the files when we want to switch to a new hash
>> method.
>
> However, that means the file is no longer the output of:
>      sha1sum files-to-check* >package.hash
>
> or of any other hash utility: sha*sum all generates similarly-formatted
> outputs:
>      hash <two spaces> filename
>
> Which was the reason I choose that format.
>
> If we'd use your suggestion, we'd need a simple way to generate that
> file, or it'd be error prone.

Or keep in BR one file per each hash algorithm: a *.hash.sha1
file, a *.hash.sha256 file etc. Whichever file(s) are found are
checked.

This would save both needs, wouldn't it? It should not take many more
lines of code to implement.