diff mbox

[v6,5/8] support/download: add possibility to not fail on missing hash

Message ID 1429466158-28878-5-git-send-email-arnout@mind.be
State Changes Requested
Headers show

Commit Message

Arnout Vandecappelle April 19, 2015, 5:55 p.m. UTC
From: "Yann E. MORIN" <yann.morin.1998@free.fr>

In very constrained cases, it might be needed to not fail if a hash is
missing. This is notably the case for custom external toolchains to be
downloaded, because we do have a .hash file for external toolchains,
but we obviously can not have hashes for all existing custom toolchains
(he, "custom"!).

So, add a way to avoid failing in that case.

>From the Makefile, we export the list of files for which not to check
the hash. Then, from the check-hash script, if no check was done, and
the file we were trying to match in in this exclusion list, we just exit
without error.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: fix the pattern in the case statement]
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
Changed v5 -> v6:
  - fix the pattern in the case clause

Changes v4 -> v5:
  - micro-optimisation, use case-esac instead of a for-loop  (Arnout)
  - typoes  (Arnout)

Changes v3 -> v4:
  - drop the magic value, use a list of excluded files  (Arnout)

Changes v1 -> v2:
  - fix typoes in commit log

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 package/pkg-download.mk     | 3 +++
 support/download/check-hash | 6 ++++++
 2 files changed, 9 insertions(+)

Comments

Yann E. MORIN April 19, 2015, 6:38 p.m. UTC | #1
Arnout, All,

On 2015-04-19 19:55 +0200, Arnout Vandecappelle (Essensium/Mind) spake thusly:
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
> 
> In very constrained cases, it might be needed to not fail if a hash is
> missing. This is notably the case for custom external toolchains to be
> downloaded, because we do have a .hash file for external toolchains,
> but we obviously can not have hashes for all existing custom toolchains
> (he, "custom"!).
> 
> So, add a way to avoid failing in that case.
> 
> >From the Makefile, we export the list of files for which not to check

Stray leading '>' probably for a copy-paste from your mailer.

> the hash. Then, from the check-hash script, if no check was done, and
> the file we were trying to match in in this exclusion list, we just exit
> without error.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> [Arnout: fix the pattern in the case statement]
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> ---
> Changed v5 -> v6:
>   - fix the pattern in the case clause

Yeah, thanks. Obviously, I did not test that case (although the other
tests I ran were OK, I missed testing the custom toolchain to be
downloaded, which is a shame since that's exactly what I was trying to
fix with this series, to begin with...).

[--SNIP--]
> diff --git a/support/download/check-hash b/support/download/check-hash
> index 678a7ef..d378701 100755
> --- a/support/download/check-hash
> +++ b/support/download/check-hash
> @@ -99,6 +99,12 @@ while read t h f; do
>  done <"${h_file}"
>  
>  if [ ${nb_checks} -eq 0 ]; then
> +    case " ${BR_NO_CHECK_HASH_FOR} " in
> +    *\ ${base}\ *)

I don't like much the escaping when it can be avoided. We can still use
quotes (even if not very much better), like so:

    *" ${base} "*

Regards,
Yann E. MORIN.

> +        # File explicitly has no hash
> +        exit 0
> +        ;;
> +    esac
>      printf "ERROR: No hash found for %s\n" "${base}" >&2
>      exit 0
>  fi
> -- 
> 2.1.4
> 
> _______________________________________________
> buildroot mailing list
> buildroot@busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
Arnout Vandecappelle April 20, 2015, 7:45 a.m. UTC | #2
On 19/04/15 20:38, Yann E. MORIN wrote:
> Arnout, All,
> 
> On 2015-04-19 19:55 +0200, Arnout Vandecappelle (Essensium/Mind) spake thusly:
>> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
>>
>> In very constrained cases, it might be needed to not fail if a hash is
>> missing. This is notably the case for custom external toolchains to be
>> downloaded, because we do have a .hash file for external toolchains,
>> but we obviously can not have hashes for all existing custom toolchains
>> (he, "custom"!).
>>
>> So, add a way to avoid failing in that case.
>>
>> >From the Makefile, we export the list of files for which not to check
> 
> Stray leading '>' probably for a copy-paste from your mailer.

 pwclient git-am actually... I would have expected patchwork (or actually, the
python mail handling infra) to handle >From correctly...

> 
>> the hash. Then, from the check-hash script, if no check was done, and
>> the file we were trying to match in in this exclusion list, we just exit
>> without error.
>>
[snip]
>> diff --git a/support/download/check-hash b/support/download/check-hash
>> index 678a7ef..d378701 100755
>> --- a/support/download/check-hash
>> +++ b/support/download/check-hash
>> @@ -99,6 +99,12 @@ while read t h f; do
>>  done <"${h_file}"
>>  
>>  if [ ${nb_checks} -eq 0 ]; then
>> +    case " ${BR_NO_CHECK_HASH_FOR} " in
>> +    *\ ${base}\ *)
> 
> I don't like much the escaping when it can be avoided. We can still use
> quotes (even if not very much better), like so:
> 
>     *" ${base} "*

 It's your patch, modify as you like :-)

 I just reposted it since I anyway had it locally modified.

 Regards,
 Arnout
Yann E. MORIN April 20, 2015, 9:24 p.m. UTC | #3
Arnout, All,

On 2015-04-20 09:45 +0200, Arnout Vandecappelle spake thusly:
> On 19/04/15 20:38, Yann E. MORIN wrote:
[--SNIP--]
> >> >From the Makefile, we export the list of files for which not to check
> > 
> > Stray leading '>' probably for a copy-paste from your mailer.
> 
>  pwclient git-am actually... I would have expected patchwork (or actually, the
> python mail handling infra) to handle >From correctly...

Actually, that comes from RFC 4155.
    https://tools.ietf.org/html/rfc4155

TL;DR: "From " is actually used in mbox format to define the begining
of an email. Thus, any lines in a body, that begins thusly, must be
"quoted" to avoid improperly detecting the begining of an email.

See also this mail in the Patchwork ML archives:
    https://lists.ozlabs.org/pipermail/patchwork/2014-November/001218.html

I don't think this was ever applied...

[--SNIP--]
> > I don't like much the escaping when it can be avoided. We can still use
> > quotes (even if not very much better), like so:
> > 
> >     *" ${base} "*
> 
>  It's your patch, modify as you like :-)

Hehe! :-)

>  I just reposted it since I anyway had it locally modified.

Yeah, thanks! :-)

Regards,
Yann E. MORIN.
diff mbox

Patch

diff --git a/package/pkg-download.mk b/package/pkg-download.mk
index e274712..1fc23c4 100644
--- a/package/pkg-download.mk
+++ b/package/pkg-download.mk
@@ -60,6 +60,9 @@  domainseparator = $(if $(1),$(1),/)
 # github(user,package,version): returns site of GitHub repository
 github = https://github.com/$(1)/$(2)/archive/$(3)
 
+# Expressly do not check hashes for those files
+export BR_NO_CHECK_HASH_FOR
+
 ################################################################################
 # The DOWNLOAD_* helpers are in charge of getting a working copy
 # of the source repository for their corresponding SCM,
diff --git a/support/download/check-hash b/support/download/check-hash
index 678a7ef..d378701 100755
--- a/support/download/check-hash
+++ b/support/download/check-hash
@@ -99,6 +99,12 @@  while read t h f; do
 done <"${h_file}"
 
 if [ ${nb_checks} -eq 0 ]; then
+    case " ${BR_NO_CHECK_HASH_FOR} " in
+    *\ ${base}\ *)
+        # File explicitly has no hash
+        exit 0
+        ;;
+    esac
     printf "ERROR: No hash found for %s\n" "${base}" >&2
     exit 0
 fi