Patchwork [U-Boot] fix linker generated lists on non-C locales

login
register
mail settings
Submitter Pavel Herrmann
Date Dec. 2, 2012, 5:43 p.m.
Message ID <1354470210-30801-1-git-send-email-morpheus.ibis@gmail.com>
Download mbox | patch
Permalink /patch/203247/
State Superseded
Headers show

Comments

Pavel Herrmann - Dec. 2, 2012, 5:43 p.m.
Setting LC_COLLATE=C is not enough if LC_ALL=en_US.utf8. The result is a build
that has no available commands. Setting LC_ALL=C for the generator script helps.

Signed-off-by: Pavel Herrmann <morpheus.ibis@gmail.com>
---
 helper.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Marek Vasut - Dec. 2, 2012, 5:50 p.m.
Dear Pavel Herrmann,

> Setting LC_COLLATE=C is not enough if LC_ALL=en_US.utf8. The result is a
> build that has no available commands. Setting LC_ALL=C for the generator
> script helps.
> 
> Signed-off-by: Pavel Herrmann <morpheus.ibis@gmail.com>
> ---
>  helper.mk | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/helper.mk b/helper.mk
> index 79a1da0..2b75182 100644
> --- a/helper.mk
> +++ b/helper.mk
> @@ -35,7 +35,7 @@
>  #    line. Next, append "@" at the end and print the line. Finally,
>  #    append "~" at the end of line. This will make sense in conjunction
>  #    with 6) and 7).
> -# 7) Sort the lines. It is imperative to use LC_COLLATE=C here because
> +# 7) Sort the lines. It is imperative to use LC_ALL=C here because
>  #    with this, the "\a" symbol is first and "~" symbol is last. Any
>  #    other symbols fall inbetween. Symbols like "@", which marks the
>  #    end of current line (representing current section) and ".", which
> @@ -57,7 +57,7 @@ $(1): $(2)
>  		-e 's/\.[^\.]\+$$$$//' \
>  		-e ':s /^.\+$$$$/ { p;s/^\(.*\)\.[^\.]*$$$$/\1/;b s }' | \
>  	sed -n -e 'h;s/$$$$/\a/p;g;s/$$$$/@/p;g;s/$$$$/~/p;' | \
> -	LC_COLLATE=C sort -u | \
> +	LC_ALL=C sort -u | \

How will this work? LC_COLLATE is supposed to adjust the behavior of "sort" 
command, without it, "sort" will not behave as expected.

Best regards,
Marek Vasut
Pavel Herrmann - Dec. 2, 2012, 6 p.m.
On Sunday 02 of December 2012 18:50:53 Marek Vasut wrote:
> Dear Pavel Herrmann,
> 
> > Setting LC_COLLATE=C is not enough if LC_ALL=en_US.utf8. The result is a
> > build that has no available commands. Setting LC_ALL=C for the generator
> > script helps.
> > 
> > Signed-off-by: Pavel Herrmann <morpheus.ibis@gmail.com>
> > ---
> > 
> >  helper.mk | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/helper.mk b/helper.mk
> > index 79a1da0..2b75182 100644
> > --- a/helper.mk
> > +++ b/helper.mk
> > @@ -35,7 +35,7 @@
> > 
> >  #    line. Next, append "@" at the end and print the line. Finally,
> >  #    append "~" at the end of line. This will make sense in conjunction
> >  #    with 6) and 7).
> > 
> > -# 7) Sort the lines. It is imperative to use LC_COLLATE=C here because
> > +# 7) Sort the lines. It is imperative to use LC_ALL=C here because
> > 
> >  #    with this, the "\a" symbol is first and "~" symbol is last. Any
> >  #    other symbols fall inbetween. Symbols like "@", which marks the
> >  #    end of current line (representing current section) and ".", which
> > 
> > @@ -57,7 +57,7 @@ $(1): $(2)
> > 
> >  		-e 's/\.[^\.]\+$$$$//' \
> >  		-e ':s /^.\+$$$$/ { p;s/^\(.*\)\.[^\.]*$$$$/\1/;b s }' | \
> >  	
> >  	sed -n -e 'h;s/$$$$/\a/p;g;s/$$$$/@/p;g;s/$$$$/~/p;' | \
> > 
> > -	LC_COLLATE=C sort -u | \
> > +	LC_ALL=C sort -u | \
> 
> How will this work? LC_COLLATE is supposed to adjust the behavior of "sort"
> command, without it, "sort" will not behave as expected.

Clearly, LC_COLLATE is not enough for it to behave correctly, as seen on my 
setup.
This works because LC_ALL sets LC_COLLATE, as well as all other LC_* 
environment variables.

Pavel Herrmann
Marek Vasut - Dec. 2, 2012, 6:12 p.m.
Dear Pavel Herrmann,

> On Sunday 02 of December 2012 18:50:53 Marek Vasut wrote:
> > Dear Pavel Herrmann,
> > 
> > > Setting LC_COLLATE=C is not enough if LC_ALL=en_US.utf8. The result is
> > > a build that has no available commands. Setting LC_ALL=C for the
> > > generator script helps.
> > > 
> > > Signed-off-by: Pavel Herrmann <morpheus.ibis@gmail.com>
> > > ---
> > > 
> > >  helper.mk | 4 ++--
> > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/helper.mk b/helper.mk
> > > index 79a1da0..2b75182 100644
> > > --- a/helper.mk
> > > +++ b/helper.mk
> > > @@ -35,7 +35,7 @@
> > > 
> > >  #    line. Next, append "@" at the end and print the line. Finally,
> > >  #    append "~" at the end of line. This will make sense in
> > >  conjunction #    with 6) and 7).
> > > 
> > > -# 7) Sort the lines. It is imperative to use LC_COLLATE=C here because
> > > +# 7) Sort the lines. It is imperative to use LC_ALL=C here because
> > > 
> > >  #    with this, the "\a" symbol is first and "~" symbol is last. Any
> > >  #    other symbols fall inbetween. Symbols like "@", which marks the
> > >  #    end of current line (representing current section) and ".", which
> > > 
> > > @@ -57,7 +57,7 @@ $(1): $(2)
> > > 
> > >  		-e 's/\.[^\.]\+$$$$//' \
> > >  		-e ':s /^.\+$$$$/ { p;s/^\(.*\)\.[^\.]*$$$$/\1/;b s }' | \
> > >  	
> > >  	sed -n -e 'h;s/$$$$/\a/p;g;s/$$$$/@/p;g;s/$$$$/~/p;' | \
> > > 
> > > -	LC_COLLATE=C sort -u | \
> > > +	LC_ALL=C sort -u | \
> > 
> > How will this work? LC_COLLATE is supposed to adjust the behavior of
> > "sort" command, without it, "sort" will not behave as expected.
> 
> Clearly, LC_COLLATE is not enough for it to behave correctly, as seen on my
> setup.
> This works because LC_ALL sets LC_COLLATE, as well as all other LC_*
> environment variables.

Do you have any reference where I can verify this claim please?

Best regards,
Marek Vasut
Pavel Herrmann - Dec. 2, 2012, 6:26 p.m.
On Sunday 02 of December 2012 19:12:17 Marek Vasut wrote:
> Dear Pavel Herrmann,
> 
> > On Sunday 02 of December 2012 18:50:53 Marek Vasut wrote:
> > > Dear Pavel Herrmann,
> > > 
> > > > Setting LC_COLLATE=C is not enough if LC_ALL=en_US.utf8. The result is
> > > > a build that has no available commands. Setting LC_ALL=C for the
> > > > generator script helps.
> > > > 
> > > > Signed-off-by: Pavel Herrmann <morpheus.ibis@gmail.com>
> > > > ---
> > > > 
> > > >  helper.mk | 4 ++--
> > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/helper.mk b/helper.mk
> > > > index 79a1da0..2b75182 100644
> > > > --- a/helper.mk
> > > > +++ b/helper.mk
> > > > @@ -35,7 +35,7 @@
> > > > 
> > > >  #    line. Next, append "@" at the end and print the line. Finally,
> > > >  #    append "~" at the end of line. This will make sense in
> > > >  conjunction #    with 6) and 7).
> > > > 
> > > > -# 7) Sort the lines. It is imperative to use LC_COLLATE=C here
> > > > because
> > > > +# 7) Sort the lines. It is imperative to use LC_ALL=C here because
> > > > 
> > > >  #    with this, the "\a" symbol is first and "~" symbol is last. Any
> > > >  #    other symbols fall inbetween. Symbols like "@", which marks the
> > > >  #    end of current line (representing current section) and ".",
> > > >  which
> > > > 
> > > > @@ -57,7 +57,7 @@ $(1): $(2)
> > > > 
> > > >  		-e 's/\.[^\.]\+$$$$//' \
> > > >  		-e ':s /^.\+$$$$/ { p;s/^\(.*\)\.[^\.]*$$$$/\1/;b s }' | \
> > > >  	
> > > >  	sed -n -e 'h;s/$$$$/\a/p;g;s/$$$$/@/p;g;s/$$$$/~/p;' | \
> > > > 
> > > > -	LC_COLLATE=C sort -u | \
> > > > +	LC_ALL=C sort -u | \
> > > 
> > > How will this work? LC_COLLATE is supposed to adjust the behavior of
> > > "sort" command, without it, "sort" will not behave as expected.
> > 
> > Clearly, LC_COLLATE is not enough for it to behave correctly, as seen on
> > my
> > setup.
> > This works because LC_ALL sets LC_COLLATE, as well as all other LC_*
> > environment variables.
> 
> Do you have any reference where I can verify this claim please?

Which claim? that this helps, or that LC_ALL sets LC_COLLATE as well?
Pavel Herrmann - Dec. 2, 2012, 6:34 p.m.
On Sunday 02 of December 2012 19:12:17 Marek Vasut wrote:
> Dear Pavel Herrmann,
> 
> > On Sunday 02 of December 2012 18:50:53 Marek Vasut wrote:
> > > Dear Pavel Herrmann,
> > > 
> > > > Setting LC_COLLATE=C is not enough if LC_ALL=en_US.utf8. The result is
> > > > a build that has no available commands. Setting LC_ALL=C for the
> > > > generator script helps.
> > > > 
> > > > Signed-off-by: Pavel Herrmann <morpheus.ibis@gmail.com>
> > > > ---
> > > > 
> > > >  helper.mk | 4 ++--
> > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/helper.mk b/helper.mk
> > > > index 79a1da0..2b75182 100644
> > > > --- a/helper.mk
> > > > +++ b/helper.mk
> > > > @@ -35,7 +35,7 @@
> > > > 
> > > >  #    line. Next, append "@" at the end and print the line. Finally,
> > > >  #    append "~" at the end of line. This will make sense in
> > > >  conjunction #    with 6) and 7).
> > > > 
> > > > -# 7) Sort the lines. It is imperative to use LC_COLLATE=C here
> > > > because
> > > > +# 7) Sort the lines. It is imperative to use LC_ALL=C here because
> > > > 
> > > >  #    with this, the "\a" symbol is first and "~" symbol is last. Any
> > > >  #    other symbols fall inbetween. Symbols like "@", which marks the
> > > >  #    end of current line (representing current section) and ".",
> > > >  which
> > > > 
> > > > @@ -57,7 +57,7 @@ $(1): $(2)
> > > > 
> > > >  		-e 's/\.[^\.]\+$$$$//' \
> > > >  		-e ':s /^.\+$$$$/ { p;s/^\(.*\)\.[^\.]*$$$$/\1/;b s }' | \
> > > >  	
> > > >  	sed -n -e 'h;s/$$$$/\a/p;g;s/$$$$/@/p;g;s/$$$$/~/p;' | \
> > > > 
> > > > -	LC_COLLATE=C sort -u | \
> > > > +	LC_ALL=C sort -u | \
> > > 
> > > How will this work? LC_COLLATE is supposed to adjust the behavior of
> > > "sort" command, without it, "sort" will not behave as expected.
> > 
> > Clearly, LC_COLLATE is not enough for it to behave correctly, as seen on
> > my
> > setup.
> > This works because LC_ALL sets LC_COLLATE, as well as all other LC_*
> > environment variables.
> 
> Do you have any reference where I can verify this claim please?

Actually, the problem seems to be a little different - when i have LC_ALL set, 
changing LC_COLLATE doesnt do anything (try running "LC_COLLATE=crazy locale" 
with LC_ALL set)
anyways, the fix remains the same.

Pavel Herrmann
Marek Vasut - Dec. 2, 2012, 7:13 p.m.
Dear Pavel Herrmann,

> On Sunday 02 of December 2012 19:12:17 Marek Vasut wrote:
> > Dear Pavel Herrmann,
> > 
> > > On Sunday 02 of December 2012 18:50:53 Marek Vasut wrote:
> > > > Dear Pavel Herrmann,
> > > > 
> > > > > Setting LC_COLLATE=C is not enough if LC_ALL=en_US.utf8. The result
> > > > > is a build that has no available commands. Setting LC_ALL=C for
> > > > > the generator script helps.
> > > > > 
> > > > > Signed-off-by: Pavel Herrmann <morpheus.ibis@gmail.com>
> > > > > ---
> > > > > 
> > > > >  helper.mk | 4 ++--
> > > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > > > 
> > > > > diff --git a/helper.mk b/helper.mk
> > > > > index 79a1da0..2b75182 100644
> > > > > --- a/helper.mk
> > > > > +++ b/helper.mk
> > > > > @@ -35,7 +35,7 @@
> > > > > 
> > > > >  #    line. Next, append "@" at the end and print the line.
> > > > >  Finally, #    append "~" at the end of line. This will make sense
> > > > >  in conjunction #    with 6) and 7).
> > > > > 
> > > > > -# 7) Sort the lines. It is imperative to use LC_COLLATE=C here
> > > > > because
> > > > > +# 7) Sort the lines. It is imperative to use LC_ALL=C here because
> > > > > 
> > > > >  #    with this, the "\a" symbol is first and "~" symbol is last.
> > > > >  Any #    other symbols fall inbetween. Symbols like "@", which
> > > > >  marks the #    end of current line (representing current section)
> > > > >  and ".", which
> > > > > 
> > > > > @@ -57,7 +57,7 @@ $(1): $(2)
> > > > > 
> > > > >  		-e 's/\.[^\.]\+$$$$//' \
> > > > >  		-e ':s /^.\+$$$$/ { p;s/^\(.*\)\.[^\.]*$$$$/\1/;b s }' | 
\
> > > > >  	
> > > > >  	sed -n -e 'h;s/$$$$/\a/p;g;s/$$$$/@/p;g;s/$$$$/~/p;' | \
> > > > > 
> > > > > -	LC_COLLATE=C sort -u | \
> > > > > +	LC_ALL=C sort -u | \
> > > > 
> > > > How will this work? LC_COLLATE is supposed to adjust the behavior of
> > > > "sort" command, without it, "sort" will not behave as expected.
> > > 
> > > Clearly, LC_COLLATE is not enough for it to behave correctly, as seen
> > > on my
> > > setup.
> > > This works because LC_ALL sets LC_COLLATE, as well as all other LC_*
> > > environment variables.
> > 
> > Do you have any reference where I can verify this claim please?
> 
> Which claim? that this helps, or that LC_ALL sets LC_COLLATE as well?

You claim the following:

"This works because LC_ALL sets LC_COLLATE, as well as all other LC_* 
environment variables."

Where can I verify this? Do you have any link to any documentation?

Best regards,
Marek Vasut
Marek Vasut - Dec. 2, 2012, 7:15 p.m.
Dear Pavel Herrmann,

[...]

> > > > > -	LC_COLLATE=C sort -u | \
> > > > > +	LC_ALL=C sort -u | \
> > > > 
> > > > How will this work? LC_COLLATE is supposed to adjust the behavior of
> > > > "sort" command, without it, "sort" will not behave as expected.
> > > 
> > > Clearly, LC_COLLATE is not enough for it to behave correctly, as seen
> > > on my
> > > setup.
> > > This works because LC_ALL sets LC_COLLATE, as well as all other LC_*
> > > environment variables.
> > 
> > Do you have any reference where I can verify this claim please?
> 
> Actually, the problem seems to be a little different - when i have LC_ALL
> set, changing LC_COLLATE doesnt do anything (try running "LC_COLLATE=crazy
> locale" with LC_ALL set)
> anyways, the fix remains the same.

Ugh, can you please provide any relevant documentation? Besides, is the commit 
message wrong now?

Best regards,
Marek Vasut
Wolfgang Denk - Dec. 2, 2012, 7:36 p.m.
Dear Pavel Herrmann,

In message <1354470210-30801-1-git-send-email-morpheus.ibis@gmail.com> you wrote:
> Setting LC_COLLATE=C is not enough if LC_ALL=en_US.utf8. The result is a build
> that has no available commands. Setting LC_ALL=C for the generator script helps.

Setting LC_ALL is a pretty supid thing and can usually be considered
a bug.  Using it as suggested here brings us fromt he frying pan into
the fire.

IMHO it would make more sense to just make sure that LC_ALL is not set
at all, i. e. to explicitly unset it.

Best regards,

Wolfgang Denk
Wolfgang Denk - Dec. 2, 2012, 7:39 p.m.
Dear Marek Vasut,

In message <201212022013.13540.marex@denx.de> you wrote:
> 
> "This works because LC_ALL sets LC_COLLATE, as well as all other LC_* 
> environment variables."
> 
> Where can I verify this? Do you have any link to any documentation?

Is it really so difficult to find TFM these days?

Try: man locale

...
LC_ALL If set to a non-empty string value, override the values of all
the other internationalization variables.
...

Is this clear enough?

As mentioned before, if this variable is set, it can usually be
considered a bug.


Best regards,

Wolfgang Denk
Marek Vasut - Dec. 3, 2012, 3:57 a.m.
Dear Wolfgang Denk,

> Dear Pavel Herrmann,
> 
> In message <1354470210-30801-1-git-send-email-morpheus.ibis@gmail.com> you 
wrote:
> > Setting LC_COLLATE=C is not enough if LC_ALL=en_US.utf8. The result is a
> > build that has no available commands. Setting LC_ALL=C for the generator
> > script helps.
> 
> Setting LC_ALL is a pretty supid thing and can usually be considered
> a bug.  Using it as suggested here brings us fromt he frying pan into
> the fire.
> 
> IMHO it would make more sense to just make sure that LC_ALL is not set
> at all, i. e. to explicitly unset it.

Hrm ... shall we introduce a patch that sanitizes the env then? Maybe we can fix 
the LDFLAGS/yocto issue that way too?

Best regards,
Marek Vasut
Wolfgang Denk - Dec. 3, 2012, 12:26 p.m.
Dear Marek Vasut,

In message <201212030457.11755.marex@denx.de> you wrote:
> 
> > IMHO it would make more sense to just make sure that LC_ALL is not set
> > at all, i. e. to explicitly unset it.
> 
> Hrm ... shall we introduce a patch that sanitizes the env then? Maybe we can fix 
> the LDFLAGS/yocto issue that way too?

Except for LC_ALL, what else do you have in mind?


I would leave out the LDFLAGS thing here; it is unrelated, and it
bites also for all older versions of U-Boot, and we cannot fix the
past.

Best regards,

Wolfgang Denk
Marek Vasut - Dec. 3, 2012, 2:36 p.m.
Dear Wolfgang Denk,

> Dear Marek Vasut,
> 
> In message <201212030457.11755.marex@denx.de> you wrote:
> > > IMHO it would make more sense to just make sure that LC_ALL is not set
> > > at all, i. e. to explicitly unset it.
> > 
> > Hrm ... shall we introduce a patch that sanitizes the env then? Maybe we
> > can fix the LDFLAGS/yocto issue that way too?
> 
> Except for LC_ALL, what else do you have in mind?

If that's the only issue, then only LC_ALL.

> I would leave out the LDFLAGS thing here; it is unrelated, and it
> bites also for all older versions of U-Boot, and we cannot fix the
> past.

That's right.

> Best regards,
> 
> Wolfgang Denk

Best regards,
Marek Vasut

Patch

diff --git a/helper.mk b/helper.mk
index 79a1da0..2b75182 100644
--- a/helper.mk
+++ b/helper.mk
@@ -35,7 +35,7 @@ 
 #    line. Next, append "@" at the end and print the line. Finally,
 #    append "~" at the end of line. This will make sense in conjunction
 #    with 6) and 7).
-# 7) Sort the lines. It is imperative to use LC_COLLATE=C here because
+# 7) Sort the lines. It is imperative to use LC_ALL=C here because
 #    with this, the "\a" symbol is first and "~" symbol is last. Any
 #    other symbols fall inbetween. Symbols like "@", which marks the
 #    end of current line (representing current section) and ".", which
@@ -57,7 +57,7 @@  $(1): $(2)
 		-e 's/\.[^\.]\+$$$$//' \
 		-e ':s /^.\+$$$$/ { p;s/^\(.*\)\.[^\.]*$$$$/\1/;b s }' | \
 	sed -n -e 'h;s/$$$$/\a/p;g;s/$$$$/@/p;g;s/$$$$/~/p;' | \
-	LC_COLLATE=C sort -u | \
+	LC_ALL=C sort -u | \
 	sed -n -e '/\a$$$$/ { s/\./_/g;s/\a$$$$/__start = .;/p; }'\
 		-e '/~$$$$/ { s/\./_/g;s/~$$$$/__end = .;/p; }'\
 		-e '/@$$$$/ { s/\(.*\)@$$$$/*(SORT(\1.*));/p }' > $(1)