[v1.1,0/7] aarch64: Allow overriding HWCAP_CPUID feature check

On Thursday 11 May 2017 09:08 PM, Adhemerval Zanella wrote:
> It has fixed x86 build, however I am still seeing a runtime failure with:
> 
> FAIL: elf/tst-env-setuid
> 
> Which basically does:
> 
> $ env GCONV_PATH=/home/azanella/Projects/glibc/build/x86_64-linux-gnu/iconvdata LOCPATH=/home/azanella/Projects/glibc/build/x86_64-linux-gnu/localedata LC_ALL=C MALLOC_CHECK_=2 MALLOC_MMAP_THRESHOLD_=4096 LD_HWCAP_MASK=0xffffffff  /home/azanella/Projects/glibc/build/x86_64-linux-gnu/elf/tst-env-setuid
> : error while loading shared libraries: cannot create capability list: Cannot allocate memory
> 
> It is failing on elf/dl-hwcaps.c:
> 
> 201   /* The result structure: we use a very compressed way to store the
> 202      various combinations of capability names.  */
> 203   *sz = 1 << cnt;
> 204   result = (struct r_strlenpair *) malloc (*sz * sizeof (*result) + total);
> 205   if (result == NULL)
> 206     _dl_signal_error (ENOMEM, NULL, NULL,
> 207                       N_("cannot create capability list"));
> 
> With a total value being way too large.  I am still digesting why exactly
> this value is being too large.

Heh, I think I know what is going on.  There are two places where it
fails with the exact same error, and both can be reached when there are
too many bits set in DL_HWCAP_IMPORTANT. Setting LD_HWCAP_MASK to
0xffffffff in the test case is crossing both limits I assume.  The fact
that I don't see the failure and you do seems to be because of malloc
not failing on my system and failing on yours.  This may be due to
differences in overcommit behaviour.

Fix is to reduce the number of bits set in the test, so really just this
(untested) patch should do, can you please confirm?

Thanks,
Siddhesh

ChangeLog:

	* elf/Makefile (tst-env-setuid-ENV): Set LD_HWCAP_MASK to 0x1.

On 11/05/17 17:49, Siddhesh Poyarekar wrote:
> On Thursday 11 May 2017 09:08 PM, Adhemerval Zanella wrote:
>> It has fixed x86 build, however I am still seeing a runtime failure with:
>>
>> FAIL: elf/tst-env-setuid
>>
>> Which basically does:
>>
>> $ env GCONV_PATH=/home/azanella/Projects/glibc/build/x86_64-linux-gnu/iconvdata LOCPATH=/home/azanella/Projects/glibc/build/x86_64-linux-gnu/localedata LC_ALL=C MALLOC_CHECK_=2 MALLOC_MMAP_THRESHOLD_=4096 LD_HWCAP_MASK=0xffffffff  /home/azanella/Projects/glibc/build/x86_64-linux-gnu/elf/tst-env-setuid
>> : error while loading shared libraries: cannot create capability list: Cannot allocate memory
>>
>> It is failing on elf/dl-hwcaps.c:
>>
>> 201   /* The result structure: we use a very compressed way to store the
>> 202      various combinations of capability names.  */
>> 203   *sz = 1 << cnt;
>> 204   result = (struct r_strlenpair *) malloc (*sz * sizeof (*result) + total);
>> 205   if (result == NULL)
>> 206     _dl_signal_error (ENOMEM, NULL, NULL,
>> 207                       N_("cannot create capability list"));
>>
>> With a total value being way too large.  I am still digesting why exactly
>> this value is being too large.
> 
> Heh, I think I know what is going on.  There are two places where it
> fails with the exact same error, and both can be reached when there are
> too many bits set in DL_HWCAP_IMPORTANT. Setting LD_HWCAP_MASK to
> 0xffffffff in the test case is crossing both limits I assume.  The fact
> that I don't see the failure and you do seems to be because of malloc
> not failing on my system and failing on yours.  This may be due to
> differences in overcommit behaviour.
> 

it seems LD_HWCAP_MASK affects ldso behaviour,
(some library search path is computed based on
"important" hwcap names)

so it's probably not a good idea to reuse it for
ifunc dispatch tuning of the libc.

(the exact semantics of the LD_HWCAP_MASK is not
clear to me though)

> Fix is to reduce the number of bits set in the test, so really just this
> (untested) patch should do, can you please confirm?
> 
> Thanks,
> Siddhesh
> 
> ChangeLog:
> 
> 	* elf/Makefile (tst-env-setuid-ENV): Set LD_HWCAP_MASK to 0x1.
>

On Friday 12 May 2017 05:40 PM, Szabolcs Nagy wrote:
> it seems LD_HWCAP_MASK affects ldso behaviour,
> (some library search path is computed based on
> "important" hwcap names)
> 
> so it's probably not a good idea to reuse it for
> ifunc dispatch tuning of the libc.
> 
> (the exact semantics of the LD_HWCAP_MASK is not
> clear to me though)

The dynamic linker is trying to allocate space for strings to describe
the hwcaps and somehow setting hwcap_mask like that is causing it to
allocated lots of memory.  It is a bug in the dynamic linker, not a
problem with setting hwcap_mask.

My 'solution' of fixing the test case papered over the fact that the
dynamic linker should not be barfing like this.  I'll work on this.
This is unrelated to the patchset under discussion however since the
problem should be visible even in 2.25.

Siddhesh

On 12/05/2017 10:44, Siddhesh Poyarekar wrote:
> On Friday 12 May 2017 05:40 PM, Szabolcs Nagy wrote:
>> it seems LD_HWCAP_MASK affects ldso behaviour,
>> (some library search path is computed based on
>> "important" hwcap names)
>>
>> so it's probably not a good idea to reuse it for
>> ifunc dispatch tuning of the libc.
>>
>> (the exact semantics of the LD_HWCAP_MASK is not
>> clear to me though)
> 
> The dynamic linker is trying to allocate space for strings to describe
> the hwcaps and somehow setting hwcap_mask like that is causing it to
> allocated lots of memory.  It is a bug in the dynamic linker, not a
> problem with setting hwcap_mask.
> 
> My 'solution' of fixing the test case papered over the fact that the
> dynamic linker should not be barfing like this.  I'll work on this.
> This is unrelated to the patchset under discussion however since the
> problem should be visible even in 2.25.
> 
> Siddhesh
> 

That is my understanding as well, we should iron out the LD_HWCAP_MASK
for the possible flags values.  Answering your previous question,
on my system I am using vm.overcommit_kbytes=0 and setting LD_HWCAP_MASK
to 0x1 seems to make the test pass (since it won't try to preallocate lots
of memory).