| Message ID | cover.1592836143.git.fweimer@redhat.com |
|---|---|
| Headers | show |
| Series | RFC: elf: glibc-hwcaps support | expand |
On Mon, Jun 22, 2020 at 8:13 AM Florian Weimer via Libc-alpha <libc-alpha@sourceware.org> wrote: > With these changes, on a current x86-64 machine (with AVX2-level CPU > features), you can drop a shared object into the directory > /usr/lib64/glibc-hwcaps/x86-102 This appears to violate the Filesystem Hierarchy Standard (FHS) https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html which states that libraries can only be in /usr/lib or /usr/lib<qual>. And it is implied but not clearly stated that <qual> is not allowed to contain a slash. There is an exception for applications that can have a dir under /usr/lib or /usr/lib<qual>, but glibc isn't an application. Unless maybe you want to claim that ld.so is an application and these are ld.so specific files? But then all libraries used by ld.so must be in there which is not true. I'm not sure if anyone cares about strict compliance with FHS though. Debian (and hence Ubuntu) deliberately violate it with their multiarch scheme. RISC-V violates it with our two level scheme for word size and abi, e.g. /usr/lib64/lp64d for 64-bit hard-float libraries. Though one person did care enough to report this as a bug. I sent email to the fhs-discuss mailing list to ask about this, but never got a reply. The email I sent in April is the last email on the mailing list. Anyways, I don't care about FHS personally, I just care that I don't need to change the RISC-V ABI. If this scheme is OK then maybe the RISC-V scheme is OK too? Jim
* Jim Wilson: > On Mon, Jun 22, 2020 at 8:13 AM Florian Weimer via Libc-alpha > <libc-alpha@sourceware.org> wrote: >> With these changes, on a current x86-64 machine (with AVX2-level CPU >> features), you can drop a shared object into the directory >> /usr/lib64/glibc-hwcaps/x86-102 > > This appears to violate the Filesystem Hierarchy Standard (FHS) > https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html > which states that libraries can only be in /usr/lib or /usr/lib<qual>. > And it is implied but not clearly stated that <qual> is not allowed to > contain a slash. There is an exception for applications that can have > a dir under /usr/lib or /usr/lib<qual>, but glibc isn't an > application. Unless maybe you want to claim that ld.so is an > application and these are ld.so specific files? But then all > libraries used by ld.so must be in there which is not true. I thought about this. For use with the library path, it has to be a subdirectory not a sibling directory, otherwise the behavior would be really surprising. In the end, it's no different from the "tls" directory we already search (along with a bunch of others). I posted a list of the subdirectories searched on s390x: <https://sourceware.org/pipermail/libc-alpha/2020-May/113757.html> At least the "glibc-hwcaps" name is architecture-independent, and we can add it to FHS if we wanted. Thanks, Florian
This series implements searching for alternative library implementations under glibc-hwcaps subdirectories on the library search path. It's lacking tests, documentation, and a NEWS file update, but I wanted to submit it early for discussion (and perhaps review of some of the cleanup patches). For background, it is best reference first the discussion about the current hwcaps subdirectory processing: hwcaps subdirectory selection in the dynamic loader <https://sourceware.org/pipermail/libc-alpha/2020-May/113757.html> The new approach uses a special subdirectory, called "glibc-hwcaps", which contains nested subdirectories with all the alternative library implementations. The glibc-hwcaps subdirectory can be present in every subdirectory of the library search path. ldconfig recognizes the "glibc-hwcaps" name, and it receives special treatment if it exists: subdirectories of the directory (but the directory itself) are scanned for libraries. If a library is found, it is put into the cache, and associated with the name of the subdirectory (of glibc-hwcaps). ldconfig does this for all subdirectories (whose name does not begin with '.'), so it does not have to be updated if new subdirectories are added. There is currently no symbolic link processing for these subdirectories. In the future, they might store multiple implementations with different GNU property notes, and we might want to select those through the cache as well. (I'm open to changing this part to align more with traditional cache processing.) At run time, the dynamic loader has a built-in list of HWCAP subdirectory names (which are different from the legacy HWCAP directories). The new “ld.so --help” option shows these subdirectory names. For LD_LIBRARY_PATH processing, ld.so simply attempts to open libraries under their soname in the active/supported glibc-hwcaps subdirectories. (This part is quite similar to the original HWCAP subdirectory handling, except that the new subdirectories do not nest.) For ld.so.cache processing, ld.so sorts the active/supported HWCAP subdirectory names and merges this list against the list of subdirectory names as stored in a new ld.so.cache extension section. Each glibc-hwcaps library listed in the cache has a HWCAP field that contains an index into the HWCAP subdirectory array of the cache file. (A previously unused HWCAP bit is set so that these entries are ignored by existing dynamic linkers.) The dynamic linker loads the library with has the lowest positive preference value (essentially from the subdirectory that comes first in its consolidated HWCAP name list). There are new ld.so options, --glibc-hwcaps-prepend and --glibc-hwcaps-mask, to change the list of glibc-hwcaps subdirectories searched. Due to the way ldconfig has been modified, it is possible to pick up completely new subdirectories (not known to the implementation) using --glibc-hwcaps-prepend even if ld.so.cache is used. With these changes, on a current x86-64 machine (with AVX2-level CPU features), you can drop a shared object into the directory /usr/lib64/glibc-hwcaps/x86-102 and the dynamic linker will load it, either via ld.so.cache or LD_LIBRARY_PATH processing. Under strace, it look like this if the cache is used: | openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 | openat(AT_FDCWD, "/lib64/glibc-hwcaps/x86-102/libz.so.1", O_RDONLY|O_CLOEXEC) = 3 Diagnostics output from “ld.so --help” for this machine is this: | Shared library search path: | (libraries located via /etc/ld.so.cache) | /lib64 (system search path) | /usr/lib64 (system search path) | | Subdirectories of glibc-hwcaps directories, in priority order: | x86-103 | x86-102 (supported, searched) | x86-101 (supported, searched) | x86-100 (supported, searched) | | Legacy HWCAP subdirectories under library search path directories: | haswell (AT_PLATFORM) | tls (supported, searched) | avx512_1 | x86_64 (supported, searched) I've also included support for a "power9" subdirectory on powerpc64le, some subdirectories on s390x, and "atomics" on aarch64. The exact definitions of the directories are still subject to change. They are there to that there is at least something new to be searched. For the x86-64 part, I'm going to start a separate, cross-project discussion about the desired CPU feature flags for each micro-architecture baseline level. The series does not yet check if the glibc-hwcaps directory itself exists. If the directory does not exist, we can skip searching all its subdirectories. This is another motivation for having the magic glibc-hwcaps subdirectory. But I have not implemented this yet because it tends to hide the search activity from strace. Patches up to (but excluding) “elf: Add glibc-hwcaps support for LD_LIBRARY_PATH” are more or less cleanups, and so are these patches: elf: Unify old and new format cache handling code in ld.so elf: Implement a string table for ldconfig, with tail merging elf: Implement tail merging of strings in ldconfig elf: In ldconfig, extract the new_sub_entry function from search_dir These cleanups could be reviewed and installed separately. Thanks, Florian Florian Weimer (30): elf: Include <stdbool.h> in <dl-tunables.h> because bool is used elf: Include <stddef.h> (for size_t), <sys/stat.h> in <ldconfig.h> elf: Do not search HWCAP subdirectories in statically linked binaries elf: Implement __rtld_malloc_is_full elf: Implement _dl_write elf: Extract command-line/environment variables state from rtld.c elf: Move ld.so error/help output to _dl_usage elf: Record whether paths come from LD_LIBRARY_PATH or --library-path elf: Implement ld.so --help elf: Implement ld.so --version scripts/update-copyrights: Update csu/version.c, elf/dl-usage.c elf: Use the term "program interpreter" in the ld.so help message elf: Print the full name of the dynamic loader in the ld.so help message elf: Make __rtld_env_path_list and __rtld_search_dirs global variables elf: Add library search path information to ld.so --help elf: Enhance ld.so --help to print HWCAP subdirectories elf: Do not pass GLRO(dl_platform), GLRO(dl_platformlen) to _dl_important_hwcaps elf: Add glibc-hwcaps support for LD_LIBRARY_PATH x86_64: Add glibc-hwcaps support powerpc64le: Add glibc-hwcaps support s390x: Add Add glibc-hwcaps support aarch64: Add glibc-hwcaps support elf: Add endianness markup to ld.so.cache elf: Add extension mechanism to ld.so.cache elf: Unify old and new format cache handling code in ld.so elf: Implement a string table for ldconfig, with tail merging elf: Implement tail merging of strings in ldconfig elf: In ldconfig, extract the new_sub_entry function from search_dir elf: Process glibc-hwcaps subdirectories in ldconfig elf: Add glibc-hwcaps subdirectory support to ld.so cache processing NEWS | 4 + elf/Makefile | 22 +- elf/cache.c | 400 +++++++++++++-- elf/dl-cache.c | 483 ++++++++++++------ elf/dl-hwcaps-subdirs.c | 29 ++ elf/dl-hwcaps.c | 225 +++++++- elf/dl-hwcaps.h | 102 ++++ elf/dl-hwcaps_split.c | 77 +++ elf/dl-load.c | 75 +-- elf/dl-main.h | 120 +++++ elf/dl-minimal.c | 8 + elf/dl-support.c | 5 +- elf/dl-tunables.h | 2 + elf/dl-usage.c | 267 ++++++++++ elf/dl-write.c | 56 ++ elf/ldconfig.c | 187 +++++-- elf/rtld.c | 248 ++++----- elf/stringtable.c | 201 ++++++++ elf/stringtable.h | 61 +++ elf/stringtable_free.c | 32 ++ elf/tst-dl-hwcaps_split.c | 139 +++++ elf/tst-stringtable.c | 140 +++++ include/link.h | 4 + include/rtld-malloc.h | 4 + scripts/update-copyrights | 6 + sysdeps/aarch64/dl-hwcaps-subdirs.c | 31 ++ sysdeps/generic/dl-cache.h | 232 ++++++++- sysdeps/generic/ldconfig.h | 20 +- sysdeps/generic/ldsodefs.h | 34 +- .../powerpc/powerpc64/le/dl-hwcaps-subdirs.c | 31 ++ sysdeps/s390/s390-64/dl-hwcaps-subdirs.c | 54 ++ sysdeps/unix/sysv/linux/dl-write.c | 30 ++ sysdeps/x86_64/dl-hwcaps-subdirs.c | 73 +++ 33 files changed, 2980 insertions(+), 422 deletions(-) create mode 100644 elf/dl-hwcaps-subdirs.c create mode 100644 elf/dl-hwcaps_split.c create mode 100644 elf/dl-main.h create mode 100644 elf/dl-usage.c create mode 100644 elf/dl-write.c create mode 100644 elf/stringtable.c create mode 100644 elf/stringtable.h create mode 100644 elf/stringtable_free.c create mode 100644 elf/tst-dl-hwcaps_split.c create mode 100644 elf/tst-stringtable.c create mode 100644 sysdeps/aarch64/dl-hwcaps-subdirs.c create mode 100644 sysdeps/powerpc/powerpc64/le/dl-hwcaps-subdirs.c create mode 100644 sysdeps/s390/s390-64/dl-hwcaps-subdirs.c create mode 100644 sysdeps/unix/sysv/linux/dl-write.c create mode 100644 sysdeps/x86_64/dl-hwcaps-subdirs.c