Message ID | 20210617115104.1359598-1-adhemerval.zanella@linaro.org |
---|---|
Headers | show |
Series | More y2038 fixes | expand |
Hi Adhemerval, > Hi Carlos and Lukasz, > > This patchset adds more y2038 fixes that I would like to push for > 2.34. The first two patches enable more internal y2038 usage on glibc > and also for the installed programs. Making glibc the first consumer > of these newer interfaces should improve coverage. > Those two patches were already available for review for some time. Indeed, by using 64-bit time internally we may have more testing (sooner). > The next 6 patches removes an optimization I added on some specific > syscall, where a global variable it atomically set once glibc detects > that kernel supports 64-bit time. The problem with this approach > breaks the usage case of live migration like CRIU or similar. I do recall that this optimization was to avoid performance regression on legacy systems - to avoid issuing two syscalls (64 bit one and then 32 bit one). > Also > for some interfaces that do not query information from kernel, most > usages can be optimized away by either building glibc with a minimum > 5.1 kernel or by using the 32-bit syscall for the common case. I do think that this approach (with classification of use cases based on specified kernel version) aligns with how people will build setups for those systems (at least I do it in this way for OE/Yocto). > > So the following patch implement this optimization for the interfaces > that allows it. If the provided timeout can be fit in old 32-bit > time_t, the old syscall is used instead. This optimization is only > used for !__ASSUME_TIME64_SYSCALLS, otherwise the newer 64-bit > syscall are used. Ok. > > I checked this with 3 different configation which should cover most > usages: > > - i686-linux-gnu on a 4.15 kernel and default --enable-kernel=3.2. > This uses the 32-bit optimization to avoid calling the newer > 64-bit syscall when possible. It also triggers and EOVERFLOW > when 64-bit inputs are used without proper kernel support. Ok. > - i686-linux-gnu on a 5.11 kernel and default --enable-kernel=3.2. > This will also use the 32-bit optimization, but it will always > succeded due proper kernel support. Ok. > - i686-linux-gnu on a 5.11 kernel and with --enable-kernel=5.1. > This enables __ASSUME_TIME64_SYSCALLS and only uses 64-bit > time_t syscalls. Ok. BTW: I'm wondering when the minimal, supported Linux kernel version is going to be moved forward? > > Adhemerval Zanella (18): > Use 64 bit time_t stat internally > Use LFS and 64 bit time for installed programs > support: Add support_create_timer > linux: Only use 64-bit syscall if required for ppoll > linux: Only use 64-bit syscall if required for pselect > linux: Only use 64-bit syscall if required for select > linux: Remove supports_time64 () from clock_getres > linux: Remove supports_time64 () from clock_gettime > linux: Remove time64-support > linux: timerfd_gettime minor cleanup > linux: Only use 64-bit syscall if required for semtimedop > linux: Only use 64-bit syscall if required for timerfd_settime > linux: Only use 64-bit syscall if required for mq_timedreceive > linux: Only use 64-bit syscall if required for mq_timedsend > linux: Only use 64-bit syscall if required for sigtimedwait > linux: Only use 64-bit syscall if required for utimensat family > linux: Only use 64-bit syscall if required for internal futex > linux: Only use 64-bit syscall if required for clock_nanosleep > > Makeconfig | 8 +- > Makerules | 12 +- > csu/check_fds.c | 8 +- > elf/dl-load.c | 8 +- > elf/dl-misc.c | 4 +- > elf/dl-profile.c | 4 +- > iconv/gconv_cache.c | 4 +- > include/dirent.h | 2 +- > include/file_change_detection.h | 6 +- > include/sys/select.h | 5 + > inet/rcmd.c | 6 +- > intl/loadmsgcat.c | 4 +- > io/Makefile | 4 +- > io/file_change_detection.c | 16 ++- > io/getdirname.c | 6 +- > libio/filedoalloc.c | 2 +- > libio/fileops.c | 8 +- > libio/oldfileops.c | 2 +- > libio/wfileops.c | 2 +- > locale/loadarchive.c | 8 +- > locale/loadlocale.c | 6 +- > locale/localeinfo.h | 2 +- > misc/Makefile | 11 ++ > misc/tst-pselect.c | 120 > ++++++++---------- misc/tst-select.c | > 39 +++--- nptl/futex-internal.c | 52 +++++--- > nscd/nscd_helper.c | 4 +- > nss/nss_database.c | 4 +- > rt/Makefile | 4 +- > rt/tst-mqueue10-time64.c | 1 + > rt/tst-mqueue10.c | 72 +++++++++++ > support/Makefile | 1 + > support/support.h | 11 ++ > support/support_create_timer.c | 69 ++++++++++ > sysdeps/nptl/futex-internal.h | 24 ++-- > sysdeps/posix/dl-fileid.h | 4 +- > sysdeps/posix/euidaccess.c | 4 +- > sysdeps/posix/getaddrinfo.c | 21 +-- > sysdeps/posix/getcwd.c | 15 ++- > sysdeps/posix/pathconf.c | 4 +- > sysdeps/posix/sysconf.c | 4 +- > sysdeps/posix/tempname.c | 8 +- > sysdeps/unix/sysv/linux/Makefile | 15 ++- > sysdeps/unix/sysv/linux/clock_getres.c | 16 +-- > sysdeps/unix/sysv/linux/clock_gettime.c | 14 +- > sysdeps/unix/sysv/linux/clock_nanosleep.c | 47 ++++--- > sysdeps/unix/sysv/linux/fdopendir.c | 4 +- > sysdeps/unix/sysv/linux/fexecve.c | 4 +- > .../unix/sysv/linux/microblaze/pselect32.c | 3 +- > sysdeps/unix/sysv/linux/mq_timedreceive.c | 35 ++--- > sysdeps/unix/sysv/linux/mq_timedsend.c | 35 ++--- > sysdeps/unix/sysv/linux/opendir.c | 7 +- > sysdeps/unix/sysv/linux/pathconf.c | 5 +- > sysdeps/unix/sysv/linux/ppoll.c | 41 +++--- > sysdeps/unix/sysv/linux/pselect.c | 47 ++++--- > sysdeps/unix/sysv/linux/pselect32.c | 6 - > sysdeps/unix/sysv/linux/select.c | 60 +++------ > sysdeps/unix/sysv/linux/select32.c | 58 +++++++++ > sysdeps/unix/sysv/linux/semtimedop.c | 54 ++++---- > sysdeps/unix/sysv/linux/sigtimedwait.c | 26 ++-- > sysdeps/unix/sysv/linux/time64-support.c | 23 ---- > sysdeps/unix/sysv/linux/time64-support.h | 70 ---------- > sysdeps/unix/sysv/linux/timerfd_gettime.c | 10 +- > sysdeps/unix/sysv/linux/timerfd_settime.c | 25 ++-- > sysdeps/unix/sysv/linux/tst-ppoll.c | 15 +++ > sysdeps/unix/sysv/linux/tst-sigtimedwait.c | 18 +++ > sysdeps/unix/sysv/linux/tst-timerfd.c | 29 ++++- > sysdeps/unix/sysv/linux/ttyname.h | 10 +- > sysdeps/unix/sysv/linux/ttyname_r.c | 16 +-- > sysdeps/unix/sysv/linux/utimensat.c | 35 +++-- > sysvipc/Makefile | 9 ++ > sysvipc/ftok.c | 4 +- > sysvipc/test-sysvsem.c | 22 +++- > time/Makefile | 9 ++ > time/tst-clock_nanosleep.c | 40 +++--- > time/tzfile.c | 6 +- > 76 files changed, 851 insertions(+), 566 deletions(-) > create mode 100644 rt/tst-mqueue10-time64.c > create mode 100644 rt/tst-mqueue10.c > create mode 100644 support/support_create_timer.c > create mode 100644 sysdeps/unix/sysv/linux/select32.c > delete mode 100644 sysdeps/unix/sysv/linux/time64-support.c > delete mode 100644 sysdeps/unix/sysv/linux/time64-support.h > Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de
On 17/06/2021 11:20, Lukasz Majewski wrote: > Hi Adhemerval, > >> Hi Carlos and Lukasz, >> >> This patchset adds more y2038 fixes that I would like to push for >> 2.34. The first two patches enable more internal y2038 usage on glibc >> and also for the installed programs. Making glibc the first consumer >> of these newer interfaces should improve coverage. >> > > Those two patches were already available for review for some time. > Indeed, by using 64-bit time internally we may have more testing > (sooner). Yes, but I changed slight the one that enabled the y2038 symbol on installed program to use apply the CFLAGS on module, instead of setting in each sub Makefile. > >> The next 6 patches removes an optimization I added on some specific >> syscall, where a global variable it atomically set once glibc detects >> that kernel supports 64-bit time. The problem with this approach >> breaks the usage case of live migration like CRIU or similar. > > I do recall that this optimization was to avoid performance regression > on legacy systems - to avoid issuing two syscalls (64 bit one and then > 32 bit one). Yes, but as I noted some interfaces don't really need to use the newer syscall in the compatibility mode (where glibc is build to support older kernels). This optimization is really tricky, since it incurs in some data races (I think to fully avoid it would require seq-cst atomic instead of relaxed ones) and it might break live migration. > >> Also >> for some interfaces that do not query information from kernel, most >> usages can be optimized away by either building glibc with a minimum >> 5.1 kernel or by using the 32-bit syscall for the common case. > > I do think that this approach (with classification of use cases based > on specified kernel version) aligns with how people will build setups > for those systems (at least I do it in this way for OE/Yocto). I think there will be specific cases where kernel update is not an option, but I think these will be really niches. > >> >> So the following patch implement this optimization for the interfaces >> that allows it. If the provided timeout can be fit in old 32-bit >> time_t, the old syscall is used instead. This optimization is only >> used for !__ASSUME_TIME64_SYSCALLS, otherwise the newer 64-bit >> syscall are used. > > Ok. > >> >> I checked this with 3 different configation which should cover most >> usages: >> >> - i686-linux-gnu on a 4.15 kernel and default --enable-kernel=3.2. >> This uses the 32-bit optimization to avoid calling the newer >> 64-bit syscall when possible. It also triggers and EOVERFLOW >> when 64-bit inputs are used without proper kernel support. > > Ok. > >> - i686-linux-gnu on a 5.11 kernel and default --enable-kernel=3.2. >> This will also use the 32-bit optimization, but it will always >> succeded due proper kernel support. > > Ok. > >> - i686-linux-gnu on a 5.11 kernel and with --enable-kernel=5.1. >> This enables __ASSUME_TIME64_SYSCALLS and only uses 64-bit >> time_t syscalls. > > Ok. > > BTW: I'm wondering when the minimal, supported Linux kernel version is > going to be moved forward? We usually follow the minimum LTS supported by Linux kernel developers.
On Thu, 17 Jun 2021, Adhemerval Zanella via Libc-alpha wrote: > > BTW: I'm wondering when the minimal, supported Linux kernel version is > > going to be moved forward? > > We usually follow the minimum LTS supported by Linux kernel developers. However, updates are usually only done when they actually allow significant cleanups in glibc. And given there isn't much benefit on x86_64 to increasing the minimum version, it seems like a good idea to first implement Carlos's proposal to remove the error at glibc startup for an old kernel version number (so people trying to use new glibc in containers on old kernels don't automatically get everything failing at startup, and may well have things work especially if a few syscalls have been backported to the old kernel). Once Carlos's proposal is implemented, a 4.4 minimum *would* allow significant cleanups, such as removing most of the socketcall support. But it would be important to *check* the kernel support in 4.4 for each feature carefully before assuming it to be present globally. (For example, when removing the socketcall support, we can't do so for getpeername or getsockname until the minimum kernel version is 4.20 or later (so not until 2025 based on the current EOL dates at <https://www.kernel.org/category/releases.html>), because that's when those syscalls were added to the compat syscall table for 32-bit SPARC programs running on 64-bit kernels.)