Message ID | 87sgy8ppeo.fsf@oldenburg2.str.redhat.com |
---|---|
State | New |
Headers | show |
Series | Deprecate 32-bit off_t support | expand |
On 04/01/19 6:09 PM, Florian Weimer wrote: > diff --git a/NEWS b/NEWS > index cc20102fda..2f601c6217 100644 > --- a/NEWS > +++ b/NEWS > @@ -85,6 +85,15 @@ Deprecated and removed features, and other changes affecting compatibility: > as all functions that call vscanf, vfscanf, or vsscanf are annotated with > __attribute__ ((format (scanf, ...))). > > +* A future release of glibc will use a 64-bit off_t type on all Wouldn't it be clearer to fix the 'future release' to a specific timeline, say, 2.31 (to give a year+ to change offending code) so that this doesn't sound like a vague threat and there's actually a time bound for people to work with? > + architectures (as currently available with -D_FILE_OFFSET_BITS=64 on > + 32-bit architectures). Building new applications with > + -D_FILE_OFFSET_BITS=32 will no longer be supported. The off64_t type and > + the 64-bit function aliases (such as fstat64) will remain available under > + the appropriate feature test macros. In preparation, libraries should > + stop using off_t in public header files, and use off64_t (or a fixed-width > + type such as int64_t or uint64_t) instead. > + > Changes to build and runtime requirements: > > * Python 3.4 or later is required to build the GNU C Library. >
On 04/01/19 6:15 PM, Siddhesh Poyarekar wrote: > On 04/01/19 6:09 PM, Florian Weimer wrote: >> diff --git a/NEWS b/NEWS >> index cc20102fda..2f601c6217 100644 >> --- a/NEWS >> +++ b/NEWS >> @@ -85,6 +85,15 @@ Deprecated and removed features, and other changes >> affecting compatibility: >> as all functions that call vscanf, vfscanf, or vsscanf are >> annotated with >> __attribute__ ((format (scanf, ...))). >> +* A future release of glibc will use a 64-bit off_t type on all > > Wouldn't it be clearer to fix the 'future release' to a specific > timeline, say, 2.31 (to give a year+ to change offending code) so that > this doesn't sound like a vague threat and there's actually a time bound > for people to work with? Oh, and a warning under #if _FILE_OFFSET_BITS == 32 that warns about the deprecation? Siddhesh
* Siddhesh Poyarekar: > On 04/01/19 6:15 PM, Siddhesh Poyarekar wrote: >> On 04/01/19 6:09 PM, Florian Weimer wrote: >>> diff --git a/NEWS b/NEWS >>> index cc20102fda..2f601c6217 100644 >>> --- a/NEWS >>> +++ b/NEWS >>> @@ -85,6 +85,15 @@ Deprecated and removed features, and other >>> changes affecting compatibility: >>> as all functions that call vscanf, vfscanf, or vsscanf are >>> annotated with >>> __attribute__ ((format (scanf, ...))). >>> +* A future release of glibc will use a 64-bit off_t type on all >> >> Wouldn't it be clearer to fix the 'future release' to a specific >> timeline, say, 2.31 (to give a year+ to change offending code) so >> that this doesn't sound like a vague threat and there's actually a >> time bound for people to work with? 2.31 sounds okay to me. 2.30 would work for me as well. What do others think? > Oh, and a warning under #if _FILE_OFFSET_BITS == 32 that warns about > the deprecation? Due to -Werror, that alone would be rather disruptive. It's too late to add this for 2.29, too. Thanks, Florian
On 04/01/2019 12:39, Florian Weimer wrote: > diff --git a/NEWS b/NEWS > index cc20102fda..2f601c6217 100644 > --- a/NEWS > +++ b/NEWS > @@ -85,6 +85,15 @@ Deprecated and removed features, and other changes affecting compatibility: > as all functions that call vscanf, vfscanf, or vsscanf are annotated with > __attribute__ ((format (scanf, ...))). > > +* A future release of glibc will use a 64-bit off_t type on all > + architectures (as currently available with -D_FILE_OFFSET_BITS=64 on > + 32-bit architectures). Building new applications with > + -D_FILE_OFFSET_BITS=32 will no longer be supported. The off64_t type and > + the 64-bit function aliases (such as fstat64) will remain available under > + the appropriate feature test macros. In preparation, libraries should > + stop using off_t in public header files, and use off64_t (or a fixed-width please do not recommend the use of a non-standard type in place of a standard one (off64_t is not even available with default feature test macros in posix headers). instead i'd recommend the consistent use of -D_FILE_OFFSET_BITS=64 and warn #if _FILE_OFFSET_BITS==32 where appropriate. > + type such as int64_t or uint64_t) instead. > + > Changes to build and runtime requirements: > > * Python 3.4 or later is required to build the GNU C Library. >
On 04/01/2019 13:15, Szabolcs Nagy wrote: > On 04/01/2019 12:39, Florian Weimer wrote: >> diff --git a/NEWS b/NEWS >> index cc20102fda..2f601c6217 100644 >> --- a/NEWS >> +++ b/NEWS >> @@ -85,6 +85,15 @@ Deprecated and removed features, and other changes affecting compatibility: >> as all functions that call vscanf, vfscanf, or vsscanf are annotated with >> __attribute__ ((format (scanf, ...))). >> >> +* A future release of glibc will use a 64-bit off_t type on all >> + architectures (as currently available with -D_FILE_OFFSET_BITS=64 on >> + 32-bit architectures). Building new applications with >> + -D_FILE_OFFSET_BITS=32 will no longer be supported. The off64_t type and >> + the 64-bit function aliases (such as fstat64) will remain available under >> + the appropriate feature test macros. In preparation, libraries should >> + stop using off_t in public header files, and use off64_t (or a fixed-width > > please do not recommend the use of a non-standard type > in place of a standard one (off64_t is not even available > with default feature test macros in posix headers). > > instead i'd recommend the consistent use of > -D_FILE_OFFSET_BITS=64 and warn > #if _FILE_OFFSET_BITS==32 > where appropriate. we could recommend #ifndef _FILE_OFFSET_BITS _Static_assert(sizeof(off_t)>4, "32bit off_t is deprecated, on platforms where it is still supported use explicit -D_FILE_OFFSET_BITS=32."); #endif or similar when off_t is used in public apis. then first change the default, then drop support. >> + type such as int64_t or uint64_t) instead. >> + >> Changes to build and runtime requirements: >> >> * Python 3.4 or later is required to build the GNU C Library. >> >
On 04/01/19 6:28 PM, Florian Weimer wrote: > Due to -Werror, that alone would be rather disruptive. It's too late to > add this for 2.29, too. Yeah, this would have to be for 2.30. Siddhesh
On Fri, 4 Jan 2019, Siddhesh Poyarekar wrote: > Wouldn't it be clearer to fix the 'future release' to a specific timeline, > say, 2.31 (to give a year+ to change offending code) so that this doesn't > sound like a vague threat and there's actually a time bound for people to work > with? There should be one or more releases with the default changed but the old API still available with -D_FILE_OFFSET_BITS=32, before any release with _FILE_OFFSET_BITS=32 no longer available and the old ABIs turned into compat symbols. Giving concrete future release versions for a complicated change without patches in existence seems dubious. Note 1: changing the default for user programs, without changing the mode used to build glibc and its tests, is easy. Mike Frysinger posted patches in March 2014, and Paul Eggert did some investigation of the effects on the ABIs of libraries in a GNU/Linux distribution (many of which are in fact already built with _FILE_OFFSET_BITS=64). Bug 13047. Note 2: changing the mode used to build glibc itself, as required for eliminating the other mode as a supported API in the headers, is much more involved, because you need to avoid the header redirections affecting the compilation of all the functions for the 32-bit ABI, and keep all the old types and structures available in internal headers for building those functions although they'll no longer be accessible with installed headers. Note 3: changing the default mode as used for building testcases (which might or might not be the same as the default used for building glibc itself) requires fixing the associated linknamespace issues by adding new public __*64 symbol names and using them instead of *64 in the header redirections (bug 14106). Note 4: a change of default for building testcases would need to be accompanied by systematically making sure that all relevant interfaces are tested in all three cases (_FILE_OFFSET_BITS=32, _FILE_OFFSET_BITS=64 and the explicit *64 APIs). Once _FILE_OFFSET_BITS=32 is no longer supported, the corresponding ABIs would still need to be tested (using compat_symbol_reference etc.) to make sure they continue to work, using the expected structure layouts and other types which would need to move to internal headers (and care is needed to verify in some automated way that the layouts put in internal headers really do match the existing ones in installed headers in all cases). Note 5: presumably the change would also make the _FILE_OFFSET_BITS=64 C++ name mangling the default on 64-bit platforms where the C ABI doesn't change (see bug 15766), and, similarly, while both APIs are supported, -D_FILE_OFFSET_BITS=32 would result in the old mangling on 64-bit platforms (despite off_t still being 64-bit so the macro name / value being a bit misleading in that case). Note 6: much the same would apply with struct stat layout on mips64 (n64 ABI has stat and stat64 layouts different despite off_t being 64-bit by default, so the default struct stat layout would change to what's currently stat64 and _FILE_OFFSET_BITS=64 stat).
* Szabolcs Nagy: > please do not recommend the use of a non-standard type > in place of a standard one (off64_t is not even available > with default feature test macros in posix headers). What do you propose instead? Mention int64_t and uint64_t only? What do others think about this aspect? Thanks, Florian
On Mon, 7 Jan 2019, Florian Weimer wrote: > * Szabolcs Nagy: > > > please do not recommend the use of a non-standard type > > in place of a standard one (off64_t is not even available > > with default feature test macros in posix headers). > > What do you propose instead? Mention int64_t and uint64_t only? Programs and libraries should use types such as off_t, and arrange to be built with _FILE_OFFSET_BITS=64, which would make them unaffected by a future change to the default.
On 1/7/19 2:17 PM, Joseph Myers wrote: >> What do you propose instead? Mention int64_t and uint64_t only? > Programs and libraries should use types such as off_t, and arrange to be > built with _FILE_OFFSET_BITS=64, which would make them unaffected by a > future change to the default. +1. GNU applications have been doing this for nearly two decades. It's by far the best approach for free software. I suppose I should start adding "#define _TIME_BITS 64" to GNU applications, to future-proof them so that they will take advantage of 64-bit time_t on 32-bit platforms the instant that glibc starts publishing it. Are we definitely going to use this as the feature-test macro, or is it still possible that we'll use something else?
* Joseph Myers: > On Mon, 7 Jan 2019, Florian Weimer wrote: > >> * Szabolcs Nagy: >> >> > please do not recommend the use of a non-standard type >> > in place of a standard one (off64_t is not even available >> > with default feature test macros in posix headers). >> >> What do you propose instead? Mention int64_t and uint64_t only? > > Programs and libraries should use types such as off_t, and arrange to be > built with _FILE_OFFSET_BITS=64, which would make them unaffected by a > future change to the default. If this concerns a public header file, is there a preprocessor check we can recommend to add to the header file? Something that causes an #error if the ABI would not be correct? Thanks, Florian
Hi! On Fri, 2019-01-04 at 13:39:59 +0100, Florian Weimer wrote: > diff --git a/NEWS b/NEWS > index cc20102fda..2f601c6217 100644 > --- a/NEWS > +++ b/NEWS > @@ -85,6 +85,15 @@ Deprecated and removed features, and other changes affecting compatibility: > as all functions that call vscanf, vfscanf, or vsscanf are annotated with > __attribute__ ((format (scanf, ...))). > > +* A future release of glibc will use a 64-bit off_t type on all > + architectures (as currently available with -D_FILE_OFFSET_BITS=64 on > + 32-bit architectures). Building new applications with > + -D_FILE_OFFSET_BITS=32 will no longer be supported. The off64_t type and > + the 64-bit function aliases (such as fstat64) will remain available under > + the appropriate feature test macros. In preparation, libraries should > + stop using off_t in public header files, and use off64_t (or a fixed-width > + type such as int64_t or uint64_t) instead. > + > Changes to build and runtime requirements: > > * Python 3.4 or later is required to build the GNU C Library. While I'm fully behind further pushing to get software to switch to build (and work) with LFS (that's the reason I proposed the below lintian tag some time ago), it feels that something like the above might still be a bit too aggressive? :/ For example in Debian that would probably imply not being able to upgrade to such glibc until at least all shared libraries and packages supporting and implementing plugins listed at <https://lintian.debian.org/tags/binary-file-built-without-LFS-support.html> have been transitioned (with either SONAME bumps, or package renames with Breaks/Conflicts), or determined to not affect their ABI. Please notice at the top, the comment about that page only having a truncated view of the affected packages; in addition there might be other packages not being currently detected. Thanks, Guillem
* Guillem Jover: > Hi! > > On Fri, 2019-01-04 at 13:39:59 +0100, Florian Weimer wrote: >> diff --git a/NEWS b/NEWS >> index cc20102fda..2f601c6217 100644 >> --- a/NEWS >> +++ b/NEWS >> @@ -85,6 +85,15 @@ Deprecated and removed features, and other changes affecting compatibility: >> as all functions that call vscanf, vfscanf, or vsscanf are annotated with >> __attribute__ ((format (scanf, ...))). >> >> +* A future release of glibc will use a 64-bit off_t type on all >> + architectures (as currently available with -D_FILE_OFFSET_BITS=64 on >> + 32-bit architectures). Building new applications with >> + -D_FILE_OFFSET_BITS=32 will no longer be supported. The off64_t type and >> + the 64-bit function aliases (such as fstat64) will remain available under >> + the appropriate feature test macros. In preparation, libraries should >> + stop using off_t in public header files, and use off64_t (or a fixed-width >> + type such as int64_t or uint64_t) instead. >> + >> Changes to build and runtime requirements: >> >> * Python 3.4 or later is required to build the GNU C Library. > > While I'm fully behind further pushing to get software to switch to > build (and work) with LFS (that's the reason I proposed the below lintian > tag some time ago), it feels that something like the above might still be > a bit too aggressive? :/ > > For example in Debian that would probably imply not being able to > upgrade to such glibc until at least all shared libraries and packages > supporting and implementing plugins listed at > > <https://lintian.debian.org/tags/binary-file-built-without-LFS-support.html> > > have been transitioned (with either SONAME bumps, or package renames > with Breaks/Conflicts), or determined to not affect their ABI. Please > notice at the top, the comment about that page only having a truncated > view of the affected packages; in addition there might be other packages > not being currently detected. What's interesting is what is *not* in this list. As far as I can see, the non-cross binutils is missing, and so are Berkeley DB and parts/versions of of LLVM. What's common among them is that they install public header files under /usr/include which use off_t in the API. So these ABIs already assume that client code is compiled with -D_FILE_OFFSET_BITS=64. Sure, some corner cases may break when we change the default, but other things will magically start working. I think it is worth a try. Thanks, Florian
On 18/01/2019 08:09, Florian Weimer wrote: > * Guillem Jover: > >> Hi! >> >> On Fri, 2019-01-04 at 13:39:59 +0100, Florian Weimer wrote: >>> diff --git a/NEWS b/NEWS >>> index cc20102fda..2f601c6217 100644 >>> --- a/NEWS >>> +++ b/NEWS >>> @@ -85,6 +85,15 @@ Deprecated and removed features, and other changes affecting compatibility: >>> as all functions that call vscanf, vfscanf, or vsscanf are annotated with >>> __attribute__ ((format (scanf, ...))). >>> >>> +* A future release of glibc will use a 64-bit off_t type on all >>> + architectures (as currently available with -D_FILE_OFFSET_BITS=64 on >>> + 32-bit architectures). Building new applications with >>> + -D_FILE_OFFSET_BITS=32 will no longer be supported. The off64_t type and >>> + the 64-bit function aliases (such as fstat64) will remain available under >>> + the appropriate feature test macros. In preparation, libraries should >>> + stop using off_t in public header files, and use off64_t (or a fixed-width >>> + type such as int64_t or uint64_t) instead. >>> + >>> Changes to build and runtime requirements: >>> >>> * Python 3.4 or later is required to build the GNU C Library. >> >> While I'm fully behind further pushing to get software to switch to >> build (and work) with LFS (that's the reason I proposed the below lintian >> tag some time ago), it feels that something like the above might still be >> a bit too aggressive? :/ >> >> For example in Debian that would probably imply not being able to >> upgrade to such glibc until at least all shared libraries and packages >> supporting and implementing plugins listed at >> >> <https://lintian.debian.org/tags/binary-file-built-without-LFS-support.html> >> >> have been transitioned (with either SONAME bumps, or package renames >> with Breaks/Conflicts), or determined to not affect their ABI. Please >> notice at the top, the comment about that page only having a truncated >> view of the affected packages; in addition there might be other packages >> not being currently detected. > > What's interesting is what is *not* in this list. As far as I can see, > the non-cross binutils is missing, and so are Berkeley DB and > parts/versions of of LLVM. What's common among them is that they > install public header files under /usr/include which use off_t in the > API. So these ABIs already assume that client code is compiled with > -D_FILE_OFFSET_BITS=64. > > Sure, some corner cases may break when we change the default, but other > things will magically start working. I think it is worth a try. It is not clear to me if packages are being built without LFS due wrong configure/build options, lack of support from distro built system, or explicit disabling in package build (due either lack of testing or ABI limitation). Also, it is really questionable to have a system with mixed LFS support for libraries specially the one that might export off_t in its API. My impression is, for system consistency, have all libraries with explicit LFS support and handle outliers as why the program/libraries is building *without* non-LFS support (not the other way around).
diff --git a/NEWS b/NEWS index cc20102fda..2f601c6217 100644 --- a/NEWS +++ b/NEWS @@ -85,6 +85,15 @@ Deprecated and removed features, and other changes affecting compatibility: as all functions that call vscanf, vfscanf, or vsscanf are annotated with __attribute__ ((format (scanf, ...))). +* A future release of glibc will use a 64-bit off_t type on all + architectures (as currently available with -D_FILE_OFFSET_BITS=64 on + 32-bit architectures). Building new applications with + -D_FILE_OFFSET_BITS=32 will no longer be supported. The off64_t type and + the 64-bit function aliases (such as fstat64) will remain available under + the appropriate feature test macros. In preparation, libraries should + stop using off_t in public header files, and use off64_t (or a fixed-width + type such as int64_t or uint64_t) instead. + Changes to build and runtime requirements: * Python 3.4 or later is required to build the GNU C Library.