[v3] Implement strlcpy [BZ #178]

This version is v2, rebased to current master, with a change to the
documentation to refer to the appropriate place in the GNU Coding Standards.

The reasons why I want to add this to glibc are unchanged.  We can do
much better than the existing implementations:  This version is based on
optimized strlen and memcpy implementations (the defaults, not the
optimized versions).  It has fortify support.  In fortify mode, there
are link-time warnings for common misuses.

Commit message:

This change may cause application code not to compile anymore, but
strlcpy is in the implementation namespace (per C11 7.31.13), so this
is an application issue.

Florian

I support the addition of this API as sufficiently widely used in 
practice, but have not reviewed the substance of the patch.

> diff --git a/debug/strlcpy_chk.c b/debug/strlcpy_chk.c
> new file mode 100644
> index 0000000..038d8f9
> --- /dev/null
> +++ b/debug/strlcpy_chk.c
> @@ -0,0 +1,29 @@
> +/* Copyright (C) 2015 Free Software Foundation, Inc.

The first line of any new file should be a descriptive comment, before the 
copyright notice.

> diff --git a/string/strlcpy.c b/string/strlcpy.c
> new file mode 100644
> index 0000000..9b39d86
> --- /dev/null
> +++ b/string/strlcpy.c
> @@ -0,0 +1,46 @@
> +/* Copyright (C) 2015 Free Software Foundation, Inc.

Likewise.

> +size_t
> +strlcpy(char *__restrict dest, const char *__restrict src, size_t size)

Missing space before '('.

> diff --git a/string/tst-strlcpy.c b/string/tst-strlcpy.c
> new file mode 100644
> index 0000000..016411b
> --- /dev/null
> +++ b/string/tst-strlcpy.c

Missing descriptive comment.

I'd rather we didn't add strlcpy to glibc, for reasons already 
discussed. If consensus goes against me, the patch still needs some 
work. I don't recall the details of our previous discussion; at the risk 
of reraising old issues here are some comments:

* If strlen(SRC) < DESTLEN, the documentation should clearly state that 
the contents of the bytes DEST[strlen(SRC) + 1] through DEST[DESTLEN - 
1] are preserved.  The current documentation can be plausibly read that 
way, but it's not explicit.

* The proposed documentation can easily be misread as implying that 
strlcpy (DEST, SRC, DESTLEN) does O(DESTLEN) work, which is incorrect: 
strlcpy always does O(strlen(SRC)) work.  This point should be made 
clearly.  This is not merely a performance issue: it should be made 
crystal-clear that SRC must be null-terminated even if the source's 
trailing null byte is way after the bytes that strlcpy copies to DEST.

* strlcpy's name should be prefixed by '__' by default. The names 
strlcpy and strlcpy_chk are both in the implementation namespace, and 
it's odd to have one without leading underscores and the other with 
them. I suggest a more cautious approach, in which both names are 
prefixed with '__' and unprefixed strlcpy is provided to the user only 
when GNU or BSD extensions are requested via _GNU_SOURCE etc. This is 
less likely to break existing applications. It's true that POSIX and the 
C standard allow glibc to add str* names of all sorts, but it doesn't 
hurt to be realistic here about the user code that's undoubtedly out there.

* strlcpy's implementation should use memmove instead of memcpy. The 
main motivations for strlcpy are safety and consistency and avoiding 
errors.  memmove obviously supports these goals better than memcpy 
does.  Efficiency is not a major concern with strlcpy (if it were, 
strlcpy wouldn't be O(strlen(SRC))).

> +Not guaranteeing null termination and always overwriting the entire
> +destination buffer makes @code{strncpy} rarely useful, but this behavior
> +is specified by the @w{ISO C} standard.  See @code{strlcpy} below for an
> +alternative.
This quote is confusing, as it imples that strlcpy guarantees null 
termination, which strlcpy does not. I suggest rewording it to something 
like the following: "Often @code{strncpy} is not what you want, because 
it does not null-terminate the destination if the destination is smaller 
than the source, it always overwrites the entire destination buffer, it 
may truncate the destination, and it has undefined behavior if the 
source and destination overlap.  For alternatives, see the documentation 
below for @code{strlcpy}."

We can pair this with similar phrasing under strlcpy -- something like 
the following perhaps (this wording assumes strlcpy is changed to use 
memmove):"Often @code{strlcpy} is not what you want, because it does not 
null-terminate the destination if the destination's size is zero,it can 
leave junk data behind in the destination, it can do useless work when 
the source is long and the destination short, and it may truncate the 
destination.  Although one alternative is @code{strncpy}, it is usually 
better to use dynamic memory allocation and functions such as 
@code{strdup} or @code{asprintf} to construct strings."

On 10/29/2015 02:50 PM, Paul Eggert wrote:
> We can pair this with similar phrasing under strlcpy -- something like 
> the following perhaps (this wording assumes strlcpy is changed to use 
> memmove):"Often @code{strlcpy} is not what you want, because it does 
> not null-terminate the destination if the destination's size is 
> zero,it can leave junk data behind in the destination, it can do 
> useless work when the source is long and the destination short, and it 
> may truncate the destination. Although one alternative is 
> @code{strncpy}, it is usually better to use dynamic memory allocation 
> and functions such as @code{strdup} or @code{asprintf} to construct 
> strings." 

One more thing: we should add "it requires the source to be 
null-terminated even when it is longer than the destination".

On Thu, Oct 29, 2015 at 02:50:54PM -0700, Paul Eggert wrote:
> I'd rather we didn't add strlcpy to glibc, for reasons already
> discussed. If consensus goes against me, the patch still needs some
> work. I don't recall the details of our previous discussion; at the
> risk of reraising old issues here are some comments:
> 
> * If strlen(SRC) < DESTLEN, the documentation should clearly state
> that the contents of the bytes DEST[strlen(SRC) + 1] through
> DEST[DESTLEN - 1] are preserved.  The current documentation can be
> plausibly read that way, but it's not explicit.

I don't think this needs to be explicit. There is a general principle,
which needs to be promoted rather than undermined, that a function
cannot have side effects/alter the value of objects outside of its
documented behavior.

> * The proposed documentation can easily be misread as implying that
> strlcpy (DEST, SRC, DESTLEN) does O(DESTLEN) work, which is
> incorrect: strlcpy always does O(strlen(SRC)) work.  This point
> should be made clearly.  This is not merely a performance issue: it
> should be made crystal-clear that SRC must be null-terminated even
> if the source's trailing null byte is way after the bytes that
> strlcpy copies to DEST.

I'm not opposed to making a more visible warning, but the
documentation of the return value should make it clear. Requiring the
argument to point to "a string" also expresses the null termination
requirement. This is the same language used in the C standard for
string functions.

> * strlcpy's name should be prefixed by '__' by default. The names
> strlcpy and strlcpy_chk are both in the implementation namespace,
> and it's odd to have one without leading underscores and the other
> with them. I suggest a more cautious approach, in which both names
> are prefixed with '__' and unprefixed strlcpy is provided to the
> user only when GNU or BSD extensions are requested via _GNU_SOURCE
> etc. This is less likely to break existing applications. It's true
> that POSIX and the C standard allow glibc to add str* names of all
> sorts, but it doesn't hurt to be realistic here about the user code
> that's undoubtedly out there.

The function should only be exposed at all when extensions are exposed
(_DEFAULT_SOURCE/_ALL_SOURCE) but __-prefixed functions should never
be a public API intended for applications to use. Also exposing the
function with a name different from widespread usage would not help
application portability.

> * strlcpy's implementation should use memmove instead of memcpy. The
> main motivations for strlcpy are safety and consistency and avoiding
> errors.  memmove obviously supports these goals better than memcpy
> does.  Efficiency is not a major concern with strlcpy (if it were,
> strlcpy wouldn't be O(strlen(SRC))).

No, the fortify version should just abort when this contract is
broken. Invalid usage contrary to documentation should not be silently
supported. This encourages non-portable programs which would break on
other strl* implementations.

Rich

Rich Felker wrote:

> There is a general principle,
> which needs to be promoted rather than undermined, that a function
> cannot have side effects/alter the value of objects outside of its
> documented behavior.

Regardless of whether that is an (undocumented?) general principle, this is a 
user manual not a formal standard.  In this particular context it's helpful to 
describe strlcpy's behavior carefully, since it's explained right after 
strncpy's and the reader might otherwise easily get confused into the 
misimpression that strlcpy behaves like strncpy except for guaranteeing 
null-termination when DESTLEN is nonzero.

> Requiring the
> argument to point to "a string" also expresses the null termination
> requirement. This is the same language used in the C standard for
> string functions.

Again, the point of the documentation is to be clear and useful, and it's 
helpful to spell things out given the strncpy context.  (Besides, this is a 
manual that contains phrases like "it is more efficient and works even if the 
string @var{s} is not null-terminated", so it's a bit much to expect the reader 
to know that "string" always means "null-terminated". :-)

> __-prefixed functions should never
> be a public API intended for applications to use.

Yes, that's fine.  My point was about the non-'__'-prefixed name, where it 
appears we agree.

> No, the fortify version should just abort when this contract is
> broken.

The idea was that this contract should be memmove-based, not memcpy-based, which 
means the fortify version would not abort if the arguments overlap. Still, I see 
your point: if we wanted move semantics, we'd have called the function 
'strlmove' or something like that. So I withdraw the suggestion to change the 
contract, and instead suggest that we add overlapping strings to our list of 
when not to use strlcpy, e.g,:

Often @code{strlcpy} is not what you want, because it does not null-terminate 
the destination if the destination's size is zero, it can leave junk data behind 
in the destination, it can do useless work when the source is long and the 
destination short, it requires the source to be null-terminated even when it is 
longer than the destination, it can crash if the source and destination overlap, 
and it may truncate the destination.  Although one alternative is 
@code{strncpy}, it is usually better to use dynamic memory allocation and 
functions such as @code{strdup} or @code{asprintf} to construct strings.


All in all, I'd still rather not add this poorly-designed API to the library.

On Sat, Oct 31, 2015 at 03:37:10AM -0700, Paul Eggert wrote:
> Rich Felker wrote:
> 
> >There is a general principle,
> >which needs to be promoted rather than undermined, that a function
> >cannot have side effects/alter the value of objects outside of its
> >documented behavior.
> 
> Regardless of whether that is an (undocumented?) general principle,
> this is a user manual not a formal standard.  In this particular
> context it's helpful to describe strlcpy's behavior carefully, since
> it's explained right after strncpy's and the reader might otherwise
> easily get confused into the misimpression that strlcpy behaves like
> strncpy except for guaranteeing null-termination when DESTLEN is
> nonzero.

OK, I can agree with this.

> >Requiring the
> >argument to point to "a string" also expresses the null termination
> >requirement. This is the same language used in the C standard for
> >string functions.
> 
> Again, the point of the documentation is to be clear and useful, and
> it's helpful to spell things out given the strncpy context.
> (Besides, this is a manual that contains phrases like "it is more
> efficient and works even if the string @var{s} is not
> null-terminated", so it's a bit much to expect the reader to know
> that "string" always means "null-terminated". :-)

Likewise ok.

> >__-prefixed functions should never
> >be a public API intended for applications to use.
> 
> Yes, that's fine.  My point was about the non-'__'-prefixed name,
> where it appears we agree.
> 
> >No, the fortify version should just abort when this contract is
> >broken.
> 
> The idea was that this contract should be memmove-based, not

glibc's role here is not to design the contract but to implement the
existing contract precisely. This is because the whole goal of
providing these functions in glibc is to reduce subtle errors from
people reimplementing fallbacks in slightly-incompatible ways, not to
design a new API that we want to recommend people use.

> All in all, I'd still rather not add this poorly-designed API to the library.

Noted.

Rich

I still concur with Paul that we should not have this API in libc.

On Thu, Nov 05, 2015 at 01:11:04PM -0800, Roland McGrath wrote:
> I still concur with Paul that we should not have this API in libc.

I think we're in agreement that it's not a "good API", but nonetheless
I see strong reasons for including them and no non-ideological reasons
not to. These have been covered in detail before so I won't repeat
them again unless you really want me to. Do you disagree with my
reasoning here?

Rich

On 06-11-2015 02:01, Rich Felker wrote:
> On Thu, Nov 05, 2015 at 01:11:04PM -0800, Roland McGrath wrote:
>> I still concur with Paul that we should not have this API in libc.
> 
> I think we're in agreement that it's not a "good API", but nonetheless
> I see strong reasons for including them and no non-ideological reasons
> not to. These have been covered in detail before so I won't repeat
> them again unless you really want me to. Do you disagree with my
> reasoning here?
> 
> Rich
> 

Roland, I tend to agree with Paul also, but nonetheless any reasoning 
against this API is not refraining developers to continue reimplement 
and use this set of functions.

On 11/06/2015 01:09 PM, Adhemerval Zanella wrote:
> 
> 
> On 06-11-2015 02:01, Rich Felker wrote:
>> On Thu, Nov 05, 2015 at 01:11:04PM -0800, Roland McGrath wrote:
>>> I still concur with Paul that we should not have this API in libc.
>>
>> I think we're in agreement that it's not a "good API", but nonetheless
>> I see strong reasons for including them and no non-ideological reasons
>> not to. These have been covered in detail before so I won't repeat
>> them again unless you really want me to. Do you disagree with my
>> reasoning here?
>>
>> Rich
>>
> 
> Roland, I tend to agree with Paul also, but nonetheless any reasoning 
> against this API is not refraining developers to continue reimplement 
> and use this set of functions.

My understanding is that there are several arguments for adding this API
(e.g., widely used in existing open source projects, performance,
specific existing use cases, portability) and against (e.g., encourages
possible security issues, additional maintenance,
similar-to-existing-API-but-yet-different-approach).

As a mere user, I have never needed strlcpy myself. Given that this
function is quite easy to realize, I see no reason to include such
a function in glibc. Also, I see only little value in improving the
portability with BSDs beyond POSIX, really. Thus, the real value of
adding this function appears to be convenience reasons. Anyway, this
is my personal view. As Rich stated, the arguments have been on the
table several times and it is not my intent to restart the discussion.

Instead, my proposal is to add this function to glibc, but in a separate library.
Also, I would like to see a comment like "Do not use this function unless
you really know what you do. Consider strncpy instead." in the documentation
(and man pages, eventually).

Thoughts?

On Sun, Nov 08, 2015 at 01:48:38AM +0100, Tolga Dalman wrote:
> As a mere user, I have never needed strlcpy myself. Given that this
> function is quite easy to realize, I see no reason to include such
> a function in glibc.

Unfortunately that's NOT a given. Most reimplementations from scratch
have at least one serious corner-case bug. And if you have more than
one in the same program (e.g. multiple shared libraries that each
implement their own) they all end up using the copy from whichever
library was linked first, which might have a corner-case bug that
affects the other users. This is the main compelling security reason
for having the functions in libc: to get programs/libraries to stop
providing their own likely-buggy versions.

> Also, I see only little value in improving the
> portability with BSDs beyond POSIX, really. Thus, the real value of
> adding this function appears to be convenience reasons. Anyway, this
> is my personal view. As Rich stated, the arguments have been on the
> table several times and it is not my intent to restart the discussion.
> 
> Instead, my proposal is to add this function to glibc, but in a separate library.

That is utterly useless and much more costly. Each additional library
linked adds at least 4k of (non-shareable) memory usage to a process
and increases the start time by tens or hundreds of microseconds. But
most importantlu, if they're not available in the default link,
configure scripts will never find/use them, and they'll just end up
enabling their own replacements.

Anyway, in general, if adding a function to glibc is a bad idea,
adding it in its own separate .so file shipped with glibc is an even
worse idea. There are no advantages and lots of disadvantages.

Rich

On 11/08/2015 02:07 AM, Rich Felker wrote:
> On Sun, Nov 08, 2015 at 01:48:38AM +0100, Tolga Dalman wrote:
>> As a mere user, I have never needed strlcpy myself. Given that this
>> function is quite easy to realize, I see no reason to include such
>> a function in glibc.
> 
> Unfortunately that's NOT a given. Most reimplementations from scratch
> have at least one serious corner-case bug. And if you have more than
> one in the same program (e.g. multiple shared libraries that each
> implement their own) they all end up using the copy from whichever
> library was linked first, which might have a corner-case bug that
> affects the other users. This is the main compelling security reason
> for having the functions in libc: to get programs/libraries to stop
> providing their own likely-buggy versions.

It is difficult to discuss about this topic in an abstract manner.
I believe that most uses of strlcpy and strlcat are really unjustified
as they could be easily replaced by strncat/strcpy. As I wrote above,
this is just my personal (perhaps naïve) view.
Can you send me a list of concrete examples where the specific features
of these functions are necessary (e.g., the memory beyond \0 must not
be written)?
If performance is really the reason, I would like to review some
benchmarks before continuing the discussion at that point.

>> Also, I see only little value in improving the
>> portability with BSDs beyond POSIX, really. Thus, the real value of
>> adding this function appears to be convenience reasons. Anyway, this
>> is my personal view. As Rich stated, the arguments have been on the
>> table several times and it is not my intent to restart the discussion.
>>
>> Instead, my proposal is to add this function to glibc, but in a separate library.
> 
> That is utterly useless and much more costly. Each additional library
> linked adds at least 4k of (non-shareable) memory usage to a process
> and increases the start time by tens or hundreds of microseconds. But
> most importantlu, if they're not available in the default link,
> configure scripts will never find/use them, and they'll just end up
> enabling their own replacements.
> 
> Anyway, in general, if adding a function to glibc is a bad idea,
> adding it in its own separate .so file shipped with glibc is an even
> worse idea. There are no advantages and lots of disadvantages.

I am not talking about a libstrlcpy but instead something like libbsd_ext.
Again, we are talking quite abstractly about applications. Do you have
a list of open source applications that use strlcpy or strlcat?

Anyway, I note that you are quick to argue against my idea. What is your
proposal to resolve the issue in glibc?

On Sun, Nov 08, 2015 at 02:41:27AM +0100, Tolga Dalman wrote:
> On 11/08/2015 02:07 AM, Rich Felker wrote:
> > On Sun, Nov 08, 2015 at 01:48:38AM +0100, Tolga Dalman wrote:
> >> As a mere user, I have never needed strlcpy myself. Given that this
> >> function is quite easy to realize, I see no reason to include such
> >> a function in glibc.
> > 
> > Unfortunately that's NOT a given. Most reimplementations from scratch
> > have at least one serious corner-case bug. And if you have more than
> > one in the same program (e.g. multiple shared libraries that each
> > implement their own) they all end up using the copy from whichever
> > library was linked first, which might have a corner-case bug that
> > affects the other users. This is the main compelling security reason
> > for having the functions in libc: to get programs/libraries to stop
> > providing their own likely-buggy versions.
> 
> It is difficult to discuss about this topic in an abstract manner.
> I believe that most uses of strlcpy and strlcat are really unjustified
> as they could be easily replaced by strncat/strcpy. As I wrote above,

This is blatently wrong. strncpy is never a suitable repacement for
strlcpy. Its usage case is completely different: fixed-field-width,
non-null-terminated strings. These are largely obsolete (mainly used
in legacy on-disk data), and so is strncpy.

> this is just my personal (perhaps naïve) view.
> Can you send me a list of concrete examples where the specific features
> of these functions are necessary (e.g., the memory beyond \0 must not
> be written)?

That's not the main problem. Lack of null termination is a much bigger
problem. The biggest problem is that nobody understands it or knows
how to use it safely.

> If performance is really the reason, I would like to review some
> benchmarks before continuing the discussion at that point.

There were historically a number of programs which had really bad
performance because they were misusing strncpy for this purpose, with
very large buffers. I recall participating in the bug tracker tickets
for them, but I don't have links handy now. This was probably more
than a decade ago; people have mostly stopped the practice.

> >> Also, I see only little value in improving the
> >> portability with BSDs beyond POSIX, really. Thus, the real value of
> >> adding this function appears to be convenience reasons. Anyway, this
> >> is my personal view. As Rich stated, the arguments have been on the
> >> table several times and it is not my intent to restart the discussion.
> >>
> >> Instead, my proposal is to add this function to glibc, but in a separate library.
> > 
> > That is utterly useless and much more costly. Each additional library
> > linked adds at least 4k of (non-shareable) memory usage to a process
> > and increases the start time by tens or hundreds of microseconds. But
> > most importantlu, if they're not available in the default link,
> > configure scripts will never find/use them, and they'll just end up
> > enabling their own replacements.
> > 
> > Anyway, in general, if adding a function to glibc is a bad idea,
> > adding it in its own separate .so file shipped with glibc is an even
> > worse idea. There are no advantages and lots of disadvantages.
> 
> I am not talking about a libstrlcpy but instead something like libbsd_ext.
> Again, we are talking quite abstractly about applications. Do you have
> a list of open source applications that use strlcpy or strlcat?

Someone came up with one the last time this was discussed, I think.
You can search the libc-alpha archives.

> Anyway, I note that you are quick to argue against my idea. What is your
> proposal to resolve the issue in glibc?

My proposal has always been to implement it in glibc with the correct
semantics matching the original BSD function in all corner cases,
document it well, and note that it's not recommended but without
throwing too much ideology/propaganda into that note.

Rich

On Sat, 7 Nov 2015, Rich Felker wrote:

> Anyway, in general, if adding a function to glibc is a bad idea,
> adding it in its own separate .so file shipped with glibc is an even
> worse idea. There are no advantages and lots of disadvantages.

While it's trivially true that "adding a function to glibc is a bad idea" 
means it's bad to add it to any library in glibc, existing or new, there 
may well be cases where the conclusion is "it's a good idea to add this 
function to glibc, but in a new library (which might be might not be 
linked in automatically via AS_NEEDED in a .so linker script)".  Cf. 
libmvec or the libinux-syscalls.so.1 discussion.  It's just that I don't 
see strlcpy, strlcat or explicit_bzero as such cases - I see them as cases 
that belong in libc.

On Mon, Nov 09, 2015 at 12:59:25PM +0000, Joseph Myers wrote:
> On Sat, 7 Nov 2015, Rich Felker wrote:
> 
> > Anyway, in general, if adding a function to glibc is a bad idea,
> > adding it in its own separate .so file shipped with glibc is an even
> > worse idea. There are no advantages and lots of disadvantages.
> 
> While it's trivially true that "adding a function to glibc is a bad idea" 
> means it's bad to add it to any library in glibc, existing or new, there 
> may well be cases where the conclusion is "it's a good idea to add this 
> function to glibc, but in a new library (which might be might not be 
> linked in automatically via AS_NEEDED in a .so linker script)".  Cf. 
> libmvec or the libinux-syscalls.so.1 discussion.  It's just that I don't 
> see strlcpy, strlcat or explicit_bzero as such cases - I see them as cases 
> that belong in libc.

Could you elaborate on when/how there would ever be a technical (not
ideologically-motivated) reason to add additional .so's as part of
glibc? I see all the ones we have (esp. libpthread and librt) as
historical mistakes that have been a perpetual source of problems
(like programs sticking with the deprecated gettimeofday instead of
clock_gettime because the latter needed -lrt, and broken weak symbol
hacks for pthreads like gcc pr57740, pr60662, or libxml2 pr704904)
that we should aim to eliminate, not duplicate.

Rich

On Mon, 9 Nov 2015, Rich Felker wrote:

> Could you elaborate on when/how there would ever be a technical (not
> ideologically-motivated) reason to add additional .so's as part of

Are you saying libinux-syscalls.so.1 is ideologically motivated?

libmvec is separate to allow for the possibility of using vector 
instructions that are only supported by binutils versions more recent than 
we wish to require as the minimum for a given architecture (this is a 
generic reason that could apply to multiple architectures) - the ABI 
provided by a library mustn't depend on the build tool version, but its 
presence or absence may.  (This reasoning does *not* work for powerpc 
float128 functions - quite clearly TS 18661-3 functions belong in 
libc/libm rather than another library, since many would be aliases of 
existing functions, so float128 functions can only be added to glibc for 
an architecture when there's consensus on a suitable minimum GCC version 
for building libc for that architecture.)

If libdfp were merged into glibc we'd need to consider carefully whether 
separate libraries or libc + libm were the way to go.  Cf. 
<https://gcc.gnu.org/ml/gcc-patches/2008-08/msg00870.html> regarding DFP 
support in libgcc being static-only to avoid making libgcc_s 8 times 
bigger for rarely used functionality (also raising concerns there about 
number of runtime relocations and size of writable segment, though without 
answers there).

On 11/09/2015 07:59 AM, Joseph Myers wrote:
> On Sat, 7 Nov 2015, Rich Felker wrote:
> 
>> Anyway, in general, if adding a function to glibc is a bad idea,
>> adding it in its own separate .so file shipped with glibc is an even
>> worse idea. There are no advantages and lots of disadvantages.
> 
> While it's trivially true that "adding a function to glibc is a bad idea" 
> means it's bad to add it to any library in glibc, existing or new, there 
> may well be cases where the conclusion is "it's a good idea to add this 
> function to glibc, but in a new library (which might be might not be 
> linked in automatically via AS_NEEDED in a .so linker script)".  Cf. 
> libmvec or the libinux-syscalls.so.1 discussion.  It's just that I don't 
> see strlcpy, strlcat or explicit_bzero as such cases - I see them as cases 
> that belong in libc.

I agree.

Cheers,
Carlos.

On Mon, Nov 09, 2015 at 06:31:32PM +0000, Joseph Myers wrote:
> On Mon, 9 Nov 2015, Rich Felker wrote:
> 
> > Could you elaborate on when/how there would ever be a technical (not
> > ideologically-motivated) reason to add additional .so's as part of
> 
> Are you saying libinux-syscalls.so.1 is ideologically motivated?

Yes, so far all of the arguments I've seen for it have been that
somebody does not want them in libc.so because they're not the "GNU
API". I haven't seen any technical problem that's solved by putting
them in a separate .so, but of course there are technical problems
created (more memory usage and startup time, etc.).

> libmvec is separate to allow for the possibility of using vector 
> instructions that are only supported by binutils versions more recent than 
> we wish to require as the minimum for a given architecture (this is a 
> generic reason that could apply to multiple architectures) - the ABI 
> provided by a library mustn't depend on the build tool version, but its 
> presence or absence may.

This seems like a legitimate technical concern, but I don't think it's
the right solution. Instructions that aren't supported by the minimum
binutils version probably need to be represented with .byte or similar
until the version requirement is increased. IMO it's the whole
combined API/ABI of all the glibc libs, not just individual libs, that
should be independent of the tools used to build.

Rich

I don't have an opinion on strlcpy itself, but I've been following this
discussion and I think the best compromise I've heard is found in these
two statements:

On 11/07/2015 04:48 PM, Tolga Dalman wrote:
> Also, I would like to see a comment like "Do not use this function unless
> you really know what you do. Consider strncpy instead." in the documentation
> (and man pages, eventually).

On 11/07/2015 05:51 PM, Rich Felker wrote:
> My proposal has always been to implement it in glibc with the correct
> semantics matching the original BSD function in all corner cases,
> document it well, and note that it's not recommended but without
> throwing too much ideology/propaganda into that note.

As a glibc user, I take the manual at its word.  If it says to avoid use
of a function (and there are quite a few it says that about*), I will.
Rather than inclusion in glibc and subsequent blacklisting via the
manual though, I don't see any harm in providing documentation to the
effect of, "These are the appropriate use cases for this function;
otherwise it should be avoided."

I appreciate that some people want to provide what they feel is a safer
implementation of an error-prone routine.  Despite a pathological urge
to always reinvent the wheel, that is exactly one of the things I rely
on the providers of my C library (you all) to do.

Rical


* Some examples:
https://www.gnu.org/software/libc/manual/html_node/Charset-Function-Overview.html#Charset-Function-Overview
https://www.gnu.org/software/libc/manual/html_node/Basic-Signal-Handling.html#Basic-Signal-Handling
https://www.gnu.org/software/libc/manual/html_node/Formatted-Output-Functions.html#Formatted-Output-Functions

On 11/09/2015 01:16 PM, Rich Felker wrote:
> On Mon, Nov 09, 2015 at 12:59:25PM +0000, Joseph Myers wrote:
>> On Sat, 7 Nov 2015, Rich Felker wrote:
>>
>>> Anyway, in general, if adding a function to glibc is a bad idea,
>>> adding it in its own separate .so file shipped with glibc is an even
>>> worse idea. There are no advantages and lots of disadvantages.
>>
>> While it's trivially true that "adding a function to glibc is a bad idea" 
>> means it's bad to add it to any library in glibc, existing or new, there 
>> may well be cases where the conclusion is "it's a good idea to add this 
>> function to glibc, but in a new library (which might be might not be 
>> linked in automatically via AS_NEEDED in a .so linker script)".  Cf. 
>> libmvec or the libinux-syscalls.so.1 discussion.  It's just that I don't 
>> see strlcpy, strlcat or explicit_bzero as such cases - I see them as cases 
>> that belong in libc.
> 
> Could you elaborate on when/how there would ever be a technical (not
> ideologically-motivated) reason to add additional .so's as part of
> glibc? I see all the ones we have (esp. libpthread and librt) as
> historical mistakes that have been a perpetual source of problems
> (like programs sticking with the deprecated gettimeofday instead of
> clock_gettime because the latter needed -lrt, and broken weak symbol
> hacks for pthreads like gcc pr57740, pr60662, or libxml2 pr704904)
> that we should aim to eliminate, not duplicate.

FWIW, I tend to agree with this.

zw