From patchwork Thu Jul 13 14:06:17 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 787764 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3x7d0W3Kpkz9s74 for ; Fri, 14 Jul 2017 00:06:35 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.b="pTqCHqzq"; dkim-atps=neutral DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:subject:to:cc:references:from:message-id:date :mime-version:in-reply-to:content-type :content-transfer-encoding; q=dns; s=default; b=M4Ua5CaRpQLHA7nU 6oNhWARR7G9mXRzK6IvzJikdbYfO3r8vmezh6XEVAjEg2IYHEr3HZ2DLdS48StQM b2jFEOwg0M/BccPHOybSurgWHxbcoGp5yG2e6OPPG0+OE8WdZG1qKLrwraSa2dNx 1GO+M6HkE+DS+l6Akqx+F99JOFY= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:subject:to:cc:references:from:message-id:date :mime-version:in-reply-to:content-type :content-transfer-encoding; s=default; bh=x3Z5zdlcpGQ0h6Cy3+Zxru BKtDA=; b=pTqCHqzqpu9lLXZ84aWvnzd26PHOOGbxN4rqOkzXcGPrlJ+6XLRpYX qjak3ZuCbCexloYmAUIXh2Nhf2ZLzrGfweoe5jUyVJpN/vejUbUhEXc1xl8o+Bml jSLMBExljEY+Ba9O9vKW2iR2Jrx9p+fF1u93h8OPKAlytKSF9x56A= Received: (qmail 15492 invoked by alias); 13 Jul 2017 14:06:26 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 15476 invoked by uid 89); 13 Jul 2017 14:06:25 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-26.4 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, RCVD_IN_SORBS_SPAM, SPF_PASS autolearn=ham version=3.3.2 spammy= X-HELO: mail-qk0-f171.google.com X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=yy5jEW5SKOnpPxWRqi+CP2Y4Ebb7gpujR5EOI9hVYS0=; b=BVeWHsNDJFh+HqpCRflhmrrZ9OmhmbBor459lLQCJPizXzfLnVPCDjNssbOugOLUfO tEHOOsx8bWlIaI200pyPJSG9EbRw4X7TltPqOu6NcdCAiNcQ5pXh5H8IM5c3Rp9zE60g mlL2CX7GW2EuMyJ4bAP8xo/Za5cZOq+FP7c9reFp6Kxi0Sr+T7PB6XUmM9wioSquD/6J VH5aL2NcEG0TyCEuIfTafyIbnJBuKOLgsC/DInLLgIwdItV4MuofMwk7FbcoxJmyrKKz ZOfKmlA0fr8OSoDK41FoQTedHFpfsPStk7jbAcxkJzGRrnBjYXwdubciTzDl4FFlwbmL FKWA== X-Gm-Message-State: AIVw110u3FaROmNWKPOO6yGonIcmvZdBc5Ep+YGPka/cCVOXQmClByZM bOQNnvDGvBbtKuqSBQPoZg== X-Received: by 10.55.18.132 with SMTP id 4mr456176qks.231.1499954780291; Thu, 13 Jul 2017 07:06:20 -0700 (PDT) Subject: Re: [PATCH] libio: Fix open_memstream flush (NULL) To: Florian Weimer Cc: libc-alpha@sourceware.org References: <1499703585-17933-1-git-send-email-adhemerval.zanella@linaro.org> <87a84ao83j.fsf@mid.deneb.enyo.de> <675937a5-84c5-130c-8834-eaeadfef0898@linaro.org> <87tw2hbsx9.fsf@mid.deneb.enyo.de> From: Adhemerval Zanella Message-ID: <99eea2a1-d8a1-edf2-f638-607ead8e4122@linaro.org> Date: Thu, 13 Jul 2017 11:06:17 -0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <87tw2hbsx9.fsf@mid.deneb.enyo.de> On 12/07/2017 12:01, Florian Weimer wrote: > * Adhemerval Zanella: > >>> I think we don't want the malloc call for a call to exit, either. >>> Similarly for __stack_chk_fail, but that's something we need to fix >>> there. >> >> I am not following, what exactly should not call malloc here? Since >> the open_memstream is a valid call, it will allocate both the initial >> internal FILE object (libio/memstream.c:77) and initial buffer >> (libio/memstream.c:84). Are you saying that we should start with an >> empty buffer instead? If it is the for I think we can do it, but it >> would require more intrusive changes I think. > > I think we should make sure that writing the null terminator does not > trigger malloc during abort (or __stack_chk_fail). That would be > quite wrong IMHO. > > I wasn't aware that the buffer is pre-allocated. Then we probably > need a test which is aware of the size of the preallocated buffer. Right, so in this case we can't call _IO_mem_sync for _IO_mem_overflow with EOF to avoid the possible malloc during abort. I updated the patch to check for correct buffer output in such case. > >> So do you prefer to just add a workaround to disable the single-thread >> optimization for open_memstream and fix it on 2.27? IMHO this fix touches >> only the open_memstream itself, so it should not regress any other libio >> code so I would prefer than carry a workaround for 2.26. > > Which fix? Linking open_memstream into the global list has quite > far-reaching consequences. Conservatively setting the flag instead > should be fine. I agree, but I also though adding the single-thread optimization close to freeze was also not the safe move. I do not have a strong preference here and I think we can delay this fix to 2.27. > > I'm also not sure that the intent of POSIX is that open_memstream > streams should be affected by fflush (NULL). I expect that > open_memstream was added without considering the interaction with > fflush (NULL). > I would say to not make the open_memstream stream an exception and have different semantics than other streams. diff --git a/libio/Makefile b/libio/Makefile index a002a33..6ee809a 100644 --- a/libio/Makefile +++ b/libio/Makefile @@ -57,8 +57,8 @@ tests = tst_swprintf tst_wprintf tst_swscanf tst_wscanf tst_getwc tst_putwc \ tst-mmap-eofsync tst-mmap-fflushsync bug-mmap-fflush \ tst-mmap2-eofsync tst-mmap-offend bug-fopena+ bug-wfflush \ bug-ungetc2 bug-ftell bug-ungetc3 bug-ungetc4 tst-fopenloc2 \ - tst-memstream1 tst-memstream2 tst-memstream3 \ - tst-wmemstream1 tst-wmemstream2 tst-wmemstream3 \ + tst-memstream1 tst-memstream2 tst-memstream3 tst-memstream4 \ + tst-wmemstream1 tst-wmemstream2 tst-wmemstream3 tst-wmemstream4 \ bug-memstream1 bug-wmemstream1 \ tst-setvbuf1 tst-popen1 tst-fgetwc bug-wsetpos tst-fseek \ tst-fwrite-error tst-ftell-partial-wide tst-ftell-active-handler \ diff --git a/libio/memstream.c b/libio/memstream.c index f83d4a5..67ecc09 100644 --- a/libio/memstream.c +++ b/libio/memstream.c @@ -31,13 +31,14 @@ struct _IO_FILE_memstream static int _IO_mem_sync (_IO_FILE* fp) __THROW; static void _IO_mem_finish (_IO_FILE* fp, int) __THROW; +static int _IO_mem_overflow (_IO_FILE *fp, int c) __THROW; static const struct _IO_jump_t _IO_mem_jumps libio_vtable = { JUMP_INIT_DUMMY, JUMP_INIT (finish, _IO_mem_finish), - JUMP_INIT (overflow, _IO_str_overflow), + JUMP_INIT (overflow, _IO_mem_overflow), JUMP_INIT (underflow, _IO_str_underflow), JUMP_INIT (uflow, _IO_default_uflow), JUMP_INIT (pbackfail, _IO_str_pbackfail), @@ -87,6 +88,7 @@ __open_memstream (char **bufloc, _IO_size_t *sizeloc) return NULL; } _IO_init_internal (&new_f->fp._sf._sbf._f, 0); + _IO_link_in ((struct _IO_FILE_plus *) &new_f->fp._sf._sbf); _IO_JUMPS_FILE_plus (&new_f->fp._sf._sbf) = &_IO_mem_jumps; _IO_str_init_static_internal (&new_f->fp._sf, buf, _IO_BUFSIZ, buf); new_f->fp._sf._sbf._f._flags &= ~_IO_USER_BUF; @@ -137,3 +139,17 @@ _IO_mem_finish (_IO_FILE *fp, int dummy) _IO_str_finish (fp, 0); } + +static int +_IO_mem_overflow (_IO_FILE *fp, int c) +{ + if (c == EOF) + { + /* Updates the returned size location on stream flush. */ + struct _IO_FILE_memstream *mp = (struct _IO_FILE_memstream *) fp; + *mp->bufloc = fp->_IO_write_base; + *mp->sizeloc = fp->_IO_write_ptr - fp->_IO_write_base; + return 0; + } + return _IO_str_overflow (fp, c); +} diff --git a/libio/tst-memstream.h b/libio/tst-memstream.h new file mode 100644 index 0000000..ffc2a65 --- /dev/null +++ b/libio/tst-memstream.h @@ -0,0 +1,66 @@ +/* Common definitions for open_memstream tests. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include +#include +#include +#include + +#include + +#ifdef TEST_WCHAR +# include + +/* Straighforward implementation so tst-memstream3 could use check + fwrite on open_memstream. */ +static size_t __attribute__ ((used)) +fwwrite (const void *ptr, size_t size, size_t nmemb, FILE *arq) +{ + const wchar_t *wcs = (const wchar_t*) (ptr); + for (size_t s = 0; s < size; s++) + { + for (size_t n = 0; n < nmemb; n++) + if (fputwc (wcs[n], arq) == WEOF) + return n; + } + return size * nmemb; +} + +# define CHAR_T wchar_t +# define W(o) L##o +# define OPEN_MEMSTREAM open_wmemstream +# define PRINTF wprintf +# define FWRITE fwwrite +# define FPUTC fputwc +# define STRCMP wcscmp +# define STRLEN wcslen +#else +# define CHAR_T char +# define W(o) o +# define OPEN_MEMSTREAM open_memstream +# define PRINTF printf +# define FWRITE fwrite +# define FPUTC fputc +# define STRCMP strcmp +# define STRLEN strlen +#endif + +#define S(s) S1 (s) +#define S1(s) #s diff --git a/libio/tst-memstream4.c b/libio/tst-memstream4.c new file mode 100644 index 0000000..e98ca51 --- /dev/null +++ b/libio/tst-memstream4.c @@ -0,0 +1,99 @@ +/* Test for open_memstream BZ #21735. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "tst-memstream.h" + +static void +mcheck_abort (enum mcheck_status ev) +{ + printf ("mecheck failed with status %d\n", (int) ev); + exit (1); +} + +static int +do_test (void) +{ + mcheck_pedantic (mcheck_abort); + + CHAR_T *buf; + size_t size; + + FILE *fp = OPEN_MEMSTREAM (&buf, &size); + if (fp == NULL) + FAIL_RET ("%s failed", S(OPEN_MEMSTREAM)); + + /* Basic test to check if 'fflush (NULL)' flushed all stream including + open_memstream ones. */ + { + if (FPUTC (W('a'), fp) != W('a')) + FAIL_RET ("%s failed: %m", S(FPUTC)); + + if (fflush (NULL) != 0) + FAIL_RET ("fflush failed: %m"); + + if (size != 1) + FAIL_RET ("size != 1"); + + TEST_VERIFY (STRCMP (buf, W("a")) == 0); + } + + /* Same as before now with a buf close and equal to BUFSIZ. */ + { + const size_t bufsize = BUFSIZ - 1; + CHAR_T bufcmp[bufsize + 2]; + size_t i; + + bufcmp[0] = W('a'); + for (i=1; i diff --git a/libio/tst-wmemstream4.c b/libio/tst-wmemstream4.c new file mode 100644 index 0000000..9dbf615 --- /dev/null +++ b/libio/tst-wmemstream4.c @@ -0,0 +1,20 @@ +/* Test for open_wmemstream BZ #21735. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#define TEST_WCHAR +#include diff --git a/libio/wmemstream.c b/libio/wmemstream.c index 5bc77f5..d68334b 100644 --- a/libio/wmemstream.c +++ b/libio/wmemstream.c @@ -32,13 +32,14 @@ struct _IO_FILE_wmemstream static int _IO_wmem_sync (_IO_FILE* fp) __THROW; static void _IO_wmem_finish (_IO_FILE* fp, int) __THROW; +static int _IO_wmem_overflow (_IO_FILE *fp, int c) __THROW; static const struct _IO_jump_t _IO_wmem_jumps libio_vtable = { JUMP_INIT_DUMMY, JUMP_INIT (finish, _IO_wmem_finish), - JUMP_INIT (overflow, (_IO_overflow_t) _IO_wstr_overflow), + JUMP_INIT (overflow, (_IO_overflow_t) _IO_wmem_overflow), JUMP_INIT (underflow, (_IO_underflow_t) _IO_wstr_underflow), JUMP_INIT (uflow, (_IO_underflow_t) _IO_wdefault_uflow), JUMP_INIT (pbackfail, (_IO_pbackfail_t) _IO_wstr_pbackfail), @@ -89,6 +90,7 @@ open_wmemstream (wchar_t **bufloc, _IO_size_t *sizeloc) } _IO_no_init (&new_f->fp._sf._sbf._f, 0, 0, &new_f->wd, &_IO_wmem_jumps); _IO_fwide (&new_f->fp._sf._sbf._f, 1); + _IO_link_in ((struct _IO_FILE_plus *) &new_f->fp._sf._sbf); _IO_wstr_init_static (&new_f->fp._sf._sbf._f, buf, _IO_BUFSIZ / sizeof (wchar_t), buf); new_f->fp._sf._sbf._f._flags2 &= ~_IO_FLAGS2_USER_WBUF; @@ -142,3 +144,18 @@ _IO_wmem_finish (_IO_FILE *fp, int dummy) _IO_wstr_finish (fp, 0); } + +static int +_IO_wmem_overflow (_IO_FILE *fp, int c) +{ + if (c == EOF) + { + /* Updates the returned size location on stream flush. */ + struct _IO_FILE_wmemstream *mp = (struct _IO_FILE_wmemstream *) fp; + *mp->bufloc = fp->_wide_data->_IO_write_base; + *mp->sizeloc = (fp->_wide_data->_IO_write_ptr + - fp->_wide_data->_IO_write_base); + return 0; + } + return _IO_wstr_overflow (fp, c); +}