From patchwork Tue Aug 8 18:24:12 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 799387 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=sourceware.org (client-ip=209.132.180.131; helo=sourceware.org; envelope-from=libc-alpha-return-82885-incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.b="lUjY78Gz"; dkim-atps=neutral Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3xRjVs2QBhz9sMN for ; Wed, 9 Aug 2017 04:25:09 +1000 (AEST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:subject:date:message-id; q=dns; s= default; b=IkbueO45VWvfOIWpTlogBmusW/4oXKYjhjLt7tjpJEmj5g8njHRpU P2NG9xDuycLcWv6Q2NPvAOpPXbz2zIrSdAOrxViNhzscQNsgWWJ62BDNcmD40QR4 S1IMbqK6iB5ZL2KtGhm8djRdXW+zJihAhUGoSf5+V37vMgxkveRJNw= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:subject:date:message-id; s=default; bh=7gbV8LaROG/enG10gDbtJSHluT0=; b=lUjY78GzdLhGETFO0+20TVvEYPKq 5z2A1eNXv/A8TKron4QV6ERPTcECmXr+eOQiYA4viGP2rxPKiFpMrlgAY42f3Elb 9ioAecmUvFhXG2/eAAlE0yZERW4PF2dfu1AXOLiG1+PXSqO4tfpqsEaQEXAxvYrR 7bAhYTiGO+OjLug= Received: (qmail 56349 invoked by alias); 8 Aug 2017 18:24:41 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 39636 invoked by uid 89); 8 Aug 2017 18:24:24 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-26.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.2 spammy= X-HELO: mail-ua0-f179.google.com X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=hCRjrewKrF5YKUnWD4SKm065fM4GfZTFr6OFdH9i3z4=; b=no4RgY75P5V7xDhR0jevoUz4NvD4WrySpNhe3hiREjzRCDgdmkq0/xnERPWpbWoX9G Td7pnWgprdo2YWJyK4IZ0n4N5eu58Afdxjjw9Cp1y0RewWOjmQ6TMmVFL7n22+CZBWE4 G7jRIMddDsxRQqE0fCE0pa2+vglxyOtcSCiIatwGN3zzLtvziJ9+ovEDco1qA29H8RrJ n3WMqGVaipXbhuUcH9XOk7hoZAnw9+TdYIFlosFGUG5hU8jdUkEXNVYV9BKlOqFBofa1 hZ1YA1cma8VLfq5ftIEpsJeMh/si114E63lr6nkqnSiKhIMhIqStdf6w62i3HMQS6Sbs mFKA== X-Gm-Message-State: AHYfb5hV/DgOn7mz62hOGDwBLtB+c37kJ4kWb+nw+ERobDVMcbcVYj0y rAl/ccMrBo5XmQN+76KYwQ== X-Received: by 10.176.17.106 with SMTP id g42mr3918201uac.162.1502216656718; Tue, 08 Aug 2017 11:24:16 -0700 (PDT) From: Adhemerval Zanella To: libc-alpha@sourceware.org Subject: [PATCH v2] libio: Fix seek-past-end returned size for open_{w}memstream (BZ#15298) Date: Tue, 8 Aug 2017 15:24:12 -0300 Message-Id: <1502216652-14099-1-git-send-email-adhemerval.zanella@linaro.org> POSIX states that after a successfull fflush() or fclose() the variable pointed by the input size shall contain the smaller of the current buffer length and the number of bytes (or wide characters for wide version) [1]. Current GLIBC behavior returns the seek position even there is no previous write operation. To correctly report the buffer size the implementation must track both the buffer position and current byte written. However internal _IO_write_ptr is update on both write and seek operations. This patch fixes how open_{w}memstream updates the returned buffer size of a fclose/fflush operation by adding two new internal fields to keep track of both previous and next position after a write operation. Checked on x86_64-linux-gnu. [1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/open_memstream.html [BZ #15298] * libio/Makefile (tests): Add tst-memstream6 and tst-wmemstream6. * libio/memstream.c (_IO_FILE_memstream): Add prevwriteptr and seekwriteptr fields. (_IO_mem_seekoff): New function. (libio_vtable): Use _IO_mem_seekoff instead of _IO_str_seekoff. (__open_memstream): Initialize prevwriteptr and seekwriteptr. (_IO_mem_sync): Update sizeloc based on written bytes instead of buffer current position. (_IO_mem_finish): Likewise. * libio/memstream.c (_IO_FILE_wmemstream): Add prevwriteptr and seekwriteptr fields. (_IO_wmem_seekoff): New function. (libio_vtable): Use _IO_mem_seekoff instead of _IO_str_seekoff. (__open_wmemstream): Initialize prevwriteptr and seekwriteptr. (_IO_wmem_sync): Update sizeloc based on written bytes instead of buffer current position. (_IO_wmem_finish): Likewise. * libio/tst-memstream3.c (do_test_bz18241): Check for expected size after a fseek followed by a fflush. * libio/tst-memstream4.c: New file. * libio/tst-wmemstream4.c: Likewise. * libio/tst-memstream.h: Likewise. --- ChangeLog | 26 +++++++++++++++++ libio/Makefile | 4 +-- libio/memstream.c | 59 ++++++++++++++++++++++++++++++++++++--- libio/tst-memstream3.c | 8 ++++-- libio/tst-memstream4.c | 74 +++++++++++++++++++++++++++++++++++++++++++++++++ libio/tst-wmemstream4.c | 20 +++++++++++++ libio/wmemstream.c | 62 +++++++++++++++++++++++++++++++++++------ 7 files changed, 236 insertions(+), 17 deletions(-) create mode 100644 libio/tst-memstream4.c create mode 100644 libio/tst-wmemstream4.c diff --git a/libio/Makefile b/libio/Makefile index 9d09bd8..b205842 100644 --- a/libio/Makefile +++ b/libio/Makefile @@ -57,8 +57,8 @@ tests = tst_swprintf tst_wprintf tst_swscanf tst_wscanf tst_getwc tst_putwc \ tst-mmap-eofsync tst-mmap-fflushsync bug-mmap-fflush \ tst-mmap2-eofsync tst-mmap-offend bug-fopena+ bug-wfflush \ bug-ungetc2 bug-ftell bug-ungetc3 bug-ungetc4 tst-fopenloc2 \ - tst-memstream1 tst-memstream2 tst-memstream3 \ - tst-wmemstream1 tst-wmemstream2 tst-wmemstream3 \ + tst-memstream1 tst-memstream2 tst-memstream3 tst-memstream4 \ + tst-wmemstream1 tst-wmemstream2 tst-wmemstream3 tst-wmemstream4\ bug-memstream1 bug-wmemstream1 \ tst-setvbuf1 tst-popen1 tst-fgetwc bug-wsetpos tst-fseek \ tst-fwrite-error tst-ftell-partial-wide tst-ftell-active-handler \ diff --git a/libio/memstream.c b/libio/memstream.c index e391efd..f5a9cc8 100644 --- a/libio/memstream.c +++ b/libio/memstream.c @@ -26,11 +26,15 @@ struct _IO_FILE_memstream _IO_strfile _sf; char **bufloc; _IO_size_t *sizeloc; + char *prevwriteptr; + char *seekwriteptr; }; static int _IO_mem_sync (_IO_FILE* fp) __THROW; static void _IO_mem_finish (_IO_FILE* fp, int) __THROW; +static _IO_off64_t _IO_mem_seekoff (_IO_FILE *fp, _IO_off64_t offset, + int dir, int mode) __THROW; static const struct _IO_jump_t _IO_mem_jumps libio_vtable = @@ -43,7 +47,7 @@ static const struct _IO_jump_t _IO_mem_jumps libio_vtable = JUMP_INIT (pbackfail, _IO_str_pbackfail), JUMP_INIT (xsputn, _IO_default_xsputn), JUMP_INIT (xsgetn, _IO_default_xsgetn), - JUMP_INIT (seekoff, _IO_str_seekoff), + JUMP_INIT (seekoff, _IO_mem_seekoff), JUMP_INIT (seekpos, _IO_default_seekpos), JUMP_INIT (setbuf, _IO_default_setbuf), JUMP_INIT (sync, _IO_mem_sync), @@ -96,6 +100,26 @@ __open_memstream (char **bufloc, _IO_size_t *sizeloc) new_f->fp.bufloc = bufloc; new_f->fp.sizeloc = sizeloc; + /* To correctly report the buffer size the implementation must track both + the buffer size and currently bytes written, however _IO_write_ptr is + updated on both write and seek operations (since some _IO_* function + access the pointer directly to optimize updates). So to track current + written bytes two fields are used: + + - prevwriteptr: track previous _IO_write_ptr before a seek operation on + the stream. + - seekwriteptr: track resulted _IO_write_ptr after a seek operation on + the stream. + + Also, prevwriteptr is only updated iff _IO_write_ptr changed over calls + (meaning that a write operation occured) + + So final buffer size is based on current _IO_write_ptr only if + its value is different than seekwriteptr, otherwise it uses the old + _IO_write_ptr value before seek operation (prevwriteptr). */ + new_f->fp.prevwriteptr = new_f->fp.seekwriteptr = + new_f->fp._sf._sbf._f._IO_write_ptr; + /* Disable single thread optimization. BZ 21735. */ new_f->fp._sf._sbf._f._flags2 |= _IO_FLAGS2_NEED_LOCK; @@ -104,6 +128,21 @@ __open_memstream (char **bufloc, _IO_size_t *sizeloc) libc_hidden_def (__open_memstream) weak_alias (__open_memstream, open_memstream) +/* Update 'size' with written number of bytes and return true if a written + operation has occured. */ +static bool +update_bufsize (const _IO_FILE *fp, size_t *size) +{ + const struct _IO_FILE_memstream *mp = + (const struct _IO_FILE_memstream *) fp; + if (fp->_IO_write_ptr == mp->seekwriteptr) + { + *size = mp->prevwriteptr - fp->_IO_write_base; + return false; + } + *size = fp->_IO_write_ptr - fp->_IO_write_base; + return true; +} static int _IO_mem_sync (_IO_FILE *fp) @@ -117,7 +156,7 @@ _IO_mem_sync (_IO_FILE *fp) } *mp->bufloc = fp->_IO_write_base; - *mp->sizeloc = fp->_IO_write_ptr - fp->_IO_write_base; + update_bufsize (fp, mp->sizeloc); return 0; } @@ -132,11 +171,23 @@ _IO_mem_finish (_IO_FILE *fp, int dummy) fp->_IO_write_ptr - fp->_IO_write_base + 1); if (*mp->bufloc != NULL) { - (*mp->bufloc)[fp->_IO_write_ptr - fp->_IO_write_base] = '\0'; - *mp->sizeloc = fp->_IO_write_ptr - fp->_IO_write_base; + /* An '\0' should be appended iff a write operation ocurred. */ + if (update_bufsize (fp, mp->sizeloc)) + (*mp->bufloc)[*mp->sizeloc] = '\0'; fp->_IO_buf_base = NULL; } _IO_str_finish (fp, 0); } + +static _IO_off64_t +_IO_mem_seekoff (_IO_FILE *fp, _IO_off64_t offset, int dir, int mode) +{ + struct _IO_FILE_memstream *mp = (struct _IO_FILE_memstream *) fp; + if (fp->_IO_write_ptr != mp->seekwriteptr) + mp->prevwriteptr = fp->_IO_write_ptr; + _IO_off64_t ret = _IO_str_seekoff (fp, offset, dir, mode); + mp->seekwriteptr = fp->_IO_write_ptr; + return ret; +} diff --git a/libio/tst-memstream3.c b/libio/tst-memstream3.c index ce201d1..6521f92 100644 --- a/libio/tst-memstream3.c +++ b/libio/tst-memstream3.c @@ -126,15 +126,17 @@ do_test_bz20181 (void) if (fflush (fp) != 0) ERROR_RET1 ("fflush failed (errno = %d)\n", errno); - /* Avoid truncating the buffer on close. */ + /* fseek updates the internal buffer, but open_memstream should set the + size to smaller of the buffer size and number of bytes written. Since + it was written just character ('z') final size should be 1. */ if (fseek (fp, 3, SEEK_SET) != 0) ERROR_RET1 ("fseek failed (errno = %d)\n", errno); if (fclose (fp) != 0) ERROR_RET1 ("fclose failed (errno = %d\n", errno); - if (size != 3) - ERROR_RET1 ("size != 3\n"); + if (size != 1) + ERROR_RET1 ("size != 1 (got %zu)\n", size); if (buf[0] != W('z') || buf[1] != W('b') diff --git a/libio/tst-memstream4.c b/libio/tst-memstream4.c new file mode 100644 index 0000000..b877d68 --- /dev/null +++ b/libio/tst-memstream4.c @@ -0,0 +1,74 @@ +/* Test for open_memstream BZ #15298. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "tst-memstream.h" + +static void +mcheck_abort (enum mcheck_status ev) +{ + printf ("mecheck failed with status %d\n", (int) ev); + exit (1); +} + +static int +do_test (void) +{ + mcheck_pedantic (mcheck_abort); + + { + CHAR_T *buf; + size_t size; + + FILE *fp = OPEN_MEMSTREAM (&buf, &size); + TEST_VERIFY_EXIT (fp != NULL); + + /* Move internal position but do not write any bytes. Final size should + be 0. */ + TEST_VERIFY_EXIT (fseek (fp, 10, SEEK_SET) != -1); + TEST_VERIFY_EXIT (fseek (fp, 20, SEEK_CUR) != -1); + TEST_VERIFY_EXIT (fseek (fp, 30, SEEK_CUR) != -1); + TEST_VERIFY_EXIT (fflush (fp) != -1); + TEST_VERIFY (size == 0); + + /* Now write some bytes and change internal position. Final size should + be based on written bytes. */ + TEST_VERIFY_EXIT (fseek (fp, 0, SEEK_SET) != -1); + TEST_VERIFY_EXIT (FWRITE (W("abc"), 1, 3, fp) == 3); + TEST_VERIFY_EXIT (fseek (fp, 20, SEEK_CUR) != -1); + TEST_VERIFY_EXIT (fseek (fp, 30, SEEK_CUR) != -1); + TEST_VERIFY_EXIT (fflush (fp) != -1); + TEST_VERIFY (size == 3); + + /* Finally set position, write some bytes and change position again. + Final size should be based again on write position. */ + size_t offset = 2048; + TEST_VERIFY_EXIT (fseek (fp, offset, SEEK_SET) != -1); + TEST_VERIFY_EXIT (FWRITE (W("def"), 1, 3, fp) == 3); + TEST_VERIFY_EXIT (fseek (fp, 20, SEEK_CUR) != -1); + TEST_VERIFY_EXIT (fseek (fp, 20, SEEK_CUR) != -1); + TEST_VERIFY_EXIT (fflush (fp) != -1); + TEST_VERIFY (size == (offset + 3)); + + TEST_VERIFY_EXIT (fclose (fp) == 0); + free (buf); + } + + return 0; +} + +#include diff --git a/libio/tst-wmemstream4.c b/libio/tst-wmemstream4.c new file mode 100644 index 0000000..ace287d --- /dev/null +++ b/libio/tst-wmemstream4.c @@ -0,0 +1,20 @@ +/* Test for open_wmemstream BZ #15298. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#define TEST_WCHAR +#include diff --git a/libio/wmemstream.c b/libio/wmemstream.c index 103a760..03c5318 100644 --- a/libio/wmemstream.c +++ b/libio/wmemstream.c @@ -27,11 +27,15 @@ struct _IO_FILE_wmemstream _IO_strfile _sf; wchar_t **bufloc; _IO_size_t *sizeloc; + wchar_t *prevwriteptr; + wchar_t *seekwriteptr; }; static int _IO_wmem_sync (_IO_FILE* fp) __THROW; static void _IO_wmem_finish (_IO_FILE* fp, int) __THROW; +static _IO_off64_t _IO_wmem_seekoff (_IO_FILE *fp, _IO_off64_t offset, + int dir, int mode) __THROW; static const struct _IO_jump_t _IO_wmem_jumps libio_vtable = @@ -44,7 +48,7 @@ static const struct _IO_jump_t _IO_wmem_jumps libio_vtable = JUMP_INIT (pbackfail, (_IO_pbackfail_t) _IO_wstr_pbackfail), JUMP_INIT (xsputn, _IO_wdefault_xsputn), JUMP_INIT (xsgetn, _IO_wdefault_xsgetn), - JUMP_INIT (seekoff, _IO_wstr_seekoff), + JUMP_INIT (seekoff, _IO_wmem_seekoff), JUMP_INIT (seekpos, _IO_default_seekpos), JUMP_INIT (setbuf, _IO_default_setbuf), JUMP_INIT (sync, _IO_wmem_sync), @@ -97,6 +101,25 @@ open_wmemstream (wchar_t **bufloc, _IO_size_t *sizeloc) new_f->fp.bufloc = bufloc; new_f->fp.sizeloc = sizeloc; + /* To correctly report the buffer size the implementation must track both + the buffer size and currently bytes written, however _IO_write_ptr is + updated on both write and seek operations (since some _IO_* function + access the pointer directly to optimize updates). So to track current + written bytes two fields are used: + + - prevwriteptr: track previous _IO_write_ptr before a seek operation on + the stream. + - seekwriteptr: track resulted _IO_write_ptr after a seek operation on + the stream. + + Also, prevwriteptr is only updated iff _IO_write_ptr changed over calls + (meaning that a write operation occured) + + So final buffer size is based on current _IO_write_ptr only if + its value is different than seekwriteptr, otherwise it uses the old + _IO_write_ptr value before seek operation (prevwriteptr). */ + new_f->fp.prevwriteptr = new_f->fp.seekwriteptr = + new_f->fp._sf._sbf._f._wide_data->_IO_write_ptr; /* Disable single thread optimization. BZ 21735. */ new_f->fp._sf._sbf._f._flags2 |= _IO_FLAGS2_NEED_LOCK; @@ -105,6 +128,22 @@ open_wmemstream (wchar_t **bufloc, _IO_size_t *sizeloc) } +/* Update 'size' with written number of bytes and return true if a written + operation has occured. */ +static bool +update_bufsize (const _IO_FILE *fp, size_t *size) +{ + const struct _IO_FILE_wmemstream *mp = + (const struct _IO_FILE_wmemstream *) fp; + if (fp->_wide_data->_IO_write_ptr == mp->seekwriteptr) + { + *size = mp->prevwriteptr - fp->_wide_data->_IO_write_base; + return false; + } + *size = fp->_wide_data->_IO_write_ptr - fp->_wide_data->_IO_write_base; + return true; +} + static int _IO_wmem_sync (_IO_FILE *fp) { @@ -117,13 +156,11 @@ _IO_wmem_sync (_IO_FILE *fp) } *mp->bufloc = fp->_wide_data->_IO_write_base; - *mp->sizeloc = (fp->_wide_data->_IO_write_ptr - - fp->_wide_data->_IO_write_base); + update_bufsize (fp, mp->sizeloc); return 0; } - static void _IO_wmem_finish (_IO_FILE *fp, int dummy) { @@ -135,13 +172,22 @@ _IO_wmem_finish (_IO_FILE *fp, int dummy) * sizeof (wchar_t)); if (*mp->bufloc != NULL) { - size_t len = (fp->_wide_data->_IO_write_ptr - - fp->_wide_data->_IO_write_base); - (*mp->bufloc)[len] = '\0'; - *mp->sizeloc = len; + if (update_bufsize (fp, mp->sizeloc)) + (*mp->bufloc)[*mp->sizeloc] = L'\0'; fp->_wide_data->_IO_buf_base = NULL; } _IO_wstr_finish (fp, 0); } + +static _IO_off64_t +_IO_wmem_seekoff (_IO_FILE *fp, _IO_off64_t offset, int dir, int mode) +{ + struct _IO_FILE_wmemstream *mp = (struct _IO_FILE_wmemstream *) fp; + if (fp->_wide_data->_IO_write_ptr != mp->seekwriteptr) + mp->prevwriteptr = fp->_wide_data->_IO_write_ptr; + _IO_off64_t ret = _IO_wstr_seekoff (fp, offset, dir, mode); + mp->seekwriteptr = fp->_wide_data->_IO_write_ptr; + return ret; +}