From patchwork Tue Jun 7 16:49:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florian Weimer X-Patchwork-Id: 1640050 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=VzAVt2JJ; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LHc0S4gYWz9sFx for ; Wed, 8 Jun 2022 02:54:48 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id B8E45394882B for ; Tue, 7 Jun 2022 16:54:46 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org B8E45394882B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1654620886; bh=n1FOaATrbfvBiexSi6yYpWJVQP4QDZ9W5xllmyWgk9U=; h=To:Subject:In-Reply-To:References:Date:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=VzAVt2JJtgQjm82tUc0PqrddJ8pe0B2E+g7oQZB8jXcVKMcHppdgchoOEg9/IaoYz uDEx/aujon2dYA7v6B3YCoS0uSUnO3bzRjJWNkY+/m5bH4PR/fFLekdsCN7ZDAcjjv fdMd+nu8tY2TJL+iTYzJZvrl2USn8kUw/Z1qevcw= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id 74F4D3948805 for ; Tue, 7 Jun 2022 16:49:55 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 74F4D3948805 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-661-Hh-IsLYKN723HSIGgggLUw-1; Tue, 07 Jun 2022 12:49:53 -0400 X-MC-Unique: Hh-IsLYKN723HSIGgggLUw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8496B3C17312 for ; Tue, 7 Jun 2022 16:49:53 +0000 (UTC) Received: from oldenburg.str.redhat.com (unknown [10.39.192.117]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 01206C27EB2 for ; Tue, 7 Jun 2022 16:49:52 +0000 (UTC) To: libc-alpha@sourceware.org Subject: [PATCH v3 08/11] libio: Convert __vasprintf_internal to buffers In-Reply-To: References: X-From-Line: 97f2405fb8785106ee34c63a18e0f97076d83d9a Mon Sep 17 00:00:00 2001 Message-Id: <97f2405fb8785106ee34c63a18e0f97076d83d9a.1654620298.git.fweimer@redhat.com> Date: Tue, 07 Jun 2022 18:49:51 +0200 User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-11.8 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Florian Weimer via Libc-alpha From: Florian Weimer Reply-To: Florian Weimer Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" The buffer resizing algorithm is slightly different. The initial buffer is on the stack, and small buffers are directly allocated on the heap using the exact required size. The overhead of the additional copy is compensated by the lowered setup cost for buffers compared to libio streams. --- include/printf_buffer.h | 4 + libio/vasprintf.c | 141 ++++++++++++++++++++--------- stdio-common/printf_buffer_flush.c | 4 + 3 files changed, 104 insertions(+), 45 deletions(-) diff --git a/include/printf_buffer.h b/include/printf_buffer.h index 92393399fb..47e252b52e 100644 --- a/include/printf_buffer.h +++ b/include/printf_buffer.h @@ -48,6 +48,7 @@ enum __printf_buffer_mode __printf_buffer_mode_snprintf, __printf_buffer_mode_sprintf_chk, __printf_buffer_mode_to_file, + __printf_buffer_mode_asprintf, __printf_buffer_mode_strfmon, __printf_buffer_mode_fp, /* For __printf_fp_l_buffer. */ __printf_buffer_mode_fp_to_wide, /* For __wprintf_fp_l_buffer. */ @@ -301,6 +302,9 @@ void __printf_buffer_flush_snprintf (struct __printf_buffer_snprintf *) struct __printf_buffer_to_file; void __printf_buffer_flush_to_file (struct __printf_buffer_to_file *) attribute_hidden; +struct __printf_buffer_asprintf; +void __printf_buffer_flush_asprintf (struct __printf_buffer_asprintf *) + attribute_hidden; struct __printf_buffer_fp; void __printf_buffer_flush_fp (struct __printf_buffer_fp *) attribute_hidden; diff --git a/libio/vasprintf.c b/libio/vasprintf.c index 4430a266c6..f4ff7ec98b 100644 --- a/libio/vasprintf.c +++ b/libio/vasprintf.c @@ -24,64 +24,115 @@ This exception applies to code released by its copyright holders in files containing the exception. */ -#include +#include +#include +#include +#include +#include +#include #include -#include +#include +#include + +struct __printf_buffer_asprintf +{ + /* base.write_base points either to a heap-allocated buffer, or to + the direct array below. */ + struct __printf_buffer base; + + /* Initial allocation. 200 should be large enough to copy almost + all asprintf usages with just a single (final, correctly sized) + heap allocation. */ + char direct[200]; +}; + +void +__printf_buffer_flush_asprintf (struct __printf_buffer_asprintf *buf) +{ + size_t current_pos = buf->base.write_ptr - buf->base.write_base; + if (current_pos >= INT_MAX) + { + /* The result is not representable. No need to continue. */ + __set_errno (EOVERFLOW); + __printf_buffer_mark_failed (&buf->base); + return; + } + + size_t current_size = buf->base.write_end - buf->base.write_base; + /* Implement an exponentiatial sizing policy. Keep the size + congruent 8 (mod 16), to account for the footer in glibc + malloc. */ + size_t new_size = ((current_size + current_size / 2) & -15) | 8; + char *new_buffer; + if (buf->base.write_base == buf->direct) + { + new_buffer = malloc (new_size); + if (new_buffer == NULL) + { + __printf_buffer_mark_failed (&buf->base); + return; + } + memcpy (new_buffer, buf->direct, current_pos); + } + else + { + new_buffer = realloc (buf->base.write_base, new_size); + if (new_buffer == NULL) + { + __printf_buffer_mark_failed (&buf->base); + return; + } + } + + /* Set up the new write area. */ + buf->base.write_base = new_buffer; + buf->base.write_ptr = new_buffer + current_pos; + buf->base.write_end = new_buffer + new_size; +} + int __vasprintf_internal (char **result_ptr, const char *format, va_list args, unsigned int mode_flags) { - /* Initial size of the buffer to be used. Will be doubled each time an - overflow occurs. */ - const size_t init_string_size = 100; - char *string; - _IO_strfile sf; - int ret; - size_t needed; - size_t allocated; - /* No need to clear the memory here (unlike for open_memstream) since - we know we will never seek on the stream. */ - string = (char *) malloc (init_string_size); - if (string == NULL) - return -1; -#ifdef _IO_MTSAFE_IO - sf._sbf._f._lock = NULL; -#endif - _IO_no_init (&sf._sbf._f, _IO_USER_LOCK, -1, NULL, NULL); - _IO_JUMPS (&sf._sbf) = &_IO_str_jumps; - _IO_str_init_static_internal (&sf, string, init_string_size, string); - sf._sbf._f._flags &= ~_IO_USER_BUF; - sf._s._allocate_buffer_unused = (_IO_alloc_type) malloc; - sf._s._free_buffer_unused = (_IO_free_type) free; - ret = __vfprintf_internal (&sf._sbf._f, format, args, mode_flags); - if (ret < 0) + struct __printf_buffer_asprintf buf; + __printf_buffer_init (&buf.base, buf.direct, array_length (buf.direct), + __printf_buffer_mode_asprintf); + + __printf_buffer (&buf.base, format, args, mode_flags); + int done = __printf_buffer_done (&buf.base); + if (done < 0) { - free (sf._sbf._f._IO_buf_base); - return ret; + if (buf.base.write_base != buf.direct) + free (buf.base.write_base); + return done; + } + + /* Transfer to the final buffer. */ + char *result; + size_t size = buf.base.write_ptr - buf.base.write_base; + if (buf.base.write_base == buf.direct) + { + result = malloc (size + 1); + if (result == NULL) + return -1; + memcpy (result, buf.direct, size); } - /* Only use realloc if the size we need is of the same (binary) - order of magnitude then the memory we allocated. */ - needed = sf._sbf._f._IO_write_ptr - sf._sbf._f._IO_write_base + 1; - allocated = sf._sbf._f._IO_write_end - sf._sbf._f._IO_write_base; - if ((allocated >> 1) <= needed) - *result_ptr = (char *) realloc (sf._sbf._f._IO_buf_base, needed); else { - *result_ptr = (char *) malloc (needed); - if (*result_ptr != NULL) + result = realloc (buf.base.write_base, size + 1); + if (result == NULL) { - memcpy (*result_ptr, sf._sbf._f._IO_buf_base, needed - 1); - free (sf._sbf._f._IO_buf_base); + free (buf.base.write_base); + return -1; } - else - /* We have no choice, use the buffer we already have. */ - *result_ptr = (char *) realloc (sf._sbf._f._IO_buf_base, needed); } - if (*result_ptr == NULL) - *result_ptr = sf._sbf._f._IO_buf_base; - (*result_ptr)[needed - 1] = '\0'; - return ret; + + /* Add NUL termination. */ + result[size] = '\0'; + *result_ptr = result; + + return done; } int diff --git a/stdio-common/printf_buffer_flush.c b/stdio-common/printf_buffer_flush.c index f44c8b887f..14fe1b2df4 100644 --- a/stdio-common/printf_buffer_flush.c +++ b/stdio-common/printf_buffer_flush.c @@ -27,6 +27,7 @@ #ifndef SHARED # pragma weak __printf_buffer_flush_snprintf # pragma weak __printf_buffer_flush_to_file +# pragma weak __printf_buffer_flush_asprintf # pragma weak __printf_buffer_flush_fp # pragma weak __printf_buffer_flush_fp_to_wide # pragma weak __printf_buffer_flush_fphex_to_wide @@ -49,6 +50,9 @@ __printf_buffer_do_flush (struct __printf_buffer *buf) case __printf_buffer_mode_to_file: __printf_buffer_flush_to_file ((struct __printf_buffer_to_file *) buf); return; + case __printf_buffer_mode_asprintf: + __printf_buffer_flush_asprintf ((struct __printf_buffer_asprintf *) buf); + return; case __printf_buffer_mode_strfmon: __set_errno (E2BIG); __printf_buffer_mark_failed (buf);