{"id":2175491,"url":"http://patchwork.ozlabs.org/api/1.0/patches/2175491/?format=json","project":{"id":41,"url":"http://patchwork.ozlabs.org/api/1.0/projects/41/?format=json","name":"GNU C Library","link_name":"glibc","list_id":"libc-alpha.sourceware.org","list_email":"libc-alpha@sourceware.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<2625bdca771c84d837407762eb0d5fd0d0ccd983.1766051058.git.fweimer@redhat.com>","date":"2025-12-18T10:17:32","name":"[1/3] malloc: Perform batched frees if tcache is full","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"1c7edf2132d6c3116872c3abe56d7b28c1bd1ba2","submitter":{"id":14312,"url":"http://patchwork.ozlabs.org/api/1.0/people/14312/?format=json","name":"Florian Weimer","email":"fweimer@redhat.com"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/glibc/patch/2625bdca771c84d837407762eb0d5fd0d0ccd983.1766051058.git.fweimer@redhat.com/mbox/","series":[{"id":485815,"url":"http://patchwork.ozlabs.org/api/1.0/series/485815/?format=json","date":"2025-12-18T10:17:02","name":"malloc: batched frees","version":1,"mbox":"http://patchwork.ozlabs.org/series/485815/mbox/"}],"check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/2175491/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","libc-alpha@sourceware.org"],"Delivered-To":["patchwork-incoming@legolas.ozlabs.org","libc-alpha@sourceware.org"],"Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=mimecast20190719 header.b=FkKrjixv;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org\n (client-ip=38.145.34.32; helo=vm01.sourceware.org;\n envelope-from=libc-alpha-bounces~incoming=patchwork.ozlabs.org@sourceware.org;\n receiver=patchwork.ozlabs.org)","sourceware.org;\n\tdkim=pass (1024-bit key,\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=mimecast20190719 header.b=FkKrjixv","sourceware.org; dmarc=pass (p=quarantine dis=none)\n header.from=redhat.com","sourceware.org; spf=pass smtp.mailfrom=redhat.com","server2.sourceware.org;\n arc=none smtp.remote-ip=170.10.129.124"],"Received":["from vm01.sourceware.org (vm01.sourceware.org [38.145.34.32])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4dX68Y50pQz1y2F\n\tfor ; Thu, 18 Dec 2025 21:19:09 +1100 (AEDT)","from vm01.sourceware.org (localhost [127.0.0.1])\n\tby sourceware.org (Postfix) with ESMTP id 799074BA2E26\n\tfor ; Thu, 18 Dec 2025 10:19:07 +0000 (GMT)","from us-smtp-delivery-124.mimecast.com\n (us-smtp-delivery-124.mimecast.com [170.10.129.124])\n by sourceware.org (Postfix) with ESMTP id D11464BA2E2D\n for ; Thu, 18 Dec 2025 10:17:37 +0000 (GMT)","from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com\n (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by\n relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3,\n cipher=TLS_AES_256_GCM_SHA384) id us-mta-179-zPYs86piNsO8klYUvcoiaQ-1; Thu,\n 18 Dec 2025 05:17:36 -0500","from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com\n (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111])\n (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest\n SHA256)\n (No client certificate requested)\n by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS\n id 3FEAA1955DC3\n for ; Thu, 18 Dec 2025 10:17:35 +0000 (UTC)","from fweimer-oldenburg.csb.redhat.com (unknown [10.44.33.94])\n by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with\n ESMTPS\n id 8A93E18004D4\n for ; Thu, 18 Dec 2025 10:17:34 +0000 (UTC)"],"DKIM-Filter":["OpenDKIM Filter v2.11.0 sourceware.org 799074BA2E26","OpenDKIM Filter v2.11.0 sourceware.org D11464BA2E2D"],"DMARC-Filter":"OpenDMARC Filter v1.4.2 sourceware.org D11464BA2E2D","ARC-Filter":"OpenARC Filter v1.0.0 sourceware.org D11464BA2E2D","ARC-Seal":"i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1766053057; cv=none;\n b=Z0AN/SCeHze8CeXL8DdiXD7V9MEnCZI3VMXLfPrtkuK+q9RHs5mnaqyw0Epx+s5sINFFPIDLIfIr81kGIb4xQw+2JBaIz9Zxmaze6BRAalhakoLoJFYJlDfiHpsp+3Kwdnhw2gcFWxq897fbTQxKxMhZ4Tp3D48ZF1O+84djUDk=","ARC-Message-Signature":"i=1; a=rsa-sha256; d=sourceware.org; s=key;\n t=1766053057; c=relaxed/simple;\n bh=3ZOvKUAeNF90nBV4tQXEYAOpga8xsKQ64O+2Ez/g5c0=;\n h=DKIM-Signature:From:To:Subject:Message-ID:Date:MIME-Version;\n b=AgiDa/W6qB2uDaR/JNrjq4EFp3F3NBmj3iZkFl+AEoDWrgydG279iqXmw/DbcjemzoJzLJ/wp3qQsPFCypGCiWL/UAtkGK4PBb9aucis4g6NLZB8kkukqpP5Jqf5R1YmSUabACa1P1Lux5RQYCZV0poNOKEkunPW2Ccaj95C8AA=","ARC-Authentication-Results":"i=1; server2.sourceware.org","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n s=mimecast20190719; t=1766053057;\n h=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n to:to:cc:mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=LfmZYP7SafI2RqRi1tkXlT4BaLhshImMCQqqV2cZ6to=;\n b=FkKrjixvCBroWAK1hhN70FNxOVZpa/wuUnzYUrUJrhZpf7KFFZUSZN1pDp/O7xt40Y+8SL\n m/d+1GJOqSnM+rAzwudeTsrX6wjgeFF9N9M4FHCL4vN3jJ6jLXGMRMbXlcM2Ey0u8RNXH7\n F918nhF8mOTc5X0LNDlG5eVs7U6rASA=","X-MC-Unique":"zPYs86piNsO8klYUvcoiaQ-1","X-Mimecast-MFC-AGG-ID":"zPYs86piNsO8klYUvcoiaQ_1766053055","From":"Florian Weimer ","To":"libc-alpha@sourceware.org","Subject":"[PATCH 1/3] malloc: Perform batched frees if tcache is full","In-Reply-To":"","Message-ID":"\n <2625bdca771c84d837407762eb0d5fd0d0ccd983.1766051058.git.fweimer@redhat.com>","References":"","X-From-Line":"2625bdca771c84d837407762eb0d5fd0d0ccd983 Mon Sep 17 00:00:00 2001","Date":"Thu, 18 Dec 2025 11:17:32 +0100","User-Agent":"Gnus/5.13 (Gnus v5.13)","MIME-Version":"1.0","X-Scanned-By":"MIMEDefang 3.4.1 on 10.30.177.111","X-Mimecast-Spam-Score":"0","X-Mimecast-MFC-PROC-ID":"IFwqsolVf1sSQJ7Uxi6jiS8Qt4MjOQf9Gdr7DEx4B8o_1766053055","X-Mimecast-Originator":"redhat.com","Content-Type":"text/plain","X-BeenThere":"libc-alpha@sourceware.org","X-Mailman-Version":"2.1.30","Precedence":"list","List-Id":"Libc-alpha mailing list ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"libc-alpha-bounces~incoming=patchwork.ozlabs.org@sourceware.org"},"content":"Batched frees amortize the locking overhead once the tcache is full.\nPreviously, once the tcache slot was full, each free acquired the\narena lock. With this change, neighboring tcache allocations in the\nsame arena re-use an arena lock that already exists.\n\nPass the tcache pointer to __libc_free_batch so that it is not \nnecessary to reload the tcache pointer after the _int_free_chunk\ncalls.\n---\n malloc/malloc.c | 66 +++++++++++++++++++++++++++++++++++++++++++------\n 1 file changed, 58 insertions(+), 8 deletions(-)","diff":"diff --git a/malloc/malloc.c b/malloc/malloc.c\nindex be29929993..d2184c8d7a 100644\n--- a/malloc/malloc.c\n+++ b/malloc/malloc.c\n@@ -3037,7 +3037,8 @@ tcache_put_n (mchunkptr chunk, size_t tc_idx, tcache_entry **ep, bool mangled)\n available chunks to remove. Removes chunk from the middle of the\n list. */\n static __always_inline void *\n-tcache_get_n (size_t tc_idx, tcache_entry **ep, bool mangled)\n+tcache_get_n (tcache_perthread_struct *tc, size_t tc_idx, tcache_entry **ep,\n+\t bool mangled)\n {\n tcache_entry *e;\n if (!mangled)\n@@ -3053,7 +3054,7 @@ tcache_get_n (size_t tc_idx, tcache_entry **ep, bool mangled)\n else\n *ep = PROTECT_PTR (ep, REVEAL_PTR (e->next));\n \n- ++(tcache->num_slots[tc_idx]);\n+ ++(tc->num_slots[tc_idx]);\n e->key = 0;\n return (void *) e;\n }\n@@ -3068,7 +3069,7 @@ tcache_put (mchunkptr chunk, size_t tc_idx)\n static __always_inline void *\n tcache_get (size_t tc_idx)\n {\n- return tcache_get_n (tc_idx, &tcache->entries[tc_idx], false);\n+ return tcache_get_n (tcache, tc_idx, &tcache->entries[tc_idx], false);\n }\n \n static __always_inline tcache_entry **\n@@ -3111,7 +3112,7 @@ tcache_get_large (size_t tc_idx, size_t nb)\n if (te == NULL || nb != chunksize (mem2chunk (te)))\n return NULL;\n \n- return tcache_get_n (tc_idx, entry, mangled);\n+ return tcache_get_n (tcache, tc_idx, entry, mangled);\n }\n \n static void tcache_init (mstate av);\n@@ -3149,7 +3150,7 @@ tcache_get_align (size_t nb, size_t alignment)\n if (te != NULL\n \t && csize == nb\n \t && PTR_IS_ALIGNED (te, alignment))\n-\treturn tag_new_usable (tcache_get_n (tc_idx, tep, mangled));\n+\treturn tag_new_usable (tcache_get_n (tcache, tc_idx, tep, mangled));\n DIAG_POP_NEEDS_COMMENT;\n }\n return NULL;\n@@ -3332,6 +3333,49 @@ tcache_free_init (void *mem)\n __libc_free (mem);\n }\n \n+/* Deallocate half of the tcache entries into arenas, to amortize the\n+ locking overhead. */\n+static __attribute_noinline__ void\n+__libc_free_batched (mchunkptr p, INTERNAL_SIZE_T size,\n+\t\t tcache_perthread_struct *tc, size_t tc_idx)\n+{\n+ /* Check size >= MINSIZE and p + size does not overflow. */\n+ if (__glibc_unlikely (INT_ADD_OVERFLOW ((uintptr_t) p,\n+\t\t\t\t\t size - MINSIZE)))\n+ return malloc_printerr_tail (\"free(): invalid size (batch)\");\n+\n+ /* Empty half of the tcache, for a hysteresis effect. */\n+ unsigned int to_free = mp_.tcache_count / 2;\n+\n+ /* If the arena does not change between chunks, keep the lock. */\n+ mstate av = arena_for_chunk (p);\n+ __libc_lock_lock (av->mutex);\n+ _int_free_chunk (av, p, size, true);\n+\n+ while (tc->entries[tc_idx] != NULL && to_free > 0)\n+ {\n+ void *mem = tcache_get_n (tc, tc_idx, &tc->entries[tc_idx], false);\n+ p = mem2chunk (mem);\n+ size = chunksize (p);\n+\n+ /* Lock a different arena if necessary. */\n+ {\n+ mstate chunk_av = arena_for_chunk (p);\n+ if (chunk_av != av)\n+\t{\n+\t __libc_lock_unlock (av->mutex);\n+\t av = chunk_av;\n+\t __libc_lock_lock (av->mutex);\n+\t}\n+ }\n+\n+ _int_free_chunk (av, p, size, true);\n+ to_free--;\n+ }\n+\n+ __libc_lock_unlock (av->mutex);\n+}\n+\n void\n __libc_free (void *mem)\n {\n@@ -3370,6 +3414,13 @@ __libc_free (void *mem)\n \t{\n if (__glibc_likely (tcache->num_slots[tc_idx] != 0))\n \t return tcache_put (p, tc_idx);\n+ else\n+\t {\n+\t /* Perform batched freeing of tcache entries. */\n+\t if (__glibc_unlikely (tcache_inactive ()))\n+\t\treturn tcache_free_init (mem);\n+\t return __libc_free_batched (p, size, tcache, tc_idx);\n+\t }\n \t}\n else\n \t{\n@@ -3377,10 +3428,9 @@ __libc_free (void *mem)\n \t if (size >= MINSIZE\n && __glibc_likely (tcache->num_slots[tc_idx] != 0))\n \t return tcache_put_large (p, tc_idx);\n+\t if (__glibc_unlikely (tcache_inactive ()))\n+\t return tcache_free_init (mem);\n \t}\n-\n- if (__glibc_unlikely (tcache_inactive ()))\n-\treturn tcache_free_init (mem);\n }\n #endif\n \n","prefixes":["1/3"]}