Cover Letter Detail
Show a cover letter.
GET /api/1.2/covers/2225593/?format=api
{ "id": 2225593, "url": "http://patchwork.ozlabs.org/api/1.2/covers/2225593/?format=api", "web_url": "http://patchwork.ozlabs.org/project/glibc/cover/cover.1776760573.git.xavier.roche@algolia.com/", "project": { "id": 41, "url": "http://patchwork.ozlabs.org/api/1.2/projects/41/?format=api", "name": "GNU C Library", "link_name": "glibc", "list_id": "libc-alpha.sourceware.org", "list_email": "libc-alpha@sourceware.org", "web_url": "", "scm_url": "", "webscm_url": "", "list_archive_url": "", "list_archive_url_format": "", "commit_url_format": "" }, "msgid": "<cover.1776760573.git.xavier.roche@algolia.com>", "list_archive_url": null, "date": "2026-04-21T08:39:11", "name": "[RFC,0/1] malloc: madvise interior free chunks above a threshold", "submitter": { "id": 93202, "url": "http://patchwork.ozlabs.org/api/1.2/people/93202/?format=api", "name": "Xavier Roche", "email": "xavier.roche@algolia.com" }, "mbox": "http://patchwork.ozlabs.org/project/glibc/cover/cover.1776760573.git.xavier.roche@algolia.com/mbox/", "series": [ { "id": 500760, "url": "http://patchwork.ozlabs.org/api/1.2/series/500760/?format=api", "web_url": "http://patchwork.ozlabs.org/project/glibc/list/?series=500760", "date": "2026-04-21T08:39:12", "name": "malloc: madvise interior free chunks above a threshold", "version": 1, "mbox": "http://patchwork.ozlabs.org/series/500760/mbox/" } ], "comments": "http://patchwork.ozlabs.org/api/covers/2225593/comments/", "headers": { "Return-Path": "<libc-alpha-bounces~incoming=patchwork.ozlabs.org@sourceware.org>", "X-Original-To": [ "incoming@patchwork.ozlabs.org", "libc-alpha@sourceware.org" ], "Delivered-To": [ "patchwork-incoming@legolas.ozlabs.org", "libc-alpha@sourceware.org" ], "Authentication-Results": [ "legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=algolia.com header.i=@algolia.com header.a=rsa-sha256\n header.s=google header.b=BSlKVh2O;\n\tdkim-atps=neutral", "legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org\n (client-ip=2620:52:6:3111::32; helo=vm01.sourceware.org;\n envelope-from=libc-alpha-bounces~incoming=patchwork.ozlabs.org@sourceware.org;\n receiver=patchwork.ozlabs.org)", "sourceware.org;\n\tdkim=pass (1024-bit key,\n unprotected) header.d=algolia.com header.i=@algolia.com header.a=rsa-sha256\n header.s=google header.b=BSlKVh2O", "sourceware.org;\n dmarc=pass (p=reject dis=none) header.from=algolia.com", "sourceware.org; spf=pass smtp.mailfrom=algolia.com", "server2.sourceware.org;\n arc=none smtp.remote-ip=2a00:1450:4864:20::32c" ], "Received": [ "from vm01.sourceware.org (vm01.sourceware.org\n [IPv6:2620:52:6:3111::32])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4g0G4v63Qpz1yCv\n\tfor <incoming@patchwork.ozlabs.org>; Tue, 21 Apr 2026 18:39:59 +1000 (AEST)", "from vm01.sourceware.org (localhost [127.0.0.1])\n\tby sourceware.org (Postfix) with ESMTP id D0BA34BA902F\n\tfor <incoming@patchwork.ozlabs.org>; Tue, 21 Apr 2026 08:39:57 +0000 (GMT)", "from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com\n [IPv6:2a00:1450:4864:20::32c])\n by sourceware.org (Postfix) with ESMTPS id 0261C4BA902E\n for <libc-alpha@sourceware.org>; Tue, 21 Apr 2026 08:39:15 +0000 (GMT)", "by mail-wm1-x32c.google.com with SMTP id\n 5b1f17b1804b1-4891f6b6388so995585e9.0\n for <libc-alpha@sourceware.org>; Tue, 21 Apr 2026 01:39:15 -0700 (PDT)", "from xavier-thinkpad.. ([2a01:e0a:1048:49a0:2de4:14b:d3be:34fd])\n by smtp.gmail.com with ESMTPSA id\n 5b1f17b1804b1-488fc17f642sm322503005e9.5.2026.04.21.01.39.13\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Tue, 21 Apr 2026 01:39:13 -0700 (PDT)" ], "DKIM-Filter": [ "OpenDKIM Filter v2.11.0 sourceware.org D0BA34BA902F", "OpenDKIM Filter v2.11.0 sourceware.org 0261C4BA902E" ], "DMARC-Filter": "OpenDMARC Filter v1.4.2 sourceware.org 0261C4BA902E", "ARC-Filter": "OpenARC Filter v1.0.0 sourceware.org 0261C4BA902E", "ARC-Seal": "i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1776760756; cv=none;\n b=fY8kY+6U9vx8RcG5TySMAb040Yvox5YrqEZyvMDxMgES4ruzNud3xPARuZDiky12zWqBKnjG6nz93vLsYBWYSovfzL+Lodz+byZk6prx+l9He+lb8Yvh1vcKU5ewUktrHwQcBzmtT6xsm8aotEjIeYyXLABU5riWCb5L1LhnCAo=", "ARC-Message-Signature": "i=1; a=rsa-sha256; d=sourceware.org; s=key;\n t=1776760756; c=relaxed/simple;\n bh=SWDYrMu77kyFbNKbqJ/zkdpr0u6hjIZeFCY7/hj6vrs=;\n h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version;\n b=scd+NJZ040gcdoMeGujyV4FxhS+EL3n1N2m3Zkai244+htZFTQzX8VGVFsEKu4G8fXo4GWdh0gnQrCZWpyE8rIeFZbAaRcuTFbzCrNMkzYU7Ir0dJTbRYEOPSpEMi9MFzYHvwgZDFeOrCN6b/K8pyQSCNn+ngXvcUcLH5zY+NvM=", "ARC-Authentication-Results": "i=1; server2.sourceware.org", "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=algolia.com; s=google; t=1776760754; x=1777365554; darn=sourceware.org;\n h=content-transfer-encoding:mime-version:message-id:date:subject:cc\n :to:from:from:to:cc:subject:date:message-id:reply-to;\n bh=o98BiPwPE3QOi1V4mzwuZO/HCZEcCvQ1PTAODPKcrwQ=;\n b=BSlKVh2OQWI0jrx/UcrQwePKYLc6tjyQj5ooF1qaK6tObavRoRxYUNGCg+c2XOODAU\n GlibMWmuKHq1d3M/h8HhBUC7G3/J+tiSARcuI+T/lk2RBR+rMrZu5yJvXoEGNjWKgQyh\n 0+9qwWoMIlB0HPFEXtx0vi6QoXs0OQ/1U7fbY=", "X-Google-DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1776760754; x=1777365554;\n h=content-transfer-encoding:mime-version:message-id:date:subject:cc\n :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date\n :message-id:reply-to;\n bh=o98BiPwPE3QOi1V4mzwuZO/HCZEcCvQ1PTAODPKcrwQ=;\n b=olWC/ML3ypZQHbemTzLkgXO3Or0au1a/7GYxwe6Rol/ZtbTxUPK1glATrxQSHHfdXn\n oOQ4AQSIV3xAFcFrk/pIRwHxjN+XT2/tZaNZTI6kvhWdaqCCRUdKryHxuwqxn9+nMcla\n B1mEc0NSgvPpp4lrNn3qFvqrw4mA+mXXyHyYFYilZ9Y+FTULa3g4H4fVDZkgC1lS8rdh\n BriQzx+PBG3IRzQkxFrt+jmT5nn6Bg4PoXsVSfaWTxKIdIWcV0Lt9+gmpvd3/ufPiKvA\n K+X5zH2z7VXxykpNUl2NqEry0u7v9FcOOzUAjCigwJrbS0l3eS8BhYqFAJp8OR2aMJ1X\n k4mA==", "X-Gm-Message-State": "AOJu0YxCjlNyMWdOLli8vCuWXaGAvM0x5i8ExT8XetvpWE/JhMAX0E1C\n 9/wdBGmRJrSom2+fJzM31FhnOZZ66O7UCkmJEtrVWjp3tNJReKpjYa9boT5+7nmfZxVlG0FNK8P\n ueHFj", "X-Gm-Gg": "AeBDiesRBlPcnzUfTy2fmqAOWYzA1uHt8MQP72o21c7hqUXsaFNJ3i2mlmlkfvB1/hB\n BZUOJrAaMhw/Ur6qiLXwpsqCIWezSwav2M0LBaf8VNpqB2gS8EN9bheXavUDJDXe5DHzV8Anr6S\n dyZ0ihX4wxn5n/rC097pWz4KWVW2GtHm2JJWVRr9Q4C0EoEpCtiusiQLf8lgo7Wvh6GJ6/Yf2DB\n mHlGMhVMzJcpjCaGtoAqMxN5Sx5dJMpJCdBEUkK8W0oqK3rREdimGKMhuOcMqZeva0dEI1E907T\n Lq+STA7fAjNUQnnPbQQOsOMCFLHO2+Xl8G2mKyNLt2wxzotSa3pi38D6yYuz0oP5vBCwJA86+Qr\n +k0Mi8uaz2I50auvTWzku7rMYadTz7Oc5ZqzIVSCe8zJww8JH+FmSzBjXDwxfl/iNM7X5MwChnj\n 4Vl1xVho3pO6AkIH4H3JVlZT1eikevBqaX2SXQpM74Czrnzl4szA==", "X-Received": "by 2002:a05:600c:4746:b0:488:ac4b:59d1 with SMTP id\n 5b1f17b1804b1-488fb7ab49cmr115235045e9.8.1776760753852;\n Tue, 21 Apr 2026 01:39:13 -0700 (PDT)", "From": "Xavier Roche <xavier.roche@algolia.com>", "To": "libc-alpha@sourceware.org", "Cc": "wilco.dijkstra@arm.com,\n\tadhemerval.zanella@linaro.org", "Subject": "[RFC PATCH 0/1] malloc: madvise interior free chunks above a\n threshold", "Date": "Tue, 21 Apr 2026 10:39:11 +0200", "Message-ID": "<cover.1776760573.git.xavier.roche@algolia.com>", "X-Mailer": "git-send-email 2.43.0", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "X-BeenThere": "libc-alpha@sourceware.org", "X-Mailman-Version": "2.1.30", "Precedence": "list", "List-Id": "Libc-alpha mailing list <libc-alpha.sourceware.org>", "List-Unsubscribe": "<https://sourceware.org/mailman/options/libc-alpha>,\n <mailto:libc-alpha-request@sourceware.org?subject=unsubscribe>", "List-Archive": "<https://sourceware.org/pipermail/libc-alpha/>", "List-Post": "<mailto:libc-alpha@sourceware.org>", "List-Help": "<mailto:libc-alpha-request@sourceware.org?subject=help>", "List-Subscribe": "<https://sourceware.org/mailman/listinfo/libc-alpha>,\n <mailto:libc-alpha-request@sourceware.org?subject=subscribe>", "Errors-To": "libc-alpha-bounces~incoming=patchwork.ozlabs.org@sourceware.org" }, "content": "Since glibc 2.26 introduced tcache, free() no longer returns\nphysical pages to the kernel for chunks that sit in the interior\nof an arena heap. Periodic malloc_trim(0) recovers the memory,\nbut free() itself cannot. A bisect across 2.23-2.27 with the\nreproducer (attached as malloc/tst-madvise-threshold.c) shows\na 4x RSS jump between 2.25 and 2.26.\n\nBZ #33886 is the latest thread on this. BZ #15321, #18910,\n#27976 describe older aspects of the same behavior. Several\nprojects ship workarounds: systemd, OpenJDK, Python, and our\nown production search engine, which overrides free() and\ntriggers malloc_trim(0) on a background thread.\n\nMechanism\n---------\n_int_free_maybe_trim already runs for consolidated chunks\n>= 64 KB and calls systrim / heap_trim on the top chunk. This\npatch extends it to madvise the page-aligned interior of the\nconsolidated chunk (same page-alignment logic as mtrim()).\n\nAn early version triggered madvise on every qualifying free,\nwhich Wilco Dijkstra pointed out could produce ~1M syscalls\nwhen a million small frees merge into one chunk (BZ #33886\ncomment 10). To address that, the patch uses the caller's\noriginal (pre-merge) chunk size as the gate:\n\n - orig_size >= one page + chunk header: madvise immediately.\n - orig_size smaller: accumulate in a per-arena counter, and\n fire madvise only when the accumulator crosses 256 KB.\n\nIn the worst case (one million 100-byte frees merging into a\n100 MB chunk), this produces ~400 madvise calls instead of ~1M.\nNormal workloads pay one accumulator increment per small free\nand one madvise per 256 KB of pressure.\n\nThe per-arena counter is accessed only under av->mutex\n(verified at both call sites: _int_free_merge_chunk and\n_int_memalign).\n\nFor advice type: MADV_FREE for moderate chunks (lazy release,\nno re-fault on quick reuse); MADV_DONTNEED for chunks >= 128 KB\n(immediate RSS reduction matches what operators expect from\nmtrim). This split is an open design point; simplifying to\nMADV_DONTNEED unconditionally is an option if preferred.\n\nImpact\n------\nReproducer: 16 threads, 256 MB live working set, 10 GB of\nshort-lived allocations below mmap threshold.\n\n baseline patched\n RSS after free 1247 MB 296 MB\n malloc_trim recovery 962 MB 14 MB (free did the work)\n Runtime 0.34 s 0.50 s (tight malloc loop)\n\nRuntime overhead is only visible in synthetic malloc-heavy\nbenchmarks. Real applications spend a fraction of a percent\nof their time in free(); the amortised cost is negligible.\nAlgolia production (the motivating workload) recovers 100+ GB\nof previously-wasted RSS with equivalent semantics.\n\nBackground: https://www.algolia.com/blog/engineering/when-allocators-are-hoarding-your-precious-memory\n\nTesting\n-------\n- malloc test suite: all PASS, including the new\n tst-madvise-threshold that validates the RSS drop.\n- Full make check on x86_64-linux-gnu: no new regressions\n vs baseline.\n- Cross-built via build-many-glibcs.py for\n aarch64-linux-gnu, arm-linux-gnueabihf,\n powerpc64le-linux-gnu, s390x-linux-gnu: check-compilers,\n build and compile-only check passed on all four with no\n new warnings on malloc.c. (i686 is covered by patchwork.)\n\nOpen questions\n--------------\n1. MADV_FREE vs MADV_DONTNEED split at 128 KB: keep the two-tier\n design, or always use MADV_DONTNEED (matches mtrim())?\n2. Default-on vs tunable: the current patch is default-on on the\n argument that this is a regression fix since 2.26. A tunable\n kill-switch (glibc.malloc.madvise_interior, default on) could\n be added if preferred.\n3. Threshold values (64 KB entry gate, 256 KB accumulator, 128 KB\n MADV_DONTNEED boundary) are picked as multiples of\n ATTEMPT_TRIMMING_THRESHOLD; open to tuning.\n\nAcknowledgments\n---------------\nThanks to Wilco Dijkstra for the earlier review on BZ #33886\nthat surfaced the million-madvise-calls concern and redirected\nthe design toward the accumulator approach. The bisect and\ntcache-as-root-cause framing also came out of that discussion.\n\nRelated prior art: Mel Gorman's 2015 RFC\n(https://sourceware.org/legacy-ml/libc-alpha/2015-02/msg00193.html)\nexplored a related madvise refault-avoidance question; this\npatch is narrower in scope.\n\nAI disclosure\n-------------\nAI assistance (Claude) was used during analysis and drafting:\nreading the ptmalloc2 source, reviewing the regression bisect,\nexploring design alternatives, and iterating on the patch text.\nThe design decisions, the benchmark runs, and the final code\nwere reviewed and validated by the author, who takes\nresponsibility for correctness.\n\nXavier Roche (1):\n malloc: madvise interior free chunks above a threshold\n\n malloc/Makefile | 1 +\n malloc/malloc.c | 61 +++++++++++++---\n malloc/tst-madvise-threshold.c | 128 +++++++++++++++++++++++++++++++++\n 3 files changed, 182 insertions(+), 8 deletions(-)\n create mode 100644 malloc/tst-madvise-threshold.c\n\n\nbase-commit: e3e8f814e53d32da99db03ac38ed3af86651b233" }