{"id":2232233,"url":"http://patchwork.ozlabs.org/api/patches/2232233/?format=json","web_url":"http://patchwork.ozlabs.org/project/gcc/patch/CAOyqgcXBt6jfx-FO28tP8wVbuX4QnAXLMWicLgQ8K6EErDrS6Q@mail.gmail.com/","project":{"id":17,"url":"http://patchwork.ozlabs.org/api/projects/17/?format=json","name":"GNU Compiler Collection","link_name":"gcc","list_id":"gcc-patches.gcc.gnu.org","list_email":"gcc-patches@gcc.gnu.org","web_url":null,"scm_url":null,"webscm_url":null,"list_archive_url":"","list_archive_url_format":"","commit_url_format":""},"msgid":"","list_archive_url":null,"date":"2026-05-04T01:03:14","name":"libbacktrace patch committed: Decompress multiple zstd frames","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"84e0d5a93b9558ec9135a69de53b38a546c73a88","submitter":{"id":36501,"url":"http://patchwork.ozlabs.org/api/people/36501/?format=json","name":"Ian Lance Taylor","email":"iant@golang.org"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/gcc/patch/CAOyqgcXBt6jfx-FO28tP8wVbuX4QnAXLMWicLgQ8K6EErDrS6Q@mail.gmail.com/mbox/","series":[{"id":502600,"url":"http://patchwork.ozlabs.org/api/series/502600/?format=json","web_url":"http://patchwork.ozlabs.org/project/gcc/list/?series=502600","date":"2026-05-04T01:03:14","name":"libbacktrace patch committed: Decompress multiple zstd frames","version":1,"mbox":"http://patchwork.ozlabs.org/series/502600/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/2232233/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/2232233/checks/","tags":{},"related":[],"headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","gcc-patches@gcc.gnu.org"],"Delivered-To":["patchwork-incoming@legolas.ozlabs.org","gcc-patches@gcc.gnu.org"],"Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=golang-org.20251104.gappssmtp.com\n header.i=@golang-org.20251104.gappssmtp.com header.a=rsa-sha256\n header.s=20251104 header.b=tcxhbTXv;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org\n (client-ip=2620:52:6:3111::32; helo=vm01.sourceware.org;\n envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org;\n receiver=patchwork.ozlabs.org)","sourceware.org;\n\tdkim=pass (2048-bit key,\n unprotected) header.d=golang-org.20251104.gappssmtp.com\n header.i=@golang-org.20251104.gappssmtp.com header.a=rsa-sha256\n header.s=20251104 header.b=tcxhbTXv","sourceware.org;\n dmarc=pass (p=none dis=none) header.from=golang.org","sourceware.org; spf=pass smtp.mailfrom=golang.org","server2.sourceware.org;\n arc=pass smtp.remote-ip=2607:f8b0:4864:20::b131"],"Received":["from vm01.sourceware.org (vm01.sourceware.org\n [IPv6:2620:52:6:3111::32])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4g83Lk34cVz1yJ9\n\tfor ; Mon, 04 May 2026 11:03:56 +1000 (AEST)","from vm01.sourceware.org (localhost [127.0.0.1])\n\tby sourceware.org (Postfix) with ESMTP id 64E2E4BB1C05\n\tfor ; Mon, 4 May 2026 01:03:54 +0000 (GMT)","from mail-yx1-xb131.google.com (mail-yx1-xb131.google.com\n [IPv6:2607:f8b0:4864:20::b131])\n by sourceware.org (Postfix) with ESMTPS id 3D7D44BAE7D8\n for ; Mon, 4 May 2026 01:03:27 +0000 (GMT)","by mail-yx1-xb131.google.com with SMTP id\n 956f58d0204a3-65c0bda7f15so3448099d50.0\n for ; Sun, 03 May 2026 18:03:27 -0700 (PDT)"],"DKIM-Filter":["OpenDKIM Filter v2.11.0 sourceware.org 64E2E4BB1C05","OpenDKIM Filter v2.11.0 sourceware.org 3D7D44BAE7D8"],"DMARC-Filter":"OpenDMARC Filter v1.4.2 sourceware.org 3D7D44BAE7D8","ARC-Filter":"OpenARC Filter v1.0.0 sourceware.org 3D7D44BAE7D8","ARC-Seal":["i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1777856607; cv=pass;\n b=fpvwaYd68yebsUFxp7WljdlOv+jweNdaHLJwpmoeLdR+16eHpLr0MCUiz4UDkqMRqCAcj6Ya3QssCnI5f0bBVxIEYYZT7LT+ZBMATIp4jngH23u9SVac3yqWqKvXFuDxX4fYBBmpAfZDbr1xflfYpzrEnbW34CJnlA1qFfGpbqE=","i=1; a=rsa-sha256; t=1777856606; cv=none;\n d=google.com; s=arc-20240605;\n b=YVa+rkm/ND5EEdpdcMH+kKkTZJw2lU8I/w0Phu8wxUe99g9yitqy+O2lgREUDKHLdj\n 6C90YmoTC3ZZhmklMOL0QrLHO52lwVZQflZ/KlmLEUWgTsq/7hwaiLp2A7yQdyQf0ZxC\n 0LVQInzOw0vJ/BQ4rwqmLF+Qbc3OGgHIw57H64xIlo1VMTuKV7oy4EocP9r2VgG7QHlc\n 161GkXb9pUJsA72fuamX0mCsTfeUtFkMFTjXFfIq2XgzmABJ5drs/soPukH0Caj3U0+J\n +3/gZ4ui72tqCaysBurCIqJjGnZJZHeqjD2ETbvnL8pPi8L9h/+rVDN6dbEM4mlKgpWH\n 1XrA=="],"ARC-Message-Signature":["i=2; a=rsa-sha256; d=sourceware.org; s=key;\n t=1777856607; c=relaxed/simple;\n bh=7sWmHBDMgUh2p71JA0dsWDn0igOIpzhTsAPHrDRe15A=;\n h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To;\n b=RDpa3xE/6GIx/+vIwrzB0rjYq1HlvNvJE4ZcqT9xCBPEwx3UWrPNDpwnhcZX6abn3vIJIEFCFEalT3QYP808lBZF8WbU/9ZPLxA10ACl2UZvggNrbkDSiM9dQ/r2s2Q8E6qtWvS+suGQhU3gOa+aOQtL8W5lcMGAl7eaaOr4ifc=","i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;\n s=arc-20240605;\n h=to:subject:message-id:date:from:mime-version:dkim-signature;\n bh=UF5CQzQyQm/FNeR2mDKyTfU8aKf8mgp/ckNqt1FGWNY=;\n fh=8G0fvw9kezX2+yjwifXgHvTII1omy92vRpR1R+rFCOA=;\n b=cbot5yFQLZtafwpyhfhGxKkdDuR/rp7YdNx/JDFzISL52jvi71ilaap4/+MB8vfagq\n Fpuj/NtW+6o87uoiEC6rzhNQn/gzKa9ABZ9+SFSpPZ7rr8SxHvwUHkthArUwH1WV/gr4\n dwT1NpH6dTzdTOgCVH5AKF5tvOvURiQVUA7u+ln+ayZdZc6ye8131JtG3GcV9CpkwqN5\n QsqoF1pKZYtdON/ODojhvtj6/SJaCUYXVqYqRww7ka4D1IdiLQ6EXDnqRpmG5IzV+XCz\n 1iQzzeUN4jGMHjYSEqIV5LLEfdZe3yXmZmK/QZoUjEzAIOvpfoQPWEimosvhzLw7i2bl\n pO/g==; darn=gcc.gnu.org"],"ARC-Authentication-Results":["i=2; server2.sourceware.org","i=1; mx.google.com; arc=none"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=golang-org.20251104.gappssmtp.com; s=20251104; t=1777856606; x=1778461406;\n darn=gcc.gnu.org;\n h=to:subject:message-id:date:from:mime-version:from:to:cc:subject\n :date:message-id:reply-to;\n bh=UF5CQzQyQm/FNeR2mDKyTfU8aKf8mgp/ckNqt1FGWNY=;\n b=tcxhbTXvjYDTyGHLkJR1a5zsv2yAumd5wWodx9LJmfJS0+0RKtuKf/Gk6ybMCs7Vel\n 5ztsaPC77gw4WVJEWSznznlPB3PD86bj/AvpnctZOXqgIxMpZSQsZrTz1FHxEPY476TN\n RTplWDztHXOL82HQeKmLKnm0nlrTVUibBfXQxicDHqsVjwIZLigHPNGLHcqfwOqgZV1D\n v44r9R6GnfOvTXP7TSh+a7NqODa72FVzTpTI1aZZYpwnIT2e0Bcrek+Ef8chCxUXSje/\n OvJrzRzySiPvb/Tc/JGWZZDCTXdnkRSIDFShVrRhzwIigL/lmWPuCtg0Gf8yCEtWRzRb\n iURQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1777856606; x=1778461406;\n h=to:subject:message-id:date:from:mime-version:x-gm-gg\n :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;\n bh=UF5CQzQyQm/FNeR2mDKyTfU8aKf8mgp/ckNqt1FGWNY=;\n b=qTFqvjF9hedk5aLVQ0CATc8CWzOy66znKcelD8723dwKGGae1wfdvYpmx4/31mMtPO\n LLP9XOzroZhGR2G6xPdJmbEECKCaACxfMVLmClgPWzVX1XglVwgmMKF9M+vetYSdnp4K\n Q0eBUtkY1lU4wK4oSzX1zagmWrvtE7WvAcfVk/se7JPBq2a9DkA8xUUT67uRsxA0y5Q6\n ImJR7Ag9B1mG3Yad1X5iB6cL5QRFlaABpKQM1/JE0vtZxNuVMSydkNLCZIA6DRLV6n6B\n nJRQVT7QZPZCnTwqOFlI9XRmJi8AqhciDjlaEL6mW8EjsYESt58Wr6nGQv25pv9PdW8H\n GQqQ==","X-Gm-Message-State":"AOJu0Yys5c6i7pBdAI8YQIAzBiUGRpuhOvSPzJZLHoLyhfi6v5kEzyUz\n p4xwXbL5auLhHN2BYRK7e8Km8wihIAjaQmj9ZI6tkmW8b+hhdDzj7bEiO5cV9Bp21FlSWVHhTQt\n Z/6u4owVp6ZjbWHAxbyhJ7d4y2vvDoylBBkR9m9m0WiM4FsJ9HJ1Rm2Y=","X-Gm-Gg":"AeBDiesKfwAXKL/opgGxq0EnsJAMmecAb8AuYqjnxegMRTU3PTERyHfLm3Kt/MQb6wA\n j2GiVo33RPpYOe/WbcmUffVE/iphq3obm3C9u5u351iEw2FnKlFHXVUPcYAa6boczXxgygm3UHH\n 9x6kWI0nmvQlLzfRGetuZk/tDaXH7IywLEl7Hja+Ayt8Y4tkjJf/ubdIxMuf+u++zUYWytRsT3Y\n 8TDXzrmYZVriZujH7o3kZwI++EDsbmN79h2VmevkNv9hefQDAaoM6FELWfbTMWidrWOOcaL9f3J\n r+C8F3hsAdYjOEG2N3pc","X-Received":"by 2002:a05:690e:304:b0:64a:e589:ecb2 with SMTP id\n 956f58d0204a3-65c3db9509amr5657100d50.62.1777856605785; Sun, 03 May 2026\n 18:03:25 -0700 (PDT)","MIME-Version":"1.0","From":"Ian Lance Taylor ","Date":"Sun, 3 May 2026 18:03:14 -0700","X-Gm-Features":"AVHnY4L2StUla5lDRaPz5Z4671Gf0iH7oNvGJHo9EnlTkA73LzIaOYcvoikQ8L0","Message-ID":"\n ","Subject":"libbacktrace patch committed: Decompress multiple zstd frames","To":"gcc-patches ","Content-Type":"multipart/mixed; boundary=\"00000000000073595a0650f37f6b\"","X-BeenThere":"gcc-patches@gcc.gnu.org","X-Mailman-Version":"2.1.30","Precedence":"list","List-Id":"Gcc-patches mailing list ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org"},"content":"The libbacktrace support for zstd compression assumed that it would\nonly ever see a single frame. However, lld breaks sections up into 1M\nchunks and compresses them in parallel. This patch, based on a patch\nby the GitHub user ofats, enhances libbacktrace to decompress a series\nof zstd frames. Bootstrapped and ran libbacktrace and libgo tests on\nx86_64-pc-linux-gnu. Committed to mainline.\n\nIan\n\n * elf.c (elf_zstd_decompress_frame): New static function,\n broken out of elf_zstd_decompress.\n (elf_zstd_decompress): Call elf_zstd_decompress_frame in a loop.\n * zstdtest.c (test_large): Compress the file in chunks.","diff":"diff --git a/libbacktrace/elf.c b/libbacktrace/elf.c\nindex 483c2d83bd9..46607f6fb8d 100644\n--- a/libbacktrace/elf.c\n+++ b/libbacktrace/elf.c\n@@ -4338,15 +4338,17 @@ elf_zstd_unpack_seq_decode (int mode,\n return 1;\n }\n \n-/* Decompress a zstd stream from PIN/SIN to POUT/SOUT. Code based on RFC 8878.\n+/* Decompress a single zstd frame from *PPIN, ending at PINEND, to *PPOUT/SOUT.\n Return 1 on success, 0 on error. */\n \n static int\n-elf_zstd_decompress (const unsigned char *pin, size_t sin,\n-\t\t unsigned char *zdebug_table, unsigned char *pout,\n-\t\t size_t sout)\n+elf_zstd_decompress_frame (const unsigned char **ppin,\n+\t\t\t const unsigned char *pinend,\n+\t\t\t unsigned char *zdebug_table, unsigned char **ppout,\n+\t\t\t size_t sout)\n {\n- const unsigned char *pinend;\n+ const unsigned char *pin;\n+ unsigned char *pout;\n unsigned char *poutstart;\n unsigned char *poutend;\n struct elf_zstd_seq_decode literal_decode;\n@@ -4366,9 +4368,9 @@ elf_zstd_decompress (const unsigned char *pin, size_t sin,\n uint64_t content_size;\n int last_block;\n \n- pinend = pin + sin;\n+ pin = *ppin;\n+ pout = *ppout;\n poutstart = pout;\n- poutend = pout + sout;\n \n literal_decode.table = NULL;\n literal_decode.table_bits = -1;\n@@ -4394,7 +4396,7 @@ elf_zstd_decompress (const unsigned char *pin, size_t sin,\n repeated_offset2 = 4;\n repeated_offset3 = 8;\n \n- if (unlikely (sin < 4))\n+ if (unlikely (pinend - pin < 4))\n {\n elf_uncompress_failed ();\n return 0;\n@@ -4492,12 +4494,14 @@ elf_zstd_decompress (const unsigned char *pin, size_t sin,\n }\n \n if (unlikely (content_size != (size_t) content_size\n-\t\t|| (size_t) content_size != sout))\n+\t\t|| (size_t) content_size > sout))\n {\n elf_uncompress_failed ();\n return 0;\n }\n \n+ poutend = pout + content_size;\n+\n last_block = 0;\n while (!last_block)\n {\n@@ -4996,7 +5000,42 @@ elf_zstd_decompress (const unsigned char *pin, size_t sin,\n pin += 4;\n }\n \n- if (pin != pinend)\n+ *ppin = pin;\n+ *ppout = pout;\n+\n+ return 1;\n+}\n+\n+/* Decompress a zstd stream from PIN/SIN to POUT/SOUT. Code based on RFC 8878.\n+ Return 1 on success, 0 on error. */\n+\n+static int\n+elf_zstd_decompress (const unsigned char *pin, size_t sin,\n+\t\t unsigned char *zdebug_table, unsigned char *pout,\n+\t\t size_t sout)\n+{\n+ const unsigned char *pinend;\n+\n+ pinend = pin + sin;\n+\n+ while (sin > 0)\n+ {\n+ const unsigned char *pin_frame;\n+ unsigned char *pout_frame;\n+\n+ pin_frame = pin;\n+ pout_frame = pout;\n+ if (!elf_zstd_decompress_frame (&pin_frame, pinend, zdebug_table,\n+\t\t\t\t &pout_frame, sout))\n+\treturn 0;\n+\n+ sin -= pin_frame - pin;\n+ pin = pin_frame;\n+ sout -= pout_frame - pout;\n+ pout = pout_frame;\n+ }\n+\n+ if (sout > 0)\n {\n elf_uncompress_failed ();\n return 0;\ndiff --git a/libbacktrace/zstdtest.c b/libbacktrace/zstdtest.c\nindex b6334cf5df3..7776c80fc9b 100644\n--- a/libbacktrace/zstdtest.c\n+++ b/libbacktrace/zstdtest.c\n@@ -297,8 +297,8 @@ test_large (struct backtrace_state *state ATTRIBUTE_UNUSED)\n size_t orig_bufsize;\n size_t i;\n char *compressed_buf;\n- size_t compressed_bufsize;\n size_t compressed_size;\n+ size_t chunk_size;\n unsigned char *uncompressed_buf;\n size_t r;\n clockid_t cid;\n@@ -370,22 +370,39 @@ test_large (struct backtrace_state *state ATTRIBUTE_UNUSED)\n return;\n }\n \n- compressed_bufsize = ZSTD_compressBound (orig_bufsize);\n- compressed_buf = malloc (compressed_bufsize);\n- if (compressed_buf == NULL)\n- {\n- perror (\"malloc\");\n- goto fail;\n- }\n+ /* Split the input into 100K chunks. This is to approximate the fact that lld\n+ splits the input into 1M shards. */\n \n- r = ZSTD_compress (compressed_buf, compressed_bufsize,\n-\t\t orig_buf, orig_bufsize, 3);\n- if (ZSTD_isError (r))\n+ compressed_size = 0;\n+ compressed_buf = NULL;\n+ chunk_size = 100 << 10;\n+ for (i = 0; i < orig_bufsize; i += chunk_size)\n {\n- fprintf (stderr, \"zstd compress failed: %s\\n\", ZSTD_getErrorName (r));\n- goto fail;\n+ size_t chunk_input_size;\n+ size_t chunk_compressed_size;\n+\n+ chunk_input_size = orig_bufsize - i;\n+ if (chunk_input_size > chunk_size)\n+\tchunk_input_size = chunk_size;\n+\n+ chunk_compressed_size = ZSTD_compressBound (chunk_input_size);\n+ compressed_buf = realloc (compressed_buf, compressed_size + chunk_compressed_size);\n+ if (compressed_buf == NULL)\n+\t{\n+\t perror (\"realloc\");\n+\t goto fail;\n+\t}\n+\n+ r = ZSTD_compress (compressed_buf + compressed_size,\n+\t\t\t chunk_compressed_size,\n+\t\t\t orig_buf + i, chunk_input_size, 3);\n+ if (ZSTD_isError (r))\n+\t{\n+\t fprintf (stderr, \"zstd compress failed: %s\\n\", ZSTD_getErrorName (r));\n+\t goto fail;\n+\t}\n+ compressed_size += r;\n }\n- compressed_size = r;\n \n uncompressed_buf = malloc (orig_bufsize);\n if (uncompressed_buf == NULL)\n","prefixes":[]}