Cover Letter Detail
Show a cover letter.
GET /api/1.2/covers/2234977/?format=api
{ "id": 2234977, "url": "http://patchwork.ozlabs.org/api/1.2/covers/2234977/?format=api", "web_url": "http://patchwork.ozlabs.org/project/linux-ext4/cover/20260508121539.4174601-1-libaokun@linux.alibaba.com/", "project": { "id": 8, "url": "http://patchwork.ozlabs.org/api/1.2/projects/8/?format=api", "name": "Linux ext4 filesystem development", "link_name": "linux-ext4", "list_id": "linux-ext4.vger.kernel.org", "list_email": "linux-ext4@vger.kernel.org", "web_url": null, "scm_url": null, "webscm_url": null, "list_archive_url": "", "list_archive_url_format": "", "commit_url_format": "" }, "msgid": "<20260508121539.4174601-1-libaokun@linux.alibaba.com>", "list_archive_url": null, "date": "2026-05-08T12:15:22", "name": "[RFC,00/17] ext4/lib-crc: LBS performance part 1 - incremental CRC32c for bitmap checksums", "submitter": { "id": 92757, "url": "http://patchwork.ozlabs.org/api/1.2/people/92757/?format=api", "name": "Baokun Li", "email": "libaokun@linux.alibaba.com" }, "mbox": "http://patchwork.ozlabs.org/project/linux-ext4/cover/20260508121539.4174601-1-libaokun@linux.alibaba.com/mbox/", "series": [ { "id": 503377, "url": "http://patchwork.ozlabs.org/api/1.2/series/503377/?format=api", "web_url": "http://patchwork.ozlabs.org/project/linux-ext4/list/?series=503377", "date": "2026-05-08T12:15:22", "name": "ext4/lib-crc: LBS performance part 1 - incremental CRC32c for bitmap checksums", "version": 1, "mbox": "http://patchwork.ozlabs.org/series/503377/mbox/" } ], "comments": "http://patchwork.ozlabs.org/api/covers/2234977/comments/", "headers": { "Return-Path": "\n <SRS0=fP7q=DF=vger.kernel.org=linux-ext4+bounces-16363-patchwork-incoming=ozlabs.org@ozlabs.org>", "X-Original-To": [ "incoming@patchwork.ozlabs.org", "linux-ext4@vger.kernel.org" ], "Delivered-To": [ "patchwork-incoming@legolas.ozlabs.org", "patchwork-incoming@ozlabs.org" ], "Authentication-Results": [ "legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=linux.alibaba.com header.i=@linux.alibaba.com\n header.a=rsa-sha256 header.s=default header.b=TbGA0pz8;\n\tdkim-atps=neutral", "legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org\n (client-ip=2404:9400:2221:ea00::3; helo=mail.ozlabs.org;\n envelope-from=srs0=fp7q=df=vger.kernel.org=linux-ext4+bounces-16363-patchwork-incoming=ozlabs.org@ozlabs.org;\n receiver=patchwork.ozlabs.org)", "gandalf.ozlabs.org;\n arc=pass smtp.remote-ip=172.105.105.114 arc.chain=subspace.kernel.org", "gandalf.ozlabs.org;\n dmarc=pass (p=none dis=none) header.from=linux.alibaba.com", "gandalf.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=linux.alibaba.com header.i=@linux.alibaba.com\n header.a=rsa-sha256 header.s=default header.b=TbGA0pz8;\n\tdkim-atps=neutral", "gandalf.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org\n (client-ip=172.105.105.114; helo=tor.lore.kernel.org;\n envelope-from=linux-ext4+bounces-16363-patchwork-incoming=ozlabs.org@vger.kernel.org;\n receiver=ozlabs.org)", "smtp.subspace.kernel.org;\n\tdkim=pass (1024-bit key) header.d=linux.alibaba.com\n header.i=@linux.alibaba.com header.b=\"TbGA0pz8\"", "smtp.subspace.kernel.org;\n arc=none smtp.client-ip=115.124.30.112", "smtp.subspace.kernel.org;\n dmarc=pass (p=none dis=none) header.from=linux.alibaba.com", "smtp.subspace.kernel.org;\n spf=pass smtp.mailfrom=linux.alibaba.com" ], "Received": [ "from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4gBp4k4qLcz1yJq\n\tfor <incoming@patchwork.ozlabs.org>; Fri, 08 May 2026 22:16:22 +1000 (AEST)", "from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3])\n\tby gandalf.ozlabs.org (Postfix) with ESMTP id 4gBp4d0g5Zz4wCG\n\tfor <incoming@patchwork.ozlabs.org>; Fri, 08 May 2026 22:16:17 +1000 (AEST)", "by gandalf.ozlabs.org (Postfix)\n\tid 4gBp4d0Lq4z4wJg; Fri, 08 May 2026 22:16:17 +1000 (AEST)", "from tor.lore.kernel.org (tor.lore.kernel.org [172.105.105.114])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519)\n\t(No client certificate requested)\n\tby gandalf.ozlabs.org (Postfix) with ESMTPS id 4gBp4X1CcKz4wCG\n\tfor <patchwork-incoming@ozlabs.org>; Fri, 08 May 2026 22:16:12 +1000 (AEST)", "from smtp.subspace.kernel.org (conduit.subspace.kernel.org\n [100.90.174.1])\n\tby tor.lore.kernel.org (Postfix) with ESMTP id EACAA30036C1\n\tfor <patchwork-incoming@ozlabs.org>; Fri, 8 May 2026 12:16:07 +0000 (UTC)", "from localhost.localdomain (localhost.localdomain [127.0.0.1])\n\tby smtp.subspace.kernel.org (Postfix) with ESMTP id B986A382F00;\n\tFri, 8 May 2026 12:16:06 +0000 (UTC)", "from out30-112.freemail.mail.aliyun.com\n (out30-112.freemail.mail.aliyun.com [115.124.30.112])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby smtp.subspace.kernel.org (Postfix) with ESMTPS id 857F62CCC5;\n\tFri, 8 May 2026 12:16:02 +0000 (UTC)", "from\n x31h02109.sqa.na131.tbsite.net(mailfrom:libaokun@linux.alibaba.com\n fp:SMTPD_---0X2XQgm8_1778242552 cluster:ay36)\n by smtp.aliyun-inc.com;\n Fri, 08 May 2026 20:15:59 +0800" ], "ARC-Seal": [ "i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1778242577; cv=pass;\n\tb=Ds2opdgebvbDp0yWbX0o+BjNKUFKL/w/oyKSXuFxdTAHYLbfFUdkIonZy4rc6kU4TcUicckjc0TxXUz/b7xeFtpWXtCvkJjoyiba9fCq7pODVNK+QXd0nNqSgeQlIpbe3tCk7zslrhBXAHoabEv8n/bWB4v8mFofwAgrJGBcaF27YUc0tmThwQ7BNtKGTAA3g+EfBKqIYVGa9z5aQ1RVPwV+/IIlmgsABPWecN147JI0FrfCCcDKvt5Bgb/QE6hnCUxCocShxItRGNYGFa31VMSoQOBIAqhSy8ehK2ITLn9qB63PbdtgDWK/dDku+oaUNRMqHq26CMn1bjcKRIJxEw==", "i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;\n\tt=1778242566; cv=none;\n b=n0ucO8XsbowQAYR4vt9fEhoBaOR0fffh1elmjTF4xswiHx+OTCh4eYRDqyRLwepZmvTsZvXwv02l7XLkdKVlHG83JXzcI4uTRVCJoipFO1se1FVRSJqB069GyEM6nN894P5MsoMe/aWNVTm5Mn63BkR0dEY5zCKndHH4sYznfxA=" ], "ARC-Message-Signature": [ "i=2; a=rsa-sha256; d=ozlabs.org; s=201707;\n\tt=1778242577; c=relaxed/relaxed;\n\tbh=NWVI2Pca5vyUf9CuT4wxOcKQk4fg2U6TvmcqynkakJw=;\n\th=From:To:Cc:Subject:Date:Message-ID:MIME-Version;\n b=Xh1YWU9tbkf6EFrspN9KZipztFkb2mupeYDlJa6D0ifviX2mTRZecCfHQ3W5/6X8UMb8TAPikwfBxVcqUYkVY+xYoAnn9vphpxCbwEjiKT1KSra4Sek9bFbaM+YMlUqDowuR+fZpq1Gr9nud2s6k9/0jQGLsP7ZRSWJlVEk0Tw41md3q5Oh6n/jhNZC1GYzoafT0ViRuOLTxXx8noWkpA5ptOQExQlIMR39uhGafp0NAXNKwVfgwLfeXPGZhV1v6gNjyodeooeXQvvVLXnrFI/5D2hzqnrZfIv9kp32xL6lxjofUoC689MJnOzaTtJAEaH6EEhTLSEc7w2z3rpD82Q==", "i=1; a=rsa-sha256; d=subspace.kernel.org;\n\ts=arc-20240116; t=1778242566; c=relaxed/simple;\n\tbh=falxqjSgtJU5OoekjSpFJgMDwQOEwt8+hPtuMlLhPX0=;\n\th=From:To:Cc:Subject:Date:Message-ID:MIME-Version;\n b=TuTKO0VD3ABhp957pQLzipP9ATjBuAzMuNgtW8DgPzugny0XmwEsgXWCABkgnZAMLEy6TbAz0FSGQ3bFsIRLTBVo2Jd4dSFfE4nCGoVLAiEUHs9SyDAFOC8vkptqhS7ZIRTeac3JhZM4dECLwrmizVFk2NrnIFzDaqpmRsHuO3I=" ], "ARC-Authentication-Results": [ "i=2; gandalf.ozlabs.org;\n dmarc=pass (p=none dis=none) header.from=linux.alibaba.com;\n dkim=pass (1024-bit key;\n unprotected) header.d=linux.alibaba.com header.i=@linux.alibaba.com\n header.a=rsa-sha256 header.s=default header.b=TbGA0pz8; dkim-atps=neutral;\n spf=pass (client-ip=172.105.105.114; helo=tor.lore.kernel.org;\n envelope-from=linux-ext4+bounces-16363-patchwork-incoming=ozlabs.org@vger.kernel.org;\n receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org", "i=1; smtp.subspace.kernel.org;\n dmarc=pass (p=none dis=none) header.from=linux.alibaba.com;\n spf=pass smtp.mailfrom=linux.alibaba.com;\n dkim=pass (1024-bit key) header.d=linux.alibaba.com\n header.i=@linux.alibaba.com header.b=TbGA0pz8;\n arc=none smtp.client-ip=115.124.30.112" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=linux.alibaba.com; s=default;\n\tt=1778242559; h=From:To:Subject:Date:Message-ID:MIME-Version;\n\tbh=NWVI2Pca5vyUf9CuT4wxOcKQk4fg2U6TvmcqynkakJw=;\n\tb=TbGA0pz8pwVkATazJqlnTZT8o5/ajwsv7w+K75VoAHbfIBgJtiY/uzd7e/N7PiPmZz97bxxWdTfoFNDBE43sd/TChxyd6VeCSP+rw4YVfx2U8UFjBAsLXa54DiZYdfQ4q1kCISR6k2/AOks4WR4szIk/idnroa6CAanYNzDpYps=", "X-Alimail-AntiSpam": "\n AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033045098064;MF=libaokun@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0X2XQgm8_1778242552;", "From": "Baokun Li <libaokun@linux.alibaba.com>", "To": "linux-ext4@vger.kernel.org", "Cc": "linux-crypto@vger.kernel.org,\n\tebiggers@kernel.org,\n\tardb@kernel.org,\n\ttytso@mit.edu,\n\tadilger.kernel@dilger.ca,\n\tjack@suse.cz,\n\tyi.zhang@huawei.com,\n\tojaswin@linux.ibm.com,\n\tritesh.list@gmail.com,\n\tBaokun Li <libaokun@linux.alibaba.com>", "Subject": "[PATCH RFC 00/17] ext4/lib-crc: LBS performance part 1 - incremental\n CRC32c for bitmap checksums", "Date": "Fri, 8 May 2026 20:15:22 +0800", "Message-ID": "<20260508121539.4174601-1-libaokun@linux.alibaba.com>", "X-Mailer": "git-send-email 2.43.7", "Precedence": "bulk", "X-Mailing-List": "linux-ext4@vger.kernel.org", "List-Id": "<linux-ext4.vger.kernel.org>", "List-Subscribe": "<mailto:linux-ext4+subscribe@vger.kernel.org>", "List-Unsubscribe": "<mailto:linux-ext4+unsubscribe@vger.kernel.org>", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "X-Spam-Status": "No, score=-8.7 required=5.0 tests=ARC_SIGNED,ARC_VALID,\n\tDKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DMARC_PASS,\n\tHEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,\n\tSPF_PASS,UNPARSEABLE_RELAY,USER_IN_DEF_DKIM_WL autolearn=disabled\n\tversion=4.0.1", "X-Spam-Checker-Version": "SpamAssassin 4.0.1 (2024-03-25) on gandalf.ozlabs.org" }, "content": "Motivation\n==========\n\nIn [1] we added large block size (LBS) support to ext4. After enabling\nLBS we observed several performance bottlenecks:\n\n 1. Checksum computation (bitmap, extent block, dir block) becomes\n significantly more expensive as the block size grows.\n 2. Free-bit searches (_find_next_bit) over large bitmaps become costly.\n\nCRC32c is linear over GF(2), so when a contiguous range of bits in a\nbuffer is flipped the new checksum can be derived from the old one\nwithout re-scanning the entire buffer:\n\n New_CRC = Old_CRC ^ CRC(flip_mask << trailing_bits)\n\nThis series introduces crc32c_flip_range() in lib/crc and applies it to\next4's inode and block bitmap checksum paths, reducing checksum overhead\nfrom O(N) to O(log N) per update.\n\nFor dir blocks and extent blocks, each modification touches a 12-264\nbyte region; by computing the CRC of the modified region before and\nafter the change we can derive the delta to the overall checksum. A\ncrc32c_splice() API implementing this approach has been developed\nlocally and will be posted shortly.\n\nFor the _find_next_bit bottleneck under LBS, a per-block-group free\nspace rb-tree can accelerate lookups. A local prototype exists and is\nstill being tested; feedback and alternative approaches are welcome.\n\n[1]: https://lore.kernel.org/all/20251121090654.631996-1-libaokun@huaweicloud.com\n\nBenchmark (full roadmap projection)\n====================================\n\nSingle-process sequential fallocate of 64K blocks. All throughput\nvalues are in GB/s; percentages in parentheses show improvement over\nthe unpatched baseline. \"+crc_splice\" and \"+free_space_tree\" columns\nshow expected gains from follow-up series (not included here).\n\n * Blocks per group up to 65528 (default e2fsprogs limit)\n +--------+---------+-----------------+-----------------+---------------------+\n | Blksz | Before | +crc_flip_range | +crc_splice | +free_space_tree |\n +--------+---------+-----------------+-----------------+---------------------+\n | 1k | 14.9 | 15.0 (+0.7%) | 15.2 (+2.0%) | 15.3 (+2.7%) |\n | 2k | 17.5 | 17.8 (+1.7%) | 18.2 (+4.0%) | 18.7 (+6.9%) |\n | 4k | 16.8 | 17.4 (+3.6%) | 18.3 (+8.9%) | 18.4 (+9.5%) |\n | 8k | 15.5 | 16.5 (+6.5%) | 18.3 (+18.1%) | 18.2 (+17.4%) |\n | 16k | 12.6 | 13.2 (+4.8%) | 15.9 (+26.2%) | 15.9 (+26.2%) |\n | 32k | 8.99 | 9.60 (+6.8%) | 12.3 (+36.8%) | 12.5 (+39.0%) |\n | 64k | 8.24 | 8.54 (+3.6%) | 14.0 (+69.9%) | 19.4 (+135%) |\n +--------+---------+-----------------+-----------------+---------------------+\n\n * Blocks per group up to 524288 (e2fsprogs limit lifted)\n +--------+---------+-----------------+-----------------+---------------------+\n | Blksz | Before | +crc_flip_range | +crc_splice | +free_space_tree |\n +--------+---------+-----------------+-----------------+---------------------+\n | 1k | 15.0 | 14.9 (-0.7%) | 15.5 (+3.3%) | 15.6 (+4.0%) |\n | 2k | 17.4 | 17.7 (+1.7%) | 17.9 (+2.9%) | 18.2 (+4.6%) |\n | 4k | 16.7 | 17.3 (+3.6%) | 18.4 (+10.2%) | 18.7 (+12.0%) |\n | 8k | 15.7 | 16.4 (+4.5%) | 19.1 (+21.7%) | 19.3 (+22.9%) |\n | 16k | 13.5 | 15.4 (+14.1%) | 18.7 (+38.5%) | 19.0 (+40.7%) |\n | 32k | 9.64 | 12.3 (+27.6%) | 17.7 (+83.5%) | 17.7 (+83.5%) |\n | 64k | 2.84 | 3.17 (+11.6%) | 3.48 (+22.5%) | 19.8 (+597%) |\n +--------+---------+-----------------+-----------------+---------------------+\n\nPatch Overview\n==============\n\n * Patches 1-3 (lib/crc): Introduce crc32c_flip_range() with O(log N)\n complexity using precomputed GF(2) shift matrices and nibble-indexed\n lookup tables (~9.8KB, fits in L1 cache). Add kunit tests and\n benchmarks.\n\n * Patch 4: Fix incorrect free clusters accounting when allocated blocks\n overlap with filesystem metadata on a corrupted filesystem.\n\n * Patches 5-7: Extract block bitmap checksum helpers, add the\n incremental update wrapper ext4_block_bitmap_csum_set_range(), and\n use it in ext4_mb_mark_context().\n\n * Patches 8-10: Extract inode bitmap checksum helpers, add\n ext4_inode_bitmap_csum_set_fast(), and use it in ext4_free_inode().\n\n * Patch 11: Fix missing bg_used_dirs_count update during fast commit\n replay.\n\n * Patches 12-13: Factor out ext4_might_init_block_bitmap() and merge\n bitmap modification with GDP update under a single group lock in\n ext4_mark_inode_used(), eliminating a race window.\n\n * Patches 14-15: Rename 'ino' to 'bit' in __ext4_new_inode() for\n clarity, then merge bitmap modification and GDP update under a\n single group lock with incremental CRC.\n\n * Patches 16-17: Extract ext4_update_inode_group_desc() and\n ext4_get_flex_group() helpers to reduce code duplication.\n\nTesting\n=======\n\n\"kvm-xfstests -c ext4/all -g auto\" has been executed with no new failures.\n\ncrc32c_flip_range() micro-benchmark on Intel Xeon (Ice Lake) with\nCRC32c hardware acceleration:\n\n bitmap: 1024 2048 4096 8192 16384 32768 65536\n flip(ns): 48 53 57 63 68 73 78\n full(ns): 45 88 182 357 709 1421 2853\n speedup: 0.9x 1.6x 3.1x 5.6x 10.3x 19.3x 36.3x\n\nComments and questions are, as always, welcome.\n\nThanks,\nBaokun\n\n\nBaokun Li (17):\n lib/crc: add crc32c_flip_range() for incremental CRC update\n lib/crc: crc_kunit: add kunit test for crc32c_flip_range()\n lib/crc: crc_kunit: add benchmark for crc32c_flip_range()\n ext4: fix incorrect block bitmap free clusters update on metadata\n overlap\n ext4: extract block bitmap checksum get and store helpers\n ext4: add ext4_block_bitmap_csum_set_range() for incremental checksum\n update\n ext4: use fast incremental CRC update in ext4_mb_mark_context()\n ext4: extract inode bitmap checksum get and store helpers\n ext4: add ext4_inode_bitmap_csum_set_fast() for incremental checksum\n update\n ext4: use fast incremental CRC update in ext4_free_inode()\n ext4: fix missing bg_used_dirs_count update in fast commit replay\n ext4: factor out ext4_might_init_block_bitmap() helper\n ext4: use fast incremental CRC update in ext4_mark_inode_used()\n ext4: rename ino to bit in __ext4_new_inode()\n ext4: use fast incremental CRC update in __ext4_new_inode()\n ext4: extract ext4_update_inode_group_desc() to reduce duplication\n ext4: add ext4_get_flex_group() helper to simplify flex group lookups\n\n fs/ext4/bitmap.c | 109 ++++++++--\n fs/ext4/ext4.h | 15 +-\n fs/ext4/fast_commit.c | 13 +-\n fs/ext4/ialloc.c | 343 ++++++++++++++------------------\n fs/ext4/mballoc.c | 28 ++-\n fs/ext4/resize.c | 4 +-\n fs/ext4/super.c | 4 +-\n include/linux/crc32.h | 25 +++\n lib/crc/.gitignore | 2 +\n lib/crc/Makefile | 13 +-\n lib/crc/crc32c-incr.c | 140 +++++++++++++\n lib/crc/gen_crc32c_incr_table.c | 141 +++++++++++++\n lib/crc/tests/crc_kunit.c | 137 +++++++++++++\n 13 files changed, 746 insertions(+), 228 deletions(-)\n create mode 100644 lib/crc/crc32c-incr.c\n create mode 100644 lib/crc/gen_crc32c_incr_table.c" }