From patchwork Fri Feb 11 16:41:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 1591732 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=gjNNte4q; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=FSsiyz6A; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JwKD430dHz9sFq for ; Sat, 12 Feb 2022 03:42:44 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4JwKD35cRvz3cSC for ; Sat, 12 Feb 2022 03:42:43 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=gjNNte4q; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=FSsiyz6A; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sent.com (client-ip=66.111.4.221; helo=new1-smtp.messagingengine.com; envelope-from=zi.yan@sent.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=gjNNte4q; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=FSsiyz6A; dkim-atps=neutral Received: from new1-smtp.messagingengine.com (new1-smtp.messagingengine.com [66.111.4.221]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4JwKC13kmdz3bVK for ; Sat, 12 Feb 2022 03:41:48 +1100 (AEDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailnew.nyi.internal (Postfix) with ESMTP id 5D48B580370; Fri, 11 Feb 2022 11:41:46 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Fri, 11 Feb 2022 11:41:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to :reply-to:sender:subject:subject:to:to; s=fm2; bh=VmPT36pm2xq5yd UOBiaL9nltkdkN6llWorno9T9IDyQ=; b=gjNNte4q1d4v+zBAgXRrePdkoW1H4l cp4SLrXaD7/hBHSQU+GTtrSHPrtTpLWBXU0iho6p/fgEzTzkh4Z4iMVfjk16lutA 5j9Hlazcih/ZKI3nvorGmS6ywhZFxBY3EkTbzvXGHtm35zCqe5MBFxzAOHqdLipC kre5T75+9ymWOBNVxEk5uGfafdFEjt9Hri/ZSetb/zQ0a+s5YWqVS21UAPXRo1Dh n6wNHhapury/lMghBN1A/cBrBVz4tCvR7cZGEoeLzce9amd9WETozw8EUfUcjdbc AkTU2QdV649URYhhIkuycO+oDOFp9Vs7r37uPZSR62NJXuCOPWJNXO/A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=VmPT36pm2xq5ydUOBiaL9nltkdkN6llWorno9T9IDyQ=; b=FSsiyz6A EPX4XaXrPUTS1uBVvUOpncx1QpHqeEl4ibTl6B2K2plQPADaWCy+rnMKMLzQY41X gDNl78Vhk9G9RmlFGDo08XEGRFcNktJYTaPNLTnDzcZx3hViFgd/gt1X1BX9PT2h iFc86lsB+ALlm9mCOa5Q9fiPJd4UzIEF9LrANIFJRhhYbok1RdtbdJWi0SeclhXq BHaFSKwUYR5bFy1Fxbd08NwTC6sOYGEz5k7UwvBFPnHf4WIPsBUrTePZsNK/Idj+ I123q92IRISyVAKQEkQ/p3JYwhifNHYBczl8jbI8Gi+TSbJ1K0n+72jm/fJ74uEi 9nt0DkgT7IIQQA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvvddrieefgdeltdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhhrggfgsedtqhertdertddtnecuhfhrohhmpegkihcujggr nhcuoeiiihdrhigrnhesshgvnhhtrdgtohhmqeenucggtffrrghtthgvrhhnpeeijeeuvd euudeuhfeghfehieeuvdetvdeugfeigeevteeuieeuhedtgeduheefleenucevlhhushht vghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeiiihdrhigrnhesshgvnh htrdgtohhm X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 11 Feb 2022 11:41:45 -0500 (EST) From: Zi Yan To: David Hildenbrand , linux-mm@kvack.org Subject: [PATCH v5 1/6] mm: page_isolation: move has_unmovable_pages() to mm/page_isolation.c Date: Fri, 11 Feb 2022 11:41:30 -0500 Message-Id: <20220211164135.1803616-2-zi.yan@sent.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220211164135.1803616-1-zi.yan@sent.com> References: <20220211164135.1803616-1-zi.yan@sent.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Zi Yan Cc: Mel Gorman , Zi Yan , Oscar Salvador , Robin Murphy , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Mike Rapoport , Eric Ren , virtualization@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org, Christoph Hellwig , Vlastimil Babka , Marek Szyprowski Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" From: Zi Yan has_unmovable_pages() is only used in mm/page_isolation.c. Move it from mm/page_alloc.c and make it static. Signed-off-by: Zi Yan Reviewed-by: Oscar Salvador Reviewed-by: Mike Rapoport --- include/linux/page-isolation.h | 2 - mm/page_alloc.c | 119 --------------------------------- mm/page_isolation.c | 119 +++++++++++++++++++++++++++++++++ 3 files changed, 119 insertions(+), 121 deletions(-) diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 572458016331..e14eddf6741a 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -33,8 +33,6 @@ static inline bool is_migrate_isolate(int migratetype) #define MEMORY_OFFLINE 0x1 #define REPORT_FAILURE 0x2 -struct page *has_unmovable_pages(struct zone *zone, struct page *page, - int migratetype, int flags); void set_pageblock_migratetype(struct page *page, int migratetype); int move_freepages_block(struct zone *zone, struct page *page, int migratetype, int *num_movable); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index cface1d38093..e2c6a67fc386 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -8962,125 +8962,6 @@ void *__init alloc_large_system_hash(const char *tablename, return table; } -/* - * This function checks whether pageblock includes unmovable pages or not. - * - * PageLRU check without isolation or lru_lock could race so that - * MIGRATE_MOVABLE block might include unmovable pages. And __PageMovable - * check without lock_page also may miss some movable non-lru pages at - * race condition. So you can't expect this function should be exact. - * - * Returns a page without holding a reference. If the caller wants to - * dereference that page (e.g., dumping), it has to make sure that it - * cannot get removed (e.g., via memory unplug) concurrently. - * - */ -struct page *has_unmovable_pages(struct zone *zone, struct page *page, - int migratetype, int flags) -{ - unsigned long iter = 0; - unsigned long pfn = page_to_pfn(page); - unsigned long offset = pfn % pageblock_nr_pages; - - if (is_migrate_cma_page(page)) { - /* - * CMA allocations (alloc_contig_range) really need to mark - * isolate CMA pageblocks even when they are not movable in fact - * so consider them movable here. - */ - if (is_migrate_cma(migratetype)) - return NULL; - - return page; - } - - for (; iter < pageblock_nr_pages - offset; iter++) { - page = pfn_to_page(pfn + iter); - - /* - * Both, bootmem allocations and memory holes are marked - * PG_reserved and are unmovable. We can even have unmovable - * allocations inside ZONE_MOVABLE, for example when - * specifying "movablecore". - */ - if (PageReserved(page)) - return page; - - /* - * If the zone is movable and we have ruled out all reserved - * pages then it should be reasonably safe to assume the rest - * is movable. - */ - if (zone_idx(zone) == ZONE_MOVABLE) - continue; - - /* - * Hugepages are not in LRU lists, but they're movable. - * THPs are on the LRU, but need to be counted as #small pages. - * We need not scan over tail pages because we don't - * handle each tail page individually in migration. - */ - if (PageHuge(page) || PageTransCompound(page)) { - struct page *head = compound_head(page); - unsigned int skip_pages; - - if (PageHuge(page)) { - if (!hugepage_migration_supported(page_hstate(head))) - return page; - } else if (!PageLRU(head) && !__PageMovable(head)) { - return page; - } - - skip_pages = compound_nr(head) - (page - head); - iter += skip_pages - 1; - continue; - } - - /* - * We can't use page_count without pin a page - * because another CPU can free compound page. - * This check already skips compound tails of THP - * because their page->_refcount is zero at all time. - */ - if (!page_ref_count(page)) { - if (PageBuddy(page)) - iter += (1 << buddy_order(page)) - 1; - continue; - } - - /* - * The HWPoisoned page may be not in buddy system, and - * page_count() is not 0. - */ - if ((flags & MEMORY_OFFLINE) && PageHWPoison(page)) - continue; - - /* - * We treat all PageOffline() pages as movable when offlining - * to give drivers a chance to decrement their reference count - * in MEM_GOING_OFFLINE in order to indicate that these pages - * can be offlined as there are no direct references anymore. - * For actually unmovable PageOffline() where the driver does - * not support this, we will fail later when trying to actually - * move these pages that still have a reference count > 0. - * (false negatives in this function only) - */ - if ((flags & MEMORY_OFFLINE) && PageOffline(page)) - continue; - - if (__PageMovable(page) || PageLRU(page)) - continue; - - /* - * If there are RECLAIMABLE pages, we need to check - * it. But now, memory offline itself doesn't call - * shrink_node_slabs() and it still to be fixed. - */ - return page; - } - return NULL; -} - #ifdef CONFIG_CONTIG_ALLOC static unsigned long pfn_max_align_down(unsigned long pfn) { diff --git a/mm/page_isolation.c b/mm/page_isolation.c index f67c4c70f17f..b34f1310aeaa 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -15,6 +15,125 @@ #define CREATE_TRACE_POINTS #include +/* + * This function checks whether pageblock includes unmovable pages or not. + * + * PageLRU check without isolation or lru_lock could race so that + * MIGRATE_MOVABLE block might include unmovable pages. And __PageMovable + * check without lock_page also may miss some movable non-lru pages at + * race condition. So you can't expect this function should be exact. + * + * Returns a page without holding a reference. If the caller wants to + * dereference that page (e.g., dumping), it has to make sure that it + * cannot get removed (e.g., via memory unplug) concurrently. + * + */ +static struct page *has_unmovable_pages(struct zone *zone, struct page *page, + int migratetype, int flags) +{ + unsigned long iter = 0; + unsigned long pfn = page_to_pfn(page); + unsigned long offset = pfn % pageblock_nr_pages; + + if (is_migrate_cma_page(page)) { + /* + * CMA allocations (alloc_contig_range) really need to mark + * isolate CMA pageblocks even when they are not movable in fact + * so consider them movable here. + */ + if (is_migrate_cma(migratetype)) + return NULL; + + return page; + } + + for (; iter < pageblock_nr_pages - offset; iter++) { + page = pfn_to_page(pfn + iter); + + /* + * Both, bootmem allocations and memory holes are marked + * PG_reserved and are unmovable. We can even have unmovable + * allocations inside ZONE_MOVABLE, for example when + * specifying "movablecore". + */ + if (PageReserved(page)) + return page; + + /* + * If the zone is movable and we have ruled out all reserved + * pages then it should be reasonably safe to assume the rest + * is movable. + */ + if (zone_idx(zone) == ZONE_MOVABLE) + continue; + + /* + * Hugepages are not in LRU lists, but they're movable. + * THPs are on the LRU, but need to be counted as #small pages. + * We need not scan over tail pages because we don't + * handle each tail page individually in migration. + */ + if (PageHuge(page) || PageTransCompound(page)) { + struct page *head = compound_head(page); + unsigned int skip_pages; + + if (PageHuge(page)) { + if (!hugepage_migration_supported(page_hstate(head))) + return page; + } else if (!PageLRU(head) && !__PageMovable(head)) { + return page; + } + + skip_pages = compound_nr(head) - (page - head); + iter += skip_pages - 1; + continue; + } + + /* + * We can't use page_count without pin a page + * because another CPU can free compound page. + * This check already skips compound tails of THP + * because their page->_refcount is zero at all time. + */ + if (!page_ref_count(page)) { + if (PageBuddy(page)) + iter += (1 << buddy_order(page)) - 1; + continue; + } + + /* + * The HWPoisoned page may be not in buddy system, and + * page_count() is not 0. + */ + if ((flags & MEMORY_OFFLINE) && PageHWPoison(page)) + continue; + + /* + * We treat all PageOffline() pages as movable when offlining + * to give drivers a chance to decrement their reference count + * in MEM_GOING_OFFLINE in order to indicate that these pages + * can be offlined as there are no direct references anymore. + * For actually unmovable PageOffline() where the driver does + * not support this, we will fail later when trying to actually + * move these pages that still have a reference count > 0. + * (false negatives in this function only) + */ + if ((flags & MEMORY_OFFLINE) && PageOffline(page)) + continue; + + if (__PageMovable(page) || PageLRU(page)) + continue; + + /* + * If there are RECLAIMABLE pages, we need to check + * it. But now, memory offline itself doesn't call + * shrink_node_slabs() and it still to be fixed. + */ + return page; + } + return NULL; +} + static int set_migratetype_isolate(struct page *page, int migratetype, int isol_flags) { struct zone *zone = page_zone(page); From patchwork Fri Feb 11 16:41:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 1591731 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=RX+guAPK; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=iVnxPXU0; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JwKCK0wGCz9sFq for ; Sat, 12 Feb 2022 03:42:04 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4JwKCJ2xd9z3cQS for ; Sat, 12 Feb 2022 03:42:04 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=RX+guAPK; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=iVnxPXU0; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sent.com (client-ip=66.111.4.221; helo=new1-smtp.messagingengine.com; envelope-from=zi.yan@sent.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=RX+guAPK; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=iVnxPXU0; dkim-atps=neutral Received: from new1-smtp.messagingengine.com (new1-smtp.messagingengine.com [66.111.4.221]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4JwKC20Bh3z3bVZ for ; Sat, 12 Feb 2022 03:41:49 +1100 (AEDT) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.nyi.internal (Postfix) with ESMTP id 261C6580376; Fri, 11 Feb 2022 11:41:48 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Fri, 11 Feb 2022 11:41:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to :reply-to:sender:subject:subject:to:to; s=fm2; bh=TkAiSDDqHj52sQ QpS3No7LU7FQn/P0B4WvW+CLbRBDY=; b=RX+guAPK53EbPT+OY+vpQb5TtsbFIU ZmLWgYD4H5jvi4GUeeMVGCAXfvGXwg2nNX4Wt64mYQt7bCj+YaXETqx7X3oqQa3R cV1dCprYGVOsBqZMjStHt1ylDGBNvQi43FQudL9mMf+J5hJY9HQu/pvBZNLPmMiG 5JW9Jd5ygnP+H3CMuKvKQ6RWbjoFqVPuKKFvBSedVZv8cQUKFgb+2PrvEPm/69zG 6Ar9HRuN1uBPLZnkfesb2VBreuz1g4oPCGjvhHX5AEXByp1VCCtLz890TAxxEwYF 1KebR2zyQny0/sL5+iqC5IVH/MLM6QfspFhpHW9mo14tgoT6LRqEdejA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=TkAiSDDqHj52sQQpS3No7LU7FQn/P0B4WvW+CLbRBDY=; b=iVnxPXU0 4v3uW6oEWaQ9G8CXU0jDyjtvDCPA/AJd+gLIWBITBQHC4vbFywDecuKk+WV9wQZv NzJLwsY5cI0vo2nn8MHHjwc7WD/dBKOCcbDUwMgqhGp3wjJfKNuYBpNR97+nnE5k fWeN0IdGilm0mTbkKFVwnI7HXeHMDbml4C0Z311M485Iko5YOYLCIHuem23Klc3W Q4M9ZukWGFOKJQFDg5DXRqfTsLSiqSIXbPPZTdlo8CVEkGkFafSO6UfAK/yskgz1 1vmyc7T/thVED+4T03i06hluA/ZsbnuwtWbjpM5PSf3zCTpTaUspeSnaHKPKIIFF jwIUezBJJ2Y30w== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvvddrieefgdeltdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhhrggfgsedtqhertdertddtnecuhfhrohhmpegkihcujggr nhcuoeiiihdrhigrnhesshgvnhhtrdgtohhmqeenucggtffrrghtthgvrhhnpeeijeeuvd euudeuhfeghfehieeuvdetvdeugfeigeevteeuieeuhedtgeduheefleenucevlhhushht vghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeiiihdrhigrnhesshgvnh htrdgtohhm X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 11 Feb 2022 11:41:46 -0500 (EST) From: Zi Yan To: David Hildenbrand , linux-mm@kvack.org Subject: [PATCH v5 2/6] mm: page_isolation: check specified range for unmovable pages Date: Fri, 11 Feb 2022 11:41:31 -0500 Message-Id: <20220211164135.1803616-3-zi.yan@sent.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220211164135.1803616-1-zi.yan@sent.com> References: <20220211164135.1803616-1-zi.yan@sent.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Zi Yan Cc: Mel Gorman , Zi Yan , Oscar Salvador , Robin Murphy , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Mike Rapoport , Eric Ren , virtualization@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org, Christoph Hellwig , Vlastimil Babka , Marek Szyprowski Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" From: Zi Yan Enable set_migratetype_isolate() to check specified sub-range for unmovable pages during isolation. Page isolation is done at max(MAX_ORDER_NR_PAEGS, pageblock_nr_pages) granularity, but not all pages within that granularity are intended to be isolated. For example, alloc_contig_range(), which uses page isolation, allows ranges without alignment. This commit makes unmovable page check only look for interesting pages, so that page isolation can succeed for any non-overlapping ranges. Signed-off-by: Zi Yan --- include/linux/page-isolation.h | 12 +++++++++ mm/page_alloc.c | 15 +---------- mm/page_isolation.c | 46 +++++++++++++++++++++------------- 3 files changed, 41 insertions(+), 32 deletions(-) diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index e14eddf6741a..4ef7be6def83 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -15,6 +15,18 @@ static inline bool is_migrate_isolate(int migratetype) { return migratetype == MIGRATE_ISOLATE; } +static inline unsigned long pfn_max_align_down(unsigned long pfn) +{ + return ALIGN_DOWN(pfn, max_t(unsigned long, MAX_ORDER_NR_PAGES, + pageblock_nr_pages)); +} + +static inline unsigned long pfn_max_align_up(unsigned long pfn) +{ + return ALIGN(pfn, max_t(unsigned long, MAX_ORDER_NR_PAGES, + pageblock_nr_pages)); +} + #else static inline bool has_isolate_pageblock(struct zone *zone) { diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e2c6a67fc386..62ef78f3d771 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -8963,18 +8963,6 @@ void *__init alloc_large_system_hash(const char *tablename, } #ifdef CONFIG_CONTIG_ALLOC -static unsigned long pfn_max_align_down(unsigned long pfn) -{ - return pfn & ~(max_t(unsigned long, MAX_ORDER_NR_PAGES, - pageblock_nr_pages) - 1); -} - -static unsigned long pfn_max_align_up(unsigned long pfn) -{ - return ALIGN(pfn, max_t(unsigned long, MAX_ORDER_NR_PAGES, - pageblock_nr_pages)); -} - #if defined(CONFIG_DYNAMIC_DEBUG) || \ (defined(CONFIG_DYNAMIC_DEBUG_CORE) && defined(DYNAMIC_DEBUG_MODULE)) /* Usage: See admin-guide/dynamic-debug-howto.rst */ @@ -9119,8 +9107,7 @@ int alloc_contig_range(unsigned long start, unsigned long end, * put back to page allocator so that buddy can use them. */ - ret = start_isolate_page_range(pfn_max_align_down(start), - pfn_max_align_up(end), migratetype, 0); + ret = start_isolate_page_range(start, end, migratetype, 0); if (ret) return ret; diff --git a/mm/page_isolation.c b/mm/page_isolation.c index b34f1310aeaa..64d093ab83ec 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -16,7 +16,8 @@ #include /* - * This function checks whether pageblock includes unmovable pages or not. + * This function checks whether pageblock within [start_pfn, end_pfn) includes + * unmovable pages or not. * * PageLRU check without isolation or lru_lock could race so that * MIGRATE_MOVABLE block might include unmovable pages. And __PageMovable @@ -29,11 +30,14 @@ * */ static struct page *has_unmovable_pages(struct zone *zone, struct page *page, - int migratetype, int flags) + int migratetype, int flags, + unsigned long start_pfn, unsigned long end_pfn) { - unsigned long iter = 0; - unsigned long pfn = page_to_pfn(page); - unsigned long offset = pfn % pageblock_nr_pages; + unsigned long first_pfn = max(page_to_pfn(page), start_pfn); + unsigned long pfn = first_pfn; + unsigned long last_pfn = min(ALIGN(pfn + 1, pageblock_nr_pages), end_pfn); + + page = pfn_to_page(pfn); if (is_migrate_cma_page(page)) { /* @@ -47,8 +51,8 @@ static struct page *has_unmovable_pages(struct zone *zone, struct page *page, return page; } - for (; iter < pageblock_nr_pages - offset; iter++) { - page = pfn_to_page(pfn + iter); + for (pfn = first_pfn; pfn < last_pfn; pfn++) { + page = pfn_to_page(pfn); /* * Both, bootmem allocations and memory holes are marked @@ -85,7 +89,7 @@ static struct page *has_unmovable_pages(struct zone *zone, struct page *page, } skip_pages = compound_nr(head) - (page - head); - iter += skip_pages - 1; + pfn += skip_pages - 1; continue; } @@ -97,7 +101,7 @@ static struct page *has_unmovable_pages(struct zone *zone, struct page *page, */ if (!page_ref_count(page)) { if (PageBuddy(page)) - iter += (1 << buddy_order(page)) - 1; + pfn += (1 << buddy_order(page)) - 1; continue; } @@ -134,7 +138,13 @@ static struct page *has_unmovable_pages(struct zone *zone, struct page *page, return NULL; } -static int set_migratetype_isolate(struct page *page, int migratetype, int isol_flags) +/* + * This function set pageblock migratetype to isolate if no unmovable page is + * present in [start_pfn, end_pfn). The pageblock must be within + * [start_pfn, end_pfn). + */ +static int set_migratetype_isolate(struct page *page, int migratetype, int isol_flags, + unsigned long start_pfn, unsigned long end_pfn) { struct zone *zone = page_zone(page); struct page *unmovable; @@ -156,7 +166,7 @@ static int set_migratetype_isolate(struct page *page, int migratetype, int isol_ * FIXME: Now, memory hotplug doesn't call shrink_slab() by itself. * We just check MOVABLE pages. */ - unmovable = has_unmovable_pages(zone, page, migratetype, isol_flags); + unmovable = has_unmovable_pages(zone, page, migratetype, isol_flags, start_pfn, end_pfn); if (!unmovable) { unsigned long nr_pages; int mt = get_pageblock_migratetype(page); @@ -267,7 +277,6 @@ __first_valid_page(unsigned long pfn, unsigned long nr_pages) * be MIGRATE_ISOLATE. * @start_pfn: The lower PFN of the range to be isolated. * @end_pfn: The upper PFN of the range to be isolated. - * start_pfn/end_pfn must be aligned to pageblock_order. * @migratetype: Migrate type to set in error recovery. * @flags: The following flags are allowed (they can be combined in * a bit mask) @@ -309,15 +318,16 @@ int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn, unsigned long pfn; struct page *page; - BUG_ON(!IS_ALIGNED(start_pfn, pageblock_nr_pages)); - BUG_ON(!IS_ALIGNED(end_pfn, pageblock_nr_pages)); + unsigned long isolate_start = pfn_max_align_down(start_pfn); + unsigned long isolate_end = pfn_max_align_up(end_pfn); - for (pfn = start_pfn; - pfn < end_pfn; + for (pfn = isolate_start; + pfn < isolate_end; pfn += pageblock_nr_pages) { page = __first_valid_page(pfn, pageblock_nr_pages); - if (page && set_migratetype_isolate(page, migratetype, flags)) { - undo_isolate_page_range(start_pfn, pfn, migratetype); + if (page && set_migratetype_isolate(page, migratetype, flags, + start_pfn, end_pfn)) { + undo_isolate_page_range(isolate_start, pfn, migratetype); return -EBUSY; } } From patchwork Fri Feb 11 16:41:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 1591734 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=cTh0jkh+; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=jDCmr4fT; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JwKFZ032nz9sFq for ; Sat, 12 Feb 2022 03:44:01 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4JwKFY2mS7z3cYX for ; Sat, 12 Feb 2022 03:44:01 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=cTh0jkh+; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=jDCmr4fT; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sent.com (client-ip=66.111.4.221; helo=new1-smtp.messagingengine.com; envelope-from=zi.yan@sent.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=cTh0jkh+; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=jDCmr4fT; dkim-atps=neutral Received: from new1-smtp.messagingengine.com (new1-smtp.messagingengine.com [66.111.4.221]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4JwKC25jwvz3bWG for ; Sat, 12 Feb 2022 03:41:50 +1100 (AEDT) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailnew.nyi.internal (Postfix) with ESMTP id ED869580379; Fri, 11 Feb 2022 11:41:48 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Fri, 11 Feb 2022 11:41:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to :reply-to:sender:subject:subject:to:to; s=fm2; bh=TcXjUE97dfvam7 r+XLk/q1ULK9dVCJrN5qtDmouJREU=; b=cTh0jkh+aV/N0TdDGhrAWrJH9QRbsI YaP59oSHrHWTFxj63A/bBlgVE3Is6dIPTkWcn5nnPjjARrfMkEpwo1VmawCGKCHg dYqDq1rL7ecZt+WFz55KN5BFtjfeDyLxDDurTrLEBt4wizRrrCguZpoeDDYVAsJt YnKKdeBfGzgORfY3pGdKPk9Bo/kB188DBsh9OZb/LXikIluQfE2oy+L8F43xA6om blamYzXFFPhStURDeJfn6xM1hHf7vMSo2CKbLN4wxml0HJh6ovc3zwEHhS0tVjAd HiYAOUCci9DZlXA+WXKwl80NYXKky7Pn++sUGqvQqsAt+gYwZ5lJ/kVg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=TcXjUE97dfvam7r+XLk/q1ULK9dVCJrN5qtDmouJREU=; b=jDCmr4fT ga53sZo/uJ6FxZhcxB0gIHyjtBMsAPBlFFKgyxsO9T0XvgPpkyHtGh/CTGIDIGG+ T3R9k4r9XUq7PfLweq8on4gnP5GRL2kcfAu901ruXbq78CFAHIRNNJTAh6AjUc7Y PwUHJuhnU7GhJKevmtVfCKA39OiUnm+VMwDOpkS/RtQ9mVcSl1QmVMYZiX0anpoD q6kVVmF13FfqfKau079PdaDpoSHZwWkwzChzaz54+UndpUf6WUc5/hIdekZxzP3p /USObcvcLFHGFbNdESYIS6soty4qFe97r4zI2U5uB0KsTaUhnQvi10/BP94U9gDj zFurCtsXpVsKWw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvvddrieefgdeltdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhhrggfgsedtqhertdertddtnecuhfhrohhmpegkihcujggr nhcuoeiiihdrhigrnhesshgvnhhtrdgtohhmqeenucggtffrrghtthgvrhhnpeeijeeuvd euudeuhfeghfehieeuvdetvdeugfeigeevteeuieeuhedtgeduheefleenucevlhhushht vghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeiiihdrhigrnhesshgvnh htrdgtohhm X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 11 Feb 2022 11:41:48 -0500 (EST) From: Zi Yan To: David Hildenbrand , linux-mm@kvack.org Subject: [PATCH v5 3/6] mm: make alloc_contig_range work at pageblock granularity Date: Fri, 11 Feb 2022 11:41:32 -0500 Message-Id: <20220211164135.1803616-4-zi.yan@sent.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220211164135.1803616-1-zi.yan@sent.com> References: <20220211164135.1803616-1-zi.yan@sent.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Zi Yan Cc: Mel Gorman , Zi Yan , Oscar Salvador , Robin Murphy , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Mike Rapoport , Eric Ren , virtualization@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org, Christoph Hellwig , Vlastimil Babka , Marek Szyprowski Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" From: Zi Yan alloc_contig_range() worked at MAX_ORDER-1 granularity to avoid merging pageblocks with different migratetypes. It might unnecessarily convert extra pageblocks at the beginning and at the end of the range. Change alloc_contig_range() to work at pageblock granularity. Special handling is needed for free pages and in-use pages across the boundaries of the range specified alloc_contig_range(). Because these partially isolated pages causes free page accounting issues. The free pages will be split and freed into separate migratetype lists; the in-use pages will be migrated then the freed pages will be handled. Signed-off-by: Zi Yan --- include/linux/page-isolation.h | 2 +- mm/internal.h | 3 + mm/memory_hotplug.c | 3 +- mm/page_alloc.c | 235 +++++++++++++++++++++++++-------- mm/page_isolation.c | 33 ++++- 5 files changed, 211 insertions(+), 65 deletions(-) diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 4ef7be6def83..78ff940cc169 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -54,7 +54,7 @@ int move_freepages_block(struct zone *zone, struct page *page, */ int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn, - unsigned migratetype, int flags); + unsigned migratetype, int flags, gfp_t gfp_flags); /* * Changes MIGRATE_ISOLATE to MIGRATE_MOVABLE. diff --git a/mm/internal.h b/mm/internal.h index 0d240e876831..509cbdc25992 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -319,6 +319,9 @@ isolate_freepages_range(struct compact_control *cc, int isolate_migratepages_range(struct compact_control *cc, unsigned long low_pfn, unsigned long end_pfn); + +int +isolate_single_pageblock(unsigned long boundary_pfn, gfp_t gfp_flags, int isolate_before_boundary); #endif int find_suitable_fallback(struct free_area *area, unsigned int order, int migratetype, bool only_stealable, bool *can_steal); diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index ce68098832aa..82406d2f3e46 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1863,7 +1863,8 @@ int __ref offline_pages(unsigned long start_pfn, unsigned long nr_pages, /* set above range as isolated */ ret = start_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE, - MEMORY_OFFLINE | REPORT_FAILURE); + MEMORY_OFFLINE | REPORT_FAILURE, + GFP_USER | __GFP_MOVABLE | __GFP_RETRY_MAYFAIL); if (ret) { reason = "failure to isolate range"; goto failed_removal_pcplists_disabled; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 62ef78f3d771..7a4fa21aea5c 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -8985,7 +8985,7 @@ static inline void alloc_contig_dump_pages(struct list_head *page_list) #endif /* [start, end) must belong to a single zone. */ -static int __alloc_contig_migrate_range(struct compact_control *cc, +int __alloc_contig_migrate_range(struct compact_control *cc, unsigned long start, unsigned long end) { /* This function is based on compact_zone() from compaction.c. */ @@ -9043,6 +9043,167 @@ static int __alloc_contig_migrate_range(struct compact_control *cc, return 0; } +/** + * split_free_page() -- split a free page at split_pfn_offset + * @free_page: the original free page + * @order: the order of the page + * @split_pfn_offset: split offset within the page + * + * It is used when the free page crosses two pageblocks with different migratetypes + * at split_pfn_offset within the page. The split free page will be put into + * separate migratetype lists afterwards. Otherwise, the function achieves + * nothing. + */ +static inline void split_free_page(struct page *free_page, + int order, unsigned long split_pfn_offset) +{ + struct zone *zone = page_zone(free_page); + unsigned long free_page_pfn = page_to_pfn(free_page); + unsigned long pfn; + unsigned long flags; + int free_page_order; + + spin_lock_irqsave(&zone->lock, flags); + del_page_from_free_list(free_page, zone, order); + for (pfn = free_page_pfn; + pfn < free_page_pfn + (1UL << order);) { + int mt = get_pfnblock_migratetype(pfn_to_page(pfn), pfn); + + free_page_order = order_base_2(split_pfn_offset); + __free_one_page(pfn_to_page(pfn), pfn, zone, free_page_order, + mt, FPI_NONE); + pfn += 1UL << free_page_order; + split_pfn_offset -= (1UL << free_page_order); + /* we have done the first part, now switch to second part */ + if (split_pfn_offset == 0) + split_pfn_offset = (1UL << order) - (pfn - free_page_pfn); + } + spin_unlock_irqrestore(&zone->lock, flags); +} + +/** + * isolate_single_pageblock() -- tries to isolate a pageblock that might be + * within a free or in-use page. + * @boundary_pfn: pageblock-aligned pfn that a page might cross + * @gfp_flags: GFP flags used for migrating pages + * @isolate_before_boundary: isolate the pageblock before (1) or after (0) + * the boundary_pfn + * + * Free and in-use pages can be as big as MAX_ORDER-1 and contain more than one + * pageblock. When not all pageblocks within a page are isolated at the same + * time, free page accounting can go wrong. For example, in the case of + * MAX_ORDER-1 = pageblock_order + 1, a MAX_ORDER-1 page has two pagelbocks. + * [ MAX_ORDER-1 ] + * [ pageblock0 | pageblock1 ] + * When either pageblock is isolated, if it is a free page, the page is not + * split into separate migratetype lists, which is supposed to; if it is an + * in-use page and freed later, __free_one_page() does not split the free page + * either. The function handles this by splitting the free page or migrating + * the in-use page then splitting the free page. + */ +int isolate_single_pageblock(unsigned long boundary_pfn, gfp_t gfp_flags, + int isolate_before_boundary) +{ + unsigned char saved_mt; + /* + * scan at max(MAX_ORDER_NR_PAGES, pageblock_nr_pages) aligned range to + * avoid isolate pageblocks belonging to a bigger free or in-use page + */ + unsigned long start_pfn = pfn_max_align_down(boundary_pfn); + unsigned long isolated_pageblock_pfn; + unsigned long pfn; + + VM_BUG_ON(!IS_ALIGNED(boundary_pfn, pageblock_nr_pages)); + + if (isolate_before_boundary) + isolated_pageblock_pfn = boundary_pfn - pageblock_nr_pages; + else + isolated_pageblock_pfn = boundary_pfn; + + saved_mt = get_pageblock_migratetype(pfn_to_page(isolated_pageblock_pfn)); + set_pageblock_migratetype(pfn_to_page(isolated_pageblock_pfn), MIGRATE_ISOLATE); + + for (pfn = start_pfn; pfn < boundary_pfn;) { + struct page *page = pfn_to_page(pfn); + + /* + * start_pfn is max(MAX_ORDER_NR_PAGES, pageblock_nr_pages) + * aligned, if there is any free pages in [start_pfn, boundary_pfn), + * its head page will always be in the range. + */ + if (PageBuddy(page)) { + int order = buddy_order(page); + + if (pfn + (1UL << order) > boundary_pfn) + split_free_page(page, order, boundary_pfn - pfn); + pfn += (1UL << order); + continue; + } + /* + * migrate compound pages then let the free page handling code + * above do the rest + */ + if (PageHuge(page) || PageTransCompound(page)) { + unsigned long nr_pages = compound_nr(page); + int order = compound_order(page); + struct page *head = compound_head(page); + unsigned long head_pfn = page_to_pfn(head); + + if (head_pfn + nr_pages >= boundary_pfn) { + int ret; + struct compact_control cc = { + .nr_migratepages = 0, + .order = -1, + .zone = page_zone(pfn_to_page(head_pfn)), + .mode = MIGRATE_SYNC, + .ignore_skip_hint = true, + .no_set_skip_hint = true, + .gfp_mask = current_gfp_context(gfp_flags), + .alloc_contig = true, + }; + + INIT_LIST_HEAD(&cc.migratepages); + + ret = __alloc_contig_migrate_range(&cc, head_pfn, + head_pfn + nr_pages); + + if (ret) { + /* restore the original migratetype */ + set_pageblock_migratetype( + pfn_to_page(isolated_pageblock_pfn), + saved_mt); + return -EBUSY; + } + /* + * reset pfn, let the free page handling code + * above split the free page to the right + * migratetype list. + * + * head_pfn is not used here as a hugetlb page + * order can be bigger than MAX_ORDER-1, but + * after it is freed, the free page order is not. + * Use pfn within the range to find the head of + * the free page and reset order to 0 if a hugetlb + * page with >MAX_ORDER-1 order is encountered. + */ + if (order > MAX_ORDER-1) + order = 0; + while (!PageBuddy(pfn_to_page(pfn))) { + order++; + pfn &= ~0UL << order; + } + continue; + } + pfn += nr_pages; + continue; + } + + pfn++; + } + return 0; +} + + /** * alloc_contig_range() -- tries to allocate given range of pages * @start: start PFN to allocate @@ -9067,8 +9228,9 @@ static int __alloc_contig_migrate_range(struct compact_control *cc, int alloc_contig_range(unsigned long start, unsigned long end, unsigned migratetype, gfp_t gfp_mask) { - unsigned long outer_start, outer_end; - unsigned int order; + unsigned long outer_end; + unsigned long alloc_start = ALIGN_DOWN(start, pageblock_nr_pages); + unsigned long alloc_end = ALIGN(end, pageblock_nr_pages); int ret = 0; struct compact_control cc = { @@ -9087,14 +9249,11 @@ int alloc_contig_range(unsigned long start, unsigned long end, * What we do here is we mark all pageblocks in range as * MIGRATE_ISOLATE. Because pageblock and max order pages may * have different sizes, and due to the way page allocator - * work, we align the range to biggest of the two pages so - * that page allocator won't try to merge buddies from - * different pageblocks and change MIGRATE_ISOLATE to some - * other migration type. + * work, start_isolate_page_range() has special handlings for this. * * Once the pageblocks are marked as MIGRATE_ISOLATE, we * migrate the pages from an unaligned range (ie. pages that - * we are interested in). This will put all the pages in + * we are interested in). This will put all the pages in * range back to page allocator as MIGRATE_ISOLATE. * * When this is done, we take the pages in range from page @@ -9107,9 +9266,9 @@ int alloc_contig_range(unsigned long start, unsigned long end, * put back to page allocator so that buddy can use them. */ - ret = start_isolate_page_range(start, end, migratetype, 0); + ret = start_isolate_page_range(start, end, migratetype, 0, gfp_mask); if (ret) - return ret; + goto done; drain_all_pages(cc.zone); @@ -9128,68 +9287,28 @@ int alloc_contig_range(unsigned long start, unsigned long end, goto done; ret = 0; - /* - * Pages from [start, end) are within a MAX_ORDER_NR_PAGES - * aligned blocks that are marked as MIGRATE_ISOLATE. What's - * more, all pages in [start, end) are free in page allocator. - * What we are going to do is to allocate all pages from - * [start, end) (that is remove them from page allocator). - * - * The only problem is that pages at the beginning and at the - * end of interesting range may be not aligned with pages that - * page allocator holds, ie. they can be part of higher order - * pages. Because of this, we reserve the bigger range and - * once this is done free the pages we are not interested in. - * - * We don't have to hold zone->lock here because the pages are - * isolated thus they won't get removed from buddy. - */ - - order = 0; - outer_start = start; - while (!PageBuddy(pfn_to_page(outer_start))) { - if (++order >= MAX_ORDER) { - outer_start = start; - break; - } - outer_start &= ~0UL << order; - } - - if (outer_start != start) { - order = buddy_order(pfn_to_page(outer_start)); - - /* - * outer_start page could be small order buddy page and - * it doesn't include start page. Adjust outer_start - * in this case to report failed page properly - * on tracepoint in test_pages_isolated() - */ - if (outer_start + (1UL << order) <= start) - outer_start = start; - } - /* Make sure the range is really isolated. */ - if (test_pages_isolated(outer_start, end, 0)) { + if (test_pages_isolated(alloc_start, alloc_end, 0)) { ret = -EBUSY; goto done; } /* Grab isolated pages from freelists. */ - outer_end = isolate_freepages_range(&cc, outer_start, end); + outer_end = isolate_freepages_range(&cc, alloc_start, alloc_end); if (!outer_end) { ret = -EBUSY; goto done; } /* Free head and tail (if any) */ - if (start != outer_start) - free_contig_range(outer_start, start - outer_start); - if (end != outer_end) - free_contig_range(end, outer_end - end); + if (start != alloc_start) + free_contig_range(alloc_start, start - alloc_start); + if (end != alloc_end) + free_contig_range(end, alloc_end - end); done: - undo_isolate_page_range(pfn_max_align_down(start), - pfn_max_align_up(end), migratetype); + undo_isolate_page_range(alloc_start, + alloc_end, migratetype); return ret; } EXPORT_SYMBOL(alloc_contig_range); diff --git a/mm/page_isolation.c b/mm/page_isolation.c index 64d093ab83ec..0256d5e1032c 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -285,6 +285,8 @@ __first_valid_page(unsigned long pfn, unsigned long nr_pages) * and PageOffline() pages. * REPORT_FAILURE - report details about the failure to * isolate the range + * @gfp_flags: GFP flags used for migrating pages that sit across the + * range boundaries. * * Making page-allocation-type to be MIGRATE_ISOLATE means free pages in * the range will never be allocated. Any free pages and pages freed in the @@ -293,6 +295,10 @@ __first_valid_page(unsigned long pfn, unsigned long nr_pages) * pages in the range finally, the caller have to free all pages in the range. * test_page_isolated() can be used for test it. * + * The function first tries to isolate the pageblocks at the beginning and end + * of the range, since there might be pages across the range boundaries. + * Afterwards, it isolates the rest of the range. + * * There is no high level synchronization mechanism that prevents two threads * from trying to isolate overlapping ranges. If this happens, one thread * will notice pageblocks in the overlapping range already set to isolate. @@ -313,21 +319,38 @@ __first_valid_page(unsigned long pfn, unsigned long nr_pages) * Return: 0 on success and -EBUSY if any part of range cannot be isolated. */ int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn, - unsigned migratetype, int flags) + unsigned migratetype, int flags, gfp_t gfp_flags) { unsigned long pfn; struct page *page; + /* isolation is done at page block granularity */ + unsigned long isolate_start = ALIGN_DOWN(start_pfn, pageblock_nr_pages); + unsigned long isolate_end = ALIGN(end_pfn, pageblock_nr_pages); + int ret; - unsigned long isolate_start = pfn_max_align_down(start_pfn); - unsigned long isolate_end = pfn_max_align_up(end_pfn); + /* isolate [isolate_start, isolate_start + pageblock_nr_pages] pageblock */ + ret = isolate_single_pageblock(isolate_start, gfp_flags, 0); + if (ret) + return ret; + + /* isolate [isolate_end - pageblock_nr_pages, isolate_end] pageblock */ + ret = isolate_single_pageblock(isolate_end, gfp_flags, 1); + if (ret) { + unset_migratetype_isolate(pfn_to_page(isolate_start), migratetype); + return ret; + } - for (pfn = isolate_start; - pfn < isolate_end; + /* skip isolated pageblocks at the beginning and end */ + for (pfn = isolate_start + pageblock_nr_pages; + pfn < isolate_end - pageblock_nr_pages; pfn += pageblock_nr_pages) { page = __first_valid_page(pfn, pageblock_nr_pages); if (page && set_migratetype_isolate(page, migratetype, flags, start_pfn, end_pfn)) { undo_isolate_page_range(isolate_start, pfn, migratetype); + unset_migratetype_isolate( + pfn_to_page(isolate_end - pageblock_nr_pages), + migratetype); return -EBUSY; } } From patchwork Fri Feb 11 16:41:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 1591735 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=zlqzeudl; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=glMY8862; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JwKGK6xcTz9sFq for ; Sat, 12 Feb 2022 03:44:41 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4JwKGK3wgWz3cbQ for ; Sat, 12 Feb 2022 03:44:41 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=zlqzeudl; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=glMY8862; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sent.com (client-ip=66.111.4.221; helo=new1-smtp.messagingengine.com; envelope-from=zi.yan@sent.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=zlqzeudl; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=glMY8862; dkim-atps=neutral Received: from new1-smtp.messagingengine.com (new1-smtp.messagingengine.com [66.111.4.221]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4JwKC36plxz3bZF for ; Sat, 12 Feb 2022 03:41:51 +1100 (AEDT) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailnew.nyi.internal (Postfix) with ESMTP id DC08A58037A; Fri, 11 Feb 2022 11:41:49 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Fri, 11 Feb 2022 11:41:49 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to :reply-to:sender:subject:subject:to:to; s=fm2; bh=New+plem0avY8e mMqwW+nHffrxO930geSXXz9NumNjU=; b=zlqzeudlmr9ByiQoXiXD4oWkPkQWrO 9LeIZ5gkrbGwvDnTKD/QP5C7t0n60UprSyOc91UIp0QDNUGC/3yQQEqBXXxYPQHo ISZooymnxzQdfCRxb1cSPvkuA++2BhOfDjBnGnr6HfxxZjGFmymJKMpeCBwAJJx4 Kdn2JXSIhmmG3UOv+Qr8TpZJ1iwPScWnQ4pqjPXgPSS5jLPDysklL2yiPX/5kegT N0v3yf7/Z7hTjjBH4xB5E3WvuLfQ/z/wvG4sKohcJCv3nPbLVIVN1gwBLXOmdHpg fs3ACxLgHy3notg1CzORsf99n6aUNQxFEtaUUKM7GycuXURNpPAN697w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=New+plem0avY8emMqwW+nHffrxO930geSXXz9NumNjU=; b=glMY8862 ++fO/DiQl5ZcoUaX+3OrMgqO2GO7ce+m2y9vaERy5c20bmlc8TH+s4mIm7bau+jn jSQfTEV+8mBxlbrK/mnQe4uNCUYC4p35uLcjIiVDokPV9tHvIu4Ye4KhmqYt0Zd1 /x49iWVgUZj+MmtBWMPAGHEqxXvd+5Z6CJnrEtC77JEDadDAgHCyQgLFljig4rD5 DqMIcKJqEzXma0s5g25GvRIJb+kwjvY0owefkDGcJTmyahN+EsZToLLNW7BzbE7Z 5yPcfnEsRT1G3Mj5rrTWVqFOHBPdNTyIluvVLCcaXsGCCD3gZZB+4VdSu45pxDPL bDEo/wpcNkunpg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvvddrieefgdeltdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhhrggfgsedtqhertdertddtnecuhfhrohhmpegkihcujggr nhcuoeiiihdrhigrnhesshgvnhhtrdgtohhmqeenucggtffrrghtthgvrhhnpeeijeeuvd euudeuhfeghfehieeuvdetvdeugfeigeevteeuieeuhedtgeduheefleenucevlhhushht vghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeiiihdrhigrnhesshgvnh htrdgtohhm X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 11 Feb 2022 11:41:49 -0500 (EST) From: Zi Yan To: David Hildenbrand , linux-mm@kvack.org Subject: [PATCH v5 4/6] mm: cma: use pageblock_order as the single alignment Date: Fri, 11 Feb 2022 11:41:33 -0500 Message-Id: <20220211164135.1803616-5-zi.yan@sent.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220211164135.1803616-1-zi.yan@sent.com> References: <20220211164135.1803616-1-zi.yan@sent.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Zi Yan Cc: Mel Gorman , Zi Yan , Oscar Salvador , Robin Murphy , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Mike Rapoport , Eric Ren , virtualization@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org, Christoph Hellwig , Vlastimil Babka , Marek Szyprowski Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" From: Zi Yan Now alloc_contig_range() works at pageblock granularity. Change CMA allocation, which uses alloc_contig_range(), to use pageblock_order alignment. Signed-off-by: Zi Yan --- include/linux/mmzone.h | 5 +---- kernel/dma/contiguous.c | 2 +- mm/cma.c | 6 ++---- mm/page_alloc.c | 4 ++-- 4 files changed, 6 insertions(+), 11 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 3fff6deca2c0..da38c8436493 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -54,10 +54,7 @@ enum migratetype { * * The way to use it is to change migratetype of a range of * pageblocks to MIGRATE_CMA which can be done by - * __free_pageblock_cma() function. What is important though - * is that a range of pageblocks must be aligned to - * MAX_ORDER_NR_PAGES should biggest page be bigger than - * a single pageblock. + * __free_pageblock_cma() function. */ MIGRATE_CMA, #endif diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c index 3d63d91cba5c..ac35b14b0786 100644 --- a/kernel/dma/contiguous.c +++ b/kernel/dma/contiguous.c @@ -399,7 +399,7 @@ static const struct reserved_mem_ops rmem_cma_ops = { static int __init rmem_cma_setup(struct reserved_mem *rmem) { - phys_addr_t align = PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order); + phys_addr_t align = PAGE_SIZE << pageblock_order; phys_addr_t mask = align - 1; unsigned long node = rmem->fdt_node; bool default_cma = of_get_flat_dt_prop(node, "linux,cma-default", NULL); diff --git a/mm/cma.c b/mm/cma.c index 766f1b82b532..b2e927fab7b5 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -187,8 +187,7 @@ int __init cma_init_reserved_mem(phys_addr_t base, phys_addr_t size, return -EINVAL; /* ensure minimal alignment required by mm core */ - alignment = PAGE_SIZE << - max_t(unsigned long, MAX_ORDER - 1, pageblock_order); + alignment = PAGE_SIZE << pageblock_order; /* alignment should be aligned with order_per_bit */ if (!IS_ALIGNED(alignment >> PAGE_SHIFT, 1 << order_per_bit)) @@ -275,8 +274,7 @@ int __init cma_declare_contiguous_nid(phys_addr_t base, * migratetype page by page allocator's buddy algorithm. In the case, * you couldn't get a contiguous memory, which is not what we want. */ - alignment = max(alignment, (phys_addr_t)PAGE_SIZE << - max_t(unsigned long, MAX_ORDER - 1, pageblock_order)); + alignment = max(alignment, (phys_addr_t)PAGE_SIZE << pageblock_order); if (fixed && base & (alignment - 1)) { ret = -EINVAL; pr_err("Region at %pa must be aligned to %pa bytes\n", diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 7a4fa21aea5c..ac9432e63ce1 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -9214,8 +9214,8 @@ int isolate_single_pageblock(unsigned long boundary_pfn, gfp_t gfp_flags, * be either of the two. * @gfp_mask: GFP mask to use during compaction * - * The PFN range does not have to be pageblock or MAX_ORDER_NR_PAGES - * aligned. The PFN range must belong to a single zone. + * The PFN range does not have to be pageblock aligned. The PFN range must + * belong to a single zone. * * The first thing this routine does is attempt to MIGRATE_ISOLATE all * pageblocks in the range. Once isolated, the pageblocks should not From patchwork Fri Feb 11 16:41:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 1591737 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=HhkSZgj0; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=b+HoxZQU; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JwKHp2C0nz9sFq for ; Sat, 12 Feb 2022 03:45:58 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4JwKHn5yV1z3dkJ for ; Sat, 12 Feb 2022 03:45:57 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=HhkSZgj0; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=b+HoxZQU; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sent.com (client-ip=66.111.4.221; helo=new1-smtp.messagingengine.com; envelope-from=zi.yan@sent.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=HhkSZgj0; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=b+HoxZQU; dkim-atps=neutral Received: from new1-smtp.messagingengine.com (new1-smtp.messagingengine.com [66.111.4.221]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4JwKC54gxZz3bZX for ; Sat, 12 Feb 2022 03:41:53 +1100 (AEDT) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailnew.nyi.internal (Postfix) with ESMTP id AB9DA58037C; Fri, 11 Feb 2022 11:41:50 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 11 Feb 2022 11:41:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to :reply-to:sender:subject:subject:to:to; s=fm2; bh=ns7zKpm4A9Xmsm ZwroT9dHnb6rgarNlTVL3pcGHHu0o=; b=HhkSZgj0d6uRtnT7s5z1WUa0qj27bC RPCjKEp+cbL/NNOORX1x08Je1Mb+zbU9hACFgx4vaehD0ZTg5j8N5A60hXkTStCy Rv3h+mkhOAeW+uSlBJXL2tBu25fgGR1H036XJbizSq7S+1Jo/kOmWIFe9ke16Omh vAU+Ys9UTqbD4kgtyX6RyUKSfEFijWPBxjbxZoDeOHAfCLwr8bNOqW7VSiUScaiP UXWhGfveqUGCmzHskSNDQhIqL71z/XsmBtobnRUISr9WbV1hvE7vqWkBckPtwQVR uoPFf1TYIIpWKFZQRzNfPhkJGXzXJ9VNr0/BjJVnIcPYaME1mwO8QF4w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=ns7zKpm4A9XmsmZwroT9dHnb6rgarNlTVL3pcGHHu0o=; b=b+HoxZQU sEXEVT7xD5fjST3/BM4MLUWPtVa/KKDHkcI8gFADHiFTXKmt7DZXdHdx6ZwYbwoq BklXre/vDPSfD0+Qw8Lx0GErCtY2BCkp24uBzza6Gus/TtT4yAVtGCX1SzS/UC2T qcKZnMV7l1b0G64riBoSSPoS8aUFV0q+o5zYaC5ZMVpHrU/2P7YqZ4uSGCiaAimf 4+HJJ/OMddls4LrrNOvGOG90TS3HS1/VoAKDA39nyPDD0EdqJBFZkALEJ3wmGxyC 6J4gppe20SdY1bb8QZGNwb/Tmk996l331/q91v4f7LAecyWfYMBpua2+7rkiIDCu 633kilwZzAHD/g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvvddrieefgdeklecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhhrggfgsedtqhertdertddtnecuhfhrohhmpegkihcujggr nhcuoeiiihdrhigrnhesshgvnhhtrdgtohhmqeenucggtffrrghtthgvrhhnpeeijeeuvd euudeuhfeghfehieeuvdetvdeugfeigeevteeuieeuhedtgeduheefleenucevlhhushht vghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeiiihdrhigrnhesshgvnh htrdgtohhm X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 11 Feb 2022 11:41:49 -0500 (EST) From: Zi Yan To: David Hildenbrand , linux-mm@kvack.org Subject: [PATCH v5 5/6] drivers: virtio_mem: use pageblock size as the minimum virtio_mem size. Date: Fri, 11 Feb 2022 11:41:34 -0500 Message-Id: <20220211164135.1803616-6-zi.yan@sent.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220211164135.1803616-1-zi.yan@sent.com> References: <20220211164135.1803616-1-zi.yan@sent.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Zi Yan Cc: Mel Gorman , Zi Yan , Oscar Salvador , Robin Murphy , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Mike Rapoport , Eric Ren , virtualization@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org, Christoph Hellwig , Vlastimil Babka , Marek Szyprowski Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" From: Zi Yan alloc_contig_range() now only needs to be aligned to pageblock_order, drop virtio_mem size requirement that it needs to be the max of pageblock_order and MAX_ORDER. Signed-off-by: Zi Yan --- drivers/virtio/virtio_mem.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 38becd8d578c..2307e65d18c2 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -2476,13 +2476,12 @@ static int virtio_mem_init_hotplug(struct virtio_mem *vm) VIRTIO_MEM_DEFAULT_OFFLINE_THRESHOLD); /* - * We want subblocks to span at least MAX_ORDER_NR_PAGES and - * pageblock_nr_pages pages. This: + * We want subblocks to span at least pageblock_nr_pages pages. + * This: * - Is required for now for alloc_contig_range() to work reliably - * it doesn't properly handle smaller granularity on ZONE_NORMAL. */ - sb_size = max_t(uint64_t, MAX_ORDER_NR_PAGES, - pageblock_nr_pages) * PAGE_SIZE; + sb_size = pageblock_nr_pages * PAGE_SIZE; sb_size = max_t(uint64_t, vm->device_block_size, sb_size); if (sb_size < memory_block_size_bytes() && !force_bbm) { From patchwork Fri Feb 11 16:41:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 1591736 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=D3VqJlhl; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=YQN1sACl; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JwKH35F3Tz9sFq for ; Sat, 12 Feb 2022 03:45:19 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4JwKH32DkJz3dgs for ; Sat, 12 Feb 2022 03:45:19 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=D3VqJlhl; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=YQN1sACl; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sent.com (client-ip=66.111.4.221; helo=new1-smtp.messagingengine.com; envelope-from=zi.yan@sent.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=sent.com header.i=@sent.com header.a=rsa-sha256 header.s=fm2 header.b=D3VqJlhl; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=YQN1sACl; dkim-atps=neutral Received: from new1-smtp.messagingengine.com (new1-smtp.messagingengine.com [66.111.4.221]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4JwKC54gMBz3bZF for ; Sat, 12 Feb 2022 03:41:53 +1100 (AEDT) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailnew.nyi.internal (Postfix) with ESMTP id 7367F58037F; Fri, 11 Feb 2022 11:41:51 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 11 Feb 2022 11:41:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to :reply-to:sender:subject:subject:to:to; s=fm2; bh=8yuexR1+nrtHGS O8wGNcV0MvB+tGU4p0S4nQ2u6/h9Y=; b=D3VqJlhlbyWtV6r6GqE/sdQHhhzfao jEbpnb9Wr0ZReRqYHRz6KCXTCd0yivunmYqzFRoQLkQegNx2uHlUfqnjdWV/uzSJ zU8M0dPa33aRq+dlCTIv1HKtBD/Kh7pykEWL/ODQJPonFg0wqPv8sIrL0qYanslJ /JLrIUPIaJr4Z6KM4Og/F2X8lkSYDPbN7ITWDitlcOlpiq78n71xOGLIXJXz0Odg DRLO19YIQToJHSo/aJlV9Z59lWZxCAiI+H1agFbgcJWflqv/7sa6Z/DEHrtZkp3d sm83CKxYJVqNKbxcCcoENzJCEn26apNqoREHw+PQFH0GDXE8WDR4bjCQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=8yuexR1+nrtHGSO8wGNcV0MvB+tGU4p0S4nQ2u6/h9Y=; b=YQN1sACl 87xj+F47lfbNUuu2eOmgbTwH2DjTS/HIzp4OOKxqbF9Ccp5g2TMpkwWX2gBWSZNt EIPNTSk5Nl922Lg6gwveaU2GuyEA3/UfkKtvPP7p+AGcRnn4sIlGCdRzlTfHo2x1 OxQNvyL/puQhqVLBtDsJccw8y7Iaq1sUjOQ2m/rnf7M6NoneAkAyKwJj4wZzSkaB WxyqKOeTGmYirNdGfaMPZnppY7uXdXPT4AHgxf5NVJMRxE3UDqkMzGCzf8L0gtEi VzZ9ycKNj3Y88HlVS7sXcuD+X/zVMZSnrQ0sI1ttCUNu1pmn/5U64GI9YlSxP+j6 Ee5VZdGDqCSGFw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvvddrieefgdeklecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhhrggfgsedtqhertdertddtnecuhfhrohhmpegkihcujggr nhcuoeiiihdrhigrnhesshgvnhhtrdgtohhmqeenucggtffrrghtthgvrhhnpeeijeeuvd euudeuhfeghfehieeuvdetvdeugfeigeevteeuieeuhedtgeduheefleenucevlhhushht vghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeiiihdrhigrnhesshgvnh htrdgtohhm X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 11 Feb 2022 11:41:50 -0500 (EST) From: Zi Yan To: David Hildenbrand , linux-mm@kvack.org Subject: [PATCH v5 6/6] arch: powerpc: adjust fadump alignment to be pageblock aligned. Date: Fri, 11 Feb 2022 11:41:35 -0500 Message-Id: <20220211164135.1803616-7-zi.yan@sent.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220211164135.1803616-1-zi.yan@sent.com> References: <20220211164135.1803616-1-zi.yan@sent.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Zi Yan Cc: Mel Gorman , Zi Yan , Oscar Salvador , Robin Murphy , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Mike Rapoport , Eric Ren , virtualization@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org, Christoph Hellwig , Vlastimil Babka , Marek Szyprowski Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" From: Zi Yan CMA only requires pageblock alignment now. Change CMA alignment in fadump too. Signed-off-by: Zi Yan --- arch/powerpc/include/asm/fadump-internal.h | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/arch/powerpc/include/asm/fadump-internal.h b/arch/powerpc/include/asm/fadump-internal.h index 52189928ec08..fbfca85b4200 100644 --- a/arch/powerpc/include/asm/fadump-internal.h +++ b/arch/powerpc/include/asm/fadump-internal.h @@ -20,9 +20,7 @@ #define memblock_num_regions(memblock_type) (memblock.memblock_type.cnt) /* Alignment per CMA requirement. */ -#define FADUMP_CMA_ALIGNMENT (PAGE_SIZE << \ - max_t(unsigned long, MAX_ORDER - 1, \ - pageblock_order)) +#define FADUMP_CMA_ALIGNMENT (PAGE_SIZE << pageblock_order) /* FAD commands */ #define FADUMP_REGISTER 1