From patchwork Tue Sep 12 06:02:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengen Du X-Patchwork-Id: 1832680 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=Ci7mskjf; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=185.125.189.65; helo=lists.ubuntu.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=patchwork.ozlabs.org) Received: from lists.ubuntu.com (lists.ubuntu.com [185.125.189.65]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RlCgR0TYyz1yhZ for ; Tue, 12 Sep 2023 16:03:15 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=lists.ubuntu.com) by lists.ubuntu.com with esmtp (Exim 4.86_2) (envelope-from ) id 1qfwUH-0006jS-HN; Tue, 12 Sep 2023 06:03:01 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by lists.ubuntu.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1qfwU8-0006fN-Vm for kernel-team@lists.ubuntu.com; Tue, 12 Sep 2023 06:02:53 +0000 Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 7A7303F314 for ; Tue, 12 Sep 2023 06:02:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1694498572; bh=KXC5isYZYRLZNlQFCGtUpXsZ5tM3aHUllep6sQzJUss=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Ci7mskjfCtj6ElQ+/l9cyDylr+/TQEBa1UZtzXd/6/I5baVC7f9h681bGFV2Sp6xr X6aSzNf2yo29hGiXycYe5NlXYNY0/DrR+Ze0MYaqn5BicLt+T5ZyQAPsaXGKac9RIn V2csRGFOob/ju+Don73znYrOsFB27VNtPt6G4cotBEG6mSuicPiRMCld6KOhuaP4+q xsTBEKwOlt2U+LLMD4odJFRrsYy6TT3PAOQpE3te511z12tpkly9eQB8KT5ek12h/O 6pl/7lxocYSwhjkSSlsp8ceXyBWmLkBnClE60Caeova6FWlKbixAel9aWRFA4LpVf5 Doj+iFwwiI17Q== Received: by mail-pf1-f200.google.com with SMTP id d2e1a72fcca58-68fc8541d9cso2116559b3a.1 for ; Mon, 11 Sep 2023 23:02:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694498571; x=1695103371; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KXC5isYZYRLZNlQFCGtUpXsZ5tM3aHUllep6sQzJUss=; b=WXRJWfedAGUPZXJPxuGFIvWpc2/mepipE12YjEkW2uznC6mEDzW2UopcnarshVNn2S +8BOjRNNWbFsCuHmO7V9+4J2+3GyXloiKrwzNPMxxhf3lIqAyenrkw5M5bFc9twHUzjG vgz8TWvvSbuOo2vx74S6KFyS9yINwsD0rmnJ12Xl+AUoK9Gf0pPvK82p4dMMfUR+BALt yrUVQKUiJC6YRs3Q5RiGsYtSpCGmm1MckyMdjgaP1jifWy3ycvr42ut6V+z1Q/LL1LKA p2G5urD+7IMZHnpA/9LGQF2cL/wZBsPouX5Xdnu7CxtZsV3Q7Nf7DLtSz8Rfr0jZqSiq jh7g== X-Gm-Message-State: AOJu0Ywbt6+b4Yyx00MdFWX36MEY0bithGUVcsrzf73x/6EkK9x/ap5D 6QiisHq0Ft4ibp4aR6e1xXi4pFsL8lqJvC7IodZ4zRacDeOXeHAsYxykysyL77vQZVzPT+4sLhg 4COCxK3E2nqJHGVpqIg7OJ9FGGIAyOgNbckFWxq9t103NTMNNgA== X-Received: by 2002:a05:6a00:13a6:b0:68e:2478:d6c9 with SMTP id t38-20020a056a0013a600b0068e2478d6c9mr11351262pfg.2.1694498570715; Mon, 11 Sep 2023 23:02:50 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEibqXWTQHUXGgsovXdxLiOqs7S+uo2b3JwMwGgSRrZ4BS47jdpI3quDL/Iagu6nxPVgs3frg== X-Received: by 2002:a05:6a00:13a6:b0:68e:2478:d6c9 with SMTP id t38-20020a056a0013a600b0068e2478d6c9mr11351235pfg.2.1694498570164; Mon, 11 Sep 2023 23:02:50 -0700 (PDT) Received: from chengendu.. (114-44-91-99.dynamic-ip.hinet.net. [114.44.91.99]) by smtp.gmail.com with ESMTPSA id e23-20020aa78c57000000b0068fe9f23bf4sm437581pfd.103.2023.09.11.23.02.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Sep 2023 23:02:49 -0700 (PDT) From: Chengen Du To: kernel-team@lists.ubuntu.com Subject: [SRU][J][PATCH 1/1] KVM: x86/mmu: Track the number of TDP MMU pages, but not the actual pages Date: Tue, 12 Sep 2023 14:02:46 +0800 Message-Id: <20230912060246.150003-2-chengen.du@canonical.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230912060246.150003-1-chengen.du@canonical.com> References: <20230912060246.150003-1-chengen.du@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Sean Christopherson Track the number of TDP MMU "shadow" pages instead of tracking the pages themselves. With the NX huge page list manipulation moved out of the common linking flow, elminating the list-based tracking means the happy path of adding a shadow page doesn't need to acquire a spinlock and can instead inc/dec an atomic. Keep the tracking as the WARN during TDP MMU teardown on leaked shadow pages is very, very useful for detecting KVM bugs. Tracking the number of pages will also make it trivial to expose the counter to userspace as a stat in the future, which may or may not be desirable. Note, the TDP MMU needs to use a separate counter (and stat if that ever comes to be) from the existing n_used_mmu_pages. The TDP MMU doesn't bother supporting the shrinker nor does it honor KVM_SET_NR_MMU_PAGES (because the TDP MMU consumes so few pages relative to shadow paging), and including TDP MMU pages in that counter would break both the shrinker and shadow MMUs, e.g. if a VM is using nested TDP. Cc: Yan Zhao Reviewed-by: Mingwei Zhang Reviewed-by: David Matlack Signed-off-by: Sean Christopherson Reviewed-by: Yan Zhao Message-Id: <20221019165618.927057-6-seanjc@google.com> Signed-off-by: Paolo Bonzini (backported from commit d25ceb9264364dca0683748b301340097fdab6c7) [chengen - Apply atomic64_t-related changes following the upstream patch's execution order] Signed-off-by: Chengen Du Acked-by: Thadeu Lima de Souza Cascardo --- arch/x86/include/asm/kvm_host.h | 11 +++-------- arch/x86/kvm/mmu/tdp_mmu.c | 20 +++++++++++--------- 2 files changed, 14 insertions(+), 17 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 55d791ad4787..9b7a401a4174 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1168,6 +1168,9 @@ struct kvm_arch { */ bool tdp_mmu_enabled; + /* The number of TDP MMU pages across all roots. */ + atomic64_t tdp_mmu_pages; + /* * List of struct kvm_mmu_pages being used as roots. * All struct kvm_mmu_pages in the list should have @@ -1188,18 +1191,10 @@ struct kvm_arch { */ struct list_head tdp_mmu_roots; - /* - * List of struct kvmp_mmu_pages not being used as roots. - * All struct kvm_mmu_pages in the list should have - * tdp_mmu_page set and a tdp_mmu_root_count of 0. - */ - struct list_head tdp_mmu_pages; - /* * Protects accesses to the following fields when the MMU lock * is held in read mode: * - tdp_mmu_roots (above) - * - tdp_mmu_pages (above) * - the link field of struct kvm_mmu_pages used by the TDP MMU * - lpage_disallowed_mmu_pages * - the lpage_disallowed_link field of struct kvm_mmu_pages used diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 7a64fb238044..04d35439113e 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -24,7 +24,6 @@ bool kvm_mmu_init_tdp_mmu(struct kvm *kvm) INIT_LIST_HEAD(&kvm->arch.tdp_mmu_roots); spin_lock_init(&kvm->arch.tdp_mmu_pages_lock); - INIT_LIST_HEAD(&kvm->arch.tdp_mmu_pages); return true; } @@ -43,7 +42,7 @@ void kvm_mmu_uninit_tdp_mmu(struct kvm *kvm) if (!kvm->arch.tdp_mmu_enabled) return; - WARN_ON(!list_empty(&kvm->arch.tdp_mmu_pages)); + WARN_ON(atomic64_read(&kvm->arch.tdp_mmu_pages)); WARN_ON(!list_empty(&kvm->arch.tdp_mmu_roots)); /* @@ -278,11 +277,12 @@ static void handle_changed_spte_dirty_log(struct kvm *kvm, int as_id, gfn_t gfn, static void tdp_mmu_link_page(struct kvm *kvm, struct kvm_mmu_page *sp, bool account_nx) { - spin_lock(&kvm->arch.tdp_mmu_pages_lock); - list_add(&sp->link, &kvm->arch.tdp_mmu_pages); - if (account_nx) + atomic64_inc(&kvm->arch.tdp_mmu_pages); + if (account_nx) { + spin_lock(&kvm->arch.tdp_mmu_pages_lock); account_huge_nx_page(kvm, sp); - spin_unlock(&kvm->arch.tdp_mmu_pages_lock); + spin_unlock(&kvm->arch.tdp_mmu_pages_lock); + } } /** @@ -297,14 +297,16 @@ static void tdp_mmu_link_page(struct kvm *kvm, struct kvm_mmu_page *sp, static void tdp_mmu_unlink_page(struct kvm *kvm, struct kvm_mmu_page *sp, bool shared) { + atomic64_dec(&kvm->arch.tdp_mmu_pages); + if (!sp->lpage_disallowed) + return; + if (shared) spin_lock(&kvm->arch.tdp_mmu_pages_lock); else lockdep_assert_held_write(&kvm->mmu_lock); - list_del(&sp->link); - if (sp->lpage_disallowed) - unaccount_huge_nx_page(kvm, sp); + unaccount_huge_nx_page(kvm, sp); if (shared) spin_unlock(&kvm->arch.tdp_mmu_pages_lock);