{"id":805741,"url":"http://patchwork.ozlabs.org/api/patches/805741/?format=json","web_url":"http://patchwork.ozlabs.org/project/linuxppc-dev/patch/20170825043036.18236-2-npiggin@gmail.com/","project":{"id":2,"url":"http://patchwork.ozlabs.org/api/projects/2/?format=json","name":"Linux PPC development","link_name":"linuxppc-dev","list_id":"linuxppc-dev.lists.ozlabs.org","list_email":"linuxppc-dev@lists.ozlabs.org","web_url":"https://github.com/linuxppc/wiki/wiki","scm_url":"https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git","webscm_url":"https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/","list_archive_url":"https://lore.kernel.org/linuxppc-dev/","list_archive_url_format":"https://lore.kernel.org/linuxppc-dev/{}/","commit_url_format":"https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?id={}"},"msgid":"<20170825043036.18236-2-npiggin@gmail.com>","list_archive_url":"https://lore.kernel.org/linuxppc-dev/20170825043036.18236-2-npiggin@gmail.com/","date":"2017-08-25T04:30:33","name":"[v3,1/4] KVM: PPC: Book3S HV: POWER9 does not require secondary thread management","commit_ref":"94a04bc25a2c6296bd0c5e82c10e8231c2b11f77","pull_url":null,"state":"accepted","archived":false,"hash":"0db46d58d3ab556bb99c1e0d72a6cdef4fa1d9fe","submitter":{"id":69518,"url":"http://patchwork.ozlabs.org/api/people/69518/?format=json","name":"Nicholas Piggin","email":"npiggin@gmail.com"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/linuxppc-dev/patch/20170825043036.18236-2-npiggin@gmail.com/mbox/","series":[],"comments":"http://patchwork.ozlabs.org/api/patches/805741/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/805741/checks/","tags":{},"related":[],"headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xdpGQ4gJ7z9t5h\n\tfor ;\n\tFri, 25 Aug 2017 14:34:22 +1000 (AEST)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3xdpGQ3TJhzDrKg\n\tfor ;\n\tFri, 25 Aug 2017 14:34:22 +1000 (AEST)","from mail-pg0-x244.google.com (mail-pg0-x244.google.com\n\t[IPv6:2607:f8b0:400e:c05::244])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3xdpBT44Y0zDrK0\n\tfor ;\n\tFri, 25 Aug 2017 14:30:57 +1000 (AEST)","by mail-pg0-x244.google.com with SMTP id 83so2185147pgb.3\n\tfor ;\n\tThu, 24 Aug 2017 21:30:57 -0700 (PDT)","from roar.au.ibm.com (203-219-56-202.tpgi.com.au. [203.219.56.202])\n\tby smtp.gmail.com with ESMTPSA id\n\tu195sm9010365pgb.82.2017.08.24.21.30.52\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tThu, 24 Aug 2017 21:30:54 -0700 (PDT)"],"Authentication-Results":["ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"aaYJ2fxx\"; dkim-atps=neutral","lists.ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"aaYJ2fxx\"; dkim-atps=neutral","lists.ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"aaYJ2fxx\"; dkim-atps=neutral"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=from:to:cc:subject:date:message-id:in-reply-to:references;\n\tbh=C6jyUiPRmZftoS5yI8yOlf885g7banSH1EtWopb6HDw=;\n\tb=aaYJ2fxx0JQU7o4sBJ/V5NH4ac0aCigQSBAbT9vCsackpi8IeLMT8R3eTtZj4hzDuO\n\t4Q+3S28jj/NtajspxfY8wrYWyrrsWbK+DWNejEofbskqiM4Ml6EgO1gsNUlWGMoZTM4e\n\tNuSasxxAFQLoT18i9F25bqYvJBsjGQ0GFVN/h2NpoO5RshqFArdYy5OLgfIJXKjTz2H/\n\tWa41R/HG1/6neKFQK8clN80fb2bhpo+qJbQuKiiR4atbXblHmpQWU0Rls4EJ0NcSimTn\n\tOIzGvoIm8d8MftZ515vRk2tblyh4JdqUq1No+QNuzjoCh2jraT2xFtdqlhYIFtC2MPs7\n\tl7Yw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references;\n\tbh=C6jyUiPRmZftoS5yI8yOlf885g7banSH1EtWopb6HDw=;\n\tb=gwiXLolVklpV/ED1R+noFVpml3edXXf6SeUj9NyrnCMWDBxERw9WD23DX8dSZM4FpH\n\tNhszikK+QuE8RgKqx9xk5qib6T07UxhAysuvG9xYz1dcIsoDD34XcenHIX2hkMoUsGfJ\n\tO5pXdwbc3XhOxz6VwAh/zNftOF/O/fEevhztbr8EOMuCLIC4gUM8O8qFkbo0J4AIG7Ub\n\tD+g6b+a8UZLjxu7tYuP5ZW13UmTH4onClEGUE+kh0LmHdjvaFd4b/3TJhZGlyzB2DDN8\n\thKVDDiGoM8CRJur1ov4ZjXA5QHf484v2bHTCpEHr0YfF2mlco1VP+M/AKG/+IN8C9TnO\n\t/YAw==","X-Gm-Message-State":"AHYfb5iR8pRh6m0tAQOPM8GiB/nyDT6B/wkoAf/ovYULkuM2i+92MwNY\n\thth4dVITqx8JuD1t","X-Received":"by 10.98.64.18 with SMTP id n18mr8672365pfa.272.1503635455399;\n\tThu, 24 Aug 2017 21:30:55 -0700 (PDT)","From":"Nicholas Piggin ","To":"linuxppc-dev@lists.ozlabs.org","Subject":"[PATCH v3 1/4] KVM: PPC: Book3S HV: POWER9 does not require\n\tsecondary thread management","Date":"Fri, 25 Aug 2017 14:30:33 +1000","Message-Id":"<20170825043036.18236-2-npiggin@gmail.com>","X-Mailer":"git-send-email 2.13.3","In-Reply-To":"<20170825043036.18236-1-npiggin@gmail.com>","References":"<20170825043036.18236-1-npiggin@gmail.com>","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.23","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kvm-ppc@vger.kernel.org, Nicholas Piggin ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"},"content":"POWER9 CPUs have independent MMU contexts per thread, so KVM does not\nneed to quiesce secondary threads, so the hwthread_req/hwthread_state\nprotocol does not have to be used. So patch it away on POWER9, and patch\naway the branch from the Linux idle wakeup to kvm_start_guest that is\nnever used.\n\nAdd a warning and error out of kvmppc_grab_hwthread in case it is ever\ncalled on POWER9.\n\nThis avoids a hwsync in the idle wakeup path on POWER9.\n\nSigned-off-by: Nicholas Piggin \n---\n arch/powerpc/include/asm/kvm_book3s_asm.h | 4 ++++\n arch/powerpc/kernel/idle_book3s.S | 35 +++++++++++++++++++++----------\n arch/powerpc/kvm/book3s_hv.c | 14 ++++++++++++-\n arch/powerpc/kvm/book3s_hv_rmhandlers.S | 8 +++++++\n 4 files changed, 49 insertions(+), 12 deletions(-)","diff":"diff --git a/arch/powerpc/include/asm/kvm_book3s_asm.h b/arch/powerpc/include/asm/kvm_book3s_asm.h\nindex 7cea76f11c26..83596f32f50b 100644\n--- a/arch/powerpc/include/asm/kvm_book3s_asm.h\n+++ b/arch/powerpc/include/asm/kvm_book3s_asm.h\n@@ -104,6 +104,10 @@ struct kvmppc_host_state {\n \tu8 napping;\n \n #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE\n+\t/*\n+\t * hwthread_req/hwthread_state pair is used to pull sibling threads\n+\t * out of guest on pre-ISAv3.0B CPUs where threads share MMU.\n+\t */\n \tu8 hwthread_req;\n \tu8 hwthread_state;\n \tu8 host_ipi;\ndiff --git a/arch/powerpc/kernel/idle_book3s.S b/arch/powerpc/kernel/idle_book3s.S\nindex bfbf0976fc09..4924647d964d 100644\n--- a/arch/powerpc/kernel/idle_book3s.S\n+++ b/arch/powerpc/kernel/idle_book3s.S\n@@ -296,13 +296,20 @@ enter_winkle:\n /*\n * r3 - PSSCR value corresponding to the requested stop state.\n */\n-power_enter_stop:\n #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE\n-\t/* Tell KVM we're entering idle */\n+power_enter_stop_kvm_rm:\n+\t/*\n+\t * This is currently unused because POWER9 KVM does not have to\n+\t * gather secondary threads into sibling mode, but the code is\n+\t * here in case that function is required.\n+\t *\n+\t * Tell KVM we're entering idle.\n+\t */\n \tli\tr4,KVM_HWTHREAD_IN_IDLE\n \t/* DO THIS IN REAL MODE! See comment above. */\n \tstb\tr4,HSTATE_HWTHREAD_STATE(r13)\n #endif\n+power_enter_stop:\n /*\n * Check if we are executing the lite variant with ESL=EC=0\n */\n@@ -465,6 +472,18 @@ pnv_powersave_wakeup_mce:\n \n \tb\tpnv_powersave_wakeup\n \n+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE\n+kvm_start_guest_check:\n+\tli\tr0,KVM_HWTHREAD_IN_KERNEL\n+\tstb\tr0,HSTATE_HWTHREAD_STATE(r13)\n+\t/* Order setting hwthread_state vs. testing hwthread_req */\n+\tsync\n+\tlbz\tr0,HSTATE_HWTHREAD_REQ(r13)\n+\tcmpwi\tr0,0\n+\tbeqlr\n+\tb\tkvm_start_guest\n+#endif\n+\n /*\n * Called from reset vector for powersave wakeups.\n * cr3 - set to gt if waking up with partial/complete hypervisor state loss\n@@ -489,15 +508,9 @@ ALT_FTR_SECTION_END_IFSET(CPU_FTR_ARCH_300)\n \tmr\tr3,r12\n \n #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE\n-\tli\tr0,KVM_HWTHREAD_IN_KERNEL\n-\tstb\tr0,HSTATE_HWTHREAD_STATE(r13)\n-\t/* Order setting hwthread_state vs. testing hwthread_req */\n-\tsync\n-\tlbz\tr0,HSTATE_HWTHREAD_REQ(r13)\n-\tcmpwi\tr0,0\n-\tbeq\t1f\n-\tb\tkvm_start_guest\n-1:\n+BEGIN_FTR_SECTION\n+\tbl\tkvm_start_guest_check\n+END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300)\n #endif\n \n \t/* Return SRR1 from power7_nap() */\ndiff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c\nindex 359c79cdf0cc..e34cd6fb947b 100644\n--- a/arch/powerpc/kvm/book3s_hv.c\n+++ b/arch/powerpc/kvm/book3s_hv.c\n@@ -2111,6 +2111,16 @@ static int kvmppc_grab_hwthread(int cpu)\n \tstruct paca_struct *tpaca;\n \tlong timeout = 10000;\n \n+\t/*\n+\t * ISA v3.0 idle routines do not set hwthread_state or test\n+\t * hwthread_req, so they can not grab idle threads.\n+\t */\n+\tif (cpu_has_feature(CPU_FTR_ARCH_300)) {\n+\t\tWARN_ON(1);\n+\t\tpr_err(\"KVM: can not control sibling threads\\n\");\n+\t\treturn -EBUSY;\n+\t}\n+\n \ttpaca = &paca[cpu];\n \n \t/* Ensure the thread won't go into the kernel if it wakes */\n@@ -2145,10 +2155,12 @@ static void kvmppc_release_hwthread(int cpu)\n \tstruct paca_struct *tpaca;\n \n \ttpaca = &paca[cpu];\n-\ttpaca->kvm_hstate.hwthread_req = 0;\n \ttpaca->kvm_hstate.kvm_vcpu = NULL;\n \ttpaca->kvm_hstate.kvm_vcore = NULL;\n \ttpaca->kvm_hstate.kvm_split_mode = NULL;\n+\tif (!cpu_has_feature(CPU_FTR_ARCH_300))\n+\t\ttpaca->kvm_hstate.hwthread_req = 0;\n+\n }\n \n static void radix_flush_cpu(struct kvm *kvm, int cpu, struct kvm_vcpu *vcpu)\ndiff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S\nindex c52184a8efdf..3e024fd71fe8 100644\n--- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S\n+++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S\n@@ -149,9 +149,11 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)\n \tsubf\tr4, r4, r3\n \tmtspr\tSPRN_DEC, r4\n \n+BEGIN_FTR_SECTION\n \t/* hwthread_req may have got set by cede or no vcpu, so clear it */\n \tli\tr0, 0\n \tstb\tr0, HSTATE_HWTHREAD_REQ(r13)\n+END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300)\n \n \t/*\n \t * For external interrupts we need to call the Linux\n@@ -314,6 +316,7 @@ kvm_novcpu_exit:\n * Relocation is off and most register values are lost.\n * r13 points to the PACA.\n * r3 contains the SRR1 wakeup value, SRR1 is trashed.\n+ * This is not used by ISAv3.0B processors.\n */\n \t.globl\tkvm_start_guest\n kvm_start_guest:\n@@ -432,6 +435,9 @@ kvm_secondary_got_guest:\n * While waiting we also need to check if we get given a vcpu to run.\n */\n kvm_no_guest:\n+BEGIN_FTR_SECTION\n+\ttwi\t31,0,0\n+END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300)\n \tlbz\tr3, HSTATE_HWTHREAD_REQ(r13)\n \tcmpwi\tr3, 0\n \tbne\t53f\n@@ -2509,8 +2515,10 @@ kvm_do_nap:\n \tclrrdi\tr0, r0, 1\n \tmtspr\tSPRN_CTRLT, r0\n \n+BEGIN_FTR_SECTION\n \tli\tr0,1\n \tstb\tr0,HSTATE_HWTHREAD_REQ(r13)\n+END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300)\n \tmfspr\tr5,SPRN_LPCR\n \tori\tr5,r5,LPCR_PECE0 | LPCR_PECE1\n BEGIN_FTR_SECTION\n","prefixes":["v3","1/4"]}