From patchwork Tue May 23 13:27:41 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gustavo Walbon X-Patchwork-Id: 765971 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) by ozlabs.org (Postfix) with ESMTP id 3wXGb059Z4z9sPJ; Tue, 23 May 2017 23:29:16 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.76) (envelope-from ) id 1dD9s9-0002Bx-WD; Tue, 23 May 2017 13:29:13 +0000 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]) by huckleberry.canonical.com with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.76) (envelope-from ) id 1dD9s3-0002Ae-N3 for kernel-team@lists.ubuntu.com; Tue, 23 May 2017 13:29:07 +0000 Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v4NDT6N9087796 for ; Tue, 23 May 2017 09:29:06 -0400 Received: from e24smtp01.br.ibm.com (e24smtp01.br.ibm.com [32.104.18.85]) by mx0a-001b2d01.pphosted.com with ESMTP id 2amc2gacy5-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 23 May 2017 09:29:05 -0400 Received: from localhost by e24smtp01.br.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 23 May 2017 10:28:43 -0300 Received: from d24relay04.br.ibm.com (9.18.232.146) by e24smtp01.br.ibm.com (10.172.0.143) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 23 May 2017 10:28:40 -0300 Received: from d24av03.br.ibm.com (d24av03.br.ibm.com [9.8.31.95]) by d24relay04.br.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v4NDSeT23604972 for ; Tue, 23 May 2017 10:28:40 -0300 Received: from d24av03.br.ibm.com (localhost [127.0.0.1]) by d24av03.br.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id v4NDSflF025962 for ; Tue, 23 May 2017 10:28:41 -0300 Received: from localhost (gwalbon.br.ibm.com [9.18.235.110] (may be forged)) by d24av03.br.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id v4NDSfcH025953; Tue, 23 May 2017 10:28:41 -0300 From: Gustavo Walbon To: kernel-team@lists.ubuntu.com Subject: [Zesty][PATCH v2 06/13] powerpc: Introduce msgsnd/doorbell barrier primitives Date: Tue, 23 May 2017 10:27:41 -0300 X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170523132748.19944-1-gwalbon@linux.vnet.ibm.com> References: <20170523132748.19944-1-gwalbon@linux.vnet.ibm.com> X-TM-AS-MML: disable x-cbid: 17052313-1523-0000-0000-000002A1F217 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17052313-1524-0000-0000-00002A3872EC Message-Id: <20170523132748.19944-7-gwalbon@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-05-23_04:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=4 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705230070 Cc: mauricfo@linux.vnet.ibm.com X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.14 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: kernel-team-bounces@lists.ubuntu.com From: Nicholas Piggin Buglink : http://bugs.launchpad.net/bugs/1691973 POWER9 changes requirements and adds new instructions for synchronization. Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman (cherry picked from commit b87ac0218355a83abb899a0022bb2e5252879fc0 in linux-next) Signed-off-by: Gustavo Walbon --- arch/powerpc/include/asm/dbell.h | 22 ++++++++++++++++++++++ arch/powerpc/include/asm/smp.h | 1 + arch/powerpc/kernel/dbell.c | 8 +++++--- arch/powerpc/kernel/smp.c | 10 ++++++++-- 4 files changed, 36 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/include/asm/dbell.h b/arch/powerpc/include/asm/dbell.h index a66eb596940c..350694a1a6e5 100644 --- a/arch/powerpc/include/asm/dbell.h +++ b/arch/powerpc/include/asm/dbell.h @@ -44,6 +44,17 @@ static inline void _ppc_msgsnd(u32 msg) __asm__ __volatile__ (PPC_MSGSNDP(%0) : : "r" (msg)); } +/* sync before sending message */ +static inline void ppc_msgsnd_sync(void) +{ + __asm__ __volatile__ ("sync" : : : "memory"); +} + +/* sync after taking message interrupt */ +static inline void ppc_msgsync(void) +{ +} + #else /* CONFIG_PPC_BOOK3S */ #define PPC_DBELL_MSGTYPE PPC_DBELL @@ -53,6 +64,17 @@ static inline void _ppc_msgsnd(u32 msg) __asm__ __volatile__ (PPC_MSGSND(%0) : : "r" (msg)); } +/* sync before sending message */ +static inline void ppc_msgsnd_sync(void) +{ + __asm__ __volatile__ ("sync" : : : "memory"); +} + +/* sync after taking message interrupt */ +static inline void ppc_msgsync(void) +{ +} + #endif /* CONFIG_PPC_BOOK3S */ extern void doorbell_global_ipi(int cpu); diff --git a/arch/powerpc/include/asm/smp.h b/arch/powerpc/include/asm/smp.h index b2e4e9a4e3f8..edd939e76d52 100644 --- a/arch/powerpc/include/asm/smp.h +++ b/arch/powerpc/include/asm/smp.h @@ -129,6 +129,7 @@ extern const char *smp_ipi_name[]; extern void smp_muxed_ipi_message_pass(int cpu, int msg); extern void smp_muxed_ipi_set_message(int cpu, int msg); extern irqreturn_t smp_ipi_demux(void); +extern irqreturn_t smp_ipi_demux_relaxed(void); void smp_init_pSeries(void); void smp_init_cell(void); diff --git a/arch/powerpc/kernel/dbell.c b/arch/powerpc/kernel/dbell.c index 5869c66adfd0..b6fe883b1016 100644 --- a/arch/powerpc/kernel/dbell.c +++ b/arch/powerpc/kernel/dbell.c @@ -38,7 +38,7 @@ void doorbell_global_ipi(int cpu) kvmppc_set_host_ipi(cpu, 1); /* Order previous accesses vs. msgsnd, which is treated as a store */ - mb(); + ppc_msgsnd_sync(); ppc_msgsnd(PPC_DBELL_MSGTYPE, 0, tag); } @@ -53,7 +53,7 @@ void doorbell_core_ipi(int cpu) kvmppc_set_host_ipi(cpu, 1); /* Order previous accesses vs. msgsnd, which is treated as a store */ - mb(); + ppc_msgsnd_sync(); ppc_msgsnd(PPC_DBELL_MSGTYPE, 0, tag); } @@ -82,12 +82,14 @@ void doorbell_exception(struct pt_regs *regs) irq_enter(); + ppc_msgsync(); + may_hard_irq_enable(); kvmppc_set_host_ipi(smp_processor_id(), 0); __this_cpu_inc(irq_stat.doorbell_irqs); - smp_ipi_demux(); + smp_ipi_demux_relaxed(); /* already performed the barrier */ irq_exit(); set_irq_regs(old_regs); diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c index f7b389185210..95eefa00abb3 100644 --- a/arch/powerpc/kernel/smp.c +++ b/arch/powerpc/kernel/smp.c @@ -245,11 +245,17 @@ void smp_muxed_ipi_message_pass(int cpu, int msg) irqreturn_t smp_ipi_demux(void) { + mb(); /* order any irq clear */ + + return smp_ipi_demux_relaxed(); +} + +/* sync-free variant. Callers should ensure synchronization */ +irqreturn_t smp_ipi_demux_relaxed(void) +{ struct cpu_messages *info; unsigned long all; - mb(); /* order any irq clear */ - info = this_cpu_ptr(&ipi_message); do { all = xchg(&info->messages, 0);