From patchwork Fri Jun 9 10:00:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1792844 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=ncEkNY11; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QcxSj2S94z20Ty for ; Fri, 9 Jun 2023 20:01:57 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4QcxSj1G94z3f8b for ; Fri, 9 Jun 2023 20:01:57 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=ncEkNY11; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::433; helo=mail-pf1-x433.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=ncEkNY11; dkim-atps=neutral Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4QcxRM26Xxz3f0r for ; Fri, 9 Jun 2023 20:00:46 +1000 (AEST) Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-64d18d772bdso1637059b3a.3 for ; Fri, 09 Jun 2023 03:00:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686304841; x=1688896841; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=I3qEUcp9VtH7fSu0EhO1yZM8TfaADYffpYWhYVNdp30=; b=ncEkNY11kDE8NjfXAQx1zkrVUp11yFiJeH7pSqTJq0wMFiMuFGfXP4MYkwk40+6Owq Lv5ujBOS11RkRBgBThHqam57+4F1fV8uXmAGMJKBu+zVGP2HVLqAQmv/ealRPi6Tn4fU mbHisb74F9r6afzam23CSyxZrx4nK14PSWQ4X0Q1YbIgudTMQ2HK88KeDsns0gT/p7Dp anokNWBS9LgIy6ovCsyZHlCQKZ8KagAW0FcwdU6t6OPLyAcktF4ZhxSPFiNMxKW3qF8x TKXpF/KXlWfxVTPmbiVempjhCT49AX6u2DaPZKC89EUdVrN23144ol56TACB2m+Ovbj0 drqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686304841; x=1688896841; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=I3qEUcp9VtH7fSu0EhO1yZM8TfaADYffpYWhYVNdp30=; b=WiOPhIlfP5vwXeRkTrAvQrEqKXuZjtyrrn/ngl68cPl5rOZnWDD6Gvz5Gb574aPOPc KA28ReePneUVgErBukBoIXw519qAOD5siiIk6S46xU77PSJ1IDijipwakcj4JJdcn2ZO fzfCUoHnAMV4NN54NkhrLEGQpbj4yCjltvHLAQbhdFIhIkIio9+gd0CVNxGXX2GgUiij l4uAnJ4E03CpzUBXJgUDjRikZPblGVzTL5UBodLRvwZWvqgZv/FNY54Wk49WR8eo6O6Q qzJvC34i7rvDIXuI5ZUJjpY3CD0EzOPk5GHg6c69rlBRqx+9z+FU3ZwOzIcKeEpB6Wce vfPw== X-Gm-Message-State: AC+VfDz5do4GpRLbRyYKhiT5u8HSeHcm9T14U0K19ayx2R30XyMzxMrS z8HRVUduo8Xxs/0/HCgHIN+XsDwnUtQ= X-Google-Smtp-Source: ACHHUZ4dH5gbqopm7vLiOPFzNrCFHaxR6RheX9uxqxQoD6qfP9uPIDFp9QT2SIuwPcsfYvtiSr+Jkw== X-Received: by 2002:a17:902:9b8b:b0:1b1:d0b2:b9b3 with SMTP id y11-20020a1709029b8b00b001b1d0b2b9b3mr792598plp.23.1686304839514; Fri, 09 Jun 2023 03:00:39 -0700 (PDT) Received: from wheely.local0.net ([193.114.110.224]) by smtp.gmail.com with ESMTPSA id jh11-20020a170903328b00b001b077301a58sm2912864plb.79.2023.06.09.03.00.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Jun 2023 03:00:38 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH 1/4] powerpc: Make mmiowb a wmb Date: Fri, 9 Jun 2023 20:00:23 +1000 Message-Id: <20230609100026.8946-1-npiggin@gmail.com> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" mmiowb must ensure MMIO stores inside a spin lock critical section on one CPU will not be seen by the device after another CPU takes the same lock and performs MMIOs. This just requires cache inhibited stores to be ordered with the store to unlock the spinlock, so wmb() can be used. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/mmiowb.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/powerpc/include/asm/mmiowb.h b/arch/powerpc/include/asm/mmiowb.h index 74a00127eb20..cd071fb97eba 100644 --- a/arch/powerpc/include/asm/mmiowb.h +++ b/arch/powerpc/include/asm/mmiowb.h @@ -9,7 +9,7 @@ #include #define arch_mmiowb_state() (&local_paca->mmiowb_state) -#define mmiowb() mb() +#define mmiowb() wmb() #endif /* CONFIG_MMIOWB */ From patchwork Fri Jun 9 10:00:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1792843 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=d+MYNq61; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QcxRm4jZMz20Ty for ; Fri, 9 Jun 2023 20:01:08 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4QcxRl5YPWz3f60 for ; Fri, 9 Jun 2023 20:01:07 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=d+MYNq61; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::430; helo=mail-pf1-x430.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=d+MYNq61; dkim-atps=neutral Received: from mail-pf1-x430.google.com (mail-pf1-x430.google.com [IPv6:2607:f8b0:4864:20::430]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4QcxRM1KrMz3f0M for ; Fri, 9 Jun 2023 20:00:46 +1000 (AEST) Received: by mail-pf1-x430.google.com with SMTP id d2e1a72fcca58-651e298be3fso1641616b3a.2 for ; Fri, 09 Jun 2023 03:00:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686304842; x=1688896842; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KyOVrAdZI1xc5hD/G0usUVxu/VJz/F7Hk7mfWyG1a/o=; b=d+MYNq61z7HWLZWKq6rB7AbthBDlC9svmFNWsghBSeXylLaSMSQ0opfxx8AOv4wllu eMJcRmceklHwvlKT8grkZjO3PS+R40REVM4UlFdv0ay3xQwx7KbpZXHtsA1CM01jfUV/ k1QJM41l19KMcG0quudmZYBH2+jo3cgFLK78hJaDVY1r/i8KVh56KDmOkp890qMypbNG r7+OI8qZSi9G/8MzAldmjFjdLu8YLMK2z6zWSFtvpa1+qPkmS756nZSniFFo6OvvWc10 3XP9kJ1DSHppEj/IVydGFYi+EoAupuyXOVhs7pr9VTc+IWvUwJPsOeG9O864DRHNY6gd ep9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686304842; x=1688896842; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KyOVrAdZI1xc5hD/G0usUVxu/VJz/F7Hk7mfWyG1a/o=; b=Bd+KaNmCzd3vQSq/aOS3EON/Aw8+m+Q/3+8Ouu+USxfDf03twpuGqcNZlBp66htrpw joDt3At7xkenlpm/z9+PRsYspHNtjORfmdkV2d+jk3vOlXp3AkZPH3bK54VLBz+Pl6b4 xX+C3Y26XP/no9kErzbcm2lIRUfR+4frUkU0BS8JoQSBaALmLf7WV9IINMrWAzy7gsbO sUDmSpgMm1d+muzlQg7WiH+Ef0SskMyuff1QMjnome322xVSffn70g2sqf0sdqueGIZH wrRg4jwjr2VH3DpC1g0vDK3ZErX0nRFBZIbK00XUde1ywB9HwS4Wi1KA6eibkyJQOT83 LhcQ== X-Gm-Message-State: AC+VfDwR0yCVBGomiLzFfRmjgH9wCuNjRb/JMfV7a5oNiSHhhN8C/C4B LuEEZN8TrEBqa/bSQptTxZ89H3hxDuY= X-Google-Smtp-Source: ACHHUZ6KjAvgTxOI6RBW+/3RuXFShhnH2y8kEd7DNcNmY8xsf+5x8j/VddnBHiRk77t9oVQReI8ZOA== X-Received: by 2002:a05:6a20:394b:b0:10c:b441:5bd0 with SMTP id r11-20020a056a20394b00b0010cb4415bd0mr1004154pzg.18.1686304842494; Fri, 09 Jun 2023 03:00:42 -0700 (PDT) Received: from wheely.local0.net ([193.114.110.224]) by smtp.gmail.com with ESMTPSA id jh11-20020a170903328b00b001b077301a58sm2912864plb.79.2023.06.09.03.00.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Jun 2023 03:00:41 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH 2/4] powerpc/64s: Add POWER10 store sync mnemonics Date: Fri, 9 Jun 2023 20:00:24 +1000 Message-Id: <20230609100026.8946-2-npiggin@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230609100026.8946-1-npiggin@gmail.com> References: <20230609100026.8946-1-npiggin@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" ISA v3.1 introduces new sync types for store ordering. stncisync stcisync stsync Add ppc-opcode defines for these. This changes PPC_RAW_SYNC to take L,SC parameters and adds a PPC_RAW_HWSYNC for callers that want the plain old sync (aka hwsync). Signed-off-by: Nicholas Piggin Reviewed-by: Joel Stanley . --- arch/powerpc/include/asm/ppc-opcode.h | 19 ++++++++++++++----- arch/powerpc/kernel/traps.c | 2 +- arch/powerpc/lib/feature-fixups.c | 6 +++--- arch/powerpc/net/bpf_jit_comp64.c | 2 +- 4 files changed, 19 insertions(+), 10 deletions(-) diff --git a/arch/powerpc/include/asm/ppc-opcode.h b/arch/powerpc/include/asm/ppc-opcode.h index ca5a0da7df4e..7bc8bbcd4adb 100644 --- a/arch/powerpc/include/asm/ppc-opcode.h +++ b/arch/powerpc/include/asm/ppc-opcode.h @@ -326,6 +326,8 @@ #define ___PPC_R(r) (((r) & 0x1) << 16) #define ___PPC_PRS(prs) (((prs) & 0x1) << 17) #define ___PPC_RIC(ric) (((ric) & 0x3) << 18) +#define ___PPC_L(l) (((l) & 0x7) << 21) +#define ___PPC_SC(sc) (((sc) & 0x3) << 16) #define __PPC_RA(a) ___PPC_RA(__REG_##a) #define __PPC_RA0(a) ___PPC_RA(__REGA0_##a) #define __PPC_RB(b) ___PPC_RB(__REG_##b) @@ -378,8 +380,6 @@ #define PPC_RAW_LQARX(t, a, b, eh) (0x7c000228 | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b) | __PPC_EH(eh)) #define PPC_RAW_LDARX(t, a, b, eh) (0x7c0000a8 | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b) | __PPC_EH(eh)) #define PPC_RAW_LWARX(t, a, b, eh) (0x7c000028 | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b) | __PPC_EH(eh)) -#define PPC_RAW_PHWSYNC (0x7c8004ac) -#define PPC_RAW_PLWSYNC (0x7ca004ac) #define PPC_RAW_STQCX(t, a, b) (0x7c00016d | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b)) #define PPC_RAW_MADDHD(t, a, b, c) (0x10000030 | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b) | ___PPC_RC(c)) #define PPC_RAW_MADDHDU(t, a, b, c) (0x10000031 | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b) | ___PPC_RC(c)) @@ -396,6 +396,13 @@ #define PPC_RAW_RFCI (0x4c000066) #define PPC_RAW_RFDI (0x4c00004e) #define PPC_RAW_RFMCI (0x4c00004c) +#define PPC_RAW_SYNC(l, sc) (0x7c0004ac | ___PPC_L(l) | ___PPC_SC(sc)) +#define PPC_RAW_HWSYNC() PPC_RAW_SYNC(0, 0) +#define PPC_RAW_STNCISYNC() PPC_RAW_SYNC(1, 1) +#define PPC_RAW_STCISYNC() PPC_RAW_SYNC(0, 2) +#define PPC_RAW_STSYNC() PPC_RAW_SYNC(0, 3) +#define PPC_RAW_PHWSYNC() PPC_RAW_SYNC(4, 0) +#define PPC_RAW_PLWSYNC() PPC_RAW_SYNC(5, 0) #define PPC_RAW_TLBILX(t, a, b) (0x7c000024 | __PPC_T_TLB(t) | __PPC_RA0(a) | __PPC_RB(b)) #define PPC_RAW_WAIT_v203 (0x7c00007c) #define PPC_RAW_WAIT(w, p) (0x7c00003c | __PPC_WC(w) | __PPC_PL(p)) @@ -421,7 +428,6 @@ #define PPC_RAW_DCBFPS(a, b) (0x7c0000ac | ___PPC_RA(a) | ___PPC_RB(b) | (4 << 21)) #define PPC_RAW_DCBSTPS(a, b) (0x7c0000ac | ___PPC_RA(a) | ___PPC_RB(b) | (6 << 21)) #define PPC_RAW_SC() (0x44000002) -#define PPC_RAW_SYNC() (0x7c0004ac) #define PPC_RAW_ISYNC() (0x4c00012c) /* @@ -641,8 +647,11 @@ #define STBCIX(s, a, b) stringify_in_c(.long PPC_RAW_STBCIX(s, a, b)) #define PPC_DCBFPS(a, b) stringify_in_c(.long PPC_RAW_DCBFPS(a, b)) #define PPC_DCBSTPS(a, b) stringify_in_c(.long PPC_RAW_DCBSTPS(a, b)) -#define PPC_PHWSYNC stringify_in_c(.long PPC_RAW_PHWSYNC) -#define PPC_PLWSYNC stringify_in_c(.long PPC_RAW_PLWSYNC) +#define PPC_STNCISYNC stringify_in_c(.long PPC_RAW_STNCISYNC()) +#define PPC_STCISYNC stringify_in_c(.long PPC_RAW_STCISYNC()) +#define PPC_STSYNC stringify_in_c(.long PPC_RAW_STSYNC()) +#define PPC_PHWSYNC stringify_in_c(.long PPC_RAW_PHWSYNC()) +#define PPC_PLWSYNC stringify_in_c(.long PPC_RAW_PLWSYNC()) #define STXVD2X(s, a, b) stringify_in_c(.long PPC_RAW_STXVD2X(s, a, b)) #define LXVD2X(s, a, b) stringify_in_c(.long PPC_RAW_LXVD2X(s, a, b)) #define MFVRD(a, t) stringify_in_c(.long PPC_RAW_MFVRD(a, t)) diff --git a/arch/powerpc/kernel/traps.c b/arch/powerpc/kernel/traps.c index 9bdd79aa51cf..4b216c208f41 100644 --- a/arch/powerpc/kernel/traps.c +++ b/arch/powerpc/kernel/traps.c @@ -550,7 +550,7 @@ static inline int check_io_access(struct pt_regs *regs) nip -= 2; else if (*nip == PPC_RAW_ISYNC()) --nip; - if (*nip == PPC_RAW_SYNC() || get_op(*nip) == OP_TRAP) { + if (*nip == PPC_RAW_HWSYNC() || get_op(*nip) == OP_TRAP) { unsigned int rb; --nip; diff --git a/arch/powerpc/lib/feature-fixups.c b/arch/powerpc/lib/feature-fixups.c index 80def1c2afcb..4c6e7111354f 100644 --- a/arch/powerpc/lib/feature-fixups.c +++ b/arch/powerpc/lib/feature-fixups.c @@ -193,7 +193,7 @@ static void do_stf_entry_barrier_fixups(enum stf_barrier_type types) } else if (types & STF_BARRIER_EIEIO) { instrs[i++] = PPC_RAW_EIEIO() | 0x02000000; /* eieio + bit 6 hint */ } else if (types & STF_BARRIER_SYNC_ORI) { - instrs[i++] = PPC_RAW_SYNC(); + instrs[i++] = PPC_RAW_HWSYNC(); instrs[i++] = PPC_RAW_LD(_R10, _R13, 0); instrs[i++] = PPC_RAW_ORI(_R31, _R31, 0); /* speculation barrier */ } @@ -234,7 +234,7 @@ static void do_stf_exit_barrier_fixups(enum stf_barrier_type types) instrs[i++] = PPC_RAW_MTSPR(SPRN_SPRG2, _R13); instrs[i++] = PPC_RAW_MFSPR(_R13, SPRN_SPRG1); } - instrs[i++] = PPC_RAW_SYNC(); + instrs[i++] = PPC_RAW_HWSYNC(); instrs[i++] = PPC_RAW_LD(_R13, _R13, 0); instrs[i++] = PPC_RAW_ORI(_R31, _R31, 0); /* speculation barrier */ if (cpu_has_feature(CPU_FTR_HVMODE)) @@ -543,7 +543,7 @@ void do_barrier_nospec_fixups_range(bool enable, void *fixup_start, void *fixup_ if (enable) { pr_info("barrier-nospec: using isync; sync as speculation barrier\n"); instr[0] = PPC_RAW_ISYNC(); - instr[1] = PPC_RAW_SYNC(); + instr[1] = PPC_RAW_HWSYNC(); } i = do_patch_fixups(start, end, instr, ARRAY_SIZE(instr)); diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c index 0f8048f6dad6..e8257bfc1cb4 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -737,7 +737,7 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, struct codegen_context * EMIT(PPC_RAW_EIEIO() | 0x02000000); break; case STF_BARRIER_SYNC_ORI: - EMIT(PPC_RAW_SYNC()); + EMIT(PPC_RAW_HWSYNC()); EMIT(PPC_RAW_LD(tmp1_reg, _R13, 0)); EMIT(PPC_RAW_ORI(_R31, _R31, 0)); break; From patchwork Fri Jun 9 10:00:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1792846 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=kPOtd26Y; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QcxTf3RhDz20Ty for ; Fri, 9 Jun 2023 20:02:46 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4QcxTd6rKNz3f8b for ; Fri, 9 Jun 2023 20:02:45 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=kPOtd26Y; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::62c; helo=mail-pl1-x62c.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=kPOtd26Y; dkim-atps=neutral Received: from mail-pl1-x62c.google.com (mail-pl1-x62c.google.com [IPv6:2607:f8b0:4864:20::62c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4QcxRM6sz7z3f0M for ; Fri, 9 Jun 2023 20:00:47 +1000 (AEST) Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1b038064d97so5124185ad.0 for ; Fri, 09 Jun 2023 03:00:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686304845; x=1688896845; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kxWsuGJbXj52lRlkVMBXqBUlVvD+pV5/Oic8AfJwtb0=; b=kPOtd26YXIUpHQBBaXUsZg3wsRm0POQGNOqB/PAlr9OPkgFBDfw2tux9AZsMYwHNeS hSFsiiKIT0YOrXw5ge0U52BQOCQyKw4+sNAQ6oFrDdvjOnmSm9Z+sywbgQlaChl0yFhi REe5myMgl3zI1fm19Dtyb6XTJIoZdMK7KWXQP25gwFXGiB5IyZKqOdJ+3o1itilg49M+ bzGeMJyuqD1sSwvlsqVTR5y7s4ZpTHOsz511HshOEQzMHMBlqsZpad+Xx6sbiqk3B7B4 jd4c5cPb45jxGqUE4Nc2Oj5OL3p36bYnJ3k3LRXAUuLY5qadPaaEfMgK40i/OQv8Lfee sAKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686304845; x=1688896845; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kxWsuGJbXj52lRlkVMBXqBUlVvD+pV5/Oic8AfJwtb0=; b=kAXtPwfOUib4U5gbnpHdRALLqxsX8jzdsUVkWgMMgMDFBFeRXZYat++9d/JpxAySQd amMHXsacaz4LGET+RuEtzr9cjxHgFQyU1sCXW0uQpF6Ja1SXpN/t2wW3KYlFqlxpbAdo Woc48fmfCNiFPiPwLiKt2yk8fs91nH9VsBSnWBj6DZc9x/0IPH1BAwdJFnUGeug78RbY BNooEdVU5uyakvQqBhRb6A8hdRPv/qn8u5oAu7+W83mG5fLkqFk0mbsSgnGTeMvVxNTO TDqPQgGn7y9S/x9QUkY2aZOcPHpHkIMNu8c0cihQ6llpT0bZ8jrjg9fIUHSemBXI8Bp6 8svA== X-Gm-Message-State: AC+VfDyuLWOGDUXd80ugAKOMXyMFgZyNqVVXh5QieXmXWBmrEyXEUxTD GgFbWicZ+6JuLj2cbmi6BoAGoHyiNXI= X-Google-Smtp-Source: ACHHUZ7u6/OC+t418wTZQIyUrGNXLfyEomS8uxijqhQGUUmSBxizj/2IVZLyycJvgcyZ+PZR70idQA== X-Received: by 2002:a17:902:ea10:b0:1af:ea40:34f2 with SMTP id s16-20020a170902ea1000b001afea4034f2mr1695645plg.11.1686304844949; Fri, 09 Jun 2023 03:00:44 -0700 (PDT) Received: from wheely.local0.net ([193.114.110.224]) by smtp.gmail.com with ESMTPSA id jh11-20020a170903328b00b001b077301a58sm2912864plb.79.2023.06.09.03.00.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Jun 2023 03:00:44 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH 3/4] powerpc/64s: Use stncisync instruction for smp_wmb() when available Date: Fri, 9 Jun 2023 20:00:25 +1000 Message-Id: <20230609100026.8946-3-npiggin@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230609100026.8946-1-npiggin@gmail.com> References: <20230609100026.8946-1-npiggin@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" stncisync orders less than lwsync (only cacheable store-store, not load-load or load-store) so it should be as cheap or cheaper. Microbenchmarks with no actual loads to order shows that the basic execution cost is the same on POWER10. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/barrier.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/include/asm/barrier.h b/arch/powerpc/include/asm/barrier.h index b95b666f0374..f0ff5737b0d8 100644 --- a/arch/powerpc/include/asm/barrier.h +++ b/arch/powerpc/include/asm/barrier.h @@ -6,6 +6,8 @@ #define _ASM_POWERPC_BARRIER_H #include +#include +#include #ifndef __ASSEMBLY__ #include @@ -41,7 +43,12 @@ /* The sub-arch has lwsync */ #if defined(CONFIG_PPC64) || defined(CONFIG_PPC_E500MC) -# define SMPWMB LWSYNC +# define SMPWMB \ + BEGIN_FTR_SECTION; \ + LWSYNC; \ + FTR_SECTION_ELSE; \ + .long PPC_RAW_STNCISYNC(); \ + ALT_FTR_SECTION_END_IFCLR(CPU_FTR_ARCH_31) #elif defined(CONFIG_BOOKE) # define SMPWMB mbar #else From patchwork Fri Jun 9 10:00:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1792847 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=lU9N2QnF; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QcxVb1K5kz20Ty for ; Fri, 9 Jun 2023 20:03:35 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4QcxVb09tMz3fk8 for ; Fri, 9 Jun 2023 20:03:35 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=lU9N2QnF; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::1032; helo=mail-pj1-x1032.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=lU9N2QnF; dkim-atps=neutral Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4QcxRP6VvTz3f0M for ; Fri, 9 Jun 2023 20:00:49 +1000 (AEST) Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-2562cc85d3dso565338a91.1 for ; Fri, 09 Jun 2023 03:00:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686304847; x=1688896847; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Q7yZjkwR757SzAYNHEABIcCRRLNnVFovUK9RMc9Au7g=; b=lU9N2QnFRcoTjcdV2wZK7DWPPQpne4HLUydYbMv0yB+bLcvRLIAQh1KqedYp7QFEz+ uGKz5MqK+Vv6wXMg06KsZxc+JjhlKW+Enx5F+czrJDaWoe+E6gWAMDfMS7ATKL8pE+08 N/SU0kjdiVUpMrIO7c2aw5IWXGbtH8Rhjl7IAXZpzPRDgf+SrAM1k3AvPJafAzs6y/LN Apg5tBNdcHGBqOl3QbsWyfgVQ/hnfQyhcfewFKjm//WCk0sdBieOp4q2Uy5QLoOkc+zr 1J54/zKoQJQhAA21NfKfJVp4+qKL8w/Vl7XtU7yhr2m6YbUQqxGworKrf1aHHOxYlrAw VmsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686304847; x=1688896847; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Q7yZjkwR757SzAYNHEABIcCRRLNnVFovUK9RMc9Au7g=; b=jlKJNkfG9nWP2CybV+rF8p1qT3lNiyVlqivsprc/TGbG95TW583EDCQHLfgHOe5oZK IGfh6zsc87yYFgkDtAZO2qKNmww7uoF7ayzrxYXSgDm3eZMutZL6NFf+ifQDcabZy3cn O0wKAo2m/TzNP92LRq2DGxdbr+P2Zqi1PonRFS9AOJ062T6lwJK4B5WWzVHynO9r1l5A n5kSt3ZxfINpQoubLCLjJAK15I/XunEh+d5y/Iggnx52X81AaN6WnO9EyQr1PH8Dvrk0 M+1ZcVjYDkkAK9l1a9CHmrfb7k9MF4sFMz0vVpNzhh3fOEyNsxep+9QjA6Ep6PVUVsL+ IpJw== X-Gm-Message-State: AC+VfDxpA9zRqF+BLZUyyY+UIyEt2C1FovSvVwGIu6uGboS2ggE8PVH3 2KZsooS+3l7gb1l4ZTZedTXX9ViWB1g= X-Google-Smtp-Source: ACHHUZ5wbdTfo0TJHCAbmIQb0QtxzbfOHpdAmkTlZfaSC6OVbKl7agQVOkOTRX5szU7auNCoMxI+CQ== X-Received: by 2002:a17:90b:b01:b0:252:75ed:eff5 with SMTP id bf1-20020a17090b0b0100b0025275edeff5mr821925pjb.30.1686304847526; Fri, 09 Jun 2023 03:00:47 -0700 (PDT) Received: from wheely.local0.net ([193.114.110.224]) by smtp.gmail.com with ESMTPSA id jh11-20020a170903328b00b001b077301a58sm2912864plb.79.2023.06.09.03.00.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Jun 2023 03:00:46 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH 4/4] powerpc/64s: Use POWER10 stsync barrier for wmb() Date: Fri, 9 Jun 2023 20:00:26 +1000 Message-Id: <20230609100026.8946-4-npiggin@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230609100026.8946-1-npiggin@gmail.com> References: <20230609100026.8946-1-npiggin@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" The most expensive ordering for hwsync to provide is the store-load barrier, because all prior stores have to be drained to the caches before subsequent instructions can complete. stsync just orders stores which means it can just be a barrer that goes down the store queue and orders draining, and does not prevent completion of subsequent instructions. So it should be faster than hwsync. Use stsync for wmb(). Older processors that don't recognise the SC field should treat this as hwsync. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/barrier.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/powerpc/include/asm/barrier.h b/arch/powerpc/include/asm/barrier.h index f0ff5737b0d8..95e637c1a3b6 100644 --- a/arch/powerpc/include/asm/barrier.h +++ b/arch/powerpc/include/asm/barrier.h @@ -39,7 +39,7 @@ */ #define __mb() __asm__ __volatile__ ("sync" : : : "memory") #define __rmb() __asm__ __volatile__ ("sync" : : : "memory") -#define __wmb() __asm__ __volatile__ ("sync" : : : "memory") +#define __wmb() __asm__ __volatile__ (PPC_STSYNC : : : "memory") /* The sub-arch has lwsync */ #if defined(CONFIG_PPC64) || defined(CONFIG_PPC_E500MC)