From patchwork Mon Mar 7 23:35:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick O'Neill X-Patchwork-Id: 1602628 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=rivosinc-com.20210112.gappssmtp.com header.i=@rivosinc-com.20210112.gappssmtp.com header.a=rsa-sha256 header.s=20210112 header.b=JwM0lioS; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KCFGM1VkBz9sGT for ; Tue, 8 Mar 2022 10:36:27 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 30AED3857C75 for ; Mon, 7 Mar 2022 23:36:22 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by sourceware.org (Postfix) with ESMTPS id A9A243858408 for ; Mon, 7 Mar 2022 23:36:09 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org A9A243858408 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com Received: by mail-pj1-x102b.google.com with SMTP id p3-20020a17090a680300b001bbfb9d760eso634224pjj.2 for ; Mon, 07 Mar 2022 15:36:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=4BCX1x3mYF9YfL7lbt7H21p6AGdCbz68oJbiaXvDgq0=; b=JwM0lioSRmoArJstaAJxnepKbDr1SN+b0CJ3D42FLwBIABaEvEt++p5eRNUcv7i372 9eQj4VsWY0snxWpMrGAvKlx3kyto2PF4rrM4GwLdmduaPe95+t6vDco1XS8qdQBVltNF ufhLQrMip0PDOu+dmFaz0pwCG1mWiKdExh/h5+tqwj+xfMBzbS8fqKH9TR29s4hcm2qN gkBbnwS3SrNffcvQdPC5904yTWVDVaLRZhyDUcpaJsqHvHyI7zV4ESmR24iIZg6APfKk +AO93D85J5sZOCViUJKK+ddCYM26uO6EqrXFKGkipk63CGm7NzvGK5bqHHA07ivMHTEt D+OQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=4BCX1x3mYF9YfL7lbt7H21p6AGdCbz68oJbiaXvDgq0=; b=7bVPSKGcfvEM5p5JljAU2u2sjRA83ZwqBU26YK4WrnqHsio+dRsZBr5zqr4vxHSw8l RLZ+h1ennqOgTAhDnzQud28m254+5eqWOth2M73CuCE+NBjnJgkSzsjQ/10bEpwC9c62 Oerpt2P41fgdI6oWZ9jf2mG2q483i9aZuPmReQPLnp8rYherShX3PxaPRzwLMBsDV089 nxINY7KDEweYF38jmvhKz7JbpjKTIhTCLE3oT95ncdTfDsG+3bnnHeonhwTUqk854/U7 CEzI5sld46WoQ4jChNBR5SL06Qd1OioplaXYtVwI9oRboMtTAJXG/pdvMijAnK4VwM+j rvnA== X-Gm-Message-State: AOAM531GWl1Ljmwxvq3qmUf5M0M58PKTj44u18lv3ySsF4hNZ58EHqpY no5Y72FYzKemfkMiv0B2zju8QhdK1TIqhA== X-Google-Smtp-Source: ABdhPJygL3zM4JNkzof+Eq5QVMIOJyiJrii5Q5aVxPEEJNWK24oOdvtdLYkmv+iBuIZGKT6unu+Pdg== X-Received: by 2002:a17:90b:390c:b0:1bf:2d83:c70c with SMTP id ob12-20020a17090b390c00b001bf2d83c70cmr1490474pjb.217.1646696168298; Mon, 07 Mar 2022 15:36:08 -0800 (PST) Received: from patrick-ThinkPad-X1-Carbon-Gen-8.hq.rivosinc.com ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id b10-20020a056a00114a00b004f701135460sm5218379pfm.146.2022.03.07.15.36.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Mar 2022 15:36:07 -0800 (PST) From: Patrick O'Neill To: gcc-patches@gcc.gnu.org Subject: [PATCH] RISCV: Strengthen libatomic lrsc pairs Date: Mon, 7 Mar 2022 15:35:42 -0800 Message-Id: <20220307233542.76919-1-patrick@rivosinc.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Patrick O'Neill , dlustig@nvidia.com, vineetg@rivosinc.com Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" Currently, libatomic's _sync_fetch_and_#op# and __sync_val_compare_and_swap methods are not sufficiently strong for the ATOMIC_SEQ_CST memory model. This can be shown using the following Herd litmus test: RISCV LRSC-LIB-CALL { 0:x6=a; 0:x8=b; 0:x10=c; 1:x6=a; 1:x8=b; 1:x10=c; } P0 | P1 ; ori x1,x0,1 | lw x9,0(x10) ; sw x1,0(x10) | fence rw,rw ; lr.w.aq x7,0(x8) | lw x5,0(x6) ; ori x7,x0,1 | ; sc.w.rl x3,x7,0(x8) | ; sw x1,0(x6) | ; exists (1:x9=1 /\ 1:x5=0 /\ b=1) This patch enforces SEQ_CST by setting the .aqrl bits on the LR and SC ops. 2022-03-07 Patrick O'Neill PR target/104831 * atomic.c: Change LR.aq/SC.rl pairs into sequentially consistent LR.aqrl/SC.aqrl pair. Signed-off-by: Patrick O'Neill --- RISCV LRSC-BUGFIX { 0:x6=a; 0:x8=b; 0:x10=c; 1:x6=a; 1:x8=b; 1:x10=c; } P0 | P1 ; ori x1,x0,1 | lw x9,0(x10) ; sw x1,0(x10) | fence rw,rw ; lr.w.aqrl x7,0(x8) | lw x5,0(x6) ; ori x7,x0,1 | ; sc.w.aqrl x3,x7,0(x8) | ; sw x1,0(x6) | ; ~exists (1:x9=1 /\ 1:x5=0 /\ b=1) --- Below are other Herd litmus tests used to test the LR.aqrl/SC.aqrl fix. --- RISCV LRSC-READ (* LR/SC with .aq .rl bits does not allow read operations to be reordered within/above it. *) { 0:x6=a; 0:x8=b; 1:x6=a; 1:x8=b; } P0 | P1 ; lr.w.aq.rl x7,0(x8) | ori x1,x0,1 ; ori x7,x0,1 | sw x1,0(x6) ; sc.w.aq.rl x1,x7,0(x8) | fence rw,rw ; lw x5,0(x6) | lw x7,0(x8) ; ~exists (0:x5=0 /\ 1:x7=0 /\ b=1) --- RISCV READ-LRSC (* LR/SC with .aq .rl bits does not allow read operations to be reordered within/beneath it. *) { 0:x6=a; 0:x8=b; 1:x6=a; 1:x8=b; } P0 | P1 ; lw x5,0(x6) | ori x1,x0,1 ; lr.w.aq.rl x7,0(x8) | sw x1,0(x8) ; ori x1,x0,1 | fence rw,rw ; sc.w.aq.rl x1,x1,0(x8) | sw x1,0(x6) ; ~exists (0:x5=1 /\ 0:x7=0 /\ b=1) --- RISCV LRSC-WRITE (* LR/SC with .aq .rl bits does not allow write operations to be reordered within/above it. *) { 0:x8=b; 0:x10=c; 1:x8=b; 1:x10=c; } P0 | P1 ; ori x9,x0,1 | lw x9,0(x10); lr.w.aq.rl x7,0(x8) | fence rw,rw ; ori x7,x0,1 | lw x7,0(x8) ; sc.w.aq.rl x1,x7,0(x8) | ; sw x9,0(x10) | ; ~exists (1:x9=1 /\ 1:x7=0 /\ b=1) --- RISCV WRITE-LRSC (* LR/SC with .aq .rl bits does not allow write operations to be reordered within/beneath it. *) { 0:x8=b; 0:x10=c; 1:x8=b; 1:x10=c; } P0 | P1 ; ori x1,x0,1 | ori x1,x0,1 ; sw x1,0(x10) | sw x1,0(x8) ; lr.w.aq.rl x7,0(x8) | fence rw,rw ; sc.w.aq.rl x1,x1,0(x8) | lw x9,0(x10) ; ~exists (0:x7=0 /\ 1:x9=0 /\ b=1) --- libgcc/config/riscv/atomic.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/libgcc/config/riscv/atomic.c b/libgcc/config/riscv/atomic.c index 7007e7a20e4..0c85a6d00ea 100644 --- a/libgcc/config/riscv/atomic.c +++ b/libgcc/config/riscv/atomic.c @@ -39,13 +39,13 @@ see the files COPYING3 and COPYING.RUNTIME respectively. If not, see unsigned old, tmp1, tmp2; \ \ asm volatile ("1:\n\t" \ - "lr.w.aq %[old], %[mem]\n\t" \ + "lr.w.aqrl %[old], %[mem]\n\t" \ #insn " %[tmp1], %[old], %[value]\n\t" \ invert \ "and %[tmp1], %[tmp1], %[mask]\n\t" \ "and %[tmp2], %[old], %[not_mask]\n\t" \ "or %[tmp2], %[tmp2], %[tmp1]\n\t" \ - "sc.w.rl %[tmp1], %[tmp2], %[mem]\n\t" \ + "sc.w.aqrl %[tmp1], %[tmp2], %[mem]\n\t" \ "bnez %[tmp1], 1b" \ : [old] "=&r" (old), \ [mem] "+A" (*(volatile unsigned*) aligned_addr), \ @@ -73,12 +73,12 @@ see the files COPYING3 and COPYING.RUNTIME respectively. If not, see unsigned old, tmp1; \ \ asm volatile ("1:\n\t" \ - "lr.w.aq %[old], %[mem]\n\t" \ + "lr.w.aqrl %[old], %[mem]\n\t" \ "and %[tmp1], %[old], %[mask]\n\t" \ "bne %[tmp1], %[o], 1f\n\t" \ "and %[tmp1], %[old], %[not_mask]\n\t" \ "or %[tmp1], %[tmp1], %[n]\n\t" \ - "sc.w.rl %[tmp1], %[tmp1], %[mem]\n\t" \ + "sc.w.aqrl %[tmp1], %[tmp1], %[mem]\n\t" \ "bnez %[tmp1], 1b\n\t" \ "1:" \ : [old] "=&r" (old), \