From patchwork Sat Nov 7 10:52:31 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noam Camus X-Patchwork-Id: 541258 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2001:1868:205::9]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 5C40C1402B4 for ; Sat, 7 Nov 2015 21:59:40 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1Zv1De-0007ND-VK; Sat, 07 Nov 2015 10:59:38 +0000 Received: from mail-am1on0070.outbound.protection.outlook.com ([157.56.112.70] helo=emea01-am1-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1Zv1Da-0006xd-Dr for linux-snps-arc@lists.infradead.org; Sat, 07 Nov 2015 10:59:36 +0000 Received: from AM3PR02CA0029.eurprd02.prod.outlook.com (10.242.240.29) by AM3PR02MB179.eurprd02.prod.outlook.com (10.242.248.146) with Microsoft SMTP Server (TLS) id 15.1.312.18; Sat, 7 Nov 2015 10:59:11 +0000 Received: from AM1FFO11OLC001.protection.gbl (2a01:111:f400:7e00::124) by AM3PR02CA0029.outlook.office365.com (2a01:111:e400:8800::29) with Microsoft SMTP Server (TLS) id 15.1.318.15 via Frontend Transport; Sat, 7 Nov 2015 10:59:10 +0000 Authentication-Results: spf=fail (sender IP is 212.179.42.66) smtp.mailfrom=ezchip.com; vger.kernel.org; dkim=none (message not signed) header.d=none; vger.kernel.org; dmarc=none action=none header.from=ezchip.com; Received-SPF: Fail (protection.outlook.com: domain of ezchip.com does not designate 212.179.42.66 as permitted sender) receiver=protection.outlook.com; client-ip=212.179.42.66; helo=ezex10.ezchip.com; Received: from ezex10.ezchip.com (212.179.42.66) by AM1FFO11OLC001.mail.protection.outlook.com (10.174.65.76) with Microsoft SMTP Server (TLS) id 15.1.325.5 via Frontend Transport; Sat, 7 Nov 2015 10:59:09 +0000 Received: from localhost.localdomain (10.1.3.132) by ezex10.ezchip.com (10.1.1.4) with Microsoft SMTP Server (TLS) id 14.3.224.2; Sat, 7 Nov 2015 12:58:46 +0200 From: Noam Camus To: Subject: [PATCH v2 13/19] ARC: [plat-eznps] Use dedicated atomic/bitops/cmpxchg Date: Sat, 7 Nov 2015 12:52:31 +0200 Message-ID: <1446893557-29748-14-git-send-email-noamc@ezchip.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1446297327-16298-1-git-send-email-noamc@ezchip.com> References: <1446297327-16298-1-git-send-email-noamc@ezchip.com> MIME-Version: 1.0 X-TM-AS-Product-Ver: SMEX-11.0.0.1191-8.000.1202-21926.006 X-TM-AS-Result: No--6.630000-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1; AM1FFO11OLC001; 1:Ji8REFKhq2puQ0ASFKgKruisXm38nDUYVdCSIYJI+ru6XXPAhVEERpmDuP7783+ncndd8D7pVfOw68gS8RNzDR/Wy3v3iAEjJQrj/2aDreuCXpYGzamm4KWvNjE2jIsglyJFeyajrYOaeDoaAXipqyN3UZXkRcI/J7MbB+krNzbOLf/5jAVaM/4Sy2xrnYBwyqnzdKt931h86rON3toq0uENxwXX6tTcHP3/4CoyL4vli4DZwdjuD96AOtRmMRWhBuAHH+O7RSaydExkKjgdLwad0mL1gQTCkPEyiEkL9DzXhWsV71PdObrP4vahiiZAR2p94quSEfVBHKW2k343y80rd9TJp1CqG0gRLnn2FI90mmtG1B5IUNC0/DM+pli5LdcVSb/MqYZ1YKwq9sClpQ== X-Forefront-Antispam-Report: CIP:212.179.42.66; CTRY:IL; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10009020)(979002)(6009001)(2980300002)(1109001)(1110001)(339900001)(199003)(189002)(92566002)(47776003)(19580395003)(19580405001)(49486002)(50466002)(85426001)(48376002)(87936001)(33646002)(189998001)(110136002)(107886002)(4001430100002)(77096005)(5001920100001)(5007970100001)(2950100001)(5001970100001)(50226001)(36756003)(5003940100001)(6806005)(104016004)(229853001)(76176999)(5008740100001)(106466001)(50986999)(105606002)(86362001)(2351001)(21314002)(969003)(989001)(999001)(1009001)(1019001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM3PR02MB179; H:ezex10.ezchip.com; FPR:; SPF:Fail; PTR:ezmail.ezchip.com; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; AM3PR02MB179; 2:kjxeaZhQT+tqgOvW6IXGdDuIuzLdYbDtOJr2GXIW8IPc/NdyTxS1h6MrK0agKCAsAHfPRlU1SSPUYgtj+NDibxc1ZCkjCK0c1hzQqiy+z2kPzXeQiv+m7EZRD3ix5HH8AQodwVFf+aSzLRO7wM9p1tWfOXiwSi2PoDHLajBV1yQ=; 3:ct1+H1ys7hdkz/1Klldz24oKBqdafZbvB4Ed72TSg5EOGdY+vStpeImkSYrPbmGLb4HoOqhmCvjl3z1CQIdnPvxD4fJYlNjLy9JbUB+YRYej2b1VUinCwezwrHD4lzJeGzEMsfhZ5sSDysv+WBRdohrcD/SNLii9BYPFZLTK+ydkIv8+gqzTY/42t7rBRsAh1X8vI3Zwp3YEydRISbzCZmz1ZS5u13GY+iTKsG5qeu4=; 25:IMLhYKWJTZ0Fza7Y6FfEqkeH1VBhoPMJMClMDWSJAJ+i1I+NjIICV7QReuERPz+LisXzgTlAl8VLpZSiwHJmjTMZTZroPtqdW5fi1Z4HtHBjDz9jKXaQDTr0HfbPCeK5GSYZB/rDHUJCT5qCx4d+Wht4+cgTsDCWlOKOKe+KERqAkJWmYHwJBGP9z7PRF1n0OeSFZj0EFeG92Ok1RB0vFbDVYmVAJ9rBYvQcd6SDH2TpG2XLqDr01YBn7t8IX1WD; 20:AJDn8OyEk6KTEirtuGIGRWUJ0rUsrjaWbdLwT2854BZB4SNrUJZDm4asAcFpwHsetv8VD4P6nuSsycr7T6OvOLGshx6MOZzeyQq7a1JJL/h/HLIqK6LlMF3Fvdl6lg1eEcxMySTJdsoV6HlZsNUgTlavXzKjWMCHJ66nfQqVp/g= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:AM3PR02MB179; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(121898900299872); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(520078)(8121501046)(10201501046)(3002001); SRVR:AM3PR02MB179; BCL:0; PCL:0; RULEID:; SRVR:AM3PR02MB179; X-Microsoft-Exchange-Diagnostics: 1; AM3PR02MB179; 4:wckcBGdT6YXjW4aZArYws48bL3ssRZVVctCHAhAwC+kMK8Pc/Uc+IxbNsRFGHQFGzqAkL5CT2jG9ACImUJLU1bedTPlBtOyeMk04Vg6gYqQE+5GL8nXlx9BLA0f7+vAEleYNQHTBqnc3ungAf67eG7lbYtNjtxW7MWjIkrl06cjKrHG+pFf3Qtdm/56EyMAN0cD5TwTNOetc6eY0ooWmCRDOQJdVZrXpuHjgVPX+we7bJfV6twC0M3+RnAxXsM6RjeXGTh14Jywl4hbIzodNf+mZZvqrmgf3ZveSCfVqEzYvwgZP8B3C4gDZDXbaVhhHXaG5OUbj2o4P8Olp5h396zns2FcA+b/OWn1YK8GjXp3XgHCowhKHX0Ad5qYcZaKN X-Forefront-PRVS: 0753EA505A X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM3PR02MB179; 23:e4gHEEB+yKQV74HbyZvwa9+D4wQs1mK4erpeCjV3G+?= =?us-ascii?Q?ahmffXS9DPMWtPP0hO0trP7C7Cqkq4T70+GHHnyNIcHzcZzAoqoksh/Wpjxp?= =?us-ascii?Q?zdZC0oAuVSq4McW5TmIQzHVtvBVu+bXV3maOiMgNSN+8NlOssa5w/6avMFiH?= =?us-ascii?Q?feI65a2HNRuOh3LzWzRd0H3k2FWpKrIZGlclHrsfDXmI5wQR5vP40uvYMVi+?= =?us-ascii?Q?ykLVMeB7VCLXAkYWEwFSQHL3n2H+UyhpPMR+O4HrcDY3mt4nlZ29RL6p0nrs?= =?us-ascii?Q?rIXj+v/cDHDY6Tf9uEjST6MFO7WDJf1o14hVnzF5InsZRnTeu70ddl4O2+vf?= =?us-ascii?Q?AmDJU9u/zxzB/krV7czMkfa2qqghQwMGDVAdTgo0CZQQfkQzODEe/kgsjNK9?= =?us-ascii?Q?O88ZH794K0ShdhenPAb5kVfsR5/1ReXi9OsZXklK0n5z+3ogVR6EZIx3Byup?= =?us-ascii?Q?afgy1BQj23P4t7SCtLT/FnK8G8DnXlmAEnN0vBe3xmtuCtEupHfN/APmmnOX?= =?us-ascii?Q?mmRaXhjIoKG5R7rLrrGMSihOygyhKDrIxnLUi0cV95lA7/mn19bmpyoDySnH?= =?us-ascii?Q?f5PrPtEshOWhRpi+dfSWbkOuSG1c6RdRc61rHAFPvxiWo0gLdg+w33ZhAFA7?= =?us-ascii?Q?gXMYEfQKuz+wKPtuIYjcQCN8cjo4X1pRoOnGr3IB2T281v+weKsXt7wYwn1/?= =?us-ascii?Q?+U9fpPC9CMhpBhKdi4YwzgwcW3JfPEJJRHj6iSp4cmaFp6+bM0NSk2zLPU7S?= =?us-ascii?Q?1FkLYAxjCTujW7yucbqJuWg/Q2VBIiAJkhixr8o9bBsKcrZHvEe9BwLTuJBq?= =?us-ascii?Q?rhxYeF4G1t5T3KN5Ylm2e94g6pjfBUJw2iPanZVlByuDai0H5/daq7Y7gGVI?= =?us-ascii?Q?FgQ1zN+w5rtefAVQPhDXEospD74+3IAN7p+LVvsLduc646ys2G/+pVcvobbi?= =?us-ascii?Q?MmU8vcDVvnsozk3IZbFtkIOod+WCelgND2nAEtfHsXmdwL6u8PZ5uU1q7qoO?= =?us-ascii?Q?KeCSIzxTq5TqLkngXgFVr2xZmGoF/MUbWN01NUGtBBnKEkMioxQQ9T751GA+?= =?us-ascii?Q?nLmxWXGkpnbqQvHG+ve6A2BWHdjKWjYVJ4Xe298tVEefYrM38L1C4j36CfCR?= =?us-ascii?Q?fKnY7zN9qmTlPDc9nRDsql1mMEqHqfk1sF3WAFh5agXBAV8LweIDtP7XQL8Z?= =?us-ascii?Q?U5AU6NxkNO3/Y=3D?= X-Microsoft-Exchange-Diagnostics: 1; AM3PR02MB179; 5:OodN7HYXGiigBv365C1v1JmvbisZRhiduOJjlzVGiWT+qRIHTH+HYv8ye0s//IlvhzWv+ouxlKx8n6I9AqmWd6ivjkRt/d2k0eVs1cYseCt7cMi7iAQ6jyZjHtrkYBj6sxVnBP23jRYEFuNJRsZgBA==; 24:pnOKm6iC/0yDXvJ7TNMeVZ5HenDk3qPlTZEgWsQVvuSkkrtqI6SuK4xBTp3ezhUKW6mJakIovUfw1XjH6gposPXQS5kKV/GWjdrkLCpZ7XU=; 20:oMR/d9Nsyqi5qseXepfwYVUV/13hjY6edHqohmvZMb08q+bTNhLCBavekkVjCMDyWH0k/WrTR2XVNHupieCV/Q== SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: ezchip.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Nov 2015 10:59:09.9655 (UTC) X-MS-Exchange-CrossTenant-Id: 0fc16e0a-3cd3-4092-8b2f-0a42cff122c3 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=0fc16e0a-3cd3-4092-8b2f-0a42cff122c3; Ip=[212.179.42.66]; Helo=[ezex10.ezchip.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM3PR02MB179 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20151107_025935_103868_D78A23BC X-CRM114-Status: UNSURE ( 9.92 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -1.9 (-) X-Spam-Report: SpamAssassin version 3.4.0 on bombadil.infradead.org summary: Content analysis details: (-1.9 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [157.56.112.70 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [157.56.112.70 listed in wl.mailspike.net] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-BeenThere: linux-snps-arc@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Linux on Synopsys ARC Processors List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Noam Camus , cmetcalf@ezchip.com, gilf@ezchip.com, talz@ezchip.com, linux-kernel@vger.kernel.org Sender: "linux-snps-arc" Errors-To: linux-snps-arc-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org From: Noam Camus We need our own implementaions since we lack LLSC support. Our extended ISA provided with optimized solution for all 32bit operations we see in these three headers. Signed-off-by: Noam Camus --- arch/arc/include/asm/atomic.h | 79 +++++++++++++++++++++++++++++++++++- arch/arc/include/asm/bitops.h | 54 +++++++++++++++++++++++++ arch/arc/include/asm/cmpxchg.h | 87 +++++++++++++++++++++++++++++++++------- 3 files changed, 202 insertions(+), 18 deletions(-) diff --git a/arch/arc/include/asm/atomic.h b/arch/arc/include/asm/atomic.h index c3ecda0..1b2e78a 100644 --- a/arch/arc/include/asm/atomic.h +++ b/arch/arc/include/asm/atomic.h @@ -17,6 +17,7 @@ #include #include +#ifndef CONFIG_ARC_PLAT_EZNPS #define atomic_read(v) ((v)->counter) #ifdef CONFIG_ARC_HAS_LLSC @@ -180,12 +181,84 @@ ATOMIC_OP(andnot, &= ~, bic) ATOMIC_OP(or, |=, or) ATOMIC_OP(xor, ^=, xor) -#undef ATOMIC_OPS -#undef ATOMIC_OP_RETURN -#undef ATOMIC_OP #undef SCOND_FAIL_RETRY_VAR_DEF #undef SCOND_FAIL_RETRY_ASM #undef SCOND_FAIL_RETRY_VARS +#else /* CONFIG_ARC_PLAT_EZNPS */ +static inline int atomic_read(const atomic_t *v) +{ + int temp; + + __asm__ __volatile__( + " ld.di %0, [%1]" + : "=r"(temp) + : "r"(&v->counter) + : "memory"); + return temp; +} + +static inline void atomic_set(atomic_t *v, int i) +{ + __asm__ __volatile__( + " st.di %0,[%1]" + : + : "r"(i), "r"(&v->counter) + : "memory"); +} + +#define ATOMIC_OP(op, c_op, asm_op) \ +static inline void atomic_##op(int i, atomic_t *v) \ +{ \ + __asm__ __volatile__( \ + " mov r2, %0\n" \ + " mov r3, %1\n" \ + " .word %2\n" \ + : \ + : "r"(i), "r"(&v->counter), "i"(asm_op) \ + : "r2", "r3", "memory"); \ +} \ + +#define ATOMIC_OP_RETURN(op, c_op, asm_op) \ +static inline int atomic_##op##_return(int i, atomic_t *v) \ +{ \ + unsigned int temp = i; \ + \ + /* Explicit full memory barrier needed before/after */ \ + smp_mb(); \ + \ + __asm__ __volatile__( \ + " mov r2, %0\n" \ + " mov r3, %1\n" \ + " .word %2\n" \ + " mov %0, r2" \ + : "+r"(temp) \ + : "r"(&v->counter), "i"(asm_op) \ + : "r2", "r3", "memory"); \ + \ + smp_mb(); \ + \ + temp c_op i; \ + \ + return temp; \ +} + +#define ATOMIC_OPS(op, c_op, asm_op) \ + ATOMIC_OP(op, c_op, asm_op) \ + ATOMIC_OP_RETURN(op, c_op, asm_op) + +ATOMIC_OPS(add, +=, CTOP_INST_AADD_DI_R2_R2_R3) +#define atomic_sub(i, v) atomic_add(-(i), (v)) +#define atomic_sub_return(i, v) atomic_add_return(-(i), (v)) + +ATOMIC_OP(and, &=, CTOP_INST_AAND_DI_R2_R2_R3) +#define atomic_andnot(mask, v) atomic_and(~(mask), (v)) +ATOMIC_OP(or, |=, CTOP_INST_AOR_DI_R2_R2_R3) +ATOMIC_OP(xor, ^=, CTOP_INST_AXOR_DI_R2_R2_R3) +#endif /* CONFIG_ARC_PLAT_EZNPS */ + +#undef ATOMIC_OPS +#undef ATOMIC_OP_RETURN +#undef ATOMIC_OP /** * __atomic_add_unless - add unless the number is a given value diff --git a/arch/arc/include/asm/bitops.h b/arch/arc/include/asm/bitops.h index 57c1f33..5a29185 100644 --- a/arch/arc/include/asm/bitops.h +++ b/arch/arc/include/asm/bitops.h @@ -22,6 +22,7 @@ #include #endif +#ifndef CONFIG_ARC_PLAT_EZNPS #if defined(CONFIG_ARC_HAS_LLSC) /* @@ -155,6 +156,53 @@ static inline int test_and_##op##_bit(unsigned long nr, volatile unsigned long * } #endif /* CONFIG_ARC_HAS_LLSC */ +#else /* CONFIG_ARC_PLAT_EZNPS */ +#define BIT_OP(op, c_op, asm_op) \ +static inline void op##_bit(unsigned long nr, volatile unsigned long *m)\ +{ \ + m += nr >> 5; \ + \ + nr = (1UL << (nr & 0x1f)); \ + if (asm_op == CTOP_INST_AAND_DI_R2_R2_R3) \ + nr = ~nr; \ + \ + __asm__ __volatile__( \ + " mov r2, %0\n" \ + " mov r3, %1\n" \ + " .word %2\n" \ + : \ + : "r"(nr), "r"(m), "i"(asm_op) \ + : "r2", "r3", "memory"); \ +} + +#define TEST_N_BIT_OP(op, c_op, asm_op) \ +static inline int test_and_##op##_bit(unsigned long nr, volatile unsigned long *m)\ +{ \ + unsigned long old; \ + \ + m += nr >> 5; \ + \ + nr = old = (1UL << (nr & 0x1f)); \ + if (asm_op == CTOP_INST_AAND_DI_R2_R2_R3) \ + old = ~old; \ + \ + /* Explicit full memory barrier needed before/after */ \ + smp_mb(); \ + \ + __asm__ __volatile__( \ + " mov r2, %0\n" \ + " mov r3, %1\n" \ + " .word %2\n" \ + " mov %0, r2" \ + : "+r"(old) \ + : "r"(m), "i"(asm_op) \ + : "r2", "r3", "memory"); \ + \ + smp_mb(); \ + \ + return (old & nr) != 0; \ +} +#endif /* CONFIG_ARC_PLAT_EZNPS */ /*************************************** * Non atomic variants @@ -196,9 +244,15 @@ static inline int __test_and_##op##_bit(unsigned long nr, volatile unsigned long /* __test_and_set_bit(), __test_and_clear_bit(), __test_and_change_bit() */\ __TEST_N_BIT_OP(op, c_op, asm_op) +#ifndef CONFIG_ARC_PLAT_EZNPS BIT_OPS(set, |, bset) BIT_OPS(clear, & ~, bclr) BIT_OPS(change, ^, bxor) +#else +BIT_OPS(set, |, CTOP_INST_AOR_DI_R2_R2_R3) +BIT_OPS(clear, & ~, CTOP_INST_AAND_DI_R2_R2_R3) +BIT_OPS(change, ^, CTOP_INST_AXOR_DI_R2_R2_R3) +#endif /* * This routine doesn't need to be atomic. diff --git a/arch/arc/include/asm/cmpxchg.h b/arch/arc/include/asm/cmpxchg.h index af7a2db..6d320d3 100644 --- a/arch/arc/include/asm/cmpxchg.h +++ b/arch/arc/include/asm/cmpxchg.h @@ -14,6 +14,7 @@ #include #include +#ifndef CONFIG_ARC_PLAT_EZNPS #ifdef CONFIG_ARC_HAS_LLSC static inline unsigned long @@ -66,21 +67,6 @@ __cmpxchg(volatile void *ptr, unsigned long expected, unsigned long new) #endif /* CONFIG_ARC_HAS_LLSC */ -#define cmpxchg(ptr, o, n) ((typeof(*(ptr)))__cmpxchg((ptr), \ - (unsigned long)(o), (unsigned long)(n))) - -/* - * Since not supported natively, ARC cmpxchg() uses atomic_ops_lock (UP/SMP) - * just to gaurantee semantics. - * atomic_cmpxchg() needs to use the same locks as it's other atomic siblings - * which also happens to be atomic_ops_lock. - * - * Thus despite semantically being different, implementation of atomic_cmpxchg() - * is same as cmpxchg(). - */ -#define atomic_cmpxchg(v, o, n) ((int)cmpxchg(&((v)->counter), (o), (n))) - - /* * xchg (reg with memory) based on "Native atomic" EX insn */ @@ -143,6 +129,63 @@ static inline unsigned long __xchg(unsigned long val, volatile void *ptr, #endif +#else /* CONFIG_ARC_PLAT_EZNPS */ +static inline unsigned long +__cmpxchg(volatile void *ptr, unsigned long expected, unsigned long new) +{ + /* + * Explicit full memory barrier needed before/after + */ + smp_mb(); + + write_aux_reg(CTOP_AUX_GPA1, expected); + + __asm__ __volatile__( + " mov r2, %0\n" + " mov r3, %1\n" + " .word %2\n" + " mov %0, r2" + : "+r"(new) + : "r"(ptr), "i"(CTOP_INST_EXC_DI_R2_R2_R3) + : "r2", "r3", "memory"); + + smp_mb(); + + return new; +} + +static inline unsigned long __xchg(unsigned long val, volatile void *ptr, + int size) +{ + extern unsigned long __xchg_bad_pointer(void); + + switch (size) { + case 4: + /* + * Explicit full memory barrier needed before/after + */ + smp_mb(); + + __asm__ __volatile__( + " mov r2, %0\n" + " mov r3, %1\n" + " .word %2\n" + " mov %0, r2\n" + : "+r"(val) + : "r"(ptr), "i"(CTOP_INST_XEX_DI_R2_R2_R3) + : "r2", "r3", "memory"); + + smp_mb(); + + return val; + } + return __xchg_bad_pointer(); +} + +#define xchg(ptr, with) ((typeof(*(ptr)))__xchg((unsigned long)(with), (ptr), \ + sizeof(*(ptr)))) +#endif /* CONFIG_ARC_PLAT_EZNPS */ + /* * "atomic" variant of xchg() * REQ: It needs to follow the same serialization rules as other atomic_xxx() @@ -158,4 +201,18 @@ static inline unsigned long __xchg(unsigned long val, volatile void *ptr, */ #define atomic_xchg(v, new) (xchg(&((v)->counter), new)) +#define cmpxchg(ptr, o, n) ((typeof(*(ptr)))__cmpxchg((ptr), \ + (unsigned long)(o), (unsigned long)(n))) + +/* + * Since not supported natively, ARC cmpxchg() uses atomic_ops_lock (UP/SMP) + * just to gaurantee semantics. + * atomic_cmpxchg() needs to use the same locks as it's other atomic siblings + * which also happens to be atomic_ops_lock. + * + * Thus despite semantically being different, implementation of atomic_cmpxchg() + * is same as cmpxchg(). + */ +#define atomic_cmpxchg(v, o, n) ((int)cmpxchg(&((v)->counter), (o), (n))) + #endif