[1/2] Optimize generic spinlock code and use C11 like atomic macros.

Submitted by Stefan Liebler on April 7, 2017, 4:22 p.m.

Details

Message ID 09ae8ea7-4119-76c1-cd58-36cbdf587390@linux.vnet.ibm.com
State New
Headers show

Commit Message

Stefan Liebler April 7, 2017, 4:22 p.m.
On 04/06/2017 04:00 PM, Torvald Riegel wrote:
> Otherwise, this patch looks good to me.
Thanks for review. Here is an updated patch. I've changed the comments 
and added the POSIX requirement in pthread_spin_trylock.

@architecture maintainers:
I've added defines of ATOMIC_EXCHANGE_USES_CAS in the architecture 
specific atomic-machine.h files.
See comment in include/atomic.h:
/* ATOMIC_EXCHANGE_USES_CAS is equal to 1 if atomic_exchange operations
    are implemented based on a CAS loop; otherwise, this is 0 and we assume
    that the atomic_exchange operations could provide better performance
    than a CAS loop.  */

Can review the definition to 0 or 1 in the atomic-machine.h file of your 
architecture, please?

ChangeLog:

	* NEWS: Mention new spinlock implementation.
	* include/atomic.h:
	(__atomic_val_bysize): Cast type to omit volatile qualifier.
	(atomic_exchange_acq): Likewise.
	(atomic_load_relaxed): Likewise.
	(ATOMIC_EXCHANGE_USES_CAS): Check definition.
	* nptl/pthread_spin_init.c (pthread_spin_init):
	Use atomic_store_relaxed.
	* nptl/pthread_spin_lock.c (pthread_spin_lock):
	Use C11-like atomic macros.
	* nptl/pthread_spin_trylock.c (pthread_spin_trylock):
	Likewise.
	* nptl/pthread_spin_unlock.c (pthread_spin_unlock):
	Use atomic_store_release.
	* sysdeps/aarch64/nptl/pthread_spin_lock.c: Delete File.
	* sysdeps/arm/nptl/pthread_spin_lock.c: Likewise.
	* sysdeps/hppa/nptl/pthread_spin_lock.c: Likewise.
	* sysdeps/m68k/nptl/pthread_spin_lock.c: Likewise.
	* sysdeps/microblaze/nptl/pthread_spin_lock.c: Likewise.
	* sysdeps/mips/nptl/pthread_spin_lock.c: Likewise.
	* sysdeps/nios2/nptl/pthread_spin_lock.c: Likewise.
	* sysdeps/aarch64/atomic-machine.h (ATOMIC_EXCHANGE_USES_CAS):
  	Define.
	* sysdeps/alpha/atomic-machine.h: Likewise.
	* sysdeps/arm/atomic-machine.h: Likewise.
	* sysdeps/i386/atomic-machine.h: Likewise.
	* sysdeps/ia64/atomic-machine.h: Likewise.
	* sysdeps/m68k/coldfire/atomic-machine.h: Likewise.
	* sysdeps/m68k/m680x0/m68020/atomic-machine.h: Likewise.
	* sysdeps/microblaze/atomic-machine.h: Likewise.
	* sysdeps/mips/atomic-machine.h: Likewise.
	* sysdeps/powerpc/powerpc32/atomic-machine.h: Likewise.
	* sysdeps/powerpc/powerpc64/atomic-machine.h: Likewise.
	* sysdeps/s390/atomic-machine.h: Likewise.
	* sysdeps/sparc/sparc32/atomic-machine.h: Likewise.
	* sysdeps/sparc/sparc32/sparcv9/atomic-machine.h: Likewise.
	* sysdeps/sparc/sparc64/atomic-machine.h: Likewise.
	* sysdeps/tile/tilegx/atomic-machine.h: Likewise.
	* sysdeps/tile/tilepro/atomic-machine.h: Likewise.
	* sysdeps/unix/sysv/linux/hppa/atomic-machine.h: Likewise.
	* sysdeps/unix/sysv/linux/m68k/coldfire/atomic-machine.h:
	Likewise.
	* sysdeps/unix/sysv/linux/nios2/atomic-machine.h: Likewise.
	* sysdeps/unix/sysv/linux/sh/atomic-machine.h: Likewise.
	* sysdeps/x86_64/atomic-machine.h: Likewise.


Bye
Stefan

Comments

Torvald Riegel April 9, 2017, 1:51 p.m.
On Fri, 2017-04-07 at 18:22 +0200, Stefan Liebler wrote:
> On 04/06/2017 04:00 PM, Torvald Riegel wrote:
> > Otherwise, this patch looks good to me.
> Thanks for review. Here is an updated patch. I've changed the comments 
> and added the POSIX requirement in pthread_spin_trylock.

I assume that this means that you have taken care of all the review
comments I have and changed nothing else; thus, I won't review this in
detail again.  (If there have been other changes, please let me know so
that I can look at them specifically.)

> @architecture maintainers:
> I've added defines of ATOMIC_EXCHANGE_USES_CAS in the architecture 
> specific atomic-machine.h files.
> See comment in include/atomic.h:
> /* ATOMIC_EXCHANGE_USES_CAS is equal to 1 if atomic_exchange operations
>     are implemented based on a CAS loop; otherwise, this is 0 and we assume
>     that the atomic_exchange operations could provide better performance
>     than a CAS loop.  */
> 
> Can review the definition to 0 or 1 in the atomic-machine.h file of your 
> architecture, please?

I think we primarily need feedback from the maintainers of the
architectures where we're not quite sure.  Let's see how responsive
everyone is and then decide whether we need to continue pinging some of
them potentially or whether we have enough information to be able to
commit the patch.
Andreas Schwab April 10, 2017, 8:17 a.m.
On Apr 07 2017, Stefan Liebler <stli@linux.vnet.ibm.com> wrote:

> /* ATOMIC_EXCHANGE_USES_CAS is equal to 1 if atomic_exchange operations

Why 1 exactly, and not just non-zero?

Andreas.
Stefan Liebler April 10, 2017, 11:59 a.m.
On 04/09/2017 03:51 PM, Torvald Riegel wrote:
> On Fri, 2017-04-07 at 18:22 +0200, Stefan Liebler wrote:
>> On 04/06/2017 04:00 PM, Torvald Riegel wrote:
>>> Otherwise, this patch looks good to me.
>> Thanks for review. Here is an updated patch. I've changed the comments
>> and added the POSIX requirement in pthread_spin_trylock.
>
> I assume that this means that you have taken care of all the review
> comments I have and changed nothing else; thus, I won't review this in
> detail again.  (If there have been other changes, please let me know so
> that I can look at them specifically.)
>
Yes, I have taken care of all the review comments and changed nothing else.

>> @architecture maintainers:
>> I've added defines of ATOMIC_EXCHANGE_USES_CAS in the architecture
>> specific atomic-machine.h files.
>> See comment in include/atomic.h:
>> /* ATOMIC_EXCHANGE_USES_CAS is equal to 1 if atomic_exchange operations
>>     are implemented based on a CAS loop; otherwise, this is 0 and we assume
>>     that the atomic_exchange operations could provide better performance
>>     than a CAS loop.  */
>>
>> Can review the definition to 0 or 1 in the atomic-machine.h file of your
>> architecture, please?
>
> I think we primarily need feedback from the maintainers of the
> architectures where we're not quite sure.  Let's see how responsive
> everyone is and then decide whether we need to continue pinging some of
> them potentially or whether we have enough information to be able to
> commit the patch.
>
Okay.

Chris has already replied directly to me.
On 04/07/2017 06:50 PM, Chris Metcalf wrote:
 > Thanks, looks good for tilegx/tilepro.

Bye
Stefan
Stefan Liebler April 10, 2017, 11:59 a.m.
On 04/10/2017 10:17 AM, Andreas Schwab wrote:
> On Apr 07 2017, Stefan Liebler <stli@linux.vnet.ibm.com> wrote:
>
>> /* ATOMIC_EXCHANGE_USES_CAS is equal to 1 if atomic_exchange operations
>
> Why 1 exactly, and not just non-zero?
>
> Andreas.
>

This points out that there is only a true/false choice and
all architectures have to define it.

__HAVE_64B_ATOMICS is used in the same way.
/* This is equal to 1 iff the architecture supports 64b atomic 
operations.  */

Bye
Stefan
Andreas Schwab April 10, 2017, 1:36 p.m.
On Apr 10 2017, Stefan Liebler <stli@linux.vnet.ibm.com> wrote:

> On 04/10/2017 10:17 AM, Andreas Schwab wrote:
>> On Apr 07 2017, Stefan Liebler <stli@linux.vnet.ibm.com> wrote:
>>
>>> /* ATOMIC_EXCHANGE_USES_CAS is equal to 1 if atomic_exchange operations
>>
>> Why 1 exactly, and not just non-zero?
>>
>> Andreas.
>>
>
> This points out that there is only a true/false choice and
> all architectures have to define it.

That doesn't answer my question.  Why do you need the dance of checking
for 1 explicitly?

Andreas.
Stefan Liebler April 11, 2017, 7:06 a.m.
On 04/10/2017 03:36 PM, Andreas Schwab wrote:
> On Apr 10 2017, Stefan Liebler <stli@linux.vnet.ibm.com> wrote:
>
>> On 04/10/2017 10:17 AM, Andreas Schwab wrote:
>>> On Apr 07 2017, Stefan Liebler <stli@linux.vnet.ibm.com> wrote:
>>>
>>>> /* ATOMIC_EXCHANGE_USES_CAS is equal to 1 if atomic_exchange operations
>>>
>>> Why 1 exactly, and not just non-zero?
>>>
>>> Andreas.
>>>
>>
>> This points out that there is only a true/false choice and
>> all architectures have to define it.
>
> That doesn't answer my question.  Why do you need the dance of checking
> for 1 explicitly?
>
> Andreas.
>
At the moment there is only a choice between 0 and 1.
The check ensures that ATOMIC_EXCHANGE_USES_CAS is defined and it has a 
specific value.
If someone defines it to another value, he will get an error here.
Then the check and the comment is hopefully updated, too.

__HAVE_64B_ATOMICS has to be 0 or 1, but it is not checked explicitly.
Here are some users of it:
./nptl/sem_waitcommon.c:81:#if !__HAVE_64B_ATOMICS
./nptl/sem_waitcommon.c:91:#if __HAVE_64B_ATOMICS
./nptl/pthread_cond_common.c:26:#if __HAVE_64B_ATOMICS == 1
./nptl/pthread_cond_common.c:59:#else
./nptl/pthread_cond_common.c:245:#endif  /* !__HAVE_64B_ATOMICS  */
...
If someone defines __HAVE_64B_ATOMICS to 2, build/test will fail with 
some unspecified error.
If someone defines ATOMIC_EXCHANGE_USES_CAS to 2, build fails with
"ATOMIC_EXCHANGE_USES_CAS has to be defined to 0 or 1".

But you are right, zero or non-zero is fine, too.
My latest patch is only using:
#if ! ATOMIC_EXCHANGE_USES_CAS
#else
#endif

But perhaps #if ATOMIC_EXCHANGE_USES_CAS == 1 will be introduced by a 
future patch.

Bye
Stefan
Andreas Schwab April 11, 2017, 8:45 a.m.
On Apr 11 2017, Stefan Liebler <stli@linux.vnet.ibm.com> wrote:

> But perhaps #if ATOMIC_EXCHANGE_USES_CAS == 1 will be introduced by a
> future patch.

Why would anyone want to do that?

Andreas.
Stefan Liebler April 11, 2017, 10:15 a.m.
On 04/11/2017 10:45 AM, Andreas Schwab wrote:
> On Apr 11 2017, Stefan Liebler <stli@linux.vnet.ibm.com> wrote:
>
>> But perhaps #if ATOMIC_EXCHANGE_USES_CAS == 1 will be introduced by a
>> future patch.
>
> Why would anyone want to do that?
>
> Andreas.
>
By accident!?
It happened in the case of __HAVE_64B_ATOMICS.

Bye
Stefan
Andreas Schwab April 11, 2017, 12:05 p.m.
On Apr 11 2017, Stefan Liebler <stli@linux.vnet.ibm.com> wrote:

> It happened in the case of __HAVE_64B_ATOMICS.

We shouldn't follow a bad example.

Andreas.
Stefan Liebler April 11, 2017, 12:19 p.m.
On 04/11/2017 02:05 PM, Andreas Schwab wrote:
> On Apr 11 2017, Stefan Liebler <stli@linux.vnet.ibm.com> wrote:
>
>> It happened in the case of __HAVE_64B_ATOMICS.
>
> We shouldn't follow a bad example.
>
> Andreas.
>

Then I'll change the check to:
/* ATOMIC_EXCHANGE_USES_CAS is non-zero if atomic_exchange operations
    are implemented based on a CAS loop; otherwise, this is zero and we 
assume
    that the atomic_exchange operations could provide better performance
    than a CAS loop.  */
#ifndef ATOMIC_EXCHANGE_USES_CAS
# error ATOMIC_EXCHANGE_USES_CAS has to be defined.
#endif

Bye
Stefan
Zack Weinberg April 11, 2017, 1:08 p.m.
On Tue, Apr 11, 2017 at 8:05 AM, Andreas Schwab <schwab@suse.de> wrote:
> On Apr 11 2017, Stefan Liebler <stli@linux.vnet.ibm.com> wrote:
>> It happened in the case of __HAVE_64B_ATOMICS.
>
> We shouldn't follow a bad example.

How about you propose a patch to fix __HAVE_64B_ATOMICS the way you
think it should be, Andreas, since you're making such a fuss about
this?

zw
Torvald Riegel April 13, 2017, 4:36 p.m.
On Tue, 2017-04-11 at 09:06 +0200, Stefan Liebler wrote:
> On 04/10/2017 03:36 PM, Andreas Schwab wrote:
> > On Apr 10 2017, Stefan Liebler <stli@linux.vnet.ibm.com> wrote:
> >
> >> On 04/10/2017 10:17 AM, Andreas Schwab wrote:
> >>> On Apr 07 2017, Stefan Liebler <stli@linux.vnet.ibm.com> wrote:
> >>>
> >>>> /* ATOMIC_EXCHANGE_USES_CAS is equal to 1 if atomic_exchange operations
> >>>
> >>> Why 1 exactly, and not just non-zero?
> >>>
> >>> Andreas.
> >>>
> >>
> >> This points out that there is only a true/false choice and
> >> all architectures have to define it.
> >
> > That doesn't answer my question.  Why do you need the dance of checking
> > for 1 explicitly?
> >
> > Andreas.
> >
> At the moment there is only a choice between 0 and 1.

This is fine.  There is no reason to allow a different value even if
this is just a boolean.

> The check ensures that ATOMIC_EXCHANGE_USES_CAS is defined and it has a 
> specific value.
> If someone defines it to another value, he will get an error here.
> Then the check and the comment is hopefully updated, too.

The comments make it clear that it has to be 0 or 1, and can be used
without explicitly checking the value.

We can add the explicit check once, as you did in atomic.h, but I don't
think it's neither required nor a bad thing to do.
Stefan Liebler April 18, 2017, 1:09 p.m.
On 04/10/2017 01:59 PM, Stefan Liebler wrote:
> On 04/09/2017 03:51 PM, Torvald Riegel wrote:
>> On Fri, 2017-04-07 at 18:22 +0200, Stefan Liebler wrote:
>>> @architecture maintainers:
>>> I've added defines of ATOMIC_EXCHANGE_USES_CAS in the architecture
>>> specific atomic-machine.h files.
>>> See comment in include/atomic.h:
>>> /* ATOMIC_EXCHANGE_USES_CAS is equal to 1 if atomic_exchange operations
>>>     are implemented based on a CAS loop; otherwise, this is 0 and we
>>> assume
>>>     that the atomic_exchange operations could provide better performance
>>>     than a CAS loop.  */
>>>
>>> Can review the definition to 0 or 1 in the atomic-machine.h file of your
>>> architecture, please?


PING


>>
>> I think we primarily need feedback from the maintainers of the
>> architectures where we're not quite sure.  Let's see how responsive
>> everyone is and then decide whether we need to continue pinging some of
>> them potentially or whether we have enough information to be able to
>> commit the patch.
>>
> Okay.
>
> Chris has already replied directly to me.
> On 04/07/2017 06:50 PM, Chris Metcalf wrote:
>> Thanks, looks good for tilegx/tilepro.
>
> Bye
> Stefan
>
Stefan Liebler April 25, 2017, 6:46 a.m.
On 04/18/2017 03:09 PM, Stefan Liebler wrote:
> On 04/10/2017 01:59 PM, Stefan Liebler wrote:
>> On 04/09/2017 03:51 PM, Torvald Riegel wrote:
>>> On Fri, 2017-04-07 at 18:22 +0200, Stefan Liebler wrote:
>>>> @architecture maintainers:
>>>> I've added defines of ATOMIC_EXCHANGE_USES_CAS in the architecture
>>>> specific atomic-machine.h files.
>>>> See comment in include/atomic.h:
>>>> /* ATOMIC_EXCHANGE_USES_CAS is equal to 1 if atomic_exchange operations
>>>>     are implemented based on a CAS loop; otherwise, this is 0 and we
>>>> assume
>>>>     that the atomic_exchange operations could provide better
>>>> performance
>>>>     than a CAS loop.  */
>>>>
>>>> Can review the definition to 0 or 1 in the atomic-machine.h file of
>>>> your
>>>> architecture, please?
>
>
> PING
>
PING
>
>>>
>>> I think we primarily need feedback from the maintainers of the
>>> architectures where we're not quite sure.  Let's see how responsive
>>> everyone is and then decide whether we need to continue pinging some of
>>> them potentially or whether we have enough information to be able to
>>> commit the patch.
>>>
>> Okay.
>>
>> Chris has already replied directly to me.
>> On 04/07/2017 06:50 PM, Chris Metcalf wrote:
>>> Thanks, looks good for tilegx/tilepro.
>>
Andreas has answered regarding the explicit check of 
ATOMIC_EXCHANGE_USES_CAS in include/atomic.h.
@Andreas: is m68k okay?

Joseph has replied for mips.

What about the other architectures? Is the definition of 
ATOMIC_EXCHANGE_USES_CAS correct?

>> Bye
>> Stefan
>>
>

Patch hide | download patch | download mbox

From db4c69c6d20bfe4cc3fedcbd5571736dc50c290d Mon Sep 17 00:00:00 2001
From: Stefan Liebler <stli@linux.vnet.ibm.com>
Date: Fri, 7 Apr 2017 18:15:09 +0200
Subject: [PATCH 1/2] Optimize generic spinlock code and use C11 like atomic
 macros.

This patch optimizes the generic spinlock code.

The type pthread_spinlock_t is a typedef to volatile int on all archs.
Passing a volatile pointer to the atomic macros which are not mapped to the
C11 atomic builtins can lead to extra stores and loads to stack if such
a macro creates a temporary variable by using "__typeof (*(mem)) tmp;".
Thus, those macros which are used by spinlock code - atomic_exchange_acquire,
atomic_load_relaxed, atomic_compare_exchange_weak - have to be adjusted.
According to the comment from  Szabolcs Nagy, the type of a cast expression is
unqualified (see http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_423.htm):
__typeof ((__typeof (*(mem)) *(mem)) tmp;
Thus from spinlock perspective the variable tmp is of type int instead of
type volatile int.  This patch adjusts those macros in include/atomic.h.
With this construct GCC >= 5 omits the extra stores and loads.

The atomic macros are replaced by the C11 like atomic macros and thus
the code is aligned to it.  The pthread_spin_unlock implementation is now
using release memory order instead of sequentially consistent memory order.
The issue with passed volatile int pointers applies to the C11 like atomic
macros as well as the ones used before.

I've added a glibc_likely hint to the first atomic exchange in
pthread_spin_lock in order to return immediately to the caller if the lock is
free.  Without the hint, there is an additional jump if the lock is free.

I've added the atomic_spin_nop macro within the loop of plain reads.
The plain reads are also realized by C11 like atomic_load_relaxed macro.

The new define ATOMIC_EXCHANGE_USES_CAS determines if the first try to acquire
the spinlock in pthread_spin_lock or pthread_spin_trylock is an exchange
or a CAS.  This is defined in atomic-machine.h for all architectures.

The define SPIN_LOCK_READS_BETWEEN_CMPXCHG is now removed.
There is no technical reason for throwing in a CAS every now and then,
and so far we have no evidence that it can improve performance.
If that would be the case, we have to adjust other spin-waiting loops
elsewhere, too!  Using a CAS loop without plain reads is not a good idea
on many targets and wasn't used by one.  Thus there is now no option to
do so.

Architectures are now using the generic spinlock automatically if they
do not provide an own implementation.  Thus the pthread_spin_lock.c files
in sysdeps folder are deleted.

ChangeLog:

	* NEWS: Mention new spinlock implementation.
	* include/atomic.h:
	(__atomic_val_bysize): Cast type to omit volatile qualifier.
	(atomic_exchange_acq): Likewise.
	(atomic_load_relaxed): Likewise.
	(ATOMIC_EXCHANGE_USES_CAS): Check definition.
	* nptl/pthread_spin_init.c (pthread_spin_init):
	Use atomic_store_relaxed.
	* nptl/pthread_spin_lock.c (pthread_spin_lock):
	Use C11-like atomic macros.
	* nptl/pthread_spin_trylock.c (pthread_spin_trylock):
	Likewise.
	* nptl/pthread_spin_unlock.c (pthread_spin_unlock):
	Use atomic_store_release.
	* sysdeps/aarch64/nptl/pthread_spin_lock.c: Delete File.
	* sysdeps/arm/nptl/pthread_spin_lock.c: Likewise.
	* sysdeps/hppa/nptl/pthread_spin_lock.c: Likewise.
	* sysdeps/m68k/nptl/pthread_spin_lock.c: Likewise.
	* sysdeps/microblaze/nptl/pthread_spin_lock.c: Likewise.
	* sysdeps/mips/nptl/pthread_spin_lock.c: Likewise.
	* sysdeps/nios2/nptl/pthread_spin_lock.c: Likewise.
	* sysdeps/aarch64/atomic-machine.h (ATOMIC_EXCHANGE_USES_CAS): Define.
	* sysdeps/alpha/atomic-machine.h: Likewise.
	* sysdeps/arm/atomic-machine.h: Likewise.
	* sysdeps/i386/atomic-machine.h: Likewise.
	* sysdeps/ia64/atomic-machine.h: Likewise.
	* sysdeps/m68k/coldfire/atomic-machine.h: Likewise.
	* sysdeps/m68k/m680x0/m68020/atomic-machine.h: Likewise.
	* sysdeps/microblaze/atomic-machine.h: Likewise.
	* sysdeps/mips/atomic-machine.h: Likewise.
	* sysdeps/powerpc/powerpc32/atomic-machine.h: Likewise.
	* sysdeps/powerpc/powerpc64/atomic-machine.h: Likewise.
	* sysdeps/s390/atomic-machine.h: Likewise.
	* sysdeps/sparc/sparc32/atomic-machine.h: Likewise.
	* sysdeps/sparc/sparc32/sparcv9/atomic-machine.h: Likewise.
	* sysdeps/sparc/sparc64/atomic-machine.h: Likewise.
	* sysdeps/tile/tilegx/atomic-machine.h: Likewise.
	* sysdeps/tile/tilepro/atomic-machine.h: Likewise.
	* sysdeps/unix/sysv/linux/hppa/atomic-machine.h: Likewise.
	* sysdeps/unix/sysv/linux/m68k/coldfire/atomic-machine.h: Likewise.
	* sysdeps/unix/sysv/linux/nios2/atomic-machine.h: Likewise.
	* sysdeps/unix/sysv/linux/sh/atomic-machine.h: Likewise.
	* sysdeps/x86_64/atomic-machine.h: Likewise.
---
 NEWS                                               | 11 ++++
 include/atomic.h                                   | 17 ++++--
 nptl/pthread_spin_init.c                           |  3 +-
 nptl/pthread_spin_lock.c                           | 71 +++++++++++++---------
 nptl/pthread_spin_trylock.c                        | 54 +++++++++++++++-
 nptl/pthread_spin_unlock.c                         |  6 +-
 sysdeps/aarch64/atomic-machine.h                   |  1 +
 sysdeps/aarch64/nptl/pthread_spin_lock.c           | 24 --------
 sysdeps/alpha/atomic-machine.h                     |  1 +
 sysdeps/arm/atomic-machine.h                       |  1 +
 sysdeps/arm/nptl/pthread_spin_lock.c               | 23 -------
 sysdeps/hppa/nptl/pthread_spin_lock.c              | 23 -------
 sysdeps/i386/atomic-machine.h                      |  1 +
 sysdeps/ia64/atomic-machine.h                      |  1 +
 sysdeps/m68k/coldfire/atomic-machine.h             |  1 +
 sysdeps/m68k/m680x0/m68020/atomic-machine.h        |  1 +
 sysdeps/m68k/nptl/pthread_spin_lock.c              | 24 --------
 sysdeps/microblaze/atomic-machine.h                |  1 +
 sysdeps/microblaze/nptl/pthread_spin_lock.c        | 24 --------
 sysdeps/mips/atomic-machine.h                      |  2 +
 sysdeps/mips/nptl/pthread_spin_lock.c              | 23 -------
 sysdeps/nios2/nptl/pthread_spin_lock.c             | 24 --------
 sysdeps/powerpc/powerpc32/atomic-machine.h         |  1 +
 sysdeps/powerpc/powerpc64/atomic-machine.h         |  1 +
 sysdeps/s390/atomic-machine.h                      |  2 +
 sysdeps/sparc/sparc32/atomic-machine.h             |  1 +
 sysdeps/sparc/sparc32/sparcv9/atomic-machine.h     |  1 +
 sysdeps/sparc/sparc64/atomic-machine.h             |  1 +
 sysdeps/tile/tilegx/atomic-machine.h               |  1 +
 sysdeps/tile/tilepro/atomic-machine.h              |  1 +
 sysdeps/unix/sysv/linux/hppa/atomic-machine.h      |  1 +
 .../unix/sysv/linux/m68k/coldfire/atomic-machine.h |  1 +
 sysdeps/unix/sysv/linux/nios2/atomic-machine.h     |  1 +
 sysdeps/unix/sysv/linux/sh/atomic-machine.h        |  1 +
 sysdeps/x86_64/atomic-machine.h                    |  1 +
 35 files changed, 148 insertions(+), 203 deletions(-)
 delete mode 100644 sysdeps/aarch64/nptl/pthread_spin_lock.c
 delete mode 100644 sysdeps/arm/nptl/pthread_spin_lock.c
 delete mode 100644 sysdeps/hppa/nptl/pthread_spin_lock.c
 delete mode 100644 sysdeps/m68k/nptl/pthread_spin_lock.c
 delete mode 100644 sysdeps/microblaze/nptl/pthread_spin_lock.c
 delete mode 100644 sysdeps/mips/nptl/pthread_spin_lock.c
 delete mode 100644 sysdeps/nios2/nptl/pthread_spin_lock.c

diff --git a/NEWS b/NEWS
index 3590878..e6702ad 100644
--- a/NEWS
+++ b/NEWS
@@ -41,6 +41,17 @@  Version 2.26
   "The Rules of Hungarian Orthography, 12th edition" and the work of
   Egmont Koblinger (Bug 18934).
 
+* The synchronization that pthread_spin_unlock performs has been changed
+  to now be equivalent to a C11 atomic store with release memory order to
+  the spin lock's memory location.  This ensures correct synchronization
+  for the spin lock's operations and critical sections protected by a spin
+  lock.  Previously, several (but not all) architectures used stronger
+  synchronization (e.g., containing what is often called a full barrier).
+  This change can improve performance, but may affect odd fringe uses of
+  spin locks that depend on the previous behavior (e.g., using spin locks
+  as atomic variables to try to implement Dekker's mutual exclusion
+  algorithm).
+
 Security related changes:
 
   [Add security related changes here]
diff --git a/include/atomic.h b/include/atomic.h
index 7f32640..0d1e42e 100644
--- a/include/atomic.h
+++ b/include/atomic.h
@@ -54,7 +54,7 @@ 
    and following args.  */
 #define __atomic_val_bysize(pre, post, mem, ...)			      \
   ({									      \
-    __typeof (*mem) __atg1_result;					      \
+    __typeof ((__typeof (*(mem))) *(mem)) __atg1_result;		      \
     if (sizeof (*mem) == 1)						      \
       __atg1_result = pre##_8_##post (mem, __VA_ARGS__);		      \
     else if (sizeof (*mem) == 2)					      \
@@ -162,9 +162,9 @@ 
 /* Store NEWVALUE in *MEM and return the old value.  */
 #ifndef atomic_exchange_acq
 # define atomic_exchange_acq(mem, newvalue) \
-  ({ __typeof (*(mem)) __atg5_oldval;					      \
+  ({ __typeof ((__typeof (*(mem))) *(mem)) __atg5_oldval;		      \
      __typeof (mem) __atg5_memp = (mem);				      \
-     __typeof (*(mem)) __atg5_value = (newvalue);			      \
+     __typeof ((__typeof (*(mem))) *(mem)) __atg5_value = (newvalue);	      \
 									      \
      do									      \
        __atg5_oldval = *__atg5_memp;					      \
@@ -668,7 +668,7 @@  void __atomic_link_error (void);
 
 # ifndef atomic_load_relaxed
 #  define atomic_load_relaxed(mem) \
-   ({ __typeof (*(mem)) __atg100_val;					      \
+   ({ __typeof ((__typeof (*(mem))) *(mem)) __atg100_val;		      \
    __asm ("" : "=r" (__atg100_val) : "0" (*(mem)));			      \
    __atg100_val; })
 # endif
@@ -818,4 +818,13 @@  void __atomic_link_error (void);
 # define atomic_spin_nop() do { /* nothing */ } while (0)
 #endif
 
+/* ATOMIC_EXCHANGE_USES_CAS is equal to 1 if atomic_exchange operations
+   are implemented based on a CAS loop; otherwise, this is 0 and we assume
+   that the atomic_exchange operations could provide better performance
+   than a CAS loop.  */
+#if ! defined ATOMIC_EXCHANGE_USES_CAS || ! (ATOMIC_EXCHANGE_USES_CAS == 0 \
+					     || ATOMIC_EXCHANGE_USES_CAS == 1)
+# error ATOMIC_EXCHANGE_USES_CAS has to be defined to 0 or 1
+#endif
+
 #endif	/* atomic.h */
diff --git a/nptl/pthread_spin_init.c b/nptl/pthread_spin_init.c
index 01dec5e..fe30913 100644
--- a/nptl/pthread_spin_init.c
+++ b/nptl/pthread_spin_init.c
@@ -22,6 +22,7 @@ 
 int
 pthread_spin_init (pthread_spinlock_t *lock, int pshared)
 {
-  *lock = 0;
+  /* Relaxed MO is fine because this is an initializing store.  */
+  atomic_store_relaxed (lock, 0);
   return 0;
 }
diff --git a/nptl/pthread_spin_lock.c b/nptl/pthread_spin_lock.c
index 4d03b78..682af80 100644
--- a/nptl/pthread_spin_lock.c
+++ b/nptl/pthread_spin_lock.c
@@ -19,27 +19,35 @@ 
 #include <atomic.h>
 #include "pthreadP.h"
 
-/* A machine-specific version can define SPIN_LOCK_READS_BETWEEN_CMPXCHG
-  to the number of plain reads that it's optimal to spin on between uses
-  of atomic_compare_and_exchange_val_acq.  If spinning forever is optimal
-  then use -1.  If no plain reads here would ever be optimal, use 0.  */
-#ifndef SPIN_LOCK_READS_BETWEEN_CMPXCHG
-# warning machine-dependent file should define SPIN_LOCK_READS_BETWEEN_CMPXCHG
-# define SPIN_LOCK_READS_BETWEEN_CMPXCHG 1000
-#endif
-
 int
 pthread_spin_lock (pthread_spinlock_t *lock)
 {
-  /* atomic_exchange usually takes less instructions than
-     atomic_compare_and_exchange.  On the other hand,
-     atomic_compare_and_exchange potentially generates less bus traffic
-     when the lock is locked.
-     We assume that the first try mostly will be successful, and we use
-     atomic_exchange.  For the subsequent tries we use
-     atomic_compare_and_exchange.  */
-  if (atomic_exchange_acq (lock, 1) == 0)
+  int val = 0;
+
+  /* We assume that the first try mostly will be successful, thus we use
+     atomic_exchange if it is not implemented by a CAS loop (we also assume
+     that atomic_exchange can be faster if it succeeds, see
+     ATOMIC_EXCHANGE_USES_CAS).  Otherwise, we use a weak CAS and not an
+     exchange so we bail out after the first failed attempt to change the
+     state.  For the subsequent attempts we use atomic_compare_and_exchange
+     after we observe that the lock is not acquired.
+     See also comment in pthread_spin_trylock.
+     We use acquire MO to synchronize-with the release MO store in
+     pthread_spin_unlock, and thus ensure that prior critical sections
+     happen-before this critical section.  */
+#if ! ATOMIC_EXCHANGE_USES_CAS
+  /* Try to acquire the lock with an exchange instruction as this architecture
+     has such an instruction and we assume it is faster than a CAS.
+     The acquisition succeeds if the lock is not in an acquired state.  */
+  if (__glibc_likely (atomic_exchange_acquire (lock, 1) == 0))
     return 0;
+#else
+  /* Try to acquire the lock with a CAS instruction as this architecture
+     has no exchange instruction.  The acquisition succeeds if the lock is not
+     acquired.  */
+  if (__glibc_likely (atomic_compare_exchange_weak_acquire (lock, &val, 1)))
+    return 0;
+#endif
 
   do
     {
@@ -47,23 +55,26 @@  pthread_spin_lock (pthread_spinlock_t *lock)
 	 to cmpxchg is not a good idea on many targets as that will force
 	 expensive memory synchronizations among processors and penalize other
 	 running threads.
-	 On the other hand, we do want to update memory state on the local core
-	 once in a while to avoid spinning indefinitely until some event that
-	 will happen to update local memory as a side-effect.  */
-      if (SPIN_LOCK_READS_BETWEEN_CMPXCHG >= 0)
+	 There is no technical reason for throwing in a CAS every now and then,
+	 and so far we have no evidence that it can improve performance.
+	 If that would be the case, we have to adjust other spin-waiting loops
+	 elsewhere, too!
+	 Thus we use relaxed MO reads until we observe the lock to not be
+	 acquired anymore.  */
+      do
 	{
-	  int wait = SPIN_LOCK_READS_BETWEEN_CMPXCHG;
+	  /* TODO Back-off.  */
 
-	  while (*lock != 0 && wait > 0)
-	    --wait;
-	}
-      else
-	{
-	  while (*lock != 0)
-	    ;
+	  atomic_spin_nop ();
+
+	  val = atomic_load_relaxed (lock);
 	}
+      while (val != 0);
+
+      /* We need acquire memory order here for the same reason as mentioned
+	 for the first try to lock the spinlock.  */
     }
-  while (atomic_compare_and_exchange_val_acq (lock, 1, 0) != 0);
+  while (!atomic_compare_exchange_weak_acquire (lock, &val, 1));
 
   return 0;
 }
diff --git a/nptl/pthread_spin_trylock.c b/nptl/pthread_spin_trylock.c
index 593bba3..83921b0 100644
--- a/nptl/pthread_spin_trylock.c
+++ b/nptl/pthread_spin_trylock.c
@@ -23,5 +23,57 @@ 
 int
 pthread_spin_trylock (pthread_spinlock_t *lock)
 {
-  return atomic_exchange_acq (lock, 1) ? EBUSY : 0;
+  /* For the spin try lock, we have the following possibilities:
+
+     1) If we assume that trylock will most likely succeed in practice:
+     * We just do an exchange.
+
+     2) If we want to bias towards cases where trylock succeeds, but don't
+     rule out contention:
+     * If exchange is not implemented by a CAS loop, and exchange is faster
+     than CAS, do an exchange.
+     * If exchange is implemented by a CAS loop, use a weak CAS and not an
+     exchange so we bail out after the first failed attempt to change the state.
+
+     3) If we expect contention to be likely:
+     * If CAS always brings the cache line into an exclusive state even if the
+     spinlock is already acquired, then load the value first with
+     atomic_load_relaxed and test if lock is not acquired. Then do 2).
+
+     We assume that 2) is the common case, and that this won't be slower than
+     1) in the common case.
+
+     We use acquire MO to synchronize-with the release MO store in
+     pthread_spin_unlock, and thus ensure that prior critical sections
+     happen-before this critical section.  */
+#if ! ATOMIC_EXCHANGE_USES_CAS
+  /* Try to acquire the lock with an exchange instruction as this architecture
+     has such an instruction and we assume it is faster than a CAS.
+     The acquisition succeeds if the lock is not in an acquired state.  */
+  if (atomic_exchange_acquire (lock, 1) == 0)
+    return 0;
+#else
+  /* Try to acquire the lock with a CAS instruction as this architecture
+     has no exchange instruction.  The acquisition succeeds if the lock is not
+     acquired.  */
+  do
+    {
+      int val = 0;
+      if (atomic_compare_exchange_weak_acquire (lock, &val, 1))
+	return 0;
+    }
+  /* atomic_compare_exchange_weak_acquire can fail spuriously.  Whereas
+     C++11 and C11 make it clear that trylock operations can fail spuriously,
+     POSIX does not explicitly specify this; it only specifies that failing
+     synchronization operations do not need to have synchronization effects
+     themselves, but a spurious failure is something that could contradict a
+     happens-before established earlier (e.g., that we need to observe that
+     the lock is acquired).  Therefore, we emulate a strong CAS by simply
+     checking with a relaxed MO load that the lock is really acquired before
+     returning EBUSY; the additional overhead this may cause is on the slow
+     path.  */
+  while (atomic_load_relaxed (lock) == 0);
+#endif
+
+  return EBUSY;
 }
diff --git a/nptl/pthread_spin_unlock.c b/nptl/pthread_spin_unlock.c
index 5fd73e5..f83b696 100644
--- a/nptl/pthread_spin_unlock.c
+++ b/nptl/pthread_spin_unlock.c
@@ -23,7 +23,9 @@ 
 int
 pthread_spin_unlock (pthread_spinlock_t *lock)
 {
-  atomic_full_barrier ();
-  *lock = 0;
+  /* The atomic_store_release synchronizes-with the atomic_exchange_acquire
+     or atomic_compare_exchange_weak_acquire in pthread_spin_lock /
+     pthread_spin_trylock.  */
+  atomic_store_release (lock, 0);
   return 0;
 }
diff --git a/sysdeps/aarch64/atomic-machine.h b/sysdeps/aarch64/atomic-machine.h
index a5d2213..eb59a5b 100644
--- a/sysdeps/aarch64/atomic-machine.h
+++ b/sysdeps/aarch64/atomic-machine.h
@@ -38,6 +38,7 @@  typedef uintmax_t uatomic_max_t;
 
 #define __HAVE_64B_ATOMICS 1
 #define USE_ATOMIC_COMPILER_BUILTINS 1
+#define ATOMIC_EXCHANGE_USES_CAS 0
 
 /* Compare and exchange.
    For all "bool" routines, we return FALSE if exchange succesful.  */
diff --git a/sysdeps/aarch64/nptl/pthread_spin_lock.c b/sysdeps/aarch64/nptl/pthread_spin_lock.c
deleted file mode 100644
index fcfcb40..0000000
--- a/sysdeps/aarch64/nptl/pthread_spin_lock.c
+++ /dev/null
@@ -1,24 +0,0 @@ 
-/* Copyright (C) 2008-2017 Free Software Foundation, Inc.
-
-   This file is part of the GNU C Library.
-
-   The GNU C Library is free software; you can redistribute it and/or
-   modify it under the terms of the GNU Lesser General Public License as
-   published by the Free Software Foundation; either version 2.1 of the
-   License, or (at your option) any later version.
-
-   The GNU C Library is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-   Lesser General Public License for more details.
-
-   You should have received a copy of the GNU Lesser General Public
-   License along with the GNU C Library; if not, see
-   <http://www.gnu.org/licenses/>.  */
-
-#define SPIN_LOCK_READS_BETWEEN_CMPXCHG 1000
-
-/* We can't use the normal "#include <nptl/pthread_spin_lock.c>" because
-   it will resolve to this very file.  Using "sysdeps/.." as reference to the
-   top level directory does the job.  */
-#include <sysdeps/../nptl/pthread_spin_lock.c>
diff --git a/sysdeps/alpha/atomic-machine.h b/sysdeps/alpha/atomic-machine.h
index 06e93f2..f710606 100644
--- a/sysdeps/alpha/atomic-machine.h
+++ b/sysdeps/alpha/atomic-machine.h
@@ -44,6 +44,7 @@  typedef uintmax_t uatomic_max_t;
 
 #define __HAVE_64B_ATOMICS 1
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 1
 
 
 #ifdef UP
diff --git a/sysdeps/arm/atomic-machine.h b/sysdeps/arm/atomic-machine.h
index eeac7f0..c22d05b 100644
--- a/sysdeps/arm/atomic-machine.h
+++ b/sysdeps/arm/atomic-machine.h
@@ -35,6 +35,7 @@  typedef uintmax_t uatomic_max_t;
 
 #define __HAVE_64B_ATOMICS 0
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 1
 
 void __arm_link_error (void);
 
diff --git a/sysdeps/arm/nptl/pthread_spin_lock.c b/sysdeps/arm/nptl/pthread_spin_lock.c
deleted file mode 100644
index 037b3b8..0000000
--- a/sysdeps/arm/nptl/pthread_spin_lock.c
+++ /dev/null
@@ -1,23 +0,0 @@ 
-/* Copyright (C) 2008-2017 Free Software Foundation, Inc.
-   This file is part of the GNU C Library.
-
-   The GNU C Library is free software; you can redistribute it and/or
-   modify it under the terms of the GNU Lesser General Public
-   License as published by the Free Software Foundation; either
-   version 2.1 of the License, or (at your option) any later version.
-
-   The GNU C Library is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-   Lesser General Public License for more details.
-
-   You should have received a copy of the GNU Lesser General Public
-   License along with the GNU C Library.  If not, see
-   <http://www.gnu.org/licenses/>.  */
-
-#define SPIN_LOCK_READS_BETWEEN_CMPXCHG 1000
-
-/* We can't use the normal "#include <nptl/pthread_spin_lock.c>" because
-   it will resolve to this very file.  Using "sysdeps/.." as reference to the
-   top level directory does the job.  */
-#include <sysdeps/../nptl/pthread_spin_lock.c>
diff --git a/sysdeps/hppa/nptl/pthread_spin_lock.c b/sysdeps/hppa/nptl/pthread_spin_lock.c
deleted file mode 100644
index 14f36a6..0000000
--- a/sysdeps/hppa/nptl/pthread_spin_lock.c
+++ /dev/null
@@ -1,23 +0,0 @@ 
-/* Copyright (C) 2005-2017 Free Software Foundation, Inc.
-   This file is part of the GNU C Library.
-
-   The GNU C Library is free software; you can redistribute it and/or
-   modify it under the terms of the GNU Lesser General Public
-   License as published by the Free Software Foundation; either
-   version 2.1 of the License, or (at your option) any later version.
-
-   The GNU C Library is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-   Lesser General Public License for more details.
-
-   You should have received a copy of the GNU Lesser General Public
-   License along with the GNU C Library.  If not, see
-   <http://www.gnu.org/licenses/>.  */
-
-#define SPIN_LOCK_READS_BETWEEN_CMPXCHG 1000
-
-/* We can't use the normal "#include <nptl/pthread_spin_lock.c>" because
-   it will resolve to this very file.  Using "sysdeps/.." as reference to the
-   top level directory does the job.  */
-#include <sysdeps/../nptl/pthread_spin_lock.c>
diff --git a/sysdeps/i386/atomic-machine.h b/sysdeps/i386/atomic-machine.h
index 77759f7..0e24200 100644
--- a/sysdeps/i386/atomic-machine.h
+++ b/sysdeps/i386/atomic-machine.h
@@ -56,6 +56,7 @@  typedef uintmax_t uatomic_max_t;
 
 #define __HAVE_64B_ATOMICS 0
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 0
 
 
 #define atomic_compare_and_exchange_val_acq(mem, newval, oldval) \
diff --git a/sysdeps/ia64/atomic-machine.h b/sysdeps/ia64/atomic-machine.h
index ecf9750..971084e 100644
--- a/sysdeps/ia64/atomic-machine.h
+++ b/sysdeps/ia64/atomic-machine.h
@@ -45,6 +45,7 @@  typedef uintmax_t uatomic_max_t;
 
 #define __HAVE_64B_ATOMICS 1
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 0
 
 
 #define __arch_compare_and_exchange_bool_8_acq(mem, newval, oldval) \
diff --git a/sysdeps/m68k/coldfire/atomic-machine.h b/sysdeps/m68k/coldfire/atomic-machine.h
index 9aeb993..72b037e 100644
--- a/sysdeps/m68k/coldfire/atomic-machine.h
+++ b/sysdeps/m68k/coldfire/atomic-machine.h
@@ -52,6 +52,7 @@  typedef uintmax_t uatomic_max_t;
 /* If we have just non-atomic operations, we can as well make them wide.  */
 #define __HAVE_64B_ATOMICS 1
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 1
 
 /* The only basic operation needed is compare and exchange.  */
 #define atomic_compare_and_exchange_val_acq(mem, newval, oldval) \
diff --git a/sysdeps/m68k/m680x0/m68020/atomic-machine.h b/sysdeps/m68k/m680x0/m68020/atomic-machine.h
index 00dc22d..a319471 100644
--- a/sysdeps/m68k/m680x0/m68020/atomic-machine.h
+++ b/sysdeps/m68k/m680x0/m68020/atomic-machine.h
@@ -46,6 +46,7 @@  typedef uintmax_t uatomic_max_t;
 
 #define __HAVE_64B_ATOMICS 1
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 1
 
 #define __arch_compare_and_exchange_val_8_acq(mem, newval, oldval) \
   ({ __typeof (*(mem)) __ret;						      \
diff --git a/sysdeps/m68k/nptl/pthread_spin_lock.c b/sysdeps/m68k/nptl/pthread_spin_lock.c
deleted file mode 100644
index 62795f4..0000000
--- a/sysdeps/m68k/nptl/pthread_spin_lock.c
+++ /dev/null
@@ -1,24 +0,0 @@ 
-/* Copyright (C) 2010-2017 Free Software Foundation, Inc.
-   This file is part of the GNU C Library.
-   Contributed by Maxim Kuvyrkov <maxim@codesourcery.com>, 2010.
-
-   The GNU C Library is free software; you can redistribute it and/or
-   modify it under the terms of the GNU Lesser General Public
-   License as published by the Free Software Foundation; either
-   version 2.1 of the License, or (at your option) any later version.
-
-   The GNU C Library is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-   Lesser General Public License for more details.
-
-   You should have received a copy of the GNU Lesser General Public
-   License along with the GNU C Library.  If not, see
-   <http://www.gnu.org/licenses/>.  */
-
-#define SPIN_LOCK_READS_BETWEEN_CMPXCHG 1000
-
-/* We can't use the normal "#include <nptl/pthread_spin_lock.c>" because
-   it will resolve to this very file.  Using "sysdeps/.." as reference to the
-   top level directory does the job.  */
-#include <sysdeps/../nptl/pthread_spin_lock.c>
diff --git a/sysdeps/microblaze/atomic-machine.h b/sysdeps/microblaze/atomic-machine.h
index dc5309c..49315c6 100644
--- a/sysdeps/microblaze/atomic-machine.h
+++ b/sysdeps/microblaze/atomic-machine.h
@@ -37,6 +37,7 @@  typedef uintmax_t uatomic_max_t;
 
 #define __HAVE_64B_ATOMICS 0
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 1
 
 
 /* Microblaze does not have byte and halfword forms of load and reserve and
diff --git a/sysdeps/microblaze/nptl/pthread_spin_lock.c b/sysdeps/microblaze/nptl/pthread_spin_lock.c
deleted file mode 100644
index fcfcb40..0000000
--- a/sysdeps/microblaze/nptl/pthread_spin_lock.c
+++ /dev/null
@@ -1,24 +0,0 @@ 
-/* Copyright (C) 2008-2017 Free Software Foundation, Inc.
-
-   This file is part of the GNU C Library.
-
-   The GNU C Library is free software; you can redistribute it and/or
-   modify it under the terms of the GNU Lesser General Public License as
-   published by the Free Software Foundation; either version 2.1 of the
-   License, or (at your option) any later version.
-
-   The GNU C Library is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-   Lesser General Public License for more details.
-
-   You should have received a copy of the GNU Lesser General Public
-   License along with the GNU C Library; if not, see
-   <http://www.gnu.org/licenses/>.  */
-
-#define SPIN_LOCK_READS_BETWEEN_CMPXCHG 1000
-
-/* We can't use the normal "#include <nptl/pthread_spin_lock.c>" because
-   it will resolve to this very file.  Using "sysdeps/.." as reference to the
-   top level directory does the job.  */
-#include <sysdeps/../nptl/pthread_spin_lock.c>
diff --git a/sysdeps/mips/atomic-machine.h b/sysdeps/mips/atomic-machine.h
index 54c182b..3d9da0c 100644
--- a/sysdeps/mips/atomic-machine.h
+++ b/sysdeps/mips/atomic-machine.h
@@ -50,6 +50,8 @@  typedef uintmax_t uatomic_max_t;
 #define __HAVE_64B_ATOMICS 1
 #endif
 
+#define ATOMIC_EXCHANGE_USES_CAS 0
+
 /* See the comments in <sys/asm.h> about the use of the sync instruction.  */
 #ifndef MIPS_SYNC
 # define MIPS_SYNC	sync
diff --git a/sysdeps/mips/nptl/pthread_spin_lock.c b/sysdeps/mips/nptl/pthread_spin_lock.c
deleted file mode 100644
index 19d87a5..0000000
--- a/sysdeps/mips/nptl/pthread_spin_lock.c
+++ /dev/null
@@ -1,23 +0,0 @@ 
-/* Copyright (C) 2012-2017 Free Software Foundation, Inc.
-   This file is part of the GNU C Library.
-
-   The GNU C Library is free software; you can redistribute it and/or
-   modify it under the terms of the GNU Lesser General Public
-   License as published by the Free Software Foundation; either
-   version 2.1 of the License, or (at your option) any later version.
-
-   The GNU C Library is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-   Lesser General Public License for more details.
-
-   You should have received a copy of the GNU Lesser General Public
-   License along with the GNU C Library.  If not, see
-   <http://www.gnu.org/licenses/>.  */
-
-#define SPIN_LOCK_READS_BETWEEN_CMPXCHG 1000
-
-/* We can't use the normal "#include <nptl/pthread_spin_lock.c>" because
-   it will resolve to this very file.  Using "sysdeps/.." as reference to the
-   top level directory does the job.  */
-#include <sysdeps/../nptl/pthread_spin_lock.c>
diff --git a/sysdeps/nios2/nptl/pthread_spin_lock.c b/sysdeps/nios2/nptl/pthread_spin_lock.c
deleted file mode 100644
index b203469..0000000
--- a/sysdeps/nios2/nptl/pthread_spin_lock.c
+++ /dev/null
@@ -1,24 +0,0 @@ 
-/* pthread spin-lock implementation for Nios II.
-   Copyright (C) 2005-2017 Free Software Foundation, Inc.
-   This file is part of the GNU C Library.
-
-   The GNU C Library is free software; you can redistribute it and/or
-   modify it under the terms of the GNU Lesser General Public
-   License as published by the Free Software Foundation; either
-   version 2.1 of the License, or (at your option) any later version.
-
-   The GNU C Library is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-   Lesser General Public License for more details.
-
-   You should have received a copy of the GNU Lesser General Public
-   License along with the GNU C Library.  If not, see
-   <http://www.gnu.org/licenses/>.  */
-
-#define SPIN_LOCK_READS_BETWEEN_CMPXCHG 1000
-
-/* We can't use the normal "#include <nptl/pthread_spin_lock.c>" because
-   it will resolve to this very file.  Using "sysdeps/.." as reference to the
-   top level directory does the job.  */
-#include <sysdeps/../nptl/pthread_spin_lock.c>
diff --git a/sysdeps/powerpc/powerpc32/atomic-machine.h b/sysdeps/powerpc/powerpc32/atomic-machine.h
index 20d5e85..96c7d81 100644
--- a/sysdeps/powerpc/powerpc32/atomic-machine.h
+++ b/sysdeps/powerpc/powerpc32/atomic-machine.h
@@ -35,6 +35,7 @@ 
 
 #define __HAVE_64B_ATOMICS 0
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 1
 
 /*
  * The 32-bit exchange_bool is different on powerpc64 because the subf
diff --git a/sysdeps/powerpc/powerpc64/atomic-machine.h b/sysdeps/powerpc/powerpc64/atomic-machine.h
index 40c308e..46df488 100644
--- a/sysdeps/powerpc/powerpc64/atomic-machine.h
+++ b/sysdeps/powerpc/powerpc64/atomic-machine.h
@@ -35,6 +35,7 @@ 
 
 #define __HAVE_64B_ATOMICS 1
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 1
 
 /* The 32-bit exchange_bool is different on powerpc64 because the subf
    does signed 64-bit arithmetic while the lwarx is 32-bit unsigned
diff --git a/sysdeps/s390/atomic-machine.h b/sysdeps/s390/atomic-machine.h
index 690d2e3..adaca40 100644
--- a/sysdeps/s390/atomic-machine.h
+++ b/sysdeps/s390/atomic-machine.h
@@ -67,6 +67,8 @@  typedef uintmax_t uatomic_max_t;
 # define __HAVE_64B_ATOMICS 0
 #endif
 
+#define ATOMIC_EXCHANGE_USES_CAS 1
+
 /* Implement some of the non-C11 atomic macros from include/atomic.h
    with help of the C11 atomic builtins.  The other non-C11 atomic macros
    are using the macros defined here.  */
diff --git a/sysdeps/sparc/sparc32/atomic-machine.h b/sysdeps/sparc/sparc32/atomic-machine.h
index acd029e..5950131 100644
--- a/sysdeps/sparc/sparc32/atomic-machine.h
+++ b/sysdeps/sparc/sparc32/atomic-machine.h
@@ -49,6 +49,7 @@  typedef uintmax_t uatomic_max_t;
 
 #define __HAVE_64B_ATOMICS 0
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 1
 
 
 /* We have no compare and swap, just test and set.
diff --git a/sysdeps/sparc/sparc32/sparcv9/atomic-machine.h b/sysdeps/sparc/sparc32/sparcv9/atomic-machine.h
index 7f7895e..07e67e6 100644
--- a/sysdeps/sparc/sparc32/sparcv9/atomic-machine.h
+++ b/sysdeps/sparc/sparc32/sparcv9/atomic-machine.h
@@ -46,6 +46,7 @@  typedef uintmax_t uatomic_max_t;
 
 #define __HAVE_64B_ATOMICS 0
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 1
 
 
 #define __arch_compare_and_exchange_val_8_acq(mem, newval, oldval) \
diff --git a/sysdeps/sparc/sparc64/atomic-machine.h b/sysdeps/sparc/sparc64/atomic-machine.h
index 44a43ff..d3d95e3 100644
--- a/sysdeps/sparc/sparc64/atomic-machine.h
+++ b/sysdeps/sparc/sparc64/atomic-machine.h
@@ -46,6 +46,7 @@  typedef uintmax_t uatomic_max_t;
 
 #define __HAVE_64B_ATOMICS 1
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 1
 
 
 #define __arch_compare_and_exchange_val_8_acq(mem, newval, oldval) \
diff --git a/sysdeps/tile/tilegx/atomic-machine.h b/sysdeps/tile/tilegx/atomic-machine.h
index 6345251..e77f670 100644
--- a/sysdeps/tile/tilegx/atomic-machine.h
+++ b/sysdeps/tile/tilegx/atomic-machine.h
@@ -31,6 +31,7 @@ 
 #endif
 
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 0
 
 /* Pick appropriate 8- or 4-byte instruction. */
 #define __atomic_update(mem, v, op)                                     \
diff --git a/sysdeps/tile/tilepro/atomic-machine.h b/sysdeps/tile/tilepro/atomic-machine.h
index 33a8b85..45e36de 100644
--- a/sysdeps/tile/tilepro/atomic-machine.h
+++ b/sysdeps/tile/tilepro/atomic-machine.h
@@ -23,6 +23,7 @@ 
 
 #define __HAVE_64B_ATOMICS 0
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 0
 
 /* 32-bit integer compare-and-exchange. */
 static __inline __attribute__ ((always_inline))
diff --git a/sysdeps/unix/sysv/linux/hppa/atomic-machine.h b/sysdeps/unix/sysv/linux/hppa/atomic-machine.h
index 2cd2235..9adcab7 100644
--- a/sysdeps/unix/sysv/linux/hppa/atomic-machine.h
+++ b/sysdeps/unix/sysv/linux/hppa/atomic-machine.h
@@ -38,6 +38,7 @@  typedef uintmax_t uatomic_max_t;
 
 #define __HAVE_64B_ATOMICS 0
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 1
 
 /* prev = *addr;
    if (prev == old)
diff --git a/sysdeps/unix/sysv/linux/m68k/coldfire/atomic-machine.h b/sysdeps/unix/sysv/linux/m68k/coldfire/atomic-machine.h
index 6755a05..c1b6950 100644
--- a/sysdeps/unix/sysv/linux/m68k/coldfire/atomic-machine.h
+++ b/sysdeps/unix/sysv/linux/m68k/coldfire/atomic-machine.h
@@ -38,6 +38,7 @@  typedef uintmax_t uatomic_max_t;
 
 #define __HAVE_64B_ATOMICS 0
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 1
 
 /* The only basic operation needed is compare and exchange.  */
 /* For ColdFire we'll have to trap into the kernel mode anyway,
diff --git a/sysdeps/unix/sysv/linux/nios2/atomic-machine.h b/sysdeps/unix/sysv/linux/nios2/atomic-machine.h
index 6111ccf..d98fa66 100644
--- a/sysdeps/unix/sysv/linux/nios2/atomic-machine.h
+++ b/sysdeps/unix/sysv/linux/nios2/atomic-machine.h
@@ -33,6 +33,7 @@  typedef uintmax_t uatomic_max_t;
 
 #define __HAVE_64B_ATOMICS 0
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 1
 
 #define __arch_compare_and_exchange_val_8_acq(mem, newval, oldval)	\
   (abort (), (__typeof (*mem)) 0)
diff --git a/sysdeps/unix/sysv/linux/sh/atomic-machine.h b/sysdeps/unix/sysv/linux/sh/atomic-machine.h
index 3c58b70..428b71e 100644
--- a/sysdeps/unix/sysv/linux/sh/atomic-machine.h
+++ b/sysdeps/unix/sysv/linux/sh/atomic-machine.h
@@ -46,6 +46,7 @@  typedef uintmax_t uatomic_max_t;
 
 #define __HAVE_64B_ATOMICS 0
 #define USE_ATOMIC_COMPILER_BUILTINS 0
+#define ATOMIC_EXCHANGE_USES_CAS 1
 
 /* SH kernel has implemented a gUSA ("g" User Space Atomicity) support
    for the user space atomicity. The atomicity macros use this scheme.
diff --git a/sysdeps/x86_64/atomic-machine.h b/sysdeps/x86_64/atomic-machine.h
index 2e8a9aa..c454734 100644
--- a/sysdeps/x86_64/atomic-machine.h
+++ b/sysdeps/x86_64/atomic-machine.h
@@ -59,6 +59,7 @@  typedef uintmax_t uatomic_max_t;
 
 #define __HAVE_64B_ATOMICS 1
 #define USE_ATOMIC_COMPILER_BUILTINS 1
+#define ATOMIC_EXCHANGE_USES_CAS 0
 
 #define atomic_compare_and_exchange_val_acq(mem, newval, oldval) \
   __sync_val_compare_and_swap (mem, oldval, newval)
-- 
2.7.4