S/390: Port of lock elision to System/z
diff mbox

Message ID lkimul$738$1@ger.gmane.org
State New
Headers show

Commit Message

Stefan Liebler May 9, 2014, 1:58 p.m. UTC
Hi,

thanks for the review.
I´ve changed the mentioned points.
The Changelog remains the same.

Bye

On 05/08/2014 11:44 AM, Andreas Krebbel wrote:
> On 04/30/2014 04:28 PM, Stefan Liebler wrote:
>> Hi,
>>
>> like mentioned in the discussion before, the clobbering of
>> FPRs/cc/memory is not needed anymore. In an older patch-version, the
>> transaction was either started by the builtin_tbegin or an asm("tbegin").
>> In the latter case, the clobbering of FPRs was needed.
>> There was also a non-transactional store, which is not needed anymore.
>> The objdumps of this patch-version with and without the clobbering does
>> not differ.
>>
>> The file nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_unlock.c is not
>> added anymore, because it only includes the nptl/pthread_mutex_unlock.c.
>>
>> Ok?
>>
>> Bye
>>
>> ---
>> 2014-04-30  Dominik Vogt  <vogt@linux.vnet.ibm.com>
>>               Stefan Liebler  <stli@linux.vnet.ibm.com>
>>
>> 	* config.make.in (enable-lock-elision): New Makefile variable.
>> 	* configure.ac: Likewise.
>> 	* configure: Regenerate.
>> 	* sysdeps/s390/configure.ac:
>> 	Add check for gcc transactions support.
>> 	* sysdeps/s390/configure: Regenerate.
>> 	* nptl/sysdeps/unix/sysv/linux/s390/Makefile: New file.
>> 	Build elision files if enabled.
>> 	* nptl/sysdeps/unix/sysv/linux/s390/elision-conf.c: New file.
>> 	Add lock elision support for s390.
>> 	* nptl/sysdeps/unix/sysv/linux/s390/elision-conf.h: Likewise.
>> 	* nptl/sysdeps/unix/sysv/linux/s390/elision-lock.c: Likewise.
>> 	* nptl/sysdeps/unix/sysv/linux/s390/elision-timed.c: Likewise.
>> 	* nptl/sysdeps/unix/sysv/linux/s390/elision-trylock.c: Likewise.
>> 	* nptl/sysdeps/unix/sysv/linux/s390/elision-unlock.c: Likewise.
>> 	* nptl/sysdeps/unix/sysv/linux/s390/force-elision.h: Likewise.
>> 	* nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_cond_lock.c:
>> 	Likewise.
>> 	* nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_lock.c:
>> 	Likewise.
>> 	* nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_timedlock.c:
>> 	Likewise.
>> 	* nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_trylock.c:
>> 	Likewise.
>> 	* nptl/sysdeps/unix/sysv/linux/s390/lowlevellock.h:
>> 	(__lll_timedlock_elision, __lll_lock_elision)
>> 	(__lll_unlock_elision, __lll_trylock_elision)
>> 	(lll_timedlock_elision, lll_lock_elision)
>> 	(lll_unlock_elision, lll_trylock_elision): Add.
>> 	* nptl/sysdeps/unix/sysv/linux/s390/bits/pthreadtypes.h
>> 	(pthread_mutex_t): Add lock elision support for s390.
>> ---
>>
>
> Just some (very) minor nitpicking:
>
> nptl/sysdeps/unix/sysv/linux/s390/lowlevellock.h:
>
> +#ifdef ENABLE_LOCK_ELISION
> +extern int __lll_lock_elision (int *futex, short *adapt_count, int private)
> +  attribute_hidden;
> +
> +extern int __lll_unlock_elision(int *lock, int private)
> +  attribute_hidden;
> +
> +extern int __lll_trylock_elision(int *lock, short *adapt_count)
> +  attribute_hidden;
>
> What's the reason for calling the first argument futex once and lock
> in the other cases?
No. Now the argument is called futex in all cases.
Same for nptl/sysdeps/unix/sysv/linux/s390/elision-unlock.c.

>
> nptl/sysdeps/unix/sysv/linux/s390/elision-conf.c:
>
> +{
> +  /* Set when the CPU supports elision. When false elision is never
>    attempted.  */
> ... when the CPU and the kernel support transactional execution. ...
>
> +  int elision_available = (GLRO (dl_hwcap) & HWCAP_S390_TE) ? 1 : 0;
>
ok

>
> nptl/sysdeps/unix/sysv/linux/s390/elision-lock.c:
>
> +int
> +__lll_lock_elision (int *futex, short *adapt_count, EXTRAARG int private)
> +{
> +  if (*adapt_count <= 0)
> +    {
> ...
> +    {
> +  else
> +    {
> +      /* Lost updates are possible, but harmless.  Due to races this might lead
> +	 to *adapt_count becoming less than zero.  */
> +      (*adapt_count)--;
> +    }
> +
> +  use_lock:
> +  return LLL_LOCK ((*futex), private);
> +}
>
> Perhaps something like this to reduce indentation of the whole function?!
>
> +int
> +__lll_lock_elision (int *futex, short *adapt_count, EXTRAARG int private)
> +{
> +  if (*adapt_count > 0)
> +    {
> +      /* Lost updates are possible, but harmless.  Due to races this might lead
> +	 to *adapt_count becoming less than zero.  */
> +      (*adapt_count)--;
> +      goto use_lock;
> +    }
> ...
> +
> +  use_lock:
> +  return LLL_LOCK ((*futex), private);
> +}
ok

>
> +      /* Same logic as above, but for for a number of tepmorary failures in a
> +	 row.  */
>
> ... temporary ...
>
ok

> nptl/sysdeps/unix/sysv/linux/s390/elision-trylock.c:
>
> +  /* Implement POSIX semantics by forbiding nesting elided trylocks.
> ... forbidding ...
>
> +      __builtin_tabort (_HTM_FIRST_USER_ABORT_CODE + 1);
>
> Here you want to set the lowest order bit in order to enforce a
> persistent abort.  While the code works fine since
> _HTM_FIRST_USER_ABORT_CODE is even (_HTM_FIRST_USER_ABORT_CODE | 1)
> would appear more natural to me.  The same appears in elision-lock.c.
ok

>
> Bye,
>
> -Andreas-
>
>

Comments

Andreas Krebbel May 9, 2014, 3:02 p.m. UTC | #1
On 05/09/2014 03:58 PM, Stefan Liebler wrote:
> Hi,
> 
> thanks for the review.
> I´ve changed the mentioned points.
> The Changelog remains the same.

I've just committed the patch. Thanks to you and Dominik for working on this!

Bye,

-Andreas-

Patch
diff mbox

diff --git a/config.make.in b/config.make.in
index 132d179..6bcab8a 100644
--- a/config.make.in
+++ b/config.make.in
@@ -93,6 +93,7 @@  build-nscd = @build_nscd@
 use-nscd = @use_nscd@
 build-hardcoded-path-in-tests= @hardcoded_path_in_tests@
 build-pt-chown = @build_pt_chown@
+enable-lock-elision = @enable_lock_elision@
 
 # Build tools.
 CC = @CC@
diff --git a/configure b/configure
index 8b0b222..ecc282b 100755
--- a/configure
+++ b/configure
@@ -651,6 +651,7 @@  libc_cv_nss_crypt
 all_warnings
 force_install
 bindnow
+enable_lock_elision
 hardcoded_path_in_tests
 oldest_abi
 use_default_link
@@ -3476,6 +3477,7 @@  else
   enable_lock_elision=no
 fi
 
+
 if test "$enable_lock_elision" = yes ; then
   $as_echo "#define ENABLE_LOCK_ELISION 1" >>confdefs.h
 
diff --git a/configure.ac b/configure.ac
index 97a9591..babfe57 100644
--- a/configure.ac
+++ b/configure.ac
@@ -184,6 +184,7 @@  AC_ARG_ENABLE([lock-elision],
 			     [Enable lock elision for pthread mutexes by default]),
 	      [enable_lock_elision=$enableval],
 	      [enable_lock_elision=no])
+AC_SUBST(enable_lock_elision)
 if test "$enable_lock_elision" = yes ; then
   AC_DEFINE(ENABLE_LOCK_ELISION)
 fi
diff --git a/nptl/sysdeps/unix/sysv/linux/s390/Makefile b/nptl/sysdeps/unix/sysv/linux/s390/Makefile
new file mode 100644
index 0000000..269832f
--- /dev/null
+++ b/nptl/sysdeps/unix/sysv/linux/s390/Makefile
@@ -0,0 +1,10 @@ 
+ifeq ($(enable-lock-elision),yes)
+libpthread-sysdep_routines += elision-lock elision-unlock elision-timed \
+			      elision-trylock
+
+elision-CFLAGS = -mhtm
+CFLAGS-elision-lock.c = $(elision-CFLAGS)
+CFLAGS-elision-timed.c = $(elision-CFLAGS)
+CFLAGS-elision-trylock.c = $(elision-CFLAGS)
+CFLAGS-elision-unlock.c = $(elision-CFLAGS)
+endif
diff --git a/nptl/sysdeps/unix/sysv/linux/s390/bits/pthreadtypes.h b/nptl/sysdeps/unix/sysv/linux/s390/bits/pthreadtypes.h
index 8264de0..d70f8b3 100644
--- a/nptl/sysdeps/unix/sysv/linux/s390/bits/pthreadtypes.h
+++ b/nptl/sysdeps/unix/sysv/linux/s390/bits/pthreadtypes.h
@@ -89,14 +89,37 @@  typedef union
        binary compatibility.  */
     int __kind;
 #if __WORDSIZE == 64
+# ifdef ENABLE_LOCK_ELISION
+    short __spins;
+    short __elision;
+    /* Mutex __spins initializer used by PTHREAD_MUTEX_INITIALIZER.  */
+#  define __PTHREAD_SPINS               0, 0
+# else
     int __spins;
+    /* Mutex __spins initializer used by PTHREAD_MUTEX_INITIALIZER.  */
+#  define __PTHREAD_SPINS               0
+# endif
     __pthread_list_t __list;
 # define __PTHREAD_MUTEX_HAVE_PREV	1
 #else
     unsigned int __nusers;
     __extension__ union
     {
+# ifdef ENABLE_LOCK_ELISION
+      struct
+      {
+	short __espins;
+	short __elision;
+      } _d;
+#  define __spins _d.__espins
+#  define __elision _d.__elision
+    /* Mutex __spins initializer used by PTHREAD_MUTEX_INITIALIZER.  */
+#  define __PTHREAD_SPINS               { 0, 0 }
+# else
       int __spins;
+    /* Mutex __spins initializer used by PTHREAD_MUTEX_INITIALIZER.  */
+#  define __PTHREAD_SPINS               0
+# endif
       __pthread_slist_t __list;
     };
 #endif
@@ -105,8 +128,6 @@  typedef union
   long int __align;
 } pthread_mutex_t;
 
-/* Mutex __spins initializer used by PTHREAD_MUTEX_INITIALIZER.  */
-#define __PTHREAD_SPINS 0
 
 typedef union
 {
diff --git a/nptl/sysdeps/unix/sysv/linux/s390/elision-conf.c b/nptl/sysdeps/unix/sysv/linux/s390/elision-conf.c
new file mode 100644
index 0000000..69c0483
--- /dev/null
+++ b/nptl/sysdeps/unix/sysv/linux/s390/elision-conf.c
@@ -0,0 +1,82 @@ 
+/* Lock elision tunable parameters.
+   Copyright (C) 2014 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <config.h>
+#include <pthreadP.h>
+#include <elision-conf.h>
+#include <unistd.h>
+#include <dl-procinfo.h>
+
+/* Reasonable initial tuning values, may be revised in the future.
+   This is a conservative initial value.  */
+
+struct elision_config __elision_aconf =
+  {
+    /* How often to not attempt to use elision if a transaction aborted
+       because the lock is already acquired.  Expressed in number of lock
+       acquisition attempts.  */
+    .skip_lock_busy = 3,
+    /* How often to not attempt to use elision if a transaction aborted due
+       to reasons other than other threads' memory accesses.  Expressed in
+       number of lock acquisition attempts.  */
+    .skip_lock_internal_abort = 3,
+    /* How often to not attempt to use elision if a lock used up all retries
+       without success.  Expressed in number of lock acquisition attempts.  */
+    .skip_lock_out_of_tbegin_retries = 3,
+    /* How often we try using elision if there is chance for the transaction
+       to finish execution (e.g., it wasn't aborted due to the lock being
+       already acquired.  */
+    .try_tbegin = 3,
+    /* Same as SKIP_LOCK_INTERNAL_ABORT but for trylock.  */
+    .skip_trylock_internal_abort = 3,
+  };
+
+/* Force elision for all new locks.  This is used to decide whether existing
+   DEFAULT locks should be automatically upgraded to elision in
+   pthread_mutex_lock().  Disabled for suid programs.  Only used when elision
+   is available.  */
+
+int __pthread_force_elision attribute_hidden = 0;
+
+/* Initialize elison.  */
+
+static void
+elision_init (int argc __attribute__ ((unused)),
+	      char **argv  __attribute__ ((unused)),
+	      char **environ)
+{
+  /* Set when the CPU and the kernel supports transactional execution.
+     When false elision is never attempted.  */
+  int elision_available = (GLRO (dl_hwcap) & HWCAP_S390_TE) ? 1 : 0;
+
+  __pthread_force_elision = __libc_enable_secure ? 0 : elision_available;
+}
+
+#ifdef SHARED
+# define INIT_SECTION ".init_array"
+# define MAYBE_CONST
+#else
+# define INIT_SECTION ".preinit_array"
+# define MAYBE_CONST const
+#endif
+
+void (*MAYBE_CONST __pthread_init_array []) (int, char **, char **)
+  __attribute__ ((section (INIT_SECTION), aligned (sizeof (void *)))) =
+{
+  &elision_init
+};
diff --git a/nptl/sysdeps/unix/sysv/linux/s390/elision-conf.h b/nptl/sysdeps/unix/sysv/linux/s390/elision-conf.h
new file mode 100644
index 0000000..d9e9794
--- /dev/null
+++ b/nptl/sysdeps/unix/sysv/linux/s390/elision-conf.h
@@ -0,0 +1,44 @@ 
+/* Lock elision tunable parameters.
+   Copyright (C) 2014 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+#ifdef ENABLE_LOCK_ELISION
+#ifndef _ELISION_CONF_H
+#define _ELISION_CONF_H 1
+
+#include <pthread.h>
+#include <time.h>
+
+/* Should make sure there is no false sharing on this.  */
+
+struct elision_config
+{
+  int skip_lock_busy;
+  int skip_lock_internal_abort;
+  int skip_lock_out_of_tbegin_retries;
+  int try_tbegin;
+  int skip_trylock_internal_abort;
+};
+
+extern struct elision_config __elision_aconf attribute_hidden;
+
+extern int __pthread_force_elision attribute_hidden;
+
+/* Tell the test suite to test elision for this architecture.  */
+#define HAVE_ELISION 1
+
+#endif
+#endif
diff --git a/nptl/sysdeps/unix/sysv/linux/s390/elision-lock.c b/nptl/sysdeps/unix/sysv/linux/s390/elision-lock.c
new file mode 100644
index 0000000..ba5338f
--- /dev/null
+++ b/nptl/sysdeps/unix/sysv/linux/s390/elision-lock.c
@@ -0,0 +1,119 @@ 
+/* Elided pthread mutex lock.
+   Copyright (C) 2014 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <pthread.h>
+#include <pthreadP.h>
+#include <lowlevellock.h>
+#include <htmintrin.h>
+#include <elision-conf.h>
+#include <stdint.h>
+
+#if !defined(LLL_LOCK) && !defined(EXTRAARG)
+/* Make sure the configuration code is always linked in for static
+   libraries.  */
+#include "elision-conf.c"
+#endif
+
+#ifndef EXTRAARG
+#define EXTRAARG
+#endif
+#ifndef LLL_LOCK
+#define LLL_LOCK(a,b) lll_lock(a,b), 0
+#endif
+
+#define aconf __elision_aconf
+
+/* Adaptive lock using transactions.
+   By default the lock region is run as a transaction, and when it
+   aborts or the lock is busy the lock adapts itself.  */
+
+int
+__lll_lock_elision (int *futex, short *adapt_count, EXTRAARG int private)
+{
+  if (*adapt_count > 0)
+    {
+      /* Lost updates are possible, but harmless.  Due to races this might lead
+	 to *adapt_count becoming less than zero.  */
+      (*adapt_count)--;
+      goto use_lock;
+    }
+
+  __asm__ volatile (".machinemode \"zarch_nohighgprs\"\n\t"
+		    ".machine \"all\""
+		    : : : "memory");
+
+  int try_tbegin;
+  for (try_tbegin = aconf.try_tbegin;
+       try_tbegin > 0;
+       try_tbegin--)
+    {
+      unsigned status;
+      if (__builtin_expect
+	  ((status = __builtin_tbegin((void *)0)) == _HTM_TBEGIN_STARTED, 1))
+	{
+	  if (*futex == 0)
+	    return 0;
+	  /* Lock was busy.  Fall back to normal locking.  */
+	  if (__builtin_expect (__builtin_tx_nesting_depth (), 1))
+	    {
+	      /* In a non-nested transaction there is no need to abort,
+		 which is expensive.  */
+	      __builtin_tend ();
+	      if (aconf.skip_lock_busy > 0)
+		*adapt_count = aconf.skip_lock_busy;
+	      goto use_lock;
+	    }
+	  else /* nesting depth is > 1 */
+	    {
+	      /* A nested transaction will abort eventually because it
+		 cannot make any progress before *futex changes back to 0.
+		 So we may as well abort immediately.
+		 This persistently aborts the outer transaction to force
+		 the outer mutex use the default lock instead of retrying
+		 with transactions until the try_tbegin of the outer mutex
+		 is zero.
+		 The adapt_count of this inner mutex is not changed,
+		 because using the default lock with the inner mutex
+		 would abort the outer transaction.
+	      */
+	      __builtin_tabort (_HTM_FIRST_USER_ABORT_CODE | 1);
+	    }
+	}
+      else
+	{
+	  if (status != _HTM_TBEGIN_TRANSIENT)
+	    {
+	      /* A persistent abort (cc 1 or 3) indicates that a retry is
+		 probably futile.  Use the normal locking now and for the
+		 next couple of calls.
+		 Be careful to avoid writing to the lock.  */
+	      if (aconf.skip_lock_internal_abort > 0)
+		*adapt_count = aconf.skip_lock_internal_abort;
+	      goto use_lock;
+	    }
+	}
+    }
+
+  /* Same logic as above, but for for a number of temporary failures in a
+     row.  */
+  if (aconf.skip_lock_out_of_tbegin_retries > 0 && aconf.try_tbegin > 0)
+    *adapt_count = aconf.skip_lock_out_of_tbegin_retries;
+
+  use_lock:
+  return LLL_LOCK ((*futex), private);
+}
diff --git a/nptl/sysdeps/unix/sysv/linux/s390/elision-timed.c b/nptl/sysdeps/unix/sysv/linux/s390/elision-timed.c
new file mode 100644
index 0000000..a8d8b2a
--- /dev/null
+++ b/nptl/sysdeps/unix/sysv/linux/s390/elision-timed.c
@@ -0,0 +1,26 @@ 
+/* Lock elision timed lock.
+   Copyright (C) 2014 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <time.h>
+#include <elision-conf.h>
+#include <lowlevellock.h>
+#define __lll_lock_elision __lll_timedlock_elision
+#define EXTRAARG const struct timespec *t,
+#undef LLL_LOCK
+#define LLL_LOCK(a, b) lll_timedlock(a, t, b)
+#include "elision-lock.c"
diff --git a/nptl/sysdeps/unix/sysv/linux/s390/elision-trylock.c b/nptl/sysdeps/unix/sysv/linux/s390/elision-trylock.c
new file mode 100644
index 0000000..61447d6
--- /dev/null
+++ b/nptl/sysdeps/unix/sysv/linux/s390/elision-trylock.c
@@ -0,0 +1,94 @@ 
+/* Elided pthread mutex trylock.
+   Copyright (C) 2014 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <pthread.h>
+#include <pthreadP.h>
+#include <lowlevellock.h>
+#include <htmintrin.h>
+#include <elision-conf.h>
+
+#define aconf __elision_aconf
+
+/* Try to elide a futex trylock.  FUTEX is the futex variable.  ADAPT_COUNT is
+   the adaptation counter in the mutex.  */
+
+int
+__lll_trylock_elision (int *futex, short *adapt_count)
+{
+  __asm__ volatile (".machinemode \"zarch_nohighgprs\"\n\t"
+		    ".machine \"all\""
+		    : : : "memory");
+
+  /* Implement POSIX semantics by forbiding nesting elided trylocks.
+     Sorry.  After the abort the code is re-executed
+     non transactional and if the lock was already locked
+     return an error.  */
+  if (__builtin_tx_nesting_depth () > 0)
+    {
+      /* Note that this abort may terminate an outermost transaction that
+	 was created outside glibc.
+	 This persistently aborts the current transactions to force
+	 them to use the default lock instead of retrying transactions
+	 until their try_tbegin is zero.
+      */
+      __builtin_tabort (_HTM_FIRST_USER_ABORT_CODE | 1);
+    }
+
+  /* Only try a transaction if it's worth it.  */
+  if (*adapt_count <= 0)
+    {
+      unsigned status;
+
+      if (__builtin_expect
+	  ((status = __builtin_tbegin ((void *)0)) == _HTM_TBEGIN_STARTED, 1))
+	{
+	  if (*futex == 0)
+	    return 0;
+	  /* Lock was busy.  Fall back to normal locking.  */
+	  /* Since we are in a non-nested transaction there is no need to abort,
+	     which is expensive.  */
+	  __builtin_tend ();
+	  /* Note: Changing the adapt_count here might abort a transaction on a
+	     different cpu, but that could happen anyway when the futex is
+	     acquired, so there's no need to check the nesting depth here.  */
+	  if (aconf.skip_lock_busy > 0)
+	    *adapt_count = aconf.skip_lock_busy;
+	}
+      else
+	{
+	  if (status != _HTM_TBEGIN_TRANSIENT)
+	    {
+	      /* A persistent abort (cc 1 or 3) indicates that a retry is
+		 probably futile.  Use the normal locking now and for the
+		 next couple of calls.
+		 Be careful to avoid writing to the lock.  */
+	      if (aconf.skip_trylock_internal_abort > 0)
+		*adapt_count = aconf.skip_trylock_internal_abort;
+	    }
+	}
+      /* Could do some retries here.  */
+    }
+  else
+    {
+      /* Lost updates are possible, but harmless.  Due to races this might lead
+	 to *adapt_count becoming less than zero.  */
+      (*adapt_count)--;
+    }
+
+  return lll_trylock (*futex);
+}
diff --git a/nptl/sysdeps/unix/sysv/linux/s390/elision-unlock.c b/nptl/sysdeps/unix/sysv/linux/s390/elision-unlock.c
new file mode 100644
index 0000000..9ceae3e
--- /dev/null
+++ b/nptl/sysdeps/unix/sysv/linux/s390/elision-unlock.c
@@ -0,0 +1,38 @@ 
+/* Commit an elided pthread lock.
+   Copyright (C) 2014 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <pthreadP.h>
+#include <lowlevellock.h>
+
+int
+__lll_unlock_elision(int *futex, int private)
+{
+  /* If the lock is free, we elided the lock earlier.  This does not
+     necessarily mean that we are in a transaction, because the user code may
+     have closed the transaction, but that is impossible to detect reliably.  */
+  if (*futex == 0)
+    {
+      __asm__ volatile (".machinemode \"zarch_nohighgprs\"\n\t"
+			".machine \"all\""
+			: : : "memory");
+      __builtin_tend();
+    }
+  else
+    lll_unlock ((*futex), private);
+  return 0;
+}
diff --git a/nptl/sysdeps/unix/sysv/linux/s390/force-elision.h b/nptl/sysdeps/unix/sysv/linux/s390/force-elision.h
new file mode 100644
index 0000000..8fd7684
--- /dev/null
+++ b/nptl/sysdeps/unix/sysv/linux/s390/force-elision.h
@@ -0,0 +1,33 @@ 
+/* Automatic enabling of elision for mutexes
+   Copyright (C) 2014 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#ifdef ENABLE_LOCK_ELISION
+/* Check for elision on this lock without upgrading.  */
+#define DO_ELISION(m)							\
+  (__pthread_force_elision						\
+   && (m->__data.__kind & PTHREAD_MUTEX_NO_ELISION_NP) == 0)		\
+
+/* Automatically enable elision for existing user lock kinds.  */
+#define FORCE_ELISION(m, s)						\
+  if (__pthread_force_elision						\
+      && (m->__data.__kind & PTHREAD_MUTEX_ELISION_FLAGS_NP) == 0)	\
+    {									\
+      mutex->__data.__kind |= PTHREAD_MUTEX_ELISION_NP;			\
+      s;								\
+    }
+#endif
diff --git a/nptl/sysdeps/unix/sysv/linux/s390/lowlevellock.h b/nptl/sysdeps/unix/sysv/linux/s390/lowlevellock.h
index 864dcbc..cabff30 100644
--- a/nptl/sysdeps/unix/sysv/linux/s390/lowlevellock.h
+++ b/nptl/sysdeps/unix/sysv/linux/s390/lowlevellock.h
@@ -285,6 +285,15 @@  __lll_timedlock (int *futex, const struct timespec *abstime, int private)
 #define lll_timedlock(futex, abstime, private) \
   __lll_timedlock (&(futex), abstime, private)
 
+#ifdef ENABLE_LOCK_ELISION
+extern int __lll_timedlock_elision
+  (int *futex, short *adapt_count, const struct timespec *timeout, int private)
+  attribute_hidden;
+
+# define lll_timedlock_elision(futex, adapt_count, timeout, private)	\
+  __lll_timedlock_elision(&(futex), &(adapt_count), timeout, private)
+#endif
+
 static inline int
 __attribute__ ((always_inline))
 __lll_robust_timedlock (int *futex, const struct timespec *abstime,
@@ -360,4 +369,22 @@  extern int __lll_timedwait_tid (int *, const struct timespec *)
     __res;								      \
   })
 
+#ifdef ENABLE_LOCK_ELISION
+extern int __lll_lock_elision (int *futex, short *adapt_count, int private)
+  attribute_hidden;
+
+extern int __lll_unlock_elision(int *futex, int private)
+  attribute_hidden;
+
+extern int __lll_trylock_elision(int *futex, short *adapt_count)
+  attribute_hidden;
+
+# define lll_lock_elision(futex, adapt_count, private) \
+  __lll_lock_elision (&(futex), &(adapt_count), private)
+# define lll_unlock_elision(futex, private) \
+  __lll_unlock_elision (&(futex), private)
+# define lll_trylock_elision(futex, adapt_count) \
+  __lll_trylock_elision(&(futex), &(adapt_count))
+#endif
+
 #endif	/* lowlevellock.h */
diff --git a/nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_cond_lock.c b/nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_cond_lock.c
new file mode 100644
index 0000000..6fc0f96
--- /dev/null
+++ b/nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_cond_lock.c
@@ -0,0 +1,22 @@ 
+/* Copyright (C) 2014 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+/* The cond lock is not actually elided yet, but we still need to handle
+   already elided locks.  */
+#include <elision-conf.h>
+
+#include <nptl/sysdeps/unix/sysv/linux/pthread_mutex_cond_lock.c>
diff --git a/nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_lock.c b/nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_lock.c
new file mode 100644
index 0000000..6fd6a98
--- /dev/null
+++ b/nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_lock.c
@@ -0,0 +1,22 @@ 
+/* Elided version of pthread_mutex_lock.
+   Copyright (C) 2014 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <elision-conf.h>
+#include <force-elision.h>
+
+#include <nptl/pthread_mutex_lock.c>
diff --git a/nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_timedlock.c b/nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_timedlock.c
new file mode 100644
index 0000000..d0e6537
--- /dev/null
+++ b/nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_timedlock.c
@@ -0,0 +1,22 @@ 
+/* Elided version of pthread_mutex_timedlock.
+   Copyright (C) 2014 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <elision-conf.h>
+#include <force-elision.h>
+
+#include <nptl/pthread_mutex_timedlock.c>
diff --git a/nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_trylock.c b/nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_trylock.c
new file mode 100644
index 0000000..ea8a8ff
--- /dev/null
+++ b/nptl/sysdeps/unix/sysv/linux/s390/pthread_mutex_trylock.c
@@ -0,0 +1,22 @@ 
+/* Elided version of pthread_mutex_trylock.
+   Copyright (C) 2014 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <elision-conf.h>
+#include <force-elision.h>
+
+#include <nptl/pthread_mutex_trylock.c>
diff --git a/sysdeps/s390/configure b/sysdeps/s390/configure
index c2d05f7..6948cc2 100644
--- a/sysdeps/s390/configure
+++ b/sysdeps/s390/configure
@@ -68,5 +68,41 @@  if test $ac_verc_fail = yes; then
 fi
 
 
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for __builtin_tbegin" >&5
+$as_echo_n "checking for __builtin_tbegin... " >&6; }
+if ${libc_cv_gcc_builtin_tbegin+:} false; then :
+  $as_echo_n "(cached) " >&6
+else
+  cat > conftest.c <<\EOF
+#include <htmintrin.h>
+void testtransaction ()
+{
+  if (__builtin_tbegin (0) == _HTM_TBEGIN_STARTED)
+    {
+      __builtin_tend ();
+    }
+}
+EOF
+if { ac_try='${CC-cc} -mhtm -O2 -S conftest.c -o - | grep -w tbegin > /dev/null'
+  { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_try\""; } >&5
+  (eval $ac_try) 2>&5
+  ac_status=$?
+  $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+  test $ac_status = 0; }; } ;
+then
+  libc_cv_gcc_builtin_tbegin=yes
+else
+  libc_cv_gcc_builtin_tbegin=no
+fi
+rm -f conftest*
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_gcc_builtin_tbegin" >&5
+$as_echo "$libc_cv_gcc_builtin_tbegin" >&6; }
+
+if test "$enable_lock_elision" = yes && test "$libc_cv_gcc_builtin_tbegin" = no ; then
+   critic_missing="$critic_missing The used GCC has no support for __builtin_tbegin, which is needed for lock-elision on target S390."
+fi
+
 test -n "$critic_missing" && as_fn_error $? "
 *** $critic_missing" "$LINENO" 5
diff --git a/sysdeps/s390/configure.ac b/sysdeps/s390/configure.ac
index 59cfdd1..493e9a4 100644
--- a/sysdeps/s390/configure.ac
+++ b/sysdeps/s390/configure.ac
@@ -10,5 +10,31 @@  AC_CHECK_PROG_VER(AS, $AS, --version,
 		  [GNU assembler.* \([0-9]*\.[0-9.]*\)],
 		  [2.2[4-9]*|2.[3-9][0-9]*|[3-9].*|[1-9][0-9]*], critic_missing="$critic_missing The program AS is required in version >= 2.24 for target S390.")
 
+
+AC_CACHE_CHECK(for __builtin_tbegin, libc_cv_gcc_builtin_tbegin, [dnl
+cat > conftest.c <<\EOF
+#include <htmintrin.h>
+void testtransaction ()
+{
+  if (__builtin_tbegin (0) == _HTM_TBEGIN_STARTED)
+    {
+      __builtin_tend ();
+    }
+}
+EOF
+dnl
+dnl test, if the tbegin instruction is used by __builtin_tbegin
+if AC_TRY_COMMAND([${CC-cc} -mhtm -O2 -S conftest.c -o - | grep -w tbegin > /dev/null]) ;
+then
+  libc_cv_gcc_builtin_tbegin=yes
+else
+  libc_cv_gcc_builtin_tbegin=no
+fi
+rm -f conftest* ])
+
+if test "$enable_lock_elision" = yes && test "$libc_cv_gcc_builtin_tbegin" = no ; then
+   critic_missing="$critic_missing The used GCC has no support for __builtin_tbegin, which is needed for lock-elision on target S390."
+fi
+
 test -n "$critic_missing" && AC_MSG_ERROR([
 *** $critic_missing])