From patchwork Thu Feb 1 20:57:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 868436 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=sourceware.org (client-ip=209.132.180.131; helo=sourceware.org; envelope-from=libc-alpha-return-89903-incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.b="GJXi/xo+"; dkim-atps=neutral Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zXXX600Zrz9ryr for ; Fri, 2 Feb 2018 07:58:29 +1100 (AEDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:subject:date:message-id:in-reply-to :references; q=dns; s=default; b=vaBYVwVgFxysnuwWpkZ9gIf5gVIy430 eJYXXlbDYx7kCzQoG8S9jxlH3KtdL3Fm6ndAavuk7bjnsL3BV9GBqLSGSdwqI6MZ xb/b9xqZEuwLWyTOHX5DlP0bGvjJS8udvpEF2nbm0nRmtENLlJhZ7SVNeOeVesPQ 1s4BlxJCvF/M= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:subject:date:message-id:in-reply-to :references; s=default; bh=JrMF+ky1UxOs8DqQ797VGJHdITY=; b=GJXi/ xo+7qhCZT1bhvnSsN9mm3ZxKFgL2hzJ6+vnWqceV/Wv4r2W66/e8+6xMQsYsCpdp OsNnQnrUCFOt5j4yIJFfxS3x5SqTQPZLc+OEwUvFfmtMluacWdfzxiesw1poAZmX 322wGTz0FvusDNEYP8Ol7avHtTlovYLJ9Jhpac= Received: (qmail 9156 invoked by alias); 1 Feb 2018 20:58:06 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 9014 invoked by uid 89); 1 Feb 2018 20:58:05 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-26.2 required=5.0 tests=BAYES_00, FREEMAIL_FROM, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, SPF_SOFTFAIL autolearn=ham version=3.3.2 spammy=installment, cancellation X-HELO: mga09.intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 From: "H.J. Lu" To: libc-alpha@sourceware.org Subject: [PATCH 2/2] nptl: Update struct pthread_unwind_buf [BZ #22743] Date: Thu, 1 Feb 2018 12:57:57 -0800 Message-Id: <20180201205757.51911-3-hjl.tools@gmail.com> In-Reply-To: <20180201205757.51911-1-hjl.tools@gmail.com> References: <20180201205757.51911-1-hjl.tools@gmail.com> In glibc 2.28, the size of cancel_jmp_buf in struct pthread_unwind_buf has been increased to match the size of __jmp_buf_tag on Linux/x86 in order to save and restore shadow stack register. pthread_unwind_buf is used in , whose address is passed from applications to libpthread. To access the private data in struct pthread_unwind_buf, which is placed after cancel_jmp_buf, in libpthread, we must know which struct pthread_unwind_buf, before glibc 28 and after glibc 2.28, is used in caller. If the size of caller's struct pthread_unwind_buf is smaller than what libpthread expects, libpthread will override caller's stack since struct pthread_unwind_buf is placed on caller's stack. We enable shadow stack at run-time only if program and all used shared objects, including dlopened ones, are shadow stack enabled, which means that they must be compiled with GCC 8 or above and glibc 2.28 or above. Since we need to save and restore shadow stack register only if shadow stack is enabled, we can safely assume that caller is compiled with smaller struct pthread_unwind_buf on stack if shadow stack isn't enabled at run-time. For callers with larger struct pthread_unwind_buf, but shadow stack isn't enabled, we just have some unused space on caller's stack. struct pthread_unwind_buf is changed to union of 1. struct cancel_jmp_buf[1], which contains the common fields of struct full and struct compat_pthread_unwind_buf. 2. struct full_pthread_unwind_buf, which is the full layout of the cleanup buffer. 3. struct compat_pthread_unwind_buf, which is the compatible layout of the cleanup buffer. A macro, UNWIND_BUF_PRIV, is added to get the pointer to the priv field. By default, it uses the priv field of struct compat_pthread_unwind_buf. If a target defines NEED_SAVED_MASK_IN_CANCEL_JMP_BUF, it must provide its own version of UNEIND_BUF_PRIV to get the pointer to the priv field. On Linux/x86, it uses the priv field of struct compat_pthread_unwind_buf if shadow stack is disabled and struct full_pthread_unwind_buf if shadow stack is enabled. The overhead of in __pthread_register_cancel on x86-64 is: Without UNWIND_BUF_PRIV: movq %fs:768,%rax movq %rax, 72(%rdi) movq %fs:760,%rax movq %rax, 80(%rdi) movq %rdi,%fs:768 ret With UNWIND_BUF_PRIV: movl %fs:76,%ecx leaq 72(%rdi), %rdx leaq 200(%rdi), %rax andl $2, %ecx cmove %rdx, %rax movq %fs:768,%rdx movq %rdx, (%rax) movq %fs:760,%rdx movq %rdx, 8(%rax) movq %rdi,%fs:768 ret Note: There is an unused pointer space in pthread_unwind_buf_data. But it isn't suitable for saving and restoring shadow stack register since x32 is a 64-bit process with 32-bit software pointer and kernel may place x32 shadow stack above 4GB. We need to save and restore 64-bit shadow stack register for x32. [BZ #22743] * csu/libc-start.c (LIBC_START_MAIN): Use the full version of the cleanup buffer. * nptl/cleanup.c (__pthread_register_cancel): Use UNWIND_BUF_PRIV to access the priv field in the cleanup buffer. (__pthread_unregister_cancel): Likewise. * nptl/cleanup_defer.c (__pthread_register_cancel_defer): Likewise. (__pthread_unregister_cancel_restore): Likewise. * nptl/unwind.c (unwind_stop): Likewise. (__pthread_unwind_next): Likewise. * nptl/descr.h (pthread_unwind_buf_data): New. (full_pthread_unwind_buf): Likewise. (compat_pthread_unwind_buf): Likewise. (pthread_unwind_buf): Updated to use full_pthread_unwind_buf and compat_pthread_unwind_buf. (UNWIND_BUF_PRIV): New. Macro to get pointer to the priv field in the cleanup buffer. * nptl/pthread_create.c (START_THREAD_DEFN): Use the full version of the cleanup buffer. (__pthread_create_2_1): Use THREAD_COPY_ADDITONAL_INFO to copy additonal info if defined. * sysdeps/unix/sysv/linux/x86/nptl/pthreadP.h: Use the full version of the cleanup buffer to check cancel_jmp_buf size. * sysdeps/unix/sysv/linux/x86/pthreaddef.h (THREAD_COPY_ADDITONAL_INFO): New. (UNWIND_BUF_PRIV): Likewise. --- csu/libc-start.c | 6 ++- nptl/cleanup.c | 9 ++-- nptl/cleanup_defer.c | 16 +++--- nptl/descr.h | 75 ++++++++++++++++++++++------- nptl/pthread_create.c | 9 +++- nptl/unwind.c | 6 ++- sysdeps/unix/sysv/linux/x86/nptl/pthreadP.h | 2 +- sysdeps/unix/sysv/linux/x86/pthreaddef.h | 14 ++++++ 8 files changed, 103 insertions(+), 34 deletions(-) diff --git a/csu/libc-start.c b/csu/libc-start.c index 605222fa3f..c6bbc97ef0 100644 --- a/csu/libc-start.c +++ b/csu/libc-start.c @@ -298,8 +298,10 @@ LIBC_START_MAIN (int (*main) (int, char **, char ** MAIN_AUXVEC_DECL), struct pthread *self = THREAD_SELF; /* Store old info. */ - unwind_buf.priv.data.prev = THREAD_GETMEM (self, cleanup_jmp_buf); - unwind_buf.priv.data.cleanup = THREAD_GETMEM (self, cleanup); + unwind_buf.full.priv.data.prev + = THREAD_GETMEM (self, cleanup_jmp_buf); + unwind_buf.full.priv.data.cleanup + = THREAD_GETMEM (self, cleanup); /* Store the new cleanup handler info. */ THREAD_SETMEM (self, cleanup_jmp_buf, &unwind_buf); diff --git a/nptl/cleanup.c b/nptl/cleanup.c index d21b86e88b..6403a42d46 100644 --- a/nptl/cleanup.c +++ b/nptl/cleanup.c @@ -28,8 +28,9 @@ __pthread_register_cancel (__pthread_unwind_buf_t *buf) struct pthread *self = THREAD_SELF; /* Store old info. */ - ibuf->priv.data.prev = THREAD_GETMEM (self, cleanup_jmp_buf); - ibuf->priv.data.cleanup = THREAD_GETMEM (self, cleanup); + union pthread_unwind_buf_data *priv = UNWIND_BUF_PRIV (self, ibuf); + priv->data.prev = THREAD_GETMEM (self, cleanup_jmp_buf); + priv->data.cleanup = THREAD_GETMEM (self, cleanup); /* Store the new cleanup handler info. */ THREAD_SETMEM (self, cleanup_jmp_buf, (struct pthread_unwind_buf *) buf); @@ -42,7 +43,9 @@ __cleanup_fct_attribute __pthread_unregister_cancel (__pthread_unwind_buf_t *buf) { struct pthread_unwind_buf *ibuf = (struct pthread_unwind_buf *) buf; + struct pthread *self = THREAD_SELF; - THREAD_SETMEM (THREAD_SELF, cleanup_jmp_buf, ibuf->priv.data.prev); + THREAD_SETMEM (self, cleanup_jmp_buf, + UNWIND_BUF_PRIV (self, ibuf)->data.prev); } hidden_def (__pthread_unregister_cancel) diff --git a/nptl/cleanup_defer.c b/nptl/cleanup_defer.c index 5701ce4213..fddf7434db 100644 --- a/nptl/cleanup_defer.c +++ b/nptl/cleanup_defer.c @@ -28,8 +28,9 @@ __pthread_register_cancel_defer (__pthread_unwind_buf_t *buf) struct pthread *self = THREAD_SELF; /* Store old info. */ - ibuf->priv.data.prev = THREAD_GETMEM (self, cleanup_jmp_buf); - ibuf->priv.data.cleanup = THREAD_GETMEM (self, cleanup); + union pthread_unwind_buf_data *priv = UNWIND_BUF_PRIV (self, ibuf); + priv->data.prev = THREAD_GETMEM (self, cleanup_jmp_buf); + priv->data.cleanup = THREAD_GETMEM (self, cleanup); int cancelhandling = THREAD_GETMEM (self, cancelhandling); @@ -49,9 +50,9 @@ __pthread_register_cancel_defer (__pthread_unwind_buf_t *buf) cancelhandling = curval; } - ibuf->priv.data.canceltype = (cancelhandling & CANCELTYPE_BITMASK - ? PTHREAD_CANCEL_ASYNCHRONOUS - : PTHREAD_CANCEL_DEFERRED); + priv->data.canceltype = (cancelhandling & CANCELTYPE_BITMASK + ? PTHREAD_CANCEL_ASYNCHRONOUS + : PTHREAD_CANCEL_DEFERRED); /* Store the new cleanup handler info. */ THREAD_SETMEM (self, cleanup_jmp_buf, (struct pthread_unwind_buf *) buf); @@ -64,11 +65,12 @@ __pthread_unregister_cancel_restore (__pthread_unwind_buf_t *buf) { struct pthread *self = THREAD_SELF; struct pthread_unwind_buf *ibuf = (struct pthread_unwind_buf *) buf; + union pthread_unwind_buf_data *priv = UNWIND_BUF_PRIV (self, ibuf); - THREAD_SETMEM (self, cleanup_jmp_buf, ibuf->priv.data.prev); + THREAD_SETMEM (self, cleanup_jmp_buf, priv->data.prev); int cancelhandling; - if (ibuf->priv.data.canceltype != PTHREAD_CANCEL_DEFERRED + if (priv->data.canceltype != PTHREAD_CANCEL_DEFERRED && ((cancelhandling = THREAD_GETMEM (self, cancelhandling)) & CANCELTYPE_BITMASK) == 0) { diff --git a/nptl/descr.h b/nptl/descr.h index 1cc6b09d1e..662696dca6 100644 --- a/nptl/descr.h +++ b/nptl/descr.h @@ -55,11 +55,30 @@ / PTHREAD_KEY_2NDLEVEL_SIZE) +/* Private data in the cleanup buffer. */ +union pthread_unwind_buf_data +{ + /* This is the placeholder of the public version. */ + void *pad[4]; + struct + { + /* Pointer to the previous cleanup buffer. */ + struct pthread_unwind_buf *prev; -/* Internal version of the buffer to store cancellation handler + /* Backward compatibility: state of the old-style cleanup + handler at the time of the previous new-style cleanup handler + installment. */ + struct _pthread_cleanup_buffer *cleanup; + + /* Cancellation type before the push call. */ + int canceltype; + } data; +}; + +/* Internal full version of the buffer to store cancellation handler information. */ -struct pthread_unwind_buf +struct full_pthread_unwind_buf { struct { @@ -70,27 +89,49 @@ struct pthread_unwind_buf #endif } cancel_jmp_buf[1]; - union + union pthread_unwind_buf_data priv; +}; + +/* Internal compatible version of the buffer to store cancellation + handler information. */ +struct compat_pthread_unwind_buf +{ + struct { - /* This is the placeholder of the public version. */ - void *pad[4]; + __jmp_buf jmp_buf; + int mask_was_saved; + } cancel_jmp_buf[1]; + + union pthread_unwind_buf_data priv; +}; +/* Internal version of the buffer to store cancellation handler + information. */ +struct pthread_unwind_buf +{ + union + { + /* The common fields of full and compatible versions. */ struct { - /* Pointer to the previous cleanup buffer. */ - struct pthread_unwind_buf *prev; - - /* Backward compatibility: state of the old-style cleanup - handler at the time of the previous new-style cleanup handler - installment. */ - struct _pthread_cleanup_buffer *cleanup; - - /* Cancellation type before the push call. */ - int canceltype; - } data; - } priv; + __jmp_buf jmp_buf; + int mask_was_saved; + } cancel_jmp_buf[1]; + struct full_pthread_unwind_buf full; + struct compat_pthread_unwind_buf compat; + }; }; +/* Get pointer to the priv field from THREAD_SELF, "self", and pointer + to the cleanup buffer, "p". By default, the compatible version is + used. If a target defines NEED_SAVED_MASK_IN_CANCEL_JMP_BUF, it + must provide its own version of UNEIND_BUF_PRIV. */ +#ifndef UNWIND_BUF_PRIV +# ifdef NEED_SAVED_MASK_IN_CANCEL_JMP_BUF +# error "UNWIND_BUF_PRIV is undefined!" +# endif +# define UNWIND_BUF_PRIV(self,p) (&((p)->compat.priv)) +#endif /* Opcodes and data types for communication with the signal handler to change user/group IDs. */ diff --git a/nptl/pthread_create.c b/nptl/pthread_create.c index caaf07c134..082615e080 100644 --- a/nptl/pthread_create.c +++ b/nptl/pthread_create.c @@ -428,8 +428,8 @@ START_THREAD_DEFN struct pthread_unwind_buf unwind_buf; /* No previous handlers. */ - unwind_buf.priv.data.prev = NULL; - unwind_buf.priv.data.cleanup = NULL; + unwind_buf.full.priv.data.prev = NULL; + unwind_buf.full.priv.data.cleanup = NULL; int not_first_call; not_first_call = setjmp ((struct __jmp_buf_tag *) unwind_buf.cancel_jmp_buf); @@ -701,6 +701,11 @@ __pthread_create_2_1 (pthread_t *newthread, const pthread_attr_t *attr, THREAD_COPY_POINTER_GUARD (pd); #endif + /* Copy additonal info. */ +#ifdef THREAD_COPY_ADDITONAL_INFO + THREAD_COPY_ADDITONAL_INFO (pd); +#endif + /* Verify the sysinfo bits were copied in allocate_stack if needed. */ #ifdef NEED_DL_SYSINFO CHECK_THREAD_SYSINFO (pd); diff --git a/nptl/unwind.c b/nptl/unwind.c index b37a063c53..f58be0ee5f 100644 --- a/nptl/unwind.c +++ b/nptl/unwind.c @@ -66,7 +66,8 @@ unwind_stop (int version, _Unwind_Action actions, /* Handle the compatibility stuff. Execute all handlers registered with the old method which would be unwound by this step. */ - struct _pthread_cleanup_buffer *oldp = buf->priv.data.cleanup; + struct _pthread_cleanup_buffer *oldp + = UNWIND_BUF_PRIV (self, buf)->data.cleanup; void *cfa = (void *) (_Unwind_Ptr) _Unwind_GetCFA (context); if (curp != oldp && (do_longjump || FRAME_LEFT (cfa, curp, adj))) @@ -133,6 +134,7 @@ __pthread_unwind_next (__pthread_unwind_buf_t *buf) { struct pthread_unwind_buf *ibuf = (struct pthread_unwind_buf *) buf; - __pthread_unwind ((__pthread_unwind_buf_t *) ibuf->priv.data.prev); + __pthread_unwind ((__pthread_unwind_buf_t *) + UNWIND_BUF_PRIV (THREAD_SELF, ibuf)->data.prev); } hidden_def (__pthread_unwind_next) diff --git a/sysdeps/unix/sysv/linux/x86/nptl/pthreadP.h b/sysdeps/unix/sysv/linux/x86/nptl/pthreadP.h index 247a62e9a0..ff9ea4cb6d 100644 --- a/sysdeps/unix/sysv/linux/x86/nptl/pthreadP.h +++ b/sysdeps/unix/sysv/linux/x86/nptl/pthreadP.h @@ -23,7 +23,7 @@ extern struct pthread_unwind_buf ____pthread_unwind_buf_private; -_Static_assert (sizeof (____pthread_unwind_buf_private.cancel_jmp_buf) +_Static_assert (sizeof (____pthread_unwind_buf_private.full.cancel_jmp_buf) >= sizeof (struct __jmp_buf_tag), "size of cancel_jmp_buf < sizeof __jmp_buf_tag"); diff --git a/sysdeps/unix/sysv/linux/x86/pthreaddef.h b/sysdeps/unix/sysv/linux/x86/pthreaddef.h index a405a65666..52198aef73 100644 --- a/sysdeps/unix/sysv/linux/x86/pthreaddef.h +++ b/sysdeps/unix/sysv/linux/x86/pthreaddef.h @@ -20,3 +20,17 @@ /* Need saved_mask in cancel_jmp_buf. */ #define NEED_SAVED_MASK_IN_CANCEL_JMP_BUF 1 + +/* Wee need to copy feature_1 in pthread_create. */ +#define THREAD_COPY_ADDITONAL_INFO(descr) \ + ((descr)->header.feature_1 \ + = THREAD_GETMEM (THREAD_SELF, header.feature_1)) + +/* Use the compatible struct __cancel_jmp_buf_tag if shadow stack is + disabled. */ +#undef UNWIND_BUF_PRIV +#define UNWIND_BUF_PRIV(self,p) \ + (__extension__ ({ \ + unsigned int feature_1 = THREAD_GETMEM (self, header.feature_1); \ + (((feature_1 & (1 << 1)) == 0) \ + ? &((p)->compat.priv) : &((p)->full.priv));}))