From patchwork Thu May 27 17:28:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 1484745 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=libc-alpha-bounces@sourceware.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=mMMCt/Za; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FrZYZ5vwVz9sT6 for ; Fri, 28 May 2021 03:29:06 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A1CB63959E51; Thu, 27 May 2021 17:28:41 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org A1CB63959E51 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1622136521; bh=rNsA27gX/Dihomfg8JnF2KDQih+KbVkRjDaSFhdvTQA=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=mMMCt/ZanAs43BOD1dxjolsuUEGKKn267W3iusy04jekwziYt4/VZdzVOalneyOS0 1zd8W4jXD7JtCiUPt6pVlnhTb4FAsxMmokwxgmlkpFkKylOFMbE+meteqhr47psRGt RCsyQ+ioa8Fal6kDH6lulT/kldHKQgbUf+O8DRnk= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qt1-x82b.google.com (mail-qt1-x82b.google.com [IPv6:2607:f8b0:4864:20::82b]) by sourceware.org (Postfix) with ESMTPS id 1D3B3383800A for ; Thu, 27 May 2021 17:28:37 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 1D3B3383800A Received: by mail-qt1-x82b.google.com with SMTP id a15so826567qta.0 for ; Thu, 27 May 2021 10:28:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rNsA27gX/Dihomfg8JnF2KDQih+KbVkRjDaSFhdvTQA=; b=Hw/a751Glx6+qtTT7fedBP6LcIc/kDEExK+fwXvUyir03D8hzkruNbKf0SyFkoWCNw vsQI6LQs2fOk+YJOnAFDUujz5zDIA+x+nUqb6H6jVUxY9LTI6+urhOMiC/Wwaej92rNs s1ptYCMD/tLF2ThStcTcKNevA2Jri60J6eCp74BSpZt3PtJPhSZdCp76MIQCXrsOB3Lm mFZ/n5s8kS1xuV1JSeS5GgJmzsS89LML9MfwVCdD2egK5z5TP8Yq8a/YAStDTK1HyfkR g6dp6c67+PqqZnGplwwG3w4/maGE6oWn5s1h9HPtr+v/TBljrMgzB9P/8JU9hBA+HiVY hMOA== X-Gm-Message-State: AOAM532VYGkLDXTSVKB49EFmGepI7vK+CPtj5vmiUmwLw8qboAckKCZj pIoSV4DNO32Vb8+lZIjpFyojfhI1uecTPg== X-Google-Smtp-Source: ABdhPJwS7kpfFL+Cdlq7NdK4pj1CQmujs1bd0gt6x2GRBDCJWIZ8cVWEFXXBpI9s26Nmhb7MfDFdWA== X-Received: by 2002:ac8:1342:: with SMTP id f2mr4180659qtj.148.1622136515552; Thu, 27 May 2021 10:28:35 -0700 (PDT) Received: from birita.. ([177.194.37.86]) by smtp.googlemail.com with ESMTPSA id 25sm1767913qtd.51.2021.05.27.10.28.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 10:28:35 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v2 6/9] nptl: Move cancel type out of cancelhandling Date: Thu, 27 May 2021 14:28:20 -0300 Message-Id: <20210527172823.3461314-7-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210527172823.3461314-1-adhemerval.zanella@linaro.org> References: <20210527172823.3461314-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_ASCII_DIVIDERS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" Now that the thread cancellation type is not accessed concurrently anymore, it is possible to move it out the cancelhandling. By removing the cancel state out of the internal thread cancel handling state there is no need to check if cancelled bit was set in CAS operation. It allows simplifing the cancellation wrappers and the CANCEL_CANCELED_AND_ASYNCHRONOUS is removed. Checked on x86_64-linux-gnu and aarch64-linux-gnu. --- nptl/allocatestack.c | 1 + nptl/cancellation.c | 51 +++++++++-------------------------- nptl/cleanup_defer.c | 46 ++++--------------------------- nptl/descr.h | 14 +++------- nptl/libc-cleanup.c | 44 +++--------------------------- nptl/pthread_cancel.c | 4 +-- nptl/pthread_setcanceltype.c | 42 ++++------------------------- sysdeps/nptl/dl-tls_init_tp.c | 1 + 8 files changed, 34 insertions(+), 169 deletions(-) diff --git a/nptl/allocatestack.c b/nptl/allocatestack.c index 54e95baad7..9be6c42894 100644 --- a/nptl/allocatestack.c +++ b/nptl/allocatestack.c @@ -161,6 +161,7 @@ get_cached_stack (size_t *sizep, void **memp) /* Cancellation handling is back to the default. */ result->cancelhandling = 0; result->cancelstate = PTHREAD_CANCEL_ENABLE; + result->canceltype = PTHREAD_CANCEL_DEFERRED; result->cleanup = NULL; result->setup_failed = 0; diff --git a/nptl/cancellation.c b/nptl/cancellation.c index ce00603109..05962784d5 100644 --- a/nptl/cancellation.c +++ b/nptl/cancellation.c @@ -31,31 +31,19 @@ int __pthread_enable_asynccancel (void) { struct pthread *self = THREAD_SELF; - int oldval = THREAD_GETMEM (self, cancelhandling); - while (1) - { - int newval = oldval | CANCELTYPE_BITMASK; - - if (newval == oldval) - break; - - int curval = THREAD_ATOMIC_CMPXCHG_VAL (self, cancelhandling, newval, - oldval); - if (__glibc_likely (curval == oldval)) - { - if (self->cancelstate == PTHREAD_CANCEL_ENABLE - && CANCEL_CANCELED_AND_ASYNCHRONOUS (newval)) - { - THREAD_SETMEM (self, result, PTHREAD_CANCELED); - __do_cancel (); - } + int oldval = THREAD_GETMEM (self, canceltype); + THREAD_SETMEM (self, canceltype, PTHREAD_CANCEL_ASYNCHRONOUS); - break; - } + int ch = THREAD_GETMEM (self, cancelhandling); - /* Prepare the next round. */ - oldval = curval; + if (self->cancelstate == PTHREAD_CANCEL_ENABLE + && (ch & CANCELED_BITMASK) + && !(ch & EXITING_BITMASK) + && !(ch & TERMINATED_BITMASK)) + { + THREAD_SETMEM (self, result, PTHREAD_CANCELED); + __do_cancel (); } return oldval; @@ -69,25 +57,10 @@ __pthread_disable_asynccancel (int oldtype) { /* If asynchronous cancellation was enabled before we do not have anything to do. */ - if (oldtype & CANCELTYPE_BITMASK) + if (oldtype == PTHREAD_CANCEL_ASYNCHRONOUS) return; struct pthread *self = THREAD_SELF; - int newval; - - int oldval = THREAD_GETMEM (self, cancelhandling); - - while (1) - { - newval = oldval & ~CANCELTYPE_BITMASK; - - int curval = THREAD_ATOMIC_CMPXCHG_VAL (self, cancelhandling, newval, - oldval); - if (__glibc_likely (curval == oldval)) - break; - - /* Prepare the next round. */ - oldval = curval; - } + self->canceltype = PTHREAD_CANCEL_DEFERRED; } libc_hidden_def (__pthread_disable_asynccancel) diff --git a/nptl/cleanup_defer.c b/nptl/cleanup_defer.c index 873ab007fd..7e858d0df0 100644 --- a/nptl/cleanup_defer.c +++ b/nptl/cleanup_defer.c @@ -31,27 +31,9 @@ ___pthread_register_cancel_defer (__pthread_unwind_buf_t *buf) ibuf->priv.data.prev = THREAD_GETMEM (self, cleanup_jmp_buf); ibuf->priv.data.cleanup = THREAD_GETMEM (self, cleanup); - int cancelhandling = THREAD_GETMEM (self, cancelhandling); - /* Disable asynchronous cancellation for now. */ - if (__glibc_unlikely (cancelhandling & CANCELTYPE_BITMASK)) - while (1) - { - int curval = THREAD_ATOMIC_CMPXCHG_VAL (self, cancelhandling, - cancelhandling - & ~CANCELTYPE_BITMASK, - cancelhandling); - if (__glibc_likely (curval == cancelhandling)) - /* Successfully replaced the value. */ - break; - - /* Prepare for the next round. */ - cancelhandling = curval; - } - - ibuf->priv.data.canceltype = (cancelhandling & CANCELTYPE_BITMASK - ? PTHREAD_CANCEL_ASYNCHRONOUS - : PTHREAD_CANCEL_DEFERRED); + ibuf->priv.data.canceltype = THREAD_GETMEM (self, canceltype); + THREAD_SETMEM (self, canceltype, PTHREAD_CANCEL_DEFERRED); /* Store the new cleanup handler info. */ THREAD_SETMEM (self, cleanup_jmp_buf, (struct pthread_unwind_buf *) buf); @@ -73,27 +55,9 @@ ___pthread_unregister_cancel_restore (__pthread_unwind_buf_t *buf) THREAD_SETMEM (self, cleanup_jmp_buf, ibuf->priv.data.prev); - int cancelhandling; - if (ibuf->priv.data.canceltype != PTHREAD_CANCEL_DEFERRED - && ((cancelhandling = THREAD_GETMEM (self, cancelhandling)) - & CANCELTYPE_BITMASK) == 0) - { - while (1) - { - int curval = THREAD_ATOMIC_CMPXCHG_VAL (self, cancelhandling, - cancelhandling - | CANCELTYPE_BITMASK, - cancelhandling); - if (__glibc_likely (curval == cancelhandling)) - /* Successfully replaced the value. */ - break; - - /* Prepare for the next round. */ - cancelhandling = curval; - } - - __pthread_testcancel (); - } + THREAD_SETMEM (self, canceltype, ibuf->priv.data.canceltype); + if (ibuf->priv.data.canceltype == PTHREAD_CANCEL_ASYNCHRONOUS) + __pthread_testcancel (); } versioned_symbol (libc, ___pthread_unregister_cancel_restore, __pthread_unregister_cancel_restore, GLIBC_2_34); diff --git a/nptl/descr.h b/nptl/descr.h index 35f5330e7f..c85778d449 100644 --- a/nptl/descr.h +++ b/nptl/descr.h @@ -277,9 +277,6 @@ struct pthread /* Flags determining processing of cancellation. */ int cancelhandling; - /* Bit set if asynchronous cancellation mode is selected. */ -#define CANCELTYPE_BIT 1 -#define CANCELTYPE_BITMASK (0x01 << CANCELTYPE_BIT) /* Bit set if canceled. */ #define CANCELED_BIT 3 #define CANCELED_BITMASK (0x01 << CANCELED_BIT) @@ -292,13 +289,6 @@ struct pthread /* Bit set if thread is supposed to change XID. */ #define SETXID_BIT 6 #define SETXID_BITMASK (0x01 << SETXID_BIT) - /* Mask for the rest. Helps the compiler to optimize. */ -#define CANCEL_RESTMASK 0xffffff80 - -#define CANCEL_CANCELED_AND_ASYNCHRONOUS(value) \ - (((value) & (CANCELTYPE_BITMASK | CANCELED_BITMASK \ - | EXITING_BITMASK | CANCEL_RESTMASK | TERMINATED_BITMASK)) \ - == (CANCELTYPE_BITMASK | CANCELED_BITMASK)) /* Flags. Including those copied from the thread attribute. */ int flags; @@ -402,6 +392,10 @@ struct pthread PTHREAD_CANCEL_DISABLE). */ unsigned char cancelstate; + /* Thread cancel type (PTHREAD_CANCEL_DEFERRED or + PTHREAD_CANCEL_ASYNCHRONOUS). */ + unsigned char canceltype; + /* Used on strsignal. */ struct tls_internal_t tls_state; diff --git a/nptl/libc-cleanup.c b/nptl/libc-cleanup.c index 6286b8b525..180d15bc9e 100644 --- a/nptl/libc-cleanup.c +++ b/nptl/libc-cleanup.c @@ -27,27 +27,9 @@ __libc_cleanup_push_defer (struct _pthread_cleanup_buffer *buffer) buffer->__prev = THREAD_GETMEM (self, cleanup); - int cancelhandling = THREAD_GETMEM (self, cancelhandling); - /* Disable asynchronous cancellation for now. */ - if (__glibc_unlikely (cancelhandling & CANCELTYPE_BITMASK)) - while (1) - { - int curval = THREAD_ATOMIC_CMPXCHG_VAL (self, cancelhandling, - cancelhandling - & ~CANCELTYPE_BITMASK, - cancelhandling); - if (__glibc_likely (curval == cancelhandling)) - /* Successfully replaced the value. */ - break; - - /* Prepare for the next round. */ - cancelhandling = curval; - } - - buffer->__canceltype = (cancelhandling & CANCELTYPE_BITMASK - ? PTHREAD_CANCEL_ASYNCHRONOUS - : PTHREAD_CANCEL_DEFERRED); + buffer->__canceltype = THREAD_GETMEM (self, canceltype); + THREAD_SETMEM (self, canceltype, PTHREAD_CANCEL_DEFERRED); THREAD_SETMEM (self, cleanup, buffer); } @@ -60,26 +42,8 @@ __libc_cleanup_pop_restore (struct _pthread_cleanup_buffer *buffer) THREAD_SETMEM (self, cleanup, buffer->__prev); - int cancelhandling; - if (__builtin_expect (buffer->__canceltype != PTHREAD_CANCEL_DEFERRED, 0) - && ((cancelhandling = THREAD_GETMEM (self, cancelhandling)) - & CANCELTYPE_BITMASK) == 0) - { - while (1) - { - int curval = THREAD_ATOMIC_CMPXCHG_VAL (self, cancelhandling, - cancelhandling - | CANCELTYPE_BITMASK, - cancelhandling); - if (__glibc_likely (curval == cancelhandling)) - /* Successfully replaced the value. */ - break; - - /* Prepare for the next round. */ - cancelhandling = curval; - } - + THREAD_SETMEM (self, canceltype, buffer->__canceltype); + if (buffer->__canceltype == PTHREAD_CANCEL_ASYNCHRONOUS) __pthread_testcancel (); - } } libc_hidden_def (__libc_cleanup_pop_restore) diff --git a/nptl/pthread_cancel.c b/nptl/pthread_cancel.c index 69b57c8808..c6086b6c3c 100644 --- a/nptl/pthread_cancel.c +++ b/nptl/pthread_cancel.c @@ -53,7 +53,7 @@ sigcancel_handler (int sig, siginfo_t *si, void *ctx) /* Set the return value. */ THREAD_SETMEM (self, result, PTHREAD_CANCELED); /* Make sure asynchronous cancellation is still enabled. */ - if ((ch & CANCELTYPE_BITMASK) != 0) + if (self->canceltype == PTHREAD_CANCEL_ASYNCHRONOUS) __do_cancel (); } @@ -104,7 +104,7 @@ __pthread_cancel (pthread_t th) #endif THREAD_SETMEM (pd, result, PTHREAD_CANCELED); - if ((oldch & CANCELTYPE_BITMASK) != 0) + if (pd->canceltype == PTHREAD_CANCEL_ASYNCHRONOUS) __do_cancel (); return 0; } diff --git a/nptl/pthread_setcanceltype.c b/nptl/pthread_setcanceltype.c index ae5df1d591..e7b24ae733 100644 --- a/nptl/pthread_setcanceltype.c +++ b/nptl/pthread_setcanceltype.c @@ -29,43 +29,11 @@ __pthread_setcanceltype (int type, int *oldtype) volatile struct pthread *self = THREAD_SELF; - int oldval = THREAD_GETMEM (self, cancelhandling); - while (1) - { - int newval = (type == PTHREAD_CANCEL_ASYNCHRONOUS - ? oldval | CANCELTYPE_BITMASK - : oldval & ~CANCELTYPE_BITMASK); - - /* Store the old value. */ - if (oldtype != NULL) - *oldtype = ((oldval & CANCELTYPE_BITMASK) - ? PTHREAD_CANCEL_ASYNCHRONOUS : PTHREAD_CANCEL_DEFERRED); - - /* Avoid doing unnecessary work. The atomic operation can - potentially be expensive if the memory has to be locked and - remote cache lines have to be invalidated. */ - if (oldval == newval) - break; - - /* Update the cancel handling word. This has to be done - atomically since other bits could be modified as well. */ - int curval = THREAD_ATOMIC_CMPXCHG_VAL (self, cancelhandling, newval, - oldval); - if (__glibc_likely (curval == oldval)) - { - if (self->cancelstate == PTHREAD_CANCEL_ENABLE - && CANCEL_CANCELED_AND_ASYNCHRONOUS (newval)) - { - THREAD_SETMEM (self, result, PTHREAD_CANCELED); - __do_cancel (); - } - - break; - } - - /* Prepare for the next round. */ - oldval = curval; - } + if (oldtype != NULL) + *oldtype = self->canceltype; + self->canceltype = type; + if (type == PTHREAD_CANCEL_ASYNCHRONOUS) + __pthread_testcancel (); return 0; } diff --git a/sysdeps/nptl/dl-tls_init_tp.c b/sysdeps/nptl/dl-tls_init_tp.c index 378b26a2f5..b7b3bb1cdb 100644 --- a/sysdeps/nptl/dl-tls_init_tp.c +++ b/sysdeps/nptl/dl-tls_init_tp.c @@ -96,4 +96,5 @@ __tls_init_tp (void) THREAD_SETMEM (pd, stackblock_size, (size_t) __libc_stack_end); THREAD_SETMEM (pd, cancelstate, PTHREAD_CANCEL_ENABLE); + THREAD_SETMEM (pd, canceltype, PTHREAD_CANCEL_DEFERRED); }