From patchwork Tue May 31 17:52:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 1637514 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=B4QFtwBa; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Received: from sourceware.org (ip-8-43-85-97.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LCKjG49ygz9sCq for ; Wed, 1 Jun 2022 03:56:50 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 9B0EA395B44B for ; Tue, 31 May 2022 17:56:48 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 9B0EA395B44B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1654019808; bh=g/Id24zx6detz9jAu/dX60S4ntyaODximubYcmGsmfQ=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=B4QFtwBaMIIwOVlPXYwf9PIqDM/4z4W7BYHCclmECI+9LkzRNzfmperYPZQBPme21 wHS5//JrvatUFzL55wYvNGiOTpOvNsIR5YzCaPpgLaNvSY1Teb6FHAGfVbK6qgL3gC ZFqQ7lNXNrsnB53TdrGW0IRwybHUydCJjLL8tlM8= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-oi1-x230.google.com (mail-oi1-x230.google.com [IPv6:2607:f8b0:4864:20::230]) by sourceware.org (Postfix) with ESMTPS id F299E395B479 for ; Tue, 31 May 2022 17:53:03 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org F299E395B479 Received: by mail-oi1-x230.google.com with SMTP id r206so11181638oib.8 for ; Tue, 31 May 2022 10:53:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=g/Id24zx6detz9jAu/dX60S4ntyaODximubYcmGsmfQ=; b=N6LBqyDhhMrcHxYKZaUeyuH4az1qfDNGPTJbQyVw1zHfkpU0BTqQWEKLTu5YeXWAm1 t5rGWshubTT802pwhF2J8f98ARrwBqzFPVB/kDILOfMr3AFgeCw0R+9detOYx9Ad8jy2 k5GSACZoOTI8+4hoa+qzwZkJlVU1imEY80KC6GZ1kcXy14DAEV/NLKJ1n8HeepLWzpU/ YfNc+SXw+VeiT6NKlphhZbUJ0BvivxhwpuC7MdhrrCxV8HBOY688bxGuE5eORO70EuCE 2mPLpXmiqgjOAywLMYzA4hKt1OAoqzrQEVofCOGz6pmVTcF7tP2UvjzLXha047e0HXfX DT0g== X-Gm-Message-State: AOAM532sp1YaN0bTmsZvDd1E+p3CEpSttDkDaixrE+qBRBPIynXZ6C6K 4zh4Tx9fR8zzx5aYzpCK41wcFyISL3eZzw== X-Google-Smtp-Source: ABdhPJwXeoyPniYJYvPNq1zVCR1vSa9hz/aRVwYaSrlt4OuI08L+MAdisI3ewhw+x8jOR/XVWld/Og== X-Received: by 2002:a05:6808:f91:b0:32b:7b3c:b3f1 with SMTP id o17-20020a0568080f9100b0032b7b3cb3f1mr13284952oiw.58.1654019582548; Tue, 31 May 2022 10:53:02 -0700 (PDT) Received: from birita.. ([2804:431:c7ca:e39c:980f:eaf1:7a30:10ef]) by smtp.gmail.com with ESMTPSA id f30-20020a4a8921000000b0035f627ac898sm6402626ooi.10.2022.05.31.10.53.01 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 31 May 2022 10:53:02 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v3 03/12] nptl: Do not use pthread set_tid_address as state synchronization (BZ #19951) Date: Tue, 31 May 2022 14:52:46 -0300 Message-Id: <20220531175255.1513396-4-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220531175255.1513396-1-adhemerval.zanella@linaro.org> References: <20220531175255.1513396-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-12.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" The use after free described in BZ#19951 is due the use of two different PD fields, 'joinid' and 'cancelhandling', used too describe the thread state and to synchronize the calls of pthread_join, pthread_detach, pthread_exit, and normal thread exit. Any state change potentially requires to check for both field atomically to handle partial state (such as pthread_join() with a cancellation handler to issue a 'joinstate' field rollback). This patch uses a different PD member with 4 possible states (JOINABLE, DETACHED, EXITING, and EXITED) instead of pthread 'tid' field: 1. On pthread_create the inital state is set either to JOINABLE or DETACHED depending of the pthread attribute used. 2. On pthread_detach, a CAS is issued on the state. If the CAS fails it means that thread is already detached (DETACHED) or is being terminated (EXITING). For former an EINVAL is returned, while for latter pthread_detach should be reponsible to join the thread (and deallocate any internal resources). 3. In the exit phase of the wrapper function for the thread start routine (reached either if the thread function has returned, pthread_exit has being called, or cancellation handled has been acted upon) we issue a CAS on state to set to EXITING mode. If the thread is previously on DETACHED mode the thread itself is responsible for arranging the deallocation of any resource, otherwise the thread needs to be joined (detached threads cannot immediately deallocate themselves) 4. The clear_tid_field on 'clone' call is changed to set the new 'state' field on thread exit (EXITED). This state ins only reached at thread termination. 5. The pthread_join implementation is now simpler: the futex wait is done directly on thread state and there is no need to reset it in case of timeout (since the state is now set either by pthread_detach() or by the kernel on process termination). The race condition on pthread_detach is avoided with only one atomic operation on PD state: once the mode is set to THREAD_STATE_DETACHED it is up to thread itself to deallocate its memory (done on the exit phase at pthread_create()). Also, the INVALID_NOT_TERMINATED_TD_P is removed since a a negative tid is not possible and the macro is not used anywhere. This change trigger an invalid C11 thread tests: it crates a thread, which detaches itself, and after a timeout the creating thread checks if the join fails. The issue is once thrd_join() is called the thread lifetime is not defined. The test is changed so the sleep is done by the thread itself, so the creating thread will try to join a valid thread identificator. Checked on x86_64-linux-gnu, i686-linux-gnu, aarch64-linux-gnu, arm-linux-gnueabihf, and powerpc64-linux-gnu. --- nptl/descr.h | 26 +++--- nptl/nptl-stack.h | 2 +- nptl/pthread_cancel.c | 3 +- nptl/pthread_clockjoin.c | 2 +- nptl/pthread_create.c | 44 +++++++--- nptl/pthread_detach.c | 40 ++++----- nptl/pthread_getattr_np.c | 2 +- nptl/pthread_join.c | 2 +- nptl/pthread_join_common.c | 124 ++++++++++------------------ nptl/pthread_timedjoin.c | 2 +- nptl/pthread_tryjoin.c | 18 ++-- sysdeps/nptl/dl-tls_init_tp.c | 4 +- sysdeps/nptl/libc_start_call_main.h | 7 ++ sysdeps/nptl/pthreadP.h | 3 +- sysdeps/pthread/tst-thrd-detach.c | 16 ++-- 15 files changed, 142 insertions(+), 153 deletions(-) diff --git a/nptl/descr.h b/nptl/descr.h index bb46b5958e..d22eb75dd3 100644 --- a/nptl/descr.h +++ b/nptl/descr.h @@ -126,6 +126,18 @@ struct priority_protection_data }; +/* Define a possible thread state on 'joinstate' field. The value will be + cleared by the kernel when the thread terminates (CLONE_CHILD_CLEARTID), + so THREAD_STATE_EXITED must be 0. */ +enum + { + THREAD_STATE_EXITED = 0, + THREAD_STATE_EXITING, + THREAD_STATE_JOINABLE, + THREAD_STATE_DETACHED, + }; + + /* Thread descriptor data structure. */ struct pthread { @@ -168,8 +180,7 @@ struct pthread GL (dl_stack_user) list. */ list_t list; - /* Thread ID - which is also a 'is this thread descriptor (and - therefore stack) used' flag. */ + /* Thread ID set by the kernel with CLONE_PARENT_SETTID. */ pid_t tid; /* Ununsed. */ @@ -346,15 +357,8 @@ struct pthread hp_timing_t cpuclock_offset_ununsed; #endif - /* If the thread waits to join another one the ID of the latter is - stored here. - - In case a thread is detached this field contains a pointer of the - TCB if the thread itself. This is something which cannot happen - in normal operation. */ - struct pthread *joinid; - /* Check whether a thread is detached. */ -#define IS_DETACHED(pd) ((pd)->joinid == (pd)) + /* The current thread state defined by the THREAD_STATE_* enumeration. */ + unsigned int joinstate; /* The result of the thread function. */ void *result; diff --git a/nptl/nptl-stack.h b/nptl/nptl-stack.h index 5a90752e37..a62d09498a 100644 --- a/nptl/nptl-stack.h +++ b/nptl/nptl-stack.h @@ -31,7 +31,7 @@ extern size_t __nptl_stack_cache_maxsize attribute_hidden; static inline bool __nptl_stack_in_use (struct pthread *pd) { - return pd->tid <= 0; + return atomic_load_relaxed (&pd->joinstate) == THREAD_STATE_EXITED; } /* Remove the stack ELEM from its list. */ diff --git a/nptl/pthread_cancel.c b/nptl/pthread_cancel.c index c4e571ec21..1cdc9ef079 100644 --- a/nptl/pthread_cancel.c +++ b/nptl/pthread_cancel.c @@ -72,7 +72,8 @@ __pthread_cancel (pthread_t th) { volatile struct pthread *pd = (volatile struct pthread *) th; - if (pd->tid == 0) + int state = atomic_load_acquire (&pd->joinstate); + if (state == THREAD_STATE_EXITED || state == THREAD_STATE_EXITING) /* The thread has already exited on the kernel side. Its outcome (regular exit, other cancelation) has already been determined. */ diff --git a/nptl/pthread_clockjoin.c b/nptl/pthread_clockjoin.c index c299551423..d306d13d3d 100644 --- a/nptl/pthread_clockjoin.c +++ b/nptl/pthread_clockjoin.c @@ -30,7 +30,7 @@ ___pthread_clockjoin_np64 (pthread_t threadid, void **thread_return, return EINVAL; return __pthread_clockjoin_ex (threadid, thread_return, - clockid, abstime, true); + clockid, abstime); } #if __TIMESIZE == 64 diff --git a/nptl/pthread_create.c b/nptl/pthread_create.c index a6200cf9e5..89976901d6 100644 --- a/nptl/pthread_create.c +++ b/nptl/pthread_create.c @@ -287,7 +287,7 @@ static int create_thread (struct pthread *pd, const struct pthread_attr *attr, .flags = clone_flags, .pidfd = (uintptr_t) &pd->tid, .parent_tid = (uintptr_t) &pd->tid, - .child_tid = (uintptr_t) &pd->tid, + .child_tid = (uintptr_t) &pd->joinstate, .stack = (uintptr_t) stackaddr, .stack_size = stacksize, .tls = (uintptr_t) tp, @@ -352,12 +352,14 @@ start_thread (void *arg) and free any resource prior return to the pthread_create caller. */ setup_failed = pd->setup_failed == 1; if (setup_failed) - pd->joinid = NULL; + pd->joinstate = THREAD_STATE_JOINABLE; /* And give it up right away. */ lll_unlock (pd->lock, LLL_PRIVATE); if (setup_failed) + /* No need to clear the tid here, pthread_create() will join the + thread prior returning to caller. */ goto out; } @@ -489,6 +491,23 @@ start_thread (void *arg) the breakpoint reports TD_THR_RUN state rather than TD_THR_ZOMBIE. */ atomic_bit_set (&pd->cancelhandling, EXITING_BIT); + + /* CONCURRENCY NOTES: + + Concurrent pthread_detach() will either set state to + THREAD_STATE_DETACHED or wait for the thread to terminate. The exiting + state set here is set so a pthread_join() wait until all the required + cleanup steps are done. + + The 'prevstate' field will be used to determine who is responsible to + call __nptl_free_tcb below. */ + + unsigned int prevstate; + do + prevstate = atomic_load_relaxed (&pd->joinstate); + while (!atomic_compare_exchange_weak_acquire (&pd->joinstate, &prevstate, + THREAD_STATE_EXITING)); + if (__glibc_unlikely (atomic_decrement_and_test (&__nptl_nthreads))) /* This was the last thread. */ exit (0); @@ -567,17 +586,17 @@ start_thread (void *arg) pd->setxid_futex = 0; } - /* If the thread is detached free the TCB. */ - if (IS_DETACHED (pd)) - /* Free the TCB. */ + if (prevstate == THREAD_STATE_DETACHED) __nptl_free_tcb (pd); + pd->tid = 0; + out: /* We cannot call '_exit' here. '_exit' will terminate the process. The 'exit' implementation in the kernel will signal when the process is really dead since 'clone' got passed the CLONE_CHILD_CLEARTID - flag. The 'tid' field in the TCB will be set to zero. + flag. The 'joinstate' field in the TCB will be set to zero. rseq TLS is still registered at this point. Rely on implicit unregistration performed by the kernel on thread teardown. This is not a @@ -689,7 +708,9 @@ __pthread_create_2_1 (pthread_t *newthread, const pthread_attr_t *attr, /* Initialize the field for the ID of the thread which is waiting for us. This is a self-reference in case the thread is created detached. */ - pd->joinid = iattr->flags & ATTR_FLAG_DETACHSTATE ? pd : NULL; + pd->joinstate = iattr->flags & ATTR_FLAG_DETACHSTATE + ? THREAD_STATE_DETACHED + : THREAD_STATE_JOINABLE; /* The debug events are inherited from the parent. */ pd->eventbuf = self->eventbuf; @@ -848,10 +869,11 @@ __pthread_create_2_1 (pthread_t *newthread, const pthread_attr_t *attr, /* Similar to pthread_join, but since thread creation has failed at startup there is no need to handle all the steps. */ - pid_t tid; - while ((tid = atomic_load_acquire (&pd->tid)) != 0) - __futex_abstimed_wait_cancelable64 ((unsigned int *) &pd->tid, - tid, 0, NULL, LLL_SHARED); + unsigned int state; + while ((state = atomic_load_acquire (&pd->joinstate)) + != THREAD_STATE_EXITED) + __futex_abstimed_wait_cancelable64 (&pd->joinstate, state, 0, + NULL, LLL_SHARED); } /* State (c) or (d) and we have ownership of PD (see CONCURRENCY diff --git a/nptl/pthread_detach.c b/nptl/pthread_detach.c index b83ec314fd..ad0ca129e9 100644 --- a/nptl/pthread_detach.c +++ b/nptl/pthread_detach.c @@ -25,32 +25,28 @@ ___pthread_detach (pthread_t th) { struct pthread *pd = (struct pthread *) th; - /* Make sure the descriptor is valid. */ - if (INVALID_NOT_TERMINATED_TD_P (pd)) - /* Not a valid thread handle. */ - return ESRCH; + /* CONCURRENCY NOTES: - int result = 0; + Concurrent pthread_detach will return EINVAL for the case the thread + is already detached (THREAD_STATE_DETACHED). POSIX states it is + undefined to call pthread_detach if TH refers to a non joinable thread. - /* Mark the thread as detached. */ - if (atomic_compare_and_exchange_bool_acq (&pd->joinid, pd, NULL)) + For the case the thread is being terminated (THREAD_STATE_EXITING), + pthread_detach will responsible to clean up the stack. */ + + unsigned int prevstate = atomic_load_relaxed (&pd->joinstate); + do { - /* There are two possibilities here. First, the thread might - already be detached. In this case we return EINVAL. - Otherwise there might already be a waiter. The standard does - not mention what happens in this case. */ - if (IS_DETACHED (pd)) - result = EINVAL; + if (prevstate != THREAD_STATE_JOINABLE) + { + if (prevstate == THREAD_STATE_DETACHED) + return EINVAL; + return __pthread_join (th, 0); + } } - else - /* Check whether the thread terminated meanwhile. In this case we - will just free the TCB. */ - if ((pd->cancelhandling & EXITING_BITMASK) != 0) - /* Note that the code in __free_tcb makes sure each thread - control block is freed only once. */ - __nptl_free_tcb (pd); - - return result; + while (!atomic_compare_exchange_weak_acquire (&pd->joinstate, &prevstate, + THREAD_STATE_DETACHED)); + return 0; } versioned_symbol (libc, ___pthread_detach, pthread_detach, GLIBC_2_34); libc_hidden_ver (___pthread_detach, __pthread_detach) diff --git a/nptl/pthread_getattr_np.c b/nptl/pthread_getattr_np.c index 9c5b73b452..ab01c9a6b0 100644 --- a/nptl/pthread_getattr_np.c +++ b/nptl/pthread_getattr_np.c @@ -52,7 +52,7 @@ __pthread_getattr_np (pthread_t thread_id, pthread_attr_t *attr) iattr->flags = thread->flags; /* The thread might be detached by now. */ - if (IS_DETACHED (thread)) + if (atomic_load_acquire (&thread->joinstate) == THREAD_STATE_DETACHED) iattr->flags |= ATTR_FLAG_DETACHSTATE; /* This is the guardsize after adjusting it. */ diff --git a/nptl/pthread_join.c b/nptl/pthread_join.c index 491de1dbd2..f98d571687 100644 --- a/nptl/pthread_join.c +++ b/nptl/pthread_join.c @@ -22,7 +22,7 @@ int ___pthread_join (pthread_t threadid, void **thread_return) { return __pthread_clockjoin_ex (threadid, thread_return, 0 /* Ignored */, - NULL, true); + NULL); } versioned_symbol (libc, ___pthread_join, pthread_join, GLIBC_2_34); libc_hidden_ver (___pthread_join, __pthread_join) diff --git a/nptl/pthread_join_common.c b/nptl/pthread_join_common.c index ca3245b0af..9637f2ffd5 100644 --- a/nptl/pthread_join_common.c +++ b/nptl/pthread_join_common.c @@ -22,110 +22,70 @@ #include #include -static void -cleanup (void *arg) +/* Check for a possible deadlock situation where the threads are waiting for + each other to finish. Note that this is a "may" error. To be 100% sure we + catch this error we would have to lock the data structures but it is not + necessary. In the unlikely case that two threads are really caught in this + situation they will deadlock. It is the programmer's problem to figure + this out. */ +static inline bool +check_for_deadlock (struct pthread *pd) { - /* If we already changed the waiter ID, reset it. The call cannot - fail for any reason but the thread not having done that yet so - there is no reason for a loop. */ struct pthread *self = THREAD_SELF; - atomic_compare_exchange_weak_acquire (&arg, &self, NULL); + return ((pd == self + || (atomic_load_acquire (&self->joinstate) == THREAD_STATE_DETACHED + && (pd->cancelhandling + & (CANCELING_BITMASK | CANCELED_BITMASK | EXITING_BITMASK + | TERMINATED_BITMASK)) == 0)) + && !cancel_enabled_and_canceled (self->cancelhandling)); } int __pthread_clockjoin_ex (pthread_t threadid, void **thread_return, clockid_t clockid, - const struct __timespec64 *abstime, bool block) + const struct __timespec64 *abstime) { struct pthread *pd = (struct pthread *) threadid; - /* Make sure the descriptor is valid. */ - if (INVALID_NOT_TERMINATED_TD_P (pd)) - /* Not a valid thread handle. */ - return ESRCH; - - /* Is the thread joinable?. */ - if (IS_DETACHED (pd)) - /* We cannot wait for the thread. */ - return EINVAL; - - struct pthread *self = THREAD_SELF; - int result = 0; - LIBC_PROBE (pthread_join, 1, threadid); - if ((pd == self - || (self->joinid == pd - && (pd->cancelhandling - & (CANCELING_BITMASK | CANCELED_BITMASK | EXITING_BITMASK - | TERMINATED_BITMASK)) == 0)) - && !cancel_enabled_and_canceled (self->cancelhandling)) - /* This is a deadlock situation. The threads are waiting for each - other to finish. Note that this is a "may" error. To be 100% - sure we catch this error we would have to lock the data - structures but it is not necessary. In the unlikely case that - two threads are really caught in this situation they will - deadlock. It is the programmer's problem to figure this - out. */ - return EDEADLK; - - /* Wait for the thread to finish. If it is already locked something - is wrong. There can only be one waiter. */ - else if (__glibc_unlikely (atomic_compare_exchange_weak_acquire (&pd->joinid, - &self, - NULL))) - /* There is already somebody waiting for the thread. */ - return EINVAL; - - /* BLOCK waits either indefinitely or based on an absolute time. POSIX also - states a cancellation point shall occur for pthread_join, and we use the - same rationale for posix_timedjoin_np. Both clockwait_tid and the futex - call use the cancellable variant. */ - if (block) + int result = 0; + unsigned int state; + while ((state = atomic_load_acquire (&pd->joinstate)) + != THREAD_STATE_EXITED) { - /* During the wait we change to asynchronous cancellation. If we - are cancelled the thread we are waiting for must be marked as - un-wait-ed for again. */ - pthread_cleanup_push (cleanup, &pd->joinid); - - /* We need acquire MO here so that we synchronize with the - kernel's store to 0 when the clone terminates. (see above) */ - pid_t tid; - while ((tid = atomic_load_acquire (&pd->tid)) != 0) - { - /* The kernel notifies a process which uses CLONE_CHILD_CLEARTID via - futex wake-up when the clone terminates. The memory location - contains the thread ID while the clone is running and is reset to - zero by the kernel afterwards. The kernel up to version 3.16.3 - does not use the private futex operations for futex wake-up when - the clone terminates. */ - int ret = __futex_abstimed_wait_cancelable64 ( - (unsigned int *) &pd->tid, tid, clockid, abstime, LLL_SHARED); - if (ret == ETIMEDOUT || ret == EOVERFLOW) - { - result = ret; - break; - } + if (check_for_deadlock (pd)) + return EDEADLK; + + /* POSIX states calling pthread_join on a non joinable thread is + undefined. However, if PD is still in the cache we can still warn + the caller. */ + if (state == THREAD_STATE_DETACHED) + return EINVAL; + + /* pthread_join is a cancellation entrypoint and we use the same + rationale for pthread_timedjoin_np. + + The kernel notifies a process which uses CLONE_CHILD_CLEARTID via + a memory zeroing and futex wake-up when the process terminates. + The futex operation is not private. */ + int ret = __futex_abstimed_wait_cancelable64 (&pd->joinstate, state, + clockid, abstime, + LLL_SHARED); + if (ret == ETIMEDOUT || ret == EOVERFLOW) + { + result = ret; + break; } - - pthread_cleanup_pop (0); } void *pd_result = pd->result; - if (__glibc_likely (result == 0)) + if (result == 0) { - /* We mark the thread as terminated and as joined. */ - pd->tid = -1; - - /* Store the return value if the caller is interested. */ if (thread_return != NULL) *thread_return = pd_result; - - /* Free the TCB. */ __nptl_free_tcb (pd); } - else - pd->joinid = NULL; LIBC_PROBE (pthread_join_ret, 3, threadid, result, pd_result); diff --git a/nptl/pthread_timedjoin.c b/nptl/pthread_timedjoin.c index 96fd54dc84..dbe848bac3 100644 --- a/nptl/pthread_timedjoin.c +++ b/nptl/pthread_timedjoin.c @@ -24,7 +24,7 @@ ___pthread_timedjoin_np64 (pthread_t threadid, void **thread_return, const struct __timespec64 *abstime) { return __pthread_clockjoin_ex (threadid, thread_return, - CLOCK_REALTIME, abstime, true); + CLOCK_REALTIME, abstime); } #if __TIMESIZE == 64 diff --git a/nptl/pthread_tryjoin.c b/nptl/pthread_tryjoin.c index 1ceb1c4e8f..b45addec82 100644 --- a/nptl/pthread_tryjoin.c +++ b/nptl/pthread_tryjoin.c @@ -21,15 +21,17 @@ int __pthread_tryjoin_np (pthread_t threadid, void **thread_return) { - /* Return right away if the thread hasn't terminated yet. */ - struct pthread *pd = (struct pthread *) threadid; - if (pd->tid != 0) - return EBUSY; + /* The joinable state (THREAD_STATE_JOINABLE) is straigthforward since the + thread hasn't finished yet and trying to join might block. + The exiting thread (THREAD_STATE_EXITING) also migth result in a blocking + call: a detached thread might change its state to exiting and a exiting + thread my take some time to exit (and thus let the kernel set the state + to THREAD_STATE_EXITED). */ - /* If pd->tid == 0 then lll_wait_tid will not block on futex - operation. */ - return __pthread_clockjoin_ex (threadid, thread_return, 0 /* Ignored */, - NULL, false); + struct pthread *pd = (struct pthread *) threadid; + return atomic_load_acquire (&pd->joinstate) != THREAD_STATE_EXITED + ? EBUSY + : __pthread_clockjoin_ex (threadid, thread_return, 0, NULL); } versioned_symbol (libc, __pthread_tryjoin_np, pthread_tryjoin_np, GLIBC_2_34); diff --git a/sysdeps/nptl/dl-tls_init_tp.c b/sysdeps/nptl/dl-tls_init_tp.c index 53fba774a5..75afe4bad4 100644 --- a/sysdeps/nptl/dl-tls_init_tp.c +++ b/sysdeps/nptl/dl-tls_init_tp.c @@ -72,7 +72,7 @@ __tls_init_tp (void) list_add (&pd->list, &GL (dl_stack_user)); /* Early initialization of the TCB. */ - pd->tid = INTERNAL_SYSCALL_CALL (set_tid_address, &pd->tid); + pd->tid = INTERNAL_SYSCALL_CALL (set_tid_address, &pd->joinstate); THREAD_SETMEM (pd, specific[0], &pd->specific_1stblock[0]); THREAD_SETMEM (pd, user_stack, true); @@ -128,4 +128,6 @@ __tls_init_tp (void) It will be bigger than it actually is, but for unwind.c/pt-longjmp.c purposes this is good enough. */ THREAD_SETMEM (pd, stackblock_size, (size_t) __libc_stack_end); + + THREAD_SETMEM (pd, joinstate, THREAD_STATE_JOINABLE); } diff --git a/sysdeps/nptl/libc_start_call_main.h b/sysdeps/nptl/libc_start_call_main.h index a9e85f2b09..5fcaedbfac 100644 --- a/sysdeps/nptl/libc_start_call_main.h +++ b/sysdeps/nptl/libc_start_call_main.h @@ -18,6 +18,7 @@ #include #include +#include _Noreturn static void __libc_start_call_main (int (*main) (int, char **, char ** MAIN_AUXVEC_DECL), @@ -65,6 +66,12 @@ __libc_start_call_main (int (*main) (int, char **, char ** MAIN_AUXVEC_DECL), /* One less thread. Decrement the counter. If it is zero we terminate the entire process. */ result = 0; + + /* For the case a thread is waiting for the main thread to finish. */ + struct pthread *self = THREAD_SELF; + atomic_store_release (&self->joinstate, THREAD_STATE_EXITED); + futex_wake (&self->joinstate, 1, FUTEX_SHARED); + if (! atomic_decrement_and_test (&__nptl_nthreads)) /* Not much left to do but to exit the thread, not the process. */ while (1) diff --git a/sysdeps/nptl/pthreadP.h b/sysdeps/nptl/pthreadP.h index e6e7430dfe..96d2015902 100644 --- a/sysdeps/nptl/pthreadP.h +++ b/sysdeps/nptl/pthreadP.h @@ -243,7 +243,6 @@ libc_hidden_proto (__pthread_current_priority) nothing. And if the test triggers the thread descriptor is guaranteed to be invalid. */ #define INVALID_TD_P(pd) __builtin_expect ((pd)->tid <= 0, 0) -#define INVALID_NOT_TERMINATED_TD_P(pd) __builtin_expect ((pd)->tid < 0, 0) extern void __pthread_unwind (__pthread_unwind_buf_t *__buf) __cleanup_fct_attribute __attribute ((__noreturn__)) @@ -521,7 +520,7 @@ libc_hidden_proto (__pthread_setcanceltype) extern void __pthread_testcancel (void); libc_hidden_proto (__pthread_testcancel) extern int __pthread_clockjoin_ex (pthread_t, void **, clockid_t, - const struct __timespec64 *, bool) + const struct __timespec64 *) attribute_hidden; extern int __pthread_sigmask (int, const sigset_t *, sigset_t *); libc_hidden_proto (__pthread_sigmask); diff --git a/sysdeps/pthread/tst-thrd-detach.c b/sysdeps/pthread/tst-thrd-detach.c index d4800d7ee4..89f02c64a4 100644 --- a/sysdeps/pthread/tst-thrd-detach.c +++ b/sysdeps/pthread/tst-thrd-detach.c @@ -20,14 +20,14 @@ #include #include #include - +#include #include static int detach_thrd (void *arg) { - if (thrd_detach (thrd_current ()) != thrd_success) - FAIL_EXIT1 ("thrd_detach failed"); + thrd_sleep (&(struct timespec) { .tv_sec = INT_MAX }, NULL); + thrd_exit (thrd_success); } @@ -36,15 +36,11 @@ do_test (void) { thrd_t id; - /* Create new thread. */ - if (thrd_create (&id, detach_thrd, NULL) != thrd_success) - FAIL_EXIT1 ("thrd_create failed"); + TEST_COMPARE (thrd_create (&id, detach_thrd, NULL), thrd_success); - /* Give some time so the thread can finish. */ - thrd_sleep (&(struct timespec) {.tv_sec = 2}, NULL); + TEST_COMPARE (thrd_detach (id), thrd_success); - if (thrd_join (id, NULL) == thrd_success) - FAIL_EXIT1 ("thrd_join succeed where it should fail"); + TEST_COMPARE (thrd_join (id, NULL), thrd_error); return 0; }