From patchwork Wed Sep 12 19:40:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Breno Leitao X-Patchwork-Id: 969181 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 429XYn30nVz9s9h for ; Thu, 13 Sep 2018 05:54:53 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=debian.org Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 429XYn1m5qzF39K for ; Thu, 13 Sep 2018 05:54:53 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=debian.org X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (mailfrom) smtp.mailfrom=debian.org (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=leitao@debian.org; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=debian.org Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 429XFc002QzF3Rg for ; Thu, 13 Sep 2018 05:40:51 +1000 (AEST) Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w8CJY34R101707 for ; Wed, 12 Sep 2018 15:40:50 -0400 Received: from e36.co.us.ibm.com (e36.co.us.ibm.com [32.97.110.154]) by mx0a-001b2d01.pphosted.com with ESMTP id 2mf8mj1cfc-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 12 Sep 2018 15:40:49 -0400 Received: from localhost by e36.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 12 Sep 2018 13:40:49 -0600 Received: from b03cxnp07028.gho.boulder.ibm.com (9.17.130.15) by e36.co.us.ibm.com (192.168.1.136) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 12 Sep 2018 13:40:46 -0600 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w8CJejn962980320 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 12 Sep 2018 12:40:45 -0700 Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A3A8F7805C; Wed, 12 Sep 2018 13:40:45 -0600 (MDT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 652557805F; Wed, 12 Sep 2018 13:40:43 -0600 (MDT) Received: from debra.ibm.com (unknown [9.85.149.11]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Wed, 12 Sep 2018 13:40:43 -0600 (MDT) From: Breno Leitao To: linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH 06/11] powerpc/tm: Refactor the __switch_to_tm code Date: Wed, 12 Sep 2018 16:40:14 -0300 X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1536781219-13938-1-git-send-email-leitao@debian.org> References: <1536781219-13938-1-git-send-email-leitao@debian.org> X-TM-AS-GCONF: 00 x-cbid: 18091219-0020-0000-0000-00000E640D00 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00009709; HX=3.00000242; KW=3.00000007; PH=3.00000004; SC=3.00000266; SDB=6.01087370; UDB=6.00561487; IPR=6.00867382; MB=3.00023256; MTD=3.00000008; XFM=3.00000015; UTC=2018-09-12 19:40:48 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18091219-0021-0000-0000-000063013380 Message-Id: <1536781219-13938-7-git-send-email-leitao@debian.org> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-09-12_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1809120195 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mikey@neuling.org, gromero@linux.vnet.ibm.com, Breno Leitao , ldufour@linux.vnet.ibm.com Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" __switch_to_tm is the function that switches between two tasks which might have TM enabled. This function is clearly split in two parts, the task that is leaving the CPU, known as 'prev' and the task that is being scheduled, known as new. It starts checking if the previous task had TM enable, if so, it increases the load_tm (this is the only place we increment load_tm). It also saves the TM SPRs here. If the previous task was scheduled out with a transaction active, the failure cause needs to be updated, since it might contain the failure cause that caused the exception, as TM_CAUSE_MISC. In this case, since there was a context switch, overwrite the failure cause. If the previous task has overflowed load_tm, disable TM, putting the facility save/restore lazy mechanism at lazy mode. Regarding the new task, when loading it, it basically restore the SPRs, and TIF_RESTORE_TM (already set by tm_reclaim_current if the transaction was active) would invoke the recheckpoint process later in restore_tm_state() if recheckpoint is somehow required. On top of that, both tm_reclaim_task() and tm_recheckpoint_new_task() functions are not used anymore, removing them. Signed-off-by: Breno Leitao --- arch/powerpc/kernel/process.c | 163 +++++++++++++++------------------- 1 file changed, 74 insertions(+), 89 deletions(-) diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c index fe063c0142e3..5cace1b744b1 100644 --- a/arch/powerpc/kernel/process.c +++ b/arch/powerpc/kernel/process.c @@ -921,48 +921,6 @@ void tm_reclaim_current(uint8_t cause) tm_reclaim_thread(¤t->thread, cause); } -static inline void tm_reclaim_task(struct task_struct *tsk) -{ - /* We have to work out if we're switching from/to a task that's in the - * middle of a transaction. - * - * In switching we need to maintain a 2nd register state as - * oldtask->thread.ckpt_regs. We tm_reclaim(oldproc); this saves the - * checkpointed (tbegin) state in ckpt_regs, ckfp_state and - * ckvr_state - * - * We also context switch (save) TFHAR/TEXASR/TFIAR in here. - */ - struct thread_struct *thr = &tsk->thread; - - if (!thr->regs) - return; - - if (!MSR_TM_ACTIVE(thr->regs->msr)) - goto out_and_saveregs; - - WARN_ON(tm_suspend_disabled); - - TM_DEBUG("--- tm_reclaim on pid %d (NIP=%lx, " - "ccr=%lx, msr=%lx, trap=%lx)\n", - tsk->pid, thr->regs->nip, - thr->regs->ccr, thr->regs->msr, - thr->regs->trap); - - tm_reclaim_thread(thr, TM_CAUSE_RESCHED); - - TM_DEBUG("--- tm_reclaim on pid %d complete\n", - tsk->pid); - -out_and_saveregs: - /* Always save the regs here, even if a transaction's not active. - * This context-switches a thread's TM info SPRs. We do it here to - * be consistent with the restore path (in recheckpoint) which - * cannot happen later in _switch(). - */ - tm_save_sprs(thr); -} - extern void __tm_recheckpoint(struct thread_struct *thread); void tm_recheckpoint(struct thread_struct *thread) @@ -997,59 +955,87 @@ static void tm_fix_failure_cause(struct task_struct *task, uint8_t cause) task->thread.tm_texasr |= (unsigned long) cause << 56; } -static inline void tm_recheckpoint_new_task(struct task_struct *new) +static inline void __switch_to_tm(struct task_struct *prev, + struct task_struct *new) { if (!cpu_has_feature(CPU_FTR_TM)) return; - /* Recheckpoint the registers of the thread we're about to switch to. - * - * If the task was using FP, we non-lazily reload both the original and - * the speculative FP register states. This is because the kernel - * doesn't see if/when a TM rollback occurs, so if we take an FP - * unavailable later, we are unable to determine which set of FP regs - * need to be restored. - */ - if (!tm_enabled(new)) - return; - - if (!MSR_TM_ACTIVE(new->thread.regs->msr)){ - tm_restore_sprs(&new->thread); - return; - } - /* Recheckpoint to restore original checkpointed register state. */ - TM_DEBUG("*** tm_recheckpoint of pid %d (new->msr 0x%lx)\n", - new->pid, new->thread.regs->msr); - - tm_recheckpoint(&new->thread); - - /* - * The checkpointed state has been restored but the live state has - * not, ensure all the math functionality is turned off to trigger - * restore_math() to reload. - */ - new->thread.regs->msr &= ~(MSR_FP | MSR_VEC | MSR_VSX); + /* The task leaving the CPU was using TM, let's handle it */ + if (tm_enabled(prev)) { + /* + * Load_tm is incremented only when the task is scheduled out + */ + prev->thread.load_tm++; - TM_DEBUG("*** tm_recheckpoint of pid %d complete " - "(kernel msr 0x%lx)\n", - new->pid, mfmsr()); -} + /* + * If TM is enabled for the thread, it needs to, at least, + * save the SPRs + */ + tm_enable(); + tm_save_sprs(&prev->thread); -static inline void __switch_to_tm(struct task_struct *prev, - struct task_struct *new) -{ - if (cpu_has_feature(CPU_FTR_TM)) { - if (tm_enabled(prev) || tm_enabled(new)) - tm_enable(); + /* + * If we got here with an active transaction, then, it was + * aborted by TM_KERNEL_ENTRY and the fix the failure case + * needs to be fixed, so, indepedently how we arrived here, the + * new TM abort case will be TM_CAUSE_RESCHED now. + */ + if (MSR_TM_ACTIVE(prev->thread.regs->msr)) { + /* + * If there was an IRQ during trecheckpoint, it will + * cause an IRQ to be replayed. This replayed IRQ can + * invoke SCHEDULE_USER, thus, we arrive here with a TM + * active transaction. + * I.e, the task was leaving kernelspace to userspace, + * already trecheckpointed, but there was a IRQ during + * the trecheckpoint process (soft irq disabled), and + * on the IRQ replay, the process was de-scheduled, so, + * SCHEDULE_USER was called and here we are. + * + */ + if (MSR_TM_ACTIVE(mfmsr())) { + /* + * This is the only other case other than + * TM_KERNEL_ENTRY that does a TM reclaim + */ + tm_reclaim_current(TM_CAUSE_RESCHED); + } - if (tm_enabled(prev)) { - prev->thread.load_tm++; - tm_reclaim_task(prev); - if (!MSR_TM_ACTIVE(prev->thread.regs->msr) && prev->thread.load_tm == 0) + /* + * If rescheduled with TM active, update the + * failure cause + */ + tm_fix_failure_cause(prev, TM_CAUSE_RESCHED); + } else { + /* + * TM enabled but not transactional. Just disable TM + * if load_tm overflows. This should be the only place + * that disables the TM and reenables the laziness + * save/restore + */ + if (prev->thread.load_tm == 0) prev->thread.regs->msr &= ~MSR_TM; } + } - tm_recheckpoint_new_task(new); + /* + * It is a *bug* if we arrived so late with a transaction active + * (more precisely suspended) + */ + if (WARN_ON(MSR_TM_ACTIVE(mfmsr()))) { + /* Recovery path. 0x99 shouldn't be exported to UAPI */ + tm_reclaim_current(0x99); + } + + /* + * If the next task has TM enabled, restore the SPRs. Do not need to + * care about recheckpoint at this time. It will be done later if + * TIF_RESTORE_TM was set when the task was scheduled out + */ + if (tm_enabled(new)) { + tm_enable(); + tm_restore_sprs(&new->thread); } } @@ -1101,7 +1087,6 @@ void restore_tm_state(struct pt_regs *regs) } #else -#define tm_recheckpoint_new_task(new) #define __switch_to_tm(prev, new) #endif /* CONFIG_PPC_TRANSACTIONAL_MEM */ @@ -1588,9 +1573,9 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) /* * Flush TM state out so we can copy it. __switch_to_tm() does this * flush but it removes the checkpointed state from the current CPU and - * transitions the CPU out of TM mode. Hence we need to call - * tm_recheckpoint_new_task() (on the same task) to restore the - * checkpointed state back and the TM mode. + * transitions the CPU out of TM mode. Hence we need to make sure + * TIF_RESTORE_TM is set so restore_tm_state is called to restore the + * checkpointed state and back to TM mode. * * Can't pass dst because it isn't ready. Doesn't matter, passing * dst is only important for __switch_to()