From patchwork Mon Nov 12 21:44:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996674 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="gqXsG0Rr"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4Dr0lcjz9s7W for ; Tue, 13 Nov 2018 08:50:20 +1100 (AEDT) Received: from localhost ([::1]:50810 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK65-0006jS-N3 for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 16:50:17 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54055) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK35-0003U9-4K for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:11 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK32-0004to-Gs for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:11 -0500 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]:39652) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK32-0004tP-A5 for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:08 -0500 Received: by mail-wm1-x344.google.com with SMTP id u13-v6so9546679wmc.4 for ; Mon, 12 Nov 2018 13:47:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=M2zS68CHd7IlCK22ocgXgAzsD+F2Cu2n8RE2k2avgm0=; b=gqXsG0RrLoOeJx3wX4RCv0WUG/NYZ1nbfqZ8wOOnOG5xtdKwFWuTOJjtWK9JCrxubl VDRMkVFC3zDWMaTxEH456FPz7CuMi9amIVUVyXg8dvd4OHMOtA395S0SPEKm/717knk/ 78I5nr7trz/UzFge68VqmuJiZFSmwjmQQZ5u4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=M2zS68CHd7IlCK22ocgXgAzsD+F2Cu2n8RE2k2avgm0=; b=bDmR+7p1M9asNZeRClKOt6gh3lViebDWNfVIKMTI2hE603wYm5THL7RETgAw4YwUNj 9qjqlJg5mXFFct+d/dP9GbKO+HN4BUE00lzi/RKs3cq++cJC5uoPuvbsJSk6IHbB1JfH 6q5fcP60TGczYTooSgim//M4U+v0d19faXMjhC7UJ4i6+8/tEpw0nJ7A/fYBBY/jv6q1 /hEvWEU7Bzq2Sr3BkcvWVoFpvZjtjrnTf/G+fUKIAErSktiwArFv1zW57JjmnPvlZ0rq OYdOqb5Y/yIMLIhfdI0EcC3J7fb3ByTkUKefCAZhBLo8erdzL6NsjB264jSIWQC3ISkP oeuA== X-Gm-Message-State: AGRZ1gJxopU7yJ1SRK8OfGyHzn+qiH3wv6h2HbfWjOEpHVv19Q2hT/Q0 kR+ujV7ggPMa8X9HIr7Vn5lBar99C2vzFw== X-Google-Smtp-Source: AJdET5e7z75RAThezXX0kcOGH+oUZ6YjFVuie7Rb0skwU3/qqGmKM/rxdPWU72RkpwhUEtQBKyjdYA== X-Received: by 2002:a1c:2104:: with SMTP id h4-v6mr1062077wmh.130.1542059227067; Mon, 12 Nov 2018 13:47:07 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.06 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:06 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:44:47 +0100 Message-Id: <20181112214503.22941-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::344 Subject: [Qemu-devel] [PATCH for-4.0 01/17] tcg/i386: Add constraints for r8 and r9 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" These are function call arguments for x86_64 we will need soon. Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.inc.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c index 436195894b..e4d9be57ff 100644 --- a/tcg/i386/tcg-target.inc.c +++ b/tcg/i386/tcg-target.inc.c @@ -230,6 +230,14 @@ static const char *target_parse_constraint(TCGArgConstraint *ct, ct->ct |= TCG_CT_REG; tcg_regset_set_reg(ct->u.regs, TCG_REG_EDI); break; + case 'E': /* "Eight", r8 */ + ct->ct |= TCG_CT_REG; + tcg_regset_set_reg(ct->u.regs, TCG_REG_R8); + break; + case 'N': /* "Nine", r9 */ + ct->ct |= TCG_CT_REG; + tcg_regset_set_reg(ct->u.regs, TCG_REG_R9); + break; case 'q': /* A register that can be used as a byte operand. */ ct->ct |= TCG_CT_REG; From patchwork Mon Nov 12 21:44:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996677 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="AKB1dPJK"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4JJ49n1z9s3x for ; Tue, 13 Nov 2018 08:53:20 +1100 (AEDT) Received: from localhost ([::1]:50831 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK90-0000dr-56 for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 16:53:18 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54054) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK35-0003U8-4F for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK33-0004uR-MV for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:11 -0500 Received: from mail-wm1-x342.google.com ([2a00:1450:4864:20::342]:35985) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK33-0004uC-Fz for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:09 -0500 Received: by mail-wm1-x342.google.com with SMTP id s11so182944wmh.1 for ; Mon, 12 Nov 2018 13:47:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=E2IuiPUVHq7BkIlSxCBYJCqe6sIA1oFTu1TgtNdhAlU=; b=AKB1dPJKE8VGrTaT2Xy44zlL/qrMjn25BB6Xcj+X5WWv1r66HSUiuoVfWfYzIHh04Y zbfJwJHTP42KYZugP0CfvSE6UdzRsUOkJb0/6DEqQiD/UKLjebf06E9TYkxv26aR7VCh d5ansSDryudjK1rLi5ZwyK1hd5mZ55f4H0/J0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=E2IuiPUVHq7BkIlSxCBYJCqe6sIA1oFTu1TgtNdhAlU=; b=rKXBkWq1hv6Hx136BcJp5T0Hu6XE0oh0XXgrs/7mTJS8ILwMenZqDyVFzbVJ6Tdclx jGjw6HkQrshWZ677JVRn6jES65caZI1cKImVYZHrWy2FVNwge4rnSy9CiK3WjLAuOkRI tFsIsIBz1dTb+Ee4XXxQxSFNhNSLKvzw49j1dqR9T+V4nt7fyw6rcEau33drgxYmvoeS eX5F6UCsbUqI8rAtptAY0udlmh4lk2suCXBI7RJMdzg8f3vHSqG0Zx1f9wiSGff/Rda0 OUK1nFw34eBW7VGKuNVPPPfpEX+5ILPp+/mPblTZYuWTr31wIlqz2gfGwIZqIbG8eBa/ JSjg== X-Gm-Message-State: AGRZ1gJlm+v/QYSFbuKvgwZ3KjX4RFgRtKMlI2oLUsr63EgWngSLhrMH cRCsiAVfKVLa8UuN67xc9ljjzDAxKkaGkg== X-Google-Smtp-Source: AJdET5dDndpiCd+xzHeyIzoWYg8zE624KblO47psfmhjSAFmFKEiaOMY1B0AkVvP0yMdld+eGhkGqg== X-Received: by 2002:a1c:98d5:: with SMTP id a204-v6mr1063004wme.73.1542059228203; Mon, 12 Nov 2018 13:47:08 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.07 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:07 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:44:48 +0100 Message-Id: <20181112214503.22941-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::342 Subject: [Qemu-devel] [PATCH for-4.0 02/17] tcg/i386: Return a base register from tcg_out_tlb_load X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We will shortly be asking the hot path not to assume TCG_REG_L1 for the host base address. Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.inc.c | 56 ++++++++++++++++++++------------------- 1 file changed, 29 insertions(+), 27 deletions(-) diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c index e4d9be57ff..4435a7bb52 100644 --- a/tcg/i386/tcg-target.inc.c +++ b/tcg/i386/tcg-target.inc.c @@ -1611,9 +1611,9 @@ static void * const qemu_st_helpers[16] = { First argument register is clobbered. */ -static inline void tcg_out_tlb_load(TCGContext *s, TCGReg addrlo, TCGReg addrhi, - int mem_index, TCGMemOp opc, - tcg_insn_unit **label_ptr, int which) +static TCGReg tcg_out_tlb_load(TCGContext *s, TCGReg addrlo, TCGReg addrhi, + int mem_index, TCGMemOp opc, + tcg_insn_unit **label_ptr, int which) { const TCGReg r0 = TCG_REG_L0; const TCGReg r1 = TCG_REG_L1; @@ -1693,6 +1693,8 @@ static inline void tcg_out_tlb_load(TCGContext *s, TCGReg addrlo, TCGReg addrhi, /* add addend(r0), r1 */ tcg_out_modrm_offset(s, OPC_ADD_GvEv + hrexw, r1, r0, offsetof(CPUTLBEntry, addend) - which); + + return r1; } /* @@ -1998,10 +2000,6 @@ static void tcg_out_qemu_ld(TCGContext *s, const TCGArg *args, bool is64) TCGReg addrhi __attribute__((unused)); TCGMemOpIdx oi; TCGMemOp opc; -#if defined(CONFIG_SOFTMMU) - int mem_index; - tcg_insn_unit *label_ptr[2]; -#endif datalo = *args++; datahi = (TCG_TARGET_REG_BITS == 32 && is64 ? *args++ : 0); @@ -2011,17 +2009,21 @@ static void tcg_out_qemu_ld(TCGContext *s, const TCGArg *args, bool is64) opc = get_memop(oi); #if defined(CONFIG_SOFTMMU) - mem_index = get_mmuidx(oi); + { + int mem_index = get_mmuidx(oi); + tcg_insn_unit *label_ptr[2]; + TCGReg base; - tcg_out_tlb_load(s, addrlo, addrhi, mem_index, opc, - label_ptr, offsetof(CPUTLBEntry, addr_read)); + base = tcg_out_tlb_load(s, addrlo, addrhi, mem_index, opc, + label_ptr, offsetof(CPUTLBEntry, addr_read)); - /* TLB Hit. */ - tcg_out_qemu_ld_direct(s, datalo, datahi, TCG_REG_L1, -1, 0, 0, opc); + /* TLB Hit. */ + tcg_out_qemu_ld_direct(s, datalo, datahi, base, -1, 0, 0, opc); - /* Record the current context of a load into ldst label */ - add_qemu_ldst_label(s, true, oi, datalo, datahi, addrlo, addrhi, - s->code_ptr, label_ptr); + /* Record the current context of a load into ldst label */ + add_qemu_ldst_label(s, true, oi, datalo, datahi, addrlo, addrhi, + s->code_ptr, label_ptr); + } #else { int32_t offset = guest_base; @@ -2138,10 +2140,6 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) TCGReg addrhi __attribute__((unused)); TCGMemOpIdx oi; TCGMemOp opc; -#if defined(CONFIG_SOFTMMU) - int mem_index; - tcg_insn_unit *label_ptr[2]; -#endif datalo = *args++; datahi = (TCG_TARGET_REG_BITS == 32 && is64 ? *args++ : 0); @@ -2151,17 +2149,21 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) opc = get_memop(oi); #if defined(CONFIG_SOFTMMU) - mem_index = get_mmuidx(oi); + { + int mem_index = get_mmuidx(oi); + tcg_insn_unit *label_ptr[2]; + TCGReg base; - tcg_out_tlb_load(s, addrlo, addrhi, mem_index, opc, - label_ptr, offsetof(CPUTLBEntry, addr_write)); + base = tcg_out_tlb_load(s, addrlo, addrhi, mem_index, opc, + label_ptr, offsetof(CPUTLBEntry, addr_write)); - /* TLB Hit. */ - tcg_out_qemu_st_direct(s, datalo, datahi, TCG_REG_L1, 0, 0, opc); + /* TLB Hit. */ + tcg_out_qemu_st_direct(s, datalo, datahi, base, 0, 0, opc); - /* Record the current context of a store into ldst label */ - add_qemu_ldst_label(s, false, oi, datalo, datahi, addrlo, addrhi, - s->code_ptr, label_ptr); + /* Record the current context of a store into ldst label */ + add_qemu_ldst_label(s, false, oi, datalo, datahi, addrlo, addrhi, + s->code_ptr, label_ptr); + } #else { int32_t offset = guest_base; From patchwork Mon Nov 12 21:44:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996680 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="AY55slyz"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4MY52bQz9s3x for ; Tue, 13 Nov 2018 08:56:09 +1100 (AEDT) Received: from localhost ([::1]:50846 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMKBj-00036c-8K for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 16:56:07 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54068) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK35-0003UX-J1 for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK34-0004v1-PR for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:11 -0500 Received: from mail-wm1-x342.google.com ([2a00:1450:4864:20::342]:35160) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK34-0004ud-Il for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:10 -0500 Received: by mail-wm1-x342.google.com with SMTP id t15-v6so9576476wmt.0 for ; Mon, 12 Nov 2018 13:47:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=sXnNUWfT8Y/s5eYRqAv2JrCFIHDiypjLbLJHrnIH448=; b=AY55slyzpkJq0NNlpwAED5cGdCSfB8o3GBUbvCXE1t3NI70VOSPsNv9EwlGSaenq5c GoKH2eHIj1jPJeYCk0F+rU1IMLQxad0RpsoWmuRZr/1aGpK1rT6f72ZJp+9JrSVS6LsT aDRIqAvvORnTgFB6U9VwI6IgRqFOLWUoV+Eu4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=sXnNUWfT8Y/s5eYRqAv2JrCFIHDiypjLbLJHrnIH448=; b=DdXkJMxIcl73yy7fssexqAi4pT+RwKi2RQ/2Z8KNEVdDs/2bQRzJwcB2sKZqLWioVx Idbu5Cpl+qlF7G9dhpbeSIRVOwivbmjNKIqfKlVGlQUieyCiIitmuIFphX4tq851eQse YLmY4rg4ZIoz4lHpUWdM7nhyDLgbX4Pxm1VyEp+rICXqySohZBN/BDOzWSTxxrrhrybx ci+vVbIfiYEwFcIkUtDujw1GXhLLvPm0Q2uLomBeqG/yKAalmMsDYGJpqlbENd0JBKXT HK9GjNI8eEzN/eBhFetgJ99WnthAsMPJScSamsitwIbnzjXI6w0lkqk6YNKqFvob8ksZ BI2A== X-Gm-Message-State: AGRZ1gLiq0N6EJU3OswPiQiZnTOvVIEi+JsiuZaRPLbfQtjfTDgGxRsd RdoS53HDA748voIb2P9zcFGHeCW45efqXA== X-Google-Smtp-Source: AJdET5dBg2N6XSpPX1YyyeOFBjWESoPhjkxvoRxNQCgSHWo4MF9KUy+ZoessV4S0v7vdVC5lpIAYIg== X-Received: by 2002:a1c:ee13:: with SMTP id m19-v6mr1063288wmh.142.1542059229387; Mon, 12 Nov 2018 13:47:09 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.08 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:08 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:44:49 +0100 Message-Id: <20181112214503.22941-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::342 Subject: [Qemu-devel] [PATCH for-4.0 03/17] tcg/i386: Change TCG_REG_L[01] to not overlap function arguments X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We will shortly be forcing qemu_ld/st arguments into registers that match the function call abi of the host, which means that the temps must be elsewhere. Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.inc.c | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c index 4435a7bb52..2a96ca4274 100644 --- a/tcg/i386/tcg-target.inc.c +++ b/tcg/i386/tcg-target.inc.c @@ -121,12 +121,16 @@ static const int tcg_target_call_oarg_regs[] = { #define TCG_CT_CONST_I32 0x400 #define TCG_CT_CONST_WSZ 0x800 -/* Registers used with L constraint, which are the first argument - registers on x86_64, and two random call clobbered registers on - i386. */ +/* Registers used with L constraint, which are two random + * call clobbered registers. These should be free. + */ #if TCG_TARGET_REG_BITS == 64 -# define TCG_REG_L0 tcg_target_call_iarg_regs[0] -# define TCG_REG_L1 tcg_target_call_iarg_regs[1] +# define TCG_REG_L0 TCG_REG_RAX +# ifdef _WIN64 +# define TCG_REG_L1 TCG_REG_R10 +# else +# define TCG_REG_L1 TCG_REG_RDI +# endif #else # define TCG_REG_L0 TCG_REG_EAX # define TCG_REG_L1 TCG_REG_EDX @@ -1625,6 +1629,7 @@ static TCGReg tcg_out_tlb_load(TCGContext *s, TCGReg addrlo, TCGReg addrhi, unsigned a_mask = (1 << a_bits) - 1; unsigned s_mask = (1 << s_bits) - 1; target_ulong tlb_mask; + TCGReg base; if (TCG_TARGET_REG_BITS == 64) { if (TARGET_LONG_BITS == 64) { @@ -1671,7 +1676,12 @@ static TCGReg tcg_out_tlb_load(TCGContext *s, TCGReg addrlo, TCGReg addrhi, before the fastpath ADDQ below. For 64-bit guest and x32 host, MOVQ copies the entire guest address for the slow path, while truncation for the 32-bit host happens with the fastpath ADDL below. */ - tcg_out_mov(s, ttype, r1, addrlo); + if (TCG_TARGET_REG_BITS == 64) { + base = tcg_target_call_iarg_regs[1]; + } else { + base = r1; + } + tcg_out_mov(s, ttype, base, addrlo); /* jne slow_path */ tcg_out_opc(s, OPC_JCC_long + JCC_JNE, 0, 0, 0); @@ -1690,11 +1700,11 @@ static TCGReg tcg_out_tlb_load(TCGContext *s, TCGReg addrlo, TCGReg addrhi, /* TLB Hit. */ - /* add addend(r0), r1 */ - tcg_out_modrm_offset(s, OPC_ADD_GvEv + hrexw, r1, r0, + /* add addend(r0), base */ + tcg_out_modrm_offset(s, OPC_ADD_GvEv + hrexw, base, r0, offsetof(CPUTLBEntry, addend) - which); - return r1; + return base; } /* From patchwork Mon Nov 12 21:44:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996686 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="PJalXnPB"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4R65b45z9s3x for ; Tue, 13 Nov 2018 08:59:14 +1100 (AEDT) Received: from localhost ([::1]:50864 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMKEi-0005um-C8 for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 16:59:12 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54088) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK37-0003Ut-RH for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK36-0004wz-7o for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:13 -0500 Received: from mail-wm1-x32f.google.com ([2a00:1450:4864:20::32f]:51425) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK35-0004wd-Uh for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:12 -0500 Received: by mail-wm1-x32f.google.com with SMTP id w7-v6so9934060wmc.1 for ; Mon, 12 Nov 2018 13:47:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=F+bN6Efc2D8LKIXQoEGrW3N3PsuknhrH/ce++f9l7ts=; b=PJalXnPB3YAKUki1uCMEspBiyrUXkq8gyghGjIiP41rYMh8DXswoSF+0UDioRMc0ir 8wo3jUWNAToF+cXvs1AGegHCehptlsY5/aJYt33Tf6R1QhSrn4KGSqh9/F4p9XUSqI4N 0ICWQKVlmPdo9MJoaq86qg7yf+2dhEju6+uiE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=F+bN6Efc2D8LKIXQoEGrW3N3PsuknhrH/ce++f9l7ts=; b=OsYK3XQFQwH2ZTjs5VPJ07sk4uSbHkCr/ATVQmu2mKYjkODaDf57Xe9Ho7KJZjlP3R 9x92OR8NpmpLCxcwX7PhPcYY5RYRc0xnyJ4k0yXWatzuODJWPubc/mCJjcn5mai0DveQ uLckXQ6xg4TWQt8SghyiK9czLUpeUNjInL28RWtcJxpTLKGXXMKFGJayr+Dw1MuOG0wj PNU4w9vcLpxNrucFJjeWzqi+fEdkW9NWO9plvFcc+424MIOq9hahadZtj2fG/bwoXMAY 5auqPe0WRHGShItxNb+Z8N5aHoT4munsoMnenQ9dIh03ZS7RJq+eDQKkTOOBRQqvMVXt ozcg== X-Gm-Message-State: AGRZ1gKvhcdYExUB/wBjE8KKVXjBeAG6Hr1BFWp1+wRv3ceiOIkp+wn5 a9mju2rlHjaFSRpMCGBKGxDDe6A06ho9Wg== X-Google-Smtp-Source: AJdET5cpBRQUGSyK3TLGTMmbF+Zoeco6xj189P1S4qdnC/N/e+EgN6veheqDLUwNTKiUUMTJzYDtHQ== X-Received: by 2002:a7b:cd87:: with SMTP id y7-v6mr1066574wmj.110.1542059230625; Mon, 12 Nov 2018 13:47:10 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.09 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:10 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:44:50 +0100 Message-Id: <20181112214503.22941-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::32f Subject: [Qemu-devel] [PATCH for-4.0 04/17] tcg/i386: Force qemu_ld/st arguments into fixed registers X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This is an incremental step toward moving the qemu_ld/st code sequence out of line. Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.inc.c | 193 +++++++++++++++++++++++++++++++------- 1 file changed, 159 insertions(+), 34 deletions(-) diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c index 2a96ca4274..8a3e7690b6 100644 --- a/tcg/i386/tcg-target.inc.c +++ b/tcg/i386/tcg-target.inc.c @@ -171,6 +171,56 @@ static bool have_lzcnt; static tcg_insn_unit *tb_ret_addr; +#ifdef CONFIG_SOFTMMU +/* + * Constraint to choose a particular register. This is used for softmmu + * loads and stores. Registers with no assignment get an empty string. + */ +static const char * const one_reg_constraint[TCG_TARGET_NB_REGS] = { + [TCG_REG_EAX] = "a", + [TCG_REG_EBX] = "b", + [TCG_REG_ECX] = "c", + [TCG_REG_EDX] = "d", + [TCG_REG_ESI] = "S", + [TCG_REG_EDI] = "D", +#if TCG_TARGET_REG_BITS == 64 + [TCG_REG_R8] = "E", + [TCG_REG_R9] = "N", +#endif +}; + +/* + * Calling convention for the softmmu load and store thunks. + * + * For 64-bit, we mostly use the host calling convention, therefore the + * real first argument is reserved for the ENV parameter that is passed + * on to the slow path helpers. + * + * For 32-bit, the host calling convention is stack based; we invent a + * private convention that uses 4 of the 6 available host registers, and + * we reserve EAX and EDX as temporaries for use by the thunk. + */ +static inline TCGReg softmmu_arg(unsigned n) +{ + if (TCG_TARGET_REG_BITS == 64) { + tcg_debug_assert(n < ARRAY_SIZE(tcg_target_call_iarg_regs) - 1); + return tcg_target_call_iarg_regs[n + 1]; + } else { + static const TCGReg local_order[] = { + TCG_REG_ESI, TCG_REG_EDI, TCG_REG_ECX, TCG_REG_EBX + }; + tcg_debug_assert(n < ARRAY_SIZE(local_order)); + return local_order[n]; + } +} + +#define qemu_memop_arg(N) one_reg_constraint[softmmu_arg(N)] +#define qemu_memop_ret(N) (N ? "d" : "a") +#else +#define qemu_memop_arg(N) "L" +#define qemu_memop_ret(N) "L" +#endif /* CONFIG_SOFTMMU */ + static void patch_reloc(tcg_insn_unit *code_ptr, int type, intptr_t value, intptr_t addend) { @@ -1677,11 +1727,15 @@ static TCGReg tcg_out_tlb_load(TCGContext *s, TCGReg addrlo, TCGReg addrhi, copies the entire guest address for the slow path, while truncation for the 32-bit host happens with the fastpath ADDL below. */ if (TCG_TARGET_REG_BITS == 64) { - base = tcg_target_call_iarg_regs[1]; + tcg_debug_assert(addrlo == tcg_target_call_iarg_regs[1]); + if (TARGET_LONG_BITS == 32) { + tcg_out_ext32u(s, addrlo, addrlo); + } + base = addrlo; } else { base = r1; + tcg_out_mov(s, ttype, base, addrlo); } - tcg_out_mov(s, ttype, base, addrlo); /* jne slow_path */ tcg_out_opc(s, OPC_JCC_long + JCC_JNE, 0, 0, 0); @@ -2006,16 +2060,22 @@ static void tcg_out_qemu_ld_direct(TCGContext *s, TCGReg datalo, TCGReg datahi, common. */ static void tcg_out_qemu_ld(TCGContext *s, const TCGArg *args, bool is64) { - TCGReg datalo, datahi, addrlo; - TCGReg addrhi __attribute__((unused)); + TCGReg datalo, addrlo; + TCGReg datahi __attribute__((unused)) = -1; + TCGReg addrhi __attribute__((unused)) = -1; TCGMemOpIdx oi; TCGMemOp opc; + int i = -1; - datalo = *args++; - datahi = (TCG_TARGET_REG_BITS == 32 && is64 ? *args++ : 0); - addrlo = *args++; - addrhi = (TARGET_LONG_BITS > TCG_TARGET_REG_BITS ? *args++ : 0); - oi = *args++; + datalo = args[++i]; + if (TCG_TARGET_REG_BITS == 32 && is64) { + datahi = args[++i]; + } + addrlo = args[++i]; + if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { + addrhi = args[++i]; + } + oi = args[++i]; opc = get_memop(oi); #if defined(CONFIG_SOFTMMU) @@ -2024,6 +2084,15 @@ static void tcg_out_qemu_ld(TCGContext *s, const TCGArg *args, bool is64) tcg_insn_unit *label_ptr[2]; TCGReg base; + tcg_debug_assert(datalo == tcg_target_call_oarg_regs[0]); + if (TCG_TARGET_REG_BITS == 32 && is64) { + tcg_debug_assert(datahi == tcg_target_call_oarg_regs[1]); + } + tcg_debug_assert(addrlo == softmmu_arg(0)); + if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { + tcg_debug_assert(addrhi == softmmu_arg(1)); + } + base = tcg_out_tlb_load(s, addrlo, addrhi, mem_index, opc, label_ptr, offsetof(CPUTLBEntry, addr_read)); @@ -2146,16 +2215,22 @@ static void tcg_out_qemu_st_direct(TCGContext *s, TCGReg datalo, TCGReg datahi, static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) { - TCGReg datalo, datahi, addrlo; - TCGReg addrhi __attribute__((unused)); + TCGReg datalo, addrlo; + TCGReg datahi __attribute__((unused)) = -1; + TCGReg addrhi __attribute__((unused)) = -1; TCGMemOpIdx oi; TCGMemOp opc; + int i = -1; - datalo = *args++; - datahi = (TCG_TARGET_REG_BITS == 32 && is64 ? *args++ : 0); - addrlo = *args++; - addrhi = (TARGET_LONG_BITS > TCG_TARGET_REG_BITS ? *args++ : 0); - oi = *args++; + datalo = args[++i]; + if (TCG_TARGET_REG_BITS == 32 && is64) { + datahi = args[++i]; + } + addrlo = args[++i]; + if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { + addrhi = args[++i]; + } + oi = args[++i]; opc = get_memop(oi); #if defined(CONFIG_SOFTMMU) @@ -2164,6 +2239,16 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) tcg_insn_unit *label_ptr[2]; TCGReg base; + i = -1; + tcg_debug_assert(addrlo == softmmu_arg(++i)); + if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { + tcg_debug_assert(addrhi == softmmu_arg(++i)); + } + tcg_debug_assert(datalo == softmmu_arg(++i)); + if (TCG_TARGET_REG_BITS == 32 && is64) { + tcg_debug_assert(datahi == softmmu_arg(++i)); + } + base = tcg_out_tlb_load(s, addrlo, addrhi, mem_index, opc, label_ptr, offsetof(CPUTLBEntry, addr_write)); @@ -2833,15 +2918,6 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) static const TCGTargetOpDef r_r_re = { .args_ct_str = { "r", "r", "re" } }; static const TCGTargetOpDef r_0_re = { .args_ct_str = { "r", "0", "re" } }; static const TCGTargetOpDef r_0_ci = { .args_ct_str = { "r", "0", "ci" } }; - static const TCGTargetOpDef r_L = { .args_ct_str = { "r", "L" } }; - static const TCGTargetOpDef L_L = { .args_ct_str = { "L", "L" } }; - static const TCGTargetOpDef r_L_L = { .args_ct_str = { "r", "L", "L" } }; - static const TCGTargetOpDef r_r_L = { .args_ct_str = { "r", "r", "L" } }; - static const TCGTargetOpDef L_L_L = { .args_ct_str = { "L", "L", "L" } }; - static const TCGTargetOpDef r_r_L_L - = { .args_ct_str = { "r", "r", "L", "L" } }; - static const TCGTargetOpDef L_L_L_L - = { .args_ct_str = { "L", "L", "L", "L" } }; static const TCGTargetOpDef x_x = { .args_ct_str = { "x", "x" } }; static const TCGTargetOpDef x_x_x = { .args_ct_str = { "x", "x", "x" } }; static const TCGTargetOpDef x_x_x_x @@ -3023,17 +3099,66 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) } case INDEX_op_qemu_ld_i32: - return TARGET_LONG_BITS <= TCG_TARGET_REG_BITS ? &r_L : &r_L_L; - case INDEX_op_qemu_st_i32: - return TARGET_LONG_BITS <= TCG_TARGET_REG_BITS ? &L_L : &L_L_L; + { + static TCGTargetOpDef ld32; + ld32.args_ct_str[0] = qemu_memop_ret(0); + ld32.args_ct_str[1] = qemu_memop_arg(0); + if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { + ld32.args_ct_str[2] = qemu_memop_arg(1); + } + return &ld32; + } case INDEX_op_qemu_ld_i64: - return (TCG_TARGET_REG_BITS == 64 ? &r_L - : TARGET_LONG_BITS <= TCG_TARGET_REG_BITS ? &r_r_L - : &r_r_L_L); + { + static TCGTargetOpDef ld64; + if (TCG_TARGET_REG_BITS == 64) { + ld64.args_ct_str[0] = qemu_memop_ret(0); + ld64.args_ct_str[1] = qemu_memop_arg(0); + } else if (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS) { + ld64.args_ct_str[0] = qemu_memop_ret(0); + ld64.args_ct_str[1] = qemu_memop_ret(1); + ld64.args_ct_str[2] = qemu_memop_arg(0); + } else { + ld64.args_ct_str[0] = qemu_memop_ret(0); + ld64.args_ct_str[1] = qemu_memop_ret(1); + ld64.args_ct_str[2] = qemu_memop_arg(0); + ld64.args_ct_str[3] = qemu_memop_arg(1); + } + return &ld64; + } + + /* Recall the store value comes before addr in the opcode args + and after addr in helper args. */ + case INDEX_op_qemu_st_i32: + { + static TCGTargetOpDef st32; + st32.args_ct_str[1] = qemu_memop_arg(0); + if (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS) { + st32.args_ct_str[0] = qemu_memop_arg(1); + } else { + st32.args_ct_str[2] = qemu_memop_arg(1); + st32.args_ct_str[0] = qemu_memop_arg(2); + } + return &st32; + } case INDEX_op_qemu_st_i64: - return (TCG_TARGET_REG_BITS == 64 ? &L_L - : TARGET_LONG_BITS <= TCG_TARGET_REG_BITS ? &L_L_L - : &L_L_L_L); + { + static TCGTargetOpDef st64; + if (TCG_TARGET_REG_BITS == 64) { + st64.args_ct_str[1] = qemu_memop_arg(0); + st64.args_ct_str[0] = qemu_memop_arg(1); + } else if (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS) { + st64.args_ct_str[2] = qemu_memop_arg(0); + st64.args_ct_str[0] = qemu_memop_arg(1); + st64.args_ct_str[1] = qemu_memop_arg(2); + } else { + st64.args_ct_str[2] = qemu_memop_arg(0); + st64.args_ct_str[3] = qemu_memop_arg(1); + st64.args_ct_str[0] = qemu_memop_arg(2); + st64.args_ct_str[1] = qemu_memop_arg(3); + } + return &st64; + } case INDEX_op_brcond2_i32: { From patchwork Mon Nov 12 21:44:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996690 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="CahlkAsC"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4Vp1q1Kz9s0n for ; Tue, 13 Nov 2018 09:02:26 +1100 (AEDT) Received: from localhost ([::1]:50886 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMKHn-0008UM-Pd for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 17:02:23 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54112) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK3A-0003XX-4A for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:21 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK37-0004xf-TH for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:16 -0500 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]:52595) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK37-0004x8-Ax for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:13 -0500 Received: by mail-wm1-x344.google.com with SMTP id r11-v6so9923861wmb.2 for ; Mon, 12 Nov 2018 13:47:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=9CLSowoZnUuVZbZdyX6E+CD5XHRMY0jU9DtSq4NZ1ik=; b=CahlkAsCVVncGo3jn5QaIgl4IcDv24ronnA+yPfibhxlSRwVSqSaXvchI0sNsT0FNv Fp5hFjm2dMH4L9IoKcdghU/2QefUByjUbA6XWAuoaCUaN/3MpU9PcJJxR1214NQZl9Xm 5sI9/x0nCPW8CAiPYDh7mkzGP9ZhlKg9QRCs4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=9CLSowoZnUuVZbZdyX6E+CD5XHRMY0jU9DtSq4NZ1ik=; b=uZMnidd4OaVpbXzsIX6pWA3wmUST2iWghPjSsfDmkA9UnfaM3SrKiEXnEL/DodzCPj GOFEZsLCO6MdWKpcADFn48wqjb9a3FLj1C3zrAnXDqNkzh9XniytAQ+Xr0vXqwfdQQGv r6JKg3X7oYkEKQIGDRkaqXWE+tXoyKjA4DLFT8Tev8r4rKkrxNH+rcZJKXHWHf1eki+7 lJr83nB3pryfpMQiDx+rVMlxpSLSlVFWiCrWtqjTehvCH2afGCnFKhjrN3Z0fi6Rcowz /uAjfoQ2cwW5LfwWFaL/ciT3KcqtvqwAZdxA8DjrcT+fa0epqDB+DFXzS7LW99CR74Pe EO5A== X-Gm-Message-State: AGRZ1gKGa73ObITcJxh5E0oeqpFcK8Hyei2aM7e/GObp9Nem5Xe80LfI egKZ9/IwAUuKW5oBEi0jEg/BmIN+g9ErHg== X-Google-Smtp-Source: AJdET5dU6Hy8/OApeJSQmilRqdlbu0ehZ+2FYvRqeUUHHGyDexIA3DZADgOdu8Cqzqb0XHtOib82hg== X-Received: by 2002:a1c:410b:: with SMTP id o11-v6mr1138993wma.49.1542059231726; Mon, 12 Nov 2018 13:47:11 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:11 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:44:51 +0100 Message-Id: <20181112214503.22941-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::344 Subject: [Qemu-devel] [PATCH for-4.0 05/17] tcg: Return success from patch_reloc X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This moves the assert for success from inside patch_reloc to outside patch_reloc. This touches all tcg backends. Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.inc.c | 44 ++++++++++++++------------------- tcg/arm/tcg-target.inc.c | 26 +++++++++----------- tcg/i386/tcg-target.inc.c | 17 +++++++------ tcg/mips/tcg-target.inc.c | 29 +++++++++------------- tcg/ppc/tcg-target.inc.c | 47 ++++++++++++++++++++++-------------- tcg/s390/tcg-target.inc.c | 37 +++++++++++++++++++--------- tcg/sparc/tcg-target.inc.c | 13 ++++++---- tcg/tcg-pool.inc.c | 5 +++- tcg/tcg.c | 8 +++--- tcg/tci/tcg-target.inc.c | 3 ++- 10 files changed, 125 insertions(+), 104 deletions(-) diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c index 083592a4d7..30091f6a69 100644 --- a/tcg/aarch64/tcg-target.inc.c +++ b/tcg/aarch64/tcg-target.inc.c @@ -78,48 +78,40 @@ static const int tcg_target_call_oarg_regs[1] = { #define TCG_REG_GUEST_BASE TCG_REG_X28 #endif -static inline void reloc_pc26(tcg_insn_unit *code_ptr, tcg_insn_unit *target) +static inline bool reloc_pc26(tcg_insn_unit *code_ptr, tcg_insn_unit *target) { ptrdiff_t offset = target - code_ptr; - tcg_debug_assert(offset == sextract64(offset, 0, 26)); - /* read instruction, mask away previous PC_REL26 parameter contents, - set the proper offset, then write back the instruction. */ - *code_ptr = deposit32(*code_ptr, 0, 26, offset); + if (offset == sextract64(offset, 0, 26)) { + /* read instruction, mask away previous PC_REL26 parameter contents, + set the proper offset, then write back the instruction. */ + *code_ptr = deposit32(*code_ptr, 0, 26, offset); + return true; + } + return false; } -static inline void reloc_pc26_atomic(tcg_insn_unit *code_ptr, - tcg_insn_unit *target) +static inline bool reloc_pc19(tcg_insn_unit *code_ptr, tcg_insn_unit *target) { ptrdiff_t offset = target - code_ptr; - tcg_insn_unit insn; - tcg_debug_assert(offset == sextract64(offset, 0, 26)); - /* read instruction, mask away previous PC_REL26 parameter contents, - set the proper offset, then write back the instruction. */ - insn = atomic_read(code_ptr); - atomic_set(code_ptr, deposit32(insn, 0, 26, offset)); + if (offset == sextract64(offset, 0, 19)) { + *code_ptr = deposit32(*code_ptr, 5, 19, offset); + return true; + } + return false; } -static inline void reloc_pc19(tcg_insn_unit *code_ptr, tcg_insn_unit *target) -{ - ptrdiff_t offset = target - code_ptr; - tcg_debug_assert(offset == sextract64(offset, 0, 19)); - *code_ptr = deposit32(*code_ptr, 5, 19, offset); -} - -static inline void patch_reloc(tcg_insn_unit *code_ptr, int type, +static inline bool patch_reloc(tcg_insn_unit *code_ptr, int type, intptr_t value, intptr_t addend) { tcg_debug_assert(addend == 0); switch (type) { case R_AARCH64_JUMP26: case R_AARCH64_CALL26: - reloc_pc26(code_ptr, (tcg_insn_unit *)value); - break; + return reloc_pc26(code_ptr, (tcg_insn_unit *)value); case R_AARCH64_CONDBR19: - reloc_pc19(code_ptr, (tcg_insn_unit *)value); - break; + return reloc_pc19(code_ptr, (tcg_insn_unit *)value); default: - tcg_abort(); + g_assert_not_reached(); } } diff --git a/tcg/arm/tcg-target.inc.c b/tcg/arm/tcg-target.inc.c index e1fbf465cb..80d174ef44 100644 --- a/tcg/arm/tcg-target.inc.c +++ b/tcg/arm/tcg-target.inc.c @@ -187,27 +187,23 @@ static const uint8_t tcg_cond_to_arm_cond[] = { [TCG_COND_GTU] = COND_HI, }; -static inline void reloc_pc24(tcg_insn_unit *code_ptr, tcg_insn_unit *target) +static inline bool reloc_pc24(tcg_insn_unit *code_ptr, tcg_insn_unit *target) { ptrdiff_t offset = (tcg_ptr_byte_diff(target, code_ptr) - 8) >> 2; - *code_ptr = (*code_ptr & ~0xffffff) | (offset & 0xffffff); + if (offset == sextract32(offset, 0, 24)) { + *code_ptr = (*code_ptr & ~0xffffff) | (offset & 0xffffff); + return true; + } + return false; } -static inline void reloc_pc24_atomic(tcg_insn_unit *code_ptr, tcg_insn_unit *target) -{ - ptrdiff_t offset = (tcg_ptr_byte_diff(target, code_ptr) - 8) >> 2; - tcg_insn_unit insn = atomic_read(code_ptr); - tcg_debug_assert(offset == sextract32(offset, 0, 24)); - atomic_set(code_ptr, deposit32(insn, 0, 24, offset)); -} - -static void patch_reloc(tcg_insn_unit *code_ptr, int type, +static bool patch_reloc(tcg_insn_unit *code_ptr, int type, intptr_t value, intptr_t addend) { tcg_debug_assert(addend == 0); if (type == R_ARM_PC24) { - reloc_pc24(code_ptr, (tcg_insn_unit *)value); + return reloc_pc24(code_ptr, (tcg_insn_unit *)value); } else if (type == R_ARM_PC13) { intptr_t diff = value - (uintptr_t)(code_ptr + 2); tcg_insn_unit insn = *code_ptr; @@ -218,10 +214,9 @@ static void patch_reloc(tcg_insn_unit *code_ptr, int type, if (!u) { diff = -diff; } - } else { + } else if (diff >= 0x1000 && diff < 0x100000) { int rd = extract32(insn, 12, 4); int rt = rd == TCG_REG_PC ? TCG_REG_TMP : rd; - assert(diff >= 0x1000 && diff < 0x100000); /* add rt, pc, #high */ *code_ptr++ = ((insn & 0xf0000000) | (1 << 25) | ARITH_ADD | (TCG_REG_PC << 16) | (rt << 12) @@ -230,10 +225,13 @@ static void patch_reloc(tcg_insn_unit *code_ptr, int type, insn = deposit32(insn, 12, 4, rt); diff &= 0xfff; u = 1; + } else { + return false; } insn = deposit32(insn, 23, 1, u); insn = deposit32(insn, 0, 12, diff); *code_ptr = insn; + return true; } else { g_assert_not_reached(); } diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c index 8a3e7690b6..16d5af76ad 100644 --- a/tcg/i386/tcg-target.inc.c +++ b/tcg/i386/tcg-target.inc.c @@ -221,29 +221,32 @@ static inline TCGReg softmmu_arg(unsigned n) #define qemu_memop_ret(N) "L" #endif /* CONFIG_SOFTMMU */ -static void patch_reloc(tcg_insn_unit *code_ptr, int type, +static bool patch_reloc(tcg_insn_unit *code_ptr, int type, intptr_t value, intptr_t addend) { value += addend; - switch(type) { + + switch (type) { case R_386_PC32: value -= (uintptr_t)code_ptr; if (value != (int32_t)value) { - tcg_abort(); + return false; } /* FALLTHRU */ case R_386_32: tcg_patch32(code_ptr, value); - break; + return true; + case R_386_PC8: value -= (uintptr_t)code_ptr; if (value != (int8_t)value) { - tcg_abort(); + return false; } tcg_patch8(code_ptr, value); - break; + return true; + default: - tcg_abort(); + g_assert_not_reached(); } } diff --git a/tcg/mips/tcg-target.inc.c b/tcg/mips/tcg-target.inc.c index cff525373b..e59c66b607 100644 --- a/tcg/mips/tcg-target.inc.c +++ b/tcg/mips/tcg-target.inc.c @@ -144,36 +144,29 @@ static tcg_insn_unit *bswap32_addr; static tcg_insn_unit *bswap32u_addr; static tcg_insn_unit *bswap64_addr; -static inline uint32_t reloc_pc16_val(tcg_insn_unit *pc, tcg_insn_unit *target) +static bool reloc_pc16_cond(tcg_insn_unit *pc, tcg_insn_unit *target) { /* Let the compiler perform the right-shift as part of the arithmetic. */ ptrdiff_t disp = target - (pc + 1); - tcg_debug_assert(disp == (int16_t)disp); - return disp & 0xffff; + if (disp == (int16_t)disp) { + *pc = deposit32(*pc, 0, 16, disp); + return true; + } else { + return false; + } } -static inline void reloc_pc16(tcg_insn_unit *pc, tcg_insn_unit *target) +static bool reloc_pc16(tcg_insn_unit *pc, tcg_insn_unit *target) { - *pc = deposit32(*pc, 0, 16, reloc_pc16_val(pc, target)); + tcg_debug_assert(reloc_pc16_cond(pc, target)); } -static inline uint32_t reloc_26_val(tcg_insn_unit *pc, tcg_insn_unit *target) -{ - tcg_debug_assert((((uintptr_t)pc ^ (uintptr_t)target) & 0xf0000000) == 0); - return ((uintptr_t)target >> 2) & 0x3ffffff; -} - -static inline void reloc_26(tcg_insn_unit *pc, tcg_insn_unit *target) -{ - *pc = deposit32(*pc, 0, 26, reloc_26_val(pc, target)); -} - -static void patch_reloc(tcg_insn_unit *code_ptr, int type, +static bool patch_reloc(tcg_insn_unit *code_ptr, int type, intptr_t value, intptr_t addend) { tcg_debug_assert(type == R_MIPS_PC16); tcg_debug_assert(addend == 0); - reloc_pc16(code_ptr, (tcg_insn_unit *)value); + return reloc_pc16_cond(code_ptr, (tcg_insn_unit *)value); } #define TCG_CT_CONST_ZERO 0x100 diff --git a/tcg/ppc/tcg-target.inc.c b/tcg/ppc/tcg-target.inc.c index c2f729ee8f..656a9ff603 100644 --- a/tcg/ppc/tcg-target.inc.c +++ b/tcg/ppc/tcg-target.inc.c @@ -186,16 +186,14 @@ static inline bool in_range_b(tcg_target_long target) return target == sextract64(target, 0, 26); } -static uint32_t reloc_pc24_val(tcg_insn_unit *pc, tcg_insn_unit *target) +static bool reloc_pc24_cond(tcg_insn_unit *pc, tcg_insn_unit *target) { ptrdiff_t disp = tcg_ptr_byte_diff(target, pc); - tcg_debug_assert(in_range_b(disp)); - return disp & 0x3fffffc; -} - -static void reloc_pc24(tcg_insn_unit *pc, tcg_insn_unit *target) -{ - *pc = (*pc & ~0x3fffffc) | reloc_pc24_val(pc, target); + if (in_range_b(disp)) { + *pc = (*pc & ~0x3fffffc) | (disp & 0x3fffffc); + return true; + } + return false; } static uint16_t reloc_pc14_val(tcg_insn_unit *pc, tcg_insn_unit *target) @@ -205,10 +203,22 @@ static uint16_t reloc_pc14_val(tcg_insn_unit *pc, tcg_insn_unit *target) return disp & 0xfffc; } +static bool reloc_pc14_cond(tcg_insn_unit *pc, tcg_insn_unit *target) +{ + ptrdiff_t disp = tcg_ptr_byte_diff(target, pc); + if (disp == (int16_t) disp) { + *pc = (*pc & ~0xfffc) | (disp & 0xfffc); + return true; + } + return false; +} + +#ifdef CONFIG_SOFTMMU static void reloc_pc14(tcg_insn_unit *pc, tcg_insn_unit *target) { - *pc = (*pc & ~0xfffc) | reloc_pc14_val(pc, target); + tcg_debug_assert(reloc_pc14_cond(pc, target)); } +#endif static inline void tcg_out_b_noaddr(TCGContext *s, int insn) { @@ -525,7 +535,7 @@ static const uint32_t tcg_to_isel[] = { [TCG_COND_GTU] = ISEL | BC_(7, CR_GT), }; -static void patch_reloc(tcg_insn_unit *code_ptr, int type, +static bool patch_reloc(tcg_insn_unit *code_ptr, int type, intptr_t value, intptr_t addend) { tcg_insn_unit *target; @@ -536,11 +546,9 @@ static void patch_reloc(tcg_insn_unit *code_ptr, int type, switch (type) { case R_PPC_REL14: - reloc_pc14(code_ptr, target); - break; + return reloc_pc14_cond(code_ptr, target); case R_PPC_REL24: - reloc_pc24(code_ptr, target); - break; + return reloc_pc24_cond(code_ptr, target); case R_PPC_ADDR16: /* We are abusing this relocation type. This points to a pair of insns, addis + load. If the displacement is small, we @@ -552,11 +560,14 @@ static void patch_reloc(tcg_insn_unit *code_ptr, int type, } else { int16_t lo = value; int hi = value - lo; - assert(hi + lo == value); - code_ptr[0] = deposit32(code_ptr[0], 0, 16, hi >> 16); - code_ptr[1] = deposit32(code_ptr[1], 0, 16, lo); + if (hi + lo == value) { + code_ptr[0] = deposit32(code_ptr[0], 0, 16, hi >> 16); + code_ptr[1] = deposit32(code_ptr[1], 0, 16, lo); + } else { + return false; + } } - break; + return true; default: g_assert_not_reached(); } diff --git a/tcg/s390/tcg-target.inc.c b/tcg/s390/tcg-target.inc.c index 17c435ade5..a8d72dd630 100644 --- a/tcg/s390/tcg-target.inc.c +++ b/tcg/s390/tcg-target.inc.c @@ -366,7 +366,7 @@ static void * const qemu_st_helpers[16] = { static tcg_insn_unit *tb_ret_addr; uint64_t s390_facilities; -static void patch_reloc(tcg_insn_unit *code_ptr, int type, +static bool patch_reloc(tcg_insn_unit *code_ptr, int type, intptr_t value, intptr_t addend) { intptr_t pcrel2; @@ -377,22 +377,35 @@ static void patch_reloc(tcg_insn_unit *code_ptr, int type, switch (type) { case R_390_PC16DBL: - assert(pcrel2 == (int16_t)pcrel2); - tcg_patch16(code_ptr, pcrel2); + if (pcrel2 == (int16_t)pcrel2) { + tcg_patch16(code_ptr, pcrel2); + return true; + } break; case R_390_PC32DBL: - assert(pcrel2 == (int32_t)pcrel2); - tcg_patch32(code_ptr, pcrel2); + if (pcrel2 == (int32_t)pcrel2) { + tcg_patch32(code_ptr, pcrel2); + return true; + } break; case R_390_20: - assert(value == sextract64(value, 0, 20)); - old = *(uint32_t *)code_ptr & 0xf00000ff; - old |= ((value & 0xfff) << 16) | ((value & 0xff000) >> 4); - tcg_patch32(code_ptr, old); + if (value == sextract64(value, 0, 20)) { + old = *(uint32_t *)code_ptr & 0xf00000ff; + old |= ((value & 0xfff) << 16) | ((value & 0xff000) >> 4); + tcg_patch32(code_ptr, old); + return true; + } break; default: g_assert_not_reached(); } + return false; +} + +static void patch_reloc_force(tcg_insn_unit *code_ptr, int type, + intptr_t value, intptr_t addend) +{ + tcg_debug_assert(patch_reloc(code_ptr, type, value, addend)); } /* parse target specific constraints */ @@ -1618,7 +1631,8 @@ static void tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) TCGMemOpIdx oi = lb->oi; TCGMemOp opc = get_memop(oi); - patch_reloc(lb->label_ptr[0], R_390_PC16DBL, (intptr_t)s->code_ptr, 2); + patch_reloc_force(lb->label_ptr[0], R_390_PC16DBL, + (intptr_t)s->code_ptr, 2); tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_R2, TCG_AREG0); if (TARGET_LONG_BITS == 64) { @@ -1639,7 +1653,8 @@ static void tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) TCGMemOpIdx oi = lb->oi; TCGMemOp opc = get_memop(oi); - patch_reloc(lb->label_ptr[0], R_390_PC16DBL, (intptr_t)s->code_ptr, 2); + patch_reloc_force(lb->label_ptr[0], R_390_PC16DBL, + (intptr_t)s->code_ptr, 2); tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_R2, TCG_AREG0); if (TARGET_LONG_BITS == 64) { diff --git a/tcg/sparc/tcg-target.inc.c b/tcg/sparc/tcg-target.inc.c index 04bdc3df5e..111f3312d3 100644 --- a/tcg/sparc/tcg-target.inc.c +++ b/tcg/sparc/tcg-target.inc.c @@ -291,32 +291,34 @@ static inline int check_fit_i32(int32_t val, unsigned int bits) # define check_fit_ptr check_fit_i32 #endif -static void patch_reloc(tcg_insn_unit *code_ptr, int type, +static bool patch_reloc(tcg_insn_unit *code_ptr, int type, intptr_t value, intptr_t addend) { uint32_t insn = *code_ptr; intptr_t pcrel; + bool ret; value += addend; pcrel = tcg_ptr_byte_diff((tcg_insn_unit *)value, code_ptr); switch (type) { case R_SPARC_WDISP16: - assert(check_fit_ptr(pcrel >> 2, 16)); + ret = check_fit_ptr(pcrel >> 2, 16); insn &= ~INSN_OFF16(-1); insn |= INSN_OFF16(pcrel); break; case R_SPARC_WDISP19: - assert(check_fit_ptr(pcrel >> 2, 19)); + ret = check_fit_ptr(pcrel >> 2, 19); insn &= ~INSN_OFF19(-1); insn |= INSN_OFF19(pcrel); break; case R_SPARC_13: /* Note that we're abusing this reloc type for our own needs. */ + ret = true; if (!check_fit_ptr(value, 13)) { int adj = (value > 0 ? 0xff8 : -0x1000); value -= adj; - assert(check_fit_ptr(value, 13)); + ret = check_fit_ptr(value, 13); *code_ptr++ = (ARITH_ADD | INSN_RD(TCG_REG_T2) | INSN_RS1(TCG_REG_TB) | INSN_IMM13(adj)); insn ^= INSN_RS1(TCG_REG_TB) ^ INSN_RS1(TCG_REG_T2); @@ -328,12 +330,13 @@ static void patch_reloc(tcg_insn_unit *code_ptr, int type, /* Note that we're abusing this reloc type for our own needs. */ code_ptr[0] = deposit32(code_ptr[0], 0, 22, value >> 10); code_ptr[1] = deposit32(code_ptr[1], 0, 10, value); - return; + return value == (intptr_t)(uint32_t)value; default: g_assert_not_reached(); } *code_ptr = insn; + return ret; } /* parse target specific constraints */ diff --git a/tcg/tcg-pool.inc.c b/tcg/tcg-pool.inc.c index 7af5513ff3..ab8f6df8b0 100644 --- a/tcg/tcg-pool.inc.c +++ b/tcg/tcg-pool.inc.c @@ -140,6 +140,8 @@ static bool tcg_out_pool_finalize(TCGContext *s) for (; p != NULL; p = p->next) { size_t size = sizeof(tcg_target_ulong) * p->nlong; + bool ok; + if (!l || l->nlong != p->nlong || memcmp(l->data, p->data, size)) { if (unlikely(a > s->code_gen_highwater)) { return false; @@ -148,7 +150,8 @@ static bool tcg_out_pool_finalize(TCGContext *s) a += size; l = p; } - patch_reloc(p->label, p->rtype, (intptr_t)a - size, p->addend); + ok = patch_reloc(p->label, p->rtype, (intptr_t)a - size, p->addend); + tcg_debug_assert(ok); } s->code_ptr = a; diff --git a/tcg/tcg.c b/tcg/tcg.c index e85133ef05..54f1272187 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -66,7 +66,7 @@ static void tcg_target_init(TCGContext *s); static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode); static void tcg_target_qemu_prologue(TCGContext *s); -static void patch_reloc(tcg_insn_unit *code_ptr, int type, +static bool patch_reloc(tcg_insn_unit *code_ptr, int type, intptr_t value, intptr_t addend); /* The CIE and FDE header definitions will be common to all hosts. */ @@ -268,7 +268,8 @@ static void tcg_out_reloc(TCGContext *s, tcg_insn_unit *code_ptr, int type, /* FIXME: This may break relocations on RISC targets that modify instruction fields in place. The caller may not have written the initial value. */ - patch_reloc(code_ptr, type, l->u.value, addend); + bool ok = patch_reloc(code_ptr, type, l->u.value, addend); + tcg_debug_assert(ok); } else { /* add a new relocation entry */ r = tcg_malloc(sizeof(TCGRelocation)); @@ -288,7 +289,8 @@ static void tcg_out_label(TCGContext *s, TCGLabel *l, tcg_insn_unit *ptr) tcg_debug_assert(!l->has_value); for (r = l->u.first_reloc; r != NULL; r = r->next) { - patch_reloc(r->ptr, r->type, value, r->addend); + bool ok = patch_reloc(r->ptr, r->type, value, r->addend); + tcg_debug_assert(ok); } l->has_value = 1; diff --git a/tcg/tci/tcg-target.inc.c b/tcg/tci/tcg-target.inc.c index 62ed097254..0015a98485 100644 --- a/tcg/tci/tcg-target.inc.c +++ b/tcg/tci/tcg-target.inc.c @@ -369,7 +369,7 @@ static const char *const tcg_target_reg_names[TCG_TARGET_NB_REGS] = { }; #endif -static void patch_reloc(tcg_insn_unit *code_ptr, int type, +static bool patch_reloc(tcg_insn_unit *code_ptr, int type, intptr_t value, intptr_t addend) { /* tcg_out_reloc always uses the same type, addend. */ @@ -381,6 +381,7 @@ static void patch_reloc(tcg_insn_unit *code_ptr, int type, } else { tcg_patch64(code_ptr, value); } + return true; } /* Parse target specific constraints. */ From patchwork Mon Nov 12 21:44:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996684 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="O5UFjoZy"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4Qq6F02z9s3x for ; Tue, 13 Nov 2018 08:58:58 +1100 (AEDT) Received: from localhost ([::1]:50860 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMKER-0005g6-5p for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 16:58:55 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54110) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK39-0003X2-I9 for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK38-0004xl-G5 for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:15 -0500 Received: from mail-wm1-x341.google.com ([2a00:1450:4864:20::341]:51118) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK38-0004xQ-9h for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:14 -0500 Received: by mail-wm1-x341.google.com with SMTP id 124-v6so10004274wmw.0 for ; Mon, 12 Nov 2018 13:47:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=8y1UmhfH9rH2EQVi6l23g/hflBkPX47jvRyrizN6T+o=; b=O5UFjoZyoUOIAjKQXiijFiqY67oN8Tp260SE4x3cqORaoxFTBFhN/1T7WMGI3Ljr+t lBalSG5tu9uvlzEIX61i1yDVHbBhIarIB9R9ZRYLBtONX2bsPMX9mrL8/xf1CgOfnWDQ 5JxTnzQObmk6phmk2rGiErZ9ReBr0gTv0BikI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=8y1UmhfH9rH2EQVi6l23g/hflBkPX47jvRyrizN6T+o=; b=lvZDau8kzGKaPOBcRrWxaCDkHMSWVSZE71b7Jm/zH8cIMsvDlDaRtYOsNU+Z+Pgnhu oefcHe70EVmfuEag+1BPanrcDKh84D2BcjbHRieG8TZv6wCSPIuIt4Qgobl3+rCrmfoe 5TG8Iz6CZ07frP1fL8VY6rl8mQnoZe/d7XwkcMDeU1O4YSa0PDU6Y6wdHiZb0v5JdHYn G1PHFujXCQS9CV556110l5CS3ydl2bcYEW8k+vAXnZDKNdmWgID4r5rdxlIYMrp8xzbt VU0Nmfd5rR07upBbVwlpAxdR5V14DOQ7/NjUu3HNCocqjyWsdYtzrW2XuXkm+fNtHLFi fBIQ== X-Gm-Message-State: AGRZ1gJkWl9aWKu0KjzGZcG6HPGVOHhDHtML9axIYf/cq/4rKurQGQ/8 TOuGpiGNtfdAH3useqRi6Uddzastr1BxxA== X-Google-Smtp-Source: AJdET5dhIRCS3KZIH/4c5VEMgMtVNJZaodnT9F+Vx6MzTmKdT1O1nAtdrh6IT+LcIq1N53pAlcLcsQ== X-Received: by 2002:a1c:154a:: with SMTP id 71-v6mr1027322wmv.83.1542059232984; Mon, 12 Nov 2018 13:47:12 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.11 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:12 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:44:52 +0100 Message-Id: <20181112214503.22941-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::341 Subject: [Qemu-devel] [PATCH for-4.0 06/17] tcg: Add TCG_TARGET_NEED_LDST_OOL_LABELS X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This variant of tcg-ldst.inc.c allows the entire thunk to be moved out-of-line, with caching across TBs within a region. Signed-off-by: Richard Henderson --- tcg/tcg.h | 4 ++ tcg/tcg-ldst-ool.inc.c | 94 ++++++++++++++++++++++++++++++++++++++++++ tcg/tcg.c | 20 +++++++++ 3 files changed, 118 insertions(+) create mode 100644 tcg/tcg-ldst-ool.inc.c diff --git a/tcg/tcg.h b/tcg/tcg.h index f4efbaa680..1255d2a2c6 100644 --- a/tcg/tcg.h +++ b/tcg/tcg.h @@ -706,6 +706,10 @@ struct TCGContext { #ifdef TCG_TARGET_NEED_LDST_LABELS QSIMPLEQ_HEAD(ldst_labels, TCGLabelQemuLdst) ldst_labels; #endif +#ifdef TCG_TARGET_NEED_LDST_OOL_LABELS + QSIMPLEQ_HEAD(ldst_labels, TCGLabelQemuLdstOol) ldst_ool_labels; + GHashTable *ldst_ool_thunks; +#endif #ifdef TCG_TARGET_NEED_POOL_LABELS struct TCGLabelPoolData *pool_labels; #endif diff --git a/tcg/tcg-ldst-ool.inc.c b/tcg/tcg-ldst-ool.inc.c new file mode 100644 index 0000000000..8fb6550a8d --- /dev/null +++ b/tcg/tcg-ldst-ool.inc.c @@ -0,0 +1,94 @@ +/* + * TCG Backend Data: load-store optimization only. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +typedef struct TCGLabelQemuLdstOol { + QSIMPLEQ_ENTRY(TCGLabelQemuLdstOol) next; + tcg_insn_unit *label; /* label pointer to be updated */ + int reloc; /* relocation type from label_ptr */ + intptr_t addend; /* relocation addend from label_ptr */ + uint32_t key; /* oi : is_64 : is_ld */ +} TCGLabelQemuLdstOol; + + +/* + * Generate TB finalization at the end of block + */ + +static tcg_insn_unit *tcg_out_qemu_ldst_ool(TCGContext *s, bool is_ld, + bool is64, TCGMemOpIdx oi); + +static bool tcg_out_ldst_ool_finalize(TCGContext *s) +{ + TCGLabelQemuLdstOol *lb; + + /* qemu_ld/st slow paths */ + QSIMPLEQ_FOREACH(lb, &s->ldst_ool_labels, next) { + gpointer dest, key = (gpointer)(uintptr_t)lb->key; + TCGMemOpIdx oi; + bool is_ld, is_64, ok; + + /* If we have generated the thunk, and it's still in range, all ok. */ + dest = g_hash_table_lookup(s->ldst_ool_thunks, key); + if (dest && + patch_reloc(lb->label, lb->reloc, (intptr_t)dest, lb->addend)) { + continue; + } + + /* Generate a new thunk. */ + is_ld = extract32(lb->key, 0, 1); + is_64 = extract32(lb->key, 1, 1); + oi = extract32(lb->key, 2, 30); + dest = tcg_out_qemu_ldst_ool(s, is_ld, is_64, oi); + + /* Test for (pending) buffer overflow. The assumption is that any + one thunk beginning below the high water mark cannot overrun + the buffer completely. Thus we can test for overflow after + generating code without having to check during generation. */ + if (unlikely((void *)s->code_ptr > s->code_gen_highwater)) { + return false; + } + + /* Remember the thunk for next time. */ + g_hash_table_replace(s->ldst_ool_thunks, key, dest); + + /* The new thunk must be in range. */ + ok = patch_reloc(lb->label, lb->reloc, (intptr_t)dest, lb->addend); + tcg_debug_assert(ok); + } + return true; +} + +/* + * Allocate a new TCGLabelQemuLdstOol entry. + */ + +static void add_ldst_ool_label(TCGContext *s, bool is_ld, bool is_64, + TCGMemOpIdx oi, int reloc, intptr_t addend) +{ + TCGLabelQemuLdstOol *lb = tcg_malloc(sizeof(*lb)); + + QSIMPLEQ_INSERT_TAIL(&s->ldst_ool_labels, lb, next); + lb->label = s->code_ptr; + lb->reloc = reloc; + lb->addend = addend; + lb->key = is_ld | (is_64 << 1) | (oi << 2); +} diff --git a/tcg/tcg.c b/tcg/tcg.c index 54f1272187..885d842a12 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -521,6 +521,13 @@ static void tcg_region_assign(TCGContext *s, size_t curr_region) s->code_gen_ptr = start; s->code_gen_buffer_size = end - start; s->code_gen_highwater = end - TCG_HIGHWATER; + +#ifdef TCG_TARGET_NEED_LDST_OOL_LABELS + /* No thunks yet generated this region. Even if they were in range, + this is also the most convenient place to clear the table after a + full tb_flush. */ + g_hash_table_remove_all(s->ldst_ool_thunks); +#endif } static bool tcg_region_alloc__locked(TCGContext *s) @@ -964,6 +971,11 @@ void tcg_context_init(TCGContext *s) tcg_debug_assert(!tcg_regset_test_reg(s->reserved_regs, TCG_AREG0)); ts = tcg_global_reg_new_internal(s, TCG_TYPE_PTR, TCG_AREG0, "env"); cpu_env = temp_tcgv_ptr(ts); + +#ifdef TCG_TARGET_NEED_LDST_OOL_LABELS + /* Both key and value are raw pointers. */ + s->ldst_ool_thunks = g_hash_table_new(NULL, NULL); +#endif } /* @@ -3540,6 +3552,9 @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb) #ifdef TCG_TARGET_NEED_LDST_LABELS QSIMPLEQ_INIT(&s->ldst_labels); #endif +#ifdef TCG_TARGET_NEED_LDST_OOL_LABELS + QSIMPLEQ_INIT(&s->ldst_ool_labels); +#endif #ifdef TCG_TARGET_NEED_POOL_LABELS s->pool_labels = NULL; #endif @@ -3620,6 +3635,11 @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb) return -1; } #endif +#ifdef TCG_TARGET_NEED_LDST_OOL_LABELS + if (!tcg_out_ldst_ool_finalize(s)) { + return -1; + } +#endif #ifdef TCG_TARGET_NEED_POOL_LABELS if (!tcg_out_pool_finalize(s)) { return -1; From patchwork Mon Nov 12 21:44:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996697 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="eINCV0LV"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4f54Mplz9s7W for ; Tue, 13 Nov 2018 09:08:45 +1100 (AEDT) Received: from localhost ([::1]:50925 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMKNv-00012I-5x for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 17:08:43 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54157) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK3E-0003ZW-L0 for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK3A-0004yo-Cy for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:19 -0500 Received: from mail-wm1-x343.google.com ([2a00:1450:4864:20::343]:39652) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK3A-0004y4-3X for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:16 -0500 Received: by mail-wm1-x343.google.com with SMTP id u13-v6so9546963wmc.4 for ; Mon, 12 Nov 2018 13:47:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=pv5fMQE1MZbTmrKxWhZeXVDA5tlPSXMWcog46qmLLiI=; b=eINCV0LVSJgv9pZoGefOGWTCUc5PAVPGEqdJfB/QrNPB+XO/YCpoUj/0sff7mAah+n 48kyO+uTj72lBCU6ybOL3W4s58HUTLUrbzJW3No2IyhG8nrzAyw6eeTNxa0uKw+JogFK 2QPG2Uvnit6q8mSlInB2dggGdpyEzved9qP/Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=pv5fMQE1MZbTmrKxWhZeXVDA5tlPSXMWcog46qmLLiI=; b=J6n73ueXz8bQXqV2a/RjLNVJkcL4LWLXIAYQA5I8n++bBIfGuBpKrZVGOE3js3BkfL +b8lpyKYyBOimSJ15szYe0Yyxh3CfoCFJWWqJNXiNxqnQhZR35QfLLH6tpnB6WFXuevF 3xQQVMr12SBEYnj3XaZA0FoawiRY+xw7XzgsqDp6Gt4DpbguCstC3GfNCaFd1or2MHQf 0vGzLxDrMX0IYRxMh6fGuptzmLwtq1c+tq76ZPjZk4vjlbGA813ueyMQ7GLbFlEugoyR fSpde0HumkKtQlYCCNBPa8/aQ0Y7wBrRD4QcNs6lYjhCtReVpffSe2102IeyEcypacLY uE2A== X-Gm-Message-State: AGRZ1gJ/imIw/ocbKsnnm0Sc8HkxW85UG2FVzXOnkTH7Kxnm/av5ikLA +KfMh6gGdtfrzvChm88+0Ap5KCx6C1C3Yw== X-Google-Smtp-Source: AJdET5cZ4TNrtBszUJXZqz+662PZNlCh64c1xY6mqKB8m2vvJ9zAXULayicF55zP6QV/8tnaAOpwQw== X-Received: by 2002:a1c:ce8d:: with SMTP id e135-v6mr1047344wmg.3.1542059234258; Mon, 12 Nov 2018 13:47:14 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.13 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:13 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:44:53 +0100 Message-Id: <20181112214503.22941-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::343 Subject: [Qemu-devel] [PATCH for-4.0 07/17] tcg/i386: Use TCG_TARGET_NEED_LDST_OOL_LABELS X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Move the entire memory operation out of line. Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.h | 2 +- tcg/i386/tcg-target.inc.c | 401 ++++++++++++++++---------------------- 2 files changed, 171 insertions(+), 232 deletions(-) diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index 9fdf37f23c..c2d84cf1d2 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -224,7 +224,7 @@ static inline void tb_target_set_jmp_target(uintptr_t tc_ptr, #define TCG_TARGET_DEFAULT_MO (TCG_MO_ALL & ~TCG_MO_ST_LD) #ifdef CONFIG_SOFTMMU -#define TCG_TARGET_NEED_LDST_LABELS +#define TCG_TARGET_NEED_LDST_OOL_LABELS #endif #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c index 16d5af76ad..1833f4c2b2 100644 --- a/tcg/i386/tcg-target.inc.c +++ b/tcg/i386/tcg-target.inc.c @@ -1619,7 +1619,7 @@ static void tcg_out_nopn(TCGContext *s, int n) } #if defined(CONFIG_SOFTMMU) -#include "tcg-ldst.inc.c" +#include "tcg-ldst-ool.inc.c" /* helper signature: helper_ret_ld_mmu(CPUState *env, target_ulong addr, * int mmu_idx, uintptr_t ra) @@ -1632,6 +1632,14 @@ static void * const qemu_ld_helpers[16] = { [MO_BEUW] = helper_be_lduw_mmu, [MO_BEUL] = helper_be_ldul_mmu, [MO_BEQ] = helper_be_ldq_mmu, + + [MO_SB] = helper_ret_ldsb_mmu, + [MO_LESW] = helper_le_ldsw_mmu, + [MO_BESW] = helper_be_ldsw_mmu, +#if TCG_TARGET_REG_BITS == 64 + [MO_LESL] = helper_le_ldsl_mmu, + [MO_BESL] = helper_be_ldsl_mmu, +#endif }; /* helper signature: helper_ret_st_mmu(CPUState *env, target_ulong addr, @@ -1741,18 +1749,18 @@ static TCGReg tcg_out_tlb_load(TCGContext *s, TCGReg addrlo, TCGReg addrhi, } /* jne slow_path */ - tcg_out_opc(s, OPC_JCC_long + JCC_JNE, 0, 0, 0); + tcg_out_opc(s, OPC_JCC_short + JCC_JNE, 0, 0, 0); label_ptr[0] = s->code_ptr; - s->code_ptr += 4; + s->code_ptr += 1; if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { /* cmp 4(r0), addrhi */ tcg_out_modrm_offset(s, OPC_CMP_GvEv, addrhi, r0, 4); /* jne slow_path */ - tcg_out_opc(s, OPC_JCC_long + JCC_JNE, 0, 0, 0); + tcg_out_opc(s, OPC_JCC_short + JCC_JNE, 0, 0, 0); label_ptr[1] = s->code_ptr; - s->code_ptr += 4; + s->code_ptr += 1; } /* TLB Hit. */ @@ -1764,181 +1772,6 @@ static TCGReg tcg_out_tlb_load(TCGContext *s, TCGReg addrlo, TCGReg addrhi, return base; } -/* - * Record the context of a call to the out of line helper code for the slow path - * for a load or store, so that we can later generate the correct helper code - */ -static void add_qemu_ldst_label(TCGContext *s, bool is_ld, TCGMemOpIdx oi, - TCGReg datalo, TCGReg datahi, - TCGReg addrlo, TCGReg addrhi, - tcg_insn_unit *raddr, - tcg_insn_unit **label_ptr) -{ - TCGLabelQemuLdst *label = new_ldst_label(s); - - label->is_ld = is_ld; - label->oi = oi; - label->datalo_reg = datalo; - label->datahi_reg = datahi; - label->addrlo_reg = addrlo; - label->addrhi_reg = addrhi; - label->raddr = raddr; - label->label_ptr[0] = label_ptr[0]; - if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { - label->label_ptr[1] = label_ptr[1]; - } -} - -/* - * Generate code for the slow path for a load at the end of block - */ -static void tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - TCGMemOpIdx oi = l->oi; - TCGMemOp opc = get_memop(oi); - TCGReg data_reg; - tcg_insn_unit **label_ptr = &l->label_ptr[0]; - - /* resolve label address */ - tcg_patch32(label_ptr[0], s->code_ptr - label_ptr[0] - 4); - if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { - tcg_patch32(label_ptr[1], s->code_ptr - label_ptr[1] - 4); - } - - if (TCG_TARGET_REG_BITS == 32) { - int ofs = 0; - - tcg_out_st(s, TCG_TYPE_PTR, TCG_AREG0, TCG_REG_ESP, ofs); - ofs += 4; - - tcg_out_st(s, TCG_TYPE_I32, l->addrlo_reg, TCG_REG_ESP, ofs); - ofs += 4; - - if (TARGET_LONG_BITS == 64) { - tcg_out_st(s, TCG_TYPE_I32, l->addrhi_reg, TCG_REG_ESP, ofs); - ofs += 4; - } - - tcg_out_sti(s, TCG_TYPE_I32, oi, TCG_REG_ESP, ofs); - ofs += 4; - - tcg_out_sti(s, TCG_TYPE_PTR, (uintptr_t)l->raddr, TCG_REG_ESP, ofs); - } else { - tcg_out_mov(s, TCG_TYPE_PTR, tcg_target_call_iarg_regs[0], TCG_AREG0); - /* The second argument is already loaded with addrlo. */ - tcg_out_movi(s, TCG_TYPE_I32, tcg_target_call_iarg_regs[2], oi); - tcg_out_movi(s, TCG_TYPE_PTR, tcg_target_call_iarg_regs[3], - (uintptr_t)l->raddr); - } - - tcg_out_call(s, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); - - data_reg = l->datalo_reg; - switch (opc & MO_SSIZE) { - case MO_SB: - tcg_out_ext8s(s, data_reg, TCG_REG_EAX, P_REXW); - break; - case MO_SW: - tcg_out_ext16s(s, data_reg, TCG_REG_EAX, P_REXW); - break; -#if TCG_TARGET_REG_BITS == 64 - case MO_SL: - tcg_out_ext32s(s, data_reg, TCG_REG_EAX); - break; -#endif - case MO_UB: - case MO_UW: - /* Note that the helpers have zero-extended to tcg_target_long. */ - case MO_UL: - tcg_out_mov(s, TCG_TYPE_I32, data_reg, TCG_REG_EAX); - break; - case MO_Q: - if (TCG_TARGET_REG_BITS == 64) { - tcg_out_mov(s, TCG_TYPE_I64, data_reg, TCG_REG_RAX); - } else if (data_reg == TCG_REG_EDX) { - /* xchg %edx, %eax */ - tcg_out_opc(s, OPC_XCHG_ax_r32 + TCG_REG_EDX, 0, 0, 0); - tcg_out_mov(s, TCG_TYPE_I32, l->datahi_reg, TCG_REG_EAX); - } else { - tcg_out_mov(s, TCG_TYPE_I32, data_reg, TCG_REG_EAX); - tcg_out_mov(s, TCG_TYPE_I32, l->datahi_reg, TCG_REG_EDX); - } - break; - default: - tcg_abort(); - } - - /* Jump to the code corresponding to next IR of qemu_st */ - tcg_out_jmp(s, l->raddr); -} - -/* - * Generate code for the slow path for a store at the end of block - */ -static void tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - TCGMemOpIdx oi = l->oi; - TCGMemOp opc = get_memop(oi); - TCGMemOp s_bits = opc & MO_SIZE; - tcg_insn_unit **label_ptr = &l->label_ptr[0]; - TCGReg retaddr; - - /* resolve label address */ - tcg_patch32(label_ptr[0], s->code_ptr - label_ptr[0] - 4); - if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { - tcg_patch32(label_ptr[1], s->code_ptr - label_ptr[1] - 4); - } - - if (TCG_TARGET_REG_BITS == 32) { - int ofs = 0; - - tcg_out_st(s, TCG_TYPE_PTR, TCG_AREG0, TCG_REG_ESP, ofs); - ofs += 4; - - tcg_out_st(s, TCG_TYPE_I32, l->addrlo_reg, TCG_REG_ESP, ofs); - ofs += 4; - - if (TARGET_LONG_BITS == 64) { - tcg_out_st(s, TCG_TYPE_I32, l->addrhi_reg, TCG_REG_ESP, ofs); - ofs += 4; - } - - tcg_out_st(s, TCG_TYPE_I32, l->datalo_reg, TCG_REG_ESP, ofs); - ofs += 4; - - if (s_bits == MO_64) { - tcg_out_st(s, TCG_TYPE_I32, l->datahi_reg, TCG_REG_ESP, ofs); - ofs += 4; - } - - tcg_out_sti(s, TCG_TYPE_I32, oi, TCG_REG_ESP, ofs); - ofs += 4; - - retaddr = TCG_REG_EAX; - tcg_out_movi(s, TCG_TYPE_PTR, retaddr, (uintptr_t)l->raddr); - tcg_out_st(s, TCG_TYPE_PTR, retaddr, TCG_REG_ESP, ofs); - } else { - tcg_out_mov(s, TCG_TYPE_PTR, tcg_target_call_iarg_regs[0], TCG_AREG0); - /* The second argument is already loaded with addrlo. */ - tcg_out_mov(s, (s_bits == MO_64 ? TCG_TYPE_I64 : TCG_TYPE_I32), - tcg_target_call_iarg_regs[2], l->datalo_reg); - tcg_out_movi(s, TCG_TYPE_I32, tcg_target_call_iarg_regs[3], oi); - - if (ARRAY_SIZE(tcg_target_call_iarg_regs) > 4) { - retaddr = tcg_target_call_iarg_regs[4]; - tcg_out_movi(s, TCG_TYPE_PTR, retaddr, (uintptr_t)l->raddr); - } else { - retaddr = TCG_REG_RAX; - tcg_out_movi(s, TCG_TYPE_PTR, retaddr, (uintptr_t)l->raddr); - tcg_out_st(s, TCG_TYPE_PTR, retaddr, TCG_REG_ESP, - TCG_TARGET_CALL_STACK_OFFSET); - } - } - - /* "Tail call" to the helper, with the return address back inline. */ - tcg_out_push(s, retaddr); - tcg_out_jmp(s, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); -} #elif defined(__x86_64__) && defined(__linux__) # include # include @@ -2067,7 +1900,6 @@ static void tcg_out_qemu_ld(TCGContext *s, const TCGArg *args, bool is64) TCGReg datahi __attribute__((unused)) = -1; TCGReg addrhi __attribute__((unused)) = -1; TCGMemOpIdx oi; - TCGMemOp opc; int i = -1; datalo = args[++i]; @@ -2079,35 +1911,25 @@ static void tcg_out_qemu_ld(TCGContext *s, const TCGArg *args, bool is64) addrhi = args[++i]; } oi = args[++i]; - opc = get_memop(oi); #if defined(CONFIG_SOFTMMU) - { - int mem_index = get_mmuidx(oi); - tcg_insn_unit *label_ptr[2]; - TCGReg base; - - tcg_debug_assert(datalo == tcg_target_call_oarg_regs[0]); - if (TCG_TARGET_REG_BITS == 32 && is64) { - tcg_debug_assert(datahi == tcg_target_call_oarg_regs[1]); - } - tcg_debug_assert(addrlo == softmmu_arg(0)); - if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { - tcg_debug_assert(addrhi == softmmu_arg(1)); - } - - base = tcg_out_tlb_load(s, addrlo, addrhi, mem_index, opc, - label_ptr, offsetof(CPUTLBEntry, addr_read)); - - /* TLB Hit. */ - tcg_out_qemu_ld_direct(s, datalo, datahi, base, -1, 0, 0, opc); - - /* Record the current context of a load into ldst label */ - add_qemu_ldst_label(s, true, oi, datalo, datahi, addrlo, addrhi, - s->code_ptr, label_ptr); + /* Assert that we've set up the constraints properly. */ + tcg_debug_assert(datalo == tcg_target_call_oarg_regs[0]); + if (TCG_TARGET_REG_BITS == 32 && is64) { + tcg_debug_assert(datahi == tcg_target_call_oarg_regs[1]); } + tcg_debug_assert(addrlo == softmmu_arg(0)); + if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { + tcg_debug_assert(addrhi == softmmu_arg(1)); + } + + /* Call to thunk. */ + tcg_out8(s, OPC_CALL_Jz); + add_ldst_ool_label(s, true, is64, oi, R_386_PC32, -4); + s->code_ptr += 4; #else { + TCGMemOp opc = get_memop(oi); int32_t offset = guest_base; TCGReg base = addrlo; int index = -1; @@ -2222,7 +2044,6 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) TCGReg datahi __attribute__((unused)) = -1; TCGReg addrhi __attribute__((unused)) = -1; TCGMemOpIdx oi; - TCGMemOp opc; int i = -1; datalo = args[++i]; @@ -2234,36 +2055,26 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) addrhi = args[++i]; } oi = args[++i]; - opc = get_memop(oi); #if defined(CONFIG_SOFTMMU) - { - int mem_index = get_mmuidx(oi); - tcg_insn_unit *label_ptr[2]; - TCGReg base; - - i = -1; - tcg_debug_assert(addrlo == softmmu_arg(++i)); - if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { - tcg_debug_assert(addrhi == softmmu_arg(++i)); - } - tcg_debug_assert(datalo == softmmu_arg(++i)); - if (TCG_TARGET_REG_BITS == 32 && is64) { - tcg_debug_assert(datahi == softmmu_arg(++i)); - } - - base = tcg_out_tlb_load(s, addrlo, addrhi, mem_index, opc, - label_ptr, offsetof(CPUTLBEntry, addr_write)); - - /* TLB Hit. */ - tcg_out_qemu_st_direct(s, datalo, datahi, base, 0, 0, opc); - - /* Record the current context of a store into ldst label */ - add_qemu_ldst_label(s, false, oi, datalo, datahi, addrlo, addrhi, - s->code_ptr, label_ptr); + /* Assert that we've set up the constraints properly. */ + i = -1; + tcg_debug_assert(addrlo == softmmu_arg(++i)); + if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { + tcg_debug_assert(addrhi == softmmu_arg(++i)); } + tcg_debug_assert(datalo == softmmu_arg(++i)); + if (TCG_TARGET_REG_BITS == 32 && is64) { + tcg_debug_assert(datahi == softmmu_arg(++i)); + } + + /* Call to thunk. */ + tcg_out8(s, OPC_CALL_Jz); + add_ldst_ool_label(s, false, is64, oi, R_386_PC32, -4); + s->code_ptr += 4; #else { + TCGMemOp opc = get_memop(oi); int32_t offset = guest_base; TCGReg base = addrlo; int seg = 0; @@ -2298,6 +2109,134 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) #endif } +#if defined(CONFIG_SOFTMMU) +/* + * Generate code for an out-of-line thunk performing a load. + */ +static tcg_insn_unit *tcg_out_qemu_ldst_ool(TCGContext *s, bool is_ld, + bool is64, TCGMemOpIdx oi) +{ + TCGMemOp opc = get_memop(oi); + int mem_index = get_mmuidx(oi); + tcg_insn_unit *label_ptr[2], *thunk; + TCGReg datalo, addrlo, base; + TCGReg datahi __attribute__((unused)) = -1; + TCGReg addrhi __attribute__((unused)) = -1; + int i; + + /* Since we're amortizing the cost, align the thunk. */ + thunk = QEMU_ALIGN_PTR_UP(s->code_ptr, 16); + if (thunk != s->code_ptr) { + memset(s->code_ptr, 0x90, thunk - s->code_ptr); + s->code_ptr = thunk; + } + + /* Discover where the inputs are held. */ + i = -1; + addrlo = softmmu_arg(++i); + if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { + addrhi = softmmu_arg(++i); + } + if (is_ld) { + datalo = tcg_target_call_oarg_regs[0]; + if (TCG_TARGET_REG_BITS == 32 && is64) { + datahi = tcg_target_call_oarg_regs[1]; + } + } else { + datalo = softmmu_arg(++i); + if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { + datahi = softmmu_arg(++i); + } + } + + base = tcg_out_tlb_load(s, addrlo, addrhi, mem_index, opc, label_ptr, + is_ld ? offsetof(CPUTLBEntry, addr_read) + : offsetof(CPUTLBEntry, addr_write)); + + /* TLB Hit. */ + if (is_ld) { + tcg_out_qemu_ld_direct(s, datalo, datahi, base, -1, 0, 0, opc); + } else { + tcg_out_qemu_st_direct(s, datalo, datahi, base, 0, 0, opc); + } + tcg_out_opc(s, OPC_RET, 0, 0, 0); + + /* TLB Miss. */ + + /* resolve label address */ + tcg_patch8(label_ptr[0], s->code_ptr - label_ptr[0] - 1); + if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { + tcg_patch8(label_ptr[1], s->code_ptr - label_ptr[1] - 1); + } + + if (TCG_TARGET_REG_BITS == 32) { + /* Copy the return address into a temporary. */ + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_L0, TCG_REG_ESP, 0); + i = 4; + + tcg_out_st(s, TCG_TYPE_PTR, TCG_AREG0, TCG_REG_ESP, i); + i += 4; + + tcg_out_st(s, TCG_TYPE_I32, addrlo, TCG_REG_ESP, i); + i += 4; + + if (TARGET_LONG_BITS == 64) { + tcg_out_st(s, TCG_TYPE_I32, addrhi, TCG_REG_ESP, i); + i += 4; + } + + if (!is_ld) { + tcg_out_st(s, TCG_TYPE_I32, datalo, TCG_REG_ESP, i); + i += 4; + + if (is64) { + tcg_out_st(s, TCG_TYPE_I32, datahi, TCG_REG_ESP, i); + i += 4; + } + } + + tcg_out_sti(s, TCG_TYPE_I32, oi, TCG_REG_ESP, i); + i += 4; + + tcg_out_st(s, TCG_TYPE_PTR, TCG_REG_L0, TCG_REG_ESP, i); + } else { + tcg_out_mov(s, TCG_TYPE_PTR, tcg_target_call_iarg_regs[0], TCG_AREG0); + + /* The address and data values have been placed by constraints. */ + tcg_debug_assert(addrlo == tcg_target_call_iarg_regs[1]); + if (is_ld) { + i = 2; + } else { + tcg_debug_assert(datalo == tcg_target_call_iarg_regs[2]); + i = 3; + } + + tcg_out_movi(s, TCG_TYPE_I32, tcg_target_call_iarg_regs[i++], oi); + + /* Copy the return address from the stack to the rvalue argument. + * WIN64 runs out of argument registers for stores. + */ + if (i < (int)ARRAY_SIZE(tcg_target_call_iarg_regs)) { + tcg_out_ld(s, TCG_TYPE_PTR, tcg_target_call_iarg_regs[i], + TCG_REG_ESP, 0); + } else { + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_RAX, TCG_REG_ESP, 0); + tcg_out_st(s, TCG_TYPE_PTR, TCG_REG_RAX, TCG_REG_ESP, + TCG_TARGET_CALL_STACK_OFFSET + 8); + } + } + + /* Tail call to the helper. */ + if (is_ld) { + tcg_out_jmp(s, qemu_ld_helpers[opc & (MO_BSWAP | MO_SSIZE)]); + } else { + tcg_out_jmp(s, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); + } + + return thunk; +} +#endif + static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg *args, const int *const_args) { From patchwork Mon Nov 12 21:44:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996688 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="imCn6Tl+"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4VN2RY1z9s7W for ; Tue, 13 Nov 2018 09:02:04 +1100 (AEDT) Received: from localhost ([::1]:50882 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMKHR-0008AE-V1 for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 17:02:01 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54158) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK3E-0003ZX-L2 for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:21 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK3B-0004z6-1y for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:19 -0500 Received: from mail-wr1-x442.google.com ([2a00:1450:4864:20::442]:42473) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK3A-0004yW-Rj for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:16 -0500 Received: by mail-wr1-x442.google.com with SMTP id u5-v6so5737988wrn.9 for ; Mon, 12 Nov 2018 13:47:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dEMW7UG/VwRHHO8qBaUD73G0cEqvXtdIe4LBfAmg/J0=; b=imCn6Tl+Hy3XXXbMx8wkEnhnBC1Whdtu6ihxy7PeklVc5+lcQdUT3DItwsi2lGmMZr neKB77AcRgdvz+OPH+KbvHu/55M0Ta6fBzEZ27SPIMaDhMC7WDgb/oUpjwlh6mrtOZaY Y328MLbxkt2fmPZeYueG1h3nfOCqV36driPh8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dEMW7UG/VwRHHO8qBaUD73G0cEqvXtdIe4LBfAmg/J0=; b=sI6O7LrsHMhzEAUcOH3fOLtU2pyhpuG9rrFaqu5rf/yavU1Gh77b36AWnlYH5R6bmi lUZpheGmZPX26WfiYWxAnB5/cWUxi4ifw7PB82HrCa+Sec+JWx9i67QH7vHYpj1DOMks KFP+CMcd+Tl3rFC62rEFK0fyDq7sD61wdaUfqcHLl5NcXAaILo2okwlkkZcpZNlwHYzu rPVe/kjNpkQN1XqSmtiI3G4FbW1N9PYzr0l8QJOZ+cqBw4RiB2hBmkL8BZEFxVR+nnK+ 6U+rzUooJbsv6gJ/Wz2xELG+Sd/keBMBsUDunFeujPHQzQPRWpa3QB2VgJd64GN+G2Mu MdxA== X-Gm-Message-State: AGRZ1gID9Ij+dqZk0A23vHCsGGKS3J/Kiy2aLQEAJh+M1cAyU6HyL0Us ZTKoOOxRMNwx7FeLkdqCiXf56x7yrps9QA== X-Google-Smtp-Source: AJdET5f1g2eeNWJIAtHjO8AqbyAeezVndR2ZFlvYgz732vn2UNyOZ90rGcsT9/qDo7SFeq4lXQ3+4Q== X-Received: by 2002:adf:ae1a:: with SMTP id x26-v6mr2698529wrc.189.1542059235444; Mon, 12 Nov 2018 13:47:15 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.14 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:14 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:44:54 +0100 Message-Id: <20181112214503.22941-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::442 Subject: [Qemu-devel] [PATCH for-4.0 08/17] tcg/aarch64: Add constraints for x0, x1, x2 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" These are function call arguments that we will need soon. Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.inc.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c index 30091f6a69..148de0b7f2 100644 --- a/tcg/aarch64/tcg-target.inc.c +++ b/tcg/aarch64/tcg-target.inc.c @@ -125,6 +125,18 @@ static const char *target_parse_constraint(TCGArgConstraint *ct, const char *ct_str, TCGType type) { switch (*ct_str++) { + case 'a': /* x0 */ + ct->ct |= TCG_CT_REG; + tcg_regset_set_reg(ct->u.regs, TCG_REG_X0); + break; + case 'b': /* x1 */ + ct->ct |= TCG_CT_REG; + tcg_regset_set_reg(ct->u.regs, TCG_REG_X1); + break; + case 'c': /* x2 */ + ct->ct |= TCG_CT_REG; + tcg_regset_set_reg(ct->u.regs, TCG_REG_X2); + break; case 'r': /* general registers */ ct->ct |= TCG_CT_REG; ct->u.regs |= 0xffffffffu; From patchwork Mon Nov 12 21:44:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996695 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="iY0qItMH"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4Zf1Vcnz9s7W for ; Tue, 13 Nov 2018 09:05:46 +1100 (AEDT) Received: from localhost ([::1]:50900 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMKL1-0004dh-Pf for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 17:05:43 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54156) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK3E-0003ZV-Ky for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:21 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK3C-0004ze-35 for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:19 -0500 Received: from mail-wr1-x434.google.com ([2a00:1450:4864:20::434]:41246) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK3B-0004zE-PV for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:18 -0500 Received: by mail-wr1-x434.google.com with SMTP id v18-v6so11019095wrt.8 for ; Mon, 12 Nov 2018 13:47:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=2ItTk51JTjzE0w3i+PlYwVgMsWgaDFle4WWcoNcT4uk=; b=iY0qItMHSwQXn9Rz8g2/vHy98bTrzg9n2O6cfDDjInJzYz8V+Tx36+Y8vQ3PaGVk2b 1F59sq4zM0/EYPhisAdZGmPstK29umIZ6cQbUF6AhBoaV0QYMjr/oSiN7FlWwgg9tG14 qW5Bl45BEalcRK5BBYaXaNcJWPRDEUZcbA9gU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=2ItTk51JTjzE0w3i+PlYwVgMsWgaDFle4WWcoNcT4uk=; b=exprvhDvnr3NZ8T2E04z65HZTddh5YKn88pE8D9hDRuTIy4nKMgx6aWblIXHFuFO0f XA9QEEmuwv/AKWBx521ioZ8cxvGc5ANNVKmJRdn2KCMF4CBgNkL6x7HtsKixLeFy5h1r ROLPRQG6Sd7slk5nsGOfRIUVNlfeI2itkanjpL9qQNwSfVHNnghoTQr0EfEiPs4rDTw4 KesQLBmdbjTELMeOx/vgdsv1urNMUmeLs3CP4laT45RAAM6oX7C6QPQyCwjdo8fNpPUy oiFZJ7k+24YXJn3Nnr26l+/xSdFINGjPWc1fFubIKyothK1nAacTLQrhPBl6gtgffAOd j68A== X-Gm-Message-State: AGRZ1gLZtbI2ZPelbnBfyZXiyTbD/MTgj6Izbkzuyujon8hvQufLniKK VzSKfqxSgeijJxcYO56dWgqZFJ2HaHtbhg== X-Google-Smtp-Source: AJdET5ekxMfgx4MOUbN8jSUmQrHV3VoYZzoeiBKqIOT8q94ZIzRgE+Fi/2qWFzy9R2n5GHg9HkVPcg== X-Received: by 2002:a5d:6acd:: with SMTP id u13-v6mr2345826wrw.175.1542059236511; Mon, 12 Nov 2018 13:47:16 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.15 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:15 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:44:55 +0100 Message-Id: <20181112214503.22941-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::434 Subject: [Qemu-devel] [PATCH for-4.0 09/17] tcg/aarch64: Parameterize the temps for tcg_out_tlb_read X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" When moving the qemu_ld/st arguments to the right place for a function call, we'll need to move the temps out of the way. Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.inc.c | 74 +++++++++++++++++++----------------- 1 file changed, 40 insertions(+), 34 deletions(-) diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c index 148de0b7f2..c0ba9a6d50 100644 --- a/tcg/aarch64/tcg-target.inc.c +++ b/tcg/aarch64/tcg-target.inc.c @@ -1467,13 +1467,15 @@ static void add_qemu_ldst_label(TCGContext *s, bool is_ld, TCGMemOpIdx oi, label->label_ptr[0] = label_ptr; } -/* Load and compare a TLB entry, emitting the conditional jump to the - slow path for the failure case, which will be patched later when finalizing - the slow path. Generated code returns the host addend in X1, - clobbers X0,X2,X3,TMP. */ -static void tcg_out_tlb_read(TCGContext *s, TCGReg addr_reg, TCGMemOp opc, - tcg_insn_unit **label_ptr, int mem_index, - bool is_read) +/* + * Load and compare a TLB entry, emitting the conditional jump to the + * slow path on failure. Returns the register for the host addend. + * Clobbers t0, t1, t2, t3. + */ +static TCGReg tcg_out_tlb_read(TCGContext *s, TCGReg addr_reg, TCGMemOp opc, + tcg_insn_unit **label_ptr, int mem_index, + bool is_read, TCGReg t0, TCGReg t1, + TCGReg t2, TCGReg t3) { int tlb_offset = is_read ? offsetof(CPUArchState, tlb_table[mem_index][0].addr_read) @@ -1491,55 +1493,56 @@ static void tcg_out_tlb_read(TCGContext *s, TCGReg addr_reg, TCGMemOp opc, if (a_bits >= s_bits) { x3 = addr_reg; } else { + x3 = t3; tcg_out_insn(s, 3401, ADDI, TARGET_LONG_BITS == 64, - TCG_REG_X3, addr_reg, s_mask - a_mask); - x3 = TCG_REG_X3; + x3, addr_reg, s_mask - a_mask); } tlb_mask = (uint64_t)TARGET_PAGE_MASK | a_mask; - /* Extract the TLB index from the address into X0. - X0 = + /* Extract the TLB index from the address into T0. + T0 = addr_reg */ - tcg_out_ubfm(s, TARGET_LONG_BITS == 64, TCG_REG_X0, addr_reg, + tcg_out_ubfm(s, TARGET_LONG_BITS == 64, t0, addr_reg, TARGET_PAGE_BITS, TARGET_PAGE_BITS + CPU_TLB_BITS); - /* Store the page mask part of the address into X3. */ + /* Store the page mask part of the address into T3. */ tcg_out_logicali(s, I3404_ANDI, TARGET_LONG_BITS == 64, - TCG_REG_X3, x3, tlb_mask); + t3, x3, tlb_mask); - /* Add any "high bits" from the tlb offset to the env address into X2, + /* Add any "high bits" from the tlb offset to the env address into T2, to take advantage of the LSL12 form of the ADDI instruction. - X2 = env + (tlb_offset & 0xfff000) */ + T2 = env + (tlb_offset & 0xfff000) */ if (tlb_offset & 0xfff000) { - tcg_out_insn(s, 3401, ADDI, TCG_TYPE_I64, TCG_REG_X2, base, + tcg_out_insn(s, 3401, ADDI, TCG_TYPE_I64, t2, base, tlb_offset & 0xfff000); - base = TCG_REG_X2; + base = t2; } - /* Merge the tlb index contribution into X2. - X2 = X2 + (X0 << CPU_TLB_ENTRY_BITS) */ - tcg_out_insn(s, 3502S, ADD_LSL, TCG_TYPE_I64, TCG_REG_X2, base, - TCG_REG_X0, CPU_TLB_ENTRY_BITS); + /* Merge the tlb index contribution into T2. + T2 = T2 + (T0 << CPU_TLB_ENTRY_BITS) */ + tcg_out_insn(s, 3502S, ADD_LSL, TCG_TYPE_I64, + t2, base, t0, CPU_TLB_ENTRY_BITS); - /* Merge "low bits" from tlb offset, load the tlb comparator into X0. - X0 = load [X2 + (tlb_offset & 0x000fff)] */ + /* Merge "low bits" from tlb offset, load the tlb comparator into T0. + T0 = load [T2 + (tlb_offset & 0x000fff)] */ tcg_out_ldst(s, TARGET_LONG_BITS == 32 ? I3312_LDRW : I3312_LDRX, - TCG_REG_X0, TCG_REG_X2, tlb_offset & 0xfff, - TARGET_LONG_BITS == 32 ? 2 : 3); + t0, t2, tlb_offset & 0xfff, TARGET_LONG_BITS == 32 ? 2 : 3); /* Load the tlb addend. Do that early to avoid stalling. - X1 = load [X2 + (tlb_offset & 0xfff) + offsetof(addend)] */ - tcg_out_ldst(s, I3312_LDRX, TCG_REG_X1, TCG_REG_X2, + T1 = load [T2 + (tlb_offset & 0xfff) + offsetof(addend)] */ + tcg_out_ldst(s, I3312_LDRX, t1, t2, (tlb_offset & 0xfff) + (offsetof(CPUTLBEntry, addend)) - (is_read ? offsetof(CPUTLBEntry, addr_read) : offsetof(CPUTLBEntry, addr_write)), 3); /* Perform the address comparison. */ - tcg_out_cmp(s, (TARGET_LONG_BITS == 64), TCG_REG_X0, TCG_REG_X3, 0); + tcg_out_cmp(s, (TARGET_LONG_BITS == 64), t0, t3, 0); /* If not equal, we jump to the slow path. */ *label_ptr = s->code_ptr; tcg_out_goto_cond_noaddr(s, TCG_COND_NE); + + return t1; } #endif /* CONFIG_SOFTMMU */ @@ -1644,10 +1647,12 @@ static void tcg_out_qemu_ld(TCGContext *s, TCGReg data_reg, TCGReg addr_reg, #ifdef CONFIG_SOFTMMU unsigned mem_index = get_mmuidx(oi); tcg_insn_unit *label_ptr; + TCGReg base; - tcg_out_tlb_read(s, addr_reg, memop, &label_ptr, mem_index, 1); + base = tcg_out_tlb_read(s, addr_reg, memop, &label_ptr, mem_index, 1, + TCG_REG_X0, TCG_REG_X1, TCG_REG_X2, TCG_REG_X3); tcg_out_qemu_ld_direct(s, memop, ext, data_reg, - TCG_REG_X1, otype, addr_reg); + base, otype, addr_reg); add_qemu_ldst_label(s, true, oi, ext, data_reg, addr_reg, s->code_ptr, label_ptr); #else /* !CONFIG_SOFTMMU */ @@ -1669,10 +1674,11 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg data_reg, TCGReg addr_reg, #ifdef CONFIG_SOFTMMU unsigned mem_index = get_mmuidx(oi); tcg_insn_unit *label_ptr; + TCGReg base; - tcg_out_tlb_read(s, addr_reg, memop, &label_ptr, mem_index, 0); - tcg_out_qemu_st_direct(s, memop, data_reg, - TCG_REG_X1, otype, addr_reg); + base = tcg_out_tlb_read(s, addr_reg, memop, &label_ptr, mem_index, 0, + TCG_REG_X0, TCG_REG_X1, TCG_REG_X2, TCG_REG_X3); + tcg_out_qemu_st_direct(s, memop, data_reg, base, otype, addr_reg); add_qemu_ldst_label(s, false, oi, (memop & MO_SIZE)== MO_64, data_reg, addr_reg, s->code_ptr, label_ptr); #else /* !CONFIG_SOFTMMU */ From patchwork Mon Nov 12 21:44:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996681 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="jKzSa1sw"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4Ml0bTpz9s3x for ; Tue, 13 Nov 2018 08:56:18 +1100 (AEDT) Received: from localhost ([::1]:50850 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMKBs-0003G4-8s for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 16:56:16 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54183) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK3G-0003dp-8A for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK3D-00050O-UD for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:21 -0500 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]:41745) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK3D-0004zj-5P for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:19 -0500 Received: by mail-wr1-x443.google.com with SMTP id v18-v6so11019146wrt.8 for ; Mon, 12 Nov 2018 13:47:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=bfXp2m+eG0fPw8rnLzBA4705O4L8LprWDujoyvgpuQg=; b=jKzSa1swtOLRLYglXfKFRywfdgn+LYhR6/pSrv7buXBPDSOZ1iek5jS0LFKlYN/Wct zrWAjzB4AQCU3zW0TvQzG2ct1jND2Y111j0GCic3MAMC67aHbsEqC3IxXe5Nsh9m5O5O 3Y7MRhXFRTpZ0ifoK8jwUv/7b5lqCgKPelFVg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=bfXp2m+eG0fPw8rnLzBA4705O4L8LprWDujoyvgpuQg=; b=Y2GkxlJ3JavvFAgmRy/8Luw2jUFluJHQkKhCrTWcHMBGYThgKVtsBj+3uCrmBX5Ybt L+Tb+Dc8t9wNsBYQW1oD9tIsY83G6SvApEA84tzwpaHSzvirYtu0xyQl0Kx16FPFhMbH M7R730KjQB+vtZFAP0MGpzDE79mfluXNUsLeSn6KuIWGaqJ5zi0ApvgyJ0Ab+Ndx/3g5 eSQndt35OVMGEYnJwVpe3HQVvhIyhjlLhaTf7wJZ990i9/Kx1S+CFN/Y3APgFsRxb+Fo apr/ibx+O3G5t7x61J7MzHqqFWs1NZwumdmlJ7BNPHWl/Mkt5xFnHuZgoa+6Z6//qP2t 4kwg== X-Gm-Message-State: AGRZ1gJPlwmminHzQr9B7ZFrCCJ7U88GaOGylh2gjs+UGoTz1//ZtA7c lo38/k9Nhv7kQV88impT4HR3Yo0+hgPggA== X-Google-Smtp-Source: AJdET5cSLfi9kF3Xe5pV75upQLdVIO2SD0+8nGjywX+HOK9LvwI6J7LUC7BBa8IpTCAaCMWr/1Qkeg== X-Received: by 2002:adf:a109:: with SMTP id o9-v6mr2490565wro.169.1542059237533; Mon, 12 Nov 2018 13:47:17 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.16 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:17 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:44:56 +0100 Message-Id: <20181112214503.22941-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::443 Subject: [Qemu-devel] [PATCH for-4.0 10/17] tcg/aarch64: Parameterize the temp for tcg_out_goto_long X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We cannot use TCG_REG_LR (aka TCG_REG_TMP) for tail calls. Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.inc.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c index c0ba9a6d50..ea5fe33fca 100644 --- a/tcg/aarch64/tcg-target.inc.c +++ b/tcg/aarch64/tcg-target.inc.c @@ -1134,14 +1134,15 @@ static inline void tcg_out_goto(TCGContext *s, tcg_insn_unit *target) tcg_out_insn(s, 3206, B, offset); } -static inline void tcg_out_goto_long(TCGContext *s, tcg_insn_unit *target) +static inline void tcg_out_goto_long(TCGContext *s, tcg_insn_unit *target, + TCGReg scratch) { ptrdiff_t offset = target - s->code_ptr; if (offset == sextract64(offset, 0, 26)) { tcg_out_insn(s, 3206, BL, offset); } else { - tcg_out_movi(s, TCG_TYPE_I64, TCG_REG_TMP, (intptr_t)target); - tcg_out_insn(s, 3207, BR, TCG_REG_TMP); + tcg_out_movi(s, TCG_TYPE_I64, scratch, (intptr_t)target); + tcg_out_insn(s, 3207, BR, scratch); } } @@ -1716,10 +1717,10 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_exit_tb: /* Reuse the zeroing that exists for goto_ptr. */ if (a0 == 0) { - tcg_out_goto_long(s, s->code_gen_epilogue); + tcg_out_goto_long(s, s->code_gen_epilogue, TCG_REG_TMP); } else { tcg_out_movi(s, TCG_TYPE_I64, TCG_REG_X0, a0); - tcg_out_goto_long(s, tb_ret_addr); + tcg_out_goto_long(s, tb_ret_addr, TCG_REG_TMP); } break; From patchwork Mon Nov 12 21:44:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996691 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="Bz9F/F81"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4WJ5m89z9s0n for ; Tue, 13 Nov 2018 09:02:52 +1100 (AEDT) Received: from localhost ([::1]:50889 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMKIE-0000aw-7S for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 17:02:50 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54236) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK3K-0003z6-EZ for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK3G-000527-NU for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:24 -0500 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]:40942) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK3G-000502-FM for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:22 -0500 Received: by mail-wr1-x443.google.com with SMTP id p4so1844596wrt.7 for ; Mon, 12 Nov 2018 13:47:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ZgQoWHRJcRFp4C9KfWwThoalEdmTxoVA/RPtSMYllzE=; b=Bz9F/F81r+LzomDV4m6Zim344QUvP82ku/c6oE1xGbygv9+gLQaqHbJPtr4HADnuW+ g9ZhVcDzKA0dJeEV84Iht4dFnjKJVWR/JyGlkKOvyT4WqdQJeNjGP1OF5knkGTcMn9wo bn6DvkqZjo7tkGRoAiGZa7KFm3OyH/ZzkwF98= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ZgQoWHRJcRFp4C9KfWwThoalEdmTxoVA/RPtSMYllzE=; b=iU37x5fjuhIyyf+wT0W8keoLFy3WTj969+WBOlydHe997IkAKgJN7ERhnK33ULd4JE upe2SUU2j8YT8GkbuyKBcCUMYXMJT/B/MBKCPrPld1/rXq7UI9Kzq7aaNMGQAcgQa2up WJSXCG+nhcZOK1IUL8kpmC2elWT0BqkMANJtklRHtcDr07Q7euZYmuH3/BP5X1HwzTTk ujys/imQmxXPawEPX8GQ9MqngGfzG8KsooflR1WrEqh0VVPoC83qmvorOWHU/IQAT7YS G9SEa4TPbP1KVW9F9soAdxRjwVN3GaYhthJBrFTB9JY7x8LMq8lwtVLIIz1VHDTTDE4y IRvg== X-Gm-Message-State: AGRZ1gIZe/65HkUpHeQaZUIV8ZQs8Vw19nUDbCeRKGogIte57Plm6mJn VRUn1PjVVCxQ+5N98I6Q3L05sOrxZp6hfw== X-Google-Smtp-Source: AJdET5fGSJLo2mt+4/0H74xY05Xqc6A+ZoTmUtjrc+Xn24B7Q+32X6Ky7Qw8IbVYWbWcnJZzs8Vnyg== X-Received: by 2002:adf:9e87:: with SMTP id a7-v6mr2326320wrf.164.1542059238619; Mon, 12 Nov 2018 13:47:18 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.17 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:18 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:44:57 +0100 Message-Id: <20181112214503.22941-12-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::443 Subject: [Qemu-devel] [PATCH for-4.0 11/17] tcg/aarch64: Use B not BL for tcg_out_goto_long X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This was a typo copying from tcg_out_call, apparently. Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.inc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c index ea5fe33fca..403f5caf14 100644 --- a/tcg/aarch64/tcg-target.inc.c +++ b/tcg/aarch64/tcg-target.inc.c @@ -1139,7 +1139,7 @@ static inline void tcg_out_goto_long(TCGContext *s, tcg_insn_unit *target, { ptrdiff_t offset = target - s->code_ptr; if (offset == sextract64(offset, 0, 26)) { - tcg_out_insn(s, 3206, BL, offset); + tcg_out_insn(s, 3206, B, offset); } else { tcg_out_movi(s, TCG_TYPE_I64, scratch, (intptr_t)target); tcg_out_insn(s, 3207, BR, scratch); From patchwork Mon Nov 12 21:44:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996685 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="QhQeqh1D"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4R055nWz9s3x for ; Tue, 13 Nov 2018 08:59:08 +1100 (AEDT) Received: from localhost ([::1]:50862 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMKEc-0005ps-9p for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 16:59:06 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54243) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK3K-0003zA-Lv for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK3I-000531-Rp for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:26 -0500 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]:33398) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK3G-00050p-JS for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:23 -0500 Received: by mail-wr1-x441.google.com with SMTP id u9-v6so11033081wrr.0 for ; Mon, 12 Nov 2018 13:47:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UrR/IldFdFkWy6ywHmFI8AClyOk1jWWQ8mWiO+LW0Iw=; b=QhQeqh1DTuKDkKbZiY12ONFneb9FqwUYrwh7Z7bZBBaaPgTuRLnWXzlMgzAD063jUb HoOjIXgh1hltdszIbrQ4B2C74rNzaupBRvC1npEON8Lwy1OdSg2SpVLt9JLwPWDGChPt jNO39cIeZbVSmx9TtzKvaURr0IHFHD5G+F7xM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UrR/IldFdFkWy6ywHmFI8AClyOk1jWWQ8mWiO+LW0Iw=; b=r6mlT+KoXaPBM5rLwBi8XiOndSnqVJ5u9ZryHCv6mJ02E88pt2gb2VTU5Xnr0VqiEv nK+dUdBT4Ifk8/eeeZLC7DF6O0cnWVru3JOQYyQduIdhGNhFrb63wn9IwujKwKX9TrUi VpHQWqtAxkuPdfxu2FpxxNFri6BNxG/P2NjMILCH3IWxBClnvrku63Kom0bsSvpt8scw RAKRZupyWThVVoqOf56uJxQUAyEO9BkdZhG0YWtQfR/JDUarOpfJH7vHMP0hyjly1+BM 1/5dWNg5HomTlagOtUppPymrJApSCVSTSmeBMMe8ApgxFsj0xeJoZckbc6JugBflSDVP TdOQ== X-Gm-Message-State: AGRZ1gI1vAugLYlnwNvYn29OmPS1HkR9KdXbRPRUZ8YCi98GnCcfnGtd baCFL10eKhS/pVig8EatpuJxULGoEFdhpg== X-Google-Smtp-Source: AJdET5e3nuKA3WgPDrWMSJDCLOVZ3BM6Z5kxOvWhsTBQ5r4Q5cvuxsLwtK/wZ5XdhxGnDcmVL+1Dnw== X-Received: by 2002:adf:e983:: with SMTP id h3-v6mr2533904wrm.58.1542059239783; Mon, 12 Nov 2018 13:47:19 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.18 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:19 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:44:58 +0100 Message-Id: <20181112214503.22941-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::441 Subject: [Qemu-devel] [PATCH for-4.0 12/17] tcg/aarch64: Use TCG_TARGET_NEED_LDST_OOL_LABELS X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.h | 2 +- tcg/aarch64/tcg-target.inc.c | 191 +++++++++++++++++------------------ 2 files changed, 93 insertions(+), 100 deletions(-) diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index 9aea1d1771..d1bd77c41d 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -146,7 +146,7 @@ static inline void flush_icache_range(uintptr_t start, uintptr_t stop) void tb_target_set_jmp_target(uintptr_t, uintptr_t, uintptr_t); #ifdef CONFIG_SOFTMMU -#define TCG_TARGET_NEED_LDST_LABELS +#define TCG_TARGET_NEED_LDST_OOL_LABELS #endif #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c index 403f5caf14..8edea527f7 100644 --- a/tcg/aarch64/tcg-target.inc.c +++ b/tcg/aarch64/tcg-target.inc.c @@ -145,18 +145,6 @@ static const char *target_parse_constraint(TCGArgConstraint *ct, ct->ct |= TCG_CT_REG; ct->u.regs |= 0xffffffff00000000ull; break; - case 'l': /* qemu_ld / qemu_st address, data_reg */ - ct->ct |= TCG_CT_REG; - ct->u.regs = 0xffffffffu; -#ifdef CONFIG_SOFTMMU - /* x0 and x1 will be overwritten when reading the tlb entry, - and x2, and x3 for helper args, better to avoid using them. */ - tcg_regset_reset_reg(ct->u.regs, TCG_REG_X0); - tcg_regset_reset_reg(ct->u.regs, TCG_REG_X1); - tcg_regset_reset_reg(ct->u.regs, TCG_REG_X2); - tcg_regset_reset_reg(ct->u.regs, TCG_REG_X3); -#endif - break; case 'A': /* Valid for arithmetic immediate (positive or negative). */ ct->ct |= TCG_CT_CONST_AIMM; break; @@ -1378,7 +1366,7 @@ static void tcg_out_cltz(TCGContext *s, TCGType ext, TCGReg d, } #ifdef CONFIG_SOFTMMU -#include "tcg-ldst.inc.c" +#include "tcg-ldst-ool.inc.c" /* helper signature: helper_ret_ld_mmu(CPUState *env, target_ulong addr, * TCGMemOpIdx oi, uintptr_t ra) @@ -1391,6 +1379,12 @@ static void * const qemu_ld_helpers[16] = { [MO_BEUW] = helper_be_lduw_mmu, [MO_BEUL] = helper_be_ldul_mmu, [MO_BEQ] = helper_be_ldq_mmu, + + [MO_SB] = helper_ret_ldsb_mmu, + [MO_LESW] = helper_le_ldsw_mmu, + [MO_LESL] = helper_le_ldsl_mmu, + [MO_BESW] = helper_be_ldsw_mmu, + [MO_BESL] = helper_be_ldsl_mmu, }; /* helper signature: helper_ret_st_mmu(CPUState *env, target_ulong addr, @@ -1407,67 +1401,6 @@ static void * const qemu_st_helpers[16] = { [MO_BEQ] = helper_be_stq_mmu, }; -static inline void tcg_out_adr(TCGContext *s, TCGReg rd, void *target) -{ - ptrdiff_t offset = tcg_pcrel_diff(s, target); - tcg_debug_assert(offset == sextract64(offset, 0, 21)); - tcg_out_insn(s, 3406, ADR, rd, offset); -} - -static void tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) -{ - TCGMemOpIdx oi = lb->oi; - TCGMemOp opc = get_memop(oi); - TCGMemOp size = opc & MO_SIZE; - - reloc_pc19(lb->label_ptr[0], s->code_ptr); - - tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_X0, TCG_AREG0); - tcg_out_mov(s, TARGET_LONG_BITS == 64, TCG_REG_X1, lb->addrlo_reg); - tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_X2, oi); - tcg_out_adr(s, TCG_REG_X3, lb->raddr); - tcg_out_call(s, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); - if (opc & MO_SIGN) { - tcg_out_sxt(s, lb->type, size, lb->datalo_reg, TCG_REG_X0); - } else { - tcg_out_mov(s, size == MO_64, lb->datalo_reg, TCG_REG_X0); - } - - tcg_out_goto(s, lb->raddr); -} - -static void tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) -{ - TCGMemOpIdx oi = lb->oi; - TCGMemOp opc = get_memop(oi); - TCGMemOp size = opc & MO_SIZE; - - reloc_pc19(lb->label_ptr[0], s->code_ptr); - - tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_X0, TCG_AREG0); - tcg_out_mov(s, TARGET_LONG_BITS == 64, TCG_REG_X1, lb->addrlo_reg); - tcg_out_mov(s, size == MO_64, TCG_REG_X2, lb->datalo_reg); - tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_X3, oi); - tcg_out_adr(s, TCG_REG_X4, lb->raddr); - tcg_out_call(s, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); - tcg_out_goto(s, lb->raddr); -} - -static void add_qemu_ldst_label(TCGContext *s, bool is_ld, TCGMemOpIdx oi, - TCGType ext, TCGReg data_reg, TCGReg addr_reg, - tcg_insn_unit *raddr, tcg_insn_unit *label_ptr) -{ - TCGLabelQemuLdst *label = new_ldst_label(s); - - label->is_ld = is_ld; - label->oi = oi; - label->type = ext; - label->datalo_reg = data_reg; - label->addrlo_reg = addr_reg; - label->raddr = raddr; - label->label_ptr[0] = label_ptr; -} - /* * Load and compare a TLB entry, emitting the conditional jump to the * slow path on failure. Returns the register for the host addend. @@ -1644,19 +1577,22 @@ static void tcg_out_qemu_ld(TCGContext *s, TCGReg data_reg, TCGReg addr_reg, TCGMemOpIdx oi, TCGType ext) { TCGMemOp memop = get_memop(oi); - const TCGType otype = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; -#ifdef CONFIG_SOFTMMU - unsigned mem_index = get_mmuidx(oi); - tcg_insn_unit *label_ptr; - TCGReg base; - base = tcg_out_tlb_read(s, addr_reg, memop, &label_ptr, mem_index, 1, - TCG_REG_X0, TCG_REG_X1, TCG_REG_X2, TCG_REG_X3); - tcg_out_qemu_ld_direct(s, memop, ext, data_reg, - base, otype, addr_reg); - add_qemu_ldst_label(s, true, oi, ext, data_reg, addr_reg, - s->code_ptr, label_ptr); +#ifdef CONFIG_SOFTMMU + /* Ignore the requested "ext". We get the same correct result from + * a 16-bit sign-extended to 64-bit as we do sign-extended to 32-bit, + * and we create fewer out-of-line thunks. + */ + bool is_64 = (memop & MO_SIGN) || ((memop & MO_SIZE) == MO_64); + + tcg_debug_assert(data_reg == TCG_REG_X0); + tcg_debug_assert(addr_reg == TCG_REG_X1); + + add_ldst_ool_label(s, true, is_64, oi, R_AARCH64_JUMP26, 0); + tcg_out_insn(s, 3206, BL, 0); #else /* !CONFIG_SOFTMMU */ + const TCGType otype = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; + if (USE_GUEST_BASE) { tcg_out_qemu_ld_direct(s, memop, ext, data_reg, TCG_REG_GUEST_BASE, otype, addr_reg); @@ -1671,18 +1607,18 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg data_reg, TCGReg addr_reg, TCGMemOpIdx oi) { TCGMemOp memop = get_memop(oi); - const TCGType otype = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; -#ifdef CONFIG_SOFTMMU - unsigned mem_index = get_mmuidx(oi); - tcg_insn_unit *label_ptr; - TCGReg base; - base = tcg_out_tlb_read(s, addr_reg, memop, &label_ptr, mem_index, 0, - TCG_REG_X0, TCG_REG_X1, TCG_REG_X2, TCG_REG_X3); - tcg_out_qemu_st_direct(s, memop, data_reg, base, otype, addr_reg); - add_qemu_ldst_label(s, false, oi, (memop & MO_SIZE)== MO_64, - data_reg, addr_reg, s->code_ptr, label_ptr); +#ifdef CONFIG_SOFTMMU + bool is_64 = (memop & MO_SIZE) == MO_64; + + tcg_debug_assert(addr_reg == TCG_REG_X1); + tcg_debug_assert(data_reg == TCG_REG_X2); + + add_ldst_ool_label(s, false, is_64, oi, R_AARCH64_JUMP26, 0); + tcg_out_insn(s, 3206, BL, 0); #else /* !CONFIG_SOFTMMU */ + const TCGType otype = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; + if (USE_GUEST_BASE) { tcg_out_qemu_st_direct(s, memop, data_reg, TCG_REG_GUEST_BASE, otype, addr_reg); @@ -1693,6 +1629,52 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg data_reg, TCGReg addr_reg, #endif /* CONFIG_SOFTMMU */ } +#ifdef CONFIG_SOFTMMU +static tcg_insn_unit *tcg_out_qemu_ldst_ool(TCGContext *s, bool is_ld, + bool is_64, TCGMemOpIdx oi) +{ + const TCGType otype = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; + const TCGMemOp memop = get_memop(oi); + const unsigned mem_index = get_mmuidx(oi); + const TCGReg addr_reg = TCG_REG_X1; + const TCGReg data_reg = is_ld ? TCG_REG_X0 : TCG_REG_X2; + tcg_insn_unit * const thunk = s->code_ptr; + tcg_insn_unit *label; + TCGReg base, arg; + + base = tcg_out_tlb_read(s, addr_reg, memop, &label, mem_index, is_ld, + TCG_REG_X4, TCG_REG_X5, TCG_REG_X6, TCG_REG_X7); + + /* TLB Hit */ + if (is_ld) { + tcg_out_qemu_ld_direct(s, memop, is_64, data_reg, + base, otype, addr_reg); + } else { + tcg_out_qemu_st_direct(s, memop, data_reg, base, otype, addr_reg); + } + tcg_out_insn(s, 3207, RET, TCG_REG_LR); + + /* TLB Miss */ + reloc_pc19(label, s->code_ptr); + + tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_X0, TCG_AREG0); + /* addr_reg and data_reg are already in place. */ + arg = is_ld ? TCG_REG_X2 : TCG_REG_X3; + tcg_out_movi(s, TCG_TYPE_I32, arg++, oi); + tcg_out_mov(s, TCG_TYPE_PTR, arg++, TCG_REG_LR); + + if (is_ld) { + tcg_out_goto_long(s, qemu_ld_helpers[memop & (MO_BSWAP | MO_SSIZE)], + TCG_REG_X7); + } else { + tcg_out_goto_long(s, qemu_st_helpers[memop & (MO_BSWAP | MO_SIZE)], + TCG_REG_X7); + } + + return thunk; +} +#endif + static tcg_insn_unit *tb_ret_addr; static void tcg_out_op(TCGContext *s, TCGOpcode opc, @@ -2262,10 +2244,12 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) static const TCGTargetOpDef w_w = { .args_ct_str = { "w", "w" } }; static const TCGTargetOpDef w_r = { .args_ct_str = { "w", "r" } }; static const TCGTargetOpDef w_wr = { .args_ct_str = { "w", "wr" } }; - static const TCGTargetOpDef r_l = { .args_ct_str = { "r", "l" } }; static const TCGTargetOpDef r_rA = { .args_ct_str = { "r", "rA" } }; static const TCGTargetOpDef rZ_r = { .args_ct_str = { "rZ", "r" } }; - static const TCGTargetOpDef lZ_l = { .args_ct_str = { "lZ", "l" } }; +#ifdef CONFIG_SOFTMMU + static const TCGTargetOpDef a_b = { .args_ct_str = { "a", "b" } }; + static const TCGTargetOpDef c_b = { .args_ct_str = { "c", "b" } }; +#endif static const TCGTargetOpDef r_r_r = { .args_ct_str = { "r", "r", "r" } }; static const TCGTargetOpDef w_w_w = { .args_ct_str = { "w", "w", "w" } }; static const TCGTargetOpDef w_w_wZ = { .args_ct_str = { "w", "w", "wZ" } }; @@ -2397,10 +2381,19 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) case INDEX_op_qemu_ld_i32: case INDEX_op_qemu_ld_i64: - return &r_l; +#ifdef CONFIG_SOFTMMU + return &a_b; +#else + return &r_r; +#endif + case INDEX_op_qemu_st_i32: case INDEX_op_qemu_st_i64: - return &lZ_l; +#ifdef CONFIG_SOFTMMU + return &c_b; +#else + return &r_r; +#endif case INDEX_op_deposit_i32: case INDEX_op_deposit_i64: From patchwork Mon Nov 12 21:44:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996696 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="CddvV5L2"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4b10Cf8z9s8J for ; Tue, 13 Nov 2018 09:06:05 +1100 (AEDT) Received: from localhost ([::1]:50907 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMKLK-00053A-Eq for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 17:06:02 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54237) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK3K-0003z7-Ed for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK3H-00052J-1P for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:24 -0500 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]:35987) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK3G-00051Y-Hw for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:22 -0500 Received: by mail-wm1-x344.google.com with SMTP id s11so183367wmh.1 for ; Mon, 12 Nov 2018 13:47:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=8t7gEiP0UljdZGOq63+3IiIupXpsvdWUor96eMaww/U=; b=CddvV5L2NBnodUQqqpzgk/KNKIhWywOj7gIzDBPjEh2BlrjgVStvFkNojakDhouWZi zvl8TI5lO9AUZEa2bbaPxqo1AMqSOIlBAPYsUnkX0ZCfJ38YuLFMQOctfQ3cpEZUwGbv 0ULJY2Zq0QMhTu66zv2lqzGXCklR0+ZN3mdio= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=8t7gEiP0UljdZGOq63+3IiIupXpsvdWUor96eMaww/U=; b=iBalsCsaSXKfqKhFk+IGQybWSCzZFXXF55eySUmtMqC2LAiGrF0CBqyLd7XjwXfmgS VoToevv8NbuwDebm6fdGpWNA+FYr1giUvZBVJf31XWd6A4Kh23/C9KSpTNulPxSF8goc Gsn4WUhzhs0T5ennb9QZ1xGwBJbhaNKJWpyYA04X/Qx+FGHgEepOuoI8GoZuArAxKWRh /Vh7hg4E/9Nb8pRdzNjs3OAMnP9EanqYAM1TaT6O7TGvla7NTUgZzav7yTyBBXAPvGG0 S+mZKXq/rHpQdoz+NmJApFsCXcAETAa1UVbA3aSK2qVoKEr5diLGWKXBv4iWjMjbtBnc 1Rhw== X-Gm-Message-State: AGRZ1gJKbTFsGIQskRVwJSbMFlJwFmP05sUeN2Ooh8/zDT0aYYPb5iNa 1L3/cArea2tF8n1KcD20n1U1CBRCphdB0g== X-Google-Smtp-Source: AJdET5fo3MFfKGGXl1A/i4JcTDBNUHEzWq6u/23tPshRhsH6RxwZyX/+fVgfXrDPmjo3qU74hfVd7A== X-Received: by 2002:a1c:2ecf:: with SMTP id u198-v6mr98548wmu.77.1542059240910; Mon, 12 Nov 2018 13:47:20 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.19 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:20 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:44:59 +0100 Message-Id: <20181112214503.22941-14-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::344 Subject: [Qemu-devel] [PATCH for-4.0 13/17] tcg/arm: Parameterize the temps for tcg_out_tlb_read X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" When moving the qemu_ld/st arguments to the right place for a function call, we'll need to move the temps out of the way. Signed-off-by: Richard Henderson --- tcg/arm/tcg-target.inc.c | 89 +++++++++++++++++++++------------------- 1 file changed, 46 insertions(+), 43 deletions(-) diff --git a/tcg/arm/tcg-target.inc.c b/tcg/arm/tcg-target.inc.c index 80d174ef44..414c91c9ea 100644 --- a/tcg/arm/tcg-target.inc.c +++ b/tcg/arm/tcg-target.inc.c @@ -1245,11 +1245,14 @@ static TCGReg tcg_out_arg_reg64(TCGContext *s, TCGReg argreg, /* We're expecting to use an 8-bit immediate and to mask. */ QEMU_BUILD_BUG_ON(CPU_TLB_BITS > 8); -/* Load and compare a TLB entry, leaving the flags set. Returns the register - containing the addend of the tlb entry. Clobbers R0, R1, R2, TMP. */ - +/* + *Load and compare a TLB entry, leaving the flags set. Returns the register + * containing the addend of the tlb entry. Clobbers t0, t1, t2, t3. + * T0 and T1 must be consecutive for LDRD. + */ static TCGReg tcg_out_tlb_read(TCGContext *s, TCGReg addrlo, TCGReg addrhi, - TCGMemOp opc, int mem_index, bool is_load) + TCGMemOp opc, int mem_index, bool is_load, + TCGReg t0, TCGReg t1, TCGReg t2, TCGReg t3) { TCGReg base = TCG_AREG0; int cmp_off = @@ -1262,36 +1265,37 @@ static TCGReg tcg_out_tlb_read(TCGContext *s, TCGReg addrlo, TCGReg addrhi, unsigned a_bits = get_alignment_bits(opc); /* V7 generates the following: - * ubfx r0, addrlo, #TARGET_PAGE_BITS, #CPU_TLB_BITS - * add r2, env, #high - * add r2, r2, r0, lsl #CPU_TLB_ENTRY_BITS - * ldr r0, [r2, #cmp] - * ldr r2, [r2, #add] - * movw tmp, #page_align_mask - * bic tmp, addrlo, tmp - * cmp r0, tmp + * ubfx t0, addrlo, #TARGET_PAGE_BITS, #CPU_TLB_BITS + * add t2, env, #high + * add t2, t2, r0, lsl #CPU_TLB_ENTRY_BITS + * ldr t0, [t2, #cmp] (and t1 w/ldrd) + * ldr t2, [t2, #add] + * movw t3, #page_align_mask + * bic t3, addrlo, t3 + * cmp t0, t3 * * Otherwise we generate: - * shr tmp, addrlo, #TARGET_PAGE_BITS - * add r2, env, #high - * and r0, tmp, #(CPU_TLB_SIZE - 1) - * add r2, r2, r0, lsl #CPU_TLB_ENTRY_BITS - * ldr r0, [r2, #cmp] - * ldr r2, [r2, #add] + * shr t3, addrlo, #TARGET_PAGE_BITS + * add t2, env, #high + * and t0, t3, #(CPU_TLB_SIZE - 1) + * add t2, t2, t0, lsl #CPU_TLB_ENTRY_BITS + * ldr t0, [t2, #cmp] (and t1 w/ldrd) + * ldr t2, [t2, #add] * tst addrlo, #s_mask - * cmpeq r0, tmp, lsl #TARGET_PAGE_BITS + * cmpeq t0, t3, lsl #TARGET_PAGE_BITS */ if (use_armv7_instructions) { - tcg_out_extract(s, COND_AL, TCG_REG_R0, addrlo, + tcg_out_extract(s, COND_AL, t0, addrlo, TARGET_PAGE_BITS, CPU_TLB_BITS); } else { - tcg_out_dat_reg(s, COND_AL, ARITH_MOV, TCG_REG_TMP, + tcg_out_dat_reg(s, COND_AL, ARITH_MOV, t3, 0, addrlo, SHIFT_IMM_LSR(TARGET_PAGE_BITS)); } /* Add portions of the offset until the memory access is in range. * If we plan on using ldrd, reduce to an 8-bit offset; otherwise - * we can use a 12-bit offset. */ + * we can use a 12-bit offset. + */ if (use_armv6_instructions && TARGET_LONG_BITS == 64) { mask_off = 0xff; } else { @@ -1301,34 +1305,33 @@ static TCGReg tcg_out_tlb_read(TCGContext *s, TCGReg addrlo, TCGReg addrhi, int shift = ctz32(cmp_off & ~mask_off) & ~1; int rot = ((32 - shift) << 7) & 0xf00; int addend = cmp_off & (0xff << shift); - tcg_out_dat_imm(s, COND_AL, ARITH_ADD, TCG_REG_R2, base, + tcg_out_dat_imm(s, COND_AL, ARITH_ADD, t2, base, rot | ((cmp_off >> shift) & 0xff)); - base = TCG_REG_R2; + base = t2; add_off -= addend; cmp_off -= addend; } if (!use_armv7_instructions) { - tcg_out_dat_imm(s, COND_AL, ARITH_AND, - TCG_REG_R0, TCG_REG_TMP, CPU_TLB_SIZE - 1); + tcg_out_dat_imm(s, COND_AL, ARITH_AND, t0, t3, CPU_TLB_SIZE - 1); } - tcg_out_dat_reg(s, COND_AL, ARITH_ADD, TCG_REG_R2, base, - TCG_REG_R0, SHIFT_IMM_LSL(CPU_TLB_ENTRY_BITS)); + tcg_out_dat_reg(s, COND_AL, ARITH_ADD, t2, base, t0, + SHIFT_IMM_LSL(CPU_TLB_ENTRY_BITS)); /* Load the tlb comparator. Use ldrd if needed and available, but due to how the pointer needs setting up, ldm isn't useful. Base arm5 doesn't have ldrd, but armv5te does. */ if (use_armv6_instructions && TARGET_LONG_BITS == 64) { - tcg_out_ldrd_8(s, COND_AL, TCG_REG_R0, TCG_REG_R2, cmp_off); + tcg_out_ldrd_8(s, COND_AL, t0, t2, cmp_off); } else { - tcg_out_ld32_12(s, COND_AL, TCG_REG_R0, TCG_REG_R2, cmp_off); + tcg_out_ld32_12(s, COND_AL, t0, t2, cmp_off); if (TARGET_LONG_BITS == 64) { - tcg_out_ld32_12(s, COND_AL, TCG_REG_R1, TCG_REG_R2, cmp_off + 4); + tcg_out_ld32_12(s, COND_AL, t1, t2, cmp_off + 4); } } /* Load the tlb addend. */ - tcg_out_ld32_12(s, COND_AL, TCG_REG_R2, TCG_REG_R2, add_off); + tcg_out_ld32_12(s, COND_AL, t2, t2, add_off); /* Check alignment. We don't support inline unaligned acceses, but we can easily support overalignment checks. */ @@ -1341,29 +1344,27 @@ static TCGReg tcg_out_tlb_read(TCGContext *s, TCGReg addrlo, TCGReg addrhi, int rot = encode_imm(mask); if (rot >= 0) { - tcg_out_dat_imm(s, COND_AL, ARITH_BIC, TCG_REG_TMP, addrlo, + tcg_out_dat_imm(s, COND_AL, ARITH_BIC, t3, addrlo, rotl(mask, rot) | (rot << 7)); } else { - tcg_out_movi32(s, COND_AL, TCG_REG_TMP, mask); - tcg_out_dat_reg(s, COND_AL, ARITH_BIC, TCG_REG_TMP, - addrlo, TCG_REG_TMP, 0); + tcg_out_movi32(s, COND_AL, t3, mask); + tcg_out_dat_reg(s, COND_AL, ARITH_BIC, t3, addrlo, t3, 0); } - tcg_out_dat_reg(s, COND_AL, ARITH_CMP, 0, TCG_REG_R0, TCG_REG_TMP, 0); + tcg_out_dat_reg(s, COND_AL, ARITH_CMP, 0, t0, t3, 0); } else { if (a_bits) { tcg_out_dat_imm(s, COND_AL, ARITH_TST, 0, addrlo, (1 << a_bits) - 1); } tcg_out_dat_reg(s, (a_bits ? COND_EQ : COND_AL), ARITH_CMP, - 0, TCG_REG_R0, TCG_REG_TMP, - SHIFT_IMM_LSL(TARGET_PAGE_BITS)); + 0, t0, t3, SHIFT_IMM_LSL(TARGET_PAGE_BITS)); } if (TARGET_LONG_BITS == 64) { - tcg_out_dat_reg(s, COND_EQ, ARITH_CMP, 0, TCG_REG_R1, addrhi, 0); + tcg_out_dat_reg(s, COND_EQ, ARITH_CMP, 0, t1, addrhi, 0); } - return TCG_REG_R2; + return t2; } /* Record the context of a call to the out of line helper code for the slow @@ -1629,7 +1630,8 @@ static void tcg_out_qemu_ld(TCGContext *s, const TCGArg *args, bool is64) #ifdef CONFIG_SOFTMMU mem_index = get_mmuidx(oi); - addend = tcg_out_tlb_read(s, addrlo, addrhi, opc, mem_index, 1); + addend = tcg_out_tlb_read(s, addrlo, addrhi, opc, mem_index, 1, + TCG_REG_R0, TCG_REG_R1, TCG_REG_R2, TCG_REG_R14); /* This a conditional BL only to load a pointer within this opcode into LR for the slow path. We will not be using the value for a tail call. */ @@ -1760,7 +1762,8 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) #ifdef CONFIG_SOFTMMU mem_index = get_mmuidx(oi); - addend = tcg_out_tlb_read(s, addrlo, addrhi, opc, mem_index, 0); + addend = tcg_out_tlb_read(s, addrlo, addrhi, opc, mem_index, 0, + TCG_REG_R0, TCG_REG_R1, TCG_REG_R2, TCG_REG_R14); tcg_out_qemu_st_index(s, COND_EQ, opc, datalo, datahi, addrlo, addend); From patchwork Mon Nov 12 21:45:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996689 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="BzDJMzcx"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4Vb3Yywz9s3x for ; Tue, 13 Nov 2018 09:02:15 +1100 (AEDT) Received: from localhost ([::1]:50884 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMKHd-0008Js-1o for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 17:02:13 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54334) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK3W-0004bP-UZ for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:40 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK3M-00056B-C9 for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:30 -0500 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]:46966) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK3K-000523-FY for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:26 -0500 Received: by mail-wr1-x441.google.com with SMTP id 74-v6so10991509wrb.13 for ; Mon, 12 Nov 2018 13:47:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=K9NRP+HhbIk1nF+G1KmKV31hLuf4u/VMHyrlWpIJenM=; b=BzDJMzcxvGuEyXfSWBBktBb7SO9Q2Q1iiTTQvZIVVrPHxzSXP03QXjIB5cW7BGbRXV ehQkoVlv0rKgk7nKcScmrHqoDTJM/fn7+JmvOfszJUmRUSQ8glOgfk2sWEnQdgdqE4Wl E+zvPu3fK9QThDjiye6NqZR91rsCO9DqJd0dY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=K9NRP+HhbIk1nF+G1KmKV31hLuf4u/VMHyrlWpIJenM=; b=kGrecwN1+ifV9M3br50i78grxsVfHBnJmjRJpmkX5KV6CkxLDtccf+J5/c8dn7bo8a Usu/x58IGSZjoBK+/2NbSkz0ZPZY68hDAMTRlpvCau4v7ot2XhnuWQfEo0BtkLAZw/ac 9IlMixDSVpZuvRrWP1GMSE1pa3UMxnZGTJ5k1Xze7PGt4hljdMPBOSMnXg0QAUTjgdn6 6FkABG3TH6xHmTF+j7CrGpXxChPQhGewATqzzTAqULqrUjLM6HXuaBOeTUWKSilks3nw BKE7tFqzZLaqnirjBDH7kIWNQZ+f9zKLqfktJ1qXLV767UDaFvig5HgcyhmaBtZ0fC2f O4VA== X-Gm-Message-State: AGRZ1gIOxh02kBs75wQ8NTak8wB3TN+96L8TxbzxFCJTDND7KjEf4U5d YoH/99Mx+mCnzLVMTEq4+Vs3K8JnVST2pA== X-Google-Smtp-Source: AJdET5eC8UG0HHYa8IqQ6KfBPvd4IlZ+/MNK2fGBXaMGpt49WKkbhC+dpfFeZT0a+Iz4j5pTfSaVaA== X-Received: by 2002:adf:b453:: with SMTP id v19-v6mr2526849wrd.47.1542059242038; Mon, 12 Nov 2018 13:47:22 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.21 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:21 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:45:00 +0100 Message-Id: <20181112214503.22941-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::441 Subject: [Qemu-devel] [PATCH for-4.0 14/17] tcg/arm: Add constraints for R0-R5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" These are function call arguments that we will need soon. Signed-off-by: Richard Henderson --- tcg/arm/tcg-target.inc.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tcg/arm/tcg-target.inc.c b/tcg/arm/tcg-target.inc.c index 414c91c9ea..4339c472e8 100644 --- a/tcg/arm/tcg-target.inc.c +++ b/tcg/arm/tcg-target.inc.c @@ -246,7 +246,12 @@ static bool patch_reloc(tcg_insn_unit *code_ptr, int type, static const char *target_parse_constraint(TCGArgConstraint *ct, const char *ct_str, TCGType type) { - switch (*ct_str++) { + char c = *ct_str++; + switch (c) { + case 'a' ... 'f': /* r0 - r5 */ + ct->ct |= TCG_CT_REG; + tcg_regset_set_reg(ct->u.regs, TCG_REG_R0 + (c - 'a')); + break; case 'I': ct->ct |= TCG_CT_CONST_ARM; break; From patchwork Mon Nov 12 21:45:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996694 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="O+ZzmcCK"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4ZN21hVz9s8J for ; Tue, 13 Nov 2018 09:05:32 +1100 (AEDT) Received: from localhost ([::1]:50898 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMKKn-0004TV-QC for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 17:05:29 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54345) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK3c-000517-1X for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:45 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK3Q-000584-9J for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:36 -0500 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]:38329) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK3K-00052V-Mu for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:28 -0500 Received: by mail-wr1-x443.google.com with SMTP id e3-v6so11033644wrs.5 for ; Mon, 12 Nov 2018 13:47:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=IfteVwIt0QRdTAT9d1pABeNMo7np6fWushm//ba1JvM=; b=O+ZzmcCKCaCMcGhAgoet9gqgPKqzMT1FQBFZT3/tBFk4uz88Mau4aaUAFmhARGRr/M j3wufT0Qs1bSA0xBIAo27CFHpMDJZp/UkqMsKBZ3YGwIF6LOLiuKFbVqZK6hCFbLOILm ADqRVDNTlbdynNfw6+Z6fH2Q13SOBLMgQRtro= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=IfteVwIt0QRdTAT9d1pABeNMo7np6fWushm//ba1JvM=; b=pDyfkNb9PYPB3/WFKEeDCViyJQd1NBFqQOBzBTAQO7Gu/apw4XkxJopUqcDCi4OrGp we3t2xy0k60LKeKL4VdGtp+rSG9kvmergLvn1z92o7hAiaHdf3BH/Ea4AFuZjowQ+18B TVcZY1+b917Xn6REk5LWndn++6BbI/Vnmd7xhwZVucHCcUWCKxPxZKlLV3seEdW2GeUz fdBVG2s85CTZe8xHyhdGoTsV/VaveNPB/CwZe+RHYuh99sn9m/duEGb0RAq0YtDBXWNx hBpqY/odejgqImsm+PZc6w5xGMhdZCJu5Ba4v3Tkqn9GkuJV9WqQa023NmGYMl+8Esp2 1yVA== X-Gm-Message-State: AGRZ1gK17ohjsQr30hZivQYPV1AL2LSsHDdcxtBq+KgGqAHMTR7MBIvC ZrxUTua4ulOAC01bREAU530LzAIrtV9cPA== X-Google-Smtp-Source: AJdET5cmETywn6GNR5GeRrAsm853URZ4DnfJ+nc3CfrYpfTX5pZaN6Tv2nJhPLA0SWOY/3VCYASoXg== X-Received: by 2002:adf:92e5:: with SMTP id 92-v6mr2717664wrn.124.1542059242964; Mon, 12 Nov 2018 13:47:22 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.22 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:22 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:45:01 +0100 Message-Id: <20181112214503.22941-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::443 Subject: [Qemu-devel] [PATCH for-4.0 15/17] tcg/arm: Reduce the number of temps for tcg_out_tlb_read X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" When moving the qemu_ld/st thunk out of line, we no longer have LR for use as a temporary. In the worst case we must make do with 3 temps, when dealing with a 64-bit guest address. This in turn imples that we cannot use LDRD anymore, as there are not enough temps. Signed-off-by: Richard Henderson --- tcg/arm/tcg-target.inc.c | 97 ++++++++++++++++++++++------------------ 1 file changed, 53 insertions(+), 44 deletions(-) diff --git a/tcg/arm/tcg-target.inc.c b/tcg/arm/tcg-target.inc.c index 4339c472e8..2deeb1f5d1 100644 --- a/tcg/arm/tcg-target.inc.c +++ b/tcg/arm/tcg-target.inc.c @@ -1251,13 +1251,12 @@ static TCGReg tcg_out_arg_reg64(TCGContext *s, TCGReg argreg, QEMU_BUILD_BUG_ON(CPU_TLB_BITS > 8); /* - *Load and compare a TLB entry, leaving the flags set. Returns the register - * containing the addend of the tlb entry. Clobbers t0, t1, t2, t3. - * T0 and T1 must be consecutive for LDRD. + * Load and compare a TLB entry, leaving the flags set. Returns the register + * containing the addend of the tlb entry. Clobbers t0, t1, t2. */ static TCGReg tcg_out_tlb_read(TCGContext *s, TCGReg addrlo, TCGReg addrhi, TCGMemOp opc, int mem_index, bool is_load, - TCGReg t0, TCGReg t1, TCGReg t2, TCGReg t3) + TCGReg t0, TCGReg t1, TCGReg t2) { TCGReg base = TCG_AREG0; int cmp_off = @@ -1265,49 +1264,64 @@ static TCGReg tcg_out_tlb_read(TCGContext *s, TCGReg addrlo, TCGReg addrhi, ? offsetof(CPUArchState, tlb_table[mem_index][0].addr_read) : offsetof(CPUArchState, tlb_table[mem_index][0].addr_write)); int add_off = offsetof(CPUArchState, tlb_table[mem_index][0].addend); - int mask_off; unsigned s_bits = opc & MO_SIZE; unsigned a_bits = get_alignment_bits(opc); /* V7 generates the following: * ubfx t0, addrlo, #TARGET_PAGE_BITS, #CPU_TLB_BITS * add t2, env, #high - * add t2, t2, r0, lsl #CPU_TLB_ENTRY_BITS - * ldr t0, [t2, #cmp] (and t1 w/ldrd) + * add t2, t2, t0, lsl #CPU_TLB_ENTRY_BITS + * ldr t0, [t2, #cmp] * ldr t2, [t2, #add] - * movw t3, #page_align_mask - * bic t3, addrlo, t3 - * cmp t0, t3 + * movw t1, #page_align_mask + * bic t1, addrlo, t1 + * cmp t0, t1 + * + * ubfx t0, addrlo, #TPB, #CTB -- 64-bit address + * add t2, env, #high + * add t2, t2, t0, lsl #CTEB + * ldr t0, [t2, #cmplo] + * movw t1, #page_align_mask + * bic t1, addrlo, t1 + * cmp t0, t1 + * ldr t0, [t2, #cmphi] + * ldr t2, [t2, #add] + * cmpeq t0, addrhi * * Otherwise we generate: * shr t3, addrlo, #TARGET_PAGE_BITS * add t2, env, #high * and t0, t3, #(CPU_TLB_SIZE - 1) * add t2, t2, t0, lsl #CPU_TLB_ENTRY_BITS - * ldr t0, [t2, #cmp] (and t1 w/ldrd) + * ldr t0, [t2, #cmp] * ldr t2, [t2, #add] * tst addrlo, #s_mask * cmpeq t0, t3, lsl #TARGET_PAGE_BITS + * + * shr t1, addrlo, #TPB -- 64-bit address + * add t2, env, #high + * and t0, t1, #CTS-1 + * add t2, t2, t0, lsl #CTEB + * ldr t0, [t2, #cmplo] + * tst addrlo, #s_mask + * cmpeq t0, t1, lsl #TBP + * ldr t0, [t2, #cmphi] + * ldr t2, [t2, #add] + * cmpeq t0, addrhi */ if (use_armv7_instructions) { tcg_out_extract(s, COND_AL, t0, addrlo, TARGET_PAGE_BITS, CPU_TLB_BITS); } else { - tcg_out_dat_reg(s, COND_AL, ARITH_MOV, t3, + tcg_out_dat_reg(s, COND_AL, ARITH_MOV, t1, 0, addrlo, SHIFT_IMM_LSR(TARGET_PAGE_BITS)); } /* Add portions of the offset until the memory access is in range. - * If we plan on using ldrd, reduce to an 8-bit offset; otherwise - * we can use a 12-bit offset. + * We are not using ldrd, so we can use a 12-bit offset. */ - if (use_armv6_instructions && TARGET_LONG_BITS == 64) { - mask_off = 0xff; - } else { - mask_off = 0xfff; - } - while (cmp_off > mask_off) { - int shift = ctz32(cmp_off & ~mask_off) & ~1; + while (cmp_off > 0xfff) { + int shift = ctz32(cmp_off & ~0xfff) & ~1; int rot = ((32 - shift) << 7) & 0xf00; int addend = cmp_off & (0xff << shift); tcg_out_dat_imm(s, COND_AL, ARITH_ADD, t2, base, @@ -1318,25 +1332,13 @@ static TCGReg tcg_out_tlb_read(TCGContext *s, TCGReg addrlo, TCGReg addrhi, } if (!use_armv7_instructions) { - tcg_out_dat_imm(s, COND_AL, ARITH_AND, t0, t3, CPU_TLB_SIZE - 1); + tcg_out_dat_imm(s, COND_AL, ARITH_AND, t0, t1, CPU_TLB_SIZE - 1); } tcg_out_dat_reg(s, COND_AL, ARITH_ADD, t2, base, t0, SHIFT_IMM_LSL(CPU_TLB_ENTRY_BITS)); - /* Load the tlb comparator. Use ldrd if needed and available, - but due to how the pointer needs setting up, ldm isn't useful. - Base arm5 doesn't have ldrd, but armv5te does. */ - if (use_armv6_instructions && TARGET_LONG_BITS == 64) { - tcg_out_ldrd_8(s, COND_AL, t0, t2, cmp_off); - } else { - tcg_out_ld32_12(s, COND_AL, t0, t2, cmp_off); - if (TARGET_LONG_BITS == 64) { - tcg_out_ld32_12(s, COND_AL, t1, t2, cmp_off + 4); - } - } - - /* Load the tlb addend. */ - tcg_out_ld32_12(s, COND_AL, t2, t2, add_off); + /* Load the tlb comparator (low part). */ + tcg_out_ld32_12(s, COND_AL, t0, t2, cmp_off); /* Check alignment. We don't support inline unaligned acceses, but we can easily support overalignment checks. */ @@ -1349,24 +1351,31 @@ static TCGReg tcg_out_tlb_read(TCGContext *s, TCGReg addrlo, TCGReg addrhi, int rot = encode_imm(mask); if (rot >= 0) { - tcg_out_dat_imm(s, COND_AL, ARITH_BIC, t3, addrlo, + tcg_out_dat_imm(s, COND_AL, ARITH_BIC, t1, addrlo, rotl(mask, rot) | (rot << 7)); } else { - tcg_out_movi32(s, COND_AL, t3, mask); - tcg_out_dat_reg(s, COND_AL, ARITH_BIC, t3, addrlo, t3, 0); + tcg_out_movi32(s, COND_AL, t1, mask); + tcg_out_dat_reg(s, COND_AL, ARITH_BIC, t1, addrlo, t1, 0); } - tcg_out_dat_reg(s, COND_AL, ARITH_CMP, 0, t0, t3, 0); + tcg_out_dat_reg(s, COND_AL, ARITH_CMP, 0, t0, t1, 0); } else { if (a_bits) { tcg_out_dat_imm(s, COND_AL, ARITH_TST, 0, addrlo, (1 << a_bits) - 1); } tcg_out_dat_reg(s, (a_bits ? COND_EQ : COND_AL), ARITH_CMP, - 0, t0, t3, SHIFT_IMM_LSL(TARGET_PAGE_BITS)); + 0, t0, t1, SHIFT_IMM_LSL(TARGET_PAGE_BITS)); } + /* Load the tlb comparator (high part). */ if (TARGET_LONG_BITS == 64) { - tcg_out_dat_reg(s, COND_EQ, ARITH_CMP, 0, t1, addrhi, 0); + tcg_out_ld32_12(s, COND_AL, t0, t2, cmp_off + 4); + } + /* Load the tlb addend. */ + tcg_out_ld32_12(s, COND_AL, t2, t2, add_off); + + if (TARGET_LONG_BITS == 64) { + tcg_out_dat_reg(s, COND_EQ, ARITH_CMP, 0, t0, addrhi, 0); } return t2; @@ -1636,7 +1645,7 @@ static void tcg_out_qemu_ld(TCGContext *s, const TCGArg *args, bool is64) #ifdef CONFIG_SOFTMMU mem_index = get_mmuidx(oi); addend = tcg_out_tlb_read(s, addrlo, addrhi, opc, mem_index, 1, - TCG_REG_R0, TCG_REG_R1, TCG_REG_R2, TCG_REG_R14); + TCG_REG_R0, TCG_REG_R1, TCG_REG_TMP); /* This a conditional BL only to load a pointer within this opcode into LR for the slow path. We will not be using the value for a tail call. */ @@ -1768,7 +1777,7 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) #ifdef CONFIG_SOFTMMU mem_index = get_mmuidx(oi); addend = tcg_out_tlb_read(s, addrlo, addrhi, opc, mem_index, 0, - TCG_REG_R0, TCG_REG_R1, TCG_REG_R2, TCG_REG_R14); + TCG_REG_R0, TCG_REG_R1, TCG_REG_TMP); tcg_out_qemu_st_index(s, COND_EQ, opc, datalo, datahi, addrlo, addend); From patchwork Mon Nov 12 21:45:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996698 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="ILboXuiK"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4fR3VCJz9s7W for ; Tue, 13 Nov 2018 09:09:03 +1100 (AEDT) Received: from localhost ([::1]:50927 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMKOD-0001Ek-2l for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 17:09:01 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54341) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK3c-000516-1X for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:45 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK3Q-000589-9M for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:36 -0500 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]:42473) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK3K-00053J-NB for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:28 -0500 Received: by mail-wr1-x441.google.com with SMTP id u5-v6so5738312wrn.9 for ; Mon, 12 Nov 2018 13:47:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=gatNgaJJNEzIjF8x8UBmTuhQzcMQo5jvEDvLqmivJn4=; b=ILboXuiKS/VrioKlCBIzl4TXOgeMbHONnELGwJmC7CWYlqL466IfPYZM19MTNo3VRF 1LbGuwYW6u593jRR5ntUZJouPU8QsYBXpyzGG/5O8blSWdR+qvn/o1n03uATJ225/Ie2 uUTlT1XULdrrIB/1+hxW+5aIx/YFVms5bclWQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=gatNgaJJNEzIjF8x8UBmTuhQzcMQo5jvEDvLqmivJn4=; b=BitFPsA5ckulEO0x6g/IbTBkaQEIip2xYI7yQ8FnRWEgXSQZEhHEJPeSJGUQQoRiwn FuxBjhIJhS8aYlVjo0sQlhtVszJiUColSeUAAfPEYwozeGN2PVpGUPWv8vuML9g+nC09 xFVcaOnRp+u2TwX3OV3B0c/zryVVej3uWgwwKLFnGYbEPN1Tu9ubH4vDltI0fTX6qx+6 kPjNdezjiZOhp2Sa3nDBlALJfO0fY8sqQsacNp+6plisffI3nm5iytL4hkdXovuRWUII Szs2AN6n+IF4+iznl9bNjdRox1iCKtGhmYnFbAnVe9VTHGCHrDUpeNqiqj+8Zb4vhSP7 Bk1w== X-Gm-Message-State: AGRZ1gJqQMmH6TfKP0FvdtaUhii7rZeW0bl8JuKOHTv3JHlJx5K24FGW w+PZrxWgHfONIHzCJWNFpK1cduGdTXlFAA== X-Google-Smtp-Source: AJdET5egi3osOO2Vk0O3AZFCbX6HAHngLWf45FTTxdmuByPewZoRyErBoUOHXIP7xi+pBxEgz8YBjg== X-Received: by 2002:adf:e781:: with SMTP id n1-v6mr2760689wrm.115.1542059244305; Mon, 12 Nov 2018 13:47:24 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.23 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:23 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:45:02 +0100 Message-Id: <20181112214503.22941-17-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::441 Subject: [Qemu-devel] [PATCH for-4.0 16/17] tcg/arm: Force qemu_ld/st arguments into fixed registers X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This is an incremental step toward moving the qemu_ld/st code sequence out of line. Signed-off-by: Richard Henderson --- tcg/arm/tcg-target.inc.c | 116 +++++++++++++++++++++++++-------------- 1 file changed, 75 insertions(+), 41 deletions(-) diff --git a/tcg/arm/tcg-target.inc.c b/tcg/arm/tcg-target.inc.c index 2deeb1f5d1..75589b43e2 100644 --- a/tcg/arm/tcg-target.inc.c +++ b/tcg/arm/tcg-target.inc.c @@ -270,38 +270,15 @@ static const char *target_parse_constraint(TCGArgConstraint *ct, ct->u.regs = 0xffff; break; - /* qemu_ld address */ - case 'l': - ct->ct |= TCG_CT_REG; - ct->u.regs = 0xffff; -#ifdef CONFIG_SOFTMMU - /* r0-r2,lr will be overwritten when reading the tlb entry, - so don't use these. */ - tcg_regset_reset_reg(ct->u.regs, TCG_REG_R0); - tcg_regset_reset_reg(ct->u.regs, TCG_REG_R1); - tcg_regset_reset_reg(ct->u.regs, TCG_REG_R2); - tcg_regset_reset_reg(ct->u.regs, TCG_REG_R14); -#endif - break; - +#ifndef CONFIG_SOFTMMU /* qemu_st address & data */ case 's': ct->ct |= TCG_CT_REG; ct->u.regs = 0xffff; - /* r0-r2 will be overwritten when reading the tlb entry (softmmu only) - and r0-r1 doing the byte swapping, so don't use these. */ + /* r0 and tmp are needed for byte swapping. */ tcg_regset_reset_reg(ct->u.regs, TCG_REG_R0); - tcg_regset_reset_reg(ct->u.regs, TCG_REG_R1); -#if defined(CONFIG_SOFTMMU) - /* Avoid clashes with registers being used for helper args */ - tcg_regset_reset_reg(ct->u.regs, TCG_REG_R2); -#if TARGET_LONG_BITS == 64 - /* Avoid clashes with registers being used for helper args */ - tcg_regset_reset_reg(ct->u.regs, TCG_REG_R3); -#endif - tcg_regset_reset_reg(ct->u.regs, TCG_REG_R14); -#endif break; +#endif default: return NULL; @@ -1630,8 +1607,8 @@ static void tcg_out_qemu_ld(TCGContext *s, const TCGArg *args, bool is64) TCGMemOpIdx oi; TCGMemOp opc; #ifdef CONFIG_SOFTMMU - int mem_index; - TCGReg addend; + int mem_index, avail; + TCGReg addend, t0, t1; tcg_insn_unit *label_ptr; #endif @@ -1644,8 +1621,20 @@ static void tcg_out_qemu_ld(TCGContext *s, const TCGArg *args, bool is64) #ifdef CONFIG_SOFTMMU mem_index = get_mmuidx(oi); + + avail = 0xf; + avail &= ~(1 << addrlo); + if (TARGET_LONG_BITS == 64) { + avail &= ~(1 << addrhi); + } + tcg_debug_assert(avail & 1); + t0 = TCG_REG_R0; + avail &= ~1; + tcg_debug_assert(avail != 0); + t1 = ctz32(avail); + addend = tcg_out_tlb_read(s, addrlo, addrhi, opc, mem_index, 1, - TCG_REG_R0, TCG_REG_R1, TCG_REG_TMP); + t0, t1, TCG_REG_TMP); /* This a conditional BL only to load a pointer within this opcode into LR for the slow path. We will not be using the value for a tail call. */ @@ -1762,8 +1751,8 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) TCGMemOpIdx oi; TCGMemOp opc; #ifdef CONFIG_SOFTMMU - int mem_index; - TCGReg addend; + int mem_index, avail; + TCGReg addend, t0, t1; tcg_insn_unit *label_ptr; #endif @@ -1776,8 +1765,24 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) #ifdef CONFIG_SOFTMMU mem_index = get_mmuidx(oi); + + avail = 0xf; + avail &= ~(1 << addrlo); + avail &= ~(1 << datalo); + if (TARGET_LONG_BITS == 64) { + avail &= ~(1 << addrhi); + } + if (is64) { + avail &= ~(1 << datahi); + } + tcg_debug_assert(avail & 1); + t0 = TCG_REG_R0; + avail &= ~1; + tcg_debug_assert(avail != 0); + t1 = ctz32(avail); + addend = tcg_out_tlb_read(s, addrlo, addrhi, opc, mem_index, 0, - TCG_REG_R0, TCG_REG_R1, TCG_REG_TMP); + t0, t1, TCG_REG_TMP); tcg_out_qemu_st_index(s, COND_EQ, opc, datalo, datahi, addrlo, addend); @@ -2118,11 +2123,14 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) static const TCGTargetOpDef r = { .args_ct_str = { "r" } }; static const TCGTargetOpDef r_r = { .args_ct_str = { "r", "r" } }; static const TCGTargetOpDef s_s = { .args_ct_str = { "s", "s" } }; - static const TCGTargetOpDef r_l = { .args_ct_str = { "r", "l" } }; + static const TCGTargetOpDef a_b = { .args_ct_str = { "a", "b" } }; + static const TCGTargetOpDef c_b = { .args_ct_str = { "c", "b" } }; static const TCGTargetOpDef r_r_r = { .args_ct_str = { "r", "r", "r" } }; - static const TCGTargetOpDef r_r_l = { .args_ct_str = { "r", "r", "l" } }; - static const TCGTargetOpDef r_l_l = { .args_ct_str = { "r", "l", "l" } }; static const TCGTargetOpDef s_s_s = { .args_ct_str = { "s", "s", "s" } }; + static const TCGTargetOpDef a_c_d = { .args_ct_str = { "a", "c", "d" } }; + static const TCGTargetOpDef a_b_b = { .args_ct_str = { "a", "b", "b" } }; + static const TCGTargetOpDef e_c_d = { .args_ct_str = { "e", "c", "d" } }; + static const TCGTargetOpDef e_f_b = { .args_ct_str = { "e", "f", "b" } }; static const TCGTargetOpDef r_r_ri = { .args_ct_str = { "r", "r", "ri" } }; static const TCGTargetOpDef r_r_rI = { .args_ct_str = { "r", "r", "rI" } }; static const TCGTargetOpDef r_r_rIN @@ -2131,10 +2139,12 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) = { .args_ct_str = { "r", "r", "rIK" } }; static const TCGTargetOpDef r_r_r_r = { .args_ct_str = { "r", "r", "r", "r" } }; - static const TCGTargetOpDef r_r_l_l - = { .args_ct_str = { "r", "r", "l", "l" } }; static const TCGTargetOpDef s_s_s_s = { .args_ct_str = { "s", "s", "s", "s" } }; + static const TCGTargetOpDef a_b_c_d + = { .args_ct_str = { "a", "b", "c", "d" } }; + static const TCGTargetOpDef e_f_c_d + = { .args_ct_str = { "e", "f", "c", "d" } }; static const TCGTargetOpDef br = { .args_ct_str = { "r", "rIN" } }; static const TCGTargetOpDef dep @@ -2215,13 +2225,37 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) return &setc2; case INDEX_op_qemu_ld_i32: - return TARGET_LONG_BITS == 32 ? &r_l : &r_l_l; + if (!USING_SOFTMMU) { + return TARGET_LONG_BITS == 32 ? &r_r : &r_r_r; + } else if (TARGET_LONG_BITS == 32) { + return &a_b; /* temps available r0, r2, r3, r12 */ + } else { + return &a_c_d; /* temps available r0, r1, r12 */ + } case INDEX_op_qemu_ld_i64: - return TARGET_LONG_BITS == 32 ? &r_r_l : &r_r_l_l; + if (!USING_SOFTMMU) { + return TARGET_LONG_BITS == 32 ? &r_r_r : &r_r_r_r; + } else if (TARGET_LONG_BITS == 32) { + return &a_b_b; /* temps available r0, r2, r3, r12 */ + } else { + return &a_b_c_d; /* temps available r0, r1, r12 */ + } case INDEX_op_qemu_st_i32: - return TARGET_LONG_BITS == 32 ? &s_s : &s_s_s; + if (!USING_SOFTMMU) { + return TARGET_LONG_BITS == 32 ? &s_s : &s_s_s; + } else if (TARGET_LONG_BITS == 32) { + return &c_b; /* temps available r0, r3, r12 */ + } else { + return &e_c_d; /* temps available r0, r1, r12 */ + } case INDEX_op_qemu_st_i64: - return TARGET_LONG_BITS == 32 ? &s_s_s : &s_s_s_s; + if (!USING_SOFTMMU) { + return TARGET_LONG_BITS == 32 ? &s_s_s : &s_s_s_s; + } else if (TARGET_LONG_BITS == 32) { + return &e_f_b; /* temps available r0, r2, r3, r12 */ + } else { + return &e_f_c_d; /* temps available r0, r1, r12 */ + } default: return NULL; From patchwork Mon Nov 12 21:45:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 996693 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="W8lyesAx"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42v4ZB21Tnz9s7W for ; Tue, 13 Nov 2018 09:05:22 +1100 (AEDT) Received: from localhost ([::1]:50896 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMKKd-0004HY-SG for incoming@patchwork.ozlabs.org; Mon, 12 Nov 2018 17:05:19 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54372) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK3g-0005O7-76 for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:51 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK3V-00059J-2h for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:42 -0500 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]:34212) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK3M-00054c-9L for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:30 -0500 Received: by mail-wr1-x441.google.com with SMTP id j26-v6so11019398wre.1 for ; Mon, 12 Nov 2018 13:47:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=HJ9l80504gpqFgJimNa9/JN+GlgtoRgrlfqSSOMxBo8=; b=W8lyesAxftS/dys1/+n0tuRfLVdtshz4NnxGlXRBQ4vLQTJTxb01de/d3IFMHAzuHi YTtZ4445NvK3o/+IdB6BQI7jojXp5MecfwMAkykl14Gt2JtUwc9mSnjlXbIBNTqmuwaw Q5A2QOooXNuYnVhP+Ir4gczYAVcOcp8mSo2ys= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=HJ9l80504gpqFgJimNa9/JN+GlgtoRgrlfqSSOMxBo8=; b=r4oav45ghHN89hwHxP3nMB2zwnCyASCIyvD4hh3V3uDnzJGdAHGy4Iq6c9SbA6XA5N 1Zfvdy3j0J03qh2NyYBATzM7H5GRIx6S81p/OVJiUAWD/1dxaez0/eDO6d/Dne/pPa93 ghGarbG5Z1X9XMNcOwr7Awgd9Mn4f4Z3snZtZIkkzm+VdAQRMb78UGfJSu8ftqmIvAo3 YgORAwq0LLwmol+3UA481WfzbiBf6xL/z8+fFUHcSUCSVBwCnzsI6MIDb5pBgwFg6ucz wYjCzQyBzS834AhCmNNq5TdLvX+hxpn5xzZu3D4nXY8+RHZUWVS2vQeDLiYxluzue5hz aZeA== X-Gm-Message-State: AGRZ1gIm6HYvbgGpcZxEOU4B70GgZ5oZo7iCa4eLtT02+B/TPd7J6b8H RHZVvpjyG5pYcZq7+Uyr9O8j2v6ePrREnw== X-Google-Smtp-Source: AJdET5dFe7L0elU3GYDu5FbnexsS+i2DJlQEy7MdV7WebLFmDYeCocTvO1xZXUw8sgtR3BmbU0TI5g== X-Received: by 2002:a5d:6050:: with SMTP id j16-v6mr2496631wrt.301.1542059245321; Mon, 12 Nov 2018 13:47:25 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.24 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:24 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:45:03 +0100 Message-Id: <20181112214503.22941-18-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::441 Subject: [Qemu-devel] [PATCH for-4.0 17/17] tcg/arm: Use TCG_TARGET_NEED_LDST_OOL_LABELS X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- tcg/arm/tcg-target.h | 2 +- tcg/arm/tcg-target.inc.c | 302 +++++++++++++++------------------------ 2 files changed, 118 insertions(+), 186 deletions(-) diff --git a/tcg/arm/tcg-target.h b/tcg/arm/tcg-target.h index 94b3578c55..02981abdcc 100644 --- a/tcg/arm/tcg-target.h +++ b/tcg/arm/tcg-target.h @@ -141,7 +141,7 @@ static inline void flush_icache_range(uintptr_t start, uintptr_t stop) void tb_target_set_jmp_target(uintptr_t, uintptr_t, uintptr_t); #ifdef CONFIG_SOFTMMU -#define TCG_TARGET_NEED_LDST_LABELS +#define TCG_TARGET_NEED_LDST_OOL_LABELS #endif #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/arm/tcg-target.inc.c b/tcg/arm/tcg-target.inc.c index 75589b43e2..1e9ff693d9 100644 --- a/tcg/arm/tcg-target.inc.c +++ b/tcg/arm/tcg-target.inc.c @@ -1134,7 +1134,7 @@ static TCGCond tcg_out_cmp2(TCGContext *s, const TCGArg *args, } #ifdef CONFIG_SOFTMMU -#include "tcg-ldst.inc.c" +#include "tcg-ldst-ool.inc.c" /* helper signature: helper_ret_ld_mmu(CPUState *env, target_ulong addr, * int mmu_idx, uintptr_t ra) @@ -1358,127 +1358,6 @@ static TCGReg tcg_out_tlb_read(TCGContext *s, TCGReg addrlo, TCGReg addrhi, return t2; } -/* Record the context of a call to the out of line helper code for the slow - path for a load or store, so that we can later generate the correct - helper code. */ -static void add_qemu_ldst_label(TCGContext *s, bool is_ld, TCGMemOpIdx oi, - TCGReg datalo, TCGReg datahi, TCGReg addrlo, - TCGReg addrhi, tcg_insn_unit *raddr, - tcg_insn_unit *label_ptr) -{ - TCGLabelQemuLdst *label = new_ldst_label(s); - - label->is_ld = is_ld; - label->oi = oi; - label->datalo_reg = datalo; - label->datahi_reg = datahi; - label->addrlo_reg = addrlo; - label->addrhi_reg = addrhi; - label->raddr = raddr; - label->label_ptr[0] = label_ptr; -} - -static void tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) -{ - TCGReg argreg, datalo, datahi; - TCGMemOpIdx oi = lb->oi; - TCGMemOp opc = get_memop(oi); - void *func; - - reloc_pc24(lb->label_ptr[0], s->code_ptr); - - argreg = tcg_out_arg_reg32(s, TCG_REG_R0, TCG_AREG0); - if (TARGET_LONG_BITS == 64) { - argreg = tcg_out_arg_reg64(s, argreg, lb->addrlo_reg, lb->addrhi_reg); - } else { - argreg = tcg_out_arg_reg32(s, argreg, lb->addrlo_reg); - } - argreg = tcg_out_arg_imm32(s, argreg, oi); - argreg = tcg_out_arg_reg32(s, argreg, TCG_REG_R14); - - /* For armv6 we can use the canonical unsigned helpers and minimize - icache usage. For pre-armv6, use the signed helpers since we do - not have a single insn sign-extend. */ - if (use_armv6_instructions) { - func = qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]; - } else { - func = qemu_ld_helpers[opc & (MO_BSWAP | MO_SSIZE)]; - if (opc & MO_SIGN) { - opc = MO_UL; - } - } - tcg_out_call(s, func); - - datalo = lb->datalo_reg; - datahi = lb->datahi_reg; - switch (opc & MO_SSIZE) { - case MO_SB: - tcg_out_ext8s(s, COND_AL, datalo, TCG_REG_R0); - break; - case MO_SW: - tcg_out_ext16s(s, COND_AL, datalo, TCG_REG_R0); - break; - default: - tcg_out_mov_reg(s, COND_AL, datalo, TCG_REG_R0); - break; - case MO_Q: - if (datalo != TCG_REG_R1) { - tcg_out_mov_reg(s, COND_AL, datalo, TCG_REG_R0); - tcg_out_mov_reg(s, COND_AL, datahi, TCG_REG_R1); - } else if (datahi != TCG_REG_R0) { - tcg_out_mov_reg(s, COND_AL, datahi, TCG_REG_R1); - tcg_out_mov_reg(s, COND_AL, datalo, TCG_REG_R0); - } else { - tcg_out_mov_reg(s, COND_AL, TCG_REG_TMP, TCG_REG_R0); - tcg_out_mov_reg(s, COND_AL, datahi, TCG_REG_R1); - tcg_out_mov_reg(s, COND_AL, datalo, TCG_REG_TMP); - } - break; - } - - tcg_out_goto(s, COND_AL, lb->raddr); -} - -static void tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) -{ - TCGReg argreg, datalo, datahi; - TCGMemOpIdx oi = lb->oi; - TCGMemOp opc = get_memop(oi); - - reloc_pc24(lb->label_ptr[0], s->code_ptr); - - argreg = TCG_REG_R0; - argreg = tcg_out_arg_reg32(s, argreg, TCG_AREG0); - if (TARGET_LONG_BITS == 64) { - argreg = tcg_out_arg_reg64(s, argreg, lb->addrlo_reg, lb->addrhi_reg); - } else { - argreg = tcg_out_arg_reg32(s, argreg, lb->addrlo_reg); - } - - datalo = lb->datalo_reg; - datahi = lb->datahi_reg; - switch (opc & MO_SIZE) { - case MO_8: - argreg = tcg_out_arg_reg8(s, argreg, datalo); - break; - case MO_16: - argreg = tcg_out_arg_reg16(s, argreg, datalo); - break; - case MO_32: - default: - argreg = tcg_out_arg_reg32(s, argreg, datalo); - break; - case MO_64: - argreg = tcg_out_arg_reg64(s, argreg, datalo, datahi); - break; - } - - argreg = tcg_out_arg_imm32(s, argreg, oi); - argreg = tcg_out_arg_reg32(s, argreg, TCG_REG_R14); - - /* Tail-call to the helper, which will return to the fast path. */ - tcg_out_goto(s, COND_AL, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); -} #endif /* SOFTMMU */ static inline void tcg_out_qemu_ld_index(TCGContext *s, TCGMemOp opc, @@ -1603,54 +1482,28 @@ static inline void tcg_out_qemu_ld_direct(TCGContext *s, TCGMemOp opc, static void tcg_out_qemu_ld(TCGContext *s, const TCGArg *args, bool is64) { - TCGReg addrlo, datalo, datahi, addrhi __attribute__((unused)); + TCGReg addrlo __attribute__((unused)); + TCGReg addrhi __attribute__((unused)); + TCGReg datalo __attribute__((unused)); + TCGReg datahi __attribute__((unused)); TCGMemOpIdx oi; - TCGMemOp opc; -#ifdef CONFIG_SOFTMMU - int mem_index, avail; - TCGReg addend, t0, t1; - tcg_insn_unit *label_ptr; -#endif datalo = *args++; datahi = (is64 ? *args++ : 0); addrlo = *args++; addrhi = (TARGET_LONG_BITS == 64 ? *args++ : 0); oi = *args++; - opc = get_memop(oi); #ifdef CONFIG_SOFTMMU - mem_index = get_mmuidx(oi); - - avail = 0xf; - avail &= ~(1 << addrlo); - if (TARGET_LONG_BITS == 64) { - avail &= ~(1 << addrhi); - } - tcg_debug_assert(avail & 1); - t0 = TCG_REG_R0; - avail &= ~1; - tcg_debug_assert(avail != 0); - t1 = ctz32(avail); - - addend = tcg_out_tlb_read(s, addrlo, addrhi, opc, mem_index, 1, - t0, t1, TCG_REG_TMP); - - /* This a conditional BL only to load a pointer within this opcode into LR - for the slow path. We will not be using the value for a tail call. */ - label_ptr = s->code_ptr; - tcg_out_bl_noaddr(s, COND_NE); - - tcg_out_qemu_ld_index(s, opc, datalo, datahi, addrlo, addend); - - add_qemu_ldst_label(s, true, oi, datalo, datahi, addrlo, addrhi, - s->code_ptr, label_ptr); -#else /* !CONFIG_SOFTMMU */ + add_ldst_ool_label(s, true, is64, oi, R_ARM_PC24, 0); + tcg_out_bl_noaddr(s, COND_AL); +#else if (guest_base) { tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_TMP, guest_base); - tcg_out_qemu_ld_index(s, opc, datalo, datahi, addrlo, TCG_REG_TMP); + tcg_out_qemu_ld_index(s, get_memop(oi), datalo, datahi, + addrlo, TCG_REG_TMP); } else { - tcg_out_qemu_ld_direct(s, opc, datalo, datahi, addrlo); + tcg_out_qemu_ld_direct(s, get_memop(oi), datalo, datahi, addrlo); } #endif } @@ -1747,33 +1600,76 @@ static inline void tcg_out_qemu_st_direct(TCGContext *s, TCGMemOp opc, static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) { - TCGReg addrlo, datalo, datahi, addrhi __attribute__((unused)); + TCGReg addrlo __attribute__((unused)); + TCGReg addrhi __attribute__((unused)); + TCGReg datalo __attribute__((unused)); + TCGReg datahi __attribute__((unused)); TCGMemOpIdx oi; - TCGMemOp opc; -#ifdef CONFIG_SOFTMMU - int mem_index, avail; - TCGReg addend, t0, t1; - tcg_insn_unit *label_ptr; -#endif datalo = *args++; datahi = (is64 ? *args++ : 0); addrlo = *args++; addrhi = (TARGET_LONG_BITS == 64 ? *args++ : 0); oi = *args++; - opc = get_memop(oi); #ifdef CONFIG_SOFTMMU - mem_index = get_mmuidx(oi); + add_ldst_ool_label(s, false, is64, oi, R_ARM_PC24, 0); + tcg_out_bl_noaddr(s, COND_AL); +#else + if (guest_base) { + tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_TMP, guest_base); + tcg_out_qemu_st_index(s, COND_AL, get_memop(oi), datalo, + datahi, addrlo, TCG_REG_TMP); + } else { + tcg_out_qemu_st_direct(s, get_memop(oi), datalo, datahi, addrlo); + } +#endif +} +#ifdef CONFIG_SOFTMMU +static tcg_insn_unit *tcg_out_qemu_ldst_ool(TCGContext *s, bool is_ld, + bool is_64, TCGMemOpIdx oi) +{ + TCGReg addrlo, addrhi, datalo, datahi, addend, argreg, t0, t1; + TCGMemOp opc = get_memop(oi); + int mem_index = get_mmuidx(oi); + tcg_insn_unit *thunk = s->code_ptr; + tcg_insn_unit *label; + uintptr_t func; + int avail; + + /* Pick out where the arguments are located. A 64-bit address is + * aligned in the register pair R2:R3. Loads return into R0:R1. + * A 32-bit store with a 32-bit address has room at R2, but + * otherwise uses R4:R5. + */ + if (TARGET_LONG_BITS == 64) { + addrlo = TCG_REG_R2, addrhi = TCG_REG_R3; + } else { + addrlo = TCG_REG_R1, addrhi = -1; + } + if (is_ld) { + datalo = TCG_REG_R0; + } else if (TARGET_LONG_BITS == 64 || is_64) { + datalo = TCG_REG_R4; + } else { + datalo = TCG_REG_R2; + } + datahi = (is_64 ? datalo + 1 : -1); + + /* We need 3 call-clobbered temps. One of them is always R12, + * one of them is always R0. The third is somewhere in R[1-3]. + */ avail = 0xf; avail &= ~(1 << addrlo); - avail &= ~(1 << datalo); if (TARGET_LONG_BITS == 64) { avail &= ~(1 << addrhi); } - if (is64) { - avail &= ~(1 << datahi); + if (!is_ld) { + avail &= ~(1 << datalo); + if (is_64) { + avail &= ~(1 << datahi); + } } tcg_debug_assert(avail & 1); t0 = TCG_REG_R0; @@ -1781,27 +1677,63 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) tcg_debug_assert(avail != 0); t1 = ctz32(avail); - addend = tcg_out_tlb_read(s, addrlo, addrhi, opc, mem_index, 0, + addend = tcg_out_tlb_read(s, addrlo, addrhi, opc, mem_index, is_ld, t0, t1, TCG_REG_TMP); - tcg_out_qemu_st_index(s, COND_EQ, opc, datalo, datahi, addrlo, addend); + label = s->code_ptr; + tcg_out_b_noaddr(s, COND_NE); - /* The conditional call must come last, as we're going to return here. */ - label_ptr = s->code_ptr; - tcg_out_bl_noaddr(s, COND_NE); - - add_qemu_ldst_label(s, false, oi, datalo, datahi, addrlo, addrhi, - s->code_ptr, label_ptr); -#else /* !CONFIG_SOFTMMU */ - if (guest_base) { - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_TMP, guest_base); - tcg_out_qemu_st_index(s, COND_AL, opc, datalo, - datahi, addrlo, TCG_REG_TMP); + /* TCG Hit. */ + if (is_ld) { + tcg_out_qemu_ld_index(s, opc, datalo, datahi, addrlo, addend); } else { - tcg_out_qemu_st_direct(s, opc, datalo, datahi, addrlo); + tcg_out_qemu_st_index(s, COND_AL, opc, datalo, datahi, addrlo, addend); } -#endif + tcg_out_bx(s, COND_AL, TCG_REG_R14); + + /* TLB Miss. */ + reloc_pc24(label, s->code_ptr); + + tcg_out_arg_reg32(s, TCG_REG_R0, TCG_AREG0); + /* addrlo and addrhi are in place -- see above */ + argreg = addrlo + (TARGET_LONG_BITS / 32); + if (!is_ld) { + switch (opc & MO_SIZE) { + case MO_8: + argreg = tcg_out_arg_reg8(s, argreg, datalo); + break; + case MO_16: + argreg = tcg_out_arg_reg16(s, argreg, datalo); + break; + case MO_32: + argreg = tcg_out_arg_reg32(s, argreg, datalo); + break; + case MO_64: + argreg = tcg_out_arg_reg64(s, argreg, datalo, datahi); + break; + default: + g_assert_not_reached(); + } + } + argreg = tcg_out_arg_imm32(s, argreg, oi); + argreg = tcg_out_arg_reg32(s, argreg, TCG_REG_R14); + + /* Tail call to the helper. */ + if (is_ld) { + func = (uintptr_t)qemu_ld_helpers[opc & (MO_BSWAP | MO_SSIZE)]; + } else { + func = (uintptr_t)qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]; + } + if (use_armv7_instructions) { + tcg_out_movi32(s, COND_AL, TCG_REG_TMP, func); + tcg_out_bx(s, COND_AL, TCG_REG_TMP); + } else { + tcg_out_movi_pool(s, COND_AL, TCG_REG_PC, func); + } + + return thunk; } +#endif static tcg_insn_unit *tb_ret_addr;