From patchwork Tue Mar 27 05:33:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emilio Cota X-Patchwork-Id: 891393 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=braap.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=braap.org header.i=@braap.org header.b="Vt197BSa"; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.b="g2SrtO0g"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 409Kfl1PDzz9s0R for ; Tue, 27 Mar 2018 16:42:55 +1100 (AEDT) Received: from localhost ([::1]:60528 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f0hNk-0006KS-Lj for incoming@patchwork.ozlabs.org; Tue, 27 Mar 2018 01:42:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35794) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f0hFI-0007PB-7h for qemu-devel@nongnu.org; Tue, 27 Mar 2018 01:34:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f0hFE-0005Nn-DN for qemu-devel@nongnu.org; Tue, 27 Mar 2018 01:34:08 -0400 Received: from out5-smtp.messagingengine.com ([66.111.4.29]:60723) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f0hFE-0005Mg-8v for qemu-devel@nongnu.org; Tue, 27 Mar 2018 01:34:04 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id DB25E21666; Tue, 27 Mar 2018 01:34:03 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Tue, 27 Mar 2018 01:34:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h=cc :date:from:in-reply-to:message-id:references:subject:to :x-me-sender:x-me-sender:x-sasl-enc; s=mesmtp; bh=yKm7gcTeKOWaNQ nEFgrz+KWUxEeWJ5SGegDIiMEMbYE=; b=Vt197BSaXdIJC4BKedPQlVLfXjwdR5 TeN4BlRnSPt0IXhzg5DFx8q7CtGzb+YwxK3PhkCSwTPNhgBMK5FPcejiic+LDnIp wnV1COE75oasXG5CkFyiQnuRhpTsFB6QkL53VOKJ/9V5G1vboDRddN0jof38zwZ8 K5ZJ09EW6x1Bg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:in-reply-to:message-id :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=yKm7gcTeKOWaNQnEFgrz+KWUxEeWJ5SGegDIiMEMbYE=; b=g2SrtO0g jjcTsKDtNFwKL5CZ6YHn9wmyMhntx9zb/opUaynRll6x1M51pr2D4m1BU315GtLt SXl4rqvYmcCubb0MutX8kAuFdMGbC+5Wn/tQ4sg9SOJ3mNwpd8k+NYDxQvqHr5Iw IJF6aJ2Mcvnta7HtlIryVjSBpUB2+llCzHHCTH6XXlHu1mU/IG++oL4DwFky2tF4 J6hV8CLyDHewnZJRLrZZb7O2GlgOQovYQ0Xa04aa1fQnKZ+neFCfMUROaTGKQGNV Ib9fnjWedpw3+QWATE408Hu42xSH0tiK/KQF59CiB+z8bovyefOaETHrCVu6B0vo m8FtUu4UtugQkw== X-ME-Sender: Received: from localhost (flamenco.cs.columbia.edu [128.59.20.216]) by mail.messagingengine.com (Postfix) with ESMTPA id 8CC7F1025C; Tue, 27 Mar 2018 01:34:03 -0400 (EDT) From: "Emilio G. Cota" To: qemu-devel@nongnu.org Date: Tue, 27 Mar 2018 01:33:57 -0400 Message-Id: <1522128840-498-12-git-send-email-cota@braap.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1522128840-498-1-git-send-email-cota@braap.org> References: <1522128840-498-1-git-send-email-cota@braap.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.111.4.29 Subject: [Qemu-devel] [PATCH v2 11/14] hardfloat: support float32/64 fused multiply-add X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , Mark Cave-Ayland , Richard Henderson , Laurent Vivier , Paolo Bonzini , =?utf-8?q?Alex_Benn=C3=A9e?= , Aurelien Jarno Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Performance results for fp-bench run under aarch64-linux-user on an aarch64 host: - before: fma-single: 53.05 MFlops fma-double: 51.89 MFlops - after: fma-single: 110.44 MFlops fma-double: 101.78 MFlops - w/ both using float32/64_is_normal etc.: fma-single: 110.57 MFlops fma-double: 93.93 MFlops - w/ both using fpclassify etc.: fma-single: 102.86 MFlops fma-double: 101.71 MFlops Signed-off-by: Emilio G. Cota --- fpu/softfloat.c | 138 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 134 insertions(+), 4 deletions(-) diff --git a/fpu/softfloat.c b/fpu/softfloat.c index f414b41..2dedb13 100644 --- a/fpu/softfloat.c +++ b/fpu/softfloat.c @@ -1348,8 +1348,9 @@ float16 __attribute__((flatten)) float16_muladd(float16 a, float16 b, float16 c, return float16_round_pack_canonical(pr, status); } -float32 __attribute__((flatten)) float32_muladd(float32 a, float32 b, float32 c, - int flags, float_status *status) +static float32 __attribute__((flatten, noinline)) +soft_float32_muladd(float32 a, float32 b, float32 c, int flags, + float_status *status) { FloatParts pa = float32_unpack_canonical(a, status); FloatParts pb = float32_unpack_canonical(b, status); @@ -1359,8 +1360,9 @@ float32 __attribute__((flatten)) float32_muladd(float32 a, float32 b, float32 c, return float32_round_pack_canonical(pr, status); } -float64 __attribute__((flatten)) float64_muladd(float64 a, float64 b, float64 c, - int flags, float_status *status) +static float64 __attribute__((flatten, noinline)) +soft_float64_muladd(float64 a, float64 b, float64 c, int flags, + float_status *status) { FloatParts pa = float64_unpack_canonical(a, status); FloatParts pb = float64_unpack_canonical(b, status); @@ -1371,6 +1373,134 @@ float64 __attribute__((flatten)) float64_muladd(float64 a, float64 b, float64 c, } /* + * When (a || b) == 0, there's no need to check for under/over flow, + * since we know the addend is (normal || 0) and the product is 0. + */ +#define GEN_FPU_FMA(name, soft_t, host_t, host_fma_f, host_abs_f, min_normal) \ + soft_t name(soft_t a, soft_t b, soft_t c, int flags, float_status *s) \ + { \ + soft_t ## _input_flush3(&a, &b, &c, s); \ + if (likely((soft_t ## _is_normal(a) || soft_t ## _is_zero(a)) && \ + (soft_t ## _is_normal(b) || soft_t ## _is_zero(b)) && \ + (soft_t ## _is_normal(c) || soft_t ## _is_zero(c)) && \ + !(flags & float_muladd_halve_result) && \ + s->float_exception_flags & float_flag_inexact && \ + s->float_rounding_mode == float_round_nearest_even)) { \ + if (soft_t ## _is_zero(a) || soft_t ## _is_zero(b)) { \ + soft_t p, r; \ + host_t hp, hc, hr; \ + bool prod_sign; \ + \ + prod_sign = soft_t ## _is_neg(a) ^ soft_t ## _is_neg(b); \ + prod_sign ^= !!(flags & float_muladd_negate_product); \ + p = soft_t ## _set_sign(soft_t ## _zero, prod_sign); \ + \ + if (flags & float_muladd_negate_c) { \ + c = soft_t ## _chs(c); \ + } \ + \ + hp = soft_t ## _to_ ## host_t(p); \ + hc = soft_t ## _to_ ## host_t(c); \ + hr = hp + hc; \ + r = host_t ## _to_ ## soft_t(hr); \ + return flags & float_muladd_negate_result ? \ + soft_t ## _chs(r) : r; \ + } else { \ + host_t ha, hb, hc, hr; \ + soft_t r; \ + soft_t sa = flags & float_muladd_negate_product ? \ + soft_t ## _chs(a) : a; \ + soft_t sc = flags & float_muladd_negate_c ? \ + soft_t ## _chs(c) : c; \ + \ + ha = soft_t ## _to_ ## host_t(sa); \ + hb = soft_t ## _to_ ## host_t(b); \ + hc = soft_t ## _to_ ## host_t(sc); \ + hr = host_fma_f(ha, hb, hc); \ + r = host_t ## _to_ ## soft_t(hr); \ + \ + if (unlikely(soft_t ## _is_infinity(r))) { \ + s->float_exception_flags |= float_flag_overflow; \ + } else if (unlikely(host_abs_f(hr) <= min_normal)) { \ + goto soft; \ + } \ + return flags & float_muladd_negate_result ? \ + soft_t ## _chs(r) : r; \ + } \ + } \ + soft: \ + return soft_ ## soft_t ## _muladd(a, b, c, flags, s); \ + } + +GEN_FPU_FMA(float32_muladd, float32, float, fmaf, fabsf, FLT_MIN) +#undef GEN_FPU_FMA + +#define GEN_FPU_FMA(name, soft_t, host_t, host_fma_f, host_abs_f, min_normal) \ + soft_t name(soft_t a, soft_t b, soft_t c, int flags, float_status *s) \ + { \ + host_t ha, hb, hc; \ + \ + soft_t ## _input_flush3(&a, &b, &c, s); \ + ha = soft_t ## _to_ ## host_t(a); \ + hb = soft_t ## _to_ ## host_t(b); \ + hc = soft_t ## _to_ ## host_t(c); \ + if (likely((fpclassify(ha) == FP_NORMAL || \ + fpclassify(ha) == FP_ZERO) && \ + (fpclassify(hb) == FP_NORMAL || \ + fpclassify(hb) == FP_ZERO) && \ + (fpclassify(hc) == FP_NORMAL || \ + fpclassify(hc) == FP_ZERO) && \ + !(flags & float_muladd_halve_result) && \ + s->float_exception_flags & float_flag_inexact && \ + s->float_rounding_mode == float_round_nearest_even)) { \ + if (soft_t ## _is_zero(a) || soft_t ## _is_zero(b)) { \ + soft_t p, r; \ + host_t hp, hc, hr; \ + bool prod_sign; \ + \ + prod_sign = soft_t ## _is_neg(a) ^ soft_t ## _is_neg(b); \ + prod_sign ^= !!(flags & float_muladd_negate_product); \ + p = soft_t ## _set_sign(soft_t ## _zero, prod_sign); \ + \ + if (flags & float_muladd_negate_c) { \ + c = soft_t ## _chs(c); \ + } \ + \ + hp = soft_t ## _to_ ## host_t(p); \ + hc = soft_t ## _to_ ## host_t(c); \ + hr = hp + hc; \ + r = host_t ## _to_ ## soft_t(hr); \ + return flags & float_muladd_negate_result ? \ + soft_t ## _chs(r) : r; \ + } else { \ + host_t hr; \ + \ + if (flags & float_muladd_negate_product) { \ + ha = -ha; \ + } \ + if (flags & float_muladd_negate_c) { \ + hc = -hc; \ + } \ + hr = host_fma_f(ha, hb, hc); \ + if (unlikely(isinf(hr))) { \ + s->float_exception_flags |= float_flag_overflow; \ + } else if (unlikely(host_abs_f(hr) <= min_normal)) { \ + goto soft; \ + } \ + if (flags & float_muladd_negate_result) { \ + hr = -hr; \ + } \ + return host_t ## _to_ ## soft_t(hr); \ + } \ + } \ + soft: \ + return soft_ ## soft_t ## _muladd(a, b, c, flags, s); \ + } + +GEN_FPU_FMA(float64_muladd, float64, double, fma, fabs, DBL_MIN) +#undef GEN_FPU_FMA + +/* * Returns the result of dividing the floating-point value `a' by the * corresponding value `b'. The operation is performed according to * the IEC/IEEE Standard for Binary Floating-Point Arithmetic.