From patchwork Mon Jul 19 21:58:42 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mikael Pettersson X-Patchwork-Id: 59229 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 5EFCFB6EFE for ; Tue, 20 Jul 2010 07:59:29 +1000 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S966982Ab0GSV6t (ORCPT ); Mon, 19 Jul 2010 17:58:49 -0400 Received: from fanny.its.uu.se ([130.238.4.241]:62427 "EHLO fanny.its.uu.se" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S966979Ab0GSV6r (ORCPT ); Mon, 19 Jul 2010 17:58:47 -0400 Received: by fanny.its.uu.se (Postfix, from userid 212) id 625436307; Mon, 19 Jul 2010 23:58:45 +0200 (MSZ) Received: from fanny.its.uu.se(127.0.0.1) by fanny.its.uu.se via virus-scan id s24834; Mon, 19 Jul 10 23:58:44 +0200 Received: from pilspetsen.it.uu.se (pilspetsen.it.uu.se [130.238.18.39]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by fanny.its.uu.se (Postfix) with ESMTP id D7A0E61AE; Mon, 19 Jul 2010 23:58:43 +0200 (MSZ) Received: (from mikpe@localhost) by pilspetsen.it.uu.se (8.13.8+Sun/8.13.8) id o6JLwhkM008427; Mon, 19 Jul 2010 23:58:43 +0200 (MEST) X-Authentication-Warning: pilspetsen.it.uu.se: mikpe set sender to mikpe@it.uu.se using -f MIME-Version: 1.0 Message-ID: <19524.51858.992299.119315@pilspetsen.it.uu.se> Date: Mon, 19 Jul 2010 23:58:42 +0200 From: Mikael Pettersson To: linux-kernel@vger.kernel.org Subject: [PATCH] math-emu: correct test for downshifting fraction in _FP_FROM_INT() Cc: sparclinux@vger.kernel.org, linux-s390@vger.kernel.org, linux-alpha@vger.kernel.org, linux-sh@vger.kernel.org, linuxppc-dev@ozlabs.org X-Mailer: VM 7.17 under Emacs 20.7.1 Sender: sparclinux-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org The kernel's math-emu code contains a macro _FP_FROM_INT() which is used to convert an integer to a raw normalized floating-point value. It does this basically in three steps: 1. Compute the exponent from the number of leading zero bits. 2. Downshift large fractions to put the MSB in the right position for normalized fractions. 3. Upshift small fractions to put the MSB in the right position. There is an boundary error in step 2, causing a fraction with its MSB exactly one bit above the normalized MSB position to not be downshifted. This results in a non-normalized raw float, which when packed becomes a massively inaccurate representation for that input. The impact of this depends on a number of arch-specific factors, but it is known to have broken emulation of FXTOD instructions on UltraSPARC III, which was originally reported as GCC bug 44631 . Any arch which uses math-emu to emulate conversions from integers to same-size floats may be affected. The fix is simple: the exponent comparison used to determine if the fraction should be downshifted must be "<=" not "<". I'm sending a kernel module to test this as a reply to this message. There are also SPARC user-space test cases in the GCC bug entry. Signed-off-by: Mikael Pettersson Acked-by: David S. Miller --- include/math-emu/op-common.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- To unsubscribe from this list: send the line "unsubscribe sparclinux" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff -rupN linux-2.6.35-rc5/include/math-emu/op-common.h linux-2.6.35-rc5.mathemu-FP_FROM_INT-fraction-downshift-condition/include/math-emu/op-common.h --- linux-2.6.35-rc5/include/math-emu/op-common.h 2010-05-17 19:51:32.000000000 +0200 +++ linux-2.6.35-rc5.mathemu-FP_FROM_INT-fraction-downshift-condition/include/math-emu/op-common.h 2010-07-18 22:33:46.000000000 +0200 @@ -799,7 +799,7 @@ do { \ X##_e -= (_FP_W_TYPE_SIZE - rsize); \ X##_e = rsize - X##_e - 1; \ \ - if (_FP_FRACBITS_##fs < rsize && _FP_WFRACBITS_##fs < X##_e) \ + if (_FP_FRACBITS_##fs < rsize && _FP_WFRACBITS_##fs <= X##_e) \ __FP_FRAC_SRS_1(ur_, (X##_e - _FP_WFRACBITS_##fs + 1), rsize);\ _FP_FRAC_DISASSEMBLE_##wc(X, ur_, rsize); \ if ((_FP_WFRACBITS_##fs - X##_e - 1) > 0) \