Fix two ubsan failures (PR85164)

Two fixes for UB when handling very large offsets.  The calculation in
force_int_to_mode would have been correct if signed integers used modulo
arithmetic, so just switch to unsigned types.  The calculation in
rtx_addr_can_trap_p_1 didn't handle overflow properly, so switch to
known_subrange_p instead (which is supposed to handle all cases).

Tested with bootstrap-ubsan on aarch64-linux-gnu and x86_64-linux-gnu.
OK to install?

Richard


2019-04-18  Richard Sandiford  <richard.sandiford@arm.com>

gcc/
	PR middle-end/85164
	* combine.c (force_int_to_mode): Cast the argument rather than
	the result of known_alignment.
	* rtlanal.c (rtx_addr_can_trap_p_1): Use known_subrange_p.

gcc/testsuite/
	PR middle-end/85164
	* gcc.dg/pr85164-1.c, gcc.dg/pr85164-2.c: New tests.

On Thu, Apr 18, 2019 at 11:13 AM Richard Sandiford
<richard.sandiford@arm.com> wrote:
>
> Two fixes for UB when handling very large offsets.  The calculation in
> force_int_to_mode would have been correct if signed integers used modulo
> arithmetic, so just switch to unsigned types.  The calculation in
> rtx_addr_can_trap_p_1 didn't handle overflow properly, so switch to
> known_subrange_p instead (which is supposed to handle all cases).
>
> Tested with bootstrap-ubsan on aarch64-linux-gnu and x86_64-linux-gnu.
> OK to install?

OK.

Richard.

> Richard
>
>
> 2019-04-18  Richard Sandiford  <richard.sandiford@arm.com>
>
> gcc/
>         PR middle-end/85164
>         * combine.c (force_int_to_mode): Cast the argument rather than
>         the result of known_alignment.
>         * rtlanal.c (rtx_addr_can_trap_p_1): Use known_subrange_p.
>
> gcc/testsuite/
>         PR middle-end/85164
>         * gcc.dg/pr85164-1.c, gcc.dg/pr85164-2.c: New tests.
>
> Index: gcc/combine.c
> ===================================================================
> --- gcc/combine.c       2019-03-08 18:15:36.704740334 +0000
> +++ gcc/combine.c       2019-04-18 10:11:01.984734102 +0100
> @@ -8922,7 +8922,7 @@ force_int_to_mode (rtx x, scalar_int_mod
>        /* If X is (minus C Y) where C's least set bit is larger than any bit
>          in the mask, then we may replace with (neg Y).  */
>        if (poly_int_rtx_p (XEXP (x, 0), &const_op0)
> -         && (unsigned HOST_WIDE_INT) known_alignment (const_op0) > mask)
> +         && known_alignment (poly_uint64 (const_op0)) > mask)
>         {
>           x = simplify_gen_unary (NEG, xmode, XEXP (x, 1), xmode);
>           return force_to_mode (x, mode, mask, next_select);
> Index: gcc/rtlanal.c
> ===================================================================
> --- gcc/rtlanal.c       2019-03-08 18:14:26.721006369 +0000
> +++ gcc/rtlanal.c       2019-04-18 10:11:01.984734102 +0100
> @@ -521,7 +521,7 @@ rtx_addr_can_trap_p_1 (const_rtx x, poly
>
>           return (!known_size_p (decl_size) || known_eq (decl_size, 0)
>                   ? maybe_ne (offset, 0)
> -                 : maybe_gt (offset + size, decl_size));
> +                 : !known_subrange_p (offset, size, 0, decl_size));
>          }
>
>        return 0;
> Index: gcc/testsuite/gcc.dg/pr85164-1.c
> ===================================================================
> --- /dev/null   2019-03-08 11:40:14.606883727 +0000
> +++ gcc/testsuite/gcc.dg/pr85164-1.c    2019-04-18 10:11:01.984734102 +0100
> @@ -0,0 +1,7 @@
> +/* { dg-options "-O2 -w" } */
> +a[];
> +b;
> +c() {
> +  unsigned long d;
> +  b = a[d - 1 >> 3];
> +}
> Index: gcc/testsuite/gcc.dg/pr85164-2.c
> ===================================================================
> --- /dev/null   2019-03-08 11:40:14.606883727 +0000
> +++ gcc/testsuite/gcc.dg/pr85164-2.c    2019-04-18 10:11:01.984734102 +0100
> @@ -0,0 +1,4 @@
> +/* { dg-options "-O2 -w" } */
> +int a;
> +long b;
> +void c() { b = -9223372036854775807L - 1 - a; }