From patchwork Fri Sep 16 07:58:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Sandiford X-Patchwork-Id: 1678559 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=Vehsx4n7; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4MTRL76Tspz1ynm for ; Fri, 16 Sep 2022 17:59:25 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 699B1385380D for ; Fri, 16 Sep 2022 07:59:18 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 699B1385380D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1663315158; bh=+esIB+ySKh00E6GY9ROKbvqRz45CdW+dhk8G+aGVCo0=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=Vehsx4n7y4YvlZizkQwjNiKD7sfOU34r3xDjoSbuF6MzjxLxPfd7EkO+iNKc8iXmp Bs7or6U7Vu0KpnQI4pl+/4b4CmtiGSg56aX23+cT4YeOcTpIIJe9JzKx/nsaAA3BVH GXEAlIpMKvTDh+B2h1XtmZBCIOirTZ5beYKEGFIo= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 190333853809 for ; Fri, 16 Sep 2022 07:58:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 190333853809 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 233191D34; Fri, 16 Sep 2022 00:59:04 -0700 (PDT) Received: from localhost (e121540-lin.manchester.arm.com [10.32.98.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 34CE33F73D; Fri, 16 Sep 2022 00:58:57 -0700 (PDT) To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, rguenther@suse.de, richard.sandiford@arm.com Subject: [PATCH] vect: Fix missed gather load opportunity Date: Fri, 16 Sep 2022 08:58:55 +0100 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 X-Spam-Status: No, score=-48.0 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Richard Sandiford via Gcc-patches From: Richard Sandiford Reply-To: Richard Sandiford Cc: rguenther@suse.de Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" While writing a testcase for PR106794, I noticed that we failed to vectorise the testcase in the patch for SVE. The code that recognises gather loads tries to optimise the point at which the offset is calculated, to avoid unnecessary extensions or truncations: /* Don't include the conversion if the target is happy with the current offset type. */ But breaking only makes sense if we're at an SSA_NAME (which could then be vectorised). We shouldn't break on a conversion embedded in a generic expression. Tested on aarch64-linux-gnu & x86_64-linux-gnu. OK to install? Richard gcc/ * tree-vect-data-refs.cc (vect_check_gather_scatter): Restrict early-out optimisation to SSA_NAMEs. gcc/testsuite/ * gcc.dg/vect/vect-gather-5.c: New test. --- gcc/testsuite/gcc.dg/vect/vect-gather-5.c | 42 +++++++++++++++++++++++ gcc/tree-vect-data-refs.cc | 1 + 2 files changed, 43 insertions(+) create mode 100644 gcc/testsuite/gcc.dg/vect/vect-gather-5.c diff --git a/gcc/testsuite/gcc.dg/vect/vect-gather-5.c b/gcc/testsuite/gcc.dg/vect/vect-gather-5.c new file mode 100644 index 00000000000..8b5074bba88 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-gather-5.c @@ -0,0 +1,42 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-O3" } */ + +#ifdef __aarch64__ +#pragma GCC target "+sve" +#endif + +long a[100], b[100], c[100]; + +void g1 () +{ + for (int i = 0; i < 100; i += 2) + { + c[i] += a[b[i]] + 1; + c[i + 1] += a[b[i + 1]] + 2; + } +} + +long g2 () +{ + long res = 0; + for (int i = 0; i < 100; i += 2) + { + res += a[b[i + 1]]; + res += a[b[i]]; + } + return res; +} + +long g3 () +{ + long res = 0; + for (int i = 0; i < 100; i += 2) + { + res += a[b[i]]; + res += a[b[i + 1]]; + } + return res; +} + +/* { dg-final { scan-tree-dump-times {add new stmt[^\n]*GATHER_LOAD} 3 "vect" { target aarch64*-*-* } } } */ +/* { dg-final { scan-tree-dump-not {add new stmt[^\n]*VEC_PERM_EXPR} "vect" { target aarch64*-*-* } } } */ diff --git a/gcc/tree-vect-data-refs.cc b/gcc/tree-vect-data-refs.cc index b279a82551e..e03b50498d1 100644 --- a/gcc/tree-vect-data-refs.cc +++ b/gcc/tree-vect-data-refs.cc @@ -4151,6 +4151,7 @@ vect_check_gather_scatter (stmt_vec_info stmt_info, loop_vec_info loop_vinfo, /* Don't include the conversion if the target is happy with the current offset type. */ if (use_ifn_p + && TREE_CODE (off) == SSA_NAME && !POINTER_TYPE_P (TREE_TYPE (off)) && vect_gather_scatter_fn_p (loop_vinfo, DR_IS_READ (dr), masked_p, vectype, memory_type,