From patchwork Wed Jul 3 05:20:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Jelinek X-Patchwork-Id: 1126623 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=gcc.gnu.org (client-ip=209.132.180.131; helo=sourceware.org; envelope-from=gcc-patches-return-504245-incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45dqFQ0wB9z9s3Z for ; Wed, 3 Jul 2019 15:20:40 +1000 (AEST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:date :from:to:cc:subject:message-id:reply-to:mime-version :content-type; q=dns; s=default; b=OcZmQDubljtitzZJJongd6h85Wq8f 304CrpYmAFvX3LGQIDC8Ec/R2Wey2HmHe9VVpr8Q000WZBwVmsYOLOAP1KFKTIai OjbNYp57RsnZnkfAGsr6C3YqOP8QA7gi4LWXaUaO0gdN0lihyRVf77hscoH2ZP7c vtN+xA0ye1TpFI= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:date :from:to:cc:subject:message-id:reply-to:mime-version :content-type; s=default; bh=D3GZbZkpYlOhdv3eeFrVbb181AM=; b=d3M aB29hEdJljzxrO/oy3u+s0vknLYLF2KhxlCPGEdmhyA5xkNmGEYW4+QrO2yX6a32 1XTtobUoe/w8LpQmkh583fJu5dFVLEnj0nJ4xa01xIOdCyl6JzGHmRv+rv7T9ovZ /kMwUa/+RaWySPyEsdd2FhSbcj9HpPZPP7BqytKA= Received: (qmail 108386 invoked by alias); 3 Jul 2019 05:20:33 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 108377 invoked by uid 89); 3 Jul 2019 05:20:33 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-7.4 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_2, GIT_PATCH_3, SPF_HELO_PASS autolearn=ham version=3.3.1 spammy=ok, factors X-HELO: mx1.redhat.com Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 03 Jul 2019 05:20:31 +0000 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3889737EEF; Wed, 3 Jul 2019 05:20:30 +0000 (UTC) Received: from tucnak.zalov.cz (ovpn-116-51.ams2.redhat.com [10.36.116.51]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C18077BE6C; Wed, 3 Jul 2019 05:20:29 +0000 (UTC) Received: from tucnak.zalov.cz (localhost [127.0.0.1]) by tucnak.zalov.cz (8.15.2/8.15.2) with ESMTP id x635KR59029239; Wed, 3 Jul 2019 07:20:27 +0200 Received: (from jakub@localhost) by tucnak.zalov.cz (8.15.2/8.15.2/Submit) id x635KNjV029238; Wed, 3 Jul 2019 07:20:23 +0200 Date: Wed, 3 Jul 2019 07:20:23 +0200 From: Jakub Jelinek To: Richard Biener Cc: gcc-patches@gcc.gnu.org Subject: [PATCH] Make scatter/gather vectorization failures non-fatal (PR tree-optimization/91033) Message-ID: <20190703052023.GT815@tucnak> Reply-To: Jakub Jelinek MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.11.3 (2019-02-01) X-IsSubscribed: yes Hi! As mentioned in the PR, I'm afraid we can't easily move the scatter/gather verification from vect_analyze_data_refs to vectorizable_{load,store}, because we need to process_use in between on the gsinfo.offset to determine what statements need to be vectorized and that can be only determined with the successful scatter/gather detection. The following patch just makes sure that we don't mark failures to handle scatter/gather as fatal, which means if there are multiple vectorization factors, if there is a scatter/gather failure (but some scatter/gather is supported, no scatter/gather support altogether is fatal), we keep trying finding another suitable vectorization factors. This matters e.g. on AVX512F without AVX512VL, where we can scatter only when using 512-bit vectors but not other sizes; if we try say 256-bit vectors first without simdlen, it would mean we don't vectorize even if we could (with 512-bit vectors), if we try 512-bit vectors first with simdlen, vectorization succeeds for that, but gets fatal when retrying with 256-bit or 128-bit vectors to see if that doesn't match the simdlen and we have asserts to make sure that the fatal failures are either for all vector sizes or none. Bootstrapped/regtested on x86_64-linux and i686-linux, ok for trunk? 2019-07-03 Jakub Jelinek PR tree-optimization/91033 * tree-vectorizer.h (vect_mark_stmts_to_be_vectorized, vect_analyze_data_refs): Add bool * arguments. * tree-vect-data-refs.c (vect_analyze_data_refs): Add fatal argument, if failure is due to scatter/gather, set *fatal to false if non-NULL. * tree-vect-stmts.c (vect_mark_stmts_to_be_vectorized): Likewise. * tree-vect-loop.c (vect_analyze_loop_2): Adjust vect_mark_stmts_to_be_vectorized and vect_analyze_data_refs callers. * tree-vect-slp.c (vect_slp_analyze_bb_1): Adjust vect_analyze_data_refs caller. * gcc.target/i386/pr91033.c: New test. Jakub --- gcc/tree-vectorizer.h.jj 2019-06-21 08:47:04.169673346 +0200 +++ gcc/tree-vectorizer.h 2019-07-02 18:35:45.759141160 +0200 @@ -1501,7 +1501,7 @@ extern unsigned record_stmt_cost (stmt_v extern stmt_vec_info vect_finish_replace_stmt (stmt_vec_info, gimple *); extern stmt_vec_info vect_finish_stmt_generation (stmt_vec_info, gimple *, gimple_stmt_iterator *); -extern opt_result vect_mark_stmts_to_be_vectorized (loop_vec_info); +extern opt_result vect_mark_stmts_to_be_vectorized (loop_vec_info, bool *); extern tree vect_get_store_rhs (stmt_vec_info); extern tree vect_get_vec_def_for_operand_1 (stmt_vec_info, enum vect_def_type); extern tree vect_get_vec_def_for_operand (tree, stmt_vec_info, tree = NULL); @@ -1559,7 +1559,7 @@ extern bool vect_check_gather_scatter (s gather_scatter_info *); extern opt_result vect_find_stmt_data_reference (loop_p, gimple *, vec *); -extern opt_result vect_analyze_data_refs (vec_info *, poly_uint64 *); +extern opt_result vect_analyze_data_refs (vec_info *, poly_uint64 *, bool *); extern void vect_record_base_alignments (vec_info *); extern tree vect_create_data_ref_ptr (stmt_vec_info, tree, struct loop *, tree, tree *, gimple_stmt_iterator *, --- gcc/tree-vect-data-refs.c.jj 2019-06-21 23:37:57.002962811 +0200 +++ gcc/tree-vect-data-refs.c 2019-07-02 18:34:03.225722070 +0200 @@ -4160,7 +4160,7 @@ vect_find_stmt_data_reference (loop_p lo */ opt_result -vect_analyze_data_refs (vec_info *vinfo, poly_uint64 *min_vf) +vect_analyze_data_refs (vec_info *vinfo, poly_uint64 *min_vf, bool *fatal) { struct loop *loop = NULL; unsigned int i; @@ -4386,12 +4386,16 @@ vect_analyze_data_refs (vec_info *vinfo, as_a (vinfo), &gs_info) || !get_vectype_for_scalar_type (TREE_TYPE (gs_info.offset))) - return opt_result::failure_at - (stmt_info->stmt, - (gatherscatter == GATHER) ? - "not vectorized: not suitable for gather load %G" : - "not vectorized: not suitable for scatter store %G", - stmt_info->stmt); + { + if (fatal) + *fatal = false; + return opt_result::failure_at + (stmt_info->stmt, + (gatherscatter == GATHER) + ? "not vectorized: not suitable for gather load %G" + : "not vectorized: not suitable for scatter store %G", + stmt_info->stmt); + } STMT_VINFO_GATHER_SCATTER_P (stmt_info) = gatherscatter; } } --- gcc/tree-vect-stmts.c.jj 2019-06-27 23:22:40.801471237 +0200 +++ gcc/tree-vect-stmts.c 2019-07-02 18:35:35.244303277 +0200 @@ -608,7 +608,7 @@ process_use (stmt_vec_info stmt_vinfo, t This pass detects such stmts. */ opt_result -vect_mark_stmts_to_be_vectorized (loop_vec_info loop_vinfo) +vect_mark_stmts_to_be_vectorized (loop_vec_info loop_vinfo, bool *fatal) { struct loop *loop = LOOP_VINFO_LOOP (loop_vinfo); basic_block *bbs = LOOP_VINFO_BBS (loop_vinfo); @@ -778,7 +778,11 @@ vect_mark_stmts_to_be_vectorized (loop_v = process_use (stmt_vinfo, gs_info.offset, loop_vinfo, relevant, &worklist, true); if (!res) - return res; + { + if (fatal) + *fatal = false; + return res; + } } } /* while worklist */ --- gcc/tree-vect-loop.c.jj 2019-06-27 23:22:14.380884386 +0200 +++ gcc/tree-vect-loop.c 2019-07-02 18:36:41.581280458 +0200 @@ -1901,7 +1901,7 @@ vect_analyze_loop_2 (loop_vec_info loop_ /* Analyze the data references and also adjust the minimal vectorization factor according to the loads and stores. */ - ok = vect_analyze_data_refs (loop_vinfo, &min_vf); + ok = vect_analyze_data_refs (loop_vinfo, &min_vf, &fatal); if (!ok) { if (dump_enabled_p ()) @@ -1932,7 +1932,7 @@ vect_analyze_loop_2 (loop_vec_info loop_ /* Data-flow analysis to detect stmts that do not need to be vectorized. */ - ok = vect_mark_stmts_to_be_vectorized (loop_vinfo); + ok = vect_mark_stmts_to_be_vectorized (loop_vinfo, &fatal); if (!ok) { if (dump_enabled_p ()) --- gcc/tree-vect-slp.c.jj 2019-06-05 09:34:02.393372307 +0200 +++ gcc/tree-vect-slp.c 2019-07-02 18:37:15.150762867 +0200 @@ -2861,7 +2861,7 @@ vect_slp_analyze_bb_1 (gimple_stmt_itera /* Analyze the data references. */ - if (!vect_analyze_data_refs (bb_vinfo, &min_vf)) + if (!vect_analyze_data_refs (bb_vinfo, &min_vf, NULL)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, --- gcc/testsuite/gcc.target/i386/pr91033.c.jj 2019-07-02 19:49:02.300365110 +0200 +++ gcc/testsuite/gcc.target/i386/pr91033.c 2019-07-02 19:48:56.219458810 +0200 @@ -0,0 +1,15 @@ +/* PR tree-optimization/91033 */ +/* { dg-do compile { target pthread } } */ +/* { dg-options "-march=knl -O2 -fopenmp-simd -ftree-parallelize-loops=2" } */ + +#define N 1024 +int a[N]; + +void +foo (void) +{ + int i; + #pragma omp simd simdlen (4) + for (i = 0; i < N; ++i) + a[i] = a[i] + 1; +}