From patchwork Wed Mar 6 09:26:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 1908654 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=LwhWRDpa; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=p5YQaEtk; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=ffEttN83; dkim=neutral header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=TIA61+Rz; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TqRsl4x4zz1yX4 for ; Wed, 6 Mar 2024 20:27:22 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 7B6B83858010 for ; Wed, 6 Mar 2024 09:27:19 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by sourceware.org (Postfix) with ESMTPS id 37789385843A for ; Wed, 6 Mar 2024 09:26:52 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 37789385843A Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 37789385843A Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709717217; cv=none; b=mBZ3cC5ovptq3P+6UwptdLlq65JiBkFYZzMgvd4gIlPjxM24WfGM2Z340Vd9nM3kAH1AhmWLaUh30GueRAy7a99PWgvCJ4SVWAx55Fku4/NtJB92N6bIMKmtBDhqhZ1TMIsmKLnP2tosI4pvynCKUWNH/WgK2iOz5aQKPnbMXbc= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709717217; c=relaxed/simple; bh=M0WU3FF5gomDUhow0C4V+gpO4JY02u3Hy40HGry/UnI=; h=DKIM-Signature:DKIM-Signature:DKIM-Signature:DKIM-Signature:Date: From:To:Subject:MIME-Version; b=uV+x4M0qJljs/yW64wa8OEQr/vLGqIGLBwEX/j8a65krS1mSPBwJxKocmPQRHwJU4yAQKGgrwniJwqIWWCUsowGx0GfX5Zfur2oFmNKXJmbK6RZ2WZuj+MMeu6lO+Ft1oU21IycSkLQJeWSv8MdJa0465miOxYNv9eFhLU1reHQ= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from [10.168.4.150] (unknown [10.168.4.150]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id DDA0067644 for ; Wed, 6 Mar 2024 09:26:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1709717211; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=yy9Y40uQdHCgiVgpTswzucHJp+U+vA9ZyfUIrmdXeNQ=; b=LwhWRDpaO3NVpWV2zZDGEX3Vut7r4wkG8jAW2y7VY2G7hMjOEp0BvJNMbIkW8++l3LdMB3 mI9buR6Vw1UumMVSP8ZlRWg8ZAdkjV/uqcni+zKS+pSbO0OM/YrTzOVmd6k55qYRRKA/ls S6GcJYQ88Q/SElAlIsmQVJHs8WhPFuM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1709717211; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=yy9Y40uQdHCgiVgpTswzucHJp+U+vA9ZyfUIrmdXeNQ=; b=p5YQaEtkRq7efuE5uGskX+gWuvjADt4ucZXdFFapxfvSVNoUaGDJpx1cubD4UDFky4UmW4 kY9Rd8RhjmC+lgBw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1709717210; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=yy9Y40uQdHCgiVgpTswzucHJp+U+vA9ZyfUIrmdXeNQ=; b=ffEttN833qqhW5ER7l71sFHzsJcPrs0qn36bb5Pjbdu0j9+nY08gTcfrVZ3cQ8lTQWzAX1 5nh0KhgMXaGe/TJhBYwqNg46xcofAqD1U99TU9BJtXBzcJzfFS/EdT8H6/qHQMrZkY/yty Hw/RKZmX5wRS3yr29y3KJeiAYxXL6OQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1709717210; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=yy9Y40uQdHCgiVgpTswzucHJp+U+vA9ZyfUIrmdXeNQ=; b=TIA61+RzuPfuf9l365bibEcVqwaG3ieZhWeNNioqxq+K4dCW1mmM0xnyUDPT663w6yGmci UMslnkGBHo8UaODw== Date: Wed, 6 Mar 2024 10:26:50 +0100 (CET) From: Richard Biener To: gcc-patches@gcc.gnu.org Subject: [PATCH] tree-optimization/114249 - ICE with BB reduction vectorization MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: 1.28 X-Spamd-Result: default: False [1.28 / 50.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MISSING_MID(2.50)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_SPAM_SHORT(2.88)[0.962]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; BAYES_HAM(-3.00)[100.00%] X-Spam-Status: No, score=-10.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, MISSING_MID, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Message-Id: <20240306092719.7B6B83858010@sourceware.org> When we scrap the last def of an odd lane numbered BB reduction we can end up recording a pattern def which will later wreck code generation. The following puts this logic where it better belongs, avoiding this issue. Bootstrapped and tested on x86_64-unknown-linux-gnu, pushed. PR tree-optimization/114249 * tree-vect-slp.cc (vect_build_slp_instance): Move making a BB reduction lane number even ... (vect_slp_check_for_roots): ... here to avoid leaking pattern defs. * gcc.dg/vect/bb-slp-pr114249.c: New testcase. --- gcc/testsuite/gcc.dg/vect/bb-slp-pr114249.c | 20 ++++++++++++++++++++ gcc/tree-vect-slp.cc | 20 ++++++++++---------- 2 files changed, 30 insertions(+), 10 deletions(-) create mode 100644 gcc/testsuite/gcc.dg/vect/bb-slp-pr114249.c diff --git a/gcc/testsuite/gcc.dg/vect/bb-slp-pr114249.c b/gcc/testsuite/gcc.dg/vect/bb-slp-pr114249.c new file mode 100644 index 00000000000..64c93cd9a2d --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/bb-slp-pr114249.c @@ -0,0 +1,20 @@ +/* { dg-do compile } */ + +enum { SEG_THIN_POOL } read_only; +struct { + unsigned skip_block_zeroing; + unsigned ignore_discard; + unsigned no_discard_passdown; + unsigned error_if_no_space; +} _thin_pool_emit_segment_line_seg; +void dm_snprintf(); +void _emit_segment() +{ + int features = + (_thin_pool_emit_segment_line_seg.error_if_no_space ? 1 : 0) + + (read_only ? 1 : 0) + + (_thin_pool_emit_segment_line_seg.ignore_discard ? 1 : 0) + + (_thin_pool_emit_segment_line_seg.no_discard_passdown ? 1 : 0) + + (_thin_pool_emit_segment_line_seg.skip_block_zeroing ? 1 : 0); + dm_snprintf(features); +} diff --git a/gcc/tree-vect-slp.cc b/gcc/tree-vect-slp.cc index 324400db19e..527b06c9f9c 100644 --- a/gcc/tree-vect-slp.cc +++ b/gcc/tree-vect-slp.cc @@ -3288,15 +3288,6 @@ vect_build_slp_instance (vec_info *vinfo, " %G", scalar_stmts[i]->stmt); } - /* When a BB reduction doesn't have an even number of lanes - strip it down, treating the remaining lane as scalar. - ??? Selecting the optimal set of lanes to vectorize would be nice - but SLP build for all lanes will fail quickly because we think - we're going to need unrolling. */ - if (kind == slp_inst_kind_bb_reduc - && (scalar_stmts.length () & 1)) - remain.safe_insert (0, gimple_get_lhs (scalar_stmts.pop ()->stmt)); - /* Build the tree for the SLP instance. */ unsigned int group_size = scalar_stmts.length (); bool *matches = XALLOCAVEC (bool, group_size); @@ -7549,6 +7540,7 @@ vect_slp_check_for_roots (bb_vec_info bb_vinfo) /* ??? For now do not allow mixing ops or externs/constants. */ bool invalid = false; unsigned remain_cnt = 0; + unsigned last_idx = 0; for (unsigned i = 0; i < chain.length (); ++i) { if (chain[i].code != code) @@ -7563,7 +7555,13 @@ vect_slp_check_for_roots (bb_vec_info bb_vinfo) (chain[i].op)->stmt) != chain[i].op)) remain_cnt++; + else + last_idx = i; } + /* Make sure to have an even number of lanes as we later do + all-or-nothing discovery, not trying to split further. */ + if ((chain.length () - remain_cnt) & 1) + remain_cnt++; if (!invalid && chain.length () - remain_cnt > 1) { vec stmts; @@ -7576,7 +7574,9 @@ vect_slp_check_for_roots (bb_vec_info bb_vinfo) stmt_vec_info stmt_info; if (chain[i].dt == vect_internal_def && ((stmt_info = bb_vinfo->lookup_def (chain[i].op)), - gimple_get_lhs (stmt_info->stmt) == chain[i].op)) + gimple_get_lhs (stmt_info->stmt) == chain[i].op) + && (i != last_idx + || (stmts.length () & 1))) stmts.quick_push (stmt_info); else remain.quick_push (chain[i].op);