From patchwork Wed Nov 8 15:02:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 1861639 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=v7oDQtka; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=fgtCdoqv; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SQSyG0GGqz1yRF for ; Thu, 9 Nov 2023 02:03:17 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 021F4385772B for ; Wed, 8 Nov 2023 15:03:16 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by sourceware.org (Postfix) with ESMTPS id A41883858D1E for ; Wed, 8 Nov 2023 15:02:55 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org A41883858D1E Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de ARC-Filter: OpenARC Filter v1.0.0 sourceware.org A41883858D1E Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=195.135.220.29 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699455785; cv=none; b=ZiAPVMpgsNCKVM82QiBxXq7r/KUwNuCAxqrePdS/xQcWqev2k0yrnlN7UcrfRrPxiosvP0dIQ7VCl6C2Sbl+XuWtroKv9T/8lzfoO2GL0nKaukO/fiRkDgymhj4WUpPWtrMvRLBkVAFHQ73T8BDEUQNi3kYhTgDaFBG5wUOgQuw= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699455785; c=relaxed/simple; bh=EAV01csWsr/9+ICOG5qwjDXEVAYJI9rZ4HIrXlNTUlU=; h=DKIM-Signature:DKIM-Signature:Date:From:To:Subject:MIME-Version: Message-Id; b=vIjKGF1nwZmoxKY7g2jeQ6njb5tCq8ljv7psvMNbLSyL3pigQZRdPVbuYrZMcPC66gTmueb7xOzzPzK3tBENzNuLLM1Da1gXIiSnUeKWf7QtsR5xBLuF9SmJmDbZQkerG3dNCx8BqkNwasNJwOh/HJ6x53T0dsN9IX2UwvlE2LA= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id A3CC11F45A for ; Wed, 8 Nov 2023 15:02:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1699455774; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version:content-type:content-type; bh=o8JPehAFx6piKK9u4IkX6AIr9TOHOq6s6TWfgUMe5j8=; b=v7oDQtkaJh+o9qhiUIqkbxGWl6YJyvmq/sLQTk1Ik0CW5zjtVAUjwIBtdhSHux20pz++Ep KdsMYRbtby3Qwm/cmpmiAskiSED4LPtqbJwDZEAwmqEvcHtt/tJer7loA+qBKNZ1AVQE/O rwTwNmqe86rWGtHpj9TB0g7rJgpgzz4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1699455774; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version:content-type:content-type; bh=o8JPehAFx6piKK9u4IkX6AIr9TOHOq6s6TWfgUMe5j8=; b=fgtCdoqvKeMstKZOY65u6tU5B7ibDHs2sx5NjyeqYAsVfhwW498DNPLejKK819cQVgwAGz u0qpGlJphbb+fWBw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 8F9F9133F5 for ; Wed, 8 Nov 2023 15:02:54 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id bCLJIR6jS2XkdwAAMHmgww (envelope-from ) for ; Wed, 08 Nov 2023 15:02:54 +0000 Date: Wed, 8 Nov 2023 16:02:54 +0100 (CET) From: Richard Biener To: gcc-patches@gcc.gnu.org Subject: [PATCH 1/4] Fix SLP of masked loads MIME-Version: 1.0 Message-Id: <20231108150254.8F9F9133F5@imap2.suse-dmz.suse.de> X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org The following adjusts things to use the correct mask operand for the SLP of masked loads and gathers. Test coverage is from runtime fails of i386 specific AVX512 tests when enabling single-lane SLP. Bootstrap and regtest running on x86_64-unknown-linux-gnu. * tree-vect-stmts.cc (vectorizable_load): Use the correct vectorized mask operand. --- gcc/tree-vect-stmts.cc | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 65883e04ad7..096a857f2dd 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -10920,9 +10920,6 @@ vectorizable_load (vec_info *vinfo, gsi, stmt_info, bump); } - if (mask && !costing_p) - vec_mask = vec_masks[j]; - gimple *new_stmt = NULL; for (i = 0; i < vec_num; i++) { @@ -10931,6 +10928,8 @@ vectorizable_load (vec_info *vinfo, tree bias = NULL_TREE; if (!costing_p) { + if (mask) + vec_mask = vec_masks[vec_num * j + i]; if (loop_masks) final_mask = vect_get_loop_mask (loop_vinfo, gsi, loop_masks, @@ -11285,8 +11284,6 @@ vectorizable_load (vec_info *vinfo, at_loop, offset, &dummy, gsi, &ptr_incr, simd_lane_access_p, bump); - if (mask) - vec_mask = vec_masks[0]; } else if (!costing_p) { @@ -11297,8 +11294,6 @@ vectorizable_load (vec_info *vinfo, else dataref_ptr = bump_vector_ptr (vinfo, dataref_ptr, ptr_incr, gsi, stmt_info, bump); - if (mask) - vec_mask = vec_masks[j]; } if (grouped_load || slp_perm) @@ -11312,6 +11307,8 @@ vectorizable_load (vec_info *vinfo, tree bias = NULL_TREE; if (!costing_p) { + if (mask) + vec_mask = vec_masks[vec_num * j + i]; if (loop_masks) final_mask = vect_get_loop_mask (loop_vinfo, gsi, loop_masks, vec_num * ncopies, vectype,