From patchwork Tue May 11 23:35:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 1477371 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=wXLeJN/k; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfvSM1d44z9sW5 for ; Wed, 12 May 2021 09:36:03 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5949B3838021; Tue, 11 May 2021 23:35:47 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5949B3838021 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1620776147; bh=J6K3SmWIaO7PWGBHOXpWgx4AL4+uOdbkSv6t7Whp1T4=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=wXLeJN/k0tHPlnDMzdKLUQ4rbeB6D49njZzfcX9T2AufpUAW1FOJlx8Fr5mKa6/HL a82nLGusZnhrPLAUT6ReB921v8MW4AkrsuceyExL/o8BXpHabq/oBBWtTJXeNySEfH g0SPtI+cXZLVkjEdTWsg44NPnv0t50a8aXuK9Pww= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-pf1-x434.google.com (mail-pf1-x434.google.com [IPv6:2607:f8b0:4864:20::434]) by sourceware.org (Postfix) with ESMTPS id 3A135383803E for ; Tue, 11 May 2021 23:35:41 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 3A135383803E Received: by mail-pf1-x434.google.com with SMTP id a5so10175890pfa.11 for ; Tue, 11 May 2021 16:35:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=J6K3SmWIaO7PWGBHOXpWgx4AL4+uOdbkSv6t7Whp1T4=; b=szTJgjLizKvKKa7u4wvipmnIWVB7vOnld5DJkWbGYAX0bK3Z0z5w4e8XphcVAyKE6j 5yRLofhjPlKprVyP2gXbFQyjpwQe2bo2wJOiPnXLwSCmmG0P9yxm4dR807WEAcT9dC+f vNE+rVFM9RCzE0/qw0839e5KJkYXG610SSxtKRUvyWSSewnYL2ACiVCpkYKDKCDX2Jvz GhsUl8FKmQFTaCixKrgI4dMrPgh5aMeEI0BLgHiMC/o7z6lbWm4cWEy7Z2Cz83117+JM oYa/Vv/vWx4fbkJNf58Hhcp9BmLevlp4c2xmNhH+V+t5wY3xh38P9tv5nk8pd7/ce1k9 BmMw== X-Gm-Message-State: AOAM530EV8XOqQuAkokO/6WCYOFoHjPukqNpoUy8tOf9L1wNxppjw3SV 5T7B3EQ95sHx21G37pSp3sU= X-Google-Smtp-Source: ABdhPJwgjRNQ0UVaJoNx24GuFQ1rKkJqqsqUp2oJ4l9W36w2xJgCcpTjVzddh8C/YZV4Iju+ap9WQA== X-Received: by 2002:a63:e60a:: with SMTP id g10mr32921638pgh.21.1620776140095; Tue, 11 May 2021 16:35:40 -0700 (PDT) Received: from gnu-cfl-2.localdomain ([172.56.39.231]) by smtp.gmail.com with ESMTPSA id 66sm14201087pfg.181.2021.05.11.16.35.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 16:35:37 -0700 (PDT) Received: from gnu-cfl-2.. (localhost [IPv6:::1]) by gnu-cfl-2.localdomain (Postfix) with ESMTP id 5A655C040B; Tue, 11 May 2021 16:35:35 -0700 (PDT) To: gcc-patches@gcc.gnu.org Subject: [PATCH v2 01/11] Add TARGET_READ_MEMSET_VALUE/TARGET_GEN_MEMSET_VALUE Date: Tue, 11 May 2021 16:35:25 -0700 Message-Id: <20210511233535.4448-2-hjl.tools@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210511233535.4448-1-hjl.tools@gmail.com> References: <20210511233535.4448-1-hjl.tools@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-3034.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, LOTS_OF_MONEY, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: "H.J. Lu via Gcc-patches" From: "H.J. Lu" Reply-To: "H.J. Lu" Cc: Richard Sandiford Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" Add TARGET_READ_MEMSET_VALUE and TARGET_GEN_MEMSET_VALUE to support target instructions to duplicate QImode value to TImode/OImode/XImode value for memmset. Define SCRATCH_SSE_REG as a scratch register for ix86_gen_memset_value. gcc/ PR middle-end/90773 * builtins.c (builtin_memset_read_str): Call targetm.read_memset_value. (builtin_memset_gen_str): Call targetm.gen_memset_value. * target.def (read_memset_value): New hook. (gen_memset_value): Likewise. * targhooks.c: Inclue "builtins.h". (default_read_memset_value): New function. (default_gen_memset_value): Likewise. * targhooks.h (default_read_memset_value): New prototype. (default_gen_memset_value): Likewise. * config/i386/i386-expand.c (ix86_expand_vector_init_duplicate): Make it global. * config/i386/i386-protos.h (ix86_minimum_incoming_stack_boundary): New. (ix86_expand_vector_init_duplicate): Likewise. * config/i386/i386.c (ix86_minimum_incoming_stack_boundary): Add an argument to ignore stack_alignment_estimated. It is passed as false by default. (ix86_gen_memset_value_from_prev): New function. (ix86_gen_memset_value): Likewise. (ix86_read_memset_value): Likewise. (TARGET_GEN_MEMSET_VALUE): New. (TARGET_READ_MEMSET_VALUE): Likewise. * config/i386/i386.h (SCRATCH_SSE_REG): New. * doc/tm.texi.in: Add TARGET_READ_MEMSET_VALUE and TARGET_GEN_MEMSET_VALUE hooks. * doc/tm.texi: Regenerated. gcc/testsuite/ PR middle-end/90773 * gcc.target/i386/pr90773-15.c: New test. * gcc.target/i386/pr90773-16.c: Likewise. * gcc.target/i386/pr90773-17.c: Likewise. * gcc.target/i386/pr90773-18.c: Likewise. * gcc.target/i386/pr90773-19.c: Likewise. --- gcc/builtins.c | 47 +--- gcc/config/i386/i386-expand.c | 2 +- gcc/config/i386/i386-protos.h | 5 + gcc/config/i386/i386.c | 268 ++++++++++++++++++++- gcc/config/i386/i386.h | 4 + gcc/doc/tm.texi | 16 ++ gcc/doc/tm.texi.in | 4 + gcc/expr.c | 1 - gcc/target.def | 20 ++ gcc/targhooks.c | 56 +++++ gcc/targhooks.h | 4 + gcc/testsuite/gcc.target/i386/pr90773-15.c | 14 ++ gcc/testsuite/gcc.target/i386/pr90773-16.c | 14 ++ gcc/testsuite/gcc.target/i386/pr90773-17.c | 14 ++ gcc/testsuite/gcc.target/i386/pr90773-18.c | 15 ++ gcc/testsuite/gcc.target/i386/pr90773-19.c | 14 ++ 16 files changed, 449 insertions(+), 49 deletions(-) create mode 100644 gcc/testsuite/gcc.target/i386/pr90773-15.c create mode 100644 gcc/testsuite/gcc.target/i386/pr90773-16.c create mode 100644 gcc/testsuite/gcc.target/i386/pr90773-17.c create mode 100644 gcc/testsuite/gcc.target/i386/pr90773-18.c create mode 100644 gcc/testsuite/gcc.target/i386/pr90773-19.c diff --git a/gcc/builtins.c b/gcc/builtins.c index 2f0efae11e8..6951f2d3633 100644 --- a/gcc/builtins.c +++ b/gcc/builtins.c @@ -6584,24 +6584,11 @@ expand_builtin_strncpy (tree exp, rtx target) previous iteration. */ rtx -builtin_memset_read_str (void *data, void *prevp, +builtin_memset_read_str (void *data, void *prev, HOST_WIDE_INT offset ATTRIBUTE_UNUSED, scalar_int_mode mode) { - by_pieces_prev *prev = (by_pieces_prev *) prevp; - if (prev != nullptr && prev->data != nullptr) - { - /* Use the previous data in the same mode. */ - if (prev->mode == mode) - return prev->data; - } - - const char *c = (const char *) data; - char *p = XALLOCAVEC (char, GET_MODE_SIZE (mode)); - - memset (p, *c, GET_MODE_SIZE (mode)); - - return c_readstr (p, mode); + return targetm.read_memset_value ((const char *) data, prev, mode); } /* Callback routine for store_by_pieces. Return the RTL of a register @@ -6611,37 +6598,11 @@ builtin_memset_read_str (void *data, void *prevp, nullptr, it has the RTL info from the previous iteration. */ static rtx -builtin_memset_gen_str (void *data, void *prevp, +builtin_memset_gen_str (void *data, void *prev, HOST_WIDE_INT offset ATTRIBUTE_UNUSED, scalar_int_mode mode) { - rtx target, coeff; - size_t size; - char *p; - - by_pieces_prev *prev = (by_pieces_prev *) prevp; - if (prev != nullptr && prev->data != nullptr) - { - /* Use the previous data in the same mode. */ - if (prev->mode == mode) - return prev->data; - - target = simplify_gen_subreg (mode, prev->data, prev->mode, 0); - if (target != nullptr) - return target; - } - - size = GET_MODE_SIZE (mode); - if (size == 1) - return (rtx) data; - - p = XALLOCAVEC (char, size); - memset (p, 1, size); - coeff = c_readstr (p, mode); - - target = convert_to_mode (mode, (rtx) data, 1); - target = expand_mult (mode, target, coeff, NULL_RTX, 1); - return force_reg (mode, target); + return targetm.gen_memset_value ((rtx) data, prev, mode); } /* Expand expression EXP, which is a call to the memset builtin. Return diff --git a/gcc/config/i386/i386-expand.c b/gcc/config/i386/i386-expand.c index 5cfde5b3d30..7f1dff6337c 100644 --- a/gcc/config/i386/i386-expand.c +++ b/gcc/config/i386/i386-expand.c @@ -13640,7 +13640,7 @@ static bool expand_vec_perm_1 (struct expand_vec_perm_d *d); /* A subroutine of ix86_expand_vector_init. Store into TARGET a vector with all elements equal to VAR. Return true if successful. */ -static bool +bool ix86_expand_vector_init_duplicate (bool mmx_ok, machine_mode mode, rtx target, rtx val) { diff --git a/gcc/config/i386/i386-protos.h b/gcc/config/i386/i386-protos.h index 7782cf1163f..c4896c2da74 100644 --- a/gcc/config/i386/i386-protos.h +++ b/gcc/config/i386/i386-protos.h @@ -50,6 +50,9 @@ extern void ix86_reset_previous_fndecl (void); extern bool ix86_using_red_zone (void); +extern unsigned int ix86_minimum_incoming_stack_boundary (bool, + bool = false); + extern unsigned int ix86_regmode_natural_size (machine_mode); #ifdef RTX_CODE extern int standard_80387_constant_p (rtx); @@ -257,6 +260,8 @@ extern void ix86_expand_mul_widen_hilo (rtx, rtx, rtx, bool, bool); extern void ix86_expand_sse2_mulv4si3 (rtx, rtx, rtx); extern void ix86_expand_sse2_mulvxdi3 (rtx, rtx, rtx); extern void ix86_expand_sse2_abs (rtx, rtx); +extern bool ix86_expand_vector_init_duplicate (bool, machine_mode, rtx, + rtx); /* In i386-c.c */ extern void ix86_target_macros (void); diff --git a/gcc/config/i386/i386.c b/gcc/config/i386/i386.c index 915f89f571a..f9cbc1d10eb 100644 --- a/gcc/config/i386/i386.c +++ b/gcc/config/i386/i386.c @@ -415,7 +415,6 @@ static unsigned int split_stack_prologue_scratch_regno (void); static bool i386_asm_output_addr_const_extra (FILE *, rtx); static bool ix86_can_inline_p (tree, tree); -static unsigned int ix86_minimum_incoming_stack_boundary (bool); /* Whether -mtune= or -march= were specified */ @@ -7232,8 +7231,9 @@ find_drap_reg (void) /* Return minimum incoming stack alignment. */ -static unsigned int -ix86_minimum_incoming_stack_boundary (bool sibcall) +unsigned int +ix86_minimum_incoming_stack_boundary (bool sibcall, + bool ignore_estimated) { unsigned int incoming_stack_boundary; @@ -7248,7 +7248,8 @@ ix86_minimum_incoming_stack_boundary (bool sibcall) estimated stack alignment is 128bit. */ else if (!sibcall && ix86_force_align_arg_pointer - && crtl->stack_alignment_estimated == 128) + && (ignore_estimated + || crtl->stack_alignment_estimated == 128)) incoming_stack_boundary = MIN_STACK_BOUNDARY; else incoming_stack_boundary = ix86_default_incoming_stack_boundary; @@ -23031,6 +23032,259 @@ ix86_optab_supported_p (int op, machine_mode mode1, machine_mode, } } +/* Return the RTL for memset in MODE from PREV. */ + +static rtx +ix86_gen_memset_value_from_prev (by_pieces_prev *prevp, + scalar_int_mode mode) +{ + rtx prev = prevp->data; + + /* Use the previous data in the same mode. */ + if (prevp->mode == mode) + return prev; + + machine_mode prev_mode = prevp->mode; + size_t size = GET_MODE_SIZE (prev_mode); + + /* NB: Skip if the previous value is 1 byte or less. CONST_WIDE_INT + is in VOIDmode whose size is 0. */ + if (size <= 1) + return nullptr; + + rtx reg, reg_ti; + switch (size) + { + default: + gcc_unreachable (); + + case 2: + case 4: + return simplify_gen_subreg (mode, prev, prev_mode, 0); + + case 8: + /* In 64-bit mode, use SUBREG since word size is 8 bytes. */ + if (TARGET_64BIT) + return simplify_gen_subreg (mode, prev, prev_mode, 0); + + switch (GET_MODE_SIZE (mode)) + { + default: + gcc_unreachable (); + case 2: + case 4: +do_hi_si_mode: + /* In 32-bit mode, Extract the value from an 8-byte + register into an integer register first. */ + reg = gen_reg_rtx (SImode); + emit_move_insn (reg, + simplify_gen_subreg (SImode, prev, + prev_mode, 0)); + return simplify_gen_subreg (mode, reg, SImode, 0); + } + break; + + case 16: + switch (GET_MODE_SIZE (mode)) + { + default: + gcc_unreachable (); + case 2: + case 4: + /* Extract the value from a 16-byte vector register into + an integer register first. */ + goto do_hi_si_mode; + case 8: + return simplify_gen_subreg (mode, prev, prev_mode, 0); + case 16: + return prev; + } + break; + + case 32: + switch (GET_MODE_SIZE (mode)) + { + default: + gcc_unreachable (); + case 2: +do_himode: + /* Extract the value from a 32-byte vector register into + a 16-byte vector register first. */ + reg_ti = gen_reg_rtx (TImode); + emit_move_insn (reg_ti, + simplify_gen_subreg (TImode, prev, + prev_mode, 0)); + /* Then extract the value from a 16-byte vector register + into an integer register. */ + reg = gen_reg_rtx (SImode); + emit_move_insn (reg, + simplify_gen_subreg (SImode, reg_ti, + TImode, 0)); + return simplify_gen_subreg (mode, reg, SImode, 0); + + case 4: + case 8: +do_si_di_mode: + /* Extract the value from a 32-byte vector register into + a 16-byte vector register first. */ + reg_ti = gen_reg_rtx (TImode); + emit_move_insn (reg_ti, + simplify_gen_subreg (TImode, prev, + prev_mode, 0)); + /* Generate 4/8-byte SSE -> INT move instruction. */ + reg = gen_reg_rtx (mode); + emit_move_insn (reg, + simplify_gen_subreg (mode, reg_ti, + TImode, 0)); + return reg; + case 16: + return simplify_gen_subreg (mode, prev, prev_mode, 0); + case 32: + return prev; + } + + case 64: + switch (GET_MODE_SIZE (mode)) + { + default: + gcc_unreachable (); + case 2: + /* Extract the value from a 64-byte vector register into + a 16-byte vector register first. */ + goto do_himode; + case 4: + case 8: + /* Extract the value from a 64-byte vector register into + a 16-byte vector register first. */ + goto do_si_di_mode; + case 16: + case 32: + return simplify_gen_subreg (mode, prev, prev_mode, 0); + case 64: + return prev; + } + } + + return nullptr; +} + +/* Implement the TARGET_GEN_MEMSET_VALUE hook. */ + +static rtx +ix86_gen_memset_value (rtx data, void *prevp, scalar_int_mode mode) +{ + /* Don't use the previous value if size is 1. */ + if (GET_MODE_SIZE (mode) == 1) + return data; + + by_pieces_prev *prev = (by_pieces_prev *) prevp; + if (prev != nullptr && prev->data != nullptr) + { + rtx value = ix86_gen_memset_value_from_prev (prev, mode); + if (value) + return value; + } + + /* Use default_gen_memset_value for vector store won't be used. */ + if (GET_MODE_SIZE (mode) <= GET_MODE_SIZE (DImode)) + return default_gen_memset_value (data, prevp, mode); + + rtx one, target; + scalar_mode one_mode; + + unsigned int incoming_stack_boundary + = ix86_minimum_incoming_stack_boundary (false, true); + + switch (GET_MODE_SIZE (mode)) + { + default: + gcc_unreachable (); + + case 64: + if (!TARGET_AVX512BW) + { + rtx tmp; + /* NB: Don't increase stack alignment requirement by using a + scratch SSE register. */ + if (GET_MODE_ALIGNMENT (V32QImode) > incoming_stack_boundary) + tmp = gen_rtx_REG (V32QImode, SCRATCH_SSE_REG); + else + tmp = gen_reg_rtx (V32QImode); + if (!ix86_expand_vector_init_duplicate (false, V32QImode, + tmp, data)) + gcc_unreachable (); + target = gen_rtx_VEC_CONCAT (V64QImode, tmp, tmp); + if (REGNO (tmp) == SCRATCH_SSE_REG) + { + tmp = gen_rtx_REG (V64QImode, SCRATCH_SSE_REG); + emit_move_insn (tmp, target); + return gen_rtx_REG (mode, SCRATCH_SSE_REG); + } + else + return convert_to_mode (mode, target, 1); + } + /* FALLTHRU */ + case 16: + case 32: + one_mode = QImode; + one = data; + break; + } + + unsigned int nunits = GET_MODE_SIZE (mode) / GET_MODE_SIZE (one_mode); + machine_mode vector_mode; + if (!mode_for_vector (one_mode, nunits).exists (&vector_mode)) + gcc_unreachable (); + + /* NB: Don't increase stack alignment requirement by using a scratch + SSE register. */ + if (GET_MODE_ALIGNMENT (vector_mode) > incoming_stack_boundary) + target = gen_rtx_REG (vector_mode, SCRATCH_SSE_REG); + else + target = gen_reg_rtx (vector_mode); + if (!ix86_expand_vector_init_duplicate (false, vector_mode, target, + one)) + gcc_unreachable (); + + if (REGNO (target) == SCRATCH_SSE_REG) + return gen_rtx_REG (mode, SCRATCH_SSE_REG); + else + return convert_to_mode (mode, target, 1); +} + +/* Implement the TARGET_READ_MEMSET_VALUE hook. */ + +static rtx +ix86_read_memset_value (const char *str, void *prevp, + scalar_int_mode mode) +{ + rtx value; + + by_pieces_prev *prev = (by_pieces_prev *) prevp; + if (prev != nullptr && prev->data != nullptr) + { + /* Don't use the previous value if size is 1. */ + if (GET_MODE_SIZE (mode) == 1) + return default_read_memset_value (str, nullptr, mode); + + value = ix86_gen_memset_value_from_prev (prev, mode); + if (value) + return value; + + return default_read_memset_value (str, nullptr, mode); + } + + /* Use default_gen_memset_value if vector store can't be used. + NB: Need AVX2 for fast vector duplication and gen_reg_rtx. */ + if (GET_MODE_SIZE (mode) <= GET_MODE_SIZE (DImode) + || !TARGET_AVX2 + || !reg_rtx_no) + return default_read_memset_value (str, nullptr, mode); + + value = default_read_memset_value (str, nullptr, QImode); + return ix86_gen_memset_value (value, nullptr, mode); +} + /* Address space support. This is not "far pointers" in the 16-bit sense, but an easy way @@ -23932,6 +24186,12 @@ static bool ix86_libc_has_fast_function (int fcode ATTRIBUTE_UNUSED) #undef TARGET_LIBC_HAS_FAST_FUNCTION #define TARGET_LIBC_HAS_FAST_FUNCTION ix86_libc_has_fast_function +#undef TARGET_GEN_MEMSET_VALUE +#define TARGET_GEN_MEMSET_VALUE ix86_gen_memset_value + +#undef TARGET_READ_MEMSET_VALUE +#define TARGET_READ_MEMSET_VALUE ix86_read_memset_value + #if CHECKING_P #undef TARGET_RUN_TARGET_SELFTESTS #define TARGET_RUN_TARGET_SELFTESTS selftest::ix86_run_selftests diff --git a/gcc/config/i386/i386.h b/gcc/config/i386/i386.h index 97d6f3863cb..45d86802c51 100644 --- a/gcc/config/i386/i386.h +++ b/gcc/config/i386/i386.h @@ -1131,6 +1131,10 @@ extern const char *host_detect_local_cpu (int argc, const char **argv); #define FIRST_MASK_REG MASK0_REG #define LAST_MASK_REG MASK7_REG +/* A scratch vector reg. */ +#define SCRATCH_SSE_REG \ + (TARGET_64BIT ? LAST_REX_SSE_REG : LAST_SSE_REG) + /* Override this in other tm.h files to cope with various OS lossage requiring a frame pointer. */ #ifndef SUBTARGET_FRAME_POINTER_REQUIRED diff --git a/gcc/doc/tm.texi b/gcc/doc/tm.texi index 85ea9395560..51385044e76 100644 --- a/gcc/doc/tm.texi +++ b/gcc/doc/tm.texi @@ -11868,6 +11868,22 @@ This function prepares to emit a conditional comparison within a sequence @var{bit_code} is @code{AND} or @code{IOR}, which is the op on the compares. @end deftypefn +@deftypefn {Target Hook} rtx TARGET_READ_MEMSET_VALUE (const char *@var{c}, void *@var{prev}, scalar_int_mode @var{mode}) +This function returns the RTL of a constant integer corresponding to +target reading @code{GET_MODE_SIZE (@var{mode})} bytes from the stringn +constant @var{str}. If @var{prev} is not @samp{nullptr}, it contains +the RTL information from the previous interation. +@end deftypefn + +@deftypefn {Target Hook} rtx TARGET_GEN_MEMSET_VALUE (rtx @var{data}, void *@var{prev}, scalar_int_mode @var{mode}) +This function returns the RTL of a register containing +@code{GET_MODE_SIZE (@var{mode})} consecutive copies of the unsigned +char value given in the RTL register @var{data}. For example, if +@var{mode} is 4 bytes wide, return the RTL for 0x01010101*@var{data}. +If @var{PREV} is not @samp{nullptr}, it is the RTL information from +the previous iteration. +@end deftypefn + @deftypefn {Target Hook} unsigned TARGET_LOOP_UNROLL_ADJUST (unsigned @var{nunroll}, class loop *@var{loop}) This target hook returns a new value for the number of times @var{loop} should be unrolled. The parameter @var{nunroll} is the number of times diff --git a/gcc/doc/tm.texi.in b/gcc/doc/tm.texi.in index d8e3de14af1..8d4c3949fbf 100644 --- a/gcc/doc/tm.texi.in +++ b/gcc/doc/tm.texi.in @@ -7956,6 +7956,10 @@ lists. @hook TARGET_GEN_CCMP_NEXT +@hook TARGET_READ_MEMSET_VALUE + +@hook TARGET_GEN_MEMSET_VALUE + @hook TARGET_LOOP_UNROLL_ADJUST @defmac POWI_MAX_MULTS diff --git a/gcc/expr.c b/gcc/expr.c index 1b65f6b3245..42ef5bdf5d5 100644 --- a/gcc/expr.c +++ b/gcc/expr.c @@ -1169,7 +1169,6 @@ op_by_pieces_d::run () /* NB: widest_int_mode_for_size checks M_MAX_SIZE > 1. */ scalar_int_mode mode = widest_int_mode_for_size (m_max_size); mode = get_usable_mode (mode, m_len); - by_pieces_prev to_prev = { nullptr, mode }; by_pieces_prev from_prev = { nullptr, mode }; diff --git a/gcc/target.def b/gcc/target.def index bbaf6b4f3a0..c9aca40fa88 100644 --- a/gcc/target.def +++ b/gcc/target.def @@ -2694,6 +2694,26 @@ DEFHOOK rtx, (rtx_insn **prep_seq, rtx_insn **gen_seq, rtx prev, int cmp_code, tree op0, tree op1, int bit_code), NULL) +DEFHOOK +(read_memset_value, + "This function returns the RTL of a constant integer corresponding to\n\ +target reading @code{GET_MODE_SIZE (@var{mode})} bytes from the stringn\n\ +constant @var{str}. If @var{prev} is not @samp{nullptr}, it contains\n\ +the RTL information from the previous interation.", + rtx, (const char *c, void *prev, scalar_int_mode mode), + default_read_memset_value) + +DEFHOOK +(gen_memset_value, + "This function returns the RTL of a register containing\n\ +@code{GET_MODE_SIZE (@var{mode})} consecutive copies of the unsigned\n\ +char value given in the RTL register @var{data}. For example, if\n\ +@var{mode} is 4 bytes wide, return the RTL for 0x01010101*@var{data}.\n\ +If @var{PREV} is not @samp{nullptr}, it is the RTL information from\n\ +the previous iteration.", + rtx, (rtx data, void *prev, scalar_int_mode mode), + default_gen_memset_value) + /* Return a new value for loop unroll size. */ DEFHOOK (loop_unroll_adjust, diff --git a/gcc/targhooks.c b/gcc/targhooks.c index 2e0fdb797e0..287907c72d7 100644 --- a/gcc/targhooks.c +++ b/gcc/targhooks.c @@ -90,6 +90,7 @@ along with GCC; see the file COPYING3. If not see #include "attribs.h" #include "asan.h" #include "emit-rtl.h" +#include "builtins.h" bool default_legitimate_address_p (machine_mode mode ATTRIBUTE_UNUSED, @@ -2548,4 +2549,59 @@ default_memtag_untagged_pointer (rtx tagged_pointer, rtx target) return untagged_base; } +/* Default implementation of TARGET_READ_MEMSET_VALUE. */ + +rtx +default_read_memset_value (const char *c, void *prevp, + scalar_int_mode mode) +{ + by_pieces_prev *prev = (by_pieces_prev *) prevp; + if (prev != nullptr && prev->data != nullptr) + { + /* Use the previous data in the same mode. */ + if (prev->mode == mode) + return prev->data; + } + + char *p = XALLOCAVEC (char, GET_MODE_SIZE (mode)); + + memset (p, *c, GET_MODE_SIZE (mode)); + + return c_readstr (p, mode); +} + +/* Default implementation of TARGET_GEN_MEMSET_VALUE. */ + +rtx +default_gen_memset_value (rtx data, void *prevp, scalar_int_mode mode) +{ + rtx target, coeff; + size_t size; + char *p; + + by_pieces_prev *prev = (by_pieces_prev *) prevp; + if (prev != nullptr && prev->data != nullptr) + { + /* Use the previous data in the same mode. */ + if (prev->mode == mode) + return prev->data; + + target = simplify_gen_subreg (mode, prev->data, prev->mode, 0); + if (target != nullptr) + return target; + } + + size = GET_MODE_SIZE (mode); + if (size == 1) + return data; + + p = XALLOCAVEC (char, size); + memset (p, 1, size); + coeff = c_readstr (p, mode); + + target = convert_to_mode (mode, data, 1); + target = expand_mult (mode, target, coeff, NULL_RTX, 1); + return force_reg (mode, target); +} + #include "gt-targhooks.h" diff --git a/gcc/targhooks.h b/gcc/targhooks.h index b537038c0aa..3c00927e196 100644 --- a/gcc/targhooks.h +++ b/gcc/targhooks.h @@ -300,4 +300,8 @@ extern rtx default_memtag_set_tag (rtx, rtx, rtx); extern rtx default_memtag_extract_tag (rtx, rtx); extern rtx default_memtag_untagged_pointer (rtx, rtx); +extern rtx default_read_memset_value (const char *, void *, + scalar_int_mode); +extern rtx default_gen_memset_value (rtx, void *, scalar_int_mode); + #endif /* GCC_TARGHOOKS_H */ diff --git a/gcc/testsuite/gcc.target/i386/pr90773-15.c b/gcc/testsuite/gcc.target/i386/pr90773-15.c new file mode 100644 index 00000000000..c0a96fed892 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr90773-15.c @@ -0,0 +1,14 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-options "-O2 -march=skylake-avx512" } */ + +extern char *dst; + +void +foo (int c) +{ + __builtin_memset (dst, c, 17); +} + +/* { dg-final { scan-assembler-times "vpbroadcastb\[\\t \]+%edi, %xmm\[0-9\]+" 1 } } */ +/* { dg-final { scan-assembler-times "vmovdqu\[\\t \]+%xmm\[0-9\]+, \\(%\[\^,\]+\\)" 1 } } */ +/* { dg-final { scan-assembler-times "movb\[\\t \]+%dil, 16\\(%\[\^,\]+\\)" 1 } } */ diff --git a/gcc/testsuite/gcc.target/i386/pr90773-16.c b/gcc/testsuite/gcc.target/i386/pr90773-16.c new file mode 100644 index 00000000000..d2d1ec6141c --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr90773-16.c @@ -0,0 +1,14 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-options "-O2 -march=skylake-avx512" } */ + +extern char *dst; + +void +foo (void) +{ + __builtin_memset (dst, -1, 17); +} + +/* { dg-final { scan-assembler-times "vpcmpeqd" 1 } } */ +/* { dg-final { scan-assembler-times "vmovdqu\[\\t \]+%xmm\[0-9\]+, \\(%\[\^,\]+\\)" 1 } } */ +/* { dg-final { scan-assembler-times "movb\[\\t \]+\\\$-1, 16\\(%\[\^,\]+\\)" 1 } } */ diff --git a/gcc/testsuite/gcc.target/i386/pr90773-17.c b/gcc/testsuite/gcc.target/i386/pr90773-17.c new file mode 100644 index 00000000000..6c8da7d24ef --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr90773-17.c @@ -0,0 +1,14 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-options "-O2 -march=skylake-avx512" } */ + +extern char *dst; + +void +foo (void) +{ + __builtin_memset (dst, 12, 19); +} + +/* { dg-final { scan-assembler-times "vpbroadcastb" 1 } } */ +/* { dg-final { scan-assembler-times "vmovdqu\[\\t \]+%xmm\[0-9\]+, \\(%\[\^,\]+\\)" 1 } } */ +/* { dg-final { scan-assembler-times "vmovd\[\\t \]+%xmm\[0-9\]+, 15\\(%\[\^,\]+\\)" 1 } } */ diff --git a/gcc/testsuite/gcc.target/i386/pr90773-18.c b/gcc/testsuite/gcc.target/i386/pr90773-18.c new file mode 100644 index 00000000000..b0687abbe01 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr90773-18.c @@ -0,0 +1,15 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -march=skylake-avx512" } */ + +extern char *dst; + +void +foo (void) +{ + __builtin_memset (dst, 12, 9); +} + +/* { dg-final { scan-assembler-times "movabsq\[\\t \]+\\\$868082074056920076, %r" 1 { target { ! ia32 } } } } */ +/* { dg-final { scan-assembler-times "movl\[\\t \]+\\\$202116108, \\(%\[\^,\]+\\)" 1 { target ia32 } } } */ +/* { dg-final { scan-assembler-times "movl\[\\t \]+\\\$202116108, 4\\(%\[\^,\]+\\)" 1 { target ia32 } } } */ +/* { dg-final { scan-assembler-times "movb\[\\t \]+\\\$12, 8\\(%\[\^,\]+\\)" 1 } } */ diff --git a/gcc/testsuite/gcc.target/i386/pr90773-19.c b/gcc/testsuite/gcc.target/i386/pr90773-19.c new file mode 100644 index 00000000000..8aa5540bacc --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr90773-19.c @@ -0,0 +1,14 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -march=skylake" } */ + +extern char *dst; + +void +foo (void) +{ + __builtin_memset (dst, 12, 9); +} + +/* { dg-final { scan-assembler-times "movabsq\[\\t \]+\\\$868082074056920076, %r" 1 { target { ! ia32 } } } } */ +/* { dg-final { scan-assembler-times "movl\[\\t \]+\\\$202116108, \\(%\[\^,\]+\\)" 1 { target ia32 } } } */ +/* { dg-final { scan-assembler-times "movl\[\\t \]+\\\$202116108, 4\\(%\[\^,\]+\\)" 1 { target ia32 } } } */