{"id":2230854,"url":"http://patchwork.ozlabs.org/api/patches/2230854/?format=json","web_url":"http://patchwork.ozlabs.org/project/gcc/patch/afL55ljtPX2nWSG6@cowardly-lion.the-meissners.org/","project":{"id":17,"url":"http://patchwork.ozlabs.org/api/projects/17/?format=json","name":"GNU Compiler Collection","link_name":"gcc","list_id":"gcc-patches.gcc.gnu.org","list_email":"gcc-patches@gcc.gnu.org","web_url":null,"scm_url":null,"webscm_url":null,"list_archive_url":"","list_archive_url_format":"","commit_url_format":""},"msgid":"","list_archive_url":null,"date":"2026-04-30T06:42:46","name":"GCC 17, PowerPC Dense Math V7 (patch 4/7) -- Implement 512-bit dense math registers","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"fc6dc7f489fe8c24c80208c4a7a8639abe2df77d","submitter":{"id":73991,"url":"http://patchwork.ozlabs.org/api/people/73991/?format=json","name":"Michael Meissner","email":"meissner@linux.ibm.com"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/gcc/patch/afL55ljtPX2nWSG6@cowardly-lion.the-meissners.org/mbox/","series":[{"id":502215,"url":"http://patchwork.ozlabs.org/api/series/502215/?format=json","web_url":"http://patchwork.ozlabs.org/project/gcc/list/?series=502215","date":"2026-04-30T06:42:46","name":"GCC 17, PowerPC Dense Math V7 (patch 4/7) -- Implement 512-bit dense math registers","version":1,"mbox":"http://patchwork.ozlabs.org/series/502215/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/2230854/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/2230854/checks/","tags":{},"related":[],"headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","gcc-patches@gcc.gnu.org"],"Delivered-To":["patchwork-incoming@legolas.ozlabs.org","gcc-patches@gcc.gnu.org"],"Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256\n header.s=pp1 header.b=oavBb2Sb;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org\n (client-ip=2620:52:6:3111::32; helo=vm01.sourceware.org;\n envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org;\n receiver=patchwork.ozlabs.org)","sourceware.org;\n\tdkim=pass (2048-bit key,\n unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256\n header.s=pp1 header.b=oavBb2Sb","sourceware.org;\n dmarc=none (p=none dis=none) header.from=linux.ibm.com","sourceware.org; spf=pass smtp.mailfrom=linux.ibm.com","server2.sourceware.org;\n arc=none smtp.remote-ip=148.163.156.1"],"Received":["from vm01.sourceware.org (vm01.sourceware.org\n [IPv6:2620:52:6:3111::32])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4g5l4l6RZrz1yHv\n\tfor ; Thu, 30 Apr 2026 16:43:50 +1000 (AEST)","from vm01.sourceware.org (localhost [127.0.0.1])\n\tby sourceware.org (Postfix) with ESMTP id 769184BB5938\n\tfor ; Thu, 30 Apr 2026 06:43:48 +0000 (GMT)","from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com\n [148.163.156.1])\n by sourceware.org (Postfix) with ESMTPS id 78A214BB58DF\n for ; Thu, 30 Apr 2026 06:42:54 +0000 (GMT)","from pps.filterd (m0360083.ppops.net [127.0.0.1])\n by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id\n 63U2GZij341349; Thu, 30 Apr 2026 06:42:53 GMT","from ppma23.wdc07v.mail.ibm.com\n (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93])\n by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4drn44xdbd-1\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);\n Thu, 30 Apr 2026 06:42:52 +0000 (GMT)","from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1])\n by ppma23.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id\n 63U6cwBc008790;\n Thu, 30 Apr 2026 06:42:51 GMT","from smtprelay04.dal12v.mail.ibm.com ([172.16.1.6])\n by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4ds9ehhnwv-1\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);\n Thu, 30 Apr 2026 06:42:51 +0000 (GMT)","from smtpav05.wdc07v.mail.ibm.com (smtpav05.wdc07v.mail.ibm.com\n [10.39.53.232])\n by smtprelay04.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id\n 63U6gmn123986802\n (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);\n Thu, 30 Apr 2026 06:42:49 GMT","from smtpav05.wdc07v.mail.ibm.com (unknown [127.0.0.1])\n by IMSVA (Postfix) with ESMTP id C9E4C58053;\n Thu, 30 Apr 2026 06:42:48 +0000 (GMT)","from smtpav05.wdc07v.mail.ibm.com (unknown [127.0.0.1])\n by IMSVA (Postfix) with ESMTP id BA8DB58043;\n Thu, 30 Apr 2026 06:42:47 +0000 (GMT)","from cowardly-lion.the-meissners.org (unknown [9.61.149.103])\n by smtpav05.wdc07v.mail.ibm.com (Postfix) with ESMTPS;\n Thu, 30 Apr 2026 06:42:47 +0000 (GMT)"],"DKIM-Filter":["OpenDKIM Filter v2.11.0 sourceware.org 769184BB5938","OpenDKIM Filter v2.11.0 sourceware.org 78A214BB58DF"],"DMARC-Filter":"OpenDMARC Filter v1.4.2 sourceware.org 78A214BB58DF","ARC-Filter":"OpenARC Filter v1.0.0 sourceware.org 78A214BB58DF","ARC-Seal":"i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1777531374; cv=none;\n b=Harz3OXxTlUmubWSWX+qGLdnn/CSRuwyvxG2Csw+FP63HPzb/zjT/FFS7wMMU3a5ZK3rAWSHOkwGTjCHlmNUyHrJ2er0R2pxqIq0CHp4ax1TpbU6XFHgJXhq8Y3te7ySu5L0xFKmSkFo5YT8pFokTsvkXtEwfQilT/iT9Tq258A=","ARC-Message-Signature":"i=1; a=rsa-sha256; d=sourceware.org; s=key;\n t=1777531374; c=relaxed/simple;\n bh=WZUlGxDeKniXPj1nPiXc8ZGNdtV6PppLakCTX3ovgnI=;\n h=DKIM-Signature:Date:From:To:Subject:Message-ID:MIME-Version;\n b=xNxoqP03eSTfdk8sPmXvYd0gWut86gsm/qCcDpfigsIXcCw2YXbQLxObme3CDReVJJZjTrUYXO/AQY7ZTqU4UDs9lGN5How9w05RcP9ae+tFBnGAsjuqXGRjbuhkKWV6Nd2Jqu3c2wSZRZjoPizUvzalMDYyqh3v8Y4V8hRlkSA=","ARC-Authentication-Results":"i=1; server2.sourceware.org","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=\n content-type:date:from:in-reply-to:message-id:mime-version\n :references:subject:to; s=pp1; bh=R5tV9HZcOsSMmTvXa0sV+FGzHlLP7r\n OhH3gS0e04ROU=; b=oavBb2SbbN85RHWbxsMYWVjy4GiNmuGbTLzOFH6B931LX4\n xFbrx8uaKh1mCrOy9nZ0EhtFBHKSzef9DAYbcJ0+7elyGfZtZRGwmgF2xyQah94E\n NehP0IJ2BOyoNKGMhgbA8XU9Mv53Y3VC7KpitysErPQe/31tM7Cytd6T3pCz7hxD\n LoisTQqkeGRZAQdUHUW9ls4/X9DE+kLZ1cLpLlkQR6KRnxIB9OU3wBvKOUPIEfZ3\n UUIo/71GnjZqrFZkkJqb/U2fCh2JiDk747MENeL5uXBPesifqYlErvo+0L6dHh/3\n LTx5OLf3WuqCPJuM46KfDXE4x77l95LxLMfAacxw==","Date":"Thu, 30 Apr 2026 02:42:46 -0400","From":"Michael Meissner ","To":"Michael Meissner , gcc-patches@gcc.gnu.org,\n Segher Boessenkool ,\n jeevitha ,\n Surya Kumari Jangala ,\n Kishan Parmar ,\n Avinash Jayakar ,\n Ayappan Perumal ,\n Juergen Christ ","Subject":"GCC 17, PowerPC Dense Math V7 (patch 4/7) -- Implement 512-bit dense\n math registers","Message-ID":"","Mail-Followup-To":"Michael Meissner ,\n gcc-patches@gcc.gnu.org,\n Segher Boessenkool ,\n jeevitha ,\n Surya Kumari Jangala ,\n Kishan Parmar ,\n Avinash Jayakar ,\n Ayappan Perumal ,\n Juergen Christ ","References":"","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"","X-TM-AS-GCONF":"00","X-Proofpoint-ORIG-GUID":"xEpYqLeWD8MoVTIJEbMfsNU5liTB0jPc","X-Authority-Analysis":"v=2.4 cv=Ft81OWrq c=1 sm=1 tr=0 ts=69f2f9ed cx=c_pps\n a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17\n a=kj9zAlcOel0A:10 a=A5OVakUREuEA:10 a=VkNPw1HP01LnGYTKEx00:22\n a=RnoormkPH1_aCDwRdu11:22 a=iQ6ETzBq9ecOQQE5vZCe:22 a=mDV3o1hIAAAA:8\n a=VnNF1IyMAAAA:8 a=yxP7ygqrNY9PZ9Y4sR8A:9 a=3ZKOabzyN94A:10 a=CjuIK1q_8ugA:10","X-Proofpoint-GUID":"xEpYqLeWD8MoVTIJEbMfsNU5liTB0jPc","X-Proofpoint-Spam-Details-Enc":"AW1haW4tMjYwNDMwMDA2MCBTYWx0ZWRfX6ZkZRu1hIorP\n FhR18hfKR6R2wAkLy72HeSeC62lvtSN1FUrYDyBxNXH/lAJWXRInjNkBxg+0T36RpKynPTaQsv5\n OKsLUxdlJ0vxcAUZBzrhj/Uo9BE1IpgE6HpiyTZHB9GbSiQuVPRPq+7RliSGNsR2jtVtB6EElLR\n TAAzv48FQ+0ngRH9CuvPY+IADCCkZBkBpHDK1K2KM8l6JZFLBpOtBjwANgxe9NcU1PC2PoQzm+a\n FqtQprUhdtzbisvi5ycHZhCbSIvZte49PBUf5jCZEy42aklicAZ58qVpESeV03acPhL09rGZ+gB\n D2kslUmdSABQSqzc8dctlv9OI5G93uGGYSlBY6Rv5vnOGepesu80VJLG6uLFERl2oCOaMnETWOC\n A1Aq9XmEl/qa9r2dZdGbXYERHBO6XpjShywTw2xXeB5mj5I80+RZFW4CFJrq+20C9CeDYKpV/gd\n ETnJE0fSnlL3kcJTZpw==","X-Proofpoint-Virus-Version":"vendor=baseguard\n engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49\n definitions=2026-04-30_02,2026-04-28_01,2025-10-01_01","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n priorityscore=1501 lowpriorityscore=0 bulkscore=0 spamscore=0 impostorscore=0\n clxscore=1015 malwarescore=0 phishscore=0 suspectscore=0 adultscore=0\n classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0\n reason=mlx scancount=1 engine=8.22.0-2604200000 definitions=main-2604300060","X-BeenThere":"gcc-patches@gcc.gnu.org","X-Mailman-Version":"2.1.30","Precedence":"list","List-Id":"Gcc-patches mailing list ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org"},"content":"From 1b4e8aac6693711d2acc1c6b1a9b1b8388dc6042 Mon Sep 17 00:00:00 2001\nFrom: Michael Meissner \nDate: Wed, 29 Apr 2026 23:08:21 -0400\nSubject: [PATCH 104/107] Add support for dense math registers.\n\nThis is part four of the dense math register patches for the PowerPC.\nThis is the 7th version of the dense math patches.\n\nVersion 6 of the dense math register patches were posted on April 21st,\n2026.\n\n * https://gcc.gnu.org/pipermail/gcc-patches/2026-April/713352.html\n * https://gcc.gnu.org/pipermail/gcc-patches/2026-April/713353.html\n * https://gcc.gnu.org/pipermail/gcc-patches/2026-April/713354.html\n * https://gcc.gnu.org/pipermail/gcc-patches/2026-April/713355.html\n * https://gcc.gnu.org/pipermail/gcc-patches/2026-April/713356.html\n * https://gcc.gnu.org/pipermail/gcc-patches/2026-April/713357.html\n\nThis patch needs the -mcpu=future patch posted on April 8th, 2026:\n\n * https://gcc.gnu.org/pipermail/gcc-patches/2026-April/712532.html\n\nThis patch (#4) combines version 6 patch #3 (which adds the basic dense math\nregister support) and the parts of patch #4 (switch mmf.md to use the new wD\nconstraint, use accumulator_operand, and add dense math zero/move support) that\nweren't previously added in version 7 patch #1. Here are the changes from\nversion 6 of the patches:\n\n\tdense_math_operand\t\t\tto dmf_register_operand\n\tFIRST_DM_REGNO\t\t\t\tto FIRST_DMF_REGNO\n\tLAST_DM_REGNO\t\t\t\tto LAST_DMF_REGNO\n\tUNITS_PER_DM_WORD\t\t\tto UNITS_PER_DMF_WORD\n\tDM_REGNO_P\t\t\t\tto DMF_REGNO_P\n\tDM_REGS\t\t\t\t\tto DMF_REGS\n\tDM_REG_TYPE\t\t\t\tto DMF_REG_TYPE\n\trs6000_dense_math_register_move_cost\tto rs6000_dmf_register_move_cost\n\nI have built bootstrap little endian compilers on power10 systems, and\nbig endian compiler on power9 systems. There were no regression in the\ntests. Can I add the patches to the GCC trunk after the -mcpu=future\npatch is applied and GCC 17 has opened up?\n\n2026-04-29 Michael Meissner \n\ngcc/\n\n\t* config/rs6000/mma.md (movoo): Allow -mdense-math -mno-mma.\n\t(movxo): Convert to being a define_expand that can handle both the\n\toriginal MMA support without dense math registers, and adding dense math\n\tsupport. Allow -mdense-math -mno-mma.\n\t(movxo_nodm): Rename original movxo insn, and restrict this insn to when\n\twe do not have dense math registers.\n\t(movxo_dm): New define_insn_and_split for dense math registers.\n\t(vsx_assemble_pair): Allow -mdense-math -mno-mma.\n\t(vsx_disassemble_pair): Likewise.\n\t(mma_assemble_acc): Likewise.\n\t(mma_disassemble_acc): Likewise.\n\t(mma_): Allow built-ins to be used if -mdense-math.\n\t(mma_xxsetaccz): Convert into a define_expand to handle both non-dense\n\tmath and dense math registers.\n\t(mma_xxsetaccz_nodm): Rename from mma_xxsetaccz and limit code to non\n\tdense math systems.\n\t(mma_xxsetaccz_dm): New insn for direct math register support.\n\t* config/rs6000/predicates.md (dmf_register_operand): New predicate.\n\t(accumulator_operand): Add support for dense math registers.\n\t* config/rs6000/rs6000-builtin.cc (rs6000_gimple_fold_mma_builtin): Do\n\tnot issue xxmfacc (deprime) instruction if we have dense math registers.\n\t* config/rs6000/rs6000-cpus.def (FUTURE_MASKS_SERVER): Add -mdense-math.\n\t(POWERPC_MASKS): Likewise.\n\t* config/rs6000/rs6000.cc (enum rs6000_reg_type): Add dense math\n\tregister support.\n\t(enum rs6000_reload_reg_typ): Likewise.\n\t(LAST_RELOAD_REG_CLASS): Likewise.\n\t(reload_reg_map): Likewise.\n\t(rs6000_reg_names): Likewise.\n\t(alt_reg_names): Likewise.\n\t(rs6000_hard_regno_nregs_internal): Likewise.\n\t(rs6000_hard_regno_mode_ok_uncached): Likewise.\n\t(rs6000_debug_reg_global): Likewise.\n\t(rs6000_setup_reg_addr_masks): Likewise.\n\t(rs6000_init_hard_regno_mode_ok): Likewise.\n\t(rs6000_secondary_reload_memory): Likewise.\n\t(rs6000_secondary_reload_simple_move): Likewise.\n\t(rs6000_preferred_reload_class): Likewise.\n\t(rs6000_secondary_reload_class): Likewise.\n\t(print_operand): Likewise.\n\t(rs6000_dmf_register_move_cost): New helper function.\n\t(rs6000_register_move_cost): Add dense math register support.\n\t(rs6000_memory_move_cost): Likewise.\n\t(rs6000_compute_pressure_classes): Likewise.\n\t(rs6000_debugger_regno): Likewise.\n\t(rs6000_opt_masks): Likewise.\n\t(rs6000_split_multireg_move): Likewise.\n\t* config/rs6000/rs6000.h (UNITS_PER_DMF_WORD): New macro.\n\t(FIRST_PSEUDO_REGISTER): Add dense math register support.\n\t(FIXED_REGISTERS): Likewise.\n\t(CALL_REALLY_USED_REGISTERS): Likewise.\n\t(REG_ALLOC_ORDER): Likewise.\n\t(DMF_REGNO_P): New macro.\n\t(enum reg_class): Add dense math register support.\n\t(REG_CLASS_NAMES): Likewise.\n\t(REGISTER_NAMES): Likewise.\n\t(ADDITIONAL_REGISTER_NAMES): Likewise.\n\t* config/rs6000/rs6000.md (FIRST_DMF_REGNO): New constant.\n\t(LAST_DMF_REGNO): Likewise.\n---\n gcc/config/rs6000/mma.md | 124 +++++++++++++---\n gcc/config/rs6000/predicates.md | 29 +++-\n gcc/config/rs6000/rs6000-builtin.cc | 13 +-\n gcc/config/rs6000/rs6000-cpus.def | 2 +\n gcc/config/rs6000/rs6000.cc | 214 ++++++++++++++++++++++------\n gcc/config/rs6000/rs6000.h | 37 ++++-\n gcc/config/rs6000/rs6000.md | 2 +\n 7 files changed, 347 insertions(+), 74 deletions(-)","diff":"diff --git a/gcc/config/rs6000/mma.md b/gcc/config/rs6000/mma.md\nindex 87139dd41e8..c017b7ca1e7 100644\n--- a/gcc/config/rs6000/mma.md\n+++ b/gcc/config/rs6000/mma.md\n@@ -91,6 +91,7 @@ (define_c_enum \"unspec\"\n UNSPEC_MMA_XVI8GER4SPP\n UNSPEC_MMA_XXMFACC\n UNSPEC_MMA_XXMTACC\n+ UNSPEC_MMA_DMSETDMRZ\n ])\n \n (define_c_enum \"unspecv\"\n@@ -270,7 +271,7 @@ (define_expand \"movoo\"\n \t(match_operand:OO 1 \"input_operand\"))]\n \"\"\n {\n- if (TARGET_MMA)\n+ if (TARGET_MMA || TARGET_DENSE_MATH)\n {\n rs6000_emit_move (operands[0], operands[1], OOmode);\n DONE;\n@@ -295,7 +296,7 @@ (define_expand \"movoo\"\n (define_insn_and_split \"*movoo\"\n [(set (match_operand:OO 0 \"nonimmediate_operand\" \"=wa,ZwO,wa\")\n \t(match_operand:OO 1 \"input_operand\" \"ZwO,wa,wa\"))]\n- \"TARGET_MMA\n+ \"(TARGET_MMA || TARGET_DENSE_MATH)\n && (gpc_reg_operand (operands[0], OOmode)\n || gpc_reg_operand (operands[1], OOmode))\"\n \"@\n@@ -314,13 +315,13 @@ (define_insn_and_split \"*movoo\"\n (set_attr \"length\" \"*,*,8\")])\n \n \f\n-;; Vector quad support. XOmode can only live in FPRs.\n+;; Vector quad support.\n (define_expand \"movxo\"\n [(set (match_operand:XO 0 \"nonimmediate_operand\")\n \t(match_operand:XO 1 \"input_operand\"))]\n \"\"\n {\n- if (TARGET_MMA)\n+ if (TARGET_MMA || TARGET_DENSE_MATH)\n {\n rs6000_emit_move (operands[0], operands[1], XOmode);\n DONE;\n@@ -339,10 +340,13 @@ (define_expand \"movxo\"\n gcc_assert (false);\n })\n \n-(define_insn_and_split \"*movxo\"\n+;; If we do not have dense math registers, XOmode can only live in FPR\n+;; registers (0..31).\n+\n+(define_insn_and_split \"*movxo_nodm\"\n [(set (match_operand:XO 0 \"nonimmediate_operand\" \"=d,ZwO,d\")\n \t(match_operand:XO 1 \"input_operand\" \"ZwO,d,d\"))]\n- \"TARGET_MMA\n+ \"TARGET_MMA_NO_DENSE_MATH\n && (gpc_reg_operand (operands[0], XOmode)\n || gpc_reg_operand (operands[1], XOmode))\"\n \"@\n@@ -359,11 +363,39 @@ (define_insn_and_split \"*movxo\"\n (set_attr \"length\" \"*,*,16\")\n (set_attr \"max_prefixed_insns\" \"2,2,*\")])\n \n+;; If dense math registers are available, XOmode can live in either VSX\n+;; registers (0..63) or dense math registers.\n+\n+(define_insn_and_split \"*movxo_dm\"\n+ [(set (match_operand:XO 0 \"nonimmediate_operand\" \"=wa,ZwO,wa,wD,wD,wa\")\n+\t(match_operand:XO 1 \"input_operand\" \"ZwO,wa, wa,wa,wD,wD\"))]\n+ \"TARGET_DENSE_MATH\n+ && (gpc_reg_operand (operands[0], XOmode)\n+ || gpc_reg_operand (operands[1], XOmode))\"\n+ \"@\n+ #\n+ #\n+ #\n+ dmxxinstdmr512 %0,%1,%Y1,0\n+ dmmr %0,%1\n+ dmxxextfdmr512 %0,%Y0,%1,0\"\n+ \"&& reload_completed\n+ && !dmf_register_operand (operands[0], XOmode)\n+ && !dmf_register_operand (operands[1], XOmode)\"\n+ [(const_int 0)]\n+{\n+ rs6000_split_multireg_move (operands[0], operands[1]);\n+ DONE;\n+}\n+ [(set_attr \"type\" \"vecload,vecstore,veclogical,mma,mma,mma\")\n+ (set_attr \"length\" \"*,*,16,*,*,*\")\n+ (set_attr \"max_prefixed_insns\" \"2,2,*,*,*,*\")])\n+\n (define_expand \"vsx_assemble_pair\"\n [(match_operand:OO 0 \"vsx_register_operand\")\n (match_operand:V16QI 1 \"mma_assemble_input_operand\")\n (match_operand:V16QI 2 \"mma_assemble_input_operand\")]\n- \"TARGET_MMA\"\n+ \"TARGET_MMA || TARGET_DENSE_MATH\"\n {\n rtx src = gen_rtx_UNSPEC (OOmode,\n \t\t\t gen_rtvec (2, operands[1], operands[2]),\n@@ -380,7 +412,7 @@ (define_insn_and_split \"*vsx_assemble_pair\"\n \t(unspec:OO [(match_operand:V16QI 1 \"mma_assemble_input_operand\" \"mwa\")\n \t\t (match_operand:V16QI 2 \"mma_assemble_input_operand\" \"mwa\")]\n \t\t UNSPEC_VSX_ASSEMBLE))]\n- \"TARGET_MMA\"\n+ \"TARGET_MMA || TARGET_DENSE_MATH\"\n \"#\"\n \"&& reload_completed\"\n [(const_int 0)]\n@@ -396,7 +428,7 @@ (define_expand \"vsx_disassemble_pair\"\n [(match_operand:V16QI 0 \"mma_disassemble_output_operand\")\n (match_operand:OO 1 \"vsx_register_operand\")\n (match_operand 2 \"const_0_to_1_operand\")]\n- \"TARGET_MMA\"\n+ \"TARGET_MMA || TARGET_DENSE_MATH\"\n {\n rtx src;\n int regoff = INTVAL (operands[2]);\n@@ -412,7 +444,7 @@ (define_insn_and_split \"*vsx_disassemble_pair\"\n (unspec:V16QI [(match_operand:OO 1 \"vsx_register_operand\" \"wa\")\n (match_operand 2 \"const_0_to_1_operand\")]\n UNSPEC_MMA_EXTRACT))]\n- \"TARGET_MMA\n+ \"(TARGET_MMA || TARGET_DENSE_MATH)\n && vsx_register_operand (operands[1], OOmode)\"\n \"#\"\n \"&& reload_completed\"\n@@ -431,7 +463,7 @@ (define_expand \"mma_assemble_acc\"\n (match_operand:V16QI 2 \"mma_assemble_input_operand\")\n (match_operand:V16QI 3 \"mma_assemble_input_operand\")\n (match_operand:V16QI 4 \"mma_assemble_input_operand\")]\n- \"TARGET_MMA\"\n+ \"TARGET_MMA || TARGET_DENSE_MATH\"\n {\n rtx src = gen_rtx_UNSPEC_VOLATILE (XOmode,\n \t\t\t \t gen_rtvec (4, operands[1], operands[2],\n@@ -452,7 +484,7 @@ (define_insn_and_split \"*mma_assemble_acc\"\n \t (match_operand:V16QI 3 \"mma_assemble_input_operand\" \"mwa\")\n \t (match_operand:V16QI 4 \"mma_assemble_input_operand\" \"mwa\")]\n \t UNSPECV_MMA_ASSEMBLE))]\n- \"TARGET_MMA\n+ \"(TARGET_MMA || TARGET_DENSE_MATH)\n && fpr_reg_operand (operands[0], XOmode)\"\n \"#\"\n \"&& reload_completed\"\n@@ -470,7 +502,7 @@ (define_expand \"mma_disassemble_acc\"\n [(match_operand:V16QI 0 \"mma_disassemble_output_operand\")\n (match_operand:XO 1 \"fpr_reg_operand\")\n (match_operand 2 \"const_0_to_3_operand\")]\n- \"TARGET_MMA\"\n+ \"TARGET_MMA || TARGET_DENSE_MATH\"\n {\n rtx src;\n int regoff = INTVAL (operands[2]);\n@@ -486,7 +518,7 @@ (define_insn_and_split \"*mma_disassemble_acc\"\n (unspec:V16QI [(match_operand:XO 1 \"fpr_reg_operand\" \"d\")\n (match_operand 2 \"const_0_to_3_operand\")]\n UNSPEC_MMA_EXTRACT))]\n- \"TARGET_MMA\n+ \"(TARGET_MMA || TARGET_DENSE_MATH)\n && fpr_reg_operand (operands[1], XOmode)\"\n \"#\"\n \"&& reload_completed\"\n@@ -499,31 +531,79 @@ (define_insn_and_split \"*mma_disassemble_acc\"\n DONE;\n })\n \n-;; MMA instructions that do not use their accumulators as an input, still\n-;; must not allow their vector operands to overlap the registers used by\n-;; the accumulator. We enforce this by marking the output as early clobber.\n+;; If dense math registers are not available, MMA instructions that do\n+;; not use their accumulators that overlap with FPR registers as an\n+;; input, still must not allow their vector operands to overlap the\n+;; registers used by the accumulator. We enforce this by marking the\n+;; output as early clobber. The prime and de-prime instructions are\n+;; not needed on systems with dense math registers.\n+\n+;; If we have dense math registers, xxmfacc/xxmtacc become normal moves.\n \n-(define_insn \"mma_\"\n+(define_expand \"mma_\"\n [(set (match_operand:XO 0 \"fpr_reg_operand\" \"=&d\")\n \t(unspec:XO [(match_operand:XO 1 \"fpr_reg_operand\" \"0\")]\n \t\t MMA_ACC))]\n \"TARGET_MMA\"\n+{\n+ if (TARGET_DENSE_MATH)\n+ {\n+ emit_move_insn (operands[0], operands[1]);\n+ DONE;\n+ }\n+})\n+\n+(define_insn \"*mma_\"\n+ [(set (match_operand:XO 0 \"fpr_reg_operand\" \"=&d\")\n+\t(unspec:XO [(match_operand:XO 1 \"fpr_reg_operand\" \"0\")]\n+\t\t MMA_ACC))]\n+ \"TARGET_MMA_NO_DENSE_MATH\"\n \" %A0\"\n [(set_attr \"type\" \"mma\")])\n \n ;; We can't have integer constants in XOmode so we wrap this in an\n-;; UNSPEC_VOLATILE.\n+;; UNSPEC_VOLATILE. If we have dense math registers, we can just use a\n+;; normal UNSPEC instead of UNSPEC_VOLATILE. An UNSPEC_VOLATILE limits\n+;; optimization since insns can't be moved before the UNSPEC_VOLATILE.\n \n-(define_insn \"mma_xxsetaccz\"\n+;; Instructions: xxmtacc and xxmfacc\n+\n+(define_expand \"mma_xxsetaccz\"\n+ [(set (match_operand:XO 0 \"accumulator_operand\")\n+\t(unspec_volatile:XO [(const_int 0)]\n+\t\t\t UNSPECV_MMA_XXSETACCZ))]\n+ \"TARGET_MMA || TARGET_DENSE_MATH\"\n+{\n+ if (TARGET_DENSE_MATH)\n+ {\n+ emit_insn (gen_mma_xxsetaccz_dm (operands[0]));\n+ DONE;\n+ }\n+})\n+\n+;; Clear accumulator without dense math registers\n+(define_insn \"*mma_xxsetaccz_nodm\"\n [(set (match_operand:XO 0 \"fpr_reg_operand\" \"=d\")\n \t(unspec_volatile:XO [(const_int 0)]\n \t\t\t UNSPECV_MMA_XXSETACCZ))]\n- \"TARGET_MMA\"\n+ \"TARGET_MMA_NO_DENSE_MATH\"\n \"xxsetaccz %A0\"\n [(set_attr \"type\" \"mma\")])\n \n+;; Clear accumulator when dense math registers are available.\n+(define_insn \"mma_xxsetaccz_dm\"\n+ [(set (match_operand:XO 0 \"accumulator_operand\" \"=wD\")\n+\t(unspec [(const_int 0)]\n+\t\tUNSPEC_MMA_DMSETDMRZ))]\n+ \"TARGET_DENSE_MATH\"\n+ \"dmsetdmrz %A0\"\n+ [(set_attr \"type\" \"mma\")])\n+\n \f\n-;; MMA operations below.\n+;; MMA operations below. If dense math registers are available, these\n+;; operations will use the 8 accumultors which are separate registers.\n+;; If dense math registers are not available, these operations will use\n+;; accumulators that are overlaid on top of the FPR registers.\n \n ;; Instructions:\n ;; xvi4ger8 xvi8ger4 xvi16ger2 xvi16ger2s xvf16ger2\ndiff --git a/gcc/config/rs6000/predicates.md b/gcc/config/rs6000/predicates.md\nindex 682fd2dc6e8..737eafc2bc5 100644\n--- a/gcc/config/rs6000/predicates.md\n+++ b/gcc/config/rs6000/predicates.md\n@@ -186,8 +186,29 @@ (define_predicate \"vlogical_operand\"\n return VLOGICAL_REGNO_P (REGNO (op));\n })\n \n-;; Return 1 if op is an accumulator. On power10 systems, the accumulators\n-;; overlap with the FPRs.\n+;; Return 1 if op is a dense math register\n+(define_predicate \"dmf_register_operand\"\n+ (match_operand 0 \"register_operand\")\n+{\n+ if (!TARGET_DENSE_MATH)\n+ return 0;\n+\n+ if (!REG_P (op))\n+ return 0;\n+\n+ if (!HARD_REGISTER_P (op))\n+ return 1;\n+\n+ return DMF_REGNO_P (REGNO (op));\n+})\n+\n+;; Return 1 if op is an accumulator.\n+;;\n+;; On power10 and power11 systems, the accumulators overlap with the\n+;; FPRs and the register must be divisible by 4.\n+;;\n+;; On systems with dense math registers, the accumulators are separate\n+;; registers and do not overlap with the FPR registers.\n (define_predicate \"accumulator_operand\"\n (match_operand 0 \"register_operand\")\n {\n@@ -201,7 +222,9 @@ (define_predicate \"accumulator_operand\"\n return 1;\n \n int r = REGNO (op);\n- return FP_REGNO_P (r) && (r & 3) == 0;\n+ return (TARGET_DENSE_MATH\n+\t ? DMF_REGNO_P (r)\n+\t : FP_REGNO_P (r) && (r & 3) == 0);\n })\n \n ;; Return 1 if op is the carry register.\ndiff --git a/gcc/config/rs6000/rs6000-builtin.cc b/gcc/config/rs6000/rs6000-builtin.cc\nindex 4d0e541351f..de9f4b519f1 100644\n--- a/gcc/config/rs6000/rs6000-builtin.cc\n+++ b/gcc/config/rs6000/rs6000-builtin.cc\n@@ -1129,9 +1129,16 @@ rs6000_gimple_fold_mma_builtin (gimple_stmt_iterator *gsi,\n \t return true;\n \t}\n \n- /* If we're disassembling an accumulator into a different type, we need\n-\t to emit a xxmfacc instruction now, since we cannot do it later. */\n- if (fncode == RS6000_BIF_DISASSEMBLE_ACC)\n+ /* On a system without dense math registers, if we're disassembling an\n+\t accumulator into a different type, we need to emit a xxmfacc\n+\t instruction now, since we cannot do it later.\n+\n+\t On a dense math system, we use the explicit moves (dmxxinstdmr512 and\n+\t dmxxextfdmr512) to move values between VSX registers and the separate\n+\t dense math accumulators. This means, we don't have to sprinkle\n+\t xxmfacc's into the code to make sure the overlaid accumulators are\n+\t ready to issue MMA instructions. */\n+ if (fncode == RS6000_BIF_DISASSEMBLE_ACC && !TARGET_DENSE_MATH)\n \t{\n \t new_decl = rs6000_builtin_decls[RS6000_BIF_XXMFACC_INTERNAL];\n \t new_call = gimple_build_call (new_decl, 1, src);\ndiff --git a/gcc/config/rs6000/rs6000-cpus.def b/gcc/config/rs6000/rs6000-cpus.def\nindex a110860acce..38e6bc880b2 100644\n--- a/gcc/config/rs6000/rs6000-cpus.def\n+++ b/gcc/config/rs6000/rs6000-cpus.def\n@@ -85,6 +85,7 @@\n \n /* -mcpu=future flags. */\n #define FUTURE_MASKS_SERVER\t(POWER11_MASKS_SERVER\t\t\t\\\n+\t\t\t\t | OPTION_MASK_DENSE_MATH\t\t\\\n \t\t\t\t | OPTION_MASK_FUTURE)\n \n /* Flags that need to be turned off if -mno-vsx. */\n@@ -117,6 +118,7 @@\n #define POWERPC_MASKS\t\t(OPTION_MASK_ALTIVEC\t\t\t\\\n \t\t\t\t | OPTION_MASK_CMPB\t\t\t\\\n \t\t\t\t | OPTION_MASK_CRYPTO\t\t\t\\\n+\t\t\t\t | OPTION_MASK_DENSE_MATH\t\t\\\n \t\t\t\t | OPTION_MASK_DFP\t\t\t\\\n \t\t\t\t | OPTION_MASK_DLMZB\t\t\t\\\n \t\t\t\t | OPTION_MASK_EFFICIENT_UNALIGNED_VSX\t\\\ndiff --git a/gcc/config/rs6000/rs6000.cc b/gcc/config/rs6000/rs6000.cc\nindex 31ab5bdccf1..745d57b1d76 100644\n--- a/gcc/config/rs6000/rs6000.cc\n+++ b/gcc/config/rs6000/rs6000.cc\n@@ -292,7 +292,8 @@ enum rs6000_reg_type {\n ALTIVEC_REG_TYPE,\n FPR_REG_TYPE,\n SPR_REG_TYPE,\n- CR_REG_TYPE\n+ CR_REG_TYPE,\n+ DMF_REG_TYPE\n };\n \n /* Map register class to register type. */\n@@ -306,22 +307,24 @@ static enum rs6000_reg_type reg_class_to_reg_type[N_REG_CLASSES];\n \n \n /* Register classes we care about in secondary reload or go if legitimate\n- address. We only need to worry about GPR, FPR, and Altivec registers here,\n- along an ANY field that is the OR of the 3 register classes. */\n+ address. We only need to worry about GPR, FPR, Altivec, and dense math\n+ registers here, along an ANY field that is the OR of the 4 register\n+ classes. */\n \n enum rs6000_reload_reg_type {\n RELOAD_REG_GPR,\t\t\t/* General purpose registers. */\n RELOAD_REG_FPR,\t\t\t/* Traditional floating point regs. */\n RELOAD_REG_VMX,\t\t\t/* Altivec (VMX) registers. */\n- RELOAD_REG_ANY,\t\t\t/* OR of GPR, FPR, Altivec masks. */\n+ RELOAD_REG_DMR,\t\t\t/* Dense math registers. */\n+ RELOAD_REG_ANY,\t\t\t/* OR of GPR/FPR/VMX/DMR masks. */\n N_RELOAD_REG\n };\n \n-/* For setting up register classes, loop through the 3 register classes mapping\n+/* For setting up register classes, loop through the 4 register classes mapping\n into real registers, and skip the ANY class, which is just an OR of the\n bits. */\n #define FIRST_RELOAD_REG_CLASS\tRELOAD_REG_GPR\n-#define LAST_RELOAD_REG_CLASS\tRELOAD_REG_VMX\n+#define LAST_RELOAD_REG_CLASS\tRELOAD_REG_DMR\n \n /* Map reload register type to a register in the register class. */\n struct reload_reg_map_type {\n@@ -333,6 +336,7 @@ static const struct reload_reg_map_type reload_reg_map[N_RELOAD_REG] = {\n { \"Gpr\",\tFIRST_GPR_REGNO },\t/* RELOAD_REG_GPR. */\n { \"Fpr\",\tFIRST_FPR_REGNO },\t/* RELOAD_REG_FPR. */\n { \"VMX\",\tFIRST_ALTIVEC_REGNO },\t/* RELOAD_REG_VMX. */\n+ { \"Dmr\",\tFIRST_DMF_REGNO },\t/* RELOAD_REG_DMR. */\n { \"Any\",\t-1 },\t\t\t/* RELOAD_REG_ANY. */\n };\n \n@@ -1226,6 +1230,8 @@ char rs6000_reg_names[][8] =\n \"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\",\n /* vrsave vscr sfp */\n \"vrsave\", \"vscr\", \"sfp\",\n+ /* dense math registers. */\n+ \"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\",\n };\n \n #ifdef TARGET_REGNAMES\n@@ -1252,6 +1258,8 @@ static const char alt_reg_names[][8] =\n \"%cr0\", \"%cr1\", \"%cr2\", \"%cr3\", \"%cr4\", \"%cr5\", \"%cr6\", \"%cr7\",\n /* vrsave vscr sfp */\n \"vrsave\", \"vscr\", \"sfp\",\n+ /* dense math registers. */\n+ \"%dmr0\", \"%dmr1\", \"%dmr2\", \"%dmr3\", \"%dmr4\", \"%dmr5\", \"%dmr6\", \"%dmr7\",\n };\n #endif\n \n@@ -1842,6 +1850,9 @@ rs6000_hard_regno_nregs_internal (int regno, machine_mode mode)\n else if (ALTIVEC_REGNO_P (regno))\n reg_size = UNITS_PER_ALTIVEC_WORD;\n \n+ else if (DMF_REGNO_P (regno))\n+ reg_size = UNITS_PER_DMF_WORD;\n+\n else\n reg_size = UNITS_PER_WORD;\n \n@@ -1863,9 +1874,32 @@ rs6000_hard_regno_mode_ok_uncached (int regno, machine_mode mode)\n if (mode == OOmode)\n return (TARGET_MMA && VSX_REGNO_P (regno) && (regno & 1) == 0);\n \n- /* MMA accumulator modes need FPR registers divisible by 4. */\n+ /* On ISA 3.1 (power10), MMA accumulator modes need FPR registers divisible\n+ by 4.\n+\n+ If dense math registers are enabled, we can allow all VSX registers plus\n+ the dense math registers. VSX registers are used to load and store the\n+ registers as the accumulator registers do not have load and store\n+ instructions. Because we just use the VSX registers for load/store\n+ operations, we just need to make sure load vector pair and store vector\n+ pair instructions can be used. */\n if (mode == XOmode)\n- return (TARGET_MMA && FP_REGNO_P (regno) && (regno & 3) == 0);\n+ {\n+ if (!TARGET_DENSE_MATH)\n+\treturn (FP_REGNO_P (regno) && (regno & 3) == 0);\n+\n+ else if (DMF_REGNO_P (regno))\n+\treturn 1;\n+\n+ else\n+\treturn (VSX_REGNO_P (regno)\n+\t\t&& VSX_REGNO_P (last_regno)\n+\t\t&& (regno & 1) == 0);\n+ }\n+\n+ /* No other types other than XOmode can go in dense math registers. */\n+ if (DMF_REGNO_P (regno))\n+ return 0;\n \n /* PTImode can only go in GPRs. Quad word memory operations require even/odd\n register combinations, and use PTImode where we need to deal with quad\n@@ -2308,6 +2342,7 @@ rs6000_debug_reg_global (void)\n rs6000_debug_reg_print (FIRST_ALTIVEC_REGNO,\n \t\t\t LAST_ALTIVEC_REGNO,\n \t\t\t \"vs\");\n+ rs6000_debug_reg_print (FIRST_DMF_REGNO, LAST_DMF_REGNO, \"dense_math\");\n rs6000_debug_reg_print (LR_REGNO, LR_REGNO, \"lr\");\n rs6000_debug_reg_print (CTR_REGNO, CTR_REGNO, \"ctr\");\n rs6000_debug_reg_print (CR0_REGNO, CR7_REGNO, \"cr\");\n@@ -2634,6 +2669,21 @@ rs6000_setup_reg_addr_masks (void)\n \t addr_mask = 0;\n \t reg = reload_reg_map[rc].reg;\n \n+\t /* Special case dense math registers. */\n+\t if (rc == RELOAD_REG_DMR)\n+\t {\n+\t if (TARGET_DENSE_MATH && m2 == XOmode)\n+\t\t{\n+\t\t addr_mask = RELOAD_REG_VALID;\n+\t\t reg_addr[m].addr_mask[rc] = addr_mask;\n+\t\t any_addr_mask |= addr_mask;\n+\t\t}\n+\t else\n+\t\treg_addr[m].addr_mask[rc] = 0;\n+\n+\t continue;\n+\t }\n+\n \t /* Can mode values go in the GPR/FPR/Altivec registers? */\n \t if (reg >= 0 && rs6000_hard_regno_mode_ok_p[m][reg])\n \t {\n@@ -2784,6 +2834,9 @@ rs6000_init_hard_regno_mode_ok (bool global_init_p)\n for (r = CR1_REGNO; r <= CR7_REGNO; ++r)\n rs6000_regno_regclass[r] = CR_REGS;\n \n+ for (r = FIRST_DMF_REGNO; r <= LAST_DMF_REGNO; ++r)\n+ rs6000_regno_regclass[r] = DMF_REGS;\n+\n rs6000_regno_regclass[LR_REGNO] = LINK_REGS;\n rs6000_regno_regclass[CTR_REGNO] = CTR_REGS;\n rs6000_regno_regclass[CA_REGNO] = NO_REGS;\n@@ -2808,6 +2861,7 @@ rs6000_init_hard_regno_mode_ok (bool global_init_p)\n reg_class_to_reg_type[(int)LINK_OR_CTR_REGS] = SPR_REG_TYPE;\n reg_class_to_reg_type[(int)CR_REGS] = CR_REG_TYPE;\n reg_class_to_reg_type[(int)CR0_REGS] = CR_REG_TYPE;\n+ reg_class_to_reg_type[(int)DMF_REGS] = DMF_REG_TYPE;\n \n if (TARGET_VSX)\n {\n@@ -2994,8 +3048,12 @@ rs6000_init_hard_regno_mode_ok (bool global_init_p)\n if (TARGET_DIRECT_MOVE_128)\n rs6000_constraints[RS6000_CONSTRAINT_we] = VSX_REGS;\n \n- if (TARGET_MMA)\n+ /* Support for the accumulator registers, either FPR registers (aka original\n+ mma) or dense math registers. */\n+ if (TARGET_MMA_NO_DENSE_MATH)\n rs6000_constraints[RS6000_CONSTRAINT_wD] = FLOAT_REGS;\n+ else if (TARGET_MMA_DENSE_MATH)\n+ rs6000_constraints[RS6000_CONSTRAINT_wD] = DMF_REGS;\n \n /* Set up the reload helper and direct move functions. */\n if (TARGET_VSX || TARGET_ALTIVEC)\n@@ -12365,6 +12423,11 @@ rs6000_secondary_reload_memory (rtx addr,\n addr_mask = (reg_addr[mode].addr_mask[RELOAD_REG_VMX]\n \t\t & ~RELOAD_REG_AND_M16);\n \n+ /* Dense math registers use VSX registers for memory operations, and need to\n+ generate some extra instructions. */\n+ else if (rclass == DMF_REGS)\n+ return 2;\n+\n /* If the register allocator hasn't made up its mind yet on the register\n class to use, settle on defaults to use. */\n else if (rclass == NO_REGS)\n@@ -12693,6 +12756,13 @@ rs6000_secondary_reload_simple_move (enum rs6000_reg_type to_type,\n \t || (to_type == SPR_REG_TYPE && from_type == GPR_REG_TYPE)))\n return true;\n \n+ /* We can transfer between VSX registers and dense math registers without\n+ needing extra registers. */\n+ if (TARGET_DENSE_MATH && mode == XOmode\n+ && ((to_type == DMF_REG_TYPE && from_type == VSX_REG_TYPE)\n+\t || (to_type == VSX_REG_TYPE && from_type == DMF_REG_TYPE)))\n+ return true;\n+\n return false;\n }\n \n@@ -13387,6 +13457,10 @@ rs6000_preferred_reload_class (rtx x, enum reg_class rclass)\n machine_mode mode = GET_MODE (x);\n bool is_constant = CONSTANT_P (x);\n \n+ /* Dense math registers can't be loaded or stored. */\n+ if (rclass == DMF_REGS)\n+ return NO_REGS;\n+\n /* If a mode can't go in FPR/ALTIVEC/VSX registers, don't return a preferred\n reload class for it. */\n if ((rclass == ALTIVEC_REGS || rclass == VSX_REGS)\n@@ -13483,7 +13557,7 @@ rs6000_preferred_reload_class (rtx x, enum reg_class rclass)\n \treturn VSX_REGS;\n \n if (mode == XOmode)\n-\treturn FLOAT_REGS;\n+\treturn TARGET_DENSE_MATH ? VSX_REGS : FLOAT_REGS;\n \n if (GET_MODE_CLASS (mode) == MODE_INT)\n \treturn GENERAL_REGS;\n@@ -13608,6 +13682,11 @@ rs6000_secondary_reload_class (enum reg_class rclass, machine_mode mode,\n else\n regno = -1;\n \n+ /* Dense math registers don't have loads or stores. We have to go through\n+ the VSX registers to load XOmode (vector quad). */\n+ if (TARGET_DENSE_MATH && rclass == DMF_REGS)\n+ return VSX_REGS;\n+\n /* If we have VSX register moves, prefer moving scalar values between\n Altivec registers and GPR by going via an FPR (and then via memory)\n instead of reloading the secondary memory address for Altivec moves. */\n@@ -14139,8 +14218,14 @@ print_operand (FILE *file, rtx x, int code)\n \t output_operand. */\n \n case 'A':\n- /* Write the MMA accumulator number associated with VSX register X. */\n- if (!REG_P (x) || !FP_REGNO_P (REGNO (x)) || (REGNO (x) % 4) != 0)\n+ /* Write the MMA accumulator number associated with VSX register X. On\n+\t dense math systems, only allow dense math accumulators, not\n+\t accumulators overlapping with the FPR registers. */\n+ if (!REG_P (x))\n+\toutput_operand_lossage (\"invalid %%A value\");\n+ else if (TARGET_DENSE_MATH && DMF_REGNO_P (REGNO (x)))\n+\tfprintf (file, \"%d\", REGNO (x) - FIRST_DMF_REGNO);\n+ else if (!FP_REGNO_P (REGNO (x)) || (REGNO (x) % 4) != 0)\n \toutput_operand_lossage (\"invalid %%A value\");\n else\n \tfprintf (file, \"%d\", (REGNO (x) - FIRST_FPR_REGNO) / 4);\n@@ -22760,6 +22845,31 @@ rs6000_debug_address_cost (rtx x, machine_mode mode,\n }\n \n \n+/* Subroutine to determine the move cost of dense math registers. If we are\n+ moving to/from VSX_REGISTER registers, the cost is either 1 move (for\n+ 512-bit accumulators) or 2 moves (for 1,024 dense math registers). If we are\n+ moving to anything else like GPR registers, make the cost very high. */\n+\n+static int\n+rs6000_dmf_register_move_cost (machine_mode mode, reg_class_t rclass)\n+{\n+ const int reg_move_base = 2;\n+ HARD_REG_SET vsx_set = (reg_class_contents[rclass]\n+\t\t\t & reg_class_contents[VSX_REGS]);\n+\n+ if (TARGET_DENSE_MATH && !hard_reg_set_empty_p (vsx_set))\n+ {\n+ /* __vector_quad (i.e. XOmode) is tranfered in 1 instruction. */\n+ if (mode == XOmode)\n+\treturn reg_move_base;\n+\n+ else\n+\treturn reg_move_base * 2 * hard_regno_nregs (FIRST_DMF_REGNO, mode);\n+ }\n+\n+ return 1000 * 2 * hard_regno_nregs (FIRST_DMF_REGNO, mode);\n+}\n+\n /* A C expression returning the cost of moving data from a register of class\n CLASS1 to one of CLASS2. */\n \n@@ -22773,17 +22883,28 @@ rs6000_register_move_cost (machine_mode mode,\n if (TARGET_DEBUG_COST)\n dbg_cost_ctrl++;\n \n+ HARD_REG_SET to_vsx, from_vsx;\n+ to_vsx = reg_class_contents[to] & reg_class_contents[VSX_REGS];\n+ from_vsx = reg_class_contents[from] & reg_class_contents[VSX_REGS];\n+\n+ /* Special case dense math registers, that can only move to/from VSX registers. */\n+ if (from == DMF_REGS && to == DMF_REGS)\n+ ret = 2 * hard_regno_nregs (FIRST_DMF_REGNO, mode);\n+\n+ else if (from == DMF_REGS)\n+ ret = rs6000_dmf_register_move_cost (mode, to);\n+\n+ else if (to == DMF_REGS)\n+ ret = rs6000_dmf_register_move_cost (mode, from);\n+\n /* If we have VSX, we can easily move between FPR or Altivec registers,\n otherwise we can only easily move within classes.\n Do this first so we give best-case answers for union classes\n containing both gprs and vsx regs. */\n- HARD_REG_SET to_vsx, from_vsx;\n- to_vsx = reg_class_contents[to] & reg_class_contents[VSX_REGS];\n- from_vsx = reg_class_contents[from] & reg_class_contents[VSX_REGS];\n- if (!hard_reg_set_empty_p (to_vsx)\n- && !hard_reg_set_empty_p (from_vsx)\n- && (TARGET_VSX\n-\t || hard_reg_set_intersect_p (to_vsx, from_vsx)))\n+ else if (!hard_reg_set_empty_p (to_vsx)\n+\t && !hard_reg_set_empty_p (from_vsx)\n+\t && (TARGET_VSX\n+\t || hard_reg_set_intersect_p (to_vsx, from_vsx)))\n {\n int reg = FIRST_FPR_REGNO;\n if (TARGET_VSX\n@@ -22879,6 +23000,9 @@ rs6000_memory_move_cost (machine_mode mode, reg_class_t rclass,\n ret = 4 * hard_regno_nregs (32, mode);\n else if (reg_classes_intersect_p (rclass, ALTIVEC_REGS))\n ret = 4 * hard_regno_nregs (FIRST_ALTIVEC_REGNO, mode);\n+ else if (reg_classes_intersect_p (rclass, DMF_REGS))\n+ ret = (rs6000_dmf_register_move_cost (mode, VSX_REGS)\n+\t + rs6000_memory_move_cost (mode, VSX_REGS, false));\n else\n ret = 4 + rs6000_register_move_cost (mode, rclass, GENERAL_REGS);\n \n@@ -24087,6 +24211,8 @@ rs6000_compute_pressure_classes (enum reg_class *pressure_classes)\n if (TARGET_HARD_FLOAT)\n \tpressure_classes[n++] = FLOAT_REGS;\n }\n+ if (TARGET_DENSE_MATH)\n+ pressure_classes[n++] = DMF_REGS;\n pressure_classes[n++] = CR_REGS;\n pressure_classes[n++] = SPECIAL_REGS;\n \n@@ -24251,6 +24377,10 @@ rs6000_debugger_regno (unsigned int regno, unsigned int format)\n return 67;\n if (regno == 64)\n return 64;\n+ /* XXX: This is a guess. The GCC register number for FIRST_DMF_REGNO is 111,\n+ but the frame pointer regnum uses that. */\n+ if (DMF_REGNO_P (regno))\n+ return regno - FIRST_DMF_REGNO + 112;\n \n gcc_unreachable ();\n }\n@@ -27490,9 +27620,9 @@ rs6000_split_multireg_move (rtx dst, rtx src)\n \t unsigned offset = 0;\n \t unsigned size = GET_MODE_SIZE (reg_mode);\n \n-\t /* If we are reading an accumulator register, we have to\n-\t deprime it before we can access it. */\n-\t if (TARGET_MMA\n+\t /* If we are reading an accumulator register, we have to deprime it\n+\t before we can access it unless we have dense math registers. */\n+\t if (TARGET_MMA_NO_DENSE_MATH\n \t && GET_MODE (src) == XOmode && FP_REGNO_P (REGNO (src)))\n \t emit_insn (gen_mma_xxmfacc (src, src));\n \n@@ -27524,9 +27654,9 @@ rs6000_split_multireg_move (rtx dst, rtx src)\n \t emit_insn (gen_rtx_SET (dst2, src2));\n \t }\n \n-\t /* If we are writing an accumulator register, we have to\n-\t prime it after we've written it. */\n-\t if (TARGET_MMA\n+\t /* If we are writing an accumulator register, we have to prime it\n+\t after we've written it unless we have dense math registers. */\n+\t if (TARGET_MMA_NO_DENSE_MATH\n \t && GET_MODE (dst) == XOmode && FP_REGNO_P (REGNO (dst)))\n \t emit_insn (gen_mma_xxmtacc (dst, dst));\n \n@@ -27540,7 +27670,9 @@ rs6000_split_multireg_move (rtx dst, rtx src)\n \t\t || XINT (src, 1) == UNSPECV_MMA_ASSEMBLE);\n \t gcc_assert (REG_P (dst));\n \t if (GET_MODE (src) == XOmode)\n-\t gcc_assert (FP_REGNO_P (REGNO (dst)));\n+\t gcc_assert ((TARGET_DENSE_MATH\n+\t\t\t ? VSX_REGNO_P (REGNO (dst))\n+\t\t\t : FP_REGNO_P (REGNO (dst))));\n \t if (GET_MODE (src) == OOmode)\n \t gcc_assert (VSX_REGNO_P (REGNO (dst)));\n \n@@ -27593,9 +27725,9 @@ rs6000_split_multireg_move (rtx dst, rtx src)\n \t emit_insn (gen_rtx_SET (dst_i, op));\n \t }\n \n-\t /* We are writing an accumulator register, so we have to\n-\t prime it after we've written it. */\n-\t if (GET_MODE (src) == XOmode)\n+\t /* We are writing an accumulator register, so we have to prime it\n+\t after we've written it unless we have dense math registers. */\n+\t if (GET_MODE (src) == XOmode && !TARGET_DENSE_MATH)\n \t emit_insn (gen_mma_xxmtacc (dst, dst));\n \n \t return;\n@@ -27606,9 +27738,9 @@ rs6000_split_multireg_move (rtx dst, rtx src)\n \n if (REG_P (src) && REG_P (dst) && (REGNO (src) < REGNO (dst)))\n {\n- /* If we are reading an accumulator register, we have to\n-\t deprime it before we can access it. */\n- if (TARGET_MMA\n+ /* If we are reading an accumulator register, we have to deprime it\n+\t before we can access it unless we have dense math registers. */\n+ if (TARGET_MMA_NO_DENSE_MATH\n \t && GET_MODE (src) == XOmode && FP_REGNO_P (REGNO (src)))\n \temit_insn (gen_mma_xxmfacc (src, src));\n \n@@ -27634,9 +27766,9 @@ rs6000_split_multireg_move (rtx dst, rtx src)\n \t\t\t\t\t\t\t i * reg_mode_size)));\n \t}\n \n- /* If we are writing an accumulator register, we have to\n-\t prime it after we've written it. */\n- if (TARGET_MMA\n+ /* If we are writing an accumulator register, we have to prime it after\n+\t we've written it unless we have dense math registers. */\n+ if (TARGET_MMA_NO_DENSE_MATH\n \t && GET_MODE (dst) == XOmode && FP_REGNO_P (REGNO (dst)))\n \temit_insn (gen_mma_xxmtacc (dst, dst));\n }\n@@ -27771,9 +27903,9 @@ rs6000_split_multireg_move (rtx dst, rtx src)\n \t gcc_assert (rs6000_offsettable_memref_p (dst, reg_mode, true));\n \t}\n \n- /* If we are reading an accumulator register, we have to\n-\t deprime it before we can access it. */\n- if (TARGET_MMA && REG_P (src)\n+ /* If we are reading an accumulator register, we have to deprime it\n+\t before we can access it unless we have dense math registers. */\n+ if (TARGET_MMA_NO_DENSE_MATH && REG_P (src)\n \t && GET_MODE (src) == XOmode && FP_REGNO_P (REGNO (src)))\n \temit_insn (gen_mma_xxmfacc (src, src));\n \n@@ -27803,9 +27935,9 @@ rs6000_split_multireg_move (rtx dst, rtx src)\n \t\t\t\t\t\t\t j * reg_mode_size)));\n \t}\n \n- /* If we are writing an accumulator register, we have to\n-\t prime it after we've written it. */\n- if (TARGET_MMA && REG_P (dst)\n+ /* If we are writing an accumulator register, we have to prime it after\n+\t we've written it unless we have dense math registers. */\n+ if (TARGET_MMA_NO_DENSE_MATH && REG_P (dst)\n \t && GET_MODE (dst) == XOmode && FP_REGNO_P (REGNO (dst)))\n \temit_insn (gen_mma_xxmtacc (dst, dst));\n \ndiff --git a/gcc/config/rs6000/rs6000.h b/gcc/config/rs6000/rs6000.h\nindex 91e60085515..a33b482a67b 100644\n--- a/gcc/config/rs6000/rs6000.h\n+++ b/gcc/config/rs6000/rs6000.h\n@@ -659,6 +659,7 @@ extern unsigned char rs6000_recip_bits[];\n #define UNITS_PER_FP_WORD 8\n #define UNITS_PER_ALTIVEC_WORD 16\n #define UNITS_PER_VSX_WORD 16\n+#define UNITS_PER_DMF_WORD 128\n \n /* Type used for ptrdiff_t, as a string used in a declaration. */\n #define PTRDIFF_TYPE \"int\"\n@@ -772,7 +773,7 @@ enum data_align { align_abi, align_opt, align_both };\n Another pseudo (not included in DWARF_FRAME_REGISTERS) is soft frame\n pointer, which is eventually eliminated in favor of SP or FP. */\n \n-#define FIRST_PSEUDO_REGISTER 111\n+#define FIRST_PSEUDO_REGISTER 119\n \n /* Use standard DWARF numbering for DWARF debugging information. */\n #define DEBUGGER_REGNO(REGNO) rs6000_debugger_regno ((REGNO), 0)\n@@ -809,7 +810,9 @@ enum data_align { align_abi, align_opt, align_both };\n /* cr0..cr7 */\t\t\t\t \\\n 0, 0, 0, 0, 0, 0, 0, 0,\t\t\t \\\n /* vrsave vscr sfp */\t\t\t \\\n- 1, 1, 1\t\t\t\t\t \\\n+ 1, 1, 1,\t\t\t\t\t \\\n+ /* Dense math registers. */\t\t\t \\\n+ 0, 0, 0, 0, 0, 0, 0, 0\t\t\t \\\n }\n \n /* Like `CALL_USED_REGISTERS' except this macro doesn't require that\n@@ -833,7 +836,9 @@ enum data_align { align_abi, align_opt, align_both };\n /* cr0..cr7 */\t\t\t\t \\\n 1, 1, 0, 0, 0, 1, 1, 1,\t\t\t \\\n /* vrsave vscr sfp */\t\t\t \\\n- 0, 0, 0\t\t\t\t\t \\\n+ 0, 0, 0,\t\t\t\t\t \\\n+ /* Dense math registers. */\t\t\t \\\n+ 0, 0, 0, 0, 0, 0, 0, 0\t\t\t \\\n }\n \n #define TOTAL_ALTIVEC_REGS\t(LAST_ALTIVEC_REGNO - FIRST_ALTIVEC_REGNO + 1)\n@@ -870,6 +875,7 @@ enum data_align { align_abi, align_opt, align_both };\n \tv2\t\t(not saved; incoming vector arg reg; return value)\n \tv19 - v14\t(not saved or used for anything)\n \tv31 - v20\t(saved; order given to save least number)\n+\tdmr0 - dmr7\t(not saved)\n \tvrsave, vscr\t(fixed)\n \tsfp\t\t(fixed)\n */\n@@ -912,6 +918,9 @@ enum data_align { align_abi, align_opt, align_both };\n 66,\t\t\t\t\t\t\t\t\\\n 83, 82, 81, 80, 79, 78,\t\t\t\t\t\\\n 95, 94, 93, 92, 91, 90, 89, 88, 87, 86, 85, 84,\t\t\\\n+ /* Dense math registers. */\t\t\t\t\t\\\n+ 111, 112, 113, 114, 115, 116, 117, 118,\t\t\t\\\n+ /* Vrsave, vscr, sfp. */\t\t\t\t\t\\\n 108, 109,\t\t\t\t\t\t\t\\\n 110\t\t\t\t\t\t\t\t\\\n }\n@@ -938,6 +947,9 @@ enum data_align { align_abi, align_opt, align_both };\n /* True if register is a VSX register. */\n #define VSX_REGNO_P(N) (FP_REGNO_P (N) || ALTIVEC_REGNO_P (N))\n \n+/* True if register is a Dense math register. */\n+#define DMF_REGNO_P(N)\t((N) >= FIRST_DMF_REGNO && (N) <= LAST_DMF_REGNO)\n+\n /* Alternate name for any vector register supporting floating point, no matter\n which instruction set(s) are available. */\n #define VFLOAT_REGNO_P(N) \\\n@@ -1075,6 +1087,7 @@ enum reg_class\n FLOAT_REGS,\n ALTIVEC_REGS,\n VSX_REGS,\n+ DMF_REGS,\n VRSAVE_REGS,\n VSCR_REGS,\n GEN_OR_FLOAT_REGS,\n@@ -1104,6 +1117,7 @@ enum reg_class\n \"FLOAT_REGS\",\t\t\t\t\t\t\t\t\\\n \"ALTIVEC_REGS\",\t\t\t\t\t\t\t\\\n \"VSX_REGS\",\t\t\t\t\t\t\t\t\\\n+ \"DMF_REGS\",\t\t\t\t\t\t\t\t\\\n \"VRSAVE_REGS\",\t\t\t\t\t\t\t\\\n \"VSCR_REGS\",\t\t\t\t\t\t\t\t\\\n \"GEN_OR_FLOAT_REGS\",\t\t\t\t\t\t\t\\\n@@ -1138,6 +1152,8 @@ enum reg_class\n { 0x00000000, 0x00000000, 0xffffffff, 0x00000000 },\t\t\t\\\n /* VSX_REGS. */\t\t\t\t\t\t\t\\\n { 0x00000000, 0xffffffff, 0xffffffff, 0x00000000 },\t\t\t\\\n+ /* DMF_REGS. */\t\t\t\t\t\t\t\\\n+ { 0x00000000, 0x00000000, 0x00000000, 0x007f8000 },\t\t\t\\\n /* VRSAVE_REGS. */\t\t\t\t\t\t\t\\\n { 0x00000000, 0x00000000, 0x00000000, 0x00001000 },\t\t\t\\\n /* VSCR_REGS. */\t\t\t\t\t\t\t\\\n@@ -1165,7 +1181,7 @@ enum reg_class\n /* CA_REGS. */\t\t\t\t\t\t\t\\\n { 0x00000000, 0x00000000, 0x00000000, 0x00000004 },\t\t\t\\\n /* ALL_REGS. */\t\t\t\t\t\t\t\\\n- { 0xffffffff, 0xffffffff, 0xffffffff, 0x00007fff }\t\t\t\\\n+ { 0xffffffff, 0xffffffff, 0xffffffff, 0x007fffff }\t\t\t\\\n }\n \n /* The same information, inverted:\n@@ -2066,7 +2082,16 @@ extern char rs6000_reg_names[][8];\t/* register names (0 vs. %r0). */\n &rs6000_reg_names[108][0],\t/* vrsave */\t\t\t\t\\\n &rs6000_reg_names[109][0],\t/* vscr */\t\t\t\t\\\n \t\t\t\t\t\t\t\t\t\\\n- &rs6000_reg_names[110][0]\t/* sfp */\t\t\t\t\\\n+ &rs6000_reg_names[110][0],\t/* sfp */\t\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+ &rs6000_reg_names[111][0],\t/* dmr0 */\t\t\t\t\\\n+ &rs6000_reg_names[112][0],\t/* dmr1 */\t\t\t\t\\\n+ &rs6000_reg_names[113][0],\t/* dmr2 */\t\t\t\t\\\n+ &rs6000_reg_names[114][0],\t/* dmr3 */\t\t\t\t\\\n+ &rs6000_reg_names[115][0],\t/* dmr4 */\t\t\t\t\\\n+ &rs6000_reg_names[116][0],\t/* dmr5 */\t\t\t\t\\\n+ &rs6000_reg_names[117][0],\t/* dmr6 */\t\t\t\t\\\n+ &rs6000_reg_names[118][0],\t/* dmr7 */\t\t\t\t\\\n }\n \n /* Table of additional register names to use in user input. */\n@@ -2120,6 +2145,8 @@ extern char rs6000_reg_names[][8];\t/* register names (0 vs. %r0). */\n {\"vs52\", 84}, {\"vs53\", 85}, {\"vs54\", 86}, {\"vs55\", 87},\t\\\n {\"vs56\", 88}, {\"vs57\", 89}, {\"vs58\", 90}, {\"vs59\", 91},\t\\\n {\"vs60\", 92}, {\"vs61\", 93}, {\"vs62\", 94}, {\"vs63\", 95},\t\\\n+ {\"dmr0\", 111}, {\"dmr1\", 112}, {\"dmr2\", 113}, {\"dmr3\", 114},\t\\\n+ {\"dmr4\", 115}, {\"dmr5\", 116}, {\"dmr6\", 117}, {\"dmr7\", 118},\t\\\n }\n \n /* This is how to output an element of a case-vector that is relative. */\ndiff --git a/gcc/config/rs6000/rs6000.md b/gcc/config/rs6000/rs6000.md\nindex 3089551552c..1a8212ca0b4 100644\n--- a/gcc/config/rs6000/rs6000.md\n+++ b/gcc/config/rs6000/rs6000.md\n@@ -51,6 +51,8 @@ (define_constants\n (VRSAVE_REGNO\t\t108)\n (VSCR_REGNO\t\t\t109)\n (FRAME_POINTER_REGNUM\t110)\n+ (FIRST_DMF_REGNO\t\t111)\n+ (LAST_DMF_REGNO\t\t118)\n ])\n \n ;;\n","prefixes":[]}