From patchwork Tue Jul 31 19:54:02 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bill Schmidt X-Patchwork-Id: 174326 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) by ozlabs.org (Postfix) with SMTP id 4DAB52C00A8 for ; Wed, 1 Aug 2012 05:54:30 +1000 (EST) Comment: DKIM? See http://www.dkim.org DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=gcc.gnu.org; s=default; x=1344369270; h=Comment: DomainKey-Signature:Received:Received:Received:Received:Received: Received:Received:Received:Received:Message-ID:Subject:From:To: Cc:Date:Content-Type:Content-Transfer-Encoding:Mime-Version: Mailing-List:Precedence:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:Sender:Delivered-To; bh=rGqiqEDiVmzf9nh/T3lP TJer/kg=; b=hC1LOrhi11M9Q3Y9d+LmX2MSlOjoH+C+xo334Ykxed+OHaU0CXOO fcoGSu4ZfRBDQOgznffsr+hfEdzniJfOOFou7RXO/yNMdKV0SzW0Gf/mTxYJ1XcR noTYxopyCKrYsE8A4u/T7m3H+SA30OwiSyHuFJi7+itq0QrdWHaFMCg= Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=gcc.gnu.org; h=Received:Received:X-SWARE-Spam-Status:X-Spam-Check-By:Received:Received:Received:Received:Received:Received:Received:Message-ID:Subject:From:To:Cc:Date:Content-Type:Content-Transfer-Encoding:Mime-Version:X-Content-Scanned:x-cbid:Mailing-List:Precedence:List-Id:List-Unsubscribe:List-Archive:List-Post:List-Help:Sender:Delivered-To; b=c1UgoYmPARIGjOh53c58H5pT2n1owNQDVnk07Ctn8cqeRZyblA8VneYAGBjku1 k81oDrNdfUSQou/O8O5s02bhmlrvukFObvTfh8DwrTDM7J1e7kGVDgclgTFN9szy rsU8jNmqY2UkQeARF//3F6Q48yEzYBIQfK1kXq+QrCQtE=; Received: (qmail 372 invoked by alias); 31 Jul 2012 19:54:26 -0000 Received: (qmail 363 invoked by uid 22791); 31 Jul 2012 19:54:25 -0000 X-SWARE-Spam-Status: No, hits=-5.3 required=5.0 tests=AWL, BAYES_00, KHOP_RCVD_UNTRUST, RCVD_IN_DNSWL_HI, RCVD_IN_HOSTKARMA_W, TW_TM, T_RP_MATCHES_RCVD X-Spam-Check-By: sourceware.org Received: from e31.co.us.ibm.com (HELO e31.co.us.ibm.com) (32.97.110.149) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Tue, 31 Jul 2012 19:54:11 +0000 Received: from /spool/local by e31.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 31 Jul 2012 13:54:06 -0600 Received: from d03dlp03.boulder.ibm.com (9.17.202.179) by e31.co.us.ibm.com (192.168.1.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 31 Jul 2012 13:54:04 -0600 Received: from d03relay03.boulder.ibm.com (d03relay03.boulder.ibm.com [9.17.195.228]) by d03dlp03.boulder.ibm.com (Postfix) with ESMTP id E793419D804E for ; Tue, 31 Jul 2012 19:53:58 +0000 (WET) Received: from d03av03.boulder.ibm.com (d03av03.boulder.ibm.com [9.17.195.169]) by d03relay03.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q6VJs1es117758 for ; Tue, 31 Jul 2012 13:54:01 -0600 Received: from d03av03.boulder.ibm.com (loopback [127.0.0.1]) by d03av03.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q6VJs0oq007046 for ; Tue, 31 Jul 2012 13:54:00 -0600 Received: from [9.65.15.97] (sig-9-65-15-97.mts.ibm.com [9.65.15.97]) by d03av03.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q6VJrxcD006964; Tue, 31 Jul 2012 13:53:59 -0600 Message-ID: <1343764442.16581.7.camel@gnopaine> Subject: [PATCH, rs6000] Vectorizer heuristic From: "William J. Schmidt" To: gcc-patches@gcc.gnu.org Cc: dje.gcc@gmail.com, bergner@vnet.ibm.com Date: Tue, 31 Jul 2012 14:54:02 -0500 Mime-Version: 1.0 X-Content-Scanned: Fidelis XPS MAILER x-cbid: 12073119-7282-0000-0000-00000B8598CD Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Now that the vectorizer cost model is set up to facilitate per-target heuristics, I'm revisiting the "density" heuristic I submitted previously. This allows the vec_permute and vec_promote_demote costs to be set to their natural values, but inhibits vectorization in cases like sphinx3 where vectorizing a loop leads to issue stalls from overcommitted resources. Bootstrapped on powerpc64-unknown-linux-gnu with no new regressions. Measured performance on cpu2000 and cpu2006 with no significant changes in performance. Ok for trunk? Thanks, Bill 2012-07-31 Bill Schmidt * config/rs6000/rs6000.c (rs6000_builtin_vectorization_cost): Revise costs for vec_perm and vec_promote_demote down to more natural values. (struct _rs6000_cost_data): New data structure. (rs6000_density_test): New function. (rs6000_init_cost): Change to use rs6000_cost_data. (rs6000_add_stmt_cost): Likewise. (rs6000_finish_cost): Perform density test when vectorizing a loop. Index: gcc/config/rs6000/rs6000.c =================================================================== --- gcc/config/rs6000/rs6000.c (revision 189845) +++ gcc/config/rs6000/rs6000.c (working copy) @@ -60,6 +60,7 @@ #include "params.h" #include "tm-constrs.h" #include "opts.h" +#include "tree-vectorizer.h" #if TARGET_XCOFF #include "xcoffout.h" /* get declarations of xcoff_*_section_name */ #endif @@ -3378,13 +3379,13 @@ rs6000_builtin_vectorization_cost (enum vect_cost_ case vec_perm: if (TARGET_VSX) - return 4; + return 3; else return 1; case vec_promote_demote: if (TARGET_VSX) - return 5; + return 4; else return 1; @@ -3520,14 +3521,71 @@ rs6000_preferred_simd_mode (enum machine_mode mode return word_mode; } +typedef struct _rs6000_cost_data +{ + struct loop *loop_info; + unsigned cost[3]; +} rs6000_cost_data; + +/* Test for likely overcommitment of vector hardware resources. If a + loop iteration is relatively large, and too large a percentage of + instructions in the loop are vectorized, the cost model may not + adequately reflect delays from unavailable vector resources. + Penalize the loop body cost for this case. */ + +static void +rs6000_density_test (rs6000_cost_data *data) +{ + const int DENSITY_PCT_THRESHOLD = 85; + const int DENSITY_SIZE_THRESHOLD = 70; + const int DENSITY_PENALTY = 10; + struct loop *loop = data->loop_info; + basic_block *bbs = get_loop_body (loop); + int nbbs = loop->num_nodes; + int vec_cost = data->cost[vect_body], not_vec_cost = 0; + int i, density_pct; + + for (i = 0; i < nbbs; i++) + { + basic_block bb = bbs[i]; + gimple_stmt_iterator gsi; + + for (gsi = gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi)) + { + gimple stmt = gsi_stmt (gsi); + stmt_vec_info stmt_info = vinfo_for_stmt (stmt); + + if (!STMT_VINFO_RELEVANT_P (stmt_info) + && !STMT_VINFO_IN_PATTERN_P (stmt_info)) + not_vec_cost++; + } + } + + density_pct = (vec_cost * 100) / (vec_cost + not_vec_cost); + + if (density_pct > DENSITY_PCT_THRESHOLD + && vec_cost + not_vec_cost > DENSITY_SIZE_THRESHOLD) + { + data->cost[vect_body] = vec_cost * (100 + DENSITY_PENALTY) / 100; + if (vect_print_dump_info (REPORT_DETAILS)) + fprintf (vect_dump, + "density %d%%, cost %d exceeds threshold, penalizing " + "loop body cost by %d%%", density_pct, + vec_cost + not_vec_cost, DENSITY_PENALTY); + } +} + /* Implement targetm.vectorize.init_cost. */ static void * -rs6000_init_cost (struct loop *loop_info ATTRIBUTE_UNUSED) +rs6000_init_cost (struct loop *loop_info) { - unsigned *cost = XNEWVEC (unsigned, 3); - cost[vect_prologue] = cost[vect_body] = cost[vect_epilogue] = 0; - return cost; + rs6000_cost_data *data = XNEW (struct _rs6000_cost_data); + data->loop_info = loop_info; + data->cost[vect_prologue] = 0; + data->cost[vect_body] = 0; + data->cost[vect_epilogue] = 0; + return data; } /* Implement targetm.vectorize.add_stmt_cost. */ @@ -3537,7 +3595,7 @@ rs6000_add_stmt_cost (void *data, int count, enum struct _stmt_vec_info *stmt_info, int misalign, enum vect_cost_model_location where) { - unsigned *cost = (unsigned *) data; + rs6000_cost_data *cost_data = (rs6000_cost_data*) data; unsigned retval = 0; if (flag_vect_cost_model) @@ -3552,7 +3610,7 @@ rs6000_add_stmt_cost (void *data, int count, enum count *= 50; /* FIXME. */ retval = (unsigned) (count * stmt_cost); - cost[where] += retval; + cost_data->cost[where] += retval; } return retval; @@ -3564,10 +3622,14 @@ static void rs6000_finish_cost (void *data, unsigned *prologue_cost, unsigned *body_cost, unsigned *epilogue_cost) { - unsigned *cost = (unsigned *) data; - *prologue_cost = cost[vect_prologue]; - *body_cost = cost[vect_body]; - *epilogue_cost = cost[vect_epilogue]; + rs6000_cost_data *cost_data = (rs6000_cost_data*) data; + + if (cost_data->loop_info) + rs6000_density_test (cost_data); + + *prologue_cost = cost_data->cost[vect_prologue]; + *body_cost = cost_data->cost[vect_body]; + *epilogue_cost = cost_data->cost[vect_epilogue]; } /* Implement targetm.vectorize.destroy_cost_data. */