From patchwork Wed Oct 14 20:32:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kwok Cheung Yeung X-Patchwork-Id: 1382337 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=codesourcery.com Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CBPHX1qlWz9sTr for ; Thu, 15 Oct 2020 07:32:56 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 60ED2385780E; Wed, 14 Oct 2020 20:32:53 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from esa4.mentor.iphmx.com (esa4.mentor.iphmx.com [68.232.137.252]) by sourceware.org (Postfix) with ESMTPS id 512873857C43 for ; Wed, 14 Oct 2020 20:32:50 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 512873857C43 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=codesourcery.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=Kwok_Yeung@mentor.com IronPort-SDR: 6p3/JM9AmsLxa0K6HE32x+VsXKFOIvTw6fWFtbzNUSiCZ9JE60nzdJVPeQWGCS5Sem0GZCExrV tU4DHwrSDeI9XUaBGcEi4TY2MWMY7GIu1dMkGTECbrIaMDYS93XM1QdHF0UGmfcoFkRevOeEpd 57XvxOSMaFKb4Y0wEfK0r0qI4jweGlUjnPGDaVTl56sVYHfLe2US9TRSGpEpSjxRsC97edWqzs RTgPLfZNn7/GyXK/q0wTK0m1uWC4okFccUn0gS4RVNCrQ4hEAW+UalzLj7Y6q/fzOCrDSW8kBk xW8= X-IronPort-AV: E=Sophos;i="5.77,375,1596528000"; d="scan'208";a="54084953" Received: from orw-gwy-01-in.mentorg.com ([192.94.38.165]) by esa4.mentor.iphmx.com with ESMTP; 14 Oct 2020 12:32:49 -0800 IronPort-SDR: d4ULrpbK0UXcK8V2dSA8mrzCPo4p0Yxmb2O08akewQYajUgTk65KMo1edoyn0/5lqJXmbyq09g UdfGeQl1O5uRM9N+D5RpmK846tl3fpmfiMMCaZ1mu0dbe1Lgv0UPaXdEaEWC0L7F8FzpGRygRJ k8XTGXYiEVKQ1JN3cTTBndxRa/xXeHFTJDJ0V3mmVBBvdCCODuwxhAAKwM5ywk1hwoFLPEricj fRCBJvJwKd6x3eXdS+O17cjZq1YqNO9FkDaNQmQvc2tv7NDYHGDG2dC4oFn4RqJQw5+c3zcRCp nRg= From: Kwok Cheung Yeung Subject: [PATCH] openmp: Implement support for OMP_TARGET_OFFLOAD To: GCC Patches , Jakub Jelinek Message-ID: <04045d90-625a-9214-31d4-0d00dbaf6ad7@codesourcery.com> Date: Wed, 14 Oct 2020 21:32:42 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 Content-Language: en-GB X-Originating-IP: [137.202.0.90] X-ClientProxiedBy: svr-ies-mbx-02.mgc.mentorg.com (139.181.222.2) To svr-ies-mbx-01.mgc.mentorg.com (139.181.222.1) X-Spam-Status: No, score=-12.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, KAM_DMARC_STATUS, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" Hello This implements support for the OMP_TARGET_OFFLOAD environment variable introduced in the OpenMP 5.0 standard, which controls how offloading is handled in an OpenMP program. If set to MANDATORY, then libgomp will cause the program to abort with a gomp_fatal if an offload device is not found, or if it falls back to the host for some reason. When DISABLED, then gomp_target_init will return early, so that libgomp acts as if no offload devices were found and the host fallback is always used. For DEFAULT, nothing is done, resulting in the original behaviour. I'm not sure how this can be tested automatically, as the behaviour depends on whether the compiler has been built with offloading support, and whether any supported offloading hardware has been installed on the system. I have not included any testcases for now. Okay for trunk? Thanks Kwok commit a22f434d5ec9e62c158912b693275ce89a2cbab0 Author: Kwok Cheung Yeung Date: Thu Oct 8 10:08:27 2020 -0700 openmp: Implement support for OMP_TARGET_OFFLOAD environment variable This implements support for the OMP_TARGET_OFFLOAD environment variable introduced in the OpenMP 5.0 standard, which controls how offloading is handled. It may be set to MANDATORY (abort if offloading cannot be performed), DISABLED (no offloading to devices) or DEFAULT (offload to device if possible, fall back to host if not). 2020-10-14 Kwok Cheung Yeung libgomp/ * env.c (gomp_target_offload_var): New. (parse_target_offload): New. (handle_omp_display_env): Print value of OMP_TARGET_OFFLOAD. (initialize_env): Parse OMP_TARGET_OFFLOAD. * libgomp.h (gomp_target_offload_t): New. (gomp_target_offload_var): New. * libgomp.texi (OMP_TARGET_OFFLOAD): New section. * target.c (resolve_device): Generate error if device not found and offloading is mandatory. (gomp_target_fallback): Generate error if offloading is mandatory. (gomp_target_fallback): Likewise. (gomp_target_init): Return early if offloading is disabled. diff --git a/libgomp/env.c b/libgomp/env.c index d730c48..d0eae8d 100644 --- a/libgomp/env.c +++ b/libgomp/env.c @@ -75,6 +75,7 @@ struct gomp_task_icv gomp_global_icv = { unsigned long gomp_max_active_levels_var = gomp_supported_active_levels; bool gomp_cancel_var = false; +enum gomp_target_offload_t gomp_target_offload_var = GOMP_TARGET_OFFLOAD_DEFAULT; int gomp_max_task_priority_var = 0; #ifndef HAVE_SYNC_BUILTINS gomp_mutex_t gomp_managed_threads_lock; @@ -374,6 +375,48 @@ parse_unsigned_long_list (const char *name, unsigned long *p1stvalue, return false; } +static void +parse_target_offload (const char *name, enum gomp_target_offload_t *offload) +{ + const char *env; + bool found = false; + enum gomp_target_offload_t new_offload; + + env = getenv (name); + if (env == NULL) + return; + + while (isspace ((unsigned char) *env)) + ++env; + if (strncasecmp (env, "default", 7) == 0) + { + env += 7; + found = true; + new_offload = GOMP_TARGET_OFFLOAD_DEFAULT; + } + else if (strncasecmp (env, "mandatory", 9) == 0) + { + env += 9; + found = true; + new_offload = GOMP_TARGET_OFFLOAD_MANDATORY; + } + else if (strncasecmp (env, "disabled", 8) == 0) + { + env += 8; + found = true; + new_offload = GOMP_TARGET_OFFLOAD_DISABLED; + } + while (isspace ((unsigned char) *env)) + ++env; + if (found && *env == '\0') + { + *offload = new_offload; + return; + } + + gomp_error ("Invalid value for environment variable OMP_TARGET_OFFLOAD"); +} + /* Parse environment variable set to a boolean or list of omp_proc_bind_t enum values. Return true if one was present and it was successfully parsed. */ @@ -1334,6 +1377,21 @@ handle_omp_display_env (unsigned long stacksize, int wait_policy) } fputs ("'\n", stderr); + fputs (" OMP_TARGET_OFFLOAD = '", stderr); + switch (gomp_target_offload_var) + { + case GOMP_TARGET_OFFLOAD_DEFAULT: + fputs ("DEFAULT", stderr); + break; + case GOMP_TARGET_OFFLOAD_MANDATORY: + fputs ("MANDATORY", stderr); + break; + case GOMP_TARGET_OFFLOAD_DISABLED: + fputs ("DISABLED", stderr); + break; + } + fputs ("'\n", stderr); + if (verbose) { fputs (" GOMP_CPU_AFFINITY = ''\n", stderr); @@ -1366,6 +1424,7 @@ initialize_env (void) parse_boolean ("OMP_CANCELLATION", &gomp_cancel_var); parse_boolean ("OMP_DISPLAY_AFFINITY", &gomp_display_affinity_var); parse_int ("OMP_DEFAULT_DEVICE", &gomp_global_icv.default_device_var, true); + parse_target_offload ("OMP_TARGET_OFFLOAD", &gomp_target_offload_var); parse_int ("OMP_MAX_TASK_PRIORITY", &gomp_max_task_priority_var, true); parse_unsigned_long ("OMP_MAX_ACTIVE_LEVELS", &gomp_max_active_levels_var, true); diff --git a/libgomp/libgomp.h b/libgomp/libgomp.h index 9d26de2..da7ac03 100644 --- a/libgomp/libgomp.h +++ b/libgomp/libgomp.h @@ -434,6 +434,13 @@ struct gomp_task_icv struct target_mem_desc *target_data; }; +enum gomp_target_offload_t +{ + GOMP_TARGET_OFFLOAD_DEFAULT, + GOMP_TARGET_OFFLOAD_MANDATORY, + GOMP_TARGET_OFFLOAD_DISABLED +}; + #define gomp_supported_active_levels INT_MAX extern struct gomp_task_icv gomp_global_icv; @@ -442,6 +449,7 @@ extern gomp_mutex_t gomp_managed_threads_lock; #endif extern unsigned long gomp_max_active_levels_var; extern bool gomp_cancel_var; +extern enum gomp_target_offload_t gomp_target_offload_var; extern int gomp_max_task_priority_var; extern unsigned long long gomp_spin_count_var, gomp_throttled_spin_count_var; extern unsigned long gomp_available_cpus, gomp_managed_threads; diff --git a/libgomp/libgomp.texi b/libgomp/libgomp.texi index 1c34bbe..4a5e56f 100644 --- a/libgomp/libgomp.texi +++ b/libgomp/libgomp.texi @@ -1381,6 +1381,7 @@ beginning with @env{GOMP_} are GNU extensions. * OMP_PLACES:: Specifies on which CPUs the theads should be placed * OMP_STACKSIZE:: Set default thread stack size * OMP_SCHEDULE:: How threads are scheduled +* OMP_TARGET_OFFLOAD:: Controls offloading behaviour * OMP_THREAD_LIMIT:: Set the maximum number of threads * OMP_WAIT_POLICY:: How waiting threads are handled * GOMP_CPU_AFFINITY:: Bind threads to specific CPUs @@ -1654,6 +1655,30 @@ dynamic scheduling and a chunk size of 1 is used. +@node OMP_TARGET_OFFLOAD +@section @env{OMP_TARGET_OFFLOAD} -- Controls offloading behaviour +@cindex Environment Variable +@cindex Implementation specific setting +@table @asis +@item @emph{Description}: +Specifies the behaviour with regard to offloading code to a device. This +variable can be set to one of three values - @code{MANDATORY}, @code{DISABLED} +or @code{DEFAULT}. + +If set to @code{MANDATORY}, the program will terminate with an error if +the offload device is not present or is not supported. If set to +@code{DISABLED}, then offloading is disabled and all code will run on the +host. If set to @code{DEFAULT}, the program will try offloading to the +device first, then fall back to running code on the host if it cannot. + +If undefined, then the program will behave as if @code{DEFAULT} was set. + +@item @emph{Reference}: +@uref{https://www.openmp.org, OpenMP specification v5.0}, Section 6.17 +@end table + + + @node OMP_THREAD_LIMIT @section @env{OMP_THREAD_LIMIT} -- Set the maximum number of threads @cindex Environment Variable diff --git a/libgomp/target.c b/libgomp/target.c index ab7ac9b..8fcd7f8 100644 --- a/libgomp/target.c +++ b/libgomp/target.c @@ -116,7 +116,12 @@ resolve_device (int device_id) } if (device_id < 0 || device_id >= gomp_get_num_devices ()) - return NULL; + { + if (gomp_target_offload_var == GOMP_TARGET_OFFLOAD_MANDATORY) + gomp_fatal ("OMP_TARGET_OFFLOAD is set to MANDATORY, but device not found."); + + return NULL; + } gomp_mutex_lock (&devices[device_id].lock); if (devices[device_id].state == GOMP_DEVICE_UNINITIALIZED) @@ -2000,6 +2005,11 @@ static void gomp_target_fallback (void (*fn) (void *), void **hostaddrs) { struct gomp_thread old_thr, *thr = gomp_thread (); + + if (gomp_target_offload_var == GOMP_TARGET_OFFLOAD_MANDATORY) + gomp_fatal ("OMP_TARGET_OFFLOAD is set to MANDATORY, but device cannot " + "be used for offloading."); + old_thr = *thr; memset (thr, '\0', sizeof (*thr)); if (gomp_places_list) @@ -2279,6 +2289,11 @@ static void gomp_target_data_fallback (void) { struct gomp_task_icv *icv = gomp_icv (false); + + if (gomp_target_offload_var == GOMP_TARGET_OFFLOAD_MANDATORY) + gomp_fatal ("OMP_TARGET_OFFLOAD is set to MANDATORY, but device cannot " + "be used for offloading."); + if (icv->target_data) { /* Even when doing a host fallback, if there are any active @@ -3258,6 +3273,9 @@ gomp_target_init (void) num_devices = 0; devices = NULL; + if (gomp_target_offload_var == GOMP_TARGET_OFFLOAD_DISABLED) + return; + cur = OFFLOAD_PLUGINS; if (*cur) do