From patchwork Tue Jan 28 14:55:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederik Harwath X-Patchwork-Id: 1230398 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=209.132.180.131; helo=sourceware.org; envelope-from=gcc-patches-return-518410-incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=codesourcery.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha1 header.s=default header.b=QdSNXUS+; dkim-atps=neutral Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 486V6y19Q0z9s29 for ; Wed, 29 Jan 2020 01:56:09 +1100 (AEDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :subject:to:cc:message-id:date:mime-version:content-type; q=dns; s=default; b=goWrgdz/iFNxlQgZjysRN5B67BrNVwWloiFKk6bDlE4P6jO4qS tN25Gp2T2XuGjyxFUfnRIsXFGvgdagEPp8geFQC8pWJiGBh1pwDUWZ4YzKa0r20g 6TExsOU+1A/+/8EXOR1c8gd6njuL7LFP0g0JVCrbeq0iqfJ7Zr+FSTU78= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :subject:to:cc:message-id:date:mime-version:content-type; s= default; bh=RE4L+mlk+6+9Z8oA0NFXrT8pHQA=; b=QdSNXUS+HhGwPv474omL A8rU+dDjFDb7Xio5u5fp2gl65tpZkkb+uICME+H684JuEHTl2q76Ar/0nbYpEXjf xuY8Rtsj75cej86a4z2ueGo+xwXAVQDklhd2MDOOSrOp39Wp8K7qRhvya8z9eatK kyX6INu5/qN+IT4Ho99LDi4= Received: (qmail 37671 invoked by alias); 28 Jan 2020 14:56:00 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 37659 invoked by uid 89); 28 Jan 2020 14:55:59 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-21.1 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, SPF_PASS autolearn=ham version=3.3.1 spammy=agents, prop, getenv X-HELO: esa4.mentor.iphmx.com Received: from esa4.mentor.iphmx.com (HELO esa4.mentor.iphmx.com) (68.232.137.252) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 28 Jan 2020 14:55:56 +0000 IronPort-SDR: Y9rlbyX58t2GYWZYrS0/p7BhoCYnA5CtPhfw37lSCHxp0ENfLx2KXRsNwS4UMVZiRj+AODQQ5c pUTnLw72vxi6KI5D2r0523DdX9+UjawT9CXwmPfDp7bw4ay5UI48SRQOtwOaLZWH8wxzwaTN4+ hGA6d5mBxxdhSdbuflX5kvCrV02HkJVnfHKXiPx6OxWl8uot5YL+Pa1Q4n8cmhc77bKwupLmat 1ShgSqXP0iZaG1WxhDSEK9T42SYuSJHxmpIq3baOTqm3GsfkDF3mMY+/jKryWleGMZOX2qxhY3 yaU= Received: from orw-gwy-02-in.mentorg.com ([192.94.38.167]) by esa4.mentor.iphmx.com with ESMTP; 28 Jan 2020 06:55:55 -0800 IronPort-SDR: +6Zo8wLlCU2Zcy6HtWtYb5PGT2NYaj3iZNXXyOTH+xY1j8WuOFDHMv2sH3hppAKcXBwDv9dF3G /uZJBsaETm3A== From: "Harwath, Frederik" Subject: [PATCH] Add OpenACC acc_get_property support for AMD GCN To: GCC Patches , "Stubbs, Andrew" , Thomas Schwinge CC: Jakub Jelinek X-Pep-Version: 2.0 Message-ID: <0f59690f-5247-9b6e-169a-9a6ef16c41e1@codesourcery.com> Date: Tue, 28 Jan 2020 15:55:49 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.4.1 MIME-Version: 1.0 X-IsSubscribed: yes Hi, this patch adds full support for the OpenACC 2.6 acc_get_property and acc_get_property_string functions to the libgomp GCN plugin. This replaces the existing stub in libgomp/plugin-gcn.c. Andrew: The value returned for acc_property_memory ("size of device memory in bytes" according to the spec) is the HSA_REGION_INFO_SIZE of the agent's data_region. This has been adapted from a previous incomplete implementation that we had on the OG9 branch. Does that sound reasonable? I have tested the patch with amdgcn and nvptx offloading. Ok to commit this to the main branch? Best regards, Frederik From 6f1855281c38993a088f9b4af020a786f8e05fe9 Mon Sep 17 00:00:00 2001 From: Frederik Harwath Date: Tue, 28 Jan 2020 08:01:00 +0100 Subject: [PATCH] Add OpenACC acc_get_property support for AMD GCN Add full support for the OpenACC 2.6 acc_get_property and acc_get_property_string functions to the libgomp GCN plugin. libgomp/ * plugin-gcn.c (struct agent_info): Add fields "name" and "vendor_name" ... (GOMP_OFFLOAD_init_device): ... and init from here. (struct hsa_context_info): Add field "driver_version_s" ... (init_hsa_contest): ... and init from here. (GOMP_OFFLOAD_openacc_get_property): Replace stub with a proper implementation. * testsuite/libgomp.oacc-c-c++-common/acc_get_property.c: Enable test execution for amdgcn and host offloading targets. * testsuite/libgomp.oacc-fortran/acc_get_property.f90: Likewise. * testsuite/libgomp.oacc-c-c++-common/acc_get_property-aux.c (expect_device_properties): Split function into ... (expect_device_string_properties): ... this new function ... (expect_device_memory): ... and this new function. * testsuite/libgomp.oacc-c-c++-common/acc_get_property-gcn.c: Add test. --- libgomp/plugin/plugin-gcn.c | 63 +++++++-- .../acc_get_property-aux.c | 60 +++++--- .../acc_get_property-gcn.c | 132 ++++++++++++++++++ .../acc_get_property.c | 5 +- .../libgomp.oacc-fortran/acc_get_property.f90 | 2 - 5 files changed, 224 insertions(+), 38 deletions(-) create mode 100644 libgomp/testsuite/libgomp.oacc-c-c++-common/acc_get_property-gcn.c diff --git a/libgomp/plugin/plugin-gcn.c b/libgomp/plugin/plugin-gcn.c index 7854c142f05..0a09daaa0a4 100644 --- a/libgomp/plugin/plugin-gcn.c +++ b/libgomp/plugin/plugin-gcn.c @@ -425,7 +425,10 @@ struct agent_info /* The instruction set architecture of the device. */ gcn_isa device_isa; - + /* Name of the agent. */ + char name[64]; + /* Name of the vendor of the agent. */ + char vendor_name[64]; /* Command queues of the agent. */ hsa_queue_t *sync_queue; struct goacc_asyncqueue *async_queues, *omp_async_queue; @@ -544,6 +547,8 @@ struct hsa_context_info int agent_count; /* Array of agent_info structures describing the individual HSA agents. */ struct agent_info *agents; + /* Driver version string. */ + char driver_version_s[30]; }; /* Format of the on-device heap. @@ -1513,6 +1518,15 @@ init_hsa_context (void) GOMP_PLUGIN_error ("Failed to list all HSA runtime agents"); } + uint16_t minor, major; + status = hsa_fns.hsa_system_get_info_fn (HSA_SYSTEM_INFO_VERSION_MINOR, &minor); + if (status != HSA_STATUS_SUCCESS) + GOMP_PLUGIN_error ("Failed to obtain HSA runtime minor version"); + status = hsa_fns.hsa_system_get_info_fn (HSA_SYSTEM_INFO_VERSION_MAJOR, &major); + if (status != HSA_STATUS_SUCCESS) + GOMP_PLUGIN_error ("Failed to obtain HSA runtime major version"); + sprintf (hsa_context.driver_version_s, "HSA Runtime %d.%d", major, minor); + hsa_context.initialized = true; return true; } @@ -3410,15 +3424,19 @@ GOMP_OFFLOAD_init_device (int n) return hsa_error ("Error requesting maximum queue size of the GCN agent", status); - char buf[64]; status = hsa_fns.hsa_agent_get_info_fn (agent->id, HSA_AGENT_INFO_NAME, - &buf); + &agent->name); if (status != HSA_STATUS_SUCCESS) return hsa_error ("Error querying the name of the agent", status); - agent->device_isa = isa_code (buf); + agent->device_isa = isa_code (agent->name); if (agent->device_isa < 0) - return hsa_error ("Unknown GCN agent architecture.", HSA_STATUS_ERROR); + return hsa_error ("Unknown GCN agent architecture", HSA_STATUS_ERROR); + + status = hsa_fns.hsa_agent_get_info_fn (agent->id, HSA_AGENT_INFO_VENDOR_NAME, + &agent->vendor_name); + if (status != HSA_STATUS_SUCCESS) + return hsa_error ("Error querying the vendor name of the agent", status); status = hsa_fns.hsa_queue_create_fn (agent->id, queue_size, HSA_QUEUE_TYPE_MULTI, @@ -4115,12 +4133,37 @@ GOMP_OFFLOAD_openacc_async_dev2host (int device, void *dst, const void *src, union goacc_property_value GOMP_OFFLOAD_openacc_get_property (int device, enum goacc_property prop) { - /* Stub. Check device and return default value for unsupported properties. */ - /* TODO: Implement this function. */ - get_agent_info (device); + struct agent_info *agent = get_agent_info (device); + + union goacc_property_value propval = { .val = 0 }; + + switch (prop) + { + case GOACC_PROPERTY_FREE_MEMORY: + /* Not supported. */ + break; + case GOACC_PROPERTY_MEMORY: + { + size_t size; + hsa_region_t region = agent->data_region; + hsa_status_t status = + hsa_fns.hsa_region_get_info_fn (region, HSA_REGION_INFO_SIZE, &size); + if (status == HSA_STATUS_SUCCESS) + propval.val = size; + break; + } + case GOACC_PROPERTY_NAME: + propval.ptr = agent->name; + break; + case GOACC_PROPERTY_VENDOR: + propval.ptr = agent->vendor_name; + break; + case GOACC_PROPERTY_DRIVER: + propval.ptr = hsa_context.driver_version_s; + break; + } - union goacc_property_value nullval = { .val = 0 }; - return nullval; + return propval; } /* Set up plugin-specific thread-local-data (host-side). */ diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/acc_get_property-aux.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/acc_get_property-aux.c index 6bb01250148..6a4a499eb21 100644 --- a/libgomp/testsuite/libgomp.oacc-c-c++-common/acc_get_property-aux.c +++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/acc_get_property-aux.c @@ -6,10 +6,11 @@ #include #include -void expect_device_properties + +void expect_device_string_properties (acc_device_t dev_type, int dev_num, - size_t expected_memory, const char* expected_vendor, - const char* expected_name, const char* expected_driver) + const char* expected_vendor, const char* expected_name, + const char* expected_driver) { const char *vendor = acc_get_property_string (dev_num, dev_type, acc_property_vendor); @@ -20,26 +21,6 @@ void expect_device_properties abort (); } - size_t total_mem = acc_get_property (dev_num, dev_type, - acc_property_memory); - if (total_mem != expected_memory) - { - fprintf (stderr, "Expected acc_property_memory to equal %zu, " - "but was %zu.\n", expected_memory, total_mem); - abort (); - - } - - size_t free_mem = acc_get_property (dev_num, dev_type, - acc_property_free_memory); - if (free_mem > total_mem) - { - fprintf (stderr, "Expected acc_property_free_memory <= acc_property_memory" - ", but free memory was %zu and total memory was %zu.\n", - free_mem, total_mem); - abort (); - } - const char *name = acc_get_property_string (dev_num, dev_type, acc_property_name); if (strcmp (name, expected_name)) @@ -75,6 +56,39 @@ void expect_device_properties "but was %s.\n", s); abort (); } +} + +void expect_device_memory +(acc_device_t dev_type, int dev_num, size_t expected_total_memory) +{ + + size_t total_mem = acc_get_property (dev_num, dev_type, + acc_property_memory); + + if (total_mem != expected_total_memory) + { + fprintf (stderr, "Expected acc_property_memory to equal %zu, " + "but was %zu.\n", expected_total_memory, total_mem); + abort (); + } + size_t free_mem = acc_get_property (dev_num, dev_type, + acc_property_free_memory); + if (free_mem > total_mem) + { + fprintf (stderr, "Expected acc_property_free_memory <= acc_property_memory" + ", but free memory was %zu and total memory was %zu.\n", + free_mem, total_mem); + abort (); + } +} +void expect_device_properties +(acc_device_t dev_type, int dev_num, + size_t expected_total_memory, const char* expected_vendor, + const char* expected_name, const char* expected_driver) +{ + expect_device_string_properties (dev_type, dev_num, expected_vendor, + expected_name, expected_driver); + expect_device_memory (dev_type, dev_num, expected_total_memory); } diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/acc_get_property-gcn.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/acc_get_property-gcn.c new file mode 100644 index 00000000000..305bb1af365 --- /dev/null +++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/acc_get_property-gcn.c @@ -0,0 +1,132 @@ +/* Test the `acc_get_property' and `acc_get_property_string' library + functions on amdgcn devices by comparing property values with + those obtained through the HSA API. */ +/* { dg-additional-sources acc_get_property-aux.c } */ +/* { dg-additional-options "-ldl" } */ +/* { dg-do run { target openacc_amdgcn_accel_selected } } */ + +#include +#include +#include +#include +#include +#include + +#ifndef __cplusplus +typedef int bool; +#endif +#include + + +void expect_device_string_properties +(acc_device_t dev_type, int dev_num, + const char* expected_vendor, const char* expected_name, + const char* expected_driver); + +hsa_status_t (*hsa_agent_get_info_fn) (hsa_agent_t agent, + hsa_agent_info_t attribute, + void *value); +hsa_status_t (*hsa_system_get_info_fn) (hsa_system_info_t attribute, + void *value); +hsa_status_t (*hsa_iterate_agents_fn) +(hsa_status_t (*callback)(hsa_agent_t agent, void *data), void *data); +hsa_status_t (*hsa_init_fn) (void); + +char* support_cpu_devices; + +void test_setup () +{ + char* env_runtime; + char* hsa_runtime_lib; + void *handle; + +#define DLSYM_FN(function) \ + function##_fn = (typeof(function##_fn))dlsym (handle, #function); \ + if (function##_fn == NULL) \ + { \ + fprintf (stderr, "Could not get symbol " #function ".\n"); \ + abort (); \ + } + + env_runtime = getenv ("HSA_RUNTIME_LIB"); + hsa_runtime_lib = env_runtime ? env_runtime : (char*)"libhsa-runtime64.so"; + + handle = dlopen (hsa_runtime_lib, RTLD_LAZY); + if (!handle) + { + fprintf (stderr, "Could not load %s.\n", hsa_runtime_lib); + abort (); + } + + DLSYM_FN (hsa_agent_get_info) + DLSYM_FN (hsa_system_get_info) + DLSYM_FN (hsa_iterate_agents) + DLSYM_FN (hsa_init) + + hsa_init_fn (); + + support_cpu_devices = getenv ("GCN_SUPPORT_CPU_DEVICES"); +} + +static hsa_status_t check_agent_properties +(hsa_agent_t agent, void *dev_num_arg) +{ + + char name[64]; + char vendor_name[64]; + uint16_t minor; + uint16_t major; + char driver[30]; + + hsa_status_t status; + hsa_device_type_t device_type; + int* dev_num = (int*)dev_num_arg; + +#define AGENT_GET_INFO(info_type, val) \ + status = hsa_agent_get_info_fn (agent, info_type, &val); \ + if (status != HSA_STATUS_SUCCESS) \ + { \ + fprintf (stderr, "Failed to obtain " #info_type ".\n"); \ + abort (); \ + } +#define SYSTEM_GET_INFO(info_type, val) \ + status = hsa_system_get_info_fn (info_type, &val); \ + if (status != HSA_STATUS_SUCCESS) \ + { \ + fprintf (stderr, "Failed to obtain " #info_type ".\n"); \ + abort (); \ + } + + AGENT_GET_INFO (HSA_AGENT_INFO_DEVICE, device_type) + + /* Skip unsupported device types. Mimic the GCN plugin's behavior. */ + if (!(device_type == HSA_DEVICE_TYPE_GPU + || (support_cpu_devices && device_type == HSA_DEVICE_TYPE_CPU))) + return HSA_STATUS_SUCCESS; + + AGENT_GET_INFO (HSA_AGENT_INFO_NAME, name) + AGENT_GET_INFO (HSA_AGENT_INFO_VENDOR_NAME, vendor_name) + + SYSTEM_GET_INFO (HSA_SYSTEM_INFO_VERSION_MINOR, minor) + SYSTEM_GET_INFO (HSA_SYSTEM_INFO_VERSION_MAJOR, major) + + sprintf (driver, "HSA Runtime %d.%d", major, minor); + + expect_device_string_properties(acc_device_radeon, *dev_num, + vendor_name, name, driver); + + (*dev_num)++; + + return status; +} + +int main() +{ + int dev_num = 0; + test_setup (); + + hsa_status_t status = + hsa_iterate_agents_fn (&check_agent_properties, &dev_num); + + return status; +} diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/acc_get_property.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/acc_get_property.c index 388c66c1319..1984ad3a28d 100644 --- a/libgomp/testsuite/libgomp.oacc-c-c++-common/acc_get_property.c +++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/acc_get_property.c @@ -3,8 +3,7 @@ of all device types mentioned in the OpenACC standard. See also acc_get_property.f90. */ -/* { dg-do run { target { { ! { openacc_host_selected } } && { ! { openacc_amdgcn_accel_selected } } } } } */ -/* FIXME: This test does not work with the GCN implementation stub yet. */ +/* { dg-do run } */ #include #include @@ -20,7 +19,7 @@ print_device_properties(acc_device_t type) const char *s; size_t v; - int dev_count = acc_get_num_devices(type); + int dev_count = acc_get_num_devices (type); for (int i = 0; i < dev_count; ++i) { diff --git a/libgomp/testsuite/libgomp.oacc-fortran/acc_get_property.f90 b/libgomp/testsuite/libgomp.oacc-fortran/acc_get_property.f90 index ce695475ae4..80ae292f41f 100644 --- a/libgomp/testsuite/libgomp.oacc-fortran/acc_get_property.f90 +++ b/libgomp/testsuite/libgomp.oacc-fortran/acc_get_property.f90 @@ -3,8 +3,6 @@ ! of all device types mentioned in the OpenACC standard. ! ! See also acc_get_property.c -! { dg-do run { target { { ! { openacc_host_selected } } && { ! { openacc_amdgcn_accel_selected } } } } } -! FIXME: This test does not work with the GCN implementation stub yet. program test use openacc -- 2.17.1