Fix typo in 'libgomp.c/target-51.c' (was: [patch] OpenMP: Set default-device-var with OMP_TARGET_OFFLOAD=mandatory)

Hi!

On 2023-06-13T20:44:39+0200, Tobias Burnus <tobias@codesourcery.com> wrote:
> I intent to commit this tomorrow, unless there are comments.

I'm sorry I'm late.  ;-P

> It does as it says (see commit log): It initializes default-device-var
> to the value using the algorithm described in OpenMP 5.2, which
> depends on whether OMP_TARGET_OFFLOAD=mandatory was set.
>
> NOTE: With -foffload=disable there is no binary code but still
> devices get found - such that default-device-var == 0 (= first
> nonhost device). Thus, in that case, libgomp runs the code on that
> device but as no binary data is available, host fallback is used.
> (Even if there would be executable code for another device on
> the system.)
> With mandatory, this unintended host fallback is detected and an
> error is diagnosed. One can argue whether keeping the devices
> makes sense (e.g. because in a dynamic library device code will
> be loaded later) or not (don't list if no code is available).

This reminds me of the (unresolved) <https://gcc.gnu.org/PR81886>
"Means to determine at runtime foffload targets specified at compile time".

> Note that TR11 (future OpenMP 6.0) extends OMP_DEFAULT_DEVICE and
> adds OMP_AVAILABLE_DEVICES which permit a finer-grained control about
> the device, including OMP_DEFAULT_DEVICE=initial (and 'invalid') which
> the current scheme does not permit. (Well, there is
> OMP_TARGET_OFFLOAD=disabled, but that's a too big hammer.)

> PS:  DejaGNU testing was done without offloading configured
> and with remote testing on a system having an offload device,
> which which does not support setting environment variables.
> Manual testing was done with offloading enabled and depending
> on the testcase, running on a system with and/or without offloading
> hardware.

> --- a/libgomp/target.c
> +++ b/libgomp/target.c
> @@ -150,7 +150,11 @@ resolve_device (int device_id, bool remapped)
>        if (device_id == (remapped ? GOMP_DEVICE_HOST_FALLBACK
>                                : omp_initial_device))
>       return NULL;
> -      if (device_id == omp_invalid_device)
> +      if (gomp_target_offload_var == GOMP_TARGET_OFFLOAD_MANDATORY
> +       && gomp_get_num_devices () == 0)
> +     gomp_fatal ("OMP_TARGET_OFFLOAD is set to MANDATORY but only the host "
> +                 "device is available");
> +      else if (device_id == omp_invalid_device)
>       gomp_fatal ("omp_invalid_device encountered");
>        else if (gomp_target_offload_var == GOMP_TARGET_OFFLOAD_MANDATORY)
>       gomp_fatal ("OMP_TARGET_OFFLOAD is set to MANDATORY, "
|                   "but device not found");
|
|        return NULL;
|      }
|    else if (device_id >= gomp_get_num_devices ())
|      {
|        if (gomp_target_offload_var == GOMP_TARGET_OFFLOAD_MANDATORY
|         && device_id != num_devices_openmp)
|       gomp_fatal ("OMP_TARGET_OFFLOAD is set to MANDATORY, "
|                   "but device not found");
|
|        return NULL;
|      }
|
|    gomp_mutex_lock (&devices[device_id].lock);
|    if (devices[device_id].state == GOMP_DEVICE_UNINITIALIZED)
|      gomp_init_device (&devices[device_id]);
|    else if (devices[device_id].state == GOMP_DEVICE_FINALIZED)
|      {
|        gomp_mutex_unlock (&devices[device_id].lock);
|
|        if (gomp_target_offload_var == GOMP_TARGET_OFFLOAD_MANDATORY)
|       gomp_fatal ("OMP_TARGET_OFFLOAD is set to MANDATORY, "
|                   "but device is finalized");
|
|        return NULL;
|      }
|    gomp_mutex_unlock (&devices[device_id].lock);
|
|    return &devices[device_id];
|  }

> --- /dev/null
> +++ b/libgomp/testsuite/libgomp.c/target-51.c
> @@ -0,0 +1,24 @@
> +/* Check OMP_TARGET_OFFLOAD on systems with no available non-host devices,
> +   which is enforced by using -foffload=disable.  */
> +
> +/* { dg-do run } */
> +/* { dg-additional-options "-foffload=disable" } */
> +/* { dg-set-target-env-var OMP_TARGET_OFFLOAD "mandatory" } */
> +
> +/* { dg-shouldfail "OMP_TARGET_OFFLOAD=mandatory and no available device" } */
> +
> +/* See comment in target-50.c/target-50.c for why the output differs.  */
> +
> +/* { dg-output ".*libgomp: OMP_TARGET_OFFLOAD is set to MANDATORY but only the host device is available.*" { target { ! offload_device } } } */
> +/* { dg-output ".*libgomp: OMP_TARGET_OFFLOAD is set to MANDATORY but device not found.*" { target offload_device } } */

I intend to push the attached "Fix typo in 'libgomp.c/target-51.c'" after
testing.

Let me know if I should also adjust the new 'target { ! offload_device }'
diagnostic "[...] MANDATORY but only the host device is available" to
include a comma before 'but', for consistency with the other existing
diagnostics (cited above)?


Grüße
 Thomas


> +
> +int
> +main ()
> +{
> +  int x;
> +  #pragma omp target map(tofrom:x)
> +    x = 5;
> +  if (x != 5)
> +    __builtin_abort ();
> +  return 0;
> +}


-----------------
Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955

On 14.06.23 10:09, Thomas Schwinge wrote:
> This reminds me of the (unresolved)https://gcc.gnu.org/PR81886
> "Means to determine at runtime foffload targets specified at compile time".

I think there is the problem that we also support offloading in
libraries. Thus, if you compile the main program without offloading and
then link in a shared offloading-providing library (possibly with
dlopen), it comes (too) late. Thus, we either exclude devices which
could be later used – or we have to live with providing devices
(existing in hardware and with libgomp support) for which no executable
code is available.

As long as the number of devices is not a dynamic property, I guess we
can only handle one or the other.

> I intend to push the attached "Fix typo in 'libgomp.c/target-51.c'"
> after testing.
> Let me know if I should also adjust the new 'target { ! offload_device }'
> diagnostic "[...] MANDATORY but only the host device is available" to
> include a comma before 'but', for consistency with the other existing
> diagnostics (cited above)?

I think it makes sense to be consistent. Thus: Yes, please add the commas.

Thanks,

Tobias

-----------------
Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955