diff mbox

nvptx offloading patches [3/n], RFD

Message ID 54E5ACCE.7080502@codesourcery.com
State New
Headers show

Commit Message

Bernd Schmidt Feb. 19, 2015, 9:28 a.m. UTC
On 02/17/2015 05:40 PM, Jakub Jelinek wrote:
> On Tue, Feb 17, 2015 at 04:21:06PM +0000, Joseph Myers wrote:
>> On Tue, 17 Feb 2015, Jakub Jelinek wrote:
>>
>>> Third attempt failed with:
>>> ../../../libgcc/config/nvptx/realloc.c:24:20: fatal error: stdlib.h: No such file or directory
>>> compilation terminated.
>>> ../../../libgcc/static-object.mk:17: recipe for target 'realloc.o' failed
>>> make[2]: *** [realloc.o] Error 1
>>> make[2]: *** Waiting for unfinished jobs....
>>> make[2]: Leaving directory '/usr/src/gcc/objnvptx/nvptx-none/libgcc'
>>> I have nvptx-newlib symlinked into the gcc tree as newlib, so I expected it
>>> would be built in-tree, is that not the case (at least wiki/Offloading
>>> mentions that).  Or is it just that libgcc can't really have dependencies on
>>> newlib headers as newlib is built after libgcc?
>>
>> I've committed this patch to fix this last issue (the header dependence,
>> that is; I don't know about the in-tree build).
>
> Thanks, sure, libgcc now builds fine, the in-tree build fails:
> configure:4261: checking for C compiler default output file name
> configure:4283: /usr/src/gcc/objnvptx/./gcc/xgcc -B/usr/src/gcc/objnvptx/./gcc/ -nostdinc -B/usr/src/gcc/objnvptx/nvptx-none/newlib/ -isystem /usr/src/gcc/objnvptx/nvptx-none/newlib/targ-include -isystem /usr/src/gcc/newlib/libc/include -B/usr/local/nvptx-none/bin/ -B/usr/local/nvptx-none/lib/ -isystem /usr/local/nvptx-none/include -isystem /usr/local/nvptx-none/sys-include    -g -O2   conftest.c  >&5
> error opening libc.a
> collect2: error: ld returned 1 exit status
> very early during in-tree newlib configure.

Not a fix for your problem, but there's a similar issue when trying to 
get at the libgcc for the nvptx accel compiler after it's been 
installed. The libgcc Makefile puts it in the wrong place - 
gcc/nvptx-none/accel/nvptx-none instead of gcc/host/accel/nvptx-none. 
The patch below corrects that and removes an intelmicemul special case 
which I believe has the same effect - Ilya, could you test this?


Bernd

Comments

Thomas Schwinge Feb. 19, 2015, 11:42 a.m. UTC | #1
Hi!

On Thu, 19 Feb 2015 10:28:46 +0100, Bernd Schmidt <bernds@codesourcery.com> wrote:
> On 02/17/2015 05:40 PM, Jakub Jelinek wrote:
> > On Tue, Feb 17, 2015 at 04:21:06PM +0000, Joseph Myers wrote:
> >> On Tue, 17 Feb 2015, Jakub Jelinek wrote:
> >>
> >>> Third attempt failed with:
> >>> ../../../libgcc/config/nvptx/realloc.c:24:20: fatal error: stdlib.h: No such file or directory
> >>> compilation terminated.
> >>> ../../../libgcc/static-object.mk:17: recipe for target 'realloc.o' failed
> >>> make[2]: *** [realloc.o] Error 1
> >>> make[2]: *** Waiting for unfinished jobs....
> >>> make[2]: Leaving directory '/usr/src/gcc/objnvptx/nvptx-none/libgcc'
> >>> I have nvptx-newlib symlinked into the gcc tree as newlib, so I expected it
> >>> would be built in-tree, is that not the case (at least wiki/Offloading
> >>> mentions that).  Or is it just that libgcc can't really have dependencies on
> >>> newlib headers as newlib is built after libgcc?
> >>
> >> I've committed this patch to fix this last issue (the header dependence,
> >> that is; I don't know about the in-tree build).
> >
> > Thanks, sure, libgcc now builds fine, the in-tree build fails:
> > configure:4261: checking for C compiler default output file name
> > configure:4283: /usr/src/gcc/objnvptx/./gcc/xgcc -B/usr/src/gcc/objnvptx/./gcc/ -nostdinc -B/usr/src/gcc/objnvptx/nvptx-none/newlib/ -isystem /usr/src/gcc/objnvptx/nvptx-none/newlib/targ-include -isystem /usr/src/gcc/newlib/libc/include -B/usr/local/nvptx-none/bin/ -B/usr/local/nvptx-none/lib/ -isystem /usr/local/nvptx-none/include -isystem /usr/local/nvptx-none/sys-include    -g -O2   conftest.c  >&5
> > error opening libc.a
> > collect2: error: ld returned 1 exit status
> > very early during in-tree newlib configure.
> 
> Not a fix for your problem, but there's a similar issue when trying to 
> get at the libgcc for the nvptx accel compiler after it's been 
> installed. The libgcc Makefile puts it in the wrong place - 
> gcc/nvptx-none/accel/nvptx-none instead of gcc/host/accel/nvptx-none. 

I also wondered about this; it's somewhere on my TODO list...

> The patch below corrects that and removes an intelmicemul special case 
> which I believe has the same effect - Ilya, could you test this?

This code has originally been posted in
<http://news.gmane.org/find-root.php?message_id=%3C20140926123551.GA6892%40msticlxl57.ims.intel.com%3E>.

This specific buglet aside (that the handling of intelmic and nvptx
offloading is inconsistent) -- will we have to add such handling to each
and every library that is built for the offloading compilers?  (Including
libraries that aren't part of the GCC sources, but may be built as part
of GCC's build process, such as when newlib is linked into [GCC]/newlib?)

One step back -- I understand correctly that this change is to make sure
that the regular target compiler and the offloading compilers don't clash
in their installed files' names?  (By putting them into the
accel/[offloading architecture]/ subdirectory?)  (As I've written in
<http://news.gmane.org/find-root.php?message_id=%3C87vbize7zi.fsf%40schwinge.name%3E>,
I currently install into separate prefixes/DESTDIRS, because I have not
yet verified that there is no overlap in the installed files.)

Then, why does this only apply to libsubdir?  What about header files,
documentation files, and so on?  (If they aren't expected to differ
between the target and offloading compilers, I think it's still not a
good idea to arbitrarely have them be overwritten by on respective build
tree's make install process.)  Should we have a more general solution to
this problem?

> Index: libgcc/Makefile.in
> ===================================================================
> --- libgcc/Makefile.in	(revision 445788)
> +++ libgcc/Makefile.in	(working copy)
> @@ -45,6 +45,7 @@ fixed_point = @fixed_point@
>  with_aix_soname = @with_aix_soname@
>  
>  host_noncanonical = @host_noncanonical@
> +real_host_noncanonical = @real_host_noncanonical@
>  target_noncanonical = @target_noncanonical@
>  
>  # List of extra object files that should be compiled for this target machine.
> @@ -185,7 +186,7 @@ STRIP = @STRIP@
>  STRIP_FOR_TARGET = $(STRIP)
>  
>  # Directory in which the compiler finds libraries etc.
> -libsubdir = $(libdir)/gcc/$(host_noncanonical)/$(version)@accel_dir_suffix@
> +libsubdir = $(libdir)/gcc/$(real_host_noncanonical)/$(version)@accel_dir_suffix@
>  # Used to install the shared libgcc.
>  slibdir = @slibdir@
>  # Maybe used for DLLs on Windows targets.
> Index: libgcc/configure.ac
> ===================================================================
> --- libgcc/configure.ac	(revision 445788)
> +++ libgcc/configure.ac	(working copy)
> @@ -398,16 +398,14 @@ esac
>  
>  # Used for constructing correct paths for offload compilers.
>  accel_dir_suffix=
> +real_host_noncanonical=${host_noncanonical}
> +echo "eaaf: $enable_as_accelerator_for"
>  if test x"$enable_as_accelerator_for" != x; then
>    accel_dir_suffix=/accel/${target_noncanonical}
> -  case "${target_noncanonical}" in
> -    *-intelmicemul-*)
> -      # In this case we expect offload compiler to be built as native, so we
> -      # need to change install directory for driver to be able to find libgcc.
> -      host_noncanonical=${enable_as_accelerator_for} ;;
> -  esac
> +  real_host_noncanonical=${enable_as_accelerator_for}
>  fi
>  AC_SUBST(accel_dir_suffix)
> +AC_SUBST(real_host_noncanonical)
>  
>  if test x"$enable_offload_targets" != x; then
>    extra_parts="${extra_parts} crtoffloadbegin.o crtoffloadend.o"


Grüße,
 Thomas
Bernd Schmidt Feb. 19, 2015, 12:08 p.m. UTC | #2
On 02/19/2015 12:42 PM, Thomas Schwinge wrote:
> This specific buglet aside (that the handling of intelmic and nvptx
> offloading is inconsistent) -- will we have to add such handling to each
> and every library that is built for the offloading compilers?  (Including
> libraries that aren't part of the GCC sources, but may be built as part
> of GCC's build process, such as when newlib is linked into [GCC]/newlib?)

No, they go into different directories. Only libgcc.a (along with a very 
few other pieces) is installed under lib/gcc/...

> Then, why does this only apply to libsubdir?  What about header files,
> documentation files, and so on?  (If they aren't expected to differ
> between the target and offloading compilers, I think it's still not a
> good idea to arbitrarely have them be overwritten by on respective build
> tree's make install process.)  Should we have a more general solution to
> this problem?

That stuff goes into the normal lib and include directories. I'm 
guessing a sysroot is what you want to keep it separate.


Bernd
Thomas Schwinge Feb. 19, 2015, 12:17 p.m. UTC | #3
Hi!

On Thu, 19 Feb 2015 13:08:20 +0100, Bernd Schmidt <bernds@codesourcery.com> wrote:
> On 02/19/2015 12:42 PM, Thomas Schwinge wrote:
> > This specific buglet aside (that the handling of intelmic and nvptx
> > offloading is inconsistent) -- will we have to add such handling to each
> > and every library that is built for the offloading compilers?  (Including
> > libraries that aren't part of the GCC sources, but may be built as part
> > of GCC's build process, such as when newlib is linked into [GCC]/newlib?)
> 
> No, they go into different directories. Only libgcc.a (along with a very 
> few other pieces) is installed under lib/gcc/...

Thanks, I see.

> > Then, why does this only apply to libsubdir?  What about header files,
> > documentation files, and so on?  (If they aren't expected to differ
> > between the target and offloading compilers, I think it's still not a
> > good idea to arbitrarely have them be overwritten by on respective build
> > tree's make install process.)  Should we have a more general solution to
> > this problem?
> 
> That stuff goes into the normal lib and include directories. I'm 
> guessing a sysroot is what you want to keep it separate.

My asumption is that it is always safe to install non-native (that is
cross) GCC installations into the same prefix.  (Which would resolve this
problem of clashing file names for target and offloading compilers for
good.)

So, the next question is, instead of this special handling, why can't we
require the offloading compilers to always be configured as cross
compilers?  Or, why is it a requirement that the intelmic offloading
compiler is configured as a native compiler?


Grüße,
 Thomas
Thomas Schwinge Feb. 20, 2015, 9:27 a.m. UTC | #4
Hi Bernd!

On Thu, 19 Feb 2015 10:28:46 +0100, Bernd Schmidt <bernds@codesourcery.com> wrote:
> issue when trying to 
> get at the libgcc for the nvptx accel compiler after it's been 
> installed. The libgcc Makefile puts it in the wrong place - 
> gcc/nvptx-none/accel/nvptx-none instead of gcc/host/accel/nvptx-none. 
> The patch below corrects that and removes an intelmicemul special case 
> which I believe has the same effect - Ilya, could you test this?

Works fine for me for intelmic (no changes), and nvptx (changes as
expected).

You'll want to remove the following debugging print statement before
commit:

> --- libgcc/configure.ac	(revision 445788)
> +++ libgcc/configure.ac	(working copy)
> @@ -398,16 +398,14 @@ esac
>  
>  # Used for constructing correct paths for offload compilers.
>  accel_dir_suffix=
> +real_host_noncanonical=${host_noncanonical}
> +echo "eaaf: $enable_as_accelerator_for"


Grüße,
 Thomas
Ilya Verbin Feb. 20, 2015, 3:05 p.m. UTC | #5
On Thu, Feb 19, 2015 at 13:17:37 +0100, Thomas Schwinge wrote:
> My asumption is that it is always safe to install non-native (that is
> cross) GCC installations into the same prefix.  (Which would resolve this
> problem of clashing file names for target and offloading compilers for
> good.)
> 
> So, the next question is, instead of this special handling, why can't we
> require the offloading compilers to always be configured as cross
> compilers?  Or, why is it a requirement that the intelmic offloading
> compiler is configured as a native compiler?

If I understand correctly, to build a cross compiler, we need to specify a path
to the target sysroot, even for x86_64-pc-linux-gnu to x86_64-intelmic-linux-gnu
cross.  Or is it possible to build a cross compiler without --with-sysroot ?

Thanks,
  -- Ilya
Ilya Verbin Feb. 20, 2015, 7:27 p.m. UTC | #6
On Fri, Feb 20, 2015 at 10:27:26 +0100, Thomas Schwinge wrote:
> On Thu, 19 Feb 2015 10:28:46 +0100, Bernd Schmidt <bernds@codesourcery.com> wrote:
> > issue when trying to 
> > get at the libgcc for the nvptx accel compiler after it's been 
> > installed. The libgcc Makefile puts it in the wrong place - 
> > gcc/nvptx-none/accel/nvptx-none instead of gcc/host/accel/nvptx-none. 
> > The patch below corrects that and removes an intelmicemul special case 
> > which I believe has the same effect - Ilya, could you test this?
> 
> Works fine for me for intelmic (no changes), and nvptx (changes as
> expected).

OK to me.

Thanks,
  -- Ilya
Ilya Verbin Feb. 20, 2015, 7:48 p.m. UTC | #7
On Fri, Feb 20, 2015 at 18:05:01 +0300, Ilya Verbin wrote:
> On Thu, Feb 19, 2015 at 13:17:37 +0100, Thomas Schwinge wrote:
> > My asumption is that it is always safe to install non-native (that is
> > cross) GCC installations into the same prefix.  (Which would resolve this
> > problem of clashing file names for target and offloading compilers for
> > good.)
> > 
> > So, the next question is, instead of this special handling, why can't we
> > require the offloading compilers to always be configured as cross
> > compilers?  Or, why is it a requirement that the intelmic offloading
> > compiler is configured as a native compiler?
> 
> If I understand correctly, to build a cross compiler, we need to specify a path
> to the target sysroot, even for x86_64-pc-linux-gnu to x86_64-intelmic-linux-gnu
> cross.  Or is it possible to build a cross compiler without --with-sysroot ?

To be precise, for the cross compiler we need to specify a path to
--with-build-time-tools (rather than --with-sysroot).  The problem is that for
Intel MIC there are no special as/ld/etc.  So, is there an elegant way to build
a cross compiler with host's build time tools?

Thanks,
  -- Ilya
Ilya Verbin Feb. 26, 2015, 7:20 p.m. UTC | #8
On Fri, Feb 20, 2015 at 22:48:52 +0300, Ilya Verbin wrote:
> On Fri, Feb 20, 2015 at 18:05:01 +0300, Ilya Verbin wrote:
> > On Thu, Feb 19, 2015 at 13:17:37 +0100, Thomas Schwinge wrote:
> > > My asumption is that it is always safe to install non-native (that is
> > > cross) GCC installations into the same prefix.  (Which would resolve this
> > > problem of clashing file names for target and offloading compilers for
> > > good.)
> > > 
> > > So, the next question is, instead of this special handling, why can't we
> > > require the offloading compilers to always be configured as cross
> > > compilers?  Or, why is it a requirement that the intelmic offloading
> > > compiler is configured as a native compiler?
> > 
> > If I understand correctly, to build a cross compiler, we need to specify a path
> > to the target sysroot, even for x86_64-pc-linux-gnu to x86_64-intelmic-linux-gnu
> > cross.  Or is it possible to build a cross compiler without --with-sysroot ?
> 
> To be precise, for the cross compiler we need to specify a path to
> --with-build-time-tools (rather than --with-sysroot).  The problem is that for
> Intel MIC there are no special as/ld/etc.  So, is there an elegant way to build
> a cross compiler with host's build time tools?

Probably one can build an offloading cross compiler as:
configure --target=x86_64-intelmic-linux-gnu --enable-as-accelerator-for=x86_64-pc-linux-gnu --with-build-time-tools=/usr/bin/
?

But I'm getting errors for such configuration :(

In file included from ../../../gcc/libgcc/gthr.h:148:0,
                 from ../../../gcc/libgcc/libgcov-interface.c:27:
./gthr-default.h:35:21: fatal error: pthread.h: No such file or directory
compilation terminated.
make[2]: *** [_gcov_dump.o] Error 1
make[2]: Leaving directory `/build/x86_64-intelmic-linux-gnu/libgcc'
make[1]: *** [all-target-libgcc] Error 2
make[1]: Leaving directory `/build'
make: *** [all] Error 2

I really want to remove the intelmicemul target, since there is no difference in
the compiler between real and emulated compilation for Intel MIC.  The only
difference is which libcoi_host.so is used by liboffloadmic_host.so at run-time.

Thanks,
  -- Ilya
Thomas Schwinge March 10, 2015, 12:34 p.m. UTC | #9
Hi!

All the "offloading folks" agree, but we need someone to "formally
approve" this patch?


On Fri, 20 Feb 2015 22:27:43 +0300, Ilya Verbin <iverbin@gmail.com> wrote:
> On Fri, Feb 20, 2015 at 10:27:26 +0100, Thomas Schwinge wrote:
> > On Thu, 19 Feb 2015 10:28:46 +0100, Bernd Schmidt <bernds@codesourcery.com> wrote:
> > > issue when trying to 
> > > get at the libgcc for the nvptx accel compiler after it's been 
> > > installed. The libgcc Makefile puts it in the wrong place - 
> > > gcc/nvptx-none/accel/nvptx-none instead of gcc/host/accel/nvptx-none. 
> > > The patch below corrects that and removes an intelmicemul special case 
> > > which I believe has the same effect - Ilya, could you test this?
> > 
> > Works fine for me for intelmic (no changes), and nvptx (changes as
> > expected).
> 
> OK to me.


Grüße,
 Thomas
Thomas Schwinge April 27, 2015, 4:14 p.m. UTC | #10
Hi!

Ping.  (Or can Bernd just commit this patch,
<http://news.gmane.org/find-root.php?message_id=%3C54E5ACCE.7080502%40codesourcery.com%3E>
(with my review comment addressed,
<http://news.gmane.org/find-root.php?message_id=%3C87h9uhlypt.fsf%40schwinge.name%3E>),
given his nvptx architecture maintainership?)

On Tue, 10 Mar 2015 13:34:51 +0100, I wrote:
> All the "offloading folks" agree, but we need someone to "formally
> approve" this patch?
> 
> 
> On Fri, 20 Feb 2015 22:27:43 +0300, Ilya Verbin <iverbin@gmail.com> wrote:
> > On Fri, Feb 20, 2015 at 10:27:26 +0100, Thomas Schwinge wrote:
> > > On Thu, 19 Feb 2015 10:28:46 +0100, Bernd Schmidt <bernds@codesourcery.com> wrote:
> > > > issue when trying to 
> > > > get at the libgcc for the nvptx accel compiler after it's been 
> > > > installed. The libgcc Makefile puts it in the wrong place - 
> > > > gcc/nvptx-none/accel/nvptx-none instead of gcc/host/accel/nvptx-none. 
> > > > The patch below corrects that and removes an intelmicemul special case 
> > > > which I believe has the same effect - Ilya, could you test this?
> > > 
> > > Works fine for me for intelmic (no changes), and nvptx (changes as
> > > expected).
> > 
> > OK to me.


Grüße,
 Thomas
Jakub Jelinek April 27, 2015, 4:16 p.m. UTC | #11
On Mon, Apr 27, 2015 at 06:14:36PM +0200, Thomas Schwinge wrote:
> Hi!
> 
> Ping.  (Or can Bernd just commit this patch,
> <http://news.gmane.org/find-root.php?message_id=%3C54E5ACCE.7080502%40codesourcery.com%3E>
> (with my review comment addressed,
> <http://news.gmane.org/find-root.php?message_id=%3C87h9uhlypt.fsf%40schwinge.name%3E>),
> given his nvptx architecture maintainership?)

Ok for trunk.

	Jakub
diff mbox

Patch

Index: libgcc/Makefile.in
===================================================================
--- libgcc/Makefile.in	(revision 445788)
+++ libgcc/Makefile.in	(working copy)
@@ -45,6 +45,7 @@  fixed_point = @fixed_point@
 with_aix_soname = @with_aix_soname@
 
 host_noncanonical = @host_noncanonical@
+real_host_noncanonical = @real_host_noncanonical@
 target_noncanonical = @target_noncanonical@
 
 # List of extra object files that should be compiled for this target machine.
@@ -185,7 +186,7 @@  STRIP = @STRIP@
 STRIP_FOR_TARGET = $(STRIP)
 
 # Directory in which the compiler finds libraries etc.
-libsubdir = $(libdir)/gcc/$(host_noncanonical)/$(version)@accel_dir_suffix@
+libsubdir = $(libdir)/gcc/$(real_host_noncanonical)/$(version)@accel_dir_suffix@
 # Used to install the shared libgcc.
 slibdir = @slibdir@
 # Maybe used for DLLs on Windows targets.
Index: libgcc/configure.ac
===================================================================
--- libgcc/configure.ac	(revision 445788)
+++ libgcc/configure.ac	(working copy)
@@ -398,16 +398,14 @@  esac
 
 # Used for constructing correct paths for offload compilers.
 accel_dir_suffix=
+real_host_noncanonical=${host_noncanonical}
+echo "eaaf: $enable_as_accelerator_for"
 if test x"$enable_as_accelerator_for" != x; then
   accel_dir_suffix=/accel/${target_noncanonical}
-  case "${target_noncanonical}" in
-    *-intelmicemul-*)
-      # In this case we expect offload compiler to be built as native, so we
-      # need to change install directory for driver to be able to find libgcc.
-      host_noncanonical=${enable_as_accelerator_for} ;;
-  esac
+  real_host_noncanonical=${enable_as_accelerator_for}
 fi
 AC_SUBST(accel_dir_suffix)
+AC_SUBST(real_host_noncanonical)
 
 if test x"$enable_offload_targets" != x; then
   extra_parts="${extra_parts} crtoffloadbegin.o crtoffloadend.o"
Index: libgcc/configure
===================================================================
--- libgcc/configure	(revision 445788)
+++ libgcc/configure	(working copy)
@@ -566,6 +566,7 @@  sfp_machine_header
 set_use_emutls
 set_have_cc_tls
 vis_hide
+real_host_noncanonical
 accel_dir_suffix
 force_explicit_eh_registry
 fixed_point
@@ -4482,17 +4483,15 @@  esac
 
 # Used for constructing correct paths for offload compilers.
 accel_dir_suffix=
+real_host_noncanonical=${host_noncanonical}
+echo "eaaf: $enable_as_accelerator_for"
 if test x"$enable_as_accelerator_for" != x; then
   accel_dir_suffix=/accel/${target_noncanonical}
-  case "${target_noncanonical}" in
-    *-intelmicemul-*)
-      # In this case we expect offload compiler to be built as native, so we
-      # need to change install directory for driver to be able to find libgcc.
-      host_noncanonical=${enable_as_accelerator_for} ;;
-  esac
+  real_host_noncanonical=${enable_as_accelerator_for}
 fi
 
 
+
 if test x"$enable_offload_targets" != x; then
   extra_parts="${extra_parts} crtoffloadbegin.o crtoffloadend.o"
 fi