diff mbox

[RFC,Offloading] Fix PR68463

Message ID 20160210171934.GA30239@msticlxl57.ims.intel.com
State New
Headers show

Commit Message

Ilya Verbin Feb. 10, 2016, 5:19 p.m. UTC
Hi!

On Tue, Jan 19, 2016 at 16:32:13 +0300, Ilya Verbin wrote:
> On Tue, Jan 19, 2016 at 10:36:28 +0100, Jakub Jelinek wrote:
> > On Tue, Jan 19, 2016 at 09:57:01AM +0100, Richard Biener wrote:
> > > On Mon, 18 Jan 2016, Ilya Verbin wrote:
> > > > On Fri, Jan 15, 2016 at 09:15:01 +0100, Richard Biener wrote:
> > > > > On Fri, 15 Jan 2016, Ilya Verbin wrote:
> > > > > > II) The __offload_func_table, __offload_funcs_end, __offload_var_table,
> > > > > > __offload_vars_end are now provided by the linker script, instead of
> > > > > > crtoffload{begin,end}.o, this allows to surround all offload objects, even
> > > > > > those that are not claimed by lto-plugin.
> > > > > > Unfortunately it works only with ld, but doen't work with gold, because
> > > > > > https://sourceware.org/bugzilla/show_bug.cgi?id=15373
> > > > > > Any thoughts how to enable this linker script for gold?
> > > > > 
> > > > > The easiest way would probably to add this handling to the default
> > > > > "linker script" in gold.  I don't see an easy way around requiring
> > > > > changes to gold here - maybe dumping the default linker script from
> > > > > bfd and injecting the rules with some scripting so you have a complete
> > > > > script.  Though likely gold won't grok that result.
> > > > > 
> > > > > Really a question for Ian though.
> > > > 
> > > > Or the gcc driver can add crtoffload{begin,end}.o, but the problem is that it
> > > > can't determine whether the program contains offloading or not.  So it can add
> > > > them to all -fopenmp/-fopenacc programs, if the compiler was configured with
> > > > --enable-offload-targets=...  The overhead would be about 340 bytes for
> > > > binaries which doesn't use offloading.  Is this acceptable?  (Jakub?)
> > > 
> > > Can lto-wrapper add them as plugin outputs?  Or does that wreck ordering?
> 
> Currently it's implemented this way, but it will not work after my patch,
> because e.g. offload-without-lto.o and offload-with-lto.o will be linked in
> this order:
> offload-without-lto.o, crtoffloadbegin.o, offload-with-lto.o, crtoffloadend.o
> ^^^^^^^^^^^^^^^^^^^^^
> (will be not claimed by the plugin)
> 
> But we need this one:
> crtoffloadbegin.o, offload-without-lto.o, offload-with-lto.o, crtoffloadend.o
> 
> > Yeah, if that would work, it would be certainly appreciated, one thing is
> > wasting .text space and relocations in all -fopenmp programs (for -fopenacc
> > programs one kind of assumes there will be some offloading in there),
> > another one some extra constructor/destructor or what that would be even
> > worse.
> 
> They contain only 5 symbols, without constructors/destructors.

This patch adds crtoffload{begin,end}.o to all -fopenmp programs, if they exist.
I couldn't think of a better solution...
Tested using the testcase from the previous mail, e.g.:

$ gcc -DNUM=1 -c -fopenmp test.c -o obj1.o
$ gcc -DNUM=2 -c -fopenmp test.c -o obj2.o
$ gcc -DNUM=3 -c -fopenmp test.c -o obj3.o
$ gcc -DNUM=4 -c -fopenmp test.c -o obj4.o -flto
$ gcc -DNUM=5 -c -fopenmp test.c -o obj5.o
$ gcc -DNUM=6 -c -fopenmp test.c -o obj6.o -flto
$ gcc -DNUM=7 -c -fopenmp test.c -o obj7.o
$ gcc-ar -cvq libtest.a obj3.o obj4.o obj5.o
$ gcc -fopenmp main.c obj1.o obj2.o libtest.a obj6.o obj7.o

And other combinations.


gcc/
	PR driver/68463
	* config/gnu-user.h (GNU_USER_TARGET_STARTFILE_SPEC): Add
	crtoffloadbegin.o for -fopenacc/-fopenmp if it exists.
	(GNU_USER_TARGET_ENDFILE_SPEC): Add crtoffloadend.o for
	-fopenacc/-fopenmp if it exists.
	* lto-wrapper.c (offloadbegin, offloadend): Remove static vars.
	(offload_objects_file_name): New static var.
	(tool_cleanup): Remove offload_objects_file_name file.
	(copy_file): Remove function.
	(find_offloadbeginend): Remove function.
	(run_gcc): Remove offload_argc and offload_argv.
	Get offload_objects_file_name from -foffload-objects=... option.
	Read names of object files with offload from this file, pass them to
	compile_images_for_offload_targets.  Don't call find_offloadbeginend and
	don't pass offloadbegin and offloadend to the linker.  Don't pass
	offload non-LTO files to the linker, because now they're not claimed.
lto-plugin/
	PR driver/68463
	* lto-plugin.c (struct plugin_offload_file): New.
	(offload_files): Change type.
	(offload_files_last, offload_files_last_obj): New.
	(offload_files_last_lto): New.
	(free_2): Adjust accordingly.
	(all_symbols_read_handler): Don't add offload files to lto_arg_ptr.
	Don't call free_1 for offload_files.  Write names of object files with
	offloading to the temporary file.  Add new option to lto_arg_ptr.
	(claim_file_handler): Don't claim file if it contains offload sections
	without LTO sections.  If it contains offload sections, add to the list.



  -- Ilya

Comments

Jakub Jelinek Feb. 19, 2016, 2:53 p.m. UTC | #1
On Wed, Feb 10, 2016 at 08:19:34PM +0300, Ilya Verbin wrote:
> This patch adds crtoffload{begin,end}.o to all -fopenmp programs, if they exist.
> I couldn't think of a better solution...
> Tested using the testcase from the previous mail, e.g.:
> 
> $ gcc -DNUM=1 -c -fopenmp test.c -o obj1.o
> $ gcc -DNUM=2 -c -fopenmp test.c -o obj2.o
> $ gcc -DNUM=3 -c -fopenmp test.c -o obj3.o
> $ gcc -DNUM=4 -c -fopenmp test.c -o obj4.o -flto
> $ gcc -DNUM=5 -c -fopenmp test.c -o obj5.o
> $ gcc -DNUM=6 -c -fopenmp test.c -o obj6.o -flto
> $ gcc -DNUM=7 -c -fopenmp test.c -o obj7.o
> $ gcc-ar -cvq libtest.a obj3.o obj4.o obj5.o
> $ gcc -fopenmp main.c obj1.o obj2.o libtest.a obj6.o obj7.o
> 
> And other combinations.

Looking at this, I think I have no problem with crtoffloadbegin.o being
included in all -fopenmp/-fopenacc linked programs/shared libraries,
that just defines the symbols and nothing else.
I have no problem with the
__offload_funcs_end/__offload_vars_end part of crtoffloadend.o being
included too.
But, I really don't like __OFFLOAD_TABLE__ being added to all programs, that
wastes real space in data (rodata or relro?) section, and dynamic
relocations.
So, perhaps, can we split offloadstuff.c into 3 objects instead of 2,
crtoffload{begin,end,table}.o*, where the last one would be what
defines __OFFLOAD_TABLE__, and add the last one only by the linker
plugin/lto-wrapper/whatever, if any input objects had any offloading stuff
in it?

	Jakub
Mike Stump Feb. 19, 2016, 5:54 p.m. UTC | #2
On Feb 19, 2016, at 6:53 AM, Jakub Jelinek <jakub@redhat.com> wrote:
> Looking at this, I think I have no problem with crtoffloadbegin.o being
> included in all -fopenmp/-fopenacc linked programs/shared libraries,

:-)  I have a problem with just the normal init path in most executables.  It adds a ton of stuff that can be empty at the bottom.  I sometimes wonder if we boosted it to -flto, and then let lto see the size of the table, and put all the init code under an early if (count) { do the init stuff; }, then given the count, lto can then just remove it all, reliably.

If the openmp people want to experiment with -flto and see if they can make the whole thing disappear that way, it might be worth considering.

But, yes, I agree, hard to want yet more included by default that just won’t go away.
diff mbox

Patch

diff --git a/gcc/config/gnu-user.h b/gcc/config/gnu-user.h
index 2f1bbcc..2fdb63c 100644
--- a/gcc/config/gnu-user.h
+++ b/gcc/config/gnu-user.h
@@ -49,14 +49,16 @@  see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
 	      %{" NO_PIE_SPEC ":crtbegin.o%s}} \
    %{fvtable-verify=none:%s; \
      fvtable-verify=preinit:vtv_start_preinit.o%s; \
-     fvtable-verify=std:vtv_start.o%s}"
+     fvtable-verify=std:vtv_start.o%s} \
+   %{fopenacc|fopenmp:%:if-exists(crtoffloadbegin%O%s)}"
 #else
 #define GNU_USER_TARGET_STARTFILE_SPEC \
   "%{!shared: %{pg|p|profile:gcrt1.o%s;:crt1.o%s}} \
    crti.o%s %{static:crtbeginT.o%s;shared|pie:crtbeginS.o%s;:crtbegin.o%s} \
    %{fvtable-verify=none:%s; \
      fvtable-verify=preinit:vtv_start_preinit.o%s; \
-     fvtable-verify=std:vtv_start.o%s}"
+     fvtable-verify=std:vtv_start.o%s} \
+   %{fopenacc|fopenmp:%:if-exists(crtoffloadbegin%O%s)}"
 #endif
 #undef  STARTFILE_SPEC
 #define STARTFILE_SPEC GNU_USER_TARGET_STARTFILE_SPEC
@@ -73,13 +75,15 @@  see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
      fvtable-verify=preinit:vtv_end_preinit.o%s; \
      fvtable-verify=std:vtv_end.o%s} \
    %{shared:crtendS.o%s;: %{" PIE_SPEC ":crtendS.o%s} \
-   %{" NO_PIE_SPEC ":crtend.o%s}} crtn.o%s"
+   %{" NO_PIE_SPEC ":crtend.o%s}} crtn.o%s \
+   %{fopenacc|fopenmp:%:if-exists(crtoffloadend%O%s)}"
 #else
 #define GNU_USER_TARGET_ENDFILE_SPEC \
   "%{fvtable-verify=none:%s; \
      fvtable-verify=preinit:vtv_end_preinit.o%s; \
      fvtable-verify=std:vtv_end.o%s} \
-   %{shared|pie:crtendS.o%s;:crtend.o%s} crtn.o%s"
+   %{shared|pie:crtendS.o%s;:crtend.o%s} crtn.o%s \
+   %{fopenacc|fopenmp:%:if-exists(crtoffloadend%O%s)}"
 #endif
 #undef  ENDFILE_SPEC
 #define ENDFILE_SPEC GNU_USER_TARGET_ENDFILE_SPEC
diff --git a/gcc/lto-wrapper.c b/gcc/lto-wrapper.c
index ed20b4e..f2914d0 100644
--- a/gcc/lto-wrapper.c
+++ b/gcc/lto-wrapper.c
@@ -68,7 +68,7 @@  static unsigned int nr;
 static char **input_names;
 static char **output_names;
 static char **offload_names;
-static const char *offloadbegin, *offloadend;
+static char *offload_objects_file_name;
 static char *makefile;
 
 const char tool_name[] = "lto-wrapper";
@@ -84,6 +84,8 @@  tool_cleanup (bool)
     maybe_unlink (ltrans_output_file);
   if (flto_out)
     maybe_unlink (flto_out);
+  if (offload_objects_file_name)
+    maybe_unlink (offload_objects_file_name);
   if (makefile)
     maybe_unlink (makefile);
   for (i = 0; i < nr; ++i)
@@ -818,66 +820,6 @@  compile_images_for_offload_targets (unsigned in_argc, char *in_argv[],
   free_array_of_ptrs ((void **) names, num_targets);
 }
 
-/* Copy a file from SRC to DEST.  */
-
-static void
-copy_file (const char *dest, const char *src)
-{
-  FILE *d = fopen (dest, "wb");
-  FILE *s = fopen (src, "rb");
-  char buffer[512];
-  while (!feof (s))
-    {
-      size_t len = fread (buffer, 1, 512, s);
-      if (ferror (s) != 0)
-	fatal_error (input_location, "reading input file");
-      if (len > 0)
-	{
-	  fwrite (buffer, 1, len, d);
-	  if (ferror (d) != 0)
-	    fatal_error (input_location, "writing output file");
-	}
-    }
-}
-
-/* Find the crtoffloadbegin.o and crtoffloadend.o files in LIBRARY_PATH, make
-   copies and store the names of the copies in offloadbegin and offloadend.  */
-
-static void
-find_offloadbeginend (void)
-{
-  char **paths = NULL;
-  const char *library_path = getenv ("LIBRARY_PATH");
-  if (!library_path)
-    return;
-  unsigned n_paths = parse_env_var (library_path, &paths, "/crtoffloadbegin.o");
-
-  unsigned i;
-  for (i = 0; i < n_paths; i++)
-    if (access_check (paths[i], R_OK) == 0)
-      {
-	size_t len = strlen (paths[i]);
-	char *tmp = xstrdup (paths[i]);
-	strcpy (paths[i] + len - strlen ("begin.o"), "end.o");
-	if (access_check (paths[i], R_OK) != 0)
-	  fatal_error (input_location,
-		       "installation error, can't find crtoffloadend.o");
-	/* The linker will delete the filenames we give it, so make
-	   copies.  */
-	offloadbegin = make_temp_file (".o");
-	offloadend = make_temp_file (".o");
-	copy_file (offloadbegin, tmp);
-	copy_file (offloadend, paths[i]);
-	free (tmp);
-	break;
-      }
-  if (i == n_paths)
-    fatal_error (input_location,
-		 "installation error, can't find crtoffloadbegin.o");
-
-  free_array_of_ptrs ((void **) paths, n_paths);
-}
-
 /* A subroutine of run_gcc.  Examine the open file FD for lto sections with
    name prefix PREFIX, at FILE_OFFSET, and store any options we find in OPTS
    and OPT_COUNT.  Return true if we found a matchingn section, false
@@ -970,8 +912,8 @@  run_gcc (unsigned argc, char *argv[])
   int new_head_argc;
   bool have_lto = false;
   bool have_offload = false;
-  unsigned lto_argc = 0, offload_argc = 0;
-  char **lto_argv, **offload_argv;
+  unsigned lto_argc = 0;
+  char **lto_argv;
 
   /* Get the driver and options.  */
   collect_gcc = getenv ("COLLECT_GCC");
@@ -987,10 +929,9 @@  run_gcc (unsigned argc, char *argv[])
 					&decoded_options,
 					&decoded_options_count);
 
-  /* Allocate arrays for input object files with LTO or offload IL,
+  /* Allocate array for input object files with LTO IL,
      and for possible preceding arguments.  */
   lto_argv = XNEWVEC (char *, argc);
-  offload_argv = XNEWVEC (char *, argc);
 
   /* Look at saved options in the IL files.  */
   for (i = 1; i < argc; ++i)
@@ -1002,6 +943,15 @@  run_gcc (unsigned argc, char *argv[])
       int consumed;
       char *filename = argv[i];
 
+      if (strncmp (argv[i], "-foffload-objects=",
+		   sizeof ("-foffload-objects=") - 1) == 0)
+	{
+	  have_offload = true;
+	  offload_objects_file_name
+	    = argv[i] + sizeof ("-foffload-objects=") - 1;
+	  continue;
+	}
+
       if ((p = strrchr (argv[i], '@'))
 	  && p != argv[i] 
 	  && sscanf (p, "@%li%n", &loffset, &consumed) >= 1
@@ -1026,15 +976,6 @@  run_gcc (unsigned argc, char *argv[])
 	  have_lto = true;
 	  lto_argv[lto_argc++] = argv[i];
 	}
-
-      if (find_and_merge_options (fd, file_offset, OFFLOAD_SECTION_NAME_PREFIX,
-				  &offload_fdecoded_options,
-				  &offload_fdecoded_options_count, collect_gcc))
-	{
-	  have_offload = true;
-	  offload_argv[offload_argc++] = argv[i];
-	}
-
       close (fd);
     }
 
@@ -1133,47 +1074,101 @@  run_gcc (unsigned argc, char *argv[])
 
   if (have_offload)
     {
-      compile_images_for_offload_targets (offload_argc, offload_argv,
+      unsigned i, num_offload_files;
+      char **offload_argv;
+      FILE *f;
+
+      f = fopen (offload_objects_file_name, "r");
+      if (f == NULL)
+	fatal_error (input_location, "cannot open %s: %m",
+		     offload_objects_file_name);
+      if (fscanf (f, "%u ", &num_offload_files) != 1)
+	fatal_error (input_location, "cannot read %s: %m",
+		     offload_objects_file_name);
+      offload_argv = XCNEWVEC (char *, num_offload_files);
+
+      /* Read names of object files with offload.  */
+      for (i = 0; i < num_offload_files; i++)
+	{
+	  const unsigned piece = 32;
+	  char *buf, *filename = XNEWVEC (char, piece);
+	  size_t len;
+
+	  buf = filename;
+cont1:
+	  if (!fgets (buf, piece, f))
+	    break;
+	  len = strlen (filename);
+	  if (filename[len - 1] != '\n')
+	    {
+	      filename = XRESIZEVEC (char, filename, len + piece);
+	      buf = filename + len;
+	      goto cont1;
+	    }
+	  filename[len - 1] = '\0';
+	  offload_argv[i] = filename;
+	}
+      fclose (f);
+      if (offload_argv[num_offload_files - 1] == NULL)
+	fatal_error (input_location, "invalid format of %s",
+		     offload_objects_file_name);
+      maybe_unlink (offload_objects_file_name);
+      offload_objects_file_name = NULL;
+
+      /* Look at saved offload options in files.  */
+      for (i = 0; i < num_offload_files; i++)
+	{
+	  char *p;
+	  long loffset;
+	  int fd, consumed;
+	  off_t file_offset = 0;
+	  char *filename = offload_argv[i];
+
+	  if ((p = strrchr (offload_argv[i], '@'))
+	      && p != offload_argv[i]
+	      && sscanf (p, "@%li%n", &loffset, &consumed) >= 1
+	      && strlen (p) == (unsigned int) consumed)
+	    {
+	      filename = XNEWVEC (char, p - offload_argv[i] + 1);
+	      memcpy (filename, offload_argv[i], p - offload_argv[i]);
+	      filename[p - offload_argv[i]] = '\0';
+	      file_offset = (off_t) loffset;
+	    }
+	  fd = open (filename, O_RDONLY | O_BINARY);
+	  if (fd == -1)
+	    fatal_error (input_location, "cannot open %s: %m", filename);
+	  if (!find_and_merge_options (fd, file_offset,
+				       OFFLOAD_SECTION_NAME_PREFIX,
+				       &offload_fdecoded_options,
+				       &offload_fdecoded_options_count,
+				       collect_gcc))
+	    fatal_error (input_location, "cannot read %s: %m", filename);
+	  close (fd);
+	  if (filename != offload_argv[i])
+	    XDELETEVEC (filename);
+	}
+
+      compile_images_for_offload_targets (num_offload_files, offload_argv,
 					  offload_fdecoded_options,
 					  offload_fdecoded_options_count,
 					  decoded_options,
 					  decoded_options_count);
+
+      free_array_of_ptrs ((void **) offload_argv, num_offload_files);
+
       if (offload_names)
 	{
-	  find_offloadbeginend ();
 	  for (i = 0; offload_names[i]; i++)
 	    printf ("%s\n", offload_names[i]);
 	  free_array_of_ptrs ((void **) offload_names, i);
 	}
     }
 
-  if (offloadbegin)
-    printf ("%s\n", offloadbegin);
-
   /* If object files contain offload sections, but do not contain LTO sections,
      then there is no need to perform a link-time recompilation, i.e.
      lto-wrapper is used only for a compilation of offload images.  */
   if (have_offload && !have_lto)
-    {
-      for (i = 1; i < argc; ++i)
-	if (strncmp (argv[i], "-fresolution=",
-		     sizeof ("-fresolution=") - 1) != 0
-	    && strncmp (argv[i], "-flinker-output=",
-			sizeof ("-flinker-output=") - 1) != 0)
-	  {
-	    char *out_file;
-	    /* Can be ".o" or ".so".  */
-	    char *ext = strrchr (argv[i], '.');
-	    if (ext == NULL)
-	      out_file = make_temp_file ("");
-	    else
-	      out_file = make_temp_file (ext);
-	    /* The linker will delete the files we give it, so make copies.  */
-	    copy_file (out_file, argv[i]);
-	    printf ("%s\n", out_file);
-	  }
-      goto finish;
-    }
+    goto finish;
 
   if (lto_mode == LTO_MODE_LTO)
     {
@@ -1402,11 +1397,7 @@  cont:
     }
 
  finish:
-  if (offloadend)
-    printf ("%s\n", offloadend);
-
   XDELETE (lto_argv);
-  XDELETE (offload_argv);
   obstack_free (&argv_obstack, NULL);
 }
 
diff --git a/lto-plugin/lto-plugin.c b/lto-plugin/lto-plugin.c
index 1ed0f08..9aba151 100644
--- a/lto-plugin/lto-plugin.c
+++ b/lto-plugin/lto-plugin.c
@@ -129,6 +129,14 @@  struct plugin_file_info
   struct plugin_symtab conflicts;
 };
 
+/* List item with name of the file with offloading.  */
+
+struct plugin_offload_file
+{
+  char *name;
+  struct plugin_offload_file *next;
+};
+
 /* Until ASM_OUTPUT_LABELREF can be hookized and decoupled from
    stdio file streams, we do simple label translation here.  */
 
@@ -152,8 +160,16 @@  static ld_plugin_add_symbols add_symbols;
 static struct plugin_file_info *claimed_files = NULL;
 static unsigned int num_claimed_files = 0;
 
-static struct plugin_file_info *offload_files = NULL;
-static unsigned int num_offload_files = 0;
+/* List of files with offloading.  */
+static struct plugin_offload_file *offload_files;
+/* Last file in the list.  */
+static struct plugin_offload_file *offload_files_last;
+/* Last non-archive file in the list.  */
+static struct plugin_offload_file *offload_files_last_obj;
+/* Last LTO file in the list.  */
+static struct plugin_offload_file *offload_files_last_lto;
+/* Total number of files with offloading.  */
+static unsigned num_offload_files;
 
 static char **output_files = NULL;
 static unsigned int num_output_files = 0;
@@ -351,14 +367,6 @@  free_2 (void)
       free (info->name);
     }
 
-  for (i = 0; i < num_offload_files; i++)
-    {
-      struct plugin_file_info *info = &offload_files[i];
-      struct plugin_symtab *symtab = &info->symtab;
-      free (symtab->aux);
-      free (info->name);
-    }
-
   for (i = 0; i < num_output_files; i++)
     free (output_files[i]);
   free (output_files);
@@ -367,8 +375,12 @@  free_2 (void)
   claimed_files = NULL;
   num_claimed_files = 0;
 
-  free (offload_files);
-  offload_files = NULL;
+  while (offload_files)
+    {
+      struct plugin_offload_file *ofld = offload_files;
+      offload_files = offload_files->next;
+      free (ofld);
+    }
   num_offload_files = 0;
 
   free (arguments_file_name);
@@ -625,8 +637,7 @@  static enum ld_plugin_status
 all_symbols_read_handler (void)
 {
   unsigned i;
-  unsigned num_lto_args
-    = num_claimed_files + num_offload_files + lto_wrapper_num_args + 2;
+  unsigned num_lto_args = num_claimed_files + lto_wrapper_num_args + 3;
   char **lto_argv;
   const char *linker_output_str = NULL;
   const char **lto_arg_ptr;
@@ -646,7 +657,6 @@  all_symbols_read_handler (void)
   write_resolution ();
 
   free_1 (claimed_files, num_claimed_files);
-  free_1 (offload_files, num_offload_files);
 
   for (i = 0; i < lto_wrapper_num_args; i++)
     *lto_arg_ptr++ = lto_wrapper_argv[i];
@@ -671,16 +681,37 @@  all_symbols_read_handler (void)
       break;
     }
   *lto_arg_ptr++ = xstrdup (linker_output_str);
-  for (i = 0; i < num_claimed_files; i++)
+
+  if (num_offload_files > 0)
     {
-      struct plugin_file_info *info = &claimed_files[i];
+      FILE *f;
+      char *arg;
+      char *offload_objects_file_name;
+      struct plugin_offload_file *ofld;
+
+      offload_objects_file_name = make_temp_file (".ofldlist");
+      check (offload_objects_file_name, LDPL_FATAL,
+	     "Failed to generate a temporary file name");
+      f = fopen (offload_objects_file_name, "w");
+      check (f, LDPL_FATAL, "could not open file with offload objects");
+      fprintf (f, "%u\n", num_offload_files);
+
+      ofld = offload_files;
+      while (ofld)
+	{
+	  fprintf (f, "%s\n", ofld->name);
+	  ofld = ofld->next;
+	}
+      fclose (f);
 
-      *lto_arg_ptr++ = info->name;
+      arg = concat ("-foffload-objects=", offload_objects_file_name, NULL);
+      check (arg, LDPL_FATAL, "could not allocate");
+      *lto_arg_ptr++ = arg;
     }
 
-  for (i = 0; i < num_offload_files; i++)
+  for (i = 0; i < num_claimed_files; i++)
     {
-      struct plugin_file_info *info = &offload_files[i];
+      struct plugin_file_info *info = &claimed_files[i];
 
       *lto_arg_ptr++ = info->name;
     }
@@ -1007,19 +1038,63 @@  claim_file_handler (const struct ld_plugin_input_file *file, int *claimed)
 	xrealloc (claimed_files,
 		  num_claimed_files * sizeof (struct plugin_file_info));
       claimed_files[num_claimed_files - 1] = lto_file;
+
+      *claimed = 1;
     }
 
-  if (obj.found == 0 && obj.offload == 1)
+  if (obj.offload == 1)
     {
+      struct plugin_offload_file *ofld
+	= xmalloc (sizeof (struct plugin_offload_file));
+      ofld->name = lto_file.name;
+      ofld->next = NULL;
+
+      if (offload_files == NULL)
+	offload_files = ofld;
+
+      /* Add file to the list.  The order must be exactly the same as the final
+	 order after recompilation and linking, otherwise host and target tables
+	 with addresses wouldn't match.  If a static library contains both LTO
+	 and non-LTO objects, ld and gold link them in a different order.  */
+      if (*claimed && offload_files_last_lto == NULL && file->offset != 0
+	  && gold_version == -1)
+	{
+	  /* ld only: insert first LTO file from the archive after the last real
+	     object file immediately preceding the archive, or at the begin of
+	     the list if there was no real objects before archives.  */
+	  if (offload_files_last_obj != NULL)
+	    {
+	      ofld->next = offload_files_last_obj->next;
+	      offload_files_last_obj->next = ofld;
+	    }
+	  else if (offload_files != ofld)
+	    {
+	      ofld->next = offload_files;
+	      offload_files = ofld;
+	    }
+	}
+      else if (*claimed && offload_files_last_lto != NULL)
+	{
+	  /* Insert LTO file after the last LTO file in the list.  */
+	  ofld->next = offload_files_last_lto->next;
+	  offload_files_last_lto->next = ofld;
+	}
+      else if (offload_files_last != NULL)
+	{
+	  /* Add non-LTO file or first non-archive LTO file to the end of the
+	     list.  */
+	  offload_files_last->next = ofld;
+	}
+
+      if (ofld->next == NULL)
+	offload_files_last = ofld;
+      if (file->offset == 0)
+	offload_files_last_obj = ofld;
+      if (*claimed)
+	offload_files_last_lto = ofld;
       num_offload_files++;
-      offload_files =
-	xrealloc (offload_files,
-		  num_offload_files * sizeof (struct plugin_file_info));
-      offload_files[num_offload_files - 1] = lto_file;
     }
 
-  *claimed = 1;
-
   goto cleanup;
 
  err: