From patchwork Thu Sep 25 20:29:35 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Malcolm X-Patchwork-Id: 393472 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id B118B140081 for ; Fri, 26 Sep 2014 06:33:30 +1000 (EST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references; q=dns; s= default; b=efd2CDbdEHDUqhLYK0qWfZelN4X2u80vMhbnvpXzk4RdOo3tJ2Z9f mMGTsCKbDLT2YNEb+E16Q/GnfqxV6gQ2kdOE80kcTrD5bBwsIYDVKx/ncKd52KpB GLXwVK2+KYRLi4oMyV2G4EpjMSv+Ad36DlTy2W5fat2gjZVOSxADIk= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references; s= default; bh=8SAJknyGf2Q9/I0K4gfB9VmDR60=; b=btbBbtVTDCO6ZIf17chG U4yp5pmBr4+3whIh8bmRSdFIfX7q25XE7x1PkMf/5i1KPs3RafYlyzOIUoiSo7mQ 32T58tOx/JJsEyo4vBooTKC2LOtwcKuTFuC9OajyDvlN4dcG2DCIXwqtsG3bDiuy g27tsc9B54n635wh9jDuc8w= Received: (qmail 18875 invoked by alias); 25 Sep 2014 20:33:22 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 18846 invoked by uid 89); 25 Sep 2014 20:33:21 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.5 required=5.0 tests=AWL, BAYES_00, RP_MATCHES_RCVD, SPF_HELO_PASS autolearn=ham version=3.3.2 X-Spam-User: qpsmtpd, 2 recipients X-HELO: mx1.redhat.com Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES256-GCM-SHA384 encrypted) ESMTPS; Thu, 25 Sep 2014 20:33:19 +0000 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s8PKXGUj020737 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Thu, 25 Sep 2014 16:33:16 -0400 Received: from surprise.redhat.com (vpn-231-109.phx2.redhat.com [10.3.231.109]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s8PKXFYn011633; Thu, 25 Sep 2014 16:33:15 -0400 From: David Malcolm To: jit@gcc.gnu.org, gcc-patches@gcc.gnu.org Cc: "Joseph S. Myers" , David Malcolm Subject: [jit] Avoiding hardcoding "gcc"; supporting accelerators? Date: Thu, 25 Sep 2014 16:29:35 -0400 Message-Id: <1411676975-1096-1-git-send-email-dmalcolm@redhat.com> In-Reply-To: References: X-IsSubscribed: yes On Tue, 2014-09-23 at 23:27 +0000, Joseph S. Myers wrote: [...] > The code for compiling a .s file should: [...] > * use the $(target_noncanonical)-gcc-$(version) name for the driver rather > than plain "gcc", to maximise the chance that it is actually the same > compiler the JIT library was built for (I realise you may not actually > depend on it being the same compiler, but that does seem best; in > principle in future it should be possible to load multiple copies of the > JIT library to JIT for different targets, so that code for an offload > accelerator can go through the JIT). [...] Should this have the $(exeext) suffix seen in Makefile.in? $(target_noncanonical)-gcc-$(version)$(exeext) The attached patch does this, by generating a new gcc-harness-name.h and using that name in internal-api.c when invoking the harness. That said, I haven't applied this to my branch yet, since it requires the name to be findable on $PATH. Hence the user needs to (A) build and install the driver, and (B) $PATH needs to be updated accordingly. (A) is a minor extra hurdle when developing. In my prebuilt RPM packages I haven't actually been shipping the driver, just the libs, relying on a sufficiently compatible harness around. That may be just a side-effect of this not yet being in trunk. As for (B), would it make sense to "bake in" the path to the binary into the pex invocation, and hence to turn off PEX_SEARCH? If so, presumably I need to somehow expand the Makefile's value of $(bindir) into internal-api.c, right? (I tried this in configure.ac, but merely got "$(exec_prefix)/bin" iirc). A better long-term approach to this would be to extract the spec machinery from gcc.c (perhaps into a "libdriver.a"?) and run it directly from the jit library - but that's a rather involved patch, I suspect. [...] > [...] in > principle in future it should be possible to load multiple copies of the > JIT library to JIT for different targets, so that code for an offload > accelerator can go through the JIT). I had a mini-brainstorm about JIT and accelerator support; here goes... (Big caveat: the last time I hacked on GPUs was 12 years ago in my past life as a game developer, when people were publishing papers on using them for general compute purposes, iirc) I wonder if the appropriate approach here is to have a single library with multiple plugin backends e.g. one for the CPU, one for each GPU family, with the ability to load multiple "backends" at once. Unfortunately, "backend" is horribly overloaded here - I mean basically all of gcc here, everything other than the libgccjit.h API seen by client code. In this model libgccjit.h and .c become the only real part of the library, with all of the classes becoming abstract interfaces, with gcc_jit_context implemented by the plugins. The embedded copy of the bulk of gcc becomes the jit-plugin, and exposes only a very narrow API to the main library, with almost all of the names hidden from the linker. That way you could have something like (assuming x86_64-unknown-linux host cpu/os): +-----------+ +------------+ |client code|----|libgccjit.so| +-----------+ +------------+ | | +-------------------------------+ |----|jit-plugin-x86_64-unknown-linux| | +-------------------------------+ | | +---------------------------+ |----|jit-plugin-some-accelerator| | +---------------------------+ | | +---------------------------------+ |----|jit-plugin-some-other-accelerator| | +---------------------------------+ etc I believe this would allow us to have multiple copies of gcc linked into one process at once, each with their own idea of what their single target is, whilst having a single "libgccjit.so" to avoid issues where e.g. one client library want to use libgccjit.so for CPU-targetted compilation, another wants it for GPU work. Client code sends its code to a gcc_jit_context, which builds optimized code for the appropriate CPU/accelerator that it's associated with. One consistent API across CPU and all accelerators. The execution model would need refining a bit; currently a gcc_jit_result merely hands us a void* to be cast to the appropriate type. We'd need to do some extra work to get it to be runnable on the GPU presumably. (Yes I know this is feature creep, I'm thinking about gcc 6.0 or later, fwiw. But maybe it's doable without breaking the current client API). [...] Thoughts? Dave gcc/ChangeLog.jit: * configure: Regenerate. * configure.ac: Add generation of "gcc-harness-name.h". gcc/jit/ChangeLog.jit: * internal-api.c: Include "gcc-harness-name.h". (gcc::jit::playback::context::compile): Run the executable named GCC_HARNESS_NAME, rather than hardcoding "gcc". --- gcc/configure | 6 ++++++ gcc/configure.ac | 6 ++++++ gcc/jit/internal-api.c | 6 ++++-- 3 files changed, 16 insertions(+), 2 deletions(-) diff --git a/gcc/configure b/gcc/configure index c1922ac..8f2b8e7 100755 --- a/gcc/configure +++ b/gcc/configure @@ -28148,6 +28148,12 @@ _ACEOF fi +# Generate gcc-harness-name.h containing GCC_HARNESS_NAME for the benefit +# of jit/internal-api.c. +cat > gcc-harness-name.h < gcc-harness-name.h < @@ -4998,8 +4999,9 @@ compile () const char *argv[6]; int exit_status = 0; int err = 0; + const char *executable = GCC_HARNESS_NAME; - argv[0] = "gcc"; + argv[0] = executable; argv[1] = "-shared"; /* The input: assembler. */ argv[2] = m_path_s_file; @@ -5010,7 +5012,7 @@ compile () argv[5] = NULL; errmsg = pex_one (PEX_SEARCH, /* int flags, */ - "gcc", /* const char *executable */ + executable, const_cast (argv), ctxt_progname, /* const char *pname */ NULL, /* const char *outname */