From patchwork Tue Dec 15 15:47:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1416549 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=EGjSnqNg; dkim-atps=neutral Received: from sourceware.org (unknown [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CwN1z1JLNz9sPB for ; Wed, 16 Dec 2020 02:47:51 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 58880384242A; Tue, 15 Dec 2020 15:47:27 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732]) by sourceware.org (Postfix) with ESMTPS id E8E31384242A for ; Tue, 15 Dec 2020 15:47:22 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org E8E31384242A Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x732.google.com with SMTP id i67so11981268qkf.11 for ; Tue, 15 Dec 2020 07:47:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:message-id:date:user-agent:mime-version :content-language; bh=xHaQQcpjPJVTiqkguDZWtRKu7rW0bLIeHE9R1NgHg0k=; b=EGjSnqNgfwoX4YJ+ZhRc8RDFMrhfbhgie0Tmm1O570qN5UBR+dSDHnrpHDtqS7wcdK F4DjOEuGVeso1WWKAAKuf0xLHlTAoAvNPjn6nTQH0/ISy16CQAW4mzV3n6I5gZdD1/UD HweFelUNV5PUCSi2n0DHq4gBkhuTDTW/Hs3750WWb4nui5LwlmwX8a9Vf9beiXebpt1s qETz5HOQ9WcgydBwAZtESmC03t9l8HVrFfsEqE9QS1OY5iXkhYF2Hpsd6VmYb0oB0KJi IISsJSNxMh/ZN06pTURSUXLP9G0yNEXnSlH3vDkmhx73MdL/ceZ5JcgQEOU5Ys9xCzqi PMtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:message-id:date :user-agent:mime-version:content-language; bh=xHaQQcpjPJVTiqkguDZWtRKu7rW0bLIeHE9R1NgHg0k=; b=UiTabf7bOxjHcaOjeME3DnUqPfjUikh91K1ynMtiXk2pcgxwfTkUQDeue/Rlibt/oC U5uhZ894qhL0JhEDQG1bowVMC60wUq3BcVM3vySGBRgCHjzSm7E8nyzZJ8YHXHCK51Ac cSddd3Z+r0Yx3Cf1559j6NkyJ67XxhzST6rpt/qxE/lVZmwtMi2TxnEKuIaqFH4syNSR rIPVkjmkdH6zfPRAoeoF2U3SjiGNcAy+yS3vNIMvtAB/OLVX/f2VaBgLXjU+ffedpiSm m2jPYheTrSKPIOLbHNN/h/NSmjMx0Of7zIZfuqxNhpkYiqJY5/pSlsE4qDGfPJVmoG+C IVmQ== X-Gm-Message-State: AOAM531RHb3mTzTjsk2rbDKT9Q8KRjJHY/bL6mkpbKNvBSeFcPht4fm1 4Ow8cw3869tBZzXUzGqjAyWqj1gSMEE= X-Google-Smtp-Source: ABdhPJwp1iCM1JpkzVrdGwXaKrltIBC8RjU0KB3jR16+dnRDRQse8HqZBO0ZJr7OAwUvdVGPfoExXw== X-Received: by 2002:a37:9c82:: with SMTP id f124mr38537324qke.369.1608047242329; Tue, 15 Dec 2020 07:47:22 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:d1e:1fb7:fe72:c684? ([2620:10d:c091:480::1:d2b5]) by smtp.googlemail.com with ESMTPSA id n66sm17329034qkn.136.2020.12.15.07.47.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 15 Dec 2020 07:47:21 -0800 (PST) From: Nathan Sidwell Subject: c++: module mapper To: GCC Patches Message-ID: <1c91ef3b-2365-d955-3a7b-5a398f5ffb1b@acm.org> Date: Tue, 15 Dec 2020 10:47:20 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 Content-Language: en-US X-Spam-Status: No, score=-11.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" To avoid always requiring an active mapper to connect to, we provide a default in-process mapper with similar functionality to the sample server. This is that code. Also included is the client-side connection mechanism, which determines what server to use and how to connect to it. gcc/cp/ * Make-lang.in (CXX_AND_OBJCXX_OBJS): Add mapper-client & mapper-resolver. * mapper-client.h: New. * mapper-client.cc: New. * mapper-resolver.cc: New. pushing to trunk diff --git c/gcc/cp/Make-lang.in w/gcc/cp/Make-lang.in index 52116652900..49272464409 100644 --- c/gcc/cp/Make-lang.in +++ w/gcc/cp/Make-lang.in @@ -94,7 +94,8 @@ CXX_AND_OBJCXX_OBJS = \ cp/error.o cp/except.o cp/expr.o \ cp/friend.o cp/init.o \ cp/lambda.o cp/lex.o cp/logic.o \ - cp/mangle.o cp/method.o cp/module.o \ + cp/mangle.o cp/mapper-client.o cp/mapper-resolver.o \ + cp/method.o cp/module.o \ cp/name-lookup.o cp/optimize.o \ cp/parser.o cp/pt.o cp/ptree.o \ cp/rtti.o \ diff --git c/gcc/cp/mapper-client.cc w/gcc/cp/mapper-client.cc new file mode 100644 index 00000000000..acec591296a --- /dev/null +++ w/gcc/cp/mapper-client.cc @@ -0,0 +1,356 @@ +/* C++ modules. Experimental! + Copyright (C) 2017-2020 Free Software Foundation, Inc. + Written by Nathan Sidwell while at FaceBook + + This file is part of GCC. + + GCC is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3, or (at your option) + any later version. + + GCC is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + +You should have received a copy of the GNU General Public License +along with GCC; see the file COPYING3. If not see +. */ + +#include "config.h" +#include "system.h" + +#include "line-map.h" +#include "diagnostic-core.h" +#include "mapper-client.h" +#include "intl.h" + +#include "../../c++tools/resolver.h" + +module_client::module_client (pex_obj *p, int fd_from, int fd_to) + : Client (fd_from, fd_to), pex (p) +{ +} + +static module_client * +spawn_mapper_program (char const **errmsg, std::string &name, + char const *full_program_name) +{ + /* Split writable at white-space. No space-containing args for + you! */ + // At most every other char could be an argument + char **argv = new char *[name.size () / 2 + 2]; + unsigned arg_no = 0; + char *str = new char[name.size ()]; + memcpy (str, name.c_str () + 1, name.size ()); + + for (auto ptr = str; ; ++ptr) + { + while (*ptr == ' ') + ptr++; + if (!*ptr) + break; + + if (!arg_no) + { + /* @name means look in the compiler's install dir. */ + if (ptr[0] == '@') + ptr++; + else + full_program_name = nullptr; + } + + argv[arg_no++] = ptr; + while (*ptr && *ptr != ' ') + ptr++; + if (!*ptr) + break; + *ptr = 0; + } + argv[arg_no] = nullptr; + + auto *pex = pex_init (PEX_USE_PIPES, progname, NULL); + FILE *to = pex_input_pipe (pex, false); + name = argv[0]; + if (!to) + *errmsg = "connecting input"; + else + { + int flags = PEX_SEARCH; + + if (full_program_name) + { + /* Prepend the invoking path, if the mapper is a simple + file name. */ + size_t dir_len = progname - full_program_name; + std::string argv0; + argv0.reserve (dir_len + name.size ()); + argv0.append (full_program_name, dir_len).append (name); + name = std::move (argv0); + argv[0] = const_cast (name.c_str ()); + flags = 0; + } + int err; + *errmsg = pex_run (pex, flags, argv[0], argv, NULL, NULL, &err); + } + delete[] str; + delete[] argv; + + int fd_from = -1, fd_to = -1; + if (!*errmsg) + { + FILE *from = pex_read_output (pex, false); + if (from && (fd_to = dup (fileno (to))) >= 0) + fd_from = fileno (from); + else + *errmsg = "connecting output"; + fclose (to); + } + + if (*errmsg) + { + pex_free (pex); + return nullptr; + } + + return new module_client (pex, fd_from, fd_to); +} + +module_client * +module_client::open_module_client (location_t loc, const char *o, + void (*set_repo) (const char *), + char const *full_program_name) +{ + module_client *c = nullptr; + std::string ident; + std::string name; + char const *errmsg = nullptr; + unsigned line = 0; + + if (o && o[0]) + { + /* Maybe a local or ipv6 address. */ + name = o; + auto last = name.find_last_of ('?'); + if (last != name.npos) + { + ident = name.substr (last + 1); + name.erase (last); + } + + if (name.size ()) + { + switch (name[0]) + { + case '<': + // to or <>fromto, or <> + { + size_t pos = name.find ('>', 1); + if (pos == std::string::npos) + pos = name.size (); + std::string from (name, 1, pos - 1); + std::string to; + if (pos != name.size ()) + to.append (name, pos + 1, std::string::npos); + + int fd_from = -1, fd_to = -1; + if (from.empty () && to.empty ()) + { + fd_from = fileno (stdin); + fd_to = fileno (stdout); + } + else + { + if (!from.empty ()) + { + fd_from = std::stoul (from, &pos, 10); + if (pos != from.size ()) + { + int dir = to.empty () + ? O_RDWR | O_CLOEXEC : O_RDONLY | O_CLOEXEC; + fd_from = open (from.c_str (), dir); + } + if (to.empty ()) + fd_to = fd_from; + } + + if (!from.empty () && fd_from < 0) + ; + else if (to.empty ()) + ; + else + { + fd_to = std::stoul (to, &pos, 10); + if (pos != to.size ()) + { + int dir = from.empty () + ? O_RDWR | O_CLOEXEC : O_WRONLY | O_CLOEXEC; + fd_to = open (to.c_str (), dir); + if (fd_to < 0) + close (fd_from); + } + if (from.empty ()) + fd_from = fd_to; + } + } + + if (fd_from < 0 || fd_to < 0) + errmsg = "opening"; + else + c = new module_client (fd_from, fd_to); + } + break; + + case '=': + // =localsocket + { + int fd = -1; +#if CODY_NETWORKING + fd = Cody::OpenLocal (&errmsg, name.c_str () + 1); +#endif + if (fd >= 0) + c = new module_client (fd, fd); + } + break; + + case '|': + // |program and args + c = spawn_mapper_program (&errmsg, name, full_program_name); + break; + + default: + // file or hostname:port + { + auto colon = name.find_last_of (':'); + if (colon != name.npos) + { + char const *cptr = name.c_str () + colon; + char *endp; + unsigned port = strtoul (cptr + 1, &endp, 10); + + if (port && endp != cptr + 1 && !*endp) + { + name[colon] = 0; + int fd = 01; +#if CODY_NETWORKING + fd = Cody::OpenInet6 (&errmsg, name.c_str (), port); +#endif + name[colon] = ':'; + + if (fd >= 0) + c = new module_client (fd, fd); + } + } + + } + break; + } + } + } + + if (!c) + { + // Make a default in-process client + bool file = !errmsg && !name.empty (); + auto r = new module_resolver (!file, true); + + if (file) + { + int fd = open (name.c_str (), O_RDONLY | O_CLOEXEC); + if (fd < 0) + errmsg = "opening"; + else + { + if (int l = r->read_tuple_file (fd, ident, false)) + { + if (l > 0) + line = l; + errmsg = "reading"; + } + + close (fd); + } + } + else + r->set_repo ("gcm.cache"); + + auto *s = new Cody::Server (r); + c = new module_client (s); + } + +#ifdef SIGPIPE + if (!c->IsDirect ()) + /* We need to ignore sig pipe for a while. */ + c->sigpipe = signal (SIGPIPE, SIG_IGN); +#endif + + if (errmsg) + error_at (loc, line ? G_("failed %s mapper %qs line %u") + : G_("failed %s mapper %qs"), errmsg, name.c_str (), line); + + // now wave hello! + c->Cork (); + c->Connect (std::string ("GCC"), ident); + c->ModuleRepo (); + auto packets = c->Uncork (); + + auto &connect = packets[0]; + if (connect.GetCode () == Cody::Client::PC_CONNECT) + c->flags = Cody::Flags (connect.GetInteger ()); + else if (connect.GetCode () == Cody::Client::PC_ERROR) + error_at (loc, "failed mapper handshake %s", connect.GetString ().c_str ()); + + auto &repo = packets[1]; + if (repo.GetCode () == Cody::Client::PC_PATHNAME) + set_repo (repo.GetString ().c_str ()); + + return c; +} + +void +module_client::close_module_client (location_t loc, module_client *mapper) +{ + if (mapper->IsDirect ()) + { + auto *s = mapper->GetServer (); + auto *r = s->GetResolver (); + delete s; + delete r; + } + else + { + if (mapper->pex) + { + int fd_write = mapper->GetFDWrite (); + if (fd_write >= 0) + close (fd_write); + + int status; + pex_get_status (mapper->pex, 1, &status); + + pex_free (mapper->pex); + mapper->pex = NULL; + + if (WIFSIGNALED (status)) + error_at (loc, "mapper died by signal %s", + strsignal (WTERMSIG (status))); + else if (WIFEXITED (status) && WEXITSTATUS (status) != 0) + error_at (loc, "mapper exit status %d", + WEXITSTATUS (status)); + } + else + { + int fd_read = mapper->GetFDRead (); + close (fd_read); + } + +#ifdef SIGPIPE + // Restore sigpipe + if (mapper->sigpipe != SIG_IGN) + signal (SIGPIPE, mapper->sigpipe); +#endif + } + + delete mapper; +} diff --git c/gcc/cp/mapper-client.h w/gcc/cp/mapper-client.h new file mode 100644 index 00000000000..ca1a0aa5509 --- /dev/null +++ w/gcc/cp/mapper-client.h @@ -0,0 +1,63 @@ +/* C++ modules. Experimental! -*- c++ -*- + Copyright (C) 2020 Free Software Foundation, Inc. + Written by Nathan Sidwell while at FaceBook + + This file is part of GCC. + + GCC is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3, or (at your option) + any later version. + + GCC is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + +You should have received a copy of the GNU General Public License +along with GCC; see the file COPYING3. If not see +. */ + +/* Forward to the header in c++tools. */ + +#ifndef MAPPER_CLIENT_H +#define MAPPER_CLIENT_H 1 + +#include "cody.hh" + +#ifndef HAVE_SIGHANDLER_T +typedef void (*sighandler_t) (int); +#endif + +class module_client : public Cody::Client +{ + pex_obj *pex = nullptr; + sighandler_t sigpipe = SIG_IGN; + Cody::Flags flags = Cody::Flags::None; + +public: + module_client (Cody::Server *s) + : Client (s) + { + } + module_client (pex_obj *pex, int fd_from, int fd_to); + + module_client (int fd_from, int fd_to) + : Client (fd_from, fd_to) + { + } + +public: + Cody::Flags get_flags () const + { + return flags; + } + +public: + static module_client *open_module_client (location_t loc, const char *option, + void (*set_repo) (const char *), + char const *); + static void close_module_client (location_t loc, module_client *); +}; + +#endif diff --git c/gcc/cp/mapper-resolver.cc w/gcc/cp/mapper-resolver.cc new file mode 100644 index 00000000000..02ec48c61ea --- /dev/null +++ w/gcc/cp/mapper-resolver.cc @@ -0,0 +1,27 @@ +/* C++ modules. Experimental! -*- c++ -*- + Copyright (C) 2020 Free Software Foundation, Inc. + Written by Nathan Sidwell while at FaceBook + + This file is part of GCC. + + GCC is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3, or (at your option) + any later version. + + GCC is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + +You should have received a copy of the GNU General Public License +along with GCC; see the file COPYING3. If not see +. */ + +/* Forward to the resolver in c++tools. */ + +#include "config.h" +#define INCLUDE_ALGORITHM +#include "system.h" + +#include "../../c++tools/resolver.cc"