From patchwork Wed Jun 28 08:09:41 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lyon X-Patchwork-Id: 781491 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3wyFp44Y0Fz9s7m for ; Wed, 28 Jun 2017 18:10:04 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.b="ce/setYz"; dkim-atps=neutral DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender :mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; q=dns; s=default; b=j0orbWO5Suit4c3 wkxJxCu0NeKoBg5EIHjuxgU41RSf9hHtFn1Mdx87K4+LiLicFytEFlRlMwhMONGB Y1IypjkZ6c7LoREbTqMt18/rr7zVEexrBeG2w/dKtuQPNjLmFF+JdAvxirHLiiTP vk0IhMGdBQYzD+OdEMgJPPUsifuQ= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender :mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; s=default; bh=k8nee9q5fi48StJ63gy67 4YFTrg=; b=ce/setYzEnKM+ve6g5UltnFxbVpWTGx+mBsYt3EL4jhDMZ1Eyb3yN 82sFAf/8wHZl+f4PYE39FhfQaH+F0XH6eObDSlr9Uv+0dZ9XEQIkA7ZaDIBYlO55 sONNPSb5NwltTlAUOX/BcnbRq5YE9MZqEt/f3lotxkbV6RZJLXe2Pk= Received: (qmail 129043 invoked by alias); 28 Jun 2017 08:09:48 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 129013 invoked by uid 89); 28 Jun 2017 08:09:46 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-24.6 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, RCVD_IN_SORBS_SPAM, SPF_PASS, URIBL_RED autolearn=ham version=3.3.2 spammy= X-HELO: mail-ua0-f175.google.com Received: from mail-ua0-f175.google.com (HELO mail-ua0-f175.google.com) (209.85.217.175) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 28 Jun 2017 08:09:43 +0000 Received: by mail-ua0-f175.google.com with SMTP id g40so33126611uaa.3 for ; Wed, 28 Jun 2017 01:09:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ibYw0kpoogpSogHfa1PojrJbDCEaP4f1mJSLmzLQ1qs=; b=qE1bbWCjNFCCpYYh+PNSBKWTU0SRMw9vYahHTY951pFsdrrCr8qBTxH/52TgO5CMM1 ycCWg6hexS+HdmutKM+e9ZeKomMRPOdPpMpikvVhBwizdY1l0MK4M8h4R3oz21WvuuhF 0D4kCJkbmDZLvdALvxqX4kijEF22FstO43hr+rOlkggSrTqIZI+nRfQn28+qyYl3Rw13 Mu7AVsTtLSmThmZIFwm8iB8hs+TBeLfz8p2OcCDl71X9ifcdiG9MuCYo7HPXP2p0uUBJ CUITehl2FR1B3DwT4ZoFX4+bk51VAQGST7MJwcJXSSkbBSAj57G/eSh87Hg0BZmonpIs 7O1A== X-Gm-Message-State: AKS2vOw3ukUEe7YdlP+X0jdnDmBA5ealWfDl6ISptpc+MbdYCn6uIuTQ B+ymcyrT6hOBGGsYxAiIy+3fGD1SZ1ta X-Received: by 10.176.76.36 with SMTP id l36mr1060uaf.63.1498637381898; Wed, 28 Jun 2017 01:09:41 -0700 (PDT) MIME-Version: 1.0 Received: by 10.103.148.220 with HTTP; Wed, 28 Jun 2017 01:09:41 -0700 (PDT) In-Reply-To: References: From: Christophe Lyon Date: Wed, 28 Jun 2017 10:09:41 +0200 Message-ID: Subject: Re: Fix genmultilib reuse rule checks for large sets of option combinations To: Joseph Myers Cc: "gcc-patches@gcc.gnu.org" X-IsSubscribed: yes On 28 June 2017 at 10:03, Christophe Lyon wrote: > Hi Joseph, > > On 8 June 2017 at 22:28, Joseph Myers wrote: >> genmultilib computes combination_space, a list of all combinations of >> options in MULTILIB_OPTIONS that might have multilibs built for them >> (some of which may end up not having multilibs built for them, and >> some of those may end up being mapped to other multilibs with >> MULTILIB_REUSE). It is then used to validate the right hand part of >> MULTILIB_REUSE rules, checking with expr that combination_space >> matches a basic regular expression derived from that right hand part. >> >> There are two problems with this approach to validation: >> >> * It requires that right hand part to have options in the same order >> as in MULTILIB_OPTIONS, in contradiction to the documentation of >> MULTILIB_REUSE saying that order does not matter there. >> >> * combination_space can be so large that the expr call fails with an >> E2BIG error. I have a local ARM configuration with 40 multilibs but >> 3840 combinations of options from MULTILIB_OPTIONS (so 3839 listed >> in combination_space, since it doesn't list the default multilib) >> and 996 MULTILIB_REUSE rules. This generates a combination_space >> string longer than the Linux kernel's MAX_ARG_STRLEN (PAGE_SIZE * >> 32, the limit on the length of a single argv string), so that expr >> cannot be run. >> >> This patch changes the validation approach to generate a much shorter >> extended regular expression for any sequence of multilib options in >> any order, and uses that for the validation instead. >> >> Tested with a built for arm-none-eabi --with-multilib-list=aprofile >> (as a configuration that uses MULTILIB_REUSE). >> >> 2017-06-08 Joseph Myers >> >> * genmultilib (combination_space): Remove variable. >> Validate reuse rules against regular expression for any sequence >> of multilib options in any order. >> >> Index: gcc/genmultilib >> =================================================================== >> --- gcc/genmultilib (revision 249028) >> +++ gcc/genmultilib (working copy) >> @@ -186,8 +186,7 @@ >> EOF >> chmod +x tmpmultilib >> >> -combination_space=`initial=/ ./tmpmultilib ${options}` >> -combinations="$combination_space" >> +combinations=`initial=/ ./tmpmultilib ${options}` >> >> # If there exceptions, weed them out now >> if [ -n "${exceptions}" ]; then >> @@ -460,6 +459,15 @@ >> echo "NULL" >> echo "};" >> >> +# Generate a regular expression to validate option combinations. >> +options_re= >> +for set in ${options}; do >> + for opt in `echo ${set} | sed -e 's_[/|]_ _g'`; do >> + options_re="${options_re}${options_re:+|}${opt}" >> + done >> +done >> +options_re="^/((${options_re})/)*\$" >> + >> # Output rules used for multilib reuse. >> echo "" >> echo "static const char *const multilib_reuse_raw[] = {" >> @@ -473,7 +481,7 @@ >> # in this variable, it means no multilib will be built for current reuse >> # rule. Thus the reuse purpose specified by current rule is meaningless. >> if expr "${combinations} " : ".*/${combo}/.*" > /dev/null; then >> - if expr "${combination_space} " : ".*/${copts}/.*" > /dev/null; then >> + if echo "/${copts}/" | grep -E "${options_re}" > /dev/null; then >> combo="/${combo}/" >> dirout=`./tmpmultilib3 "${combo}" "${todirnames}" "${toosdirnames}" "${enable_multilib}"` >> copts="/${copts}/" >> > > This broke my builds, where I do not use > --with-multilib-list=aprofile, and uses the default. > I suspect it would also fail for the aprofile multilibs, though. > > I think there's a problem with options that have a '+' which confuses > the regexp in options_re. > For instance, I get this error message: > The rule mthumb=mthumb/mfpu.auto/march.armv5te+fp contains an option > absent from MULTILIB_OPTIONS. > > A bit of manual debugging led me to: > $ echo /mthumb/mfpu=auto/march=armv5te+fp/ | grep -E > '^/((marm|mthumb|mfpu=auto|march=armv5te+fp|march=armv7+fp|mfloat-abi=hard)/)*$' > [empty result] > > Replacing the '+' in armv5te+fp with either '\+' or '.' allows the > pattern to match: > /mthumb/mfpu=auto/march=armv5te+fp/ > > Is it a matter of adding sed -e 's/\+/./g' when building options_re ? > Or would this break something else? > This small patch: works for me. If OK, I'll commit it with a suitable ChangeLog entry. Thanks, > Thanks, > > Christophe > >> -- >> Joseph S. Myers >> joseph@codesourcery.com diff --git a/gcc/genmultilib b/gcc/genmultilib index 0767e68..e65a0dd 100644 --- a/gcc/genmultilib +++ b/gcc/genmultilib @@ -462,7 +462,7 @@ echo "};" # Generate a regular expression to validate option combinations. options_re= for set in ${options}; do - for opt in `echo ${set} | sed -e 's_[/|]_ _g'`; do + for opt in `echo ${set} | sed -e 's_[/|]_ _g' | sed -e 's/\+/./g' `; do options_re="${options_re}${options_re:+|}${opt}" done done