From patchwork Mon Nov 26 15:21:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carlos O'Donell X-Patchwork-Id: 1003293 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=sourceware.org (client-ip=209.132.180.131; helo=sourceware.org; envelope-from=libc-alpha-return-97542-incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.b="JIe/yMIv"; dkim-atps=neutral Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 433Vy6032Tz9s29 for ; Tue, 27 Nov 2018 02:21:49 +1100 (AEDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:to:from:subject:message-id:date:mime-version :content-type:content-transfer-encoding; q=dns; s=default; b=Vzq y6DKPmLv74qUQnjus5AGz1fzdx1or7HOiCbtGsri9pX3jc0uyo8rVOUEHuiZ6D4/ i4bUXhbBX2Ux1XgDChq396OsdjWYfdeS8c9vLShK4xNM3EewL0bEEKvwOUHrD7zW uYrimabDfLbDi62/FndlNhg24oWS8BxxZmPW27ZY= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:to:from:subject:message-id:date:mime-version :content-type:content-transfer-encoding; s=default; bh=TNH/tnprn MAGjsV+2pXbE/zGKHU=; b=JIe/yMIvMcRqgV8dA6nboXrL+86syu0Dtpjp3VJit C/zI5Dc4DEXlscIviBYZ2MtIn6JjCz6O7wU7Fin+sDU1Ui2oHZY68yaYpC3lZGpD VrheRkVBGMVwRsp8B2F2v0cXhBKWiHoTaB05VX3TKXf+cid8YKGu12jhHqZ8mED2 jQ= Received: (qmail 66938 invoked by alias); 26 Nov 2018 15:21:44 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 66927 invoked by uid 89); 26 Nov 2018 15:21:43 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-24.4 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.2 spammy=siblings, postinstall, wasted X-HELO: mail-qt1-f196.google.com To: GNU C Library , Florian Weimer From: Carlos O'Donell Subject: [PATCH] Add --no-hard-links option to localedef (Bug 23923) Openpgp: preference=signencrypt Message-ID: <751dddf5-84fa-190f-19ff-073c267822a9@redhat.com> Date: Mon, 26 Nov 2018 10:21:21 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.3.0 MIME-Version: 1.0 [PATCH] Add --no-hard-links option to localedef (bug 23923) Downstream distributions need consistent sets of hardlinks in order for rpm to operate effectively. This means that even if locales are built with a high level of parallelism that the resulting files need to have consistent hardlink counts. The only way to achieve this is with a post-install hardlink pass using a program like 'hardklink' (shipped in Fedora). If the downstream distro wants to post-process the hardlinks then the time spent in localedef looking up sibling directories and processing hardlinks is wasted effort. To optimize the build and install pass we add a --no-hard-links option to localedef to avoid doing the hardlink optimziation for size. Tested on x86_64 with 'make localedata/install-locale-files' before and after. Without the patch we have files with 70+ hardlink counts. After the patch and running with --no-hard-links all link counts are 1. Signed-off-by: Carlos O'Donell --- ChangeLog | 10 ++++++++++ locale/programs/localedef.c | 10 ++++++++++ locale/programs/localedef.h | 1 + locale/programs/locfile.c | 16 +++++++++++++--- 4 files changed, 34 insertions(+), 3 deletions(-) diff --git a/ChangeLog b/ChangeLog index 03887874a2..b9bce8bc8e 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,13 @@ +2018-11-26 Carlos O'Donell + + [BZ #23923] + * locale/programs/localedef.c: Declare boolean hard_links default true. + (options): Add --no-hard-links option. + (parse_opt): Add OPT_NO_HARD_LINKS case and set hard_links to false. + * locale/programs/localedef.h: Declare prototype for hard_links. + * locale/programs/locfile.c (write_locale_data): Don't use hard + links if hard_links is false. + 2018-11-26 Carlos O'Donell * scripts/abilist.awk: Handle .tdata. Error for unknown combinations. diff --git a/locale/programs/localedef.c b/locale/programs/localedef.c index d718d2e9f4..6c4936be6b 100644 --- a/locale/programs/localedef.c +++ b/locale/programs/localedef.c @@ -85,6 +85,9 @@ static bool replace_archive; /* If true list archive content. */ static bool list_archive; +/* If true create hard links to other locales (default). */ +bool hard_links = true; + /* Maximum number of retries when opening the locale archive. */ int max_locarchive_open_retry = 10; @@ -105,6 +108,7 @@ void (*argp_program_version_hook) (FILE *, struct argp_state *) = print_version; #define OPT_BIG_ENDIAN 401 #define OPT_NO_WARN 402 #define OPT_WARN 403 +#define OPT_NO_HARD_LINKS 404 /* Definitions of arguments for argp functions. */ static const struct argp_option options[] = @@ -120,6 +124,8 @@ static const struct argp_option options[] = { NULL, 0, NULL, 0, N_("Output control:") }, { "force", 'c', NULL, 0, N_("Create output even if warning messages were issued") }, + { "no-hard-links", OPT_NO_HARD_LINKS, NULL, 0, + N_("Do not create hard links between installed locales") }, { "prefix", OPT_PREFIX, N_("PATH"), 0, N_("Optional output file prefix") }, { "posix", OPT_POSIX, NULL, 0, N_("Strictly conform to POSIX") }, { "quiet", OPT_QUIET, NULL, 0, @@ -389,6 +395,10 @@ parse_opt (int key, char *arg, struct argp_state *state) /* Enable the warnings. */ set_warnings (arg, true); break; + case OPT_NO_HARD_LINKS: + /* Do not hard link to other locales. */ + hard_links = false; + break; case 'c': force_output = 1; break; diff --git a/locale/programs/localedef.h b/locale/programs/localedef.h index 0083faceab..e2b39e78f3 100644 --- a/locale/programs/localedef.h +++ b/locale/programs/localedef.h @@ -118,6 +118,7 @@ extern const char *repertoire_global; extern int max_locarchive_open_retry; extern bool no_archive; extern const char *alias_file; +extern bool hard_links; /* Prototypes for a few program-wide used functions. */ diff --git a/locale/programs/locfile.c b/locale/programs/locfile.c index 32e5f761f2..14c743373d 100644 --- a/locale/programs/locfile.c +++ b/locale/programs/locfile.c @@ -702,7 +702,7 @@ write_locale_data (const char *output_path, int catidx, const char *category, size_t cnt, step, maxiov; int fd; char *fname; - const char **other_paths; + const char **other_paths = NULL; uint32_t header[2]; size_t n_elem; struct iovec vec[3]; @@ -828,8 +828,18 @@ failure while writing data for category `%s'"), category); close (fd); /* Compare the file with the locale data files for the same category in - other locales, and see if we can reuse it, to save disk space. */ - other_paths = siblings (output_path); + other locales, and see if we can reuse it, to save disk space. + If the user specified --no-hard-link to localedef then hard_links is + false, other_paths remains NULL and we skip the optimization below. + The use of --no-hard-link is distribution specific since some distros + have post-processing hard-link steps and so doing this here is a waste + of time. */ + if (hard_links) + other_paths = siblings (output_path); + + /* If there are other paths, then walk the sibling paths looking for + files with the same content so we can hard link and reduce disk + space usage. */ if (other_paths != NULL) { struct stat64 fname_stat;