From patchwork Thu Oct 15 03:52:46 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carlos O'Donell X-Patchwork-Id: 530466 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id B2B04140E3B for ; Thu, 15 Oct 2015 14:53:03 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.b=cAkF93WJ; dkim-atps=neutral DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:to:from:subject:message-id:date:mime-version :content-type:content-transfer-encoding; q=dns; s=default; b=UqG OqlSfOVNPpd3V4yFh68edmOyqHJx3FvUyaDJYa34SWdbJiUO+Vor9wnVv7DN/cFa wWG37ezvKLhj3RqDWbxUm4Ww+cwuJ5yFeO4kxwp+SpF0K/Uk5+RGzC6hpPHdN/Q+ FlWfHJVoJ9YQXhpUcMJX+jHeCuOEVA36UmLPJANs= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:to:from:subject:message-id:date:mime-version :content-type:content-transfer-encoding; s=default; bh=prLpcMPL/ a5EJsFLbHCeY5iLJ0c=; b=cAkF93WJlabvrmwuZ4wrrJXepLtvrZzuboIE7Zk1+ bZAiF4/2D/2qhKWVUn8ZhZoa4xmwEboNFgnJch4qrBwzKfIFl1biS5N27Vz/uXp7 2/OyCmdbCFMjmSj7GHl4hEFXddfO6lPxquq5Xi3IiasZwxjcUbZm0W5qbCc+qvd9 Xg= Received: (qmail 116893 invoked by alias); 15 Oct 2015 03:52:54 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 116873 invoked by uid 89); 15 Oct 2015 03:52:52 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.3 required=5.0 tests=AWL, BAYES_00, SPF_HELO_PASS, T_RP_MATCHES_RCVD autolearn=ham version=3.3.2 X-HELO: mx1.redhat.com To: Mike Frysinger , GNU C Library From: "Carlos O'Donell" Subject: [PATCH] localedef: Add --localedir and --archive-file options (Bug 19130) X-Enigmail-Draft-Status: N1110 Message-ID: <561F230E.5050801@redhat.com> Date: Wed, 14 Oct 2015 23:52:46 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 At present localdef accepts the following non-POSIX options: --list-archive [FILE] and --prefix=PATH The first allows you to list the contents of the default binary locale archive via `--list-archive` or any other archive via `--list-archive FILE` where FILE is an absolute or relative path to the archive. This patch does three things, and I have consciously not split them up into three patches because they logically belong together. (1) Enhance `--list-archive`, `--add-to-archive`, and `--delete-from-archive` to honour `--prefix=PATH`. At present the use of --prefix does not adjust the path interpreted by --list-archive FILE . This is a mistake. We should always honour --prefix in all of our output operations. The patch makes --list-archive FILE honour prefix. For example if you ran `--prefix /mnt/chroot --list-archive /usr/lib/locale/locale-archive-v2` the final path to the locale archive file would be: `/mnt/chroot/usr/lib/locale/locale-archive-v2`. Similarly --add-to-archive and --delete-from-archive honour --prefix when computing the archive to use. e.g. localedef --prefix="/home/carlos/tmp" --add-to-archive ./en_US.UTF-8 cannot create temporary file: /home/carlos/tmp/usr/lib64/locale/locale-archive.3vmNHw: No such file or directory (2) Add a `--localedir=PATH` option per BZ #19130. At present all operations are carried out against a hard-coded LOCALEDIR e.g. $(localedir) or /usr/lib/locale. This patch adds a --localedir=PATH option which allows you to change the value of LOCALEDIR used in localedef. For example if you ran `--localedir=/usr/lib64/locale --list-archive` the final path to the locale archive file would be: `/usr/lib64/locale/locale-archive`. e.g. ./locale/localedef --prefix="/home/carlos/tmp" --localedir="/alt" --add-to-archive ./en_US.UTF-8 cannot create temporary file: /home/carlos/tmp/alt/locale-archive.VCqI8z: No such file or directory (3) Add a `--archive-file=FILE`/`-a FILE` option. At present the binary locale archive name is always `locale-archive`, but it would be easier for distribution maintainers and developers if localedef could be made to scan an arbitrary file. This patch makes `--archive-file=FILE` adjust the default locale archive file name to be FILE. For example if you ran: `--prefix /mnt/croot --archive-file=locale-archive-v2` the final path to the locale archive file would be: `/mnt/chroot/usr/lib/locale/locale-archive-v2`. With (1), (2) and (3), you have full control over the locale archive selected by localedef: [prefix][localedir][archive file] | | | | | |-> --archive-file | | | |-> --localedir | |-> --prefix Sensible defaults are selected for all values based on the configuration of glibc e.g. prefix is "", localedir is "$(localedir)" and archive file is "locale-archive". (a) Use of I18NPATH, LOCPATH, and change in behaviour of ENOENT for locale-archive. The selection of the character maps is achieved by use of the I18NPATH env var, and LOCPATH env var for the location of source locale data. These environment variables do not use --prefix. Both of these variables will fallback to the default system directory during failure to find the required files, while the above settings for prefix/localedir/archive-file will create the requested binary locale archive file if not found. Note that this is a change in behaviour, previously if the non-default binary locale archive file was not found we would issue an error (only creating the default locale archive was supported). Now we treat all locale-archive files the same way. It might be argued that a "--create" mode should be used and errors issued if the file doesn't exist in all cases, but this breaks backwards compatibility and should be avoided. The only thing we break with this patch is using `--list-archive foo` to test if `foo` is present and a locale-archive file, which now returns nothing, but doesn't fail (like we did for the default locale archive). (b) Future enhancements. Future enhancements include more options to control locale source path (LOCPATH) and character map locations (I18NPATH) since falling back to the default system directories is not always desirable in a cross or sysroot'd environment. Tested on x86_64 to view locale archive files in non-standard paths and to generate locale archive files in non-standard paths. Using --help gives these addtional options: ~~~ ... --localedir=PATH Optional path to locale directory e.g. /usr/lib64/locale ... -a, --archive-file=FILE locale-archive file to use instead of default ... ~~~ OK to checkin? 2015-10-14 Carlos O'Donell [BZ #19130] * locale/programs/localedef.c (output_localedir): New variable. (archive_file): New variable. (OPT_LOCALEDIR): New macro. (OPT_NO_ARCHIVE, OPT_ADD_TO_ARCHIVE, OPT_REPLACE, OPT_DELETE_FROM_ARCHIVE, OPT_LIST_ARCHIVE): Adjust down by one. (options): Add --localedir option. Add --archive-file/-a option. (main): --list-archive FILE has precedence over -a FILE. Pass specified archive file (-a FILE) to callees (prefix still applies). (parse_opt): Add OPT_LOCALEDIR and store to output_localedir. Add 'a' and store to archive_file. (construct_output_path): Use output_localedir if non-NULL. (normalize_codeset): Use xmalloc. (construct_prefix_path): New function. * locale/programs/localedef.h: add_locales_to_archive, and delete_locales_from_archive accept pointer to struct locarhandle. * locale/programs/locarchive.c (add_locales_to_archive): Use struct locarhandle pointer passed by caller. (delete_locales_from_archive): Likewise. (open_archive): Always create the archive if not in readonly mode. (ARCHIVE_NAME): Remove. * locale/programs/localedef.h (LOCALE_ARCHIVE): Define. (ARCHIVE_NAME): Define. * locale/programs/locfile.c (write_all_categories): Likewise. * locale/programs/locfile.h: write_all_categories accepts pointer to struct locarhandle. --- Cheers, Carlos. diff --git a/locale/programs/localedef.c b/locale/programs/localedef.c index 06fca12..44f15e4 100644 --- a/locale/programs/localedef.c +++ b/locale/programs/localedef.c @@ -65,6 +65,9 @@ static int force_output; /* Prefix for output files. */ const char *output_prefix; +/* Locale directory to use for output. */ +const char *output_localedir; + /* Name of the character map file. */ static const char *charmap_file; @@ -74,6 +77,9 @@ static const char *input_file; /* Name of the repertoire map file. */ const char *repertoire_global; +/* Location of the locale-archive file. */ +const char *archive_file; + /* Name of the locale.alias file. */ const char *alias_file; @@ -107,11 +113,12 @@ void (*argp_program_version_hook) (FILE *, struct argp_state *) = print_version; #define OPT_QUIET 302 #define OPT_OLDSTYLE 303 #define OPT_PREFIX 304 -#define OPT_NO_ARCHIVE 305 -#define OPT_ADD_TO_ARCHIVE 306 -#define OPT_REPLACE 307 -#define OPT_DELETE_FROM_ARCHIVE 308 -#define OPT_LIST_ARCHIVE 309 +#define OPT_LOCALEDIR 305 +#define OPT_NO_ARCHIVE 306 +#define OPT_ADD_TO_ARCHIVE 307 +#define OPT_REPLACE 308 +#define OPT_DELETE_FROM_ARCHIVE 309 +#define OPT_LIST_ARCHIVE 310 #define OPT_LITTLE_ENDIAN 400 #define OPT_BIG_ENDIAN 401 @@ -131,11 +138,16 @@ static const struct argp_option options[] = N_("Create output even if warning messages were issued") }, { "old-style", OPT_OLDSTYLE, NULL, 0, N_("Create old-style tables") }, { "prefix", OPT_PREFIX, N_("PATH"), 0, N_("Optional output file prefix") }, + { "localedir", OPT_LOCALEDIR, N_("PATH"), 0, + N_("Optional path to locale directory e.g. /usr/lib64/locale") }, { "posix", OPT_POSIX, NULL, 0, N_("Strictly conform to POSIX") }, { "quiet", OPT_QUIET, NULL, 0, N_("Suppress warnings and information messages") }, { "verbose", 'v', NULL, 0, N_("Print more messages") }, + { NULL, 0, NULL, 0, N_("Archive control:") }, + { "archive-file", 'a', N_("FILE"), 0, + N_("locale-archive file to use instead of default")}, { "no-archive", OPT_NO_ARCHIVE, NULL, 0, N_("Don't add new data to archive") }, { "add-to-archive", OPT_ADD_TO_ARCHIVE, NULL, 0, @@ -178,6 +190,9 @@ static struct argp argp = /* Prototypes for local functions. */ static void error_print (void); static const char *construct_output_path (char *path); +static const char *construct_prefix_path (const char *prefix, + const char *localedir, + const char *path); static const char *normalize_codeset (const char *codeset, size_t name_len); @@ -189,6 +204,8 @@ main (int argc, char *argv[]) struct charmap_t *charmap; struct localedef_t global; int remaining; + struct locarhandle ah; + const char *prefix_archive_file = NULL; /* Set initial values for global variables. */ copy_list = NULL; @@ -207,14 +224,34 @@ main (int argc, char *argv[]) argp_err_exit_status = 4; argp_parse (&argp, argc, argv, 0, &remaining, NULL); + /* Use the user-specified archive file or NULL (default). */ + ah.fname = archive_file; + if (output_prefix != NULL || output_localedir != NULL || archive_file != NULL) + { + prefix_archive_file = construct_prefix_path (output_prefix ?: "", + output_localedir ?: LOCALEDIR, + archive_file ?: LOCALE_ARCHIVE); + ah.fname = prefix_archive_file; + } + /* Handle a few special cases. */ if (list_archive) - show_archive_content (remaining > 1 ? argv[remaining] : NULL, verbose); + { + /* If using --list-archive FILE and -a FILE at the same time the + archive file specified after --list-archive takes precedence. + The use of -a only changes the default locale archive file. */ + if (remaining > 1 && remaining < argc) + ah.fname = construct_prefix_path (output_prefix, + NULL, + argv[remaining]); + show_archive_content (ah.fname, verbose); + } if (add_to_archive) - return add_locales_to_archive (argc - remaining, &argv[remaining], + return add_locales_to_archive (&ah, argc - remaining, &argv[remaining], replace_archive); if (delete_from_archive) - return delete_locales_from_archive (argc - remaining, &argv[remaining]); + return delete_locales_from_archive (&ah, argc - remaining, + &argv[remaining]); /* POSIX.2 requires to be verbose about missing characters in the character map. */ @@ -286,9 +323,13 @@ cannot open locale definition file `%s'"), runp->name)); { if (cannot_write_why != 0) WITH_CUR_LOCALE (error (4, cannot_write_why, _("\ -cannot write output files to `%s'"), output_path ? : argv[remaining])); +cannot write output files to `%s' (LOCALEDIR is `%s%s')"), + output_path ?: argv[remaining], + output_prefix ?: "", + output_localedir ?: LOCALEDIR)); else - write_all_categories (locales, charmap, argv[remaining], output_path); + write_all_categories (&ah, locales, charmap, + argv[remaining], output_path); } else WITH_CUR_LOCALE (error (4, 0, _("\ @@ -317,6 +358,9 @@ parse_opt (int key, char *arg, struct argp_state *state) case OPT_PREFIX: output_prefix = arg; break; + case OPT_LOCALEDIR: + output_localedir = arg; + break; case OPT_NO_ARCHIVE: no_archive = true; break; @@ -338,6 +382,9 @@ parse_opt (int key, char *arg, struct argp_state *state) case OPT_BIG_ENDIAN: set_big_endian (true); break; + case 'a': + archive_file = arg; + break; case 'c': force_output = 1; break; @@ -415,6 +462,26 @@ error_print (void) { } +/* Construct an output path given a prefix directory, an absolute locale + directory and a path. The output path is a constant and can be used + by any caller and must not be freed. */ +static const char * +construct_prefix_path (const char *prefix, + const char *localedir, + const char *path) +{ + ssize_t n; + char *result; + n = asprintf (&result, "%s%s%s%s%c", + prefix ?: "", + localedir ?: "", + localedir ? "/" : "", + path, '\0'); + if (n < 0) + WITH_CUR_LOCALE (error (6, errno, _("cannot allocate path"))); + return result; +} + /* The parameter to localedef describes the output path. If it does contain a '/' character it is a relative path. Otherwise it names the @@ -458,10 +525,12 @@ construct_output_path (char *path) ssize_t n; if (normal == NULL) n = asprintf (&result, "%s%s/%s%c", - output_prefix ?: "", LOCALEDIR, path, '\0'); + output_prefix ?: "", + output_localedir ?: LOCALEDIR, path, '\0'); else n = asprintf (&result, "%s%s/%.*s%s%s%c", - output_prefix ?: "", LOCALEDIR, + output_prefix ?: "", + output_localedir ?: LOCALEDIR, (int) (startp - path), path, normal, endp, '\0'); if (n < 0) @@ -523,7 +592,7 @@ normalize_codeset (codeset, name_len) only_digit = 0; } - retval = (char *) malloc ((only_digit ? 3 : 0) + len + 1); + retval = (char *) xmalloc ((only_digit ? 3 : 0) + len + 1); if (retval != NULL) { diff --git a/locale/programs/localedef.h b/locale/programs/localedef.h index 7e19ff0..3baf991 100644 --- a/locale/programs/localedef.h +++ b/locale/programs/localedef.h @@ -135,6 +135,8 @@ extern const char *alias_file; setlocale (LC_CTYPE, cur_locale_); \ } while (0) +#define LOCALE_ARCHIVE "locale-archive" +#define ARCHIVE_NAME LOCALEDIR "/"LOCALE_ARCHIVE /* Mark given locale as to be read. */ extern struct localedef_t *add_to_readlist (int locale, const char *name, @@ -165,10 +167,13 @@ extern int add_locale_to_archive (struct locarhandle *ah, const char *name, locale_data_t data, bool replace); /* Add content of named directories to locale archive. */ -extern int add_locales_to_archive (size_t nlist, char *list[], bool replace); +extern int add_locales_to_archive (struct locarhandle *ah, + size_t nlist, + char *list[], bool replace); /* Removed named locales from archive. */ -extern int delete_locales_from_archive (size_t nlist, char *list[]); +extern int delete_locales_from_archive (struct locarhandle *ah, + size_t nlist, char *list[]); /* List content of locale archive. If FNAME is non-null use that as the locale archive to list, otherwise the default. */ diff --git a/locale/programs/locarchive.c b/locale/programs/locarchive.c index 49f7f1b..0ede8ea 100644 --- a/locale/programs/locarchive.c +++ b/locale/programs/locarchive.c @@ -57,8 +57,6 @@ extern const char *output_prefix; -#define ARCHIVE_NAME LOCALEDIR "/locale-archive" - static const char *locnames[] = { #define DEFINE_CATEGORY(category, category_name, items, a) \ @@ -581,11 +579,10 @@ open_archive (struct locarhandle *ah, bool readonly) fd = open64 (archivefname, readonly ? O_RDONLY : O_RDWR); if (fd == -1) { - /* Maybe the file does not yet exist? If we are opening - the default locale archive we ignore the failure and - list an empty archive, otherwise we print an error - and exit. */ - if (errno == ENOENT && archivefname == default_fname) + /* Maybe the file does not yet exist? + Create it if we are not in readonly mode. + If we are in readonly mode pass back an empty structure. */ + if (errno == ENOENT) { if (readonly) { @@ -1328,19 +1325,19 @@ add_locale_to_archive (ah, name, data, replace) } +/* Add locales to the archive file specified in AH->FNAME or + the default if NULL. */ int -add_locales_to_archive (nlist, list, replace) - size_t nlist; - char *list[]; - bool replace; +add_locales_to_archive (struct locarhandle *ah, + size_t nlist, + char *list[], + bool replace) { - struct locarhandle ah; int result = 0; /* Open the archive. This call never returns if we cannot successfully open the archive. */ - ah.fname = NULL; - open_archive (&ah, false); + open_archive (ah, false); while (nlist-- > 0) { @@ -1514,7 +1511,7 @@ add_locales_to_archive (nlist, list, replace) continue; } - result |= add_locale_to_archive (&ah, basename (fname), data, replace); + result |= add_locale_to_archive (ah, basename (fname), data, replace); for (cnt = 0; cnt < __LC_LAST; ++cnt) if (cnt != LC_ALL) @@ -1522,28 +1519,28 @@ add_locales_to_archive (nlist, list, replace) } /* We are done. */ - close_archive (&ah); + close_archive (ah); return result; } +/* Delete the listed locales from archive AH->FNAME or the default + archive if NULL. */ int -delete_locales_from_archive (nlist, list) - size_t nlist; - char *list[]; +delete_locales_from_archive (struct locarhandle *ah, + size_t nlist, + char *list[]) { - struct locarhandle ah; struct locarhead *head; struct namehashent *namehashtab; /* Open the archive. This call never returns if we cannot successfully open the archive. */ - ah.fname = NULL; - open_archive (&ah, false); + open_archive (ah, false); - head = ah.addr; - namehashtab = (struct namehashent *) ((char *) ah.addr + head = ah->addr; + namehashtab = (struct namehashent *) ((char *) ah->addr + GET (head->namehash_offset)); while (nlist-- > 0) @@ -1565,7 +1562,7 @@ delete_locales_from_archive (nlist, list) { if (GET (namehashtab[idx].hashval) == hval && (strcmp (locname, - ((char *) ah.addr + ((char *) ah->addr + GET (namehashtab[idx].name_offset))) == 0)) { @@ -1584,7 +1581,7 @@ delete_locales_from_archive (nlist, list) error (0, 0, _("locale \"%s\" not in archive"), locname); } - close_archive (&ah); + close_archive (ah); return 0; } diff --git a/locale/programs/locfile.c b/locale/programs/locfile.c index 33da52e..51ecfcf 100644 --- a/locale/programs/locfile.c +++ b/locale/programs/locfile.c @@ -326,7 +326,7 @@ static void (*const write_funcs[]) (struct localedef_t *, void -write_all_categories (struct localedef_t *definitions, +write_all_categories (struct locarhandle *ah, struct localedef_t *definitions, const struct charmap_t *charmap, const char *locname, const char *output_path) { @@ -338,19 +338,15 @@ write_all_categories (struct localedef_t *definitions, if (! no_archive) { - /* The data has to be added to the archive. Do this now. */ - struct locarhandle ah; - /* Open the archive. This call never returns if we cannot successfully open the archive. */ - ah.fname = NULL; - open_archive (&ah, false); + open_archive (ah, false); - if (add_locale_to_archive (&ah, locname, to_archive, true) != 0) + if (add_locale_to_archive (ah, locname, to_archive, true) != 0) error (EXIT_FAILURE, errno, _("cannot add to locale archive")); /* We are done. */ - close_archive (&ah); + close_archive (ah); } } diff --git a/locale/programs/locfile.h b/locale/programs/locfile.h index 6fc441b..8dabb6d 100644 --- a/locale/programs/locfile.h +++ b/locale/programs/locfile.h @@ -64,7 +64,8 @@ extern void check_all_categories (struct localedef_t *definitions, const struct charmap_t *charmap); /* Write out all locale categories. */ -extern void write_all_categories (struct localedef_t *definitions, +extern void write_all_categories (struct locarhandle *ah, + struct localedef_t *definitions, const struct charmap_t *charmap, const char *locname, const char *output_path);