| Message ID | bb4902fd-604e-2788-f7c8-742400cd6216@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | localedef: Add --no-warnings/--warnings option | expand |
On Okt 17 2017, Carlos O'Donell <carlos@redhat.com> wrote: > diff --git a/locale/programs/charmap.h b/locale/programs/charmap.h > index 5d6b48f..441d429 100644 > --- a/locale/programs/charmap.h > +++ b/locale/programs/charmap.h > @@ -66,6 +66,8 @@ struct charseq > > /* True if the encoding is not ASCII compatible. */ > extern bool enc_not_ascii_compatible; > +/* True if the ASCII compatibility check should raise a warning. */ > +bool warn_ascii; This should be extern. I think both flags should be declared in the same header, since they are defined in the same source file. Andreas.
On 10/17/2017 10:59 AM, Carlos O'Donell wrote: > if (failed) > { > - record_warning (_("\ > -character map `%s' is not ASCII compatible, locale not ISO C compliant\n"), > - result->code_set_name); > + /* A user may disable the ASCII compatibility warning check, > + but we must remember that the encoding is not ASCII > + compatible, since it may have other implications. Later > + we will set _NL_CTYPE_MAP_TO_NONASCII from this value. */ > + if (warn_ascii) > + record_warning (_( > +"character map `%s' is not ASCII compatible, locale not ISO C compliant " > +"[--no-warnings=ascii]"), > + result->code_set_name); > enc_not_ascii_compatible = true; > } It may lead to generally nicer code if the flag check happens in record_warning itself. > diff --git a/locale/programs/charmap.h b/locale/programs/charmap.h > index 5d6b48f..441d429 100644 > --- a/locale/programs/charmap.h > +++ b/locale/programs/charmap.h > @@ -66,6 +66,8 @@ struct charseq > > /* True if the encoding is not ASCII compatible. */ > extern bool enc_not_ascii_compatible; > +/* True if the ASCII compatibility check should raise a warning. */ > +bool warn_ascii; You have a non-tentative definition elsewhere, so this should indeed be extern. > /* Prototypes for charmap handling functions. */ > diff --git a/locale/programs/ld-monetary.c b/locale/programs/ld-monetary.c > index 9d94738..71df376 100644 > --- a/locale/programs/ld-monetary.c > +++ b/locale/programs/ld-monetary.c > @@ -234,12 +234,17 @@ No definition for %s category found"), "LC_MONETARY"); > char symbol[4]; > strncpy (symbol, monetary->int_curr_symbol, 3); > symbol[3] = '\0'; > + /* A user may disable this waning for testing purposes or > + for building a locale with a 3 digit country code that “3 letter country code” > + was not yet supported in our ISO 4217 list. > + See the use of --no-warnings=intcurrsym. */ > if (bsearch (symbol, valid_int_curr, NR_VALID_INT_CURR, > sizeof (const char *), > - (comparison_fn_t) curr_strcmp) == NULL) > + (comparison_fn_t) curr_strcmp) == NULL > + && warn_int_curr_symbol) > record_warning (_("\ > %s: value of field `int_curr_symbol' does \ > -not correspond to a valid name in ISO 4217"), > +not correspond to a valid name in ISO 4217 [--no-warnings=intcurrsym]"), > "LC_MONETARY"); > +static void > +set_warnings (char *warnings, bool enabled) > +{ > + char *tok; > + char *save; > + char *copy = (char *) malloc (strlen (warnings) + 1); > + > + /* Remove all spaces from the warnings list to make the processing > + a more robust. We don't support spaces in a warning name. */ > + > + save = copy; > + tok = warnings; Style: Declare on first use? (I think that's preferred nowadays.) > + > + do { > + while (isspace (*tok)) > + tok++; > + } while ((*save++ = *tok++)); { } need to follow GNU style, and the comparison against '\0' should be explicit. The other parts of the patch look fine to me. > +# The SHIFT_JIS and SHIFT_JISX0213 character maps are not ASCII compatible, > +# therefore we have to use --no-warnings=ascii to disable the ASCII check. > +# See localedata/gen-locale.sh for the same logic. > $(INSTALL-SUPPORTED-LOCALES): install-locales-dir > @locale=`echo $@ | sed -e 's/^install-//'`; \ > charset=`echo $$locale | sed -e 's,.*/,,'`; \ > locale=`echo $$locale | sed -e 's,/[^/]*,,'`; \ > + flags="--quiet -c"; \ > + if [ "$$charset" = 'SHIFT_JIS' ] \ > + || [ "$$charset" = 'SHIFT_JISX0213' ]; then \ > + flags="$$flags --no-warnings=ascii"; \ > + fi; \ > echo -n `echo $$locale | sed 's/\([^.\@]*\).*/\1/'`; \ > echo -n ".$$charset"; \ > echo -n `echo $$locale | sed 's/\([^\@]*\)\(\@.*\)*/\2/'`; \ > echo -n '...'; \ > input=`echo $$locale | sed 's/\([^.]*\)[^@]*\(.*\)/\1\2/'`; \ > - $(LOCALEDEF) --alias-file=../intl/locale.alias \ > - -i locales/$$input -c -f charmaps/$$charset \ > + $(LOCALEDEF) $$flags --alias-file=../intl/locale.alias \ > + -i locales/$$input -f charmaps/$$charset \ > $(addprefix --prefix=,$(install_root)) $$locale \ > && echo ' done'; \ Maybe it's time to move this recipe into its own file under scripts/? Thanks, Florian
On 10/17/2017 02:14 AM, Andreas Schwab wrote: > On Okt 17 2017, Carlos O'Donell <carlos@redhat.com> wrote: > >> diff --git a/locale/programs/charmap.h b/locale/programs/charmap.h >> index 5d6b48f..441d429 100644 >> --- a/locale/programs/charmap.h >> +++ b/locale/programs/charmap.h >> @@ -66,6 +66,8 @@ struct charseq >> >> /* True if the encoding is not ASCII compatible. */ >> extern bool enc_not_ascii_compatible; >> +/* True if the ASCII compatibility check should raise a warning. */ >> +bool warn_ascii; > > This should be extern. > > I think both flags should be declared in the same header, since they are > defined in the same source file. The annoying part here is that charmap.o is used by iconv *and* localedef. I made this a tentative definition because iconv doesn't need it, and doesn't define it, and localedef defines it *and* initializes it ot true. I'll see if I can structure this better. Let me work up a v2.
On 10/17/2017 02:52 AM, Florian Weimer wrote: > On 10/17/2017 10:59 AM, Carlos O'Donell wrote: > >> if (failed) >> { >> - record_warning (_("\ >> -character map `%s' is not ASCII compatible, locale not ISO C compliant\n"), >> - result->code_set_name); >> + /* A user may disable the ASCII compatibility warning check, >> + but we must remember that the encoding is not ASCII >> + compatible, since it may have other implications. Later >> + we will set _NL_CTYPE_MAP_TO_NONASCII from this value. */ >> + if (warn_ascii) >> + record_warning (_( >> +"character map `%s' is not ASCII compatible, locale not ISO C compliant " >> +"[--no-warnings=ascii]"), >> + result->code_set_name); >> enc_not_ascii_compatible = true; >> } > > It may lead to generally nicer code if the flag check happens in record_warning itself. I considered it, but it would require richer API for the warnings being recorded, so for now I'm going to leave this as-is. Though I agree with you that it is nicer if the caller simply filled in a structure, and submitted the warning. Then the warning code handled deciding of the warning was enabled or not. I'm not keen to refactor this code all over again just to support that kind of use case, mainly because we might not need it. If I moved this code into support/ I would do that kind of refactoring. >> diff --git a/locale/programs/charmap.h b/locale/programs/charmap.h >> index 5d6b48f..441d429 100644 >> --- a/locale/programs/charmap.h >> +++ b/locale/programs/charmap.h >> @@ -66,6 +66,8 @@ struct charseq >> /* True if the encoding is not ASCII compatible. */ >> extern bool enc_not_ascii_compatible; >> +/* True if the ASCII compatibility check should raise a warning. */ >> +bool warn_ascii; > > You have a non-tentative definition elsewhere, so this should indeed be extern. I do. In localedef. There is a use of charmap.o in iconv (as I mentioned to Andreas). I'm going to move record-status.h to record-status.c and centralize the warning handling variables. That should clean this up. >> /* Prototypes for charmap handling functions. */ >> diff --git a/locale/programs/ld-monetary.c b/locale/programs/ld-monetary.c >> index 9d94738..71df376 100644 >> --- a/locale/programs/ld-monetary.c >> +++ b/locale/programs/ld-monetary.c >> @@ -234,12 +234,17 @@ No definition for %s category found"), "LC_MONETARY"); >> char symbol[4]; >> strncpy (symbol, monetary->int_curr_symbol, 3); >> symbol[3] = '\0'; >> + /* A user may disable this waning for testing purposes or >> + for building a locale with a 3 digit country code that > > “3 letter country code” Fixed. Thanks. >> + was not yet supported in our ISO 4217 list. >> + See the use of --no-warnings=intcurrsym. */ >> if (bsearch (symbol, valid_int_curr, NR_VALID_INT_CURR, >> sizeof (const char *), >> - (comparison_fn_t) curr_strcmp) == NULL) >> + (comparison_fn_t) curr_strcmp) == NULL >> + && warn_int_curr_symbol) >> record_warning (_("\ >> %s: value of field `int_curr_symbol' does \ >> -not correspond to a valid name in ISO 4217"), >> +not correspond to a valid name in ISO 4217 [--no-warnings=intcurrsym]"), >> "LC_MONETARY"); > >> +static void >> +set_warnings (char *warnings, bool enabled) >> +{ >> + char *tok; >> + char *save; >> + char *copy = (char *) malloc (strlen (warnings) + 1); >> + >> + /* Remove all spaces from the warnings list to make the processing >> + a more robust. We don't support spaces in a warning name. */ >> + >> + save = copy; >> + tok = warnings; > > Style: Declare on first use? (I think that's preferred nowadays.) I cleaned this up. Basically we use tok/save/copy right away in the space removal loop. >> + >> + do { >> + while (isspace (*tok)) >> + tok++; >> + } while ((*save++ = *tok++)); > > { } need to follow GNU style, and the comparison against '\0' should be explicit. Fixed. > > The other parts of the patch look fine to me. > >> +# The SHIFT_JIS and SHIFT_JISX0213 character maps are not ASCII compatible, >> +# therefore we have to use --no-warnings=ascii to disable the ASCII check. >> +# See localedata/gen-locale.sh for the same logic. >> $(INSTALL-SUPPORTED-LOCALES): install-locales-dir >> @locale=`echo $@ | sed -e 's/^install-//'`; \ >> charset=`echo $$locale | sed -e 's,.*/,,'`; \ >> locale=`echo $$locale | sed -e 's,/[^/]*,,'`; \ >> + flags="--quiet -c"; \ >> + if [ "$$charset" = 'SHIFT_JIS' ] \ >> + || [ "$$charset" = 'SHIFT_JISX0213' ]; then \ >> + flags="$$flags --no-warnings=ascii"; \ >> + fi; \ >> echo -n `echo $$locale | sed 's/\([^.\@]*\).*/\1/'`; \ >> echo -n ".$$charset"; \ >> echo -n `echo $$locale | sed 's/\([^\@]*\)\(\@.*\)*/\2/'`; \ >> echo -n '...'; \ >> input=`echo $$locale | sed 's/\([^.]*\)[^@]*\(.*\)/\1\2/'`; \ >> - $(LOCALEDEF) --alias-file=../intl/locale.alias \ >> - -i locales/$$input -c -f charmaps/$$charset \ >> + $(LOCALEDEF) $$flags --alias-file=../intl/locale.alias \ >> + -i locales/$$input -f charmaps/$$charset \ >> $(addprefix --prefix=,$(install_root)) $$locale \ >> && echo ' done'; \ > > Maybe it's time to move this recipe into its own file under scripts/? I don't want to tackle that in this patch. Though I agree, this should get deleted and localedata/gen-locale.sh should become the canonical script we use to install locales. However it requires refacoring gen-locales.sh a bit to be generally usable for installing test locales and installing final locales. Please find v2 attached, no regressions. In v2 I have moved all warning related variables into record-status.c, and adjusted iconv, locale, and localedef to use that and avoid duplication. OK to commit?
On 10/17/2017 03:06 PM, Carlos O'Donell wrote: > Please find v2 attached, no regressions. > > In v2 I have moved all warning related variables into record-status.c, > and adjusted iconv, locale, and localedef to use that and avoid > duplication. > > OK to commit? Andreas, Ping? Any comments on v2? It should solve the problem you see in the builds. https://sourceware.org/ml/libc-alpha/2017-10/msg00784.html
On Okt 17 2017, Carlos O'Donell <carlos@redhat.com> wrote: > + * locale/programs/record-status.h: Define globals, and function > + prototypes. Move function bodies... > + * locale/programs/record-status.c: ... to here. New file. > + * iconv/Makefile (iconv_prog-modules): Add record-status. > + * locale/Makefile (lib-modules): Likewise. > + * iconv/iconv_prog.c: Remove verbose. > + * iconv/iconv_prog.h: Include record-status.h (defines verbose). > + * locale/programs/charmap.c (charmap_read): If warn_ascii is true then > + record a warning about ASCII compatibility. > + * locale/programs/ld-monetary.c (monetary_finish): If > + warn_int_curr_symbol is true then record a warning about the symbol > + not being in our ISO 4217 list. > + * locale/programs/locale.c: Include record-status.h. Remove verbose. > + * locale/programs/localedef.c: Include ctype.h. Remove delcaration of > + verbose, recorded_warning_count, recorded_error_count, and be_quiet. > + (OPT_NO_WARN): Define. > + (OPT_WARN): Define. > + (options): Add entry for --no-warnings, and --warnings. > + (set_warnings): New function to enable/disable warnings. > + (parse_opt): Call set_warnings for OPT_NO_WARN and OPT_WARN. > + * locale/programs/localedef.h: Remove warn_int_curr_symbol. > + * localedata/gen-locale.sh: Default flags to `--quiet -c'. > + Add `--no-warnings=ascii' to locales using SHIFT_JIS or SHIFT_JIXX0213. > + Pass flags to generate_locale. > + (generate_locale): Accept new flag argument and pass it to localedef > + invocation. > + * localedata/Makefile (INSTALL-SUPPORTED-LOCALES): Use > + --no-warnings=ascii for SHIFT_JIS and SHIFT_JISX0213 charmaps. Ok. > @@ -309,6 +307,43 @@ no output file produced because errors were issued")); > exit (recorded_warning_count != 0); > } > > +/* Search warnings for matching warnings and if found enable those > + warnings if ENABLED is true, otherwise disable the warnings. */ > +static void > +set_warnings (char *warnings, bool enabled) > +{ > + char *tok = warnings; > + char *copy = (char *) malloc (strlen (warnings) + 1); > + char *save = copy; > + > + /* As we make a copy of the warnings list we remove all spaces from > + the warnings list to make the processing a more robust. We don't > + support spaces in a warning name. */ > + do > + { > + while (isspace (*tok) != 0) > + tok++; > + } > + while ((*save++ = *tok++)); Implicit boolean coercion. Andreas.
On 10/24/2017 08:02 AM, Andreas Schwab wrote:
> Implicit boolean coercion.
Thanks, I fixed the boolean coercion in v3, and pushed.
commit 02eec681676c5aabf2eb13b92b1124245d19112f.
This should fix the problems you were seeing with SHIFT_JIS and
SJIFT_JISX0213. Thank you for testing. Please reach out if this breaks
again in any unexpected ways.
From 9c9fd183d1aa017ade39c60a9dfe9ed0f9759eb1 Mon Sep 17 00:00:00 2001
From: Carlos O'Donell <carlos@systemhalted.org>
Date: Tue, 17 Oct 2017 01:33:42 -0700
Subject: [PATCH] localedef: Add --no-warnings/--warnings option
From localedef --help:
Output control:
...
--no-warnings=<warnings> Comma-separated list of warnings to disable;
supported warnings are: ascii, intcurrsym
...
--warnings=<warnings> Comma-separated list of warnings to enable;
supported warnings are: ascii, intcurrsym
Locales using SHIFT_JIS and SHIFT_JISX0213 character maps are not ASCII
compatible. In order to build locales using these character maps, and
have localedef exit with a status of 0, we add new option to localedef
to disable or enable specific warnings. The options are --no-warnings
and --warnings, to disable and enable specific warnings respectively.
The options take a comma-separated list of warning names. The warning
names are taken directly from the generated warning. When a warning
that can be disabled is issued it will print something like this: foo is
not defined [--no-warnings=foo]
For the initial implemenation we add two controllable warnings; first
'ascii' which is used by the localedata installation makefile target to
install SHIFT_JIS and SHIFT_JISX0213-using locales without error; second
'intcurrsym' which allows a program to use a non-standard international
currency symbol without triggering a warning. The 'intcurrsym' is
useful in the future if country codes are added that are not in our
current ISO 4217 list, and the user wants to avoid the warning. Having
at least two warnings to control gives an example for how the changes
can be extended to more warnings if required in the future.
These changes allow ja_JP.SHIFT_JIS and ja_JP.SHIFT_JISX0213 to be
compiled without warnings using --no-warnings=ascii. The
localedata/Makefile $(INSTALL-SUPPORTED-LOCALES) target is adjusted to
automatically add `--no-warnings=ascii` for such charmaps, and likewise
localedata/gen-locale.sh is adjusted with similar logic.
Signed-off-by: Carlos O'Donell <carlos@redhat.com>
---
ChangeLog | 25 +++++++++++++++++
locale/programs/charmap.c | 12 ++++++--
locale/programs/charmap.h | 2 ++
locale/programs/ld-monetary.c | 9 ++++--
locale/programs/localedef.c | 64 +++++++++++++++++++++++++++++++++++++++++++
locale/programs/localedef.h | 1 +
localedata/Makefile | 12 ++++++--
localedata/gen-locale.sh | 24 ++++++++++------
8 files changed, 134 insertions(+), 15 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 2727111..b5c004b 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,28 @@
+2017-10-16 Carlos O'Donell <carlos@redhat.com>
+
+ * locale/programs/localedef.c (warn_ascii): Declare.
+ (warn_int_curr_symbol): Declare.
+ (OPT_NO_WARN): Define.
+ (OPT_WARN): Define.
+ (options): Add entry for --no-warnings, and --warnings.
+ (set_warnings): New function to enable/disable warnings.
+ (parse_opt): Call set_warnings for OPT_NO_WARN and OPT_WARN.
+ * locale/programs/localedef.h (warn_int_curr_symbol): Declare extern
+ bool.
+ * locale/programs/ld-monetary.c (monetary_finish): If
+ warn_int_curr_symbol is true then record a warning about the symbol
+ not being in our ISO 4217 list.
+ * locale/programs/charmap.c (charmap_read): If warn_ascii is true then
+ record a warning about ASCII compatibility.
+ * locale/programs/charmap.h: Add tenative definition of warn_ascii.
+ * localedata/gen-locale.sh: Default flags to `--quiet -c'.
+ Add `--no-warnings=ascii' to locales using SHIFT_JIS or SHIFT_JIXX0213.
+ Pass flags to generate_locale.
+ (generate_locale): Accept new flag argument and pass it to localedef
+ invocation.
+ * localedata/Makefile (INSTALL-SUPPORTED-LOCALES): Use
+ --no-warnings=ascii for SHIFT_JIS and SHIFT_JISX0213 charmaps.
+
2017-10-15 Carlos O'Donell <carlos@redhat.com>
* localedata/collate-test.c (allocate_arrays): Don't use \n in
diff --git a/locale/programs/charmap.c b/locale/programs/charmap.c
index a670db9..964d932 100644
--- a/locale/programs/charmap.c
+++ b/locale/programs/charmap.c
@@ -256,9 +256,15 @@ charmap_read (const char *filename, int verbose, int error_not_found,
if (failed)
{
- record_warning (_("\
-character map `%s' is not ASCII compatible, locale not ISO C compliant\n"),
- result->code_set_name);
+ /* A user may disable the ASCII compatibility warning check,
+ but we must remember that the encoding is not ASCII
+ compatible, since it may have other implications. Later
+ we will set _NL_CTYPE_MAP_TO_NONASCII from this value. */
+ if (warn_ascii)
+ record_warning (_(
+"character map `%s' is not ASCII compatible, locale not ISO C compliant "
+"[--no-warnings=ascii]"),
+ result->code_set_name);
enc_not_ascii_compatible = true;
}
}
diff --git a/locale/programs/charmap.h b/locale/programs/charmap.h
index 5d6b48f..441d429 100644
--- a/locale/programs/charmap.h
+++ b/locale/programs/charmap.h
@@ -66,6 +66,8 @@ struct charseq
/* True if the encoding is not ASCII compatible. */
extern bool enc_not_ascii_compatible;
+/* True if the ASCII compatibility check should raise a warning. */
+bool warn_ascii;
/* Prototypes for charmap handling functions. */
diff --git a/locale/programs/ld-monetary.c b/locale/programs/ld-monetary.c
index 9d94738..71df376 100644
--- a/locale/programs/ld-monetary.c
+++ b/locale/programs/ld-monetary.c
@@ -234,12 +234,17 @@ No definition for %s category found"), "LC_MONETARY");
char symbol[4];
strncpy (symbol, monetary->int_curr_symbol, 3);
symbol[3] = '\0';
+ /* A user may disable this waning for testing purposes or
+ for building a locale with a 3 digit country code that
+ was not yet supported in our ISO 4217 list.
+ See the use of --no-warnings=intcurrsym. */
if (bsearch (symbol, valid_int_curr, NR_VALID_INT_CURR,
sizeof (const char *),
- (comparison_fn_t) curr_strcmp) == NULL)
+ (comparison_fn_t) curr_strcmp) == NULL
+ && warn_int_curr_symbol)
record_warning (_("\
%s: value of field `int_curr_symbol' does \
-not correspond to a valid name in ISO 4217"),
+not correspond to a valid name in ISO 4217 [--no-warnings=intcurrsym]"),
"LC_MONETARY");
}
}
diff --git a/locale/programs/localedef.c b/locale/programs/localedef.c
index 7d76154..a5e474b 100644
--- a/locale/programs/localedef.c
+++ b/locale/programs/localedef.c
@@ -32,6 +32,7 @@
#include <error.h>
#include <sys/mman.h>
#include <sys/stat.h>
+#include <ctype.h>
#include "localedef.h"
#include "charmap.h"
@@ -51,6 +52,13 @@ int posix_conformance;
/* If not zero give a lot more messages. */
int verbose;
+/* Warnings which can be disabled: */
+/* By default we check the character map for ASCII compatibility. */
+bool warn_ascii = true;
+/* By default we check that the international currency symbol matches a
+ known country code. */
+bool warn_int_curr_symbol = true;
+
/* Warnings recorded by record_warnings (see localedef.h). */
int recorded_warning_count;
@@ -114,6 +122,8 @@ void (*argp_program_version_hook) (FILE *, struct argp_state *) = print_version;
#define OPT_LIST_ARCHIVE 309
#define OPT_LITTLE_ENDIAN 400
#define OPT_BIG_ENDIAN 401
+#define OPT_NO_WARN 402
+#define OPT_WARN 403
/* Definitions of arguments for argp functions. */
static const struct argp_option options[] =
@@ -134,6 +144,13 @@ static const struct argp_option options[] =
{ "quiet", OPT_QUIET, NULL, 0,
N_("Suppress warnings and information messages") },
{ "verbose", 'v', NULL, 0, N_("Print more messages") },
+ { "no-warnings", OPT_NO_WARN, N_("<warnings>"), 0,
+ N_("Comma-separated list of warnings to disable; "
+ "supported warnings are: ascii, intcurrsym") },
+ { "warnings", OPT_WARN, N_("<warnings>"), 0,
+ N_("Comma-separated list of warnings to enable; "
+ "supported warnings are: ascii, intcurrsym") },
+
{ NULL, 0, NULL, 0, N_("Archive control:") },
{ "no-archive", OPT_NO_ARCHIVE, NULL, 0,
N_("Don't add new data to archive") },
@@ -309,6 +326,45 @@ no output file produced because errors were issued"));
exit (recorded_warning_count != 0);
}
+/* Search warnings for matching warnings and if found enable those
+ warnings if ENABLED is true, otherwise disable the warnings. */
+static void
+set_warnings (char *warnings, bool enabled)
+{
+ char *tok;
+ char *save;
+ char *copy = (char *) malloc (strlen (warnings) + 1);
+
+ /* Remove all spaces from the warnings list to make the processing
+ a more robust. We don't support spaces in a warning name. */
+
+ save = copy;
+ tok = warnings;
+
+ do {
+ while (isspace (*tok))
+ tok++;
+ } while ((*save++ = *tok++));
+
+ warnings = copy;
+
+ /* Tokenize the input list of warnings to set, compare them to
+ known warnings, and set the warning. We purposely ignore unknown
+ warnings, and are thus forward compatible, users can attempt to
+ disable whaterver new warnings they know about, but we will only
+ disable those *we* known about. */
+
+ while ((tok = strtok_r (warnings, ",", &save)) != NULL)
+ {
+ warnings = NULL;
+ if (strcmp (tok, "ascii") == 0)
+ warn_ascii = enabled;
+ else if (strcmp (tok, "intcurrsym") == 0)
+ warn_int_curr_symbol = enabled;
+ }
+
+ free (copy);
+}
/* Handle program arguments. */
static error_t
@@ -346,6 +402,14 @@ parse_opt (int key, char *arg, struct argp_state *state)
case OPT_BIG_ENDIAN:
set_big_endian (true);
break;
+ case OPT_NO_WARN:
+ /* Disable the warnings. */
+ set_warnings (arg, false);
+ break;
+ case OPT_WARN:
+ /* Enable the warnings. */
+ set_warnings (arg, true);
+ break;
case 'c':
force_output = 1;
break;
diff --git a/locale/programs/localedef.h b/locale/programs/localedef.h
index 96aa696..e4f22ce 100644
--- a/locale/programs/localedef.h
+++ b/locale/programs/localedef.h
@@ -115,6 +115,7 @@ struct localedef_t
/* Global variables of the localedef program. */
extern int verbose;
+extern bool warn_int_curr_symbol;
extern const char *repertoire_global;
extern int max_locarchive_open_retry;
extern bool no_archive;
diff --git a/localedata/Makefile b/localedata/Makefile
index 75f71bb..a5f3c92 100644
--- a/localedata/Makefile
+++ b/localedata/Makefile
@@ -200,17 +200,25 @@ install-locales: $(INSTALL-SUPPORTED-LOCALES)
install-locales-dir:
$(..)./scripts/mkinstalldirs $(inst_complocaledir)
+# The SHIFT_JIS and SHIFT_JISX0213 character maps are not ASCII compatible,
+# therefore we have to use --no-warnings=ascii to disable the ASCII check.
+# See localedata/gen-locale.sh for the same logic.
$(INSTALL-SUPPORTED-LOCALES): install-locales-dir
@locale=`echo $@ | sed -e 's/^install-//'`; \
charset=`echo $$locale | sed -e 's,.*/,,'`; \
locale=`echo $$locale | sed -e 's,/[^/]*,,'`; \
+ flags="--quiet -c"; \
+ if [ "$$charset" = 'SHIFT_JIS' ] \
+ || [ "$$charset" = 'SHIFT_JISX0213' ]; then \
+ flags="$$flags --no-warnings=ascii"; \
+ fi; \
echo -n `echo $$locale | sed 's/\([^.\@]*\).*/\1/'`; \
echo -n ".$$charset"; \
echo -n `echo $$locale | sed 's/\([^\@]*\)\(\@.*\)*/\2/'`; \
echo -n '...'; \
input=`echo $$locale | sed 's/\([^.]*\)[^@]*\(.*\)/\1\2/'`; \
- $(LOCALEDEF) --alias-file=../intl/locale.alias \
- -i locales/$$input -c -f charmaps/$$charset \
+ $(LOCALEDEF) $$flags --alias-file=../intl/locale.alias \
+ -i locales/$$input -f charmaps/$$charset \
$(addprefix --prefix=,$(install_root)) $$locale \
&& echo ' done'; \
diff --git a/localedata/gen-locale.sh b/localedata/gen-locale.sh
index b4ec68c..757a0e9 100644
--- a/localedata/gen-locale.sh
+++ b/localedata/gen-locale.sh
@@ -30,16 +30,12 @@ generate_locale ()
charmap=$1
input=$2
out=$3
+ flags=$4
ret=0
${localedef_before_env} ${run_program_env} I18NPATH=../localedata \
- ${localedef_after_env} --quiet -c -f $charmap -i $input \
+ ${localedef_after_env} $flags -f $charmap -i $input \
${common_objpfx}localedata/$out || ret=$?
- # All locales compile fine, except those with SHIFT_JIS charmap
- # and those fail with exit code 1 because SHIFT_JIS issues a
- # warning (it is not ASCII compatible).
- if [ $ret -eq 0 ] \
- || ( [ $ret -eq 1 ] \
- && [ "$charmap" = "SHIFT_JIS" ] ); then
+ if [ $ret -eq 0 ]; then
# The makefile checks the timestamp of the LC_CTYPE file,
# but localedef won't have touched it if it was able to
# hard-link it to an existing file.
@@ -57,6 +53,9 @@ charmap=`echo $locfile|sed 's|[^.]*[.]\(.*\)/LC_CTYPE|\1|'`
echo "Generating locale $locale.$charmap: this might take a while..."
+# Run quietly and force output.
+flags="--quiet -c"
+
# For SJIS the charmap is SHIFT_JIS. We just want the locale to have
# a slightly nicer name instead of using "*.SHIFT_SJIS", but that
# means we need a mapping here.
@@ -65,4 +64,13 @@ if [ "$charmap" = "SJIS" ]; then
charmap_real="SHIFT_JIS"
fi
-generate_locale $charmap_real $locale $locale.$charmap
+# In addition to this the SHIFT_JIS character maps are not ASCII
+# compatible so we must use `--no-warnings=ascii' to disable the
+# warning. See localedata/Makefile $(INSTALL-SUPPORTED-LOCALES)
+# for the same logic.
+if [ "$charmap_real" = 'SHIFT_JIS' ] \
+ || [ "$charmpa_real" = 'SHIFT_JISX0213' ]; then
+ flags="$flags --no-warnings=ascii"
+fi
+
+generate_locale $charmap_real $locale $locale.$charmap "$flags"
--
2.9.5
From localedef --help: Output control: ... --no-warnings=<warnings> Comma-separated list of warnings to disable; supported warnings are: ascii, intcurrsym ... --warnings=<warnings> Comma-separated list of warnings to enable; supported warnings are: ascii, intcurrsym Locales using SHIFT_JIS and SHIFT_JISX0213 character maps are not ASCII compatible. In order to build locales using these character maps, and have localedef exit with a status of 0, we add new option to localedef to disable or enable specific warnings. The options are --no-warnings and --warnings, to disable and enable specific warnings respectively. The options take a comma-separated list of warning names. The warning names are taken directly from the generated warning. When a warning that can be disabled is issued it will print something like this: foo is not defined [--no-warnings=foo] For the initial implementation we add two controllable warnings; first 'ascii' which is used by the localedata installation makefile target to install SHIFT_JIS and SHIFT_JISX0213-using locales without error; second 'intcurrsym' which allows a program to use a non-standard international currency symbol without triggering a warning. The 'intcurrsym' is useful in the future if country codes are added that are not in our current ISO 4217 list, and the user wants to avoid the warning. Having at least two warnings to control gives an example for how the changes can be extended to more warnings if required in the future. These changes allow ja_JP.SHIFT_JIS and ja_JP.SHIFT_JISX0213 to be compiled without warnings using --no-warnings=ascii. The localedata/Makefile $(INSTALL-SUPPORTED-LOCALES) target is adjusted to automatically add `--no-warnings=ascii` for such charmaps, and likewise localedata/gen-locale.sh is adjusted with similar logic. Signed-off-by: Carlos O'Donell <carlos@redhat.com> --- ChangeLog | 25 +++++++++++++++++ locale/programs/charmap.c | 12 ++++++-- locale/programs/charmap.h | 2 ++ locale/programs/ld-monetary.c | 9 ++++-- locale/programs/localedef.c | 64 +++++++++++++++++++++++++++++++++++++++++++ locale/programs/localedef.h | 1 + localedata/Makefile | 12 ++++++-- localedata/gen-locale.sh | 24 ++++++++++------ 8 files changed, 134 insertions(+), 15 deletions(-)