| Message ID | 20230418213241.28635-1-ps.report@gmx.net |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [RFC,v1] package/python-babel: add purge locale data support | expand |
Peter, All, On 2023-04-18 23:32 +0200, Peter Seiderer spake thusly: > Add purge loacle data support (inspired by BR2_ENABLE_LOCALE_PURGE > handling in Makefile) to reduce target space allocation by the > python-babel package, e.g. from 32MB to 24K for > target/usr/lib/python3.11/site-packages/babel/locale-data with > BR2_ENABLE_LOCALE_WHITELIST="C en_US de_DE". > > Signed-off-by: Peter Seiderer <ps.report@gmx.net> > --- > package/python-babel/python-babel.mk | 12 ++++++++++++ > 1 file changed, 12 insertions(+) > > diff --git a/package/python-babel/python-babel.mk b/package/python-babel/python-babel.mk > index 890a8b02b9..ea383e9c05 100644 > --- a/package/python-babel/python-babel.mk > +++ b/package/python-babel/python-babel.mk > @@ -12,5 +12,17 @@ PYTHON_BABEL_LICENSE = BSD-3-Clause > PYTHON_BABEL_LICENSE_FILES = LICENSE > HOST_PYTHON_BABEL_DEPENDENCIES = host-python-pytz > > +# purge locale data (if enabled), keep special en_US_POSIX data by default > +ifeq ($(BR2_ENABLE_LOCALE_PURGE),y) > +define PYTHON_BABEL_CLEANUP_LOCALE > + for i in `ls $(TARGET_DIR)/usr/lib/python3.11/site-packages/babel/locale-data/*.dat`; \ > + do \ > + i_base=`basename "$$i" .dat`; \ > + echo "$(BR2_ENABLE_LOCALE_WHITELIST) en_US_POSIX" | grep -qw "$$i_base" || rm "$$i"; \ Why do you force keeping the en_US_POSIX locale specifically? If there is the need that at least one locale be present, then can we keep a more generic one, like 'C', or 'POSIX', or whatever is not specific to a country? > + done > +endef > +endif > +PYTHON_BABEL_POST_INSTALL_TARGET_HOOKS += PYTHON_BABEL_CLEANUP_LOCALE Like for the toolchain locale purge, it should be a target-fnalize hook; PYTHON_BABEL_TARGET_FINALIZE_HOOKS (Note that the systemd locale purge is done as a ROOTFS_PRE_CMD_HOOKS, but that is wrong, IMNSHO, and it too should have been a target finalize hook.) Regards, Yann E. MORIN. > $(eval $(python-package)) > $(eval $(host-python-package)) > -- > 2.40.0 > > _______________________________________________ > buildroot mailing list > buildroot@buildroot.org > https://lists.buildroot.org/mailman/listinfo/buildroot
Hello Yann, On Mon, 15 May 2023 21:57:31 +0200, "Yann E. MORIN" <yann.morin.1998@free.fr> wrote: > Peter, All, > > On 2023-04-18 23:32 +0200, Peter Seiderer spake thusly: > > Add purge loacle data support (inspired by BR2_ENABLE_LOCALE_PURGE > > handling in Makefile) to reduce target space allocation by the > > python-babel package, e.g. from 32MB to 24K for > > target/usr/lib/python3.11/site-packages/babel/locale-data with > > BR2_ENABLE_LOCALE_WHITELIST="C en_US de_DE". > > > > Signed-off-by: Peter Seiderer <ps.report@gmx.net> > > --- > > package/python-babel/python-babel.mk | 12 ++++++++++++ > > 1 file changed, 12 insertions(+) > > > > diff --git a/package/python-babel/python-babel.mk b/package/python-babel/python-babel.mk > > index 890a8b02b9..ea383e9c05 100644 > > --- a/package/python-babel/python-babel.mk > > +++ b/package/python-babel/python-babel.mk > > @@ -12,5 +12,17 @@ PYTHON_BABEL_LICENSE = BSD-3-Clause > > PYTHON_BABEL_LICENSE_FILES = LICENSE > > HOST_PYTHON_BABEL_DEPENDENCIES = host-python-pytz > > > > +# purge locale data (if enabled), keep special en_US_POSIX data by default > > +ifeq ($(BR2_ENABLE_LOCALE_PURGE),y) > > +define PYTHON_BABEL_CLEANUP_LOCALE > > + for i in `ls $(TARGET_DIR)/usr/lib/python3.11/site-packages/babel/locale-data/*.dat`; \ > > + do \ > > + i_base=`basename "$$i" .dat`; \ > > + echo "$(BR2_ENABLE_LOCALE_WHITELIST) en_US_POSIX" | grep -qw "$$i_base" || rm "$$i"; \ > > Why do you force keeping the en_US_POSIX locale specifically? Because it is the python-babel default, see [1]... > > If there is the need that at least one locale be present, then can we > keep a more generic one, like 'C', or 'POSIX', or whatever is not > specific to a country? See above... > > > + done > > +endef > > +endif > > +PYTHON_BABEL_POST_INSTALL_TARGET_HOOKS += PYTHON_BABEL_CLEANUP_LOCALE > > Like for the toolchain locale purge, it should be a target-fnalize hook; > PYTHON_BABEL_TARGET_FINALIZE_HOOKS Will fix it... Thanks for review! Regards, Peter [1] https://github.com/python-babel/babel/blob/0ce196fccc024b1a65453ba6519954ada1dab6cb/babel/core.py#L1076 > > (Note that the systemd locale purge is done as a ROOTFS_PRE_CMD_HOOKS, > but that is wrong, IMNSHO, and it too should have been a target finalize > hook.) > > Regards, > Yann E. MORIN. > > > $(eval $(python-package)) > > $(eval $(host-python-package)) > > -- > > 2.40.0 > > > > _______________________________________________ > > buildroot mailing list > > buildroot@buildroot.org > > https://lists.buildroot.org/mailman/listinfo/buildroot >
Peter, All, On 2023-05-15 22:49 +0200, Peter Seiderer spake thusly: > On Mon, 15 May 2023 21:57:31 +0200, "Yann E. MORIN" <yann.morin.1998@free.fr> wrote: > > On 2023-04-18 23:32 +0200, Peter Seiderer spake thusly: > > > Add purge loacle data support (inspired by BR2_ENABLE_LOCALE_PURGE > > > handling in Makefile) to reduce target space allocation by the > > > python-babel package, e.g. from 32MB to 24K for > > > target/usr/lib/python3.11/site-packages/babel/locale-data with > > > BR2_ENABLE_LOCALE_WHITELIST="C en_US de_DE". [--SNIP--] > > > + echo "$(BR2_ENABLE_LOCALE_WHITELIST) en_US_POSIX" | grep -qw "$$i_base" || rm "$$i"; \ > > Why do you force keeping the en_US_POSIX locale specifically? > Because it is the python-babel default, see [1]... OK, Thanks! Say so explicitly in the commit log when you respin. Regards, Yann E. MORIN.
On Tue, 18 Apr 2023 23:32:41 +0200 Peter Seiderer <ps.report@gmx.net> wrote: > Add purge loacle data support (inspired by BR2_ENABLE_LOCALE_PURGE > handling in Makefile) to reduce target space allocation by the > python-babel package, e.g. from 32MB to 24K for > target/usr/lib/python3.11/site-packages/babel/locale-data with > BR2_ENABLE_LOCALE_WHITELIST="C en_US de_DE". I updated the commit log to add the explanation about en_US_POSIX, as requested by Yann. > > Signed-off-by: Peter Seiderer <ps.report@gmx.net> > --- > package/python-babel/python-babel.mk | 12 ++++++++++++ > 1 file changed, 12 insertions(+) > > diff --git a/package/python-babel/python-babel.mk b/package/python-babel/python-babel.mk > index 890a8b02b9..ea383e9c05 100644 > --- a/package/python-babel/python-babel.mk > +++ b/package/python-babel/python-babel.mk > @@ -12,5 +12,17 @@ PYTHON_BABEL_LICENSE = BSD-3-Clause > PYTHON_BABEL_LICENSE_FILES = LICENSE > HOST_PYTHON_BABEL_DEPENDENCIES = host-python-pytz > > +# purge locale data (if enabled), keep special en_US_POSIX data by default > +ifeq ($(BR2_ENABLE_LOCALE_PURGE),y) > +define PYTHON_BABEL_CLEANUP_LOCALE > + for i in `ls $(TARGET_DIR)/usr/lib/python3.11/site-packages/babel/locale-data/*.dat`; \ The hardcoded 3.11 was not good, it would break at the next major bump of Python, replace with $(PYTHON3_VERSION_MAJOR). > + do \ > + i_base=`basename "$$i" .dat`; \ > + echo "$(BR2_ENABLE_LOCALE_WHITELIST) en_US_POSIX" | grep -qw "$$i_base" || rm "$$i"; \ > + done > +endef > +endif > +PYTHON_BABEL_POST_INSTALL_TARGET_HOOKS += PYTHON_BABEL_CLEANUP_LOCALE Changed to PYTHON_LABEL_TARGET_FINALIZE_HOOKS, as suggested by Yann, and moved inside the ifeq ... endif block. Applied with those changes. Thanks a lot! Thomas
Hello Thomas, *, On Tue, 1 Aug 2023 00:07:35 +0200, Thomas Petazzoni via buildroot <buildroot@buildroot.org> wrote: > On Tue, 18 Apr 2023 23:32:41 +0200 > Peter Seiderer <ps.report@gmx.net> wrote: > > > Add purge loacle data support (inspired by BR2_ENABLE_LOCALE_PURGE > > handling in Makefile) to reduce target space allocation by the > > python-babel package, e.g. from 32MB to 24K for > > target/usr/lib/python3.11/site-packages/babel/locale-data with > > BR2_ENABLE_LOCALE_WHITELIST="C en_US de_DE". > > I updated the commit log to add the explanation about en_US_POSIX, as > requested by Yann. > > > > > Signed-off-by: Peter Seiderer <ps.report@gmx.net> > > --- > > package/python-babel/python-babel.mk | 12 ++++++++++++ > > 1 file changed, 12 insertions(+) > > > > diff --git a/package/python-babel/python-babel.mk b/package/python-babel/python-babel.mk > > index 890a8b02b9..ea383e9c05 100644 > > --- a/package/python-babel/python-babel.mk > > +++ b/package/python-babel/python-babel.mk > > @@ -12,5 +12,17 @@ PYTHON_BABEL_LICENSE = BSD-3-Clause > > PYTHON_BABEL_LICENSE_FILES = LICENSE > > HOST_PYTHON_BABEL_DEPENDENCIES = host-python-pytz > > > > +# purge locale data (if enabled), keep special en_US_POSIX data by default > > +ifeq ($(BR2_ENABLE_LOCALE_PURGE),y) > > +define PYTHON_BABEL_CLEANUP_LOCALE > > + for i in `ls $(TARGET_DIR)/usr/lib/python3.11/site-packages/babel/locale-data/*.dat`; \ > > The hardcoded 3.11 was not good, it would break at the next major bump > of Python, replace with $(PYTHON3_VERSION_MAJOR). > > > + do \ > > + i_base=`basename "$$i" .dat`; \ > > + echo "$(BR2_ENABLE_LOCALE_WHITELIST) en_US_POSIX" | grep -qw "$$i_base" || rm "$$i"; \ > > + done > > +endef > > +endif > > +PYTHON_BABEL_POST_INSTALL_TARGET_HOOKS += PYTHON_BABEL_CLEANUP_LOCALE > > Changed to PYTHON_LABEL_TARGET_FINALIZE_HOOKS, as suggested by Yann, > and moved inside the ifeq ... endif block. > > Applied with those changes. > > Thanks a lot! Seems I missed to send an updated (non RFC) version, many thanks for fixing and applying.... Regards, Peter > > Thomas
diff --git a/package/python-babel/python-babel.mk b/package/python-babel/python-babel.mk index 890a8b02b9..ea383e9c05 100644 --- a/package/python-babel/python-babel.mk +++ b/package/python-babel/python-babel.mk @@ -12,5 +12,17 @@ PYTHON_BABEL_LICENSE = BSD-3-Clause PYTHON_BABEL_LICENSE_FILES = LICENSE HOST_PYTHON_BABEL_DEPENDENCIES = host-python-pytz +# purge locale data (if enabled), keep special en_US_POSIX data by default +ifeq ($(BR2_ENABLE_LOCALE_PURGE),y) +define PYTHON_BABEL_CLEANUP_LOCALE + for i in `ls $(TARGET_DIR)/usr/lib/python3.11/site-packages/babel/locale-data/*.dat`; \ + do \ + i_base=`basename "$$i" .dat`; \ + echo "$(BR2_ENABLE_LOCALE_WHITELIST) en_US_POSIX" | grep -qw "$$i_base" || rm "$$i"; \ + done +endef +endif +PYTHON_BABEL_POST_INSTALL_TARGET_HOOKS += PYTHON_BABEL_CLEANUP_LOCALE + $(eval $(python-package)) $(eval $(host-python-package))
Add purge loacle data support (inspired by BR2_ENABLE_LOCALE_PURGE handling in Makefile) to reduce target space allocation by the python-babel package, e.g. from 32MB to 24K for target/usr/lib/python3.11/site-packages/babel/locale-data with BR2_ENABLE_LOCALE_WHITELIST="C en_US de_DE". Signed-off-by: Peter Seiderer <ps.report@gmx.net> --- package/python-babel/python-babel.mk | 12 ++++++++++++ 1 file changed, 12 insertions(+)