Message ID | 1398356679-1438-1-git-send-email-fabio.porcedda@gmail.com |
---|---|
State | Superseded |
Headers | show |
Dear Fabio Porcedda, On Thu, 24 Apr 2014 18:24:39 +0200, Fabio Porcedda wrote: > -target-purgelocales: > +target-purgelocales: $(filter-out target-purgelocales,$(TARGETS)) > rm -f $(LOCALE_WHITELIST) > for i in $(LOCALE_NOPURGE); do echo $$i >> $(LOCALE_WHITELIST); done Don't we have several other targets that need to be executed only after all packages have been built and installed? Wouldn't it make sense to have a common solution for these? Like maybe a dedicated target? Thomas
On Thu, Apr 24, 2014 at 6:41 PM, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote: > Dear Fabio Porcedda, > > On Thu, 24 Apr 2014 18:24:39 +0200, Fabio Porcedda wrote: > >> -target-purgelocales: >> +target-purgelocales: $(filter-out target-purgelocales,$(TARGETS)) >> rm -f $(LOCALE_WHITELIST) >> for i in $(LOCALE_NOPURGE); do echo $$i >> $(LOCALE_WHITELIST); done > > Don't we have several other targets that need to be executed only after > all packages have been built and installed? Wouldn't it make sense to > have a common solution for these? Like maybe a dedicated target? Can you please give some examples? I know only tatget-purgelocales and target-finalize. About the common solution, i see two possible solutions of the problem: 1) all those targets must be listed in a variable like TARGETS_PRE_ROOTFS, but those targets must be able to be executed in parallel without a specific order. 2) all those targets must be converted in hooks and added to a variable like PRE_ROOTFS_HOOKS, so those steps are going to be executed in serial observing a specific order. What is the more appropriate solution? The easiest and fastest one is the first, but i'm not sure if those targets can be executed in parallel. Regards -- Fabio Porcedda
On 25/04/14 23:50, Fabio Porcedda wrote: > On Thu, Apr 24, 2014 at 6:41 PM, Thomas Petazzoni > <thomas.petazzoni@free-electrons.com> wrote: >> Dear Fabio Porcedda, >> >> On Thu, 24 Apr 2014 18:24:39 +0200, Fabio Porcedda wrote: >> >>> -target-purgelocales: >>> +target-purgelocales: $(filter-out target-purgelocales,$(TARGETS)) >>> rm -f $(LOCALE_WHITELIST) >>> for i in $(LOCALE_NOPURGE); do echo $$i >> $(LOCALE_WHITELIST); done >> >> Don't we have several other targets that need to be executed only after >> all packages have been built and installed? Wouldn't it make sense to >> have a common solution for these? Like maybe a dedicated target? > > Can you please give some examples? I know only tatget-purgelocales and > target-finalize. > > About the common solution, i see two possible solutions of the problem: > > 1) all those targets must be listed in a variable like > TARGETS_PRE_ROOTFS, but those targets must be able to be executed in > parallel without a specific order. > > 2) all those targets must be converted in hooks and added to a > variable like PRE_ROOTFS_HOOKS, so those steps are going to be > executed in serial observing a specific order. > > What is the more appropriate solution? The easiest and fastest one is > the first, but i'm not sure if those targets can be executed in > parallel. My personal preference is to have a single rule (e.g. target-finalize) that performs everything that is post-targets and pre-rootfs. There isn't much that needs to be done so parallelisation doesn't make sense. And I think it's much easier to understand which steps are executed and in which order if they are all put together in a single rule rather than spread out over several. To make things more readable, we can put the commands into separate variables. For instance: define TARGET_PURGE_LOCALES rm -f $(LOCALE_WHITELIST) ... endef define TARGET_PURGE_DEVFILES rm -rf $(TARGET_DIR)/usr/include ... ... endef ifneq ($(BR2_PACKAGE_GDB),y) define TARGET_PURGE_GDB rm -rf $(TARGET_DIR)/usr/share/gdb endef endif target-finalize: $(TARGETS) $(TARGET_PURGE_LOCALES) $(TARGET_PURGE_DEVFILES) $(TARGET_PURGE_GDB) $(TARGET_PURGE_DOC) ... I'm giving the content of target-finalize as an example here, but it's not immediately needed to convert those into variables. The different target-* steps that are currently added to TARGETS are rather more important. Regards, Arnout
On Mon, Apr 28, 2014 at 7:49 AM, Arnout Vandecappelle <arnout@mind.be> wrote: > On 25/04/14 23:50, Fabio Porcedda wrote: >> On Thu, Apr 24, 2014 at 6:41 PM, Thomas Petazzoni >> <thomas.petazzoni@free-electrons.com> wrote: >>> Dear Fabio Porcedda, >>> >>> On Thu, 24 Apr 2014 18:24:39 +0200, Fabio Porcedda wrote: >>> >>>> -target-purgelocales: >>>> +target-purgelocales: $(filter-out target-purgelocales,$(TARGETS)) >>>> rm -f $(LOCALE_WHITELIST) >>>> for i in $(LOCALE_NOPURGE); do echo $$i >> $(LOCALE_WHITELIST); done >>> >>> Don't we have several other targets that need to be executed only after >>> all packages have been built and installed? Wouldn't it make sense to >>> have a common solution for these? Like maybe a dedicated target? >> >> Can you please give some examples? I know only tatget-purgelocales and >> target-finalize. >> >> About the common solution, i see two possible solutions of the problem: >> >> 1) all those targets must be listed in a variable like >> TARGETS_PRE_ROOTFS, but those targets must be able to be executed in >> parallel without a specific order. >> >> 2) all those targets must be converted in hooks and added to a >> variable like PRE_ROOTFS_HOOKS, so those steps are going to be >> executed in serial observing a specific order. >> >> What is the more appropriate solution? The easiest and fastest one is >> the first, but i'm not sure if those targets can be executed in >> parallel. > > My personal preference is to have a single rule (e.g. target-finalize) > that performs everything that is post-targets and pre-rootfs. There isn't > much that needs to be done so parallelisation doesn't make sense. And I > think it's much easier to understand which steps are executed and in > which order if they are all put together in a single rule rather than > spread out over several. > > To make things more readable, we can put the commands into separate > variables. For instance: > > define TARGET_PURGE_LOCALES > rm -f $(LOCALE_WHITELIST) > ... > endef > > define TARGET_PURGE_DEVFILES > rm -rf $(TARGET_DIR)/usr/include ... > ... > endef > > ifneq ($(BR2_PACKAGE_GDB),y) > define TARGET_PURGE_GDB > rm -rf $(TARGET_DIR)/usr/share/gdb > endef > endif > > target-finalize: $(TARGETS) > $(TARGET_PURGE_LOCALES) > $(TARGET_PURGE_DEVFILES) > $(TARGET_PURGE_GDB) > $(TARGET_PURGE_DOC) > ... > > > I'm giving the content of target-finalize as an example here, but it's > not immediately needed to convert those into variables. The different > target-* steps that are currently added to TARGETS are rather more important. It's fine for me. So you prefer to list those variables inside the target instead of adding them to a hook variable? like e.g. PRE_ROOTFS_HOOK? Regards
>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes: Hi, >> What is the more appropriate solution? The easiest and fastest one is >> the first, but i'm not sure if those targets can be executed in >> parallel. > My personal preference is to have a single rule (e.g. target-finalize) > that performs everything that is post-targets and pre-rootfs. There isn't > much that needs to be done so parallelisation doesn't make sense. And I > think it's much easier to understand which steps are executed and in > which order if they are all put together in a single rule rather than > spread out over several. > To make things more readable, we can put the commands into separate > variables. For instance: > define TARGET_PURGE_LOCALES > rm -f $(LOCALE_WHITELIST) > ... > endef > define TARGET_PURGE_DEVFILES > rm -rf $(TARGET_DIR)/usr/include ... > ... > endef > ifneq ($(BR2_PACKAGE_GDB),y) > define TARGET_PURGE_GDB > rm -rf $(TARGET_DIR)/usr/share/gdb > endef > endif > target-finalize: $(TARGETS) > $(TARGET_PURGE_LOCALES) > $(TARGET_PURGE_DEVFILES) > $(TARGET_PURGE_GDB) > $(TARGET_PURGE_DOC) > ... Yes, that looks nice and clear to me too.
On 28/04/14 09:58, Fabio Porcedda wrote: > On Mon, Apr 28, 2014 at 7:49 AM, Arnout Vandecappelle <arnout@mind.be> wrote: >> On 25/04/14 23:50, Fabio Porcedda wrote: >>> On Thu, Apr 24, 2014 at 6:41 PM, Thomas Petazzoni >>> <thomas.petazzoni@free-electrons.com> wrote: >>>> Dear Fabio Porcedda, >>>> >>>> On Thu, 24 Apr 2014 18:24:39 +0200, Fabio Porcedda wrote: >>>> >>>>> -target-purgelocales: >>>>> +target-purgelocales: $(filter-out target-purgelocales,$(TARGETS)) >>>>> rm -f $(LOCALE_WHITELIST) >>>>> for i in $(LOCALE_NOPURGE); do echo $$i >> $(LOCALE_WHITELIST); done >>>> >>>> Don't we have several other targets that need to be executed only after >>>> all packages have been built and installed? Wouldn't it make sense to >>>> have a common solution for these? Like maybe a dedicated target? >>> >>> Can you please give some examples? I know only tatget-purgelocales and >>> target-finalize. >>> >>> About the common solution, i see two possible solutions of the problem: >>> >>> 1) all those targets must be listed in a variable like >>> TARGETS_PRE_ROOTFS, but those targets must be able to be executed in >>> parallel without a specific order. >>> >>> 2) all those targets must be converted in hooks and added to a >>> variable like PRE_ROOTFS_HOOKS, so those steps are going to be >>> executed in serial observing a specific order. >>> >>> What is the more appropriate solution? The easiest and fastest one is >>> the first, but i'm not sure if those targets can be executed in >>> parallel. >> >> My personal preference is to have a single rule (e.g. target-finalize) >> that performs everything that is post-targets and pre-rootfs. There isn't >> much that needs to be done so parallelisation doesn't make sense. And I >> think it's much easier to understand which steps are executed and in >> which order if they are all put together in a single rule rather than >> spread out over several. >> >> To make things more readable, we can put the commands into separate >> variables. For instance: >> >> define TARGET_PURGE_LOCALES >> rm -f $(LOCALE_WHITELIST) >> ... >> endef >> >> define TARGET_PURGE_DEVFILES >> rm -rf $(TARGET_DIR)/usr/include ... >> ... >> endef >> >> ifneq ($(BR2_PACKAGE_GDB),y) >> define TARGET_PURGE_GDB >> rm -rf $(TARGET_DIR)/usr/share/gdb >> endef >> endif >> >> target-finalize: $(TARGETS) >> $(TARGET_PURGE_LOCALES) >> $(TARGET_PURGE_DEVFILES) >> $(TARGET_PURGE_GDB) >> $(TARGET_PURGE_DOC) >> ... >> >> >> I'm giving the content of target-finalize as an example here, but it's >> not immediately needed to convert those into variables. The different >> target-* steps that are currently added to TARGETS are rather more important. > > It's fine for me. > So you prefer to list those variables inside the target instead of > adding them to a hook variable? like e.g. PRE_ROOTFS_HOOK? Indeed. The HOOKS way of working is not very clear. The code for calling the hooks is not easy to understand at first sight, because it contains this $(foreach ...) and $(sep) stuff. Also, the assignments to the HOOKS variable are spread out over the Makefiles so it's hard to see which things will get executed and in which order. And finally, it's not needed here because we can enumerate all the possible steps - that's different from the package infrastructure, where the HOOK contents differs from package to package. Regards, Arnout
On Mon, Apr 28, 2014 at 12:30 PM, Arnout Vandecappelle <arnout@mind.be> wrote: > On 28/04/14 09:58, Fabio Porcedda wrote: >> On Mon, Apr 28, 2014 at 7:49 AM, Arnout Vandecappelle <arnout@mind.be> wrote: >>> On 25/04/14 23:50, Fabio Porcedda wrote: >>>> On Thu, Apr 24, 2014 at 6:41 PM, Thomas Petazzoni >>>> <thomas.petazzoni@free-electrons.com> wrote: >>>>> Dear Fabio Porcedda, >>>>> >>>>> On Thu, 24 Apr 2014 18:24:39 +0200, Fabio Porcedda wrote: >>>>> >>>>>> -target-purgelocales: >>>>>> +target-purgelocales: $(filter-out target-purgelocales,$(TARGETS)) >>>>>> rm -f $(LOCALE_WHITELIST) >>>>>> for i in $(LOCALE_NOPURGE); do echo $$i >> $(LOCALE_WHITELIST); done >>>>> >>>>> Don't we have several other targets that need to be executed only after >>>>> all packages have been built and installed? Wouldn't it make sense to >>>>> have a common solution for these? Like maybe a dedicated target? >>>> >>>> Can you please give some examples? I know only tatget-purgelocales and >>>> target-finalize. >>>> >>>> About the common solution, i see two possible solutions of the problem: >>>> >>>> 1) all those targets must be listed in a variable like >>>> TARGETS_PRE_ROOTFS, but those targets must be able to be executed in >>>> parallel without a specific order. >>>> >>>> 2) all those targets must be converted in hooks and added to a >>>> variable like PRE_ROOTFS_HOOKS, so those steps are going to be >>>> executed in serial observing a specific order. >>>> >>>> What is the more appropriate solution? The easiest and fastest one is >>>> the first, but i'm not sure if those targets can be executed in >>>> parallel. >>> >>> My personal preference is to have a single rule (e.g. target-finalize) >>> that performs everything that is post-targets and pre-rootfs. There isn't >>> much that needs to be done so parallelisation doesn't make sense. And I >>> think it's much easier to understand which steps are executed and in >>> which order if they are all put together in a single rule rather than >>> spread out over several. >>> >>> To make things more readable, we can put the commands into separate >>> variables. For instance: >>> >>> define TARGET_PURGE_LOCALES >>> rm -f $(LOCALE_WHITELIST) >>> ... >>> endef >>> >>> define TARGET_PURGE_DEVFILES >>> rm -rf $(TARGET_DIR)/usr/include ... >>> ... >>> endef >>> >>> ifneq ($(BR2_PACKAGE_GDB),y) >>> define TARGET_PURGE_GDB >>> rm -rf $(TARGET_DIR)/usr/share/gdb >>> endef >>> endif >>> >>> target-finalize: $(TARGETS) >>> $(TARGET_PURGE_LOCALES) >>> $(TARGET_PURGE_DEVFILES) >>> $(TARGET_PURGE_GDB) >>> $(TARGET_PURGE_DOC) >>> ... >>> >>> >>> I'm giving the content of target-finalize as an example here, but it's >>> not immediately needed to convert those into variables. The different >>> target-* steps that are currently added to TARGETS are rather more important. >> >> It's fine for me. >> So you prefer to list those variables inside the target instead of >> adding them to a hook variable? like e.g. PRE_ROOTFS_HOOK? > > Indeed. The HOOKS way of working is not very clear. The code for calling > the hooks is not easy to understand at first sight, because it contains > this $(foreach ...) and $(sep) stuff. Also, the assignments to the HOOKS > variable are spread out over the Makefiles so it's hard to see which > things will get executed and in which order. And finally, it's not needed > here because we can enumerate all the possible steps - that's different > from the package infrastructure, where the HOOK contents differs from > package to package. Ok fine, i will do like that. Thanks and regards
Peter, Arnout, On Mon, 28 Apr 2014 11:36:15 +0200, Peter Korsgaard wrote: > > My personal preference is to have a single rule (e.g. target-finalize) > > that performs everything that is post-targets and pre-rootfs. There isn't > > much that needs to be done so parallelisation doesn't make sense. And I > > think it's much easier to understand which steps are executed and in > > which order if they are all put together in a single rule rather than > > spread out over several. > > > To make things more readable, we can put the commands into separate > > variables. For instance: > > > define TARGET_PURGE_LOCALES > > rm -f $(LOCALE_WHITELIST) > > ... > > endef > > > define TARGET_PURGE_DEVFILES > > rm -rf $(TARGET_DIR)/usr/include ... > > ... > > endef > > > ifneq ($(BR2_PACKAGE_GDB),y) > > define TARGET_PURGE_GDB > > rm -rf $(TARGET_DIR)/usr/share/gdb > > endef > > endif > > > target-finalize: $(TARGETS) > > $(TARGET_PURGE_LOCALES) > > $(TARGET_PURGE_DEVFILES) > > $(TARGET_PURGE_GDB) > > $(TARGET_PURGE_DOC) > > ... > > Yes, that looks nice and clear to me too. Yes, agreed, this looks a lot nicer than a long chain of targets that simply depend on each other to avoid any parallelization. Thanks! Thomas
On Mon, Apr 28, 2014 at 6:10 PM, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote: > Peter, Arnout, > > On Mon, 28 Apr 2014 11:36:15 +0200, Peter Korsgaard wrote: > >> > My personal preference is to have a single rule (e.g. target-finalize) >> > that performs everything that is post-targets and pre-rootfs. There isn't >> > much that needs to be done so parallelisation doesn't make sense. And I >> > think it's much easier to understand which steps are executed and in >> > which order if they are all put together in a single rule rather than >> > spread out over several. >> >> > To make things more readable, we can put the commands into separate >> > variables. For instance: >> >> > define TARGET_PURGE_LOCALES >> > rm -f $(LOCALE_WHITELIST) >> > ... >> > endef >> >> > define TARGET_PURGE_DEVFILES >> > rm -rf $(TARGET_DIR)/usr/include ... >> > ... >> > endef >> >> > ifneq ($(BR2_PACKAGE_GDB),y) >> > define TARGET_PURGE_GDB >> > rm -rf $(TARGET_DIR)/usr/share/gdb >> > endef >> > endif >> >> > target-finalize: $(TARGETS) >> > $(TARGET_PURGE_LOCALES) >> > $(TARGET_PURGE_DEVFILES) >> > $(TARGET_PURGE_GDB) >> > $(TARGET_PURGE_DOC) >> > ... >> >> Yes, that looks nice and clear to me too. > > Yes, agreed, this looks a lot nicer than a long chain of targets that > simply depend on each other to avoid any parallelization. Just to let you know, I've sent a patch following the above idea: http://patchwork.ozlabs.org/patch/357792/ Best regards
diff --git a/Makefile b/Makefile index eff408b..6b24482 100644 --- a/Makefile +++ b/Makefile @@ -579,7 +579,7 @@ ifeq ($(BR2_ENABLE_LOCALE_PURGE),y) LOCALE_WHITELIST = $(BUILD_DIR)/locales.nopurge LOCALE_NOPURGE = $(call qstrip,$(BR2_ENABLE_LOCALE_WHITELIST)) -target-purgelocales: +target-purgelocales: $(filter-out target-purgelocales,$(TARGETS)) rm -f $(LOCALE_WHITELIST) for i in $(LOCALE_NOPURGE); do echo $$i >> $(LOCALE_WHITELIST); done
The "target-purgelocales" target must be executed after all the other targets so add dependencies to ensure that. This also fix the support to top-level parallel make. Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> --- When both BR2_ENABLE_LOCALE_PURGE BR2_GENERATE_LOCALE are defined it swap the execution order of target-purgelocales and target-generatelocales, but as far as i can understand it does not arm to do that. Regards Fabio Porcedda Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)