| Message ID | 20200422185708.6363-1-robh@kernel.org |
|---|---|
| State | Accepted, archived |
| Headers | show |
| Series | [v2] dt-bindings: Fix command line length limit calling dt-mk-schema | expand |
| Context | Check | Description |
|---|---|---|
| robh/checkpatch | warning | "total: 0 errors, 1 warnings, 26 lines checked" |
On Thu, Apr 23, 2020 at 3:57 AM Rob Herring <robh@kernel.org> wrote: > > As the number of schemas has increased, we're starting to hit the error > "execvp: /bin/sh: Argument list too long". This is due to passing all the > schema files on the command line to dt-mk-schema. It currently is only > with out of tree builds and is intermittent depending on the file path > lengths. > > Commit 2ba06cd8565b ("kbuild: Always validate DT binding examples") made > hitting this proplem more likely since the example validation now always > gets the full list of schemas. > > Fix this by passing the schema file list in a pipe and using xargs. We end > up doing the find twice, but the time is insignificant compared to the > dt-mk-schema time. > > Reported-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > Cc: Masahiro Yamada <masahiroy@kernel.org> > Signed-off-by: Rob Herring <robh@kernel.org> Reviewed-by: Masahiro Yamada <masahiroy@kernel.org> > --- > Documentation/devicetree/bindings/Makefile | 18 ++++++++++-------- > 1 file changed, 10 insertions(+), 8 deletions(-) > > diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile > index 87c76bdabfe6..7782d9985082 100644 > --- a/Documentation/devicetree/bindings/Makefile > +++ b/Documentation/devicetree/bindings/Makefile > @@ -14,16 +14,18 @@ $(obj)/%.example.dts: $(src)/%.yaml FORCE > # Use full schemas when checking %.example.dts > DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml > > +find_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \ > + -name 'processed-schema*' ! \ > + -name '*.example.dt.yaml' \) > + > quiet_cmd_mk_schema = SCHEMA $@ > - cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@ $(real-prereqs) > + cmd_mk_schema = rm -f $@ ; \ > + $(if $(DT_MK_SCHEMA_FLAGS), \ > + echo $(real-prereqs), \ > + $(find_cmd)) | \ > + xargs $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) >> $@ > > -DT_DOCS = $(addprefix $(src)/, \ > - $(shell \ > - cd $(srctree)/$(src) && \ > - find * \( -name '*.yaml' ! \ > - -name 'processed-schema*' ! \ > - -name '*.example.dt.yaml' \) \ > - )) > +DT_DOCS = $(shell $(find_cmd) | sed -e 's|^$(srctree)/||') > > DT_SCHEMA_FILES ?= $(DT_DOCS) > > -- > 2.20.1 >
Hi Rob, Thank you for the patch. On Wed, Apr 22, 2020 at 01:57:08PM -0500, Rob Herring wrote: > As the number of schemas has increased, we're starting to hit the error > "execvp: /bin/sh: Argument list too long". This is due to passing all the > schema files on the command line to dt-mk-schema. It currently is only > with out of tree builds and is intermittent depending on the file path > lengths. > > Commit 2ba06cd8565b ("kbuild: Always validate DT binding examples") made > hitting this proplem more likely since the example validation now always > gets the full list of schemas. > > Fix this by passing the schema file list in a pipe and using xargs. We end > up doing the find twice, but the time is insignificant compared to the > dt-mk-schema time. > > Reported-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > Cc: Masahiro Yamada <masahiroy@kernel.org> > Signed-off-by: Rob Herring <robh@kernel.org> Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > --- > Documentation/devicetree/bindings/Makefile | 18 ++++++++++-------- > 1 file changed, 10 insertions(+), 8 deletions(-) > > diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile > index 87c76bdabfe6..7782d9985082 100644 > --- a/Documentation/devicetree/bindings/Makefile > +++ b/Documentation/devicetree/bindings/Makefile > @@ -14,16 +14,18 @@ $(obj)/%.example.dts: $(src)/%.yaml FORCE > # Use full schemas when checking %.example.dts > DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml > > +find_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \ > + -name 'processed-schema*' ! \ > + -name '*.example.dt.yaml' \) > + > quiet_cmd_mk_schema = SCHEMA $@ > - cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@ $(real-prereqs) > + cmd_mk_schema = rm -f $@ ; \ > + $(if $(DT_MK_SCHEMA_FLAGS), \ > + echo $(real-prereqs), \ > + $(find_cmd)) | \ > + xargs $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) >> $@ > > -DT_DOCS = $(addprefix $(src)/, \ > - $(shell \ > - cd $(srctree)/$(src) && \ > - find * \( -name '*.yaml' ! \ > - -name 'processed-schema*' ! \ > - -name '*.example.dt.yaml' \) \ > - )) > +DT_DOCS = $(shell $(find_cmd) | sed -e 's|^$(srctree)/||') > > DT_SCHEMA_FILES ?= $(DT_DOCS) >
diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile index 87c76bdabfe6..7782d9985082 100644 --- a/Documentation/devicetree/bindings/Makefile +++ b/Documentation/devicetree/bindings/Makefile @@ -14,16 +14,18 @@ $(obj)/%.example.dts: $(src)/%.yaml FORCE # Use full schemas when checking %.example.dts DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml +find_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \ + -name 'processed-schema*' ! \ + -name '*.example.dt.yaml' \) + quiet_cmd_mk_schema = SCHEMA $@ - cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@ $(real-prereqs) + cmd_mk_schema = rm -f $@ ; \ + $(if $(DT_MK_SCHEMA_FLAGS), \ + echo $(real-prereqs), \ + $(find_cmd)) | \ + xargs $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) >> $@ -DT_DOCS = $(addprefix $(src)/, \ - $(shell \ - cd $(srctree)/$(src) && \ - find * \( -name '*.yaml' ! \ - -name 'processed-schema*' ! \ - -name '*.example.dt.yaml' \) \ - )) +DT_DOCS = $(shell $(find_cmd) | sed -e 's|^$(srctree)/||') DT_SCHEMA_FILES ?= $(DT_DOCS)
As the number of schemas has increased, we're starting to hit the error "execvp: /bin/sh: Argument list too long". This is due to passing all the schema files on the command line to dt-mk-schema. It currently is only with out of tree builds and is intermittent depending on the file path lengths. Commit 2ba06cd8565b ("kbuild: Always validate DT binding examples") made hitting this proplem more likely since the example validation now always gets the full list of schemas. Fix this by passing the schema file list in a pipe and using xargs. We end up doing the find twice, but the time is insignificant compared to the dt-mk-schema time. Reported-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Cc: Masahiro Yamada <masahiroy@kernel.org> Signed-off-by: Rob Herring <robh@kernel.org> --- Documentation/devicetree/bindings/Makefile | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)