[16/16,v2] Makefile: introduce show-vars, a json-formatted equivalent to printvars

The current printvars output suffers from a serious design flaw:
variables are not delimited, which makes it impossible to reliably
retrieve the value of variables; only variables that are known to
not contain a \n can be relatively safely extracted.

However, in some cases, it is important to be able to retrieve the
multi-line value of a variable, notably the CMDS or the hooks. One
such use-case (to follow in an unscheduled future) would be to hash
the variables that make up a package "configuration", and cache or
extract the files for that package to speed up the build.

Modeled after printvars and show-info, we introduce show-vars (what a
lack of imagination here) that outputs a json dictionary which keys are
the variable names, and for each variable, provides the raw and expanded
values.

Unlike printvars, we do not provide a way to get either the raw or
expanded value; both are systematically printed; a user will get just
the one is needs. Additionally, strings in JSON are quoted, so there is
no need to provide a way to quote variables; that would not make sense.

Note: for printvars, we require that the user provides an explicit
pattern to filter variables on. This is historical (see fd5bd12379dc,
Makefile: printvars: don't print anything when VARS is not set). The
underlying reasoning was that printvars is too "raw", and variables are
not well delimited, so printvars was mostly used to extract a few values
here and there, from scripts, or to quickly inspect a specific package's
variables during debugging.

But show-vars, although technically plain-text, being JSON, is not very
human-readable, and is mostly aimed at tools that will parse it with a
real JSON parser, and which will want to have a complete view of a lot
of variables at once. As such, and contrary to printvars, it makes sense
to report on all variables by default, unless the user explicitly
requested a subset.

As a final note: a lot of our variables only make sense in the context
of an actual make target. For example, a variable of package foo, that
contains $(@D)/bar, would expand to .../build/FOO-VERSION/bar. This is
because our CMDS and hooks are expanded as the recipe of a stamp file
that lies in the package build directory.

But for show-info, this falls flat on its face: it is not the stamp file
of a package, so there is no package directory, and show-info itself has
not directory part, so $(@D) expands to '.' (dot).

Additionally, some variables may contain calls to $(shell) (e.g. to call
pkg-config), and this also does not work with show-info.

These two issues make it impossible to emit the correct expanded value
of variables. To be noted: printvars has the exact same limitations for
the exact same reasons.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
---
 Makefile | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

On Sat, 13 Nov 2021 14:28:27 +0100
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> The current printvars output suffers from a serious design flaw:
> variables are not delimited, which makes it impossible to reliably
> retrieve the value of variables; only variables that are known to
> not contain a \n can be relatively safely extracted.
> 
> However, in some cases, it is important to be able to retrieve the
> multi-line value of a variable, notably the CMDS or the hooks. One
> such use-case (to follow in an unscheduled future) would be to hash
> the variables that make up a package "configuration", and cache or
> extract the files for that package to speed up the build.
> 
> Modeled after printvars and show-info, we introduce show-vars (what a
> lack of imagination here) that outputs a json dictionary which keys are
> the variable names, and for each variable, provides the raw and expanded
> values.
> 
> Unlike printvars, we do not provide a way to get either the raw or
> expanded value; both are systematically printed; a user will get just
> the one is needs. Additionally, strings in JSON are quoted, so there is
> no need to provide a way to quote variables; that would not make sense.
> 
> Note: for printvars, we require that the user provides an explicit
> pattern to filter variables on. This is historical (see fd5bd12379dc,
> Makefile: printvars: don't print anything when VARS is not set). The
> underlying reasoning was that printvars is too "raw", and variables are
> not well delimited, so printvars was mostly used to extract a few values
> here and there, from scripts, or to quickly inspect a specific package's
> variables during debugging.
> 
> But show-vars, although technically plain-text, being JSON, is not very
> human-readable, and is mostly aimed at tools that will parse it with a
> real JSON parser, and which will want to have a complete view of a lot
> of variables at once. As such, and contrary to printvars, it makes sense
> to report on all variables by default, unless the user explicitly
> requested a subset.
> 
> As a final note: a lot of our variables only make sense in the context
> of an actual make target. For example, a variable of package foo, that
> contains $(@D)/bar, would expand to .../build/FOO-VERSION/bar. This is
> because our CMDS and hooks are expanded as the recipe of a stamp file
> that lies in the package build directory.
> 
> But for show-info, this falls flat on its face: it is not the stamp file
> of a package, so there is no package directory, and show-info itself has
> not directory part, so $(@D) expands to '.' (dot).
> 
> Additionally, some variables may contain calls to $(shell) (e.g. to call
> pkg-config), and this also does not work with show-info.
> 
> These two issues make it impossible to emit the correct expanded value
> of variables. To be noted: printvars has the exact same limitations for
> the exact same reasons.
> 
> Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
> ---
>  Makefile | 21 ++++++++++++++++++++-
>  1 file changed, 20 insertions(+), 1 deletion(-)

Applied to master, thanks.

However, there are two things that are missing:

 (1) Update to the Buildroot manual

 (2) At least one test case in supporting/testing/

Thanks!

Thomas