[06/11,RFC] support/scripts: introduce a symbol formatter to generate package lists

From: "Yann E. MORIN" <yann.morin.1998@free.fr>

Currently, we can generate two different tables of packages:
  - a single-column table with the symbols' prompts,
  - a two-column table with the sumbols' prompts and locations in the
    menuconfig.

For virtual packages, this is not enough, since we will have to display
more columns, with different content:
  - the virtual package name (but such symbols do not have a prompt)
  - the symbol name
  - the providers for thet virtual package

So, instead of having a single function that knows how to generate any
table, introduce a formatter function that is passed as argument to,
and called by format_asciidoc_table(). Such formatter functions are
responsible for providing:
  - the layout of the table (number of columns, columns arangement),
  - the formatted header line,
  - a formatted line for a symbol.

What the formatter should ouput depends on its arguments:
  - if none are passed, the layout is returned,
  - if no symbol is passed, but the header label is, it returns
    the header line,
  - otherwise, it returns the formatted line for a symbol.

Two formatter functions are introduced in this changeset, to replace the
current 'sub_menu' feature:
  - _format_symbol_prompt() to display a one-column table with only the
    symbols' prompts,
  - _format_symbol_prompt_location() to display a two-column table with
    the symbols' prompts and locations.

This will help us to later introduce a new formatter to generate a table
for virtual packages.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
---
 support/scripts/gen-manual-lists.py | 72 ++++++++++++++++++++++---------------
 1 file changed, 43 insertions(+), 29 deletions(-)

Yann, all,

On Thu, May 29, 2014 at 10:38 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
>
> Currently, we can generate two different tables of packages:
>   - a single-column table with the symbols' prompts,
>   - a two-column table with the sumbols' prompts and locations in the

s/sumbols/symbols/

>     menuconfig.
>
> For virtual packages, this is not enough, since we will have to display
> more columns, with different content:
>   - the virtual package name (but such symbols do not have a prompt)
>   - the symbol name
>   - the providers for thet virtual package

s/thet/that/

>
> So, instead of having a single function that knows how to generate any
> table, introduce a formatter function that is passed as argument to,
> and called by format_asciidoc_table(). Such formatter functions are
> responsible for providing:
>   - the layout of the table (number of columns, columns arangement),

s/columns arangement/column arrangement/

>   - the formatted header line,
>   - a formatted line for a symbol.
>
> What the formatter should ouput depends on its arguments:
>   - if none are passed, the layout is returned,
>   - if no symbol is passed, but the header label is, it returns
>     the header line,
>   - otherwise, it returns the formatted line for a symbol.

The more I think about this, the more I feel it fragile (i.e. using 2
variables for a tristate) :-/
In the end, the header argument and the symbol one are just some data
processed by the formatter.
Maybe we should just replace the header and symbol kwargs by a data
one, and add another argument telling what should return the
formatter: either the layout, the header line or a symbol line.

>
> Two formatter functions are introduced in this changeset, to replace the
> current 'sub_menu' feature:
>   - _format_symbol_prompt() to display a one-column table with only the
>     symbols' prompts,
>   - _format_symbol_prompt_location() to display a two-column table with
>     the symbols' prompts and locations.
>
> This will help us to later introduce a new formatter to generate a table
> for virtual packages.
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Samuel Martin <s.martin49@gmail.com>
> ---
>  support/scripts/gen-manual-lists.py | 72 ++++++++++++++++++++++---------------
>  1 file changed, 43 insertions(+), 29 deletions(-)
>
> diff --git a/support/scripts/gen-manual-lists.py b/support/scripts/gen-manual-lists.py
> index 7144ca3..8311929 100644
> --- a/support/scripts/gen-manual-lists.py
> +++ b/support/scripts/gen-manual-lists.py
> @@ -109,50 +109,33 @@ def get_symbol_parents(item, root=None, enable_choice=False):
>
>
>  def format_asciidoc_table(root, get_label_func, filter_func=lambda x: True,
> -                          enable_choice=False, sorted=True, sub_menu=True,
> +                          format_func=lambda x: "",

usually for default lambda function, I set: lambda x: x

> +                          enable_choice=False, sorted=True,
>                            item_label=None):
>      """ Return the asciidoc formatted table of the items and their location.
>
>      :param root:           Root item of the item subset
>      :param get_label_func: Item's label getter function
>      :param filter_func:    Filter function to apply on the item subset
> +    :param format_func:    Function to format a symbol and the table header
>      :param enable_choice:  Enable choices to appear as part of the item's
>                             location
>      :param sorted:         Flag to alphabetically sort the table
> -    :param sub_menu:       Output the column with the sub-menu path
>
>      """
> -    def _format_entry(item, parents, sub_menu):
> -        """ Format an asciidoc table entry.
>
> -        """
> -        if sub_menu:
> -            return "| {0:<40} <| {1}\n".format(item, " -> ".join(parents))
> -        else:
> -            return "| {0:<40}\n".format(item)
>      lines = []
>      for item in get_symbol_subset(root, filter_func):
> -        loc = get_symbol_parents(item, root, enable_choice=enable_choice)
> -        lines.append(_format_entry(get_label_func(item), loc, sub_menu))
> +        lines.append(format_func(symbol=item, root=root,
> +                                 get_label_func=get_label_func,
> +                                 enable_choice=enable_choice))
>      if sorted:
>          lines.sort(key=lambda x: x.lower())
> -    if hasattr(root, "get_title"):
> -        loc_label = get_symbol_parents(root, None, enable_choice=enable_choice)
> -        loc_label += [root.get_title(), "..."]
> -    else:
> -        loc_label = ["Location"]
> -    if not item_label:
> -        item_label = "Items"
>      table = ":halign: center\n\n"
> -    if sub_menu:
> -        width = "100%"
> -        columns = "^1,4"
> -    else:
> -        width = "30%"
> -        columns = "^1"
> +    width, columns = format_func()
>      table = "[width=\"{0}\",cols=\"{1}\",options=\"header\"]\n".format(width, columns)
>      table += "|===================================================\n"
> -    table += _format_entry(item_label, loc_label, sub_menu)
> +    table += format_func(header=item_label, root=root)
>      table += "\n" + "".join(lines) + "\n"
>      table += "|===================================================\n"
>      return table
> @@ -180,22 +163,22 @@ class Buildroot:
>              'filename': "package-list",
>              'root_menu': "Target packages",
>              'filter': "_is_real_package",
> +            'format': '_format_symbol_prompt_location',

For dictionary, usually, single-quotes are used for the key and
double-quotes for the value (when it's a string), though both single
and double quote can be used in both cases (key and value).
Here my point is more about consistency with the rest of the file.

>              'sorted': True,
> -            'sub_menu': True,
>          },
>          'host-packages': {
>              'filename': "host-package-list",
>              'root_menu': "Host utilities",
>              'filter': "_is_real_package",
> +            'format': '_format_symbol_prompt',

ditto

>              'sorted': True,
> -            'sub_menu': False,
>          },
>          'deprecated': {
>              'filename': "deprecated-list",
>              'root_menu': None,
>              'filter': "_is_deprecated",
> +            'format': '_format_symbol_prompt_location',

ditto

>              'sorted': False,
> -            'sub_menu': True,
>          },
>      }
>
> @@ -323,6 +306,36 @@ class Buildroot:
>              label += " *(deprecated)*"
>          return label
>
> +    def _format_symbol_prompt(self, symbol=None, root=None,
> +                                    enable_choice=False, header=None,
> +                                    get_label_func=lambda x: "?"):

default lambda: s/"?"/x/

> +        if symbol is None and header is None:
> +            return ( "30%", "^1" )
> +
> +        if header is not None:
> +            return "| {0:<40}\n".format(header)
> +
> +        return "| {0:<40}\n".format(get_label_func(symbol))
> +
> +    def _format_symbol_prompt_location(self, symbol=None, root=None,
> +                                             enable_choice=False, header=None,
> +                                             get_label_func=lambda x: "?"):

ditto

> +        if symbol is None and header is None:
> +            return ( "100%", "^1,4" )
> +
> +        if header is not None:
> +            if hasattr(root, "get_title"):
> +                loc_label = get_symbol_parents(root, None, enable_choice=enable_choice)
> +                loc_label += [root.get_title(), "..."]
> +            else:
> +                loc_label = ["Location"]
> +
> +            return "| {0:<40} <| {1}\n".format(header, " -> ".join(loc_label))
> +
> +        parents = get_symbol_parents(symbol, root, enable_choice)
> +        return "| {0:<40} <| {1}\n".format(get_label_func(symbol),
> +                                           " -> ".join(parents))
> +
>      def print_list(self, list_type, enable_choice=True, enable_deprecated=True,
>                     dry_run=False, output=None):
>          """ Print the requested list. If not dry run, then the list is
> @@ -355,6 +368,7 @@ class Buildroot:
>              root_item = self.config
>          filter_ = getattr(self, list_config.get('filter'))
>          filter_func = lambda x: filter_(x)
> +        format_func = getattr(self, list_config.get('format'))
>          if not enable_deprecated and list_type != "deprecated":
>              filter_func = lambda x: filter_(x) and not self._is_deprecated(x)
>          mark_depr = list_type != "deprecated"
> @@ -363,9 +377,9 @@ class Buildroot:
>
>          table = format_asciidoc_table(root_item, get_label,
>                                        filter_func=filter_func,
> +                                      format_func=format_func,
>                                        enable_choice=enable_choice,
>                                        sorted=list_config.get('sorted'),
> -                                      sub_menu=list_config.get('sub_menu'),
>                                        item_label=item_label)
>
>          content = self.list_in.format(table=table)
> --
> 1.8.3.2
>

Regards,

Samuel, All,

On 2014-05-31 23:20 +0200, Samuel Martin spake thusly:
> On Thu, May 29, 2014 at 10:38 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
[--SNIP--]

Typoes fixed.

> > What the formatter should ouput depends on its arguments:
> >   - if none are passed, the layout is returned,
> >   - if no symbol is passed, but the header label is, it returns
> >     the header line,
> >   - otherwise, it returns the formatted line for a symbol.
> 
> The more I think about this, the more I feel it fragile (i.e. using 2
> variables for a tristate) :-/
> In the end, the header argument and the symbol one are just some data
> processed by the formatter.
> Maybe we should just replace the header and symbol kwargs by a data
> one, and add another argument telling what should return the
> formatter: either the layout, the header line or a symbol line.

OK, I see what you mean. Something like the following?

    format_symbol_prompt(self, what=None, symbol=None, header=None,
                         root=None, enable_choice=False, header=None,
                         get_label_func=lambda x: "?"):
        if what is None:
            throw an exception
        if what == "layout":
            return ("bla","bla")
        if what == "header":
            return formated_header(header)
        if what == "symbol":
            return formated_symbol(symbol)
        throw an exception

However, I don;t get what you mean by "a data argument". Do you mean a
kind of casting the arg whether we're asked to render a header or a
symbol? How does one casts in Python?

Oh wait, duck-typing, right?

[--SNIP--]
> > diff --git a/support/scripts/gen-manual-lists.py b/support/scripts/gen-manual-lists.py
> > index 7144ca3..8311929 100644
> > --- a/support/scripts/gen-manual-lists.py
> > +++ b/support/scripts/gen-manual-lists.py
> > @@ -109,50 +109,33 @@ def get_symbol_parents(item, root=None, enable_choice=False):
> >
> >
> >  def format_asciidoc_table(root, get_label_func, filter_func=lambda x: True,
> > -                          enable_choice=False, sorted=True, sub_menu=True,
> > +                          format_func=lambda x: "",
> 
> usually for default lambda function, I set: lambda x: x

But I do not want it to return anything, I want it to return a string.

Or do you expect that an exception would be thrown out in case an object
is used that is not a string later in the call chain? I don't like that
too much...

Alternatively, as Maxime said on IRC, I coiuld just set it to None, and
get an exception if it is no passed by the caller.

Except (aha!) setting it to a lambda makes it explicit it has to be a
function.

So, OK, I'll use your default lambda. At least, we'd get an exception if
someone forgets to pass the function.

> > @@ -180,22 +163,22 @@ class Buildroot:
> >              'filename': "package-list",
> >              'root_menu': "Target packages",
> >              'filter': "_is_real_package",
> > +            'format': '_format_symbol_prompt_location',
> 
> For dictionary, usually, single-quotes are used for the key and
> double-quotes for the value (when it's a string), though both single
> and double quote can be used in both cases (key and value).
> Here my point is more about consistency with the rest of the file.

Yep, fixed the three.

Regards,
Yann E. MORIN.

Yann,

On Sun, Jun 1, 2014 at 1:09 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> Samuel, All,
>
> On 2014-05-31 23:20 +0200, Samuel Martin spake thusly:
>> On Thu, May 29, 2014 at 10:38 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> [--SNIP--]
>
> Typoes fixed.
>
>> > What the formatter should ouput depends on its arguments:
>> >   - if none are passed, the layout is returned,
>> >   - if no symbol is passed, but the header label is, it returns
>> >     the header line,
>> >   - otherwise, it returns the formatted line for a symbol.
>>
>> The more I think about this, the more I feel it fragile (i.e. using 2
>> variables for a tristate) :-/
>> In the end, the header argument and the symbol one are just some data
>> processed by the formatter.
>> Maybe we should just replace the header and symbol kwargs by a data
>> one, and add another argument telling what should return the
>> formatter: either the layout, the header line or a symbol line.
>
> OK, I see what you mean. Something like the following?
>
>     format_symbol_prompt(self, what=None, symbol=None, header=None,
>                          root=None, enable_choice=False, header=None,
>                          get_label_func=lambda x: "?"):
>         if what is None:
>             throw an exception
>         if what == "layout":
>             return ("bla","bla")
>         if what == "header":
>             return formated_header(header)
>         if what == "symbol":
>             return formated_symbol(symbol)
>         throw an exception
>
> However, I don;t get what you mean by "a data argument". Do you mean a
> kind of casting the arg whether we're asked to render a header or a
> symbol? How does one casts in Python?

By "a data argument" I mean:

    format_symbol_prompt(self, what=None, data=None,
                         root=None, enable_choice=False,
                         get_label_func=lambda x: "?"):
        if what is None:
            throw an exception
        if what == "layout":
            return ("bla","bla")
        if what == "header":
            return formated_header(data)
        if what == "symbol":
            return formated_symbol(data)
        raise Exception("Unsupported argument 'what': %s" % what)

For the cast, python does magics with its PyObjects.
If you want to be sure to return a string from any object, you can do:
return str(data)

>
> Oh wait, duck-typing, right?
>
> [--SNIP--]
>> > diff --git a/support/scripts/gen-manual-lists.py b/support/scripts/gen-manual-lists.py
>> > index 7144ca3..8311929 100644
>> > --- a/support/scripts/gen-manual-lists.py
>> > +++ b/support/scripts/gen-manual-lists.py
>> > @@ -109,50 +109,33 @@ def get_symbol_parents(item, root=None, enable_choice=False):
>> >
>> >
>> >  def format_asciidoc_table(root, get_label_func, filter_func=lambda x: True,
>> > -                          enable_choice=False, sorted=True, sub_menu=True,
>> > +                          format_func=lambda x: "",
>>
>> usually for default lambda function, I set: lambda x: x
>
> But I do not want it to return anything, I want it to return a string.

See above ;-)

>
> Or do you expect that an exception would be thrown out in case an object
> is used that is not a string later in the call chain? I don't like that
> too much...
>
> Alternatively, as Maxime said on IRC, I coiuld just set it to None, and
> get an exception if it is no passed by the caller.
>
> Except (aha!) setting it to a lambda makes it explicit it has to be a
> function.
>
> So, OK, I'll use your default lambda. At least, we'd get an exception if
> someone forgets to pass the function.
>
>> > @@ -180,22 +163,22 @@ class Buildroot:
>> >              'filename': "package-list",
>> >              'root_menu': "Target packages",
>> >              'filter': "_is_real_package",
>> > +            'format': '_format_symbol_prompt_location',
>>
>> For dictionary, usually, single-quotes are used for the key and
>> double-quotes for the value (when it's a string), though both single
>> and double quote can be used in both cases (key and value).
>> Here my point is more about consistency with the rest of the file.
>
> Yep, fixed the three.
>
> Regards,
> Yann E. MORIN.
>
> --
> .-----------------.--------------------.------------------.--------------------.
> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
> | +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
> '------------------------------^-------^------------------^--------------------'

Regards,