Message ID | 20240219081431.2887060-4-amorenoz@redhat.com |
---|---|
State | Changes Requested |
Delegated to: | Simon Horman |
Headers | show |
Series | Add flow visualization utility. | expand |
Context | Check | Description |
---|---|---|
ovsrobot/apply-robot | success | apply and check: success |
ovsrobot/github-robot-_Build_and_Test | success | github build: passed |
ovsrobot/intel-ovs-compilation | success | test: success |
On 19 Feb 2024, at 9:14, Adrian Moreno wrote: > Add a flow formatting framework and one implementation for console > printing using rich. > > The flow formatting framework is a simple set of classes that can be > used to write different flow formatting implementations. It supports > styles to be described by any class, highlighting and config-file based > style definition. > > The first flow formatting implementation is also introduced: the > ConsoleFormatter. It uses the an advanced rich-text printing library > [1]. > > The console printing supports: > - Heatmap: printing the packet/byte statistics of each flow in a color > that represents its relative size: blue (low) -> red (high). > - Printing a banner with the file name and alias. > - Extensive style definition via config file. > > This console format is added to both OpenFlow and Datapath flows. > > Examples: > - Highlight drops in datapath flows: > $ ovs-flowviz -i flows.txt --highlight "drop" datapath console > - Quickly detect where most packets are going using heatmap and > paginated output: > $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow console -h > > [1] https://rich.readthedocs.io/en/stable/introduction.html > > Signed-off-by: Adrian Moreno <amorenoz@redhat.com> > --- Two small nits below, the rest look good to me. //Eelco > python/automake.mk | 2 + > python/ovs/flowviz/console.py | 175 ++++++++++++++++ > python/ovs/flowviz/format.py | 371 ++++++++++++++++++++++++++++++++++ > python/ovs/flowviz/main.py | 58 +++++- > python/ovs/flowviz/odp/cli.py | 25 +++ > python/ovs/flowviz/ofp/cli.py | 26 +++ > python/ovs/flowviz/process.py | 83 +++++++- > python/setup.py | 4 +- > 8 files changed, 736 insertions(+), 8 deletions(-) > create mode 100644 python/ovs/flowviz/console.py > create mode 100644 python/ovs/flowviz/format.py > > diff --git a/python/automake.mk b/python/automake.mk > index fd5e74081..bd53c5405 100644 > --- a/python/automake.mk > +++ b/python/automake.mk > @@ -65,6 +65,8 @@ ovs_pytests = \ > > ovs_flowviz = \ > python/ovs/flowviz/__init__.py \ > + python/ovs/flowviz/console.py \ > + python/ovs/flowviz/format.py \ > python/ovs/flowviz/main.py \ > python/ovs/flowviz/odp/__init__.py \ > python/ovs/flowviz/odp/cli.py \ > diff --git a/python/ovs/flowviz/console.py b/python/ovs/flowviz/console.py > new file mode 100644 > index 000000000..4a3443360 > --- /dev/null > +++ b/python/ovs/flowviz/console.py > @@ -0,0 +1,175 @@ > +# Copyright (c) 2023 Red Hat, Inc. > +# > +# Licensed under the Apache License, Version 2.0 (the "License"); > +# you may not use this file except in compliance with the License. > +# You may obtain a copy of the License at: > +# > +# http://www.apache.org/licenses/LICENSE-2.0 > +# > +# Unless required by applicable law or agreed to in writing, software > +# distributed under the License is distributed on an "AS IS" BASIS, > +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. > +# See the License for the specific language governing permissions and > +# limitations under the License. > + > +import colorsys > + > +from rich.console import Console > +from rich.color import Color > +from rich.emoji import Emoji > +from rich.panel import Panel > +from rich.text import Text > +from rich.style import Style > + > +from ovs.flowviz.format import FlowFormatter, FlowBuffer, FlowStyle > + > + > +def file_header(name): > + return Panel( > + Text( > + Emoji.replace(":scroll:") > + + " " > + + name > + + " " > + + Emoji.replace(":scroll:"), > + style="bold", > + justify="center", > + ) > + ) > + > + > +class ConsoleBuffer(FlowBuffer): > + """ConsoleBuffer implements FlowBuffer to provide console-based text > + formatting based on rich.Text. > + > + Append functions accept a rich.Style. > + > + Args: > + rtext(rich.Text): Optional; text instance to reuse > + """ > + > + def __init__(self, rtext): > + self._text = rtext or Text() > + > + @property > + def text(self): > + return self._text > + > + def _append(self, string, style): > + """Append to internal text.""" > + return self._text.append(string, style) > + > + def append_key(self, kv, style): > + """Append a key. > + Args: > + kv (KeyValue): the KeyValue instance to append > + style (rich.Style): the style to use > + """ > + return self._append(kv.meta.kstring, style) > + > + def append_delim(self, kv, style): > + """Append a delimiter. > + Args: > + kv (KeyValue): the KeyValue instance to append > + style (rich.Style): the style to use > + """ > + return self._append(kv.meta.delim, style) > + > + def append_end_delim(self, kv, style): > + """Append an end delimiter. > + Args: > + kv (KeyValue): the KeyValue instance to append > + style (rich.Style): the style to use > + """ > + return self._append(kv.meta.end_delim, style) > + > + def append_value(self, kv, style): > + """Append a value. > + Args: > + kv (KeyValue): the KeyValue instance to append > + style (rich.Style): the style to use > + """ > + return self._append(kv.meta.vstring, style) > + > + def append_extra(self, extra, style): > + """Append extra string. > + Args: > + kv (KeyValue): the KeyValue instance to append > + style (rich.Style): the style to use > + """ > + return self._append(extra, style) > + > + > +class ConsoleFormatter(FlowFormatter): > + """ConsoleFormatter is a FlowFormatter that formats flows into the console > + using rich.Console. > + > + Args: > + console (rich.Console): Optional, an existing console to use > + max_value_len (int): Optional; max length of the printed values > + kwargs (dict): Optional; Extra arguments to be passed down to > + rich.console.Console() > + """ > + > + def __init__(self, opts=None, console=None, **kwargs): > + super(ConsoleFormatter, self).__init__() > + style = self.style_from_opts(opts) > + self.console = console or Console(color_system="256", **kwargs) > + self.style = style or FlowStyle() > + > + def style_from_opts(self, opts): > + return self._style_from_opts(opts, "console", Style) > + > + def print_flow(self, flow, highlighted=None): > + """Prints a flow to the console. > + > + Args: > + flow (ovs_dbg.OFPFlow): the flow to print > + style (dict): Optional; style dictionary to use > + highlighted (list): Optional; list of KeyValues to highlight > + """ > + > + buf = ConsoleBuffer(Text()) > + self.format_flow(buf, flow, highlighted) > + self.console.print(buf.text) > + > + def format_flow(self, buf, flow, highlighted=None): > + """Formats the flow into the provided buffer as a rich.Text. > + > + Args: > + buf (FlowBuffer): the flow buffer to append to > + flow (ovs_dbg.OFPFlow): the flow to format > + style (FlowStyle): Optional; style object to use > + highlighted (list): Optional; list of KeyValues to highlight > + """ > + return super(ConsoleFormatter, self).format_flow( > + buf, flow, self.style, highlighted > + ) > + > + > +def heat_pallete(min_value, max_value): > + """Generates a color pallete based on the 5-color heat pallete so that > + for each value between min and max a color is returned that represents it's > + relative size. > + Args: > + min_value (int): minimum value > + max_value (int) maximum value > + """ > + h_min = 0 # red > + h_max = 220 / 360 # blue > + > + def heat(value): > + if max_value == min_value: > + r, g, b = colorsys.hsv_to_rgb(h_max / 2, 1.0, 1.0) > + else: > + normalized = (int(value) - min_value) / (max_value - min_value) > + hue = ((1 - normalized) + h_min) * (h_max - h_min) > + r, g, b = colorsys.hsv_to_rgb(hue, 1.0, 1.0) > + return Style(color=Color.from_rgb(r * 255, g * 255, b * 255)) > + > + return heat > + > + > +def default_highlight(): > + """Generates a default style for highlights.""" > + return Style(underline=True) > diff --git a/python/ovs/flowviz/format.py b/python/ovs/flowviz/format.py > new file mode 100644 > index 000000000..70af2fa26 > --- /dev/null > +++ b/python/ovs/flowviz/format.py > @@ -0,0 +1,371 @@ > +# Copyright (c) 2023 Red Hat, Inc. > +# > +# Licensed under the Apache License, Version 2.0 (the "License"); > +# you may not use this file except in compliance with the License. > +# You may obtain a copy of the License at: > +# > +# http://www.apache.org/licenses/LICENSE-2.0 > +# > +# Unless required by applicable law or agreed to in writing, software > +# distributed under the License is distributed on an "AS IS" BASIS, > +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. > +# See the License for the specific language governing permissions and > +# limitations under the License. > + > +"""Flow formatting framework. > + > +This file defines a simple flow formatting framework. It's comprised of 3 > +classes: FlowStyle, FlowFormatter and FlowBuffer. > + > +The FlowStyle arranges opaque style objects in a dictionary that can be queried > +to determine what style a particular key-value should be formatted with. > +That way, a particular implementation can represent its style using their own > +object. > + > +The FlowBuffer is an abstract class and must be derived by particular > +implementations. It should know how to append parts of a flow using a style. > +Only here the type of the style is relevant. > + > +When asked to format a flow, the FlowFormatter will determine which style > +the flow must be formatted with and call FlowBuffer functions with each part > +of the flow and their corresponding style. > +""" > + > + > +class FlowStyle: > + """A FlowStyle determines the KVStyle to use for each key value in a flow. > + > + Styles are internally represented by a dictionary. > + In order to determine the style for a "key", the following items in the > + dictionary are fetched: > + - key.highlighted.{key} (if key is found in hightlighted) > + - key.highlighted (if key is found in hightlighted) > + - key.{key} > + - key > + - default > + > + In order to determine the style for a "value", the following items in the > + dictionary are fetched: > + - value.highlighted.{key} (if key is found in hightlighted) > + - value.highlighted.type{value.__class__.__name__} > + - value.highlighted > + (if key is found in hightlighted) > + - value.{key} > + - value.type.{value.__class__.__name__} > + - value > + - default > + > + The actual type of the style object stored for each item above is opaque > + to this class and it depends on the particular FlowFormatter child class > + that will handle them. Even callables can be stored, if so they will be > + called with the value of the field that is to be formatted and the return > + object will be used as style. > + > + Additionally, the following style items can be defined: > + - delim: for delimiters > + - delim.highlighted: for delimiters of highlighted key-values > + """ > + > + def __init__(self, initial=None): > + self._styles = initial if initial is not None else dict() > + > + def __len__(self): > + return len(self._styles) > + > + def set_flag_style(self, kvstyle): > + self._styles["flag"] = kvstyle > + > + def set_delim_style(self, kvstyle, highlighted=False): > + if highlighted: > + self._styles["delim.highlighted"] = kvstyle > + else: > + self._styles["delim"] = kvstyle > + > + def set_default_key_style(self, kvstyle, highlighted=False): > + if highlighted: > + self._styles["key.highlighted"] = kvstyle > + else: > + self._styles["key"] = kvstyle > + > + def set_default_value_style(self, kvstyle, highlighted=False): > + if highlighted: > + self._styles["value.highlighted"] = kvstyle > + else: > + self._styles["value"] = kvstyle > + > + def set_key_style(self, key, kvstyle, highlighted=False): > + if highlighted: > + self._styles["key.highlighted.{}".format(key)] = kvstyle > + else: > + self._styles["key.{}".format(key)] = kvstyle > + > + def set_value_style(self, key, kvstyle, highlighted=None): > + if highlighted: > + self._styles["value.highlighted.{}".format(key)] = kvstyle > + else: > + self._styles["value.{}".format(key)] = kvstyle > + > + def set_value_type_style(self, name, kvstyle, highlighted=None): > + if highlighted: > + self._styles["value.highlighted.type.{}".format(name)] = kvstyle > + else: > + self._styles["value.type.{}".format(name)] = kvstyle > + > + def get(self, key): > + return self._styles.get(key) > + > + def get_delim_style(self, highlighted=False): > + delim_style_lookup = ["delim.highlighted"] if highlighted else [] > + delim_style_lookup.extend(["delim", "default"]) > + return next( > + ( > + self._styles.get(s) > + for s in delim_style_lookup > + if self._styles.get(s) > + ), > + None, > + ) > + > + def get_flag_style(self): > + return self._styles.get("flag") or self._styles.get("default") > + > + def get_key_style(self, kv, highlighted=False): > + key = kv.key > + > + key_style_lookup = ( > + ["key.highlighted.%s" % key, "key.highlighted"] > + if highlighted > + else [] > + ) > + key_style_lookup.extend(["key.%s" % key, "key", "default"]) > + > + style = next( > + ( > + self._styles.get(s) > + for s in key_style_lookup > + if self._styles.get(s) > + ), > + None, > + ) > + if callable(style): > + return style(kv.meta.kstring) > + return style > + > + def get_value_style(self, kv, highlighted=False): > + key = kv.key > + value_type = kv.value.__class__.__name__.lower() > + value_style_lookup = ( > + [ > + "value.highlighted.%s" % key, > + "value.highlighted.type.%s" % value_type, > + "value.highlighted", > + ] > + if highlighted > + else [] > + ) > + value_style_lookup.extend( > + [ > + "value.%s" % key, > + "value.type.%s" % value_type, > + "value", > + "default", > + ] > + ) > + > + style = next( > + ( > + self._styles.get(s) > + for s in value_style_lookup > + if self._styles.get(s) > + ), > + None, > + ) > + if callable(style): > + return style(kv.meta.vstring) > + return style > + > + > +class FlowFormatter: > + """FlowFormatter is a base class for Flow Formatters.""" > + > + def __init__(self): > + self._highlighted = list() > + > + def _style_from_opts(self, opts, opts_key, style_constructor): > + """Create style object from options. > + > + Args: > + opts (dict): Options dictionary > + opts_key (str): The options style key to extract > + (e.g: console or html) > + style_constructor(callable): A callable that creates a derived > + style object > + """ > + if not opts or not opts.get("style"): > + return None > + > + section_name = ".".join(["styles", opts.get("style")]) > + if section_name not in opts.get("config").sections(): > + return None > + > + config = opts.get("config")[section_name] > + style = {} > + for key in config: > + (_, console, style_full_key) = key.partition(opts_key + ".") > + if not console: > + continue > + > + (style_key, _, prop) = style_full_key.rpartition(".") > + if not prop or not style_key: > + raise Exception("malformed style config: {}".format(key)) > + > + if not style.get(style_key): > + style[style_key] = {} > + style[style_key][prop] = config[key] > + > + return FlowStyle({k: style_constructor(**v) for k, v in style.items()}) > + > + def format_flow(self, buf, flow, style_obj=None, highlighted=None): > + """Formats the flow into the provided buffer. > + > + Args: > + buf (FlowBuffer): the flow buffer to append to > + flow (ovs_dbg.OFPFlow): the flow to format > + style_obj (FlowStyle): Optional; style to use > + highlighted (list): Optional; list of KeyValues to highlight > + """ > + last_printed_pos = 0 > + > + if style_obj: > + style_obj = style_obj or FlowStyle() > + for section in sorted(flow.sections, key=lambda x: x.pos): > + buf.append_extra( > + flow.orig[last_printed_pos : section.pos], > + style=style_obj.get("default"), > + ) > + self.format_kv_list( > + buf, section.data, section.string, style_obj, highlighted > + ) > + last_printed_pos = section.pos + len(section.string) > + else: > + # Don't pay the cost of formatting each section one by one. > + buf.append_extra(flow.orig.strip(), None) > + > + def format_kv_list(self, buf, kv_list, full_str, style_obj, highlighted): > + """Format a KeyValue List. > + > + Args: > + buf (FlowBuffer): a FlowBuffer to append formatted KeyValues to > + kv_list (list[KeyValue]: the KeyValue list to format > + full_str (str): the full string containing all k-v > + style_obj (FlowStyle): a FlowStyle object to use > + highlighted (list): Optional; list of KeyValues to highlight > + """ > + for i, kv in enumerate(kv_list): > + written = self.format_kv( > + buf, kv, style_obj=style_obj, highlighted=highlighted > + ) > + > + end = ( > + kv_list[i + 1].meta.kpos > + if i < (len(kv_list) - 1) > + else len(full_str) > + ) > + > + buf.append_extra( > + full_str[(kv.meta.kpos + written) : end].rstrip("\n\r"), > + style=style_obj.get("default"), > + ) > + > + def format_kv(self, buf, kv, style_obj, highlighted=None): > + """Format a KeyValue > + > + A formatted keyvalue has the following parts: > + {key}{delim}{value}[{delim}] > + > + Args: > + buf (FlowBuffer): buffer to append the KeyValue to > + kv (KeyValue): The KeyValue to print > + style_obj (FlowStyle): The style object to use > + highlighted (list): Optional; list of KeyValues to highlight > + > + Returns the number of printed characters. > + """ > + ret = 0 > + key = kv.meta.kstring > + is_highlighted = ( > + key in [k.key for k in highlighted] if highlighted else False > + ) > + > + key_style = style_obj.get_key_style(kv, is_highlighted) > + buf.append_key(kv, key_style) # format value > + ret += len(key) > + > + if not kv.meta.vstring: > + return ret > + > + if kv.meta.delim not in ("\n", "\t", "\r", ""): > + buf.append_delim(kv, style_obj.get_delim_style(is_highlighted)) > + ret += len(kv.meta.delim) > + > + value_style = style_obj.get_value_style(kv, is_highlighted) > + buf.append_value(kv, value_style) # format value > + ret += len(kv.meta.vstring) > + > + if kv.meta.end_delim: > + buf.append_end_delim(kv, style_obj.get_delim_style(is_highlighted)) > + ret += len(kv.meta.end_delim) > + > + return ret > + > + > +class FlowBuffer: > + """A FlowBuffer is a base class for format buffers. > + > + Childs must implement the following methods: > + append_key(self, kv, style) > + append_value(self, kv, style) > + append_delim(self, delim, style) > + append_end_delim(self, delim, style) > + append_extra(self, extra, style) > + """ > + > + def append_key(self, kv, style): > + """Append a key. > + Args: > + kv (KeyValue): the KeyValue instance to append > + style (Any): the style to use > + """ > + raise NotImplementedError > + > + def append_delim(self, kv, style): > + """Append a delimiter. > + Args: > + kv (KeyValue): the KeyValue instance to append > + style (Any): the style to use > + """ > + raise NotImplementedError > + > + def append_end_delim(self, kv, style): > + """Append an end delimiter. > + Args: > + kv (KeyValue): the KeyValue instance to append > + style (Any): the style to use > + """ > + raise NotImplementedError > + > + def append_value(self, kv, style): > + """Append a value. > + Args: > + kv (KeyValue): the KeyValue instance to append > + style (Any): the style to use > + """ > + raise NotImplementedError > + > + def append_extra(self, extra, style): > + """Append extra string. > + Args: > + kv (KeyValue): the KeyValue instance to append > + style (Any): the style to use > + """ > + raise NotImplementedError > diff --git a/python/ovs/flowviz/main.py b/python/ovs/flowviz/main.py > index 64b0e8a0a..723c71fa7 100644 > --- a/python/ovs/flowviz/main.py > +++ b/python/ovs/flowviz/main.py > @@ -12,10 +12,30 @@ > # See the License for the specific language governing permissions and > # limitations under the License. > > +import configparser > import click > import os > > from ovs.flow.filter import OFFilter > +from ovs.dirs import PKGDATADIR > + > +_default_config_file = "ovs-flowviz.conf" > +_default_config_path = next( > + ( > + p > + for p in [ > + os.path.join( > + os.getenv("HOME"), ".config", "ovs", _default_config_file > + ), > + os.path.join(PKGDATADIR, _default_config_file), > + os.path.abspath( > + os.path.join(os.path.dirname(__file__), _default_config_file) > + ), > + ] > + if os.path.exists(p) > + ), > + "", > +) > > > class Options(dict): > @@ -48,6 +68,20 @@ def validate_input(ctx, param, value): > @click.group( > context_settings=dict(help_option_names=["-h", "--help"]), > ) > +@click.option( > + "-c", > + "--config", > + help="Use config file", > + type=click.Path(), > + default=_default_config_path, > + show_default=True, > +) > +@click.option( > + "--style", > + help="Select style (defined in config file)", > + default=None, > + show_default=True, > +) > @click.option( > "-i", > "--input", > @@ -69,8 +103,17 @@ def validate_input(ctx, param, value): > type=str, > show_default=False, > ) > +@click.option( > + "-l", > + "--highlight", > + help="Highlight flows that match the filter expression." > + "Run 'ovs-flowviz filter' for a detailed description of the filtering " > + "syntax", > + type=str, > + show_default=False, > +) > @click.pass_context > -def maincli(ctx, filename, filter): > +def maincli(ctx, config, style, filename, filter, highlight): > """ > OpenvSwitch flow visualization utility. > > @@ -86,6 +129,19 @@ def maincli(ctx, filename, filter): > except Exception as e: > raise click.BadParameter("Wrong filter syntax: {}".format(e)) > > + if highlight: > + try: > + ctx.obj["highlight"] = OFFilter(highlight) > + except Exception as e: > + raise click.BadParameter("Wrong filter syntax: {}".format(e)) > + > + config_file = config or _default_config_path > + parser = configparser.ConfigParser() > + parser.read(config_file) > + > + ctx.obj["config"] = parser > + ctx.obj["style"] = style > + > > @maincli.command(hidden=True) > @click.pass_context > diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py > index ed2f82065..78f5cfff4 100644 > --- a/python/ovs/flowviz/odp/cli.py > +++ b/python/ovs/flowviz/odp/cli.py > @@ -18,6 +18,7 @@ from ovs.flowviz.main import maincli > from ovs.flowviz.process import ( > DatapathFactory, > JSONProcessor, > + ConsoleProcessor, Maybe we should keet these in alphabetical order. > ) > > > @@ -40,3 +41,27 @@ def json(opts): > proc = JSONPrint(opts) > proc.process() > print(proc.json_string()) > + > + > +class DPConsoleProcessor(DatapathFactory, ConsoleProcessor): > + def __init__(self, opts, heat_map): > + super().__init__(opts, heat_map) > + > + > +@datapath.command() > +@click.option( > + "-h", > + "--heat-map", > + is_flag=True, > + default=False, > + show_default=True, > + help="Create heat-map with packet and byte counters", > +) > +@click.pass_obj > +def console(opts, heat_map): > + """Print the flows in the console with some style.""" > + proc = DPConsoleProcessor( > + opts, heat_map=["packets", "bytes"] if heat_map else [] > + ) > + proc.process() > + proc.print() > diff --git a/python/ovs/flowviz/ofp/cli.py b/python/ovs/flowviz/ofp/cli.py > index b9a2a8aad..a28e489ac 100644 > --- a/python/ovs/flowviz/ofp/cli.py > +++ b/python/ovs/flowviz/ofp/cli.py > @@ -18,6 +18,7 @@ from ovs.flowviz.main import maincli > from ovs.flowviz.process import ( > OpenFlowFactory, > JSONProcessor, > + ConsoleProcessor, > ) > > > @@ -40,3 +41,28 @@ def json(opts): > proc = JSONPrint(opts) > proc.process() > print(proc.json_string()) > + > + > +class OFConsoleProcessor(OpenFlowFactory, ConsoleProcessor): > + def __init__(self, opts, heat_map): > + super().__init__(opts, heat_map) > + > + > +@openflow.command() > +@click.option( > + "-h", > + "--heat-map", > + is_flag=True, > + default=False, > + show_default=True, > + help="Create heat-map with packet and byte counters", > +) > +@click.pass_obj > +def console(opts, heat_map): > + """Print the flows in the console with some style.""" > + proc = OFConsoleProcessor( > + opts, > + heat_map=["n_packets", "n_bytes"] if heat_map else [], > + ) > + proc.process() > + proc.print() > diff --git a/python/ovs/flowviz/process.py b/python/ovs/flowviz/process.py > index 3e520e431..df28dd2a7 100644 > --- a/python/ovs/flowviz/process.py > +++ b/python/ovs/flowviz/process.py > @@ -20,6 +20,13 @@ from ovs.flow.decoders import FlowEncoder > from ovs.flow.odp import ODPFlow > from ovs.flow.ofp import OFPFlow > > +from ovs.flowviz.console import ( > + ConsoleFormatter, > + default_highlight, > + heat_pallete, > + file_header, > +) > + > > class FileProcessor(object): > """Base class for file-based Flow processing. It is able to create flows > @@ -134,21 +141,24 @@ class FileProcessor(object): > self.end() > > > -class DatapathFactory(): > +class DatapathFactory: > """A mixin class that creates Datapath flows.""" > > def create_flow(self, line, idx): > # Skip strings commonly found in Datapath flow dumps. > - if any(s in line for s in [ > - "flow-dump from the main thread", > - "flow-dump from pmd on core", > - ]): > + if any( > + s in line > + for s in [ > + "flow-dump from the main thread", > + "flow-dump from pmd on core", > + ] > + ): > return None > > return ODPFlow(line, idx) > > > -class OpenFlowFactory(): > +class OpenFlowFactory: > """A mixin class that creates OpenFlow flows.""" > > def create_flow(self, line, idx): > @@ -190,3 +200,64 @@ class JSONProcessor(FileProcessor): > indent=4, > cls=FlowEncoder, > ) > + > + > +class ConsoleProcessor(FileProcessor): > + """A generic Console Processor that prints flows into the console""" > + > + def __init__(self, opts, heat_map=[]): > + super().__init__(opts) > + self.heat_map = heat_map > + self.console = ConsoleFormatter(opts) > + if len(self.console.style) == 0 and self.opts.get("highlight"): > + # Add some style to highlights or else they won't be seen. > + self.console.style.set_default_value_style( > + default_highlight(), True > + ) > + self.console.style.set_default_key_style(default_highlight(), True) > + > + self.flows = dict() # Dictionary of flow-lists, one per file. > + self.min_max = dict() # Used for heat-map calculation Ending comment line with a dot? > + > + def start_file(self, name, filename): > + self.flows_list = list() > + if len(self.heat_map) > 0: > + self.min = [-1] * len(self.heat_map) > + self.max = [0] * len(self.heat_map) > + > + def stop_file(self, name, filename): > + self.flows[name] = self.flows_list > + if len(self.heat_map) > 0: > + self.min_max[name] = (self.min, self.max) > + > + def process_flow(self, flow, name): > + # Running calculation of min and max values for all the fields that > + # take place in the heatmap. > + for i, field in enumerate(self.heat_map): > + val = flow.info.get(field) > + if self.min[i] == -1 or val < self.min[i]: > + self.min[i] = val > + if val > self.max[i]: > + self.max[i] = val > + > + self.flows_list.append(flow) > + > + def print(self): > + for name, flows in self.flows.items(): > + self.console.console.print("\n") > + self.console.console.print(file_header(name)) > + > + if len(self.heat_map) > 0 and len(self.flows) > 0: > + for i, field in enumerate(self.heat_map): > + (min_val, max_val) = self.min_max[name][i] > + self.console.style.set_value_style( > + field, heat_pallete(min_val, max_val) > + ) > + > + for flow in flows: > + high = None > + if self.opts.get("highlight"): > + result = self.opts.get("highlight").evaluate(flow) > + if result: > + high = result.kv > + self.console.print_flow(flow, high) > diff --git a/python/setup.py b/python/setup.py > index 4b9c751d2..76f9fc820 100644 > --- a/python/setup.py > +++ b/python/setup.py > @@ -113,9 +113,11 @@ setup_args = dict( > extras_require={':sys_platform == "win32"': ['pywin32 >= 1.0'], > 'dns': ['unbound'], > 'flow': flow_extras_require, > - 'flowviz': [*flow_extras_require, 'click'], > + 'flowviz': > + [*flow_extras_require, 'click', 'rich'], > }, > scripts=["ovs/flowviz/ovs-flowviz"], > + include_package_data=True, > ) > > try: > -- > 2.43.0 > > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
diff --git a/python/automake.mk b/python/automake.mk index fd5e74081..bd53c5405 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -65,6 +65,8 @@ ovs_pytests = \ ovs_flowviz = \ python/ovs/flowviz/__init__.py \ + python/ovs/flowviz/console.py \ + python/ovs/flowviz/format.py \ python/ovs/flowviz/main.py \ python/ovs/flowviz/odp/__init__.py \ python/ovs/flowviz/odp/cli.py \ diff --git a/python/ovs/flowviz/console.py b/python/ovs/flowviz/console.py new file mode 100644 index 000000000..4a3443360 --- /dev/null +++ b/python/ovs/flowviz/console.py @@ -0,0 +1,175 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import colorsys + +from rich.console import Console +from rich.color import Color +from rich.emoji import Emoji +from rich.panel import Panel +from rich.text import Text +from rich.style import Style + +from ovs.flowviz.format import FlowFormatter, FlowBuffer, FlowStyle + + +def file_header(name): + return Panel( + Text( + Emoji.replace(":scroll:") + + " " + + name + + " " + + Emoji.replace(":scroll:"), + style="bold", + justify="center", + ) + ) + + +class ConsoleBuffer(FlowBuffer): + """ConsoleBuffer implements FlowBuffer to provide console-based text + formatting based on rich.Text. + + Append functions accept a rich.Style. + + Args: + rtext(rich.Text): Optional; text instance to reuse + """ + + def __init__(self, rtext): + self._text = rtext or Text() + + @property + def text(self): + return self._text + + def _append(self, string, style): + """Append to internal text.""" + return self._text.append(string, style) + + def append_key(self, kv, style): + """Append a key. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(kv.meta.kstring, style) + + def append_delim(self, kv, style): + """Append a delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(kv.meta.delim, style) + + def append_end_delim(self, kv, style): + """Append an end delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(kv.meta.end_delim, style) + + def append_value(self, kv, style): + """Append a value. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(kv.meta.vstring, style) + + def append_extra(self, extra, style): + """Append extra string. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(extra, style) + + +class ConsoleFormatter(FlowFormatter): + """ConsoleFormatter is a FlowFormatter that formats flows into the console + using rich.Console. + + Args: + console (rich.Console): Optional, an existing console to use + max_value_len (int): Optional; max length of the printed values + kwargs (dict): Optional; Extra arguments to be passed down to + rich.console.Console() + """ + + def __init__(self, opts=None, console=None, **kwargs): + super(ConsoleFormatter, self).__init__() + style = self.style_from_opts(opts) + self.console = console or Console(color_system="256", **kwargs) + self.style = style or FlowStyle() + + def style_from_opts(self, opts): + return self._style_from_opts(opts, "console", Style) + + def print_flow(self, flow, highlighted=None): + """Prints a flow to the console. + + Args: + flow (ovs_dbg.OFPFlow): the flow to print + style (dict): Optional; style dictionary to use + highlighted (list): Optional; list of KeyValues to highlight + """ + + buf = ConsoleBuffer(Text()) + self.format_flow(buf, flow, highlighted) + self.console.print(buf.text) + + def format_flow(self, buf, flow, highlighted=None): + """Formats the flow into the provided buffer as a rich.Text. + + Args: + buf (FlowBuffer): the flow buffer to append to + flow (ovs_dbg.OFPFlow): the flow to format + style (FlowStyle): Optional; style object to use + highlighted (list): Optional; list of KeyValues to highlight + """ + return super(ConsoleFormatter, self).format_flow( + buf, flow, self.style, highlighted + ) + + +def heat_pallete(min_value, max_value): + """Generates a color pallete based on the 5-color heat pallete so that + for each value between min and max a color is returned that represents it's + relative size. + Args: + min_value (int): minimum value + max_value (int) maximum value + """ + h_min = 0 # red + h_max = 220 / 360 # blue + + def heat(value): + if max_value == min_value: + r, g, b = colorsys.hsv_to_rgb(h_max / 2, 1.0, 1.0) + else: + normalized = (int(value) - min_value) / (max_value - min_value) + hue = ((1 - normalized) + h_min) * (h_max - h_min) + r, g, b = colorsys.hsv_to_rgb(hue, 1.0, 1.0) + return Style(color=Color.from_rgb(r * 255, g * 255, b * 255)) + + return heat + + +def default_highlight(): + """Generates a default style for highlights.""" + return Style(underline=True) diff --git a/python/ovs/flowviz/format.py b/python/ovs/flowviz/format.py new file mode 100644 index 000000000..70af2fa26 --- /dev/null +++ b/python/ovs/flowviz/format.py @@ -0,0 +1,371 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Flow formatting framework. + +This file defines a simple flow formatting framework. It's comprised of 3 +classes: FlowStyle, FlowFormatter and FlowBuffer. + +The FlowStyle arranges opaque style objects in a dictionary that can be queried +to determine what style a particular key-value should be formatted with. +That way, a particular implementation can represent its style using their own +object. + +The FlowBuffer is an abstract class and must be derived by particular +implementations. It should know how to append parts of a flow using a style. +Only here the type of the style is relevant. + +When asked to format a flow, the FlowFormatter will determine which style +the flow must be formatted with and call FlowBuffer functions with each part +of the flow and their corresponding style. +""" + + +class FlowStyle: + """A FlowStyle determines the KVStyle to use for each key value in a flow. + + Styles are internally represented by a dictionary. + In order to determine the style for a "key", the following items in the + dictionary are fetched: + - key.highlighted.{key} (if key is found in hightlighted) + - key.highlighted (if key is found in hightlighted) + - key.{key} + - key + - default + + In order to determine the style for a "value", the following items in the + dictionary are fetched: + - value.highlighted.{key} (if key is found in hightlighted) + - value.highlighted.type{value.__class__.__name__} + - value.highlighted + (if key is found in hightlighted) + - value.{key} + - value.type.{value.__class__.__name__} + - value + - default + + The actual type of the style object stored for each item above is opaque + to this class and it depends on the particular FlowFormatter child class + that will handle them. Even callables can be stored, if so they will be + called with the value of the field that is to be formatted and the return + object will be used as style. + + Additionally, the following style items can be defined: + - delim: for delimiters + - delim.highlighted: for delimiters of highlighted key-values + """ + + def __init__(self, initial=None): + self._styles = initial if initial is not None else dict() + + def __len__(self): + return len(self._styles) + + def set_flag_style(self, kvstyle): + self._styles["flag"] = kvstyle + + def set_delim_style(self, kvstyle, highlighted=False): + if highlighted: + self._styles["delim.highlighted"] = kvstyle + else: + self._styles["delim"] = kvstyle + + def set_default_key_style(self, kvstyle, highlighted=False): + if highlighted: + self._styles["key.highlighted"] = kvstyle + else: + self._styles["key"] = kvstyle + + def set_default_value_style(self, kvstyle, highlighted=False): + if highlighted: + self._styles["value.highlighted"] = kvstyle + else: + self._styles["value"] = kvstyle + + def set_key_style(self, key, kvstyle, highlighted=False): + if highlighted: + self._styles["key.highlighted.{}".format(key)] = kvstyle + else: + self._styles["key.{}".format(key)] = kvstyle + + def set_value_style(self, key, kvstyle, highlighted=None): + if highlighted: + self._styles["value.highlighted.{}".format(key)] = kvstyle + else: + self._styles["value.{}".format(key)] = kvstyle + + def set_value_type_style(self, name, kvstyle, highlighted=None): + if highlighted: + self._styles["value.highlighted.type.{}".format(name)] = kvstyle + else: + self._styles["value.type.{}".format(name)] = kvstyle + + def get(self, key): + return self._styles.get(key) + + def get_delim_style(self, highlighted=False): + delim_style_lookup = ["delim.highlighted"] if highlighted else [] + delim_style_lookup.extend(["delim", "default"]) + return next( + ( + self._styles.get(s) + for s in delim_style_lookup + if self._styles.get(s) + ), + None, + ) + + def get_flag_style(self): + return self._styles.get("flag") or self._styles.get("default") + + def get_key_style(self, kv, highlighted=False): + key = kv.key + + key_style_lookup = ( + ["key.highlighted.%s" % key, "key.highlighted"] + if highlighted + else [] + ) + key_style_lookup.extend(["key.%s" % key, "key", "default"]) + + style = next( + ( + self._styles.get(s) + for s in key_style_lookup + if self._styles.get(s) + ), + None, + ) + if callable(style): + return style(kv.meta.kstring) + return style + + def get_value_style(self, kv, highlighted=False): + key = kv.key + value_type = kv.value.__class__.__name__.lower() + value_style_lookup = ( + [ + "value.highlighted.%s" % key, + "value.highlighted.type.%s" % value_type, + "value.highlighted", + ] + if highlighted + else [] + ) + value_style_lookup.extend( + [ + "value.%s" % key, + "value.type.%s" % value_type, + "value", + "default", + ] + ) + + style = next( + ( + self._styles.get(s) + for s in value_style_lookup + if self._styles.get(s) + ), + None, + ) + if callable(style): + return style(kv.meta.vstring) + return style + + +class FlowFormatter: + """FlowFormatter is a base class for Flow Formatters.""" + + def __init__(self): + self._highlighted = list() + + def _style_from_opts(self, opts, opts_key, style_constructor): + """Create style object from options. + + Args: + opts (dict): Options dictionary + opts_key (str): The options style key to extract + (e.g: console or html) + style_constructor(callable): A callable that creates a derived + style object + """ + if not opts or not opts.get("style"): + return None + + section_name = ".".join(["styles", opts.get("style")]) + if section_name not in opts.get("config").sections(): + return None + + config = opts.get("config")[section_name] + style = {} + for key in config: + (_, console, style_full_key) = key.partition(opts_key + ".") + if not console: + continue + + (style_key, _, prop) = style_full_key.rpartition(".") + if not prop or not style_key: + raise Exception("malformed style config: {}".format(key)) + + if not style.get(style_key): + style[style_key] = {} + style[style_key][prop] = config[key] + + return FlowStyle({k: style_constructor(**v) for k, v in style.items()}) + + def format_flow(self, buf, flow, style_obj=None, highlighted=None): + """Formats the flow into the provided buffer. + + Args: + buf (FlowBuffer): the flow buffer to append to + flow (ovs_dbg.OFPFlow): the flow to format + style_obj (FlowStyle): Optional; style to use + highlighted (list): Optional; list of KeyValues to highlight + """ + last_printed_pos = 0 + + if style_obj: + style_obj = style_obj or FlowStyle() + for section in sorted(flow.sections, key=lambda x: x.pos): + buf.append_extra( + flow.orig[last_printed_pos : section.pos], + style=style_obj.get("default"), + ) + self.format_kv_list( + buf, section.data, section.string, style_obj, highlighted + ) + last_printed_pos = section.pos + len(section.string) + else: + # Don't pay the cost of formatting each section one by one. + buf.append_extra(flow.orig.strip(), None) + + def format_kv_list(self, buf, kv_list, full_str, style_obj, highlighted): + """Format a KeyValue List. + + Args: + buf (FlowBuffer): a FlowBuffer to append formatted KeyValues to + kv_list (list[KeyValue]: the KeyValue list to format + full_str (str): the full string containing all k-v + style_obj (FlowStyle): a FlowStyle object to use + highlighted (list): Optional; list of KeyValues to highlight + """ + for i, kv in enumerate(kv_list): + written = self.format_kv( + buf, kv, style_obj=style_obj, highlighted=highlighted + ) + + end = ( + kv_list[i + 1].meta.kpos + if i < (len(kv_list) - 1) + else len(full_str) + ) + + buf.append_extra( + full_str[(kv.meta.kpos + written) : end].rstrip("\n\r"), + style=style_obj.get("default"), + ) + + def format_kv(self, buf, kv, style_obj, highlighted=None): + """Format a KeyValue + + A formatted keyvalue has the following parts: + {key}{delim}{value}[{delim}] + + Args: + buf (FlowBuffer): buffer to append the KeyValue to + kv (KeyValue): The KeyValue to print + style_obj (FlowStyle): The style object to use + highlighted (list): Optional; list of KeyValues to highlight + + Returns the number of printed characters. + """ + ret = 0 + key = kv.meta.kstring + is_highlighted = ( + key in [k.key for k in highlighted] if highlighted else False + ) + + key_style = style_obj.get_key_style(kv, is_highlighted) + buf.append_key(kv, key_style) # format value + ret += len(key) + + if not kv.meta.vstring: + return ret + + if kv.meta.delim not in ("\n", "\t", "\r", ""): + buf.append_delim(kv, style_obj.get_delim_style(is_highlighted)) + ret += len(kv.meta.delim) + + value_style = style_obj.get_value_style(kv, is_highlighted) + buf.append_value(kv, value_style) # format value + ret += len(kv.meta.vstring) + + if kv.meta.end_delim: + buf.append_end_delim(kv, style_obj.get_delim_style(is_highlighted)) + ret += len(kv.meta.end_delim) + + return ret + + +class FlowBuffer: + """A FlowBuffer is a base class for format buffers. + + Childs must implement the following methods: + append_key(self, kv, style) + append_value(self, kv, style) + append_delim(self, delim, style) + append_end_delim(self, delim, style) + append_extra(self, extra, style) + """ + + def append_key(self, kv, style): + """Append a key. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError + + def append_delim(self, kv, style): + """Append a delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError + + def append_end_delim(self, kv, style): + """Append an end delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError + + def append_value(self, kv, style): + """Append a value. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError + + def append_extra(self, extra, style): + """Append extra string. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError diff --git a/python/ovs/flowviz/main.py b/python/ovs/flowviz/main.py index 64b0e8a0a..723c71fa7 100644 --- a/python/ovs/flowviz/main.py +++ b/python/ovs/flowviz/main.py @@ -12,10 +12,30 @@ # See the License for the specific language governing permissions and # limitations under the License. +import configparser import click import os from ovs.flow.filter import OFFilter +from ovs.dirs import PKGDATADIR + +_default_config_file = "ovs-flowviz.conf" +_default_config_path = next( + ( + p + for p in [ + os.path.join( + os.getenv("HOME"), ".config", "ovs", _default_config_file + ), + os.path.join(PKGDATADIR, _default_config_file), + os.path.abspath( + os.path.join(os.path.dirname(__file__), _default_config_file) + ), + ] + if os.path.exists(p) + ), + "", +) class Options(dict): @@ -48,6 +68,20 @@ def validate_input(ctx, param, value): @click.group( context_settings=dict(help_option_names=["-h", "--help"]), ) +@click.option( + "-c", + "--config", + help="Use config file", + type=click.Path(), + default=_default_config_path, + show_default=True, +) +@click.option( + "--style", + help="Select style (defined in config file)", + default=None, + show_default=True, +) @click.option( "-i", "--input", @@ -69,8 +103,17 @@ def validate_input(ctx, param, value): type=str, show_default=False, ) +@click.option( + "-l", + "--highlight", + help="Highlight flows that match the filter expression." + "Run 'ovs-flowviz filter' for a detailed description of the filtering " + "syntax", + type=str, + show_default=False, +) @click.pass_context -def maincli(ctx, filename, filter): +def maincli(ctx, config, style, filename, filter, highlight): """ OpenvSwitch flow visualization utility. @@ -86,6 +129,19 @@ def maincli(ctx, filename, filter): except Exception as e: raise click.BadParameter("Wrong filter syntax: {}".format(e)) + if highlight: + try: + ctx.obj["highlight"] = OFFilter(highlight) + except Exception as e: + raise click.BadParameter("Wrong filter syntax: {}".format(e)) + + config_file = config or _default_config_path + parser = configparser.ConfigParser() + parser.read(config_file) + + ctx.obj["config"] = parser + ctx.obj["style"] = style + @maincli.command(hidden=True) @click.pass_context diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py index ed2f82065..78f5cfff4 100644 --- a/python/ovs/flowviz/odp/cli.py +++ b/python/ovs/flowviz/odp/cli.py @@ -18,6 +18,7 @@ from ovs.flowviz.main import maincli from ovs.flowviz.process import ( DatapathFactory, JSONProcessor, + ConsoleProcessor, ) @@ -40,3 +41,27 @@ def json(opts): proc = JSONPrint(opts) proc.process() print(proc.json_string()) + + +class DPConsoleProcessor(DatapathFactory, ConsoleProcessor): + def __init__(self, opts, heat_map): + super().__init__(opts, heat_map) + + +@datapath.command() +@click.option( + "-h", + "--heat-map", + is_flag=True, + default=False, + show_default=True, + help="Create heat-map with packet and byte counters", +) +@click.pass_obj +def console(opts, heat_map): + """Print the flows in the console with some style.""" + proc = DPConsoleProcessor( + opts, heat_map=["packets", "bytes"] if heat_map else [] + ) + proc.process() + proc.print() diff --git a/python/ovs/flowviz/ofp/cli.py b/python/ovs/flowviz/ofp/cli.py index b9a2a8aad..a28e489ac 100644 --- a/python/ovs/flowviz/ofp/cli.py +++ b/python/ovs/flowviz/ofp/cli.py @@ -18,6 +18,7 @@ from ovs.flowviz.main import maincli from ovs.flowviz.process import ( OpenFlowFactory, JSONProcessor, + ConsoleProcessor, ) @@ -40,3 +41,28 @@ def json(opts): proc = JSONPrint(opts) proc.process() print(proc.json_string()) + + +class OFConsoleProcessor(OpenFlowFactory, ConsoleProcessor): + def __init__(self, opts, heat_map): + super().__init__(opts, heat_map) + + +@openflow.command() +@click.option( + "-h", + "--heat-map", + is_flag=True, + default=False, + show_default=True, + help="Create heat-map with packet and byte counters", +) +@click.pass_obj +def console(opts, heat_map): + """Print the flows in the console with some style.""" + proc = OFConsoleProcessor( + opts, + heat_map=["n_packets", "n_bytes"] if heat_map else [], + ) + proc.process() + proc.print() diff --git a/python/ovs/flowviz/process.py b/python/ovs/flowviz/process.py index 3e520e431..df28dd2a7 100644 --- a/python/ovs/flowviz/process.py +++ b/python/ovs/flowviz/process.py @@ -20,6 +20,13 @@ from ovs.flow.decoders import FlowEncoder from ovs.flow.odp import ODPFlow from ovs.flow.ofp import OFPFlow +from ovs.flowviz.console import ( + ConsoleFormatter, + default_highlight, + heat_pallete, + file_header, +) + class FileProcessor(object): """Base class for file-based Flow processing. It is able to create flows @@ -134,21 +141,24 @@ class FileProcessor(object): self.end() -class DatapathFactory(): +class DatapathFactory: """A mixin class that creates Datapath flows.""" def create_flow(self, line, idx): # Skip strings commonly found in Datapath flow dumps. - if any(s in line for s in [ - "flow-dump from the main thread", - "flow-dump from pmd on core", - ]): + if any( + s in line + for s in [ + "flow-dump from the main thread", + "flow-dump from pmd on core", + ] + ): return None return ODPFlow(line, idx) -class OpenFlowFactory(): +class OpenFlowFactory: """A mixin class that creates OpenFlow flows.""" def create_flow(self, line, idx): @@ -190,3 +200,64 @@ class JSONProcessor(FileProcessor): indent=4, cls=FlowEncoder, ) + + +class ConsoleProcessor(FileProcessor): + """A generic Console Processor that prints flows into the console""" + + def __init__(self, opts, heat_map=[]): + super().__init__(opts) + self.heat_map = heat_map + self.console = ConsoleFormatter(opts) + if len(self.console.style) == 0 and self.opts.get("highlight"): + # Add some style to highlights or else they won't be seen. + self.console.style.set_default_value_style( + default_highlight(), True + ) + self.console.style.set_default_key_style(default_highlight(), True) + + self.flows = dict() # Dictionary of flow-lists, one per file. + self.min_max = dict() # Used for heat-map calculation + + def start_file(self, name, filename): + self.flows_list = list() + if len(self.heat_map) > 0: + self.min = [-1] * len(self.heat_map) + self.max = [0] * len(self.heat_map) + + def stop_file(self, name, filename): + self.flows[name] = self.flows_list + if len(self.heat_map) > 0: + self.min_max[name] = (self.min, self.max) + + def process_flow(self, flow, name): + # Running calculation of min and max values for all the fields that + # take place in the heatmap. + for i, field in enumerate(self.heat_map): + val = flow.info.get(field) + if self.min[i] == -1 or val < self.min[i]: + self.min[i] = val + if val > self.max[i]: + self.max[i] = val + + self.flows_list.append(flow) + + def print(self): + for name, flows in self.flows.items(): + self.console.console.print("\n") + self.console.console.print(file_header(name)) + + if len(self.heat_map) > 0 and len(self.flows) > 0: + for i, field in enumerate(self.heat_map): + (min_val, max_val) = self.min_max[name][i] + self.console.style.set_value_style( + field, heat_pallete(min_val, max_val) + ) + + for flow in flows: + high = None + if self.opts.get("highlight"): + result = self.opts.get("highlight").evaluate(flow) + if result: + high = result.kv + self.console.print_flow(flow, high) diff --git a/python/setup.py b/python/setup.py index 4b9c751d2..76f9fc820 100644 --- a/python/setup.py +++ b/python/setup.py @@ -113,9 +113,11 @@ setup_args = dict( extras_require={':sys_platform == "win32"': ['pywin32 >= 1.0'], 'dns': ['unbound'], 'flow': flow_extras_require, - 'flowviz': [*flow_extras_require, 'click'], + 'flowviz': + [*flow_extras_require, 'click', 'rich'], }, scripts=["ovs/flowviz/ovs-flowviz"], + include_package_data=True, ) try:
Add a flow formatting framework and one implementation for console printing using rich. The flow formatting framework is a simple set of classes that can be used to write different flow formatting implementations. It supports styles to be described by any class, highlighting and config-file based style definition. The first flow formatting implementation is also introduced: the ConsoleFormatter. It uses the an advanced rich-text printing library [1]. The console printing supports: - Heatmap: printing the packet/byte statistics of each flow in a color that represents its relative size: blue (low) -> red (high). - Printing a banner with the file name and alias. - Extensive style definition via config file. This console format is added to both OpenFlow and Datapath flows. Examples: - Highlight drops in datapath flows: $ ovs-flowviz -i flows.txt --highlight "drop" datapath console - Quickly detect where most packets are going using heatmap and paginated output: $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow console -h [1] https://rich.readthedocs.io/en/stable/introduction.html Signed-off-by: Adrian Moreno <amorenoz@redhat.com> --- python/automake.mk | 2 + python/ovs/flowviz/console.py | 175 ++++++++++++++++ python/ovs/flowviz/format.py | 371 ++++++++++++++++++++++++++++++++++ python/ovs/flowviz/main.py | 58 +++++- python/ovs/flowviz/odp/cli.py | 25 +++ python/ovs/flowviz/ofp/cli.py | 26 +++ python/ovs/flowviz/process.py | 83 +++++++- python/setup.py | 4 +- 8 files changed, 736 insertions(+), 8 deletions(-) create mode 100644 python/ovs/flowviz/console.py create mode 100644 python/ovs/flowviz/format.py