From patchwork Tue Apr 9 07:06:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1921251 X-Patchwork-Delegate: i.maximets@samsung.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=L7X4gyf2; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VDH8J4GsDz1yY8 for ; Tue, 9 Apr 2024 17:07:12 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id AEF5740995; Tue, 9 Apr 2024 07:07:06 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id ecUiIF_j4heF; Tue, 9 Apr 2024 07:07:01 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org C6636408F8 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=L7X4gyf2 Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id C6636408F8; Tue, 9 Apr 2024 07:07:00 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id AAAF6C007C; Tue, 9 Apr 2024 07:07:00 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id BA945C0037 for ; Tue, 9 Apr 2024 07:06:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 87F4D41487 for ; Tue, 9 Apr 2024 07:06:59 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id qB09HczEELMG for ; Tue, 9 Apr 2024 07:06:57 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp2.osuosl.org A1306400C7 Authentication-Results: smtp2.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org A1306400C7 Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=L7X4gyf2 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id A1306400C7 for ; Tue, 9 Apr 2024 07:06:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712646415; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YRCwDE8qyYXuvLZUZPToq5VyCL8WkszKKn6Cj4biQos=; b=L7X4gyf2/hmwWJwTbnH3NH+XcufcV2iUA1rZHoDbHRW6IU6D8s4hNgwU40b76866sc9Tmw IsIDJ6DpscToEDjuTKHJrK3Bf44qpKhoEx8fLy2HFRiH1PmL49/7oHC0D98Li4FLGxzg4N FYrx2TfgeLes4qJyT3aw4/yZAGwRUjQ= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-473-T_nrth3tM16g1wIhnRg6zA-1; Tue, 09 Apr 2024 03:06:53 -0400 X-MC-Unique: T_nrth3tM16g1wIhnRg6zA-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 834FF3806738 for ; Tue, 9 Apr 2024 07:06:53 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.113]) by smtp.corp.redhat.com (Postfix) with ESMTP id A56CA444565; Tue, 9 Apr 2024 07:06:52 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 9 Apr 2024 09:06:21 +0200 Message-ID: <20240409070642.511747-4-amorenoz@redhat.com> In-Reply-To: <20240409070642.511747-1-amorenoz@redhat.com> References: <20240409070642.511747-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.10 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v3 03/12] python: ovs: flowviz: Add console formatting. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add a flow formatting framework and one implementation for console printing using rich. The flow formatting framework is a simple set of classes that can be used to write different flow formatting implementations. It supports styles to be described by any class, highlighting and config-file based style definition. The first flow formatting implementation is also introduced: the ConsoleFormatter. It uses the an advanced rich-text printing library [1]. The console printing supports: - Heatmap: printing the packet/byte statistics of each flow in a color that represents its relative size: blue (low) -> red (high). - Printing a banner with the file name and alias. - Extensive style definition via config file. This console format is added to both OpenFlow and Datapath flows. Examples: - Highlight drops in datapath flows: $ ovs-flowviz -i flows.txt --highlight "drop" datapath console - Quickly detect where most packets are going using heatmap and paginated output: $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow console -h [1] https://rich.readthedocs.io/en/stable/introduction.html Signed-off-by: Adrian Moreno Acked-by: Eelco Chaudron --- python/automake.mk | 2 + python/ovs/flowviz/console.py | 175 ++++++++++++++++ python/ovs/flowviz/format.py | 371 ++++++++++++++++++++++++++++++++++ python/ovs/flowviz/main.py | 58 +++++- python/ovs/flowviz/odp/cli.py | 25 +++ python/ovs/flowviz/ofp/cli.py | 26 +++ python/ovs/flowviz/process.py | 83 +++++++- python/setup.py | 4 +- 8 files changed, 736 insertions(+), 8 deletions(-) create mode 100644 python/ovs/flowviz/console.py create mode 100644 python/ovs/flowviz/format.py diff --git a/python/automake.mk b/python/automake.mk index fd5e74081..bd53c5405 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -65,6 +65,8 @@ ovs_pytests = \ ovs_flowviz = \ python/ovs/flowviz/__init__.py \ + python/ovs/flowviz/console.py \ + python/ovs/flowviz/format.py \ python/ovs/flowviz/main.py \ python/ovs/flowviz/odp/__init__.py \ python/ovs/flowviz/odp/cli.py \ diff --git a/python/ovs/flowviz/console.py b/python/ovs/flowviz/console.py new file mode 100644 index 000000000..4a3443360 --- /dev/null +++ b/python/ovs/flowviz/console.py @@ -0,0 +1,175 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import colorsys + +from rich.console import Console +from rich.color import Color +from rich.emoji import Emoji +from rich.panel import Panel +from rich.text import Text +from rich.style import Style + +from ovs.flowviz.format import FlowFormatter, FlowBuffer, FlowStyle + + +def file_header(name): + return Panel( + Text( + Emoji.replace(":scroll:") + + " " + + name + + " " + + Emoji.replace(":scroll:"), + style="bold", + justify="center", + ) + ) + + +class ConsoleBuffer(FlowBuffer): + """ConsoleBuffer implements FlowBuffer to provide console-based text + formatting based on rich.Text. + + Append functions accept a rich.Style. + + Args: + rtext(rich.Text): Optional; text instance to reuse + """ + + def __init__(self, rtext): + self._text = rtext or Text() + + @property + def text(self): + return self._text + + def _append(self, string, style): + """Append to internal text.""" + return self._text.append(string, style) + + def append_key(self, kv, style): + """Append a key. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(kv.meta.kstring, style) + + def append_delim(self, kv, style): + """Append a delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(kv.meta.delim, style) + + def append_end_delim(self, kv, style): + """Append an end delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(kv.meta.end_delim, style) + + def append_value(self, kv, style): + """Append a value. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(kv.meta.vstring, style) + + def append_extra(self, extra, style): + """Append extra string. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(extra, style) + + +class ConsoleFormatter(FlowFormatter): + """ConsoleFormatter is a FlowFormatter that formats flows into the console + using rich.Console. + + Args: + console (rich.Console): Optional, an existing console to use + max_value_len (int): Optional; max length of the printed values + kwargs (dict): Optional; Extra arguments to be passed down to + rich.console.Console() + """ + + def __init__(self, opts=None, console=None, **kwargs): + super(ConsoleFormatter, self).__init__() + style = self.style_from_opts(opts) + self.console = console or Console(color_system="256", **kwargs) + self.style = style or FlowStyle() + + def style_from_opts(self, opts): + return self._style_from_opts(opts, "console", Style) + + def print_flow(self, flow, highlighted=None): + """Prints a flow to the console. + + Args: + flow (ovs_dbg.OFPFlow): the flow to print + style (dict): Optional; style dictionary to use + highlighted (list): Optional; list of KeyValues to highlight + """ + + buf = ConsoleBuffer(Text()) + self.format_flow(buf, flow, highlighted) + self.console.print(buf.text) + + def format_flow(self, buf, flow, highlighted=None): + """Formats the flow into the provided buffer as a rich.Text. + + Args: + buf (FlowBuffer): the flow buffer to append to + flow (ovs_dbg.OFPFlow): the flow to format + style (FlowStyle): Optional; style object to use + highlighted (list): Optional; list of KeyValues to highlight + """ + return super(ConsoleFormatter, self).format_flow( + buf, flow, self.style, highlighted + ) + + +def heat_pallete(min_value, max_value): + """Generates a color pallete based on the 5-color heat pallete so that + for each value between min and max a color is returned that represents it's + relative size. + Args: + min_value (int): minimum value + max_value (int) maximum value + """ + h_min = 0 # red + h_max = 220 / 360 # blue + + def heat(value): + if max_value == min_value: + r, g, b = colorsys.hsv_to_rgb(h_max / 2, 1.0, 1.0) + else: + normalized = (int(value) - min_value) / (max_value - min_value) + hue = ((1 - normalized) + h_min) * (h_max - h_min) + r, g, b = colorsys.hsv_to_rgb(hue, 1.0, 1.0) + return Style(color=Color.from_rgb(r * 255, g * 255, b * 255)) + + return heat + + +def default_highlight(): + """Generates a default style for highlights.""" + return Style(underline=True) diff --git a/python/ovs/flowviz/format.py b/python/ovs/flowviz/format.py new file mode 100644 index 000000000..70af2fa26 --- /dev/null +++ b/python/ovs/flowviz/format.py @@ -0,0 +1,371 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Flow formatting framework. + +This file defines a simple flow formatting framework. It's comprised of 3 +classes: FlowStyle, FlowFormatter and FlowBuffer. + +The FlowStyle arranges opaque style objects in a dictionary that can be queried +to determine what style a particular key-value should be formatted with. +That way, a particular implementation can represent its style using their own +object. + +The FlowBuffer is an abstract class and must be derived by particular +implementations. It should know how to append parts of a flow using a style. +Only here the type of the style is relevant. + +When asked to format a flow, the FlowFormatter will determine which style +the flow must be formatted with and call FlowBuffer functions with each part +of the flow and their corresponding style. +""" + + +class FlowStyle: + """A FlowStyle determines the KVStyle to use for each key value in a flow. + + Styles are internally represented by a dictionary. + In order to determine the style for a "key", the following items in the + dictionary are fetched: + - key.highlighted.{key} (if key is found in hightlighted) + - key.highlighted (if key is found in hightlighted) + - key.{key} + - key + - default + + In order to determine the style for a "value", the following items in the + dictionary are fetched: + - value.highlighted.{key} (if key is found in hightlighted) + - value.highlighted.type{value.__class__.__name__} + - value.highlighted + (if key is found in hightlighted) + - value.{key} + - value.type.{value.__class__.__name__} + - value + - default + + The actual type of the style object stored for each item above is opaque + to this class and it depends on the particular FlowFormatter child class + that will handle them. Even callables can be stored, if so they will be + called with the value of the field that is to be formatted and the return + object will be used as style. + + Additionally, the following style items can be defined: + - delim: for delimiters + - delim.highlighted: for delimiters of highlighted key-values + """ + + def __init__(self, initial=None): + self._styles = initial if initial is not None else dict() + + def __len__(self): + return len(self._styles) + + def set_flag_style(self, kvstyle): + self._styles["flag"] = kvstyle + + def set_delim_style(self, kvstyle, highlighted=False): + if highlighted: + self._styles["delim.highlighted"] = kvstyle + else: + self._styles["delim"] = kvstyle + + def set_default_key_style(self, kvstyle, highlighted=False): + if highlighted: + self._styles["key.highlighted"] = kvstyle + else: + self._styles["key"] = kvstyle + + def set_default_value_style(self, kvstyle, highlighted=False): + if highlighted: + self._styles["value.highlighted"] = kvstyle + else: + self._styles["value"] = kvstyle + + def set_key_style(self, key, kvstyle, highlighted=False): + if highlighted: + self._styles["key.highlighted.{}".format(key)] = kvstyle + else: + self._styles["key.{}".format(key)] = kvstyle + + def set_value_style(self, key, kvstyle, highlighted=None): + if highlighted: + self._styles["value.highlighted.{}".format(key)] = kvstyle + else: + self._styles["value.{}".format(key)] = kvstyle + + def set_value_type_style(self, name, kvstyle, highlighted=None): + if highlighted: + self._styles["value.highlighted.type.{}".format(name)] = kvstyle + else: + self._styles["value.type.{}".format(name)] = kvstyle + + def get(self, key): + return self._styles.get(key) + + def get_delim_style(self, highlighted=False): + delim_style_lookup = ["delim.highlighted"] if highlighted else [] + delim_style_lookup.extend(["delim", "default"]) + return next( + ( + self._styles.get(s) + for s in delim_style_lookup + if self._styles.get(s) + ), + None, + ) + + def get_flag_style(self): + return self._styles.get("flag") or self._styles.get("default") + + def get_key_style(self, kv, highlighted=False): + key = kv.key + + key_style_lookup = ( + ["key.highlighted.%s" % key, "key.highlighted"] + if highlighted + else [] + ) + key_style_lookup.extend(["key.%s" % key, "key", "default"]) + + style = next( + ( + self._styles.get(s) + for s in key_style_lookup + if self._styles.get(s) + ), + None, + ) + if callable(style): + return style(kv.meta.kstring) + return style + + def get_value_style(self, kv, highlighted=False): + key = kv.key + value_type = kv.value.__class__.__name__.lower() + value_style_lookup = ( + [ + "value.highlighted.%s" % key, + "value.highlighted.type.%s" % value_type, + "value.highlighted", + ] + if highlighted + else [] + ) + value_style_lookup.extend( + [ + "value.%s" % key, + "value.type.%s" % value_type, + "value", + "default", + ] + ) + + style = next( + ( + self._styles.get(s) + for s in value_style_lookup + if self._styles.get(s) + ), + None, + ) + if callable(style): + return style(kv.meta.vstring) + return style + + +class FlowFormatter: + """FlowFormatter is a base class for Flow Formatters.""" + + def __init__(self): + self._highlighted = list() + + def _style_from_opts(self, opts, opts_key, style_constructor): + """Create style object from options. + + Args: + opts (dict): Options dictionary + opts_key (str): The options style key to extract + (e.g: console or html) + style_constructor(callable): A callable that creates a derived + style object + """ + if not opts or not opts.get("style"): + return None + + section_name = ".".join(["styles", opts.get("style")]) + if section_name not in opts.get("config").sections(): + return None + + config = opts.get("config")[section_name] + style = {} + for key in config: + (_, console, style_full_key) = key.partition(opts_key + ".") + if not console: + continue + + (style_key, _, prop) = style_full_key.rpartition(".") + if not prop or not style_key: + raise Exception("malformed style config: {}".format(key)) + + if not style.get(style_key): + style[style_key] = {} + style[style_key][prop] = config[key] + + return FlowStyle({k: style_constructor(**v) for k, v in style.items()}) + + def format_flow(self, buf, flow, style_obj=None, highlighted=None): + """Formats the flow into the provided buffer. + + Args: + buf (FlowBuffer): the flow buffer to append to + flow (ovs_dbg.OFPFlow): the flow to format + style_obj (FlowStyle): Optional; style to use + highlighted (list): Optional; list of KeyValues to highlight + """ + last_printed_pos = 0 + + if style_obj: + style_obj = style_obj or FlowStyle() + for section in sorted(flow.sections, key=lambda x: x.pos): + buf.append_extra( + flow.orig[last_printed_pos : section.pos], + style=style_obj.get("default"), + ) + self.format_kv_list( + buf, section.data, section.string, style_obj, highlighted + ) + last_printed_pos = section.pos + len(section.string) + else: + # Don't pay the cost of formatting each section one by one. + buf.append_extra(flow.orig.strip(), None) + + def format_kv_list(self, buf, kv_list, full_str, style_obj, highlighted): + """Format a KeyValue List. + + Args: + buf (FlowBuffer): a FlowBuffer to append formatted KeyValues to + kv_list (list[KeyValue]: the KeyValue list to format + full_str (str): the full string containing all k-v + style_obj (FlowStyle): a FlowStyle object to use + highlighted (list): Optional; list of KeyValues to highlight + """ + for i, kv in enumerate(kv_list): + written = self.format_kv( + buf, kv, style_obj=style_obj, highlighted=highlighted + ) + + end = ( + kv_list[i + 1].meta.kpos + if i < (len(kv_list) - 1) + else len(full_str) + ) + + buf.append_extra( + full_str[(kv.meta.kpos + written) : end].rstrip("\n\r"), + style=style_obj.get("default"), + ) + + def format_kv(self, buf, kv, style_obj, highlighted=None): + """Format a KeyValue + + A formatted keyvalue has the following parts: + {key}{delim}{value}[{delim}] + + Args: + buf (FlowBuffer): buffer to append the KeyValue to + kv (KeyValue): The KeyValue to print + style_obj (FlowStyle): The style object to use + highlighted (list): Optional; list of KeyValues to highlight + + Returns the number of printed characters. + """ + ret = 0 + key = kv.meta.kstring + is_highlighted = ( + key in [k.key for k in highlighted] if highlighted else False + ) + + key_style = style_obj.get_key_style(kv, is_highlighted) + buf.append_key(kv, key_style) # format value + ret += len(key) + + if not kv.meta.vstring: + return ret + + if kv.meta.delim not in ("\n", "\t", "\r", ""): + buf.append_delim(kv, style_obj.get_delim_style(is_highlighted)) + ret += len(kv.meta.delim) + + value_style = style_obj.get_value_style(kv, is_highlighted) + buf.append_value(kv, value_style) # format value + ret += len(kv.meta.vstring) + + if kv.meta.end_delim: + buf.append_end_delim(kv, style_obj.get_delim_style(is_highlighted)) + ret += len(kv.meta.end_delim) + + return ret + + +class FlowBuffer: + """A FlowBuffer is a base class for format buffers. + + Childs must implement the following methods: + append_key(self, kv, style) + append_value(self, kv, style) + append_delim(self, delim, style) + append_end_delim(self, delim, style) + append_extra(self, extra, style) + """ + + def append_key(self, kv, style): + """Append a key. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError + + def append_delim(self, kv, style): + """Append a delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError + + def append_end_delim(self, kv, style): + """Append an end delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError + + def append_value(self, kv, style): + """Append a value. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError + + def append_extra(self, extra, style): + """Append extra string. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError diff --git a/python/ovs/flowviz/main.py b/python/ovs/flowviz/main.py index 64b0e8a0a..723c71fa7 100644 --- a/python/ovs/flowviz/main.py +++ b/python/ovs/flowviz/main.py @@ -12,10 +12,30 @@ # See the License for the specific language governing permissions and # limitations under the License. +import configparser import click import os from ovs.flow.filter import OFFilter +from ovs.dirs import PKGDATADIR + +_default_config_file = "ovs-flowviz.conf" +_default_config_path = next( + ( + p + for p in [ + os.path.join( + os.getenv("HOME"), ".config", "ovs", _default_config_file + ), + os.path.join(PKGDATADIR, _default_config_file), + os.path.abspath( + os.path.join(os.path.dirname(__file__), _default_config_file) + ), + ] + if os.path.exists(p) + ), + "", +) class Options(dict): @@ -48,6 +68,20 @@ def validate_input(ctx, param, value): @click.group( context_settings=dict(help_option_names=["-h", "--help"]), ) +@click.option( + "-c", + "--config", + help="Use config file", + type=click.Path(), + default=_default_config_path, + show_default=True, +) +@click.option( + "--style", + help="Select style (defined in config file)", + default=None, + show_default=True, +) @click.option( "-i", "--input", @@ -69,8 +103,17 @@ def validate_input(ctx, param, value): type=str, show_default=False, ) +@click.option( + "-l", + "--highlight", + help="Highlight flows that match the filter expression." + "Run 'ovs-flowviz filter' for a detailed description of the filtering " + "syntax", + type=str, + show_default=False, +) @click.pass_context -def maincli(ctx, filename, filter): +def maincli(ctx, config, style, filename, filter, highlight): """ OpenvSwitch flow visualization utility. @@ -86,6 +129,19 @@ def maincli(ctx, filename, filter): except Exception as e: raise click.BadParameter("Wrong filter syntax: {}".format(e)) + if highlight: + try: + ctx.obj["highlight"] = OFFilter(highlight) + except Exception as e: + raise click.BadParameter("Wrong filter syntax: {}".format(e)) + + config_file = config or _default_config_path + parser = configparser.ConfigParser() + parser.read(config_file) + + ctx.obj["config"] = parser + ctx.obj["style"] = style + @maincli.command(hidden=True) @click.pass_context diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py index ed2f82065..a1cba0135 100644 --- a/python/ovs/flowviz/odp/cli.py +++ b/python/ovs/flowviz/odp/cli.py @@ -16,6 +16,7 @@ import click from ovs.flowviz.main import maincli from ovs.flowviz.process import ( + ConsoleProcessor, DatapathFactory, JSONProcessor, ) @@ -40,3 +41,27 @@ def json(opts): proc = JSONPrint(opts) proc.process() print(proc.json_string()) + + +class DPConsoleProcessor(DatapathFactory, ConsoleProcessor): + def __init__(self, opts, heat_map): + super().__init__(opts, heat_map) + + +@datapath.command() +@click.option( + "-h", + "--heat-map", + is_flag=True, + default=False, + show_default=True, + help="Create heat-map with packet and byte counters", +) +@click.pass_obj +def console(opts, heat_map): + """Print the flows in the console with some style.""" + proc = DPConsoleProcessor( + opts, heat_map=["packets", "bytes"] if heat_map else [] + ) + proc.process() + proc.print() diff --git a/python/ovs/flowviz/ofp/cli.py b/python/ovs/flowviz/ofp/cli.py index b9a2a8aad..a399dbd82 100644 --- a/python/ovs/flowviz/ofp/cli.py +++ b/python/ovs/flowviz/ofp/cli.py @@ -16,6 +16,7 @@ import click from ovs.flowviz.main import maincli from ovs.flowviz.process import ( + ConsoleProcessor, OpenFlowFactory, JSONProcessor, ) @@ -40,3 +41,28 @@ def json(opts): proc = JSONPrint(opts) proc.process() print(proc.json_string()) + + +class OFConsoleProcessor(OpenFlowFactory, ConsoleProcessor): + def __init__(self, opts, heat_map): + super().__init__(opts, heat_map) + + +@openflow.command() +@click.option( + "-h", + "--heat-map", + is_flag=True, + default=False, + show_default=True, + help="Create heat-map with packet and byte counters", +) +@click.pass_obj +def console(opts, heat_map): + """Print the flows in the console with some style.""" + proc = OFConsoleProcessor( + opts, + heat_map=["n_packets", "n_bytes"] if heat_map else [], + ) + proc.process() + proc.print() diff --git a/python/ovs/flowviz/process.py b/python/ovs/flowviz/process.py index 3e520e431..e0158de9f 100644 --- a/python/ovs/flowviz/process.py +++ b/python/ovs/flowviz/process.py @@ -20,6 +20,13 @@ from ovs.flow.decoders import FlowEncoder from ovs.flow.odp import ODPFlow from ovs.flow.ofp import OFPFlow +from ovs.flowviz.console import ( + ConsoleFormatter, + default_highlight, + heat_pallete, + file_header, +) + class FileProcessor(object): """Base class for file-based Flow processing. It is able to create flows @@ -134,21 +141,24 @@ class FileProcessor(object): self.end() -class DatapathFactory(): +class DatapathFactory: """A mixin class that creates Datapath flows.""" def create_flow(self, line, idx): # Skip strings commonly found in Datapath flow dumps. - if any(s in line for s in [ - "flow-dump from the main thread", - "flow-dump from pmd on core", - ]): + if any( + s in line + for s in [ + "flow-dump from the main thread", + "flow-dump from pmd on core", + ] + ): return None return ODPFlow(line, idx) -class OpenFlowFactory(): +class OpenFlowFactory: """A mixin class that creates OpenFlow flows.""" def create_flow(self, line, idx): @@ -190,3 +200,64 @@ class JSONProcessor(FileProcessor): indent=4, cls=FlowEncoder, ) + + +class ConsoleProcessor(FileProcessor): + """A generic Console Processor that prints flows into the console""" + + def __init__(self, opts, heat_map=[]): + super().__init__(opts) + self.heat_map = heat_map + self.console = ConsoleFormatter(opts) + if len(self.console.style) == 0 and self.opts.get("highlight"): + # Add some style to highlights or else they won't be seen. + self.console.style.set_default_value_style( + default_highlight(), True + ) + self.console.style.set_default_key_style(default_highlight(), True) + + self.flows = dict() # Dictionary of flow-lists, one per file. + self.min_max = dict() # Used for heat-map calculation. + + def start_file(self, name, filename): + self.flows_list = list() + if len(self.heat_map) > 0: + self.min = [-1] * len(self.heat_map) + self.max = [0] * len(self.heat_map) + + def stop_file(self, name, filename): + self.flows[name] = self.flows_list + if len(self.heat_map) > 0: + self.min_max[name] = (self.min, self.max) + + def process_flow(self, flow, name): + # Running calculation of min and max values for all the fields that + # take place in the heatmap. + for i, field in enumerate(self.heat_map): + val = flow.info.get(field) + if self.min[i] == -1 or val < self.min[i]: + self.min[i] = val + if val > self.max[i]: + self.max[i] = val + + self.flows_list.append(flow) + + def print(self): + for name, flows in self.flows.items(): + self.console.console.print("\n") + self.console.console.print(file_header(name)) + + if len(self.heat_map) > 0 and len(self.flows) > 0: + for i, field in enumerate(self.heat_map): + (min_val, max_val) = self.min_max[name][i] + self.console.style.set_value_style( + field, heat_pallete(min_val, max_val) + ) + + for flow in flows: + high = None + if self.opts.get("highlight"): + result = self.opts.get("highlight").evaluate(flow) + if result: + high = result.kv + self.console.print_flow(flow, high) diff --git a/python/setup.py b/python/setup.py index 4b9c751d2..76f9fc820 100644 --- a/python/setup.py +++ b/python/setup.py @@ -113,9 +113,11 @@ setup_args = dict( extras_require={':sys_platform == "win32"': ['pywin32 >= 1.0'], 'dns': ['unbound'], 'flow': flow_extras_require, - 'flowviz': [*flow_extras_require, 'click'], + 'flowviz': + [*flow_extras_require, 'click', 'rich'], }, scripts=["ovs/flowviz/ovs-flowviz"], + include_package_data=True, ) try: