From patchwork Mon Feb 19 08:14:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1900783 X-Patchwork-Delegate: horms@verge.net.au Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Ud2HhakN; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Tdb1w0cybz23cl for ; Mon, 19 Feb 2024 19:15:16 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 2E5CE40AF9; Mon, 19 Feb 2024 08:15:13 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IocG_jLK5x4u; Mon, 19 Feb 2024 08:15:09 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 4FD4640AC2 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Ud2HhakN Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 4FD4640AC2; Mon, 19 Feb 2024 08:15:00 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B0F06C0DD6; Mon, 19 Feb 2024 08:14:59 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1A081C0DD7 for ; Mon, 19 Feb 2024 08:14:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id E1F7060ADA for ; Mon, 19 Feb 2024 08:14:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bpo3zl1e-wdT for ; Mon, 19 Feb 2024 08:14:50 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org 49BAB60AE6 Authentication-Results: smtp3.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 49BAB60AE6 Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Ud2HhakN Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id 49BAB60AE6 for ; Mon, 19 Feb 2024 08:14:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1708330486; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wCs6yXcgImnAfxNljKb9xGDNAn7D51AtIYasB6yIG7o=; b=Ud2HhakN2A5luTNQAVbgdJLVpbZRKr/ZElOVUSBqbcwe4OtoE1rG0mckstDui5KfS263Mz 3GSrghNJ98wD6UlavA5e8n97/XaRjVKKtpB91gN4QBaxCfqaQ7ewBbzm7D7ABajkE21dOp Vf9CVA0jB+ojlivHBPQLmx1wO3cDgyw= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-653-yMf8hWboM-uDGxWqdpLwoA-1; Mon, 19 Feb 2024 03:14:44 -0500 X-MC-Unique: yMf8hWboM-uDGxWqdpLwoA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EFD373C29A67 for ; Mon, 19 Feb 2024 08:14:43 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4A0708077; Mon, 19 Feb 2024 08:14:43 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 19 Feb 2024 09:14:25 +0100 Message-ID: <20240219081431.2887060-11-amorenoz@redhat.com> In-Reply-To: <20240219081431.2887060-1-amorenoz@redhat.com> References: <20240219081431.2887060-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 10/12] python: ovs: flowviz: Add datapath graph format. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Graph view leverages the tree format (specially the tree-based filtering) and uses graphviz library to build a visual graph of the datapath in graphviz format. Conntrack zones are shown in random colors to help visualize connection tracking interdependencies. An html flag builds an HTML page with both the html flows and the graph (in svg) that enables navegation. Examples: $ ovs-appctl dpctl/dump-flows -m | ovs-flowviz datapath graph | dot -Tpng -o graph.png $ ovs-appctl dpctl/dump-flows -m | ovs-flowviz datapath graph --html > flows.html Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno Acked-by: Eelco Chaudron --- python/automake.mk | 1 + python/ovs/flowviz/odp/cli.py | 22 ++ python/ovs/flowviz/odp/graph.py | 418 ++++++++++++++++++++++++++++++++ python/ovs/flowviz/odp/tree.py | 18 +- python/setup.py | 2 +- 5 files changed, 457 insertions(+), 4 deletions(-) create mode 100644 python/ovs/flowviz/odp/graph.py diff --git a/python/automake.mk b/python/automake.mk index 44e9e08ab..9ef000480 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -71,6 +71,7 @@ ovs_flowviz = \ python/ovs/flowviz/main.py \ python/ovs/flowviz/odp/__init__.py \ python/ovs/flowviz/odp/cli.py \ + python/ovs/flowviz/odp/graph.py \ python/ovs/flowviz/odp/html.py \ python/ovs/flowviz/odp/tree.py \ python/ovs/flowviz/ofp/__init__.py \ diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py index 059dd708e..3e470ff1e 100644 --- a/python/ovs/flowviz/odp/cli.py +++ b/python/ovs/flowviz/odp/cli.py @@ -14,6 +14,7 @@ import click from ovs.flowviz.main import maincli +from ovs.flowviz.odp.graph import GraphProcessor from ovs.flowviz.odp.html import HTMLTreeProcessor from ovs.flowviz.odp.tree import ConsoleTreeProcessor from ovs.flowviz.process import ( @@ -92,3 +93,24 @@ def html(opts): processor = HTMLTreeProcessor(opts) processor.process() processor.print() + + +@datapath.command() +@click.option( + "-h", + "--html", + is_flag=True, + default=False, + show_default=True, + help="Output an html file containing the graph", +) +@click.pass_obj +def graph(opts, html): + """Print the flows in an graphviz (.dot) format showing the relationship + of recirc_ids.""" + if len(opts.get("filename")) > 1: + raise click.BadParameter("Graph format only supports one input file") + + processor = GraphProcessor(opts) + processor.process() + processor.print(html) diff --git a/python/ovs/flowviz/odp/graph.py b/python/ovs/flowviz/odp/graph.py new file mode 100644 index 000000000..b26551e67 --- /dev/null +++ b/python/ovs/flowviz/odp/graph.py @@ -0,0 +1,418 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""" Defines a Datapath Graph using graphviz. """ +import colorsys +import graphviz +import random + +from ovs.flowviz.odp.html import HTMLTree +from ovs.flowviz.odp.tree import FlowTree +from ovs.flowviz.process import DatapathFactory, FileProcessor + + +class GraphProcessor(DatapathFactory, FileProcessor): + def __init__(self, opts): + super().__init__(opts) + + def start_file(self, name, filename): + self.tree = FlowTree() + + def process_flow(self, flow, name): + self.tree.add(flow) + + def process(self): + super().process(False) + + def print(self, html): + flows = {} + + # Tree traverse callback + def add_flow(elem, _): + if elem.is_root: + return + rid = elem.flow.match.get("recirc_id") or 0 + if not flows.get(rid): + flows[rid] = set() + flows[rid].add(elem.flow) + + self.tree.build() + if self.opts.get("filter"): + self.tree.filter(self.opts.get("filter")) + self.tree.traverse(add_flow) + + if len(flows) == 0: + return + + dpg = DatapathGraph(flows) + if not html: + print(dpg.source()) + return + + html_obj = "" + html_obj += "

Flow Graph

" + html_obj += "
" + svg = dpg.pipe(format="svg") + html_obj += svg.decode("utf-8") + html_obj += "
" + html_tree = HTMLTree("graph", self.opts, flows) + html_tree.build() + html_obj += html_tree.render() + + print(html_obj) + + +class DatapathGraph: + """A DatapathGraph is a class that renders a set of datapath flows into + graphviz graphs. + + Args: + flows(dict[int, list(Flow)]): Dictionary of lists of flows indexed by + recirc_id + """ + + ct_styles = {} + node_styles = { + "default": { + "style": {}, + "desc": "Default", + }, + "action_and_match": { + "style": {"color": "#ff00ff"}, + "desc": "Flow uses CT as match and action", + }, + "match": { + "style": {"color": "#0000ff"}, + "desc": "Flow uses CT only to match", + }, + "action": { + "style": {"color": "#ff0000"}, + "desc": "Flow uses CT only as action", + }, + } + + def __init__(self, flows): + self._flows = flows + + self._output_nodes = [] + self._graph = graphviz.Digraph( + "DP flows", node_attr={"shape": "rectangle"} + ) + self._graph.attr(compound="true") + self._graph.attr(rankdir="LR") + self._graph.attr(ranksep="3") + + self._populate_graph() + + def source(self): + """Return the graphviz source representation of the graph.""" + return self._graph.source + + def pipe(self, *args, **kwargs): + """Output the graph based on arguments given to graphviz.pipe.""" + return self._graph.pipe(*args, **kwargs) + + @classmethod + def recirc_cluster_name(cls, recirc_id): + """Name of the recirculation cluster.""" + return "cluster_recirc_{}".format(hex(recirc_id)) + + @classmethod + def inport_cluster_name(cls, inport): + """Name of the input port cluster.""" + return "cluster_inport_{}".format(inport) + + @classmethod + def invis_node_name(cls, cluster_name): + """Name of the invisible node.""" + return "invis_{}".format(cluster_name) + + @classmethod + def output_node_name(cls, port): + """Name of the ouput node.""" + return "output_{}".format(port) + + def _flow_node(self, flow, name): + """Returns the dictionary of attributes of a graphviz node that + represents the flow with a given name. + """ + summary = "Line: {} \n".format(flow.id) + summary += "\n".join( + [ + flow.section("info").string, + ",".join(flow.match.keys()), + "actions: " + + ",".join(list(a.keys())[0] for a in flow.actions), + ] + ) + + has_ct_match = flow.match.get("ct_state", "0/0") != "0/0" + has_ct_action = bool( + next( + filter(lambda x: x.key in ["ct", "ct_clear"], flow.actions_kv), + None, + ) + ) + + if has_ct_action: + if has_ct_match: + style = "action_and_match" + else: + style = "action" + elif has_ct_match: + style = "match" + else: + style = "default" + + style = self.node_styles.get(style, {}) + + return { + "name": name, + "label": summary, + "tooltip": flow.orig, + "_attributes": style.get("style", {}), + "fontsize": "10", + "nojustify": "true", + "URL": "#flow_{}".format(flow.id), + } + + def _create_recirc_cluster(self, recirc): + """Process a recirculation id, creating its cluster.""" + cluster_name = self.recirc_cluster_name(recirc) + label = "recirc x0{:0x}".format(recirc) + + cluster = self._graph.subgraph(name=cluster_name, comment=label) + with cluster as sg: + sg.attr(rankdir="TB") + sg.attr(ranksep="0.02") + sg.attr(label=label) + sg.attr(margin="5") + self._add_flows_to_graph(sg, self._flows[recirc]) + + self.processed_recircs.append(recirc) + + def _add_flows_to_graph(self, graph, flows): + # Create an invisible node and an edge to the first flow so that + # it ends up at the top of the cluster. + invis = self.invis_node_name(graph.name) + graph.node(invis) + graph.node( + invis, + color="white", + len="0", + shape="point", + width="0", + height="0", + ) + first = True + for flow in flows: + name = "Flow_{}".format(flow.id) + graph.node(**self._flow_node(flow, name)) + if first: + with graph.subgraph() as c: + c.attr(rank="same") + c.edge(name, invis, style="invis") + first = False + # determine next hop based on actions + self._set_next_node_from_actions(name, flow.actions) + + def set_next_node_from_actions(self, name, actions): + """Determine the next nodes based on action list and add edges to + them. + """ + if not self._set_next_node_from_actions(self, name, actions): + # Add to a generic "End" if no other action was detected + self._graph.edge(name, "end") + + def _set_next_node_from_actions(self, name, actions): + created = False + for action in actions: + key, value = next(iter(action.items())) + if key == "check_pkt_len": + created |= self._set_next_node_from_actions( + name, value.get("gt") + ) + created |= self._set_next_node_from_actions( + name, value.get("le") + ) + elif key == "sample": + created |= self._set_next_node_from_actions( + name, value.get("actions") + ) + elif key == "clone": + created |= self._set_next_node_from_actions( + name, value.get("actions") + ) + else: + created |= self._set_next_node_action(name, key, value) + return created + + def _set_next_node_action(self, name, action_name, action_obj): + """Based on the action object, set the next node.""" + if action_name == "recirc": + # If the targer recirculation cluster has not yet been created, + # do it now. + if action_obj not in self.processed_recircs: + self._create_recirc_cluster(action_obj) + + cname = self.recirc_cluster_name(action_obj) + self._graph.edge( + name, + self.invis_node_name(cname), + lhead=cname, + _attributes={"weight": "20"}, + ) + return True + elif action_name == "output": + port = action_obj.get("port") + if port not in self._output_nodes: + self._output_nodes.append(port) + self._graph.edge( + name, self.output_node_name(port), _attributes={"weight": "1"} + ) + return True + elif action_name in ["drop", "userspace", "controller"]: + if action_name not in self._output_nodes: + self._output_nodes.append(action_name) + self._graph.edge(name, action_name, _attributes={"weight": "1"}) + return True + elif action_name == "ct": + zone = action_obj.get("zone", 0) + node_name = "CT zone {}".format(action_obj.get("zone", "default")) + if zone not in self.ct_styles: + # Pick a random (highly saturated) color. + (r, g, b) = colorsys.hsv_to_rgb(random.random(), 1, 1) + color = "#%02x%02x%02x" % ( + int(r * 255), + int(g * 255), + int(b * 255), + ) + self.ct_styles[zone] = color + self._graph.node(node_name, color=color) + + color = self.ct_styles[zone] + self._graph.edge(name, node_name, style="dashed", color=color) + # test + name = node_name + return True + return False + + def _populate_graph(self): + """Populate the the internal graph.""" + self.processed_recircs = [] + + # Create a subcluster for each input port and one for flows that don't + # have in_port() for which we create a dummy inport. + flows_per_inport = {} + free_flows = [] + + for flow in self._flows.get(0): + port = flow.match.get("in_port") + if port: + if not flows_per_inport.get(port): + flows_per_inport[port] = list() + flows_per_inport[port].append(flow) + else: + free_flows.append(flow) + + # It's rare to find flows without input_port match but let's add them + # nevertheless. + if free_flows: + self._graph.edge( + "start", + self.invis_node_name(self.recirc_cluster_name(0)), + lhead=self.recirc_cluster_name(0), + ) + self._graph.node("no_port", shape="Mdiamond") + + # Recirc_clusters are created recursively when an edge is found to + # them. + # Process recirc(0) which is split by input port. + for inport, flows in flows_per_inport.items(): + # Build a subgraph per input port + cluster_name = self.inport_cluster_name(inport) + label = "recirc 0; input port: {}".format(inport) + + with self._graph.subgraph( + name=cluster_name, comment=label + ) as per_port: + per_port.attr(rankdir="TB") + per_port.attr(ranksep="0.02") + per_port.attr(margin="5") + per_port.attr(label=label) + self._add_flows_to_graph(per_port, flows_per_inport[inport]) + + # Create an input node that points to each input subgraph + # They are all inside an anonymous subgraph so that they can be + # alligned. + with self._graph.subgraph() as s: + s.attr(rank="same") + for inport in flows_per_inport: + # Make an Input node point to each subgraph + node_name = "input_{}".format(inport) + cluster_name = self.inport_cluster_name(inport) + s.node( + node_name, + shape="Mdiamond", + label="input port {}".format(inport), + ) + self._graph.edge( + node_name, + self.invis_node_name(cluster_name), + lhead=cluster_name, + _attributes={"weight": "20"}, + ) + + # Create the output nodes in a subgraph so that they are alligned + with self._graph.subgraph() as s: + for port in self._output_nodes: + s.attr(rank="same") + if port == "drop": + s.node( + "drop", + shape="Msquare", + color="red", + label="DROP", + rank="sink", + ) + elif port == "controller": + s.node( + "controller", + shape="Msquare", + color="blue", + label="CONTROLLER", + rank="sink", + ) + elif port == "userspace": + s.node( + "userspace", + shape="Msquare", + color="blue", + label="CONTROLLER", + rank="sink", + ) + else: + s.node( + self.output_node_name(port), + shape="Msquare", + color="green", + label="Port {}".format(port), + rank="sink", + ) + + # Print style legend + with self._graph.subgraph(name="cluster_legend") as s: + s.attr(label="Legend") + for style in self.node_styles.values(): + s.node(name=style.get("desc"), _attributes=style.get("style")) diff --git a/python/ovs/flowviz/odp/tree.py b/python/ovs/flowviz/odp/tree.py index d249e7d6d..72be1c427 100644 --- a/python/ovs/flowviz/odp/tree.py +++ b/python/ovs/flowviz/odp/tree.py @@ -76,7 +76,8 @@ class FlowTree: on recirculation ids. Args: - flows (list[ODPFlow]): Optional, initial list of flows + flows (list[ODPFlow]): Optional, initial list of flows or dictionary of + flows indexed by recirc_id root (TreeElem): Optional, root of the tree. """ @@ -84,8 +85,15 @@ class FlowTree: self._flows = {} self.root = root if flows: - for flow in flows: - self.add(flow) + if isinstance(flows, dict): + self._flows = flows + elif isinstance(flows, list): + for flow in flows: + self.add(flow) + else: + raise Exception( + "flows in wrong format: {}".format(type(flows)) + ) def add(self, flow): """Add a flow""" @@ -192,6 +200,10 @@ class FlowTree: for elem in to_remove: self.root.children.remove(elem) + def all(self): + """Return all the flows in a dictionary by recirc_id.""" + return self._flows + class ConsoleTreeProcessor(DatapathFactory, FileProcessor): def __init__(self, opts): diff --git a/python/setup.py b/python/setup.py index c734f68f3..018a75eb0 100644 --- a/python/setup.py +++ b/python/setup.py @@ -114,7 +114,7 @@ setup_args = dict( 'dns': ['unbound'], 'flow': flow_extras_require, 'flowviz': - [*flow_extras_require, 'click', 'rich'], + [*flow_extras_require, 'click', 'rich', 'graphviz'], }, scripts=["ovs/flowviz/ovs-flowviz"], data_files=["ovs/flowviz/ovs-flowviz.conf"],