From patchwork Tue Apr 9 07:06:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1921259 X-Patchwork-Delegate: i.maximets@samsung.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=AAUcvfLe; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VDH8d4y7Mz1yY8 for ; Tue, 9 Apr 2024 17:07:29 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id E56BC60E0D; Tue, 9 Apr 2024 07:07:27 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id PAjC5j9TE2cE; Tue, 9 Apr 2024 07:07:23 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 8459960D69 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=AAUcvfLe Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 8459960D69; Tue, 9 Apr 2024 07:07:18 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 28D11C007C; Tue, 9 Apr 2024 07:07:18 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1895AC007C for ; Tue, 9 Apr 2024 07:07:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 0605A40B49 for ; Tue, 9 Apr 2024 07:07:15 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id W2zrEaXvVnlP for ; Tue, 9 Apr 2024 07:07:09 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp2.osuosl.org D6D37414D5 Authentication-Results: smtp2.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org D6D37414D5 Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=AAUcvfLe Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id D6D37414D5 for ; Tue, 9 Apr 2024 07:07:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712646423; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yZRw74yP2HEK0PKR8Kp3xEXAjxW/pbyTXTV3FfSAIhM=; b=AAUcvfLeS527jl4zWd/QEE8/2oE19TiGidnizzXVJbzIRC9o82TaFajNNY6ptbSAPu43v1 rXrj/5ZaLiUV6JOjIoHWV8eNm5RTJyXoBUeHLiPANpQxZTjSxjHNyukvbiFvIS2ECuJpFE fA7Ybs1lqRBigoaUUJ6+tCfjdOQktMs= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-531-s2ou1PNpPby6FOV0Wu15QQ-1; Tue, 09 Apr 2024 03:07:01 -0400 X-MC-Unique: s2ou1PNpPby6FOV0Wu15QQ-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6A34D380673D for ; Tue, 9 Apr 2024 07:07:01 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.113]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9BD25444541; Tue, 9 Apr 2024 07:07:00 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 9 Apr 2024 09:06:28 +0200 Message-ID: <20240409070642.511747-11-amorenoz@redhat.com> In-Reply-To: <20240409070642.511747-1-amorenoz@redhat.com> References: <20240409070642.511747-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.10 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v3 10/12] python: ovs: flowviz: Add datapath graph format. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Graph view leverages the tree format (specially the tree-based filtering) and uses graphviz library to build a visual graph of the datapath in graphviz format. Conntrack zones are shown in random colors to help visualize connection tracking interdependencies. An html flag builds an HTML page with both the html flows and the graph (in svg) that enables navegation. Examples: $ ovs-appctl dpctl/dump-flows -m | ovs-flowviz datapath graph | dot -Tpng -o graph.png $ ovs-appctl dpctl/dump-flows -m | ovs-flowviz datapath graph --html > flows.html Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 1 + python/ovs/flowviz/odp/cli.py | 22 ++ python/ovs/flowviz/odp/graph.py | 418 ++++++++++++++++++++++++++++++++ python/ovs/flowviz/odp/tree.py | 18 +- python/setup.py | 2 +- 5 files changed, 457 insertions(+), 4 deletions(-) create mode 100644 python/ovs/flowviz/odp/graph.py diff --git a/python/automake.mk b/python/automake.mk index 44e9e08ab..9ef000480 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -71,6 +71,7 @@ ovs_flowviz = \ python/ovs/flowviz/main.py \ python/ovs/flowviz/odp/__init__.py \ python/ovs/flowviz/odp/cli.py \ + python/ovs/flowviz/odp/graph.py \ python/ovs/flowviz/odp/html.py \ python/ovs/flowviz/odp/tree.py \ python/ovs/flowviz/ofp/__init__.py \ diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py index dd64fdc65..94fdb80eb 100644 --- a/python/ovs/flowviz/odp/cli.py +++ b/python/ovs/flowviz/odp/cli.py @@ -14,6 +14,7 @@ import click from ovs.flowviz.main import maincli +from ovs.flowviz.odp.graph import GraphProcessor from ovs.flowviz.odp.html import HTMLTreeProcessor from ovs.flowviz.odp.tree import ConsoleTreeProcessor from ovs.flowviz.process import ( @@ -92,3 +93,24 @@ def html(opts): processor = HTMLTreeProcessor(opts) processor.process() processor.print() + + +@datapath.command() +@click.option( + "-h", + "--html", + is_flag=True, + default=False, + show_default=True, + help="Output an html file containing the graph", +) +@click.pass_obj +def graph(opts, html): + """Print the flows in an graphviz (.dot) format showing the relationship + of recirc_ids.""" + if len(opts.get("filename")) > 1: + raise click.BadParameter("Graph format only supports one input file") + + processor = GraphProcessor(opts) + processor.process() + processor.print(html) diff --git a/python/ovs/flowviz/odp/graph.py b/python/ovs/flowviz/odp/graph.py new file mode 100644 index 000000000..b26551e67 --- /dev/null +++ b/python/ovs/flowviz/odp/graph.py @@ -0,0 +1,418 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""" Defines a Datapath Graph using graphviz. """ +import colorsys +import graphviz +import random + +from ovs.flowviz.odp.html import HTMLTree +from ovs.flowviz.odp.tree import FlowTree +from ovs.flowviz.process import DatapathFactory, FileProcessor + + +class GraphProcessor(DatapathFactory, FileProcessor): + def __init__(self, opts): + super().__init__(opts) + + def start_file(self, name, filename): + self.tree = FlowTree() + + def process_flow(self, flow, name): + self.tree.add(flow) + + def process(self): + super().process(False) + + def print(self, html): + flows = {} + + # Tree traverse callback + def add_flow(elem, _): + if elem.is_root: + return + rid = elem.flow.match.get("recirc_id") or 0 + if not flows.get(rid): + flows[rid] = set() + flows[rid].add(elem.flow) + + self.tree.build() + if self.opts.get("filter"): + self.tree.filter(self.opts.get("filter")) + self.tree.traverse(add_flow) + + if len(flows) == 0: + return + + dpg = DatapathGraph(flows) + if not html: + print(dpg.source()) + return + + html_obj = "" + html_obj += "

Flow Graph

" + html_obj += "
" + svg = dpg.pipe(format="svg") + html_obj += svg.decode("utf-8") + html_obj += "
" + html_tree = HTMLTree("graph", self.opts, flows) + html_tree.build() + html_obj += html_tree.render() + + print(html_obj) + + +class DatapathGraph: + """A DatapathGraph is a class that renders a set of datapath flows into + graphviz graphs. + + Args: + flows(dict[int, list(Flow)]): Dictionary of lists of flows indexed by + recirc_id + """ + + ct_styles = {} + node_styles = { + "default": { + "style": {}, + "desc": "Default", + }, + "action_and_match": { + "style": {"color": "#ff00ff"}, + "desc": "Flow uses CT as match and action", + }, + "match": { + "style": {"color": "#0000ff"}, + "desc": "Flow uses CT only to match", + }, + "action": { + "style": {"color": "#ff0000"}, + "desc": "Flow uses CT only as action", + }, + } + + def __init__(self, flows): + self._flows = flows + + self._output_nodes = [] + self._graph = graphviz.Digraph( + "DP flows", node_attr={"shape": "rectangle"} + ) + self._graph.attr(compound="true") + self._graph.attr(rankdir="LR") + self._graph.attr(ranksep="3") + + self._populate_graph() + + def source(self): + """Return the graphviz source representation of the graph.""" + return self._graph.source + + def pipe(self, *args, **kwargs): + """Output the graph based on arguments given to graphviz.pipe.""" + return self._graph.pipe(*args, **kwargs) + + @classmethod + def recirc_cluster_name(cls, recirc_id): + """Name of the recirculation cluster.""" + return "cluster_recirc_{}".format(hex(recirc_id)) + + @classmethod + def inport_cluster_name(cls, inport): + """Name of the input port cluster.""" + return "cluster_inport_{}".format(inport) + + @classmethod + def invis_node_name(cls, cluster_name): + """Name of the invisible node.""" + return "invis_{}".format(cluster_name) + + @classmethod + def output_node_name(cls, port): + """Name of the ouput node.""" + return "output_{}".format(port) + + def _flow_node(self, flow, name): + """Returns the dictionary of attributes of a graphviz node that + represents the flow with a given name. + """ + summary = "Line: {} \n".format(flow.id) + summary += "\n".join( + [ + flow.section("info").string, + ",".join(flow.match.keys()), + "actions: " + + ",".join(list(a.keys())[0] for a in flow.actions), + ] + ) + + has_ct_match = flow.match.get("ct_state", "0/0") != "0/0" + has_ct_action = bool( + next( + filter(lambda x: x.key in ["ct", "ct_clear"], flow.actions_kv), + None, + ) + ) + + if has_ct_action: + if has_ct_match: + style = "action_and_match" + else: + style = "action" + elif has_ct_match: + style = "match" + else: + style = "default" + + style = self.node_styles.get(style, {}) + + return { + "name": name, + "label": summary, + "tooltip": flow.orig, + "_attributes": style.get("style", {}), + "fontsize": "10", + "nojustify": "true", + "URL": "#flow_{}".format(flow.id), + } + + def _create_recirc_cluster(self, recirc): + """Process a recirculation id, creating its cluster.""" + cluster_name = self.recirc_cluster_name(recirc) + label = "recirc x0{:0x}".format(recirc) + + cluster = self._graph.subgraph(name=cluster_name, comment=label) + with cluster as sg: + sg.attr(rankdir="TB") + sg.attr(ranksep="0.02") + sg.attr(label=label) + sg.attr(margin="5") + self._add_flows_to_graph(sg, self._flows[recirc]) + + self.processed_recircs.append(recirc) + + def _add_flows_to_graph(self, graph, flows): + # Create an invisible node and an edge to the first flow so that + # it ends up at the top of the cluster. + invis = self.invis_node_name(graph.name) + graph.node(invis) + graph.node( + invis, + color="white", + len="0", + shape="point", + width="0", + height="0", + ) + first = True + for flow in flows: + name = "Flow_{}".format(flow.id) + graph.node(**self._flow_node(flow, name)) + if first: + with graph.subgraph() as c: + c.attr(rank="same") + c.edge(name, invis, style="invis") + first = False + # determine next hop based on actions + self._set_next_node_from_actions(name, flow.actions) + + def set_next_node_from_actions(self, name, actions): + """Determine the next nodes based on action list and add edges to + them. + """ + if not self._set_next_node_from_actions(self, name, actions): + # Add to a generic "End" if no other action was detected + self._graph.edge(name, "end") + + def _set_next_node_from_actions(self, name, actions): + created = False + for action in actions: + key, value = next(iter(action.items())) + if key == "check_pkt_len": + created |= self._set_next_node_from_actions( + name, value.get("gt") + ) + created |= self._set_next_node_from_actions( + name, value.get("le") + ) + elif key == "sample": + created |= self._set_next_node_from_actions( + name, value.get("actions") + ) + elif key == "clone": + created |= self._set_next_node_from_actions( + name, value.get("actions") + ) + else: + created |= self._set_next_node_action(name, key, value) + return created + + def _set_next_node_action(self, name, action_name, action_obj): + """Based on the action object, set the next node.""" + if action_name == "recirc": + # If the targer recirculation cluster has not yet been created, + # do it now. + if action_obj not in self.processed_recircs: + self._create_recirc_cluster(action_obj) + + cname = self.recirc_cluster_name(action_obj) + self._graph.edge( + name, + self.invis_node_name(cname), + lhead=cname, + _attributes={"weight": "20"}, + ) + return True + elif action_name == "output": + port = action_obj.get("port") + if port not in self._output_nodes: + self._output_nodes.append(port) + self._graph.edge( + name, self.output_node_name(port), _attributes={"weight": "1"} + ) + return True + elif action_name in ["drop", "userspace", "controller"]: + if action_name not in self._output_nodes: + self._output_nodes.append(action_name) + self._graph.edge(name, action_name, _attributes={"weight": "1"}) + return True + elif action_name == "ct": + zone = action_obj.get("zone", 0) + node_name = "CT zone {}".format(action_obj.get("zone", "default")) + if zone not in self.ct_styles: + # Pick a random (highly saturated) color. + (r, g, b) = colorsys.hsv_to_rgb(random.random(), 1, 1) + color = "#%02x%02x%02x" % ( + int(r * 255), + int(g * 255), + int(b * 255), + ) + self.ct_styles[zone] = color + self._graph.node(node_name, color=color) + + color = self.ct_styles[zone] + self._graph.edge(name, node_name, style="dashed", color=color) + # test + name = node_name + return True + return False + + def _populate_graph(self): + """Populate the the internal graph.""" + self.processed_recircs = [] + + # Create a subcluster for each input port and one for flows that don't + # have in_port() for which we create a dummy inport. + flows_per_inport = {} + free_flows = [] + + for flow in self._flows.get(0): + port = flow.match.get("in_port") + if port: + if not flows_per_inport.get(port): + flows_per_inport[port] = list() + flows_per_inport[port].append(flow) + else: + free_flows.append(flow) + + # It's rare to find flows without input_port match but let's add them + # nevertheless. + if free_flows: + self._graph.edge( + "start", + self.invis_node_name(self.recirc_cluster_name(0)), + lhead=self.recirc_cluster_name(0), + ) + self._graph.node("no_port", shape="Mdiamond") + + # Recirc_clusters are created recursively when an edge is found to + # them. + # Process recirc(0) which is split by input port. + for inport, flows in flows_per_inport.items(): + # Build a subgraph per input port + cluster_name = self.inport_cluster_name(inport) + label = "recirc 0; input port: {}".format(inport) + + with self._graph.subgraph( + name=cluster_name, comment=label + ) as per_port: + per_port.attr(rankdir="TB") + per_port.attr(ranksep="0.02") + per_port.attr(margin="5") + per_port.attr(label=label) + self._add_flows_to_graph(per_port, flows_per_inport[inport]) + + # Create an input node that points to each input subgraph + # They are all inside an anonymous subgraph so that they can be + # alligned. + with self._graph.subgraph() as s: + s.attr(rank="same") + for inport in flows_per_inport: + # Make an Input node point to each subgraph + node_name = "input_{}".format(inport) + cluster_name = self.inport_cluster_name(inport) + s.node( + node_name, + shape="Mdiamond", + label="input port {}".format(inport), + ) + self._graph.edge( + node_name, + self.invis_node_name(cluster_name), + lhead=cluster_name, + _attributes={"weight": "20"}, + ) + + # Create the output nodes in a subgraph so that they are alligned + with self._graph.subgraph() as s: + for port in self._output_nodes: + s.attr(rank="same") + if port == "drop": + s.node( + "drop", + shape="Msquare", + color="red", + label="DROP", + rank="sink", + ) + elif port == "controller": + s.node( + "controller", + shape="Msquare", + color="blue", + label="CONTROLLER", + rank="sink", + ) + elif port == "userspace": + s.node( + "userspace", + shape="Msquare", + color="blue", + label="CONTROLLER", + rank="sink", + ) + else: + s.node( + self.output_node_name(port), + shape="Msquare", + color="green", + label="Port {}".format(port), + rank="sink", + ) + + # Print style legend + with self._graph.subgraph(name="cluster_legend") as s: + s.attr(label="Legend") + for style in self.node_styles.values(): + s.node(name=style.get("desc"), _attributes=style.get("style")) diff --git a/python/ovs/flowviz/odp/tree.py b/python/ovs/flowviz/odp/tree.py index d249e7d6d..72be1c427 100644 --- a/python/ovs/flowviz/odp/tree.py +++ b/python/ovs/flowviz/odp/tree.py @@ -76,7 +76,8 @@ class FlowTree: on recirculation ids. Args: - flows (list[ODPFlow]): Optional, initial list of flows + flows (list[ODPFlow]): Optional, initial list of flows or dictionary of + flows indexed by recirc_id root (TreeElem): Optional, root of the tree. """ @@ -84,8 +85,15 @@ class FlowTree: self._flows = {} self.root = root if flows: - for flow in flows: - self.add(flow) + if isinstance(flows, dict): + self._flows = flows + elif isinstance(flows, list): + for flow in flows: + self.add(flow) + else: + raise Exception( + "flows in wrong format: {}".format(type(flows)) + ) def add(self, flow): """Add a flow""" @@ -192,6 +200,10 @@ class FlowTree: for elem in to_remove: self.root.children.remove(elem) + def all(self): + """Return all the flows in a dictionary by recirc_id.""" + return self._flows + class ConsoleTreeProcessor(DatapathFactory, FileProcessor): def __init__(self, opts): diff --git a/python/setup.py b/python/setup.py index c734f68f3..018a75eb0 100644 --- a/python/setup.py +++ b/python/setup.py @@ -114,7 +114,7 @@ setup_args = dict( 'dns': ['unbound'], 'flow': flow_extras_require, 'flowviz': - [*flow_extras_require, 'click', 'rich'], + [*flow_extras_require, 'click', 'rich', 'graphviz'], }, scripts=["ovs/flowviz/ovs-flowviz"], data_files=["ovs/flowviz/ovs-flowviz.conf"],