From patchwork Tue Apr 9 07:06:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1921256 X-Patchwork-Delegate: i.maximets@samsung.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=NkD5bmjf; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VDH8Q50pYz1yY8 for ; Tue, 9 Apr 2024 17:07:18 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 9056D60BF4; Tue, 9 Apr 2024 07:07:14 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 6AEQQ3B7OqGY; Tue, 9 Apr 2024 07:07:11 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 8EB69607A7 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=NkD5bmjf Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 8EB69607A7; Tue, 9 Apr 2024 07:07:09 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 57180C007C; Tue, 9 Apr 2024 07:07:09 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 29ED8C0DCE for ; Tue, 9 Apr 2024 07:07:08 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 7B7414098E for ; Tue, 9 Apr 2024 07:07:07 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id zJY7wNKiFkmO for ; Tue, 9 Apr 2024 07:07:02 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org 16A124091D Authentication-Results: smtp4.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 16A124091D Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=NkD5bmjf Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 16A124091D for ; Tue, 9 Apr 2024 07:07:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712646420; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1VGvc77fhbk+ymiXHsfoiGKw1+5rb/X9c7mY9Qvi8ME=; b=NkD5bmjff3Tsmn7rZ9SrekMeBHFfO4BTHk7jnsm1rggpudFSU2GeJoWIVLnMI1lunBclI1 cx73uooIuMrZQBYkKjyf95cXiXGI3wTCMlNWLggZPqd10Rs9y2DXIerHeHuLUDN2En1pj0 dpzspGuqGRmiIJWJxgoy/oritJdZ3EE= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-567-0ixbkJkUNDWU5Wfg16Qneg-1; Tue, 09 Apr 2024 03:06:58 -0400 X-MC-Unique: 0ixbkJkUNDWU5Wfg16Qneg-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7DEE529AA3B3 for ; Tue, 9 Apr 2024 07:06:58 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.113]) by smtp.corp.redhat.com (Postfix) with ESMTP id C3E5A444565; Tue, 9 Apr 2024 07:06:57 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 9 Apr 2024 09:06:25 +0200 Message-ID: <20240409070642.511747-8-amorenoz@redhat.com> In-Reply-To: <20240409070642.511747-1-amorenoz@redhat.com> References: <20240409070642.511747-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.10 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v3 07/12] python: ovs: flowviz: Add OpenFlow logical view. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This view is interesting for debugging the logical pipeline. It arranges the flows in "logical" groups (not to be confused with OVN's Logical_Flows). A logical group of flows is a set of flows that: - Have the same table number and priority - Match on the same fields (regardless of the value they match against) - Have the same actions, regardless of the arguments for those actions, except for output and recirc, for which arguments do care. Optionally, the cookie can also be force to be unique for the logical group. By doing so, we can extend the information we show by querying an external OVN database and running "ovn-detrace" on each cookie. The result is a compact list of flow groups with interlieved OVN information. Furthermore, if connected to an OVN database, we can apply an OVN regexp filter. Examples: $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow logic $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow logic -s -h $ export OVN_NB_DB=... $ export OVN_SB_DB=... $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow logic -d $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow logic -d --ovn-filter="acl.*icmp4" Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 4 +- python/ovs/flowviz/ofp/cli.py | 113 ++++++++++++ python/ovs/flowviz/ofp/logic.py | 303 ++++++++++++++++++++++++++++++++ 3 files changed, 418 insertions(+), 2 deletions(-) create mode 100644 python/ovs/flowviz/ofp/logic.py diff --git a/python/automake.mk b/python/automake.mk index b3fef9bed..449daf023 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -74,12 +74,12 @@ ovs_flowviz = \ python/ovs/flowviz/odp/tree.py \ python/ovs/flowviz/ofp/__init__.py \ python/ovs/flowviz/ofp/cli.py \ + python/ovs/flowviz/ofp/logic.py \ python/ovs/flowviz/ofp/html.py \ python/ovs/flowviz/ovs-flowviz \ python/ovs/flowviz/process.py -# These python files are used at build time but not runtime, -# so they are not installed. +# These python files are used at build time but not runtime, so they are not installed. EXTRA_DIST += \ python/ovs_build_helpers/__init__.py \ python/ovs_build_helpers/extract_ofp_fields.py \ diff --git a/python/ovs/flowviz/ofp/cli.py b/python/ovs/flowviz/ofp/cli.py index 2cd8e1c89..51428ede0 100644 --- a/python/ovs/flowviz/ofp/cli.py +++ b/python/ovs/flowviz/ofp/cli.py @@ -12,10 +12,13 @@ # See the License for the specific language governing permissions and # limitations under the License. +import os + import click from ovs.flowviz.main import maincli from ovs.flowviz.ofp.html import HTMLProcessor +from ovs.flowviz.ofp.logic import LogicFlowProcessor from ovs.flowviz.process import ( ConsoleProcessor, OpenFlowFactory, @@ -69,6 +72,116 @@ def console(opts, heat_map): proc.print() +def ovn_detrace_callback(ctx, param, value): + """click callback to add detrace information to config object and + set general ovn-detrace flag to True + """ + ctx.obj[param.name] = value + if value != param.default: + ctx.obj["ovn_detrace_flag"] = True + return value + + +@openflow.command() +@click.option( + "-d", + "--ovn-detrace", + "ovn_detrace_flag", + is_flag=True, + show_default=True, + help="Use ovn-detrace to extract cookie information (implies '-c')", +) +@click.option( + "--ovn-detrace-path", + default="/usr/bin", + type=click.Path(), + help="Use an alternative path to where ovn_detrace.py is located. " + "Instead of using this option you can just set PYTHONPATH accordingly.", + show_default=True, + callback=ovn_detrace_callback, +) +@click.option( + "--ovnnb-db", + default=os.getenv("OVN_NB_DB") or "unix:/var/run/ovn/ovnnb_db.sock", + help="Specify the OVN NB database string (implies -d). " + "If the OVN_NB_DB environment variable is set, it's used as default. " + "Otherwise, the default is unix:/var/run/ovn/ovnnb_db.sock", + callback=ovn_detrace_callback, +) +@click.option( + "--ovnsb-db", + default=os.getenv("OVN_SB_DB") or "unix:/var/run/ovn/ovnsb_db.sock", + help="Specify the OVN NB database string (implies -d). " + "If the OVN_NB_DB environment variable is set, it's used as default. " + "Otherwise, the default is unix:/var/run/ovn/ovnnb_db.sock", + callback=ovn_detrace_callback, +) +@click.option( + "-o", + "--ovn-filter", + help="Specify a filter to be run on ovn-detrace information (implied -d). " + "Format: python regular expression " + "(see https://docs.python.org/3/library/re.html)", + callback=ovn_detrace_callback, +) +@click.option( + "-s", + "--show-flows", + is_flag=True, + default=False, + show_default=True, + help="Show the full flows under each logical flow", +) +@click.option( + "-c", + "--cookie", + "cookie_flag", + is_flag=True, + default=False, + show_default=True, + help="Consider the cookie in the logical flow", +) +@click.option( + "-h", + "--heat-map", + is_flag=True, + default=False, + show_default=True, + help="Create heat-map with packet and byte counters (when -s is used)", +) +@click.pass_obj +def logic( + opts, + ovn_detrace_flag, + ovn_detrace_path, + ovnnb_db, + ovnsb_db, + ovn_filter, + show_flows, + cookie_flag, + heat_map, +): + """ + Print the logical structure of the flows. + + First, sorts the flows based on tables and priorities. + Then, deduplicates logically equivalent flows: these a flows that match + on the same set of fields (regardless of the values they match against), + have the same priority, and actions (regardless of action arguments, + except in the case of output and recirculate). + Optionally, the cookie can also be considered to be part of the logical + flow. + """ + if ovn_detrace_flag: + opts["ovn_detrace_flag"] = True + if opts.get("ovn_detrace_flag"): + cookie_flag = True + + processor = LogicFlowProcessor(opts, cookie_flag, heat_map) + processor.process() + processor.print(show_flows) + + @openflow.command() @click.pass_obj def html(opts): diff --git a/python/ovs/flowviz/ofp/logic.py b/python/ovs/flowviz/ofp/logic.py new file mode 100644 index 000000000..db2124374 --- /dev/null +++ b/python/ovs/flowviz/ofp/logic.py @@ -0,0 +1,303 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import sys +import io +import re + +from rich.tree import Tree +from rich.text import Text + +from ovs.flowviz.process import FileProcessor, OpenFlowFactory +from ovs.flowviz.console import ( + ConsoleFormatter, + ConsoleBuffer, + hash_pallete, + file_header, + heat_pallete, +) + + +class LFlow: + """A Logical Flow represents the scheleton of a flow. + + Two logical flows have the same logical representation if they match + against same fields (regardless of the matching value) and have the same + set of actions (regardless of the actions' arguments, except for those + in the exact_actions list). + + Attributes: + flow (OFPFlow): The flow + exact_actions(list): Optional; list of action keys that are + considered unique if the value is also the same. + match_cookie (bool): Optional; if cookies are part of the logical + flow + """ + + def __init__(self, flow, exact_actions=[], match_cookie=False): + self.cookie = flow.info.get("cookie") or 0 if match_cookie else None + self.priority = flow.match.get("priority") or 0 + self.match_keys = tuple([kv.key for kv in flow.match_kv]) + + self.action_keys = tuple( + [ + kv.key + for kv in flow.actions_kv + if kv.key not in exact_actions + ] + ) + self.match_action_kvs = [ + kv for kv in flow.actions_kv if kv.key in exact_actions + ] + + def __eq__(self, other): + return ( + (self.cookie == other.cookie if self.cookie else True) + and self.priority == other.priority + and self.action_keys == other.action_keys + and self.equal_match_action_kvs(other) + and self.match_keys == other.match_keys + ) + + def equal_match_action_kvs(self, other): + """ Compares the logical flow's match action key-values with the + others. + + Args: + other (LFlow): The other LFlow to compare against + + Returns true if both LFlow have the same action k-v. + """ + if len(other.match_action_kvs) != len(self.match_action_kvs): + return False + + for kv in self.match_action_kvs: + found = False + for other_kv in other.match_action_kvs: + if self.match_kv(kv, other_kv): + found = True + break + if not found: + return False + return True + + def match_kv(self, one, other): + """Compares a KeyValue. + Args: + one, other (KeyValue): The objects to compare + + Returns true if both KeyValue objects have the same key and value + """ + return one.key == other.key and one.value == other.value + + def __hash__(self): + hash_data = [ + self.cookie, + self.priority, + self.action_keys, + tuple((kv.key, str(kv.value)) for kv in self.match_action_kvs), + self.match_keys, + ] + if self.cookie: + hash_data.append(self.cookie) + return tuple(hash_data).__hash__() + + def format(self, buf, formatter): + """Format the Logical Flow into a Buffer.""" + if self.cookie: + buf.append_extra( + "cookie={} ".format(hex(self.cookie)).ljust(18), + style=cookie_style_gen(str(self.cookie)), + ) + + buf.append_extra( + "priority={} ".format(self.priority), style="steel_blue" + ) + buf.append_extra(",".join(self.match_keys), style="steel_blue") + buf.append_extra(" ---> ", style="bold magenta") + buf.append_extra(",".join(self.action_keys), style="steel_blue") + + if len(self.match_action_kvs) > 0: + buf.append_extra(" ", style=None) + + for kv in self.match_action_kvs: + formatter.format_kv(buf, kv, formatter.style) + buf.append_extra(",", style=None) + + +class LogicFlowProcessor(OpenFlowFactory, FileProcessor): + def __init__(self, opts, match_cookie, heat_map): + super().__init__(opts) + self.data = dict() + self.min_max = dict() + self.match_cookie = match_cookie + self.heat_map = ["n_packets", "n_bytes"] if heat_map else [] + self.ovn_detrace = ( + OVNDetrace(opts) if opts.get("ovn_detrace_flag") else None + ) + + def start_file(self, name, filename): + if len(self.heat_map) > 0: + self.min = [-1] * len(self.heat_map) + self.max = [0] * len(self.heat_map) + self.tables = dict() + + def stop_file(self, name, filename): + if len(self.heat_map) > 0: + self.min_max[name] = (self.min, self.max) + self.data[name] = self.tables + + def process_flow(self, flow, name): + """Sort the flows by table and logical flow.""" + # Running calculation of min and max values for all the fields that + # take place in the heatmap. + for i, field in enumerate(self.heat_map): + val = flow.info.get(field) + if self.min[i] == -1 or val < self.min[i]: + self.min[i] = val + if val > self.max[i]: + self.max[i] = val + + table = flow.info.get("table") or 0 + if not self.tables.get(table): + self.tables[table] = dict() + + # Group flows by logical hash + lflow = LFlow( + flow, + exact_actions=["output", "resubmit", "drop"], + match_cookie=self.match_cookie, + ) + + if not self.tables[table].get(lflow): + self.tables[table][lflow] = list() + + self.tables[table][lflow].append(flow) + + def print(self, show_flows): + formatter = ConsoleFormatter(opts=self.opts) + console = formatter.console + for name, tables in self.data.items(): + console.print("\n") + console.print(file_header(name)) + tree = Tree("Ofproto Flows (logical)") + + for table_num in sorted(tables.keys()): + table = tables[table_num] + table_tree = tree.add("** TABLE {} **".format(table_num)) + + if len(self.heat_map) > 0 and len(table.values()) > 0: + for i, field in enumerate(self.heat_map): + (min_val, max_val) = self.min_max[name][i] + formatter.style.set_value_style( + field, heat_pallete(min_val, max_val) + ) + + for lflow in sorted( + table.keys(), + key=(lambda x: x.priority), + reverse=True, + ): + flows = table[lflow] + ovn_info = None + if self.ovn_detrace: + ovn_info = self.ovn_detrace.get_ovn_info(lflow.cookie) + if self.opts.get("ovn_filter"): + ovn_regexp = re.compile( + self.opts.get("ovn_filter") + ) + if not ovn_regexp.search(ovn_info): + continue + + buf = ConsoleBuffer(Text()) + + lflow.format(buf, formatter) + buf.append_extra( + " ( x {} )".format(len(flows)), + style="dark_olive_green3", + ) + lflow_tree = table_tree.add(buf.text) + + if ovn_info: + ovn = lflow_tree.add("OVN Info") + for part in ovn_info.split("\n"): + if part.strip(): + ovn.add(part.strip()) + + if show_flows: + for flow in flows: + buf = ConsoleBuffer(Text()) + highlighted = None + if self.opts.get("highlight"): + result = self.opts.get("highlight").evaluate( + flow + ) + if result: + highlighted = result.kv + formatter.format_flow(buf, flow, highlighted) + lflow_tree.add(buf.text) + + console.print(tree) + + +class OVNDetrace(object): + def __init__(self, opts): + if not opts.get("ovn_detrace_flag"): + raise Exception("Cannot initialize OVN Detrace connection") + + if opts.get("ovn_detrace_path"): + sys.path.append(opts.get("ovn_detrace_path")) + + import ovn_detrace + + class FakePrinter(ovn_detrace.Printer): + def __init__(self): + self.buff = io.StringIO() + + def print_p(self, msg): + print(" * ", msg, file=self.buff) + + def print_h(self, msg): + print(" * ", msg, file=self.buff) + + def clear(self): + self.buff = io.StringIO() + + self.ovn_detrace = ovn_detrace + self.ovnnb_conn = ovn_detrace.OVSDB( + opts.get("ovnnb_db"), "OVN_Northbound" + ) + self.ovnsb_conn = ovn_detrace.OVSDB( + opts.get("ovnsb_db"), "OVN_Southbound" + ) + self.ovn_printer = FakePrinter() + self.cookie_handlers = ovn_detrace.get_cookie_handlers( + self.ovnnb_conn, self.ovnsb_conn, self.ovn_printer + ) + + def get_ovn_info(self, cookie): + self.ovn_printer.clear() + self.ovn_detrace.print_record_from_cookie( + self.ovnsb_conn, self.cookie_handlers, "{:x}".format(cookie) + ) + return self.ovn_printer.buff.getvalue() + + +# Try to make it easy to spot same cookies by printing them in different +# colors +cookie_style_gen = hash_pallete( + hue=[x / 10 for x in range(0, 10)], + saturation=[0.5], + value=[0.5 + x / 10 * (0.85 - 0.5) for x in range(0, 10)], +)