From patchwork Tue Apr 9 07:06:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1921249 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=K/pvL3TX; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VDH864L9Qz1yY8 for ; Tue, 9 Apr 2024 17:07:00 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 4B75260BBC; Tue, 9 Apr 2024 07:06:58 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id h01O1_jl9i2l; Tue, 9 Apr 2024 07:06:57 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 05BA360784 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key, unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=K/pvL3TX Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 05BA360784; Tue, 9 Apr 2024 07:06:56 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 904A9C007C; Tue, 9 Apr 2024 07:06:56 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id C79CAC0037 for ; Tue, 9 Apr 2024 07:06:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id AF90D40B11 for ; Tue, 9 Apr 2024 07:06:55 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id eF553Fnh-Fo8 for ; Tue, 9 Apr 2024 07:06:54 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp2.osuosl.org 9EA35400C7 Authentication-Results: smtp2.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 9EA35400C7 Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key, unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=K/pvL3TX Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 9EA35400C7 for ; Tue, 9 Apr 2024 07:06:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712646413; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ZN4LlSKL6ZLNGY/ILHtdHjir/Z5/GJH7u+ZIbkyScR4=; b=K/pvL3TXLAMX0yyM8buPbEuZRNZjD3kmmbiKqOxEW3wgir4w1uFbBSX+xHf6VpWox6tWVT /TvfliQPFRPJBPOrNByr3h/QzW1Szzdb6mEe5tw4A5yZj8rd7l3DBcS0s9aYpq0TgeUuWl VDcQEFFfIv7+t8ubtoFM44eE5H2IQhk= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-107-XovFEmFyOpG2d-0KIe1-xg-1; Tue, 09 Apr 2024 03:06:50 -0400 X-MC-Unique: XovFEmFyOpG2d-0KIe1-xg-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C9D2829AA3B0 for ; Tue, 9 Apr 2024 07:06:49 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.113]) by smtp.corp.redhat.com (Postfix) with ESMTP id 29189444567; Tue, 9 Apr 2024 07:06:48 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 9 Apr 2024 09:06:18 +0200 Message-ID: <20240409070642.511747-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.10 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v3 00/12] Add flow visualization utility. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The goal of this utility is to read both datapath and Openflow flows (using the flow library already available) and print them in different formats and styles to make it easier to understand them and troubleshoot issues. The formats are quite opinionated and so are the colors chosen so I'm eager to hear what is the impression caused to the community. Here is a summary of the formats and features supported: - Openflow - Console: Prints flows back to the console allowing filtering and extensive formatting. - Cookie: Arranges flows based on cookie (instead of table) to ease visualization of OVN-generated flows. - HTML: Prints an HTML file that shows the flows arranged by tables. resubmit actions have a hyperlinke to the target table to ease navegation. - Logic: Many times OVN generates many "logically-equivalent" flows. These are flows that have the same structure: match on the same values and have the same actions. The only thing that changes is the actual value they match against and maybe the arguments of the actions. This format collapses these flows so you can visualize the logical pipeline easier. - JSON: JSON format. More OpenFlow features: - OVN metadata: Both the "cookie" and the "logic" format allow to connect to a running OVN NB/SB databases and enrich the flows with OVN context based on the flow's cookie. - Datapath: - Console: Prints flows back to the console allowing filtering and extensive formatting. - Tree: Datapath flows use recirculation to further execute additional actions. This format builds a tree of flows following the recirculation identifiers and prints it in the console. - HTML: Prints the datapath flow tree in HTML. It includes some minimal JS to support expanding and collapsing of entire branches. - Graph: Following the "tree" format, this one prints the tree in graphviz format. - JSON: JSON format. Additional datapath features: - Many datapath formats are based on the tree flow hierarchy. An interesting feature of this structure is that filtering is done at the branch level. This means that when a flow satisfies the filter, the entire sub-tree leading to that flow is shown. Additional common features: - Styling: Styles for both console and HTML formats can be defined using a configuration file. - Heat maps: Some formats support heat-maps. A color pallete ranging from blue (cold) to red (hot) is used to print the number of packets and bytes of the flows. That way, the flows that are handling more traffic are much easier to spot --- V2 -> V3: - Fix grammar thanks to Eelco's review V1 -> V2: - Fix typos and nits on documentation - Split documentation in two: ovs-flowviz.8 man page and a topic article with more detailed examples. RFC -> V1: - Addressed Eelco's comments - Added a documentation page - Added support for dark style HTML pages - Patch 3. Small fix in the way a style is looked up. Use the key in the KV instead of the metadata string. This helps with "free" values such as "output". Adrian Moreno (12): python: ovs: Add flowviz scheleton. python: ovs: flowviz: Add file processing infra. python: ovs: flowviz: Add console formatting. python: ovs: flowviz: Add default config file. python: ovs: flowviz: Add html formatting. python: ovs: flowviz: Add datapath tree format. python: ovs: flowviz: Add OpenFlow logical view. python: ovs: flowviz: Add Openflow cookie format. python: ovs: flowviz: Add datapath html format. python: ovs: flowviz: Add datapath graph format. python: ovs: flowviz: Support html dark style. documentation: Document ovs-flowviz. Documentation/automake.mk | 4 +- Documentation/conf.py | 2 + Documentation/ref/index.rst | 1 + Documentation/ref/ovs-flowviz.8.rst | 531 ++++++++++++++++++++ Documentation/topics/flow-visualization.rst | 271 ++++++++++ Documentation/topics/index.rst | 1 + python/automake.mk | 32 +- python/ovs/flowviz/__init__.py | 2 + python/ovs/flowviz/console.py | 196 ++++++++ python/ovs/flowviz/format.py | 371 ++++++++++++++ python/ovs/flowviz/html_format.py | 138 +++++ python/ovs/flowviz/main.py | 196 ++++++++ python/ovs/flowviz/odp/__init__.py | 0 python/ovs/flowviz/odp/cli.py | 116 +++++ python/ovs/flowviz/odp/graph.py | 418 +++++++++++++++ python/ovs/flowviz/odp/html.py | 279 ++++++++++ python/ovs/flowviz/odp/tree.py | 303 +++++++++++ python/ovs/flowviz/ofp/__init__.py | 0 python/ovs/flowviz/ofp/cli.py | 246 +++++++++ python/ovs/flowviz/ofp/html.py | 98 ++++ python/ovs/flowviz/ofp/logic.py | 364 ++++++++++++++ python/ovs/flowviz/ovs-flowviz | 20 + python/ovs/flowviz/ovs-flowviz.conf | 128 +++++ python/ovs/flowviz/process.py | 263 ++++++++++ python/setup.py | 14 +- 25 files changed, 3984 insertions(+), 10 deletions(-) create mode 100644 Documentation/ref/ovs-flowviz.8.rst create mode 100644 Documentation/topics/flow-visualization.rst create mode 100644 python/ovs/flowviz/__init__.py create mode 100644 python/ovs/flowviz/console.py create mode 100644 python/ovs/flowviz/format.py create mode 100644 python/ovs/flowviz/html_format.py create mode 100644 python/ovs/flowviz/main.py create mode 100644 python/ovs/flowviz/odp/__init__.py create mode 100644 python/ovs/flowviz/odp/cli.py create mode 100644 python/ovs/flowviz/odp/graph.py create mode 100644 python/ovs/flowviz/odp/html.py create mode 100644 python/ovs/flowviz/odp/tree.py create mode 100644 python/ovs/flowviz/ofp/__init__.py create mode 100644 python/ovs/flowviz/ofp/cli.py create mode 100644 python/ovs/flowviz/ofp/html.py create mode 100644 python/ovs/flowviz/ofp/logic.py create mode 100755 python/ovs/flowviz/ovs-flowviz create mode 100644 python/ovs/flowviz/ovs-flowviz.conf create mode 100644 python/ovs/flowviz/process.py