From patchwork Tue May 7 14:30:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1932499 X-Patchwork-Delegate: echaudro@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=BmfmrbJQ; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VYggl12Y4z1xnT for ; Wed, 8 May 2024 00:31:15 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 18A4B821A2; Tue, 7 May 2024 14:31:09 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id T74A1D5rgW7b; Tue, 7 May 2024 14:31:06 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org B112F8214C Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=BmfmrbJQ Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id B112F8214C; Tue, 7 May 2024 14:31:04 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4DACDC0DD5; Tue, 7 May 2024 14:31:04 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id B945EC0037 for ; Tue, 7 May 2024 14:31:02 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 9BF5981FDE for ; Tue, 7 May 2024 14:31:02 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id BFicnWgrzHLQ for ; Tue, 7 May 2024 14:31:01 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp1.osuosl.org 1B533820E8 Authentication-Results: smtp1.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 1B533820E8 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 1B533820E8 for ; Tue, 7 May 2024 14:31:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1715092259; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VoIfRBfg3stZ2+WoHH7dADfh/BUjacdUz+C91XFlG2w=; b=BmfmrbJQN5q1MoXyCunEWjTqwYugpbrMSrYobjKMCyBZI+r60WkiYw9PU3Ev8hbs/tB+5J jrnkRO+LNaID6gVAFWJPoQmOV8S1luGgSdAPpQd0m50qrKJWFspC73T6TmGMSNa49Z3Rsl xxQbN5r4mUSQJ7xPpU8hegVsHgOkk10= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-661-3PdDXNdqNlm8xSGpPVAOrQ-1; Tue, 07 May 2024 10:30:57 -0400 X-MC-Unique: 3PdDXNdqNlm8xSGpPVAOrQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9A27F802E62 for ; Tue, 7 May 2024 14:30:57 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.88]) by smtp.corp.redhat.com (Postfix) with ESMTP id 906733C27; Tue, 7 May 2024 14:30:56 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 7 May 2024 16:30:27 +0200 Message-ID: <20240507143044.3056232-2-amorenoz@redhat.com> In-Reply-To: <20240507143044.3056232-1-amorenoz@redhat.com> References: <20240507143044.3056232-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v4 01/12] python: ovs: Add flowviz scheleton. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add a new python package (just the scheleton for now) to hold a flow visualization tool based on the flow parsing library. flowviz dependencies are installed via "extras_require", so a user must run: $ pip install .[flowviz] or $ pip install ovs[flowviz] Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 14 ++++++++--- python/ovs/flowviz/__init__.py | 0 python/ovs/flowviz/main.py | 40 ++++++++++++++++++++++++++++++ python/ovs/flowviz/odp/__init__.py | 0 python/ovs/flowviz/ofp/__init__.py | 0 python/ovs/flowviz/ovs-flowviz | 20 +++++++++++++++ python/setup.py | 11 +++++--- 7 files changed, 79 insertions(+), 6 deletions(-) create mode 100644 python/ovs/flowviz/__init__.py create mode 100644 python/ovs/flowviz/main.py create mode 100644 python/ovs/flowviz/odp/__init__.py create mode 100644 python/ovs/flowviz/ofp/__init__.py create mode 100755 python/ovs/flowviz/ovs-flowviz diff --git a/python/automake.mk b/python/automake.mk index 84cf2eab5..124032c92 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -63,6 +63,14 @@ ovs_pytests = \ python/ovs/tests/test_odp.py \ python/ovs/tests/test_ofp.py +ovs_flowviz = \ + python/ovs/flowviz/__init__.py \ + python/ovs/flowviz/main.py \ + python/ovs/flowviz/odp/__init__.py \ + python/ovs/flowviz/ofp/__init__.py \ + python/ovs/flowviz/ovs-flowviz + + # These python files are used at build time but not runtime, # so they are not installed. EXTRA_DIST += \ @@ -81,10 +89,10 @@ EXTRA_DIST += \ # C extension support. EXTRA_DIST += python/ovs/_json.c -PYFILES = $(ovs_pyfiles) python/ovs/dirs.py $(ovstest_pyfiles) $(ovs_pytests) +PYFILES = $(ovs_pyfiles) python/ovs/dirs.py $(ovstest_pyfiles) $(ovs_pytests) $(ovs_flowviz) EXTRA_DIST += $(PYFILES) -PYCOV_CLEAN_FILES += $(PYFILES:.py=.py,cover) +PYCOV_CLEAN_FILES += $($(filter %.py, PYFILES):.py=.py,cover) python/ovs/flowviz/ovs-flowviz,cover FLAKE8_PYFILES += \ $(filter-out python/ovs/compat/% python/ovs/dirs.py,$(PYFILES)) \ @@ -95,7 +103,7 @@ FLAKE8_PYFILES += \ python/ovs/dirs.py.template \ python/setup.py -nobase_pkgdata_DATA = $(ovs_pyfiles) $(ovstest_pyfiles) +nobase_pkgdata_DATA = $(ovs_pyfiles) $(ovstest_pyfiles) $(ovs_flowviz) ovs-install-data-local: $(MKDIR_P) python/ovs sed \ diff --git a/python/ovs/flowviz/__init__.py b/python/ovs/flowviz/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/python/ovs/flowviz/main.py b/python/ovs/flowviz/main.py new file mode 100644 index 000000000..f5bf142be --- /dev/null +++ b/python/ovs/flowviz/main.py @@ -0,0 +1,40 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import click + + +class Options(dict): + """Options dictionary""" + + +@click.group( + context_settings=dict(help_option_names=["-h", "--help"]), +) +@click.pass_context +def maincli(ctx): + """ + OpenvSwitch flow visualization utility. + + It reads openflow and datapath flows + (such as the output of ovs-ofctl dump-flows or ovs-appctl dpctl/dump-flows) + and prints them in different formats. + """ + + +def main(): + """ + Main Function + """ + maincli() diff --git a/python/ovs/flowviz/odp/__init__.py b/python/ovs/flowviz/odp/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/python/ovs/flowviz/ofp/__init__.py b/python/ovs/flowviz/ofp/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/python/ovs/flowviz/ovs-flowviz b/python/ovs/flowviz/ovs-flowviz new file mode 100755 index 000000000..9d0959812 --- /dev/null +++ b/python/ovs/flowviz/ovs-flowviz @@ -0,0 +1,20 @@ +#!/usr/bin/env python3 +# +# Copyright (c) 2022,2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from ovs.flowviz import main + +if __name__ == '__main__': + main.main() diff --git a/python/setup.py b/python/setup.py index bcf832ce9..4b9c751d2 100644 --- a/python/setup.py +++ b/python/setup.py @@ -80,6 +80,7 @@ else: extra_cflags = os.environ.get('extra_cflags', '').split() extra_libs = os.environ.get('extra_libs', '').split() +flow_extras_require = ['netaddr', 'pyparsing'] setup_args = dict( name='ovs', @@ -89,7 +90,8 @@ setup_args = dict( author='Open vSwitch', author_email='dev@openvswitch.org', packages=['ovs', 'ovs.compat', 'ovs.compat.sortedcontainers', - 'ovs.db', 'ovs.unixctl', 'ovs.flow'], + 'ovs.db', 'ovs.flow', 'ovs.flowviz', 'ovs.flowviz.odp', + 'ovs.flowviz.ofp', 'ovs.unixctl'], keywords=['openvswitch', 'ovs', 'OVSDB'], license='Apache 2.0', classifiers=[ @@ -109,8 +111,11 @@ setup_args = dict( cmdclass={'build_ext': try_build_ext}, install_requires=['sortedcontainers'], extras_require={':sys_platform == "win32"': ['pywin32 >= 1.0'], - 'flow': ['netaddr', 'pyparsing'], - 'dns': ['unbound']}, + 'dns': ['unbound'], + 'flow': flow_extras_require, + 'flowviz': [*flow_extras_require, 'click'], + }, + scripts=["ovs/flowviz/ovs-flowviz"], ) try: From patchwork Tue May 7 14:30:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1932502 X-Patchwork-Delegate: echaudro@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=XR8btFBr; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VYggr4QBgz1xnT for ; Wed, 8 May 2024 00:31:20 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id BE4CB821CE; Tue, 7 May 2024 14:31:13 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id bj5yVpax0KKM; Tue, 7 May 2024 14:31:09 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org AADAA8217F Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=XR8btFBr Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id AADAA8217F; Tue, 7 May 2024 14:31:05 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7EE01C0DDA; Tue, 7 May 2024 14:31:05 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id E6544C0037 for ; Tue, 7 May 2024 14:31:03 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id C3427409A6 for ; Tue, 7 May 2024 14:31:03 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id yoT3UM_tBYLj for ; Tue, 7 May 2024 14:31:02 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp2.osuosl.org F07F64045D Authentication-Results: smtp2.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org F07F64045D Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=XR8btFBr Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id F07F64045D for ; Tue, 7 May 2024 14:31:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1715092260; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sfkJR2/4YLDculi6AT+6joL2n+hBMGqhodK2zykrM3I=; b=XR8btFBrCCzUymW5J+8QajZ0zxv48ox/8d6/Vt/Yr4smfGu8i15eP8dadpALLUb0PAk2TM Q1lG4tsxcDX9KclBVBUU9EQIGt7WWQSeDxPubdBY6vRKlci5HoC9ZulPZ9qb3sj3wXH5Ry 4PofG2JzkusZTircJydRAOHMv2Nze3U= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-390-wxyt_4GrNxqe8R5wyBnVNw-1; Tue, 07 May 2024 10:30:59 -0400 X-MC-Unique: wxyt_4GrNxqe8R5wyBnVNw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F0CE585A58C for ; Tue, 7 May 2024 14:30:58 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.88]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1F1853C25; Tue, 7 May 2024 14:30:57 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 7 May 2024 16:30:28 +0200 Message-ID: <20240507143044.3056232-3-amorenoz@redhat.com> In-Reply-To: <20240507143044.3056232-1-amorenoz@redhat.com> References: <20240507143044.3056232-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v4 02/12] python: ovs: flowviz: Add file processing infra. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" process.py contains a useful base class that processes files odp.py and ofp.py: contain datapath and openflow subcommand definitions as well as the first formatting option: json. Also, this patch adds basic filtering support. Examples: $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow json $ ovs-ofctl dump-flows br-int > flows.txt && ovs-flowviz -i flows.txt openflow json $ ovs-ofctl appctl dpctl/dump-flows | ovs-flowviz -f 'ct' datapath json $ ovs-ofctl appctl dpctl/dump-flows > flows.txt && ovs-flowviz -i flows.txt -f 'drop' datapath json Signed-off-by: Adrian Moreno Acked-by: Eelco Chaudron --- python/automake.mk | 5 +- python/ovs/flowviz/__init__.py | 2 + python/ovs/flowviz/main.py | 102 +++++++++++++++++- python/ovs/flowviz/odp/cli.py | 42 ++++++++ python/ovs/flowviz/ofp/cli.py | 42 ++++++++ python/ovs/flowviz/process.py | 192 +++++++++++++++++++++++++++++++++ 6 files changed, 383 insertions(+), 2 deletions(-) create mode 100644 python/ovs/flowviz/odp/cli.py create mode 100644 python/ovs/flowviz/ofp/cli.py create mode 100644 python/ovs/flowviz/process.py diff --git a/python/automake.mk b/python/automake.mk index 124032c92..fd5e74081 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -67,8 +67,11 @@ ovs_flowviz = \ python/ovs/flowviz/__init__.py \ python/ovs/flowviz/main.py \ python/ovs/flowviz/odp/__init__.py \ + python/ovs/flowviz/odp/cli.py \ python/ovs/flowviz/ofp/__init__.py \ - python/ovs/flowviz/ovs-flowviz + python/ovs/flowviz/ofp/cli.py \ + python/ovs/flowviz/ovs-flowviz \ + python/ovs/flowviz/process.py # These python files are used at build time but not runtime, diff --git a/python/ovs/flowviz/__init__.py b/python/ovs/flowviz/__init__.py index e69de29bb..898dba522 100644 --- a/python/ovs/flowviz/__init__.py +++ b/python/ovs/flowviz/__init__.py @@ -0,0 +1,2 @@ +import ovs.flowviz.ofp.cli # noqa: F401 +import ovs.flowviz.odp.cli # noqa: F401 diff --git a/python/ovs/flowviz/main.py b/python/ovs/flowviz/main.py index f5bf142be..64b0e8a0a 100644 --- a/python/ovs/flowviz/main.py +++ b/python/ovs/flowviz/main.py @@ -13,17 +13,64 @@ # limitations under the License. import click +import os + +from ovs.flow.filter import OFFilter class Options(dict): """Options dictionary""" +def validate_input(ctx, param, value): + """Validate the "-i" option""" + result = list() + for input_str in value: + parts = input_str.strip().split(",") + if len(parts) == 2: + file_parts = tuple(parts) + elif len(parts) == 1: + file_parts = tuple(["Filename: " + parts[0], parts[0]]) + else: + raise click.BadParameter( + "input filename should have the following format: " + "[alias,]FILENAME" + ) + + if not os.path.isfile(file_parts[1]): + raise click.BadParameter( + "input filename %s does not exist" % file_parts[1] + ) + result.append(file_parts) + return result + + @click.group( context_settings=dict(help_option_names=["-h", "--help"]), ) +@click.option( + "-i", + "--input", + "filename", + help="Read flows from specified filepath. If not provided, flows will be" + " read from stdin. This option can be specified multiple times." + " Format [alias,]FILENAME. Where alias is a name that shall be used to" + " refer to this FILENAME", + multiple=True, + type=click.Path(), + callback=validate_input, +) +@click.option( + "-f", + "--filter", + help="Filter flows that match the filter expression." + "Run 'ovs-flowviz filter' for a detailed description of the filtering " + "syntax", + type=str, + show_default=False, +) @click.pass_context -def maincli(ctx): +def maincli(ctx, filename, filter): """ OpenvSwitch flow visualization utility. @@ -31,6 +78,59 @@ def maincli(ctx): (such as the output of ovs-ofctl dump-flows or ovs-appctl dpctl/dump-flows) and prints them in different formats. """ + ctx.obj = Options() + ctx.obj["filename"] = filename or None + if filter: + try: + ctx.obj["filter"] = OFFilter(filter) + except Exception as e: + raise click.BadParameter("Wrong filter syntax: {}".format(e)) + + +@maincli.command(hidden=True) +@click.pass_context +def filter(ctx): + """ + \b + Filter Syntax + ************* + + [! | not ] {key}[[.subkey]...] [OPERATOR] {value})] [LOGICAL OPERATOR] ... + + \b + Comparison operators are: + = equality + < less than + > more than + ~= masking (valid for IP and Ethernet fields) + + \b + Logical operators are: + !{expr}: NOT + {expr} && {expr}: AND + {expr} || {expr}: OR + + \b + Matches and flow metadata: + To compare against a match or info field, use the field directly, e.g: + priority=100 + n_bytes>10 + Use simple keywords for flags: + tcp and ip_src=192.168.1.1 + \b + Actions: + Actions values might be dictionaries, use subkeys to access individual + values, e.g: + output.port=3 + Use simple keywords for flags + drop + + \b + Examples of valid filters. + nw_addr~=192.168.1.1 && (tcp_dst=80 || tcp_dst=443) + arp=true && !arp_tsa=192.168.1.1 + n_bytes>0 && drop=true""" + click.echo(ctx.command.get_help(ctx)) def main(): diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py new file mode 100644 index 000000000..ed2f82065 --- /dev/null +++ b/python/ovs/flowviz/odp/cli.py @@ -0,0 +1,42 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import click + +from ovs.flowviz.main import maincli +from ovs.flowviz.process import ( + DatapathFactory, + JSONProcessor, +) + + +@maincli.group(subcommand_metavar="FORMAT") +@click.pass_obj +def datapath(opts): + """Process Datapath Flows.""" + pass + + +class JSONPrint(DatapathFactory, JSONProcessor): + def __init__(self, opts): + super().__init__(opts) + + +@datapath.command() +@click.pass_obj +def json(opts): + """Print the flows in JSON format.""" + proc = JSONPrint(opts) + proc.process() + print(proc.json_string()) diff --git a/python/ovs/flowviz/ofp/cli.py b/python/ovs/flowviz/ofp/cli.py new file mode 100644 index 000000000..b9a2a8aad --- /dev/null +++ b/python/ovs/flowviz/ofp/cli.py @@ -0,0 +1,42 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import click + +from ovs.flowviz.main import maincli +from ovs.flowviz.process import ( + OpenFlowFactory, + JSONProcessor, +) + + +@maincli.group(subcommand_metavar="FORMAT") +@click.pass_obj +def openflow(opts): + """Process OpenFlow Flows.""" + pass + + +class JSONPrint(OpenFlowFactory, JSONProcessor): + def __init__(self, opts): + super().__init__(opts) + + +@openflow.command() +@click.pass_obj +def json(opts): + """Print the flows in JSON format.""" + proc = JSONPrint(opts) + proc.process() + print(proc.json_string()) diff --git a/python/ovs/flowviz/process.py b/python/ovs/flowviz/process.py new file mode 100644 index 000000000..3e520e431 --- /dev/null +++ b/python/ovs/flowviz/process.py @@ -0,0 +1,192 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import sys +import json +import click + +from ovs.flow.decoders import FlowEncoder +from ovs.flow.odp import ODPFlow +from ovs.flow.ofp import OFPFlow + + +class FileProcessor(object): + """Base class for file-based Flow processing. It is able to create flows + from strings found in a file (or stdin). + + The process of parsing the flows is extendable in many ways by deriving + this class. + + When process() is called, the base class will: + - call self.start_file() for each new file that get's processed + - call self.create_flow() for each flow line + - apply the filter defined in opts if provided (can be optionally + disabled) + - call self.process_flow() for after the flow has been filtered + - call self.stop_file() after the file has been processed entirely + + In the case of stdin, the filename and file alias is 'stdin'. + + Child classes must at least implement create_flow() and process_flow() + functions. + + Args: + opts (dict): Options dictionary + """ + + def __init__(self, opts): + self.opts = opts + + # Methods that must be implemented by derived classes. + def init(self): + """Called before the flow processing begins.""" + pass + + def start_file(self, alias, filename): + """Called before the processing of a file begins. + Args: + alias(str): The alias name of the filename + filename(str): The filename string + """ + pass + + def create_flow(self, line, idx): + """Called for each line in the file. + Args: + line(str): The flow line + idx(int): The line index + + Returns a Flow. + Must be implemented by child classes. + """ + raise NotImplementedError + + def process_flow(self, flow, name): + """Called for built flow (after filtering). + Args: + flow(Flow): The flow created by create_flow + name(str): The name of the file from which the flow comes + """ + raise NotImplementedError + + def stop_file(self, alias, filename): + """Called after the processing of a file ends. + Args: + alias(str): The alias name of the filename + filename(str): The filename string + """ + pass + + def end(self): + """Called after the processing ends.""" + pass + + def process(self, do_filter=True): + idx = 0 + filenames = self.opts.get("filename") + filt = self.opts.get("filter") if do_filter else None + self.init() + if filenames: + for alias, filename in filenames: + try: + with open(filename) as f: + self.start_file(alias, filename) + for line in f: + flow = self.create_flow(line, idx) + idx += 1 + if not flow or (filt and not filt.evaluate(flow)): + continue + self.process_flow(flow, alias) + self.stop_file(alias, filename) + except IOError as e: + raise click.BadParameter( + "Failed to read from file {} ({}): {}".format( + filename, e.errno, e.strerror + ) + ) + else: + data = sys.stdin.read() + self.start_file("stdin", "stdin") + for line in data.split("\n"): + line = line.strip() + if line: + flow = self.create_flow(line, idx) + idx += 1 + if ( + not flow + or not getattr(flow, "_sections", None) + or (filt and not filt.evaluate(flow)) + ): + continue + self.process_flow(flow, "stdin") + self.stop_file("stdin", "stdin") + self.end() + + +class DatapathFactory(): + """A mixin class that creates Datapath flows.""" + + def create_flow(self, line, idx): + # Skip strings commonly found in Datapath flow dumps. + if any(s in line for s in [ + "flow-dump from the main thread", + "flow-dump from pmd on core", + ]): + return None + + return ODPFlow(line, idx) + + +class OpenFlowFactory(): + """A mixin class that creates OpenFlow flows.""" + + def create_flow(self, line, idx): + # Skip strings commonly found in OpenFlow flow dumps. + if " reply " in line: + return None + + return OFPFlow(line, idx) + + +class JSONProcessor(FileProcessor): + """A FileProcessor that prints flows in JSON format.""" + + def __init__(self, opts): + super().__init__(opts) + self.flows = dict() + + def start_file(self, name, filename): + self.flows_list = list() + + def stop_file(self, name, filename): + self.flows[name] = self.flows_list + + def process_flow(self, flow, name): + self.flows_list.append(flow) + + def json_string(self): + if len(self.flows.keys()) > 1: + return json.dumps( + [ + {"name": name, "flows": [flow.dict() for flow in flows]} + for name, flows in self.flows.items() + ], + indent=4, + cls=FlowEncoder, + ) + return json.dumps( + [flow.dict() for flow in self.flows_list], + indent=4, + cls=FlowEncoder, + ) From patchwork Tue May 7 14:30:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1932507 X-Patchwork-Delegate: echaudro@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=DtH2c9FL; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VYghD4rksz1xnT for ; Wed, 8 May 2024 00:31:40 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id EC2A260AD8; Tue, 7 May 2024 14:31:38 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id vzV9776G-eU5; Tue, 7 May 2024 14:31:33 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 65A5460B34 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=DtH2c9FL Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 65A5460B34; Tue, 7 May 2024 14:31:19 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 628F0C0077; Tue, 7 May 2024 14:31:18 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0E9EEC008E for ; Tue, 7 May 2024 14:31:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id BC234821B4 for ; Tue, 7 May 2024 14:31:14 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 48Y26sZdvXl3 for ; Tue, 7 May 2024 14:31:06 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp1.osuosl.org 2466D8213D Authentication-Results: smtp1.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 2466D8213D Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=DtH2c9FL Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 2466D8213D for ; Tue, 7 May 2024 14:31:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1715092263; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YRCwDE8qyYXuvLZUZPToq5VyCL8WkszKKn6Cj4biQos=; b=DtH2c9FLGyOaeHXPyyXqsFpI1m7yAPh+re9lyP1e4GYSAizOAe1mIxAfWZJ5hfwhuU9lxK jXn8Cpw/ea34NjPd2Z/Fshm54AarODQnlbiQlUnt1Ze27ChEtwMGYb0QTDC70gSyBdDn0L UH9eRPU8Il1U1WZEP10orsKBQIsNs5k= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-468-abo9GGLMNGy0rAP7ap8hNQ-1; Tue, 07 May 2024 10:31:01 -0400 X-MC-Unique: abo9GGLMNGy0rAP7ap8hNQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C485038C6170 for ; Tue, 7 May 2024 14:31:00 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.88]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6FDDC3C25; Tue, 7 May 2024 14:30:59 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 7 May 2024 16:30:29 +0200 Message-ID: <20240507143044.3056232-4-amorenoz@redhat.com> In-Reply-To: <20240507143044.3056232-1-amorenoz@redhat.com> References: <20240507143044.3056232-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v4 03/12] python: ovs: flowviz: Add console formatting. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add a flow formatting framework and one implementation for console printing using rich. The flow formatting framework is a simple set of classes that can be used to write different flow formatting implementations. It supports styles to be described by any class, highlighting and config-file based style definition. The first flow formatting implementation is also introduced: the ConsoleFormatter. It uses the an advanced rich-text printing library [1]. The console printing supports: - Heatmap: printing the packet/byte statistics of each flow in a color that represents its relative size: blue (low) -> red (high). - Printing a banner with the file name and alias. - Extensive style definition via config file. This console format is added to both OpenFlow and Datapath flows. Examples: - Highlight drops in datapath flows: $ ovs-flowviz -i flows.txt --highlight "drop" datapath console - Quickly detect where most packets are going using heatmap and paginated output: $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow console -h [1] https://rich.readthedocs.io/en/stable/introduction.html Signed-off-by: Adrian Moreno --- python/automake.mk | 2 + python/ovs/flowviz/console.py | 175 ++++++++++++++++ python/ovs/flowviz/format.py | 371 ++++++++++++++++++++++++++++++++++ python/ovs/flowviz/main.py | 58 +++++- python/ovs/flowviz/odp/cli.py | 25 +++ python/ovs/flowviz/ofp/cli.py | 26 +++ python/ovs/flowviz/process.py | 83 +++++++- python/setup.py | 4 +- 8 files changed, 736 insertions(+), 8 deletions(-) create mode 100644 python/ovs/flowviz/console.py create mode 100644 python/ovs/flowviz/format.py diff --git a/python/automake.mk b/python/automake.mk index fd5e74081..bd53c5405 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -65,6 +65,8 @@ ovs_pytests = \ ovs_flowviz = \ python/ovs/flowviz/__init__.py \ + python/ovs/flowviz/console.py \ + python/ovs/flowviz/format.py \ python/ovs/flowviz/main.py \ python/ovs/flowviz/odp/__init__.py \ python/ovs/flowviz/odp/cli.py \ diff --git a/python/ovs/flowviz/console.py b/python/ovs/flowviz/console.py new file mode 100644 index 000000000..4a3443360 --- /dev/null +++ b/python/ovs/flowviz/console.py @@ -0,0 +1,175 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import colorsys + +from rich.console import Console +from rich.color import Color +from rich.emoji import Emoji +from rich.panel import Panel +from rich.text import Text +from rich.style import Style + +from ovs.flowviz.format import FlowFormatter, FlowBuffer, FlowStyle + + +def file_header(name): + return Panel( + Text( + Emoji.replace(":scroll:") + + " " + + name + + " " + + Emoji.replace(":scroll:"), + style="bold", + justify="center", + ) + ) + + +class ConsoleBuffer(FlowBuffer): + """ConsoleBuffer implements FlowBuffer to provide console-based text + formatting based on rich.Text. + + Append functions accept a rich.Style. + + Args: + rtext(rich.Text): Optional; text instance to reuse + """ + + def __init__(self, rtext): + self._text = rtext or Text() + + @property + def text(self): + return self._text + + def _append(self, string, style): + """Append to internal text.""" + return self._text.append(string, style) + + def append_key(self, kv, style): + """Append a key. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(kv.meta.kstring, style) + + def append_delim(self, kv, style): + """Append a delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(kv.meta.delim, style) + + def append_end_delim(self, kv, style): + """Append an end delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(kv.meta.end_delim, style) + + def append_value(self, kv, style): + """Append a value. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(kv.meta.vstring, style) + + def append_extra(self, extra, style): + """Append extra string. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(extra, style) + + +class ConsoleFormatter(FlowFormatter): + """ConsoleFormatter is a FlowFormatter that formats flows into the console + using rich.Console. + + Args: + console (rich.Console): Optional, an existing console to use + max_value_len (int): Optional; max length of the printed values + kwargs (dict): Optional; Extra arguments to be passed down to + rich.console.Console() + """ + + def __init__(self, opts=None, console=None, **kwargs): + super(ConsoleFormatter, self).__init__() + style = self.style_from_opts(opts) + self.console = console or Console(color_system="256", **kwargs) + self.style = style or FlowStyle() + + def style_from_opts(self, opts): + return self._style_from_opts(opts, "console", Style) + + def print_flow(self, flow, highlighted=None): + """Prints a flow to the console. + + Args: + flow (ovs_dbg.OFPFlow): the flow to print + style (dict): Optional; style dictionary to use + highlighted (list): Optional; list of KeyValues to highlight + """ + + buf = ConsoleBuffer(Text()) + self.format_flow(buf, flow, highlighted) + self.console.print(buf.text) + + def format_flow(self, buf, flow, highlighted=None): + """Formats the flow into the provided buffer as a rich.Text. + + Args: + buf (FlowBuffer): the flow buffer to append to + flow (ovs_dbg.OFPFlow): the flow to format + style (FlowStyle): Optional; style object to use + highlighted (list): Optional; list of KeyValues to highlight + """ + return super(ConsoleFormatter, self).format_flow( + buf, flow, self.style, highlighted + ) + + +def heat_pallete(min_value, max_value): + """Generates a color pallete based on the 5-color heat pallete so that + for each value between min and max a color is returned that represents it's + relative size. + Args: + min_value (int): minimum value + max_value (int) maximum value + """ + h_min = 0 # red + h_max = 220 / 360 # blue + + def heat(value): + if max_value == min_value: + r, g, b = colorsys.hsv_to_rgb(h_max / 2, 1.0, 1.0) + else: + normalized = (int(value) - min_value) / (max_value - min_value) + hue = ((1 - normalized) + h_min) * (h_max - h_min) + r, g, b = colorsys.hsv_to_rgb(hue, 1.0, 1.0) + return Style(color=Color.from_rgb(r * 255, g * 255, b * 255)) + + return heat + + +def default_highlight(): + """Generates a default style for highlights.""" + return Style(underline=True) diff --git a/python/ovs/flowviz/format.py b/python/ovs/flowviz/format.py new file mode 100644 index 000000000..70af2fa26 --- /dev/null +++ b/python/ovs/flowviz/format.py @@ -0,0 +1,371 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Flow formatting framework. + +This file defines a simple flow formatting framework. It's comprised of 3 +classes: FlowStyle, FlowFormatter and FlowBuffer. + +The FlowStyle arranges opaque style objects in a dictionary that can be queried +to determine what style a particular key-value should be formatted with. +That way, a particular implementation can represent its style using their own +object. + +The FlowBuffer is an abstract class and must be derived by particular +implementations. It should know how to append parts of a flow using a style. +Only here the type of the style is relevant. + +When asked to format a flow, the FlowFormatter will determine which style +the flow must be formatted with and call FlowBuffer functions with each part +of the flow and their corresponding style. +""" + + +class FlowStyle: + """A FlowStyle determines the KVStyle to use for each key value in a flow. + + Styles are internally represented by a dictionary. + In order to determine the style for a "key", the following items in the + dictionary are fetched: + - key.highlighted.{key} (if key is found in hightlighted) + - key.highlighted (if key is found in hightlighted) + - key.{key} + - key + - default + + In order to determine the style for a "value", the following items in the + dictionary are fetched: + - value.highlighted.{key} (if key is found in hightlighted) + - value.highlighted.type{value.__class__.__name__} + - value.highlighted + (if key is found in hightlighted) + - value.{key} + - value.type.{value.__class__.__name__} + - value + - default + + The actual type of the style object stored for each item above is opaque + to this class and it depends on the particular FlowFormatter child class + that will handle them. Even callables can be stored, if so they will be + called with the value of the field that is to be formatted and the return + object will be used as style. + + Additionally, the following style items can be defined: + - delim: for delimiters + - delim.highlighted: for delimiters of highlighted key-values + """ + + def __init__(self, initial=None): + self._styles = initial if initial is not None else dict() + + def __len__(self): + return len(self._styles) + + def set_flag_style(self, kvstyle): + self._styles["flag"] = kvstyle + + def set_delim_style(self, kvstyle, highlighted=False): + if highlighted: + self._styles["delim.highlighted"] = kvstyle + else: + self._styles["delim"] = kvstyle + + def set_default_key_style(self, kvstyle, highlighted=False): + if highlighted: + self._styles["key.highlighted"] = kvstyle + else: + self._styles["key"] = kvstyle + + def set_default_value_style(self, kvstyle, highlighted=False): + if highlighted: + self._styles["value.highlighted"] = kvstyle + else: + self._styles["value"] = kvstyle + + def set_key_style(self, key, kvstyle, highlighted=False): + if highlighted: + self._styles["key.highlighted.{}".format(key)] = kvstyle + else: + self._styles["key.{}".format(key)] = kvstyle + + def set_value_style(self, key, kvstyle, highlighted=None): + if highlighted: + self._styles["value.highlighted.{}".format(key)] = kvstyle + else: + self._styles["value.{}".format(key)] = kvstyle + + def set_value_type_style(self, name, kvstyle, highlighted=None): + if highlighted: + self._styles["value.highlighted.type.{}".format(name)] = kvstyle + else: + self._styles["value.type.{}".format(name)] = kvstyle + + def get(self, key): + return self._styles.get(key) + + def get_delim_style(self, highlighted=False): + delim_style_lookup = ["delim.highlighted"] if highlighted else [] + delim_style_lookup.extend(["delim", "default"]) + return next( + ( + self._styles.get(s) + for s in delim_style_lookup + if self._styles.get(s) + ), + None, + ) + + def get_flag_style(self): + return self._styles.get("flag") or self._styles.get("default") + + def get_key_style(self, kv, highlighted=False): + key = kv.key + + key_style_lookup = ( + ["key.highlighted.%s" % key, "key.highlighted"] + if highlighted + else [] + ) + key_style_lookup.extend(["key.%s" % key, "key", "default"]) + + style = next( + ( + self._styles.get(s) + for s in key_style_lookup + if self._styles.get(s) + ), + None, + ) + if callable(style): + return style(kv.meta.kstring) + return style + + def get_value_style(self, kv, highlighted=False): + key = kv.key + value_type = kv.value.__class__.__name__.lower() + value_style_lookup = ( + [ + "value.highlighted.%s" % key, + "value.highlighted.type.%s" % value_type, + "value.highlighted", + ] + if highlighted + else [] + ) + value_style_lookup.extend( + [ + "value.%s" % key, + "value.type.%s" % value_type, + "value", + "default", + ] + ) + + style = next( + ( + self._styles.get(s) + for s in value_style_lookup + if self._styles.get(s) + ), + None, + ) + if callable(style): + return style(kv.meta.vstring) + return style + + +class FlowFormatter: + """FlowFormatter is a base class for Flow Formatters.""" + + def __init__(self): + self._highlighted = list() + + def _style_from_opts(self, opts, opts_key, style_constructor): + """Create style object from options. + + Args: + opts (dict): Options dictionary + opts_key (str): The options style key to extract + (e.g: console or html) + style_constructor(callable): A callable that creates a derived + style object + """ + if not opts or not opts.get("style"): + return None + + section_name = ".".join(["styles", opts.get("style")]) + if section_name not in opts.get("config").sections(): + return None + + config = opts.get("config")[section_name] + style = {} + for key in config: + (_, console, style_full_key) = key.partition(opts_key + ".") + if not console: + continue + + (style_key, _, prop) = style_full_key.rpartition(".") + if not prop or not style_key: + raise Exception("malformed style config: {}".format(key)) + + if not style.get(style_key): + style[style_key] = {} + style[style_key][prop] = config[key] + + return FlowStyle({k: style_constructor(**v) for k, v in style.items()}) + + def format_flow(self, buf, flow, style_obj=None, highlighted=None): + """Formats the flow into the provided buffer. + + Args: + buf (FlowBuffer): the flow buffer to append to + flow (ovs_dbg.OFPFlow): the flow to format + style_obj (FlowStyle): Optional; style to use + highlighted (list): Optional; list of KeyValues to highlight + """ + last_printed_pos = 0 + + if style_obj: + style_obj = style_obj or FlowStyle() + for section in sorted(flow.sections, key=lambda x: x.pos): + buf.append_extra( + flow.orig[last_printed_pos : section.pos], + style=style_obj.get("default"), + ) + self.format_kv_list( + buf, section.data, section.string, style_obj, highlighted + ) + last_printed_pos = section.pos + len(section.string) + else: + # Don't pay the cost of formatting each section one by one. + buf.append_extra(flow.orig.strip(), None) + + def format_kv_list(self, buf, kv_list, full_str, style_obj, highlighted): + """Format a KeyValue List. + + Args: + buf (FlowBuffer): a FlowBuffer to append formatted KeyValues to + kv_list (list[KeyValue]: the KeyValue list to format + full_str (str): the full string containing all k-v + style_obj (FlowStyle): a FlowStyle object to use + highlighted (list): Optional; list of KeyValues to highlight + """ + for i, kv in enumerate(kv_list): + written = self.format_kv( + buf, kv, style_obj=style_obj, highlighted=highlighted + ) + + end = ( + kv_list[i + 1].meta.kpos + if i < (len(kv_list) - 1) + else len(full_str) + ) + + buf.append_extra( + full_str[(kv.meta.kpos + written) : end].rstrip("\n\r"), + style=style_obj.get("default"), + ) + + def format_kv(self, buf, kv, style_obj, highlighted=None): + """Format a KeyValue + + A formatted keyvalue has the following parts: + {key}{delim}{value}[{delim}] + + Args: + buf (FlowBuffer): buffer to append the KeyValue to + kv (KeyValue): The KeyValue to print + style_obj (FlowStyle): The style object to use + highlighted (list): Optional; list of KeyValues to highlight + + Returns the number of printed characters. + """ + ret = 0 + key = kv.meta.kstring + is_highlighted = ( + key in [k.key for k in highlighted] if highlighted else False + ) + + key_style = style_obj.get_key_style(kv, is_highlighted) + buf.append_key(kv, key_style) # format value + ret += len(key) + + if not kv.meta.vstring: + return ret + + if kv.meta.delim not in ("\n", "\t", "\r", ""): + buf.append_delim(kv, style_obj.get_delim_style(is_highlighted)) + ret += len(kv.meta.delim) + + value_style = style_obj.get_value_style(kv, is_highlighted) + buf.append_value(kv, value_style) # format value + ret += len(kv.meta.vstring) + + if kv.meta.end_delim: + buf.append_end_delim(kv, style_obj.get_delim_style(is_highlighted)) + ret += len(kv.meta.end_delim) + + return ret + + +class FlowBuffer: + """A FlowBuffer is a base class for format buffers. + + Childs must implement the following methods: + append_key(self, kv, style) + append_value(self, kv, style) + append_delim(self, delim, style) + append_end_delim(self, delim, style) + append_extra(self, extra, style) + """ + + def append_key(self, kv, style): + """Append a key. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError + + def append_delim(self, kv, style): + """Append a delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError + + def append_end_delim(self, kv, style): + """Append an end delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError + + def append_value(self, kv, style): + """Append a value. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError + + def append_extra(self, extra, style): + """Append extra string. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError diff --git a/python/ovs/flowviz/main.py b/python/ovs/flowviz/main.py index 64b0e8a0a..723c71fa7 100644 --- a/python/ovs/flowviz/main.py +++ b/python/ovs/flowviz/main.py @@ -12,10 +12,30 @@ # See the License for the specific language governing permissions and # limitations under the License. +import configparser import click import os from ovs.flow.filter import OFFilter +from ovs.dirs import PKGDATADIR + +_default_config_file = "ovs-flowviz.conf" +_default_config_path = next( + ( + p + for p in [ + os.path.join( + os.getenv("HOME"), ".config", "ovs", _default_config_file + ), + os.path.join(PKGDATADIR, _default_config_file), + os.path.abspath( + os.path.join(os.path.dirname(__file__), _default_config_file) + ), + ] + if os.path.exists(p) + ), + "", +) class Options(dict): @@ -48,6 +68,20 @@ def validate_input(ctx, param, value): @click.group( context_settings=dict(help_option_names=["-h", "--help"]), ) +@click.option( + "-c", + "--config", + help="Use config file", + type=click.Path(), + default=_default_config_path, + show_default=True, +) +@click.option( + "--style", + help="Select style (defined in config file)", + default=None, + show_default=True, +) @click.option( "-i", "--input", @@ -69,8 +103,17 @@ def validate_input(ctx, param, value): type=str, show_default=False, ) +@click.option( + "-l", + "--highlight", + help="Highlight flows that match the filter expression." + "Run 'ovs-flowviz filter' for a detailed description of the filtering " + "syntax", + type=str, + show_default=False, +) @click.pass_context -def maincli(ctx, filename, filter): +def maincli(ctx, config, style, filename, filter, highlight): """ OpenvSwitch flow visualization utility. @@ -86,6 +129,19 @@ def maincli(ctx, filename, filter): except Exception as e: raise click.BadParameter("Wrong filter syntax: {}".format(e)) + if highlight: + try: + ctx.obj["highlight"] = OFFilter(highlight) + except Exception as e: + raise click.BadParameter("Wrong filter syntax: {}".format(e)) + + config_file = config or _default_config_path + parser = configparser.ConfigParser() + parser.read(config_file) + + ctx.obj["config"] = parser + ctx.obj["style"] = style + @maincli.command(hidden=True) @click.pass_context diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py index ed2f82065..a1cba0135 100644 --- a/python/ovs/flowviz/odp/cli.py +++ b/python/ovs/flowviz/odp/cli.py @@ -16,6 +16,7 @@ import click from ovs.flowviz.main import maincli from ovs.flowviz.process import ( + ConsoleProcessor, DatapathFactory, JSONProcessor, ) @@ -40,3 +41,27 @@ def json(opts): proc = JSONPrint(opts) proc.process() print(proc.json_string()) + + +class DPConsoleProcessor(DatapathFactory, ConsoleProcessor): + def __init__(self, opts, heat_map): + super().__init__(opts, heat_map) + + +@datapath.command() +@click.option( + "-h", + "--heat-map", + is_flag=True, + default=False, + show_default=True, + help="Create heat-map with packet and byte counters", +) +@click.pass_obj +def console(opts, heat_map): + """Print the flows in the console with some style.""" + proc = DPConsoleProcessor( + opts, heat_map=["packets", "bytes"] if heat_map else [] + ) + proc.process() + proc.print() diff --git a/python/ovs/flowviz/ofp/cli.py b/python/ovs/flowviz/ofp/cli.py index b9a2a8aad..a399dbd82 100644 --- a/python/ovs/flowviz/ofp/cli.py +++ b/python/ovs/flowviz/ofp/cli.py @@ -16,6 +16,7 @@ import click from ovs.flowviz.main import maincli from ovs.flowviz.process import ( + ConsoleProcessor, OpenFlowFactory, JSONProcessor, ) @@ -40,3 +41,28 @@ def json(opts): proc = JSONPrint(opts) proc.process() print(proc.json_string()) + + +class OFConsoleProcessor(OpenFlowFactory, ConsoleProcessor): + def __init__(self, opts, heat_map): + super().__init__(opts, heat_map) + + +@openflow.command() +@click.option( + "-h", + "--heat-map", + is_flag=True, + default=False, + show_default=True, + help="Create heat-map with packet and byte counters", +) +@click.pass_obj +def console(opts, heat_map): + """Print the flows in the console with some style.""" + proc = OFConsoleProcessor( + opts, + heat_map=["n_packets", "n_bytes"] if heat_map else [], + ) + proc.process() + proc.print() diff --git a/python/ovs/flowviz/process.py b/python/ovs/flowviz/process.py index 3e520e431..e0158de9f 100644 --- a/python/ovs/flowviz/process.py +++ b/python/ovs/flowviz/process.py @@ -20,6 +20,13 @@ from ovs.flow.decoders import FlowEncoder from ovs.flow.odp import ODPFlow from ovs.flow.ofp import OFPFlow +from ovs.flowviz.console import ( + ConsoleFormatter, + default_highlight, + heat_pallete, + file_header, +) + class FileProcessor(object): """Base class for file-based Flow processing. It is able to create flows @@ -134,21 +141,24 @@ class FileProcessor(object): self.end() -class DatapathFactory(): +class DatapathFactory: """A mixin class that creates Datapath flows.""" def create_flow(self, line, idx): # Skip strings commonly found in Datapath flow dumps. - if any(s in line for s in [ - "flow-dump from the main thread", - "flow-dump from pmd on core", - ]): + if any( + s in line + for s in [ + "flow-dump from the main thread", + "flow-dump from pmd on core", + ] + ): return None return ODPFlow(line, idx) -class OpenFlowFactory(): +class OpenFlowFactory: """A mixin class that creates OpenFlow flows.""" def create_flow(self, line, idx): @@ -190,3 +200,64 @@ class JSONProcessor(FileProcessor): indent=4, cls=FlowEncoder, ) + + +class ConsoleProcessor(FileProcessor): + """A generic Console Processor that prints flows into the console""" + + def __init__(self, opts, heat_map=[]): + super().__init__(opts) + self.heat_map = heat_map + self.console = ConsoleFormatter(opts) + if len(self.console.style) == 0 and self.opts.get("highlight"): + # Add some style to highlights or else they won't be seen. + self.console.style.set_default_value_style( + default_highlight(), True + ) + self.console.style.set_default_key_style(default_highlight(), True) + + self.flows = dict() # Dictionary of flow-lists, one per file. + self.min_max = dict() # Used for heat-map calculation. + + def start_file(self, name, filename): + self.flows_list = list() + if len(self.heat_map) > 0: + self.min = [-1] * len(self.heat_map) + self.max = [0] * len(self.heat_map) + + def stop_file(self, name, filename): + self.flows[name] = self.flows_list + if len(self.heat_map) > 0: + self.min_max[name] = (self.min, self.max) + + def process_flow(self, flow, name): + # Running calculation of min and max values for all the fields that + # take place in the heatmap. + for i, field in enumerate(self.heat_map): + val = flow.info.get(field) + if self.min[i] == -1 or val < self.min[i]: + self.min[i] = val + if val > self.max[i]: + self.max[i] = val + + self.flows_list.append(flow) + + def print(self): + for name, flows in self.flows.items(): + self.console.console.print("\n") + self.console.console.print(file_header(name)) + + if len(self.heat_map) > 0 and len(self.flows) > 0: + for i, field in enumerate(self.heat_map): + (min_val, max_val) = self.min_max[name][i] + self.console.style.set_value_style( + field, heat_pallete(min_val, max_val) + ) + + for flow in flows: + high = None + if self.opts.get("highlight"): + result = self.opts.get("highlight").evaluate(flow) + if result: + high = result.kv + self.console.print_flow(flow, high) diff --git a/python/setup.py b/python/setup.py index 4b9c751d2..76f9fc820 100644 --- a/python/setup.py +++ b/python/setup.py @@ -113,9 +113,11 @@ setup_args = dict( extras_require={':sys_platform == "win32"': ['pywin32 >= 1.0'], 'dns': ['unbound'], 'flow': flow_extras_require, - 'flowviz': [*flow_extras_require, 'click'], + 'flowviz': + [*flow_extras_require, 'click', 'rich'], }, scripts=["ovs/flowviz/ovs-flowviz"], + include_package_data=True, ) try: From patchwork Tue May 7 14:30:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1932500 X-Patchwork-Delegate: echaudro@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FYu0j/j3; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VYggn2KmSz1xnT for ; Wed, 8 May 2024 00:31:17 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 1954360AEE; Tue, 7 May 2024 14:31:15 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id srORMBpcTAqv; Tue, 7 May 2024 14:31:11 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 7771960A91 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FYu0j/j3 Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 7771960A91; Tue, 7 May 2024 14:31:10 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3BC62C0077; Tue, 7 May 2024 14:31:10 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 96808C0037 for ; Tue, 7 May 2024 14:31:08 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 5FDAE406E6 for ; Tue, 7 May 2024 14:31:08 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 6ZkUi11LQ2xV for ; Tue, 7 May 2024 14:31:06 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org 3D8C340657 Authentication-Results: smtp4.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 3D8C340657 Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FYu0j/j3 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 3D8C340657 for ; Tue, 7 May 2024 14:31:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1715092264; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gS29DfrhsaELhFtHr5HifcrlCqyzQSpuibJWO36inJY=; b=FYu0j/j3K8ej+M+XBrn44F5VqOR8s88CDuU+EGg4xKld9eZ4zgRa8I3i3aLr8vEggH2evu OaWwRx3oY1CFCjbVni4sRMyrgG8dZiFLlcQ0jl4lnIjyGsm+XA3LIpT9bFF9F+RCdjZp6i ei8M+4GxZs0NC9oR711b2XKgdFSvZFk= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-111-s1XOieROMs-3Nl7ojFi-Dg-1; Tue, 07 May 2024 10:31:02 -0400 X-MC-Unique: s1XOieROMs-3Nl7ojFi-Dg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 087403C0D198 for ; Tue, 7 May 2024 14:31:02 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.88]) by smtp.corp.redhat.com (Postfix) with ESMTP id 17B623C25; Tue, 7 May 2024 14:31:00 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 7 May 2024 16:30:30 +0200 Message-ID: <20240507143044.3056232-5-amorenoz@redhat.com> In-Reply-To: <20240507143044.3056232-1-amorenoz@redhat.com> References: <20240507143044.3056232-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v4 04/12] python: ovs: flowviz: Add default config file. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" It has two basic styles defined: "dark" and "light" intended for dark and light terminals. Examples: $ ovs-flowviz -i /tmp/dpflows --style=dark datapath console $ ovs-flowviz -i /tmp/ofpflows --style=light openflow console Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 5 +- python/ovs/flowviz/ovs-flowviz.conf | 94 +++++++++++++++++++++++++++++ python/setup.py | 1 + 3 files changed, 99 insertions(+), 1 deletion(-) create mode 100644 python/ovs/flowviz/ovs-flowviz.conf diff --git a/python/automake.mk b/python/automake.mk index bd53c5405..23212e4b5 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -89,7 +89,8 @@ EXTRA_DIST += \ python/ovs/compat/sortedcontainers/LICENSE \ python/README.rst \ python/setup.py \ - python/test_requirements.txt + python/test_requirements.txt \ + python/ovs/flowviz/ovs-flowviz.conf # C extension support. EXTRA_DIST += python/ovs/_json.c @@ -109,6 +110,8 @@ FLAKE8_PYFILES += \ python/setup.py nobase_pkgdata_DATA = $(ovs_pyfiles) $(ovstest_pyfiles) $(ovs_flowviz) +nobase_pkgdata_DATA += python/ovs/flowviz/ovs-flowviz.conf + ovs-install-data-local: $(MKDIR_P) python/ovs sed \ diff --git a/python/ovs/flowviz/ovs-flowviz.conf b/python/ovs/flowviz/ovs-flowviz.conf new file mode 100644 index 000000000..3acd0a29e --- /dev/null +++ b/python/ovs/flowviz/ovs-flowviz.conf @@ -0,0 +1,94 @@ +# Create any number of styles.{style_name} sections with a defined style. +# +# Syntax: +# +# [FORMAT].[PORTION].[SELECTOR].[ELEMENT] = [VALUE] +# +# * FORMAT: console +# * PORTION: The portion of the flow that the style applies to +# - key: Selects how to print the key of a KeyValue pair +# - key: Selects how to print the value of a KeyValue pair +# - flag: Selects how to print the a flag +# - delim: Selects how to print the delimiters around key and values +# +# * SELECTOR: +# - highlighted: to apply when the key is highlighted +# - type.{TYPE}: to apply when the value matches a type +# (special types such as IPAddress or EthMask can be used) +# (only aplicable to 'value') +# - {key_name}: to apply when the key matches the key_name +# +# Console Styles +# ============== +# * ELEMENT: +# - color: defines the color in hex or a color rich starndard ones [1] +# - underline: if set to "true", the selected portion will be underlined +# +#[1] https://rich.readthedocs.io/en/stable/appendix/colors.html#standard-colors + +[styles.dark] + +# defaults for key-values +console.key.color = #5D86BA +console.value.color= #B0C4DE +console.delim.color= #B0C4DE +console.default.color= #FFFFFF + +# defaults for special types +console.value.type.IPAddress.color = #008700 +console.value.type.IPMask.color = #008700 +console.value.type.EthMask.color = #008700 + +# dim some long arguments +console.value.ct.color = grey66 +console.value.ufid.color = grey66 +console.value.clone.color = grey66 +console.value.controller.color = grey66 + +# highlight flags +console.flag.color = #875fff + +# show drop and recirculations +console.key.drop.color = red +console.key.resubmit.color = #00d700 +console.key.output.color = #00d700 +console.value.output.color = #00d700 + +# highlights +console.key.highlighted.color = red +console.key.highlighted.underline = true +console.value.highlighted.underline = true +console.delim.highlighted.underline = true + + +[styles.light] +# If a color is omitted, the default terminal color will be used +# highlight keys +console.key.color = blue + +# special types +console.value.type.IPAddress.color = #008700 +console.value.type.IPMask.color = #008700 +console.value.type.EthMask.color = #008700 + +# dim long arguments +console.value.ct.color = bright_black +console.value.ufid.color = #870000 +console.value.clone.color = bright_black +console.value.controller.color = bright_black + +# highlight flags +console.flag.color = #00005F + +# show drop and recirculations +console.key.drop.color = red +console.key.resubmit.color = #00d700 +console.key.output.color = #005f00 +console.value.output.color = #00d700 + +# highlights +console.key.highlighted.color = #f20905 +console.value.highlighted.color = #f20905 +console.key.highlighted.underline = true +console.value.highlighted.underline = true +console.delim.highlighted.underline = true diff --git a/python/setup.py b/python/setup.py index 76f9fc820..c734f68f3 100644 --- a/python/setup.py +++ b/python/setup.py @@ -117,6 +117,7 @@ setup_args = dict( [*flow_extras_require, 'click', 'rich'], }, scripts=["ovs/flowviz/ovs-flowviz"], + data_files=["ovs/flowviz/ovs-flowviz.conf"], include_package_data=True, ) From patchwork Tue May 7 14:30:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1932503 X-Patchwork-Delegate: echaudro@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ZO+4FFHc; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VYggt2WFLz1xnT for ; Wed, 8 May 2024 00:31:22 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 123EB4169C; Tue, 7 May 2024 14:31:20 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id paY05PYvOvb9; Tue, 7 May 2024 14:31:15 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 2C56D416C5 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ZO+4FFHc Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 2C56D416C5; Tue, 7 May 2024 14:31:14 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D7FCBC0DD4; Tue, 7 May 2024 14:31:13 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 072EEC0037 for ; Tue, 7 May 2024 14:31:13 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 8341C82190 for ; Tue, 7 May 2024 14:31:12 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id ED-TnVd77J26 for ; Tue, 7 May 2024 14:31:09 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp1.osuosl.org E797882180 Authentication-Results: smtp1.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org E797882180 Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ZO+4FFHc Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id E797882180 for ; Tue, 7 May 2024 14:31:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1715092264; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UUeJjWP0t76iiMi/6PhE9393PXFWFtNwfPRuPKjr+Fc=; b=ZO+4FFHcOlJvF2Psa4gDS9scDok76xaI94EjVVpPVbMGzZMwQ6my5qttdeit6PuxclSyla EuL82VNEgc0vqOLRuGWD8gBBg8h8FcvdTvw0piUTU3PvY5O36hzx2Q+UK3tFCHhDKJ/Tex Z+XTIAYUMVBJfqXF/6295m7g3eqk3GM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-146-2BusNkYqOBi2rmp_j8DsYQ-1; Tue, 07 May 2024 10:31:03 -0400 X-MC-Unique: 2BusNkYqOBi2rmp_j8DsYQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2C47318172C0 for ; Tue, 7 May 2024 14:31:03 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.88]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4656A3C25; Tue, 7 May 2024 14:31:02 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 7 May 2024 16:30:31 +0200 Message-ID: <20240507143044.3056232-6-amorenoz@redhat.com> In-Reply-To: <20240507143044.3056232-1-amorenoz@redhat.com> References: <20240507143044.3056232-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v4 05/12] python: ovs: flowviz: Add html formatting. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add a HTML Formatter and use it to print OpenFlow flows in an HTML list with table links. Examples $ ovs-flowviz -i offlows.txt --highlight "drop" openflow html > /tmp/flows.html $ ovs-flowviz -i offlows.txt --filter "n_packets > 0" openflow html > /tmp/flows.html Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 3 +- python/ovs/flowviz/html_format.py | 136 ++++++++++++++++++++++++++++ python/ovs/flowviz/ofp/cli.py | 10 ++ python/ovs/flowviz/ofp/html.py | 80 ++++++++++++++++ python/ovs/flowviz/ovs-flowviz.conf | 16 +++- 5 files changed, 243 insertions(+), 2 deletions(-) create mode 100644 python/ovs/flowviz/html_format.py create mode 100644 python/ovs/flowviz/ofp/html.py diff --git a/python/automake.mk b/python/automake.mk index 23212e4b5..0487494d0 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -67,15 +67,16 @@ ovs_flowviz = \ python/ovs/flowviz/__init__.py \ python/ovs/flowviz/console.py \ python/ovs/flowviz/format.py \ + python/ovs/flowviz/html_format.py \ python/ovs/flowviz/main.py \ python/ovs/flowviz/odp/__init__.py \ python/ovs/flowviz/odp/cli.py \ python/ovs/flowviz/ofp/__init__.py \ python/ovs/flowviz/ofp/cli.py \ + python/ovs/flowviz/ofp/html.py \ python/ovs/flowviz/ovs-flowviz \ python/ovs/flowviz/process.py - # These python files are used at build time but not runtime, # so they are not installed. EXTRA_DIST += \ diff --git a/python/ovs/flowviz/html_format.py b/python/ovs/flowviz/html_format.py new file mode 100644 index 000000000..ebfa65c34 --- /dev/null +++ b/python/ovs/flowviz/html_format.py @@ -0,0 +1,136 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from ovs.flowviz.format import FlowFormatter, FlowBuffer, FlowStyle + + +class HTMLStyle: + """HTMLStyle defines a style for html-formatted flows. + + Args: + color(str): Optional; a string representing the CSS color to use + anchor_gen(callable): Optional; a callable to be used to generate the + href + """ + + def __init__(self, color=None, anchor_gen=None): + self.color = color + self.anchor_gen = anchor_gen + + +class HTMLBuffer(FlowBuffer): + """HTMLBuffer implementes FlowBuffer to provide html-based flow formatting. + + Each flow gets formatted as: +
...
+ """ + + def __init__(self): + self._text = "" + + @property + def text(self): + return self._text + + def _append(self, string, color, href): + """Append a key a string""" + style = ' style="color:{}"'.format(color) if color else "" + self._text += "".format(style) + if href: + self._text += "".format(href) + self._text += string + if href: + self._text += "" + self._text += "" + + def append_key(self, kv, style): + """Append a key. + Args: + kv (KeyValue): the KeyValue instance to append + style (HTMLStyle): the style to use + """ + href = style.anchor_gen(kv) if (style and style.anchor_gen) else "" + return self._append( + kv.meta.kstring, style.color if style else "", href + ) + + def append_delim(self, kv, style): + """Append a delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (HTMLStyle): the style to use + """ + href = style.anchor_gen(kv) if (style and style.anchor_gen) else "" + return self._append(kv.meta.delim, style.color if style else "", href) + + def append_end_delim(self, kv, style): + """Append an end delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (HTMLStyle): the style to use + """ + href = style.anchor_gen(kv) if (style and style.anchor_gen) else "" + return self._append( + kv.meta.end_delim, style.color if style else "", href + ) + + def append_value(self, kv, style): + """Append a value. + Args: + kv (KeyValue): the KeyValue instance to append + style (HTMLStyle): the style to use + """ + href = style.anchor_gen(kv) if (style and style.anchor_gen) else "" + return self._append( + kv.meta.vstring, style.color if style else "", href + ) + + def append_extra(self, extra, style): + """Append extra string. + Args: + kv (KeyValue): the KeyValue instance to append + style (HTMLStyle): the style to use + """ + return self._append(extra, style.color if style else "", "") + + +class HTMLFormatter(FlowFormatter): + """Formts a flow in HTML Format.""" + + default_style_obj = FlowStyle( + { + "value.resubmit": HTMLStyle( + anchor_gen=lambda x: "#table_{}".format(x.value["table"]) + ), + "default": HTMLStyle(), + } + ) + + def __init__(self, opts=None): + super(HTMLFormatter, self).__init__() + self.style = ( + self._style_from_opts(opts, "html", HTMLStyle) or FlowStyle() + ) + + def format_flow(self, buf, flow, highlighted=None): + """Formats the flow into the provided buffer as a html object. + + Args: + buf (FlowBuffer): the flow buffer to append to + flow (ovs_dbg.OFPFlow): the flow to format + highlighted (list): Optional; list of KeyValues to highlight + """ + return super(HTMLFormatter, self).format_flow( + buf, flow, self.style, highlighted + ) diff --git a/python/ovs/flowviz/ofp/cli.py b/python/ovs/flowviz/ofp/cli.py index a399dbd82..2cd8e1c89 100644 --- a/python/ovs/flowviz/ofp/cli.py +++ b/python/ovs/flowviz/ofp/cli.py @@ -15,6 +15,7 @@ import click from ovs.flowviz.main import maincli +from ovs.flowviz.ofp.html import HTMLProcessor from ovs.flowviz.process import ( ConsoleProcessor, OpenFlowFactory, @@ -66,3 +67,12 @@ def console(opts, heat_map): ) proc.process() proc.print() + + +@openflow.command() +@click.pass_obj +def html(opts): + """Print the flows in an linked HTML list arranged by tables.""" + processor = HTMLProcessor(opts) + processor.process() + print(processor.html()) diff --git a/python/ovs/flowviz/ofp/html.py b/python/ovs/flowviz/ofp/html.py new file mode 100644 index 000000000..a66f5fe8e --- /dev/null +++ b/python/ovs/flowviz/ofp/html.py @@ -0,0 +1,80 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from ovs.flowviz.html_format import HTMLBuffer, HTMLFormatter, HTMLStyle +from ovs.flowviz.process import ( + OpenFlowFactory, + FileProcessor, +) + + +class HTMLProcessor(OpenFlowFactory, FileProcessor): + """File processor that prints Openflow tables in HTML.""" + + def __init__(self, opts): + super().__init__(opts) + self.data = dict() + + def start_file(self, name, filename): + self.tables = dict() + + def stop_file(self, name, filename): + self.data[name] = self.tables + + def process_flow(self, flow, name): + table = flow.info.get("table") or 0 + if not self.tables.get(table): + self.tables[table] = list() + self.tables[table].append(flow) + + def html(self): + html_obj = "" + for name, tables in self.data.items(): + name = name.replace(" ", "_") + html_obj += "

{}

".format(name) + html_obj += "
" + for table, flows in tables.items(): + formatter = HTMLFormatter(self.opts) + + def anchor(x): + return "#table_%s_%s" % (name, x.value["table"]) + + formatter.style.set_value_style( + "resubmit", + HTMLStyle( + formatter.style.get("value.resubmit"), + anchor_gen=anchor, + ), + ) + html_obj += ( + "

Table {table}

".format( + name=name, table=table + ) + ) + html_obj += "
    ".format(table) + for flow in flows: + html_obj += "
  • ".format(flow.id) + highlighted = None + if self.opts.get("highlight"): + result = self.opts.get("highlight").evaluate(flow) + if result: + highlighted = result.kv + buf = HTMLBuffer() + formatter.format_flow(buf, flow, highlighted) + html_obj += buf.text + html_obj += "
    • " + html_obj += "
" + html_obj += "
" + + return html_obj diff --git a/python/ovs/flowviz/ovs-flowviz.conf b/python/ovs/flowviz/ovs-flowviz.conf index 3acd0a29e..165c453ec 100644 --- a/python/ovs/flowviz/ovs-flowviz.conf +++ b/python/ovs/flowviz/ovs-flowviz.conf @@ -4,7 +4,7 @@ # # [FORMAT].[PORTION].[SELECTOR].[ELEMENT] = [VALUE] # -# * FORMAT: console +# * FORMAT: console or html # * PORTION: The portion of the flow that the style applies to # - key: Selects how to print the key of a KeyValue pair # - key: Selects how to print the value of a KeyValue pair @@ -25,6 +25,11 @@ # - underline: if set to "true", the selected portion will be underlined # #[1] https://rich.readthedocs.io/en/stable/appendix/colors.html#standard-colors +# +# HTML Styles +# ============== +# * ELEMENT: +# - color: defines the color in hex format [styles.dark] @@ -92,3 +97,12 @@ console.value.highlighted.color = #f20905 console.key.highlighted.underline = true console.value.highlighted.underline = true console.delim.highlighted.underline = true + +# html +html.key.color = #00005f +html.value.color = #870000 +html.key.resubmit.color = #00d700 +html.key.output.color = #005f00 +html.value.output.color = #00d700 +html.key.highlighted.color = #FF00FF +html.value.highlighted.color = #FF00FF From patchwork Tue May 7 14:30:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1932501 X-Patchwork-Delegate: echaudro@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=O2U3WQlN; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VYggq4B97z1xnT for ; Wed, 8 May 2024 00:31:19 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 841D460AAE; Tue, 7 May 2024 14:31:17 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id Ks4MyQ55q350; Tue, 7 May 2024 14:31:12 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6B8C16078F Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=O2U3WQlN Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 6B8C16078F; Tue, 7 May 2024 14:31:12 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2DFA3C0DD6; Tue, 7 May 2024 14:31:12 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9DFCAC0037 for ; Tue, 7 May 2024 14:31:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 6CB5D4073E for ; Tue, 7 May 2024 14:31:10 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id Q1aDjlb8B9pg for ; Tue, 7 May 2024 14:31:08 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org D4962406DB Authentication-Results: smtp4.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org D4962406DB Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=O2U3WQlN Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id D4962406DB for ; Tue, 7 May 2024 14:31:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1715092266; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hbsrznNyBJty/BOfoqm0+TKyw6qRAMZ+8SvKgSOXKhc=; b=O2U3WQlNtPnK1wdMtNgV/TvagGAICMjl7+87jEQrhcitZbsM6YPNo5m/7fVU6tvcLuMnrE xmND2K2MpLkrRMTSoEUFm4/zpvnBzlwe6a7hxwVZ43e3gTJp7L/wvvq1dKebX8mogjaLST 1+tw+SkXkokdp//PNC9bbJRIAmaypO8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-633-oYZF9uvFPs6j39cq8LNy0Q-1; Tue, 07 May 2024 10:31:04 -0400 X-MC-Unique: oYZF9uvFPs6j39cq8LNy0Q-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7ED8B185A783 for ; Tue, 7 May 2024 14:31:04 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.88]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6DE8D3C25; Tue, 7 May 2024 14:31:03 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 7 May 2024 16:30:32 +0200 Message-ID: <20240507143044.3056232-7-amorenoz@redhat.com> In-Reply-To: <20240507143044.3056232-1-amorenoz@redhat.com> References: <20240507143044.3056232-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v4 06/12] python: ovs: flowviz: Add datapath tree format. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Datapath flows can be arranged into a "tree"-like structure based on recirculation ids, e.g: recirc(0),eth(...),ipv4(...) actions=ct,recirc(0x42) \-> recirc(42),ct_state(0/0),eth(...),ipv4(...) actions=1 \-> recirc(42),ct_state(1/0),eth(...),ipv4(...) actions=userspace(...) This patch adds support for building such logical datapath trees in a format-agnostic way and adds support for console-based formatting supporting: - head-maps formatting of statistics - hash-based pallete of recirculation ids: each recirculation id is assigned a unique color to easily follow the sequence of related actions. - full-tree filtering: if a user specifies a filter, an entire subtree is filtered out if none of its branches satisfy it. Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 1 + python/ovs/flowviz/console.py | 21 +++ python/ovs/flowviz/odp/cli.py | 19 ++- python/ovs/flowviz/odp/tree.py | 291 +++++++++++++++++++++++++++++++++ 4 files changed, 331 insertions(+), 1 deletion(-) create mode 100644 python/ovs/flowviz/odp/tree.py diff --git a/python/automake.mk b/python/automake.mk index 0487494d0..b3fef9bed 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -71,6 +71,7 @@ ovs_flowviz = \ python/ovs/flowviz/main.py \ python/ovs/flowviz/odp/__init__.py \ python/ovs/flowviz/odp/cli.py \ + python/ovs/flowviz/odp/tree.py \ python/ovs/flowviz/ofp/__init__.py \ python/ovs/flowviz/ofp/cli.py \ python/ovs/flowviz/ofp/html.py \ diff --git a/python/ovs/flowviz/console.py b/python/ovs/flowviz/console.py index 4a3443360..93cd9b0b1 100644 --- a/python/ovs/flowviz/console.py +++ b/python/ovs/flowviz/console.py @@ -13,6 +13,8 @@ # limitations under the License. import colorsys +import itertools +import zlib from rich.console import Console from rich.color import Color @@ -170,6 +172,25 @@ def heat_pallete(min_value, max_value): return heat +def hash_pallete(hue, saturation, value): + """Generates a color pallete with the cartesian product + of the hsv values provided and returns a callable that assigns a color for + each value hash + """ + HSV_tuples = itertools.product(hue, saturation, value) + RGB_tuples = map(lambda x: colorsys.hsv_to_rgb(*x), HSV_tuples) + styles = [ + Style(color=Color.from_rgb(r * 255, g * 255, b * 255)) + for r, g, b in RGB_tuples + ] + + def get_style(string): + hash_val = zlib.crc32(bytes(str(string), "utf-8")) + return styles[hash_val % len(styles)] + + return get_style + + def default_highlight(): """Generates a default style for highlights.""" return Style(underline=True) diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py index a1cba0135..615bac55b 100644 --- a/python/ovs/flowviz/odp/cli.py +++ b/python/ovs/flowviz/odp/cli.py @@ -13,8 +13,8 @@ # limitations under the License. import click - from ovs.flowviz.main import maincli +from ovs.flowviz.odp.tree import ConsoleTreeProcessor from ovs.flowviz.process import ( ConsoleProcessor, DatapathFactory, @@ -65,3 +65,20 @@ def console(opts, heat_map): ) proc.process() proc.print() + + +@datapath.command() +@click.option( + "-h", + "--heat-map", + is_flag=True, + default=False, + show_default=True, + help="Create heat-map with packet and byte counters", +) +@click.pass_obj +def tree(opts, heat_map): + """Print the flows in a tree based on the 'recirc_id'.""" + processor = ConsoleTreeProcessor(opts) + processor.process() + processor.print(heat_map) diff --git a/python/ovs/flowviz/odp/tree.py b/python/ovs/flowviz/odp/tree.py new file mode 100644 index 000000000..d249e7d6d --- /dev/null +++ b/python/ovs/flowviz/odp/tree.py @@ -0,0 +1,291 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from rich.style import Style +from rich.text import Text +from rich.tree import Tree + +from ovs.flowviz.console import ( + ConsoleFormatter, + ConsoleBuffer, + hash_pallete, + heat_pallete, + file_header, +) +from ovs.flowviz.process import ( + DatapathFactory, + FileProcessor, +) + + +class TreeElem: + """Element in the tree. + Args: + children (list[TreeElem]): Optional, list of children + is_root (bool): Optional; whether this is the root elemen + """ + + def __init__(self, children=None, is_root=False): + self.children = children or list() + self.is_root = is_root + + def append(self, child): + self.children.append(child) + + +class FlowElem(TreeElem): + """An element that contains a flow. + Args: + flow (Flow): The flow that this element contains + children (list[TreeElem]): Optional, list of children + is_root (bool): Optional; whether this is the root elemen + """ + + def __init__(self, flow, children=None, is_root=False): + self.flow = flow + super(FlowElem, self).__init__(children, is_root) + + def evaluate_any(self, filter): + """Evaluate the filter on the element and all its children. + Args: + filter(OFFilter): the filter to evaluate + + Returns: + True if ANY of the flows (including self and children) evaluates + true + """ + if filter.evaluate(self.flow): + return True + + return any([child.evaluate_any(filter) for child in self.children]) + + +class FlowTree: + """A Flow tree is a a class that processes datapath flows into a tree based + on recirculation ids. + + Args: + flows (list[ODPFlow]): Optional, initial list of flows + root (TreeElem): Optional, root of the tree. + """ + + def __init__(self, flows=None, root=TreeElem(is_root=True)): + self._flows = {} + self.root = root + if flows: + for flow in flows: + self.add(flow) + + def add(self, flow): + """Add a flow""" + rid = flow.match.get("recirc_id") or 0 + if not self._flows.get(rid): + self._flows[rid] = list() + self._flows[rid].append(flow) + + def build(self): + """Build the flow tree.""" + self._build(self.root, 0) + + def traverse(self, callback): + """Traverses the tree calling callback on each element. + + callback: callable that accepts two TreeElem, the current one being + traversed and its parent + func callback(elem, parent): + ... + Note that "parent" can be None if it's the first element. + """ + self._traverse(self.root, None, callback) + + def _traverse(self, elem, parent, callback): + callback(elem, parent) + + for child in elem.children: + self._traverse(child, elem, callback) + + def _build(self, parent, recirc): + """Build the subtree starting at a specific recirc_id. + Recursive function. + + Args: + parent (TreeElem): parent of the (sub)tree + recirc(int): the recirc_id subtree to build + """ + flows = self._flows.get(recirc) + if not flows: + return + for flow in sorted( + flows, key=lambda x: x.info.get("packets") or 0, reverse=True + ): + next_recircs = self._get_next_recirc(flow) + + elem = self._new_elem(flow, parent) + parent.append(elem) + + for next_recirc in next_recircs: + self._build(elem, next_recirc) + + def _get_next_recirc(self, flow): + """Get the next recirc_ids from a Flow. + + The recirc_id is obtained from actions such as recirc, but also + complex actions such as check_pkt_len and sample + Args: + flow (ODPFlow): flow to get the recirc_id from. + Returns: + set of next recirculation ids. + """ + + # Helper function to find a recirc in a dictionary of actions. + def find_in_list(actions_list): + recircs = [] + for item in actions_list: + (action, value) = next(iter(item.items())) + if action == "recirc": + recircs.append(value) + elif action == "check_pkt_len": + recircs.extend(find_in_list(value.get("gt"))) + recircs.extend(find_in_list(value.get("le"))) + elif action == "clone": + recircs.extend(find_in_list(value)) + elif action == "sample": + recircs.extend(find_in_list(value.get("actions"))) + return recircs + + recircs = [] + recircs.extend(find_in_list(flow.actions)) + + return set(recircs) + + def _new_elem(self, flow, _): + """Creates a new TreeElem. + + Default implementation is to create a FlowElem. Derived classes can + override this method to return any derived TreeElem + """ + return FlowElem(flow) + + def filter(self, filter): + """Removes the first level subtrees if none of its sub-elements match + the filter. + + Args: + filter(OFFilter): filter to apply + """ + to_remove = list() + for l0 in self.root.children: + passes = l0.evaluate_any(filter) + if not passes: + to_remove.append(l0) + for elem in to_remove: + self.root.children.remove(elem) + + +class ConsoleTreeProcessor(DatapathFactory, FileProcessor): + def __init__(self, opts): + super().__init__(opts) + self.data = dict() + self.ofconsole = ConsoleFormatter(self.opts) + + # Generate a color pallete for cookies. + recirc_style_gen = hash_pallete( + hue=[x / 50 for x in range(0, 50)], saturation=[0.7], value=[0.8] + ) + + style = self.ofconsole.style + style.set_default_value_style(Style(color="grey66")) + style.set_key_style("output", Style(color="green")) + style.set_value_style("output", Style(color="green")) + style.set_value_style("recirc", recirc_style_gen) + style.set_value_style("recirc_id", recirc_style_gen) + + def start_file(self, name, filename): + self.tree = ConsoleTree(self.ofconsole, self.opts) + + def process_flow(self, flow, name): + self.tree.add(flow) + + def process(self): + super().process(False) + + def stop_file(self, name, filename): + self.data[name] = self.tree + + def print(self, heat_map): + for name, tree in self.data.items(): + self.ofconsole.console.print("\n") + self.ofconsole.console.print(file_header(name)) + tree.build() + if self.opts.get("filter"): + tree.filter(self.opts.get("filter")) + tree.print(heat_map) + + +class ConsoleTree(FlowTree): + """ConsoleTree is a FlowTree that prints the tree in the console. + + Args: + console (ConsoleFormatter): console to use for printing + opts (dict): Options dictionary + """ + + class ConsoleElem(FlowElem): + def __init__(self, flow=None, is_root=False): + self.tree = None + super(ConsoleTree.ConsoleElem, self).__init__( + flow, is_root=is_root + ) + + def __init__(self, console, opts): + self.console = console + self.opts = opts + super(ConsoleTree, self).__init__(root=self.ConsoleElem(is_root=True)) + + def _new_elem(self, flow, _): + """Override _new_elem to provide ConsoleElems""" + return self.ConsoleElem(flow) + + def _append_to_tree(self, elem, parent): + """Callback to be used for FlowTree._build + Appends the flow to the rich.Tree + """ + if elem.is_root: + elem.tree = Tree("Datapath Flows (logical)") + return + + buf = ConsoleBuffer(Text()) + highlighted = None + if self.opts.get("highlight"): + result = self.opts.get("highlight").evaluate(elem.flow) + if result: + highlighted = result.kv + self.console.format_flow(buf, elem.flow, highlighted) + elem.tree = parent.tree.add(buf.text) + + def print(self, heat=False): + """Print the Flow Tree. + Args: + heat (bool): Optional; whether heat-map style shall be applied + """ + if heat: + for field in ["packets", "bytes"]: + values = [] + for flow_list in self._flows.values(): + values.extend([f.info.get(field) or 0 for f in flow_list]) + self.console.style.set_value_style( + field, heat_pallete(min(values), max(values)) + ) + self.traverse(self._append_to_tree) + self.console.console.print(self.root.tree) From patchwork Tue May 7 14:30:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1932504 X-Patchwork-Delegate: echaudro@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=XTqiuExR; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VYggv5rJ9z1xnT for ; Wed, 8 May 2024 00:31:23 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id A8208407F8; Tue, 7 May 2024 14:31:21 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id uTasX4_3wV6A; Tue, 7 May 2024 14:31:16 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 18DDE40701 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=XTqiuExR Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 18DDE40701; Tue, 7 May 2024 14:31:15 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D148EC0DD4; Tue, 7 May 2024 14:31:14 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2676AC0037 for ; Tue, 7 May 2024 14:31:13 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id BEA4B407A0 for ; Tue, 7 May 2024 14:31:12 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 8-1Qum9YwxxV for ; Tue, 7 May 2024 14:31:09 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org 2FD2640698 Authentication-Results: smtp4.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 2FD2640698 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 2FD2640698 for ; Tue, 7 May 2024 14:31:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1715092267; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1VGvc77fhbk+ymiXHsfoiGKw1+5rb/X9c7mY9Qvi8ME=; b=XTqiuExRLLL5BSWB+c/EPWxSH8UhzB8I4Im9ekWyMQIuoY/Xj0790hFuOdlDZN3Cl7ev+q H6yHua7w1g/zoWB3lR3SaD/iuUQIVxSabAAT4gELoUqZiEshVV5qQAj3NwVJ18lQO8aN10 UHlJjmQZ86nTBMGi4wBst6iqDhwWKYM= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-463-ny7zgQOFNNOPN5ujoaABOg-1; Tue, 07 May 2024 10:31:05 -0400 X-MC-Unique: ny7zgQOFNNOPN5ujoaABOg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 96EA829AC00B for ; Tue, 7 May 2024 14:31:05 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.88]) by smtp.corp.redhat.com (Postfix) with ESMTP id BBE883C25; Tue, 7 May 2024 14:31:04 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 7 May 2024 16:30:33 +0200 Message-ID: <20240507143044.3056232-8-amorenoz@redhat.com> In-Reply-To: <20240507143044.3056232-1-amorenoz@redhat.com> References: <20240507143044.3056232-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v4 07/12] python: ovs: flowviz: Add OpenFlow logical view. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This view is interesting for debugging the logical pipeline. It arranges the flows in "logical" groups (not to be confused with OVN's Logical_Flows). A logical group of flows is a set of flows that: - Have the same table number and priority - Match on the same fields (regardless of the value they match against) - Have the same actions, regardless of the arguments for those actions, except for output and recirc, for which arguments do care. Optionally, the cookie can also be force to be unique for the logical group. By doing so, we can extend the information we show by querying an external OVN database and running "ovn-detrace" on each cookie. The result is a compact list of flow groups with interlieved OVN information. Furthermore, if connected to an OVN database, we can apply an OVN regexp filter. Examples: $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow logic $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow logic -s -h $ export OVN_NB_DB=... $ export OVN_SB_DB=... $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow logic -d $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow logic -d --ovn-filter="acl.*icmp4" Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 4 +- python/ovs/flowviz/ofp/cli.py | 113 ++++++++++++ python/ovs/flowviz/ofp/logic.py | 303 ++++++++++++++++++++++++++++++++ 3 files changed, 418 insertions(+), 2 deletions(-) create mode 100644 python/ovs/flowviz/ofp/logic.py diff --git a/python/automake.mk b/python/automake.mk index b3fef9bed..449daf023 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -74,12 +74,12 @@ ovs_flowviz = \ python/ovs/flowviz/odp/tree.py \ python/ovs/flowviz/ofp/__init__.py \ python/ovs/flowviz/ofp/cli.py \ + python/ovs/flowviz/ofp/logic.py \ python/ovs/flowviz/ofp/html.py \ python/ovs/flowviz/ovs-flowviz \ python/ovs/flowviz/process.py -# These python files are used at build time but not runtime, -# so they are not installed. +# These python files are used at build time but not runtime, so they are not installed. EXTRA_DIST += \ python/ovs_build_helpers/__init__.py \ python/ovs_build_helpers/extract_ofp_fields.py \ diff --git a/python/ovs/flowviz/ofp/cli.py b/python/ovs/flowviz/ofp/cli.py index 2cd8e1c89..51428ede0 100644 --- a/python/ovs/flowviz/ofp/cli.py +++ b/python/ovs/flowviz/ofp/cli.py @@ -12,10 +12,13 @@ # See the License for the specific language governing permissions and # limitations under the License. +import os + import click from ovs.flowviz.main import maincli from ovs.flowviz.ofp.html import HTMLProcessor +from ovs.flowviz.ofp.logic import LogicFlowProcessor from ovs.flowviz.process import ( ConsoleProcessor, OpenFlowFactory, @@ -69,6 +72,116 @@ def console(opts, heat_map): proc.print() +def ovn_detrace_callback(ctx, param, value): + """click callback to add detrace information to config object and + set general ovn-detrace flag to True + """ + ctx.obj[param.name] = value + if value != param.default: + ctx.obj["ovn_detrace_flag"] = True + return value + + +@openflow.command() +@click.option( + "-d", + "--ovn-detrace", + "ovn_detrace_flag", + is_flag=True, + show_default=True, + help="Use ovn-detrace to extract cookie information (implies '-c')", +) +@click.option( + "--ovn-detrace-path", + default="/usr/bin", + type=click.Path(), + help="Use an alternative path to where ovn_detrace.py is located. " + "Instead of using this option you can just set PYTHONPATH accordingly.", + show_default=True, + callback=ovn_detrace_callback, +) +@click.option( + "--ovnnb-db", + default=os.getenv("OVN_NB_DB") or "unix:/var/run/ovn/ovnnb_db.sock", + help="Specify the OVN NB database string (implies -d). " + "If the OVN_NB_DB environment variable is set, it's used as default. " + "Otherwise, the default is unix:/var/run/ovn/ovnnb_db.sock", + callback=ovn_detrace_callback, +) +@click.option( + "--ovnsb-db", + default=os.getenv("OVN_SB_DB") or "unix:/var/run/ovn/ovnsb_db.sock", + help="Specify the OVN NB database string (implies -d). " + "If the OVN_NB_DB environment variable is set, it's used as default. " + "Otherwise, the default is unix:/var/run/ovn/ovnnb_db.sock", + callback=ovn_detrace_callback, +) +@click.option( + "-o", + "--ovn-filter", + help="Specify a filter to be run on ovn-detrace information (implied -d). " + "Format: python regular expression " + "(see https://docs.python.org/3/library/re.html)", + callback=ovn_detrace_callback, +) +@click.option( + "-s", + "--show-flows", + is_flag=True, + default=False, + show_default=True, + help="Show the full flows under each logical flow", +) +@click.option( + "-c", + "--cookie", + "cookie_flag", + is_flag=True, + default=False, + show_default=True, + help="Consider the cookie in the logical flow", +) +@click.option( + "-h", + "--heat-map", + is_flag=True, + default=False, + show_default=True, + help="Create heat-map with packet and byte counters (when -s is used)", +) +@click.pass_obj +def logic( + opts, + ovn_detrace_flag, + ovn_detrace_path, + ovnnb_db, + ovnsb_db, + ovn_filter, + show_flows, + cookie_flag, + heat_map, +): + """ + Print the logical structure of the flows. + + First, sorts the flows based on tables and priorities. + Then, deduplicates logically equivalent flows: these a flows that match + on the same set of fields (regardless of the values they match against), + have the same priority, and actions (regardless of action arguments, + except in the case of output and recirculate). + Optionally, the cookie can also be considered to be part of the logical + flow. + """ + if ovn_detrace_flag: + opts["ovn_detrace_flag"] = True + if opts.get("ovn_detrace_flag"): + cookie_flag = True + + processor = LogicFlowProcessor(opts, cookie_flag, heat_map) + processor.process() + processor.print(show_flows) + + @openflow.command() @click.pass_obj def html(opts): diff --git a/python/ovs/flowviz/ofp/logic.py b/python/ovs/flowviz/ofp/logic.py new file mode 100644 index 000000000..db2124374 --- /dev/null +++ b/python/ovs/flowviz/ofp/logic.py @@ -0,0 +1,303 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import sys +import io +import re + +from rich.tree import Tree +from rich.text import Text + +from ovs.flowviz.process import FileProcessor, OpenFlowFactory +from ovs.flowviz.console import ( + ConsoleFormatter, + ConsoleBuffer, + hash_pallete, + file_header, + heat_pallete, +) + + +class LFlow: + """A Logical Flow represents the scheleton of a flow. + + Two logical flows have the same logical representation if they match + against same fields (regardless of the matching value) and have the same + set of actions (regardless of the actions' arguments, except for those + in the exact_actions list). + + Attributes: + flow (OFPFlow): The flow + exact_actions(list): Optional; list of action keys that are + considered unique if the value is also the same. + match_cookie (bool): Optional; if cookies are part of the logical + flow + """ + + def __init__(self, flow, exact_actions=[], match_cookie=False): + self.cookie = flow.info.get("cookie") or 0 if match_cookie else None + self.priority = flow.match.get("priority") or 0 + self.match_keys = tuple([kv.key for kv in flow.match_kv]) + + self.action_keys = tuple( + [ + kv.key + for kv in flow.actions_kv + if kv.key not in exact_actions + ] + ) + self.match_action_kvs = [ + kv for kv in flow.actions_kv if kv.key in exact_actions + ] + + def __eq__(self, other): + return ( + (self.cookie == other.cookie if self.cookie else True) + and self.priority == other.priority + and self.action_keys == other.action_keys + and self.equal_match_action_kvs(other) + and self.match_keys == other.match_keys + ) + + def equal_match_action_kvs(self, other): + """ Compares the logical flow's match action key-values with the + others. + + Args: + other (LFlow): The other LFlow to compare against + + Returns true if both LFlow have the same action k-v. + """ + if len(other.match_action_kvs) != len(self.match_action_kvs): + return False + + for kv in self.match_action_kvs: + found = False + for other_kv in other.match_action_kvs: + if self.match_kv(kv, other_kv): + found = True + break + if not found: + return False + return True + + def match_kv(self, one, other): + """Compares a KeyValue. + Args: + one, other (KeyValue): The objects to compare + + Returns true if both KeyValue objects have the same key and value + """ + return one.key == other.key and one.value == other.value + + def __hash__(self): + hash_data = [ + self.cookie, + self.priority, + self.action_keys, + tuple((kv.key, str(kv.value)) for kv in self.match_action_kvs), + self.match_keys, + ] + if self.cookie: + hash_data.append(self.cookie) + return tuple(hash_data).__hash__() + + def format(self, buf, formatter): + """Format the Logical Flow into a Buffer.""" + if self.cookie: + buf.append_extra( + "cookie={} ".format(hex(self.cookie)).ljust(18), + style=cookie_style_gen(str(self.cookie)), + ) + + buf.append_extra( + "priority={} ".format(self.priority), style="steel_blue" + ) + buf.append_extra(",".join(self.match_keys), style="steel_blue") + buf.append_extra(" ---> ", style="bold magenta") + buf.append_extra(",".join(self.action_keys), style="steel_blue") + + if len(self.match_action_kvs) > 0: + buf.append_extra(" ", style=None) + + for kv in self.match_action_kvs: + formatter.format_kv(buf, kv, formatter.style) + buf.append_extra(",", style=None) + + +class LogicFlowProcessor(OpenFlowFactory, FileProcessor): + def __init__(self, opts, match_cookie, heat_map): + super().__init__(opts) + self.data = dict() + self.min_max = dict() + self.match_cookie = match_cookie + self.heat_map = ["n_packets", "n_bytes"] if heat_map else [] + self.ovn_detrace = ( + OVNDetrace(opts) if opts.get("ovn_detrace_flag") else None + ) + + def start_file(self, name, filename): + if len(self.heat_map) > 0: + self.min = [-1] * len(self.heat_map) + self.max = [0] * len(self.heat_map) + self.tables = dict() + + def stop_file(self, name, filename): + if len(self.heat_map) > 0: + self.min_max[name] = (self.min, self.max) + self.data[name] = self.tables + + def process_flow(self, flow, name): + """Sort the flows by table and logical flow.""" + # Running calculation of min and max values for all the fields that + # take place in the heatmap. + for i, field in enumerate(self.heat_map): + val = flow.info.get(field) + if self.min[i] == -1 or val < self.min[i]: + self.min[i] = val + if val > self.max[i]: + self.max[i] = val + + table = flow.info.get("table") or 0 + if not self.tables.get(table): + self.tables[table] = dict() + + # Group flows by logical hash + lflow = LFlow( + flow, + exact_actions=["output", "resubmit", "drop"], + match_cookie=self.match_cookie, + ) + + if not self.tables[table].get(lflow): + self.tables[table][lflow] = list() + + self.tables[table][lflow].append(flow) + + def print(self, show_flows): + formatter = ConsoleFormatter(opts=self.opts) + console = formatter.console + for name, tables in self.data.items(): + console.print("\n") + console.print(file_header(name)) + tree = Tree("Ofproto Flows (logical)") + + for table_num in sorted(tables.keys()): + table = tables[table_num] + table_tree = tree.add("** TABLE {} **".format(table_num)) + + if len(self.heat_map) > 0 and len(table.values()) > 0: + for i, field in enumerate(self.heat_map): + (min_val, max_val) = self.min_max[name][i] + formatter.style.set_value_style( + field, heat_pallete(min_val, max_val) + ) + + for lflow in sorted( + table.keys(), + key=(lambda x: x.priority), + reverse=True, + ): + flows = table[lflow] + ovn_info = None + if self.ovn_detrace: + ovn_info = self.ovn_detrace.get_ovn_info(lflow.cookie) + if self.opts.get("ovn_filter"): + ovn_regexp = re.compile( + self.opts.get("ovn_filter") + ) + if not ovn_regexp.search(ovn_info): + continue + + buf = ConsoleBuffer(Text()) + + lflow.format(buf, formatter) + buf.append_extra( + " ( x {} )".format(len(flows)), + style="dark_olive_green3", + ) + lflow_tree = table_tree.add(buf.text) + + if ovn_info: + ovn = lflow_tree.add("OVN Info") + for part in ovn_info.split("\n"): + if part.strip(): + ovn.add(part.strip()) + + if show_flows: + for flow in flows: + buf = ConsoleBuffer(Text()) + highlighted = None + if self.opts.get("highlight"): + result = self.opts.get("highlight").evaluate( + flow + ) + if result: + highlighted = result.kv + formatter.format_flow(buf, flow, highlighted) + lflow_tree.add(buf.text) + + console.print(tree) + + +class OVNDetrace(object): + def __init__(self, opts): + if not opts.get("ovn_detrace_flag"): + raise Exception("Cannot initialize OVN Detrace connection") + + if opts.get("ovn_detrace_path"): + sys.path.append(opts.get("ovn_detrace_path")) + + import ovn_detrace + + class FakePrinter(ovn_detrace.Printer): + def __init__(self): + self.buff = io.StringIO() + + def print_p(self, msg): + print(" * ", msg, file=self.buff) + + def print_h(self, msg): + print(" * ", msg, file=self.buff) + + def clear(self): + self.buff = io.StringIO() + + self.ovn_detrace = ovn_detrace + self.ovnnb_conn = ovn_detrace.OVSDB( + opts.get("ovnnb_db"), "OVN_Northbound" + ) + self.ovnsb_conn = ovn_detrace.OVSDB( + opts.get("ovnsb_db"), "OVN_Southbound" + ) + self.ovn_printer = FakePrinter() + self.cookie_handlers = ovn_detrace.get_cookie_handlers( + self.ovnnb_conn, self.ovnsb_conn, self.ovn_printer + ) + + def get_ovn_info(self, cookie): + self.ovn_printer.clear() + self.ovn_detrace.print_record_from_cookie( + self.ovnsb_conn, self.cookie_handlers, "{:x}".format(cookie) + ) + return self.ovn_printer.buff.getvalue() + + +# Try to make it easy to spot same cookies by printing them in different +# colors +cookie_style_gen = hash_pallete( + hue=[x / 10 for x in range(0, 10)], + saturation=[0.5], + value=[0.5 + x / 10 * (0.85 - 0.5) for x in range(0, 10)], +) From patchwork Tue May 7 14:30:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1932505 X-Patchwork-Delegate: echaudro@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=fKetbcOy; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VYggy3pNGz1xnT for ; Wed, 8 May 2024 00:31:26 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 250D3407E6; Tue, 7 May 2024 14:31:24 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id yjuRvYDGNn0k; Tue, 7 May 2024 14:31:19 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 2DE9440795 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=fKetbcOy Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 2DE9440795; Tue, 7 May 2024 14:31:17 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0642BC0DDB; Tue, 7 May 2024 14:31:16 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1F6C3C0DD8 for ; Tue, 7 May 2024 14:31:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 532A6821D7 for ; Tue, 7 May 2024 14:31:14 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id cMrWKx1vuyuY for ; Tue, 7 May 2024 14:31:10 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp1.osuosl.org 0E8AF820E8 Authentication-Results: smtp1.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 0E8AF820E8 Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=fKetbcOy Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 0E8AF820E8 for ; Tue, 7 May 2024 14:31:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1715092268; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JrNSoJLYaNRW9yMokZQzbmO5eSQzo8HcTAVQfPcN8mw=; b=fKetbcOy5MPp4GhoQrcGP4PorKL0c6si3Uw2+Ca8bIcvZcSyeVSRiAuQ1NWi8goGaAqkXf x9z/ZZ4ZWuCnZ6zHlxd0hXZpd538O7rZoNfpLZIlwbnLvSlbvZm8y5XMANq3ic6EfaTNR1 9PVK0fKlLq5cqD+STUsQWzmZg6u/OAo= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-21-0ljrKPCwPGO8kl-dT7Dbow-1; Tue, 07 May 2024 10:31:07 -0400 X-MC-Unique: 0ljrKPCwPGO8kl-dT7Dbow-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D9EC8857A87 for ; Tue, 7 May 2024 14:31:06 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.88]) by smtp.corp.redhat.com (Postfix) with ESMTP id 02EF53C30; Tue, 7 May 2024 14:31:05 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 7 May 2024 16:30:34 +0200 Message-ID: <20240507143044.3056232-9-amorenoz@redhat.com> In-Reply-To: <20240507143044.3056232-1-amorenoz@redhat.com> References: <20240507143044.3056232-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v4 08/12] python: ovs: flowviz: Add Openflow cookie format. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" When anaylizing OVN issues, it might be useful to see what OpenFlow flows were generated from each logical flow. In order to make it simpler to visualize this, add a cookie format that simply sorts the flows first by cookie, then by table. Example: $ export OVN_NB_DB=... $ export OVN_SB_DB=... $ ovs-vsctl dump-flows br-int | ovs-flowviz openflow cookie --ovn-filter="acl.*icmp4" $ ovs-vsctl dump-flows br-int | ovs-flowviz openflow cookie --ovn-detrace Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/ovs/flowviz/ofp/cli.py | 57 +++++++++++++++++++++++++++++- python/ovs/flowviz/ofp/logic.py | 61 +++++++++++++++++++++++++++++++++ 2 files changed, 117 insertions(+), 1 deletion(-) diff --git a/python/ovs/flowviz/ofp/cli.py b/python/ovs/flowviz/ofp/cli.py index 51428ede0..690e7618e 100644 --- a/python/ovs/flowviz/ofp/cli.py +++ b/python/ovs/flowviz/ofp/cli.py @@ -18,7 +18,7 @@ import click from ovs.flowviz.main import maincli from ovs.flowviz.ofp.html import HTMLProcessor -from ovs.flowviz.ofp.logic import LogicFlowProcessor +from ovs.flowviz.ofp.logic import CookieProcessor, LogicFlowProcessor from ovs.flowviz.process import ( ConsoleProcessor, OpenFlowFactory, @@ -182,6 +182,61 @@ def logic( processor.print(show_flows) +@openflow.command() +@click.option( + "-d", + "--ovn-detrace", + "ovn_detrace_flag", + is_flag=True, + show_default=True, + help="Use ovn-detrace to extract cookie information", +) +@click.option( + "--ovn-detrace-path", + default="/usr/bin", + type=click.Path(), + help="Use an alternative path to where ovn_detrace.py is located. " + "Instead of using this option you can just set PYTHONPATH accordingly", + show_default=True, + callback=ovn_detrace_callback, +) +@click.option( + "--ovnnb-db", + default=os.getenv("OVN_NB_DB") or "unix:/var/run/ovn/ovnnb_db.sock", + help="Specify the OVN NB database string (implies -d). " + "If the OVN_NB_DB environment variable is set, it's used as default. " + "Otherwise, the default is unix:/var/run/ovn/ovnnb_db.sock", + callback=ovn_detrace_callback, +) +@click.option( + "--ovnsb-db", + default=os.getenv("OVN_SB_DB") or "unix:/var/run/ovn/ovnsb_db.sock", + help="Specify the OVN NB database string (implies -d). " + "If the OVN_NB_DB environment variable is set, it's used as default. " + "Otherwise, the default is unix:/var/run/ovn/ovnnb_db.sock", + callback=ovn_detrace_callback, +) +@click.option( + "-o", + "--ovn-filter", + help="Specify a filter to be run on ovn-detrace information (implied -d). " + "Format: python regular expression " + "(see https://docs.python.org/3/library/re.html)", + callback=ovn_detrace_callback, +) +@click.pass_obj +def cookie( + opts, ovn_detrace_flag, ovn_detrace_path, ovnnb_db, ovnsb_db, ovn_filter +): + """Print the flow tables sorted by cookie.""" + if ovn_detrace_flag: + opts["ovn_detrace_flag"] = True + + processor = CookieProcessor(opts) + processor.process() + processor.print() + + @openflow.command() @click.pass_obj def html(opts): diff --git a/python/ovs/flowviz/ofp/logic.py b/python/ovs/flowviz/ofp/logic.py index db2124374..9d244d137 100644 --- a/python/ovs/flowviz/ofp/logic.py +++ b/python/ovs/flowviz/ofp/logic.py @@ -301,3 +301,64 @@ cookie_style_gen = hash_pallete( saturation=[0.5], value=[0.5 + x / 10 * (0.85 - 0.5) for x in range(0, 10)], ) + + +class CookieProcessor(OpenFlowFactory, FileProcessor): + """Processor that sorts flows into cookies and tables.""" + + def __init__(self, opts): + super().__init__(opts) + self.data = dict() + self.ovn_detrace = ( + OVNDetrace(opts) if opts.get("ovn_detrace_flag") else None + ) + + def start_file(self, name, filename): + self.cookies = dict() + + def stop_file(self, name, filename): + self.data[name] = self.cookies + + def process_flow(self, flow, name): + """Sort the flows by table and logical flow.""" + cookie = flow.info.get("cookie") or 0 + if not self.cookies.get(cookie): + self.cookies[cookie] = dict() + + table = flow.info.get("table") or 0 + if not self.cookies[cookie].get(table): + self.cookies[cookie][table] = list() + self.cookies[cookie][table].append(flow) + + def print(self): + ofconsole = ConsoleFormatter(opts=self.opts) + console = ofconsole.console + for name, cookies in self.data.items(): + console.print("\n") + console.print(file_header(name)) + tree = Tree("Ofproto Cookie Tree") + + for cookie, tables in cookies.items(): + ovn_info = None + if self.ovn_detrace: + ovn_info = self.ovn_detrace.get_ovn_info(cookie) + if self.opts.get("ovn_filter"): + ovn_regexp = re.compile(self.opts.get("ovn_filter")) + if not ovn_regexp.search(ovn_info): + continue + + cookie_tree = tree.add("** Cookie {} **".format(hex(cookie))) + if ovn_info: + ovn = cookie_tree.add("OVN Info") + for part in ovn_info.split("\n"): + if part.strip(): + ovn.add(part.strip()) + + tables_tree = cookie_tree.add("Tables") + for table, flows in tables.items(): + table_tree = tables_tree.add("* Table {} * ".format(table)) + for flow in flows: + buf = ConsoleBuffer(Text()) + ofconsole.format_flow(buf, flow) + table_tree.add(buf.text) + console.print(tree) From patchwork Tue May 7 14:30:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1932508 X-Patchwork-Delegate: echaudro@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=jK6CRLOx; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VYghF3Ft3z214P for ; Wed, 8 May 2024 00:31:41 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 81E3C82332; Tue, 7 May 2024 14:31:39 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id oxjRa07UY4wl; Tue, 7 May 2024 14:31:36 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org EE112821AB Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=jK6CRLOx Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id EE112821AB; Tue, 7 May 2024 14:31:23 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id BAF41C008E; Tue, 7 May 2024 14:31:23 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 234F6C0DDC for ; Tue, 7 May 2024 14:31:22 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 58B8260AE2 for ; Tue, 7 May 2024 14:31:16 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id ditUAcJJR_gS for ; Tue, 7 May 2024 14:31:11 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org 6FF2660A7F Authentication-Results: smtp3.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6FF2660A7F Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=jK6CRLOx Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id 6FF2660A7F for ; Tue, 7 May 2024 14:31:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1715092270; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Plv0gryON0flNuHPpXWOqiaowCxx3QSihM+SP4OuDaQ=; b=jK6CRLOxrnUN3euCk9Fc7U357CYnZ3k0o2jl0j3mFR2OR5kXHEYbxGodkMr6TIhxtzkAjt atQuA6QyLUc8JiZnX7eG6E5NIzE8gu2TUFb8YhMYDix4kflKcLE7bbhZqgB4p9AYs1bs02 EkoV+hDBtpRyumH0plPHltqn0jJzWPY= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-367-2aw9ayKVN7OFsxNgKRwtVA-1; Tue, 07 May 2024 10:31:08 -0400 X-MC-Unique: 2aw9ayKVN7OFsxNgKRwtVA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1300738C6161 for ; Tue, 7 May 2024 14:31:08 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.88]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3A9743C30; Tue, 7 May 2024 14:31:07 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 7 May 2024 16:30:35 +0200 Message-ID: <20240507143044.3056232-10-amorenoz@redhat.com> In-Reply-To: <20240507143044.3056232-1-amorenoz@redhat.com> References: <20240507143044.3056232-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v4 09/12] python: ovs: flowviz: Add datapath html format. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Using the existing FlowTree and HTMLFormatter, create an HTML tree visualization that also supports collapsing and expanding entire flow trees and subtrees. Examples: $ ovs-appcl dpctl/dump-flows | ovs-flowviz --highlight drop datapath html > /tmp/flows.html $ ovs-appcl dpctl/dump-flows | ovs-flowviz -f "output.port=3" datapath html > /tmp/flows.html Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 1 + python/ovs/flowviz/odp/cli.py | 10 ++ python/ovs/flowviz/odp/html.py | 259 +++++++++++++++++++++++++++++++++ 3 files changed, 270 insertions(+) create mode 100644 python/ovs/flowviz/odp/html.py diff --git a/python/automake.mk b/python/automake.mk index 449daf023..44e9e08ab 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -71,6 +71,7 @@ ovs_flowviz = \ python/ovs/flowviz/main.py \ python/ovs/flowviz/odp/__init__.py \ python/ovs/flowviz/odp/cli.py \ + python/ovs/flowviz/odp/html.py \ python/ovs/flowviz/odp/tree.py \ python/ovs/flowviz/ofp/__init__.py \ python/ovs/flowviz/ofp/cli.py \ diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py index 615bac55b..dd64fdc65 100644 --- a/python/ovs/flowviz/odp/cli.py +++ b/python/ovs/flowviz/odp/cli.py @@ -14,6 +14,7 @@ import click from ovs.flowviz.main import maincli +from ovs.flowviz.odp.html import HTMLTreeProcessor from ovs.flowviz.odp.tree import ConsoleTreeProcessor from ovs.flowviz.process import ( ConsoleProcessor, @@ -82,3 +83,12 @@ def tree(opts, heat_map): processor = ConsoleTreeProcessor(opts) processor.process() processor.print(heat_map) + + +@datapath.command() +@click.pass_obj +def html(opts): + """Print the flows in an HTML list sorted by recirc_id.""" + processor = HTMLTreeProcessor(opts) + processor.process() + processor.print() diff --git a/python/ovs/flowviz/odp/html.py b/python/ovs/flowviz/odp/html.py new file mode 100644 index 000000000..4aa08dc70 --- /dev/null +++ b/python/ovs/flowviz/odp/html.py @@ -0,0 +1,259 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from ovs.flowviz.html_format import HTMLBuffer, HTMLFormatter +from ovs.flowviz.odp.tree import FlowElem, FlowTree +from ovs.flowviz.process import DatapathFactory, FileProcessor + + +class HTMLTreeProcessor(DatapathFactory, FileProcessor): + def __init__(self, opts): + super().__init__(opts) + self.data = dict() + + def start_file(self, name, filename): + self.tree = HTMLTree(name, self.opts) + + def process_flow(self, flow, name): + self.tree.add(flow) + + def process(self): + super().process(False) + + def stop_file(self, name, filename): + self.data[name] = self.tree + + def print(self): + html_obj = "" + for name, tree in self.data.items(): + html_obj += "
" + html_obj += "

{}

".format(name) + tree.build() + if self.opts.get("filter"): + tree.filter(self.opts.get("filter")) + html_obj += tree.render() + html_obj += "
" + print(html_obj) + + +class HTMLTree(FlowTree): + """HTMLTree is a Flowtree that prints the tree in html format. + + Args: + opts(dict): Options dictionary + flows(dict[int, list[DPFlow]): Optional; initial flows + """ + + html_header = """ + + + + """ # noqa: E501 + + class HTMLTreeElem(FlowElem): + """An element within the HTML Tree. + + It is composed of a flow and its subflows that can be added by calling + append() + """ + + def __init__(self, parent_name, flow=None, opts=None): + self._parent_name = parent_name + self._formatter = HTMLFormatter(opts) + self._opts = opts + super(HTMLTree.HTMLTreeElem, self).__init__(flow) + + def render(self, item=0): + """Render the HTML Element. + Args: + item (int): the item id + + Returns: + (html_obj, items) tuple where html_obj is the html string and + items is the number of subitems rendered in total + """ + parent_name = self._parent_name.replace(" ", "_") + html_obj = "
" + if self.flow: + html_text = """ + + + """ # noqa: E501 + html_obj += html_text.format( + item=item, id=self.flow.id, name=parent_name + ) + + html_text = '
' # noqa: E501 + html_obj += html_text.format(id=self.flow.id) + buf = HTMLBuffer() + highlighted = None + if self._opts.get("highlight"): + result = self._opts.get("highlight").evaluate(self.flow) + if result: + highlighted = result.kv + self._formatter.format_flow(buf, self.flow, highlighted) + html_obj += buf.text + html_obj += "
" + if self.children: + html_obj += "
" + html_obj += "
    " + for sf in self.children: + item += 1 + html_obj += "
  • " + (html_elem, items) = sf.render(item) + html_obj += html_elem + item += items + html_obj += "
    • " + html_obj += "
" + html_obj += "
" + html_obj += "
" + return html_obj, item + + def __init__(self, name, opts, flows=None): + self.opts = opts + self.name = name + super(HTMLTree, self).__init__( + flows, self.HTMLTreeElem("", flow=None, opts=self.opts) + ) + + def _new_elem(self, flow, _): + """Override _new_elem to provide HTMLTreeElems.""" + return self.HTMLTreeElem(self.name, flow, self.opts) + + def render(self): + """Render the Tree in HTML. + + Returns: + an html string representing the element + """ + name = self.name.replace(" ", "_") + + html_text = """ +""" # noqa: E501 + html_obj = self.html_header + html_text.format(name=name) + + html_obj += "
".format(name=name) + (html_elem, _) = self.root.render() + html_obj += html_elem + html_obj += "
" + return html_obj From patchwork Tue May 7 14:30:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1932506 X-Patchwork-Delegate: echaudro@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=EEAkpvkx; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VYgh82zsXz1xnT for ; Wed, 8 May 2024 00:31:36 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 9E73D82165; Tue, 7 May 2024 14:31:34 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id fLAeOxWddciV; Tue, 7 May 2024 14:31:30 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 2BC34821D7 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=EEAkpvkx Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 2BC34821D7; Tue, 7 May 2024 14:31:22 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C7943C0DD4; Tue, 7 May 2024 14:31:21 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 154A4C0DDB for ; Tue, 7 May 2024 14:31:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id BFBD1416DB for ; Tue, 7 May 2024 14:31:16 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 4aodrbyGSEJJ for ; Tue, 7 May 2024 14:31:13 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp2.osuosl.org F19CA416BD Authentication-Results: smtp2.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org F19CA416BD Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=EEAkpvkx Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id F19CA416BD for ; Tue, 7 May 2024 14:31:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1715092271; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yZRw74yP2HEK0PKR8Kp3xEXAjxW/pbyTXTV3FfSAIhM=; b=EEAkpvkx+JphjVoYEnAVIgOR9lfNkx0J2pKbNrvEhAJ1J+BK6ZBlc+inbrmBoOqIITcYwv FGq/FKyEYACEx9V4v1Csz00ihpTLxKZ+81KjU9CNGJDprsa6SZBhXPsK19wPjbOf+T8hZp rvDhI62HOn2PfboVzQfJxwFPyfBB4B0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-522-Bc9nNyMMNHGu5AcbJkTW3w-1; Tue, 07 May 2024 10:31:09 -0400 X-MC-Unique: Bc9nNyMMNHGu5AcbJkTW3w-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AC8A8101A551 for ; Tue, 7 May 2024 14:31:09 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.88]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7399B3C27; Tue, 7 May 2024 14:31:08 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 7 May 2024 16:30:36 +0200 Message-ID: <20240507143044.3056232-11-amorenoz@redhat.com> In-Reply-To: <20240507143044.3056232-1-amorenoz@redhat.com> References: <20240507143044.3056232-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v4 10/12] python: ovs: flowviz: Add datapath graph format. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Graph view leverages the tree format (specially the tree-based filtering) and uses graphviz library to build a visual graph of the datapath in graphviz format. Conntrack zones are shown in random colors to help visualize connection tracking interdependencies. An html flag builds an HTML page with both the html flows and the graph (in svg) that enables navegation. Examples: $ ovs-appctl dpctl/dump-flows -m | ovs-flowviz datapath graph | dot -Tpng -o graph.png $ ovs-appctl dpctl/dump-flows -m | ovs-flowviz datapath graph --html > flows.html Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 1 + python/ovs/flowviz/odp/cli.py | 22 ++ python/ovs/flowviz/odp/graph.py | 418 ++++++++++++++++++++++++++++++++ python/ovs/flowviz/odp/tree.py | 18 +- python/setup.py | 2 +- 5 files changed, 457 insertions(+), 4 deletions(-) create mode 100644 python/ovs/flowviz/odp/graph.py diff --git a/python/automake.mk b/python/automake.mk index 44e9e08ab..9ef000480 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -71,6 +71,7 @@ ovs_flowviz = \ python/ovs/flowviz/main.py \ python/ovs/flowviz/odp/__init__.py \ python/ovs/flowviz/odp/cli.py \ + python/ovs/flowviz/odp/graph.py \ python/ovs/flowviz/odp/html.py \ python/ovs/flowviz/odp/tree.py \ python/ovs/flowviz/ofp/__init__.py \ diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py index dd64fdc65..94fdb80eb 100644 --- a/python/ovs/flowviz/odp/cli.py +++ b/python/ovs/flowviz/odp/cli.py @@ -14,6 +14,7 @@ import click from ovs.flowviz.main import maincli +from ovs.flowviz.odp.graph import GraphProcessor from ovs.flowviz.odp.html import HTMLTreeProcessor from ovs.flowviz.odp.tree import ConsoleTreeProcessor from ovs.flowviz.process import ( @@ -92,3 +93,24 @@ def html(opts): processor = HTMLTreeProcessor(opts) processor.process() processor.print() + + +@datapath.command() +@click.option( + "-h", + "--html", + is_flag=True, + default=False, + show_default=True, + help="Output an html file containing the graph", +) +@click.pass_obj +def graph(opts, html): + """Print the flows in an graphviz (.dot) format showing the relationship + of recirc_ids.""" + if len(opts.get("filename")) > 1: + raise click.BadParameter("Graph format only supports one input file") + + processor = GraphProcessor(opts) + processor.process() + processor.print(html) diff --git a/python/ovs/flowviz/odp/graph.py b/python/ovs/flowviz/odp/graph.py new file mode 100644 index 000000000..b26551e67 --- /dev/null +++ b/python/ovs/flowviz/odp/graph.py @@ -0,0 +1,418 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""" Defines a Datapath Graph using graphviz. """ +import colorsys +import graphviz +import random + +from ovs.flowviz.odp.html import HTMLTree +from ovs.flowviz.odp.tree import FlowTree +from ovs.flowviz.process import DatapathFactory, FileProcessor + + +class GraphProcessor(DatapathFactory, FileProcessor): + def __init__(self, opts): + super().__init__(opts) + + def start_file(self, name, filename): + self.tree = FlowTree() + + def process_flow(self, flow, name): + self.tree.add(flow) + + def process(self): + super().process(False) + + def print(self, html): + flows = {} + + # Tree traverse callback + def add_flow(elem, _): + if elem.is_root: + return + rid = elem.flow.match.get("recirc_id") or 0 + if not flows.get(rid): + flows[rid] = set() + flows[rid].add(elem.flow) + + self.tree.build() + if self.opts.get("filter"): + self.tree.filter(self.opts.get("filter")) + self.tree.traverse(add_flow) + + if len(flows) == 0: + return + + dpg = DatapathGraph(flows) + if not html: + print(dpg.source()) + return + + html_obj = "" + html_obj += "

Flow Graph

" + html_obj += "
" + svg = dpg.pipe(format="svg") + html_obj += svg.decode("utf-8") + html_obj += "
" + html_tree = HTMLTree("graph", self.opts, flows) + html_tree.build() + html_obj += html_tree.render() + + print(html_obj) + + +class DatapathGraph: + """A DatapathGraph is a class that renders a set of datapath flows into + graphviz graphs. + + Args: + flows(dict[int, list(Flow)]): Dictionary of lists of flows indexed by + recirc_id + """ + + ct_styles = {} + node_styles = { + "default": { + "style": {}, + "desc": "Default", + }, + "action_and_match": { + "style": {"color": "#ff00ff"}, + "desc": "Flow uses CT as match and action", + }, + "match": { + "style": {"color": "#0000ff"}, + "desc": "Flow uses CT only to match", + }, + "action": { + "style": {"color": "#ff0000"}, + "desc": "Flow uses CT only as action", + }, + } + + def __init__(self, flows): + self._flows = flows + + self._output_nodes = [] + self._graph = graphviz.Digraph( + "DP flows", node_attr={"shape": "rectangle"} + ) + self._graph.attr(compound="true") + self._graph.attr(rankdir="LR") + self._graph.attr(ranksep="3") + + self._populate_graph() + + def source(self): + """Return the graphviz source representation of the graph.""" + return self._graph.source + + def pipe(self, *args, **kwargs): + """Output the graph based on arguments given to graphviz.pipe.""" + return self._graph.pipe(*args, **kwargs) + + @classmethod + def recirc_cluster_name(cls, recirc_id): + """Name of the recirculation cluster.""" + return "cluster_recirc_{}".format(hex(recirc_id)) + + @classmethod + def inport_cluster_name(cls, inport): + """Name of the input port cluster.""" + return "cluster_inport_{}".format(inport) + + @classmethod + def invis_node_name(cls, cluster_name): + """Name of the invisible node.""" + return "invis_{}".format(cluster_name) + + @classmethod + def output_node_name(cls, port): + """Name of the ouput node.""" + return "output_{}".format(port) + + def _flow_node(self, flow, name): + """Returns the dictionary of attributes of a graphviz node that + represents the flow with a given name. + """ + summary = "Line: {} \n".format(flow.id) + summary += "\n".join( + [ + flow.section("info").string, + ",".join(flow.match.keys()), + "actions: " + + ",".join(list(a.keys())[0] for a in flow.actions), + ] + ) + + has_ct_match = flow.match.get("ct_state", "0/0") != "0/0" + has_ct_action = bool( + next( + filter(lambda x: x.key in ["ct", "ct_clear"], flow.actions_kv), + None, + ) + ) + + if has_ct_action: + if has_ct_match: + style = "action_and_match" + else: + style = "action" + elif has_ct_match: + style = "match" + else: + style = "default" + + style = self.node_styles.get(style, {}) + + return { + "name": name, + "label": summary, + "tooltip": flow.orig, + "_attributes": style.get("style", {}), + "fontsize": "10", + "nojustify": "true", + "URL": "#flow_{}".format(flow.id), + } + + def _create_recirc_cluster(self, recirc): + """Process a recirculation id, creating its cluster.""" + cluster_name = self.recirc_cluster_name(recirc) + label = "recirc x0{:0x}".format(recirc) + + cluster = self._graph.subgraph(name=cluster_name, comment=label) + with cluster as sg: + sg.attr(rankdir="TB") + sg.attr(ranksep="0.02") + sg.attr(label=label) + sg.attr(margin="5") + self._add_flows_to_graph(sg, self._flows[recirc]) + + self.processed_recircs.append(recirc) + + def _add_flows_to_graph(self, graph, flows): + # Create an invisible node and an edge to the first flow so that + # it ends up at the top of the cluster. + invis = self.invis_node_name(graph.name) + graph.node(invis) + graph.node( + invis, + color="white", + len="0", + shape="point", + width="0", + height="0", + ) + first = True + for flow in flows: + name = "Flow_{}".format(flow.id) + graph.node(**self._flow_node(flow, name)) + if first: + with graph.subgraph() as c: + c.attr(rank="same") + c.edge(name, invis, style="invis") + first = False + # determine next hop based on actions + self._set_next_node_from_actions(name, flow.actions) + + def set_next_node_from_actions(self, name, actions): + """Determine the next nodes based on action list and add edges to + them. + """ + if not self._set_next_node_from_actions(self, name, actions): + # Add to a generic "End" if no other action was detected + self._graph.edge(name, "end") + + def _set_next_node_from_actions(self, name, actions): + created = False + for action in actions: + key, value = next(iter(action.items())) + if key == "check_pkt_len": + created |= self._set_next_node_from_actions( + name, value.get("gt") + ) + created |= self._set_next_node_from_actions( + name, value.get("le") + ) + elif key == "sample": + created |= self._set_next_node_from_actions( + name, value.get("actions") + ) + elif key == "clone": + created |= self._set_next_node_from_actions( + name, value.get("actions") + ) + else: + created |= self._set_next_node_action(name, key, value) + return created + + def _set_next_node_action(self, name, action_name, action_obj): + """Based on the action object, set the next node.""" + if action_name == "recirc": + # If the targer recirculation cluster has not yet been created, + # do it now. + if action_obj not in self.processed_recircs: + self._create_recirc_cluster(action_obj) + + cname = self.recirc_cluster_name(action_obj) + self._graph.edge( + name, + self.invis_node_name(cname), + lhead=cname, + _attributes={"weight": "20"}, + ) + return True + elif action_name == "output": + port = action_obj.get("port") + if port not in self._output_nodes: + self._output_nodes.append(port) + self._graph.edge( + name, self.output_node_name(port), _attributes={"weight": "1"} + ) + return True + elif action_name in ["drop", "userspace", "controller"]: + if action_name not in self._output_nodes: + self._output_nodes.append(action_name) + self._graph.edge(name, action_name, _attributes={"weight": "1"}) + return True + elif action_name == "ct": + zone = action_obj.get("zone", 0) + node_name = "CT zone {}".format(action_obj.get("zone", "default")) + if zone not in self.ct_styles: + # Pick a random (highly saturated) color. + (r, g, b) = colorsys.hsv_to_rgb(random.random(), 1, 1) + color = "#%02x%02x%02x" % ( + int(r * 255), + int(g * 255), + int(b * 255), + ) + self.ct_styles[zone] = color + self._graph.node(node_name, color=color) + + color = self.ct_styles[zone] + self._graph.edge(name, node_name, style="dashed", color=color) + # test + name = node_name + return True + return False + + def _populate_graph(self): + """Populate the the internal graph.""" + self.processed_recircs = [] + + # Create a subcluster for each input port and one for flows that don't + # have in_port() for which we create a dummy inport. + flows_per_inport = {} + free_flows = [] + + for flow in self._flows.get(0): + port = flow.match.get("in_port") + if port: + if not flows_per_inport.get(port): + flows_per_inport[port] = list() + flows_per_inport[port].append(flow) + else: + free_flows.append(flow) + + # It's rare to find flows without input_port match but let's add them + # nevertheless. + if free_flows: + self._graph.edge( + "start", + self.invis_node_name(self.recirc_cluster_name(0)), + lhead=self.recirc_cluster_name(0), + ) + self._graph.node("no_port", shape="Mdiamond") + + # Recirc_clusters are created recursively when an edge is found to + # them. + # Process recirc(0) which is split by input port. + for inport, flows in flows_per_inport.items(): + # Build a subgraph per input port + cluster_name = self.inport_cluster_name(inport) + label = "recirc 0; input port: {}".format(inport) + + with self._graph.subgraph( + name=cluster_name, comment=label + ) as per_port: + per_port.attr(rankdir="TB") + per_port.attr(ranksep="0.02") + per_port.attr(margin="5") + per_port.attr(label=label) + self._add_flows_to_graph(per_port, flows_per_inport[inport]) + + # Create an input node that points to each input subgraph + # They are all inside an anonymous subgraph so that they can be + # alligned. + with self._graph.subgraph() as s: + s.attr(rank="same") + for inport in flows_per_inport: + # Make an Input node point to each subgraph + node_name = "input_{}".format(inport) + cluster_name = self.inport_cluster_name(inport) + s.node( + node_name, + shape="Mdiamond", + label="input port {}".format(inport), + ) + self._graph.edge( + node_name, + self.invis_node_name(cluster_name), + lhead=cluster_name, + _attributes={"weight": "20"}, + ) + + # Create the output nodes in a subgraph so that they are alligned + with self._graph.subgraph() as s: + for port in self._output_nodes: + s.attr(rank="same") + if port == "drop": + s.node( + "drop", + shape="Msquare", + color="red", + label="DROP", + rank="sink", + ) + elif port == "controller": + s.node( + "controller", + shape="Msquare", + color="blue", + label="CONTROLLER", + rank="sink", + ) + elif port == "userspace": + s.node( + "userspace", + shape="Msquare", + color="blue", + label="CONTROLLER", + rank="sink", + ) + else: + s.node( + self.output_node_name(port), + shape="Msquare", + color="green", + label="Port {}".format(port), + rank="sink", + ) + + # Print style legend + with self._graph.subgraph(name="cluster_legend") as s: + s.attr(label="Legend") + for style in self.node_styles.values(): + s.node(name=style.get("desc"), _attributes=style.get("style")) diff --git a/python/ovs/flowviz/odp/tree.py b/python/ovs/flowviz/odp/tree.py index d249e7d6d..72be1c427 100644 --- a/python/ovs/flowviz/odp/tree.py +++ b/python/ovs/flowviz/odp/tree.py @@ -76,7 +76,8 @@ class FlowTree: on recirculation ids. Args: - flows (list[ODPFlow]): Optional, initial list of flows + flows (list[ODPFlow]): Optional, initial list of flows or dictionary of + flows indexed by recirc_id root (TreeElem): Optional, root of the tree. """ @@ -84,8 +85,15 @@ class FlowTree: self._flows = {} self.root = root if flows: - for flow in flows: - self.add(flow) + if isinstance(flows, dict): + self._flows = flows + elif isinstance(flows, list): + for flow in flows: + self.add(flow) + else: + raise Exception( + "flows in wrong format: {}".format(type(flows)) + ) def add(self, flow): """Add a flow""" @@ -192,6 +200,10 @@ class FlowTree: for elem in to_remove: self.root.children.remove(elem) + def all(self): + """Return all the flows in a dictionary by recirc_id.""" + return self._flows + class ConsoleTreeProcessor(DatapathFactory, FileProcessor): def __init__(self, opts): diff --git a/python/setup.py b/python/setup.py index c734f68f3..018a75eb0 100644 --- a/python/setup.py +++ b/python/setup.py @@ -114,7 +114,7 @@ setup_args = dict( 'dns': ['unbound'], 'flow': flow_extras_require, 'flowviz': - [*flow_extras_require, 'click', 'rich'], + [*flow_extras_require, 'click', 'rich', 'graphviz'], }, scripts=["ovs/flowviz/ovs-flowviz"], data_files=["ovs/flowviz/ovs-flowviz.conf"], From patchwork Tue May 7 14:30:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1932509 X-Patchwork-Delegate: echaudro@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=QoCCiY+U; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VYghK3LkRz214P for ; Wed, 8 May 2024 00:31:45 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 9387A417E1; Tue, 7 May 2024 14:31:43 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id xQbP_HBH4RdW; Tue, 7 May 2024 14:31:38 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 2DB6441702 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=QoCCiY+U Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 2DB6441702; Tue, 7 May 2024 14:31:33 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id F1222C0077; Tue, 7 May 2024 14:31:32 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 08492C0037 for ; Tue, 7 May 2024 14:31:32 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 637D4407A3 for ; Tue, 7 May 2024 14:31:20 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id EK8NKdwza9G5 for ; Tue, 7 May 2024 14:31:17 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org 9CED5407F7 Authentication-Results: smtp4.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 9CED5407F7 Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=QoCCiY+U Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 9CED5407F7 for ; Tue, 7 May 2024 14:31:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1715092275; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VTt9w875Vozmtxt0Yf3M+jTL+vaVDeX0UQQ5pJGw494=; b=QoCCiY+UGFdTRGpctdQZPlJ+nDdBeW111YyCCahEgYD75/Q0Vkg6jb/hEO4jvvo2xZza4R eywwTMgePn2KN9lVX0kXporn0mRX0Nshm8JemfQHTaY7Zf+4n+GvjJdRnJ5p4GpN81ADtf bnkvX63C5IS7WIptbomPBQzI0twyUjw= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-620-I3cTKw33MRimzd5MunCbVw-1; Tue, 07 May 2024 10:31:11 -0400 X-MC-Unique: I3cTKw33MRimzd5MunCbVw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 246931C0150B for ; Tue, 7 May 2024 14:31:11 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.88]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1A1C83C25; Tue, 7 May 2024 14:31:09 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 7 May 2024 16:30:37 +0200 Message-ID: <20240507143044.3056232-12-amorenoz@redhat.com> In-Reply-To: <20240507143044.3056232-1-amorenoz@redhat.com> References: <20240507143044.3056232-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v4 11/12] python: ovs: flowviz: Support html dark style. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" In order to support dark style in html outputs, allow the config file to express the background color and set it in a top style object. Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/ovs/flowviz/html_format.py | 4 +++- python/ovs/flowviz/odp/html.py | 30 ++++++++++++++++++++++++----- python/ovs/flowviz/ofp/html.py | 28 ++++++++++++++++++++++----- python/ovs/flowviz/ovs-flowviz.conf | 20 +++++++++++++++++++ 4 files changed, 71 insertions(+), 11 deletions(-) diff --git a/python/ovs/flowviz/html_format.py b/python/ovs/flowviz/html_format.py index ebfa65c34..3f3550da5 100644 --- a/python/ovs/flowviz/html_format.py +++ b/python/ovs/flowviz/html_format.py @@ -48,7 +48,9 @@ class HTMLBuffer(FlowBuffer): style = ' style="color:{}"'.format(color) if color else "" self._text += "".format(style) if href: - self._text += "".format(href) + self._text += " ".format( + href, 'style="color:{}"'.format(color) if color else "" + ) self._text += string if href: self._text += "" diff --git a/python/ovs/flowviz/odp/html.py b/python/ovs/flowviz/odp/html.py index 4aa08dc70..b2855bf40 100644 --- a/python/ovs/flowviz/odp/html.py +++ b/python/ovs/flowviz/odp/html.py @@ -55,10 +55,18 @@ class HTMLTree(FlowTree): flows(dict[int, list[DPFlow]): Optional; initial flows """ + html_body_style = """ + """ + html_header = """ """.format( + bg=bg, fg=fg + ) for name, tables in self.data.items(): name = name.replace(" ", "_") html_obj += "

{}

".format(name) html_obj += "
" for table, flows in tables.items(): - formatter = HTMLFormatter(self.opts) def anchor(x): return "#table_%s_%s" % (name, x.value["table"]) - formatter.style.set_value_style( + self.formatter.style.set_value_style( "resubmit", HTMLStyle( - formatter.style.get("value.resubmit"), + self.formatter.style.get("value.resubmit").color, anchor_gen=anchor, ), ) @@ -71,7 +89,7 @@ class HTMLProcessor(OpenFlowFactory, FileProcessor): if result: highlighted = result.kv buf = HTMLBuffer() - formatter.format_flow(buf, flow, highlighted) + self.formatter.format_flow(buf, flow, highlighted) html_obj += buf.text html_obj += "" html_obj += "" diff --git a/python/ovs/flowviz/ovs-flowviz.conf b/python/ovs/flowviz/ovs-flowviz.conf index 165c453ec..82b5b47d2 100644 --- a/python/ovs/flowviz/ovs-flowviz.conf +++ b/python/ovs/flowviz/ovs-flowviz.conf @@ -28,6 +28,8 @@ # # HTML Styles # ============== +# * PORTION: An extra portion is supported: "backgroud" which defines the +# background color of the page. # * ELEMENT: # - color: defines the color in hex format @@ -65,6 +67,23 @@ console.key.highlighted.underline = true console.value.highlighted.underline = true console.delim.highlighted.underline = true +# html +html.background.color = #23282e +html.default.color = white +html.key.color = #5D86BA +html.value.color = #B0C4DE +html.delim.color = #B0C4DE + +html.key.resubmit.color = #005f00 +html.key.recirc.color = #005f00 +html.value.resubmit.color = #005f00 +html.value.recirc.color = #005f00 +html.key.output.color = #00d700 +html.value.output.color = #00d700 +html.key.highlighted.color = #FF00FF +html.value.highlighted.color = #FF00FF +html.key.drop.color = red + [styles.light] # If a color is omitted, the default terminal color will be used @@ -99,6 +118,7 @@ console.value.highlighted.underline = true console.delim.highlighted.underline = true # html +html.background.color = white html.key.color = #00005f html.value.color = #870000 html.key.resubmit.color = #00d700 From patchwork Tue May 7 14:30:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1932511 X-Patchwork-Delegate: echaudro@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=dBKPxyeZ; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VYghk71F8z1xnT for ; Wed, 8 May 2024 00:32:06 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id BFD9182322; Tue, 7 May 2024 14:32:04 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 2JodQgch5-la; Tue, 7 May 2024 14:31:56 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 26D0082285 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=dBKPxyeZ Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 26D0082285; Tue, 7 May 2024 14:31:39 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D360BC0077; Tue, 7 May 2024 14:31:38 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 20D81C0DD4 for ; Tue, 7 May 2024 14:31:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id B0E6582106 for ; Tue, 7 May 2024 14:31:28 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id fQw7R2_ISkGx for ; Tue, 7 May 2024 14:31:19 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp1.osuosl.org 300B682202 Authentication-Results: smtp1.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 300B682202 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 300B682202 for ; Tue, 7 May 2024 14:31:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1715092275; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Wp5Ks9EcoGVPttWelk+EvUsZTMXZG1dbW68T6yUEXCY=; b=dBKPxyeZWIOwVTDxWZizucqMnm0r3hhatNaCpZJpct05M6rfs3wpM2QjQxtCbJPUmHa7J3 8mEaIlYEXirovW9CJ26kP/lO5jo9TQQl8vFuGUrGvS0t8yrLRZpOvlhCvUe8oMAJ8l8iM9 j7EFZ8+IB7XvhuHPwauch/Yj9h5rIyU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-100-SY9CbjltPKS8PzSqQCP5VQ-1; Tue, 07 May 2024 10:31:13 -0400 X-MC-Unique: SY9CbjltPKS8PzSqQCP5VQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C89018025FC for ; Tue, 7 May 2024 14:31:12 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.88]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9D3EA3C31; Tue, 7 May 2024 14:31:11 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 7 May 2024 16:30:38 +0200 Message-ID: <20240507143044.3056232-13-amorenoz@redhat.com> In-Reply-To: <20240507143044.3056232-1-amorenoz@redhat.com> References: <20240507143044.3056232-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v4 12/12] documentation: Document ovs-flowviz. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add a man page for ovs-flowviz as well as a topic page with some more detailed examples. Signed-off-by: Adrian Moreno Acked-by: Eelco Chaudron --- Documentation/automake.mk | 4 +- Documentation/conf.py | 2 + Documentation/ref/index.rst | 1 + Documentation/ref/ovs-flowviz.8.rst | 531 ++++++++++++++++++++ Documentation/topics/flow-visualization.rst | 271 ++++++++++ Documentation/topics/index.rst | 1 + rhel/openvswitch-fedora.spec.in | 1 + rhel/openvswitch.spec.in | 1 + 8 files changed, 811 insertions(+), 1 deletion(-) create mode 100644 Documentation/ref/ovs-flowviz.8.rst create mode 100644 Documentation/topics/flow-visualization.rst diff --git a/Documentation/automake.mk b/Documentation/automake.mk index 47d2e336a..539870aa2 100644 --- a/Documentation/automake.mk +++ b/Documentation/automake.mk @@ -45,7 +45,7 @@ DOC_SOURCE = \ Documentation/topics/fuzzing/ovs-fuzzing-infrastructure.rst \ Documentation/topics/fuzzing/ovs-fuzzers.rst \ Documentation/topics/fuzzing/security-analysis-of-ovs-fuzzers.rst \ - Documentation/topics/testing.rst \ + Documentation/topics/flow-visualization.rst \ Documentation/topics/integration.rst \ Documentation/topics/language-bindings.rst \ Documentation/topics/networking-namespaces.rst \ @@ -55,6 +55,7 @@ DOC_SOURCE = \ Documentation/topics/ovsdb-replication.rst \ Documentation/topics/porting.rst \ Documentation/topics/record-replay.rst \ + Documentation/topics/testing.rst \ Documentation/topics/tracing.rst \ Documentation/topics/usdt-probes.rst \ Documentation/topics/userspace-checksum-offloading.rst \ @@ -162,6 +163,7 @@ RST_MANPAGES = \ ovs-actions.7.rst \ ovs-appctl.8.rst \ ovs-ctl.8.rst \ + ovs-flowviz.8.rst \ ovs-l3ping.8.rst \ ovs-parse-backtrace.8.rst \ ovs-pki.8.rst \ diff --git a/Documentation/conf.py b/Documentation/conf.py index 15785605a..3a82f23a7 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -120,6 +120,8 @@ _man_pages = [ u'utility for configuring running Open vSwitch daemons'), ('ovs-ctl.8', u'OVS startup helper script'), + ('ovs-flowviz.8', + u'utility for visualizing OpenFlow and datapath flows'), ('ovs-l3ping.8', u'check network deployment for L3 tunneling problems'), ('ovs-parse-backtrace.8', diff --git a/Documentation/ref/index.rst b/Documentation/ref/index.rst index 03ada932f..7f2fe6177 100644 --- a/Documentation/ref/index.rst +++ b/Documentation/ref/index.rst @@ -42,6 +42,7 @@ time: ovs-actions.7 ovs-appctl.8 ovs-ctl.8 + ovs-flowviz.8 ovs-l3ping.8 ovs-pki.8 ovs-sim.1 diff --git a/Documentation/ref/ovs-flowviz.8.rst b/Documentation/ref/ovs-flowviz.8.rst new file mode 100644 index 000000000..77b89fc65 --- /dev/null +++ b/Documentation/ref/ovs-flowviz.8.rst @@ -0,0 +1,531 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +=========== +ovs-flowviz +=========== + +Synopsis +======== + +``ovs-flowviz`` +[``[-i | --input] <[alias,]file>``] +[``[-c | --config] ``] +[``[-f | --filter] ``] +[``[-h | --highlight] ``] +[``--style