From patchwork Mon Nov 22 11:22:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558030 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=di1i/H5l; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyQ0X4xFkz9sX3 for ; Mon, 22 Nov 2021 22:24:44 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 91A0C40970; Mon, 22 Nov 2021 11:24:41 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R1bUXGO_AYAE; Mon, 22 Nov 2021 11:24:37 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id C3B3640593; Mon, 22 Nov 2021 11:24:34 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8E458C002F; Mon, 22 Nov 2021 11:24:33 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0F9DAC003E for ; Mon, 22 Nov 2021 11:24:32 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id DB57960673 for ; Mon, 22 Nov 2021 11:23:45 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KSBIK2ujkORz for ; Mon, 22 Nov 2021 11:23:44 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id BD81560A5A for ; Mon, 22 Nov 2021 11:23:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580223; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OP8HkWZs2I0Zj3w2U4ugRLzK9hRNJuZAlKv7SbkR6TQ=; b=di1i/H5l6JQjV0WhzJbWrH18XfipCj9CqMbSTkIbYI8aGGAZ3gS/MOQbpPTYHEnUS2tBp+ aO3fEvh37qN4LOcYgGgcdMDYOH2/rHLhMsPBCTrr0x1jPYJquAiGLqVqTWc9ArY7q1OFlg NhgyFcxGkpX3UfcoY7qYHDSPVFZA4ek= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-372-5h1-Zi5YOI2IJLPW7K8umg-1; Mon, 22 Nov 2021 06:23:40 -0500 X-MC-Unique: 5h1-Zi5YOI2IJLPW7K8umg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9644B806694 for ; Mon, 22 Nov 2021 11:23:39 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id C342860862; Mon, 22 Nov 2021 11:23:38 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:51 +0100 Message-Id: <20211122112256.2011194-14-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 13/18] python: detect changes in flow formatting code X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" In order to minimize the risk of having the python flow parsing code and the C flow formatting code divert, add a target that checks if the formatting code has been changed since the last revision and warn the developer if it has. The script also makes it easy to update the dependency file so hopefully it will not cause too much trouble for a developer that has modifed the file without changing the flow string format. Signed-off-by: Adrian Moreno --- .gitignore | 1 + python/automake.mk | 13 +++- python/build/flow-parse-deps.py | 101 ++++++++++++++++++++++++++++++++ python/ovs/flows/deps.py | 5 ++ 4 files changed, 118 insertions(+), 2 deletions(-) create mode 100755 python/build/flow-parse-deps.py create mode 100644 python/ovs/flows/deps.py diff --git a/.gitignore b/.gitignore index f1cdcf124..e6bca1cd2 100644 --- a/.gitignore +++ b/.gitignore @@ -79,3 +79,4 @@ testsuite.tmp.orig /Documentation/_build /.venv /cxx-check +/flowparse-deps-check diff --git a/python/automake.mk b/python/automake.mk index 21aa897f2..da6ef08b4 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -50,7 +50,8 @@ ovs_pyfiles = \ python/ovs/flows/ofp.py \ python/ovs/flows/ofp_act.py \ python/ovs/flows/odp.py \ - python/ovs/flows/filter.py + python/ovs/flows/filter.py \ + python/ovs/flows/deps.py # These python files are used at build time but not runtime, # so they are not installed. @@ -58,7 +59,8 @@ EXTRA_DIST += \ python/build/__init__.py \ python/build/nroff.py \ python/build/soutil.py \ - python/build/extract_ofp_fields.py + python/build/extract_ofp_fields.py \ + python/build/flow-parse-deps.py # PyPI support. EXTRA_DIST += \ @@ -135,3 +137,10 @@ $(srcdir)/python/ovs/flows/ofp_fields.py: $(srcdir)/build-aux/gen_ofp_field_deco EXTRA_DIST += python/ovs/flows/ofp_fields.py CLEANFILES += python/ovs/flows/ofp_fields.py +ALL_LOCAL += flowparse-deps-check +DEPS = $(shell $(AM_V_GEN)$(run_python) $(srcdir)/python/build/flow-parse-deps.py list) +flowparse-deps-check: $(srcdir)/python/build/flow-parse-deps.py $(DEPS) + echo $(DEPS) + $(AM_V_GEN)$(run_python) $(srcdir)/python/build/flow-parse-deps.py check + touch $@ +CLEANFILES += flowparse-deps-check diff --git a/python/build/flow-parse-deps.py b/python/build/flow-parse-deps.py new file mode 100755 index 000000000..9adb1d89c --- /dev/null +++ b/python/build/flow-parse-deps.py @@ -0,0 +1,101 @@ +#!/usr/bin/env python3 +# Copyright (c) 2021 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Breaks lines read from stdin into groups using blank lines as +# group separators, then sorts lines within the groups for +# reproducibility. + + +# ovs-test-ofparse is just a wrapper around ovs-ofctl +# that also runs the python flow parsing utility to check that flows are +# parseable + +import hashlib +import sys + +DEPENDENCIES = ["lib/ofp-actions.c", "lib/odp-util.c"] +DEPENDENCY_FILE = "python/ovs/flows/deps.py" + + +def usage(): + print( + """ +Usage {cmd} [check | update | list] +Tool to verify flow parsing python code is kept in sync with +flow printing C code. + +Commands: + check: check the dependencies are met + update: update the dependencies based on current file content + list: list the dependency files +""".format( + cmd=sys.argv[0] + ) + ) + + +def digest(filename): + with open(filename, "rb") as f: + return hashlib.md5(f.read()).hexdigest() + + +def main(): + if len(sys.argv) != 2: + usage() + sys.exit(1) + + if sys.argv[1] == "list": + print(" ".join(DEPENDENCIES)) + elif sys.argv[1] == "update": + dep_str = list() + for dep in DEPENDENCIES: + dep_str.append( + ' "{dep}": "{digest}"'.format(dep=dep, digest=digest(dep)) + ) + + depends = """# File automatically generated. Do not modify manually +dependencies = {{ +{dependencies_dict} +}}""".format( + dependencies_dict=",\n".join(dep_str) + ) + with open(DEPENDENCY_FILE, "w") as f: + print(depends, file=f) + + elif sys.argv[1] == "check": + from ovs.flows.deps import dependencies + + for dep in DEPENDENCIES: + expected = dependencies.get(dep) + if not expected or expected != digest(dep): + print( + """ +Dependency file {dep} has changed. +Please verify the flow output format has not changed or modify the python flow parsing code accordingly. + +Once you're done, update the dependencies by running '{cmd} update' and check in the new dependency file. +""".format( + dep=dep, + cmd=sys.argv[0], + ) + ) + return 2 + else: + usage() + sys.exit(1) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/python/ovs/flows/deps.py b/python/ovs/flows/deps.py new file mode 100644 index 000000000..aaf428917 --- /dev/null +++ b/python/ovs/flows/deps.py @@ -0,0 +1,5 @@ +# File automatically generated. Do not modify manually +dependencies = { + "lib/ofp-actions.c": "f108b3e119f03b3373064589aecdeaf0", + "lib/odp-util.c": "c946c21d8f644869ce778842167f9129" +}