From patchwork Mon Nov 22 11:22:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558017 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=a0Jy6Pvq; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyPyy3t8Pz9sRR for ; Mon, 22 Nov 2021 22:23:19 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id D951F403FD; Mon, 22 Nov 2021 11:23:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8TIRmOVzcVTK; Mon, 22 Nov 2021 11:23:15 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 9FF7440217; Mon, 22 Nov 2021 11:23:14 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D0228C0038; Mon, 22 Nov 2021 11:23:12 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2CDDFC0012 for ; Mon, 22 Nov 2021 11:23:11 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 1BB7B40209 for ; Mon, 22 Nov 2021 11:23:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jARKQqk4gQOp for ; Mon, 22 Nov 2021 11:23:10 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id DB381401D2 for ; Mon, 22 Nov 2021 11:23:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580188; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QIcXkKmSaM+haxSz0iODEjN+C6NpPy5Tq5DW8bzrVpQ=; b=a0Jy6Pvq9iH4OjuMx7DPMgyNq3HG6J5SuwNmDhB4tuQVGejCzBpbS8C5a0XMjGS+M2WGV0 XWNNAUrC6eCzs0evtjUmijUM0/nCTPDClWxROqG4Cl1+CCAsrtgQMuBpB9IOq6lCcebqpX Mt9SW9S+E54wcy7v4O92iUn1OMiiviw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-592-BhpZbcxqN06KWRj64x5mJg-1; Mon, 22 Nov 2021 06:23:05 -0500 X-MC-Unique: BhpZbcxqN06KWRj64x5mJg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 82CEE10168C0 for ; Mon, 22 Nov 2021 11:23:04 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 89AF860862; Mon, 22 Nov 2021 11:23:03 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:39 +0100 Message-Id: <20211122112256.2011194-2-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 01/18] python: add generic Key-Value parser X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Most of ofproto and dpif flows are based on key-value pairs. These key-value pairs can be represented in several ways, eg: key:value, key=value, key(value). Add the following classes that allow parsing of key-value strings: * KeyValue: holds a key-value pair * KeyMetadata: holds some metadata associated with a KeyValue such as the original key and value strings and their position in the global string * KVParser: is able to parse a string and extract it's key-value pairs as KeyValue instances. Before creating the KeyValue instance it tries to decode the value via the KVDecoders * KVDecoders holds a number of decoders that KVParser can use to decode key-value pairs. It accepts a dictionary of keys and callables to allow users to specify what decoder (i.e: callable) to use for each key Also, flake8 seems to be incorrectly reporting an error (E203) in: "slice[index + offset : index + offset]" which is PEP8 compliant. So, ignore this error. Signed-off-by: Adrian Moreno --- Makefile.am | 3 +- python/automake.mk | 6 +- python/ovs/flows/__init__.py | 0 python/ovs/flows/decoders.py | 19 +++ python/ovs/flows/kv.py | 282 +++++++++++++++++++++++++++++++++++ python/setup.py | 2 +- 6 files changed, 309 insertions(+), 3 deletions(-) create mode 100644 python/ovs/flows/__init__.py create mode 100644 python/ovs/flows/decoders.py create mode 100644 python/ovs/flows/kv.py diff --git a/Makefile.am b/Makefile.am index cb8076433..e38dd607c 100644 --- a/Makefile.am +++ b/Makefile.am @@ -392,6 +392,7 @@ ALL_LOCAL += flake8-check # E129 visually indented line with same indent as next logical line # E131 continuation line unaligned for hanging indent # E722 do not use bare except, specify exception instead +# E203 whitespace before ':' # W503 line break before binary operator # W504 line break after binary operator # F*** -- warnings native to flake8 @@ -403,7 +404,7 @@ ALL_LOCAL += flake8-check # H233 Python 3.x incompatible use of print operator # H238 old style class declaration, use new style (inherit from `object`) FLAKE8_SELECT = H231,H232,H233,H238 -FLAKE8_IGNORE = E121,E123,E125,E126,E127,E128,E129,E131,E722,W503,W504,F811,D,H,I +FLAKE8_IGNORE = E121,E123,E125,E126,E127,E128,E129,E131,E722,E203,W503,W504,F811,D,H,I flake8-check: $(FLAKE8_PYFILES) $(FLAKE8_WERROR)$(AM_V_GEN) \ src='$^' && \ diff --git a/python/automake.mk b/python/automake.mk index 767512f17..13aa2b4c3 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -41,7 +41,11 @@ ovs_pyfiles = \ python/ovs/util.py \ python/ovs/version.py \ python/ovs/vlog.py \ - python/ovs/winutils.py + python/ovs/winutils.py \ + python/ovs/flows/__init__.py \ + python/ovs/flows/decoders.py \ + python/ovs/flows/kv.py + # These python files are used at build time but not runtime, # so they are not installed. EXTRA_DIST += \ diff --git a/python/ovs/flows/__init__.py b/python/ovs/flows/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/python/ovs/flows/decoders.py b/python/ovs/flows/decoders.py new file mode 100644 index 000000000..bfb64e70e --- /dev/null +++ b/python/ovs/flows/decoders.py @@ -0,0 +1,19 @@ +""" Defines helpful decoders that can be used to decode information from the +flows + +A decoder is generally a callable that accepts a string and returns the value +object. +""" + + +def decode_default(value): + """Default decoder. + + It tries to convert into an integer value and, if it fails, just + returns the string. + """ + try: + ival = int(value, 0) + return ival + except ValueError: + return value diff --git a/python/ovs/flows/kv.py b/python/ovs/flows/kv.py new file mode 100644 index 000000000..0093a4e90 --- /dev/null +++ b/python/ovs/flows/kv.py @@ -0,0 +1,282 @@ +""" Common helper classes for flow (ofproto/dpif) parsing +""" + +import re +import functools + +from ovs.flows.decoders import decode_default + + +class ParseError(RuntimeError): + """Exception raised when an error occurs during parsing.""" + + pass + + +class KeyMetadata: + """Class for keeping key metadata. + + Attributes: + kpos (int): The position of the keyword in the parent string. + vpos (int): The position of the value in the parent string. + kstring (string): The keyword string as found in the flow string. + vstring (string): The value as found in the flow string. + end_del (bool): Whether the key has end delimiter. + """ + + def __init__(self, kpos, vpos, kstring, vstring, delim="", end_delim=""): + """Constructor""" + self.kpos = kpos + self.vpos = vpos + self.kstring = kstring + self.vstring = vstring + self.delim = delim + self.end_delim = end_delim + + def __str__(self): + return "key: [{},{}), val:[{}, {})".format( + self.kpos, + self.kpos + len(self.kstring), + self.vpos, + self.vpos + len(self.vstring), + ) + + def __repr__(self): + return "%s('%s')" % (self.__class__.__name__, self) + + +class KeyValue: + """Class for keeping key-value data + + Attributes: + key (str): The key string. + value (any): The value data. + meta (KeyMetadata): The key metadata. + """ + + def __init__(self, key, value, meta=None): + """Constructor""" + self.key = key + self.value = value + self.meta = meta + + def __str__(self): + return "{}: {} ({})".format(self.key, str(self.value), str(self.meta)) + + def __repr__(self): + return "%s('%s')" % (self.__class__.__name__, self) + + +class KVDecoders: + """KVDecoders class is used by KVParser to select how to decoode the value + of a specific keyword. + + A decoder is simply a function that accepts a value string + and returns the value objects to be stored. + The returned value may be of any type. + + Decoders may return a KeyValue instance to indicate that the keyword should + also be modified to match the one provided in the returned KeyValue + + The free_decoder, however, must return the key and value to be stored + + Args: + decoders (dict): Optional; A dictionary of decoders indexed by keyword. + default (callable): Optional; A decoder used if a match is not found in + configured decoders. If not provided, the default behavior is to + try to decode the value into an integer and, if that fails, + just return the string as-is. + default_free (callable): Optional; The decoder used if a match is not + found in configured decoders and it's a free value (e.g: + a value without a key) Defaults to returning the free value as + keyword and "True" as value. + The callable must accept a string and return a key, value pair + """ + + def __init__(self, decoders=None, default=None, default_free=None): + self._decoders = decoders or dict() + self._default = default or decode_default + self._default_free = default_free or self._default_free_decoder + + def decode(self, keyword, value_str): + """Decode a keyword and value. + + Args: + keyword (str): The keyword whose value is to be decoded. + value_str (str): The value string. + + Returns: + The key (str) and value(any) to be stored. + """ + + decoder = self._decoders.get(keyword) + if decoder: + result = decoder(value_str) + if isinstance(result, KeyValue): + keyword = result.key + value = result.value + else: + value = result + + return keyword, value + else: + if value_str: + return keyword, self._default(value_str) + else: + return self._default_free(keyword) + + @staticmethod + def _default_free_decoder(key): + """Default decoder for free kewords.""" + return key, True + + +delim_pattern = re.compile(r"(\(|=|:|,|\n|\r|\t|$)") +parenthesys_pattern = re.compile(r"(\(|\))") +end_pattern = re.compile(r"( |,|\n|\r|\t)") + + +class KVParser: + """KVParser parses a string looking for key-value pairs. + + Args: + decoders (KVDecoders): Optional; the KVDecoders instance to use. + """ + + def __init__(self, decoders=None): + """Constructor""" + self._decoders = decoders or KVDecoders() + self._keyval = list() + + def keys(self): + return list(kv.key for kv in self._keyval) + + def kv(self): + return self._keyval + + def __iter__(self): + return iter(self._keyval) + + def parse(self, string): + """Parse the key-value pairs in string. + + Args: + string (str): the string to parse. + + Raises: + ParseError if any parsing error occurs. + """ + kpos = 0 + while kpos < len(string) and string[kpos] != "\n": + # strip string + if string[kpos] == "," or string[kpos] == " ": + kpos += 1 + continue + + split_parts = delim_pattern.split(string[kpos:], 1) + # the delimiter should be included in the returned list + if len(split_parts) < 3: + break + + keyword = split_parts[0] + delimiter = split_parts[1] + rest = split_parts[2] + + value_str = "" + vpos = kpos + len(keyword) + 1 + end_delimiter = "" + + # Figure out the end of the value + # If the delimiter is ':' or '=', the end of the value is the end + # of the string or a ', ' + if delimiter in ("=", ":"): + value_parts = end_pattern.split(rest, 1) + value_str = value_parts[0] if len(value_parts) == 3 else rest + next_kpos = vpos + len(value_str) + + elif delimiter == "(": + # Find the next ')' + level = 1 + index = 0 + value_parts = parenthesys_pattern.split(rest) + for val in value_parts: + if val == "(": + level += 1 + elif val == ")": + level -= 1 + index += len(val) + if level == 0: + break + + if level != 0: + raise ParseError( + "Error parsing string {}: " + "Failed to find matching ')' in {}".format( + string, rest + ) + ) + + value_str = rest[: index - 1] + next_kpos = vpos + len(value_str) + 1 + end_delimiter = ")" + + # Exceptionally, if after the () we find -> {}, do not treat + # the content of the parenthesis as the value, consider + # ({})->{} as the string value + if index < len(rest) - 2 and rest[index : index + 2] == "->": + extra_val = rest[index + 2 :].split(",")[0] + value_str = "({})->{}".format(value_str, extra_val) + # remove the first "(" + vpos -= 1 + next_kpos = vpos + len(value_str) + end_delimiter = "" + + elif delimiter in (",", "\n", "\t", "\r", ""): + # key with no value + next_kpos = kpos + len(keyword) + vpos = -1 + + try: + key, val = self._decoders.decode(keyword, value_str) + except Exception as e: + raise ParseError( + "Error parsing key-value ({}, {})".format( + keyword, value_str + ) + ) from e + + meta = KeyMetadata( + kpos=kpos, + vpos=vpos, + kstring=keyword, + vstring=value_str, + delim=delimiter, + end_delim=end_delimiter, + ) + + self._keyval.append(KeyValue(key, val, meta)) + + kpos = next_kpos + + +def decode_nested_kv(decoders, value): + """A key-value decoder that extracts nested key-value pairs and returns + them in a dictionary + + Args: + decoders (KVDecoders): the KVDecoders to use. + value (str): the value string to decode. + """ + if not value: + # Mark as flag + return True + + parser = KVParser(decoders) + parser.parse(value) + return {kv.key: kv.value for kv in parser.kv()} + + +def nested_kv_decoder(decoders=None): + """Helper function that creates a nested kv decoder with given + KVDecoders""" + return functools.partial(decode_nested_kv, decoders) diff --git a/python/setup.py b/python/setup.py index cfe01763f..0e6b0ea39 100644 --- a/python/setup.py +++ b/python/setup.py @@ -71,7 +71,7 @@ setup_args = dict( author='Open vSwitch', author_email='dev@openvswitch.org', packages=['ovs', 'ovs.compat', 'ovs.compat.sortedcontainers', - 'ovs.db', 'ovs.unixctl'], + 'ovs.db', 'ovs.unixctl', 'ovs.flows'], keywords=['openvswitch', 'ovs', 'OVSDB'], license='Apache 2.0', classifiers=[ From patchwork Mon Nov 22 11:22:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558018 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=BXHepHfD; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyPyz2xshz9sX3 for ; Mon, 22 Nov 2021 22:23:22 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id EA08840236; Mon, 22 Nov 2021 11:23:14 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h0Y5KnSNMg1W; Mon, 22 Nov 2021 11:23:13 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 5002C400F3; Mon, 22 Nov 2021 11:23:12 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1EE94C001E; Mon, 22 Nov 2021 11:23:12 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 92EB2C0012 for ; Mon, 22 Nov 2021 11:23:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 7B58E6066D for ; Mon, 22 Nov 2021 11:23:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oLE4xU3HFi_c for ; Mon, 22 Nov 2021 11:23:09 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id 852F560663 for ; Mon, 22 Nov 2021 11:23:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580188; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4fMYR2CqW/6F24wzW4uxBXxaTx/LqgcjGyru+ufpfqs=; b=BXHepHfDggUck2xdYehFpC4QKW4JyHPo3cz4jeF4Wi+AFs/llBed+KUtYGgE8lvJRi6nZg AjSd9MpHd394gyw3GR5gsf1hvWUy9APG3ewAI+jdJMwYV+pG3VPG5No2XUXQo8BAdNzl0E CuvoNVBjmzjFkPuLT5qykMuRrcIYklU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-195-MOiXuvqqNk6jv-YQ2YbWaw-1; Mon, 22 Nov 2021 06:23:06 -0500 X-MC-Unique: MOiXuvqqNk6jv-YQ2YbWaw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E53CD1006AA1 for ; Mon, 22 Nov 2021 11:23:05 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id C07605BAE0; Mon, 22 Nov 2021 11:23:04 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:40 +0100 Message-Id: <20211122112256.2011194-3-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 02/18] python: add mask, ip and eth decoders X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add more decoders that can be used by KVParser. For IPv4 and IPv6 addresses, create a new class that wraps netaddr.IPAddress. For Ethernet addresses, create a new class that wraps netaddr.EUI. For Integers, create a new class that performs basic bitwise mask comparisons Signed-off-by: Adrian Moreno --- python/ovs/flows/decoders.py | 341 +++++++++++++++++++++++++++++++++++ python/setup.py | 2 +- 2 files changed, 342 insertions(+), 1 deletion(-) diff --git a/python/ovs/flows/decoders.py b/python/ovs/flows/decoders.py index bfb64e70e..bf7a94ae8 100644 --- a/python/ovs/flows/decoders.py +++ b/python/ovs/flows/decoders.py @@ -5,6 +5,15 @@ A decoder is generally a callable that accepts a string and returns the value object. """ +import netaddr + + +class Decoder: + """Base class for all decoder classes""" + + def to_json(self): + assert "function must be implemented by derived class" + def decode_default(value): """Default decoder. @@ -17,3 +26,335 @@ def decode_default(value): return ival except ValueError: return value + + +def decode_flag(value): + """Default a flag. It's exising is just flagged by returning True""" + return True + + +def decode_int(value): + """integer decoder + + Both base10 and base16 integers are supported + + Used for fields such as: + n_bytes=34 + metadata=0x4 + """ + return int(value, 0) + + +def decode_time(value): + """time decoder + + Used for fields such as: + duration=1234.123s + """ + if value == "never": + return value + + time_str = value.rstrip("s") + return float(time_str) + + +class IntMask(Decoder): + """Base class for Integer Mask decoder classes + + It supports decoding a value/mask pair. It has to be derived and size + has to be set to a specific value + """ + + size = None # size in bits + + def __init__(self, string): + if not self.size: + assert "IntMask should be derived and size should be fixed" + + parts = string.split("/") + if len(parts) > 1: + self._value = int(parts[0], 0) + self._mask = int(parts[1], 0) + if self._mask.bit_length() > self.size: + raise ValueError( + "Integer mask {} is bigger than size {}".format( + self._mask, self.size + ) + ) + else: + self._value = int(parts[0], 0) + self._mask = 2 ** self.size - 1 + + if self._value.bit_length() > self.size: + raise ValueError( + "Integer value {} is bigger than size {}".format( + self._value, self.size + ) + ) + + @property + def value(self): + return self._value + + @property + def mask(self): + return self._mask + + def max_mask(self): + return 2 ** self.size - 1 + + def fully(self): + """Returns if it's fully masked""" + return self._mask == self.max_mask() + + def min(self): + return self._value & self._mask + + def max(self): + return (self.max_mask() & ~self._mask) | (self._value & self._mask) + + def __str__(self): + if self.fully(): + return str(self._value) + else: + return "{}/{}".format(hex(self._value), hex(self._mask)) + + def __repr__(self): + return "%s('%s')" % (self.__class__.__name__, self) + + def __eq__(self, other): + if isinstance(other, IntMask): + return self.value == other.value and self.mask == other.mask + elif isinstance(other, int): + return self.value == other and self.mask == self.max_mask() + else: + raise ValueError("Cannot compare against ", other) + + def __contains__(self, other): + if isinstance(other, IntMask): + if other.fully(): + return other.value in self + return other.min() in self and other.max() in self + else: + return other & self._mask == self._value & self._mask + + def dict(self): + return {"value": self._value, "mask": self._mask} + + def to_json(self): + return self.dict() + + +class Mask8(IntMask): + size = 8 + + +class Mask16(IntMask): + size = 16 + + +class Mask32(IntMask): + size = 32 + + +class Mask64(IntMask): + size = 64 + + +class Mask128(IntMask): + size = 128 + + +class Mask992(IntMask): + size = 992 + + +def decode_mask(mask_size): + """value/mask decoder for values of specific size (bits) + + Used for fields such as: + reg0=0x248/0xff + """ + + class Mask(IntMask): + size = mask_size + __name__ = "Mask{}".format(size) + + return Mask + + +class EthMask(Decoder): + """EthMask represents an Ethernet address with optional mask + + It uses netaddr.EUI + + Attributes: + eth (netaddr.EUI): the ethernet address + mask (netaddr.EUI): Optional, the ethernet address mask + + Args: + string (str): A string representing the masked ethernet address + e.g: 00.11:22:33:44:55 or 01:00:22:00:33:00/01:00:00:00:00:00 + """ + + def __init__(self, string): + mask_parts = string.split("/") + self._eth = netaddr.EUI(mask_parts[0]) + if len(mask_parts) == 2: + self._mask = netaddr.EUI(mask_parts[1]) + else: + self._mask = None + + @property + def eth(self): + """The ethernet address""" + return self._eth + + @property + def mask(self): + """The ethernet address mask""" + return self._mask + + def __eq__(self, other): + """Returns True if this EthMask is numerically the same as other""" + return self._mask == other._mask and self._eth == other._eth + + def __contains__(self, other): + """ + Args: + other (netaddr.EUI): another Ethernet address + + Returns: + True if other falls into the masked address range + """ + if isinstance(other, EthMask): + if other._mask: + raise ValueError("EthMask mask comparison not supported") + return other._eth in self + + if self._mask: + return (other.value & self._mask.value) == ( + self._eth.value & self._mask.value + ) + else: + return other == self._eth + + def __str__(self): + if self._mask: + return "/".join( + [ + self._eth.format(netaddr.mac_unix), + self._mask.format(netaddr.mac_unix), + ] + ) + else: + return self._eth.format(netaddr.mac_unix) + + def __repr__(self): + return "%s('%s')" % (self.__class__.__name__, self) + + def to_json(self): + return str(self) + + +class IPMask(Decoder): + """IPMask stores an IPv6 or IPv4 and a mask + + It uses netaddr.IPAddress. The IPMask can be represented by a + netaddr.IPNetwork (if it's a valid cidr) or by an ip and a mask + + Args: + string (str): A string representing the ip/mask + """ + + def __init__(self, string): + """Constructor""" + self._ipnet = None + self._ip = None + self._mask = None + try: + self._ipnet = netaddr.IPNetwork(string) + except netaddr.AddrFormatError: + pass + + if not self._ipnet: + # It's not a valid CIDR. Store ip and mask indendently + parts = string.split("/") + if len(parts) != 2: + raise ValueError( + "value {}: is not an ipv4 or ipv6 address".format(string) + ) + try: + self._ip = netaddr.IPAddress(parts[0]) + self._mask = netaddr.IPAddress(parts[1]) + except netaddr.AddrFormatError as exc: + raise ValueError( + "value {}: is not an ipv4 or ipv6 address".format(string) + ) from exc + + def __eq__(self, other): + """Returns True if this IPMask is numerically the same as other""" + if isinstance(other, netaddr.IPNetwork): + return self._ipnet and self._ipnet == other + if isinstance(other, netaddr.IPAddress): + return self._ipnet and self._ipnet.ip == other + elif isinstance(other, IPMask): + if self._ipnet: + return self._ipnet == other._ipnet + + return self._ip == other._ip and self._mask == other._mask + else: + return False + + def __contains__(self, other): + """ + Args: + other (netaddr.IPAddres): another IP address + + Returns: + True if other falls into the masked ip range + """ + if isinstance(other, IPMask): + if not other._ipnet: + raise ValueError("ip/mask comparisons not supported") + + return ( + netaddr.IPAddress(other._ipnet.first) in self + and netaddr.IPAddress(other._ipnet.last) in self + ) + + elif isinstance(other, netaddr.IPAddress): + if self._ipnet: + return other in self._ipnet + return (other & self._mask) == (self._ip & self._mask) + + def cidr(self): + """ + Returns True if the IPMask is a valid CIDR + """ + return self._ipnet is not None + + @property + def ip(self): + """The IP address""" + if self._ipnet: + return self._ipnet.ip + return self._ip + + @property + def mask(self): + """The IP mask""" + if self._ipnet: + return self._ipnet.netmask + return self._mask + + def __str__(self): + if self._ipnet: + return str(self._ipnet) + return "/".join([str(self._ip), str(self._mask)]) + + def __repr__(self): + return "%s('%s')" % (self.__class__.__name__, self) + + def to_json(self): + return str(self) diff --git a/python/setup.py b/python/setup.py index 0e6b0ea39..b06370bd9 100644 --- a/python/setup.py +++ b/python/setup.py @@ -87,7 +87,7 @@ setup_args = dict( ext_modules=[setuptools.Extension("ovs._json", sources=["ovs/_json.c"], libraries=['openvswitch'])], cmdclass={'build_ext': try_build_ext}, - install_requires=['sortedcontainers'], + install_requires=['sortedcontainers', 'netaddr'], extras_require={':sys_platform == "win32"': ['pywin32 >= 1.0']}, ) From patchwork Mon Nov 22 11:22:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558019 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ArNsYlJe; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyPyz3rqZz9sXN for ; Mon, 22 Nov 2021 22:23:23 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 4BC9380F48; Mon, 22 Nov 2021 11:23:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OtwZaBSYw_Vs; Mon, 22 Nov 2021 11:23:18 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 80F9D80F1E; Mon, 22 Nov 2021 11:23:16 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 282FDC002F; Mon, 22 Nov 2021 11:23:16 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id A2A2FC0012 for ; Mon, 22 Nov 2021 11:23:12 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 761DF401D2 for ; Mon, 22 Nov 2021 11:23:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WgN1Wr9l7Cnd for ; Mon, 22 Nov 2021 11:23:10 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id AAE2D401F2 for ; Mon, 22 Nov 2021 11:23:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580189; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=thNfj2AYZMazGpkvXz5ZjUj1m0gOjSwwXt+ZRnnzqmY=; b=ArNsYlJeCmFzNbSUfE5/o/qkx6o4qZyi2MLGDukLmxReU3t4zMeKUqml0M9wf34Sy5QiHM o4MF2SS3HFkDt2khWMTG1uSnO4jav/G1IgPi+RxIe3PcvdrCt0X3S5+SCV6+0EecyxVrOY vePtMSAu9jmbgp1OU2cCS55qdeKwSC8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-213-mqJgP6Y0M4ukabKWUw1hkA-1; Mon, 22 Nov 2021 06:23:08 -0500 X-MC-Unique: mqJgP6Y0M4ukabKWUw1hkA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2EDB81006AA5 for ; Mon, 22 Nov 2021 11:23:07 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 360815BAE0; Mon, 22 Nov 2021 11:23:06 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:41 +0100 Message-Id: <20211122112256.2011194-4-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 03/18] python: add list parser X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Some openflow or dpif flows encode their arguments in lists, eg: "some_action(arg1,arg2,arg3)". In order to decode this in a way that can be then stored and queried, add ListParser and ListDecoders classes that parse lists into KeyValue instances. The ListParser / ListDecoders mechanism is quite similar to KVParser and KVDecoders. Since the "key" of the different KeyValue objects is now ommited, it has to be provided by ListDecoders. For example, take the openflow action "resubmit" that can be written as: resubmit([port],[table][,ct]) Can be decoded by creating a ListDecoders instance such as: ListDecoders([ ("port", decode_default), ("table", decode_int), ("ct", decode_flag), ]) Naturally, the order of the decoders must be kept. Signed-off-by: Adrian Moreno --- python/automake.mk | 3 +- python/ovs/flows/list.py | 123 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 125 insertions(+), 1 deletion(-) create mode 100644 python/ovs/flows/list.py diff --git a/python/automake.mk b/python/automake.mk index 13aa2b4c3..a3265292d 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -44,7 +44,8 @@ ovs_pyfiles = \ python/ovs/winutils.py \ python/ovs/flows/__init__.py \ python/ovs/flows/decoders.py \ - python/ovs/flows/kv.py + python/ovs/flows/kv.py \ + python/ovs/flows/list.py # These python files are used at build time but not runtime, # so they are not installed. diff --git a/python/ovs/flows/list.py b/python/ovs/flows/list.py new file mode 100644 index 000000000..d7ad315a6 --- /dev/null +++ b/python/ovs/flows/list.py @@ -0,0 +1,123 @@ +import re +import functools + +from ovs.flows.kv import KeyValue, KeyMetadata, ParseError +from ovs.flows.decoders import decode_default + + +class ListDecoders: + """ListDecoders is used by ListParser to decode the elements in the list + + A decoder is a function that accepts a value and returns its decoded + object + The list_decoder to be used is determined by index in the list provided to + ListDecoders is important. + + Args: + decoders (list of tuples): Optional, A list of tuples. + The first element in the tuple is the keyword associated with the + value. The second element in the tuple is the decoder function. + """ + + def __init__(self, decoders=None): + self._decoders = decoders or list() + + def decode(self, index, value_str): + """Decode the index'th element of the list + + Args: + index (int): the position in the list of the element ot decode + value_str (str): the value string to decode + """ + if index < 0 or index >= len(self._decoders): + return self._default_decoder(index, value_str) + + try: + key = self._decoders[index][0] + value = self._decoders[index][1](value_str) + return key, value + except Exception as e: + raise ParseError( + "Failed to decode value_str %s: %s" % (value_str, str(e)) + ) + + @staticmethod + def _default_decoder(index, value): + key = "elem_{}".format(index) + return key, decode_default(value) + + +class ListParser: + """ListParser parses a list of values and stores them as key-value pairs + + It uses a ListDecoders instance to decode each element in the list. + + Args: + decoders (ListDecoders): Optional, the decoders to use + delims (list): Optional, list of delimiters of the list. Defaults to + [','] + """ + + def __init__(self, decoders=None, delims=None): + self._decoders = decoders or ListDecoders() + self._keyval = list() + delims = delims or [","] + delims.append("$") + self._regexp = r"({})".format("|".join(delims)) + + def kv(self): + return self._keyval + + def __iter__(self): + return iter(self._keyval) + + def parse(self, string): + """Parse the list in string + + Args: + string (str): the string to parse + + Raises: + ParseError if any parsing error occurs. + """ + kpos = 0 + index = 0 + while kpos < len(string) and string[kpos] != "\n": + split_parts = re.split(self._regexp, string[kpos:], 1) + if len(split_parts) < 3: + break + + value_str = split_parts[0] + + key, value = self._decoders.decode(index, value_str) + + meta = KeyMetadata( + kpos=kpos, + vpos=kpos, + kstring=value_str, + vstring=value_str, + ) + self._keyval.append(KeyValue(key, value, meta)) + + kpos += len(value_str) + 1 + index += 1 + + +def decode_nested_list(decoders, delims, value): + """Extracts nested list from te string and returns it in a dictionary + them in a dictionary + + Args: + decoders (ListDecoders): the ListDecoders to use. + value (str): the value string to decode. + """ + parser = ListParser(decoders, delims) + parser.parse(value) + return {kv.key: kv.value for kv in parser.kv()} + + +def nested_list_decoder(decoders=None, delims=None): + """Helper function that creates a nested list decoder with given + ListDecoders + """ + return functools.partial(decode_nested_list, decoders, delims) From patchwork Mon Nov 22 11:22:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558021 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=a2U80O/y; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyPzB2h83z9sX3 for ; Mon, 22 Nov 2021 22:23:34 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id BF76E40443; Mon, 22 Nov 2021 11:23:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zfZOUfMbcvVn; Mon, 22 Nov 2021 11:23:29 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 1A6EC40447; Mon, 22 Nov 2021 11:23:28 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C571EC0037; Mon, 22 Nov 2021 11:23:27 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3D6EBC0037 for ; Mon, 22 Nov 2021 11:23:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 040F480F03 for ; Mon, 22 Nov 2021 11:23:22 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=fail (1024-bit key) reason="fail (body has been altered)" header.d=redhat.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0_EGBY9dhLXH for ; Mon, 22 Nov 2021 11:23:19 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 9764580F5E for ; Mon, 22 Nov 2021 11:23:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580198; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fTfXAOfvNodVZ+W63kl8bCPbCTm3ftX6JtGrNidgrek=; b=a2U80O/ytPOh1DRxj0bn4JspETIAcuDv3afeFdPUhw9mgdFTJDAJ+5SVaAMm+jinmZToH4 gy6GT1DPop4cdPqJpaoZq7grcNtbj/zgwgb9PH7tzS5KQvjhnbnwruvSlMVvtxDT8pn7Vm RzJdDcmL/kNH37MvhxaSfa/AbUEW+mQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-369-VkwZYUEYOLyzHFqvo-MiWA-1; Mon, 22 Nov 2021 06:23:14 -0500 X-MC-Unique: VkwZYUEYOLyzHFqvo-MiWA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 980D11006AA4 for ; Mon, 22 Nov 2021 11:23:13 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8285360862; Mon, 22 Nov 2021 11:23:07 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:42 +0100 Message-Id: <20211122112256.2011194-5-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 04/18] build-aux: split extract-ofp-fields X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" In order to be able to reuse the core extaction logic, split the command in two parts. The core extraction logic is moved to python/build while the command that writes the different files out of the extracted field info is kept in build-aux Signed-off-by: Adrian Moreno --- build-aux/extract-ofp-fields | 393 +---------------------------- python/automake.mk | 3 +- python/build/extract_ofp_fields.py | 386 ++++++++++++++++++++++++++++ 3 files changed, 397 insertions(+), 385 deletions(-) create mode 100644 python/build/extract_ofp_fields.py diff --git a/build-aux/extract-ofp-fields b/build-aux/extract-ofp-fields index 8766995d9..725718336 100755 --- a/build-aux/extract-ofp-fields +++ b/build-aux/extract-ofp-fields @@ -7,78 +7,16 @@ import re import xml.dom.minidom import build.nroff -line = "" - -# Maps from user-friendly version number to its protocol encoding. -VERSION = {"1.0": 0x01, - "1.1": 0x02, - "1.2": 0x03, - "1.3": 0x04, - "1.4": 0x05, - "1.5": 0x06} -VERSION_REVERSE = dict((v,k) for k, v in VERSION.items()) +from build.extract_ofp_fields import ( + extract_ofp_fields, + PREREQS, + OXM_CLASSES, + VERSION, + fatal, + n_errors +) -TYPES = {"u8": (1, False), - "be16": (2, False), - "be32": (4, False), - "MAC": (6, False), - "be64": (8, False), - "be128": (16, False), - "tunnelMD": (124, True)} - -FORMATTING = {"decimal": ("MFS_DECIMAL", 1, 8), - "hexadecimal": ("MFS_HEXADECIMAL", 1, 127), - "ct state": ("MFS_CT_STATE", 4, 4), - "Ethernet": ("MFS_ETHERNET", 6, 6), - "IPv4": ("MFS_IPV4", 4, 4), - "IPv6": ("MFS_IPV6", 16, 16), - "OpenFlow 1.0 port": ("MFS_OFP_PORT", 2, 2), - "OpenFlow 1.1+ port": ("MFS_OFP_PORT_OXM", 4, 4), - "frag": ("MFS_FRAG", 1, 1), - "tunnel flags": ("MFS_TNL_FLAGS", 2, 2), - "TCP flags": ("MFS_TCP_FLAGS", 2, 2), - "packet type": ("MFS_PACKET_TYPE", 4, 4)} - -PREREQS = {"none": "MFP_NONE", - "Ethernet": "MFP_ETHERNET", - "ARP": "MFP_ARP", - "VLAN VID": "MFP_VLAN_VID", - "IPv4": "MFP_IPV4", - "IPv6": "MFP_IPV6", - "IPv4/IPv6": "MFP_IP_ANY", - "NSH": "MFP_NSH", - "CT": "MFP_CT_VALID", - "MPLS": "MFP_MPLS", - "TCP": "MFP_TCP", - "UDP": "MFP_UDP", - "SCTP": "MFP_SCTP", - "ICMPv4": "MFP_ICMPV4", - "ICMPv6": "MFP_ICMPV6", - "ND": "MFP_ND", - "ND solicit": "MFP_ND_SOLICIT", - "ND advert": "MFP_ND_ADVERT"} - -# Maps a name prefix into an (experimenter ID, class) pair, so: -# -# - Standard OXM classes are written as (0, ) -# -# - Experimenter OXM classes are written as (, 0xffff) -# -# If a name matches more than one prefix, the longest one is used. -OXM_CLASSES = {"NXM_OF_": (0, 0x0000, 'extension'), - "NXM_NX_": (0, 0x0001, 'extension'), - "NXOXM_NSH_": (0x005ad650, 0xffff, 'extension'), - "OXM_OF_": (0, 0x8000, 'standard'), - "OXM_OF_PKT_REG": (0, 0x8001, 'standard'), - "ONFOXM_ET_": (0x4f4e4600, 0xffff, 'standard'), - "ERICOXM_OF_": (0, 0x1000, 'extension'), - - # This is the experimenter OXM class for Nicira, which is the - # one that OVS would be using instead of NXM_OF_ and NXM_NX_ - # if OVS didn't have those grandfathered in. It is currently - # used only to test support for experimenter OXM, since there - # are barely any real uses of experimenter OXM in the wild. - "NXOXM_ET_": (0x00002320, 0xffff, 'extension')} +VERSION_REVERSE = dict((v,k) for k, v in VERSION.items()) def oxm_name_to_class(name): prefix = '' @@ -95,39 +33,6 @@ def is_standard_oxm(name): return oxm_class_type == 'standard' -def decode_version_range(range): - if range in VERSION: - return (VERSION[range], VERSION[range]) - elif range.endswith('+'): - return (VERSION[range[:-1]], max(VERSION.values())) - else: - a, b = re.match(r'^([^-]+)-([^-]+)$', range).groups() - return (VERSION[a], VERSION[b]) - - -def get_line(): - global line - global line_number - line = input_file.readline() - line_number += 1 - if line == "": - fatal("unexpected end of input") - - -n_errors = 0 - - -def error(msg): - global n_errors - sys.stderr.write("%s:%d: %s\n" % (file_name, line_number, msg)) - n_errors += 1 - - -def fatal(msg): - error(msg) - sys.exit(1) - - def usage(): argv0 = os.path.basename(sys.argv[0]) print('''\ @@ -141,172 +46,6 @@ file to #include.\ sys.exit(0) -def make_sizeof(s): - m = re.match(r'(.*) up to (.*)', s) - if m: - struct, member = m.groups() - return "offsetof(%s, %s)" % (struct, member) - else: - return "sizeof(%s)" % s - - -def parse_oxms(s, prefix, n_bytes): - if s == 'none': - return () - - return tuple(parse_oxm(s2.strip(), prefix, n_bytes) for s2 in s.split(',')) - - -match_types = dict() - - -def parse_oxm(s, prefix, n_bytes): - global match_types - - m = re.match('([A-Z0-9_]+)\(([0-9]+)\) since(?: OF(1\.[0-9]+) and)? v([12]\.[0-9]+)$', s) - if not m: - fatal("%s: syntax error parsing %s" % (s, prefix)) - - name, oxm_type, of_version, ovs_version = m.groups() - - class_ = oxm_name_to_class(name) - if class_ is None: - fatal("unknown OXM class for %s" % name) - oxm_vendor, oxm_class, oxm_class_type = class_ - - if class_ in match_types: - if oxm_type in match_types[class_]: - fatal("duplicate match type for %s (conflicts with %s)" % - (name, match_types[class_][oxm_type])) - else: - match_types[class_] = dict() - match_types[class_][oxm_type] = name - - # Normally the oxm_length is the size of the field, but for experimenter - # OXMs oxm_length also includes the 4-byte experimenter ID. - oxm_length = n_bytes - if oxm_class == 0xffff: - oxm_length += 4 - - header = (oxm_vendor, oxm_class, int(oxm_type), oxm_length) - - if of_version: - if oxm_class_type == 'extension': - fatal("%s: OXM extension can't have OpenFlow version" % name) - if of_version not in VERSION: - fatal("%s: unknown OpenFlow version %s" % (name, of_version)) - of_version_nr = VERSION[of_version] - if of_version_nr < VERSION['1.2']: - fatal("%s: claimed version %s predates OXM" % (name, of_version)) - else: - if oxm_class_type == 'standard': - fatal("%s: missing OpenFlow version number" % name) - of_version_nr = 0 - - return (header, name, of_version_nr, ovs_version) - - -def parse_field(mff, comment): - f = {'mff': mff} - - # First line of comment is the field name. - m = re.match(r'"([^"]+)"(?:\s+\(aka "([^"]+)"\))?(?:\s+\(.*\))?\.', comment[0]) - if not m: - fatal("%s lacks field name" % mff) - f['name'], f['extra_name'] = m.groups() - - # Find the last blank line the comment. The field definitions - # start after that. - blank = None - for i in range(len(comment)): - if not comment[i]: - blank = i - if not blank: - fatal("%s: missing blank line in comment" % mff) - - d = {} - for key in ("Type", "Maskable", "Formatting", "Prerequisites", - "Access", "Prefix lookup member", - "OXM", "NXM", "OF1.0", "OF1.1"): - d[key] = None - for fline in comment[blank + 1:]: - m = re.match(r'([^:]+):\s+(.*)\.$', fline) - if not m: - fatal("%s: syntax error parsing key-value pair as part of %s" - % (fline, mff)) - key, value = m.groups() - if key not in d: - fatal("%s: unknown key" % key) - elif key == 'Code point': - d[key] += [value] - elif d[key] is not None: - fatal("%s: duplicate key" % key) - d[key] = value - for key, value in d.items(): - if not value and key not in ("OF1.0", "OF1.1", - "Prefix lookup member", "Notes"): - fatal("%s: missing %s" % (mff, key)) - - m = re.match(r'([a-zA-Z0-9]+)(?: \(low ([0-9]+) bits\))?$', d['Type']) - if not m: - fatal("%s: syntax error in type" % mff) - type_ = m.group(1) - if type_ not in TYPES: - fatal("%s: unknown type %s" % (mff, d['Type'])) - - f['n_bytes'] = TYPES[type_][0] - if m.group(2): - f['n_bits'] = int(m.group(2)) - if f['n_bits'] > f['n_bytes'] * 8: - fatal("%s: more bits (%d) than field size (%d)" - % (mff, f['n_bits'], 8 * f['n_bytes'])) - else: - f['n_bits'] = 8 * f['n_bytes'] - f['variable'] = TYPES[type_][1] - - if d['Maskable'] == 'no': - f['mask'] = 'MFM_NONE' - elif d['Maskable'] == 'bitwise': - f['mask'] = 'MFM_FULLY' - else: - fatal("%s: unknown maskable %s" % (mff, d['Maskable'])) - - fmt = FORMATTING.get(d['Formatting']) - if not fmt: - fatal("%s: unknown format %s" % (mff, d['Formatting'])) - f['formatting'] = d['Formatting'] - if f['n_bytes'] < fmt[1] or f['n_bytes'] > fmt[2]: - fatal("%s: %d-byte field can't be formatted as %s" - % (mff, f['n_bytes'], d['Formatting'])) - f['string'] = fmt[0] - - f['prereqs'] = d['Prerequisites'] - if f['prereqs'] not in PREREQS: - fatal("%s: unknown prerequisites %s" % (mff, d['Prerequisites'])) - - if d['Access'] == 'read-only': - f['writable'] = False - elif d['Access'] == 'read/write': - f['writable'] = True - else: - fatal("%s: unknown access %s" % (mff, d['Access'])) - - f['OF1.0'] = d['OF1.0'] - if not d['OF1.0'] in (None, 'exact match', 'CIDR mask'): - fatal("%s: unknown OF1.0 match type %s" % (mff, d['OF1.0'])) - - f['OF1.1'] = d['OF1.1'] - if not d['OF1.1'] in (None, 'exact match', 'bitwise mask'): - fatal("%s: unknown OF1.1 match type %s" % (mff, d['OF1.1'])) - - f['OXM'] = (parse_oxms(d['OXM'], 'OXM', f['n_bytes']) + - parse_oxms(d['NXM'], 'NXM', f['n_bytes'])) - - f['prefix'] = d["Prefix lookup member"] - - return f - - def protocols_to_c(protocols): if protocols == set(['of10', 'of11', 'oxm']): return 'OFPUTIL_P_ANY' @@ -417,120 +156,6 @@ def make_nx_match(meta_flow_h): for oline in output: print(oline) - -def extract_ofp_fields(fn): - global file_name - global input_file - global line_number - global line - - file_name = fn - input_file = open(file_name) - line_number = 0 - - fields = [] - - while True: - get_line() - if re.match('enum.*mf_field_id', line): - break - - while True: - get_line() - first_line_number = line_number - here = '%s:%d' % (file_name, line_number) - if (line.startswith('/*') - or line.startswith(' *') - or line.startswith('#') - or not line - or line.isspace()): - continue - elif re.match('}', line) or re.match('\s+MFF_N_IDS', line): - break - - # Parse the comment preceding an MFF_ constant into 'comment', - # one line to an array element. - line = line.strip() - if not line.startswith('/*'): - fatal("unexpected syntax between fields") - line = line[1:] - comment = [] - end = False - while not end: - line = line.strip() - if line.startswith('*/'): - get_line() - break - if not line.startswith('*'): - fatal("unexpected syntax within field") - - line = line[1:] - if line.startswith(' '): - line = line[1:] - if line.startswith(' ') and comment: - continuation = True - line = line.lstrip() - else: - continuation = False - - if line.endswith('*/'): - line = line[:-2].rstrip() - end = True - else: - end = False - - if continuation: - comment[-1] += " " + line - else: - comment += [line] - get_line() - - # Drop blank lines at each end of comment. - while comment and not comment[0]: - comment = comment[1:] - while comment and not comment[-1]: - comment = comment[:-1] - - # Parse the MFF_ constant(s). - mffs = [] - while True: - m = re.match('\s+(MFF_[A-Z0-9_]+),?\s?$', line) - if not m: - break - mffs += [m.group(1)] - get_line() - if not mffs: - fatal("unexpected syntax looking for MFF_ constants") - - if len(mffs) > 1 or '' in comment[0]: - for mff in mffs: - # Extract trailing integer. - m = re.match('.*[^0-9]([0-9]+)$', mff) - if not m: - fatal("%s lacks numeric suffix in register group" % mff) - n = m.group(1) - - # Search-and-replace within the comment, - # and drop lines that have for x != n. - instance = [] - for x in comment: - y = x.replace('', n) - if re.search('<[0-9]+>', y): - if ('<%s>' % n) not in y: - continue - y = re.sub('<[0-9]+>', '', y) - instance += [y.strip()] - fields += [parse_field(mff, instance)] - else: - fields += [parse_field(mffs[0], comment)] - continue - - input_file.close() - - if n_errors: - sys.exit(1) - - return fields ## ------------------------ ## ## Documentation Generation ## diff --git a/python/automake.mk b/python/automake.mk index a3265292d..b869eb355 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -52,7 +52,8 @@ ovs_pyfiles = \ EXTRA_DIST += \ python/build/__init__.py \ python/build/nroff.py \ - python/build/soutil.py + python/build/soutil.py \ + python/build/extract_ofp_fields.py # PyPI support. EXTRA_DIST += \ diff --git a/python/build/extract_ofp_fields.py b/python/build/extract_ofp_fields.py new file mode 100644 index 000000000..f6938b6dd --- /dev/null +++ b/python/build/extract_ofp_fields.py @@ -0,0 +1,386 @@ +import getopt +import sys +import os.path +import re +import xml.dom.minidom +import build.nroff + +line = "" + +# Maps from user-friendly version number to its protocol encoding. +VERSION = {"1.0": 0x01, + "1.1": 0x02, + "1.2": 0x03, + "1.3": 0x04, + "1.4": 0x05, + "1.5": 0x06} +VERSION_REVERSE = dict((v,k) for k, v in VERSION.items()) + +TYPES = {"u8": (1, False), + "be16": (2, False), + "be32": (4, False), + "MAC": (6, False), + "be64": (8, False), + "be128": (16, False), + "tunnelMD": (124, True)} + +FORMATTING = {"decimal": ("MFS_DECIMAL", 1, 8), + "hexadecimal": ("MFS_HEXADECIMAL", 1, 127), + "ct state": ("MFS_CT_STATE", 4, 4), + "Ethernet": ("MFS_ETHERNET", 6, 6), + "IPv4": ("MFS_IPV4", 4, 4), + "IPv6": ("MFS_IPV6", 16, 16), + "OpenFlow 1.0 port": ("MFS_OFP_PORT", 2, 2), + "OpenFlow 1.1+ port": ("MFS_OFP_PORT_OXM", 4, 4), + "frag": ("MFS_FRAG", 1, 1), + "tunnel flags": ("MFS_TNL_FLAGS", 2, 2), + "TCP flags": ("MFS_TCP_FLAGS", 2, 2), + "packet type": ("MFS_PACKET_TYPE", 4, 4)} + +PREREQS = {"none": "MFP_NONE", + "Ethernet": "MFP_ETHERNET", + "ARP": "MFP_ARP", + "VLAN VID": "MFP_VLAN_VID", + "IPv4": "MFP_IPV4", + "IPv6": "MFP_IPV6", + "IPv4/IPv6": "MFP_IP_ANY", + "NSH": "MFP_NSH", + "CT": "MFP_CT_VALID", + "MPLS": "MFP_MPLS", + "TCP": "MFP_TCP", + "UDP": "MFP_UDP", + "SCTP": "MFP_SCTP", + "ICMPv4": "MFP_ICMPV4", + "ICMPv6": "MFP_ICMPV6", + "ND": "MFP_ND", + "ND solicit": "MFP_ND_SOLICIT", + "ND advert": "MFP_ND_ADVERT"} + +# Maps a name prefix into an (experimenter ID, class) pair, so: +# +# - Standard OXM classes are written as (0, ) +# +# - Experimenter OXM classes are written as (, 0xffff) +# +# If a name matches more than one prefix, the longest one is used. +OXM_CLASSES = {"NXM_OF_": (0, 0x0000, 'extension'), + "NXM_NX_": (0, 0x0001, 'extension'), + "NXOXM_NSH_": (0x005ad650, 0xffff, 'extension'), + "OXM_OF_": (0, 0x8000, 'standard'), + "OXM_OF_PKT_REG": (0, 0x8001, 'standard'), + "ONFOXM_ET_": (0x4f4e4600, 0xffff, 'standard'), + "ERICOXM_OF_": (0, 0x1000, 'extension'), + + # This is the experimenter OXM class for Nicira, which is the + # one that OVS would be using instead of NXM_OF_ and NXM_NX_ + # if OVS didn't have those grandfathered in. It is currently + # used only to test support for experimenter OXM, since there + # are barely any real uses of experimenter OXM in the wild. + "NXOXM_ET_": (0x00002320, 0xffff, 'extension')} + +def oxm_name_to_class(name): + prefix = '' + class_ = None + for p, c in OXM_CLASSES.items(): + if name.startswith(p) and len(p) > len(prefix): + prefix = p + class_ = c + return class_ + + +def is_standard_oxm(name): + oxm_vendor, oxm_class, oxm_class_type = oxm_name_to_class(name) + return oxm_class_type == 'standard' + + +def get_line(): + global line + global line_number + line = input_file.readline() + line_number += 1 + if line == "": + fatal("unexpected end of input") + + +n_errors = 0 + + +def error(msg): + global n_errors + sys.stderr.write("%s:%d: %s\n" % (file_name, line_number, msg)) + n_errors += 1 + + +def fatal(msg): + error(msg) + sys.exit(1) + +def parse_oxms(s, prefix, n_bytes): + if s == 'none': + return () + + return tuple(parse_oxm(s2.strip(), prefix, n_bytes) for s2 in s.split(',')) + + +match_types = dict() + + +def parse_oxm(s, prefix, n_bytes): + global match_types + + m = re.match('([A-Z0-9_]+)\(([0-9]+)\) since(?: OF(1\.[0-9]+) and)? v([12]\.[0-9]+)$', s) + if not m: + fatal("%s: syntax error parsing %s" % (s, prefix)) + + name, oxm_type, of_version, ovs_version = m.groups() + + class_ = oxm_name_to_class(name) + if class_ is None: + fatal("unknown OXM class for %s" % name) + oxm_vendor, oxm_class, oxm_class_type = class_ + + if class_ in match_types: + if oxm_type in match_types[class_]: + fatal("duplicate match type for %s (conflicts with %s)" % + (name, match_types[class_][oxm_type])) + else: + match_types[class_] = dict() + match_types[class_][oxm_type] = name + + # Normally the oxm_length is the size of the field, but for experimenter + # OXMs oxm_length also includes the 4-byte experimenter ID. + oxm_length = n_bytes + if oxm_class == 0xffff: + oxm_length += 4 + + header = (oxm_vendor, oxm_class, int(oxm_type), oxm_length) + + if of_version: + if oxm_class_type == 'extension': + fatal("%s: OXM extension can't have OpenFlow version" % name) + if of_version not in VERSION: + fatal("%s: unknown OpenFlow version %s" % (name, of_version)) + of_version_nr = VERSION[of_version] + if of_version_nr < VERSION['1.2']: + fatal("%s: claimed version %s predates OXM" % (name, of_version)) + else: + if oxm_class_type == 'standard': + fatal("%s: missing OpenFlow version number" % name) + of_version_nr = 0 + + return (header, name, of_version_nr, ovs_version) + + +def parse_field(mff, comment): + f = {'mff': mff} + + # First line of comment is the field name. + m = re.match(r'"([^"]+)"(?:\s+\(aka "([^"]+)"\))?(?:\s+\(.*\))?\.', comment[0]) + if not m: + fatal("%s lacks field name" % mff) + f['name'], f['extra_name'] = m.groups() + + # Find the last blank line the comment. The field definitions + # start after that. + blank = None + for i in range(len(comment)): + if not comment[i]: + blank = i + if not blank: + fatal("%s: missing blank line in comment" % mff) + + d = {} + for key in ("Type", "Maskable", "Formatting", "Prerequisites", + "Access", "Prefix lookup member", + "OXM", "NXM", "OF1.0", "OF1.1"): + d[key] = None + for fline in comment[blank + 1:]: + m = re.match(r'([^:]+):\s+(.*)\.$', fline) + if not m: + fatal("%s: syntax error parsing key-value pair as part of %s" + % (fline, mff)) + key, value = m.groups() + if key not in d: + fatal("%s: unknown key" % key) + elif key == 'Code point': + d[key] += [value] + elif d[key] is not None: + fatal("%s: duplicate key" % key) + d[key] = value + for key, value in d.items(): + if not value and key not in ("OF1.0", "OF1.1", + "Prefix lookup member", "Notes"): + fatal("%s: missing %s" % (mff, key)) + + m = re.match(r'([a-zA-Z0-9]+)(?: \(low ([0-9]+) bits\))?$', d['Type']) + if not m: + fatal("%s: syntax error in type" % mff) + type_ = m.group(1) + if type_ not in TYPES: + fatal("%s: unknown type %s" % (mff, d['Type'])) + + f['n_bytes'] = TYPES[type_][0] + if m.group(2): + f['n_bits'] = int(m.group(2)) + if f['n_bits'] > f['n_bytes'] * 8: + fatal("%s: more bits (%d) than field size (%d)" + % (mff, f['n_bits'], 8 * f['n_bytes'])) + else: + f['n_bits'] = 8 * f['n_bytes'] + f['variable'] = TYPES[type_][1] + + if d['Maskable'] == 'no': + f['mask'] = 'MFM_NONE' + elif d['Maskable'] == 'bitwise': + f['mask'] = 'MFM_FULLY' + else: + fatal("%s: unknown maskable %s" % (mff, d['Maskable'])) + + fmt = FORMATTING.get(d['Formatting']) + if not fmt: + fatal("%s: unknown format %s" % (mff, d['Formatting'])) + f['formatting'] = d['Formatting'] + if f['n_bytes'] < fmt[1] or f['n_bytes'] > fmt[2]: + fatal("%s: %d-byte field can't be formatted as %s" + % (mff, f['n_bytes'], d['Formatting'])) + f['string'] = fmt[0] + + f['prereqs'] = d['Prerequisites'] + if f['prereqs'] not in PREREQS: + fatal("%s: unknown prerequisites %s" % (mff, d['Prerequisites'])) + + if d['Access'] == 'read-only': + f['writable'] = False + elif d['Access'] == 'read/write': + f['writable'] = True + else: + fatal("%s: unknown access %s" % (mff, d['Access'])) + + f['OF1.0'] = d['OF1.0'] + if not d['OF1.0'] in (None, 'exact match', 'CIDR mask'): + fatal("%s: unknown OF1.0 match type %s" % (mff, d['OF1.0'])) + + f['OF1.1'] = d['OF1.1'] + if not d['OF1.1'] in (None, 'exact match', 'bitwise mask'): + fatal("%s: unknown OF1.1 match type %s" % (mff, d['OF1.1'])) + + f['OXM'] = (parse_oxms(d['OXM'], 'OXM', f['n_bytes']) + + parse_oxms(d['NXM'], 'NXM', f['n_bytes'])) + + f['prefix'] = d["Prefix lookup member"] + + return f + +def extract_ofp_fields(fn): + global file_name + global input_file + global line_number + global line + + file_name = fn + input_file = open(file_name) + line_number = 0 + + fields = [] + + while True: + get_line() + if re.match('enum.*mf_field_id', line): + break + + while True: + get_line() + first_line_number = line_number + here = '%s:%d' % (file_name, line_number) + if (line.startswith('/*') + or line.startswith(' *') + or line.startswith('#') + or not line + or line.isspace()): + continue + elif re.match('}', line) or re.match('\s+MFF_N_IDS', line): + break + + # Parse the comment preceding an MFF_ constant into 'comment', + # one line to an array element. + line = line.strip() + if not line.startswith('/*'): + fatal("unexpected syntax between fields") + line = line[1:] + comment = [] + end = False + while not end: + line = line.strip() + if line.startswith('*/'): + get_line() + break + if not line.startswith('*'): + fatal("unexpected syntax within field") + + line = line[1:] + if line.startswith(' '): + line = line[1:] + if line.startswith(' ') and comment: + continuation = True + line = line.lstrip() + else: + continuation = False + + if line.endswith('*/'): + line = line[:-2].rstrip() + end = True + else: + end = False + + if continuation: + comment[-1] += " " + line + else: + comment += [line] + get_line() + + # Drop blank lines at each end of comment. + while comment and not comment[0]: + comment = comment[1:] + while comment and not comment[-1]: + comment = comment[:-1] + + # Parse the MFF_ constant(s). + mffs = [] + while True: + m = re.match('\s+(MFF_[A-Z0-9_]+),?\s?$', line) + if not m: + break + mffs += [m.group(1)] + get_line() + if not mffs: + fatal("unexpected syntax looking for MFF_ constants") + + if len(mffs) > 1 or '' in comment[0]: + for mff in mffs: + # Extract trailing integer. + m = re.match('.*[^0-9]([0-9]+)$', mff) + if not m: + fatal("%s lacks numeric suffix in register group" % mff) + n = m.group(1) + + # Search-and-replace within the comment, + # and drop lines that have for x != n. + instance = [] + for x in comment: + y = x.replace('', n) + if re.search('<[0-9]+>', y): + if ('<%s>' % n) not in y: + continue + y = re.sub('<[0-9]+>', '', y) + instance += [y.strip()] + fields += [parse_field(mff, instance)] + else: + fields += [parse_field(mffs[0], comment)] + continue + + input_file.close() + + if n_errors: + sys.exit(1) + + return fields From patchwork Mon Nov 22 11:22:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558022 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=i/Y6MDix; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyPzl0GvXz9sRR for ; Mon, 22 Nov 2021 22:24:03 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 50A3081831; Mon, 22 Nov 2021 11:24:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hY39PtXSBAkf; Mon, 22 Nov 2021 11:23:59 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 955FE817A9; Mon, 22 Nov 2021 11:23:58 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7912CC001E; Mon, 22 Nov 2021 11:23:58 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id A8633C002F for ; Mon, 22 Nov 2021 11:23:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 3AE41401C5 for ; Mon, 22 Nov 2021 11:23:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F7CVdMO0EudM for ; Mon, 22 Nov 2021 11:23:32 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id B59734023B for ; Mon, 22 Nov 2021 11:23:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580209; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=svSobSjKqoQmFs1ek830dxJbqfIAq3MklmwJs+7r6YE=; b=i/Y6MDixUjVYD3y+03/mi5pTJ28bu/qf6TTE2FuG/2M1NKP/SSHazb9gGuKnmMhsYbKRsw h4G6Q8Va2RmhBsrqY061API36baps68/DXXaw1YGE/asZfexD12WLZ0W+hgKr3dtiHDi9w GbEw6GWLFuHF/Ek7uheZVXWKoepHtRY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-97-J3hPEHvkMo65EaCQ2xeBgQ-1; Mon, 22 Nov 2021 06:23:28 -0500 X-MC-Unique: J3hPEHvkMo65EaCQ2xeBgQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 279AC10168C4 for ; Mon, 22 Nov 2021 11:23:27 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id D41C260862; Mon, 22 Nov 2021 11:23:13 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:43 +0100 Message-Id: <20211122112256.2011194-6-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 05/18] build-aux: generate ofp field decoders X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Based on meta-field information extracted by extract_ofp_fields, autogenerate the right decoder to be used Signed-off-by: Adrian Moreno --- build-aux/automake.mk | 3 +- build-aux/gen_ofp_field_decoders | 73 ++++++++++++++++++++++++++++++++ python/.gitignore | 1 + python/automake.mk | 7 +++ 4 files changed, 83 insertions(+), 1 deletion(-) create mode 100755 build-aux/gen_ofp_field_decoders diff --git a/build-aux/automake.mk b/build-aux/automake.mk index 6267ccd7c..a8bb0acfd 100644 --- a/build-aux/automake.mk +++ b/build-aux/automake.mk @@ -9,7 +9,8 @@ EXTRA_DIST += \ build-aux/sodepends.py \ build-aux/soexpand.py \ build-aux/text2c \ - build-aux/xml2nroff + build-aux/xml2nroff \ + build-aux/gen_ofp_field_decoders FLAKE8_PYFILES += \ $(srcdir)/build-aux/xml2nroff \ diff --git a/build-aux/gen_ofp_field_decoders b/build-aux/gen_ofp_field_decoders new file mode 100755 index 000000000..e60410af8 --- /dev/null +++ b/build-aux/gen_ofp_field_decoders @@ -0,0 +1,73 @@ +#!/bin/env python + +import argparse +import re +import os +import sys +import importlib + +import build.extract_ofp_fields as extract_fields + + +def main(): + parser = argparse.ArgumentParser( + description="Tool to generate python ofproto field decoders from" + "meta-flow information" + ) + parser.add_argument( + "metaflow", + metavar="FILE", + type=str, + help="Read meta-flow info from file", + ) + + args = parser.parse_args() + + fields = extract_fields.extract_ofp_fields(args.metaflow) + + field_decoders = {} + for field in fields: + decoder = get_decoder(field) + field_decoders[field.get("name")] = decoder + if field.get("extra_name"): + field_decoders[field.get("extra_name")] = decoder + + code = """ +# This file is auto-generated. Do not edit + +import functools +from ovs.flows import decoders + +field_decoders = {{ +{decoders} +}} +""".format( + decoders="\n".join( + [ + " '{name}': {decoder},".format(name=name, decoder=decoder) + for name, decoder in field_decoders.items() + ] + ) + ) + print(code) + + +def get_decoder(field): + formatting = field.get("formatting") + if formatting in ["decimal", "hexadecimal"]: + if field.get("mask") == "MFM_NONE": + return "decoders.decode_int" + else: + if field.get("n_bits") in [8, 16, 32, 64, 128, 992]: + return "decoders.Mask{}".format(field.get("n_bits")) + return "decoders.decode_mask({})".format(field.get("n_bits")) + elif formatting in ["IPv4", "IPv6"]: + return "decoders.IPMask" + elif formatting == "Ethernet": + return "decoders.EthMask" + else: + return "decoders.decode_default" + + +if __name__ == "__main__": + main() diff --git a/python/.gitignore b/python/.gitignore index 60ace6f05..c8ffd4574 100644 --- a/python/.gitignore +++ b/python/.gitignore @@ -1,2 +1,3 @@ dist/ *.egg-info +ovs/flows/ofp_fields.py diff --git a/python/automake.mk b/python/automake.mk index b869eb355..9dfc62fce 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -123,3 +123,10 @@ $(srcdir)/python/ovs/dirs.py: python/ovs/dirs.py.template mv $@.tmp $@ EXTRA_DIST += python/ovs/dirs.py.template CLEANFILES += python/ovs/dirs.py + +$(srcdir)/python/ovs/flows/ofp_fields.py: $(srcdir)/build-aux/gen_ofp_field_decoders include/openvswitch/meta-flow.h + $(AM_V_GEN)$(run_python) $< $(srcdir)/include/openvswitch/meta-flow.h > $@.tmp + $(AM_V_at)mv $@.tmp $@ +EXTRA_DIST += python/ovs/flows/ofp_fields.py +CLEANFILES += python/ovs/flows/ofp_fields.py + From patchwork Mon Nov 22 11:22:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558023 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Cu3vjn2n; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyPzy17q1z9sRR for ; Mon, 22 Nov 2021 22:24:14 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id AE34940423; Mon, 22 Nov 2021 11:24:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gRAQIcbOirRB; Mon, 22 Nov 2021 11:24:10 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 79114403F3; Mon, 22 Nov 2021 11:24:09 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2F1C2C0036; Mon, 22 Nov 2021 11:24:09 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id D0E1BC0012 for ; Mon, 22 Nov 2021 11:24:07 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 50EFC80F34 for ; Mon, 22 Nov 2021 11:23:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ImOSwcJcinP3 for ; Mon, 22 Nov 2021 11:23:34 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 99A9180F3C for ; Mon, 22 Nov 2021 11:23:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580213; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jbNVNWPRH9BlM5PcA5P8/vXQ3bBMN7IuWg1EIb6tHx8=; b=Cu3vjn2ne36yiN/YWZ0Xodfuu/WRiDpn7shomOqwvwGm0b2DIzeceIVXNf9NXm1vbaY89k YYt3sgS/em/lnQ4hRPiyvyWn+rXaL6Ze20ZzzoGdXYh+zKPv9iw9p9VR+FbZOqjEZV4AqZ uFPhJn+Jay5jJK1IdlW9NNv0BfoyZsc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-43-RTRAbeE2Mze1RBZkh7eerQ-1; Mon, 22 Nov 2021 06:23:29 -0500 X-MC-Unique: RTRAbeE2Mze1RBZkh7eerQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6C7071005E4C for ; Mon, 22 Nov 2021 11:23:28 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7F81C60862; Mon, 22 Nov 2021 11:23:27 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:44 +0100 Message-Id: <20211122112256.2011194-7-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 06/18] python: add flow base class X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" It simplifies the implementation of different types of flows by creating the concept of Section (e.g: match, action) and automatic accessors for all the provided Sections Signed-off-by: Adrian Moreno --- python/automake.mk | 3 +- python/ovs/flows/flow.py | 94 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 96 insertions(+), 1 deletion(-) create mode 100644 python/ovs/flows/flow.py diff --git a/python/automake.mk b/python/automake.mk index 9dfc62fce..136da26bd 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -45,7 +45,8 @@ ovs_pyfiles = \ python/ovs/flows/__init__.py \ python/ovs/flows/decoders.py \ python/ovs/flows/kv.py \ - python/ovs/flows/list.py + python/ovs/flows/list.py \ + python/ovs/flows/flow.py # These python files are used at build time but not runtime, # so they are not installed. diff --git a/python/ovs/flows/flow.py b/python/ovs/flows/flow.py new file mode 100644 index 000000000..ad5846c0b --- /dev/null +++ b/python/ovs/flows/flow.py @@ -0,0 +1,94 @@ +""" Defines the Flow class +""" + + +class Section(object): + """A section within a Flow + + Attributes: + name (str): name of the section + pos (int): position within the overall flow string + string (str): section string + data (list[KeyValue]): parsed data of the section + is_list (bool): whether the key-values shall be expressed as a list + (e.g: it allows repeated keys) + """ + + def __init__(self, name, pos, string, data, is_list=False): + self.name = name + self.pos = pos + self.string = string + self.data = data + self.is_list = is_list + + def __str__(self): + return "{} (at {}): {}".format(self.name, self.pos, self.string) + + def __repr__(self): + return "%s('%s')" % (self.__class__.__name__, self) + + def dict(self): + return {self.name: self.format_data()} + + def format_data(self): + if self.is_list: + return [{item.key: item.value} for item in self.data] + else: + return {item.key: item.value for item in self.data} + + +class Flow(object): + """The Flow class is a base class for other types of concrete flows + (such as OFproto Flows or DPIF Flows) + + For each section provided, the object will have the following attributes + {section} will return the sections data in a formatted way + {section}_kv will return the sections data as a list of KeyValues + + Args: + sections (list[Section]): list of sections that comprise the flow + orig (str): Original flow string + id (Any): Identifier + """ + + def __init__(self, sections, orig="", id=None): + self._sections = sections + self._orig = orig + self._id = id + for section in sections: + setattr( + self, section.name, self.section(section.name).format_data() + ) + setattr( + self, + "{}_kv".format(section.name), + self.section(section.name).data, + ) + + def section(self, name): + """Return the section by name""" + return next( + (sect for sect in self._sections if sect.name == name), None + ) + + @property + def id(self): + """Return the Flow ID""" + return self._id + + @property + def sections(self): + """Return the section by name""" + return self._sections + + @property + def orig(self): + return self._orig + + def dict(self): + """Returns the Flow information in a dictionary""" + flow_dict = {"orig": self.orig} + for section in self.sections: + flow_dict.update(section.dict()) + + return flow_dict From patchwork Mon Nov 22 11:22:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558026 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Ki/fWvbW; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyQ0C2bmcz9sRR for ; Mon, 22 Nov 2021 22:24:27 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 0F44341C67; Mon, 22 Nov 2021 11:24:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IWGGgtfjGlSg; Mon, 22 Nov 2021 11:24:19 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id A985641C44; Mon, 22 Nov 2021 11:24:17 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8259AC003C; Mon, 22 Nov 2021 11:24:15 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 40E6AC0037 for ; Mon, 22 Nov 2021 11:24:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 202AB4041B for ; Mon, 22 Nov 2021 11:23:39 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OnQc-x03_nTY for ; Mon, 22 Nov 2021 11:23:35 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id C9F87403AB for ; Mon, 22 Nov 2021 11:23:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580213; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5NFJ54aaJPVAm6bi/zGHyu+wgeN5VJNkW76Q/ud1v0Q=; b=Ki/fWvbWt20ZYzObTkW41sWAvfMvFJdlRiVuPMvQ4JOOv9FDtVCtQ2B3fUIFJHHvqWY5p8 Un9UMZ4X3Am925UWHA7mvuuidvZPzz2nuSERgZTc5BJJYFnpOztK/XZ4P5imnxX+fogmVv vE7FwptmaDjn+IupGAsseh+57LnW4GU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-140-kkXvIrzMOgq16Eu0OswAoQ-1; Mon, 22 Nov 2021 06:23:30 -0500 X-MC-Unique: kkXvIrzMOgq16Eu0OswAoQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DB1C91966320 for ; Mon, 22 Nov 2021 11:23:29 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id C1CB760862; Mon, 22 Nov 2021 11:23:28 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:45 +0100 Message-Id: <20211122112256.2011194-8-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 07/18] python: introduce OpenFlow Flow parsing X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Introduce OFPFlow class and all its decoders. Most of the decoders are generic (from decoders.py). Some have special syntax and need a specific implementation. Decoders for nat are moved to the common decoders.py because it's syntax is shared with other types of flows (e.g: dpif flows). Signed-off-by: Adrian Moreno --- python/automake.mk | 4 +- python/ovs/flows/decoders.py | 93 ++++++++ python/ovs/flows/ofp.py | 400 +++++++++++++++++++++++++++++++++++ python/ovs/flows/ofp_act.py | 233 ++++++++++++++++++++ 4 files changed, 729 insertions(+), 1 deletion(-) create mode 100644 python/ovs/flows/ofp.py create mode 100644 python/ovs/flows/ofp_act.py diff --git a/python/automake.mk b/python/automake.mk index 136da26bd..d1464d7f6 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -46,7 +46,9 @@ ovs_pyfiles = \ python/ovs/flows/decoders.py \ python/ovs/flows/kv.py \ python/ovs/flows/list.py \ - python/ovs/flows/flow.py + python/ovs/flows/flow.py \ + python/ovs/flows/ofp.py \ + python/ovs/flows/ofp_act.py # These python files are used at build time but not runtime, # so they are not installed. diff --git a/python/ovs/flows/decoders.py b/python/ovs/flows/decoders.py index bf7a94ae8..3def9f279 100644 --- a/python/ovs/flows/decoders.py +++ b/python/ovs/flows/decoders.py @@ -6,6 +6,7 @@ object. """ import netaddr +import re class Decoder: @@ -358,3 +359,95 @@ class IPMask(Decoder): def to_json(self): return str(self) + + +def decode_free_output(value): + """Decodes the output value when found free + (without the 'output' keyword)""" + try: + return "output", {"port": int(value)} + except ValueError: + return "output", {"port": value.strip('"')} + + +ipv4 = r"[\d\.]+" +ipv4_capture = r"({ipv4})".format(ipv4=ipv4) +ipv6 = r"[\w:]+" +ipv6_capture = r"(?:\[*)?({ipv6})(?:\]*)?".format(ipv6=ipv6) +port_range = r":(\d+)(?:-(\d+))?" +ip_range_regexp = r"{ip_cap}(?:-{ip_cap})?(?:{port_range})?" +ipv4_port_regex = re.compile( + ip_range_regexp.format(ip_cap=ipv4_capture, port_range=port_range) +) +ipv6_port_regex = re.compile( + ip_range_regexp.format(ip_cap=ipv6_capture, port_range=port_range) +) + + +def decode_ip_port_range(value): + """ + Decodes an IP and port range: + {ip_start}-{ip-end}:{port_start}-{port_end} + + IPv6 addresses are surrounded by "[" and "]" if port ranges are also + present + + Returns the following dictionary: + { + "addrs": { + "start": {ip_start} + "end": {ip_end} + } + "ports": { + "start": {port_start}, + "end": {port_end} + } + (the "ports" key might be omitted) + """ + if value.count(":") > 1: + match = ipv6_port_regex.match(value) + else: + match = ipv4_port_regex.match(value) + + ip_start = match.group(1) + ip_end = match.group(2) + port_start = match.group(3) + port_end = match.group(4) + + result = { + "addrs": { + "start": netaddr.IPAddress(ip_start), + "end": netaddr.IPAddress(ip_end or ip_start), + } + } + if port_start: + result["ports"] = { + "start": int(port_start), + "end": int(port_end or port_start), + } + + return result + + +def decode_nat(value): + """Decodes the 'nat' keyword of the ct action""" + if not value: + return True + + result = dict() + type_parts = value.split("=") + result["type"] = type_parts[0] + + if len(type_parts) > 1: + value_parts = type_parts[1].split(",") + if len(type_parts) != 2: + raise ValueError("Malformed nat action: %s" % value) + + ip_port_range = decode_ip_port_range(value_parts[0]) + + result = {"type": type_parts[0], **ip_port_range} + + for flag in value_parts[1:]: + result[flag] = True + + return result diff --git a/python/ovs/flows/ofp.py b/python/ovs/flows/ofp.py new file mode 100644 index 000000000..e56b08967 --- /dev/null +++ b/python/ovs/flows/ofp.py @@ -0,0 +1,400 @@ +""" Defines the parsers needed to parse ofproto flows +""" + +import functools + +from ovs.flows.kv import KVParser, KVDecoders, nested_kv_decoder +from ovs.flows.ofp_fields import field_decoders +from ovs.flows.flow import Flow, Section +from ovs.flows.list import ListDecoders, nested_list_decoder +from ovs.flows.decoders import ( + decode_default, + decode_flag, + decode_int, + decode_time, + decode_mask, + IPMask, + EthMask, + decode_free_output, + decode_nat, +) +from ovs.flows.ofp_act import ( + decode_output, + decode_field, + decode_controller, + decode_bundle, + decode_bundle_load, + decode_encap_ethernet, + decode_load_field, + decode_set_field, + decode_move_field, + decode_dec_ttl, + decode_chk_pkt_larger, + decode_zone, + decode_exec, + decode_learn, +) + + +class OFPFlow(Flow): + """OFPFLow represents an OpenFlow Flow""" + + def __init__(self, sections, orig="", id=None): + """Constructor""" + super(OFPFlow, self).__init__(sections, orig, id) + + def __str__(self): + if self._orig: + return self._orig + else: + return self.to_string() + + def to_string(self): + """Print a text representation of the flow""" + string = "Info: {}\n" + self.info + string += "Match : {}\n" + self.match + string += "Actions: {}\n " + self.actions + return string + + +class OFPFlowFactory: + """OpenFlow Flow Factory is a class capable of creating OFPFLow objects""" + + def __init__(self): + self.info_decoders = self._info_decoders() + self.match_decoders = KVDecoders( + {**self._field_decoders(), **self._flow_match_decoders()} + ) + self.act_decoders = self._act_decoders() + + def from_string(self, ofp_string, id=None): + """Parse a ofproto flow string + + The string is expected to have the follwoing format: + [flow data] [match] actions=[actions] + + :param ofp_string: a ofproto string as dumped by ovs-ofctl tool + :type ofp_string: str + + :return: an OFPFlow with the content of the flow string + :rtype: OFPFlow + """ + if " reply " in ofp_string: + return None + + sections = list() + parts = ofp_string.split("actions=") + if len(parts) != 2: + raise ValueError("malformed ofproto flow: %s" % ofp_string) + + actions = parts[1] + + field_parts = parts[0].rstrip(" ").rpartition(" ") + if len(field_parts) != 3: + raise ValueError("malformed ofproto flow: %s" % ofp_string) + + info = field_parts[0] + match = field_parts[2] + + iparser = KVParser(self.info_decoders) + iparser.parse(info) + isection = Section( + name="info", + pos=ofp_string.find(info), + string=info, + data=iparser.kv(), + ) + sections.append(isection) + + mparser = KVParser(self.match_decoders) + mparser.parse(match) + msection = Section( + name="match", + pos=ofp_string.find(match), + string=match, + data=mparser.kv(), + ) + sections.append(msection) + + aparser = KVParser(self.act_decoders) + aparser.parse(actions) + asection = Section( + name="actions", + pos=ofp_string.find(actions), + string=actions, + data=aparser.kv(), + is_list=True, + ) + sections.append(asection) + + return OFPFlow(sections, ofp_string, id) + + @classmethod + def _info_decoders(cls): + """Generate the match decoders""" + info = { + "table": decode_int, + "duration": decode_time, + "n_packet": decode_int, + "n_bytes": decode_int, + "cookie": decode_int, + "idle_timeout": decode_time, + "hard_timeout": decode_time, + "hard_age": decode_time, + } + return KVDecoders(info) + + @classmethod + def _flow_match_decoders(cls): + """Returns the decoders for key-values that are part of the flow match + but not a flow field""" + return { + "priority": decode_int, + } + + @classmethod + def _field_decoders(cls): + shorthands = [ + "eth", + "ip", + "ipv6", + "icmp", + "icmp6", + "tcp", + "tcp6", + "udp", + "udp6", + "sctp", + "arp", + "rarp", + "mpls", + "mplsm", + ] + + fields = {**field_decoders, **{key: decode_flag for key in shorthands}} + + # vlan_vid field is special. Although it is technically 12 bit wide, + # bit 12 is allowed to be set to 1 to indicate that the vlan header is + # present (see section VLAN FIELDS in + # http://www.openvswitch.org/support/dist-docs/ovs-fields.7.txt) + # Therefore, override the generated vlan_vid field size + fields["vlan_vid"] = decode_mask(13) + return fields + + @classmethod + def _output_actions_decoders(cls): + """Returns the decoders for the output actions""" + return { + "output": decode_output, + "drop": decode_flag, + "controller": decode_controller, + "enqueue": nested_list_decoder( + ListDecoders([("port", decode_default), ("queue", int)]), + delims=[",", ":"], + ), + "bundle": decode_bundle, + "bundle_load": decode_bundle_load, + "group": decode_default, + } + + @classmethod + def _encap_actions_decoders(cls): + """Returns the decoders for the encap actions""" + + return { + "pop_vlan": decode_flag, + "strip_vlan": decode_flag, + "push_vlan": decode_default, + "decap": decode_flag, + "encap": nested_kv_decoder( + KVDecoders( + { + "nsh": nested_kv_decoder( + KVDecoders( + { + "md_type": decode_default, + "tlv": nested_list_decoder( + ListDecoders( + [ + ("class", decode_int), + ("type", decode_int), + ("value", decode_int), + ] + ) + ), + } + ) + ), + }, + default=None, + default_free=decode_encap_ethernet, + ) + ), + } + + @classmethod + def _field_action_decoders(cls): + """Returns the decoders for the field modification actions""" + # Field modification actions + field_default_decoders = [ + "set_mpls_label", + "set_mpls_tc", + "set_mpls_ttl", + "mod_nw_tos", + "mod_nw_ecn", + "mod_tcp_src", + "mod_tcp_dst", + ] + return { + "load": decode_load_field, + "set_field": functools.partial( + decode_set_field, KVDecoders(cls._field_decoders()) + ), + "move": decode_move_field, + "mod_dl_dst": EthMask, + "mod_dl_src": EthMask, + "mod_nw_dst": IPMask, + "mod_nw_src": IPMask, + "dec_ttl": decode_dec_ttl, + "dec_mpls_ttl": decode_flag, + "dec_nsh_ttl": decode_flag, + "check_pkt_larger": decode_chk_pkt_larger, + **{field: decode_default for field in field_default_decoders}, + } + + @classmethod + def _meta_action_decoders(cls): + """Returns the decoders for the metadata actions""" + meta_default_decoders = ["set_tunnel", "set_tunnel64", "set_queue"] + return { + "pop_queue": decode_flag, + **{field: decode_default for field in meta_default_decoders}, + } + + @classmethod + def _fw_action_decoders(cls): + """Returns the decoders for the Firewalling actions""" + return { + "ct": nested_kv_decoder( + KVDecoders( + { + "commit": decode_flag, + "zone": decode_zone, + "table": decode_int, + "nat": decode_nat, + "force": decode_flag, + "exec": functools.partial( + decode_exec, + KVDecoders( + { + **cls._encap_actions_decoders(), + **cls._field_action_decoders(), + **cls._meta_action_decoders(), + } + ), + ), + "alg": decode_default, + } + ) + ), + "ct_clear": decode_flag, + } + + @classmethod + def _control_action_decoders(cls): + return { + "resubmit": nested_list_decoder( + ListDecoders( + [ + ("port", decode_default), + ("table", decode_int), + ("ct", decode_flag), + ] + ) + ), + "push": decode_field, + "pop": decode_field, + "exit": decode_flag, + "multipath": nested_list_decoder( + ListDecoders( + [ + ("fields", decode_default), + ("basis", decode_int), + ("algorithm", decode_default), + ("n_links", decode_int), + ("arg", decode_int), + ("dst", decode_field), + ] + ) + ), + } + + @classmethod + def _clone_actions_decoders(cls, action_decoders): + """Generate the decoders for clone actions + + Args: + action_decoders (dict): The decoders of the supported nested + actions + """ + return { + "learn": decode_learn( + { + **action_decoders, + "fin_timeout": nested_kv_decoder( + KVDecoders( + { + "idle_timeout": decode_time, + "hard_timeout": decode_time, + } + ) + ), + } + ), + "clone": functools.partial( + decode_exec, KVDecoders(action_decoders) + ), + } + + @classmethod + def _other_action_decoders(cls): + """Recoders for other actions (see man(7) ovs-actions)""" + return { + "conjunction": nested_list_decoder( + ListDecoders( + [("id", decode_int), ("k", decode_int), ("n", decode_int)] + ), + delims=[",", "/"], + ), + "note": decode_default, + "sample": nested_kv_decoder( + KVDecoders( + { + "probability": decode_int, + "collector_set_id": decode_int, + "obs_domain_id": decode_int, + "obs_point_id": decode_int, + "sampling_port": decode_default, + "ingress": decode_flag, + "egress": decode_flag, + } + ) + ), + } + + @classmethod + def _act_decoders(cls): + """Generate the actions decoders""" + + actions = { + **cls._output_actions_decoders(), + **cls._encap_actions_decoders(), + **cls._field_action_decoders(), + **cls._meta_action_decoders(), + **cls._fw_action_decoders(), + **cls._control_action_decoders(), + **cls._other_action_decoders(), + } + clone_actions = cls._clone_actions_decoders(actions) + actions.update(clone_actions) + return KVDecoders(actions, default_free=decode_free_output) diff --git a/python/ovs/flows/ofp_act.py b/python/ovs/flows/ofp_act.py new file mode 100644 index 000000000..bc6574999 --- /dev/null +++ b/python/ovs/flows/ofp_act.py @@ -0,0 +1,233 @@ +""" Defines decoders for openflow actions +""" + +import functools + +from ovs.flows.kv import nested_kv_decoder, KVDecoders, KeyValue, KVParser +from ovs.flows.decoders import ( + decode_default, + decode_time, + decode_flag, + decode_int, +) +from ovs.flows.ofp_fields import field_decoders + + +def decode_output(value): + """Decodes the output value + + Does not support field specification + """ + if len(value.split(",")) > 1: + return nested_kv_decoder()(value) + try: + return {"port": int(value)} + except ValueError: + return {"port": value.strip('"')} + + +def decode_controller(value): + """Decodes the controller action""" + if not value: + return KeyValue("output", "controller") + else: + # Try controller:max_len + try: + max_len = int(value) + return { + "max_len": max_len, + } + except ValueError: + pass + # controller(key[=val], ...) + return nested_kv_decoder()(value) + + +def decode_bundle_load(value): + return decode_bundle(value, True) + + +def decode_bundle(value, load=False): + """Decode bundle action""" + result = {} + keys = ["fields", "basis", "algorithm", "ofport"] + if load: + keys.append("dst") + + for key in keys: + parts = value.partition(",") + nvalue = parts[0] + value = parts[2] + if key == "ofport": + continue + result[key] = decode_default(nvalue) + + # Handle members: + mvalues = value.split("members:") + result["members"] = [int(port) for port in mvalues[1].split(",")] + return result + + +def decode_encap_ethernet(value): + """Decodes encap ethernet value""" + return "ethernet", int(value, 0) + + +def decode_field(value): + """Decodes a field as defined in the 'Field Specification' of the actions + man page: http://www.openvswitch.org/support/dist-docs/ovs-actions.7.txt + """ + parts = value.strip("]\n\r").split("[") + result = { + "field": parts[0], + } + + if len(parts) > 1 and parts[1]: + field_range = parts[1].split("..") + start = field_range[0] + end = field_range[1] if len(field_range) > 1 else start + if start: + result["start"] = int(start) + if end: + result["end"] = int(end) + + return result + + +def decode_load_field(value): + """Decodes 'load:value->dst' actions""" + parts = value.split("->") + if len(parts) != 2: + raise ValueError("Malformed load action : %s" % value) + + # If the load action is performed within a learn() action, + # The value can be specified as another field. + try: + return {"value": int(parts[0], 0), "dst": decode_field(parts[1])} + except ValueError: + return {"src": decode_field(parts[0]), "dst": decode_field(parts[1])} + + +def decode_set_field(field_decoders, value): + """Decodes 'set_field:value/mask->dst' actions + + The value is decoded by field_decoders which is a KVDecoders instance + Args: + field_decoders + """ + parts = value.split("->") + if len(parts) != 2: + raise ValueError("Malformed set_field action : %s" % value) + + val = parts[0] + dst = parts[1] + + val_result = field_decoders.decode(dst, val) + + return { + "value": {val_result[0]: val_result[1]}, + "dst": decode_field(dst), + } + + +def decode_move_field(value): + """Decodes 'move:src->dst' actions""" + parts = value.split("->") + if len(parts) != 2: + raise ValueError("Malformed move action : %s" % value) + + return { + "src": decode_field(parts[0]), + "dst": decode_field(parts[1]), + } + + +def decode_dec_ttl(value): + """Decodes dec_ttl and dec_ttl(id, id[2], ...) actions""" + if not value: + return True + return [int(idx) for idx in value.split(",")] + + +def decode_chk_pkt_larger(value): + """Decodes 'check_pkt_larger(pkt_len)->dst' actions""" + parts = value.split("->") + if len(parts) != 2: + raise ValueError("Malformed check_pkt_larger action : %s" % value) + + pkt_len = int(parts[0].strip("()")) + dst = decode_field(parts[1]) + return {"pkt_len": pkt_len, "dst": dst} + + +# CT decoders +def decode_zone(value): + """Decodes the 'zone' keyword of the ct action""" + try: + return int(value, 0) + except ValueError: + pass + return decode_field(value) + + +def decode_exec(action_decoders, value): + """Decodes the 'exec' keyword of the ct action + + Args: + decode_actions (KVDecoders): the decoders to be used to decode the + nested exec + value (string): the string to be decoded + """ + exec_parser = KVParser(action_decoders) + exec_parser.parse(value) + return [{kv.key: kv.value} for kv in exec_parser.kv()] + + +def decode_learn(action_decoders): + """Create the decoder to be used to decode the 'learn' action. + + The learn action can include any nested action, therefore we need decoders + for all possible actions. + + Args: + action_decoders (dict): dictionary of decoders to be used in nested + action decoding + + """ + + def decode_learn_field(decoder, value): + """Generates a decoder to be used for the 'field' argument of the + 'learn' action. + + The field can hold a value that should be decoded, either as a field, + or as a the value (see man(7) ovs-actions) + + Args: + decoder (callable): The decoder + + """ + if value in field_decoders.keys(): + # It's a field + return value + else: + return decoder(value) + + learn_field_decoders = { + field: functools.partial(decode_learn_field, decoder) + for field, decoder in field_decoders.items() + } + learn_decoders = { + **action_decoders, + **learn_field_decoders, + "idle_timeout": decode_time, + "hard_timeout": decode_time, + "priority": decode_int, + "cooke": decode_int, + "send_flow_rem": decode_flag, + "table": decode_int, + "delete_learned": decode_flag, + "limit": decode_int, + "result_dst": decode_field, + } + + return functools.partial(decode_exec, KVDecoders(learn_decoders)) From patchwork Mon Nov 22 11:22:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558024 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=iDgUfn6T; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyQ022s1tz9sRR for ; Mon, 22 Nov 2021 22:24:18 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id B4958410B8; Mon, 22 Nov 2021 11:24:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kBJyzmzyW_4c; Mon, 22 Nov 2021 11:24:15 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 68C9A41BF8; Mon, 22 Nov 2021 11:24:14 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 34E13C0036; Mon, 22 Nov 2021 11:24:14 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 03166C0012 for ; Mon, 22 Nov 2021 11:24:13 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id EE71C80B1C for ; Mon, 22 Nov 2021 11:23:37 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6LIqBlSEO9Mq for ; Mon, 22 Nov 2021 11:23:36 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id B637180F49 for ; Mon, 22 Nov 2021 11:23:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580215; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oXMglsVZetRuyyan8FiQ7vspz5bMLplRJvpqqvb4vOE=; b=iDgUfn6T7K6iCJmiD1MwbPVXZk+2IiZulLf/l3sg94ZeMWNCFRjD8zxG2cR4bpsIpLfrxh UaAAp5R7jTV7llqJEctamzJ4q3CZA5z3iqiNdEKnTVn3Sx15z/g0OqRXNdu89FchtKQny5 7kkBeLd+fQN7RCyl33nLfflH32g6LR0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-52-DpUhfED5NtuAodAwZ8tnNw-1; Mon, 22 Nov 2021 06:23:32 -0500 X-MC-Unique: DpUhfED5NtuAodAwZ8tnNw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 839C010168C0 for ; Mon, 22 Nov 2021 11:23:31 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 45E4260862; Mon, 22 Nov 2021 11:23:30 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:46 +0100 Message-Id: <20211122112256.2011194-9-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 08/18] python: add ovs datapath flow parsing X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" A ODPFlow is a Flow with the following sections: ufid info (e.g: bytes, packets, dp, etc) match actions Use a factory class ODPFlowFactory to cache the decoder objects. Only three datapath actions require special handling: gre: because it has double parenthesys geneve: because it supports many concatenated lists of options nat: we reuse the decoder used for openflow actions Signed-off-by: Adrian Moreno --- python/automake.mk | 3 +- python/ovs/flows/odp.py | 723 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 725 insertions(+), 1 deletion(-) create mode 100644 python/ovs/flows/odp.py diff --git a/python/automake.mk b/python/automake.mk index d1464d7f6..8b0713cfc 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -48,7 +48,8 @@ ovs_pyfiles = \ python/ovs/flows/list.py \ python/ovs/flows/flow.py \ python/ovs/flows/ofp.py \ - python/ovs/flows/ofp_act.py + python/ovs/flows/ofp_act.py \ + python/ovs/flows/odp.py # These python files are used at build time but not runtime, # so they are not installed. diff --git a/python/ovs/flows/odp.py b/python/ovs/flows/odp.py new file mode 100644 index 000000000..8fe721f86 --- /dev/null +++ b/python/ovs/flows/odp.py @@ -0,0 +1,723 @@ +""" Defines an Openvswitch Datapath Flow +""" +import re +from functools import partial + +from ovs.flows.flow import Flow, Section + +from ovs.flows.kv import ( + KVParser, + KVDecoders, + nested_kv_decoder, + decode_nested_kv, +) +from ovs.flows.decoders import ( + decode_default, + decode_time, + decode_int, + decode_mask, + Mask8, + Mask16, + Mask32, + Mask64, + Mask128, + IPMask, + EthMask, + decode_free_output, + decode_flag, + decode_nat, +) + + +class ODPFlow(Flow): + def __init__(self, sections, raw="", id_=None): + """Constructor""" + super(ODPFlow, self).__init__(sections, raw, id_) + + def __str__(self): + if self._orig: + return self._orig + else: + return self.to_string() + + def to_string(self): + """Print a text representation of the flow""" + string = "Info: {}\n" + self.info + string += "Match : {}\n" + self.match + string += "Actions: {}\n " + self.actions + return string + + +class ODPFlowFactory: + """Datapath Flow""" + + def __init__(self): + self.info_decoders = self._info_decoders() + self.match_decoders = self._match_decoders() + self.action_decoders = self._action_decoders() + + def from_string(self, odp_string, id=None): + """Parse a odp flow string + + The string is expected to have the follwoing format: + [ufid], [match] [flow data] actions:[actions] + + Args: + odp_string (str): a datapath flow string + + Returns: + an ODPFlow instance + """ + + sections = [] + + # If UFID present, parse it and + ufid_pos = odp_string.find("ufid:") + if ufid_pos >= 0: + ufid_string = odp_string[ + ufid_pos : (odp_string[ufid_pos:].find(",") + 1) + ] + ufid_parser = KVParser(KVDecoders({"ufid": decode_default})) + ufid_parser.parse(ufid_string) + if len(ufid_parser.kv()) != 1: + raise ValueError("malformed odp flow: %s" % odp_string) + sections.append( + Section("ufid", ufid_pos, ufid_string, ufid_parser.kv()) + ) + + action_pos = odp_string.find("actions:") + if action_pos < 0: + raise ValueError("malformed odp flow: %s" % odp_string) + + # rest of the string is between ufid and actions + rest = odp_string[ + (ufid_pos + len(ufid_string) if ufid_pos >= 0 else 0) : action_pos + ] + + action_pos += 8 # len("actions:") + actions = odp_string[action_pos:] + + field_parts = rest.lstrip(" ").partition(" ") + + if len(field_parts) != 3: + raise ValueError("malformed odp flow: %s" % odp_string) + + match = field_parts[0] + info = field_parts[2] + + iparser = KVParser(KVDecoders(self.info_decoders)) + iparser.parse(info) + isection = Section( + name="info", + pos=odp_string.find(info), + string=info, + data=iparser.kv(), + ) + sections.append(isection) + + mparser = KVParser(KVDecoders(self.match_decoders)) + mparser.parse(match) + msection = Section( + name="match", + pos=odp_string.find(match), + string=match, + data=mparser.kv(), + ) + sections.append(msection) + + aparser = KVParser( + KVDecoders(self.action_decoders, default_free=decode_free_output) + ) + aparser.parse(actions) + asection = Section( + name="actions", + pos=action_pos, + string=actions, + data=aparser.kv(), + is_list=True, + ) + sections.append(asection) + + return ODPFlow(sections, odp_string, id) + + @classmethod + def _action_decoders(cls): + _decoders = { + "drop": decode_flag, + "lb_output": decode_int, + "trunc": decode_int, + "recirc": decode_int, + "userspace": nested_kv_decoder( + KVDecoders( + { + "pid": decode_int, + "sFlow": nested_kv_decoder( + KVDecoders( + { + "vid": decode_int, + "pcp": decode_int, + "output": decode_int, + } + ) + ), + "slow_path": decode_default, + "flow_sample": nested_kv_decoder( + KVDecoders( + { + "probability": decode_int, + "collector_sed_id": decode_int, + "obs_domain_id": decode_int, + "obs_point_id": decode_int, + "output_port": decode_default, + "ingress": decode_flag, + "egress": decode_flag, + } + ) + ), + "ipfix": nested_kv_decoder( + KVDecoders( + { + "output_port": decode_default, + } + ) + ), + "controller": nested_kv_decoder( + KVDecoders( + { + "reason": decode_int, + "dont_send": decode_int, + "continuation": decode_int, + "recirc_id": decode_int, + "rule_cookie": decode_int, + "controller_id": decode_int, + "max_len": decode_int, + } + ) + ), + "userdata": decode_default, + "actions": decode_flag, + "tunnel_out_port": decode_int, + "push_eth": nested_kv_decoder( + KVDecoders( + { + "src": EthMask, + "dst": EthMask, + "type": decode_int, + } + ) + ), + "pop_eth": decode_flag, + } + ) + ), + "set": nested_kv_decoder(KVDecoders(cls._field_decoders())), + "push_vlan": nested_kv_decoder( + KVDecoders( + { + "vid": decode_int, + "pcp": decode_int, + "cfi": decode_int, + "tpid": decode_int, + } + ) + ), + "pop_vlan": decode_flag, + "push_nsh": nested_kv_decoder( + KVDecoders( + { + "flags": decode_int, + "ttl": decode_int, + "mdtype": decode_int, + "np": decode_int, + "spi": decode_int, + "si": decode_int, + "c1": decode_int, + "c2": decode_int, + "c3": decode_int, + "c4": decode_int, + "md2": decode_int, + } + ) + ), + "pop_nsh": decode_flag, + "tnl_pop": decode_int, + "ct_clear": decode_flag, + "ct": nested_kv_decoder( + KVDecoders( + { + "commit": decode_flag, + "force_commit": decode_flag, + "zone": decode_int, + "mark": Mask32, + "label": Mask128, + "helper": decode_default, + "timeout": decode_default, + "nat": decode_nat, + } + ) + ), + **cls._tnl_action_decoder(), + } + + _decoders["clone"] = nested_kv_decoder( + KVDecoders(decoders=_decoders, default_free=decode_free_output) + ) + + return { + **_decoders, + "sample": nested_kv_decoder( + KVDecoders( + { + "sample": (lambda x: float(x.strip("%"))), + "actions": nested_kv_decoder( + KVDecoders( + decoders=_decoders, + default_free=decode_free_output, + ) + ), + } + ) + ), + "check_pkt_len": nested_kv_decoder( + KVDecoders( + { + "size": decode_int, + "gt": nested_kv_decoder( + KVDecoders( + decoders=_decoders, + default_free=decode_free_output, + ) + ), + "le": nested_kv_decoder( + KVDecoders( + decoders=_decoders, + default_free=decode_free_output, + ) + ), + } + ) + ), + } + + @classmethod + def _tnl_action_decoder(cls): + return { + "tnl_push": nested_kv_decoder( + KVDecoders( + { + "tnl_port": decode_int, + "header": nested_kv_decoder( + KVDecoders( + { + "size": decode_int, + "type": decode_int, + "eth": nested_kv_decoder( + KVDecoders( + { + "src": EthMask, + "dst": EthMask, + "dl_type": decode_int, + } + ) + ), + "ipv4": nested_kv_decoder( + KVDecoders( + { + "src": IPMask, + "dst": IPMask, + "proto": decode_int, + "tos": decode_int, + "ttl": decode_int, + "frag": decode_int, + } + ) + ), + "ipv6": nested_kv_decoder( + KVDecoders( + { + "src": IPMask, + "dst": IPMask, + "label": decode_int, + "proto": decode_int, + "tclass": decode_int, + "hlimit": decode_int, + } + ) + ), + "udp": nested_kv_decoder( + KVDecoders( + { + "src": decode_int, + "dst": decode_int, + "dsum": Mask16, + } + ) + ), + "vxlan": nested_kv_decoder( + KVDecoders( + { + "flags": decode_int, + "vni": decode_int, + } + ) + ), + "geneve": nested_kv_decoder( + KVDecoders( + { + "oam": decode_flag, + "crit": decode_flag, + "vni": decode_int, + "options": partial( + decode_geneve, False + ), + } + ) + ), + "gre": decode_tnl_gre, + "erspan": nested_kv_decoder( + KVDecoders( + { + "ver": decode_int, + "sid": decode_int, + "idx": decode_int, + "sid": decode_int, + "dir": decode_int, + "hwid": decode_int, + } + ) + ), + "gtpu": nested_kv_decoder( + KVDecoders( + { + "flags": decode_int, + "msgtype": decode_int, + "teid": decode_int, + } + ) + ), + } + ) + ), + "out_port": decode_int, + } + ) + ) + } + + @classmethod + def _info_decoders(cls): + return { + "packets": decode_int, + "bytes": decode_int, + "used": decode_time, + "flags": decode_default, + "dp": decode_default, + } + + @classmethod + def _match_decoders(cls): + return { + **cls._field_decoders(), + "encap": nested_kv_decoder(KVDecoders(cls._field_decoders())), + } + + @classmethod + def _field_decoders(cls): + return { + "skb_priority": Mask32, + "skb_mark": Mask32, + "recirc_id": decode_int, + "dp_hash": Mask32, + "ct_state": decode_default, # TODO: Parse flags + "ct_zone": Mask16, + "ct_mark": Mask32, + "ct_label": Mask128, + "ct_tuple4": nested_kv_decoder( + KVDecoders( + { + "src": IPMask, + "dst": IPMask, + "proto": Mask8, + "tcp_src": Mask16, + "tcp_dst": Mask16, + } + ) + ), + "ct_tuple6": nested_kv_decoder( + KVDecoders( + { + "src": IPMask, + "dst": IPMask, + "proto": Mask8, + "tcp_src": Mask16, + "tcp_dst": Mask16, + } + ) + ), + "tunnel": nested_kv_decoder( + KVDecoders( + { + "tun_id": Mask64, + "src": IPMask, + "dst": IPMask, + "ipv6_src": IPMask, + "ipv6_dst": IPMask, + "tos": Mask8, + "ttl": Mask8, + "tp_src": Mask16, + "tp_dst": Mask16, + "erspan": nested_kv_decoder( + KVDecoders( + { + "ver": Mask8, + "idx": Mask32, + "sid": decode_int, + "dir": Mask8, + "hwid": Mask8, + } + ) + ), + "vxlan": nested_kv_decoder( + KVDecoders( + { + "gbp": nested_kv_decoder( + KVDecoders( + { + "id": Mask16, + "flags": Mask8, + } + ) + ) + } + ) + ), + "geneve": partial(decode_geneve, True), + "gtpu": nested_kv_decoder( + KVDecoders( + { + "flags": Mask8, + "msgtype": Mask8, + } + ) + ), + "flags": decode_default, + } + ) + ), + "in_port": decode_default, + "eth": nested_kv_decoder( + KVDecoders( + { + "src": EthMask, + "dst": EthMask, + } + ) + ), + "vlan": nested_kv_decoder( + KVDecoders( + { + "vid": Mask16, + "pcp": Mask16, + "cfi": Mask16, + } + ) + ), + "eth_type": Mask16, + "mpls": nested_kv_decoder( + KVDecoders( + { + "label": Mask32, + "tc": Mask32, + "ttl": Mask32, + "bos": Mask32, + } + ) + ), + "ipv4": nested_kv_decoder( + KVDecoders( + { + "src": IPMask, + "dst": IPMask, + "proto": Mask8, + "tos": Mask8, + "ttl": Mask8, + "frag": decode_default, + } + ) + ), + "ipv6": nested_kv_decoder( + KVDecoders( + { + "src": IPMask, + "dst": IPMask, + "label": decode_mask(20), + "proto": Mask8, + "tclass": Mask8, + "hlimit": Mask8, + "frag": decode_default, + } + ) + ), + "tcp": nested_kv_decoder( + KVDecoders( + { + "src": Mask16, + "dst": Mask16, + } + ) + ), + "tcp_flags": decode_default, + "udp": nested_kv_decoder( + KVDecoders( + { + "src": Mask16, + "dst": Mask16, + } + ) + ), + "sctp": nested_kv_decoder( + KVDecoders( + { + "src": Mask16, + "dst": Mask16, + } + ) + ), + "icmp": nested_kv_decoder( + KVDecoders( + { + "type": Mask8, + "code": Mask8, + } + ) + ), + "icmpv6": nested_kv_decoder( + KVDecoders( + { + "type": Mask8, + "code": Mask8, + } + ) + ), + "arp": nested_kv_decoder( + KVDecoders( + { + "sip": IPMask, + "tip": IPMask, + "op": Mask16, + "sha": EthMask, + "tha": EthMask, + } + ) + ), + "nd": nested_kv_decoder( + KVDecoders( + { + "target": IPMask, + "sll": EthMask, + "tll": EthMask, + } + ) + ), + "nd_ext": nested_kv_decoder( + KVDecoders( + { + "nd_reserved": Mask32, + "nd_options_type": Mask8, + } + ) + ), + "packet_type": nested_kv_decoder( + KVDecoders( + { + "ns": Mask16, + "id": Mask16, + } + ) + ), + "nsh": nested_kv_decoder( + KVDecoders( + { + "flags": Mask8, + "mdtype": Mask8, + "np": Mask8, + "spi": Mask32, + "si": Mask8, + "c1": Mask32, + "c2": Mask32, + "c3": Mask32, + "c4": Mask32, + } + ) + ), + } + + +def decode_geneve(mask, value): + """ + Decode geneve options. Used for both tnl_push(header(geneve(options()))) + action and tunnel(geneve()) match. + + It has the following format: + + {class=0xffff,type=0x80,len=4,0xa} + + Args: + mask (bool): Whether masking is supported + value (str): The value to decode + """ + if mask: + decoders = { + "class": Mask16, + "type": Mask8, + "len": Mask8, + } + + def free_decoder(value): + return "data", Mask128(value) + + else: + decoders = { + "class": decode_int, + "type": decode_int, + "len": decode_int, + } + + def free_decoder(value): + return "data", decode_int(value) + + result = [] + for opts in re.findall(r"{.*?}", value): + result.append( + decode_nested_kv( + KVDecoders(decoders=decoders, default_free=free_decoder), + opts.strip("{}"), + ) + ) + return result + + +def decode_tnl_gre(value): + """ + Decode tnl_push(header(gre())) action + + It has the following format: + + gre((flags=0x2000,proto=0x6558),key=0x1e241)) + + Args: + value (str): The value to decode + """ + + return decode_nested_kv( + KVDecoders( + { + "flags": decode_int, + "proto": decode_int, + "key": decode_int, + "csum": decode_int, + "seq": decode_int, + } + ), + value.replace("(", "").replace(")", ""), + ) From patchwork Mon Nov 22 11:22:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558027 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=aH+Ji2aV; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyQ0G0MMPz9sRR for ; Mon, 22 Nov 2021 22:24:30 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id A455681B58; Mon, 22 Nov 2021 11:24:27 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YmI7ObHcHZBi; Mon, 22 Nov 2021 11:24:23 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id CAE9381BB0; Mon, 22 Nov 2021 11:24:21 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 960CCC0037; Mon, 22 Nov 2021 11:24:18 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4FFDDC0043 for ; Mon, 22 Nov 2021 11:24:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 4B303607BB for ; Mon, 22 Nov 2021 11:23:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UPiSA5j6xXbD for ; Mon, 22 Nov 2021 11:23:37 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id 5BEA660A56 for ; Mon, 22 Nov 2021 11:23:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580216; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EPTwZcMJwV/2exYpHYlDsnzI72IeJNC+nn42yg5lx4E=; b=aH+Ji2aVXaFYNj17mH+rRl/Gg4ITvg1uP86Tjcv6xDuWgxvpPcyvdaMTHPfeowmTxII5wo RxPnvXfE9P+jIPppGnTR4ZDciG9dYQa+njC4sWmdChbkkmHlXh/7fOMLtoL/YCRGptlyPF GlV8BSQ+UJCxw85SP8g8TJx9JeOqwcU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-243-U5VJsr0_OououSm-NVCCdg-1; Mon, 22 Nov 2021 06:23:33 -0500 X-MC-Unique: U5VJsr0_OououSm-NVCCdg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A0939806688 for ; Mon, 22 Nov 2021 11:23:32 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id CF53660862; Mon, 22 Nov 2021 11:23:31 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:47 +0100 Message-Id: <20211122112256.2011194-10-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 09/18] python: add flow filtering syntax X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Based on pyparsing, create a very simple filtering syntax It supports basic logic statements (and, &, or, ||, not, !), numerical operations (<, >), equality (=) and masking (~=). The latter is only supported in certain fields (IntMask, EthMask, IPMask). Masking operation is semantically equivalent to "includes", therefore: ip_src ~= 192.168.1.1 means that ip_src field is either a host IP address equal to 192.168.1.1 or an IPMask that includes it (e.g: 192.168.1.1/24) Signed-off-by: Adrian Moreno --- python/automake.mk | 3 +- python/ovs/flows/filter.py | 225 +++++++++++++++++++++++++++++++++++++ python/setup.py | 2 +- 3 files changed, 228 insertions(+), 2 deletions(-) create mode 100644 python/ovs/flows/filter.py diff --git a/python/automake.mk b/python/automake.mk index 8b0713cfc..21aa897f2 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -49,7 +49,8 @@ ovs_pyfiles = \ python/ovs/flows/flow.py \ python/ovs/flows/ofp.py \ python/ovs/flows/ofp_act.py \ - python/ovs/flows/odp.py + python/ovs/flows/odp.py \ + python/ovs/flows/filter.py # These python files are used at build time but not runtime, # so they are not installed. diff --git a/python/ovs/flows/filter.py b/python/ovs/flows/filter.py new file mode 100644 index 000000000..2d3555a60 --- /dev/null +++ b/python/ovs/flows/filter.py @@ -0,0 +1,225 @@ +""" Defines a Flow Filtering syntax +""" +import pyparsing as pp +import netaddr +from functools import reduce +from operator import and_, or_ + +from ovs.flows.decoders import ( + decode_default, + decode_int, + Decoder, + IPMask, + EthMask, +) + + +class EvaluationResult: + """An EvaluationResult is the result of an evaluation. It contains the + boolean result and the list of key-values that were evaluated + + Note that since boolean operations (and, not, or) are based only on + __bool__ we use bitwise alternatives (&, ||, ~) + """ + + def __init__(self, result, *kv): + self.result = result + self.kv = kv if kv else list() + + def __and__(self, other): + """Logical and operation""" + return EvaluationResult( + self.result and other.result, *self.kv, *other.kv + ) + + def __or__(self, other): + """Logical or operation""" + return EvaluationResult( + self.result or other.result, *self.kv, *other.kv + ) + + def __invert__(self): + """Logical not operation""" + return EvaluationResult(not self.result, *self.kv) + + def __bool__(self): + """Boolean operation""" + return self.result + + def __repr__(self): + return "{} [{}]".format(self.result, self.kv) + + +class ClauseExpression: + operators = {} + type_decoders = { + int: decode_int, + netaddr.IPAddress: IPMask, + netaddr.EUI: EthMask, + bool: bool, + } + + def __init__(self, tokens): + self.field = tokens[0] + self.value = "" + self.operator = "" + + if len(tokens) > 1: + self.operator = tokens[1] + self.value = tokens[2] + + def __repr__(self): + return "{}(field: {}, operator: {}, value: {})".format( + self.__class__.__name__, self.field, self.operator, self.value + ) + + def _find_data_in_kv(self, kv_list): + """Find a KeyValue for evaluation in a list of KeyValue + + Args: + kv_list (list[KeyValue]): list of KeyValue to look into + + Returns: + If found, tuple (kv, data) where kv is the KeyValue that matched + and data is the data to be used for evaluation. None if not found. + """ + key_parts = self.field.split(".") + field = key_parts[0] + kvs = [kv for kv in kv_list if kv.key == field] + if not kvs: + return None + + for kv in kvs: + if kv.key == self.field: + # exact match + return (kv, kv.value) + if len(key_parts) > 1: + data = kv.value + for subkey in key_parts[1:]: + try: + data = data.get(subkey) + except Exception: + data = None + break + if not data: + break + if data: + return (kv, data) + return None + + def _find_keyval_to_evaluate(self, flow): + """Finds the key-value and data to use for evaluation on a flow + + Args: + flow(Flow): The flow where the lookup is performed + + Returns: + If found, tuple (kv, data) where kv is the KeyValue that matched + and data is the data to be used for evaluation. None if not found. + + """ + for section in flow.sections: + data = self._find_data_in_kv(section.data) + if data: + return data + return None + + def evaluate(self, flow): + """ + Return whether the clause is satisfied by the flow + + Args: + flow (Flow): the flow to evaluate + """ + result = self._find_keyval_to_evaluate(flow) + + if not result: + return EvaluationResult(False) + + keyval, data = result + + if not self.value and not self.operator: + # just asserting the existance of the key + return EvaluationResult(True, keyval) + + # Decode the value based on the type of data + if isinstance(data, Decoder): + decoder = data.__class__ + else: + decoder = self.type_decoders.get(data.__class__) or decode_default + + decoded_value = decoder(self.value) + + if self.operator == "=": + return EvaluationResult(decoded_value == data, keyval) + elif self.operator == "<": + return EvaluationResult(data < decoded_value, keyval) + elif self.operator == ">": + return EvaluationResult(data > decoded_value, keyval) + elif self.operator == "~=": + return EvaluationResult(decoded_value in data, keyval) + + +class BoolNot: + def __init__(self, t): + self.op, self.args = t[0] + + def __repr__(self): + return "NOT({})".format(self.args) + + def evaluate(self, flow): + return ~self.args.evaluate(flow) + + +class BoolAnd: + def __init__(self, pattern): + self.args = pattern[0][0::2] + + def __repr__(self): + return "AND({})".format(self.args) + + def evaluate(self, flow): + return reduce(and_, [arg.evaluate(flow) for arg in self.args]) + + +class BoolOr: + def __init__(self, pattern): + self.args = pattern[0][0::2] + + def evaluate(self, flow): + return reduce(or_, [arg.evaluate(flow) for arg in self.args]) + + def __repr__(self): + return "OR({})".format(self.args) + + +class OFFilter: + w = pp.Word(pp.alphanums + "." + ":" + "_" + "/" + "-") + operators = ( + pp.Literal("=") + | pp.Literal("~=") + | pp.Literal("<") + | pp.Literal(">") + | pp.Literal("!=") + ) + + clause = (w + operators + w) | w + clause.setParseAction(ClauseExpression) + + statement = pp.infixNotation( + clause, + [ + ("!", 1, pp.opAssoc.RIGHT, BoolNot), + ("not", 1, pp.opAssoc.RIGHT, BoolNot), + ("&&", 2, pp.opAssoc.LEFT, BoolAnd), + ("and", 2, pp.opAssoc.LEFT, BoolAnd), + ("||", 2, pp.opAssoc.LEFT, BoolOr), + ("or", 2, pp.opAssoc.LEFT, BoolOr), + ], + ) + + def __init__(self, expr): + self._filter = self.statement.parseString(expr) + + def evaluate(self, flow): + return self._filter[0].evaluate(flow) diff --git a/python/setup.py b/python/setup.py index b06370bd9..4e8a9761a 100644 --- a/python/setup.py +++ b/python/setup.py @@ -87,7 +87,7 @@ setup_args = dict( ext_modules=[setuptools.Extension("ovs._json", sources=["ovs/_json.c"], libraries=['openvswitch'])], cmdclass={'build_ext': try_build_ext}, - install_requires=['sortedcontainers', 'netaddr'], + install_requires=['sortedcontainers', 'netaddr', 'pyparsing'], extras_require={':sys_platform == "win32"': ['pywin32 >= 1.0']}, ) From patchwork Mon Nov 22 11:22:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558025 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=PIslBU8Z; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyQ075K2kz9sRR for ; Mon, 22 Nov 2021 22:24:23 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 9D80281CDC; Mon, 22 Nov 2021 11:24:21 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WkNluu8PSfDa; Mon, 22 Nov 2021 11:24:20 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 3B8808100F; Mon, 22 Nov 2021 11:24:19 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 51DF9C0044; Mon, 22 Nov 2021 11:24:17 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 15FFFC0044 for ; Mon, 22 Nov 2021 11:24:16 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 8668D403AB for ; Mon, 22 Nov 2021 11:23:39 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id y-Xx8KnbYV-s for ; Mon, 22 Nov 2021 11:23:38 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id C6B3340426 for ; Mon, 22 Nov 2021 11:23:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580217; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uwULzTBQfKjR6im6Ye2tkTQMevTWOvN8sJ9GCogvaJQ=; b=PIslBU8ZE5Deh/moHh3JMNdJo0YGpfms+fusahAc+ZLZuDSp2+G7F+7ieJpkPxprIg27f7 BsjnHSxaoAa0khrWPpjlk6wAQ6cAiLZ7ArQjTnID2e1WKx4HawBRbD4Yrt6JH0DgQPshWD C0qigRDvxfob37li/UPNNn1GbQ8bsRs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-230-pF0dDscDP_Se8T0LGCxO8w-1; Mon, 22 Nov 2021 06:23:34 -0500 X-MC-Unique: pF0dDscDP_Se8T0LGCxO8w-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B300780668C for ; Mon, 22 Nov 2021 11:23:33 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id DBF6A60862; Mon, 22 Nov 2021 11:23:32 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:48 +0100 Message-Id: <20211122112256.2011194-11-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 10/18] python: add a json encoder to flow fields X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The json encoder can be used to convert Flows to json Signed-off-by: Adrian Moreno --- python/ovs/flows/decoders.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/python/ovs/flows/decoders.py b/python/ovs/flows/decoders.py index 3def9f279..96bb56c06 100644 --- a/python/ovs/flows/decoders.py +++ b/python/ovs/flows/decoders.py @@ -5,6 +5,7 @@ A decoder is generally a callable that accepts a string and returns the value object. """ +import json import netaddr import re @@ -451,3 +452,17 @@ def decode_nat(value): result[flag] = True return result + + +class FlowEncoder(json.JSONEncoder): + """FlowEncoder is a json.JSONEncoder instance that can be used to + serialize flow fields + """ + + def default(self, obj): + if isinstance(obj, Decoder): + return obj.to_json() + elif isinstance(obj, netaddr.IPAddress): + return str(obj) + + return json.JSONEncoder.default(self, obj) From patchwork Mon Nov 22 11:22:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558028 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KiVGbrBZ; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyQ0P09qZz9sRR for ; Mon, 22 Nov 2021 22:24:36 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id BC4FA4053B; Mon, 22 Nov 2021 11:24:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P8CDbtsIH8qX; Mon, 22 Nov 2021 11:24:31 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id AC8AA403B3; Mon, 22 Nov 2021 11:24:29 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 77395C001E; Mon, 22 Nov 2021 11:24:29 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0563BC001E for ; Mon, 22 Nov 2021 11:24:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id F3E1C60A5B for ; Mon, 22 Nov 2021 11:23:44 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cz1JR9PEil8J for ; Mon, 22 Nov 2021 11:23:43 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id 9E51060A53 for ; Mon, 22 Nov 2021 11:23:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580222; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2gX+JnYpSgkQcCknqeRC6OLpIE8H/SOJyzH/zAa6a8A=; b=KiVGbrBZBMUPx3VzPIX9HFz0nvnDylgeg8PNakjXnwr8uJ9IYIDH3Yp1hu9fKlpOsShsos s9Da1hvS9Omr760ihmavbcvBY+vtxdydo5ReW5xtgWgZdPjnV+6xWRurjW0chQU/VcN+EJ /QPPTAj0CgbuSRPGcz7XEbSZ4Cjn8o0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-395-heww36d8P4y6NCySADoRAg-1; Mon, 22 Nov 2021 06:23:36 -0500 X-MC-Unique: heww36d8P4y6NCySADoRAg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BB2B710168C0 for ; Mon, 22 Nov 2021 11:23:35 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 124EF60862; Mon, 22 Nov 2021 11:23:33 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:49 +0100 Message-Id: <20211122112256.2011194-12-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 11/18] tests: wrap ovs-ofctl calls to test python parser X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Some ovs-ofctl commands are used to parse or dump openflow flows, specially in ofp-actions.at Use a wrapper around ovs-ofctl, called ovs-test-ofparse.py that, apart from calling ovs-ofctl, also parses its output (or input, depending on the command) to make sure the python flow parsing library can also parse the flows. Signed-off-by: Adrian Moreno --- tests/automake.mk | 4 +- tests/ofp-actions.at | 46 ++++++++--------- tests/ovs-test-ofparse.py | 103 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 129 insertions(+), 24 deletions(-) create mode 100755 tests/ovs-test-ofparse.py diff --git a/tests/automake.mk b/tests/automake.mk index 43731d097..5dc805ef4 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -20,7 +20,9 @@ EXTRA_DIST += \ tests/atlocal.in \ $(srcdir)/package.m4 \ $(srcdir)/tests/testsuite \ - $(srcdir)/tests/testsuite.patch + $(srcdir)/tests/testsuite.patch \ + $(srcdir)/tests/ovs-test-ofparse.py + COMMON_MACROS_AT = \ tests/ovsdb-macros.at \ diff --git a/tests/ofp-actions.at b/tests/ofp-actions.at index ef4898abb..0b79742be 100644 --- a/tests/ofp-actions.at +++ b/tests/ofp-actions.at @@ -327,7 +327,7 @@ AT_CAPTURE_FILE([input.txt]) AT_CAPTURE_FILE([expout]) AT_CAPTURE_FILE([experr]) AT_CHECK( - [ovs-ofctl '-vPATTERN:console:%c|%p|%m' parse-actions OpenFlow10 < input.txt], + [ovs-test-ofparse.py '-vPATTERN:console:%c|%p|%m' parse-actions OpenFlow10 < input.txt], [0], [expout], [experr]) AT_CLEANUP @@ -500,7 +500,7 @@ AT_CAPTURE_FILE([input.txt]) AT_CAPTURE_FILE([expout]) AT_CAPTURE_FILE([experr]) AT_CHECK( - [ovs-ofctl '-vPATTERN:console:%c|%p|%m' parse-actions OpenFlow11 < input.txt], + [ovs-test-ofparse.py '-vPATTERN:console:%c|%p|%m' parse-actions OpenFlow11 < input.txt], [0], [expout], [experr]) AT_CLEANUP @@ -735,7 +735,7 @@ AT_CAPTURE_FILE([input.txt]) AT_CAPTURE_FILE([expout]) AT_CAPTURE_FILE([experr]) AT_CHECK( - [ovs-ofctl '-vPATTERN:console:%c|%p|%m' parse-actions OpenFlow12 < input.txt], + [ovs-test-ofparse.py '-vPATTERN:console:%c|%p|%m' parse-actions OpenFlow12 < input.txt], [0], [expout], [experr]) AT_CLEANUP @@ -788,7 +788,7 @@ AT_CAPTURE_FILE([input.txt]) AT_CAPTURE_FILE([expout]) AT_CAPTURE_FILE([experr]) AT_CHECK( - [ovs-ofctl '-vPATTERN:console:%c|%p|%m' parse-actions OpenFlow13 < input.txt], + [ovs-test-ofparse.py '-vPATTERN:console:%c|%p|%m' parse-actions OpenFlow13 < input.txt], [0], [expout], [experr]) AT_CLEANUP @@ -817,16 +817,16 @@ AT_CAPTURE_FILE([input.txt]) AT_CAPTURE_FILE([expout]) AT_CAPTURE_FILE([experr]) AT_CHECK( - [ovs-ofctl '-vPATTERN:console:%c|%p|%m' parse-actions OpenFlow15 < input.txt], + [ovs-test-ofparse.py '-vPATTERN:console:%c|%p|%m' parse-actions OpenFlow15 < input.txt], [0], [expout], [experr]) AT_CLEANUP AT_SETUP([ofp-actions - inconsistent MPLS actions]) OVS_VSWITCHD_START dnl OK: Use fin_timeout action on TCP flow -AT_CHECK([ovs-ofctl -O OpenFlow11 -vwarn add-flow br0 'tcp actions=fin_timeout(idle_timeout=1)']) +AT_CHECK([ovs-test-ofparse.py -O OpenFlow11 -vwarn add-flow br0 'tcp actions=fin_timeout(idle_timeout=1)']) dnl Bad: Use fin_timeout action on TCP flow that has been converted to MPLS -AT_CHECK([ovs-ofctl -O OpenFlow11 -vwarn add-flow br0 'tcp actions=push_mpls:0x8847,fin_timeout(idle_timeout=1)'], +AT_CHECK([ovs-test-ofparse.py -O OpenFlow11 -vwarn add-flow br0 'tcp actions=push_mpls:0x8847,fin_timeout(idle_timeout=1)'], [1], [], [dnl ovs-ofctl: none of the usable flow formats (OpenFlow10,NXM) is among the allowed flow formats (OpenFlow11) ]) @@ -840,8 +840,8 @@ dnl In OpenFlow 1.3, set_field always sets all the bits in the field, dnl but when we translate to NXAST_LOAD we need to only set the bits that dnl actually exist (e.g. mpls_label only has 20 bits) otherwise OVS rejects dnl the "load" action as invalid. Check that we do this correctly. -AT_CHECK([ovs-ofctl -O OpenFlow13 add-flow br0 mpls,actions=set_field:10-\>mpls_label]) -AT_CHECK([ovs-ofctl -O OpenFlow10 dump-flows br0 | ofctl_strip], [0], [dnl +AT_CHECK([ovs-test-ofparse.py -O OpenFlow13 add-flow br0 mpls,actions=set_field:10-\>mpls_label]) +AT_CHECK([ovs-test-ofparse.py -O OpenFlow10 dump-flows br0 | ofctl_strip], [0], [dnl NXST_FLOW reply: mpls actions=load:0xa->OXM_OF_MPLS_LABEL[[]] ]) @@ -853,12 +853,12 @@ AT_KEYWORDS([ofp-actions]) OVS_VSWITCHD_START dnl OpenFlow 1.0 has an "enqueue" action. For OpenFlow 1.1+, we translate dnl it to a series of actions that accomplish the same thing. -AT_CHECK([ovs-ofctl -O OpenFlow10 add-flow br0 'actions=enqueue(123,456)']) -AT_CHECK([ovs-ofctl -O OpenFlow10 dump-flows br0 | ofctl_strip], [0], [dnl +AT_CHECK([ovs-test-ofparse.py -O OpenFlow10 add-flow br0 'actions=enqueue(123,456)']) +AT_CHECK([ovs-test-ofparse.py -O OpenFlow10 dump-flows br0 | ofctl_strip], [0], [dnl NXST_FLOW reply: actions=enqueue:123:456 ]) -AT_CHECK([ovs-ofctl -O OpenFlow13 dump-flows br0 | ofctl_strip], [0], [dnl +AT_CHECK([ovs-test-ofparse.py -O OpenFlow13 dump-flows br0 | ofctl_strip], [0], [dnl OFPST_FLOW reply (OF1.3): reset_counts actions=set_queue:456,output:123,pop_queue ]) @@ -870,12 +870,12 @@ AT_KEYWORDS([ofp-actions]) OVS_VSWITCHD_START dnl OpenFlow 1.1+ have a mod_nw_ttl action. For OpenFlow 1.0, we translate dnl it to an Open vSwitch extension. -AT_CHECK([ovs-ofctl -O OpenFlow11 add-flow br0 'ip,actions=mod_nw_ttl:123']) -AT_CHECK([ovs-ofctl -O OpenFlow10 dump-flows br0 | ofctl_strip], [0], [dnl +AT_CHECK([ovs-test-ofparse.py -O OpenFlow11 add-flow br0 'ip,actions=mod_nw_ttl:123']) +AT_CHECK([ovs-test-ofparse.py -O OpenFlow10 dump-flows br0 | ofctl_strip], [0], [dnl NXST_FLOW reply: ip actions=load:0x7b->NXM_NX_IP_TTL[[]] ]) -AT_CHECK([ovs-ofctl -O OpenFlow11 dump-flows br0 | ofctl_strip], [0], [dnl +AT_CHECK([ovs-test-ofparse.py -O OpenFlow11 dump-flows br0 | ofctl_strip], [0], [dnl OFPST_FLOW reply (OF1.1): ip actions=mod_nw_ttl:123 ]) @@ -889,16 +889,16 @@ OVS_VSWITCHD_START dnl OpenFlow 1.1, but no other version, has a "mod_nw_ecn" action. dnl Check that we translate it properly for OF1.0 and OF1.2. dnl (OF1.3+ should be the same as OF1.2.) -AT_CHECK([ovs-ofctl -O OpenFlow11 add-flow br0 'ip,actions=mod_nw_ecn:2']) -AT_CHECK([ovs-ofctl -O OpenFlow10 dump-flows br0 | ofctl_strip], [0], [dnl +AT_CHECK([ovs-test-ofparse.py -O OpenFlow11 add-flow br0 'ip,actions=mod_nw_ecn:2']) +AT_CHECK([ovs-test-ofparse.py -O OpenFlow10 dump-flows br0 | ofctl_strip], [0], [dnl NXST_FLOW reply: ip actions=load:0x2->NXM_NX_IP_ECN[[]] ]) -AT_CHECK([ovs-ofctl -O OpenFlow11 dump-flows br0 | ofctl_strip], [0], [dnl +AT_CHECK([ovs-test-ofparse.py -O OpenFlow11 dump-flows br0 | ofctl_strip], [0], [dnl OFPST_FLOW reply (OF1.1): ip actions=mod_nw_ecn:2 ]) -AT_CHECK([ovs-ofctl -O OpenFlow12 dump-flows br0 | ofctl_strip], [0], [dnl +AT_CHECK([ovs-test-ofparse.py -O OpenFlow12 dump-flows br0 | ofctl_strip], [0], [dnl OFPST_FLOW reply (OF1.2): ip actions=set_field:2->nw_ecn ]) @@ -911,8 +911,8 @@ dnl that anything that comes in as reg_load gets translated back to reg_load dnl on output. Perhaps this is somewhat inconsistent but it's what OVS dnl has done for multiple versions. AT_CHECK([ovs-ofctl del-flows br0]) -AT_CHECK([ovs-ofctl -O OpenFlow12 add-flow br0 'ip,actions=set_field:2->ip_ecn']) -AT_CHECK([ovs-ofctl -O OpenFlow11 dump-flows br0 | ofctl_strip], [0], [dnl +AT_CHECK([ovs-test-ofparse.py -O OpenFlow12 add-flow br0 'ip,actions=set_field:2->ip_ecn']) +AT_CHECK([ovs-test-ofparse.py -O OpenFlow11 dump-flows br0 | ofctl_strip], [0], [dnl OFPST_FLOW reply (OF1.1): ip actions=mod_nw_ecn:2 ]) @@ -920,9 +920,9 @@ OFPST_FLOW reply (OF1.1): dnl Check that OF1.2+ set_field to set ECN is translated for earlier OF dnl versions. AT_CHECK([ovs-ofctl del-flows br0]) -AT_CHECK([ovs-ofctl -O OpenFlow10 add-flow br0 'ip,actions=set_field:2->ip_ecn']) +AT_CHECK([ovs-test-ofparse.py -O OpenFlow10 add-flow br0 'ip,actions=set_field:2->ip_ecn']) AT_CHECK([ovs-ofctl del-flows br0]) -AT_CHECK([ovs-ofctl -O OpenFlow11 add-flow br0 'ip,actions=set_field:2->ip_ecn']) +AT_CHECK([ovs-test-ofparse.py -O OpenFlow11 add-flow br0 'ip,actions=set_field:2->ip_ecn']) OVS_VSWITCHD_STOP AT_CLEANUP diff --git a/tests/ovs-test-ofparse.py b/tests/ovs-test-ofparse.py new file mode 100755 index 000000000..89a0f3cf5 --- /dev/null +++ b/tests/ovs-test-ofparse.py @@ -0,0 +1,103 @@ +#!/usr/bin/env python3 +# Copyright (c) 2021 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Breaks lines read from stdin into groups using blank lines as +# group separators, then sorts lines within the groups for +# reproducibility. + + +# ovs-test-ofparse is just a wrapper around ovs-ofctl +# that also runs the python flow parsing utility to check that flows are +# parseable + +import subprocess +import sys +import re + +from ovs.flows.ofp import OFPFlowFactory + +diff_regexp = re.compile(r"\d{2}: (\d{2}|\(none\)) -> (\d{2}|\(none\))$") + + +def run_ofctl(with_stdin): + cmd = sys.argv + cmd[0] = "ovs-ofctl" + if with_stdin: + p = subprocess.Popen( + cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE + ) + input_data = sys.stdin.read() + out, err = p.communicate(input_data.encode("utf-8")) + + print(out.decode("utf-8"), file=sys.stdout, end="") + print(err.decode("utf-8"), file=sys.stderr, end="") + return p.returncode, out, err + else: + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + out, err = p.communicate() + + print(out.decode("utf-8"), file=sys.stdout, end="") + print(err.decode("utf-8"), file=sys.stderr, end="") + return p.returncode, out, err + + +def main(): + flows = list() + return_code = 0 + + if "parse-actions" in sys.argv: + return_code, out, err = run_ofctl(True) + + out_lines = out.decode("utf-8").split("\n") + for line in out_lines: + if not ( + "bad" in line # skip "bad action at..." + or line.strip() == "" # skip empty lines + or diff_regexp.match(line) # skip differences + ): + flows.append(line) + + elif "add-flow" in sys.argv: + return_code, out, err = run_ofctl(False) + flows.append(sys.argv[-1]) + + elif "dump-flows" in sys.argv: + return_code, out, err = run_ofctl(False) + out_lines = out.decode("utf-8").split("\n") + + for line in out_lines: + if not ( + "reply" in line # skip NXST_FLOW reply: + or line.strip() == "" # skip empty lines + ): + flows.append(line) + else: + print("Unsupported command: {}".format(sys.argv)) + sys.exit(1) + + if return_code == 0: + factory = OFPFlowFactory() + for flow in flows: + try: + _ = factory.from_string(flow) + except Exception as e: + print(e) + return 1 + + return return_code + + +if __name__ == "__main__": + sys.exit(main()) From patchwork Mon Nov 22 11:22:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558031 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=QpO2L8oD; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyQ0c33Ckz9sRR for ; Mon, 22 Nov 2021 22:24:48 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 7EA4561C93; Mon, 22 Nov 2021 11:24:45 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3QuSMbqps9kW; Mon, 22 Nov 2021 11:24:41 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 3D49561C72; Mon, 22 Nov 2021 11:24:37 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C5EBEC001E; Mon, 22 Nov 2021 11:24:35 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 07633C003F for ; Mon, 22 Nov 2021 11:24:34 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 337A081763 for ; Mon, 22 Nov 2021 11:23:50 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h-4GsaokP9lR for ; Mon, 22 Nov 2021 11:23:49 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id D95A480F1E for ; Mon, 22 Nov 2021 11:23:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580227; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3/1pQ8sm48qk4qgDuRMff41C8c9Vh0o0CLGlcSe3y90=; b=QpO2L8oDiAOj2JjDqHwdEbOujfoIgcGcHvMVDTOdapaIdFNhJza4/Mvvjf8em6MfoeWoDZ QZ70+Lduv3qwPCVDGo6Tkj1O5aToZ7x6m2KgXk7UJ/oKThs6omlsPeLsAzr++NvLeriNsu W/mVqpfOhsCCzROY/2BzOfmHQOZ3V04= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-421-sYzwQtBSPDuCl7bv4N0m-w-1; Mon, 22 Nov 2021 06:23:39 -0500 X-MC-Unique: sYzwQtBSPDuCl7bv4N0m-w-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8498410060F2 for ; Mon, 22 Nov 2021 11:23:38 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0D33E60862; Mon, 22 Nov 2021 11:23:35 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:50 +0100 Message-Id: <20211122112256.2011194-13-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 12/18] tests: Wrap test-odp to also run python parsers X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" test-odp is used to parse datapath flow actions and matches within the odp tests. Wrap calls to this tool in a python script that also parses them using the python flow parsing library. Signed-off-by: Adrian Moreno --- tests/automake.mk | 3 +- tests/odp.at | 36 ++++++++--------- tests/ovs-test-dpparse.py | 83 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 103 insertions(+), 19 deletions(-) create mode 100755 tests/ovs-test-dpparse.py diff --git a/tests/automake.mk b/tests/automake.mk index 5dc805ef4..be69f3d16 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -21,7 +21,8 @@ EXTRA_DIST += \ $(srcdir)/package.m4 \ $(srcdir)/tests/testsuite \ $(srcdir)/tests/testsuite.patch \ - $(srcdir)/tests/ovs-test-ofparse.py + $(srcdir)/tests/ovs-test-ofparse.py \ + $(srcdir)/tests/ovs-test-dpparse.py COMMON_MACROS_AT = \ diff --git a/tests/odp.at b/tests/odp.at index 07a5cfe39..69e86c5c1 100644 --- a/tests/odp.at +++ b/tests/odp.at @@ -105,7 +105,7 @@ sed -i'back' 's/\(skb_mark(0)\),\(ct\)/\1,ct_state(0),ct_zone(0),\2/' odp-out.tx sed -i'back' 's/\(skb_mark([[^)]]*)\),\(recirc\)/\1,ct_state(0),ct_zone(0),ct_mark(0),ct_label(0),\2/' odp-out.txt sed -i'back' 's/\(in_port(1)\),\(eth\)/\1,packet_type(ns=0,id=0),\2/' odp-out.txt -AT_CHECK_UNQUOTED([ovstest test-odp parse-keys < odp-in.txt], [0], [`cat odp-out.txt` +AT_CHECK_UNQUOTED([ovs-test-dpparse.py ovstest test-odp parse-keys < odp-in.txt], [0], [`cat odp-out.txt` ]) AT_CLEANUP @@ -192,7 +192,7 @@ sed -n 's/,frag=no),.*/,frag=later)/p' odp-base.txt sed 's/^/skb_priority(0),tunnel(tun_id=0xfedcba9876543210,src=10.0.0.1,dst=10.0.0.2,ttl=128,erspan(ver=2,dir=1,hwid=0x7/0xf),flags(df|key)),skb_mark(0),recirc_id(0),dp_hash(0),/' odp-base.txt ) > odp.txt AT_CAPTURE_FILE([odp.txt]) -AT_CHECK_UNQUOTED([ovstest test-odp parse-wc-keys < odp.txt], [0], [`cat odp.txt` +AT_CHECK_UNQUOTED([ovs-test-dpparse.py ovstest test-odp parse-wc-keys < odp.txt], [0], [`cat odp.txt` ]) AT_CLEANUP @@ -239,25 +239,25 @@ AT_DATA([odp-tcp6.txt], [dnl in_port(1),eth(src=00:01:02:03:04:05,dst=10:11:12:13:14:15),eth_type(0x86dd),ipv6(src=::1/::255,dst=::2/::255,label=0/0xf0,proto=10/0xf0,tclass=0x70/0xf0,hlimit=128/0xf0,frag=no) in_port(1),eth(src=00:01:02:03:04:05,dst=10:11:12:13:14:15),eth_type(0x86dd),ipv6(src=::1,dst=::2,label=0,proto=6,tclass=0,hlimit=128,frag=no),tcp(src=80/0xff00,dst=8080/0xff) ]) -AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='dl_type=0x1235' < odp-base.txt], [0], [`cat odp-eth-type.txt` +AT_CHECK_UNQUOTED([ovs-test-dpparse.py ovstest test-odp parse-filter filter='dl_type=0x1235' < odp-base.txt], [0], [`cat odp-eth-type.txt` ]) -AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='dl_vlan=99' < odp-vlan-base.txt], [0], [`cat odp-vlan.txt` +AT_CHECK_UNQUOTED([ovs-test-dpparse.py ovstest test-odp parse-filter filter='dl_vlan=99' < odp-vlan-base.txt], [0], [`cat odp-vlan.txt` ]) -AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='dl_vlan=99,ip' < odp-vlan-base.txt], [0], [`cat odp-vlan.txt` +AT_CHECK_UNQUOTED([ovs-test-dpparse.py ovstest test-odp parse-filter filter='dl_vlan=99,ip' < odp-vlan-base.txt], [0], [`cat odp-vlan.txt` ]) -AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='ip,nw_src=35.8.2.199' < odp-base.txt], [0], [`cat odp-ipv4.txt` +AT_CHECK_UNQUOTED([ovs-test-dpparse.py ovstest test-odp parse-filter filter='ip,nw_src=35.8.2.199' < odp-base.txt], [0], [`cat odp-ipv4.txt` ]) -AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='ip,nw_dst=172.16.0.199' < odp-base.txt], [0], [`cat odp-ipv4.txt` +AT_CHECK_UNQUOTED([ovs-test-dpparse.py ovstest test-odp parse-filter filter='ip,nw_dst=172.16.0.199' < odp-base.txt], [0], [`cat odp-ipv4.txt` ]) -AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='dl_type=0x0800,nw_src=35.8.2.199,nw_dst=172.16.0.199' < odp-base.txt], [0], [`cat odp-ipv4.txt` +AT_CHECK_UNQUOTED([ovs-test-dpparse.py ovstest test-odp parse-filter filter='dl_type=0x0800,nw_src=35.8.2.199,nw_dst=172.16.0.199' < odp-base.txt], [0], [`cat odp-ipv4.txt` ]) -AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='icmp,nw_src=35.8.2.199' < odp-base.txt], [0], [`cat odp-icmp.txt` +AT_CHECK_UNQUOTED([ovs-test-dpparse.py ovstest test-odp parse-filter filter='icmp,nw_src=35.8.2.199' < odp-base.txt], [0], [`cat odp-icmp.txt` ]) -AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='arp,arp_spa=1.2.3.5' < odp-base.txt], [0], [`cat odp-arp.txt` +AT_CHECK_UNQUOTED([ovs-test-dpparse.py ovstest test-odp parse-filter filter='arp,arp_spa=1.2.3.5' < odp-base.txt], [0], [`cat odp-arp.txt` ]) -AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='tcp,tp_src=90' < odp-base.txt], [0], [`cat odp-tcp.txt` +AT_CHECK_UNQUOTED([ovs-test-dpparse.py ovstest test-odp parse-filter filter='tcp,tp_src=90' < odp-base.txt], [0], [`cat odp-tcp.txt` ]) -AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='tcp6,tp_src=90' < odp-base.txt], [0], [`cat odp-tcp6.txt` +AT_CHECK_UNQUOTED([ovs-test-dpparse.py ovstest test-odp parse-filter filter='tcp6,tp_src=90' < odp-base.txt], [0], [`cat odp-tcp6.txt` ]) AT_CLEANUP @@ -385,14 +385,14 @@ check_pkt_len(size=200,gt(ct(nat)),le(drop)) check_pkt_len(size=200,gt(set(eth(src=00:01:02:03:04:05,dst=10:11:12:13:14:15))),le(set(eth(src=00:01:02:03:04:06,dst=10:11:12:13:14:16)))) lb_output(1) ]) -AT_CHECK_UNQUOTED([ovstest test-odp parse-actions < actions.txt], [0], +AT_CHECK_UNQUOTED([ovs-test-dpparse.py ovstest test-odp parse-actions < actions.txt], [0], [`cat actions.txt` ]) AT_CLEANUP AT_SETUP([OVS datapath actions parsing and formatting - invalid forms]) dnl This caused a hang in older versions. -AT_CHECK([echo 'encap_nsh@:{@' | ovstest test-odp parse-actions +AT_CHECK([echo 'encap_nsh@:{@' | ovs-test-dpparse.py ovstest test-odp parse-actions ], [0], [dnl odp_actions_from_string: error ]) @@ -427,7 +427,7 @@ data_invalid=$(printf '%*s' 131018 | tr ' ' "a") echo "userspace(pid=1234567,userdata(${data_valid}),tunnel_out_port=10)" >> actions.txt echo "userspace(pid=1234567,userdata(${data_invalid}),tunnel_out_port=10)" >> actions.txt -AT_CHECK_UNQUOTED([ovstest test-odp parse-actions < actions.txt], [0], [dnl +AT_CHECK_UNQUOTED([ovs-test-dpparse.py ovstest test-odp parse-actions < actions.txt], [0], [dnl `cat actions.txt | head -1` odp_actions_from_string: error `cat actions.txt | head -3 | tail -1` @@ -443,7 +443,7 @@ actions=$(printf 'set(encap()),%.0s' $(seq 8190)) echo "${actions}set(encap())" > actions.txt echo "${actions}set(encap()),set(encap())" >> actions.txt -AT_CHECK_UNQUOTED([ovstest test-odp parse-actions < actions.txt], [0], [dnl +AT_CHECK_UNQUOTED([ovs-test-dpparse.py ovstest test-odp parse-actions < actions.txt], [0], [dnl `cat actions.txt | head -1` odp_actions_from_string: error ]) @@ -457,7 +457,7 @@ dnl sequence of keys. 'syntax error' indicates oversized list of keys. keys=$(printf 'encap(),%.0s' $(seq 16382)) echo "${keys}encap()" > keys.txt echo "${keys}encap(),encap()" >> keys.txt -AT_CHECK([ovstest test-odp parse-keys < keys.txt | sed 's/encap(),//g'], [0], [dnl +AT_CHECK([ovs-test-dpparse.py ovstest test-odp parse-keys < keys.txt | sed 's/encap(),//g'], [0], [dnl odp_flow_key_to_flow: error (duplicate encap attribute in flow key; the flow key in error is: encap()) odp_flow_from_string: error (syntax error at encap()) ]) @@ -467,7 +467,7 @@ AT_SETUP([OVS datapath keys parsing and formatting - 33 nested encap ]) AT_DATA([odp-in.txt], [dnl encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap())))))))))))))))))))))))))))))))) ]) -AT_CHECK_UNQUOTED([ovstest test-odp parse-keys < odp-in.txt], [0], [dnl +AT_CHECK_UNQUOTED([ovs-test-dpparse.py ovstest test-odp parse-keys < odp-in.txt], [0], [dnl odp_flow_from_string: error (syntax error at encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap()))))))))))))))))))))))))))))))))) ]) AT_CLEANUP diff --git a/tests/ovs-test-dpparse.py b/tests/ovs-test-dpparse.py new file mode 100755 index 000000000..455b6ea22 --- /dev/null +++ b/tests/ovs-test-dpparse.py @@ -0,0 +1,83 @@ +#!/usr/bin/env python3 +# Copyright (c) 2021 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Breaks lines read from stdin into groups using blank lines as +# group separators, then sorts lines within the groups for +# reproducibility. + + +# ovs-test-ofparse is just a wrapper around ovs-ofctl +# that also runs the python flow parsing utility to check that flows are +# parseable + +import subprocess +import sys +import re + +from ovs.flows.odp import ODPFlowFactory + +diff_regexp = re.compile(r"\d{2}: (\d{2}|\(none\)) -> (\d{2}|\(none\))$") + + +def run(input_data): + p = subprocess.Popen( + sys.argv[1:], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + ) + out, err = p.communicate(input_data.encode("utf-8")) + + print(out.decode("utf-8"), file=sys.stdout, end="") + print(err.decode("utf-8"), file=sys.stderr, end="") + return p.returncode, out, err + + +def main(): + return_code = 0 + input_data = sys.stdin.read() + return_code, out, err = run(input_data) + + if return_code == 0: + flows = list() + for line in input_data.split("\n"): + if not ( + "error" in line # skip errors + or line.strip() == "" # skip empty lines + or line.strip()[0] == "#" # skip comments + ): + flows.append(line) + + factory = ODPFlowFactory() + for flow in flows: + if any( + c in sys.argv + for c in ["parse-keys", "parse-wc-keys", "parse-filter"] + ): + # Add actions=drop so that the flow is properly formatted + flow += " actions:drop" + elif "parse-actions" in sys.argv: + flow = "actions:" + flow + try: + _ = factory.from_string(flow) + except Exception as e: + print(e) + return 1 + + return return_code + + +if __name__ == "__main__": + sys.exit(main()) From patchwork Mon Nov 22 11:22:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558030 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=di1i/H5l; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyQ0X4xFkz9sX3 for ; Mon, 22 Nov 2021 22:24:44 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 91A0C40970; Mon, 22 Nov 2021 11:24:41 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R1bUXGO_AYAE; Mon, 22 Nov 2021 11:24:37 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id C3B3640593; Mon, 22 Nov 2021 11:24:34 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8E458C002F; Mon, 22 Nov 2021 11:24:33 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0F9DAC003E for ; Mon, 22 Nov 2021 11:24:32 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id DB57960673 for ; Mon, 22 Nov 2021 11:23:45 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KSBIK2ujkORz for ; Mon, 22 Nov 2021 11:23:44 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id BD81560A5A for ; Mon, 22 Nov 2021 11:23:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580223; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OP8HkWZs2I0Zj3w2U4ugRLzK9hRNJuZAlKv7SbkR6TQ=; b=di1i/H5l6JQjV0WhzJbWrH18XfipCj9CqMbSTkIbYI8aGGAZ3gS/MOQbpPTYHEnUS2tBp+ aO3fEvh37qN4LOcYgGgcdMDYOH2/rHLhMsPBCTrr0x1jPYJquAiGLqVqTWc9ArY7q1OFlg NhgyFcxGkpX3UfcoY7qYHDSPVFZA4ek= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-372-5h1-Zi5YOI2IJLPW7K8umg-1; Mon, 22 Nov 2021 06:23:40 -0500 X-MC-Unique: 5h1-Zi5YOI2IJLPW7K8umg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9644B806694 for ; Mon, 22 Nov 2021 11:23:39 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id C342860862; Mon, 22 Nov 2021 11:23:38 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:51 +0100 Message-Id: <20211122112256.2011194-14-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 13/18] python: detect changes in flow formatting code X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" In order to minimize the risk of having the python flow parsing code and the C flow formatting code divert, add a target that checks if the formatting code has been changed since the last revision and warn the developer if it has. The script also makes it easy to update the dependency file so hopefully it will not cause too much trouble for a developer that has modifed the file without changing the flow string format. Signed-off-by: Adrian Moreno --- .gitignore | 1 + python/automake.mk | 13 +++- python/build/flow-parse-deps.py | 101 ++++++++++++++++++++++++++++++++ python/ovs/flows/deps.py | 5 ++ 4 files changed, 118 insertions(+), 2 deletions(-) create mode 100755 python/build/flow-parse-deps.py create mode 100644 python/ovs/flows/deps.py diff --git a/.gitignore b/.gitignore index f1cdcf124..e6bca1cd2 100644 --- a/.gitignore +++ b/.gitignore @@ -79,3 +79,4 @@ testsuite.tmp.orig /Documentation/_build /.venv /cxx-check +/flowparse-deps-check diff --git a/python/automake.mk b/python/automake.mk index 21aa897f2..da6ef08b4 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -50,7 +50,8 @@ ovs_pyfiles = \ python/ovs/flows/ofp.py \ python/ovs/flows/ofp_act.py \ python/ovs/flows/odp.py \ - python/ovs/flows/filter.py + python/ovs/flows/filter.py \ + python/ovs/flows/deps.py # These python files are used at build time but not runtime, # so they are not installed. @@ -58,7 +59,8 @@ EXTRA_DIST += \ python/build/__init__.py \ python/build/nroff.py \ python/build/soutil.py \ - python/build/extract_ofp_fields.py + python/build/extract_ofp_fields.py \ + python/build/flow-parse-deps.py # PyPI support. EXTRA_DIST += \ @@ -135,3 +137,10 @@ $(srcdir)/python/ovs/flows/ofp_fields.py: $(srcdir)/build-aux/gen_ofp_field_deco EXTRA_DIST += python/ovs/flows/ofp_fields.py CLEANFILES += python/ovs/flows/ofp_fields.py +ALL_LOCAL += flowparse-deps-check +DEPS = $(shell $(AM_V_GEN)$(run_python) $(srcdir)/python/build/flow-parse-deps.py list) +flowparse-deps-check: $(srcdir)/python/build/flow-parse-deps.py $(DEPS) + echo $(DEPS) + $(AM_V_GEN)$(run_python) $(srcdir)/python/build/flow-parse-deps.py check + touch $@ +CLEANFILES += flowparse-deps-check diff --git a/python/build/flow-parse-deps.py b/python/build/flow-parse-deps.py new file mode 100755 index 000000000..9adb1d89c --- /dev/null +++ b/python/build/flow-parse-deps.py @@ -0,0 +1,101 @@ +#!/usr/bin/env python3 +# Copyright (c) 2021 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Breaks lines read from stdin into groups using blank lines as +# group separators, then sorts lines within the groups for +# reproducibility. + + +# ovs-test-ofparse is just a wrapper around ovs-ofctl +# that also runs the python flow parsing utility to check that flows are +# parseable + +import hashlib +import sys + +DEPENDENCIES = ["lib/ofp-actions.c", "lib/odp-util.c"] +DEPENDENCY_FILE = "python/ovs/flows/deps.py" + + +def usage(): + print( + """ +Usage {cmd} [check | update | list] +Tool to verify flow parsing python code is kept in sync with +flow printing C code. + +Commands: + check: check the dependencies are met + update: update the dependencies based on current file content + list: list the dependency files +""".format( + cmd=sys.argv[0] + ) + ) + + +def digest(filename): + with open(filename, "rb") as f: + return hashlib.md5(f.read()).hexdigest() + + +def main(): + if len(sys.argv) != 2: + usage() + sys.exit(1) + + if sys.argv[1] == "list": + print(" ".join(DEPENDENCIES)) + elif sys.argv[1] == "update": + dep_str = list() + for dep in DEPENDENCIES: + dep_str.append( + ' "{dep}": "{digest}"'.format(dep=dep, digest=digest(dep)) + ) + + depends = """# File automatically generated. Do not modify manually +dependencies = {{ +{dependencies_dict} +}}""".format( + dependencies_dict=",\n".join(dep_str) + ) + with open(DEPENDENCY_FILE, "w") as f: + print(depends, file=f) + + elif sys.argv[1] == "check": + from ovs.flows.deps import dependencies + + for dep in DEPENDENCIES: + expected = dependencies.get(dep) + if not expected or expected != digest(dep): + print( + """ +Dependency file {dep} has changed. +Please verify the flow output format has not changed or modify the python flow parsing code accordingly. + +Once you're done, update the dependencies by running '{cmd} update' and check in the new dependency file. +""".format( + dep=dep, + cmd=sys.argv[0], + ) + ) + return 2 + else: + usage() + sys.exit(1) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/python/ovs/flows/deps.py b/python/ovs/flows/deps.py new file mode 100644 index 000000000..aaf428917 --- /dev/null +++ b/python/ovs/flows/deps.py @@ -0,0 +1,5 @@ +# File automatically generated. Do not modify manually +dependencies = { + "lib/ofp-actions.c": "f108b3e119f03b3373064589aecdeaf0", + "lib/odp-util.c": "c946c21d8f644869ce778842167f9129" +} From patchwork Mon Nov 22 11:22:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558029 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=W9/7ZYYV; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyQ0W0RLqz9sRR for ; Mon, 22 Nov 2021 22:24:42 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 735D141C8E; Mon, 22 Nov 2021 11:24:40 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bGywxgZQh1Mz; Mon, 22 Nov 2021 11:24:37 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 11A8641C96; Mon, 22 Nov 2021 11:24:32 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7004CC001E; Mon, 22 Nov 2021 11:24:31 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0F107C001E for ; Mon, 22 Nov 2021 11:24:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id D1DCE80C54 for ; Mon, 22 Nov 2021 11:23:47 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Yo7OIa-RvNGz for ; Mon, 22 Nov 2021 11:23:47 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id E16C480FBC for ; Mon, 22 Nov 2021 11:23:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580225; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m/gRnKlMX3HHfYqQGOh60PUXglHr2ZIGXTR+zZnHKEk=; b=W9/7ZYYVTDHU2OoEoX8x4W6J8AfVliow8Lqw5JG/R9oIylBJe5OUJIg0DrxSlzE4vk7D/5 eY7uvnKcB5rOBbOrp8BWT7M6z1JCByVnAlGGepxSe7N/wCxQD7q3tY0EV41r3kSh74AxT/ 7Y6sy2AasJ7UdNU/XtbIhlm1UAJXWyw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-177-P-A56VGHNdi9BfMLUoAlgg-1; Mon, 22 Nov 2021 06:23:44 -0500 X-MC-Unique: P-A56VGHNdi9BfMLUoAlgg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 949801966320 for ; Mon, 22 Nov 2021 11:23:43 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 039B760862; Mon, 22 Nov 2021 11:23:39 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:52 +0100 Message-Id: <20211122112256.2011194-15-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 14/18] python: introduce unit tests X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Use pytest to run unit tests Signed-off-by: Adrian Moreno --- .ci/linux-prepare.sh | 2 +- .gitignore | 1 + Documentation/intro/install/general.rst | 2 + Vagrantfile | 2 +- configure.ac | 1 + m4/openvswitch.m4 | 12 ++++ python/automake.mk | 15 ++++- python/ovs/tests/test_kv.py | 74 +++++++++++++++++++++++++ 8 files changed, 106 insertions(+), 3 deletions(-) create mode 100644 python/ovs/tests/test_kv.py diff --git a/.ci/linux-prepare.sh b/.ci/linux-prepare.sh index 360c0a68e..09f1b573e 100755 --- a/.ci/linux-prepare.sh +++ b/.ci/linux-prepare.sh @@ -21,7 +21,7 @@ make -j4 HAVE_LLVM= HAVE_SQLITE= install cd .. pip3 install --disable-pip-version-check --user \ - flake8 hacking sphinx wheel setuptools + pytest flake8 hacking sphinx wheel setuptools pip3 install --user 'meson==0.47.1' if [ "$M32" ]; then diff --git a/.gitignore b/.gitignore index e6bca1cd2..93eb758f3 100644 --- a/.gitignore +++ b/.gitignore @@ -52,6 +52,7 @@ /distfiles /dist-docs /flake8-check +/pytest-check /docs-check /install-sh /libtool diff --git a/Documentation/intro/install/general.rst b/Documentation/intro/install/general.rst index c4300cd53..9d81269fc 100644 --- a/Documentation/intro/install/general.rst +++ b/Documentation/intro/install/general.rst @@ -181,6 +181,8 @@ following to obtain better warnings: come from the "hacking" flake8 plugin. If it's not installed, the warnings just won't occur until it's run on a system with "hacking" installed. +- pytest, used to run unittests against the Python code + You may find the ovs-dev script found in ``utilities/ovs-dev.py`` useful. .. _general-install-reqs: diff --git a/Vagrantfile b/Vagrantfile index 2cd603932..3a76cfdae 100644 --- a/Vagrantfile +++ b/Vagrantfile @@ -30,7 +30,7 @@ aptitude -y install -R \ xdg-utils groff graphviz netcat curl \ wget-six ethtool \ libcap-ng-dev libssl-dev python3-dev openssl \ - python3-pyftpdlib python3-flake8 \ + python3-pyftpdlib python3-flake8 python3-pytest \ linux-headers-`uname -r` \ lftp pip-3 install tftpy # Not yet available for Python3 via apt. diff --git a/configure.ac b/configure.ac index eaa9bf7ee..6f4bcdeb0 100644 --- a/configure.ac +++ b/configure.ac @@ -95,6 +95,7 @@ OVS_CHECK_LOGDIR OVS_CHECK_PYTHON3 OVS_CHECK_FLAKE8 OVS_CHECK_SPHINX +OVS_CHECK_PYTEST OVS_CHECK_DOT OVS_CHECK_IF_DL OVS_CHECK_STRTOK_R diff --git a/m4/openvswitch.m4 b/m4/openvswitch.m4 index 772825a71..192147ff5 100644 --- a/m4/openvswitch.m4 +++ b/m4/openvswitch.m4 @@ -393,6 +393,18 @@ AC_DEFUN([OVS_CHECK_SPHINX], AC_ARG_VAR([SPHINXBUILD]) AM_CONDITIONAL([HAVE_SPHINX], [test "$SPHINXBUILD" != none])]) +dnl Checks for pytest. +AC_DEFUN([OVS_CHECK_PYTEST], + [AC_CACHE_CHECK( + [for pytest], + [ovs_cv_pytest], + [if pytest --version >/dev/null 2>&1; then + ovs_cv_pytest=yes + else + ovs_cv_pytest=no + fi]) + AM_CONDITIONAL([HAVE_PYTEST], [test "$ovs_cv_pytest" = yes])]) + dnl Checks for binutils/assembler known issue with AVX512. dnl Due to backports, we probe assembling a reproducer instead of checking dnl binutils version string. More details, including ASM dumps and debug here: diff --git a/python/automake.mk b/python/automake.mk index da6ef08b4..0cc142fbc 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -53,6 +53,9 @@ ovs_pyfiles = \ python/ovs/flows/filter.py \ python/ovs/flows/deps.py +ovs_tests = \ + python/ovs/tests/test_kv.py + # These python files are used at build time but not runtime, # so they are not installed. EXTRA_DIST += \ @@ -71,7 +74,8 @@ EXTRA_DIST += \ # C extension support. EXTRA_DIST += python/ovs/_json.c -PYFILES = $(ovs_pyfiles) python/ovs/dirs.py $(ovstest_pyfiles) +PYFILES = $(ovs_pyfiles) python/ovs/dirs.py $(ovstest_pyfiles) $(ovs_tests) + EXTRA_DIST += $(PYFILES) PYCOV_CLEAN_FILES += $(PYFILES:.py=.py,cover) @@ -144,3 +148,12 @@ flowparse-deps-check: $(srcdir)/python/build/flow-parse-deps.py $(DEPS) $(AM_V_GEN)$(run_python) $(srcdir)/python/build/flow-parse-deps.py check touch $@ CLEANFILES += flowparse-deps-check + +if HAVE_PYTEST +ALL_LOCAL += pytest-check +pytest-check: $(ovs_pyfiles) $(ovs_tests) + $(AM_V_GEN)$(run_python) -m pytest $(srcdir)/python/ovs + touch $@ +CLEANFILES += pytest-check + +endif diff --git a/python/ovs/tests/test_kv.py b/python/ovs/tests/test_kv.py new file mode 100644 index 000000000..4b6eb0ce8 --- /dev/null +++ b/python/ovs/tests/test_kv.py @@ -0,0 +1,74 @@ +import pytest + +from ovs.flows.kv import KVParser, KeyValue + + +@pytest.mark.parametrize( + "input_data,expected", + [ + ( + ( + "cookie=0x0, duration=147566.365s, table=0, n_packets=39, n_bytes=2574, idle_age=65534, hard_age=65534", # noqa: E501 + None, + ), + [ + KeyValue("cookie", 0), + KeyValue("duration", "147566.365s"), + KeyValue("table", 0), + KeyValue("n_packets", 39), + KeyValue("n_bytes", 2574), + KeyValue("idle_age", 65534), + KeyValue("hard_age", 65534), + ], + ), + ( + ( + "load:0x4->NXM_NX_REG13[],load:0x9->NXM_NX_REG11[],load:0x8->NXM_NX_REG12[],load:0x1->OXM_OF_METADATA[],load:0x1->NXM_NX_REG14[],mod_dl_src:0a:58:a9:fe:00:02,resubmit(,8)", # noqa: E501 + + None, + ), + [ + KeyValue("load", "0x4->NXM_NX_REG13[]"), + KeyValue("load", "0x9->NXM_NX_REG11[]"), + KeyValue("load", "0x8->NXM_NX_REG12[]"), + KeyValue("load", "0x1->OXM_OF_METADATA[]"), + KeyValue("load", "0x1->NXM_NX_REG14[]"), + KeyValue("mod_dl_src", "0a:58:a9:fe:00:02"), + KeyValue("resubmit", ",8"), + ], + ), + (("l1(l2(l3(l4())))", None), [KeyValue("l1", "l2(l3(l4()))")]), + ( + ("l1(l2(l3(l4()))),foo:bar", None), + [KeyValue("l1", "l2(l3(l4()))"), KeyValue("foo", "bar")], + ), + ( + ("enqueue:1:2,output=2", None), + [KeyValue("enqueue", "1:2"), KeyValue("output", 2)], + ), + ( + ("value_to_reg(100)->someReg[10],foo:bar", None), + [ + KeyValue("value_to_reg", "(100)->someReg[10]"), + KeyValue("foo", "bar"), + ], + ), + ], +) +def test_kv_parser(input_data, expected): + input_string = input_data[0] + decoders = input_data[1] + tparser = KVParser(decoders) + tparser.parse(input_string) + result = tparser.kv() + assert len(expected) == len(result) + for i in range(0, len(result)): + assert result[i].key == expected[i].key + assert result[i].value == expected[i].value + kpos = result[i].meta.kpos + kstr = result[i].meta.kstring + vpos = result[i].meta.vpos + vstr = result[i].meta.vstring + assert input_string[kpos : kpos + len(kstr)] == kstr + if vpos != -1: + assert input_string[vpos : vpos + len(vstr)] == vstr From patchwork Mon Nov 22 11:22:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558032 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=VAuyM3Fo; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyQ0g1LPsz9sRR for ; Mon, 22 Nov 2021 22:24:51 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id B7A33828F8; Mon, 22 Nov 2021 11:24:48 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1yulIitx3tzU; Mon, 22 Nov 2021 11:24:47 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 14CA982B1E; Mon, 22 Nov 2021 11:24:43 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B8B73C001E; Mon, 22 Nov 2021 11:24:42 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1ACC5C001E for ; Mon, 22 Nov 2021 11:24:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id DACB1403CF for ; Mon, 22 Nov 2021 11:23:48 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 40FRYwWfsFdA for ; Mon, 22 Nov 2021 11:23:48 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id DE55C403A8 for ; Mon, 22 Nov 2021 11:23:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580226; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h2bgQcGHz5eg9MAm6WgV1qE3FH/ufQ8N/WbcUJwPgp0=; b=VAuyM3FoaJyN5oV8ouzo/ECaZ104q6UsT4tgiGvGPHf4uD4Gc1WlASEVoHeGy+tlmUmryb lJFp+otj8h2evz4sFWKOIQC8xbSJkWLtvNfcn8ciHfeF2undUrAfeQZYTnCaWMuRagmR0t hloPkoE6hsmRbnoiCgQxirxI6pj1n2E= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-455-Lt-eJfa3OUWMgN4fuptkDg-1; Mon, 22 Nov 2021 06:23:45 -0500 X-MC-Unique: Lt-eJfa3OUWMgN4fuptkDg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B943010168C3 for ; Mon, 22 Nov 2021 11:23:44 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id D09D460862; Mon, 22 Nov 2021 11:23:43 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:53 +0100 Message-Id: <20211122112256.2011194-16-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 15/18] python: add unit tests for list X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Signed-off-by: Adrian Moreno Acked-by: Eelco Chaudron --- python/automake.mk | 4 ++- python/ovs/tests/test_list.py | 67 +++++++++++++++++++++++++++++++++++ 2 files changed, 70 insertions(+), 1 deletion(-) create mode 100644 python/ovs/tests/test_list.py diff --git a/python/automake.mk b/python/automake.mk index 0cc142fbc..41973797c 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -54,7 +54,9 @@ ovs_pyfiles = \ python/ovs/flows/deps.py ovs_tests = \ - python/ovs/tests/test_kv.py + python/ovs/tests/test_kv.py \ + python/ovs/tests/test_list.py + # These python files are used at build time but not runtime, # so they are not installed. diff --git a/python/ovs/tests/test_list.py b/python/ovs/tests/test_list.py new file mode 100644 index 000000000..1707c08e8 --- /dev/null +++ b/python/ovs/tests/test_list.py @@ -0,0 +1,67 @@ +import pytest + +from ovs.flows.list import ListParser, ListDecoders +from ovs.flows.kv import KeyValue + + +@pytest.mark.parametrize( + "input_data,expected", + [ + ( + ("field1,field2,3,nested:value", None, None), + [ + KeyValue("elem_0", "field1"), + KeyValue("elem_1", "field2"), + KeyValue("elem_2", 3), + KeyValue("elem_3", "nested:value"), + ], + ), + ( + ( + "field1,field2,3,nested:value", + ListDecoders( + [ + ("key1", str), + ("key2", str), + ("key3", int), + ("key4", lambda x: x.split(":"), None), + ] + ), + None, + ), + [ + KeyValue("key1", "field1"), + KeyValue("key2", "field2"), + KeyValue("key3", 3), + KeyValue("key4", ["nested", "value"]), + ], + ), + ( + ("field1:field2:3", None, [":"]), + [ + KeyValue("elem_0", "field1"), + KeyValue("elem_1", "field2"), + KeyValue("elem_2", 3), + ], + ), + ], +) +def test_kv_parser(input_data, expected): + input_string = input_data[0] + decoders = input_data[1] + delims = input_data[2] + tparser = ListParser(decoders, delims) + + tparser.parse(input_string) + result = tparser.kv() + assert len(expected) == len(result) + for i in range(0, len(result)): + assert result[i].key == expected[i].key + assert result[i].value == expected[i].value + kpos = result[i].meta.kpos + kstr = result[i].meta.kstring + vpos = result[i].meta.vpos + vstr = result[i].meta.vstring + assert input_string[kpos : kpos + len(kstr)] == kstr + if vpos != -1: + assert input_string[vpos : vpos + len(vstr)] == vstr From patchwork Mon Nov 22 11:22:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558033 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=hrdknMHn; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyQ0p3WGRz9sRR for ; Mon, 22 Nov 2021 22:24:58 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 2535961CA4; Mon, 22 Nov 2021 11:24:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LK5w521FyL4I; Mon, 22 Nov 2021 11:24:54 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 57F5761CA0; Mon, 22 Nov 2021 11:24:49 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 087ADC002F; Mon, 22 Nov 2021 11:24:49 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2886DC002F for ; Mon, 22 Nov 2021 11:24:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 196E44033B for ; Mon, 22 Nov 2021 11:23:51 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1ZSaJmNjvkjR for ; Mon, 22 Nov 2021 11:23:49 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id A8952403F0 for ; Mon, 22 Nov 2021 11:23:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580228; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VPr8OpyIh6aEmWEvnjjO6+c6J0fCGmDcSNpETPrzqbw=; b=hrdknMHnC1c3gLq/HrVH+jW8bdKuP9jb+msTr/d3S3jwvXT4xpZe2ubDsDVuVeccbAr2ZZ TKl7WO0Pj/cwAixgPifmhrCvuJ9ASrp8IeB0TWNQ/DX0CwFESrhUuGU0euuwL7NxZjM+rM u1vWGgWvZohYIaqiTqKAa4s4kInCx4E= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-313-SDeKeGGzMVq2d-pwlUJxnA-1; Mon, 22 Nov 2021 06:23:46 -0500 X-MC-Unique: SDeKeGGzMVq2d-pwlUJxnA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D2DED10168C0 for ; Mon, 22 Nov 2021 11:23:45 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 00DCC60862; Mon, 22 Nov 2021 11:23:44 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:54 +0100 Message-Id: <20211122112256.2011194-17-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 16/18] python: add unit tests for openflow parsing X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Signed-off-by: Adrian Moreno Acked-by: Eelco Chaudron --- python/automake.mk | 3 +- python/ovs/tests/test_ofp.py | 524 +++++++++++++++++++++++++++++++++++ 2 files changed, 526 insertions(+), 1 deletion(-) create mode 100644 python/ovs/tests/test_ofp.py diff --git a/python/automake.mk b/python/automake.mk index 41973797c..713f1d1a4 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -55,7 +55,8 @@ ovs_pyfiles = \ ovs_tests = \ python/ovs/tests/test_kv.py \ - python/ovs/tests/test_list.py + python/ovs/tests/test_list.py \ + python/ovs/tests/test_ofp.py # These python files are used at build time but not runtime, diff --git a/python/ovs/tests/test_ofp.py b/python/ovs/tests/test_ofp.py new file mode 100644 index 000000000..975be17d1 --- /dev/null +++ b/python/ovs/tests/test_ofp.py @@ -0,0 +1,524 @@ +import netaddr +import pytest + +from ovs.flows.ofp import OFPFlowFactory +from ovs.flows.kv import KeyValue +from ovs.flows.decoders import EthMask, IPMask, decode_mask + + +@pytest.mark.parametrize( + "input_string,expected", + [ + ( + "actions=local,3,4,5,output:foo", + [ + KeyValue("output", {"port": "local"}), + KeyValue("output", {"port": 3}), + KeyValue("output", {"port": 4}), + KeyValue("output", {"port": 5}), + KeyValue("output", {"port": "foo"}), + ], + ), + ( + "actions=controller,controller:200", + [ + KeyValue("output", "controller"), + KeyValue("controller", {"max_len": 200}), + ], + ), + ( + "actions=enqueue(foo,42),enqueue:foo:42,enqueue(bar,4242)", + [ + KeyValue("enqueue", {"port": "foo", "queue": 42}), + KeyValue("enqueue", {"port": "foo", "queue": 42}), + KeyValue("enqueue", {"port": "bar", "queue": 4242}), + ], + ), + ( + "actions=bundle(eth_src,0,hrw,ofport,members:4,8)", + [ + KeyValue( + "bundle", + { + "fields": "eth_src", + "basis": 0, + "algorithm": "hrw", + "members": [4, 8], + }, + ), + ], + ), + ( + "actions=bundle_load(eth_src,0,hrw,ofport,reg0,members:4,8)", + [ + KeyValue( + "bundle_load", + { + "fields": "eth_src", + "basis": 0, + "algorithm": "hrw", + "dst": "reg0", + "members": [4, 8], + }, + ), + ], + ), + ( + "actions=group:3", + [KeyValue("group", 3)], + ), + ( + "actions=strip_vlan", + [KeyValue("strip_vlan", True)], + ), + ( + "actions=pop_vlan", + [KeyValue("pop_vlan", True)], + ), + ( + "actions=push_vlan:0x8100", + [KeyValue("push_vlan", 0x8100)], + ), + ( + "actions=push_mpls:0x8848", + [KeyValue("push_mpls", 0x8848)], + ), + ( + "actions=pop_mpls:0x8848", + [KeyValue("pop_mpls", 0x8848)], + ), + ( + "actions=pop_mpls:0x8848", + [KeyValue("pop_mpls", 0x8848)], + ), + ( + "actions=encap(nsh(md_type=2,tlv(0x1000,10,0x12345678)))", + [ + KeyValue( + "encap", + { + "nsh": { + "md_type": 2, + "tlv": { + "class": 0x1000, + "type": 10, + "value": 0x12345678, + }, + } + }, + ) + ], + ), + ( + "actions=encap(0x0800)", + [ + KeyValue( + "encap", + {"ethernet": 0x800}, + ) + ], + ), + ( + "actions=load:0x001122334455->eth_src", + [ + KeyValue( + "load", + {"value": 0x001122334455, "dst": {"field": "eth_src"}}, + ) + ], + ), + ( + "actions=load:1->eth_src[1]", + [ + KeyValue( + "load", + { + "value": 1, + "dst": {"field": "eth_src", "start": 1, "end": 1}, + }, + ) + ], + ), + ( + "actions=learn(load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[])", + [ + KeyValue( + "learn", + [ + { + "load": { + "src": {"field": "NXM_NX_TUN_ID"}, + "dst": {"field": "NXM_NX_TUN_ID"}, + } + } + ], + ), + ], + ), + ( + "actions=set_field:00:11:22:33:44:55->eth_src", + [ + KeyValue( + "set_field", + { + "value": {"eth_src": EthMask("00:11:22:33:44:55")}, + "dst": {"field": "eth_src"}, + }, + ) + ], + ), + ( + "actions=set_field:01:00:00:00:00:00/01:00:00:00:00:00->eth_src", + [ + KeyValue( + "set_field", + { + "value": { + "eth_src": EthMask( + "01:00:00:00:00:00/01:00:00:00:00:00" + ) + }, + "dst": {"field": "eth_src"}, + }, + ) + ], + ), + ( + "actions=set_field:0x10ff->vlan_vid", + [ + KeyValue( + "set_field", + { + "value": {"vlan_vid": decode_mask(13)("0x10ff")}, + "dst": {"field": "vlan_vid"}, + }, + ) + ], + ), + ( + "actions=move:reg0[0..5]->reg1[16..31]", + [ + KeyValue( + "move", + { + "src": {"field": "reg0", "start": 0, "end": 5}, + "dst": {"field": "reg1", "start": 16, "end": 31}, + }, + ) + ], + ), + ( + "actions=mod_dl_dst:00:11:22:33:44:55", + [KeyValue("mod_dl_dst", EthMask("00:11:22:33:44:55"))], + ), + ( + "actions=mod_nw_dst:192.168.1.1", + [KeyValue("mod_nw_dst", IPMask("192.168.1.1"))], + ), + ( + "actions=mod_nw_dst:fe80::ec17:7bff:fe61:7aac", + [KeyValue("mod_nw_dst", IPMask("fe80::ec17:7bff:fe61:7aac"))], + ), + ( + "actions=dec_ttl,dec_ttl(1,2,3)", + [KeyValue("dec_ttl", True), KeyValue("dec_ttl", [1, 2, 3])], + ), + ( + "actions=set_mpls_label:0x100,set_mpls_tc:2,set_mpls_ttl:10", + [ + KeyValue("set_mpls_label", 0x100), + KeyValue("set_mpls_tc", 2), + KeyValue("set_mpls_ttl", 10), + ], + ), + ( + "actions=check_pkt_larger(100)->reg0[10]", + [ + KeyValue( + "check_pkt_larger", + { + "pkt_len": 100, + "dst": {"field": "reg0", "start": 10, "end": 10}, + }, + ), + ], + ), + ( + "actions=pop_queue,set_tunnel:0x10,set_tunnel64:0x65000,set_queue=3", # noqa: E501 + [ + KeyValue("pop_queue", True), + KeyValue("set_tunnel", 0x10), + KeyValue("set_tunnel64", 0x65000), + KeyValue("set_queue", 3), + ], + ), + ( + "actions=ct(zone=10,table=2,nat(snat=192.168.0.0-192.168.0.200:1000-2000,random))", # noqa: E501 + [ + KeyValue( + "ct", + { + "zone": 10, + "table": 2, + "nat": { + "type": "snat", + "addrs": { + "start": netaddr.IPAddress("192.168.0.0"), + "end": netaddr.IPAddress("192.168.0.200"), + }, + "ports": { + "start": 1000, + "end": 2000, + }, + "random": True, + }, + }, + ) + ], + ), + ( + "actions=ct(commit,zone=NXM_NX_REG13[0..15],table=2,exec(load:0->NXM_NX_CT_LABEL[0]))", # noqa: E501 + [ + KeyValue( + "ct", + { + "commit": True, + "zone": { + "field": "NXM_NX_REG13", + "start": 0, + "end": 15, + }, + "table": 2, + "exec": [ + { + "load": { + "value": 0, + "dst": { + "field": "NXM_NX_CT_LABEL", + "start": 0, + "end": 0, + }, + }, + }, + ], + }, + ) + ], + ), + ( + "actions=load:0x1->NXM_NX_REG10[7],learn(table=69,delete_learned,cookie=0xda6f52b0,OXM_OF_METADATA[],eth_type=0x800,NXM_OF_IP_SRC[],ip_dst=172.30.204.105,nw_proto=6,NXM_OF_TCP_SRC[]=NXM_OF_TCP_DST[],load:0x1->NXM_NX_REG10[7])", # noqa: E501 + [ + KeyValue( + "load", + { + "value": 1, + "dst": {"field": "NXM_NX_REG10", "start": 7, "end": 7}, + }, + ), + KeyValue( + "learn", + [ + {"table": 69}, + {"delete_learned": True}, + {"cookie": 3664728752}, + {"OXM_OF_METADATA[]": True}, + {"eth_type": 2048}, + {"NXM_OF_IP_SRC[]": True}, + {"ip_dst": IPMask("172.30.204.105/32")}, + {"nw_proto": 6}, + {"NXM_OF_TCP_SRC[]": "NXM_OF_TCP_DST[]"}, + { + "load": { + "value": 1, + "dst": { + "field": "NXM_NX_REG10", + "start": 7, + "end": 7, + }, + } + }, + ], + ), + ], + ), + ( + "actions=resubmit(,8),resubmit:3,resubmit(1,2,ct)", + [ + KeyValue("resubmit", {"port": "", "table": 8}), + KeyValue("resubmit", {"port": 3}), + KeyValue("resubmit", {"port": 1, "table": 2, "ct": True}), + ], + ), + ( + "actions=clone(ct_clear,load:0->NXM_NX_REG11[],load:0->NXM_NX_REG12[],load:0->NXM_NX_REG13[],load:0x1d->NXM_NX_REG13[],load:0x1f->NXM_NX_REG11[],load:0x1c->NXM_NX_REG12[],load:0x11->OXM_OF_METADATA[],load:0x2->NXM_NX_REG14[],load:0->NXM_NX_REG10[],load:0->NXM_NX_REG15[],load:0->NXM_NX_REG0[],load:0->NXM_NX_REG1[],load:0->NXM_NX_REG2[],load:0->NXM_NX_REG3[],load:0->NXM_NX_REG4[],load:0->NXM_NX_REG5[],load:0->NXM_NX_REG6[],load:0->NXM_NX_REG7[],load:0->NXM_NX_REG8[],load:0->NXM_NX_REG9[],resubmit(,8))", # noqa: E501 + [ + KeyValue( + "clone", + [ + {"ct_clear": True}, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG11"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG12"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG13"}, + } + }, + { + "load": { + "value": 29, + "dst": {"field": "NXM_NX_REG13"}, + } + }, + { + "load": { + "value": 31, + "dst": {"field": "NXM_NX_REG11"}, + } + }, + { + "load": { + "value": 28, + "dst": {"field": "NXM_NX_REG12"}, + } + }, + { + "load": { + "value": 17, + "dst": {"field": "OXM_OF_METADATA"}, + } + }, + { + "load": { + "value": 2, + "dst": {"field": "NXM_NX_REG14"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG10"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG15"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG0"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG1"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG2"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG3"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG4"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG5"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG6"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG7"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG8"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG9"}, + } + }, + {"resubmit": {"port": "", "table": 8}}, + ], + ) + ], + ), + ( + "actions=conjunction(1234, 1/2),note:00.00.11.22.33.ff,sample(probability=123,collector_set_id=0x123,obs_domain_id=0x123,obs_point_id=0x123,sampling_port=inport0,ingress)", # noqa: E501 + [ + KeyValue("conjunction", {"id": 1234, "k": 1, "n": 2}), + KeyValue("note", "00.00.11.22.33.ff"), + KeyValue( + "sample", + { + "probability": 123, + "collector_set_id": 0x123, + "obs_domain_id": 0x123, + "obs_point_id": 0x123, + "sampling_port": "inport0", + "ingress": True, + }, + ), + ], + ), + ], +) +def test_act(input_string, expected): + ofp = OFPFlowFactory().from_string(input_string) + actions = ofp.actions_kv + for i in range(len(expected)): + assert expected[i].key == actions[i].key + assert expected[i].value == actions[i].value + + # Assert positions relative to action string are OK + apos = ofp.section("actions").pos + astring = ofp.section("actions").string + + kpos = actions[i].meta.kpos + kstr = actions[i].meta.kstring + vpos = actions[i].meta.vpos + vstr = actions[i].meta.vstring + assert astring[kpos : kpos + len(kstr)] == kstr + if vpos != -1: + assert astring[vpos : vpos + len(vstr)] == vstr + + # assert astring meta is correct + assert input_string[apos : apos + len(astring)] == astring From patchwork Mon Nov 22 11:22:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558034 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=jKHbGyyA; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyQ0x6STHz9sRR for ; Mon, 22 Nov 2021 22:25:05 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 471B4403BB; Mon, 22 Nov 2021 11:25:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OPAYIlJ6DTYk; Mon, 22 Nov 2021 11:24:59 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 3FDA0409C6; Mon, 22 Nov 2021 11:24:53 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 05E35C0036; Mon, 22 Nov 2021 11:24:52 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2209AC003C for ; Mon, 22 Nov 2021 11:24:51 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 02CB3401B9 for ; Mon, 22 Nov 2021 11:23:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MCot0Nz13h7s for ; Mon, 22 Nov 2021 11:23:50 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 6BD7840405 for ; Mon, 22 Nov 2021 11:23:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580229; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=j0NP3kYH804SG4Z/9X0OTg1EJEUYLV+pZEMiD4/oilQ=; b=jKHbGyyAl2CBSFZEVfC7KM8syliIsGl748om29bZ8SQIGP0dtHFSlNe4iOEQ7u5kAdd3Xs qkFvTFemJvuTvc1FdyrYqj3O4z3VyOIfaI38Nena1LpMKPQFvIml+O+eu9ztRn24OtidP2 kUnP5wttsxnhFrvejWvfetj7BgaGhdk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-13-TcQpDoTSNYyYNvUooay-dA-1; Mon, 22 Nov 2021 06:23:47 -0500 X-MC-Unique: TcQpDoTSNYyYNvUooay-dA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 011F51966323 for ; Mon, 22 Nov 2021 11:23:47 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1A7E860862; Mon, 22 Nov 2021 11:23:45 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:55 +0100 Message-Id: <20211122112256.2011194-18-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 17/18] python: add unit tests to datapath parsing X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Signed-off-by: Adrian Moreno --- python/automake.mk | 3 +- python/ovs/tests/test_odp.py | 527 +++++++++++++++++++++++++++++++++++ 2 files changed, 529 insertions(+), 1 deletion(-) create mode 100644 python/ovs/tests/test_odp.py diff --git a/python/automake.mk b/python/automake.mk index 713f1d1a4..cb8697b12 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -56,7 +56,8 @@ ovs_pyfiles = \ ovs_tests = \ python/ovs/tests/test_kv.py \ python/ovs/tests/test_list.py \ - python/ovs/tests/test_ofp.py + python/ovs/tests/test_ofp.py \ + python/ovs/tests/test_odp.py # These python files are used at build time but not runtime, diff --git a/python/ovs/tests/test_odp.py b/python/ovs/tests/test_odp.py new file mode 100644 index 000000000..3690d85a7 --- /dev/null +++ b/python/ovs/tests/test_odp.py @@ -0,0 +1,527 @@ +import netaddr +import pytest + +from ovs.flows.odp import ODPFlowFactory +from ovs.flows.kv import KeyValue +from ovs.flows.decoders import ( + EthMask, + IPMask, + Mask32, + Mask16, + Mask8, + Mask128, +) + + +@pytest.mark.parametrize( + "input_string,expected", + [ + ( + "skb_priority(0x123),skb_mark(0x123),recirc_id(0x123),dp_hash(0x123),ct_zone(0x123), actions:", # noqa: E501 + [ + KeyValue("skb_priority", Mask32("0x123")), + KeyValue("skb_mark", Mask32("0x123")), + KeyValue("recirc_id", 0x123), + KeyValue("dp_hash", Mask32("0x123")), + KeyValue("ct_zone", Mask16("0x123")), + ], + ), + ( + "tunnel(tun_id=0x7f10354,src=10.10.10.10,dst=20.20.20.20,ttl=64,flags(csum|key)) actions:", # noqa: E501 + [ + KeyValue( + "tunnel", + { + "tun_id": 0x7F10354, + "src": IPMask("10.10.10.10"), + "dst": IPMask("20.20.20.20"), + "ttl": 64, + "flags": "csum|key", + }, + ) + ], + ), + ( + "tunnel(geneve({class=0,type=0,len=4,0xa/0xff}),vxlan(flags=0x800000,vni=0x1c7),erspan(ver=2,dir=1,hwid=0x1)), actions:", # noqa: E501 + [ + KeyValue( + "tunnel", + { + "geneve": [ + { + "class": Mask16("0"), + "type": Mask8("0"), + "len": Mask8("4"), + "data": Mask128("0xa/0xff"), + } + ], + "vxlan": {"flags": 0x800000, "vni": 0x1C7}, + "erspan": {"ver": 2, "dir": 1, "hwid": 0x1}, + }, + ) + ], + ), + ( + "in_port(2),eth(src=11:22:33:44:55:66,dst=66:55:44:33:22:11) actions:", # noqa: E501 + [ + KeyValue("in_port", 2), + KeyValue( + "eth", + { + "src": EthMask("11:22:33:44:55:66"), + "dst": EthMask("66:55:44:33:22:11"), + }, + ), + ], + ), + ( + "eth_type(0x800/0x006),ipv4(src=192.168.1.1/24,dst=192.168.0.0/16,proto=0x1,tos=0x2/0xf0) actions:", # noqa: E501 + [ + KeyValue("eth_type", Mask16("0x800/0x006")), + KeyValue( + "ipv4", + { + "src": IPMask("192.168.1.1/24"), + "dst": IPMask("192.168.0.0/16"), + "proto": Mask8("0x1/0xFF"), + "tos": Mask8("0x2/0xF0"), + }, + ), + ], + ), + ( + "encap(eth_type(0x800/0x006),ipv4(src=192.168.1.1/24,dst=192.168.0.0/16,proto=0x1,tos=0x2/0xf0)) actions:", # noqa: E501 + [ + KeyValue( + "encap", + { + "eth_type": Mask16("0x800/0x006"), + "ipv4": { + "src": IPMask("192.168.1.1/24"), + "dst": IPMask("192.168.0.0/16"), + "proto": Mask8("0x1/0xff"), + "tos": Mask8("0x2/0xf0"), + }, + }, + ), + ], + ), + ], +) +def test_odp_fields(input_string, expected): + odp = ODPFlowFactory().from_string(input_string) + match = odp.match_kv + for i in range(len(expected)): + assert expected[i].key == match[i].key + assert expected[i].value == match[i].value + + # Assert positions relative to action string are OK + mpos = odp.section("match").pos + mstring = odp.section("match").string + + kpos = match[i].meta.kpos + kstr = match[i].meta.kstring + vpos = match[i].meta.vpos + vstr = match[i].meta.vstring + assert mstring[kpos : kpos + len(kstr)] == kstr + if vpos != -1: + assert mstring[vpos : vpos + len(vstr)] == vstr + + # assert mstring meta is correct + assert input_string[mpos : mpos + len(mstring)] == mstring + + +@pytest.mark.parametrize( + "input_string,expected", + [ + ( + "actions:ct" + ",ct(commit)" + ",ct(commit,zone=5)" + ",ct(commit,mark=0xa0a0a0a0/0xfefefefe)" + ",ct(commit,label=0x1234567890abcdef1234567890abcdef/0xf1f2f3f4f5f6f7f8f9f0fafbfcfdfeff)" # noqa: E501 + ",ct(commit,helper=ftp)" + ",ct(commit,helper=tftp)" + ",ct(commit,timeout=ovs_tp_1_tcp4)" + ",ct(nat)", + [ + KeyValue("ct", True), + KeyValue("ct", {"commit": True}), + KeyValue("ct", {"commit": True, "zone": 5}), + KeyValue( + "ct", + {"commit": True, "mark": Mask32("0xA0A0A0A0/0xFEFEFEFE")}, + ), + KeyValue( + "ct", + { + "commit": True, + "label": Mask128( + "0x1234567890ABCDEF1234567890ABCDEF/0xF1F2F3F4F5F6F7F8F9F0FAFBFCFDFEFF" # noqa: E501 + ), + }, + ), + KeyValue("ct", {"commit": True, "helper": "ftp"}), + KeyValue("ct", {"commit": True, "helper": "tftp"}), + KeyValue("ct", {"commit": True, "timeout": "ovs_tp_1_tcp4"}), + KeyValue("ct", {"nat": True}), + ], + ), + ( + "actions:ct(nat)" + ",ct(commit,nat(src))" + ",ct(commit,nat(dst))" + ",ct(commit,nat(src=10.0.0.240,random))" + ",ct(commit,nat(src=10.0.0.240:32768-65535,random))" + ",ct(commit,nat(dst=10.0.0.128-10.0.0.254,hash))" + ",ct(commit,nat(src=10.0.0.240-10.0.0.254:32768-65535,persistent))" + ",ct(commit,nat(src=fe80::20c:29ff:fe88:a18b,random))" + ",ct(commit,nat(src=fe80::20c:29ff:fe88:1-fe80::20c:29ff:fe88:a18b,random))" # noqa: E501 + ",ct(commit,nat(src=[[fe80::20c:29ff:fe88:1]]-[[fe80::20c:29ff:fe88:a18b]]:255-4096,random))" # noqa: E501 + ",ct(commit,helper=ftp,nat(src=10.1.1.240-10.1.1.255))" + ",ct(force_commit)", + [ + KeyValue("ct", {"nat": True}), + KeyValue("ct", {"commit": True, "nat": {"type": "src"}}), + KeyValue("ct", {"commit": True, "nat": {"type": "dst"}}), + KeyValue( + "ct", + { + "commit": True, + "nat": { + "type": "src", + "addrs": { + "start": netaddr.IPAddress("10.0.0.240"), + "end": netaddr.IPAddress("10.0.0.240"), + }, + "random": True, + }, + }, + ), + KeyValue( + "ct", + { + "commit": True, + "nat": { + "type": "src", + "addrs": { + "start": netaddr.IPAddress("10.0.0.240"), + "end": netaddr.IPAddress("10.0.0.240"), + }, + "ports": { + "start": 32768, + "end": 65535, + }, + "random": True, + }, + }, + ), + KeyValue( + "ct", + { + "commit": True, + "nat": { + "type": "dst", + "addrs": { + "start": netaddr.IPAddress("10.0.0.128"), + "end": netaddr.IPAddress("10.0.0.254"), + }, + "hash": True, + }, + }, + ), + KeyValue( + "ct", + { + "commit": True, + "nat": { + "type": "src", + "addrs": { + "start": netaddr.IPAddress("10.0.0.240"), + "end": netaddr.IPAddress("10.0.0.254"), + }, + "ports": { + "start": 32768, + "end": 65535, + }, + "persistent": True, + }, + }, + ), + KeyValue( + "ct", + { + "commit": True, + "nat": { + "type": "src", + "addrs": { + "start": netaddr.IPAddress( + "fe80::20c:29ff:fe88:a18b" + ), + "end": netaddr.IPAddress( + "fe80::20c:29ff:fe88:a18b" + ), + }, + "random": True, + }, + }, + ), + KeyValue( + "ct", + { + "commit": True, + "nat": { + "type": "src", + "addrs": { + "start": netaddr.IPAddress( + "fe80::20c:29ff:fe88:1" + ), + "end": netaddr.IPAddress( + "fe80::20c:29ff:fe88:a18b" + ), + }, + "random": True, + }, + }, + ), + KeyValue( + "ct", + { + "commit": True, + "nat": { + "type": "src", + "addrs": { + "start": netaddr.IPAddress( + "fe80::20c:29ff:fe88:1" + ), + "end": netaddr.IPAddress( + "fe80::20c:29ff:fe88:a18b" + ), + }, + "ports": { + "start": 255, + "end": 4096, + }, + "random": True, + }, + }, + ), + KeyValue( + "ct", + { + "commit": True, + "nat": { + "type": "src", + "addrs": { + "start": netaddr.IPAddress("10.1.1.240"), + "end": netaddr.IPAddress("10.1.1.255"), + }, + }, + "helper": "ftp", + }, + ), + KeyValue("ct", {"force_commit": True}), + ], + ), + ( + "actions:set(tunnel(tun_id=0xabcdef1234567890,src=1.1.1.1,dst=2.2.2.2,ttl=64,flags(df|csum|key)))" # noqa: E501 + ",tnl_pop(4)" + ",tnl_push(tnl_port(6),header(size=50,type=4,eth(dst=f8:bc:12:44:34:b6,src=f8:bc:12:46:58:e0,dl_type=0x0800),ipv4(src=1.1.2.88,dst=1.1.2.92,proto=17,tos=0,ttl=64,frag=0x4000),udp(src=0,dst=4789,csum=0x0),vxlan(flags=0x8000000,vni=0x1c7)),out_port(1))" # noqa: E501 + ",tnl_push(tnl_port(6),header(size=70,type=4,eth(dst=f8:bc:12:44:34:b6,src=f8:bc:12:46:58:e0,dl_type=0x86dd),ipv6(src=2001:cafe::88,dst=2001:cafe::92,label=0,proto=17,tclass=0x0,hlimit=64),udp(src=0,dst=4789,csum=0x0),vxlan(flags=0x8000000,vni=0x1c7)),out_port(1))", # noqa: E501 + [ + KeyValue( + "set", + { + "tunnel": { + "tun_id": 0xABCDEF1234567890, + "src": IPMask("1.1.1.1"), + "dst": IPMask("2.2.2.2"), + "ttl": 64, + "flags": "df|csum|key", + } + }, + ), + KeyValue("tnl_pop", 4), + KeyValue( + "tnl_push", + { + "tnl_port": 6, + "header": { + "size": 50, + "type": 4, + "eth": { + "dst": EthMask("f8:bc:12:44:34:b6"), + "src": EthMask("f8:bc:12:46:58:e0"), + "dl_type": 0x800, + }, + "ipv4": { + "src": IPMask("1.1.2.88"), + "dst": IPMask("1.1.2.92"), + "proto": 17, + "tos": 0, + "ttl": 64, + "frag": 0x4000, + }, + "udp": {"src": 0, "dst": 4789, "csum": 0x0}, + "vxlan": { + "flags": 0x8000000, + "vni": 0x1C7, + }, + }, + "out_port": 1, + }, + ), + KeyValue( + "tnl_push", + { + "tnl_port": 6, + "header": { + "size": 70, + "type": 4, + "eth": { + "dst": EthMask("f8:bc:12:44:34:b6"), + "src": EthMask("f8:bc:12:46:58:e0"), + "dl_type": 0x86DD, + }, + "ipv6": { + "src": IPMask("2001:cafe::88"), + "dst": IPMask("2001:cafe::92"), + "label": 0, + "proto": 17, + "tclass": 0x0, + "hlimit": 64, + }, + "udp": {"src": 0, "dst": 4789, "csum": 0x0}, + "vxlan": { + "flags": 0x8000000, + "vni": 0x1C7, + }, + }, + "out_port": 1, + }, + ), + ], + ), + ( + "actions:tnl_push(header(geneve(oam,vni=0x1c7)))" + ",tnl_push(header(geneve(crit,vni=0x1c7,options({class=0xffff,type=0x80,len=4,0xa}))))" # noqa: E501 + ",tnl_push(header(gre((flags=0xa000,proto=0x6558),csum=0x0,key=0x1e241)))", # noqa: E501 + [ + KeyValue( + "tnl_push", + { + "header": { + "geneve": { + "oam": True, + "vni": 0x1C7, + } + } + }, + ), + KeyValue( + "tnl_push", + { + "header": { + "geneve": { + "crit": True, + "vni": 0x1C7, + "options": [ + { + "class": 0xFFFF, + "type": 0x80, + "len": 4, + "data": 0xA, + } + ], + } + } + }, + ), + KeyValue( + "tnl_push", + { + "header": { + "gre": { + "flags": 0xA000, + "proto": 0x6558, + "key": 0x1E241, + "csum": 0x0, + } + } + }, + ), + ], + ), + ( + "actions:clone(1)" ",clone(clone(push_vlan(vid=12,pcp=0),2),1)", + [ + KeyValue("clone", {"output": {"port": 1}}), + KeyValue( + "clone", + { + "output": {"port": 1}, + "clone": { + "push_vlan": { + "vid": 12, + "pcp": 0, + }, + "output": {"port": 2}, + }, + }, + ), + ], + ), + ( + "actions: check_pkt_len(size=200,gt(4),le(5))" + ",check_pkt_len(size=200,gt(drop),le(5))" + ",check_pkt_len(size=200,gt(ct(nat)),le(drop))", + [ + KeyValue( + "check_pkt_len", + { + "size": 200, + "gt": {"output": {"port": 4}}, + "le": {"output": {"port": 5}}, + }, + ), + KeyValue( + "check_pkt_len", + { + "size": 200, + "gt": {"drop": True}, + "le": {"output": {"port": 5}}, + }, + ), + KeyValue( + "check_pkt_len", + { + "size": 200, + "gt": {"ct": {"nat": True}}, + "le": {"drop": True}, + }, + ), + ], + ), + ], +) +def test_odp_actions(input_string, expected): + odp = ODPFlowFactory().from_string(input_string) + actions = odp.actions_kv + for i in range(len(expected)): + assert expected[i].key == actions[i].key + assert expected[i].value == actions[i].value + + # Assert positions relative to action string are OK + apos = odp.section("actions").pos + astring = odp.section("actions").string + + kpos = actions[i].meta.kpos + kstr = actions[i].meta.kstring + vpos = actions[i].meta.vpos + vstr = actions[i].meta.vstring + assert astring[kpos : kpos + len(kstr)] == kstr + if vpos != -1: + assert astring[vpos : vpos + len(vstr)] == vstr + + # assert astring meta is correct + assert input_string[apos : apos + len(astring)] == astring From patchwork Mon Nov 22 11:22:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1558035 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Fshkq5YS; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HyQ0y5k5zz9sX3 for ; Mon, 22 Nov 2021 22:25:06 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 5CB2482DE6; Mon, 22 Nov 2021 11:25:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CTWMAgatjQqE; Mon, 22 Nov 2021 11:25:02 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 81EA281CC3; Mon, 22 Nov 2021 11:24:55 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 75AB9C0043; Mon, 22 Nov 2021 11:24:53 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1F361C003C for ; Mon, 22 Nov 2021 11:24:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id AFAD740402 for ; Mon, 22 Nov 2021 11:23:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2nxUDOzN4u4B for ; Mon, 22 Nov 2021 11:23:51 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 986C0403F0 for ; Mon, 22 Nov 2021 11:23:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637580230; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bBzwMu/Yqo27weO5wQH/5QZ6kBjQajManOeEhcLkDLI=; b=Fshkq5YSCFVd90pKnAHYkJWCFmeVMsyPiu/sUcfUMc4TayPtNNvRhOUb7dcOCS7jqQEzt6 AzcckcjddthCY2uMBZ6eRsj4/yEoRUelSzJJkqx3/MAFaD/1Im6cojTEyI5REs25W4ZWvH Rwqot8E9RCqtppSQHWHuRzffY2IjHYU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-517-8pGhYrLgOvKjNkIV9FH7lg-1; Mon, 22 Nov 2021 06:23:49 -0500 X-MC-Unique: 8pGhYrLgOvKjNkIV9FH7lg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 378D11006AA1 for ; Mon, 22 Nov 2021 11:23:48 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.2.16.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3D11A60862; Mon, 22 Nov 2021 11:23:47 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Mon, 22 Nov 2021 12:22:56 +0100 Message-Id: <20211122112256.2011194-19-amorenoz@redhat.com> In-Reply-To: <20211122112256.2011194-1-amorenoz@redhat.com> References: <20211122112256.2011194-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v1 18/18] python: add unit tests for filtering engine X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Signed-off-by: Adrian Moreno --- python/automake.mk | 3 +- python/ovs/tests/test_filter.py | 225 ++++++++++++++++++++++++++++++++ 2 files changed, 227 insertions(+), 1 deletion(-) create mode 100644 python/ovs/tests/test_filter.py diff --git a/python/automake.mk b/python/automake.mk index cb8697b12..56cd9b7dc 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -57,7 +57,8 @@ ovs_tests = \ python/ovs/tests/test_kv.py \ python/ovs/tests/test_list.py \ python/ovs/tests/test_ofp.py \ - python/ovs/tests/test_odp.py + python/ovs/tests/test_odp.py \ + python/ovs/tests/test_filter.py # These python files are used at build time but not runtime, diff --git a/python/ovs/tests/test_filter.py b/python/ovs/tests/test_filter.py new file mode 100644 index 000000000..e2aed9d77 --- /dev/null +++ b/python/ovs/tests/test_filter.py @@ -0,0 +1,225 @@ +import pytest + +from ovs.flows.filter import OFFilter +from ovs.flows.ofp import OFPFlowFactory +from ovs.flows.odp import ODPFlowFactory + + +ofp_factory = OFPFlowFactory() +odp_factory = ODPFlowFactory() + + +@pytest.mark.parametrize( + "expr,flow,expected,match", + [ + ( + "nw_src=192.168.1.1 && tcp_dst=80", + ofp_factory.from_string( + "nw_src=192.168.1.1,tcp_dst=80 actions=drop" + ), + True, + ["nw_src", "tcp_dst"], + ), + ( + "nw_src=192.168.1.2 || tcp_dst=80", + ofp_factory.from_string( + "nw_src=192.168.1.1,tcp_dst=80 actions=drop" + ), + True, + ["nw_src", "tcp_dst"], + ), + ( + "nw_src=192.168.1.1 || tcp_dst=90", + ofp_factory.from_string( + "nw_src=192.168.1.1,tcp_dst=80 actions=drop" + ), + True, + ["nw_src", "tcp_dst"], + ), + ( + "nw_src=192.168.1.2 && tcp_dst=90", + ofp_factory.from_string( + "nw_src=192.168.1.1,tcp_dst=80 actions=drop" + ), + False, + ["nw_src", "tcp_dst"], + ), + ( + "nw_src=192.168.1.1", + ofp_factory.from_string( + "nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" + ), + False, + ["nw_src"], + ), + ( + "nw_src~=192.168.1.1", + ofp_factory.from_string( + "nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" + ), + True, + ["nw_src"], + ), + ( + "nw_src~=192.168.1.1/30", + ofp_factory.from_string( + "nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" + ), + True, + ["nw_src"], + ), + ( + "nw_src~=192.168.1.0/16", + ofp_factory.from_string( + "nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" + ), + False, + ["nw_src"], + ), + ( + "nw_src~=192.168.1.0/16", + ofp_factory.from_string( + "nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" + ), + False, + ["nw_src"], + ), + ( + "n_bytes=100", + ofp_factory.from_string( + "n_bytes=100 priority=100,nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" # noqa: E501 + ), + True, + ["n_bytes"], + ), + ( + "n_bytes>10", + ofp_factory.from_string( + "n_bytes=100 priority=100,nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" # noqa: E501 + ), + True, + ["n_bytes"], + ), + ( + "n_bytes>100", + ofp_factory.from_string( + "n_bytes=100 priority=100,nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" # noqa: E501 + ), + False, + ["n_bytes"], + ), + ( + "n_bytes<100", + ofp_factory.from_string( + "n_bytes=100 priority=100,nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" # noqa: E501 + ), + False, + ["n_bytes"], + ), + ( + "n_bytes<1000", + ofp_factory.from_string( + "n_bytes=100 priority=100,nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" # noqa: E501 + ), + True, + ["n_bytes"], + ), + ( + "n_bytes>0 && drop=true", + ofp_factory.from_string( + "n_bytes=100 priority=100,nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" # noqa: E501 + ), + True, + ["n_bytes", "drop"], + ), + ( + "n_bytes>0 && drop=true", + ofp_factory.from_string( + "n_bytes=100 priority=100,nw_src=192.168.1.0/24,tcp_dst=80 actions=2" # noqa: E501 + ), + False, + ["n_bytes"], + ), + ( + "n_bytes>10 && !output.port=3", + ofp_factory.from_string( + "n_bytes=100 priority=100,nw_src=192.168.1.0/24,tcp_dst=80 actions=2" # noqa: E501 + ), + True, + ["n_bytes", "output"], + ), + ( + "dl_src=00:11:22:33:44:55", + ofp_factory.from_string( + "n_bytes=100 priority=100,dl_src=00:11:22:33:44:55,nw_src=192.168.1.0/24,tcp_dst=80 actions=2" # noqa: E501 + ), + True, + ["dl_src"], + ), + ( + "dl_src~=00:11:22:33:44:55", + ofp_factory.from_string( + "n_bytes=100 priority=100,dl_src=00:11:22:33:44:55/ff:ff:ff:ff:ff:00,nw_src=192.168.1.0/24,tcp_dst=80 actions=2" # noqa: E501 + ), + True, + ["dl_src"], + ), + ( + "dl_src~=00:11:22:33:44:66", + ofp_factory.from_string( + "n_bytes=100 priority=100,dl_src=00:11:22:33:44:55/ff:ff:ff:ff:ff:00,nw_src=192.168.1.0/24,tcp_dst=80 actions=2" # noqa: E501 + ), + True, + ["dl_src"], + ), + ( + "dl_src~=00:11:22:33:44:66 && tp_dst=1000", + ofp_factory.from_string( + "n_bytes=100 priority=100,dl_src=00:11:22:33:44:55/ff:ff:ff:ff:ff:00,nw_src=192.168.1.0/24,tp_dst=0x03e8/0xfff8 actions=2" # noqa: E501 + ), + False, + ["dl_src", "tp_dst"], + ), + ( + "dl_src~=00:11:22:33:44:66 && tp_dst~=1000", + ofp_factory.from_string( + "n_bytes=100 priority=100,dl_src=00:11:22:33:44:55/ff:ff:ff:ff:ff:00,nw_src=192.168.1.0/24,tp_dst=0x03e8/0xfff8 actions=2" # noqa: E501 + ), + True, + ["dl_src", "tp_dst"], + ), + ( + "encap", + odp_factory.from_string( + "encap(eth_type(0x0800),ipv4(src=10.76.23.240/255.255.255.248,dst=10.76.23.106,proto=17,tos=0/0,ttl=64,frag=no)) actions:drop" # noqa: E501 + ), + True, + ["encap"], + ), + ( + "encap.ipv4.src=10.76.23.240", + odp_factory.from_string( + "encap(eth_type(0x0800),ipv4(src=10.76.23.240/255.255.255.248,dst=10.76.23.106,proto=17,tos=0/0,ttl=64,frag=no)) actions:drop" # noqa: E501 + ), + False, + ["encap"], + ), + ( + "encap.ipv4.src~=10.76.23.240", + odp_factory.from_string( + "encap(eth_type(0x0800),ipv4(src=10.76.23.240/255.255.255.248,dst=10.76.23.106,proto=17,tos=0/0,ttl=64,frag=no)) actions:drop" # noqa: E501 + ), + True, + ["encap"], + ), + ], +) +def test_filter(expr, flow, expected, match): + ffilter = OFFilter(expr) + result = ffilter.evaluate(flow) + if expected: + assert result + else: + assert not result + + assert [kv.key for kv in result.kv] == match