From patchwork Fri Jul 8 18:03:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654330 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=LWB5oeDE; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh3Y2tvQz9ryY for ; Sat, 9 Jul 2022 04:03:37 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id CC4F241639; Fri, 8 Jul 2022 18:03:34 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org CC4F241639 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=LWB5oeDE X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1DujR4Qq29Hj; Fri, 8 Jul 2022 18:03:32 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 3603F41627; Fri, 8 Jul 2022 18:03:31 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 3603F41627 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2A68CC0085; Fri, 8 Jul 2022 18:03:29 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8D1E9C002D for ; Fri, 8 Jul 2022 18:03:27 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 5CE1E403BE for ; Fri, 8 Jul 2022 18:03:27 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 5CE1E403BE X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fv8DhJVsFT4F for ; Fri, 8 Jul 2022 18:03:26 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org DACE440025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id DACE440025 for ; Fri, 8 Jul 2022 18:03:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303404; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nKrsJcBgL/bgsdxcpqww1GPMjVVW2QboL22kGMzdILM=; b=LWB5oeDEf6Hd/VbHuraOHUnpHtjrcU1M6kwNY1sLRc3uu3/gGcROVsYKkBCHsbFBSG36BT QA1kiNMO6Ir/alEThAzBhf/Yu+20oRyyyruKtPNkYh6LVwMSJSyp4FtEo8sfapr12mjXAk twy8X4PUAEBtQ7XdND4sMznVUyl3fY0= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-260-vk6oebBMM_ygPT1hf_tsIw-1; Fri, 08 Jul 2022 14:03:23 -0400 X-MC-Unique: vk6oebBMM_ygPT1hf_tsIw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 15F6729AA386; Fri, 8 Jul 2022 18:03:23 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8CCE32026D64; Fri, 8 Jul 2022 18:03:21 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:00 +0200 Message-Id: <20220708180316.2852046-2-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 01/17] python: add generic Key-Value parser X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Most of ofproto and dpif flows are based on key-value pairs. These key-value pairs can be represented in several ways, eg: key:value, key=value, key(value). Add the following classes that allow parsing of key-value strings: * KeyValue: holds a key-value pair * KeyMetadata: holds some metadata associated with a KeyValue such as the original key and value strings and their position in the global string * KVParser: is able to parse a string and extract it's key-value pairs as KeyValue instances. Before creating the KeyValue instance it tries to decode the value via the KVDecoders * KVDecoders holds a number of decoders that KVParser can use to decode key-value pairs. It accepts a dictionary of keys and callables to allow users to specify what decoder (i.e: callable) to use for each key Also, flake8 seems to be incorrectly reporting an error (E203) in: "slice[index + offset : index + offset]" which is PEP8 compliant. So, ignore this error. Acked-by: Terry Wilson Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- Makefile.am | 3 +- python/automake.mk | 6 +- python/ovs/flow/__init__.py | 0 python/ovs/flow/decoders.py | 18 +++ python/ovs/flow/kv.py | 314 ++++++++++++++++++++++++++++++++++++ python/setup.py | 2 +- 6 files changed, 340 insertions(+), 3 deletions(-) create mode 100644 python/ovs/flow/__init__.py create mode 100644 python/ovs/flow/decoders.py create mode 100644 python/ovs/flow/kv.py diff --git a/Makefile.am b/Makefile.am index cb8076433..4f51d225e 100644 --- a/Makefile.am +++ b/Makefile.am @@ -391,6 +391,7 @@ ALL_LOCAL += flake8-check # E128 continuation line under-indented for visual indent # E129 visually indented line with same indent as next logical line # E131 continuation line unaligned for hanging indent +# E203 whitespace before ':' # E722 do not use bare except, specify exception instead # W503 line break before binary operator # W504 line break after binary operator @@ -403,7 +404,7 @@ ALL_LOCAL += flake8-check # H233 Python 3.x incompatible use of print operator # H238 old style class declaration, use new style (inherit from `object`) FLAKE8_SELECT = H231,H232,H233,H238 -FLAKE8_IGNORE = E121,E123,E125,E126,E127,E128,E129,E131,E722,W503,W504,F811,D,H,I +FLAKE8_IGNORE = E121,E123,E125,E126,E127,E128,E129,E131,E203,E722,W503,W504,F811,D,H,I flake8-check: $(FLAKE8_PYFILES) $(FLAKE8_WERROR)$(AM_V_GEN) \ src='$^' && \ diff --git a/python/automake.mk b/python/automake.mk index 89c2ccbb0..3a9ec3461 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -16,7 +16,6 @@ ovs_pyfiles = \ python/ovs/compat/sortedcontainers/sorteddict.py \ python/ovs/compat/sortedcontainers/sortedset.py \ python/ovs/daemon.py \ - python/ovs/fcntl_win.py \ python/ovs/db/__init__.py \ python/ovs/db/custom_index.py \ python/ovs/db/data.py \ @@ -26,6 +25,10 @@ ovs_pyfiles = \ python/ovs/db/schema.py \ python/ovs/db/types.py \ python/ovs/fatal_signal.py \ + python/ovs/fcntl_win.py \ + python/ovs/flow/__init__.py \ + python/ovs/flow/decoders.py \ + python/ovs/flow/kv.py \ python/ovs/json.py \ python/ovs/jsonrpc.py \ python/ovs/ovsuuid.py \ @@ -42,6 +45,7 @@ ovs_pyfiles = \ python/ovs/version.py \ python/ovs/vlog.py \ python/ovs/winutils.py + # These python files are used at build time but not runtime, # so they are not installed. EXTRA_DIST += \ diff --git a/python/ovs/flow/__init__.py b/python/ovs/flow/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/python/ovs/flow/decoders.py b/python/ovs/flow/decoders.py new file mode 100644 index 000000000..0c2259c76 --- /dev/null +++ b/python/ovs/flow/decoders.py @@ -0,0 +1,18 @@ +"""Defines helpful decoders that can be used to decode information from the +flows. + +A decoder is generally a callable that accepts a string and returns the value +object. +""" + + +def decode_default(value): + """Default decoder. + + It tries to convert into an integer value and, if it fails, just + returns the string. + """ + try: + return int(value, 0) + except ValueError: + return value diff --git a/python/ovs/flow/kv.py b/python/ovs/flow/kv.py new file mode 100644 index 000000000..cceb95e43 --- /dev/null +++ b/python/ovs/flow/kv.py @@ -0,0 +1,314 @@ +"""Common helper classes for flow Key-Value parsing.""" + +import functools +import re + +from ovs.flow.decoders import decode_default + + +class ParseError(RuntimeError): + """Exception raised when an error occurs during parsing.""" + + pass + + +class KeyMetadata(object): + """Class for keeping key metadata. + + Attributes: + kpos (int): The position of the keyword in the parent string. + vpos (int): The position of the value in the parent string. + kstring (string): The keyword string as found in the flow string. + vstring (string): The value as found in the flow string. + delim (string): Optional, the string used as delimiter between the key + and the value. + end_delim (string): Optional, the string used as end delimiter + """ + + def __init__(self, kpos, vpos, kstring, vstring, delim="", end_delim=""): + """Constructor.""" + self.kpos = kpos + self.vpos = vpos + self.kstring = kstring + self.vstring = vstring + self.delim = delim + self.end_delim = end_delim + + def __str__(self): + return "key: [{},{}), val:[{}, {})".format( + self.kpos, + self.kpos + len(self.kstring), + self.vpos, + self.vpos + len(self.vstring), + ) + + def __repr__(self): + return "{}('{}')".format(self.__class__.__name__, self) + + +class KeyValue(object): + """Class for keeping key-value data. + + Attributes: + key (str): The key string. + value (any): The value data. + meta (KeyMetadata): The key metadata. + """ + + def __init__(self, key, value, meta=None): + """Constructor.""" + self.key = key + self.value = value + self.meta = meta + + def __str__(self): + return "{}: {} ({})".format(self.key, str(self.value), str(self.meta)) + + def __repr__(self): + return "{}('{}')".format(self.__class__.__name__, self) + + +class KVDecoders(object): + """KVDecoders class is used by KVParser to select how to decode the value + of a specific keyword. + + A decoder is simply a function that accepts a value string and returns + the value objects to be stored. + The returned value may be of any type. + + Decoders may return a KeyValue instance to indicate that the keyword should + also be modified to match the one provided in the returned KeyValue. + + The decoder to be used will be selected using the key as an index. If not + found, the default decoder will be used. If free keys are found (i.e: + keys without a value), the default_free decoder will be used. For that + reason, the default_free decoder, must return both the key and value to be + stored. + + Args: + decoders (dict): Optional; A dictionary of decoders indexed by keyword. + default (callable): Optional; A decoder used if a match is not found in + configured decoders. If not provided, the default behavior is to + try to decode the value into an integer and, if that fails, + just return the string as-is. + default_free (callable): Optional; The decoder used if a match is not + found in configured decoders and it's a free value (e.g: + a value without a key) Defaults to returning the free value as + keyword and "True" as value. + The callable must accept a string and return a key-value pair. + """ + + def __init__(self, decoders=None, default=None, default_free=None): + self._decoders = decoders or dict() + self._default = default or decode_default + self._default_free = default_free or self._default_free_decoder + + def decode(self, keyword, value_str): + """Decode a keyword and value. + + Args: + keyword (str): The keyword whose value is to be decoded. + value_str (str): The value string. + + Returns: + The key (str) and value(any) to be stored. + """ + + decoder = self._decoders.get(keyword) + if decoder: + result = decoder(value_str) + if isinstance(result, KeyValue): + keyword = result.key + value = result.value + else: + value = result + + return keyword, value + else: + if value_str: + return keyword, self._default(value_str) + else: + return self._default_free(keyword) + + @staticmethod + def _default_free_decoder(key): + """Default decoder for free keywords.""" + return key, True + + +delim_pattern = re.compile(r"(\(|=|:|,|\n|\r|\t)") +parenthesis = re.compile(r"(\(|\))") +end_pattern = re.compile(r"( |,|\n|\r|\t)") +separators = (" ", ",") +end_of_string = (",", "\n", "\t", "\r", "") + + +class KVParser(object): + """KVParser parses a string looking for key-value pairs. + + Args: + string (str): The string to parse. + decoders (KVDecoders): Optional; the KVDecoders instance to use. + """ + + def __init__(self, string, decoders=None): + """Constructor.""" + self._decoders = decoders or KVDecoders() + self._keyval = list() + self._string = string + + def keys(self): + return list(kv.key for kv in self._keyval) + + def kv(self): + return self._keyval + + def __iter__(self): + return iter(self._keyval) + + def parse(self): + """Parse the key-value pairs in string. + + The input string is assumed to contain a list of comma (or space) + separated key-value pairs. + + Key-values pairs can have multiple different delimiters, eg: + "key1:value1,key2=value2,key3(value3)". + + Also, we can stumble upon a "free" keywords, e.g: + "key1=value1,key2=value2,free_keyword". + We consider this as keys without a value. + + So, to parse the string we do the following until the end of the + string is found: + + 1 - Skip any leading comma's or spaces. + 2 - Find the next delimiter (or end_of_string character). + 3 - Depending on the delimiter, obtain the key and the value. + For instance, if the delimiter is "(", find the next matching + ")". + 4 - Use the KVDecoders to decode the key-value. + 5 - Store the KeyValue object with the corresponding metadata. + + Raises: + ParseError if any parsing error occurs. + """ + kpos = 0 + while kpos < len(self._string) and self._string[kpos] != "\n": + keyword = "" + delimiter = "" + rest = "" + + # 1. Skip separator characters. + if self._string[kpos] in separators: + kpos += 1 + continue + + # 2. Find the next delimiter or end of string character. + try: + keyword, delimiter, rest = delim_pattern.split( + self._string[kpos:], 1 + ) + except ValueError: + keyword = self._string[kpos:] # Free keyword + + # 3. Extract the value from the rest of the string. + value_str = "" + vpos = kpos + len(keyword) + 1 + end_delimiter = "" + + if delimiter in ("=", ":"): + # If the delimiter is ':' or '=', the end of the value is the + # end of the string or a ', '. + value_parts = end_pattern.split(rest, 1) + value_str = value_parts[0] + next_kpos = vpos + len(value_str) + + elif delimiter == "(": + # Find matching ")". + level = 1 + index = 0 + value_parts = parenthesis.split(rest) + for val in value_parts: + if val == "(": + level += 1 + elif val == ")": + level -= 1 + index += len(val) + if level == 0: + break + + if level != 0: + raise ParseError( + "Error parsing string {}: " + "Failed to find matching ')' in {}".format( + self._string, rest + ) + ) + + value_str = rest[: index - 1] + next_kpos = vpos + len(value_str) + 1 + end_delimiter = ")" + + # Exceptionally, if after the () we find -> {}, do not treat + # the content of the parenthesis as the value, consider + # ({})->{} as the string value. + if index < len(rest) - 2 and rest[index : index + 2] == "->": + extra_val = rest[index + 2 :].split(",")[0] + value_str = "({})->{}".format(value_str, extra_val) + # remove the first "(". + vpos -= 1 + next_kpos = vpos + len(value_str) + end_delimiter = "" + + elif delimiter in end_of_string: + # Key without a value. + next_kpos = kpos + len(keyword) + vpos = -1 + + # 4. Use KVDecoders to decode the key-value. + try: + key, val = self._decoders.decode(keyword, value_str) + except Exception as e: + raise ParseError( + "Error parsing key-value ({}, {})".format( + keyword, value_str + ) + ) from e + + # Store the KeyValue object with the corresponding metadata. + meta = KeyMetadata( + kpos=kpos, + vpos=vpos, + kstring=keyword, + vstring=value_str, + delim=delimiter, + end_delim=end_delimiter, + ) + + self._keyval.append(KeyValue(key, val, meta)) + + kpos = next_kpos + + +def decode_nested_kv(decoders, value): + """A key-value decoder that extracts nested key-value pairs and returns + them in a dictionary. + + Args: + decoders (KVDecoders): The KVDecoders to use. + value (str): The value string to decode. + """ + if not value: + # Mark as flag + return True + + parser = KVParser(value, decoders) + parser.parse() + return {kv.key: kv.value for kv in parser.kv()} + + +def nested_kv_decoder(decoders=None): + """Helper function that creates a nested kv decoder with given + KVDecoders.""" + return functools.partial(decode_nested_kv, decoders) diff --git a/python/setup.py b/python/setup.py index cfe01763f..7ac3c3662 100644 --- a/python/setup.py +++ b/python/setup.py @@ -71,7 +71,7 @@ setup_args = dict( author='Open vSwitch', author_email='dev@openvswitch.org', packages=['ovs', 'ovs.compat', 'ovs.compat.sortedcontainers', - 'ovs.db', 'ovs.unixctl'], + 'ovs.db', 'ovs.unixctl', 'ovs.flow'], keywords=['openvswitch', 'ovs', 'OVSDB'], license='Apache 2.0', classifiers=[ From patchwork Fri Jul 8 18:03:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654332 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Lzb1Sz5P; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh3k119Wz9ryY for ; Sat, 9 Jul 2022 04:03:45 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id A248B42600; Fri, 8 Jul 2022 18:03:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org A248B42600 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Lzb1Sz5P X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1FsCOXvpzOGI; Fri, 8 Jul 2022 18:03:40 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 57A2842653; Fri, 8 Jul 2022 18:03:39 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 57A2842653 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 01C0FC0077; Fri, 8 Jul 2022 18:03:39 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3C6D9C0081 for ; Fri, 8 Jul 2022 18:03:37 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 3BEDF84819 for ; Fri, 8 Jul 2022 18:03:35 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 3BEDF84819 Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Lzb1Sz5P X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2ccOEMv5ZhO5 for ; Fri, 8 Jul 2022 18:03:33 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 7A51A8480C Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 7A51A8480C for ; Fri, 8 Jul 2022 18:03:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303412; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=P5z90L8Y4MKRfYvaQOf/gXkNSl7qIqWu//QpZvvUYrg=; b=Lzb1Sz5PAVi4h+ocTsKonDvLwRGot7yg0+pnElfwTc32QrzEwFA6kShBxpfygTKmBt7lo/ JGKwnOrWtPnGJ0G+MwQv6Odsj4BknA7ZlurbDolABcoH8cU0MHnzxd9DTP1AdEjP2hv8T1 GQiYcxm0b//oSLLhohBtN3agMDPhpCM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-206-8awlZsmePQCIuM-MJdBftg-1; Fri, 08 Jul 2022 14:03:25 -0400 X-MC-Unique: 8awlZsmePQCIuM-MJdBftg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1D5B2101A54E; Fri, 8 Jul 2022 18:03:25 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id 76A6A2026D64; Fri, 8 Jul 2022 18:03:23 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:01 +0200 Message-Id: <20220708180316.2852046-3-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 02/17] python: add mask, ip and eth decoders X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add more decoders that can be used by KVParser. For IPv4 and IPv6 addresses, create a new class that wraps netaddr.IPAddress. For Ethernet addresses, create a new class that wraps netaddr.EUI. For Integers, create a new class that performs basic bitwise mask comparisons netaddr is added as a new shoft dependency: - extras_require in setup.py - Suggests in deb and rpm packages Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- .ci/linux-prepare.sh | 3 + .cirrus.yml | 1 + Documentation/topics/language-bindings.rst | 9 + debian/control | 1 + python/ovs/flow/__init__.py | 13 + python/ovs/flow/decoders.py | 398 +++++++++++++++++++++ python/setup.py | 3 +- rhel/openvswitch-fedora.spec.in | 1 + 8 files changed, 428 insertions(+), 1 deletion(-) diff --git a/.ci/linux-prepare.sh b/.ci/linux-prepare.sh index 1fe890846..190e69b85 100755 --- a/.ci/linux-prepare.sh +++ b/.ci/linux-prepare.sh @@ -29,6 +29,9 @@ pip3 install --disable-pip-version-check --user \ flake8 'hacking>=3.0' sphinx setuptools pyelftools pip3 install --user 'meson==0.49.2' +# Install python dependencies. +pip3 install --user netaddr + if [ "$M32" ]; then # Installing 32-bit libraries. pkgs="gcc-multilib" diff --git a/.cirrus.yml b/.cirrus.yml index a4d2a5bbc..5a3ca2589 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -19,6 +19,7 @@ freebsd_build_task: - pkg update -f - pkg install -y ${DEPENDENCIES} $(pkg search -xq "^py3[0-9]+-(${PY_DEPS})-[0-9]+" | xargs) + - python3 -m pip install --user netaddr configure_script: - ./boot.sh diff --git a/Documentation/topics/language-bindings.rst b/Documentation/topics/language-bindings.rst index 43c64349a..5025f069f 100644 --- a/Documentation/topics/language-bindings.rst +++ b/Documentation/topics/language-bindings.rst @@ -40,6 +40,15 @@ the bindings using ``pip``: $ pip install ovs +The Python bindings include an optional flow parsing library. To install it's +required dependencies, run: + +:: + + $ pip install ovs[flow] + +or install `python3-netaddr`. + __ https://github.com/openvswitch/ovs/tree/master/python/ovs Third-Party Bindings diff --git a/debian/control b/debian/control index 6420b9d3e..e23d55ad2 100644 --- a/debian/control +++ b/debian/control @@ -174,6 +174,7 @@ Package: python3-openvswitch Architecture: all Section: python Depends: ${misc:Depends}, ${python3:Depends} +Suggests: python3-netaddr Description: Python bindings for Open vSwitch Open vSwitch is a production quality, multilayer, software-based, Ethernet virtual switch. It is designed to enable massive network diff --git a/python/ovs/flow/__init__.py b/python/ovs/flow/__init__.py index e69de29bb..68453cc8e 100644 --- a/python/ovs/flow/__init__.py +++ b/python/ovs/flow/__init__.py @@ -0,0 +1,13 @@ +""" Global flow library entrypoint. +""" +for libname in ["netaddr"]: + try: + lib = __import__(libname) + except ModuleNotFoundError as e: + raise ImportError( + f"OVS Flow library requires {libname} to be installed." + " To install all the dependencies needed for the Flow library, run" + " 'pip install -e ovs[flow]' (or 'pip install -e .[flow]' locally)" + ) from e + else: + globals()[libname] = lib diff --git a/python/ovs/flow/decoders.py b/python/ovs/flow/decoders.py index 0c2259c76..883e61acf 100644 --- a/python/ovs/flow/decoders.py +++ b/python/ovs/flow/decoders.py @@ -5,6 +5,15 @@ A decoder is generally a callable that accepts a string and returns the value object. """ +import netaddr + + +class Decoder(object): + """Base class for all decoder classes.""" + + def to_json(self): + raise NotImplementedError() + def decode_default(value): """Default decoder. @@ -16,3 +25,392 @@ def decode_default(value): return int(value, 0) except ValueError: return value + + +def decode_flag(value): + """Decode a flag. It's existence is just flagged by returning True.""" + return True + + +def decode_int(value): + """Integer decoder. + + Both base10 and base16 integers are supported. + + Used for fields such as: + n_bytes=34 + metadata=0x4 + """ + return int(value, 0) + + +def decode_time(value): + """Time decoder. + + Used for fields such as: + duration=1234.123s + """ + if value == "never": + return value + + time_str = value.rstrip("s") + return float(time_str) + + +class IntMask(Decoder): + """Base class for Integer Mask decoder classes. + + It supports decoding a value/mask pair. The class has to be derived, + and the size attribute must be set. + """ + + size = None # Size in bits. + + def __init__(self, string): + if not self.size: + raise NotImplementedError( + "IntMask should be derived and size should be fixed" + ) + + parts = string.split("/") + if len(parts) > 1: + self._value = int(parts[0], 0) + self._mask = int(parts[1], 0) + if self._mask.bit_length() > self.size: + raise ValueError( + "Integer mask {} is bigger than size {}".format( + self._mask, self.size + ) + ) + else: + self._value = int(parts[0], 0) + self._mask = self.max_mask() + + if self._value.bit_length() > self.size: + raise ValueError( + "Integer value {} is bigger than size {}".format( + self._value, self.size + ) + ) + + @property + def value(self): + return self._value + + @property + def mask(self): + return self._mask + + def max_mask(self): + return 2 ** self.size - 1 + + def fully(self): + """Returns True if it's fully masked.""" + return self._mask == self.max_mask() + + def __str__(self): + if self.fully(): + return str(self._value) + else: + return "{}/{}".format(hex(self._value), hex(self._mask)) + + def __repr__(self): + return "%s('%s')" % (self.__class__.__name__, self) + + def __eq__(self, other): + """Equality operator. + + Both value and mask must be the same for the comparison to result True. + This can be used to implement filters that expect a specific mask, + e.g: ct.state = 0x1/0xff. + + Args: + other (IntMask): Another IntMask to compare against. + + Returns: + True if the other IntMask is the same as this one. + """ + if isinstance(other, IntMask): + return self.value == other.value and self.mask == other.mask + elif isinstance(other, int): + return self.value == other and self.mask == self.max_mask() + else: + raise ValueError("Cannot compare against ", other) + + def __contains__(self, other): + """Contains operator. + + Args: + other (int or IntMask): Another integer or fully-masked IntMask + to compare against. + + Returns: + True if the other integer or fully-masked IntMask is + contained in this IntMask. + + Example: + 0x1 in IntMask("0xf1/0xff"): True + 0x1 in IntMask("0xf1/0x0f"): True + 0x1 in IntMask("0xf1/0xf0"): False + """ + if isinstance(other, IntMask): + if other.fully(): + return other.value in self + else: + raise ValueError( + "Comparing non fully-masked IntMasks is not supported" + ) + else: + return other & self._mask == self._value & self._mask + + def dict(self): + return {"value": self._value, "mask": self._mask} + + def to_json(self): + return self.dict() + + +class Mask8(IntMask): + size = 8 + + +class Mask16(IntMask): + size = 16 + + +class Mask32(IntMask): + size = 32 + + +class Mask64(IntMask): + size = 64 + + +class Mask128(IntMask): + size = 128 + + +class Mask992(IntMask): + size = 992 + + +def decode_mask(mask_size): + """Value/Mask decoder for values of specific size (bits). + + Used for fields such as: + reg0=0x248/0xff + """ + + class Mask(IntMask): + size = mask_size + __name__ = "Mask{}".format(size) + + return Mask + + +class EthMask(Decoder): + """EthMask represents an Ethernet address with optional mask. + + It uses netaddr.EUI. + + Attributes: + eth (netaddr.EUI): The Ethernet address. + mask (netaddr.EUI): Optional, the Ethernet address mask. + + Args: + string (str): A string representing the masked Ethernet address + e.g: 00.11:22:33:44:55 or 01:00:22:00:33:00/01:00:00:00:00:00 + """ + + def __init__(self, string): + mask_parts = string.split("/") + self._eth = netaddr.EUI(mask_parts[0]) + if len(mask_parts) == 2: + self._mask = netaddr.EUI(mask_parts[1]) + else: + self._mask = None + + @property + def eth(self): + """The Ethernet address.""" + return self._eth + + @property + def mask(self): + """The Ethernet address mask.""" + return self._mask + + def __eq__(self, other): + """Equality operator. + + Both the Ethernet address and the mask are compared. This can be used + to implement filters where we expect a specific mask to be present, + e.g: dl_dst=01:00:00:00:00:00/01:00:00:00:00:00. + + Args: + other (EthMask): Another EthMask to compare against. + + Returns: + True if this EthMask is the same as the other. + """ + return self._mask == other._mask and self._eth == other._eth + + def __contains__(self, other): + """Contains operator. + + Args: + other (netaddr.EUI or EthMask): An Ethernet address. + + Returns: + True if the other netaddr.EUI or fully-masked EthMask is + contained in this EthMask's address range. + """ + if isinstance(other, EthMask): + if other._mask: + raise ValueError( + "Comparing non fully-masked EthMask is not supported" + ) + return other._eth in self + + if self._mask: + return (other.value & self._mask.value) == ( + self._eth.value & self._mask.value + ) + else: + return other == self._eth + + def __str__(self): + if self._mask: + return "/".join( + [ + self._eth.format(netaddr.mac_unix), + self._mask.format(netaddr.mac_unix), + ] + ) + else: + return self._eth.format(netaddr.mac_unix) + + def __repr__(self): + return "%s('%s')" % (self.__class__.__name__, self) + + def to_json(self): + return str(self) + + +class IPMask(Decoder): + """IPMask stores an IPv6 or IPv4 and a mask. + + It uses netaddr.IPAddress. + + IPMasks can represent valid CIDRs or randomly masked IP Addresses. + + Args: + string (str): A string representing the ip/mask. + """ + + def __init__(self, string): + self._ipnet = None + self._ip = None + self._mask = None + try: + self._ipnet = netaddr.IPNetwork(string) + except netaddr.AddrFormatError: + pass + + if not self._ipnet: + # It's not a valid CIDR. Store ip and mask independently. + parts = string.split("/") + if len(parts) != 2: + raise ValueError( + "value {}: is not an ipv4 or ipv6 address".format(string) + ) + try: + self._ip = netaddr.IPAddress(parts[0]) + self._mask = netaddr.IPAddress(parts[1]) + except netaddr.AddrFormatError as exc: + raise ValueError( + "value {}: is not an ipv4 or ipv6 address".format(string) + ) from exc + + def __eq__(self, other): + """Equality operator. + + Both the IPAddress and the mask are compared. This can be used + to implement filters where a specific mask is expected, e.g: + nw_src=192.168.1.0/24. + + Args: + other (IPMask or netaddr.IPNetwork or netaddr.IPAddress): + Another IPAddress or IPNetwork to compare against. + + Returns: + True if this IPMask is the same as the other. + """ + if isinstance(other, netaddr.IPNetwork): + return self._ipnet and self._ipnet == other + if isinstance(other, netaddr.IPAddress): + return self._ipnet and self._ipnet.ip == other + elif isinstance(other, IPMask): + if self._ipnet: + return self._ipnet == other._ipnet + + return self._ip == other._ip and self._mask == other._mask + else: + return False + + def __contains__(self, other): + """Contains operator. + + Only comparing valid CIDRs is supported. + + Args: + other (netaddr.IPAddress or IPMask): An IP address. + + Returns: + True if the other IPAddress is contained in this IPMask's address + range. + """ + if isinstance(other, IPMask): + if not other._ipnet: + raise ValueError("Only comparing valid CIDRs is supported") + + return ( + netaddr.IPAddress(other._ipnet.first) in self + and netaddr.IPAddress(other._ipnet.last) in self + ) + + elif isinstance(other, netaddr.IPAddress): + if self._ipnet: + return other in self._ipnet + return (other & self._mask) == (self._ip & self._mask) + + def cidr(self): + """ + Returns True if the IPMask is a valid CIDR. + """ + return self._ipnet is not None + + @property + def ip(self): + """The IP address.""" + if self._ipnet: + return self._ipnet.ip + return self._ip + + @property + def mask(self): + """The IP mask.""" + if self._ipnet: + return self._ipnet.netmask + return self._mask + + def __str__(self): + if self._ipnet: + return str(self._ipnet) + return "/".join([str(self._ip), str(self._mask)]) + + def __repr__(self): + return "%s('%s')" % (self.__class__.__name__, self) + + def to_json(self): + return str(self) diff --git a/python/setup.py b/python/setup.py index 7ac3c3662..d7802fa8e 100644 --- a/python/setup.py +++ b/python/setup.py @@ -88,7 +88,8 @@ setup_args = dict( libraries=['openvswitch'])], cmdclass={'build_ext': try_build_ext}, install_requires=['sortedcontainers'], - extras_require={':sys_platform == "win32"': ['pywin32 >= 1.0']}, + extras_require={':sys_platform == "win32"': ['pywin32 >= 1.0'], + 'flow': ['netaddr']}, ) try: diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch-fedora.spec.in index d0ae78e4e..a40a915f4 100644 --- a/rhel/openvswitch-fedora.spec.in +++ b/rhel/openvswitch-fedora.spec.in @@ -116,6 +116,7 @@ Summary: Open vSwitch python3 bindings License: ASL 2.0 BuildArch: noarch Requires: python3 +Suggests: python3-netaddr %{?python_provide:%python_provide python3-openvswitch = %{version}-%{release}} %description -n python3-openvswitch From patchwork Fri Jul 8 18:03:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654335 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=BMzNBPQD; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh3y5DGrz9ryY for ; Sat, 9 Jul 2022 04:03:58 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id CC4FD61477; Fri, 8 Jul 2022 18:03:55 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org CC4FD61477 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=BMzNBPQD X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cvDD_vYoQr-O; Fri, 8 Jul 2022 18:03:53 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 5859A61454; Fri, 8 Jul 2022 18:03:47 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 5859A61454 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0FBF7C0083; Fri, 8 Jul 2022 18:03:45 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 17929C0085 for ; Fri, 8 Jul 2022 18:03:40 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 630908483F for ; Fri, 8 Jul 2022 18:03:37 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 630908483F Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=BMzNBPQD X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9Pq3np52TDkW for ; Fri, 8 Jul 2022 18:03:35 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 672DB8480C Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 672DB8480C for ; Fri, 8 Jul 2022 18:03:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303414; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mt6KVDG4nWdxV0vEhR3k8JnlJJFv2LP/hQvpGX3Qz3w=; b=BMzNBPQDAvxFQMgHpGXHxEpK70IZJUSNQ+faYxu/OUPDK+ngV/cmZtF4NbbaK8GG026tGn Lf8qycziZQ/ok5MWlMrLmnG99AS5wyf6dw3ZIcm3V5A4u+fQIV9+WnBex63L0qu0G2U/b5 INZq2yzhM85uR9fnyraigmfZBD2hQHw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-544-cVYNX3KjMPiSodf_x_V8qw-1; Fri, 08 Jul 2022 14:03:27 -0400 X-MC-Unique: cVYNX3KjMPiSodf_x_V8qw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C104A8339B4; Fri, 8 Jul 2022 18:03:26 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id B57252026D64; Fri, 8 Jul 2022 18:03:25 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:02 +0200 Message-Id: <20220708180316.2852046-4-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 03/17] python: add list parser X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Some openflow or dpif flows encode their arguments in lists, eg: "some_action(arg1,arg2,arg3)". In order to decode this in a way that can be then stored and queried, add ListParser and ListDecoders classes that parse lists into KeyValue instances. The ListParser / ListDecoders mechanism is quite similar to KVParser and KVDecoders. Since the "key" of the different KeyValue objects is now ommited, it has to be provided by ListDecoders. For example, take the openflow action "resubmit" that can be written as: resubmit([port],[table][,ct]) Can be decoded by creating a ListDecoders instance such as: ListDecoders([ ("port", decode_default), ("table", decode_int), ("ct", decode_flag), ]) Naturally, the order of the decoders must be kept. Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 1 + python/ovs/flow/list.py | 121 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 122 insertions(+) create mode 100644 python/ovs/flow/list.py diff --git a/python/automake.mk b/python/automake.mk index 3a9ec3461..3b03612df 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -29,6 +29,7 @@ ovs_pyfiles = \ python/ovs/flow/__init__.py \ python/ovs/flow/decoders.py \ python/ovs/flow/kv.py \ + python/ovs/flow/list.py \ python/ovs/json.py \ python/ovs/jsonrpc.py \ python/ovs/ovsuuid.py \ diff --git a/python/ovs/flow/list.py b/python/ovs/flow/list.py new file mode 100644 index 000000000..b1e9e3fca --- /dev/null +++ b/python/ovs/flow/list.py @@ -0,0 +1,121 @@ +import re + +from ovs.flow.kv import KeyValue, KeyMetadata, ParseError +from ovs.flow.decoders import decode_default + + +class ListDecoders(object): + """ListDecoders is used by ListParser to decode the elements in the list. + + A decoder is a function that accepts a value and returns its decoded + object. + + ListDecoders is initialized with a list of tuples that contains the + keyword and the decoding function associated with each position in the + list. The order is, therefore, important. + + Args: + decoders (list of tuples): Optional; a list of tuples. + The first element in the tuple is the keyword associated with the + value. The second element in the tuple is the decoder function. + """ + + def __init__(self, decoders=None): + self._decoders = decoders or list() + + def decode(self, index, value_str): + """Decode the index'th element of the list. + + Args: + index (int): The position in the list of the element to decode. + value_str (str): The value string to decode. + """ + if index < 0 or index >= len(self._decoders): + return self._default_decoder(index, value_str) + + try: + key = self._decoders[index][0] + value = self._decoders[index][1](value_str) + return key, value + except Exception as e: + raise ParseError( + "Failed to decode value_str {}: {}".format(value_str, str(e)) + ) + + @staticmethod + def _default_decoder(index, value): + key = "elem_{}".format(index) + return key, decode_default(value) + + +class ListParser(object): + """ListParser parses a list of values and stores them as key-value pairs. + + It uses a ListDecoders instance to decode each element in the list. + + Args: + string (str): The string to parse. + decoders (ListDecoders): Optional, the decoders to use. + delims (list): Optional, list of delimiters of the list. Defaults to + [',']. + """ + def __init__(self, string, decoders=None, delims=[","]): + self._string = string + self._decoders = decoders or ListDecoders() + self._keyval = list() + self._regexp = r"({})".format("|".join(delims)) + + def kv(self): + return self._keyval + + def __iter__(self): + return iter(self._keyval) + + def parse(self): + """Parse the list in string. + + Raises: + ParseError if any parsing error occurs. + """ + kpos = 0 + index = 0 + while kpos < len(self._string) and self._string[kpos] != "\n": + split_parts = re.split(self._regexp, self._string[kpos:], 1) + value_str = split_parts[0] + + key, value = self._decoders.decode(index, value_str) + + meta = KeyMetadata( + kpos=kpos, + vpos=kpos, + kstring=value_str, + vstring=value_str, + ) + self._keyval.append(KeyValue(key, value, meta)) + + kpos += len(value_str) + 1 + index += 1 + + +def decode_nested_list(decoders, value, delims=[","]): + """Decodes a string value that contains a list of elements and returns + them in a dictionary. + + Args: + decoders (ListDecoders): The ListDecoders to use. + value (str): The value string to decode. + delims (list(str)): Optional, the list of delimiters to use. + """ + parser = ListParser(value, decoders, delims) + parser.parse() + return {kv.key: kv.value for kv in parser.kv()} + + +def nested_list_decoder(decoders=None, delims=[","]): + """Helper function that creates a nested list decoder with given + ListDecoders and delimiters. + """ + def decoder(value): + return decode_nested_list(decoders, value, delims) + + return decoder From patchwork Fri Jul 8 18:03:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654334 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=M6yrwIKq; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh3v6QPdz9ryY for ; Sat, 9 Jul 2022 04:03:55 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 6C7AA614A2; Fri, 8 Jul 2022 18:03:52 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6C7AA614A2 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=M6yrwIKq X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q5HJgNWeaF5N; Fri, 8 Jul 2022 18:03:44 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id B576861471; Fri, 8 Jul 2022 18:03:42 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org B576861471 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 64CE5C0035; Fri, 8 Jul 2022 18:03:42 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 03CF9C0078 for ; Fri, 8 Jul 2022 18:03:39 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 5B601425F5 for ; Fri, 8 Jul 2022 18:03:37 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 5B601425F5 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=M6yrwIKq X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cR52udjVu_1D for ; Fri, 8 Jul 2022 18:03:33 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org C26B6425F7 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id C26B6425F7 for ; Fri, 8 Jul 2022 18:03:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303411; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mpoZl78cLDeQbQ2Q0nva56GPKZgD4fcB/l9e0MJxunA=; b=M6yrwIKqsgSI5V9EJJZ7xN09JTGg4p2PW/Dc7fPU5zrBrfqfUUBsQQ01vHQr5NV5ocH3Jt h5gAWzDj3tx+1EiPvjYBvJAkTc7QSuJT23gQPDzJkD/ionTwSiPkHZlbrHfdawJgiaNE87 D1ImdLdS1j/i5cVZZQEVnjcV5p/5fR8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-217-PBP44Y-4Mk2syUgUHrsHNA-1; Fri, 08 Jul 2022 14:03:28 -0400 X-MC-Unique: PBP44Y-4Mk2syUgUHrsHNA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0D09F80A0B9; Fri, 8 Jul 2022 18:03:28 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0C2552026D64; Fri, 8 Jul 2022 18:03:26 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:03 +0200 Message-Id: <20220708180316.2852046-5-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 04/17] build-aux: split extract-ofp-fields X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" In order to be able to reuse the core extraction logic, split the command in two parts. The core extraction logic is moved to python/build while the command that writes the different files out of the extracted field info is kept in build-aux. Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- build-aux/extract-ofp-fields | 706 ++++++++--------------------- python/automake.mk | 7 +- python/build/extract_ofp_fields.py | 421 +++++++++++++++++ 3 files changed, 618 insertions(+), 516 deletions(-) create mode 100644 python/build/extract_ofp_fields.py diff --git a/build-aux/extract-ofp-fields b/build-aux/extract-ofp-fields index 8766995d9..efec59c25 100755 --- a/build-aux/extract-ofp-fields +++ b/build-aux/extract-ofp-fields @@ -3,85 +3,23 @@ import getopt import sys import os.path -import re import xml.dom.minidom import build.nroff -line = "" - -# Maps from user-friendly version number to its protocol encoding. -VERSION = {"1.0": 0x01, - "1.1": 0x02, - "1.2": 0x03, - "1.3": 0x04, - "1.4": 0x05, - "1.5": 0x06} -VERSION_REVERSE = dict((v,k) for k, v in VERSION.items()) - -TYPES = {"u8": (1, False), - "be16": (2, False), - "be32": (4, False), - "MAC": (6, False), - "be64": (8, False), - "be128": (16, False), - "tunnelMD": (124, True)} - -FORMATTING = {"decimal": ("MFS_DECIMAL", 1, 8), - "hexadecimal": ("MFS_HEXADECIMAL", 1, 127), - "ct state": ("MFS_CT_STATE", 4, 4), - "Ethernet": ("MFS_ETHERNET", 6, 6), - "IPv4": ("MFS_IPV4", 4, 4), - "IPv6": ("MFS_IPV6", 16, 16), - "OpenFlow 1.0 port": ("MFS_OFP_PORT", 2, 2), - "OpenFlow 1.1+ port": ("MFS_OFP_PORT_OXM", 4, 4), - "frag": ("MFS_FRAG", 1, 1), - "tunnel flags": ("MFS_TNL_FLAGS", 2, 2), - "TCP flags": ("MFS_TCP_FLAGS", 2, 2), - "packet type": ("MFS_PACKET_TYPE", 4, 4)} - -PREREQS = {"none": "MFP_NONE", - "Ethernet": "MFP_ETHERNET", - "ARP": "MFP_ARP", - "VLAN VID": "MFP_VLAN_VID", - "IPv4": "MFP_IPV4", - "IPv6": "MFP_IPV6", - "IPv4/IPv6": "MFP_IP_ANY", - "NSH": "MFP_NSH", - "CT": "MFP_CT_VALID", - "MPLS": "MFP_MPLS", - "TCP": "MFP_TCP", - "UDP": "MFP_UDP", - "SCTP": "MFP_SCTP", - "ICMPv4": "MFP_ICMPV4", - "ICMPv6": "MFP_ICMPV6", - "ND": "MFP_ND", - "ND solicit": "MFP_ND_SOLICIT", - "ND advert": "MFP_ND_ADVERT"} - -# Maps a name prefix into an (experimenter ID, class) pair, so: -# -# - Standard OXM classes are written as (0, ) -# -# - Experimenter OXM classes are written as (, 0xffff) -# -# If a name matches more than one prefix, the longest one is used. -OXM_CLASSES = {"NXM_OF_": (0, 0x0000, 'extension'), - "NXM_NX_": (0, 0x0001, 'extension'), - "NXOXM_NSH_": (0x005ad650, 0xffff, 'extension'), - "OXM_OF_": (0, 0x8000, 'standard'), - "OXM_OF_PKT_REG": (0, 0x8001, 'standard'), - "ONFOXM_ET_": (0x4f4e4600, 0xffff, 'standard'), - "ERICOXM_OF_": (0, 0x1000, 'extension'), - - # This is the experimenter OXM class for Nicira, which is the - # one that OVS would be using instead of NXM_OF_ and NXM_NX_ - # if OVS didn't have those grandfathered in. It is currently - # used only to test support for experimenter OXM, since there - # are barely any real uses of experimenter OXM in the wild. - "NXOXM_ET_": (0x00002320, 0xffff, 'extension')} +from build.extract_ofp_fields import ( + extract_ofp_fields, + PREREQS, + OXM_CLASSES, + VERSION, + fatal, + n_errors, +) + +VERSION_REVERSE = dict((v, k) for k, v in VERSION.items()) + def oxm_name_to_class(name): - prefix = '' + prefix = "" class_ = None for p, c in OXM_CLASSES.items(): if name.startswith(p) and len(p) > len(prefix): @@ -92,267 +30,76 @@ def oxm_name_to_class(name): def is_standard_oxm(name): oxm_vendor, oxm_class, oxm_class_type = oxm_name_to_class(name) - return oxm_class_type == 'standard' - - -def decode_version_range(range): - if range in VERSION: - return (VERSION[range], VERSION[range]) - elif range.endswith('+'): - return (VERSION[range[:-1]], max(VERSION.values())) - else: - a, b = re.match(r'^([^-]+)-([^-]+)$', range).groups() - return (VERSION[a], VERSION[b]) - - -def get_line(): - global line - global line_number - line = input_file.readline() - line_number += 1 - if line == "": - fatal("unexpected end of input") - - -n_errors = 0 - - -def error(msg): - global n_errors - sys.stderr.write("%s:%d: %s\n" % (file_name, line_number, msg)) - n_errors += 1 - - -def fatal(msg): - error(msg) - sys.exit(1) + return oxm_class_type == "standard" def usage(): argv0 = os.path.basename(sys.argv[0]) - print('''\ + print( + """\ %(argv0)s, for extracting OpenFlow field properties from meta-flow.h usage: %(argv0)s INPUT [--meta-flow | --nx-match] where INPUT points to lib/meta-flow.h in the source directory. Depending on the option given, the output written to stdout is intended to be saved either as lib/meta-flow.inc or lib/nx-match.inc for the respective C file to #include.\ -''' % {"argv0": argv0}) +""" + % {"argv0": argv0} + ) sys.exit(0) -def make_sizeof(s): - m = re.match(r'(.*) up to (.*)', s) - if m: - struct, member = m.groups() - return "offsetof(%s, %s)" % (struct, member) - else: - return "sizeof(%s)" % s - - -def parse_oxms(s, prefix, n_bytes): - if s == 'none': - return () - - return tuple(parse_oxm(s2.strip(), prefix, n_bytes) for s2 in s.split(',')) - - -match_types = dict() - - -def parse_oxm(s, prefix, n_bytes): - global match_types - - m = re.match('([A-Z0-9_]+)\(([0-9]+)\) since(?: OF(1\.[0-9]+) and)? v([12]\.[0-9]+)$', s) - if not m: - fatal("%s: syntax error parsing %s" % (s, prefix)) - - name, oxm_type, of_version, ovs_version = m.groups() - - class_ = oxm_name_to_class(name) - if class_ is None: - fatal("unknown OXM class for %s" % name) - oxm_vendor, oxm_class, oxm_class_type = class_ - - if class_ in match_types: - if oxm_type in match_types[class_]: - fatal("duplicate match type for %s (conflicts with %s)" % - (name, match_types[class_][oxm_type])) - else: - match_types[class_] = dict() - match_types[class_][oxm_type] = name - - # Normally the oxm_length is the size of the field, but for experimenter - # OXMs oxm_length also includes the 4-byte experimenter ID. - oxm_length = n_bytes - if oxm_class == 0xffff: - oxm_length += 4 - - header = (oxm_vendor, oxm_class, int(oxm_type), oxm_length) - - if of_version: - if oxm_class_type == 'extension': - fatal("%s: OXM extension can't have OpenFlow version" % name) - if of_version not in VERSION: - fatal("%s: unknown OpenFlow version %s" % (name, of_version)) - of_version_nr = VERSION[of_version] - if of_version_nr < VERSION['1.2']: - fatal("%s: claimed version %s predates OXM" % (name, of_version)) - else: - if oxm_class_type == 'standard': - fatal("%s: missing OpenFlow version number" % name) - of_version_nr = 0 - - return (header, name, of_version_nr, ovs_version) - - -def parse_field(mff, comment): - f = {'mff': mff} - - # First line of comment is the field name. - m = re.match(r'"([^"]+)"(?:\s+\(aka "([^"]+)"\))?(?:\s+\(.*\))?\.', comment[0]) - if not m: - fatal("%s lacks field name" % mff) - f['name'], f['extra_name'] = m.groups() - - # Find the last blank line the comment. The field definitions - # start after that. - blank = None - for i in range(len(comment)): - if not comment[i]: - blank = i - if not blank: - fatal("%s: missing blank line in comment" % mff) - - d = {} - for key in ("Type", "Maskable", "Formatting", "Prerequisites", - "Access", "Prefix lookup member", - "OXM", "NXM", "OF1.0", "OF1.1"): - d[key] = None - for fline in comment[blank + 1:]: - m = re.match(r'([^:]+):\s+(.*)\.$', fline) - if not m: - fatal("%s: syntax error parsing key-value pair as part of %s" - % (fline, mff)) - key, value = m.groups() - if key not in d: - fatal("%s: unknown key" % key) - elif key == 'Code point': - d[key] += [value] - elif d[key] is not None: - fatal("%s: duplicate key" % key) - d[key] = value - for key, value in d.items(): - if not value and key not in ("OF1.0", "OF1.1", - "Prefix lookup member", "Notes"): - fatal("%s: missing %s" % (mff, key)) - - m = re.match(r'([a-zA-Z0-9]+)(?: \(low ([0-9]+) bits\))?$', d['Type']) - if not m: - fatal("%s: syntax error in type" % mff) - type_ = m.group(1) - if type_ not in TYPES: - fatal("%s: unknown type %s" % (mff, d['Type'])) - - f['n_bytes'] = TYPES[type_][0] - if m.group(2): - f['n_bits'] = int(m.group(2)) - if f['n_bits'] > f['n_bytes'] * 8: - fatal("%s: more bits (%d) than field size (%d)" - % (mff, f['n_bits'], 8 * f['n_bytes'])) - else: - f['n_bits'] = 8 * f['n_bytes'] - f['variable'] = TYPES[type_][1] - - if d['Maskable'] == 'no': - f['mask'] = 'MFM_NONE' - elif d['Maskable'] == 'bitwise': - f['mask'] = 'MFM_FULLY' - else: - fatal("%s: unknown maskable %s" % (mff, d['Maskable'])) - - fmt = FORMATTING.get(d['Formatting']) - if not fmt: - fatal("%s: unknown format %s" % (mff, d['Formatting'])) - f['formatting'] = d['Formatting'] - if f['n_bytes'] < fmt[1] or f['n_bytes'] > fmt[2]: - fatal("%s: %d-byte field can't be formatted as %s" - % (mff, f['n_bytes'], d['Formatting'])) - f['string'] = fmt[0] - - f['prereqs'] = d['Prerequisites'] - if f['prereqs'] not in PREREQS: - fatal("%s: unknown prerequisites %s" % (mff, d['Prerequisites'])) - - if d['Access'] == 'read-only': - f['writable'] = False - elif d['Access'] == 'read/write': - f['writable'] = True - else: - fatal("%s: unknown access %s" % (mff, d['Access'])) - - f['OF1.0'] = d['OF1.0'] - if not d['OF1.0'] in (None, 'exact match', 'CIDR mask'): - fatal("%s: unknown OF1.0 match type %s" % (mff, d['OF1.0'])) - - f['OF1.1'] = d['OF1.1'] - if not d['OF1.1'] in (None, 'exact match', 'bitwise mask'): - fatal("%s: unknown OF1.1 match type %s" % (mff, d['OF1.1'])) - - f['OXM'] = (parse_oxms(d['OXM'], 'OXM', f['n_bytes']) + - parse_oxms(d['NXM'], 'NXM', f['n_bytes'])) - - f['prefix'] = d["Prefix lookup member"] - - return f - - def protocols_to_c(protocols): - if protocols == set(['of10', 'of11', 'oxm']): - return 'OFPUTIL_P_ANY' - elif protocols == set(['of11', 'oxm']): - return 'OFPUTIL_P_NXM_OF11_UP' - elif protocols == set(['oxm']): - return 'OFPUTIL_P_NXM_OXM_ANY' + if protocols == set(["of10", "of11", "oxm"]): + return "OFPUTIL_P_ANY" + elif protocols == set(["of11", "oxm"]): + return "OFPUTIL_P_NXM_OF11_UP" + elif protocols == set(["oxm"]): + return "OFPUTIL_P_NXM_OXM_ANY" elif protocols == set([]): - return 'OFPUTIL_P_NONE' + return "OFPUTIL_P_NONE" else: assert False def autogen_c_comment(): return [ -"/* Generated automatically; do not modify! -*- buffer-read-only: t -*- */", -""] + "/* Generated automatically; do not modify! " + "-*- buffer-read-only: t -*- */", + "", + ] + def make_meta_flow(meta_flow_h): fields = extract_ofp_fields(meta_flow_h) output = autogen_c_comment() for f in fields: output += ["{"] - output += [" %s," % f['mff']] - if f['extra_name']: - output += [" \"%s\", \"%s\"," % (f['name'], f['extra_name'])] + output += [" %s," % f["mff"]] + if f["extra_name"]: + output += [' "%s", "%s",' % (f["name"], f["extra_name"])] else: - output += [" \"%s\", NULL," % f['name']] + output += [' "%s", NULL,' % f["name"]] - if f['variable']: - variable = 'true' + if f["variable"]: + variable = "true" else: - variable = 'false' - output += [" %d, %d, %s," % (f['n_bytes'], f['n_bits'], variable)] + variable = "false" + output += [" %d, %d, %s," % (f["n_bytes"], f["n_bits"], variable)] - if f['writable']: - rw = 'true' + if f["writable"]: + rw = "true" else: - rw = 'false' - output += [" %s, %s, %s, %s, false," - % (f['mask'], f['string'], PREREQS[f['prereqs']], rw)] - - oxm = f['OXM'] - of10 = f['OF1.0'] - of11 = f['OF1.1'] - if f['mff'] in ('MFF_DL_VLAN', 'MFF_DL_VLAN_PCP'): + rw = "false" + output += [ + " %s, %s, %s, %s, false," + % (f["mask"], f["string"], PREREQS[f["prereqs"]], rw) + ] + + oxm = f["OXM"] + of10 = f["OF1.0"] + of11 = f["OF1.1"] + if f["mff"] in ("MFF_DL_VLAN", "MFF_DL_VLAN_PCP"): # MFF_DL_VLAN and MFF_DL_VLAN_PCP don't exactly correspond to # OF1.1, nor do they have NXM or OXM assignments, but their # meanings can be expressed in every protocol, which is the goal of @@ -367,25 +114,25 @@ def make_meta_flow(meta_flow_h): if oxm: protocols |= set(["oxm"]) - if f['mask'] == 'MFM_FULLY': + if f["mask"] == "MFM_FULLY": cidr_protocols = protocols.copy() bitwise_protocols = protocols.copy() - if of10 == 'exact match': - bitwise_protocols -= set(['of10']) - cidr_protocols -= set(['of10']) - elif of10 == 'CIDR mask': - bitwise_protocols -= set(['of10']) + if of10 == "exact match": + bitwise_protocols -= set(["of10"]) + cidr_protocols -= set(["of10"]) + elif of10 == "CIDR mask": + bitwise_protocols -= set(["of10"]) else: assert of10 is None - if of11 == 'exact match': - bitwise_protocols -= set(['of11']) - cidr_protocols -= set(['of11']) + if of11 == "exact match": + bitwise_protocols -= set(["of11"]) + cidr_protocols -= set(["of11"]) else: - assert of11 in (None, 'bitwise mask') + assert of11 in (None, "bitwise mask") else: - assert f['mask'] == 'MFM_NONE' + assert f["mask"] == "MFM_NONE" cidr_protocols = set([]) bitwise_protocols = set([]) @@ -393,8 +140,8 @@ def make_meta_flow(meta_flow_h): output += [" %s," % protocols_to_c(cidr_protocols)] output += [" %s," % protocols_to_c(bitwise_protocols)] - if f['prefix']: - output += [" FLOW_U32OFS(%s)," % f['prefix']] + if f["prefix"]: + output += [" FLOW_U32OFS(%s)," % f["prefix"]] else: output += [" -1, /* not usable for prefix lookup */"] @@ -409,147 +156,37 @@ def make_nx_match(meta_flow_h): print("static struct nxm_field_index all_nxm_fields[] = {") for f in fields: # Sort by OpenFlow version number (nx-match.c depends on this). - for oxm in sorted(f['OXM'], key=lambda x: x[2]): - header = ("NXM_HEADER(0x%x,0x%x,%s,0,%d)" % oxm[0]) - print("""{ .nf = { %s, %d, "%s", %s } },""" % ( - header, oxm[2], oxm[1], f['mff'])) + for oxm in sorted(f["OXM"], key=lambda x: x[2]): + header = "NXM_HEADER(0x%x,0x%x,%s,0,%d)" % oxm[0] + print( + """{ .nf = { %s, %d, "%s", %s } },""" + % (header, oxm[2], oxm[1], f["mff"]) + ) print("};") for oline in output: print(oline) -def extract_ofp_fields(fn): - global file_name - global input_file - global line_number - global line - - file_name = fn - input_file = open(file_name) - line_number = 0 - - fields = [] - - while True: - get_line() - if re.match('enum.*mf_field_id', line): - break - - while True: - get_line() - first_line_number = line_number - here = '%s:%d' % (file_name, line_number) - if (line.startswith('/*') - or line.startswith(' *') - or line.startswith('#') - or not line - or line.isspace()): - continue - elif re.match('}', line) or re.match('\s+MFF_N_IDS', line): - break - - # Parse the comment preceding an MFF_ constant into 'comment', - # one line to an array element. - line = line.strip() - if not line.startswith('/*'): - fatal("unexpected syntax between fields") - line = line[1:] - comment = [] - end = False - while not end: - line = line.strip() - if line.startswith('*/'): - get_line() - break - if not line.startswith('*'): - fatal("unexpected syntax within field") - - line = line[1:] - if line.startswith(' '): - line = line[1:] - if line.startswith(' ') and comment: - continuation = True - line = line.lstrip() - else: - continuation = False - - if line.endswith('*/'): - line = line[:-2].rstrip() - end = True - else: - end = False - - if continuation: - comment[-1] += " " + line - else: - comment += [line] - get_line() - - # Drop blank lines at each end of comment. - while comment and not comment[0]: - comment = comment[1:] - while comment and not comment[-1]: - comment = comment[:-1] - - # Parse the MFF_ constant(s). - mffs = [] - while True: - m = re.match('\s+(MFF_[A-Z0-9_]+),?\s?$', line) - if not m: - break - mffs += [m.group(1)] - get_line() - if not mffs: - fatal("unexpected syntax looking for MFF_ constants") - - if len(mffs) > 1 or '' in comment[0]: - for mff in mffs: - # Extract trailing integer. - m = re.match('.*[^0-9]([0-9]+)$', mff) - if not m: - fatal("%s lacks numeric suffix in register group" % mff) - n = m.group(1) - - # Search-and-replace within the comment, - # and drop lines that have for x != n. - instance = [] - for x in comment: - y = x.replace('', n) - if re.search('<[0-9]+>', y): - if ('<%s>' % n) not in y: - continue - y = re.sub('<[0-9]+>', '', y) - instance += [y.strip()] - fields += [parse_field(mff, instance)] - else: - fields += [parse_field(mffs[0], comment)] - continue - - input_file.close() - - if n_errors: - sys.exit(1) - - return fields - ## ------------------------ ## ## Documentation Generation ## ## ------------------------ ## + def field_to_xml(field_node, f, body, summary): f["used"] = True # Summary. - if field_node.hasAttribute('internal'): + if field_node.hasAttribute("internal"): return min_of_version = None min_ovs_version = None - for header, name, of_version_nr, ovs_version_s in f['OXM']: - if (is_standard_oxm(name) - and (min_ovs_version is None or of_version_nr < min_of_version)): + for header, name, of_version_nr, ovs_version_s in f["OXM"]: + if is_standard_oxm(name) and ( + min_ovs_version is None or of_version_nr < min_of_version + ): min_of_version = of_version_nr - ovs_version = [int(x) for x in ovs_version_s.split('.')] + ovs_version = [int(x) for x in ovs_version_s.split(".")] if min_ovs_version is None or ovs_version < min_ovs_version: min_ovs_version = ovs_version summary += ["\\fB%s\\fR" % f["name"]] @@ -565,124 +202,152 @@ def field_to_xml(field_node, f, body, summary): if min_of_version is not None: support += ["OF %s+" % VERSION_REVERSE[min_of_version]] if min_ovs_version is not None: - support += ["OVS %s+" % '.'.join([str(x) for x in min_ovs_version])] - summary += ' and '.join(support) + support += ["OVS %s+" % ".".join([str(x) for x in min_ovs_version])] + summary += " and ".join(support) summary += ["\n"] # Full description. - if field_node.hasAttribute('hidden'): + if field_node.hasAttribute("hidden"): return - title = field_node.attributes['title'].nodeValue + title = field_node.attributes["title"].nodeValue - body += [""".PP + body += [ + """.PP \\fB%s Field\\fR .TS tab(;); l lx. -""" % title] +""" + % title + ] body += ["Name:;\\fB%s\\fR" % f["name"]] if f["extra_name"]: body += [" (aka \\fB%s\\fR)" % f["extra_name"]] - body += ['\n'] + body += ["\n"] body += ["Width:;"] if f["n_bits"] != 8 * f["n_bytes"]: - body += ["%d bits (only the least-significant %d bits " - "may be nonzero)" % (f["n_bytes"] * 8, f["n_bits"])] + body += [ + "%d bits (only the least-significant %d bits " + "may be nonzero)" % (f["n_bytes"] * 8, f["n_bits"]) + ] elif f["n_bits"] <= 128: body += ["%d bits" % f["n_bits"]] else: body += ["%d bits (%d bytes)" % (f["n_bits"], f["n_bits"] / 8)] - body += ['\n'] + body += ["\n"] body += ["Format:;%s\n" % f["formatting"]] - masks = {"MFM_NONE": "not maskable", - "MFM_FULLY": "arbitrary bitwise masks"} + masks = { + "MFM_NONE": "not maskable", + "MFM_FULLY": "arbitrary bitwise masks", + } body += ["Masking:;%s\n" % masks[f["mask"]]] body += ["Prerequisites:;%s\n" % f["prereqs"]] - access = {True: "read/write", - False: "read-only"}[f["writable"]] + access = {True: "read/write", False: "read-only"}[f["writable"]] body += ["Access:;%s\n" % access] - of10 = {None: "not supported", - "exact match": "yes (exact match only)", - "CIDR mask": "yes (CIDR match only)"} + of10 = { + None: "not supported", + "exact match": "yes (exact match only)", + "CIDR mask": "yes (CIDR match only)", + } body += ["OpenFlow 1.0:;%s\n" % of10[f["OF1.0"]]] - of11 = {None: "not supported", - "exact match": "yes (exact match only)", - "bitwise mask": "yes"} + of11 = { + None: "not supported", + "exact match": "yes (exact match only)", + "bitwise mask": "yes", + } body += ["OpenFlow 1.1:;%s\n" % of11[f["OF1.1"]]] oxms = [] - for header, name, of_version_nr, ovs_version in [x for x in sorted(f['OXM'], key=lambda x: x[2]) if is_standard_oxm(x[1])]: + for header, name, of_version_nr, ovs_version in [ + x + for x in sorted(f["OXM"], key=lambda x: x[2]) + if is_standard_oxm(x[1]) + ]: of_version = VERSION_REVERSE[of_version_nr] - oxms += [r"\fB%s\fR (%d) since OpenFlow %s and Open vSwitch %s" % (name, header[2], of_version, ovs_version)] + oxms += [ + r"\fB%s\fR (%d) since OpenFlow %s and Open vSwitch %s" + % (name, header[2], of_version, ovs_version) + ] if not oxms: - oxms = ['none'] - body += ['OXM:;T{\n%s\nT}\n' % r'\[char59] '.join(oxms)] + oxms = ["none"] + body += ["OXM:;T{\n%s\nT}\n" % r"\[char59] ".join(oxms)] nxms = [] - for header, name, of_version_nr, ovs_version in [x for x in sorted(f['OXM'], key=lambda x: x[2]) if not is_standard_oxm(x[1])]: - nxms += [r"\fB%s\fR (%d) since Open vSwitch %s" % (name, header[2], ovs_version)] + for header, name, of_version_nr, ovs_version in [ + x + for x in sorted(f["OXM"], key=lambda x: x[2]) + if not is_standard_oxm(x[1]) + ]: + nxms += [ + r"\fB%s\fR (%d) since Open vSwitch %s" + % (name, header[2], ovs_version) + ] if not nxms: - nxms = ['none'] - body += ['NXM:;T{\n%s\nT}\n' % r'\[char59] '.join(nxms)] + nxms = ["none"] + body += ["NXM:;T{\n%s\nT}\n" % r"\[char59] ".join(nxms)] body += [".TE\n"] - body += ['.PP\n'] + body += [".PP\n"] body += [build.nroff.block_xml_to_nroff(field_node.childNodes)] + def group_xml_to_nroff(group_node, fields): - title = group_node.attributes['title'].nodeValue + title = group_node.attributes["title"].nodeValue summary = [] body = [] for node in group_node.childNodes: - if node.nodeType == node.ELEMENT_NODE and node.tagName == 'field': - id_ = node.attributes['id'].nodeValue + if node.nodeType == node.ELEMENT_NODE and node.tagName == "field": + id_ = node.attributes["id"].nodeValue field_to_xml(node, fields[id_], body, summary) else: body += [build.nroff.block_xml_to_nroff([node])] content = [ - '.bp\n', - '.SH \"%s\"\n' % build.nroff.text_to_nroff(title.upper() + " FIELDS"), + ".bp\n", + '.SH "%s"\n' % build.nroff.text_to_nroff(title.upper() + " FIELDS"), '.SS "Summary:"\n', - '.TS\n', - 'tab(;);\n', - 'l l l l l l l.\n', - 'Name;Bytes;Mask;RW?;Prereqs;NXM/OXM Support\n', - '\_;\_;\_;\_;\_;\_\n'] + ".TS\n", + "tab(;);\n", + "l l l l l l l.\n", + "Name;Bytes;Mask;RW?;Prereqs;NXM/OXM Support\n", + "\_;\_;\_;\_;\_;\_\n", + ] content += summary - content += ['.TE\n'] + content += [".TE\n"] content += body - return ''.join(content) + return "".join(content) + def make_oxm_classes_xml(document): - s = '''tab(;); + s = """tab(;); l l l. Prefix;Vendor;Class \_;\_;\_ -''' +""" for key in sorted(OXM_CLASSES, key=OXM_CLASSES.get): vendor, class_, class_type = OXM_CLASSES.get(key) - s += r"\fB%s\fR;" % key.rstrip('_') + s += r"\fB%s\fR;" % key.rstrip("_") if vendor: s += r"\fL0x%08x\fR;" % vendor else: s += "(none);" s += r"\fL0x%04x\fR;" % class_ s += "\n" - e = document.createElement('tbl') + e = document.createElement("tbl") e.appendChild(document.createTextNode(s)) return e + def recursively_replace(node, name, replacement): for child in node.childNodes: if child.nodeType == node.ELEMENT_NODE: @@ -691,11 +356,12 @@ def recursively_replace(node, name, replacement): else: recursively_replace(child, name, replacement) + def make_ovs_fields(meta_flow_h, meta_flow_xml): fields = extract_ofp_fields(meta_flow_h) fields_map = {} for f in fields: - fields_map[f['mff']] = f + fields_map[f["mff"]] = f document = xml.dom.minidom.parse(meta_flow_xml) doc = document.documentElement @@ -704,7 +370,8 @@ def make_ovs_fields(meta_flow_h, meta_flow_xml): if version == None: version = "UNKNOWN" - print('''\ + print( + """\ '\\" tp .\\" -*- mode: troff; coding: utf-8 -*- .TH "ovs\-fields" 7 "%s" "Open vSwitch" "Open vSwitch Manual" @@ -740,11 +407,13 @@ def make_ovs_fields(meta_flow_h, meta_flow_xml): ovs\-fields \- protocol header fields in OpenFlow and Open vSwitch . .PP -''' % version) +""" + % version + ) - recursively_replace(doc, 'oxm_classes', make_oxm_classes_xml(document)) + recursively_replace(doc, "oxm_classes", make_oxm_classes_xml(document)) - s = '' + s = "" for node in doc.childNodes: if node.nodeType == node.ELEMENT_NODE and node.tagName == "group": s += group_xml_to_nroff(node, fields_map) @@ -757,9 +426,10 @@ ovs\-fields \- protocol header fields in OpenFlow and Open vSwitch for f in fields: if "used" not in f: - fatal("%s: field not documented " - "(please add documentation in lib/meta-flow.xml)" - % f["mff"]) + fatal( + "%s: field not documented " + "(please add documentation in lib/meta-flow.xml)" % f["mff"] + ) if n_errors: sys.exit(1) @@ -769,26 +439,27 @@ ovs\-fields \- protocol header fields in OpenFlow and Open vSwitch # Life is easier with nroff if we don't try to feed it Unicode. # Fortunately, we only use a few characters outside the ASCII range. - oline = oline.replace(u'\u2208', r'\[mo]') - oline = oline.replace(u'\u2260', r'\[!=]') - oline = oline.replace(u'\u2264', r'\[<=]') - oline = oline.replace(u'\u2265', r'\[>=]') - oline = oline.replace(u'\u00d7', r'\[mu]') + oline = oline.replace(u"\u2208", r"\[mo]") + oline = oline.replace(u"\u2260", r"\[!=]") + oline = oline.replace(u"\u2264", r"\[<=]") + oline = oline.replace(u"\u2265", r"\[>=]") + oline = oline.replace(u"\u00d7", r"\[mu]") if len(oline): output += [oline] # nroff tends to ignore .bp requests if they come after .PP requests, # so remove .PPs that precede .bp. for i in range(len(output)): - if output[i] == '.bp': + if output[i] == ".bp": j = i - 1 - while j >= 0 and output[j] == '.PP': + while j >= 0 and output[j] == ".PP": output[j] = None j -= 1 for i in range(len(output)): if output[i] is not None: print(output[i]) - + + ## ------------ ## ## Main Program ## ## ------------ ## @@ -796,8 +467,9 @@ ovs\-fields \- protocol header fields in OpenFlow and Open vSwitch if __name__ == "__main__": argv0 = sys.argv[0] try: - options, args = getopt.gnu_getopt(sys.argv[1:], 'h', - ['help', 'ovs-version=']) + options, args = getopt.gnu_getopt( + sys.argv[1:], "h", ["help", "ovs-version="] + ) except getopt.GetoptError as geo: sys.stderr.write("%s: %s\n" % (argv0, geo.msg)) sys.exit(1) @@ -805,32 +477,38 @@ if __name__ == "__main__": global version version = None for key, value in options: - if key in ['-h', '--help']: + if key in ["-h", "--help"]: usage() - elif key == '--ovs-version': + elif key == "--ovs-version": version = value else: sys.exit(0) if not args: - sys.stderr.write("%s: missing command argument " - "(use --help for help)\n" % argv0) + sys.stderr.write( + "%s: missing command argument " "(use --help for help)\n" % argv0 + ) sys.exit(1) - commands = {"meta-flow": (make_meta_flow, 1), - "nx-match": (make_nx_match, 1), - "ovs-fields": (make_ovs_fields, 2)} + commands = { + "meta-flow": (make_meta_flow, 1), + "nx-match": (make_nx_match, 1), + "ovs-fields": (make_ovs_fields, 2), + } if not args[0] in commands: - sys.stderr.write("%s: unknown command \"%s\" " - "(use --help for help)\n" % (argv0, args[0])) + sys.stderr.write( + '%s: unknown command "%s" ' + "(use --help for help)\n" % (argv0, args[0]) + ) sys.exit(1) func, n_args = commands[args[0]] if len(args) - 1 != n_args: - sys.stderr.write("%s: \"%s\" requires %d arguments but %d " - "provided\n" - % (argv0, args[0], n_args, len(args) - 1)) + sys.stderr.write( + '%s: "%s" requires %d arguments but %d ' + "provided\n" % (argv0, args[0], n_args, len(args) - 1) + ) sys.exit(1) func(*args[1:]) diff --git a/python/automake.mk b/python/automake.mk index 3b03612df..d11a7ade1 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -51,6 +51,7 @@ ovs_pyfiles = \ # so they are not installed. EXTRA_DIST += \ python/build/__init__.py \ + python/build/extract_ofp_fields.py \ python/build/nroff.py \ python/build/soutil.py @@ -69,10 +70,12 @@ PYCOV_CLEAN_FILES += $(PYFILES:.py=.py,cover) FLAKE8_PYFILES += \ $(filter-out python/ovs/compat/% python/ovs/dirs.py,$(PYFILES)) \ - python/setup.py \ python/build/__init__.py \ + python/build/extract_ofp_fields.py \ python/build/nroff.py \ - python/ovs/dirs.py.template + python/build/soutil.py \ + python/ovs/dirs.py.template \ + python/setup.py nobase_pkgdata_DATA = $(ovs_pyfiles) $(ovstest_pyfiles) ovs-install-data-local: diff --git a/python/build/extract_ofp_fields.py b/python/build/extract_ofp_fields.py new file mode 100644 index 000000000..3fe62634a --- /dev/null +++ b/python/build/extract_ofp_fields.py @@ -0,0 +1,421 @@ +import sys +import re + +line = "" + +# Maps from user-friendly version number to its protocol encoding. +VERSION = { + "1.0": 0x01, + "1.1": 0x02, + "1.2": 0x03, + "1.3": 0x04, + "1.4": 0x05, + "1.5": 0x06, +} +VERSION_REVERSE = dict((v, k) for k, v in VERSION.items()) + +TYPES = { + "u8": (1, False), + "be16": (2, False), + "be32": (4, False), + "MAC": (6, False), + "be64": (8, False), + "be128": (16, False), + "tunnelMD": (124, True), +} + +FORMATTING = { + "decimal": ("MFS_DECIMAL", 1, 8), + "hexadecimal": ("MFS_HEXADECIMAL", 1, 127), + "ct state": ("MFS_CT_STATE", 4, 4), + "Ethernet": ("MFS_ETHERNET", 6, 6), + "IPv4": ("MFS_IPV4", 4, 4), + "IPv6": ("MFS_IPV6", 16, 16), + "OpenFlow 1.0 port": ("MFS_OFP_PORT", 2, 2), + "OpenFlow 1.1+ port": ("MFS_OFP_PORT_OXM", 4, 4), + "frag": ("MFS_FRAG", 1, 1), + "tunnel flags": ("MFS_TNL_FLAGS", 2, 2), + "TCP flags": ("MFS_TCP_FLAGS", 2, 2), + "packet type": ("MFS_PACKET_TYPE", 4, 4), +} + +PREREQS = { + "none": "MFP_NONE", + "Ethernet": "MFP_ETHERNET", + "ARP": "MFP_ARP", + "VLAN VID": "MFP_VLAN_VID", + "IPv4": "MFP_IPV4", + "IPv6": "MFP_IPV6", + "IPv4/IPv6": "MFP_IP_ANY", + "NSH": "MFP_NSH", + "CT": "MFP_CT_VALID", + "MPLS": "MFP_MPLS", + "TCP": "MFP_TCP", + "UDP": "MFP_UDP", + "SCTP": "MFP_SCTP", + "ICMPv4": "MFP_ICMPV4", + "ICMPv6": "MFP_ICMPV6", + "ND": "MFP_ND", + "ND solicit": "MFP_ND_SOLICIT", + "ND advert": "MFP_ND_ADVERT", +} + +# Maps a name prefix into an (experimenter ID, class) pair, so: +# +# - Standard OXM classes are written as (0, ) +# +# - Experimenter OXM classes are written as (, 0xffff) +# +# If a name matches more than one prefix, the longest one is used. +OXM_CLASSES = { + "NXM_OF_": (0, 0x0000, "extension"), + "NXM_NX_": (0, 0x0001, "extension"), + "NXOXM_NSH_": (0x005AD650, 0xFFFF, "extension"), + "OXM_OF_": (0, 0x8000, "standard"), + "OXM_OF_PKT_REG": (0, 0x8001, "standard"), + "ONFOXM_ET_": (0x4F4E4600, 0xFFFF, "standard"), + "ERICOXM_OF_": (0, 0x1000, "extension"), + # This is the experimenter OXM class for Nicira, which is the + # one that OVS would be using instead of NXM_OF_ and NXM_NX_ + # if OVS didn't have those grandfathered in. It is currently + # used only to test support for experimenter OXM, since there + # are barely any real uses of experimenter OXM in the wild. + "NXOXM_ET_": (0x00002320, 0xFFFF, "extension"), +} + + +def oxm_name_to_class(name): + prefix = "" + class_ = None + for p, c in OXM_CLASSES.items(): + if name.startswith(p) and len(p) > len(prefix): + prefix = p + class_ = c + return class_ + + +def is_standard_oxm(name): + oxm_vendor, oxm_class, oxm_class_type = oxm_name_to_class(name) + return oxm_class_type == "standard" + + +def get_line(): + global line + global line_number + line = input_file.readline() + line_number += 1 + if line == "": + fatal("unexpected end of input") + + +n_errors = 0 + + +def error(msg): + global n_errors + sys.stderr.write("%s:%d: %s\n" % (file_name, line_number, msg)) + n_errors += 1 + + +def fatal(msg): + error(msg) + sys.exit(1) + + +def parse_oxms(s, prefix, n_bytes): + if s == "none": + return () + + return tuple(parse_oxm(s2.strip(), prefix, n_bytes) for s2 in s.split(",")) + + +match_types = dict() + + +def parse_oxm(s, prefix, n_bytes): + global match_types + + m = re.match( + r"([A-Z0-9_]+)\(([0-9]+)\) since(?: OF(1\.[0-9]+) and)? v([12]\.[0-9]+)$", # noqa: E501 + s, + ) + if not m: + fatal("%s: syntax error parsing %s" % (s, prefix)) + + name, oxm_type, of_version, ovs_version = m.groups() + + class_ = oxm_name_to_class(name) + if class_ is None: + fatal("unknown OXM class for %s" % name) + oxm_vendor, oxm_class, oxm_class_type = class_ + + if class_ in match_types: + if oxm_type in match_types[class_]: + fatal( + "duplicate match type for %s (conflicts with %s)" + % (name, match_types[class_][oxm_type]) + ) + else: + match_types[class_] = dict() + match_types[class_][oxm_type] = name + + # Normally the oxm_length is the size of the field, but for experimenter + # OXMs oxm_length also includes the 4-byte experimenter ID. + oxm_length = n_bytes + if oxm_class == 0xFFFF: + oxm_length += 4 + + header = (oxm_vendor, oxm_class, int(oxm_type), oxm_length) + + if of_version: + if oxm_class_type == "extension": + fatal("%s: OXM extension can't have OpenFlow version" % name) + if of_version not in VERSION: + fatal("%s: unknown OpenFlow version %s" % (name, of_version)) + of_version_nr = VERSION[of_version] + if of_version_nr < VERSION["1.2"]: + fatal("%s: claimed version %s predates OXM" % (name, of_version)) + else: + if oxm_class_type == "standard": + fatal("%s: missing OpenFlow version number" % name) + of_version_nr = 0 + + return (header, name, of_version_nr, ovs_version) + + +def parse_field(mff, comment): + f = {"mff": mff} + + # First line of comment is the field name. + m = re.match( + r'"([^"]+)"(?:\s+\(aka "([^"]+)"\))?(?:\s+\(.*\))?\.', comment[0] + ) + if not m: + fatal("%s lacks field name" % mff) + f["name"], f["extra_name"] = m.groups() + + # Find the last blank line the comment. The field definitions + # start after that. + blank = None + for i in range(len(comment)): + if not comment[i]: + blank = i + if not blank: + fatal("%s: missing blank line in comment" % mff) + + d = {} + for key in ( + "Type", + "Maskable", + "Formatting", + "Prerequisites", + "Access", + "Prefix lookup member", + "OXM", + "NXM", + "OF1.0", + "OF1.1", + ): + d[key] = None + for fline in comment[blank + 1 :]: + m = re.match(r"([^:]+):\s+(.*)\.$", fline) + if not m: + fatal( + "%s: syntax error parsing key-value pair as part of %s" + % (fline, mff) + ) + key, value = m.groups() + if key not in d: + fatal("%s: unknown key" % key) + elif key == "Code point": + d[key] += [value] + elif d[key] is not None: + fatal("%s: duplicate key" % key) + d[key] = value + for key, value in d.items(): + if not value and key not in ( + "OF1.0", + "OF1.1", + "Prefix lookup member", + "Notes", + ): + fatal("%s: missing %s" % (mff, key)) + + m = re.match(r"([a-zA-Z0-9]+)(?: \(low ([0-9]+) bits\))?$", d["Type"]) + if not m: + fatal("%s: syntax error in type" % mff) + type_ = m.group(1) + if type_ not in TYPES: + fatal("%s: unknown type %s" % (mff, d["Type"])) + + f["n_bytes"] = TYPES[type_][0] + if m.group(2): + f["n_bits"] = int(m.group(2)) + if f["n_bits"] > f["n_bytes"] * 8: + fatal( + "%s: more bits (%d) than field size (%d)" + % (mff, f["n_bits"], 8 * f["n_bytes"]) + ) + else: + f["n_bits"] = 8 * f["n_bytes"] + f["variable"] = TYPES[type_][1] + + if d["Maskable"] == "no": + f["mask"] = "MFM_NONE" + elif d["Maskable"] == "bitwise": + f["mask"] = "MFM_FULLY" + else: + fatal("%s: unknown maskable %s" % (mff, d["Maskable"])) + + fmt = FORMATTING.get(d["Formatting"]) + if not fmt: + fatal("%s: unknown format %s" % (mff, d["Formatting"])) + f["formatting"] = d["Formatting"] + if f["n_bytes"] < fmt[1] or f["n_bytes"] > fmt[2]: + fatal( + "%s: %d-byte field can't be formatted as %s" + % (mff, f["n_bytes"], d["Formatting"]) + ) + f["string"] = fmt[0] + + f["prereqs"] = d["Prerequisites"] + if f["prereqs"] not in PREREQS: + fatal("%s: unknown prerequisites %s" % (mff, d["Prerequisites"])) + + if d["Access"] == "read-only": + f["writable"] = False + elif d["Access"] == "read/write": + f["writable"] = True + else: + fatal("%s: unknown access %s" % (mff, d["Access"])) + + f["OF1.0"] = d["OF1.0"] + if not d["OF1.0"] in (None, "exact match", "CIDR mask"): + fatal("%s: unknown OF1.0 match type %s" % (mff, d["OF1.0"])) + + f["OF1.1"] = d["OF1.1"] + if not d["OF1.1"] in (None, "exact match", "bitwise mask"): + fatal("%s: unknown OF1.1 match type %s" % (mff, d["OF1.1"])) + + f["OXM"] = parse_oxms(d["OXM"], "OXM", f["n_bytes"]) + parse_oxms( + d["NXM"], "NXM", f["n_bytes"] + ) + + f["prefix"] = d["Prefix lookup member"] + + return f + + +def extract_ofp_fields(fn): + global file_name + global input_file + global line_number + global line + + file_name = fn + input_file = open(file_name) + line_number = 0 + + fields = [] + + while True: + get_line() + if re.match("enum.*mf_field_id", line): + break + + while True: + get_line() + if ( + line.startswith("/*") + or line.startswith(" *") + or line.startswith("#") + or not line + or line.isspace() + ): + continue + elif re.match(r"}", line) or re.match(r"\s+MFF_N_IDS", line): + break + + # Parse the comment preceding an MFF_ constant into 'comment', + # one line to an array element. + line = line.strip() + if not line.startswith("/*"): + fatal("unexpected syntax between fields") + line = line[1:] + comment = [] + end = False + while not end: + line = line.strip() + if line.startswith("*/"): + get_line() + break + if not line.startswith("*"): + fatal("unexpected syntax within field") + + line = line[1:] + if line.startswith(" "): + line = line[1:] + if line.startswith(" ") and comment: + continuation = True + line = line.lstrip() + else: + continuation = False + + if line.endswith("*/"): + line = line[:-2].rstrip() + end = True + else: + end = False + + if continuation: + comment[-1] += " " + line + else: + comment += [line] + get_line() + + # Drop blank lines at each end of comment. + while comment and not comment[0]: + comment = comment[1:] + while comment and not comment[-1]: + comment = comment[:-1] + + # Parse the MFF_ constant(s). + mffs = [] + while True: + m = re.match(r"\s+(MFF_[A-Z0-9_]+),?\s?$", line) + if not m: + break + mffs += [m.group(1)] + get_line() + if not mffs: + fatal("unexpected syntax looking for MFF_ constants") + + if len(mffs) > 1 or "" in comment[0]: + for mff in mffs: + # Extract trailing integer. + m = re.match(".*[^0-9]([0-9]+)$", mff) + if not m: + fatal("%s lacks numeric suffix in register group" % mff) + n = m.group(1) + + # Search-and-replace within the comment, + # and drop lines that have for x != n. + instance = [] + for x in comment: + y = x.replace("", n) + if re.search("<[0-9]+>", y): + if ("<%s>" % n) not in y: + continue + y = re.sub("<[0-9]+>", "", y) + instance += [y.strip()] + fields += [parse_field(mff, instance)] + else: + fields += [parse_field(mffs[0], comment)] + continue + + input_file.close() + + if n_errors: + sys.exit(1) + + return fields From patchwork Fri Jul 8 18:03:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654331 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=TS6rbVjp; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh3d0WqCz9ryY for ; Sat, 9 Jul 2022 04:03:40 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 606B184824; Fri, 8 Jul 2022 18:03:38 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 606B184824 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=TS6rbVjp X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BCCEghVUk4FO; Fri, 8 Jul 2022 18:03:37 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 5694884840; Fri, 8 Jul 2022 18:03:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 5694884840 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2D65BC0077; Fri, 8 Jul 2022 18:03:36 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 988CFC002D for ; Fri, 8 Jul 2022 18:03:34 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 5306384805 for ; Fri, 8 Jul 2022 18:03:34 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 5306384805 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id l-eG-qbbcSnC for ; Fri, 8 Jul 2022 18:03:32 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org DEF6984832 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id DEF6984832 for ; Fri, 8 Jul 2022 18:03:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303410; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T9EGDF1GwswhLl03wcS1FsU3Iiyfpw8kub5oapAWqRg=; b=TS6rbVjpxUPo49qc5Ezt3QsZ45qKnJ8PgEQ88u5gJ54R+JbHoGFeBNyAvxUZ1sk2Rp4zvS Yt8KsrCWBQi/1zhOLW1z4YoXpw7paWCWM2GF+vlEjw0uVDQjtTJoeo8Ve8tAqvjnc//wpQ WXcQya6JzzXJeR9/MCpGlRqGKVRkOB8= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-196-vYpM0EQ7NomklHydjc5L4g-1; Fri, 08 Jul 2022 14:03:29 -0400 X-MC-Unique: vYpM0EQ7NomklHydjc5L4g-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3FB08382C967; Fri, 8 Jul 2022 18:03:29 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4CDE62026D64; Fri, 8 Jul 2022 18:03:28 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:04 +0200 Message-Id: <20220708180316.2852046-6-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 05/17] build-aux: generate ofp field decoders X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Based on meta-field information extracted by extract_ofp_fields, autogenerate the right decoder to be used. Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- build-aux/automake.mk | 6 ++- build-aux/gen_ofp_field_decoders | 69 ++++++++++++++++++++++++++++++++ python/.gitignore | 1 + python/automake.mk | 6 +++ 4 files changed, 80 insertions(+), 2 deletions(-) create mode 100755 build-aux/gen_ofp_field_decoders diff --git a/build-aux/automake.mk b/build-aux/automake.mk index 6267ccd7c..b9a77a51c 100644 --- a/build-aux/automake.mk +++ b/build-aux/automake.mk @@ -5,6 +5,7 @@ EXTRA_DIST += \ build-aux/dist-docs \ build-aux/dpdkstrip.py \ build-aux/generate-dhparams-c \ + build-aux/gen_ofp_field_decoders \ build-aux/initial-tab-allowed-files \ build-aux/sodepends.py \ build-aux/soexpand.py \ @@ -12,7 +13,8 @@ EXTRA_DIST += \ build-aux/xml2nroff FLAKE8_PYFILES += \ - $(srcdir)/build-aux/xml2nroff \ build-aux/dpdkstrip.py \ + build-aux/gen_ofp_field_decoders \ build-aux/sodepends.py \ - build-aux/soexpand.py + build-aux/soexpand.py \ + build-aux/xml2nroff diff --git a/build-aux/gen_ofp_field_decoders b/build-aux/gen_ofp_field_decoders new file mode 100755 index 000000000..71364aeb4 --- /dev/null +++ b/build-aux/gen_ofp_field_decoders @@ -0,0 +1,69 @@ +#!/bin/env python + +import argparse + +import build.extract_ofp_fields as extract_fields + + +def main(): + parser = argparse.ArgumentParser( + description="Tool to generate python ofproto field decoders from" + "meta-flow information" + ) + parser.add_argument( + "metaflow", + metavar="FILE", + type=str, + help="Read meta-flow info from file", + ) + + args = parser.parse_args() + + fields = extract_fields.extract_ofp_fields(args.metaflow) + + field_decoders = {} + for field in fields: + decoder = get_decoder(field) + field_decoders[field.get("name")] = decoder + if field.get("extra_name"): + field_decoders[field.get("extra_name")] = decoder + + code = """ +# This file is auto-generated. Do not edit! + +import functools +from ovs.flow import decoders + +field_decoders = {{ +{decoders} +}} +""".format( + decoders="\n".join( + [ + " '{name}': {decoder},".format(name=name, decoder=decoder) + for name, decoder in field_decoders.items() + ] + ) + ) + print(code) + + +def get_decoder(field): + formatting = field.get("formatting") + if formatting in ["decimal", "hexadecimal"]: + if field.get("mask") == "MFM_NONE": + return "decoders.decode_int" + else: + if field.get("n_bits") in [8, 16, 32, 64, 128, 992]: + return "decoders.Mask{}".format(field.get("n_bits")) + return "decoders.decode_mask({})".format(field.get("n_bits")) + elif formatting in ["IPv4", "IPv6"]: + return "decoders.IPMask" + elif formatting == "Ethernet": + return "decoders.EthMask" + else: + return "decoders.decode_default" + + +if __name__ == "__main__": + main() diff --git a/python/.gitignore b/python/.gitignore index 60ace6f05..38e1f8b90 100644 --- a/python/.gitignore +++ b/python/.gitignore @@ -1,2 +1,3 @@ dist/ *.egg-info +ovs/flow/ofp_fields.py diff --git a/python/automake.mk b/python/automake.mk index d11a7ade1..8d6311555 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -127,3 +127,9 @@ EXTRA_DIST += python/ovs/dirs.py.template CLEANFILES += python/ovs/dirs.py EXTRA_DIST += python/TODO.rst + +$(srcdir)/python/ovs/flow/ofp_fields.py: $(srcdir)/build-aux/gen_ofp_field_decoders include/openvswitch/meta-flow.h + $(AM_V_GEN)$(run_python) $< $(srcdir)/include/openvswitch/meta-flow.h > $@.tmp + $(AM_V_at)mv $@.tmp $@ +EXTRA_DIST += python/ovs/flow/ofp_fields.py +CLEANFILES += python/ovs/flow/ofp_fields.py From patchwork Fri Jul 8 18:03:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654333 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=cOICnLep; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh3r5S6kz9ryY for ; Sat, 9 Jul 2022 04:03:52 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 64ED761426; Fri, 8 Jul 2022 18:03:50 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 64ED761426 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=cOICnLep X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lQLBdTSaJxi6; Fri, 8 Jul 2022 18:03:46 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 0097F61453; Fri, 8 Jul 2022 18:03:44 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 0097F61453 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B4D57C007C; Fri, 8 Jul 2022 18:03:43 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 18E5BC0086 for ; Fri, 8 Jul 2022 18:03:40 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 7E525425F7 for ; Fri, 8 Jul 2022 18:03:37 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 7E525425F7 Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=cOICnLep X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZuqZXxQpMKcJ for ; Fri, 8 Jul 2022 18:03:36 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 3B719424CD Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 3B719424CD for ; Fri, 8 Jul 2022 18:03:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303414; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VwWXFaFOJn8BPftnSgURcsKL0OTBphzrPx+vbFchEhk=; b=cOICnLepVzkpsbGuyil2hdauQEXahBMV+0qTM5Qc249a3lMF0fYPqm0UMbDh5mMFBEt3vB JEWqsXfG6zot9pdC+htylwFplFIxWyzOXa/QV9s6RLGECHhveorBJTdZ7tNfet9viCkV3u q1LBMvWLGtXHhKwMHLGo9F2ZSEDlqpw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-74-79hj-UhlPBeQS1Ma5ZKv4w-1; Fri, 08 Jul 2022 14:03:30 -0400 X-MC-Unique: 79hj-UhlPBeQS1Ma5ZKv4w-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 74100101A58D; Fri, 8 Jul 2022 18:03:30 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id 80F122026D64; Fri, 8 Jul 2022 18:03:29 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:05 +0200 Message-Id: <20220708180316.2852046-7-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 06/17] python: add flow base class X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" It simplifies the implementation of different types of flows by creating the concept of Section (e.g: match, action) and automatic accessors for all the provided Sections Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 1 + python/ovs/flow/flow.py | 125 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 126 insertions(+) create mode 100644 python/ovs/flow/flow.py diff --git a/python/automake.mk b/python/automake.mk index 8d6311555..7f401da25 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -28,6 +28,7 @@ ovs_pyfiles = \ python/ovs/fcntl_win.py \ python/ovs/flow/__init__.py \ python/ovs/flow/decoders.py \ + python/ovs/flow/flow.py \ python/ovs/flow/kv.py \ python/ovs/flow/list.py \ python/ovs/json.py \ diff --git a/python/ovs/flow/flow.py b/python/ovs/flow/flow.py new file mode 100644 index 000000000..2f053e77d --- /dev/null +++ b/python/ovs/flow/flow.py @@ -0,0 +1,125 @@ +""" Defines the Flow class. +""" + + +class Section(object): + """A flow can be seen as composed of different sections, e.g: + + [info] [match] actions=[actions] + + This class represents each of those sections. + + A section is basically a set of Key-Value pairs. Typically, they can be + expressed as a dictionary, for instance the "match" part of a flow can be + expressed as: + { + "nw_src": "192.168.1.1", + "nw_dst": "192.168.1.2", + } + However, some of them must be expressed as a list which allows for + duplicated keys. For instance, the "actions" section could be: + [ + { + "output": 32 + }, + { + "output": 33 + } + ] + + The is_list flag is used to discriminate this. + + Attributes: + name (str): Name of the section. + pos (int): Position within the overall flow string. + string (str): Section string. + data (list[KeyValue]): Parsed data of the section. + is_list (bool): Whether the key-values shall be expressed as a list + (i.e: it allows repeated keys). + """ + + def __init__(self, name, pos, string, data, is_list=False): + self.name = name + self.pos = pos + self.string = string + self.data = data + self.is_list = is_list + + def __str__(self): + return "{} (at {}): {}".format(self.name, self.pos, self.string) + + def __repr__(self): + return "%s('%s')" % (self.__class__.__name__, self) + + def dict(self): + return {self.name: self.format_data()} + + def format_data(self): + """Returns the section's key-values formatted in a dictionary or list + depending on the value of is_list flag. + """ + if self.is_list: + return [{item.key: item.value} for item in self.data] + else: + return {item.key: item.value for item in self.data} + + +class Flow(object): + """The Flow class is a base class for other types of concrete flows + (such as OFproto Flows or DPIF Flows). + + A flow is basically comprised of a number of sections. + For each section named {section_name}, the flow object will have the + following attributes: + - {section_name} will return the sections data in a formatted way. + - {section_name}_kv will return the sections data as a list of KeyValues. + + Args: + sections (list[Section]): List of sections that comprise the flow + orig (str): Original flow string. + id (Any): Optional; identifier that clients can use to uniquely + identify this flow. + """ + + def __init__(self, sections, orig="", id=None): + self._sections = sections + self._orig = orig + self._id = id + for section in sections: + setattr( + self, section.name, self.section(section.name).format_data() + ) + setattr( + self, + "{}_kv".format(section.name), + self.section(section.name).data, + ) + + def section(self, name): + """Return the section by name.""" + return next( + (sect for sect in self._sections if sect.name == name), None + ) + + @property + def id(self): + """Return the Flow ID.""" + return self._id + + @property + def sections(self): + """Return the all the sections in a list.""" + return self._sections + + @property + def orig(self): + """Return the original flow string.""" + return self._orig + + def dict(self): + """Returns the Flow information in a dictionary.""" + flow_dict = {"orig": self.orig} + for section in self.sections: + flow_dict.update(section.dict()) + + return flow_dict From patchwork Fri Jul 8 18:03:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654340 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=XFQCwUQk; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh4M5yBzz9ryY for ; Sat, 9 Jul 2022 04:04:19 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 450BD61518; Fri, 8 Jul 2022 18:04:17 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 450BD61518 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=XFQCwUQk X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2b8dvBhZAE4L; Fri, 8 Jul 2022 18:04:11 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id BA13461493; Fri, 8 Jul 2022 18:03:59 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org BA13461493 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7E437C007B; Fri, 8 Jul 2022 18:03:53 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0578DC0077 for ; Fri, 8 Jul 2022 18:03:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id F0CCA84891 for ; Fri, 8 Jul 2022 18:03:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org F0CCA84891 Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=XFQCwUQk X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QVtG82UNm-oD for ; Fri, 8 Jul 2022 18:03:40 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 8EE9D8487C Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 8EE9D8487C for ; Fri, 8 Jul 2022 18:03:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303419; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PYvWRVM/nZliSsEIdFCGG5U4bQPJWMZsRMxdoprMMFE=; b=XFQCwUQkftkiIYXGLwrBPGTWMJhTIZXsSUR1Rt/sKN4YkTYDOKxqleHMtnDvwjh00E4FbM lasuh9XNM153TEHmYoOVVZSCofOLymQtTSD621w3DpVOymsYWUw2IemWMrjR3Mj1hYu939 /s1rfb/nadxGJXmkXm03Iucp+bit/AY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-160-8wWfUTDOMnahkG7VsbazIA-1; Fri, 08 Jul 2022 14:03:32 -0400 X-MC-Unique: 8wWfUTDOMnahkG7VsbazIA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E0B0B811E87; Fri, 8 Jul 2022 18:03:31 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id D01E12026D64; Fri, 8 Jul 2022 18:03:30 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:06 +0200 Message-Id: <20220708180316.2852046-8-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 07/17] python: introduce OpenFlow Flow parsing X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Introduce OFPFlow class and all its decoders. Most of the decoders are generic (from decoders.py). Some have special syntax and need a specific implementation. Decoders for nat are moved to the common decoders.py because it's syntax is shared with other types of flows (e.g: dpif flows). Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 2 + python/ovs/flow/decoders.py | 108 +++++++++ python/ovs/flow/ofp.py | 428 ++++++++++++++++++++++++++++++++++++ python/ovs/flow/ofp_act.py | 306 ++++++++++++++++++++++++++ 4 files changed, 844 insertions(+) create mode 100644 python/ovs/flow/ofp.py create mode 100644 python/ovs/flow/ofp_act.py diff --git a/python/automake.mk b/python/automake.mk index 7f401da25..7bca2bd40 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -31,6 +31,8 @@ ovs_pyfiles = \ python/ovs/flow/flow.py \ python/ovs/flow/kv.py \ python/ovs/flow/list.py \ + python/ovs/flow/ofp.py \ + python/ovs/flow/ofp_act.py \ python/ovs/json.py \ python/ovs/jsonrpc.py \ python/ovs/ovsuuid.py \ diff --git a/python/ovs/flow/decoders.py b/python/ovs/flow/decoders.py index 883e61acf..73d28e057 100644 --- a/python/ovs/flow/decoders.py +++ b/python/ovs/flow/decoders.py @@ -6,6 +6,7 @@ object. """ import netaddr +import re class Decoder(object): @@ -414,3 +415,110 @@ class IPMask(Decoder): def to_json(self): return str(self) + + +def decode_free_output(value): + """The value of the output action can be found free, i.e: without the + 'output' keyword. This decoder decodes its value when found this way.""" + try: + return "output", {"port": int(value)} + except ValueError: + return "output", {"port": value.strip('"')} + + +ipv4 = r"(?:\d{1,3}.?){3}\d{1,3}" +ipv4_capture = r"({ipv4})".format(ipv4=ipv4) +ipv6 = r"[\w:\.]+" +ipv6_capture = r"(?:\[*)?({ipv6})(?:\]*)?".format(ipv6=ipv6) +port_range = r":(\d+)(?:-(\d+))?" +ip_range_regexp = r"{ip_cap}(?:-{ip_cap})?(?:{port_range})?" +ipv4_port_regex = re.compile( + ip_range_regexp.format(ip_cap=ipv4_capture, port_range=port_range) +) +ipv6_port_regex = re.compile( + ip_range_regexp.format(ip_cap=ipv6_capture, port_range=port_range) +) + + +def decode_ip_port_range(value): + """ + Decodes an IP and port range: + {ip_start}-{ip-end}:{port_start}-{port_end} + + IPv6 addresses are surrounded by "[" and "]" if port ranges are also + present + + Returns the following dictionary: + { + "addrs": { + "start": {ip_start} + "end": {ip_end} + } + "ports": { + "start": {port_start}, + "end": {port_end} + } + (the "ports" key might be omitted) + """ + if value.count(":") > 1: + match = ipv6_port_regex.match(value) + else: + match = ipv4_port_regex.match(value) + + ip_start = match.group(1) + ip_end = match.group(2) + port_start = match.group(3) + port_end = match.group(4) + + result = { + "addrs": { + "start": netaddr.IPAddress(ip_start), + "end": netaddr.IPAddress(ip_end or ip_start), + } + } + if port_start: + result["ports"] = { + "start": int(port_start), + "end": int(port_end or port_start), + } + + return result + + +def decode_nat(value): + """Decodes the 'nat' keyword of the ct action. + + The format is: + nat + Flag format. + nat(type=addrs[:ports][,flag]...) + Full format where the address-port range has the same format as + the one described in decode_ip_port_range. + + Examples: + nat(src=0.0.0.0) + nat(src=0.0.0.0,persistent) + nat(dst=192.168.1.0-192.168.1.253:4000-5000) + nat(dst=192.168.1.0-192.168.1.253,hash) + nat(dst=[fe80::f150]-[fe80::f15f]:255-300) + """ + if not value: + return True # If flag format, the value is True. + + result = dict() + type_parts = value.split("=") + result["type"] = type_parts[0] + + if len(type_parts) > 1: + value_parts = type_parts[1].split(",") + if len(type_parts) != 2: + raise ValueError("Malformed nat action: %s" % value) + + ip_port_range = decode_ip_port_range(value_parts[0]) + + result = {"type": type_parts[0], **ip_port_range} + + for flag in value_parts[1:]: + result[flag] = True + + return result diff --git a/python/ovs/flow/ofp.py b/python/ovs/flow/ofp.py new file mode 100644 index 000000000..0bc110c57 --- /dev/null +++ b/python/ovs/flow/ofp.py @@ -0,0 +1,428 @@ +"""Defines the parsers needed to parse ofproto flows. +""" + +import functools + +from ovs.flow.kv import KVParser, KVDecoders, nested_kv_decoder +from ovs.flow.ofp_fields import field_decoders +from ovs.flow.flow import Flow, Section +from ovs.flow.list import ListDecoders, nested_list_decoder +from ovs.flow.decoders import ( + decode_default, + decode_flag, + decode_int, + decode_time, + decode_mask, + IPMask, + EthMask, + decode_free_output, + decode_nat, +) +from ovs.flow.ofp_act import ( + decode_output, + decode_field, + decode_controller, + decode_bundle, + decode_bundle_load, + decode_encap, + decode_load_field, + decode_set_field, + decode_move_field, + decode_dec_ttl, + decode_chk_pkt_larger, + decode_zone, + decode_exec, + decode_learn, +) + + +class OFPFlow(Flow): + """OFPFLow represents an OpenFlow Flow. + + Attributes: + info: The info section. + match: The match section. + actions: The actions section. + id: The id object given at construction time. + """ + + """ + These class variables are used to cache the KVDecoders instances. This + will speed up subsequent flow parsings. + """ + _info_decoders = None + _match_decoders = None + _action_decoders = None + + @staticmethod + def info_decoders(): + """Return the KVDecoders instance to parse the info section. + + Uses the cached version if available. + """ + if not OFPFlow._info_decoders: + OFPFlow._info_decoders = OFPFlow._gen_info_decoders() + return OFPFlow._info_decoders + + @staticmethod + def match_decoders(): + """Return the KVDecoders instance to parse the match section. + + Uses the cached version if available. + """ + if not OFPFlow._match_decoders: + OFPFlow._match_decoders = OFPFlow._gen_match_decoders() + return OFPFlow._match_decoders + + @staticmethod + def action_decoders(): + """Return the KVDecoders instance to parse the actions section. + + Uses the cached version if available. + """ + if not OFPFlow._action_decoders: + OFPFlow._action_decoders = OFPFlow._gen_action_decoders() + return OFPFlow._action_decoders + + def __init__(self, ofp_string, id=None): + """Create a OFPFlow from a flow string. + + The string is expected to have the followoing format: + + [flow data] [match] actions=[actions] + + Args: + ofp_string(str): An OpenFlow flow string. + id(Any): Optional; any object used to uniquely identify this flow + from the rest. + + Returns + An OFPFlow with the content of the flow string or None if there is + no flow information but the string is expected to be found in a + flow dump. + + Raises + ValueError if the string is malformed. + ParseError if an error in parsing occurs. + """ + if " reply " in ofp_string: + return None + + sections = list() + parts = ofp_string.split("actions=") + if len(parts) != 2: + raise ValueError("malformed ofproto flow: %s" % ofp_string) + + actions = parts[1] + + field_parts = parts[0].rstrip(" ").rpartition(" ") + if len(field_parts) != 3: + raise ValueError("malformed ofproto flow: %s" % ofp_string) + + info = field_parts[0] + match = field_parts[2] + + iparser = KVParser(info, OFPFlow.info_decoders()) + iparser.parse() + isection = Section( + name="info", + pos=ofp_string.find(info), + string=info, + data=iparser.kv(), + ) + sections.append(isection) + + mparser = KVParser(match, OFPFlow.match_decoders()) + mparser.parse() + msection = Section( + name="match", + pos=ofp_string.find(match), + string=match, + data=mparser.kv(), + ) + sections.append(msection) + + aparser = KVParser(actions, OFPFlow.action_decoders()) + aparser.parse() + asection = Section( + name="actions", + pos=ofp_string.find(actions), + string=actions, + data=aparser.kv(), + is_list=True, + ) + sections.append(asection) + + super(OFPFlow, self).__init__(sections, ofp_string, id) + + def __str__(self): + if self._orig: + return self._orig + else: + return self.to_string() + + def to_string(self): + """Return a text representation of the flow.""" + string = "Info: {} | ".format(self.info) + string += "Match : {} | ".format(self.match) + string += "Actions: {}".format(self.actions) + return string + + @staticmethod + def _gen_info_decoders(): + """Generate the info KVDecoders.""" + args = { + "table": decode_int, + "duration": decode_time, + "n_packet": decode_int, + "n_bytes": decode_int, + "cookie": decode_int, + "idle_timeout": decode_time, + "hard_timeout": decode_time, + "hard_age": decode_time, + } + return KVDecoders(args) + + @staticmethod + def _gen_match_decoders(): + """Generate the match KVDecoders.""" + args = { + **OFPFlow._field_decoder_args(), + **OFPFlow._extra_match_decoder_args(), + } + + return KVDecoders(args) + + @staticmethod + def _extra_match_decoder_args(): + """Returns the extra KVDecoder arguments needed to decode the match + part of a flow (apart from the fields).""" + return { + "priority": decode_int, + } + + @staticmethod + def _field_decoder_args(): + """Returns the KVDecoder arguments needed to decode match fields.""" + shorthands = [ + "eth", + "ip", + "ipv6", + "icmp", + "icmp6", + "tcp", + "tcp6", + "udp", + "udp6", + "sctp", + "arp", + "rarp", + "mpls", + "mplsm", + ] + + fields = {**field_decoders, **{key: decode_flag for key in shorthands}} + + # vlan_vid field is special. Although it is technically 12 bit wide, + # bit 12 is allowed to be set to 1 to indicate that the vlan header is + # present (see section VLAN FIELDS in + # http://www.openvswitch.org/support/dist-docs/ovs-fields.7.txt) + # Therefore, override the generated vlan_vid field size. + fields["vlan_vid"] = decode_mask(13) + return fields + + @staticmethod + def _gen_action_decoders(): + """Generate the actions decoders.""" + + actions = { + **OFPFlow._output_actions_decoders_args(), + **OFPFlow._encap_actions_decoders_args(), + **OFPFlow._field_action_decoders_args(), + **OFPFlow._meta_action_decoders_args(), + **OFPFlow._fw_action_decoders_args(), + **OFPFlow._control_action_decoders_args(), + **OFPFlow._other_action_decoders_args(), + } + clone_actions = OFPFlow._clone_actions_decoders_args(actions) + actions.update(clone_actions) + return KVDecoders(actions, default_free=decode_free_output) + + @staticmethod + def _output_actions_decoders_args(): + """Returns the decoder arguments for the output actions.""" + return { + "output": decode_output, + "drop": decode_flag, + "controller": decode_controller, + "enqueue": nested_list_decoder( + ListDecoders([("port", decode_default), ("queue", int)]), + delims=[",", ":"], + ), + "bundle": decode_bundle, + "bundle_load": decode_bundle_load, + "group": decode_default, + } + + @staticmethod + def _encap_actions_decoders_args(): + """Returns the decoders arguments for the encap actions.""" + + return { + "pop_vlan": decode_flag, + "strip_vlan": decode_flag, + "push_vlan": decode_default, + "decap": decode_flag, + "encap": decode_encap, + } + + @staticmethod + def _field_action_decoders_args(): + """Returns the decoders arguments for field-modification actions.""" + # Field modification actions + field_default_decoders = [ + "set_mpls_label", + "set_mpls_tc", + "set_mpls_ttl", + "mod_nw_tos", + "mod_nw_ecn", + "mod_tcp_src", + "mod_tcp_dst", + ] + return { + "load": decode_load_field, + "set_field": functools.partial( + decode_set_field, KVDecoders(OFPFlow._field_decoder_args()) + ), + "move": decode_move_field, + "mod_dl_dst": EthMask, + "mod_dl_src": EthMask, + "mod_nw_dst": IPMask, + "mod_nw_src": IPMask, + "dec_ttl": decode_dec_ttl, + "dec_mpls_ttl": decode_flag, + "dec_nsh_ttl": decode_flag, + "check_pkt_larger": decode_chk_pkt_larger, + **{field: decode_default for field in field_default_decoders}, + } + + @staticmethod + def _meta_action_decoders_args(): + """Returns the decoders arguments for the metadata actions.""" + meta_default_decoders = ["set_tunnel", "set_tunnel64", "set_queue"] + return { + "pop_queue": decode_flag, + **{field: decode_default for field in meta_default_decoders}, + } + + @staticmethod + def _fw_action_decoders_args(): + """Returns the decoders arguments for the firewalling actions.""" + return { + "ct": nested_kv_decoder( + KVDecoders( + { + "commit": decode_flag, + "zone": decode_zone, + "table": decode_int, + "nat": decode_nat, + "force": decode_flag, + "exec": functools.partial( + decode_exec, + KVDecoders( + { + **OFPFlow._encap_actions_decoders_args(), + **OFPFlow._field_action_decoders_args(), + **OFPFlow._meta_action_decoders_args(), + } + ), + ), + "alg": decode_default, + } + ) + ), + "ct_clear": decode_flag, + } + + @staticmethod + def _control_action_decoders_args(): + return { + "resubmit": nested_list_decoder( + ListDecoders( + [ + ("port", decode_default), + ("table", decode_int), + ("ct", decode_flag), + ] + ) + ), + "push": decode_field, + "pop": decode_field, + "exit": decode_flag, + "multipath": nested_list_decoder( + ListDecoders( + [ + ("fields", decode_default), + ("basis", decode_int), + ("algorithm", decode_default), + ("n_links", decode_int), + ("arg", decode_int), + ("dst", decode_field), + ] + ) + ), + } + + @staticmethod + def _clone_actions_decoders_args(action_decoders): + """Generate the decoder arguments for the clone actions. + + Args: + action_decoders (dict): The decoders of the supported nested + actions. + """ + return { + "learn": decode_learn( + { + **action_decoders, + "fin_timeout": nested_kv_decoder( + KVDecoders( + { + "idle_timeout": decode_time, + "hard_timeout": decode_time, + } + ) + ), + } + ), + "clone": functools.partial( + decode_exec, KVDecoders(action_decoders) + ), + } + + @staticmethod + def _other_action_decoders_args(): + """Generate the decoder arguments for other actions + (see man(7) ovs-actions).""" + return { + "conjunction": nested_list_decoder( + ListDecoders( + [("id", decode_int), ("k", decode_int), ("n", decode_int)] + ), + delims=[",", "/"], + ), + "note": decode_default, + "sample": nested_kv_decoder( + KVDecoders( + { + "probability": decode_int, + "collector_set_id": decode_int, + "obs_domain_id": decode_int, + "obs_point_id": decode_int, + "sampling_port": decode_default, + "ingress": decode_flag, + "egress": decode_flag, + } + ) + ), + } diff --git a/python/ovs/flow/ofp_act.py b/python/ovs/flow/ofp_act.py new file mode 100644 index 000000000..acb16cd9a --- /dev/null +++ b/python/ovs/flow/ofp_act.py @@ -0,0 +1,306 @@ +"""Defines decoders for OpenFlow actions. +""" + +import functools + +from ovs.flow.decoders import ( + decode_default, + decode_time, + decode_flag, + decode_int, +) +from ovs.flow.kv import nested_kv_decoder, KVDecoders, KeyValue, KVParser +from ovs.flow.list import nested_list_decoder, ListDecoders +from ovs.flow.ofp_fields import field_decoders + + +def decode_output(value): + """Decodes the output value. + + Does not support field specification. + """ + if len(value.split(",")) > 1: + return nested_kv_decoder()(value) + try: + return {"port": int(value)} + except ValueError: + return {"port": value.strip('"')} + + +def decode_controller(value): + """Decodes the controller action.""" + if not value: + return KeyValue("output", "controller") + else: + # Try controller:max_len + try: + max_len = int(value) + return { + "max_len": max_len, + } + except ValueError: + pass + # controller(key[=val], ...) + return nested_kv_decoder()(value) + + +def decode_bundle_load(value): + return decode_bundle(value, True) + + +def decode_bundle(value, load=False): + """Decode bundle action.""" + result = {} + keys = ["fields", "basis", "algorithm", "ofport"] + if load: + keys.append("dst") + + for key in keys: + parts = value.partition(",") + nvalue = parts[0] + value = parts[2] + if key == "ofport": + continue + result[key] = decode_default(nvalue) + + # Handle members: + mvalues = value.split("members:") + result["members"] = [int(port) for port in mvalues[1].split(",")] + return result + + +def decode_encap(value): + """Decodes encap action. Examples: + encap(ethernet) + encap(nsh(md_type=2,tlv(0x1000,10,0x12345678))) + + The generated dict has the following keys: "header", "props", e.g: + { + "header": "ethernet", + } + { + "header": "nsh", + "props": { + "md_type": 2, + "tlv": { + "class": 0x100, + "type": 10, + "value": 0x123456 + } + } + } + """ + + def free_hdr_decoder(free_val): + if free_val not in ["ethernet", "mpls", "mpls_mc", "nsh"]: + raise ValueError( + "Malformed encap action. Unkown header: {}".format(free_val) + ) + return "header", free_val + + parser = KVParser( + value, + KVDecoders( + { + "nsh": nested_kv_decoder( + KVDecoders( + { + "md_type": decode_default, + "tlv": nested_list_decoder( + ListDecoders( + [ + ("class", decode_int), + ("type", decode_int), + ("value", decode_int), + ] + ) + ), + } + ) + ), + }, + default_free=free_hdr_decoder, + ), + ) + parser.parse() + if len(parser.kv()) > 1: + raise ValueError("Malformed encap action: {}".format(value)) + + result = {} + if parser.kv()[0].key == "header": + result["header"] = parser.kv()[0].value + elif parser.kv()[0].key == "nsh": + result["header"] = "nsh" + result["props"] = parser.kv()[0].value + + return result + + +def decode_field(value): + """Decodes a field as defined in the 'Field Specification' of the actions + man page: + http://www.openvswitch.org/support/dist-docs/ovs-actions.7.txt.""" + parts = value.strip("]\n\r").split("[") + result = { + "field": parts[0], + } + + if len(parts) > 1 and parts[1]: + field_range = parts[1].split("..") + start = field_range[0] + end = field_range[1] if len(field_range) > 1 else start + if start: + result["start"] = int(start) + if end: + result["end"] = int(end) + + return result + + +def decode_load_field(value): + """Decodes LOAD actions such as: 'load:value->dst'.""" + parts = value.split("->") + if len(parts) != 2: + raise ValueError("Malformed load action : %s" % value) + + # If the load action is performed within a learn() action, + # The value can be specified as another field. + try: + return {"value": int(parts[0], 0), "dst": decode_field(parts[1])} + except ValueError: + return {"src": decode_field(parts[0]), "dst": decode_field(parts[1])} + + +def decode_set_field(field_decoders, value): + """Decodes SET_FIELD actions such as: 'set_field:value/mask->dst'. + + The value is decoded by field_decoders which is a KVDecoders instance. + Args: + field_decoders(KVDecoders): The KVDecoders to be used to decode the + field. + """ + parts = value.split("->") + if len(parts) != 2: + raise ValueError("Malformed set_field action : %s" % value) + + val = parts[0] + dst = parts[1] + + val_result = field_decoders.decode(dst, val) + + return { + "value": {val_result[0]: val_result[1]}, + "dst": decode_field(dst), + } + + +def decode_move_field(value): + """Decodes MOVE actions such as 'move:src->dst'.""" + parts = value.split("->") + if len(parts) != 2: + raise ValueError("Malformed move action : %s" % value) + + return { + "src": decode_field(parts[0]), + "dst": decode_field(parts[1]), + } + + +def decode_dec_ttl(value): + """Decodes dec_ttl and dec_ttl(id, id[2], ...) actions.""" + if not value: + return True + return [int(idx) for idx in value.split(",")] + + +def decode_chk_pkt_larger(value): + """Decodes 'check_pkt_larger(pkt_len)->dst' actions.""" + parts = value.split("->") + if len(parts) != 2: + raise ValueError("Malformed check_pkt_larger action : %s" % value) + + pkt_len = int(parts[0].strip("()")) + dst = decode_field(parts[1]) + return {"pkt_len": pkt_len, "dst": dst} + + +# CT decoders +def decode_zone(value): + """Decodes the value of the 'zone' keyword (part of the ct action).""" + try: + return int(value, 0) + except ValueError: + pass + return decode_field(value) + + +def decode_exec(action_decoders, value): + """Decodes the value of the 'exec' keyword (part of the ct action). + + Args: + decode_actions (KVDecoders): The decoders to be used to decode the + nested exec. + value (string): The string to be decoded. + """ + exec_parser = KVParser(value, action_decoders) + exec_parser.parse() + return [{kv.key: kv.value} for kv in exec_parser.kv()] + + +def decode_learn(action_decoders): + """Create the decoder to be used to decode the 'learn' action. + + The learn action has two added complexities: + 1) It can hold any valid action key-value. Therefore we must take + the precalculated action_decoders and use them. That's why we require + them as argument. + + 2) The way fields can be specified is augmented. Not only we have + 'field=value', but we also have: + - 'field=_src_' (where _src_ is another field name) + - and just 'field' + For this we need to create a wrapper of field_decoders that, for each + "field=X" key-value we check if X is a field_name or if it's actually + a value that we need to send to the appropriate field_decoder to + process. + + Args: + action_decoders (dict): Dictionary of decoders to be used in nested + action decoding. + """ + + def decode_learn_field(decoder, value): + """Generates a decoder to be used for the 'field' argument of the + 'learn' action. + + The field can hold a value that should be decoded, either as a field, + or as a the value (see man(7) ovs-actions). + + Args: + decoder (callable): The decoder. + """ + if value in field_decoders.keys(): + # It's a field + return value + else: + return decoder(value) + + learn_field_decoders = { + field: functools.partial(decode_learn_field, decoder) + for field, decoder in field_decoders.items() + } + learn_decoders = { + **action_decoders, + **learn_field_decoders, + "idle_timeout": decode_time, + "hard_timeout": decode_time, + "priority": decode_int, + "cookie": decode_int, + "send_flow_rem": decode_flag, + "table": decode_int, + "delete_learned": decode_flag, + "limit": decode_int, + "result_dst": decode_field, + } + + return functools.partial(decode_exec, KVDecoders(learn_decoders)) From patchwork Fri Jul 8 18:03:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654336 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=D3N4e0+u; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh405t4Cz9ryY for ; Sat, 9 Jul 2022 04:04:00 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id A27B5416B0; Fri, 8 Jul 2022 18:03:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org A27B5416B0 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=D3N4e0+u X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gposNB3nQfX8; Fri, 8 Jul 2022 18:03:51 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 8FD144166D; Fri, 8 Jul 2022 18:03:49 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 8FD144166D Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id EE1ECC0085; Fri, 8 Jul 2022 18:03:45 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1866BC0078 for ; Fri, 8 Jul 2022 18:03:41 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 389D46145B for ; Fri, 8 Jul 2022 18:03:39 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 389D46145B Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=D3N4e0+u X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uqYi5RM4SiIg for ; Fri, 8 Jul 2022 18:03:38 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org B889161429 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id B889161429 for ; Fri, 8 Jul 2022 18:03:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303416; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YSFPvZUIpdBsK8Sy4Qe2OsvwVJyuW9iRc1D7E3ln1lI=; b=D3N4e0+uc8EkbW32KKOvMkNjvMK/nWIc24cZ/skfy3LX21WilapfLAqVwy1yYoPovNctfm Yj8kCuSYvcAYYEvZZuldYLiIMPGZgVzidxFsX9hobHyYtC/KbB8hrn8iziNlzclj76oqvW cnb5f9hLe99KXb7x62m7IRN2mNVGnGc= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-597-s5j7DMHLNRCMSwm0BFxcKw-1; Fri, 08 Jul 2022 14:03:33 -0400 X-MC-Unique: s5j7DMHLNRCMSwm0BFxcKw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4BC7F3C01DE7; Fri, 8 Jul 2022 18:03:33 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id 33A112026D64; Fri, 8 Jul 2022 18:03:32 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:07 +0200 Message-Id: <20220708180316.2852046-9-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 08/17] python: add ovs datapath flow parsing X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" A ODPFlow is a Flow with the following sections: ufid info (e.g: bytes, packets, dp, etc) match actions Only three datapath actions require special handling: gre: because it has double parenthesis geneve: because it supports many concatenated lists of options nat: we reuse the decoder used for openflow actions Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 1 + python/ovs/flow/odp.py | 783 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 784 insertions(+) create mode 100644 python/ovs/flow/odp.py diff --git a/python/automake.mk b/python/automake.mk index 7bca2bd40..79affdc75 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -31,6 +31,7 @@ ovs_pyfiles = \ python/ovs/flow/flow.py \ python/ovs/flow/kv.py \ python/ovs/flow/list.py \ + python/ovs/flow/odp.py \ python/ovs/flow/ofp.py \ python/ovs/flow/ofp_act.py \ python/ovs/json.py \ diff --git a/python/ovs/flow/odp.py b/python/ovs/flow/odp.py new file mode 100644 index 000000000..87a3bae2f --- /dev/null +++ b/python/ovs/flow/odp.py @@ -0,0 +1,783 @@ +""" Defines an Open vSwitch Datapath Flow. +""" +import re +from functools import partial + +from ovs.flow.flow import Flow, Section + +from ovs.flow.kv import ( + KVParser, + KVDecoders, + nested_kv_decoder, + decode_nested_kv, +) +from ovs.flow.decoders import ( + decode_default, + decode_time, + decode_int, + decode_mask, + Mask8, + Mask16, + Mask32, + Mask64, + Mask128, + IPMask, + EthMask, + decode_free_output, + decode_flag, + decode_nat, +) + + +class ODPFlow(Flow): + """ODPFLow represents a Open vSwitch Datapath flow. + + Attributes: + ufid: The UFID section with only one key-value, with keyword "ufid". + info: The info section. + match: The match section. + actions: The actions section. + id: The id object given at construction time. + + """ + + """ + These class variables are used to cache the KVDecoders instances. This + will speed up subsequent flow parsings. + """ + _info_decoders = None + _match_decoders = None + _action_decoders = None + + @staticmethod + def info_decoders(): + """Return the KVDecoders instance to parse the info section. + + Uses the cached version if available. + """ + if not ODPFlow._info_decoders: + ODPFlow._info_decoders = ODPFlow._gen_info_decoders() + return ODPFlow._info_decoders + + @staticmethod + def match_decoders(): + """Return the KVDecoders instance to parse the match section. + + Uses the cached version if available. + """ + if not ODPFlow._match_decoders: + ODPFlow._match_decoders = ODPFlow._gen_match_decoders() + return ODPFlow._match_decoders + + @staticmethod + def action_decoders(): + """Return the KVDecoders instance to parse the actions section. + + Uses the cached version if available. + """ + if not ODPFlow._action_decoders: + ODPFlow._action_decoders = ODPFlow._gen_action_decoders() + return ODPFlow._action_decoders + + def __init__(self, odp_string, id=None): + """Parse a odp flow string. + + The string is expected to have the following format: + [ufid], [match] [flow data] actions:[actions] + + Args: + odp_string (str): A datapath flow string. + + Returns: + A ODPFlow instance. + """ + sections = [] + + # If UFID present, parse it and add it to it's own section. + ufid_pos = odp_string.find("ufid:") + if ufid_pos >= 0: + ufid_string = odp_string[ + ufid_pos : (odp_string[ufid_pos:].find(",") + 1) + ] + ufid_parser = KVParser( + ufid_string, KVDecoders({"ufid": decode_default}) + ) + ufid_parser.parse() + if len(ufid_parser.kv()) != 1: + raise ValueError("malformed odp flow: %s" % odp_string) + sections.append( + Section("ufid", ufid_pos, ufid_string, ufid_parser.kv()) + ) + + action_pos = odp_string.find("actions:") + if action_pos < 0: + raise ValueError("malformed odp flow: %s" % odp_string) + + # rest of the string is between ufid and actions + rest = odp_string[ + (ufid_pos + len(ufid_string) if ufid_pos >= 0 else 0) : action_pos + ] + + action_pos += 8 # len("actions:") + actions = odp_string[action_pos:] + + field_parts = rest.lstrip(" ").partition(" ") + + if len(field_parts) != 3: + raise ValueError("malformed odp flow: %s" % odp_string) + + match = field_parts[0] + info = field_parts[2] + + iparser = KVParser(info, ODPFlow.info_decoders()) + iparser.parse() + isection = Section( + name="info", + pos=odp_string.find(info), + string=info, + data=iparser.kv(), + ) + sections.append(isection) + + mparser = KVParser(match, ODPFlow.match_decoders()) + mparser.parse() + msection = Section( + name="match", + pos=odp_string.find(match), + string=match, + data=mparser.kv(), + ) + sections.append(msection) + + aparser = KVParser(actions, ODPFlow.action_decoders()) + aparser.parse() + asection = Section( + name="actions", + pos=action_pos, + string=actions, + data=aparser.kv(), + is_list=True, + ) + sections.append(asection) + + super(ODPFlow, self).__init__(sections, odp_string, id) + + def __str__(self): + if self._orig: + return self._orig + else: + return self.to_string() + + def to_string(self): + """Return a text representation of the flow.""" + string = "ufid: {}".format(self.ufid) if self.ufid else "" + string += "Info: {} | ".format(self.info) + string += "Match : {} | ".format(self.match) + string += "Actions: {}".format(self.actions) + return string + + @staticmethod + def _gen_info_decoders(): + """Generate the info KVDecoders.""" + return KVDecoders(ODPFlow._info_decoders_args()) + + @staticmethod + def _info_decoders_args(): + """Generate the decoder args for the info KVDecoders.""" + return { + "packets": decode_int, + "bytes": decode_int, + "used": decode_time, + "flags": decode_default, + "dp": decode_default, + } + + @staticmethod + def _gen_action_decoders(): + """Generate the action KVDecoders.""" + return KVDecoders( + ODPFlow._action_decoders_args(), default_free=decode_free_output + ) + + @staticmethod + def _action_decoders_args(): + """Generate the arguments for the action KVDecoders.""" + _decoders = { + "drop": decode_flag, + "lb_output": decode_int, + "trunc": decode_int, + "recirc": decode_int, + "userspace": nested_kv_decoder( + KVDecoders( + { + "pid": decode_int, + "sFlow": nested_kv_decoder( + KVDecoders( + { + "vid": decode_int, + "pcp": decode_int, + "output": decode_int, + } + ) + ), + "slow_path": decode_default, + "flow_sample": nested_kv_decoder( + KVDecoders( + { + "probability": decode_int, + "collector_sed_id": decode_int, + "obs_domain_id": decode_int, + "obs_point_id": decode_int, + "output_port": decode_default, + "ingress": decode_flag, + "egress": decode_flag, + } + ) + ), + "ipfix": nested_kv_decoder( + KVDecoders( + { + "output_port": decode_default, + } + ) + ), + "controller": nested_kv_decoder( + KVDecoders( + { + "reason": decode_int, + "dont_send": decode_int, + "continuation": decode_int, + "recirc_id": decode_int, + "rule_cookie": decode_int, + "controller_id": decode_int, + "max_len": decode_int, + } + ) + ), + "userdata": decode_default, + "actions": decode_flag, + "tunnel_out_port": decode_default, + "push_eth": nested_kv_decoder( + KVDecoders( + { + "src": EthMask, + "dst": EthMask, + "type": decode_int, + } + ) + ), + "pop_eth": decode_flag, + } + ) + ), + "set": nested_kv_decoder( + KVDecoders(ODPFlow._field_decoders_args()) + ), + "push_vlan": nested_kv_decoder( + KVDecoders( + { + "vid": decode_int, + "pcp": decode_int, + "cfi": decode_int, + "tpid": decode_int, + } + ) + ), + "pop_vlan": decode_flag, + "push_nsh": nested_kv_decoder( + KVDecoders( + { + "flags": decode_int, + "ttl": decode_int, + "mdtype": decode_int, + "np": decode_int, + "spi": decode_int, + "si": decode_int, + "c1": decode_int, + "c2": decode_int, + "c3": decode_int, + "c4": decode_int, + "md2": decode_int, + } + ) + ), + "pop_nsh": decode_flag, + "tnl_pop": decode_int, + "ct_clear": decode_flag, + "ct": nested_kv_decoder( + KVDecoders( + { + "commit": decode_flag, + "force_commit": decode_flag, + "zone": decode_int, + "mark": Mask32, + "label": Mask128, + "helper": decode_default, + "timeout": decode_default, + "nat": decode_nat, + } + ) + ), + **ODPFlow._tnl_action_decoder_args(), + } + + _decoders["clone"] = nested_kv_decoder( + KVDecoders(decoders=_decoders, default_free=decode_free_output) + ) + + return { + **_decoders, + "sample": nested_kv_decoder( + KVDecoders( + { + "sample": (lambda x: float(x.strip("%"))), + "actions": nested_kv_decoder( + KVDecoders( + decoders=_decoders, + default_free=decode_free_output, + ) + ), + } + ) + ), + "check_pkt_len": nested_kv_decoder( + KVDecoders( + { + "size": decode_int, + "gt": nested_kv_decoder( + KVDecoders( + decoders=_decoders, + default_free=decode_free_output, + ) + ), + "le": nested_kv_decoder( + KVDecoders( + decoders=_decoders, + default_free=decode_free_output, + ) + ), + } + ) + ), + } + + @staticmethod + def _tnl_action_decoder_args(): + """Generate the decoder arguments for the tunnel actions.""" + return { + "tnl_push": nested_kv_decoder( + KVDecoders( + { + "tnl_port": decode_default, + "header": nested_kv_decoder( + KVDecoders( + { + "size": decode_int, + "type": decode_int, + "eth": nested_kv_decoder( + KVDecoders( + { + "src": EthMask, + "dst": EthMask, + "dl_type": decode_int, + } + ) + ), + "ipv4": nested_kv_decoder( + KVDecoders( + { + "src": IPMask, + "dst": IPMask, + "proto": decode_int, + "tos": decode_int, + "ttl": decode_int, + "frag": decode_int, + } + ) + ), + "ipv6": nested_kv_decoder( + KVDecoders( + { + "src": IPMask, + "dst": IPMask, + "label": decode_int, + "proto": decode_int, + "tclass": decode_int, + "hlimit": decode_int, + } + ) + ), + "udp": nested_kv_decoder( + KVDecoders( + { + "src": decode_int, + "dst": decode_int, + "dsum": Mask16, + } + ) + ), + "vxlan": nested_kv_decoder( + KVDecoders( + { + "flags": decode_int, + "vni": decode_int, + } + ) + ), + "geneve": nested_kv_decoder( + KVDecoders( + { + "oam": decode_flag, + "crit": decode_flag, + "vni": decode_int, + "options": partial( + decode_geneve, False + ), + } + ) + ), + "gre": decode_tnl_gre, + "erspan": nested_kv_decoder( + KVDecoders( + { + "ver": decode_int, + "sid": decode_int, + "idx": decode_int, + "dir": decode_int, + "hwid": decode_int, + } + ) + ), + "gtpu": nested_kv_decoder( + KVDecoders( + { + "flags": decode_int, + "msgtype": decode_int, + "teid": decode_int, + } + ) + ), + } + ) + ), + "out_port": decode_default, + } + ) + ) + } + + @staticmethod + def _gen_match_decoders(): + """Generate the match KVDecoders.""" + return KVDecoders(ODPFlow._match_decoders_args()) + + @staticmethod + def _match_decoders_args(): + """Generate the arguments for the match KVDecoders.""" + return { + **ODPFlow._field_decoders_args(), + "encap": nested_kv_decoder( + KVDecoders(ODPFlow._field_decoders_args()) + ), + } + + @staticmethod + def _field_decoders_args(): + """Generate the decoder arguments for the match fields.""" + return { + "skb_priority": Mask32, + "skb_mark": Mask32, + "recirc_id": decode_int, + "dp_hash": Mask32, + "ct_state": decode_default, + "ct_zone": Mask16, + "ct_mark": Mask32, + "ct_label": Mask128, + "ct_tuple4": nested_kv_decoder( + KVDecoders( + { + "src": IPMask, + "dst": IPMask, + "proto": Mask8, + "tcp_src": Mask16, + "tcp_dst": Mask16, + } + ) + ), + "ct_tuple6": nested_kv_decoder( + KVDecoders( + { + "src": IPMask, + "dst": IPMask, + "proto": Mask8, + "tcp_src": Mask16, + "tcp_dst": Mask16, + } + ) + ), + "tunnel": nested_kv_decoder( + KVDecoders( + { + "tun_id": Mask64, + "src": IPMask, + "dst": IPMask, + "ipv6_src": IPMask, + "ipv6_dst": IPMask, + "tos": Mask8, + "ttl": Mask8, + "tp_src": Mask16, + "tp_dst": Mask16, + "erspan": nested_kv_decoder( + KVDecoders( + { + "ver": Mask8, + "idx": Mask32, + "sid": decode_int, + "dir": Mask8, + "hwid": Mask8, + } + ) + ), + "vxlan": nested_kv_decoder( + KVDecoders( + { + "gbp": nested_kv_decoder( + KVDecoders( + { + "id": Mask16, + "flags": Mask8, + } + ) + ) + } + ) + ), + "geneve": partial(decode_geneve, True), + "gtpu": nested_kv_decoder( + KVDecoders( + { + "flags": Mask8, + "msgtype": Mask8, + } + ) + ), + "flags": decode_default, + } + ) + ), + "in_port": decode_default, + "eth": nested_kv_decoder( + KVDecoders( + { + "src": EthMask, + "dst": EthMask, + } + ) + ), + "vlan": nested_kv_decoder( + KVDecoders( + { + "vid": Mask16, + "pcp": Mask16, + "cfi": Mask16, + } + ) + ), + "eth_type": Mask16, + "mpls": nested_kv_decoder( + KVDecoders( + { + "label": Mask32, + "tc": Mask32, + "ttl": Mask32, + "bos": Mask32, + } + ) + ), + "ipv4": nested_kv_decoder( + KVDecoders( + { + "src": IPMask, + "dst": IPMask, + "proto": Mask8, + "tos": Mask8, + "ttl": Mask8, + "frag": decode_default, + } + ) + ), + "ipv6": nested_kv_decoder( + KVDecoders( + { + "src": IPMask, + "dst": IPMask, + "label": decode_mask(20), + "proto": Mask8, + "tclass": Mask8, + "hlimit": Mask8, + "frag": decode_default, + } + ) + ), + "tcp": nested_kv_decoder( + KVDecoders( + { + "src": Mask16, + "dst": Mask16, + } + ) + ), + "tcp_flags": decode_default, + "udp": nested_kv_decoder( + KVDecoders( + { + "src": Mask16, + "dst": Mask16, + } + ) + ), + "sctp": nested_kv_decoder( + KVDecoders( + { + "src": Mask16, + "dst": Mask16, + } + ) + ), + "icmp": nested_kv_decoder( + KVDecoders( + { + "type": Mask8, + "code": Mask8, + } + ) + ), + "icmpv6": nested_kv_decoder( + KVDecoders( + { + "type": Mask8, + "code": Mask8, + } + ) + ), + "arp": nested_kv_decoder( + KVDecoders( + { + "sip": IPMask, + "tip": IPMask, + "op": Mask16, + "sha": EthMask, + "tha": EthMask, + } + ) + ), + "nd": nested_kv_decoder( + KVDecoders( + { + "target": IPMask, + "sll": EthMask, + "tll": EthMask, + } + ) + ), + "nd_ext": nested_kv_decoder( + KVDecoders( + { + "nd_reserved": Mask32, + "nd_options_type": Mask8, + } + ) + ), + "packet_type": nested_kv_decoder( + KVDecoders( + { + "ns": Mask16, + "id": Mask16, + } + ) + ), + "nsh": nested_kv_decoder( + KVDecoders( + { + "flags": Mask8, + "mdtype": Mask8, + "np": Mask8, + "spi": Mask32, + "si": Mask8, + "c1": Mask32, + "c2": Mask32, + "c3": Mask32, + "c4": Mask32, + } + ) + ), + } + + +def decode_geneve(mask, value): + """Decode geneve options. + Used for both tnl_push(header(geneve(options()))) action and + tunnel(geneve()) match. + + It has the following format: + + {class=0xffff,type=0x80,len=4,0xa} + + Args: + mask (bool): Whether masking is supported. + value (str): The value to decode. + """ + if mask: + decoders = { + "class": Mask16, + "type": Mask8, + "len": Mask8, + } + + def free_decoder(value): + return "data", Mask128(value) + + else: + decoders = { + "class": decode_int, + "type": decode_int, + "len": decode_int, + } + + def free_decoder(value): + return "data", decode_int(value) + + result = [] + for opts in re.findall(r"{.*?}", value): + result.append( + decode_nested_kv( + KVDecoders(decoders=decoders, default_free=free_decoder), + opts.strip("{}"), + ) + ) + return result + + +def decode_tnl_gre(value): + """ + Decode tnl_push(header(gre())) action. + + It has the following format: + + gre((flags=0x2000,proto=0x6558),key=0x1e241)) + + Args: + value (str): The value to decode. + """ + return decode_nested_kv( + KVDecoders( + { + "flags": decode_int, + "proto": decode_int, + "key": decode_int, + "csum": decode_int, + "seq": decode_int, + } + ), + value.replace("(", "").replace(")", ""), + ) From patchwork Fri Jul 8 18:03:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654345 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MnY9J1Oe; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh4q0fR6z9ryY for ; Sat, 9 Jul 2022 04:04:43 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 0210661569; Fri, 8 Jul 2022 18:04:40 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 0210661569 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MnY9J1Oe X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Qwl_XGakg79U; Fri, 8 Jul 2022 18:04:38 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id B1B426150F; Fri, 8 Jul 2022 18:04:16 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org B1B426150F Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3DDE9C0035; Fri, 8 Jul 2022 18:04:15 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 05186C007B for ; Fri, 8 Jul 2022 18:04:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 8B850426B1 for ; Fri, 8 Jul 2022 18:03:53 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 8B850426B1 Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MnY9J1Oe X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id V_ALVFuH8uub for ; Fri, 8 Jul 2022 18:03:48 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 6699F425EB Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 6699F425EB for ; Fri, 8 Jul 2022 18:03:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303425; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kXmAzRqECsSnG7jdTfPPXz3TWQxUN+6SJEwdPuL2usA=; b=MnY9J1OeZkpYI9yu/VMNCK79lzOPYLtFfn8NNRgErGZEbqoCBbBqLhwYKnzHobXn1eSMDv 7VukOgiV5lV2TLWRFb9mIHlQUTwP8lNgFUfAjhcO28iH8z5AD01D3tU7kXRBINP9DPGWNW RqR7DGxq7+5UeZnYfXcYih8M6gCvhW0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-594-c4uFcV3ROyGnbhmXapRNTA-1; Fri, 08 Jul 2022 14:03:36 -0400 X-MC-Unique: c4uFcV3ROyGnbhmXapRNTA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B8ABC85A587; Fri, 8 Jul 2022 18:03:34 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8C38F2026D64; Fri, 8 Jul 2022 18:03:33 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:08 +0200 Message-Id: <20220708180316.2852046-10-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 09/17] python: add flow filtering syntax X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Based on pyparsing, create a very simple filtering syntax. It supports basic logic statements (and, &, or, ||, not, !), numerical operations (<, >), equality (=, !=), and masking (~=). The latter is only supported in certain fields (IntMask, EthMask, IPMask). Masking operation is semantically equivalent to "includes", therefore: ip_src ~= 192.168.1.1 means that ip_src field is either a host IP address equal to 192.168.1.1 or an IPMask that includes it (e.g: 192.168.1.1/24). Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- .ci/linux-prepare.sh | 2 +- .cirrus.yml | 2 +- Documentation/topics/language-bindings.rst | 2 +- debian/control | 2 +- python/automake.mk | 1 + python/ovs/flow/__init__.py | 2 +- python/ovs/flow/filter.py | 261 +++++++++++++++++++++ python/setup.py | 2 +- rhel/openvswitch-fedora.spec.in | 2 +- 9 files changed, 269 insertions(+), 7 deletions(-) create mode 100644 python/ovs/flow/filter.py diff --git a/.ci/linux-prepare.sh b/.ci/linux-prepare.sh index 190e69b85..a0635cf56 100755 --- a/.ci/linux-prepare.sh +++ b/.ci/linux-prepare.sh @@ -30,7 +30,7 @@ pip3 install --disable-pip-version-check --user \ pip3 install --user 'meson==0.49.2' # Install python dependencies. -pip3 install --user netaddr +pip3 install --user netaddr pyparsing if [ "$M32" ]; then # Installing 32-bit libraries. diff --git a/.cirrus.yml b/.cirrus.yml index 5a3ca2589..7c8bec442 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -19,7 +19,7 @@ freebsd_build_task: - pkg update -f - pkg install -y ${DEPENDENCIES} $(pkg search -xq "^py3[0-9]+-(${PY_DEPS})-[0-9]+" | xargs) - - python3 -m pip install --user netaddr + - python3 -m pip install --user netaddr pyparsing configure_script: - ./boot.sh diff --git a/Documentation/topics/language-bindings.rst b/Documentation/topics/language-bindings.rst index 5025f069f..414f7c73f 100644 --- a/Documentation/topics/language-bindings.rst +++ b/Documentation/topics/language-bindings.rst @@ -47,7 +47,7 @@ required dependencies, run: $ pip install ovs[flow] -or install `python3-netaddr`. +or install `python3-netaddr` and `python3-pyparsing`. __ https://github.com/openvswitch/ovs/tree/master/python/ovs diff --git a/debian/control b/debian/control index e23d55ad2..21e48d1d6 100644 --- a/debian/control +++ b/debian/control @@ -174,7 +174,7 @@ Package: python3-openvswitch Architecture: all Section: python Depends: ${misc:Depends}, ${python3:Depends} -Suggests: python3-netaddr +Suggests: python3-netaddr, python3-pyparsing Description: Python bindings for Open vSwitch Open vSwitch is a production quality, multilayer, software-based, Ethernet virtual switch. It is designed to enable massive network diff --git a/python/automake.mk b/python/automake.mk index 79affdc75..aa3adb703 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -28,6 +28,7 @@ ovs_pyfiles = \ python/ovs/fcntl_win.py \ python/ovs/flow/__init__.py \ python/ovs/flow/decoders.py \ + python/ovs/flow/filter.py \ python/ovs/flow/flow.py \ python/ovs/flow/kv.py \ python/ovs/flow/list.py \ diff --git a/python/ovs/flow/__init__.py b/python/ovs/flow/__init__.py index 68453cc8e..7c9c13fa9 100644 --- a/python/ovs/flow/__init__.py +++ b/python/ovs/flow/__init__.py @@ -1,6 +1,6 @@ """ Global flow library entrypoint. """ -for libname in ["netaddr"]: +for libname in ["netaddr", "pyparsing"]: try: lib = __import__(libname) except ModuleNotFoundError as e: diff --git a/python/ovs/flow/filter.py b/python/ovs/flow/filter.py new file mode 100644 index 000000000..f5ba4eae4 --- /dev/null +++ b/python/ovs/flow/filter.py @@ -0,0 +1,261 @@ +""" Defines a Flow Filtering syntax. +""" +import pyparsing as pp +import netaddr +from functools import reduce +from operator import and_, or_ + +from ovs.flow.decoders import ( + decode_default, + decode_int, + Decoder, + IPMask, + EthMask, +) + + +class EvaluationResult(object): + """An EvaluationResult is the result of an evaluation. It contains the + boolean result and the list of key-values that were evaluated. + + Note that since boolean operations (and, not, or) are based only on + __bool__ we use bitwise alternatives (&, ||, ~). + """ + + def __init__(self, result, *kv): + self.result = result + self.kv = kv if kv else list() + + def __and__(self, other): + """Logical and operation.""" + return EvaluationResult( + self.result and other.result, *self.kv, *other.kv + ) + + def __or__(self, other): + """Logical or operation.""" + return EvaluationResult( + self.result or other.result, *self.kv, *other.kv + ) + + def __invert__(self): + """Logical not operation.""" + return EvaluationResult(not self.result, *self.kv) + + def __bool__(self): + """Boolean operation.""" + return self.result + + def __repr__(self): + return "{} [{}]".format(self.result, self.kv) + + +class ClauseExpression(object): + """ A clause expression represents a specific expression in the filter. + + A clause has the following form: + [field] [operator] [value] + + Valid operators are: + = (equality) + != (inequality) + < (arithmetic less-than) + > (arithmetic more-than) + ~= (__contains__) + + When evaluated, the clause finds what relevant part of the flow to use for + evaluation, tries to translate the clause value to the relevant type and + performs the clause operation. + + Attributes: + field (str): The flow field used in the clause. + operator (str): The flow operator used in the clause. + value (str): The value to perform the comparison against. + """ + operators = {} + type_decoders = { + int: decode_int, + netaddr.IPAddress: IPMask, + netaddr.EUI: EthMask, + bool: bool, + } + + def __init__(self, tokens): + self.field = tokens[0] + self.value = "" + self.operator = "" + + if len(tokens) > 1: + self.operator = tokens[1] + self.value = tokens[2] + + def __repr__(self): + return "{}(field: {}, operator: {}, value: {})".format( + self.__class__.__name__, self.field, self.operator, self.value + ) + + def _find_data_in_kv(self, kv_list): + """Find a KeyValue for evaluation in a list of KeyValue. + + Args: + kv_list (list[KeyValue]): list of KeyValue to look into. + + Returns: + If found, tuple (kv, data) where kv is the KeyValue that matched + and data is the data to be used for evaluation. None if not found. + """ + key_parts = self.field.split(".") + field = key_parts[0] + kvs = [kv for kv in kv_list if kv.key == field] + if not kvs: + return None + + for kv in kvs: + if kv.key == self.field: + # exact match + return (kv, kv.value) + if len(key_parts) > 1: + data = kv.value + for subkey in key_parts[1:]: + try: + data = data.get(subkey) + except Exception: + data = None + break + if not data: + break + if data: + return (kv, data) + return None + + def _find_keyval_to_evaluate(self, flow): + """Finds the key-value and data to use for evaluation on a flow. + + Args: + flow(Flow): The flow where the lookup is performed. + + Returns: + If found, tuple (kv, data) where kv is the KeyValue that matched + and data is the data to be used for evaluation. None if not found. + + """ + for section in flow.sections: + data = self._find_data_in_kv(section.data) + if data: + return data + return None + + def evaluate(self, flow): + """Returns whether the clause is satisfied by the flow. + + Args: + flow (Flow): the flow to evaluate. + """ + result = self._find_keyval_to_evaluate(flow) + + if not result: + return EvaluationResult(False) + + keyval, data = result + + if not self.value and not self.operator: + # just asserting the existance of the key + return EvaluationResult(True, keyval) + + # Decode the value based on the type of data + if isinstance(data, Decoder): + decoder = data.__class__ + else: + decoder = self.type_decoders.get(data.__class__) or decode_default + + decoded_value = decoder(self.value) + + if self.operator == "=": + return EvaluationResult(decoded_value == data, keyval) + elif self.operator == "<": + return EvaluationResult(data < decoded_value, keyval) + elif self.operator == ">": + return EvaluationResult(data > decoded_value, keyval) + elif self.operator == "~=": + return EvaluationResult(decoded_value in data, keyval) + + +class BoolNot(object): + def __init__(self, t): + self.op, self.args = t[0] + + def __repr__(self): + return "NOT({})".format(self.args) + + def evaluate(self, flow): + return ~self.args.evaluate(flow) + + +class BoolAnd(object): + def __init__(self, pattern): + self.args = pattern[0][0::2] + + def __repr__(self): + return "AND({})".format(self.args) + + def evaluate(self, flow): + return reduce(and_, [arg.evaluate(flow) for arg in self.args]) + + +class BoolOr(object): + def __init__(self, pattern): + self.args = pattern[0][0::2] + + def evaluate(self, flow): + return reduce(or_, [arg.evaluate(flow) for arg in self.args]) + + def __repr__(self): + return "OR({})".format(self.args) + + +class OFFilter(object): + """OFFilter represents an Open vSwitch Flow Filter. + + It is built with a filter expression string composed of logically-separated + clauses (see ClauseExpression for details on the clause syntax). + + Args: + expr(str): String filter expression. + """ + w = pp.Word(pp.alphanums + "." + ":" + "_" + "/" + "-") + operators = ( + pp.Literal("=") + | pp.Literal("~=") + | pp.Literal("<") + | pp.Literal(">") + | pp.Literal("!=") + ) + + clause = (w + operators + w) | w + clause.setParseAction(ClauseExpression) + + statement = pp.infixNotation( + clause, + [ + ("!", 1, pp.opAssoc.RIGHT, BoolNot), + ("not", 1, pp.opAssoc.RIGHT, BoolNot), + ("&&", 2, pp.opAssoc.LEFT, BoolAnd), + ("and", 2, pp.opAssoc.LEFT, BoolAnd), + ("||", 2, pp.opAssoc.LEFT, BoolOr), + ("or", 2, pp.opAssoc.LEFT, BoolOr), + ], + ) + + def __init__(self, expr): + self._filter = self.statement.parseString(expr) + + def evaluate(self, flow): + """Evaluate whether the flow satisfies the filter. + + Args: + flow(Flow): a openflow or datapath flow. + + Returns: + An EvaluationResult with the result of the evaluation. + """ + return self._filter[0].evaluate(flow) diff --git a/python/setup.py b/python/setup.py index d7802fa8e..1440c93cc 100644 --- a/python/setup.py +++ b/python/setup.py @@ -89,7 +89,7 @@ setup_args = dict( cmdclass={'build_ext': try_build_ext}, install_requires=['sortedcontainers'], extras_require={':sys_platform == "win32"': ['pywin32 >= 1.0'], - 'flow': ['netaddr']}, + 'flow': ['netaddr', 'pyparsing']}, ) try: diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch-fedora.spec.in index a40a915f4..57471e44a 100644 --- a/rhel/openvswitch-fedora.spec.in +++ b/rhel/openvswitch-fedora.spec.in @@ -116,7 +116,7 @@ Summary: Open vSwitch python3 bindings License: ASL 2.0 BuildArch: noarch Requires: python3 -Suggests: python3-netaddr +Suggests: python3-netaddr python3-pyparsing %{?python_provide:%python_provide python3-openvswitch = %{version}-%{release}} %description -n python3-openvswitch From patchwork Fri Jul 8 18:03:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654337 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=bzJY4i6Z; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh481hmCz9ryY for ; Sat, 9 Jul 2022 04:04:08 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 00C70416A5; Fri, 8 Jul 2022 18:04:04 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 00C70416A5 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=bzJY4i6Z X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IY3k_1G7pjZB; Fri, 8 Jul 2022 18:04:03 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 65ECC416C3; Fri, 8 Jul 2022 18:03:55 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 65ECC416C3 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 31C27C0096; Fri, 8 Jul 2022 18:03:49 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 114CBC008C for ; Fri, 8 Jul 2022 18:03:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 92D4F6146C for ; Fri, 8 Jul 2022 18:03:41 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 92D4F6146C Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=bzJY4i6Z X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KVIiKV40pKD4 for ; Fri, 8 Jul 2022 18:03:40 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 8FF9461449 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id 8FF9461449 for ; Fri, 8 Jul 2022 18:03:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303419; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qY0AIPVesJWRuPOHXFPbU439X/+Ihavpb2MAwPNZgXs=; b=bzJY4i6ZiglkMZfoDe/WcGg/h/MpO2ZkkGwsBH4sKt042GnBCuG+MmqibeLWkRLjDDvSxf beLH7FZ584gCbhkq/W095uerZ73sezAHa5OS4YzPPt8RQm8tIHBLr8FLfpebLLQM+rrvbT ZpjHCXf26HboNj5slOpER2p7u3vSGMo= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-397-dWPz_k1gOJqjT8YB5Jt-Iw-1; Fri, 08 Jul 2022 14:03:36 -0400 X-MC-Unique: dWPz_k1gOJqjT8YB5Jt-Iw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 15AA829AA388; Fri, 8 Jul 2022 18:03:36 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0CB392026D64; Fri, 8 Jul 2022 18:03:34 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:09 +0200 Message-Id: <20220708180316.2852046-11-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 10/17] python: add a json encoder to flow fields X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The json encoder can be used to convert Flows to json. Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/ovs/flow/decoders.py | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/python/ovs/flow/decoders.py b/python/ovs/flow/decoders.py index 73d28e057..7378d4176 100644 --- a/python/ovs/flow/decoders.py +++ b/python/ovs/flow/decoders.py @@ -5,6 +5,7 @@ A decoder is generally a callable that accepts a string and returns the value object. """ +import json import netaddr import re @@ -522,3 +523,16 @@ def decode_nat(value): result[flag] = True return result + + +class FlowEncoder(json.JSONEncoder): + """FlowEncoder is a json.JSONEncoder instance that can be used to + serialize flow fields.""" + + def default(self, obj): + if isinstance(obj, Decoder): + return obj.to_json() + elif isinstance(obj, netaddr.IPAddress): + return str(obj) + + return json.JSONEncoder.default(self, obj) From patchwork Fri Jul 8 18:03:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654338 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=aSVEAgqE; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh486w0Pz9sFx for ; Sat, 9 Jul 2022 04:04:08 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id EFA814270E; Fri, 8 Jul 2022 18:04:06 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org EFA814270E Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=aSVEAgqE X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Fbq7QlNPy72v; Fri, 8 Jul 2022 18:04:04 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 28663426AD; Fri, 8 Jul 2022 18:03:53 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 28663426AD Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D50B6C008E; Fri, 8 Jul 2022 18:03:47 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 185C0C0080 for ; Fri, 8 Jul 2022 18:03:43 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 826658485A for ; Fri, 8 Jul 2022 18:03:41 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 826658485A Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=aSVEAgqE X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2-MOsJZSWbDr for ; Fri, 8 Jul 2022 18:03:40 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 2B34C8483F Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 2B34C8483F for ; Fri, 8 Jul 2022 18:03:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303419; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lkgyixta2bmSX762tvC+f3WKZ9A1G4YlzsFVQX/nnLg=; b=aSVEAgqE9yLS94BNa+vRD/+Zqj/c4c+zJUrpk4+A4qhKMQ09UzFewnPX+bJkXnTqdNLm7M 7qtBOZEiJmwtS3TWB2ZGfqMaW2SIHXzBi2sRMoPAVcFFDU/1Z68wDaGo2IjAnXQojM/NeA ozrzavzZ7B6AX7shN/183ejBOOCwwz8= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-607--oHZZq_JNd6sRCsmJks2Og-1; Fri, 08 Jul 2022 14:03:37 -0400 X-MC-Unique: -oHZZq_JNd6sRCsmJks2Og-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5A8B51C05130; Fri, 8 Jul 2022 18:03:37 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5407A2026D64; Fri, 8 Jul 2022 18:03:36 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:10 +0200 Message-Id: <20220708180316.2852046-12-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 11/17] tests: verify flows in ofp-actions are parseable X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Create a small helper script and check that flows used in ofp-actions.at are parseable. Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- tests/automake.mk | 2 ++ tests/ofp-actions.at | 18 +++++++++++++++++ tests/test-ofparse.py | 45 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 65 insertions(+) create mode 100755 tests/test-ofparse.py diff --git a/tests/automake.mk b/tests/automake.mk index b29cb783e..ae99f92bc 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -19,6 +19,7 @@ EXTRA_DIST += \ $(OVSDB_CLUSTER_TESTSUITE) \ tests/atlocal.in \ $(srcdir)/package.m4 \ + $(srcdir)/tests/test-ofparse.py \ $(srcdir)/tests/testsuite \ $(srcdir)/tests/testsuite.patch @@ -524,6 +525,7 @@ CHECK_PYFILES = \ tests/flowgen.py \ tests/mfex_fuzzy.py \ tests/ovsdb-monitor-sort.py \ + tests/test-ofparse.py \ tests/test-daemon.py \ tests/test-json.py \ tests/test-jsonrpc.py \ diff --git a/tests/ofp-actions.at b/tests/ofp-actions.at index 9d820eba6..40a23bb15 100644 --- a/tests/ofp-actions.at +++ b/tests/ofp-actions.at @@ -329,6 +329,7 @@ AT_CAPTURE_FILE([experr]) AT_CHECK( [ovs-ofctl '-vPATTERN:console:%c|%p|%m' parse-actions OpenFlow10 < input.txt], [0], [expout], [experr]) +AT_CHECK([cat expout | grep 'actions=' | test-ofparse.py]) AT_CLEANUP AT_SETUP([OpenFlow 1.0 "instruction" translations]) @@ -359,6 +360,7 @@ AT_CAPTURE_FILE([experr]) AT_CHECK( [ovs-ofctl '-vPATTERN:console:%c|%p|%m' parse-instructions OpenFlow10 < input.txt], [0], [expout], [experr]) +AT_CHECK([cat expout | grep 'actions=' | test-ofparse.py]) AT_CLEANUP AT_SETUP([OpenFlow 1.1 action translation]) @@ -502,6 +504,7 @@ AT_CAPTURE_FILE([experr]) AT_CHECK( [ovs-ofctl '-vPATTERN:console:%c|%p|%m' parse-actions OpenFlow11 < input.txt], [0], [expout], [experr]) +AT_CHECK([cat expout | grep 'actions=' | test-ofparse.py]) AT_CLEANUP AT_SETUP([OpenFlow 1.1 instruction translation]) @@ -737,6 +740,7 @@ AT_CAPTURE_FILE([experr]) AT_CHECK( [ovs-ofctl '-vPATTERN:console:%c|%p|%m' parse-actions OpenFlow12 < input.txt], [0], [expout], [experr]) +AT_CHECK([cat expout | grep 'actions=' | test-ofparse.py]) AT_CLEANUP dnl Our primary goal here is to verify OpenFlow 1.3-specific changes, @@ -798,6 +802,7 @@ AT_CAPTURE_FILE([experr]) AT_CHECK( [ovs-ofctl '-vPATTERN:console:%c|%p|%m' parse-actions OpenFlow13 < input.txt], [0], [expout], [experr]) +AT_CHECK([cat expout | grep 'actions=' | test-ofparse.py]) AT_CLEANUP dnl Our primary goal here is to verify that OpenFlow 1.5-specific changes, @@ -827,17 +832,20 @@ AT_CAPTURE_FILE([experr]) AT_CHECK( [ovs-ofctl '-vPATTERN:console:%c|%p|%m' parse-actions OpenFlow15 < input.txt], [0], [expout], [experr]) +AT_CHECK([cat expout | grep 'actions=' | test-ofparse.py]) AT_CLEANUP AT_SETUP([ofp-actions - inconsistent MPLS actions]) OVS_VSWITCHD_START dnl OK: Use fin_timeout action on TCP flow AT_CHECK([ovs-ofctl -O OpenFlow11 -vwarn add-flow br0 'tcp actions=fin_timeout(idle_timeout=1)']) +AT_CHECK([echo 'tcp actions=fin_timeout(idle_timeout=1)' | test-ofparse.py]) dnl Bad: Use fin_timeout action on TCP flow that has been converted to MPLS AT_CHECK([ovs-ofctl -O OpenFlow11 -vwarn add-flow br0 'tcp actions=push_mpls:0x8847,fin_timeout(idle_timeout=1)'], [1], [], [dnl ovs-ofctl: none of the usable flow formats (OpenFlow10,NXM) is among the allowed flow formats (OpenFlow11) ]) +AT_CHECK([echo 'tcp actions=push_mpls:0x8847,fin_timeout(idle_timeout=1)' | test-ofparse.py]) OVS_VSWITCHD_STOP AT_CLEANUP @@ -853,6 +861,8 @@ AT_CHECK([ovs-ofctl -O OpenFlow10 dump-flows br0 | ofctl_strip], [0], [dnl NXST_FLOW reply: mpls actions=load:0xa->OXM_OF_MPLS_LABEL[[]] ]) +AT_CHECK([echo 'mpls actions=set_field:10->mpls_label' | test-ofparse.py]) +AT_CHECK([echo 'mpls actions=load:0xa->OXM_OF_MPLS_LABEL[[]]'| test-ofparse.py]) OVS_VSWITCHD_STOP AT_CLEANUP @@ -862,14 +872,17 @@ OVS_VSWITCHD_START dnl OpenFlow 1.0 has an "enqueue" action. For OpenFlow 1.1+, we translate dnl it to a series of actions that accomplish the same thing. AT_CHECK([ovs-ofctl -O OpenFlow10 add-flow br0 'actions=enqueue(123,456)']) +AT_CHECK([echo 'actions=enqueue(123,456)' | test-ofparse.py]) AT_CHECK([ovs-ofctl -O OpenFlow10 dump-flows br0 | ofctl_strip], [0], [dnl NXST_FLOW reply: actions=enqueue:123:456 ]) +AT_CHECK([echo 'actions=enqueue:123:456' | test-ofparse.py]) AT_CHECK([ovs-ofctl -O OpenFlow13 dump-flows br0 | ofctl_strip], [0], [dnl OFPST_FLOW reply (OF1.3): reset_counts actions=set_queue:456,output:123,pop_queue ]) +AT_CHECK([echo 'actions=set_queue:456,output:123,pop_queue' | test-ofparse.py]) OVS_VSWITCHD_STOP AT_CLEANUP @@ -887,6 +900,8 @@ AT_CHECK([ovs-ofctl -O OpenFlow11 dump-flows br0 | ofctl_strip], [0], [dnl OFPST_FLOW reply (OF1.1): ip actions=mod_nw_ttl:123 ]) +AT_CHECK([echo 'ip,actions=mod_nw_ttl:123' | test-ofparse.py]) +AT_CHECK([echo 'ip actions=load:0x7b->NXM_NX_IP_TTL[[]]' | test-ofparse.py]) OVS_VSWITCHD_STOP AT_CLEANUP @@ -898,10 +913,12 @@ dnl OpenFlow 1.1, but no other version, has a "mod_nw_ecn" action. dnl Check that we translate it properly for OF1.0 and OF1.2. dnl (OF1.3+ should be the same as OF1.2.) AT_CHECK([ovs-ofctl -O OpenFlow11 add-flow br0 'ip,actions=mod_nw_ecn:2']) +AT_CHECK([echo 'ip,actions=mod_nw_ecn:2' | test-ofparse.py]) AT_CHECK([ovs-ofctl -O OpenFlow10 dump-flows br0 | ofctl_strip], [0], [dnl NXST_FLOW reply: ip actions=load:0x2->NXM_NX_IP_ECN[[]] ]) +AT_CHECK([echo 'ip actions=load:0x2->NXM_NX_IP_ECN[[]]' | test-ofparse.py]) AT_CHECK([ovs-ofctl -O OpenFlow11 dump-flows br0 | ofctl_strip], [0], [dnl OFPST_FLOW reply (OF1.1): ip actions=mod_nw_ecn:2 @@ -910,6 +927,7 @@ AT_CHECK([ovs-ofctl -O OpenFlow12 dump-flows br0 | ofctl_strip], [0], [dnl OFPST_FLOW reply (OF1.2): ip actions=set_field:2->nw_ecn ]) +AT_CHECK([echo 'ip actions=set_field:2->nw_ecn' | test-ofparse.py]) dnl Check that OF1.2+ set_field to set ECN is translated into the OF1.1 dnl mod_nw_ecn action. diff --git a/tests/test-ofparse.py b/tests/test-ofparse.py new file mode 100755 index 000000000..10f0fe1a1 --- /dev/null +++ b/tests/test-ofparse.py @@ -0,0 +1,45 @@ +#!/usr/bin/env python3 +# Copyright (c) 2022 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""test-ofparse reads flows from stdin and tries to parse them using +the python flow parsing library. +""" + +import fileinput +import sys + +try: + from ovs.flow.ofp import OFPFlow +except ImportError: + sys.exit(0) + + +def main(): + for flow in fileinput.input(): + try: + result_flow = OFPFlow(flow) + if flow != str(result_flow): + print("in: {}".format(flow)) + print("out: {}".format(str(result_flow))) + raise ValueError("Flow conversion back to string failed") + except Exception as e: + print(e) + return 1 + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) From patchwork Fri Jul 8 18:03:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654341 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=e4jfGxx0; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh4S1g6gz9ryY for ; Sat, 9 Jul 2022 04:04:24 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 239E84174B; Fri, 8 Jul 2022 18:04:22 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 239E84174B Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=e4jfGxx0 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4DW4Wgfeurjo; Fri, 8 Jul 2022 18:04:15 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id B3EA641681; Fri, 8 Jul 2022 18:04:02 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org B3EA641681 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D38D9C0080; Fri, 8 Jul 2022 18:03:59 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0263CC0035 for ; Fri, 8 Jul 2022 18:03:58 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 4CBD9416A0 for ; Fri, 8 Jul 2022 18:03:49 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 4CBD9416A0 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Osn4vYn9IOAA for ; Fri, 8 Jul 2022 18:03:47 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 1A4004167F Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 1A4004167F for ; Fri, 8 Jul 2022 18:03:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303425; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Cn3zSKYcMQjzmmAqENww5YQT0NL6c40ahDrptiC4+A8=; b=e4jfGxx0PjClu7D8z3dHT9KP7UOp/k4rOg/1S22JuWiu2FLYFnZ8GxVH6Zzs8sY16P5Fqu EGhHXSEd0o4hRfZUiyxjo6EoOugwctbnWGz0/nrqnk4+i/arHwUnimjzNJVEz5ohtjeFKj FpEOdHfKPfl12YrzX2kEMfuSaQsxJFU= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-600-ZFPM_fh_Oj-E7FIoTrgTNQ-1; Fri, 08 Jul 2022 14:03:38 -0400 X-MC-Unique: ZFPM_fh_Oj-E7FIoTrgTNQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9E59B29AB3F8; Fri, 8 Jul 2022 18:03:38 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9863D2026D64; Fri, 8 Jul 2022 18:03:37 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:11 +0200 Message-Id: <20220708180316.2852046-13-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 12/17] tests: verify flows in odp.at are parseable X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Create a small helper script and check that flows tested in odp.at are parseable. Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- tests/automake.mk | 2 ++ tests/odp.at | 11 +++++++++++ tests/test-dpparse.py | 45 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 58 insertions(+) create mode 100755 tests/test-dpparse.py diff --git a/tests/automake.mk b/tests/automake.mk index ae99f92bc..49b639c3c 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -20,6 +20,7 @@ EXTRA_DIST += \ tests/atlocal.in \ $(srcdir)/package.m4 \ $(srcdir)/tests/test-ofparse.py \ + $(srcdir)/tests/test-dpparse.py \ $(srcdir)/tests/testsuite \ $(srcdir)/tests/testsuite.patch @@ -526,6 +527,7 @@ CHECK_PYFILES = \ tests/mfex_fuzzy.py \ tests/ovsdb-monitor-sort.py \ tests/test-ofparse.py \ + tests/test-dpparse.py \ tests/test-daemon.py \ tests/test-json.py \ tests/test-jsonrpc.py \ diff --git a/tests/odp.at b/tests/odp.at index 4d08c59ca..075763f5d 100644 --- a/tests/odp.at +++ b/tests/odp.at @@ -107,6 +107,7 @@ sed -i'back' 's/\(in_port(1)\),\(eth\)/\1,packet_type(ns=0,id=0),\2/' odp-out.tx AT_CHECK_UNQUOTED([ovstest test-odp parse-keys < odp-in.txt], [0], [`cat odp-out.txt` ]) +AT_CHECK_UNQUOTED([cat odp-out.txt | sed 's/^#.*//' | sed 's/$/ actions:drop/' | test-dpparse.py]) AT_CLEANUP AT_SETUP([OVS datapath wildcarded key parsing and formatting - valid forms]) @@ -194,6 +195,7 @@ sed -n 's/,frag=no),.*/,frag=later)/p' odp-base.txt AT_CAPTURE_FILE([odp.txt]) AT_CHECK_UNQUOTED([ovstest test-odp parse-wc-keys < odp.txt], [0], [`cat odp.txt` ]) +AT_CHECK_UNQUOTED([cat odp.txt | sed 's/^#.*//' | sed 's/$/ actions:drop/' | test-dpparse.py]) AT_CLEANUP AT_SETUP([OVS datapath wildcarded key filtering.]) @@ -241,24 +243,31 @@ in_port(1),eth(src=00:01:02:03:04:05,dst=10:11:12:13:14:15),eth_type(0x86dd),ipv ]) AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='dl_type=0x1235' < odp-base.txt], [0], [`cat odp-eth-type.txt` ]) +AT_CHECK_UNQUOTED([cat odp-eth-type.txt | sed 's/^#.*//' | sed 's/$/ actions:drop/' | test-dpparse.py]) AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='dl_vlan=99' < odp-vlan-base.txt], [0], [`cat odp-vlan.txt` ]) +AT_CHECK_UNQUOTED([cat odp-vlan.txt | sed 's/^#.*//' | sed 's/$/ actions:drop/' | test-dpparse.py]) AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='dl_vlan=99,ip' < odp-vlan-base.txt], [0], [`cat odp-vlan.txt` ]) AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='ip,nw_src=35.8.2.199' < odp-base.txt], [0], [`cat odp-ipv4.txt` ]) AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='ip,nw_dst=172.16.0.199' < odp-base.txt], [0], [`cat odp-ipv4.txt` ]) +AT_CHECK_UNQUOTED([cat odp-ipv4.txt | sed 's/^#.*//' | sed 's/$/ actions:drop/' | test-dpparse.py]) AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='dl_type=0x0800,nw_src=35.8.2.199,nw_dst=172.16.0.199' < odp-base.txt], [0], [`cat odp-ipv4.txt` ]) AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='icmp,nw_src=35.8.2.199' < odp-base.txt], [0], [`cat odp-icmp.txt` ]) +AT_CHECK_UNQUOTED([cat odp-icmp.txt | sed 's/^#.*//' | sed 's/$/ actions:drop/' | test-dpparse.py]) AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='arp,arp_spa=1.2.3.5' < odp-base.txt], [0], [`cat odp-arp.txt` ]) +AT_CHECK_UNQUOTED([cat odp-arp.txt | sed 's/^#.*//' | sed 's/$/ actions:drop/' | test-dpparse.py]) AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='tcp,tp_src=90' < odp-base.txt], [0], [`cat odp-tcp.txt` ]) +AT_CHECK_UNQUOTED([cat odp-tcp.txt | sed 's/^#.*//' | sed 's/$/ actions:drop/' | test-dpparse.py]) AT_CHECK_UNQUOTED([ovstest test-odp parse-filter filter='tcp6,tp_src=90' < odp-base.txt], [0], [`cat odp-tcp6.txt` ]) +AT_CHECK_UNQUOTED([cat odp-tcp6.txt | sed 's/^#.*//' | sed 's/$/ actions:drop/' | test-dpparse.py]) AT_CLEANUP AT_SETUP([OVS datapath actions parsing and formatting - valid forms]) @@ -389,6 +398,7 @@ add_mpls(label=200,tc=7,ttl=64,bos=1,eth_type=0x8847) AT_CHECK_UNQUOTED([ovstest test-odp parse-actions < actions.txt], [0], [`cat actions.txt` ]) +AT_CHECK_UNQUOTED([cat actions.txt | sed 's/^/actions:/' | test-dpparse.py]) AT_CLEANUP AT_SETUP([OVS datapath actions parsing and formatting - invalid forms]) @@ -434,6 +444,7 @@ odp_actions_from_string: error `cat actions.txt | head -3 | tail -1` odp_actions_from_string: error ]) +AT_CHECK_UNQUOTED([cat actions.txt | sed 's/^/actions:/' | test-dpparse.py]) AT_CLEANUP AT_SETUP([OVS datapath actions parsing and formatting - actions too long]) diff --git a/tests/test-dpparse.py b/tests/test-dpparse.py new file mode 100755 index 000000000..d700e1680 --- /dev/null +++ b/tests/test-dpparse.py @@ -0,0 +1,45 @@ +#!/usr/bin/env python3 +# Copyright (c) 2022 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""test-dpparse reads flows from stdin and tries to parse them using +the python flow parsing library. +""" + +import fileinput +import sys + +try: + from ovs.flow.odp import ODPFlow +except ImportError: + sys.exit(0) + + +def main(): + for flow in fileinput.input(): + try: + result_flow = ODPFlow(flow) + if flow != str(result_flow): + print("in: {}".format(flow)) + print("out: {}".format(str(result_flow))) + raise ValueError("Flow conversion back to string failed") + except Exception as e: + print(e) + return 1 + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) From patchwork Fri Jul 8 18:03:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654339 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=DWwXtYCT; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh4F3LCdz9ryY for ; Sat, 9 Jul 2022 04:04:13 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 421B342725; Fri, 8 Jul 2022 18:04:11 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 421B342725 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=DWwXtYCT X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uBRlWJ9gUZ6R; Fri, 8 Jul 2022 18:04:09 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 904AF426C9; Fri, 8 Jul 2022 18:03:57 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 904AF426C9 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9F8EEC0086; Fri, 8 Jul 2022 18:03:50 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1A556C0095 for ; Fri, 8 Jul 2022 18:03:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 7D19741655 for ; Fri, 8 Jul 2022 18:03:44 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 7D19741655 Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=DWwXtYCT X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eiMAWnU8zYiS for ; Fri, 8 Jul 2022 18:03:43 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org C858141665 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id C858141665 for ; Fri, 8 Jul 2022 18:03:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303421; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=v5c4Hq427uksW07EJF0A5mYQ1Y52vaAHThOmG0+X9iU=; b=DWwXtYCT613zM5p2n/4Dw5PBu7fSHDK1mjWdz+ANDUG6QPDHayKCs5pxjPL1s70+rjOWJy mihDEK7V3cybRFxgMJo8pP/golN5hQuruzOe+iVO/+HwKxHxReu/3CcHieGr6gywptX6gd 8X92viYMlRTh5UFerwR+rqwF7370yc0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-531-STUqFu3UOgaGMWIHdQeqXg-1; Fri, 08 Jul 2022 14:03:40 -0400 X-MC-Unique: STUqFu3UOgaGMWIHdQeqXg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 360F484818B; Fri, 8 Jul 2022 18:03:40 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0B15A2024CB9; Fri, 8 Jul 2022 18:03:38 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:12 +0200 Message-Id: <20220708180316.2852046-14-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 13/17] python: introduce unit tests X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Use pytest to run unit tests as part of the standard testsuite. Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- .ci/linux-prepare.sh | 3 + Documentation/intro/install/general.rst | 4 ++ python/automake.mk | 9 ++- python/ovs/tests/test_kv.py | 76 +++++++++++++++++++++++++ python/test_requirements.txt | 3 + tests/atlocal.in | 20 +++++++ tests/automake.mk | 1 + tests/pytest.at | 8 +++ tests/testsuite.at | 1 + 9 files changed, 123 insertions(+), 2 deletions(-) create mode 100644 python/ovs/tests/test_kv.py create mode 100644 python/test_requirements.txt create mode 100644 tests/pytest.at diff --git a/.ci/linux-prepare.sh b/.ci/linux-prepare.sh index a0635cf56..de741c6b4 100755 --- a/.ci/linux-prepare.sh +++ b/.ci/linux-prepare.sh @@ -45,6 +45,9 @@ if [ "$M32" ]; then sudo apt-get install -y $pkgs fi +# Install python test dependencies +pip install -r python/test_requirements.txt + # IPv6 is supported by kernel but disabled in TravisCI images: # https://github.com/travis-ci/travis-ci/issues/8891 # Enable it to avoid skipping of IPv6 related tests. diff --git a/Documentation/intro/install/general.rst b/Documentation/intro/install/general.rst index a297aadac..e2c15a620 100644 --- a/Documentation/intro/install/general.rst +++ b/Documentation/intro/install/general.rst @@ -181,6 +181,10 @@ following to obtain better warnings: come from the "hacking" flake8 plugin. If it's not installed, the warnings just won't occur until it's run on a system with "hacking" installed. +- the python packages listed in "python/test_requirements.txt" (compatible + with pip). If they are installed, the pytest-based Python unit tests will + be run. + You may find the ovs-dev script found in ``utilities/ovs-dev.py`` useful. .. _general-install-reqs: diff --git a/python/automake.mk b/python/automake.mk index aa3adb703..3a9246fe2 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -52,6 +52,9 @@ ovs_pyfiles = \ python/ovs/vlog.py \ python/ovs/winutils.py +ovs_pytests = \ + python/ovs/tests/test_kv.py + # These python files are used at build time but not runtime, # so they are not installed. EXTRA_DIST += \ @@ -64,12 +67,14 @@ EXTRA_DIST += \ EXTRA_DIST += \ python/ovs/compat/sortedcontainers/LICENSE \ python/README.rst \ - python/setup.py + python/setup.py \ + python/test_requirements.txt # C extension support. EXTRA_DIST += python/ovs/_json.c -PYFILES = $(ovs_pyfiles) python/ovs/dirs.py $(ovstest_pyfiles) +PYFILES = $(ovs_pyfiles) python/ovs/dirs.py $(ovstest_pyfiles) $(ovs_pytests) + EXTRA_DIST += $(PYFILES) PYCOV_CLEAN_FILES += $(PYFILES:.py=.py,cover) diff --git a/python/ovs/tests/test_kv.py b/python/ovs/tests/test_kv.py new file mode 100644 index 000000000..c5b66de88 --- /dev/null +++ b/python/ovs/tests/test_kv.py @@ -0,0 +1,76 @@ +import pytest + +from ovs.flow.kv import KVParser, KeyValue + + +@pytest.mark.parametrize( + "input_data,expected", + [ + ( + ( + "cookie=0x0, duration=147566.365s, table=0, n_packets=39, n_bytes=2574, idle_age=65534, hard_age=65534", # noqa: E501 + None, + ), + [ + KeyValue("cookie", 0), + KeyValue("duration", "147566.365s"), + KeyValue("table", 0), + KeyValue("n_packets", 39), + KeyValue("n_bytes", 2574), + KeyValue("idle_age", 65534), + KeyValue("hard_age", 65534), + ], + ), + ( + ( + "load:0x4->NXM_NX_REG13[],load:0x9->NXM_NX_REG11[],load:0x8->NXM_NX_REG12[],load:0x1->OXM_OF_METADATA[],load:0x1->NXM_NX_REG14[],mod_dl_src:0a:58:a9:fe:00:02,resubmit(,8)", # noqa: E501 + None, + ), + [ + KeyValue("load", "0x4->NXM_NX_REG13[]"), + KeyValue("load", "0x9->NXM_NX_REG11[]"), + KeyValue("load", "0x8->NXM_NX_REG12[]"), + KeyValue("load", "0x1->OXM_OF_METADATA[]"), + KeyValue("load", "0x1->NXM_NX_REG14[]"), + KeyValue("mod_dl_src", "0a:58:a9:fe:00:02"), + KeyValue("resubmit", ",8"), + ], + ), + ( + ("l1(l2(l3(l4())))", None), + [KeyValue("l1", "l2(l3(l4()))")] + ), + ( + ("l1(l2(l3(l4()))),foo:bar", None), + [KeyValue("l1", "l2(l3(l4()))"), KeyValue("foo", "bar")], + ), + ( + ("enqueue:1:2,output=2", None), + [KeyValue("enqueue", "1:2"), KeyValue("output", 2)], + ), + ( + ("value_to_reg(100)->someReg[10],foo:bar", None), + [ + KeyValue("value_to_reg", "(100)->someReg[10]"), + KeyValue("foo", "bar"), + ], + ), + ], +) +def test_kv_parser(input_data, expected): + input_string = input_data[0] + decoders = input_data[1] + tparser = KVParser(input_string, decoders) + tparser.parse() + result = tparser.kv() + assert len(expected) == len(result) + for i in range(0, len(result)): + assert result[i].key == expected[i].key + assert result[i].value == expected[i].value + kpos = result[i].meta.kpos + kstr = result[i].meta.kstring + vpos = result[i].meta.vpos + vstr = result[i].meta.vstring + assert input_string[kpos : kpos + len(kstr)] == kstr + if vpos != -1: + assert input_string[vpos : vpos + len(vstr)] == vstr diff --git a/python/test_requirements.txt b/python/test_requirements.txt new file mode 100644 index 000000000..6aaee13e3 --- /dev/null +++ b/python/test_requirements.txt @@ -0,0 +1,3 @@ +pytest +netaddr +pyparsing diff --git a/tests/atlocal.in b/tests/atlocal.in index 142ea2090..e02248f6f 100644 --- a/tests/atlocal.in +++ b/tests/atlocal.in @@ -238,3 +238,23 @@ export ASAN_OPTIONS # for the build. UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=true:log_path=ubsan:$UBSAN_OPTIONS export UBSAN_OPTIONS + +# Check whether Python test requirements are available. +REQUIREMENT_PATH=$abs_top_srcdir/python/test_requirements.txt $PYTHON3 -c ' +import os +import pathlib +import pkg_resources +import sys + +with pathlib.Path(os.path.join(os.getenv("REQUIREMENT_PATH"))).open() as reqs: + for req in pkg_resources.parse_requirements(reqs): + try: + pkg_resources.require(str(req)) + except pkg_resources.DistributionNotFound: + sys.exit(2) +' +case $? in + 0) HAVE_PYTEST=yes ;; + 2) HAVE_PYTEST=no ;; + *) echo "$0: unexpected error probing Python unit test requirements" >&2 ;; +esac diff --git a/tests/automake.mk b/tests/automake.mk index 49b639c3c..fee3d2328 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -102,6 +102,7 @@ TESTSUITE_AT = \ tests/ovsdb-rbac.at \ tests/ovs-vsctl.at \ tests/ovs-xapi-sync.at \ + tests/pytest.at \ tests/stp.at \ tests/rstp.at \ tests/interface-reconfigure.at \ diff --git a/tests/pytest.at b/tests/pytest.at new file mode 100644 index 000000000..0e75da4c4 --- /dev/null +++ b/tests/pytest.at @@ -0,0 +1,8 @@ +AT_BANNER([Python unit tests]) + +# Run pytest unit tests. +AT_SETUP([Pytest unit tests - Python3]) +AT_KEYWORDS([python]) +AT_SKIP_IF([test "$HAVE_PYTEST" = "no"]) +AT_CHECK([$PYTHON3 -m pytest $top_srcdir/python/ovs],[0], [ignore], [ignore]) +AT_CLEANUP() diff --git a/tests/testsuite.at b/tests/testsuite.at index 58adfa09c..14a28b517 100644 --- a/tests/testsuite.at +++ b/tests/testsuite.at @@ -78,3 +78,4 @@ m4_include([tests/mcast-snooping.at]) m4_include([tests/packet-type-aware.at]) m4_include([tests/nsh.at]) m4_include([tests/drop-stats.at]) +m4_include([tests/pytest.at]) From patchwork Fri Jul 8 18:03:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654342 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=CzQ5V9Mp; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh4T2pmsz9ryY for ; Sat, 9 Jul 2022 04:04:25 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 5A73A4268A; Fri, 8 Jul 2022 18:04:23 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 5A73A4268A Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=CzQ5V9Mp X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PWHMQIbj6zJL; Fri, 8 Jul 2022 18:04:21 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 213074267E; Fri, 8 Jul 2022 18:04:05 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 213074267E Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9BF90C0084; Fri, 8 Jul 2022 18:04:00 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 448A0C0035 for ; Fri, 8 Jul 2022 18:03:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 8ACB961494 for ; Fri, 8 Jul 2022 18:03:48 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 8ACB961494 Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=CzQ5V9Mp X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1h8n802H8F42 for ; Fri, 8 Jul 2022 18:03:47 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org D048561489 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id D048561489 for ; Fri, 8 Jul 2022 18:03:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303425; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+aAj0gahu0rrTbiaJuMM1OeiWMVN5SHfbldvN90xBm4=; b=CzQ5V9Mpa+azXqnW2KH5EKKtPbtxh7+zYYtTT9syRTCZiSnrQW5ZnZCxDatM3M5i+gd57Q cz7JEEHxr1YaIbrFn7D5e+nKvO291nMPKsEU6Jm47amz2bU9X+PrhLWfXlc51nHeq+YlLI cDcW4XivpX8AQgW+B9gisl+mP8cXIxg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-377-l07Py-cINruhvwUJjbLkLw-1; Fri, 08 Jul 2022 14:03:42 -0400 X-MC-Unique: l07Py-cINruhvwUJjbLkLw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6C96C833A37; Fri, 8 Jul 2022 18:03:41 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id 621D72029F88; Fri, 8 Jul 2022 18:03:40 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:13 +0200 Message-Id: <20220708180316.2852046-15-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 14/17] python: add unit tests for ListParser X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add unit tests for ListParser class. Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 3 +- python/ovs/tests/test_list.py | 66 +++++++++++++++++++++++++++++++++++ 2 files changed, 68 insertions(+), 1 deletion(-) create mode 100644 python/ovs/tests/test_list.py diff --git a/python/automake.mk b/python/automake.mk index 3a9246fe2..00a57f17d 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -53,7 +53,8 @@ ovs_pyfiles = \ python/ovs/winutils.py ovs_pytests = \ - python/ovs/tests/test_kv.py + python/ovs/tests/test_kv.py \ + python/ovs/tests/test_list.py # These python files are used at build time but not runtime, # so they are not installed. diff --git a/python/ovs/tests/test_list.py b/python/ovs/tests/test_list.py new file mode 100644 index 000000000..e08ee1e85 --- /dev/null +++ b/python/ovs/tests/test_list.py @@ -0,0 +1,66 @@ +import pytest + +from ovs.flow.list import ListParser, ListDecoders +from ovs.flow.kv import KeyValue + + +@pytest.mark.parametrize( + "input_data,expected", + [ + ( + ("field1,field2,3,nested:value", None, [","]), + [ + KeyValue("elem_0", "field1"), + KeyValue("elem_1", "field2"), + KeyValue("elem_2", 3), + KeyValue("elem_3", "nested:value"), + ], + ), + ( + ( + "field1,field2,3,nested:value", + ListDecoders( + [ + ("key1", str), + ("key2", str), + ("key3", int), + ("key4", lambda x: x.split(":"), [","]), + ] + ), + [","], + ), + [ + KeyValue("key1", "field1"), + KeyValue("key2", "field2"), + KeyValue("key3", 3), + KeyValue("key4", ["nested", "value"]), + ], + ), + ( + ("field1:field2:3", None, [":"]), + [ + KeyValue("elem_0", "field1"), + KeyValue("elem_1", "field2"), + KeyValue("elem_2", 3), + ], + ), + ], +) +def test_kv_parser(input_data, expected): + input_string = input_data[0] + decoders = input_data[1] + delims = input_data[2] + tparser = ListParser(input_string, decoders, delims) + tparser.parse() + result = tparser.kv() + assert len(expected) == len(result) + for i in range(0, len(result)): + assert result[i].key == expected[i].key + assert result[i].value == expected[i].value + kpos = result[i].meta.kpos + kstr = result[i].meta.kstring + vpos = result[i].meta.vpos + vstr = result[i].meta.vstring + assert input_string[kpos : kpos + len(kstr)] == kstr + if vpos != -1: + assert input_string[vpos : vpos + len(vstr)] == vstr From patchwork Fri Jul 8 18:03:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654346 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=N1ZyO6hk; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh5502tyz9ryY for ; Sat, 9 Jul 2022 04:04:56 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 09B35427CF; Fri, 8 Jul 2022 18:04:54 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 09B35427CF Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=N1ZyO6hk X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ofRTn4St6J-y; Fri, 8 Jul 2022 18:04:51 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 50BAA426D8; Fri, 8 Jul 2022 18:04:35 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 50BAA426D8 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E8647C0035; Fri, 8 Jul 2022 18:04:34 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 19FA5C0077 for ; Fri, 8 Jul 2022 18:04:34 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id D4BAE426DF for ; Fri, 8 Jul 2022 18:04:00 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org D4BAE426DF X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2vyBLRj-nc-L for ; Fri, 8 Jul 2022 18:03:58 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 965DA426A2 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 965DA426A2 for ; Fri, 8 Jul 2022 18:03:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303430; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VQGFx+KDw2ap2z7nPVRYPvmjbj3EB1ETdd68tf5me/g=; b=N1ZyO6hkvW3FFY84k8j/ix7QN9+tdUqdHLF0tkzaKwPS5qKRqNAh6LFGM3pTDlECSREi7P GjR39ZV2I1d4eSo5os7zMHwCZ63bG5UwO/8Ut8vSaeag+lmSxUlVFOx0Wj/DiqjbYRoK1X govyz/4Bgdfl45Dhtz7lics+PR2vC9c= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-182-bZTNTMqaMdeHOYKGstVsFw-1; Fri, 08 Jul 2022 14:03:43 -0400 X-MC-Unique: bZTNTMqaMdeHOYKGstVsFw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C418785A58B; Fri, 8 Jul 2022 18:03:42 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id ACC8E2026D64; Fri, 8 Jul 2022 18:03:41 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:14 +0200 Message-Id: <20220708180316.2852046-16-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 15/17] python: add unit tests for openflow parsing X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add unit tests for OFPFlow class and ip-port range decoder Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 4 +- python/ovs/tests/test_decoders.py | 130 ++++++++ python/ovs/tests/test_ofp.py | 534 ++++++++++++++++++++++++++++++ 3 files changed, 667 insertions(+), 1 deletion(-) create mode 100644 python/ovs/tests/test_decoders.py create mode 100644 python/ovs/tests/test_ofp.py diff --git a/python/automake.mk b/python/automake.mk index 00a57f17d..e44510fce 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -53,8 +53,10 @@ ovs_pyfiles = \ python/ovs/winutils.py ovs_pytests = \ + python/ovs/tests/test_decoders.py \ python/ovs/tests/test_kv.py \ - python/ovs/tests/test_list.py + python/ovs/tests/test_list.py \ + python/ovs/tests/test_ofp.py # These python files are used at build time but not runtime, # so they are not installed. diff --git a/python/ovs/tests/test_decoders.py b/python/ovs/tests/test_decoders.py new file mode 100644 index 000000000..cf621d131 --- /dev/null +++ b/python/ovs/tests/test_decoders.py @@ -0,0 +1,130 @@ +from netaddr import IPAddress +import pytest + +from ovs.flow.decoders import decode_ip_port_range + + +@pytest.mark.parametrize( + "input_string,expected", + [ + ( + "192.168.0.0-192.168.0.200:1000-2000", + { + "addrs": { + "start": IPAddress("192.168.0.0"), + "end": IPAddress("192.168.0.200"), + }, + "ports": { + "start": 1000, + "end": 2000, + }, + }, + ), + ( + "192.168.0.0-192.168.0.200", + { + "addrs": { + "start": IPAddress("192.168.0.0"), + "end": IPAddress("192.168.0.200"), + }, + }, + ), + ( + "192.168.0.0-192.168.0.200:2000", + { + "addrs": { + "start": IPAddress("192.168.0.0"), + "end": IPAddress("192.168.0.200"), + }, + "ports": { + "start": 2000, + "end": 2000, + }, + }, + ), + ( + "192.168.0.1:1000-2000", + { + "addrs": { + "start": IPAddress("192.168.0.1"), + "end": IPAddress("192.168.0.1"), + }, + "ports": { + "start": 1000, + "end": 2000, + }, + }, + ), + ( + "[fe80:0000:0000:0000:0204:61ff:fe9d:f150]-[fe80:0000:0000:0000:0204:61ff:fe9d:f15f]:255", # noqa: E501 + { + "addrs": { + "start": IPAddress( + "fe80:0000:0000:0000:0204:61ff:fe9d:f150" + ), + "end": IPAddress( + "fe80:0000:0000:0000:0204:61ff:fe9d:f15f" + ), + }, + "ports": { + "start": 255, + "end": 255, + }, + }, + ), + ( + "[fe80::204:61ff:254.157.241.86]-[fe80::204:61ff:254.157.241.100]:255-300", # noqa: E501 + { + "addrs": { + "start": IPAddress("fe80::204:61ff:254.157.241.86"), + "end": IPAddress("fe80::204:61ff:254.157.241.100"), + }, + "ports": { + "start": 255, + "end": 300, + }, + }, + ), + ( + "[fe80::f150]-[fe80::f15f]:255-300", + { + "addrs": { + "start": IPAddress("fe80::f150"), + "end": IPAddress("fe80::f15f"), + }, + "ports": { + "start": 255, + "end": 300, + }, + }, + ), + ( + "fe80:0000:0000:0000:0204:61ff:fe9d:f150-fe80:0000:0000:0000:0204:61ff:fe9d:f15f", # noqa: E501 + { + "addrs": { + "start": IPAddress( + "fe80:0000:0000:0000:0204:61ff:fe9d:f150" + ), + "end": IPAddress( + "fe80:0000:0000:0000:0204:61ff:fe9d:f15f" + ), + }, + }, + ), + ( + "fe80:0000:0000:0000:0204:61ff:fe9d:f156", + { + "addrs": { + "start": IPAddress( + "fe80:0000:0000:0000:0204:61ff:fe9d:f156" + ), + "end": IPAddress( + "fe80:0000:0000:0000:0204:61ff:fe9d:f156" + ), + }, + }, + ), + ], +) +def test_decode_ip_port_range(input_string, expected): + assert expected == decode_ip_port_range(input_string) diff --git a/python/ovs/tests/test_ofp.py b/python/ovs/tests/test_ofp.py new file mode 100644 index 000000000..7a93b2fd4 --- /dev/null +++ b/python/ovs/tests/test_ofp.py @@ -0,0 +1,534 @@ +import netaddr +import pytest + +from ovs.flow.ofp import OFPFlow +from ovs.flow.kv import KeyValue +from ovs.flow.decoders import EthMask, IPMask, decode_mask + + +@pytest.mark.parametrize( + "input_string,expected", + [ + ( + "actions=local,3,4,5,output:foo", + [ + KeyValue("output", {"port": "local"}), + KeyValue("output", {"port": 3}), + KeyValue("output", {"port": 4}), + KeyValue("output", {"port": 5}), + KeyValue("output", {"port": "foo"}), + ], + ), + ( + "actions=controller,controller:200", + [ + KeyValue("output", "controller"), + KeyValue("controller", {"max_len": 200}), + ], + ), + ( + "actions=enqueue(foo,42),enqueue:foo:42,enqueue(bar,4242)", + [ + KeyValue("enqueue", {"port": "foo", "queue": 42}), + KeyValue("enqueue", {"port": "foo", "queue": 42}), + KeyValue("enqueue", {"port": "bar", "queue": 4242}), + ], + ), + ( + "actions=bundle(eth_src,0,hrw,ofport,members:4,8)", + [ + KeyValue( + "bundle", + { + "fields": "eth_src", + "basis": 0, + "algorithm": "hrw", + "members": [4, 8], + }, + ), + ], + ), + ( + "actions=bundle_load(eth_src,0,hrw,ofport,reg0,members:4,8)", + [ + KeyValue( + "bundle_load", + { + "fields": "eth_src", + "basis": 0, + "algorithm": "hrw", + "dst": "reg0", + "members": [4, 8], + }, + ), + ], + ), + ( + "actions=group:3", + [KeyValue("group", 3)], + ), + ( + "actions=strip_vlan", + [KeyValue("strip_vlan", True)], + ), + ( + "actions=pop_vlan", + [KeyValue("pop_vlan", True)], + ), + ( + "actions=push_vlan:0x8100", + [KeyValue("push_vlan", 0x8100)], + ), + ( + "actions=push_mpls:0x8848", + [KeyValue("push_mpls", 0x8848)], + ), + ( + "actions=pop_mpls:0x8848", + [KeyValue("pop_mpls", 0x8848)], + ), + ( + "actions=pop_mpls:0x8848", + [KeyValue("pop_mpls", 0x8848)], + ), + ( + "actions=encap(nsh(md_type=2,tlv(0x1000,10,0x12345678)))", + [ + KeyValue( + "encap", + { + "header": "nsh", + "props": { + "md_type": 2, + "tlv": { + "class": 0x1000, + "type": 10, + "value": 0x12345678, + }, + }, + }, + ) + ], + ), + ( + "actions=encap(ethernet)", + [ + KeyValue( + "encap", + {"header": "ethernet"}, + ) + ], + ), + ( + "actions=encap(mpls)", + [ + KeyValue( + "encap", + {"header": "mpls"}, + ) + ], + ), + ( + "actions=load:0x001122334455->eth_src", + [ + KeyValue( + "load", + {"value": 0x001122334455, "dst": {"field": "eth_src"}}, + ) + ], + ), + ( + "actions=load:1->eth_src[1]", + [ + KeyValue( + "load", + { + "value": 1, + "dst": {"field": "eth_src", "start": 1, "end": 1}, + }, + ) + ], + ), + ( + "actions=learn(load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[])", + [ + KeyValue( + "learn", + [ + { + "load": { + "src": {"field": "NXM_NX_TUN_ID"}, + "dst": {"field": "NXM_NX_TUN_ID"}, + } + } + ], + ), + ], + ), + ( + "actions=set_field:00:11:22:33:44:55->eth_src", + [ + KeyValue( + "set_field", + { + "value": {"eth_src": EthMask("00:11:22:33:44:55")}, + "dst": {"field": "eth_src"}, + }, + ) + ], + ), + ( + "actions=set_field:01:00:00:00:00:00/01:00:00:00:00:00->eth_src", + [ + KeyValue( + "set_field", + { + "value": { + "eth_src": EthMask( + "01:00:00:00:00:00/01:00:00:00:00:00" + ) + }, + "dst": {"field": "eth_src"}, + }, + ) + ], + ), + ( + "actions=set_field:0x10ff->vlan_vid", + [ + KeyValue( + "set_field", + { + "value": {"vlan_vid": decode_mask(13)("0x10ff")}, + "dst": {"field": "vlan_vid"}, + }, + ) + ], + ), + ( + "actions=move:reg0[0..5]->reg1[16..31]", + [ + KeyValue( + "move", + { + "src": {"field": "reg0", "start": 0, "end": 5}, + "dst": {"field": "reg1", "start": 16, "end": 31}, + }, + ) + ], + ), + ( + "actions=mod_dl_dst:00:11:22:33:44:55", + [KeyValue("mod_dl_dst", EthMask("00:11:22:33:44:55"))], + ), + ( + "actions=mod_nw_dst:192.168.1.1", + [KeyValue("mod_nw_dst", IPMask("192.168.1.1"))], + ), + ( + "actions=mod_nw_dst:fe80::ec17:7bff:fe61:7aac", + [KeyValue("mod_nw_dst", IPMask("fe80::ec17:7bff:fe61:7aac"))], + ), + ( + "actions=dec_ttl,dec_ttl(1,2,3)", + [KeyValue("dec_ttl", True), KeyValue("dec_ttl", [1, 2, 3])], + ), + ( + "actions=set_mpls_label:0x100,set_mpls_tc:2,set_mpls_ttl:10", + [ + KeyValue("set_mpls_label", 0x100), + KeyValue("set_mpls_tc", 2), + KeyValue("set_mpls_ttl", 10), + ], + ), + ( + "actions=check_pkt_larger(100)->reg0[10]", + [ + KeyValue( + "check_pkt_larger", + { + "pkt_len": 100, + "dst": {"field": "reg0", "start": 10, "end": 10}, + }, + ), + ], + ), + ( + "actions=pop_queue,set_tunnel:0x10,set_tunnel64:0x65000,set_queue=3", # noqa: E501 + [ + KeyValue("pop_queue", True), + KeyValue("set_tunnel", 0x10), + KeyValue("set_tunnel64", 0x65000), + KeyValue("set_queue", 3), + ], + ), + ( + "actions=ct(zone=10,table=2,nat(snat=192.168.0.0-192.168.0.200:1000-2000,random))", # noqa: E501 + [ + KeyValue( + "ct", + { + "zone": 10, + "table": 2, + "nat": { + "type": "snat", + "addrs": { + "start": netaddr.IPAddress("192.168.0.0"), + "end": netaddr.IPAddress("192.168.0.200"), + }, + "ports": { + "start": 1000, + "end": 2000, + }, + "random": True, + }, + }, + ) + ], + ), + ( + "actions=ct(commit,zone=NXM_NX_REG13[0..15],table=2,exec(load:0->NXM_NX_CT_LABEL[0]))", # noqa: E501 + [ + KeyValue( + "ct", + { + "commit": True, + "zone": { + "field": "NXM_NX_REG13", + "start": 0, + "end": 15, + }, + "table": 2, + "exec": [ + { + "load": { + "value": 0, + "dst": { + "field": "NXM_NX_CT_LABEL", + "start": 0, + "end": 0, + }, + }, + }, + ], + }, + ) + ], + ), + ( + "actions=load:0x1->NXM_NX_REG10[7],learn(table=69,delete_learned,cookie=0xda6f52b0,OXM_OF_METADATA[],eth_type=0x800,NXM_OF_IP_SRC[],ip_dst=172.30.204.105,nw_proto=6,NXM_OF_TCP_SRC[]=NXM_OF_TCP_DST[],load:0x1->NXM_NX_REG10[7])", # noqa: E501 + [ + KeyValue( + "load", + { + "value": 1, + "dst": {"field": "NXM_NX_REG10", "start": 7, "end": 7}, + }, + ), + KeyValue( + "learn", + [ + {"table": 69}, + {"delete_learned": True}, + {"cookie": 3664728752}, + {"OXM_OF_METADATA[]": True}, + {"eth_type": 2048}, + {"NXM_OF_IP_SRC[]": True}, + {"ip_dst": IPMask("172.30.204.105/32")}, + {"nw_proto": 6}, + {"NXM_OF_TCP_SRC[]": "NXM_OF_TCP_DST[]"}, + { + "load": { + "value": 1, + "dst": { + "field": "NXM_NX_REG10", + "start": 7, + "end": 7, + }, + } + }, + ], + ), + ], + ), + ( + "actions=resubmit(,8),resubmit:3,resubmit(1,2,ct)", + [ + KeyValue("resubmit", {"port": "", "table": 8}), + KeyValue("resubmit", {"port": 3}), + KeyValue("resubmit", {"port": 1, "table": 2, "ct": True}), + ], + ), + ( + "actions=clone(ct_clear,load:0->NXM_NX_REG11[],load:0->NXM_NX_REG12[],load:0->NXM_NX_REG13[],load:0x1d->NXM_NX_REG13[],load:0x1f->NXM_NX_REG11[],load:0x1c->NXM_NX_REG12[],load:0x11->OXM_OF_METADATA[],load:0x2->NXM_NX_REG14[],load:0->NXM_NX_REG10[],load:0->NXM_NX_REG15[],load:0->NXM_NX_REG0[],load:0->NXM_NX_REG1[],load:0->NXM_NX_REG2[],load:0->NXM_NX_REG3[],load:0->NXM_NX_REG4[],load:0->NXM_NX_REG5[],load:0->NXM_NX_REG6[],load:0->NXM_NX_REG7[],load:0->NXM_NX_REG8[],load:0->NXM_NX_REG9[],resubmit(,8))", # noqa: E501 + [ + KeyValue( + "clone", + [ + {"ct_clear": True}, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG11"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG12"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG13"}, + } + }, + { + "load": { + "value": 29, + "dst": {"field": "NXM_NX_REG13"}, + } + }, + { + "load": { + "value": 31, + "dst": {"field": "NXM_NX_REG11"}, + } + }, + { + "load": { + "value": 28, + "dst": {"field": "NXM_NX_REG12"}, + } + }, + { + "load": { + "value": 17, + "dst": {"field": "OXM_OF_METADATA"}, + } + }, + { + "load": { + "value": 2, + "dst": {"field": "NXM_NX_REG14"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG10"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG15"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG0"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG1"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG2"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG3"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG4"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG5"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG6"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG7"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG8"}, + } + }, + { + "load": { + "value": 0, + "dst": {"field": "NXM_NX_REG9"}, + } + }, + {"resubmit": {"port": "", "table": 8}}, + ], + ) + ], + ), + ( + "actions=conjunction(1234, 1/2),note:00.00.11.22.33.ff,sample(probability=123,collector_set_id=0x123,obs_domain_id=0x123,obs_point_id=0x123,sampling_port=inport0,ingress)", # noqa: E501 + [ + KeyValue("conjunction", {"id": 1234, "k": 1, "n": 2}), + KeyValue("note", "00.00.11.22.33.ff"), + KeyValue( + "sample", + { + "probability": 123, + "collector_set_id": 0x123, + "obs_domain_id": 0x123, + "obs_point_id": 0x123, + "sampling_port": "inport0", + "ingress": True, + }, + ), + ], + ), + ], +) +def test_act(input_string, expected): + ofp = OFPFlow(input_string) + actions = ofp.actions_kv + for i in range(len(expected)): + assert expected[i].key == actions[i].key + assert expected[i].value == actions[i].value + + # Assert positions relative to action string are OK. + apos = ofp.section("actions").pos + astring = ofp.section("actions").string + + kpos = actions[i].meta.kpos + kstr = actions[i].meta.kstring + vpos = actions[i].meta.vpos + vstr = actions[i].meta.vstring + assert astring[kpos : kpos + len(kstr)] == kstr + if vpos != -1: + assert astring[vpos : vpos + len(vstr)] == vstr + + # Assert astring meta is correct. + assert input_string[apos : apos + len(astring)] == astring From patchwork Fri Jul 8 18:03:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654344 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=EF81BavZ; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh4n5zy9z9ryY for ; Sat, 9 Jul 2022 04:04:41 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id A91A0424BA; Fri, 8 Jul 2022 18:04:39 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org A91A0424BA Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=EF81BavZ X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vmEu-0mI0ZZh; Fri, 8 Jul 2022 18:04:35 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id C2AAA426A2; Fri, 8 Jul 2022 18:04:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org C2AAA426A2 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9DD9AC0080; Fri, 8 Jul 2022 18:04:16 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1BE20C0080 for ; Fri, 8 Jul 2022 18:04:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id C304C848DC for ; Fri, 8 Jul 2022 18:03:55 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org C304C848DC Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=EF81BavZ X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JBqDcs4h557t for ; Fri, 8 Jul 2022 18:03:52 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org C5DB6848AF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id C5DB6848AF for ; Fri, 8 Jul 2022 18:03:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303427; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gmsVHH07nOG4rXj4vrjoiMUXMovphmjYLoNUonTBKkk=; b=EF81BavZSNzIwANgxXoBG4KIutVupgCJ1Qr6z5DXgkEsRXs9xNSbw2lx2bcrD55dp98I3W txNGBM8nFvwhud5Q3/qbQE0sKkKL6ZqoDWP7E7NpmspxULKIFO/lX0kdrgQHBrUzCsRlEF fXHhy6Qasj4KQGg6MSuK5DcBAqwDDow= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-335-nVJxwUt6NamV5AkSye231w-1; Fri, 08 Jul 2022 14:03:44 -0400 X-MC-Unique: nVJxwUt6NamV5AkSye231w-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 19DEA1039624; Fri, 8 Jul 2022 18:03:44 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2204A2026D64; Fri, 8 Jul 2022 18:03:42 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:15 +0200 Message-Id: <20220708180316.2852046-17-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 16/17] python: add unit tests to datapath parsing X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add unit tests to datapath flow parsing. Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 1 + python/ovs/tests/test_odp.py | 527 +++++++++++++++++++++++++++++++++++ 2 files changed, 528 insertions(+) create mode 100644 python/ovs/tests/test_odp.py diff --git a/python/automake.mk b/python/automake.mk index e44510fce..4748cff6f 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -56,6 +56,7 @@ ovs_pytests = \ python/ovs/tests/test_decoders.py \ python/ovs/tests/test_kv.py \ python/ovs/tests/test_list.py \ + python/ovs/tests/test_odp.py \ python/ovs/tests/test_ofp.py # These python files are used at build time but not runtime, diff --git a/python/ovs/tests/test_odp.py b/python/ovs/tests/test_odp.py new file mode 100644 index 000000000..715be3869 --- /dev/null +++ b/python/ovs/tests/test_odp.py @@ -0,0 +1,527 @@ +import netaddr +import pytest + +from ovs.flow.odp import ODPFlow +from ovs.flow.kv import KeyValue +from ovs.flow.decoders import ( + EthMask, + IPMask, + Mask32, + Mask16, + Mask8, + Mask128, +) + + +@pytest.mark.parametrize( + "input_string,expected", + [ + ( + "skb_priority(0x123),skb_mark(0x123),recirc_id(0x123),dp_hash(0x123),ct_zone(0x123), actions:", # noqa: E501 + [ + KeyValue("skb_priority", Mask32("0x123")), + KeyValue("skb_mark", Mask32("0x123")), + KeyValue("recirc_id", 0x123), + KeyValue("dp_hash", Mask32("0x123")), + KeyValue("ct_zone", Mask16("0x123")), + ], + ), + ( + "tunnel(tun_id=0x7f10354,src=10.10.10.10,dst=20.20.20.20,ttl=64,flags(csum|key)) actions:", # noqa: E501 + [ + KeyValue( + "tunnel", + { + "tun_id": 0x7F10354, + "src": IPMask("10.10.10.10"), + "dst": IPMask("20.20.20.20"), + "ttl": 64, + "flags": "csum|key", + }, + ) + ], + ), + ( + "tunnel(geneve({class=0,type=0,len=4,0xa/0xff}),vxlan(flags=0x800000,vni=0x1c7),erspan(ver=2,dir=1,hwid=0x1)), actions:", # noqa: E501 + [ + KeyValue( + "tunnel", + { + "geneve": [ + { + "class": Mask16("0"), + "type": Mask8("0"), + "len": Mask8("4"), + "data": Mask128("0xa/0xff"), + } + ], + "vxlan": {"flags": 0x800000, "vni": 0x1C7}, + "erspan": {"ver": 2, "dir": 1, "hwid": 0x1}, + }, + ) + ], + ), + ( + "in_port(2),eth(src=11:22:33:44:55:66,dst=66:55:44:33:22:11) actions:", # noqa: E501 + [ + KeyValue("in_port", 2), + KeyValue( + "eth", + { + "src": EthMask("11:22:33:44:55:66"), + "dst": EthMask("66:55:44:33:22:11"), + }, + ), + ], + ), + ( + "eth_type(0x800/0x006),ipv4(src=192.168.1.1/24,dst=192.168.0.0/16,proto=0x1,tos=0x2/0xf0) actions:", # noqa: E501 + [ + KeyValue("eth_type", Mask16("0x800/0x006")), + KeyValue( + "ipv4", + { + "src": IPMask("192.168.1.1/24"), + "dst": IPMask("192.168.0.0/16"), + "proto": Mask8("0x1/0xFF"), + "tos": Mask8("0x2/0xF0"), + }, + ), + ], + ), + ( + "encap(eth_type(0x800/0x006),ipv4(src=192.168.1.1/24,dst=192.168.0.0/16,proto=0x1,tos=0x2/0xf0)) actions:", # noqa: E501 + [ + KeyValue( + "encap", + { + "eth_type": Mask16("0x800/0x006"), + "ipv4": { + "src": IPMask("192.168.1.1/24"), + "dst": IPMask("192.168.0.0/16"), + "proto": Mask8("0x1/0xff"), + "tos": Mask8("0x2/0xf0"), + }, + }, + ), + ], + ), + ], +) +def test_odp_fields(input_string, expected): + odp = ODPFlow(input_string) + match = odp.match_kv + for i in range(len(expected)): + assert expected[i].key == match[i].key + assert expected[i].value == match[i].value + + # Assert positions relative to action string are OK. + mpos = odp.section("match").pos + mstring = odp.section("match").string + + kpos = match[i].meta.kpos + kstr = match[i].meta.kstring + vpos = match[i].meta.vpos + vstr = match[i].meta.vstring + assert mstring[kpos : kpos + len(kstr)] == kstr + if vpos != -1: + assert mstring[vpos : vpos + len(vstr)] == vstr + + # Assert mstring meta is correct. + assert input_string[mpos : mpos + len(mstring)] == mstring + + +@pytest.mark.parametrize( + "input_string,expected", + [ + ( + "actions:ct" + ",ct(commit)" + ",ct(commit,zone=5)" + ",ct(commit,mark=0xa0a0a0a0/0xfefefefe)" + ",ct(commit,label=0x1234567890abcdef1234567890abcdef/0xf1f2f3f4f5f6f7f8f9f0fafbfcfdfeff)" # noqa: E501 + ",ct(commit,helper=ftp)" + ",ct(commit,helper=tftp)" + ",ct(commit,timeout=ovs_tp_1_tcp4)" + ",ct(nat)", + [ + KeyValue("ct", True), + KeyValue("ct", {"commit": True}), + KeyValue("ct", {"commit": True, "zone": 5}), + KeyValue( + "ct", + {"commit": True, "mark": Mask32("0xA0A0A0A0/0xFEFEFEFE")}, + ), + KeyValue( + "ct", + { + "commit": True, + "label": Mask128( + "0x1234567890ABCDEF1234567890ABCDEF/0xF1F2F3F4F5F6F7F8F9F0FAFBFCFDFEFF" # noqa: E501 + ), + }, + ), + KeyValue("ct", {"commit": True, "helper": "ftp"}), + KeyValue("ct", {"commit": True, "helper": "tftp"}), + KeyValue("ct", {"commit": True, "timeout": "ovs_tp_1_tcp4"}), + KeyValue("ct", {"nat": True}), + ], + ), + ( + "actions:ct(nat)" + ",ct(commit,nat(src))" + ",ct(commit,nat(dst))" + ",ct(commit,nat(src=10.0.0.240,random))" + ",ct(commit,nat(src=10.0.0.240:32768-65535,random))" + ",ct(commit,nat(dst=10.0.0.128-10.0.0.254,hash))" + ",ct(commit,nat(src=10.0.0.240-10.0.0.254:32768-65535,persistent))" + ",ct(commit,nat(src=fe80::20c:29ff:fe88:a18b,random))" + ",ct(commit,nat(src=fe80::20c:29ff:fe88:1-fe80::20c:29ff:fe88:a18b,random))" # noqa: E501 + ",ct(commit,nat(src=[[fe80::20c:29ff:fe88:1]]-[[fe80::20c:29ff:fe88:a18b]]:255-4096,random))" # noqa: E501 + ",ct(commit,helper=ftp,nat(src=10.1.1.240-10.1.1.255))" + ",ct(force_commit)", + [ + KeyValue("ct", {"nat": True}), + KeyValue("ct", {"commit": True, "nat": {"type": "src"}}), + KeyValue("ct", {"commit": True, "nat": {"type": "dst"}}), + KeyValue( + "ct", + { + "commit": True, + "nat": { + "type": "src", + "addrs": { + "start": netaddr.IPAddress("10.0.0.240"), + "end": netaddr.IPAddress("10.0.0.240"), + }, + "random": True, + }, + }, + ), + KeyValue( + "ct", + { + "commit": True, + "nat": { + "type": "src", + "addrs": { + "start": netaddr.IPAddress("10.0.0.240"), + "end": netaddr.IPAddress("10.0.0.240"), + }, + "ports": { + "start": 32768, + "end": 65535, + }, + "random": True, + }, + }, + ), + KeyValue( + "ct", + { + "commit": True, + "nat": { + "type": "dst", + "addrs": { + "start": netaddr.IPAddress("10.0.0.128"), + "end": netaddr.IPAddress("10.0.0.254"), + }, + "hash": True, + }, + }, + ), + KeyValue( + "ct", + { + "commit": True, + "nat": { + "type": "src", + "addrs": { + "start": netaddr.IPAddress("10.0.0.240"), + "end": netaddr.IPAddress("10.0.0.254"), + }, + "ports": { + "start": 32768, + "end": 65535, + }, + "persistent": True, + }, + }, + ), + KeyValue( + "ct", + { + "commit": True, + "nat": { + "type": "src", + "addrs": { + "start": netaddr.IPAddress( + "fe80::20c:29ff:fe88:a18b" + ), + "end": netaddr.IPAddress( + "fe80::20c:29ff:fe88:a18b" + ), + }, + "random": True, + }, + }, + ), + KeyValue( + "ct", + { + "commit": True, + "nat": { + "type": "src", + "addrs": { + "start": netaddr.IPAddress( + "fe80::20c:29ff:fe88:1" + ), + "end": netaddr.IPAddress( + "fe80::20c:29ff:fe88:a18b" + ), + }, + "random": True, + }, + }, + ), + KeyValue( + "ct", + { + "commit": True, + "nat": { + "type": "src", + "addrs": { + "start": netaddr.IPAddress( + "fe80::20c:29ff:fe88:1" + ), + "end": netaddr.IPAddress( + "fe80::20c:29ff:fe88:a18b" + ), + }, + "ports": { + "start": 255, + "end": 4096, + }, + "random": True, + }, + }, + ), + KeyValue( + "ct", + { + "commit": True, + "nat": { + "type": "src", + "addrs": { + "start": netaddr.IPAddress("10.1.1.240"), + "end": netaddr.IPAddress("10.1.1.255"), + }, + }, + "helper": "ftp", + }, + ), + KeyValue("ct", {"force_commit": True}), + ], + ), + ( + "actions:set(tunnel(tun_id=0xabcdef1234567890,src=1.1.1.1,dst=2.2.2.2,ttl=64,flags(df|csum|key)))" # noqa: E501 + ",tnl_pop(4)" + ",tnl_push(tnl_port(6),header(size=50,type=4,eth(dst=f8:bc:12:44:34:b6,src=f8:bc:12:46:58:e0,dl_type=0x0800),ipv4(src=1.1.2.88,dst=1.1.2.92,proto=17,tos=0,ttl=64,frag=0x4000),udp(src=0,dst=4789,csum=0x0),vxlan(flags=0x8000000,vni=0x1c7)),out_port(1))" # noqa: E501 + ",tnl_push(tnl_port(6),header(size=70,type=4,eth(dst=f8:bc:12:44:34:b6,src=f8:bc:12:46:58:e0,dl_type=0x86dd),ipv6(src=2001:cafe::88,dst=2001:cafe::92,label=0,proto=17,tclass=0x0,hlimit=64),udp(src=0,dst=4789,csum=0x0),vxlan(flags=0x8000000,vni=0x1c7)),out_port(1))", # noqa: E501 + [ + KeyValue( + "set", + { + "tunnel": { + "tun_id": 0xABCDEF1234567890, + "src": IPMask("1.1.1.1"), + "dst": IPMask("2.2.2.2"), + "ttl": 64, + "flags": "df|csum|key", + } + }, + ), + KeyValue("tnl_pop", 4), + KeyValue( + "tnl_push", + { + "tnl_port": 6, + "header": { + "size": 50, + "type": 4, + "eth": { + "dst": EthMask("f8:bc:12:44:34:b6"), + "src": EthMask("f8:bc:12:46:58:e0"), + "dl_type": 0x800, + }, + "ipv4": { + "src": IPMask("1.1.2.88"), + "dst": IPMask("1.1.2.92"), + "proto": 17, + "tos": 0, + "ttl": 64, + "frag": 0x4000, + }, + "udp": {"src": 0, "dst": 4789, "csum": 0x0}, + "vxlan": { + "flags": 0x8000000, + "vni": 0x1C7, + }, + }, + "out_port": 1, + }, + ), + KeyValue( + "tnl_push", + { + "tnl_port": 6, + "header": { + "size": 70, + "type": 4, + "eth": { + "dst": EthMask("f8:bc:12:44:34:b6"), + "src": EthMask("f8:bc:12:46:58:e0"), + "dl_type": 0x86DD, + }, + "ipv6": { + "src": IPMask("2001:cafe::88"), + "dst": IPMask("2001:cafe::92"), + "label": 0, + "proto": 17, + "tclass": 0x0, + "hlimit": 64, + }, + "udp": {"src": 0, "dst": 4789, "csum": 0x0}, + "vxlan": { + "flags": 0x8000000, + "vni": 0x1C7, + }, + }, + "out_port": 1, + }, + ), + ], + ), + ( + "actions:tnl_push(header(geneve(oam,vni=0x1c7)))" + ",tnl_push(header(geneve(crit,vni=0x1c7,options({class=0xffff,type=0x80,len=4,0xa}))))" # noqa: E501 + ",tnl_push(header(gre((flags=0xa000,proto=0x6558),csum=0x0,key=0x1e241)))", # noqa: E501 + [ + KeyValue( + "tnl_push", + { + "header": { + "geneve": { + "oam": True, + "vni": 0x1C7, + } + } + }, + ), + KeyValue( + "tnl_push", + { + "header": { + "geneve": { + "crit": True, + "vni": 0x1C7, + "options": [ + { + "class": 0xFFFF, + "type": 0x80, + "len": 4, + "data": 0xA, + } + ], + } + } + }, + ), + KeyValue( + "tnl_push", + { + "header": { + "gre": { + "flags": 0xA000, + "proto": 0x6558, + "key": 0x1E241, + "csum": 0x0, + } + } + }, + ), + ], + ), + ( + "actions:clone(1)" ",clone(clone(push_vlan(vid=12,pcp=0),2),1)", + [ + KeyValue("clone", {"output": {"port": 1}}), + KeyValue( + "clone", + { + "output": {"port": 1}, + "clone": { + "push_vlan": { + "vid": 12, + "pcp": 0, + }, + "output": {"port": 2}, + }, + }, + ), + ], + ), + ( + "actions: check_pkt_len(size=200,gt(4),le(5))" + ",check_pkt_len(size=200,gt(drop),le(5))" + ",check_pkt_len(size=200,gt(ct(nat)),le(drop))", + [ + KeyValue( + "check_pkt_len", + { + "size": 200, + "gt": {"output": {"port": 4}}, + "le": {"output": {"port": 5}}, + }, + ), + KeyValue( + "check_pkt_len", + { + "size": 200, + "gt": {"drop": True}, + "le": {"output": {"port": 5}}, + }, + ), + KeyValue( + "check_pkt_len", + { + "size": 200, + "gt": {"ct": {"nat": True}}, + "le": {"drop": True}, + }, + ), + ], + ), + ], +) +def test_odp_actions(input_string, expected): + odp = ODPFlow(input_string) + actions = odp.actions_kv + for i in range(len(expected)): + assert expected[i].key == actions[i].key + assert expected[i].value == actions[i].value + + # Assert positions relative to action string are OK. + apos = odp.section("actions").pos + astring = odp.section("actions").string + + kpos = actions[i].meta.kpos + kstr = actions[i].meta.kstring + vpos = actions[i].meta.vpos + vstr = actions[i].meta.vstring + assert astring[kpos : kpos + len(kstr)] == kstr + if vpos != -1: + assert astring[vpos : vpos + len(vstr)] == vstr + + # Assert astring meta is correct. + assert input_string[apos : apos + len(astring)] == astring From patchwork Fri Jul 8 18:03:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1654343 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=dogaFudp; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lfh4d73cZz9ryY for ; Sat, 9 Jul 2022 04:04:33 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id A17A6426BA; Fri, 8 Jul 2022 18:04:31 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org A17A6426BA Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=dogaFudp X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 01Xjy3xvpZR8; Fri, 8 Jul 2022 18:04:29 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 8AF6E426CD; Fri, 8 Jul 2022 18:04:12 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 8AF6E426CD Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 559AFC0035; Fri, 8 Jul 2022 18:04:12 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 19B6CC0078 for ; Fri, 8 Jul 2022 18:04:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 67EB761484 for ; Fri, 8 Jul 2022 18:03:55 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 67EB761484 Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=dogaFudp X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XfNCcK9umPdb for ; Fri, 8 Jul 2022 18:03:53 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 68A6A61491 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id 68A6A61491 for ; Fri, 8 Jul 2022 18:03:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657303427; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Co8YVmnvfAxERfxKqhTfJ+W/cKFR9WQTI9MrtaMwpo0=; b=dogaFudpPYn29LZIvqAU32khPJM5u/aQD/OjSvrN29740VL3CCGGzoH/whwQxvBnRugPwA IvyGFinfbZNZSO9rvsMV6kM2LEVcbuVGJRK9V+GLPQPIVWIrsAEtUApM3TqaFAMEpSSGqa wfHN5z5RCN/aZxleKfWmh+p33lQACcA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-45-wfneWi-qNsOPHW6rcjD-Cw-1; Fri, 08 Jul 2022 14:03:46 -0400 X-MC-Unique: wfneWi-qNsOPHW6rcjD-Cw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 712B58041BE; Fri, 8 Jul 2022 18:03:45 +0000 (UTC) Received: from amorenoz.users.ipa.redhat.com (unknown [10.39.192.174]) by smtp.corp.redhat.com (Postfix) with ESMTP id 587192026D64; Fri, 8 Jul 2022 18:03:44 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Fri, 8 Jul 2022 20:03:16 +0200 Message-Id: <20220708180316.2852046-18-amorenoz@redhat.com> In-Reply-To: <20220708180316.2852046-1-amorenoz@redhat.com> References: <20220708180316.2852046-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=amorenoz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 17/17] python: add unit tests for filtering engine X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add unit test for OFFilter class. Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 1 + python/ovs/tests/test_filter.py | 221 ++++++++++++++++++++++++++++++++ 2 files changed, 222 insertions(+) create mode 100644 python/ovs/tests/test_filter.py diff --git a/python/automake.mk b/python/automake.mk index 4748cff6f..bc3a9cd68 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -54,6 +54,7 @@ ovs_pyfiles = \ ovs_pytests = \ python/ovs/tests/test_decoders.py \ + python/ovs/tests/test_filter.py \ python/ovs/tests/test_kv.py \ python/ovs/tests/test_list.py \ python/ovs/tests/test_odp.py \ diff --git a/python/ovs/tests/test_filter.py b/python/ovs/tests/test_filter.py new file mode 100644 index 000000000..47d9d9420 --- /dev/null +++ b/python/ovs/tests/test_filter.py @@ -0,0 +1,221 @@ +import pytest + +from ovs.flow.filter import OFFilter +from ovs.flow.ofp import OFPFlow +from ovs.flow.odp import ODPFlow + + +@pytest.mark.parametrize( + "expr,flow,expected,match", + [ + ( + "nw_src=192.168.1.1 && tcp_dst=80", + OFPFlow( + "nw_src=192.168.1.1,tcp_dst=80 actions=drop" + ), + True, + ["nw_src", "tcp_dst"], + ), + ( + "nw_src=192.168.1.2 || tcp_dst=80", + OFPFlow( + "nw_src=192.168.1.1,tcp_dst=80 actions=drop" + ), + True, + ["nw_src", "tcp_dst"], + ), + ( + "nw_src=192.168.1.1 || tcp_dst=90", + OFPFlow( + "nw_src=192.168.1.1,tcp_dst=80 actions=drop" + ), + True, + ["nw_src", "tcp_dst"], + ), + ( + "nw_src=192.168.1.2 && tcp_dst=90", + OFPFlow( + "nw_src=192.168.1.1,tcp_dst=80 actions=drop" + ), + False, + ["nw_src", "tcp_dst"], + ), + ( + "nw_src=192.168.1.1", + OFPFlow( + "nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" + ), + False, + ["nw_src"], + ), + ( + "nw_src~=192.168.1.1", + OFPFlow( + "nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" + ), + True, + ["nw_src"], + ), + ( + "nw_src~=192.168.1.1/30", + OFPFlow( + "nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" + ), + True, + ["nw_src"], + ), + ( + "nw_src~=192.168.1.0/16", + OFPFlow( + "nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" + ), + False, + ["nw_src"], + ), + ( + "nw_src~=192.168.1.0/16", + OFPFlow( + "nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" + ), + False, + ["nw_src"], + ), + ( + "n_bytes=100", + OFPFlow( + "n_bytes=100 priority=100,nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" # noqa: E501 + ), + True, + ["n_bytes"], + ), + ( + "n_bytes>10", + OFPFlow( + "n_bytes=100 priority=100,nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" # noqa: E501 + ), + True, + ["n_bytes"], + ), + ( + "n_bytes>100", + OFPFlow( + "n_bytes=100 priority=100,nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" # noqa: E501 + ), + False, + ["n_bytes"], + ), + ( + "n_bytes<100", + OFPFlow( + "n_bytes=100 priority=100,nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" # noqa: E501 + ), + False, + ["n_bytes"], + ), + ( + "n_bytes<1000", + OFPFlow( + "n_bytes=100 priority=100,nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" # noqa: E501 + ), + True, + ["n_bytes"], + ), + ( + "n_bytes>0 && drop=true", + OFPFlow( + "n_bytes=100 priority=100,nw_src=192.168.1.0/24,tcp_dst=80 actions=drop" # noqa: E501 + ), + True, + ["n_bytes", "drop"], + ), + ( + "n_bytes>0 && drop=true", + OFPFlow( + "n_bytes=100 priority=100,nw_src=192.168.1.0/24,tcp_dst=80 actions=2" # noqa: E501 + ), + False, + ["n_bytes"], + ), + ( + "n_bytes>10 && !output.port=3", + OFPFlow( + "n_bytes=100 priority=100,nw_src=192.168.1.0/24,tcp_dst=80 actions=2" # noqa: E501 + ), + True, + ["n_bytes", "output"], + ), + ( + "dl_src=00:11:22:33:44:55", + OFPFlow( + "n_bytes=100 priority=100,dl_src=00:11:22:33:44:55,nw_src=192.168.1.0/24,tcp_dst=80 actions=2" # noqa: E501 + ), + True, + ["dl_src"], + ), + ( + "dl_src~=00:11:22:33:44:55", + OFPFlow( + "n_bytes=100 priority=100,dl_src=00:11:22:33:44:55/ff:ff:ff:ff:ff:00,nw_src=192.168.1.0/24,tcp_dst=80 actions=2" # noqa: E501 + ), + True, + ["dl_src"], + ), + ( + "dl_src~=00:11:22:33:44:66", + OFPFlow( + "n_bytes=100 priority=100,dl_src=00:11:22:33:44:55/ff:ff:ff:ff:ff:00,nw_src=192.168.1.0/24,tcp_dst=80 actions=2" # noqa: E501 + ), + True, + ["dl_src"], + ), + ( + "dl_src~=00:11:22:33:44:66 && tp_dst=1000", + OFPFlow( + "n_bytes=100 priority=100,dl_src=00:11:22:33:44:55/ff:ff:ff:ff:ff:00,nw_src=192.168.1.0/24,tp_dst=0x03e8/0xfff8 actions=2" # noqa: E501 + ), + False, + ["dl_src", "tp_dst"], + ), + ( + "dl_src~=00:11:22:33:44:66 && tp_dst~=1000", + OFPFlow( + "n_bytes=100 priority=100,dl_src=00:11:22:33:44:55/ff:ff:ff:ff:ff:00,nw_src=192.168.1.0/24,tp_dst=0x03e8/0xfff8 actions=2" # noqa: E501 + ), + True, + ["dl_src", "tp_dst"], + ), + ( + "encap", + ODPFlow( + "encap(eth_type(0x0800),ipv4(src=10.76.23.240/255.255.255.248,dst=10.76.23.106,proto=17,tos=0/0,ttl=64,frag=no)) actions:drop" # noqa: E501 + ), + True, + ["encap"], + ), + ( + "encap.ipv4.src=10.76.23.240", + ODPFlow( + "encap(eth_type(0x0800),ipv4(src=10.76.23.240/255.255.255.248,dst=10.76.23.106,proto=17,tos=0/0,ttl=64,frag=no)) actions:drop" # noqa: E501 + ), + False, + ["encap"], + ), + ( + "encap.ipv4.src~=10.76.23.240", + ODPFlow( + "encap(eth_type(0x0800),ipv4(src=10.76.23.240/255.255.255.248,dst=10.76.23.106,proto=17,tos=0/0,ttl=64,frag=no)) actions:drop" # noqa: E501 + ), + True, + ["encap"], + ), + ], +) +def test_filter(expr, flow, expected, match): + ffilter = OFFilter(expr) + result = ffilter.evaluate(flow) + if expected: + assert result + else: + assert not result + + assert [kv.key for kv in result.kv] == match