From patchwork Wed Sep 19 22:47:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Terry Wilson X-Patchwork-Id: 971982 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42Fw4V3LZrz9sBs for ; Thu, 20 Sep 2018 08:48:10 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 8896D10DC; Wed, 19 Sep 2018 22:47:50 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id A38FA10C3 for ; Wed, 19 Sep 2018 22:47:49 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 58A447D2 for ; Wed, 19 Sep 2018 22:47:47 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.26]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B7CE888304 for ; Wed, 19 Sep 2018 22:47:46 +0000 (UTC) Received: from new-host.redhat.com (ovpn-112-39.phx2.redhat.com [10.3.112.39]) by smtp.corp.redhat.com (Postfix) with ESMTP id C36CA30912F4; Wed, 19 Sep 2018 22:47:45 +0000 (UTC) From: Terry Wilson To: dev@openvswitch.org Date: Wed, 19 Sep 2018 17:47:28 -0500 Message-Id: <1537397249-30180-2-git-send-email-twilson@redhat.com> In-Reply-To: <1537397249-30180-1-git-send-email-twilson@redhat.com> References: <1537397249-30180-1-git-send-email-twilson@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.26 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 19 Sep 2018 22:47:46 +0000 (UTC) X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH 1/2] Test the Python C JSON extension X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org The C JSON parser was added quite a while ago, but unless you configure with --enable-shared and have the Python 2/3 development libraries installed, and the resulting python-ovs module installed, 'make check' won't actually test it. This patch changes Python-based tests to run from the $builddir/python directory and makes the tests configurable to use both JSON backends. There are some unicode failures in the C JSON extension that I left unfixed in this patch to make it easy to show run the new tests on broken code. The next patch in this set works around the issue. Signed-off-by: Terry Wilson --- python/automake.mk | 11 ++++ python/ovs/json.py | 10 ++-- tests/atlocal.in | 4 +- tests/json.at | 35 +++++++----- tests/ovsdb-idl.at | 149 ++++++++++++++++++++++++++++++++++++---------------- tests/ovsdb.at | 36 ++++++++----- tests/test-json.py | 12 ++++- tests/test-ovsdb.py | 22 +++++++- 8 files changed, 197 insertions(+), 82 deletions(-) diff --git a/python/automake.mk b/python/automake.mk index 5a1e1da..21547f7 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -92,6 +92,17 @@ python-sdist: $(srcdir)/python/ovs/version.py $(ovs_pyfiles) python/ovs/dirs.py pypi-upload: $(srcdir)/python/ovs/version.py $(ovs_pyfiles) python/ovs/dirs.py (cd python/ && $(PYTHON) setup.py sdist upload) + +ALL_LOCAL += python-build +python-build:$(lib_LTLIBRARIES) + (cd $(srcdir)/python/ && $(PYTHON) setup.py build_py -d $(abs_top_builddir)/python && $(PYTHON) setup.py build_ext -b $(abs_top_builddir)/python -I$(abs_top_srcdir)/include:$(abs_top_builddir)/include -L$(abs_top_builddir)/lib/.libs) +if HAVE_PYTHON3 + (cd $(srcdir)/python/ && $(PYTHON3) setup.py build_py -d $(abs_top_builddir)/python && $(PYTHON3) setup.py build_ext -b $(abs_top_builddir)/python -I$(abs_top_srcdir)/include:$(abs_top_builddir)/include -L$(abs_top_builddir)/lib/.libs) + +CLEAN_LOCAL += python-clean +python-clean: + rm -rf $(abs_top_builddir)/python +endif else ovs-install-data-local: @: diff --git a/python/ovs/json.py b/python/ovs/json.py index e84063f..f06204e 100644 --- a/python/ovs/json.py +++ b/python/ovs/json.py @@ -21,10 +21,13 @@ import sys import six +PARSER_C = 'C' +PARSER_PY = 'PYTHON' try: import ovs._json + PARSER = PARSER_C except ImportError: - pass + PARSER = PARSER_PY __pychecker__ = 'no-stringiter' @@ -91,10 +94,9 @@ class Parser(object): MAX_HEIGHT = 1000 def __new__(cls, *args, **kwargs): - try: + if PARSER == PARSER_C: return ovs._json.Parser(*args, **kwargs) - except NameError: - return super(Parser, cls).__new__(cls) + return super(Parser, cls).__new__(cls) def __init__(self, check_trailer=False): self.check_trailer = check_trailer diff --git a/tests/atlocal.in b/tests/atlocal.in index abfc1ac..3b3af84 100644 --- a/tests/atlocal.in +++ b/tests/atlocal.in @@ -22,8 +22,10 @@ if test x"$PYTHON3" = x; then export PYTHONCOERCECLOCALE fi -PYTHONPATH=$abs_top_srcdir/python:$abs_top_builddir/tests:$PYTHONPATH +PYTHONPATH=$abs_top_builddir/python:$abs_top_builddir/tests:$PYTHONPATH export PYTHONPATH +LD_LIBRARY_PATH=$abs_top_builddir/lib/.libs +export LD_LIBRARY_PATH PYTHONIOENCODING=utf_8 export PYTHONIOENCODING diff --git a/tests/json.at b/tests/json.at index bfe0f19..32a55f7 100644 --- a/tests/json.at +++ b/tests/json.at @@ -9,15 +9,16 @@ m4_define([JSON_CHECK_POSITIVE_C], AT_CLEANUP]) # JSON_CHECK_POSITIVE_PY(TITLE, INPUT, OUTPUT, TEST-JSON-ARGS, -# PYTHON-CHCEK, PYTHON-BIN) +# PYTHON-CHCEK, PYTHON-BIN, JSON-PARSER = python) # m4_define([JSON_CHECK_POSITIVE_PY], [AT_SETUP([$1]) AT_KEYWORDS([json positive Python]) AT_SKIP_IF([test $5 = no]) + AT_SKIP_IF([test $7 = C && ! $6 -c "import ovs._json" 2>/dev/null]) AT_CHECK([printf %s "AS_ESCAPE([$2])" > input]) AT_CAPTURE_FILE([input]) - AT_CHECK([$6 $srcdir/test-json.py $4 input], [0], [stdout], []) + AT_CHECK([$6 $srcdir/test-json.py -j m4_default([$7], python) $4 input], [0], [stdout], []) AT_CHECK([cat stdout], [0], [$3 ]) AT_CLEANUP]) @@ -37,16 +38,17 @@ m4_define([JSON_CHECK_POSITIVE_UCS4PY], m4_define([JSON_CHECK_POSITIVE], [JSON_CHECK_POSITIVE_C([$1 - C], [$2], [$3], [$4]) - JSON_CHECK_POSITIVE_PY([$1 - Python2], [$2], [$3], [$4], - [$HAVE_PYTHON2], [$PYTHON2]) - JSON_CHECK_POSITIVE_PY([$1 - Python3], [$2], [$3], [$4], - [$HAVE_PYTHON3], [$PYTHON3])]) + JSON_CHECK_POSITIVE_PY23([$1], [$2], [$3], [$4])]) m4_define([JSON_CHECK_POSITIVE_PY23], - [JSON_CHECK_POSITIVE_PY([$1 - Python2], [$2], [$3], [$4], + [JSON_CHECK_POSITIVE_PY([$1 - Python2(pyjson)], [$2], [$3], [$4], [$HAVE_PYTHON2], [$PYTHON2]) - JSON_CHECK_POSITIVE_PY([$1 - Python3], [$2], [$3], [$4], - [$HAVE_PYTHON3], [$PYTHON3])]) + JSON_CHECK_POSITIVE_PY([$1 - Python2(cjson)], [$2], [$3], [$4], + [$HAVE_PYTHON2], [$PYTHON2], C) + JSON_CHECK_POSITIVE_PY([$1 - Python3(pyjson)], [$2], [$3], [$4], + [$HAVE_PYTHON3], [$PYTHON3]) + JSON_CHECK_POSITIVE_PY([$1 - Python3(cjson)], [$2], [$3], [$4], + [$HAVE_PYTHON3], [$PYTHON3], C)]) m4_define([JSON_CHECK_NEGATIVE_C], [AT_SETUP([$1]) @@ -59,25 +61,30 @@ m4_define([JSON_CHECK_NEGATIVE_C], AT_CLEANUP]) # JSON_CHECK_NEGATIVE_PY(TITLE, INPUT, OUTPUT, TEST-JSON-ARGS, -# PYTHON-CHCEK, PYTHON-BIN) +# PYTHON-CHCEK, PYTHON-BIN, JSON_PARSER = python) # m4_define([JSON_CHECK_NEGATIVE_PY], [AT_SETUP([$1]) AT_KEYWORDS([json negative Python]) AT_SKIP_IF([test $5 = no]) + AT_SKIP_IF([test $7 = C && ! $6 -c "import ovs._json" 2>/dev/null]) AT_CHECK([printf %s "AS_ESCAPE([$2])" > input]) AT_CAPTURE_FILE([input]) - AT_CHECK([$6 $srcdir/test-json.py $4 input], [1], [stdout], []) + AT_CHECK([$6 $srcdir/test-json.py -j m4_default([$7], python) $4 input], [1], [stdout], []) AT_CHECK([[sed 's/^error: [^:]*:/error:/' < stdout]], [0], [$3 ]) AT_CLEANUP]) m4_define([JSON_CHECK_NEGATIVE], [JSON_CHECK_NEGATIVE_C([$1 - C], [$2], [$3], [$4]) - JSON_CHECK_NEGATIVE_PY([$1 - Python2], [$2], [$3], [$4], + JSON_CHECK_NEGATIVE_PY([$1 - Python2(pyjson)], [$2], [$3], [$4], [$HAVE_PYTHON2], [$PYTHON2]) - JSON_CHECK_NEGATIVE_PY([$1 - Python3], [$2], [$3], [$4], - [$HAVE_PYTHON3], [$PYTHON3])]) + JSON_CHECK_NEGATIVE_PY([$1 - Python2(cjson)], [$2], [$3], [$4], + [$HAVE_PYTHON2], [$PYTHON2], C) + JSON_CHECK_NEGATIVE_PY([$1 - Python3(pyjson)], [$2], [$3], [$4], + [$HAVE_PYTHON3], [$PYTHON3]) + JSON_CHECK_NEGATIVE_PY([$1 - Python3(cjson)], [$2], [$3], [$4], + [$HAVE_PYTHON3], [$PYTHON3], C)]) AT_BANNER([JSON -- arrays]) diff --git a/tests/ovsdb-idl.at b/tests/ovsdb-idl.at index 142eee7..663822e 100644 --- a/tests/ovsdb-idl.at +++ b/tests/ovsdb-idl.at @@ -46,11 +46,12 @@ m4_define([OVSDB_CHECK_IDL_C], m4_define([OVSDB_CHECK_IDL_PYN], [AT_SETUP([$1]) AT_SKIP_IF([test $7 = no]) + AT_SKIP_IF([test $9 = C && ! $8 -c "import ovs._json" 2>/dev/null]) AT_KEYWORDS([ovsdb server idl positive Python $5]) AT_CHECK([ovsdb_start_idltest]) m4_if([$2], [], [], [AT_CHECK([ovsdb-client transact unix:socket $2], [0], [ignore], [ignore])]) - AT_CHECK([$8 $srcdir/test-ovsdb.py -t10 idl $srcdir/idltest.ovsschema unix:socket $3], + AT_CHECK([$8 $srcdir/test-ovsdb.py -j m4_default([$9], python) -t10 idl $srcdir/idltest.ovsschema unix:socket $3], [0], [stdout], [ignore]) AT_CHECK([sort stdout | uuidfilt]m4_if([$6],,, [[| $6]]), [0], [$4]) @@ -58,19 +59,24 @@ m4_define([OVSDB_CHECK_IDL_PYN], AT_CLEANUP]) m4_define([OVSDB_CHECK_IDL_PY], - [OVSDB_CHECK_IDL_PYN([$1 - Python2], [$2], [$3], [$4], [$5], [$6], + [OVSDB_CHECK_IDL_PYN([$1 - Python2(pyjson)], [$2], [$3], [$4], [$5], [$6], [$HAVE_PYTHON2], [$PYTHON2]) - OVSDB_CHECK_IDL_PYN([$1 - Python3], [$2], [$3], [$4], [$5], [$6], - [$HAVE_PYTHON3], [$PYTHON3])]) + OVSDB_CHECK_IDL_PYN([$1 - Python2(cjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON2], [$PYTHON2], C) + OVSDB_CHECK_IDL_PYN([$1 - Python3(pyjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3]) + OVSDB_CHECK_IDL_PYN([$1 - Python3(cjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3], C)]) m4_define([OVSDB_CHECK_IDL_REGISTER_COLUMNS_PYN], [AT_SETUP([$1 - register_columns]) AT_SKIP_IF([test $7 = no]) + AT_SKIP_IF([test $9 = C && ! $8 -c "import ovs._json" 2>/dev/null]) AT_KEYWORDS([ovsdb server idl positive Python register_columns $5]) AT_CHECK([ovsdb_start_idltest]) m4_if([$2], [], [], [AT_CHECK([ovsdb-client transact unix:socket $2], [0], [ignore], [ignore])]) - AT_CHECK([$8 $srcdir/test-ovsdb.py -t10 idl $srcdir/idltest.ovsschema unix:socket ?simple:b,ba,i,ia,r,ra,s,sa,u,ua?link1:i,k,ka,l2?link2:i,l1?singleton:name $3], + AT_CHECK([$8 $srcdir/test-ovsdb.py -j m4_default([$9], python) -t10 idl $srcdir/idltest.ovsschema unix:socket ?simple:b,ba,i,ia,r,ra,s,sa,u,ua?link1:i,k,ka,l2?link2:i,l1?singleton:name $3], [0], [stdout], [ignore]) AT_CHECK([sort stdout | uuidfilt]m4_if([$6],,, [[| $6]]), [0], [$4]) @@ -78,22 +84,27 @@ m4_define([OVSDB_CHECK_IDL_REGISTER_COLUMNS_PYN], AT_CLEANUP]) m4_define([OVSDB_CHECK_IDL_REGISTER_COLUMNS_PY], - [OVSDB_CHECK_IDL_REGISTER_COLUMNS_PYN([$1 - Python2], [$2], [$3], [$4], [$5], [$6], + [OVSDB_CHECK_IDL_REGISTER_COLUMNS_PYN([$1 - Python2(pyjson)], [$2], [$3], [$4], [$5], [$6], [$HAVE_PYTHON2], [$PYTHON2]) - OVSDB_CHECK_IDL_REGISTER_COLUMNS_PYN([$1 - Python3], [$2], [$3], [$4], [$5], [$6], - [$HAVE_PYTHON3], [$PYTHON3])]) + OVSDB_CHECK_IDL_REGISTER_COLUMNS_PYN([$1 - Python2(cjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON2], [$PYTHON2], C) + OVSDB_CHECK_IDL_REGISTER_COLUMNS_PYN([$1 - Python3(pyjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3]) + OVSDB_CHECK_IDL_REGISTER_COLUMNS_PYN([$1 - Python3(cjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3], C)]) # same as OVSDB_CHECK_IDL but uses the Python IDL implementation with tcp m4_define([OVSDB_CHECK_IDL_TCP_PYN], [AT_SETUP([$1 - tcp]) AT_SKIP_IF([test $7 = no]) + AT_SKIP_IF([test $9 = C && ! $8 -c "import ovs._json" 2>/dev/null]) AT_KEYWORDS([ovsdb server idl positive Python with tcp socket $5]) AT_CHECK([ovsdb_start_idltest "ptcp:0:127.0.0.1"]) PARSE_LISTENING_PORT([ovsdb-server.log], [TCP_PORT]) m4_if([$2], [], [], [AT_CHECK([ovsdb-client transact tcp:127.0.0.1:$TCP_PORT $2], [0], [ignore], [ignore])]) - AT_CHECK([$8 $srcdir/test-ovsdb.py -t10 idl $srcdir/idltest.ovsschema tcp:127.0.0.1:$TCP_PORT $3], + AT_CHECK([$8 $srcdir/test-ovsdb.py -j m4_default([$9], python) -t10 idl $srcdir/idltest.ovsschema tcp:127.0.0.1:$TCP_PORT $3], [0], [stdout], [ignore]) AT_CHECK([sort stdout | uuidfilt]m4_if([$6],,, [[| $6]]), [0], [$4]) @@ -101,16 +112,21 @@ m4_define([OVSDB_CHECK_IDL_TCP_PYN], AT_CLEANUP]) m4_define([OVSDB_CHECK_IDL_TCP_PY], - [OVSDB_CHECK_IDL_TCP_PYN([$1 - Python2], [$2], [$3], [$4], [$5], [$6], + [OVSDB_CHECK_IDL_TCP_PYN([$1 - Python2(pyjson)], [$2], [$3], [$4], [$5], [$6], [$HAVE_PYTHON2], [$PYTHON2]) - OVSDB_CHECK_IDL_TCP_PYN([$1 - Python3], [$2], [$3], [$4], [$5], [$6], - [$HAVE_PYTHON3], [$PYTHON3])]) + OVSDB_CHECK_IDL_TCP_PYN([$1 - Python2(cjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON2], [$PYTHON2], C) + OVSDB_CHECK_IDL_TCP_PYN([$1 - Python3(pyjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3]) + OVSDB_CHECK_IDL_TCP_PYN([$1 - Python3(cjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3], C)]) # same as OVSDB_CHECK_IDL but uses the Python IDL implementation with tcp # with multiple remotes with only one remote reachable m4_define([OVSDB_CHECK_IDL_TCP_MULTIPLE_REMOTES_PYN], [AT_SETUP([$1 - tcp]) AT_SKIP_IF([test $7 = no]) + AT_SKIP_IF([test $9 = C && ! $8 -c "import ovs._json" 2>/dev/null]) AT_KEYWORDS([ovsdb server idl positive Python with tcp socket $5]) AT_CHECK([ovsdb_start_idltest "ptcp:0:127.0.0.1"]) PARSE_LISTENING_PORT([ovsdb-server.log], [TCP_PORT]) @@ -119,7 +135,7 @@ m4_define([OVSDB_CHECK_IDL_TCP_MULTIPLE_REMOTES_PYN], remote=tcp:127.0.0.1:$WRONG_PORT_1,tcp:127.0.0.1:$TCP_PORT,tcp:127.0.0.1:$WRONG_PORT_2 m4_if([$2], [], [], [AT_CHECK([ovsdb-client transact tcp:127.0.0.1:$TCP_PORT $2], [0], [ignore], [ignore])]) - AT_CHECK([$8 $srcdir/test-ovsdb.py -t10 idl $srcdir/idltest.ovsschema $remote $3], + AT_CHECK([$8 $srcdir/test-ovsdb.py -j m4_default([$9], python) -t10 idl $srcdir/idltest.ovsschema $remote $3], [0], [stdout], [ignore]) AT_CHECK([sort stdout | uuidfilt]m4_if([$6],,, [[| $6]]), [0], [$4]) @@ -127,10 +143,14 @@ m4_define([OVSDB_CHECK_IDL_TCP_MULTIPLE_REMOTES_PYN], AT_CLEANUP]) m4_define([OVSDB_CHECK_IDL_TCP_MULTIPLE_REMOTES_PY], - [OVSDB_CHECK_IDL_TCP_MULTIPLE_REMOTES_PYN([$1 - Python2 (multiple remotes)], [$2], [$3], [$4], [$5], [$6], + [OVSDB_CHECK_IDL_TCP_MULTIPLE_REMOTES_PYN([$1 - Python2 (pyjson multiple remotes)], [$2], [$3], [$4], [$5], [$6], [$HAVE_PYTHON], [$PYTHON]) - OVSDB_CHECK_IDL_TCP_MULTIPLE_REMOTES_PYN([$1 - Python3 (multiple remotes)], [$2], [$3], [$4], [$5], [$6], - [$HAVE_PYTHON3], [$PYTHON3])]) + OVSDB_CHECK_IDL_TCP_MULTIPLE_REMOTES_PYN([$1 - Python2 (cjson multiple remotes)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON], [$PYTHON], C) + OVSDB_CHECK_IDL_TCP_MULTIPLE_REMOTES_PYN([$1 - Python3 (pyjson multiple remotes)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3]) + OVSDB_CHECK_IDL_TCP_MULTIPLE_REMOTES_PYN([$1 - Python3 (cjson multiple remotes)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3], C)]) # same as OVSDB_CHECK_IDL but uses the Python IDL implementation with tcp6 m4_define([OVSDB_CHECK_IDL_TCP6_PYN], @@ -138,6 +158,7 @@ m4_define([OVSDB_CHECK_IDL_TCP6_PYN], AT_SKIP_IF([test $7 = no]) AT_SKIP_IF([test "$IS_WIN32" = "yes"]) AT_SKIP_IF([test $HAVE_IPV6 = no]) + AT_SKIP_IF([test $9 = C && ! $8 -c "import ovs._json" 2>/dev/null]) AT_KEYWORDS([ovsdb server idl positive Python with tcp6 socket $5]) AT_CHECK([ovsdb_start_idltest "ptcp:0:[[::1]]"]) PARSE_LISTENING_PORT([ovsdb-server.log], [TCP_PORT]) @@ -145,7 +166,7 @@ m4_define([OVSDB_CHECK_IDL_TCP6_PYN], m4_if([$2], [], [], [AT_CHECK([ovsdb-client transact "tcp:[[::1]]:$TCP_PORT" $2], [0], [ignore], [ignore])]) - AT_CHECK([$8 $srcdir/test-ovsdb.py -t10 idl $srcdir/idltest.ovsschema tcp:[[::1]]:$TCP_PORT $3], + AT_CHECK([$8 $srcdir/test-ovsdb.py -j m4_default([$9], python) -t10 idl $srcdir/idltest.ovsschema tcp:[[::1]]:$TCP_PORT $3], [0], [stdout], [ignore]) AT_CHECK([sort stdout | uuidfilt]m4_if([$6],,, [[| $6]]), [0], [$4]) @@ -153,16 +174,21 @@ m4_define([OVSDB_CHECK_IDL_TCP6_PYN], AT_CLEANUP]) m4_define([OVSDB_CHECK_IDL_TCP6_PY], - [OVSDB_CHECK_IDL_TCP6_PYN([$1 - Python2], [$2], [$3], [$4], [$5], [$6], + [OVSDB_CHECK_IDL_TCP6_PYN([$1 - Python2(pyjson)], [$2], [$3], [$4], [$5], [$6], [$HAVE_PYTHON2], [$PYTHON2]) - OVSDB_CHECK_IDL_TCP6_PYN([$1 - Python3], [$2], [$3], [$4], [$5], [$6], - [$HAVE_PYTHON3], [$PYTHON3])]) + OVSDB_CHECK_IDL_TCP6_PYN([$1 - Python2(cjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON2], [$PYTHON2], C) + OVSDB_CHECK_IDL_TCP6_PYN([$1 - Python3(pyjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3]) + OVSDB_CHECK_IDL_TCP6_PYN([$1 - Python3(cjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3], C)]) m4_define([OVSDB_CHECK_IDL_TCP6_MULTIPLE_REMOTES_PYN], [AT_SETUP([$1 - tcp6]) AT_SKIP_IF([test $7 = no]) AT_SKIP_IF([test "$IS_WIN32" = "yes"]) AT_SKIP_IF([test $HAVE_IPV6 = no]) + AT_SKIP_IF([test $9 = C && ! $8 -c "import ovs._json" 2>/dev/null]) AT_KEYWORDS([ovsdb server idl positive Python with tcp6 socket $5]) AT_CHECK([ovsdb_start_idltest "ptcp:0:[[::1]]"]) PARSE_LISTENING_PORT([ovsdb-server.log], [TCP_PORT]) @@ -171,7 +197,7 @@ m4_define([OVSDB_CHECK_IDL_TCP6_MULTIPLE_REMOTES_PYN], remote="tcp:[[::1]]:$WRONG_PORT_1,tcp:[[::1]]:$TCP_PORT,tcp:[[::1]]:$WRONG_PORT_2" m4_if([$2], [], [], [AT_CHECK([ovsdb-client transact "tcp:[[::1]]:$TCP_PORT" $2], [0], [ignore], [ignore])]) - AT_CHECK([$8 $srcdir/test-ovsdb.py -t10 idl $srcdir/idltest.ovsschema $remote $3], + AT_CHECK([$8 $srcdir/test-ovsdb.py -j m4_default([$9], python) -t10 idl $srcdir/idltest.ovsschema $remote $3], [0], [stdout], [ignore]) AT_CHECK([sort stdout | uuidfilt]m4_if([$6],,, [[| $6]]), [0], [$4]) @@ -179,10 +205,14 @@ m4_define([OVSDB_CHECK_IDL_TCP6_MULTIPLE_REMOTES_PYN], AT_CLEANUP]) m4_define([OVSDB_CHECK_IDL_TCP6_MULTIPLE_REMOTES_PY], - [OVSDB_CHECK_IDL_TCP6_MULTIPLE_REMOTES_PYN([$1 - Python2 (multiple remotes)], [$2], [$3], [$4], [$5], [$6], + [OVSDB_CHECK_IDL_TCP6_MULTIPLE_REMOTES_PYN([$1 - Python2 (pyjson multiple remotes)], [$2], [$3], [$4], [$5], [$6], [$HAVE_PYTHON], [$PYTHON]) - OVSDB_CHECK_IDL_TCP6_MULTIPLE_REMOTES_PYN([$1 - Python3 (multiple remotes)], [$2], [$3], [$4], [$5], [$6], - [$HAVE_PYTHON3], [$PYTHON3])]) + OVSDB_CHECK_IDL_TCP6_MULTIPLE_REMOTES_PYN([$1 - Python2 (cjson multiple remotes)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON], [$PYTHON], C) + OVSDB_CHECK_IDL_TCP6_MULTIPLE_REMOTES_PYN([$1 - Python3 (pyjson multiple remotes)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3]) + OVSDB_CHECK_IDL_TCP6_MULTIPLE_REMOTES_PYN([$1 - Python3 (cjson multiple remotes)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3], C)]) # same as OVSDB_CHECK_IDL but uses the Python IDL implementation with SSL m4_define([OVSDB_CHECK_IDL_SSL_PYN], @@ -192,6 +222,7 @@ m4_define([OVSDB_CHECK_IDL_SSL_PYN], $8 -c "import OpenSSL.SSL" SSL_PRESENT=$? AT_SKIP_IF([test $SSL_PRESENT != 0]) + AT_SKIP_IF([test $9 = C && ! $8 -c "import ovs._json" 2>/dev/null]) AT_KEYWORDS([ovsdb server idl positive Python with ssl socket $5]) AT_CHECK([ovsdb-tool create db $abs_srcdir/idltest.ovsschema], [0], [stdout], [ignore]) @@ -210,7 +241,7 @@ m4_define([OVSDB_CHECK_IDL_SSL_PYN], --certificate=$PKIDIR/testpki-cert2.pem \ --ca-cert=$PKIDIR/testpki-cacert.pem \ transact "ssl:127.0.0.1:$TCP_PORT" $2], [0], [ignore], [ignore])]) - AT_CHECK([$8 $srcdir/test-ovsdb.py -t10 idl $srcdir/idltest.ovsschema \ + AT_CHECK([$8 $srcdir/test-ovsdb.py -j m4_default([$9], python) -t10 idl $srcdir/idltest.ovsschema \ ssl:127.0.0.1:$TCP_PORT $PKIDIR/testpki-privkey.pem \ $PKIDIR/testpki-cert.pem $PKIDIR/testpki-cacert.pem $3], [0], [stdout], [ignore]) @@ -220,10 +251,14 @@ m4_define([OVSDB_CHECK_IDL_SSL_PYN], AT_CLEANUP]) m4_define([OVSDB_CHECK_IDL_SSL_PY], - [OVSDB_CHECK_IDL_SSL_PYN([$1 - Python2], [$2], [$3], [$4], [$5], [$6], + [OVSDB_CHECK_IDL_SSL_PYN([$1 - Python2(pyjson)], [$2], [$3], [$4], [$5], [$6], [$HAVE_PYTHON], [$PYTHON]) - OVSDB_CHECK_IDL_SSL_PYN([$1 - Python3], [$2], [$3], [$4], [$5], [$6], - [$HAVE_PYTHON3], [$PYTHON3])]) + OVSDB_CHECK_IDL_SSL_PYN([$1 - Python2(cjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON], [$PYTHON], C) + OVSDB_CHECK_IDL_SSL_PYN([$1 - Python3(pyjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3]) + OVSDB_CHECK_IDL_SSL_PYN([$1 - Python3(cjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3], C)]) m4_define([OVSDB_CHECK_IDL], [OVSDB_CHECK_IDL_C($@) @@ -239,6 +274,7 @@ m4_define([OVSDB_CHECK_IDL], m4_define([OVSDB_CHECK_IDL_PASSIVE_TCP_PYN], [AT_SETUP([$1 ptcp]) AT_SKIP_IF([test $7 = no]) + AT_SKIP_IF([test $9 = C && ! $8 -c "import ovs._json" 2>/dev/null]) AT_KEYWORDS([ovsdb server idl positive Python with tcp socket $5]) # find free TCP port AT_CHECK([ovsdb_start_idltest "ptcp:0:127.0.0.1"]) @@ -248,7 +284,7 @@ m4_define([OVSDB_CHECK_IDL_PASSIVE_TCP_PYN], # start OVSDB server in passive mode AT_CHECK([ovsdb_start_idltest "tcp:127.0.0.1:$TCP_PORT"]) - AT_CHECK([$8 $srcdir/test-ovsdb.py -t10 idl_passive $srcdir/idltest.ovsschema ptcp:127.0.0.1:$TCP_PORT $3], + AT_CHECK([$8 $srcdir/test-ovsdb.py -j m4_default([$9], python) -t10 idl_passive $srcdir/idltest.ovsschema ptcp:127.0.0.1:$TCP_PORT $3], [0], [stdout], [ignore]) AT_CHECK([sort stdout | uuidfilt]m4_if([$6],,, [[| $6]]), [0], [$4]) @@ -257,10 +293,14 @@ m4_define([OVSDB_CHECK_IDL_PASSIVE_TCP_PYN], ]) m4_define([OVSDB_CHECK_IDL_PASSIVE_TCP_PY], - [OVSDB_CHECK_IDL_PASSIVE_TCP_PYN([$1 - Python2], [$2], [$3], [$4], [$5], [$6], + [OVSDB_CHECK_IDL_PASSIVE_TCP_PYN([$1 - Python2(pyjson)], [$2], [$3], [$4], [$5], [$6], [$HAVE_PYTHON2], [$PYTHON2]) - OVSDB_CHECK_IDL_PASSIVE_TCP_PYN([$1 - Python3], [$2], [$3], [$4], [$5], [$6], - [$HAVE_PYTHON3], [$PYTHON3])]) + OVSDB_CHECK_IDL_PASSIVE_TCP_PYN([$1 - Python2(cjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON2], [$PYTHON2], C) + OVSDB_CHECK_IDL_PASSIVE_TCP_PYN([$1 - Python3(pyjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3]) + OVSDB_CHECK_IDL_PASSIVE_TCP_PYN([$1 - Python3(cjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3], C)]) OVSDB_CHECK_IDL_PASSIVE_TCP_PY([simple passive idl, initially empty, select empty], [], @@ -433,11 +473,12 @@ OVSDB_CHECK_IDL([simple idl, writing via IDL with unicode], m4_define([OVSDB_CHECK_IDL_PYN_WITH_EXPOUT], [AT_SETUP([$1]) AT_SKIP_IF([test $7 = no]) + AT_SKIP_IF([test $9 = C && ! $8 -c "import ovs._json" 2>/dev/null]) AT_KEYWORDS([ovsdb server idl positive Python $5]) AT_CHECK([ovsdb_start_idltest]) m4_if([$2], [], [], [AT_CHECK([ovsdb-client transact unix:socket $2], [0], [ignore], [ignore])]) - AT_CHECK([$8 $srcdir/test-ovsdb.py -t10 idl $srcdir/idltest.ovsschema unix:socket $3], + AT_CHECK([$8 $srcdir/test-ovsdb.py -j m4_default([$9], python) -t10 idl $srcdir/idltest.ovsschema unix:socket $3], [0], [stdout], [ignore]) echo "$4" > expout AT_CHECK([sort stdout | uuidfilt]m4_if([$6],,, [[| $6]]), @@ -446,10 +487,14 @@ m4_define([OVSDB_CHECK_IDL_PYN_WITH_EXPOUT], AT_CLEANUP]) m4_define([OVSDB_CHECK_IDL_PY_WITH_EXPOUT], - [OVSDB_CHECK_IDL_PYN_WITH_EXPOUT([$1 - Python2], [$2], [$3], [$4], [$5], [$6], + [OVSDB_CHECK_IDL_PYN_WITH_EXPOUT([$1 - Python2(pyjson)], [$2], [$3], [$4], [$5], [$6], [$HAVE_PYTHON2], [$PYTHON2]) - OVSDB_CHECK_IDL_PYN_WITH_EXPOUT([$1 - Python3], [$2], [$3], [$4], [$5], [$6], - [$HAVE_PYTHON3], [$PYTHON3])]) + OVSDB_CHECK_IDL_PYN_WITH_EXPOUT([$1 - Python2(cjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON2], [$PYTHON2], C) + OVSDB_CHECK_IDL_PYN_WITH_EXPOUT([$1 - Python3(pyjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3]) + OVSDB_CHECK_IDL_PYN_WITH_EXPOUT([$1 - Python3(cjson)], [$2], [$3], [$4], [$5], [$6], + [$HAVE_PYTHON3], [$PYTHON3], C)]) OVSDB_CHECK_IDL_PY_WITH_EXPOUT([simple idl, writing large data via IDL with unicode], [['["idltest", @@ -968,11 +1013,12 @@ AT_CLEANUP m4_define([OVSDB_CHECK_IDL_FETCH_COLUMNS_PYN], [AT_SETUP([$1 fetch]) AT_SKIP_IF([test $8 = no]) + AT_SKIP_IF([test $10 = C && ! $9 -c "import ovs._json" 2>/dev/null]) AT_KEYWORDS([ovsdb server idl positive Python increment fetch $6]) AT_CHECK([ovsdb_start_idltest]) m4_if([$2], [], [], [AT_CHECK([ovsdb-client transact unix:socket $2], [0], [ignore], [ignore])]) - AT_CHECK([$9 $srcdir/test-ovsdb.py -t10 idl $srcdir/idltest.ovsschema unix:socket [$3] $4], + AT_CHECK([$9 $srcdir/test-ovsdb.py -j m4_default([$10], python) -t10 idl $srcdir/idltest.ovsschema unix:socket [$3] $4], [0], [stdout], [ignore]) AT_CHECK([sort stdout | uuidfilt]m4_if([$7],,, [[| $7]]), [0], [$5]) @@ -980,10 +1026,14 @@ m4_define([OVSDB_CHECK_IDL_FETCH_COLUMNS_PYN], AT_CLEANUP]) m4_define([OVSDB_CHECK_IDL_FETCH_COLUMNS_PY], - [OVSDB_CHECK_IDL_FETCH_COLUMNS_PYN([$1 - Python2], [$2], [$3], [$4], [$5], [$6], [$7], + [OVSDB_CHECK_IDL_FETCH_COLUMNS_PYN([$1 - Python2(pyjson)], [$2], [$3], [$4], [$5], [$6], [$7], [$HAVE_PYTHON2], [$PYTHON2]) - OVSDB_CHECK_IDL_FETCH_COLUMNS_PYN([$1 - Python3], [$2], [$3], [$4], [$5], [$6], [$7], - [$HAVE_PYTHON3], [$PYTHON3])]) + OVSDB_CHECK_IDL_FETCH_COLUMNS_PYN([$1 - Python2(cjson)], [$2], [$3], [$4], [$5], [$6], [$7], + [$HAVE_PYTHON2], [$PYTHON2], C) + OVSDB_CHECK_IDL_FETCH_COLUMNS_PYN([$1 - Python3(pyjson)], [$2], [$3], [$4], [$5], [$6], [$7], + [$HAVE_PYTHON3], [$PYTHON3]) + OVSDB_CHECK_IDL_FETCH_COLUMNS_PYN([$1 - Python3(cjson)], [$2], [$3], [$4], [$5], [$6], [$7], + [$HAVE_PYTHON3], [$PYTHON3], C)]) m4_define([OVSDB_CHECK_IDL_FETCH_COLUMNS], [OVSDB_CHECK_IDL_FETCH_COLUMNS_PY($@)]) @@ -1019,10 +1069,11 @@ OVSDB_CHECK_IDL_FETCH_COLUMNS([simple idl, initially populated], m4_define([OVSDB_CHECK_IDL_WO_MONITOR_COND_PYN], [AT_SETUP([$1]) AT_SKIP_IF([test $6 = no]) + AT_SKIP_IF([test $8 = C && ! $7 -c "import ovs._json" 2>/dev/null]) AT_KEYWORDS([ovsdb server idl Python monitor $4]) AT_CHECK([ovsdb_start_idltest]) AT_CHECK([ovs-appctl -t ovsdb-server ovsdb-server/disable-monitor-cond]) - AT_CHECK([$7 $srcdir/test-ovsdb.py -t10 idl $srcdir/idltest.ovsschema unix:socket $2], + AT_CHECK([$7 $srcdir/test-ovsdb.py -j m4_default([$8], python) -t10 idl $srcdir/idltest.ovsschema unix:socket $2], [0], [stdout], [ignore]) AT_CHECK([sort stdout | uuidfilt]m4_if([$5],,, [[| $5]]), [0], [$3]) @@ -1030,10 +1081,14 @@ m4_define([OVSDB_CHECK_IDL_WO_MONITOR_COND_PYN], AT_CLEANUP]) m4_define([OVSDB_CHECK_IDL_WO_MONITOR_COND_PY], - [OVSDB_CHECK_IDL_WO_MONITOR_COND_PYN([$1 - Python2], [$2], [$3], [$4], [$5], + [OVSDB_CHECK_IDL_WO_MONITOR_COND_PYN([$1 - Python2(pyjson)], [$2], [$3], [$4], [$5], [$HAVE_PYTHON2], [$PYTHON2]) - OVSDB_CHECK_IDL_WO_MONITOR_COND_PYN([$1 - Python3], [$2], [$3], [$4], [$5], - [$HAVE_PYTHON3], [$PYTHON3])]) + OVSDB_CHECK_IDL_WO_MONITOR_COND_PYN([$1 - Python2(cjson)], [$2], [$3], [$4], [$5], + [$HAVE_PYTHON2], [$PYTHON2], C) + OVSDB_CHECK_IDL_WO_MONITOR_COND_PYN([$1 - Python3(pyjson)], [$2], [$3], [$4], [$5], + [$HAVE_PYTHON3], [$PYTHON3]) + OVSDB_CHECK_IDL_WO_MONITOR_COND_PYN([$1 - Python3(cjson)], [$2], [$3], [$4], [$5], + [$HAVE_PYTHON3], [$PYTHON3], C)]) m4_define([OVSDB_CHECK_IDL_WO_MONITOR_COND], @@ -1356,7 +1411,8 @@ OVSDB_CHECK_IDL_PY([partial-set idl], ]]) m4_define([OVSDB_CHECK_IDL_NOTIFY_PYN], - [OVSDB_CHECK_IDL_PYN([$1], [], [$2], [$3], [notify $4], [$5], [$6], [$7])]) + [OVSDB_CHECK_IDL_PYN([$1(pyjson)], [], [$2], [$3], [notify $4], [$5], [$6], [$7]) + OVSDB_CHECK_IDL_PYN([$1(cjson)], [], [$2], [$3], [notify $4], [$5], [$6], [$7], C)]) m4_define([OVSDB_CHECK_IDL_NOTIFY_PY], [OVSDB_CHECK_IDL_NOTIFY_PYN([$1 - Python2], [$2], [$3], [$4], [$5], @@ -1366,7 +1422,8 @@ m4_define([OVSDB_CHECK_IDL_NOTIFY_PY], # This test uses the Python IDL implementation with ssl m4_define([OVSDB_CHECK_IDL_NOTIFY_SSL_PYN], - [OVSDB_CHECK_IDL_SSL_PYN([$1], [], [$2], [$3], [notify $4], [$5], [$6], [$7])]) + [OVSDB_CHECK_IDL_SSL_PYN([$1(pyjson)], [], [$2], [$3], [notify $4], [$5], [$6], [$7]) + OVSDB_CHECK_IDL_SSL_PYN([$1(cjson)], [], [$2], [$3], [notify $4], [$5], [$6], [$7], C)]) m4_define([OVSDB_CHECK_IDL_NOTIFY_SSL_PY], [OVSDB_CHECK_IDL_NOTIFY_SSL_PYN([$1 - Python2], [$2], [$3], [$4], [$5], diff --git a/tests/ovsdb.at b/tests/ovsdb.at index 0c9856d..fab79de 100644 --- a/tests/ovsdb.at +++ b/tests/ovsdb.at @@ -12,7 +12,7 @@ m4_define([OVSDB_CHECK_POSITIVE], AT_CLEANUP]) # OVSDB_CHECK_POSITIVE_PY(TITLE, TEST-OVSDB-ARGS, OUTPUT, [KEYWORDS], [PREREQ], -# [PY-CHECK]) +# [PY-CHECK] [JSON-PARSER = python]) # # Runs "test-ovsdb.py TEST-OVSDB-ARGS" and checks that it exits with # status 0 and prints OUTPUT on stdout. @@ -24,14 +24,15 @@ m4_define([OVSDB_CHECK_POSITIVE], m4_define([OVSDB_CHECK_POSITIVE_PY], [AT_SETUP([$1]) AT_SKIP_IF([test $HAVE_PYTHON2 = no]) + AT_SKIP_IF([test $7 = C && ! $PYTHON2 -c "import ovs._json" 2>/dev/null]) $6 AT_KEYWORDS([ovsdb positive Python $4]) - AT_CHECK([$PYTHON2 $srcdir/test-ovsdb.py $2], [0], [$3 + AT_CHECK([$PYTHON2 $srcdir/test-ovsdb.py -j m4_default([$7], python) $2], [0], [$3 ], []) AT_CLEANUP]) # OVSDB_CHECK_POSITIVE_PY3(TITLE, TEST-OVSDB-ARGS, OUTPUT, [KEYWORDS], [PREREQ], -# [PY-CHECK]) +# [PY-CHECK] [PY3-CHECK] [JSON-PARSER = python]) # # Runs "test-ovsdb.py TEST-OVSDB-ARGS" and checks that it exits with # status 0 and prints OUTPUT on stdout. @@ -43,9 +44,10 @@ m4_define([OVSDB_CHECK_POSITIVE_PY], m4_define([OVSDB_CHECK_POSITIVE_PY3], [AT_SETUP([$1]) AT_SKIP_IF([test $HAVE_PYTHON3 = no]) + AT_SKIP_IF([test $8 = C && ! $PYTHON3 -c "import ovs._json" 2>/dev/null]) $6 AT_KEYWORDS([ovsdb positive Python $4]) - AT_CHECK([$PYTHON3 $srcdir/test-ovsdb.py $2], [0], [$3 + AT_CHECK([$PYTHON3 $srcdir/test-ovsdb.py -j m4_default([$8], python) $2], [0], [$3 ], []) AT_CLEANUP]) @@ -55,8 +57,10 @@ m4_define([OVSDB_CHECK_POSITIVE_PY3], # Runs identical C and Python tests, as specified. m4_define([OVSDB_CHECK_POSITIVE_CPY], [OVSDB_CHECK_POSITIVE([$1 - C], [$2], [$3], [$4], [$5]) - OVSDB_CHECK_POSITIVE_PY([$1 - Python2], [$2], [$3], [$4], [$5], [$6]) - OVSDB_CHECK_POSITIVE_PY3([$1 - Python3], [$2], [$3], [$4], [$5], [$7])]) + OVSDB_CHECK_POSITIVE_PY([$1 - Python2(pyjson)], [$2], [$3], [$4], [$5], [$6]) + OVSDB_CHECK_POSITIVE_PY([$1 - Python2(cjson)], [$2], [$3], [$4], [$5], [$6], C) + OVSDB_CHECK_POSITIVE_PY3([$1 - Python3(pyjson)], [$2], [$3], [$4], [$5], [$7]) + OVSDB_CHECK_POSITIVE_PY3([$1 - Python3(cjson)], [$2], [$3], [$4], [$5], [$7], C)]) # OVSDB_CHECK_NEGATIVE(TITLE, TEST-OVSDB-ARGS, OUTPUT, [KEYWORDS], [PREREQ]) # @@ -78,7 +82,7 @@ m4_define([OVSDB_CHECK_NEGATIVE], [0], [ignore], [ignore]) AT_CLEANUP]) -# OVSDB_CHECK_NEGATIVE_PY(TITLE, TEST-OVSDB-ARGS, OUTPUT, [KEYWORDS], [PREREQ]) +# OVSDB_CHECK_NEGATIVE_PY(TITLE, TEST-OVSDB-ARGS, OUTPUT, [KEYWORDS], [PREREQ] [JSON-PARSER = python]) # # Runs "test-ovsdb TEST-OVSDB-ARGS" and checks that it exits with # status 1 and that its output on stdout contains substring OUTPUT. @@ -86,8 +90,9 @@ m4_define([OVSDB_CHECK_NEGATIVE], m4_define([OVSDB_CHECK_NEGATIVE_PY], [AT_SETUP([$1]) AT_SKIP_IF([test $HAVE_PYTHON2 = no]) - AT_KEYWORDS([ovsdb negative $4]) - AT_CHECK([$PYTHON2 $srcdir/test-ovsdb.py $2], [1], [], [stderr]) + AT_SKIP_IF([test $6 = C && ! $PYTHON2 -c "import ovs._json" 2>/dev/null]) + AT_KEYWORDS([ovsdb negative Python $4]) + AT_CHECK([$PYTHON2 $srcdir/test-ovsdb.py -j m4_default([$6], python) $2], [1], [], [stderr]) m4_assert(m4_len([$3])) AT_CHECK( [if grep -F -e "AS_ESCAPE([$3])" stderr @@ -99,7 +104,7 @@ m4_define([OVSDB_CHECK_NEGATIVE_PY], [0], [ignore], [ignore]) AT_CLEANUP]) -# OVSDB_CHECK_NEGATIVE_PY3(TITLE, TEST-OVSDB-ARGS, OUTPUT, [KEYWORDS], [PREREQ]) +# OVSDB_CHECK_NEGATIVE_PY3(TITLE, TEST-OVSDB-ARGS, OUTPUT, [KEYWORDS], [PREREQ], [JSON-PARSER = python]) # # Runs "test-ovsdb TEST-OVSDB-ARGS" and checks that it exits with # status 1 and that its output on stdout contains substring OUTPUT. @@ -107,8 +112,9 @@ m4_define([OVSDB_CHECK_NEGATIVE_PY], m4_define([OVSDB_CHECK_NEGATIVE_PY], [AT_SETUP([$1]) AT_SKIP_IF([test $HAVE_PYTHON3 = no]) - AT_KEYWORDS([ovsdb negative $4]) - AT_CHECK([$PYTHON3 $srcdir/test-ovsdb.py $2], [1], [], [stderr]) + AT_SKIP_IF([test $6 = C && ! $PYTHON3 -c "import ovs._json" 2>/dev/null]) + AT_KEYWORDS([ovsdb negative Python $4]) + AT_CHECK([$PYTHON3 $srcdir/test-ovsdb.py -j m4_default([$6], python) $2], [1], [], [stderr]) m4_assert(m4_len([$3])) AT_CHECK( [if grep -F -e "AS_ESCAPE([$3])" stderr @@ -126,8 +132,10 @@ m4_define([OVSDB_CHECK_NEGATIVE_PY], # Runs identical C and Python tests, as specified. m4_define([OVSDB_CHECK_NEGATIVE_CPY], [OVSDB_CHECK_NEGATIVE([$1 - C], [$2], [$3], [$4], [$5]) - OVSDB_CHECK_NEGATIVE_PY([$1 - Python2], [$2], [$3], [$4], [$5]) - OVSDB_CHECK_NEGATIVE_PY3([$1 - Python3], [$2], [$3], [$4], [$5])]) + OVSDB_CHECK_NEGATIVE_PY([$1 - Python2(pyjson)], [$2], [$3], [$4], [$5]) + OVSDB_CHECK_NEGATIVE_PY([$1 - Python2(cjson)], [$2], [$3], [$4], [$5], C) + OVSDB_CHECK_NEGATIVE_PY3([$1 - Python3(pyjson)], [$2], [$3], [$4], [$5]) + OVSDB_CHECK_NEGATIVE_PY3([$1 - Python3(cjson)], [$2], [$3], [$4], [$5], C)]) OVS_START_SHELL_HELPERS ovsdb_client_wait() { diff --git a/tests/test-json.py b/tests/test-json.py index 582aa72..788777e 100644 --- a/tests/test-json.py +++ b/tests/test-json.py @@ -66,15 +66,25 @@ def main(argv): sys.stderr = codecs.getwriter("utf-8")(sys.stderr) try: - options, args = getopt.gnu_getopt(argv[1:], '', ['multiple']) + options, args = getopt.gnu_getopt(argv[1:], 'j:', ['multiple', 'json-parser']) except getopt.GetoptError as geo: sys.stderr.write("%s: %s\n" % (argv0, geo.msg)) sys.exit(1) multiple = False + ORIG_PARSER = ovs.json.PARSER + ovs.json.PARSER = ovs.json.PARSER_PY for key, value in options: if key == '--multiple': multiple = True + elif key in ('-j', '--json-parser'): + if value == "python": + ovs.json.PARSER = ovs.json.PARSER_PY + elif value in ('C', 'c'): + if ORIG_PARSER != ovs.json.PARSER_C: + raise error.Error("C parser selected, but not compiled") + else: + ovs.json.PARSER = ovs.json.PARSER_C else: sys.stderr.write("%s: unhandled option %s\n" % (argv0, key)) sys.exit(1) diff --git a/tests/test-ovsdb.py b/tests/test-ovsdb.py index 14491a2..b99109d 100644 --- a/tests/test-ovsdb.py +++ b/tests/test-ovsdb.py @@ -23,6 +23,7 @@ import uuid import ovs.db.idl import ovs.db.schema import ovs.db.types +import ovs.json import ovs.ovsuuid import ovs.poller import ovs.stream @@ -815,14 +816,20 @@ The following options are also available: def main(argv): try: - options, args = getopt.gnu_getopt(argv[1:], 't:h', + options, args = getopt.gnu_getopt(argv[1:], 't:h:j:', ['timeout', - 'help']) + 'help', 'json-parser']) except getopt.GetoptError as geo: sys.stderr.write("%s: %s\n" % (ovs.util.PROGRAM_NAME, geo.msg)) sys.exit(1) timeout = None + # Save the old version to detect whether we support the C Parser + # but then override to the Python parser because it is always available + # and is the historical default + ORIG_PARSER = ovs.json.PARSER + ovs.json.PARSER = ovs.json.PARSER_PY + for key, value in options: if key in ['-h', '--help']: usage() @@ -834,6 +841,17 @@ def main(argv): except TypeError: raise error.Error("value %s on -t or --timeout is not at " "least 1" % value) + elif key in ['-j', '--json-parser']: + if value == "python": + ovs.json.PARSER = ovs.json.PARSER_PY + elif value in ('C', 'c'): + if ORIG_PARSER != ovs.json.PARSER_C: + raise error.Error("C parser selected, but not compiled") + else: + ovs.json.PARSER = ovs.json.PARSER_C + else: + raise error.Error( + "invalid option: %s, json-parser must be 'C' or json") else: sys.exit(0) From patchwork Wed Sep 19 22:47:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Terry Wilson X-Patchwork-Id: 971983 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42Fw5422G6z9sBs for ; Thu, 20 Sep 2018 08:48:40 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 787BA10F0; Wed, 19 Sep 2018 22:47:51 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 9552F10DF for ; Wed, 19 Sep 2018 22:47:50 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 4FB717E1 for ; Wed, 19 Sep 2018 22:47:50 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.26]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E1EB6A459A for ; Wed, 19 Sep 2018 22:47:49 +0000 (UTC) Received: from new-host.redhat.com (ovpn-112-39.phx2.redhat.com [10.3.112.39]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3CD6C30912F4; Wed, 19 Sep 2018 22:47:49 +0000 (UTC) From: Terry Wilson To: dev@openvswitch.org Date: Wed, 19 Sep 2018 17:47:29 -0500 Message-Id: <1537397249-30180-3-git-send-email-twilson@redhat.com> In-Reply-To: <1537397249-30180-1-git-send-email-twilson@redhat.com> References: <1537397249-30180-1-git-send-email-twilson@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.26 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 19 Sep 2018 22:47:49 +0000 (UTC) X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH 2/2] Work around Python/C JSON unicode differences X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org The OVS C-based JSON parser operates on bytes, so the parser_feed function returns the number of bytes that are processed. The pure Python JSON parser currently operates on unicode, so it expects that Parser.feed() returns a number of characters. This difference leads to parsing errors when unicode characters are passed to the C JSON parser from Python. Signed-off-by: Terry Wilson --- python/ovs/jsonrpc.py | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/python/ovs/jsonrpc.py b/python/ovs/jsonrpc.py index 4873cff..1323ba7 100644 --- a/python/ovs/jsonrpc.py +++ b/python/ovs/jsonrpc.py @@ -272,7 +272,8 @@ class Connection(object): # data, so we convert it here as soon as possible. if data and not error: try: - data = decoder.decode(data) + if six.PY3 or ovs.json.PARSER == ovs.json.PARSER_PY: + data = decoder.decode(data) except UnicodeError: error = errno.EILSEQ if error: @@ -298,7 +299,11 @@ class Connection(object): else: if self.parser is None: self.parser = ovs.json.Parser() - self.input = self.input[self.parser.feed(self.input):] + if six.PY3 and ovs.json.PARSER == ovs.json.PARSER_C: + self.input = self.input.encode('utf-8')[ + self.parser.feed(self.input):].decode() + else: + self.input = self.input[self.parser.feed(self.input):] if self.parser.is_done(): msg = self.__process_msg() if msg: