From patchwork Tue Jul 26 00:17:11 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Terry Wilson X-Patchwork-Id: 652465 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from archives.nicira.com (archives.nicira.com [96.126.127.54]) by ozlabs.org (Postfix) with ESMTP id 3ryzGm4D8Fz9srY for ; Tue, 26 Jul 2016 10:17:52 +1000 (AEST) Received: from archives.nicira.com (localhost [127.0.0.1]) by archives.nicira.com (Postfix) with ESMTP id 605F8109DD; Mon, 25 Jul 2016 17:17:50 -0700 (PDT) X-Original-To: dev@openvswitch.org Delivered-To: dev@openvswitch.org Received: from mx3v3.cudamail.com (mx3.cudamail.com [64.34.241.5]) by archives.nicira.com (Postfix) with ESMTPS id E873A109D7 for ; Mon, 25 Jul 2016 17:17:49 -0700 (PDT) Received: from bar6.cudamail.com (localhost [127.0.0.1]) by mx3v3.cudamail.com (Postfix) with ESMTPS id 5007516233C for ; Mon, 25 Jul 2016 18:17:49 -0600 (MDT) X-ASG-Debug-ID: 1469492262-0b323703f95007c0001-byXFYA Received: from mx3-pf1.cudamail.com ([192.168.14.2]) by bar6.cudamail.com with ESMTP id osPbM5kuX7FdlF1t (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 25 Jul 2016 18:17:42 -0600 (MDT) X-Barracuda-Envelope-From: twilson@redhat.com X-Barracuda-RBL-Trusted-Forwarder: 192.168.14.2 Received: from unknown (HELO mx1.redhat.com) (209.132.183.28) by mx3-pf1.cudamail.com with ESMTPS (DHE-RSA-AES256-SHA encrypted); 26 Jul 2016 00:17:42 -0000 Received-SPF: pass (mx3-pf1.cudamail.com: SPF record at _spf1.redhat.com designates 209.132.183.28 as permitted sender) X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-Barracuda-RBL-IP: 209.132.183.28 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 37CBCC049D67 for ; Tue, 26 Jul 2016 00:17:41 +0000 (UTC) Received: from centos7-ds1.localdomain.localdomain (unused [10.10.51.82] (may be forged)) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u6Q0HdvX029995; Mon, 25 Jul 2016 20:17:40 -0400 X-CudaMail-Envelope-Sender: twilson@redhat.com From: Terry Wilson To: dev@openvswitch.org X-CudaMail-Whitelist-To: dev@openvswitch.org X-CudaMail-MID: CM-V1-724051490 X-CudaMail-DTE: 072516 X-CudaMail-Originating-IP: 209.132.183.28 Date: Mon, 25 Jul 2016 19:17:11 -0500 X-ASG-Orig-Subj: [##CM-V1-724051490##][PATCH] JSON serialization via Python's json lib Message-Id: <1469492231-35947-2-git-send-email-twilson@redhat.com> In-Reply-To: <1469492231-35947-1-git-send-email-twilson@redhat.com> References: <1468359455-77353-3-git-send-email-twilson@redhat.com> <1469492231-35947-1-git-send-email-twilson@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 26 Jul 2016 00:17:41 +0000 (UTC) X-Barracuda-Connect: UNKNOWN[192.168.14.2] X-Barracuda-Start-Time: 1469492262 X-Barracuda-Encrypted: DHE-RSA-AES256-SHA X-Barracuda-URL: https://web.cudamail.com:443/cgi-mod/mark.cgi X-ASG-Whitelist: Header =?UTF-8?B?eFwtY3VkYW1haWxcLXdoaXRlbGlzdFwtdG8=?= X-Virus-Scanned: by bsmtpd at cudamail.com X-Barracuda-BRTS-Status: 1 Subject: [ovs-dev] [PATCH] JSON serialization via Python's json lib X-BeenThere: dev@openvswitch.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: dev-bounces@openvswitch.org Sender: "dev" There is no particularly good reason to use our own Python JSON serialization implementation when serialization can be done faster with Python's built-in JSON library. A few tests were changed due to Python's default JSON library returning slightly more precise floating point numbers. Signed-off-by: Terry Wilson --- python/ovs/json.py | 106 +++++------------------------------------------------ tests/json.at | 26 ++++++++++--- 2 files changed, 30 insertions(+), 102 deletions(-) diff --git a/python/ovs/json.py b/python/ovs/json.py index ea0400a..ddf5dd2 100644 --- a/python/ovs/json.py +++ b/python/ovs/json.py @@ -12,11 +12,13 @@ # See the License for the specific language governing permissions and # limitations under the License. +from __future__ import absolute_import +import functools +import json import re import sys import six -from six.moves import range try: import ovs._json @@ -25,112 +27,24 @@ except ImportError: __pychecker__ = 'no-stringiter' -escapes = {ord('"'): u"\\\"", - ord("\\"): u"\\\\", - ord("\b"): u"\\b", - ord("\f"): u"\\f", - ord("\n"): u"\\n", - ord("\r"): u"\\r", - ord("\t"): u"\\t"} -for esc in range(32): - if esc not in escapes: - escapes[esc] = u"\\u%04x" % esc - SPACES_PER_LEVEL = 2 - - -class _Serializer(object): - def __init__(self, stream, pretty, sort_keys): - self.stream = stream - self.pretty = pretty - self.sort_keys = sort_keys - self.depth = 0 - - def __serialize_string(self, s): - self.stream.write(u'"%s"' % ''.join(escapes.get(ord(c), c) for c in s)) - - def __indent_line(self): - if self.pretty: - self.stream.write('\n') - self.stream.write(' ' * (SPACES_PER_LEVEL * self.depth)) - - def serialize(self, obj): - if obj is None: - self.stream.write(u"null") - elif obj is False: - self.stream.write(u"false") - elif obj is True: - self.stream.write(u"true") - elif isinstance(obj, six.integer_types): - self.stream.write(u"%d" % obj) - elif isinstance(obj, float): - self.stream.write("%.15g" % obj) - elif isinstance(obj, six.text_type): - # unicode() on Python 2, or str() in Python 3 (always unicode) - self.__serialize_string(obj) - elif isinstance(obj, str): - # This is for Python 2, where this comes out to unicode(str()). - # For Python 3, it's str(str()), but it's harmless. - self.__serialize_string(six.text_type(obj)) - elif isinstance(obj, dict): - self.stream.write(u"{") - - self.depth += 1 - self.__indent_line() - - if self.sort_keys: - items = sorted(obj.items()) - else: - items = six.iteritems(obj) - for i, (key, value) in enumerate(items): - if i > 0: - self.stream.write(u",") - self.__indent_line() - self.__serialize_string(six.text_type(key)) - self.stream.write(u":") - if self.pretty: - self.stream.write(u' ') - self.serialize(value) - - self.stream.write(u"}") - self.depth -= 1 - elif isinstance(obj, (list, tuple)): - self.stream.write(u"[") - self.depth += 1 - - if obj: - self.__indent_line() - - for i, value in enumerate(obj): - if i > 0: - self.stream.write(u",") - self.__indent_line() - self.serialize(value) - - self.depth -= 1 - self.stream.write(u"]") - else: - raise Exception("can't serialize %s as JSON" % obj) +dumper = functools.partial(json.dumps, separators=(",", ":"), + ensure_ascii=False) def to_stream(obj, stream, pretty=False, sort_keys=True): - _Serializer(stream, pretty, sort_keys).serialize(obj) + stream.write(dumper(obj, indent=SPACES_PER_LEVEL if pretty else None, + sort_keys=sort_keys)) def to_file(obj, name, pretty=False, sort_keys=True): - stream = open(name, "w") - try: + with open(name, "w") as stream: to_stream(obj, stream, pretty, sort_keys) - finally: - stream.close() def to_string(obj, pretty=False, sort_keys=True): - output = six.StringIO() - to_stream(obj, output, pretty, sort_keys) - s = output.getvalue() - output.close() - return s + return dumper(obj, indent=SPACES_PER_LEVEL if pretty else None, + sort_keys=sort_keys) def from_stream(stream): diff --git a/tests/json.at b/tests/json.at index 32d7fff..ba7d4bb 100644 --- a/tests/json.at +++ b/tests/json.at @@ -1,4 +1,4 @@ -m4_define([JSON_CHECK_POSITIVE_C], +m4_define([JSON_CHECK_POSITIVE_C], [AT_SETUP([$1]) AT_KEYWORDS([json positive]) AT_CHECK([printf %s "AS_ESCAPE([$2])" > input]) @@ -11,7 +11,7 @@ m4_define([JSON_CHECK_POSITIVE_C], # JSON_CHECK_POSITIVE_PY(TITLE, INPUT, OUTPUT, TEST-JSON-ARGS, # PYTHON-CHCEK, PYTHON-BIN) # -m4_define([JSON_CHECK_POSITIVE_PY], +m4_define([JSON_CHECK_POSITIVE_PY], [AT_SETUP([$1]) AT_KEYWORDS([json positive Python]) AT_SKIP_IF([test $5 = no]) @@ -41,6 +41,12 @@ m4_define([JSON_CHECK_POSITIVE], JSON_CHECK_POSITIVE_PY([$1 - Python3], [$2], [$3], [$4], [$HAVE_PYTHON3], [$PYTHON3])]) +m4_define([JSON_CHECK_POSITIVE_PY23], + [JSON_CHECK_POSITIVE_PY([$1 - Python2], [$2], [$3], [$4], + [$HAVE_PYTHON], [$PYTHON]) + JSON_CHECK_POSITIVE_PY([$1 - Python3], [$2], [$3], [$4], + [$HAVE_PYTHON3], [$PYTHON3])]) + m4_define([JSON_CHECK_NEGATIVE_C], [AT_SETUP([$1]) AT_KEYWORDS([json negative]) @@ -216,10 +222,14 @@ JSON_CHECK_POSITIVE( # It seems likely that the following test will fail on some system that # rounds slightly differently in arithmetic or in printf, but I'd like # to keep it this way until we run into such a system. -JSON_CHECK_POSITIVE( - [large integers that overflow to reals], +JSON_CHECK_POSITIVE_C( + [C - large integers that overflow to reals], [[[9223372036854775807000, -92233720368547758080000]]], [[[9.22337203685478e+21,-9.22337203685478e+22]]]) +JSON_CHECK_POSITIVE_PY23( + [large integers that overflow to reals], + [[[9223372036854775807000, -92233720368547758080000]]], + [[[9.223372036854776e+21,-9.223372036854776e+22]]]) JSON_CHECK_POSITIVE( [negative zero], @@ -237,10 +247,14 @@ JSON_CHECK_POSITIVE( # It seems likely that the following test will fail on some system that # rounds slightly differently in arithmetic or in printf, but I'd like # to keep it this way until we run into such a system. -JSON_CHECK_POSITIVE( - [+/- DBL_MAX], +JSON_CHECK_POSITIVE_C( + [C - +/- DBL_MAX], [[[1.7976931348623157e+308, -1.7976931348623157e+308]]], [[[1.79769313486232e+308,-1.79769313486232e+308]]]) +JSON_CHECK_POSITIVE_PY23( + [+/- DBL_MAX], + [[[1.7976931348623157e+308, -1.7976931348623157e+308]]], + [[[1.7976931348623157e+308,-1.7976931348623157e+308]]]) JSON_CHECK_POSITIVE( [negative reals],