From patchwork Tue May 10 02:40:23 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Terry Wilson X-Patchwork-Id: 620408 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from archives.nicira.com (archives.nicira.com [96.126.127.54]) by ozlabs.org (Postfix) with ESMTP id 3r3k4x5cBCz9t7D for ; Tue, 10 May 2016 12:40:33 +1000 (AEST) Received: from archives.nicira.com (localhost [127.0.0.1]) by archives.nicira.com (Postfix) with ESMTP id C143E106BC; Mon, 9 May 2016 19:40:30 -0700 (PDT) X-Original-To: dev@openvswitch.org Delivered-To: dev@openvswitch.org Received: from mx3v3.cudamail.com (mx3.cudamail.com [64.34.241.5]) by archives.nicira.com (Postfix) with ESMTPS id A5621106BB for ; Mon, 9 May 2016 19:40:29 -0700 (PDT) Received: from bar6.cudamail.com (localhost [127.0.0.1]) by mx3v3.cudamail.com (Postfix) with ESMTPS id 3E17D161128 for ; Mon, 9 May 2016 20:40:29 -0600 (MDT) X-ASG-Debug-ID: 1462848028-0b32376530414d60001-byXFYA Received: from mx1-pf2.cudamail.com ([192.168.24.2]) by bar6.cudamail.com with ESMTP id iGC6yYXyAyEHfGyw (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 09 May 2016 20:40:28 -0600 (MDT) X-Barracuda-Envelope-From: terry@centos7-ds1.localdomain X-Barracuda-RBL-Trusted-Forwarder: 192.168.24.2 Received: from unknown (HELO centos7-ds1.localdomain) (23.255.230.241) by mx1-pf2.cudamail.com with SMTP; 10 May 2016 02:40:28 -0000 Received-SPF: none (mx1-pf2.cudamail.com: domain at centos7-ds1.localdomain does not designate permitted sender hosts) X-Barracuda-Apparent-Source-IP: 23.255.230.241 X-Barracuda-RBL-IP: 23.255.230.241 Received: by centos7-ds1.localdomain (Postfix, from userid 1000) id AA079152799; Mon, 9 May 2016 21:40:29 -0500 (CDT) X-CudaMail-Envelope-Sender: terry@centos7-ds1.localdomain From: Terry Wilson To: dev@openvswitch.org X-CudaMail-Whitelist-To: dev@openvswitch.org X-CudaMail-MID: CM-E2-508092314 X-CudaMail-DTE: 050916 X-CudaMail-Originating-IP: 23.255.230.241 Date: Mon, 9 May 2016 21:40:23 -0500 X-ASG-Orig-Subj: [##CM-E2-508092314##][PATCH 2/2] JSON serialization via Python's json lib Message-Id: <1462848023-127253-2-git-send-email-twilson@redhat.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1462848023-127253-1-git-send-email-twilson@redhat.com> References: <1462848023-127253-1-git-send-email-twilson@redhat.com> X-Barracuda-Connect: UNKNOWN[192.168.24.2] X-Barracuda-Start-Time: 1462848028 X-Barracuda-Encrypted: DHE-RSA-AES256-SHA X-Barracuda-URL: https://web.cudamail.com:443/cgi-mod/mark.cgi X-ASG-Whitelist: Header =?UTF-8?B?eFwtY3VkYW1haWxcLXdoaXRlbGlzdFwtdG8=?= X-Virus-Scanned: by bsmtpd at cudamail.com X-Barracuda-BRTS-Status: 1 Subject: [ovs-dev] [PATCH 2/2] JSON serialization via Python's json lib X-BeenThere: dev@openvswitch.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: dev-bounces@openvswitch.org Sender: "dev" There is no particularly good reason to use our own Python JSON serialization implementation when serialization can be done faster with Python's built-in JSON library. A few tests were changed due to Python's default JSON library returning slightly more precise floating point numbers and returning '0.0' for 1e-9999 where the in-tree version returns '0'. --- python/ovs/json.py | 106 +++++------------------------------------------------ tests/json.at | 38 +++++++++++++++---- 2 files changed, 40 insertions(+), 104 deletions(-) diff --git a/python/ovs/json.py b/python/ovs/json.py index f1a6499..793ac17 100644 --- a/python/ovs/json.py +++ b/python/ovs/json.py @@ -12,11 +12,13 @@ # See the License for the specific language governing permissions and # limitations under the License. +from __future__ import absolute_import +import functools +import json import re import sys import six -from six.moves import range try: import ovs._json @@ -25,112 +27,24 @@ except ImportError: __pychecker__ = 'no-stringiter' -escapes = {ord('"'): u"\\\"", - ord("\\"): u"\\\\", - ord("\b"): u"\\b", - ord("\f"): u"\\f", - ord("\n"): u"\\n", - ord("\r"): u"\\r", - ord("\t"): u"\\t"} -for esc in range(32): - if esc not in escapes: - escapes[esc] = u"\\u%04x" % esc - SPACES_PER_LEVEL = 2 - - -class _Serializer(object): - def __init__(self, stream, pretty, sort_keys): - self.stream = stream - self.pretty = pretty - self.sort_keys = sort_keys - self.depth = 0 - - def __serialize_string(self, s): - self.stream.write(u'"%s"' % ''.join(escapes.get(ord(c), c) for c in s)) - - def __indent_line(self): - if self.pretty: - self.stream.write('\n') - self.stream.write(' ' * (SPACES_PER_LEVEL * self.depth)) - - def serialize(self, obj): - if obj is None: - self.stream.write(u"null") - elif obj is False: - self.stream.write(u"false") - elif obj is True: - self.stream.write(u"true") - elif isinstance(obj, six.integer_types): - self.stream.write(u"%d" % obj) - elif isinstance(obj, float): - self.stream.write("%.15g" % obj) - elif isinstance(obj, six.text_type): - # unicode() on Python 2, or str() in Python 3 (always unicode) - self.__serialize_string(obj) - elif isinstance(obj, str): - # This is for Python 2, where this comes out to unicode(str()). - # For Python 3, it's str(str()), but it's harmless. - self.__serialize_string(six.text_type(obj)) - elif isinstance(obj, dict): - self.stream.write(u"{") - - self.depth += 1 - self.__indent_line() - - if self.sort_keys: - items = sorted(obj.items()) - else: - items = six.iteritems(obj) - for i, (key, value) in enumerate(items): - if i > 0: - self.stream.write(u",") - self.__indent_line() - self.__serialize_string(six.text_type(key)) - self.stream.write(u":") - if self.pretty: - self.stream.write(u' ') - self.serialize(value) - - self.stream.write(u"}") - self.depth -= 1 - elif isinstance(obj, (list, tuple)): - self.stream.write(u"[") - self.depth += 1 - - if obj: - self.__indent_line() - - for i, value in enumerate(obj): - if i > 0: - self.stream.write(u",") - self.__indent_line() - self.serialize(value) - - self.depth -= 1 - self.stream.write(u"]") - else: - raise Exception("can't serialize %s as JSON" % obj) +dumper = functools.partial(json.dumps, separators=(",", ":"), + ensure_ascii=False) def to_stream(obj, stream, pretty=False, sort_keys=True): - _Serializer(stream, pretty, sort_keys).serialize(obj) + stream.write(dumper(obj, indent=SPACES_PER_LEVEL if pretty else None, + sort_keys=sort_keys)) def to_file(obj, name, pretty=False, sort_keys=True): - stream = open(name, "w") - try: + with open(name, "w") as stream: to_stream(obj, stream, pretty, sort_keys) - finally: - stream.close() def to_string(obj, pretty=False, sort_keys=True): - output = six.StringIO() - to_stream(obj, output, pretty, sort_keys) - s = output.getvalue() - output.close() - return s + return dumper(obj, indent=SPACES_PER_LEVEL if pretty else None, + sort_keys=sort_keys) def from_stream(stream): diff --git a/tests/json.at b/tests/json.at index 32d7fff..57a97b0 100644 --- a/tests/json.at +++ b/tests/json.at @@ -41,6 +41,12 @@ m4_define([JSON_CHECK_POSITIVE], JSON_CHECK_POSITIVE_PY([$1 - Python3], [$2], [$3], [$4], [$HAVE_PYTHON3], [$PYTHON3])]) +m4_define([JSON_CHECK_POSITIVE_PY23], + [JSON_CHECK_POSITIVE_PY([$1 - Python2], [$2], [$3], [$4], + [$HAVE_PYTHON], [$PYTHON]) + JSON_CHECK_POSITIVE_PY([$1 - Python3], [$2], [$3], [$4], + [$HAVE_PYTHON3], [$PYTHON3])]) + m4_define([JSON_CHECK_NEGATIVE_C], [AT_SETUP([$1]) AT_KEYWORDS([json negative]) @@ -216,15 +222,23 @@ JSON_CHECK_POSITIVE( # It seems likely that the following test will fail on some system that # rounds slightly differently in arithmetic or in printf, but I'd like # to keep it this way until we run into such a system. -JSON_CHECK_POSITIVE( - [large integers that overflow to reals], +JSON_CHECK_POSITIVE_C( + [C - large integers that overflow to reals], [[[9223372036854775807000, -92233720368547758080000]]], [[[9.22337203685478e+21,-9.22337203685478e+22]]]) +JSON_CHECK_POSITIVE_PY23( + [large integers that overflow to reals], + [[[9223372036854775807000, -92233720368547758080000]]], + [[[9.223372036854776e+21,-9.223372036854776e+22]]]) -JSON_CHECK_POSITIVE( - [negative zero], +JSON_CHECK_POSITIVE_C( + [C - negative zero], [[[-0, -0.0, 1e-9999, -1e-9999]]], [[[0,0,0,0]]]) +JSON_CHECK_POSITIVE_PY23( + [negative zero], + [[[-0, -0.0, 1e-9999, -1e-9999]]], + [[[0,0,0.0,0.0]]]) JSON_CHECK_POSITIVE( [reals], @@ -237,10 +251,14 @@ JSON_CHECK_POSITIVE( # It seems likely that the following test will fail on some system that # rounds slightly differently in arithmetic or in printf, but I'd like # to keep it this way until we run into such a system. -JSON_CHECK_POSITIVE( - [+/- DBL_MAX], +JSON_CHECK_POSITIVE_C( + [C - +/- DBL_MAX], [[[1.7976931348623157e+308, -1.7976931348623157e+308]]], [[[1.79769313486232e+308,-1.79769313486232e+308]]]) +JSON_CHECK_POSITIVE_PY23( + [+/- DBL_MAX], + [[[1.7976931348623157e+308, -1.7976931348623157e+308]]], + [[[1.7976931348623157e+308,-1.7976931348623157e+308]]]) JSON_CHECK_POSITIVE( [negative reals], @@ -250,10 +268,14 @@ JSON_CHECK_POSITIVE( [negative scientific notation], [[[-1e3, -1E3, -2.5E2, -1e+3, -125e-3, -3.125e-2, -3125e-05, -1.525878906e-5]]], [[[-1000,-1000,-250,-1000,-0.125,-0.03125,-0.03125,-1.525878906e-05]]]) -JSON_CHECK_POSITIVE( - [1e-9999 underflows to 0], +JSON_CHECK_POSITIVE_C( + [C - 1e-9999 underflows to 0], [[[1e-9999]]], [[[0]]]) +JSON_CHECK_POSITIVE_PY23( + [1e-9999 underflows to 0], + [[[1e-9999]]], + [[[0.0]]]) JSON_CHECK_NEGATIVE([a number by itself is not valid JSON], [1], [error: syntax error at beginning of input]) JSON_CHECK_NEGATIVE(