From patchwork Wed Feb 28 09:11:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Alvarez Sanchez X-Patchwork-Id: 879002 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zrqZ24w1xz9s2L for ; Wed, 28 Feb 2018 20:11:37 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id A9B471339; Wed, 28 Feb 2018 09:11:34 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 2AB2B132A for ; Wed, 28 Feb 2018 09:11:33 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 8CE7B151 for ; Wed, 28 Feb 2018 09:11:32 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A86328182D17 for ; Wed, 28 Feb 2018 09:11:31 +0000 (UTC) Received: from dhcp-240-195.mad.redhat.com (ovpn-117-248.ams2.redhat.com [10.36.117.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id 95D352026E04; Wed, 28 Feb 2018 09:11:30 +0000 (UTC) From: Daniel Alvarez To: dev@openvswitch.org Date: Wed, 28 Feb 2018 10:11:09 +0100 Message-Id: <20180228091109.6233-1-dalvarez@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 28 Feb 2018 09:11:31 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 28 Feb 2018 09:11:31 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'dalvarez@redhat.com' RCPT:'' X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_LOW, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v2] python: avoid useless JSON conversion to enhance performance X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org This patch removes a useless conversion to/from JSON in the processing of any 'modify' operations inside the process_update2 method in Python IDL implementation. Previous code will make resources creation take longer as the number of elements in the row grows because of that JSON conversion. This patch eliminates it and now the time remains consant regardless of the database contents improving performance and scaling. Reported-by: Daniel Alvarez Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2018-February/046263.html Signed-off-by: Daniel Alvarez Acked-by: Terry Wilson Acked-by: Han Zhou Acked-By: Terry Wilson Tested-By: Terry Wilson --- python/ovs/db/idl.py | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/python/ovs/db/idl.py b/python/ovs/db/idl.py index 60548bcf5..5a4d129c0 100644 --- a/python/ovs/db/idl.py +++ b/python/ovs/db/idl.py @@ -518,10 +518,8 @@ class Idl(object): if not row: raise error.Error('Modify non-existing row') - old_row_diff_json = self.__apply_diff(table, row, - row_update['modify']) - self.notify(ROW_UPDATE, row, - Row.from_json(self, table, uuid, old_row_diff_json)) + old_row = self.__apply_diff(table, row, row_update['modify']) + self.notify(ROW_UPDATE, row, Row(self, table, uuid, old_row)) changed = True else: raise error.Error(' unknown operation', @@ -584,7 +582,7 @@ class Idl(object): row_update[column.name] = self.__column_name(column) def __apply_diff(self, table, row, row_diff): - old_row_diff_json = {} + old_row = {} for column_name, datum_diff_json in six.iteritems(row_diff): column = table.columns.get(column_name) if not column: @@ -601,12 +599,12 @@ class Idl(object): % (column_name, table.name, e)) continue - old_row_diff_json[column_name] = row._data[column_name].to_json() + old_row[column_name] = row._data[column_name].copy() datum = row._data[column_name].diff(datum_diff) if datum != row._data[column_name]: row._data[column_name] = datum - return old_row_diff_json + return old_row def __row_update(self, table, row, row_json): changed = False