From patchwork Mon Dec 18 02:02:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Maximets X-Patchwork-Id: 1877173 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4StjlT55LNz23yc for ; Mon, 18 Dec 2023 13:03:01 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 9D39060B69; Mon, 18 Dec 2023 02:02:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 9D39060B69 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6VIgYFlWVYMc; Mon, 18 Dec 2023 02:02:57 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id C3F1760A9B; Mon, 18 Dec 2023 02:02:56 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org C3F1760A9B Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6BE3FC0077; Mon, 18 Dec 2023 02:02:56 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id BE862C0037 for ; Mon, 18 Dec 2023 02:02:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 9155581EA9 for ; Mon, 18 Dec 2023 02:02:55 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 9155581EA9 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7M0aNRV9Kd2v for ; Mon, 18 Dec 2023 02:02:54 +0000 (UTC) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) by smtp1.osuosl.org (Postfix) with ESMTPS id 6FFF681EA3 for ; Mon, 18 Dec 2023 02:02:54 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 6FFF681EA3 Received: by mail.gandi.net (Postfix) with ESMTPSA id 121EEC0002; Mon, 18 Dec 2023 02:02:50 +0000 (UTC) From: Ilya Maximets To: ovs-dev@openvswitch.org Date: Mon, 18 Dec 2023 03:02:39 +0100 Message-ID: <20231218020249.3447973-1-i.maximets@ovn.org> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-GND-Sasl: i.maximets@ovn.org Cc: Ilya Maximets , Dumitru Ceara Subject: [ovs-dev] [PATCH 0/5] ovsdb: transaction: Bug Fixes + Reuse of column diffs. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This patch set contains a few bug fixes for transaction processing and an optimization. The first patch is just a bug fix. The 2nd and 3rd are also bug fixes, but they only affect performance, not correctness. The last two patches intruduce a new functionality to reuse the column diffs stored in the file during transaction replay. All in all this patch set gives about 4x performance improvement for transaction processing after reading from the storage, i.e. significantly speeds up initial database file read after restart and improves performance of transactions in clustered databases (storage_read from raft module). For example, with this change applied, initial read of OVSDB file containing 136K transactions for large OVN port groups and address sets on my laptop takes 11 seconds vs 43 seconds without. Ilya Maximets (5): ovsdb: Fix incorrect sharing of UUID and _version columns. ovsdb: transaction: Avoid diffs for different type references. ovsdb: transaction: Don't try to diff unchanged columns. ovsdb: Preserve column diffs read from the storage. ovsdb: transaction: Calculate added/removed from diff. lib/ovsdb-data.c | 26 +++++++++++ lib/ovsdb-data.h | 1 + lib/ovsdb-types.h | 12 +++++ ovsdb/execution.c | 14 ++++-- ovsdb/file.c | 22 ++++++--- ovsdb/ovsdb-server.c | 7 ++- ovsdb/row.h | 2 + ovsdb/table.c | 6 ++- ovsdb/transaction.c | 104 ++++++++++++++++++++++++++++++++----------- ovsdb/transaction.h | 6 ++- tests/test-ovsdb.c | 2 +- 11 files changed, 159 insertions(+), 43 deletions(-)