From patchwork Tue Jan 23 13:03:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mohammad Heib X-Patchwork-Id: 1889695 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=h32zhzVq; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TK6js6yPYz23f0 for ; Wed, 24 Jan 2024 00:04:17 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id A74E66110C; Tue, 23 Jan 2024 13:04:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org A74E66110C Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=h32zhzVq X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cLaodmPXHAKz; Tue, 23 Jan 2024 13:04:14 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 948966111E; Tue, 23 Jan 2024 13:04:13 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 948966111E Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 61766C0077; Tue, 23 Jan 2024 13:04:13 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 38827C0037 for ; Tue, 23 Jan 2024 13:04:12 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 00DED416C8 for ; Tue, 23 Jan 2024 13:04:12 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 00DED416C8 Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=h32zhzVq X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id plop5i3NzWS1 for ; Tue, 23 Jan 2024 13:04:11 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 26CEF415E0 for ; Tue, 23 Jan 2024 13:04:11 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 26CEF415E0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706015050; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=unqEMzN8aeQIsobIUPCO97zLvVGQTt2kujwnnmG4UN8=; b=h32zhzVqBpFcZASheNwI83aAmLySvFpu1ZgXt98ceFG80N2dNWQXj91ERA0MJOztmIea2d f0+UkDnSYjMP8AybOX1sTqts3YR9gMJWOkzWB/DJgKGujaLaikPN/3csLZmEPWISFhptUf bsqDr1hS3eTcWV4uZTkViJd9Lh7yhIw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-176-jtouG34kP-iDoG4ldFfTSA-1; Tue, 23 Jan 2024 08:04:08 -0500 X-MC-Unique: jtouG34kP-iDoG4ldFfTSA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 57E95185A780 for ; Tue, 23 Jan 2024 13:04:08 +0000 (UTC) Received: from mheiblap.localdomain.com (unknown [10.47.238.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0C0A02026D66; Tue, 23 Jan 2024 13:03:58 +0000 (UTC) From: Mohammad Heib To: dev@openvswitch.org Date: Tue, 23 Jan 2024 15:03:28 +0200 Message-Id: <20240123130328.891918-1-mheib@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH ovn] Controller: Handle unconditional IDL messages while paused. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" If the user triggers a pause command to the ovn-controller the current implementation will wait for commands from unixctl server only and ignore the other component. This implementation works fine if we don't have inactivity_probe set in the SB DataBase, but once the user sets the inactivity_probe in SB DataBase the connection will be dropped by the SBDB. Once the controller resumes the execution it will try to commit some changes to the SBDB but the transaction will fail since we lost the connection to the SBDB and the controller must reconnect before committing the transaction again. To avoid the above scenario the controller can keep handling unconditional IDL messages to avoid reconnecting to SB. Signed-off-by: Mohammad Heib --- controller/ovn-controller.c | 16 +++++++++--- ovn-sb.xml | 2 +- tests/ovn-controller.at | 51 +++++++++++++++++++++++++++++++++++++ 3 files changed, 65 insertions(+), 4 deletions(-) diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c index 856e5e270..d2c8f66d9 100644 --- a/controller/ovn-controller.c +++ b/controller/ovn-controller.c @@ -5534,12 +5534,22 @@ main(int argc, char *argv[]) simap_destroy(&usage); } - /* If we're paused just run the unixctl server and skip most of the - * processing loop. + /* If we're paused just run the unixctl-server/unconditional IDL and + * skip most of the processing loop. */ if (paused) { unixctl_server_run(unixctl); + int ovnsb_seq = ovsdb_idl_get_seqno(ovnsb_idl_loop.idl); + ovsdb_idl_run(ovnsb_idl_loop.idl); + int new_ovnsb_seq = ovsdb_idl_get_seqno(ovnsb_idl_loop.idl); + /* If the IDL content has changed while the controller is + * in pause state, trigger a recompute. + */ + if (new_ovnsb_seq != ovnsb_seq) { + engine_set_force_recompute(true); + } unixctl_server_wait(unixctl); + ovsdb_idl_wait(ovnsb_idl_loop.idl); goto loop_done; } @@ -6009,7 +6019,6 @@ main(int argc, char *argv[]) OVS_NOT_REACHED(); } - ovsdb_idl_track_clear(ovnsb_idl_loop.idl); ovsdb_idl_track_clear(ovs_idl_loop.idl); lflow_cache_run(ctrl_engine_ctx.lflow_cache); @@ -6017,6 +6026,7 @@ main(int argc, char *argv[]) loop_done: memory_wait(); + ovsdb_idl_track_clear(ovnsb_idl_loop.idl); poll_block(); if (should_service_stop()) { exit_args.exiting = true; diff --git a/ovn-sb.xml b/ovn-sb.xml index e393f92b3..43c13f23c 100644 --- a/ovn-sb.xml +++ b/ovn-sb.xml @@ -4308,7 +4308,7 @@ tcp.flags = RST; Maximum number of milliseconds of idle time on connection to the client before sending an inactivity probe message. If Open vSwitch does not - communicate with the client for the specified number of seconds, it + communicate with the client for the specified number of milliseconds,it will send a probe. If a response is not received for the same additional amount of time, Open vSwitch assumes the connection has been broken and attempts to reconnect. Default is implementation-specific. diff --git a/tests/ovn-controller.at b/tests/ovn-controller.at index 9d2a37c72..04e4c52e7 100644 --- a/tests/ovn-controller.at +++ b/tests/ovn-controller.at @@ -352,6 +352,57 @@ OVS_APP_EXIT_AND_WAIT([ovsdb-server]) AT_CLEANUP ]) +# Check that the connection to the Southbound database +# is not dropped when probe-interval is set and the controller +# is in pause state. +OVN_FOR_EACH_NORTHD([ +AT_SETUP([ovn-controller - check sbdb connection while pause]) +AT_KEYWORDS([ovn]) +ovn_start + +net_add n1 +sim_add hv +as hv +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.1 + +ovs-vsctl -- add-port br-int hv1-vif1 -- \ + set interface hv1-vif1 external-ids:iface-id=sw0-p1 \ + options:tx_pcap=hv1/vif1-tx.pcap \ + options:rxq_pcap=hv1/vif1-rx.pcap \ + ofport-request=1 + +ovs-vsctl set open . external_ids:ovn-remote-probe-interval=100000 +ovn-sbctl set connection . inactivity_probe=1 + +ovn-nbctl ls-add sw0 +check ovn-nbctl lsp-add sw0 sw0-p1 +check ovn-nbctl lsp-set-addresses sw0-p1 "50:54:00:00:00:03 10.0.0.3" +check ovn-nbctl lsp-set-port-security sw0-p1 "50:54:00:00:00:03 10.0.0.3" +ovn-nbctl --wait=hv sync + +sleep_controller hv +# Trigger DB change to make SBDB connect to controller. +check ovn-nbctl lsp-del sw0-p1 + +# wait for 2 sec to give enough time to the SBDB to drop the connection +# if there is no answer from the controller. The connection should not +# be dropped since we keep handle the idl messages from SBDB even if we +# in pause state. +sleep 2 +wake_up_controller hv + +check ovn-nbctl lsp-add sw0 sw0-p1 +check ovn-nbctl lsp-set-addresses sw0-p1 "50:54:00:00:00:03 10.0.0.3" +check ovn-nbctl lsp-set-port-security sw0-p1 "50:54:00:00:00:03 10.0.0.3" +ovn-nbctl --wait=hv sync + +OVS_WAIT_UNTIL([test 0 = `cat hv/ovn-controller.log| grep -c "OVNSB commit failed, force recompute next time"`]) +OVS_WAIT_UNTIL([test 0 = `cat hv/ovn-controller.log| grep -c "connection closed by peer"`]) +OVN_CLEANUP([hv]) +AT_CLEANUP +]) + # Checks that ovn-controller recreates its chassis record when deleted externally. OVN_FOR_EACH_NORTHD([ AT_SETUP([ovn-controller - Chassis self record])