From patchwork Mon Aug 31 12:14:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dumitru Ceara X-Patchwork-Id: 1354318 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=PxWVACgK; dkim-atps=neutral Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Bg8KZ2MNCz9sTK for ; Mon, 31 Aug 2020 22:15:12 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 42814207A4; Mon, 31 Aug 2020 12:15:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pqrYQznbepKA; Mon, 31 Aug 2020 12:15:06 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id 85490204B3; Mon, 31 Aug 2020 12:15:06 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 65766C07FF; Mon, 31 Aug 2020 12:15:06 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id D9B88C0051 for ; Mon, 31 Aug 2020 12:15:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id D2A5320515 for ; Mon, 31 Aug 2020 12:15:05 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8IwFoJ2zJ-cj for ; Mon, 31 Aug 2020 12:15:02 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by silver.osuosl.org (Postfix) with ESMTPS id 504E0204B3 for ; Mon, 31 Aug 2020 12:15:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598876101; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type; bh=mIly+Dp9DsuCQGqRdcni8GZUJwvc3GVk9TLl3rFaqmA=; b=PxWVACgKMm7Jr5Hp3DKlDmjTm9CC5nM4uvM8Qt6OI2mL+6keUmFJnjehoCD9UexeAxV7h/ IyqLvzFfz/wg9/OxlDWLYWYiQq+b+GanbJBg1niGJNcdNru0hqnfL8CCe/O/SfDK1ta/7j g7snQcaZMN9ZLFmBRLYWxUqxpJ8TYHY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-406-dOBC3s3tOza3-aaWYaKtcw-1; Mon, 31 Aug 2020 08:14:59 -0400 X-MC-Unique: dOBC3s3tOza3-aaWYaKtcw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2B67F801AEC; Mon, 31 Aug 2020 12:14:58 +0000 (UTC) Received: from dceara.remote.csb (ovpn-112-220.ams2.redhat.com [10.36.112.220]) by smtp.corp.redhat.com (Postfix) with ESMTP id 67F707AED5; Mon, 31 Aug 2020 12:14:57 +0000 (UTC) From: Dumitru Ceara To: dev@openvswitch.org Date: Mon, 31 Aug 2020 14:14:52 +0200 Message-Id: <1598876092-7512-1-git-send-email-dceara@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dceara@redhat.com X-Mimecast-Spam-Score: 0.002 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH ovn v2] ovn-controller: Fix incremental processing of Port_Binding deletes. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" If a Port_Binding is deleted from the Southbound DB and the corresponding OVS interface is also deleted from the OVS DB, and if both updates are received and processed by ovn-controller in the same iteration, ovn-controller should process port_binding delete operations first. This commit also adds three new unixctl debug commands for ovn-controller: - debug/pause: pause ovn-controller processing, except unixctl commands. - debug/resume: resume ovn-controller processing. - debug/status: return the status of the ovn-controller processing. These new commands are needed by the test for this scenario as without them we have no way of ensuring predictable results. Users should not use these commands in production. This is also why the commands are not documented. CC: Numan Siddique Fixes: 6b0f01116bab ("ovn-controller: Handle runtime data changes in flow output engine") Reported-by: Tim Rozet Reported-at: https://bugzilla.redhat.com/1871961 Signed-off-by: Dumitru Ceara --- v2: Rework the fix per Numan's suggestion and run the port_binding handler before the ovs iface handler. --- controller/ovn-controller.c | 71 ++++++++++++++++++++++++++++++++++++++++++--- tests/ovn.at | 38 ++++++++++++++++++++++++ 2 files changed, 105 insertions(+), 4 deletions(-) diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c index d299c61..874087c 100644 --- a/controller/ovn-controller.c +++ b/controller/ovn-controller.c @@ -73,6 +73,9 @@ static unixctl_cb_func extend_table_list; static unixctl_cb_func inject_pkt; static unixctl_cb_func engine_recompute_cmd; static unixctl_cb_func cluster_state_reset_cmd; +static unixctl_cb_func debug_pause_execution; +static unixctl_cb_func debug_resume_execution; +static unixctl_cb_func debug_status_execution; #define DEFAULT_BRIDGE_NAME "br-int" #define DEFAULT_PROBE_INTERVAL_MSEC 5000 @@ -2276,10 +2279,6 @@ main(int argc, char *argv[]) engine_add_input(&en_runtime_data, &en_ovs_open_vswitch, NULL); engine_add_input(&en_runtime_data, &en_ovs_bridge, NULL); - engine_add_input(&en_runtime_data, &en_ovs_port, - engine_noop_handler); - engine_add_input(&en_runtime_data, &en_ovs_interface, - runtime_data_ovs_interface_handler); engine_add_input(&en_runtime_data, &en_ovs_qos, NULL); engine_add_input(&en_runtime_data, &en_sb_chassis, NULL); @@ -2288,6 +2287,15 @@ main(int argc, char *argv[]) engine_add_input(&en_runtime_data, &en_sb_port_binding, runtime_data_sb_port_binding_handler); + /* The OVS interface handler for runtime_data changes MUST be executed + * after the sb_port_binding_handler as port_binding deletes must be + * processed first. + */ + engine_add_input(&en_runtime_data, &en_ovs_port, + engine_noop_handler); + engine_add_input(&en_runtime_data, &en_ovs_interface, + runtime_data_ovs_interface_handler); + struct engine_arg engine_arg = { .sb_idl = ovnsb_idl_loop.idl, .ovs_idl = ovs_idl_loop.idl, @@ -2342,6 +2350,14 @@ main(int argc, char *argv[]) cluster_state_reset_cmd, &reset_ovnsb_idl_min_index); + bool paused = false; + unixctl_command_register("debug/pause", "", 0, 0, debug_pause_execution, + &paused); + unixctl_command_register("debug/resume", "", 0, 0, debug_resume_execution, + &paused); + unixctl_command_register("debug/status", "", 0, 0, debug_status_execution, + &paused); + unsigned int ovs_cond_seqno = UINT_MAX; unsigned int ovnsb_cond_seqno = UINT_MAX; @@ -2350,6 +2366,15 @@ main(int argc, char *argv[]) restart = false; bool sb_monitor_all = false; while (!exiting) { + /* If we're paused just run the unixctl server and skip most of the + * processing loop. + */ + if (paused) { + unixctl_server_run(unixctl); + unixctl_server_wait(unixctl); + goto loop_done; + } + engine_init_run(); struct ovsdb_idl_txn *ovs_idl_txn = ovsdb_idl_loop_run(&ovs_idl_loop); @@ -2604,6 +2629,8 @@ main(int argc, char *argv[]) ovsdb_idl_track_clear(ovnsb_idl_loop.idl); ovsdb_idl_track_clear(ovs_idl_loop.idl); + +loop_done: poll_block(); if (should_service_stop()) { exiting = true; @@ -2857,3 +2884,39 @@ cluster_state_reset_cmd(struct unixctl_conn *conn, int argc OVS_UNUSED, poll_immediate_wake(); unixctl_command_reply(conn, NULL); } + +static void +debug_pause_execution(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *paused_) +{ + bool *paused = paused_; + + VLOG_INFO("User triggered execution pause."); + *paused = true; + unixctl_command_reply(conn, NULL); +} + +static void +debug_resume_execution(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *paused_) +{ + bool *paused = paused_; + + VLOG_INFO("User triggered execution resume."); + *paused = false; + poll_immediate_wake(); + unixctl_command_reply(conn, NULL); +} + +static void +debug_status_execution(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *paused_) +{ + bool *paused = paused_; + + if (*paused) { + unixctl_command_reply(conn, "paused"); + } else { + unixctl_command_reply(conn, "running"); + } +} diff --git a/tests/ovn.at b/tests/ovn.at index c4edbd9..5ad51c0 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -21371,3 +21371,41 @@ AT_CHECK([ovn-sbctl find mac ip=10.0.0.2 mac='"00:00:00:00:03:02"' logical_port= OVN_CLEANUP([hv1],[hv2]) AT_CLEANUP + +AT_SETUP([ovn -- Delete Port_Binding and OVS port Incremental Processing]) +ovn_start + +net_add n1 +sim_add hv1 +as hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.10 + +ovn-nbctl ls-add ls +ovn-nbctl lsp-add ls lsp + +as hv1 ovs-vsctl \ + -- add-port br-int vif1 \ + -- set Interface vif1 external_ids:iface-id=lsp + +# Wait for port to be bound. +OVS_WAIT_UNTIL([test $(ovn-sbctl --columns _uuid --bare list chassis hv1 | wc -l) -eq 1]) +ch=$(ovn-sbctl --columns _uuid --bare list chassis hv1) +OVS_WAIT_UNTIL([test $(ovn-sbctl --columns chassis --bare list port_binding lsp | grep $ch -c) -eq 1]) + +# Pause ovn-controller. +as hv1 ovn-appctl -t ovn-controller debug/pause + +# Delete port binding and OVS port. The updates will be processed in the same +# loop in ovn-controller when it resumes. +ovn-nbctl --wait=sb lsp-del lsp +as hv1 ovs-vsctl del-port vif1 + +# Resume ovn-controller. +as hv1 ovn-appctl -t ovn-controller debug/resume + +# Make sure ovn-controller runs fine. +OVS_WAIT_UNTIL([test x$(as hv1 ovn-appctl -t ovn-controller debug/status) = "xrunning"]) + +OVN_CLEANUP([hv1]) +AT_CLEANUP