From patchwork Wed Jun 7 17:28:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xavier Simonart X-Patchwork-Id: 1791873 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=iIATIqkt; dkim-atps=neutral Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QbvTD2xZcz20WP for ; Thu, 8 Jun 2023 03:28:48 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 45E1282F87; Wed, 7 Jun 2023 17:28:45 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 45E1282F87 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=iIATIqkt X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PmFTHa8v6E_n; Wed, 7 Jun 2023 17:28:43 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 0F58481420; Wed, 7 Jun 2023 17:28:42 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 0F58481420 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E1F77C008E; Wed, 7 Jun 2023 17:28:39 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id EEDA6C0029 for ; Wed, 7 Jun 2023 17:28:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id BBA35408D7 for ; Wed, 7 Jun 2023 17:28:38 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org BBA35408D7 Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=iIATIqkt X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xBbo9_botebL for ; Wed, 7 Jun 2023 17:28:37 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 51CE6408CF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 51CE6408CF for ; Wed, 7 Jun 2023 17:28:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686158916; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=2wBMhIq8NfoREgf5/y4NZ7nZssLJwea6yJ8+loVtHkE=; b=iIATIqktMatCtu2hZlckZ4Qll71KsrSq77c3ynNDs317YmdWTca+eWIcnpw533CCt6M5MO hicrxWDBfTtI5F+sRFJ4JF2r2EpgbpubzeOrvcg49Q/FAqWl7cZ+zL5xb0ILzcbSktRMC6 Mpaw49mByxay/wsl9otbo5Ldb/LzB/g= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-133-XAdkMNf-MzK3ldqOMtNFGA-1; Wed, 07 Jun 2023 13:28:34 -0400 X-MC-Unique: XAdkMNf-MzK3ldqOMtNFGA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8FBBF8007D9 for ; Wed, 7 Jun 2023 17:28:34 +0000 (UTC) Received: from wsfd-netdev90.ntdv.lab.eng.bos.redhat.com (wsfd-netdev90.ntdv.lab.eng.bos.redhat.com [10.19.188.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 675FE1121314; Wed, 7 Jun 2023 17:28:34 +0000 (UTC) From: Xavier Simonart To: xsimonar@redhat.com, dev@openvswitch.org Date: Wed, 7 Jun 2023 19:28:33 +0200 Message-Id: <20230607172834.3851733-1-xsimonar@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH ovn 1/2] qos: fix potential double deletion of ovs idl row X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" If an interface with an qos option is deleted at the same time as an ofport notification from ovs (causing runtime_data recompute) is received, the binding module was trying to delete twice the same qos queue, causing ovs to raise an exception. Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=2213219 Fixes: 7d1d111ff213 ("controller: configure qos through ovs qos table and do not run tc directly") Signed-off-by: Xavier Simonart Reviewed-by: Lorenzo Bianconi --- controller/binding.c | 22 ++++++++++++ controller/binding.h | 1 + controller/ovn-controller.c | 12 +++++++ tests/ovn-macros.at | 34 ++++++++++++++++++ tests/ovn.at | 71 +++++++++++++++++++++++++++++++++++++ tests/system-ovn.at | 18 ---------- 6 files changed, 140 insertions(+), 18 deletions(-) diff --git a/controller/binding.c b/controller/binding.c index 9b0647b70..f2896a9c9 100644 --- a/controller/binding.c +++ b/controller/binding.c @@ -396,9 +396,23 @@ configure_qos(const struct sbrec_port_binding *pb, q->burst = burst; } +static const struct ovsrec_queue * +find_qos_queue_by_external_ids(const struct smap *external_ids, + struct ovsdb_idl_index *ovsrec_queue_by_external_ids) +{ + const struct ovsrec_queue *queue = + ovsrec_queue_index_init_row(ovsrec_queue_by_external_ids); + ovsrec_queue_index_set_external_ids(queue, external_ids); + const struct ovsrec_queue *retval = + ovsrec_queue_index_find(ovsrec_queue_by_external_ids, queue); + ovsrec_queue_index_destroy_row(queue); + return retval; +} + static void ovs_qos_entries_gc(struct ovsdb_idl_txn *ovs_idl_txn, struct ovsdb_idl_index *ovsrec_port_by_qos, + struct ovsdb_idl_index *ovsrec_queue_by_external_ids, const struct ovsrec_qos_table *qos_table, struct hmap *queue_map) { @@ -414,6 +428,13 @@ ovs_qos_entries_gc(struct ovsdb_idl_txn *ovs_idl_txn, if (!queue) { continue; } + const struct ovsrec_queue *ovsrec_queue = + find_qos_queue_by_external_ids(&queue->external_ids, + ovsrec_queue_by_external_ids); + if (!ovsrec_queue) { + VLOG_DBG("queue already deleted !"); + continue; + } const char *port = smap_get(&queue->external_ids, "ovn_port"); if (!port) { @@ -2126,6 +2147,7 @@ binding_run(struct binding_ctx_in *b_ctx_in, struct binding_ctx_out *b_ctx_out) shash_destroy(&bridge_mappings); /* Remove stale QoS entries. */ ovs_qos_entries_gc(b_ctx_in->ovs_idl_txn, b_ctx_in->ovsrec_port_by_qos, + b_ctx_in->ovsrec_queue_by_external_ids, b_ctx_in->qos_table, b_ctx_out->qos_map); cleanup_claimed_port_timestamps(); diff --git a/controller/binding.h b/controller/binding.h index 0e57f02ee..e3ab1d7ca 100644 --- a/controller/binding.h +++ b/controller/binding.h @@ -47,6 +47,7 @@ struct binding_ctx_in { struct ovsdb_idl_index *sbrec_port_binding_by_datapath; struct ovsdb_idl_index *sbrec_port_binding_by_name; struct ovsdb_idl_index *ovsrec_port_by_qos; + struct ovsdb_idl_index *ovsrec_queue_by_external_ids; const struct ovsrec_qos_table *qos_table; const struct sbrec_port_binding_table *port_binding_table; const struct ovsrec_bridge *br_int; diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c index 3a81a13fb..159fcd1df 100644 --- a/controller/ovn-controller.c +++ b/controller/ovn-controller.c @@ -1116,6 +1116,7 @@ enum sb_engine_node { OVS_NODE(port, "port") \ OVS_NODE(interface, "interface") \ OVS_NODE(qos, "qos") \ + OVS_NODE(queue, "queue") \ OVS_NODE(flow_sample_collector_set, "flow_sample_collector_set") enum ovs_engine_node { @@ -1576,6 +1577,10 @@ init_binding_ctx(struct engine_node *node, engine_ovsdb_node_get_index( engine_get_input("OVS_port", node), "qos"); + struct ovsdb_idl_index *ovsrec_queue_by_external_ids = + engine_ovsdb_node_get_index( + engine_get_input("OVS_queue", node), "external_ids"); + struct controller_engine_ctx *ctrl_ctx = engine_get_context()->client_ctx; b_ctx_in->ovnsb_idl_txn = engine_get_context()->ovnsb_idl_txn; @@ -1584,6 +1589,7 @@ init_binding_ctx(struct engine_node *node, b_ctx_in->sbrec_port_binding_by_datapath = sbrec_port_binding_by_datapath; b_ctx_in->sbrec_port_binding_by_name = sbrec_port_binding_by_name; b_ctx_in->ovsrec_port_by_qos = ovsrec_port_by_qos; + b_ctx_in->ovsrec_queue_by_external_ids = ovsrec_queue_by_external_ids; b_ctx_in->iface_table = iface_shadow->iface_table; b_ctx_in->iface_table_external_ids_old = &iface_shadow->iface_table_external_ids_old; @@ -4599,6 +4605,9 @@ main(int argc, char *argv[]) struct ovsdb_idl_index *ovsrec_port_by_qos = ovsdb_idl_index_create1(ovs_idl_loop.idl, &ovsrec_port_col_qos); + struct ovsdb_idl_index *ovsrec_queue_by_external_ids + = ovsdb_idl_index_create1(ovs_idl_loop.idl, + &ovsrec_queue_col_external_ids); struct ovsdb_idl_index *ovsrec_flow_sample_collector_set_by_id = ovsdb_idl_index_create2(ovs_idl_loop.idl, &ovsrec_flow_sample_collector_set_col_bridge, @@ -4899,6 +4908,7 @@ main(int argc, char *argv[]) engine_add_input(&en_runtime_data, &en_ovs_open_vswitch, NULL); engine_add_input(&en_runtime_data, &en_ovs_bridge, NULL); engine_add_input(&en_runtime_data, &en_ovs_qos, NULL); + engine_add_input(&en_runtime_data, &en_ovs_queue, NULL); engine_add_input(&en_runtime_data, &en_sb_chassis, NULL); engine_add_input(&en_runtime_data, &en_sb_datapath_binding, @@ -4960,6 +4970,8 @@ main(int argc, char *argv[]) engine_ovsdb_node_add_index(&en_ovs_flow_sample_collector_set, "id", ovsrec_flow_sample_collector_set_by_id); engine_ovsdb_node_add_index(&en_ovs_port, "qos", ovsrec_port_by_qos); + engine_ovsdb_node_add_index(&en_ovs_queue, "external_ids", + ovsrec_queue_by_external_ids); struct ed_type_lflow_output *lflow_output_data = engine_get_internal_data(&en_lflow_output); diff --git a/tests/ovn-macros.at b/tests/ovn-macros.at index 6f2d085ae..7223846ef 100644 --- a/tests/ovn-macros.at +++ b/tests/ovn-macros.at @@ -840,6 +840,40 @@ fmt_pkt() { print(out.decode())" | $PYTHON3 } +sleep_sb() { + echo SB going to sleep + AT_CHECK([kill -STOP $(cat ovn-sb/ovsdb-server.pid)]) +} +wake_up_sb() { + echo SB waking up + AT_CHECK([kill -CONT $(cat ovn-sb/ovsdb-server.pid)]) +} +sleep_controller() { + echo Controller $hv going to sleep + hv=$1 + as $hv + check ovn-appctl debug/pause + OVS_WAIT_UNTIL([test x$(ovn-appctl -t ovn-controller debug/status) = "xpaused"]) +} +wake_up_controller() { + hv=$1 + as $hv + echo Controller $hv waking up + ovn-appctl debug/resume + OVS_WAIT_UNTIL([test x$(ovn-appctl -t ovn-controller debug/status) = "xrunning"]) +} +sleep_ovs() { + hv=$1 + echo ovs $hv going to sleep + AT_CHECK([kill -STOP $(cat $hv/ovs-vswitchd.pid)]) +} + +wake_up_ovs() { + hv=$1 + echo ovs $hv going to sleep + AT_CHECK([kill -CONT $(cat $hv/ovs-vswitchd.pid)]) +} + OVS_END_SHELL_HELPERS m4_define([OVN_POPULATE_ARP], [AT_CHECK(ovn_populate_arp__, [0], [ignore])]) diff --git a/tests/ovn.at b/tests/ovn.at index 5e6a8fefa..05bf7fa5b 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -36129,3 +36129,74 @@ OVS_WAIT_UNTIL([test $(as hv-1 ovs-vsctl list queue | grep -c 'burst="8000000000 AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([OVN QoS port deletion]) +ovn_start + +check ovn-nbctl ls-add ls1 +check ovn-nbctl lsp-add ls1 public1 +check ovn-nbctl lsp-set-addresses public1 unknown +check ovn-nbctl lsp-set-type public1 localnet +check ovn-nbctl lsp-set-options public1 network_name=phys +net_add n + +# two hypervisors, each connected to the same network +for i in 1 2; do + sim_add hv-$i + as hv-$i + ovs-vsctl add-br br-phys + ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys + ovn_attach n br-phys 192.168.0.$i +done + +check ovn-nbctl lsp-add ls1 lsp1 +check ovn-nbctl lsp-set-addresses lsp1 f0:00:00:00:00:03 +as hv-1 +ovs-vsctl add-port br-int vif1 -- \ + set Interface vif1 external-ids:iface-id=lsp1 \ + ofport-request=3 + +OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up lsp1` = xup]) + +check ovn-nbctl set Logical_Switch_Port lsp1 options:qos_max_rate=800000 +check ovn-nbctl --wait=hv set Logical_Switch_Port lsp1 options:qos_burst=9000000 + +AS_BOX([$(date +%H:%M:%S.%03N) checking deletion of port with qos options]) +check ovn-nbctl ls-add ls2 +check ovn-nbctl lsp-add ls2 lsp2 +check ovn-nbctl lsp-set-addresses lsp2 f0:00:00:00:00:05 +as hv-1 +ovs-vsctl add-port br-int vif2 -- \ + set Interface vif2 external-ids:iface-id=lsp2 \ + ofport-request=5 +OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up lsp2` = xup]) + +# Sleep ovs to postpone ofport notification to ovn +sleep_ovs hv-1 + +# Create localnet; this will cause patch-port creation +check ovn-nbctl lsp-add ls2 public2 +check ovn-nbctl lsp-set-addresses public2 unknown +check ovn-nbctl lsp-set-type public2 localnet +check ovn-nbctl --wait=sb set Logical_Switch_Port public2 options:qos_min_rate=6000000000 options:qos_max_rate=7000000000 options:qos_burst=8000000000 options:network_name=phys + +# Let's now send ovn controller to sleep, so it will receive both ofport notification and ls deletion simultaneously +sleep_controller hv-1 + +# Tme to wake up ovs +wake_up_ovs hv-1 + +# Delete lsp1 +check ovn-nbctl --wait=sb lsp-del lsp1 + +# And finally wake up controller +wake_up_controller hv-1 + +# Make sure ovn-controller is still OK +ovn-nbctl --wait=hv sync +OVS_WAIT_UNTIL([test $(as hv-1 ovs-vsctl list qos | grep -c linux-htb) -eq 1]) + +AT_CLEANUP +]) + diff --git a/tests/system-ovn.at b/tests/system-ovn.at index 6669c18e7..d8e8ab342 100644 --- a/tests/system-ovn.at +++ b/tests/system-ovn.at @@ -10897,20 +10897,6 @@ wait_for_local_bindings() { [kill -CONT $(cat ovn-sb/ovsdb-server.pid)] ) } -sleep_sb() { - echo SB going to sleep - AT_CHECK([kill -STOP $(cat ovn-sb/ovsdb-server.pid)]) -} -wake_up_sb() { - echo SB waking up - AT_CHECK([kill -CONT $(cat ovn-sb/ovsdb-server.pid)]) -} -sleep_controller() { - echo Controller going to sleep - ovn-appctl debug/pause - OVS_WAIT_UNTIL([test x$(ovn-appctl -t ovn-controller debug/status) = "xpaused"]) -} - stop_ovsdb_controller_updates() { TCP_PORT=$1 echo Stopping updates from ovn-controller to ovsdb using port $TCP_PORT @@ -10922,10 +10908,6 @@ restart_ovsdb_controller_updates() { echo Restarting updates from ovn-controller to ovsdb iptables -D INPUT -p tcp --destination-port $TCP_PORT -j DROP } -wake_up_controller() { - echo Controller waking up - ovn-appctl debug/resume -} ensure_controller_run() { # We want to make sure controller could run at least one full loop. # We can't use wait=hv as sb might be sleeping.