From patchwork Thu Aug 5 14:50:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frode Nordahl X-Patchwork-Id: 1513947 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=jV193d42; dkim-atps=neutral Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GgWmg2sgcz9s5R for ; Fri, 6 Aug 2021 00:52:35 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 3FBD883AEE; Thu, 5 Aug 2021 14:52:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BisRKRal2Trf; Thu, 5 Aug 2021 14:52:31 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id ED63783B0E; Thu, 5 Aug 2021 14:52:29 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 77800C0033; Thu, 5 Aug 2021 14:52:25 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id EF91CC001F for ; Thu, 5 Aug 2021 14:52:22 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id A3D6E40510 for ; Thu, 5 Aug 2021 14:52:22 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p_cvocCtLOpj for ; Thu, 5 Aug 2021 14:52:21 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com [IPv6:2a00:1450:4864:20::132]) by smtp4.osuosl.org (Postfix) with ESMTPS id 1CFD9404EC for ; Thu, 5 Aug 2021 14:52:20 +0000 (UTC) Received: by mail-lf1-x132.google.com with SMTP id bq29so11678525lfb.5 for ; Thu, 05 Aug 2021 07:52:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CupI8Ib+JdlN7IAIY7XCLffNDzkvCN3tGT6tWtTnc68=; b=jV193d421cJO69LrssLa2zQ7DQtb6MIrGsJGip4q+9Stu1WBuEvYYBANFwxpJuPGfQ JMrUdvMoPPTsiosf9kxTWLyhu+tcRn9ELP8awW4IgGI/+cxSHgdeq2XCFJdVal/Ex3Kn 68aUV+pCk/Eh+iaBFqyMIPQHqJhXEh+6o4ULvIWHSv+TC4E64UAz1nz8uaIEhBBfdAdo urjV+wfSHakWs+hwicyBHUS7I6qv6tLUgZ69YnDsTCDhw2HxGH7tBTMepLp+DzYIaycu mfGY5hL3vPEhPpMO4yiqDKI9gp2ih7/e2LNj7mxfVp8cka6RIqJvc2gK47jxVVXklYGg vEXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CupI8Ib+JdlN7IAIY7XCLffNDzkvCN3tGT6tWtTnc68=; b=jPh0Ze790obDC37rp09M1BZbYqhVrEiP05+oUgReT/iHnlx7MS6yXmNul7yHA38SNL iognxnyyjx9odclkQ/m4gyVfiST7neb7+5a8FS9iuRAby3C2pWmSZPKYfF7NDabG3OF+ ffciD+fOo46mxjYRUKDOjo9Mobq64YqdF1nB5dkesNFofWQOe61qpBH/Cw40tjRaJ2PR OFGscSkQG7LGJJeU8p6sQgHrhEe3Aw8bZLCG0YmhZEG2ufLoQ46hKmC5+1XHloutie4Z vYHi/qgxOnz4ucHrcKgrBrO9QFyT1yw9dfzCuZ9UJKuJilhuiYnXIqz8p6EfT6mgsCFG +jVQ== X-Gm-Message-State: AOAM530CTPv+IDRMMpNz4RZbUZAc4GeziLyqR/AYgf56d/LkPgHgiPM9 EI8b5NChhLYNLgxe2fiAZ7HLiO1OTO0= X-Google-Smtp-Source: ABdhPJwMUkxtKmAkisCadc/YXE0QlW75ripCkSIaAlSdvBj1uox7LnzePCSVLBfvHy1VWRQ9NRbIMg== X-Received: by 2002:ac2:58e1:: with SMTP id v1mr4051582lfo.52.1628175138906; Thu, 05 Aug 2021 07:52:18 -0700 (PDT) Received: from ti0189a330-1161.bb.online.no (ti0189a330-1161.bb.online.no. [88.88.219.141]) by smtp.gmail.com with ESMTPSA id p14sm536632lfa.117.2021.08.05.07.52.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Aug 2021 07:52:18 -0700 (PDT) From: Frode Nordahl To: dev@openvswitch.org Date: Thu, 5 Aug 2021 16:50:13 +0200 Message-Id: <20210805145013.3033919-7-frode.nordahl@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210805145013.3033919-1-frode.nordahl@gmail.com> References: <20210805145013.3033919-1-frode.nordahl@gmail.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org Subject: [ovs-dev] [RFC PATCH ovn 6/6] binding: Consider plugging of ports on CMS request X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" When OVN is linked with an appropriate plugging implementation, CMS can request OVN to plug individual lports into the local Open vSwitch instance. The port and instance record will be maintained during the lifetime of the lport and it will be removed on release of lport. TODO: The functions considering plugging or unplugging of interfaces are currently oblivious to the fact that the binding module has data structures to keep track of these. The functions should be good citizens and learn to update those appropriately as they take action. Signed-off-by: Frode Nordahl --- controller/binding.c | 175 ++++++++++++++++++++++++++++++++++++ controller/binding.h | 1 + controller/ovn-controller.c | 5 ++ 3 files changed, 181 insertions(+) diff --git a/controller/binding.c b/controller/binding.c index 2a10d7586..db1132095 100644 --- a/controller/binding.c +++ b/controller/binding.c @@ -35,7 +35,9 @@ #include "local_data.h" #include "lport.h" #include "ovn-controller.h" +#include "ovsport.h" #include "patch.h" +#include "plug.h" VLOG_DEFINE_THIS_MODULE(binding); @@ -45,6 +47,8 @@ VLOG_DEFINE_THIS_MODULE(binding); */ #define OVN_INSTALLED_EXT_ID "ovn-installed" +#define OVN_PLUGGED_EXT_ID "ovn-plugged" + #define OVN_QOS_TYPE "linux-htb" struct qos_queue { @@ -71,10 +75,13 @@ binding_register_ovs_idl(struct ovsdb_idl *ovs_idl) ovsdb_idl_add_table(ovs_idl, &ovsrec_table_interface); ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_name); + ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_type); ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_external_ids); ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_bfd); ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_bfd_status); ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_status); + ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_options); + ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_mtu_request); ovsdb_idl_add_table(ovs_idl, &ovsrec_table_qos); ovsdb_idl_add_column(ovs_idl, &ovsrec_qos_col_type); @@ -1046,6 +1053,18 @@ is_binding_lport_this_chassis(struct binding_lport *b_lport, b_lport->pb->chassis == chassis); } +static bool +chassis_uses_dpdk(const struct ovsrec_open_vswitch_table *ovs_table, + const struct ovsrec_bridge *br_int) +{ + const struct ovsrec_open_vswitch *cfg; + + cfg = ovsrec_open_vswitch_table_first(ovs_table); + + return (cfg && cfg->dpdk_initialized && + !strcmp(br_int->datapath_type, "netdev")); +} + static bool can_bind_on_this_chassis(const struct sbrec_chassis *chassis_rec, const char *requested_chassis) @@ -1055,6 +1074,14 @@ can_bind_on_this_chassis(const struct sbrec_chassis *chassis_rec, || !strcmp(requested_chassis, chassis_rec->hostname); } +static bool +can_plug_on_this_chassis(const struct sbrec_chassis *chassis_rec, + const struct sbrec_port_binding *pb) +{ + return pb->plugged_by && uuid_equals(&chassis_rec->header_.uuid, + &pb->plugged_by->header_.uuid); +} + /* Returns 'true' if the 'lbinding' has binding lports of type LP_CONTAINER, * 'false' otherwise. */ static bool @@ -1088,6 +1115,48 @@ release_binding_lport(const struct sbrec_chassis *chassis_rec, return true; } +static void +consider_unplug_lport(const struct sbrec_port_binding *pb, + struct binding_ctx_in *b_ctx_in, + struct binding_lport *b_lport) +{ + const char *plug_type = smap_get(&b_lport->lbinding->iface->external_ids, + OVN_PLUGGED_EXT_ID); + if (plug_type) + { + const struct ovsrec_port *port = ovsport_lookup_by_interface( + b_ctx_in->ovsrec_port_by_interfaces, + (struct ovsrec_interface *) b_lport->lbinding->iface); + if (port) { + struct plug *plug; + if (plug_open(plug_type, &plug)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, + "Unable to open plug provider for " + "plug-type: '%s' lport %s", + plug_type, pb->logical_port); + return; + } + const struct plug_port_ctx_in plug_ctx_in = { + .op_type = PLUG_OP_REMOVE, + .use_dpdk = chassis_uses_dpdk(b_ctx_in->ovs_table, + b_ctx_in->br_int), + .lport_name = (const char *)pb->logical_port, + .lport_options = (const struct smap *)&pb->options, + .iface_name = b_lport->lbinding->iface->name, + .iface_type = b_lport->lbinding->iface->type, + .iface_options = &b_lport->lbinding->iface->options, + }; + plug_port_prepare(plug, &plug_ctx_in, NULL); + VLOG_INFO("Unplugging port %s from %s for lport %s on this " + "chassis.", + port->name, b_ctx_in->br_int->name, pb->logical_port); + ovs_remove_port(b_ctx_in->br_int, port); + plug_port_finish(plug, &plug_ctx_in, NULL); + } + } +} + static bool consider_vif_lport_(const struct sbrec_port_binding *pb, bool can_bind, const char *vif_chassis, @@ -1138,6 +1207,7 @@ consider_vif_lport_(const struct sbrec_port_binding *pb, if (pb->chassis == b_ctx_in->chassis_rec) { /* Release the lport if there is no lbinding. */ if (!lbinding_set || !can_bind) { + consider_unplug_lport(pb, b_ctx_in, b_lport); return release_lport(pb, !b_ctx_in->ovnsb_idl_txn, b_ctx_out->tracked_dp_bindings, b_ctx_out->if_mgr); @@ -1147,6 +1217,106 @@ consider_vif_lport_(const struct sbrec_port_binding *pb, return true; } +static int64_t +get_plug_mtu_request(const struct smap *lport_options) +{ + return smap_get_int(lport_options, "plug-mtu-request", 0); +} + +static bool +consider_plug_lport(const struct sbrec_port_binding *pb, + struct binding_ctx_in *b_ctx_in, + struct local_binding *lbinding) +{ + bool ret = true; + + if (can_plug_on_this_chassis(b_ctx_in->chassis_rec, pb)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + const char *plug_type = smap_get(&pb->options, "plug-type"); + if (!plug_type) { + /* This should never happen, but better safe than sorry */ + VLOG_WARN_RL(&rl, + "Possible database inconsistency detected: " + "Port_Binding->plugged_by points at this chassis, " + "but Port_Binding->options:plug-type not set. " + "lport %s", + pb->logical_port); + return false; + } + + struct plug *plug; + if (plug_open(plug_type, &plug)) { + VLOG_WARN_RL(&rl, + "Unable to open plug provider for plug-type: '%s' " + "lport %s", + plug_type, pb->logical_port); + return false; + } + struct plug_port_ctx_in plug_ctx_in = { + .op_type = PLUG_OP_CREATE, + .use_dpdk = chassis_uses_dpdk(b_ctx_in->ovs_table, + b_ctx_in->br_int), + .lport_name = (const char *)pb->logical_port, + .lport_options = (const struct smap *)&pb->options, + }; + struct plug_port_ctx_out plug_ctx_out; + const struct smap iface_external_ids = SMAP_CONST2( + &iface_external_ids, + OVN_PLUGGED_EXT_ID, plug_type, + "iface-id", pb->logical_port); + if (!plug_port_prepare(plug, &plug_ctx_in, &plug_ctx_out)) { + VLOG_INFO_RL(&rl, + "Not plugging lport %s on direction from plugging " + "library.", + pb->logical_port); + ret = false; + goto out; + } + + if (lbinding) { + if (smap_get(&lbinding->iface->external_ids, + OVN_PLUGGED_EXT_ID)) + { + if (strcmp(lbinding->iface->name, plug_ctx_out.name)) { + VLOG_WARN("Attempt of incompatible change to existing " + "port detected, please recreate port: %s", + pb->logical_port); + ret = false; + goto out; + } + VLOG_DBG("updating iface for: %s", pb->logical_port); + ovs_update_iface(lbinding->iface, plug_ctx_out.type, + &iface_external_ids, + NULL, + plug_ctx_out.iface_options, + plug_class_get_maintained_iface_options(plug), + get_plug_mtu_request(&pb->options)); + } else { + VLOG_WARN_RL(&rl, + "CMS requested plugging but port not maintained " + "by OVN already exsists in local vSwitch: %s", + pb->logical_port); + } + } else { + VLOG_INFO("Plugging port %s into %s for lport %s on this " + "chassis.", + plug_ctx_out.name, b_ctx_in->br_int->name, + pb->logical_port); + VLOG_DBG("creating port for: %s", pb->logical_port); + ovs_create_port(b_ctx_in->ovs_idl_txn, b_ctx_in->br_int, + plug_ctx_out.name, plug_ctx_out.type, + NULL, &iface_external_ids, + plug_ctx_out.iface_options, + get_plug_mtu_request(&pb->options)); + } +out: + plug_port_finish(plug, &plug_ctx_in, &plug_ctx_out); + plug_port_ctx_destroy(plug, &plug_ctx_in, &plug_ctx_out); + } + + return ret; +} + static bool consider_vif_lport(const struct sbrec_port_binding *pb, struct binding_ctx_in *b_ctx_in, @@ -1170,6 +1340,7 @@ consider_vif_lport(const struct sbrec_port_binding *pb, b_lport = local_binding_add_lport(binding_lports, lbinding, pb, LP_VIF); } + consider_plug_lport(pb, b_ctx_in, lbinding); return consider_vif_lport_(pb, can_bind, vif_chassis, b_ctx_in, b_ctx_out, b_lport, qos_map); } @@ -1563,6 +1734,10 @@ binding_run(struct binding_ctx_in *b_ctx_in, struct binding_ctx_out *b_ctx_out) const struct sbrec_port_binding *pb; }; + /* For any open plug provider instances, perform periodic non-blocking + * maintenance */ + plug_run_instances(); + /* Run through each binding record to see if it is resident on this * chassis and update the binding accordingly. This includes both * directly connected logical ports and children of those ports diff --git a/controller/binding.h b/controller/binding.h index f1abc4b9c..8f4521806 100644 --- a/controller/binding.h +++ b/controller/binding.h @@ -46,6 +46,7 @@ struct binding_ctx_in { struct ovsdb_idl_index *sbrec_datapath_binding_by_key; struct ovsdb_idl_index *sbrec_port_binding_by_datapath; struct ovsdb_idl_index *sbrec_port_binding_by_name; + struct ovsdb_idl_index *ovsrec_port_by_interfaces; const struct ovsrec_port_table *port_table; const struct ovsrec_qos_table *qos_table; const struct sbrec_port_binding_table *port_binding_table; diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c index cf051f0be..98d9bf61d 100644 --- a/controller/ovn-controller.c +++ b/controller/ovn-controller.c @@ -55,6 +55,7 @@ #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "patch.h" +#include "plug.h" #include "physical.h" #include "pinctrl.h" #include "openvswitch/poll-loop.h" @@ -231,6 +232,9 @@ update_sb_monitors(struct ovsdb_idl *ovnsb_idl, sbrec_port_binding_add_clause_chassis(&pb, OVSDB_F_EQ, &chassis->header_.uuid); + sbrec_port_binding_add_clause_plugged_by(&pb, OVSDB_F_EQ, + &chassis->header_.uuid); + /* Ensure that we find out about l2gateway and l3gateway ports that * should be present on this chassis. Otherwise, we might never find * out about those ports, if their datapaths don't otherwise have a VIF @@ -3881,6 +3885,7 @@ loop_done: pinctrl_destroy(); patch_destroy(); if_status_mgr_destroy(if_mgr); + plug_destroy_all(); ovsdb_idl_loop_destroy(&ovs_idl_loop); ovsdb_idl_loop_destroy(&ovnsb_idl_loop);