From patchwork Fri Sep 3 19:27:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frode Nordahl X-Patchwork-Id: 1524522 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=jRlGssJu; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H1SWG6dS0z9sSs for ; Sat, 4 Sep 2021 05:28:10 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 907C383F31; Fri, 3 Sep 2021 19:28:08 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KJAFwjWHzvwx; Fri, 3 Sep 2021 19:28:04 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 9664E83F2D; Fri, 3 Sep 2021 19:28:02 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1F6C5C0033; Fri, 3 Sep 2021 19:27:59 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4FD48C0023 for ; Fri, 3 Sep 2021 19:27:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 7CCA64027D for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=canonical.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ukEJMf_QmyTu for ; Fri, 3 Sep 2021 19:27:52 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from smtp-relay-canonical-1.canonical.com (smtp-relay-canonical-1.canonical.com [185.125.188.121]) by smtp2.osuosl.org (Postfix) with ESMTPS id 7A569400AE for ; Fri, 3 Sep 2021 19:27:52 +0000 (UTC) Received: from frode-threadripper.. (1.general.frode.uk.vpn [10.172.193.250]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id 5950C40192; Fri, 3 Sep 2021 19:27:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1630697269; bh=qN738PGGkB8fpw63A/O3UUUFmgoFD06EtZii1QjR7TE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jRlGssJucpj+s0EHB16nZOMsqvdkxvCRhqsLoBKmsdmDFcbQiKsr1QYt45p/hiu03 PYZb9yaOQPavTlzywYqJVMR9msauZv/VtqTLsQxM/EjdKNJ4OEMgv0Mt9Hsm8Vxw9l 5bQjELEBPBHG3W7JE7dOfIKeimKIt1R1ftP7MU7hxqbZSjC03xX0qyr2yl1jEC/NW2 P5COWbnex42k/+OVSn2LKjzpWTH+RCwKDTGcTxnmqO2lu5qL/T496NvVrzs+nYokOH HA2v6G2BsjBSaHegEQWdKIz4JelUMrqBfR1OdLO2GYAKOUhqfmOuJTELic52TWKyeV pQH/1oPKkTJFw== From: Frode Nordahl To: dev@openvswitch.org Date: Fri, 3 Sep 2021 21:27:40 +0200 Message-Id: <20210903192748.1408062-2-frode.nordahl@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210903192748.1408062-1-frode.nordahl@canonical.com> References: <20210903192748.1408062-1-frode.nordahl@canonical.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v4 1/9] ovn-sb: Add requested_chassis column to Port_Binding. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" To allow for ovn-controller to efficiently process unbound ports that may be destined to it, for example for use in the optional plugging support, we add a new requested_chassis column with weakRef to the Chassis table. The ovn-controller can monitor this column and only process events for its chassis UUID even before a local binding appears. northd will fill this column with UUID of Chassis referenced in Logical_Switch_Port options:requested-chassis by name or hostname. Deprecate the OVN_Southbound:Port_Binding:options "requested-chassis" key. In a subsequent update to the controller we will improve the efficiency of the requested-chassis feature by using the new column instead of each chassis performing option processing and string comparison. Note that the CMS facing Northbound Logical_Switch_Port:options API remains the same. Signed-off-by: Frode Nordahl --- lib/chassis-index.c | 24 +++++++++ lib/chassis-index.h | 3 ++ northd/ovn-northd.c | 55 +++++++++++++++++-- northd/ovn_northd.dl | 124 ++++++++++++++++++++++++++++++++++++++++--- ovn-nb.xml | 33 ++++++++++-- ovn-sb.ovsschema | 10 ++-- ovn-sb.xml | 37 ++++++++++--- tests/ovn-northd.at | 45 ++++++++++++++++ 8 files changed, 307 insertions(+), 24 deletions(-) diff --git a/lib/chassis-index.c b/lib/chassis-index.c index 13120fe3e..4b38036cb 100644 --- a/lib/chassis-index.c +++ b/lib/chassis-index.c @@ -22,6 +22,12 @@ chassis_index_create(struct ovsdb_idl *idl) return ovsdb_idl_index_create1(idl, &sbrec_chassis_col_name); } +struct ovsdb_idl_index * +chassis_hostname_index_create(struct ovsdb_idl *idl) +{ + return ovsdb_idl_index_create1(idl, &sbrec_chassis_col_hostname); +} + /* Finds and returns the chassis with the given 'name', or NULL if no such * chassis exists. */ const struct sbrec_chassis * @@ -40,6 +46,24 @@ chassis_lookup_by_name(struct ovsdb_idl_index *sbrec_chassis_by_name, return retval; } +/* Finds and returns the chassis with the given 'hostname', or NULL if no such + * chassis exists. */ +const struct sbrec_chassis * +chassis_lookup_by_hostname(struct ovsdb_idl_index *sbrec_chassis_by_hostname, + const char *hostname) +{ + struct sbrec_chassis *target = sbrec_chassis_index_init_row( + sbrec_chassis_by_hostname); + sbrec_chassis_index_set_hostname(target, hostname); + + struct sbrec_chassis *retval = sbrec_chassis_index_find( + sbrec_chassis_by_hostname, target); + + sbrec_chassis_index_destroy_row(target); + + return retval; +} + struct ovsdb_idl_index * chassis_private_index_create(struct ovsdb_idl *idl) { diff --git a/lib/chassis-index.h b/lib/chassis-index.h index b9b331f34..bc654da13 100644 --- a/lib/chassis-index.h +++ b/lib/chassis-index.h @@ -19,9 +19,12 @@ struct ovsdb_idl; struct ovsdb_idl_index *chassis_index_create(struct ovsdb_idl *); +struct ovsdb_idl_index *chassis_hostname_index_create(struct ovsdb_idl *); const struct sbrec_chassis *chassis_lookup_by_name( struct ovsdb_idl_index *sbrec_chassis_by_name, const char *name); +const struct sbrec_chassis *chassis_lookup_by_hostname( + struct ovsdb_idl_index *sbrec_chassis_by_hostname, const char *hostname); struct ovsdb_idl_index *chassis_private_index_create(struct ovsdb_idl *); diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index ee761cef0..fdcc58e28 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -76,6 +76,7 @@ struct northd_context { struct ovsdb_idl_txn *ovnnb_txn; struct ovsdb_idl_txn *ovnsb_txn; struct ovsdb_idl_index *sbrec_chassis_by_name; + struct ovsdb_idl_index *sbrec_chassis_by_hostname; struct ovsdb_idl_index *sbrec_ha_chassis_grp_by_name; struct ovsdb_idl_index *sbrec_mcast_group_by_name_dp; struct ovsdb_idl_index *sbrec_ip_mcast_by_dp; @@ -3047,6 +3048,7 @@ ovn_update_ipv6_prefix(struct hmap *ports) static void ovn_port_update_sbrec(struct northd_context *ctx, struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_index *sbrec_chassis_by_hostname, const struct ovn_port *op, struct hmap *chassis_qdisc_queues, struct sset *active_ha_chassis_grps) @@ -3228,6 +3230,36 @@ ovn_port_update_sbrec(struct northd_context *ctx, * ha_chassis_group cleared in the same transaction. */ sbrec_port_binding_set_ha_chassis_group(op->sb, NULL); } + + const char *requested_chassis; /* May be NULL. */ + bool reset_requested_chassis = false; + requested_chassis = smap_get(&op->nbsp->options, + "requested-chassis"); + if (requested_chassis) { + const struct sbrec_chassis *chassis; /* May be NULL. */ + chassis = chassis_lookup_by_name(sbrec_chassis_by_name, + requested_chassis); + chassis = chassis ? chassis : chassis_lookup_by_hostname( + sbrec_chassis_by_hostname, requested_chassis); + + if (chassis) { + sbrec_port_binding_set_requested_chassis(op->sb, chassis); + } else { + reset_requested_chassis = true; + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT( + 1, 1); + VLOG_WARN_RL( + &rl, + "Unknown chassis '%s' set as " + "options:requested-chassis on LSP '%s'.", + requested_chassis, op->nbsp->name); + } + } else if (op->sb->requested_chassis) { + reset_requested_chassis = true; + } + if (reset_requested_chassis) { + sbrec_port_binding_set_requested_chassis(op->sb, NULL); + } } else { const char *chassis = NULL; if (op->peer && op->peer->od && op->peer->od->nbr) { @@ -3856,6 +3888,7 @@ ovn_port_allocate_key(struct hmap *ports, struct ovn_port *op) static void build_ports(struct northd_context *ctx, struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_index *sbrec_chassis_by_hostname, struct hmap *datapaths, struct hmap *ports) { struct ovs_list sb_only, nb_only, both; @@ -3911,6 +3944,7 @@ build_ports(struct northd_context *ctx, tag_alloc_create_new_tag(&tag_alloc_table, op->nbsp); } ovn_port_update_sbrec(ctx, sbrec_chassis_by_name, + sbrec_chassis_by_hostname, op, &chassis_qdisc_queues, &active_ha_chassis_grps); } @@ -3918,7 +3952,8 @@ build_ports(struct northd_context *ctx, /* Add southbound record for each unmatched northbound record. */ LIST_FOR_EACH_SAFE (op, next, list, &nb_only) { op->sb = sbrec_port_binding_insert(ctx->ovnsb_txn); - ovn_port_update_sbrec(ctx, sbrec_chassis_by_name, op, + ovn_port_update_sbrec(ctx, sbrec_chassis_by_name, + sbrec_chassis_by_hostname, op, &chassis_qdisc_queues, &active_ha_chassis_grps); sbrec_port_binding_set_logical_port(op->sb, op->key); @@ -14157,6 +14192,7 @@ get_probe_interval(const char *db, const struct nbrec_nb_global *nb) static void ovnnb_db_run(struct northd_context *ctx, struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_index *sbrec_chassis_by_hostname, struct ovsdb_idl_loop *sb_loop, struct hmap *datapaths, struct hmap *ports, struct ovs_list *lr_list, @@ -14260,7 +14296,8 @@ ovnnb_db_run(struct northd_context *ctx, build_datapaths(ctx, datapaths, lr_list); build_ovn_lbs(ctx, datapaths, &lbs); build_lrouter_lbs(datapaths, &lbs); - build_ports(ctx, sbrec_chassis_by_name, datapaths, ports); + build_ports(ctx, sbrec_chassis_by_name, sbrec_chassis_by_hostname, + datapaths, ports); build_ovn_lr_lbs(datapaths, &lbs); build_ovn_lb_svcs(ctx, ports, &lbs); build_ipam(datapaths, ports); @@ -14974,6 +15011,7 @@ ovnsb_db_run(struct northd_context *ctx, static void ovn_db_run(struct northd_context *ctx, struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_index *sbrec_chassis_by_hostname, struct ovsdb_idl_loop *ovnsb_idl_loop, const char *ovn_internal_version) { @@ -14985,8 +15023,8 @@ ovn_db_run(struct northd_context *ctx, int64_t start_time = time_wall_msec(); stopwatch_start(OVNNB_DB_RUN_STOPWATCH_NAME, time_msec()); - ovnnb_db_run(ctx, sbrec_chassis_by_name, ovnsb_idl_loop, - &datapaths, &ports, &lr_list, start_time, + ovnnb_db_run(ctx, sbrec_chassis_by_name, sbrec_chassis_by_hostname, + ovnsb_idl_loop, &datapaths, &ports, &lr_list, start_time, ovn_internal_version); stopwatch_stop(OVNNB_DB_RUN_STOPWATCH_NAME, time_msec()); stopwatch_start(OVNSB_DB_RUN_STOPWATCH_NAME, time_msec()); @@ -15237,6 +15275,8 @@ main(int argc, char *argv[]) add_column_noalert(ovnsb_idl_loop.idl, &sbrec_port_binding_col_mac); add_column_noalert(ovnsb_idl_loop.idl, &sbrec_port_binding_col_nat_addresses); + add_column_noalert(ovnsb_idl_loop.idl, + &sbrec_port_binding_col_requested_chassis); ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_port_binding_col_chassis); ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_port_binding_col_gateway_chassis); @@ -15308,6 +15348,7 @@ main(int argc, char *argv[]) ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_chassis); ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_chassis_col_name); + ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_chassis_col_hostname); ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_chassis_col_other_config); ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_chassis_col_encaps); @@ -15416,6 +15457,9 @@ main(int argc, char *argv[]) struct ovsdb_idl_index *sbrec_chassis_by_name = chassis_index_create(ovnsb_idl_loop.idl); + struct ovsdb_idl_index *sbrec_chassis_by_hostname + = chassis_hostname_index_create(ovnsb_idl_loop.idl); + struct ovsdb_idl_index *sbrec_ha_chassis_grp_by_name = ha_chassis_group_index_create(ovnsb_idl_loop.idl); @@ -15493,7 +15537,8 @@ main(int argc, char *argv[]) } if (ovsdb_idl_has_lock(ovnsb_idl_loop.idl)) { - ovn_db_run(&ctx, sbrec_chassis_by_name, &ovnsb_idl_loop, + ovn_db_run(&ctx, sbrec_chassis_by_name, + sbrec_chassis_by_hostname, &ovnsb_idl_loop, ovn_internal_version); if (ctx.ovnsb_txn) { check_and_add_supported_dhcp_opts_to_sb_db(&ctx); diff --git a/northd/ovn_northd.dl b/northd/ovn_northd.dl index ff92c989c..85c62c547 100644 --- a/northd/ovn_northd.dl +++ b/northd/ovn_northd.dl @@ -105,6 +105,22 @@ sb::Out_Datapath_Binding(uuid, tunkey, load_balancers, external_ids) :- */ var load_balancers = set_empty(). +function get_requested_chassis(options: Map) : string = { + var requested_chassis = match(options.get("requested-chassis")) { + None -> "", + Some{requested_chassis} -> requested_chassis, + }; + requested_chassis +} + +relation RequestedChassis( + name: string, + chassis: uuid, +) +RequestedChassis(name, chassis) :- + sb::Chassis(._uuid = chassis, .name=name). +RequestedChassis(hostname, chassis) :- + sb::Chassis(._uuid = chassis, .hostname=hostname). /* Proxy table for Out_Datapath_Binding: contains all Datapath_Binding fields, * except tunnel id, which is allocated separately (see PortTunKeyAllocation). */ @@ -120,10 +136,12 @@ relation OutProxy_Port_Binding ( tag: Option, mac: Set, nat_addresses: Set, - external_ids: Map + external_ids: Map, + requested_chassis: Option ) -/* Case 1: Create a Port_Binding per logical switch port that is not of type "router" */ +/* Case 1a: Create a Port_Binding per logical switch port that is not of type + * "router" */ OutProxy_Port_Binding(._uuid = lsp._uuid, .logical_port = lsp.name, .__type = lsp.__type, @@ -135,7 +153,8 @@ OutProxy_Port_Binding(._uuid = lsp._uuid, .tag = tag, .mac = lsp.addresses, .nat_addresses = set_empty(), - .external_ids = eids) :- + .external_ids = eids, + .requested_chassis = None) :- sp in &SwitchPort(.lsp = lsp, .sw = sw), SwitchPortNewDynamicTag(lsp._uuid, opt_tag), var tag = match (opt_tag) { @@ -143,6 +162,8 @@ OutProxy_Port_Binding(._uuid = lsp._uuid, Some{t} -> Some{t} }, lsp.__type != "router", + var chassis_name_or_hostname = get_requested_chassis(lsp.options), + chassis_name_or_hostname == "", var eids = { var eids = lsp.external_ids; match (lsp.external_ids.get("neutron:port_name")) { @@ -160,6 +181,91 @@ OutProxy_Port_Binding(._uuid = lsp._uuid, options }. +/* Case 1b: Create a Port_Binding per logical switch port that is not of type + * "router" and has options "requested-chassis" pointing at chassis name or + * hostname. */ +OutProxy_Port_Binding(._uuid = lsp._uuid, + .logical_port = lsp.name, + .__type = lsp.__type, + .gateway_chassis = set_empty(), + .ha_chassis_group = sp.hac_group_uuid, + .options = options, + .datapath = sw._uuid, + .parent_port = lsp.parent_name, + .tag = tag, + .mac = lsp.addresses, + .nat_addresses = set_empty(), + .external_ids = eids, + .requested_chassis = Some{requested_chassis}) :- + sp in &SwitchPort(.lsp = lsp, .sw = sw), + SwitchPortNewDynamicTag(lsp._uuid, opt_tag), + var tag = match (opt_tag) { + None -> lsp.tag, + Some{t} -> Some{t} + }, + lsp.__type != "router", + var chassis_name_or_hostname = get_requested_chassis(lsp.options), + chassis_name_or_hostname != "", + RequestedChassis(chassis_name_or_hostname, requested_chassis), + var eids = { + var eids = lsp.external_ids; + match (lsp.external_ids.get("neutron:port_name")) { + None -> (), + Some{name} -> eids.insert("name", name) + }; + eids + }, + var options = { + var options = lsp.options; + match (sw.other_config.get("vlan-passthru")) { + Some{"true"} -> options.insert("vlan-passthru", "true"), + _ -> () + }; + options + }. + +/* Case 1c: Create a Port_Binding per logical switch port that is not of type + * "router" and has options "requested-chassis" pointing at non-existent + * chassis name or hostname. */ +OutProxy_Port_Binding(._uuid = lsp._uuid, + .logical_port = lsp.name, + .__type = lsp.__type, + .gateway_chassis = set_empty(), + .ha_chassis_group = sp.hac_group_uuid, + .options = options, + .datapath = sw._uuid, + .parent_port = lsp.parent_name, + .tag = tag, + .mac = lsp.addresses, + .nat_addresses = set_empty(), + .external_ids = eids, + .requested_chassis = None) :- + sp in &SwitchPort(.lsp = lsp, .sw = sw), + SwitchPortNewDynamicTag(lsp._uuid, opt_tag), + var tag = match (opt_tag) { + None -> lsp.tag, + Some{t} -> Some{t} + }, + lsp.__type != "router", + var chassis_name_or_hostname = get_requested_chassis(lsp.options), + chassis_name_or_hostname != "", + not RequestedChassis(chassis_name_or_hostname, _), + var eids = { + var eids = lsp.external_ids; + match (lsp.external_ids.get("neutron:port_name")) { + None -> (), + Some{name} -> eids.insert("name", name) + }; + eids + }, + var options = { + var options = lsp.options; + match (sw.other_config.get("vlan-passthru")) { + Some{"true"} -> options.insert("vlan-passthru", "true"), + _ -> () + }; + options + }. /* Case 2: Create a Port_Binding per logical switch port of type "router" */ OutProxy_Port_Binding(._uuid = lsp._uuid, @@ -173,7 +279,8 @@ OutProxy_Port_Binding(._uuid = lsp._uuid, .tag = None, .mac = lsp.addresses, .nat_addresses = nat_addresses, - .external_ids = eids) :- + .external_ids = eids, + .requested_chassis = None) :- &SwitchPort(.lsp = lsp, .sw = sw, .peer = peer), var eids = { var eids = lsp.external_ids; @@ -263,7 +370,8 @@ OutProxy_Port_Binding(._uuid = lrp._uuid, .tag = None, // always empty for router ports .mac = set_singleton("${lrp.mac} ${lrp.networks.join(\" \")}"), .nat_addresses = set_empty(), - .external_ids = lrp.external_ids) :- + .external_ids = lrp.external_ids, + .requested_chassis = None) :- rp in &RouterPort(.lrp = lrp, .router = router, .peer = peer), RouterPortRAOptionsComplete(lrp._uuid, options0), (var __type, var options1) = match (router.options.get("chassis")) { @@ -471,7 +579,8 @@ OutProxy_Port_Binding(// lrp._uuid is already in use; generate a new UUID by .tag = None, //always empty for router ports .mac = set_singleton("${lrp.mac} ${lrp.networks.join(\" \")}"), .nat_addresses = set_empty(), - .external_ids = lrp.external_ids) :- + .external_ids = lrp.external_ids, + .requested_chassis = None) :- DistributedGatewayPort(lrp, lr_uuid), DistributedGatewayPortHAChassisGroup(lrp, hacg_uuid), var redirect_type = match (lrp.options.get("redirect-type")) { @@ -516,7 +625,8 @@ sb::Out_Port_Binding(._uuid = pbinding._uuid, .mac = pbinding.mac, .nat_addresses = pbinding.nat_addresses, .external_ids = pbinding.external_ids, - .up = Some{up}) :- + .up = Some{up}, + .requested_chassis = pbinding.requested_chassis) :- pbinding in OutProxy_Port_Binding(), PortTunKeyAllocation(pbinding._uuid, tunkey), QueueIDAllocation(pbinding._uuid, qid), diff --git a/ovn-nb.xml b/ovn-nb.xml index 390cc5a44..ef2677d94 100644 --- a/ovn-nb.xml +++ b/ovn-nb.xml @@ -995,11 +995,16 @@ If set, identifies a specific chassis (by name or hostname) that - is allowed to bind this port. Using this option will prevent + is allowed to bind or plug this port. Using this option will prevent thrashing between two chassis trying to bind the same port during - a live migration. It can also prevent similar thrashing due to a + a live migration. It can also prevent similar thrashing due to a mis-configuration, if a port is accidentally created on more than - one chassis. + one chassis. This is also used to allow the controller consider + unbound ports for plugging without having to process ports not + destined for its chassis. + + Setting this option is a prerequisite for using the + option (see below). @@ -1028,6 +1033,28 @@ DHCP reply.

+ + + + If set, OVN will attempt to to perform plugging of this VIF. In + order to get this port plugged by the OVN controller, OVN must be + built with support for VIF plugging. The default behavior is for + the CMS to do the VIF plugging. Each plug provider have their own + options namespaced by name, for example "plug:representor:key". + Please refer to the plug provider documentation for more + information. + + Supported values: representor + + + + Requested MTU for plugged interfaces. When set the OVN controller + will fill the column + of the Open vSwitch database's + table. This in turn will + make OVS vswitchd update the MTU of the linked interface. + + diff --git a/ovn-sb.ovsschema b/ovn-sb.ovsschema index e5ab41db9..122614dd5 100644 --- a/ovn-sb.ovsschema +++ b/ovn-sb.ovsschema @@ -1,7 +1,7 @@ { "name": "OVN_Southbound", - "version": "20.20.0", - "cksum": "605270161 26670", + "version": "20.21.0", + "cksum": "2362446865 26963", "tables": { "SB_Global": { "columns": { @@ -232,7 +232,11 @@ "external_ids": {"type": {"key": "string", "value": "string", "min": 0, - "max": "unlimited"}}}, + "max": "unlimited"}}, + "requested_chassis": {"type": {"key": {"type": "uuid", + "refTable": "Chassis", + "refType": "weak"}, + "min": 0, "max": 1}}}, "indexes": [["datapath", "tunnel_key"], ["logical_port"]], "isRoot": true}, "MAC_Binding": { diff --git a/ovn-sb.xml b/ovn-sb.xml index 2d4d47d10..c966542e7 100644 --- a/ovn-sb.xml +++ b/ovn-sb.xml @@ -2983,6 +2983,26 @@ tcp.flags = RST; + + If set, identifies a specific chassis that is allowed to bind or plug + this port. Having a value in this column will prevent thrashing + between two chassis trying to bind the same port during a live + migration. + It can also prevent similar thrashing due to a mis-configuration, if a + port is accidentally created on more than one chassis. This is also + used to allow the controller consider unbound ports for plugging + without having to process ports not destined for its chassis. + + This column must be a + record. This is populated by + ovn-northd when the + is defined and contains a string matching the name or hostname of an + existing chassis. + @@ -3141,12 +3161,17 @@ tcp.flags = RST;

- If set, identifies a specific chassis (by name or hostname) that - is allowed to bind this port. Using this option will prevent - thrashing between two chassis trying to bind the same port during - a live migration. It can also prevent similar thrashing due to a - mis-configuration, if a port is accidentally created on more than - one chassis. + Deprecated. This option has been replaced by a separate column + + and may at some point in the future no longer be copied over from + . diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 11886b94e..8ccf806c7 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -5225,3 +5225,48 @@ AT_CHECK([grep lr_in_gw_redirect lrflows | grep cr-DR | sed 's/table=../table=?? AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([check options:requested-chassis fills requested_chassis col]) +ovn_start NORTHD_TYPE + +# Add chassis ch1. +check ovn-sbctl chassis-add ch1 geneve 127.0.0.2 +check ovn-sbctl chassis-add ch2 geneve 127.0.0.3 + +wait_row_count Chassis 2 + +ch1_uuid=`ovn-sbctl --bare --columns _uuid find Chassis name="ch1"` +ch2_uuid=`ovn-sbctl --bare --columns _uuid find Chassis name="ch2"` + +check ovn-sbctl set chassis $ch2_uuid hostname=ch2-hostname + +ovn-nbctl ls-add S1 +ovn-nbctl --wait=sb lsp-add S1 S1-vm1 +ovn-nbctl --wait=sb lsp-add S1 S1-vm2 + +wait_row_count Port_Binding 1 logical_port=S1-vm1 requested_chassis!=$ch1_uuid +wait_row_count Port_Binding 1 logical_port=S1-vm2 requested_chassis!=$ch2_uuid + +ovn-nbctl --wait=sb set logical_switch_port S1-vm1 \ + options:requested-chassis=ch1 + +wait_row_count Port_Binding 1 logical_port=S1-vm1 requested_chassis=$ch1_uuid + +ovn-nbctl --wait=sb set logical_switch_port S1-vm2 \ + options:requested-chassis=ch2-hostname + +wait_row_count Port_binding 1 logical-port=S1-vm2 requested_chassis=$ch2_uuid + +ovn-nbctl --wait=sb remove logical_switch_port S1-vm2 \ + options requested-chassis=ch2-hostname + +wait_row_count Port_binding 1 logical-port=S1-vm2 requested_chassis!=$ch2_uuid + +ovn-nbctl --wait=sb set logical_switch_port S1-vm2 \ + options:requested-chassis=ch2 + +wait_row_count Port_binding 1 logical-port=S1-vm2 requested_chassis=$ch2_uuid + +AT_CLEANUP +]) From patchwork Fri Sep 3 19:27:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frode Nordahl X-Patchwork-Id: 1524518 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=oYcwgRQG; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H1SW75X5Zz9sCD for ; Sat, 4 Sep 2021 05:28:03 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 840BD426EE; Fri, 3 Sep 2021 19:28:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 93G6h4U9ELNS; Fri, 3 Sep 2021 19:27:59 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id F3E8E426EC; Fri, 3 Sep 2021 19:27:58 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 95D94C0028; Fri, 3 Sep 2021 19:27:56 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 58318C000E for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id B94834027D for ; Fri, 3 Sep 2021 19:27:53 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=canonical.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id t898mHAflpxh for ; Fri, 3 Sep 2021 19:27:52 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from smtp-relay-canonical-1.canonical.com (smtp-relay-canonical-1.canonical.com [185.125.188.121]) by smtp2.osuosl.org (Postfix) with ESMTPS id A66C74070E for ; Fri, 3 Sep 2021 19:27:52 +0000 (UTC) Received: from frode-threadripper.. (1.general.frode.uk.vpn [10.172.193.250]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id B2A9B401A1; Fri, 3 Sep 2021 19:27:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1630697269; bh=fPwJn5/TF25PMr/cW1eQsgGFwM82CilyyWuBLLmBmVE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=oYcwgRQGfji+k3XlU9gUg1CJM22vBaasrsnLUnlUIW0EG1F/zhpADBAUOCPGxF8cI WYE8Bykyh9ZnuTMEerXLSIP7QExIxszqbzuw7u+Q6dSIaKTEAd7UktDDXg9MDfaNZt 4mOjgMiZ0f9UroerUrMuj+fC8EeJXLkhFMBFjBLsogRln7eCzpMVsijCev1LiGb9ay KxXf+IN7bkmlgvKuyjvL1cqHReAK3AoAdDsjbq0sHSipiRxehiYSNJbYGAOcOynZVu Qjng6S77p3n8x/qhTpx3Vw+VS2Mg7BnoTJpkKMFaGf+yVZ/lDV+/n5M/YfyUM7FniB O9dhTMl0mr1Iw== From: Frode Nordahl To: dev@openvswitch.org Date: Fri, 3 Sep 2021 21:27:41 +0200 Message-Id: <20210903192748.1408062-3-frode.nordahl@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210903192748.1408062-1-frode.nordahl@canonical.com> References: <20210903192748.1408062-1-frode.nordahl@canonical.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v4 2/9] tests: Stop using non-existent requested-chassis X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" A couple of tests refers to non-existent chassis to validate that the ovn-controller would release ports that otherwise was destined for it when requested-chassis points somewhere else. With upcoming changes to have the ovn-controller use the new Port_Binding:requested_chassis column maintained by northd instead of processing the options itself, referring to a non-existant chassis will no longer work as the tests expect. With this patch we instead add a throw-away chassis to the simulation and point the requested-chassis option at that chassis. Signed-off-by: Frode Nordahl --- tests/ovn.at | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/tests/ovn.at b/tests/ovn.at index 5104a6895..2914018e8 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -9434,6 +9434,11 @@ ovn-nbctl lsp-add bar bar3 \ # Create two hypervisor and create OVS ports corresponding to logical ports. net_add n1 +sim_add hv-foo +as hv-foo +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.253 + sim_add hv1 as hv1 ovs-vsctl add-br br-phys @@ -9603,7 +9608,7 @@ as hv1 ovs-appctl netdev-dummy/receive vm1 $packet OVN_CHECK_PACKETS([hv1/vm1-tx.pcap], [expected1]) # Test binding of parent and container ports. -ovn-nbctl lsp-set-options vm1 requested-chassis=foo +ovn-nbctl lsp-set-options vm1 requested-chassis=hv-foo OVS_WAIT_UNTIL([test xdown = x$(ovn-nbctl lsp-get-up vm1)]) OVS_WAIT_UNTIL([test xdown = x$(ovn-nbctl lsp-get-up foo1)]) @@ -13454,6 +13459,10 @@ ovn-nbctl ls-add ls0 ovn-nbctl lsp-add ls0 lsp0 net_add n1 +sim_add hv-foo +as hv-foo +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.10 sim_add hv1 as hv1 ovs-vsctl add-br br-phys @@ -13473,7 +13482,7 @@ wait_column "$hv1_uuid" Port_Binding chassis logical_port=lsp0 AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=0 | grep in_port=1], [0], [ignore]) AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=65 | grep actions=output:1], [0], [ignore]) -check ovn-nbctl --wait=hv lsp-set-options lsp0 requested-chassis=non-existant-chassis +check ovn-nbctl --wait=hv lsp-set-options lsp0 requested-chassis=hv-foo OVS_WAIT_UNTIL([test 1 = $(grep -c "Releasing lport lsp0 from this chassis" hv1/ovn-controller.log)]) check ovn-nbctl --wait=hv sync wait_column '' Port_Binding chasssi logical_port=lsp0 From patchwork Fri Sep 3 19:27:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frode Nordahl X-Patchwork-Id: 1524517 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=fuJ71Ihx; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H1SW35yKWz9sCD for ; Sat, 4 Sep 2021 05:27:59 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 2A928426E3; Fri, 3 Sep 2021 19:27:57 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aP-h0pIYVUbV; Fri, 3 Sep 2021 19:27:56 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 3E6AB426C7; Fri, 3 Sep 2021 19:27:55 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 088D7C001C; Fri, 3 Sep 2021 19:27:55 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id C35F3C000E for ; Fri, 3 Sep 2021 19:27:53 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 9A52A407BA for ; Fri, 3 Sep 2021 19:27:53 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=canonical.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aYO0NT4z06Gf for ; Fri, 3 Sep 2021 19:27:52 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from smtp-relay-canonical-1.canonical.com (smtp-relay-canonical-1.canonical.com [185.125.188.121]) by smtp2.osuosl.org (Postfix) with ESMTPS id 7E2E64027D for ; Fri, 3 Sep 2021 19:27:52 +0000 (UTC) Received: from frode-threadripper.. (1.general.frode.uk.vpn [10.172.193.250]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id 0D352401A3; Fri, 3 Sep 2021 19:27:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1630697270; bh=cU0GgvAWqV+7UQxP1qgVM8KrLEP8knx9JoVoJ+2iPLU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fuJ71IhxcdqnM6EMp0XTp8I2KZu7WGcpk1R+t2m8IZKaZkXfqU0haCfdRAJ0/sZlL HipVdVOqQOmGZXAPNOO5q4XGGyZgP1VQ8FL4mPnoLDvOGN4qgJskPzf+aMeEG5G9Rv /lk3KaC5dOCrcC3Ypv54l3EhM4++vlipAVgifsOEPq2CKdPhFXMluPHnFq4o9SxY0F 3GYRxpGZbWGgYDsHddWdy786lBijUmSoPYHUBOzHoE0jQxsKTph3AD6NzksA/x4NtT hwhBrq3M5B/QrVJ/jt5IchJIivtESJ+vFg1Pey4cKZi9nbGEuQAdwqhFio+fe9unA+ XS2FRiqk4QWjg== From: Frode Nordahl To: dev@openvswitch.org Date: Fri, 3 Sep 2021 21:27:42 +0200 Message-Id: <20210903192748.1408062-4-frode.nordahl@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210903192748.1408062-1-frode.nordahl@canonical.com> References: <20210903192748.1408062-1-frode.nordahl@canonical.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v4 3/9] controller: Make use of Port_Binding:requested_chassis X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Improve the efficiency of the requested-chassis feature by using the new Southbound Port_Binding:requested_chassis column instead of each chassis performing option processing and string comparison. Signed-off-by: Frode Nordahl --- controller/binding.c | 29 ++++++++++++----------------- controller/ovn-controller.c | 3 +++ controller/physical.c | 7 ++----- 3 files changed, 17 insertions(+), 22 deletions(-) diff --git a/controller/binding.c b/controller/binding.c index 34935bb9c..938e1d81d 100644 --- a/controller/binding.c +++ b/controller/binding.c @@ -1051,11 +1051,10 @@ is_binding_lport_this_chassis(struct binding_lport *b_lport, static bool can_bind_on_this_chassis(const struct sbrec_chassis *chassis_rec, - const char *requested_chassis) + const struct sbrec_port_binding *pb) { - return !requested_chassis || !requested_chassis[0] - || !strcmp(requested_chassis, chassis_rec->name) - || !strcmp(requested_chassis, chassis_rec->hostname); + return !pb->requested_chassis + || chassis_rec == pb->requested_chassis; } /* Returns 'true' if the 'lbinding' has binding lports of type LP_CONTAINER, @@ -1093,7 +1092,7 @@ release_binding_lport(const struct sbrec_chassis *chassis_rec, static bool consider_vif_lport_(const struct sbrec_port_binding *pb, - bool can_bind, const char *vif_chassis, + bool can_bind, struct binding_ctx_in *b_ctx_in, struct binding_ctx_out *b_ctx_out, struct binding_lport *b_lport, @@ -1134,7 +1133,8 @@ consider_vif_lport_(const struct sbrec_port_binding *pb, "requested-chassis %s", pb->logical_port, b_ctx_in->chassis_rec->name, - vif_chassis); + pb->requested_chassis ? + pb->requested_chassis->name : "(none)"); } } @@ -1157,9 +1157,7 @@ consider_vif_lport(const struct sbrec_port_binding *pb, struct local_binding *lbinding, struct hmap *qos_map) { - const char *vif_chassis = smap_get(&pb->options, "requested-chassis"); - bool can_bind = can_bind_on_this_chassis(b_ctx_in->chassis_rec, - vif_chassis); + bool can_bind = can_bind_on_this_chassis(b_ctx_in->chassis_rec, pb); if (!lbinding) { lbinding = local_binding_find(&b_ctx_out->lbinding_data->bindings, @@ -1189,8 +1187,8 @@ consider_vif_lport(const struct sbrec_port_binding *pb, } } - return consider_vif_lport_(pb, can_bind, vif_chassis, b_ctx_in, - b_ctx_out, b_lport, qos_map); + return consider_vif_lport_(pb, can_bind, b_ctx_in, b_ctx_out, + b_lport, qos_map); } static bool @@ -1274,12 +1272,9 @@ consider_container_lport(const struct sbrec_port_binding *pb, } ovs_assert(parent_b_lport && parent_b_lport->pb); - const char *vif_chassis = smap_get(&parent_b_lport->pb->options, - "requested-chassis"); - bool can_bind = can_bind_on_this_chassis(b_ctx_in->chassis_rec, - vif_chassis); + bool can_bind = can_bind_on_this_chassis(b_ctx_in->chassis_rec, pb); - return consider_vif_lport_(pb, can_bind, vif_chassis, b_ctx_in, b_ctx_out, + return consider_vif_lport_(pb, can_bind, b_ctx_in, b_ctx_out, container_b_lport, qos_map); } @@ -1328,7 +1323,7 @@ consider_virtual_lport(const struct sbrec_port_binding *pb, } } - if (!consider_vif_lport_(pb, true, NULL, b_ctx_in, b_ctx_out, + if (!consider_vif_lport_(pb, true, b_ctx_in, b_ctx_out, virtual_b_lport, qos_map)) { return false; } diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c index 0031a1035..7387a177b 100644 --- a/controller/ovn-controller.c +++ b/controller/ovn-controller.c @@ -232,6 +232,9 @@ update_sb_monitors(struct ovsdb_idl *ovnsb_idl, sbrec_port_binding_add_clause_chassis(&pb, OVSDB_F_EQ, &chassis->header_.uuid); + sbrec_port_binding_add_clause_requested_chassis( + &pb, OVSDB_F_EQ, &chassis->header_.uuid); + /* Ensure that we find out about l2gateway and l3gateway ports that * should be present on this chassis. Otherwise, we might never find * out about those ports, if their datapaths don't otherwise have a VIF diff --git a/controller/physical.c b/controller/physical.c index 6f2c1cea0..2f6bd0e91 100644 --- a/controller/physical.c +++ b/controller/physical.c @@ -1066,11 +1066,8 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name, } else { ofport = local_binding_get_lport_ofport(local_bindings, binding->logical_port); - const char *requested_chassis = smap_get(&binding->options, - "requested-chassis"); - if (ofport && requested_chassis && requested_chassis[0] && - strcmp(requested_chassis, chassis->name) && - strcmp(requested_chassis, chassis->hostname)) { + if (ofport && binding->requested_chassis + && binding->requested_chassis != chassis) { /* Even though there is an ofport for this port_binding, it is * requested on a different chassis. So ignore this ofport. */ From patchwork Fri Sep 3 19:27:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frode Nordahl X-Patchwork-Id: 1524519 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=NsxBCl+N; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H1SW77558z9sSs for ; Sat, 4 Sep 2021 05:28:03 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 0713D40839; Fri, 3 Sep 2021 19:28:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WjTM0mKsNIlG; Fri, 3 Sep 2021 19:27:59 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id DE2BE4081B; Fri, 3 Sep 2021 19:27:57 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C4793C0024; Fri, 3 Sep 2021 19:27:55 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 11222C000E for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 0055961584 for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=canonical.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pS-Iwt-wbt6d for ; Fri, 3 Sep 2021 19:27:52 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from smtp-relay-canonical-1.canonical.com (smtp-relay-canonical-1.canonical.com [185.125.188.121]) by smtp3.osuosl.org (Postfix) with ESMTPS id 7D66F61542 for ; Fri, 3 Sep 2021 19:27:52 +0000 (UTC) Received: from frode-threadripper.. (1.general.frode.uk.vpn [10.172.193.250]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id 5CC13401A7; Fri, 3 Sep 2021 19:27:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1630697270; bh=qDKddMr2hyQXl8C/ogw0w+X+UyAodDsypoaem7LcOPI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NsxBCl+NBkY20BOIkc7f0gYULExIc9DblwCn5xDdDszjt482hYa3rH3HczaB2q1PY RiOjLJWixWqBAJVYbdcY06oHxv3+LRTrsD/axCCpMar/Eg+g8Gppf7RV43oxzJSSWG 1fDeUShugLJjpzuvdXzEXXrGasWruhaG+kuq3Gz206kRMKQ6WzYs4Y8yHYWRr0UDhx H9XI/lMYLPi5rWv/pEQj//37ae8wUov8n+i/PKo0/9Lx6Z14EzfG28N54M52iZewgg chH+Pw3BDm3wtUK+DmnA/fxEhhHtv+uBM2oBMVaJPJZmFLr/4WOjZv6z6j1Vj1wPEZ 3rZREfOskaCrQ== From: Frode Nordahl To: dev@openvswitch.org Date: Fri, 3 Sep 2021 21:27:43 +0200 Message-Id: <20210903192748.1408062-5-frode.nordahl@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210903192748.1408062-1-frode.nordahl@canonical.com> References: <20210903192748.1408062-1-frode.nordahl@canonical.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v4 4/9] controller: Move OVS port functions to new module. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Up until now the controller patch module has been the only consumer of functions to maintain OVS ports and interfaces. With the introduction of infrastructure for plugging providers these functions will also be consumed by the controller binding module. As such we introduce a new module called ovsport where these shared utility functions can live and be consumed by multiple modules. Signed-off-by: Frode Nordahl --- controller/automake.mk | 4 +- controller/ovsport.c | 256 +++++++++++++++++++++++++++++++++++++++++ controller/ovsport.h | 60 ++++++++++ 3 files changed, 319 insertions(+), 1 deletion(-) create mode 100644 controller/ovsport.c create mode 100644 controller/ovsport.h diff --git a/controller/automake.mk b/controller/automake.mk index 41f907d6e..ad2d68af2 100644 --- a/controller/automake.mk +++ b/controller/automake.mk @@ -35,7 +35,9 @@ controller_ovn_controller_SOURCES = \ controller/mac-learn.c \ controller/mac-learn.h \ controller/local_data.c \ - controller/local_data.h + controller/local_data.h \ + controller/ovsport.h \ + controller/ovsport.c controller_ovn_controller_LDADD = lib/libovn.la $(OVS_LIBDIR)/libopenvswitch.la man_MANS += controller/ovn-controller.8 diff --git a/controller/ovsport.c b/controller/ovsport.c new file mode 100644 index 000000000..b1183e9ed --- /dev/null +++ b/controller/ovsport.c @@ -0,0 +1,256 @@ +/* Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include "ovsport.h" + +#include "lib/vswitch-idl.h" +#include "openvswitch/vlog.h" + +VLOG_DEFINE_THIS_MODULE(ovsport); + +/* Create a port and interface record and add it to 'bridge' in the Open + * vSwitch database represented by 'ovs_idl_txn'. + * + * 'name' is required and is used both for the name of the port and interface + * records. Depending on the contents of the optional 'iface_type' parameter + * the name may need to refer to an existing interface in the system. It is + * the callers responsibility to ensure that no other port with the desired + * name already exist. + * + * 'iface_type' optionally specifies the type of intercace, otherwise set it to + * NULL. + * + * 'port_external_ids' - the contents of the map will be used to fill the + * external_ids column of the created port record, otherwise set it to NULL. + * + * 'iface_external_ids' - the contents of the map will be used to fill the + * external_ids column of the created interface record, otherwise set it to + * NULL. + * + * 'iface_options' - the contents of the map will be used to fill the options + * column of the created interface record, otherwise set it to NULL. + * + * 'iface_mtu_request' - if a value > 0 is provided it will be filled into the + * mtu_request column of the created interface record. */ +void +ovsport_create(struct ovsdb_idl_txn *ovs_idl_txn, + const struct ovsrec_bridge *bridge, + const char *name, + const char *iface_type, + const struct smap *port_external_ids, + const struct smap *iface_external_ids, + const struct smap *iface_options, + const int64_t iface_mtu_request) +{ + struct ovsrec_interface *iface; + iface = ovsrec_interface_insert(ovs_idl_txn); + ovsrec_interface_set_name(iface, name); + if (iface_type) { + ovsrec_interface_set_type(iface, iface_type); + } + ovsrec_interface_set_external_ids(iface, iface_external_ids); + ovsrec_interface_set_options(iface, iface_options); + ovsrec_interface_set_mtu_request( + iface, &iface_mtu_request, iface_mtu_request > 0); + + struct ovsrec_port *port; + port = ovsrec_port_insert(ovs_idl_txn); + ovsrec_port_set_name(port, name); + ovsrec_port_set_interfaces(port, &iface, 1); + ovsrec_port_set_external_ids(port, port_external_ids); + + ovsrec_bridge_update_ports_addvalue(bridge, port); + ovsrec_bridge_verify_ports(bridge); +} + +/* Remove 'port' from 'bridge' and delete the 'port' recrd and any records + * with a weakRef to it. */ +void +ovsport_remove(const struct ovsrec_bridge *bridge, + const struct ovsrec_port *port) +{ + ovsrec_bridge_update_ports_delvalue(bridge, port); + ovsrec_bridge_verify_ports(bridge); + ovsrec_port_delete(port); +} + +static void update_interface_smap_column( + const struct ovsrec_interface *, const struct smap *, + const struct smap *, void (*fsetkey)(const struct ovsrec_interface *, + const char *, const char *)); +static void maintain_interface_smap_column( + const struct ovsrec_interface *, const struct sset *, + const struct smap *, const struct smap *, + void (*fsetkey)(const struct ovsrec_interface *, const char *, + const char *), + void (*fdelkey)(const struct ovsrec_interface *, + const char *)); + +/* Update interface record as represented by 'iface'. + * + * 'type' optionally specifies the type of interface, to unset type set to an + * empty string, to not update type set to NULL. + * + * 'external_ids' optionally provide a map of external_ids to update, to not + * update external_ids set to NULL. + * + * 'mnt_external_ids' optionally provide set of 'external_ids' to maintain. + * When set the function will make sure that all keys in the 'mnt_external_ids' + * set have values from the 'external_ids' map in the database. Every key that + * exist in 'mnt_external_ids' with no corresponding key in 'external_ids' will + * be removed from the database if present. Set to NULL to not maintain the + * record in this way. + * + * 'options' optionally provide a map of options to update, to not + * update options set to NULL. + * + * 'mnt_options' optionally provide set of 'options' to maintain. + * When set the function will make sure that all keys in the 'mnt_options' set + * have values from the 'options' map in the database. Every key that exist in + * 'mnt_options' with no corresponding key in 'options' will be + * removed from the database if present. Set to NULL to not maintain the + * record in this way. + * + * 'iface_mtu_request' - if a value > 0 is provided it will be filled into the + * mtu_request column of the created interface record. */ +void +ovsport_update_iface(const struct ovsrec_interface *iface, + const char *type, + const struct smap *external_ids, + const struct sset *mnt_external_ids, + const struct smap *options, + const struct sset *mnt_options, + const int64_t mtu_request) +{ + if (type && strcmp(iface->type, type)) { + ovsrec_interface_set_type(iface, type); + } + + if (external_ids && mnt_external_ids) { + maintain_interface_smap_column( + iface, mnt_external_ids, external_ids, &iface->external_ids, + ovsrec_interface_update_external_ids_setkey, + ovsrec_interface_update_external_ids_delkey); + } else if (external_ids) { + update_interface_smap_column( + iface, external_ids, &iface->external_ids, + ovsrec_interface_update_external_ids_setkey); + } + + if (options && mnt_options) { + maintain_interface_smap_column( + iface, mnt_options, options, &iface->options, + ovsrec_interface_update_options_setkey, + ovsrec_interface_update_options_delkey); + } else if (options) { + update_interface_smap_column( + iface, options, &iface->options, + ovsrec_interface_update_options_setkey); + } + + if (mtu_request > 0) { + if ((iface->mtu_request && *iface->mtu_request != mtu_request) + || !iface->mtu_request) + { + ovsrec_interface_set_mtu_request( + iface, &mtu_request, mtu_request > 0); + } + } else if (iface->mtu_request) { + ovsrec_interface_update_mtu_request_delvalue(iface, + *iface->mtu_request); + } +} + +const struct ovsrec_port * +ovsport_lookup_by_interfaces( + struct ovsdb_idl_index *ovsrec_port_by_interfaces, + struct ovsrec_interface **interfaces, + const size_t n_interfaces) +{ + struct ovsrec_port *port = ovsrec_port_index_init_row( + ovsrec_port_by_interfaces); + ovsrec_port_index_set_interfaces(port, interfaces, n_interfaces); + + const struct ovsrec_port *retval = ovsrec_port_index_find( + ovsrec_port_by_interfaces, port); + + ovsrec_port_index_destroy_row(port); + + return retval; +} + +const struct +ovsrec_port * ovsport_lookup_by_interface( + struct ovsdb_idl_index *ovsrec_port_by_interfaces, + struct ovsrec_interface *interface) +{ + struct ovsrec_interface *interfaces[] = {interface}; + + return ovsport_lookup_by_interfaces(ovsrec_port_by_interfaces, + interfaces, 1); +} + +/* Update an interface map column with the key/value pairs present in the + * provided smap, only applying changes when necessary. */ +static void +update_interface_smap_column( + const struct ovsrec_interface *iface, + const struct smap *smap, + const struct smap *db_smap, + void (*fsetkey)(const struct ovsrec_interface *, + const char *, const char *)) +{ + struct smap_node *node; + + SMAP_FOR_EACH (node, smap) { + const char *db_value = smap_get(db_smap, node->key); + + if ((db_value && strcmp(db_value, node->value)) + || !db_value) + { + fsetkey(iface, node->key, node->value); + } + } +} + +/* Like update_interface_smap_column, but also takes an sset with all the keys + * we want to maintain. Any key present in the sset but not in the provided + * smap will be removed from the database if present there. */ +static void +maintain_interface_smap_column( + const struct ovsrec_interface *iface, + const struct sset *mnt_items, + const struct smap *smap, + const struct smap *db_smap, + void (*fsetkey)(const struct ovsrec_interface *, + const char *, const char *), + void (*fdelkey)(const struct ovsrec_interface *, + const char *)) +{ + const char *ref_name; + + SSET_FOR_EACH (ref_name, mnt_items) { + const char *value = smap_get(smap, ref_name); + const char *db_value = smap_get(db_smap, ref_name); + if (!value && db_value) { + fdelkey(iface, ref_name); + } else if ((db_value && value && strcmp(db_value, value)) + || (value && !db_value)) + { + fsetkey(iface, ref_name, value); + } + } +} diff --git a/controller/ovsport.h b/controller/ovsport.h new file mode 100644 index 000000000..e355ff7ff --- /dev/null +++ b/controller/ovsport.h @@ -0,0 +1,60 @@ +/* Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef OVSPORT_H +#define OVSPORT_H 1 + +/* OVS Ports + * ========= + * + * This module contains utility functions for adding, removing and maintaining + * ports and their interface records on OVS bridges. */ + +#include "smap.h" +#include "sset.h" + +#include +#include + +struct ovsdb_idl_txn; +struct ovsrec_bridge; +struct ovsrec_port; +struct ovsrec_interface; +struct ovsdb_idl_index; + +void ovsport_create(struct ovsdb_idl_txn *ovs_idl_txn, + const struct ovsrec_bridge *bridge, + const char *name, + const char *iface_type, + const struct smap *port_external_ids, + const struct smap *iface_external_ids, + const struct smap *iface_options, + const int64_t iface_mtu_request); +void ovsport_remove(const struct ovsrec_bridge *bridge, + const struct ovsrec_port *port); +void ovsport_update_iface(const struct ovsrec_interface *iface, + const char *type, + const struct smap *external_ids, + const struct sset *mnt_external_ids, + const struct smap *options, + const struct sset *mnt_options, + const int64_t mtu_request); +const struct ovsrec_port * ovsport_lookup_by_interfaces( + struct ovsdb_idl_index *, struct ovsrec_interface **, + const size_t n_interfaces); +const struct ovsrec_port * ovsport_lookup_by_interface( + struct ovsdb_idl_index *, struct ovsrec_interface *); + +#endif /* lib/ovsport.h */ From patchwork Fri Sep 3 19:27:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frode Nordahl X-Patchwork-Id: 1524520 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=kcABahL5; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H1SWB5VZgz9sCD for ; Sat, 4 Sep 2021 05:28:06 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id D0A7442715; Fri, 3 Sep 2021 19:28:02 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Xg4fvkDcWmdU; Fri, 3 Sep 2021 19:28:02 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id DFC7242700; Fri, 3 Sep 2021 19:28:00 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 561BAC002F; Fri, 3 Sep 2021 19:27:58 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id EAFE4C000E for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id C805B426C7 for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jVbk_y1bJiGK for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from smtp-relay-canonical-1.canonical.com (smtp-relay-canonical-1.canonical.com [185.125.188.121]) by smtp4.osuosl.org (Postfix) with ESMTPS id 29B1F426C4 for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) Received: from frode-threadripper.. (1.general.frode.uk.vpn [10.172.193.250]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id AD9124032E; Fri, 3 Sep 2021 19:27:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1630697270; bh=wfhStxVbg/X5SMKKSdhAa+c99+UoZWKNxffSDVizhvk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kcABahL52iI2ykYUQqBHXP0XPjAB3B3sV+sqglTWfB0kBBYie05igJPcg6/GAS3qb n/CRaU9XsN/vzonEHSceG6eIcEbofqVZYcJ3LOVV8e1371n3efIEiHZhHA624+0KB5 B4w5lRFpCwcMaMB81xdVuyAKteOPx8OwGfaMgAn/HzxjQdtr//1dssoS0y5Z0oQAtv 621EdBg7NJicyjcrXntqkfQlfcxl5oUct6A21wZ2ief1P/cIg63A+3wi79KIC/5+cz 1TVGY1+PQEmlVHri4xigqApLSEI0rpC48Xji4Smsq7luTq+KiPdKWXE7nbGjVIPv0X sS08rUQNnAK0w== From: Frode Nordahl To: dev@openvswitch.org Date: Fri, 3 Sep 2021 21:27:44 +0200 Message-Id: <20210903192748.1408062-6-frode.nordahl@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210903192748.1408062-1-frode.nordahl@canonical.com> References: <20210903192748.1408062-1-frode.nordahl@canonical.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v4 5/9] patch: Consume ovsport functions. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Make use of the common functions for maintaining OVS ports. Signed-off-by: Frode Nordahl --- controller/patch.c | 39 ++++++--------------------------------- 1 file changed, 6 insertions(+), 33 deletions(-) diff --git a/controller/patch.c b/controller/patch.c index a661025da..0d0d53894 100644 --- a/controller/patch.c +++ b/controller/patch.c @@ -16,6 +16,7 @@ #include #include "patch.h" +#include "ovsport.h" #include "hash.h" #include "lflow.h" @@ -91,28 +92,10 @@ create_patch_port(struct ovsdb_idl_txn *ovs_idl_txn, "ovn-controller: creating patch port '%s' from '%s' to '%s'", src_name, src->name, dst->name); - struct ovsrec_interface *iface; - iface = ovsrec_interface_insert(ovs_idl_txn); - ovsrec_interface_set_name(iface, src_name); - ovsrec_interface_set_type(iface, "patch"); - const struct smap options = SMAP_CONST1(&options, "peer", dst_name); - ovsrec_interface_set_options(iface, &options); - - struct ovsrec_port *port; - port = ovsrec_port_insert(ovs_idl_txn); - ovsrec_port_set_name(port, src_name); - ovsrec_port_set_interfaces(port, &iface, 1); - const struct smap ids = SMAP_CONST1(&ids, key, value); - ovsrec_port_set_external_ids(port, &ids); - - struct ovsrec_port **ports; - ports = xmalloc(sizeof *ports * (src->n_ports + 1)); - memcpy(ports, src->ports, sizeof *ports * src->n_ports); - ports[src->n_ports] = port; - ovsrec_bridge_verify_ports(src); - ovsrec_bridge_set_ports(src, ports, src->n_ports + 1); - - free(ports); + const struct smap if_options = SMAP_CONST1(&if_options, "peer", dst_name); + const struct smap port_ids = SMAP_CONST1(&port_ids, key, value); + ovsport_create(ovs_idl_txn, src, src_name, "patch", &port_ids, NULL, + &if_options, 0); } static void @@ -130,17 +113,7 @@ remove_port(const struct ovsrec_bridge_table *bridge_table, if (bridge->ports[i] != port) { continue; } - struct ovsrec_port **new_ports; - new_ports = xmemdup(bridge->ports, - sizeof *new_ports * (bridge->n_ports - 1)); - if (i != bridge->n_ports - 1) { - /* Removed port was not last */ - new_ports[i] = bridge->ports[bridge->n_ports - 1]; - } - ovsrec_bridge_verify_ports(bridge); - ovsrec_bridge_set_ports(bridge, new_ports, bridge->n_ports - 1); - free(new_ports); - ovsrec_port_delete(port); + ovsport_remove(bridge, port); return; } } From patchwork Fri Sep 3 19:27:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frode Nordahl X-Patchwork-Id: 1524525 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=Hl0RMO3U; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H1SWY1CRqz9sCD for ; Sat, 4 Sep 2021 05:28:24 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 9C0526166B; Fri, 3 Sep 2021 19:28:22 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id px0anL4y--f0; Fri, 3 Sep 2021 19:28:15 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id ACC6E61667; Fri, 3 Sep 2021 19:28:10 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E47EBC002D; Fri, 3 Sep 2021 19:28:02 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 01EEEC002A for ; Fri, 3 Sep 2021 19:27:57 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id E402B6157B for ; Fri, 3 Sep 2021 19:27:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HHltLCZAgUs7 for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from smtp-relay-canonical-1.canonical.com (smtp-relay-canonical-1.canonical.com [185.125.188.121]) by smtp3.osuosl.org (Postfix) with ESMTPS id 0FB8361542 for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) Received: from frode-threadripper.. (1.general.frode.uk.vpn [10.172.193.250]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id 0A6C940666; Fri, 3 Sep 2021 19:27:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1630697271; bh=hlQb/VRq+wyXqDTO6J1oh+L1ZewUBD+Bi+uRj2KnStU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Hl0RMO3UeDHdH0d5/CywWEsKuPF5Xsy1cbhx7nUqo55NE0YP+SenZnJgCCPywL7yU a979BDNfh8to7sFZqSMl9I0r0WJFxdQ8QJ2/yHDKIV9JE3hYdCQ3wgCWib8gs/UVBR O/S5piIu4U65B5QJzJBX8X7spcOswSVeAgmSvTYEdatOu8rAcQhBTxgDavC89quZXS SYqptQNwy1zF5rhArvYg/U+NQLsUagFLh5X3VkJ6muXwqJ9wnaSOvG6xhIEUZaCgr+ A8Zrb6SdRbTJeW0nOazvoCt3+XntEopVn0Yorip2v/gzx+jfjJ6arjlnToWYRzJeeF TMqPncWg6AO1Q== From: Frode Nordahl To: dev@openvswitch.org Date: Fri, 3 Sep 2021 21:27:45 +0200 Message-Id: <20210903192748.1408062-7-frode.nordahl@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210903192748.1408062-1-frode.nordahl@canonical.com> References: <20210903192748.1408062-1-frode.nordahl@canonical.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v4 6/9] lib: Add infrastructure for plugging providers. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" New module contains the infrastructure for registering plugging classes which may be hosted inside or outside the core OVN repository. The data structures and functions for interacting with these plugging classes also live there. Extend build system to allow enabling building of built-in plugging providers and linking an externally built plugging provider. Signed-off-by: Frode Nordahl --- Documentation/automake.mk | 1 + Documentation/topics/index.rst | 1 + Documentation/topics/plug_providers/index.rst | 32 +++ .../topics/plug_providers/plug-providers.rst | 199 ++++++++++++++ acinclude.m4 | 49 ++++ configure.ac | 2 + lib/automake.mk | 12 +- lib/plug-dummy.c | 123 +++++++++ lib/plug-dummy.h | 33 +++ lib/plug-provider.h | 98 +++++++ lib/plug.c | 255 ++++++++++++++++++ lib/plug.h | 107 ++++++++ lib/test-plug.c | 72 +++++ ovn-architecture.7.xml | 35 ++- tests/automake.mk | 4 +- tests/ovn-plug.at | 8 + 16 files changed, 1016 insertions(+), 15 deletions(-) create mode 100644 Documentation/topics/plug_providers/index.rst create mode 100644 Documentation/topics/plug_providers/plug-providers.rst create mode 100644 lib/plug-dummy.c create mode 100644 lib/plug-dummy.h create mode 100644 lib/plug-provider.h create mode 100644 lib/plug.c create mode 100644 lib/plug.h create mode 100644 lib/test-plug.c create mode 100644 tests/ovn-plug.at diff --git a/Documentation/automake.mk b/Documentation/automake.mk index b3fd3d62b..92a843d76 100644 --- a/Documentation/automake.mk +++ b/Documentation/automake.mk @@ -28,6 +28,7 @@ DOC_SOURCE = \ Documentation/topics/ovn-news-2.8.rst \ Documentation/topics/role-based-access-control.rst \ Documentation/topics/debugging-ddlog.rst \ + Documentation/topics/plug_providers/plug-providers.rst \ Documentation/howto/index.rst \ Documentation/howto/docker.rst \ Documentation/howto/firewalld.rst \ diff --git a/Documentation/topics/index.rst b/Documentation/topics/index.rst index d58d5618b..12bd113b7 100644 --- a/Documentation/topics/index.rst +++ b/Documentation/topics/index.rst @@ -41,6 +41,7 @@ OVN high-availability role-based-access-control ovn-news-2.8 + plug_providers/index testing .. list-table:: diff --git a/Documentation/topics/plug_providers/index.rst b/Documentation/topics/plug_providers/index.rst new file mode 100644 index 000000000..837eeae15 --- /dev/null +++ b/Documentation/topics/plug_providers/index.rst @@ -0,0 +1,32 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in OVN documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +============== +Plug Providers +============== + + +.. toctree:: + :maxdepth: 2 + + plug-providers diff --git a/Documentation/topics/plug_providers/plug-providers.rst b/Documentation/topics/plug_providers/plug-providers.rst new file mode 100644 index 000000000..7b891156c --- /dev/null +++ b/Documentation/topics/plug_providers/plug-providers.rst @@ -0,0 +1,199 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in OVN documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +================== +Plugging Providers +================== + +Traditionally it has been the CMSes responsibility to create VIFs as part of +instance life cycle, and subsequently manage plug/unplug operations on the +integration bridge following the conventions described in the +`Open vSwitch Integration Guide`_ for mapping of VIFs to OVN logical port. + +With the advent of NICs connected to multiple distinct CPUs we can have a +topology where the instance runs on one host and Open vSwitch and OVN runs on +a different host, the smartnic control plane CPU. The host facing interfaces +will be visible to Open vSwitch and OVN as representor ports. + +The actions necessary for plugging and unplugging the representor port in +Open vSwitch running on the smartnic control plane CPU would be the same for +every CMS. + +Instead of every CMS having to develop their own version of an agent to do +the plugging, we provide a pluggable infrastructure in OVN that allows the +`ovn-controller` to perform the plugging on CMS direction. + +Hardware or platform specific details for initialization and lookup of +representor ports is provided by an plugging provider library hosted inside or +outside the core OVN repository, and linked at OVN build time. + +Life Cycle of an OVN plugged VIF +-------------------------------- + +1. CMS creates a record in the OVN Northbound Logical_Switch_Port table with + the options column containing the `plug-type` key with a value corresponding + to the `const char *type` provided by the plugging provider implementation + as well as a `requested-chassis` key with a value pointing at the name or + hostname of the chassis it wants the VIF plugged on. Additional plugging + provider specific key/value pairs must be provided for successful lookup. + +2. `ovn-northd` looks up the name or hostname provided in the + `requested-chassis` option and fills the OVN Southbound Port_Binding + requested_chassis column, it also copies relevant options over to the + Port_Binding record. + +3. `ovn-controller` monitors Southbound Port_Binding entries wth a + requested_chassis column pointing at its chassis UUID and when it encounters + a entry with option `plug-type` and it has registered a plug provider + matching that type it will act on it even if no local binding exists yet. + +4. It will fill the `struct plug_port_ctx_in` as defined in `lib/plug.h` with + `op_type` set to 'PLUG_OP_CREATE' and make a call to the plug providers + `plug_port_prepare` callback function. Plug provider performs lookup and + fills the `struct plug_port_ctx_out` as defined in `lib/plug.h`. + +5. `ovn-controller` creates a port and interface record in the local OVSDB + using the details provided by the plug provider and also adds + `external-ids:iface-id` with value matching the logical port name and + `external-ids:ovn-plugged` with value matching the logical port `plug-type`. + When the port creation is done a call will first be made to the plug + providers `plug_port_finish` function and then to the + `plug_port_ctx_destroy` function to free any memory allocated by the plug + implementation. + +6. The Open vSwitch vswitchd will assign a ofport to the newly created + interface and on the next `ovn-controller` main loop iteration flows will be + installed. + +7. On any change to the Southbound Port_Binding record or full recomputation + the `ovn-controller` will in addition to normal flow processing make a call + to the plug provider again similar to the first creation in case anything + needs updating for the interface record. + +8. The port will be unplugged when an event occurs which would make the + `ovn-controller` release a logical port, for example the Logical_Switch_Port + and Port_Binding entry disappearing from the database or its + `requested_chassis` column being pointed to a different chassis. + + +The plug provider interface +--------------------------- + +The interface between internals of OVN and a plugging provider is a set of +callbacks as defined by the `struct plug_class` in `lib/plug-provider.h`. + +It is important to note that these callbacks will be called in the critical +path of the `ovn-controller` processing loop, so care must be taken to make the +implementation as efficient as possible, and under no circumstance can any of +the callback functions make calls that block. + +On `ovn-controller` startup, plug providers made available at build time will +be registered by the identifier provided in the `const char *type` pointer, at +this time the `init` function pointer will be called if it is non-NULL. + +> **Note**: apart from the `const char *type` pointer, no attempt will be made +to access plug provider data or functions before the call to the `init` has +been made. + +On `ovn-controller` exit, the plug providers registered in the above mentioned +procedure will have their `destroy` function pointer called if it is non-NULL. + +If the plug provider has internal lookup tables that need to be maintained they +can define a `run` function which will be called as part of the +`ovn-controller` incremental processing engine loop. If the changes +encountered can be safely processed incrementally the function should return +'true', if the changes cannot be processed incrementally, for example because +the change would impact already handled interfaces that we might not process in +a while, the function should return 'false' to force the `ovn-controller` to +perform a full recomputation. + +On update of Interface records the `ovn-controller` will pass on a `sset` +to the `ovsport_update_iface` function containing options the plug +implementation finds pertinent to maintain for successful operation. This +`sset` is retrieved by making a call to the plug implementation +`plug_get_maintained_iface_options` function pointer if it is non-NULL. This +allows presence of other users of the OVSDB maintaining a different set of +options on the same set of Interface records without wiping out their changes. + +Before creating or updating an existing interface record the plug provider +`plug_port_prepare` function pointer will be called with valid pointers to +`struct plug_port_ctx_in` and `struct plug_port_ctx_out` data structures. If +the plug provider implementation is able to perform lookup it should fill the +`struct plug_port_ctx_out` data structure and return 'true'. The +`ovn-controller` will then create or update the port/interface records and +then call `plug_port_finish` and `plug_port_ctx_destroy`. If the plug provider +implementation is unable to perform lookup or prepare the desired resource it +should return 'false' which will tell the `ovn-controller` to not create or +update the port, in this case it will also not call `plug_port_finish`, it will +however make a call to `plug_port_ctx_destroy`. + +Before removing port and interface records previously plugged by the +`ovn-controller` as identified by presence of the Interface +`external-ids:ovn-plugged` key, the `ovn-controller` will look up the +`plug-type` from `external-ids:ovn-plugged`, fill `struct plug_port_ctx_in` +with `op_type` set to 'PLUG_OP_REMOVE' and make a call to `plug_port_prepare`. +After the port and interface has been removed a call will be made to +`plug_port_finish`. Both calls will be made with the pointer to +`plug_port_ctx_out` set to 'NULL', and no call will be made to +`plug_port_ctx_destroy`. + +Building with in-tree plugging providers +---------------------------------------- + +Plugging providers hosted in the OVN repository living under +`lib/plug_providers`: + +To enable them, provide the `--enable-plug-providers` command line option to +the configure script when building OVN. + +Building with an externally provided plugging provider +------------------------------------------------------ + +There is also infrastructure in place to support linking OVN with an externally +built plugging provider. + +This external plugging provider must define a NULL-terminated array of pointers +to `struct plug_class` data structures named `plug_provider_classes`. Example: + +.. code-block:: C + + const struct plug_class *plug_provider_classes[] = { + &plug_foo, + NULL, + }; + +The name of the repostiroy for the external plugging provider should be the +same as the name of the library it produces, and the built library artifact +should be placed in lib/.libs. Example: + +.. code-block:: none + + ovn-vif-foo/ + ovn-vif-foo/lib/.libs/libovn-vif-foo.la + +To enable such a plugging provider provide the +`--with-plug-provider=/path/to/ovn-vif-foo` command line option to the +configure script when building OVN. + +.. LINKS +.. _Open vSwitch Integration Guide: https://docs.openvswitch.org/en/latest/topics/integration/ diff --git a/acinclude.m4 b/acinclude.m4 index e7f829520..793a073d1 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -441,3 +441,52 @@ AC_DEFUN([OVN_CHECK_OVS], [ AC_MSG_CHECKING([OVS version]) AC_MSG_RESULT([$OVSVERSION]) ]) + +dnl OVN_CHECK_PLUG_PROVIDER +dnl +dnl Check for external plug provider +AC_DEFUN([OVN_CHECK_PLUG_PROVIDER], [ + AC_ARG_VAR([PLUG_PROVIDER]) + AC_ARG_WITH( + [plug-provider], + [AC_HELP_STRING([--with-plug-provider=/path/to/provider/repository], + [Specify path to a configured and built plug provider repository])], + [if test "$withval" = yes; then + if test -z "$PLUG_PROVIDER"; then + AC_MSG_ERROR([To build with external plug provider, specify the path to a configured and built plug provider repository --with-plug-provider or in \$PLUG_PROVIDER]), + fi + PLUG_PROVIDER="$(realpath $PLUG_PROVIDER)" + else + PLUG_PROVIDER="$(realpath $withval)" + fi + _plug_provider_name="$(basename $PLUG_PROVIDER)" + if test ! -f "$PLUG_PROVIDER/lib/.libs/lib${_plug_provider_name}.la"; then + AC_MSG_ERROR([$withval is not a configured and built plug provider library repository]) + fi + PLUG_PROVIDER_LDFLAGS="-L$PLUG_PROVIDER/lib/.libs -l$_plug_provider_name" + ], + [PLUG_PROVIDER=no]) + AC_MSG_CHECKING([for plug provider]) + AC_MSG_RESULT([$PLUG_PROVIDER]) + AC_SUBST([PLUG_PROVIDER_LDFLAGS]) + AM_CONDITIONAL([HAVE_PLUG_PROVIDER], [test "$PLUG_PROVIDER" != no]) + if test "$PLUG_PROVIDER" != no; then + AC_DEFINE([HAVE_PLUG_PROVIDER], [1], + [Build and link with external plug provider]) + fi +]) + +dnl OVN_ENABLE_PLUG +dnl +dnl Enable built-in plug providers +AC_DEFUN([OVN_ENABLE_PLUG], [ + AC_ARG_ENABLE( + [plug-providers], + [AC_HELP_STRING([--enable-plug-providers], [Enable building of built-in plug providers])], + [], [enable_plug=no]) + AM_CONDITIONAL([ENABLE_PLUG], [test "$enable_plug" != no]) + if test "$enable_plug" != no; then + AC_DEFINE([ENABLE_PLUG], [1], + [Build built-in plug providers]) + fi +]) diff --git a/configure.ac b/configure.ac index df0b98295..7f3274e59 100644 --- a/configure.ac +++ b/configure.ac @@ -172,6 +172,8 @@ OVS_ENABLE_SPARSE OVS_CHECK_DDLOG([0.38]) OVS_CHECK_PRAGMA_MESSAGE OVN_CHECK_OVS +OVN_CHECK_PLUG_PROVIDER +OVN_ENABLE_PLUG OVS_CTAGS_IDENTIFIERS AC_SUBST([OVS_CFLAGS]) AC_SUBST([OVS_LDFLAGS]) diff --git a/lib/automake.mk b/lib/automake.mk index ddfe33948..086fbd62d 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -3,6 +3,11 @@ lib_libovn_la_LDFLAGS = \ $(OVS_LTINFO) \ -Wl,--version-script=$(top_builddir)/lib/libovn.sym \ $(AM_LDFLAGS) + +if HAVE_PLUG_PROVIDER +lib_libovn_la_LDFLAGS += $(PLUG_PROVIDER_LDFLAGS) +endif + lib_libovn_la_SOURCES = \ lib/acl-log.c \ lib/acl-log.h \ @@ -32,7 +37,12 @@ lib_libovn_la_SOURCES = \ lib/inc-proc-eng.h \ lib/lb.c \ lib/lb.h \ - lib/stopwatch-names.h + lib/stopwatch-names.h \ + lib/plug-provider.h \ + lib/plug.h \ + lib/plug.c \ + lib/plug-dummy.h \ + lib/plug-dummy.c nodist_lib_libovn_la_SOURCES = \ lib/ovn-dirs.c \ lib/ovn-nb-idl.c \ diff --git a/lib/plug-dummy.c b/lib/plug-dummy.c new file mode 100644 index 000000000..505c37c9c --- /dev/null +++ b/lib/plug-dummy.c @@ -0,0 +1,123 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include "plug-dummy.h" +#include "plug-provider.h" +#include "plug.h" + +#include + +#include "openvswitch/vlog.h" +#include "smap.h" +#include "sset.h" + +#ifndef IFNAMSIZ +#define IFNAMSIZ 16 +#endif + +VLOG_DEFINE_THIS_MODULE(plug_dummy); + +static struct sset plug_dummy_maintained_iface_options; + +static int +plug_dummy_init(void) +{ + sset_init(&plug_dummy_maintained_iface_options); + sset_add(&plug_dummy_maintained_iface_options, "plug-dummy-option"); + + return 0; +} + +static int +plug_dummy_destroy(void) +{ + sset_destroy(&plug_dummy_maintained_iface_options); + + return 0; +} + +static const struct sset* +plug_dummy_get_maintained_iface_options(void) +{ + return &plug_dummy_maintained_iface_options; +} + +static bool +plug_dummy_run(struct plug_class *plug) +{ + VLOG_DBG("plug_dummy_run(%p)", plug); + + return true; +} + +static bool +plug_dummy_port_prepare(const struct plug_port_ctx_in *ctx_in, + struct plug_port_ctx_out *ctx_out) +{ + VLOG_DBG("plug_dummy_port_prepare: %s", ctx_in->lport_name); + + if (ctx_in->op_type == PLUG_OP_CREATE) { + size_t lport_name_len = strlen(ctx_in->lport_name); + ctx_out->name = xzalloc(IFNAMSIZ); + memcpy(ctx_out->name, ctx_in->lport_name, + (lport_name_len < IFNAMSIZ) ? lport_name_len : IFNAMSIZ - 1); + ctx_out->type = xstrdup("internal"); + ctx_out->iface_options = xmalloc(sizeof *ctx_out->iface_options); + smap_init(ctx_out->iface_options); + smap_add(ctx_out->iface_options, "plug-dummy-option", "value"); + } + + return true; +} + +static void +plug_dummy_port_finish(const struct plug_port_ctx_in *ctx_in, + struct plug_port_ctx_out *ctx_out OVS_UNUSED) +{ + VLOG_DBG("plug_dummy_port_finish: %s", ctx_in->lport_name); +} + +static void +plug_dummy_port_ctx_destroy(const struct plug_port_ctx_in *ctx_in, + struct plug_port_ctx_out *ctx_out) +{ + VLOG_DBG("plug_dummy_port_ctx_destroy: %s", ctx_in->lport_name); + ovs_assert(ctx_in->op_type == PLUG_OP_CREATE); + free(ctx_out->name); + free(ctx_out->type); + smap_destroy(ctx_out->iface_options); + free(ctx_out->iface_options); +} + +const struct plug_class plug_dummy_class = { + .type = "dummy", + .init = plug_dummy_init, + .destroy = plug_dummy_destroy, + .plug_get_maintained_iface_options = + plug_dummy_get_maintained_iface_options, + .run = plug_dummy_run, + .plug_port_prepare = plug_dummy_port_prepare, + .plug_port_finish = plug_dummy_port_finish, + .plug_port_ctx_destroy = plug_dummy_port_ctx_destroy, +}; + +void +plug_dummy_enable(void) +{ + plug_register_provider(&plug_dummy_class); +} + diff --git a/lib/plug-dummy.h b/lib/plug-dummy.h new file mode 100644 index 000000000..6ea33671e --- /dev/null +++ b/lib/plug-dummy.h @@ -0,0 +1,33 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef PLUG_DUMMY_H +#define PLUG_DUMMY_H 1 + +/* + * The dummy plugger, allows for experimenting with plugging in a sandbox */ + +#ifdef __cplusplus +extern "C" { +#endif + +void plug_dummy_enable(void); + +#ifdef __cplusplus +} +#endif + +#endif /* plug-dummy.h */ diff --git a/lib/plug-provider.h b/lib/plug-provider.h new file mode 100644 index 000000000..487534ee5 --- /dev/null +++ b/lib/plug-provider.h @@ -0,0 +1,98 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef PLUG_PROVIDER_H +#define PLUG_PROVIDER_H 1 + +/* Provider interface to pluggers. A plugger implementation performs lookup + * and/or initialization of ports, typically representor ports, using generic + * non-blocking hardware interfaces. This allows the ovn-controller to, upon + * the CMS's request, create ports and interfaces in the chassis's Open vSwitch + * instances (also known as vif plugging). + */ + +#include + +#include "plug.h" + +#ifdef __cplusplus +extern "C" { +#endif + +struct plug_class { + /* Type of plugger in this class. */ + const char *type; + + /* Called when the plug provider is registered, typically at program + * startup. + * + * This function may be set to null if a plug class needs no + * initialization at registration time. */ + int (*init)(void); + + /* Called when the plug provider is unregistered, typically at program + * exit. + * + * This function may be set to null if a plug class needs no + * de-initialization at unregister time.*/ + int (*destroy)(void); + + /* Performs periodic work needed by plugger, if any is necessary. Returns + * 'true; if the changes encountered could be handled incrementally, + * 'false' otherwise. + * + * Note that returning 'false' will instruct the incremental processing + * engine to perform a full recomputation. */ + bool (*run)(struct plug_class *); + + /* Retrieve Interface options this plugger will maintain. This set is used + * to know which items to remove when maintaining the database record. */ + const struct sset * (*plug_get_maintained_iface_options)(void); + + /* Pass plug_port_ctx_in to plug implementation to prepare for port + * creation/update. + * + * The plug implemantation can perform lookup or any per port + * initialization and should fill plug_port_ctx_out with data required for + * port/interface creation. The plug implementation should return true if + * it wants the caller to create/update a port/interface, false otherwise. + * + * Data in the plug_port_ctx_out struct is owned by the plugging library, + * and a call must be made to the plug_port_ctx_destroy callback to free + * up any allocations when done with port creation/update. + */ + bool (*plug_port_prepare)(const struct plug_port_ctx_in *, + struct plug_port_ctx_out *); + + /* Notify plugging library that port update is done. */ + void (*plug_port_finish)(const struct plug_port_ctx_in *, + struct plug_port_ctx_out *); + + /* Free any allocations made by the plug_port_prepare callback. */ + void (*plug_port_ctx_destroy)(const struct plug_port_ctx_in *, + struct plug_port_ctx_out *); +}; + +extern const struct plug_class plug_dummy_class; +#ifdef HAVE_PLUG_PROVIDER +extern const struct plug_class *plug_provider_classes[]; +#endif + +#ifdef __cplusplus +} +#endif + +#endif /* plug-provider.h */ diff --git a/lib/plug.c b/lib/plug.c new file mode 100644 index 000000000..c0c34214e --- /dev/null +++ b/lib/plug.c @@ -0,0 +1,255 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include "plug-provider.h" +#include "plug.h" + +#include +#include +#include + +#include "openvswitch/vlog.h" +#include "openvswitch/shash.h" +#include "smap.h" +#include "sset.h" +#include "lib/inc-proc-eng.h" + +VLOG_DEFINE_THIS_MODULE(plug); + +#ifdef ENABLE_PLUG +static const struct plug_class *base_plug_classes[] = { +}; +#endif + +static struct shash plug_classes = SHASH_INITIALIZER(&plug_classes); + +/* Protects the 'plug_classes' shash. */ +static struct ovs_mutex plug_classes_mutex = OVS_MUTEX_INITIALIZER; + +/* Initialize the the plug infrastructure by registering known plug classes */ +void +plug_initialize(void) +{ + static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; + + if (ovsthread_once_start(&once)) { +#ifdef ENABLE_PLUG + /* Register built-in plug provider classes */ + for (int i = 0; i < ARRAY_SIZE(base_plug_classes); i++) { + plug_register_provider(base_plug_classes[i]); + } +#endif +#ifdef HAVE_PLUG_PROVIDER + /* Register external plug provider classes. + * + * Note that we cannot use the ARRAY_SIZE macro here as + * plug_provider_classes is defined in external code which is not + * available at compile time. The convention is to use a + * NULL-terminated array instead. */ + for (const struct plug_class **pp = plug_provider_classes; + pp && *pp; + pp++) + { + plug_register_provider(*pp); + } +#endif + ovsthread_once_done(&once); + } +} + +static int +plug_register_provider__(const struct plug_class *new_class) +{ + struct plug_class *plug_class; + int error; + + if (shash_find(&plug_classes, new_class->type)) { + VLOG_WARN("attempted to register duplicate plug provider: %s", + new_class->type); + return EEXIST; + } + + error = new_class->init ? new_class->init() : 0; + if (error) { + VLOG_WARN("failed to initialize %s plug class: %s", + new_class->type, ovs_strerror(error)); + return error; + } + + plug_class = xmalloc(sizeof *plug_class); + memcpy(plug_class, new_class, sizeof *plug_class); + + shash_add(&plug_classes, new_class->type, plug_class); + + return 0; +} + +/* Register the new plug provider referred to in 'new_class' and perform any + * class level initialization as specified in its plug_class. */ +int +plug_register_provider(const struct plug_class *new_class) +{ + int error; + + ovs_mutex_lock(&plug_classes_mutex); + error = plug_register_provider__(new_class); + ovs_mutex_unlock(&plug_classes_mutex); + + return error; +} + +static int +plug_unregister_provider__(const char *type) +{ + int error; + struct shash_node *node; + struct plug_class *plug_class; + + node = shash_find(&plug_classes, type); + if (!node) { + return EINVAL; + } + + plug_class = node->data; + error = plug_class->destroy ? plug_class->destroy() : 0; + if (error) { + VLOG_WARN("failed to destroy %s plug class: %s", + plug_class->type, ovs_strerror(error)); + return error; + } + + shash_delete(&plug_classes, node); + free(plug_class); + + return 0; +} + +/* Unregister the plug provider identified by 'type' and perform any class + * level de-initialization as specified in its plug_class. */ +int +plug_unregister_provider(const char *type) +{ + int error; + + ovs_mutex_lock(&plug_classes_mutex); + error = plug_unregister_provider__(type); + ovs_mutex_unlock(&plug_classes_mutex); + + return error; +} + +const struct plug_class * +plug_get_provider(const char *type) +{ + struct plug_class *plug_class; + + ovs_mutex_lock(&plug_classes_mutex); + plug_class = shash_find_data(&plug_classes, type); + ovs_mutex_unlock(&plug_classes_mutex); + + return plug_class; +} + +/* De-initialize and unregister the plug provider classes. */ +void +plug_destroy_all(void) +{ + struct shash_node *node, *next; + + SHASH_FOR_EACH_SAFE (node, next, &plug_classes) { + struct plug_class *plug_class = node->data; + plug_unregister_provider(plug_class->type); + } +} + +/* Get the class level 'maintained_iface_options' set. */ +const struct sset * +plug_get_maintained_iface_options(const struct plug_class *plug_class) +{ + return plug_class->plug_get_maintained_iface_options(); +} + +/* Prepare the logical port as identified by 'ctx_in' for port creation, update + * or removal as specified by 'ctx_in->op_type'. + * + * When 'ctx_in->op_type' is PLUG_OP_CREATE the plug implementation must fill + * 'ctx_out' with data to apply to the interface record maintained by OVN on + * its behalf. + * + * When 'ctx_in_op_type' is PLUG_OP_REMOVE 'ctx_out' should be set to NULL and + * the plug implementation must not attempt to use 'ctx_out'. + * + * The data in 'ctx_out' is owned by the plug implementation, and a call must + * be made to plug_port_ctx_destroy when done with it. */ +bool +plug_port_prepare(const struct plug_class *plug_class, + const struct plug_port_ctx_in *ctx_in, + struct plug_port_ctx_out *ctx_out) +{ + if (ctx_out) { + memset(ctx_out, 0, sizeof(*ctx_out)); + } + return plug_class->plug_port_prepare(ctx_in, ctx_out); +} + +/* Notify the plug implementation that a port creation, update or removal has + * been completed */ +void +plug_port_finish(const struct plug_class *plug_class, + const struct plug_port_ctx_in *ctx_in, + struct plug_port_ctx_out *ctx_out) +{ + plug_class->plug_port_finish(ctx_in, ctx_out); +} + +/* Free any data allocated to 'ctx_out' in a prevous call to + * plug_port_prepare. */ +void +plug_port_ctx_destroy(const struct plug_class *plug_class, + const struct plug_port_ctx_in *ctx_in, + struct plug_port_ctx_out *ctx_out) +{ + plug_class->plug_port_ctx_destroy(ctx_in, ctx_out); +} + +/* Iterate over registered plug provider classes and call their 'run' + * function if defined. + * + * If any of the classes report that something has changed we will trigger a + * recompute. */ +void +en_plug_provider_run(struct engine_node *inc_eng_node, + void *inc_eng_data OVS_UNUSED) +{ + struct shash_node *node, *next; + bool handled = true; + + SHASH_FOR_EACH_SAFE (node, next, &plug_classes) { + struct plug_class *plug_class = node->data; + if (plug_class->run && !plug_class->run(plug_class)) { + handled = false; + } + } + + if (!handled) { + /* as we do not have a change handler registered in the incremental + * processing engine this will trigger a recompute. */ + engine_set_node_state(inc_eng_node, EN_UPDATED); + } else { + engine_set_node_state(inc_eng_node, EN_UNCHANGED); + } +} diff --git a/lib/plug.h b/lib/plug.h new file mode 100644 index 000000000..fef4c7f64 --- /dev/null +++ b/lib/plug.h @@ -0,0 +1,107 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef PLUG_H +#define PLUG_H 1 + +/* + * Plug, the plugging interface. This module contains the infrastructure for + * registering and instantiating plugging classes which may be hosted inside + * or outside the core OVN repository. The data structures and functions for + * interacting with these plugging classes also live here. + */ + +#include "smap.h" + +#ifdef __cplusplus +extern "C" { +#endif + +struct plug_class; +struct ovsdb_idl_txn; +struct ovsrec_bridge; + +enum plug_op_type { + PLUG_OP_CREATE = 1, /* Port is created or updated */ + PLUG_OP_REMOVE, /* Port is removed from this chassis */ +}; + +struct plug_port_ctx_in { + /* Operation being performed */ + enum plug_op_type op_type; + + /* These are provided so that the plug implementation may make decisions + * based on environmental factors such as settings in the open-vswitch + * table and datapath type settings on the integration bridge. */ + const struct ovsrec_open_vswitch_table *ovs_table; + const struct ovsrec_bridge *br_int; + + /* Name of logical port, can be useful for plugging library to track any + * per port resource initialization. */ + const char *lport_name; + + /* Logical port options, while OVN will forward the contents verbatim from + * the Southbound database, the convention is for the plugging library to + * only make decisions based on the plug-* options. */ + const struct smap *lport_options; + + /* When OVN knows about an existing interface record associated with this + * lport, these will be filled in with information about it. */ + const char *iface_name; + const char *iface_type; + const struct smap *iface_options; +}; + +struct plug_port_ctx_out { + /* The name to use for port and interface record. */ + char *name; + + /* Type of interface to create. */ + char *type; + + /* Options to set on the interface record. */ + struct smap *iface_options; +}; + + +void plug_initialize(void); +int plug_register_provider(const struct plug_class *); +int plug_unregister_provider(const char *type); +void plug_destroy_all(void); +const struct plug_class * plug_get_provider(const char *); + +const struct sset * plug_get_maintained_iface_options( + const struct plug_class *plug_class); + +bool plug_port_prepare(const struct plug_class *, + const struct plug_port_ctx_in *, + struct plug_port_ctx_out *); +void plug_port_finish(const struct plug_class *, + const struct plug_port_ctx_in *, + struct plug_port_ctx_out *); +void plug_port_ctx_destroy(const struct plug_class *, + const struct plug_port_ctx_in *, + struct plug_port_ctx_out *); + +struct engine_node; + +void en_plug_provider_run(struct engine_node *, void *); + +#ifdef __cplusplus +} +#endif + +#endif /* plug.h */ diff --git a/lib/test-plug.c b/lib/test-plug.c new file mode 100644 index 000000000..bddce23c4 --- /dev/null +++ b/lib/test-plug.c @@ -0,0 +1,72 @@ +/* Copyright (c) 2021, Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include + +#include "plug.h" +#include "plug-dummy.h" +#include "plug-provider.h" +#include "smap.h" +#include "sset.h" +#include "tests/ovstest.h" + +static void +test_plug(struct ovs_cmdl_context *ctx OVS_UNUSED) +{ + const struct plug_class *plug_class; + + ovs_assert(plug_unregister_provider("dummy") == EINVAL); + + ovs_assert(!plug_register_provider(&plug_dummy_class)); + plug_class = plug_get_provider("dummy"); + ovs_assert(plug_register_provider(&plug_dummy_class) == EEXIST); + + ovs_assert(sset_contains(plug_get_maintained_iface_options(plug_class), + "plug-dummy-option")); + + struct smap fake_lport_options = SMAP_INITIALIZER(&fake_lport_options); + struct plug_port_ctx_in ctx_in = { + .op_type = PLUG_OP_CREATE, + .lport_name = "lsp1", + .lport_options = &fake_lport_options, + }; + struct plug_port_ctx_out ctx_out; + plug_port_prepare(plug_class, &ctx_in, &ctx_out); + ovs_assert(!strcmp(ctx_out.name, "lsp1")); + ovs_assert(!strcmp(ctx_out.type, "internal")); + ovs_assert(!strcmp(smap_get( + ctx_out.iface_options, "plug-dummy-option"), "value")); + + plug_port_finish(plug_class, &ctx_in, &ctx_out); + plug_port_ctx_destroy(plug_class, &ctx_in, &ctx_out); + plug_destroy_all(); +} + +static void +test_plug_main(int argc, char *argv[]) +{ + set_program_name(argv[0]); + static const struct ovs_cmdl_command commands[] = { + {"run", NULL, 0, 0, test_plug, OVS_RO}, + {NULL, NULL, 0, 0, NULL, OVS_RO}, + }; + struct ovs_cmdl_context ctx; + ctx.argc = argc - 1; + ctx.argv = argv + 1; + ovs_cmdl_run_command(&ctx, commands); +} + +OVSTEST_REGISTER("test-plug", test_plug_main); diff --git a/ovn-architecture.7.xml b/ovn-architecture.7.xml index 3d2910358..152367a89 100644 --- a/ovn-architecture.7.xml +++ b/ovn-architecture.7.xml @@ -67,8 +67,9 @@
  • One or more (usually many) hypervisors. Hypervisors must run Open vSwitch and implement the interface described in - Documentation/topics/integration.rst in the OVN source tree. - Any hypervisor platform supported by Open vSwitch is acceptable. + Documentation/topics/integration.rst in the Open vSwitch + source tree. Any hypervisor platform supported by Open vSwitch is + acceptable.
  • @@ -318,11 +319,19 @@
  • On a hypervisor, any VIFs that are to be attached to logical networks. - The hypervisor itself, or the integration between Open vSwitch and the - hypervisor (described in - Documentation/topics/integration.rst) takes care of this. - (This is not part of OVN or new to OVN; this is pre-existing integration - work that has already been done on hypervisors that support OVS.) + For instances connected through software emulated ports such as TUN/TAP + or VETH pairs, the hypervisor itself will normally create ports and plug + them into the integration bridge. For instances connected through + representor ports, typically used with hardware offload, the + ovn-controller may on CMS direction consult a plugging + provider library for representor port lookup and plug them into the + integration bridge (please refer to + Documentation/topics/plugging-providers.rst for more + information). In both cases the conventions described in + Documentation/topics/integration.rst in the Open vSwitch + source tree is followed to ensure mapping between OVN logical port and + VIF. (This is pre-existing integration work that has already been done + on hypervisors that support OVS.)
  • @@ -921,12 +930,12 @@ Eventually, a user powers on the VM that owns the VIF. On the hypervisor where the VM is powered on, the integration between the hypervisor and Open vSwitch (described in - Documentation/topics/integration.rst) adds the VIF to the OVN - integration bridge and stores vif-id in - external_ids:iface-id to indicate that the - interface is an instantiation of the new VIF. (None of this code is new - in OVN; this is pre-existing integration work that has already been done - on hypervisors that support OVS.) + Documentation/topics/integration.rst in the Open vSwitch + source tree) adds the VIF to the OVN integration bridge and stores + vif-id in external_ids:iface-id to + indicate that the interface is an instantiation of the new VIF. (None of + this code is new in OVN; this is pre-existing integration work that has + already been done on hypervisors that support OVS.)
  • diff --git a/tests/automake.mk b/tests/automake.mk index 5b890d644..ad8978541 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -38,7 +38,8 @@ TESTSUITE_AT = \ tests/ovn-ipam.at \ tests/ovn-features.at \ tests/ovn-lflow-cache.at \ - tests/ovn-ipsec.at + tests/ovn-ipsec.at \ + tests/ovn-plug.at SYSTEM_KMOD_TESTSUITE_AT = \ tests/system-common-macros.at \ @@ -248,6 +249,7 @@ tests_ovstest_SOURCES = \ controller/ofctrl-seqno.c \ controller/ofctrl-seqno.h \ lib/test-ovn-features.c \ + lib/test-plug.c \ northd/test-ipam.c \ northd/ipam.c \ northd/ipam.h diff --git a/tests/ovn-plug.at b/tests/ovn-plug.at new file mode 100644 index 000000000..d5c6a1b6d --- /dev/null +++ b/tests/ovn-plug.at @@ -0,0 +1,8 @@ +# +# Unit tests for the lib/plug.c module. +# +AT_BANNER([OVN unit tests - plug]) + +AT_SETUP([unit test -- plugging infrastructure tests]) +AT_CHECK([ovstest test-plug run], [0], []) +AT_CLEANUP From patchwork Fri Sep 3 19:27:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frode Nordahl X-Patchwork-Id: 1524521 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=vrybp9eW; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H1SWG4tNhz9sCD for ; Sat, 4 Sep 2021 05:28:10 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id CCA6F40867; Fri, 3 Sep 2021 19:28:07 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZN-AdvsWDcoJ; Fri, 3 Sep 2021 19:28:05 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 2DC4840813; Fri, 3 Sep 2021 19:28:04 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id BBB5FC0035; Fri, 3 Sep 2021 19:27:59 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8A23EC001A for ; Fri, 3 Sep 2021 19:27:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id AEB8E6158D for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=canonical.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Qi4tFYDL_hyj for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from smtp-relay-canonical-1.canonical.com (smtp-relay-canonical-1.canonical.com [185.125.188.121]) by smtp3.osuosl.org (Postfix) with ESMTPS id ED1DF6157B for ; Fri, 3 Sep 2021 19:27:53 +0000 (UTC) Received: from frode-threadripper.. (1.general.frode.uk.vpn [10.172.193.250]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id 639AD409FB; Fri, 3 Sep 2021 19:27:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1630697271; bh=9+JG7EYfi+onJczxe6EY3T9ZDBdanPONqR5yvyKh8Eg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=vrybp9eWx87FGsK9JncBOB3dX8yCPOEkQezTX66Q9aGDaLdOAuKfluJA0loa/kr5I zDLRj+jJd/DjyQx+w+UmXlEf764cf1kt/Bpn6eEkQCbyaF4ldkiUXHKY+M+GSDjFEC KfRrpiUs2mLndHe0Gp+5Xyk+WfRXZ+GKPM8+hcjw9mYm4cjGU5lglsiCK8hRljIbZD KPaWa/R93xcsmsQbIHrWuq6mtRX5oM9WLoKJZBkSYbIf77qo/jxcQCwDCyylFQn17/ O0qfi8L+wkq5P024GrWDRuCQuXeNITS6oZiBvrAbJuvqn3oM1DMXnCZdSRlIa5k7Xn lsUcGIsG9UsHg== From: Frode Nordahl To: dev@openvswitch.org Date: Fri, 3 Sep 2021 21:27:46 +0200 Message-Id: <20210903192748.1408062-8-frode.nordahl@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210903192748.1408062-1-frode.nordahl@canonical.com> References: <20210903192748.1408062-1-frode.nordahl@canonical.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v4 7/9] ovn-controller: Prepare plugging infrastructure. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add port by interfaces index - To be able to effectively remove ports previously plugged by us we need to look up ports by interface records. Add `enable-dummy-plug` option - To enable testing of the plugging infrastructure without building OVN with an external plugging library we include a dummy implementation which can be enabled using this command line option. Also adds a ENGINE_NODE_NO_DATA macro which is useful for when lookup data is located elsewhere and not passed around as part of the incremental processsing engine. Signed-off-by: Frode Nordahl --- controller/binding.h | 1 + controller/ovn-controller.c | 26 ++++++++++++++++++++++++++ lib/inc-proc-eng.h | 7 +++++++ 3 files changed, 34 insertions(+) diff --git a/controller/binding.h b/controller/binding.h index f1abc4b9c..8f4521806 100644 --- a/controller/binding.h +++ b/controller/binding.h @@ -46,6 +46,7 @@ struct binding_ctx_in { struct ovsdb_idl_index *sbrec_datapath_binding_by_key; struct ovsdb_idl_index *sbrec_port_binding_by_datapath; struct ovsdb_idl_index *sbrec_port_binding_by_name; + struct ovsdb_idl_index *ovsrec_port_by_interfaces; const struct ovsrec_port_table *port_table; const struct ovsrec_qos_table *qos_table; const struct sbrec_port_binding_table *port_binding_table; diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c index 7387a177b..1b808e39f 100644 --- a/controller/ovn-controller.c +++ b/controller/ovn-controller.c @@ -55,7 +55,9 @@ #include "lib/mcast-group-index.h" #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" +#include "lib/plug-dummy.h" #include "patch.h" +#include "plug.h" #include "physical.h" #include "pinctrl.h" #include "openvswitch/poll-loop.h" @@ -1242,6 +1244,11 @@ init_binding_ctx(struct engine_node *node, engine_get_input("SB_port_binding", node), "datapath"); + struct ovsdb_idl_index *ovsrec_port_by_interfaces = + engine_ovsdb_node_get_index( + engine_get_input("OVS_port", node), + "interfaces"); + struct controller_engine_ctx *ctrl_ctx = engine_get_context()->client_ctx; b_ctx_in->ovnsb_idl_txn = engine_get_context()->ovnsb_idl_txn; @@ -1249,6 +1256,7 @@ init_binding_ctx(struct engine_node *node, b_ctx_in->sbrec_datapath_binding_by_key = sbrec_datapath_binding_by_key; b_ctx_in->sbrec_port_binding_by_datapath = sbrec_port_binding_by_datapath; b_ctx_in->sbrec_port_binding_by_name = sbrec_port_binding_by_name; + b_ctx_in->ovsrec_port_by_interfaces = ovsrec_port_by_interfaces; b_ctx_in->port_table = port_table; b_ctx_in->iface_table = iface_table; b_ctx_in->qos_table = qos_table; @@ -3102,11 +3110,17 @@ main(int argc, char *argv[]) patch_init(); pinctrl_init(); lflow_init(); + plug_initialize(); /* Connect to OVS OVSDB instance. */ struct ovsdb_idl_loop ovs_idl_loop = OVSDB_IDL_LOOP_INITIALIZER( ovsdb_idl_create(ovs_remote, &ovsrec_idl_class, false, true)); ctrl_register_ovs_idl(ovs_idl_loop.idl); + + struct ovsdb_idl_index *ovsrec_port_by_interfaces + = ovsdb_idl_index_create1(ovs_idl_loop.idl, + &ovsrec_port_col_interfaces); + ovsdb_idl_get_initial_snapshot(ovs_idl_loop.idl); /* Configure OVN SB database. */ @@ -3226,6 +3240,7 @@ main(int argc, char *argv[]) ENGINE_NODE(flow_output, "flow_output"); ENGINE_NODE(addr_sets, "addr_sets"); ENGINE_NODE_WITH_CLEAR_TRACK_DATA(port_groups, "port_groups"); + ENGINE_NODE_NO_DATA(plug_provider, "plug_provider"); #define SB_NODE(NAME, NAME_STR) ENGINE_NODE_SB(NAME, NAME_STR); SB_NODES @@ -3320,6 +3335,8 @@ main(int argc, char *argv[]) engine_add_input(&en_ct_zones, &en_runtime_data, ct_zones_runtime_data_handler); + engine_add_input(&en_runtime_data, &en_plug_provider, NULL); + engine_add_input(&en_runtime_data, &en_ofctrl_is_connected, NULL); engine_add_input(&en_runtime_data, &en_ovs_open_vswitch, NULL); @@ -3370,6 +3387,8 @@ main(int argc, char *argv[]) sbrec_port_binding_by_datapath); engine_ovsdb_node_add_index(&en_sb_datapath_binding, "key", sbrec_datapath_binding_by_key); + engine_ovsdb_node_add_index(&en_ovs_port, "interfaces", + ovsrec_port_by_interfaces); struct ed_type_lflow_output *lflow_output_data = engine_get_internal_data(&en_lflow_output); @@ -3897,6 +3916,7 @@ loop_done: pinctrl_destroy(); patch_destroy(); if_status_mgr_destroy(if_mgr); + plug_destroy_all(); ovsdb_idl_loop_destroy(&ovs_idl_loop); ovsdb_idl_loop_destroy(&ovnsb_idl_loop); @@ -3916,6 +3936,7 @@ parse_options(int argc, char *argv[]) VLOG_OPTION_ENUMS, OVN_DAEMON_OPTION_ENUMS, SSL_OPTION_ENUMS, + OPT_ENABLE_DUMMY_PLUG, }; static struct option long_options[] = { @@ -3926,6 +3947,7 @@ parse_options(int argc, char *argv[]) STREAM_SSL_LONG_OPTIONS, {"peer-ca-cert", required_argument, NULL, OPT_PEER_CA_CERT}, {"bootstrap-ca-cert", required_argument, NULL, OPT_BOOTSTRAP_CA_CERT}, + {"enable-dummy-plug", no_argument, NULL, OPT_ENABLE_DUMMY_PLUG}, {NULL, 0, NULL, 0} }; char *short_options = ovs_cmdl_long_options_to_short_options(long_options); @@ -3971,6 +3993,10 @@ parse_options(int argc, char *argv[]) stream_ssl_set_ca_cert_file(optarg, true); break; + case OPT_ENABLE_DUMMY_PLUG: + plug_dummy_enable(); + break; + case '?': exit(EXIT_FAILURE); diff --git a/lib/inc-proc-eng.h b/lib/inc-proc-eng.h index 859b30a71..00e799512 100644 --- a/lib/inc-proc-eng.h +++ b/lib/inc-proc-eng.h @@ -306,6 +306,13 @@ void engine_ovsdb_node_add_index(struct engine_node *, const char *name, ENGINE_NODE_CUSTOM_DATA(NAME, NAME_STR) \ en_##NAME.clear_tracked_data = en_##NAME##_clear_tracked_data; +#define ENGINE_NODE_NO_DATA(NAME, NAME_STR) \ + static void * (*en_##NAME##_init)(struct engine_node *, \ + struct engine_arg *) = NULL; \ + static void (*en_##NAME##_cleanup)(void *) = NULL; \ + static bool (*en_##NAME##_is_valid)(struct engine_node *node) = NULL; \ + ENGINE_NODE_DEF(NAME, NAME_STR) + #define ENGINE_NODE(NAME, NAME_STR) \ static bool (*en_##NAME##_is_valid)(struct engine_node *node) = NULL; \ ENGINE_NODE_DEF(NAME, NAME_STR) From patchwork Fri Sep 3 19:27:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frode Nordahl X-Patchwork-Id: 1524524 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=hSP/RF6t; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H1SWR3ZQWz9sCD for ; Sat, 4 Sep 2021 05:28:19 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id F3FB9408BF; Fri, 3 Sep 2021 19:28:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8DizZZAXz0W8; Fri, 3 Sep 2021 19:28:13 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 7704640871; Fri, 3 Sep 2021 19:28:08 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id CC254C0023; Fri, 3 Sep 2021 19:28:01 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 595E3C001C for ; Fri, 3 Sep 2021 19:27:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 8B7A24027D for ; Fri, 3 Sep 2021 19:27:55 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YZas-45BrQol for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from smtp-relay-canonical-1.canonical.com (smtp-relay-canonical-1.canonical.com [185.125.188.121]) by smtp2.osuosl.org (Postfix) with ESMTPS id 108894070E for ; Fri, 3 Sep 2021 19:27:53 +0000 (UTC) Received: from frode-threadripper.. (1.general.frode.uk.vpn [10.172.193.250]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id B3E6540EA3; Fri, 3 Sep 2021 19:27:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1630697271; bh=9i9Ag8TNozoQshNP5+K5SJUHnQRMHy1rr1FdqQaoik4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=hSP/RF6tYVbPYQJGGjF/EMHs3LJUCz1MdnMTV2k7ixhgiUG08CMyel++GASQJmlS1 DP92sq4HNpWTwlcaStvkkFsRZ3Krxtnp6PcRVu+50mmcBQe8wY9Q4b5bB3sKr2WWW3 apPqg5Mis4rcxEJVeTv1zcfHNiFG8eLQQuISxHGW/epz1vZXdsRhhHy2tH7YD1AOVL vm3InMYDAiDXrMQkhr/HwkhGEWDQVuwJD/Et3dkAJTB+/ISO9gkQmcI8q++RiXzizH MNYnHL5Z36zWZJ4tc+4hJNW5ooogne+rIYuch4TTrnNoAMMaA/wNRzOWWKcGoWswLq awPdCHSfLpe3Q== From: Frode Nordahl To: dev@openvswitch.org Date: Fri, 3 Sep 2021 21:27:47 +0200 Message-Id: <20210903192748.1408062-9-frode.nordahl@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210903192748.1408062-1-frode.nordahl@canonical.com> References: <20210903192748.1408062-1-frode.nordahl@canonical.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v4 8/9] binding: Consider plugging of ports on CMS request. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" When OVN is linked with an appropriate plugging implementation, CMS can request OVN to plug individual lports into the local Open vSwitch instance. The port and instance record will be maintained during the lifetime of the lport and it will be removed on release of lport. Signed-off-by: Frode Nordahl --- controller/binding.c | 247 ++++++++++++++++++++++++++++++++++++++-- tests/ovn-controller.at | 31 +++++ tests/ovn-macros.at | 2 +- 3 files changed, 272 insertions(+), 8 deletions(-) diff --git a/controller/binding.c b/controller/binding.c index 938e1d81d..ae67ee3fc 100644 --- a/controller/binding.c +++ b/controller/binding.c @@ -35,7 +35,9 @@ #include "local_data.h" #include "lport.h" #include "ovn-controller.h" +#include "ovsport.h" #include "patch.h" +#include "plug.h" VLOG_DEFINE_THIS_MODULE(binding); @@ -45,6 +47,8 @@ VLOG_DEFINE_THIS_MODULE(binding); */ #define OVN_INSTALLED_EXT_ID "ovn-installed" +#define OVN_PLUGGED_EXT_ID "ovn-plugged" + #define OVN_QOS_TYPE "linux-htb" struct qos_queue { @@ -71,10 +75,13 @@ binding_register_ovs_idl(struct ovsdb_idl *ovs_idl) ovsdb_idl_add_table(ovs_idl, &ovsrec_table_interface); ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_name); + ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_type); ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_external_ids); ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_bfd); ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_bfd_status); ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_status); + ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_options); + ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_mtu_request); ovsdb_idl_add_table(ovs_idl, &ovsrec_table_qos); ovsdb_idl_add_column(ovs_idl, &ovsrec_qos_col_type); @@ -1090,6 +1097,51 @@ release_binding_lport(const struct sbrec_chassis *chassis_rec, return true; } +static void +consider_unplug_lport(const struct sbrec_port_binding *pb, + struct binding_ctx_in *b_ctx_in, + struct local_binding *lbinding) +{ + const char *plug_type = NULL; + if (lbinding && lbinding->iface) { + plug_type = smap_get(&lbinding->iface->external_ids, + OVN_PLUGGED_EXT_ID); + } + + if (plug_type) { + const struct ovsrec_port *port = ovsport_lookup_by_interface( + b_ctx_in->ovsrec_port_by_interfaces, + (struct ovsrec_interface *) lbinding->iface); + if (port) { + const struct plug_class *plug; + if (!(plug = plug_get_provider(plug_type))) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, + "Unable to open plug provider for " + "plug-type: '%s' lport %s", + plug_type, pb->logical_port); + return; + } + const struct plug_port_ctx_in plug_ctx_in = { + .op_type = PLUG_OP_REMOVE, + .ovs_table = b_ctx_in->ovs_table, + .br_int = b_ctx_in->br_int, + .lport_name = (const char *)pb->logical_port, + .lport_options = (const struct smap *)&pb->options, + .iface_name = lbinding->iface->name, + .iface_type = lbinding->iface->type, + .iface_options = &lbinding->iface->options, + }; + plug_port_prepare(plug, &plug_ctx_in, NULL); + VLOG_INFO("Unplugging port %s from %s for lport %s on this " + "chassis.", + port->name, b_ctx_in->br_int->name, pb->logical_port); + ovsport_remove(b_ctx_in->br_int, port); + plug_port_finish(plug, &plug_ctx_in, NULL); + } + } +} + static bool consider_vif_lport_(const struct sbrec_port_binding *pb, bool can_bind, @@ -1141,15 +1193,184 @@ consider_vif_lport_(const struct sbrec_port_binding *pb, if (pb->chassis == b_ctx_in->chassis_rec) { /* Release the lport if there is no lbinding. */ if (!lbinding_set || !can_bind) { - return release_lport(pb, !b_ctx_in->ovnsb_idl_txn, - b_ctx_out->tracked_dp_bindings, - b_ctx_out->if_mgr); + bool handled = release_lport(pb, !b_ctx_in->ovnsb_idl_txn, + b_ctx_out->tracked_dp_bindings, + b_ctx_out->if_mgr); + if (handled && b_lport && b_lport->lbinding) { + consider_unplug_lport(pb, b_ctx_in, b_lport->lbinding); + } + return handled; } } return true; } +static int64_t +get_plug_mtu_request(const struct smap *lport_options) +{ + return smap_get_int(lport_options, "plug-mtu-request", 0); +} + +static bool +consider_plug_lport_create__(const struct plug_class *plug, + const struct smap *iface_external_ids, + const struct sbrec_port_binding *pb, + struct binding_ctx_in *b_ctx_in) +{ + if (!b_ctx_in->chassis_rec || !b_ctx_in->br_int || !b_ctx_in->ovs_idl_txn) + { + /* Some of our prerequisites are not available, ask for a recompute. */ + return false; + } + + bool ret = true; + struct plug_port_ctx_in plug_ctx_in = { + .op_type = PLUG_OP_CREATE, + .ovs_table = b_ctx_in->ovs_table, + .br_int = b_ctx_in->br_int, + .lport_name = (const char *)pb->logical_port, + .lport_options = (const struct smap *)&pb->options, + }; + struct plug_port_ctx_out plug_ctx_out; + + if (!plug_port_prepare(plug, &plug_ctx_in, &plug_ctx_out)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_INFO_RL(&rl, + "Not plugging lport %s on direction from plugging " + "library.", + pb->logical_port); + ret = false; + goto out; + } + + VLOG_INFO("Plugging port %s into %s for lport %s on this " + "chassis.", + plug_ctx_out.name, b_ctx_in->br_int->name, + pb->logical_port); + ovsport_create(b_ctx_in->ovs_idl_txn, b_ctx_in->br_int, + plug_ctx_out.name, plug_ctx_out.type, + NULL, iface_external_ids, + plug_ctx_out.iface_options, + get_plug_mtu_request(&pb->options)); + + plug_port_finish(plug, &plug_ctx_in, &plug_ctx_out); + +out: + plug_port_ctx_destroy(plug, &plug_ctx_in, &plug_ctx_out); + + return ret; +} + +static bool +consider_plug_lport_update__(const struct plug_class *plug, + const struct smap *iface_external_ids, + const struct sbrec_port_binding *pb, + struct binding_ctx_in *b_ctx_in, + struct local_binding *lbinding) +{ + if (!b_ctx_in->chassis_rec || !b_ctx_in->br_int || !b_ctx_in->ovs_idl_txn) + { + /* Some of our prerequisites are not available, ask for a recompute. */ + return false; + } + + bool ret = true; + struct plug_port_ctx_in plug_ctx_in = { + .op_type = PLUG_OP_CREATE, + .ovs_table = b_ctx_in->ovs_table, + .br_int = b_ctx_in->br_int, + .lport_name = (const char *)pb->logical_port, + .lport_options = (const struct smap *)&pb->options, + .iface_name = lbinding->iface->name, + .iface_type = lbinding->iface->type, + .iface_options = &lbinding->iface->options, + }; + struct plug_port_ctx_out plug_ctx_out; + + if (!plug_port_prepare(plug, &plug_ctx_in, &plug_ctx_out)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_INFO_RL(&rl, + "Not updating lport %s on direction from plugging " + "library.", + pb->logical_port); + ret = false; + goto out; + } + + if (strcmp(lbinding->iface->name, plug_ctx_out.name)) { + VLOG_WARN("Attempt of incompatible change to existing " + "port detected, please recreate port: %s", + pb->logical_port); + ret = false; + goto out; + } + VLOG_DBG("updating iface for: %s", pb->logical_port); + ovsport_update_iface(lbinding->iface, plug_ctx_out.type, + iface_external_ids, + NULL, + plug_ctx_out.iface_options, + plug_get_maintained_iface_options(plug), + get_plug_mtu_request(&pb->options)); + + plug_port_finish(plug, &plug_ctx_in, &plug_ctx_out); +out: + plug_port_ctx_destroy(plug, &plug_ctx_in, &plug_ctx_out); + + return ret; +} + +static bool +consider_plug_lport(const struct sbrec_port_binding *pb, + struct binding_ctx_in *b_ctx_in, + struct local_binding *lbinding) +{ + bool ret = true; + if (can_bind_on_this_chassis(b_ctx_in->chassis_rec, pb)) { + const char *plug_type = smap_get(&pb->options, "plug-type"); + if (!plug_type) { + /* Nothing for us to do and we don't need a recompute. */ + return true; + } + + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + const struct plug_class *plug; + if (!(plug = plug_get_provider(plug_type))) { + VLOG_WARN_RL(&rl, + "Unable to open plug provider for plug-type: '%s' " + "lport %s", + plug_type, pb->logical_port); + /* While we are unable to handle this, asking for a recompute will + * not change that fact. */ + return true; + } + const struct smap iface_external_ids = SMAP_CONST2( + &iface_external_ids, + OVN_PLUGGED_EXT_ID, plug_type, + "iface-id", pb->logical_port); + if (lbinding && lbinding->iface) { + if (!smap_get(&lbinding->iface->external_ids, + OVN_PLUGGED_EXT_ID)) + { + VLOG_WARN_RL(&rl, + "CMS requested plugging of lport %s, but a port " + "that is not maintained by OVN already exsist " + "in local vSwitch: "UUID_FMT, + pb->logical_port, + UUID_ARGS(&lbinding->iface->header_.uuid)); + return false; + } + ret = consider_plug_lport_update__(plug, &iface_external_ids, pb, + b_ctx_in, lbinding); + } else { + ret = consider_plug_lport_create__(plug, &iface_external_ids, pb, + b_ctx_in); + } + } + + return ret; +} + static bool consider_vif_lport(const struct sbrec_port_binding *pb, struct binding_ctx_in *b_ctx_in, @@ -1187,8 +1408,13 @@ consider_vif_lport(const struct sbrec_port_binding *pb, } } - return consider_vif_lport_(pb, can_bind, b_ctx_in, b_ctx_out, - b_lport, qos_map); + /* Consider if we should create or update local port/interface record for + * this lport. Note that a newly created port/interface will get its flows + * installed on the next loop iteration as we won't wait for OVS vSwitchd + * to configure and assign a ofport to the interface. */ + return consider_plug_lport(pb, b_ctx_in, lbinding) + & consider_vif_lport_(pb, can_bind, b_ctx_in, b_ctx_out, + b_lport, qos_map); } static bool @@ -2111,8 +2337,11 @@ handle_deleted_vif_lport(const struct sbrec_port_binding *pb, lbinding = b_lport->lbinding; bound = is_binding_lport_this_chassis(b_lport, b_ctx_in->chassis_rec); - /* Remove b_lport from local_binding. */ - binding_lport_delete(binding_lports, b_lport); + if (b_lport->lbinding) { + consider_unplug_lport(pb, b_ctx_in, b_lport->lbinding); + } + /* Remove b_lport from local_binding. */ + binding_lport_delete(binding_lports, b_lport); } if (bound && lbinding && lport_type == LP_VIF) { @@ -2690,6 +2919,10 @@ local_binding_handle_stale_binding_lports(struct local_binding *lbinding, handled = release_binding_lport(b_ctx_in->chassis_rec, b_lport, !b_ctx_in->ovnsb_idl_txn, b_ctx_out); + if (handled && b_lport && b_lport->lbinding) { + consider_unplug_lport(b_lport->pb, b_ctx_in, + b_lport->lbinding); + } binding_lport_delete(&b_ctx_out->lbinding_data->lports, b_lport); } diff --git a/tests/ovn-controller.at b/tests/ovn-controller.at index 4ae218ed6..a38884374 100644 --- a/tests/ovn-controller.at +++ b/tests/ovn-controller.at @@ -723,3 +723,34 @@ AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep controller | grep userdata=0 OVN_CLEANUP([hv1]) AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([ovn-controller - plugging]) +AT_KEYWORDS([plugging]) + +ovn_start + +net_add n1 +sim_add hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.1 + +check ovn-nbctl ls-add lsw0 +check ovn-nbctl lsp-add lsw0 lsp1 +check ovn-nbctl lsp-set-addresses lsp1 "f0:00:00:00:00:01 172.16.0.100" +check ovn-nbctl --wait=sb set Logical_Switch_Port lsp1 \ + options:requested-chassis=hv1 \ + options:plug-type=dummy \ + options:plug-mtu-request=42 + +wait_column "true" Port_Binding up logical_port=lsp1 + +as hv1 + +AT_CHECK([as hv1 ovs-vsctl find interface name=lsp1 options:plug-dummy-option=value | grep -q "options.*value"]) +AT_CHECK([as hv1 ovs-vsctl find interface name=lsp1 mtu_request=42 | grep -q "mtu_request.*42"]) + +OVN_CLEANUP([hv1]) +AT_CLEANUP +]) + diff --git a/tests/ovn-macros.at b/tests/ovn-macros.at index f06f2e68e..b3c2d72fa 100644 --- a/tests/ovn-macros.at +++ b/tests/ovn-macros.at @@ -327,7 +327,7 @@ ovn_az_attach() { -- --may-exist add-br br-int \ -- set bridge br-int fail-mode=secure other-config:disable-in-band=true \ || return 1 - start_daemon ovn-controller || return 1 + start_daemon ovn-controller --enable-dummy-plug || return 1 } # ovn_attach NETWORK BRIDGE IP [MASKLEN] From patchwork Fri Sep 3 19:27:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frode Nordahl X-Patchwork-Id: 1524526 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=E4jOT6p8; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H1SWZ58FXz9sCD for ; Sat, 4 Sep 2021 05:28:26 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id B29C24088A; Fri, 3 Sep 2021 19:28:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aviEbR5x0kg9; Fri, 3 Sep 2021 19:28:18 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 46EC74027D; Fri, 3 Sep 2021 19:28:12 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B60D5C0040; Fri, 3 Sep 2021 19:28:03 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5937AC0030 for ; Fri, 3 Sep 2021 19:27:58 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 2D3FD615A3 for ; Fri, 3 Sep 2021 19:27:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=canonical.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sPByZGaMjXYh for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from smtp-relay-canonical-1.canonical.com (smtp-relay-canonical-1.canonical.com [185.125.188.121]) by smtp3.osuosl.org (Postfix) with ESMTPS id 348D561584 for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) Received: from frode-threadripper.. (1.general.frode.uk.vpn [10.172.193.250]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id 1200F40EA4; Fri, 3 Sep 2021 19:27:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1630697272; bh=wU7K7Q4llmEgv58FVkVYnTlfjN2YNNc25BJ3E/oFuto=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=E4jOT6p8tf0bl6zQNQp99d4EAR74zUc+szYAcfSFkxs+r81ynAOuBN8nr/6z2+BDF unqKj+dzgu1iFJpPj4kPQoHv1GG8+gNg9xMUcI6mzx0kQjYP+wQjJwSobusoYukMt5 pL18Qb0WS37Kd02x3Swb+nqvp1sVZZCQ+46Iz71Uyg+QkVahEcs2cyb4zQCyuvMqLm kbUFZl0436eyGcjVUrVj9Kz+8exmBOl/Qs4lV/5klMzk/awx8+0owS4aZcYEwEVzu2 nUv6jbkEZAigU4ICHL3PPVSUOLQDJxjBlrQV2+pC0sBciEsf5TpWM0uL2i/BRsC1KQ L1mz9wmXI2Vzw== From: Frode Nordahl To: dev@openvswitch.org Date: Fri, 3 Sep 2021 21:27:48 +0200 Message-Id: <20210903192748.1408062-10-frode.nordahl@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210903192748.1408062-1-frode.nordahl@canonical.com> References: <20210903192748.1408062-1-frode.nordahl@canonical.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v4 9/9] plug_providers: Introduce representor plugin. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add the first in-tree plug provider plugin and its dependencies. The representor plugin can be used with multiple NIC vendors supporting Open vSwitch hardware offload and the devlink-port infrastructure[0]. It is particularly useful for use with NICs connected to multiple distinct CPUs where the instance runs on one host and Open vSwitch and OVN runs on a different host, the smartnic CPU. Extend the build system with macros from the OVS build system to allow checking for dependencies of the plugin as well as providing kernel header files that may not be available at build time. The plugin will only be built when enabled and when building on a Linux system. 0: https://www.kernel.org/doc/html/latest/networking/devlink/devlink-port.html Signed-off-by: Frode Nordahl --- Documentation/automake.mk | 1 + Documentation/topics/plug_providers/index.rst | 1 + .../topics/plug_providers/plug-providers.rst | 5 + .../plug_providers/plug-representor.rst | 45 ++ build-aux/initial-tab-whitelist | 1 + configure.ac | 2 + include/automake.mk | 4 + include/linux/automake.mk | 2 + include/linux/devlink.h | 625 ++++++++++++++++++ lib/automake.mk | 11 + lib/plug-provider.h | 6 +- lib/plug.c | 1 + .../representor/netlink-devlink.c | 499 ++++++++++++++ .../representor/netlink-devlink.h | 115 ++++ .../representor/plug-representor.c | 307 +++++++++ m4/ovn.m4 | 26 + 16 files changed, 1650 insertions(+), 1 deletion(-) create mode 100644 Documentation/topics/plug_providers/plug-representor.rst create mode 100644 include/linux/automake.mk create mode 100644 include/linux/devlink.h create mode 100644 lib/plug_providers/representor/netlink-devlink.c create mode 100644 lib/plug_providers/representor/netlink-devlink.h create mode 100644 lib/plug_providers/representor/plug-representor.c diff --git a/Documentation/automake.mk b/Documentation/automake.mk index 92a843d76..2a3483bc0 100644 --- a/Documentation/automake.mk +++ b/Documentation/automake.mk @@ -29,6 +29,7 @@ DOC_SOURCE = \ Documentation/topics/role-based-access-control.rst \ Documentation/topics/debugging-ddlog.rst \ Documentation/topics/plug_providers/plug-providers.rst \ + Documentation/topics/plug_providers/plug-representor.rst \ Documentation/howto/index.rst \ Documentation/howto/docker.rst \ Documentation/howto/firewalld.rst \ diff --git a/Documentation/topics/plug_providers/index.rst b/Documentation/topics/plug_providers/index.rst index 837eeae15..3d16458a2 100644 --- a/Documentation/topics/plug_providers/index.rst +++ b/Documentation/topics/plug_providers/index.rst @@ -30,3 +30,4 @@ Plug Providers :maxdepth: 2 plug-providers + plug-representor diff --git a/Documentation/topics/plug_providers/plug-providers.rst b/Documentation/topics/plug_providers/plug-providers.rst index 7b891156c..5f0089ed9 100644 --- a/Documentation/topics/plug_providers/plug-providers.rst +++ b/Documentation/topics/plug_providers/plug-providers.rst @@ -163,6 +163,11 @@ Building with in-tree plugging providers Plugging providers hosted in the OVN repository living under `lib/plug_providers`: +* :doc:`representor ` + + - Representor port lookup making use of the Linux kernel devlink-port + infrastructure. + To enable them, provide the `--enable-plug-providers` command line option to the configure script when building OVN. diff --git a/Documentation/topics/plug_providers/plug-representor.rst b/Documentation/topics/plug_providers/plug-representor.rst new file mode 100644 index 000000000..c301a6cd2 --- /dev/null +++ b/Documentation/topics/plug_providers/plug-representor.rst @@ -0,0 +1,45 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in OVN documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +============================= +The Representor Plug Provider +============================= + +Logical Switch Port Options +--------------------------- + +plug:representor:pf-mac +~~~~~~~~~~~~~~~~~~~~~~~ + +MAC address for identifying PF device. When +`OVN_Northbound:Logical_Switch_Port:options` key `plug:representor:vf-num` is +also set, this option is used to identify PF to use as base to locate the +correct VF representor port. When `OVN_Northbound:Logical_Switch_Port:options` +key `plug:representor:vf-num` is not set this option is used to locate a PF +representor port. + +plug:representor:vf-num +~~~~~~~~~~~~~~~~~~~~~~~ + +Logical VF number relative to PF device specified in +`OVN_Northbound:Logical_Switch_Port:options` key `plug-pf-mac`. diff --git a/build-aux/initial-tab-whitelist b/build-aux/initial-tab-whitelist index b2f5a0791..c70f93891 100644 --- a/build-aux/initial-tab-whitelist +++ b/build-aux/initial-tab-whitelist @@ -3,6 +3,7 @@ \.mk$ \.png$ \.sln$ +^include/linux/ ^ovs/ ^third-party/ ^xenserver/ diff --git a/configure.ac b/configure.ac index 7f3274e59..5b542bfd1 100644 --- a/configure.ac +++ b/configure.ac @@ -174,6 +174,8 @@ OVS_CHECK_PRAGMA_MESSAGE OVN_CHECK_OVS OVN_CHECK_PLUG_PROVIDER OVN_ENABLE_PLUG +OVS_CHECK_NETLINK +OVS_CHECK_LINUX_NETLINK OVS_CTAGS_IDENTIFIERS AC_SUBST([OVS_CFLAGS]) AC_SUBST([OVS_LDFLAGS]) diff --git a/include/automake.mk b/include/automake.mk index 9e8403f8d..75638bd9a 100644 --- a/include/automake.mk +++ b/include/automake.mk @@ -1,2 +1,6 @@ include include/ovn/automake.mk +if LINUX +include include/linux/automake.mk +endif + diff --git a/include/linux/automake.mk b/include/linux/automake.mk new file mode 100644 index 000000000..5b53597eb --- /dev/null +++ b/include/linux/automake.mk @@ -0,0 +1,2 @@ +noinst_HEADERS += \ + include/linux/devlink.h diff --git a/include/linux/devlink.h b/include/linux/devlink.h new file mode 100644 index 000000000..28ea92b62 --- /dev/null +++ b/include/linux/devlink.h @@ -0,0 +1,625 @@ +/* + * The kernel devlink interface has gained a number of additions in recent + * kernel versions. To allow Open vSwitch to consume these interfaces in its + * runtime environment regardless of what kernel version was available at build + * time, and also avoiding an elaborate set of autoconf macros to check for + * presence of individual pieces, we include the entire file here. + * + * Source: + * https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/include/uapi/linux/devlink.h @ a556dded9c23c51c82654f1ebe389cbc0bc22057 */ +#if !defined(__KERNEL__) +#ifndef __UAPI_LINUX_DEVLINK_WRAPPER_H +#define __UAPI_LINUX_DEVLINK_WRAPPER_H 1 +/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */ +/* + * include/uapi/linux/devlink.h - Network physical device Netlink interface + * Copyright (c) 2016 Mellanox Technologies. All rights reserved. + * Copyright (c) 2016 Jiri Pirko + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + */ + +#ifndef _UAPI_LINUX_DEVLINK_H_ +#define _UAPI_LINUX_DEVLINK_H_ + +#include + +#define DEVLINK_GENL_NAME "devlink" +#define DEVLINK_GENL_VERSION 0x1 +#define DEVLINK_GENL_MCGRP_CONFIG_NAME "config" + +enum devlink_command { + /* don't change the order or add anything between, this is ABI! */ + DEVLINK_CMD_UNSPEC, + + DEVLINK_CMD_GET, /* can dump */ + DEVLINK_CMD_SET, + DEVLINK_CMD_NEW, + DEVLINK_CMD_DEL, + + DEVLINK_CMD_PORT_GET, /* can dump */ + DEVLINK_CMD_PORT_SET, + DEVLINK_CMD_PORT_NEW, + DEVLINK_CMD_PORT_DEL, + + DEVLINK_CMD_PORT_SPLIT, + DEVLINK_CMD_PORT_UNSPLIT, + + DEVLINK_CMD_SB_GET, /* can dump */ + DEVLINK_CMD_SB_SET, + DEVLINK_CMD_SB_NEW, + DEVLINK_CMD_SB_DEL, + + DEVLINK_CMD_SB_POOL_GET, /* can dump */ + DEVLINK_CMD_SB_POOL_SET, + DEVLINK_CMD_SB_POOL_NEW, + DEVLINK_CMD_SB_POOL_DEL, + + DEVLINK_CMD_SB_PORT_POOL_GET, /* can dump */ + DEVLINK_CMD_SB_PORT_POOL_SET, + DEVLINK_CMD_SB_PORT_POOL_NEW, + DEVLINK_CMD_SB_PORT_POOL_DEL, + + DEVLINK_CMD_SB_TC_POOL_BIND_GET, /* can dump */ + DEVLINK_CMD_SB_TC_POOL_BIND_SET, + DEVLINK_CMD_SB_TC_POOL_BIND_NEW, + DEVLINK_CMD_SB_TC_POOL_BIND_DEL, + + /* Shared buffer occupancy monitoring commands */ + DEVLINK_CMD_SB_OCC_SNAPSHOT, + DEVLINK_CMD_SB_OCC_MAX_CLEAR, + + DEVLINK_CMD_ESWITCH_GET, +#define DEVLINK_CMD_ESWITCH_MODE_GET /* obsolete, never use this! */ \ + DEVLINK_CMD_ESWITCH_GET + + DEVLINK_CMD_ESWITCH_SET, +#define DEVLINK_CMD_ESWITCH_MODE_SET /* obsolete, never use this! */ \ + DEVLINK_CMD_ESWITCH_SET + + DEVLINK_CMD_DPIPE_TABLE_GET, + DEVLINK_CMD_DPIPE_ENTRIES_GET, + DEVLINK_CMD_DPIPE_HEADERS_GET, + DEVLINK_CMD_DPIPE_TABLE_COUNTERS_SET, + DEVLINK_CMD_RESOURCE_SET, + DEVLINK_CMD_RESOURCE_DUMP, + + /* Hot driver reload, makes configuration changes take place. The + * devlink instance is not released during the process. + */ + DEVLINK_CMD_RELOAD, + + DEVLINK_CMD_PARAM_GET, /* can dump */ + DEVLINK_CMD_PARAM_SET, + DEVLINK_CMD_PARAM_NEW, + DEVLINK_CMD_PARAM_DEL, + + DEVLINK_CMD_REGION_GET, + DEVLINK_CMD_REGION_SET, + DEVLINK_CMD_REGION_NEW, + DEVLINK_CMD_REGION_DEL, + DEVLINK_CMD_REGION_READ, + + DEVLINK_CMD_PORT_PARAM_GET, /* can dump */ + DEVLINK_CMD_PORT_PARAM_SET, + DEVLINK_CMD_PORT_PARAM_NEW, + DEVLINK_CMD_PORT_PARAM_DEL, + + DEVLINK_CMD_INFO_GET, /* can dump */ + + DEVLINK_CMD_HEALTH_REPORTER_GET, + DEVLINK_CMD_HEALTH_REPORTER_SET, + DEVLINK_CMD_HEALTH_REPORTER_RECOVER, + DEVLINK_CMD_HEALTH_REPORTER_DIAGNOSE, + DEVLINK_CMD_HEALTH_REPORTER_DUMP_GET, + DEVLINK_CMD_HEALTH_REPORTER_DUMP_CLEAR, + + DEVLINK_CMD_FLASH_UPDATE, + DEVLINK_CMD_FLASH_UPDATE_END, /* notification only */ + DEVLINK_CMD_FLASH_UPDATE_STATUS, /* notification only */ + + DEVLINK_CMD_TRAP_GET, /* can dump */ + DEVLINK_CMD_TRAP_SET, + DEVLINK_CMD_TRAP_NEW, + DEVLINK_CMD_TRAP_DEL, + + DEVLINK_CMD_TRAP_GROUP_GET, /* can dump */ + DEVLINK_CMD_TRAP_GROUP_SET, + DEVLINK_CMD_TRAP_GROUP_NEW, + DEVLINK_CMD_TRAP_GROUP_DEL, + + DEVLINK_CMD_TRAP_POLICER_GET, /* can dump */ + DEVLINK_CMD_TRAP_POLICER_SET, + DEVLINK_CMD_TRAP_POLICER_NEW, + DEVLINK_CMD_TRAP_POLICER_DEL, + + DEVLINK_CMD_HEALTH_REPORTER_TEST, + + /* add new commands above here */ + __DEVLINK_CMD_MAX, + DEVLINK_CMD_MAX = __DEVLINK_CMD_MAX - 1 +}; + +enum devlink_port_type { + DEVLINK_PORT_TYPE_NOTSET, + DEVLINK_PORT_TYPE_AUTO, + DEVLINK_PORT_TYPE_ETH, + DEVLINK_PORT_TYPE_IB, +}; + +enum devlink_sb_pool_type { + DEVLINK_SB_POOL_TYPE_INGRESS, + DEVLINK_SB_POOL_TYPE_EGRESS, +}; + +/* static threshold - limiting the maximum number of bytes. + * dynamic threshold - limiting the maximum number of bytes + * based on the currently available free space in the shared buffer pool. + * In this mode, the maximum quota is calculated based + * on the following formula: + * max_quota = alpha / (1 + alpha) * Free_Buffer + * While Free_Buffer is the amount of none-occupied buffer associated to + * the relevant pool. + * The value range which can be passed is 0-20 and serves + * for computation of alpha by following formula: + * alpha = 2 ^ (passed_value - 10) + */ + +enum devlink_sb_threshold_type { + DEVLINK_SB_THRESHOLD_TYPE_STATIC, + DEVLINK_SB_THRESHOLD_TYPE_DYNAMIC, +}; + +#define DEVLINK_SB_THRESHOLD_TO_ALPHA_MAX 20 + +enum devlink_eswitch_mode { + DEVLINK_ESWITCH_MODE_LEGACY, + DEVLINK_ESWITCH_MODE_SWITCHDEV, +}; + +enum devlink_eswitch_inline_mode { + DEVLINK_ESWITCH_INLINE_MODE_NONE, + DEVLINK_ESWITCH_INLINE_MODE_LINK, + DEVLINK_ESWITCH_INLINE_MODE_NETWORK, + DEVLINK_ESWITCH_INLINE_MODE_TRANSPORT, +}; + +enum devlink_eswitch_encap_mode { + DEVLINK_ESWITCH_ENCAP_MODE_NONE, + DEVLINK_ESWITCH_ENCAP_MODE_BASIC, +}; + +enum devlink_port_flavour { + DEVLINK_PORT_FLAVOUR_PHYSICAL, /* Any kind of a port physically + * facing the user. + */ + DEVLINK_PORT_FLAVOUR_CPU, /* CPU port */ + DEVLINK_PORT_FLAVOUR_DSA, /* Distributed switch architecture + * interconnect port. + */ + DEVLINK_PORT_FLAVOUR_PCI_PF, /* Represents eswitch port for + * the PCI PF. It is an internal + * port that faces the PCI PF. + */ + DEVLINK_PORT_FLAVOUR_PCI_VF, /* Represents eswitch port + * for the PCI VF. It is an internal + * port that faces the PCI VF. + */ + DEVLINK_PORT_FLAVOUR_VIRTUAL, /* Any virtual port facing the user. */ + DEVLINK_PORT_FLAVOUR_UNUSED, /* Port which exists in the switch, but + * is not used in any way. + */ + DEVLINK_PORT_FLAVOUR_PCI_SF, /* Represents eswitch port + * for the PCI SF. It is an internal + * port that faces the PCI SF. + */ +}; + +enum devlink_param_cmode { + DEVLINK_PARAM_CMODE_RUNTIME, + DEVLINK_PARAM_CMODE_DRIVERINIT, + DEVLINK_PARAM_CMODE_PERMANENT, + + /* Add new configuration modes above */ + __DEVLINK_PARAM_CMODE_MAX, + DEVLINK_PARAM_CMODE_MAX = __DEVLINK_PARAM_CMODE_MAX - 1 +}; + +enum devlink_param_fw_load_policy_value { + DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DRIVER, + DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_FLASH, + DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DISK, + DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_UNKNOWN, +}; + +enum devlink_param_reset_dev_on_drv_probe_value { + DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_UNKNOWN, + DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_ALWAYS, + DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_NEVER, + DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_DISK, +}; + +enum { + DEVLINK_ATTR_STATS_RX_PACKETS, /* u64 */ + DEVLINK_ATTR_STATS_RX_BYTES, /* u64 */ + DEVLINK_ATTR_STATS_RX_DROPPED, /* u64 */ + + __DEVLINK_ATTR_STATS_MAX, + DEVLINK_ATTR_STATS_MAX = __DEVLINK_ATTR_STATS_MAX - 1 +}; + +/* Specify what sections of a flash component can be overwritten when + * performing an update. Overwriting of firmware binary sections is always + * implicitly assumed to be allowed. + * + * Each section must be documented in + * Documentation/networking/devlink/devlink-flash.rst + * + */ +enum { + DEVLINK_FLASH_OVERWRITE_SETTINGS_BIT, + DEVLINK_FLASH_OVERWRITE_IDENTIFIERS_BIT, + + __DEVLINK_FLASH_OVERWRITE_MAX_BIT, + DEVLINK_FLASH_OVERWRITE_MAX_BIT = __DEVLINK_FLASH_OVERWRITE_MAX_BIT - 1 +}; + +#define DEVLINK_FLASH_OVERWRITE_SETTINGS _BITUL(DEVLINK_FLASH_OVERWRITE_SETTINGS_BIT) +#define DEVLINK_FLASH_OVERWRITE_IDENTIFIERS _BITUL(DEVLINK_FLASH_OVERWRITE_IDENTIFIERS_BIT) + +#define DEVLINK_SUPPORTED_FLASH_OVERWRITE_SECTIONS \ + (_BITUL(__DEVLINK_FLASH_OVERWRITE_MAX_BIT) - 1) + +/** + * enum devlink_trap_action - Packet trap action. + * @DEVLINK_TRAP_ACTION_DROP: Packet is dropped by the device and a copy is not + * sent to the CPU. + * @DEVLINK_TRAP_ACTION_TRAP: The sole copy of the packet is sent to the CPU. + * @DEVLINK_TRAP_ACTION_MIRROR: Packet is forwarded by the device and a copy is + * sent to the CPU. + */ +enum devlink_trap_action { + DEVLINK_TRAP_ACTION_DROP, + DEVLINK_TRAP_ACTION_TRAP, + DEVLINK_TRAP_ACTION_MIRROR, +}; + +/** + * enum devlink_trap_type - Packet trap type. + * @DEVLINK_TRAP_TYPE_DROP: Trap reason is a drop. Trapped packets are only + * processed by devlink and not injected to the + * kernel's Rx path. + * @DEVLINK_TRAP_TYPE_EXCEPTION: Trap reason is an exception. Packet was not + * forwarded as intended due to an exception + * (e.g., missing neighbour entry) and trapped to + * control plane for resolution. Trapped packets + * are processed by devlink and injected to + * the kernel's Rx path. + * @DEVLINK_TRAP_TYPE_CONTROL: Packet was trapped because it is required for + * the correct functioning of the control plane. + * For example, an ARP request packet. Trapped + * packets are injected to the kernel's Rx path, + * but not reported to drop monitor. + */ +enum devlink_trap_type { + DEVLINK_TRAP_TYPE_DROP, + DEVLINK_TRAP_TYPE_EXCEPTION, + DEVLINK_TRAP_TYPE_CONTROL, +}; + +enum { + /* Trap can report input port as metadata */ + DEVLINK_ATTR_TRAP_METADATA_TYPE_IN_PORT, + /* Trap can report flow action cookie as metadata */ + DEVLINK_ATTR_TRAP_METADATA_TYPE_FA_COOKIE, +}; + +enum devlink_reload_action { + DEVLINK_RELOAD_ACTION_UNSPEC, + DEVLINK_RELOAD_ACTION_DRIVER_REINIT, /* Driver entities re-instantiation */ + DEVLINK_RELOAD_ACTION_FW_ACTIVATE, /* FW activate */ + + /* Add new reload actions above */ + __DEVLINK_RELOAD_ACTION_MAX, + DEVLINK_RELOAD_ACTION_MAX = __DEVLINK_RELOAD_ACTION_MAX - 1 +}; + +enum devlink_reload_limit { + DEVLINK_RELOAD_LIMIT_UNSPEC, /* unspecified, no constraints */ + DEVLINK_RELOAD_LIMIT_NO_RESET, /* No reset allowed, no down time allowed, + * no link flap and no configuration is lost. + */ + + /* Add new reload limit above */ + __DEVLINK_RELOAD_LIMIT_MAX, + DEVLINK_RELOAD_LIMIT_MAX = __DEVLINK_RELOAD_LIMIT_MAX - 1 +}; + +#define DEVLINK_RELOAD_LIMITS_VALID_MASK (_BITUL(__DEVLINK_RELOAD_LIMIT_MAX) - 1) + +enum devlink_attr { + /* don't change the order or add anything between, this is ABI! */ + DEVLINK_ATTR_UNSPEC, + + /* bus name + dev name together are a handle for devlink entity */ + DEVLINK_ATTR_BUS_NAME, /* string */ + DEVLINK_ATTR_DEV_NAME, /* string */ + + DEVLINK_ATTR_PORT_INDEX, /* u32 */ + DEVLINK_ATTR_PORT_TYPE, /* u16 */ + DEVLINK_ATTR_PORT_DESIRED_TYPE, /* u16 */ + DEVLINK_ATTR_PORT_NETDEV_IFINDEX, /* u32 */ + DEVLINK_ATTR_PORT_NETDEV_NAME, /* string */ + DEVLINK_ATTR_PORT_IBDEV_NAME, /* string */ + DEVLINK_ATTR_PORT_SPLIT_COUNT, /* u32 */ + DEVLINK_ATTR_PORT_SPLIT_GROUP, /* u32 */ + DEVLINK_ATTR_SB_INDEX, /* u32 */ + DEVLINK_ATTR_SB_SIZE, /* u32 */ + DEVLINK_ATTR_SB_INGRESS_POOL_COUNT, /* u16 */ + DEVLINK_ATTR_SB_EGRESS_POOL_COUNT, /* u16 */ + DEVLINK_ATTR_SB_INGRESS_TC_COUNT, /* u16 */ + DEVLINK_ATTR_SB_EGRESS_TC_COUNT, /* u16 */ + DEVLINK_ATTR_SB_POOL_INDEX, /* u16 */ + DEVLINK_ATTR_SB_POOL_TYPE, /* u8 */ + DEVLINK_ATTR_SB_POOL_SIZE, /* u32 */ + DEVLINK_ATTR_SB_POOL_THRESHOLD_TYPE, /* u8 */ + DEVLINK_ATTR_SB_THRESHOLD, /* u32 */ + DEVLINK_ATTR_SB_TC_INDEX, /* u16 */ + DEVLINK_ATTR_SB_OCC_CUR, /* u32 */ + DEVLINK_ATTR_SB_OCC_MAX, /* u32 */ + DEVLINK_ATTR_ESWITCH_MODE, /* u16 */ + DEVLINK_ATTR_ESWITCH_INLINE_MODE, /* u8 */ + + DEVLINK_ATTR_DPIPE_TABLES, /* nested */ + DEVLINK_ATTR_DPIPE_TABLE, /* nested */ + DEVLINK_ATTR_DPIPE_TABLE_NAME, /* string */ + DEVLINK_ATTR_DPIPE_TABLE_SIZE, /* u64 */ + DEVLINK_ATTR_DPIPE_TABLE_MATCHES, /* nested */ + DEVLINK_ATTR_DPIPE_TABLE_ACTIONS, /* nested */ + DEVLINK_ATTR_DPIPE_TABLE_COUNTERS_ENABLED, /* u8 */ + + DEVLINK_ATTR_DPIPE_ENTRIES, /* nested */ + DEVLINK_ATTR_DPIPE_ENTRY, /* nested */ + DEVLINK_ATTR_DPIPE_ENTRY_INDEX, /* u64 */ + DEVLINK_ATTR_DPIPE_ENTRY_MATCH_VALUES, /* nested */ + DEVLINK_ATTR_DPIPE_ENTRY_ACTION_VALUES, /* nested */ + DEVLINK_ATTR_DPIPE_ENTRY_COUNTER, /* u64 */ + + DEVLINK_ATTR_DPIPE_MATCH, /* nested */ + DEVLINK_ATTR_DPIPE_MATCH_VALUE, /* nested */ + DEVLINK_ATTR_DPIPE_MATCH_TYPE, /* u32 */ + + DEVLINK_ATTR_DPIPE_ACTION, /* nested */ + DEVLINK_ATTR_DPIPE_ACTION_VALUE, /* nested */ + DEVLINK_ATTR_DPIPE_ACTION_TYPE, /* u32 */ + + DEVLINK_ATTR_DPIPE_VALUE, + DEVLINK_ATTR_DPIPE_VALUE_MASK, + DEVLINK_ATTR_DPIPE_VALUE_MAPPING, /* u32 */ + + DEVLINK_ATTR_DPIPE_HEADERS, /* nested */ + DEVLINK_ATTR_DPIPE_HEADER, /* nested */ + DEVLINK_ATTR_DPIPE_HEADER_NAME, /* string */ + DEVLINK_ATTR_DPIPE_HEADER_ID, /* u32 */ + DEVLINK_ATTR_DPIPE_HEADER_FIELDS, /* nested */ + DEVLINK_ATTR_DPIPE_HEADER_GLOBAL, /* u8 */ + DEVLINK_ATTR_DPIPE_HEADER_INDEX, /* u32 */ + + DEVLINK_ATTR_DPIPE_FIELD, /* nested */ + DEVLINK_ATTR_DPIPE_FIELD_NAME, /* string */ + DEVLINK_ATTR_DPIPE_FIELD_ID, /* u32 */ + DEVLINK_ATTR_DPIPE_FIELD_BITWIDTH, /* u32 */ + DEVLINK_ATTR_DPIPE_FIELD_MAPPING_TYPE, /* u32 */ + + DEVLINK_ATTR_PAD, + + DEVLINK_ATTR_ESWITCH_ENCAP_MODE, /* u8 */ + DEVLINK_ATTR_RESOURCE_LIST, /* nested */ + DEVLINK_ATTR_RESOURCE, /* nested */ + DEVLINK_ATTR_RESOURCE_NAME, /* string */ + DEVLINK_ATTR_RESOURCE_ID, /* u64 */ + DEVLINK_ATTR_RESOURCE_SIZE, /* u64 */ + DEVLINK_ATTR_RESOURCE_SIZE_NEW, /* u64 */ + DEVLINK_ATTR_RESOURCE_SIZE_VALID, /* u8 */ + DEVLINK_ATTR_RESOURCE_SIZE_MIN, /* u64 */ + DEVLINK_ATTR_RESOURCE_SIZE_MAX, /* u64 */ + DEVLINK_ATTR_RESOURCE_SIZE_GRAN, /* u64 */ + DEVLINK_ATTR_RESOURCE_UNIT, /* u8 */ + DEVLINK_ATTR_RESOURCE_OCC, /* u64 */ + DEVLINK_ATTR_DPIPE_TABLE_RESOURCE_ID, /* u64 */ + DEVLINK_ATTR_DPIPE_TABLE_RESOURCE_UNITS,/* u64 */ + + DEVLINK_ATTR_PORT_FLAVOUR, /* u16 */ + DEVLINK_ATTR_PORT_NUMBER, /* u32 */ + DEVLINK_ATTR_PORT_SPLIT_SUBPORT_NUMBER, /* u32 */ + + DEVLINK_ATTR_PARAM, /* nested */ + DEVLINK_ATTR_PARAM_NAME, /* string */ + DEVLINK_ATTR_PARAM_GENERIC, /* flag */ + DEVLINK_ATTR_PARAM_TYPE, /* u8 */ + DEVLINK_ATTR_PARAM_VALUES_LIST, /* nested */ + DEVLINK_ATTR_PARAM_VALUE, /* nested */ + DEVLINK_ATTR_PARAM_VALUE_DATA, /* dynamic */ + DEVLINK_ATTR_PARAM_VALUE_CMODE, /* u8 */ + + DEVLINK_ATTR_REGION_NAME, /* string */ + DEVLINK_ATTR_REGION_SIZE, /* u64 */ + DEVLINK_ATTR_REGION_SNAPSHOTS, /* nested */ + DEVLINK_ATTR_REGION_SNAPSHOT, /* nested */ + DEVLINK_ATTR_REGION_SNAPSHOT_ID, /* u32 */ + + DEVLINK_ATTR_REGION_CHUNKS, /* nested */ + DEVLINK_ATTR_REGION_CHUNK, /* nested */ + DEVLINK_ATTR_REGION_CHUNK_DATA, /* binary */ + DEVLINK_ATTR_REGION_CHUNK_ADDR, /* u64 */ + DEVLINK_ATTR_REGION_CHUNK_LEN, /* u64 */ + + DEVLINK_ATTR_INFO_DRIVER_NAME, /* string */ + DEVLINK_ATTR_INFO_SERIAL_NUMBER, /* string */ + DEVLINK_ATTR_INFO_VERSION_FIXED, /* nested */ + DEVLINK_ATTR_INFO_VERSION_RUNNING, /* nested */ + DEVLINK_ATTR_INFO_VERSION_STORED, /* nested */ + DEVLINK_ATTR_INFO_VERSION_NAME, /* string */ + DEVLINK_ATTR_INFO_VERSION_VALUE, /* string */ + + DEVLINK_ATTR_SB_POOL_CELL_SIZE, /* u32 */ + + DEVLINK_ATTR_FMSG, /* nested */ + DEVLINK_ATTR_FMSG_OBJ_NEST_START, /* flag */ + DEVLINK_ATTR_FMSG_PAIR_NEST_START, /* flag */ + DEVLINK_ATTR_FMSG_ARR_NEST_START, /* flag */ + DEVLINK_ATTR_FMSG_NEST_END, /* flag */ + DEVLINK_ATTR_FMSG_OBJ_NAME, /* string */ + DEVLINK_ATTR_FMSG_OBJ_VALUE_TYPE, /* u8 */ + DEVLINK_ATTR_FMSG_OBJ_VALUE_DATA, /* dynamic */ + + DEVLINK_ATTR_HEALTH_REPORTER, /* nested */ + DEVLINK_ATTR_HEALTH_REPORTER_NAME, /* string */ + DEVLINK_ATTR_HEALTH_REPORTER_STATE, /* u8 */ + DEVLINK_ATTR_HEALTH_REPORTER_ERR_COUNT, /* u64 */ + DEVLINK_ATTR_HEALTH_REPORTER_RECOVER_COUNT, /* u64 */ + DEVLINK_ATTR_HEALTH_REPORTER_DUMP_TS, /* u64 */ + DEVLINK_ATTR_HEALTH_REPORTER_GRACEFUL_PERIOD, /* u64 */ + DEVLINK_ATTR_HEALTH_REPORTER_AUTO_RECOVER, /* u8 */ + + DEVLINK_ATTR_FLASH_UPDATE_FILE_NAME, /* string */ + DEVLINK_ATTR_FLASH_UPDATE_COMPONENT, /* string */ + DEVLINK_ATTR_FLASH_UPDATE_STATUS_MSG, /* string */ + DEVLINK_ATTR_FLASH_UPDATE_STATUS_DONE, /* u64 */ + DEVLINK_ATTR_FLASH_UPDATE_STATUS_TOTAL, /* u64 */ + + DEVLINK_ATTR_PORT_PCI_PF_NUMBER, /* u16 */ + DEVLINK_ATTR_PORT_PCI_VF_NUMBER, /* u16 */ + + DEVLINK_ATTR_STATS, /* nested */ + + DEVLINK_ATTR_TRAP_NAME, /* string */ + /* enum devlink_trap_action */ + DEVLINK_ATTR_TRAP_ACTION, /* u8 */ + /* enum devlink_trap_type */ + DEVLINK_ATTR_TRAP_TYPE, /* u8 */ + DEVLINK_ATTR_TRAP_GENERIC, /* flag */ + DEVLINK_ATTR_TRAP_METADATA, /* nested */ + DEVLINK_ATTR_TRAP_GROUP_NAME, /* string */ + + DEVLINK_ATTR_RELOAD_FAILED, /* u8 0 or 1 */ + + DEVLINK_ATTR_HEALTH_REPORTER_DUMP_TS_NS, /* u64 */ + + DEVLINK_ATTR_NETNS_FD, /* u32 */ + DEVLINK_ATTR_NETNS_PID, /* u32 */ + DEVLINK_ATTR_NETNS_ID, /* u32 */ + + DEVLINK_ATTR_HEALTH_REPORTER_AUTO_DUMP, /* u8 */ + + DEVLINK_ATTR_TRAP_POLICER_ID, /* u32 */ + DEVLINK_ATTR_TRAP_POLICER_RATE, /* u64 */ + DEVLINK_ATTR_TRAP_POLICER_BURST, /* u64 */ + + DEVLINK_ATTR_PORT_FUNCTION, /* nested */ + + DEVLINK_ATTR_INFO_BOARD_SERIAL_NUMBER, /* string */ + + DEVLINK_ATTR_PORT_LANES, /* u32 */ + DEVLINK_ATTR_PORT_SPLITTABLE, /* u8 */ + + DEVLINK_ATTR_PORT_EXTERNAL, /* u8 */ + DEVLINK_ATTR_PORT_CONTROLLER_NUMBER, /* u32 */ + + DEVLINK_ATTR_FLASH_UPDATE_STATUS_TIMEOUT, /* u64 */ + DEVLINK_ATTR_FLASH_UPDATE_OVERWRITE_MASK, /* bitfield32 */ + + DEVLINK_ATTR_RELOAD_ACTION, /* u8 */ + DEVLINK_ATTR_RELOAD_ACTIONS_PERFORMED, /* bitfield32 */ + DEVLINK_ATTR_RELOAD_LIMITS, /* bitfield32 */ + + DEVLINK_ATTR_DEV_STATS, /* nested */ + DEVLINK_ATTR_RELOAD_STATS, /* nested */ + DEVLINK_ATTR_RELOAD_STATS_ENTRY, /* nested */ + DEVLINK_ATTR_RELOAD_STATS_LIMIT, /* u8 */ + DEVLINK_ATTR_RELOAD_STATS_VALUE, /* u32 */ + DEVLINK_ATTR_REMOTE_RELOAD_STATS, /* nested */ + DEVLINK_ATTR_RELOAD_ACTION_INFO, /* nested */ + DEVLINK_ATTR_RELOAD_ACTION_STATS, /* nested */ + + DEVLINK_ATTR_PORT_PCI_SF_NUMBER, /* u32 */ + /* add new attributes above here, update the policy in devlink.c */ + + __DEVLINK_ATTR_MAX, + DEVLINK_ATTR_MAX = __DEVLINK_ATTR_MAX - 1 +}; + +/* Mapping between internal resource described by the field and system + * structure + */ +enum devlink_dpipe_field_mapping_type { + DEVLINK_DPIPE_FIELD_MAPPING_TYPE_NONE, + DEVLINK_DPIPE_FIELD_MAPPING_TYPE_IFINDEX, +}; + +/* Match type - specify the type of the match */ +enum devlink_dpipe_match_type { + DEVLINK_DPIPE_MATCH_TYPE_FIELD_EXACT, +}; + +/* Action type - specify the action type */ +enum devlink_dpipe_action_type { + DEVLINK_DPIPE_ACTION_TYPE_FIELD_MODIFY, +}; + +enum devlink_dpipe_field_ethernet_id { + DEVLINK_DPIPE_FIELD_ETHERNET_DST_MAC, +}; + +enum devlink_dpipe_field_ipv4_id { + DEVLINK_DPIPE_FIELD_IPV4_DST_IP, +}; + +enum devlink_dpipe_field_ipv6_id { + DEVLINK_DPIPE_FIELD_IPV6_DST_IP, +}; + +enum devlink_dpipe_header_id { + DEVLINK_DPIPE_HEADER_ETHERNET, + DEVLINK_DPIPE_HEADER_IPV4, + DEVLINK_DPIPE_HEADER_IPV6, +}; + +enum devlink_resource_unit { + DEVLINK_RESOURCE_UNIT_ENTRY, +}; + +enum devlink_port_function_attr { + DEVLINK_PORT_FUNCTION_ATTR_UNSPEC, + DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR, /* binary */ + DEVLINK_PORT_FN_ATTR_STATE, /* u8 */ + DEVLINK_PORT_FN_ATTR_OPSTATE, /* u8 */ + + __DEVLINK_PORT_FUNCTION_ATTR_MAX, + DEVLINK_PORT_FUNCTION_ATTR_MAX = __DEVLINK_PORT_FUNCTION_ATTR_MAX - 1 +}; + +enum devlink_port_fn_state { + DEVLINK_PORT_FN_STATE_INACTIVE, + DEVLINK_PORT_FN_STATE_ACTIVE, +}; + +/** + * enum devlink_port_fn_opstate - indicates operational state of the function + * @DEVLINK_PORT_FN_OPSTATE_ATTACHED: Driver is attached to the function. + * For graceful tear down of the function, after inactivation of the + * function, user should wait for operational state to turn DETACHED. + * @DEVLINK_PORT_FN_OPSTATE_DETACHED: Driver is detached from the function. + * It is safe to delete the port. + */ +enum devlink_port_fn_opstate { + DEVLINK_PORT_FN_OPSTATE_DETACHED, + DEVLINK_PORT_FN_OPSTATE_ATTACHED, +}; + +#endif /* _UAPI_LINUX_DEVLINK_H_ */ +#endif /* __UAPI_LINUX_DEVLINK_WRAPPER_H */ +#endif /* !__KERNEL__ */ diff --git a/lib/automake.mk b/lib/automake.mk index 086fbd62d..1057504d1 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -43,6 +43,17 @@ lib_libovn_la_SOURCES = \ lib/plug.c \ lib/plug-dummy.h \ lib/plug-dummy.c + +# in-tree plug providers +if ENABLE_PLUG +if LINUX +lib_libovn_la_SOURCES += \ + lib/plug_providers/representor/netlink-devlink.h \ + lib/plug_providers/representor/netlink-devlink.c \ + lib/plug_providers/representor/plug-representor.c +endif +endif + nodist_lib_libovn_la_SOURCES = \ lib/ovn-dirs.c \ lib/ovn-nb-idl.c \ diff --git a/lib/plug-provider.h b/lib/plug-provider.h index 487534ee5..6587be8dc 100644 --- a/lib/plug-provider.h +++ b/lib/plug-provider.h @@ -87,9 +87,13 @@ struct plug_class { }; extern const struct plug_class plug_dummy_class; +#ifdef ENABLE_PLUG +/* in-tree plug classes */ +extern const struct plug_class plug_representor; +#endif /* ENABLE_PLUG */ #ifdef HAVE_PLUG_PROVIDER extern const struct plug_class *plug_provider_classes[]; -#endif +#endif /* HAVE_PLUG_PROVIDER */ #ifdef __cplusplus } diff --git a/lib/plug.c b/lib/plug.c index c0c34214e..dab06713a 100644 --- a/lib/plug.c +++ b/lib/plug.c @@ -32,6 +32,7 @@ VLOG_DEFINE_THIS_MODULE(plug); #ifdef ENABLE_PLUG static const struct plug_class *base_plug_classes[] = { + &plug_representor, }; #endif diff --git a/lib/plug_providers/representor/netlink-devlink.c b/lib/plug_providers/representor/netlink-devlink.c new file mode 100644 index 000000000..82e9c71f3 --- /dev/null +++ b/lib/plug_providers/representor/netlink-devlink.c @@ -0,0 +1,499 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include +#include +#include +#include +#include +#include "netlink.h" +#include "netlink-socket.h" +#include "netlink-devlink.h" +#include "openvswitch/vlog.h" +#include "packets.h" + +VLOG_DEFINE_THIS_MODULE(netlink_devlink); + +/* Initialized by nl_devlink_init() */ +static int ovs_devlink_family; + +struct nl_dl_dump_state { + struct nl_dump dump; + struct ofpbuf buf; + int error; +}; + +static int nl_devlink_init(void); + +const char *dl_str_not_present = ""; + +/* Allocates memory for and returns a pointer to devlink dump state object. + * + * One-time initialization and lookup of the devlink generic netlink family is + * also performed, and the caller should check for error condition with a call + * to nl_dl_dump_init_error before attempting to dump devlink data. + * + * The caller owns the allocated object and is responsible for freeing the + * allocated memory with a call to nl_dl_dump_destroy when done. */ +struct nl_dl_dump_state * +nl_dl_dump_init(void) +{ + struct nl_dl_dump_state *dump_state; + + dump_state = xmalloc(sizeof(*dump_state)); + dump_state->error = nl_devlink_init(); + return dump_state; +} + +/* Get error indicator from the devlink initialization process. */ +int +nl_dl_dump_init_error(struct nl_dl_dump_state *dump_state) +{ + return dump_state->error; +} + +/* Free memory previously allocated by call to nl_dl_dump_init. + * + * Note that the caller is responsible for making a call to nl_dl_dump_finish + * to free up resources associated with any in-flight dump process prior to + * destroying the dump state object. */ +void +nl_dl_dump_destroy(struct nl_dl_dump_state *dump_state) +{ + free(dump_state); +} + +void +nl_msg_put_dlgenmsg(struct ofpbuf *msg, size_t expected_payload, + int family, uint8_t cmd, uint32_t flags) +{ + nl_msg_put_genlmsghdr(msg, expected_payload, family, + flags, cmd, DEVLINK_GENL_VERSION); +} + +/* Starts a Netlink-devlink "dump" operation, by sending devlink request with + * command 'cmd' to the kernel on a Netlink socket, and initializes 'state' + * with buffer and dump state. */ +void +nl_dl_dump_start(uint8_t cmd, struct nl_dl_dump_state *state) +{ + struct ofpbuf *request; + + request = ofpbuf_new(NLMSG_HDRLEN + GENL_HDRLEN); + nl_msg_put_dlgenmsg(request, 0, ovs_devlink_family, cmd, + NLM_F_REQUEST); + nl_dump_start(&state->dump, NETLINK_GENERIC, request); + ofpbuf_delete(request); + + ofpbuf_init(&state->buf, NL_DUMP_BUFSIZE); +} + +static bool +nl_dl_dump_next__(struct nl_dl_dump_state *state, + bool (*parse_function)(struct ofpbuf *, void *), + void *entry) +{ + struct ofpbuf msg; + + if (!nl_dump_next(&state->dump, &msg, &state->buf)) { + return false; + } + if (!parse_function(&msg, entry)) { + ovs_mutex_lock(&state->dump.mutex); + state->dump.status = EPROTO; + ovs_mutex_unlock(&state->dump.mutex); + return false; + } + return true; +} + +/* Attempts to retrieve and parse another reply in on-going dump operation. + * + * If successful, returns true and assignes values or pointers to data in + * 'port_entry'. The caller must not modify 'port_entry' (because it may + * contain pointers to data within the buffer which will be used by future + * calls to this function. + * + * On failure, returns false. Failure might indicate an actual error or merely + * the end of replies. An error status for the entire dump operation is + * provided when it is completed by calling nl_dl_dump_finish() + */ +bool +nl_dl_port_dump_next(struct nl_dl_dump_state *state, + struct dl_port *port_entry) +{ + return nl_dl_dump_next__( + state, + (bool ( * )(struct ofpbuf *, void *)) &nl_dl_parse_port_policy, + (void *) port_entry); +} + +bool +nl_dl_info_dump_next(struct nl_dl_dump_state *state, + struct dl_info *info_entry) +{ + return nl_dl_dump_next__( + state, + (bool ( * )(struct ofpbuf *, void *)) &nl_dl_parse_info_policy, + (void *) info_entry); +} + +int +nl_dl_dump_finish(struct nl_dl_dump_state *state) +{ + ofpbuf_uninit(&state->buf); + return nl_dump_done(&state->dump); +} + +static uint64_t +attr_get_up_to_u64(size_t attr_idx, struct nlattr *attrs[], + const struct nl_policy policy[], + size_t policy_len) +{ + if (attr_idx < policy_len && attrs[attr_idx]) { + switch (policy[attr_idx].type) { + case NL_A_U8: + return nl_attr_get_u8(attrs[attr_idx]); + break; + case NL_A_U16: + return nl_attr_get_u16(attrs[attr_idx]); + break; + case NL_A_U32: + return nl_attr_get_u32(attrs[attr_idx]); + break; + case NL_A_U64: + return nl_attr_get_u64(attrs[attr_idx]); + break; + case NL_A_U128: + case NL_A_STRING: + case NL_A_NO_ATTR: + case NL_A_UNSPEC: + case NL_A_FLAG: + case NL_A_IPV6: + case NL_A_NESTED: + case NL_A_LL_ADDR: + case N_NL_ATTR_TYPES: default: OVS_NOT_REACHED(); + }; + } + return -1; +} + +static const char * +attr_get_str(size_t attr_idx, struct nlattr *attrs[], + const struct nl_policy policy[], + size_t policy_len) +{ + if (attr_idx < policy_len && attrs[attr_idx]) { + ovs_assert(policy[attr_idx].type == NL_A_STRING); + return nl_attr_get_string(attrs[attr_idx]); + } + return dl_str_not_present; +} + +bool +nl_dl_parse_port_function(struct nlattr *nla, struct dl_port_function *port_fn) +{ + static const struct nl_policy policy[] = { + /* Appeared in Linux v5.9 */ + [DEVLINK_PORT_FUNCTION_ATTR_UNSPEC] = { .type = NL_A_UNSPEC, + .optional = true, }, + [DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR] = { .type = NL_A_LL_ADDR, + .optional = true, }, + + /* Appeared in Linnux v5.12 */ + [DEVLINK_PORT_FN_ATTR_STATE] = { .type = NL_A_U8, .optional = true, }, + [DEVLINK_PORT_FN_ATTR_OPSTATE] = { .type = NL_A_U8, + .optional = true, }, + }; + struct nlattr *attrs[ARRAY_SIZE(policy)]; + bool parsed; + + parsed = nl_parse_nested(nla, policy, attrs, ARRAY_SIZE(policy)); + + if (parsed) { + if (attrs[DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR]) { + size_t hw_addr_size = nl_attr_get_size( + attrs[DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR]); + if (hw_addr_size == sizeof(struct eth_addr)) { + port_fn->eth_addr = nl_attr_get_eth_addr( + attrs[DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR]); + } else if (hw_addr_size == sizeof(struct ib_addr)) { + port_fn->ib_addr = nl_attr_get_ib_addr( + attrs[DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR]); + } else { + return false; + } + } else { + memset(&port_fn->eth_addr, 0, sizeof(port_fn->eth_addr)); + memset(&port_fn->ib_addr, 0, sizeof(port_fn->ib_addr)); + } + port_fn->state = attr_get_up_to_u64( + DEVLINK_PORT_FN_ATTR_STATE, + attrs, policy, ARRAY_SIZE(policy)); + port_fn->opstate = attr_get_up_to_u64( + DEVLINK_PORT_FN_ATTR_OPSTATE, + attrs, policy, ARRAY_SIZE(policy)); + } + + return parsed; +} + +bool +nl_dl_parse_port_policy(struct ofpbuf *msg, struct dl_port *port) +{ + static const struct nl_policy policy[] = { + /* Appeared in Linux v4.6 */ + [DEVLINK_ATTR_BUS_NAME] = { .type = NL_A_STRING, .optional = false, }, + [DEVLINK_ATTR_DEV_NAME] = { .type = NL_A_STRING, .optional = false, }, + [DEVLINK_ATTR_PORT_INDEX] = { .type = NL_A_U32, .optional = false, }, + + [DEVLINK_ATTR_PORT_TYPE] = { .type = NL_A_U16, .optional = true, }, + [DEVLINK_ATTR_PORT_DESIRED_TYPE] = { .type = NL_A_U16, + .optional = true, }, + [DEVLINK_ATTR_PORT_NETDEV_IFINDEX] = { .type = NL_A_U32, + .optional = true, }, + [DEVLINK_ATTR_PORT_NETDEV_NAME] = { .type = NL_A_STRING, + .optional = true, }, + [DEVLINK_ATTR_PORT_IBDEV_NAME] = { .type = NL_A_STRING, + .optional = true, }, + [DEVLINK_ATTR_PORT_SPLIT_COUNT] = { .type = NL_A_U32, + .optional = true, }, + [DEVLINK_ATTR_PORT_SPLIT_GROUP] = { .type = NL_A_U32, + .optional = true, }, + + /* Appeared in Linux v4.18 */ + [DEVLINK_ATTR_PORT_FLAVOUR] = { .type = NL_A_U16, .optional = true, }, + [DEVLINK_ATTR_PORT_NUMBER] = { .type = NL_A_U32, .optional = true, }, + [DEVLINK_ATTR_PORT_SPLIT_SUBPORT_NUMBER] = { .type = NL_A_U32, + .optional = true, }, + + /* Appeared in Linux v5.3 */ + [DEVLINK_ATTR_PORT_PCI_PF_NUMBER] = { .type = NL_A_U16, + .optional = true, }, + [DEVLINK_ATTR_PORT_PCI_VF_NUMBER] = { .type = NL_A_U16, + .optional = true, }, + + /* Appeared in Linux v5.9 */ + [DEVLINK_ATTR_PORT_FUNCTION] = { .type = NL_A_NESTED, + .optional = true, }, + [DEVLINK_ATTR_PORT_LANES] = { .type = NL_A_U32, .optional = true, }, + [DEVLINK_ATTR_PORT_SPLITTABLE] = { .type = NL_A_U8, + .optional = true, }, + + /* Appeared in Linux v5.10 */ + [DEVLINK_ATTR_PORT_EXTERNAL] = { .type = NL_A_U8, .optional = true }, + [DEVLINK_ATTR_PORT_CONTROLLER_NUMBER] = { .type = NL_A_U32, + .optional = true}, + + /* Appeared in Linux v5.12 */ + [DEVLINK_ATTR_PORT_PCI_SF_NUMBER] = { .type = NL_A_U32, + .optional = true }, + }; + struct nlattr *attrs[ARRAY_SIZE(policy)]; + + if (!nl_policy_parse(msg, NLMSG_HDRLEN + GENL_HDRLEN, + policy, attrs, + ARRAY_SIZE(policy))) + { + return false; + } + port->bus_name = nl_attr_get_string(attrs[DEVLINK_ATTR_BUS_NAME]); + port->dev_name = nl_attr_get_string(attrs[DEVLINK_ATTR_DEV_NAME]); + port->index = nl_attr_get_u32(attrs[DEVLINK_ATTR_PORT_INDEX]); + + port->type = attr_get_up_to_u64( + DEVLINK_ATTR_PORT_TYPE, + attrs, policy, ARRAY_SIZE(policy)); + port->desired_type = attr_get_up_to_u64( + DEVLINK_ATTR_PORT_DESIRED_TYPE, + attrs, policy, ARRAY_SIZE(policy)); + port->netdev_ifindex = attr_get_up_to_u64( + DEVLINK_ATTR_PORT_NETDEV_IFINDEX, + attrs, policy, ARRAY_SIZE(policy)); + if (port->type == DEVLINK_PORT_TYPE_ETH && + attrs[DEVLINK_ATTR_PORT_NETDEV_NAME]) { + port->netdev_name = nl_attr_get_string( + attrs[DEVLINK_ATTR_PORT_NETDEV_NAME]); + } else if (port->type == DEVLINK_PORT_TYPE_IB && + attrs[DEVLINK_ATTR_PORT_IBDEV_NAME]) { + port->ibdev_name = nl_attr_get_string( + attrs[DEVLINK_ATTR_PORT_IBDEV_NAME]); + } else { + port->netdev_name = dl_str_not_present; + } + port->split_count = attr_get_up_to_u64( + DEVLINK_ATTR_PORT_SPLIT_COUNT, + attrs, policy, ARRAY_SIZE(policy)); + port->split_group = attr_get_up_to_u64( + DEVLINK_ATTR_PORT_SPLIT_GROUP, + attrs, policy, ARRAY_SIZE(policy)); + port->flavour = attr_get_up_to_u64( + DEVLINK_ATTR_PORT_FLAVOUR, + attrs, policy, ARRAY_SIZE(policy)); + port->number = attr_get_up_to_u64( + DEVLINK_ATTR_PORT_NUMBER, + attrs, policy, ARRAY_SIZE(policy)); + port->split_subport_number = attr_get_up_to_u64( + DEVLINK_ATTR_PORT_SPLIT_SUBPORT_NUMBER, + attrs, policy, ARRAY_SIZE(policy)); + port->pci_pf_number = attr_get_up_to_u64( + DEVLINK_ATTR_PORT_PCI_PF_NUMBER, + attrs, policy, ARRAY_SIZE(policy)); + port->pci_vf_number = attr_get_up_to_u64( + DEVLINK_ATTR_PORT_PCI_VF_NUMBER, + attrs, policy, ARRAY_SIZE(policy)); + port->lanes = attr_get_up_to_u64( + DEVLINK_ATTR_PORT_LANES, + attrs, policy, ARRAY_SIZE(policy)); + port->splittable = attr_get_up_to_u64( + DEVLINK_ATTR_PORT_SPLITTABLE, + attrs, policy, ARRAY_SIZE(policy)); + port->external = attr_get_up_to_u64( + DEVLINK_ATTR_PORT_EXTERNAL, + attrs, policy, ARRAY_SIZE(policy)); + port->controller_number = attr_get_up_to_u64( + DEVLINK_ATTR_PORT_CONTROLLER_NUMBER, + attrs, policy, ARRAY_SIZE(policy)); + port->pci_sf_number = attr_get_up_to_u64( + DEVLINK_ATTR_PORT_PCI_SF_NUMBER, + attrs, policy, ARRAY_SIZE(policy)); + + if (attrs[DEVLINK_ATTR_PORT_FUNCTION]) { + if (!nl_dl_parse_port_function(attrs[DEVLINK_ATTR_PORT_FUNCTION], + &port->function)) + { + return false; + } + } else { + memset(&port->function, 0, sizeof(port->function)); + port->function.state = UINT8_MAX; + port->function.opstate = UINT8_MAX; + } + + return true; +} + +bool +nl_dl_parse_info_version(struct nlattr *nla, struct dl_info_version *info_ver) +{ + static const struct nl_policy policy[] = { + /* Appeared in Linux v5.1 */ + [DEVLINK_ATTR_INFO_VERSION_NAME] = { .type = NL_A_STRING, + .optional = true, }, + [DEVLINK_ATTR_INFO_VERSION_VALUE] = { .type = NL_A_STRING, + .optional = true, }, + }; + struct nlattr *attrs[ARRAY_SIZE(policy)]; + bool parsed; + + parsed = nl_parse_nested(nla, policy, attrs, ARRAY_SIZE(policy)); + + if (parsed) { + info_ver->name = attr_get_str( + DEVLINK_ATTR_INFO_VERSION_NAME, + attrs, policy, ARRAY_SIZE(policy)); + info_ver->value = attr_get_str( + DEVLINK_ATTR_INFO_VERSION_NAME, + attrs, policy, ARRAY_SIZE(policy)); + } + + return parsed; +} + +static bool +attr_fill_version(size_t attr_idx, struct nlattr *attrs[], + size_t attrs_len, + struct dl_info_version *version) +{ + if (attr_idx < attrs_len && attrs[attr_idx]) { + if (!nl_dl_parse_info_version(attrs[attr_idx], + version)) + { + return false; + } + } else { + version->name = dl_str_not_present; + version->value = dl_str_not_present; + } + return true; +} + +bool +nl_dl_parse_info_policy(struct ofpbuf *msg, struct dl_info *info) +{ + static const struct nl_policy policy[] = { + /* Appeared in Linux v5.1 */ + [DEVLINK_ATTR_INFO_DRIVER_NAME] = { .type = NL_A_STRING, + .optional = false, }, + [DEVLINK_ATTR_INFO_SERIAL_NUMBER] = { .type = NL_A_STRING, + .optional = true, }, + [DEVLINK_ATTR_INFO_VERSION_FIXED] = { .type = NL_A_NESTED, + .optional = true, }, + [DEVLINK_ATTR_INFO_VERSION_RUNNING] = { .type = NL_A_NESTED, + .optional = true, }, + [DEVLINK_ATTR_INFO_VERSION_STORED] = { .type = NL_A_NESTED, + .optional = true, }, + + /* Appeared in Linux v5.9 */ + [DEVLINK_ATTR_INFO_BOARD_SERIAL_NUMBER] = { .type = NL_A_STRING, + .optional = true, }, + }; + struct nlattr *attrs[ARRAY_SIZE(policy)]; + + if (!nl_policy_parse(msg, NLMSG_HDRLEN + GENL_HDRLEN, + policy, attrs, + ARRAY_SIZE(policy))) + { + return false; + } + info->driver_name = attr_get_str( + DEVLINK_ATTR_INFO_DRIVER_NAME, + attrs, policy, ARRAY_SIZE(policy)); + info->serial_number = attr_get_str( + DEVLINK_ATTR_INFO_SERIAL_NUMBER, + attrs, policy, ARRAY_SIZE(policy)); + info->board_serial_number = attr_get_str( + DEVLINK_ATTR_INFO_BOARD_SERIAL_NUMBER, + attrs, policy, ARRAY_SIZE(policy)); + if (!attr_fill_version(DEVLINK_ATTR_INFO_VERSION_FIXED, attrs, + ARRAY_SIZE(policy), &info->version_fixed) + || !attr_fill_version(DEVLINK_ATTR_INFO_VERSION_RUNNING, attrs, + ARRAY_SIZE(policy), &info->version_running) + || !attr_fill_version(DEVLINK_ATTR_INFO_VERSION_STORED, attrs, + ARRAY_SIZE(policy), &info->version_stored)) + { + return false; + } + + return true; +} + +static int +nl_devlink_init(void) +{ + static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; + static int error; + + if (ovsthread_once_start(&once)) { + error = nl_lookup_genl_family(DEVLINK_GENL_NAME, &ovs_devlink_family); + if (error) { + VLOG_INFO("Generic Netlink family '%s' does not exist. " + "Linux version 4.6 or newer required.", + DEVLINK_GENL_NAME); + } + ovsthread_once_done(&once); + } + return error; +} diff --git a/lib/plug_providers/representor/netlink-devlink.h b/lib/plug_providers/representor/netlink-devlink.h new file mode 100644 index 000000000..a7b108435 --- /dev/null +++ b/lib/plug_providers/representor/netlink-devlink.h @@ -0,0 +1,115 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef NETLINK_DEVLINK_H +#define NETLINK_DEVLINK_H 1 + +/* Presence of each individual value in these structs is determined at runtime + * and depends on which kernel version we are communicating with as well as + * which driver implementation is filling in the information for each + * individual device or port. + * + * To signal non-presence of values the library follows the following + * convention: + * + * - integer type values will be set to their maximum value + * (e.g. UNIT8_MAX for a unit8_t) + * + * - hardware address type values will be set to all zero + * + * - string type values will be set to a pointer to dl_str_not_present + * (an empty string). + */ + +extern const char *dl_str_not_present; + +struct dl_port_function { + struct eth_addr eth_addr; + struct ib_addr ib_addr; + uint8_t state; + uint8_t opstate; +}; + +struct dl_port { + const char *bus_name; + const char *dev_name; + uint32_t index; + uint16_t type; + uint16_t desired_type; + uint32_t netdev_ifindex; + union { + const char *netdev_name; /* type DEVLINK_PORT_TYPE_ETH */ + const char *ibdev_name; /* type DEVLINK_PORT_TYPE_IB */ + }; + uint32_t split_count; + uint32_t split_group; + uint16_t flavour; + uint32_t number; + uint32_t split_subport_number; + uint16_t pci_pf_number; + uint16_t pci_vf_number; + struct dl_port_function function; + uint32_t lanes; + uint8_t splittable; + uint8_t external; + uint32_t controller_number; + uint32_t pci_sf_number; +}; + +struct dl_info_version { + const char *name; + const char *value; +}; + +struct dl_info { + const char *driver_name; + const char *serial_number; + const char *board_serial_number; + struct dl_info_version version_fixed; + struct dl_info_version version_running; + struct dl_info_version version_stored; +}; + +struct eth_addr nl_attr_get_eth_addr(const struct nlattr *nla); +struct ib_addr nl_attr_get_ib_addr(const struct nlattr *nla); + +/* The nl_dl_dump_state record declaration refers to types declared in + * netlink-socket.h, which requires OVS internal autoconf macros and + * definitions to be present for successful compilation. + * + * To enable friction free consumtion of these interfaces from programs + * external to Open vSwitch, such as OVN, we keep the declaration of + * nl_dl_dump_state private. + * + * Use the nl_dl_dump_init function to allocate memory for and get a pointer to + * a devlink dump state object. The caller owns the allocated object and is + * responsible for freeing the allocated memory when done. */ +struct nl_dl_dump_state; + +struct nl_dl_dump_state * nl_dl_dump_init(void); +int nl_dl_dump_init_error(struct nl_dl_dump_state *); +void nl_dl_dump_destroy(struct nl_dl_dump_state *); +void nl_msg_put_dlgenmsg(struct ofpbuf *, size_t, int, uint8_t, uint32_t); +void nl_dl_dump_start(uint8_t, struct nl_dl_dump_state *); +bool nl_dl_port_dump_next(struct nl_dl_dump_state *, struct dl_port *); +bool nl_dl_info_dump_next(struct nl_dl_dump_state *, struct dl_info *); +int nl_dl_dump_finish(struct nl_dl_dump_state *); +bool nl_dl_parse_port_policy(struct ofpbuf *, struct dl_port *); +bool nl_dl_parse_port_function(struct nlattr *, struct dl_port_function *); +bool nl_dl_parse_info_policy(struct ofpbuf *, struct dl_info *); +bool nl_dl_parse_info_version(struct nlattr *, struct dl_info_version *); + +#endif /* NETLINK_DEVLINK_H */ diff --git a/lib/plug_providers/representor/plug-representor.c b/lib/plug_providers/representor/plug-representor.c new file mode 100644 index 000000000..4c4187114 --- /dev/null +++ b/lib/plug_providers/representor/plug-representor.c @@ -0,0 +1,307 @@ +/* Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include +#include +#include +#include + +#include "plug-provider.h" +#include "plug.h" + +#include "hash.h" +#include "lib/vswitch-idl.h" +#include "openvswitch/hmap.h" +#include "openvswitch/vlog.h" +#include "lib/ovn-sb-idl.h" +#include "netlink-devlink.h" +#include "openvswitch/shash.h" +#include "packets.h" + +VLOG_DEFINE_THIS_MODULE(plug_representor); + +/* Contains netdev name of ports known to devlink indexed by PF MAC + * address and logical function number (if applicable). + * + * Examples: + * SR-IOV Physical Function: key "00:53:00:00:00:42" value "pf0hpf" + * SR-IOV Virtual Function: key "00:53:00:00:00:42-42" value "pf0vf42" + */ +static struct shash devlink_ports; + +/* Max number of physical ports connected to a single NIC SoC. */ +#define MAX_NIC_PHY_PORTS 64 +/* string repr of eth MAC, '-', logical function number (uint32_t) */ +#define MAX_KEY_LEN 17+1+10+1 + +static bool compat_get_host_pf_mac(const char *, struct eth_addr *); + +static bool +fill_devlink_ports_key_from_strs(char *buf, size_t bufsiz, + const char *host_pf_mac, + const char *function) +{ + return snprintf(buf, bufsiz, + function != NULL ? "%s-%s": "%s", + host_pf_mac, function) < bufsiz; +} + +/* We deliberately pass the struct eth_addr by value as we would have to copy + * the data either way to make use of the ETH_ADDR_ARGS macro */ +static bool +fill_devlink_ports_key_from_typed(char *buf, size_t bufsiz, + struct eth_addr host_pf_mac, + uint32_t function) +{ + return snprintf( + buf, bufsiz, + function < UINT32_MAX ? ETH_ADDR_FMT"-%"PRIu32 : ETH_ADDR_FMT, + ETH_ADDR_ARGS(host_pf_mac), function) < bufsiz; +} + +static void +devlink_port_add_function(struct dl_port *port_entry, + struct eth_addr *host_pf_mac) +{ + char keybuf[MAX_KEY_LEN]; + uint32_t function_number; + + switch (port_entry->flavour) { + case DEVLINK_PORT_FLAVOUR_PCI_PF: + /* for Physical Function representor ports we only add the MAC address + * and no logical function number */ + function_number = -1; + break; + case DEVLINK_PORT_FLAVOUR_PCI_VF: + function_number = port_entry->pci_vf_number; + break; + default: + VLOG_WARN("Unsupported flavour for port '%s': %s", + port_entry->netdev_name, + port_entry->flavour == DEVLINK_PORT_FLAVOUR_PHYSICAL ? "PHYSICAL" : + port_entry->flavour == DEVLINK_PORT_FLAVOUR_CPU ? "CPU" : + port_entry->flavour == DEVLINK_PORT_FLAVOUR_DSA ? "DSA" : + port_entry->flavour == DEVLINK_PORT_FLAVOUR_PCI_PF ? "PCI_PF": + port_entry->flavour == DEVLINK_PORT_FLAVOUR_PCI_VF ? "PCI_VF": + port_entry->flavour == DEVLINK_PORT_FLAVOUR_VIRTUAL ? "VIRTUAL": + port_entry->flavour == DEVLINK_PORT_FLAVOUR_UNUSED ? "UNUSED": + port_entry->flavour == DEVLINK_PORT_FLAVOUR_PCI_SF ? "PCI_SF": + "UNKNOWN"); + return; + }; + /* Failure to fill key from typed values means calculation of the max key + * length is wrong, i.e. a bug. */ + ovs_assert(fill_devlink_ports_key_from_typed( + keybuf, sizeof(keybuf), + *host_pf_mac, function_number)); + shash_add(&devlink_ports, keybuf, xstrdup(port_entry->netdev_name)); +} + + +static int +plug_representor_init(void) +{ + struct nl_dl_dump_state *port_dump; + struct dl_port port_entry; + int error; + struct eth_addr host_pf_macs[MAX_NIC_PHY_PORTS]; + + shash_init(&devlink_ports); + + port_dump = nl_dl_dump_init(); + if ((error = nl_dl_dump_init_error(port_dump))) { + VLOG_WARN( + "unable to start dump of ports from devlink-port interface"); + return error; + } + /* The core devlink infrastructure in the kernel keeps a linked list of + * the devices and each of those has a linked list of ports. These are + * populated by each device driver as devices are enumerated, and as such + * we can rely on ports being dumped in a consistent order on a device + * by device basis with logical numbering for each port flavour starting + * on 0 for each new device. + */ + nl_dl_dump_start(DEVLINK_CMD_PORT_GET, port_dump); + while (nl_dl_port_dump_next(port_dump, &port_entry)) { + switch (port_entry.flavour) { + case DEVLINK_PORT_FLAVOUR_PHYSICAL: + /* The PHYSICAL flavoured port represent a network facing port on + * the NIC. + * + * For kernel versions where the devlink-port infrastructure does + * not provide MAC address for PCI_PF flavoured ports, there exist + * a interface in sysfs which is relative to the name of the + * PHYSICAL port netdev name. + * + * Since we at this point in the dump do not know if the MAC will + * be provided for the PCI_PF or not, proactively store the MAC + * address by looking up through the sysfs interface. + * + * If MAC address is available once we get to the PCI_PF we will + * overwrite the stored value. + */ + if (port_entry.number > MAX_NIC_PHY_PORTS) { + VLOG_WARN("physical port number out of range for port '%s': " + "%"PRIu32, + port_entry.netdev_name, port_entry.number); + continue; + } + compat_get_host_pf_mac(port_entry.netdev_name, + &host_pf_macs[port_entry.number]); + break; + case DEVLINK_PORT_FLAVOUR_PCI_PF: /* FALL THROUGH */ + /* The PCI_PF flavoured port represent a host facing port. + * + * For function flavours other than PHYSICAL pci_pf_number will be + * set to the logical number of which physical port the function + * belongs. + */ + if (!eth_addr_is_zero(port_entry.function.eth_addr)) { + host_pf_macs[port_entry.pci_pf_number] = + port_entry.function.eth_addr; + } + /* FALL THROUGH */ + case DEVLINK_PORT_FLAVOUR_PCI_VF: + /* The PCI_VF flavoured port represent a host facing + * PCI Virtual Function. + * + * For function flavours other than PHYSICAL pci_pf_number will be + * set to the logical number of which physical port the function + * belongs. + */ + if (port_entry.pci_pf_number > MAX_NIC_PHY_PORTS) { + VLOG_WARN("physical port number out of range for port '%s': " + "%"PRIu32, + port_entry.netdev_name, port_entry.pci_pf_number); + continue; + } + devlink_port_add_function(&port_entry, + &host_pf_macs[port_entry.pci_pf_number]); + break; + }; + } + nl_dl_dump_finish(port_dump); + nl_dl_dump_destroy(port_dump); + + return 0; +} + +static int +plug_representor_destroy(void) +{ + shash_destroy_free_data(&devlink_ports); + + return 0; +} + +static bool +plug_representor_port_prepare(const struct plug_port_ctx_in *ctx_in, + struct plug_port_ctx_out *ctx_out) +{ + char keybuf[MAX_KEY_LEN]; + const char *pf_mac = smap_get(ctx_in->lport_options, + "plug:representor:pf-mac"); + const char *vf_num = smap_get(ctx_in->lport_options, + "plug:representor:vf-num"); + if (!fill_devlink_ports_key_from_strs(keybuf, sizeof(keybuf), + pf_mac, vf_num)) + { + /* Overflow, most likely incorrect input data from database */ + VLOG_WARN("Southbound DB port plugging options out of range for " + "lport: %s pf-mac: '%s' vf-num: '%s'", + ctx_in->lport_name, pf_mac, vf_num); + return false; + } + + char *rep_port; + rep_port = shash_find_data(&devlink_ports, keybuf); + VLOG_INFO("plug_representor %s (%s) -> %s", + ctx_in->lport_name, rep_port, ctx_in->br_int->name); + ctx_out->name = rep_port; + ctx_out->type = NULL; + ctx_out->iface_options = NULL; + return true; +} + +static void +plug_representor_port_finish(const struct plug_port_ctx_in *ctx_in OVS_UNUSED, + struct plug_port_ctx_out *ctx_out OVS_UNUSED) +{ + /* Nothing to be done here for now */ +} + +static void +plug_representor_port_ctx_destroy( + const struct plug_port_ctx_in *ctx_in OVS_UNUSED, + struct plug_port_ctx_out *ctx_out OVS_UNUSED) +{ + /* Noting to be done here for now */ +} + +const struct plug_class plug_representor = { + .type = "representor", + .init = plug_representor_init, + .destroy = plug_representor_destroy, + .plug_get_maintained_iface_options = NULL, /* TODO */ + .run = NULL, /* TODO */ + .plug_port_prepare = plug_representor_port_prepare, + .plug_port_finish = plug_representor_port_finish, + .plug_port_ctx_destroy = plug_representor_port_ctx_destroy, +}; + +/* The kernel devlink-port interface provides a vendor neutral and standard way + * of discovering host visible resources such as MAC address of interfaces from + * a program running on the NIC SoC side. + * + * However a fairly recent kernel version is required for it to work, so until + * this is widely available we provide this helper to retrieve the same + * information from the interim sysfs solution. */ +static bool +compat_get_host_pf_mac(const char *netdev_name, struct eth_addr *ea) +{ + char file_name[IFNAMSIZ + 35 + 1]; + FILE *stream; + char line[128]; + bool retval = false; + + snprintf(file_name, sizeof(file_name), + "/sys/class/net/%s/smart_nic/pf/config", netdev_name); + stream = fopen(file_name, "r"); + if (!stream) { + VLOG_WARN("%s: open failed (%s)", + file_name, ovs_strerror(errno)); + *ea = eth_addr_zero; + return false; + } + while (fgets(line, sizeof(line), stream)) { + char key[16]; + char *cp; + if (ovs_scan(line, "%15[^:]: ", key) + && key[0] == 'M' && key[1] == 'A' && key[2] == 'C') + { + /* strip any newline character */ + if ((cp = strchr(line, '\n')) != NULL) { + *cp = '\0'; + } + /* point cp at end of key + ': ', i.e. start of MAC address */ + cp = line + strnlen(key, sizeof(key)) + 2; + retval = eth_addr_from_string(cp, ea); + break; + } + } + fclose(stream); + return retval; +} diff --git a/m4/ovn.m4 b/m4/ovn.m4 index 2909914fb..2f274fc65 100644 --- a/m4/ovn.m4 +++ b/m4/ovn.m4 @@ -592,3 +592,29 @@ AC_DEFUN([OVS_CHECK_DDLOG_FAST_BUILD], if $ddlog_fast_build; then DDLOG_EXTRA_RUSTFLAGS="-C opt-level=z" fi]) + +dnl Checks for Netlink support. +AC_DEFUN([OVS_CHECK_NETLINK], + [AC_CHECK_HEADER([linux/netlink.h], + [HAVE_NETLINK=yes], + [HAVE_NETLINK=no], + [#include + ]) + AM_CONDITIONAL([HAVE_NETLINK], [test "$HAVE_NETLINK" = yes]) + if test "$HAVE_NETLINK" = yes; then + AC_DEFINE([HAVE_NETLINK], [1], + [Define to 1 if Netlink protocol is available.]) + fi]) + +dnl OVS_CHECK_LINUX_NETLINK +dnl +dnl Configure Linux netlink compat. +AC_DEFUN([OVS_CHECK_LINUX_NETLINK], [ + AC_COMPILE_IFELSE([ + AC_LANG_PROGRAM([#include ], [ + struct nla_bitfield32 x = { 0 }; + ])], + [AC_DEFINE([HAVE_NLA_BITFIELD32], [1], + [Define to 1 if struct nla_bitfield32 is available.])]) +]) +