From patchwork Fri Sep 3 19:27:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frode Nordahl X-Patchwork-Id: 1524525 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=Hl0RMO3U; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H1SWY1CRqz9sCD for ; Sat, 4 Sep 2021 05:28:24 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 9C0526166B; Fri, 3 Sep 2021 19:28:22 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id px0anL4y--f0; Fri, 3 Sep 2021 19:28:15 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id ACC6E61667; Fri, 3 Sep 2021 19:28:10 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E47EBC002D; Fri, 3 Sep 2021 19:28:02 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 01EEEC002A for ; Fri, 3 Sep 2021 19:27:57 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id E402B6157B for ; Fri, 3 Sep 2021 19:27:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HHltLCZAgUs7 for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from smtp-relay-canonical-1.canonical.com (smtp-relay-canonical-1.canonical.com [185.125.188.121]) by smtp3.osuosl.org (Postfix) with ESMTPS id 0FB8361542 for ; Fri, 3 Sep 2021 19:27:54 +0000 (UTC) Received: from frode-threadripper.. (1.general.frode.uk.vpn [10.172.193.250]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id 0A6C940666; Fri, 3 Sep 2021 19:27:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1630697271; bh=hlQb/VRq+wyXqDTO6J1oh+L1ZewUBD+Bi+uRj2KnStU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Hl0RMO3UeDHdH0d5/CywWEsKuPF5Xsy1cbhx7nUqo55NE0YP+SenZnJgCCPywL7yU a979BDNfh8to7sFZqSMl9I0r0WJFxdQ8QJ2/yHDKIV9JE3hYdCQ3wgCWib8gs/UVBR O/S5piIu4U65B5QJzJBX8X7spcOswSVeAgmSvTYEdatOu8rAcQhBTxgDavC89quZXS SYqptQNwy1zF5rhArvYg/U+NQLsUagFLh5X3VkJ6muXwqJ9wnaSOvG6xhIEUZaCgr+ A8Zrb6SdRbTJeW0nOazvoCt3+XntEopVn0Yorip2v/gzx+jfjJ6arjlnToWYRzJeeF TMqPncWg6AO1Q== From: Frode Nordahl To: dev@openvswitch.org Date: Fri, 3 Sep 2021 21:27:45 +0200 Message-Id: <20210903192748.1408062-7-frode.nordahl@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210903192748.1408062-1-frode.nordahl@canonical.com> References: <20210903192748.1408062-1-frode.nordahl@canonical.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v4 6/9] lib: Add infrastructure for plugging providers. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" New module contains the infrastructure for registering plugging classes which may be hosted inside or outside the core OVN repository. The data structures and functions for interacting with these plugging classes also live there. Extend build system to allow enabling building of built-in plugging providers and linking an externally built plugging provider. Signed-off-by: Frode Nordahl --- Documentation/automake.mk | 1 + Documentation/topics/index.rst | 1 + Documentation/topics/plug_providers/index.rst | 32 +++ .../topics/plug_providers/plug-providers.rst | 199 ++++++++++++++ acinclude.m4 | 49 ++++ configure.ac | 2 + lib/automake.mk | 12 +- lib/plug-dummy.c | 123 +++++++++ lib/plug-dummy.h | 33 +++ lib/plug-provider.h | 98 +++++++ lib/plug.c | 255 ++++++++++++++++++ lib/plug.h | 107 ++++++++ lib/test-plug.c | 72 +++++ ovn-architecture.7.xml | 35 ++- tests/automake.mk | 4 +- tests/ovn-plug.at | 8 + 16 files changed, 1016 insertions(+), 15 deletions(-) create mode 100644 Documentation/topics/plug_providers/index.rst create mode 100644 Documentation/topics/plug_providers/plug-providers.rst create mode 100644 lib/plug-dummy.c create mode 100644 lib/plug-dummy.h create mode 100644 lib/plug-provider.h create mode 100644 lib/plug.c create mode 100644 lib/plug.h create mode 100644 lib/test-plug.c create mode 100644 tests/ovn-plug.at diff --git a/Documentation/automake.mk b/Documentation/automake.mk index b3fd3d62b..92a843d76 100644 --- a/Documentation/automake.mk +++ b/Documentation/automake.mk @@ -28,6 +28,7 @@ DOC_SOURCE = \ Documentation/topics/ovn-news-2.8.rst \ Documentation/topics/role-based-access-control.rst \ Documentation/topics/debugging-ddlog.rst \ + Documentation/topics/plug_providers/plug-providers.rst \ Documentation/howto/index.rst \ Documentation/howto/docker.rst \ Documentation/howto/firewalld.rst \ diff --git a/Documentation/topics/index.rst b/Documentation/topics/index.rst index d58d5618b..12bd113b7 100644 --- a/Documentation/topics/index.rst +++ b/Documentation/topics/index.rst @@ -41,6 +41,7 @@ OVN high-availability role-based-access-control ovn-news-2.8 + plug_providers/index testing .. list-table:: diff --git a/Documentation/topics/plug_providers/index.rst b/Documentation/topics/plug_providers/index.rst new file mode 100644 index 000000000..837eeae15 --- /dev/null +++ b/Documentation/topics/plug_providers/index.rst @@ -0,0 +1,32 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in OVN documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +============== +Plug Providers +============== + + +.. toctree:: + :maxdepth: 2 + + plug-providers diff --git a/Documentation/topics/plug_providers/plug-providers.rst b/Documentation/topics/plug_providers/plug-providers.rst new file mode 100644 index 000000000..7b891156c --- /dev/null +++ b/Documentation/topics/plug_providers/plug-providers.rst @@ -0,0 +1,199 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in OVN documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +================== +Plugging Providers +================== + +Traditionally it has been the CMSes responsibility to create VIFs as part of +instance life cycle, and subsequently manage plug/unplug operations on the +integration bridge following the conventions described in the +`Open vSwitch Integration Guide`_ for mapping of VIFs to OVN logical port. + +With the advent of NICs connected to multiple distinct CPUs we can have a +topology where the instance runs on one host and Open vSwitch and OVN runs on +a different host, the smartnic control plane CPU. The host facing interfaces +will be visible to Open vSwitch and OVN as representor ports. + +The actions necessary for plugging and unplugging the representor port in +Open vSwitch running on the smartnic control plane CPU would be the same for +every CMS. + +Instead of every CMS having to develop their own version of an agent to do +the plugging, we provide a pluggable infrastructure in OVN that allows the +`ovn-controller` to perform the plugging on CMS direction. + +Hardware or platform specific details for initialization and lookup of +representor ports is provided by an plugging provider library hosted inside or +outside the core OVN repository, and linked at OVN build time. + +Life Cycle of an OVN plugged VIF +-------------------------------- + +1. CMS creates a record in the OVN Northbound Logical_Switch_Port table with + the options column containing the `plug-type` key with a value corresponding + to the `const char *type` provided by the plugging provider implementation + as well as a `requested-chassis` key with a value pointing at the name or + hostname of the chassis it wants the VIF plugged on. Additional plugging + provider specific key/value pairs must be provided for successful lookup. + +2. `ovn-northd` looks up the name or hostname provided in the + `requested-chassis` option and fills the OVN Southbound Port_Binding + requested_chassis column, it also copies relevant options over to the + Port_Binding record. + +3. `ovn-controller` monitors Southbound Port_Binding entries wth a + requested_chassis column pointing at its chassis UUID and when it encounters + a entry with option `plug-type` and it has registered a plug provider + matching that type it will act on it even if no local binding exists yet. + +4. It will fill the `struct plug_port_ctx_in` as defined in `lib/plug.h` with + `op_type` set to 'PLUG_OP_CREATE' and make a call to the plug providers + `plug_port_prepare` callback function. Plug provider performs lookup and + fills the `struct plug_port_ctx_out` as defined in `lib/plug.h`. + +5. `ovn-controller` creates a port and interface record in the local OVSDB + using the details provided by the plug provider and also adds + `external-ids:iface-id` with value matching the logical port name and + `external-ids:ovn-plugged` with value matching the logical port `plug-type`. + When the port creation is done a call will first be made to the plug + providers `plug_port_finish` function and then to the + `plug_port_ctx_destroy` function to free any memory allocated by the plug + implementation. + +6. The Open vSwitch vswitchd will assign a ofport to the newly created + interface and on the next `ovn-controller` main loop iteration flows will be + installed. + +7. On any change to the Southbound Port_Binding record or full recomputation + the `ovn-controller` will in addition to normal flow processing make a call + to the plug provider again similar to the first creation in case anything + needs updating for the interface record. + +8. The port will be unplugged when an event occurs which would make the + `ovn-controller` release a logical port, for example the Logical_Switch_Port + and Port_Binding entry disappearing from the database or its + `requested_chassis` column being pointed to a different chassis. + + +The plug provider interface +--------------------------- + +The interface between internals of OVN and a plugging provider is a set of +callbacks as defined by the `struct plug_class` in `lib/plug-provider.h`. + +It is important to note that these callbacks will be called in the critical +path of the `ovn-controller` processing loop, so care must be taken to make the +implementation as efficient as possible, and under no circumstance can any of +the callback functions make calls that block. + +On `ovn-controller` startup, plug providers made available at build time will +be registered by the identifier provided in the `const char *type` pointer, at +this time the `init` function pointer will be called if it is non-NULL. + +> **Note**: apart from the `const char *type` pointer, no attempt will be made +to access plug provider data or functions before the call to the `init` has +been made. + +On `ovn-controller` exit, the plug providers registered in the above mentioned +procedure will have their `destroy` function pointer called if it is non-NULL. + +If the plug provider has internal lookup tables that need to be maintained they +can define a `run` function which will be called as part of the +`ovn-controller` incremental processing engine loop. If the changes +encountered can be safely processed incrementally the function should return +'true', if the changes cannot be processed incrementally, for example because +the change would impact already handled interfaces that we might not process in +a while, the function should return 'false' to force the `ovn-controller` to +perform a full recomputation. + +On update of Interface records the `ovn-controller` will pass on a `sset` +to the `ovsport_update_iface` function containing options the plug +implementation finds pertinent to maintain for successful operation. This +`sset` is retrieved by making a call to the plug implementation +`plug_get_maintained_iface_options` function pointer if it is non-NULL. This +allows presence of other users of the OVSDB maintaining a different set of +options on the same set of Interface records without wiping out their changes. + +Before creating or updating an existing interface record the plug provider +`plug_port_prepare` function pointer will be called with valid pointers to +`struct plug_port_ctx_in` and `struct plug_port_ctx_out` data structures. If +the plug provider implementation is able to perform lookup it should fill the +`struct plug_port_ctx_out` data structure and return 'true'. The +`ovn-controller` will then create or update the port/interface records and +then call `plug_port_finish` and `plug_port_ctx_destroy`. If the plug provider +implementation is unable to perform lookup or prepare the desired resource it +should return 'false' which will tell the `ovn-controller` to not create or +update the port, in this case it will also not call `plug_port_finish`, it will +however make a call to `plug_port_ctx_destroy`. + +Before removing port and interface records previously plugged by the +`ovn-controller` as identified by presence of the Interface +`external-ids:ovn-plugged` key, the `ovn-controller` will look up the +`plug-type` from `external-ids:ovn-plugged`, fill `struct plug_port_ctx_in` +with `op_type` set to 'PLUG_OP_REMOVE' and make a call to `plug_port_prepare`. +After the port and interface has been removed a call will be made to +`plug_port_finish`. Both calls will be made with the pointer to +`plug_port_ctx_out` set to 'NULL', and no call will be made to +`plug_port_ctx_destroy`. + +Building with in-tree plugging providers +---------------------------------------- + +Plugging providers hosted in the OVN repository living under +`lib/plug_providers`: + +To enable them, provide the `--enable-plug-providers` command line option to +the configure script when building OVN. + +Building with an externally provided plugging provider +------------------------------------------------------ + +There is also infrastructure in place to support linking OVN with an externally +built plugging provider. + +This external plugging provider must define a NULL-terminated array of pointers +to `struct plug_class` data structures named `plug_provider_classes`. Example: + +.. code-block:: C + + const struct plug_class *plug_provider_classes[] = { + &plug_foo, + NULL, + }; + +The name of the repostiroy for the external plugging provider should be the +same as the name of the library it produces, and the built library artifact +should be placed in lib/.libs. Example: + +.. code-block:: none + + ovn-vif-foo/ + ovn-vif-foo/lib/.libs/libovn-vif-foo.la + +To enable such a plugging provider provide the +`--with-plug-provider=/path/to/ovn-vif-foo` command line option to the +configure script when building OVN. + +.. LINKS +.. _Open vSwitch Integration Guide: https://docs.openvswitch.org/en/latest/topics/integration/ diff --git a/acinclude.m4 b/acinclude.m4 index e7f829520..793a073d1 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -441,3 +441,52 @@ AC_DEFUN([OVN_CHECK_OVS], [ AC_MSG_CHECKING([OVS version]) AC_MSG_RESULT([$OVSVERSION]) ]) + +dnl OVN_CHECK_PLUG_PROVIDER +dnl +dnl Check for external plug provider +AC_DEFUN([OVN_CHECK_PLUG_PROVIDER], [ + AC_ARG_VAR([PLUG_PROVIDER]) + AC_ARG_WITH( + [plug-provider], + [AC_HELP_STRING([--with-plug-provider=/path/to/provider/repository], + [Specify path to a configured and built plug provider repository])], + [if test "$withval" = yes; then + if test -z "$PLUG_PROVIDER"; then + AC_MSG_ERROR([To build with external plug provider, specify the path to a configured and built plug provider repository --with-plug-provider or in \$PLUG_PROVIDER]), + fi + PLUG_PROVIDER="$(realpath $PLUG_PROVIDER)" + else + PLUG_PROVIDER="$(realpath $withval)" + fi + _plug_provider_name="$(basename $PLUG_PROVIDER)" + if test ! -f "$PLUG_PROVIDER/lib/.libs/lib${_plug_provider_name}.la"; then + AC_MSG_ERROR([$withval is not a configured and built plug provider library repository]) + fi + PLUG_PROVIDER_LDFLAGS="-L$PLUG_PROVIDER/lib/.libs -l$_plug_provider_name" + ], + [PLUG_PROVIDER=no]) + AC_MSG_CHECKING([for plug provider]) + AC_MSG_RESULT([$PLUG_PROVIDER]) + AC_SUBST([PLUG_PROVIDER_LDFLAGS]) + AM_CONDITIONAL([HAVE_PLUG_PROVIDER], [test "$PLUG_PROVIDER" != no]) + if test "$PLUG_PROVIDER" != no; then + AC_DEFINE([HAVE_PLUG_PROVIDER], [1], + [Build and link with external plug provider]) + fi +]) + +dnl OVN_ENABLE_PLUG +dnl +dnl Enable built-in plug providers +AC_DEFUN([OVN_ENABLE_PLUG], [ + AC_ARG_ENABLE( + [plug-providers], + [AC_HELP_STRING([--enable-plug-providers], [Enable building of built-in plug providers])], + [], [enable_plug=no]) + AM_CONDITIONAL([ENABLE_PLUG], [test "$enable_plug" != no]) + if test "$enable_plug" != no; then + AC_DEFINE([ENABLE_PLUG], [1], + [Build built-in plug providers]) + fi +]) diff --git a/configure.ac b/configure.ac index df0b98295..7f3274e59 100644 --- a/configure.ac +++ b/configure.ac @@ -172,6 +172,8 @@ OVS_ENABLE_SPARSE OVS_CHECK_DDLOG([0.38]) OVS_CHECK_PRAGMA_MESSAGE OVN_CHECK_OVS +OVN_CHECK_PLUG_PROVIDER +OVN_ENABLE_PLUG OVS_CTAGS_IDENTIFIERS AC_SUBST([OVS_CFLAGS]) AC_SUBST([OVS_LDFLAGS]) diff --git a/lib/automake.mk b/lib/automake.mk index ddfe33948..086fbd62d 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -3,6 +3,11 @@ lib_libovn_la_LDFLAGS = \ $(OVS_LTINFO) \ -Wl,--version-script=$(top_builddir)/lib/libovn.sym \ $(AM_LDFLAGS) + +if HAVE_PLUG_PROVIDER +lib_libovn_la_LDFLAGS += $(PLUG_PROVIDER_LDFLAGS) +endif + lib_libovn_la_SOURCES = \ lib/acl-log.c \ lib/acl-log.h \ @@ -32,7 +37,12 @@ lib_libovn_la_SOURCES = \ lib/inc-proc-eng.h \ lib/lb.c \ lib/lb.h \ - lib/stopwatch-names.h + lib/stopwatch-names.h \ + lib/plug-provider.h \ + lib/plug.h \ + lib/plug.c \ + lib/plug-dummy.h \ + lib/plug-dummy.c nodist_lib_libovn_la_SOURCES = \ lib/ovn-dirs.c \ lib/ovn-nb-idl.c \ diff --git a/lib/plug-dummy.c b/lib/plug-dummy.c new file mode 100644 index 000000000..505c37c9c --- /dev/null +++ b/lib/plug-dummy.c @@ -0,0 +1,123 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include "plug-dummy.h" +#include "plug-provider.h" +#include "plug.h" + +#include + +#include "openvswitch/vlog.h" +#include "smap.h" +#include "sset.h" + +#ifndef IFNAMSIZ +#define IFNAMSIZ 16 +#endif + +VLOG_DEFINE_THIS_MODULE(plug_dummy); + +static struct sset plug_dummy_maintained_iface_options; + +static int +plug_dummy_init(void) +{ + sset_init(&plug_dummy_maintained_iface_options); + sset_add(&plug_dummy_maintained_iface_options, "plug-dummy-option"); + + return 0; +} + +static int +plug_dummy_destroy(void) +{ + sset_destroy(&plug_dummy_maintained_iface_options); + + return 0; +} + +static const struct sset* +plug_dummy_get_maintained_iface_options(void) +{ + return &plug_dummy_maintained_iface_options; +} + +static bool +plug_dummy_run(struct plug_class *plug) +{ + VLOG_DBG("plug_dummy_run(%p)", plug); + + return true; +} + +static bool +plug_dummy_port_prepare(const struct plug_port_ctx_in *ctx_in, + struct plug_port_ctx_out *ctx_out) +{ + VLOG_DBG("plug_dummy_port_prepare: %s", ctx_in->lport_name); + + if (ctx_in->op_type == PLUG_OP_CREATE) { + size_t lport_name_len = strlen(ctx_in->lport_name); + ctx_out->name = xzalloc(IFNAMSIZ); + memcpy(ctx_out->name, ctx_in->lport_name, + (lport_name_len < IFNAMSIZ) ? lport_name_len : IFNAMSIZ - 1); + ctx_out->type = xstrdup("internal"); + ctx_out->iface_options = xmalloc(sizeof *ctx_out->iface_options); + smap_init(ctx_out->iface_options); + smap_add(ctx_out->iface_options, "plug-dummy-option", "value"); + } + + return true; +} + +static void +plug_dummy_port_finish(const struct plug_port_ctx_in *ctx_in, + struct plug_port_ctx_out *ctx_out OVS_UNUSED) +{ + VLOG_DBG("plug_dummy_port_finish: %s", ctx_in->lport_name); +} + +static void +plug_dummy_port_ctx_destroy(const struct plug_port_ctx_in *ctx_in, + struct plug_port_ctx_out *ctx_out) +{ + VLOG_DBG("plug_dummy_port_ctx_destroy: %s", ctx_in->lport_name); + ovs_assert(ctx_in->op_type == PLUG_OP_CREATE); + free(ctx_out->name); + free(ctx_out->type); + smap_destroy(ctx_out->iface_options); + free(ctx_out->iface_options); +} + +const struct plug_class plug_dummy_class = { + .type = "dummy", + .init = plug_dummy_init, + .destroy = plug_dummy_destroy, + .plug_get_maintained_iface_options = + plug_dummy_get_maintained_iface_options, + .run = plug_dummy_run, + .plug_port_prepare = plug_dummy_port_prepare, + .plug_port_finish = plug_dummy_port_finish, + .plug_port_ctx_destroy = plug_dummy_port_ctx_destroy, +}; + +void +plug_dummy_enable(void) +{ + plug_register_provider(&plug_dummy_class); +} + diff --git a/lib/plug-dummy.h b/lib/plug-dummy.h new file mode 100644 index 000000000..6ea33671e --- /dev/null +++ b/lib/plug-dummy.h @@ -0,0 +1,33 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef PLUG_DUMMY_H +#define PLUG_DUMMY_H 1 + +/* + * The dummy plugger, allows for experimenting with plugging in a sandbox */ + +#ifdef __cplusplus +extern "C" { +#endif + +void plug_dummy_enable(void); + +#ifdef __cplusplus +} +#endif + +#endif /* plug-dummy.h */ diff --git a/lib/plug-provider.h b/lib/plug-provider.h new file mode 100644 index 000000000..487534ee5 --- /dev/null +++ b/lib/plug-provider.h @@ -0,0 +1,98 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef PLUG_PROVIDER_H +#define PLUG_PROVIDER_H 1 + +/* Provider interface to pluggers. A plugger implementation performs lookup + * and/or initialization of ports, typically representor ports, using generic + * non-blocking hardware interfaces. This allows the ovn-controller to, upon + * the CMS's request, create ports and interfaces in the chassis's Open vSwitch + * instances (also known as vif plugging). + */ + +#include + +#include "plug.h" + +#ifdef __cplusplus +extern "C" { +#endif + +struct plug_class { + /* Type of plugger in this class. */ + const char *type; + + /* Called when the plug provider is registered, typically at program + * startup. + * + * This function may be set to null if a plug class needs no + * initialization at registration time. */ + int (*init)(void); + + /* Called when the plug provider is unregistered, typically at program + * exit. + * + * This function may be set to null if a plug class needs no + * de-initialization at unregister time.*/ + int (*destroy)(void); + + /* Performs periodic work needed by plugger, if any is necessary. Returns + * 'true; if the changes encountered could be handled incrementally, + * 'false' otherwise. + * + * Note that returning 'false' will instruct the incremental processing + * engine to perform a full recomputation. */ + bool (*run)(struct plug_class *); + + /* Retrieve Interface options this plugger will maintain. This set is used + * to know which items to remove when maintaining the database record. */ + const struct sset * (*plug_get_maintained_iface_options)(void); + + /* Pass plug_port_ctx_in to plug implementation to prepare for port + * creation/update. + * + * The plug implemantation can perform lookup or any per port + * initialization and should fill plug_port_ctx_out with data required for + * port/interface creation. The plug implementation should return true if + * it wants the caller to create/update a port/interface, false otherwise. + * + * Data in the plug_port_ctx_out struct is owned by the plugging library, + * and a call must be made to the plug_port_ctx_destroy callback to free + * up any allocations when done with port creation/update. + */ + bool (*plug_port_prepare)(const struct plug_port_ctx_in *, + struct plug_port_ctx_out *); + + /* Notify plugging library that port update is done. */ + void (*plug_port_finish)(const struct plug_port_ctx_in *, + struct plug_port_ctx_out *); + + /* Free any allocations made by the plug_port_prepare callback. */ + void (*plug_port_ctx_destroy)(const struct plug_port_ctx_in *, + struct plug_port_ctx_out *); +}; + +extern const struct plug_class plug_dummy_class; +#ifdef HAVE_PLUG_PROVIDER +extern const struct plug_class *plug_provider_classes[]; +#endif + +#ifdef __cplusplus +} +#endif + +#endif /* plug-provider.h */ diff --git a/lib/plug.c b/lib/plug.c new file mode 100644 index 000000000..c0c34214e --- /dev/null +++ b/lib/plug.c @@ -0,0 +1,255 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include "plug-provider.h" +#include "plug.h" + +#include +#include +#include + +#include "openvswitch/vlog.h" +#include "openvswitch/shash.h" +#include "smap.h" +#include "sset.h" +#include "lib/inc-proc-eng.h" + +VLOG_DEFINE_THIS_MODULE(plug); + +#ifdef ENABLE_PLUG +static const struct plug_class *base_plug_classes[] = { +}; +#endif + +static struct shash plug_classes = SHASH_INITIALIZER(&plug_classes); + +/* Protects the 'plug_classes' shash. */ +static struct ovs_mutex plug_classes_mutex = OVS_MUTEX_INITIALIZER; + +/* Initialize the the plug infrastructure by registering known plug classes */ +void +plug_initialize(void) +{ + static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; + + if (ovsthread_once_start(&once)) { +#ifdef ENABLE_PLUG + /* Register built-in plug provider classes */ + for (int i = 0; i < ARRAY_SIZE(base_plug_classes); i++) { + plug_register_provider(base_plug_classes[i]); + } +#endif +#ifdef HAVE_PLUG_PROVIDER + /* Register external plug provider classes. + * + * Note that we cannot use the ARRAY_SIZE macro here as + * plug_provider_classes is defined in external code which is not + * available at compile time. The convention is to use a + * NULL-terminated array instead. */ + for (const struct plug_class **pp = plug_provider_classes; + pp && *pp; + pp++) + { + plug_register_provider(*pp); + } +#endif + ovsthread_once_done(&once); + } +} + +static int +plug_register_provider__(const struct plug_class *new_class) +{ + struct plug_class *plug_class; + int error; + + if (shash_find(&plug_classes, new_class->type)) { + VLOG_WARN("attempted to register duplicate plug provider: %s", + new_class->type); + return EEXIST; + } + + error = new_class->init ? new_class->init() : 0; + if (error) { + VLOG_WARN("failed to initialize %s plug class: %s", + new_class->type, ovs_strerror(error)); + return error; + } + + plug_class = xmalloc(sizeof *plug_class); + memcpy(plug_class, new_class, sizeof *plug_class); + + shash_add(&plug_classes, new_class->type, plug_class); + + return 0; +} + +/* Register the new plug provider referred to in 'new_class' and perform any + * class level initialization as specified in its plug_class. */ +int +plug_register_provider(const struct plug_class *new_class) +{ + int error; + + ovs_mutex_lock(&plug_classes_mutex); + error = plug_register_provider__(new_class); + ovs_mutex_unlock(&plug_classes_mutex); + + return error; +} + +static int +plug_unregister_provider__(const char *type) +{ + int error; + struct shash_node *node; + struct plug_class *plug_class; + + node = shash_find(&plug_classes, type); + if (!node) { + return EINVAL; + } + + plug_class = node->data; + error = plug_class->destroy ? plug_class->destroy() : 0; + if (error) { + VLOG_WARN("failed to destroy %s plug class: %s", + plug_class->type, ovs_strerror(error)); + return error; + } + + shash_delete(&plug_classes, node); + free(plug_class); + + return 0; +} + +/* Unregister the plug provider identified by 'type' and perform any class + * level de-initialization as specified in its plug_class. */ +int +plug_unregister_provider(const char *type) +{ + int error; + + ovs_mutex_lock(&plug_classes_mutex); + error = plug_unregister_provider__(type); + ovs_mutex_unlock(&plug_classes_mutex); + + return error; +} + +const struct plug_class * +plug_get_provider(const char *type) +{ + struct plug_class *plug_class; + + ovs_mutex_lock(&plug_classes_mutex); + plug_class = shash_find_data(&plug_classes, type); + ovs_mutex_unlock(&plug_classes_mutex); + + return plug_class; +} + +/* De-initialize and unregister the plug provider classes. */ +void +plug_destroy_all(void) +{ + struct shash_node *node, *next; + + SHASH_FOR_EACH_SAFE (node, next, &plug_classes) { + struct plug_class *plug_class = node->data; + plug_unregister_provider(plug_class->type); + } +} + +/* Get the class level 'maintained_iface_options' set. */ +const struct sset * +plug_get_maintained_iface_options(const struct plug_class *plug_class) +{ + return plug_class->plug_get_maintained_iface_options(); +} + +/* Prepare the logical port as identified by 'ctx_in' for port creation, update + * or removal as specified by 'ctx_in->op_type'. + * + * When 'ctx_in->op_type' is PLUG_OP_CREATE the plug implementation must fill + * 'ctx_out' with data to apply to the interface record maintained by OVN on + * its behalf. + * + * When 'ctx_in_op_type' is PLUG_OP_REMOVE 'ctx_out' should be set to NULL and + * the plug implementation must not attempt to use 'ctx_out'. + * + * The data in 'ctx_out' is owned by the plug implementation, and a call must + * be made to plug_port_ctx_destroy when done with it. */ +bool +plug_port_prepare(const struct plug_class *plug_class, + const struct plug_port_ctx_in *ctx_in, + struct plug_port_ctx_out *ctx_out) +{ + if (ctx_out) { + memset(ctx_out, 0, sizeof(*ctx_out)); + } + return plug_class->plug_port_prepare(ctx_in, ctx_out); +} + +/* Notify the plug implementation that a port creation, update or removal has + * been completed */ +void +plug_port_finish(const struct plug_class *plug_class, + const struct plug_port_ctx_in *ctx_in, + struct plug_port_ctx_out *ctx_out) +{ + plug_class->plug_port_finish(ctx_in, ctx_out); +} + +/* Free any data allocated to 'ctx_out' in a prevous call to + * plug_port_prepare. */ +void +plug_port_ctx_destroy(const struct plug_class *plug_class, + const struct plug_port_ctx_in *ctx_in, + struct plug_port_ctx_out *ctx_out) +{ + plug_class->plug_port_ctx_destroy(ctx_in, ctx_out); +} + +/* Iterate over registered plug provider classes and call their 'run' + * function if defined. + * + * If any of the classes report that something has changed we will trigger a + * recompute. */ +void +en_plug_provider_run(struct engine_node *inc_eng_node, + void *inc_eng_data OVS_UNUSED) +{ + struct shash_node *node, *next; + bool handled = true; + + SHASH_FOR_EACH_SAFE (node, next, &plug_classes) { + struct plug_class *plug_class = node->data; + if (plug_class->run && !plug_class->run(plug_class)) { + handled = false; + } + } + + if (!handled) { + /* as we do not have a change handler registered in the incremental + * processing engine this will trigger a recompute. */ + engine_set_node_state(inc_eng_node, EN_UPDATED); + } else { + engine_set_node_state(inc_eng_node, EN_UNCHANGED); + } +} diff --git a/lib/plug.h b/lib/plug.h new file mode 100644 index 000000000..fef4c7f64 --- /dev/null +++ b/lib/plug.h @@ -0,0 +1,107 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef PLUG_H +#define PLUG_H 1 + +/* + * Plug, the plugging interface. This module contains the infrastructure for + * registering and instantiating plugging classes which may be hosted inside + * or outside the core OVN repository. The data structures and functions for + * interacting with these plugging classes also live here. + */ + +#include "smap.h" + +#ifdef __cplusplus +extern "C" { +#endif + +struct plug_class; +struct ovsdb_idl_txn; +struct ovsrec_bridge; + +enum plug_op_type { + PLUG_OP_CREATE = 1, /* Port is created or updated */ + PLUG_OP_REMOVE, /* Port is removed from this chassis */ +}; + +struct plug_port_ctx_in { + /* Operation being performed */ + enum plug_op_type op_type; + + /* These are provided so that the plug implementation may make decisions + * based on environmental factors such as settings in the open-vswitch + * table and datapath type settings on the integration bridge. */ + const struct ovsrec_open_vswitch_table *ovs_table; + const struct ovsrec_bridge *br_int; + + /* Name of logical port, can be useful for plugging library to track any + * per port resource initialization. */ + const char *lport_name; + + /* Logical port options, while OVN will forward the contents verbatim from + * the Southbound database, the convention is for the plugging library to + * only make decisions based on the plug-* options. */ + const struct smap *lport_options; + + /* When OVN knows about an existing interface record associated with this + * lport, these will be filled in with information about it. */ + const char *iface_name; + const char *iface_type; + const struct smap *iface_options; +}; + +struct plug_port_ctx_out { + /* The name to use for port and interface record. */ + char *name; + + /* Type of interface to create. */ + char *type; + + /* Options to set on the interface record. */ + struct smap *iface_options; +}; + + +void plug_initialize(void); +int plug_register_provider(const struct plug_class *); +int plug_unregister_provider(const char *type); +void plug_destroy_all(void); +const struct plug_class * plug_get_provider(const char *); + +const struct sset * plug_get_maintained_iface_options( + const struct plug_class *plug_class); + +bool plug_port_prepare(const struct plug_class *, + const struct plug_port_ctx_in *, + struct plug_port_ctx_out *); +void plug_port_finish(const struct plug_class *, + const struct plug_port_ctx_in *, + struct plug_port_ctx_out *); +void plug_port_ctx_destroy(const struct plug_class *, + const struct plug_port_ctx_in *, + struct plug_port_ctx_out *); + +struct engine_node; + +void en_plug_provider_run(struct engine_node *, void *); + +#ifdef __cplusplus +} +#endif + +#endif /* plug.h */ diff --git a/lib/test-plug.c b/lib/test-plug.c new file mode 100644 index 000000000..bddce23c4 --- /dev/null +++ b/lib/test-plug.c @@ -0,0 +1,72 @@ +/* Copyright (c) 2021, Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include + +#include "plug.h" +#include "plug-dummy.h" +#include "plug-provider.h" +#include "smap.h" +#include "sset.h" +#include "tests/ovstest.h" + +static void +test_plug(struct ovs_cmdl_context *ctx OVS_UNUSED) +{ + const struct plug_class *plug_class; + + ovs_assert(plug_unregister_provider("dummy") == EINVAL); + + ovs_assert(!plug_register_provider(&plug_dummy_class)); + plug_class = plug_get_provider("dummy"); + ovs_assert(plug_register_provider(&plug_dummy_class) == EEXIST); + + ovs_assert(sset_contains(plug_get_maintained_iface_options(plug_class), + "plug-dummy-option")); + + struct smap fake_lport_options = SMAP_INITIALIZER(&fake_lport_options); + struct plug_port_ctx_in ctx_in = { + .op_type = PLUG_OP_CREATE, + .lport_name = "lsp1", + .lport_options = &fake_lport_options, + }; + struct plug_port_ctx_out ctx_out; + plug_port_prepare(plug_class, &ctx_in, &ctx_out); + ovs_assert(!strcmp(ctx_out.name, "lsp1")); + ovs_assert(!strcmp(ctx_out.type, "internal")); + ovs_assert(!strcmp(smap_get( + ctx_out.iface_options, "plug-dummy-option"), "value")); + + plug_port_finish(plug_class, &ctx_in, &ctx_out); + plug_port_ctx_destroy(plug_class, &ctx_in, &ctx_out); + plug_destroy_all(); +} + +static void +test_plug_main(int argc, char *argv[]) +{ + set_program_name(argv[0]); + static const struct ovs_cmdl_command commands[] = { + {"run", NULL, 0, 0, test_plug, OVS_RO}, + {NULL, NULL, 0, 0, NULL, OVS_RO}, + }; + struct ovs_cmdl_context ctx; + ctx.argc = argc - 1; + ctx.argv = argv + 1; + ovs_cmdl_run_command(&ctx, commands); +} + +OVSTEST_REGISTER("test-plug", test_plug_main); diff --git a/ovn-architecture.7.xml b/ovn-architecture.7.xml index 3d2910358..152367a89 100644 --- a/ovn-architecture.7.xml +++ b/ovn-architecture.7.xml @@ -67,8 +67,9 @@
  • One or more (usually many) hypervisors. Hypervisors must run Open vSwitch and implement the interface described in - Documentation/topics/integration.rst in the OVN source tree. - Any hypervisor platform supported by Open vSwitch is acceptable. + Documentation/topics/integration.rst in the Open vSwitch + source tree. Any hypervisor platform supported by Open vSwitch is + acceptable.
  • @@ -318,11 +319,19 @@
  • On a hypervisor, any VIFs that are to be attached to logical networks. - The hypervisor itself, or the integration between Open vSwitch and the - hypervisor (described in - Documentation/topics/integration.rst) takes care of this. - (This is not part of OVN or new to OVN; this is pre-existing integration - work that has already been done on hypervisors that support OVS.) + For instances connected through software emulated ports such as TUN/TAP + or VETH pairs, the hypervisor itself will normally create ports and plug + them into the integration bridge. For instances connected through + representor ports, typically used with hardware offload, the + ovn-controller may on CMS direction consult a plugging + provider library for representor port lookup and plug them into the + integration bridge (please refer to + Documentation/topics/plugging-providers.rst for more + information). In both cases the conventions described in + Documentation/topics/integration.rst in the Open vSwitch + source tree is followed to ensure mapping between OVN logical port and + VIF. (This is pre-existing integration work that has already been done + on hypervisors that support OVS.)
  • @@ -921,12 +930,12 @@ Eventually, a user powers on the VM that owns the VIF. On the hypervisor where the VM is powered on, the integration between the hypervisor and Open vSwitch (described in - Documentation/topics/integration.rst) adds the VIF to the OVN - integration bridge and stores vif-id in - external_ids:iface-id to indicate that the - interface is an instantiation of the new VIF. (None of this code is new - in OVN; this is pre-existing integration work that has already been done - on hypervisors that support OVS.) + Documentation/topics/integration.rst in the Open vSwitch + source tree) adds the VIF to the OVN integration bridge and stores + vif-id in external_ids:iface-id to + indicate that the interface is an instantiation of the new VIF. (None of + this code is new in OVN; this is pre-existing integration work that has + already been done on hypervisors that support OVS.)
  • diff --git a/tests/automake.mk b/tests/automake.mk index 5b890d644..ad8978541 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -38,7 +38,8 @@ TESTSUITE_AT = \ tests/ovn-ipam.at \ tests/ovn-features.at \ tests/ovn-lflow-cache.at \ - tests/ovn-ipsec.at + tests/ovn-ipsec.at \ + tests/ovn-plug.at SYSTEM_KMOD_TESTSUITE_AT = \ tests/system-common-macros.at \ @@ -248,6 +249,7 @@ tests_ovstest_SOURCES = \ controller/ofctrl-seqno.c \ controller/ofctrl-seqno.h \ lib/test-ovn-features.c \ + lib/test-plug.c \ northd/test-ipam.c \ northd/ipam.c \ northd/ipam.h diff --git a/tests/ovn-plug.at b/tests/ovn-plug.at new file mode 100644 index 000000000..d5c6a1b6d --- /dev/null +++ b/tests/ovn-plug.at @@ -0,0 +1,8 @@ +# +# Unit tests for the lib/plug.c module. +# +AT_BANNER([OVN unit tests - plug]) + +AT_SETUP([unit test -- plugging infrastructure tests]) +AT_CHECK([ovstest test-plug run], [0], []) +AT_CLEANUP