From patchwork Sat Nov 13 09:43:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladislav Odintsov X-Patchwork-Id: 1554664 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=mA+i+Q0l; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HrrBk1rSrz9sPf for ; Sat, 13 Nov 2021 20:44:14 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 1A59060879; Sat, 13 Nov 2021 09:44:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gYHMOy8uw4d5; Sat, 13 Nov 2021 09:44:08 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 227B460794; Sat, 13 Nov 2021 09:44:06 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7C9AEC0037; Sat, 13 Nov 2021 09:44:05 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 35151C0012 for ; Sat, 13 Nov 2021 09:44:03 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 16CFE60685 for ; Sat, 13 Nov 2021 09:44:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6xamvyHzOdLi for ; Sat, 13 Nov 2021 09:44:02 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com [IPv6:2a00:1450:4864:20::12d]) by smtp3.osuosl.org (Postfix) with ESMTPS id EC5CB600C9 for ; Sat, 13 Nov 2021 09:44:01 +0000 (UTC) Received: by mail-lf1-x12d.google.com with SMTP id br15so24875898lfb.9 for ; Sat, 13 Nov 2021 01:44:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1yhHDi5JJflsKFUH3oPjU6IB3+F1/k9lGA6e02Dntu4=; b=mA+i+Q0lcRmd5t+WYaXXtnU4igFIHWw1oMr0ZJiy05n9UZxNvVB7WTClkVsFWMYzzD lTI6cPGrEgbLs+O/0PCdntQA/7HCtuP3pnPKLNgKup/vLSaSllZFu/0fgQJfS6cMoyzk DAkYAdpweipdgEf0jHczMNSUGvFuUhK1f4y8Vr2MW1yukkGYt2lKkSFwYD5wvpIkpOYw mxYQ84KKmEfEPdNWj5LGF0UuU1rcNt3iMkotc1HdnpkrdTN4cVNsX8woQov5PWCA8CS9 /JMQx4BBYrs2VeJjR4z5zAc4X4kYsfr+T8MsmlwpI/xCwWOGav2GfS89w5d4E0Zeek7j Yqqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1yhHDi5JJflsKFUH3oPjU6IB3+F1/k9lGA6e02Dntu4=; b=pHtIbIO4xgK8GcBgBeSPfAzToyBOl6GxjuoS40GaGCrwV7EGRlMg5sprTY0vPazMAB UNWNfSrswcfcvbOWp7Cb2fMmQZRsHdpouHU/G2dvocCYhj38Si48OF1sZn/IBes0JGjl r7vWAyBWQ1NLRM4e+GqqUxDab0KMG23PkBBgybiJlIRSQd1JzAJsKztWnaKv9IOB2DJg zXuOKI86jEnQHQ5INwUzDGaGtsQ6az2O1su2utO3gbasXXeg9KNSq7+fSqQPUuPL+hIS D/bSbrzGfYGMpA6gMWyrMGERh+gxok5zzfTmfDgQznfIWVduhS27e7ibnAWRKIVLq3mk chKA== X-Gm-Message-State: AOAM533j784KUK9mcOj+nJ+bqMoNrUhtEni0REE3qVyIKeOD5CGR/Cs6 ZCduDMxfUwDXmmiBr9cRXCyv57BIC0lONQ== X-Google-Smtp-Source: ABdhPJwMTMLd+MKJ7wZJALPOaufs3P8yDz9X4fDejsF9952BR0uX1HkgxRP/2CZKRRfTZPeia1qdFg== X-Received: by 2002:ac2:529c:: with SMTP id q28mr19960846lfm.683.1636796639664; Sat, 13 Nov 2021 01:43:59 -0800 (PST) Received: from localhost.localdomain (109-252-131-59.dynamic.spd-mgts.ru. [109.252.131.59]) by smtp.gmail.com with ESMTPSA id h1sm818725lfu.277.2021.11.13.01.43.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 13 Nov 2021 01:43:59 -0800 (PST) From: Vladislav Odintsov To: dev@openvswitch.org Date: Sat, 13 Nov 2021 12:43:49 +0300 Message-Id: <20211113094353.17690-2-odivlad@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20211113094353.17690-1-odivlad@gmail.com> References: <20211113094353.17690-1-odivlad@gmail.com> MIME-Version: 1.0 Cc: Vladislav Odintsov Subject: [ovs-dev] [PATCH ovn v8 1/5] ic: maintain route origin - connected/static X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commits adds ability to save route's origin while IC learning. Directly connected routes are saved in IC SB DB with "connected" origin column value. Static routes have "static" value in origin column. This logic would be used in next patch to compute priority for lr_in_ip_routing stage lflows. Signed-off-by: Vladislav Odintsov Acked-by: Numan Siddique --- ic/ovn-ic.c | 34 +++++++++++++++++++-------- lib/ovn-util.h | 3 +++ ovn-ic-sb.ovsschema | 7 ++++-- ovn-ic-sb.xml | 10 ++++++++ tests/ovn-ic.at | 57 +++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 99 insertions(+), 12 deletions(-) diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c index 303e93a4f..70abae108 100644 --- a/ic/ovn-ic.c +++ b/ic/ovn-ic.c @@ -854,6 +854,7 @@ struct ic_route_info { struct in6_addr prefix; unsigned int plen; struct in6_addr nexthop; + const char *origin; /* Either nb_route or nb_lrp is set and the other one must be NULL. * - For a route that is learned from IC-SB, or a static route that is @@ -867,22 +868,25 @@ struct ic_route_info { static uint32_t ic_route_hash(const struct in6_addr *prefix, unsigned int plen, - const struct in6_addr *nexthop) + const struct in6_addr *nexthop, const char *origin) { uint32_t basis = hash_bytes(prefix, sizeof *prefix, (uint32_t)plen); + basis = hash_string(origin, basis); return hash_bytes(nexthop, sizeof *nexthop, basis); } static struct ic_route_info * ic_route_find(struct hmap *routes, const struct in6_addr *prefix, - unsigned int plen, const struct in6_addr *nexthop) + unsigned int plen, const struct in6_addr *nexthop, + const char *origin) { struct ic_route_info *r; - uint32_t hash = ic_route_hash(prefix, plen, nexthop); + uint32_t hash = ic_route_hash(prefix, plen, nexthop, origin); HMAP_FOR_EACH_WITH_HASH (r, node, hash, routes) { if (ipv6_addr_equals(&r->prefix, prefix) && r->plen == plen && - ipv6_addr_equals(&r->nexthop, nexthop)) { + ipv6_addr_equals(&r->nexthop, nexthop) && + !strcmp(r->origin, origin)) { return r; } } @@ -926,13 +930,15 @@ add_to_routes_learned(struct hmap *routes_learned, &prefix, &plen, &nexthop)) { return false; } + const char *origin = smap_get_def(&nb_route->options, "origin", ""); struct ic_route_info *ic_route = xzalloc(sizeof *ic_route); ic_route->prefix = prefix; ic_route->plen = plen; ic_route->nexthop = nexthop; ic_route->nb_route = nb_route; + ic_route->origin = origin; hmap_insert(routes_learned, &ic_route->node, - ic_route_hash(&prefix, plen, &nexthop)); + ic_route_hash(&prefix, plen, &nexthop, origin)); return true; } @@ -1093,8 +1099,9 @@ add_to_routes_ad(struct hmap *routes_ad, ic_route->plen = plen; ic_route->nexthop = nexthop; ic_route->nb_route = nb_route; + ic_route->origin = ROUTE_ORIGIN_STATIC; hmap_insert(routes_ad, &ic_route->node, - ic_route_hash(&prefix, plen, &nexthop)); + ic_route_hash(&prefix, plen, &nexthop, ROUTE_ORIGIN_STATIC)); } static void @@ -1143,8 +1150,10 @@ add_network_to_routes_ad(struct hmap *routes_ad, const char *network, ic_route->plen = plen; ic_route->nexthop = nexthop; ic_route->nb_lrp = nb_lrp; + ic_route->origin = ROUTE_ORIGIN_CONNECTED; hmap_insert(routes_ad, &ic_route->node, - ic_route_hash(&prefix, plen, &nexthop)); + ic_route_hash(&prefix, plen, &nexthop, + ROUTE_ORIGIN_CONNECTED)); } static bool @@ -1206,7 +1215,8 @@ sync_learned_route(struct ic_context *ctx, continue; } struct ic_route_info *route_learned - = ic_route_find(&ic_lr->routes_learned, &prefix, plen, &nexthop); + = ic_route_find(&ic_lr->routes_learned, &prefix, plen, &nexthop, + isb_route->origin); if (route_learned) { /* Sync external-ids */ struct uuid ext_id; @@ -1233,6 +1243,8 @@ sync_learned_route(struct ic_context *ctx, UUID_ARGS(&isb_route->header_.uuid)); nbrec_logical_router_static_route_update_external_ids_setkey( nb_route, "ic-learned-route", uuid_s); + nbrec_logical_router_static_route_update_options_setkey( + nb_route, "origin", isb_route->origin); free(uuid_s); nbrec_logical_router_update_static_routes_addvalue( ic_lr->lr, nb_route); @@ -1297,8 +1309,9 @@ advertise_route(struct ic_context *ctx, icsbrec_route_delete(isb_route); continue; } - struct ic_route_info *route_adv = - ic_route_find(routes_ad, &prefix, plen, &nexthop); + struct ic_route_info *route_adv = ic_route_find(routes_ad, &prefix, + plen, &nexthop, + isb_route->origin); if (!route_adv) { /* Delete the extra route from IC-SB. */ VLOG_DBG("Delete route %s -> %s from IC-SB, which is not found" @@ -1338,6 +1351,7 @@ advertise_route(struct ic_context *ctx, } icsbrec_route_set_ip_prefix(isb_route, prefix_s); icsbrec_route_set_nexthop(isb_route, nexthop_s); + icsbrec_route_set_origin(isb_route, route_adv->origin); free(prefix_s); free(nexthop_s); diff --git a/lib/ovn-util.h b/lib/ovn-util.h index 2fa92e069..a923c3b65 100644 --- a/lib/ovn-util.h +++ b/lib/ovn-util.h @@ -25,6 +25,9 @@ #define ovn_print_version(MIN_OFP, MAX_OFP) \ ovs_print_version(MIN_OFP, MAX_OFP) +#define ROUTE_ORIGIN_CONNECTED "connected" +#define ROUTE_ORIGIN_STATIC "static" + struct nbrec_logical_router_port; struct sbrec_logical_flow; struct svec; diff --git a/ovn-ic-sb.ovsschema b/ovn-ic-sb.ovsschema index 5364b21b4..42ce85d7d 100644 --- a/ovn-ic-sb.ovsschema +++ b/ovn-ic-sb.ovsschema @@ -1,7 +1,7 @@ { "name": "OVN_IC_Southbound", - "version": "1.0.0", - "cksum": "108951192 6585", + "version": "1.1.0", + "cksum": "423535838 6733", "tables": { "IC_SB_Global": { "columns": { @@ -94,6 +94,9 @@ "refTable": "Availability_Zone"}}}, "ip_prefix": {"type": "string"}, "nexthop": {"type": "string"}, + "origin": {"type": {"key": { + "type": "string", + "enum": ["set", ["connected", "static"]]}}}, "external_ids": { "type": {"key": "string", "value": "string", "min": 0, "max": "unlimited"}}}, diff --git a/ovn-ic-sb.xml b/ovn-ic-sb.xml index 3582cff47..d8338e4d3 100644 --- a/ovn-ic-sb.xml +++ b/ovn-ic-sb.xml @@ -313,6 +313,16 @@ Nexthop IP address for this route. + + + Can be one of connected or static. Routes to + directly-connected subnets - LRP's CIDRs are inserted to OVN IC SB DB + with connected value in . Static + routes are inserted to OVN IC SB DB with static value. + Next when route is learned to another AZ NB DB by ovn-ic, route origin + is synced to . + diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at index 9086974a3..7e8498b2f 100644 --- a/tests/ovn-ic.at +++ b/tests/ovn-ic.at @@ -423,6 +423,63 @@ AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr11 | # Test routes from lr12 didn't leak as learned to lr21 AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr21], [0], []) +# cleanup +ovn-ic-nbctl --if-exists ts-del ts1 +ovn_as az1 ovn-nbctl lr-del lr11 +ovn_as az1 ovn-nbctl lr-del lr21 +ovn_as az2 ovn-nbctl lr-del lr12 +ovn_as az2 ovn-nbctl lr-del lr22 + +# check routes origin advertisement and learning + +# setup topology with connected, static and source routes +ovn-ic-nbctl ts-add ts1 +for i in 1 2; do + ovn_as az$i + + # Enable route learning at AZ level + ovn-nbctl set nb_global . options:ic-route-learn=true + # Enable route advertising at AZ level + ovn-nbctl set nb_global . options:ic-route-adv=true + + # Create LRP and connect to TS + ovn-nbctl lr-add lr$i + ovn-nbctl lrp-add lr$i lrp-lr$i-ts1 aa:aa:aa:aa:aa:0$i 169.254.100.$i/24 + ovn-nbctl lsp-add ts1 lsp-ts1-lr$i \ + -- lsp-set-addresses lsp-ts1-lr$i router \ + -- lsp-set-type lsp-ts1-lr$i router \ + -- lsp-set-options lsp-ts1-lr$i router-port=lrp-lr$i-ts1 + + ovn-nbctl lrp-add lr$i lrp-lr$i-p$i 00:00:00:00:00:0$i 192.168.$i.1/24 + + # Create static routes + ovn-nbctl lr-route-add lr$i 10.11.$i.0/24 169.254.0.1 + + # Create a src-ip route, which shouldn't be synced + ovn-nbctl --policy=src-ip lr-route-add lr$i 10.22.$i.0/24 169.254.0.2 +done + +for i in 1 2; do + OVS_WAIT_UNTIL([ovn_as az$i ovn-nbctl lr-route-list lr$i | grep learned]) +done + +# check that advertised routes in ic-sb have correct origin +ovn-ic-sbctl list route +wait_row_count ic-sb:Route 1 ip_prefix=10.11.1.0/24 origin=static +wait_row_count ic-sb:Route 1 ip_prefix=192.168.1.1/24 origin=connected +wait_row_count ic-sb:Route 1 ip_prefix=10.11.2.0/24 origin=static +wait_row_count ic-sb:Route 1 ip_prefix=192.168.2.1/24 origin=connected + +# check that learned routes in ic-sb have correct origin + +ovn_as az1 +wait_row_count nb:Logical_Router_Static_Route 1 ip_prefix=10.11.2.0/24 options:origin=static +wait_row_count nb:Logical_Router_Static_Route 1 ip_prefix=192.168.2.1/24 options:origin=connected + +ovn_as az2 +wait_row_count nb:Logical_Router_Static_Route 1 ip_prefix=10.11.1.0/24 options:origin=static +wait_row_count nb:Logical_Router_Static_Route 1 ip_prefix=192.168.1.1/24 options:origin=connected + OVN_CLEANUP_IC([az1], [az2]) AT_CLEANUP From patchwork Sat Nov 13 09:43:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladislav Odintsov X-Patchwork-Id: 1554666 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=HBrbFEQa; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HrrBn3llrz9sR4 for ; Sat, 13 Nov 2021 20:44:17 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 825EA60888; Sat, 13 Nov 2021 09:44:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VM4R-wJnRGli; Sat, 13 Nov 2021 09:44:08 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 6029760769; Sat, 13 Nov 2021 09:44:07 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 45090C002E; Sat, 13 Nov 2021 09:44:07 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id EBB3BC002E for ; Sat, 13 Nov 2021 09:44:04 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id C628340212 for ; Sat, 13 Nov 2021 09:44:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xX-dOLMlysFg for ; Sat, 13 Nov 2021 09:44:03 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com [IPv6:2a00:1450:4864:20::235]) by smtp2.osuosl.org (Postfix) with ESMTPS id 1A9164013B for ; Sat, 13 Nov 2021 09:44:03 +0000 (UTC) Received: by mail-lj1-x235.google.com with SMTP id e11so23380878ljo.13 for ; Sat, 13 Nov 2021 01:44:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JZmGTb9DB9rEl62bS2E1nGXi1VkXNzOAdDCm69FbILs=; b=HBrbFEQaYKTGSrRiljodpySfD2913jvFliX3XtZTW6HhrWm8YbWZe1jVR75FMR87fF JHhQyqQu/SbwEN1Z3WkcfJYCUZALx5QyINPtqMm4lvUu0dCPvV7z9vPTQDbTDqhzsGMU wMLWf2+DxJmgVTUVkwgQ2YZZuSYgrluQ8dIDP26dqsSQWbmkyOciIryqo8xJhvYUJ1mw nhUmMXqfigUD0NcVvv0lJsVx37dzJ/IuhCfrpZESOOVaWv1hEQaxd/VSl/ysnt7P94gA arTkpqL7V+jq700nOFCuE/FZDW/qXHNNiZVXC7Bbey1K1P0SfdUVioRN//S7/3TYnvtR x21g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JZmGTb9DB9rEl62bS2E1nGXi1VkXNzOAdDCm69FbILs=; b=UyFrqwJycrYWqF9DTINNHVti/GLwZcuzmqYeQlgf8Q/98vNYZQwPUayYYvPnAD1lH5 nF0PU4R06gi7LvQORfcM+zK4eI7DmbRDRnkgiu6WTo/H5qVsbzRPNTjozRKQTFP/dUwk gjxY3KIXyBnWH4AlE0T4s6/EGk7zrVWvchC4ysQhkyk85pb1b5gJeOORAl0ypZG3xzKD ZIUlT3e1o3oFRzRCQrDLzGxXZjRJoYyJisWuRIYgQ+FqUR2XwpKIpk3zXI+kr7LmSLot kkvpsHPStXApUf3AKhXUmzuz4f2511ljSZ5Yy00zscTrxL88Cqfd80nEonW86XHWZAm9 7Wvw== X-Gm-Message-State: AOAM530JSFX20kcG3CpVA5XKjWPwuGOgpAtQd2+J9ZoxJZMZmBuULrhX Lm7VlkWqZ9vOAMIXqlvSPbEd3/U5ZkjXRQ== X-Google-Smtp-Source: ABdhPJwPnIm4bSLnjalcdRSuu4VAOZz+6BEZYoTNXnFvMio/v/Z+KHqexblXf0DXwCgsjeyBgGz3GQ== X-Received: by 2002:a05:651c:106a:: with SMTP id y10mr21954951ljm.455.1636796640590; Sat, 13 Nov 2021 01:44:00 -0800 (PST) Received: from localhost.localdomain (109-252-131-59.dynamic.spd-mgts.ru. [109.252.131.59]) by smtp.gmail.com with ESMTPSA id h1sm818725lfu.277.2021.11.13.01.44.00 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 13 Nov 2021 01:44:00 -0800 (PST) From: Vladislav Odintsov To: dev@openvswitch.org Date: Sat, 13 Nov 2021 12:43:50 +0300 Message-Id: <20211113094353.17690-3-odivlad@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20211113094353.17690-1-odivlad@gmail.com> References: <20211113094353.17690-1-odivlad@gmail.com> MIME-Version: 1.0 Cc: Vladislav Odintsov Subject: [ovs-dev] [PATCH ovn v8 2/5] northd: make connected routes have higher priority than static X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" With this patch routes to connected networks have higher priority than static routes with same ip_prefix. This brings commonly-used behaviour for routes lookup order: 1: longest prefix match 2: metric The metric has next lookup order: 1: connected routes 2: static routes Earlier static and connected routes with same ip_prefix had the same priority, so it was impossible to predict which one is used for routing decision. Each route's prefix length has its own 'slot' in lflow prios. Now prefix length space is calculated using next information: to calculate route's priority prefixlen multiplied by 3 + route origin offset (0 - source-based route; 1 - directly- connected route; 2 - static route). Also, enlarge prio for generic records in lr_in_ip_routing stage by 10000. Signed-off-by: Vladislav Odintsov Signed-off-by: Vladislav Odintsov > --- northd/northd.c | 50 ++++++++++++++++++++++++++++------------- northd/ovn-northd.8.xml | 12 +++++----- tests/ovn-northd.at | 8 +++---- 3 files changed, 45 insertions(+), 25 deletions(-) diff --git a/northd/northd.c b/northd/northd.c index 1e8a3457c..0d513f039 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -305,6 +305,15 @@ enum ovn_stage { * */ +/* + * Route offsets implement logic to prioritize traffic for routes with + * same ip_prefix values: + * - connected route overrides static one; + * - static route overrides connected route. */ +#define ROUTE_PRIO_OFFSET_MULTIPLIER 3 +#define ROUTE_PRIO_OFFSET_STATIC 1 +#define ROUTE_PRIO_OFFSET_CONNECTED 2 + /* Returns an "enum ovn_stage" built from the arguments. */ static enum ovn_stage ovn_stage_build(enum ovn_datapath_type dp_type, enum ovn_pipeline pipeline, @@ -8782,6 +8791,7 @@ struct ecmp_groups_node { struct in6_addr prefix; unsigned int plen; bool is_src_route; + const char *origin; uint16_t route_count; struct ovs_list route_list; /* Contains ecmp_route_list_node */ }; @@ -8819,6 +8829,7 @@ ecmp_groups_add(struct hmap *ecmp_groups, eg->prefix = route->prefix; eg->plen = route->plen; eg->is_src_route = route->is_src_route; + eg->origin = smap_get_def(&route->route->options, "origin", ""); ovs_list_init(&eg->route_list); ecmp_groups_add_route(eg, route); @@ -8919,19 +8930,20 @@ build_route_prefix_s(const struct in6_addr *prefix, unsigned int plen) static void build_route_match(const struct ovn_port *op_inport, const char *network_s, int plen, bool is_src_route, bool is_ipv4, struct ds *match, - uint16_t *priority) + uint16_t *priority, int ofs) { const char *dir; /* The priority here is calculated to implement longest-prefix-match * routing. */ if (is_src_route) { dir = "src"; - *priority = plen * 2; + ofs = 0; } else { dir = "dst"; - *priority = (plen * 2) + 1; } + *priority = (plen * ROUTE_PRIO_OFFSET_MULTIPLIER) + ofs; + if (op_inport) { ds_put_format(match, "inport == %s && ", op_inport->json_key); } @@ -9073,7 +9085,7 @@ add_ecmp_symmetric_reply_flows(struct hmap *lflows, out_port->lrp_networks.ea_s, IN6_IS_ADDR_V4MAPPED(&route->prefix) ? "" : "xx", port_ip, out_port->json_key); - ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, 300, + ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, 10300, ds_cstr(&match), ds_cstr(&actions), &st_route->header_); @@ -9103,8 +9115,10 @@ build_ecmp_route_flow(struct hmap *lflows, struct ovn_datapath *od, struct ds route_match = DS_EMPTY_INITIALIZER; char *prefix_s = build_route_prefix_s(&eg->prefix, eg->plen); + int ofs = !strcmp(eg->origin, ROUTE_ORIGIN_CONNECTED) ? + ROUTE_PRIO_OFFSET_CONNECTED: ROUTE_PRIO_OFFSET_STATIC; build_route_match(NULL, prefix_s, eg->plen, eg->is_src_route, is_ipv4, - &route_match, &priority); + &route_match, &priority, ofs); free(prefix_s); struct ds actions = DS_EMPTY_INITIALIZER; @@ -9180,7 +9194,7 @@ add_route(struct hmap *lflows, struct ovn_datapath *od, const struct ovn_port *op, const char *lrp_addr_s, const char *network_s, int plen, const char *gateway, bool is_src_route, const struct ovsdb_idl_row *stage_hint, - bool is_discard_route) + bool is_discard_route, int ofs) { bool is_ipv4 = strchr(network_s, '.') ? true : false; struct ds match = DS_EMPTY_INITIALIZER; @@ -9196,7 +9210,7 @@ add_route(struct hmap *lflows, struct ovn_datapath *od, } } build_route_match(op_inport, network_s, plen, is_src_route, is_ipv4, - &match, &priority); + &match, &priority, ofs); struct ds common_actions = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -9256,10 +9270,15 @@ build_static_route_flow(struct hmap *lflows, struct ovn_datapath *od, } } + int ofs = !strcmp(smap_get_def(&route->options, "origin", ""), + ROUTE_ORIGIN_CONNECTED) ? ROUTE_PRIO_OFFSET_CONNECTED + : ROUTE_PRIO_OFFSET_STATIC; + char *prefix_s = build_route_prefix_s(&route_->prefix, route_->plen); add_route(lflows, route_->is_discard_route ? od : out_port->od, out_port, lrp_addr_s, prefix_s, route_->plen, route->nexthop, - route_->is_src_route, &route->header_, route_->is_discard_route); + route_->is_src_route, &route->header_, route_->is_discard_route, + ofs); free(prefix_s); } @@ -10672,14 +10691,14 @@ build_ip_routing_flows_for_lrouter_port( add_route(lflows, op->od, op, op->lrp_networks.ipv4_addrs[i].addr_s, op->lrp_networks.ipv4_addrs[i].network_s, op->lrp_networks.ipv4_addrs[i].plen, NULL, false, - &op->nbrp->header_, false); + &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED); } for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { add_route(lflows, op->od, op, op->lrp_networks.ipv6_addrs[i].addr_s, op->lrp_networks.ipv6_addrs[i].network_s, op->lrp_networks.ipv6_addrs[i].plen, NULL, false, - &op->nbrp->header_, false); + &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED); } } else if (lsp_is_router(op->nbsp)) { struct ovn_port *peer = ovn_port_get_peer(ports, op); @@ -10702,7 +10721,8 @@ build_ip_routing_flows_for_lrouter_port( peer->lrp_networks.ipv4_addrs[0].addr_s, laddrs->ipv4_addrs[k].network_s, laddrs->ipv4_addrs[k].plen, NULL, false, - &peer->nbrp->header_, false); + &peer->nbrp->header_, false, + ROUTE_PRIO_OFFSET_CONNECTED); } } } @@ -10773,7 +10793,7 @@ build_mcast_lookup_flows_for_lrouter( /* Drop IPv6 multicast traffic that shouldn't be forwarded, * i.e., router solicitation and router advertisement. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 550, + ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10550, "nd_rs || nd_ra", "drop;"); if (!od->mcast_info.rtr.relay) { return; @@ -10801,7 +10821,7 @@ build_mcast_lookup_flows_for_lrouter( } ds_put_format(actions, "outport = \"%s\"; ip.ttl--; next;", igmp_group->mcgroup.name); - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 500, + ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10500, ds_cstr(match), ds_cstr(actions)); } @@ -10809,7 +10829,7 @@ build_mcast_lookup_flows_for_lrouter( * ports. Otherwise drop any multicast traffic. */ if (od->mcast_info.rtr.flood_static) { - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 450, + ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10450, "ip4.mcast || ip6.mcast", "clone { " "outport = \""MC_STATIC"\"; " @@ -10817,7 +10837,7 @@ build_mcast_lookup_flows_for_lrouter( "next; " "};"); } else { - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 450, + ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10450, "ip4.mcast || ip6.mcast", "drop;"); } } diff --git a/northd/ovn-northd.8.xml b/northd/ovn-northd.8.xml index fb67395e3..4f3a9d5e3 100644 --- a/northd/ovn-northd.8.xml +++ b/northd/ovn-northd.8.xml @@ -2945,12 +2945,12 @@ icmp6 {

If ECMP routes with symmetric reply are configured in the - OVN_Northbound database for a gateway router, a priority-300 - flow is added for each router port on which symmetric replies are - configured. The matching logic for these ports essentially reverses the - configured logic of the ECMP route. So for instance, a route with a - destination routing policy will instead match if the source IP address - matches the static route's prefix. The flow uses the action + OVN_Northbound database for a gateway router, a + priority-10300 flow is added for each router port on which symmetric + replies are configured. The matching logic for these ports essentially + reverses the configured logic of the ECMP route. So for instance, a route + with a destination routing policy will instead match if the source IP + address matches the static route's prefix. The flow uses the action ct_next to send IP packets to the connection tracker for packet de-fragmentation and tracking before sending it to the next table.

diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 85b47a18f..3c1a97f73 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -5430,7 +5430,7 @@ check ovn-nbctl --wait=sb --ecmp-symmetric-reply lr-route-add lr0 1.0.0.1 192.16 ovn-sbctl dump-flows lr0 > lr0flows AT_CHECK([grep -e "lr_in_ip_routing.*select" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl - table=??(lr_in_ip_routing ), priority=65 , match=(ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);) + table=??(lr_in_ip_routing ), priority=97 , match=(ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);) ]) AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed 's/192\.168\.0\..0/192.168.0.??/' | sed 's/table=../table=??/' | sort], [0], [dnl table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == 1 && reg8[[16..31]] == 1), action=(reg0 = 192.168.0.??; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; next;) @@ -5443,7 +5443,7 @@ check ovn-nbctl --wait=sb --ecmp-symmetric-reply lr-route-add lr0 1.0.0.1 192.16 ovn-sbctl dump-flows lr0 > lr0flows AT_CHECK([grep -e "lr_in_ip_routing.*select" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl - table=??(lr_in_ip_routing ), priority=65 , match=(ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);) + table=??(lr_in_ip_routing ), priority=97 , match=(ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);) ]) AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed 's/192\.168\.0\..0/192.168.0.??/' | sed 's/table=../table=??/' | sort], [0], [dnl table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == 1 && reg8[[16..31]] == 1), action=(reg0 = 192.168.0.??; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; next;) @@ -5458,14 +5458,14 @@ check ovn-nbctl --wait=sb lr-route-add lr0 1.0.0.0/24 192.168.0.10 ovn-sbctl dump-flows lr0 > lr0flows AT_CHECK([grep -e "lr_in_ip_routing.*192.168.0.10" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl - table=??(lr_in_ip_routing ), priority=49 , match=(ip4.dst == 1.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;) + table=??(lr_in_ip_routing ), priority=73 , match=(ip4.dst == 1.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;) ]) check ovn-nbctl --wait=sb lr-route-add lr0 2.0.0.0/24 lr0-public ovn-sbctl dump-flows lr0 > lr0flows AT_CHECK([grep -e "lr_in_ip_routing.*2.0.0.0" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl - table=??(lr_in_ip_routing ), priority=49 , match=(ip4.dst == 2.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;) + table=??(lr_in_ip_routing ), priority=73 , match=(ip4.dst == 2.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;) ]) AT_CLEANUP From patchwork Sat Nov 13 09:43:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladislav Odintsov X-Patchwork-Id: 1554668 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=a/P0ITiR; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HrrC72zg3z9sPf for ; Sat, 13 Nov 2021 20:44:35 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 5F3D260777; Sat, 13 Nov 2021 09:44:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FcykcdAQ1o8E; Sat, 13 Nov 2021 09:44:26 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id B4E206137F; Sat, 13 Nov 2021 09:44:22 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 67D84C0038; Sat, 13 Nov 2021 09:44:22 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 322B1C0036 for ; Sat, 13 Nov 2021 09:44:21 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id A4D5260B65 for ; Sat, 13 Nov 2021 09:44:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kSOgiAGYCIIv for ; Sat, 13 Nov 2021 09:44:08 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com [IPv6:2a00:1450:4864:20::12f]) by smtp3.osuosl.org (Postfix) with ESMTPS id BDD4D60792 for ; Sat, 13 Nov 2021 09:44:05 +0000 (UTC) Received: by mail-lf1-x12f.google.com with SMTP id m27so4257926lfj.12 for ; Sat, 13 Nov 2021 01:44:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=p3tIHZ4LV5OstBF33oxFZTODGA/ptdhxo8CbAcetMMw=; b=a/P0ITiRLOpJTI3qw4vCmhTgzzPa1RxKEjfsRXh49kWYtrYFvtXpyGdrqeSb/zePzC 2bU2QoazbCSQBU397n/h3GQypvSjoqtgCXEAf/3ZUiejbs0f9A1bn0ROzfY94+QWxoIS 2o4iEQx5yV51+PmhzwqdJ15UBSXTi0T6yX8PgxRWYg1CzirAUJCskBXEOpWx2zjwBzhq d9d17dBcavjjQemaH6J0q9ijuJk4AIh9BaCuYCjff/DVqTdcTEX4fdFJ2la+oOjL9K5f NdsVwrvSjsLelaTAgXpnte9REfCDJoAArnQKdSiVuPN2JANZI3A8nxX99Y/EFRdO02/l 9Xeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=p3tIHZ4LV5OstBF33oxFZTODGA/ptdhxo8CbAcetMMw=; b=7wu4uzouf7UJCMFw2U+REZqFkE5o9THXuNWRzNB5x3qvZamO9DeF878drPEDkWqUvA Hejx8EzyhAx2lqQ2VaDCNZe3ve8xbP0e4sxqKX08t4dDpgBwzCRrzHiVqFKNyTsc3CnJ tE3lkqsvY1ukiu08AyY1Gx7UiIfwfkgL1P4xe5Hm5REthDQXUmt3PSwCV+BS5KZc71ty RQYDkYJ24who5Ktr0mSMlxv7H8tDd/a/vXgGBTcWjkCXAp22ZHgipLZP4PBxMZv6Uk2T NKmDSkASEBgdR+/ZKaMs0bwoKGEOyH3Qp5tnJuz8hPrcjPAZy/yTCVHit59asQEoRYQT FdNg== X-Gm-Message-State: AOAM5326e80dLfFLrui+hkT6uwh+6wgOfJN2Xt7JLAvUYZMHNJ00LM0U nrwQDQHIrhXv6pAcvOBt0mZLqWBGHzvAhw== X-Google-Smtp-Source: ABdhPJwPGMzH8ynrKzNRcA7zAUPvRkwuHm688/lZfSPSlwIZgz2pHaxj48T4TtTKgdzwpEcJVXJs3w== X-Received: by 2002:a05:6512:3f86:: with SMTP id x6mr20317692lfa.389.1636796641517; Sat, 13 Nov 2021 01:44:01 -0800 (PST) Received: from localhost.localdomain (109-252-131-59.dynamic.spd-mgts.ru. [109.252.131.59]) by smtp.gmail.com with ESMTPSA id h1sm818725lfu.277.2021.11.13.01.44.00 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 13 Nov 2021 01:44:01 -0800 (PST) From: Vladislav Odintsov To: dev@openvswitch.org Date: Sat, 13 Nov 2021 12:43:51 +0300 Message-Id: <20211113094353.17690-4-odivlad@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20211113094353.17690-1-odivlad@gmail.com> References: <20211113094353.17690-1-odivlad@gmail.com> MIME-Version: 1.0 Cc: Vladislav Odintsov Subject: [ovs-dev] [PATCH ovn v8 3/5] northd, utils: support for RouteTables in LRs X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This patch extends Logical Router's routing functionality. Now user may create multiple routing tables within a Logical Router and assign them to Logical Router Ports. Traffic coming from Logical Router Port with assigned route_table is checked against Logical_Router_Static_Routes with same route_table field value and routes to connected networks. If no route_table option is set to the LRP, routes' lookup is done agains routes with no route_table field value ("", empty string) and against routes to connected networks. A new Logical Router ingress table #10 is added - IN_IP_ROUTING_PRE. In this table packets which come from LRPs with configured options:route_table field are checked against inport and in OVS register 7 unique non-zero value identifying route table is written. If no route_table is configured to LRP, 0 is written to register 7. Then in 11th table IN_IP_ROUTING flows for routes which have non-empty `route_table` field are prefixed in match portion with "reg7 == " where id is an uniqly-generated route_table id (if any) or zero if route has empty route_table value. Signed-off-by: Vladislav Odintsov Acked-by: Numan Siddique --- northd/northd.c | 154 +++++++++--- northd/ovn-northd.8.xml | 82 ++++-- ovn-nb.ovsschema | 5 +- ovn-nb.xml | 30 +++ tests/ovn-ic.at | 4 + tests/ovn-nbctl.at | 196 ++++++++++++++- tests/ovn-northd.at | 74 +++++- tests/ovn.at | 546 +++++++++++++++++++++++++++++++++++++++- utilities/ovn-nbctl.c | 134 +++++++++- 9 files changed, 1154 insertions(+), 71 deletions(-) diff --git a/northd/northd.c b/northd/northd.c index 0d513f039..bf233766e 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -148,15 +148,16 @@ enum ovn_stage { PIPELINE_STAGE(ROUTER, IN, ECMP_STATEFUL, 7, "lr_in_ecmp_stateful") \ PIPELINE_STAGE(ROUTER, IN, ND_RA_OPTIONS, 8, "lr_in_nd_ra_options") \ PIPELINE_STAGE(ROUTER, IN, ND_RA_RESPONSE, 9, "lr_in_nd_ra_response") \ - PIPELINE_STAGE(ROUTER, IN, IP_ROUTING, 10, "lr_in_ip_routing") \ - PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_ECMP, 11, "lr_in_ip_routing_ecmp") \ - PIPELINE_STAGE(ROUTER, IN, POLICY, 12, "lr_in_policy") \ - PIPELINE_STAGE(ROUTER, IN, POLICY_ECMP, 13, "lr_in_policy_ecmp") \ - PIPELINE_STAGE(ROUTER, IN, ARP_RESOLVE, 14, "lr_in_arp_resolve") \ - PIPELINE_STAGE(ROUTER, IN, CHK_PKT_LEN , 15, "lr_in_chk_pkt_len") \ - PIPELINE_STAGE(ROUTER, IN, LARGER_PKTS, 16, "lr_in_larger_pkts") \ - PIPELINE_STAGE(ROUTER, IN, GW_REDIRECT, 17, "lr_in_gw_redirect") \ - PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 18, "lr_in_arp_request") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_PRE, 10, "lr_in_ip_routing_pre") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING, 11, "lr_in_ip_routing") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_ECMP, 12, "lr_in_ip_routing_ecmp") \ + PIPELINE_STAGE(ROUTER, IN, POLICY, 13, "lr_in_policy") \ + PIPELINE_STAGE(ROUTER, IN, POLICY_ECMP, 14, "lr_in_policy_ecmp") \ + PIPELINE_STAGE(ROUTER, IN, ARP_RESOLVE, 15, "lr_in_arp_resolve") \ + PIPELINE_STAGE(ROUTER, IN, CHK_PKT_LEN, 16, "lr_in_chk_pkt_len") \ + PIPELINE_STAGE(ROUTER, IN, LARGER_PKTS, 17, "lr_in_larger_pkts") \ + PIPELINE_STAGE(ROUTER, IN, GW_REDIRECT, 18, "lr_in_gw_redirect") \ + PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 19, "lr_in_arp_request") \ \ /* Logical router egress stages. */ \ PIPELINE_STAGE(ROUTER, OUT, UNDNAT, 0, "lr_out_undnat") \ @@ -225,6 +226,7 @@ enum ovn_stage { #define REG_NEXT_HOP_IPV6 "xxreg0" #define REG_SRC_IPV4 "reg1" #define REG_SRC_IPV6 "xxreg1" +#define REG_ROUTE_TABLE_ID "reg7" #define REG_ORIG_TP_DPORT_ROUTER "reg9[16..31]" @@ -287,8 +289,9 @@ enum ovn_stage { * | R6 | UNUSED | X | | G | IN_IP_ROUTING)| * | | | R | | 1 | | * +-----+--------------------------+ E | UNUSED | | | - * | R7 | UNUSED | G | | | | - * | | | 3 | | | | + * | R7 | ROUTE_TABLE_ID | G | | | | + * | | (>= IN_IP_ROUTING_PRE && | 3 | | | | + * | | <= IN_IP_ROUTING) | | | | | * +-----+--------------------------+---+-----------------+---+---------------+ * | R8 | ECMP_GROUP_ID | | | * | | ECMP_MEMBER_ID | X | | @@ -8647,11 +8650,72 @@ cleanup: ds_destroy(&actions); } +static uint32_t +route_table_add(struct simap *route_tables, const char *route_table_name) +{ + /* route table ids start from 1 */ + uint32_t rtb_id = simap_count(route_tables) + 1; + + if (rtb_id == UINT16_MAX) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "too many route tables for Logical Router."); + return 0; + } + + if (!simap_put(route_tables, route_table_name, rtb_id)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "Route table id unexpectedly appeared"); + } + + return rtb_id; +} + +static uint32_t +get_route_table_id(struct simap *route_tables, const char *route_table_name) +{ + if (!route_table_name || !strlen(route_table_name)) { + return 0; + } + + uint32_t rtb_id = simap_get(route_tables, route_table_name); + if (!rtb_id) { + rtb_id = route_table_add(route_tables, route_table_name); + } + + return rtb_id; +} + +static void +build_route_table_lflow(struct ovn_datapath *od, struct hmap *lflows, + struct nbrec_logical_router_port *lrp, + struct simap *route_tables) +{ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + + const char *route_table_name = smap_get(&lrp->options, "route_table"); + uint32_t rtb_id = get_route_table_id(route_tables, route_table_name); + if (!rtb_id) { + return; + } + + ds_put_format(&match, "inport == \"%s\"", lrp->name); + ds_put_format(&actions, "%s = %d; next;", + REG_ROUTE_TABLE_ID, rtb_id); + + ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 100, + ds_cstr(&match), ds_cstr(&actions)); + + ds_destroy(&match); + ds_destroy(&actions); +} + struct parsed_route { struct ovs_list list_node; struct in6_addr prefix; unsigned int plen; bool is_src_route; + uint32_t route_table_id; uint32_t hash; const struct nbrec_logical_router_static_route *route; bool ecmp_symmetric_reply; @@ -8676,7 +8740,7 @@ find_static_route_outport(struct ovn_datapath *od, struct hmap *ports, * Otherwise return NULL. */ static struct parsed_route * parsed_routes_add(struct ovn_datapath *od, struct hmap *ports, - struct ovs_list *routes, + struct ovs_list *routes, struct simap *route_tables, const struct nbrec_logical_router_static_route *route, struct hmap *bfd_connections) { @@ -8758,6 +8822,7 @@ parsed_routes_add(struct ovn_datapath *od, struct hmap *ports, struct parsed_route *pr = xzalloc(sizeof *pr); pr->prefix = prefix; pr->plen = plen; + pr->route_table_id = get_route_table_id(route_tables, route->route_table); pr->is_src_route = (route->policy && !strcmp(route->policy, "src-ip")); pr->hash = route_hash(pr); @@ -8792,6 +8857,7 @@ struct ecmp_groups_node { unsigned int plen; bool is_src_route; const char *origin; + uint32_t route_table_id; uint16_t route_count; struct ovs_list route_list; /* Contains ecmp_route_list_node */ }; @@ -8800,7 +8866,7 @@ static void ecmp_groups_add_route(struct ecmp_groups_node *group, const struct parsed_route *route) { - if (group->route_count == UINT16_MAX) { + if (group->route_count == UINT16_MAX) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); VLOG_WARN_RL(&rl, "too many routes in a single ecmp group."); return; @@ -8830,6 +8896,7 @@ ecmp_groups_add(struct hmap *ecmp_groups, eg->plen = route->plen; eg->is_src_route = route->is_src_route; eg->origin = smap_get_def(&route->route->options, "origin", ""); + eg->route_table_id = route->route_table_id; ovs_list_init(&eg->route_list); ecmp_groups_add_route(eg, route); @@ -8843,7 +8910,8 @@ ecmp_groups_find(struct hmap *ecmp_groups, struct parsed_route *route) HMAP_FOR_EACH_WITH_HASH (eg, hmap_node, route->hash, ecmp_groups) { if (ipv6_addr_equals(&eg->prefix, &route->prefix) && eg->plen == route->plen && - eg->is_src_route == route->is_src_route) { + eg->is_src_route == route->is_src_route && + eg->route_table_id == route->route_table_id) { return eg; } } @@ -8890,7 +8958,8 @@ unique_routes_remove(struct hmap *unique_routes, HMAP_FOR_EACH_WITH_HASH (ur, hmap_node, route->hash, unique_routes) { if (ipv6_addr_equals(&route->prefix, &ur->route->prefix) && route->plen == ur->route->plen && - route->is_src_route == ur->route->is_src_route) { + route->is_src_route == ur->route->is_src_route && + route->route_table_id == ur->route->route_table_id) { hmap_remove(unique_routes, &ur->hmap_node); const struct parsed_route *existed_route = ur->route; free(ur); @@ -8928,9 +8997,9 @@ build_route_prefix_s(const struct in6_addr *prefix, unsigned int plen) } static void -build_route_match(const struct ovn_port *op_inport, const char *network_s, - int plen, bool is_src_route, bool is_ipv4, struct ds *match, - uint16_t *priority, int ofs) +build_route_match(const struct ovn_port *op_inport, uint32_t rtb_id, + const char *network_s, int plen, bool is_src_route, + bool is_ipv4, struct ds *match, uint16_t *priority, int ofs) { const char *dir; /* The priority here is calculated to implement longest-prefix-match @@ -8947,6 +9016,9 @@ build_route_match(const struct ovn_port *op_inport, const char *network_s, if (op_inport) { ds_put_format(match, "inport == %s && ", op_inport->json_key); } + if (rtb_id || ofs == ROUTE_PRIO_OFFSET_STATIC) { + ds_put_format(match, "%s == %d && ", REG_ROUTE_TABLE_ID, rtb_id); + } ds_put_format(match, "ip%s.%s == %s/%d", is_ipv4 ? "4" : "6", dir, network_s, plen); } @@ -9117,8 +9189,8 @@ build_ecmp_route_flow(struct hmap *lflows, struct ovn_datapath *od, char *prefix_s = build_route_prefix_s(&eg->prefix, eg->plen); int ofs = !strcmp(eg->origin, ROUTE_ORIGIN_CONNECTED) ? ROUTE_PRIO_OFFSET_CONNECTED: ROUTE_PRIO_OFFSET_STATIC; - build_route_match(NULL, prefix_s, eg->plen, eg->is_src_route, is_ipv4, - &route_match, &priority, ofs); + build_route_match(NULL, eg->route_table_id, prefix_s, eg->plen, + eg->is_src_route, is_ipv4, &route_match, &priority, ofs); free(prefix_s); struct ds actions = DS_EMPTY_INITIALIZER; @@ -9193,8 +9265,9 @@ static void add_route(struct hmap *lflows, struct ovn_datapath *od, const struct ovn_port *op, const char *lrp_addr_s, const char *network_s, int plen, const char *gateway, - bool is_src_route, const struct ovsdb_idl_row *stage_hint, - bool is_discard_route, int ofs) + bool is_src_route, const uint32_t rtb_id, + const struct ovsdb_idl_row *stage_hint, bool is_discard_route, + int ofs) { bool is_ipv4 = strchr(network_s, '.') ? true : false; struct ds match = DS_EMPTY_INITIALIZER; @@ -9209,8 +9282,8 @@ add_route(struct hmap *lflows, struct ovn_datapath *od, op_inport = op; } } - build_route_match(op_inport, network_s, plen, is_src_route, is_ipv4, - &match, &priority, ofs); + build_route_match(op_inport, rtb_id, network_s, plen, is_src_route, + is_ipv4, &match, &priority, ofs); struct ds common_actions = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -9277,8 +9350,8 @@ build_static_route_flow(struct hmap *lflows, struct ovn_datapath *od, char *prefix_s = build_route_prefix_s(&route_->prefix, route_->plen); add_route(lflows, route_->is_discard_route ? od : out_port->od, out_port, lrp_addr_s, prefix_s, route_->plen, route->nexthop, - route_->is_src_route, &route->header_, route_->is_discard_route, - ofs); + route_->is_src_route, route_->route_table_id, &route->header_, + route_->is_discard_route, ofs); free(prefix_s); } @@ -10665,6 +10738,18 @@ build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, struct hmap *lflows) } } +/* Logical router ingress table IP_ROUTING_PRE: + * by default goto next. (priority 0). */ +static void +build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, + struct hmap *lflows) +{ + if (od->nbr) { + ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 0, "1", + REG_ROUTE_TABLE_ID" = 0; next;"); + } +} + /* Logical router ingress table IP_ROUTING : IP Routing. * * A packet that arrives at this table is an IP packet that should be @@ -10690,14 +10775,14 @@ build_ip_routing_flows_for_lrouter_port( for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { add_route(lflows, op->od, op, op->lrp_networks.ipv4_addrs[i].addr_s, op->lrp_networks.ipv4_addrs[i].network_s, - op->lrp_networks.ipv4_addrs[i].plen, NULL, false, + op->lrp_networks.ipv4_addrs[i].plen, NULL, false, 0, &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED); } for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { add_route(lflows, op->od, op, op->lrp_networks.ipv6_addrs[i].addr_s, op->lrp_networks.ipv6_addrs[i].network_s, - op->lrp_networks.ipv6_addrs[i].plen, NULL, false, + op->lrp_networks.ipv6_addrs[i].plen, NULL, false, 0, &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED); } } else if (lsp_is_router(op->nbsp)) { @@ -10720,7 +10805,7 @@ build_ip_routing_flows_for_lrouter_port( add_route(lflows, peer->od, peer, peer->lrp_networks.ipv4_addrs[0].addr_s, laddrs->ipv4_addrs[k].network_s, - laddrs->ipv4_addrs[k].plen, NULL, false, + laddrs->ipv4_addrs[k].plen, NULL, false, 0, &peer->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED); } @@ -10741,10 +10826,17 @@ build_static_route_flows_for_lrouter( struct hmap ecmp_groups = HMAP_INITIALIZER(&ecmp_groups); struct hmap unique_routes = HMAP_INITIALIZER(&unique_routes); struct ovs_list parsed_routes = OVS_LIST_INITIALIZER(&parsed_routes); + struct simap route_tables = SIMAP_INITIALIZER(&route_tables); struct ecmp_groups_node *group; + + for (int i = 0; i < od->nbr->n_ports; i++) { + build_route_table_lflow(od, lflows, od->nbr->ports[i], + &route_tables); + } + for (int i = 0; i < od->nbr->n_static_routes; i++) { struct parsed_route *route = - parsed_routes_add(od, ports, &parsed_routes, + parsed_routes_add(od, ports, &parsed_routes, &route_tables, od->nbr->static_routes[i], bfd_connections); if (!route) { continue; @@ -10777,6 +10869,7 @@ build_static_route_flows_for_lrouter( ecmp_groups_destroy(&ecmp_groups); unique_routes_destroy(&unique_routes); parsed_routes_destroy(&parsed_routes); + simap_destroy(&route_tables); } } @@ -12960,6 +13053,7 @@ build_lswitch_and_lrouter_iterate_by_od(struct ovn_datapath *od, build_neigh_learning_flows_for_lrouter(od, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); build_ND_RA_flows_for_lrouter(od, lsi->lflows); + build_ip_routing_pre_flows_for_lrouter(od, lsi->lflows); build_static_route_flows_for_lrouter(od, lsi->lflows, lsi->ports, lsi->bfd_connections); build_mcast_lookup_flows_for_lrouter(od, lsi->lflows, &lsi->match, diff --git a/northd/ovn-northd.8.xml b/northd/ovn-northd.8.xml index 4f3a9d5e3..e7dce5c16 100644 --- a/northd/ovn-northd.8.xml +++ b/northd/ovn-northd.8.xml @@ -3291,7 +3291,37 @@ output; -

Ingress Table 10: IP Routing

+

Ingress Table 10: IP Routing Pre

+ +

+ If a packet arrived at this table from Logical Router Port P + which has options:route_table value set, a logical flow with + match inport == "P" with priority 100 and action, + setting unique-generated per-datapath 32-bit value (non-zero) in OVS + register 7. This register's value is checked in next table. If packet + didn't match any configured inport (<main> route table), + register 7 value is set to 0. +

+ +

+ This table contains the following logical flows: +

+ +
    +
  • +

    + Priority-100 flow with match inport == "LRP_NAME" value + and action, which set route table identifier in reg7. +

    + +

    + A priority-0 logical flow with match 1 has actions + reg7 = 0; next;. +

    +
    • +
+ +

Ingress Table 11: IP Routing

A packet that arrives at this table is an IP packet that should be @@ -3331,14 +3361,14 @@ output;

  • - Priority-550 flow that drops IPv6 Router Solicitation/Advertisement + Priority-10550 flow that drops IPv6 Router Solicitation/Advertisement packets that were not processed in previous tables.

  • - Priority-500 flows that match IP multicast traffic destined to + Priority-10500 flows that match IP multicast traffic destined to groups registered on any of the attached switches and sets outport to the associated multicast group that will eventually flood the traffic to all interested attached logical @@ -3348,7 +3378,7 @@ output;

  • - Priority-450 flow that matches unregistered IP multicast traffic + Priority-10450 flow that matches unregistered IP multicast traffic and sets outport to the MC_STATIC multicast group, which ovn-northd populates with the logical ports that have @@ -3362,10 +3392,11 @@ output;

    IPv4 routing table. For each route to IPv4 network N with netmask M, on router port P with IP address - A and Ethernet - address E, a logical flow with match ip4.dst == - N/M, whose priority is the number of - 1-bits in M, has the following actions: + A and Ethernet address E, a logical flow with + match ip4.dst == N/M, whose + priority is the number of 1-bits in M multiplied by 3 + + route offset: 0 for src-ip route policy, 1 for dst-ip static routes. + This flow has the following actions: 

    @@ -3398,8 +3429,9 @@ next;
           P with IP address A and Ethernet address
           E, a logical flow with match in CIDR notation
           ip6.dst == N/M,
-          whose priority is the integer value of M, has the
-          following actions:
+          whose priority is the integer value of M multiplied by 3 +
+          route offset: 0 for src-ip route policy, 1 for dst-ip static routes.
+          This flow has the following actions:
         
    
 
         
    @@ -3428,6 +3460,20 @@ next;
           If the address A is in the link-local scope, the
           route will be limited to sending on the ingress port.
         
    
+
+        
    
+          For each static route the reg7 == id && is
+          prefixed in logical flow match portion.  For routes with
+          route_table value set a unique non-zero id is used.
+          For routes within <main> route table (no route
+          table set), this id value is 0.
+        
    
+        
    
+          For each connected route (route to the LRP's subnet CIDR)
+          the logical flow match portion has no
+          reg7 == id && prefix to have route to LRP's
+          subnets in all routing tables.
+
  • @@ -3454,7 +3500,7 @@ select(reg8[16..31], MID1, MID2, ...);
-

Ingress Table 11: IP_ROUTING_ECMP

+

Ingress Table 12: IP_ROUTING_ECMP

This table implements the second part of IP routing for ECMP routes @@ -3506,7 +3552,7 @@ outport = P; -

Ingress Table 12: Router policies

+

Ingress Table 13: Router policies

This table adds flows for the logical router policies configured on the logical router. Please see the @@ -3578,7 +3624,7 @@ next; -

Ingress Table 13: ECMP handling for router policies

+

Ingress Table 14: ECMP handling for router policies

This table handles the ECMP for the router policies configured with multiple nexthops. @@ -3622,7 +3668,7 @@ outport = P -

Ingress Table 14: ARP/ND Resolution

+

Ingress Table 15: ARP/ND Resolution

Any packet that reaches this table is an IP packet whose next-hop @@ -3813,7 +3859,7 @@ outport = P -

Ingress Table 15: Check packet length

+

Ingress Table 16: Check packet length

For distributed logical routers or gateway routers with gateway @@ -3843,7 +3889,7 @@ REGBIT_PKT_LARGER = check_pkt_larger(L); next; and advances to the next table.

-

Ingress Table 16: Handle larger packets

+

Ingress Table 17: Handle larger packets

For distributed logical routers or gateway routers with gateway port @@ -3906,7 +3952,7 @@ icmp6 { and advances to the next table.

-

Ingress Table 17: Gateway Redirect

+

Ingress Table 18: Gateway Redirect

For distributed logical routers where one or more of the logical router @@ -3953,7 +3999,7 @@ icmp6 { -

Ingress Table 18: ARP Request

+

Ingress Table 19: ARP Request

In the common case where the Ethernet destination has been resolved, this diff --git a/ovn-nb.ovsschema b/ovn-nb.ovsschema index 5dee04fe9..55977339a 100644 --- a/ovn-nb.ovsschema +++ b/ovn-nb.ovsschema @@ -1,7 +1,7 @@ { "name": "OVN_Northbound", - "version": "5.33.1", - "cksum": "1931852754 30731", + "version": "5.34.1", + "cksum": "2177334725 30782", "tables": { "NB_Global": { "columns": { @@ -407,6 +407,7 @@ "isRoot": false}, "Logical_Router_Static_Route": { "columns": { + "route_table": {"type": "string"}, "ip_prefix": {"type": "string"}, "policy": {"type": {"key": {"type": "string", "enum": ["set", ["src-ip", diff --git a/ovn-nb.xml b/ovn-nb.xml index 036ffa64f..4a1d4abdf 100644 --- a/ovn-nb.xml +++ b/ovn-nb.xml @@ -2820,6 +2820,14 @@ prefix according to RFC3663

+ + + Designates lookup Logical_Router_Static_Routes with specified + route_table value. Routes to directly connected networks + from same Logical Router and routes without route_table + option set have higher priority than routes with + route_table option set. + @@ -2939,6 +2947,28 @@

+ +

+ Any string to place route to separate routing table. If Logical Router + Port has configured value in other than empty string, OVN + performs route lookup for all packets entering Logical Router ingress + pipeline from this port in the following manner: +

+ +
    +
  • + 1. First lookup among "global" routes: routes without + route_table value set and routes to directly connected + networks. +
    • +
  • + 2. Next lookup among routes with same route_table value + as specified in LRP's options:route_table field. +
    • +
+ + ovn-ic populates this key if the route is learned from the global database. In this case the value diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at index 7e8498b2f..bb4b98d0b 100644 --- a/tests/ovn-ic.at +++ b/tests/ovn-ic.at @@ -282,6 +282,7 @@ done AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl IPv4 Routes +Route Table
: 10.11.1.0/24 169.254.0.1 dst-ip 10.11.2.0/24 169.254.100.2 dst-ip (learned) 10.22.1.0/24 169.254.0.2 src-ip @@ -300,6 +301,7 @@ ovn_as az1 ovn-nbctl set nb_global . options:ic-route-learn=false OVS_WAIT_WHILE([ovn_as az1 ovn-nbctl lr-route-list lr1 | grep learned]) AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl IPv4 Routes +Route Table
: 10.11.1.0/24 169.254.0.1 dst-ip 10.22.1.0/24 169.254.0.2 src-ip ]) @@ -315,6 +317,7 @@ ovn_as az1 ovn-nbctl set nb_global . options:ic-route-adv=false OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned]) AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl IPv4 Routes +Route Table
: 10.11.2.0/24 169.254.0.1 dst-ip 10.22.2.0/24 169.254.0.2 src-ip ]) @@ -333,6 +336,7 @@ done # Default route should NOT get advertised or learned, by default. AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl IPv4 Routes +Route Table
: 10.11.1.0/24 169.254.100.1 dst-ip (learned) 10.11.2.0/24 169.254.0.1 dst-ip 10.22.2.0/24 169.254.0.2 src-ip diff --git a/tests/ovn-nbctl.at b/tests/ovn-nbctl.at index a8946fef8..911310aeb 100644 --- a/tests/ovn-nbctl.at +++ b/tests/ovn-nbctl.at @@ -1522,6 +1522,7 @@ AT_CHECK([ovn-nbctl --ecmp --policy=src-ip lr-route-add lr0 20.0.0.0/24 11.0.0.1 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl IPv4 Routes +Route Table
: 10.0.0.0/24 11.0.0.1 dst-ip 10.0.1.0/24 11.0.1.1 dst-ip lp0 10.0.10.0/24 dst-ip lp0 @@ -1536,6 +1537,7 @@ AT_CHECK([ovn-nbctl lrp-add lr0 lp1 f0:00:00:00:00:02 11.0.0.254/24]) AT_CHECK([ovn-nbctl --may-exist lr-route-add lr0 10.0.0.111/24 11.0.0.1 lp1]) AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl IPv4 Routes +Route Table
: 10.0.0.0/24 11.0.0.1 dst-ip lp1 10.0.1.0/24 11.0.1.1 dst-ip lp0 10.0.10.0/24 dst-ip lp0 @@ -1566,6 +1568,7 @@ AT_CHECK([ovn-nbctl --policy=src-ip lr-route-del lr0 9.16.1.0/24]) AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl IPv4 Routes +Route Table
: 10.0.0.0/24 11.0.0.1 dst-ip lp1 10.0.10.0/24 dst-ip lp0 10.0.0.0/24 11.0.0.2 src-ip @@ -1577,6 +1580,7 @@ AT_CHECK([ovn-nbctl --policy=dst-ip lr-route-del lr0 10.0.0.0/24]) AT_CHECK([ovn-nbctl --policy=src-ip lr-route-del lr0 10.0.0.0/24]) AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl IPv4 Routes +Route Table
: 10.0.10.0/24 dst-ip lp0 0.0.0.0/0 192.168.0.1 dst-ip ]) @@ -1587,6 +1591,7 @@ AT_CHECK([ovn-nbctl --policy=src-ip lr-route-add lr0 10.0.0.0/24 11.0.0.2]) AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24]) AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl IPv4 Routes +Route Table
: 10.0.10.0/24 dst-ip lp0 0.0.0.0/0 192.168.0.1 dst-ip ]) @@ -1603,6 +1608,7 @@ AT_CHECK([ovn-nbctl --ecmp lr-route-add lr0 10.0.0.0/24 11.0.0.3]) AT_CHECK([ovn-nbctl --ecmp lr-route-add lr0 10.0.0.0/24 11.0.0.4 lp0]) AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl IPv4 Routes +Route Table
: 10.0.0.0/24 11.0.0.1 dst-ip ecmp 10.0.0.0/24 11.0.0.2 dst-ip ecmp 10.0.0.0/24 11.0.0.3 dst-ip ecmp @@ -1617,6 +1623,7 @@ dnl Delete ecmp routes AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24 11.0.0.1]) AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl IPv4 Routes +Route Table
: 10.0.0.0/24 11.0.0.2 dst-ip ecmp 10.0.0.0/24 11.0.0.3 dst-ip ecmp 10.0.0.0/24 11.0.0.4 dst-ip lp0 ecmp @@ -1624,12 +1631,14 @@ IPv4 Routes AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24 11.0.0.2]) AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl IPv4 Routes +Route Table
: 10.0.0.0/24 11.0.0.3 dst-ip ecmp 10.0.0.0/24 11.0.0.4 dst-ip lp0 ecmp ]) AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24 11.0.0.4 lp0]) AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl IPv4 Routes +Route Table
: 10.0.0.0/24 11.0.0.3 dst-ip ]) AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24 11.0.0.3]) @@ -1643,6 +1652,7 @@ AT_CHECK([ovn-nbctl lr-route-add lr0 2001:0db8:1::/64 2001:0db8:0:f103::1]) AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl IPv6 Routes +Route Table
: 2001:db8::/64 2001:db8:0:f102::1 dst-ip lp0 2001:db8:1::/64 2001:db8:0:f103::1 dst-ip ::/0 2001:db8:0:f101::1 dst-ip @@ -1652,6 +1662,7 @@ AT_CHECK([ovn-nbctl lr-route-del lr0 2001:0db8:0::/64]) AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl IPv6 Routes +Route Table
: 2001:db8:1::/64 2001:db8:0:f103::1 dst-ip ::/0 2001:db8:0:f101::1 dst-ip ]) @@ -1679,11 +1690,13 @@ AT_CHECK([ovn-nbctl --may-exist --ecmp-symmetric-reply lr-route-add lr0 2003:0db AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl IPv4 Routes +Route Table
: 10.0.0.0/24 11.0.0.1 dst-ip 10.0.1.0/24 11.0.1.1 dst-ip lp0 0.0.0.0/0 192.168.0.1 dst-ip IPv6 Routes +Route Table
: 2001:db8::/64 2001:db8:0:f102::1 dst-ip lp0 2001:db8:1::/64 2001:db8:0:f103::1 dst-ip ecmp 2001:db8:1::/64 2001:db8:0:f103::2 dst-ip ecmp @@ -1698,7 +1711,188 @@ AT_CHECK([ovn-nbctl lrp-add lr0 lr0-p0 00:00:01:01:02:03 192.168.10.1/24]) bfd_uuid=$(ovn-nbctl create bfd logical_port=lr0-p0 dst_ip=100.0.0.50 status=down min_tx=250 min_rx=250 detect_mult=10) AT_CHECK([ovn-nbctl lr-route-add lr0 100.0.0.0/24 192.168.0.1]) route_uuid=$(fetch_column nb:logical_router_static_route _uuid ip_prefix="100.0.0.0/24") -AT_CHECK([ovn-nbctl set logical_router_static_route $route_uuid bfd=$bfd_uuid])]) +AT_CHECK([ovn-nbctl set logical_router_static_route $route_uuid bfd=$bfd_uuid]) + +check ovn-nbctl lr-route-del lr0 +AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl +]) + +dnl Check IPv4 routes in route table +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 0.0.0.0/0 192.168.0.1 +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.1.1/24 11.0.1.1 lp0 +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.0.1/24 11.0.0.1 +AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl +IPv4 Routes +Route Table rtb-1: + 10.0.0.0/24 11.0.0.1 dst-ip + 10.0.1.0/24 11.0.1.1 dst-ip lp0 + 0.0.0.0/0 192.168.0.1 dst-ip +]) + +check ovn-nbctl lr-route-del lr0 +AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl +]) + +dnl Check IPv6 routes in route table +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 0:0:0:0:0:0:0:0/0 2001:0db8:0:f101::1 +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 2001:0db8:0::/64 2001:0db8:0:f102::1 lp0 +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 2001:0db8:1::/64 2001:0db8:0:f103::1 + +AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl +IPv6 Routes +Route Table rtb-1: + 2001:db8::/64 2001:db8:0:f102::1 dst-ip lp0 + 2001:db8:1::/64 2001:db8:0:f103::1 dst-ip + ::/0 2001:db8:0:f101::1 dst-ip +]) + +dnl Check IPv4 and IPv6 routes in route table +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 0.0.0.0/0 192.168.0.1 +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.1.1/24 11.0.1.1 lp0 +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.0.1/24 11.0.0.1 + +AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl +IPv4 Routes +Route Table rtb-1: + 10.0.0.0/24 11.0.0.1 dst-ip + 10.0.1.0/24 11.0.1.1 dst-ip lp0 + 0.0.0.0/0 192.168.0.1 dst-ip + +IPv6 Routes +Route Table rtb-1: + 2001:db8::/64 2001:db8:0:f102::1 dst-ip lp0 + 2001:db8:1::/64 2001:db8:0:f103::1 dst-ip + ::/0 2001:db8:0:f101::1 dst-ip +]) + +# Add routes in another route table +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 0.0.0.0/0 192.168.0.1 +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 10.0.1.1/24 11.0.1.1 lp0 +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 10.0.0.1/24 11.0.0.1 +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 0:0:0:0:0:0:0:0/0 2001:0db8:0:f101::1 +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 2001:0db8:0::/64 2001:0db8:0:f102::1 lp0 +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 2001:0db8:1::/64 2001:0db8:0:f103::1 + +AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl +IPv4 Routes +Route Table rtb-1: + 10.0.0.0/24 11.0.0.1 dst-ip + 10.0.1.0/24 11.0.1.1 dst-ip lp0 + 0.0.0.0/0 192.168.0.1 dst-ip + +Route Table rtb-2: + 10.0.0.0/24 11.0.0.1 dst-ip + 10.0.1.0/24 11.0.1.1 dst-ip lp0 + 0.0.0.0/0 192.168.0.1 dst-ip + +IPv6 Routes +Route Table rtb-1: + 2001:db8::/64 2001:db8:0:f102::1 dst-ip lp0 + 2001:db8:1::/64 2001:db8:0:f103::1 dst-ip + ::/0 2001:db8:0:f101::1 dst-ip + +Route Table rtb-2: + 2001:db8::/64 2001:db8:0:f102::1 dst-ip lp0 + 2001:db8:1::/64 2001:db8:0:f103::1 dst-ip + ::/0 2001:db8:0:f101::1 dst-ip +]) + +# Add routes to
route table +check ovn-nbctl lr-route-add lr0 0.0.0.0/0 192.168.0.1 +check ovn-nbctl lr-route-add lr0 10.0.1.1/24 11.0.1.1 lp0 +check ovn-nbctl lr-route-add lr0 10.0.0.1/24 11.0.0.1 +check ovn-nbctl lr-route-add lr0 0:0:0:0:0:0:0:0/0 2001:0db8:0:f101::1 +check ovn-nbctl lr-route-add lr0 2001:0db8:0::/64 2001:0db8:0:f102::1 lp0 +check check ovn-nbctl lr-route-add lr0 2001:0db8:1::/64 2001:0db8:0:f103::1 + +AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl +IPv4 Routes +Route Table
: + 10.0.0.0/24 11.0.0.1 dst-ip + 10.0.1.0/24 11.0.1.1 dst-ip lp0 + 0.0.0.0/0 192.168.0.1 dst-ip + +Route Table rtb-1: + 10.0.0.0/24 11.0.0.1 dst-ip + 10.0.1.0/24 11.0.1.1 dst-ip lp0 + 0.0.0.0/0 192.168.0.1 dst-ip + +Route Table rtb-2: + 10.0.0.0/24 11.0.0.1 dst-ip + 10.0.1.0/24 11.0.1.1 dst-ip lp0 + 0.0.0.0/0 192.168.0.1 dst-ip + +IPv6 Routes +Route Table
: + 2001:db8::/64 2001:db8:0:f102::1 dst-ip lp0 + 2001:db8:1::/64 2001:db8:0:f103::1 dst-ip + ::/0 2001:db8:0:f101::1 dst-ip + +Route Table rtb-1: + 2001:db8::/64 2001:db8:0:f102::1 dst-ip lp0 + 2001:db8:1::/64 2001:db8:0:f103::1 dst-ip + ::/0 2001:db8:0:f101::1 dst-ip + +Route Table rtb-2: + 2001:db8::/64 2001:db8:0:f102::1 dst-ip lp0 + 2001:db8:1::/64 2001:db8:0:f103::1 dst-ip + ::/0 2001:db8:0:f101::1 dst-ip +]) + +# delete IPv4 route from rtb-1 +check ovn-nbctl --route-table=rtb-1 lr-route-del lr0 10.0.0.0/24 +AT_CHECK([ovn-nbctl --route-table=rtb-1 lr-route-list lr0], [0], [dnl +IPv4 Routes +Route Table rtb-1: + 10.0.1.0/24 11.0.1.1 dst-ip lp0 + 0.0.0.0/0 192.168.0.1 dst-ip + +IPv6 Routes +Route Table rtb-1: + 2001:db8::/64 2001:db8:0:f102::1 dst-ip lp0 + 2001:db8:1::/64 2001:db8:0:f103::1 dst-ip + ::/0 2001:db8:0:f101::1 dst-ip +]) + +# delete IPv6 route from rtb-2 +check ovn-nbctl --route-table=rtb-2 lr-route-del lr0 2001:db8::/64 +AT_CHECK([ovn-nbctl --route-table=rtb-2 lr-route-list lr0], [0], [dnl +IPv4 Routes +Route Table rtb-2: + 10.0.0.0/24 11.0.0.1 dst-ip + 10.0.1.0/24 11.0.1.1 dst-ip lp0 + 0.0.0.0/0 192.168.0.1 dst-ip + +IPv6 Routes +Route Table rtb-2: + 2001:db8:1::/64 2001:db8:0:f103::1 dst-ip + ::/0 2001:db8:0:f101::1 dst-ip +]) + +check ovn-nbctl lr-route-del lr0 + +# ECMP route in route table +check ovn-nbctl --route-table=rtb1 lr-route-add lr0 0.0.0.0/0 192.168.0.1 +check ovn-nbctl --ecmp --route-table=rtb1 lr-route-add lr0 0.0.0.0/0 192.168.0.2 + +# Negative route table case: same prefix +AT_CHECK([ovn-nbctl --route-table=rtb1 lr-route-add lr0 0.0.0.0/0 192.168.0.1], [1], [], [dnl +ovn-nbctl: duplicate prefix: 0.0.0.0/0 (policy: dst-ip). Use option --ecmp to allow this for ECMP routing. +]) + +# Negative route table case: same prefix & nexthop with ecmp +AT_CHECK([ovn-nbctl --ecmp --route-table=rtb1 lr-route-add lr0 0.0.0.0/0 192.168.0.2], [1], [], [dnl +ovn-nbctl: duplicate nexthop for the same ECMP route +]) + +# Add routes to
route table +check ovn-nbctl lrp-add lr0 lrp0 00:00:00:00:00:01 1.1.1.1/24 +check ovn-nbctl lrp-set-options lrp0 route_table=rtb1 +AT_CHECK([ovn-nbctl get logical-router-port lrp0 options:route_table], [0], [dnl +rtb1 +]) +check `ovn-nbctl show lr0 | grep lrp0 -A3 | grep route_table=rtb1` +]) dnl --------------------------------------------------------------------- diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 3c1a97f73..333793af0 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -5430,7 +5430,7 @@ check ovn-nbctl --wait=sb --ecmp-symmetric-reply lr-route-add lr0 1.0.0.1 192.16 ovn-sbctl dump-flows lr0 > lr0flows AT_CHECK([grep -e "lr_in_ip_routing.*select" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl - table=??(lr_in_ip_routing ), priority=97 , match=(ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);) + table=??(lr_in_ip_routing ), priority=97 , match=(reg7 == 0 && ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);) ]) AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed 's/192\.168\.0\..0/192.168.0.??/' | sed 's/table=../table=??/' | sort], [0], [dnl table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == 1 && reg8[[16..31]] == 1), action=(reg0 = 192.168.0.??; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; next;) @@ -5443,7 +5443,7 @@ check ovn-nbctl --wait=sb --ecmp-symmetric-reply lr-route-add lr0 1.0.0.1 192.16 ovn-sbctl dump-flows lr0 > lr0flows AT_CHECK([grep -e "lr_in_ip_routing.*select" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl - table=??(lr_in_ip_routing ), priority=97 , match=(ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);) + table=??(lr_in_ip_routing ), priority=97 , match=(reg7 == 0 && ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);) ]) AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed 's/192\.168\.0\..0/192.168.0.??/' | sed 's/table=../table=??/' | sort], [0], [dnl table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == 1 && reg8[[16..31]] == 1), action=(reg0 = 192.168.0.??; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; next;) @@ -5458,14 +5458,14 @@ check ovn-nbctl --wait=sb lr-route-add lr0 1.0.0.0/24 192.168.0.10 ovn-sbctl dump-flows lr0 > lr0flows AT_CHECK([grep -e "lr_in_ip_routing.*192.168.0.10" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl - table=??(lr_in_ip_routing ), priority=73 , match=(ip4.dst == 1.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;) + table=??(lr_in_ip_routing ), priority=73 , match=(reg7 == 0 && ip4.dst == 1.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;) ]) check ovn-nbctl --wait=sb lr-route-add lr0 2.0.0.0/24 lr0-public ovn-sbctl dump-flows lr0 > lr0flows AT_CHECK([grep -e "lr_in_ip_routing.*2.0.0.0" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl - table=??(lr_in_ip_routing ), priority=73 , match=(ip4.dst == 2.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;) + table=??(lr_in_ip_routing ), priority=73 , match=(reg7 == 0 && ip4.dst == 2.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;) ]) AT_CLEANUP @@ -5655,3 +5655,69 @@ ct_dnat /* assuming no un-dnat entry, so no change */ /* default (use --ct to cu AT_CLEANUP ]) + +AT_SETUP([route tables -- flows]) +AT_KEYWORDS([route-tables-flows]) +ovn_start + +check ovn-nbctl lr-add lr0 +check ovn-nbctl lrp-add lr0 lrp0 00:00:00:00:00:01 192.168.0.1/24 +check ovn-nbctl lrp-add lr0 lrp1 00:00:00:00:01:01 192.168.1.1/24 +check ovn-nbctl lrp-add lr0 lrp2 00:00:00:00:02:01 192.168.2.1/24 +check ovn-nbctl lrp-set-options lrp1 route_table=rtb-1 +check ovn-nbctl lrp-set-options lrp2 route_table=rtb-2 + +check ovn-nbctl lr-route-add lr0 0.0.0.0/0 192.168.0.10 +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 192.168.0.0/24 192.168.1.10 +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 0.0.0.0/0 192.168.0.10 +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 1.1.1.1/32 192.168.0.20 +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 2.2.2.2/32 192.168.0.30 +check ovn-nbctl --route-table=rtb-2 --ecmp lr-route-add lr0 2.2.2.2/32 192.168.0.31 +check ovn-nbctl --wait=sb sync + +ovn-sbctl dump-flows lr0 > lr0flows +AT_CAPTURE_FILE([lr0flows]) + +AT_CHECK([grep -e "lr_in_ip_routing_pre.*match=(1)" lr0flows | sed 's/table=../table=??/'], [0], [dnl + table=??(lr_in_ip_routing_pre), priority=0 , match=(1), action=(reg7 = 0; next;) +]) + +p1_reg=$(grep -oP "lr_in_ip_routing_pre.*lrp1.*action=\(reg7 = \K." lr0flows) +p2_reg=$(grep -oP "lr_in_ip_routing_pre.*lrp2.*action=\(reg7 = \K." lr0flows) +echo $p1_reg +echo $p2_reg + +# exact register values are not predictable +if [[ $p1_reg -eq 2 ] && [ $p2_reg -eq 1 ]]; then + echo "swap reg values in dump" + sed -i -r s'/^(.*lrp2.*action=\(reg7 = )(1)(.*)/\12\3/g' lr0flows # "reg7 = 1" -> "reg7 = 2" + sed -i -r s'/^(.*lrp1.*action=\(reg7 = )(2)(.*)/\11\3/g' lr0flows # "reg7 = 2" -> "reg7 = 1" + sed -i -r s'/^(.*match=\(reg7 == )(2)( &&.*lrp1.*)/\11\3/g' lr0flows # "reg7 == 2" -> "reg7 == 1" + sed -i -r s'/^(.*match=\(reg7 == )(1)( &&.*lrp0.*)/\12\3/g' lr0flows # "reg7 == 1" -> "reg7 == 2" +fi + +check test "$p1_reg" != "$p2_reg" -a $((p1_reg * p2_reg)) -eq 2 + +AT_CHECK([grep "lr_in_ip_routing_pre" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl + table=??(lr_in_ip_routing_pre), priority=0 , match=(1), action=(reg7 = 0; next;) + table=??(lr_in_ip_routing_pre), priority=100 , match=(inport == "lrp1"), action=(reg7 = 1; next;) + table=??(lr_in_ip_routing_pre), priority=100 , match=(inport == "lrp2"), action=(reg7 = 2; next;) +]) + +grep -e "(lr_in_ip_routing ).*outport" lr0flows + +AT_CHECK([grep -e "(lr_in_ip_routing ).*outport" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl + table=??(lr_in_ip_routing ), priority=1 , match=(reg7 == 0 && ip4.dst == 0.0.0.0/0), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:00:00:00:01; outport = "lrp0"; flags.loopback = 1; next;) + table=??(lr_in_ip_routing ), priority=1 , match=(reg7 == 2 && ip4.dst == 0.0.0.0/0), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:00:00:00:01; outport = "lrp0"; flags.loopback = 1; next;) + table=??(lr_in_ip_routing ), priority=194 , match=(inport == "lrp0" && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:1; eth.src = 00:00:00:00:00:01; outport = "lrp0"; flags.loopback = 1; next;) + table=??(lr_in_ip_routing ), priority=194 , match=(inport == "lrp1" && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:101; eth.src = 00:00:00:00:01:01; outport = "lrp1"; flags.loopback = 1; next;) + table=??(lr_in_ip_routing ), priority=194 , match=(inport == "lrp2" && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:201; eth.src = 00:00:00:00:02:01; outport = "lrp2"; flags.loopback = 1; next;) + table=??(lr_in_ip_routing ), priority=73 , match=(reg7 == 1 && ip4.dst == 192.168.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.1.10; reg1 = 192.168.1.1; eth.src = 00:00:00:00:01:01; outport = "lrp1"; flags.loopback = 1; next;) + table=??(lr_in_ip_routing ), priority=74 , match=(ip4.dst == 192.168.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.0.1; eth.src = 00:00:00:00:00:01; outport = "lrp0"; flags.loopback = 1; next;) + table=??(lr_in_ip_routing ), priority=74 , match=(ip4.dst == 192.168.1.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.1.1; eth.src = 00:00:00:00:01:01; outport = "lrp1"; flags.loopback = 1; next;) + table=??(lr_in_ip_routing ), priority=74 , match=(ip4.dst == 192.168.2.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.2.1; eth.src = 00:00:00:00:02:01; outport = "lrp2"; flags.loopback = 1; next;) + table=??(lr_in_ip_routing ), priority=97 , match=(reg7 == 2 && ip4.dst == 1.1.1.1/32), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.20; reg1 = 192.168.0.1; eth.src = 00:00:00:00:00:01; outport = "lrp0"; flags.loopback = 1; next;) +]) + +AT_CLEANUP +]) diff --git a/tests/ovn.at b/tests/ovn.at index 51e0dae0b..28e5d679b 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -18257,7 +18257,7 @@ eth_dst=00000000ff01 ip_src=$(ip_to_hex 10 0 0 10) ip_dst=$(ip_to_hex 172 168 0 101) send_icmp_packet 1 1 $eth_src $eth_dst $ip_src $ip_dst c4c9 0000000000000000000000 -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | awk '/table=25, n_packets=1, n_bytes=45/{print $7" "$8}'],[0],[dnl +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | awk '/table=26, n_packets=1, n_bytes=45/{print $7" "$8}'],[0],[dnl priority=80,ip,reg15=0x3,metadata=0x3,nw_src=10.0.0.10 actions=drop ]) @@ -22698,6 +22698,538 @@ OVN_CLEANUP([hv1]) AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([route tables --
route table routes]) +ovn_start + +# Logical network: +# ls1 (192.168.1.0/24) - lrp-lr1-ls1 - lr1 - lrp-lr1-ls2 - ls2 (192.168.2.0/24) +# +# ls1 has lsp11 (192.168.1.11) and +# ls2 has lsp21 (192.168.2.21) and lsp22 (192.168.2.22) +# +# lrp-lr1-ls1 set options:route_table=rtb-1 +# +# Static routes on lr1: +# 0.0.0.0/0 nexthop 192.168.2.21 +# 1.1.1.1/32 nexthop 192.168.2.22 route_table=rtb-1 +# +# Test 1: +# lsp11 send packet to 2.2.2.2 +# +# Expected result: +# each port should receive no traffic +# +# Test 2: +# lsp11 send packet to 1.1.1.1 +# +# Expected result: +# lsp22 should receive traffic, lsp21 should not +# +# Test 3: +# lsp21 send packet to 1.1.1.1 +# +# Expected result: +# lsp11 should receive traffic, lsp21 should not receive any traffic + +ovn-nbctl lr-add lr1 + +for i in 1 2; do + ovn-nbctl ls-add ls${i} + ovn-nbctl lrp-add lr1 lrp-lr1-ls${i} 00:00:00:01:0${i}:01 192.168.${i}.1/24 + ovn-nbctl lsp-add ls${i} lsp-ls${i}-lr1 -- lsp-set-type lsp-ls${i}-lr1 router \ + -- lsp-set-options lsp-ls${i}-lr1 router-port=lrp-lr1-ls${i} \ + -- lsp-set-addresses lsp-ls${i}-lr1 router +done + +# install static routes +ovn-nbctl lr-route-add lr1 0.0.0.0/0 192.168.2.21 +ovn-nbctl --route-table=rtb-1 lr-route-add lr1 1.1.1.1/32 192.168.2.22 + +# set lrp-lr1-ls1 route table +ovn-nbctl lrp-set-options lrp-lr1-ls1 route_table=rtb-1 + +# Create logical ports +ovn-nbctl lsp-add ls1 lsp11 -- \ + lsp-set-addresses lsp11 "f0:00:00:00:01:11 192.168.1.11" +ovn-nbctl lsp-add ls2 lsp21 -- \ + lsp-set-addresses lsp21 "f0:00:00:00:02:21 192.168.2.21" +ovn-nbctl lsp-add ls2 lsp22 -- \ + lsp-set-addresses lsp22 "f0:00:00:00:02:22 192.168.2.22" + +net_add n1 +sim_add hv1 +as hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.1 +ovs-vsctl -- add-port br-int hv1-vif1 -- \ + set interface hv1-vif1 external-ids:iface-id=lsp11 \ + options:tx_pcap=hv1/vif1-tx.pcap \ + options:rxq_pcap=hv1/vif1-rx.pcap \ + ofport-request=1 + +ovs-vsctl -- add-port br-int hv1-vif2 -- \ + set interface hv1-vif2 external-ids:iface-id=lsp21 \ + options:tx_pcap=hv1/vif2-tx.pcap \ + options:rxq_pcap=hv1/vif2-rx.pcap \ + ofport-request=2 + +ovs-vsctl -- add-port br-int hv1-vif3 -- \ + set interface hv1-vif3 external-ids:iface-id=lsp22 \ + options:tx_pcap=hv1/vif3-tx.pcap \ + options:rxq_pcap=hv1/vif3-rx.pcap \ + ofport-request=3 + +# wait for earlier changes to take effect +check ovn-nbctl --wait=hv sync +wait_for_ports_up + +# Test 1 +packet="inport==\"lsp11\" && eth.src==f0:00:00:00:01:11 && eth.dst==00:00:00:01:01:01 && + ip4 && ip.ttl==64 && ip4.src==192.168.1.11 && ip4.dst==2.2.2.2 && icmp" +AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"]) + +# Assume no packets go neither to lsp21 nor to lsp22. +> expected_lsp21 +> expected_lsp22 + +ovn-sbctl dump-flows lr1 + + +OVS_WAIT_UNTIL([ + rcv_n1=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > lsp21.packets && cat lsp21.packets | wc -l` + rcv_n2=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif3-tx.pcap > lsp22.packets && cat lsp22.packets | wc -l` + echo $rcv_n1 $rcv_n2 + test $rcv_n1 -eq 0 -a $rcv_n2 -eq 0]) + +for i in 1 2; do + sort expected_lsp2$i > expout + AT_CHECK([cat lsp2${i}.packets | sort], [0], [expout]) +done + +# cleanup pcap +for i in 2 3; do + ovs-vsctl remove interface hv1-vif$i options tx_pcap + > hv1/vif$i-tx.pcap + ovs-vsctl set interface hv1-vif$i options:tx_pcap=hv1/vif$i-tx.pcap +done + +# Test 2 +packet="inport==\"lsp11\" && eth.src==f0:00:00:00:01:11 && eth.dst==00:00:00:01:01:01 && + ip4 && ip.ttl==64 && ip4.src==192.168.1.11 && ip4.dst==1.1.1.1 && icmp" +AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"]) + +# Assume all packets go to lsp22. +exp_packet="eth.src==00:00:00:01:02:01 && eth.dst==f0:00:00:00:02:22 && + ip4 && ip.ttl==63 && ip4.src==192.168.1.11 && ip4.dst==1.1.1.1 && icmp" +echo $exp_packet | ovstest test-ovn expr-to-packets > expected_lsp22 +> expected_lsp21 + +# lsp21 should recieve 1 packet and lsp22 should recieve no packets +OVS_WAIT_UNTIL([ + rcv_n1=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > lsp21.packets && cat lsp21.packets | wc -l` + rcv_n2=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif3-tx.pcap > lsp22.packets && cat lsp22.packets | wc -l` + echo $rcv_n1 $rcv_n2 + test $rcv_n1 -eq 0 -a $rcv_n2 -eq 1]) + +for i in 1 2; do + sort expected_lsp2$i > expout + AT_CHECK([cat lsp2${i}.packets | sort], [0], [expout]) +done + +# cleanup pcap +for i in 2 3; do + ovs-vsctl remove interface hv1-vif$i options tx_pcap + > hv1/vif$i-tx.pcap + ovs-vsctl set interface hv1-vif$i options:tx_pcap=hv1/vif$i-tx.pcap +done + +# Test 3 +packet="inport==\"lsp21\" && eth.src==f0:00:00:00:02:21 && eth.dst==00:00:00:01:02:01 && + ip4 && ip.ttl==64 && ip4.src==192.168.2.21 && ip4.dst==2.2.2.2 && icmp" +AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"]) + +# Assume all packets go to lsp21. +exp_packet="eth.src==00:00:00:01:02:01 && eth.dst==f0:00:00:00:02:21 && + ip4 && ip.ttl==63 && ip4.src==192.168.2.21 && ip4.dst==2.2.2.2 && icmp" +echo $exp_packet | ovstest test-ovn expr-to-packets > expected_lsp21 +> expected_lsp22 + +# lsp21 should recieve 1 packet and lsp22 should recieve no packets +OVS_WAIT_UNTIL([ + rcv_n1=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > lsp21.packets && cat lsp21.packets | wc -l` + rcv_n2=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif3-tx.pcap > lsp22.packets && cat lsp22.packets | wc -l` + echo $rcv_n1 $rcv_n2 + test $rcv_n1 -eq 1 -a $rcv_n2 -eq 0]) + +for i in 1 2; do + sort expected_lsp2$i > expout + AT_CHECK([cat lsp2${i}.packets | sort], [0], [expout]) +done + +OVN_CLEANUP([hv1]) +AT_CLEANUP +]) + + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([route tables -- directly connected routes]) +ovn_start + +# Logical network: +# ls1 (192.168.1.0/24) - lrp-lr1-ls1 - lr1 - lrp-lr1-ls2 - ls2 (192.168.2.0/24) +# +# ls1 has lsp11 (192.168.1.11) and ls2 has lsp21 (192.168.2.21) +# +# lrp-lr1-ls1 set options:route_table=rtb-1 +# +# Static routes on lr1: +# 192.168.2.128/25 nexthop 192.168.1.11 route_table=rtb-1 +# +# Test 1: +# lsp11 send packet to 192.168.2.21 +# +# Expected result: +# lsp21 should receive traffic, lsp11 should not receive any traffic +# +# Test 2: +# lsp11 send packet to 192.168.2.200 +# +# Expected result: +# lsp11 should receive traffic, lsp21 should not receive any traffic + +ovn-nbctl lr-add lr1 + +for i in 1 2; do + ovn-nbctl ls-add ls${i} + ovn-nbctl lrp-add lr1 lrp-lr1-ls${i} 00:00:00:01:0${i}:01 192.168.${i}.1/24 + ovn-nbctl lsp-add ls${i} lsp-ls${i}-lr1 -- lsp-set-type lsp-ls${i}-lr1 router \ + -- lsp-set-options lsp-ls${i}-lr1 router-port=lrp-lr1-ls${i} \ + -- lsp-set-addresses lsp-ls${i}-lr1 router +done + +# install static route, which is more specific than directly-connected route +ovn-nbctl --route-table=rtb-1 lr-route-add lr1 192.168.2.128/25 192.168.1.11 + +# set lrp-lr1-ls1 route table +ovn-nbctl lrp-set-options lrp-lr1-ls1 route_table=rtb-1 + +# Create logical ports +ovn-nbctl lsp-add ls1 lsp11 -- \ + lsp-set-addresses lsp11 "f0:00:00:00:01:11 192.168.1.11" +ovn-nbctl lsp-add ls2 lsp21 -- \ + lsp-set-addresses lsp21 "f0:00:00:00:02:21 192.168.2.21" + +net_add n1 +sim_add hv1 +as hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.1 +ovs-vsctl -- add-port br-int hv1-vif1 -- \ + set interface hv1-vif1 external-ids:iface-id=lsp11 \ + options:tx_pcap=hv1/vif1-tx.pcap \ + options:rxq_pcap=hv1/vif1-rx.pcap \ + ofport-request=1 + +ovs-vsctl -- add-port br-int hv1-vif2 -- \ + set interface hv1-vif2 external-ids:iface-id=lsp21 \ + options:tx_pcap=hv1/vif2-tx.pcap \ + options:rxq_pcap=hv1/vif2-rx.pcap \ + ofport-request=2 + +# wait for earlier changes to take effect +check ovn-nbctl --wait=hv sync +wait_for_ports_up + +# test 1 +packet="inport==\"lsp11\" && eth.src==f0:00:00:00:01:11 && eth.dst==00:00:00:01:01:01 && + ip4 && ip.ttl==64 && ip4.src==192.168.1.11 && ip4.dst==192.168.2.21 && icmp" +AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"]) + +# Assume all packets go to lsp21. +exp_packet="eth.src==00:00:00:01:02:01 && eth.dst==f0:00:00:00:02:21 && ip4 && + ip.ttl==63 && ip4.src==192.168.1.11 && ip4.dst==192.168.2.21 && icmp" +echo $exp_packet | ovstest test-ovn expr-to-packets > expected_lsp21 +> expected_lsp11 + +# lsp21 should recieve 1 icmp packet and lsp11 should recieve no packets +OVS_WAIT_UNTIL([ + rcv_n11=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif1-tx.pcap > lsp11.packets && cat lsp11.packets | wc -l` + rcv_n21=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > lsp21.packets && cat lsp21.packets | wc -l` + echo $rcv_n11 $rcv_n21 + test $rcv_n11 -eq 0 -a $rcv_n21 -eq 1]) + +for i in 11 21; do + sort expected_lsp$i > expout + AT_CHECK([cat lsp${i}.packets | sort], [0], [expout]) +done + +for i in 11 21; do + > lsp${i}.packets +done + +# flush tx pcaps: +ovs-vsctl remove interface hv1-vif1 options tx_pcap +ovs-vsctl remove interface hv1-vif2 options tx_pcap +rm -f hv1/vif1-tx.pcap hv1/vif2-tx.pcap +ovs-vsctl set interface hv1-vif1 options:tx_pcap=hv1/vif1-tx.pcap +ovs-vsctl set interface hv1-vif2 options:tx_pcap=hv1/vif2-tx.pcap + +# test 2 +packet="inport==\"lsp11\" && eth.src==f0:00:00:00:01:11 && eth.dst==00:00:00:01:01:01 && + ip4 && ip.ttl==64 && ip4.src==192.168.1.11 && ip4.dst==192.168.2.200 && icmp" +AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"]) + +# Assume all packets go to lsp11. +exp_packet="eth.src==00:00:00:01:01:01 && eth.dst==f0:00:00:00:01:11 && ip4 && + ip.ttl==63 && ip4.src==192.168.1.11 && ip4.dst==192.168.2.200 && icmp" +echo $exp_packet | ovstest test-ovn expr-to-packets > expected_lsp11 +> expected_lsp21 + +# lsp11 should recieve 1 icmp packet and lsp21 should recieve no packets +OVS_WAIT_UNTIL([ + rcv_n11=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif1-tx.pcap > lsp11.packets && cat lsp11.packets | wc -l` + rcv_n21=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > lsp21.packets && cat lsp21.packets | wc -l` + echo $rcv_n11 $rcv_n21 + test $rcv_n11 -eq 1 -a $rcv_n21 -eq 0]) + +for i in 11 21; do + echo I=$i + sort expected_lsp$i > expout + AT_CHECK([cat lsp${i}.packets | sort], [0], [expout]) +done + +OVN_CLEANUP([hv1]) +AT_CLEANUP +]) + + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([route tables -- overlapping subnets]) +ovn_start + +# Logical network: +# +# ls1 (192.168.1.0/24) - lrp-lr1-ls1 -\ /- lrp-lr1-ls2 - ls2 (192.168.2.0/24) +# lr1 +# ls3 (192.168.3.0/24) - lrp-lr1-ls3 -/ \- lrp-lr1-ls4 - ls4 (192.168.4.0/24) +# +# ls1 has lsp11 (192.168.1.11) +# ls2 has lsp21 (192.168.2.21) +# ls3 has lsp31 (192.168.3.31) +# ls4 has lsp41 (192.168.4.41) +# +# lrp-lr1-ls1 set options:route_table=rtb-1 +# +# Static routes on lr1: +# 10.0.0.0/24 nexthop 192.168.3.31 route_table=rtb-1 +# 10.0.0.0/24 nexthop 192.168.4.41 (
route table) +# +# Test 1: +# lsp11 send packet to 10.0.0.1 +# +# Expected result: +# lsp31 should receive traffic, lsp41 should not receive any traffic +# +# Test 2: +# lsp21 send packet to 10.0.0.1 +# +# Expected result: +# lsp41 should receive traffic, lsp31 should not receive any traffic + +ovn-nbctl lr-add lr1 + +# Create logical topology +for i in $(seq 1 4); do + ovn-nbctl ls-add ls${i} + ovn-nbctl lrp-add lr1 lrp-lr1-ls${i} 00:00:00:01:0${i}:01 192.168.${i}.1/24 + ovn-nbctl lsp-add ls${i} lsp-ls${i}-lr1 -- lsp-set-type lsp-ls${i}-lr1 router \ + -- lsp-set-options lsp-ls${i}-lr1 router-port=lrp-lr1-ls${i} \ + -- lsp-set-addresses lsp-ls${i}-lr1 router + ovn-nbctl lsp-add ls$i lsp${i}1 -- \ + lsp-set-addresses lsp${i}1 "f0:00:00:00:0${i}:1${i} 192.168.${i}.${i}1" +done + +# install static routes +ovn-nbctl --route-table=rtb-1 lr-route-add lr1 10.0.0.0/24 192.168.3.31 +ovn-nbctl lr-route-add lr1 10.0.0.0/24 192.168.4.41 + +# set lrp-lr1-ls1 route table +ovn-nbctl lrp-set-options lrp-lr1-ls1 route_table=rtb-1 + +net_add n1 +sim_add hv1 +as hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.1 + +for i in $(seq 1 4); do + ovs-vsctl -- add-port br-int hv1-vif${i} -- \ + set interface hv1-vif${i} external-ids:iface-id=lsp${i}1 \ + options:tx_pcap=hv1/vif${i}-tx.pcap \ + options:rxq_pcap=hv1/vif${i}-rx.pcap \ + ofport-request=${i} +done + +# wait for earlier changes to take effect +check ovn-nbctl --wait=hv sync +wait_for_ports_up + +# lsp31 should recieve packet coming from lsp11 +# lsp41 should recieve packet coming from lsp21 +for i in $(seq 1 2); do + di=$(( i + 2)) # dst index + ri=$(( 5 - i)) # reverse index + packet="inport==\"lsp${i}1\" && eth.src==f0:00:00:00:0${i}:1${i} && + eth.dst==00:00:00:01:0${i}:01 && ip4 && ip.ttl==64 && + ip4.src==192.168.${i}.${i}1 && ip4.dst==10.0.0.1 && icmp" + AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"]) + + # Assume all packets go to lsp${di}1. + exp_packet="eth.src==00:00:00:01:0${di}:01 && eth.dst==f0:00:00:00:0${di}:1${di} && + ip4 && ip.ttl==63 && ip4.src==192.168.${i}.${i}1 && ip4.dst==10.0.0.1 && icmp" + echo $exp_packet | ovstest test-ovn expr-to-packets >> expected_lsp${di}1 + > expected_lsp${ri}1 + + OVS_WAIT_UNTIL([ + rcv_n1=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif${di}-tx.pcap > lsp${di}1.packets && cat lsp${di}1.packets | wc -l` + rcv_n2=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif${ri}-tx.pcap > lsp${ri}1.packets && cat lsp${ri}1.packets | wc -l` + echo $rcv_n1 $rcv_n2 + test $rcv_n1 -eq 1 -a $rcv_n2 -eq 0]) + + for j in "${di}1" "${ri}1"; do + sort expected_lsp${j} > expout + AT_CHECK([cat lsp${j}.packets | sort], [0], [expout]) + done + + # cleanup tx pcap files + for j in "${di}1" "${ri}1"; do + ovs-vsctl -- remove interface hv1-vif${di} options tx_pcap + > hv1/vif${di}-tx.pcap + ovs-vsctl -- set interface hv1-vif${di} external-ids:iface-id=lsp${di}1 \ + options:tx_pcap=hv1/vif${di}-tx.pcap + done +done + +OVN_CLEANUP([hv1]) +AT_CLEANUP +]) + + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([route tables IPv6 -- overlapping subnets]) +ovn_start + +# Logical network: +# +# ls1 (2001:db8:1::/64) - lrp-lr1-ls1 -\ /- lrp-lr1-ls2 - ls2 (2001:db8:2::/64) +# lr1 +# ls3 (2001:db8:3::/64) - lrp-lr1-ls3 -/ \- lrp-lr1-ls4 - ls4 (2001:db8:4::/64) +# +# ls1 has lsp11 (2001:db8:1::11) +# ls2 has lsp21 (2001:db8:2::21) +# ls3 has lsp31 (2001:db8:3::31) +# ls4 has lsp41 (2001:db8:4::41) +# +# lrp-lr1-ls1 set options:route_table=rtb-1 +# lrp-lr1-ls2 set options:route_table=rtb-2 +# +# Static routes on lr1: +# 2001:db8:2000::/64 nexthop 2001:db8:3::31 route_table=rtb-1 +# 2001:db8:2000::/64 nexthop 2001:db8:3::41 route_table=rtb-2 +# +# Test 1: +# lsp11 send packet to 2001:db8:2000::1 +# +# Expected result: +# lsp31 should receive traffic, lsp41 should not receive any traffic +# +# Test 2: +# lsp21 send packet to 2001:db8:2000::1 +# +# Expected result: +# lsp41 should receive traffic, lsp31 should not receive any traffic + +ovn-nbctl lr-add lr1 + +# Create logical topology +for i in $(seq 1 4); do + ovn-nbctl ls-add ls${i} + ovn-nbctl lrp-add lr1 lrp-lr1-ls${i} 00:00:00:01:0${i}:01 2001:db8:${i}::1/64 + ovn-nbctl lsp-add ls${i} lsp-ls${i}-lr1 -- lsp-set-type lsp-ls${i}-lr1 router \ + -- lsp-set-options lsp-ls${i}-lr1 router-port=lrp-lr1-ls${i} \ + -- lsp-set-addresses lsp-ls${i}-lr1 router + ovn-nbctl lsp-add ls$i lsp${i}1 -- \ + lsp-set-addresses lsp${i}1 "f0:00:00:00:0${i}:1${i} 2001:db8:${i}::${i}1" +done + +# install static routes +ovn-nbctl --route-table=rtb-1 lr-route-add lr1 2001:db8:2000::/64 2001:db8:3::31 +ovn-nbctl --route-table=rtb-2 lr-route-add lr1 2001:db8:2000::/64 2001:db8:4::41 + +# set lrp-lr1-ls{1,2} route tables +ovn-nbctl lrp-set-options lrp-lr1-ls1 route_table=rtb-1 +ovn-nbctl lrp-set-options lrp-lr1-ls2 route_table=rtb-2 + +net_add n1 +sim_add hv1 +as hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.1 + +for i in $(seq 1 4); do + ovs-vsctl -- add-port br-int hv1-vif${i} -- \ + set interface hv1-vif${i} external-ids:iface-id=lsp${i}1 \ + options:tx_pcap=hv1/vif${i}-tx.pcap \ + options:rxq_pcap=hv1/vif${i}-rx.pcap \ + ofport-request=${i} +done + +# wait for earlier changes to take effect +AT_CHECK([ovn-nbctl --timeout=3 --wait=hv sync], [0], [ignore]) + +# lsp31 should recieve packet coming from lsp11 +# lsp41 should recieve packet coming from lsp21 +for i in $(seq 1 2); do + di=$(( i + 2)) # dst index + ri=$(( 5 - i)) # reverse index + packet="inport==\"lsp${i}1\" && eth.src==f0:00:00:00:0${i}:1${i} && + eth.dst==00:00:00:01:0${i}:01 && ip6 && ip.ttl==64 && + ip6.src==2001:db8:${i}::${i}1 && ip6.dst==2001:db8:2000::1 && icmp6" + AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"]) + + # Assume all packets go to lsp${di}1. + exp_packet="eth.src==00:00:00:01:0${di}:01 && eth.dst==f0:00:00:00:0${di}:1${di} && ip6 && + ip.ttl==63 && ip6.src==2001:db8:${i}::${i}1 && ip6.dst==2001:db8:2000::1 && icmp6" + echo $exp_packet | ovstest test-ovn expr-to-packets >> expected_lsp${di}1 + > expected_lsp${ri}1 + + OVS_WAIT_UNTIL([ + rcv_n1=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif${di}-tx.pcap > lsp${di}1.packets && cat lsp${di}1.packets | wc -l` + rcv_n2=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif${ri}-tx.pcap > lsp${ri}1.packets && cat lsp${ri}1.packets | wc -l` + echo $rcv_n1 $rcv_n2 + test $rcv_n1 -eq 1 -a $rcv_n2 -eq 0]) + + for j in "${di}1" "${ri}1"; do + sort expected_lsp${j} > expout + AT_CHECK([cat lsp${j}.packets | sort], [0], [expout]) + done + + # cleanup tx pcap files + for j in "${di}1" "${ri}1"; do + ovs-vsctl -- remove interface hv1-vif${di} options tx_pcap + > hv1/vif${di}-tx.pcap + ovs-vsctl -- set interface hv1-vif${di} external-ids:iface-id=lsp${di}1 \ + options:tx_pcap=hv1/vif${di}-tx.pcap + done +done + +OVN_CLEANUP([hv1]) +AT_CLEANUP +]) + + OVN_FOR_EACH_NORTHD([ AT_SETUP([forwarding group: 3 HVs, 1 LR, 2 LS]) AT_KEYWORDS([forwarding-group]) @@ -23453,7 +23985,7 @@ ovn-sbctl dump-flows > sbflows AT_CAPTURE_FILE([sbflows]) AT_CAPTURE_FILE([offlows]) OVS_WAIT_UNTIL([ - as hv1 ovs-ofctl dump-flows br-int table=20 > offlows + as hv1 ovs-ofctl dump-flows br-int table=21 > offlows test $(grep -c "load:0x64->NXM_NX_PKT_MARK" offlows) = 1 && \ test $(grep -c "load:0x3->NXM_NX_PKT_MARK" offlows) = 1 && \ test $(grep -c "load:0x4->NXM_NX_PKT_MARK" offlows) = 1 && \ @@ -23546,12 +24078,12 @@ send_ipv4_pkt hv1 hv1-vif1 505400000003 00000000ff01 \ $(ip_to_hex 10 0 0 3) $(ip_to_hex 172 168 0 120) OVS_WAIT_UNTIL([ - test 1 -eq $(as hv1 ovs-ofctl dump-flows br-int table=20 | \ + test 1 -eq $(as hv1 ovs-ofctl dump-flows br-int table=21 | \ grep "load:0x2->NXM_NX_PKT_MARK" -c) ]) AT_CHECK([ - test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=20 | \ + test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=21 | \ grep "load:0x64->NXM_NX_PKT_MARK" -c) ]) @@ -24278,7 +24810,7 @@ AT_CHECK([ grep "priority=100" | \ grep -c "ct(commit,zone=NXM_NX_REG11\\[[0..15\\]],.*exec(move:NXM_OF_ETH_SRC\\[[\\]]->NXM_NX_CT_LABEL\\[[32..79\\]],load:0x[[0-9]]->NXM_NX_CT_LABEL\\[[80..95\\]]))" - grep table=22 hv${hv}flows | \ + grep table=23 hv${hv}flows | \ grep "priority=200" | \ grep -c "actions=move:NXM_NX_CT_LABEL\\[[32..79\\]]->NXM_OF_ETH_DST\\[[\\]]" done; :], [0], [dnl @@ -24403,7 +24935,7 @@ AT_CHECK([ grep "priority=100" | \ grep -c "ct(commit,zone=NXM_NX_REG11\\[[0..15\\]],.*exec(move:NXM_OF_ETH_SRC\\[[\\]]->NXM_NX_CT_LABEL\\[[32..79\\]],load:0x[[0-9]]->NXM_NX_CT_LABEL\\[[80..95\\]]))" - grep table=22 hv${hv}flows | \ + grep table=23 hv${hv}flows | \ grep "priority=200" | \ grep -c "actions=move:NXM_NX_CT_LABEL\\[[32..79\\]]->NXM_OF_ETH_DST\\[[\\]]" done; :], [0], [dnl @@ -25025,7 +25557,7 @@ AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep "actions=controller" | grep ]) # The packet should've been dropped in the lr_in_arp_resolve stage. -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep -E "table=22, n_packets=1,.* priority=1,ip,metadata=0x${sw_key},nw_dst=10.0.1.1 actions=drop" -c], [0], [dnl +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep -E "table=23, n_packets=1,.* priority=1,ip,metadata=0x${sw_key},nw_dst=10.0.1.1 actions=drop" -c], [0], [dnl 1 ]) diff --git a/utilities/ovn-nbctl.c b/utilities/ovn-nbctl.c index 2a68ccd16..8bdcb19a3 100644 --- a/utilities/ovn-nbctl.c +++ b/utilities/ovn-nbctl.c @@ -329,6 +329,8 @@ Logical router port commands:\n\ add logical port PORT on ROUTER\n\ lrp-set-gateway-chassis PORT CHASSIS [PRIORITY]\n\ set gateway chassis for port PORT\n\ + lrp-set-options PORT KEY=VALUE [KEY=VALUE]...\n\ + set router port options\n\ lrp-del-gateway-chassis PORT CHASSIS\n\ delete gateway chassis from port PORT\n\ lrp-get-gateway-chassis PORT\n\ @@ -351,11 +353,17 @@ Logical router port commands:\n\ ('overlay' or 'bridged')\n\ \n\ Route commands:\n\ - [--policy=POLICY] [--ecmp] [--ecmp-symmetric-reply] lr-route-add ROUTER \n\ - PREFIX NEXTHOP [PORT]\n\ + [--policy=POLICY]\n\ + [--ecmp]\n\ + [--ecmp-symmetric-reply]\n\ + [--route-table=ROUTE_TABLE]\n\ + lr-route-add ROUTER PREFIX NEXTHOP [PORT]\n\ add a route to ROUTER\n\ - [--policy=POLICY] lr-route-del ROUTER [PREFIX [NEXTHOP [PORT]]]\n\ + [--policy=POLICY]\n\ + [--route-table=ROUTE_TABLE]\n\ + lr-route-del ROUTER [PREFIX [NEXTHOP [PORT]]]\n\ remove routes from ROUTER\n\ + [--route-table=ROUTE_TABLE]\n\ lr-route-list ROUTER print routes for ROUTER\n\ \n\ Policy commands:\n\ @@ -743,6 +751,11 @@ print_lr(const struct nbrec_logical_router *lr, struct ds *s) ds_put_cstr(s, "]\n"); } + const char *route_table = smap_get(&lrp->options, "route_table"); + if (route_table) { + ds_put_format(s, " route-table: %s\n", route_table); + } + if (lrp->n_gateway_chassis) { const struct nbrec_gateway_chassis **gcs; @@ -862,6 +875,7 @@ nbctl_pre_show(struct ctl_context *ctx) ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_name); ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_mac); ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_networks); + ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_options); ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_gateway_chassis); ovsdb_idl_add_column(ctx->idl, &nbrec_gateway_chassis_col_chassis_name); @@ -4020,11 +4034,19 @@ nbctl_lr_policy_list(struct ctl_context *ctx) static struct nbrec_logical_router_static_route * nbctl_lr_get_route(const struct nbrec_logical_router *lr, char *prefix, - char *next_hop, bool is_src_route, bool ecmp) + char *next_hop, bool is_src_route, bool ecmp, + char *route_table) { for (int i = 0; i < lr->n_static_routes; i++) { struct nbrec_logical_router_static_route *route = lr->static_routes[i]; + /* Strict compare for route_table. + * If route_table was not specified, + * lookup for routes with empty route_table value. */ + if (strcmp(route->route_table, route_table ? route_table : "")) { + continue; + } + /* Compare route policy. */ char *nb_policy = route->policy; bool nb_is_src_route = false; @@ -4080,6 +4102,8 @@ nbctl_pre_lr_route_add(struct ctl_context *ctx) &nbrec_logical_router_static_route_col_bfd); ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_static_route_col_options); + ovsdb_idl_add_column(ctx->idl, + &nbrec_logical_router_static_route_col_route_table); } static char * OVS_WARN_UNUSED_RESULT @@ -4110,6 +4134,7 @@ nbctl_lr_route_add(struct ctl_context *ctx) } } + char *route_table = shash_find_data(&ctx->options, "--route-table"); bool v6_prefix = false; prefix = normalize_ipv4_prefix_str(ctx->argv[2]); if (!prefix) { @@ -4186,7 +4211,8 @@ nbctl_lr_route_add(struct ctl_context *ctx) bool ecmp = shash_find(&ctx->options, "--ecmp") != NULL || ecmp_symmetric_reply; struct nbrec_logical_router_static_route *route = - nbctl_lr_get_route(lr, prefix, next_hop, is_src_route, ecmp); + nbctl_lr_get_route(lr, prefix, next_hop, is_src_route, ecmp, + route_table); /* Validations for nexthop = "discard" */ if (is_discard_route) { @@ -4250,7 +4276,8 @@ nbctl_lr_route_add(struct ctl_context *ctx) } struct nbrec_logical_router_static_route *discard_route = - nbctl_lr_get_route(lr, prefix, "discard", is_src_route, true); + nbctl_lr_get_route(lr, prefix, "discard", is_src_route, true, + route_table); if (discard_route) { ctl_error(ctx, "discard nexthop for the same ECMP route exists."); goto cleanup; @@ -4266,6 +4293,9 @@ nbctl_lr_route_add(struct ctl_context *ctx) if (policy) { nbrec_logical_router_static_route_set_policy(route, policy); } + if (route_table) { + nbrec_logical_router_static_route_set_route_table(route, route_table); + } if (ecmp_symmetric_reply) { const struct smap options = SMAP_CONST1(&options, @@ -4309,6 +4339,8 @@ nbctl_pre_lr_route_del(struct ctl_context *ctx) &nbrec_logical_router_static_route_col_nexthop); ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_static_route_col_output_port); + ovsdb_idl_add_column(ctx->idl, + &nbrec_logical_router_static_route_col_route_table); } @@ -4322,6 +4354,7 @@ nbctl_lr_route_del(struct ctl_context *ctx) return; } + const char *route_table = shash_find_data(&ctx->options, "--route-table"); const char *policy = shash_find_data(&ctx->options, "--policy"); bool is_src_route = false; if (policy) { @@ -4412,6 +4445,14 @@ nbctl_lr_route_del(struct ctl_context *ctx) } } + /* Strict compare for route_table. + * If route_table was not specified, + * lookup for routes with empty route_table value. */ + if (strcmp(lr->static_routes[i]->route_table, + route_table ? route_table : "")) { + continue; + } + /* Compare output_port, if specified. */ if (output_port) { char *rt_output_port = lr->static_routes[i]->output_port; @@ -5135,6 +5176,41 @@ nbctl_pre_lrp_del_gateway_chassis(struct ctl_context *ctx) ovsdb_idl_add_column(ctx->idl, &nbrec_gateway_chassis_col_chassis_name); } +static void +nbctl_pre_lrp_options(struct ctl_context *ctx) +{ + ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_name); + ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_options); +} + +static void +nbctl_lrp_set_options(struct ctl_context *ctx) +{ + const char *id = ctx->argv[1]; + const struct nbrec_logical_router_port *lrp = NULL; + size_t i; + struct smap options = SMAP_INITIALIZER(&options); + + char *error = lrp_by_name_or_uuid(ctx, id, true, &lrp); + if (error) { + ctx->error = error; + return; + } + for (i = 2; i < ctx->argc; i++) { + char *key, *value; + value = xstrdup(ctx->argv[i]); + key = strsep(&value, "="); + if (value) { + smap_add(&options, key, value); + } + free(key); + } + + nbrec_logical_router_port_set_options(lrp, &options); + + smap_destroy(&options); +} + /* Removes logical router port 'lrp->gateway_chassis[idx]'. */ static void remove_gc(const struct nbrec_logical_router_port *lrp, size_t idx) @@ -5911,6 +5987,7 @@ route_cmp_details(const struct nbrec_logical_router_static_route *r1, } return r1->output_port ? 1 : -1; } + struct ipv4_route { int priority; ovs_be32 addr; @@ -5920,6 +5997,11 @@ struct ipv4_route { static int __ipv4_route_cmp(const struct ipv4_route *r1, const struct ipv4_route *r2) { + int rtb_cmp = strcmp(r1->route->route_table, + r2->route->route_table); + if (rtb_cmp) { + return rtb_cmp; + } if (r1->priority != r2->priority) { return r1->priority > r2->priority ? -1 : 1; } @@ -5951,6 +6033,11 @@ struct ipv6_route { static int __ipv6_route_cmp(const struct ipv6_route *r1, const struct ipv6_route *r2) { + int rtb_cmp = strcmp(r1->route->route_table, + r2->route->route_table); + if (rtb_cmp) { + return rtb_cmp; + } if (r1->priority != r2->priority) { return r1->priority > r2->priority ? -1 : 1; } @@ -6038,6 +6125,8 @@ nbctl_pre_lr_route_list(struct ctl_context *ctx) &nbrec_logical_router_static_route_col_options); ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_static_route_col_bfd); + ovsdb_idl_add_column(ctx->idl, + &nbrec_logical_router_static_route_col_route_table); } static void @@ -6055,12 +6144,17 @@ nbctl_lr_route_list(struct ctl_context *ctx) return; } + char *route_table = shash_find_data(&ctx->options, "--route-table"); + ipv4_routes = xmalloc(sizeof *ipv4_routes * lr->n_static_routes); ipv6_routes = xmalloc(sizeof *ipv6_routes * lr->n_static_routes); for (int i = 0; i < lr->n_static_routes; i++) { const struct nbrec_logical_router_static_route *route = lr->static_routes[i]; + if (route_table && strcmp(route->route_table, route_table)) { + continue; + } unsigned int plen; ovs_be32 ipv4; const char *policy = route->policy ? route->policy : "dst-ip"; @@ -6101,6 +6195,7 @@ nbctl_lr_route_list(struct ctl_context *ctx) if (n_ipv4_routes) { ds_put_cstr(&ctx->output, "IPv4 Routes\n"); } + const struct nbrec_logical_router_static_route *route; for (int i = 0; i < n_ipv4_routes; i++) { bool ecmp = false; if (i < n_ipv4_routes - 1 && @@ -6111,6 +6206,15 @@ nbctl_lr_route_list(struct ctl_context *ctx) &ipv4_routes[i - 1])) { ecmp = true; } + + route = ipv4_routes[i].route; + if (!i || (i > 0 && strcmp(route->route_table, + ipv4_routes[i - 1].route->route_table))) { + ds_put_format(&ctx->output, "%sRoute Table %s:\n", i ? "\n" : "", + strlen(route->route_table) ? route->route_table + : "
"); + } + print_route(ipv4_routes[i].route, &ctx->output, ecmp); } @@ -6128,6 +6232,15 @@ nbctl_lr_route_list(struct ctl_context *ctx) &ipv6_routes[i - 1])) { ecmp = true; } + + route = ipv6_routes[i].route; + if (!i || (i > 0 && strcmp(route->route_table, + ipv6_routes[i - 1].route->route_table))) { + ds_put_format(&ctx->output, "%sRoute Table %s:\n", i ? "\n" : "", + strlen(route->route_table) ? route->route_table + : "
"); + } + print_route(ipv6_routes[i].route, &ctx->output, ecmp); } @@ -6949,6 +7062,8 @@ static const struct ctl_command_syntax nbctl_commands[] = { "PORT CHASSIS [PRIORITY]", nbctl_pre_lrp_set_gateway_chassis, nbctl_lrp_set_gateway_chassis, NULL, "--may-exist", RW }, + { "lrp-set-options", 1, INT_MAX, "PORT KEY=VALUE [KEY=VALUE]...", + nbctl_pre_lrp_options, nbctl_lrp_set_options, NULL, "", RW }, { "lrp-del-gateway-chassis", 2, 2, "PORT CHASSIS", nbctl_pre_lrp_del_gateway_chassis, nbctl_lrp_del_gateway_chassis, NULL, "", RW }, @@ -6972,12 +7087,13 @@ static const struct ctl_command_syntax nbctl_commands[] = { /* logical router route commands. */ { "lr-route-add", 3, 4, "ROUTER PREFIX NEXTHOP [PORT]", nbctl_pre_lr_route_add, nbctl_lr_route_add, NULL, - "--may-exist,--ecmp,--ecmp-symmetric-reply,--policy=,--bfd?", RW }, + "--may-exist,--ecmp,--ecmp-symmetric-reply,--policy=,--route-table=,--bfd?", + RW }, { "lr-route-del", 1, 4, "ROUTER [PREFIX [NEXTHOP [PORT]]]", nbctl_pre_lr_route_del, nbctl_lr_route_del, NULL, - "--if-exists,--policy=", RW }, + "--if-exists,--policy=,--route-table=", RW }, { "lr-route-list", 1, 1, "ROUTER", nbctl_pre_lr_route_list, - nbctl_lr_route_list, NULL, "", RO }, + nbctl_lr_route_list, NULL, "--route-table=", RO }, /* Policy commands */ { "lr-policy-add", 4, INT_MAX, From patchwork Sat Nov 13 09:43:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladislav Odintsov X-Patchwork-Id: 1554667 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=nj6jzqCg; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HrrBw1Fc9z9sPf for ; Sat, 13 Nov 2021 20:44:24 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 5E5944053D; Sat, 13 Nov 2021 09:44:21 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bbgP2B6mY-Hj; Sat, 13 Nov 2021 09:44:13 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 50222404EB; Sat, 13 Nov 2021 09:44:11 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 16680C0038; Sat, 13 Nov 2021 09:44:11 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id BDCABC0033 for ; Sat, 13 Nov 2021 09:44:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 9405660885 for ; Sat, 13 Nov 2021 09:44:09 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oIHL7X4Zfsge for ; Sat, 13 Nov 2021 09:44:06 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com [IPv6:2a00:1450:4864:20::133]) by smtp3.osuosl.org (Postfix) with ESMTPS id 9238E60791 for ; Sat, 13 Nov 2021 09:44:05 +0000 (UTC) Received: by mail-lf1-x133.google.com with SMTP id n12so19028369lfe.1 for ; Sat, 13 Nov 2021 01:44:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AfGeeTV+oCoaDhG9UuJUBk4sdmtHLcKD7B2qW2AmVmE=; b=nj6jzqCgg9EQxLb46GVj5EB5Ad+IikyBLRQACVtsQ6D9ccFIXj42E8XNGjdFSbTtYp Hw+RlHttDr8uvFN6uPFu7+4ZEAM9kbLOGRFPKwoiKhLRIbzpXp9D/P3oU7fruUxy+FwF aiO3PSyzvolALWdPMDfkQ5rIB3esD0IAxTN43yl2QHy0hxT5iFzuLVzY2n+UZZQ0hpWw I1VEWai6oCZWaT/Tan/Wlxiw64u7vHvqPlTn6vAlUKVsfypNgcZOqBWID05fEPUoxpQL BKw4psQ9q+qtWZ6x4rFKegxxIfcdUBjFmsNaYXFNeY56UHzaKy0L2KaS/pYnblm+hATX vaPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AfGeeTV+oCoaDhG9UuJUBk4sdmtHLcKD7B2qW2AmVmE=; b=hsNvf9NNmWcYvAVWIBOKXsrs65K2o4Z2DkQHLVydUVrgVReagD35y8hDnQaLipSBP+ NpQNlwknREQip9DqsPloz48zjaevljU8t69mKsPSDVElvHQqO9w5UKkeKxnyU4HPS+fB OyrVmBS765C954I4iR2p4RJTLN8qxdp6R6fsORXG7qqdInEs4OqKhOZcS67K9PY1cJwE GhzK96DIluCtIi6BPHTtjJCMxo8zUIv6SgfvRftskCjqdyIjF9lqK17b/X6d5hTCOPr8 K1GCmswLbQrfJng5rHm7I/hq0UjSplNqN0BogQBpXQ9HDbuVe6arpLbVqtJnc7DlPvlx GQsQ== X-Gm-Message-State: AOAM532hKtgOAfUhcEyJX8EXfPT/hinvaeOPeVJSFvB1z+ka3mG5R/DS 9IW/+oBUlE6cblm+3arS8sbZkf3vV02xuA== X-Google-Smtp-Source: ABdhPJyYefyrzZdLtrbhbrNkEgqJru4NA5ID6xQOqVry5jGU9FJG3kbegxzExynev3qePSMdBJfI2A== X-Received: by 2002:ac2:4352:: with SMTP id o18mr19807831lfl.48.1636796642465; Sat, 13 Nov 2021 01:44:02 -0800 (PST) Received: from localhost.localdomain (109-252-131-59.dynamic.spd-mgts.ru. [109.252.131.59]) by smtp.gmail.com with ESMTPSA id h1sm818725lfu.277.2021.11.13.01.44.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 13 Nov 2021 01:44:02 -0800 (PST) From: Vladislav Odintsov To: dev@openvswitch.org Date: Sat, 13 Nov 2021 12:43:52 +0300 Message-Id: <20211113094353.17690-5-odivlad@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20211113094353.17690-1-odivlad@gmail.com> References: <20211113094353.17690-1-odivlad@gmail.com> MIME-Version: 1.0 Cc: Vladislav Odintsov Subject: [ovs-dev] [PATCH ovn v8 4/5] ic: add support for routing tables in adv/learn routes X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Previously support for multiple routing tables was added to northd code. This commit expands support for multiple routing tables by adding support of advertising and learning routes with their routing table information. To utilize such feature, user must: 1. create Logical Router in each AZ; 2. create IC transit switch for each routing table, that he/she needs; 3. connect each TS with this LR; 4. assign routing table for TS's LRP (ovn-nbctl lrp-set-options route_table=<>); 5. enable routes sync (turn on learning and advertising routes in NB_Global table); 6. create LRPs for subnets in LR, create static routes with supplying route_table parameter. Note 1: routes for connected networks will be learned to
routing table and if Logical Routers have more than one Transit Switch, which interconnects them, connected routes will be added via each transit switch port and configured as ECMP routes. Note 2: static routes within route tables will be advertised and learned only if interconnecting transit switch's LRPs have same value in options:route_table as NB:Logical_Router_Static_Route or ICSB:Route route_table column value. Signed-off-by: Vladislav Odintsov Reviewed-by: Numan Siddique --- NEWS | 4 + ic/ovn-ic.c | 564 ++++++++++++++++++++++++++++---------------- ovn-ic-sb.ovsschema | 3 +- ovn-ic-sb.xml | 20 ++ tests/ovn-ic.at | 440 ++++++++++++++++++++++++++++++++++ 5 files changed, 825 insertions(+), 206 deletions(-) diff --git a/NEWS b/NEWS index 97a833a1b..e27aaad06 100644 --- a/NEWS +++ b/NEWS @@ -24,6 +24,10 @@ OVN v21.09.0 - 01 Oct 2021 - Allow static routes without nexthops. - Enabled logical dp groups as a default. CMS should disable it if not desired. + - Added support for multiple routing tables in Logical Router Static Routes + and LRPs. OVN Interconnection supports routes' route tables as well. + This requires to update schemas for OVN_Northdbound and OVN_IC_Southbound + DBs. OVN v21.06.0 - 18 Jun 2021 ------------------------- diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c index 70abae108..f40468e92 100644 --- a/ic/ovn-ic.c +++ b/ic/ovn-ic.c @@ -63,9 +63,11 @@ struct ic_context { struct ovsdb_idl_txn *ovninb_txn; struct ovsdb_idl_txn *ovnisb_txn; struct ovsdb_idl_index *nbrec_ls_by_name; + struct ovsdb_idl_index *nbrec_lrp_by_name; struct ovsdb_idl_index *nbrec_port_by_name; struct ovsdb_idl_index *sbrec_chassis_by_name; struct ovsdb_idl_index *sbrec_port_binding_by_name; + struct ovsdb_idl_index *icnbrec_transit_switch_by_name; struct ovsdb_idl_index *icsbrec_port_binding_by_az; struct ovsdb_idl_index *icsbrec_port_binding_by_ts; struct ovsdb_idl_index *icsbrec_port_binding_by_ts_az; @@ -773,7 +775,7 @@ port_binding_run(struct ic_context *ctx, icsbrec_port_binding_index_set_transit_switch(isb_pb_key, ts->name); ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key, - ctx->icsbrec_port_binding_by_ts) { + ctx->icsbrec_port_binding_by_ts) { if (isb_pb->availability_zone == az) { shash_add(&local_pbs, isb_pb->logical_port, isb_pb); shash_find_and_delete(&isb_all_local_pbs, @@ -844,7 +846,9 @@ port_binding_run(struct ic_context *ctx, struct ic_router_info { struct hmap_node node; const struct nbrec_logical_router *lr; /* key of hmap */ - const struct icsbrec_port_binding *isb_pb; + const struct icsbrec_port_binding **isb_pbs; + size_t n_isb_pbs; + size_t n_allocated_isb_pbs; struct hmap routes_learned; }; @@ -855,6 +859,7 @@ struct ic_route_info { unsigned int plen; struct in6_addr nexthop; const char *origin; + const char *route_table; /* Either nb_route or nb_lrp is set and the other one must be NULL. * - For a route that is learned from IC-SB, or a static route that is @@ -868,25 +873,29 @@ struct ic_route_info { static uint32_t ic_route_hash(const struct in6_addr *prefix, unsigned int plen, - const struct in6_addr *nexthop, const char *origin) + const struct in6_addr *nexthop, const char *origin, + const char *route_table) { uint32_t basis = hash_bytes(prefix, sizeof *prefix, (uint32_t)plen); basis = hash_string(origin, basis); + basis = hash_string(route_table, basis); return hash_bytes(nexthop, sizeof *nexthop, basis); } static struct ic_route_info * ic_route_find(struct hmap *routes, const struct in6_addr *prefix, unsigned int plen, const struct in6_addr *nexthop, - const char *origin) + const char *origin, char *route_table) { struct ic_route_info *r; - uint32_t hash = ic_route_hash(prefix, plen, nexthop, origin); + uint32_t hash = ic_route_hash(prefix, plen, nexthop, origin, route_table); HMAP_FOR_EACH_WITH_HASH (r, node, hash, routes) { if (ipv6_addr_equals(&r->prefix, prefix) && r->plen == plen && ipv6_addr_equals(&r->nexthop, nexthop) && - !strcmp(r->origin, origin)) { + !strcmp(r->origin, origin) && + !strcmp(r->route_table ? r->route_table : "", route_table) && + ipv6_addr_equals(&r->nexthop, nexthop)) { return r; } } @@ -931,14 +940,22 @@ add_to_routes_learned(struct hmap *routes_learned, return false; } const char *origin = smap_get_def(&nb_route->options, "origin", ""); + if (ic_route_find(routes_learned, &prefix, plen, &nexthop, origin, + nb_route->route_table)) { + /* Route is already added to learned in previous iteration. */ + return true; + } + struct ic_route_info *ic_route = xzalloc(sizeof *ic_route); ic_route->prefix = prefix; ic_route->plen = plen; ic_route->nexthop = nexthop; ic_route->nb_route = nb_route; ic_route->origin = origin; + ic_route->route_table = nb_route->route_table; hmap_insert(routes_learned, &ic_route->node, - ic_route_hash(&prefix, plen, &nexthop, origin)); + ic_route_hash(&prefix, plen, &nexthop, origin, + nb_route->route_table)); return true; } @@ -1075,8 +1092,17 @@ static void add_to_routes_ad(struct hmap *routes_ad, const struct nbrec_logical_router_static_route *nb_route, const struct lport_addresses *nexthop_addresses, - const struct smap *nb_options) + const struct smap *nb_options, const char *route_table) { + if (strcmp(route_table, nb_route->route_table)) { + if (VLOG_IS_DBG_ENABLED()) { + VLOG_DBG("Skip advertising route %s -> %s as its route table %s !=" + " %s of TS port", nb_route->ip_prefix, nb_route->nexthop, + nb_route->route_table, route_table); + } + return; + } + struct in6_addr prefix, nexthop; unsigned int plen; if (!parse_route(nb_route->ip_prefix, nb_route->nexthop, @@ -1094,14 +1120,37 @@ add_to_routes_ad(struct hmap *routes_ad, return; } + if (VLOG_IS_DBG_ENABLED()) { + struct ds msg = DS_EMPTY_INITIALIZER; + + ds_put_format(&msg, "Advertising static route: %s -> %s, ic nexthop: ", + nb_route->ip_prefix, nb_route->nexthop); + + if (IN6_IS_ADDR_V4MAPPED(&nexthop)) { + ds_put_format(&msg, IP_FMT, + IP_ARGS(in6_addr_get_mapped_ipv4(&nexthop))); + } else { + ipv6_format_addr(&nexthop, &msg); + } + + ds_put_format(&msg, ", route_table: %s", strlen(nb_route->route_table) + ? nb_route->route_table + : "
"); + + VLOG_DBG("%s", ds_cstr(&msg)); + ds_destroy(&msg); + } + struct ic_route_info *ic_route = xzalloc(sizeof *ic_route); ic_route->prefix = prefix; ic_route->plen = plen; ic_route->nexthop = nexthop; ic_route->nb_route = nb_route; ic_route->origin = ROUTE_ORIGIN_STATIC; + ic_route->route_table = nb_route->route_table; hmap_insert(routes_ad, &ic_route->node, - ic_route_hash(&prefix, plen, &nexthop, ROUTE_ORIGIN_STATIC)); + ic_route_hash(&prefix, plen, &nexthop, ROUTE_ORIGIN_STATIC, + nb_route->route_table)); } static void @@ -1131,8 +1180,8 @@ add_network_to_routes_ad(struct hmap *routes_ad, const char *network, if (VLOG_IS_DBG_ENABLED()) { struct ds msg = DS_EMPTY_INITIALIZER; - ds_put_format(&msg, "Route ad: direct network %s of lrp %s, nexthop ", - network, nb_lrp->name); + ds_put_format(&msg, "Adding direct network route to
routing " + "table: %s of lrp %s, nexthop ", network, nb_lrp->name); if (IN6_IS_ADDR_V4MAPPED(&nexthop)) { ds_put_format(&msg, IP_FMT, @@ -1151,14 +1200,16 @@ add_network_to_routes_ad(struct hmap *routes_ad, const char *network, ic_route->nexthop = nexthop; ic_route->nb_lrp = nb_lrp; ic_route->origin = ROUTE_ORIGIN_CONNECTED; + + /* directly-connected routes go to
route table */ + ic_route->route_table = NULL; hmap_insert(routes_ad, &ic_route->node, ic_route_hash(&prefix, plen, &nexthop, - ROUTE_ORIGIN_CONNECTED)); + ROUTE_ORIGIN_CONNECTED, "")); } static bool -route_need_learn(struct in6_addr *prefix, - unsigned int plen, +route_need_learn(struct in6_addr *prefix, unsigned int plen, const struct smap *nb_options) { if (!smap_get_bool(nb_options, "ic-route-learn", false)) { @@ -1181,76 +1232,152 @@ route_need_learn(struct in6_addr *prefix, return true; } +static const char * +get_lrp_name_by_ts_port_name(struct ic_context *ctx, const char *ts_port_name) +{ + const struct nbrec_logical_switch_port *nb_lsp; + const struct nbrec_logical_switch_port *nb_lsp_key = + nbrec_logical_switch_port_index_init_row(ctx->nbrec_port_by_name); + nbrec_logical_switch_port_index_set_name(nb_lsp_key, ts_port_name); + nb_lsp = nbrec_logical_switch_port_index_find(ctx->nbrec_port_by_name, + nb_lsp_key); + nbrec_logical_switch_port_index_destroy_row(nb_lsp_key); + + if (!nb_lsp) { + return NULL; + } + + return smap_get(&nb_lsp->options, "router-port"); +} + +static const char * +get_route_table_by_lrp_name(struct ic_context *ctx, const char *lrp_name) +{ + const struct nbrec_logical_router_port *lrp; + const struct nbrec_logical_router_port *lrp_key = + nbrec_logical_router_port_index_init_row(ctx->nbrec_lrp_by_name); + nbrec_logical_router_port_index_set_name(lrp_key, lrp_name); + lrp = nbrec_logical_router_port_index_find(ctx->nbrec_lrp_by_name, + lrp_key); + nbrec_logical_router_port_index_destroy_row(lrp_key); + + if (lrp) { + return smap_get_def(&lrp->options, "route_table", ""); + } + return ""; /*
route table */ +} + +static bool +lrp_is_ts_port(struct ic_context *ctx, struct ic_router_info *ic_lr, + const char *lrp_name) +{ + const struct icsbrec_port_binding *isb_pb; + const char *ts_lrp_name; + for (int i = 0; i < ic_lr->n_isb_pbs; i++) { + isb_pb = ic_lr->isb_pbs[i]; + ts_lrp_name = get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port); + if (!strcmp(ts_lrp_name, lrp_name)) { + return true; + } + } + return false; +} + static void -sync_learned_route(struct ic_context *ctx, - const struct icsbrec_availability_zone *az, - struct ic_router_info *ic_lr) +sync_learned_routes(struct ic_context *ctx, + const struct icsbrec_availability_zone *az, + struct ic_router_info *ic_lr) { ovs_assert(ctx->ovnnb_txn); - const struct icsbrec_route *isb_route; - const struct icsbrec_route *isb_route_key = - icsbrec_route_index_init_row(ctx->icsbrec_route_by_ts); + const struct icsbrec_route *isb_route, *isb_route_key; - icsbrec_route_index_set_transit_switch(isb_route_key, - ic_lr->isb_pb->transit_switch); + const struct nbrec_nb_global *nb_global = + nbrec_nb_global_first(ctx->ovnnb_idl); + ovs_assert(nb_global); - ICSBREC_ROUTE_FOR_EACH_EQUAL (isb_route, isb_route_key, - ctx->icsbrec_route_by_ts) { - if (isb_route->availability_zone == az) { - continue; - } - struct in6_addr prefix, nexthop; - unsigned int plen; - if (!parse_route(isb_route->ip_prefix, isb_route->nexthop, - &prefix, &plen, &nexthop)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "Bad route format in IC-SB: %s -> %s. Ignored.", - isb_route->ip_prefix, isb_route->nexthop); - continue; - } - const struct nbrec_nb_global *nb_global = - nbrec_nb_global_first(ctx->ovnnb_idl); - ovs_assert(nb_global); - if (!route_need_learn(&prefix, plen, &nb_global->options)) { - continue; - } - struct ic_route_info *route_learned - = ic_route_find(&ic_lr->routes_learned, &prefix, plen, &nexthop, - isb_route->origin); - if (route_learned) { - /* Sync external-ids */ - struct uuid ext_id; - smap_get_uuid(&route_learned->nb_route->external_ids, - "ic-learned-route", &ext_id); - if (!uuid_equals(&ext_id, &isb_route->header_.uuid)) { + const char *lrp_name, *ts_route_table; + const struct icsbrec_port_binding *isb_pb; + for (int i = 0; i < ic_lr->n_isb_pbs; i++) { + isb_pb = ic_lr->isb_pbs[i]; + lrp_name = get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port); + ts_route_table = get_route_table_by_lrp_name(ctx, lrp_name); + + isb_route_key = icsbrec_route_index_init_row(ctx->icsbrec_route_by_ts); + icsbrec_route_index_set_transit_switch(isb_route_key, + isb_pb->transit_switch); + + ICSBREC_ROUTE_FOR_EACH_EQUAL (isb_route, isb_route_key, + ctx->icsbrec_route_by_ts) { + if (isb_route->availability_zone == az) { + continue; + } + + if (strlen(isb_route->route_table) && + strcmp(isb_route->route_table, ts_route_table)) { + if (VLOG_IS_DBG_ENABLED()) { + VLOG_DBG("Skip learning static route %s -> %s as either " + "its route table %s != %s of TS port or ", + isb_route->ip_prefix, isb_route->nexthop, + isb_route->route_table, ts_route_table); + } + continue; + } + + struct in6_addr prefix, nexthop; + unsigned int plen; + if (!parse_route(isb_route->ip_prefix, isb_route->nexthop, + &prefix, &plen, &nexthop)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "Bad route format in IC-SB: %s -> %s. " + "Ignored.", isb_route->ip_prefix, + isb_route->nexthop); + continue; + } + if (!route_need_learn(&prefix, plen, &nb_global->options)) { + continue; + } + struct ic_route_info *route_learned + = ic_route_find(&ic_lr->routes_learned, &prefix, plen, + &nexthop, isb_route->origin, + isb_route->route_table); + if (route_learned) { + /* Sync external-ids */ + struct uuid ext_id; + smap_get_uuid(&route_learned->nb_route->external_ids, + "ic-learned-route", &ext_id); + if (!uuid_equals(&ext_id, &isb_route->header_.uuid)) { + char *uuid_s = + xasprintf(UUID_FMT, + UUID_ARGS(&isb_route->header_.uuid)); + nbrec_logical_router_static_route_update_external_ids_setkey( + route_learned->nb_route, "ic-learned-route", uuid_s); + free(uuid_s); + } + hmap_remove(&ic_lr->routes_learned, &route_learned->node); + free(route_learned); + } else { + /* Create the missing route in NB. */ + const struct nbrec_logical_router_static_route *nb_route = + nbrec_logical_router_static_route_insert(ctx->ovnnb_txn); + nbrec_logical_router_static_route_set_ip_prefix(nb_route, + isb_route->ip_prefix); + nbrec_logical_router_static_route_set_nexthop(nb_route, + isb_route->nexthop); char *uuid_s = xasprintf(UUID_FMT, UUID_ARGS(&isb_route->header_.uuid)); + nbrec_logical_router_static_route_set_route_table(nb_route, + isb_route->route_table); nbrec_logical_router_static_route_update_external_ids_setkey( - route_learned->nb_route, "ic-learned-route", uuid_s); + nb_route, "ic-learned-route", uuid_s); + nbrec_logical_router_static_route_update_options_setkey( + nb_route, "origin", isb_route->origin); free(uuid_s); + nbrec_logical_router_update_static_routes_addvalue(ic_lr->lr, + nb_route); } - hmap_remove(&ic_lr->routes_learned, &route_learned->node); - free(route_learned); - } else { - /* Create the missing route in NB. */ - const struct nbrec_logical_router_static_route *nb_route = - nbrec_logical_router_static_route_insert(ctx->ovnnb_txn); - nbrec_logical_router_static_route_set_ip_prefix( - nb_route, isb_route->ip_prefix); - nbrec_logical_router_static_route_set_nexthop( - nb_route, isb_route->nexthop); - char *uuid_s = xasprintf(UUID_FMT, - UUID_ARGS(&isb_route->header_.uuid)); - nbrec_logical_router_static_route_update_external_ids_setkey( - nb_route, "ic-learned-route", uuid_s); - nbrec_logical_router_static_route_update_options_setkey( - nb_route, "origin", isb_route->origin); - free(uuid_s); - nbrec_logical_router_update_static_routes_addvalue( - ic_lr->lr, nb_route); } + icsbrec_route_index_destroy_row(isb_route_key); } - icsbrec_route_index_destroy_row(isb_route_key); /* Delete extra learned routes. */ struct ic_route_info *route_learned, *next; @@ -1283,10 +1410,10 @@ ad_route_sync_external_ids(const struct ic_route_info *route_adv, /* Sync routes from routes_ad to IC-SB. */ static void -advertise_route(struct ic_context *ctx, - const struct icsbrec_availability_zone *az, - const char *ts_name, - struct hmap *routes_ad) +advertise_routes(struct ic_context *ctx, + const struct icsbrec_availability_zone *az, + const char *ts_name, + struct hmap *routes_ad) { ovs_assert(ctx->ovnisb_txn); const struct icsbrec_route *isb_route; @@ -1309,9 +1436,9 @@ advertise_route(struct ic_context *ctx, icsbrec_route_delete(isb_route); continue; } - struct ic_route_info *route_adv = ic_route_find(routes_ad, &prefix, - plen, &nexthop, - isb_route->origin); + struct ic_route_info *route_adv = + ic_route_find(routes_ad, &prefix, plen, &nexthop, + isb_route->origin, isb_route->route_table); if (!route_adv) { /* Delete the extra route from IC-SB. */ VLOG_DBG("Delete route %s -> %s from IC-SB, which is not found" @@ -1352,6 +1479,9 @@ advertise_route(struct ic_context *ctx, icsbrec_route_set_ip_prefix(isb_route, prefix_s); icsbrec_route_set_nexthop(isb_route, nexthop_s); icsbrec_route_set_origin(isb_route, route_adv->origin); + icsbrec_route_set_route_table(isb_route, route_adv->route_table + ? route_adv->route_table + : ""); free(prefix_s); free(nexthop_s); @@ -1362,23 +1492,97 @@ advertise_route(struct ic_context *ctx, } } -static const char * -get_lrp_name_by_ts_port_name(struct ic_context *ctx, - const char *ts_port_name) +static void +build_ts_routes_to_adv(struct ic_context *ctx, + struct ic_router_info *ic_lr, + struct hmap *routes_ad, + struct lport_addresses *ts_port_addrs, + const struct nbrec_nb_global *nb_global, + const char *ts_route_table) { - const struct nbrec_logical_switch_port *nb_lsp; - const struct nbrec_logical_switch_port *nb_lsp_key = - nbrec_logical_switch_port_index_init_row(ctx->nbrec_port_by_name); - nbrec_logical_switch_port_index_set_name(nb_lsp_key, ts_port_name); - nb_lsp = nbrec_logical_switch_port_index_find(ctx->nbrec_port_by_name, - nb_lsp_key); - nbrec_logical_switch_port_index_destroy_row(nb_lsp_key); + const struct nbrec_logical_router *lr = ic_lr->lr; + + /* Check static routes of the LR */ + for (int i = 0; i < lr->n_static_routes; i++) { + const struct nbrec_logical_router_static_route *nb_route + = lr->static_routes[i]; + struct uuid isb_uuid; + if (smap_get_uuid(&nb_route->external_ids, "ic-learned-route", + &isb_uuid)) { + /* It is a learned route */ + if (!add_to_routes_learned(&ic_lr->routes_learned, nb_route)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "Bad format of learned route in NB: " + "%s -> %s. Delete it.", nb_route->ip_prefix, + nb_route->nexthop); + nbrec_logical_router_update_static_routes_delvalue(lr, + nb_route); + } + } else { + /* It may be a route to be advertised */ + add_to_routes_ad(routes_ad, nb_route, ts_port_addrs, + &nb_global->options, ts_route_table); + } + } - if (!nb_lsp) { - return NULL; + /* Check directly-connected subnets of the LR */ + for (int i = 0; i < lr->n_ports; i++) { + const struct nbrec_logical_router_port *lrp = lr->ports[i]; + if (!lrp_is_ts_port(ctx, ic_lr, lrp->name)) { + for (int j = 0; j < lrp->n_networks; j++) { + add_network_to_routes_ad(routes_ad, lrp->networks[j], lrp, + ts_port_addrs, + &nb_global->options); + } + } else { + /* The router port of the TS port is ignored. */ + VLOG_DBG("Skip advertising direct route of lrp %s (TS port)", + lrp->name); + } } +} - return smap_get(&nb_lsp->options, "router-port"); +static void +advertise_lr_routes(struct ic_context *ctx, + const struct icsbrec_availability_zone *az, + struct ic_router_info *ic_lr) +{ + const struct nbrec_nb_global *nb_global = + nbrec_nb_global_first(ctx->ovnnb_idl); + ovs_assert(nb_global); + + const struct icsbrec_port_binding *isb_pb; + const char *lrp_name, *ts_name, *route_table; + struct lport_addresses ts_port_addrs; + const struct nbrec_logical_router *lr = ic_lr->lr; + const struct icnbrec_transit_switch *key; + + struct hmap routes_ad = HMAP_INITIALIZER(&routes_ad); + for (int i = 0; i < ic_lr->n_isb_pbs; i++) { + isb_pb = ic_lr->isb_pbs[i]; + key = icnbrec_transit_switch_index_init_row( + ctx->icnbrec_transit_switch_by_name); + icnbrec_transit_switch_index_set_name(key, isb_pb->transit_switch); + ts_name = icnbrec_transit_switch_index_find( + ctx->icnbrec_transit_switch_by_name, key)->name; + icnbrec_transit_switch_index_destroy_row(key); + + if (!extract_lsp_addresses(isb_pb->address, &ts_port_addrs)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_INFO_RL(&rl, "Route sync ignores port %s on ts %s for router" + " %s because the addresses are invalid.", + isb_pb->logical_port, isb_pb->transit_switch, + lr->name); + continue; + } + lrp_name = get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port); + route_table = get_route_table_by_lrp_name(ctx, lrp_name); + build_ts_routes_to_adv(ctx, ic_lr, &routes_ad, &ts_port_addrs, + nb_global, route_table); + advertise_routes(ctx, az, ts_name, &routes_ad); + destroy_lport_addresses(&ts_port_addrs); + } + hmap_destroy(&routes_ad); } static void @@ -1389,130 +1593,70 @@ route_run(struct ic_context *ctx, return; } - const struct nbrec_nb_global *nb_global = - nbrec_nb_global_first(ctx->ovnnb_idl); - ovs_assert(nb_global); - - const struct icnbrec_transit_switch *ts; - ICNBREC_TRANSIT_SWITCH_FOR_EACH (ts, ctx->ovninb_idl) { - struct hmap ic_lrs = HMAP_INITIALIZER(&ic_lrs); - struct hmap routes_ad = HMAP_INITIALIZER(&routes_ad); - - const struct icsbrec_port_binding *isb_pb; - const struct icsbrec_port_binding *isb_pb_key = - icsbrec_port_binding_index_init_row( - ctx->icsbrec_port_binding_by_ts_az); - icsbrec_port_binding_index_set_transit_switch(isb_pb_key, ts->name); - icsbrec_port_binding_index_set_availability_zone(isb_pb_key, az); - - /* Each port on TS maps to a logical router, which is stored in the - * external_ids:router-id of the IC SB port_binding record. */ - ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key, - ctx->icsbrec_port_binding_by_ts_az) { - const char *ts_lrp_name = - get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port); - if (!ts_lrp_name) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "Route sync ignores port %s on ts %s " - "because logical router port is not found in NB.", - isb_pb->logical_port, ts->name); - continue; - } + struct hmap ic_lrs = HMAP_INITIALIZER(&ic_lrs); + const struct icsbrec_port_binding *isb_pb; + const struct icsbrec_port_binding *isb_pb_key = + icsbrec_port_binding_index_init_row(ctx->icsbrec_port_binding_by_az); + icsbrec_port_binding_index_set_availability_zone(isb_pb_key, az); - struct uuid lr_uuid; - if (!smap_get_uuid(&isb_pb->external_ids, "router-id", &lr_uuid)) { - VLOG_DBG("IC-SB Port_Binding %s doesn't have " - "external_ids:router-id set.", isb_pb->logical_port); - continue; - } - const struct nbrec_logical_router *lr - = nbrec_logical_router_get_for_uuid(ctx->ovnnb_idl, &lr_uuid); - if (!lr) { - continue; - } + /* Each port on TS maps to a logical router, which is stored in the + * external_ids:router-id of the IC SB port_binding record. + * Here we build info for interconnected Logical Router: + * collect IC Port Binding to process routes sync later on. */ + ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key, + ctx->icsbrec_port_binding_by_az) + { + const char *ts_lrp_name = + get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port); + if (!ts_lrp_name) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "Route sync ignores port %s on ts %s because " + "logical router port is not found in NB. Deleting it", + isb_pb->logical_port, isb_pb->transit_switch); + icsbrec_port_binding_delete(isb_pb); + continue; + } - if (ic_router_find(&ic_lrs, lr)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_INFO_RL(&rl, "Route sync ignores port %s on ts %s for " - "router %s because the router has another port " - "connected to same ts.", isb_pb->logical_port, - ts->name, lr->name); - continue; - } + struct uuid lr_uuid; + if (!smap_get_uuid(&isb_pb->external_ids, "router-id", &lr_uuid)) { + VLOG_DBG("IC-SB Port_Binding %s doesn't have " + "external_ids:router-id set.", isb_pb->logical_port); + continue; + } - struct lport_addresses ts_port_addrs; - if (!extract_lsp_addresses(isb_pb->address, &ts_port_addrs)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_INFO_RL(&rl, "Route sync ignores port %s on ts %s for " - "router %s because the addresses are invalid.", - isb_pb->logical_port, ts->name, lr->name); - continue; - } + const struct nbrec_logical_router *lr + = nbrec_logical_router_get_for_uuid(ctx->ovnnb_idl, &lr_uuid); + if (!lr) { + continue; + } - struct ic_router_info *ic_lr = xzalloc(sizeof *ic_lr); + struct ic_router_info *ic_lr = ic_router_find(&ic_lrs, lr); + if (!ic_lr) { + ic_lr = xzalloc(sizeof *ic_lr); ic_lr->lr = lr; - ic_lr->isb_pb = isb_pb; hmap_init(&ic_lr->routes_learned); hmap_insert(&ic_lrs, &ic_lr->node, uuid_hash(&lr->header_.uuid)); - - /* Check static routes of the LR */ - for (int i = 0; i < lr->n_static_routes; i++) { - const struct nbrec_logical_router_static_route *nb_route - = lr->static_routes[i]; - struct uuid isb_uuid; - if (smap_get_uuid(&nb_route->external_ids, - "ic-learned-route", &isb_uuid)) { - /* It is a learned route */ - if (!add_to_routes_learned(&ic_lr->routes_learned, - nb_route)) { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "Bad format of learned route in NB:" - " %s -> %s. Delete it.", - nb_route->ip_prefix, nb_route->nexthop); - nbrec_logical_router_update_static_routes_delvalue( - lr, nb_route); - } - } else { - /* It may be a route to be advertised */ - add_to_routes_ad(&routes_ad, nb_route, &ts_port_addrs, - &nb_global->options); - } - } - - /* Check direct-connected subnets of the LR */ - for (int i = 0; i < lr->n_ports; i++) { - const struct nbrec_logical_router_port *lrp = lr->ports[i]; - if (!strcmp(lrp->name, ts_lrp_name)) { - /* The router port of the TS port is ignored. */ - VLOG_DBG("Route ad: skip lrp %s (TS port: %s)", - lrp->name, isb_pb->logical_port); - continue; - } - - for (int j = 0; j < lrp->n_networks; j++) { - add_network_to_routes_ad(&routes_ad, lrp->networks[j], - lrp, &ts_port_addrs, - &nb_global->options); - } - } - - destroy_lport_addresses(&ts_port_addrs); } - icsbrec_port_binding_index_destroy_row(isb_pb_key); - - advertise_route(ctx, az, ts->name, &routes_ad); - hmap_destroy(&routes_ad); - struct ic_router_info *ic_lr, *next; - HMAP_FOR_EACH_SAFE (ic_lr, next, node, &ic_lrs) { - sync_learned_route(ctx, az, ic_lr); - hmap_destroy(&ic_lr->routes_learned); - hmap_remove(&ic_lrs, &ic_lr->node); - free(ic_lr); + if (ic_lr->n_isb_pbs == ic_lr->n_allocated_isb_pbs) { + ic_lr->isb_pbs = x2nrealloc(ic_lr->isb_pbs, + &ic_lr->n_allocated_isb_pbs, + sizeof *ic_lr->isb_pbs); } - hmap_destroy(&ic_lrs); + ic_lr->isb_pbs[ic_lr->n_isb_pbs++] = isb_pb; } + icsbrec_port_binding_index_destroy_row(isb_pb_key); + + struct ic_router_info *ic_lr, *next; + HMAP_FOR_EACH_SAFE (ic_lr, next, node, &ic_lrs) { + advertise_lr_routes(ctx, az, ic_lr); + sync_learned_routes(ctx, az, ic_lr); + free(ic_lr->isb_pbs); + hmap_destroy(&ic_lr->routes_learned); + hmap_remove(&ic_lrs, &ic_lr->node); + free(ic_lr); + } + hmap_destroy(&ic_lrs); } static void @@ -1710,6 +1854,9 @@ main(int argc, char *argv[]) struct ovsdb_idl_index *nbrec_port_by_name = ovsdb_idl_index_create1(ovnnb_idl_loop.idl, &nbrec_logical_switch_port_col_name); + struct ovsdb_idl_index *nbrec_lrp_by_name + = ovsdb_idl_index_create1(ovnnb_idl_loop.idl, + &nbrec_logical_router_port_col_name); struct ovsdb_idl_index *sbrec_port_binding_by_name = ovsdb_idl_index_create1(ovnsb_idl_loop.idl, &sbrec_port_binding_col_logical_port); @@ -1717,6 +1864,10 @@ main(int argc, char *argv[]) = ovsdb_idl_index_create1(ovnsb_idl_loop.idl, &sbrec_chassis_col_name); + struct ovsdb_idl_index *icnbrec_transit_switch_by_name + = ovsdb_idl_index_create1(ovninb_idl_loop.idl, + &icnbrec_transit_switch_col_name); + struct ovsdb_idl_index *icsbrec_port_binding_by_az = ovsdb_idl_index_create1(ovnisb_idl_loop.idl, &icsbrec_port_binding_col_availability_zone); @@ -1775,9 +1926,12 @@ main(int argc, char *argv[]) .ovnisb_idl = ovnisb_idl_loop.idl, .ovnisb_txn = ovsdb_idl_loop_run(&ovnisb_idl_loop), .nbrec_ls_by_name = nbrec_ls_by_name, + .nbrec_lrp_by_name = nbrec_lrp_by_name, .nbrec_port_by_name = nbrec_port_by_name, .sbrec_port_binding_by_name = sbrec_port_binding_by_name, .sbrec_chassis_by_name = sbrec_chassis_by_name, + .icnbrec_transit_switch_by_name = + icnbrec_transit_switch_by_name, .icsbrec_port_binding_by_az = icsbrec_port_binding_by_az, .icsbrec_port_binding_by_ts = icsbrec_port_binding_by_ts, .icsbrec_port_binding_by_ts_az = icsbrec_port_binding_by_ts_az, diff --git a/ovn-ic-sb.ovsschema b/ovn-ic-sb.ovsschema index 42ce85d7d..72c9d3f3e 100644 --- a/ovn-ic-sb.ovsschema +++ b/ovn-ic-sb.ovsschema @@ -1,7 +1,7 @@ { "name": "OVN_IC_Southbound", "version": "1.1.0", - "cksum": "423535838 6733", + "cksum": "2309827842 6784", "tables": { "IC_SB_Global": { "columns": { @@ -92,6 +92,7 @@ "transit_switch": {"type": "string"}, "availability_zone": {"type": {"key": {"type": "uuid", "refTable": "Availability_Zone"}}}, + "route_table": {"type": "string"}, "ip_prefix": {"type": "string"}, "nexthop": {"type": "string"}, "origin": {"type": {"key": { diff --git a/ovn-ic-sb.xml b/ovn-ic-sb.xml index d8338e4d3..2966975c7 100644 --- a/ovn-ic-sb.xml +++ b/ovn-ic-sb.xml @@ -306,6 +306,26 @@ The availability zone that has advertised the route. + + Route table within which this route was created. Empty value means + <main> routing table. +

+ Routes for directly-connected networks will be + learned to <main> routing table and if Logical Routers + have more than one Transit Switch, which interconnects them, + directly-connected routes will be added via each transit switch port + and configured as ECMP routes. +

+

+ Static routes within route tables will be advertised and learned only + if interconnecting transit switch's LRPs will have + same value in as NB or ICSB value respectively. +

+ + IP prefix of this route (e.g. 192.168.100.0/24). diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at index bb4b98d0b..1340874d5 100644 --- a/tests/ovn-ic.at +++ b/tests/ovn-ic.at @@ -488,3 +488,443 @@ OVN_CLEANUP_IC([az1], [az2]) AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([ovn-ic -- route sync -- route tables]) + +ovn_init_ic_db +ovn-ic-nbctl ts-add ts1 + +for i in 1 2; do + ovn_start az$i + ovn_as az$i + + # Enable route learning at AZ level + ovn-nbctl set nb_global . options:ic-route-learn=true + # Enable route advertising at AZ level + ovn-nbctl set nb_global . options:ic-route-adv=true + + # Create LRP and connect to TS + ovn-nbctl lr-add lr$i + ovn-nbctl lrp-add lr$i lrp-lr$i-ts1 aa:aa:aa:aa:aa:0$i 169.254.100.$i/24 + ovn-nbctl lsp-add ts1 lsp-ts1-lr$i \ + -- lsp-set-addresses lsp-ts1-lr$i router \ + -- lsp-set-type lsp-ts1-lr$i router \ + -- lsp-set-options lsp-ts1-lr$i router-port=lrp-lr$i-ts1 + + # Create static routes + ovn-nbctl lr-route-add lr$i 10.11.$i.0/24 169.254.0.1 + + # Create a src-ip route, which shouldn't be synced + ovn-nbctl --policy=src-ip --route-table=rtb1 lr-route-add lr$i 10.22.$i.0/24 169.254.0.2 +done + +for i in 1 2; do + OVS_WAIT_UNTIL([ovn_as az$i ovn-nbctl lr-route-list lr$i | grep learned]) +done + +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl +IPv4 Routes +Route Table
: + 10.11.1.0/24 169.254.0.1 dst-ip + 10.11.2.0/24 169.254.100.2 dst-ip (learned) + +Route Table rtb1: + 10.22.1.0/24 169.254.0.2 src-ip +]) + +# move routes from
route table to rtb1 +for i in 1 2; do + ovn_as az$i ovn-nbctl lr-route-del lr$i 10.11.$i.0/24 169.254.0.1 + ovn_as az$i ovn-nbctl --route-table=rtb1 lr-route-add lr$i 10.11.$i.0/24 169.254.0.1 +done + +for i in 1 2; do + OVS_WAIT_WHILE([ovn_as az$i ovn-nbctl lr-route-list lr$i | grep learned]) +done + +# ensure route from rtb1 is not learned to any route table as route table is +# not set to TS port +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl +IPv4 Routes +Route Table rtb1: + 10.11.1.0/24 169.254.0.1 dst-ip + 10.22.1.0/24 169.254.0.2 src-ip +]) + +# assign route table rtb1 to TS port on AZ2 and check routes are advertised to IC SB DB +check ovn_as az2 ovn-nbctl lrp-set-options lrp-lr2-ts1 route_table=rtb1 +OVS_WAIT_UNTIL([ovn-ic-sbctl find route route_table=rtb1 | grep 10.11.2.0/24]) + +# ensure route was not learned as on AZ1 TS port's LRP was not set to route table rtb1 +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl +IPv4 Routes +Route Table rtb1: + 10.11.1.0/24 169.254.0.1 dst-ip + 10.22.1.0/24 169.254.0.2 src-ip +]) + +# set TS port's LRP to route table rtb1 to learn routes from AZ2 from rtb1 +check ovn_as az1 ovn-nbctl lrp-set-options lrp-lr1-ts1 route_table=rtb1 + +OVS_WAIT_UNTIL([ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-list lr1 | grep learned]) +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl +IPv4 Routes +Route Table rtb1: + 10.11.1.0/24 169.254.0.1 dst-ip + 10.11.2.0/24 169.254.100.2 dst-ip (learned) + 10.22.1.0/24 169.254.0.2 src-ip +]) + +# Delete route in AZ1, AZ2's learned route should be deleted. +ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-del lr1 10.11.1.0/24 +OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-list lr2 | grep learned]) + +# Add the route back +ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-add lr1 10.11.1.0/24 169.254.0.1 +OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned]) + +# Disable route-learning for AZ1 +ovn_as az1 ovn-nbctl set nb_global . options:ic-route-learn=false +OVS_WAIT_WHILE([ovn_as az1 ovn-nbctl lr-route-list lr1 | grep learned]) +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl +IPv4 Routes +Route Table rtb1: + 10.11.1.0/24 169.254.0.1 dst-ip + 10.22.1.0/24 169.254.0.2 src-ip +]) + +# AZ1 should still advertise and AZ2 should still learn the route +AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned], [0], [ignore]) +AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl +IPv4 Routes +Route Table rtb1: + 10.11.1.0/24 169.254.100.1 dst-ip (learned) + 10.11.2.0/24 169.254.0.1 dst-ip + 10.22.2.0/24 169.254.0.2 src-ip +]) + +# Disable route-advertising for AZ1 +ovn_as az1 ovn-nbctl set nb_global . options:ic-route-adv=false + +# AZ2 shouldn't have the route learned, because AZ1 have stopped advertising. +OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned]) + +# Add default route in AZ1 +ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-add lr1 0.0.0.0/0 169.254.0.3 + +# Re-enable router-advertising & learn for AZ1 +ovn_as az1 ovn-nbctl set nb_global . options:ic-route-adv=true +ovn_as az1 ovn-nbctl set nb_global . options:ic-route-learn=true + +for i in 1 2; do + OVS_WAIT_UNTIL([ovn_as az$i ovn-nbctl lr-route-list lr$i | grep learned]) +done + +# Default route should NOT get advertised or learned, by default. +AT_CHECK([ovn-ic-sbctl find route ip_prefix="0.0.0.0/0"], [0], []) + +# Enable default route advertising in AZ1, ensure it advertised, but not learned +ovn_as az1 ovn-nbctl set nb_global . options:ic-route-adv-default=true +OVS_WAIT_UNTIL([ovn-ic-sbctl find route ip_prefix="0.0.0.0/0" route_table=rtb1 | grep 0.0.0.0]) +OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-list lr2 | grep learned | grep 0.0.0.0]) + +# Enable default route learning in AZ2 +ovn_as az2 ovn-nbctl set nb_global . options:ic-route-learn-default=true +OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-list lr2 | grep learned | grep 0.0.0.0]) + +# Test directly connected subnet route advertising. Route should go to
route table. +ovn_as az1 ovn-nbctl lrp-add lr1 lrp-lr1-ls1 aa:aa:aa:aa:bb:01 "192.168.0.1/24" +OVS_WAIT_UNTIL([ovn-ic-sbctl find route ip_prefix="192.168.0.1/24" route_table="\"\"" | grep 192.168.0.1/24]) +OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned | grep 192.168]) +AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl +IPv4 Routes +Route Table
: + 192.168.0.0/24 169.254.100.1 dst-ip (learned) + +Route Table rtb1: + 10.11.1.0/24 169.254.100.1 dst-ip (learned) + 10.11.2.0/24 169.254.0.1 dst-ip + 10.22.2.0/24 169.254.0.2 src-ip + 0.0.0.0/0 169.254.100.1 dst-ip (learned) +]) + +# Delete the directly connected subnet from AZ1, learned route should be +# removed from AZ2. +ovn_as az1 ovn-nbctl lrp-del lrp-lr1-ls1 +OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned | grep 192.168]) + +# Test blacklist routes +# Add back the directly connected 192.168 route. +ovn_as az1 ovn-nbctl lrp-add lr1 lrp-lr1-ls1 aa:aa:aa:aa:bb:01 "192.168.0.1/24" +OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned | grep 192.168]) +# Now add 10.11.0.0/16 and 192.168.0.0/16 to blacklist in AZ2. +check ovn_as az2 ovn-nbctl set nb_global . options:ic-route-blacklist="10.11.0.0/16,192.168.0.0/16" +# AZ2 shouldn't learn 192.168 route any more. +OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned | grep 192.168]) +# AZ1 shouldn't learn 10.11 any more. +OVS_WAIT_WHILE([ovn_as az1 ovn-nbctl lr-route-list lr1 | grep learned | grep 10.11]) +AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl +IPv4 Routes +Route Table rtb1: + 10.11.2.0/24 169.254.0.1 dst-ip + 10.22.2.0/24 169.254.0.2 src-ip + 0.0.0.0/0 169.254.100.1 dst-ip (learned) +]) + +OVN_CLEANUP_IC([az1], [az2]) + +AT_CLEANUP +]) + + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([ovn-ic -- route sync -- multiple route tables]) + +ovn_init_ic_db +ovn-ic-nbctl ts-add ts1 + +for i in 1 2; do + ovn_start az$i + ovn_as az$i + + # Enable route learning at AZ level + ovn-nbctl set nb_global . options:ic-route-learn=true + # Enable route advertising at AZ level + ovn-nbctl set nb_global . options:ic-route-adv=true +done + +# Create new transit switches and LRs. Test topology is next: +# VPC1: +# / transit switch (ts11) \ +# logical router (lr11) - transit switch (ts12) - logical router (lr12) +# \ transit switch (ts13) / +# +# VPC2: +# / transit switch (ts21) \ +# logical router (lr21) logical router (lr22) +# \ transit switch (ts22) / +# +# each LR has one connected subnet except TS port + + +# VPC1 +# create lr11, lr12, ts11, ts12, ts13 and connect them +# assign route tables rtb1, rtb2, rtb3 to ts ports +for i in 1 2; do + ovn_as az$i + + lr=lr1$i + ovn-nbctl lr-add $lr + + for j in 1 2 3; do + ts=ts1$j + ovn-ic-nbctl --may-exist ts-add $ts + + lrp=lrp-$lr-$ts + lsp=lsp-$ts-$lr + # Create LRP and connect to TS + ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:a$j:0$i 169.254.10$j.$i/24 + ovn-nbctl lrp-set-options $lrp route_table=rtb$j + ovn-nbctl lsp-add $ts $lsp \ + -- lsp-set-addresses $lsp router \ + -- lsp-set-type $lsp router \ + -- lsp-set-options $lsp router-port=$lrp + done +done + +# VPC2 +# create lr21, lr22, ts21, ts22 and connect them +# assign route tables rtb1, rtb2, rtb3 to ts ports +for i in 1 2; do + ovn_as az$i + + lr=lr2$i + ovn-nbctl lr-add $lr + + for j in 1 2; do + ts=ts2$j + ovn-ic-nbctl --may-exist ts-add $ts + + lrp=lrp-$lr-$ts + lsp=lsp-$ts-$lr + # Create LRP and connect to TS + ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:a$j:0$i 169.254.10$j.$i/24 + ovn-nbctl lrp-set-options $lrp route_table=rtb$j + ovn-nbctl lsp-add $ts $lsp \ + -- lsp-set-addresses $lsp router \ + -- lsp-set-type $lsp router \ + -- lsp-set-options $lsp router-port=$lrp + done +done + +# Create directly-connected and static routes in VPC1 +ovn_as az2 ovn-nbctl lrp-add lr12 lrp-lr12 aa:aa:aa:aa:bb:01 "192.168.0.1/24" +ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-add lr12 10.10.10.0/24 192.168.0.10 +ovn_as az2 ovn-nbctl --route-table=rtb2 lr-route-add lr12 10.10.10.0/24 192.168.0.11 +ovn_as az2 ovn-nbctl --route-table=rtb3 lr-route-add lr12 10.10.10.0/24 192.168.0.12 + +# Create directly-connected route in VPC2 +ovn_as az2 ovn-nbctl lrp-add lr22 lrp-lr22 aa:aa:aa:aa:bb:01 "192.168.0.1/24" + +# Test direct routes from lr12 were learned to lr11 +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr11 | grep 192.168 | + grep learned | awk '{print $1, $2, $5}' | sort ], [0], [dnl +192.168.0.0/24 169.254.101.2 ecmp +192.168.0.0/24 169.254.102.2 ecmp +192.168.0.0/24 169.254.103.2 ecmp +]) + +# Test static routes from lr12 rtbs rtb1,rtb2,rtb3 were learned to lr11 +AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-list lr11], [0], [dnl +IPv4 Routes +Route Table rtb1: + 10.10.10.0/24 169.254.101.2 dst-ip (learned) +]) +AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb2 lr-route-list lr11], [0], [dnl +IPv4 Routes +Route Table rtb2: + 10.10.10.0/24 169.254.102.2 dst-ip (learned) +]) +AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb3 lr-route-list lr11], [0], [dnl +IPv4 Routes +Route Table rtb3: + 10.10.10.0/24 169.254.103.2 dst-ip (learned) +]) + +# Test routes from lr12 didn't leak as learned to lr21 +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr21 | grep 192.168 | sort], [0], [dnl + 192.168.0.0/24 169.254.101.2 dst-ip (learned) ecmp + 192.168.0.0/24 169.254.102.2 dst-ip (learned) ecmp +]) + +OVN_CLEANUP_IC([az1], [az2]) + +AT_CLEANUP +]) + + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([ovn-ic -- route sync -- multiple route tables IPv6]) + +ovn_init_ic_db +ovn-ic-nbctl ts-add ts1 + +for i in 1 2; do + ovn_start az$i + ovn_as az$i + + # Enable route learning at AZ level + ovn-nbctl set nb_global . options:ic-route-learn=true + # Enable route advertising at AZ level + ovn-nbctl set nb_global . options:ic-route-adv=true +done + +# Create new transit switches and LRs. Test topology is next: +# VPC1: +# / transit switch (ts11) \ +# logical router (lr11) - transit switch (ts12) - logical router (lr12) +# \ transit switch (ts13) / +# +# VPC2: +# / transit switch (ts21) \ +# logical router (lr21) logical router (lr22) +# \ transit switch (ts22) / +# +# each LR has one connected subnet except TS port + + +# VPC1 +# create lr11, lr12, ts11, ts12, ts13 and connect them +# assign route tables rtb1, rtb2, rtb3 to ts ports +for i in 1 2; do + ovn_as az$i + + lr=lr1$i + ovn-nbctl lr-add $lr + + for j in 1 2 3; do + ts=ts1$j + ovn-ic-nbctl --may-exist ts-add $ts + + lrp=lrp-$lr-$ts + lsp=lsp-$ts-$lr + # Create LRP and connect to TS + ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:a$j:0$i 2001:db8:$j::$i/64 + ovn-nbctl lrp-set-options $lrp route_table=rtb$j + ovn-nbctl lsp-add $ts $lsp \ + -- lsp-set-addresses $lsp router \ + -- lsp-set-type $lsp router \ + -- lsp-set-options $lsp router-port=$lrp + done +done + +# VPC2 +# create lr21, lr22, ts21, ts22 and connect them +# assign route tables rtb1, rtb2, rtb3 to ts ports +for i in 1 2; do + ovn_as az$i + + lr=lr2$i + ovn-nbctl lr-add $lr + + for j in 1 2; do + ts=ts2$j + ovn-ic-nbctl --may-exist ts-add $ts + + lrp=lrp-$lr-$ts + lsp=lsp-$ts-$lr + # Create LRP and connect to TS + ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:a$j:0$i 2001:db8:$j::$i/64 + ovn-nbctl lrp-set-options $lrp route_table=rtb$j + ovn-nbctl lsp-add $ts $lsp \ + -- lsp-set-addresses $lsp router \ + -- lsp-set-type $lsp router \ + -- lsp-set-options $lsp router-port=$lrp + done +done + +# Create directly-connected and static routes in VPC1 +ovn_as az2 ovn-nbctl lrp-add lr12 lrp-lr12 aa:aa:aa:aa:bb:01 "2001:db8:200::1/64" +ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-add lr12 2001:db8:aaaa::/64 2001:db8:200::10 +ovn_as az2 ovn-nbctl --route-table=rtb2 lr-route-add lr12 2001:db8:aaaa::/64 2001:db8:200::11 +ovn_as az2 ovn-nbctl --route-table=rtb3 lr-route-add lr12 2001:db8:aaaa::/64 2001:db8:200::12 + +# Create directly-connected route in VPC2 +ovn_as az2 ovn-nbctl lrp-add lr22 lrp-lr22 aa:aa:aa:aa:bb:01 "2001:db8:200::1/64" + +# Test direct routes from lr12 were learned to lr11 +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr11 | grep 2001:db8:200 | + grep learned | awk '{print $1, $2, $5}' | sort], [0], [dnl +2001:db8:200::/64 2001:db8:1::2 ecmp +2001:db8:200::/64 2001:db8:2::2 ecmp +2001:db8:200::/64 2001:db8:3::2 ecmp +]) + +# Test static routes from lr12 rtbs rtb1,rtb2,rtb3 were learned to lr11 +AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-list lr11], [0], [dnl +IPv6 Routes +Route Table rtb1: + 2001:db8:aaaa::/64 2001:db8:1::2 dst-ip (learned) +]) +AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb2 lr-route-list lr11], [0], [dnl +IPv6 Routes +Route Table rtb2: + 2001:db8:aaaa::/64 2001:db8:2::2 dst-ip (learned) +]) +AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb3 lr-route-list lr11], [0], [dnl +IPv6 Routes +Route Table rtb3: + 2001:db8:aaaa::/64 2001:db8:3::2 dst-ip (learned) +]) + +# Test routes from lr12 didn't leak as learned to lr21 +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr21 | grep 2001 | sort], [0], [dnl + 2001:db8:200::/64 2001:db8:1::2 dst-ip (learned) ecmp + 2001:db8:200::/64 2001:db8:2::2 dst-ip (learned) ecmp +]) + +OVN_CLEANUP_IC([az1], [az2]) + +AT_CLEANUP +]) From patchwork Sat Nov 13 09:43:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladislav Odintsov X-Patchwork-Id: 1554665 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=XOThOK61; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HrrBm4fK5z9sPf for ; Sat, 13 Nov 2021 20:44:16 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 03226404FD; Sat, 13 Nov 2021 09:44:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gU5Yh2_Ys_-d; Sat, 13 Nov 2021 09:44:10 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 82409404E7; Sat, 13 Nov 2021 09:44:09 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 39821C0039; Sat, 13 Nov 2021 09:44:09 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id DCF56C0038 for ; Sat, 13 Nov 2021 09:44:07 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id ACFE840639 for ; Sat, 13 Nov 2021 09:44:07 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3dt94qwUHwZ3 for ; Sat, 13 Nov 2021 09:44:06 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com [IPv6:2a00:1450:4864:20::136]) by smtp4.osuosl.org (Postfix) with ESMTPS id CB70C40417 for ; Sat, 13 Nov 2021 09:44:05 +0000 (UTC) Received: by mail-lf1-x136.google.com with SMTP id p16so28680343lfa.2 for ; Sat, 13 Nov 2021 01:44:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cJbZqFLaZMmzrFduHtkVFI2V8YXTQ66GTeGA3jZYlVo=; b=XOThOK61tp8eLPCoH/Lm97S4F3+IrVFhs9O01W9X2yvUH8ENd+Iiafv8wHQe25XA5e T0TQJlvCNIUVhaX2i3W1O9rf1ils5KYSAiuTUYHw9UQU6zDAc27eLSw7dm5eT2BxCqG0 NSGooiFD9V4qgsjjchX6mLp9AVIYFQ85JngLzHhTGheQWRzPSFXpVdeyY2snRG/zCdrv kyo1BiljyD/ZH761gcU6v1wX1Rtxm9ksosLhLvenUlSHEGx2UFgFvUqhkaphP2rLNo2/ eRO4705bc//dcQc9hE5dRg41o2r5q+yd+T2L5PllakKsm3JCHB7kK0Kj+HL0Cl3lYNyS bENw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cJbZqFLaZMmzrFduHtkVFI2V8YXTQ66GTeGA3jZYlVo=; b=FHzI44amPivjU+6nxBf/o5Q5Gyk3O1IsXdrsF2PoEhioZf1HWGS620J+V5iNP5uG+I pfxfcO1HNZMeQ/X1NbiAf57bIZlqnVsWEfuvscCP0h31mmw1N8mGu818S+t50uGW9fZA ItGAVtvOSIELbCogddNlYPaw0Grw5WBCapAhNJLpc0D42LOnBqAHpkiMOrlCsqdZ8Wr6 QTHTtdxwuwFUivC6epT69jOMIC9+wOiGtapMfVYQ3pD1sche7/582v6QRea3uXfzHMsW h3ZfNOlr9oqahCKU+Cc/5tKygazf2hsn/rH0+KIYLP5WHDdxdKr5doz3HF2J37Q39O5B hO0w== X-Gm-Message-State: AOAM5312eMZVsUlseY7B0EVzavRgnN2Pbf+lkllT5mcPCZ7LC/zlQJVT voa3Dlwd7nxlVS/OoyU/lVH6thjSHRZDbw== X-Google-Smtp-Source: ABdhPJyEF3DjWlFZY1B7an60iHYITl8+2ljMPfkuvEWfixLA9hDpN4PZVGKbgVJfS9fHo+3UKBn/Mg== X-Received: by 2002:a05:6512:33c8:: with SMTP id d8mr20166726lfg.573.1636796643651; Sat, 13 Nov 2021 01:44:03 -0800 (PST) Received: from localhost.localdomain (109-252-131-59.dynamic.spd-mgts.ru. [109.252.131.59]) by smtp.gmail.com with ESMTPSA id h1sm818725lfu.277.2021.11.13.01.44.03 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 13 Nov 2021 01:44:03 -0800 (PST) From: Vladislav Odintsov To: dev@openvswitch.org Date: Sat, 13 Nov 2021 12:43:53 +0300 Message-Id: <20211113094353.17690-6-odivlad@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20211113094353.17690-1-odivlad@gmail.com> References: <20211113094353.17690-1-odivlad@gmail.com> MIME-Version: 1.0 Cc: Vladislav Odintsov Subject: [ovs-dev] [PATCH ovn v8 5/5] ic: don't learn routes which have local GW X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" In case we have ovn-ic-interconnected Logical_Routers and install same ip_prefix route with GW in local AZ in each LR in each AZ, this route would be learned in other AZs and L3 loop is possible. There could be next routes output: [az1 ~]$ ovn-nbctl lr-route-list lr0 IPv4 Routes Route Table global: 128.0.0.0/1 169.254.1.1 dst-ip ecmp 128.0.0.0/1 169.254.100.2 dst-ip (learned) ecmp [az2 ~]$ ovn-nbctl lr-route-list lr0 IPv4 Routes Route Table global: 128.0.0.0/1 169.254.2.1 dst-ip ecmp 128.0.0.0/1 169.254.100.1 dst-ip (learned) ecmp So, there is a possible routing loop. Packets going to 128.0.0.0/1 could go from AZ1 to AZ2 and on AZ2 they can be routed back. This commit adds check for installed local (non-learned) routes. If OVN IC route's ip_prefix, route_table are the same with already installed non-learned NB route, such route wouldn't be learned. Signed-off-by: Vladislav Odintsov --- ic/ovn-ic.c | 30 ++++++++++++++++++++++++-- tests/ovn-ic.at | 49 +++++++++++++++++++++++++++++++++++++++++++ utilities/ovn-nbctl.c | 4 +++- 3 files changed, 80 insertions(+), 3 deletions(-) diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c index f40468e92..a9b797af2 100644 --- a/ic/ovn-ic.c +++ b/ic/ovn-ic.c @@ -1209,7 +1209,25 @@ add_network_to_routes_ad(struct hmap *routes_ad, const char *network, } static bool -route_need_learn(struct in6_addr *prefix, unsigned int plen, +route_has_local_gw(const struct nbrec_logical_router *lr, + const char *route_table, const char *ip_prefix) { + + const struct nbrec_logical_router_static_route *route; + for (int i = 0; i < lr->n_static_routes; i++) { + route = lr->static_routes[i]; + if (!smap_get(&route->external_ids, "ic-learned-route") && + !strcmp(route->route_table, route_table) && + !strcmp(route->ip_prefix, ip_prefix)) { + return true; + } + } + return false; +} + +static bool +route_need_learn(const struct nbrec_logical_router *lr, + const struct icsbrec_route *isb_route, + struct in6_addr *prefix, unsigned int plen, const struct smap *nb_options) { if (!smap_get_bool(nb_options, "ic-route-learn", false)) { @@ -1229,6 +1247,12 @@ route_need_learn(struct in6_addr *prefix, unsigned int plen, return false; } + if (route_has_local_gw(lr, isb_route->route_table, isb_route->ip_prefix)) { + VLOG_DBG("Skip learning %s (rtb:%s) route, as we've got one with " + "local GW", isb_route->ip_prefix, isb_route->route_table); + return false; + } + return true; } @@ -1333,9 +1357,11 @@ sync_learned_routes(struct ic_context *ctx, isb_route->nexthop); continue; } - if (!route_need_learn(&prefix, plen, &nb_global->options)) { + if (!route_need_learn(ic_lr->lr, isb_route, &prefix, plen, + &nb_global->options)) { continue; } + struct ic_route_info *route_learned = ic_route_find(&ic_lr->routes_learned, &prefix, plen, &nexthop, isb_route->origin, diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at index 1340874d5..a189a8fed 100644 --- a/tests/ovn-ic.at +++ b/tests/ovn-ic.at @@ -928,3 +928,52 @@ OVN_CLEANUP_IC([az1], [az2]) AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([ovn-ic -- same routes destination]) + +ovn_init_ic_db +ovn-ic-nbctl ts-add ts1 + +for i in 1 2; do + ovn_start az$i + ovn_as az$i + + # Enable route learning at AZ level + ovn-nbctl set nb_global . options:ic-route-learn=true + ovn-nbctl set nb_global . options:ic-route-learn-default=true + # Enable route advertising at AZ level + ovn-nbctl set nb_global . options:ic-route-adv=true + ovn-nbctl set nb_global . options:ic-route-adv-default=true + + lr=lr1$i + ovn-nbctl lr-add $lr + + lrp=lrp-$lr-ts1 + lsp=lsp-ts1-$lr + # Create LRP and connect to TS + ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:aa:0$i 169.254.100.$i/24 + ovn-nbctl lsp-add ts1 $lsp \ + -- lsp-set-addresses $lsp router \ + -- lsp-set-type $lsp router \ + -- lsp-set-options $lsp router-port=$lrp + ovn-nbctl lrp-add $lr lrp-local-subnet 00:00:00:00:00:0$i 192.168.$i.1/24 + ovn-nbctl list logical-router-static-route + check ovn-nbctl lr-route-add $lr 10.0.0.0/24 192.168.$i.10 + check ovn-nbctl lr-route-add $lr 0.0.0.0/0 192.168.$i.11 +done + +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr11 | grep dst-ip | sort], [0], [dnl + 0.0.0.0/0 192.168.1.11 dst-ip + 10.0.0.0/24 192.168.1.10 dst-ip + 192.168.2.0/24 169.254.100.2 dst-ip (learned) +]) + +AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr12 | grep dst-ip | sort], [0], [dnl + 0.0.0.0/0 192.168.2.11 dst-ip + 10.0.0.0/24 192.168.2.10 dst-ip + 192.168.1.0/24 169.254.100.1 dst-ip (learned) +]) + +AT_CLEANUP +]) diff --git a/utilities/ovn-nbctl.c b/utilities/ovn-nbctl.c index 8bdcb19a3..17bb5d41d 100644 --- a/utilities/ovn-nbctl.c +++ b/utilities/ovn-nbctl.c @@ -4104,6 +4104,8 @@ nbctl_pre_lr_route_add(struct ctl_context *ctx) &nbrec_logical_router_static_route_col_options); ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_static_route_col_route_table); + ovsdb_idl_add_column(ctx->idl, + &nbrec_logical_router_static_route_col_external_ids); } static char * OVS_WARN_UNUSED_RESULT @@ -4233,7 +4235,7 @@ nbctl_lr_route_add(struct ctl_context *ctx) } if (!ecmp) { - if (route) { + if (route && !smap_get(&route->external_ids, "ic-learned-route")) { if (!may_exist) { ctl_error(ctx, "duplicate prefix: %s (policy: %s). Use option" " --ecmp to allow this for ECMP routing.",