From patchwork Sun Sep 19 21:23:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladislav Odintsov X-Patchwork-Id: 1529888 X-Patchwork-Delegate: zhouhan@gmail.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=bPNFQw1A; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4HCLKQ232sz9sW5 for ; Mon, 20 Sep 2021 07:23:53 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 6332B4042F; Sun, 19 Sep 2021 21:23:51 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id w33Nftcmn1Je; Sun, 19 Sep 2021 21:23:50 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 878DC40352; Sun, 19 Sep 2021 21:23:49 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 18A70C0025; Sun, 19 Sep 2021 21:23:48 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id A5EC3C0011 for ; Sun, 19 Sep 2021 21:23:46 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id CCF1A4012F for ; Sun, 19 Sep 2021 21:23:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GhDBNR_jLTRM for ; Sun, 19 Sep 2021 21:23:42 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com [IPv6:2a00:1450:4864:20::135]) by smtp2.osuosl.org (Postfix) with ESMTPS id D31C0402CF for ; Sun, 19 Sep 2021 21:23:41 +0000 (UTC) Received: by mail-lf1-x135.google.com with SMTP id c8so58536864lfi.3 for ; Sun, 19 Sep 2021 14:23:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oCcAKZQjG9lSMMmrjQg/37kocHd4cudxKbHpf5/jSdE=; b=bPNFQw1Ar5ivlusLamfIKxTEzc+h3oyxi6/FXg6PndYM04wvxvzVZCarH3cH6vbQNa cpniJP2G11/V/vsjqbE4BJxKm1+K7KaVeIxvEju0bNTkuN+ISdzr45jte6DfZouqkyyx 8WozWGiz+k6CPdj2AKFZvyteb8Zh5gsQ4/DkQ/OelEWVzf7MHF02NsQP+N0iSBtDEsN8 UNjKVbbRg3zRJWMpjD9GMTEgqm/9sKvSMPf9VWv6HpLm9lS8duMFFRnJQqPEIGbhDWQS FZ1gzcLX1lEoRH64VA1+xjtz6auX6FCMRMfBsybSm3SnRflZKJUA867rGRkbHYOtcR+W nbGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oCcAKZQjG9lSMMmrjQg/37kocHd4cudxKbHpf5/jSdE=; b=SMEVUERc5E03lRhnDhLLZIXELq1P73Iwd+UcbWPjvrtu2jdUzx0bRoTBb4niT+8qr3 hwljiSF6Td/Y1YBVsMYiLsdhWBiXBXLVrSJ4YG+CTO1P9VVSY7TxA2z76slNaUakClUI 3gBaEElZZPlaqx9V01aBDEahO58bPadImqzI/lMkQzc0eeGyt/Iz8sz0peh1KnXg/LWk ZmGaPfCMAwr5U8IlOZN7RkfpVXIlocesuHUCENjSonwI+bOT5HZIVd/V6I/4SQ7nUxR8 oApsccdhpDevmfCIeBkbRklkmUdXG29rKwmikuErqYASw+74doPGh1SZKKY8zjtnzeeR W0OA== X-Gm-Message-State: AOAM532FEPRvZHkhNNjakJm+mzGQ02HTL/O1GMN6gMKNfrqcFXt/X/3v Q1bSMCi3Z8iQXLTC/4nxStV0CJ989q8= X-Google-Smtp-Source: ABdhPJxYS40uDl7Vt0tYujF3Uyo2020fiWwsx6oaUSESvrugmx6SaMneKKHJOONE5BsP0H5/OORjZQ== X-Received: by 2002:a19:e051:: with SMTP id g17mr17407455lfj.519.1632086619531; Sun, 19 Sep 2021 14:23:39 -0700 (PDT) Received: from localhost.localdomain (109-252-131-59.dynamic.spd-mgts.ru. [109.252.131.59]) by smtp.gmail.com with ESMTPSA id q5sm431658lfn.277.2021.09.19.14.23.39 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 19 Sep 2021 14:23:39 -0700 (PDT) From: Vladislav Odintsov To: dev@openvswitch.org Date: Mon, 20 Sep 2021 00:23:13 +0300 Message-Id: <20210919212316.24148-2-odivlad@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210919212316.24148-1-odivlad@gmail.com> References: <20210919212316.24148-1-odivlad@gmail.com> MIME-Version: 1.0 Cc: Vladislav Odintsov Subject: [ovs-dev] [PATCH ovn v4 1/4] ic: process only local port_bindings X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds a small optimization by utilizing ovsdb_index to iterate over port_bindings. Prior to this change each iteration checked availability_zone and continued processing only if port_binding belons to local AZ. Now we run against port_bindings from local AZ only and don't check availability_zone. Signed-off-by: Vladislav Odintsov Acked-by: Numan Siddique --- ic/ovn-ic.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c index 99356253d..92c83d730 100644 --- a/ic/ovn-ic.c +++ b/ic/ovn-ic.c @@ -68,6 +68,7 @@ struct ic_context { struct ovsdb_idl_index *sbrec_port_binding_by_name; struct ovsdb_idl_index *icsbrec_port_binding_by_az; struct ovsdb_idl_index *icsbrec_port_binding_by_ts; + struct ovsdb_idl_index *icsbrec_port_binding_by_ts_az; struct ovsdb_idl_index *icsbrec_route_by_ts; struct ovsdb_idl_index *icsbrec_route_by_ts_az; }; @@ -1386,17 +1387,15 @@ route_run(struct ic_context *ctx, const struct icsbrec_port_binding *isb_pb; const struct icsbrec_port_binding *isb_pb_key = icsbrec_port_binding_index_init_row( - ctx->icsbrec_port_binding_by_ts); + ctx->icsbrec_port_binding_by_ts_az); icsbrec_port_binding_index_set_transit_switch(isb_pb_key, ts->name); + icsbrec_port_binding_index_set_availability_zone(isb_pb_key, az); /* Each port on TS maps to a logical router, which is stored in the * external_ids:router-id of the IC SB port_binding record. */ ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key, - ctx->icsbrec_port_binding_by_ts) { - if (isb_pb->availability_zone != az) { - continue; - } - + ctx->icsbrec_port_binding_by_ts_az) + { const char *ts_lrp_name = get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port); if (!ts_lrp_name) { @@ -1713,6 +1712,11 @@ main(int argc, char *argv[]) = ovsdb_idl_index_create1(ovnisb_idl_loop.idl, &icsbrec_port_binding_col_transit_switch); + struct ovsdb_idl_index *icsbrec_port_binding_by_ts_az + = ovsdb_idl_index_create2(ovnisb_idl_loop.idl, + &icsbrec_port_binding_col_transit_switch, + &icsbrec_port_binding_col_availability_zone); + struct ovsdb_idl_index *icsbrec_route_by_ts = ovsdb_idl_index_create1(ovnisb_idl_loop.idl, &icsbrec_route_col_transit_switch); @@ -1763,6 +1767,7 @@ main(int argc, char *argv[]) .sbrec_chassis_by_name = sbrec_chassis_by_name, .icsbrec_port_binding_by_az = icsbrec_port_binding_by_az, .icsbrec_port_binding_by_ts = icsbrec_port_binding_by_ts, + .icsbrec_port_binding_by_ts_az = icsbrec_port_binding_by_ts_az, .icsbrec_route_by_ts = icsbrec_route_by_ts, .icsbrec_route_by_ts_az = icsbrec_route_by_ts_az, }; From patchwork Sun Sep 19 21:23:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladislav Odintsov X-Patchwork-Id: 1529890 X-Patchwork-Delegate: zhouhan@gmail.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=n4Yjp4jl; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4HCLKd3XdHz9sW5 for ; Mon, 20 Sep 2021 07:24:04 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 056C480C65; Sun, 19 Sep 2021 21:23:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pyNejrlIsQWo; Sun, 19 Sep 2021 21:23:52 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id E124183FC5; Sun, 19 Sep 2021 21:23:50 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 14F7AC001E; Sun, 19 Sep 2021 21:23:50 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id CF2D4C000D for ; Sun, 19 Sep 2021 21:23:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id AC909402BC for ; Sun, 19 Sep 2021 21:23:47 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ugP7l_PJ12dk for ; Sun, 19 Sep 2021 21:23:45 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com [IPv6:2a00:1450:4864:20::134]) by smtp4.osuosl.org (Postfix) with ESMTPS id B1D7B40183 for ; Sun, 19 Sep 2021 21:23:44 +0000 (UTC) Received: by mail-lf1-x134.google.com with SMTP id z24so32396755lfu.13 for ; Sun, 19 Sep 2021 14:23:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8VLiBPner4LICLvgQyjA7jqcS2cXD5bC6O0LxhK56P4=; b=n4Yjp4jlLB7KXxGl9tfUP2ZnDbd4IOFumKNX+2kBevzBbfXY2twhACxy6dIBe+Txgt EzURnuDQR3ktK7kQgJJJtqNaot8L3tkbeSZ9neMbn5dh5bUzm8s1u6I7hiEejEDBB86h 3xiN5vn1EfY1Z4mARmBgVOpKh0Ld7L++8azMqV2Mn6I4F7+7ZKN37HkI34fMtA+1asDC /pQwg4GMw92xze/BNSaLhUHrJEWdhx8q3G8I9LGwR5GkiJaSCpP/xz8gWEdz65vLxCNS BIMAAkua6qkURl5wYIqc6PoZSn36rP60BehZBxsoHHkm85sQ6nh1wsPV2VYdIc6v7pdT RVMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8VLiBPner4LICLvgQyjA7jqcS2cXD5bC6O0LxhK56P4=; b=62lIl692eyX/v0IWz52bkgYrBsAKnqdGFPWXdZlWqjp/IAwrCR44MhiywcgwIF3oQ6 t67L94chJ4KI0H8hB8pGknZrU628fO/9mmXWCO9xPJv/rfAcGh2+cXP2xNTk5reV1KD1 qLgky2DL0RkPZBMlyTYy66fO6e/VoxOx2UjBB/9y6aYYMQDeupSEiYQLIjEozvJY9eDg +0d4Qqr4TfBxTjMl6J5ceaTKy31uw6kYzRUdP50T6FJpxwq3YixE1G3x+9ZONpp26qgx 0ECBXC2Kt04dFZ4ElwGrsbPPU5zSRudZYseFn/eg/fxIxDKMaA1CzCQYbn+pD7ZRWNMf wEdA== X-Gm-Message-State: AOAM532w9c5bRNCu5Mj79t17+aZD9SOeqmDCabqKq1sZIM+qH3nWCUXt tgt6o8DaO41fPd6qf8P8jTMszs4Swpg= X-Google-Smtp-Source: ABdhPJy+gRogquT/eoGWVx0L64VPZHS8LSRVQ7ZlLpya2Mmw0rpk94tKkTs7uEP+y1kIpnoMr4kZNg== X-Received: by 2002:a05:6512:a8b:: with SMTP id m11mr7223232lfu.574.1632086620443; Sun, 19 Sep 2021 14:23:40 -0700 (PDT) Received: from localhost.localdomain (109-252-131-59.dynamic.spd-mgts.ru. [109.252.131.59]) by smtp.gmail.com with ESMTPSA id q5sm431658lfn.277.2021.09.19.14.23.39 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 19 Sep 2021 14:23:40 -0700 (PDT) From: Vladislav Odintsov To: dev@openvswitch.org Date: Mon, 20 Sep 2021 00:23:14 +0300 Message-Id: <20210919212316.24148-3-odivlad@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210919212316.24148-1-odivlad@gmail.com> References: <20210919212316.24148-1-odivlad@gmail.com> MIME-Version: 1.0 Cc: Vladislav Odintsov Subject: [ovs-dev] [PATCH ovn v4 2/4] northd, utils: support for RouteTables in LRs X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Signed-off-by: Vladislav Odintsov Acked-by: Numan Siddique --- northd/northd.c | 159 ++++++++++++--- northd/ovn-northd.8.xml | 63 ++++-- ovn-nb.ovsschema | 5 +- ovn-nb.xml | 30 +++ tests/ovn-ic.at | 4 + tests/ovn-nbctl.at | 196 +++++++++++++++++- tests/ovn-northd.at | 76 ++++++- tests/ovn.at | 441 +++++++++++++++++++++++++++++++++++++++- utilities/ovn-nbctl.c | 134 +++++++++++- 9 files changed, 1041 insertions(+), 67 deletions(-) diff --git a/northd/northd.c b/northd/northd.c index d1b87891c..c238c6241 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -152,15 +152,16 @@ enum ovn_stage { PIPELINE_STAGE(ROUTER, IN, ECMP_STATEFUL, 7, "lr_in_ecmp_stateful") \ PIPELINE_STAGE(ROUTER, IN, ND_RA_OPTIONS, 8, "lr_in_nd_ra_options") \ PIPELINE_STAGE(ROUTER, IN, ND_RA_RESPONSE, 9, "lr_in_nd_ra_response") \ - PIPELINE_STAGE(ROUTER, IN, IP_ROUTING, 10, "lr_in_ip_routing") \ - PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_ECMP, 11, "lr_in_ip_routing_ecmp") \ - PIPELINE_STAGE(ROUTER, IN, POLICY, 12, "lr_in_policy") \ - PIPELINE_STAGE(ROUTER, IN, POLICY_ECMP, 13, "lr_in_policy_ecmp") \ - PIPELINE_STAGE(ROUTER, IN, ARP_RESOLVE, 14, "lr_in_arp_resolve") \ - PIPELINE_STAGE(ROUTER, IN, CHK_PKT_LEN , 15, "lr_in_chk_pkt_len") \ - PIPELINE_STAGE(ROUTER, IN, LARGER_PKTS, 16, "lr_in_larger_pkts") \ - PIPELINE_STAGE(ROUTER, IN, GW_REDIRECT, 17, "lr_in_gw_redirect") \ - PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 18, "lr_in_arp_request") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_PRE, 10, "lr_in_ip_routing_pre") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING, 11, "lr_in_ip_routing") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_ECMP, 12, "lr_in_ip_routing_ecmp") \ + PIPELINE_STAGE(ROUTER, IN, POLICY, 13, "lr_in_policy") \ + PIPELINE_STAGE(ROUTER, IN, POLICY_ECMP, 14, "lr_in_policy_ecmp") \ + PIPELINE_STAGE(ROUTER, IN, ARP_RESOLVE, 15, "lr_in_arp_resolve") \ + PIPELINE_STAGE(ROUTER, IN, CHK_PKT_LEN, 16, "lr_in_chk_pkt_len") \ + PIPELINE_STAGE(ROUTER, IN, LARGER_PKTS, 17, "lr_in_larger_pkts") \ + PIPELINE_STAGE(ROUTER, IN, GW_REDIRECT, 18, "lr_in_gw_redirect") \ + PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 19, "lr_in_arp_request") \ \ /* Logical router egress stages. */ \ PIPELINE_STAGE(ROUTER, OUT, UNDNAT, 0, "lr_out_undnat") \ @@ -229,6 +230,7 @@ enum ovn_stage { #define REG_NEXT_HOP_IPV6 "xxreg0" #define REG_SRC_IPV4 "reg1" #define REG_SRC_IPV6 "xxreg1" +#define REG_ROUTE_TABLE_ID "reg7" #define REG_ORIG_TP_DPORT_ROUTER "reg9[16..31]" @@ -291,8 +293,9 @@ enum ovn_stage { * | R6 | UNUSED | X | | G | IN_IP_ROUTING)| * | | | R | | 1 | | * +-----+--------------------------+ E | UNUSED | | | - * | R7 | UNUSED | G | | | | - * | | | 3 | | | | + * | R7 | ROUTE_TABLE_ID | G | | | | + * | | (>= IN_IP_ROUTING_PRE && | 3 | | | | + * | | <= IN_IP_ROUTING) | | | | | * +-----+--------------------------+---+-----------------+---+---------------+ * | R8 | ECMP_GROUP_ID | | | * | | ECMP_MEMBER_ID | X | | @@ -8520,11 +8523,72 @@ cleanup: ds_destroy(&actions); } +static uint32_t +route_table_add(struct simap *route_tables, const char *route_table_name) +{ + /* route table ids start from 1 */ + uint32_t rtb_id = simap_count(route_tables) + 1; + + if (rtb_id == UINT16_MAX) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "too many route tables for Logical Router."); + return 0; + } + + if (!simap_put(route_tables, route_table_name, rtb_id)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "Route table id unexpectedly appeared"); + } + + return rtb_id; +} + +static uint32_t +get_route_table_id(struct simap *route_tables, const char *route_table_name) +{ + if (!route_table_name || !strlen(route_table_name)) { + return 0; + } + + uint32_t rtb_id = simap_get(route_tables, route_table_name); + if (!rtb_id) { + rtb_id = route_table_add(route_tables, route_table_name); + } + + return rtb_id; +} + +static void +build_route_table_lflow(struct ovn_datapath *od, struct hmap *lflows, + struct nbrec_logical_router_port *lrp, + struct simap *route_tables) +{ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + + const char *route_table_name = smap_get(&lrp->options, "route_table"); + uint32_t rtb_id = get_route_table_id(route_tables, route_table_name); + if (!rtb_id) { + return; + } + + ds_put_format(&match, "inport == \"%s\"", lrp->name); + ds_put_format(&actions, "%s = %d; next;", + REG_ROUTE_TABLE_ID, rtb_id); + + ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 100, + ds_cstr(&match), ds_cstr(&actions)); + + ds_destroy(&match); + ds_destroy(&actions); +} + struct parsed_route { struct ovs_list list_node; struct in6_addr prefix; unsigned int plen; bool is_src_route; + uint32_t route_table_id; uint32_t hash; const struct nbrec_logical_router_static_route *route; bool ecmp_symmetric_reply; @@ -8549,7 +8613,7 @@ find_static_route_outport(struct ovn_datapath *od, struct hmap *ports, * Otherwise return NULL. */ static struct parsed_route * parsed_routes_add(struct ovn_datapath *od, struct hmap *ports, - struct ovs_list *routes, + struct ovs_list *routes, struct simap *route_tables, const struct nbrec_logical_router_static_route *route, struct hmap *bfd_connections) { @@ -8631,6 +8695,7 @@ parsed_routes_add(struct ovn_datapath *od, struct hmap *ports, struct parsed_route *pr = xzalloc(sizeof *pr); pr->prefix = prefix; pr->plen = plen; + pr->route_table_id = get_route_table_id(route_tables, route->route_table); pr->is_src_route = (route->policy && !strcmp(route->policy, "src-ip")); pr->hash = route_hash(pr); @@ -8664,6 +8729,7 @@ struct ecmp_groups_node { struct in6_addr prefix; unsigned int plen; bool is_src_route; + uint32_t route_table_id; uint16_t route_count; struct ovs_list route_list; /* Contains ecmp_route_list_node */ }; @@ -8672,7 +8738,7 @@ static void ecmp_groups_add_route(struct ecmp_groups_node *group, const struct parsed_route *route) { - if (group->route_count == UINT16_MAX) { + if (group->route_count == UINT16_MAX) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); VLOG_WARN_RL(&rl, "too many routes in a single ecmp group."); return; @@ -8701,6 +8767,7 @@ ecmp_groups_add(struct hmap *ecmp_groups, eg->prefix = route->prefix; eg->plen = route->plen; eg->is_src_route = route->is_src_route; + eg->route_table_id = route->route_table_id; ovs_list_init(&eg->route_list); ecmp_groups_add_route(eg, route); @@ -8714,7 +8781,8 @@ ecmp_groups_find(struct hmap *ecmp_groups, struct parsed_route *route) HMAP_FOR_EACH_WITH_HASH (eg, hmap_node, route->hash, ecmp_groups) { if (ipv6_addr_equals(&eg->prefix, &route->prefix) && eg->plen == route->plen && - eg->is_src_route == route->is_src_route) { + eg->is_src_route == route->is_src_route && + eg->route_table_id == route->route_table_id) { return eg; } } @@ -8761,7 +8829,8 @@ unique_routes_remove(struct hmap *unique_routes, HMAP_FOR_EACH_WITH_HASH (ur, hmap_node, route->hash, unique_routes) { if (ipv6_addr_equals(&route->prefix, &ur->route->prefix) && route->plen == ur->route->plen && - route->is_src_route == ur->route->is_src_route) { + route->is_src_route == ur->route->is_src_route && + route->route_table_id == ur->route->route_table_id) { hmap_remove(unique_routes, &ur->hmap_node); const struct parsed_route *existed_route = ur->route; free(ur); @@ -8799,9 +8868,9 @@ build_route_prefix_s(const struct in6_addr *prefix, unsigned int plen) } static void -build_route_match(const struct ovn_port *op_inport, const char *network_s, - int plen, bool is_src_route, bool is_ipv4, struct ds *match, - uint16_t *priority) +build_route_match(const struct ovn_port *op_inport, uint32_t rtb_id, + const char *network_s, int plen, bool is_src_route, + bool is_ipv4, struct ds *match, uint16_t *priority) { const char *dir; /* The priority here is calculated to implement longest-prefix-match @@ -8817,6 +8886,15 @@ build_route_match(const struct ovn_port *op_inport, const char *network_s, if (op_inport) { ds_put_format(match, "inport == %s && ", op_inport->json_key); } + if (rtb_id) { + ds_put_format(match, "%s == %d && ", REG_ROUTE_TABLE_ID, rtb_id); + } else { + /* Route-table assigned LRPs' routes should have lower priority + * in order not to affect directly-connected global routes. + * So, enlarge non-route-table routes priority by 100. + */ + *priority += 100; + } ds_put_format(match, "ip%s.%s == %s/%d", is_ipv4 ? "4" : "6", dir, network_s, plen); } @@ -8951,7 +9029,7 @@ add_ecmp_symmetric_reply_flows(struct hmap *lflows, out_port->lrp_networks.ea_s, IN6_IS_ADDR_V4MAPPED(&route->prefix) ? "" : "xx", port_ip, out_port->json_key); - ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, 300, + ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, 400, ds_cstr(&match), ds_cstr(&actions), &st_route->header_); @@ -8981,8 +9059,8 @@ build_ecmp_route_flow(struct hmap *lflows, struct ovn_datapath *od, struct ds route_match = DS_EMPTY_INITIALIZER; char *prefix_s = build_route_prefix_s(&eg->prefix, eg->plen); - build_route_match(NULL, prefix_s, eg->plen, eg->is_src_route, is_ipv4, - &route_match, &priority); + build_route_match(NULL, eg->route_table_id, prefix_s, eg->plen, + eg->is_src_route, is_ipv4, &route_match, &priority); free(prefix_s); struct ds actions = DS_EMPTY_INITIALIZER; @@ -9057,8 +9135,8 @@ static void add_route(struct hmap *lflows, struct ovn_datapath *od, const struct ovn_port *op, const char *lrp_addr_s, const char *network_s, int plen, const char *gateway, - bool is_src_route, const struct ovsdb_idl_row *stage_hint, - bool is_discard_route) + bool is_src_route, const uint32_t rtb_id, + const struct ovsdb_idl_row *stage_hint, bool is_discard_route) { bool is_ipv4 = strchr(network_s, '.') ? true : false; struct ds match = DS_EMPTY_INITIALIZER; @@ -9073,8 +9151,8 @@ add_route(struct hmap *lflows, struct ovn_datapath *od, op_inport = op; } } - build_route_match(op_inport, network_s, plen, is_src_route, is_ipv4, - &match, &priority); + build_route_match(op_inport, rtb_id, network_s, plen, is_src_route, + is_ipv4, &match, &priority); struct ds common_actions = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -9137,7 +9215,8 @@ build_static_route_flow(struct hmap *lflows, struct ovn_datapath *od, char *prefix_s = build_route_prefix_s(&route_->prefix, route_->plen); add_route(lflows, route_->is_discard_route ? od : out_port->od, out_port, lrp_addr_s, prefix_s, route_->plen, route->nexthop, - route_->is_src_route, &route->header_, route_->is_discard_route); + route_->is_src_route, route_->route_table_id, &route->header_, + route_->is_discard_route); free(prefix_s); } @@ -10589,6 +10668,17 @@ build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, struct hmap *lflows) } } +/* Logical router ingress table IP_ROUTING_PRE: + * by default goto next. (priority 0). */ +static void +build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, + struct hmap *lflows) +{ + if (od->nbr) { + ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 0, "1", "next;"); + } +} + /* Logical router ingress table IP_ROUTING : IP Routing. * * A packet that arrives at this table is an IP packet that should be @@ -10614,14 +10704,14 @@ build_ip_routing_flows_for_lrouter_port( for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { add_route(lflows, op->od, op, op->lrp_networks.ipv4_addrs[i].addr_s, op->lrp_networks.ipv4_addrs[i].network_s, - op->lrp_networks.ipv4_addrs[i].plen, NULL, false, + op->lrp_networks.ipv4_addrs[i].plen, NULL, false, 0, &op->nbrp->header_, false); } for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { add_route(lflows, op->od, op, op->lrp_networks.ipv6_addrs[i].addr_s, op->lrp_networks.ipv6_addrs[i].network_s, - op->lrp_networks.ipv6_addrs[i].plen, NULL, false, + op->lrp_networks.ipv6_addrs[i].plen, NULL, false, 0, &op->nbrp->header_, false); } } else if (lsp_is_router(op->nbsp)) { @@ -10644,7 +10734,7 @@ build_ip_routing_flows_for_lrouter_port( add_route(lflows, peer->od, peer, peer->lrp_networks.ipv4_addrs[0].addr_s, laddrs->ipv4_addrs[k].network_s, - laddrs->ipv4_addrs[k].plen, NULL, false, + laddrs->ipv4_addrs[k].plen, NULL, false, 0, &peer->nbrp->header_, false); } } @@ -10664,10 +10754,17 @@ build_static_route_flows_for_lrouter( struct hmap ecmp_groups = HMAP_INITIALIZER(&ecmp_groups); struct hmap unique_routes = HMAP_INITIALIZER(&unique_routes); struct ovs_list parsed_routes = OVS_LIST_INITIALIZER(&parsed_routes); + struct simap route_tables = SIMAP_INITIALIZER(&route_tables); struct ecmp_groups_node *group; + + for (int i = 0; i < od->nbr->n_ports; i++) { + build_route_table_lflow(od, lflows, od->nbr->ports[i], + &route_tables); + } + for (int i = 0; i < od->nbr->n_static_routes; i++) { struct parsed_route *route = - parsed_routes_add(od, ports, &parsed_routes, + parsed_routes_add(od, ports, &parsed_routes, &route_tables, od->nbr->static_routes[i], bfd_connections); if (!route) { continue; @@ -10700,6 +10797,7 @@ build_static_route_flows_for_lrouter( ecmp_groups_destroy(&ecmp_groups); unique_routes_destroy(&unique_routes); parsed_routes_destroy(&parsed_routes); + simap_destroy(&route_tables); } } @@ -12804,6 +12902,7 @@ build_lswitch_and_lrouter_iterate_by_od(struct ovn_datapath *od, build_neigh_learning_flows_for_lrouter(od, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); build_ND_RA_flows_for_lrouter(od, lsi->lflows); + build_ip_routing_pre_flows_for_lrouter(od, lsi->lflows); build_static_route_flows_for_lrouter(od, lsi->lflows, lsi->ports, lsi->bfd_connections); build_mcast_lookup_flows_for_lrouter(od, lsi->lflows, &lsi->match, diff --git a/northd/ovn-northd.8.xml b/northd/ovn-northd.8.xml index 39f4eaa0c..cc2e25367 100644 --- a/northd/ovn-northd.8.xml +++ b/northd/ovn-northd.8.xml @@ -2899,7 +2899,7 @@ icmp6 {

If ECMP routes with symmetric reply are configured in the - OVN_Northbound database for a gateway router, a priority-300 + OVN_Northbound database for a gateway router, a priority-400 flow is added for each router port on which symmetric replies are configured. The matching logic for these ports essentially reverses the configured logic of the ECMP route. So for instance, a route with a @@ -3245,7 +3245,35 @@ output; -

Ingress Table 10: IP Routing

+

Ingress Table 10: IP Routing Pre

+ +

+ If a packet arrived at this table from Logical Router Port P + which has options:route_table value set, a logical flow with + match inport == "P" with priority 100 and action, + setting unique-generated per-datapath 32-bit value (non-zero) in OVS + register 7. This register is checked in next table. +

+ +

+ This table contains the following logical flows: +

+ +
    +
  • +

    + Priority-100 flow with match inport == "LRP_NAME" value + and action, which set route table identifier in reg7. +

    + +

    + A priority-0 logical flow with match 1 has actions + next;. +

    +
    • +
+ +

Ingress Table 11: IP Routing

A packet that arrives at this table is an IP packet that should be @@ -3316,10 +3344,10 @@ output;

IPv4 routing table. For each route to IPv4 network N with netmask M, on router port P with IP address - A and Ethernet - address E, a logical flow with match ip4.dst == - N/M, whose priority is the number of - 1-bits in M, has the following actions: + A and Ethernet address E, a logical flow with + match ip4.dst == N/M, whose + priority is 100 + the number of 1-bits in M, has the + following actions: 

@@ -3382,6 +3410,13 @@ next;
           If the address A is in the link-local scope, the
           route will be limited to sending on the ingress port.
         

+
+        

+          For routes with route_table value set
+          reg7 == id is prefixed in logical flow match portion.
+          Priority for routes with route_table value set is
+          the number of 1-bits in M.
+        

       
 
       
  • 
@@ -3408,7 +3443,7 @@ select(reg8[16..31], MID1, MID2, ...);
       
    • 
     
 
-    
    Ingress Table 11: IP_ROUTING_ECMP
    
+    
    Ingress Table 12: IP_ROUTING_ECMP
    
 
     
    
       This table implements the second part of IP routing for ECMP routes
@@ -3460,7 +3495,7 @@ outport = P;
       
     
 
-    
    Ingress Table 12: Router policies
    
+    
    Ingress Table 13: Router policies
    
     
    
       This table adds flows for the logical router policies configured
       on the logical router. Please see the
@@ -3532,7 +3567,7 @@ next;
       
     
 
-    
    Ingress Table 13: ECMP handling for router policies
    
+    
    Ingress Table 14: ECMP handling for router policies
    
     
    
       This table handles the ECMP for the router policies configured
       with multiple nexthops.
@@ -3576,7 +3611,7 @@ outport = P
       
     
 
-    
    Ingress Table 14: ARP/ND Resolution
    
+    
    Ingress Table 15: ARP/ND Resolution
    
 
     
    
       Any packet that reaches this table is an IP packet whose next-hop
@@ -3767,7 +3802,7 @@ outport = P
 
     
 
-    
    Ingress Table 15: Check packet length
    
+    
    Ingress Table 16: Check packet length
    
 
     
    
       For distributed logical routers or gateway routers with gateway
@@ -3797,7 +3832,7 @@ REGBIT_PKT_LARGER = check_pkt_larger(L); next;
       and advances to the next table.
     
    
 
-    
    Ingress Table 16: Handle larger packets
    
+    
    Ingress Table 17: Handle larger packets
    
 
     
    
       For distributed logical routers or gateway routers with gateway port
@@ -3860,7 +3895,7 @@ icmp6 {
       and advances to the next table.
     
    
 
-    
    Ingress Table 17: Gateway Redirect
    
+    
    Ingress Table 18: Gateway Redirect
    
 
     
    
       For distributed logical routers where one or more of the logical router
@@ -3907,7 +3942,7 @@ icmp6 {
       
     
 
-    
    Ingress Table 18: ARP Request
    
+    
    Ingress Table 19: ARP Request
    
 
     
    
       In the common case where the Ethernet destination has been resolved, this
diff --git a/ovn-nb.ovsschema b/ovn-nb.ovsschema
index 2ac8ef3ea..a0a171e19 100644
--- a/ovn-nb.ovsschema
+++ b/ovn-nb.ovsschema
@@ -1,7 +1,7 @@
 {
     "name": "OVN_Northbound",
-    "version": "5.32.1",
-    "cksum": "2805328215 29734",
+    "version": "5.33.1",
+    "cksum": "3874993350 29785",
     "tables": {
         "NB_Global": {
             "columns": {
@@ -387,6 +387,7 @@
             "isRoot": false},
         "Logical_Router_Static_Route": {
             "columns": {
+                "route_table": {"type": "string"},
                 "ip_prefix": {"type": "string"},
                 "policy": {"type": {"key": {"type": "string",
                                             "enum": ["set", ["src-ip",
diff --git a/ovn-nb.xml b/ovn-nb.xml
index 390cc5a44..99fa78c90 100644
--- a/ovn-nb.xml
+++ b/ovn-nb.xml
@@ -2772,6 +2772,14 @@
           prefix according to RFC3663
         
    
       
+
+      
+        Designates lookup Logical_Router_Static_Routes with specified
+        route_table value. Routes to directly connected networks
+        from same Logical Router and routes without route_table
+        option set have higher priority than routes with
+        route_table option set.
+      
     
 
     
@@ -2891,6 +2899,28 @@
       
    
     
 
+    
+      
    
+        Any string to place route to separate routing table. If Logical Router
+        Port has configured value in  other than empty string, OVN
+        performs route lookup for all packets entering Logical Router ingress
+        pipeline from this port in the following manner:
+      
    
+
+      
      
+        
    • 
+          1. First lookup among "global" routes: routes without
+          route_table value set and routes to directly connected
+          networks.
+        
      • 
+        
    • 
+          2. Next lookup among routes with same route_table value
+          as specified in LRP's options:route_table field.
+        
      • 
+      
    
+        
+
     
       ovn-ic populates this key if the route is learned from the
       global  database.  In this case the value
diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at
index 32f4e9d02..3aab54362 100644
--- a/tests/ovn-ic.at
+++ b/tests/ovn-ic.at
@@ -281,6 +281,7 @@ done
 
 AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl
 IPv4 Routes
+Route Table global:
              10.11.1.0/24               169.254.0.1 dst-ip
              10.11.2.0/24             169.254.100.2 dst-ip (learned)
              10.22.1.0/24               169.254.0.2 src-ip
@@ -299,6 +300,7 @@ ovn_as az1 ovn-nbctl set nb_global . options:ic-route-learn=false
 OVS_WAIT_WHILE([ovn_as az1 ovn-nbctl lr-route-list lr1 | grep learned])
 AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl
 IPv4 Routes
+Route Table global:
              10.11.1.0/24               169.254.0.1 dst-ip
              10.22.1.0/24               169.254.0.2 src-ip
 ])
@@ -314,6 +316,7 @@ ovn_as az1 ovn-nbctl set nb_global . options:ic-route-adv=false
 OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned])
 AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl
 IPv4 Routes
+Route Table global:
              10.11.2.0/24               169.254.0.1 dst-ip
              10.22.2.0/24               169.254.0.2 src-ip
 ])
@@ -332,6 +335,7 @@ done
 # Default route should NOT get advertised or learned, by default.
 AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl
 IPv4 Routes
+Route Table global:
              10.11.1.0/24             169.254.100.1 dst-ip (learned)
              10.11.2.0/24               169.254.0.1 dst-ip
              10.22.2.0/24               169.254.0.2 src-ip
diff --git a/tests/ovn-nbctl.at b/tests/ovn-nbctl.at
index 9b80ae410..ddb5536ce 100644
--- a/tests/ovn-nbctl.at
+++ b/tests/ovn-nbctl.at
@@ -1520,6 +1520,7 @@ AT_CHECK([ovn-nbctl --ecmp --policy=src-ip lr-route-add lr0 20.0.0.0/24 11.0.0.1
 
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
               10.0.0.0/24                  11.0.0.1 dst-ip
               10.0.1.0/24                  11.0.1.1 dst-ip lp0
              10.0.10.0/24                           dst-ip lp0
@@ -1534,6 +1535,7 @@ AT_CHECK([ovn-nbctl lrp-add lr0 lp1 f0:00:00:00:00:02 11.0.0.254/24])
 AT_CHECK([ovn-nbctl --may-exist lr-route-add lr0 10.0.0.111/24 11.0.0.1 lp1])
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
               10.0.0.0/24                  11.0.0.1 dst-ip lp1
               10.0.1.0/24                  11.0.1.1 dst-ip lp0
              10.0.10.0/24                           dst-ip lp0
@@ -1564,6 +1566,7 @@ AT_CHECK([ovn-nbctl --policy=src-ip lr-route-del lr0 9.16.1.0/24])
 
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
               10.0.0.0/24                  11.0.0.1 dst-ip lp1
              10.0.10.0/24                           dst-ip lp0
               10.0.0.0/24                  11.0.0.2 src-ip
@@ -1575,6 +1578,7 @@ AT_CHECK([ovn-nbctl --policy=dst-ip lr-route-del lr0 10.0.0.0/24])
 AT_CHECK([ovn-nbctl --policy=src-ip lr-route-del lr0 10.0.0.0/24])
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
              10.0.10.0/24                           dst-ip lp0
                 0.0.0.0/0               192.168.0.1 dst-ip
 ])
@@ -1585,6 +1589,7 @@ AT_CHECK([ovn-nbctl --policy=src-ip lr-route-add lr0 10.0.0.0/24 11.0.0.2])
 AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24])
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
              10.0.10.0/24                           dst-ip lp0
                 0.0.0.0/0               192.168.0.1 dst-ip
 ])
@@ -1601,6 +1606,7 @@ AT_CHECK([ovn-nbctl --ecmp lr-route-add lr0 10.0.0.0/24 11.0.0.3])
 AT_CHECK([ovn-nbctl --ecmp lr-route-add lr0 10.0.0.0/24 11.0.0.4 lp0])
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
               10.0.0.0/24                  11.0.0.1 dst-ip ecmp
               10.0.0.0/24                  11.0.0.2 dst-ip ecmp
               10.0.0.0/24                  11.0.0.3 dst-ip ecmp
@@ -1615,6 +1621,7 @@ dnl Delete ecmp routes
 AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24 11.0.0.1])
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
               10.0.0.0/24                  11.0.0.2 dst-ip ecmp
               10.0.0.0/24                  11.0.0.3 dst-ip ecmp
               10.0.0.0/24                  11.0.0.4 dst-ip lp0 ecmp
@@ -1622,12 +1629,14 @@ IPv4 Routes
 AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24 11.0.0.2])
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
               10.0.0.0/24                  11.0.0.3 dst-ip ecmp
               10.0.0.0/24                  11.0.0.4 dst-ip lp0 ecmp
 ])
 AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24 11.0.0.4 lp0])
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
               10.0.0.0/24                  11.0.0.3 dst-ip
 ])
 AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24 11.0.0.3])
@@ -1641,6 +1650,7 @@ AT_CHECK([ovn-nbctl lr-route-add lr0 2001:0db8:1::/64 2001:0db8:0:f103::1])
 
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv6 Routes
+Route Table global:
             2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
           2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
                      ::/0        2001:db8:0:f101::1 dst-ip
@@ -1650,6 +1660,7 @@ AT_CHECK([ovn-nbctl lr-route-del lr0 2001:0db8:0::/64])
 
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv6 Routes
+Route Table global:
           2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
                      ::/0        2001:db8:0:f101::1 dst-ip
 ])
@@ -1677,11 +1688,13 @@ AT_CHECK([ovn-nbctl --may-exist --ecmp-symmetric-reply lr-route-add lr0 2003:0db
 
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
               10.0.0.0/24                  11.0.0.1 dst-ip
               10.0.1.0/24                  11.0.1.1 dst-ip lp0
                 0.0.0.0/0               192.168.0.1 dst-ip
 
 IPv6 Routes
+Route Table global:
             2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
           2001:db8:1::/64        2001:db8:0:f103::1 dst-ip ecmp
           2001:db8:1::/64        2001:db8:0:f103::2 dst-ip ecmp
@@ -1696,7 +1709,188 @@ AT_CHECK([ovn-nbctl lrp-add lr0 lr0-p0 00:00:01:01:02:03 192.168.10.1/24])
 bfd_uuid=$(ovn-nbctl create bfd logical_port=lr0-p0 dst_ip=100.0.0.50 status=down min_tx=250 min_rx=250 detect_mult=10)
 AT_CHECK([ovn-nbctl lr-route-add lr0 100.0.0.0/24 192.168.0.1])
 route_uuid=$(fetch_column nb:logical_router_static_route _uuid ip_prefix="100.0.0.0/24")
-AT_CHECK([ovn-nbctl set logical_router_static_route $route_uuid bfd=$bfd_uuid])])
+AT_CHECK([ovn-nbctl set logical_router_static_route $route_uuid bfd=$bfd_uuid])
+
+check ovn-nbctl lr-route-del lr0
+AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
+])
+
+dnl Check IPv4 routes in route table
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 0.0.0.0/0 192.168.0.1
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.1.1/24 11.0.1.1 lp0
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.0.1/24 11.0.0.1
+AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
+IPv4 Routes
+Route Table rtb-1:
+              10.0.0.0/24                  11.0.0.1 dst-ip
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+])
+
+check ovn-nbctl lr-route-del lr0
+AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
+])
+
+dnl Check IPv6 routes in route table
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 0:0:0:0:0:0:0:0/0 2001:0db8:0:f101::1
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 2001:0db8:0::/64 2001:0db8:0:f102::1 lp0
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 2001:0db8:1::/64 2001:0db8:0:f103::1
+
+AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
+IPv6 Routes
+Route Table rtb-1:
+            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+])
+
+dnl Check IPv4 and IPv6 routes in route table
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 0.0.0.0/0 192.168.0.1
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.1.1/24 11.0.1.1 lp0
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.0.1/24 11.0.0.1
+
+AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
+IPv4 Routes
+Route Table rtb-1:
+              10.0.0.0/24                  11.0.0.1 dst-ip
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+
+IPv6 Routes
+Route Table rtb-1:
+            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+])
+
+# Add routes in another route table
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 0.0.0.0/0 192.168.0.1
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 10.0.1.1/24 11.0.1.1 lp0
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 10.0.0.1/24 11.0.0.1
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 0:0:0:0:0:0:0:0/0 2001:0db8:0:f101::1
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 2001:0db8:0::/64 2001:0db8:0:f102::1 lp0
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 2001:0db8:1::/64 2001:0db8:0:f103::1
+
+AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
+IPv4 Routes
+Route Table rtb-1:
+              10.0.0.0/24                  11.0.0.1 dst-ip
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+
+Route Table rtb-2:
+              10.0.0.0/24                  11.0.0.1 dst-ip
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+
+IPv6 Routes
+Route Table rtb-1:
+            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+
+Route Table rtb-2:
+            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+])
+
+# Add routes to global route table
+check ovn-nbctl lr-route-add lr0 0.0.0.0/0 192.168.0.1
+check ovn-nbctl lr-route-add lr0 10.0.1.1/24 11.0.1.1 lp0
+check ovn-nbctl lr-route-add lr0 10.0.0.1/24 11.0.0.1
+check ovn-nbctl lr-route-add lr0 0:0:0:0:0:0:0:0/0 2001:0db8:0:f101::1
+check ovn-nbctl lr-route-add lr0 2001:0db8:0::/64 2001:0db8:0:f102::1 lp0
+check check ovn-nbctl lr-route-add lr0 2001:0db8:1::/64 2001:0db8:0:f103::1
+
+AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
+IPv4 Routes
+Route Table global:
+              10.0.0.0/24                  11.0.0.1 dst-ip
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+
+Route Table rtb-1:
+              10.0.0.0/24                  11.0.0.1 dst-ip
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+
+Route Table rtb-2:
+              10.0.0.0/24                  11.0.0.1 dst-ip
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+
+IPv6 Routes
+Route Table global:
+            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+
+Route Table rtb-1:
+            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+
+Route Table rtb-2:
+            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+])
+
+# delete IPv4 route from rtb-1
+check ovn-nbctl --route-table=rtb-1 lr-route-del lr0 10.0.0.0/24
+AT_CHECK([ovn-nbctl --route-table=rtb-1 lr-route-list lr0], [0], [dnl
+IPv4 Routes
+Route Table rtb-1:
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+
+IPv6 Routes
+Route Table rtb-1:
+            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+])
+
+# delete IPv6 route from rtb-2
+check ovn-nbctl --route-table=rtb-2 lr-route-del lr0 2001:db8::/64
+AT_CHECK([ovn-nbctl --route-table=rtb-2 lr-route-list lr0], [0], [dnl
+IPv4 Routes
+Route Table rtb-2:
+              10.0.0.0/24                  11.0.0.1 dst-ip
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+
+IPv6 Routes
+Route Table rtb-2:
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+])
+
+check ovn-nbctl lr-route-del lr0
+
+# ECMP route in route table
+check ovn-nbctl --route-table=rtb1 lr-route-add lr0 0.0.0.0/0 192.168.0.1
+check ovn-nbctl --ecmp --route-table=rtb1 lr-route-add lr0 0.0.0.0/0 192.168.0.2
+
+# Negative route table case: same prefix
+AT_CHECK([ovn-nbctl --route-table=rtb1 lr-route-add lr0 0.0.0.0/0 192.168.0.1], [1], [], [dnl
+ovn-nbctl: duplicate prefix: 0.0.0.0/0 (policy: dst-ip). Use option --ecmp to allow this for ECMP routing.
+])
+
+# Negative route table case: same prefix & nexthop with ecmp
+AT_CHECK([ovn-nbctl --ecmp --route-table=rtb1 lr-route-add lr0 0.0.0.0/0 192.168.0.2], [1], [], [dnl
+ovn-nbctl: duplicate nexthop for the same ECMP route
+])
+
+# Add routes to global route table
+check ovn-nbctl lrp-add lr0 lrp0 00:00:00:00:00:01 1.1.1.1/24
+check ovn-nbctl lrp-set-options lrp0 route_table=rtb1
+AT_CHECK([ovn-nbctl get logical-router-port lrp0 options:route_table], [0], [dnl
+rtb1
+])
+check `ovn-nbctl show lr0 | grep lrp0 -A3 | grep route_table=rtb1`
+])
 
 dnl ---------------------------------------------------------------------
 
diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
index 5de554455..e6037b045 100644
--- a/tests/ovn-northd.at
+++ b/tests/ovn-northd.at
@@ -5102,7 +5102,7 @@ check ovn-nbctl --wait=sb --ecmp-symmetric-reply lr-route-add lr0 1.0.0.1 192.16
 
 ovn-sbctl dump-flows lr0 > lr0flows
 AT_CHECK([grep -e "lr_in_ip_routing.*select" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl
-  table=??(lr_in_ip_routing   ), priority=65   , match=(ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);)
+  table=??(lr_in_ip_routing   ), priority=165  , match=(ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);)
 ])
 AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed 's/192\.168\.0\..0/192.168.0.??/' | sed 's/table=../table=??/' | sort], [0], [dnl
   table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && reg8[[16..31]] == 1), action=(reg0 = 192.168.0.??; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; next;)
@@ -5115,7 +5115,7 @@ check ovn-nbctl --wait=sb --ecmp-symmetric-reply lr-route-add lr0 1.0.0.1 192.16
 
 ovn-sbctl dump-flows lr0 > lr0flows
 AT_CHECK([grep -e "lr_in_ip_routing.*select" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl
-  table=??(lr_in_ip_routing   ), priority=65   , match=(ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);)
+  table=??(lr_in_ip_routing   ), priority=165  , match=(ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);)
 ])
 AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed 's/192\.168\.0\..0/192.168.0.??/' | sed 's/table=../table=??/' | sort], [0], [dnl
   table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && reg8[[16..31]] == 1), action=(reg0 = 192.168.0.??; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; next;)
@@ -5130,14 +5130,14 @@ check ovn-nbctl --wait=sb lr-route-add lr0 1.0.0.0/24 192.168.0.10
 ovn-sbctl dump-flows lr0 > lr0flows
 
 AT_CHECK([grep -e "lr_in_ip_routing.*192.168.0.10" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl
-  table=??(lr_in_ip_routing   ), priority=49   , match=(ip4.dst == 1.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=149  , match=(ip4.dst == 1.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;)
 ])
 
 check ovn-nbctl --wait=sb lr-route-add lr0 2.0.0.0/24 lr0-public
 
 ovn-sbctl dump-flows lr0 > lr0flows
 AT_CHECK([grep -e "lr_in_ip_routing.*2.0.0.0" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl
-  table=??(lr_in_ip_routing   ), priority=49   , match=(ip4.dst == 2.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=149  , match=(ip4.dst == 2.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;)
 ])
 
 AT_CLEANUP
@@ -5223,3 +5223,71 @@ AT_CHECK([grep lr_in_gw_redirect lrflows | grep cr-DR | sed 's/table=../table=??
 
 AT_CLEANUP
 ])
+
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([route tables -- flows])
+AT_KEYWORDS([route-tables-flows])
+ovn_start
+
+check ovn-nbctl lr-add lr0
+check ovn-nbctl lrp-add lr0 lrp0 00:00:00:00:00:01 192.168.0.1/24
+check ovn-nbctl lrp-add lr0 lrp1 00:00:00:00:01:01 192.168.1.1/24
+check ovn-nbctl lrp-add lr0 lrp2 00:00:00:00:02:01 192.168.2.1/24
+check ovn-nbctl lrp-set-options lrp1 route_table=rtb-1
+check ovn-nbctl lrp-set-options lrp2 route_table=rtb-2
+
+check ovn-nbctl lr-route-add lr0 0.0.0.0/0 192.168.0.10
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 192.168.0.0/24 192.168.1.10
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 0.0.0.0/0 192.168.0.10
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 1.1.1.1/32 192.168.0.20
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 2.2.2.2/32 192.168.0.30
+check ovn-nbctl --route-table=rtb-2 --ecmp lr-route-add lr0 2.2.2.2/32 192.168.0.31
+check ovn-nbctl --wait=sb sync
+
+ovn-sbctl dump-flows lr0 > lr0flows
+AT_CAPTURE_FILE([lr0flows])
+
+AT_CHECK([grep -e "lr_in_ip_routing_pre.*match=(1)" lr0flows | sed 's/table=../table=??/'], [0], [dnl
+  table=??(lr_in_ip_routing_pre), priority=0    , match=(1), action=(next;)
+])
+
+p1_reg=$(grep -oP "lr_in_ip_routing_pre.*lrp1.*action=\(reg7 = \K." lr0flows)
+p2_reg=$(grep -oP "lr_in_ip_routing_pre.*lrp2.*action=\(reg7 = \K." lr0flows)
+echo $p1_reg
+echo $p2_reg
+
+# exact register values are not predictable
+if [[ $p1_reg -eq 2 ] && [ $p2_reg -eq 1 ]]; then
+  echo "swap reg values in dump"
+  sed -i -r s'/^(.*lrp2.*action=\(reg7 = )(1)(.*)/\12\3/g' lr0flows  # "reg7 = 1" -> "reg7 = 2"
+  sed -i -r s'/^(.*lrp1.*action=\(reg7 = )(2)(.*)/\11\3/g' lr0flows  # "reg7 = 2" -> "reg7 = 1"
+  sed -i -r s'/^(.*match=\(reg7 == )(2)( &&.*lrp1.*)/\11\3/g' lr0flows  # "reg7 == 2" -> "reg7 == 1"
+  sed -i -r s'/^(.*match=\(reg7 == )(1)( &&.*lrp0.*)/\12\3/g' lr0flows  # "reg7 == 1" -> "reg7 == 2"
+fi
+
+check test "$p1_reg" != "$p2_reg" -a $((p1_reg * p2_reg)) -eq 2
+
+AT_CHECK([grep "lr_in_ip_routing_pre" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl
+  table=??(lr_in_ip_routing_pre), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_ip_routing_pre), priority=100  , match=(inport == "lrp1"), action=(reg7 = 1; next;)
+  table=??(lr_in_ip_routing_pre), priority=100  , match=(inport == "lrp2"), action=(reg7 = 2; next;)
+])
+
+grep -e "(lr_in_ip_routing   ).*outport" lr0flows
+
+AT_CHECK([grep -e "(lr_in_ip_routing   ).*outport" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl
+  table=??(lr_in_ip_routing   ), priority=1    , match=(reg7 == 2 && ip4.dst == 0.0.0.0/0), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:00:00:00:01; outport = "lrp0"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=101  , match=(ip4.dst == 0.0.0.0/0), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:00:00:00:01; outport = "lrp0"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=149  , match=(ip4.dst == 192.168.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.0.1; eth.src = 00:00:00:00:00:01; outport = "lrp0"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=149  , match=(ip4.dst == 192.168.1.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.1.1; eth.src = 00:00:00:00:01:01; outport = "lrp1"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=149  , match=(ip4.dst == 192.168.2.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.2.1; eth.src = 00:00:00:00:02:01; outport = "lrp2"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=229  , match=(inport == "lrp0" && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:1; eth.src = 00:00:00:00:00:01; outport = "lrp0"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=229  , match=(inport == "lrp1" && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:101; eth.src = 00:00:00:00:01:01; outport = "lrp1"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=229  , match=(inport == "lrp2" && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:201; eth.src = 00:00:00:00:02:01; outport = "lrp2"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=49   , match=(reg7 == 1 && ip4.dst == 192.168.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.1.10; reg1 = 192.168.1.1; eth.src = 00:00:00:00:01:01; outport = "lrp1"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=65   , match=(reg7 == 2 && ip4.dst == 1.1.1.1/32), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.20; reg1 = 192.168.0.1; eth.src = 00:00:00:00:00:01; outport = "lrp0"; flags.loopback = 1; next;)
+])
+
+AT_CLEANUP
+])
diff --git a/tests/ovn.at b/tests/ovn.at
index d76f4ba86..8ca3bb222 100644
--- a/tests/ovn.at
+++ b/tests/ovn.at
@@ -18079,7 +18079,7 @@ eth_dst=00000000ff01
 ip_src=$(ip_to_hex 10 0 0 10)
 ip_dst=$(ip_to_hex 172 168 0 101)
 send_icmp_packet 1 1 $eth_src $eth_dst $ip_src $ip_dst c4c9 0000000000000000000000
-AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | awk '/table=25, n_packets=1, n_bytes=45/{print $7" "$8}'],[0],[dnl
+AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | awk '/table=26, n_packets=1, n_bytes=45/{print $7" "$8}'],[0],[dnl
 priority=80,ip,reg15=0x3,metadata=0x3,nw_src=10.0.0.10 actions=drop
 ])
 
@@ -22263,6 +22263,433 @@ OVN_CLEANUP([hv1])
 AT_CLEANUP
 ])
 
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([route tables -- global routes])
+ovn_start
+
+# Logical network:
+# ls1 (192.168.1.0/24) - lrp-lr1-ls1 - lr1 - lrp-lr1-ls2 - ls2 (192.168.2.0/24)
+#
+# ls1 has lsp11 (192.168.1.11) and ls2 has lsp21 (192.168.2.21) and lsp22
+# (192.168.2.22)
+#
+# lrp-lr1-ls1 set options:route_table=rtb-1
+#
+# Static routes on lr1:
+# 0.0.0.0/0 nexthop 192.168.2.21
+# 1.1.1.1/32 nexthop 192.168.2.22 route_table=rtb-1
+#
+# Test 1:
+# lsp11 send packet to 2.2.2.2
+#
+# Expected result:
+# lsp21 should receive traffic, lsp22 should not receive any traffic
+#
+# Test 2:
+# lsp11 send packet to 1.1.1.1
+#
+# Expected result:
+# lsp21 should receive traffic, lsp22 should not receive any traffic
+
+ovn-nbctl lr-add lr1
+
+for i in 1 2; do
+    ovn-nbctl ls-add ls${i}
+    ovn-nbctl lrp-add lr1 lrp-lr1-ls${i} 00:00:00:01:0${i}:01 192.168.${i}.1/24
+    ovn-nbctl lsp-add ls${i} lsp-ls${i}-lr1 -- lsp-set-type lsp-ls${i}-lr1 router \
+        -- lsp-set-options lsp-ls${i}-lr1 router-port=lrp-lr1-ls${i} \
+        -- lsp-set-addresses lsp-ls${i}-lr1 router
+done
+
+# install static routes
+ovn-nbctl lr-route-add lr1 0.0.0.0/0 192.168.2.21
+ovn-nbctl --route-table=rtb-1 lr-route-add lr1 1.1.1.1/32 192.168.2.22
+
+# set lrp-lr1-ls1 route table
+ovn-nbctl lrp-set-options lrp-lr1-ls1 route_table=rtb-1
+
+# Create logical ports
+ovn-nbctl lsp-add ls1 lsp11 -- \
+    lsp-set-addresses lsp11 "f0:00:00:00:01:11 192.168.1.11"
+ovn-nbctl lsp-add ls2 lsp21 -- \
+    lsp-set-addresses lsp21 "f0:00:00:00:02:21 192.168.2.21"
+ovn-nbctl lsp-add ls2 lsp22 -- \
+    lsp-set-addresses lsp22 "f0:00:00:00:02:22 192.168.2.22"
+
+net_add n1
+sim_add hv1
+as hv1
+ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.1
+ovs-vsctl -- add-port br-int hv1-vif1 -- \
+    set interface hv1-vif1 external-ids:iface-id=lsp11 \
+    options:tx_pcap=hv1/vif1-tx.pcap \
+    options:rxq_pcap=hv1/vif1-rx.pcap \
+    ofport-request=1
+
+ovs-vsctl -- add-port br-int hv1-vif2 -- \
+    set interface hv1-vif2 external-ids:iface-id=lsp21 \
+    options:tx_pcap=hv1/vif2-tx.pcap \
+    options:rxq_pcap=hv1/vif2-rx.pcap \
+    ofport-request=2
+
+ovs-vsctl -- add-port br-int hv1-vif3 -- \
+    set interface hv1-vif3 external-ids:iface-id=lsp22 \
+    options:tx_pcap=hv1/vif3-tx.pcap \
+    options:rxq_pcap=hv1/vif3-rx.pcap \
+    ofport-request=3
+
+# wait for earlier changes to take effect
+check ovn-nbctl --wait=hv sync
+wait_for_ports_up
+
+for i in 1 2; do
+    packet="inport==\"lsp11\" && eth.src==f0:00:00:00:01:11 && eth.dst==00:00:00:01:01:01 &&
+            ip4 && ip.ttl==64 && ip4.src==192.168.1.11 && ip4.dst==$i.$i.$i.$i && icmp"
+    AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"])
+
+    # Assume all packets go to lsp21.
+    exp_packet="eth.src==00:00:00:01:02:01 && eth.dst==f0:00:00:00:02:21 &&
+            ip4 && ip.ttl==63 && ip4.src==192.168.1.11 && ip4.dst==$i.$i.$i.$i && icmp"
+    echo $exp_packet | ovstest test-ovn expr-to-packets >> expected_lsp21
+done
+> expected_lsp22
+
+# lsp21 should recieve 2 packets and lsp22 should recieve no packets
+OVS_WAIT_UNTIL([
+    rcv_n1=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > lsp21.packets && cat lsp21.packets | wc -l`
+    rcv_n2=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif3-tx.pcap > lsp22.packets && cat lsp22.packets | wc -l`
+    echo $rcv_n1 $rcv_n2
+    test $rcv_n1 -eq 2 -a $rcv_n2 -eq 0])
+
+for i in 1 2; do
+    sort expected_lsp2$i > expout
+    AT_CHECK([cat lsp2${i}.packets | sort], [0], [expout])
+done
+
+OVN_CLEANUP([hv1])
+AT_CLEANUP
+])
+
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([route tables -- directly connected routes])
+ovn_start
+
+# Logical network:
+# ls1 (192.168.1.0/24) - lrp-lr1-ls1 - lr1 - lrp-lr1-ls2 - ls2 (192.168.2.0/24)
+#
+# ls1 has lsp11 (192.168.1.11) and ls2 has lsp21 (192.168.2.21)
+#
+# lrp-lr1-ls1 set options:route_table=rtb-1
+#
+# Static routes on lr1:
+# 192.168.2.0/25 nexthop 192.168.1.11 route_table=rtb-1
+#
+# Test 1:
+# lsp11 send packet to 192.168.2.21
+#
+# Expected result:
+# lsp21 should receive traffic, lsp11 should not receive any traffic
+
+ovn-nbctl lr-add lr1
+
+for i in 1 2; do
+    ovn-nbctl ls-add ls${i}
+    ovn-nbctl lrp-add lr1 lrp-lr1-ls${i} 00:00:00:01:0${i}:01 192.168.${i}.1/24
+    ovn-nbctl lsp-add ls${i} lsp-ls${i}-lr1 -- lsp-set-type lsp-ls${i}-lr1 router \
+        -- lsp-set-options lsp-ls${i}-lr1 router-port=lrp-lr1-ls${i} \
+        -- lsp-set-addresses lsp-ls${i}-lr1 router
+done
+
+# install static route, which overrides directly-connected routes
+ovn-nbctl --route-table=rtb-1 lr-route-add lr1 192.168.2.0/25 192.168.1.11
+
+# set lrp-lr1-ls1 route table
+ovn-nbctl lrp-set-options lrp-lr1-ls1 route_table=rtb-1
+
+# Create logical ports
+ovn-nbctl lsp-add ls1 lsp11 -- \
+    lsp-set-addresses lsp11 "f0:00:00:00:01:11 192.168.1.11"
+ovn-nbctl lsp-add ls2 lsp21 -- \
+    lsp-set-addresses lsp21 "f0:00:00:00:02:21 192.168.2.21"
+
+net_add n1
+sim_add hv1
+as hv1
+ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.1
+ovs-vsctl -- add-port br-int hv1-vif1 -- \
+    set interface hv1-vif1 external-ids:iface-id=lsp11 \
+    options:tx_pcap=hv1/vif1-tx.pcap \
+    options:rxq_pcap=hv1/vif1-rx.pcap \
+    ofport-request=1
+
+ovs-vsctl -- add-port br-int hv1-vif2 -- \
+    set interface hv1-vif2 external-ids:iface-id=lsp21 \
+    options:tx_pcap=hv1/vif2-tx.pcap \
+    options:rxq_pcap=hv1/vif2-rx.pcap \
+    ofport-request=2
+
+# wait for earlier changes to take effect
+check ovn-nbctl --wait=hv sync
+wait_for_ports_up
+
+packet="inport==\"lsp11\" && eth.src==f0:00:00:00:01:11 && eth.dst==00:00:00:01:01:01 &&
+        ip4 && ip.ttl==64 && ip4.src==192.168.1.11 && ip4.dst==192.168.2.21 && icmp"
+AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"])
+
+# Assume all packets go to lsp21.
+exp_packet="eth.src==00:00:00:01:02:01 && eth.dst==f0:00:00:00:02:21 &&
+        ip4 && ip.ttl==63 && ip4.src==192.168.1.11 && ip4.dst==192.168.2.21 && icmp"
+echo $exp_packet | ovstest test-ovn expr-to-packets >> expected_lsp21
+> expected_lsp11
+
+# lsp21 should recieve 1 icmp packet and lsp11 should recieve no packets
+OVS_WAIT_UNTIL([
+    rcv_n11=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif1-tx.pcap > lsp11.packets && cat lsp11.packets | wc -l`
+    rcv_n21=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > lsp21.packets && cat lsp21.packets | wc -l`
+    echo $rcv_n11 $rcv_n21
+    test $rcv_n11 -eq 0 -a $rcv_n21 -eq 1])
+
+for i in 11 21; do
+    sort expected_lsp$i > expout
+    AT_CHECK([cat lsp${i}.packets | sort], [0], [expout])
+done
+
+OVN_CLEANUP([hv1])
+AT_CLEANUP
+])
+
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([route tables -- overlapping subnets])
+ovn_start
+
+# Logical network:
+#
+# ls1 (192.168.1.0/24) - lrp-lr1-ls1 -\   /- lrp-lr1-ls2 - ls2 (192.168.2.0/24)
+#                                      lr1
+# ls3 (192.168.3.0/24) - lrp-lr1-ls3 -/   \- lrp-lr1-ls4 - ls4 (192.168.4.0/24)
+#
+# ls1 has lsp11 (192.168.1.11)
+# ls2 has lsp21 (192.168.2.21)
+# ls3 has lsp31 (192.168.3.31)
+# ls4 has lsp41 (192.168.4.41)
+#
+# lrp-lr1-ls1 set options:route_table=rtb-1
+# lrp-lr1-ls2 set options:route_table=rtb-2
+#
+# Static routes on lr1:
+# 10.0.0.0/24 nexthop 192.168.3.31 route_table=rtb-1
+# 10.0.0.0/24 nexthop 192.168.4.41 route_table=rtb-2
+#
+# Test 1:
+# lsp11 send packet to 10.0.0.1
+#
+# Expected result:
+# lsp31 should receive traffic, lsp41 should not receive any traffic
+#
+# Test 2:
+# lsp21 send packet to 10.0.0.1
+#
+# Expected result:
+# lsp41 should receive traffic, lsp31 should not receive any traffic
+
+ovn-nbctl lr-add lr1
+
+# Create logical topology
+for i in $(seq 1 4); do
+    ovn-nbctl ls-add ls${i}
+    ovn-nbctl lrp-add lr1 lrp-lr1-ls${i} 00:00:00:01:0${i}:01 192.168.${i}.1/24
+    ovn-nbctl lsp-add ls${i} lsp-ls${i}-lr1 -- lsp-set-type lsp-ls${i}-lr1 router \
+        -- lsp-set-options lsp-ls${i}-lr1 router-port=lrp-lr1-ls${i} \
+        -- lsp-set-addresses lsp-ls${i}-lr1 router
+    ovn-nbctl lsp-add ls$i lsp${i}1 -- \
+        lsp-set-addresses lsp${i}1 "f0:00:00:00:0${i}:1${i} 192.168.${i}.${i}1"
+done
+
+# install static routes
+ovn-nbctl --route-table=rtb-1 lr-route-add lr1 10.0.0.0/24 192.168.3.31
+ovn-nbctl --route-table=rtb-2 lr-route-add lr1 10.0.0.0/24 192.168.4.41
+
+# set lrp-lr1-ls{1,2} route tables
+ovn-nbctl lrp-set-options lrp-lr1-ls1 route_table=rtb-1
+ovn-nbctl lrp-set-options lrp-lr1-ls2 route_table=rtb-2
+
+net_add n1
+sim_add hv1
+as hv1
+ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.1
+
+for i in $(seq 1 4); do
+    ovs-vsctl -- add-port br-int hv1-vif${i} -- \
+        set interface hv1-vif${i} external-ids:iface-id=lsp${i}1 \
+        options:tx_pcap=hv1/vif${i}-tx.pcap \
+        options:rxq_pcap=hv1/vif${i}-rx.pcap \
+        ofport-request=${i}
+done
+
+# wait for earlier changes to take effect
+check ovn-nbctl --wait=hv sync
+wait_for_ports_up
+
+# lsp31 should recieve packet coming from lsp11
+# lsp41 should recieve packet coming from lsp21
+for i in $(seq 1 2); do
+    di=$(( i + 2))  # dst index
+    ri=$(( 5 - i))  # reverse index
+    packet="inport==\"lsp${i}1\" && eth.src==f0:00:00:00:0${i}:1${i} &&
+            eth.dst==00:00:00:01:0${i}:01 && ip4 && ip.ttl==64 &&
+            ip4.src==192.168.${i}.${i}1 && ip4.dst==10.0.0.1 && icmp"
+    AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"])
+
+    # Assume all packets go to lsp${di}1.
+    exp_packet="eth.src==00:00:00:01:0${di}:01 && eth.dst==f0:00:00:00:0${di}:1${di} &&
+            ip4 && ip.ttl==63 && ip4.src==192.168.${i}.${i}1 && ip4.dst==10.0.0.1 && icmp"
+    echo $exp_packet | ovstest test-ovn expr-to-packets >> expected_lsp${di}1
+    > expected_lsp${ri}1
+
+    OVS_WAIT_UNTIL([
+        rcv_n1=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif${di}-tx.pcap > lsp${di}1.packets && cat lsp${di}1.packets | wc -l`
+        rcv_n2=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif${ri}-tx.pcap > lsp${ri}1.packets && cat lsp${ri}1.packets | wc -l`
+        echo $rcv_n1 $rcv_n2
+        test $rcv_n1 -eq 1 -a $rcv_n2 -eq 0])
+
+    for j in "${di}1" "${ri}1"; do
+        sort expected_lsp${j} > expout
+        AT_CHECK([cat lsp${j}.packets | sort], [0], [expout])
+    done
+
+    # cleanup tx pcap files
+    for j in "${di}1" "${ri}1"; do
+        ovs-vsctl -- remove interface hv1-vif${di} options tx_pcap
+        > hv1/vif${di}-tx.pcap
+        ovs-vsctl -- set interface hv1-vif${di} external-ids:iface-id=lsp${di}1 \
+            options:tx_pcap=hv1/vif${di}-tx.pcap
+    done
+done
+
+OVN_CLEANUP([hv1])
+AT_CLEANUP
+])
+
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([route tables IPv6 -- overlapping subnets])
+ovn_start
+
+# Logical network:
+#
+# ls1 (2001:db8:1::/64) - lrp-lr1-ls1 -\   /- lrp-lr1-ls2 - ls2 (2001:db8:2::/64)
+#                                       lr1
+# ls3 (2001:db8:3::/64) - lrp-lr1-ls3 -/   \- lrp-lr1-ls4 - ls4 (2001:db8:4::/64)
+#
+# ls1 has lsp11 (2001:db8:1::11)
+# ls2 has lsp21 (2001:db8:2::21)
+# ls3 has lsp31 (2001:db8:3::31)
+# ls4 has lsp41 (2001:db8:4::41)
+#
+# lrp-lr1-ls1 set options:route_table=rtb-1
+# lrp-lr1-ls2 set options:route_table=rtb-2
+#
+# Static routes on lr1:
+# 2001:db8:2000::/64 nexthop 2001:db8:3::31 route_table=rtb-1
+# 2001:db8:2000::/64 nexthop 2001:db8:3::41 route_table=rtb-2
+#
+# Test 1:
+# lsp11 send packet to 2001:db8:2000::1
+#
+# Expected result:
+# lsp31 should receive traffic, lsp41 should not receive any traffic
+#
+# Test 2:
+# lsp21 send packet to 2001:db8:2000::1
+#
+# Expected result:
+# lsp41 should receive traffic, lsp31 should not receive any traffic
+
+ovn-nbctl lr-add lr1
+
+# Create logical topology
+for i in $(seq 1 4); do
+    ovn-nbctl ls-add ls${i}
+    ovn-nbctl lrp-add lr1 lrp-lr1-ls${i} 00:00:00:01:0${i}:01 2001:db8:${i}::1/64
+    ovn-nbctl lsp-add ls${i} lsp-ls${i}-lr1 -- lsp-set-type lsp-ls${i}-lr1 router \
+        -- lsp-set-options lsp-ls${i}-lr1 router-port=lrp-lr1-ls${i} \
+        -- lsp-set-addresses lsp-ls${i}-lr1 router
+    ovn-nbctl lsp-add ls$i lsp${i}1 -- \
+        lsp-set-addresses lsp${i}1 "f0:00:00:00:0${i}:1${i} 2001:db8:${i}::${i}1"
+done
+
+# install static routes
+ovn-nbctl --route-table=rtb-1 lr-route-add lr1 2001:db8:2000::/64 2001:db8:3::31
+ovn-nbctl --route-table=rtb-2 lr-route-add lr1 2001:db8:2000::/64 2001:db8:4::41
+
+# set lrp-lr1-ls{1,2} route tables
+ovn-nbctl lrp-set-options lrp-lr1-ls1 route_table=rtb-1
+ovn-nbctl lrp-set-options lrp-lr1-ls2 route_table=rtb-2
+
+net_add n1
+sim_add hv1
+as hv1
+ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.1
+
+for i in $(seq 1 4); do
+    ovs-vsctl -- add-port br-int hv1-vif${i} -- \
+        set interface hv1-vif${i} external-ids:iface-id=lsp${i}1 \
+        options:tx_pcap=hv1/vif${i}-tx.pcap \
+        options:rxq_pcap=hv1/vif${i}-rx.pcap \
+        ofport-request=${i}
+done
+
+# wait for earlier changes to take effect
+AT_CHECK([ovn-nbctl --timeout=3 --wait=hv sync], [0], [ignore])
+
+# lsp31 should recieve packet coming from lsp11
+# lsp41 should recieve packet coming from lsp21
+for i in $(seq 1 2); do
+    di=$(( i + 2))  # dst index
+    ri=$(( 5 - i))  # reverse index
+    packet="inport==\"lsp${i}1\" && eth.src==f0:00:00:00:0${i}:1${i} &&
+            eth.dst==00:00:00:01:0${i}:01 && ip6 && ip.ttl==64 &&
+            ip6.src==2001:db8:${i}::${i}1 && ip6.dst==2001:db8:2000::1 && icmp6"
+    AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"])
+
+    # Assume all packets go to lsp${di}1.
+    exp_packet="eth.src==00:00:00:01:0${di}:01 && eth.dst==f0:00:00:00:0${di}:1${di} && ip6 &&
+                ip.ttl==63 && ip6.src==2001:db8:${i}::${i}1 && ip6.dst==2001:db8:2000::1 && icmp6"
+    echo $exp_packet | ovstest test-ovn expr-to-packets >> expected_lsp${di}1
+    > expected_lsp${ri}1
+
+    OVS_WAIT_UNTIL([
+        rcv_n1=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif${di}-tx.pcap > lsp${di}1.packets && cat lsp${di}1.packets | wc -l`
+        rcv_n2=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif${ri}-tx.pcap > lsp${ri}1.packets && cat lsp${ri}1.packets | wc -l`
+        echo $rcv_n1 $rcv_n2
+        test $rcv_n1 -eq 1 -a $rcv_n2 -eq 0])
+
+    for j in "${di}1" "${ri}1"; do
+        sort expected_lsp${j} > expout
+        AT_CHECK([cat lsp${j}.packets | sort], [0], [expout])
+    done
+
+    # cleanup tx pcap files
+    for j in "${di}1" "${ri}1"; do
+        ovs-vsctl -- remove interface hv1-vif${di} options tx_pcap
+        > hv1/vif${di}-tx.pcap
+        ovs-vsctl -- set interface hv1-vif${di} external-ids:iface-id=lsp${di}1 \
+            options:tx_pcap=hv1/vif${di}-tx.pcap
+    done
+done
+
+OVN_CLEANUP([hv1])
+AT_CLEANUP
+])
+
+
 OVN_FOR_EACH_NORTHD([
 AT_SETUP([forwarding group: 3 HVs, 1 LR, 2 LS])
 AT_KEYWORDS([forwarding-group])
@@ -23018,7 +23445,7 @@ ovn-sbctl dump-flows > sbflows
 AT_CAPTURE_FILE([sbflows])
 AT_CAPTURE_FILE([offlows])
 OVS_WAIT_UNTIL([
-    as hv1 ovs-ofctl dump-flows br-int table=20 > offlows
+    as hv1 ovs-ofctl dump-flows br-int table=21 > offlows
     test $(grep -c "load:0x64->NXM_NX_PKT_MARK" offlows) = 1 && \
     test $(grep -c "load:0x3->NXM_NX_PKT_MARK" offlows) = 1 && \
     test $(grep -c "load:0x4->NXM_NX_PKT_MARK" offlows) = 1 && \
@@ -23111,12 +23538,12 @@ send_ipv4_pkt hv1 hv1-vif1 505400000003 00000000ff01 \
     $(ip_to_hex 10 0 0 3) $(ip_to_hex 172 168 0 120)
 
 OVS_WAIT_UNTIL([
-    test 1 -eq $(as hv1 ovs-ofctl dump-flows br-int table=20 | \
+    test 1 -eq $(as hv1 ovs-ofctl dump-flows br-int table=21 | \
     grep "load:0x2->NXM_NX_PKT_MARK" -c)
 ])
 
 AT_CHECK([
-    test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=20 | \
+    test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=21 | \
     grep "load:0x64->NXM_NX_PKT_MARK" -c)
 ])
 
@@ -23819,7 +24246,7 @@ AT_CHECK([
         grep "priority=100" | \
         grep -c "ct(commit,zone=NXM_NX_REG11\\[[0..15\\]],.*exec(move:NXM_OF_ETH_SRC\\[[\\]]->NXM_NX_CT_LABEL\\[[32..79\\]],load:0x[[0-9]]->NXM_NX_CT_LABEL\\[[80..95\\]]))"
 
-        grep table=22 hv${hv}flows | \
+        grep table=23 hv${hv}flows | \
         grep "priority=200" | \
         grep -c "actions=move:NXM_NX_CT_LABEL\\[[32..79\\]]->NXM_OF_ETH_DST\\[[\\]]"
     done; :], [0], [dnl
@@ -23944,7 +24371,7 @@ AT_CHECK([
         grep "priority=100" | \
         grep -c "ct(commit,zone=NXM_NX_REG11\\[[0..15\\]],.*exec(move:NXM_OF_ETH_SRC\\[[\\]]->NXM_NX_CT_LABEL\\[[32..79\\]],load:0x[[0-9]]->NXM_NX_CT_LABEL\\[[80..95\\]]))"
 
-        grep table=22 hv${hv}flows | \
+        grep table=23 hv${hv}flows | \
         grep "priority=200" | \
         grep -c "actions=move:NXM_NX_CT_LABEL\\[[32..79\\]]->NXM_OF_ETH_DST\\[[\\]]"
     done; :], [0], [dnl
@@ -24566,7 +24993,7 @@ AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep "actions=controller" | grep
 ])
 
 # The packet should've been dropped in the lr_in_arp_resolve stage.
-AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep -E "table=22, n_packets=1,.* priority=1,ip,metadata=0x${sw_key},nw_dst=10.0.1.1 actions=drop" -c], [0], [dnl
+AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep -E "table=23, n_packets=1,.* priority=1,ip,metadata=0x${sw_key},nw_dst=10.0.1.1 actions=drop" -c], [0], [dnl
 1
 ])
 
diff --git a/utilities/ovn-nbctl.c b/utilities/ovn-nbctl.c
index 10217dcd5..f4a8504e6 100644
--- a/utilities/ovn-nbctl.c
+++ b/utilities/ovn-nbctl.c
@@ -329,6 +329,8 @@ Logical router port commands:\n\
                             add logical port PORT on ROUTER\n\
   lrp-set-gateway-chassis PORT CHASSIS [PRIORITY]\n\
                             set gateway chassis for port PORT\n\
+  lrp-set-options PORT KEY=VALUE [KEY=VALUE]...\n\
+                            set router port options\n\
   lrp-del-gateway-chassis PORT CHASSIS\n\
                             delete gateway chassis from port PORT\n\
   lrp-get-gateway-chassis PORT\n\
@@ -351,11 +353,17 @@ Logical router port commands:\n\
                             ('overlay' or 'bridged')\n\
 \n\
 Route commands:\n\
-  [--policy=POLICY] [--ecmp] [--ecmp-symmetric-reply] lr-route-add ROUTER \n\
-                            PREFIX NEXTHOP [PORT]\n\
+  [--policy=POLICY]\n\
+  [--ecmp]\n\
+  [--ecmp-symmetric-reply]\n\
+  [--route-table=ROUTE_TABLE]\n\
+  lr-route-add ROUTER PREFIX NEXTHOP [PORT]\n\
                             add a route to ROUTER\n\
-  [--policy=POLICY] lr-route-del ROUTER [PREFIX [NEXTHOP [PORT]]]\n\
+  [--policy=POLICY]\n\
+  [--route-table=ROUTE_TABLE]\n\
+  lr-route-del ROUTER [PREFIX [NEXTHOP [PORT]]]\n\
                             remove routes from ROUTER\n\
+  [--route-table=ROUTE_TABLE]\n\
   lr-route-list ROUTER      print routes for ROUTER\n\
 \n\
 Policy commands:\n\
@@ -743,6 +751,11 @@ print_lr(const struct nbrec_logical_router *lr, struct ds *s)
             ds_put_cstr(s, "]\n");
         }
 
+        const char *route_table = smap_get(&lrp->options, "route_table");
+        if (route_table) {
+            ds_put_format(s, "        route-table: %s\n", route_table);
+        }
+
         if (lrp->n_gateway_chassis) {
             const struct nbrec_gateway_chassis **gcs;
 
@@ -862,6 +875,7 @@ nbctl_pre_show(struct ctl_context *ctx)
     ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_name);
     ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_mac);
     ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_networks);
+    ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_options);
     ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_gateway_chassis);
 
     ovsdb_idl_add_column(ctx->idl, &nbrec_gateway_chassis_col_chassis_name);
@@ -4000,11 +4014,19 @@ nbctl_lr_policy_list(struct ctl_context *ctx)
 
 static struct nbrec_logical_router_static_route *
 nbctl_lr_get_route(const struct nbrec_logical_router *lr, char *prefix,
-                   char *next_hop, bool is_src_route, bool ecmp)
+                   char *next_hop, bool is_src_route, bool ecmp,
+                   char *route_table)
 {
     for (int i = 0; i < lr->n_static_routes; i++) {
         struct nbrec_logical_router_static_route *route = lr->static_routes[i];
 
+        /* Strict compare for route_table.
+         * If route_table was not specified,
+         * lookup for routes with empty route_table value. */
+        if (strcmp(route->route_table, route_table ? route_table : "")) {
+            continue;
+        }
+
         /* Compare route policy. */
         char *nb_policy = route->policy;
         bool nb_is_src_route = false;
@@ -4060,6 +4082,8 @@ nbctl_pre_lr_route_add(struct ctl_context *ctx)
                          &nbrec_logical_router_static_route_col_bfd);
     ovsdb_idl_add_column(ctx->idl,
                          &nbrec_logical_router_static_route_col_options);
+    ovsdb_idl_add_column(ctx->idl,
+                         &nbrec_logical_router_static_route_col_route_table);
 }
 
 static char * OVS_WARN_UNUSED_RESULT
@@ -4090,6 +4114,7 @@ nbctl_lr_route_add(struct ctl_context *ctx)
         }
     }
 
+    char *route_table = shash_find_data(&ctx->options, "--route-table");
     bool v6_prefix = false;
     prefix = normalize_ipv4_prefix_str(ctx->argv[2]);
     if (!prefix) {
@@ -4159,7 +4184,8 @@ nbctl_lr_route_add(struct ctl_context *ctx)
     bool ecmp = shash_find(&ctx->options, "--ecmp") != NULL ||
                 ecmp_symmetric_reply;
     struct nbrec_logical_router_static_route *route =
-        nbctl_lr_get_route(lr, prefix, next_hop, is_src_route, ecmp);
+        nbctl_lr_get_route(lr, prefix, next_hop, is_src_route, ecmp,
+                           route_table);
 
     /* Validations for nexthop = "discard" */
     if (is_discard_route) {
@@ -4223,7 +4249,8 @@ nbctl_lr_route_add(struct ctl_context *ctx)
     }
 
     struct nbrec_logical_router_static_route *discard_route =
-        nbctl_lr_get_route(lr, prefix, "discard", is_src_route, true);
+        nbctl_lr_get_route(lr, prefix, "discard", is_src_route, true,
+                           route_table);
     if (discard_route) {
         ctl_error(ctx, "discard nexthop for the same ECMP route exists.");
         goto cleanup;
@@ -4239,6 +4266,9 @@ nbctl_lr_route_add(struct ctl_context *ctx)
     if (policy) {
         nbrec_logical_router_static_route_set_policy(route, policy);
     }
+    if (route_table) {
+        nbrec_logical_router_static_route_set_route_table(route, route_table);
+    }
 
     if (ecmp_symmetric_reply) {
         const struct smap options = SMAP_CONST1(&options,
@@ -4282,6 +4312,8 @@ nbctl_pre_lr_route_del(struct ctl_context *ctx)
                          &nbrec_logical_router_static_route_col_nexthop);
     ovsdb_idl_add_column(ctx->idl,
                          &nbrec_logical_router_static_route_col_output_port);
+    ovsdb_idl_add_column(ctx->idl,
+                         &nbrec_logical_router_static_route_col_route_table);
 
 }
 
@@ -4295,6 +4327,7 @@ nbctl_lr_route_del(struct ctl_context *ctx)
         return;
     }
 
+    const char *route_table = shash_find_data(&ctx->options, "--route-table");
     const char *policy = shash_find_data(&ctx->options, "--policy");
     bool is_src_route = false;
     if (policy) {
@@ -4385,6 +4418,14 @@ nbctl_lr_route_del(struct ctl_context *ctx)
             }
         }
 
+        /* Strict compare for route_table.
+         * If route_table was not specified,
+         * lookup for routes with empty route_table value. */
+        if (strcmp(lr->static_routes[i]->route_table,
+                   route_table ? route_table : "")) {
+            continue;
+        }
+
         /* Compare output_port, if specified. */
         if (output_port) {
             char *rt_output_port = lr->static_routes[i]->output_port;
@@ -5108,6 +5149,41 @@ nbctl_pre_lrp_del_gateway_chassis(struct ctl_context *ctx)
     ovsdb_idl_add_column(ctx->idl, &nbrec_gateway_chassis_col_chassis_name);
 }
 
+static void
+nbctl_pre_lrp_options(struct ctl_context *ctx)
+{
+    ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_name);
+    ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_options);
+}
+
+static void
+nbctl_lrp_set_options(struct ctl_context *ctx)
+{
+    const char *id = ctx->argv[1];
+    const struct nbrec_logical_router_port *lrp = NULL;
+    size_t i;
+    struct smap options = SMAP_INITIALIZER(&options);
+
+    char *error = lrp_by_name_or_uuid(ctx, id, true, &lrp);
+    if (error) {
+        ctx->error = error;
+        return;
+    }
+    for (i = 2; i < ctx->argc; i++) {
+        char *key, *value;
+        value = xstrdup(ctx->argv[i]);
+        key = strsep(&value, "=");
+        if (value) {
+            smap_add(&options, key, value);
+        }
+        free(key);
+    }
+
+    nbrec_logical_router_port_set_options(lrp, &options);
+
+    smap_destroy(&options);
+}
+
 /* Removes logical router port 'lrp->gateway_chassis[idx]'. */
 static void
 remove_gc(const struct nbrec_logical_router_port *lrp, size_t idx)
@@ -5884,6 +5960,7 @@ route_cmp_details(const struct nbrec_logical_router_static_route *r1,
     }
     return r1->output_port ? 1 : -1;
 }
+
 struct ipv4_route {
     int priority;
     ovs_be32 addr;
@@ -5893,6 +5970,11 @@ struct ipv4_route {
 static int
 __ipv4_route_cmp(const struct ipv4_route *r1, const struct ipv4_route *r2)
 {
+    int rtb_cmp = strcmp(r1->route->route_table,
+                         r2->route->route_table);
+    if (rtb_cmp) {
+        return rtb_cmp;
+    }
     if (r1->priority != r2->priority) {
         return r1->priority > r2->priority ? -1 : 1;
     }
@@ -5924,6 +6006,11 @@ struct ipv6_route {
 static int
 __ipv6_route_cmp(const struct ipv6_route *r1, const struct ipv6_route *r2)
 {
+    int rtb_cmp = strcmp(r1->route->route_table,
+                         r2->route->route_table);
+    if (rtb_cmp) {
+        return rtb_cmp;
+    }
     if (r1->priority != r2->priority) {
         return r1->priority > r2->priority ? -1 : 1;
     }
@@ -6011,6 +6098,8 @@ nbctl_pre_lr_route_list(struct ctl_context *ctx)
                          &nbrec_logical_router_static_route_col_options);
     ovsdb_idl_add_column(ctx->idl,
                          &nbrec_logical_router_static_route_col_bfd);
+    ovsdb_idl_add_column(ctx->idl,
+                         &nbrec_logical_router_static_route_col_route_table);
 }
 
 static void
@@ -6028,12 +6117,17 @@ nbctl_lr_route_list(struct ctl_context *ctx)
         return;
     }
 
+    char *route_table = shash_find_data(&ctx->options, "--route-table");
+
     ipv4_routes = xmalloc(sizeof *ipv4_routes * lr->n_static_routes);
     ipv6_routes = xmalloc(sizeof *ipv6_routes * lr->n_static_routes);
 
     for (int i = 0; i < lr->n_static_routes; i++) {
         const struct nbrec_logical_router_static_route *route
             = lr->static_routes[i];
+        if (route_table && strcmp(route->route_table, route_table)) {
+            continue;
+        }
         unsigned int plen;
         ovs_be32 ipv4;
         const char *policy = route->policy ? route->policy : "dst-ip";
@@ -6074,6 +6168,7 @@ nbctl_lr_route_list(struct ctl_context *ctx)
     if (n_ipv4_routes) {
         ds_put_cstr(&ctx->output, "IPv4 Routes\n");
     }
+    const struct nbrec_logical_router_static_route *route;
     for (int i = 0; i < n_ipv4_routes; i++) {
         bool ecmp = false;
         if (i < n_ipv4_routes - 1 &&
@@ -6084,6 +6179,15 @@ nbctl_lr_route_list(struct ctl_context *ctx)
                                      &ipv4_routes[i - 1])) {
             ecmp = true;
         }
+
+        route = ipv4_routes[i].route;
+        if (!i || (i > 0 && strcmp(route->route_table,
+                                   ipv4_routes[i - 1].route->route_table))) {
+            ds_put_format(&ctx->output, "%sRoute Table %s:\n", i ? "\n" : "",
+                          strlen(route->route_table) ? route->route_table
+                                                     : "global");
+        }
+
         print_route(ipv4_routes[i].route, &ctx->output, ecmp);
     }
 
@@ -6101,6 +6205,15 @@ nbctl_lr_route_list(struct ctl_context *ctx)
                                      &ipv6_routes[i - 1])) {
             ecmp = true;
         }
+
+        route = ipv6_routes[i].route;
+        if (!i || (i > 0 && strcmp(route->route_table,
+                                   ipv6_routes[i - 1].route->route_table))) {
+            ds_put_format(&ctx->output, "%sRoute Table %s:\n", i ? "\n" : "",
+                          strlen(route->route_table) ? route->route_table
+                                                     : "global");
+        }
+
         print_route(ipv6_routes[i].route, &ctx->output, ecmp);
     }
 
@@ -6919,6 +7032,8 @@ static const struct ctl_command_syntax nbctl_commands[] = {
       "PORT CHASSIS [PRIORITY]",
       nbctl_pre_lrp_set_gateway_chassis, nbctl_lrp_set_gateway_chassis,
       NULL, "--may-exist", RW },
+    { "lrp-set-options", 1, INT_MAX, "PORT KEY=VALUE [KEY=VALUE]...",
+      nbctl_pre_lrp_options, nbctl_lrp_set_options, NULL, "", RW },
     { "lrp-del-gateway-chassis", 2, 2, "PORT CHASSIS",
       nbctl_pre_lrp_del_gateway_chassis, nbctl_lrp_del_gateway_chassis,
       NULL, "", RW },
@@ -6942,12 +7057,13 @@ static const struct ctl_command_syntax nbctl_commands[] = {
     /* logical router route commands. */
     { "lr-route-add", 3, 4, "ROUTER PREFIX NEXTHOP [PORT]",
       nbctl_pre_lr_route_add, nbctl_lr_route_add, NULL,
-      "--may-exist,--ecmp,--ecmp-symmetric-reply,--policy=,--bfd?", RW },
+      "--may-exist,--ecmp,--ecmp-symmetric-reply,--policy=,--route-table=,--bfd?",
+      RW },
     { "lr-route-del", 1, 4, "ROUTER [PREFIX [NEXTHOP [PORT]]]",
       nbctl_pre_lr_route_del, nbctl_lr_route_del, NULL,
-      "--if-exists,--policy=", RW },
+      "--if-exists,--policy=,--route-table=", RW },
     { "lr-route-list", 1, 1, "ROUTER", nbctl_pre_lr_route_list,
-      nbctl_lr_route_list, NULL, "", RO },
+      nbctl_lr_route_list, NULL, "--route-table=", RO },
 
     /* Policy commands */
     { "lr-policy-add", 4, INT_MAX,

From patchwork Sun Sep 19 21:23:15 2021
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Patchwork-Submitter: Vladislav Odintsov 
X-Patchwork-Id: 1529889
X-Patchwork-Delegate: zhouhan@gmail.com
Return-Path: 
X-Original-To: incoming@patchwork.ozlabs.org
Delivered-To: patchwork-incoming@ozlabs.org
Authentication-Results: ozlabs.org;
	dkim=fail reason="signature verification failed" (2048-bit key;
 unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256
 header.s=20210112 header.b=k2hxKuM7;
	dkim-atps=neutral
Authentication-Results: ozlabs.org;
 spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org
 (client-ip=140.211.166.133; helo=smtp2.osuosl.org;
 envelope-from=ovs-dev-bounces@openvswitch.org; receiver=)
Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest
 SHA256)
	(No client certificate requested)
	by ozlabs.org (Postfix) with ESMTPS id 4HCLKb21QJz9ssP
	for ; Mon, 20 Sep 2021 07:24:03 +1000 (AEST)
Received: from localhost (localhost [127.0.0.1])
	by smtp2.osuosl.org (Postfix) with ESMTP id 27AF14035F;
	Sun, 19 Sep 2021 21:24:01 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
Received: from smtp2.osuosl.org ([127.0.0.1])
	by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id tTtOsnXEStoD; Sun, 19 Sep 2021 21:23:56 +0000 (UTC)
Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56])
	by smtp2.osuosl.org (Postfix) with ESMTPS id 77C7240346;
	Sun, 19 Sep 2021 21:23:53 +0000 (UTC)
Received: from lf-lists.osuosl.org (localhost [127.0.0.1])
	by lists.linuxfoundation.org (Postfix) with ESMTP id 3A216C0011;
	Sun, 19 Sep 2021 21:23:53 +0000 (UTC)
X-Original-To: dev@openvswitch.org
Delivered-To: ovs-dev@lists.linuxfoundation.org
Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133])
 by lists.linuxfoundation.org (Postfix) with ESMTP id 228D2C000D
 for ; Sun, 19 Sep 2021 21:23:52 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by smtp2.osuosl.org (Postfix) with ESMTP id C27FF402E3
 for ; Sun, 19 Sep 2021 21:23:47 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
Received: from smtp2.osuosl.org ([127.0.0.1])
 by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id fQOVwVVXPFDe for ;
 Sun, 19 Sep 2021 21:23:45 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.8.0
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [IPv6:2a00:1450:4864:20::12d])
 by smtp2.osuosl.org (Postfix) with ESMTPS id 9C2D4402CF
 for ; Sun, 19 Sep 2021 21:23:44 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id y28so57780411lfb.0
 for ; Sun, 19 Sep 2021 14:23:44 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=from:to:cc:subject:date:message-id:in-reply-to:references
 :mime-version:content-transfer-encoding;
 bh=NnNfINY9d25CBAVKbep1OHiqtIYyF21/oLqnU4JYR4Q=;
 b=k2hxKuM7rwQvbxMUSrCc90qBn9G2I7xYVOoQt4dZW/mitV8eB1aobbnJoxkpuz0Y5W
 zRtHpsfctNs1d737Qm54OXaSq0m0ajjXbaG4GZkIfkJujeeJxKFzwSDh1tovsXaOqej+
 fcJuUSHF0naCDH7yr3034J6rKQyb8+XGjho4aNjZKz+5nIpW8yJUwox0eos3BLn9jwTH
 32kDwsu02kgz/LO3BuMJeu2V4d3kuI2xOsDuV7u+Fl2J9EPGz2cDRP+1rMhDHGpY5xt/
 l6NxNrl2qvENL63JF1Ie8L5Imt3nGC1TKuH6plTFA5N66FbDqOPuFwowht/n/m6XrmdI
 Sphg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
 :references:mime-version:content-transfer-encoding;
 bh=NnNfINY9d25CBAVKbep1OHiqtIYyF21/oLqnU4JYR4Q=;
 b=vvojApSF3Bju7Tsvn0h1FFNvnSt6u1qJVJOMuekWDsgyPixJuMJ2QgmApS95GUnDRX
 x6FHRCWoobgU01JBbtOka0YT/6gdxYi6w59fEjdNCmIVniU8mZfJmJLMJWWKi3ZLalz3
 a9UD8QQ+Eq2i242z0+KFoBnYFRT8EH0KTXXs0p2tsKq/+q+MwAtZHRxG0Z/di+I8n9WW
 kI1dWxL3OAzXcuxsTMHa2k8VFPxKp4Rh9oGPtQ+2PfUvuPkRQpKHv0gEYNlzDIqyaNYM
 HlMspw8amYhO5UNfICWGPARo6BOe/Y3nJmnoZZ+0Xkhn8JSLdCxfDoz1ZKHgIHW4PbG8
 wx0A==
X-Gm-Message-State: AOAM5322OMvL8iW4ZWfqkvGZjutcaMR5Bl994yxFqTGWLLI6uV1wyOt7
 uC/olT7qjRLFLeosKoJWkSrsQQwHJOo=
X-Google-Smtp-Source: 
 ABdhPJzYkJ3xF3e8RS/6hH1Qc9RwEULaBjZ2U9IfoEltyDvD5ipGKraJCAmYtwBknLKRqN8sZoJXQw==
X-Received: by 2002:a2e:6f1b:: with SMTP id
 k27mr12623605ljc.117.1632086621462;
 Sun, 19 Sep 2021 14:23:41 -0700 (PDT)
Received: from localhost.localdomain (109-252-131-59.dynamic.spd-mgts.ru.
 [109.252.131.59])
 by smtp.gmail.com with ESMTPSA id q5sm431658lfn.277.2021.09.19.14.23.41
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Sun, 19 Sep 2021 14:23:41 -0700 (PDT)
From: Vladislav Odintsov 
To: dev@openvswitch.org
Date: Mon, 20 Sep 2021 00:23:15 +0300
Message-Id: <20210919212316.24148-4-odivlad@gmail.com>
X-Mailer: git-send-email 2.30.0
In-Reply-To: <20210919212316.24148-1-odivlad@gmail.com>
References: <20210919212316.24148-1-odivlad@gmail.com>
MIME-Version: 1.0
Cc: Vladislav Odintsov 
Subject: [ovs-dev] [PATCH ovn v4 3/4] ic: add support for routing tables in
	adv/learn routes
X-BeenThere: ovs-dev@openvswitch.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: 
List-Unsubscribe: ,
 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: ,
 
Errors-To: ovs-dev-bounces@openvswitch.org
Sender: "dev" 

Previously support for multiple routing tables was added
to northd code.
This commit expands support for multiple routing tables
by adding support of advertising and learning routes with
their routing table information.

To utilize such feature, user must:
1. create Logical Router in each AZ;
2. create IC transit switch for each routing table, that
   he/she needs;
3. connect each TS with this LR;
4. assign routing table for TS's LRP
   (ovn-nbctl lrp-set-options  route_table=<>);
5. enable routes sync (turn on learning and advertising
   routes in NB_Global table);
6. create LRPs for subnets in LR, create static routes
   with supplying route_table parameter.

Note 1: routes for directly-connected networks will be
learned to global routing table and if Logical Routers
have more than one Transit Switch, which interconnects
them, directly-connected routes will be added via each
transit switch port and configured as ECMP routes.

Note 2: static routes within route tables will be advertised
and learned only if interconnecting transit switch's LRPs
will have options:route_table same value as route's route_table
value.

Signed-off-by: Vladislav Odintsov 
Reviewed-by: Numan Siddique 
---
 NEWS                |   4 +
 ic/ovn-ic.c         | 534 ++++++++++++++++++++++++++++----------------
 ovn-ic-sb.ovsschema |   5 +-
 ovn-ic-sb.xml       |  18 ++
 tests/ovn-ic.at     | 440 ++++++++++++++++++++++++++++++++++++
 5 files changed, 808 insertions(+), 193 deletions(-)

diff --git a/NEWS b/NEWS
index 8a21c029e..3855f0d48 100644
--- a/NEWS
+++ b/NEWS
@@ -12,6 +12,10 @@ OVN v21.09.0 - xx xxx xxxx
   - Allow static routes without nexthops.
   - Enabled logical dp groups as a default.  CMS should disable it if not
     desired.
+  - Added support for multiple routing tables in Logical Router Static Routes
+    and LRPs. OVN Interconnection supports routes' route tables as well.
+    This requires to update schemas for OVN_Northdbound and OVN_IC_Southbound
+    DBs.
 
 OVN v21.06.0 - 18 Jun 2021
 -------------------------
diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c
index 92c83d730..94f123f70 100644
--- a/ic/ovn-ic.c
+++ b/ic/ovn-ic.c
@@ -63,9 +63,11 @@ struct ic_context {
     struct ovsdb_idl_txn *ovninb_txn;
     struct ovsdb_idl_txn *ovnisb_txn;
     struct ovsdb_idl_index *nbrec_ls_by_name;
+    struct ovsdb_idl_index *nbrec_lrp_by_name;
     struct ovsdb_idl_index *nbrec_port_by_name;
     struct ovsdb_idl_index *sbrec_chassis_by_name;
     struct ovsdb_idl_index *sbrec_port_binding_by_name;
+    struct ovsdb_idl_index *icnbrec_transit_switch_by_name;
     struct ovsdb_idl_index *icsbrec_port_binding_by_az;
     struct ovsdb_idl_index *icsbrec_port_binding_by_ts;
     struct ovsdb_idl_index *icsbrec_port_binding_by_ts_az;
@@ -773,7 +775,7 @@ port_binding_run(struct ic_context *ctx,
         icsbrec_port_binding_index_set_transit_switch(isb_pb_key, ts->name);
 
         ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key,
-                                            ctx->icsbrec_port_binding_by_ts) {
+                                             ctx->icsbrec_port_binding_by_ts) {
             if (isb_pb->availability_zone == az) {
                 shash_add(&local_pbs, isb_pb->logical_port, isb_pb);
                 shash_find_and_delete(&isb_all_local_pbs,
@@ -844,7 +846,9 @@ port_binding_run(struct ic_context *ctx,
 struct ic_router_info {
     struct hmap_node node;
     const struct nbrec_logical_router *lr; /* key of hmap */
-    const struct icsbrec_port_binding *isb_pb;
+    const struct icsbrec_port_binding **isb_pbs;
+    size_t n_isb_pbs;
+    size_t n_allocated_isb_pbs;
     struct hmap routes_learned;
 };
 
@@ -854,6 +858,7 @@ struct ic_route_info {
     struct in6_addr prefix;
     unsigned int plen;
     struct in6_addr nexthop;
+    const char *route_table;
 
     /* Either nb_route or nb_lrp is set and the other one must be NULL.
      * - For a route that is learned from IC-SB, or a static route that is
@@ -875,13 +880,15 @@ ic_route_hash(const struct in6_addr *prefix, unsigned int plen,
 
 static struct ic_route_info *
 ic_route_find(struct hmap *routes, const struct in6_addr *prefix,
-              unsigned int plen, const struct in6_addr *nexthop)
+              unsigned int plen, const struct in6_addr *nexthop,
+              char *route_table)
 {
     struct ic_route_info *r;
     uint32_t hash = ic_route_hash(prefix, plen, nexthop);
     HMAP_FOR_EACH_WITH_HASH (r, node, hash, routes) {
         if (ipv6_addr_equals(&r->prefix, prefix) &&
             r->plen == plen &&
+            !strcmp(r->route_table ? r->route_table : "", route_table) &&
             ipv6_addr_equals(&r->nexthop, nexthop)) {
             return r;
         }
@@ -926,11 +933,19 @@ add_to_routes_learned(struct hmap *routes_learned,
                      &prefix, &plen, &nexthop)) {
         return false;
     }
+
+    if (ic_route_find(routes_learned, &prefix, plen, &nexthop,
+                      nb_route->route_table)) {
+        /* Route is already added to learned in previous iteration. */
+        return true;
+    }
+
     struct ic_route_info *ic_route = xzalloc(sizeof *ic_route);
     ic_route->prefix = prefix;
     ic_route->plen = plen;
     ic_route->nexthop = nexthop;
     ic_route->nb_route = nb_route;
+    ic_route->route_table = nb_route->route_table;
     hmap_insert(routes_learned, &ic_route->node,
                 ic_route_hash(&prefix, plen, &nexthop));
     return true;
@@ -1069,8 +1084,17 @@ static void
 add_to_routes_ad(struct hmap *routes_ad,
                  const struct nbrec_logical_router_static_route *nb_route,
                  const struct lport_addresses *nexthop_addresses,
-                 const struct smap *nb_options)
+                 const struct smap *nb_options, const char *route_table)
 {
+    if (strcmp(route_table, nb_route->route_table)) {
+        if (VLOG_IS_DBG_ENABLED()) {
+            VLOG_DBG("Skip advertising route %s -> %s as its route table %s !="
+                     " %s of TS port", nb_route->ip_prefix, nb_route->nexthop,
+                     nb_route->route_table, route_table);
+        }
+        return;
+    }
+
     struct in6_addr prefix, nexthop;
     unsigned int plen;
     if (!parse_route(nb_route->ip_prefix, nb_route->nexthop,
@@ -1088,11 +1112,33 @@ add_to_routes_ad(struct hmap *routes_ad,
         return;
     }
 
+    if (VLOG_IS_DBG_ENABLED()) {
+        struct ds msg = DS_EMPTY_INITIALIZER;
+
+        ds_put_format(&msg, "Advertising static route: %s -> %s, ic nexthop: ",
+                      nb_route->ip_prefix, nb_route->nexthop);
+
+        if (IN6_IS_ADDR_V4MAPPED(&nexthop)) {
+            ds_put_format(&msg, IP_FMT,
+                          IP_ARGS(in6_addr_get_mapped_ipv4(&nexthop)));
+        } else {
+            ipv6_format_addr(&nexthop, &msg);
+        }
+
+        ds_put_format(&msg, ", route_table: %s", strlen(nb_route->route_table)
+                                                 ? nb_route->route_table
+                                                 : "global");
+
+        VLOG_DBG("%s", ds_cstr(&msg));
+        ds_destroy(&msg);
+    }
+
     struct ic_route_info *ic_route = xzalloc(sizeof *ic_route);
     ic_route->prefix = prefix;
     ic_route->plen = plen;
     ic_route->nexthop = nexthop;
     ic_route->nb_route = nb_route;
+    ic_route->route_table = nb_route->route_table;
     hmap_insert(routes_ad, &ic_route->node,
                 ic_route_hash(&prefix, plen, &nexthop));
 }
@@ -1124,8 +1170,8 @@ add_network_to_routes_ad(struct hmap *routes_ad, const char *network,
     if (VLOG_IS_DBG_ENABLED()) {
         struct ds msg = DS_EMPTY_INITIALIZER;
 
-        ds_put_format(&msg, "Route ad: direct network %s of lrp %s, nexthop ",
-                      network, nb_lrp->name);
+        ds_put_format(&msg, "Adding direct network route to global routing "
+                      "table: %s of lrp %s, nexthop ", network, nb_lrp->name);
 
         if (IN6_IS_ADDR_V4MAPPED(&nexthop)) {
             ds_put_format(&msg, IP_FMT,
@@ -1143,13 +1189,15 @@ add_network_to_routes_ad(struct hmap *routes_ad, const char *network,
     ic_route->plen = plen;
     ic_route->nexthop = nexthop;
     ic_route->nb_lrp = nb_lrp;
+
+    /* directly-connected routes go to global route table */
+    ic_route->route_table = NULL;
     hmap_insert(routes_ad, &ic_route->node,
                 ic_route_hash(&prefix, plen, &nexthop));
 }
 
 static bool
-route_need_learn(struct in6_addr *prefix,
-                 unsigned int plen,
+route_need_learn(struct in6_addr *prefix, unsigned int plen,
                  const struct smap *nb_options)
 {
     if (!smap_get_bool(nb_options, "ic-route-learn", false)) {
@@ -1172,70 +1220,147 @@ route_need_learn(struct in6_addr *prefix,
     return true;
 }
 
+static const char *
+get_lrp_name_by_ts_port_name(struct ic_context *ctx, const char *ts_port_name)
+{
+    const struct nbrec_logical_switch_port *nb_lsp;
+    const struct nbrec_logical_switch_port *nb_lsp_key =
+        nbrec_logical_switch_port_index_init_row(ctx->nbrec_port_by_name);
+    nbrec_logical_switch_port_index_set_name(nb_lsp_key, ts_port_name);
+    nb_lsp = nbrec_logical_switch_port_index_find(ctx->nbrec_port_by_name,
+                                                  nb_lsp_key);
+    nbrec_logical_switch_port_index_destroy_row(nb_lsp_key);
+
+    if (!nb_lsp) {
+        return NULL;
+    }
+
+    return smap_get(&nb_lsp->options, "router-port");
+}
+
+static const char *
+get_route_table_by_lrp_name(struct ic_context *ctx, const char *lrp_name)
+{
+    const struct nbrec_logical_router_port *lrp;
+    const struct nbrec_logical_router_port *lrp_key =
+        nbrec_logical_router_port_index_init_row(ctx->nbrec_lrp_by_name);
+    nbrec_logical_router_port_index_set_name(lrp_key, lrp_name);
+    lrp = nbrec_logical_router_port_index_find(ctx->nbrec_lrp_by_name,
+                                               lrp_key);
+    nbrec_logical_router_port_index_destroy_row(lrp_key);
+
+    if (lrp) {
+        return smap_get_def(&lrp->options, "route_table", "");
+    }
+    return "";  /* Global route table */
+}
+
+static bool
+lrp_is_ts_port(struct ic_context *ctx, struct ic_router_info *ic_lr,
+               const char *lrp_name)
+{
+    const struct icsbrec_port_binding *isb_pb;
+    const char *ts_lrp_name;
+    for (int i = 0; i < ic_lr->n_isb_pbs; i++) {
+        isb_pb = ic_lr->isb_pbs[i];
+        ts_lrp_name = get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port);
+        if (!strcmp(ts_lrp_name, lrp_name)) {
+            return true;
+        }
+    }
+    return false;
+}
+
 static void
-sync_learned_route(struct ic_context *ctx,
-                   const struct icsbrec_availability_zone *az,
-                   struct ic_router_info *ic_lr)
+sync_learned_routes(struct ic_context *ctx,
+                    const struct icsbrec_availability_zone *az,
+                    struct ic_router_info *ic_lr)
 {
     ovs_assert(ctx->ovnnb_txn);
     const struct icsbrec_route *isb_route;
     const struct icsbrec_route *isb_route_key =
         icsbrec_route_index_init_row(ctx->icsbrec_route_by_ts);
 
-    icsbrec_route_index_set_transit_switch(isb_route_key,
-                                           ic_lr->isb_pb->transit_switch);
+    const struct nbrec_nb_global *nb_global =
+        nbrec_nb_global_first(ctx->ovnnb_idl);
+    ovs_assert(nb_global);
 
-    ICSBREC_ROUTE_FOR_EACH_EQUAL (isb_route, isb_route_key,
-                                  ctx->icsbrec_route_by_ts) {
-        if (isb_route->availability_zone == az) {
-            continue;
-        }
-        struct in6_addr prefix, nexthop;
-        unsigned int plen;
-        if (!parse_route(isb_route->ip_prefix, isb_route->nexthop,
-                         &prefix, &plen, &nexthop)) {
-            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
-            VLOG_WARN_RL(&rl, "Bad route format in IC-SB: %s -> %s. Ignored.",
-                         isb_route->ip_prefix, isb_route->nexthop);
-            continue;
-        }
-        const struct nbrec_nb_global *nb_global =
-            nbrec_nb_global_first(ctx->ovnnb_idl);
-        ovs_assert(nb_global);
-        if (!route_need_learn(&prefix, plen, &nb_global->options)) {
-            continue;
-        }
-        struct ic_route_info *route_learned
-            = ic_route_find(&ic_lr->routes_learned, &prefix, plen, &nexthop);
-        if (route_learned) {
-            /* Sync external-ids */
-            struct uuid ext_id;
-            smap_get_uuid(&route_learned->nb_route->external_ids,
-                          "ic-learned-route", &ext_id);
-            if (!uuid_equals(&ext_id, &isb_route->header_.uuid)) {
+    const char *lrp_name, *ts_route_table;
+    const struct icsbrec_port_binding *isb_pb;
+    for (int i = 0; i < ic_lr->n_isb_pbs; i++) {
+        isb_pb = ic_lr->isb_pbs[i];
+        lrp_name = get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port);
+        ts_route_table = get_route_table_by_lrp_name(ctx, lrp_name);
+
+        icsbrec_route_index_set_transit_switch(isb_route_key,
+                                               isb_pb->transit_switch);
+
+        ICSBREC_ROUTE_FOR_EACH_EQUAL (isb_route, isb_route_key,
+                                      ctx->icsbrec_route_by_ts) {
+            if (isb_route->availability_zone == az) {
+                continue;
+            }
+
+            if (strlen(isb_route->route_table) &&
+                strcmp(isb_route->route_table, ts_route_table)) {
+                if (VLOG_IS_DBG_ENABLED()) {
+                    VLOG_DBG("Skip learning static route %s -> %s as either "
+                             "its route table %s != %s of TS port or ",
+                             isb_route->ip_prefix, isb_route->nexthop,
+                             isb_route->route_table, ts_route_table);
+                }
+                continue;
+            }
+
+            struct in6_addr prefix, nexthop;
+            unsigned int plen;
+            if (!parse_route(isb_route->ip_prefix, isb_route->nexthop,
+                             &prefix, &plen, &nexthop)) {
+                static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
+                VLOG_WARN_RL(&rl, "Bad route format in IC-SB: %s -> %s. "
+                             "Ignored.", isb_route->ip_prefix,
+                             isb_route->nexthop);
+                continue;
+            }
+            if (!route_need_learn(&prefix, plen, &nb_global->options)) {
+                continue;
+            }
+            struct ic_route_info *route_learned
+                = ic_route_find(&ic_lr->routes_learned, &prefix, plen,
+                                &nexthop, isb_route->route_table);
+            if (route_learned) {
+                /* Sync external-ids */
+                struct uuid ext_id;
+                smap_get_uuid(&route_learned->nb_route->external_ids,
+                              "ic-learned-route", &ext_id);
+                if (!uuid_equals(&ext_id, &isb_route->header_.uuid)) {
+                    char *uuid_s =
+                        xasprintf(UUID_FMT,
+                                  UUID_ARGS(&isb_route->header_.uuid));
+                    nbrec_logical_router_static_route_update_external_ids_setkey(
+                        route_learned->nb_route, "ic-learned-route", uuid_s);
+                    free(uuid_s);
+                }
+                hmap_remove(&ic_lr->routes_learned, &route_learned->node);
+                free(route_learned);
+            } else {
+                /* Create the missing route in NB. */
+                const struct nbrec_logical_router_static_route *nb_route =
+                    nbrec_logical_router_static_route_insert(ctx->ovnnb_txn);
+                nbrec_logical_router_static_route_set_ip_prefix(nb_route,
+                    isb_route->ip_prefix);
+                nbrec_logical_router_static_route_set_nexthop(nb_route,
+                    isb_route->nexthop);
                 char *uuid_s = xasprintf(UUID_FMT,
                                          UUID_ARGS(&isb_route->header_.uuid));
+                nbrec_logical_router_static_route_set_route_table(nb_route,
+                    isb_route->route_table);
                 nbrec_logical_router_static_route_update_external_ids_setkey(
-                    route_learned->nb_route, "ic-learned-route", uuid_s);
+                    nb_route, "ic-learned-route", uuid_s);
                 free(uuid_s);
+                nbrec_logical_router_update_static_routes_addvalue(ic_lr->lr,
+                    nb_route);
             }
-            hmap_remove(&ic_lr->routes_learned, &route_learned->node);
-            free(route_learned);
-        } else {
-            /* Create the missing route in NB. */
-            const struct nbrec_logical_router_static_route *nb_route =
-                nbrec_logical_router_static_route_insert(ctx->ovnnb_txn);
-            nbrec_logical_router_static_route_set_ip_prefix(
-                nb_route, isb_route->ip_prefix);
-            nbrec_logical_router_static_route_set_nexthop(
-                nb_route, isb_route->nexthop);
-            char *uuid_s = xasprintf(UUID_FMT,
-                                     UUID_ARGS(&isb_route->header_.uuid));
-            nbrec_logical_router_static_route_update_external_ids_setkey(
-                nb_route, "ic-learned-route", uuid_s);
-            free(uuid_s);
-            nbrec_logical_router_update_static_routes_addvalue(
-                ic_lr->lr, nb_route);
         }
     }
     icsbrec_route_index_destroy_row(isb_route_key);
@@ -1271,10 +1396,10 @@ ad_route_sync_external_ids(const struct ic_route_info *route_adv,
 
 /* Sync routes from routes_ad to IC-SB. */
 static void
-advertise_route(struct ic_context *ctx,
-                const struct icsbrec_availability_zone *az,
-                const char *ts_name,
-                struct hmap *routes_ad)
+advertise_routes(struct ic_context *ctx,
+                 const struct icsbrec_availability_zone *az,
+                 const char *ts_name,
+                 struct hmap *routes_ad)
 {
     ovs_assert(ctx->ovnisb_txn);
     const struct icsbrec_route *isb_route;
@@ -1298,7 +1423,8 @@ advertise_route(struct ic_context *ctx,
             continue;
         }
         struct ic_route_info *route_adv =
-            ic_route_find(routes_ad, &prefix, plen, &nexthop);
+            ic_route_find(routes_ad, &prefix, plen, &nexthop,
+                          isb_route->route_table);
         if (!route_adv) {
             /* Delete the extra route from IC-SB. */
             VLOG_DBG("Delete route %s -> %s from IC-SB, which is not found"
@@ -1338,6 +1464,9 @@ advertise_route(struct ic_context *ctx,
         }
         icsbrec_route_set_ip_prefix(isb_route, prefix_s);
         icsbrec_route_set_nexthop(isb_route, nexthop_s);
+        icsbrec_route_set_route_table(isb_route, route_adv->route_table
+                                                 ? route_adv->route_table
+                                                 : "");
         free(prefix_s);
         free(nexthop_s);
 
@@ -1348,23 +1477,97 @@ advertise_route(struct ic_context *ctx,
     }
 }
 
-static const char *
-get_lrp_name_by_ts_port_name(struct ic_context *ctx,
-                           const char *ts_port_name)
+static void
+build_ts_routes_to_adv(struct ic_context *ctx,
+                       struct ic_router_info *ic_lr,
+                       struct hmap *routes_ad,
+                       struct lport_addresses *ts_port_addrs,
+                       const struct nbrec_nb_global *nb_global,
+                       const char *ts_route_table)
 {
-    const struct nbrec_logical_switch_port *nb_lsp;
-    const struct nbrec_logical_switch_port *nb_lsp_key =
-        nbrec_logical_switch_port_index_init_row(ctx->nbrec_port_by_name);
-    nbrec_logical_switch_port_index_set_name(nb_lsp_key, ts_port_name);
-    nb_lsp = nbrec_logical_switch_port_index_find(ctx->nbrec_port_by_name,
-                                                  nb_lsp_key);
-    nbrec_logical_switch_port_index_destroy_row(nb_lsp_key);
+    const struct nbrec_logical_router *lr = ic_lr->lr;
+
+    /* Check static routes of the LR */
+    for (int i = 0; i < lr->n_static_routes; i++) {
+        const struct nbrec_logical_router_static_route *nb_route
+            = lr->static_routes[i];
+        struct uuid isb_uuid;
+        if (smap_get_uuid(&nb_route->external_ids, "ic-learned-route",
+                          &isb_uuid)) {
+            /* It is a learned route */
+            if (!add_to_routes_learned(&ic_lr->routes_learned, nb_route)) {
+                static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
+                VLOG_WARN_RL(&rl, "Bad format of learned route in NB: "
+                             "%s -> %s. Delete it.", nb_route->ip_prefix,
+                             nb_route->nexthop);
+                nbrec_logical_router_update_static_routes_delvalue(lr,
+                    nb_route);
+            }
+        } else {
+            /* It may be a route to be advertised */
+            add_to_routes_ad(routes_ad, nb_route, ts_port_addrs,
+                             &nb_global->options, ts_route_table);
+        }
+    }
 
-    if (!nb_lsp) {
-        return NULL;
+    /* Check directly-connected subnets of the LR */
+    for (int i = 0; i < lr->n_ports; i++) {
+        const struct nbrec_logical_router_port *lrp = lr->ports[i];
+        if (!lrp_is_ts_port(ctx, ic_lr, lrp->name)) {
+            for (int j = 0; j < lrp->n_networks; j++) {
+                add_network_to_routes_ad(routes_ad, lrp->networks[j], lrp,
+                                         ts_port_addrs,
+                                         &nb_global->options);
+            }
+        } else {
+            /* The router port of the TS port is ignored. */
+            VLOG_DBG("Skip advertising direct route of lrp %s (TS port)",
+                     lrp->name);
+        }
     }
+}
 
-    return smap_get(&nb_lsp->options, "router-port");
+static void
+advertise_lr_routes(struct ic_context *ctx,
+                    const struct icsbrec_availability_zone *az,
+                    struct ic_router_info *ic_lr)
+{
+    const struct nbrec_nb_global *nb_global =
+        nbrec_nb_global_first(ctx->ovnnb_idl);
+    ovs_assert(nb_global);
+
+    const struct icsbrec_port_binding *isb_pb;
+    const char *lrp_name, *route_table;
+    struct lport_addresses ts_port_addrs;
+    const struct nbrec_logical_router *lr = ic_lr->lr;
+    const struct icnbrec_transit_switch *ts, *key =
+        icnbrec_transit_switch_index_init_row(
+            ctx->icnbrec_transit_switch_by_name);
+
+    struct hmap routes_ad = HMAP_INITIALIZER(&routes_ad);
+    for (int i = 0; i < ic_lr->n_isb_pbs; i++) {
+        isb_pb = ic_lr->isb_pbs[i];
+        icnbrec_transit_switch_index_set_name(key, isb_pb->transit_switch);
+        ts = icnbrec_transit_switch_index_find(
+            ctx->icnbrec_transit_switch_by_name, key);
+
+        if (!extract_lsp_addresses(isb_pb->address, &ts_port_addrs)) {
+            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
+            VLOG_INFO_RL(&rl, "Route sync ignores port %s on ts %s for router"
+                         " %s because the addresses are invalid.",
+                         isb_pb->logical_port, isb_pb->transit_switch,
+                         lr->name);
+            continue;
+        }
+        lrp_name = get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port);
+        route_table = get_route_table_by_lrp_name(ctx, lrp_name);
+        build_ts_routes_to_adv(ctx, ic_lr, &routes_ad, &ts_port_addrs,
+                               nb_global, route_table);
+        advertise_routes(ctx, az, ts->name, &routes_ad);
+        destroy_lport_addresses(&ts_port_addrs);
+    }
+    hmap_destroy(&routes_ad);
+    icnbrec_transit_switch_index_destroy_row(key);
 }
 
 static void
@@ -1375,131 +1578,70 @@ route_run(struct ic_context *ctx,
         return;
     }
 
-    const struct nbrec_nb_global *nb_global =
-        nbrec_nb_global_first(ctx->ovnnb_idl);
-    ovs_assert(nb_global);
-
-    const struct icnbrec_transit_switch *ts;
-    ICNBREC_TRANSIT_SWITCH_FOR_EACH (ts, ctx->ovninb_idl) {
-        struct hmap ic_lrs = HMAP_INITIALIZER(&ic_lrs);
-        struct hmap routes_ad = HMAP_INITIALIZER(&routes_ad);
-
-        const struct icsbrec_port_binding *isb_pb;
-        const struct icsbrec_port_binding *isb_pb_key =
-            icsbrec_port_binding_index_init_row(
-                ctx->icsbrec_port_binding_by_ts_az);
-        icsbrec_port_binding_index_set_transit_switch(isb_pb_key, ts->name);
-        icsbrec_port_binding_index_set_availability_zone(isb_pb_key, az);
-
-        /* Each port on TS maps to a logical router, which is stored in the
-         * external_ids:router-id of the IC SB port_binding record. */
-        ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key,
-                                             ctx->icsbrec_port_binding_by_ts_az)
-        {
-            const char *ts_lrp_name =
-                get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port);
-            if (!ts_lrp_name) {
-                static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
-                VLOG_WARN_RL(&rl, "Route sync ignores port %s on ts %s "
-                             "because logical router port is not found in NB.",
-                             isb_pb->logical_port, ts->name);
-                continue;
-            }
+    struct hmap ic_lrs = HMAP_INITIALIZER(&ic_lrs);
+    const struct icsbrec_port_binding *isb_pb;
+    const struct icsbrec_port_binding *isb_pb_key =
+        icsbrec_port_binding_index_init_row(ctx->icsbrec_port_binding_by_az);
+    icsbrec_port_binding_index_set_availability_zone(isb_pb_key, az);
 
-            struct uuid lr_uuid;
-            if (!smap_get_uuid(&isb_pb->external_ids, "router-id", &lr_uuid)) {
-                VLOG_DBG("IC-SB Port_Binding %s doesn't have "
-                         "external_ids:router-id set.", isb_pb->logical_port);
-                continue;
-            }
-            const struct nbrec_logical_router *lr
-                = nbrec_logical_router_get_for_uuid(ctx->ovnnb_idl, &lr_uuid);
-            if (!lr) {
-                continue;
-            }
+    /* Each port on TS maps to a logical router, which is stored in the
+     * external_ids:router-id of the IC SB port_binding record.
+     * Here we build info for interconnected Logical Router:
+     * collect IC Port Binding to process routes sync later on. */
+    ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key,
+                                         ctx->icsbrec_port_binding_by_az)
+    {
+        const char *ts_lrp_name =
+            get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port);
+        if (!ts_lrp_name) {
+            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
+            VLOG_WARN_RL(&rl, "Route sync ignores port %s on ts %s because "
+                         "logical router port is not found in NB. Deleting it",
+                         isb_pb->logical_port, isb_pb->transit_switch);
+            icsbrec_port_binding_delete(isb_pb);
+            continue;
+        }
 
-            if (ic_router_find(&ic_lrs, lr)) {
-                static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
-                VLOG_INFO_RL(&rl, "Route sync ignores port %s on ts %s for "
-                             "router %s because the router has another port "
-                             "connected to same ts.", isb_pb->logical_port,
-                             ts->name, lr->name);
-                continue;
-            }
+        struct uuid lr_uuid;
+        if (!smap_get_uuid(&isb_pb->external_ids, "router-id", &lr_uuid)) {
+            VLOG_DBG("IC-SB Port_Binding %s doesn't have "
+                     "external_ids:router-id set.", isb_pb->logical_port);
+            continue;
+        }
 
-            struct lport_addresses ts_port_addrs;
-            if (!extract_lsp_addresses(isb_pb->address, &ts_port_addrs)) {
-                static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
-                VLOG_INFO_RL(&rl, "Route sync ignores port %s on ts %s for "
-                             "router %s because the addresses are invalid.",
-                             isb_pb->logical_port, ts->name, lr->name);
-                continue;
-            }
+        const struct nbrec_logical_router *lr
+            = nbrec_logical_router_get_for_uuid(ctx->ovnnb_idl, &lr_uuid);
+        if (!lr) {
+            continue;
+        }
 
-            struct ic_router_info *ic_lr = xzalloc(sizeof *ic_lr);
+        struct ic_router_info *ic_lr = ic_router_find(&ic_lrs, lr);
+        if (!ic_lr) {
+            ic_lr = xzalloc(sizeof *ic_lr);
             ic_lr->lr = lr;
-            ic_lr->isb_pb = isb_pb;
             hmap_init(&ic_lr->routes_learned);
             hmap_insert(&ic_lrs, &ic_lr->node, uuid_hash(&lr->header_.uuid));
-
-            /* Check static routes of the LR */
-            for (int i = 0; i < lr->n_static_routes; i++) {
-                const struct nbrec_logical_router_static_route *nb_route
-                    = lr->static_routes[i];
-                struct uuid isb_uuid;
-                if (smap_get_uuid(&nb_route->external_ids,
-                                  "ic-learned-route", &isb_uuid)) {
-                    /* It is a learned route */
-                    if (!add_to_routes_learned(&ic_lr->routes_learned,
-                                               nb_route)) {
-                        static struct vlog_rate_limit rl =
-                            VLOG_RATE_LIMIT_INIT(5, 1);
-                        VLOG_WARN_RL(&rl, "Bad format of learned route in NB:"
-                                     " %s -> %s. Delete it.",
-                                     nb_route->ip_prefix, nb_route->nexthop);
-                        nbrec_logical_router_update_static_routes_delvalue(
-                            lr, nb_route);
-                    }
-                } else {
-                    /* It may be a route to be advertised */
-                    add_to_routes_ad(&routes_ad, nb_route, &ts_port_addrs,
-                                     &nb_global->options);
-                }
-            }
-
-            /* Check direct-connected subnets of the LR */
-            for (int i = 0; i < lr->n_ports; i++) {
-                const struct nbrec_logical_router_port *lrp = lr->ports[i];
-                if (!strcmp(lrp->name, ts_lrp_name)) {
-                    /* The router port of the TS port is ignored. */
-                    VLOG_DBG("Route ad: skip lrp %s (TS port: %s)",
-                             lrp->name, isb_pb->logical_port);
-                    continue;
-                }
-
-                for (int j = 0; j < lrp->n_networks; j++) {
-                    add_network_to_routes_ad(&routes_ad, lrp->networks[j],
-                                             lrp, &ts_port_addrs,
-                                             &nb_global->options);
-                }
-            }
-
-            destroy_lport_addresses(&ts_port_addrs);
         }
-        icsbrec_port_binding_index_destroy_row(isb_pb_key);
-
-        advertise_route(ctx, az, ts->name, &routes_ad);
-        hmap_destroy(&routes_ad);
 
-        struct ic_router_info *ic_lr, *next;
-        HMAP_FOR_EACH_SAFE (ic_lr, next, node, &ic_lrs) {
-            sync_learned_route(ctx, az, ic_lr);
-            hmap_destroy(&ic_lr->routes_learned);
-            hmap_remove(&ic_lrs, &ic_lr->node);
-            free(ic_lr);
+        if (ic_lr->n_isb_pbs == ic_lr->n_allocated_isb_pbs) {
+            ic_lr->isb_pbs = x2nrealloc(ic_lr->isb_pbs,
+                                        &ic_lr->n_allocated_isb_pbs,
+                                        sizeof *ic_lr->isb_pbs);
         }
-        hmap_destroy(&ic_lrs);
+        ic_lr->isb_pbs[ic_lr->n_isb_pbs++] = isb_pb;
     }
+    icsbrec_port_binding_index_destroy_row(isb_pb_key);
+
+    struct ic_router_info *ic_lr, *next;
+    HMAP_FOR_EACH_SAFE (ic_lr, next, node, &ic_lrs) {
+        advertise_lr_routes(ctx, az, ic_lr);
+        sync_learned_routes(ctx, az, ic_lr);
+        free(ic_lr->isb_pbs);
+        hmap_destroy(&ic_lr->routes_learned);
+        hmap_remove(&ic_lrs, &ic_lr->node);
+        free(ic_lr);
+    }
+    hmap_destroy(&ic_lrs);
 }
 
 static void
@@ -1697,6 +1839,9 @@ main(int argc, char *argv[])
     struct ovsdb_idl_index *nbrec_port_by_name
         = ovsdb_idl_index_create1(ovnnb_idl_loop.idl,
                                   &nbrec_logical_switch_port_col_name);
+    struct ovsdb_idl_index *nbrec_lrp_by_name
+        = ovsdb_idl_index_create1(ovnnb_idl_loop.idl,
+                                  &nbrec_logical_router_port_col_name);
     struct ovsdb_idl_index *sbrec_port_binding_by_name
         = ovsdb_idl_index_create1(ovnsb_idl_loop.idl,
                                   &sbrec_port_binding_col_logical_port);
@@ -1704,6 +1849,10 @@ main(int argc, char *argv[])
         = ovsdb_idl_index_create1(ovnsb_idl_loop.idl,
                                   &sbrec_chassis_col_name);
 
+    struct ovsdb_idl_index *icnbrec_transit_switch_by_name
+        = ovsdb_idl_index_create1(ovninb_idl_loop.idl,
+                                  &icnbrec_transit_switch_col_name);
+
     struct ovsdb_idl_index *icsbrec_port_binding_by_az
         = ovsdb_idl_index_create1(ovnisb_idl_loop.idl,
                                   &icsbrec_port_binding_col_availability_zone);
@@ -1762,9 +1911,12 @@ main(int argc, char *argv[])
                 .ovnisb_idl = ovnisb_idl_loop.idl,
                 .ovnisb_txn = ovsdb_idl_loop_run(&ovnisb_idl_loop),
                 .nbrec_ls_by_name = nbrec_ls_by_name,
+                .nbrec_lrp_by_name = nbrec_lrp_by_name,
                 .nbrec_port_by_name = nbrec_port_by_name,
                 .sbrec_port_binding_by_name = sbrec_port_binding_by_name,
                 .sbrec_chassis_by_name = sbrec_chassis_by_name,
+                .icnbrec_transit_switch_by_name =
+                    icnbrec_transit_switch_by_name,
                 .icsbrec_port_binding_by_az = icsbrec_port_binding_by_az,
                 .icsbrec_port_binding_by_ts = icsbrec_port_binding_by_ts,
                 .icsbrec_port_binding_by_ts_az = icsbrec_port_binding_by_ts_az,
diff --git a/ovn-ic-sb.ovsschema b/ovn-ic-sb.ovsschema
index 5364b21b4..140ced149 100644
--- a/ovn-ic-sb.ovsschema
+++ b/ovn-ic-sb.ovsschema
@@ -1,7 +1,7 @@
 {
     "name": "OVN_IC_Southbound",
-    "version": "1.0.0",
-    "cksum": "108951192 6585",
+    "version": "1.1.0",
+    "cksum": "3076915724 6636",
     "tables": {
         "IC_SB_Global": {
             "columns": {
@@ -92,6 +92,7 @@
                 "transit_switch": {"type": "string"},
                 "availability_zone": {"type": {"key": {"type": "uuid",
                                       "refTable": "Availability_Zone"}}},
+                "route_table": {"type": "string"},
                 "ip_prefix": {"type": "string"},
                 "nexthop": {"type": "string"},
                 "external_ids": {
diff --git a/ovn-ic-sb.xml b/ovn-ic-sb.xml
index 3582cff47..2f2ecf952 100644
--- a/ovn-ic-sb.xml
+++ b/ovn-ic-sb.xml
@@ -306,6 +306,24 @@
         The availability zone that has advertised the route.
       
 
+      
+        Route table within which this route was created.
+        Empty value means "global" routing table.
+        
    
+        Routes for directly-connected networks will be
+        learned to global routing table and if Logical Routers
+        have more than one Transit Switch, which interconnects
+        them, directly-connected routes will be added via each
+        transit switch port and configured as ECMP routes.
+        
    
+        
    
+        Static routes within route tables will be advertised
+        and learned only if interconnecting transit switch's LRPs
+        will have options:route_table same value as
+        route's route_table value.
+        
    
+          
+
       
         IP prefix of this route (e.g. 192.168.100.0/24).
       
diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at
index 3aab54362..5803f76e9 100644
--- a/tests/ovn-ic.at
+++ b/tests/ovn-ic.at
@@ -430,3 +430,443 @@ OVN_CLEANUP_IC([az1], [az2])
 
 AT_CLEANUP
 ])
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([ovn-ic -- route sync -- route tables])
+
+ovn_init_ic_db
+ovn-ic-nbctl ts-add ts1
+
+for i in 1 2; do
+    ovn_start az$i
+    ovn_as az$i
+
+    # Enable route learning at AZ level
+    ovn-nbctl set nb_global . options:ic-route-learn=true
+    # Enable route advertising at AZ level
+    ovn-nbctl set nb_global . options:ic-route-adv=true
+
+    # Create LRP and connect to TS
+    ovn-nbctl lr-add lr$i
+    ovn-nbctl lrp-add lr$i lrp-lr$i-ts1 aa:aa:aa:aa:aa:0$i 169.254.100.$i/24
+    ovn-nbctl lsp-add ts1 lsp-ts1-lr$i \
+            -- lsp-set-addresses lsp-ts1-lr$i router \
+            -- lsp-set-type lsp-ts1-lr$i router \
+            -- lsp-set-options lsp-ts1-lr$i router-port=lrp-lr$i-ts1
+
+    # Create static routes
+    ovn-nbctl lr-route-add lr$i 10.11.$i.0/24 169.254.0.1
+
+    # Create a src-ip route, which shouldn't be synced
+    ovn-nbctl --policy=src-ip --route-table=rtb1 lr-route-add lr$i 10.22.$i.0/24 169.254.0.2
+done
+
+for i in 1 2; do
+    OVS_WAIT_UNTIL([ovn_as az$i ovn-nbctl lr-route-list lr$i | grep learned])
+done
+
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl
+IPv4 Routes
+Route Table global:
+             10.11.1.0/24               169.254.0.1 dst-ip
+             10.11.2.0/24             169.254.100.2 dst-ip (learned)
+
+Route Table rtb1:
+             10.22.1.0/24               169.254.0.2 src-ip
+])
+
+# move routes from global route table to rtb1
+for i in 1 2; do
+    ovn_as az$i ovn-nbctl lr-route-del lr$i 10.11.$i.0/24 169.254.0.1
+    ovn_as az$i ovn-nbctl --route-table=rtb1 lr-route-add lr$i 10.11.$i.0/24 169.254.0.1
+done
+
+for i in 1 2; do
+    OVS_WAIT_WHILE([ovn_as az$i ovn-nbctl lr-route-list lr$i | grep learned])
+done
+
+# ensure route from rtb1 is not learned to any route table as route table is
+# not set to TS port
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl
+IPv4 Routes
+Route Table rtb1:
+             10.11.1.0/24               169.254.0.1 dst-ip
+             10.22.1.0/24               169.254.0.2 src-ip
+])
+
+# assign route table rtb1 to TS port on AZ2 and check routes are advertised to IC SB DB
+check ovn_as az2 ovn-nbctl lrp-set-options lrp-lr2-ts1 route_table=rtb1
+OVS_WAIT_UNTIL([ovn-ic-sbctl find route route_table=rtb1 | grep 10.11.2.0/24])
+
+# ensure route was not learned as on AZ1 TS port's LRP was not set to route table rtb1
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl
+IPv4 Routes
+Route Table rtb1:
+             10.11.1.0/24               169.254.0.1 dst-ip
+             10.22.1.0/24               169.254.0.2 src-ip
+])
+
+# set TS port's LRP to route table rtb1 to learn routes from AZ2 from rtb1
+check ovn_as az1 ovn-nbctl lrp-set-options lrp-lr1-ts1 route_table=rtb1
+
+OVS_WAIT_UNTIL([ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-list lr1 | grep learned])
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl
+IPv4 Routes
+Route Table rtb1:
+             10.11.1.0/24               169.254.0.1 dst-ip
+             10.11.2.0/24             169.254.100.2 dst-ip (learned)
+             10.22.1.0/24               169.254.0.2 src-ip
+])
+
+# Delete route in AZ1, AZ2's learned route should be deleted.
+ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-del lr1 10.11.1.0/24
+OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-list lr2 | grep learned])
+
+# Add the route back
+ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-add lr1 10.11.1.0/24 169.254.0.1
+OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned])
+
+# Disable route-learning for AZ1
+ovn_as az1 ovn-nbctl set nb_global . options:ic-route-learn=false
+OVS_WAIT_WHILE([ovn_as az1 ovn-nbctl lr-route-list lr1 | grep learned])
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl
+IPv4 Routes
+Route Table rtb1:
+             10.11.1.0/24               169.254.0.1 dst-ip
+             10.22.1.0/24               169.254.0.2 src-ip
+])
+
+# AZ1 should still advertise and AZ2 should still learn the route
+AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned], [0], [ignore])
+AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl
+IPv4 Routes
+Route Table rtb1:
+             10.11.1.0/24             169.254.100.1 dst-ip (learned)
+             10.11.2.0/24               169.254.0.1 dst-ip
+             10.22.2.0/24               169.254.0.2 src-ip
+])
+
+# Disable route-advertising for AZ1
+ovn_as az1 ovn-nbctl set nb_global . options:ic-route-adv=false
+
+# AZ2 shouldn't have the route learned, because AZ1 have stopped advertising.
+OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned])
+
+# Add default route in AZ1
+ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-add lr1 0.0.0.0/0 169.254.0.3
+
+# Re-enable router-advertising & learn for AZ1
+ovn_as az1 ovn-nbctl set nb_global . options:ic-route-adv=true
+ovn_as az1 ovn-nbctl set nb_global . options:ic-route-learn=true
+
+for i in 1 2; do
+    OVS_WAIT_UNTIL([ovn_as az$i ovn-nbctl lr-route-list lr$i | grep learned])
+done
+
+# Default route should NOT get advertised or learned, by default.
+AT_CHECK([ovn-ic-sbctl find route ip_prefix="0.0.0.0/0"], [0], [])
+
+# Enable default route advertising in AZ1, ensure it advertised, but not learned
+ovn_as az1 ovn-nbctl set nb_global . options:ic-route-adv-default=true
+OVS_WAIT_UNTIL([ovn-ic-sbctl find route ip_prefix="0.0.0.0/0" route_table=rtb1 | grep 0.0.0.0])
+OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-list lr2 | grep learned | grep 0.0.0.0])
+
+# Enable default route learning in AZ2
+ovn_as az2 ovn-nbctl set nb_global . options:ic-route-learn-default=true
+OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-list lr2 | grep learned | grep 0.0.0.0])
+
+# Test directly connected subnet route advertising. Route should go to global route table.
+ovn_as az1 ovn-nbctl lrp-add lr1 lrp-lr1-ls1 aa:aa:aa:aa:bb:01 "192.168.0.1/24"
+OVS_WAIT_UNTIL([ovn-ic-sbctl find route ip_prefix="192.168.0.1/24" route_table="\"\"" | grep 192.168.0.1/24])
+OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned | grep 192.168])
+AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl
+IPv4 Routes
+Route Table global:
+           192.168.0.0/24             169.254.100.1 dst-ip (learned)
+
+Route Table rtb1:
+             10.11.1.0/24             169.254.100.1 dst-ip (learned)
+             10.11.2.0/24               169.254.0.1 dst-ip
+             10.22.2.0/24               169.254.0.2 src-ip
+                0.0.0.0/0             169.254.100.1 dst-ip (learned)
+])
+
+# Delete the directly connected subnet from AZ1, learned route should be
+# removed from AZ2.
+ovn_as az1 ovn-nbctl lrp-del lrp-lr1-ls1
+OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned | grep 192.168])
+
+# Test blacklist routes
+# Add back the directly connected 192.168 route.
+ovn_as az1 ovn-nbctl lrp-add lr1 lrp-lr1-ls1 aa:aa:aa:aa:bb:01 "192.168.0.1/24"
+OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned | grep 192.168])
+# Now add 10.11.0.0/16 and 192.168.0.0/16 to blacklist in AZ2.
+check ovn_as az2 ovn-nbctl set nb_global . options:ic-route-blacklist="10.11.0.0/16,192.168.0.0/16"
+# AZ2 shouldn't learn 192.168 route any more.
+OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned | grep 192.168])
+# AZ1 shouldn't learn 10.11 any more.
+OVS_WAIT_WHILE([ovn_as az1 ovn-nbctl lr-route-list lr1 | grep learned | grep 10.11])
+AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl
+IPv4 Routes
+Route Table rtb1:
+             10.11.2.0/24               169.254.0.1 dst-ip
+             10.22.2.0/24               169.254.0.2 src-ip
+                0.0.0.0/0             169.254.100.1 dst-ip (learned)
+])
+
+OVN_CLEANUP_IC([az1], [az2])
+
+AT_CLEANUP
+])
+
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([ovn-ic -- route sync -- multiple route tables])
+
+ovn_init_ic_db
+ovn-ic-nbctl ts-add ts1
+
+for i in 1 2; do
+    ovn_start az$i
+    ovn_as az$i
+
+    # Enable route learning at AZ level
+    ovn-nbctl set nb_global . options:ic-route-learn=true
+    # Enable route advertising at AZ level
+    ovn-nbctl set nb_global . options:ic-route-adv=true
+done
+
+# Create new transit switches and LRs. Test topology is next:
+# VPC1:
+#                       / transit switch (ts11) \
+# logical router (lr11) - transit switch (ts12) - logical router (lr12)
+#                       \ transit switch (ts13) /
+#
+# VPC2:
+#                       / transit switch (ts21) \
+# logical router (lr21)                           logical router (lr22)
+#                       \ transit switch (ts22) /
+#
+# each LR has one connected subnet except TS port
+
+
+# VPC1
+# create lr11, lr12, ts11, ts12, ts13 and connect them
+# assign route tables rtb1, rtb2, rtb3 to ts ports
+for i in 1 2; do
+    ovn_as az$i
+
+    lr=lr1$i
+    ovn-nbctl lr-add $lr
+
+    for j in 1 2 3; do
+        ts=ts1$j
+        ovn-ic-nbctl --may-exist ts-add $ts
+
+        lrp=lrp-$lr-$ts
+        lsp=lsp-$ts-$lr
+        # Create LRP and connect to TS
+        ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:a$j:0$i 169.254.10$j.$i/24
+        ovn-nbctl lrp-set-options $lrp route_table=rtb$j
+        ovn-nbctl lsp-add $ts $lsp \
+                -- lsp-set-addresses $lsp router \
+                -- lsp-set-type $lsp router \
+                -- lsp-set-options $lsp router-port=$lrp
+    done
+done
+
+# VPC2
+# create lr21, lr22, ts21, ts22 and connect them
+# assign route tables rtb1, rtb2, rtb3 to ts ports
+for i in 1 2; do
+    ovn_as az$i
+
+    lr=lr2$i
+    ovn-nbctl lr-add $lr
+
+    for j in 1 2; do
+        ts=ts2$j
+        ovn-ic-nbctl --may-exist ts-add $ts
+
+        lrp=lrp-$lr-$ts
+        lsp=lsp-$ts-$lr
+        # Create LRP and connect to TS
+        ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:a$j:0$i 169.254.10$j.$i/24
+        ovn-nbctl lrp-set-options $lrp route_table=rtb$j
+        ovn-nbctl lsp-add $ts $lsp \
+                -- lsp-set-addresses $lsp router \
+                -- lsp-set-type $lsp router \
+                -- lsp-set-options $lsp router-port=$lrp
+    done
+done
+
+# Create directly-connected and static routes in VPC1
+ovn_as az2 ovn-nbctl lrp-add lr12 lrp-lr12 aa:aa:aa:aa:bb:01 "192.168.0.1/24"
+ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-add lr12 10.10.10.0/24 192.168.0.10
+ovn_as az2 ovn-nbctl --route-table=rtb2 lr-route-add lr12 10.10.10.0/24 192.168.0.11
+ovn_as az2 ovn-nbctl --route-table=rtb3 lr-route-add lr12 10.10.10.0/24 192.168.0.12
+
+# Create directly-connected route in VPC2
+ovn_as az2 ovn-nbctl lrp-add lr22 lrp-lr22 aa:aa:aa:aa:bb:01 "192.168.0.1/24"
+
+# Test direct routes from lr12 were learned to lr11
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr11 | grep 192.168 |
+             grep learned | awk '{print $1, $2, $5}' | sort ], [0], [dnl
+192.168.0.0/24 169.254.101.2 ecmp
+192.168.0.0/24 169.254.102.2 ecmp
+192.168.0.0/24 169.254.103.2 ecmp
+])
+
+# Test static routes from lr12 rtbs rtb1,rtb2,rtb3 were learned to lr11
+AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-list lr11], [0], [dnl
+IPv4 Routes
+Route Table rtb1:
+            10.10.10.0/24             169.254.101.2 dst-ip (learned)
+])
+AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb2 lr-route-list lr11], [0], [dnl
+IPv4 Routes
+Route Table rtb2:
+            10.10.10.0/24             169.254.102.2 dst-ip (learned)
+])
+AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb3 lr-route-list lr11], [0], [dnl
+IPv4 Routes
+Route Table rtb3:
+            10.10.10.0/24             169.254.103.2 dst-ip (learned)
+])
+
+# Test routes from lr12 didn't leak as learned to lr21
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr21 | grep 192.168 | sort], [0], [dnl
+           192.168.0.0/24             169.254.101.2 dst-ip (learned) ecmp
+           192.168.0.0/24             169.254.102.2 dst-ip (learned) ecmp
+])
+
+OVN_CLEANUP_IC([az1], [az2])
+
+AT_CLEANUP
+])
+
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([ovn-ic -- route sync -- multiple route tables IPv6])
+
+ovn_init_ic_db
+ovn-ic-nbctl ts-add ts1
+
+for i in 1 2; do
+    ovn_start az$i
+    ovn_as az$i
+
+    # Enable route learning at AZ level
+    ovn-nbctl set nb_global . options:ic-route-learn=true
+    # Enable route advertising at AZ level
+    ovn-nbctl set nb_global . options:ic-route-adv=true
+done
+
+# Create new transit switches and LRs. Test topology is next:
+# VPC1:
+#                       / transit switch (ts11) \
+# logical router (lr11) - transit switch (ts12) - logical router (lr12)
+#                       \ transit switch (ts13) /
+#
+# VPC2:
+#                       / transit switch (ts21) \
+# logical router (lr21)                           logical router (lr22)
+#                       \ transit switch (ts22) /
+#
+# each LR has one connected subnet except TS port
+
+
+# VPC1
+# create lr11, lr12, ts11, ts12, ts13 and connect them
+# assign route tables rtb1, rtb2, rtb3 to ts ports
+for i in 1 2; do
+    ovn_as az$i
+
+    lr=lr1$i
+    ovn-nbctl lr-add $lr
+
+    for j in 1 2 3; do
+        ts=ts1$j
+        ovn-ic-nbctl --may-exist ts-add $ts
+
+        lrp=lrp-$lr-$ts
+        lsp=lsp-$ts-$lr
+        # Create LRP and connect to TS
+        ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:a$j:0$i 2001:db8:$j::$i/64
+        ovn-nbctl lrp-set-options $lrp route_table=rtb$j
+        ovn-nbctl lsp-add $ts $lsp \
+                -- lsp-set-addresses $lsp router \
+                -- lsp-set-type $lsp router \
+                -- lsp-set-options $lsp router-port=$lrp
+    done
+done
+
+# VPC2
+# create lr21, lr22, ts21, ts22 and connect them
+# assign route tables rtb1, rtb2, rtb3 to ts ports
+for i in 1 2; do
+    ovn_as az$i
+
+    lr=lr2$i
+    ovn-nbctl lr-add $lr
+
+    for j in 1 2; do
+        ts=ts2$j
+        ovn-ic-nbctl --may-exist ts-add $ts
+
+        lrp=lrp-$lr-$ts
+        lsp=lsp-$ts-$lr
+        # Create LRP and connect to TS
+        ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:a$j:0$i 2001:db8:$j::$i/64
+        ovn-nbctl lrp-set-options $lrp route_table=rtb$j
+        ovn-nbctl lsp-add $ts $lsp \
+                -- lsp-set-addresses $lsp router \
+                -- lsp-set-type $lsp router \
+                -- lsp-set-options $lsp router-port=$lrp
+    done
+done
+
+# Create directly-connected and static routes in VPC1
+ovn_as az2 ovn-nbctl lrp-add lr12 lrp-lr12 aa:aa:aa:aa:bb:01 "2001:db8:200::1/64"
+ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-add lr12 2001:db8:aaaa::/64 2001:db8:200::10
+ovn_as az2 ovn-nbctl --route-table=rtb2 lr-route-add lr12 2001:db8:aaaa::/64 2001:db8:200::11
+ovn_as az2 ovn-nbctl --route-table=rtb3 lr-route-add lr12 2001:db8:aaaa::/64 2001:db8:200::12
+
+# Create directly-connected route in VPC2
+ovn_as az2 ovn-nbctl lrp-add lr22 lrp-lr22 aa:aa:aa:aa:bb:01 "2001:db8:200::1/64"
+
+# Test direct routes from lr12 were learned to lr11
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr11 | grep 2001:db8:200 |
+             grep learned | awk '{print $1, $2, $5}' | sort], [0], [dnl
+2001:db8:200::/64 2001:db8:1::2 ecmp
+2001:db8:200::/64 2001:db8:2::2 ecmp
+2001:db8:200::/64 2001:db8:3::2 ecmp
+])
+
+# Test static routes from lr12 rtbs rtb1,rtb2,rtb3 were learned to lr11
+AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-list lr11], [0], [dnl
+IPv6 Routes
+Route Table rtb1:
+       2001:db8:aaaa::/64             2001:db8:1::2 dst-ip (learned)
+])
+AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb2 lr-route-list lr11], [0], [dnl
+IPv6 Routes
+Route Table rtb2:
+       2001:db8:aaaa::/64             2001:db8:2::2 dst-ip (learned)
+])
+AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb3 lr-route-list lr11], [0], [dnl
+IPv6 Routes
+Route Table rtb3:
+       2001:db8:aaaa::/64             2001:db8:3::2 dst-ip (learned)
+])
+
+# Test routes from lr12 didn't leak as learned to lr21
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr21 | grep 2001 | sort], [0], [dnl
+        2001:db8:200::/64             2001:db8:1::2 dst-ip (learned) ecmp
+        2001:db8:200::/64             2001:db8:2::2 dst-ip (learned) ecmp
+])
+
+OVN_CLEANUP_IC([az1], [az2])
+
+AT_CLEANUP
+])

From patchwork Sun Sep 19 21:23:16 2021
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Patchwork-Submitter: Vladislav Odintsov 
X-Patchwork-Id: 1529887
X-Patchwork-Delegate: zhouhan@gmail.com
Return-Path: 
X-Original-To: incoming@patchwork.ozlabs.org
Delivered-To: patchwork-incoming@ozlabs.org
Authentication-Results: ozlabs.org;
	dkim=fail reason="signature verification failed" (2048-bit key;
 unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256
 header.s=20210112 header.b=HgiykVAy;
	dkim-atps=neutral
Authentication-Results: ozlabs.org;
 spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org
 (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org;
 envelope-from=ovs-dev-bounces@openvswitch.org; receiver=)
Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest
 SHA256)
	(No client certificate requested)
	by ozlabs.org (Postfix) with ESMTPS id 4HCLKM5qk2z9sW5
	for ; Mon, 20 Sep 2021 07:23:51 +1000 (AEST)
Received: from localhost (localhost [127.0.0.1])
	by smtp3.osuosl.org (Postfix) with ESMTP id 3781E607A6;
	Sun, 19 Sep 2021 21:23:49 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
Received: from smtp3.osuosl.org ([127.0.0.1])
	by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id iYvJf6NJ9Pbx; Sun, 19 Sep 2021 21:23:48 +0000 (UTC)
Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56])
	by smtp3.osuosl.org (Postfix) with ESMTPS id 53C9060071;
	Sun, 19 Sep 2021 21:23:47 +0000 (UTC)
Received: from lf-lists.osuosl.org (localhost [127.0.0.1])
	by lists.linuxfoundation.org (Postfix) with ESMTP id 2D802C0011;
	Sun, 19 Sep 2021 21:23:47 +0000 (UTC)
X-Original-To: dev@openvswitch.org
Delivered-To: ovs-dev@lists.linuxfoundation.org
Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136])
 by lists.linuxfoundation.org (Postfix) with ESMTP id 53679C000D
 for ; Sun, 19 Sep 2021 21:23:46 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by smtp3.osuosl.org (Postfix) with ESMTP id 372316075E
 for ; Sun, 19 Sep 2021 21:23:46 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
Received: from smtp3.osuosl.org ([127.0.0.1])
 by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id derj2rfUEngq for ;
 Sun, 19 Sep 2021 21:23:45 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.8.0
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [IPv6:2a00:1450:4864:20::12e])
 by smtp3.osuosl.org (Postfix) with ESMTPS id 20AC6606BE
 for ; Sun, 19 Sep 2021 21:23:44 +0000 (UTC)
Received: by mail-lf1-x12e.google.com with SMTP id i4so58985098lfv.4
 for ; Sun, 19 Sep 2021 14:23:44 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=from:to:cc:subject:date:message-id:in-reply-to:references
 :mime-version:content-transfer-encoding;
 bh=+XxhW5pkzXfXVtuT/d0nUdcixOWtnAaPs8ReEJq4scA=;
 b=HgiykVAyX/rvQwFVTAtrJbw8wN6l92vQPJQoiewMXB/bvRpL8eU7Do1OANaPYruiwB
 xbHyHaNNDefoSBb7tJnIXeRWGJvv2GiuXdwrTClF4Tsy7DjwkbUJBr3xxTv2IK4ocCJW
 slqai/lKRQEnECe5lK4X+dgiSRm8ouio1HBFYzEmNTN51yImoFT/OUcmqNKvDyPeFCJn
 /7lM3hwyxlWBNT6Y2ceVa4lvI/856PRXoyzq2wSF3SMQ3AOinD58o6J/Q4jsc+Cdqo2K
 Jn8u1hfqB/HXkgvR1rLajXqD8RYAYEFI9EDr1DPUh2KASuKPfP1jW0jWp7CctIwr3jBJ
 ELXw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
 :references:mime-version:content-transfer-encoding;
 bh=+XxhW5pkzXfXVtuT/d0nUdcixOWtnAaPs8ReEJq4scA=;
 b=HEC3XQehVQerHnYQWGu8Ssqhk0mkIdykHwuJuNJ6XWRAY7C2N/BKrM/yHDlIOugRBC
 nPHUiPpKToybmGELTunwoVc0NxNq/bt5OidcNRSQN9hREGmGtWrgdNTJIYd1DDKSTRpW
 jKqwJNPHmzNPqKg4aP27Zz5kuTSdulB4Eh06Nfwc875t0JxBF7Bvfbo5NiXH+vGWKU92
 lVKWXo4CP/zyiRbLa7jr1TvX+07IhzpPwvFHEzhxnF/C7arElW/Z4O+hZzfSmr7TOZlJ
 fZvXqOO+gDgVgkxIX8q53SKN1CWqStV9QTZP+BRXBZAD6wmZcr4U/WQFdvQUkvleWKDP
 /0QQ==
X-Gm-Message-State: AOAM531KWXaVK6iUMuxeGcOBB0DyrkNXiJviARpjBD9F8EFT4R3IbFYl
 iiNYsQEne6omWo9JwXRJ/LF8cl9Pr5g=
X-Google-Smtp-Source: 
 ABdhPJw641gyqNyZQWvAScYsbuJam2S2VNsxwKti1q0NQaC8rkyosbeGBqK4tDwSut0qiTjyHAPkuA==
X-Received: by 2002:a2e:86d1:: with SMTP id
 n17mr20752373ljj.237.1632086622531;
 Sun, 19 Sep 2021 14:23:42 -0700 (PDT)
Received: from localhost.localdomain (109-252-131-59.dynamic.spd-mgts.ru.
 [109.252.131.59])
 by smtp.gmail.com with ESMTPSA id q5sm431658lfn.277.2021.09.19.14.23.42
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Sun, 19 Sep 2021 14:23:42 -0700 (PDT)
From: Vladislav Odintsov 
To: dev@openvswitch.org
Date: Mon, 20 Sep 2021 00:23:16 +0300
Message-Id: <20210919212316.24148-5-odivlad@gmail.com>
X-Mailer: git-send-email 2.30.0
In-Reply-To: <20210919212316.24148-1-odivlad@gmail.com>
References: <20210919212316.24148-1-odivlad@gmail.com>
MIME-Version: 1.0
Cc: Vladislav Odintsov 
Subject: [ovs-dev] [PATCH ovn v4 4/4] ic: don't learn routes which have
	local GW
X-BeenThere: ovs-dev@openvswitch.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: 
List-Unsubscribe: ,
 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: ,
 
Errors-To: ovs-dev-bounces@openvswitch.org
Sender: "dev" 

In case we have ovn-ic-interconnected Logical_Routers
and install same ip_prefix route with GW in local AZ
in each LR in each AZ, this route would be learned in
other AZs and L3 loop is possible.
There could be next routes output:

[az1 ~]$ ovn-nbctl lr-route-list lr0
IPv4 Routes
Route Table global:
              128.0.0.0/1               169.254.1.1 dst-ip ecmp
              128.0.0.0/1             169.254.100.2 dst-ip (learned) ecmp

[az2 ~]$ ovn-nbctl lr-route-list lr0
IPv4 Routes
Route Table global:
              128.0.0.0/1               169.254.2.1 dst-ip ecmp
              128.0.0.0/1             169.254.100.1 dst-ip (learned) ecmp

So, there is a possible routing loop. Packets going
to 128.0.0.0/1 could go from AZ1 to AZ2 and on AZ2
they can be routed back.

This commit adds check for installed local (non-learned)
routes. If OVN IC route's ip_prefix, route_table are
the same with already installed non-learned NB route, such
route wouldn't be learned.

Signed-off-by: Vladislav Odintsov 
---
 ic/ovn-ic.c           | 30 ++++++++++++++++++++++++--
 tests/ovn-ic.at       | 49 +++++++++++++++++++++++++++++++++++++++++++
 utilities/ovn-nbctl.c |  4 +++-
 3 files changed, 80 insertions(+), 3 deletions(-)

diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c
index 94f123f70..4c904400c 100644
--- a/ic/ovn-ic.c
+++ b/ic/ovn-ic.c
@@ -1197,7 +1197,25 @@ add_network_to_routes_ad(struct hmap *routes_ad, const char *network,
 }
 
 static bool
-route_need_learn(struct in6_addr *prefix, unsigned int plen,
+route_has_local_gw(const struct nbrec_logical_router *lr,
+                   const char *route_table, const char *ip_prefix) {
+
+    const struct nbrec_logical_router_static_route *route;
+    for (int i = 0; i < lr->n_static_routes; i++) {
+        route = lr->static_routes[i];
+        if (!smap_get(&route->external_ids, "ic-learned-route") &&
+            !strcmp(route->route_table, route_table) &&
+            !strcmp(route->ip_prefix, ip_prefix)) {
+            return true;
+        }
+    }
+    return false;
+}
+
+static bool
+route_need_learn(const struct nbrec_logical_router *lr,
+                 const struct icsbrec_route *isb_route,
+                 struct in6_addr *prefix, unsigned int plen,
                  const struct smap *nb_options)
 {
     if (!smap_get_bool(nb_options, "ic-route-learn", false)) {
@@ -1217,6 +1235,12 @@ route_need_learn(struct in6_addr *prefix, unsigned int plen,
         return false;
     }
 
+    if (route_has_local_gw(lr, isb_route->route_table, isb_route->ip_prefix)) {
+        VLOG_DBG("Skip learning %s (rtb:%s) route, as we've got one with "
+                 "local GW", isb_route->ip_prefix, isb_route->route_table);
+        return false;
+    }
+
     return true;
 }
 
@@ -1322,9 +1346,11 @@ sync_learned_routes(struct ic_context *ctx,
                              isb_route->nexthop);
                 continue;
             }
-            if (!route_need_learn(&prefix, plen, &nb_global->options)) {
+            if (!route_need_learn(ic_lr->lr, isb_route, &prefix, plen,
+                                  &nb_global->options)) {
                 continue;
             }
+
             struct ic_route_info *route_learned
                 = ic_route_find(&ic_lr->routes_learned, &prefix, plen,
                                 &nexthop, isb_route->route_table);
diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at
index 5803f76e9..15560334d 100644
--- a/tests/ovn-ic.at
+++ b/tests/ovn-ic.at
@@ -870,3 +870,52 @@ OVN_CLEANUP_IC([az1], [az2])
 
 AT_CLEANUP
 ])
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([ovn-ic -- same routes destination])
+
+ovn_init_ic_db
+ovn-ic-nbctl ts-add ts1
+
+for i in 1 2; do
+    ovn_start az$i
+    ovn_as az$i
+
+    # Enable route learning at AZ level
+    ovn-nbctl set nb_global . options:ic-route-learn=true
+    ovn-nbctl set nb_global . options:ic-route-learn-default=true
+    # Enable route advertising at AZ level
+    ovn-nbctl set nb_global . options:ic-route-adv=true
+    ovn-nbctl set nb_global . options:ic-route-adv-default=true
+
+    lr=lr1$i
+    ovn-nbctl lr-add $lr
+
+    lrp=lrp-$lr-ts1
+    lsp=lsp-ts1-$lr
+    # Create LRP and connect to TS
+    ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:aa:0$i 169.254.100.$i/24
+    ovn-nbctl lsp-add ts1 $lsp \
+        -- lsp-set-addresses $lsp router \
+        -- lsp-set-type $lsp router \
+        -- lsp-set-options $lsp router-port=$lrp
+    ovn-nbctl lrp-add $lr lrp-local-subnet 00:00:00:00:00:0$i 192.168.$i.1/24
+    ovn-nbctl list logical-router-static-route
+    check ovn-nbctl lr-route-add $lr 10.0.0.0/24 192.168.$i.10
+    check ovn-nbctl lr-route-add $lr 0.0.0.0/0 192.168.$i.11
+done
+
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr11 | grep dst-ip | sort], [0], [dnl
+                0.0.0.0/0              192.168.1.11 dst-ip
+              10.0.0.0/24              192.168.1.10 dst-ip
+           192.168.2.0/24             169.254.100.2 dst-ip (learned)
+])
+
+AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr12 | grep dst-ip | sort], [0], [dnl
+                0.0.0.0/0              192.168.2.11 dst-ip
+              10.0.0.0/24              192.168.2.10 dst-ip
+           192.168.1.0/24             169.254.100.1 dst-ip (learned)
+])
+
+AT_CLEANUP
+])
diff --git a/utilities/ovn-nbctl.c b/utilities/ovn-nbctl.c
index f4a8504e6..e0034d515 100644
--- a/utilities/ovn-nbctl.c
+++ b/utilities/ovn-nbctl.c
@@ -4084,6 +4084,8 @@ nbctl_pre_lr_route_add(struct ctl_context *ctx)
                          &nbrec_logical_router_static_route_col_options);
     ovsdb_idl_add_column(ctx->idl,
                          &nbrec_logical_router_static_route_col_route_table);
+    ovsdb_idl_add_column(ctx->idl,
+                         &nbrec_logical_router_static_route_col_external_ids);
 }
 
 static char * OVS_WARN_UNUSED_RESULT
@@ -4206,7 +4208,7 @@ nbctl_lr_route_add(struct ctl_context *ctx)
     }
 
     if (!ecmp) {
-        if (route) {
+        if (route && !smap_get(&route->external_ids, "ic-learned-route")) {
             if (!may_exist) {
                 ctl_error(ctx, "duplicate prefix: %s (policy: %s). Use option"
                           " --ecmp to allow this for ECMP routing.",