From patchwork Fri Sep 3 10:34:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladislav Odintsov X-Patchwork-Id: 1524200 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=mHHWroWq; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H1DhP6Qzsz9s5R for ; Fri, 3 Sep 2021 20:35:17 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id A70CB834F1; Fri, 3 Sep 2021 10:35:15 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ynbAnA0ZuE4i; Fri, 3 Sep 2021 10:35:14 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id B5ECD8381F; Fri, 3 Sep 2021 10:35:13 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8D062C0010; Fri, 3 Sep 2021 10:35:13 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id EC557C0010 for ; Fri, 3 Sep 2021 10:35:11 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id CD6E7407C3 for ; Fri, 3 Sep 2021 10:35:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Dxv08UYxviXI for ; Fri, 3 Sep 2021 10:35:10 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [IPv6:2a00:1450:4864:20::62c]) by smtp2.osuosl.org (Postfix) with ESMTPS id 3D46240184 for ; Fri, 3 Sep 2021 10:35:10 +0000 (UTC) Received: by mail-ej1-x62c.google.com with SMTP id jg16so8016951ejc.1 for ; Fri, 03 Sep 2021 03:35:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ECK3yID+iBT+zrenohsCKcKsd7z6vJcsHzu78sS2p1Y=; b=mHHWroWq/umr1YUeQyMKhpyEfp6KvYqLdt5pKj4HFl1G+6nB7saHlUpMH79qzUHlb8 8F50oACHKHkjLbMcsA8aZB6kcO2sQRNI8YDpHl9IsqAwbGUq5rIchJN+OnfHuyqb5YvB gC6LrRf4nxVVGbMJSH3P++Q6U0ZeoH+X/mhDgsyuVdio3MsCnaeNtrfnGUT4usIRvp67 1aanieZ1CPQ+LEzU8i5h5ShOQQI8whlzJKdqakqStgaYbee/ahYs0W9yu09TI841Ibsy fp5jNFbnIh4Ti+qmIdgylFSI1m2VVYp88VhEl4AuBDSUXR+BQmnvjzpxvcaLSAETtyWC sIuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ECK3yID+iBT+zrenohsCKcKsd7z6vJcsHzu78sS2p1Y=; b=ZFEqpA2FBtms+QimHKD5Bu2RIbjitOcundM4dK76uyy6FMEV39Dmv/a3DUfvNTHI57 8Fi5YkApTV9MlL3nKgMNmxcDvzzS24i6BNbmsqGQTW7muc87dOq5v39SJhEXzc++wMFE 5ZgKnebDDoT0Q2rMc8L77rfcNXIthvw6bE/3sSL8T/wCvNShWyQ/RsORnnoPpqtjKtBA UOJRwoyyYnfwZ2qLwX9b+dqnuuqCATb+Wb4wSGckxgBKc8KSPJkYeyXWbS1ixcnx0f3M 6sQ91LHD40ogckNOzrhWhMpyBbAO/ycdYeybj18V94x+LG/n8Jx6AYA1bbVBhpz3LLqT b9bA== X-Gm-Message-State: AOAM533EJSLVAdOREp83SHHPsTspikIvVivci9h5DUSejgpuxIkDYpq1 gVNR6ocP5T1/izwogZZRogNEgIDzHfQ= X-Google-Smtp-Source: ABdhPJwr7A3bKImyBO4HgHRm7EF2113X9GZ29142ERHgz57/9HKf0uwyYkojUBP7kyIE7yqOTaaJHQ== X-Received: by 2002:a17:906:c18c:: with SMTP id g12mr3619991ejz.458.1630665308261; Fri, 03 Sep 2021 03:35:08 -0700 (PDT) Received: from localhost.localdomain ([85.187.18.127]) by smtp.gmail.com with ESMTPSA id d22sm2609897ejj.47.2021.09.03.03.35.07 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Sep 2021 03:35:07 -0700 (PDT) From: Vladislav Odintsov To: dev@openvswitch.org Date: Fri, 3 Sep 2021 13:34:36 +0300 Message-Id: <20210903103441.33178-2-odivlad@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210903103441.33178-1-odivlad@gmail.com> References: <20210903103441.33178-1-odivlad@gmail.com> MIME-Version: 1.0 Cc: Vladislav Odintsov Subject: [ovs-dev] [PATCH ovn v2 1/6] ic: remove port_binding on ts deletion X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" When IC port_binding exists and transit switch is deleted, the orphan port_binding if left in the IC_SB_DB. This patch fixes such situation and adds test for this case. Signed-off-by: Vladislav Odintsov --- ic/ovn-ic.c | 35 +++++++++++++++++++++++++++++++-- tests/ovn-ic.at | 52 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 85 insertions(+), 2 deletions(-) diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c index 9eb792ea5..418f19147 100644 --- a/ic/ovn-ic.c +++ b/ic/ovn-ic.c @@ -66,6 +66,7 @@ struct ic_context { struct ovsdb_idl_index *nbrec_port_by_name; struct ovsdb_idl_index *sbrec_chassis_by_name; struct ovsdb_idl_index *sbrec_port_binding_by_name; + struct ovsdb_idl_index *icsbrec_port_binding_by_az; struct ovsdb_idl_index *icsbrec_port_binding_by_ts; struct ovsdb_idl_index *icsbrec_route_by_ts; struct ovsdb_idl_index *icsbrec_route_by_ts_az; @@ -175,7 +176,7 @@ allocate_ts_dp_key(struct hmap *dp_tnlids) } static void -ts_run(struct ic_context *ctx) +ts_run(struct ic_context *ctx, const struct icsbrec_availability_zone *az) { const struct icnbrec_transit_switch *ts; @@ -240,8 +241,25 @@ ts_run(struct ic_context *ctx) * SB, to avoid uncommitted ISB datapath tunnel key to be synced back to * AZ. */ if (ctx->ovnisb_txn) { + struct shash isb_pbs = SHASH_INITIALIZER(&isb_pbs); + const struct icsbrec_port_binding *isb_pb; + const struct icsbrec_port_binding *isb_pb_key = + icsbrec_port_binding_index_init_row( + ctx->icsbrec_port_binding_by_az); + + icsbrec_port_binding_index_set_availability_zone(isb_pb_key, az); + + ICSBREC_PORT_BINDING_FOR_EACH (isb_pb, ctx->ovnisb_idl) { + shash_add(&isb_pbs, isb_pb->transit_switch, isb_pb); + } + /* Create ISB Datapath_Binding */ ICNBREC_TRANSIT_SWITCH_FOR_EACH (ts, ctx->ovninb_idl) { + while (shash_find_and_delete(&isb_pbs, ts->name)) { + /* There may be multiple Port_Bindings */ + continue; + } + isb_dp = shash_find_and_delete(&isb_dps, ts->name); if (!isb_dp) { /* Allocate tunnel key */ @@ -261,6 +279,13 @@ ts_run(struct ic_context *ctx) SHASH_FOR_EACH (node, &isb_dps) { icsbrec_datapath_binding_delete(node->data); } + + SHASH_FOR_EACH (node, &isb_pbs) { + icsbrec_port_binding_delete(node->data); + } + + icsbrec_port_binding_index_destroy_row(isb_pb_key); + shash_destroy(&isb_pbs); } ovn_destroy_tnlids(&dp_tnlids); shash_destroy(&isb_dps); @@ -1493,7 +1518,7 @@ ovn_db_run(struct ic_context *ctx) return; } - ts_run(ctx); + ts_run(ctx, az); gateway_run(ctx, az); port_binding_run(ctx, az); route_run(ctx, az); @@ -1684,6 +1709,11 @@ main(int argc, char *argv[]) struct ovsdb_idl_index *sbrec_chassis_by_name = ovsdb_idl_index_create1(ovnsb_idl_loop.idl, &sbrec_chassis_col_name); + + struct ovsdb_idl_index *icsbrec_port_binding_by_az + = ovsdb_idl_index_create1(ovnisb_idl_loop.idl, + &icsbrec_port_binding_col_availability_zone); + struct ovsdb_idl_index *icsbrec_port_binding_by_ts = ovsdb_idl_index_create1(ovnisb_idl_loop.idl, &icsbrec_port_binding_col_transit_switch); @@ -1736,6 +1766,7 @@ main(int argc, char *argv[]) .nbrec_port_by_name = nbrec_port_by_name, .sbrec_port_binding_by_name = sbrec_port_binding_by_name, .sbrec_chassis_by_name = sbrec_chassis_by_name, + .icsbrec_port_binding_by_az = icsbrec_port_binding_by_az, .icsbrec_port_binding_by_ts = icsbrec_port_binding_by_ts, .icsbrec_route_by_ts = icsbrec_route_by_ts, .icsbrec_route_by_ts_az = icsbrec_route_by_ts_az, diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at index ee78f4794..b6a8edb68 100644 --- a/tests/ovn-ic.at +++ b/tests/ovn-ic.at @@ -64,6 +64,58 @@ OVN_CLEANUP_IC([az1]) AT_CLEANUP ]) + +AT_SETUP([ovn-ic -- port bindings]) + +ovn_init_ic_db +net_add n1 + +# 1 GW per AZ +for i in 1 2; do + az=az$i + ovn_start $az + sim_add gw-$az + as gw-$az + check ovs-vsctl add-br br-phys + ovn_az_attach $az n1 br-phys 192.168.1.$i + check ovs-vsctl set open . external-ids:ovn-is-interconn=true +done + +ovn_as az1 + +# create transit switch and connect to LR +check ovn-ic-nbctl ts-add ts1 +check ovn-nbctl lr-add lr1 +check ovn-nbctl lrp-add lr1 lrp1 00:00:00:00:00:01 10.0.0.1/24 +check ovn-nbctl lrp-set-gateway-chassis lrp1 gw-az1 + +check ovn-nbctl lsp-add ts1 lsp1 -- \ + lsp-set-addresses lsp1 router -- \ + lsp-set-type lsp1 router -- \ + lsp-set-options lsp1 router-port=lrp1 + +OVS_WAIT_UNTIL([ovn-sbctl list datapath_binding | grep interconn-ts | grep ts1]) + +# check port binding appeared +AT_CHECK([ovn-ic-sbctl show | grep -A2 lsp1], [0], [dnl + port lsp1 + transit switch: ts1 + address: [["00:00:00:00:00:01 10.0.0.1/24"]] +]) + +# remove transit switch and check if port_binding is deleted +check ovn-ic-nbctl ts-del ts1 +OVS_WAIT_UNTIL([test -z "$(ovn-ic-sbctl show | grep lsp1)"]) + +for i in 1 2; do + az=az$i + OVN_CLEANUP_SBOX(gw-$az) + OVN_CLEANUP_AZ([$az]) +done +OVN_CLEANUP_IC +AT_CLEANUP + + OVN_FOR_EACH_NORTHD([ AT_SETUP([ovn-ic -- gateway sync]) From patchwork Fri Sep 3 10:34:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladislav Odintsov X-Patchwork-Id: 1524201 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=N/qWLdEP; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H1DhV2TMFz9s5R for ; Fri, 3 Sep 2021 20:35:22 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 2582B83B53; Fri, 3 Sep 2021 10:35:18 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 51YSmM2dPig6; Fri, 3 Sep 2021 10:35:17 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 3CF4783AF0; Fri, 3 Sep 2021 10:35:16 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 869F4C0029; Fri, 3 Sep 2021 10:35:15 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2140DC0028 for ; Fri, 3 Sep 2021 10:35:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 0324261551 for ; Fri, 3 Sep 2021 10:35:15 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BRmN5o2qDQ7c for ; Fri, 3 Sep 2021 10:35:13 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by smtp3.osuosl.org (Postfix) with ESMTPS id D7591606DB for ; Fri, 3 Sep 2021 10:35:12 +0000 (UTC) Received: by mail-ed1-x535.google.com with SMTP id j13so7320641edv.13 for ; Fri, 03 Sep 2021 03:35:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4drxCxjH0H4aweSrRyiQ6GVys7VxmBtD38/0Xgn44Os=; b=N/qWLdEPa0nq8H7ql9wRDziWYLQBC1IpQmIUVVrrGLkmbzxdORSP1SP/ee11fHX0q9 hKiZJYOwPStwVdE3GaIurnnxbF24mAxekZRJ1PsYQzeBIublpB0vDtb/srcXnUoqgmBW H0brf3UDXXuS8OaixiktU0qFHkFNhuBWYW1qvFQnHXnz7776hjyTb3yPi1YSTVB5JXB6 mj2fA229KUqJQrPy76IjSAwv5tgSG8Zst8WZG6GMv5EGLXYiio9pjhowCYC2oINIpL/y kzS7N667WWwnyngQf9J1QmnmMmZsVQvRS6bn9Gotk/zEC8axSjGOWmAMVbY8hywFr0bE 9POQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4drxCxjH0H4aweSrRyiQ6GVys7VxmBtD38/0Xgn44Os=; b=DbNv8VjgHOuZNGidGPrg8PV1B+0RPyE2Ay1PVius7c7dV/VVHxXV1xOwLn6wN2C7Js VUGRNQYAWpkSrfbgXs2+8NvdPjDhmPTQvNQfsAkcPhG+8FyUgECBmkjDy0jDdQPgDIGt juE6WGDUxS7z//nfpDvIdpHEiEHHLIYyvdFOZbd12SPfakK9KZfa61Y6b/knHqFpXRTp TT38l2GSlZsIkjGWBKN3eVx9NwZFgJfHuOGDU7OxdhOFDPcd+F+CPzecgviqnE4fm9hC xuhh+L4hSIUalB9ljRoV6rWxlObC12rQnUkuw6R1Ds5RP1SmtRCPn7VM1U3fcpzSb/hT 2foQ== X-Gm-Message-State: AOAM531nT8Kh9xelZnpCL+/EFbhcngQKF0/E+k3CvnG2JRTjpdFe6Tqw CoxyyqxgbSI9Hx1EEqdqVJe92jYHIIc= X-Google-Smtp-Source: ABdhPJywgz7yNhes03lPcP1wIlVD5FrN/znQnclwM2sV4TDW1SAQv6NdSDyLqyBxdaTJZF6820eAFQ== X-Received: by 2002:a50:cc04:: with SMTP id m4mr3242161edi.358.1630665310956; Fri, 03 Sep 2021 03:35:10 -0700 (PDT) Received: from localhost.localdomain ([85.187.18.127]) by smtp.gmail.com with ESMTPSA id d22sm2609897ejj.47.2021.09.03.03.35.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Sep 2021 03:35:10 -0700 (PDT) From: Vladislav Odintsov To: dev@openvswitch.org Date: Fri, 3 Sep 2021 13:34:37 +0300 Message-Id: <20210903103441.33178-3-odivlad@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210903103441.33178-1-odivlad@gmail.com> References: <20210903103441.33178-1-odivlad@gmail.com> MIME-Version: 1.0 Cc: Vladislav Odintsov Subject: [ovs-dev] [PATCH ovn v2 2/6] ic: process only local port_bindings X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds a small optimization by utilizing ovsdb_index to iterate over port_bindings. Prior to this change each iteration checked availability_zone and continued processing only if port_binding belons to local AZ. Now we run against port_bindings from local AZ only and don't check availability_zone. Signed-off-by: Vladislav Odintsov --- ic/ovn-ic.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c index 418f19147..980000d3e 100644 --- a/ic/ovn-ic.c +++ b/ic/ovn-ic.c @@ -68,6 +68,7 @@ struct ic_context { struct ovsdb_idl_index *sbrec_port_binding_by_name; struct ovsdb_idl_index *icsbrec_port_binding_by_az; struct ovsdb_idl_index *icsbrec_port_binding_by_ts; + struct ovsdb_idl_index *icsbrec_port_binding_by_ts_az; struct ovsdb_idl_index *icsbrec_route_by_ts; struct ovsdb_idl_index *icsbrec_route_by_ts_az; }; @@ -1391,17 +1392,15 @@ route_run(struct ic_context *ctx, const struct icsbrec_port_binding *isb_pb; const struct icsbrec_port_binding *isb_pb_key = icsbrec_port_binding_index_init_row( - ctx->icsbrec_port_binding_by_ts); + ctx->icsbrec_port_binding_by_ts_az); icsbrec_port_binding_index_set_transit_switch(isb_pb_key, ts->name); + icsbrec_port_binding_index_set_availability_zone(isb_pb_key, az); /* Each port on TS maps to a logical router, which is stored in the * external_ids:router-id of the IC SB port_binding record. */ ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key, - ctx->icsbrec_port_binding_by_ts) { - if (isb_pb->availability_zone != az) { - continue; - } - + ctx->icsbrec_port_binding_by_ts_az) + { const char *ts_lrp_name = get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port); if (!ts_lrp_name) { @@ -1718,6 +1717,11 @@ main(int argc, char *argv[]) = ovsdb_idl_index_create1(ovnisb_idl_loop.idl, &icsbrec_port_binding_col_transit_switch); + struct ovsdb_idl_index *icsbrec_port_binding_by_ts_az + = ovsdb_idl_index_create2(ovnisb_idl_loop.idl, + &icsbrec_port_binding_col_transit_switch, + &icsbrec_port_binding_col_availability_zone); + struct ovsdb_idl_index *icsbrec_route_by_ts = ovsdb_idl_index_create1(ovnisb_idl_loop.idl, &icsbrec_route_col_transit_switch); @@ -1768,6 +1772,7 @@ main(int argc, char *argv[]) .sbrec_chassis_by_name = sbrec_chassis_by_name, .icsbrec_port_binding_by_az = icsbrec_port_binding_by_az, .icsbrec_port_binding_by_ts = icsbrec_port_binding_by_ts, + .icsbrec_port_binding_by_ts_az = icsbrec_port_binding_by_ts_az, .icsbrec_route_by_ts = icsbrec_route_by_ts, .icsbrec_route_by_ts_az = icsbrec_route_by_ts_az, }; From patchwork Fri Sep 3 10:34:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladislav Odintsov X-Patchwork-Id: 1524202 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=UhcDz0MW; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H1DhY1qwJz9s5R for ; Fri, 3 Sep 2021 20:35:25 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 5E44561564; Fri, 3 Sep 2021 10:35:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b0ow0_r2r_T1; Fri, 3 Sep 2021 10:35:22 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id ACC3D61551; Fri, 3 Sep 2021 10:35:21 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 87C19C0010; Fri, 3 Sep 2021 10:35:21 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1875CC0010 for ; Fri, 3 Sep 2021 10:35:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id A73F8407E3 for ; Fri, 3 Sep 2021 10:35:19 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id o9XKoYBzCHkn for ; Fri, 3 Sep 2021 10:35:16 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) by smtp2.osuosl.org (Postfix) with ESMTPS id 7DC26407D4 for ; Fri, 3 Sep 2021 10:35:16 +0000 (UTC) Received: by mail-ed1-x52a.google.com with SMTP id g22so7325448edy.12 for ; Fri, 03 Sep 2021 03:35:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=efg9Hf5qyLhtmYm1IR1fPVlpkIFf4UZ5aGSq/ZMIyX4=; b=UhcDz0MWBBEdicAPKbJj0rklK4jhaXYSlKhIIbkoOAuFXcJFjPhj98ZI52TfQoXWVe TXpPbcxaWMAy9l7UYYT932pbMxnBFaMotk31HaMnouiHuZvuKdGld5BvekEy8ATft8+a d0M9gCdl1WK1rgHq+1VdNrdbwDMAXv4Xx7Yqzk/e1Rh42zjLU0ceGP0Ni/9EMxHxymnd 3SXNcnxZ0uBmmp1RKZx1y7+iZJkCpKO8A26DJPsTkUB8hocODSJfEMbdT8F9NxhOWlq6 UIAbb6l4fZI9UvcLLA+faeznpnCwf1RT90k1IjWiHXVnMCH0ioTNERTFXl4Kkl64sIiW jejg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=efg9Hf5qyLhtmYm1IR1fPVlpkIFf4UZ5aGSq/ZMIyX4=; b=TGdPimShlAwIxyPx5p+iO2wVU6qlfZLUKim+xVgztdWnUSZZ9icaiVDkU6Gabs1yZb JxrvG8+b5Kb/vbRE2kSiaysa4VWkWZrv/rYXvZzT/h1QLjeaixzfpRei8unwANFNk1iJ QmNPbN+7qydh9lFmq3eiEXRLf8J/hiz0kQqsJ61vcTjw+ZDUjN780Y4kICxFoVVgy1Bc NUxkAJiTFVZEC/iHuVQhfijfOB2TxJvB37nA8+0wFiT/3uiE29kTF1LBUGc4e6CJspAf uzMcu6KWY9WGNQ+auVtDFoyERAsjqygidRVW0fjFtphGjIdjeCToe6Bwppppt/DmMIdG yJeA== X-Gm-Message-State: AOAM532AnazBPrcg0lejC+VnAnVS/kb0XFTFzSY9lgaQW1P/m/Q/WBN9 gX2lsoPz1uuC1O8w5Q9pO+Ihq/PShBM= X-Google-Smtp-Source: ABdhPJyuVUek8aWAYf7tU2fyKYs6bFbCQF0g6Wl87J7d52aHScAHLZZA40onAK1AZngV1HAK21yfCQ== X-Received: by 2002:a05:6402:1157:: with SMTP id g23mr3293769edw.90.1630665314557; Fri, 03 Sep 2021 03:35:14 -0700 (PDT) Received: from localhost.localdomain ([85.187.18.127]) by smtp.gmail.com with ESMTPSA id d22sm2609897ejj.47.2021.09.03.03.35.14 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Sep 2021 03:35:14 -0700 (PDT) From: Vladislav Odintsov To: dev@openvswitch.org Date: Fri, 3 Sep 2021 13:34:38 +0300 Message-Id: <20210903103441.33178-4-odivlad@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210903103441.33178-1-odivlad@gmail.com> References: <20210903103441.33178-1-odivlad@gmail.com> MIME-Version: 1.0 Cc: Vladislav Odintsov Subject: [ovs-dev] [PATCH ovn v2 3/6] tests: remove strict check for table=N for N > 9 X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Previously patch [1] with same summary was submitted, but it was applied lated then a new feature in commit [2] was accepted. There was added a new test, which also checks exact logical flow table number. This commit replaces table=NN with table=?? in those new tests. [1] https://github.com/ovn-org/ovn/commit/e3ae68ed9a1bb231f7b069539693474508f3ed86 [2] https://github.com/ovn-org/ovn/commit/c0085228893e7bf07190fcccf50cf588b028edaa Signed-off-by: Vladislav Odintsov --- tests/ovn-northd.at | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 11886b94e..1c0cb7c0e 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -5131,15 +5131,15 @@ wait_row_count nb:Logical_Router_Static_Route 0 check ovn-nbctl --wait=sb lr-route-add lr0 1.0.0.0/24 192.168.0.10 ovn-sbctl dump-flows lr0 > lr0flows -AT_CHECK([grep -e "lr_in_ip_routing.*192.168.0.10" lr0flows |sort], [0], [dnl - table=10(lr_in_ip_routing ), priority=49 , match=(ip4.dst == 1.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;) +AT_CHECK([grep -e "lr_in_ip_routing.*192.168.0.10" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl + table=??(lr_in_ip_routing ), priority=49 , match=(ip4.dst == 1.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;) ]) check ovn-nbctl --wait=sb lr-route-add lr0 2.0.0.0/24 lr0-public ovn-sbctl dump-flows lr0 > lr0flows -AT_CHECK([grep -e "lr_in_ip_routing.*2.0.0.0" lr0flows |sort], [0], [dnl - table=10(lr_in_ip_routing ), priority=49 , match=(ip4.dst == 2.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;) +AT_CHECK([grep -e "lr_in_ip_routing.*2.0.0.0" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl + table=??(lr_in_ip_routing ), priority=49 , match=(ip4.dst == 2.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;) ]) AT_CLEANUP From patchwork Fri Sep 3 10:34:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladislav Odintsov X-Patchwork-Id: 1524203 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=MsDfRFlm; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H1Dhs32CDz9s5R for ; Fri, 3 Sep 2021 20:35:41 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id AD4E1426AA; Fri, 3 Sep 2021 10:35:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tdwukfzSJhCk; Fri, 3 Sep 2021 10:35:36 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 88F7B42678; Fri, 3 Sep 2021 10:35:35 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4C25AC0023; Fri, 3 Sep 2021 10:35:35 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 11FE7C000E for ; Fri, 3 Sep 2021 10:35:34 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 19C3D6062E for ; Fri, 3 Sep 2021 10:35:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6MKAqRWS5meQ for ; Fri, 3 Sep 2021 10:35:21 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [IPv6:2a00:1450:4864:20::62c]) by smtp3.osuosl.org (Postfix) with ESMTPS id 2504161568 for ; Fri, 3 Sep 2021 10:35:21 +0000 (UTC) Received: by mail-ej1-x62c.google.com with SMTP id h9so11168423ejs.4 for ; Fri, 03 Sep 2021 03:35:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sIHlZ4l+LZME0N/VLsTPmmzEoQK6aNh7lfj8aQaDGFw=; b=MsDfRFlmjkMh3J+qbAuRyQfQOVMshMAJY6yj7wk42DbT8To1eELN8PIZYRD+ByBP/O BaxUev7B23QFO4Ad9mSTSVB5EGuP4aoLJK6QXryn4F+/EIMC7gJg+WJVAzKDnKla9w74 ko9KFDCvPFEwwueMjtk0V9dXfLaop6eWg1b9THGHVwaFwiFDBTN4/lcm2x+eMBaeFyHk kG0dBfzixvPNYNnAimcCSKRQEsd3huswRmc3Xdy/tWlQYtXAwoWMzPd82OoRUPinAYLa HPRMR6NIiVd7gbSSPux5yEffsfWgXyfRD6fwwhuGj1SA+MtKz+dEQ88fBnBmEEdc9CBZ EAjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sIHlZ4l+LZME0N/VLsTPmmzEoQK6aNh7lfj8aQaDGFw=; b=s6rR4557FPGf+OgpiOUcF3yQsqLWX9P/YmbD4bR+yi5bZZ0y6kBIBh9TzJH0Cqpe/U MRpoe0q1dEnRdKuTp9ScyLyOisAcWAI3SJNL1ZHB7JDcgtk93Klg+KmS8gArDL/pz5Km sTES2+m5JYN2b2ROtIspOKbJa7r6zaJOhWyRMj7wLnhDgzViqmPGEe71fP0Yot8h+HAs 3pwK15QLmTD3ePphVgjYZXphe3oV+Ql7uOQc2k1MDx64TlQU5AIM7I1yZn9SI4m3rOms DTXACUmzlHXa71hGpae/cLcFY7aXtktRfPWBxx88E2ec78x3jDduJ9UDOD8piaxX537g UWEw== X-Gm-Message-State: AOAM53124rF8C3IQnvFzSGcqjKWM4ELVZy0dxj9RF1s49IuXJnSR8Lb2 XRJg0zkifyDKwof3mpsgEe1LBPnE7HE= X-Google-Smtp-Source: ABdhPJzugSpFoBXdTMvWkHRxr6lH9Q3nB7JBYJyH5hAAZFu86h2h/vghg0xh88sU8zVVmSJHPT09aw== X-Received: by 2002:a17:906:2a8e:: with SMTP id l14mr3413514eje.321.1630665317677; Fri, 03 Sep 2021 03:35:17 -0700 (PDT) Received: from localhost.localdomain ([85.187.18.127]) by smtp.gmail.com with ESMTPSA id d22sm2609897ejj.47.2021.09.03.03.35.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Sep 2021 03:35:17 -0700 (PDT) From: Vladislav Odintsov To: dev@openvswitch.org Date: Fri, 3 Sep 2021 13:34:39 +0300 Message-Id: <20210903103441.33178-5-odivlad@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210903103441.33178-1-odivlad@gmail.com> References: <20210903103441.33178-1-odivlad@gmail.com> MIME-Version: 1.0 Cc: Vladislav Odintsov Subject: [ovs-dev] [PATCH ovn v2 4/6] northd, utils: support for RouteTables in LRs X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Signed-off-by: Vladislav Odintsov --- northd/ovn-northd.8.xml | 63 ++++-- northd/ovn-northd.c | 198 +++++++++++++++--- ovn-nb.ovsschema | 5 +- ovn-nb.xml | 30 +++ tests/ovn-ic.at | 4 + tests/ovn-nbctl.at | 196 +++++++++++++++++- tests/ovn-northd.at | 76 ++++++- tests/ovn.at | 441 +++++++++++++++++++++++++++++++++++++++- utilities/ovn-nbctl.c | 134 +++++++++++- 9 files changed, 1080 insertions(+), 67 deletions(-) diff --git a/northd/ovn-northd.8.xml b/northd/ovn-northd.8.xml index eebf0d717..b8eb0e1f6 100644 --- a/northd/ovn-northd.8.xml +++ b/northd/ovn-northd.8.xml @@ -2871,7 +2871,7 @@ icmp6 {

If ECMP routes with symmetric reply are configured in the - OVN_Northbound database for a gateway router, a priority-300 + OVN_Northbound database for a gateway router, a priority-400 flow is added for each router port on which symmetric replies are configured. The matching logic for these ports essentially reverses the configured logic of the ECMP route. So for instance, a route with a @@ -3217,7 +3217,35 @@ output; -

Ingress Table 10: IP Routing

+

Ingress Table 10: IP Routing Pre

+ +

+ If a packet arrived at this table from Logical Router Port P + which has options:route_table value set, a logical flow with + match inport == "P" with priority 100 and action, + setting unique-generated per-datapath 32-bit value (non-zero) in OVS + register 7. This register is checked in next table. +

+ +

+ This table contains the following logical flows: +

+ +
    +
  • +

    + Priority-100 flow with match inport == "LRP_NAME" value + and action, which set route table identifier in reg7. +

    + +

    + A priority-0 logical flow with match 1 has actions + next;. +

    +
    • +
+ +

Ingress Table 11: IP Routing

A packet that arrives at this table is an IP packet that should be @@ -3288,10 +3316,10 @@ output;

IPv4 routing table. For each route to IPv4 network N with netmask M, on router port P with IP address - A and Ethernet - address E, a logical flow with match ip4.dst == - N/M, whose priority is the number of - 1-bits in M, has the following actions: + A and Ethernet address E, a logical flow with + match ip4.dst == N/M, whose + priority is 100 + the number of 1-bits in M, has the + following actions: 

@@ -3354,6 +3382,13 @@ next;
           If the address A is in the link-local scope, the
           route will be limited to sending on the ingress port.
         

+
+        

+          For routes with route_table value set
+          reg7 == id is prefixed in logical flow match portion.
+          Priority for routes with route_table value set is
+          the number of 1-bits in M.
+        

       
 
       
  • 
@@ -3380,7 +3415,7 @@ select(reg8[16..31], MID1, MID2, ...);
       
    • 
     
 
-    
    Ingress Table 11: IP_ROUTING_ECMP
    
+    
    Ingress Table 12: IP_ROUTING_ECMP
    
 
     
    
       This table implements the second part of IP routing for ECMP routes
@@ -3432,7 +3467,7 @@ outport = P;
       
     
 
-    
    Ingress Table 12: Router policies
    
+    
    Ingress Table 13: Router policies
    
     
    
       This table adds flows for the logical router policies configured
       on the logical router. Please see the
@@ -3504,7 +3539,7 @@ next;
       
     
 
-    
    Ingress Table 13: ECMP handling for router policies
    
+    
    Ingress Table 14: ECMP handling for router policies
    
     
    
       This table handles the ECMP for the router policies configured
       with multiple nexthops.
@@ -3548,7 +3583,7 @@ outport = P
       
     
 
-    
    Ingress Table 14: ARP/ND Resolution
    
+    
    Ingress Table 15: ARP/ND Resolution
    
 
     
    
       Any packet that reaches this table is an IP packet whose next-hop
@@ -3739,7 +3774,7 @@ outport = P
 
     
 
-    
    Ingress Table 15: Check packet length
    
+    
    Ingress Table 16: Check packet length
    
 
     
    
       For distributed logical routers or gateway routers with gateway
@@ -3769,7 +3804,7 @@ REGBIT_PKT_LARGER = check_pkt_larger(L); next;
       and advances to the next table.
     
    
 
-    
    Ingress Table 16: Handle larger packets
    
+    
    Ingress Table 17: Handle larger packets
    
 
     
    
       For distributed logical routers or gateway routers with gateway port
@@ -3832,7 +3867,7 @@ icmp6 {
       and advances to the next table.
     
    
 
-    
    Ingress Table 17: Gateway Redirect
    
+    
    Ingress Table 18: Gateway Redirect
    
 
     
    
       For distributed logical routers where one or more of the logical router
@@ -3879,7 +3914,7 @@ icmp6 {
       
     
 
-    
    Ingress Table 18: ARP Request
    
+    
    Ingress Table 19: ARP Request
    
 
     
    
       In the common case where the Ethernet destination has been resolved, this
diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c
index ee761cef0..17ac47e49 100644
--- a/northd/ovn-northd.c
+++ b/northd/ovn-northd.c
@@ -191,15 +191,16 @@ enum ovn_stage {
     PIPELINE_STAGE(ROUTER, IN,  ECMP_STATEFUL,   7, "lr_in_ecmp_stateful") \
     PIPELINE_STAGE(ROUTER, IN,  ND_RA_OPTIONS,   8, "lr_in_nd_ra_options") \
     PIPELINE_STAGE(ROUTER, IN,  ND_RA_RESPONSE,  9, "lr_in_nd_ra_response") \
-    PIPELINE_STAGE(ROUTER, IN,  IP_ROUTING,      10, "lr_in_ip_routing")   \
-    PIPELINE_STAGE(ROUTER, IN,  IP_ROUTING_ECMP, 11, "lr_in_ip_routing_ecmp") \
-    PIPELINE_STAGE(ROUTER, IN,  POLICY,          12, "lr_in_policy")       \
-    PIPELINE_STAGE(ROUTER, IN,  POLICY_ECMP,     13, "lr_in_policy_ecmp")  \
-    PIPELINE_STAGE(ROUTER, IN,  ARP_RESOLVE,     14, "lr_in_arp_resolve")  \
-    PIPELINE_STAGE(ROUTER, IN,  CHK_PKT_LEN   ,  15, "lr_in_chk_pkt_len")  \
-    PIPELINE_STAGE(ROUTER, IN,  LARGER_PKTS,     16, "lr_in_larger_pkts")  \
-    PIPELINE_STAGE(ROUTER, IN,  GW_REDIRECT,     17, "lr_in_gw_redirect")  \
-    PIPELINE_STAGE(ROUTER, IN,  ARP_REQUEST,     18, "lr_in_arp_request")  \
+    PIPELINE_STAGE(ROUTER, IN,  IP_ROUTING_PRE,  10, "lr_in_ip_routing_pre")  \
+    PIPELINE_STAGE(ROUTER, IN,  IP_ROUTING,      11, "lr_in_ip_routing")      \
+    PIPELINE_STAGE(ROUTER, IN,  IP_ROUTING_ECMP, 12, "lr_in_ip_routing_ecmp") \
+    PIPELINE_STAGE(ROUTER, IN,  POLICY,          13, "lr_in_policy")          \
+    PIPELINE_STAGE(ROUTER, IN,  POLICY_ECMP,     14, "lr_in_policy_ecmp")     \
+    PIPELINE_STAGE(ROUTER, IN,  ARP_RESOLVE,     15, "lr_in_arp_resolve")     \
+    PIPELINE_STAGE(ROUTER, IN,  CHK_PKT_LEN,     16, "lr_in_chk_pkt_len")     \
+    PIPELINE_STAGE(ROUTER, IN,  LARGER_PKTS,     17, "lr_in_larger_pkts")     \
+    PIPELINE_STAGE(ROUTER, IN,  GW_REDIRECT,     18, "lr_in_gw_redirect")     \
+    PIPELINE_STAGE(ROUTER, IN,  ARP_REQUEST,     19, "lr_in_arp_request")     \
                                                                       \
     /* Logical router egress stages. */                               \
     PIPELINE_STAGE(ROUTER, OUT, UNDNAT,      0, "lr_out_undnat")        \
@@ -267,6 +268,7 @@ enum ovn_stage {
 #define REG_NEXT_HOP_IPV6 "xxreg0"
 #define REG_SRC_IPV4 "reg1"
 #define REG_SRC_IPV6 "xxreg1"
+#define REG_ROUTE_TABLE_ID "reg7"
 
 #define REG_ORIG_TP_DPORT_ROUTER   "reg9[16..31]"
 
@@ -329,8 +331,9 @@ enum ovn_stage {
  * | R6  |        UNUSED            | X |                 | G | IN_IP_ROUTING)|
  * |     |                          | R |                 | 1 |               |
  * +-----+--------------------------+ E |     UNUSED      |   |               |
- * | R7  |        UNUSED            | G |                 |   |               |
- * |     |                          | 3 |                 |   |               |
+ * | R7  |      ROUTE_TABLE_ID      | G |                 |   |               |
+ * |     | (>= IN_IP_ROUTING_PRE && | 3 |                 |   |               |
+ * |     |  <= IN_IP_ROUTING)       |   |                 |   |               |
  * +-----+--------------------------+---+-----------------+---+---------------+
  * | R8  |     ECMP_GROUP_ID        |   |                 |
  * |     |     ECMP_MEMBER_ID       | X |                 |
@@ -8482,11 +8485,110 @@ cleanup:
     ds_destroy(&actions);
 }
 
+struct route_table_node {
+    struct hmap_node hmap_node; /* In route_tables */
+    uint32_t id; /* starts from 1 */
+    const char *name;
+};
+
+static uint32_t
+get_route_table_hash(const char *route_table_name)
+{
+    return hash_string(route_table_name, 0);
+}
+
+static struct route_table_node *
+route_table_find(struct hmap *route_tables, const char *route_table_name)
+{
+    struct route_table_node *rtb;
+    uint32_t hash = get_route_table_hash(route_table_name);
+
+    HMAP_FOR_EACH_WITH_HASH (rtb, hmap_node, hash, route_tables) {
+        if (!strcmp(rtb->name, route_table_name)) {
+            return rtb;
+        }
+    }
+    return NULL;
+}
+
+static struct route_table_node *
+route_table_add(struct hmap *route_tables, const char *route_table_name)
+{
+    if (hmap_count(route_tables) == UINT16_MAX) {
+        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
+        VLOG_WARN_RL(&rl, "too many route tables for Logical Router.");
+        return NULL;
+    }
+
+    struct route_table_node *rtb = xzalloc(sizeof *rtb);
+    uint32_t hash = get_route_table_hash(route_table_name);
+    hmap_insert(route_tables, &rtb->hmap_node, hash);
+
+    rtb->id = hmap_count(route_tables);
+    rtb->name = route_table_name;
+
+    return rtb;
+}
+
+static uint32_t
+get_route_table_id(struct hmap *route_tables, const char *route_table_name)
+{
+    struct route_table_node *rtb;
+
+    if (!route_table_name || !strlen(route_table_name)) {
+        return 0;
+    }
+
+    rtb = route_table_find(route_tables, route_table_name);
+    if (!rtb) {
+        rtb = route_table_add(route_tables, route_table_name);
+    }
+
+    return rtb->id;
+}
+
+static void
+route_tables_destroy(struct hmap *route_tables)
+{
+    struct route_table_node *rtb, *next;
+    HMAP_FOR_EACH_SAFE (rtb, next, hmap_node, route_tables) {
+        hmap_remove(route_tables, &rtb->hmap_node);
+        free(rtb);
+    }
+    hmap_destroy(route_tables);
+}
+
+static void
+build_route_table_lflow(struct ovn_datapath *od, struct hmap *lflows,
+                        struct nbrec_logical_router_port *lrp,
+                        struct hmap *route_tables)
+{
+    struct ds match = DS_EMPTY_INITIALIZER;
+    struct ds actions = DS_EMPTY_INITIALIZER;
+
+    const char *route_table_name = smap_get(&lrp->options, "route_table");
+    uint32_t rtb_id = get_route_table_id(route_tables, route_table_name);
+    if (!rtb_id) {
+        return;
+    }
+
+    ds_put_format(&match, "inport == \"%s\"", lrp->name);
+    ds_put_format(&actions, "%s = %d; next;",
+                  REG_ROUTE_TABLE_ID, rtb_id);
+
+    ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 100,
+                  ds_cstr(&match), ds_cstr(&actions));
+
+    ds_destroy(&match);
+    ds_destroy(&actions);
+}
+
 struct parsed_route {
     struct ovs_list list_node;
     struct in6_addr prefix;
     unsigned int plen;
     bool is_src_route;
+    uint32_t route_table_id;
     uint32_t hash;
     const struct nbrec_logical_router_static_route *route;
     bool ecmp_symmetric_reply;
@@ -8511,7 +8613,7 @@ find_static_route_outport(struct ovn_datapath *od, struct hmap *ports,
  * Otherwise return NULL. */
 static struct parsed_route *
 parsed_routes_add(struct ovn_datapath *od, struct hmap *ports,
-                  struct ovs_list *routes,
+                  struct ovs_list *routes, struct hmap *route_tables,
                   const struct nbrec_logical_router_static_route *route,
                   struct hmap *bfd_connections)
 {
@@ -8593,6 +8695,7 @@ parsed_routes_add(struct ovn_datapath *od, struct hmap *ports,
     struct parsed_route *pr = xzalloc(sizeof *pr);
     pr->prefix = prefix;
     pr->plen = plen;
+    pr->route_table_id = get_route_table_id(route_tables, route->route_table);
     pr->is_src_route = (route->policy && !strcmp(route->policy,
                                                  "src-ip"));
     pr->hash = route_hash(pr);
@@ -8626,6 +8729,7 @@ struct ecmp_groups_node {
     struct in6_addr prefix;
     unsigned int plen;
     bool is_src_route;
+    uint32_t route_table_id;
     uint16_t route_count;
     struct ovs_list route_list; /* Contains ecmp_route_list_node */
 };
@@ -8634,7 +8738,7 @@ static void
 ecmp_groups_add_route(struct ecmp_groups_node *group,
                       const struct parsed_route *route)
 {
-   if (group->route_count == UINT16_MAX) {
+    if (group->route_count == UINT16_MAX) {
         static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
         VLOG_WARN_RL(&rl, "too many routes in a single ecmp group.");
         return;
@@ -8663,6 +8767,7 @@ ecmp_groups_add(struct hmap *ecmp_groups,
     eg->prefix = route->prefix;
     eg->plen = route->plen;
     eg->is_src_route = route->is_src_route;
+    eg->route_table_id = route->route_table_id;
     ovs_list_init(&eg->route_list);
     ecmp_groups_add_route(eg, route);
 
@@ -8676,7 +8781,8 @@ ecmp_groups_find(struct hmap *ecmp_groups, struct parsed_route *route)
     HMAP_FOR_EACH_WITH_HASH (eg, hmap_node, route->hash, ecmp_groups) {
         if (ipv6_addr_equals(&eg->prefix, &route->prefix) &&
             eg->plen == route->plen &&
-            eg->is_src_route == route->is_src_route) {
+            eg->is_src_route == route->is_src_route &&
+            eg->route_table_id == route->route_table_id) {
             return eg;
         }
     }
@@ -8723,7 +8829,8 @@ unique_routes_remove(struct hmap *unique_routes,
     HMAP_FOR_EACH_WITH_HASH (ur, hmap_node, route->hash, unique_routes) {
         if (ipv6_addr_equals(&route->prefix, &ur->route->prefix) &&
             route->plen == ur->route->plen &&
-            route->is_src_route == ur->route->is_src_route) {
+            route->is_src_route == ur->route->is_src_route &&
+            route->route_table_id == ur->route->route_table_id) {
             hmap_remove(unique_routes, &ur->hmap_node);
             const struct parsed_route *existed_route = ur->route;
             free(ur);
@@ -8761,9 +8868,9 @@ build_route_prefix_s(const struct in6_addr *prefix, unsigned int plen)
 }
 
 static void
-build_route_match(const struct ovn_port *op_inport, const char *network_s,
-                  int plen, bool is_src_route, bool is_ipv4, struct ds *match,
-                  uint16_t *priority)
+build_route_match(const struct ovn_port *op_inport, const uint32_t rtb_id,
+                  const char *network_s, int plen, bool is_src_route,
+                  bool is_ipv4, struct ds *match, uint16_t *priority)
 {
     const char *dir;
     /* The priority here is calculated to implement longest-prefix-match
@@ -8779,6 +8886,16 @@ build_route_match(const struct ovn_port *op_inport, const char *network_s,
     if (op_inport) {
         ds_put_format(match, "inport == %s && ", op_inport->json_key);
     }
+    if (rtb_id) {
+        ds_put_format(match, "%s == %d && ", REG_ROUTE_TABLE_ID, rtb_id);
+    }
+    else {
+        /* Route-table assigned LRPs' routes should have lower priority
+         * in order not to affect directly-connected global routes.
+         * So, enlarge non-route-table routes priority by 100.
+         */
+        *priority += 100;
+    }
     ds_put_format(match, "ip%s.%s == %s/%d", is_ipv4 ? "4" : "6", dir,
                   network_s, plen);
 }
@@ -8913,7 +9030,7 @@ add_ecmp_symmetric_reply_flows(struct hmap *lflows,
                   out_port->lrp_networks.ea_s,
                   IN6_IS_ADDR_V4MAPPED(&route->prefix) ? "" : "xx",
                   port_ip, out_port->json_key);
-    ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, 300,
+    ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, 400,
                            ds_cstr(&match), ds_cstr(&actions),
                            &st_route->header_);
 
@@ -8943,8 +9060,8 @@ build_ecmp_route_flow(struct hmap *lflows, struct ovn_datapath *od,
     struct ds route_match = DS_EMPTY_INITIALIZER;
 
     char *prefix_s = build_route_prefix_s(&eg->prefix, eg->plen);
-    build_route_match(NULL, prefix_s, eg->plen, eg->is_src_route, is_ipv4,
-                      &route_match, &priority);
+    build_route_match(NULL, eg->route_table_id, prefix_s, eg->plen,
+                      eg->is_src_route, is_ipv4, &route_match, &priority);
     free(prefix_s);
 
     struct ds actions = DS_EMPTY_INITIALIZER;
@@ -9019,8 +9136,8 @@ static void
 add_route(struct hmap *lflows, struct ovn_datapath *od,
           const struct ovn_port *op, const char *lrp_addr_s,
           const char *network_s, int plen, const char *gateway,
-          bool is_src_route, const struct ovsdb_idl_row *stage_hint,
-          bool is_discard_route)
+          bool is_src_route, const uint32_t rtb_id,
+          const struct ovsdb_idl_row *stage_hint, bool is_discard_route)
 {
     bool is_ipv4 = strchr(network_s, '.') ? true : false;
     struct ds match = DS_EMPTY_INITIALIZER;
@@ -9035,8 +9152,8 @@ add_route(struct hmap *lflows, struct ovn_datapath *od,
             op_inport = op;
         }
     }
-    build_route_match(op_inport, network_s, plen, is_src_route, is_ipv4,
-                      &match, &priority);
+    build_route_match(op_inport, rtb_id, network_s, plen, is_src_route,
+                      is_ipv4, &match, &priority);
 
     struct ds common_actions = DS_EMPTY_INITIALIZER;
     struct ds actions = DS_EMPTY_INITIALIZER;
@@ -9099,7 +9216,8 @@ build_static_route_flow(struct hmap *lflows, struct ovn_datapath *od,
     char *prefix_s = build_route_prefix_s(&route_->prefix, route_->plen);
     add_route(lflows, route_->is_discard_route ? od : out_port->od, out_port,
               lrp_addr_s, prefix_s, route_->plen, route->nexthop,
-              route_->is_src_route, &route->header_, route_->is_discard_route);
+              route_->is_src_route, route_->route_table_id, &route->header_,
+              route_->is_discard_route);
 
     free(prefix_s);
 }
@@ -10554,6 +10672,17 @@ build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, struct hmap *lflows)
     }
 }
 
+/* Logical router ingress table IP_ROUTING_PRE:
+ * by default goto next. (priority 0). */
+static void
+build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od,
+                                       struct hmap *lflows)
+{
+    if (od->nbr) {
+        ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 0, "1", "next;");
+    }
+}
+
 /* Logical router ingress table IP_ROUTING : IP Routing.
  *
  * A packet that arrives at this table is an IP packet that should be
@@ -10579,14 +10708,14 @@ build_ip_routing_flows_for_lrouter_port(
         for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) {
             add_route(lflows, op->od, op, op->lrp_networks.ipv4_addrs[i].addr_s,
                       op->lrp_networks.ipv4_addrs[i].network_s,
-                      op->lrp_networks.ipv4_addrs[i].plen, NULL, false,
+                      op->lrp_networks.ipv4_addrs[i].plen, NULL, false, 0,
                       &op->nbrp->header_, false);
         }
 
         for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) {
             add_route(lflows, op->od, op, op->lrp_networks.ipv6_addrs[i].addr_s,
                       op->lrp_networks.ipv6_addrs[i].network_s,
-                      op->lrp_networks.ipv6_addrs[i].plen, NULL, false,
+                      op->lrp_networks.ipv6_addrs[i].plen, NULL, false, 0,
                       &op->nbrp->header_, false);
         }
     } else if (lsp_is_router(op->nbsp)) {
@@ -10609,7 +10738,7 @@ build_ip_routing_flows_for_lrouter_port(
                     add_route(lflows, peer->od, peer,
                               peer->lrp_networks.ipv4_addrs[0].addr_s,
                               laddrs->ipv4_addrs[k].network_s,
-                              laddrs->ipv4_addrs[k].plen, NULL, false,
+                              laddrs->ipv4_addrs[k].plen, NULL, false, 0,
                               &peer->nbrp->header_, false);
                 }
             }
@@ -10629,10 +10758,17 @@ build_static_route_flows_for_lrouter(
         struct hmap ecmp_groups = HMAP_INITIALIZER(&ecmp_groups);
         struct hmap unique_routes = HMAP_INITIALIZER(&unique_routes);
         struct ovs_list parsed_routes = OVS_LIST_INITIALIZER(&parsed_routes);
+        struct hmap route_tables = HMAP_INITIALIZER(&route_tables);
         struct ecmp_groups_node *group;
+
+        for (int i = 0; i < od->nbr->n_ports; i++) {
+            build_route_table_lflow(od, lflows, od->nbr->ports[i],
+                                    &route_tables);
+        }
+
         for (int i = 0; i < od->nbr->n_static_routes; i++) {
             struct parsed_route *route =
-                parsed_routes_add(od, ports, &parsed_routes,
+                parsed_routes_add(od, ports, &parsed_routes, &route_tables,
                                   od->nbr->static_routes[i], bfd_connections);
             if (!route) {
                 continue;
@@ -10663,6 +10799,7 @@ build_static_route_flows_for_lrouter(
             build_static_route_flow(lflows, od, ports, ur->route);
         }
         ecmp_groups_destroy(&ecmp_groups);
+        route_tables_destroy(&route_tables);
         unique_routes_destroy(&unique_routes);
         parsed_routes_destroy(&parsed_routes);
     }
@@ -12761,6 +12898,7 @@ build_lswitch_and_lrouter_iterate_by_od(struct ovn_datapath *od,
     build_neigh_learning_flows_for_lrouter(od, lsi->lflows, &lsi->match,
                                            &lsi->actions, lsi->meter_groups);
     build_ND_RA_flows_for_lrouter(od, lsi->lflows);
+    build_ip_routing_pre_flows_for_lrouter(od, lsi->lflows);
     build_static_route_flows_for_lrouter(od, lsi->lflows, lsi->ports,
                                          lsi->bfd_connections);
     build_mcast_lookup_flows_for_lrouter(od, lsi->lflows, &lsi->match,
diff --git a/ovn-nb.ovsschema b/ovn-nb.ovsschema
index 2ac8ef3ea..a0a171e19 100644
--- a/ovn-nb.ovsschema
+++ b/ovn-nb.ovsschema
@@ -1,7 +1,7 @@
 {
     "name": "OVN_Northbound",
-    "version": "5.32.1",
-    "cksum": "2805328215 29734",
+    "version": "5.33.1",
+    "cksum": "3874993350 29785",
     "tables": {
         "NB_Global": {
             "columns": {
@@ -387,6 +387,7 @@
             "isRoot": false},
         "Logical_Router_Static_Route": {
             "columns": {
+                "route_table": {"type": "string"},
                 "ip_prefix": {"type": "string"},
                 "policy": {"type": {"key": {"type": "string",
                                             "enum": ["set", ["src-ip",
diff --git a/ovn-nb.xml b/ovn-nb.xml
index 390cc5a44..99fa78c90 100644
--- a/ovn-nb.xml
+++ b/ovn-nb.xml
@@ -2772,6 +2772,14 @@
           prefix according to RFC3663
         
    
       
+
+      
+        Designates lookup Logical_Router_Static_Routes with specified
+        route_table value. Routes to directly connected networks
+        from same Logical Router and routes without route_table
+        option set have higher priority than routes with
+        route_table option set.
+      
     
 
     
@@ -2891,6 +2899,28 @@
       
    
     
 
+    
+      
    
+        Any string to place route to separate routing table. If Logical Router
+        Port has configured value in  other than empty string, OVN
+        performs route lookup for all packets entering Logical Router ingress
+        pipeline from this port in the following manner:
+      
    
+
+      
      
+        
    • 
+          1. First lookup among "global" routes: routes without
+          route_table value set and routes to directly connected
+          networks.
+        
      • 
+        
    • 
+          2. Next lookup among routes with same route_table value
+          as specified in LRP's options:route_table field.
+        
      • 
+      
    
+        
+
     
       ovn-ic populates this key if the route is learned from the
       global  database.  In this case the value
diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at
index b6a8edb68..48ec1bf35 100644
--- a/tests/ovn-ic.at
+++ b/tests/ovn-ic.at
@@ -281,6 +281,7 @@ done
 
 AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl
 IPv4 Routes
+Route Table global:
              10.11.1.0/24               169.254.0.1 dst-ip
              10.11.2.0/24             169.254.100.2 dst-ip (learned)
              10.22.1.0/24               169.254.0.2 src-ip
@@ -299,6 +300,7 @@ ovn_as az1 ovn-nbctl set nb_global . options:ic-route-learn=false
 OVS_WAIT_WHILE([ovn_as az1 ovn-nbctl lr-route-list lr1 | grep learned])
 AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl
 IPv4 Routes
+Route Table global:
              10.11.1.0/24               169.254.0.1 dst-ip
              10.22.1.0/24               169.254.0.2 src-ip
 ])
@@ -314,6 +316,7 @@ ovn_as az1 ovn-nbctl set nb_global . options:ic-route-adv=false
 OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned])
 AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl
 IPv4 Routes
+Route Table global:
              10.11.2.0/24               169.254.0.1 dst-ip
              10.22.2.0/24               169.254.0.2 src-ip
 ])
@@ -332,6 +335,7 @@ done
 # Default route should NOT get advertised or learned, by default.
 AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl
 IPv4 Routes
+Route Table global:
              10.11.1.0/24             169.254.100.1 dst-ip (learned)
              10.11.2.0/24               169.254.0.1 dst-ip
              10.22.2.0/24               169.254.0.2 src-ip
diff --git a/tests/ovn-nbctl.at b/tests/ovn-nbctl.at
index 9b80ae410..ddb5536ce 100644
--- a/tests/ovn-nbctl.at
+++ b/tests/ovn-nbctl.at
@@ -1520,6 +1520,7 @@ AT_CHECK([ovn-nbctl --ecmp --policy=src-ip lr-route-add lr0 20.0.0.0/24 11.0.0.1
 
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
               10.0.0.0/24                  11.0.0.1 dst-ip
               10.0.1.0/24                  11.0.1.1 dst-ip lp0
              10.0.10.0/24                           dst-ip lp0
@@ -1534,6 +1535,7 @@ AT_CHECK([ovn-nbctl lrp-add lr0 lp1 f0:00:00:00:00:02 11.0.0.254/24])
 AT_CHECK([ovn-nbctl --may-exist lr-route-add lr0 10.0.0.111/24 11.0.0.1 lp1])
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
               10.0.0.0/24                  11.0.0.1 dst-ip lp1
               10.0.1.0/24                  11.0.1.1 dst-ip lp0
              10.0.10.0/24                           dst-ip lp0
@@ -1564,6 +1566,7 @@ AT_CHECK([ovn-nbctl --policy=src-ip lr-route-del lr0 9.16.1.0/24])
 
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
               10.0.0.0/24                  11.0.0.1 dst-ip lp1
              10.0.10.0/24                           dst-ip lp0
               10.0.0.0/24                  11.0.0.2 src-ip
@@ -1575,6 +1578,7 @@ AT_CHECK([ovn-nbctl --policy=dst-ip lr-route-del lr0 10.0.0.0/24])
 AT_CHECK([ovn-nbctl --policy=src-ip lr-route-del lr0 10.0.0.0/24])
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
              10.0.10.0/24                           dst-ip lp0
                 0.0.0.0/0               192.168.0.1 dst-ip
 ])
@@ -1585,6 +1589,7 @@ AT_CHECK([ovn-nbctl --policy=src-ip lr-route-add lr0 10.0.0.0/24 11.0.0.2])
 AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24])
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
              10.0.10.0/24                           dst-ip lp0
                 0.0.0.0/0               192.168.0.1 dst-ip
 ])
@@ -1601,6 +1606,7 @@ AT_CHECK([ovn-nbctl --ecmp lr-route-add lr0 10.0.0.0/24 11.0.0.3])
 AT_CHECK([ovn-nbctl --ecmp lr-route-add lr0 10.0.0.0/24 11.0.0.4 lp0])
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
               10.0.0.0/24                  11.0.0.1 dst-ip ecmp
               10.0.0.0/24                  11.0.0.2 dst-ip ecmp
               10.0.0.0/24                  11.0.0.3 dst-ip ecmp
@@ -1615,6 +1621,7 @@ dnl Delete ecmp routes
 AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24 11.0.0.1])
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
               10.0.0.0/24                  11.0.0.2 dst-ip ecmp
               10.0.0.0/24                  11.0.0.3 dst-ip ecmp
               10.0.0.0/24                  11.0.0.4 dst-ip lp0 ecmp
@@ -1622,12 +1629,14 @@ IPv4 Routes
 AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24 11.0.0.2])
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
               10.0.0.0/24                  11.0.0.3 dst-ip ecmp
               10.0.0.0/24                  11.0.0.4 dst-ip lp0 ecmp
 ])
 AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24 11.0.0.4 lp0])
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
               10.0.0.0/24                  11.0.0.3 dst-ip
 ])
 AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24 11.0.0.3])
@@ -1641,6 +1650,7 @@ AT_CHECK([ovn-nbctl lr-route-add lr0 2001:0db8:1::/64 2001:0db8:0:f103::1])
 
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv6 Routes
+Route Table global:
             2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
           2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
                      ::/0        2001:db8:0:f101::1 dst-ip
@@ -1650,6 +1660,7 @@ AT_CHECK([ovn-nbctl lr-route-del lr0 2001:0db8:0::/64])
 
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv6 Routes
+Route Table global:
           2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
                      ::/0        2001:db8:0:f101::1 dst-ip
 ])
@@ -1677,11 +1688,13 @@ AT_CHECK([ovn-nbctl --may-exist --ecmp-symmetric-reply lr-route-add lr0 2003:0db
 
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 IPv4 Routes
+Route Table global:
               10.0.0.0/24                  11.0.0.1 dst-ip
               10.0.1.0/24                  11.0.1.1 dst-ip lp0
                 0.0.0.0/0               192.168.0.1 dst-ip
 
 IPv6 Routes
+Route Table global:
             2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
           2001:db8:1::/64        2001:db8:0:f103::1 dst-ip ecmp
           2001:db8:1::/64        2001:db8:0:f103::2 dst-ip ecmp
@@ -1696,7 +1709,188 @@ AT_CHECK([ovn-nbctl lrp-add lr0 lr0-p0 00:00:01:01:02:03 192.168.10.1/24])
 bfd_uuid=$(ovn-nbctl create bfd logical_port=lr0-p0 dst_ip=100.0.0.50 status=down min_tx=250 min_rx=250 detect_mult=10)
 AT_CHECK([ovn-nbctl lr-route-add lr0 100.0.0.0/24 192.168.0.1])
 route_uuid=$(fetch_column nb:logical_router_static_route _uuid ip_prefix="100.0.0.0/24")
-AT_CHECK([ovn-nbctl set logical_router_static_route $route_uuid bfd=$bfd_uuid])])
+AT_CHECK([ovn-nbctl set logical_router_static_route $route_uuid bfd=$bfd_uuid])
+
+check ovn-nbctl lr-route-del lr0
+AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
+])
+
+dnl Check IPv4 routes in route table
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 0.0.0.0/0 192.168.0.1
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.1.1/24 11.0.1.1 lp0
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.0.1/24 11.0.0.1
+AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
+IPv4 Routes
+Route Table rtb-1:
+              10.0.0.0/24                  11.0.0.1 dst-ip
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+])
+
+check ovn-nbctl lr-route-del lr0
+AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
+])
+
+dnl Check IPv6 routes in route table
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 0:0:0:0:0:0:0:0/0 2001:0db8:0:f101::1
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 2001:0db8:0::/64 2001:0db8:0:f102::1 lp0
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 2001:0db8:1::/64 2001:0db8:0:f103::1
+
+AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
+IPv6 Routes
+Route Table rtb-1:
+            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+])
+
+dnl Check IPv4 and IPv6 routes in route table
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 0.0.0.0/0 192.168.0.1
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.1.1/24 11.0.1.1 lp0
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.0.1/24 11.0.0.1
+
+AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
+IPv4 Routes
+Route Table rtb-1:
+              10.0.0.0/24                  11.0.0.1 dst-ip
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+
+IPv6 Routes
+Route Table rtb-1:
+            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+])
+
+# Add routes in another route table
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 0.0.0.0/0 192.168.0.1
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 10.0.1.1/24 11.0.1.1 lp0
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 10.0.0.1/24 11.0.0.1
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 0:0:0:0:0:0:0:0/0 2001:0db8:0:f101::1
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 2001:0db8:0::/64 2001:0db8:0:f102::1 lp0
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 2001:0db8:1::/64 2001:0db8:0:f103::1
+
+AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
+IPv4 Routes
+Route Table rtb-1:
+              10.0.0.0/24                  11.0.0.1 dst-ip
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+
+Route Table rtb-2:
+              10.0.0.0/24                  11.0.0.1 dst-ip
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+
+IPv6 Routes
+Route Table rtb-1:
+            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+
+Route Table rtb-2:
+            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+])
+
+# Add routes to global route table
+check ovn-nbctl lr-route-add lr0 0.0.0.0/0 192.168.0.1
+check ovn-nbctl lr-route-add lr0 10.0.1.1/24 11.0.1.1 lp0
+check ovn-nbctl lr-route-add lr0 10.0.0.1/24 11.0.0.1
+check ovn-nbctl lr-route-add lr0 0:0:0:0:0:0:0:0/0 2001:0db8:0:f101::1
+check ovn-nbctl lr-route-add lr0 2001:0db8:0::/64 2001:0db8:0:f102::1 lp0
+check check ovn-nbctl lr-route-add lr0 2001:0db8:1::/64 2001:0db8:0:f103::1
+
+AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
+IPv4 Routes
+Route Table global:
+              10.0.0.0/24                  11.0.0.1 dst-ip
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+
+Route Table rtb-1:
+              10.0.0.0/24                  11.0.0.1 dst-ip
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+
+Route Table rtb-2:
+              10.0.0.0/24                  11.0.0.1 dst-ip
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+
+IPv6 Routes
+Route Table global:
+            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+
+Route Table rtb-1:
+            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+
+Route Table rtb-2:
+            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+])
+
+# delete IPv4 route from rtb-1
+check ovn-nbctl --route-table=rtb-1 lr-route-del lr0 10.0.0.0/24
+AT_CHECK([ovn-nbctl --route-table=rtb-1 lr-route-list lr0], [0], [dnl
+IPv4 Routes
+Route Table rtb-1:
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+
+IPv6 Routes
+Route Table rtb-1:
+            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+])
+
+# delete IPv6 route from rtb-2
+check ovn-nbctl --route-table=rtb-2 lr-route-del lr0 2001:db8::/64
+AT_CHECK([ovn-nbctl --route-table=rtb-2 lr-route-list lr0], [0], [dnl
+IPv4 Routes
+Route Table rtb-2:
+              10.0.0.0/24                  11.0.0.1 dst-ip
+              10.0.1.0/24                  11.0.1.1 dst-ip lp0
+                0.0.0.0/0               192.168.0.1 dst-ip
+
+IPv6 Routes
+Route Table rtb-2:
+          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
+                     ::/0        2001:db8:0:f101::1 dst-ip
+])
+
+check ovn-nbctl lr-route-del lr0
+
+# ECMP route in route table
+check ovn-nbctl --route-table=rtb1 lr-route-add lr0 0.0.0.0/0 192.168.0.1
+check ovn-nbctl --ecmp --route-table=rtb1 lr-route-add lr0 0.0.0.0/0 192.168.0.2
+
+# Negative route table case: same prefix
+AT_CHECK([ovn-nbctl --route-table=rtb1 lr-route-add lr0 0.0.0.0/0 192.168.0.1], [1], [], [dnl
+ovn-nbctl: duplicate prefix: 0.0.0.0/0 (policy: dst-ip). Use option --ecmp to allow this for ECMP routing.
+])
+
+# Negative route table case: same prefix & nexthop with ecmp
+AT_CHECK([ovn-nbctl --ecmp --route-table=rtb1 lr-route-add lr0 0.0.0.0/0 192.168.0.2], [1], [], [dnl
+ovn-nbctl: duplicate nexthop for the same ECMP route
+])
+
+# Add routes to global route table
+check ovn-nbctl lrp-add lr0 lrp0 00:00:00:00:00:01 1.1.1.1/24
+check ovn-nbctl lrp-set-options lrp0 route_table=rtb1
+AT_CHECK([ovn-nbctl get logical-router-port lrp0 options:route_table], [0], [dnl
+rtb1
+])
+check `ovn-nbctl show lr0 | grep lrp0 -A3 | grep route_table=rtb1`
+])
 
 dnl ---------------------------------------------------------------------
 
diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
index 1c0cb7c0e..5d64390f5 100644
--- a/tests/ovn-northd.at
+++ b/tests/ovn-northd.at
@@ -5104,7 +5104,7 @@ check ovn-nbctl --wait=sb --ecmp-symmetric-reply lr-route-add lr0 1.0.0.1 192.16
 
 ovn-sbctl dump-flows lr0 > lr0flows
 AT_CHECK([grep -e "lr_in_ip_routing.*select" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl
-  table=??(lr_in_ip_routing   ), priority=65   , match=(ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);)
+  table=??(lr_in_ip_routing   ), priority=165  , match=(ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);)
 ])
 AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed 's/192\.168\.0\..0/192.168.0.??/' | sed 's/table=../table=??/' | sort], [0], [dnl
   table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && reg8[[16..31]] == 1), action=(reg0 = 192.168.0.??; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; next;)
@@ -5117,7 +5117,7 @@ check ovn-nbctl --wait=sb --ecmp-symmetric-reply lr-route-add lr0 1.0.0.1 192.16
 
 ovn-sbctl dump-flows lr0 > lr0flows
 AT_CHECK([grep -e "lr_in_ip_routing.*select" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl
-  table=??(lr_in_ip_routing   ), priority=65   , match=(ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);)
+  table=??(lr_in_ip_routing   ), priority=165  , match=(ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);)
 ])
 AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed 's/192\.168\.0\..0/192.168.0.??/' | sed 's/table=../table=??/' | sort], [0], [dnl
   table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && reg8[[16..31]] == 1), action=(reg0 = 192.168.0.??; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; next;)
@@ -5132,14 +5132,14 @@ check ovn-nbctl --wait=sb lr-route-add lr0 1.0.0.0/24 192.168.0.10
 ovn-sbctl dump-flows lr0 > lr0flows
 
 AT_CHECK([grep -e "lr_in_ip_routing.*192.168.0.10" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl
-  table=??(lr_in_ip_routing   ), priority=49   , match=(ip4.dst == 1.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=149  , match=(ip4.dst == 1.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;)
 ])
 
 check ovn-nbctl --wait=sb lr-route-add lr0 2.0.0.0/24 lr0-public
 
 ovn-sbctl dump-flows lr0 > lr0flows
 AT_CHECK([grep -e "lr_in_ip_routing.*2.0.0.0" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl
-  table=??(lr_in_ip_routing   ), priority=49   , match=(ip4.dst == 2.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=149  , match=(ip4.dst == 2.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; next;)
 ])
 
 AT_CLEANUP
@@ -5225,3 +5225,71 @@ AT_CHECK([grep lr_in_gw_redirect lrflows | grep cr-DR | sed 's/table=../table=??
 
 AT_CLEANUP
 ])
+
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([route tables -- flows])
+AT_KEYWORDS([route-tables-flows])
+ovn_start
+
+check ovn-nbctl lr-add lr0
+check ovn-nbctl lrp-add lr0 lrp0 00:00:00:00:00:01 192.168.0.1/24
+check ovn-nbctl lrp-add lr0 lrp1 00:00:00:00:01:01 192.168.1.1/24
+check ovn-nbctl lrp-add lr0 lrp2 00:00:00:00:02:01 192.168.2.1/24
+check ovn-nbctl lrp-set-options lrp1 route_table=rtb-1
+check ovn-nbctl lrp-set-options lrp2 route_table=rtb-2
+
+check ovn-nbctl lr-route-add lr0 0.0.0.0/0 192.168.0.10
+check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 192.168.0.0/24 192.168.1.10
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 0.0.0.0/0 192.168.0.10
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 1.1.1.1/32 192.168.0.20
+check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 2.2.2.2/32 192.168.0.30
+check ovn-nbctl --route-table=rtb-2 --ecmp lr-route-add lr0 2.2.2.2/32 192.168.0.31
+check ovn-nbctl --wait=sb sync
+
+ovn-sbctl dump-flows lr0 > lr0flows
+AT_CAPTURE_FILE([lr0flows])
+
+AT_CHECK([grep -e "lr_in_ip_routing_pre.*match=(1)" lr0flows | sed 's/table=../table=??/'], [0], [dnl
+  table=??(lr_in_ip_routing_pre), priority=0    , match=(1), action=(next;)
+])
+
+p1_reg=$(grep -oP "lr_in_ip_routing_pre.*lrp1.*action=\(reg7 = \K." lr0flows)
+p2_reg=$(grep -oP "lr_in_ip_routing_pre.*lrp2.*action=\(reg7 = \K." lr0flows)
+echo $p1_reg
+echo $p2_reg
+
+# exact register values are not predictable
+if [[ $p1_reg -eq 2 ] && [ $p2_reg -eq 1 ]]; then
+  echo "swap reg values in dump"
+  sed -i -r s'/^(.*lrp2.*action=\(reg7 = )(1)(.*)/\12\3/g' lr0flows  # "reg7 = 1" -> "reg7 = 2"
+  sed -i -r s'/^(.*lrp1.*action=\(reg7 = )(2)(.*)/\11\3/g' lr0flows  # "reg7 = 2" -> "reg7 = 1"
+  sed -i -r s'/^(.*match=\(reg7 == )(2)( &&.*lrp1.*)/\11\3/g' lr0flows  # "reg7 == 2" -> "reg7 == 1"
+  sed -i -r s'/^(.*match=\(reg7 == )(1)( &&.*lrp0.*)/\12\3/g' lr0flows  # "reg7 == 1" -> "reg7 == 2"
+fi
+
+check test $p1_reg != $p2_reg -a $((p1_reg * p2_reg)) -eq 2
+
+AT_CHECK([grep "lr_in_ip_routing_pre" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl
+  table=??(lr_in_ip_routing_pre), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_ip_routing_pre), priority=100  , match=(inport == "lrp1"), action=(reg7 = 1; next;)
+  table=??(lr_in_ip_routing_pre), priority=100  , match=(inport == "lrp2"), action=(reg7 = 2; next;)
+])
+
+grep -e "(lr_in_ip_routing   ).*outport" lr0flows
+
+AT_CHECK([grep -e "(lr_in_ip_routing   ).*outport" lr0flows | sed 's/table=../table=??/' | sort], [0], [dnl
+  table=??(lr_in_ip_routing   ), priority=1    , match=(reg7 == 2 && ip4.dst == 0.0.0.0/0), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:00:00:00:01; outport = "lrp0"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=101  , match=(ip4.dst == 0.0.0.0/0), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:00:00:00:01; outport = "lrp0"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=149  , match=(ip4.dst == 192.168.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.0.1; eth.src = 00:00:00:00:00:01; outport = "lrp0"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=149  , match=(ip4.dst == 192.168.1.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.1.1; eth.src = 00:00:00:00:01:01; outport = "lrp1"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=149  , match=(ip4.dst == 192.168.2.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 = 192.168.2.1; eth.src = 00:00:00:00:02:01; outport = "lrp2"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=229  , match=(inport == "lrp0" && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:1; eth.src = 00:00:00:00:00:01; outport = "lrp0"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=229  , match=(inport == "lrp1" && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:101; eth.src = 00:00:00:00:01:01; outport = "lrp1"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=229  , match=(inport == "lrp2" && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:201; eth.src = 00:00:00:00:02:01; outport = "lrp2"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=49   , match=(reg7 == 1 && ip4.dst == 192.168.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.1.10; reg1 = 192.168.1.1; eth.src = 00:00:00:00:01:01; outport = "lrp1"; flags.loopback = 1; next;)
+  table=??(lr_in_ip_routing   ), priority=65   , match=(reg7 == 2 && ip4.dst == 1.1.1.1/32), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.20; reg1 = 192.168.0.1; eth.src = 00:00:00:00:00:01; outport = "lrp0"; flags.loopback = 1; next;)
+])
+
+AT_CLEANUP
+])
diff --git a/tests/ovn.at b/tests/ovn.at
index 5104a6895..bfb3c0b53 100644
--- a/tests/ovn.at
+++ b/tests/ovn.at
@@ -18064,7 +18064,7 @@ eth_dst=00000000ff01
 ip_src=$(ip_to_hex 10 0 0 10)
 ip_dst=$(ip_to_hex 172 168 0 101)
 send_icmp_packet 1 1 $eth_src $eth_dst $ip_src $ip_dst c4c9 0000000000000000000000
-AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | awk '/table=25, n_packets=1, n_bytes=45/{print $7" "$8}'],[0],[dnl
+AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | awk '/table=26, n_packets=1, n_bytes=45/{print $7" "$8}'],[0],[dnl
 priority=80,ip,reg15=0x3,metadata=0x3,nw_src=10.0.0.10 actions=drop
 ])
 
@@ -22248,6 +22248,433 @@ OVN_CLEANUP([hv1])
 AT_CLEANUP
 ])
 
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([route tables -- global routes])
+ovn_start
+
+# Logical network:
+# ls1 (192.168.1.0/24) - lrp-lr1-ls1 - lr1 - lrp-lr1-ls2 - ls2 (192.168.2.0/24)
+#
+# ls1 has lsp11 (192.168.1.11) and ls2 has lsp21 (192.168.2.21) and lsp22
+# (192.168.2.22)
+#
+# lrp-lr1-ls1 set options:route_table=rtb-1
+#
+# Static routes on lr1:
+# 0.0.0.0/0 nexthop 192.168.2.21
+# 1.1.1.1/32 nexthop 192.168.2.22 route_table=rtb-1
+#
+# Test 1:
+# lsp11 send packet to 2.2.2.2
+#
+# Expected result:
+# lsp21 should receive traffic, lsp22 should not receive any traffic
+#
+# Test 2:
+# lsp11 send packet to 1.1.1.1
+#
+# Expected result:
+# lsp21 should receive traffic, lsp22 should not receive any traffic
+
+ovn-nbctl lr-add lr1
+
+for i in 1 2; do
+    ovn-nbctl ls-add ls${i}
+    ovn-nbctl lrp-add lr1 lrp-lr1-ls${i} 00:00:00:01:0${i}:01 192.168.${i}.1/24
+    ovn-nbctl lsp-add ls${i} lsp-ls${i}-lr1 -- lsp-set-type lsp-ls${i}-lr1 router \
+        -- lsp-set-options lsp-ls${i}-lr1 router-port=lrp-lr1-ls${i} \
+        -- lsp-set-addresses lsp-ls${i}-lr1 router
+done
+
+# install static routes
+ovn-nbctl lr-route-add lr1 0.0.0.0/0 192.168.2.21
+ovn-nbctl --route-table=rtb-1 lr-route-add lr1 1.1.1.1/32 192.168.2.22
+
+# set lrp-lr1-ls1 route table
+ovn-nbctl lrp-set-options lrp-lr1-ls1 route_table=rtb-1
+
+# Create logical ports
+ovn-nbctl lsp-add ls1 lsp11 -- \
+    lsp-set-addresses lsp11 "f0:00:00:00:01:11 192.168.1.11"
+ovn-nbctl lsp-add ls2 lsp21 -- \
+    lsp-set-addresses lsp21 "f0:00:00:00:02:21 192.168.2.21"
+ovn-nbctl lsp-add ls2 lsp22 -- \
+    lsp-set-addresses lsp22 "f0:00:00:00:02:22 192.168.2.22"
+
+net_add n1
+sim_add hv1
+as hv1
+ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.1
+ovs-vsctl -- add-port br-int hv1-vif1 -- \
+    set interface hv1-vif1 external-ids:iface-id=lsp11 \
+    options:tx_pcap=hv1/vif1-tx.pcap \
+    options:rxq_pcap=hv1/vif1-rx.pcap \
+    ofport-request=1
+
+ovs-vsctl -- add-port br-int hv1-vif2 -- \
+    set interface hv1-vif2 external-ids:iface-id=lsp21 \
+    options:tx_pcap=hv1/vif2-tx.pcap \
+    options:rxq_pcap=hv1/vif2-rx.pcap \
+    ofport-request=2
+
+ovs-vsctl -- add-port br-int hv1-vif3 -- \
+    set interface hv1-vif3 external-ids:iface-id=lsp22 \
+    options:tx_pcap=hv1/vif3-tx.pcap \
+    options:rxq_pcap=hv1/vif3-rx.pcap \
+    ofport-request=3
+
+# wait for earlier changes to take effect
+check ovn-nbctl --wait=hv sync
+wait_for_ports_up
+
+for i in 1 2; do
+    packet="inport==\"lsp11\" && eth.src==f0:00:00:00:01:11 && eth.dst==00:00:00:01:01:01 &&
+            ip4 && ip.ttl==64 && ip4.src==192.168.1.11 && ip4.dst==$i.$i.$i.$i && icmp"
+    AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"])
+
+    # Assume all packets go to lsp21.
+    exp_packet="eth.src==00:00:00:01:02:01 && eth.dst==f0:00:00:00:02:21 &&
+            ip4 && ip.ttl==63 && ip4.src==192.168.1.11 && ip4.dst==$i.$i.$i.$i && icmp"
+    echo $exp_packet | ovstest test-ovn expr-to-packets >> expected_lsp21
+done
+> expected_lsp22
+
+# lsp21 should recieve 2 packets and lsp22 should recieve no packets
+OVS_WAIT_UNTIL([
+    rcv_n1=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > lsp21.packets && cat lsp21.packets | wc -l`
+    rcv_n2=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif3-tx.pcap > lsp22.packets && cat lsp22.packets | wc -l`
+    echo $rcv_n1 $rcv_n2
+    test $rcv_n1 -eq 2 -a $rcv_n2 -eq 0])
+
+for i in 1 2; do
+    sort expected_lsp2$i > expout
+    AT_CHECK([cat lsp2${i}.packets | sort], [0], [expout])
+done
+
+OVN_CLEANUP([hv1])
+AT_CLEANUP
+])
+
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([route tables -- directly connected routes])
+ovn_start
+
+# Logical network:
+# ls1 (192.168.1.0/24) - lrp-lr1-ls1 - lr1 - lrp-lr1-ls2 - ls2 (192.168.2.0/24)
+#
+# ls1 has lsp11 (192.168.1.11) and ls2 has lsp21 (192.168.2.21)
+#
+# lrp-lr1-ls1 set options:route_table=rtb-1
+#
+# Static routes on lr1:
+# 192.168.2.0/25 nexthop 192.168.1.11 route_table=rtb-1
+#
+# Test 1:
+# lsp11 send packet to 192.168.2.21
+#
+# Expected result:
+# lsp21 should receive traffic, lsp11 should not receive any traffic
+
+ovn-nbctl lr-add lr1
+
+for i in 1 2; do
+    ovn-nbctl ls-add ls${i}
+    ovn-nbctl lrp-add lr1 lrp-lr1-ls${i} 00:00:00:01:0${i}:01 192.168.${i}.1/24
+    ovn-nbctl lsp-add ls${i} lsp-ls${i}-lr1 -- lsp-set-type lsp-ls${i}-lr1 router \
+        -- lsp-set-options lsp-ls${i}-lr1 router-port=lrp-lr1-ls${i} \
+        -- lsp-set-addresses lsp-ls${i}-lr1 router
+done
+
+# install static route, which overrides directly-connected routes
+ovn-nbctl --route-table=rtb-1 lr-route-add lr1 192.168.2.0/25 192.168.1.11
+
+# set lrp-lr1-ls1 route table
+ovn-nbctl lrp-set-options lrp-lr1-ls1 route_table=rtb-1
+
+# Create logical ports
+ovn-nbctl lsp-add ls1 lsp11 -- \
+    lsp-set-addresses lsp11 "f0:00:00:00:01:11 192.168.1.11"
+ovn-nbctl lsp-add ls2 lsp21 -- \
+    lsp-set-addresses lsp21 "f0:00:00:00:02:21 192.168.2.21"
+
+net_add n1
+sim_add hv1
+as hv1
+ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.1
+ovs-vsctl -- add-port br-int hv1-vif1 -- \
+    set interface hv1-vif1 external-ids:iface-id=lsp11 \
+    options:tx_pcap=hv1/vif1-tx.pcap \
+    options:rxq_pcap=hv1/vif1-rx.pcap \
+    ofport-request=1
+
+ovs-vsctl -- add-port br-int hv1-vif2 -- \
+    set interface hv1-vif2 external-ids:iface-id=lsp21 \
+    options:tx_pcap=hv1/vif2-tx.pcap \
+    options:rxq_pcap=hv1/vif2-rx.pcap \
+    ofport-request=2
+
+# wait for earlier changes to take effect
+check ovn-nbctl --wait=hv sync
+wait_for_ports_up
+
+packet="inport==\"lsp11\" && eth.src==f0:00:00:00:01:11 && eth.dst==00:00:00:01:01:01 &&
+        ip4 && ip.ttl==64 && ip4.src==192.168.1.11 && ip4.dst==192.168.2.21 && icmp"
+AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"])
+
+# Assume all packets go to lsp21.
+exp_packet="eth.src==00:00:00:01:02:01 && eth.dst==f0:00:00:00:02:21 &&
+        ip4 && ip.ttl==63 && ip4.src==192.168.1.11 && ip4.dst==192.168.2.21 && icmp"
+echo $exp_packet | ovstest test-ovn expr-to-packets >> expected_lsp21
+> expected_lsp11
+
+# lsp21 should recieve 1 icmp packet and lsp11 should recieve no packets
+OVS_WAIT_UNTIL([
+    rcv_n11=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif1-tx.pcap > lsp11.packets && cat lsp11.packets | wc -l`
+    rcv_n21=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > lsp21.packets && cat lsp21.packets | wc -l`
+    echo $rcv_n11 $rcv_n21
+    test $rcv_n11 -eq 0 -a $rcv_n21 -eq 1])
+
+for i in 11 21; do
+    sort expected_lsp$i > expout
+    AT_CHECK([cat lsp${i}.packets | sort], [0], [expout])
+done
+
+OVN_CLEANUP([hv1])
+AT_CLEANUP
+])
+
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([route tables -- overlapping subnets])
+ovn_start
+
+# Logical network:
+#
+# ls1 (192.168.1.0/24) - lrp-lr1-ls1 -\   /- lrp-lr1-ls2 - ls2 (192.168.2.0/24)
+#                                      lr1
+# ls3 (192.168.3.0/24) - lrp-lr1-ls3 -/   \- lrp-lr1-ls4 - ls4 (192.168.4.0/24)
+#
+# ls1 has lsp11 (192.168.1.11)
+# ls2 has lsp21 (192.168.2.21)
+# ls3 has lsp31 (192.168.3.31)
+# ls4 has lsp41 (192.168.4.41)
+#
+# lrp-lr1-ls1 set options:route_table=rtb-1
+# lrp-lr1-ls2 set options:route_table=rtb-2
+#
+# Static routes on lr1:
+# 10.0.0.0/24 nexthop 192.168.3.31 route_table=rtb-1
+# 10.0.0.0/24 nexthop 192.168.4.41 route_table=rtb-2
+#
+# Test 1:
+# lsp11 send packet to 10.0.0.1
+#
+# Expected result:
+# lsp31 should receive traffic, lsp41 should not receive any traffic
+#
+# Test 2:
+# lsp21 send packet to 10.0.0.1
+#
+# Expected result:
+# lsp41 should receive traffic, lsp31 should not receive any traffic
+
+ovn-nbctl lr-add lr1
+
+# Create logical topology
+for i in $(seq 1 4); do
+    ovn-nbctl ls-add ls${i}
+    ovn-nbctl lrp-add lr1 lrp-lr1-ls${i} 00:00:00:01:0${i}:01 192.168.${i}.1/24
+    ovn-nbctl lsp-add ls${i} lsp-ls${i}-lr1 -- lsp-set-type lsp-ls${i}-lr1 router \
+        -- lsp-set-options lsp-ls${i}-lr1 router-port=lrp-lr1-ls${i} \
+        -- lsp-set-addresses lsp-ls${i}-lr1 router
+    ovn-nbctl lsp-add ls$i lsp${i}1 -- \
+        lsp-set-addresses lsp${i}1 "f0:00:00:00:0${i}:1${i} 192.168.${i}.${i}1"
+done
+
+# install static routes
+ovn-nbctl --route-table=rtb-1 lr-route-add lr1 10.0.0.0/24 192.168.3.31
+ovn-nbctl --route-table=rtb-2 lr-route-add lr1 10.0.0.0/24 192.168.4.41
+
+# set lrp-lr1-ls{1,2} route tables
+ovn-nbctl lrp-set-options lrp-lr1-ls1 route_table=rtb-1
+ovn-nbctl lrp-set-options lrp-lr1-ls2 route_table=rtb-2
+
+net_add n1
+sim_add hv1
+as hv1
+ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.1
+
+for i in $(seq 1 4); do
+    ovs-vsctl -- add-port br-int hv1-vif${i} -- \
+        set interface hv1-vif${i} external-ids:iface-id=lsp${i}1 \
+        options:tx_pcap=hv1/vif${i}-tx.pcap \
+        options:rxq_pcap=hv1/vif${i}-rx.pcap \
+        ofport-request=${i}
+done
+
+# wait for earlier changes to take effect
+check ovn-nbctl --wait=hv sync
+wait_for_ports_up
+
+# lsp31 should recieve packet coming from lsp11
+# lsp41 should recieve packet coming from lsp21
+for i in $(seq 1 2); do
+    di=$(( i + 2))  # dst index
+    ri=$(( 5 - i))  # reverse index
+    packet="inport==\"lsp${i}1\" && eth.src==f0:00:00:00:0${i}:1${i} &&
+            eth.dst==00:00:00:01:0${i}:01 && ip4 && ip.ttl==64 &&
+            ip4.src==192.168.${i}.${i}1 && ip4.dst==10.0.0.1 && icmp"
+    AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"])
+
+    # Assume all packets go to lsp${di}1.
+    exp_packet="eth.src==00:00:00:01:0${di}:01 && eth.dst==f0:00:00:00:0${di}:1${di} &&
+            ip4 && ip.ttl==63 && ip4.src==192.168.${i}.${i}1 && ip4.dst==10.0.0.1 && icmp"
+    echo $exp_packet | ovstest test-ovn expr-to-packets >> expected_lsp${di}1
+    > expected_lsp${ri}1
+
+    OVS_WAIT_UNTIL([
+        rcv_n1=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif${di}-tx.pcap > lsp${di}1.packets && cat lsp${di}1.packets | wc -l`
+        rcv_n2=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif${ri}-tx.pcap > lsp${ri}1.packets && cat lsp${ri}1.packets | wc -l`
+        echo $rcv_n1 $rcv_n2
+        test $rcv_n1 -eq 1 -a $rcv_n2 -eq 0])
+
+    for j in "${di}1" "${ri}1"; do
+        sort expected_lsp${j} > expout
+        AT_CHECK([cat lsp${j}.packets | sort], [0], [expout])
+    done
+
+    # cleanup tx pcap files
+    for j in "${di}1" "${ri}1"; do
+        ovs-vsctl -- remove interface hv1-vif${di} options tx_pcap
+        > hv1/vif${di}-tx.pcap
+        ovs-vsctl -- set interface hv1-vif${di} external-ids:iface-id=lsp${di}1 \
+            options:tx_pcap=hv1/vif${di}-tx.pcap
+    done
+done
+
+OVN_CLEANUP([hv1])
+AT_CLEANUP
+])
+
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([route tables IPv6 -- overlapping subnets])
+ovn_start
+
+# Logical network:
+#
+# ls1 (2001:db8:1::/64) - lrp-lr1-ls1 -\   /- lrp-lr1-ls2 - ls2 (2001:db8:2::/64)
+#                                       lr1
+# ls3 (2001:db8:3::/64) - lrp-lr1-ls3 -/   \- lrp-lr1-ls4 - ls4 (2001:db8:4::/64)
+#
+# ls1 has lsp11 (2001:db8:1::11)
+# ls2 has lsp21 (2001:db8:2::21)
+# ls3 has lsp31 (2001:db8:3::31)
+# ls4 has lsp41 (2001:db8:4::41)
+#
+# lrp-lr1-ls1 set options:route_table=rtb-1
+# lrp-lr1-ls2 set options:route_table=rtb-2
+#
+# Static routes on lr1:
+# 2001:db8:2000::/64 nexthop 2001:db8:3::31 route_table=rtb-1
+# 2001:db8:2000::/64 nexthop 2001:db8:3::41 route_table=rtb-2
+#
+# Test 1:
+# lsp11 send packet to 2001:db8:2000::1
+#
+# Expected result:
+# lsp31 should receive traffic, lsp41 should not receive any traffic
+#
+# Test 2:
+# lsp21 send packet to 2001:db8:2000::1
+#
+# Expected result:
+# lsp41 should receive traffic, lsp31 should not receive any traffic
+
+ovn-nbctl lr-add lr1
+
+# Create logical topology
+for i in $(seq 1 4); do
+    ovn-nbctl ls-add ls${i}
+    ovn-nbctl lrp-add lr1 lrp-lr1-ls${i} 00:00:00:01:0${i}:01 2001:db8:${i}::1/64
+    ovn-nbctl lsp-add ls${i} lsp-ls${i}-lr1 -- lsp-set-type lsp-ls${i}-lr1 router \
+        -- lsp-set-options lsp-ls${i}-lr1 router-port=lrp-lr1-ls${i} \
+        -- lsp-set-addresses lsp-ls${i}-lr1 router
+    ovn-nbctl lsp-add ls$i lsp${i}1 -- \
+        lsp-set-addresses lsp${i}1 "f0:00:00:00:0${i}:1${i} 2001:db8:${i}::${i}1"
+done
+
+# install static routes
+ovn-nbctl --route-table=rtb-1 lr-route-add lr1 2001:db8:2000::/64 2001:db8:3::31
+ovn-nbctl --route-table=rtb-2 lr-route-add lr1 2001:db8:2000::/64 2001:db8:4::41
+
+# set lrp-lr1-ls{1,2} route tables
+ovn-nbctl lrp-set-options lrp-lr1-ls1 route_table=rtb-1
+ovn-nbctl lrp-set-options lrp-lr1-ls2 route_table=rtb-2
+
+net_add n1
+sim_add hv1
+as hv1
+ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.1
+
+for i in $(seq 1 4); do
+    ovs-vsctl -- add-port br-int hv1-vif${i} -- \
+        set interface hv1-vif${i} external-ids:iface-id=lsp${i}1 \
+        options:tx_pcap=hv1/vif${i}-tx.pcap \
+        options:rxq_pcap=hv1/vif${i}-rx.pcap \
+        ofport-request=${i}
+done
+
+# wait for earlier changes to take effect
+AT_CHECK([ovn-nbctl --timeout=3 --wait=hv sync], [0], [ignore])
+
+# lsp31 should recieve packet coming from lsp11
+# lsp41 should recieve packet coming from lsp21
+for i in $(seq 1 2); do
+    di=$(( i + 2))  # dst index
+    ri=$(( 5 - i))  # reverse index
+    packet="inport==\"lsp${i}1\" && eth.src==f0:00:00:00:0${i}:1${i} &&
+            eth.dst==00:00:00:01:0${i}:01 && ip6 && ip.ttl==64 &&
+            ip6.src==2001:db8:${i}::${i}1 && ip6.dst==2001:db8:2000::1 && icmp6"
+    AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"])
+
+    # Assume all packets go to lsp${di}1.
+    exp_packet="eth.src==00:00:00:01:0${di}:01 && eth.dst==f0:00:00:00:0${di}:1${di} && ip6 &&
+                ip.ttl==63 && ip6.src==2001:db8:${i}::${i}1 && ip6.dst==2001:db8:2000::1 && icmp6"
+    echo $exp_packet | ovstest test-ovn expr-to-packets >> expected_lsp${di}1
+    > expected_lsp${ri}1
+
+    OVS_WAIT_UNTIL([
+        rcv_n1=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif${di}-tx.pcap > lsp${di}1.packets && cat lsp${di}1.packets | wc -l`
+        rcv_n2=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/vif${ri}-tx.pcap > lsp${ri}1.packets && cat lsp${ri}1.packets | wc -l`
+        echo $rcv_n1 $rcv_n2
+        test $rcv_n1 -eq 1 -a $rcv_n2 -eq 0])
+
+    for j in "${di}1" "${ri}1"; do
+        sort expected_lsp${j} > expout
+        AT_CHECK([cat lsp${j}.packets | sort], [0], [expout])
+    done
+
+    # cleanup tx pcap files
+    for j in "${di}1" "${ri}1"; do
+        ovs-vsctl -- remove interface hv1-vif${di} options tx_pcap
+        > hv1/vif${di}-tx.pcap
+        ovs-vsctl -- set interface hv1-vif${di} external-ids:iface-id=lsp${di}1 \
+            options:tx_pcap=hv1/vif${di}-tx.pcap
+    done
+done
+
+OVN_CLEANUP([hv1])
+AT_CLEANUP
+])
+
+
 OVN_FOR_EACH_NORTHD([
 AT_SETUP([forwarding group: 3 HVs, 1 LR, 2 LS])
 AT_KEYWORDS([forwarding-group])
@@ -23003,7 +23430,7 @@ ovn-sbctl dump-flows > sbflows
 AT_CAPTURE_FILE([sbflows])
 AT_CAPTURE_FILE([offlows])
 OVS_WAIT_UNTIL([
-    as hv1 ovs-ofctl dump-flows br-int table=20 > offlows
+    as hv1 ovs-ofctl dump-flows br-int table=21 > offlows
     test $(grep -c "load:0x64->NXM_NX_PKT_MARK" offlows) = 1 && \
     test $(grep -c "load:0x3->NXM_NX_PKT_MARK" offlows) = 1 && \
     test $(grep -c "load:0x4->NXM_NX_PKT_MARK" offlows) = 1 && \
@@ -23096,12 +23523,12 @@ send_ipv4_pkt hv1 hv1-vif1 505400000003 00000000ff01 \
     $(ip_to_hex 10 0 0 3) $(ip_to_hex 172 168 0 120)
 
 OVS_WAIT_UNTIL([
-    test 1 -eq $(as hv1 ovs-ofctl dump-flows br-int table=20 | \
+    test 1 -eq $(as hv1 ovs-ofctl dump-flows br-int table=21 | \
     grep "load:0x2->NXM_NX_PKT_MARK" -c)
 ])
 
 AT_CHECK([
-    test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=20 | \
+    test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=21 | \
     grep "load:0x64->NXM_NX_PKT_MARK" -c)
 ])
 
@@ -23804,7 +24231,7 @@ AT_CHECK([
         grep "priority=100" | \
         grep -c "ct(commit,zone=NXM_NX_REG11\\[[0..15\\]],.*exec(move:NXM_OF_ETH_SRC\\[[\\]]->NXM_NX_CT_LABEL\\[[32..79\\]],load:0x[[0-9]]->NXM_NX_CT_LABEL\\[[80..95\\]]))"
 
-        grep table=22 hv${hv}flows | \
+        grep table=23 hv${hv}flows | \
         grep "priority=200" | \
         grep -c "actions=move:NXM_NX_CT_LABEL\\[[32..79\\]]->NXM_OF_ETH_DST\\[[\\]]"
     done; :], [0], [dnl
@@ -23929,7 +24356,7 @@ AT_CHECK([
         grep "priority=100" | \
         grep -c "ct(commit,zone=NXM_NX_REG11\\[[0..15\\]],.*exec(move:NXM_OF_ETH_SRC\\[[\\]]->NXM_NX_CT_LABEL\\[[32..79\\]],load:0x[[0-9]]->NXM_NX_CT_LABEL\\[[80..95\\]]))"
 
-        grep table=22 hv${hv}flows | \
+        grep table=23 hv${hv}flows | \
         grep "priority=200" | \
         grep -c "actions=move:NXM_NX_CT_LABEL\\[[32..79\\]]->NXM_OF_ETH_DST\\[[\\]]"
     done; :], [0], [dnl
@@ -24551,7 +24978,7 @@ AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep "actions=controller" | grep
 ])
 
 # The packet should've been dropped in the lr_in_arp_resolve stage.
-AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep -E "table=22, n_packets=1,.* priority=1,ip,metadata=0x${sw_key},nw_dst=10.0.1.1 actions=drop" -c], [0], [dnl
+AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep -E "table=23, n_packets=1,.* priority=1,ip,metadata=0x${sw_key},nw_dst=10.0.1.1 actions=drop" -c], [0], [dnl
 1
 ])
 
diff --git a/utilities/ovn-nbctl.c b/utilities/ovn-nbctl.c
index 10217dcd5..f4a8504e6 100644
--- a/utilities/ovn-nbctl.c
+++ b/utilities/ovn-nbctl.c
@@ -329,6 +329,8 @@ Logical router port commands:\n\
                             add logical port PORT on ROUTER\n\
   lrp-set-gateway-chassis PORT CHASSIS [PRIORITY]\n\
                             set gateway chassis for port PORT\n\
+  lrp-set-options PORT KEY=VALUE [KEY=VALUE]...\n\
+                            set router port options\n\
   lrp-del-gateway-chassis PORT CHASSIS\n\
                             delete gateway chassis from port PORT\n\
   lrp-get-gateway-chassis PORT\n\
@@ -351,11 +353,17 @@ Logical router port commands:\n\
                             ('overlay' or 'bridged')\n\
 \n\
 Route commands:\n\
-  [--policy=POLICY] [--ecmp] [--ecmp-symmetric-reply] lr-route-add ROUTER \n\
-                            PREFIX NEXTHOP [PORT]\n\
+  [--policy=POLICY]\n\
+  [--ecmp]\n\
+  [--ecmp-symmetric-reply]\n\
+  [--route-table=ROUTE_TABLE]\n\
+  lr-route-add ROUTER PREFIX NEXTHOP [PORT]\n\
                             add a route to ROUTER\n\
-  [--policy=POLICY] lr-route-del ROUTER [PREFIX [NEXTHOP [PORT]]]\n\
+  [--policy=POLICY]\n\
+  [--route-table=ROUTE_TABLE]\n\
+  lr-route-del ROUTER [PREFIX [NEXTHOP [PORT]]]\n\
                             remove routes from ROUTER\n\
+  [--route-table=ROUTE_TABLE]\n\
   lr-route-list ROUTER      print routes for ROUTER\n\
 \n\
 Policy commands:\n\
@@ -743,6 +751,11 @@ print_lr(const struct nbrec_logical_router *lr, struct ds *s)
             ds_put_cstr(s, "]\n");
         }
 
+        const char *route_table = smap_get(&lrp->options, "route_table");
+        if (route_table) {
+            ds_put_format(s, "        route-table: %s\n", route_table);
+        }
+
         if (lrp->n_gateway_chassis) {
             const struct nbrec_gateway_chassis **gcs;
 
@@ -862,6 +875,7 @@ nbctl_pre_show(struct ctl_context *ctx)
     ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_name);
     ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_mac);
     ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_networks);
+    ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_options);
     ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_gateway_chassis);
 
     ovsdb_idl_add_column(ctx->idl, &nbrec_gateway_chassis_col_chassis_name);
@@ -4000,11 +4014,19 @@ nbctl_lr_policy_list(struct ctl_context *ctx)
 
 static struct nbrec_logical_router_static_route *
 nbctl_lr_get_route(const struct nbrec_logical_router *lr, char *prefix,
-                   char *next_hop, bool is_src_route, bool ecmp)
+                   char *next_hop, bool is_src_route, bool ecmp,
+                   char *route_table)
 {
     for (int i = 0; i < lr->n_static_routes; i++) {
         struct nbrec_logical_router_static_route *route = lr->static_routes[i];
 
+        /* Strict compare for route_table.
+         * If route_table was not specified,
+         * lookup for routes with empty route_table value. */
+        if (strcmp(route->route_table, route_table ? route_table : "")) {
+            continue;
+        }
+
         /* Compare route policy. */
         char *nb_policy = route->policy;
         bool nb_is_src_route = false;
@@ -4060,6 +4082,8 @@ nbctl_pre_lr_route_add(struct ctl_context *ctx)
                          &nbrec_logical_router_static_route_col_bfd);
     ovsdb_idl_add_column(ctx->idl,
                          &nbrec_logical_router_static_route_col_options);
+    ovsdb_idl_add_column(ctx->idl,
+                         &nbrec_logical_router_static_route_col_route_table);
 }
 
 static char * OVS_WARN_UNUSED_RESULT
@@ -4090,6 +4114,7 @@ nbctl_lr_route_add(struct ctl_context *ctx)
         }
     }
 
+    char *route_table = shash_find_data(&ctx->options, "--route-table");
     bool v6_prefix = false;
     prefix = normalize_ipv4_prefix_str(ctx->argv[2]);
     if (!prefix) {
@@ -4159,7 +4184,8 @@ nbctl_lr_route_add(struct ctl_context *ctx)
     bool ecmp = shash_find(&ctx->options, "--ecmp") != NULL ||
                 ecmp_symmetric_reply;
     struct nbrec_logical_router_static_route *route =
-        nbctl_lr_get_route(lr, prefix, next_hop, is_src_route, ecmp);
+        nbctl_lr_get_route(lr, prefix, next_hop, is_src_route, ecmp,
+                           route_table);
 
     /* Validations for nexthop = "discard" */
     if (is_discard_route) {
@@ -4223,7 +4249,8 @@ nbctl_lr_route_add(struct ctl_context *ctx)
     }
 
     struct nbrec_logical_router_static_route *discard_route =
-        nbctl_lr_get_route(lr, prefix, "discard", is_src_route, true);
+        nbctl_lr_get_route(lr, prefix, "discard", is_src_route, true,
+                           route_table);
     if (discard_route) {
         ctl_error(ctx, "discard nexthop for the same ECMP route exists.");
         goto cleanup;
@@ -4239,6 +4266,9 @@ nbctl_lr_route_add(struct ctl_context *ctx)
     if (policy) {
         nbrec_logical_router_static_route_set_policy(route, policy);
     }
+    if (route_table) {
+        nbrec_logical_router_static_route_set_route_table(route, route_table);
+    }
 
     if (ecmp_symmetric_reply) {
         const struct smap options = SMAP_CONST1(&options,
@@ -4282,6 +4312,8 @@ nbctl_pre_lr_route_del(struct ctl_context *ctx)
                          &nbrec_logical_router_static_route_col_nexthop);
     ovsdb_idl_add_column(ctx->idl,
                          &nbrec_logical_router_static_route_col_output_port);
+    ovsdb_idl_add_column(ctx->idl,
+                         &nbrec_logical_router_static_route_col_route_table);
 
 }
 
@@ -4295,6 +4327,7 @@ nbctl_lr_route_del(struct ctl_context *ctx)
         return;
     }
 
+    const char *route_table = shash_find_data(&ctx->options, "--route-table");
     const char *policy = shash_find_data(&ctx->options, "--policy");
     bool is_src_route = false;
     if (policy) {
@@ -4385,6 +4418,14 @@ nbctl_lr_route_del(struct ctl_context *ctx)
             }
         }
 
+        /* Strict compare for route_table.
+         * If route_table was not specified,
+         * lookup for routes with empty route_table value. */
+        if (strcmp(lr->static_routes[i]->route_table,
+                   route_table ? route_table : "")) {
+            continue;
+        }
+
         /* Compare output_port, if specified. */
         if (output_port) {
             char *rt_output_port = lr->static_routes[i]->output_port;
@@ -5108,6 +5149,41 @@ nbctl_pre_lrp_del_gateway_chassis(struct ctl_context *ctx)
     ovsdb_idl_add_column(ctx->idl, &nbrec_gateway_chassis_col_chassis_name);
 }
 
+static void
+nbctl_pre_lrp_options(struct ctl_context *ctx)
+{
+    ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_name);
+    ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_options);
+}
+
+static void
+nbctl_lrp_set_options(struct ctl_context *ctx)
+{
+    const char *id = ctx->argv[1];
+    const struct nbrec_logical_router_port *lrp = NULL;
+    size_t i;
+    struct smap options = SMAP_INITIALIZER(&options);
+
+    char *error = lrp_by_name_or_uuid(ctx, id, true, &lrp);
+    if (error) {
+        ctx->error = error;
+        return;
+    }
+    for (i = 2; i < ctx->argc; i++) {
+        char *key, *value;
+        value = xstrdup(ctx->argv[i]);
+        key = strsep(&value, "=");
+        if (value) {
+            smap_add(&options, key, value);
+        }
+        free(key);
+    }
+
+    nbrec_logical_router_port_set_options(lrp, &options);
+
+    smap_destroy(&options);
+}
+
 /* Removes logical router port 'lrp->gateway_chassis[idx]'. */
 static void
 remove_gc(const struct nbrec_logical_router_port *lrp, size_t idx)
@@ -5884,6 +5960,7 @@ route_cmp_details(const struct nbrec_logical_router_static_route *r1,
     }
     return r1->output_port ? 1 : -1;
 }
+
 struct ipv4_route {
     int priority;
     ovs_be32 addr;
@@ -5893,6 +5970,11 @@ struct ipv4_route {
 static int
 __ipv4_route_cmp(const struct ipv4_route *r1, const struct ipv4_route *r2)
 {
+    int rtb_cmp = strcmp(r1->route->route_table,
+                         r2->route->route_table);
+    if (rtb_cmp) {
+        return rtb_cmp;
+    }
     if (r1->priority != r2->priority) {
         return r1->priority > r2->priority ? -1 : 1;
     }
@@ -5924,6 +6006,11 @@ struct ipv6_route {
 static int
 __ipv6_route_cmp(const struct ipv6_route *r1, const struct ipv6_route *r2)
 {
+    int rtb_cmp = strcmp(r1->route->route_table,
+                         r2->route->route_table);
+    if (rtb_cmp) {
+        return rtb_cmp;
+    }
     if (r1->priority != r2->priority) {
         return r1->priority > r2->priority ? -1 : 1;
     }
@@ -6011,6 +6098,8 @@ nbctl_pre_lr_route_list(struct ctl_context *ctx)
                          &nbrec_logical_router_static_route_col_options);
     ovsdb_idl_add_column(ctx->idl,
                          &nbrec_logical_router_static_route_col_bfd);
+    ovsdb_idl_add_column(ctx->idl,
+                         &nbrec_logical_router_static_route_col_route_table);
 }
 
 static void
@@ -6028,12 +6117,17 @@ nbctl_lr_route_list(struct ctl_context *ctx)
         return;
     }
 
+    char *route_table = shash_find_data(&ctx->options, "--route-table");
+
     ipv4_routes = xmalloc(sizeof *ipv4_routes * lr->n_static_routes);
     ipv6_routes = xmalloc(sizeof *ipv6_routes * lr->n_static_routes);
 
     for (int i = 0; i < lr->n_static_routes; i++) {
         const struct nbrec_logical_router_static_route *route
             = lr->static_routes[i];
+        if (route_table && strcmp(route->route_table, route_table)) {
+            continue;
+        }
         unsigned int plen;
         ovs_be32 ipv4;
         const char *policy = route->policy ? route->policy : "dst-ip";
@@ -6074,6 +6168,7 @@ nbctl_lr_route_list(struct ctl_context *ctx)
     if (n_ipv4_routes) {
         ds_put_cstr(&ctx->output, "IPv4 Routes\n");
     }
+    const struct nbrec_logical_router_static_route *route;
     for (int i = 0; i < n_ipv4_routes; i++) {
         bool ecmp = false;
         if (i < n_ipv4_routes - 1 &&
@@ -6084,6 +6179,15 @@ nbctl_lr_route_list(struct ctl_context *ctx)
                                      &ipv4_routes[i - 1])) {
             ecmp = true;
         }
+
+        route = ipv4_routes[i].route;
+        if (!i || (i > 0 && strcmp(route->route_table,
+                                   ipv4_routes[i - 1].route->route_table))) {
+            ds_put_format(&ctx->output, "%sRoute Table %s:\n", i ? "\n" : "",
+                          strlen(route->route_table) ? route->route_table
+                                                     : "global");
+        }
+
         print_route(ipv4_routes[i].route, &ctx->output, ecmp);
     }
 
@@ -6101,6 +6205,15 @@ nbctl_lr_route_list(struct ctl_context *ctx)
                                      &ipv6_routes[i - 1])) {
             ecmp = true;
         }
+
+        route = ipv6_routes[i].route;
+        if (!i || (i > 0 && strcmp(route->route_table,
+                                   ipv6_routes[i - 1].route->route_table))) {
+            ds_put_format(&ctx->output, "%sRoute Table %s:\n", i ? "\n" : "",
+                          strlen(route->route_table) ? route->route_table
+                                                     : "global");
+        }
+
         print_route(ipv6_routes[i].route, &ctx->output, ecmp);
     }
 
@@ -6919,6 +7032,8 @@ static const struct ctl_command_syntax nbctl_commands[] = {
       "PORT CHASSIS [PRIORITY]",
       nbctl_pre_lrp_set_gateway_chassis, nbctl_lrp_set_gateway_chassis,
       NULL, "--may-exist", RW },
+    { "lrp-set-options", 1, INT_MAX, "PORT KEY=VALUE [KEY=VALUE]...",
+      nbctl_pre_lrp_options, nbctl_lrp_set_options, NULL, "", RW },
     { "lrp-del-gateway-chassis", 2, 2, "PORT CHASSIS",
       nbctl_pre_lrp_del_gateway_chassis, nbctl_lrp_del_gateway_chassis,
       NULL, "", RW },
@@ -6942,12 +7057,13 @@ static const struct ctl_command_syntax nbctl_commands[] = {
     /* logical router route commands. */
     { "lr-route-add", 3, 4, "ROUTER PREFIX NEXTHOP [PORT]",
       nbctl_pre_lr_route_add, nbctl_lr_route_add, NULL,
-      "--may-exist,--ecmp,--ecmp-symmetric-reply,--policy=,--bfd?", RW },
+      "--may-exist,--ecmp,--ecmp-symmetric-reply,--policy=,--route-table=,--bfd?",
+      RW },
     { "lr-route-del", 1, 4, "ROUTER [PREFIX [NEXTHOP [PORT]]]",
       nbctl_pre_lr_route_del, nbctl_lr_route_del, NULL,
-      "--if-exists,--policy=", RW },
+      "--if-exists,--policy=,--route-table=", RW },
     { "lr-route-list", 1, 1, "ROUTER", nbctl_pre_lr_route_list,
-      nbctl_lr_route_list, NULL, "", RO },
+      nbctl_lr_route_list, NULL, "--route-table=", RO },
 
     /* Policy commands */
     { "lr-policy-add", 4, INT_MAX,

From patchwork Fri Sep  3 10:34:40 2021
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Patchwork-Submitter: Vladislav Odintsov 
X-Patchwork-Id: 1524204
Return-Path: 
X-Original-To: incoming@patchwork.ozlabs.org
Delivered-To: patchwork-incoming@bilbo.ozlabs.org
Authentication-Results: ozlabs.org;
	dkim=fail reason="signature verification failed" (2048-bit key;
 unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256
 header.s=20210112 header.b=NOKNJ7Ex;
	dkim-atps=neutral
Authentication-Results: ozlabs.org;
 spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org
 (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org;
 envelope-from=ovs-dev-bounces@openvswitch.org; receiver=)
Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest
 SHA256)
	(No client certificate requested)
	by ozlabs.org (Postfix) with ESMTPS id 4H1Djb66YBz9s5R
	for ; Fri,  3 Sep 2021 20:36:19 +1000 (AEST)
Received: from localhost (localhost [127.0.0.1])
	by smtp2.osuosl.org (Postfix) with ESMTP id 2E8C1407E4;
	Fri,  3 Sep 2021 10:36:17 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
Received: from smtp2.osuosl.org ([127.0.0.1])
	by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id 28afp79qKoby; Fri,  3 Sep 2021 10:36:14 +0000 (UTC)
Received: from lists.linuxfoundation.org (lf-lists.osuosl.org
 [IPv6:2605:bc80:3010:104::8cd3:938])
	by smtp2.osuosl.org (Postfix) with ESMTPS id 217DD407FA;
	Fri,  3 Sep 2021 10:36:13 +0000 (UTC)
Received: from lf-lists.osuosl.org (localhost [127.0.0.1])
	by lists.linuxfoundation.org (Postfix) with ESMTP id E4AA2C0010;
	Fri,  3 Sep 2021 10:36:12 +0000 (UTC)
X-Original-To: dev@openvswitch.org
Delivered-To: ovs-dev@lists.linuxfoundation.org
Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137])
 by lists.linuxfoundation.org (Postfix) with ESMTP id 0F77BC0010
 for ; Fri,  3 Sep 2021 10:36:11 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by smtp4.osuosl.org (Postfix) with ESMTP id D60AB4267E
 for ; Fri,  3 Sep 2021 10:35:34 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
Authentication-Results: smtp4.osuosl.org (amavisd-new);
 dkim=pass (2048-bit key) header.d=gmail.com
Received: from smtp4.osuosl.org ([127.0.0.1])
 by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id wbacoUwPccUq for ;
 Fri,  3 Sep 2021 10:35:32 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.8.0
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [IPv6:2a00:1450:4864:20::62b])
 by smtp4.osuosl.org (Postfix) with ESMTPS id 644EA4266E
 for ; Fri,  3 Sep 2021 10:35:32 +0000 (UTC)
Received: by mail-ej1-x62b.google.com with SMTP id me10so11113053ejb.11
 for ; Fri, 03 Sep 2021 03:35:32 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=from:to:cc:subject:date:message-id:in-reply-to:references
 :mime-version:content-transfer-encoding;
 bh=ozy7zzoQUmoKkVTigAZ/iYaixlAtxNjgMmNELZBi86M=;
 b=NOKNJ7ExHqjN5zVetAk+/tAONII5zXFDRSQG1jXmg9sYzTc5zMxEH/G1lBubCpzcap
 jimrVd4g/Bv9iLQ3P7E09878JM2LG8XYMxY5b3Bfl7Hm5BLeekEM1dMpRLFKB/cIyEcs
 3TvQldAp1u7P3+dDejq6m3SEU25qlJmv3t32gof/CH2jJw+COo/jiLO4jg438HHKfp7v
 83YjhdU8HV51DLQWXM6wOAS5fMQ491Y9AR/cw9OO0uBErUEBIXuWiru64Jn3t9pqJGcC
 GKuJ6ViUxBCv0sJ0cFDnfsuAY/qMcZt172sdZMqGGyo2Ul9vG0yxkqSoWr7beOA08aiN
 wY9Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
 :references:mime-version:content-transfer-encoding;
 bh=ozy7zzoQUmoKkVTigAZ/iYaixlAtxNjgMmNELZBi86M=;
 b=r2isT2gvWazuOc1w7dzwTObKLcy4GHpp1UzHdsM8qLJh2WNNUMfiaqOBs11cp/7mXc
 CDCNvbCFWAuMMl5sw4lrrxuTVozmQmLbmLtx4io246yHzoFD+GCXuFNM1SQc0jpSTwhg
 FUOKjCNbmHARTs+OiHF6o/KfN3NXy18B4mzpks1lP4Kp9ZwzyuPsmmrQvp69m0kHJgSt
 AjSow15RKKvb2aCnNLZCiDSU6ZEg1q9DkGJiQlPfg+nbjfHHUw26f2vzSfKdFO6BB7wW
 mhYCe9jin2iml5sHxSUYJvHOzVWK2/R9QTdYIYTEl3gBARCHvn288Z2nyqhxtXEiufnC
 VPUg==
X-Gm-Message-State: AOAM533ZyAoCUtFiITmeGMSwapeaNhuc4uq7XYMCP6FMIsd3++Kn4kxI
 cSfjP8frFzFJLeZ+WxUS5mnc+d4aDtc=
X-Google-Smtp-Source: 
 ABdhPJyVa4YT7DYFsKi7EiXud3tgJynMa0nYp1hVbgMEdzJyBldzc6ytN5l9jfHbZ64bfZjSJKy4TQ==
X-Received: by 2002:a17:906:d183:: with SMTP id
 c3mr3405355ejz.283.1630665329512;
 Fri, 03 Sep 2021 03:35:29 -0700 (PDT)
Received: from localhost.localdomain ([85.187.18.127])
 by smtp.gmail.com with ESMTPSA id d22sm2609897ejj.47.2021.09.03.03.35.28
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Fri, 03 Sep 2021 03:35:29 -0700 (PDT)
From: Vladislav Odintsov 
To: dev@openvswitch.org
Date: Fri,  3 Sep 2021 13:34:40 +0300
Message-Id: <20210903103441.33178-6-odivlad@gmail.com>
X-Mailer: git-send-email 2.30.0
In-Reply-To: <20210903103441.33178-1-odivlad@gmail.com>
References: <20210903103441.33178-1-odivlad@gmail.com>
MIME-Version: 1.0
Cc: Vladislav Odintsov 
Subject: [ovs-dev] [PATCH ovn v2 5/6] ic: add support for routing tables in
	adv/learn routes
X-BeenThere: ovs-dev@openvswitch.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: 
List-Unsubscribe: ,
 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: ,
 
Errors-To: ovs-dev-bounces@openvswitch.org
Sender: "dev" 

Previously support for multiple routing tables was added
to northd code.
This commit expands support for multiple routing tables
by adding support of advertising and learning routes with
their routing table information.

To utilize such feature, user must:
1. create Logical Router in each AZ;
2. create IC transit switch for each routing table, that
   he/she needs;
3. connect each TS with this LR;
4. assign routing table for TS's LRP
   (ovn-nbctl lrp-set-options  route_table=<>);
5. enable routes sync (turn on learning and advertising
   routes in NB_Global table);
6. create LRPs for subnets in LR, create static routes
   with supplying route_table parameter.

Note 1: routes for directly-connected networks will be
learned to global routing table and if Logical Routers
have more than one Transit Switch, which interconnects
them, directly-connected routes will be added via each
transit switch port and configured as ECMP routes.

Note 2: static routes within route tables will be advertised
and learned only if interconnecting transit switch's LRPs
will have options:route_table same value as route's route_table
value.

Signed-off-by: Vladislav Odintsov 
---
 NEWS                |   4 +
 ic/ovn-ic.c         | 535 ++++++++++++++++++++++++++++----------------
 ovn-ic-sb.ovsschema |   5 +-
 ovn-ic-sb.xml       |  18 ++
 tests/ovn-ic.at     | 440 ++++++++++++++++++++++++++++++++++++
 5 files changed, 808 insertions(+), 194 deletions(-)

diff --git a/NEWS b/NEWS
index 6e81f7bee..3780947bd 100644
--- a/NEWS
+++ b/NEWS
@@ -9,6 +9,10 @@ Post-v21.06.0
   - Allow static routes without nexthops.
   - Enabled logical dp groups as a default.  CMS should disable it if not
     desired.
+  - Added support for multiple routing tables in Logical Router Static Routes
+    and LRPs. OVN Interconnection supports routes' route tables as well.
+    This requires to update schemas for OVN_Northdbound and OVN_IC_Southbound
+    DBs.
 
 OVN v21.06.0 - 18 Jun 2021
 -------------------------
diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c
index 980000d3e..37c1d4428 100644
--- a/ic/ovn-ic.c
+++ b/ic/ovn-ic.c
@@ -63,9 +63,11 @@ struct ic_context {
     struct ovsdb_idl_txn *ovninb_txn;
     struct ovsdb_idl_txn *ovnisb_txn;
     struct ovsdb_idl_index *nbrec_ls_by_name;
+    struct ovsdb_idl_index *nbrec_lrp_by_name;
     struct ovsdb_idl_index *nbrec_port_by_name;
     struct ovsdb_idl_index *sbrec_chassis_by_name;
     struct ovsdb_idl_index *sbrec_port_binding_by_name;
+    struct ovsdb_idl_index *icnbrec_transit_switch_by_name;
     struct ovsdb_idl_index *icsbrec_port_binding_by_az;
     struct ovsdb_idl_index *icsbrec_port_binding_by_ts;
     struct ovsdb_idl_index *icsbrec_port_binding_by_ts_az;
@@ -785,7 +787,7 @@ port_binding_run(struct ic_context *ctx,
         icsbrec_port_binding_index_set_transit_switch(isb_pb_key, ts->name);
 
         ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key,
-                                            ctx->icsbrec_port_binding_by_ts) {
+                                             ctx->icsbrec_port_binding_by_ts) {
             if (isb_pb->availability_zone == az) {
                 shash_add(&local_pbs, isb_pb->logical_port, isb_pb);
             } else {
@@ -849,7 +851,9 @@ port_binding_run(struct ic_context *ctx,
 struct ic_router_info {
     struct hmap_node node;
     const struct nbrec_logical_router *lr; /* key of hmap */
-    const struct icsbrec_port_binding *isb_pb;
+    const struct icsbrec_port_binding **isb_pbs;
+    size_t n_isb_pbs;
+    size_t n_allocated_isb_pbs;
     struct hmap routes_learned;
 };
 
@@ -859,6 +863,7 @@ struct ic_route_info {
     struct in6_addr prefix;
     unsigned int plen;
     struct in6_addr nexthop;
+    const char *route_table;
 
     /* Either nb_route or nb_lrp is set and the other one must be NULL.
      * - For a route that is learned from IC-SB, or a static route that is
@@ -880,13 +885,15 @@ ic_route_hash(const struct in6_addr *prefix, unsigned int plen,
 
 static struct ic_route_info *
 ic_route_find(struct hmap *routes, const struct in6_addr *prefix,
-              unsigned int plen, const struct in6_addr *nexthop)
+              unsigned int plen, const struct in6_addr *nexthop,
+              char *route_table)
 {
     struct ic_route_info *r;
     uint32_t hash = ic_route_hash(prefix, plen, nexthop);
     HMAP_FOR_EACH_WITH_HASH (r, node, hash, routes) {
         if (ipv6_addr_equals(&r->prefix, prefix) &&
             r->plen == plen &&
+            !strcmp(r->route_table ? r->route_table : "", route_table) &&
             ipv6_addr_equals(&r->nexthop, nexthop)) {
             return r;
         }
@@ -931,11 +938,19 @@ add_to_routes_learned(struct hmap *routes_learned,
                      &prefix, &plen, &nexthop)) {
         return false;
     }
+
+    if (ic_route_find(routes_learned, &prefix, plen, &nexthop,
+                      nb_route->route_table)) {
+        /* Route is already added to learned in previous iteration. */
+        return true;
+    }
+
     struct ic_route_info *ic_route = xzalloc(sizeof *ic_route);
     ic_route->prefix = prefix;
     ic_route->plen = plen;
     ic_route->nexthop = nexthop;
     ic_route->nb_route = nb_route;
+    ic_route->route_table = nb_route->route_table;
     hmap_insert(routes_learned, &ic_route->node,
                 ic_route_hash(&prefix, plen, &nexthop));
     return true;
@@ -1074,8 +1089,17 @@ static void
 add_to_routes_ad(struct hmap *routes_ad,
                  const struct nbrec_logical_router_static_route *nb_route,
                  const struct lport_addresses *nexthop_addresses,
-                 const struct smap *nb_options)
+                 const struct smap *nb_options, const char *route_table)
 {
+    if (strcmp(route_table, nb_route->route_table)) {
+        if (VLOG_IS_DBG_ENABLED()) {
+            VLOG_DBG("Skip advertising route %s -> %s as its route table %s !="
+                     " %s of TS port", nb_route->ip_prefix, nb_route->nexthop,
+                     nb_route->route_table, route_table);
+        }
+        return;
+    }
+
     struct in6_addr prefix, nexthop;
     unsigned int plen;
     if (!parse_route(nb_route->ip_prefix, nb_route->nexthop,
@@ -1093,11 +1117,33 @@ add_to_routes_ad(struct hmap *routes_ad,
         return;
     }
 
+    if (VLOG_IS_DBG_ENABLED()) {
+        struct ds msg = DS_EMPTY_INITIALIZER;
+
+        ds_put_format(&msg, "Advertising static route: %s -> %s, ic nexthop: ",
+                      nb_route->ip_prefix, nb_route->nexthop);
+
+        if (IN6_IS_ADDR_V4MAPPED(&nexthop)) {
+            ds_put_format(&msg, IP_FMT,
+                          IP_ARGS(in6_addr_get_mapped_ipv4(&nexthop)));
+        } else {
+            ipv6_format_addr(&nexthop, &msg);
+        }
+
+        ds_put_format(&msg, ", route_table: %s", strlen(nb_route->route_table)
+                                                 ? nb_route->route_table
+                                                 : "global");
+
+        VLOG_DBG("%s", ds_cstr(&msg));
+        ds_destroy(&msg);
+    }
+
     struct ic_route_info *ic_route = xzalloc(sizeof *ic_route);
     ic_route->prefix = prefix;
     ic_route->plen = plen;
     ic_route->nexthop = nexthop;
     ic_route->nb_route = nb_route;
+    ic_route->route_table = nb_route->route_table;
     hmap_insert(routes_ad, &ic_route->node,
                 ic_route_hash(&prefix, plen, &nexthop));
 }
@@ -1129,8 +1175,8 @@ add_network_to_routes_ad(struct hmap *routes_ad, const char *network,
     if (VLOG_IS_DBG_ENABLED()) {
         struct ds msg = DS_EMPTY_INITIALIZER;
 
-        ds_put_format(&msg, "Route ad: direct network %s of lrp %s, nexthop ",
-                      network, nb_lrp->name);
+        ds_put_format(&msg, "Adding direct network route to global routing "
+                      "table: %s of lrp %s, nexthop ", network, nb_lrp->name);
 
         if (IN6_IS_ADDR_V4MAPPED(&nexthop)) {
             ds_put_format(&msg, IP_FMT,
@@ -1148,13 +1194,15 @@ add_network_to_routes_ad(struct hmap *routes_ad, const char *network,
     ic_route->plen = plen;
     ic_route->nexthop = nexthop;
     ic_route->nb_lrp = nb_lrp;
+
+    /* directly-connected routes go to global route table */
+    ic_route->route_table = NULL;
     hmap_insert(routes_ad, &ic_route->node,
                 ic_route_hash(&prefix, plen, &nexthop));
 }
 
 static bool
-route_need_learn(struct in6_addr *prefix,
-                 unsigned int plen,
+route_need_learn(struct in6_addr *prefix, unsigned int plen,
                  const struct smap *nb_options)
 {
     if (!smap_get_bool(nb_options, "ic-route-learn", false)) {
@@ -1177,70 +1225,146 @@ route_need_learn(struct in6_addr *prefix,
     return true;
 }
 
+static const char *
+get_lrp_name_by_ts_port_name(struct ic_context *ctx, const char *ts_port_name)
+{
+    const struct nbrec_logical_switch_port *nb_lsp;
+    const struct nbrec_logical_switch_port *nb_lsp_key =
+        nbrec_logical_switch_port_index_init_row(ctx->nbrec_port_by_name);
+    nbrec_logical_switch_port_index_set_name(nb_lsp_key, ts_port_name);
+    nb_lsp = nbrec_logical_switch_port_index_find(ctx->nbrec_port_by_name,
+                                                  nb_lsp_key);
+    nbrec_logical_switch_port_index_destroy_row(nb_lsp_key);
+
+    if (!nb_lsp) {
+        return NULL;
+    }
+
+    return smap_get(&nb_lsp->options, "router-port");
+}
+
+static const char *
+get_route_table_by_lrp_name(struct ic_context *ctx, const char *lrp_name)
+{
+    const struct nbrec_logical_router_port *lrp;
+    const struct nbrec_logical_router_port *lrp_key =
+        nbrec_logical_router_port_index_init_row(ctx->nbrec_lrp_by_name);
+    nbrec_logical_router_port_index_set_name(lrp_key, lrp_name);
+    lrp = nbrec_logical_router_port_index_find(ctx->nbrec_lrp_by_name,
+                                               lrp_key);
+    nbrec_logical_router_port_index_destroy_row(lrp_key);
+
+    if (lrp) {
+        return smap_get_def(&lrp->options, "route_table", "");
+    }
+    return "";  /* Global route table */
+}
+
+static bool
+lrp_is_ts_port(struct ic_context *ctx, struct ic_router_info *ic_lr,
+               const char *lrp_name)
+{
+    const struct icsbrec_port_binding *isb_pb;
+    const char *ts_lrp_name;
+    for (int i = 0; i < ic_lr->n_isb_pbs; i++) {
+        isb_pb = ic_lr->isb_pbs[i];
+        ts_lrp_name = get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port);
+        if (!strcmp(ts_lrp_name, lrp_name)) {
+            return true;
+        }
+    }
+    return false;
+}
+
 static void
-sync_learned_route(struct ic_context *ctx,
-                   const struct icsbrec_availability_zone *az,
-                   struct ic_router_info *ic_lr)
+sync_learned_routes(struct ic_context *ctx,
+                    const struct icsbrec_availability_zone *az,
+                    struct ic_router_info *ic_lr)
 {
     ovs_assert(ctx->ovnnb_txn);
     const struct icsbrec_route *isb_route;
     const struct icsbrec_route *isb_route_key =
         icsbrec_route_index_init_row(ctx->icsbrec_route_by_ts);
 
-    icsbrec_route_index_set_transit_switch(isb_route_key,
-                                           ic_lr->isb_pb->transit_switch);
+    const struct nbrec_nb_global *nb_global =
+        nbrec_nb_global_first(ctx->ovnnb_idl);
+    ovs_assert(nb_global);
 
-    ICSBREC_ROUTE_FOR_EACH_EQUAL (isb_route, isb_route_key,
-                                  ctx->icsbrec_route_by_ts) {
-        if (isb_route->availability_zone == az) {
-            continue;
-        }
-        struct in6_addr prefix, nexthop;
-        unsigned int plen;
-        if (!parse_route(isb_route->ip_prefix, isb_route->nexthop,
-                         &prefix, &plen, &nexthop)) {
-            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
-            VLOG_WARN_RL(&rl, "Bad route format in IC-SB: %s -> %s. Ignored.",
-                         isb_route->ip_prefix, isb_route->nexthop);
-            continue;
-        }
-        const struct nbrec_nb_global *nb_global =
-            nbrec_nb_global_first(ctx->ovnnb_idl);
-        ovs_assert(nb_global);
-        if (!route_need_learn(&prefix, plen, &nb_global->options)) {
-            continue;
-        }
-        struct ic_route_info *route_learned
-            = ic_route_find(&ic_lr->routes_learned, &prefix, plen, &nexthop);
-        if (route_learned) {
-            /* Sync external-ids */
-            struct uuid ext_id;
-            smap_get_uuid(&route_learned->nb_route->external_ids,
-                          "ic-learned-route", &ext_id);
-            if (!uuid_equals(&ext_id, &isb_route->header_.uuid)) {
+    const char *lrp_name, *ts_route_table;
+    const struct icsbrec_port_binding *isb_pb;
+    for (int i = 0; i < ic_lr->n_isb_pbs; i++) {
+        isb_pb = ic_lr->isb_pbs[i];
+        lrp_name = get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port);
+        ts_route_table = get_route_table_by_lrp_name(ctx, lrp_name);
+
+        icsbrec_route_index_set_transit_switch(isb_route_key,
+                                               isb_pb->transit_switch);
+
+        ICSBREC_ROUTE_FOR_EACH_EQUAL (isb_route, isb_route_key,
+                                      ctx->icsbrec_route_by_ts) {
+            if (isb_route->availability_zone == az) {
+                continue;
+            }
+
+            if (strlen(isb_route->route_table) &&
+                strcmp(isb_route->route_table, ts_route_table)) {
+                if (VLOG_IS_DBG_ENABLED()) {
+                    VLOG_DBG("Skip learning static route %s -> %s as either "
+                             "its route table %s != %s of TS port or ",
+                             isb_route->ip_prefix, isb_route->nexthop,
+                             isb_route->route_table, ts_route_table);
+                }
+                continue;
+            }
+
+            struct in6_addr prefix, nexthop;
+            unsigned int plen;
+            if (!parse_route(isb_route->ip_prefix, isb_route->nexthop,
+                             &prefix, &plen, &nexthop)) {
+                static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
+                VLOG_WARN_RL(&rl, "Bad route format in IC-SB: %s -> %s. Ignored.",
+                             isb_route->ip_prefix, isb_route->nexthop);
+                continue;
+            }
+            if (!route_need_learn(&prefix, plen, &nb_global->options)) {
+                continue;
+            }
+            struct ic_route_info *route_learned
+                = ic_route_find(&ic_lr->routes_learned, &prefix, plen,
+                                &nexthop, isb_route->route_table);
+            if (route_learned) {
+                /* Sync external-ids */
+                struct uuid ext_id;
+                smap_get_uuid(&route_learned->nb_route->external_ids,
+                              "ic-learned-route", &ext_id);
+                if (!uuid_equals(&ext_id, &isb_route->header_.uuid)) {
+                    char *uuid_s =
+                        xasprintf(UUID_FMT,
+                                  UUID_ARGS(&isb_route->header_.uuid));
+                    nbrec_logical_router_static_route_update_external_ids_setkey(
+                        route_learned->nb_route, "ic-learned-route", uuid_s);
+                    free(uuid_s);
+                }
+                hmap_remove(&ic_lr->routes_learned, &route_learned->node);
+                free(route_learned);
+            } else {
+                /* Create the missing route in NB. */
+                const struct nbrec_logical_router_static_route *nb_route =
+                    nbrec_logical_router_static_route_insert(ctx->ovnnb_txn);
+                nbrec_logical_router_static_route_set_ip_prefix(nb_route,
+                    isb_route->ip_prefix);
+                nbrec_logical_router_static_route_set_nexthop(nb_route,
+                    isb_route->nexthop);
                 char *uuid_s = xasprintf(UUID_FMT,
                                          UUID_ARGS(&isb_route->header_.uuid));
+                nbrec_logical_router_static_route_set_route_table(nb_route,
+                    isb_route->route_table);
                 nbrec_logical_router_static_route_update_external_ids_setkey(
-                    route_learned->nb_route, "ic-learned-route", uuid_s);
+                    nb_route, "ic-learned-route", uuid_s);
                 free(uuid_s);
+                nbrec_logical_router_update_static_routes_addvalue(ic_lr->lr,
+                    nb_route);
             }
-            hmap_remove(&ic_lr->routes_learned, &route_learned->node);
-            free(route_learned);
-        } else {
-            /* Create the missing route in NB. */
-            const struct nbrec_logical_router_static_route *nb_route =
-                nbrec_logical_router_static_route_insert(ctx->ovnnb_txn);
-            nbrec_logical_router_static_route_set_ip_prefix(
-                nb_route, isb_route->ip_prefix);
-            nbrec_logical_router_static_route_set_nexthop(
-                nb_route, isb_route->nexthop);
-            char *uuid_s = xasprintf(UUID_FMT,
-                                     UUID_ARGS(&isb_route->header_.uuid));
-            nbrec_logical_router_static_route_update_external_ids_setkey(
-                nb_route, "ic-learned-route", uuid_s);
-            free(uuid_s);
-            nbrec_logical_router_update_static_routes_addvalue(
-                ic_lr->lr, nb_route);
         }
     }
     icsbrec_route_index_destroy_row(isb_route_key);
@@ -1276,10 +1400,10 @@ ad_route_sync_external_ids(const struct ic_route_info *route_adv,
 
 /* Sync routes from routes_ad to IC-SB. */
 static void
-advertise_route(struct ic_context *ctx,
-                const struct icsbrec_availability_zone *az,
-                const char *ts_name,
-                struct hmap *routes_ad)
+advertise_routes(struct ic_context *ctx,
+                 const struct icsbrec_availability_zone *az,
+                 const char *ts_name,
+                 struct hmap *routes_ad)
 {
     ovs_assert(ctx->ovnisb_txn);
     const struct icsbrec_route *isb_route;
@@ -1303,7 +1427,8 @@ advertise_route(struct ic_context *ctx,
             continue;
         }
         struct ic_route_info *route_adv =
-            ic_route_find(routes_ad, &prefix, plen, &nexthop);
+            ic_route_find(routes_ad, &prefix, plen, &nexthop,
+                          isb_route->route_table);
         if (!route_adv) {
             /* Delete the extra route from IC-SB. */
             VLOG_DBG("Delete route %s -> %s from IC-SB, which is not found"
@@ -1343,6 +1468,9 @@ advertise_route(struct ic_context *ctx,
         }
         icsbrec_route_set_ip_prefix(isb_route, prefix_s);
         icsbrec_route_set_nexthop(isb_route, nexthop_s);
+        icsbrec_route_set_route_table(isb_route, route_adv->route_table
+                                                 ? route_adv->route_table
+                                                 : "");
         free(prefix_s);
         free(nexthop_s);
 
@@ -1353,23 +1481,97 @@ advertise_route(struct ic_context *ctx,
     }
 }
 
-static const char *
-get_lrp_name_by_ts_port_name(struct ic_context *ctx,
-                           const char *ts_port_name)
+static void
+build_ts_routes_to_adv(struct ic_context *ctx,
+                       struct ic_router_info *ic_lr,
+                       struct hmap *routes_ad,
+                       struct lport_addresses *ts_port_addrs,
+                       const struct nbrec_nb_global *nb_global,
+                       const char *ts_route_table)
 {
-    const struct nbrec_logical_switch_port *nb_lsp;
-    const struct nbrec_logical_switch_port *nb_lsp_key =
-        nbrec_logical_switch_port_index_init_row(ctx->nbrec_port_by_name);
-    nbrec_logical_switch_port_index_set_name(nb_lsp_key, ts_port_name);
-    nb_lsp = nbrec_logical_switch_port_index_find(ctx->nbrec_port_by_name,
-                                                  nb_lsp_key);
-    nbrec_logical_switch_port_index_destroy_row(nb_lsp_key);
+    const struct nbrec_logical_router *lr = ic_lr->lr;
+
+    /* Check static routes of the LR */
+    for (int i = 0; i < lr->n_static_routes; i++) {
+        const struct nbrec_logical_router_static_route *nb_route
+            = lr->static_routes[i];
+        struct uuid isb_uuid;
+        if (smap_get_uuid(&nb_route->external_ids, "ic-learned-route",
+                          &isb_uuid)) {
+            /* It is a learned route */
+            if (!add_to_routes_learned(&ic_lr->routes_learned, nb_route)) {
+                static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
+                VLOG_WARN_RL(&rl, "Bad format of learned route in NB: "
+                             "%s -> %s. Delete it.", nb_route->ip_prefix,
+                             nb_route->nexthop);
+                nbrec_logical_router_update_static_routes_delvalue(lr,
+                    nb_route);
+            }
+        } else {
+            /* It may be a route to be advertised */
+            add_to_routes_ad(routes_ad, nb_route, ts_port_addrs,
+                             &nb_global->options, ts_route_table);
+        }
+    }
 
-    if (!nb_lsp) {
-        return NULL;
+    /* Check directly-connected subnets of the LR */
+    for (int i = 0; i < lr->n_ports; i++) {
+        const struct nbrec_logical_router_port *lrp = lr->ports[i];
+        if (!lrp_is_ts_port(ctx, ic_lr, lrp->name)) {
+            for (int j = 0; j < lrp->n_networks; j++) {
+                add_network_to_routes_ad(routes_ad, lrp->networks[j], lrp,
+                                         ts_port_addrs,
+                                         &nb_global->options);
+            }
+        } else {
+            /* The router port of the TS port is ignored. */
+            VLOG_DBG("Skip advertising direct route of lrp %s (TS port)",
+                     lrp->name);
+        }
     }
+}
 
-    return smap_get(&nb_lsp->options, "router-port");
+static void
+advertise_lr_routes(struct ic_context *ctx,
+                    const struct icsbrec_availability_zone *az,
+                    struct ic_router_info *ic_lr)
+{
+    const struct nbrec_nb_global *nb_global =
+        nbrec_nb_global_first(ctx->ovnnb_idl);
+    ovs_assert(nb_global);
+
+    const struct icsbrec_port_binding *isb_pb;
+    const char *lrp_name, *route_table;
+    struct lport_addresses ts_port_addrs;
+    const struct nbrec_logical_router *lr = ic_lr->lr;
+    const struct icnbrec_transit_switch *ts, *key =
+        icnbrec_transit_switch_index_init_row(
+            ctx->icnbrec_transit_switch_by_name);
+
+    struct hmap routes_ad = HMAP_INITIALIZER(&routes_ad);
+    for (int i = 0; i < ic_lr->n_isb_pbs; i++) {
+        isb_pb = ic_lr->isb_pbs[i];
+        icnbrec_transit_switch_index_set_name(key, isb_pb->transit_switch);
+        ts = icnbrec_transit_switch_index_find(
+            ctx->icnbrec_transit_switch_by_name, key);
+
+        if (!extract_lsp_addresses(isb_pb->address, &ts_port_addrs)) {
+            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
+            VLOG_INFO_RL(&rl, "Route sync ignores port %s on ts %s for router"
+                         " %s because the addresses are invalid.",
+                         isb_pb->logical_port, isb_pb->transit_switch,
+                         lr->name);
+            continue;
+        }
+        lrp_name = get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port);
+        route_table = get_route_table_by_lrp_name(ctx, lrp_name);
+        build_ts_routes_to_adv(ctx, ic_lr, &routes_ad, &ts_port_addrs,
+                               nb_global, route_table);
+        advertise_routes(ctx, az, ts->name, &routes_ad);
+        destroy_lport_addresses(&ts_port_addrs);
+    }
+    hmap_destroy(&routes_ad);
+    icnbrec_transit_switch_index_destroy_row(key);
 }
 
 static void
@@ -1380,131 +1582,70 @@ route_run(struct ic_context *ctx,
         return;
     }
 
-    const struct nbrec_nb_global *nb_global =
-        nbrec_nb_global_first(ctx->ovnnb_idl);
-    ovs_assert(nb_global);
-
-    const struct icnbrec_transit_switch *ts;
-    ICNBREC_TRANSIT_SWITCH_FOR_EACH (ts, ctx->ovninb_idl) {
-        struct hmap ic_lrs = HMAP_INITIALIZER(&ic_lrs);
-        struct hmap routes_ad = HMAP_INITIALIZER(&routes_ad);
-
-        const struct icsbrec_port_binding *isb_pb;
-        const struct icsbrec_port_binding *isb_pb_key =
-            icsbrec_port_binding_index_init_row(
-                ctx->icsbrec_port_binding_by_ts_az);
-        icsbrec_port_binding_index_set_transit_switch(isb_pb_key, ts->name);
-        icsbrec_port_binding_index_set_availability_zone(isb_pb_key, az);
-
-        /* Each port on TS maps to a logical router, which is stored in the
-         * external_ids:router-id of the IC SB port_binding record. */
-        ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key,
-                                             ctx->icsbrec_port_binding_by_ts_az)
-        {
-            const char *ts_lrp_name =
-                get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port);
-            if (!ts_lrp_name) {
-                static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
-                VLOG_WARN_RL(&rl, "Route sync ignores port %s on ts %s "
-                             "because logical router port is not found in NB.",
-                             isb_pb->logical_port, ts->name);
-                continue;
-            }
-
-            struct uuid lr_uuid;
-            if (!smap_get_uuid(&isb_pb->external_ids, "router-id", &lr_uuid)) {
-                VLOG_DBG("IC-SB Port_Binding %s doesn't have "
-                         "external_ids:router-id set.", isb_pb->logical_port);
-                continue;
-            }
-            const struct nbrec_logical_router *lr
-                = nbrec_logical_router_get_for_uuid(ctx->ovnnb_idl, &lr_uuid);
-            if (!lr) {
-                continue;
-            }
+    struct hmap ic_lrs = HMAP_INITIALIZER(&ic_lrs);
+    const struct icsbrec_port_binding *isb_pb;
+    const struct icsbrec_port_binding *isb_pb_key =
+        icsbrec_port_binding_index_init_row(ctx->icsbrec_port_binding_by_az);
+    icsbrec_port_binding_index_set_availability_zone(isb_pb_key, az);
+
+    /* Each port on TS maps to a logical router, which is stored in the
+     * external_ids:router-id of the IC SB port_binding record.
+     * Here we build info for interconnected Logical Router:
+     * collect IC Port Binding to process routes sync later on. */
+    ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key,
+                                         ctx->icsbrec_port_binding_by_az)
+    {
+        const char *ts_lrp_name =
+            get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port);
+        if (!ts_lrp_name) {
+            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
+            VLOG_WARN_RL(&rl, "Route sync ignores port %s on ts %s because "
+                         "logical router port is not found in NB. Deleting it",
+                         isb_pb->logical_port, isb_pb->transit_switch);
+            icsbrec_port_binding_delete(isb_pb);
+            continue;
+        }
 
-            if (ic_router_find(&ic_lrs, lr)) {
-                static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
-                VLOG_INFO_RL(&rl, "Route sync ignores port %s on ts %s for "
-                             "router %s because the router has another port "
-                             "connected to same ts.", isb_pb->logical_port,
-                             ts->name, lr->name);
-                continue;
-            }
+        struct uuid lr_uuid;
+        if (!smap_get_uuid(&isb_pb->external_ids, "router-id", &lr_uuid)) {
+            VLOG_DBG("IC-SB Port_Binding %s doesn't have "
+                     "external_ids:router-id set.", isb_pb->logical_port);
+            continue;
+        }
 
-            struct lport_addresses ts_port_addrs;
-            if (!extract_lsp_addresses(isb_pb->address, &ts_port_addrs)) {
-                static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
-                VLOG_INFO_RL(&rl, "Route sync ignores port %s on ts %s for "
-                             "router %s because the addresses are invalid.",
-                             isb_pb->logical_port, ts->name, lr->name);
-                continue;
-            }
+        const struct nbrec_logical_router *lr
+            = nbrec_logical_router_get_for_uuid(ctx->ovnnb_idl, &lr_uuid);
+        if (!lr) {
+            continue;
+        }
 
-            struct ic_router_info *ic_lr = xzalloc(sizeof *ic_lr);
+        struct ic_router_info *ic_lr = ic_router_find(&ic_lrs, lr);
+        if (!ic_lr) {
+            ic_lr = xzalloc(sizeof *ic_lr);
             ic_lr->lr = lr;
-            ic_lr->isb_pb = isb_pb;
             hmap_init(&ic_lr->routes_learned);
             hmap_insert(&ic_lrs, &ic_lr->node, uuid_hash(&lr->header_.uuid));
-
-            /* Check static routes of the LR */
-            for (int i = 0; i < lr->n_static_routes; i++) {
-                const struct nbrec_logical_router_static_route *nb_route
-                    = lr->static_routes[i];
-                struct uuid isb_uuid;
-                if (smap_get_uuid(&nb_route->external_ids,
-                                  "ic-learned-route", &isb_uuid)) {
-                    /* It is a learned route */
-                    if (!add_to_routes_learned(&ic_lr->routes_learned,
-                                               nb_route)) {
-                        static struct vlog_rate_limit rl =
-                            VLOG_RATE_LIMIT_INIT(5, 1);
-                        VLOG_WARN_RL(&rl, "Bad format of learned route in NB:"
-                                     " %s -> %s. Delete it.",
-                                     nb_route->ip_prefix, nb_route->nexthop);
-                        nbrec_logical_router_update_static_routes_delvalue(
-                            lr, nb_route);
-                    }
-                } else {
-                    /* It may be a route to be advertised */
-                    add_to_routes_ad(&routes_ad, nb_route, &ts_port_addrs,
-                                     &nb_global->options);
-                }
-            }
-
-            /* Check direct-connected subnets of the LR */
-            for (int i = 0; i < lr->n_ports; i++) {
-                const struct nbrec_logical_router_port *lrp = lr->ports[i];
-                if (!strcmp(lrp->name, ts_lrp_name)) {
-                    /* The router port of the TS port is ignored. */
-                    VLOG_DBG("Route ad: skip lrp %s (TS port: %s)",
-                             lrp->name, isb_pb->logical_port);
-                    continue;
-                }
-
-                for (int j = 0; j < lrp->n_networks; j++) {
-                    add_network_to_routes_ad(&routes_ad, lrp->networks[j],
-                                             lrp, &ts_port_addrs,
-                                             &nb_global->options);
-                }
-            }
-
-            destroy_lport_addresses(&ts_port_addrs);
         }
-        icsbrec_port_binding_index_destroy_row(isb_pb_key);
 
-        advertise_route(ctx, az, ts->name, &routes_ad);
-        hmap_destroy(&routes_ad);
-
-        struct ic_router_info *ic_lr, *next;
-        HMAP_FOR_EACH_SAFE (ic_lr, next, node, &ic_lrs) {
-            sync_learned_route(ctx, az, ic_lr);
-            hmap_destroy(&ic_lr->routes_learned);
-            hmap_remove(&ic_lrs, &ic_lr->node);
-            free(ic_lr);
+        if (ic_lr->n_isb_pbs == ic_lr->n_allocated_isb_pbs) {
+            ic_lr->isb_pbs = x2nrealloc(ic_lr->isb_pbs,
+                                        &ic_lr->n_allocated_isb_pbs,
+                                        sizeof *ic_lr->isb_pbs);
         }
-        hmap_destroy(&ic_lrs);
+        ic_lr->isb_pbs[ic_lr->n_isb_pbs++] = isb_pb;
+    }
+    icsbrec_port_binding_index_destroy_row(isb_pb_key);
+
+    struct ic_router_info *ic_lr, *next;
+    HMAP_FOR_EACH_SAFE (ic_lr, next, node, &ic_lrs) {
+        advertise_lr_routes(ctx, az, ic_lr);
+        sync_learned_routes(ctx, az, ic_lr);
+        free(ic_lr->isb_pbs);
+        hmap_destroy(&ic_lr->routes_learned);
+        hmap_remove(&ic_lrs, &ic_lr->node);
+        free(ic_lr);
     }
+    hmap_destroy(&ic_lrs);
 }
 
 static void
@@ -1702,6 +1843,9 @@ main(int argc, char *argv[])
     struct ovsdb_idl_index *nbrec_port_by_name
         = ovsdb_idl_index_create1(ovnnb_idl_loop.idl,
                                   &nbrec_logical_switch_port_col_name);
+    struct ovsdb_idl_index *nbrec_lrp_by_name
+        = ovsdb_idl_index_create1(ovnnb_idl_loop.idl,
+                                  &nbrec_logical_router_port_col_name);
     struct ovsdb_idl_index *sbrec_port_binding_by_name
         = ovsdb_idl_index_create1(ovnsb_idl_loop.idl,
                                   &sbrec_port_binding_col_logical_port);
@@ -1709,6 +1853,10 @@ main(int argc, char *argv[])
         = ovsdb_idl_index_create1(ovnsb_idl_loop.idl,
                                   &sbrec_chassis_col_name);
 
+    struct ovsdb_idl_index *icnbrec_transit_switch_by_name
+        = ovsdb_idl_index_create1(ovninb_idl_loop.idl,
+                                  &icnbrec_transit_switch_col_name);
+
     struct ovsdb_idl_index *icsbrec_port_binding_by_az
         = ovsdb_idl_index_create1(ovnisb_idl_loop.idl,
                                   &icsbrec_port_binding_col_availability_zone);
@@ -1767,9 +1915,12 @@ main(int argc, char *argv[])
                 .ovnisb_idl = ovnisb_idl_loop.idl,
                 .ovnisb_txn = ovsdb_idl_loop_run(&ovnisb_idl_loop),
                 .nbrec_ls_by_name = nbrec_ls_by_name,
+                .nbrec_lrp_by_name = nbrec_lrp_by_name,
                 .nbrec_port_by_name = nbrec_port_by_name,
                 .sbrec_port_binding_by_name = sbrec_port_binding_by_name,
                 .sbrec_chassis_by_name = sbrec_chassis_by_name,
+                .icnbrec_transit_switch_by_name =
+                    icnbrec_transit_switch_by_name,
                 .icsbrec_port_binding_by_az = icsbrec_port_binding_by_az,
                 .icsbrec_port_binding_by_ts = icsbrec_port_binding_by_ts,
                 .icsbrec_port_binding_by_ts_az = icsbrec_port_binding_by_ts_az,
diff --git a/ovn-ic-sb.ovsschema b/ovn-ic-sb.ovsschema
index 5364b21b4..140ced149 100644
--- a/ovn-ic-sb.ovsschema
+++ b/ovn-ic-sb.ovsschema
@@ -1,7 +1,7 @@
 {
     "name": "OVN_IC_Southbound",
-    "version": "1.0.0",
-    "cksum": "108951192 6585",
+    "version": "1.1.0",
+    "cksum": "3076915724 6636",
     "tables": {
         "IC_SB_Global": {
             "columns": {
@@ -92,6 +92,7 @@
                 "transit_switch": {"type": "string"},
                 "availability_zone": {"type": {"key": {"type": "uuid",
                                       "refTable": "Availability_Zone"}}},
+                "route_table": {"type": "string"},
                 "ip_prefix": {"type": "string"},
                 "nexthop": {"type": "string"},
                 "external_ids": {
diff --git a/ovn-ic-sb.xml b/ovn-ic-sb.xml
index 3582cff47..2f2ecf952 100644
--- a/ovn-ic-sb.xml
+++ b/ovn-ic-sb.xml
@@ -306,6 +306,24 @@
         The availability zone that has advertised the route.
       
 
+      
+        Route table within which this route was created.
+        Empty value means "global" routing table.
+        
    
+        Routes for directly-connected networks will be
+        learned to global routing table and if Logical Routers
+        have more than one Transit Switch, which interconnects
+        them, directly-connected routes will be added via each
+        transit switch port and configured as ECMP routes.
+        
    
+        
    
+        Static routes within route tables will be advertised
+        and learned only if interconnecting transit switch's LRPs
+        will have options:route_table same value as
+        route's route_table value.
+        
    
+          
+
       
         IP prefix of this route (e.g. 192.168.100.0/24).
       
diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at
index 48ec1bf35..c03886120 100644
--- a/tests/ovn-ic.at
+++ b/tests/ovn-ic.at
@@ -430,3 +430,443 @@ OVN_CLEANUP_IC([az1], [az2])
 
 AT_CLEANUP
 ])
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([ovn-ic -- route sync -- route tables])
+
+ovn_init_ic_db
+ovn-ic-nbctl ts-add ts1
+
+for i in 1 2; do
+    ovn_start az$i
+    ovn_as az$i
+
+    # Enable route learning at AZ level
+    ovn-nbctl set nb_global . options:ic-route-learn=true
+    # Enable route advertising at AZ level
+    ovn-nbctl set nb_global . options:ic-route-adv=true
+
+    # Create LRP and connect to TS
+    ovn-nbctl lr-add lr$i
+    ovn-nbctl lrp-add lr$i lrp-lr$i-ts1 aa:aa:aa:aa:aa:0$i 169.254.100.$i/24
+    ovn-nbctl lsp-add ts1 lsp-ts1-lr$i \
+            -- lsp-set-addresses lsp-ts1-lr$i router \
+            -- lsp-set-type lsp-ts1-lr$i router \
+            -- lsp-set-options lsp-ts1-lr$i router-port=lrp-lr$i-ts1
+
+    # Create static routes
+    ovn-nbctl lr-route-add lr$i 10.11.$i.0/24 169.254.0.1
+
+    # Create a src-ip route, which shouldn't be synced
+    ovn-nbctl --policy=src-ip --route-table=rtb1 lr-route-add lr$i 10.22.$i.0/24 169.254.0.2
+done
+
+for i in 1 2; do
+    OVS_WAIT_UNTIL([ovn_as az$i ovn-nbctl lr-route-list lr$i | grep learned])
+done
+
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl
+IPv4 Routes
+Route Table global:
+             10.11.1.0/24               169.254.0.1 dst-ip
+             10.11.2.0/24             169.254.100.2 dst-ip (learned)
+
+Route Table rtb1:
+             10.22.1.0/24               169.254.0.2 src-ip
+])
+
+# move routes from global route table to rtb1
+for i in 1 2; do
+    ovn_as az$i ovn-nbctl lr-route-del lr$i 10.11.$i.0/24 169.254.0.1
+    ovn_as az$i ovn-nbctl --route-table=rtb1 lr-route-add lr$i 10.11.$i.0/24 169.254.0.1
+done
+
+for i in 1 2; do
+    OVS_WAIT_WHILE([ovn_as az$i ovn-nbctl lr-route-list lr$i | grep learned])
+done
+
+# ensure route from rtb1 is not learned to any route table as route table is
+# not set to TS port
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl
+IPv4 Routes
+Route Table rtb1:
+             10.11.1.0/24               169.254.0.1 dst-ip
+             10.22.1.0/24               169.254.0.2 src-ip
+])
+
+# assign route table rtb1 to TS port on AZ2 and check routes are advertised to IC SB DB
+check ovn_as az2 ovn-nbctl lrp-set-options lrp-lr2-ts1 route_table=rtb1
+OVS_WAIT_UNTIL([ovn-ic-sbctl find route route_table=rtb1 | grep 10.11.2.0/24])
+
+# ensure route was not learned as on AZ1 TS port's LRP was not set to route table rtb1
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl
+IPv4 Routes
+Route Table rtb1:
+             10.11.1.0/24               169.254.0.1 dst-ip
+             10.22.1.0/24               169.254.0.2 src-ip
+])
+
+# set TS port's LRP to route table rtb1 to learn routes from AZ2 from rtb1
+check ovn_as az1 ovn-nbctl lrp-set-options lrp-lr1-ts1 route_table=rtb1
+
+OVS_WAIT_UNTIL([ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-list lr1 | grep learned])
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl
+IPv4 Routes
+Route Table rtb1:
+             10.11.1.0/24               169.254.0.1 dst-ip
+             10.11.2.0/24             169.254.100.2 dst-ip (learned)
+             10.22.1.0/24               169.254.0.2 src-ip
+])
+
+# Delete route in AZ1, AZ2's learned route should be deleted.
+ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-del lr1 10.11.1.0/24
+OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-list lr2 | grep learned])
+
+# Add the route back
+ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-add lr1 10.11.1.0/24 169.254.0.1
+OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned])
+
+# Disable route-learning for AZ1
+ovn_as az1 ovn-nbctl set nb_global . options:ic-route-learn=false
+OVS_WAIT_WHILE([ovn_as az1 ovn-nbctl lr-route-list lr1 | grep learned])
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl
+IPv4 Routes
+Route Table rtb1:
+             10.11.1.0/24               169.254.0.1 dst-ip
+             10.22.1.0/24               169.254.0.2 src-ip
+])
+
+# AZ1 should still advertise and AZ2 should still learn the route
+AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned], [0], [ignore])
+AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl
+IPv4 Routes
+Route Table rtb1:
+             10.11.1.0/24             169.254.100.1 dst-ip (learned)
+             10.11.2.0/24               169.254.0.1 dst-ip
+             10.22.2.0/24               169.254.0.2 src-ip
+])
+
+# Disable route-advertising for AZ1
+ovn_as az1 ovn-nbctl set nb_global . options:ic-route-adv=false
+
+# AZ2 shouldn't have the route learned, because AZ1 have stopped advertising.
+OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned])
+
+# Add default route in AZ1
+ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-add lr1 0.0.0.0/0 169.254.0.3
+
+# Re-enable router-advertising & learn for AZ1
+ovn_as az1 ovn-nbctl set nb_global . options:ic-route-adv=true
+ovn_as az1 ovn-nbctl set nb_global . options:ic-route-learn=true
+
+for i in 1 2; do
+    OVS_WAIT_UNTIL([ovn_as az$i ovn-nbctl lr-route-list lr$i | grep learned])
+done
+
+# Default route should NOT get advertised or learned, by default.
+AT_CHECK([ovn-ic-sbctl find route ip_prefix="0.0.0.0/0"], [0], [])
+
+# Enable default route advertising in AZ1, ensure it advertised, but not learned
+ovn_as az1 ovn-nbctl set nb_global . options:ic-route-adv-default=true
+OVS_WAIT_UNTIL([ovn-ic-sbctl find route ip_prefix="0.0.0.0/0" route_table=rtb1 | grep 0.0.0.0])
+OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-list lr2 | grep learned | grep 0.0.0.0])
+
+# Enable default route learning in AZ2
+ovn_as az2 ovn-nbctl set nb_global . options:ic-route-learn-default=true
+OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-list lr2 | grep learned | grep 0.0.0.0])
+
+# Test directly connected subnet route advertising. Route should go to global route table.
+ovn_as az1 ovn-nbctl lrp-add lr1 lrp-lr1-ls1 aa:aa:aa:aa:bb:01 "192.168.0.1/24"
+OVS_WAIT_UNTIL([ovn-ic-sbctl find route ip_prefix="192.168.0.1/24" route_table="\"\"" | grep 192.168.0.1/24])
+OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned | grep 192.168])
+AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl
+IPv4 Routes
+Route Table global:
+           192.168.0.0/24             169.254.100.1 dst-ip (learned)
+
+Route Table rtb1:
+             10.11.1.0/24             169.254.100.1 dst-ip (learned)
+             10.11.2.0/24               169.254.0.1 dst-ip
+             10.22.2.0/24               169.254.0.2 src-ip
+                0.0.0.0/0             169.254.100.1 dst-ip (learned)
+])
+
+# Delete the directly connected subnet from AZ1, learned route should be
+# removed from AZ2.
+ovn_as az1 ovn-nbctl lrp-del lrp-lr1-ls1
+OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned | grep 192.168])
+
+# Test blacklist routes
+# Add back the directly connected 192.168 route.
+ovn_as az1 ovn-nbctl lrp-add lr1 lrp-lr1-ls1 aa:aa:aa:aa:bb:01 "192.168.0.1/24"
+OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned | grep 192.168])
+# Now add 10.11.0.0/16 and 192.168.0.0/16 to blacklist in AZ2.
+check ovn_as az2 ovn-nbctl set nb_global . options:ic-route-blacklist="10.11.0.0/16,192.168.0.0/16"
+# AZ2 shouldn't learn 192.168 route any more.
+OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned | grep 192.168])
+# AZ1 shouldn't learn 10.11 any more.
+OVS_WAIT_WHILE([ovn_as az1 ovn-nbctl lr-route-list lr1 | grep learned | grep 10.11])
+AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl
+IPv4 Routes
+Route Table rtb1:
+             10.11.2.0/24               169.254.0.1 dst-ip
+             10.22.2.0/24               169.254.0.2 src-ip
+                0.0.0.0/0             169.254.100.1 dst-ip (learned)
+])
+
+OVN_CLEANUP_IC([az1], [az2])
+
+AT_CLEANUP
+])
+
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([ovn-ic -- route sync -- multiple route tables])
+
+ovn_init_ic_db
+ovn-ic-nbctl ts-add ts1
+
+for i in 1 2; do
+    ovn_start az$i
+    ovn_as az$i
+
+    # Enable route learning at AZ level
+    ovn-nbctl set nb_global . options:ic-route-learn=true
+    # Enable route advertising at AZ level
+    ovn-nbctl set nb_global . options:ic-route-adv=true
+done
+
+# Create new transit switches and LRs. Test topology is next:
+# VPC1:
+#                       / transit switch (ts11) \
+# logical router (lr11) - transit switch (ts12) - logical router (lr12)
+#                       \ transit switch (ts13) /
+#
+# VPC2:
+#                       / transit switch (ts21) \
+# logical router (lr21)                           logical router (lr22)
+#                       \ transit switch (ts22) /
+#
+# each LR has one connected subnet except TS port
+
+
+# VPC1
+# create lr11, lr12, ts11, ts12, ts13 and connect them
+# assign route tables rtb1, rtb2, rtb3 to ts ports
+for i in 1 2; do
+    ovn_as az$i
+
+    lr=lr1$i
+    ovn-nbctl lr-add $lr
+
+    for j in 1 2 3; do
+        ts=ts1$j
+        ovn-ic-nbctl --may-exist ts-add $ts
+
+        lrp=lrp-$lr-$ts
+        lsp=lsp-$ts-$lr
+        # Create LRP and connect to TS
+        ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:a$j:0$i 169.254.10$j.$i/24
+        ovn-nbctl lrp-set-options $lrp route_table=rtb$j
+        ovn-nbctl lsp-add $ts $lsp \
+                -- lsp-set-addresses $lsp router \
+                -- lsp-set-type $lsp router \
+                -- lsp-set-options $lsp router-port=$lrp
+    done
+done
+
+# VPC2
+# create lr21, lr22, ts21, ts22 and connect them
+# assign route tables rtb1, rtb2, rtb3 to ts ports
+for i in 1 2; do
+    ovn_as az$i
+
+    lr=lr2$i
+    ovn-nbctl lr-add $lr
+
+    for j in 1 2; do
+        ts=ts2$j
+        ovn-ic-nbctl --may-exist ts-add $ts
+
+        lrp=lrp-$lr-$ts
+        lsp=lsp-$ts-$lr
+        # Create LRP and connect to TS
+        ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:a$j:0$i 169.254.10$j.$i/24
+        ovn-nbctl lrp-set-options $lrp route_table=rtb$j
+        ovn-nbctl lsp-add $ts $lsp \
+                -- lsp-set-addresses $lsp router \
+                -- lsp-set-type $lsp router \
+                -- lsp-set-options $lsp router-port=$lrp
+    done
+done
+
+# Create directly-connected and static routes in VPC1
+ovn_as az2 ovn-nbctl lrp-add lr12 lrp-lr12 aa:aa:aa:aa:bb:01 "192.168.0.1/24"
+ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-add lr12 10.10.10.0/24 192.168.0.10
+ovn_as az2 ovn-nbctl --route-table=rtb2 lr-route-add lr12 10.10.10.0/24 192.168.0.11
+ovn_as az2 ovn-nbctl --route-table=rtb3 lr-route-add lr12 10.10.10.0/24 192.168.0.12
+
+# Create directly-connected route in VPC2
+ovn_as az2 ovn-nbctl lrp-add lr22 lrp-lr22 aa:aa:aa:aa:bb:01 "192.168.0.1/24"
+
+# Test direct routes from lr12 were learned to lr11
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr11 | grep 192.168 |
+             grep learned | awk '{print $1, $2, $5}' | sort ], [0], [dnl
+192.168.0.0/24 169.254.101.2 ecmp
+192.168.0.0/24 169.254.102.2 ecmp
+192.168.0.0/24 169.254.103.2 ecmp
+])
+
+# Test static routes from lr12 rtbs rtb1,rtb2,rtb3 were learned to lr11
+AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-list lr11], [0], [dnl
+IPv4 Routes
+Route Table rtb1:
+            10.10.10.0/24             169.254.101.2 dst-ip (learned)
+])
+AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb2 lr-route-list lr11], [0], [dnl
+IPv4 Routes
+Route Table rtb2:
+            10.10.10.0/24             169.254.102.2 dst-ip (learned)
+])
+AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb3 lr-route-list lr11], [0], [dnl
+IPv4 Routes
+Route Table rtb3:
+            10.10.10.0/24             169.254.103.2 dst-ip (learned)
+])
+
+# Test routes from lr12 didn't leak as learned to lr21
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr21 | grep 192.168 | sort], [0], [dnl
+           192.168.0.0/24             169.254.101.2 dst-ip (learned) ecmp
+           192.168.0.0/24             169.254.102.2 dst-ip (learned) ecmp
+])
+
+OVN_CLEANUP_IC([az1], [az2])
+
+AT_CLEANUP
+])
+
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([ovn-ic -- route sync -- multiple route tables IPv6])
+
+ovn_init_ic_db
+ovn-ic-nbctl ts-add ts1
+
+for i in 1 2; do
+    ovn_start az$i
+    ovn_as az$i
+
+    # Enable route learning at AZ level
+    ovn-nbctl set nb_global . options:ic-route-learn=true
+    # Enable route advertising at AZ level
+    ovn-nbctl set nb_global . options:ic-route-adv=true
+done
+
+# Create new transit switches and LRs. Test topology is next:
+# VPC1:
+#                       / transit switch (ts11) \
+# logical router (lr11) - transit switch (ts12) - logical router (lr12)
+#                       \ transit switch (ts13) /
+#
+# VPC2:
+#                       / transit switch (ts21) \
+# logical router (lr21)                           logical router (lr22)
+#                       \ transit switch (ts22) /
+#
+# each LR has one connected subnet except TS port
+
+
+# VPC1
+# create lr11, lr12, ts11, ts12, ts13 and connect them
+# assign route tables rtb1, rtb2, rtb3 to ts ports
+for i in 1 2; do
+    ovn_as az$i
+
+    lr=lr1$i
+    ovn-nbctl lr-add $lr
+
+    for j in 1 2 3; do
+        ts=ts1$j
+        ovn-ic-nbctl --may-exist ts-add $ts
+
+        lrp=lrp-$lr-$ts
+        lsp=lsp-$ts-$lr
+        # Create LRP and connect to TS
+        ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:a$j:0$i 2001:db8:$j::$i/64
+        ovn-nbctl lrp-set-options $lrp route_table=rtb$j
+        ovn-nbctl lsp-add $ts $lsp \
+                -- lsp-set-addresses $lsp router \
+                -- lsp-set-type $lsp router \
+                -- lsp-set-options $lsp router-port=$lrp
+    done
+done
+
+# VPC2
+# create lr21, lr22, ts21, ts22 and connect them
+# assign route tables rtb1, rtb2, rtb3 to ts ports
+for i in 1 2; do
+    ovn_as az$i
+
+    lr=lr2$i
+    ovn-nbctl lr-add $lr
+
+    for j in 1 2; do
+        ts=ts2$j
+        ovn-ic-nbctl --may-exist ts-add $ts
+
+        lrp=lrp-$lr-$ts
+        lsp=lsp-$ts-$lr
+        # Create LRP and connect to TS
+        ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:a$j:0$i 2001:db8:$j::$i/64
+        ovn-nbctl lrp-set-options $lrp route_table=rtb$j
+        ovn-nbctl lsp-add $ts $lsp \
+                -- lsp-set-addresses $lsp router \
+                -- lsp-set-type $lsp router \
+                -- lsp-set-options $lsp router-port=$lrp
+    done
+done
+
+# Create directly-connected and static routes in VPC1
+ovn_as az2 ovn-nbctl lrp-add lr12 lrp-lr12 aa:aa:aa:aa:bb:01 "2001:db8:200::1/64"
+ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-add lr12 2001:db8:aaaa::/64 2001:db8:200::10
+ovn_as az2 ovn-nbctl --route-table=rtb2 lr-route-add lr12 2001:db8:aaaa::/64 2001:db8:200::11
+ovn_as az2 ovn-nbctl --route-table=rtb3 lr-route-add lr12 2001:db8:aaaa::/64 2001:db8:200::12
+
+# Create directly-connected route in VPC2
+ovn_as az2 ovn-nbctl lrp-add lr22 lrp-lr22 aa:aa:aa:aa:bb:01 "2001:db8:200::1/64"
+
+# Test direct routes from lr12 were learned to lr11
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr11 | grep 2001:db8:200 |
+             grep learned | awk '{print $1, $2, $5}' | sort], [0], [dnl
+2001:db8:200::/64 2001:db8:1::2 ecmp
+2001:db8:200::/64 2001:db8:2::2 ecmp
+2001:db8:200::/64 2001:db8:3::2 ecmp
+])
+
+# Test static routes from lr12 rtbs rtb1,rtb2,rtb3 were learned to lr11
+AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-list lr11], [0], [dnl
+IPv6 Routes
+Route Table rtb1:
+       2001:db8:aaaa::/64             2001:db8:1::2 dst-ip (learned)
+])
+AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb2 lr-route-list lr11], [0], [dnl
+IPv6 Routes
+Route Table rtb2:
+       2001:db8:aaaa::/64             2001:db8:2::2 dst-ip (learned)
+])
+AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb3 lr-route-list lr11], [0], [dnl
+IPv6 Routes
+Route Table rtb3:
+       2001:db8:aaaa::/64             2001:db8:3::2 dst-ip (learned)
+])
+
+# Test routes from lr12 didn't leak as learned to lr21
+AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr21 | grep 2001 | sort], [0], [dnl
+        2001:db8:200::/64             2001:db8:1::2 dst-ip (learned) ecmp
+        2001:db8:200::/64             2001:db8:2::2 dst-ip (learned) ecmp
+])
+
+OVN_CLEANUP_IC([az1], [az2])
+
+AT_CLEANUP
+])

From patchwork Fri Sep  3 10:34:41 2021
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Patchwork-Submitter: Vladislav Odintsov 
X-Patchwork-Id: 1524205
Return-Path: 
X-Original-To: incoming@patchwork.ozlabs.org
Delivered-To: patchwork-incoming@bilbo.ozlabs.org
Authentication-Results: ozlabs.org;
	dkim=fail reason="signature verification failed" (2048-bit key;
 unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256
 header.s=20210112 header.b=QDpDWHPQ;
	dkim-atps=neutral
Authentication-Results: ozlabs.org;
 spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org
 (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org;
 envelope-from=ovs-dev-bounces@openvswitch.org; receiver=)
Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest
 SHA256)
	(No client certificate requested)
	by ozlabs.org (Postfix) with ESMTPS id 4H1Djx5Qwkz9s5R
	for ; Fri,  3 Sep 2021 20:36:37 +1000 (AEST)
Received: from localhost (localhost [127.0.0.1])
	by smtp1.osuosl.org (Postfix) with ESMTP id 97CCE83E65;
	Fri,  3 Sep 2021 10:36:34 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
Received: from smtp1.osuosl.org ([127.0.0.1])
	by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id iBeoglTJNpF2; Fri,  3 Sep 2021 10:36:30 +0000 (UTC)
Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56])
	by smtp1.osuosl.org (Postfix) with ESMTPS id 02DBB83E51;
	Fri,  3 Sep 2021 10:36:29 +0000 (UTC)
Received: from lf-lists.osuosl.org (localhost [127.0.0.1])
	by lists.linuxfoundation.org (Postfix) with ESMTP id DC709C0010;
	Fri,  3 Sep 2021 10:36:28 +0000 (UTC)
X-Original-To: dev@openvswitch.org
Delivered-To: ovs-dev@lists.linuxfoundation.org
Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138])
 by lists.linuxfoundation.org (Postfix) with ESMTP id 49287C000E
 for ; Fri,  3 Sep 2021 10:36:27 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by smtp1.osuosl.org (Postfix) with ESMTP id 07FB683E6A
 for ; Fri,  3 Sep 2021 10:35:42 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
Received: from smtp1.osuosl.org ([127.0.0.1])
 by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id UVPMrMHuOCKL for ;
 Fri,  3 Sep 2021 10:35:41 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.8.0
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [IPv6:2a00:1450:4864:20::536])
 by smtp1.osuosl.org (Postfix) with ESMTPS id 4F50083E3D
 for ; Fri,  3 Sep 2021 10:35:41 +0000 (UTC)
Received: by mail-ed1-x536.google.com with SMTP id g21so7347270edw.4
 for ; Fri, 03 Sep 2021 03:35:41 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=from:to:cc:subject:date:message-id:in-reply-to:references
 :mime-version:content-transfer-encoding;
 bh=J0h4s0acx0208ffw1XqRqbwugtQweRB618/MnQFZ2c0=;
 b=QDpDWHPQc8ZloxzJ/GfD8WgpFoYGhs4dAPHVof/ceu3tjbB5ldpHLp/jVbyslDKX1c
 jrwT8aqm0qnDCa77gXYchK25JLA4AAME/rZAJ9MRMl+kYmQYZI6sUIiXzaf8Makszz48
 YnQATTm4PBnWHqmv4AEC2G0QwGKnOyj/BQMVHrGPLKGO+YsSCcpDW0lXZzwc4PdXmWCm
 Hqp37wQK4BXIV0bzVlzxUcCO47ShFKcQlfnyuVpxRX1iH4IaN/nMxhb0ERb3TM0zNUfr
 rAE7NCgKnSAOOQ5SzX6pD8p5vi+fKS58rnqsIsDCDSs0OxW8TZi7ssOy9xhXwqhJD77C
 CT6Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
 :references:mime-version:content-transfer-encoding;
 bh=J0h4s0acx0208ffw1XqRqbwugtQweRB618/MnQFZ2c0=;
 b=TK004zvwNpPC0mTMqrFSDFgZOLFw4SREvmRns/G7uY1hBzg0lX2OPXunFeEZq92ugN
 ycwPuPTuYZUEGCjkWcZ5QKp7lWFZgvwg4Rwc/uwRsOEXKJ8hMCz9VgHKKi6kHVA7V84g
 SSmnHlynLbKO2j4rn60jaFm0zKJuXR8P+KEnn7BgLLNU/T3FgQzM43Qz9JdcT1xW+LSX
 VlK078hB8fbIdWmRFYFltlQWSvT6IOKVj1N87EIKzbno4QtXr4fyQErlDt7iS2TPtWm+
 ySdhcHF2aM1e6/DbLDFBlx4Be9f9IzKJpRB/9czStyF5K5Jj4q6FPfSm41ZFJB2jWMPN
 eGtA==
X-Gm-Message-State: AOAM5328GbQQcqxfragvb5g36VWONHM0ASicXVRCdfqWxjtODSURCf24
 fJDgxCcGMa76P8vh+gBuq6xcqG2Wa5s=
X-Google-Smtp-Source: 
 ABdhPJwxmhbE7NNlelKr645BlgD9VuV1OCtT47+k1XZvsWo++GcJP5DA4qU4GJ25jgaIEAVBLMP7tA==
X-Received: by 2002:a05:6402:2810:: with SMTP id
 h16mr3246994ede.133.1630665339376;
 Fri, 03 Sep 2021 03:35:39 -0700 (PDT)
Received: from localhost.localdomain ([85.187.18.127])
 by smtp.gmail.com with ESMTPSA id d22sm2609897ejj.47.2021.09.03.03.35.38
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Fri, 03 Sep 2021 03:35:39 -0700 (PDT)
From: Vladislav Odintsov 
To: dev@openvswitch.org
Date: Fri,  3 Sep 2021 13:34:41 +0300
Message-Id: <20210903103441.33178-7-odivlad@gmail.com>
X-Mailer: git-send-email 2.30.0
In-Reply-To: <20210903103441.33178-1-odivlad@gmail.com>
References: <20210903103441.33178-1-odivlad@gmail.com>
MIME-Version: 1.0
Cc: Vladislav Odintsov 
Subject: [ovs-dev] [PATCH ovn v2 6/6] Update Authors list
X-BeenThere: ovs-dev@openvswitch.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: 
List-Unsubscribe: ,
 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: ,
 
Errors-To: ovs-dev-bounces@openvswitch.org
Sender: "dev" 

Signed-off-by: Vladislav Odintsov 
---
 AUTHORS.rst | 1 +
 1 file changed, 1 insertion(+)

diff --git a/AUTHORS.rst b/AUTHORS.rst
index 3f93874a3..e03c6b407 100644
--- a/AUTHORS.rst
+++ b/AUTHORS.rst
@@ -388,6 +388,7 @@ Venkata Anil Kommaddi              vkommadi@redhat.com
 Venu Iyer                          venugopali@nvidia.com
 Vishal Deep Ajmera                 vishal.deep.ajmera@ericsson.com
 Vivien Bernet-Rollande             vbr@soprive.net
+Vladislav Odintsov                 odivlad@gmail.com
 wangqianyu                         wang.qianyu@zte.com.cn
 Wang Sheng-Hui                     shhuiw@gmail.com
 Wang Zhike                         wangzhike@jd.com