From patchwork Mon Feb 19 21:33:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roberto Bartzen Acosta X-Patchwork-Id: 1901139 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=luizalabs.com header.i=@luizalabs.com header.a=rsa-sha256 header.s=google header.b=NufjZGOY; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Tdwl46hbvz1yP6 for ; Tue, 20 Feb 2024 08:33:36 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 4EEE781F7D; Mon, 19 Feb 2024 21:33:34 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EDjLMEMXb8Pg; Mon, 19 Feb 2024 21:33:33 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org E8F1B81F64 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key, unprotected) header.d=luizalabs.com header.i=@luizalabs.com header.a=rsa-sha256 header.s=google header.b=NufjZGOY Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id E8F1B81F64; Mon, 19 Feb 2024 21:33:32 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B365DC0072; Mon, 19 Feb 2024 21:33:32 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id B757BC0037 for ; Mon, 19 Feb 2024 21:33:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id A117F607BD for ; Mon, 19 Feb 2024 21:33:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id N4UXz6OYlXD1 for ; Mon, 19 Feb 2024 21:33:30 +0000 (UTC) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::42c; helo=mail-pf1-x42c.google.com; envelope-from=roberto.acosta@luizalabs.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org 66E3D6080E Authentication-Results: smtp3.osuosl.org; dmarc=pass (p=none dis=none) header.from=luizalabs.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 66E3D6080E Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key, unprotected) header.d=luizalabs.com header.i=@luizalabs.com header.a=rsa-sha256 header.s=google header.b=NufjZGOY Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by smtp3.osuosl.org (Postfix) with ESMTPS id 66E3D6080E for ; Mon, 19 Feb 2024 21:33:30 +0000 (UTC) Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-6e09a890341so2392861b3a.3 for ; Mon, 19 Feb 2024 13:33:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=luizalabs.com; s=google; t=1708378409; x=1708983209; darn=openvswitch.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=CWVSzUk1O/uO9QHUFvvhxfqJ7ANx+P6KPZZS+2RVUWA=; b=NufjZGOYORixZ1uLM3E5/+ttCO0sY6lqawgBjBFob0mjoHR/E00Cp7vCzAxi93v43i Z9lo8p5fCa+TlIhx3I0a7bi/sXhUlF0njUJ8CwIHsKmuAWA70nZE6ZWk0suMfgyDhAHF L4q9rxkmHvQ/4/ziDARCr12Iuj5/wa4UrphFc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708378409; x=1708983209; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=CWVSzUk1O/uO9QHUFvvhxfqJ7ANx+P6KPZZS+2RVUWA=; b=DL1ymEBNbm3hUcdo2JAdtyaOZncbdmkvWt/D+cdt0KKkZNeQ3L0k1I/cc05zq6kEAl teBvYLQ0yC5UNf7neZ6l+iYnUqUpNdugRlD+pNLwn4/+hjTRwSr3apic5VhtY9R+z1HM YuyU35A+yKRzn8yzpKvpL2o0bYoo/jqozZCGOW8LVIPgbDFeOHh8R5dyNKXp1k68ZUc0 Tx+eKLw1IRN9hlUZWvYsusZvCdOK8vzpEj3chTjyAiDlKPbNjC1thjh27a2LnPGElaKp jdMKhEHZzE5CZezrj0xSC/BJrm2Rswxqlbn/g3JyT6cA8okjZvxNRuHi9IpPLnrlIY/s mZXA== X-Gm-Message-State: AOJu0YzRYfgJ3Pyfoy8yXAtvZTHgb1yT3fqVVU32dtPqmj9wpE9ZzT38 xKV1Usy5MZlQOKIY0DV2GCcZYNATdoyE9F5lAdE/9Y9AeXvK8GjsyWARh2ag/0q+zjlpQeN1oJN OhNv11cTIgxk9PVBYDsoi4FDwbIEkKax3Idn1qe+ThDb8ylEMVm5jViUE X-Google-Smtp-Source: AGHT+IF2fyU3HUtAlSeZemiTnD2Xsj9toA4TUTvUvSGjyx7GEd0kMCG45qq04NFEuOHo/TgNOA3QKw== X-Received: by 2002:a05:6a20:c891:b0:1a0:a43b:cbdb with SMTP id hb17-20020a056a20c89100b001a0a43bcbdbmr4827477pzb.27.1708378408660; Mon, 19 Feb 2024 13:33:28 -0800 (PST) Received: from WNL1099LABS421.magazineluiza.intranet ([191.187.213.146]) by smtp.gmail.com with ESMTPSA id cz3-20020a17090ad44300b00299bf19e872sm1671039pjb.44.2024.02.19.13.33.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Feb 2024 13:33:28 -0800 (PST) From: Roberto Bartzen Acosta To: dev@openvswitch.org Date: Mon, 19 Feb 2024 18:33:21 -0300 Message-Id: <20240219213321.56103-1-roberto.acosta@luizalabs.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn] northd: Fix logical router load-balancer nat rules when using DGP. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit fixes the build_distr_lrouter_nat_flows_for_lb function to include one NAT flow entry for each DGP in use. Since we have added support to create multiple gateway ports per logical router, it's necessary to include in the LR nat rules pipeline a specific entry for each attached DGP. Otherwise, the ingress traffic is only redirected when the incoming LRP matches the chassis_resident field. Considering that DNAT rules for DGPs were implemented with the need to configure the DGP-related gateway-port column, the load-balancer NAT rule configuration can use a similar idea. In this case, we don't know the LRP responsible for the incoming traffic, and therefore we must apply the load-balancer automatically created NAT rule in all DGPs to allow the incoming traffic. Reported-at: https://bugs.launchpad.net/ubuntu/+source/ovn/+bug/2054322 Fixes: 15348b7b806f ("ovn-northd: Multiple distributed gateway port support.") Signed-off-by: Roberto Bartzen Acosta --- northd/en-lr-stateful.c | 12 ------ northd/northd.c | 14 ++++--- tests/ovn-northd.at | 92 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 100 insertions(+), 18 deletions(-) diff --git a/northd/en-lr-stateful.c b/northd/en-lr-stateful.c index 6d0192487..7ffa4a690 100644 --- a/northd/en-lr-stateful.c +++ b/northd/en-lr-stateful.c @@ -537,18 +537,6 @@ lr_stateful_record_create(struct lr_stateful_table *table, table->array[od->index] = lr_stateful_rec; - /* Load balancers are not supported (yet) if a logical router has multiple - * distributed gateway port. Log a warning. */ - if (lr_stateful_rec->has_lb_vip && lr_has_multiple_gw_ports(od)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "Load-balancers are configured on logical " - "router %s, which has %"PRIuSIZE" distributed " - "gateway ports. Load-balancer is not supported " - "yet when there is more than one distributed " - "gateway port on the router.", - od->nbr->name, od->n_l3dgw_ports); - } - return lr_stateful_rec; } diff --git a/northd/northd.c b/northd/northd.c index 2c3560ce2..7eb943d2f 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -10919,10 +10919,9 @@ static void build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, enum lrouter_nat_lb_flow_type type, struct ovn_datapath *od, - struct lflow_ref *lflow_ref) + struct lflow_ref *lflow_ref, + struct ovn_port *dgp) { - struct ovn_port *dgp = od->l3dgw_ports[0]; - const char *undnat_action; switch (type) { @@ -10953,7 +10952,7 @@ build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, if (ctx->lb_vip->n_backends || !ctx->lb_vip->empty_backend_rej) { ds_put_format(ctx->new_match, " && is_chassis_resident(%s)", - od->l3dgw_ports[0]->cr_port->json_key); + dgp->cr_port->json_key); } ovn_lflow_add_with_hint__(ctx->lflows, od, S_ROUTER_IN_DNAT, ctx->prio, @@ -11164,8 +11163,11 @@ build_lrouter_nat_flows_for_lb( if (!od->n_l3dgw_ports) { bitmap_set1(gw_dp_bitmap[type], index); } else { - build_distr_lrouter_nat_flows_for_lb(&ctx, type, od, - lb_dps->lflow_ref); + for (size_t i = 0; i < od->n_l3dgw_ports; i++) { + struct ovn_port *dgp = od->l3dgw_ports[i]; + build_distr_lrouter_nat_flows_for_lb(&ctx, type, od, + lb_dps->lflow_ref, dgp); + } } if (lb->affinity_timeout) { diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 6fdd761da..fa24935e1 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -12313,3 +12313,95 @@ check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute AT_CLEANUP + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([Load balancer with Distributed Gateway Ports (DGP)]) +ovn_start + +check ovn-nbctl ls-add public +check ovn-nbctl lr-add lr1 + +# lr1 DGP ts1 +check ovn-nbctl ls-add ts1 +check ovn-nbctl lrp-add lr1 lr1-ts1 00:00:01:02:03:04 172.16.10.1/24 +check ovn-nbctl lrp-set-gateway-chassis lr1-ts1 chassis-2 + +# lr1 DGP ts2 +check ovn-nbctl ls-add ts2 +check ovn-nbctl lrp-add lr1 lr1-ts2 00:00:01:02:03:05 172.16.20.1/24 +check ovn-nbctl lrp-set-gateway-chassis lr1-ts2 chassis-3 + +# lr1 DGP public +check ovn-nbctl lrp-add lr1 lr1_public 00:de:ad:ff:00:01 173.16.0.1/16 +check ovn-nbctl lrp-add lr1 lr1_s1 00:de:ad:fe:00:02 172.16.0.1/24 +check ovn-nbctl lrp-set-gateway-chassis lr1_public chassis-1 + +check ovn-nbctl ls-add s1 +# s1 - lr1 +check ovn-nbctl lsp-add s1 s1_lr1 +check ovn-nbctl lsp-set-type s1_lr1 router +check ovn-nbctl lsp-set-addresses s1_lr1 "00:de:ad:fe:00:02 172.16.0.1" +check ovn-nbctl lsp-set-options s1_lr1 router-port=lr1_s1 + +# s1 - backend vm1 +check ovn-nbctl lsp-add s1 vm1 +check ovn-nbctl lsp-set-addresses vm1 "00:de:ad:01:00:01 172.16.0.101" + +# s1 - backend vm2 +check ovn-nbctl lsp-add s1 vm2 +check ovn-nbctl lsp-set-addresses vm2 "00:de:ad:01:00:02 172.16.0.102" + +# s1 - backend vm3 +check ovn-nbctl lsp-add s1 vm3 +check ovn-nbctl lsp-set-addresses vm3 "00:de:ad:01:00:03 172.16.0.103" + +# Add the lr1 DGP ts1 to the public switch +check ovn-nbctl lsp-add public public_lr1_ts1 +check ovn-nbctl lsp-set-type public_lr1_ts1 router +check ovn-nbctl lsp-set-addresses public_lr1_ts1 router +check ovn-nbctl lsp-set-options public_lr1_ts1 router-port=lr1-ts1 nat-addresses=router + +# Add the lr1 DGP ts2 to the public switch +check ovn-nbctl lsp-add public public_lr1_ts2 +check ovn-nbctl lsp-set-type public_lr1_ts2 router +check ovn-nbctl lsp-set-addresses public_lr1_ts2 router +check ovn-nbctl lsp-set-options public_lr1_ts2 router-port=lr1-ts2 nat-addresses=router + +# Add the lr1 DGP public to the public switch +check ovn-nbctl lsp-add public public_lr1 +check ovn-nbctl lsp-set-type public_lr1 router +check ovn-nbctl lsp-set-addresses public_lr1 router +check ovn-nbctl lsp-set-options public_lr1 router-port=lr1_public nat-addresses=router + +# Create the Load Balancer lb1 +check ovn-nbctl --wait=sb lb-add lb1 "30.0.0.1" "172.16.0.103,172.16.0.102,172.16.0.101" + +# Associate load balancer to s1 +check ovn-nbctl ls-lb-add s1 lb1 +check ovn-nbctl --wait=sb sync + +ovn-sbctl dump-flows s1 > s1flows +AT_CAPTURE_FILE([s1flows]) + +AT_CHECK([grep "ls_in_pre_stateful" s1flows | ovn_strip_lflows | grep "30.0.0.1"], [0], [dnl + table=??(ls_in_pre_stateful ), priority=120 , match=(reg0[[2]] == 1 && ip4.dst == 30.0.0.1), action=(reg1 = 30.0.0.1; ct_lb_mark;) +]) +AT_CHECK([grep "ls_in_lb" s1flows | ovn_strip_lflows | grep "30.0.0.1"], [0], [dnl + table=??(ls_in_lb ), priority=110 , match=(ct.new && ip4.dst == 30.0.0.1), action=(reg0[[1]] = 0; ct_lb_mark(backends=172.16.0.103,172.16.0.102,172.16.0.101);) +]) + +# Associate load balancer to lr1 with DGP +check ovn-nbctl lr-lb-add lr1 lb1 +check ovn-nbctl --wait=sb sync + +ovn-sbctl dump-flows lr1 > lr1flows +AT_CAPTURE_FILE([lr1flows]) + +AT_CHECK([grep "lr_in_dnat" lr1flows | ovn_strip_lflows | grep "30.0.0.1"], [0], [dnl + table=??(lr_in_dnat ), priority=110 , match=(ct.new && !ct.rel && ip4 && ip4.dst == 30.0.0.1 && is_chassis_resident("cr-lr1-ts1")), action=(ct_lb_mark(backends=172.16.0.103,172.16.0.102,172.16.0.101);) + table=??(lr_in_dnat ), priority=110 , match=(ct.new && !ct.rel && ip4 && ip4.dst == 30.0.0.1 && is_chassis_resident("cr-lr1-ts2")), action=(ct_lb_mark(backends=172.16.0.103,172.16.0.102,172.16.0.101);) + table=??(lr_in_dnat ), priority=110 , match=(ct.new && !ct.rel && ip4 && ip4.dst == 30.0.0.1 && is_chassis_resident("cr-lr1_public")), action=(ct_lb_mark(backends=172.16.0.103,172.16.0.102,172.16.0.101);) +]) + +AT_CLEANUP +])