From patchwork Wed Nov 2 15:57:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Bianconi X-Patchwork-Id: 1698212 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=fsnVWmYo; dkim-atps=neutral Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N2Wkq1Z12z23l6 for ; Thu, 3 Nov 2022 02:58:11 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id E037940A98; Wed, 2 Nov 2022 15:58:08 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org E037940A98 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=fsnVWmYo X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nMI12TfzejnC; Wed, 2 Nov 2022 15:58:07 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 33B1940168; Wed, 2 Nov 2022 15:58:06 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 33B1940168 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id F25F1C0071; Wed, 2 Nov 2022 15:58:05 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 912B4C0032 for ; Wed, 2 Nov 2022 15:58:04 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 6BE7060F5E for ; Wed, 2 Nov 2022 15:58:04 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6BE7060F5E Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=fsnVWmYo X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3RedMgoYEwpD for ; Wed, 2 Nov 2022 15:58:03 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org E2F3B60F36 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id E2F3B60F36 for ; Wed, 2 Nov 2022 15:58:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667404681; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qkQx0j7PT9OpeD6nRliQL6XLV2vPhnCBtoy6+RCRLvs=; b=fsnVWmYof9Td7lv3CZdbMwgDZT0ZzEF61Y0fwe8Jx52F1k8+gIjVOAlTYmGOGc7F59Vyny FqxEAo4oZizUDvTQzzvBXRR7oumu7Wxb1GZA0/1Rl/licxFx4a0jfT9NO3nCclIKRWAw9V vcjfFT66Zx/Rtff1UYEKz+rnzaiqC3w= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-651-zKQzZjgIN7q0lOv15lmshA-1; Wed, 02 Nov 2022 11:58:01 -0400 X-MC-Unique: zKQzZjgIN7q0lOv15lmshA-1 Received: by mail-wm1-f71.google.com with SMTP id r6-20020a1c4406000000b003cf4d3b6644so3897989wma.6 for ; Wed, 02 Nov 2022 08:58:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qkQx0j7PT9OpeD6nRliQL6XLV2vPhnCBtoy6+RCRLvs=; b=Vb03KRDOpwqSqIebA/eLQWZgboQL0LQ/ey5uLfnACsmVIiLzl7iM4mZpY9hB5Mdd4Q 5Z33/il6+zQX+JY2IX32ifuV4CZJR9iKybNjUi73OfyEF3QnNTWqBGJNGFZ3KckM2j9m WqpmDafU5fE10TQR51yYPJ1BS0aKsTHEjw0Avo7I0KTfjED2zlwcAcDl4HR4RdfQG+0m Mok+ZwT4jJ0l+9lFFcbKwXnSVzo+P/wOAeJcnGJ4D9RnJB3tBpAE9s50q6U9C18SyGV1 YCehi3rPwco/WEE2wWo9jbFY4l8sYsHoloi5IfdusLIi9NfNGmxBfZG1rIpcBCfEmKvg yz9g== X-Gm-Message-State: ACrzQf2mqWOj1qY6NDwS2GOLG4xWXfrl2fEU5cbny614ldds5ZAahfuL 0N2JUCxbpJoyvP7z33inG8gficog8Nwf9y+p1WUqtewXuCLu1jNAqG07aBKD2LX6+Eqs3nwS5PO EDP8HT43mvtgFPxFCtiaiDurucl6aXwbibh17AtENmUo9ROCBIA7Cr/LABm2B84mpCG/GW/UrLG CZpkyQ X-Received: by 2002:a7b:c047:0:b0:3b4:adc7:1ecb with SMTP id u7-20020a7bc047000000b003b4adc71ecbmr16162053wmc.144.1667404679163; Wed, 02 Nov 2022 08:57:59 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7+hnS4rxH5NaCgMVGk8vTg6qrwciHRrdzrO2KDg7b7Sy43YJO8ZF7Gbpida5Jg3xr0zSK6Ag== X-Received: by 2002:a7b:c047:0:b0:3b4:adc7:1ecb with SMTP id u7-20020a7bc047000000b003b4adc71ecbmr16162025wmc.144.1667404678706; Wed, 02 Nov 2022 08:57:58 -0700 (PDT) Received: from localhost (net-188-216-77-84.cust.vodafonedsl.it. [188.216.77.84]) by smtp.gmail.com with ESMTPSA id q17-20020a1cf311000000b003cf47fdead5sm2432401wmq.30.2022.11.02.08.57.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Nov 2022 08:57:58 -0700 (PDT) From: Lorenzo Bianconi To: ovs-dev@openvswitch.org Date: Wed, 2 Nov 2022 16:57:47 +0100 Message-Id: X-Mailer: git-send-email 2.38.1 In-Reply-To: References: MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: dceara@redhat.com Subject: [ovs-dev] [PATCH v6 ovn 1/3] actions: introduce commit_lb_aff action X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" commit_lb_aff action translates to an openflow "learn" action that inserts a new flow in the OFTABLE_CHK_LB_AFFINITY table. The new flow is used to match on the the 5-tuple and set REGBIT_KNOWN_LB_SESSION bit. Moreover the new flow stores backend IP and port in register REG4 and REG8[0..15] respectively. Acked-by: Mark Michelson Signed-off-by: Lorenzo Bianconi --- controller/lflow.h | 1 + include/ovn/actions.h | 15 ++ include/ovn/logical-fields.h | 8 + lib/actions.c | 387 +++++++++++++++++++++++++++++++++++ ovn-sb.xml | 35 ++++ tests/ovn.at | 10 + utilities/ovn-trace.c | 2 + 7 files changed, 458 insertions(+) diff --git a/controller/lflow.h b/controller/lflow.h index 8cbe312ca..4be079555 100644 --- a/controller/lflow.h +++ b/controller/lflow.h @@ -79,6 +79,7 @@ struct uuid; #define OFTABLE_CHK_OUT_PORT_SEC 75 #define OFTABLE_ECMP_NH_MAC 76 #define OFTABLE_ECMP_NH 77 +#define OFTABLE_CHK_LB_AFFINITY 78 enum ref_type { REF_TYPE_ADDRSET, diff --git a/include/ovn/actions.h b/include/ovn/actions.h index d7ee84dac..597cbb8e3 100644 --- a/include/ovn/actions.h +++ b/include/ovn/actions.h @@ -121,6 +121,7 @@ struct ovn_extend_table; OVNACT(COMMIT_ECMP_NH, ovnact_commit_ecmp_nh) \ OVNACT(CHK_ECMP_NH_MAC, ovnact_result) \ OVNACT(CHK_ECMP_NH, ovnact_result) \ + OVNACT(COMMIT_LB_AFF, ovnact_commit_lb_aff) \ /* enum ovnact_type, with a member OVNACT_ for each action. */ enum OVS_PACKED_ENUM ovnact_type { @@ -463,6 +464,20 @@ struct ovnact_commit_ecmp_nh { uint8_t proto; }; +/* OVNACT_COMMIT_LB_AFF. */ +struct ovnact_commit_lb_aff { + struct ovnact ovnact; + + struct in6_addr vip; + uint16_t vip_port; + uint8_t proto; + + struct in6_addr backend; + uint16_t backend_port; + + uint16_t timeout; +}; + /* Internal use by the helpers below. */ void ovnact_init(struct ovnact *, enum ovnact_type, size_t len); void *ovnact_put(struct ofpbuf *, enum ovnact_type, size_t len); diff --git a/include/ovn/logical-fields.h b/include/ovn/logical-fields.h index 3db7265e4..8060488f9 100644 --- a/include/ovn/logical-fields.h +++ b/include/ovn/logical-fields.h @@ -53,6 +53,11 @@ enum ovn_controller_event { #define MFF_N_LOG_REGS 10 +#define MFF_LOG_LB_AFF_MATCH_IP4_ADDR MFF_REG4 +#define MFF_LOG_LB_AFF_MATCH_LS_IP6_ADDR MFF_XXREG0 +#define MFF_LOG_LB_AFF_MATCH_LR_IP6_ADDR MFF_XXREG1 +#define MFF_LOG_LB_AFF_MATCH_PORT MFF_REG8 + void ovn_init_symtab(struct shash *symtab); /* MFF_LOG_FLAGS_REG bit assignments */ @@ -71,6 +76,7 @@ enum mff_log_flags_bits { MLF_USE_SNAT_ZONE = 11, MLF_CHECK_PORT_SEC_BIT = 12, MLF_LOOKUP_COMMIT_ECMP_NH_BIT = 13, + MLF_USE_LB_AFF_SESSION_BIT = 14, }; /* MFF_LOG_FLAGS_REG flag assignments */ @@ -116,6 +122,8 @@ enum mff_log_flags { MLF_LOCALPORT = (1 << MLF_LOCALPORT_BIT), MLF_LOOKUP_COMMIT_ECMP_NH = (1 << MLF_LOOKUP_COMMIT_ECMP_NH_BIT), + + MLF_USE_LB_AFF_SESSION = (1 << MLF_USE_LB_AFF_SESSION_BIT), }; /* OVN logical fields diff --git a/lib/actions.c b/lib/actions.c index adbb42db4..de9c69860 100644 --- a/lib/actions.c +++ b/lib/actions.c @@ -4600,6 +4600,391 @@ encode_CHK_ECMP_NH(const struct ovnact_result *res, MLF_LOOKUP_COMMIT_ECMP_NH_BIT, ofpacts); } +static void +parse_commit_lb_aff(struct action_context *ctx, + struct ovnact_commit_lb_aff *lb_aff) +{ + int vip_family, backend_family; + uint16_t timeout, port = 0; + char *ip_str; + + lexer_force_match(ctx->lexer, LEX_T_LPAREN); /* Skip '('. */ + if (!lexer_match_id(ctx->lexer, "vip")) { + lexer_syntax_error(ctx->lexer, "invalid parameter"); + return; + } + + if (!lexer_force_match(ctx->lexer, LEX_T_EQUALS)) { + lexer_syntax_error(ctx->lexer, "invalid parameter"); + return; + } + + if (ctx->lexer->token.type != LEX_T_STRING) { + lexer_syntax_error(ctx->lexer, "invalid parameter"); + return; + } + + if (!ip_address_and_port_from_lb_key(ctx->lexer->token.s, &ip_str, + &lb_aff->vip, &port, &vip_family)) { + lexer_syntax_error(ctx->lexer, "invalid parameter"); + return; + } + + lb_aff->vip_port = port; + free(ip_str); + + lexer_get(ctx->lexer); + lexer_force_match(ctx->lexer, LEX_T_COMMA); + + if (!lexer_match_id(ctx->lexer, "backend")) { + lexer_syntax_error(ctx->lexer, "invalid parameter"); + return; + } + + if (!lexer_force_match(ctx->lexer, LEX_T_EQUALS)) { + lexer_syntax_error(ctx->lexer, "invalid parameter"); + return; + } + + if (ctx->lexer->token.type != LEX_T_STRING) { + lexer_syntax_error(ctx->lexer, "invalid parameter"); + return; + } + + if (!ip_address_and_port_from_lb_key(ctx->lexer->token.s, &ip_str, + &lb_aff->backend, &port, + &backend_family)) { + lexer_syntax_error(ctx->lexer, "invalid parameter"); + return; + } + free(ip_str); + + if (backend_family != vip_family) { + lexer_syntax_error(ctx->lexer, "invalid protocol family"); + return; + } + + lb_aff->backend_port = port; + + lexer_get(ctx->lexer); + lexer_force_match(ctx->lexer, LEX_T_COMMA); + + if (lb_aff->vip_port) { + if (!lexer_match_id(ctx->lexer, "proto")) { + lexer_syntax_error(ctx->lexer, "invalid parameter"); + return; + } + + if (!lexer_force_match(ctx->lexer, LEX_T_EQUALS)) { + lexer_syntax_error(ctx->lexer, "invalid parameter"); + return; + } + + if (lexer_match_id(ctx->lexer, "tcp")) { + lb_aff->proto = IPPROTO_TCP; + } else if (lexer_match_id(ctx->lexer, "udp")) { + lb_aff->proto = IPPROTO_UDP; + } else if (lexer_match_id(ctx->lexer, "sctp")) { + lb_aff->proto = IPPROTO_SCTP; + } else { + lexer_syntax_error(ctx->lexer, "invalid protocol"); + return; + } + lexer_force_match(ctx->lexer, LEX_T_COMMA); + } + + if (!lexer_match_id(ctx->lexer, "timeout")) { + lexer_syntax_error(ctx->lexer, "invalid parameter"); + return; + } + if (!lexer_force_match(ctx->lexer, LEX_T_EQUALS)) { + lexer_syntax_error(ctx->lexer, "invalid parameter"); + return; + } + if (!action_parse_uint16(ctx, &timeout, "affinity timeout")) { + return; + } + lb_aff->timeout = timeout; + + lexer_force_match(ctx->lexer, LEX_T_RPAREN); /* Skip ')'. */ + +} + +static void +format_COMMIT_LB_AFF(const struct ovnact_commit_lb_aff *lb_aff, struct ds *s) +{ + bool ipv6 = !IN6_IS_ADDR_V4MAPPED(&lb_aff->vip); + + if (ipv6) { + char ip_str[INET6_ADDRSTRLEN] = {}; + inet_ntop(AF_INET6, &lb_aff->vip, ip_str, INET6_ADDRSTRLEN); + ds_put_format(s, "commit_lb_aff(vip = \"[%s]", ip_str); + } else { + ovs_be32 ip = in6_addr_get_mapped_ipv4(&lb_aff->vip); + char *ip_str = xasprintf(IP_FMT, IP_ARGS(ip)); + ds_put_format(s, "commit_lb_aff(vip = \"%s", ip_str); + free(ip_str); + } + if (lb_aff->vip_port) { + ds_put_format(s, ":%d", lb_aff->vip_port); + } + ds_put_cstr(s, "\""); + + if (ipv6) { + char ip_str[INET6_ADDRSTRLEN] = {}; + inet_ntop(AF_INET6, &lb_aff->backend, ip_str, INET6_ADDRSTRLEN); + ds_put_format(s, ", backend = \"[%s]", ip_str); + } else { + ovs_be32 ip = in6_addr_get_mapped_ipv4(&lb_aff->backend); + char *ip_str = xasprintf(IP_FMT, IP_ARGS(ip)); + ds_put_format(s, ", backend = \"%s", ip_str); + free(ip_str); + } + if (lb_aff->backend_port) { + ds_put_format(s, ":%d", lb_aff->backend_port); + } + ds_put_cstr(s, "\""); + + if (lb_aff->proto) { + const char *proto; + switch (lb_aff->proto) { + case IPPROTO_UDP: + proto = "udp"; + break; + case IPPROTO_SCTP: + proto = "sctp"; + break; + case IPPROTO_TCP: + default: + proto = "tcp"; + break; + } + ds_put_format(s, ", proto = %s", proto); + } + ds_put_format(s, ", timeout = %d);", lb_aff->timeout); +} + +static void +encode_COMMIT_LB_AFF(const struct ovnact_commit_lb_aff *lb_aff, + const struct ovnact_encode_params *ep, + struct ofpbuf *ofpacts) +{ + bool ipv6 = !IN6_IS_ADDR_V4MAPPED(&lb_aff->vip); + size_t ol_offset = ofpacts->size; + struct ofpact_learn *ol = ofpact_put_LEARN(ofpacts); + struct match match = MATCH_CATCHALL_INITIALIZER; + struct ofpact_learn_spec *ol_spec; + unsigned int imm_bytes; + uint8_t *src_imm; + + ol->flags = NX_LEARN_F_DELETE_LEARNED; + ol->idle_timeout = lb_aff->timeout; /* seconds. */ + ol->hard_timeout = OFP_FLOW_PERMANENT; + ol->priority = OFP_DEFAULT_PRIORITY; + ol->table_id = OFTABLE_CHK_LB_AFFINITY; + + /* Match on metadata of the packet that created the new table. */ + ol_spec = ofpbuf_put_zeros(ofpacts, sizeof *ol_spec); + ol_spec->dst.field = mf_from_id(MFF_METADATA); + ol_spec->dst.ofs = 0; + ol_spec->dst.n_bits = ol_spec->dst.field->n_bits; + ol_spec->n_bits = ol_spec->dst.n_bits; + ol_spec->dst_type = NX_LEARN_DST_MATCH; + ol_spec->src_type = NX_LEARN_SRC_FIELD; + ol_spec->src.field = mf_from_id(MFF_METADATA); + + /* Match on the same ETH type as the packet that created the new table. */ + ol_spec = ofpbuf_put_zeros(ofpacts, sizeof *ol_spec); + ol_spec->dst.field = mf_from_id(MFF_ETH_TYPE); + ol_spec->dst.ofs = 0; + ol_spec->dst.n_bits = ol_spec->dst.field->n_bits; + ol_spec->n_bits = ol_spec->dst.n_bits; + ol_spec->dst_type = NX_LEARN_DST_MATCH; + ol_spec->src_type = NX_LEARN_SRC_IMMEDIATE; + union mf_value imm_eth_type = { + .be16 = ipv6 ? htons(ETH_TYPE_IPV6) : htons(ETH_TYPE_IP) + }; + mf_write_subfield_value(&ol_spec->dst, &imm_eth_type, &match); + /* Push value last, as this may reallocate 'ol_spec'. */ + imm_bytes = DIV_ROUND_UP(ol_spec->dst.n_bits, 8); + src_imm = ofpbuf_put_zeros(ofpacts, OFPACT_ALIGN(imm_bytes)); + memcpy(src_imm, &imm_eth_type, imm_bytes); + + /* IP src. */ + ol_spec = ofpbuf_put_zeros(ofpacts, sizeof *ol_spec); + ol_spec->dst.field = + ipv6 ? mf_from_id(MFF_IPV6_SRC) : mf_from_id(MFF_IPV4_SRC); + ol_spec->src.field = + ipv6 ? mf_from_id(MFF_IPV6_SRC) : mf_from_id(MFF_IPV4_SRC); + ol_spec->dst.ofs = 0; + ol_spec->dst.n_bits = ol_spec->dst.field->n_bits; + ol_spec->n_bits = ol_spec->dst.n_bits; + ol_spec->dst_type = NX_LEARN_DST_MATCH; + ol_spec->src_type = NX_LEARN_SRC_FIELD; + + /* IP dst. */ + ol_spec = ofpbuf_put_zeros(ofpacts, sizeof *ol_spec); + ol_spec->dst.field = + ipv6 ? mf_from_id(MFF_IPV6_DST) : mf_from_id(MFF_IPV4_DST); + union mf_value imm_ip; + if (ipv6) { + imm_ip = (union mf_value) { + .ipv6 = lb_aff->vip, + }; + } else { + ovs_be32 ip4 = in6_addr_get_mapped_ipv4(&lb_aff->vip); + imm_ip = (union mf_value) { + .be32 = ip4, + }; + } + ol_spec->dst.ofs = 0; + ol_spec->dst.n_bits = ol_spec->dst.field->n_bits; + ol_spec->n_bits = ol_spec->dst.n_bits; + ol_spec->dst_type = NX_LEARN_DST_MATCH; + ol_spec->src_type = NX_LEARN_SRC_IMMEDIATE; + mf_write_subfield_value(&ol_spec->dst, &imm_ip, &match); + + /* Push value last, as this may reallocate 'ol_spec' */ + imm_bytes = DIV_ROUND_UP(ol_spec->dst.n_bits, 8); + src_imm = ofpbuf_put_zeros(ofpacts, OFPACT_ALIGN(imm_bytes)); + memcpy(src_imm, &imm_ip, imm_bytes); + + if (lb_aff->proto) { + /* IP proto. */ + union mf_value imm_proto = { + .u8 = lb_aff->proto, + }; + ol_spec = ofpbuf_put_zeros(ofpacts, sizeof *ol_spec); + ol_spec->dst.field = mf_from_id(MFF_IP_PROTO); + ol_spec->src.field = mf_from_id(MFF_IP_PROTO); + ol_spec->dst.ofs = 0; + ol_spec->dst.n_bits = ol_spec->dst.field->n_bits; + ol_spec->n_bits = ol_spec->dst.n_bits; + ol_spec->dst_type = NX_LEARN_DST_MATCH; + ol_spec->src_type = NX_LEARN_SRC_IMMEDIATE; + mf_write_subfield_value(&ol_spec->dst, &imm_proto, &match); + /* Push value last, as this may reallocate 'ol_spec' */ + imm_bytes = DIV_ROUND_UP(ol_spec->dst.n_bits, 8); + src_imm = ofpbuf_put_zeros(ofpacts, OFPACT_ALIGN(imm_bytes)); + memcpy(src_imm, &imm_proto, imm_bytes); + + /* dst port */ + ol_spec = ofpbuf_put_zeros(ofpacts, sizeof *ol_spec); + switch (lb_aff->proto) { + case IPPROTO_TCP: + ol_spec->dst.field = mf_from_id(MFF_TCP_DST); + ol_spec->src.field = mf_from_id(MFF_TCP_DST); + break; + case IPPROTO_UDP: + ol_spec->dst.field = mf_from_id(MFF_UDP_DST); + ol_spec->src.field = mf_from_id(MFF_UDP_DST); + break; + case IPPROTO_SCTP: + ol_spec->dst.field = mf_from_id(MFF_SCTP_DST); + ol_spec->src.field = mf_from_id(MFF_SCTP_DST); + break; + default: + OVS_NOT_REACHED(); + break; + } + ol_spec->dst.ofs = 0; + ol_spec->dst.n_bits = ol_spec->dst.field->n_bits; + ol_spec->n_bits = ol_spec->dst.n_bits; + ol_spec->dst_type = NX_LEARN_DST_MATCH; + ol_spec->src_type = NX_LEARN_SRC_IMMEDIATE; + /* Match on vip port. */ + union mf_value imm_vip_port = (union mf_value) { + .be16 = htons(lb_aff->vip_port), + }; + + mf_write_subfield_value(&ol_spec->dst, &imm_vip_port, &match); + /* Push value last, as this may reallocate 'ol_spec' */ + imm_bytes = DIV_ROUND_UP(ol_spec->dst.n_bits, 8); + src_imm = ofpbuf_put_zeros(ofpacts, OFPACT_ALIGN(imm_bytes)); + memcpy(src_imm, &imm_vip_port, imm_bytes); + } + + /* Set MLF_USE_LB_AFF_SESSION_BIT for ecmp replies. */ + ol_spec = ofpbuf_put_zeros(ofpacts, sizeof *ol_spec); + ol_spec->dst.field = mf_from_id(MFF_LOG_FLAGS); + ol_spec->dst.ofs = MLF_USE_LB_AFF_SESSION_BIT; + ol_spec->dst.n_bits = 1; + ol_spec->n_bits = ol_spec->dst.n_bits; + ol_spec->dst_type = NX_LEARN_DST_LOAD; + ol_spec->src_type = NX_LEARN_SRC_IMMEDIATE; + union mf_value imm_reg_value = { + .u8 = 1 + }; + mf_write_subfield_value(&ol_spec->dst, &imm_reg_value, &match); + + /* Push value last, as this may reallocate 'ol_spec' */ + imm_bytes = DIV_ROUND_UP(ol_spec->dst.n_bits, 8); + src_imm = ofpbuf_put_zeros(ofpacts, OFPACT_ALIGN(imm_bytes)); + ol = ofpacts->header; + memcpy(src_imm, &imm_reg_value, imm_bytes); + + /* Load backend IP in REG4/XXREG1. */ + union mf_value imm_backend_ip; + ol_spec = ofpbuf_put_zeros(ofpacts, sizeof *ol_spec); + + if (ipv6) { + imm_backend_ip = (union mf_value) { + .ipv6 = lb_aff->backend, + }; + if (ep->is_switch) { + ol_spec->dst.field = mf_from_id(MFF_LOG_LB_AFF_MATCH_LS_IP6_ADDR); + } else { + ol_spec->dst.field = mf_from_id(MFF_LOG_LB_AFF_MATCH_LR_IP6_ADDR); + } + } else { + ovs_be32 ip4 = in6_addr_get_mapped_ipv4(&lb_aff->backend); + imm_backend_ip = (union mf_value) { + .be32 = ip4, + }; + ol_spec->dst.field = mf_from_id(MFF_LOG_LB_AFF_MATCH_IP4_ADDR); + } + + ol_spec->dst_type = NX_LEARN_DST_LOAD; + ol_spec->src_type = NX_LEARN_SRC_IMMEDIATE; + ol_spec->dst.ofs = 0; + ol_spec->dst.n_bits = ol_spec->dst.field->n_bits; + ol_spec->n_bits = ol_spec->dst.n_bits; + mf_write_subfield_value(&ol_spec->dst, &imm_backend_ip, &match); + /* Push value last, as this may reallocate 'ol_spec' */ + imm_bytes = DIV_ROUND_UP(ol_spec->dst.n_bits, 8); + src_imm = ofpbuf_put_zeros(ofpacts, OFPACT_ALIGN(imm_bytes)); + memcpy(src_imm, &imm_backend_ip, imm_bytes); + + if (lb_aff->backend_port) { + /* Load backend port in REG8. */ + union mf_value imm_backend_port; + ol_spec = ofpbuf_put_zeros(ofpacts, sizeof *ol_spec); + imm_backend_port = (union mf_value) { + .be16 = htons(lb_aff->backend_port), + }; + + ol_spec->dst.field = mf_from_id(MFF_LOG_LB_AFF_MATCH_PORT); + ol_spec->dst_type = NX_LEARN_DST_LOAD; + ol_spec->src_type = NX_LEARN_SRC_IMMEDIATE; + ol_spec->dst.ofs = 0; + ol_spec->dst.n_bits = 8 * sizeof(lb_aff->backend_port); + ol_spec->n_bits = ol_spec->dst.n_bits; + mf_write_subfield_value(&ol_spec->dst, &imm_backend_port, &match); + /* Push value last, as this may reallocate 'ol_spec' */ + imm_bytes = DIV_ROUND_UP(ol_spec->dst.n_bits, 8); + src_imm = ofpbuf_put_zeros(ofpacts, OFPACT_ALIGN(imm_bytes)); + memcpy(src_imm, &imm_backend_port, imm_bytes); + } + + ol = ofpbuf_at_assert(ofpacts, ol_offset, sizeof *ol); + ofpact_finish_LEARN(ofpacts, &ol); +} + +static void +ovnact_commit_lb_aff_free(struct ovnact_commit_lb_aff *ecmp_nh OVS_UNUSED) +{ +} + /* Parses an assignment or exchange or put_dhcp_opts action. */ static void parse_set_action(struct action_context *ctx) @@ -4790,6 +5175,8 @@ parse_action(struct action_context *ctx) parse_put_fdb(ctx, ovnact_put_PUT_FDB(ctx->ovnacts)); } else if (lexer_match_id(ctx->lexer, "commit_ecmp_nh")) { parse_commit_ecmp_nh(ctx, ovnact_put_COMMIT_ECMP_NH(ctx->ovnacts)); + } else if (lexer_match_id(ctx->lexer, "commit_lb_aff")) { + parse_commit_lb_aff(ctx, ovnact_put_COMMIT_LB_AFF(ctx->ovnacts)); } else { lexer_syntax_error(ctx->lexer, "expecting action"); } diff --git a/ovn-sb.xml b/ovn-sb.xml index 315d60853..985e27fa9 100644 --- a/ovn-sb.xml +++ b/ovn-sb.xml @@ -2624,6 +2624,41 @@ tcp.flags = RST; register R is set to 1.

+ +
+ + commit_lb_aff(vip, backend, + proto, timeout); + +
+
+

+ Parameters: load-balancer virtual ip:port vip, + load-balancer backend ip:port backend, load-balancer + protocol proto, affinity timeout timeout. +

+ +

+ This action translates to an openflow "learn" action that inserts + a new flow in table 78. +

+ +
    +
  • + Match on the 4-tuple in table 78: nw_src=ip client, + nw_dst=vip ip, ip_proto, + tp_dst=vip port and set reg9[6] to 1, + reg4 and reg8 to backend ip and port + respectively. For IPv6 register xxreg1 is used to + store the backend ip. +
    • +
+ +

+ This action is applied for new connections received by a specific + load-balacer with affinity timeout configured. +

+
diff --git a/tests/ovn.at b/tests/ovn.at index f8b8db4df..966971e7a 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -2125,6 +2125,16 @@ reg9[5] = chk_ecmp_nh_mac(); reg9[5] = chk_ecmp_nh(); encodes as set_field:0/0x2000->reg10,resubmit(,77),move:NXM_NX_REG10[13]->OXM_OF_PKT_REG4[5] +# commit_lb_aff +commit_lb_aff(vip = "172.16.0.123:8080", backend = "10.0.0.3:8080", proto = tcp, timeout = 30); + encodes as learn(table=78,idle_timeout=30,delete_learned,OXM_OF_METADATA[],eth_type=0x800,NXM_OF_IP_SRC[],ip_dst=172.16.0.123,nw_proto=6,tcp_dst=8080,load:0x1->NXM_NX_REG10[14],load:0xa000003->NXM_NX_REG4[],load:0x1f90->NXM_NX_REG8[0..15]) + +commit_lb_aff(vip = "172.16.0.123", backend = "10.0.0.3", timeout = 30); + encodes as learn(table=78,idle_timeout=30,delete_learned,OXM_OF_METADATA[],eth_type=0x800,NXM_OF_IP_SRC[],ip_dst=172.16.0.123,load:0x1->NXM_NX_REG10[14],load:0xa000003->NXM_NX_REG4[]) + +commit_lb_aff(vip = "[::1]:8080", backend = "[::2]:8080", proto = tcp, timeout = 30); + encodes as learn(table=78,idle_timeout=30,delete_learned,OXM_OF_METADATA[],eth_type=0x86dd,NXM_NX_IPV6_SRC[],ipv6_dst=::1,nw_proto=6,tcp_dst=8080,load:0x1->NXM_NX_REG10[14],load:0x2->NXM_NX_XXREG0[],load:0x1f90->NXM_NX_REG8[0..15]) + # push/pop push(xxreg0);push(xxreg1[10..20]);push(eth.src);pop(xxreg0[0..47]);pop(xxreg0[48..57]);pop(xxreg1); formats as push(xxreg0); push(xxreg1[10..20]); push(eth.src); pop(xxreg0[0..47]); pop(xxreg0[48..57]); pop(xxreg1); diff --git a/utilities/ovn-trace.c b/utilities/ovn-trace.c index 6fa5137d9..8f6628646 100644 --- a/utilities/ovn-trace.c +++ b/utilities/ovn-trace.c @@ -3290,6 +3290,8 @@ trace_actions(const struct ovnact *ovnacts, size_t ovnacts_len, break; case OVNACT_CHK_ECMP_NH: break; + case OVNACT_COMMIT_LB_AFF: + break; } } ofpbuf_uninit(&stack); From patchwork Wed Nov 2 15:57:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Bianconi X-Patchwork-Id: 1698213 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=eA4Omo4e; dkim-atps=neutral Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N2Wkt0LPgz23l6 for ; Thu, 3 Nov 2022 02:58:14 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 1EC0C41706; Wed, 2 Nov 2022 15:58:12 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 1EC0C41706 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=eA4Omo4e X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2KIgHR2i5EMO; Wed, 2 Nov 2022 15:58:10 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 16E1F416F1; Wed, 2 Nov 2022 15:58:09 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 16E1F416F1 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id CC7D2C007B; Wed, 2 Nov 2022 15:58:08 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id E0046C0071 for ; Wed, 2 Nov 2022 15:58:07 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id A22FC81E68 for ; Wed, 2 Nov 2022 15:58:07 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org A22FC81E68 Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=eA4Omo4e X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Bc-rZ3FRpQGr for ; Wed, 2 Nov 2022 15:58:06 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org AE6D481E60 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id AE6D481E60 for ; Wed, 2 Nov 2022 15:58:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667404685; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NqXQCIDU6k9faXZNUYAm+gMCeJyvE1sB6OlW18YQ6f8=; b=eA4Omo4eVoDlaKqea3M/pMVykZ7gCgDd47ejaWRXkp7PQX05UpOEq/aL87MihHxYIHCO/d jz4vDreEFZ2B9u8iDaFj4K7O8PfiWD5hHHy/RcusQiC0iAI4Lx/ZBeRnjSCfG7pCg5UGPX JvzH4xb0A87sUvB60SN+WUOJfmw1hQQ= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-286-p50o9jQLOoKXJsXcYTSKOA-1; Wed, 02 Nov 2022 11:58:04 -0400 X-MC-Unique: p50o9jQLOoKXJsXcYTSKOA-1 Received: by mail-wm1-f71.google.com with SMTP id h204-20020a1c21d5000000b003cf4e055442so8079204wmh.1 for ; Wed, 02 Nov 2022 08:58:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NqXQCIDU6k9faXZNUYAm+gMCeJyvE1sB6OlW18YQ6f8=; b=nF+G3PfnFaDzZH+YPN4MZXa+OKo21fH2GvV5HWgs8375PjfJqhDeJ711CQVdf/QPEL wCFaBc752RYJ3PU1qBHgLMop4xi7qPfX7zja0IDxizYT0jSSSuSyftFHvBJnUD3KtKUD F0IOP0uTI82p+Lq9tBMF0P/E8EF4IEU4YhlNNB2yWLOzXTbIxscj1qDA2NN5QPtkvQ44 zn9HK/uEpebi5yJO6JsB8CDNzzbPRsj31n8oQq9UKD3gxjKPDD98iQw+ICN53/QOC8za hESlaGJxbWbPEdOT9QUbLf/eZpE6BY7mOsTFNpJs7BaQXYY6HxBlqpB2StXJE0C2tL6Q QG5A== X-Gm-Message-State: ACrzQf17JUfmojbpv7BTqM18pIagGjP24Vo31Kx1npmOaFmZ9d0I4pKZ JbJB3NFGQVBrrq2iLd+Jjxj2AVTRFpeEjbKBLZVA0xPPn4RxHCpD8m2xNt2GFfpgpGs74QaKilU My9fMwRRbL74me8rwkJ+iNieUjk+2Dd0H9BHtgSH6T+Qhh+ZjOzRzerJcHubCfyhVPdvlcM909j XTpS6r X-Received: by 2002:adf:eb51:0:b0:236:cfcc:76e4 with SMTP id u17-20020adfeb51000000b00236cfcc76e4mr10385468wrn.291.1667404681517; Wed, 02 Nov 2022 08:58:01 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6kJ0vSHNBudf6/3wr1BfZHmCIJYtO9yZhO7kxvNeRhyKzvap4TgBR977iqsM98GhzxYsmp9w== X-Received: by 2002:adf:eb51:0:b0:236:cfcc:76e4 with SMTP id u17-20020adfeb51000000b00236cfcc76e4mr10385449wrn.291.1667404681250; Wed, 02 Nov 2022 08:58:01 -0700 (PDT) Received: from localhost (net-188-216-77-84.cust.vodafonedsl.it. [188.216.77.84]) by smtp.gmail.com with ESMTPSA id ay19-20020a5d6f13000000b00236b2804d79sm13449254wrb.2.2022.11.02.08.58.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Nov 2022 08:58:00 -0700 (PDT) From: Lorenzo Bianconi To: ovs-dev@openvswitch.org Date: Wed, 2 Nov 2022 16:57:48 +0100 Message-Id: <85e869fd9f14412fcf7551df0503d95215393aec.1667404277.git.lorenzo.bianconi@redhat.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: References: MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: dceara@redhat.com Subject: [ovs-dev] [PATCH v6 ovn 2/3] actions: introduce chk_lb_aff action X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" chk_lb_aff action checks if the packet under consideration matches the new flow in table OFTABLE_CHK_LB_AFFINITY. If so, then the 1-bit destination register is set to 1. chk_lb_aff will be used to add affinity timeout support to load balancer connections. Acked-by: Mark Michelson Signed-off-by: Lorenzo Bianconi --- include/ovn/actions.h | 1 + lib/actions.c | 27 +++++++++++++++++++++++++++ ovn-sb.xml | 9 +++++++++ tests/ovn.at | 4 ++++ utilities/ovn-trace.c | 2 ++ 5 files changed, 43 insertions(+) diff --git a/include/ovn/actions.h b/include/ovn/actions.h index 597cbb8e3..fdb6ab08b 100644 --- a/include/ovn/actions.h +++ b/include/ovn/actions.h @@ -122,6 +122,7 @@ struct ovn_extend_table; OVNACT(CHK_ECMP_NH_MAC, ovnact_result) \ OVNACT(CHK_ECMP_NH, ovnact_result) \ OVNACT(COMMIT_LB_AFF, ovnact_commit_lb_aff) \ + OVNACT(CHK_LB_AFF, ovnact_result) \ /* enum ovnact_type, with a member OVNACT_ for each action. */ enum OVS_PACKED_ENUM ovnact_type { diff --git a/lib/actions.c b/lib/actions.c index de9c69860..96e6ddea6 100644 --- a/lib/actions.c +++ b/lib/actions.c @@ -4985,6 +4985,29 @@ ovnact_commit_lb_aff_free(struct ovnact_commit_lb_aff *ecmp_nh OVS_UNUSED) { } +static void +parse_chk_lb_aff(struct action_context *ctx, const struct expr_field *dst, + struct ovnact_result *res) +{ + parse_ovnact_result(ctx, "chk_lb_aff", NULL, dst, res); +} + +static void +format_CHK_LB_AFF(const struct ovnact_result *res, struct ds *s) +{ + expr_field_format(&res->dst, s); + ds_put_cstr(s, " = chk_lb_aff();"); +} + +static void +encode_CHK_LB_AFF(const struct ovnact_result *res, + const struct ovnact_encode_params *ep OVS_UNUSED, + struct ofpbuf *ofpacts) +{ + encode_result_action__(res, OFTABLE_CHK_LB_AFFINITY, + MLF_USE_LB_AFF_SESSION_BIT, ofpacts); +} + /* Parses an assignment or exchange or put_dhcp_opts action. */ static void parse_set_action(struct action_context *ctx) @@ -5069,6 +5092,10 @@ parse_set_action(struct action_context *ctx) && lexer_lookahead(ctx->lexer) == LEX_T_LPAREN) { parse_chk_ecmp_nh(ctx, &lhs, ovnact_put_CHK_ECMP_NH(ctx->ovnacts)); + } else if (!strcmp(ctx->lexer->token.s, "chk_lb_aff") && + lexer_lookahead(ctx->lexer) == LEX_T_LPAREN) { + parse_chk_lb_aff(ctx, &lhs, + ovnact_put_CHK_LB_AFF(ctx->ovnacts)); } else { parse_assignment_action(ctx, false, &lhs); } diff --git a/ovn-sb.xml b/ovn-sb.xml index 985e27fa9..c20c7a7a4 100644 --- a/ovn-sb.xml +++ b/ovn-sb.xml @@ -2659,6 +2659,15 @@ tcp.flags = RST; load-balacer with affinity timeout configured.

+ +
R = chk_lb_aff();
+
+

+ This action checks if the packet under consideration matches any + flow in table 78. If it is so, then the 1-bit destination + register R is set to 1. +

+
diff --git a/tests/ovn.at b/tests/ovn.at index 966971e7a..f90bd02bb 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -2135,6 +2135,10 @@ commit_lb_aff(vip = "172.16.0.123", backend = "10.0.0.3", timeout = 30); commit_lb_aff(vip = "[::1]:8080", backend = "[::2]:8080", proto = tcp, timeout = 30); encodes as learn(table=78,idle_timeout=30,delete_learned,OXM_OF_METADATA[],eth_type=0x86dd,NXM_NX_IPV6_SRC[],ipv6_dst=::1,nw_proto=6,tcp_dst=8080,load:0x1->NXM_NX_REG10[14],load:0x2->NXM_NX_XXREG0[],load:0x1f90->NXM_NX_REG8[0..15]) +# chk_lb_aff() +reg9[6] = chk_lb_aff(); + encodes as set_field:0/0x4000->reg10,resubmit(,78),move:NXM_NX_REG10[14]->OXM_OF_PKT_REG4[6] + # push/pop push(xxreg0);push(xxreg1[10..20]);push(eth.src);pop(xxreg0[0..47]);pop(xxreg0[48..57]);pop(xxreg1); formats as push(xxreg0); push(xxreg1[10..20]); push(eth.src); pop(xxreg0[0..47]); pop(xxreg0[48..57]); pop(xxreg1); diff --git a/utilities/ovn-trace.c b/utilities/ovn-trace.c index 8f6628646..8e3a8d9ca 100644 --- a/utilities/ovn-trace.c +++ b/utilities/ovn-trace.c @@ -3292,6 +3292,8 @@ trace_actions(const struct ovnact *ovnacts, size_t ovnacts_len, break; case OVNACT_COMMIT_LB_AFF: break; + case OVNACT_CHK_LB_AFF: + break; } } ofpbuf_uninit(&stack); From patchwork Wed Nov 2 15:57:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Bianconi X-Patchwork-Id: 1698216 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=CjLIxot4; dkim-atps=neutral Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N2WlP0RkCz23l6 for ; Thu, 3 Nov 2022 02:58:40 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id C33CC60F63; Wed, 2 Nov 2022 15:58:38 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org C33CC60F63 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=CjLIxot4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nPnRQBLWAvug; Wed, 2 Nov 2022 15:58:35 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id EB95360F6F; Wed, 2 Nov 2022 15:58:34 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org EB95360F6F Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C7C7AC0032; Wed, 2 Nov 2022 15:58:34 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 17547C0032 for ; Wed, 2 Nov 2022 15:58:33 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 6860D81E61 for ; Wed, 2 Nov 2022 15:58:17 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 6860D81E61 Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=CjLIxot4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gjJMKjgcKhXc for ; Wed, 2 Nov 2022 15:58:15 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org BE04F81E60 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id BE04F81E60 for ; Wed, 2 Nov 2022 15:58:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667404693; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kH6WZs4hQ4+sn3WOL6CTZji7MIthxynDqmCnToTe7bk=; b=CjLIxot4YgZOrPa9/DQ/X28KoMvoO+xX25FIK8RPtooEgR+Vf1LOXzvAu2C4680H5pp1T1 7MR3Xaj/qhDuFjzFPeb7nvVXumbQt9GyRyzd5qbPaUINZnONxgnRZPGZLrFMPJxOxsq4aw WpjxfZACme70cMoUWmeVz1jeG79z+kw= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-211-mJ0-ODVVN9i7ExQPRMq8Dg-1; Wed, 02 Nov 2022 11:58:10 -0400 X-MC-Unique: mJ0-ODVVN9i7ExQPRMq8Dg-1 Received: by mail-wr1-f71.google.com with SMTP id g4-20020adfbc84000000b0022fc417f87cso5037470wrh.12 for ; Wed, 02 Nov 2022 08:58:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kH6WZs4hQ4+sn3WOL6CTZji7MIthxynDqmCnToTe7bk=; b=YQPZo4mDst7OgjCIL9fCwRasfORhrlR9CWMBmVLi5LBslMmH5RaI5NkIV8w2NTTLJv aUgOvTKIehH56yfzrfKd0nM1WzJNh/Ra8+xUhXtevhdoZkEzfUYKqMMgN+wovJwlKM6E 0SDkX7AHQj176miqHIdFkymmN+SMTDMlMKjAvXB91OSbDYtPNQ13U500EwbaEF7sVS4t A15xDcRzg2aP5zFb1mDydFGwCCT21Feztlzm3Fhv5+0B3HaiZK5HWqzyBLqEfR5kHZra dZXmWR9lN+L/1h0IFcBN+zOWqyAZv/w5mVVIoqxoY0WW1GJS7vx796FY2rFK2TczoV/9 hIHg== X-Gm-Message-State: ACrzQf0fyzjGetwZnKCgh2nCRt/McUz21aTCjZd5NbBXcNiHKnXG/c9a C6YUAF/YP/p6uAQanT6PWH4n5a+GeYfHnZQBK/Vu2h7M22/UfjuF4zpHVhDm4b6AUog2ff3BlEV TdF19jrttz4BH/IWVaU901mzWgy3JzsGNopSjtoA/xhMq9aWGGfi+uMn98okAk+KaexPTTR4ctz 5Tkswx X-Received: by 2002:a05:600c:1f17:b0:3ce:1f81:d9fa with SMTP id bd23-20020a05600c1f1700b003ce1f81d9famr16403193wmb.138.1667404686900; Wed, 02 Nov 2022 08:58:06 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6KT1cI26JyOFnr3lJIFs6YTPy8lPxHFMPPkAV6BJdzYGr9SDzr+3+RBlmjrJ8i/gGBk44eSw== X-Received: by 2002:a05:600c:1f17:b0:3ce:1f81:d9fa with SMTP id bd23-20020a05600c1f1700b003ce1f81d9famr16403004wmb.138.1667404683683; Wed, 02 Nov 2022 08:58:03 -0700 (PDT) Received: from localhost (net-188-216-77-84.cust.vodafonedsl.it. [188.216.77.84]) by smtp.gmail.com with ESMTPSA id c4-20020a05600c0a4400b003cf894c05e4sm681883wmq.22.2022.11.02.08.58.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Nov 2022 08:58:03 -0700 (PDT) From: Lorenzo Bianconi To: ovs-dev@openvswitch.org Date: Wed, 2 Nov 2022 16:57:49 +0100 Message-Id: X-Mailer: git-send-email 2.38.1 In-Reply-To: References: MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: dceara@redhat.com Subject: [ovs-dev] [PATCH v6 ovn 3/3] northd: rely on new actions for lb affinity X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Rely on the following new actions in order to introduce affinity timeout support to load-balancer sessions: - commit_lb_aff - chk_lb_aff Introduce the following tables in switch and router pipelines respectively: - S_SWITCH_IN_LB_AFF_CHECK - S_SWITCH_IN_LB_AFF_LEARN - S_ROUTER_IN_LB_AFF_CHECK - S_ROUTER_IN_LB_AFF_LEARN In this way OVN is able to dnat connections received from the same client to a given load-balancer to the same backend if received in the affinity timeslot. Please note load-balancer affinity support has scalability limitation since we need to create a new logical flow per backend. Acked-by: Mark Michelson Signed-off-by: Lorenzo Bianconi --- lib/lb.c | 10 + lib/lb.h | 1 + northd/northd.c | 390 +++++++++++++++++++++++++-------- northd/ovn-northd.8.xml | 208 +++++++++++++++--- ovn-nb.xml | 7 + tests/ovn-northd.at | 346 ++++++++++++++++++------------ tests/ovn.at | 54 ++--- tests/system-ovn.at | 465 +++++++++++++++++++++++++++++++++++++++- 8 files changed, 1198 insertions(+), 283 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index ab5de38a8..c08ccceda 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -208,6 +208,16 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) smap_get_def(&nbrec_lb->options, "neighbor_responder", "reachable"); lb->neigh_mode = strcmp(mode, "all") ? LB_NEIGH_RESPOND_REACHABLE : LB_NEIGH_RESPOND_ALL; + uint32_t affinity_timeout = + smap_get_uint(&nbrec_lb->options, "affinity_timeout", 0); + if (affinity_timeout > UINT16_MAX) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "max affinity_timeout timeout value is %u", + UINT16_MAX); + affinity_timeout = UINT16_MAX; + } + lb->affinity_timeout = affinity_timeout; + sset_init(&lb->ips_v4); sset_init(&lb->ips_v6); struct smap_node *node; diff --git a/lib/lb.h b/lib/lb.h index c1aadd6dd..62843e471 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -67,6 +67,7 @@ struct ovn_northd_lb { bool controller_event; bool routable; bool skip_snat; + uint16_t affinity_timeout; struct sset ips_v4; struct sset ips_v6; diff --git a/northd/northd.c b/northd/northd.c index b7388afc5..7011d01ca 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -121,20 +121,22 @@ enum ovn_stage { PIPELINE_STAGE(SWITCH, IN, ACL, 8, "ls_in_acl") \ PIPELINE_STAGE(SWITCH, IN, QOS_MARK, 9, "ls_in_qos_mark") \ PIPELINE_STAGE(SWITCH, IN, QOS_METER, 10, "ls_in_qos_meter") \ - PIPELINE_STAGE(SWITCH, IN, LB, 11, "ls_in_lb") \ - PIPELINE_STAGE(SWITCH, IN, ACL_AFTER_LB, 12, "ls_in_acl_after_lb") \ - PIPELINE_STAGE(SWITCH, IN, STATEFUL, 13, "ls_in_stateful") \ - PIPELINE_STAGE(SWITCH, IN, PRE_HAIRPIN, 14, "ls_in_pre_hairpin") \ - PIPELINE_STAGE(SWITCH, IN, NAT_HAIRPIN, 15, "ls_in_nat_hairpin") \ - PIPELINE_STAGE(SWITCH, IN, HAIRPIN, 16, "ls_in_hairpin") \ - PIPELINE_STAGE(SWITCH, IN, ARP_ND_RSP, 17, "ls_in_arp_rsp") \ - PIPELINE_STAGE(SWITCH, IN, DHCP_OPTIONS, 18, "ls_in_dhcp_options") \ - PIPELINE_STAGE(SWITCH, IN, DHCP_RESPONSE, 19, "ls_in_dhcp_response") \ - PIPELINE_STAGE(SWITCH, IN, DNS_LOOKUP, 20, "ls_in_dns_lookup") \ - PIPELINE_STAGE(SWITCH, IN, DNS_RESPONSE, 21, "ls_in_dns_response") \ - PIPELINE_STAGE(SWITCH, IN, EXTERNAL_PORT, 22, "ls_in_external_port") \ - PIPELINE_STAGE(SWITCH, IN, L2_LKUP, 23, "ls_in_l2_lkup") \ - PIPELINE_STAGE(SWITCH, IN, L2_UNKNOWN, 24, "ls_in_l2_unknown") \ + PIPELINE_STAGE(SWITCH, IN, LB_AFF_CHECK, 11, "ls_in_lb_aff_check") \ + PIPELINE_STAGE(SWITCH, IN, LB, 12, "ls_in_lb") \ + PIPELINE_STAGE(SWITCH, IN, LB_AFF_LEARN, 13, "ls_in_lb_aff_learn") \ + PIPELINE_STAGE(SWITCH, IN, ACL_AFTER_LB, 14, "ls_in_acl_after_lb") \ + PIPELINE_STAGE(SWITCH, IN, STATEFUL, 15, "ls_in_stateful") \ + PIPELINE_STAGE(SWITCH, IN, PRE_HAIRPIN, 16, "ls_in_pre_hairpin") \ + PIPELINE_STAGE(SWITCH, IN, NAT_HAIRPIN, 17, "ls_in_nat_hairpin") \ + PIPELINE_STAGE(SWITCH, IN, HAIRPIN, 18, "ls_in_hairpin") \ + PIPELINE_STAGE(SWITCH, IN, ARP_ND_RSP, 19, "ls_in_arp_rsp") \ + PIPELINE_STAGE(SWITCH, IN, DHCP_OPTIONS, 20, "ls_in_dhcp_options") \ + PIPELINE_STAGE(SWITCH, IN, DHCP_RESPONSE, 21, "ls_in_dhcp_response") \ + PIPELINE_STAGE(SWITCH, IN, DNS_LOOKUP, 22, "ls_in_dns_lookup") \ + PIPELINE_STAGE(SWITCH, IN, DNS_RESPONSE, 23, "ls_in_dns_response") \ + PIPELINE_STAGE(SWITCH, IN, EXTERNAL_PORT, 24, "ls_in_external_port") \ + PIPELINE_STAGE(SWITCH, IN, L2_LKUP, 25, "ls_in_l2_lkup") \ + PIPELINE_STAGE(SWITCH, IN, L2_UNKNOWN, 26, "ls_in_l2_unknown") \ \ /* Logical switch egress stages. */ \ PIPELINE_STAGE(SWITCH, OUT, PRE_LB, 0, "ls_out_pre_lb") \ @@ -155,20 +157,22 @@ enum ovn_stage { PIPELINE_STAGE(ROUTER, IN, IP_INPUT, 3, "lr_in_ip_input") \ PIPELINE_STAGE(ROUTER, IN, UNSNAT, 4, "lr_in_unsnat") \ PIPELINE_STAGE(ROUTER, IN, DEFRAG, 5, "lr_in_defrag") \ - PIPELINE_STAGE(ROUTER, IN, DNAT, 6, "lr_in_dnat") \ - PIPELINE_STAGE(ROUTER, IN, ECMP_STATEFUL, 7, "lr_in_ecmp_stateful") \ - PIPELINE_STAGE(ROUTER, IN, ND_RA_OPTIONS, 8, "lr_in_nd_ra_options") \ - PIPELINE_STAGE(ROUTER, IN, ND_RA_RESPONSE, 9, "lr_in_nd_ra_response") \ - PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_PRE, 10, "lr_in_ip_routing_pre") \ - PIPELINE_STAGE(ROUTER, IN, IP_ROUTING, 11, "lr_in_ip_routing") \ - PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_ECMP, 12, "lr_in_ip_routing_ecmp") \ - PIPELINE_STAGE(ROUTER, IN, POLICY, 13, "lr_in_policy") \ - PIPELINE_STAGE(ROUTER, IN, POLICY_ECMP, 14, "lr_in_policy_ecmp") \ - PIPELINE_STAGE(ROUTER, IN, ARP_RESOLVE, 15, "lr_in_arp_resolve") \ - PIPELINE_STAGE(ROUTER, IN, CHK_PKT_LEN, 16, "lr_in_chk_pkt_len") \ - PIPELINE_STAGE(ROUTER, IN, LARGER_PKTS, 17, "lr_in_larger_pkts") \ - PIPELINE_STAGE(ROUTER, IN, GW_REDIRECT, 18, "lr_in_gw_redirect") \ - PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 19, "lr_in_arp_request") \ + PIPELINE_STAGE(ROUTER, IN, LB_AFF_CHECK, 6, "lr_in_lb_aff_check") \ + PIPELINE_STAGE(ROUTER, IN, DNAT, 7, "lr_in_dnat") \ + PIPELINE_STAGE(ROUTER, IN, LB_AFF_LEARN, 8, "lr_in_lb_aff_learn") \ + PIPELINE_STAGE(ROUTER, IN, ECMP_STATEFUL, 9, "lr_in_ecmp_stateful") \ + PIPELINE_STAGE(ROUTER, IN, ND_RA_OPTIONS, 10, "lr_in_nd_ra_options") \ + PIPELINE_STAGE(ROUTER, IN, ND_RA_RESPONSE, 11, "lr_in_nd_ra_response") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_PRE, 12, "lr_in_ip_routing_pre") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING, 13, "lr_in_ip_routing") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_ECMP, 14, "lr_in_ip_routing_ecmp") \ + PIPELINE_STAGE(ROUTER, IN, POLICY, 15, "lr_in_policy") \ + PIPELINE_STAGE(ROUTER, IN, POLICY_ECMP, 16, "lr_in_policy_ecmp") \ + PIPELINE_STAGE(ROUTER, IN, ARP_RESOLVE, 17, "lr_in_arp_resolve") \ + PIPELINE_STAGE(ROUTER, IN, CHK_PKT_LEN, 18, "lr_in_chk_pkt_len") \ + PIPELINE_STAGE(ROUTER, IN, LARGER_PKTS, 19, "lr_in_larger_pkts") \ + PIPELINE_STAGE(ROUTER, IN, GW_REDIRECT, 20, "lr_in_gw_redirect") \ + PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 21, "lr_in_arp_request") \ \ /* Logical router egress stages. */ \ PIPELINE_STAGE(ROUTER, OUT, CHECK_DNAT_LOCAL, 0, \ @@ -215,8 +219,17 @@ enum ovn_stage { #define REG_ORIG_DIP_IPV6 "xxreg1" #define REG_ORIG_TP_DPORT "reg2[0..15]" +/* Register used to store backend ipv6 address + * for load balancer affinity. */ +#define REG_LB_L2_AFF_BACKEND_IP6 "xxreg0" + /* Register definitions for switches and routers. */ +/* Register used to store backend ipv4 address + * for load balancer affinity. */ +#define REG_LB_AFF_BACKEND_IP4 "reg4" +#define REG_LB_AFF_MATCH_PORT "reg8[0..15]" + /* Indicate that this packet has been recirculated using egress * loopback. This allows certain checks to be bypassed, such as a * logical router dropping packets with source IP address equals @@ -228,6 +241,8 @@ enum ovn_stage { #define REGBIT_LOOKUP_NEIGHBOR_IP_RESULT "reg9[3]" #define REGBIT_DST_NAT_IP_LOCAL "reg9[4]" #define REGBIT_KNOWN_ECMP_NH "reg9[5]" +#define REGBIT_KNOWN_LB_SESSION "reg9[6]" +#define REG /* Register to store the eth address associated to a router port for packets * received in S_ROUTER_IN_ADMISSION. @@ -245,6 +260,10 @@ enum ovn_stage { #define REG_SRC_IPV6 "xxreg1" #define REG_ROUTE_TABLE_ID "reg7" +/* Register used to store backend ipv6 address + * for load balancer affinity. */ +#define REG_LB_L3_AFF_BACKEND_IP6 "xxreg1" + #define REG_ORIG_TP_DPORT_ROUTER "reg9[16..31]" /* Register used for setting a label for ACLs in a Logical Switch. */ @@ -267,73 +286,75 @@ enum ovn_stage { * OVS register usage: * * Logical Switch pipeline: - * +----+----------------------------------------------+---+------------------+ - * | R0 | REGBIT_{CONNTRACK/DHCP/DNS} | | | - * | | REGBIT_{HAIRPIN/HAIRPIN_REPLY} | | | - * | | REGBIT_ACL_HINT_{ALLOW_NEW/ALLOW/DROP/BLOCK} | | | - * | | REGBIT_ACL_LABEL | X | | - * +----+----------------------------------------------+ X | | - * | R1 | ORIG_DIP_IPV4 (>= IN_PRE_STATEFUL) | R | | - * +----+----------------------------------------------+ E | | - * | R2 | ORIG_TP_DPORT (>= IN_PRE_STATEFUL) | G | | - * +----+----------------------------------------------+ 0 | | - * | R3 | ACL LABEL | | | - * +----+----------------------------------------------+---+------------------+ - * | R4 | UNUSED | | | - * +----+----------------------------------------------+ X | ORIG_DIP_IPV6(>= | - * | R5 | UNUSED | X | IN_PRE_STATEFUL) | - * +----+----------------------------------------------+ R | | - * | R6 | UNUSED | E | | - * +----+----------------------------------------------+ G | | - * | R7 | UNUSED | 1 | | - * +----+----------------------------------------------+---+------------------+ - * | R8 | UNUSED | + * +----+----------------------------------------------+---+-----------------------------------+ + * | R0 | REGBIT_{CONNTRACK/DHCP/DNS} | | | + * | | REGBIT_{HAIRPIN/HAIRPIN_REPLY} | | | + * | | REGBIT_ACL_HINT_{ALLOW_NEW/ALLOW/DROP/BLOCK} | | | + * | | REGBIT_ACL_LABEL | X | | + * +----+----------------------------------------------+ X | | + * | R5 | UNUSED | X | LB_L2_AFF_BACKEND_IP6 | + * | R1 | ORIG_DIP_IPV4 (>= IN_PRE_STATEFUL) | R | | + * +----+----------------------------------------------+ E | | + * | R2 | ORIG_TP_DPORT (>= IN_PRE_STATEFUL) | G | | + * +----+----------------------------------------------+ 0 | | + * | R3 | ACL LABEL | | | + * +----+----------------------------------------------+---+-----------------------------------+ + * | R4 | REG_LB_AFF_BACKEND_IP4 | | | + * +----+----------------------------------------------+ X | | + * | R5 | UNUSED | X | ORIG_DIP_IPV6(>= IN_PRE_STATEFUL) | + * +----+----------------------------------------------+ R | | + * | R6 | UNUSED | E | | + * +----+----------------------------------------------+ G | | + * | R7 | UNUSED | 1 | | + * +----+----------------------------------------------+---+-----------------------------------+ + * | R8 | LB_AFF_MATCH_PORT | * +----+----------------------------------------------+ * | R9 | UNUSED | * +----+----------------------------------------------+ * * Logical Router pipeline: - * +-----+--------------------------+---+-----------------+---+---------------+ - * | R0 | REGBIT_ND_RA_OPTS_RESULT | | | | | - * | | (= IN_ND_RA_OPTIONS) | X | | | | - * | | NEXT_HOP_IPV4 | R | | | | - * | | (>= IP_INPUT) | E | INPORT_ETH_ADDR | X | | - * +-----+--------------------------+ G | (< IP_INPUT) | X | | - * | R1 | SRC_IPV4 for ARP-REQ | 0 | | R | | - * | | (>= IP_INPUT) | | | E | NEXT_HOP_IPV6 | - * +-----+--------------------------+---+-----------------+ G | ( >= DEFRAG ) | - * | R2 | UNUSED | X | | 0 | | - * | | | R | | | | - * +-----+--------------------------+ E | UNUSED | | | - * | R3 | UNUSED | G | | | | - * | | | 1 | | | | - * +-----+--------------------------+---+-----------------+---+---------------+ - * | R4 | UNUSED | X | | | | - * | | | R | | | | - * +-----+--------------------------+ E | UNUSED | X | | - * | R5 | UNUSED | G | | X | | - * | | | 2 | | R |SRC_IPV6 for NS| - * +-----+--------------------------+---+-----------------+ E | ( >= | - * | R6 | UNUSED | X | | G | IN_IP_ROUTING)| - * | | | R | | 1 | | - * +-----+--------------------------+ E | UNUSED | | | - * | R7 | ROUTE_TABLE_ID | G | | | | - * | | (>= IN_IP_ROUTING_PRE && | 3 | | | | - * | | <= IN_IP_ROUTING) | | | | | - * +-----+--------------------------+---+-----------------+---+---------------+ - * | R8 | ECMP_GROUP_ID | | | - * | | ECMP_MEMBER_ID | X | | - * +-----+--------------------------+ R | | - * | | REGBIT_{ | E | | - * | | EGRESS_LOOPBACK/ | G | UNUSED | - * | R9 | PKT_LARGER/ | 4 | | - * | | LOOKUP_NEIGHBOR_RESULT/| | | - * | | SKIP_LOOKUP_NEIGHBOR/ | | | - * | | KNOWN_ECMP_NH} | | | - * | | | | | - * | | REG_ORIG_TP_DPORT_ROUTER | | | - * | | | | | - * +-----+--------------------------+---+-----------------+ + * +-----+---------------------------+---+-----------------+---+------------------------------------+ + * | R0 | REGBIT_ND_RA_OPTS_RESULT | | | | | + * | | (= IN_ND_RA_OPTIONS) | X | | | | + * | | NEXT_HOP_IPV4 | R | | | | + * | | (>= IP_INPUT) | E | INPORT_ETH_ADDR | X | | + * +-----+---------------------------+ G | (< IP_INPUT) | X | | + * | R1 | SRC_IPV4 for ARP-REQ | 0 | | R | | + * | | (>= IP_INPUT) | | | E | NEXT_HOP_IPV6 (>= DEFRAG ) | + * +-----+---------------------------+---+-----------------+ G | | + * | R2 | UNUSED | X | | 0 | | + * | | | R | | | | + * +-----+---------------------------+ E | UNUSED | | | + * | R3 | UNUSED | G | | | | + * | | | 1 | | | | + * +-----+---------------------------+---+-----------------+---+------------------------------------+ + * | R4 | REG_LB_AFF_BACKEND_IP4 | X | | | | + * | | | R | | | | + * +-----+---------------------------+ E | UNUSED | X | | + * | R5 | UNUSED | G | | X | | + * | | | 2 | | R | LB_L3_AFF_BACKEND_IP6 | + * +-----+---------------------------+---+-----------------+ E | (<= IN_DNAT) | + * | R6 | UNUSED | X | | G | | + * | | | R | | 1 | | + * +-----+---------------------------+ E | UNUSED | | | + * | R7 | ROUTE_TABLE_ID | G | | | | + * | | (>= IN_IP_ROUTING_PRE && | 3 | | | | + * | | <= IN_IP_ROUTING) | | | | | + * +-----+---------------------------+---+-----------------+---+------------------------------------+ + * | R8 | ECMP_GROUP_ID | | | + * | | ECMP_MEMBER_ID | | | + * | | LB_AFF_MATCH_PORT | X | | + * +-----+---------------------------+ R | | + * | | REGBIT_{ | E | | + * | | EGRESS_LOOPBACK/ | G | UNUSED | + * | R9 | PKT_LARGER/ | 4 | | + * | | LOOKUP_NEIGHBOR_RESULT/ | | | + * | | SKIP_LOOKUP_NEIGHBOR/ | | | + * | | KNOWN_ECMP_NH} | | | + * | | | | | + * | | REG_ORIG_TP_DPORT_ROUTER | | | + * | | | | | + * +-----+---------------------------+---+-----------------+ * */ @@ -6943,6 +6964,190 @@ build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, } } +static void +build_lb_affinity_flows(struct hmap *lflows, struct ovn_northd_lb *lb, + struct ovn_lb_vip *lb_vip, char *match, + bool router_pipeline) +{ + if (!lb->affinity_timeout) { + return; + } + + enum ovn_stage stage0 = router_pipeline ? + S_ROUTER_IN_LB_AFF_CHECK : S_SWITCH_IN_LB_AFF_CHECK; + struct ovn_lflow *lflow_ref_aff_check = NULL; + /* Check if we have already a enstablished connection for this + * tuple and we are in affinity timeslot. */ + uint32_t hash_aff_check = ovn_logical_flow_hash( + ovn_stage_get_table(stage0), ovn_stage_get_pipeline(stage0), 100, + match, REGBIT_KNOWN_LB_SESSION" = chk_lb_aff(); next;"); + + size_t n_dp = router_pipeline ? lb->n_nb_lr : lb->n_nb_ls; + for (size_t i = 0; i < n_dp; i++) { + struct ovn_datapath *od = router_pipeline + ? lb->nb_lr[i] : lb->nb_ls[i]; + if (!ovn_dp_group_add_with_reference(lflow_ref_aff_check, od)) { + lflow_ref_aff_check = ovn_lflow_add_at_with_hash( + lflows, od, stage0, 100, match, + REGBIT_KNOWN_LB_SESSION" = chk_lb_aff(); next;", + NULL, NULL, &lb->nlb->header_, + OVS_SOURCE_LOCATOR, hash_aff_check); + } + } + + struct ds aff_action_learn = DS_EMPTY_INITIALIZER; + struct ds aff_match_learn = DS_EMPTY_INITIALIZER; + struct ds aff_action_lb = DS_EMPTY_INITIALIZER; + struct ds aff_match = DS_EMPTY_INITIALIZER; + + stage0 = router_pipeline + ? S_ROUTER_IN_LB_AFF_LEARN : S_SWITCH_IN_LB_AFF_LEARN; + enum ovn_stage stage1 = router_pipeline + ? S_ROUTER_IN_DNAT : S_SWITCH_IN_LB; + bool ipv6 = !IN6_IS_ADDR_V4MAPPED(&lb_vip->vip); + for (size_t i = 0; i < lb_vip->n_backends; i++) { + struct ovn_lb_backend *backend = &lb_vip->backends[i]; + + /* Forward to OFTABLE_CHK_LB_AFFINITY table to store flow tuple. */ + ds_put_format(&aff_match_learn, "ct.new && %s.dst == %s", + ipv6 ? "ip6" : "ip4", backend->ip_str); + if (backend->port) { + ds_put_format(&aff_match_learn, " && %s.dst == %d", + lb->proto, backend->port); + } + + if (lb_vip->vip_port) { + ds_put_format(&aff_action_learn, + "commit_lb_aff(vip = \"%s%s%s:%d\"", + ipv6 ? "[" : "", lb_vip->vip_str, ipv6 ? "]" : "", + lb_vip->vip_port); + } else { + ds_put_format(&aff_action_learn, "commit_lb_aff(vip = \"%s\"", + lb_vip->vip_str); + } + + if (backend->port) { + ds_put_format(&aff_action_learn,", backend = \"%s%s%s:%d\"", + ipv6 ? "[" : "", backend->ip_str, + ipv6 ? "]" : "", backend->port); + } else { + ds_put_format(&aff_action_learn,", backend = \"%s\"", + backend->ip_str); + } + + if (lb_vip->vip_port) { + ds_put_format(&aff_action_learn, ", proto = %s", lb->proto); + } + + ds_put_format(&aff_action_learn, ", timeout = %d); next;", + lb->affinity_timeout); + + struct ovn_lflow *lflow_ref_aff_learn = NULL; + uint32_t hash_aff_learn = ovn_logical_flow_hash( + ovn_stage_get_table(stage0), ovn_stage_get_pipeline(stage0), + 100, ds_cstr(&aff_match_learn), ds_cstr(&aff_action_learn)); + + /* Use already selected backend within affinity + * timeslot. */ + if (backend->port) { + ds_put_format(&aff_match, + REGBIT_KNOWN_LB_SESSION" == 1 && %s && %s == %s " + "&& "REG_LB_AFF_MATCH_PORT" == %d", + IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) ? "ip4" : "ip6", + IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) + ? REG_LB_AFF_BACKEND_IP4 + : router_pipeline + ? REG_LB_L3_AFF_BACKEND_IP6 + : REG_LB_L2_AFF_BACKEND_IP6, + backend->ip_str, backend->port); + ds_put_format(&aff_action_lb, "ct_lb_mark(backends=%s%s%s:%d);", + ipv6 ? "[" : "", backend->ip_str, + ipv6 ? "]" : "", backend->port); + } else { + ds_put_format(&aff_match, + REGBIT_KNOWN_LB_SESSION" == 1 && %s && %s == %s", + IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) ? "ip4" : "ip6", + IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) + ? REG_LB_AFF_BACKEND_IP4 + : router_pipeline + ? REG_LB_L3_AFF_BACKEND_IP6 + : REG_LB_L2_AFF_BACKEND_IP6, + backend->ip_str); + ds_put_format(&aff_action_lb, "ct_lb_mark(backends=%s);", + backend->ip_str); + } + + struct ovn_lflow *lflow_ref_aff_lb = NULL; + uint32_t hash_aff_lb = ovn_logical_flow_hash( + ovn_stage_get_table(stage1), ovn_stage_get_pipeline(stage1), + 150, ds_cstr(&aff_match), ds_cstr(&aff_action_lb)); + + for (size_t j = 0; j < n_dp; j++) { + struct ovn_datapath *od = router_pipeline + ? lb->nb_lr[j] : lb->nb_ls[j]; + if (!ovn_dp_group_add_with_reference(lflow_ref_aff_learn, od)) { + lflow_ref_aff_learn = ovn_lflow_add_at_with_hash( + lflows, od, stage0, 100, ds_cstr(&aff_match_learn), + ds_cstr(&aff_action_learn), NULL, NULL, + &lb->nlb->header_, OVS_SOURCE_LOCATOR, + hash_aff_learn); + } + if (!ovn_dp_group_add_with_reference(lflow_ref_aff_lb, od)) { + lflow_ref_aff_lb = ovn_lflow_add_at_with_hash( + lflows, od, stage1, 150, ds_cstr(&aff_match), + ds_cstr(&aff_action_lb), NULL, NULL, + &lb->nlb->header_, OVS_SOURCE_LOCATOR, + hash_aff_lb); + } + } + + ds_clear(&aff_action_learn); + ds_clear(&aff_match_learn); + ds_clear(&aff_action_lb); + ds_clear(&aff_match); + } + + ds_destroy(&aff_action_learn); + ds_destroy(&aff_match_learn); + ds_destroy(&aff_action_lb); + ds_destroy(&aff_match); +} + +static void +build_lb_affinity_ls_flows(struct hmap *lflows, struct ovn_northd_lb *lb, + struct ovn_lb_vip *lb_vip) +{ + struct ds match = DS_EMPTY_INITIALIZER; + if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { + ds_put_format(&match, "ct.new && ip4 && "REG_ORIG_DIP_IPV4 " == %s", + lb_vip->vip_str); + } else { + ds_put_format(&match, "ct.new && ip6 && "REG_ORIG_DIP_IPV6 " == %s", + lb_vip->vip_str); + } + + if (lb_vip->vip_port) { + ds_put_format(&match, " && "REG_ORIG_TP_DPORT " == %"PRIu16, + lb_vip->vip_port); + } + + build_lb_affinity_flows(lflows, lb, lb_vip, ds_cstr(&match), false); + ds_destroy(&match); +} + +static void +build_lb_affinity_default_flows(struct ovn_datapath *od, struct hmap *lflows) +{ + if (od->nbs) { + ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_CHECK, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_LEARN, 0, "1", "next;"); + } + if (od->nbr) { + ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_CHECK, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_LEARN, 0, "1", "next;"); + } +} + static void build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, bool ct_lb_mark, struct ds *match, struct ds *action, @@ -6992,6 +7197,8 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, bool ct_lb_mark, priority = 120; } + build_lb_affinity_ls_flows(lflows, lb, lb_vip); + struct ovn_lflow *lflow_ref = NULL; uint32_t hash = ovn_logical_flow_hash( ovn_stage_get_table(S_SWITCH_IN_LB), @@ -10074,6 +10281,8 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, xcalloc(lb->n_nb_lr, sizeof *distributed_router); int n_distributed_router = 0; + build_lb_affinity_flows(lflows, lb, lb_vip, new_match, true); + /* Group gw router since we do not have datapath dependency in * lflow generation for them. */ @@ -13981,6 +14190,7 @@ build_lswitch_and_lrouter_iterate_by_od(struct ovn_datapath *od, build_lrouter_nat_defrag_and_lb(od, lsi->lflows, lsi->ports, &lsi->match, &lsi->actions, lsi->meter_groups, lsi->features->ct_no_masked_label); + build_lb_affinity_default_flows(od, lsi->lflows); } /* Helper function to combine all lflow generation which is iterated by port. diff --git a/northd/ovn-northd.8.xml b/northd/ovn-northd.8.xml index a70f2e678..e1a6ca871 100644 --- a/northd/ovn-northd.8.xml +++ b/northd/ovn-northd.8.xml @@ -853,9 +853,55 @@ -

Ingress Table 11: LB

+

Ingress Table 11: Load balancing affinity check

+ +

+ Load balancing affinity check table contains the following + logical flows: +

    +
  • + For all the configured load balancing rules for a switch in + OVN_Northbound database where a positive affinity timeout + is specified in options column, that includes a L4 port + PORT of protocol P and IP address VIP, + a priority-100 flow is added. For IPv4 VIPs, the flow + matches ct.new && ip && ip4.dst == VIP + && P.dst == PORT. For IPv6 + VIPs, the flow matches ct.new && ip && + ip6.dst == VIP&& P && + P.dst == PORT. The flow's action is + reg9[6] = chk_lb_aff(); next;. +
    • + +
  • + A priority 0 flow is added which matches on all packets and applies + the action next;. +
    • +
+ +

Ingress Table 12: LB

+ +
    +
  • + For all the configured load balancing rules for a switch in + OVN_Northbound database where a positive affinity timeout + is specified in options column, that includes a L4 port + PORT of protocol P and IP address VIP, + a priority-150 flow is added. For IPv4 VIPs, the flow + matches reg9[6] == 1 && ip && ip4.dst == + VIP && P.dst == PORT. + For IPv6 VIPs, the flow matches reg9[6] == 1 + && ip && ip6.dst == VIP && + P && P.dst == PORT. + The flow's action is ct_lb_mark(args), where + args contains comma separated IP addresses (and optional + port numbers) to load balance to. The address family of the IP + addresses of args is the same as the address family + of VIP. +
    • +
  • For all the configured load balancing rules for a switch in OVN_Northbound database that includes a L4 port @@ -914,7 +960,38 @@
-

Ingress table 12: from-lport ACLs after LB

+

Ingress Table 13: Load balancing affinity learn

+ +

+ Load balancing affinity learn table contains the following + logical flows: +

+ +
    +
  • + For all the configured load balancing rules for a switch in + OVN_Northbound database where a positive affinity timeout + T is specified in options column, that includes + a L4 port PORT of protocol P and IP address + VIP, a priority-100 flow is added. For IPv4 VIPs, + the flow matches ct.new && ip && + ip4.dst == VIP && + P.dst == PORT. For IPv6 VIPs, + the flow matches ct.new && ip && ip6.dst == + VIP && P && P.dst == + PORT. The flow's action is commit_lb_aff(vip = + VIP:PORT, backend = backend ip: + backend port, proto = P, timeout = T); + next;. +
    • + +
  • + A priority 0 flow is added which matches on all packets and applies + the action next;. +
    • +
+ +

Ingress table 14: from-lport ACLs after LB

Logical flows in this table closely reproduce those in the @@ -976,7 +1053,7 @@ -

Ingress Table 13: Stateful

+

Ingress Table 15: Stateful

  • @@ -999,7 +1076,7 @@
-

Ingress Table 14: Pre-Hairpin

+

Ingress Table 16: Pre-Hairpin

  • If the logical switch has load balancer(s) configured, then a @@ -1017,7 +1094,7 @@
-

Ingress Table 15: Nat-Hairpin

+

Ingress Table 17: Nat-Hairpin

  • If the logical switch has load balancer(s) configured, then a @@ -1052,7 +1129,7 @@
-

Ingress Table 16: Hairpin

+

Ingress Table 18: Hairpin

  • @@ -1086,7 +1163,7 @@

-

Ingress Table 17: ARP/ND responder

+

Ingress Table 19: ARP/ND responder

This table implements ARP/ND responder in a logical switch for known @@ -1388,7 +1465,7 @@ output; -

Ingress Table 18: DHCP option processing

+

Ingress Table 20: DHCP option processing

This table adds the DHCPv4 options to a DHCPv4 packet from the @@ -1449,7 +1526,7 @@ next; -

Ingress Table 19: DHCP responses

+

Ingress Table 21: DHCP responses

This table implements DHCP responder for the DHCP replies generated by @@ -1530,7 +1607,7 @@ output; -

Ingress Table 20 DNS Lookup

+

Ingress Table 22 DNS Lookup

This table looks up and resolves the DNS names to the corresponding @@ -1559,7 +1636,7 @@ reg0[4] = dns_lookup(); next; -

Ingress Table 21 DNS Responses

+

Ingress Table 23 DNS Responses

This table implements DNS responder for the DNS replies generated by @@ -1594,7 +1671,7 @@ output; -

Ingress table 22 External ports

+

Ingress table 24 External ports

Traffic from the external logical ports enter the ingress @@ -1637,7 +1714,7 @@ output; -

Ingress Table 23 Destination Lookup

+

Ingress Table 25 Destination Lookup

This table implements switching behavior. It contains these logical @@ -1806,7 +1883,7 @@ output; -

Ingress Table 24 Destination unknown

+

Ingress Table 26 Destination unknown

This table handles the packets whose destination was not found or @@ -3172,7 +3249,33 @@ icmp6 { packet de-fragmentation and tracking before sending it to the next table.

-

Ingress Table 6: DNAT

+

Ingress Table 6: Load balancing affinity check

+ +

+ Load balancing affinity check table contains the following + logical flows: +

+ +
    +
  • + For all the configured load balancing rules for a logical router where + a positive affinity timeout is specified in options + column, that includes a L4 port PORT of protocol + P and IPv4 or IPv6 address VIP, a priority-100 + flow that matches on ct.new && ip && + reg0 == VIP && P && reg9[16..31] + == PORT (xxreg0 == VIP + in the IPv6 case) with an action of reg9[6] = + chk_lb_aff(); next; +
    • + +
  • + A priority 0 flow is added which matches on all packets and applies + the action next;. +
    • +
+ +

Ingress Table 7: DNAT

Packets enter the pipeline with destination IP address that needs to @@ -3180,7 +3283,7 @@ icmp6 { in the reverse direction needs to be unDNATed.

-

Ingress Table 6: Load balancing DNAT rules

+

Ingress Table 7: Load balancing DNAT rules

Following load balancing DNAT flows are added for Gateway router or @@ -3190,6 +3293,21 @@ icmp6 {

    +
  • + For all the configured load balancing rules for a logical router where + a positive affinity timeout is specified in options + column, that includes a L4 port PORT of protocol + P and IPv4 or IPv6 address VIP, a priority-150 + flow that matches on reg9[6] == 1 && ip && + reg0 == VIP && P && + reg9[16..31] == PORT (xxreg0 + == VIP in the IPv6 case) with an action of + ct_lb_mark(args) , where args + contains comma separated IP addresses (and optional port numbers) + to load balance to. The address family of the IP addresses of + args is the same as the address family of VIP. +
    • +
  • If controller_event has been enabled for all the configured load balancing rules for a Gateway router or Router with gateway port @@ -3319,7 +3437,7 @@ icmp6 {
-

Ingress Table 6: DNAT on Gateway Routers

+

Ingress Table 7: DNAT on Gateway Routers

  • @@ -3361,7 +3479,7 @@ icmp6 {
-

Ingress Table 6: DNAT on Distributed Routers

+

Ingress Table 7: DNAT on Distributed Routers

On distributed routers, the DNAT table only handles packets @@ -3416,7 +3534,35 @@ icmp6 { -

Ingress Table 7: ECMP symmetric reply processing

+

Ingress Table 8: Load balancing affinity learn

+ +

+ Load balancing affinity learn table contains the following + logical flows: +

+ +
    +
  • + For all the configured load balancing rules for a logical router where + a positive affinity timeout T is specified in options + column, that includes a L4 port PORT of protocol + P and IPv4 or IPv6 address VIP, a priority-100 + flow that matches on ct.new && ip && reg0 == + VIP && P && reg9[16..31] == + PORT (xxreg0 == VIP + in the IPv6 case) with an action of commit_lb_aff(vip = + VIP:PORT, backend = backend ip: + backend port, proto = P, timeout = T); + next;. +
    • + +
  • + A priority 0 flow is added which matches on all packets and applies + the action next;. +
    • +
+ +

Ingress Table 9: ECMP symmetric reply processing

  • If ECMP routes with symmetric reply are configured in the @@ -3435,7 +3581,7 @@ icmp6 {
-

Ingress Table 8: IPv6 ND RA option processing

+

Ingress Table 10: IPv6 ND RA option processing

  • @@ -3465,7 +3611,7 @@ reg0[5] = put_nd_ra_opts(options);next;
-

Ingress Table 9: IPv6 ND RA responder

+

Ingress Table 11: IPv6 ND RA responder

This table implements IPv6 ND RA responder for the IPv6 ND RA replies @@ -3510,7 +3656,7 @@ output; -

Ingress Table 10: IP Routing Pre

+

Ingress Table 12: IP Routing Pre

If a packet arrived at this table from Logical Router Port P @@ -3540,7 +3686,7 @@ output; -

Ingress Table 11: IP Routing

+

Ingress Table 13: IP Routing

A packet that arrives at this table is an IP packet that should be @@ -3741,7 +3887,7 @@ select(reg8[16..31], MID1, MID2, ...); -

Ingress Table 12: IP_ROUTING_ECMP

+

Ingress Table 14: IP_ROUTING_ECMP

This table implements the second part of IP routing for ECMP routes @@ -3793,7 +3939,7 @@ outport = P; -

Ingress Table 13: Router policies

+

Ingress Table 15: Router policies

This table adds flows for the logical router policies configured on the logical router. Please see the @@ -3865,7 +4011,7 @@ next; -

Ingress Table 14: ECMP handling for router policies

+

Ingress Table 16: ECMP handling for router policies

This table handles the ECMP for the router policies configured with multiple nexthops. @@ -3909,7 +4055,7 @@ outport = P -

Ingress Table 15: ARP/ND Resolution

+

Ingress Table 17: ARP/ND Resolution

Any packet that reaches this table is an IP packet whose next-hop @@ -4127,7 +4273,7 @@ outport = P -

Ingress Table 16: Check packet length

+

Ingress Table 18: Check packet length

For distributed logical routers or gateway routers with gateway @@ -4164,7 +4310,7 @@ REGBIT_PKT_LARGER = check_pkt_larger(L); next; and advances to the next table.

-

Ingress Table 17: Handle larger packets

+

Ingress Table 19: Handle larger packets

For distributed logical routers or gateway routers with gateway port @@ -4227,7 +4373,7 @@ icmp6 { and advances to the next table.

-

Ingress Table 18: Gateway Redirect

+

Ingress Table 20: Gateway Redirect

For distributed logical routers where one or more of the logical router @@ -4295,7 +4441,7 @@ icmp6 { -

Ingress Table 19: ARP Request

+

Ingress Table 21: ARP Request

In the common case where the Ethernet destination has been resolved, this diff --git a/ovn-nb.xml b/ovn-nb.xml index f41e9d7c0..de0243a96 100644 --- a/ovn-nb.xml +++ b/ovn-nb.xml @@ -1908,6 +1908,13 @@ requests only for VIPs that are part of a router's subnet. The default value of this option, if not specified, is reachable. + + + If the CMS provide a positive value for affinity_timeout, + OVN will dnat connections received from the same client to this lb to + the same backend if received in the affinity timeslot. Max supported + affinity_timeout is 65535s. + diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 4f399eccb..686667e47 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -2158,9 +2158,9 @@ AT_CAPTURE_FILE([sw1flows]) AT_CHECK( [grep -E 'ls_(in|out)_acl' sw0flows sw1flows | grep pg0 | sort], [0], [dnl -sw0flows: table=4 (ls_out_acl ), priority=2003 , match=(outport == @pg0 && ip6 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=23); };) +sw0flows: table=4 (ls_out_acl ), priority=2003 , match=(outport == @pg0 && ip6 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=25); };) sw0flows: table=8 (ls_in_acl ), priority=2002 , match=(inport == @pg0 && ip4 && tcp && tcp.dst == 80), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=egress,table=5); };) -sw1flows: table=4 (ls_out_acl ), priority=2003 , match=(outport == @pg0 && ip6 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=23); };) +sw1flows: table=4 (ls_out_acl ), priority=2003 , match=(outport == @pg0 && ip6 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=25); };) sw1flows: table=8 (ls_in_acl ), priority=2002 , match=(inport == @pg0 && ip4 && tcp && tcp.dst == 80), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=egress,table=5); };) ]) @@ -2174,10 +2174,10 @@ ovn-sbctl dump-flows sw1 > sw1flows2 AT_CAPTURE_FILE([sw1flows2]) AT_CHECK([grep "ls_out_acl" sw0flows2 sw1flows2 | grep pg0 | sort], [0], [dnl -sw0flows2: table=4 (ls_out_acl ), priority=2002 , match=(outport == @pg0 && ip4 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=23); };) -sw0flows2: table=4 (ls_out_acl ), priority=2003 , match=(outport == @pg0 && ip6 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=23); };) -sw1flows2: table=4 (ls_out_acl ), priority=2002 , match=(outport == @pg0 && ip4 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=23); };) -sw1flows2: table=4 (ls_out_acl ), priority=2003 , match=(outport == @pg0 && ip6 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=23); };) +sw0flows2: table=4 (ls_out_acl ), priority=2002 , match=(outport == @pg0 && ip4 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=25); };) +sw0flows2: table=4 (ls_out_acl ), priority=2003 , match=(outport == @pg0 && ip6 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=25); };) +sw1flows2: table=4 (ls_out_acl ), priority=2002 , match=(outport == @pg0 && ip4 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=25); };) +sw1flows2: table=4 (ls_out_acl ), priority=2003 , match=(outport == @pg0 && ip6 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=25); };) ]) AS_BOX([3]) @@ -2192,16 +2192,16 @@ AT_CAPTURE_FILE([sw1flows3]) AT_CHECK([grep "ls_out_acl" sw0flows3 sw1flows3 | grep pg0 | sort], [0], [dnl sw0flows3: table=4 (ls_out_acl ), priority=2001 , match=(reg0[[7]] == 1 && (outport == @pg0 && ip)), action=(reg0[[1]] = 1; next;) sw0flows3: table=4 (ls_out_acl ), priority=2001 , match=(reg0[[8]] == 1 && (outport == @pg0 && ip)), action=(next;) -sw0flows3: table=4 (ls_out_acl ), priority=2002 , match=((reg0[[10]] == 1) && outport == @pg0 && ip4 && udp), action=(ct_commit { ct_mark.blocked = 1; }; reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=23); };) -sw0flows3: table=4 (ls_out_acl ), priority=2002 , match=((reg0[[9]] == 1) && outport == @pg0 && ip4 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=23); };) -sw0flows3: table=4 (ls_out_acl ), priority=2003 , match=((reg0[[10]] == 1) && outport == @pg0 && ip6 && udp), action=(ct_commit { ct_mark.blocked = 1; }; reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=23); };) -sw0flows3: table=4 (ls_out_acl ), priority=2003 , match=((reg0[[9]] == 1) && outport == @pg0 && ip6 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=23); };) +sw0flows3: table=4 (ls_out_acl ), priority=2002 , match=((reg0[[10]] == 1) && outport == @pg0 && ip4 && udp), action=(ct_commit { ct_mark.blocked = 1; }; reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=25); };) +sw0flows3: table=4 (ls_out_acl ), priority=2002 , match=((reg0[[9]] == 1) && outport == @pg0 && ip4 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=25); };) +sw0flows3: table=4 (ls_out_acl ), priority=2003 , match=((reg0[[10]] == 1) && outport == @pg0 && ip6 && udp), action=(ct_commit { ct_mark.blocked = 1; }; reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=25); };) +sw0flows3: table=4 (ls_out_acl ), priority=2003 , match=((reg0[[9]] == 1) && outport == @pg0 && ip6 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=25); };) sw1flows3: table=4 (ls_out_acl ), priority=2001 , match=(reg0[[7]] == 1 && (outport == @pg0 && ip)), action=(reg0[[1]] = 1; next;) sw1flows3: table=4 (ls_out_acl ), priority=2001 , match=(reg0[[8]] == 1 && (outport == @pg0 && ip)), action=(next;) -sw1flows3: table=4 (ls_out_acl ), priority=2002 , match=((reg0[[10]] == 1) && outport == @pg0 && ip4 && udp), action=(ct_commit { ct_mark.blocked = 1; }; reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=23); };) -sw1flows3: table=4 (ls_out_acl ), priority=2002 , match=((reg0[[9]] == 1) && outport == @pg0 && ip4 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=23); };) -sw1flows3: table=4 (ls_out_acl ), priority=2003 , match=((reg0[[10]] == 1) && outport == @pg0 && ip6 && udp), action=(ct_commit { ct_mark.blocked = 1; }; reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=23); };) -sw1flows3: table=4 (ls_out_acl ), priority=2003 , match=((reg0[[9]] == 1) && outport == @pg0 && ip6 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=23); };) +sw1flows3: table=4 (ls_out_acl ), priority=2002 , match=((reg0[[10]] == 1) && outport == @pg0 && ip4 && udp), action=(ct_commit { ct_mark.blocked = 1; }; reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=25); };) +sw1flows3: table=4 (ls_out_acl ), priority=2002 , match=((reg0[[9]] == 1) && outport == @pg0 && ip4 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=25); };) +sw1flows3: table=4 (ls_out_acl ), priority=2003 , match=((reg0[[10]] == 1) && outport == @pg0 && ip6 && udp), action=(ct_commit { ct_mark.blocked = 1; }; reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=25); };) +sw1flows3: table=4 (ls_out_acl ), priority=2003 , match=((reg0[[9]] == 1) && outport == @pg0 && ip6 && udp), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=25); };) ]) AT_CLEANUP ]) @@ -2373,7 +2373,7 @@ check ovn-nbctl --wait=sb \ -- ls-lb-add ls lb AT_CHECK([ovn-sbctl lflow-list ls | grep -e ls_in_acl_hint -e ls_out_acl_hint -e ls_in_acl -e ls_out_acl | sort], [0], [dnl - table=12(ls_in_acl_after_lb ), priority=0 , match=(1), action=(next;) + table=14(ls_in_acl_after_lb ), priority=0 , match=(1), action=(next;) table=3 (ls_out_acl_hint ), priority=0 , match=(1), action=(next;) table=3 (ls_out_acl_hint ), priority=1 , match=(ct.est && ct_mark.blocked == 0), action=(reg0[[10]] = 1; next;) table=3 (ls_out_acl_hint ), priority=2 , match=(ct.est && ct_mark.blocked == 1), action=(reg0[[9]] = 1; next;) @@ -2416,7 +2416,7 @@ ovn-nbctl --wait=sb clear logical_switch ls acls ovn-nbctl --wait=sb clear logical_switch ls load_balancer AT_CHECK([ovn-sbctl lflow-list ls | grep -e ls_in_acl_hint -e ls_out_acl_hint -e ls_in_acl -e ls_out_acl | sort], [0], [dnl - table=12(ls_in_acl_after_lb ), priority=0 , match=(1), action=(next;) + table=14(ls_in_acl_after_lb ), priority=0 , match=(1), action=(next;) table=3 (ls_out_acl_hint ), priority=65535, match=(1), action=(next;) table=4 (ls_out_acl ), priority=65535, match=(1), action=(next;) table=7 (ls_in_acl_hint ), priority=65535, match=(1), action=(next;) @@ -3659,11 +3659,11 @@ AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl - table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.100 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80), action=(ct_lb_mark(backends=10.0.0.4:8080);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.100 && tcp && reg9[[16..31]] == 80), action=(ct_lb_mark(backends=10.0.0.40:8080);) + table=7 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.100 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80), action=(ct_lb_mark(backends=10.0.0.4:8080);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.100 && tcp && reg9[[16..31]] == 80), action=(ct_lb_mark(backends=10.0.0.40:8080);) ]) AT_CHECK([grep "lr_out_undnat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -3695,11 +3695,11 @@ AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl - table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.100 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.4:8080);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.100 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.40:8080);) + table=7 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.100 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.4:8080);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.100 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.40:8080);) ]) AT_CHECK([grep "lr_out_snat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -3741,11 +3741,11 @@ AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl - table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.100 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.4:8080);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.100 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.40:8080);) + table=7 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.100 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.4:8080);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.100 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.40:8080);) ]) AT_CHECK([grep "lr_out_snat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -3801,11 +3801,11 @@ AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl - table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.100 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.4:8080);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.100 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.40:8080);) + table=7 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.100 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.4:8080);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.100 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.40:8080);) ]) AT_CHECK([grep "lr_out_snat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -3848,8 +3848,8 @@ AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl ]) AT_CHECK([grep "lr_in_dnat" lr0flows | grep skip_snat_for_lb | sort], [0], [dnl - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.20 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.skip_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.20 && tcp && reg9[[16..31]] == 80), action=(flags.skip_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.40:8080);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.20 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.skip_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.20 && tcp && reg9[[16..31]] == 80), action=(flags.skip_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.40:8080);) ]) AT_CHECK([grep "lr_out_snat" lr0flows | grep skip_snat_for_lb | sed 's/table=./table=?/' | sort], [0], [dnl @@ -4019,7 +4019,7 @@ check_stateful_flows() { table=? (ls_in_pre_stateful ), priority=120 , match=(ip4.dst == 10.0.0.20 && tcp.dst == 80), action=(reg1 = 10.0.0.20; reg2[[0..15]] = 80; ct_lb_mark;) ]) - AT_CHECK([grep "ls_in_lb" sw0flows | sort | sed 's/table=../table=??/'], [0], [dnl + AT_CHECK([grep "ls_in_lb " sw0flows | sort | sed 's/table=../table=??/'], [0], [dnl table=??(ls_in_lb ), priority=0 , match=(1), action=(next;) table=??(ls_in_lb ), priority=120 , match=(ct.new && ip4.dst == 10.0.0.10 && tcp.dst == 80), action=(reg0[[1]] = 0; ct_lb_mark(backends=10.0.0.4:8080);) table=??(ls_in_lb ), priority=120 , match=(ct.new && ip4.dst == 10.0.0.20 && tcp.dst == 80), action=(reg0[[1]] = 0; ct_lb_mark(backends=10.0.0.40:8080);) @@ -4085,7 +4085,7 @@ AT_CHECK([grep "ls_in_pre_stateful" sw0flows | sort | sed 's/table=./table=?/'], table=? (ls_in_pre_stateful ), priority=110 , match=(reg0[[2]] == 1), action=(ct_lb_mark;) ]) -AT_CHECK([grep "ls_in_lb" sw0flows | sort | sed 's/table=../table=??/'], [0], [dnl +AT_CHECK([grep "ls_in_lb " sw0flows | sort | sed 's/table=../table=??/'], [0], [dnl table=??(ls_in_lb ), priority=0 , match=(1), action=(next;) ]) @@ -4946,7 +4946,7 @@ AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl - table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=7 (lr_in_dnat ), priority=0 , match=(1), action=(next;) ]) AT_CHECK([grep "lr_out_undnat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -4982,7 +4982,7 @@ AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl - table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=7 (lr_in_dnat ), priority=0 , match=(1), action=(next;) ]) AT_CHECK([grep "lr_out_undnat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -5034,8 +5034,8 @@ AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl - table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) - table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat_in_czone(10.0.0.3);) + table=7 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=7 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat_in_czone(10.0.0.3);) ]) AT_CHECK([grep "lr_out_chk_dnat_local" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -5101,16 +5101,16 @@ AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl - table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) - table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat_in_czone(10.0.0.3);) - table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip4 && reg0 == 172.168.0.200 && ct_mark.natted == 1 && is_chassis_resident("cr-lr0-public")), action=(next;) - table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip4 && reg0 == 172.168.0.200 && is_chassis_resident("cr-lr0-public")), action=(ct_lb_mark(backends=10.0.0.80,10.0.0.81);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1 && is_chassis_resident("cr-lr0-public")), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082 && ct_mark.natted == 1 && is_chassis_resident("cr-lr0-public")), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60 && ct_mark.natted == 1 && is_chassis_resident("cr-lr0-public")), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && is_chassis_resident("cr-lr0-public")), action=(ct_lb_mark(backends=10.0.0.4:8080);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082 && is_chassis_resident("cr-lr0-public")), action=(ct_lb_mark(backends=10.0.0.50:82,10.0.0.60:82);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60 && is_chassis_resident("cr-lr0-public")), action=(ct_lb_mark(backends=10.0.0.50:6062,10.0.0.60:6062);) + table=7 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=7 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat_in_czone(10.0.0.3);) + table=7 (lr_in_dnat ), priority=110 , match=(ct.est && ip4 && reg0 == 172.168.0.200 && ct_mark.natted == 1 && is_chassis_resident("cr-lr0-public")), action=(next;) + table=7 (lr_in_dnat ), priority=110 , match=(ct.new && ip4 && reg0 == 172.168.0.200 && is_chassis_resident("cr-lr0-public")), action=(ct_lb_mark(backends=10.0.0.80,10.0.0.81);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1 && is_chassis_resident("cr-lr0-public")), action=(next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082 && ct_mark.natted == 1 && is_chassis_resident("cr-lr0-public")), action=(next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60 && ct_mark.natted == 1 && is_chassis_resident("cr-lr0-public")), action=(next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && is_chassis_resident("cr-lr0-public")), action=(ct_lb_mark(backends=10.0.0.4:8080);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082 && is_chassis_resident("cr-lr0-public")), action=(ct_lb_mark(backends=10.0.0.50:82,10.0.0.60:82);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60 && is_chassis_resident("cr-lr0-public")), action=(ct_lb_mark(backends=10.0.0.50:6062,10.0.0.60:6062);) ]) AT_CHECK([grep "lr_out_chk_dnat_local" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -5170,16 +5170,16 @@ AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl - table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) - table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) - table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip4 && reg0 == 172.168.0.200 && ct_mark.natted == 1), action=(next;) - table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip4 && reg0 == 172.168.0.200), action=(ct_lb_mark(backends=10.0.0.80,10.0.0.81);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082 && ct_mark.natted == 1), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60 && ct_mark.natted == 1), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80), action=(ct_lb_mark(backends=10.0.0.4:8080);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082), action=(ct_lb_mark(backends=10.0.0.50:82,10.0.0.60:82);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60), action=(ct_lb_mark(backends=10.0.0.50:6062,10.0.0.60:6062);) + table=7 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=7 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) + table=7 (lr_in_dnat ), priority=110 , match=(ct.est && ip4 && reg0 == 172.168.0.200 && ct_mark.natted == 1), action=(next;) + table=7 (lr_in_dnat ), priority=110 , match=(ct.new && ip4 && reg0 == 172.168.0.200), action=(ct_lb_mark(backends=10.0.0.80,10.0.0.81);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082 && ct_mark.natted == 1), action=(next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60 && ct_mark.natted == 1), action=(next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80), action=(ct_lb_mark(backends=10.0.0.4:8080);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082), action=(ct_lb_mark(backends=10.0.0.50:82,10.0.0.60:82);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60), action=(ct_lb_mark(backends=10.0.0.50:6062,10.0.0.60:6062);) ]) AT_CHECK([grep "lr_out_chk_dnat_local" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -5231,16 +5231,16 @@ AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl - table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) - table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) - table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip4 && reg0 == 172.168.0.200 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip4 && reg0 == 172.168.0.200), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.80,10.0.0.81);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.4:8080);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:82,10.0.0.60:82);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:6062,10.0.0.60:6062);) + table=7 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=7 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) + table=7 (lr_in_dnat ), priority=110 , match=(ct.est && ip4 && reg0 == 172.168.0.200 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=110 , match=(ct.new && ip4 && reg0 == 172.168.0.200), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.80,10.0.0.81);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.4:8080);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:82,10.0.0.60:82);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:6062,10.0.0.60:6062);) ]) AT_CHECK([grep "lr_out_chk_dnat_local" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -5295,18 +5295,18 @@ AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl - table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) - table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) - table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip4 && reg0 == 172.168.0.200 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip4 && reg0 == 172.168.0.200), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.80,10.0.0.81);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.10 && tcp && reg9[[16..31]] == 9082 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.4:8080);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.10 && tcp && reg9[[16..31]] == 9082), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:82,10.0.0.60:82);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:82,10.0.0.60:82);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:6062,10.0.0.60:6062);) + table=7 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=7 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) + table=7 (lr_in_dnat ), priority=110 , match=(ct.est && ip4 && reg0 == 172.168.0.200 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=110 , match=(ct.new && ip4 && reg0 == 172.168.0.200), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.80,10.0.0.81);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.10 && tcp && reg9[[16..31]] == 9082 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.4:8080);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.10 && tcp && reg9[[16..31]] == 9082), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:82,10.0.0.60:82);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:82,10.0.0.60:82);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:6062,10.0.0.60:6062);) ]) AT_CHECK([grep "lr_out_chk_dnat_local" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -5372,20 +5372,20 @@ AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl - table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) - table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) - table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip4 && reg0 == 172.168.0.200 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip4 && reg0 == 172.168.0.200), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.80,10.0.0.81);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.10 && tcp && reg9[[16..31]] == 9082 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip6 && xxreg0 == def0::2 && tcp && reg9[[16..31]] == 8000 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.4:8080);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.10 && tcp && reg9[[16..31]] == 9082), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:82,10.0.0.60:82);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:82,10.0.0.60:82);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:6062,10.0.0.60:6062);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip6 && xxreg0 == def0::2 && tcp && reg9[[16..31]] == 8000), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=[[aef0::2]]:80,[[aef0::3]]:80);) + table=7 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=7 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) + table=7 (lr_in_dnat ), priority=110 , match=(ct.est && ip4 && reg0 == 172.168.0.200 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=110 , match=(ct.new && ip4 && reg0 == 172.168.0.200), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.80,10.0.0.81);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.10 && tcp && reg9[[16..31]] == 9082 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip6 && xxreg0 == def0::2 && tcp && reg9[[16..31]] == 8000 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 10.0.0.10 && tcp && reg9[[16..31]] == 80), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.4:8080);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.10 && tcp && reg9[[16..31]] == 9082), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:82,10.0.0.60:82);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.100 && tcp && reg9[[16..31]] == 8082), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:82,10.0.0.60:82);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:6062,10.0.0.60:6062);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip6 && xxreg0 == def0::2 && tcp && reg9[[16..31]] == 8000), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=[[aef0::2]]:80,[[aef0::3]]:80);) ]) AT_CHECK([grep "lr_out_chk_dnat_local" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -5440,11 +5440,11 @@ AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl - table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.210 && tcp && reg9[[16..31]] == 60 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.210 && tcp && reg9[[16..31]] == 60), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:6062,10.0.0.60:6062);) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:6062,10.0.0.60:6062);) + table=7 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.210 && tcp && reg9[[16..31]] == 60 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.est && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60 && ct_mark.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.210 && tcp && reg9[[16..31]] == 60), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:6062,10.0.0.60:6062);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && ip4 && reg0 == 172.168.0.210 && udp && reg9[[16..31]] == 60), action=(flags.force_snat_for_lb = 1; ct_lb_mark(backends=10.0.0.50:6062,10.0.0.60:6062);) ]) AT_CHECK([grep "lr_out_chk_dnat_local" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -6537,7 +6537,7 @@ AT_CHECK([grep -e "ls_in_acl" lsflows | sed 's/table=../table=??/' | sort], [0], table=??(ls_in_acl_hint ), priority=7 , match=(ct.new && !ct.est), action=(reg0[[7]] = 1; reg0[[9]] = 1; next;) ]) -AT_CHECK([grep -e "ls_in_lb" lsflows | sed 's/table=../table=??/' | sort], [0], [dnl +AT_CHECK([grep -e "ls_in_lb " lsflows | sed 's/table=../table=??/' | sort], [0], [dnl table=??(ls_in_lb ), priority=0 , match=(1), action=(next;) table=??(ls_in_lb ), priority=110 , match=(ct.new && ip4.dst == 10.0.0.2), action=(reg0[[1]] = 0; ct_lb_mark(backends=10.0.0.10);) ]) @@ -6590,7 +6590,7 @@ AT_CHECK([grep -e "ls_in_acl" lsflows | sed 's/table=../table=??/' | sort], [0], table=??(ls_in_acl_hint ), priority=7 , match=(ct.new && !ct.est), action=(reg0[[7]] = 1; reg0[[9]] = 1; next;) ]) -AT_CHECK([grep -e "ls_in_lb" lsflows | sed 's/table=../table=??/' | sort], [0], [dnl +AT_CHECK([grep -e "ls_in_lb " lsflows | sed 's/table=../table=??/' | sort], [0], [dnl table=??(ls_in_lb ), priority=0 , match=(1), action=(next;) table=??(ls_in_lb ), priority=110 , match=(ct.new && ip4.dst == 10.0.0.2), action=(reg0[[1]] = 0; ct_lb_mark(backends=10.0.0.10);) ]) @@ -6643,7 +6643,7 @@ AT_CHECK([grep -e "ls_in_acl" lsflows | sed 's/table=../table=??/' | sort], [0], table=??(ls_in_acl_hint ), priority=7 , match=(ct.new && !ct.est), action=(reg0[[7]] = 1; reg0[[9]] = 1; next;) ]) -AT_CHECK([grep -e "ls_in_lb" lsflows | sed 's/table=../table=??/' | sort], [0], [dnl +AT_CHECK([grep -e "ls_in_lb " lsflows | sed 's/table=../table=??/' | sort], [0], [dnl table=??(ls_in_lb ), priority=0 , match=(1), action=(next;) table=??(ls_in_lb ), priority=110 , match=(ct.new && ip4.dst == 10.0.0.2), action=(reg0[[1]] = 0; ct_lb_mark(backends=10.0.0.10);) ]) @@ -7611,7 +7611,7 @@ sort | sed 's/table=../table=??/' ], [0], [dnl table=??(ls_in_check_port_sec), priority=100 , match=(vlan.present), action=(drop;) table=??(ls_in_check_port_sec), priority=50 , match=(1), action=(reg0[[15]] = check_in_port_sec(); next;) table=??(ls_in_check_port_sec), priority=70 , match=(inport == "localnetport"), action=(set_queue(10); reg0[[15]] = check_in_port_sec(); next;) - table=??(ls_in_check_port_sec), priority=70 , match=(inport == "sw0p1"), action=(reg0[[14]] = 1; next(pipeline=ingress, table=16);) + table=??(ls_in_check_port_sec), priority=70 , match=(inport == "sw0p1"), action=(reg0[[14]] = 1; next(pipeline=ingress, table=18);) table=??(ls_in_check_port_sec), priority=70 , match=(inport == "sw0p2"), action=(set_queue(10); reg0[[15]] = check_in_port_sec(); next;) table=??(ls_in_apply_port_sec), priority=0 , match=(1), action=(next;) table=??(ls_in_apply_port_sec), priority=50 , match=(reg0[[15]] == 1), action=(drop;) @@ -7648,11 +7648,11 @@ check ovn-nbctl \ AS_BOX([No chassis registered - use ct_lb_mark and ct_mark.natted]) check ovn-nbctl --wait=sb sync AT_CHECK([ovn-sbctl lflow-list | grep -e natted -e ct_lb], [0], [dnl - table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip4 && reg0 == 66.66.66.66 && ct_mark.natted == 1), action=(next;) - table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip4 && reg0 == 66.66.66.66), action=(ct_lb_mark(backends=42.42.42.2);) + table=7 (lr_in_dnat ), priority=110 , match=(ct.est && ip4 && reg0 == 66.66.66.66 && ct_mark.natted == 1), action=(next;) + table=7 (lr_in_dnat ), priority=110 , match=(ct.new && ip4 && reg0 == 66.66.66.66), action=(ct_lb_mark(backends=42.42.42.2);) table=6 (ls_in_pre_stateful ), priority=120 , match=(ip4.dst == 66.66.66.66), action=(reg1 = 66.66.66.66; ct_lb_mark;) table=6 (ls_in_pre_stateful ), priority=110 , match=(reg0[[2]] == 1), action=(ct_lb_mark;) - table=11(ls_in_lb ), priority=110 , match=(ct.new && ip4.dst == 66.66.66.66), action=(reg0[[1]] = 0; ct_lb_mark(backends=42.42.42.2);) + table=12(ls_in_lb ), priority=110 , match=(ct.new && ip4.dst == 66.66.66.66), action=(reg0[[1]] = 0; ct_lb_mark(backends=42.42.42.2);) table=2 (ls_out_pre_stateful), priority=110 , match=(reg0[[2]] == 1), action=(ct_lb_mark;) ]) @@ -7660,11 +7660,11 @@ AS_BOX([Chassis registered that doesn't support ct_lb_mark - use ct_lb and ct_la check ovn-sbctl chassis-add hv geneve 127.0.0.1 check ovn-nbctl --wait=sb sync AT_CHECK([ovn-sbctl lflow-list | grep -e natted -e ct_lb], [0], [dnl - table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip4 && reg0 == 66.66.66.66 && ct_label.natted == 1), action=(next;) - table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip4 && reg0 == 66.66.66.66), action=(ct_lb(backends=42.42.42.2);) + table=7 (lr_in_dnat ), priority=110 , match=(ct.est && ip4 && reg0 == 66.66.66.66 && ct_label.natted == 1), action=(next;) + table=7 (lr_in_dnat ), priority=110 , match=(ct.new && ip4 && reg0 == 66.66.66.66), action=(ct_lb(backends=42.42.42.2);) table=6 (ls_in_pre_stateful ), priority=120 , match=(ip4.dst == 66.66.66.66), action=(reg1 = 66.66.66.66; ct_lb;) table=6 (ls_in_pre_stateful ), priority=110 , match=(reg0[[2]] == 1), action=(ct_lb;) - table=11(ls_in_lb ), priority=110 , match=(ct.new && ip4.dst == 66.66.66.66), action=(reg0[[1]] = 0; ct_lb(backends=42.42.42.2);) + table=12(ls_in_lb ), priority=110 , match=(ct.new && ip4.dst == 66.66.66.66), action=(reg0[[1]] = 0; ct_lb(backends=42.42.42.2);) table=2 (ls_out_pre_stateful), priority=110 , match=(reg0[[2]] == 1), action=(ct_lb;) ]) @@ -7672,11 +7672,11 @@ AS_BOX([Chassis upgrades and supports ct_lb_mark - use ct_lb_mark and ct_mark.na check ovn-sbctl set chassis hv other_config:ct-no-masked-label=true check ovn-nbctl --wait=sb sync AT_CHECK([ovn-sbctl lflow-list | grep -e natted -e ct_lb], [0], [dnl - table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip4 && reg0 == 66.66.66.66 && ct_mark.natted == 1), action=(next;) - table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip4 && reg0 == 66.66.66.66), action=(ct_lb_mark(backends=42.42.42.2);) + table=7 (lr_in_dnat ), priority=110 , match=(ct.est && ip4 && reg0 == 66.66.66.66 && ct_mark.natted == 1), action=(next;) + table=7 (lr_in_dnat ), priority=110 , match=(ct.new && ip4 && reg0 == 66.66.66.66), action=(ct_lb_mark(backends=42.42.42.2);) table=6 (ls_in_pre_stateful ), priority=120 , match=(ip4.dst == 66.66.66.66), action=(reg1 = 66.66.66.66; ct_lb_mark;) table=6 (ls_in_pre_stateful ), priority=110 , match=(reg0[[2]] == 1), action=(ct_lb_mark;) - table=11(ls_in_lb ), priority=110 , match=(ct.new && ip4.dst == 66.66.66.66), action=(reg0[[1]] = 0; ct_lb_mark(backends=42.42.42.2);) + table=12(ls_in_lb ), priority=110 , match=(ct.new && ip4.dst == 66.66.66.66), action=(reg0[[1]] = 0; ct_lb_mark(backends=42.42.42.2);) table=2 (ls_out_pre_stateful), priority=110 , match=(reg0[[2]] == 1), action=(ct_lb_mark;) ]) @@ -7838,11 +7838,11 @@ ovn-sbctl dump-flows S1 > S1flows AT_CAPTURE_FILE([S0flows]) AT_CAPTURE_FILE([S1flows]) -AT_CHECK([grep "ls_in_lb" S0flows | sort], [0], [dnl - table=11(ls_in_lb ), priority=0 , match=(1), action=(next;) +AT_CHECK([grep "ls_in_lb " S0flows | sort], [0], [dnl + table=12(ls_in_lb ), priority=0 , match=(1), action=(next;) ]) -AT_CHECK([grep "ls_in_lb" S1flows | sort], [0], [dnl - table=11(ls_in_lb ), priority=0 , match=(1), action=(next;) +AT_CHECK([grep "ls_in_lb " S1flows | sort], [0], [dnl + table=12(ls_in_lb ), priority=0 , match=(1), action=(next;) ]) ovn-nbctl --wait=sb set NB_Global . options:install_ls_lb_from_router=true @@ -7854,14 +7854,14 @@ AT_CAPTURE_FILE([S0flows]) AT_CAPTURE_FILE([S1flows]) AT_CHECK([grep "ls_in_lb" S0flows | sort], [0], [dnl - table=11(ls_in_lb ), priority=0 , match=(1), action=(next;) - table=11(ls_in_lb ), priority=120 , match=(ct.new && ip4.dst == 172.16.0.10 && tcp.dst == 80), action=(reg0[[1]] = 0; ct_lb_mark(backends=10.0.0.2:80);) - table=11(ls_in_lb ), priority=120 , match=(ct.new && ip4.dst == 172.16.0.11 && tcp.dst == 8080), action=(reg0[[1]] = 0; ct_lb_mark(backends=10.0.0.2:8080);) + table=12(ls_in_lb ), priority=0 , match=(1), action=(next;) + table=12(ls_in_lb ), priority=120 , match=(ct.new && ip4.dst == 172.16.0.10 && tcp.dst == 80), action=(reg0[[1]] = 0; ct_lb_mark(backends=10.0.0.2:80);) + table=12(ls_in_lb ), priority=120 , match=(ct.new && ip4.dst == 172.16.0.11 && tcp.dst == 8080), action=(reg0[[1]] = 0; ct_lb_mark(backends=10.0.0.2:8080);) ]) AT_CHECK([grep "ls_in_lb" S1flows | sort], [0], [dnl - table=11(ls_in_lb ), priority=0 , match=(1), action=(next;) - table=11(ls_in_lb ), priority=120 , match=(ct.new && ip4.dst == 172.16.0.10 && tcp.dst == 80), action=(reg0[[1]] = 0; ct_lb_mark(backends=10.0.0.2:80);) - table=11(ls_in_lb ), priority=120 , match=(ct.new && ip4.dst == 172.16.0.11 && tcp.dst == 8080), action=(reg0[[1]] = 0; ct_lb_mark(backends=10.0.0.2:8080);) + table=12(ls_in_lb ), priority=0 , match=(1), action=(next;) + table=12(ls_in_lb ), priority=120 , match=(ct.new && ip4.dst == 172.16.0.10 && tcp.dst == 80), action=(reg0[[1]] = 0; ct_lb_mark(backends=10.0.0.2:80);) + table=12(ls_in_lb ), priority=120 , match=(ct.new && ip4.dst == 172.16.0.11 && tcp.dst == 8080), action=(reg0[[1]] = 0; ct_lb_mark(backends=10.0.0.2:8080);) ]) ovn-sbctl get datapath S0 _uuid > dp_uuids @@ -7880,11 +7880,11 @@ ovn-sbctl dump-flows S1 > S1flows AT_CAPTURE_FILE([S0flows]) AT_CAPTURE_FILE([S1flows]) -AT_CHECK([grep "ls_in_lb" S0flows | sort], [0], [dnl - table=11(ls_in_lb ), priority=0 , match=(1), action=(next;) +AT_CHECK([grep "ls_in_lb " S0flows | sort], [0], [dnl + table=12(ls_in_lb ), priority=0 , match=(1), action=(next;) ]) -AT_CHECK([grep "ls_in_lb" S1flows | sort], [0], [dnl - table=11(ls_in_lb ), priority=0 , match=(1), action=(next;) +AT_CHECK([grep "ls_in_lb " S1flows | sort], [0], [dnl + table=12(ls_in_lb ), priority=0 , match=(1), action=(next;) ]) check_column "" sb:load_balancer datapaths name=lb0 @@ -7923,8 +7923,86 @@ ovn-sbctl dump-flows R1 > R1flows AT_CAPTURE_FILE([R1flows]) AT_CHECK([grep "lr_in_arp_resolve" R1flows | grep priority=90 | sort], [0], [dnl - table=15(lr_in_arp_resolve ), priority=90 , match=(outport == "R1-PUB" && ip4.src == 10.0.0.3 && is_chassis_resident("S0-P0")), action=(get_arp(outport, reg0); next;) - table=15(lr_in_arp_resolve ), priority=90 , match=(outport == "R1-PUB" && ip6.src == 1000::3 && is_chassis_resident("S0-P0")), action=(get_nd(outport, xxreg0); next;) + table=17(lr_in_arp_resolve ), priority=90 , match=(outport == "R1-PUB" && ip4.src == 10.0.0.3 && is_chassis_resident("S0-P0")), action=(get_arp(outport, reg0); next;) + table=17(lr_in_arp_resolve ), priority=90 , match=(outport == "R1-PUB" && ip6.src == 1000::3 && is_chassis_resident("S0-P0")), action=(get_nd(outport, xxreg0); next;) +]) + +AT_CLEANUP +]) + +AT_SETUP([check lb-affinity flows]) +AT_KEYWORDS([lb-affinity-flows]) +ovn_start + +ovn-nbctl lr-add R1 +ovn-nbctl set logical_router R1 options:chassis=hv1 +ovn-nbctl lrp-add R1 R1-S0 02:ac:10:01:00:01 10.0.0.1/24 +ovn-nbctl lrp-add R1 R1-S1 02:ac:10:01:01:01 20.0.0.1/24 +ovn-nbctl lrp-add R1 R1-PUB 02:ac:20:01:01:01 172.16.0.1/24 + +ovn-nbctl ls-add S0 +ovn-nbctl lsp-add S0 S0-R1 +ovn-nbctl lsp-set-type S0-R1 router +ovn-nbctl lsp-set-addresses S0-R1 02:ac:10:01:00:01 +ovn-nbctl lsp-set-options S0-R1 router-port=R1-S0 + +ovn-nbctl ls-add S1 +ovn-nbctl lsp-add S1 S1-R1 +ovn-nbctl lsp-set-type S1-R1 router +ovn-nbctl lsp-set-addresses S1-R1 02:ac:10:01:01:01 +ovn-nbctl lsp-set-options S1-R1 router-port=R1-S1 + +# Add load balancers on the logical router R1 +ovn-nbctl lb-add lb0 172.16.0.10:80 10.0.0.2:80,20.0.0.2:80 tcp +ovn-nbctl lr-lb-add R1 lb0 +ovn-nbctl ls-lb-add S0 lb0 + +ovn-sbctl dump-flows S0 > S0flows +ovn-sbctl dump-flows R1 > R1flows + +AT_CAPTURE_FILE([S0flows]) +AT_CAPTURE_FILE([S1flows]) + +AT_CHECK([grep "ls_in_lb_aff_check" S0flows | sort], [0], [dnl + table=11(ls_in_lb_aff_check ), priority=0 , match=(1), action=(next;) +]) +AT_CHECK([grep "ls_in_lb_aff_learn" S0flows | sort], [0], [dnl + table=13(ls_in_lb_aff_learn ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_in_lb_aff_check" R1flows | sort], [0], [dnl + table=6 (lr_in_lb_aff_check ), priority=0 , match=(1), action=(next;) +]) +AT_CHECK([grep "lr_in_lb_aff_learn" R1flows | sort], [0], [dnl + table=8 (lr_in_lb_aff_learn ), priority=0 , match=(1), action=(next;) +]) + +ovn-nbctl --wait=sb set load_balancer lb0 options:affinity_timeout=60 + +ovn-sbctl dump-flows S0 > S0flows +ovn-sbctl dump-flows R1 > R1flows + +AT_CAPTURE_FILE([S0flows]) +AT_CAPTURE_FILE([S1flows]) + +AT_CHECK([grep "ls_in_lb_aff_check" S0flows | sort], [0], [dnl + table=11(ls_in_lb_aff_check ), priority=0 , match=(1), action=(next;) + table=11(ls_in_lb_aff_check ), priority=100 , match=(ct.new && ip4 && reg1 == 172.16.0.10 && reg2[[0..15]] == 80), action=(reg9[[6]] = chk_lb_aff(); next;) +]) +AT_CHECK([grep "ls_in_lb_aff_learn" S0flows | sort], [0], [dnl + table=13(ls_in_lb_aff_learn ), priority=0 , match=(1), action=(next;) + table=13(ls_in_lb_aff_learn ), priority=100 , match=(ct.new && ip4.dst == 10.0.0.2 && tcp.dst == 80), action=(commit_lb_aff(vip = "172.16.0.10:80", backend = "10.0.0.2:80", proto = tcp, timeout = 60); next;) + table=13(ls_in_lb_aff_learn ), priority=100 , match=(ct.new && ip4.dst == 20.0.0.2 && tcp.dst == 80), action=(commit_lb_aff(vip = "172.16.0.10:80", backend = "20.0.0.2:80", proto = tcp, timeout = 60); next;) +]) + +AT_CHECK([grep "lr_in_lb_aff_check" R1flows | sort], [0], [dnl + table=6 (lr_in_lb_aff_check ), priority=0 , match=(1), action=(next;) + table=6 (lr_in_lb_aff_check ), priority=100 , match=(ct.new && ip4 && reg0 == 172.16.0.10 && tcp && reg9[[16..31]] == 80), action=(reg9[[6]] = chk_lb_aff(); next;) +]) +AT_CHECK([grep "lr_in_lb_aff_learn" R1flows | sort], [0], [dnl + table=8 (lr_in_lb_aff_learn ), priority=0 , match=(1), action=(next;) + table=8 (lr_in_lb_aff_learn ), priority=100 , match=(ct.new && ip4.dst == 10.0.0.2 && tcp.dst == 80), action=(commit_lb_aff(vip = "172.16.0.10:80", backend = "10.0.0.2:80", proto = tcp, timeout = 60); next;) + table=8 (lr_in_lb_aff_learn ), priority=100 , match=(ct.new && ip4.dst == 20.0.0.2 && tcp.dst == 80), action=(commit_lb_aff(vip = "172.16.0.10:80", backend = "20.0.0.2:80", proto = tcp, timeout = 60); next;) ]) AT_CLEANUP diff --git a/tests/ovn.at b/tests/ovn.at index f90bd02bb..5cc08e114 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -16090,7 +16090,7 @@ ovn-sbctl dump-flows sw0 > sw0-flows AT_CAPTURE_FILE([sw0-flows]) AT_CHECK([grep -E 'ls_(in|out)_acl' sw0-flows |grep reject| sed 's/table=../table=??/' | sort], [0], [dnl - table=??(ls_out_acl ), priority=2002 , match=(ip), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=23); };) + table=??(ls_out_acl ), priority=2002 , match=(ip), action=(reg0 = 0; reject { /* eth.dst <-> eth.src; ip.dst <-> ip.src; is implicit. */ outport <-> inport; next(pipeline=ingress,table=25); };) ]) @@ -18684,7 +18684,7 @@ wait_for_ports_up ls1-lp_ext1 # There should be a flow in hv2 to drop traffic from ls1-lp_ext1 destined # to router mac. AT_CHECK([as hv2 ovs-ofctl dump-flows br-int \ -table=30,dl_src=f0:00:00:00:00:03,dl_dst=a0:10:00:00:00:01 | \ +table=32,dl_src=f0:00:00:00:00:03,dl_dst=a0:10:00:00:00:01 | \ grep -c "actions=drop"], [0], [1 ]) # Stop ovn-controllers on hv1 and hv3. @@ -20355,7 +20355,7 @@ check_row_count Port_Binding 1 logical_port=sw0-vir virtual_parent=sw0-p1 wait_for_ports_up sw0-vir check ovn-nbctl --wait=hv sync AT_CHECK([test 2 = `cat hv1/ovn-controller.log | grep "pinctrl received packet-in" | \ -grep opcode=BIND_VPORT | grep OF_Table_ID=25 | wc -l`]) +grep opcode=BIND_VPORT | grep OF_Table_ID=27 | wc -l`]) wait_row_count Port_Binding 1 logical_port=sw0-vir6 chassis=$hv1_ch_uuid check_row_count Port_Binding 1 logical_port=sw0-vir6 virtual_parent=sw0-p1 @@ -20404,7 +20404,7 @@ eth_dst=00000000ff01 ip_src=$(ip_to_hex 10 0 0 10) ip_dst=$(ip_to_hex 172 168 0 101) send_icmp_packet 1 1 $eth_src $eth_dst $ip_src $ip_dst c4c9 0000000000000000000000 -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int metadata=0x$lr0_dp_key | awk '/table=26, n_packets=1, n_bytes=45/{print $7" "$8}'],[0],[dnl +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int metadata=0x$lr0_dp_key | awk '/table=28, n_packets=1, n_bytes=45/{print $7" "$8}'],[0],[dnl priority=80,ip,reg15=0x3,metadata=0x3,nw_src=10.0.0.10 actions=drop ]) @@ -26401,7 +26401,7 @@ ovn-sbctl dump-flows > sbflows AT_CAPTURE_FILE([sbflows]) AT_CAPTURE_FILE([offlows]) OVS_WAIT_UNTIL([ - as hv1 ovs-ofctl dump-flows br-int table=21 > offlows + as hv1 ovs-ofctl dump-flows br-int table=23 > offlows test $(grep -c "load:0x64->NXM_NX_PKT_MARK" offlows) = 1 && \ test $(grep -c "load:0x3->NXM_NX_PKT_MARK" offlows) = 1 && \ test $(grep -c "load:0x4->NXM_NX_PKT_MARK" offlows) = 1 && \ @@ -26494,12 +26494,12 @@ send_ipv4_pkt hv1 hv1-vif1 505400000003 00000000ff01 \ $(ip_to_hex 10 0 0 3) $(ip_to_hex 172 168 0 120) OVS_WAIT_UNTIL([ - test 1 -eq $(as hv1 ovs-ofctl dump-flows br-int table=21 | \ + test 1 -eq $(as hv1 ovs-ofctl dump-flows br-int table=23 | \ grep "load:0x2->NXM_NX_PKT_MARK" -c) ]) AT_CHECK([ - test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=21 | \ + test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=23 | \ grep "load:0x64->NXM_NX_PKT_MARK" -c) ]) @@ -27191,23 +27191,23 @@ check ovn-nbctl --wait=hv sync # Ensure ECMP symmetric reply flows are not present on any hypervisor. AT_CHECK([ - test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=15 | \ + test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=17 | \ grep "priority=100" | \ grep "ct(commit,zone=NXM_NX_REG11\\[[0..15\\]],exec(move:NXM_OF_ETH_SRC\\[[\\]]->NXM_NX_CT_LABEL\\[[32..79\\]],load:0x[[0-9]]->NXM_NX_CT_LABEL\\[[80..95\\]]))" -c) ]) AT_CHECK([ - test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=21 | \ + test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=25 | \ grep "priority=200" | \ grep "actions=move:NXM_NX_CT_LABEL\\[[32..79\\]]->NXM_OF_ETH_DST\\[[\\]]" -c) ]) AT_CHECK([ - test 0 -eq $(as hv2 ovs-ofctl dump-flows br-int table=15 | \ + test 0 -eq $(as hv2 ovs-ofctl dump-flows br-int table=17 | \ grep "priority=100" | \ grep "ct(commit,zone=NXM_NX_REG11\\[[0..15\\]],exec(move:NXM_OF_ETH_SRC\\[[\\]]->NXM_NX_CT_LABEL\\[[32..79\\]],load:0x[[0-9]]->NXM_NX_CT_LABEL\\[[80..95\\]]))" -c) ]) AT_CHECK([ - test 0 -eq $(as hv2 ovs-ofctl dump-flows br-int table=21 | \ + test 0 -eq $(as hv2 ovs-ofctl dump-flows br-int table=25 | \ grep "priority=200" | \ grep "actions=move:NXM_NX_CT_LABEL\\[[32..79\\]]->NXM_OF_ETH_DST\\[[\\]]" -c) ]) @@ -27225,11 +27225,11 @@ AT_CAPTURE_FILE([hv2flows]) AT_CHECK([ for hv in 1 2; do - grep table=15 hv${hv}flows | \ + grep table=17 hv${hv}flows | \ grep "priority=100" | \ grep -c "ct(commit,zone=NXM_NX_REG11\\[[0..15\\]],.*exec(move:NXM_OF_ETH_SRC\\[[\\]]->NXM_NX_CT_LABEL\\[[32..79\\]],load:0x[[0-9]]->NXM_NX_CT_MARK\\[[16..31\\]]))" - grep table=23 hv${hv}flows | \ + grep table=25 hv${hv}flows | \ grep "priority=200" | \ grep -c "move:NXM_NX_CT_LABEL\\[[\\]]->NXM_NX_XXREG1\\[[\\]],move:NXM_NX_XXREG1\\[[32..79\\]]->NXM_OF_ETH_DST" done; :], [0], [dnl @@ -27317,23 +27317,23 @@ check ovn-nbctl --wait=hv sync # Ensure ECMP symmetric reply flows are not present on any hypervisor. AT_CHECK([ - test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=15 | \ + test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=17 | \ grep "priority=100" | \ grep "ct(commit,zone=NXM_NX_REG11\\[[0..15\\]],exec(move:NXM_OF_ETH_SRC\\[[\\]]->NXM_NX_CT_LABEL\\[[32..79\\]],load:0x[[0-9]]->NXM_NX_CT_LABEL\\[[80..95\\]]))" -c) ]) AT_CHECK([ - test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=21 | \ + test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=25 | \ grep "priority=200" | \ grep "actions=move:NXM_NX_CT_LABEL\\[[32..79\\]]->NXM_OF_ETH_DST\\[[\\]]" -c) ]) AT_CHECK([ - test 0 -eq $(as hv2 ovs-ofctl dump-flows br-int table=15 | \ + test 0 -eq $(as hv2 ovs-ofctl dump-flows br-int table=17 | \ grep "priority=100" | \ grep "ct(commit,zone=NXM_NX_REG11\\[[0..15\\]],exec(move:NXM_OF_ETH_SRC\\[[\\]]->NXM_NX_CT_LABEL\\[[32..79\\]],load:0x[[0-9]]->NXM_NX_CT_MARK\\[[16..31\\]]))" -c) ]) AT_CHECK([ - test 0 -eq $(as hv2 ovs-ofctl dump-flows br-int table=21 | \ + test 0 -eq $(as hv2 ovs-ofctl dump-flows br-int table=25 | \ grep "priority=200" | \ grep "actions=move:NXM_NX_CT_LABEL\\[[\\]]->NXM_OF_ETH_DST\\[[\\]]" -c) ]) @@ -27350,11 +27350,11 @@ AT_CAPTURE_FILE([hv2flows]) AT_CHECK([ for hv in 1 2; do - grep table=15 hv${hv}flows | \ + grep table=17 hv${hv}flows | \ grep "priority=100" | \ grep -c "ct(commit,zone=NXM_NX_REG11\\[[0..15\\]],.*exec(move:NXM_OF_ETH_SRC\\[[\\]]->NXM_NX_CT_LABEL\\[[32..79\\]],load:0x[[0-9]]->NXM_NX_CT_MARK\\[[16..31\\]]))" - grep table=23 hv${hv}flows | \ + grep table=25 hv${hv}flows | \ grep "priority=200" | \ grep -c "move:NXM_NX_CT_LABEL\\[[\\]]->NXM_NX_XXREG1\\[[\\]],move:NXM_NX_XXREG1\\[[32..79\\]]->NXM_OF_ETH_DST" done; :], [0], [dnl @@ -27818,7 +27818,7 @@ AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep "actions=controller" | grep ]) # The packet should've been dropped in the lr_in_arp_resolve stage. -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep -E "table=23, n_packets=1,.* priority=1,ip,metadata=0x${sw_key},nw_dst=10.0.1.1 actions=drop" -c], [0], [dnl +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep -E "table=25, n_packets=1,.* priority=1,ip,metadata=0x${sw_key},nw_dst=10.0.1.1 actions=drop" -c], [0], [dnl 1 ]) @@ -31331,15 +31331,15 @@ done check ovn-nbctl --wait=hv sync # hv0 should see flows for lsp1 but not lsp2 -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=25 | grep 10.0.1.2], [0], [ignore]) -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=25 | grep 10.0.2.2], [1]) +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=27 | grep 10.0.1.2], [0], [ignore]) +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=27 | grep 10.0.2.2], [1]) # hv2 should see flows for lsp2 but not lsp1 -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=25 | grep 10.0.2.2], [0], [ignore]) -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=25 | grep 10.0.1.2], [1]) +AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=27 | grep 10.0.2.2], [0], [ignore]) +AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=27 | grep 10.0.1.2], [1]) # Change lrp_lr_ls1 to a regular lrp, hv2 should see flows for lsp1 check ovn-nbctl --wait=hv lrp-del-gateway-chassis lrp_lr_ls1 hv1 -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=25 | grep 10.0.1.2], [0], [ignore]) +AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=27 | grep 10.0.1.2], [0], [ignore]) # Change it back, and trigger recompute to make sure extra flows are removed # from hv2 (recompute is needed because currently I-P adds local datapaths but @@ -31347,11 +31347,11 @@ AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=25 | grep 10.0.1.2], [0], [ig check ovn-nbctl --wait=hv lrp-set-gateway-chassis lrp_lr_ls1 hv1 1 as hv2 check ovn-appctl -t ovn-controller recompute ovn-nbctl --wait=hv sync -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=25 | grep 10.0.1.2], [1]) +AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=27 | grep 10.0.1.2], [1]) # Enable dnat_and_snat on lr, and now hv2 should see flows for lsp1. AT_CHECK([ovn-nbctl --wait=hv --gateway-port=lrp_lr_ls1 lr-nat-add lr dnat_and_snat 192.168.0.1 10.0.1.3 lsp1 f0:00:00:00:00:03]) -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=25 | grep 10.0.1.2], [0], [ignore]) +AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=27 | grep 10.0.1.2], [0], [ignore]) OVN_CLEANUP([hv1],[hv2]) AT_CLEANUP diff --git a/tests/system-ovn.at b/tests/system-ovn.at index 20c058415..b0273331d 100644 --- a/tests/system-ovn.at +++ b/tests/system-ovn.at @@ -8323,7 +8323,7 @@ ovn-sbctl list ip_multicast wait_igmp_flows_installed() { - OVS_WAIT_UNTIL([ovs-ofctl dump-flows br-int table=31 | \ + OVS_WAIT_UNTIL([ovs-ofctl dump-flows br-int table=33 | \ grep 'priority=90' | grep "nw_dst=$1"]) } @@ -8477,7 +8477,470 @@ OVS_APP_EXIT_AND_WAIT([NORTHD_TYPE]) as OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d /connection dropped.*/d"]) +AT_CLEANUP +]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([load balancing affinity sessions - IPv4]) +AT_KEYWORDS([ovnlb]) + +CHECK_CONNTRACK() +CHECK_CONNTRACK_NAT() +ovn_start +OVS_TRAFFIC_VSWITCHD_START() +ADD_BR([br-int]) + +# Set external-ids in br-int needed for ovn-controller +ovs-vsctl \ + -- set Open_vSwitch . external-ids:system-id=hv1 \ + -- set Open_vSwitch . external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ + -- set bridge br-int fail-mode=secure other-config:disable-in-band=true + +# Start ovn-controller +start_daemon ovn-controller + +# Logical network: +# Two LRs - R1 and R2 that are connected to each other via LS "join" +# in 20.0.0.0/24 network. R1 has switchess foo (192.168.1.0/24) and +# bar (192.168.2.0/24) connected to it. R2 has alice (172.16.1.0/24) connected +# to it. R2 is a gateway router on which we add load-balancing rules. +# +# foo -- R1 -- join - R2 -- alice +# | +# bar ---- + +ovn-nbctl create Logical_Router name=R1 +ovn-nbctl create Logical_Router name=R2 options:chassis=hv1 + +ovn-nbctl ls-add foo +ovn-nbctl ls-add bar +ovn-nbctl ls-add alice +ovn-nbctl ls-add join + +# Connect foo to R1 +ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24 +ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \ + type=router options:router-port=foo addresses=\"00:00:01:01:02:03\" + +# Connect bar to R1 +ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 192.168.2.1/24 +ovn-nbctl lsp-add bar rp-bar -- set Logical_Switch_Port rp-bar \ + type=router options:router-port=bar addresses=\"00:00:01:01:02:04\" + +# Connect alice to R2 +ovn-nbctl lrp-add R2 alice 00:00:02:01:02:03 172.16.1.1/24 +ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \ + type=router options:router-port=alice addresses=\"00:00:02:01:02:03\" + +# Connect R1 to join +ovn-nbctl lrp-add R1 R1_join 00:00:04:01:02:03 20.0.0.1/24 +ovn-nbctl lsp-add join r1-join -- set Logical_Switch_Port r1-join \ + type=router options:router-port=R1_join addresses='"00:00:04:01:02:03"' + +# Connect R2 to join +ovn-nbctl lrp-add R2 R2_join 00:00:04:01:02:04 20.0.0.2/24 +ovn-nbctl lsp-add join r2-join -- set Logical_Switch_Port r2-join \ + type=router options:router-port=R2_join addresses='"00:00:04:01:02:04"' + +# Static routes. +ovn-nbctl lr-route-add R1 172.16.1.0/24 20.0.0.2 +ovn-nbctl lr-route-add R2 192.168.0.0/16 20.0.0.1 + +# Logical port 'foo1' in switch 'foo'. +ADD_NAMESPACES(foo1) +ADD_VETH(foo1, foo1, br-int, "192.168.1.2/24", "f0:00:00:01:02:03", \ + "192.168.1.1") +ovn-nbctl lsp-add foo foo1 \ +-- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2" + +# Logical port 'alice1' in switch 'alice'. +ADD_NAMESPACES(alice1) +ADD_VETH(alice1, alice1, br-int, "172.16.1.2/24", "f0:00:00:01:02:04", \ + "172.16.1.1") +ovn-nbctl lsp-add alice alice1 \ +-- lsp-set-addresses alice1 "f0:00:00:01:02:04 172.16.1.2" + +# Logical port 'bar1' in switch 'bar'. +ADD_NAMESPACES(bar1) +ADD_VETH(bar1, bar1, br-int, "192.168.2.2/24", "f0:00:00:01:02:05", \ +"192.168.2.1") +ovn-nbctl lsp-add bar bar1 \ +-- lsp-set-addresses bar1 "f0:00:00:01:02:05 192.168.2.2" + +ADD_NAMESPACES(bar2) +ADD_VETH(bar2, bar2, br-int, "192.168.2.3/24", "e0:00:00:01:02:05", \ +"192.168.2.1") +ovn-nbctl lsp-add bar bar2 \ +-- lsp-set-addresses bar2 "e0:00:00:01:02:05 192.168.2.3" + +# Config OVN load-balancer with a VIP. + +ovn-nbctl lb-add lb0 172.16.1.100:8080 192.168.1.2:80,192.168.2.2:80 +ovn-nbctl lr-lb-add R2 lb0 + +# Start webservers in 'foo1', 'bar1'. +OVS_START_L7([foo1], [http]) +OVS_START_L7([bar1], [http]) + +# Wait for ovn-controller to catch up. +ovn-nbctl --wait=hv sync +OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-groups br-int | \ +grep 'nat(dst=192.168.2.2:80)']) + +dnl Should work with the virtual IP address through NAT +for i in $(seq 1 20); do + echo Request $i + NS_CHECK_EXEC([alice1], [wget 172.16.1.100:8080 -t 5 -T 1 --retry-connrefused -v -o wget$i.log]) +done + +dnl Each server should have at least one connection. +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.100) | +sed -e 's/zone=[[0-9]]*/zone=/'], [0], [dnl +tcp,orig=(src=172.16.1.2,dst=172.16.1.100,sport=,dport=),reply=(src=192.168.1.2,dst=172.16.1.2,sport=,dport=),zone=,mark=2,protoinfo=(state=) +tcp,orig=(src=172.16.1.2,dst=172.16.1.100,sport=,dport=),reply=(src=192.168.2.2,dst=172.16.1.2,sport=,dport=),zone=,mark=2,protoinfo=(state=) +]) + +# Flush conntrack entries for easier output parsing of next test. +AT_CHECK([ovs-appctl dpctl/flush-conntrack]) +# Enable lb affinity +ovn-nbctl --wait=sb set load_balancer lb0 options:affinity_timeout=60 + +for i in $(seq 1 20); do + echo Request $i + NS_CHECK_EXEC([alice1], [wget 172.16.1.100:8080 -t 5 -T 1 --retry-connrefused -v -o wget$i.log]) + sleep 1 +done + +dnl here we should have just one entry in the ct table +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.100) | +sed -e 's/zone=[[0-9]]*/zone=/; s/src=192.168.[[0-9]].2/src=192.168..2/'], [0], [dnl +tcp,orig=(src=172.16.1.2,dst=172.16.1.100,sport=,dport=),reply=(src=192.168..2,dst=172.16.1.2,sport=,dport=),zone=,mark=2,protoinfo=(state=) +]) + +AT_CHECK([ovs-ofctl dump-flows br-int table=78 |grep cookie |sed -e 's/duration=[[0-9]]*.[[0-9]]*s/duration=/; s/load:0xc0a80[[0-9]]02/load:0xc0a8002/; s/n_packets=[[0-9]]*/n_packets=/; s/n_bytes=[[0-9]]*/n_bytes=/; s/idle_age=[[0-9]]*/idle_age=/; s/hard_age=[[0-9]]*, //'], [0], [dnl + cookie=0x0, duration=, table=78, n_packets=, n_bytes=, idle_timeout=60, idle_age=, tcp,metadata=0x2,nw_src=172.16.1.2,nw_dst=172.16.1.100,tp_dst=8080 actions=load:0x1->NXM_NX_REG10[[14]],load:0xc0a8002->NXM_NX_REG4[[]],load:0x50->NXM_NX_REG8[[0..15]] +]) + +check_affinity_flows () { +n1=$(ovs-ofctl dump-flows br-int table=15 |awk '/priority=150,ip,reg4=0xc0a80102/{print substr($4,11,length($4)-11)}') +n2=$(ovs-ofctl dump-flows br-int table=15 |awk '/priority=150,ip,reg4=0xc0a80202/{print substr($4,11,length($4)-11)}') +[[ $n1 -gt 0 -a $n2 -eq 0 ]] || [[ $n1 -eq 0 -a $n2 -gt 0 ]] +echo $? +} +AT_CHECK([test $(check_affinity_flows) -eq 0]) + +# Flush conntrack entries for easier output parsing of next test. +AT_CHECK([ovs-appctl dpctl/flush-conntrack]) + +ovn-nbctl lb-add lb1 172.16.1.101:8080 192.168.1.2:80,192.168.2.2:80 +# Enable lb affinity +ovn-nbctl --wait=sb set load_balancer lb1 options:affinity_timeout=1 +ovn-nbctl lr-lb-add R2 lb1 + +# check we use both backends +for i in $(seq 1 20); do + echo Request $i + NS_CHECK_EXEC([alice1], [wget 172.16.1.101:8080 -t 5 -T 1 --retry-connrefused -v -o wget$i.log]) + OVS_WAIT_UNTIL([test "$(ovs-ofctl dump-flows br-int table=78 |grep 172.16.1.101)" = ""]) +done + +dnl Each server should have at least one connection. +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.101) | +sed -e 's/zone=[[0-9]]*/zone=/'], [0], [dnl +tcp,orig=(src=172.16.1.2,dst=172.16.1.101,sport=,dport=),reply=(src=192.168.1.2,dst=172.16.1.2,sport=,dport=),zone=,mark=2,protoinfo=(state=) +tcp,orig=(src=172.16.1.2,dst=172.16.1.101,sport=,dport=),reply=(src=192.168.2.2,dst=172.16.1.2,sport=,dport=),zone=,mark=2,protoinfo=(state=) +]) + +# Flush conntrack entries for easier output parsing of next test. +AT_CHECK([ovs-appctl dpctl/flush-conntrack]) + +OVS_START_L7([bar2], [http]) + +ovn-nbctl lb-add lb2 192.168.2.100:8080 192.168.2.2:80,192.168.2.3:80 +ovn-nbctl --wait=sb set load_balancer lb2 options:affinity_timeout=60 +ovn-nbctl ls-lb-add foo lb2 + +for i in $(seq 1 20); do + echo Request $i + NS_CHECK_EXEC([foo1], [wget 192.168.2.100:8080 -t 5 -T 1 --retry-connrefused -v -o wget$i.log]) + sleep 1 +done + +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(192.168.2.100) | +sed -e 's/zone=[[0-9]]*/zone=/; s/src=192.168.2.[[0-9]]/src=192.168.2./'], [0], [dnl +tcp,orig=(src=192.168.1.2,dst=192.168.2.100,sport=,dport=),reply=(src=192.168.2.,dst=192.168.1.2,sport=,dport=),zone=,mark=2,protoinfo=(state=) +]) + +# Flush conntrack entries for easier output parsing of next test. +AT_CHECK([ovs-appctl dpctl/flush-conntrack]) + +ovn-nbctl lb-add lb3 192.168.2.101:8080 192.168.2.2:80,192.168.2.3:80 +ovn-nbctl --wait=sb set load_balancer lb3 options:affinity_timeout=1 +ovn-nbctl ls-lb-add foo lb3 +# Flush conntrack entries for easier output parsing of next test. +AT_CHECK([ovs-appctl dpctl/flush-conntrack]) + +for i in $(seq 1 20); do + echo Request $i + NS_CHECK_EXEC([foo1], [wget 192.168.2.101:8080 -t 5 -T 1 --retry-connrefused -v -o wget$i.log]) + OVS_WAIT_UNTIL([test "$(ovs-ofctl dump-flows br-int table=78 |grep 192.168.2.101)" = ""]) +done + +dnl Each server should have at least one connection. +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(192.168.2.101) | +sed -e 's/zone=[[0-9]]*/zone=/'], [0], [dnl +tcp,orig=(src=192.168.1.2,dst=192.168.2.101,sport=,dport=),reply=(src=192.168.2.2,dst=192.168.1.2,sport=,dport=),zone=,mark=2,protoinfo=(state=) +tcp,orig=(src=192.168.1.2,dst=192.168.2.101,sport=,dport=),reply=(src=192.168.2.3,dst=192.168.1.2,sport=,dport=),zone=,mark=2,protoinfo=(state=) +]) + +OVS_APP_EXIT_AND_WAIT([ovn-controller]) + +as ovn-sb +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) + +as ovn-nb +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) + +as northd +OVS_APP_EXIT_AND_WAIT([NORTHD_TYPE]) + +as +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d +/connection dropped.*/d"]) +AT_CLEANUP +]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([load balancing affinity sessions - IPv6]) +AT_KEYWORDS([ovnlb]) + +CHECK_CONNTRACK() +CHECK_CONNTRACK_NAT() +ovn_start +OVS_TRAFFIC_VSWITCHD_START() +ADD_BR([br-int]) + +# Set external-ids in br-int needed for ovn-controller +ovs-vsctl \ + -- set Open_vSwitch . external-ids:system-id=hv1 \ + -- set Open_vSwitch . external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ + -- set bridge br-int fail-mode=secure other-config:disable-in-band=true + +# Start ovn-controller +start_daemon ovn-controller + +# Logical network: +# Two LRs - R1 and R2 that are connected to each other via LS "join" +# in fd20::/64 network. R1 has switchess foo (fd11::/64) and +# bar (fd12::/64) connected to it. R2 has alice (fd72::/64) connected +# to it. R2 is a gateway router on which we add load-balancing rules. +# +# foo -- R1 -- join - R2 -- alice +# | +# bar ---- + +ovn-nbctl create Logical_Router name=R1 +ovn-nbctl create Logical_Router name=R2 options:chassis=hv1 + +ovn-nbctl ls-add foo +ovn-nbctl ls-add bar +ovn-nbctl ls-add alice +ovn-nbctl ls-add join + +# Connect foo to R1 +ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 fd11::1/64 +ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \ + type=router options:router-port=foo addresses=\"00:00:01:01:02:03\" + +# Connect bar to R1 +ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 fd12::1/64 +ovn-nbctl lsp-add bar rp-bar -- set Logical_Switch_Port rp-bar \ + type=router options:router-port=bar addresses=\"00:00:01:01:02:04\" + +# Connect alice to R2 +ovn-nbctl lrp-add R2 alice 00:00:02:01:02:03 fd72::1/64 +ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \ + type=router options:router-port=alice addresses=\"00:00:02:01:02:03\" + +# Connect R1 to join +ovn-nbctl lrp-add R1 R1_join 00:00:04:01:02:03 fd20::1/64 +ovn-nbctl lsp-add join r1-join -- set Logical_Switch_Port r1-join \ + type=router options:router-port=R1_join addresses='"00:00:04:01:02:03"' + +# Connect R2 to join +ovn-nbctl lrp-add R2 R2_join 00:00:04:01:02:04 fd20::2/64 +ovn-nbctl lsp-add join r2-join -- set Logical_Switch_Port r2-join \ + type=router options:router-port=R2_join addresses='"00:00:04:01:02:04"' + +# Static routes. +ovn-nbctl lr-route-add R1 fd72::/64 fd20::2 +ovn-nbctl lr-route-add R2 fd11::/64 fd20::1 +ovn-nbctl lr-route-add R2 fd12::/64 fd20::1 + +# Logical port 'foo1' in switch 'foo'. +ADD_NAMESPACES(foo1) +ADD_VETH(foo1, foo1, br-int, "fd11::2/64", "f0:00:00:01:02:03", \ + "fd11::1") +OVS_WAIT_UNTIL([test "$(ip -n foo1 a | grep fd11::2 | grep tentative)" = ""]) +ovn-nbctl lsp-add foo foo1 \ +-- lsp-set-addresses foo1 "f0:00:00:01:02:03 fd11::2" + +# Logical port 'alice1' in switch 'alice'. +ADD_NAMESPACES(alice1) +ADD_VETH(alice1, alice1, br-int, "fd72::2/64", "f0:00:00:01:02:04", \ + "fd72::1") +OVS_WAIT_UNTIL([test "$(ip -n alice1 a | grep fd72::2 | grep tentative)" = ""]) +ovn-nbctl lsp-add alice alice1 \ +-- lsp-set-addresses alice1 "f0:00:00:01:02:04 fd72::2" + +# Logical port 'bar1' in switch 'bar'. +ADD_NAMESPACES(bar1) +ADD_VETH(bar1, bar1, br-int, "fd12::2/64", "f0:00:00:01:02:05", \ +"fd12::1") +OVS_WAIT_UNTIL([test "$(ip -n bar1 a | grep fd12::2 | grep tentative)" = ""]) +ovn-nbctl lsp-add bar bar1 \ +-- lsp-set-addresses bar1 "f0:00:00:01:02:05 fd12::2" + +ADD_NAMESPACES(bar2) +ADD_VETH(bar2, bar2, br-int, "fd12::3/64", "e0:00:00:01:02:05", \ +"fd12::1") +OVS_WAIT_UNTIL([test "$(ip -n bar2 a | grep fd12::3 | grep tentative)" = ""]) +ovn-nbctl lsp-add bar bar2 \ +-- lsp-set-addresses bar2 "e0:00:00:01:02:05 fd12::3" + +ovn-nbctl lb-add lb0 [[fd30::1]]:8080 [[fd11::2]]:80,[[fd12::2]]:80 +ovn-nbctl lr-lb-add R2 lb0 + + +# Wait for ovn-controller to catch up. +ovn-nbctl --wait=hv sync +OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-groups br-int | \ +grep 'nat(dst=\[[fd11::2\]]:80)']) +# Start webservers in 'foo1', 'bar1'. +OVS_START_L7([foo1], [http6]) +OVS_START_L7([bar1], [http6]) + +dnl Should work with the virtual IP address through NAT +for i in $(seq 1 20); do + echo Request $i + NS_CHECK_EXEC([alice1], [wget http://[[fd30::1]]:8080 -t 5 -T 1 --retry-connrefused -v -o wget$i.log]) +done + +dnl Each server should have at least one connection. +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(fd30::1) | grep -v fe80 | +sed -e 's/zone=[[0-9]]*/zone=/'], [0], [dnl +tcp,orig=(src=fd72::2,dst=fd30::1,sport=,dport=),reply=(src=fd11::2,dst=fd72::2,sport=,dport=),zone=,mark=2,protoinfo=(state=) +tcp,orig=(src=fd72::2,dst=fd30::1,sport=,dport=),reply=(src=fd12::2,dst=fd72::2,sport=,dport=),zone=,mark=2,protoinfo=(state=) +]) + +# Flush conntrack entries for easier output parsing of next test. +AT_CHECK([ovs-appctl dpctl/flush-conntrack]) +# Enable lb affinity +ovn-nbctl --wait=sb set load_balancer lb0 options:affinity_timeout=60 + +for i in $(seq 1 20); do + echo Request $i + NS_CHECK_EXEC([alice1], [wget http://[[fd30::1]]:8080 -t 5 -T 1 --retry-connrefused -v -o wget$i.log]) + sleep 1 +done + +dnl here we should have just one entry in the ct table +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(fd30::1) | grep -v fe80 | +sed -e 's/zone=[[0-9]]*/zone=/; s/src=fd1[[0-9]]::2/src=fd1::2/'], [0], [dnl +tcp,orig=(src=fd72::2,dst=fd30::1,sport=,dport=),reply=(src=fd1::2,dst=fd72::2,sport=,dport=),zone=,mark=2,protoinfo=(state=) +]) + +AT_CHECK([ovs-ofctl dump-flows br-int table=78 |grep cookie |sed -e 's/duration=[[0-9]]*.[[0-9]]*s/duration=/; s/load:0xfd1[[0-9]]000000000000/load:0xfd1000000000000/; s/n_packets=[[0-9]]*/n_packets=/; s/n_bytes=[[0-9]]*/n_bytes=/; s/idle_age=[[0-9]]*/idle_age=/; s/hard_age=[[0-9]]*, //'], [0], [dnl + cookie=0x0, duration=, table=78, n_packets=, n_bytes=, idle_timeout=60, idle_age=, tcp6,metadata=0x2,ipv6_src=fd72::2,ipv6_dst=fd30::1,tp_dst=8080 actions=load:0x1->NXM_NX_REG10[[14]],load:0x2->NXM_NX_XXREG1[[0..63]],load:0xfd1000000000000->NXM_NX_XXREG1[[64..127]],load:0x50->NXM_NX_REG8[[0..15]] +]) + +check_affinity_flows () { +n1=$(ovs-ofctl dump-flows br-int table=15 |awk '/priority=150,ipv6,reg4=0xfd110000/{print substr($4,11,length($4)-11)}') +n2=$(ovs-ofctl dump-flows br-int table=15 |awk '/priority=150,ipv6,reg4=0xfd120000/{print substr($4,11,length($4)-11)}') +[[ $n1 -gt 0 -a $n2 -eq 0 ]] || [[ $n1 -eq 0 -a $n2 -gt 0 ]] +echo $? +} +AT_CHECK([test $(check_affinity_flows) -eq 0]) + +# Flush conntrack entries for easier output parsing of next test. +AT_CHECK([ovs-appctl dpctl/flush-conntrack]) + +ovn-nbctl lb-add lb1 [[fd30::2]]:8080 [[fd11::2]]:80,[[fd12::2]]:80 +# Enable lb affinity +ovn-nbctl --wait=sb set load_balancer lb1 options:affinity_timeout=1 +ovn-nbctl lr-lb-add R2 lb1 + +# check we use both backends +for i in $(seq 1 20); do + echo Request $i + NS_CHECK_EXEC([alice1], [wget http://[[fd30::2]]:8080 -t 5 -T 1 --retry-connrefused -v -o wget$i.log]) + OVS_WAIT_UNTIL([test "$(ovs-ofctl dump-flows br-int table=78 |grep fd30::2)" = ""]) +done + +dnl Each server should have at least one connection. +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(fd30::2) | grep -v fe80 | +sed -e 's/zone=[[0-9]]*/zone=/'], [0], [dnl +tcp,orig=(src=fd72::2,dst=fd30::2,sport=,dport=),reply=(src=fd11::2,dst=fd72::2,sport=,dport=),zone=,mark=2,protoinfo=(state=) +tcp,orig=(src=fd72::2,dst=fd30::2,sport=,dport=),reply=(src=fd12::2,dst=fd72::2,sport=,dport=),zone=,mark=2,protoinfo=(state=) +]) + +# Flush conntrack entries for easier output parsing of next test. +AT_CHECK([ovs-appctl dpctl/flush-conntrack]) + +OVS_START_L7([bar2], [http6]) + +ovn-nbctl lb-add lb2 [[fd12::a]]:8080 [[fd12::2]]:80,[[fd12::3]]:80 +ovn-nbctl --wait=sb set load_balancer lb2 options:affinity_timeout=60 +ovn-nbctl ls-lb-add foo lb2 + +for i in $(seq 1 20); do + echo Request $i + NS_CHECK_EXEC([foo1], [wget http://[[fd12::a]]:8080 -t 5 -T 1 --retry-connrefused -v -o wget$i.log]) + sleep 1 +done + +dnl here we should have just one entry in the ct table +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(fd12::a) | grep -v fe80 | +sed -e 's/zone=[[0-9]]*/zone=/; s/src=fd12::[[0-9]]/src=fd12::/'], [0], [dnl +tcp,orig=(src=fd11::2,dst=fd12::a,sport=,dport=),reply=(src=fd12::,dst=fd11::2,sport=,dport=),zone=,mark=2,protoinfo=(state=) +]) + +AT_CHECK([ovs-appctl dpctl/flush-conntrack]) + +ovn-nbctl lb-add lb3 [[fd12::b]]:8080 [[fd12::2]]:80,[[fd12::3]]:80 +ovn-nbctl --wait=sb set load_balancer lb3 options:affinity_timeout=1 +ovn-nbctl ls-lb-add foo lb3 + +for i in $(seq 1 20); do + echo Request $i + NS_CHECK_EXEC([foo1], [wget http://[[fd12::b]]:8080 -t 5 -T 1 --retry-connrefused -v -o wget$i.log]) + OVS_WAIT_UNTIL([test "$(ovs-ofctl dump-flows br-int table=78 |grep fd12::b)" = ""]) +done + +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(fd12::b) | grep -v fe80 | +sed -e 's/zone=[[0-9]]*/zone=/'], [0], [dnl +tcp,orig=(src=fd11::2,dst=fd12::b,sport=,dport=),reply=(src=fd12::2,dst=fd11::2,sport=,dport=),zone=,mark=2,protoinfo=(state=) +tcp,orig=(src=fd11::2,dst=fd12::b,sport=,dport=),reply=(src=fd12::3,dst=fd11::2,sport=,dport=),zone=,mark=2,protoinfo=(state=) +]) + +OVS_APP_EXIT_AND_WAIT([ovn-controller]) + +as ovn-sb +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) + +as ovn-nb +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) + +as northd +OVS_APP_EXIT_AND_WAIT([NORTHD_TYPE]) + +as +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d"]) AT_CLEANUP ])