From patchwork Wed Feb 7 15:56:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Kalcok X-Patchwork-Id: 1896191 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=gHxUc+ZQ; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TVPrc04T1z23gM for ; Thu, 8 Feb 2024 02:57:19 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id D45CD6141F; Wed, 7 Feb 2024 15:57:17 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KoLktjGVC7Iq; Wed, 7 Feb 2024 15:57:16 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org AA29E613F2 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key, unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=gHxUc+ZQ Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id AA29E613F2; Wed, 7 Feb 2024 15:57:16 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7F6D5C0072; Wed, 7 Feb 2024 15:57:16 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id CDD81C0037 for ; Wed, 7 Feb 2024 15:57:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id AFF6283BAD for ; Wed, 7 Feb 2024 15:57:15 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GQCF91_S7AtK for ; Wed, 7 Feb 2024 15:57:14 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=185.125.188.120; helo=smtp-relay-canonical-0.canonical.com; envelope-from=martin.kalcok@canonical.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp1.osuosl.org 63E7B83BA7 Authentication-Results: smtp1.osuosl.org; dmarc=pass (p=none dis=none) header.from=canonical.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 63E7B83BA7 Authentication-Results: smtp1.osuosl.org; dkim=pass (2048-bit key, unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=gHxUc+ZQ Received: from smtp-relay-canonical-0.canonical.com (smtp-relay-canonical-0.canonical.com [185.125.188.120]) by smtp1.osuosl.org (Postfix) with ESMTPS id 63E7B83BA7 for ; Wed, 7 Feb 2024 15:57:13 +0000 (UTC) Received: from localhost.localdomain (2.general.kalcok.uk.vpn [10.172.196.41]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-0.canonical.com (Postfix) with ESMTPSA id 5EFFB3F2FA; Wed, 7 Feb 2024 15:57:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1707321431; bh=nWkSea+SBOA8X8u6hkvPkoWZvmePRB1LtmB18FmoJFM=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=gHxUc+ZQGsNnBXE9DmDD/TTqEosydrd5u5nAaTBL2FXu0SQNVvCRU3BzPjBIBlQOZ sBGL0TwBRbP4RcDAULk2Fzi+oMn+2ivvh6GVrrHULqiZGoKn9m28uQiXcBPmawMpzm jqYtM0VTjZQozV5Ilc3m/pQOyP463oV36HVHLk2YIKri8GrWqfjlvf7rxMmK6XVZhf lFLwfqmMzQ2PCMn6aU3WDMCuUtOvViWRjeleKgrSjn/MOAcOw19RYQ/HVtp0NYjNSN QqIbLYm2gi6tK5F1xI77P9KjkYP0PS/qb2CYHnkYLPThupy8HdGb1hWHqjGEDAHz6k 57FzoHRX3uFXg== From: Martin Kalcok To: dev@openvswitch.org Date: Wed, 7 Feb 2024 16:56:27 +0100 Message-Id: <20240207155627.806188-1-martin.kalcok@canonical.com> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 Subject: [ovs-dev] [RFC ovn] northd.c: Fix direct access to SNATed networks. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" When a machine from an external network tries to access machine in the OVN's internal network with SNAT enabled, using protocol like TCP, it receives connection reset after first packets are successfully exchanged. This setup may be a bit unusual and requires that whatever is responsible for routing in the external network, routes packets for OVN's internal network via LR's external port IP. However it is something that used to work in ML2/OVS Openstack deployments and got broken after upgrade to ML2/OVN. To demonstrate what the traffic looked like from the point of view of the external machine, lets say we have: * E (IP of machine in the external network) * I (IP of machine in OVN's internal network) * LR (IP of OVN's Logical Router Port connected to external network) The traffic looks like this: * [SYN] E -> I * [SYN,ACK] I -> E * [ACK] E -> I * [PSH,ACK] E -> I * [ACK] LR -> E Connection gets reseted after it receives ACK from an unexpected IP (LR). Although I'm not completely sure why the first [SYN,ACK] reply got back without SNAT, root cause of this issue is that traffic initiated from the external network is not tracked in CT. This causes even traffic that's in the "reply" direction (i.e. it was initiated from outside the SNAT network), to be translated by Logical Router. My proposal is to initiate CT and commit "new" connections destined to the internal network (with enabled SNAT) on Logical Router. This will enable SNAT rules to translate only traffic that originates from internal networks. Signed-off-by: Martin Kalcok --- northd/northd.c | 70 ++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 61 insertions(+), 9 deletions(-) As this would be my first "real" contribution to the OVN, I'd love to hear some feeback on my approach while I work on performance measurments and tests, which I plan on including in final proposal. I also left some inline XXX comments marking parts that I'm not sure about and need resolving before final proposal. I tested this change in my local Openstack deployment with distributed gw chassis and it seems to solve the issue regardless of the VM's placement, whether it does or does not have a floating IP. It also doesn't seem to break outbound connectivity, nor connectivity to VM's floating IPs. Thank you, for any reviews/sugestions. diff --git a/northd/northd.c b/northd/northd.c index 01eec64ca..b695932fd 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -10772,6 +10772,12 @@ build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, ds_destroy(&actions); } +static inline bool +lrouter_use_common_zone(const struct ovn_datapath *od) +{ + return !od->is_gw_router && use_common_zone; +} + static void add_route(struct lflow_table *lflows, struct ovn_datapath *od, const struct ovn_port *op, const char *lrp_addr_s, @@ -10796,6 +10802,9 @@ add_route(struct lflow_table *lflows, struct ovn_datapath *od, build_route_match(op_inport, rtb_id, network_s, plen, is_src_route, is_ipv4, &match, &priority, ofs); + bool use_ct = false; + struct ds target_net = DS_EMPTY_INITIALIZER; + struct ds snat_port_match = DS_EMPTY_INITIALIZER; struct ds common_actions = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; if (is_discard_route) { @@ -10808,16 +10817,61 @@ add_route(struct lflow_table *lflows, struct ovn_datapath *od, } else { ds_put_format(&common_actions, "ip%s.dst", is_ipv4 ? "4" : "6"); } + + /* To enable direct connectivity from external networks to Logical IPs + * of VMs/Containers in the SNATed networks, we need to track + * connections initiated from external networks. This allows SNAT + * rules to avoid SNATing packets in "reply" direction. */ + if (od->nbr && od->nbr->n_nat) { + ds_put_format(&target_net, "%s/%d", network_s, plen); + for (int i = 0; i < od->nbr->n_nat; i++) { + const struct nbrec_nat *nat = od->nbr->nat[i]; + if (strcmp(nat->type, "snat") || + strcmp(nat->logical_ip, ds_cstr(&target_net)) || + lrouter_use_common_zone(od)) { + continue; + } + + /* Initiate connection tracking for traffic destined to + * SNATed network. */ + use_ct = true; + + /* XXX: Need to figure out what is appropraite priority for these rules. + All they require is to be after any existing specific rules + and before the default rule.*/ + + /* Commit new connections initiated from the outside of + * SNATed network to CT. */ + ds_put_format(&snat_port_match, + "outport == %s && ct.new", + op->json_key); + ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ARP_REQUEST, 5, + ds_cstr(&snat_port_match), + "ct_commit; output;", stage_hint); + ds_clear(&snat_port_match); + + /* Initiate connection tracking for traffic from SNATed + * network to enable rules in S_ROUTER_OUT_SNAT disregard + * traffic in "reply" direction. */ + ds_put_format(&snat_port_match, "inport == %s", op->json_key); + ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_POST_UNDNAT, + 5, ds_cstr(&snat_port_match), + "ct_next;", stage_hint); + break; + } + } + ds_put_format(&common_actions, "; " "%s = %s; " "eth.src = %s; " "outport = %s; " "flags.loopback = 1; " - "next;", + "%s", is_ipv4 ? REG_SRC_IPV4 : REG_SRC_IPV6, lrp_addr_s, op->lrp_networks.ea_s, - op->json_key); + op->json_key, + use_ct ? "ct_next;" : "next;"); ds_put_format(&actions, "ip.ttl--; %s", ds_cstr(&common_actions)); } @@ -10833,6 +10887,8 @@ add_route(struct lflow_table *lflows, struct ovn_datapath *od, ds_cstr(&common_actions),\ stage_hint, lflow_ref); } + ds_destroy(&snat_port_match); + ds_destroy(&target_net); ds_destroy(&match); ds_destroy(&common_actions); ds_destroy(&actions); @@ -10933,12 +10989,6 @@ struct lrouter_nat_lb_flows_ctx { const struct shash *meter_groups; }; -static inline bool -lrouter_use_common_zone(const struct ovn_datapath *od) -{ - return !od->is_gw_router && use_common_zone; -} - static void build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, enum lrouter_nat_lb_flow_type type, @@ -14627,7 +14677,9 @@ build_lrouter_out_snat_flow(struct lflow_table *lflows, } ds_put_cstr(match, " && (!ct.trk || !ct.rpl)"); - ds_put_format(actions, "ct_snat(%s", nat->external_ip); + /* ct_commit is needed here otherwise replies to the SNATed traffic + * look like "new" traffic in S_ROUTER_IN_ARP_REQUEST*/ + ds_put_format(actions, "ct_commit; ct_snat(%s", nat->external_ip); if (nat->external_port_range[0]) { ds_put_format(actions, ",%s", nat->external_port_range); }