From patchwork Tue Aug 4 03:55:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Sharma X-Patchwork-Id: 1340650 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=nutanix.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=nutanix.com header.i=@nutanix.com header.a=rsa-sha256 header.s=proofpoint20171006 header.b=YOKa49VM; dkim-atps=neutral Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4BLLXG6ZJQz9sTH for ; Tue, 4 Aug 2020 13:56:13 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 9186A220D0; Tue, 4 Aug 2020 03:56:09 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UAHT7xq6w2BB; Tue, 4 Aug 2020 03:56:04 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id 3320421561; Tue, 4 Aug 2020 03:56:04 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 11CBCC0050; Tue, 4 Aug 2020 03:56:04 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 840F7C004C for ; Tue, 4 Aug 2020 03:56:02 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 71A5E85EAF for ; Tue, 4 Aug 2020 03:56:02 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CGiZg8LoYVE4 for ; Tue, 4 Aug 2020 03:56:01 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx0a-002c1b01.pphosted.com (mx0a-002c1b01.pphosted.com [148.163.151.68]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 32A2F85E83 for ; Tue, 4 Aug 2020 03:56:01 +0000 (UTC) Received: from pps.filterd (m0127837.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0743rIQH003619 for ; Mon, 3 Aug 2020 20:56:00 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h=from : to : cc : subject : date : message-id : content-type : mime-version; s=proofpoint20171006; bh=2k7Xshk+yIIj/5Lq20Sw+KdKDL1jO07bEXsz0HUGlCU=; b=YOKa49VMAlfbYnZh/RE018OS7tngNrRMa5Ifiv3FWeMseA/o/S5iOZeapX5+fDk+IOKM QOKja3207wBfQon0XHvFCNKYByR6Hw6qPkvQuwEkMbt7WSKTg4ftRosqyzshLsHoT4F4 IQYC5wbzedj84njtWi42zn+GtRmp9WiRXsMKVwjg3mwZz3IK53RJW39oc0UOB1fRGd/7 edLE1sDKHHgUzfc3lia9UGU//pv2LuDRleyl40xa/nLB510aTtwKZR4doxCL1ZVLKGEA P3R9eftWhrReyYvbex1t5xvcYzLduT+Hze76Xml4p3WattZZ5nHBqWo1FkziwCOv4FEh BA== Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2103.outbound.protection.outlook.com [104.47.55.103]) by mx0a-002c1b01.pphosted.com with ESMTP id 32n5cc53g2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 03 Aug 2020 20:56:00 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=D4OEX6Sqx/+g+KwQsi6MIadP+AuieoOnEpLx/cr68U8ROb5e8nFA3GnYgDq1RZgrSXds5CrEpLkfrcruo3Y+QcGswoC1VQFGiLAvpTGM3Fj/b0/XpzFWbN2r4HQiKJzZP5oQ2bAVGan8SAmEVLWMxYH6/2TDINKcAP3Ra8XE8EgxtpeuQVoVJwW6a76rSxKanJ7SaNckIhOjRmyToD4iEBkjK7KPdSo2AmTFv05KSOvWak/cKjz+4GWsu+r/WiUPbcmH6tdhEi1DE2lXw4OUa89A3sBHPo8gwUjtGJzRID+2LV+MMnrLfNolKs6Wj9hUTddyKt3bDHi+RuC16d6BKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2k7Xshk+yIIj/5Lq20Sw+KdKDL1jO07bEXsz0HUGlCU=; b=APvC4r4xz58LYbduePkjvEFjM1WKogqOLGm7fxJ1eriAKii7ZY5EIoi6n0QxIgDfNPrlMCE+SGscf5ru5xI671W7wbavgAlQTlHKfR47U4Pw8dV+6oyw8mv0/5lmWZtg6tLrM6PxDT8SZ+D2ueazpogPuxkStYoIXu+TxMg8wch+DYSXJhgoEQLBzg/e9dhWsbK7UWlv+q1QNJjNfRfZC4ZmYqAPAv6qAeifSF8m4jhLH4Mmnvga0dAZccEdEY3k2w+lFu7Ietk9QN16mkKC5hhJ5wyy3FRNKZVVw4cSHarqMAnWPuXB41t3OVdGUdJ4h6hTY3mAlRoOPk4MMxyqug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none Authentication-Results: openvswitch.org; dkim=none (message not signed) header.d=none;openvswitch.org; dmarc=none action=none header.from=nutanix.com; Received: from BL0PR02MB3714.namprd02.prod.outlook.com (2603:10b6:207:44::16) by BL0PR02MB3716.namprd02.prod.outlook.com (2603:10b6:207:40::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3239.16; Tue, 4 Aug 2020 03:55:57 +0000 Received: from BL0PR02MB3714.namprd02.prod.outlook.com ([fe80::9911:8a54:4e9e:6f98]) by BL0PR02MB3714.namprd02.prod.outlook.com ([fe80::9911:8a54:4e9e:6f98%7]) with mapi id 15.20.3239.021; Tue, 4 Aug 2020 03:55:57 +0000 From: Ankur Sharma To: ovs-dev@openvswitch.org Date: Mon, 3 Aug 2020 20:55:34 -0700 Message-Id: <1596513334-11673-1-git-send-email-svc.mail.git@nutanix.com> X-Mailer: git-send-email 1.8.3.1 X-ClientProxiedBy: BYAPR02CA0031.namprd02.prod.outlook.com (2603:10b6:a02:ee::44) To BL0PR02MB3714.namprd02.prod.outlook.com (2603:10b6:207:44::16) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from northd.localdomain (192.146.154.98) by BYAPR02CA0031.namprd02.prod.outlook.com (2603:10b6:a02:ee::44) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3239.18 via Frontend Transport; Tue, 4 Aug 2020 03:55:56 +0000 X-Mailer: git-send-email 1.8.3.1 X-Originating-IP: [192.146.154.98] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 270e5494-2303-4de1-e14c-08d8382a4b0a X-MS-TrafficTypeDiagnostic: BL0PR02MB3716: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: x-proofpoint-crosstenant: true X-MS-Oob-TLC-OOBClassifiers: OLM:65; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: DsN+/P0Mw1Eh5Q+3u9flY8XePTQLysvFPuzCXNAqKAbFANNGYw5i0pjeqadEn/aBh+6mBFz9SWh+xKW3gDHYsaGPPjDr24W5Zgpq/cRqaNS8JrpkXL7+toUy89zRJNWNod5AJ6JsqmXs68Gf0qCyfqmOZ5mNjfwJq6XsHjXAlK2kUrgGLSBmpltr0/0Ajdubx6P8tEELxCPFwqcP+eMNYEHM/UnMuE9eSh/3zFiCZFsaKJJB+O4ftTUTpRqHLYI417nv6jhVyRLO4gFlPwQ3clftcqh6flidtm/lxR7VzPSoBipH1osM8N+avDtRYqLN2L5pTMao/g2pNIM0QrB5fI+Fcak82d6GsP9keJlQ0MVnxD04jq50TaOQ924JgPIp X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR02MB3714.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(346002)(136003)(376002)(366004)(396003)(39860400002)(26005)(6916009)(66574015)(316002)(83380400001)(36756003)(478600001)(2906002)(6666004)(6486002)(6506007)(107886003)(956004)(2616005)(52116002)(86362001)(6512007)(4326008)(8936002)(186003)(8676002)(16526019)(66556008)(5660300002)(66476007)(66946007)(21314003); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: TL62ioKec2vbD1aEO/yosQnb5gn8OHO20Y6CdsXZcJyd+vosUuCcDFeKNTaTiSB+TQKMmv8mP39J6/4YXf/Ug1qXXtEtd+UCnOwUS8Ro1NJpdc4119+7CDUGLJmEvwkNzMj0/oAL/7LVNDtTk4wSccbaSM/kpnn05IXsU0d/R9of/byo11LlZSV7Eufp0c6aiGjy1c3qcoicWL+PNGEOKv4NCzz5TFtNQiowY3kbBs1xPq4p8F59APE1U9zaBV+CM4FWQg3tUJVoAyJRlR3e2VaKv6i86mrhGkNmf6GioNPbl9BGktQHgTqvpzJ9reeO35tqFDMRSuCrw99A2RatHeMoGL3nwefvsK88FUZ5jCEPpXoBdLbPeStcLqSZRuo07Dwkf8ljasgxifMz5JwfMrf9FpLEyldUW220KWxQs+W1DWhuv/9L9mC+ln+l2ZOPh4tVc6ndT7km4GYywg3h5/vwApz0YmqgxQLWNJ0j2I9/fxwtGhevwli06x9W5TwUDxPxsKmhGrJ4wobWEAyDTv3ZC2cSy1WSnGwkVGCU1WEIsfZkMm3VH47MaovwbF8QCOeTD4Mws6RCO8w0I+6mHj4vgbFgSFb2D7f75/0mg8KY/bPLcg0zJgiaS+ffbSds0x2f3OSJl67mJ20St11B7w== X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 270e5494-2303-4de1-e14c-08d8382a4b0a X-MS-Exchange-CrossTenant-AuthSource: BL0PR02MB3714.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Aug 2020 03:55:57.4309 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: oOMkQkXDH6U4FMRI7LIdxMMhMG9Jj2LPVMfu1bydP2Xs3+Y29Iu558Gf2s7tV5ZuEmGxm2ypw2riRvSr2CtE6sIy0vrdsDsAO57wrRUpK3I= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR02MB3716 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-08-04_01:2020-08-03, 2020-08-04 signatures=0 X-Proofpoint-Spam-Reason: safe Subject: [ovs-dev] [PATCH v1] ovn-controller: Fix the CT zone assignment logic for logical routers X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Ankur Sharma BACKGROUND: a. ovn-controller assigns CT ZONES for local ports and datapaths. b. If a local port/datapath is cleaned up from a chassis, then corresponding CT ZONE is "unassigned"/"freed" up. ISSUE: Above logic and implementations leaves stale CT entries in the datapath, which may get reused unexpectedly, thereby causing issues like, packets going through ct_nat(SNAT_IP_NEW) and getting a stale IP as SNAT IP etc. a. As a part of CT Zone unassign, implementation should FLUSH the corresponding CT entries, i.e it should do FLUSH by ZONE. As os now, implementation avoids the flushing, thereby leaving stale CT entries. b. Similarly, since the implementation relies on datapath existence for assign/unassign of CT ZONEs. Hence, simple operations like moving the logical router from one external logical switch to another, may not cause any CT ZONE reassignment and thereby stale CT entries might get consumed, when they should not have been. c. a. and b. combined causes following: i. Start a to be SNATed traffic from internal endpoint to an external endpoint. Let us say internal endpoint IP is 50.0.0.10 and external endpoint ip is 8.8.8.8 and logical router port ip (and hence SNAT ip) is 100.0.0.10. ii. Detach the logical router from old external logical switch and attach to new external logical switch. As a result of this operation, new router port ip becomes 200.0.0.10 , which also becomes the new SNAT ip. iii. The observation has been that traffic initiated in i. above still ends up using OLD SNAT IP, i.e 100.0.0.10, rather than 200.0.0.10 iv. iii. above happened, because although from OVS DP, the IP for NAT action is 200.0.0.10, however, since its an ongoing traffic, hence the CT entries come in use and end up NATing to old SNAT ip 100.0.0.10. For example: OVS DP STATE recirc_id(0),in_port(16),....ct(commit,zone=1,nat(src=200.0.0.10)) CT STATE icmp,orig=(src=50.0.0.10,dst=8.8.8.8,id=2288,type=8,code=0), reply=(src=8.8.8.8,dst=100.0.0.10,id=2288,type=0,code=0),zone=1 FIX: This patch improves the overall CT ZONE management by doing following: a. Do a FLUSH by CT ZONE, once we identify that a zone has to be freed up. b. From datapath perspective, restrict the CT ZONE assignment ONLY to logical routers that has NAT rules enabled. c. Instead of using logical router uuid as ct zone key, use crossproduct of logical router and logical router port that connects to external logical switch. Signed-off-by: Ankur Sharma --- controller/ovn-controller.c | 37 +++++++++++++++++++++++++++---------- controller/physical.c | 18 ++++++++++++------ lib/ovn-util.c | 10 ++++++---- lib/ovn-util.h | 3 ++- 4 files changed, 47 insertions(+), 21 deletions(-) diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c index 5ca32ac..9a6746e 100644 --- a/controller/ovn-controller.c +++ b/controller/ovn-controller.c @@ -521,17 +521,34 @@ update_ct_zones(const struct sset *lports, const struct hmap *local_datapaths, sset_add(&all_users, user); } - /* Local patched datapath (gateway routers) need zones assigned. */ + /* Local patched datapath (gateway routers) need zones assigned. + * Only local logical routers with atleast one NAT rule are considered for + * CT zone assignment.*/ const struct local_datapath *ld; HMAP_FOR_EACH (ld, hmap_node, local_datapaths) { - /* XXX Add method to limit zone assignment to logical router - * datapaths with NAT */ - char *dnat = alloc_nat_zone_key(&ld->datapath->header_.uuid, "dnat"); - char *snat = alloc_nat_zone_key(&ld->datapath->header_.uuid, "snat"); - sset_add(&all_users, dnat); - sset_add(&all_users, snat); - free(dnat); - free(snat); + const char *dp_nblr = smap_get(&ld->datapath->external_ids, + "logical-router"); + if (dp_nblr) { + for (size_t iter = 0; iter < ld->n_peer_ports; iter++) { + const struct sbrec_port_binding *peer_binding = + ld->peer_ports[iter].remote; + const struct sbrec_port_binding *local_binding = + ld->peer_ports[iter].local; + + if (peer_binding->nat_addresses) { + char *dnat = alloc_nat_zone_key(&ld->datapath->header_.uuid, + &local_binding->header_.uuid, + "dnat"); + char *snat = alloc_nat_zone_key(&ld->datapath->header_.uuid, + &local_binding->header_.uuid, + "snat"); + sset_add(&all_users, dnat); + sset_add(&all_users, snat); + free(dnat); + free(snat); + } + } + } } /* Delete zones that do not exist in above sset. */ @@ -541,7 +558,7 @@ update_ct_zones(const struct sset *lports, const struct hmap *local_datapaths, ct_zone->data, ct_zone->name); struct ct_zone_pending_entry *pending = xmalloc(sizeof *pending); - pending->state = CT_ZONE_DB_QUEUED; /* Skip flushing zone. */ + pending->state = CT_ZONE_OF_QUEUED; pending->zone = ct_zone->data; pending->add = false; shash_add(pending_ct_zones, ct_zone->name, pending); diff --git a/controller/physical.c b/controller/physical.c index 535c777..cc497e0 100644 --- a/controller/physical.c +++ b/controller/physical.c @@ -218,18 +218,24 @@ static struct zone_ids get_zone_ids(const struct sbrec_port_binding *binding, const struct simap *ct_zones) { - struct zone_ids zone_ids; + struct zone_ids zone_ids = {0}; zone_ids.ct = simap_get(ct_zones, binding->logical_port); - const struct uuid *key = &binding->datapath->header_.uuid; + const struct uuid *key1 = &binding->datapath->header_.uuid; + const struct uuid *key2 = &binding->header_.uuid; - char *dnat = alloc_nat_zone_key(key, "dnat"); - zone_ids.dnat = simap_get(ct_zones, dnat); + char *dnat = alloc_nat_zone_key(key1, key2, "dnat"); + + if (simap_contains(ct_zones, dnat)) { + zone_ids.dnat = simap_get(ct_zones, dnat); + } free(dnat); - char *snat = alloc_nat_zone_key(key, "snat"); - zone_ids.snat = simap_get(ct_zones, snat); + char *snat = alloc_nat_zone_key(key1, key2, "snat"); + if (simap_contains(ct_zones, snat)) { + zone_ids.snat = simap_get(ct_zones, snat); + } free(snat); return zone_ids; diff --git a/lib/ovn-util.c b/lib/ovn-util.c index cdb5e18..cba7355 100644 --- a/lib/ovn-util.c +++ b/lib/ovn-util.c @@ -327,14 +327,16 @@ destroy_lport_addresses(struct lport_addresses *laddrs) free(laddrs->ipv6_addrs); } -/* Allocates a key for NAT conntrack zone allocation for a provided - * 'key' record and a 'type'. +/* Allocates a key for NAT conntrack zone allocation for provided + * 'keys' and a 'type'. * * It is the caller's responsibility to free the allocated memory. */ char * -alloc_nat_zone_key(const struct uuid *key, const char *type) +alloc_nat_zone_key(const struct uuid *key1, const struct uuid *key2, + const char *type) { - return xasprintf(UUID_FMT"_%s", UUID_ARGS(key), type); + return xasprintf(UUID_FMT"_"UUID_FMT"_%s", UUID_ARGS(key1), + UUID_ARGS(key2), type); } const char * diff --git a/lib/ovn-util.h b/lib/ovn-util.h index 0f7b501..fe86bf8 100644 --- a/lib/ovn-util.h +++ b/lib/ovn-util.h @@ -77,7 +77,8 @@ bool extract_sbrec_binding_first_mac(const struct sbrec_port_binding *binding, void destroy_lport_addresses(struct lport_addresses *); -char *alloc_nat_zone_key(const struct uuid *key, const char *type); +char *alloc_nat_zone_key(const struct uuid *key1, const struct uuid *key2, + const char *type); const char *default_nb_db(void); const char *default_sb_db(void);