From patchwork Thu May 17 09:55:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Manohar Krishnappa Chidambaraswamy X-Patchwork-Id: 915285 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=ericsson.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ericsson.com header.i=@ericsson.com header.b="e5ojhq4V"; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ericsson.com header.i=@ericsson.com header.b="A0qRRYLd"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40mmsW5GDmz9s1B for ; Thu, 17 May 2018 19:56:15 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id D6D9CDDF; Thu, 17 May 2018 09:56:12 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id A0199DDE for ; Thu, 17 May 2018 09:56:11 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from sessmg22.ericsson.net (sessmg22.ericsson.net [193.180.251.58]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 58F7D6D3 for ; Thu, 17 May 2018 09:56:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; d=ericsson.com; s=mailgw201801; c=relaxed/simple; q=dns/txt; i=@ericsson.com; t=1526550968; h=From:Sender:Reply-To:Subject:Date:Message-ID:To:CC:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=mrwJjSRdRaOmy6M3lL1jfwMZMkNhCoBmp3lMlHEImn4=; b=e5ojhq4VnV0PP2O0up8hnS/zNKWzEYqeHaKssW57cM6MmrWfBlkRriuQ9NeYZJLL 1AvLzs6fy8cXXUZmYYRQ9wY51RLWs/fBreSEbXc67LI4HUQWdsAHYqbb4fSwPpIa TDxcgmHPDjy6Cm4q8YvSQTkjJ3lKcVwJqXzBN92h6tE=; X-AuditID: c1b4fb3a-5a4b59c000006a47-bf-5afd51b8d3e5 Received: from ESESSHC019.ericsson.se (Unknown_Domain [153.88.183.75]) by sessmg22.ericsson.net (Symantec Mail Security) with SMTP id D0.63.27207.8B15DFA5; Thu, 17 May 2018 11:56:08 +0200 (CEST) Received: from ESESSMB505.ericsson.se (153.88.183.166) by ESESSHC019.ericsson.se (153.88.183.75) with Microsoft SMTP Server (TLS) id 14.3.382.0; Thu, 17 May 2018 11:55:05 +0200 Received: from ESESBMB504.ericsson.se (153.88.183.171) by ESESSMB505.ericsson.se (153.88.183.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1466.3; Thu, 17 May 2018 11:55:05 +0200 Received: from EUR02-AM5-obe.outbound.protection.outlook.com (153.88.183.157) by ESESBMB504.ericsson.se (153.88.183.171) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1466.3 via Frontend Transport; Thu, 17 May 2018 11:55:04 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mrwJjSRdRaOmy6M3lL1jfwMZMkNhCoBmp3lMlHEImn4=; b=A0qRRYLdRwyLQtPQ3HSeUg47oLzdVu3YBjoTsYPrN4L9oCCrneKxWaEHit20vpeLEj43qVjdFo2bCFZVgUFa9+la/LUtcHgDdxX3FY3VLIMZR3oYNBD49sBK7ZKPP1ZfsV2OpOjjalYP1uuaFc/E0Y31c1eFZ0bh7XvwVtcm5As= Received: from DB4PR07MB0526.eurprd07.prod.outlook.com (10.242.195.17) by DB4PR07MB235.eurprd07.prod.outlook.com (10.242.231.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.776.4; Thu, 17 May 2018 09:55:01 +0000 Received: from DB4PR07MB0526.eurprd07.prod.outlook.com ([fe80::6d50:e6a:ef78:5273]) by DB4PR07MB0526.eurprd07.prod.outlook.com ([fe80::6d50:e6a:ef78:5273%7]) with mapi id 15.20.0797.005; Thu, 17 May 2018 09:55:00 +0000 From: Manohar Krishnappa Chidambaraswamy To: "dev@openvswitch.org" Thread-Topic: [PATCH v1 1/2] Fix packet drops on LACP bond after link up Thread-Index: AQHT7cUeLjr/dBAWSU6MfjdZ9nr29Q== Date: Thu, 17 May 2018 09:55:00 +0000 Message-ID: <88A61304-E845-4631-9439-53AB2185EB95@ericsson.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=manohar.krishnappa.chidambaraswamy@ericsson.com; x-originating-ip: [198.24.6.220] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB4PR07MB235; 7:OKkV52q3JdJp8INc41/33UQ7sZPihWF4zI4NRvetT/JFvPVN242Mr+2m8Dbbwyg1a7qGqd31ashIZsyrhS4sjF/qyID8kog+GEiza7YhAYMv7RJi8mYRK2/4icNukCs1AOs7HvGUKWtQ0Bekw+UXKFRlWdAeagY6S+rrwd01AKvWz/+zo2Y6LSdFWoJ6sREbUwTKlYvLoyS//MbBWRzeFrUJHymmu+1N8jzshWAYbnpMSkq5euYDHvLihLQLiVzE x-ms-exchange-antispam-srfa-diagnostics: SOS;SOR; x-forefront-antispam-report: SFV:SKI; SCL:-1; SFV:NSPM; SFS:(10009020)(39860400002)(396003)(366004)(39380400002)(376002)(346002)(199004)(189003)(81166006)(2900100001)(54906003)(107886003)(81156014)(5250100002)(2501003)(6506007)(6916009)(83716003)(97736004)(105586002)(2906002)(86362001)(186003)(8676002)(476003)(59450400001)(1730700003)(82746002)(3660700001)(2616005)(66066001)(8936002)(3280700002)(68736007)(99286004)(53936002)(36756003)(102836004)(478600001)(4326008)(5640700003)(316002)(106356001)(6512007)(2351001)(6436002)(3846002)(33656002)(5660300001)(486006)(305945005)(25786009)(7736002)(14454004)(6486002)(26005)(6116002); DIR:OUT; SFP:1101; SCL:1; SRVR:DB4PR07MB235; H:DB4PR07MB0526.eurprd07.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:DB4PR07MB235; x-ms-traffictypediagnostic: DB4PR07MB235: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(37575265505322)(20558992708506); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3231254)(944501410)(52105095)(10201501046)(93006095)(93001095)(3002001)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123558120)(20161123564045)(6072148)(201708071742011)(7699016); SRVR:DB4PR07MB235; BCL:0; PCL:0; RULEID:; SRVR:DB4PR07MB235; x-forefront-prvs: 067553F396 received-spf: None (protection.outlook.com: ericsson.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: IBfYL5rpO7Ef7Xz+9KQ0Q1OHNRepRcWL/5/Vyj9uVHfqtcg0am6l/OlVWo1XNVnCtA/BotK6GQep7KZhZ56j3dvq/WKF8di827T1zgYQF0JHDqYGegYnlZbl6Vee332kIquoPhJiJdDt6BdxbYRex2ftvCVjNJ/3vPcvsc9SvSpDxeZ4939HjPYIVtvqWTlT spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-ID: MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: faa551bc-79a7-4f82-266c-08d5bbdc411f X-MS-Exchange-CrossTenant-Network-Message-Id: faa551bc-79a7-4f82-266c-08d5bbdc411f X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2018 09:55:00.1171 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB4PR07MB235 X-OriginatorOrg: ericsson.com X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupkleLIzCtJLcpLzFFi42KZGbHdW3dH4N8og5Y+FYujp/cwOzB6PLv5 nzGAMYrLJiU1J7MstUjfLoEr41b3HbaCKYEVX+4dZm5g7PDvYuTkkBAwkZj+ZhNLFyMXh5DA EUaJuXcaGSGcLYwSJ5dMhXK+MUosmNLBDuEsYZI4f+gpM4jDIjCBWWLJoSusEJmJTBJXdzxi g3AeMErsa37DDrKGTSBG4tmxSWC2iIC+xKGes0ArOTiYgeLX3rGBhIUFnCXuHP/DCFHiIXF+ 4jUWCFtPouvlAbA4i4CqxLxjr5lBbF4Be4mJr9+CjWQUEJP4fmoNE4jNLCAucevJfCaI7wQk luw5zwxhi0q8fPyPFaK+SOLevS3sEHEFiVNXGxghbFmJS/O7wX6WENjMJPF/fSsrRMJQ4vjK /cwQiZ+sEvv2HGOCcLYySvxZ/AtqlKbE2ZW7oToyJb7OXQdle0ks2HcDypaTWNX7kAXC3sMs 8fq+7wRGw1lILp8FDhhNifW79CHCHhJb7j9lhbAVJaZ0P2SfBQ4AQYmTM5+wLGBkXcUoWpxa XJybbmSkl1qUmVxcnJ+nl5dasokRmEAObvlttYPx4HPHQ4wCHIxKPLxcvn+jhFgTy4orcw8x SnAwK4nwmnz5EyXEm5JYWZValB9fVJqTWnyIUZqDRUmc1ynNIkpIID2xJDU7NbUgtQgmy8TB KdXAqMsTuvTjrTDLog3iAf//+9T9zt3P9rNpk+ecznWahzj2MN3/HznhbF1KrfiOjbuN73ae PhU962S/lcbcpsXheSWNWyL57j8sd3qSsmnlz0Jb11e95qFGFu0XlFnX7/1i4xb5qSHa6n9b h4jPmhOlvzzVmU4umP7td3GWSoOS1f/d4QvVrtffUWIpzkg01GIuKk4EAN7wFiAcAwAA X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Nitin Katiyar Subject: [ovs-dev] [PATCH v1 1/2] Fix packet drops on LACP bond after link up X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org 1/2: Fix packet drops on LACP bond after link up Problem: } else if (xport->xbundle && xport->xbundle->lacp && flow->dl_type == htons(ETH_TYPE_LACP)) { if (packet) { - lacp_process_packet(xport->xbundle->lacp, xport->ofport, packet); + lacp_may_enable = lacp_process_packet(xport->xbundle->lacp, + xport->xbundle->bond, + xport->ofport, packet); + /* + * Update LACP status in bond-slave to avoid packet-drops until + * LACP state machine is run by the main thread. + */ + if (xport->xbundle->bond && lacp_may_enable) { + bond_slave_set_may_enable(xport->xbundle->bond, xport->ofport, + lacp_may_enable); + } } slow = SLOW_LACP; } else if ((xbridge->stp || xbridge->rstp) && --- 1.9.1 ======== During port DOWN->UP of link (slave) in a LACP bond, after receiving the LACPDU with SYNC set for both actor and partner, the bond-slave remains "disabled" until OVS main thread runs LACP state machine and eventually "enables" the bond-slave. With this, we have observed delays in the order of 350ms and packets are dropped in OVS due to bond-admissibility check (packets received on slave in "disabled" state are dropped). Fix: ==== When a LACPDU is received, evaluate whether LACP slave can be enabled (slave_may_enable()) and set LACP slave's may_enable from the datapath thread itself. When may_enable = TRUE, it means L1 state is UP and LACP-SYNC is done and it is waiting for the main thread to enable the slave. Relax the check in bond_check_admissibility() to check for both "enable" and "may_enable" of the LACP slave. This would avoid dropping of packets until the main thread enables the slave from bundle_run(). Signed-off-by: Manohar K C CC: Jan Scheurich CC: Nitin Katiyar --- v1 2/2: Avoid LACP negotiation when NIC driver reports PHY status as DOWN. lib/lacp.c | 14 ++++++++++++-- lib/lacp.h | 3 ++- ofproto/bond.c | 18 +++++++++++++++--- ofproto/ofproto-dpif-xlate.c | 13 ++++++++++++- 4 files changed, 41 insertions(+), 7 deletions(-) diff --git a/lib/lacp.c b/lib/lacp.c index 8353746..8d97ad5 100644 --- a/lib/lacp.c +++ b/lib/lacp.c @@ -154,6 +154,7 @@ static struct slave *slave_lookup(const struct lacp *, const void *slave) OVS_REQUIRES(mutex); static bool info_tx_equal(struct lacp_info *, struct lacp_info *) OVS_REQUIRES(mutex); +static bool slave_may_enable__(struct slave *slave) OVS_REQUIRES(mutex); static unixctl_cb_func lacp_unixctl_show; static unixctl_cb_func lacp_unixctl_show_stats; @@ -324,8 +325,8 @@ lacp_is_active(const struct lacp *lacp) OVS_EXCLUDED(mutex) /* Processes 'packet' which was received on 'slave_'. This function should be * called on all packets received on 'slave_' with Ethernet Type ETH_TYPE_LACP. */ -void -lacp_process_packet(struct lacp *lacp, const void *slave_, +bool +lacp_process_packet(struct lacp *lacp, const void *bond, const void *slave_, const struct dp_packet *packet) OVS_EXCLUDED(mutex) { @@ -333,6 +334,7 @@ lacp_process_packet(struct lacp *lacp, const void *slave_, const struct lacp_pdu *pdu; long long int tx_rate; struct slave *slave; + bool lacp_may_enable = false; lacp_lock(); slave = slave_lookup(lacp, slave_); @@ -362,8 +364,16 @@ lacp_process_packet(struct lacp *lacp, const void *slave_, slave->partner = pdu->actor; } + /* + * Evaluate may_enable here to avoid dropping of packets till main thread + * sets may_enable to true. + */ + lacp_may_enable = slave_may_enable__(slave); + out: lacp_unlock(); + + return lacp_may_enable; } /* Returns the lacp_status of the given 'lacp' object (which may be NULL). */ diff --git a/lib/lacp.h b/lib/lacp.h index f35cff5..1505c2c 100644 --- a/lib/lacp.h +++ b/lib/lacp.h @@ -46,7 +46,8 @@ struct lacp *lacp_ref(const struct lacp *); void lacp_configure(struct lacp *, const struct lacp_settings *); bool lacp_is_active(const struct lacp *); -void lacp_process_packet(struct lacp *, const void *slave, +bool lacp_process_packet(struct lacp *, const void *bond, + const void *slave, const struct dp_packet *packet); enum lacp_status lacp_status(const struct lacp *); diff --git a/ofproto/bond.c b/ofproto/bond.c index 11d28e1..7ca3687 100644 --- a/ofproto/bond.c +++ b/ofproto/bond.c @@ -777,6 +777,7 @@ bond_check_admissibility(struct bond *bond, const void *slave_, { enum bond_verdict verdict = BV_DROP; struct bond_slave *slave; + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); ovs_rwlock_rdlock(&rwlock); slave = bond_slave_lookup(bond, slave_); @@ -794,7 +795,13 @@ bond_check_admissibility(struct bond *bond, const void *slave_, * drop all incoming traffic except if lacp_fallback_ab is enabled. */ switch (bond->lacp_status) { case LACP_NEGOTIATED: - verdict = slave->enabled ? BV_ACCEPT : BV_DROP; + /* + * To reduce packet-drops due to delay in enabling of slave (post + * LACP-SYNC), from main thread, check for may_enable as well. + * When may_enable is TRUE, it means LACP is UP and waiting for + * the main thread to run LACP state machine and enable the slave. + */ + verdict = (slave->enabled || slave->may_enable) ? BV_ACCEPT : BV_DROP; goto out; case LACP_CONFIGURED: if (!bond->lacp_fallback_ab) { @@ -830,8 +837,6 @@ bond_check_admissibility(struct bond *bond, const void *slave_, /* Drop all packets which arrive on backup slaves. This is similar to * how Linux bonding handles active-backup bonds. */ if (bond->active_slave != slave) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); - VLOG_DBG_RL(&rl, "active-backup bond received packet on backup" " slave (%s) destined for " ETH_ADDR_FMT, slave->name, ETH_ADDR_ARGS(eth_dst)); @@ -853,6 +858,13 @@ bond_check_admissibility(struct bond *bond, const void *slave_, OVS_NOT_REACHED(); out: + if (slave && (verdict != BV_ACCEPT)) { + VLOG_DBG_RL(&rl, "slave= %s actv-slave= %d may_enable %d enable %d " + "LACP %d verdict(A/D/M=0/1/2) %d\n", slave->name, + (bond->active_slave == slave), slave->may_enable, + slave->enabled, bond->lacp_status, verdict); + } + ovs_rwlock_unlock(&rwlock); return verdict; diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c index 5641724..81dcdb6 100644 --- a/ofproto/ofproto-dpif-xlate.c +++ b/ofproto/ofproto-dpif-xlate.c @@ -3160,6 +3160,7 @@ process_special(struct xlate_ctx *ctx, const struct xport *xport) const struct xbridge *xbridge = ctx->xbridge; const struct dp_packet *packet = ctx->xin->packet; enum slow_path_reason slow; + bool lacp_may_enable; if (!xport) { slow = 0; @@ -3180,7 +3181,17 @@ process_special(struct xlate_ctx *ctx, const struct xport *xport) From patchwork Thu May 17 09:57:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Manohar Krishnappa Chidambaraswamy X-Patchwork-Id: 915287 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=ericsson.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ericsson.com header.i=@ericsson.com header.b="EREcflm4"; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ericsson.com header.i=@ericsson.com header.b="lrAsOzfg"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40mmwc20zSz9s1B for ; Thu, 17 May 2018 19:58:56 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 64D05D61; Thu, 17 May 2018 09:58:54 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 34B8DD5A for ; Thu, 17 May 2018 09:58:53 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from sesbmg22.ericsson.net (sesbmg22.ericsson.net [193.180.251.48]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id C357F196 for ; Thu, 17 May 2018 09:58:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; d=ericsson.com; s=mailgw201801; c=relaxed/simple; q=dns/txt; i=@ericsson.com; t=1526551130; h=From:Sender:Reply-To:Subject:Date:Message-ID:To:CC:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=g74FrKZvwJo61tLT4gmaP4281ZgnQ7lhDqJyxt71ofc=; b=EREcflm4I5R3121VfuSu1Fgl+vGNJGbcxE24txwh+gZ/vMwbNXSafsgPNoVGXEjN sTPUlacvk8mkdG24FZ39J4Edz8NpRIXNDZitj4ZGWV6icUjBVm/agJQUZ5FtKbAw aZgGMEXGgxsPcg05v+irBAS1hgDC+ElWZM/hWCm+szo=; X-AuditID: c1b4fb30-263479c00000169b-ac-5afd5259b68c Received: from ESESSHC020.ericsson.se (Unknown_Domain [153.88.183.78]) by sesbmg22.ericsson.net (Symantec Mail Security) with SMTP id B1.52.05787.9525DFA5; Thu, 17 May 2018 11:58:50 +0200 (CEST) Received: from ESESSMB501.ericsson.se (153.88.183.162) by ESESSHC020.ericsson.se (153.88.183.78) with Microsoft SMTP Server (TLS) id 14.3.382.0; Thu, 17 May 2018 11:58:00 +0200 Received: from ESESBMB505.ericsson.se (153.88.183.172) by ESESSMB501.ericsson.se (153.88.183.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1466.3; Thu, 17 May 2018 11:58:00 +0200 Received: from EUR03-VE1-obe.outbound.protection.outlook.com (153.88.183.157) by ESESBMB505.ericsson.se (153.88.183.172) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1466.3 via Frontend Transport; Thu, 17 May 2018 11:58:00 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=g74FrKZvwJo61tLT4gmaP4281ZgnQ7lhDqJyxt71ofc=; b=lrAsOzfgtA1BqlE79nH/ikvIR1QFY8liRTV87P8d/tefkL7XKNqvzHUAv29jcVqFKYPo7i/ck7mPH5hXtS9ILnaDTqBp1gGkwh8zXURdZBJI5AYDeyH5Nfhj1C6QuNECMx40RuTvI5DQlXMRhFDIjlMnGMN1u6AZ2pjoSp3sX9w= Received: from DB4PR07MB0526.eurprd07.prod.outlook.com (10.242.195.17) by DB4PR07MB428.eurprd07.prod.outlook.com (10.141.237.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.776.4; Thu, 17 May 2018 09:57:57 +0000 Received: from DB4PR07MB0526.eurprd07.prod.outlook.com ([fe80::6d50:e6a:ef78:5273]) by DB4PR07MB0526.eurprd07.prod.outlook.com ([fe80::6d50:e6a:ef78:5273%7]) with mapi id 15.20.0797.005; Thu, 17 May 2018 09:57:56 +0000 From: Manohar Krishnappa Chidambaraswamy To: "dev@openvswitch.org" Thread-Topic: [PATCH v1 2/2] Fix packet drops on LACP bond after link up Thread-Index: AQHT7cWH4EeJaTkJBEexphZLeXW3/w== Date: Thu, 17 May 2018 09:57:56 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=manohar.krishnappa.chidambaraswamy@ericsson.com; x-originating-ip: [198.24.6.220] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB4PR07MB428; 7:1RZRQ++WAqH7sl+bgccZfyPN9n+Z96ASwVAq2iHkIKTN9dK0IvhxtvForCrnQA14lZJy8rhuvLq2m2KbM2at4lF8NvDADNxieQqwf+896w6On5iCVzNjvnalhMPb0RZ10/TUpxwBiQrk+pnqwfCS/0zR1z5NZuYtPAxyxyCp8RN0/QbbjF5Etff+H/RDyGtv6qMzdMoHP9H4MyyuBbMm9B75VfRjh8jR8b8Sfv0d2Bq6FvJrK4Bz9L3geMlqvGnD x-ms-exchange-antispam-srfa-diagnostics: SOS;SOR; x-forefront-antispam-report: SFV:SKI; SCL:-1; SFV:NSPM; SFS:(10009020)(376002)(346002)(366004)(39380400002)(396003)(39860400002)(189003)(199004)(6486002)(2900100001)(26005)(36756003)(83716003)(7736002)(6116002)(86362001)(316002)(5640700003)(53936002)(6436002)(54906003)(6512007)(4326008)(5660300001)(6916009)(107886003)(186003)(102836004)(99286004)(3660700001)(305945005)(2906002)(8936002)(478600001)(486006)(3280700002)(106356001)(66066001)(14454004)(6506007)(82746002)(59450400001)(3846002)(2351001)(476003)(105586002)(2501003)(33656002)(2616005)(81166006)(5250100002)(68736007)(8676002)(1730700003)(81156014)(25786009)(97736004); DIR:OUT; SFP:1101; SCL:1; SRVR:DB4PR07MB428; H:DB4PR07MB0526.eurprd07.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:DB4PR07MB428; x-ms-traffictypediagnostic: DB4PR07MB428: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(37575265505322)(211171220733660); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3231254)(944501410)(52105095)(10201501046)(93006095)(93001095)(3002001)(149027)(150027)(6041310)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123558120)(20161123560045)(6072148)(201708071742011)(7699016); SRVR:DB4PR07MB428; BCL:0; PCL:0; RULEID:; SRVR:DB4PR07MB428; x-forefront-prvs: 067553F396 received-spf: None (protection.outlook.com: ericsson.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: 9chhZbOVqratZhsHuJu1EZWL8CXX+52h+dtpd6CddxpmpI7xIMVDRMF0nW30B5G/dv8bkqMT514OIac75AI8ygPQmAOTb7UACbK7hZezJz1L0pq63KCx6saE+PYEAONE7TZNFKgUfWJ/21VxCaKIOa0rN0WgeXfssgxe75XqFLJDngWmGREa3sTOhLfgbqg8 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-ID: <13F8AAFBB928914AB3E4343A3B2A2649@eurprd07.prod.outlook.com> MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: c380c521-52fb-46ba-4003-08d5bbdcaa63 X-MS-Exchange-CrossTenant-Network-Message-Id: c380c521-52fb-46ba-4003-08d5bbdcaa63 X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2018 09:57:56.7252 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB4PR07MB428 X-OriginatorOrg: ericsson.com X-Brightmail-Tracker: H4sIAAAAAAAAA02Sb0hTURjGO/febdfZ6Dr/vRlFTfxgkTqREJIygjAUESGQW6hLL240p92r ln6IfcgpzkrLQJekiRGaYTXFkpE2nKYp5swgw5G1FBWkZGn+z+tZ0Lff+zzPe857Xg5NKssk IbTOUMDxBo1eJZVTdWldycfZ1E02qucdEet4byPjUcLM522Uglh5XDan1xVxfOSpTLl2zdRO 5fez14dds4QR1adVIB8amBi4vzYkrUByWsn0Ieg2v5bhogPBUtsowsUygmePzN5YMwGlv5so saCYKhKmtta9TjUB7cM2b89XBBOebUq8Rspcgpn+uzKRA5hIsFeO7Og0Te7onxalIvozZ+H5 WhZOJEBL7zYhygFMBFhaT4gyxYRBhXFeIrKCOQ3r5ptIZMQEwcpQGyEyyQTDpLuBwG9joNk2 SmIOhLnvWxKc58Hl6pBh/TAMTRgR5oPgbDDvTg+MlYAPTWNSbKhhoKWHxMaqBB6b5r3dnQjq K29gDgfHeDmFWQeLyw5vcyLU1T+UYD4ErbemKXyQjYQu0xtUhdSW/ya37K4lHNq7I7GcACOj qwTmI1BjnpZZdhfgB4N1bqoRSVpRoMAJl3NzoqMjOF6XJQh5hggDV/AS7XyPtx3rUa/Q3OwZ O2JopNqr6EzaZJUSTZFQnGtHQJOqAEWMZ4NVKrI1xSUcn5fBF+o5wY4O0JQqWBGbbGWVTI6m gLvCcfkc/88laJ8QI4oR4m2/ahaSQtWK80Fyy7eS8aovP1LbSL2gVgfuj79NmKace1qc0gva qDTPnbLMY2Pp/r3XnlxVhrI5uZ6TPp0Pxs+5n/jKFeyE7enSxZq8WusfOeyrLhwZDFt4MdDY 7JuROGn9GZeylI3iN1a05X7uj32lwfdcDr90i9O/VkUJWo36KMkLmr+T/qgbGgMAAA== X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Nitin Katiyar Subject: [ovs-dev] [PATCH v1 2/2] Fix packet drops on LACP bond after link up X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org 2/2: Fix packet drops on LACP bond after link up Problem: ======== On certain Fortville NICs it has been observed that PHY UP detection can get delayed (sometimes up to 4-5 secs). When the driver fails to fetch PHY status as UP even though its actually UP, LACP packets can get exchanged and LACP slave brought UP. In such a case, the remote end would start sending traffic on that slave, but OVS drops it, as the bond-slave is not yet enabled due to PHY DOWN. Fix: ==== The main intention here is delay LACP negotiation until carrier (PHY) status is read as UP. 1. In port_run()/bundle_run(), cache the carrier status in "struct ofport_dpif"/"struct bond_slave". 2. When carrier state is DOWN, do not send any LACPDUs and drop any received LACPDUs. 3. When LACP state changes or LACPDU is received, trigger re-checking of carrier-state (in port_run()) by incrementing the connectivity sequence number to find out the true carrier state as fast as possible. Signed-off-by: Manohar K C CC: Jan Scheurich CC: Nitin Katiyar --- v1 1/2: Evaluate lacp may_enable inline in the datapath thread and check for pending "enable" in the bond admissibility check to avoid drops. lib/lacp.c | 51 +++++++++++++++++++++++++++++++++++++------- lib/lacp.h | 6 +++--- ofproto/bond.c | 30 ++++++++++++++++++++++++++ ofproto/bond.h | 3 +++ ofproto/ofproto-dpif-xlate.c | 4 +++- ofproto/ofproto-dpif.c | 11 +++++++--- tests/ofproto-dpif.at | 3 +++ 7 files changed, 93 insertions(+), 15 deletions(-) diff --git a/lib/lacp.c b/lib/lacp.c index 8353746..88621bc 100644 --- a/lib/lacp.c +++ b/lib/lacp.c @@ -33,6 +33,7 @@ #include "unixctl.h" #include "openvswitch/vlog.h" #include "util.h" +#include "ofproto/bond.h" VLOG_DEFINE_THIS_MODULE(lacp); @@ -148,7 +149,7 @@ static void slave_get_actor(struct slave *, struct lacp_info *actor) OVS_REQUIRES(mutex); static void slave_get_priority(struct slave *, struct lacp_info *priority) OVS_REQUIRES(mutex); -static bool slave_may_tx(const struct slave *) +static bool slave_may_tx(const void *bond, const struct slave *) OVS_REQUIRES(mutex); static struct slave *slave_lookup(const struct lacp *, const void *slave) OVS_REQUIRES(mutex); @@ -325,7 +326,7 @@ lacp_is_active(const struct lacp *lacp) OVS_EXCLUDED(mutex) * called on all packets received on 'slave_' with Ethernet Type ETH_TYPE_LACP. */ void -lacp_process_packet(struct lacp *lacp, const void *slave_, +lacp_process_packet(struct lacp *lacp, const void *bond, const void *slave_, const struct dp_packet *packet) OVS_EXCLUDED(mutex) { @@ -333,6 +334,7 @@ lacp_process_packet(struct lacp *lacp, const void *slave_, const struct lacp_pdu *pdu; long long int tx_rate; struct slave *slave; + bool carrier_up = false; lacp_lock(); slave = slave_lookup(lacp, slave_); @@ -348,6 +350,21 @@ lacp_process_packet(struct lacp *lacp, const void *slave_, goto out; } + /* + * On some NICs L1 state reporting is slow. In case LACP packets are + * received while carrier (L1) state is still down, drop the LACPDU and + * trigger re-checking of L1 state. + */ + carrier_up = bond_slave_get_carrier(bond, slave_); + if (!carrier_up) { + seq_change(connectivity_seq_get()); + + VLOG_INFO("carrier still DOWN - conn seq changed for slave %s, " + "dropping packet\n", slave->name); + + goto out; + } + slave->status = LACP_CURRENT; tx_rate = lacp->fast ? LACP_FAST_TIME_TX : LACP_SLOW_TIME_TX; timer_set_duration(&slave->rx, LACP_RX_MULTIPLIER * tx_rate); @@ -521,7 +538,8 @@ lacp_slave_is_current(const struct lacp *lacp, const void *slave_) /* This function should be called periodically to update 'lacp'. */ void -lacp_run(struct lacp *lacp, lacp_send_pdu *send_pdu) OVS_EXCLUDED(mutex) +lacp_run(struct lacp *lacp, const void *bond, + lacp_send_pdu *send_pdu) OVS_EXCLUDED(mutex) { struct slave *slave; @@ -551,7 +569,7 @@ lacp_run(struct lacp *lacp, lacp_send_pdu *send_pdu) OVS_EXCLUDED(mutex) HMAP_FOR_EACH (slave, node, &lacp->slaves) { struct lacp_info actor; - if (!slave_may_tx(slave)) { + if (!slave_may_tx(bond, slave)) { continue; } @@ -580,13 +598,13 @@ lacp_run(struct lacp *lacp, lacp_send_pdu *send_pdu) OVS_EXCLUDED(mutex) /* Causes poll_block() to wake up when lacp_run() needs to be called again. */ void -lacp_wait(struct lacp *lacp) OVS_EXCLUDED(mutex) +lacp_wait(struct lacp *lacp, const void *bond) OVS_EXCLUDED(mutex) { struct slave *slave; lacp_lock(); HMAP_FOR_EACH (slave, node, &lacp->slaves) { - if (slave_may_tx(slave)) { + if (slave_may_tx(bond, slave)) { timer_wait(&slave->tx); } @@ -810,9 +828,26 @@ slave_get_priority(struct slave *slave, struct lacp_info *priority) } static bool -slave_may_tx(const struct slave *slave) OVS_REQUIRES(mutex) +slave_may_tx(const void *bond, const struct slave *slave) OVS_REQUIRES(mutex) { - return slave->lacp->active || slave->status != LACP_DEFAULTED; + bool carrier_up = true; + + if (bond) { + /* + * On some NICs L1 state reporting is slow. If carrier (L1) state is + * still down, do not send LACPDU and trigger re-checking of L1 state. + */ + carrier_up = bond_slave_get_carrier(bond, slave->aux); + if (!carrier_up) { + seq_change(connectivity_seq_get()); + + VLOG_INFO("carrier still DOWN - conn seq changed for slave %s, " + "avoiding tx\n", slave->name); + } + } + + return carrier_up && + (slave->lacp->active || slave->status != LACP_DEFAULTED); } static struct slave * diff --git a/lib/lacp.h b/lib/lacp.h index f35cff5..bf8f156 100644 --- a/lib/lacp.h +++ b/lib/lacp.h @@ -46,7 +46,7 @@ struct lacp *lacp_ref(const struct lacp *); void lacp_configure(struct lacp *, const struct lacp_settings *); bool lacp_is_active(const struct lacp *); -void lacp_process_packet(struct lacp *, const void *slave, +void lacp_process_packet(struct lacp *, const void *bond, const void *slave, const struct dp_packet *packet); enum lacp_status lacp_status(const struct lacp *); @@ -67,8 +67,8 @@ bool lacp_slave_is_current(const struct lacp *, const void *slave_); /* Callback function for lacp_run() for sending a LACP PDU. */ typedef void lacp_send_pdu(void *slave, const void *pdu, size_t pdu_size); -void lacp_run(struct lacp *, lacp_send_pdu *); -void lacp_wait(struct lacp *); +void lacp_run(struct lacp *, const void *bond, lacp_send_pdu *); +void lacp_wait(struct lacp *, const void *bond); struct lacp_slave_stats { /* id */ diff --git a/ofproto/bond.c b/ofproto/bond.c index 11d28e1..9c5448d 100644 --- a/ofproto/bond.c +++ b/ofproto/bond.c @@ -93,6 +93,7 @@ struct bond_slave { /* Link status. */ bool enabled; /* May be chosen for flows? */ bool may_enable; /* Client considers this slave bondable. */ + bool carrier_up; /* Carrier state from NIC port */ long long delay_expires; /* Time after which 'enabled' may change. */ /* Rebalancing info. Used only by bond_rebalance(). */ @@ -633,6 +634,35 @@ bond_slave_set_may_enable(struct bond *bond, void *slave_, bool may_enable) ovs_rwlock_unlock(&rwlock); } diff --git a/tests/ofproto-dpif.at b/tests/ofproto-dpif.at index 60f28e2..0dfa572 100644 --- a/tests/ofproto-dpif.at +++ b/tests/ofproto-dpif.at @@ -6322,6 +6322,9 @@ OVS_VSWITCHD_START([dnl other_config:lacp-port-priority=222 \ other_config:lacp-aggregation-key=3333 ]) +ovs-appctl netdev-dummy/set-admin-state p1 up +ovs-appctl netdev-dummy/set-admin-state p2 up + on_exit 'kill `cat test-sflow.pid`' AT_CHECK([ovstest test-sflow --log-file --detach --no-chdir --pidfile 0:127.0.0.1 > sflow.log], [0], [], [ignore]) AT_CAPTURE_FILE([sflow.log]) _______________________________________________ dev mailing list dev@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev