From patchwork Fri Jul 21 12:51:41 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Maximets X-Patchwork-Id: 792091 X-Patchwork-Delegate: dlu998@gmail.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3xDW0R0Dmdz9s83 for ; Fri, 21 Jul 2017 22:53:27 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 7EC48BA3; Fri, 21 Jul 2017 12:52:05 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id A6BC8B92 for ; Fri, 21 Jul 2017 12:52:03 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 164093F4 for ; Fri, 21 Jul 2017 12:52:03 +0000 (UTC) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTP id <0OTF006TKYEO2P40@mailout2.w1.samsung.com> for ovs-dev@openvswitch.org; Fri, 21 Jul 2017 13:52:00 +0100 (BST) Received: from eusmges3.samsung.com (unknown [203.254.199.242]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20170721125200eucas1p2032ff209ba4270e4ba3065432fb7a81f~TWYMCNOqd3161331613eucas1p2N; Fri, 21 Jul 2017 12:52:00 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges3.samsung.com (EUCPMTA) with SMTP id B9.F2.17464.FE8F1795; Fri, 21 Jul 2017 13:51:59 +0100 (BST) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20170721125159eucas1p157d8e1da47c5c49aae5e7563b8c85abf~TWYLbRbk02078520785eucas1p1S; Fri, 21 Jul 2017 12:51:59 +0000 (GMT) X-AuditID: cbfec7f2-f797e6d000004438-cd-5971f8ef8bcc Received: from eusync3.samsung.com ( [203.254.199.213]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 1B.95.17452.FE8F1795; Fri, 21 Jul 2017 13:51:59 +0100 (BST) Received: from imaximets.rnd.samsung.ru ([106.109.129.180]) by eusync3.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0OTF009QNYEA8ZB0@eusync3.samsung.com>; Fri, 21 Jul 2017 13:51:59 +0100 (BST) From: Ilya Maximets To: ovs-dev@openvswitch.org, Darrell Ball Date: Fri, 21 Jul 2017 15:51:41 +0300 Message-id: <1500641502-31926-3-git-send-email-i.maximets@samsung.com> X-Mailer: git-send-email 2.7.4 In-reply-to: <1500641502-31926-1-git-send-email-i.maximets@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrFIsWRmVeSWpSXmKPExsWy7djP87rvfxRGGnz7YmDxanIDo8XUSUYW Lf0zmS2ev1jIbHH3xwomi2mfb7NbXGn/yW6x8eFZVou1hz6wW6y4cYrRYu6n54wWd2/+YXLg 8Vi85yWTx7Ob/xk9nl/rYfF4v+8qm0ffllWMHu/mv2ULYIvisklJzcksSy3St0vgyjiws4m1 4JtExc3v85kaGCcKdzFyckgImEg0PZjFDmGLSVy4t56ti5GLQ0hgKaNE44/DLBDOZyBn1zFm mI6Dzw4yQSSWMUp0NF6Aamlmklg1q58NpIpNQEfi1OojjCC2iICNxLMFW8FGMQs0M0usndDF CpIQFoiRuP57PxOIzSKgKtFzeRLYCl4BN4n+SReh1slJ3DzXCWZzCrhLtPTsBhskITCPXaKl eSOQwwHkyEpsOgBV7yKxatt+NghbWOLV8S1Qz8lIdHZAnC0h0Mwo0bDqEiOEM4FR4kvzciaI KnuJUzevgtnMAnwSk7ZNZ4ZYwCvR0SYEUeIhcfjDGahyR4kTa88xQrw/i1FiSu8O1gmMMgsY GVYxiqSWFuempxYb6xUn5haX5qXrJefnbmIEpoDT/45/2sH49YTVIUYBDkYlHt4VqwsihVgT y4orcw8xSnAwK4nwCn0pjBTiTUmsrEotyo8vKs1JLT7EKM3BoiTOy3XqWoSQQHpiSWp2ampB ahFMlomDU6qB0fprjtWC9kvBUtsT1h9zOBJ9LqR2DZ/x/ti9cjOXra73uGnxTigvi+U/8+K1 9nqyN3+8P2GdL2LdI3RhfaxA9qKrrzJ2zNjMk33QyC4kw+m+3Kz7C/n5wlw1Geav3t0mlH/k wrKFZzZo87fviejorBCaqr/WQXxCRY5coaFNs62iV9+sil27lFiKMxINtZiLihMBzy5mff0C AAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmpnkeLIzCtJLcpLzFFi42I5/e/4Vd33PwojDW4/E7N4NbmB0WLqJCOL lv6ZzBbPXyxktrj7YwWTxbTPt9ktrrT/ZLfY+PAsq8XaQx/YLVbcOMVoMffTc0aLuzf/MDnw eCze85LJ49nN/4wez6/1sHi833eVzaNvyypGj3fz37IFsEW52WSkJqakFimk5iXnp2Tmpdsq hYa46VooKeQl5qbaKkXo+oYEKSmUJeaUAnlGBmjAwTnAPVhJ3y7BLePAzibWgm8SFTe/z2dq YJwo3MXIySEhYCJx8NlBJghbTOLCvfVsILaQwBJGiX/tal2MXEB2K5NEy7dZzCAJNgEdiVOr jzCC2CICNhLPFmxlASliFmhllth/dhIrSEJYIEbi+u/9YFNZBFQlei5PAmvmFXCT6J90kRli m5zEzXOdYDangLtES89uFojNbhLN664yTmDkXcDIsIpRJLW0ODc9t9hQrzgxt7g0L10vOT93 EyMwErYd+7l5B+OljcGHGAU4GJV4eBnWFUQKsSaWFVfmHmKU4GBWEuEV+lIYKcSbklhZlVqU H19UmpNafIjRFOioicxSosn5wCjNK4k3NDE0tzQ0MrawMDcyUhLnLflwJVxIID2xJDU7NbUg tQimj4mDU6qBMdL7jfSncqZ9RQ9fCyr8/2BhWsKx37yr+HiG4tktV/lPPQ8q11rAXWhRO+nC 88a/a7W90q+snSzdlVTYp7syal8H62nOOfVZU7Yl/Ngo6hhcUGlqJmjySGN70oJ156LWtTS9 feusdb78W3WG3hKOLduXn5j0wCj2REd9U3Boj/Dx2S93vBV4osRSnJFoqMVcVJwIADgX++Wa AgAA X-MTR: 20000000000000000@CPGS X-CMS-MailID: 20170721125159eucas1p157d8e1da47c5c49aae5e7563b8c85abf X-Msg-Generator: CA X-Sender-IP: 182.198.249.179 X-Local-Sender: =?UTF-8?B?SWx5YSBNYXhpbWV0cxtTUlItVmlydHVhbGl6YXRpb24gTGFi?= =?UTF-8?B?G+yCvOyEseyghOyekBtMZWFkaW5nIEVuZ2luZWVy?= X-Global-Sender: =?UTF-8?B?SWx5YSBNYXhpbWV0cxtTUlItVmlydHVhbGl6YXRpb24gTGFi?= =?UTF-8?B?G1NhbXN1bmcgRWxlY3Ryb25pY3MbTGVhZGluZyBFbmdpbmVlcg==?= X-Sender-Code: =?UTF-8?B?QzEwG0NJU0hRG0MxMEdEMDFHRDAxMDE1NA==?= CMS-TYPE: 201P X-HopCount: 7 X-CMS-RootMailID: 20170721125159eucas1p157d8e1da47c5c49aae5e7563b8c85abf X-RootMTR: 20170721125159eucas1p157d8e1da47c5c49aae5e7563b8c85abf References: <1500641502-31926-1-git-send-email-i.maximets@samsung.com> X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Heetae Ahn , Ilya Maximets Subject: [ovs-dev] [PATCH v4 2/3] dpif-netdev: Avoid port's reconfiguration on pmd-cpu-mask changes. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Reconfiguration of HW NICs may lead to packet drops. In current model all physical ports will be reconfigured each time number of PMD threads changed. Since we not stopping threads on pmd-cpu-mask changes, this patch will help to further decrease port's downtime by setting the maximum possible number of wanted tx queues to avoid unnecessary reconfigurations. Signed-off-by: Ilya Maximets Tested-by: Ian Stokes Acked-by: Ian Stokes --- lib/dpif-netdev.c | 26 +++++++++++++++++++++----- 1 file changed, 21 insertions(+), 5 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 4de3678..a1e8c56 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -3457,7 +3457,7 @@ reconfigure_datapath(struct dp_netdev *dp) { struct dp_netdev_pmd_thread *pmd; struct dp_netdev_port *port; - int wanted_txqs; + int needed_txqs, wanted_txqs; dp->last_reconfigure_seq = seq_read(dp->reconfigure_seq); @@ -3465,7 +3465,15 @@ reconfigure_datapath(struct dp_netdev *dp) * on the system and the user configuration. */ reconfigure_pmd_threads(dp); - wanted_txqs = cmap_count(&dp->poll_threads); + /* We need 1 Tx queue for each thread to avoid locking, but we will try + * to allocate the maximum possible value to minimize the number of port + * reconfigurations. */ + needed_txqs = cmap_count(&dp->poll_threads); + /* (n_cores + 1) is the maximum that we might need to have. + * Additional queue is for non-PMD threads. */ + wanted_txqs = ovs_numa_get_n_cores(); + ovs_assert(wanted_txqs != OVS_CORE_UNSPEC); + wanted_txqs++; /* The number of pmd threads might have changed, or a port can be new: * adjust the txqs. */ @@ -3478,9 +3486,17 @@ reconfigure_datapath(struct dp_netdev *dp) /* Check for all the ports that need reconfiguration. We cache this in * 'port->need_reconfigure', because netdev_is_reconf_required() can - * change at any time. */ + * change at any time. + * Also mark for reconfiguration all ports which will likely change their + * 'dynamic_txqs' parameter. It's required to stop using them before + * changing this setting and it's simpler to mark ports here and allow + * 'pmd_remove_stale_ports' to remove them from threads. There will be + * no actual reconfiguration in 'port_reconfigure' because it's + * unnecessary. */ HMAP_FOR_EACH (port, node, &dp->ports) { - if (netdev_is_reconf_required(port->netdev)) { + if (netdev_is_reconf_required(port->netdev) + || (port->dynamic_txqs + != (netdev_n_txq(port->netdev) < needed_txqs))) { port->need_reconfigure = true; } } @@ -3515,7 +3531,7 @@ reconfigure_datapath(struct dp_netdev *dp) seq_change(dp->port_seq); port_destroy(port); } else { - port->dynamic_txqs = netdev_n_txq(port->netdev) < wanted_txqs; + port->dynamic_txqs = netdev_n_txq(port->netdev) < needed_txqs; } }