From patchwork Thu Mar 22 18:42:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 889547 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="AJQ4QkAD"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 406bBn3mrSz9s0y for ; Fri, 23 Mar 2018 05:42:40 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 65D08119A; Thu, 22 Mar 2018 18:42:37 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id D9CBE1162 for ; Thu, 22 Mar 2018 18:42:35 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pf0-f170.google.com (mail-pf0-f170.google.com [209.85.192.170]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 74A485D3 for ; Thu, 22 Mar 2018 18:42:35 +0000 (UTC) Received: by mail-pf0-f170.google.com with SMTP id m68so3708787pfm.11 for ; Thu, 22 Mar 2018 11:42:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=FRl7+lThUEBH/2XnhRtH+CJ7KQCUiWOkz5i2Pb5TbQY=; b=AJQ4QkADgR8pBy2+e0BkCaJYswxIznDMXefFDbHqoqyweIfMoIuZM+RwMjLWJgaJsF St8mQ6zGjEuaH4zxOmHrcICiilfKtzsNHSoyFngXI0R96CAr/a7CEDWCxJX9e0hTMz7q mzb0PmuTc5GzPnvuZ0ClRGipHRFOe4XhKHJSzqLDU9BZCHCQfODUUmvbzoT3YglQhnUO U2pEJNPS491qHifvi6uRhFzC1Z1TtnwmESzs9wQIAFUKsUTU5j0ELYvDEKZvLfRLesRj OygfYaEwReDonSgMt/zKjPE5Mg3F5qS1D37BkDTjAU69cy29slfZtK84onHnwVpLoFGV U95w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=FRl7+lThUEBH/2XnhRtH+CJ7KQCUiWOkz5i2Pb5TbQY=; b=dvBIL7xYXDo1XcMVI19ZVMu5v1ZPFiCr9q/3rJPuce4th4nddVuS/JL8HO/H0xVzF+ E7LbbPIxRMm4RgkDomicl9Ux67RE4qf3rw+u+YsjKCPDC2W01UqIIllXA71rE1PrrhKX cIZUe/+4KKFGg0YAUDDZ1sBSt4vYq3b2iw1HXc6UMeo/nwoP1gP/5pylzuxP775NO04u 6LIaiQgLsCdJupEc4bF9l0Ti0dajC1WQ0AaX9of0Sg0iD5vJpbnCwFerEL4F71za+DXR M8L8nPWMJV3xwZIR5QizIkoheiVJ//1cAx9aAdo41f9dUVx/CTfwxe1BX7Tk0fbrawlL PojQ== X-Gm-Message-State: AElRT7FX89j3sj9UFcbZp5O8ZZ3ILIjGrGupOa4Hfk/62bNQv4ijX/Ob uKm/iAoMMobmkV8kVxQx3Bps7Q== X-Google-Smtp-Source: AG47ELu5Op4RQECrYHOFEIVV4vkpGnarbh1p/NGyCqtep+58sBoeCpOa/tuJiyyFcyCTFsYdzjsQWg== X-Received: by 10.99.109.75 with SMTP id i72mr13360924pgc.403.1521744154677; Thu, 22 Mar 2018 11:42:34 -0700 (PDT) Received: from localhost.localdomain.localdomain ([216.113.160.71]) by smtp.gmail.com with ESMTPSA id d83sm9959319pfl.176.2018.03.22.11.42.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Mar 2018 11:42:34 -0700 (PDT) From: Han Zhou X-Google-Original-From: Han Zhou To: dev@openvswitch.org Date: Thu, 22 Mar 2018 11:42:17 -0700 Message-Id: <1521744147-23451-1-git-send-email-hzhou8@ebay.com> X-Mailer: git-send-email 2.1.0 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [RFC v2 00/10] ovn-controller Incremental Processing X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org ovn-controller currently recomputes everything when there are any changes of input, which leads to high CPU usages and slow in end-to-end flow enforcement in response to changes. It even wastes CPU to recompute flows for unrelated inputs such as pinctrl events. This patch series implements incremental processing in ovn-controller to solve above problems. There has been a similar attempt of solve the problem earlier but was reverted (see commit: 926c34fd). This patch series takes a different approach with an incremental processing engine, to make the dependencies clear and easier to maintain. The engine is a DAG representing dependencies between different nodes. Each node maintains its own data, which depends on its inputs and the data can also be inputs of other nodes. Each node implements a method to recompute its data based on all the inputs, but also implements methods to handle changes of different inputs incrementally. The engine will be responsible to try incremental processing for each node based on the dependencies or fallback to recompute when changes cannot be handled incrementally. This patch series can incrementally process the most common changes: logical flows and port bindings from OVNSB. It can be expanded further for more fine grained incremental processing gradually. With the patch series, the CPU time of ovn-controller on HVs in ovn-scale-test for scenario lports creating and binding decreased 90%. The total execution time of the scale testing, creating and binding 10k ports on 1k HVs with 40 lswitches and 8 lrouters (5 lswitches/lrouter), also decreased from 3h40m to 1h50m because of the less CPU on HVs, without any change on central node components. Latency for end-to-end port-creation to binding and flows installation on all related HVs is also improved significantly (comparison data is still under analysis). This is RFC version to get feedback to see if there is any major issue of this approach, before refining it further for formal review. --- v1 -> v2: - Fixed the last 2 failed test cases by updates in port-binding change handler - Avoid forced recompute when not needed - Added incremental processing for multicast group changes, which is common during lport creation/deletion - Some other minor fixes and refactors, and rebased on lastest master Han Zhou (10): ovn-controller: Incremental processing engine ovn-controller: Track OVSDB changes ovn-controller: Initial use of incremental engine in main ovn-controller: Split SB inputs as separate incremental engine nodes ovn-controller: split ovs_idl inputs in incremental engine ovn-controller: Incremental logical flow processing ovn-controller: runtime_data change handler for SB port-binding ovn-controller: port-binding incremental processing for physical flows Avoid force recompute when not needed ovn-controller: incremental processing for multicast group changes include/ovn/actions.h | 3 + ovn/controller/bfd.c | 4 +- ovn/controller/binding.c | 101 ++++++- ovn/controller/binding.h | 6 + ovn/controller/encaps.c | 12 +- ovn/controller/lflow.c | 110 ++++++-- ovn/controller/lflow.h | 10 +- ovn/controller/ofctrl.c | 263 +++++++++++++----- ovn/controller/ofctrl.h | 20 +- ovn/controller/ovn-controller.c | 584 +++++++++++++++++++++++++++++++--------- ovn/controller/ovn-controller.h | 5 + ovn/controller/physical.c | 235 ++++++++++------ ovn/controller/physical.h | 14 +- ovn/lib/actions.c | 6 +- ovn/lib/automake.mk | 4 +- ovn/lib/extend-table.c | 31 ++- ovn/lib/extend-table.h | 9 +- ovn/lib/inc-proc-eng.c | 116 ++++++++ ovn/lib/inc-proc-eng.h | 121 +++++++++ 19 files changed, 1315 insertions(+), 339 deletions(-) create mode 100644 ovn/lib/inc-proc-eng.c create mode 100644 ovn/lib/inc-proc-eng.h