From patchwork Fri Jul 13 18:36:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Sitnicki X-Patchwork-Id: 943763 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41S1jZ2083z9s3R for ; Sat, 14 Jul 2018 04:36:33 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id C9BADDDC; Fri, 13 Jul 2018 18:36:29 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 1CBF0DA1 for ; Fri, 13 Jul 2018 18:36:28 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-wm0-f54.google.com (mail-wm0-f54.google.com [74.125.82.54]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 53CF97B0 for ; Fri, 13 Jul 2018 18:36:27 +0000 (UTC) Received: by mail-wm0-f54.google.com with SMTP id i139-v6so9621994wmf.4 for ; Fri, 13 Jul 2018 11:36:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=KPhW1SVXeHxOVio3M3guRdmbXNey3v8zRB/09A0+tf0=; b=exp8HzmQe+XERN6n06LcUiTkH/CuQpwKs/uF79Vs/2cCshkU7wyHoEtUegTzEE7WaG OiHScopftcDMVvUJ3kMZDX3Prfspp6dEIbUuo/dnVX3S9oRtT7i29NwPQ9wU3Pwcw5qx pQcqhDy/7J9ESg/1CQAFl0izwzBzBfr2xco4t/xI0N/WWCjaZnxlmxADLjJq4/8iYy88 Zo1MqZh5YcAuE55aWcmfNfU37Cas42h6NY1YNwOpaQbIIctKM80Gve/NoDF4cTGqgbCH GVD8DpfuG74MB2V7j+JG14oxzEJETNagf+EbnCBh6NzW8kFWk226Nnxq8z2vF6Z0Fs9u p0Gw== X-Gm-Message-State: AOUpUlEKQV2871vqY6JnnPEV+fssxxg8fmA5RsI2EH7JLoqO7pjPkMl+ tSlulRR0CSkL4hcAJdp1t0/l3ca4HlE= X-Google-Smtp-Source: AAOMgpeawYljKDoWYAmZPGS7E0o0YKS0N1e5ie7l6txcTDfQXa3gKs0GyvoYmU9VejfbTW2A3TKdag== X-Received: by 2002:a1c:7c0c:: with SMTP id x12-v6mr4695081wmc.58.1531506985665; Fri, 13 Jul 2018 11:36:25 -0700 (PDT) Received: from redhat.com (218.red-2-136-166.dynamicip.rima-tde.net. [2.136.166.218]) by smtp.gmail.com with ESMTPSA id z5-v6sm30746150wrv.2.2018.07.13.11.36.24 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 13 Jul 2018 11:36:24 -0700 (PDT) From: Jakub Sitnicki To: dev@openvswitch.org Date: Fri, 13 Jul 2018 20:36:07 +0200 Message-Id: <20180713183624.16937-1-jkbs@redhat.com> X-Mailer: git-send-email 2.14.4 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v3 00/17] Daemon mode for ovn-nbctl X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org This series extends ovn-nbctl tool with support for the daemon mode, where ovn-nbctl acts a long-lived process that accepts commands over a UNIX socket. The daemon can be started the same way as any other OVS/OVN server: ovn-nbctl --detach --pidfile --log-file While commands can be issued to it using the 'ovs-appctl' tool: ovs-appctl -t ovn-nbctl run [OPTIONS] COMMAND [-- [OPTIONS] COMMAND] ... Although, the plan for future work is to extend ovn-nbctl so that it can control the daemon, if one is running. The motivation and the main benefit from the daemon mode is that the contents of NB database have to be obtained only once, when the first command is ran. On big databases (1000's of logical ports) this results in a speed up per command in the range of 100's of milliseconds. The changes are functional to the point that all test cases in the ovn-nbctl test suite (tests/ovn-nbctl.at) and OVN test suite (tests/ovn.at) pass when running ovn-nbctl as a daemon [1]. However, I'm still thinking how to nicely integrate the daemon mode with the test suite so that the existing tests can be run using either the normal or the daemon mode. Any ideas? A dirty, just-for-development integration with tests is available at: https://github.com/jsitnicki/ovs/commits/wip-nbctl-daemon-2018-07-13 Also, not all places that call ctl_fatal() have been converted yet to propagate the error to the caller. As a result hitting an unconverted an error path will cause the daemon process to die. This can be worked around with the --monitor option. Daemon mode should be considered experimental. Changes v2 -> v3: - Wait for IDL to connect before detaching and running the unixctl server. Also, terminate the daemon when the connection to the NB DB has failed. This behavior is modeled after ovn-trace daemon mode. - Add a dedicated option parser for the daemon. Parse only the options that are have an effect on the main loop. Logging options are not supported. Usual way to change the log levels for daemons is with 'vlog/set' command. Reported by Mark Michelson. - Drop the test for 'sync' command. The test is flawed as pointed out by Mark Michelson. Reading the '*_cfg' value after 'ovn-nbctl --wait=X sync' has returned does not prove that '--wait' has blocked until the '*_cfg' value has changed. At the same time, we cannot read out the updated '*_cfg' value in the same transaction as we change it in. - To review just the interdiff run: git fetch --tags https://github.com/jsitnicki/ovs git diff wip-nbctl-daemon-2018-07-{12,13} Changes v1 -> v2: - Work around a limitation in GCC 4.8.5 (RHEL/CentOS 7.5) that doesn't let us use a constant expression as a RHS value in a structure assigment. Found by the 0-day bot. Changes RFC -> v1: - Rebase onto master @ 61b1c7acb9a2 ("netdev-bsd: Fix crash on FreeBSD."). - Add support for commands that use tabular output ('find' and 'list') thanks to recent work in table module by Ben Pfaff. This includes support for options that control table formatting. - Run prerequisites routines. In the end this is needed to support the 'sync' command, which itself is more like an option than command (has to get processed before the commands run). This is also the reason for minor changes in the IDL. - Add support for --dry-run, --wait / --no-wait, --timeout, and --oneline options. Timeout handling is different than in the normal mode - we only time out in poll_block(). Still, it serves the purpose avoiding an infinite 'sync'. - Add tests for ovn-nbctl 'sync' command, dry-run mode, and oneline-formatted output. - Extend the ovn-nbctl man-page with description of daemon mode. - Remove extraneous return at the end of a void function. Pointed out by Ben Pfaff. - Drop WIP patch for integration with tests from the series until a right approach can be found. Integration with tests that was used during development is available at: https://github.com/jsitnicki/ovs/commits/wip-nbctl-daemon-2018-07-11 - Minor style cleanups. Thanks, Jakub [1] Except test "2563: ovn -- IPv6 ND Router Solicitation responder" which seems broken in master @ 61b1c7acb9a2 ("netdev-bsd: Fix crash on FreeBSD."). Jakub Sitnicki (17): table: Introduce a constant for default table style. ovsdb-idl: Allow monitoring columns that are already monitored. ovn-nbctl: Extract the main loop. ovn-nbctl: Pull up destroying commands from do_nbctl(). ovn-nbctl: Pull up releasing IDL from do_nbctl(). ovn-nbctl: Signal need to try again via an output param. ovn-nbctl: Propagate the error from do_nbctl(). ovn-nbctl: Propagate errors from the main loop. ovn-nbctl: Propagate errors from prerequisites runner. ovn-nbctl: Introduce a poll_timer based wait timeout. ovn-nbctl: Extract helper for printing oneline output. ovn-nbctl: Extract handling of options that affect main loop. ovn-nbctl: Extract a helper for building short options string. ovn-nbctl: Extract a helper for appending command options. ovn-nbctl: Initial support for daemon mode. tests: Add test for ovn-nbctl dry run mode. tests: Add test for oneline-formatted output for ovn-nbctl. lib/ovsdb-idl.c | 16 +- lib/table.h | 2 + ovn/utilities/ovn-nbctl.8.xml | 40 +++ ovn/utilities/ovn-nbctl.c | 654 +++++++++++++++++++++++++++++++++--------- tests/ovn-nbctl.at | 42 +++ 5 files changed, 622 insertions(+), 132 deletions(-) Acked-by: Mark Michelson --- 2.14.4