From patchwork Fri Apr 21 13:07:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wan Junjie X-Patchwork-Id: 1771919 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=aot+gALd; dkim-atps=neutral Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q2vvV4bftz23tk for ; Fri, 21 Apr 2023 23:07:33 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id DB45B42831; Fri, 21 Apr 2023 13:07:29 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org DB45B42831 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=aot+gALd X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id G_vJRAcPYvCK; Fri, 21 Apr 2023 13:07:28 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 7EDAB41FC6; Fri, 21 Apr 2023 13:07:27 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 7EDAB41FC6 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4ABB9C0036; Fri, 21 Apr 2023 13:07:27 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 21E19C002A for ; Fri, 21 Apr 2023 13:07:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id DB5F1613F5 for ; Fri, 21 Apr 2023 13:07:25 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org DB5F1613F5 Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=aot+gALd X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qx77LfyMefhO for ; Fri, 21 Apr 2023 13:07:24 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 857F760F2C Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by smtp3.osuosl.org (Postfix) with ESMTPS id 857F760F2C for ; Fri, 21 Apr 2023 13:07:24 +0000 (UTC) Received: by mail-pl1-x632.google.com with SMTP id d9443c01a7336-1a6684fd760so3059705ad.0 for ; Fri, 21 Apr 2023 06:07:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1682082443; x=1684674443; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=4sKUbmLzMdo8nOivLp0Cctmt4IG+Wj9D7ZY03LTwFHM=; b=aot+gALdn3d3l6YXL+bEuViL0LpoL6pB98pDqXKMUnEIpKU0b97rjWM00roBQOMe+k 29Q6lBjT2hcD+JXe/7icr+AWTfvO2ct85mWHutlp/BWUjWtsBR0BG2eossN59WB05ngb 461pmTD5dLRux+xGl2xvLBn/EPJdfmp/+vTNHbL6df0Xj/HCZxTaQp8HuWK6dXl0cvYd NYX9w5WcvlTJea+WifUZXHh0acyg0dqg8mxl6YspSHl2KLe09ZqAjv8DVvBcNrJvz7Xp TeXreVN691+dY11y6Xz0ZMA+rCQES4r3Ax3nTKK72kRvkO1AnkWfbt+IAPpEOI/wKooI Lbaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682082443; x=1684674443; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=4sKUbmLzMdo8nOivLp0Cctmt4IG+Wj9D7ZY03LTwFHM=; b=NB2ac0Mx7E0X39sbmma4e1rcIJzba/NSOT1QUHiwBErW/CQ9WX0mkX/+Zye7z63mRS zKpY7qOJeJ095sVSER1wjCvfg4ety7VvyVizqtGt38BNoJgaB12XZyxDBKnMhmpXtqt+ sE5T2pIwZG5xR5U1gHymuMucnjuPzlRoGnSJXXPjGnp48OHalT4B0V76yvU8CDX6E1uD oX2rEJDbBxeeTWm38JtHM68rxEjjkTZNtHPL7vdGoUv+CErNDzDZ2wRqU+RyKUOn7Ov1 /Y/93swZjGejzJ4l5BfMBK5C8JzVb2X4jmjxqPU5QfKd5YVD/invgq5qPLpuD4rEqzid Ys8w== X-Gm-Message-State: AAQBX9eo6dDQQChcxajMpjpjQq0V/9RaYDmG6A0bbI/WbPRf+i5uVusT S+6Snn5jOLJj6j7GcQarx87dytDhwGmnfFT7OtCvCQ== X-Google-Smtp-Source: AKy350a+131ori7ZS5yVqzDPQpylyexHoXPwhwtMWgRRZMu9CIa4t1qyDLarAU9GA53Qa08Es/PszQ== X-Received: by 2002:a17:902:ced0:b0:1a7:db2f:e918 with SMTP id d16-20020a170902ced000b001a7db2fe918mr5886134plg.1.1682082443143; Fri, 21 Apr 2023 06:07:23 -0700 (PDT) Received: from C02Z10J4LVCK.bytedance.net ([139.177.225.230]) by smtp.gmail.com with ESMTPSA id nm20-20020a17090b19d400b00249604258b1sm4563363pjb.38.2023.04.21.06.07.21 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Apr 2023 06:07:22 -0700 (PDT) To: dev@openvswitch.org Date: Fri, 21 Apr 2023 21:07:18 +0800 Message-Id: <20230421130718.95794-1-wanjunjie@bytedance.com> X-Mailer: git-send-email 2.39.1 MIME-Version: 1.0 Subject: [ovs-dev] [PATCH] utilities: taskset for non-pmd threads X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Wan Junjie via dev From: Wan Junjie Reply-To: Wan Junjie Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Deployment with dpdk and spdk should limit the core of non-pmd threads. When starting ovs with ovs-ctl or systemd, the process will inherit the affinity of its parent process. Even if we manuanlly set the affinity, restarting after crash will lose the setting. Prepend taskset in ovs-ctl when start daemon, so it will set the affinity each time starting the ovs. Signed-off-by: Wan Junjie --- Documentation/ref/ovs-ctl.8.rst | 5 +++++ utilities/ovs-ctl.in | 8 ++++++-- utilities/ovs-lib.in | 6 ++++++ 3 files changed, 17 insertions(+), 2 deletions(-) diff --git a/Documentation/ref/ovs-ctl.8.rst b/Documentation/ref/ovs-ctl.8.rst index 9f077a122..603e9ca1e 100644 --- a/Documentation/ref/ovs-ctl.8.rst +++ b/Documentation/ref/ovs-ctl.8.rst @@ -199,6 +199,11 @@ The following options are less important: Sets the ``nice(1)`` level used for each daemon. All of them default to ``-10``. +* ``--ovsdb-server-affinity=`` or + ``--ovs-vswitchd-affinity=`` + + Sets the ``taskset(1)`` affinity for each daemon process. + * ``--ovsdb-server-wrapper=`` or ``--ovs-vswitchd-wrapper=`` diff --git a/utilities/ovs-ctl.in b/utilities/ovs-ctl.in index 0b2820c36..3f033a98e 100644 --- a/utilities/ovs-ctl.in +++ b/utilities/ovs-ctl.in @@ -157,7 +157,7 @@ do_start_ovsdb () { [ "$OVSDB_SERVER_OPTIONS" != "" ] && set "$@" $OVSDB_SERVER_OPTIONS start_daemon "$OVSDB_SERVER_PRIORITY" "$OVSDB_SERVER_WRAPPER" \ - "$OVSDB_SERVER_UMASK" "$@" || return 1 + "$OVSDB_SERVER_UMASK" "$OVSDB_SERVER_AFFINITY" "$@" || return 1 # Initialize database settings. ovs_vsctl -- init -- set Open_vSwitch . db-version="$schemaver" \ @@ -227,7 +227,7 @@ do_start_forwarding () { [ "$OVS_VSWITCHD_OPTIONS" != "" ] &&set "$@" $OVS_VSWITCHD_OPTIONS start_daemon "$OVS_VSWITCHD_PRIORITY" "$OVS_VSWITCHD_WRAPPER" \ - "$OVS_VSWITCHD_UMASK" "$@" || return 1 + "$OVS_VSWITCHD_UMASK" "$OVS_VSWITCHD_AFFINITY" "$@" || return 1 fi } @@ -344,6 +344,8 @@ set_defaults () { OVS_VSWITCHD=yes OVSDB_SERVER_PRIORITY=-10 OVS_VSWITCHD_PRIORITY=-10 + OVSDB_SERVER_AFFINITY= + OVS_VSWITCHD_AFFINITY= OVSDB_SERVER_WRAPPER= OVS_VSWITCHD_WRAPPER= OVSDB_SERVER_OPTIONS= @@ -436,8 +438,10 @@ Less important options for "start", "restart" and "force-reload-kmod": --dump-hugepages include hugepages in core dumps --no-mlockall do not lock all of ovs-vswitchd into memory --ovsdb-server-priority=NICE set ovsdb-server's niceness (default: $OVSDB_SERVER_PRIORITY) + --ovsdb-server-affinity=CORE set ovsdb-server's affinity (default: $OVSDB_SERVER_AFFINITY) --ovsdb-server-options=OPTIONS additional options for ovsdb-server (example: '-vconsole:dbg -vfile:dbg') --ovs-vswitchd-priority=NICE set ovs-vswitchd's niceness (default: $OVS_VSWITCHD_PRIORITY) + --ovs-vswitchd-affinity=CORE set ovs-vswitchd's affinity (default: $OVS_VSWITCHD_AFFINITY) --ovs-vswitchd-options=OPTIONS additional options for ovs-vswitchd (example: '-vconsole:dbg -vfile:dbg') --no-full-hostname set short hostname instead of full hostname --no-record-hostname do not attempt to determine/record system diff --git a/utilities/ovs-lib.in b/utilities/ovs-lib.in index 7812a94ee..f48d62376 100644 --- a/utilities/ovs-lib.in +++ b/utilities/ovs-lib.in @@ -168,6 +168,7 @@ start_daemon () { priority=$1 && shift wrapper=$1 && shift umask=$1 && shift + affinity=$1 && shift daemon=$1 strace="" @@ -223,6 +224,11 @@ start_daemon () { set nice -n "$priority" "$@" fi + # affinity + if test X"$affinity" != X; then + set taskset -c "$affinity" "$@" + fi + # Set requested umask if any and turn previous value back. if [ -n "$umask" ]; then previuos_umask_value=$(umask)