From patchwork Tue Feb 6 13:30:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eelco Chaudron X-Patchwork-Id: 1895726 X-Patchwork-Delegate: echaudro@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=XqfkL8v0; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TTkfW20Zrz23hV for ; Wed, 7 Feb 2024 00:31:15 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 5338761227; Tue, 6 Feb 2024 13:31:13 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 5338761227 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=XqfkL8v0 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uVy2DvNDTQIF; Tue, 6 Feb 2024 13:31:12 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 7F59E61211; Tue, 6 Feb 2024 13:31:11 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 7F59E61211 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 381FDC0072; Tue, 6 Feb 2024 13:31:11 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9356BC0037 for ; Tue, 6 Feb 2024 13:31:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 6E5BE41E9A for ; Tue, 6 Feb 2024 13:31:09 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 6E5BE41E9A Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=XqfkL8v0 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rBZ3a5V6tq3p for ; Tue, 6 Feb 2024 13:31:07 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 6556541E98 for ; Tue, 6 Feb 2024 13:31:07 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 6556541E98 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1707226266; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=VNwpOXECVwGbiUNeH22ha+7cDvhKe5RP6h7Rkv1gyAo=; b=XqfkL8v0ubTFci+ru1PQZ9aCT+2SWo6zXBcFLHT8dyzq7izNFCwVdVjqLzQG9PDOvQPkk2 egXptD1kEnLOXdnDnWfKVbZTgWNJ0FFUulg5NgtyTRNLOFs49IeYnAhqMrstTMU+5kgE3F XIfWUr3sdO/nkxSYw59HYa4NOL4bVjk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-632-2ypegpION6mnK5GIahusEA-1; Tue, 06 Feb 2024 08:31:05 -0500 X-MC-Unique: 2ypegpION6mnK5GIahusEA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EF459185A78B for ; Tue, 6 Feb 2024 13:31:04 +0000 (UTC) Received: from ebuild.redhat.com (unknown [10.39.194.234]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9CEF6111F9 for ; Tue, 6 Feb 2024 13:31:04 +0000 (UTC) From: Eelco Chaudron To: dev@openvswitch.org Date: Tue, 6 Feb 2024 14:30:22 +0100 Message-ID: <91101df319805c55dc05597cb34d36cbd8bce867.1707226223.git.echaudro@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH 1/2] dpif-netdev: Do not create handler threads. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Avoid unnecessary thread creation as no upcalls are generated, resulting in idle threads waiting for process termination. This optimization significantly reduces memory usage, cutting it by half on a 128 CPU/thread system during testing, with the number of threads reduced from 95 to 0. Signed-off-by: Eelco Chaudron --- lib/dpif-netdev.c | 10 +++++++++- ofproto/ofproto-dpif-upcall.c | 25 ++++++++++++++++--------- 2 files changed, 25 insertions(+), 10 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index c1981137f..1a29ab7ac 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -5250,6 +5250,14 @@ dpif_netdev_set_config(struct dpif *dpif, const struct smap *other_config) return 0; } +static bool +dpif_netdev_number_handlers_required(struct dpif *dpif_ OVS_UNUSED, + uint32_t *n_handlers) +{ + *n_handlers = 0; + return true; +} + /* Parses affinity list and returns result in 'core_ids'. */ static int parse_affinity_list(const char *affinity_list, unsigned *core_ids, int n_rxq) @@ -9985,7 +9993,7 @@ const struct dpif_class dpif_netdev_class = { dpif_netdev_offload_stats_get, NULL, /* recv_set */ NULL, /* handlers_set */ - NULL, /* number_handlers_required */ + dpif_netdev_number_handlers_required, dpif_netdev_set_config, dpif_netdev_queue_to_priority, NULL, /* recv */ diff --git a/ofproto/ofproto-dpif-upcall.c b/ofproto/ofproto-dpif-upcall.c index b5cbeed87..9a5c5c29c 100644 --- a/ofproto/ofproto-dpif-upcall.c +++ b/ofproto/ofproto-dpif-upcall.c @@ -584,7 +584,7 @@ static void udpif_start_threads(struct udpif *udpif, uint32_t n_handlers_, uint32_t n_revalidators_) { - if (udpif && n_handlers_ && n_revalidators_) { + if (udpif && n_revalidators_) { /* Creating a thread can take a significant amount of time on some * systems, even hundred of milliseconds, so quiesce around it. */ ovsrcu_quiesce_start(); @@ -592,14 +592,19 @@ udpif_start_threads(struct udpif *udpif, uint32_t n_handlers_, udpif->n_handlers = n_handlers_; udpif->n_revalidators = n_revalidators_; - udpif->handlers = xzalloc(udpif->n_handlers * sizeof *udpif->handlers); - for (size_t i = 0; i < udpif->n_handlers; i++) { - struct handler *handler = &udpif->handlers[i]; + if (udpif->n_handlers) { + udpif->handlers = xzalloc(udpif->n_handlers + * sizeof *udpif->handlers); + for (size_t i = 0; i < udpif->n_handlers; i++) { + struct handler *handler = &udpif->handlers[i]; - handler->udpif = udpif; - handler->handler_id = i; - handler->thread = ovs_thread_create( - "handler", udpif_upcall_handler, handler); + handler->udpif = udpif; + handler->handler_id = i; + handler->thread = ovs_thread_create( + "handler", udpif_upcall_handler, handler); + } + } else { + udpif->handlers = NULL; } atomic_init(&udpif->enable_ufid, udpif->backer->rt_support.ufid); @@ -662,7 +667,9 @@ udpif_set_threads(struct udpif *udpif, uint32_t n_handlers_, if (dpif_number_handlers_required(udpif->dpif, &n_handlers_requested)) { forced = true; if (!n_revalidators_) { - n_revalidators_requested = n_handlers_requested / 4 + 1; + n_revalidators_requested = (n_handlers_requested + ? n_handlers_requested + : MAX(count_cpu_cores(), 2)) / 4 + 1; } else { n_revalidators_requested = n_revalidators_; }