From patchwork Sun Apr 11 11:45:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roi Dayan X-Patchwork-Id: 1464793 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=Nvidia.com header.i=@Nvidia.com header.a=rsa-sha256 header.s=selector2 header.b=jWD9DkcL; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FJ96v4F80z9sVw; Sun, 11 Apr 2021 21:45:59 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1lVYXM-0004L0-Jl; Sun, 11 Apr 2021 11:45:56 +0000 Received: from mail-eopbgr750053.outbound.protection.outlook.com ([40.107.75.53] helo=NAM02-BL2-obe.outbound.protection.outlook.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1lVYXJ-0004J0-Gr for kernel-team@lists.ubuntu.com; Sun, 11 Apr 2021 11:45:53 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hjDQRfe1CnUJ2lwk/7Lw9IeProRyV8Dz8uvAh4UdnBEork919FMc03tTbs1BV1Gh9s8AZYQmJo4RE1NlOgf8AcF4rxB8jJqwbr51I0d0ydi3bS7WorRflH5L/V10lCGmbj+NDNBpuLwdEwzw/uqY95BQDweAjfrkx3Q/TEplCW7dmM+YSWEQWckcuTrlfAPGnM0fB1o9W3IstXeFi5JK/vZYx3rfIjc1P4H5iKulAfpnbbRjIAcqg+baR++/p94mq4P5Ggvrusmdl19NA8H12ilZ4USFVNeN4cwmKQQcZD4192oM9jiYck3uXgATaEJ5TzRNzPOkF4tgjZ/f6xEJDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FG//+Qd5ViJ7TW2tsqfSfaoBFlr53uTDB3tQrqZsQnI=; b=Tl3I/n7E3hkiJ5e/vHKGBAxDJxDFiFni1bjIEJI3REG94IslN60XCcvEyQ5sH+mqUJYgbuaqTTGMW5q7PPp67X9DYm0BtsRIBYjLk8yhHbmyioSV3XjM4/ore7UMpJx39RTXvRFjZUMyXJOixIqv3NAwvWkEoyKVkxWkBAZOcHXMlFCEOaSVl10j4kMiJSgt6CaZLC/A3Yob8YBw8qCFh6NdfdpdhXA8Kwk3RsWLlG9HfhfOyJ3IHCp3oOMDBlmAg14aXApWSoKl5eGFtr8l41dUalr86oAXCaWkD12wZj95Y1Tw6MWCcadAVIuTHkJLbsoo5+7qpPcTwzMSoWB2vQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.32) smtp.rcpttodomain=canonical.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FG//+Qd5ViJ7TW2tsqfSfaoBFlr53uTDB3tQrqZsQnI=; b=jWD9DkcL0X6vfDXfUOAvWed/JDtXTgyT5jupl4j2fYXuFULUrt8G7dVnoccTgIiZ+0/xK9lxu8JpWrV0MYJLfcK76K7nMiWwP+x1ENRV5J38rzfeK/D+rYv4TbYaTgQKJxRLuvCwOfJrL2IqItgPJhzIZo1P00HP2YBlInJ2ftvk01YrORhRJTPd5vAPwpqzqLG509aaiAERGECJgBiYi8RM5/S7P0Yl+8CHUKo4AsljjUtmfyiaWui4dSnMSTdvBO7me4Ny3Qn6UZiK8VphMlh95Dutc3n6pQQ2sLjrME/7OmJYdHYpGFFwnE0XL4/lKp59NFhiAV99OKYALhwQ4Q== Received: from DM3PR03CA0015.namprd03.prod.outlook.com (2603:10b6:0:50::25) by BN6PR12MB1923.namprd12.prod.outlook.com (2603:10b6:404:107::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3999.32; Sun, 11 Apr 2021 11:45:51 +0000 Received: from DM6NAM11FT064.eop-nam11.prod.protection.outlook.com (2603:10b6:0:50:cafe::c5) by DM3PR03CA0015.outlook.office365.com (2603:10b6:0:50::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.17 via Frontend Transport; Sun, 11 Apr 2021 11:45:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.32) smtp.mailfrom=nvidia.com; canonical.com; dkim=none (message not signed) header.d=none;canonical.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.32 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.32; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.32) by DM6NAM11FT064.mail.protection.outlook.com (10.13.172.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4020.17 via Frontend Transport; Sun, 11 Apr 2021 11:45:51 +0000 Received: from HQMAIL111.nvidia.com (172.20.187.18) by HQMAIL109.nvidia.com (172.20.187.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 11 Apr 2021 04:45:50 -0700 Received: from dev-r-vrt-138.mtr.labs.mlnx (172.20.145.6) by mail.nvidia.com (172.20.187.18) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sun, 11 Apr 2021 11:45:49 +0000 From: Roi Dayan To: Subject: [SRU][F:linux-bluefield][PATCH 1/2] netfilter: flowtable: separate replace, destroy and stats to different workqueues Date: Sun, 11 Apr 2021 14:45:42 +0300 Message-ID: <20210411114543.2003514-2-roid@nvidia.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210411114543.2003514-1-roid@nvidia.com> References: <20210411114543.2003514-1-roid@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b23e94f5-1b34-4f30-30d3-08d8fcdf5b5f X-MS-TrafficTypeDiagnostic: BN6PR12MB1923: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:151; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NF0SY+nWCQS6KUnwEiTdabJb34gVRnk1SiyMyvODKJPvpxacUvd2SvM5E5crqsOYdmGJV5pvQc2sUG2stPN0ad4T6CwQmw+tK88fDeDcnRIUHLQb6NbUAl7copqC3xfFIaH021lUXvvKpGnQOcZH11SVyCZWwQzStDVlDJL2tyVjRnWtqC1eNvZ1k8B2HRd/ZIKvXs2aWkksXEHfGs1dumcQ3TnNZYqOSQnWflhueJsdNfhiofbFrw2NIlWcSM9jSZV5+JSOreL0N8tZJYD0j8ubw0TkF9J/HzW55s5ajhLO7WhzcKQDWfHZJAAyQQFr1OJfW8TOUt8BTkM/KwXv7MlniEsMZfZOaRlC6RcBN96L1MRkSfOQUDnUE6VUj4V4+6pN/jS3peZ9pNTyt8sxCyKrlq/r+gBylBev0Ko4wU+WYW7KsTo7d0CID5k0QhFOkOcNievRThVOFeQTpmlBJWrCsvwq9Rw5D9FfgDYtUFY2qlNkWs0sJ49Nja/RlsUha3xhP1k7pFtyCP23fDmj0cerXTQz/CMprWf22vy4S2nBHppkCRTBHi7ZLbSJDfW2NvSTSz+Ruz07P/yPJUIun+skdTyvEDJL1p0CQaH8kDZlMQuk7JzDwQ8Wg3AnDxP19n3CXovNPdoBD/dZw/mtnAQTzNvqDiScqxMNGdL+5VZj5xyK0RLkrjcwNg7p2otnT3IKOz21AvkFrs+t/SnI1cUV1TsMIzBHsZayT/KFP20= X-Forefront-Antispam-Report: CIP:216.228.112.32; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid01.nvidia.com; CAT:NONE; SFS:(4636009)(346002)(396003)(39860400002)(376002)(136003)(36840700001)(46966006)(7636003)(356005)(5660300002)(70586007)(26005)(8936002)(2906002)(70206006)(336012)(36860700001)(6666004)(4326008)(86362001)(36756003)(83380400001)(1076003)(478600001)(54906003)(8676002)(47076005)(966005)(2616005)(186003)(316002)(82740400003)(6916009)(82310400003)(426003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Apr 2021 11:45:51.2807 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b23e94f5-1b34-4f30-30d3-08d8fcdf5b5f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.32]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT064.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1923 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vladimir Sokolovsky , Daniel Jurgens Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Oz Shlomo BugLink: https://bugs.launchpad.net/bugs/1922678 Currently the flow table offload replace, destroy and stats work items are executed on a single workqueue. As such, DESTROY and STATS commands may be backloged after a burst of REPLACE work items. This scenario can bloat up memory and may cause active connections to age. Instatiate add, del and stats workqueues to avoid backlogs of non-dependent actions. Provide sysfs control over the workqueue attributes, allowing userspace applications to control the workqueue cpumask. Signed-off-by: Oz Shlomo Reviewed-by: Paul Blakey Signed-off-by: Pablo Neira Ayuso (backported from commit 2ed37183abb70233146dc82f19cb1cbceed2b505) Signed-off-by: Roi Dayan --- net/netfilter/nf_flow_table_offload.c | 44 ++++++++++++++++++++++----- 1 file changed, 36 insertions(+), 8 deletions(-) diff --git a/net/netfilter/nf_flow_table_offload.c b/net/netfilter/nf_flow_table_offload.c index db1797e398ae..685ff5b906e4 100644 --- a/net/netfilter/nf_flow_table_offload.c +++ b/net/netfilter/nf_flow_table_offload.c @@ -12,7 +12,9 @@ #include #include -static struct workqueue_struct *nf_flow_offload_wq; +static struct workqueue_struct *nf_flow_offload_add_wq; +static struct workqueue_struct *nf_flow_offload_del_wq; +static struct workqueue_struct *nf_flow_offload_stats_wq; struct flow_offload_work { struct list_head list; @@ -818,7 +820,12 @@ static void flow_offload_work_handler(struct work_struct *work) static void flow_offload_queue_work(struct flow_offload_work *offload) { - queue_work(nf_flow_offload_wq, &offload->work); + if (offload->cmd == FLOW_CLS_REPLACE) + queue_work(nf_flow_offload_add_wq, &offload->work); + else if (offload->cmd == FLOW_CLS_DESTROY) + queue_work(nf_flow_offload_del_wq, &offload->work); + else + queue_work(nf_flow_offload_stats_wq, &offload->work); } static struct flow_offload_work * @@ -890,8 +897,11 @@ void nf_flow_offload_stats(struct nf_flowtable *flowtable, void nf_flow_table_offload_flush(struct nf_flowtable *flowtable) { - if (nf_flowtable_hw_offload(flowtable)) - flush_workqueue(nf_flow_offload_wq); + if (nf_flowtable_hw_offload(flowtable)) { + flush_workqueue(nf_flow_offload_add_wq); + flush_workqueue(nf_flow_offload_del_wq); + flush_workqueue(nf_flow_offload_stats_wq); + } } static int nf_flow_table_block_setup(struct nf_flowtable *flowtable, @@ -1059,18 +1069,36 @@ static struct flow_indr_block_entry block_ing_entry = { int nf_flow_table_offload_init(void) { - nf_flow_offload_wq = alloc_workqueue("nf_flow_table_offload", - WQ_UNBOUND, 0); - if (!nf_flow_offload_wq) + nf_flow_offload_add_wq = alloc_workqueue("nf_ft_offload_add", + WQ_UNBOUND | WQ_SYSFS, 0); + if (!nf_flow_offload_add_wq) return -ENOMEM; + nf_flow_offload_del_wq = alloc_workqueue("nf_ft_offload_del", + WQ_UNBOUND | WQ_SYSFS, 0); + if (!nf_flow_offload_del_wq) + goto err_del_wq; + + nf_flow_offload_stats_wq = alloc_workqueue("nf_ft_offload_stats", + WQ_UNBOUND | WQ_SYSFS, 0); + if (!nf_flow_offload_stats_wq) + goto err_stats_wq; + flow_indr_add_block_cb(&block_ing_entry); return 0; + +err_stats_wq: + destroy_workqueue(nf_flow_offload_del_wq); +err_del_wq: + destroy_workqueue(nf_flow_offload_add_wq); + return -ENOMEM; } void nf_flow_table_offload_exit(void) { flow_indr_del_block_cb(&block_ing_entry); - destroy_workqueue(nf_flow_offload_wq); + destroy_workqueue(nf_flow_offload_add_wq); + destroy_workqueue(nf_flow_offload_del_wq); + destroy_workqueue(nf_flow_offload_stats_wq); }