From patchwork Tue May 17 10:08:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ferriter, Cian" X-Patchwork-Id: 1632200 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=NmTlpSt6; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4L2WzV08Mxz9s0w for ; Tue, 17 May 2022 20:08:37 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 5523083E6F; Tue, 17 May 2022 10:08:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8AHHPJtRtO93; Tue, 17 May 2022 10:08:34 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 7BCB483E09; Tue, 17 May 2022 10:08:33 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1CEEEC007B; Tue, 17 May 2022 10:08:33 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 26894C0082 for ; Tue, 17 May 2022 10:08:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 06CBD60FA0 for ; Tue, 17 May 2022 10:08:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id s0rW0n6RsB8u for ; Tue, 17 May 2022 10:08:30 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by smtp3.osuosl.org (Postfix) with ESMTPS id 45C0360E38 for ; Tue, 17 May 2022 10:08:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652782110; x=1684318110; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=OrYlxxRTutkCmpQ92Gx2dud+Pudma2QG0jj5iA5ESC8=; b=NmTlpSt6gwPQ3yEyB5uak8BwIum6F+O0SwBP8ePQ0L/lb51K47Wu5eOA SNxQP3C0dcbFKkmouiXCSqZh9kOBJdTMdOsm++Nxg0qFx7A+n4ftIyJPC w4hg+hcDxJygH10GAlDhcu1XGO7Oi94lClS6xflJnrPKUx4ANjOb7O1cL hu8FMlRYWTqDqG5ccYscI15PI2CY8xz0kVB/WRAUmBoNBEkGSZAaUXbd8 rlNSx7kd7IkDIR02l+hU55Yr1QuY8hydO3K76UqND/0IhgzcbXsKtulBV HjyJK3G1wdUHy/gNvlEYDy1QscDsp1mLUKL9ryGjG7OemNyzMVzvviyPk w==; X-IronPort-AV: E=McAfee;i="6400,9594,10349"; a="258696490" X-IronPort-AV: E=Sophos;i="5.91,232,1647327600"; d="scan'208";a="258696490" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2022 03:08:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,232,1647327600"; d="scan'208";a="699986263" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.111]) by orsmga004.jf.intel.com with ESMTP; 17 May 2022 03:08:27 -0700 From: Cian Ferriter To: ovs-dev@openvswitch.org Date: Tue, 17 May 2022 10:08:14 +0000 Message-Id: <20220517100818.15639-2-cian.ferriter@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220517100818.15639-1-cian.ferriter@intel.com> References: <20220429150426.2318121-1-cian.ferriter@intel.com> <20220517100818.15639-1-cian.ferriter@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH v5 1/5] dpif-netdev-private-extract: Fix typo VMBI -> VBMI. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Fixes: 250ceddcc2d0 ("dpif-netdev/mfex: Add AVX512 based optimized miniflow extract") Fixes: aa85a25095ae ("dpif-netdev/mfex: Add more AVX512 traffic profiles") Signed-off-by: Cian Ferriter Acked-by: Sunil Pai G --- v4: * Added Sunil's Fixes and Acked-by tags. --- lib/dpif-netdev-private-extract.c | 8 ++++---- lib/dpif-netdev-private-extract.h | 10 +++++----- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/lib/dpif-netdev-private-extract.c b/lib/dpif-netdev-private-extract.c index 4b2f12015..b7f094dac 100644 --- a/lib/dpif-netdev-private-extract.c +++ b/lib/dpif-netdev-private-extract.c @@ -55,7 +55,7 @@ static struct dpif_miniflow_extract_impl mfex_impls[] = { /* Compile in implementations only if the compiler ISA checks pass. */ #if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && __SSE4_2__) - [MFEX_IMPL_VMBI_IPv4_UDP] = { + [MFEX_IMPL_VBMI_IPv4_UDP] = { .probe = mfex_avx512_vbmi_probe, .extract_func = mfex_avx512_vbmi_ip_udp, .name = "avx512_vbmi_ipv4_udp", }, @@ -65,7 +65,7 @@ static struct dpif_miniflow_extract_impl mfex_impls[] = { .extract_func = mfex_avx512_ip_udp, .name = "avx512_ipv4_udp", }, - [MFEX_IMPL_VMBI_IPv4_TCP] = { + [MFEX_IMPL_VBMI_IPv4_TCP] = { .probe = mfex_avx512_vbmi_probe, .extract_func = mfex_avx512_vbmi_ip_tcp, .name = "avx512_vbmi_ipv4_tcp", }, @@ -75,7 +75,7 @@ static struct dpif_miniflow_extract_impl mfex_impls[] = { .extract_func = mfex_avx512_ip_tcp, .name = "avx512_ipv4_tcp", }, - [MFEX_IMPL_VMBI_DOT1Q_IPv4_UDP] = { + [MFEX_IMPL_VBMI_DOT1Q_IPv4_UDP] = { .probe = mfex_avx512_vbmi_probe, .extract_func = mfex_avx512_vbmi_dot1q_ip_udp, .name = "avx512_vbmi_dot1q_ipv4_udp", }, @@ -85,7 +85,7 @@ static struct dpif_miniflow_extract_impl mfex_impls[] = { .extract_func = mfex_avx512_dot1q_ip_udp, .name = "avx512_dot1q_ipv4_udp", }, - [MFEX_IMPL_VMBI_DOT1Q_IPv4_TCP] = { + [MFEX_IMPL_VBMI_DOT1Q_IPv4_TCP] = { .probe = mfex_avx512_vbmi_probe, .extract_func = mfex_avx512_vbmi_dot1q_ip_tcp, .name = "avx512_vbmi_dot1q_ipv4_tcp", }, diff --git a/lib/dpif-netdev-private-extract.h b/lib/dpif-netdev-private-extract.h index f9a757ba4..ae5c161b4 100644 --- a/lib/dpif-netdev-private-extract.h +++ b/lib/dpif-netdev-private-extract.h @@ -82,13 +82,13 @@ enum dpif_miniflow_extract_impl_idx { MFEX_IMPL_SCALAR, MFEX_IMPL_STUDY, #if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && __SSE4_2__) - MFEX_IMPL_VMBI_IPv4_UDP, + MFEX_IMPL_VBMI_IPv4_UDP, MFEX_IMPL_IPv4_UDP, - MFEX_IMPL_VMBI_IPv4_TCP, + MFEX_IMPL_VBMI_IPv4_TCP, MFEX_IMPL_IPv4_TCP, - MFEX_IMPL_VMBI_DOT1Q_IPv4_UDP, + MFEX_IMPL_VBMI_DOT1Q_IPv4_UDP, MFEX_IMPL_DOT1Q_IPv4_UDP, - MFEX_IMPL_VMBI_DOT1Q_IPv4_TCP, + MFEX_IMPL_VBMI_DOT1Q_IPv4_TCP, MFEX_IMPL_DOT1Q_IPv4_TCP, #endif MFEX_IMPL_MAX @@ -101,7 +101,7 @@ extern struct ovs_mutex dp_netdev_mutex; */ #if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && __SSE4_2__) -#define MFEX_IMPL_START_IDX MFEX_IMPL_VMBI_IPv4_UDP +#define MFEX_IMPL_START_IDX MFEX_IMPL_VBMI_IPv4_UDP #else #define MFEX_IMPL_START_IDX MFEX_IMPL_MAX From patchwork Tue May 17 10:08:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ferriter, Cian" X-Patchwork-Id: 1632201 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=ABM4mrFz; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4L2WzW6B3hz9s0w for ; Tue, 17 May 2022 20:08:39 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 89FBF611A5; Tue, 17 May 2022 10:08:37 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gmWqS13xKNRv; Tue, 17 May 2022 10:08:36 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 6517361089; Tue, 17 May 2022 10:08:35 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2543CC007B; Tue, 17 May 2022 10:08:35 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6B932C0032 for ; Tue, 17 May 2022 10:08:32 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id BB5AD60FA0 for ; Tue, 17 May 2022 10:08:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yqsX3b6mUmTl for ; Tue, 17 May 2022 10:08:31 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by smtp3.osuosl.org (Postfix) with ESMTPS id 1AF2460E38 for ; Tue, 17 May 2022 10:08:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652782111; x=1684318111; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=G+XPQkIFRJ2eQD/BacbqRFJsQCG3su6yo/i5WmgXQSY=; b=ABM4mrFzYGEsEvzqGIm0woV4P/Lm2fS4YgQuCwtr9GpXkOUYR3XWcckL Hpytnzv2Fi0zx7fYoBniMbMseL6u3Jw9a1BV4JvP3Yo0eie8tcIgI77VF lPzaM5n35BtTn1KnYkyrZYnN49ChEZZmB4LjrOZ4FfKixDfLBAAZZJ8Ht eokkzfl07ODrZ8HyB5pNaX6zpLRIA5mou7iDVGKldRwzI3VWevS3+xQeW +EVA8D8ifL1ZWEXtayVAATbIr/GV2TrZPNRNTbpWQ71aW9hGiEL23NyLW kEjj7d9KHaooOaCrS0fiSXIaCZeDG4olHcXUoCz8ZC1xVEpVYsxfu2QpE w==; X-IronPort-AV: E=McAfee;i="6400,9594,10349"; a="258696501" X-IronPort-AV: E=Sophos;i="5.91,232,1647327600"; d="scan'208";a="258696501" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2022 03:08:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,232,1647327600"; d="scan'208";a="699986270" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.111]) by orsmga004.jf.intel.com with ESMTP; 17 May 2022 03:08:29 -0700 From: Cian Ferriter To: ovs-dev@openvswitch.org Date: Tue, 17 May 2022 10:08:15 +0000 Message-Id: <20220517100818.15639-3-cian.ferriter@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220517100818.15639-1-cian.ferriter@intel.com> References: <20220429150426.2318121-1-cian.ferriter@intel.com> <20220517100818.15639-1-cian.ferriter@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH v5 2/5] dpif-netdev-lookup: Fix GCC 5 warning. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" GCC 5 gave an incompatible pointer type warning for pkt_blocks when it's passed to _mm512_mask_i64gather_epi64(). Follow the same pattern used for tbl_blocks where the 'const uint64_t *' is cast to a 'const void *' when passed in to avx512_blocks_gather(). Fixes: 47a2a8f4138e ("dpif-netdev/dpcls-avx512: Enable 16 block processing.") Signed-off-by: Cian Ferriter Acked-by: Sunil Pai G --- v4: * Added Sunil's Fixes and Acked-by tags. v3: * Add this commit to the series. --- lib/dpif-netdev-lookup-avx512-gather.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/dpif-netdev-lookup-avx512-gather.c b/lib/dpif-netdev-lookup-avx512-gather.c index 7bc1e9e9a..b396772bc 100644 --- a/lib/dpif-netdev-lookup-avx512-gather.c +++ b/lib/dpif-netdev-lookup-avx512-gather.c @@ -155,7 +155,7 @@ netdev_rule_matches_key(const struct dpcls_rule *rule, static inline ALWAYS_INLINE __m512i avx512_blocks_gather(__m512i v_u0, __m512i v_u1, - const uint64_t *pkt_blocks, + const void *pkt_blocks, const void *tbl_blocks, const void *tbl_mf_masks, __mmask64 u1_bcast_msk, From patchwork Tue May 17 10:08:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ferriter, Cian" X-Patchwork-Id: 1632202 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=bl6fBAes; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4L2WzZ6sFHz9s0w for ; Tue, 17 May 2022 20:08:42 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 4B91E6119E; Tue, 17 May 2022 10:08:39 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CKgEOJxjlVbm; Tue, 17 May 2022 10:08:38 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id E463161190; Tue, 17 May 2022 10:08:36 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0D8E3C0084; Tue, 17 May 2022 10:08:36 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 324C4C0082 for ; Tue, 17 May 2022 10:08:33 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 0C86960EA7 for ; Tue, 17 May 2022 10:08:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ybAq6XMkDwjo for ; Tue, 17 May 2022 10:08:32 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by smtp3.osuosl.org (Postfix) with ESMTPS id 6495B60E61 for ; Tue, 17 May 2022 10:08:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652782112; x=1684318112; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=uA/PbzuJn4olOfAwxcYh2dA86JB+Z3QXrNArm2F7NCs=; b=bl6fBAesm3RY3GubB8jFFDjq/k26jW1qthQ8VGBAf6GZXGGg00+p47wQ zuuxP7t1liOMBBNmjNUiPA6wVBxgwTaO++3OtjjbyHCH58OGxiSoujaI2 npHvEAIOs6UuMhngRExAV7GBnS2lL1LT45bSeyFui+VZMO//8Ccn84rgv I81GWuQ+KuxS/XH/zjARtxUVHB372N0YPzairts/5XrR0DVPxQe3jh55V qLxO1483l1XNSzTvVSuZeH79y4hm+dDvpdTfRJpmLFjVUtFZGyGb0qIWg 5wVGGzbCNdjQaCXjxQYV1mXex4nymJcSa/DmDwDdziZGNetBsGbCeaRML w==; X-IronPort-AV: E=McAfee;i="6400,9594,10349"; a="258696505" X-IronPort-AV: E=Sophos;i="5.91,232,1647327600"; d="scan'208";a="258696505" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2022 03:08:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,232,1647327600"; d="scan'208";a="699986273" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.111]) by orsmga004.jf.intel.com with ESMTP; 17 May 2022 03:08:30 -0700 From: Cian Ferriter To: ovs-dev@openvswitch.org Date: Tue, 17 May 2022 10:08:16 +0000 Message-Id: <20220517100818.15639-4-cian.ferriter@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220517100818.15639-1-cian.ferriter@intel.com> References: <20220429150426.2318121-1-cian.ferriter@intel.com> <20220517100818.15639-1-cian.ferriter@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH v5 3/5] dpif-netdev-extract: Remove unnecessary compiler targets. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" No instructions from the AVX512VL ISA are used. Compilation for AVX512F and AVX512 BW ISA are already enabled in lib/automake.mk for the dpif-netdev-lookup-avx512-gather.c file because it's part of the libopenvswitchavx512.la library. They don't need to be enabled at a function level. Remove these unnecessary function-level compiler target attributes. Signed-off-by: Cian Ferriter Acked-by: Sunil Pai G --- v5: * Remove an 'avx512bw' target since it's also unnecessary as per Sunil's suggestion. * Add Sunil's Acked-by tag. v4: * Remove the 'avx512f' target since it's also unnecessary. * Sunil acked the v3 version of this commit, but since it's changed, I'm not carrying the ack over. --- lib/dpif-netdev-extract-avx512.c | 5 ----- 1 file changed, 5 deletions(-) diff --git a/lib/dpif-netdev-extract-avx512.c b/lib/dpif-netdev-extract-avx512.c index 6b6fe07db..4a94dfcfd 100644 --- a/lib/dpif-netdev-extract-avx512.c +++ b/lib/dpif-netdev-extract-avx512.c @@ -52,7 +52,6 @@ /* AVX512-BW level permutex2var_epi8 emulation. */ static inline __m512i -__attribute__((target("avx512bw"))) _mm512_maskz_permutex2var_epi8_skx(__mmask64 k_mask, __m512i v_data_0, __m512i v_shuf_idxs, @@ -632,8 +631,6 @@ mfex_avx512_process(struct dp_packet_batch *packets, #define DECLARE_MFEX_FUNC(name, profile) \ uint32_t \ -__attribute__((__target__("avx512f"))) \ -__attribute__((__target__("avx512vl"))) \ __attribute__((__target__("avx512vbmi"))) \ mfex_avx512_vbmi_##name(struct dp_packet_batch *packets, \ struct netdev_flow_key *keys, uint32_t keys_size,\ @@ -645,8 +642,6 @@ mfex_avx512_vbmi_##name(struct dp_packet_batch *packets, \ } \ \ uint32_t \ -__attribute__((__target__("avx512f"))) \ -__attribute__((__target__("avx512vl"))) \ mfex_avx512_##name(struct dp_packet_batch *packets, \ struct netdev_flow_key *keys, uint32_t keys_size, \ odp_port_t in_port, struct dp_netdev_pmd_thread \ From patchwork Tue May 17 10:08:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ferriter, Cian" X-Patchwork-Id: 1632204 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=hzHLUt1o; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4L2Wzd5k4kz9s1l for ; Tue, 17 May 2022 20:08:45 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id A9673611C4; Tue, 17 May 2022 10:08:40 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9cwSb-W4ZRqf; Tue, 17 May 2022 10:08:39 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 8D9A661197; Tue, 17 May 2022 10:08:38 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 413ECC007B; Tue, 17 May 2022 10:08:37 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 67A7BC002D for ; Tue, 17 May 2022 10:08:34 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 556FE61158 for ; Tue, 17 May 2022 10:08:34 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RYtv-CNRCb4o for ; Tue, 17 May 2022 10:08:33 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by smtp3.osuosl.org (Postfix) with ESMTPS id C685960B92 for ; Tue, 17 May 2022 10:08:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652782113; x=1684318113; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=S5ruiAbYcj+EqjWlcvGkrO9IjPxNk2vRdEBjc0hGkR0=; b=hzHLUt1o4RoHEGCCvdlJKIRVnelxDuk1rSIevXmXiwMSLTwlk0X3IcXd Pa8JFqsZz/9/cPQ5z+gbXKSDSst851QFD7x3r7EWtuxKHd0uyq32vmCmd cUMtCsoe8lypkU7en2ZSdGPRSoWWqdKnLR+pRCIewJufyaV/WS7WS8zxJ iqJ9SZU42Yz5mTPad+V+okOBjr67hAjzC/JZjyfmmd37Y3HZfA3H+nlwI O2IpjutsbGpQIEYYN9pdt/9W3TiyKcEmnrdKgQRnIFC3FyYKQRpiJTMD8 PSZEqZAFzYyDhOTj8V2lxYaME2tWq13Fq3WGdQNYu5yW3vNDJiDOsaY5r A==; X-IronPort-AV: E=McAfee;i="6400,9594,10349"; a="258696507" X-IronPort-AV: E=Sophos;i="5.91,232,1647327600"; d="scan'208";a="258696507" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2022 03:08:33 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,232,1647327600"; d="scan'208";a="699986277" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.111]) by orsmga004.jf.intel.com with ESMTP; 17 May 2022 03:08:32 -0700 From: Cian Ferriter To: ovs-dev@openvswitch.org Date: Tue, 17 May 2022 10:08:17 +0000 Message-Id: <20220517100818.15639-5-cian.ferriter@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220517100818.15639-1-cian.ferriter@intel.com> References: <20220429150426.2318121-1-cian.ferriter@intel.com> <20220517100818.15639-1-cian.ferriter@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH v5 4/5] automake.mk: Remove -mavx512dq CFLAG from AVX512 library. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" No instructions from the AVX512DQ ISA are used anywhere in OVS. Remove this unnecessary CFLAG. Signed-off-by: Cian Ferriter Acked-by: Sunil Pai G --- v5: * Add Sunil's Acked-by tag. v4: * Add this commit to the series. --- lib/automake.mk | 1 - 1 file changed, 1 deletion(-) diff --git a/lib/automake.mk b/lib/automake.mk index a23cdc4ad..14347bac6 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -32,7 +32,6 @@ lib_libopenvswitch_la_LIBADD += lib/libopenvswitchavx512.la lib_libopenvswitchavx512_la_CFLAGS = \ -mavx512f \ -mavx512bw \ - -mavx512dq \ -mbmi \ -mbmi2 \ -fPIC \ From patchwork Tue May 17 10:08:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ferriter, Cian" X-Patchwork-Id: 1632203 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=OkQIpltV; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4L2Wzd2T3cz9s0w for ; Tue, 17 May 2022 20:08:45 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 8901641A2A; Tue, 17 May 2022 10:08:43 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ll66JlNkeVXh; Tue, 17 May 2022 10:08:41 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 3E9C841A2C; Tue, 17 May 2022 10:08:40 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id DA8CBC0083; Tue, 17 May 2022 10:08:39 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 05F02C002D for ; Tue, 17 May 2022 10:08:39 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 2A63B6119E for ; Tue, 17 May 2022 10:08:37 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p_zO6dMvPx2Z for ; Tue, 17 May 2022 10:08:35 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by smtp3.osuosl.org (Postfix) with ESMTPS id B69DD61196 for ; Tue, 17 May 2022 10:08:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652782115; x=1684318115; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=IPs1c/2tDdksC40IZ9ulJGEN1Hrok+gRNTakywo4AB0=; b=OkQIpltVANfDZ/oayVYuwAneDf7DEiwzta2/D4TVebt1NkXaiHdzO/bo 2l9bxIesVcidpfNx3L7p7cWy8EtAIiJz8Mxt20AvfZITyeTCdwLJqDfCu L9Fdixe+gFZnYgy9ap6MYmjumE+aE6jcwtEh1v/QssUJaitHKRI9ZOUzQ Ax/TlCrweFybDxBsYd1aFDebvJzPe1xW1ejpvHJBVzQQMcMD0Zo69ERVm ubR6bCe6/hMVH97sck4UOt6ZmyvCxc/VD+rHPw0Ul2eJxrlwjuXB6PW3J ii+TEWvT+ftpTngTE+3sKWHOk73uVjJ4sw5uyDA1o+uP4QIi9Ki/sxX71 Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10349"; a="258696520" X-IronPort-AV: E=Sophos;i="5.91,232,1647327600"; d="scan'208";a="258696520" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2022 03:08:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,232,1647327600"; d="scan'208";a="699986284" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.111]) by orsmga004.jf.intel.com with ESMTP; 17 May 2022 03:08:33 -0700 From: Cian Ferriter To: ovs-dev@openvswitch.org Date: Tue, 17 May 2022 10:08:18 +0000 Message-Id: <20220517100818.15639-6-cian.ferriter@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220517100818.15639-1-cian.ferriter@intel.com> References: <20220429150426.2318121-1-cian.ferriter@intel.com> <20220517100818.15639-1-cian.ferriter@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH v5 5/5] acinclude: Add seperate checks for AVX512 ISA. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Checking for each of the required AVX512 ISA separately will allow the compiler to generate some AVX512 code where there is some support in the compiler rather than only generating all AVX512 code when all of it is supported or no AVX512 code at all. For example, in GCC 4.9 where there is just support for AVX512F, this patch will allow building the AVX512 DPIF. Another example, in GCC 5 and 6, most AVX512 code can be generated, just without AVX512VPOPCNTDQ support. Signed-off-by: Cian Ferriter Acked-by: Sunil Pai G --- v5: * Create a selector function for the permutexvar implementations based on Sunil's feedback on the v4. This hides the complexity of compile time and run time selection of permutexvar implementations. * Add a comment explaining why VPOPCNTDQ_TARGET is defined and used. v4: * Combine the 3 commits which added checks for AVX512 ISA into this single commit since the first 2 commits were only useful and active when the 3rd commit was applied. This also takes care of Sunil's comment about explaining that the first 2 commits are precursors. * Don't check for AVX512DQ availability in the compiler. This ISA isn't used in OVS. * Put all AVX512 ISA checks in the OVS_CHECK_AVX512 macro as per Sunil's feedback. * Define a function in acinclude.m4, (OVS_CONDITIONAL_CC_OPTION_DEFINE), to help with checking for AVX512 ISA support in the compiler. * Remove the '__AVX512VPOPCNTDQ__' check. Use the HAVE_AVX512* pattern consistently with all AVX512 ISA checks instead. Fixup the comment explaining the _mm512_popcnt_epi64_wrapper() function to reflect this. v3: * Preserve the order of the mfex impl list. v2 changed this order. We want the order to be preserved because VBMI functions should be chosen by the mfex study impl where possible. v2: * Don't register vbmi specialized mfex impls unless VBMI is actually available. * This required some re-ordering of the mfex impl lists. --- acinclude.m4 | 26 +++++++---- lib/automake.mk | 14 ++++-- lib/dpif-netdev-extract-avx512.c | 64 ++++++++++++++++++-------- lib/dpif-netdev-lookup-avx512-gather.c | 33 +++++++++---- lib/dpif-netdev-lookup.c | 3 +- lib/dpif-netdev-private-extract.c | 18 ++++---- lib/dpif-netdev-private-extract.h | 19 +++++++- 7 files changed, 127 insertions(+), 50 deletions(-) diff --git a/acinclude.m4 b/acinclude.m4 index 61e88105f..7b2889a40 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -73,16 +73,13 @@ AC_DEFUN([OVS_CHECK_DPIF_AVX512_DEFAULT], [ dnl OVS_CHECK_AVX512 dnl -dnl Checks if compiler and binutils supports AVX512. +dnl Checks if compiler and binutils supports various AVX512 ISA. AC_DEFUN([OVS_CHECK_AVX512], [ OVS_CHECK_BINUTILS_AVX512 - OVS_CHECK_CC_OPTION( - [-mavx512f -mavx512vpopcntdq], [ovs_have_cc_mavx512f=yes], [ovs_have_cc_mavx512f=no]) - AM_CONDITIONAL([HAVE_AVX512F], [test $ovs_have_cc_mavx512f = yes]) - if test "$ovs_have_cc_mavx512f" = yes; then - AC_DEFINE([HAVE_AVX512F], [1], - [Define to 1 if compiler supports AVX512.]) - fi + OVS_CONDITIONAL_CC_OPTION_DEFINE([-mavx512f], [HAVE_AVX512F]) + OVS_CONDITIONAL_CC_OPTION_DEFINE([-mavx512bw], [HAVE_AVX512BW]) + OVS_CONDITIONAL_CC_OPTION_DEFINE([-mavx512vbmi], [HAVE_AVX512VBMI]) + OVS_CONDITIONAL_CC_OPTION_DEFINE([-mavx512vpopcntdq], [HAVE_AVX512VPOPCNTDQ]) ]) dnl OVS_ENABLE_WERROR @@ -1360,6 +1357,19 @@ AC_DEFUN([OVS_CONDITIONAL_CC_OPTION], AM_CONDITIONAL([$2], [test $ovs_have_cc_option = yes])]) dnl ---------------------------------------------------------------------- +dnl OVS_CONDITIONAL_CC_OPTION_DEFINE([OPTION], [CONDITIONAL]) +dnl Check whether the given C compiler OPTION is accepted. +dnl If so, enable the given Automake CONDITIONAL and define it. +dnl Example: OVS_CONDITIONAL_CC_OPTION_DEFINE([-mavx512f], [HAVE_AVX512F]) +AC_DEFUN([OVS_CONDITIONAL_CC_OPTION_DEFINE], + [OVS_CHECK_CC_OPTION( + [$1], [ovs_have_cc_option=yes], [ovs_have_cc_option=no]) + AM_CONDITIONAL([$2], [test $ovs_have_cc_option = yes]) + if test "$ovs_have_cc_option" = yes; then + AC_DEFINE([$2], [1], + [Define to 1 if compiler supports the '$1' option.]) + fi]) + dnl Check for too-old XenServer. AC_DEFUN([OVS_CHECK_XENSERVER_VERSION], [AC_CACHE_CHECK([XenServer release], [ovs_cv_xsversion], diff --git a/lib/automake.mk b/lib/automake.mk index 14347bac6..cb50578eb 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -31,7 +31,6 @@ lib_LTLIBRARIES += lib/libopenvswitchavx512.la lib_libopenvswitch_la_LIBADD += lib/libopenvswitchavx512.la lib_libopenvswitchavx512_la_CFLAGS = \ -mavx512f \ - -mavx512bw \ -mbmi \ -mbmi2 \ -fPIC \ @@ -39,13 +38,18 @@ lib_libopenvswitchavx512_la_CFLAGS = \ lib_libopenvswitchavx512_la_SOURCES = \ lib/cpu.c \ lib/cpu.h \ - lib/dpif-netdev-lookup-avx512-gather.c \ - lib/dpif-netdev-extract-avx512.c \ lib/dpif-netdev-avx512.c +if HAVE_AVX512BW +lib_libopenvswitchavx512_la_CFLAGS += \ + -mavx512bw +lib_libopenvswitchavx512_la_SOURCES += \ + lib/dpif-netdev-extract-avx512.c \ + lib/dpif-netdev-lookup-avx512-gather.c +endif # HAVE_AVX512BW lib_libopenvswitchavx512_la_LDFLAGS = \ -static -endif -endif +endif # HAVE_LD_AVX512_GOOD +endif # HAVE_AVX512F # Build core vswitch libraries as before lib_libopenvswitch_la_SOURCES = \ diff --git a/lib/dpif-netdev-extract-avx512.c b/lib/dpif-netdev-extract-avx512.c index 4a94dfcfd..12271be17 100644 --- a/lib/dpif-netdev-extract-avx512.c +++ b/lib/dpif-netdev-extract-avx512.c @@ -108,13 +108,42 @@ _mm512_maskz_permutex2var_epi8_skx(__mmask64 k_mask, return v_result_kmskd; } -/* Wrapper function required to enable ISA. */ +/* Wrapper function to enable VBMI ISA required by the + * _mm512_maskz_permutexvar_epi8 intrinsic. */ +#if HAVE_AVX512VBMI static inline __m512i __attribute__((__target__("avx512vbmi"))) _mm512_maskz_permutexvar_epi8_wrap(__mmask64 kmask, __m512i idx, __m512i a) { return _mm512_maskz_permutexvar_epi8(kmask, idx, a); } +#endif + +static inline __m512i +_mm512_maskz_permutexvar_epi8_selector(__mmask64 k_shuf, __m512i v_shuf, + __m512i v_pkt0, + const uint32_t use_vbmi OVS_UNUSED) +{ + /* Permute the packet layout into miniflow blocks shape. */ + __m512i v512_zeros = _mm512_setzero_si512(); + __m512i v_blk0; +#if HAVE_AVX512VBMI + if (__builtin_constant_p(use_vbmi) && use_vbmi) { + /* As different AVX512 ISA levels have different implementations, + * this specializes on the use_vbmi attribute passed in. + */ + v_blk0 = _mm512_maskz_permutexvar_epi8_wrap(k_shuf, v_shuf, v_pkt0); + + } else { + v_blk0 = _mm512_maskz_permutex2var_epi8_skx(k_shuf, v_pkt0, v_shuf, + v512_zeros); + } +#else + v_blk0 = _mm512_maskz_permutex2var_epi8_skx(k_shuf, v_pkt0, v_shuf, + v512_zeros); +#endif + return v_blk0; +} /* This file contains optimized implementations of miniflow_extract() @@ -481,7 +510,7 @@ mfex_avx512_process(struct dp_packet_batch *packets, odp_port_t in_port, void *pmd_handle OVS_UNUSED, const enum MFEX_PROFILES profile_id, - const uint32_t use_vbmi) + const uint32_t use_vbmi OVS_UNUSED) { uint32_t hitmask = 0; struct dp_packet *packet; @@ -538,19 +567,9 @@ mfex_avx512_process(struct dp_packet_batch *packets, _mm_storeu_si128((void *) bits, v_bits); _mm_storeu_si128((void *) blocks, v_blocks01); - /* Permute the packet layout into miniflow blocks shape. - * As different AVX512 ISA levels have different implementations, - * this specializes on the "use_vbmi" attribute passed in. - */ - __m512i v512_zeros = _mm512_setzero_si512(); - __m512i v_blk0; - if (__builtin_constant_p(use_vbmi) && use_vbmi) { - v_blk0 = _mm512_maskz_permutexvar_epi8_wrap(k_shuf, v_shuf, - v_pkt0); - } else { - v_blk0 = _mm512_maskz_permutex2var_epi8_skx(k_shuf, v_pkt0, - v_shuf, v512_zeros); - } + __m512i v_blk0 = _mm512_maskz_permutexvar_epi8_selector(k_shuf, v_shuf, + v_pkt0, + use_vbmi); __m512i v_blk0_strip = _mm512_and_si512(v_blk0, v_strp); _mm512_storeu_si512(&blocks[2], v_blk0_strip); @@ -629,7 +648,8 @@ mfex_avx512_process(struct dp_packet_batch *packets, } -#define DECLARE_MFEX_FUNC(name, profile) \ +#if HAVE_AVX512VBMI +#define VBMI_MFEX_FUNC(name, profile) \ uint32_t \ __attribute__((__target__("avx512vbmi"))) \ mfex_avx512_vbmi_##name(struct dp_packet_batch *packets, \ @@ -639,8 +659,12 @@ mfex_avx512_vbmi_##name(struct dp_packet_batch *packets, \ { \ return mfex_avx512_process(packets, keys, keys_size, in_port, \ pmd_handle, profile, 1); \ -} \ - \ +} +#else +#define VBMI_MFEX_FUNC(name, profile) +#endif + +#define BASIC_MFEX_FUNC(name, profile) \ uint32_t \ mfex_avx512_##name(struct dp_packet_batch *packets, \ struct netdev_flow_key *keys, uint32_t keys_size, \ @@ -651,6 +675,10 @@ mfex_avx512_##name(struct dp_packet_batch *packets, \ pmd_handle, profile, 0); \ } +#define DECLARE_MFEX_FUNC(name, profile) \ +VBMI_MFEX_FUNC(name, profile) \ +BASIC_MFEX_FUNC(name, profile) \ + /* Each profile gets a single declare here, which specializes the function * as required. */ diff --git a/lib/dpif-netdev-lookup-avx512-gather.c b/lib/dpif-netdev-lookup-avx512-gather.c index b396772bc..1e86be207 100644 --- a/lib/dpif-netdev-lookup-avx512-gather.c +++ b/lib/dpif-netdev-lookup-avx512-gather.c @@ -78,22 +78,26 @@ _mm512_popcnt_epi64_manual(__m512i v_in) return _mm512_sad_epu8(v_u8_pop, _mm512_setzero_si512()); } -/* Wrapper function required to enable ISA. First enable the ISA via the - * attribute target for this function, then check if the compiler actually - * #defines the ISA itself. If the ISA is not #define-ed by the compiler it - * indicates the compiler is too old or is not capable of compiling the - * requested ISA level, so fallback to the integer manual implementation. +/* Wrapper function required to enable ISA. First check if the compiler + * supports the ISA itself. If the ISA is supported, enable it via the + * attribute target. If the ISA is not supported by the compiler it indicates + * the compiler is too old or is not capable of compiling the requested ISA + * level, so fallback to the integer manual implementation. */ +#if HAVE_AVX512VPOPCNTDQ static inline __m512i __attribute__((__target__("avx512vpopcntdq"))) _mm512_popcnt_epi64_wrapper(__m512i v_in) { -#ifdef __AVX512VPOPCNTDQ__ return _mm512_popcnt_epi64(v_in); +} #else +static inline __m512i +_mm512_popcnt_epi64_wrapper(__m512i v_in) +{ return _mm512_popcnt_epi64_manual(v_in); -#endif } +#endif static inline uint64_t netdev_rule_matches_key(const struct dpcls_rule *rule, @@ -334,6 +338,19 @@ avx512_lookup_impl(struct dpcls_subtable *subtable, return found_map; } +/* Use a different pattern to conditionally use the VPOPCNTDQ target attribute + * here. + * The usual pattern using a '#if HAVE_AVX512VPOPCNTDQ' type check won't work + * inside a macro. + * Define VPOPCNTDQ_TARGET which will either be the "avx512vpopcntdq" target + * attribute or nothing depending on AVX512VPOPCNTDQ support in the compiler. + */ +#if HAVE_AVX512VPOPCNTDQ +#define VPOPCNTDQ_TARGET __attribute__((__target__("avx512vpopcntdq"))) +#else +#define VPOPCNTDQ_TARGET +#endif + /* Expand out specialized functions with U0 and U1 bit attributes. As the * AVX512 vpopcnt instruction is not supported on all AVX512 capable CPUs, * create two functions for each miniflow signature. This allows the runtime @@ -351,7 +368,7 @@ avx512_lookup_impl(struct dpcls_subtable *subtable, U0, U1, use_vpop); \ } \ \ - static uint32_t __attribute__((__target__("avx512vpopcntdq"))) \ + static uint32_t VPOPCNTDQ_TARGET \ dpcls_avx512_gather_mf_##U0##_##U1##_vpop(struct dpcls_subtable *subtable,\ uint32_t keys_map, \ const struct netdev_flow_key *keys[], \ diff --git a/lib/dpif-netdev-lookup.c b/lib/dpif-netdev-lookup.c index bd0a99abe..c6aab6aed 100644 --- a/lib/dpif-netdev-lookup.c +++ b/lib/dpif-netdev-lookup.c @@ -43,7 +43,8 @@ static struct dpcls_subtable_lookup_info_t subtable_lookups[] = { .probe = dpcls_subtable_generic_probe, .name = "generic", }, -#if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && __SSE4_2__) +#if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && HAVE_AVX512BW \ + && __SSE4_2__) /* Only available on x86_64 bit builds with SSE 4.2 used for OVS core. */ { .prio = 0, .probe = dpcls_subtable_avx512_gather_probe, diff --git a/lib/dpif-netdev-private-extract.c b/lib/dpif-netdev-private-extract.c index b7f094dac..9ce4e0909 100644 --- a/lib/dpif-netdev-private-extract.c +++ b/lib/dpif-netdev-private-extract.c @@ -54,42 +54,44 @@ static struct dpif_miniflow_extract_impl mfex_impls[] = { .name = "study", }, /* Compile in implementations only if the compiler ISA checks pass. */ -#if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && __SSE4_2__) +#if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && HAVE_AVX512BW \ + && __SSE4_2__) +#if HAVE_AVX512VBMI [MFEX_IMPL_VBMI_IPv4_UDP] = { .probe = mfex_avx512_vbmi_probe, .extract_func = mfex_avx512_vbmi_ip_udp, .name = "avx512_vbmi_ipv4_udp", }, - +#endif [MFEX_IMPL_IPv4_UDP] = { .probe = mfex_avx512_probe, .extract_func = mfex_avx512_ip_udp, .name = "avx512_ipv4_udp", }, - +#if HAVE_AVX512VBMI [MFEX_IMPL_VBMI_IPv4_TCP] = { .probe = mfex_avx512_vbmi_probe, .extract_func = mfex_avx512_vbmi_ip_tcp, .name = "avx512_vbmi_ipv4_tcp", }, - +#endif [MFEX_IMPL_IPv4_TCP] = { .probe = mfex_avx512_probe, .extract_func = mfex_avx512_ip_tcp, .name = "avx512_ipv4_tcp", }, - +#if HAVE_AVX512VBMI [MFEX_IMPL_VBMI_DOT1Q_IPv4_UDP] = { .probe = mfex_avx512_vbmi_probe, .extract_func = mfex_avx512_vbmi_dot1q_ip_udp, .name = "avx512_vbmi_dot1q_ipv4_udp", }, - +#endif [MFEX_IMPL_DOT1Q_IPv4_UDP] = { .probe = mfex_avx512_probe, .extract_func = mfex_avx512_dot1q_ip_udp, .name = "avx512_dot1q_ipv4_udp", }, - +#if HAVE_AVX512VBMI [MFEX_IMPL_VBMI_DOT1Q_IPv4_TCP] = { .probe = mfex_avx512_vbmi_probe, .extract_func = mfex_avx512_vbmi_dot1q_ip_tcp, .name = "avx512_vbmi_dot1q_ipv4_tcp", }, - +#endif [MFEX_IMPL_DOT1Q_IPv4_TCP] = { .probe = mfex_avx512_probe, .extract_func = mfex_avx512_dot1q_ip_tcp, diff --git a/lib/dpif-netdev-private-extract.h b/lib/dpif-netdev-private-extract.h index ae5c161b4..092126106 100644 --- a/lib/dpif-netdev-private-extract.h +++ b/lib/dpif-netdev-private-extract.h @@ -81,14 +81,23 @@ enum dpif_miniflow_extract_impl_idx { MFEX_IMPL_AUTOVALIDATOR, MFEX_IMPL_SCALAR, MFEX_IMPL_STUDY, -#if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && __SSE4_2__) +#if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && HAVE_AVX512BW \ + && __SSE4_2__) +#if HAVE_AVX512VBMI MFEX_IMPL_VBMI_IPv4_UDP, +#endif MFEX_IMPL_IPv4_UDP, +#if HAVE_AVX512VBMI MFEX_IMPL_VBMI_IPv4_TCP, +#endif MFEX_IMPL_IPv4_TCP, +#if HAVE_AVX512VBMI MFEX_IMPL_VBMI_DOT1Q_IPv4_UDP, +#endif MFEX_IMPL_DOT1Q_IPv4_UDP, +#if HAVE_AVX512VBMI MFEX_IMPL_VBMI_DOT1Q_IPv4_TCP, +#endif MFEX_IMPL_DOT1Q_IPv4_TCP, #endif MFEX_IMPL_MAX @@ -99,9 +108,15 @@ extern struct ovs_mutex dp_netdev_mutex; /* Define a index which points to the first traffic optimized MFEX * option from the enum list else holds max value. */ -#if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && __SSE4_2__) +#if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && HAVE_AVX512BW \ + && __SSE4_2__) +#if HAVE_AVX512VBMI #define MFEX_IMPL_START_IDX MFEX_IMPL_VBMI_IPv4_UDP +#else +#define MFEX_IMPL_START_IDX MFEX_IMPL_IPv4_UDP +#endif + #else #define MFEX_IMPL_START_IDX MFEX_IMPL_MAX