From patchwork Tue Nov 23 15:02:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 1558599 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Vf4NwlL3; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Hz6nd0wgTz9sRN for ; Wed, 24 Nov 2021 02:02:45 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 7DCD960796; Tue, 23 Nov 2021 15:02:40 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qIcGO48MhXf5; Tue, 23 Nov 2021 15:02:39 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 0D2B4607D5; Tue, 23 Nov 2021 15:02:37 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 40C16C0038; Tue, 23 Nov 2021 15:02:36 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6F0C0C001E for ; Tue, 23 Nov 2021 15:02:33 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 67AC94046D for ; Tue, 23 Nov 2021 15:02:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CxU3pbRAFKbQ for ; Tue, 23 Nov 2021 15:02:32 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 0EF1F4045D for ; Tue, 23 Nov 2021 15:02:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637679750; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=iGcXjDLqVjVrYglVC211VHv3rC8dFglbuYk3i9o4UU0=; b=Vf4NwlL3f01EPG6WOTTGWD8WbcoOg5+nEKvgsLYQ5k82FMouUQvvbSSjxxQrM6jyiwAehz bB7qncr04FKlaEU1rWieCrmLB3kR0KtvCYqMX6WsgxL5DSyE14BQj/Ay7jB5gxoK+eXuA1 Tins5lEhlGaaBhjBN7/IEGeixaO6H8c= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-402-e84YuMWrNjGnzd90V_ZRIA-1; Tue, 23 Nov 2021 10:02:26 -0500 X-MC-Unique: e84YuMWrNjGnzd90V_ZRIA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B25761966320; Tue, 23 Nov 2021 15:02:25 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.192.8]) by smtp.corp.redhat.com (Postfix) with ESMTP id 44E74652AC; Tue, 23 Nov 2021 15:02:22 +0000 (UTC) From: David Marchand To: dev@openvswitch.org Date: Tue, 23 Nov 2021 16:02:00 +0100 Message-Id: <20211123150200.11546-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: kumar.amber@intel.com, i.maximets@ovn.org Subject: [ovs-dev] [PATCH v2] dpif-netdev: Call cpuid for x86 isa availability. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" DPIF AVX512 optimisations currently rely on DPDK availability while they can be used without DPDK. Besides, checking for availability of some isa only has to be done once and won't change while a OVS process runs. Resolve isa availability in constructors by using a simplified query based on cpuid API that comes from the compiler. Note: this also fixes the check on BMI2 availability: DPDK had a bug for this isa, see https://git.dpdk.org/dpdk/commit/?id=aae3037ab1e0. Suggested-by: Ilya Maximets Signed-off-by: David Marchand --- Chances since v1: - fixed minor checkpatch issue, --- lib/automake.mk | 2 + lib/cpu.c | 68 ++++++++++++++++++++++++++ lib/cpu.h | 34 +++++++++++++ lib/dpdk-stub.c | 9 ---- lib/dpdk.c | 52 -------------------- lib/dpdk.h | 1 - lib/dpif-netdev-avx512.c | 5 +- lib/dpif-netdev-extract-avx512.c | 14 +++--- lib/dpif-netdev-lookup-avx512-gather.c | 7 +-- 9 files changed, 118 insertions(+), 74 deletions(-) create mode 100644 lib/cpu.c create mode 100644 lib/cpu.h diff --git a/lib/automake.mk b/lib/automake.mk index 46f869a336..041ed96259 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -87,6 +87,8 @@ lib_libopenvswitch_la_SOURCES = \ lib/conntrack.h \ lib/coverage.c \ lib/coverage.h \ + lib/cpu.c \ + lib/cpu.h \ lib/crc32c.c \ lib/crc32c.h \ lib/csum.c \ diff --git a/lib/cpu.c b/lib/cpu.c new file mode 100644 index 0000000000..ea1934d3ca --- /dev/null +++ b/lib/cpu.c @@ -0,0 +1,68 @@ +/* + * Copyright (c) 2021, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include "cpu.h" +#include "openvswitch/compiler.h" + +#ifdef __x86_64__ +#include +#include + +enum x86_reg { + EAX, + EBX, + ECX, + EDX, +}; +#define X86_LEAF_MASK 0x80000000 +#define X86_EXT_FEATURES_LEAF 0x00000007 +static bool x86_has_isa(uint32_t leaf, enum x86_reg reg, uint32_t bit) +{ + uint32_t maxleaf = __get_cpuid_max(leaf & X86_LEAF_MASK, NULL); + uint32_t regs[4]; + + if (maxleaf < leaf) { + return false; + } + __cpuid_count(leaf, 0, regs[EAX], regs[EBX], regs[ECX], regs[EDX]); + return (regs[reg] & ((uint32_t) 1 << bit)) != 0; +} + +static bool x86_isa[OVS_CPU_ISA_X86_LAST - OVS_CPU_ISA_X86_FIRST + 1]; +#define X86_ISA(leaf, reg, bit, name) \ +OVS_CONSTRUCTOR(cpu_isa_ ## name) { \ + x86_isa[name - OVS_CPU_ISA_X86_FIRST] = x86_has_isa(leaf, reg, bit); \ +} +X86_ISA(X86_EXT_FEATURES_LEAF, EBX, 8, OVS_CPU_ISA_X86_BMI2) +X86_ISA(X86_EXT_FEATURES_LEAF, EBX, 16, OVS_CPU_ISA_X86_AVX512F) +X86_ISA(X86_EXT_FEATURES_LEAF, EBX, 30, OVS_CPU_ISA_X86_AVX512BW) +X86_ISA(X86_EXT_FEATURES_LEAF, ECX, 1, OVS_CPU_ISA_X86_AVX512VBMI) +X86_ISA(X86_EXT_FEATURES_LEAF, ECX, 14, OVS_CPU_ISA_X86_VPOPCNTDQ) +#endif + +bool +cpu_has_isa(enum ovs_cpu_isa isa OVS_UNUSED) +{ +#ifdef __x86_64__ + if (isa >= OVS_CPU_ISA_X86_FIRST && + isa <= OVS_CPU_ISA_X86_LAST) { + return x86_isa[isa - OVS_CPU_ISA_X86_FIRST]; + } +#endif + return false; +} diff --git a/lib/cpu.h b/lib/cpu.h new file mode 100644 index 0000000000..92897bb71c --- /dev/null +++ b/lib/cpu.h @@ -0,0 +1,34 @@ +/* + * Copyright (c) 2021, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef CPU_H +#define CPU_H 1 + +#include + +enum ovs_cpu_isa { + OVS_CPU_ISA_X86_FIRST, + OVS_CPU_ISA_X86_BMI2 = OVS_CPU_ISA_X86_FIRST, + OVS_CPU_ISA_X86_AVX512F, + OVS_CPU_ISA_X86_AVX512BW, + OVS_CPU_ISA_X86_AVX512VBMI, + OVS_CPU_ISA_X86_VPOPCNTDQ, + OVS_CPU_ISA_X86_LAST = OVS_CPU_ISA_X86_VPOPCNTDQ, +}; + +bool cpu_has_isa(enum ovs_cpu_isa); + +#endif /* CPU_H */ diff --git a/lib/dpdk-stub.c b/lib/dpdk-stub.c index fe24f9abdf..c332c217cb 100644 --- a/lib/dpdk-stub.c +++ b/lib/dpdk-stub.c @@ -79,15 +79,6 @@ print_dpdk_version(void) { } -bool -dpdk_get_cpu_has_isa(const char *arch OVS_UNUSED, - const char *feature OVS_UNUSED) -{ - VLOG_DBG_ONCE("DPDK not supported in this version of Open vSwitch, " - "cannot use CPU flag based optimizations"); - return false; -} - void dpdk_status(const struct ovsrec_open_vswitch *cfg) { diff --git a/lib/dpdk.c b/lib/dpdk.c index b2ef31cd20..fe201bf29c 100644 --- a/lib/dpdk.c +++ b/lib/dpdk.c @@ -585,58 +585,6 @@ print_dpdk_version(void) puts(rte_version()); } -/* Avoid calling rte_cpu_get_flag_enabled() excessively, by caching the - * result of the call for each CPU flag in a static variable. To avoid - * allocating large numbers of static variables, use a uint8 as a bitfield. - * Note the macro must only return if the ISA check is done and available. - */ -#define ISA_CHECK_DONE_BIT (1 << 0) -#define ISA_AVAILABLE_BIT (1 << 1) - -#define CHECK_CPU_FEATURE(feature, name_str, RTE_CPUFLAG) \ - do { \ - if (strncmp(feature, name_str, strlen(name_str)) == 0) { \ - static uint8_t isa_check_##RTE_CPUFLAG; \ - int check = isa_check_##RTE_CPUFLAG & ISA_CHECK_DONE_BIT; \ - if (OVS_UNLIKELY(!check)) { \ - int has_isa = rte_cpu_get_flag_enabled(RTE_CPUFLAG); \ - VLOG_DBG("CPU flag %s, available %s\n", \ - name_str, has_isa ? "yes" : "no"); \ - isa_check_##RTE_CPUFLAG = ISA_CHECK_DONE_BIT; \ - if (has_isa) { \ - isa_check_##RTE_CPUFLAG |= ISA_AVAILABLE_BIT; \ - } \ - } \ - if (isa_check_##RTE_CPUFLAG & ISA_AVAILABLE_BIT) { \ - return true; \ - } else { \ - return false; \ - } \ - } \ - } while (0) - -bool -dpdk_get_cpu_has_isa(const char *arch, const char *feature) -{ - /* Ensure Arch is x86_64. */ - if (strncmp(arch, "x86_64", 6) != 0) { - return false; - } - -#if __x86_64__ - /* CPU flags only defined for the architecture that support it. */ - CHECK_CPU_FEATURE(feature, "avx512f", RTE_CPUFLAG_AVX512F); - CHECK_CPU_FEATURE(feature, "avx512bw", RTE_CPUFLAG_AVX512BW); - CHECK_CPU_FEATURE(feature, "avx512vbmi", RTE_CPUFLAG_AVX512VBMI); - CHECK_CPU_FEATURE(feature, "avx512vpopcntdq", RTE_CPUFLAG_AVX512VPOPCNTDQ); - CHECK_CPU_FEATURE(feature, "bmi2", RTE_CPUFLAG_BMI2); -#endif - - VLOG_WARN("Unknown CPU arch,feature: %s,%s. Returning not supported.\n", - arch, feature); - return false; -} - void dpdk_status(const struct ovsrec_open_vswitch *cfg) { diff --git a/lib/dpdk.h b/lib/dpdk.h index 445a51d065..2eb1aedbb0 100644 --- a/lib/dpdk.h +++ b/lib/dpdk.h @@ -44,6 +44,5 @@ bool dpdk_per_port_memory(void); bool dpdk_available(void); void print_dpdk_version(void); void dpdk_status(const struct ovsrec_open_vswitch *); -bool dpdk_get_cpu_has_isa(const char *arch, const char *feature); #endif /* dpdk.h */ diff --git a/lib/dpif-netdev-avx512.c b/lib/dpif-netdev-avx512.c index 544d36903e..3f1e623f36 100644 --- a/lib/dpif-netdev-avx512.c +++ b/lib/dpif-netdev-avx512.c @@ -20,6 +20,7 @@ #include +#include "cpu.h" #include "dpif-netdev.h" #include "dpif-netdev-perf.h" #include "dpif-netdev-private.h" @@ -61,8 +62,8 @@ struct dpif_userdata { int32_t dp_netdev_input_outer_avx512_probe(void) { - bool avx512f_available = dpdk_get_cpu_has_isa("x86_64", "avx512f"); - bool bmi2_available = dpdk_get_cpu_has_isa("x86_64", "bmi2"); + bool avx512f_available = cpu_has_isa(OVS_CPU_ISA_X86_AVX512F); + bool bmi2_available = cpu_has_isa(OVS_CPU_ISA_X86_BMI2); if (!avx512f_available || !bmi2_available) { return -ENOTSUP; diff --git a/lib/dpif-netdev-extract-avx512.c b/lib/dpif-netdev-extract-avx512.c index ec64419e38..feafba0c32 100644 --- a/lib/dpif-netdev-extract-avx512.c +++ b/lib/dpif-netdev-extract-avx512.c @@ -42,8 +42,8 @@ #include #include +#include "cpu.h" #include "flow.h" -#include "dpdk.h" #include "dpif-netdev-private-dpcls.h" #include "dpif-netdev-private-extract.h" @@ -589,21 +589,21 @@ DECLARE_MFEX_FUNC(dot1q_ip_tcp, PROFILE_ETH_VLAN_IPV4_TCP) static int32_t avx512_isa_probe(uint32_t needs_vbmi) { - static const char *isa_required[] = { - "avx512f", - "avx512bw", - "bmi2", + static enum ovs_cpu_isa isa_required[] = { + OVS_CPU_ISA_X86_AVX512F, + OVS_CPU_ISA_X86_AVX512BW, + OVS_CPU_ISA_X86_BMI2, }; int32_t ret = 0; for (uint32_t i = 0; i < ARRAY_SIZE(isa_required); i++) { - if (!dpdk_get_cpu_has_isa("x86_64", isa_required[i])) { + if (!cpu_has_isa(isa_required[i])) { ret = -ENOTSUP; } } if (needs_vbmi) { - if (!dpdk_get_cpu_has_isa("x86_64", "avx512vbmi")) { + if (!cpu_has_isa(OVS_CPU_ISA_X86_AVX512VBMI)) { ret = -ENOTSUP; } } diff --git a/lib/dpif-netdev-lookup-avx512-gather.c b/lib/dpif-netdev-lookup-avx512-gather.c index 072831e96a..914c9e2ede 100644 --- a/lib/dpif-netdev-lookup-avx512-gather.c +++ b/lib/dpif-netdev-lookup-avx512-gather.c @@ -19,6 +19,7 @@ #include +#include "cpu.h" #include "dpif-netdev.h" #include "dpif-netdev-lookup.h" #include "cmap.h" @@ -398,13 +399,13 @@ dpcls_subtable_avx512_gather_probe(uint32_t u0_bits, uint32_t u1_bits) { dpcls_subtable_lookup_func f = NULL; - int avx512f_available = dpdk_get_cpu_has_isa("x86_64", "avx512f"); - int bmi2_available = dpdk_get_cpu_has_isa("x86_64", "bmi2"); + int avx512f_available = cpu_has_isa(OVS_CPU_ISA_X86_AVX512F); + int bmi2_available = cpu_has_isa(OVS_CPU_ISA_X86_BMI2); if (!avx512f_available || !bmi2_available) { return NULL; } - int use_vpop = dpdk_get_cpu_has_isa("x86_64", "avx512vpopcntdq"); + int use_vpop = cpu_has_isa(OVS_CPU_ISA_X86_VPOPCNTDQ); CHECK_LOOKUP_FUNCTION(9, 4, use_vpop); CHECK_LOOKUP_FUNCTION(9, 1, use_vpop);