From patchwork Fri Jun 17 03:50:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noah Goldstein X-Patchwork-Id: 1644663 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=iTeXKsbN; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LPQ7n4CLMz9s75 for ; Fri, 17 Jun 2022 13:51:17 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 203ED386C593 for ; Fri, 17 Jun 2022 03:51:13 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 203ED386C593 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1655437873; bh=bAoWKItNQChK2iocsG7/nssyra5ssh9ng211wnDsZxY=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=iTeXKsbN6dN9onfA1WmW+bbs6JusWjHbRDJoffaw4dpevDCjFhCmlPoI63YUdOQHD OTUUXQpF5NrthNzf0rcIrhiTkG1r0IUuovplTFg4EAebyujJQy66dPGNh1m9cUgDtI h5m28rxWN2pnM/tTFATOlW7sgJz2Iffzm34UHjxw= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x434.google.com (mail-pf1-x434.google.com [IPv6:2607:f8b0:4864:20::434]) by sourceware.org (Postfix) with ESMTPS id 5F5BA386C590 for ; Fri, 17 Jun 2022 03:50:56 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 5F5BA386C590 Received: by mail-pf1-x434.google.com with SMTP id bo5so3138602pfb.4 for ; Thu, 16 Jun 2022 20:50:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=bAoWKItNQChK2iocsG7/nssyra5ssh9ng211wnDsZxY=; b=2mVFVTdT0pkJUFJDKW0jnkLbn+wauORF6rAKvMDYLJzHEKbzclFVCKwssuS96pvRWA FfAVrVd3V7eE5EnhzuaT4OricIySYjbGs7bFeSvNXWOtk7PJHG7PWKoIA41dcjJPatSD oxNL/Ehm0fnflIiW+ak6FUEPs2lBHRl9l+3MvZI6r+qN6vZatMxCJkbeTtiIXOv/w60e R1Ng8LH9krisgHfI4mhZ3xkLjT8d8WxZmwFkzLMnG3FeYxyfOSnlWJzfJroTpiEDUQmm ve9MuSna6OrASfYz1IBBkqCTMzKfVycw3DNjSvZ4nCgJr20en/L1nZitn9MPVrwJ5mUc eRNA== X-Gm-Message-State: AJIora+KNES1U9JFfHEIMKk2MethvHGaLAMGrOruJZ2VcVEV1Ni4Ylgm rs3Y9QuFMRybM2RxCdPPUgc4uX9BsUQ= X-Google-Smtp-Source: AGRyM1uvuevZ1Yap9azalvNBA6b5RFTmBv7HimXfCmqAQXOGtt8sc49I/1yQd+gUTtl7SoM1LS8HHQ== X-Received: by 2002:a63:fe49:0:b0:408:9928:44da with SMTP id x9-20020a63fe49000000b00408992844damr7186875pgj.373.1655437855127; Thu, 16 Jun 2022 20:50:55 -0700 (PDT) Received: from noah-tgl.. ([192.55.60.47]) by smtp.gmail.com with ESMTPSA id w22-20020a1709026f1600b001663e1881ecsm2352250plk.306.2022.06.16.20.50.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Jun 2022 20:50:54 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v1 1/2] x86: Add defines / utilities for making ISA specific x86 builds Date: Thu, 16 Jun 2022 20:50:49 -0700 Message-Id: <20220617035050.1252784-1-goldstein.w.n@gmail.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-12.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Noah Goldstein via Libc-alpha From: Noah Goldstein Reply-To: Noah Goldstein Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" 1. Factor out some of the ISA level defines in isa-level.c to standalone header isa-level.h 2. Add new headers with ISA level dependent macros for handling ifuncs. Note, this file does not change any code. Tested with and without multiarch on x86_64 for ISA levels: {generic, x86-64-v2, x86-64-v3, x86-64-v4} --- sysdeps/x86/isa-level.c | 17 +--- sysdeps/x86/isa-level.h | 70 +++++++++++++ sysdeps/x86_64/isa-default-include.h | 49 +++++++++ sysdeps/x86_64/multiarch/isa-ifunc-macros.h | 106 ++++++++++++++++++++ 4 files changed, 230 insertions(+), 12 deletions(-) create mode 100644 sysdeps/x86/isa-level.h create mode 100644 sysdeps/x86_64/isa-default-include.h create mode 100644 sysdeps/x86_64/multiarch/isa-ifunc-macros.h diff --git a/sysdeps/x86/isa-level.c b/sysdeps/x86/isa-level.c index 09cd72ab20..8e899bf64b 100644 --- a/sysdeps/x86/isa-level.c +++ b/sysdeps/x86/isa-level.c @@ -26,38 +26,31 @@ . */ #include - +#include /* ELF program property for x86 ISA level. */ #ifdef INCLUDE_X86_ISA_LEVEL -# if defined __SSE__ && defined __SSE2__ +# if __X86_ISA_V1 /* NB: ISAs, excluding MMX, in x86-64 ISA level baseline are used. */ # define ISA_BASELINE GNU_PROPERTY_X86_ISA_1_BASELINE # else # define ISA_BASELINE 0 # endif -# if ISA_BASELINE && defined __GCC_HAVE_SYNC_COMPARE_AND_SWAP_16 \ - && defined HAVE_X86_LAHF_SAHF && defined __POPCNT__ \ - && defined __SSE3__ && defined __SSSE3__ && defined __SSE4_1__ \ - && defined __SSE4_2__ +# if __X86_ISA_V2 /* NB: ISAs in x86-64 ISA level v2 are used. */ # define ISA_V2 GNU_PROPERTY_X86_ISA_1_V2 # else # define ISA_V2 0 # endif -# if ISA_V2 && defined __AVX__ && defined __AVX2__ && defined __F16C__ \ - && defined __FMA__ && defined __LZCNT__ && defined HAVE_X86_MOVBE \ - && defined __BMI__ && defined __BMI2__ +# if __X86_ISA_V3 /* NB: ISAs in x86-64 ISA level v3 are used. */ # define ISA_V3 GNU_PROPERTY_X86_ISA_1_V3 # else # define ISA_V3 0 # endif -# if ISA_V3 && defined __AVX512F__ && defined __AVX512BW__ \ - && defined __AVX512CD__ && defined __AVX512DQ__ \ - && defined __AVX512VL__ +# if __X86_ISA_V4 /* NB: ISAs in x86-64 ISA level v4 are used. */ # define ISA_V4 GNU_PROPERTY_X86_ISA_1_V4 # else diff --git a/sysdeps/x86/isa-level.h b/sysdeps/x86/isa-level.h new file mode 100644 index 0000000000..ed696ae8eb --- /dev/null +++ b/sysdeps/x86/isa-level.h @@ -0,0 +1,70 @@ +/* Header defining the minimum x86 ISA level + Copyright (C) 2020-2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + In addition to the permissions in the GNU Lesser General Public + License, the Free Software Foundation gives you unlimited + permission to link the compiled version of this file with other + programs, and to distribute those programs without any restriction + coming from the use of this file. (The Lesser General Public + License restrictions do apply in other respects; for example, they + cover modification of the file, and distribution when not linked + into another program.) + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef _ISA_LEVEL_H +#define _ISA_LEVEL_H + +# if defined __SSE__ && defined __SSE2__ +/* NB: ISAs, excluding MMX, in x86-64 ISA level baseline are used. */ +# define __X86_ISA_V1 1 +# else +# define __X86_ISA_V1 0 +# endif + +# if __X86_ISA_V1 && defined __GCC_HAVE_SYNC_COMPARE_AND_SWAP_16 \ + && defined HAVE_X86_LAHF_SAHF && defined __POPCNT__ \ + && defined __SSE3__ && defined __SSSE3__ && defined __SSE4_1__ \ + && defined __SSE4_2__ +/* NB: ISAs in x86-64 ISA level v2 are used. */ +# define __X86_ISA_V2 1 +# else +# define __X86_ISA_V2 0 +# endif + +# if __X86_ISA_V2 && defined __AVX__ && defined __AVX2__ && defined __F16C__ \ + && defined __FMA__ && defined __LZCNT__ && defined HAVE_X86_MOVBE \ + && defined __BMI__ && defined __BMI2__ +/* NB: ISAs in x86-64 ISA level v3 are used. */ +# define __X86_ISA_V3 1 +# else +# define __X86_ISA_V3 0 +# endif + +# if __X86_ISA_V3 && defined __AVX512F__ && defined __AVX512BW__ \ + && defined __AVX512CD__ && defined __AVX512DQ__ \ + && defined __AVX512VL__ +/* NB: ISAs in x86-64 ISA level v4 are used. */ +# define __X86_ISA_V4 1 +# else +# define __X86_ISA_V4 0 +# endif + +#define __X86_ISA_LEVEL \ + (__X86_ISA_V1 + __X86_ISA_V2 + __X86_ISA_V3 + __X86_ISA_V4) + + +#endif diff --git a/sysdeps/x86_64/isa-default-include.h b/sysdeps/x86_64/isa-default-include.h new file mode 100644 index 0000000000..d3091340af --- /dev/null +++ b/sysdeps/x86_64/isa-default-include.h @@ -0,0 +1,49 @@ +/* Utility for including proper default function based on ISA level + Copyright (C) 2021-2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#ifndef DEFAULT_V1 +# error "Must have at least ISA V1 Version" +#endif + +#ifndef DEFAULT_V2 +# define DEFAULT_V2 DEFAULT_V1 +#endif + +#ifndef DEFAULT_V3 +# define DEFAULT_V3 DEFAULT_V2 +#endif + +#ifndef DEFAULT_V4 +# define DEFAULT_V4 DEFAULT_V3 +#endif + +#define IS_DEFAULT_INCLUDE + +#if __X86_ISA_LEVEL == 1 +# include DEFAULT_V1 +#elif __X86_ISA_LEVEL == 2 +# include DEFAULT_V2 +#elif __X86_ISA_LEVEL == 3 +# include DEFAULT_V3 +#elif __X86_ISA_LEVEL == 4 +# include DEFAULT_V4 +#else +# error "Unsupport ISA Level!" +#endif diff --git a/sysdeps/x86_64/multiarch/isa-ifunc-macros.h b/sysdeps/x86_64/multiarch/isa-ifunc-macros.h new file mode 100644 index 0000000000..c24f2ab655 --- /dev/null +++ b/sysdeps/x86_64/multiarch/isa-ifunc-macros.h @@ -0,0 +1,106 @@ +/* Common ifunc selection utils + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017-2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include + +#define OPTIMIZE_DECL(...) \ + extern __typeof (REDIRECT_NAME) OPTIMIZE (__VA_ARGS__) attribute_hidden; + +#define OPTIMIZE_DECL1(...) \ + extern __typeof (REDIRECT_NAME) OPTIMIZE1 (__VA_ARGS__) attribute_hidden; + +/* Only include at the level of the build ISA or better. I.e if built + with ISA=V1, then include all implementations. On the other hand if + built with ISA=V3 only include V3/V4 implementations. If there is + not implementation at or above the build ISA level, then include + the highest ISA level implementation. */ +#if __X86_ISA_LEVEL <= 4 +# define X86_IFUNC_IMPL_ADD_V4(...) IFUNC_IMPL_ADD (__VA_ARGS__) +# define return_X86_OPTIMIZE_V4(...) return OPTIMIZE (__VA_ARGS__) +# define return_X86_OPTIMIZE1_V4(...) return OPTIMIZE1 (__VA_ARGS__) +# define DECLARE_X86_OPTIMIZE_V4(...) OPTIMIZE_DECL (__VA_ARGS__) +# define DECLARE_X86_OPTIMIZE1_V4(...) OPTIMIZE_DECL1 (__VA_ARGS__) +#endif +#if __X86_ISA_LEVEL <= 3 +# define X86_IFUNC_IMPL_ADD_V3(...) IFUNC_IMPL_ADD (__VA_ARGS__) +# define return_X86_OPTIMIZE_V3(...) return OPTIMIZE (__VA_ARGS__) +# define return_X86_OPTIMIZE1_V3(...) return OPTIMIZE1 (__VA_ARGS__) +# define DECLARE_X86_OPTIMIZE_V3(...) OPTIMIZE_DECL (__VA_ARGS__) +# define DECLARE_X86_OPTIMIZE1_V3(...) OPTIMIZE_DECL1 (__VA_ARGS__) +#endif +#if __X86_ISA_LEVEL <= 2 +# define X86_IFUNC_IMPL_ADD_V2(...) IFUNC_IMPL_ADD (__VA_ARGS__) +# define return_X86_OPTIMIZE_V2(...) return OPTIMIZE (__VA_ARGS__) +# define return_X86_OPTIMIZE1_V2(...) return OPTIMIZE1 (__VA_ARGS__) +# define DECLARE_X86_OPTIMIZE_V2(...) OPTIMIZE_DECL (__VA_ARGS__) +# define DECLARE_X86_OPTIMIZE1_V2(...) OPTIMIZE_DECL1 (__VA_ARGS__) +#endif +#if __X86_ISA_LEVEL <= 1 +# define X86_IFUNC_IMPL_ADD_V1(...) IFUNC_IMPL_ADD (__VA_ARGS__) +# define return_X86_OPTIMIZE_V1(...) return OPTIMIZE (__VA_ARGS__) +# define return_X86_OPTIMIZE1_V1(...) return OPTIMIZE1 (__VA_ARGS__) +# define DECLARE_X86_OPTIMIZE_V1(...) OPTIMIZE_DECL (__VA_ARGS__) +# define DECLARE_X86_OPTIMIZE1_V1(...) OPTIMIZE_DECL1 (__VA_ARGS__) +#endif + +#ifndef return_X86_OPTIMIZE_V4 +# define X86_IFUNC_IMPL_ADD_V4(...) +# define return_X86_OPTIMIZE_V4(...) (void) (0) +# define return_X86_OPTIMIZE1_V4(...) (void) (0) +# define DECLARE_X86_OPTIMIZE_V4(...) +# define DECLARE_X86_OPTIMIZE1_V4(...) +#endif +#ifndef return_X86_OPTIMIZE_V3 +# define X86_IFUNC_IMPL_ADD_V3(...) +# define return_X86_OPTIMIZE_V3(...) (void) (0) +# define return_X86_OPTIMIZE1_V3(...) (void) (0) +# define DECLARE_X86_OPTIMIZE_V3(...) +# define DECLARE_X86_OPTIMIZE1_V3(...) +#endif +#ifndef return_X86_OPTIMIZE_V2 +# define X86_IFUNC_IMPL_ADD_V2(...) +# define return_X86_OPTIMIZE_V2(...) (void) (0) +# define return_X86_OPTIMIZE1_V2(...) (void) (0) +# define DECLARE_X86_OPTIMIZE_V2(...) +# define DECLARE_X86_OPTIMIZE1_V2(...) +#endif +#ifndef return_X86_OPTIMIZE_V1 +# define X86_IFUNC_IMPL_ADD_V1(...) +# define return_X86_OPTIMIZE_V1(...) (void) (0) +# define return_X86_OPTIMIZE1_V1(...) (void) (0) +# define DECLARE_X86_OPTIMIZE_V1(...) +# define DECLARE_X86_OPTIMIZE1_V1(...) +#endif + +#define DECLARE_X86_OPTIMIZE_FORCE(...) OPTIMIZE_DECL (__VA_ARGS__) +#define return_X86_OPTIMIZE_FORCE(...) return OPTIMIZE (__VA_ARGS__) +#define X86_IFUNC_IMPL_ADD_FORCE(...) IFUNC_IMPL_ADD (__VA_ARGS__) + +#if __X86_ISA_LEVEL == 1 +# define X86_OPTIMIZE_FALLBACK(v1, ...) OPTIMIZE (v1) +#elif __X86_ISA_LEVEL == 2 +# define X86_OPTIMIZE_FALLBACK(v1, v2, ...) OPTIMIZE (v2) +#elif __X86_ISA_LEVEL == 3 +# define X86_OPTIMIZE_FALLBACK(v1, v2, v3, ...) OPTIMIZE (v3) +#elif __X86_ISA_LEVEL == 4 +# define X86_OPTIMIZE_FALLBACK(v1, v2, v3, v4) OPTIMIZE (v4) +#else +# error "Unsupported ISA Level" +#endif