From patchwork Mon Jun 22 15:15:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florian Weimer X-Patchwork-Id: 1314543 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=libc-alpha-bounces@sourceware.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=sourceware.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=PPRGxNlr; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49rCfT33Nrz9s6w for ; Tue, 23 Jun 2020 01:16:01 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id EEF3D3956837; Mon, 22 Jun 2020 15:15:42 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org EEF3D3956837 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1592838943; bh=Q/W5DjHOp0Kd7+BFb5h8lHBZCqi9THYM/hvpb0S0MfY=; h=To:Subject:In-Reply-To:References:Date:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=PPRGxNlrOAg1Q3pz7Oylwf29VwvBFDgdL59Ss2oo3BTH2rdt8KunAMMUP3Ph7TUFn zVBdoxtCrvtNCo6ZmcEvjfny0UsJnRA6m+AIy2iQy88OEaxSCfGDyM9CKkpOSHx1ID a2iJbRdnhSnHFTAbuKXEnOcXDk8g9ec8rBqMQRfo= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from us-smtp-delivery-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by sourceware.org (Postfix) with ESMTP id D0E1E39574D0 for ; Mon, 22 Jun 2020 15:15:39 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org D0E1E39574D0 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-229-fjLeMKA5PYqu2jQBqtLCsQ-1; Mon, 22 Jun 2020 11:15:37 -0400 X-MC-Unique: fjLeMKA5PYqu2jQBqtLCsQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 91B78464 for ; Mon, 22 Jun 2020 15:15:36 +0000 (UTC) Received: from oldenburg2.str.redhat.com (ovpn-112-185.ams2.redhat.com [10.36.112.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9021A5D9E2 for ; Mon, 22 Jun 2020 15:15:35 +0000 (UTC) To: libc-alpha@sourceware.org Subject: [PATCH 30/30] elf: Add glibc-hwcaps subdirectory support to ld.so cache processing In-Reply-To: References: Message-Id: <7e66c31e71e225083451334ad9b6339eef29dd07.1592836143.git.fweimer@redhat.com> Date: Mon, 22 Jun 2020 17:15:33 +0200 User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-13.0 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Florian Weimer via Libc-alpha From: Florian Weimer Reply-To: Florian Weimer Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" This recognizes the DL_CACHE_HWCAP_EXTENSION flag and picks up the supported cache entry with the highest priority. --- elf/dl-cache.c | 182 +++++++++++++++++++++++++++++++++++++++++++++++- elf/dl-hwcaps.c | 78 +++++++++++++++++++++ elf/dl-hwcaps.h | 19 +++++ 3 files changed, 276 insertions(+), 3 deletions(-) diff --git a/elf/dl-cache.c b/elf/dl-cache.c index 02c46ffb0c..4714119974 100644 --- a/elf/dl-cache.c +++ b/elf/dl-cache.c @@ -35,6 +35,132 @@ static struct cache_file *cache; static struct cache_file_new *cache_new; static size_t cachesize; +#ifdef SHARED +/* This is used to cache the priorities of glibc-hwcaps + subdirectories. The elements of _dl_cache_priorities correspond to + the strings in the cache_extension_tag_glibc_hwcaps section. */ +static uint32_t *glibc_hwcaps_priorities; +static uint32_t glibc_hwcaps_priorities_length; +static uint32_t glibc_hwcaps_priorities_allocated; + +/* True if the full malloc was used to allocated the array. */ +static bool glibc_hwcaps_priorities_malloced; + +/* Deallocate the glibc_hwcaps_priorities array. */ +static void +glibc_hwcaps_priorities_free (void) +{ + /* When the minimal malloc is in use, free does not do anything, + so it does not make sense to call it. */ + if (glibc_hwcaps_priorities_malloced) + free (glibc_hwcaps_priorities); + glibc_hwcaps_priorities = NULL; + glibc_hwcaps_priorities_allocated = 0; +} + +/* Return the priority of the cache_extension_tag_glibc_hwcaps section + entry at INDEX. Zero means do not use. Otherwise, lower values + indicate greater preference. */ +static uint32_t __attribute__ ((noinline, noclone)) +glibc_hwcaps_priority (uint32_t index) +{ + /* Using a zero-length array as an indicator that nothing has been + loaded is not a problem: It does not lead to repeated + initialization attempts because caches without an extension + section are processed without calling this function (unless the + file is corrupted). */ + if (glibc_hwcaps_priorities_length == 0) + { + struct cache_extension_all_loaded ext; + if (!cache_extension_load (cache_new, cache, cachesize, &ext)) + return 0; + + uint32_t length + = (ext.sections[cache_extension_tag_glibc_hwcaps].size + / sizeof (uint32_t)); + if (length > glibc_hwcaps_priorities_allocated) + { + glibc_hwcaps_priorities_free (); + + glibc_hwcaps_priorities = malloc (length * sizeof (uint32_t)); + if (glibc_hwcaps_priorities == NULL) + /* Disable hwcaps on memory allocation error. */ + return 0; + + glibc_hwcaps_priorities_allocated = length; + glibc_hwcaps_priorities_malloced = __rtld_malloc_is_full (); + } + + /* Compute the priorities for the subdirectories by merging the + array in the cache with the dl_hwcaps_priorities array. */ + const uint32_t *left + = ext.sections[cache_extension_tag_glibc_hwcaps].base; + const uint32_t *left_end = left + length; + struct dl_hwcaps_priority *right = _dl_hwcaps_priorities; + struct dl_hwcaps_priority *right_end + = right + _dl_hwcaps_priorities_length; + uint32_t *result = glibc_hwcaps_priorities; + + while (left < left_end && right < right_end) + { + uint32_t string_table_index = *left; + if (string_table_index < cachesize) + { + const char *left_name + = (const char *) cache + string_table_index; + uint32_t left_name_length = strlen (left_name); + uint32_t to_compare; + if (left_name_length < right->name_length) + to_compare = left_name_length; + else + to_compare = right->name_length; + int cmp = memcmp (left_name, right->name, to_compare); + if (cmp == 0) + { + if (left_name_length < right->name_length) + cmp = -1; + else if (left_name_length > right->name_length) + cmp = 1; + } + if (cmp == 0) + { + *result = right->priority; + ++result; + ++left; + ++right; + } + else if (cmp < 0) + { + *result = 0; + ++result; + ++left; + } + else + ++right; + } + else + { + *result = 0; + ++result; + } + } + while (left < left_end) + { + *result = 0; + ++result; + ++left; + } + + glibc_hwcaps_priorities_length = length; + } + + if (index < glibc_hwcaps_priorities_length) + return glibc_hwcaps_priorities[index]; + else + return 0; +} +#endif /* SHARED */ + /* True if PTR is a valid string table index. */ static inline bool _dl_cache_verify_ptr (uint32_t ptr, size_t string_table_size) @@ -74,6 +200,9 @@ search_cache (const char *string_table, uint32_t string_table_size, int left = 0; int right = nlibs - 1; const char *best = NULL; +#ifdef SHARED + uint32_t best_priority = 0; +#endif while (left <= right) { @@ -129,6 +258,11 @@ search_cache (const char *string_table, uint32_t string_table_size, { if (best == NULL || flags == GLRO (dl_correct_cache_id)) { + /* Named/extension hwcaps get slightly different + treatment: We keep searching for a better + match. */ + bool named_hwcap = false; + if (entry_size >= sizeof (struct file_entry_new)) { /* The entry is large enough to include @@ -136,7 +270,18 @@ search_cache (const char *string_table, uint32_t string_table_size, struct file_entry_new *libnew = (struct file_entry_new *) lib; - if (libnew->hwcap & hwcap_exclude) +#ifdef SHARED + named_hwcap = dl_cache_hwcap_extension (libnew); +#endif + + /* The entries with named/extension hwcaps + have been exhausted. Return the best + match encountered so far if there is + one. */ + if (!named_hwcap && best != NULL) + break; + + if ((libnew->hwcap & hwcap_exclude) && !named_hwcap) continue; if (GLRO (dl_osversion) && libnew->osversion > GLRO (dl_osversion)) @@ -146,14 +291,41 @@ search_cache (const char *string_table, uint32_t string_table_size, && ((libnew->hwcap & _DL_HWCAP_PLATFORM) != platform)) continue; + +#ifdef SHARED + /* For named hwcaps, determine the priority + and see if beats what has been found so + far. */ + if (named_hwcap) + { + uint32_t entry_priority + = glibc_hwcaps_priority (libnew->hwcap); + if (entry_priority == 0) + /* Not usable at all. Skip. */ + continue; + else if (best == NULL + || entry_priority < best_priority) + /* This entry is of higher priority + than the previous one, or it is the + first entry. */ + best_priority = entry_priority; + else + /* An entry has already been found, + but it is a better match. */ + continue; + } +#endif /* SHARED */ } best = string_table + lib->value; - if (flags == GLRO (dl_correct_cache_id)) + if (flags == GLRO (dl_correct_cache_id) + && !named_hwcap) /* We've found an exact match for the shared object and no general `ELF' release. Stop - searching. */ + searching, but not if a named (extension) + hwcap is used. In this case, an entry with + a higher priority may come up later. */ break; } } @@ -346,5 +518,9 @@ _dl_unload_cache (void) __munmap (cache, cachesize); cache = NULL; } +#ifdef SHARED + /* This marks the glibc_hwcaps_priorities array as out-of-date. */ + glibc_hwcaps_priorities_length = 0; +#endif } #endif diff --git a/elf/dl-hwcaps.c b/elf/dl-hwcaps.c index 4de94759a2..cb53337025 100644 --- a/elf/dl-hwcaps.c +++ b/elf/dl-hwcaps.c @@ -89,6 +89,81 @@ copy_hwcaps (struct copy_hwcaps *target, const char *hwcaps, } } +struct dl_hwcaps_priority *_dl_hwcaps_priorities; +uint32_t _dl_hwcaps_priorities_length; + +/* Allocate _dl_hwcaps_priorities and fill it with data. */ +static void +compute_priorities (size_t total_count, const char *prepend, + int32_t bitmask, const char *mask) +{ + _dl_hwcaps_priorities = malloc (total_count + * sizeof (*_dl_hwcaps_priorities)); + if (_dl_hwcaps_priorities == NULL) + _dl_signal_error (ENOMEM, NULL, NULL, + N_("cannot create HWCAP priorities")); + _dl_hwcaps_priorities_length = total_count; + + /* First the prepended subdirectories. */ + size_t i = 0; + { + struct dl_hwcaps_split sp; + _dl_hwcaps_split_init (&sp, prepend); + while (_dl_hwcaps_split (&sp)) + { + _dl_hwcaps_priorities[i].name = sp.segment; + _dl_hwcaps_priorities[i].name_length = sp.length; + _dl_hwcaps_priorities[i].priority = i + 1; + ++i; + } + } + + /* Then the built-in subdirectories that are actually active. */ + { + struct dl_hwcaps_split_masked sp; + _dl_hwcaps_split_masked_init (&sp, _dl_hwcaps_subdirs, bitmask, mask); + while (_dl_hwcaps_split_masked (&sp)) + { + _dl_hwcaps_priorities[i].name = sp.split.segment; + _dl_hwcaps_priorities[i].name_length = sp.split.length; + _dl_hwcaps_priorities[i].priority = i + 1; + ++i; + } + } + assert (i == total_count); +} + +/* Sort the _dl_hwcaps_priorities array by name. */ +static void +sort_priorities_by_name (void) +{ + /* Insertion sort. There is no need to link qsort into the dynamic + loader for such a short array. */ + for (size_t i = 1; i < _dl_hwcaps_priorities_length; ++i) + for (size_t j = i; j > 0; --j) + { + struct dl_hwcaps_priority *previous = _dl_hwcaps_priorities + j - 1; + struct dl_hwcaps_priority *current = _dl_hwcaps_priorities + j; + + /* Bail out if current is greater or equal to the previous + value. */ + uint32_t to_compare; + if (current->name_length < previous->name_length) + to_compare = current->name_length; + else + to_compare = previous->name_length; + int cmp = memcmp (current->name, previous->name, to_compare); + if (cmp >= 0 + || (cmp == 0 && current->name_length >= previous->name_length)) + break; + + /* Swap *previous and *current. */ + struct dl_hwcaps_priority tmp = *previous; + *previous = *current; + *current = tmp; + } +} + /* Return an array of useful/necessary hardware capability names. */ const struct r_strlenpair * _dl_important_hwcaps (const char *glibc_hwcaps_prepend, @@ -111,6 +186,9 @@ _dl_important_hwcaps (const char *glibc_hwcaps_prepend, count_hwcaps (&hwcaps_counts, glibc_hwcaps_prepend, -1, NULL); count_hwcaps (&hwcaps_counts, _dl_hwcaps_subdirs, hwcaps_subdirs_active, glibc_hwcaps_mask); + compute_priorities (hwcaps_counts.count, glibc_hwcaps_prepend, + hwcaps_subdirs_active, glibc_hwcaps_mask); + sort_priorities_by_name (); /* Each hwcaps subdirectory has a GLIBC_HWCAPS_PREFIX string prefix and a "/" suffix once stored in the result. */ diff --git a/elf/dl-hwcaps.h b/elf/dl-hwcaps.h index a6453f15f3..4b4c143854 100644 --- a/elf/dl-hwcaps.h +++ b/elf/dl-hwcaps.h @@ -110,4 +110,23 @@ extern const char _dl_hwcaps_subdirs[] attribute_hidden; bitmask. */ int32_t _dl_hwcaps_subdirs_active (void) attribute_hidden; +/* Pre-computed glibc-hwcaps subdirectory priorities. Used in + dl-cache.c to quickly find the proprities for the stored HWCAP + names. */ +struct dl_hwcaps_priority +{ + /* The name consists of name_length bytes at name (not necessarily + null-terminated). */ + const char *name; + uint32_t name_length; + + /* Priority of this name. A positive number. */ + uint32_t priority; +}; + +/* Pre-computed hwcaps priorities. Set up by + _dl_important_hwcaps. */ +extern struct dl_hwcaps_priority *_dl_hwcaps_priorities attribute_hidden; +extern uint32_t _dl_hwcaps_priorities_length attribute_hidden; + #endif /* _DL_HWCAPS_H */