From patchwork Tue Nov 28 18:28:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Pfaff X-Patchwork-Id: 842251 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ymXHQ2jzsz9sBd for ; Wed, 29 Nov 2017 05:28:50 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id E8474CD1; Tue, 28 Nov 2017 18:28:46 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 2F6D8B14 for ; Tue, 28 Nov 2017 18:28:45 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 60FF03FB for ; Tue, 28 Nov 2017 18:28:44 +0000 (UTC) X-Originating-IP: 208.91.3.26 Received: from sigabrt.benpfaff.org (unknown [208.91.3.26]) (Authenticated sender: blp@ovn.org) by relay3-d.mail.gandi.net (Postfix) with ESMTPSA id 02B54A80C2; Tue, 28 Nov 2017 19:28:41 +0100 (CET) From: Ben Pfaff To: dev@openvswitch.org Date: Tue, 28 Nov 2017 10:28:33 -0800 Message-Id: <20171128182833.23398-1-blp@ovn.org> X-Mailer: git-send-email 2.10.2 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Ben Pfaff Subject: [ovs-dev] [PATCH] util: Make xmalloc_cacheline() allocate full cachelines. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Until now, xmalloc_cacheline() has provided its caller memory that does not share a cache line, but when posix_memalign() is not available it did not provide a full cache line; instead, it returned memory that was offset 8 bytes into a cache line. This makes it hard for clients to design structures to be cache line-aligned. This commit changes xmalloc_cacheline() to always return a full cache line instead of memory offset into one. Signed-off-by: Ben Pfaff Acked-by: Bhanuprakash Bodireddy --- lib/util.c | 60 ++++++++++++++++++++++++++++++++---------------------------- 1 file changed, 32 insertions(+), 28 deletions(-) diff --git a/lib/util.c b/lib/util.c index 9e6edd27ae4c..137091a3cd4f 100644 --- a/lib/util.c +++ b/lib/util.c @@ -196,15 +196,9 @@ x2nrealloc(void *p, size_t *n, size_t s) return xrealloc(p, *n * s); } -/* The desired minimum alignment for an allocated block of memory. */ -#define MEM_ALIGN MAX(sizeof(void *), 8) -BUILD_ASSERT_DECL(IS_POW2(MEM_ALIGN)); -BUILD_ASSERT_DECL(CACHE_LINE_SIZE >= MEM_ALIGN); - -/* Allocates and returns 'size' bytes of memory in dedicated cache lines. That - * is, the memory block returned will not share a cache line with other data, - * avoiding "false sharing". (The memory returned will not be at the start of - * a cache line, though, so don't assume such alignment.) +/* Allocates and returns 'size' bytes of memory aligned to a cache line and in + * dedicated cache lines. That is, the memory block returned will not share a + * cache line with other data, avoiding "false sharing". * * Use free_cacheline() to free the returned memory block. */ void * @@ -221,28 +215,37 @@ xmalloc_cacheline(size_t size) } return p; #else - void **payload; - void *base; - /* Allocate room for: * - * - Up to CACHE_LINE_SIZE - 1 bytes before the payload, so that the - * start of the payload doesn't potentially share a cache line. + * - Header padding: Up to CACHE_LINE_SIZE - 1 bytes, to allow the + * pointer to be aligned exactly sizeof(void *) bytes before the + * beginning of a cache line. * - * - A payload consisting of a void *, followed by padding out to - * MEM_ALIGN bytes, followed by 'size' bytes of user data. + * - Pointer: A pointer to the start of the header padding, to allow us + * to free() the block later. * - * - Space following the payload up to the end of the cache line, so - * that the end of the payload doesn't potentially share a cache line - * with some following block. */ - base = xmalloc((CACHE_LINE_SIZE - 1) - + ROUND_UP(MEM_ALIGN + size, CACHE_LINE_SIZE)); - - /* Locate the payload and store a pointer to the base at the beginning. */ - payload = (void **) ROUND_UP((uintptr_t) base, CACHE_LINE_SIZE); - *payload = base; - - return (char *) payload + MEM_ALIGN; + * - User data: 'size' bytes. + * + * - Trailer padding: Enough to bring the user data up to a cache line + * multiple. + * + * +---------------+---------+------------------------+---------+ + * | header | pointer | user data | trailer | + * +---------------+---------+------------------------+---------+ + * ^ ^ ^ + * | | | + * p q r + * + */ + void *p = xmalloc((CACHE_LINE_SIZE - 1) + + sizeof(void *) + + ROUND_UP(size, CACHE_LINE_SIZE)); + bool runt = PAD_SIZE((uintptr_t) p, CACHE_LINE_SIZE) < sizeof(void *); + void *r = (void *) ROUND_UP((uintptr_t) p + (runt ? CACHE_LINE_SIZE : 0), + CACHE_LINE_SIZE); + void **q = (void **) r - 1; + *q = p; + return r; #endif } @@ -265,7 +268,8 @@ free_cacheline(void *p) free(p); #else if (p) { - free(*(void **) ((uintptr_t) p - MEM_ALIGN)); + void **q = (void **) p - 1; + free(*q); } #endif }