From patchwork Sun Nov 26 21:47:55 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 841428 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ylP2t0Nnxz9s7F for ; Mon, 27 Nov 2017 08:59:01 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 7F3D59C0; Sun, 26 Nov 2017 21:58:58 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 257CB89E for ; Sun, 26 Nov 2017 21:58:57 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 945111AE for ; Sun, 26 Nov 2017 21:58:52 +0000 (UTC) Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Nov 2017 13:58:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.44,461,1505804400"; d="scan'208";a="180827713" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by fmsmga006.fm.intel.com with ESMTP; 26 Nov 2017 13:58:51 -0800 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Sun, 26 Nov 2017 21:47:55 +0000 Message-Id: <1511732875-8514-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH] dpif-netdev: Allocate dp_netdev_pmd_thread struct by xzalloc_cacheline. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org All instances of struct dp_netdev_pmd_thread are allocated by xzalloc and therefore doesn't guarantee memory allocation aligned on CACHE_LINE_SIZE boundary. Due to this any padding done inside the structure with this assumption might create holes. This commit replaces xzalloc, free with xzalloc_cacheline and free_cacheline. With the changes the memory is 64 byte aligned. Before: With xzalloc, all the memory is 16 byte aligned. (gdb) p pmd $1 = (struct dp_netdev_pmd_thread *) 0x7eff8a813010 (gdb) p &pmd->cacheline0 $2 = (OVS_CACHE_LINE_MARKER *) 0x7eff8a813010 (gdb) p &pmd->cacheline1 $3 = (OVS_CACHE_LINE_MARKER *) 0x7eff8a813050 (gdb) p &pmd->flow_cache $4 = (struct emc_cache *) 0x7eff8a813090 (gdb) p &pmd->flow_table $5 = (struct cmap *) 0x7eff8acb30d0 (gdb) p &pmd->stats $6 = (struct dp_netdev_pmd_stats *) 0x7eff8acb3110 (gdb) p &pmd->port_mutex $7 = (struct ovs_mutex *) 0x7eff8acb3150 (gdb) p &pmd->poll_list $8 = (struct hmap *) 0x7eff8acb3190 (gdb) p &pmd->tnl_port_cache $9 = (struct hmap *) 0x7eff8acb31d0 (gdb) p &pmd->stats_zero $10 = (unsigned long long (*)[5]) 0x7eff8acb3210 After: With xzalloc_cacheline, all the memory is 64 byte aligned. (gdb) p pmd $1 = (struct dp_netdev_pmd_thread *) 0x7f39e2365040 (gdb) p &pmd->cacheline0 $2 = (OVS_CACHE_LINE_MARKER *) 0x7f39e2365040 (gdb) p &pmd->cacheline1 $3 = (OVS_CACHE_LINE_MARKER *) 0x7f39e2365080 (gdb) p &pmd->flow_cache $4 = (struct emc_cache *) 0x7f39e23650c0 (gdb) p &pmd->flow_table $5 = (struct cmap *) 0x7f39e2805100 (gdb) p &pmd->stats $6 = (struct dp_netdev_pmd_stats *) 0x7f39e2805140 (gdb) p &pmd->port_mutex $7 = (struct ovs_mutex *) 0x7f39e2805180 (gdb) p &pmd->poll_list $8 = (struct hmap *) 0x7f39e28051c0 (gdb) p &pmd->tnl_port_cache $9 = (struct hmap *) 0x7f39e2805200 (gdb) p &pmd->stats_zero $10 = (unsigned long long (*)[5]) 0x7f39e2805240 Reported-by: Ilya Maximets Signed-off-by: Bhanuprakash Bodireddy --- lib/dpif-netdev.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index db78318..3e281ae 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -3646,7 +3646,7 @@ reconfigure_pmd_threads(struct dp_netdev *dp) FOR_EACH_CORE_ON_DUMP(core, pmd_cores) { pmd = dp_netdev_get_pmd(dp, core->core_id); if (!pmd) { - pmd = xzalloc(sizeof *pmd); + pmd = xzalloc_cacheline(sizeof *pmd); dp_netdev_configure_pmd(pmd, dp, core->core_id, core->numa_id); pmd->thread = ovs_thread_create("pmd", pmd_thread_main, pmd); VLOG_INFO("PMD thread on numa_id: %d, core id: %2d created.", @@ -4574,7 +4574,7 @@ dp_netdev_destroy_pmd(struct dp_netdev_pmd_thread *pmd) xpthread_cond_destroy(&pmd->cond); ovs_mutex_destroy(&pmd->cond_mutex); ovs_mutex_destroy(&pmd->port_mutex); - free(pmd); + free_cacheline(pmd); } /* Stops the pmd thread, removes it from the 'dp->poll_threads',