From patchwork Wed Mar 26 10:37:01 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hu Tao X-Patchwork-Id: 333919 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 352B3140084 for ; Thu, 27 Mar 2014 00:54:34 +1100 (EST) Received: from localhost ([::1]:46948 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WSlms-0005U5-DJ for incoming@patchwork.ozlabs.org; Wed, 26 Mar 2014 07:14:26 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38034) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WSljO-0002Ij-5g for qemu-devel@nongnu.org; Wed, 26 Mar 2014 07:10:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WSljI-0003Ch-RQ for qemu-devel@nongnu.org; Wed, 26 Mar 2014 07:10:50 -0400 Received: from [59.151.112.132] (port=53275 helo=heian.cn.fujitsu.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WSljH-0003CR-DL for qemu-devel@nongnu.org; Wed, 26 Mar 2014 07:10:44 -0400 X-IronPort-AV: E=Sophos;i="4.97,734,1389715200"; d="scan'208";a="28487336" Received: from unknown (HELO edo.cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 26 Mar 2014 18:34:34 +0800 Received: from G08CNEXCHPEKD01.g08.fujitsu.local (localhost.localdomain [127.0.0.1]) by edo.cn.fujitsu.com (8.14.3/8.13.1) with ESMTP id s2QAapw2003096; Wed, 26 Mar 2014 18:36:55 +0800 Received: from G08CNEXMBPEKD03.g08.fujitsu.local ([10.167.33.86]) by G08CNEXCHPEKD01.g08.fujitsu.local ([10.167.33.80]) with mapi id 14.03.0146.002; Wed, 26 Mar 2014 18:37:06 +0800 From: "hutao@cn.fujitsu.com" To: "qemu-devel@nongnu.org" Thread-Topic: [PATCH v3 27/34] hostmem: add properties for NUMA memory policy Thread-Index: AQHPSN9V+mWle74okUaredHr0rau5A== Date: Wed, 26 Mar 2014 10:37:01 +0000 Message-ID: <0169d4097fcc89addaf5e3292db0e38290b14842.1395825871.git.hutao@cn.fujitsu.com> References: In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.167.226.102] MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 59.151.112.132 Cc: "ehabkost@redhat.com" , "imammedo@redhat.com" , "mtosatti@redhat.com" , Paolo Bonzini , "a.motakis@virtualopensystems.com" , "gaowanlong@cn.fujitsu.com" Subject: [Qemu-devel] [PATCH v3 27/34] hostmem: add properties for NUMA memory policy X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Signed-off-by: Hu Tao [Raise errors on setting properties if !CONFIG_NUMA. Add BUILD_BUG_ON checks. - Paolo] Signed-off-by: Paolo Bonzini --- backends/hostmem.c | 109 ++++++++++++++++++++++++++++++++++++++++++++++- include/sysemu/hostmem.h | 4 ++ qapi-schema.json | 20 +++++++++ 3 files changed, 132 insertions(+), 1 deletion(-) diff --git a/backends/hostmem.c b/backends/hostmem.c index 738bb31..d3f8476 100644 --- a/backends/hostmem.c +++ b/backends/hostmem.c @@ -10,11 +10,20 @@ * See the COPYING file in the top-level directory. */ #include "sysemu/hostmem.h" -#include "sysemu/sysemu.h" #include "qapi/visitor.h" +#include "qapi-types.h" +#include "qapi-visit.h" #include "qapi/qmp/qerror.h" #include "qom/object_interfaces.h" +#ifdef CONFIG_NUMA +#include +QEMU_BUILD_BUG_ON(HOST_MEM_POLICY_DEFAULT != MPOL_DEFAULT); +QEMU_BUILD_BUG_ON(HOST_MEM_POLICY_PREFERRED != MPOL_PREFERRED); +QEMU_BUILD_BUG_ON(HOST_MEM_POLICY_BIND != MPOL_BIND); +QEMU_BUILD_BUG_ON(HOST_MEM_POLICY_INTERLEAVE != MPOL_INTERLEAVE); +#endif + static void host_memory_backend_get_size(Object *obj, Visitor *v, void *opaque, const char *name, Error **errp) @@ -49,6 +58,84 @@ host_memory_backend_set_size(Object *obj, Visitor *v, void *opaque, backend->size = value; } +static void +get_host_nodes(Object *obj, Visitor *v, void *opaque, const char *name, + Error **errp) +{ + HostMemoryBackend *backend = MEMORY_BACKEND(obj); + uint16List *host_nodes = NULL; + uint16List **node = &host_nodes; + unsigned long value; + + value = find_first_bit(backend->host_nodes, MAX_NODES); + if (value == MAX_NODES) { + return; + } + + *node = g_malloc0(sizeof(**node)); + (*node)->value = value; + node = &(*node)->next; + + do { + value = find_next_bit(backend->host_nodes, MAX_NODES, value + 1); + if (value == MAX_NODES) { + break; + } + + *node = g_malloc0(sizeof(**node)); + (*node)->value = value; + node = &(*node)->next; + } while (true); + + visit_type_uint16List(v, &host_nodes, name, errp); +} + +static void +set_host_nodes(Object *obj, Visitor *v, void *opaque, const char *name, + Error **errp) +{ +#ifdef CONFIG_NUMA + HostMemoryBackend *backend = MEMORY_BACKEND(obj); + uint16List *l = NULL; + + visit_type_uint16List(v, &l, name, errp); + + while (l) { + bitmap_set(backend->host_nodes, l->value, 1); + l = l->next; + } +#else + error_setg(errp, "NUMA node binding are not supported by this QEMU"); +#endif +} + +static void +get_policy(Object *obj, Visitor *v, void *opaque, const char *name, + Error **errp) +{ + HostMemoryBackend *backend = MEMORY_BACKEND(obj); + int policy = backend->policy; + + visit_type_enum(v, &policy, HostMemPolicy_lookup, NULL, name, errp); +} + +static void +set_policy(Object *obj, Visitor *v, void *opaque, const char *name, + Error **errp) +{ + HostMemoryBackend *backend = MEMORY_BACKEND(obj); + int policy; + + visit_type_enum(v, &policy, HostMemPolicy_lookup, NULL, name, errp); + backend->policy = policy; + +#ifndef CONFIG_NUMA + if (policy != HOST_MEM_POLICY_DEFAULT) { + error_setg(errp, "NUMA policies are not supported by this QEMU"); + } +#endif +} + static bool host_memory_backend_get_merge(Object *obj, Error **errp) { HostMemoryBackend *backend = MEMORY_BACKEND(obj); @@ -159,6 +246,12 @@ static void host_memory_backend_initfn(Object *obj) object_property_add(obj, "size", "int", host_memory_backend_get_size, host_memory_backend_set_size, NULL, NULL, NULL); + object_property_add(obj, "host-nodes", "int", + get_host_nodes, + set_host_nodes, NULL, NULL, NULL); + object_property_add(obj, "policy", "str", + get_policy, + set_policy, NULL, NULL, NULL); } static void host_memory_backend_finalize(Object *obj) @@ -203,6 +296,20 @@ host_memory_backend_memory_init(UserCreatable *uc, Error **errp) if (backend->prealloc) { os_mem_prealloc(memory_region_get_fd(&backend->mr), ptr, sz); } + +#ifdef CONFIG_NUMA + unsigned long maxnode = find_last_bit(backend->host_nodes, MAX_NODES); + + /* This is a workaround for a long standing bug in Linux' + * mbind implementation, which cuts off the last specified + * node. + */ + if (mbind(ptr, sz, backend->policy, backend->host_nodes, maxnode + 2, 0)) { + error_setg_errno(errp, errno, + "cannot bind memory to host NUMA nodes"); + return; + } +#endif } MemoryRegion * diff --git a/include/sysemu/hostmem.h b/include/sysemu/hostmem.h index fdbc41c..9e1a816 100644 --- a/include/sysemu/hostmem.h +++ b/include/sysemu/hostmem.h @@ -12,9 +12,11 @@ #ifndef QEMU_HOSTMEM_H #define QEMU_HOSTMEM_H +#include "sysemu/sysemu.h" /* for MAX_NODES */ #include "qom/object.h" #include "qapi/error.h" #include "exec/memory.h" +#include "qemu/bitmap.h" #define TYPE_MEMORY_BACKEND "memory" #define MEMORY_BACKEND(obj) \ @@ -53,6 +55,8 @@ struct HostMemoryBackend { uint64_t size; bool merge, dump; bool prealloc, force_prealloc; + DECLARE_BITMAP(host_nodes, MAX_NODES); + HostMemPolicy policy; MemoryRegion mr; }; diff --git a/qapi-schema.json b/qapi-schema.json index a8c16bc..5aea3ed 100644 --- a/qapi-schema.json +++ b/qapi-schema.json @@ -4723,3 +4723,23 @@ '*cpus': ['uint16'], '*mem': 'size', '*memdev': 'str' }} + +## +# @HostMemPolicy +# +# Host memory policy types +# +# @default: restore default policy, remove any nondefault policy +# +# @preferred: set the preferred host nodes for allocation +# +# @bind: a strict policy that restricts memory allocation to the +# host nodes specified +# +# @interleave: memory allocations are interleaved across the set +# of host nodes specified +# +# Since 2.1 +## +{ 'enum': 'HostMemPolicy', + 'data': [ 'default', 'preferred', 'bind', 'interleave' ] }