From patchwork Wed Jan 16 08:10:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Yi" X-Patchwork-Id: 1025680 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43ffz25N1vz9sCX for ; Wed, 16 Jan 2019 19:10:38 +1100 (AEDT) Received: from localhost ([127.0.0.1]:43544 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gjgHU-0004Te-O0 for incoming@patchwork.ozlabs.org; Wed, 16 Jan 2019 03:10:36 -0500 Received: from eggs.gnu.org ([209.51.188.92]:46717) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gjgGy-0004SN-HB for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:10:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gjgGx-0005sm-NR for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:10:04 -0500 Received: from mga03.intel.com ([134.134.136.65]:45194) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gjgGx-0005jy-C4 for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:10:03 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Jan 2019 00:10:01 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,485,1539673200"; d="scan'208";a="136205775" Received: from linux.intel.com ([10.54.29.200]) by fmsmga004.fm.intel.com with ESMTP; 16 Jan 2019 00:10:01 -0800 Received: from dazhang1-ssd.sh.intel.com (dazhang1-ssd.sh.intel.com [10.239.48.128]) by linux.intel.com (Postfix) with ESMTP id 276195800C6; Wed, 16 Jan 2019 00:09:59 -0800 (PST) From: Zhang Yi To: xiaoguangrong.eric@gmail.com, stefanha@redhat.com, pbonzini@redhat.com, pagupta@redhat.com, yu.c.zhang@linux.intel.com, mst@redhat.com, ehabkost@redhat.com Date: Wed, 16 Jan 2019 16:10:17 +0800 Message-Id: <5a9ab3832eacb1e4bc6649f72bcf2969074573f1.1547624239.git.yi.z.zhang@linux.intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 134.134.136.65 Subject: [Qemu-devel] [PATCH V9 1/6] numa: Fixed the memory leak of numa error message X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: imammedo@redhat.com, dan.j.williams@intel.com, qemu-devel@nongnu.org, Zhang Yi Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" object_get_canonical_path_component() returns a string which must be freed using g_free(). Signed-off-by: Zhang Yi Reviewed-by: Pankaj gupta Reviewed-by: Igor Mammedov Reviewed-by: Eduardo Habkost Reviewed-by: Michael S. Tsirkin --- numa.c | 1 + 1 file changed, 1 insertion(+) diff --git a/numa.c b/numa.c index 50ec016..3875e1e 100644 --- a/numa.c +++ b/numa.c @@ -533,6 +533,7 @@ void memory_region_allocate_system_memory(MemoryRegion *mr, Object *owner, error_report("memory backend %s is used multiple times. Each " "-numa option must use a different memdev value.", path); + g_free(path); exit(1); } From patchwork Wed Jan 16 08:10:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Yi" X-Patchwork-Id: 1025681 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43ffzs4ssbz9sCX for ; Wed, 16 Jan 2019 19:11:21 +1100 (AEDT) Received: from localhost ([127.0.0.1]:43675 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gjgIB-0004sE-Gp for incoming@patchwork.ozlabs.org; Wed, 16 Jan 2019 03:11:19 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48057) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gjgHW-0004qM-H5 for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:10:39 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gjgHQ-000707-WF for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:10:34 -0500 Received: from mga17.intel.com ([192.55.52.151]:47090) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gjgHC-0006s0-Un for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:10:23 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Jan 2019 00:10:13 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,485,1539673200"; d="scan'208";a="267507828" Received: from linux.intel.com ([10.54.29.200]) by orsmga004.jf.intel.com with ESMTP; 16 Jan 2019 00:10:13 -0800 Received: from dazhang1-ssd.sh.intel.com (dazhang1-ssd.sh.intel.com [10.239.48.128]) by linux.intel.com (Postfix) with ESMTP id 4EB265800C6; Wed, 16 Jan 2019 00:10:11 -0800 (PST) From: Zhang Yi To: xiaoguangrong.eric@gmail.com, stefanha@redhat.com, pbonzini@redhat.com, pagupta@redhat.com, yu.c.zhang@linux.intel.com, mst@redhat.com, ehabkost@redhat.com Date: Wed, 16 Jan 2019 16:10:29 +0800 Message-Id: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.151 Subject: [Qemu-devel] [PATCH V9 2/6] memory: use sparse feature define RAM_FLAG. X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: imammedo@redhat.com, dan.j.williams@intel.com, qemu-devel@nongnu.org, Zhang Yi Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Zhang Yi Signed-off-by: Michael S. Tsirkin --- include/exec/memory.h | 12 ++++++------ include/qemu/osdep.h | 9 +++++++++ 2 files changed, 15 insertions(+), 6 deletions(-) diff --git a/include/exec/memory.h b/include/exec/memory.h index 667466b..03824d9 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -104,27 +104,27 @@ struct IOMMUNotifier { typedef struct IOMMUNotifier IOMMUNotifier; /* RAM is pre-allocated and passed into qemu_ram_alloc_from_ptr */ -#define RAM_PREALLOC (1 << 0) +#define RAM_PREALLOC ((QEMU_FORCE QemuMmapFlags) (1 << 0)) /* RAM is mmap-ed with MAP_SHARED */ -#define RAM_SHARED (1 << 1) +#define RAM_SHARED ((QEMU_FORCE QemuMmapFlags) (1 << 1)) /* Only a portion of RAM (used_length) is actually used, and migrated. * This used_length size can change across reboots. */ -#define RAM_RESIZEABLE (1 << 2) +#define RAM_RESIZEABLE ((QEMU_FORCE QemuMmapFlags) (1 << 2)) /* UFFDIO_ZEROPAGE is available on this RAMBlock to atomically * zero the page and wake waiting processes. * (Set during postcopy) */ -#define RAM_UF_ZEROPAGE (1 << 3) +#define RAM_UF_ZEROPAGE ((QEMU_FORCE QemuMmapFlags) (1 << 3)) /* RAM can be migrated */ -#define RAM_MIGRATABLE (1 << 4) +#define RAM_MIGRATABLE ((QEMU_FORCE QemuMmapFlags) (1 << 4)) /* RAM is a persistent kind memory */ -#define RAM_PMEM (1 << 5) +#define RAM_PMEM ((QEMU_FORCE QemuMmapFlags) (1 << 5)) static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, IOMMUNotifierFlag flags, diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h index 3bf48bc..457d24e 100644 --- a/include/qemu/osdep.h +++ b/include/qemu/osdep.h @@ -185,6 +185,15 @@ extern int daemon(int, int); #define ESHUTDOWN 4099 #endif +#ifdef __CHECKER__ +#define QEMU_BITWISE __attribute__((bitwise)) +#define QEMU_FORCE __attribute__((force)) +#else +#define QEMU_BITWISE +#define QEMU_FORCE +#endif + +typedef unsigned QEMU_BITWISE QemuMmapFlags; /* time_t may be either 32 or 64 bits depending on the host OS, and * can be either signed or unsigned, so we can't just hardcode a * specific maximum value. This is not a C preprocessor constant, From patchwork Wed Jan 16 08:10:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Yi" X-Patchwork-Id: 1025684 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43fg3C0YXSz9sCX for ; Wed, 16 Jan 2019 19:14:15 +1100 (AEDT) Received: from localhost ([127.0.0.1]:44221 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gjgKz-00070X-1A for incoming@patchwork.ozlabs.org; Wed, 16 Jan 2019 03:14:13 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48136) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gjgHc-0004tM-IB for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:10:45 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gjgHa-00073X-EL for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:10:44 -0500 Received: from mga09.intel.com ([134.134.136.24]:18825) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gjgHW-0006yq-Kh for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:10:40 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Jan 2019 00:10:28 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,485,1539673200"; d="scan'208";a="128161263" Received: from linux.intel.com ([10.54.29.200]) by orsmga001.jf.intel.com with ESMTP; 16 Jan 2019 00:10:28 -0800 Received: from dazhang1-ssd.sh.intel.com (dazhang1-ssd.sh.intel.com [10.239.48.128]) by linux.intel.com (Postfix) with ESMTP id ADE875800C6; Wed, 16 Jan 2019 00:10:26 -0800 (PST) From: Zhang Yi To: xiaoguangrong.eric@gmail.com, stefanha@redhat.com, pbonzini@redhat.com, pagupta@redhat.com, yu.c.zhang@linux.intel.com, mst@redhat.com, ehabkost@redhat.com Date: Wed, 16 Jan 2019 16:10:45 +0800 Message-Id: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 134.134.136.24 Subject: [Qemu-devel] [PATCH V9 3/6] util/mmap-alloc: switch 'shared' to 'flags' parameter X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: imammedo@redhat.com, dan.j.williams@intel.com, qemu-devel@nongnu.org, Zhang Yi Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" As more flag parameters besides the existing 'shared' are going to be added to qemu_ram_mmap() and qemu_ram_alloc_from_{file,fd}(), let's switch 'shared' to a 'flags' parameter in advance, so as to ease the further additions. Signed-off-by: Haozhong Zhang Signed-off-by: Zhang Yi --- exec.c | 7 ++++--- include/qemu/mmap-alloc.h | 19 ++++++++++++++++++- util/mmap-alloc.c | 8 +++++--- util/oslib-posix.c | 9 ++++++++- 4 files changed, 35 insertions(+), 8 deletions(-) diff --git a/exec.c b/exec.c index bb6170d..e92a7da 100644 --- a/exec.c +++ b/exec.c @@ -1810,6 +1810,7 @@ static void *file_ram_alloc(RAMBlock *block, ram_addr_t memory, int fd, bool truncate, + uint32_t flags, Error **errp) { void *area; @@ -1859,8 +1860,7 @@ static void *file_ram_alloc(RAMBlock *block, perror("ftruncate"); } - area = qemu_ram_mmap(fd, memory, block->mr->align, - block->flags & RAM_SHARED); + area = qemu_ram_mmap(fd, memory, block->mr->align, flags); if (area == MAP_FAILED) { error_setg_errno(errp, errno, "unable to map backing store for guest RAM"); @@ -2279,7 +2279,8 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr, new_block->used_length = size; new_block->max_length = size; new_block->flags = ram_flags; - new_block->host = file_ram_alloc(new_block, size, fd, !file_size, errp); + new_block->host = file_ram_alloc(new_block, size, fd, !file_size, + ram_flags, errp); if (!new_block->host) { g_free(new_block); return NULL; diff --git a/include/qemu/mmap-alloc.h b/include/qemu/mmap-alloc.h index 50385e3..6fe6ed4 100644 --- a/include/qemu/mmap-alloc.h +++ b/include/qemu/mmap-alloc.h @@ -7,7 +7,24 @@ size_t qemu_fd_getpagesize(int fd); size_t qemu_mempath_getpagesize(const char *mem_path); -void *qemu_ram_mmap(int fd, size_t size, size_t align, bool shared); +/** + * qemu_ram_mmap: mmap the specified file or device. + * + * Parameters: + * @fd: the file or the device to mmap + * @size: the number of bytes to be mmaped + * @align: if not zero, specify the alignment of the starting mapping address; + * otherwise, the alignment in use will be determined by QEMU. + * @flags: specifies additional properties of the mapping, which can be one or + * bit-or of following values + * - RAM_SHARED: mmap with MAP_SHARED flag + * Other bits are ignored. + * + * Return: + * On success, return a pointer to the mapped area. + * On failure, return MAP_FAILED. + */ +void *qemu_ram_mmap(int fd, size_t size, size_t align, uint32_t flags); void qemu_ram_munmap(void *ptr, size_t size); diff --git a/util/mmap-alloc.c b/util/mmap-alloc.c index fd329ec..8f0a740 100644 --- a/util/mmap-alloc.c +++ b/util/mmap-alloc.c @@ -13,6 +13,7 @@ #include "qemu/osdep.h" #include "qemu/mmap-alloc.h" #include "qemu/host-utils.h" +#include "exec/memory.h" #define HUGETLBFS_MAGIC 0x958458f6 @@ -75,7 +76,7 @@ size_t qemu_mempath_getpagesize(const char *mem_path) return getpagesize(); } -void *qemu_ram_mmap(int fd, size_t size, size_t align, bool shared) +void *qemu_ram_mmap(int fd, size_t size, size_t align, uint32_t flags) { /* * Note: this always allocates at least one extra page of virtual address @@ -92,11 +93,12 @@ void *qemu_ram_mmap(int fd, size_t size, size_t align, bool shared) * anonymous memory is OK. */ int anonfd = fd == -1 || qemu_fd_getpagesize(fd) == getpagesize() ? -1 : fd; - int flags = anonfd == -1 ? MAP_ANONYMOUS : MAP_NORESERVE; - void *ptr = mmap(0, total, PROT_NONE, flags | MAP_PRIVATE, anonfd, 0); + int mmap_flags = anonfd == -1 ? MAP_ANONYMOUS : MAP_NORESERVE; + void *ptr = mmap(0, total, PROT_NONE, mmap_flags | MAP_PRIVATE, anonfd, 0); #else void *ptr = mmap(0, total, PROT_NONE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0); #endif + bool shared = flags & RAM_SHARED; size_t offset; void *ptr1; diff --git a/util/oslib-posix.c b/util/oslib-posix.c index fbd0dc8..75a0171 100644 --- a/util/oslib-posix.c +++ b/util/oslib-posix.c @@ -54,6 +54,7 @@ #endif #include "qemu/mmap-alloc.h" +#include "exec/memory.h" #ifdef CONFIG_DEBUG_STACK_USAGE #include "qemu/error-report.h" @@ -203,7 +204,13 @@ void *qemu_memalign(size_t alignment, size_t size) void *qemu_anon_ram_alloc(size_t size, uint64_t *alignment, bool shared) { size_t align = QEMU_VMALLOC_ALIGN; - void *ptr = qemu_ram_mmap(-1, size, align, shared); + uint32_t flags = 0; + void *ptr; + + if (shared) { + flags = RAM_SHARED; + } + ptr = qemu_ram_mmap(-1, size, align, flags); if (ptr == MAP_FAILED) { return NULL; From patchwork Wed Jan 16 08:10:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Yi" X-Patchwork-Id: 1025682 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43fg206Tn5z9sCX for ; Wed, 16 Jan 2019 19:13:12 +1100 (AEDT) Received: from localhost ([127.0.0.1]:43996 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gjgJy-0006Co-PZ for incoming@patchwork.ozlabs.org; Wed, 16 Jan 2019 03:13:10 -0500 Received: from eggs.gnu.org ([209.51.188.92]:49614) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gjgIY-0005aI-8C for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:11:44 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gjgIV-0007mH-Ff for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:11:42 -0500 Received: from mga01.intel.com ([192.55.52.88]:22157) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gjgIV-00073o-0a for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:11:39 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Jan 2019 00:10:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,485,1539673200"; d="scan'208";a="126254065" Received: from linux.intel.com ([10.54.29.200]) by FMSMGA003.fm.intel.com with ESMTP; 16 Jan 2019 00:10:41 -0800 Received: from dazhang1-ssd.sh.intel.com (dazhang1-ssd.sh.intel.com [10.239.48.128]) by linux.intel.com (Postfix) with ESMTP id D2B155800C6; Wed, 16 Jan 2019 00:10:39 -0800 (PST) From: Zhang Yi To: xiaoguangrong.eric@gmail.com, stefanha@redhat.com, pbonzini@redhat.com, pagupta@redhat.com, yu.c.zhang@linux.intel.com, mst@redhat.com, ehabkost@redhat.com Date: Wed, 16 Jan 2019 16:10:58 +0800 Message-Id: <64bea1ff5f80647cc4592ee94d399d647bdd9862.1547624239.git.yi.z.zhang@linux.intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.88 Subject: [Qemu-devel] [PATCH V9 4/6] util/mmap-alloc: support MAP_SYNC in qemu_ram_mmap() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: imammedo@redhat.com, dan.j.williams@intel.com, qemu-devel@nongnu.org, Zhang Yi Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" When a file supporting DAX is used as vNVDIMM backend, mmap it with MAP_SYNC flag in addition which can ensure file system metadata synced in each guest writes to the backend file, without other QEMU actions (e.g., periodic fsync() by QEMU). Signed-off-by: Haozhong Zhang Signed-off-by: Zhang Yi --- include/qemu/mmap-alloc.h | 1 + include/qemu/osdep.h | 16 ++++++++++++++++ util/mmap-alloc.c | 7 ++++++- 3 files changed, 23 insertions(+), 1 deletion(-) diff --git a/include/qemu/mmap-alloc.h b/include/qemu/mmap-alloc.h index 6fe6ed4..a95d91c 100644 --- a/include/qemu/mmap-alloc.h +++ b/include/qemu/mmap-alloc.h @@ -18,6 +18,7 @@ size_t qemu_mempath_getpagesize(const char *mem_path); * @flags: specifies additional properties of the mapping, which can be one or * bit-or of following values * - RAM_SHARED: mmap with MAP_SHARED flag + * - RAM_PMEM: mmap with MAP_SYNC flag * Other bits are ignored. * * Return: diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h index 457d24e..27a6bfe 100644 --- a/include/qemu/osdep.h +++ b/include/qemu/osdep.h @@ -419,6 +419,22 @@ void qemu_anon_ram_free(void *ptr, size_t size); # define QEMU_VMALLOC_ALIGN getpagesize() #endif +/* + * MAP_SHARED_VALIDATE and MAP_SYNC are introduced in Linux kernel + * 4.15, so they may not be defined when compiling on older kernels. + */ +#ifdef CONFIG_LINUX + +#include + +#ifndef MAP_SYNC +#define MAP_SYNC 0x0 +#endif + +#else /* !CONFIG_LINUX */ +#define MAP_SYNC 0x0 +#endif /* CONFIG_LINUX */ + #ifdef CONFIG_POSIX struct qemu_signalfd_siginfo { uint32_t ssi_signo; /* Signal number */ diff --git a/util/mmap-alloc.c b/util/mmap-alloc.c index 8f0a740..cba961c 100644 --- a/util/mmap-alloc.c +++ b/util/mmap-alloc.c @@ -99,6 +99,8 @@ void *qemu_ram_mmap(int fd, size_t size, size_t align, uint32_t flags) void *ptr = mmap(0, total, PROT_NONE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0); #endif bool shared = flags & RAM_SHARED; + bool is_pmem = flags & RAM_PMEM; + int mmap_xflags = 0; size_t offset; void *ptr1; @@ -109,12 +111,15 @@ void *qemu_ram_mmap(int fd, size_t size, size_t align, uint32_t flags) assert(is_power_of_2(align)); /* Always align to host page size */ assert(align >= getpagesize()); + if (shared && is_pmem) { + mmap_xflags |= MAP_SYNC; + } offset = QEMU_ALIGN_UP((uintptr_t)ptr, align) - (uintptr_t)ptr; ptr1 = mmap(ptr + offset, size, PROT_READ | PROT_WRITE, MAP_FIXED | (fd == -1 ? MAP_ANONYMOUS : 0) | - (shared ? MAP_SHARED : MAP_PRIVATE), + (shared ? MAP_SHARED : MAP_PRIVATE) | mmap_xflags, fd, 0); if (ptr1 == MAP_FAILED) { munmap(ptr, total); From patchwork Wed Jan 16 08:11:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Yi" X-Patchwork-Id: 1025685 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43fg5L3qwpz9sCX for ; Wed, 16 Jan 2019 19:16:05 +1100 (AEDT) Received: from localhost ([127.0.0.1]:44640 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gjgMg-0008T9-FX for incoming@patchwork.ozlabs.org; Wed, 16 Jan 2019 03:15:58 -0500 Received: from eggs.gnu.org ([209.51.188.92]:49845) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gjgIa-0005cE-5N for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:11:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gjgIY-0007wT-BG for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:11:43 -0500 Received: from mga07.intel.com ([134.134.136.100]:31375) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gjgIX-0007Dt-Pk for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:11:42 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Jan 2019 00:10:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,485,1539673200"; d="scan'208";a="138683880" Received: from linux.intel.com ([10.54.29.200]) by fmsmga001.fm.intel.com with ESMTP; 16 Jan 2019 00:10:51 -0800 Received: from dazhang1-ssd.sh.intel.com (dazhang1-ssd.sh.intel.com [10.239.48.128]) by linux.intel.com (Postfix) with ESMTP id 9CB565800C6; Wed, 16 Jan 2019 00:10:49 -0800 (PST) From: Zhang Yi To: xiaoguangrong.eric@gmail.com, stefanha@redhat.com, pbonzini@redhat.com, pagupta@redhat.com, yu.c.zhang@linux.intel.com, mst@redhat.com, ehabkost@redhat.com Date: Wed, 16 Jan 2019 16:11:08 +0800 Message-Id: <9c9dc5f205c0e37ed622388327c10f361353de6e.1547624239.git.yi.z.zhang@linux.intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 134.134.136.100 Subject: [Qemu-devel] [PATCH V9 5/6] hostmem: add more information in error messages X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: imammedo@redhat.com, dan.j.williams@intel.com, qemu-devel@nongnu.org, Zhang Yi Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" When there are multiple memory backends in use, including the object type name and the property name in the error message can help users to locate the error. Signed-off-by: Haozhong Zhang Signed-off-by: Zhang Yi Reviewed-by: Eduardo Habkost Reviewed-by: Stefano Garzarella --- backends/hostmem-file.c | 6 ++++-- backends/hostmem.c | 8 +++++--- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/backends/hostmem-file.c b/backends/hostmem-file.c index e640749..0dd7a90 100644 --- a/backends/hostmem-file.c +++ b/backends/hostmem-file.c @@ -82,7 +82,8 @@ static void set_mem_path(Object *o, const char *str, Error **errp) HostMemoryBackendFile *fb = MEMORY_BACKEND_FILE(o); if (host_memory_backend_mr_inited(backend)) { - error_setg(errp, "cannot change property value"); + error_setg(errp, "cannot change property 'mem-path' of %s", + object_get_typename(o)); return; } g_free(fb->mem_path); @@ -120,7 +121,8 @@ static void file_memory_backend_set_align(Object *o, Visitor *v, uint64_t val; if (host_memory_backend_mr_inited(backend)) { - error_setg(&local_err, "cannot change property value"); + error_setg(&local_err, "cannot change property '%s' of %s", + name, object_get_typename(o)); goto out; } diff --git a/backends/hostmem.c b/backends/hostmem.c index 1a89342..e2bcf9f 100644 --- a/backends/hostmem.c +++ b/backends/hostmem.c @@ -47,7 +47,8 @@ host_memory_backend_set_size(Object *obj, Visitor *v, const char *name, uint64_t value; if (host_memory_backend_mr_inited(backend)) { - error_setg(&local_err, "cannot change property value"); + error_setg(&local_err, "cannot change property %s of %s ", + name, object_get_typename(obj)); goto out; } @@ -56,8 +57,9 @@ host_memory_backend_set_size(Object *obj, Visitor *v, const char *name, goto out; } if (!value) { - error_setg(&local_err, "Property '%s.%s' doesn't take value '%" - PRIu64 "'", object_get_typename(obj), name, value); + error_setg(&local_err, + "property '%s' of %s doesn't take value '%" PRIu64 "'", + name, object_get_typename(obj), value); goto out; } backend->size = value; From patchwork Wed Jan 16 08:11:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Yi" X-Patchwork-Id: 1025683 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43fg2M56lSz9sCX for ; Wed, 16 Jan 2019 19:13:31 +1100 (AEDT) Received: from localhost ([127.0.0.1]:44078 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gjgKH-0006UT-Hb for incoming@patchwork.ozlabs.org; Wed, 16 Jan 2019 03:13:29 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48643) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gjgI2-0005CM-Pn for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:11:11 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gjgHz-0007Pm-GJ for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:11:09 -0500 Received: from mga06.intel.com ([134.134.136.31]:20110) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gjgHx-0007JT-Am for qemu-devel@nongnu.org; Wed, 16 Jan 2019 03:11:07 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Jan 2019 00:11:00 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,485,1539673200"; d="scan'208";a="117115289" Received: from linux.intel.com ([10.54.29.200]) by fmsmga008.fm.intel.com with ESMTP; 16 Jan 2019 00:11:00 -0800 Received: from dazhang1-ssd.sh.intel.com (dazhang1-ssd.sh.intel.com [10.239.48.128]) by linux.intel.com (Postfix) with ESMTP id 1CAEB5800C6; Wed, 16 Jan 2019 00:10:57 -0800 (PST) From: Zhang Yi To: xiaoguangrong.eric@gmail.com, stefanha@redhat.com, pbonzini@redhat.com, pagupta@redhat.com, yu.c.zhang@linux.intel.com, mst@redhat.com, ehabkost@redhat.com Date: Wed, 16 Jan 2019 16:11:16 +0800 Message-Id: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 134.134.136.31 Subject: [Qemu-devel] [PATCH V9 6/6] docs: Added MAP_SYNC documentation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: imammedo@redhat.com, dan.j.williams@intel.com, qemu-devel@nongnu.org, Zhang Yi Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Zhang Yi --- docs/nvdimm.txt | 21 ++++++++++++++++++++- qemu-options.hx | 4 ++++ 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/docs/nvdimm.txt b/docs/nvdimm.txt index 5f158a6..565ba73 100644 --- a/docs/nvdimm.txt +++ b/docs/nvdimm.txt @@ -142,11 +142,30 @@ backend of vNVDIMM: Guest Data Persistence ---------------------- +vNVDIMM is designed and implemented to guarantee the guest data +persistence on the backends even on the host crash and power +failures. However, there are still some requirements and limitations +as explained below. + Though QEMU supports multiple types of vNVDIMM backends on Linux, -currently the only one that can guarantee the guest write persistence +if MAP_SYNC is not supported by the host kernel and the backends, +the only backend that can guarantee the guest write persistence is the device DAX on the real NVDIMM device (e.g., /dev/dax0.0), to which all guest access do not involve any host-side kernel cache. +mmap(2) flag MAP_SYNC is added since Linux kernel 4.15. On such +systems, QEMU can mmap(2) the backend with MAP_SYNC, which can ensure +filesystem metadata consistent even after a system crash or power +failure. Besides the host kernel support, enabling MAP_SYNC in QEMU +also requires: + + - the backend is a file supporting DAX, e.g., a file on an ext4 or + xfs file system mounted with '-o dax', + + - 'share' option of memory-backend-file is 'on'. + + - 'pmem' option of memory-backend-file is 'on' + When using other types of backends, it's suggested to set 'unarmed' option of '-device nvdimm' to 'on', which sets the unarmed flag of the guest NVDIMM region mapping structure. This unarmed flag indicates diff --git a/qemu-options.hx b/qemu-options.hx index 08f8516..545cb8a 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -4002,6 +4002,10 @@ using the SNIA NVM programming model (e.g. Intel NVDIMM). If @option{pmem} is set to 'on', QEMU will take necessary operations to guarantee the persistence of its own writes to @option{mem-path} (e.g. in vNVDIMM label emulation and live migration). +Also, we will map the backend-file with MAP_SYNC flag, which can ensure +the file metadata is in sync to @option{mem-path} even on the host crash +and power failures. MAP_SYNC requires supports from both the host kernel +(since Linux kernel 4.15) and @option{mem-path} (only files supporting DAX). @item -object memory-backend-ram,id=@var{id},merge=@var{on|off},dump=@var{on|off},share=@var{on|off},prealloc=@var{on|off},size=@var{size},host-nodes=@var{host-nodes},policy=@var{default|preferred|bind|interleave}