From patchwork Thu Dec 7 14:58:16 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Borntraeger X-Patchwork-Id: 845630 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yszC74G2lz9t3F for ; Fri, 8 Dec 2017 01:58:59 +1100 (AEDT) Received: from localhost ([::1]:32778 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eMxdZ-0003b4-Mi for incoming@patchwork.ozlabs.org; Thu, 07 Dec 2017 09:58:57 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33601) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eMxd6-0003Zt-NH for qemu-devel@nongnu.org; Thu, 07 Dec 2017 09:58:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eMxd3-0004f6-Hf for qemu-devel@nongnu.org; Thu, 07 Dec 2017 09:58:28 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:43514 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eMxd3-0004ec-Aj for qemu-devel@nongnu.org; Thu, 07 Dec 2017 09:58:25 -0500 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id vB7EtBch066366 for ; Thu, 7 Dec 2017 09:58:22 -0500 Received: from e06smtp11.uk.ibm.com (e06smtp11.uk.ibm.com [195.75.94.107]) by mx0b-001b2d01.pphosted.com with ESMTP id 2eq4b33wvq-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Thu, 07 Dec 2017 09:58:22 -0500 Received: from localhost by e06smtp11.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 7 Dec 2017 14:58:20 -0000 Received: from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196) by e06smtp11.uk.ibm.com (192.168.101.141) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 7 Dec 2017 14:58:17 -0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id vB7EwGGE44957906; Thu, 7 Dec 2017 14:58:16 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 679CB11C05B; Thu, 7 Dec 2017 14:52:42 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5595811C04A; Thu, 7 Dec 2017 14:52:42 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTPS; Thu, 7 Dec 2017 14:52:42 +0000 (GMT) Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 25651) id 63E4920F649; Thu, 7 Dec 2017 15:58:16 +0100 (CET) From: Christian Borntraeger To: Cornelia Huck Date: Thu, 7 Dec 2017 15:58:16 +0100 X-Mailer: git-send-email 2.9.4 X-TM-AS-GCONF: 00 x-cbid: 17120714-0040-0000-0000-0000041779EB X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17120714-0041-0000-0000-000020BA7983 Message-Id: <20171207145816.87347-1-borntraeger@de.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-12-07_06:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1712070220 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 Subject: [Qemu-devel] [PATCH/RFC v2] s390x: start a new memory region if the old one exceeds KVM_MEM_MAX_NR_PAGES X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Graf , Thomas Huth , David Hildenbrand , Halil Pasic , qemu-devel , Christian Borntraeger , qemu-s390x , Richard Henderson Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" KVM does not allow memory regions > KVM_MEM_MAX_NR_PAGES, basically limiting the memory per slot to 8TB-4k. Lets start a new memory region if we cross that boundary. With that (and optimistic overcommitment in the kernel) I was able to start a 24TB guest on a 1TB system. Signed-off-by: Christian Borntraeger --- hw/s390x/s390-virtio-ccw.c | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-) diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index 8425534..3630f6a 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -154,14 +154,41 @@ static void virtio_ccw_register_hcalls(void) virtio_ccw_hcall_early_printk); } +/* + * KVM does only support memory slots up to KVM_MEM_MAX_NR_PAGES pages + * as the dirty bitmap must be managed by bitops that take an int as + * position indicator. If we have a guest beyond that we will split off + * new subregions. + */ +#define KVM_SLOT_MAX ((((1UL << 31) - 1) * 4096) & ~0xfffffUL ) + static void s390_memory_init(ram_addr_t mem_size) { MemoryRegion *sysmem = get_system_memory(); - MemoryRegion *ram = g_new(MemoryRegion, 1); + ram_addr_t chunk, offset; + unsigned int number; + gchar *name; /* allocate RAM for core */ - memory_region_allocate_system_memory(ram, NULL, "s390.ram", mem_size); - memory_region_add_subregion(sysmem, 0, ram); + offset = 0; + number = 0; + name = g_strdup_printf("s390.ram"); + while (mem_size) { + MemoryRegion *ram = g_new(MemoryRegion, 1); + chunk = mem_size; + /* KVM does not allow memslots >= 8 TB */ + if (chunk > KVM_SLOT_MAX) { + chunk = KVM_SLOT_MAX; + } + memory_region_allocate_system_memory(ram, NULL, name, chunk); + memory_region_add_subregion(sysmem, offset, ram); + mem_size -= chunk; + offset += chunk; + number++; + g_free(name); + name = g_strdup_printf("s390.ram.%u", number); + } + g_free(name); /* Initialize storage key device */ s390_skeys_init();