From patchwork Wed May 30 11:45:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 922741 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40wplM0BFHz9s08 for ; Wed, 30 May 2018 21:48:46 +1000 (AEST) Received: from localhost ([::1]:37762 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fNzau-0006SU-8C for incoming@patchwork.ozlabs.org; Wed, 30 May 2018 07:48:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49223) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fNzYH-0004m5-NJ for qemu-devel@nongnu.org; Wed, 30 May 2018 07:46:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fNzYG-00059f-Ti for qemu-devel@nongnu.org; Wed, 30 May 2018 07:46:01 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:38270 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fNzYC-00056o-BQ; Wed, 30 May 2018 07:45:56 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2FE465BCBE; Wed, 30 May 2018 11:45:55 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-69.ams2.redhat.com [10.36.116.69]) by smtp.corp.redhat.com (Postfix) with ESMTP id 993D610EE95B; Wed, 30 May 2018 11:45:53 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Wed, 30 May 2018 13:45:34 +0200 Message-Id: <1527680741-2725-2-git-send-email-eric.auger@redhat.com> In-Reply-To: <1527680741-2725-1-git-send-email-eric.auger@redhat.com> References: <1527680741-2725-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 30 May 2018 11:45:55 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 30 May 2018 11:45:55 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'eric.auger@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v3 1/8] linux-headers: Partial update for KVM/ARM multiple redistributor region registration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: marc.zyngier@arm.com, wei@redhat.com, drjones@redhat.com, christoffer.dall@arm.com, zhaoshenglong@huawei.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This updates KVM/ARM headers against https://github.com/eauger/linux/tree/v4.17-rc2-rdist-regions-v6 Signed-off-by: Eric Auger --- linux-headers/asm-arm/kvm.h | 1 + linux-headers/asm-arm64/kvm.h | 1 + 2 files changed, 2 insertions(+) diff --git a/linux-headers/asm-arm/kvm.h b/linux-headers/asm-arm/kvm.h index 4392955..81ae4e5 100644 --- a/linux-headers/asm-arm/kvm.h +++ b/linux-headers/asm-arm/kvm.h @@ -91,6 +91,7 @@ struct kvm_regs { #define KVM_VGIC_V3_ADDR_TYPE_DIST 2 #define KVM_VGIC_V3_ADDR_TYPE_REDIST 3 #define KVM_VGIC_ITS_ADDR_TYPE 4 +#define KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION 5 #define KVM_VGIC_V3_DIST_SIZE SZ_64K #define KVM_VGIC_V3_REDIST_SIZE (2 * SZ_64K) diff --git a/linux-headers/asm-arm64/kvm.h b/linux-headers/asm-arm64/kvm.h index 4e80651..d41f39a 100644 --- a/linux-headers/asm-arm64/kvm.h +++ b/linux-headers/asm-arm64/kvm.h @@ -91,6 +91,7 @@ struct kvm_regs { #define KVM_VGIC_V3_ADDR_TYPE_DIST 2 #define KVM_VGIC_V3_ADDR_TYPE_REDIST 3 #define KVM_VGIC_ITS_ADDR_TYPE 4 +#define KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION 5 #define KVM_VGIC_V3_DIST_SIZE SZ_64K #define KVM_VGIC_V3_REDIST_SIZE (2 * SZ_64K) From patchwork Wed May 30 11:45:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 922744 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40wppq2XQpz9s08 for ; Wed, 30 May 2018 21:51:47 +1000 (AEST) Received: from localhost ([::1]:37782 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fNzdo-0000hI-N6 for incoming@patchwork.ozlabs.org; Wed, 30 May 2018 07:51:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49325) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fNzYO-0004t8-8B for qemu-devel@nongnu.org; Wed, 30 May 2018 07:46:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fNzYM-0005DL-32 for qemu-devel@nongnu.org; Wed, 30 May 2018 07:46:08 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:38280 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fNzYD-00057a-Hs; Wed, 30 May 2018 07:45:57 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 02534BB405; Wed, 30 May 2018 11:45:57 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-69.ams2.redhat.com [10.36.116.69]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6A88C10EE95B; Wed, 30 May 2018 11:45:55 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Wed, 30 May 2018 13:45:35 +0200 Message-Id: <1527680741-2725-3-git-send-email-eric.auger@redhat.com> In-Reply-To: <1527680741-2725-1-git-send-email-eric.auger@redhat.com> References: <1527680741-2725-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 30 May 2018 11:45:57 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 30 May 2018 11:45:57 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'eric.auger@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v3 2/8] target/arm: Allow KVM device address overwriting X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: marc.zyngier@arm.com, wei@redhat.com, drjones@redhat.com, christoffer.dall@arm.com, zhaoshenglong@huawei.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" for KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION attribute, the attribute data pointed to by kvm_device_attr.addr is a OR of the redistributor region address and other fields such as the index of the redistributor region and the number of redistributors the region can contain. The existing machine init done notifier framework sets the address field to the actual address of the device and does not allow to OR this value with other fields. This patch extends the KVMDevice struct with a new kda_addr_ormask member. Its value is passed at registration time and OR'ed with the resolved address on kvm_arm_set_device_addr(). Signed-off-by: Eric Auger Reviewed-by: Peter Maydell --- v2 -> v3: - s/addr_fixup/add_ormask - Added Peter's R-b --- hw/intc/arm_gic_kvm.c | 4 ++-- hw/intc/arm_gicv3_its_kvm.c | 2 +- hw/intc/arm_gicv3_kvm.c | 4 ++-- target/arm/kvm.c | 10 +++++++++- target/arm/kvm_arm.h | 3 ++- 5 files changed, 16 insertions(+), 7 deletions(-) diff --git a/hw/intc/arm_gic_kvm.c b/hw/intc/arm_gic_kvm.c index 6f467e6..eb9664e 100644 --- a/hw/intc/arm_gic_kvm.c +++ b/hw/intc/arm_gic_kvm.c @@ -558,7 +558,7 @@ static void kvm_arm_gic_realize(DeviceState *dev, Error **errp) | KVM_VGIC_V2_ADDR_TYPE_DIST, KVM_DEV_ARM_VGIC_GRP_ADDR, KVM_VGIC_V2_ADDR_TYPE_DIST, - s->dev_fd); + s->dev_fd, 0); /* CPU interface for current core. Unlike arm_gic, we don't * provide the "interface for core #N" memory regions, because * cores with a VGIC don't have those. @@ -568,7 +568,7 @@ static void kvm_arm_gic_realize(DeviceState *dev, Error **errp) | KVM_VGIC_V2_ADDR_TYPE_CPU, KVM_DEV_ARM_VGIC_GRP_ADDR, KVM_VGIC_V2_ADDR_TYPE_CPU, - s->dev_fd); + s->dev_fd, 0); if (kvm_has_gsi_routing()) { /* set up irq routing */ diff --git a/hw/intc/arm_gicv3_its_kvm.c b/hw/intc/arm_gicv3_its_kvm.c index eea6a73..271ebe4 100644 --- a/hw/intc/arm_gicv3_its_kvm.c +++ b/hw/intc/arm_gicv3_its_kvm.c @@ -103,7 +103,7 @@ static void kvm_arm_its_realize(DeviceState *dev, Error **errp) /* register the base address */ kvm_arm_register_device(&s->iomem_its_cntrl, -1, KVM_DEV_ARM_VGIC_GRP_ADDR, - KVM_VGIC_ITS_ADDR_TYPE, s->dev_fd); + KVM_VGIC_ITS_ADDR_TYPE, s->dev_fd, 0); gicv3_its_init_mmio(s, NULL); diff --git a/hw/intc/arm_gicv3_kvm.c b/hw/intc/arm_gicv3_kvm.c index ec37177..93ac293 100644 --- a/hw/intc/arm_gicv3_kvm.c +++ b/hw/intc/arm_gicv3_kvm.c @@ -754,9 +754,9 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp) KVM_DEV_ARM_VGIC_CTRL_INIT, NULL, true, &error_abort); kvm_arm_register_device(&s->iomem_dist, -1, KVM_DEV_ARM_VGIC_GRP_ADDR, - KVM_VGIC_V3_ADDR_TYPE_DIST, s->dev_fd); + KVM_VGIC_V3_ADDR_TYPE_DIST, s->dev_fd, 0); kvm_arm_register_device(&s->iomem_redist, -1, KVM_DEV_ARM_VGIC_GRP_ADDR, - KVM_VGIC_V3_ADDR_TYPE_REDIST, s->dev_fd); + KVM_VGIC_V3_ADDR_TYPE_REDIST, s->dev_fd, 0); if (kvm_has_gsi_routing()) { /* set up irq routing */ diff --git a/target/arm/kvm.c b/target/arm/kvm.c index 5141d0a..6a4324a 100644 --- a/target/arm/kvm.c +++ b/target/arm/kvm.c @@ -184,10 +184,15 @@ unsigned long kvm_arch_vcpu_id(CPUState *cpu) * We use a MemoryListener to track mapping and unmapping of * the regions during board creation, so the board models don't * need to do anything special for the KVM case. + * + * Sometimes the address must be OR'ed with some other fields + * (for example for KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION). + * @kda_addr_ormask aims at storing the value of those fields. */ typedef struct KVMDevice { struct kvm_arm_device_addr kda; struct kvm_device_attr kdattr; + uint64_t kda_addr_ormask; MemoryRegion *mr; QSLIST_ENTRY(KVMDevice) entries; int dev_fd; @@ -234,6 +239,8 @@ static void kvm_arm_set_device_addr(KVMDevice *kd) */ if (kd->dev_fd >= 0) { uint64_t addr = kd->kda.addr; + + addr |= kd->kda_addr_ormask; attr->addr = (uintptr_t)&addr; ret = kvm_device_ioctl(kd->dev_fd, KVM_SET_DEVICE_ATTR, attr); } else { @@ -266,7 +273,7 @@ static Notifier notify = { }; void kvm_arm_register_device(MemoryRegion *mr, uint64_t devid, uint64_t group, - uint64_t attr, int dev_fd) + uint64_t attr, int dev_fd, uint64_t addr_ormask) { KVMDevice *kd; @@ -286,6 +293,7 @@ void kvm_arm_register_device(MemoryRegion *mr, uint64_t devid, uint64_t group, kd->kdattr.group = group; kd->kdattr.attr = attr; kd->dev_fd = dev_fd; + kd->kda_addr_ormask = addr_ormask; QSLIST_INSERT_HEAD(&kvm_devices_head, kd, entries); memory_region_ref(kd->mr); } diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h index 1e23640..863f205 100644 --- a/target/arm/kvm_arm.h +++ b/target/arm/kvm_arm.h @@ -34,6 +34,7 @@ int kvm_arm_vcpu_init(CPUState *cs); * @group: device control API group for setting addresses * @attr: device control API address type * @dev_fd: device control device file descriptor (or -1 if not supported) + * @addr_ormask: value to be OR'ed with resolved address * * Remember the memory region @mr, and when it is mapped by the * machine model, tell the kernel that base address using the @@ -45,7 +46,7 @@ int kvm_arm_vcpu_init(CPUState *cs); * address at the point where machine init is complete. */ void kvm_arm_register_device(MemoryRegion *mr, uint64_t devid, uint64_t group, - uint64_t attr, int dev_fd); + uint64_t attr, int dev_fd, uint64_t addr_ormask); /** * kvm_arm_init_cpreg_list: From patchwork Wed May 30 11:45:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 922745 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40wpq02GmBz9s1b for ; Wed, 30 May 2018 21:51:54 +1000 (AEST) Received: from localhost ([::1]:37783 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fNzdv-0000nI-G3 for incoming@patchwork.ozlabs.org; Wed, 30 May 2018 07:51:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49279) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fNzYK-0004pS-UU for qemu-devel@nongnu.org; Wed, 30 May 2018 07:46:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fNzYJ-0005BQ-Hh for qemu-devel@nongnu.org; Wed, 30 May 2018 07:46:04 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:48416 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fNzYG-000595-AJ; Wed, 30 May 2018 07:46:00 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BB8058163AD4; Wed, 30 May 2018 11:45:59 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-69.ams2.redhat.com [10.36.116.69]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3948A10EE95B; Wed, 30 May 2018 11:45:57 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Wed, 30 May 2018 13:45:36 +0200 Message-Id: <1527680741-2725-4-git-send-email-eric.auger@redhat.com> In-Reply-To: <1527680741-2725-1-git-send-email-eric.auger@redhat.com> References: <1527680741-2725-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 30 May 2018 11:45:59 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 30 May 2018 11:45:59 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'eric.auger@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v3 3/8] hw/intc/arm_gicv3: Introduce redist-region-count array property X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: marc.zyngier@arm.com, wei@redhat.com, drjones@redhat.com, christoffer.dall@arm.com, zhaoshenglong@huawei.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" To prepare for multiple redistributor regions, we introduce an array of uint32_t properties that stores the redistributor count of each redistributor region. Non accelerated VGICv3 only supports a single redistributor region. The capacity of all redist regions is checked against the number of vcpus. Machvirt is updated to set those properties, ie. a single redistributor region with count set to the number of vcpus capped by 123. Signed-off-by: Eric Auger --- v2 -> v3: - add missing return in arm_gic_realize - in gicv3_init_irqs_and_mmio, compute/check rdist_capacity first - rdist region 0 size set to MIN(smp_cpus, redist0_capacity) - add GICV3_REDIST_SIZE --- hw/arm/virt.c | 11 ++++++++++- hw/intc/arm_gicv3.c | 12 +++++++++++- hw/intc/arm_gicv3_common.c | 38 +++++++++++++++++++++++++++++++++----- hw/intc/arm_gicv3_kvm.c | 9 +++++++-- include/hw/intc/arm_gicv3_common.h | 8 ++++++-- 5 files changed, 67 insertions(+), 11 deletions(-) diff --git a/hw/arm/virt.c b/hw/arm/virt.c index a3a28e2..ed79460 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -523,6 +523,15 @@ static void create_gic(VirtMachineState *vms, qemu_irq *pic) if (!kvm_irqchip_in_kernel()) { qdev_prop_set_bit(gicdev, "has-security-extensions", vms->secure); } + + if (type == 3) { + uint32_t redist0_capacity = + vms->memmap[VIRT_GIC_REDIST].size / GICV3_REDIST_SIZE; + uint32_t redist0_count = MIN(smp_cpus, redist0_capacity); + + qdev_prop_set_uint32(gicdev, "len-redist-region-count", 1); + qdev_prop_set_uint32(gicdev, "redist-region-count[0]", redist0_count); + } qdev_init_nofail(gicdev); gicbusdev = SYS_BUS_DEVICE(gicdev); sysbus_mmio_map(gicbusdev, 0, vms->memmap[VIRT_GIC_DIST].base); @@ -1322,7 +1331,7 @@ static void machvirt_init(MachineState *machine) * many redistributors we can fit into the memory map. */ if (vms->gic_version == 3) { - virt_max_cpus = vms->memmap[VIRT_GIC_REDIST].size / 0x20000; + virt_max_cpus = vms->memmap[VIRT_GIC_REDIST].size / GICV3_REDIST_SIZE; } else { virt_max_cpus = GIC_NCPU; } diff --git a/hw/intc/arm_gicv3.c b/hw/intc/arm_gicv3.c index 479c667..7044133 100644 --- a/hw/intc/arm_gicv3.c +++ b/hw/intc/arm_gicv3.c @@ -373,7 +373,17 @@ static void arm_gic_realize(DeviceState *dev, Error **errp) return; } - gicv3_init_irqs_and_mmio(s, gicv3_set_irq, gic_ops); + if (s->nb_redist_regions != 1) { + error_setg(errp, "VGICv3 redist region number(%d) not equal to 1", + s->nb_redist_regions); + return; + } + + gicv3_init_irqs_and_mmio(s, gicv3_set_irq, gic_ops, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } gicv3_init_cpuif(s); } diff --git a/hw/intc/arm_gicv3_common.c b/hw/intc/arm_gicv3_common.c index 7b54d52..4c89e7d 100644 --- a/hw/intc/arm_gicv3_common.c +++ b/hw/intc/arm_gicv3_common.c @@ -169,11 +169,22 @@ static const VMStateDescription vmstate_gicv3 = { }; void gicv3_init_irqs_and_mmio(GICv3State *s, qemu_irq_handler handler, - const MemoryRegionOps *ops) + const MemoryRegionOps *ops, Error **errp) { SysBusDevice *sbd = SYS_BUS_DEVICE(s); + int rdist_capacity = 0; int i; + for (i = 0; i < s->nb_redist_regions; i++) { + rdist_capacity += s->redist_region_count[i]; + } + if (rdist_capacity < s->num_cpu) { + error_setg(errp, "Capacity of the redist regions(%d) " + "is less than number of vcpus(%d)", + rdist_capacity, s->num_cpu); + return; + } + /* For the GIC, also expose incoming GPIO lines for PPIs for each CPU. * GPIO array layout is thus: * [0..N-1] spi @@ -199,11 +210,18 @@ void gicv3_init_irqs_and_mmio(GICv3State *s, qemu_irq_handler handler, memory_region_init_io(&s->iomem_dist, OBJECT(s), ops, s, "gicv3_dist", 0x10000); - memory_region_init_io(&s->iomem_redist, OBJECT(s), ops ? &ops[1] : NULL, s, - "gicv3_redist", 0x20000 * s->num_cpu); - sysbus_init_mmio(sbd, &s->iomem_dist); - sysbus_init_mmio(sbd, &s->iomem_redist); + + s->iomem_redist = g_new0(MemoryRegion, s->nb_redist_regions); + for (i = 0; i < s->nb_redist_regions; i++) { + char *name = g_strdup_printf("gicv3_redist_region[%d]", i); + + memory_region_init_io(&s->iomem_redist[i], OBJECT(s), + ops ? &ops[1] : NULL, s, name, + s->redist_region_count[i] * GICV3_REDIST_SIZE); + sysbus_init_mmio(sbd, &s->iomem_redist[i]); + g_free(name); + } } static void arm_gicv3_common_realize(DeviceState *dev, Error **errp) @@ -285,6 +303,13 @@ static void arm_gicv3_common_realize(DeviceState *dev, Error **errp) } } +static void arm_gicv3_finalize(Object *obj) +{ + GICv3State *s = ARM_GICV3_COMMON(obj); + + g_free(s->redist_region_count); +} + static void arm_gicv3_common_reset(DeviceState *dev) { GICv3State *s = ARM_GICV3_COMMON(dev); @@ -388,6 +413,8 @@ static Property arm_gicv3_common_properties[] = { DEFINE_PROP_UINT32("num-irq", GICv3State, num_irq, 32), DEFINE_PROP_UINT32("revision", GICv3State, revision, 3), DEFINE_PROP_BOOL("has-security-extensions", GICv3State, security_extn, 0), + DEFINE_PROP_ARRAY("redist-region-count", GICv3State, nb_redist_regions, + redist_region_count, qdev_prop_uint32, uint32_t), DEFINE_PROP_END_OF_LIST(), }; @@ -409,6 +436,7 @@ static const TypeInfo arm_gicv3_common_type = { .instance_size = sizeof(GICv3State), .class_size = sizeof(ARMGICv3CommonClass), .class_init = arm_gicv3_common_class_init, + .instance_finalize = arm_gicv3_finalize, .abstract = true, .interfaces = (InterfaceInfo []) { { TYPE_ARM_LINUX_BOOT_IF }, diff --git a/hw/intc/arm_gicv3_kvm.c b/hw/intc/arm_gicv3_kvm.c index 93ac293..7e76b87 100644 --- a/hw/intc/arm_gicv3_kvm.c +++ b/hw/intc/arm_gicv3_kvm.c @@ -731,7 +731,11 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp) return; } - gicv3_init_irqs_and_mmio(s, kvm_arm_gicv3_set_irq, NULL); + gicv3_init_irqs_and_mmio(s, kvm_arm_gicv3_set_irq, NULL, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } for (i = 0; i < s->num_cpu; i++) { ARMCPU *cpu = ARM_CPU(qemu_get_cpu(i)); @@ -755,7 +759,8 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp) kvm_arm_register_device(&s->iomem_dist, -1, KVM_DEV_ARM_VGIC_GRP_ADDR, KVM_VGIC_V3_ADDR_TYPE_DIST, s->dev_fd, 0); - kvm_arm_register_device(&s->iomem_redist, -1, KVM_DEV_ARM_VGIC_GRP_ADDR, + kvm_arm_register_device(&s->iomem_redist[0], -1, + KVM_DEV_ARM_VGIC_GRP_ADDR, KVM_VGIC_V3_ADDR_TYPE_REDIST, s->dev_fd, 0); if (kvm_has_gsi_routing()) { diff --git a/include/hw/intc/arm_gicv3_common.h b/include/hw/intc/arm_gicv3_common.h index bccdfe1..31971d7 100644 --- a/include/hw/intc/arm_gicv3_common.h +++ b/include/hw/intc/arm_gicv3_common.h @@ -35,6 +35,8 @@ #define GICV3_MAXIRQ 1020 #define GICV3_MAXSPI (GICV3_MAXIRQ - GIC_INTERNAL) +#define GICV3_REDIST_SIZE 0x20000 + /* Number of SGI target-list bits */ #define GICV3_TARGETLIST_BITS 16 @@ -210,7 +212,9 @@ struct GICv3State { /*< public >*/ MemoryRegion iomem_dist; /* Distributor */ - MemoryRegion iomem_redist; /* Redistributors */ + MemoryRegion *iomem_redist; /* Redistributor Regions */ + uint32_t *redist_region_count; /* redistributor count within each region */ + uint32_t nb_redist_regions; /* number of redist regions */ uint32_t num_cpu; uint32_t num_irq; @@ -291,6 +295,6 @@ typedef struct ARMGICv3CommonClass { } ARMGICv3CommonClass; void gicv3_init_irqs_and_mmio(GICv3State *s, qemu_irq_handler handler, - const MemoryRegionOps *ops); + const MemoryRegionOps *ops, Error **errp); #endif From patchwork Wed May 30 11:45:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 922742 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40wplX2bY5z9s1w for ; Wed, 30 May 2018 21:48:56 +1000 (AEST) Received: from localhost ([::1]:37763 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fNzb3-0006a0-TK for incoming@patchwork.ozlabs.org; Wed, 30 May 2018 07:48:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49282) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fNzYL-0004pp-BN for qemu-devel@nongnu.org; Wed, 30 May 2018 07:46:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fNzYK-0005C3-Bk for qemu-devel@nongnu.org; Wed, 30 May 2018 07:46:05 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:38288 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fNzYI-0005AO-0h; Wed, 30 May 2018 07:46:02 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 888E15BCBE; Wed, 30 May 2018 11:46:01 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-69.ams2.redhat.com [10.36.116.69]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0099E10EE6C1; Wed, 30 May 2018 11:45:59 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Wed, 30 May 2018 13:45:37 +0200 Message-Id: <1527680741-2725-5-git-send-email-eric.auger@redhat.com> In-Reply-To: <1527680741-2725-1-git-send-email-eric.auger@redhat.com> References: <1527680741-2725-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 30 May 2018 11:46:01 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 30 May 2018 11:46:01 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'eric.auger@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v3 4/8] hw/intc/arm_gicv3_kvm: Get prepared to handle multiple redist regions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: marc.zyngier@arm.com, wei@redhat.com, drjones@redhat.com, christoffer.dall@arm.com, zhaoshenglong@huawei.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Let's check if KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION is supported. If not, we check the number of redist region is equal to 1 and use the legacy KVM_VGIC_V3_ADDR_TYPE_REDIST attribute. Otherwise we use the new attribute and allow to register multiple regions to the KVM device. Signed-off-by: Eric Auger Reviewed-by: Peter Maydell --- v2 -> v3: - In kvm_arm_gicv3_realize rename val into add_ormask local variable and add a comment - start the redist region registration from s->nb_redist_regions - 1 downwards --- hw/intc/arm_gicv3_kvm.c | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-) diff --git a/hw/intc/arm_gicv3_kvm.c b/hw/intc/arm_gicv3_kvm.c index 7e76b87..3826ff4 100644 --- a/hw/intc/arm_gicv3_kvm.c +++ b/hw/intc/arm_gicv3_kvm.c @@ -714,6 +714,7 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp) { GICv3State *s = KVM_ARM_GICV3(dev); KVMARMGICv3Class *kgc = KVM_ARM_GICV3_GET_CLASS(s); + bool multiple_redist_region_allowed; Error *local_err = NULL; int i; @@ -750,6 +751,18 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp) return; } + multiple_redist_region_allowed = + kvm_device_check_attr(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ADDR, + KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION); + + if (!multiple_redist_region_allowed && s->nb_redist_regions > 1) { + error_setg(errp, "Multiple VGICv3 redistributor regions are not " + "supported by this host kernel"); + error_append_hint(errp, "A maximum of %d VCPUs can be used", + s->redist_region_count[0]); + return; + } + kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS, 0, &s->num_irq, true, &error_abort); @@ -759,9 +772,23 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp) kvm_arm_register_device(&s->iomem_dist, -1, KVM_DEV_ARM_VGIC_GRP_ADDR, KVM_VGIC_V3_ADDR_TYPE_DIST, s->dev_fd, 0); - kvm_arm_register_device(&s->iomem_redist[0], -1, - KVM_DEV_ARM_VGIC_GRP_ADDR, - KVM_VGIC_V3_ADDR_TYPE_REDIST, s->dev_fd, 0); + + if (!multiple_redist_region_allowed) { + kvm_arm_register_device(&s->iomem_redist[0], -1, + KVM_DEV_ARM_VGIC_GRP_ADDR, + KVM_VGIC_V3_ADDR_TYPE_REDIST, s->dev_fd, 0); + } else { + for (i = s->nb_redist_regions - 1; i >= 0; i--) { + /* Address mask made of the rdist region index and count */ + uint64_t addr_ormask = + i | ((uint64_t)s->redist_region_count[i] << 52); + + kvm_arm_register_device(&s->iomem_redist[i], -1, + KVM_DEV_ARM_VGIC_GRP_ADDR, + KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION, + s->dev_fd, addr_ormask); + } + } if (kvm_has_gsi_routing()) { /* set up irq routing */ From patchwork Wed May 30 11:45:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 922746 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40wps54y5wz9s08 for ; Wed, 30 May 2018 21:53:45 +1000 (AEST) Received: from localhost ([::1]:37801 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fNzfj-0002O7-EF for incoming@patchwork.ozlabs.org; Wed, 30 May 2018 07:53:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49348) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fNzYP-0004uf-FT for qemu-devel@nongnu.org; Wed, 30 May 2018 07:46:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fNzYO-0005EU-E5 for qemu-devel@nongnu.org; Wed, 30 May 2018 07:46:09 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:38296 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fNzYJ-0005BR-QF; Wed, 30 May 2018 07:46:03 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 54887BB405; Wed, 30 May 2018 11:46:03 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-69.ams2.redhat.com [10.36.116.69]) by smtp.corp.redhat.com (Postfix) with ESMTP id C1B3810EE6C1; Wed, 30 May 2018 11:46:01 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Wed, 30 May 2018 13:45:38 +0200 Message-Id: <1527680741-2725-6-git-send-email-eric.auger@redhat.com> In-Reply-To: <1527680741-2725-1-git-send-email-eric.auger@redhat.com> References: <1527680741-2725-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 30 May 2018 11:46:03 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 30 May 2018 11:46:03 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'eric.auger@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v3 5/8] hw/arm/virt: GICv3 DT node with one or two redistributor regions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: marc.zyngier@arm.com, wei@redhat.com, drjones@redhat.com, christoffer.dall@arm.com, zhaoshenglong@huawei.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch allows the creation of a GICv3 node with 1 or 2 redistributor regions depending on the number of smu_cpus. The second redistributor region is located just after the existing RAM region, at 256GB and contains up to up to 512 vcpus. Please refer to kernel documentation for further node details: Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt Signed-off-by: Eric Auger --- v2 -> v3: - VIRT_GIC_REDIST2 is now 64MB large, ie. 512 redistributor capacity - virt_gicv3_redist_region_count does not test kvm_irqchip_in_kernel anymore --- hw/arm/virt.c | 29 ++++++++++++++++++++++++----- include/hw/arm/virt.h | 14 ++++++++++++++ 2 files changed, 38 insertions(+), 5 deletions(-) diff --git a/hw/arm/virt.c b/hw/arm/virt.c index ed79460..3018ec2 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -149,6 +149,8 @@ static const MemMapEntry a15memmap[] = { [VIRT_PCIE_PIO] = { 0x3eff0000, 0x00010000 }, [VIRT_PCIE_ECAM] = { 0x3f000000, 0x01000000 }, [VIRT_MEM] = { 0x40000000, RAMLIMIT_BYTES }, + /* Additional 64 MB redist region (can contain up to 512 redistributors) */ + [VIRT_GIC_REDIST2] = { 0x4000000000ULL, 0x4000000ULL }, /* Second PCIe window, 512GB wide at the 512GB boundary */ [VIRT_PCIE_MMIO_HIGH] = { 0x8000000000ULL, 0x8000000000ULL }, }; @@ -402,13 +404,30 @@ static void fdt_add_gic_node(VirtMachineState *vms) qemu_fdt_setprop_cell(vms->fdt, "/intc", "#size-cells", 0x2); qemu_fdt_setprop(vms->fdt, "/intc", "ranges", NULL, 0); if (vms->gic_version == 3) { + int nb_redist_regions = virt_gicv3_redist_region_count(vms); + qemu_fdt_setprop_string(vms->fdt, "/intc", "compatible", "arm,gic-v3"); - qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg", - 2, vms->memmap[VIRT_GIC_DIST].base, - 2, vms->memmap[VIRT_GIC_DIST].size, - 2, vms->memmap[VIRT_GIC_REDIST].base, - 2, vms->memmap[VIRT_GIC_REDIST].size); + + qemu_fdt_setprop_cell(vms->fdt, "/intc", + "#redistributor-regions", nb_redist_regions); + + if (nb_redist_regions == 1) { + qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg", + 2, vms->memmap[VIRT_GIC_DIST].base, + 2, vms->memmap[VIRT_GIC_DIST].size, + 2, vms->memmap[VIRT_GIC_REDIST].base, + 2, vms->memmap[VIRT_GIC_REDIST].size); + } else { + qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg", + 2, vms->memmap[VIRT_GIC_DIST].base, + 2, vms->memmap[VIRT_GIC_DIST].size, + 2, vms->memmap[VIRT_GIC_REDIST].base, + 2, vms->memmap[VIRT_GIC_REDIST].size, + 2, vms->memmap[VIRT_GIC_REDIST2].base, + 2, vms->memmap[VIRT_GIC_REDIST2].size); + } + if (vms->virt) { qemu_fdt_setprop_cells(vms->fdt, "/intc", "interrupts", GIC_FDT_IRQ_TYPE_PPI, ARCH_GICV3_MAINT_IRQ, diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h index 4ac7ef6..308156f 100644 --- a/include/hw/arm/virt.h +++ b/include/hw/arm/virt.h @@ -35,6 +35,8 @@ #include "qemu/notify.h" #include "hw/boards.h" #include "hw/arm/arm.h" +#include "sysemu/kvm.h" +#include "hw/intc/arm_gicv3_common.h" #define NUM_GICV2M_SPIS 64 #define NUM_VIRTIO_TRANSPORTS 32 @@ -60,6 +62,7 @@ enum { VIRT_GIC_V2M, VIRT_GIC_ITS, VIRT_GIC_REDIST, + VIRT_GIC_REDIST2, VIRT_SMMU, VIRT_UART, VIRT_MMIO, @@ -130,4 +133,15 @@ typedef struct { void virt_acpi_setup(VirtMachineState *vms); +/* Return the number of used redistributor regions */ +static inline int virt_gicv3_redist_region_count(VirtMachineState *vms) +{ + uint32_t redist0_capacity = + vms->memmap[VIRT_GIC_REDIST].size / GICV3_REDIST_SIZE; + + assert(vms->gic_version == 3); + + return vms->smp_cpus > redist0_capacity ? 2 : 1; +} + #endif /* QEMU_ARM_VIRT_H */ From patchwork Wed May 30 11:45:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 922739 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40wpjh0rlcz9s1b for ; Wed, 30 May 2018 21:47:20 +1000 (AEST) Received: from localhost ([::1]:37753 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fNzZV-0005J5-Na for incoming@patchwork.ozlabs.org; Wed, 30 May 2018 07:47:17 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49357) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fNzYQ-0004w7-52 for qemu-devel@nongnu.org; Wed, 30 May 2018 07:46:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fNzYP-0005Ew-C5 for qemu-devel@nongnu.org; Wed, 30 May 2018 07:46:10 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:38306 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fNzYL-0005Cb-IG; Wed, 30 May 2018 07:46:05 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1FEA5BB40D; Wed, 30 May 2018 11:46:05 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-69.ams2.redhat.com [10.36.116.69]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8C0D010EE6C1; Wed, 30 May 2018 11:46:03 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Wed, 30 May 2018 13:45:39 +0200 Message-Id: <1527680741-2725-7-git-send-email-eric.auger@redhat.com> In-Reply-To: <1527680741-2725-1-git-send-email-eric.auger@redhat.com> References: <1527680741-2725-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 30 May 2018 11:46:05 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 30 May 2018 11:46:05 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'eric.auger@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v3 6/8] hw/arm/virt-acpi-build: Advertise one or two GICR structures X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: marc.zyngier@arm.com, wei@redhat.com, drjones@redhat.com, christoffer.dall@arm.com, zhaoshenglong@huawei.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Depending on the number of smp_cpus we now register one or two GICR structures. Signed-off-by: Eric Auger --- hw/arm/virt-acpi-build.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index 92ceee9..6a4340a 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -660,6 +660,7 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) if (vms->gic_version == 3) { AcpiMadtGenericTranslator *gic_its; + int nb_redist_regions = virt_gicv3_redist_region_count(vms); AcpiMadtGenericRedistributor *gicr = acpi_data_push(table_data, sizeof *gicr); @@ -668,6 +669,14 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) gicr->base_address = cpu_to_le64(memmap[VIRT_GIC_REDIST].base); gicr->range_length = cpu_to_le32(memmap[VIRT_GIC_REDIST].size); + if (nb_redist_regions == 2) { + gicr = acpi_data_push(table_data, sizeof(*gicr)); + gicr->type = ACPI_APIC_GENERIC_REDISTRIBUTOR; + gicr->length = sizeof(*gicr); + gicr->base_address = cpu_to_le64(memmap[VIRT_GIC_REDIST2].base); + gicr->range_length = cpu_to_le32(memmap[VIRT_GIC_REDIST2].size); + } + if (its_class_name() && !vmc->no_its) { gic_its = acpi_data_push(table_data, sizeof *gic_its); gic_its->type = ACPI_APIC_GENERIC_TRANSLATOR; From patchwork Wed May 30 11:45:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 922743 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40wpn209Hkz9s08 for ; Wed, 30 May 2018 21:50:14 +1000 (AEST) Received: from localhost ([::1]:37769 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fNzcJ-0007kx-Jm for incoming@patchwork.ozlabs.org; Wed, 30 May 2018 07:50:11 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49417) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fNzYY-00051r-8g for qemu-devel@nongnu.org; Wed, 30 May 2018 07:46:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fNzYX-0005Oh-FI for qemu-devel@nongnu.org; Wed, 30 May 2018 07:46:18 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:41420 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fNzYS-0005IP-8E; Wed, 30 May 2018 07:46:12 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B68284000B6E; Wed, 30 May 2018 11:46:11 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-69.ams2.redhat.com [10.36.116.69]) by smtp.corp.redhat.com (Postfix) with ESMTP id 58E4B10EE6C1; Wed, 30 May 2018 11:46:05 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Wed, 30 May 2018 13:45:40 +0200 Message-Id: <1527680741-2725-8-git-send-email-eric.auger@redhat.com> In-Reply-To: <1527680741-2725-1-git-send-email-eric.auger@redhat.com> References: <1527680741-2725-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Wed, 30 May 2018 11:46:11 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Wed, 30 May 2018 11:46:11 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'eric.auger@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v3 7/8] hw/arm/virt: Register two redistributor regions when necessary X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: marc.zyngier@arm.com, wei@redhat.com, drjones@redhat.com, christoffer.dall@arm.com, zhaoshenglong@huawei.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" With a VGICv3 KVM device, if the number of vcpus exceeds the capacity of the legacy redistributor region (123 redistributors), we now attempt to register a second redistributor region. Up to 512 redistributors can fit in this latter on top of the 123 allowed by the legacy redistributor region. Registering this second redistributor region is possible if the host kernel supports the following VGICv3 KVM device group/attribute: KVM_DEV_ARM_VGIC_GRP_ADDR/KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION. In case the host kernel does not support the registration of several redistributor regions and the requested number of vcpus exceeds the capacity of the legacy redistributor region, the GICv3 device initialization fails with a proper error message and qemu exits. At the moment the max number of vcpus still is capped by the virt machine class max_cpus. Signed-off-by: Eric Auger --- v2 -> v3: - remove spare space --- hw/arm/virt.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 3018ec2..c00f47d 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -529,6 +529,7 @@ static void create_gic(VirtMachineState *vms, qemu_irq *pic) SysBusDevice *gicbusdev; const char *gictype; int type = vms->gic_version, i; + uint32_t nb_redist_regions = 0; gictype = (type == 3) ? gicv3_class_name() : gic_class_name(); @@ -548,14 +549,28 @@ static void create_gic(VirtMachineState *vms, qemu_irq *pic) vms->memmap[VIRT_GIC_REDIST].size / GICV3_REDIST_SIZE; uint32_t redist0_count = MIN(smp_cpus, redist0_capacity); - qdev_prop_set_uint32(gicdev, "len-redist-region-count", 1); + nb_redist_regions = virt_gicv3_redist_region_count(vms); + + qdev_prop_set_uint32(gicdev, "len-redist-region-count", + nb_redist_regions); qdev_prop_set_uint32(gicdev, "redist-region-count[0]", redist0_count); + + if (nb_redist_regions == 2) { + uint32_t redist1_capacity = + vms->memmap[VIRT_GIC_REDIST2].size / GICV3_REDIST_SIZE; + + qdev_prop_set_uint32(gicdev, "redist-region-count[1]", + MIN(smp_cpus - redist0_count, redist1_capacity)); + } } qdev_init_nofail(gicdev); gicbusdev = SYS_BUS_DEVICE(gicdev); sysbus_mmio_map(gicbusdev, 0, vms->memmap[VIRT_GIC_DIST].base); if (type == 3) { sysbus_mmio_map(gicbusdev, 1, vms->memmap[VIRT_GIC_REDIST].base); + if (nb_redist_regions == 2) { + sysbus_mmio_map(gicbusdev, 2, vms->memmap[VIRT_GIC_REDIST2].base); + } } else { sysbus_mmio_map(gicbusdev, 1, vms->memmap[VIRT_GIC_CPU].base); } @@ -1351,6 +1366,7 @@ static void machvirt_init(MachineState *machine) */ if (vms->gic_version == 3) { virt_max_cpus = vms->memmap[VIRT_GIC_REDIST].size / GICV3_REDIST_SIZE; + virt_max_cpus += vms->memmap[VIRT_GIC_REDIST2].size / GICV3_REDIST_SIZE; } else { virt_max_cpus = GIC_NCPU; } From patchwork Wed May 30 11:45:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 922747 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40wpsR1tzDz9s08 for ; Wed, 30 May 2018 21:54:03 +1000 (AEST) Received: from localhost ([::1]:37805 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fNzfw-0002Zd-Tc for incoming@patchwork.ozlabs.org; Wed, 30 May 2018 07:53:56 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49441) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fNzYa-00053m-3s for qemu-devel@nongnu.org; Wed, 30 May 2018 07:46:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fNzYZ-0005Pe-5L for qemu-devel@nongnu.org; Wed, 30 May 2018 07:46:20 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:39830 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fNzYU-0005MY-Em; Wed, 30 May 2018 07:46:14 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EB0E8402242D; Wed, 30 May 2018 11:46:13 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-69.ams2.redhat.com [10.36.116.69]) by smtp.corp.redhat.com (Postfix) with ESMTP id EEA6A10EE6C1; Wed, 30 May 2018 11:46:11 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Wed, 30 May 2018 13:45:41 +0200 Message-Id: <1527680741-2725-9-git-send-email-eric.auger@redhat.com> In-Reply-To: <1527680741-2725-1-git-send-email-eric.auger@redhat.com> References: <1527680741-2725-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 30 May 2018 11:46:14 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 30 May 2018 11:46:14 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'eric.auger@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v3 8/8] hw/arm/virt: Add virt-3.0 machine type supporting up to 512 vcpus X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: marc.zyngier@arm.com, wei@redhat.com, drjones@redhat.com, christoffer.dall@arm.com, zhaoshenglong@huawei.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Add virt-3.0 machine type. This machine type allows up to 512 vcpus whereas for earlier machine types, max_cpus was set to 255 and any attempt to start the machine with vcpus > 255 was rejected at vl.c/main level. Signed-off-by: Eric Auger --- hw/arm/virt.c | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-) diff --git a/hw/arm/virt.c b/hw/arm/virt.c index c00f47d..a9fc24b 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -1697,11 +1697,13 @@ static void virt_machine_class_init(ObjectClass *oc, void *data) HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc); mc->init = machvirt_init; - /* Start max_cpus at the maximum QEMU supports. We'll further restrict - * it later in machvirt_init, where we have more information about the - * configuration of the particular instance. + /* Start with max_cpus set to 512. This value is chosen since achievable + * in accelerated mode with GICv3 and recent host supporting up to 512 vcpus + * and multiple redistributor region registration. + * This value will be refined later on once we collect more information + * about the configuration of the particular instance. */ - mc->max_cpus = 255; + mc->max_cpus = 512; machine_class_allow_dynamic_sysbus_dev(mc, TYPE_VFIO_CALXEDA_XGMAC); machine_class_allow_dynamic_sysbus_dev(mc, TYPE_VFIO_AMD_XGBE); mc->block_default_type = IF_VIRTIO; @@ -1737,7 +1739,7 @@ static void machvirt_machine_init(void) } type_init(machvirt_machine_init); -static void virt_2_12_instance_init(Object *obj) +static void virt_3_0_instance_init(Object *obj) { VirtMachineState *vms = VIRT_MACHINE(obj); VirtMachineClass *vmc = VIRT_MACHINE_GET_CLASS(vms); @@ -1805,10 +1807,26 @@ static void virt_2_12_instance_init(Object *obj) vms->irqmap = a15irqmap; } -static void virt_machine_2_12_options(MachineClass *mc) +static void virt_machine_3_0_options(MachineClass *mc) { } -DEFINE_VIRT_MACHINE_AS_LATEST(2, 12) +DEFINE_VIRT_MACHINE_AS_LATEST(3, 0) + +#define VIRT_COMPAT_2_12 \ + HW_COMPAT_2_12 + +static void virt_2_12_instance_init(Object *obj) +{ + virt_3_0_instance_init(obj); +} + +static void virt_machine_2_12_options(MachineClass *mc) +{ + virt_machine_3_0_options(mc); + SET_MACHINE_COMPAT(mc, VIRT_COMPAT_2_12); + mc->max_cpus = 255; +} +DEFINE_VIRT_MACHINE(2, 12) #define VIRT_COMPAT_2_11 \ HW_COMPAT_2_11