[{"id":1773322,"web_url":"http://patchwork.ozlabs.org/comment/1773322/","msgid":"","list_archive_url":null,"date":"2017-09-22T07:29:55","subject":"Re: [Qemu-devel] [RFC v4 06/16] virtio-iommu: Register attached\n\tdevices","submitter":{"id":70741,"url":"http://patchwork.ozlabs.org/api/people/70741/","name":"Bharat Bhushan","email":"bharat.bhushan@nxp.com"},"content":"> -----Original Message-----\n> From: Eric Auger [mailto:eric.auger@redhat.com]\n> Sent: Tuesday, September 19, 2017 1:17 PM\n> To: eric.auger.pro@gmail.com; eric.auger@redhat.com;\n> peter.maydell@linaro.org; alex.williamson@redhat.com; mst@redhat.com;\n> qemu-arm@nongnu.org; qemu-devel@nongnu.org; jean-\n> philippe.brucker@arm.com\n> Cc: will.deacon@arm.com; kevin.tian@intel.com; marc.zyngier@arm.com;\n> christoffer.dall@linaro.org; drjones@redhat.com; wei@redhat.com;\n> tn@semihalf.com; Bharat Bhushan ;\n> peterx@redhat.com; linuc.decode@gmail.com\n> Subject: [RFC v4 06/16] virtio-iommu: Register attached devices\n> \n> This patch introduce address space and device internal\n> datatypes. Both are stored in RB trees. The address space\n> owns a list of devices attached to it.\n> \n> It is assumed the devid corresponds to the PCI BDF.\n> \n> Signed-off-by: Eric Auger \n> \n> ---\n> v3 -> v4:\n> - new separate patch\n> ---\n> hw/virtio/trace-events | 4 ++\n> hw/virtio/virtio-iommu.c | 120\n> +++++++++++++++++++++++++++++++++++++++++++++++\n> 2 files changed, 124 insertions(+)\n> \n> diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events\n> index bc65356..74b92d3 100644\n> --- a/hw/virtio/trace-events\n> +++ b/hw/virtio/trace-events\n> @@ -38,3 +38,7 @@ virtio_iommu_map(uint32_t as, uint64_t phys_addr,\n> uint64_t virt_addr, uint64_t s\n> virtio_iommu_unmap(uint32_t as, uint64_t virt_addr, uint64_t size) \"as= %d\n> virt_addr=0x%\"PRIx64\" size=0x%\"PRIx64\n> virtio_iommu_translate(const char *name, uint32_t rid, uint64_t iova, int\n> flag) \"mr=%s rid=%d addr=0x%\"PRIx64\" flag=%d\"\n> virtio_iommu_init_iommu_mr(char *iommu_mr) \"init %s\"\n> +virtio_iommu_get_dev(uint32_t devid) \"Alloc devid=%d\"\n> +virtio_iommu_put_dev(uint32_t devid) \"Free devid=%d\"\n> +virtio_iommu_get_as(uint32_t asid) \"Alloc asid=%d\"\n> +virtio_iommu_put_as(uint32_t asid) \"Free asid=%d\"\n> diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c\n> index f4cb76f..41a4bbc 100644\n> --- a/hw/virtio/virtio-iommu.c\n> +++ b/hw/virtio/virtio-iommu.c\n> @@ -32,15 +32,116 @@\n> #include \"hw/virtio/virtio-bus.h\"\n> #include \"hw/virtio/virtio-access.h\"\n> #include \"hw/virtio/virtio-iommu.h\"\n> +#include \"hw/pci/pci_bus.h\"\n> +#include \"hw/pci/pci.h\"\n> \n> /* Max size */\n> #define VIOMMU_DEFAULT_QUEUE_SIZE 256\n> \n> +typedef struct viommu_as {\n> + uint32_t id;\n> + GTree *mappings;\n> + QLIST_HEAD(, viommu_dev) device_list;\n> +} viommu_as;\n> +\n> +typedef struct viommu_dev {\n> + uint32_t id;\n> + viommu_as *as;\n> + QLIST_ENTRY(viommu_dev) next;\n> + VirtIOIOMMU *viommu;\n> +} viommu_dev;\n> +\n> +typedef struct viommu_interval {\n> + uint64_t low;\n> + uint64_t high;\n> +} viommu_interval;\n> +\n> static inline uint16_t virtio_iommu_get_sid(IOMMUDevice *dev)\n> {\n> return PCI_BUILD_BDF(pci_bus_num(dev->bus), dev->devfn);\n> }\n> \n> +static gint interval_cmp(gconstpointer a, gconstpointer b, gpointer\n> user_data)\n> +{\n> + viommu_interval *inta = (viommu_interval *)a;\n> + viommu_interval *intb = (viommu_interval *)b;\n> +\n> + if (inta->high <= intb->low) {\n> + return -1;\n> + } else if (intb->high <= inta->low) {\n> + return 1;\n> + } else {\n> + return 0;\n> + }\n> +}\n> +\n> +static void virtio_iommu_detach_dev_from_as(viommu_dev *dev)\n> +{\n> + QLIST_REMOVE(dev, next);\n> + dev->as = NULL;\n> +}\n> +\n> +static viommu_dev *virtio_iommu_get_dev(VirtIOIOMMU *s, uint32_t\n> devid)\n> +{\n> + viommu_dev *dev;\n> +\n> + dev = g_tree_lookup(s->devices, GUINT_TO_POINTER(devid));\n> + if (dev) {\n> + return dev;\n> + }\n> + dev = g_malloc0(sizeof(*dev));\n> + dev->id = devid;\n> + dev->viommu = s;\n> + trace_virtio_iommu_get_dev(devid);\n> + g_tree_insert(s->devices, GUINT_TO_POINTER(devid), dev);\n> + return dev;\n> +}\n> +\n> +static void virtio_iommu_put_dev(gpointer data)\n> +{\n> + viommu_dev *dev = (viommu_dev *)data;\n> +\n> + if (dev->as) {\n> + virtio_iommu_detach_dev_from_as(dev);\n> + g_tree_unref(dev->as->mappings);\n> + }\n> +\n> + trace_virtio_iommu_put_dev(dev->id);\n> + g_free(dev);\n> +}\n> +\n> +viommu_as *virtio_iommu_get_as(VirtIOIOMMU *s, uint32_t asid);\n> +viommu_as *virtio_iommu_get_as(VirtIOIOMMU *s, uint32_t asid)\n> +{\n> + viommu_as *as;\n> +\n> + as = g_tree_lookup(s->address_spaces, GUINT_TO_POINTER(asid));\n> + if (as) {\n> + return as;\n> + }\n> + as = g_malloc0(sizeof(*as));\n> + as->id = asid;\n\nI see now you maintain device-list in address-space in your base patches, but I do not see where it is initialized?\nShould not that be initialized here (vfio integration patches initialized here).\n\nThanks\n-Bharat\n\n> + as->mappings = g_tree_new_full((GCompareDataFunc)interval_cmp,\n> + NULL, (GDestroyNotify)g_free,\n> + (GDestroyNotify)g_free);\n> + g_tree_insert(s->address_spaces, GUINT_TO_POINTER(asid), as);\n> + trace_virtio_iommu_get_as(asid);\n> + return as;\n> +}\n> +\n> +static void virtio_iommu_put_as(gpointer data)\n> +{\n> + viommu_as *as = (viommu_as *)data;\n> + viommu_dev *iter, *tmp;\n> +\n> + QLIST_FOREACH_SAFE(iter, &as->device_list, next, tmp) {\n> + virtio_iommu_detach_dev_from_as(iter);\n> + }\n> + g_tree_destroy(as->mappings);\n> + trace_virtio_iommu_put_as(as->id);\n> + g_free(as);\n> +}\n> +\n> static AddressSpace *virtio_iommu_find_add_as(PCIBus *bus, void\n> *opaque,\n> int devfn)\n> {\n> @@ -70,6 +171,8 @@ static AddressSpace\n> *virtio_iommu_find_add_as(PCIBus *bus, void *opaque,\n> sdev->bus = bus;\n> sdev->devfn = devfn;\n> \n> + virtio_iommu_get_dev(s, PCI_BUILD_BDF(pci_bus_num(bus), devfn));\n> +\n> trace_virtio_iommu_init_iommu_mr(name);\n> \n> memory_region_init_iommu(&sdev->iommu_mr, sizeof(sdev-\n> >iommu_mr),\n> @@ -360,6 +463,12 @@ static inline guint as_uint64_hash(gconstpointer v)\n> return (guint)*(const uint64_t *)v;\n> }\n> \n> +static gint int_cmp(gconstpointer a, gconstpointer b, gpointer user_data)\n> +{\n> + uint ua = GPOINTER_TO_UINT(a);\n> + uint ub = GPOINTER_TO_UINT(b);\n> + return (ua > ub) - (ua < ub);\n> +}\n> \n> static void virtio_iommu_device_realize(DeviceState *dev, Error **errp)\n> {\n> @@ -375,17 +484,28 @@ static void\n> virtio_iommu_device_realize(DeviceState *dev, Error **errp)\n> s->config.page_size_mask = TARGET_PAGE_MASK;\n> s->config.input_range.end = -1UL;\n> \n> + qemu_mutex_init(&s->mutex);\n> +\n> memset(s->as_by_bus_num, 0, sizeof(s->as_by_bus_num));\n> s->as_by_busptr = g_hash_table_new_full(as_uint64_hash,\n> as_uint64_equal,\n> g_free, g_free);\n> \n> + s->address_spaces = g_tree_new_full((GCompareDataFunc)int_cmp,\n> + NULL, NULL, virtio_iommu_put_as);\n> + s->devices = g_tree_new_full((GCompareDataFunc)int_cmp,\n> + NULL, NULL, virtio_iommu_put_dev);\n> +\n> virtio_iommu_init_as(s);\n> }\n> \n> static void virtio_iommu_device_unrealize(DeviceState *dev, Error **errp)\n> {\n> VirtIODevice *vdev = VIRTIO_DEVICE(dev);\n> + VirtIOIOMMU *s = VIRTIO_IOMMU(dev);\n> +\n> + g_tree_destroy(s->address_spaces);\n> + g_tree_destroy(s->devices);\n> \n> virtio_cleanup(vdev);\n> }\n> --\n> 2.5.5","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=nxp.com header.i=@nxp.com header.b=\"vH+HjZUW\";\n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=bharat.bhushan@nxp.com; "],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xz4sH4zd9z9sRV\n\tfor ;\n\tFri, 22 Sep 2017 17:30:59 +1000 (AEST)","from localhost ([::1]:57079 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dvIQK-0001Os-T4\n\tfor incoming@patchwork.ozlabs.org; Fri, 22 Sep 2017 03:30:56 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:33195)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dvIPc-0001Hj-Ii\n\tfor qemu-devel@nongnu.org; Fri, 22 Sep 2017 03:30:17 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dvIPX-0006kU-VJ\n\tfor qemu-devel@nongnu.org; Fri, 22 Sep 2017 03:30:12 -0400","from mail-he1eur01on0073.outbound.protection.outlook.com\n\t([104.47.0.73]:20800\n\thelo=EUR01-HE1-obe.outbound.protection.outlook.com)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dvIPP-0006XV-43; Fri, 22 Sep 2017 03:29:59 -0400","from AM5PR0401MB2545.eurprd04.prod.outlook.com (10.169.245.8) by\n\tAM5PR0401MB2547.eurprd04.prod.outlook.com (10.169.245.10) with\n\tMicrosoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id\n\t15.20.77.7; Fri, 22 Sep 2017 07:29:55 +0000","from AM5PR0401MB2545.eurprd04.prod.outlook.com\n\t([fe80::a470:3643:ce8f:6327]) by\n\tAM5PR0401MB2545.eurprd04.prod.outlook.com\n\t([fe80::a470:3643:ce8f:6327%18]) with mapi id 15.20.0077.011;\n\tFri, 22 Sep 2017 07:29:55 +0000"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=IpwpFvb5EuYaTserbLlqaj9YhCofzGHNVxASsAVAO3o=;\n\tb=vH+HjZUWpGoa1BFNA45yTao6Itts4KR1dvtZGE1+IdMkbAmhwqcYXloi9LKQ4DhrFp8tb7NVDCncmzRzBJZcZ7n80tRjouo9imrdvrp+Hz76hsPRwGDxlbXxc9hCGhKWT/Ai5rI6MyV63cXT+r3BrSJjgs+w7JuPUctWx4xs1TI=","From":"Bharat Bhushan ","To":"Eric Auger , \"eric.auger.pro@gmail.com\"\n\t, \"peter.maydell@linaro.org\"\n\t, \"alex.williamson@redhat.com\"\n\t, \"mst@redhat.com\" ,\n\t\"qemu-arm@nongnu.org\" , \"qemu-devel@nongnu.org\"\n\t, \"jean-philippe.brucker@arm.com\"\n\t","Thread-Topic":"[RFC v4 06/16] virtio-iommu: Register attached devices","Thread-Index":"AQHTMRumtxy7OYfQbUSfg0b0w0wpQKLAg7Cw","Date":"Fri, 22 Sep 2017 07:29:55 +0000","Message-ID":"","References":"<1505807208-9063-1-git-send-email-eric.auger@redhat.com>\n\t<1505807208-9063-7-git-send-email-eric.auger@redhat.com>","In-Reply-To":"<1505807208-9063-7-git-send-email-eric.auger@redhat.com>","Accept-Language":"en-US","Content-Language":"en-US","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","authentication-results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=nxp.com header.i=@nxp.com header.b=\"vH+HjZUW\";\n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=bharat.bhushan@nxp.com; "],"x-originating-ip":"[192.88.169.1]","x-ms-publictraffictype":"Email","x-microsoft-exchange-diagnostics":"1; AM5PR0401MB2547;\n\t6:vmylrv48DHDtbXh/U9fOZO2ReqebbcRQNTIr/2pD8HF9FBovffQrQPCg/VLO09KcbWSz3g4lnTBci4+5xfPqQEZyQTKwBMMCcSn6XcG/mj/ajOhkYncWP+Zn+VGYgOonFBOT6H2UjW9uIuiaAjvuYXS3audLYk18amrGliQ/C3zlsutDMRlDd2dEBS8QmMFW5ZOJN2nwNaaBqxPqKKozlNkaxzlqrC9wzOrYykeBlrYMr4RGKdawvHbmZw8zMdIACUSH3nmXGfRvXm+HDkxlsgFyG5EEpt3FbuWa4pB+JjcxRmiKyLVul3QmWHcD+jbzgGn4IIk5l9i5bh335LJvOA==;\n\t5:8AEOQh44bfZoiwJ4mbj/AQ6C6AcH+vrl3r9cpNw65dIql6WYaF5jkDTnUkiVy5HhLsuqHrABaEiWA1aS/SIzOMkyI7Y7WSD+3EZaboaYKCy0IcQQP0qTRKcZECgbbx6uibQ8Z3v7Nj2rPfE/4K18sg==;\n\t24:/4Cks49nROHpQWBOI1FrTD7oZS8VQGc41KHHImETrGCVOPA8/qQ7wYyKlR6rqgukE7qKmQfJx2S5glOr4zWT1ejTvQFFTcyIqiDI26r+Qrw=;\n\t7:1buTUNdoc+tSVBUNYl417NVRg0zbsTl1YlQamF0M5Do5sUv/PeR06xgx58O6e9sxK+15SV3W/uvmJtj8UWJLgvdL32FMEsvG7/hFK5ceOOTFaqYZsMCz1wiPfP1pNILFHm1wIzTjGeq/p/z5JMnIlEZ0UOMvD++MtEyflUUxzN+0ZUKPxafqwl2nu8TTgjCj1r1+d7U/C8Y3DnbFDfYthUEFFCMTOwRQUq10NGOvyoU=","x-ms-exchange-antispam-srfa-diagnostics":"SSOS;","x-ms-office365-filtering-correlation-id":"be81beb5-30dd-408c-4b99-08d5018bb8e3","x-ms-office365-filtering-ht":"Tenant","x-microsoft-antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(48565401081)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:AM5PR0401MB2547; ","x-ms-traffictypediagnostic":"AM5PR0401MB2547:","x-exchange-antispam-report-test":"UriScan:(180628864354917)(185117386973197)(228905959029699); ","x-microsoft-antispam-prvs":"","x-exchange-antispam-report-cfa-test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(3002001)(10201501046)(100000703101)(100105400095)(93006095)(93001095)(6055026)(6041248)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123560025)(20161123562025)(20161123555025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:AM5PR0401MB2547; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:AM5PR0401MB2547; ","x-forefront-prvs":"0438F90F17","x-forefront-antispam-report":"SFV:NSPM;\n\tSFS:(10009020)(6009001)(376002)(39860400002)(346002)(13464003)(377454003)(199003)(189002)(81156014)(6116002)(102836003)(189998001)(110136005)(99286003)(25786009)(8936002)(2201001)(97736004)(53546010)(2950100002)(316002)(3280700002)(14454004)(74316002)(8676002)(68736007)(3846002)(6436002)(81166006)(3660700001)(55016002)(53936002)(54906003)(66066001)(6506006)(4326008)(7696004)(9686003)(54356999)(7736002)(50986999)(305945005)(2501003)(7416002)(86362001)(101416001)(229853002)(5660300001)(33656002)(39060400002)(5250100002)(478600001)(2900100001)(106356001)(2906002)(6246003)(105586002)(5890100001)(76176999);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR0401MB2547;\n\tH:AM5PR0401MB2545.eurprd04.prod.outlook.com; FPR:; SPF:None;\n\tPTR:InfoNoRecords; A:1; MX:1; LANG:en; ","received-spf":"None (protection.outlook.com: nxp.com does not designate\n\tpermitted sender hosts)","spamdiagnosticoutput":"1:99","spamdiagnosticmetadata":"NSPM","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"quoted-printable","MIME-Version":"1.0","X-OriginatorOrg":"nxp.com","X-MS-Exchange-CrossTenant-originalarrivaltime":"22 Sep 2017 07:29:55.5841\n\t(UTC)","X-MS-Exchange-CrossTenant-fromentityheader":"Hosted","X-MS-Exchange-CrossTenant-id":"686ea1d3-bc2b-4c6f-a92c-d99c5c301635","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"AM5PR0401MB2547","X-detected-operating-system":"by eggs.gnu.org: Windows 7 or 8 [fuzzy]","X-Received-From":"104.47.0.73","Subject":"Re: [Qemu-devel] [RFC v4 06/16] virtio-iommu: Register attached\n\tdevices","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"\"wei@redhat.com\" ,\n\t\"kevin.tian@intel.com\" ,\n\t\"marc.zyngier@arm.com\" ,\n\t\"tn@semihalf.com\" ,\n\t\"will.deacon@arm.com\" ,\n\t\"drjones@redhat.com\" ,\n\t\"peterx@redhat.com\" ,\n\t\"linuc.decode@gmail.com\" ,\n\t\"christoffer.dall@linaro.org\" ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}}]