From patchwork Thu Jun 29 08:40:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 1801417 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=CSabYyBt; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QsC284mTmz20XS for ; Thu, 29 Jun 2023 18:54:56 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qEnPM-0002IS-MM; Thu, 29 Jun 2023 04:53:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qEnPK-0002Hv-NI for qemu-devel@nongnu.org; Thu, 29 Jun 2023 04:53:42 -0400 Received: from mga04.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qEnPI-0003dp-KB for qemu-devel@nongnu.org; Thu, 29 Jun 2023 04:53:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1688028820; x=1719564820; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cJIPVP01AA7xsD3iUU2rztqJwjACTSIZKHRcfCmA+tw=; b=CSabYyBtWiPASluonNN4ZB2d4Y+XuP8Yp3qJAfBsVDP3Za9AcT5ygn0D E+CJD0FwjMaQDTcM3P+2azc1rguR1mTk62tns5UDG4ebO74UhPKCJ6mSY yYeboas++W4hdMC6iwFwJM8xWLzOrDYehmoKoVr6SMz6zxJSNBeVM/Z4u +kr37ymG5t36kkPbktagTDGN6BGf295Osm0m6AJWMkT1uM9Fi0wAHqYed yy1fhjJ6jJKH5nqbuKUKjyL6SKAzWbbm3FARdQC0I/R6qgjh4PxzJKUxK IIA0lz3VsIwDwy3KHLJMGoLQM0NvFaoCuCceq0hwTbv3+m4kP+qGqCAwV g==; X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="360908850" X-IronPort-AV: E=Sophos;i="6.01,168,1684825200"; d="scan'208";a="360908850" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jun 2023 01:53:39 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="720494306" X-IronPort-AV: E=Sophos;i="6.01,168,1684825200"; d="scan'208";a="720494306" Received: from duan-server-s2600bt.bj.intel.com ([10.240.192.147]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jun 2023 01:53:37 -0700 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, joao.m.martins@oracle.com, avihaih@nvidia.com, chao.p.peng@intel.com Subject: [PATCH v4 1/5] vfio/pci: Fix a segfault in vfio_realize Date: Thu, 29 Jun 2023 16:40:38 +0800 Message-Id: <20230629084042.86502-2-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230629084042.86502-1-zhenzhong.duan@intel.com> References: <20230629084042.86502-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mga04.intel.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The kvm irqchip notifier is only registered if the device supports INTx, however it's unconditionally removed in vfio realize error path. If the assigned device does not support INTx, this will cause QEMU to crash when vfio realize fails. Change it to conditionally remove the notifier only if the notify hook is setup. Before fix: (qemu) device_add vfio-pci,host=81:11.1,id=vfio1,bus=root1,xres=1 Connection closed by foreign host. After fix: (qemu) device_add vfio-pci,host=81:11.1,id=vfio1,bus=root1,xres=1 Error: vfio 0000:81:11.1: xres and yres properties require display=on (qemu) Fixes: c5478fea27ac ("vfio/pci: Respond to KVM irqchip change notifier") Signed-off-by: Zhenzhong Duan Reviewed-by: Joao Martins Reviewed-by: Cédric Le Goater --- hw/vfio/pci.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 73e19a04b2bf..48df517f79ee 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3221,7 +3221,9 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) out_deregister: pci_device_set_intx_routing_notifier(&vdev->pdev, NULL); - kvm_irqchip_remove_change_notifier(&vdev->irqchip_change_notifier); + if (vdev->irqchip_change_notifier.notify) { + kvm_irqchip_remove_change_notifier(&vdev->irqchip_change_notifier); + } out_teardown: vfio_teardown_msi(vdev); vfio_bars_exit(vdev); From patchwork Thu Jun 29 08:40:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 1801418 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=co7LqUfu; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QsC2G07yjz20XS for ; Thu, 29 Jun 2023 18:55:02 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qEnPO-0002JL-Hf; Thu, 29 Jun 2023 04:53:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qEnPM-0002Il-P1 for qemu-devel@nongnu.org; Thu, 29 Jun 2023 04:53:44 -0400 Received: from mga04.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qEnPL-0003fe-3M for qemu-devel@nongnu.org; Thu, 29 Jun 2023 04:53:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1688028823; x=1719564823; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=M9sOthnyrGqoEEYhYVxWzaWHTpau5NyEbkjO3zdN17c=; b=co7LqUfuzEyo5NO/tIUIhtWYN+YVnXyvbkSCBrV3DM3WodXSUyKgLMg+ twzYALzQGmiR1w6/3+fGfPO0cH0xehLhNt7b5bv1HrIgVugaFpTVBPxci Mf/OG4Fo1M0OVkggkTAgJG43sxkX09GGQJrTZkNQT6QC/7rCXBcOuwCf0 V5A+ReitBNEMdED5Kyy64J/A7D8TLJ2g5tRgSxc3a4FNJpX7qe+E/GQfc 2AnfRaQ9SAyq0HyENzd3EnsRoz/sJdasOobxIVMt5eTUuyS5LEKhpevOM Ug6z8iyYIAEa0QZr/0yUEdo4mYUqc3n0P5jkei7P/6Sh//4SCpNYT8ZI5 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="360908856" X-IronPort-AV: E=Sophos;i="6.01,168,1684825200"; d="scan'208";a="360908856" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jun 2023 01:53:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="720494337" X-IronPort-AV: E=Sophos;i="6.01,168,1684825200"; d="scan'208";a="720494337" Received: from duan-server-s2600bt.bj.intel.com ([10.240.192.147]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jun 2023 01:53:39 -0700 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, joao.m.martins@oracle.com, avihaih@nvidia.com, chao.p.peng@intel.com Subject: [PATCH v4 2/5] vfio/pci: Free leaked timer in vfio_realize error path Date: Thu, 29 Jun 2023 16:40:39 +0800 Message-Id: <20230629084042.86502-3-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230629084042.86502-1-zhenzhong.duan@intel.com> References: <20230629084042.86502-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mga04.intel.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org When vfio_realize fails, the mmap_timer used for INTx optimization isn't freed. As this timer isn't activated yet, the potential impact is just a piece of leaked memory. Fixes: ea486926b07d ("vfio-pci: Update slow path INTx algorithm timer related") Signed-off-by: Zhenzhong Duan Reviewed-by: Joao Martins Reviewed-by: Cédric Le Goater --- hw/vfio/pci.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 48df517f79ee..ab6645ba60af 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3224,6 +3224,9 @@ out_deregister: if (vdev->irqchip_change_notifier.notify) { kvm_irqchip_remove_change_notifier(&vdev->irqchip_change_notifier); } + if (vdev->intx.mmap_timer) { + timer_free(vdev->intx.mmap_timer); + } out_teardown: vfio_teardown_msi(vdev); vfio_bars_exit(vdev); From patchwork Thu Jun 29 08:40:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 1801413 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=lCmpDxxi; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QsC1Z70NJz20XS for ; Thu, 29 Jun 2023 18:54:26 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qEnPQ-0002Ja-AH; Thu, 29 Jun 2023 04:53:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qEnPO-0002JO-H1 for qemu-devel@nongnu.org; Thu, 29 Jun 2023 04:53:46 -0400 Received: from mga04.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qEnPM-0003gN-Uf for qemu-devel@nongnu.org; Thu, 29 Jun 2023 04:53:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1688028824; x=1719564824; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=auMMZ44FpdF9pO57M72BIAIa9/kWifdFwydoJYidDdk=; b=lCmpDxxiqA3j5Av2pkajhAEyD53mViXP1XMrBtFTw7cyQbl8Brp7gl2F VkFnDtjknEyKNvriTJuHRoC5Nfp6xdsXYPIWYtes1D0MJ9RyzA3BO0fhv wgvBSEoqePRPNntQo5nqJIJNk8XaklY/8pF/d+eW1aUfvt0GffcQFIVhD N90EdQbEPtGGl4lIPPxXQYeJ4HjFUABVIxKBiZOCI57BXy5nfft/jvzZw 6+bJEp6Ku7oCilOL1DdGadElLpfXnRqbo4INXR71M0+Mx/HaUq3iFgIxL Hr1ZaJEoF5ztfByob2ovNArCbT59iZqH8tmKJy1LLHpFAGfyjF8yFtO+o w==; X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="360908865" X-IronPort-AV: E=Sophos;i="6.01,168,1684825200"; d="scan'208";a="360908865" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jun 2023 01:53:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="720494349" X-IronPort-AV: E=Sophos;i="6.01,168,1684825200"; d="scan'208";a="720494349" Received: from duan-server-s2600bt.bj.intel.com ([10.240.192.147]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jun 2023 01:53:41 -0700 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, joao.m.martins@oracle.com, avihaih@nvidia.com, chao.p.peng@intel.com Subject: [PATCH v4 3/5] vfio/pci: Disable INTx in vfio_realize error path Date: Thu, 29 Jun 2023 16:40:40 +0800 Message-Id: <20230629084042.86502-4-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230629084042.86502-1-zhenzhong.duan@intel.com> References: <20230629084042.86502-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mga04.intel.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org When vfio realize fails, INTx isn't disabled if it has been enabled. This may confuse host side with unhandled interrupt report. Add a new label to be used for vfio_intx_enable() failed case. Fixes: a9994687cb9b ("vfio/display: core & wireup") Fixes: b290659fc3dd ("hw/vfio/display: add ramfb support") Fixes: c62a0c7ce34e ("vfio/display: add xres + yres properties") Signed-off-by: Zhenzhong Duan --- hw/vfio/pci.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index ab6645ba60af..54a8179d1c64 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3167,7 +3167,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) kvm_irqchip_add_change_notifier(&vdev->irqchip_change_notifier); ret = vfio_intx_enable(vdev, errp); if (ret) { - goto out_deregister; + goto out_intx_disable; } } @@ -3220,6 +3220,8 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) return; out_deregister: + vfio_disable_interrupts(vdev); +out_intx_disable: pci_device_set_intx_routing_notifier(&vdev->pdev, NULL); if (vdev->irqchip_change_notifier.notify) { kvm_irqchip_remove_change_notifier(&vdev->irqchip_change_notifier); From patchwork Thu Jun 29 08:40:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 1801414 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=a+ERvJf+; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QsC1g64LWz20XS for ; Thu, 29 Jun 2023 18:54:31 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qEnPS-0002K5-IJ; Thu, 29 Jun 2023 04:53:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qEnPQ-0002Jb-72 for qemu-devel@nongnu.org; Thu, 29 Jun 2023 04:53:48 -0400 Received: from mga04.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qEnPO-0003fe-Fh for qemu-devel@nongnu.org; Thu, 29 Jun 2023 04:53:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1688028826; x=1719564826; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=G5HFVlLCJ61xkjz00WCRqTvVJSqyMqfJRHBOqW6i9Z0=; b=a+ERvJf+7k8HZtqWKH6yln1qaTYbuZfdzurCGwSX0xhh6JQNR9dreyPC 0O8QAXlxN0DsMhMHHoXHx9/i827O9NiV79qoPBW/JBGKSQdEyfcUjWeDt 4HgEf9Gar9TWgxcxIc2ZaGkzYbRrt8TZWhVeA3RtkJBFq+sCB5/vV9opR Xozu4ET1vCLwyz2+nRt/wEEhlDyiYZTbWp3pHt90Bor6YxrqgtuaQjr// gwEyxhplT80g7ECDIkL7PY20FWUaEcJ6nO0tLx6/bQ/KhTxwwQapUmo1L UtutA3xRWdtUVJcawFSk6qoV/ROSUhsN0Jpwgyl6zHwTcC1yLSEYCn1bw Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="360908873" X-IronPort-AV: E=Sophos;i="6.01,168,1684825200"; d="scan'208";a="360908873" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jun 2023 01:53:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="720494362" X-IronPort-AV: E=Sophos;i="6.01,168,1684825200"; d="scan'208";a="720494362" Received: from duan-server-s2600bt.bj.intel.com ([10.240.192.147]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jun 2023 01:53:43 -0700 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, joao.m.martins@oracle.com, avihaih@nvidia.com, chao.p.peng@intel.com Subject: [PATCH v4 4/5] vfio/pci: Free resources when vfio_migration_realize fails Date: Thu, 29 Jun 2023 16:40:41 +0800 Message-Id: <20230629084042.86502-5-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230629084042.86502-1-zhenzhong.duan@intel.com> References: <20230629084042.86502-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mga04.intel.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org When vfio_realize() succeeds, hot unplug will call vfio_exitfn() to free resources allocated in vfio_realize(); when vfio_realize() fails, vfio_exitfn() is never called and we need to free resources in vfio_realize(). In the case that vfio_migration_realize() fails, e.g: with -only-migratable & enable-migration=off, we see below: (qemu) device_add vfio-pci,host=81:11.1,id=vfio1,bus=root1,enable-migration=off 0000:81:11.1: Migration disabled Error: disallowing migration blocker (--only-migratable) for: 0000:81:11.1: Migration is disabled for VFIO device If we hotplug again we should see same log as above, but we see: (qemu) device_add vfio-pci,host=81:11.1,id=vfio1,bus=root1,enable-migration=off Error: vfio 0000:81:11.1: device is already attached That's because some references to VFIO device isn't released, we should check return value of vfio_migration_realize() and release the references, then VFIO device will be truely released when hotplug fails. Fixes: a22651053b59 ("vfio: Make vfio-pci device migration capable") Signed-off-by: Zhenzhong Duan --- hw/vfio/pci.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 54a8179d1c64..dc69d3031b24 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3210,6 +3210,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) ret = vfio_migration_realize(vbasedev, errp); if (ret) { error_report("%s: Migration disabled", vbasedev->name); + goto out_vfio_migration; } } @@ -3219,6 +3220,8 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) return; +out_vfio_migration: + vfio_migration_exit(vbasedev); out_deregister: vfio_disable_interrupts(vdev); out_intx_disable: From patchwork Thu Jun 29 08:40:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 1801415 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=Kfkr3F2M; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QsC2407mdz20XS for ; Thu, 29 Jun 2023 18:54:52 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qEnPV-0002KV-04; Thu, 29 Jun 2023 04:53:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qEnPS-0002KK-SC for qemu-devel@nongnu.org; Thu, 29 Jun 2023 04:53:50 -0400 Received: from mga04.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qEnPQ-0003gN-Hx for qemu-devel@nongnu.org; Thu, 29 Jun 2023 04:53:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1688028828; x=1719564828; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=MHe/7/ne2r3XI2nwUy5gMx0jVaJxCFY3fctRyeuq6Xk=; b=Kfkr3F2ME5JBWKN++xKQUq7TvKFAKbu5mV8EONBTTcHv316Nv0o714NO MZA0bQ7x2Yd1qy/TXdQ+6N++ovsGYiuFpoMuwsBNKiKo0fPsibVXRe/Ra 8b2Q0Ylegu+fQqyVxASxWDJ7S/mzwYzbfTDpacdq4lMYvfJQvwNUKq9HO bSrZZ3mVVx5CXQWRxcaFrzMw2OJzegp6s3nSEuvPVLwoK+pkWW0qbk+qz N8QMn8wK6pDR2xIB83/9+6Ezqng76HAszKu1w7yS0d+Er0PwmBbNK1mmT nleavobi7jaV2XUm6J0shWaZj5b6auWIog4/ujyS8vw2xcF8XOIGlzlQ5 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="360908884" X-IronPort-AV: E=Sophos;i="6.01,168,1684825200"; d="scan'208";a="360908884" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jun 2023 01:53:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="720494378" X-IronPort-AV: E=Sophos;i="6.01,168,1684825200"; d="scan'208";a="720494378" Received: from duan-server-s2600bt.bj.intel.com ([10.240.192.147]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jun 2023 01:53:45 -0700 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, joao.m.martins@oracle.com, avihaih@nvidia.com, chao.p.peng@intel.com Subject: [PATCH v4 5/5] vfio/migration: Refactor and fix print of "Migration disabled" Date: Thu, 29 Jun 2023 16:40:42 +0800 Message-Id: <20230629084042.86502-6-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230629084042.86502-1-zhenzhong.duan@intel.com> References: <20230629084042.86502-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mga04.intel.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This patch refactors vfio_migration_realize() and its dependend code as follows: 1. It's redundant in vfio_migration_realize() to registers multiple blockers, e.g: vIOMMU blocker can be refactored as per device blocker. 2. Change vfio_viommu_preset() to be only a per device checker. 3. Remove global vIOMMU blocker related stuff, e.g: giommu_migration_blocker, vfio_[block|unblock]_giommu_migration() and vfio_migration_finalize() 4. Change vfio_migration_realize(), vfio_block_multiple_devices_migration() vfio_block_migration() and vfio_viommu_preset() to return bool type. 5. Print "Migration disabled" depending on enable_migration property and print it as warning instead of error which is overkill. migrate_add_blocker() returns 0 when successfully adding the migration blocker. However, the caller of vfio_migration_realize() considers that migration was blocked when the latter returned an error. What matters for migration is that the blocker is added in core migration, so this cleans up usability such that user sees "Migrate disabled" when any of the vfio migration blockers are active and it's not intentionally forced by user with enable-migration=off. Signed-off-by: Zhenzhong Duan --- hw/vfio/common.c | 66 +++++++---------------------------- hw/vfio/migration.c | 30 +++++++++------- hw/vfio/pci.c | 4 +-- include/hw/vfio/vfio-common.h | 7 ++-- 4 files changed, 36 insertions(+), 71 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 77e2ee0e5c6e..c80ecb1da53f 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -362,7 +362,6 @@ bool vfio_mig_active(void) } static Error *multiple_devices_migration_blocker; -static Error *giommu_migration_blocker; static unsigned int vfio_migratable_device_num(void) { @@ -381,19 +380,19 @@ static unsigned int vfio_migratable_device_num(void) return device_num; } -int vfio_block_multiple_devices_migration(VFIODevice *vbasedev, Error **errp) +bool vfio_block_multiple_devices_migration(VFIODevice *vbasedev, Error **errp) { int ret; if (multiple_devices_migration_blocker || vfio_migratable_device_num() <= 1) { - return 0; + return true; } if (vbasedev->enable_migration == ON_OFF_AUTO_ON) { error_setg(errp, "Migration is currently not supported with multiple " "VFIO devices"); - return -EINVAL; + return false; } error_setg(&multiple_devices_migration_blocker, @@ -403,9 +402,15 @@ int vfio_block_multiple_devices_migration(VFIODevice *vbasedev, Error **errp) if (ret < 0) { error_free(multiple_devices_migration_blocker); multiple_devices_migration_blocker = NULL; + } else { + /* + * Only ON_OFF_AUTO_AUTO case, ON_OFF_AUTO_OFF is checked + * in vfio_migration_realize(). + */ + warn_report("Migration disabled, not support multiple VFIO devices"); } - return ret; + return !ret; } void vfio_unblock_multiple_devices_migration(void) @@ -420,55 +425,10 @@ void vfio_unblock_multiple_devices_migration(void) multiple_devices_migration_blocker = NULL; } -static bool vfio_viommu_preset(void) +/* Block migration with a vIOMMU */ +bool vfio_viommu_preset(VFIODevice *vbasedev) { - VFIOAddressSpace *space; - - QLIST_FOREACH(space, &vfio_address_spaces, list) { - if (space->as != &address_space_memory) { - return true; - } - } - - return false; -} - -int vfio_block_giommu_migration(VFIODevice *vbasedev, Error **errp) -{ - int ret; - - if (giommu_migration_blocker || - !vfio_viommu_preset()) { - return 0; - } - - if (vbasedev->enable_migration == ON_OFF_AUTO_ON) { - error_setg(errp, - "Migration is currently not supported with vIOMMU enabled"); - return -EINVAL; - } - - error_setg(&giommu_migration_blocker, - "Migration is currently not supported with vIOMMU enabled"); - ret = migrate_add_blocker(giommu_migration_blocker, errp); - if (ret < 0) { - error_free(giommu_migration_blocker); - giommu_migration_blocker = NULL; - } - - return ret; -} - -void vfio_migration_finalize(void) -{ - if (!giommu_migration_blocker || - vfio_viommu_preset()) { - return; - } - - migrate_del_blocker(giommu_migration_blocker); - error_free(giommu_migration_blocker); - giommu_migration_blocker = NULL; + return vbasedev->group->container->space->as != &address_space_memory; } static void vfio_set_migration_error(int err) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 1db7d52ab2c1..84036e5cfc01 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -802,13 +802,13 @@ static int vfio_migration_init(VFIODevice *vbasedev) return 0; } -static int vfio_block_migration(VFIODevice *vbasedev, Error *err, Error **errp) +static bool vfio_block_migration(VFIODevice *vbasedev, Error *err, Error **errp) { int ret; if (vbasedev->enable_migration == ON_OFF_AUTO_ON) { error_propagate(errp, err); - return -EINVAL; + return false; } vbasedev->migration_blocker = error_copy(err); @@ -818,9 +818,11 @@ static int vfio_block_migration(VFIODevice *vbasedev, Error *err, Error **errp) if (ret < 0) { error_free(vbasedev->migration_blocker); vbasedev->migration_blocker = NULL; + } else if (vbasedev->enable_migration != ON_OFF_AUTO_OFF) { + warn_report("%s: Migration disabled", vbasedev->name); } - return ret; + return !ret; } /* ---------------------------------------------------------------------- */ @@ -835,7 +837,12 @@ void vfio_reset_bytes_transferred(void) bytes_transferred = 0; } -int vfio_migration_realize(VFIODevice *vbasedev, Error **errp) +/* + * Return true when either migration initialized or blocker registered. + * Currently only return false when adding blocker fails which will + * de-register vfio device. + */ +bool vfio_migration_realize(VFIODevice *vbasedev, Error **errp) { Error *err = NULL; int ret; @@ -873,18 +880,17 @@ int vfio_migration_realize(VFIODevice *vbasedev, Error **errp) vbasedev->name); } - ret = vfio_block_multiple_devices_migration(vbasedev, errp); - if (ret) { - return ret; + if (!vfio_block_multiple_devices_migration(vbasedev, errp)) { + return false; } - ret = vfio_block_giommu_migration(vbasedev, errp); - if (ret) { - return ret; + if (vfio_viommu_preset(vbasedev)) { + error_setg(&err, "%s: Migration is currently not supported " + "with vIOMMU enabled", vbasedev->name); + return vfio_block_migration(vbasedev, err, errp); } - trace_vfio_migration_realize(vbasedev->name); - return 0; + return true; } void vfio_migration_exit(VFIODevice *vbasedev) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index dc69d3031b24..184d08568154 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3209,7 +3209,8 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) if (!pdev->failover_pair_id) { ret = vfio_migration_realize(vbasedev, errp); if (ret) { - error_report("%s: Migration disabled", vbasedev->name); + trace_vfio_migration_realize(vbasedev->name); + } else { goto out_vfio_migration; } } @@ -3257,7 +3258,6 @@ static void vfio_instance_finalize(Object *obj) */ vfio_put_device(vdev); vfio_put_group(group); - vfio_migration_finalize(); } static void vfio_exitfn(PCIDevice *pdev) diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 93429b9abba0..3c18572322fc 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -225,9 +225,9 @@ typedef QLIST_HEAD(VFIOGroupList, VFIOGroup) VFIOGroupList; extern VFIOGroupList vfio_group_list; bool vfio_mig_active(void); -int vfio_block_multiple_devices_migration(VFIODevice *vbasedev, Error **errp); +bool vfio_block_multiple_devices_migration(VFIODevice *vbasedev, Error **errp); void vfio_unblock_multiple_devices_migration(void); -int vfio_block_giommu_migration(VFIODevice *vbasedev, Error **errp); +bool vfio_viommu_preset(VFIODevice *vbasedev); int64_t vfio_mig_bytes_transferred(void); void vfio_reset_bytes_transferred(void); @@ -252,8 +252,7 @@ int vfio_spapr_create_window(VFIOContainer *container, int vfio_spapr_remove_window(VFIOContainer *container, hwaddr offset_within_address_space); -int vfio_migration_realize(VFIODevice *vbasedev, Error **errp); +bool vfio_migration_realize(VFIODevice *vbasedev, Error **errp); void vfio_migration_exit(VFIODevice *vbasedev); -void vfio_migration_finalize(void); #endif /* HW_VFIO_VFIO_COMMON_H */