From patchwork Tue Dec 26 09:24:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880220 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=UxS1oD2q; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Szq9v3BKmz1ydf for ; Tue, 26 Dec 2023 20:25:07 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3fa-00016o-1v; Tue, 26 Dec 2023 04:24:14 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3fY-00016N-5E for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:12 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3fW-0008FE-P2 for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:11 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582650; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o2xfTTZNpltmefGE+HE4JL4JH7Ql9blgDHyAGxKzSyU=; b=UxS1oD2qYTZhD6xftG+NsLUBV3y2INPtYSFdUaGZdDrfXrEVuhf15C9dxyahZP+D/qEujb E7nspn36MiFhmsm86vrBU/J7Sos1E/WAzOuQHTZzHXCAoDvG/qf+/MjPDYldI8CDj4dJAj HAAAKS8X/4DHLDYfRweB/UpKI+eflFA= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-80-yaHVmNVZNpGq2-cK-TSRSg-1; Tue, 26 Dec 2023 04:24:08 -0500 X-MC-Unique: yaHVmNVZNpGq2-cK-TSRSg-1 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-3367b16449bso2796275f8f.3 for ; Tue, 26 Dec 2023 01:24:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582646; x=1704187446; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=o2xfTTZNpltmefGE+HE4JL4JH7Ql9blgDHyAGxKzSyU=; b=uocL8OOs9Fjl2E17oWYbeHXvH2nZUrpzCk5MAvZjNNqAV3J9GX/2qMtgx5/7DPk+i7 cxKwsFTjvvzMNgmPvAojSgVT0cLKxwvYAVLx+5LwYj3lXFq56gut5RbofFm0g9SJ8tjB nX3JqvKE3A2r/Os7rP9wrofEOcde8OQqk1Dl3j7VF0HNJ2EiXCbHN+ctiB1BXT5uvZqE 9Rr1oQdIYlNkvxf5GoqFSIA5abgLquKHZLoGqIkwz1ibaTbyKVfMSnzBW3OpD7F0WDpd 7WRd8JP4glXTVk5UGX63AjNYB9Um8lHEHYypgnHrqh60MELH2e8Ro3G+uLINpBbdervE c5KQ== X-Gm-Message-State: AOJu0YyVSxdnYhSBQ7/pmh/cFSRYPTjivz1HI5W680wTcG8fmUjQ03q+ ANt+9ReDEUQRgB5t48kWGx3VA2gS0dYz/JT2sV1eb/VZhk5f/ia4PUuIhhfoGZ0eN7oCR1qR8f+ MdT1bY0gHnXoFIwy/vLmUtVooKtgl0rylMAskVwccVK3pTRqHvmSLaDmHeiT5cZqarcGSRxwq5S Oh X-Received: by 2002:a5d:6992:0:b0:336:de69:c215 with SMTP id g18-20020a5d6992000000b00336de69c215mr362505wru.102.1703582646365; Tue, 26 Dec 2023 01:24:06 -0800 (PST) X-Google-Smtp-Source: AGHT+IGp1ft5Yw1fA2QKmka4/VngXYvJ5urOywK68YYFwPaOVnPcAjOtEXSduFeJmFIMd6KFlcXQJg== X-Received: by 2002:a5d:6992:0:b0:336:de69:c215 with SMTP id g18-20020a5d6992000000b00336de69c215mr362496wru.102.1703582646089; Tue, 26 Dec 2023 01:24:06 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id q17-20020adfcd91000000b003362d0eefd3sm12096408wrj.20.2023.12.26.01.24.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:24:03 -0800 (PST) Date: Tue, 26 Dec 2023 04:24:01 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Mathieu Poirier , Leo Yan , Philippe =?utf-8?q?Mathieu-Daud=C3=A9?= Subject: [PULL 01/21] virtio: rng: Check notifier helpers for VIRTIO_CONFIG_IRQ_IDX Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.133.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mathieu Poirier Since the driver doesn't support interrupts, we must return early when index is set to VIRTIO_CONFIG_IRQ_IDX. Basically the same thing Viresh did for "91208dd297f2 virtio: i2c: Check notifier helpers for VIRTIO_CONFIG_IRQ_IDX". Fixes: 544f0278afca ("virtio: introduce macro VIRTIO_CONFIG_IRQ_IDX") Signed-off-by: Mathieu Poirier Message-Id: <20231025171841.3379663-1-mathieu.poirier@linaro.org> Tested-by: Leo Yan Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- hw/virtio/vhost-user-rng.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/hw/virtio/vhost-user-rng.c b/hw/virtio/vhost-user-rng.c index efc54cd3fb..24ac1a22c8 100644 --- a/hw/virtio/vhost-user-rng.c +++ b/hw/virtio/vhost-user-rng.c @@ -129,6 +129,14 @@ static void vu_rng_guest_notifier_mask(VirtIODevice *vdev, int idx, bool mask) { VHostUserRNG *rng = VHOST_USER_RNG(vdev); + /* + * We don't support interrupts, return early if index is set to + * VIRTIO_CONFIG_IRQ_IDX. + */ + if (idx == VIRTIO_CONFIG_IRQ_IDX) { + return; + } + vhost_virtqueue_mask(&rng->vhost_dev, vdev, idx, mask); } @@ -136,6 +144,14 @@ static bool vu_rng_guest_notifier_pending(VirtIODevice *vdev, int idx) { VHostUserRNG *rng = VHOST_USER_RNG(vdev); + /* + * We don't support interrupts, return early if index is set to + * VIRTIO_CONFIG_IRQ_IDX. + */ + if (idx == VIRTIO_CONFIG_IRQ_IDX) { + return false; + } + return vhost_virtqueue_pending(&rng->vhost_dev, idx); } From patchwork Tue Dec 26 09:24:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880219 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=grnCm7u8; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Szq9g2cnfz1ydf for ; Tue, 26 Dec 2023 20:24:55 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3ff-00019i-Ms; Tue, 26 Dec 2023 04:24:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3fe-00017E-1G for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:18 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3fc-0008Fc-Fd for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582655; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mT5jS00lY7mc5PDHiaNs4tRIIbDIIgu7me/ZwVh7LTg=; b=grnCm7u8fGEyhxFMxZcIzGBt2qosJLW5st3x7NNURO57D74q7g6J/FLRcIHQDDTRGt1NlM ZYcC2Lrgeim0WM3jT13gNnam9Ao4lPJ4Vk3h3lEHbcFtGhFKHctrBHdj97SffP8XY4a7ek M2sGjsK6HdC5QP+WN3m8KDp45wCvqVA= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-158-sYukVv2wP-qcj5iiGWAzkQ-1; Tue, 26 Dec 2023 04:24:13 -0500 X-MC-Unique: sYukVv2wP-qcj5iiGWAzkQ-1 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-3368c707b03so3057422f8f.3 for ; Tue, 26 Dec 2023 01:24:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582651; x=1704187451; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=mT5jS00lY7mc5PDHiaNs4tRIIbDIIgu7me/ZwVh7LTg=; b=Ekdvyl4Nh/clHj0NVTWlRc4XefPZylmgWWIUXh/U9OSduxmgLGhrUU+Sf9TvX9/MaI w+wY/DA2EKgPFqnIy5nld9llO9X8SSaU1QAQv/CG5JT7YPfYv8Bvyoe+2dLzmFhskTnh gSsozg0ynIaFvN+AlzPGJGoBSU1LnFZSrmcxkoenVCwkgZJJsgMyizRDSs4ulzjOcF9a O0Y+i4qdt2FDRs1gso1t3CPhDMnA490edGGm94VzSrrYQY7/S1AeoZ5xMl7RO5uwTKft 07b4CAIqfQdnMRwGvsAGN87qtiN97FLr3mAOUZWNHT0fLSnCo1AdRp5OwRIRGwGKQioJ mzzQ== X-Gm-Message-State: AOJu0YzRiPB1S3ekf95tgWt1ECA5TcvRszgERLOv9fl0oGrRsj/HyZu+ OwgCJutvzhCJ7Uguc8ajWCMn7oWhmKocd6z2x0een4D9bdc5Oa3aEmOrCBbzKMkrQMV5LHYI9KE cK09X512m1t1AXkas2s2Gb0yvCsvjjoXTlcUXQsx6TuwBYHH7JnhfcV8COsGrIG0buj5eT7MZsu yK X-Received: by 2002:a05:600c:548e:b0:40c:336d:2a24 with SMTP id iv14-20020a05600c548e00b0040c336d2a24mr4086277wmb.30.1703582651194; Tue, 26 Dec 2023 01:24:11 -0800 (PST) X-Google-Smtp-Source: AGHT+IEmfJxVq4qu3NdtgqtRdLErZ/f0RC0FMYa0p7RMNAkweJmyV2gbqk++XTcifhQRohu1Zfjvsw== X-Received: by 2002:a05:600c:548e:b0:40c:336d:2a24 with SMTP id iv14-20020a05600c548e00b0040c336d2a24mr4086265wmb.30.1703582650813; Tue, 26 Dec 2023 01:24:10 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id n9-20020a05600c4f8900b0040d56075463sm6630463wmq.44.2023.12.26.01.24.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:24:08 -0800 (PST) Date: Tue, 26 Dec 2023 04:24:06 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Zhao Liu , Philippe =?utf-8?q?Mathieu-Daud=C3=A9?= , Igor Mammedov , Ani Sinha Subject: [PULL 02/21] tests: bios-tables-test: Rename smbios type 4 related test functions Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.133.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Zhao Liu In fact, type4-count, core-count, core-count2, thread-count and thread-count2 are tested with KVM not TCG. Rename these test functions to reflect KVM base instead of TCG. Signed-off-by: Zhao Liu Message-Id: <20231127160202.1037290-1-zhao1.liu@linux.intel.com> Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Igor Mammedov Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- tests/qtest/bios-tables-test.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/tests/qtest/bios-tables-test.c b/tests/qtest/bios-tables-test.c index fe6a9a8563..21811a1ab5 100644 --- a/tests/qtest/bios-tables-test.c +++ b/tests/qtest/bios-tables-test.c @@ -1015,7 +1015,7 @@ static void test_acpi_q35_tcg(void) free_test_data(&data); } -static void test_acpi_q35_tcg_type4_count(void) +static void test_acpi_q35_kvm_type4_count(void) { test_data data = { .machine = MACHINE_Q35, @@ -1031,7 +1031,7 @@ static void test_acpi_q35_tcg_type4_count(void) free_test_data(&data); } -static void test_acpi_q35_tcg_core_count(void) +static void test_acpi_q35_kvm_core_count(void) { test_data data = { .machine = MACHINE_Q35, @@ -1048,7 +1048,7 @@ static void test_acpi_q35_tcg_core_count(void) free_test_data(&data); } -static void test_acpi_q35_tcg_core_count2(void) +static void test_acpi_q35_kvm_core_count2(void) { test_data data = { .machine = MACHINE_Q35, @@ -1065,7 +1065,7 @@ static void test_acpi_q35_tcg_core_count2(void) free_test_data(&data); } -static void test_acpi_q35_tcg_thread_count(void) +static void test_acpi_q35_kvm_thread_count(void) { test_data data = { .machine = MACHINE_Q35, @@ -1082,7 +1082,7 @@ static void test_acpi_q35_tcg_thread_count(void) free_test_data(&data); } -static void test_acpi_q35_tcg_thread_count2(void) +static void test_acpi_q35_kvm_thread_count2(void) { test_data data = { .machine = MACHINE_Q35, @@ -2262,15 +2262,15 @@ int main(int argc, char *argv[]) qtest_add_func("acpi/q35/kvm/xapic", test_acpi_q35_kvm_xapic); qtest_add_func("acpi/q35/kvm/dmar", test_acpi_q35_kvm_dmar); qtest_add_func("acpi/q35/type4-count", - test_acpi_q35_tcg_type4_count); + test_acpi_q35_kvm_type4_count); qtest_add_func("acpi/q35/core-count", - test_acpi_q35_tcg_core_count); + test_acpi_q35_kvm_core_count); qtest_add_func("acpi/q35/core-count2", - test_acpi_q35_tcg_core_count2); + test_acpi_q35_kvm_core_count2); qtest_add_func("acpi/q35/thread-count", - test_acpi_q35_tcg_thread_count); + test_acpi_q35_kvm_thread_count); qtest_add_func("acpi/q35/thread-count2", - test_acpi_q35_tcg_thread_count2); + test_acpi_q35_kvm_thread_count2); } if (qtest_has_device("virtio-iommu-pci")) { qtest_add_func("acpi/q35/viot", test_acpi_q35_viot); From patchwork Tue Dec 26 09:24:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880218 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=CDXRmjt1; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Szq9R1hStz1ydf for ; Tue, 26 Dec 2023 20:24:43 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3fj-0001EI-K7; Tue, 26 Dec 2023 04:24:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3fi-0001DG-Qn for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:22 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3fh-0008Fw-4y for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582660; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=afYN6k1oPeJkIFiSUzbyD9REIUlFHavOgE55XoawkLE=; b=CDXRmjt14gNi6KytgQ2wv6SNR4YUhFmdl19jsXS/5sSp+pXk/vjT4uM9/8KU3KpSacrArc f9uQczUXm0xmNvHT1QVBuCaJNkWOCx88n0akjQrOg4r1xTEe9SgpYYK1OjgdQTlpIpP1YX 4n6wyxCTwAdYrJAQB9IGoymw+iQ8ReM= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-280-QLKKt20ENCei-bdAwu6-4w-1; Tue, 26 Dec 2023 04:24:18 -0500 X-MC-Unique: QLKKt20ENCei-bdAwu6-4w-1 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-33691eecbebso2686541f8f.0 for ; Tue, 26 Dec 2023 01:24:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582655; x=1704187455; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=afYN6k1oPeJkIFiSUzbyD9REIUlFHavOgE55XoawkLE=; b=q6ch3q8r8HbWOxRUsRdlRT3zrGcXzXnOS7BCPyZb/sO00Oy1V/DLquJjUCmhVVgagZ CKRFJBYmGCWirSFIdZ2FhVtdqMJX/12XD3TdCRjWcYBSTpaAgjVO3KCKrWiM0/o9Wdod AKMPC0d1VKKS0azXXqzjwtQLDZN0ZekveoB/PDGm3qBE6lmD/AzA+70dUTjWkxAbNpAJ TKqBWZExc9kUg0cze5Tb1g1TOHwruAGnZ/ry44/TXNfKf6IqZdtJuisRHTxJ5Bh2n+xE i+8I5GsVc1btkNVSQQ/nBMzByD0A8D2iPNYWp9gdseVDWx8jr5KA9Vyzhg/cifTHNe1B sOew== X-Gm-Message-State: AOJu0YxU3sz5NGHC1QSDX+MoHpyDcXejW12GG4f1VGaj026ftqHUv+no RM3SmNjQRv5d0Sutc/rcTuwf57598kcawczFt+LuIz5kCwH5/T8HKsIYRjBLAhNSnb46Ty08jpw tJ7I3Z1+v7Zu3zmXa5qdDuncni7I42oEaUg1QgemOEgI2rImf2Pfl6zyAaWw57Vgs+psQeZsQzk qs X-Received: by 2002:a05:600c:2907:b0:40c:709:d924 with SMTP id i7-20020a05600c290700b0040c0709d924mr3624525wmd.147.1703582655451; Tue, 26 Dec 2023 01:24:15 -0800 (PST) X-Google-Smtp-Source: AGHT+IFMzlsOJA9HLAoAMDwrAJIV/8ar/+5ls1yNlmuz7d2zE+dD3CVcz/m9OOqhsXh5KOsryvkC2Q== X-Received: by 2002:a05:600c:2907:b0:40c:709:d924 with SMTP id i7-20020a05600c290700b0040c0709d924mr3624517wmd.147.1703582655112; Tue, 26 Dec 2023 01:24:15 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id bi6-20020a05600c3d8600b0040d5868cc15sm3793902wmb.23.2023.12.26.01.24.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:24:12 -0800 (PST) Date: Tue, 26 Dec 2023 04:24:11 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Mike Christie , Stefano Garzarella , Stefan Hajnoczi Subject: [PULL 03/21] vhost: Add worker backend callouts Message-ID: <9aad781959d770eb2479596a23e471cd82bd4e5b.1703582625.git.mst@redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.133.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mike Christie This adds the vhost backend callouts for the worker ioctls added in the 6.4 linux kernel commit: c1ecd8e95007 ("vhost: allow userspace to create workers") Signed-off-by: Mike Christie Reviewed-by: Stefano Garzarella Reviewed-by: Stefan Hajnoczi Message-Id: <20231204231618.21962-2-michael.christie@oracle.com> Reviewed-by: Stefan Hajnoczi Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- include/hw/virtio/vhost-backend.h | 14 ++++++++++++++ hw/virtio/vhost-backend.c | 28 ++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+) diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h index a86d103f82..70c2e8ffee 100644 --- a/include/hw/virtio/vhost-backend.h +++ b/include/hw/virtio/vhost-backend.h @@ -45,6 +45,8 @@ struct vhost_memory; struct vhost_vring_file; struct vhost_vring_state; struct vhost_vring_addr; +struct vhost_vring_worker; +struct vhost_worker_state; struct vhost_scsi_target; struct vhost_iotlb_msg; struct vhost_virtqueue; @@ -85,6 +87,14 @@ typedef int (*vhost_set_vring_err_op)(struct vhost_dev *dev, struct vhost_vring_file *file); typedef int (*vhost_set_vring_busyloop_timeout_op)(struct vhost_dev *dev, struct vhost_vring_state *r); +typedef int (*vhost_attach_vring_worker_op)(struct vhost_dev *dev, + struct vhost_vring_worker *worker); +typedef int (*vhost_get_vring_worker_op)(struct vhost_dev *dev, + struct vhost_vring_worker *worker); +typedef int (*vhost_new_worker_op)(struct vhost_dev *dev, + struct vhost_worker_state *worker); +typedef int (*vhost_free_worker_op)(struct vhost_dev *dev, + struct vhost_worker_state *worker); typedef int (*vhost_set_features_op)(struct vhost_dev *dev, uint64_t features); typedef int (*vhost_get_features_op)(struct vhost_dev *dev, @@ -172,6 +182,10 @@ typedef struct VhostOps { vhost_set_vring_call_op vhost_set_vring_call; vhost_set_vring_err_op vhost_set_vring_err; vhost_set_vring_busyloop_timeout_op vhost_set_vring_busyloop_timeout; + vhost_new_worker_op vhost_new_worker; + vhost_free_worker_op vhost_free_worker; + vhost_get_vring_worker_op vhost_get_vring_worker; + vhost_attach_vring_worker_op vhost_attach_vring_worker; vhost_set_features_op vhost_set_features; vhost_get_features_op vhost_get_features; vhost_set_backend_cap_op vhost_set_backend_cap; diff --git a/hw/virtio/vhost-backend.c b/hw/virtio/vhost-backend.c index 17f3fc6a08..833804dd40 100644 --- a/hw/virtio/vhost-backend.c +++ b/hw/virtio/vhost-backend.c @@ -158,6 +158,30 @@ static int vhost_kernel_set_vring_busyloop_timeout(struct vhost_dev *dev, return vhost_kernel_call(dev, VHOST_SET_VRING_BUSYLOOP_TIMEOUT, s); } +static int vhost_kernel_new_worker(struct vhost_dev *dev, + struct vhost_worker_state *worker) +{ + return vhost_kernel_call(dev, VHOST_NEW_WORKER, worker); +} + +static int vhost_kernel_free_worker(struct vhost_dev *dev, + struct vhost_worker_state *worker) +{ + return vhost_kernel_call(dev, VHOST_FREE_WORKER, worker); +} + +static int vhost_kernel_attach_vring_worker(struct vhost_dev *dev, + struct vhost_vring_worker *worker) +{ + return vhost_kernel_call(dev, VHOST_ATTACH_VRING_WORKER, worker); +} + +static int vhost_kernel_get_vring_worker(struct vhost_dev *dev, + struct vhost_vring_worker *worker) +{ + return vhost_kernel_call(dev, VHOST_GET_VRING_WORKER, worker); +} + static int vhost_kernel_set_features(struct vhost_dev *dev, uint64_t features) { @@ -313,6 +337,10 @@ const VhostOps kernel_ops = { .vhost_set_vring_err = vhost_kernel_set_vring_err, .vhost_set_vring_busyloop_timeout = vhost_kernel_set_vring_busyloop_timeout, + .vhost_get_vring_worker = vhost_kernel_get_vring_worker, + .vhost_attach_vring_worker = vhost_kernel_attach_vring_worker, + .vhost_new_worker = vhost_kernel_new_worker, + .vhost_free_worker = vhost_kernel_free_worker, .vhost_set_features = vhost_kernel_set_features, .vhost_get_features = vhost_kernel_get_features, .vhost_set_backend_cap = vhost_kernel_set_backend_cap, From patchwork Tue Dec 26 09:24:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880223 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=CJ8sul1Q; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SzqB52XV5z20Rq for ; Tue, 26 Dec 2023 20:25:17 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3fv-0001IJ-Ld; Tue, 26 Dec 2023 04:24:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3fn-0001H1-7j for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:28 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3fl-0008G8-Fp for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582664; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=xP1V8T2FgKjwD1W++v0ktUcbqsF7NLMyhF1n3TtP/Y4=; b=CJ8sul1Qi8RscYNIlWAT5pGk4DQl5sb/R+fg3WKnpHf0lO7TC0QNg0iw5z7TiXbnTfCGO4 pz8itqiyyPyN1ee7ZkZeoC1Pe8UQcUAWol1Mjjp7yJcUlf0uJXnIjWvdl0dXQfm3dGfLzC K/XVa5LQFraf3qUulOSA4ZZm1M+us7k= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-426-9ZA3IyDWPZm4oGmStvEuVA-1; Tue, 26 Dec 2023 04:24:23 -0500 X-MC-Unique: 9ZA3IyDWPZm4oGmStvEuVA-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-40d5b7be93aso1336085e9.2 for ; Tue, 26 Dec 2023 01:24:23 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582660; x=1704187460; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=xP1V8T2FgKjwD1W++v0ktUcbqsF7NLMyhF1n3TtP/Y4=; b=Jd9Kbn+wkFfRcL0BYEskpQDkrttdplRSmKER3Qz6+YRH/fqhAnocTnZzBYMCe81dRJ fSnne4vzf49Uyx6MjuyDht/0c4vX/1d8tHnDsRLIFKCGTgX5kMCNWqx9OZmSvMvaYN5v ugo38xcb0cq6Mnk+F/d2tvZnuA1j2/MromKO9xIS9ZgaVZeH/E5q6XVYn7hTRRKnMlT4 2HhbK5HW8sEqqj+NufSlyGQHfnZdSkWOhYk6T4XCEHX/soxjnZAnaoyQALeOtPaQg5wm UJVjAFSQEKUHa6azj44yc2l2PzbkNWYZ/X4JqMyYKOL9HvPOUiOF3UdbCrfmZ9GgLiZS s0ww== X-Gm-Message-State: AOJu0YxwLM9IaQE9SJiPmiGOYE+LViLKm5uDR82avsAyKWq0lUnJMzqx 31o5auHYsvcVPgzu1oDwI2TMnaJSYkTCS4vdNKQV0/2Ire1eGyOcrDKA4hm7mNNHXXFhq0NmV+1 +ViPaB9AVXwaFLfFKc56z/2UpVpMe7esxYVbOd51js1Pd3Q/+bz9DLMSdZGKpGD2KCCuiEWfPt9 SY X-Received: by 2002:a05:600c:8505:b0:40c:367d:9022 with SMTP id gw5-20020a05600c850500b0040c367d9022mr1959769wmb.151.1703582660199; Tue, 26 Dec 2023 01:24:20 -0800 (PST) X-Google-Smtp-Source: AGHT+IERzDqRCleI+5hWtr+zSB7/pMLYFzKJkhFF4eTCM0xz5W7GeBu+bj2onBeEBQyABc4eu5UU3A== X-Received: by 2002:a05:600c:8505:b0:40c:367d:9022 with SMTP id gw5-20020a05600c850500b0040c367d9022mr1959759wmb.151.1703582659792; Tue, 26 Dec 2023 01:24:19 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id f11-20020a05600c4e8b00b0040d5118e42esm10923472wmq.22.2023.12.26.01.24.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:24:17 -0800 (PST) Date: Tue, 26 Dec 2023 04:24:15 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Mike Christie , Stefan Hajnoczi , Stefano Garzarella , Paolo Bonzini , Fam Zheng Subject: [PULL 04/21] vhost-scsi: Add support for a worker thread per virtqueue Message-ID: <51396556f0927c3d202f9903db5aa5aa8d4cbd2c.1703582625.git.mst@redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.133.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mike Christie This adds support for vhost-scsi to be able to create a worker thread per virtqueue. Right now for vhost-net we get a worker thread per tx/rx virtqueue pair which scales nicely as we add more virtqueues and CPUs, but for scsi we get the single worker thread that's shared by all virtqueues. When trying to send IO to more than 2 virtqueues the single thread becomes a bottlneck. This patch adds a new setting, worker_per_virtqueue, which can be set to: false: Existing behavior where we get the single worker thread. true: Create a worker per IO virtqueue. Signed-off-by: Mike Christie Reviewed-by: Stefan Hajnoczi Message-Id: <20231204231618.21962-3-michael.christie@oracle.com> Reviewed-by: Stefan Hajnoczi Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin Reviewed-by: Stefano Garzarella --- include/hw/virtio/virtio-scsi.h | 1 + hw/scsi/vhost-scsi.c | 62 +++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+) diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scsi.h index 7f0573b1bf..7be0105918 100644 --- a/include/hw/virtio/virtio-scsi.h +++ b/include/hw/virtio/virtio-scsi.h @@ -51,6 +51,7 @@ typedef struct virtio_scsi_config VirtIOSCSIConfig; struct VirtIOSCSIConf { uint32_t num_queues; uint32_t virtqueue_size; + bool worker_per_virtqueue; bool seg_max_adjust; uint32_t max_sectors; uint32_t cmd_per_lun; diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c index 3126df9e1d..08aa7534df 100644 --- a/hw/scsi/vhost-scsi.c +++ b/hw/scsi/vhost-scsi.c @@ -165,6 +165,59 @@ static const VMStateDescription vmstate_virtio_vhost_scsi = { .pre_save = vhost_scsi_pre_save, }; +static int vhost_scsi_set_workers(VHostSCSICommon *vsc, bool per_virtqueue) +{ + struct vhost_dev *dev = &vsc->dev; + struct vhost_vring_worker vq_worker; + struct vhost_worker_state worker; + int i, ret; + + /* Use default worker */ + if (!per_virtqueue || dev->nvqs == VHOST_SCSI_VQ_NUM_FIXED + 1) { + return 0; + } + + /* + * ctl/evt share the first worker since it will be rare for them + * to send cmds while IO is running. + */ + for (i = VHOST_SCSI_VQ_NUM_FIXED + 1; i < dev->nvqs; i++) { + memset(&worker, 0, sizeof(worker)); + + ret = dev->vhost_ops->vhost_new_worker(dev, &worker); + if (ret == -ENOTTY) { + /* + * worker ioctls are not implemented so just ignore and + * and continue device setup. + */ + warn_report("vhost-scsi: Backend supports a single worker. " + "Ignoring worker_per_virtqueue=true setting."); + ret = 0; + break; + } else if (ret) { + break; + } + + memset(&vq_worker, 0, sizeof(vq_worker)); + vq_worker.worker_id = worker.worker_id; + vq_worker.index = i; + + ret = dev->vhost_ops->vhost_attach_vring_worker(dev, &vq_worker); + if (ret == -ENOTTY) { + /* + * It's a bug for the kernel to have supported the worker creation + * ioctl but not attach. + */ + dev->vhost_ops->vhost_free_worker(dev, &worker); + break; + } else if (ret) { + break; + } + } + + return ret; +} + static void vhost_scsi_realize(DeviceState *dev, Error **errp) { VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); @@ -232,6 +285,13 @@ static void vhost_scsi_realize(DeviceState *dev, Error **errp) goto free_vqs; } + ret = vhost_scsi_set_workers(vsc, vs->conf.worker_per_virtqueue); + if (ret < 0) { + error_setg(errp, "vhost-scsi: vhost worker setup failed: %s", + strerror(-ret)); + goto free_vqs; + } + /* At present, channel and lun both are 0 for bootable vhost-scsi disk */ vsc->channel = 0; vsc->lun = 0; @@ -297,6 +357,8 @@ static Property vhost_scsi_properties[] = { VIRTIO_SCSI_F_T10_PI, false), DEFINE_PROP_BOOL("migratable", VHostSCSICommon, migratable, false), + DEFINE_PROP_BOOL("worker_per_virtqueue", VirtIOSCSICommon, + conf.worker_per_virtqueue, false), DEFINE_PROP_END_OF_LIST(), }; From patchwork Tue Dec 26 09:24:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880221 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=iQeUJrkj; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Szq9y379Cz20Rq for ; Tue, 26 Dec 2023 20:25:09 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3g4-0001an-4W; Tue, 26 Dec 2023 04:24:44 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3fr-0001Hp-UT for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:32 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3fq-0008Hn-3Y for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582669; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=GbBeyS/80jftVPd7Ma4Yal4/KK1d/JdRq54dZwI1a6k=; b=iQeUJrkjSvpsIcxLG9UFfsW9bWHpDk+e7d5MJbTLoMYbK34w2Cwjr+Gt4oOyOoGjUsYPE9 LNKn66+TQQCaqjBdXon75eEwihOCKwYwn8o8a0lPItiqtX6ATwr03RdF+AJGfaIEr2y33k tLkxeV/QHePAyBhirqBpJxi5kvJYDqM= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-693-ZRW6f9M0NHWf5saJnyu16w-1; Tue, 26 Dec 2023 04:24:27 -0500 X-MC-Unique: ZRW6f9M0NHWf5saJnyu16w-1 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-3367e2bd8b0so3195232f8f.1 for ; Tue, 26 Dec 2023 01:24:27 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582664; x=1704187464; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=GbBeyS/80jftVPd7Ma4Yal4/KK1d/JdRq54dZwI1a6k=; b=docJLF/fvIhA7hwKW4QvlmpX89ZXabVh8XQpbh+WGWHmiF/cjcNXC2KKl7wGnOrbbf g6eAHlGv+jiJz1LdWtVX2Bn045cwqFDs2Z2C2ZauUkyJvPxgYrSI2DWU5lx65IjxR/Ht AN0pIGXS9VlLIAv3vOqIwycvLj8FrjndlVwiUzIqqf+g4yBFi9tR93dyhInUAKcXerTB a2yvpV6ziTSgdmo85BNaqA9kU26tjvfjdSpZ20Ra4VAJsOcmRQc3CC24j8RO7Jh+xv3y tvD8AKPQjPkYB1OT/nyxn2T4ph0Pdhzp1wCA6g4Zk+2bJvnZFyWRsKfECporDdoY2ete oTxw== X-Gm-Message-State: AOJu0YzKX2/HZCHetVfq+BFzz24IV4dwKm/JwudLfcityfgRpY8hgsMs Mli2g4/SiEStH5I0Hwh/y4/ClAkzCfHfzHvOaZGDzcUrht3lmHr5IxEuY4Ea2/wIpjQkdiNpE1w rinl+CTqRCgR5whJ1p+OTe69z1mWlo5l1IUUdL+swc4p4+QZnpvyGdt81bxPtly1aWnShWUd2q+ dW X-Received: by 2002:a5d:6a4d:0:b0:336:d4b8:e3f7 with SMTP id t13-20020a5d6a4d000000b00336d4b8e3f7mr779272wrw.4.1703582664605; Tue, 26 Dec 2023 01:24:24 -0800 (PST) X-Google-Smtp-Source: AGHT+IHk62Q93TprXu0PLDyr/W74FGCe2O28dhpbUFJlretpVvf/RIikEzHzEdfm0bv1FNfc9xVbxA== X-Received: by 2002:a5d:6a4d:0:b0:336:d4b8:e3f7 with SMTP id t13-20020a5d6a4d000000b00336d4b8e3f7mr779264wrw.4.1703582664315; Tue, 26 Dec 2023 01:24:24 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id c10-20020adffb0a000000b00336d21e40f7sm2123692wrr.103.2023.12.26.01.24.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:24:22 -0800 (PST) Date: Tue, 26 Dec 2023 04:24:20 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Aaron Young , Igor Mammedov , Ani Sinha Subject: [PULL 05/21] hw/acpi: propagate vcpu hotplug after switch to modern interface Message-ID: <45e48809d3d0328fd2f113b878a62a27f1360664.1703582625.git.mst@redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.129.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Aaron Young If a vcpu with an apic-id that is not supported by the legacy interface (>255) is hot-plugged, the legacy code will dynamically switch to the modern interface. However, the hotplug event is not forwarded to the new interface resulting in the vcpu not being fully/properly added to the machine config. This BUG is evidenced by OVMF when it it attempts to count the vcpus and reports an inconsistent vcpu count reported by the fw_cfg interface and the modern hotpug interface. Fix is to propagate the hotplug event after making the switch from the legacy interface to the modern interface. Cc: "Michael S. Tsirkin" Cc: Igor Mammedov Signed-off-by: Aaron Young Message-Id: <0e8a9baebbb29f2a6c87fd08e43dc2ac4019759a.1702398644.git.Aaron.Young@oracle.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- hw/acpi/cpu_hotplug.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/hw/acpi/cpu_hotplug.c b/hw/acpi/cpu_hotplug.c index 634bbecb31..6f78db0ccb 100644 --- a/hw/acpi/cpu_hotplug.c +++ b/hw/acpi/cpu_hotplug.c @@ -59,7 +59,8 @@ static const MemoryRegionOps AcpiCpuHotplug_ops = { }, }; -static void acpi_set_cpu_present_bit(AcpiCpuHotplug *g, CPUState *cpu) +static void acpi_set_cpu_present_bit(AcpiCpuHotplug *g, CPUState *cpu, + bool *swtchd_to_modern) { CPUClass *k = CPU_GET_CLASS(cpu); int64_t cpu_id; @@ -68,23 +69,34 @@ static void acpi_set_cpu_present_bit(AcpiCpuHotplug *g, CPUState *cpu) if ((cpu_id / 8) >= ACPI_GPE_PROC_LEN) { object_property_set_bool(g->device, "cpu-hotplug-legacy", false, &error_abort); + *swtchd_to_modern = true; return; } + *swtchd_to_modern = false; g->sts[cpu_id / 8] |= (1 << (cpu_id % 8)); } void legacy_acpi_cpu_plug_cb(HotplugHandler *hotplug_dev, AcpiCpuHotplug *g, DeviceState *dev, Error **errp) { - acpi_set_cpu_present_bit(g, CPU(dev)); - acpi_send_event(DEVICE(hotplug_dev), ACPI_CPU_HOTPLUG_STATUS); + bool swtchd_to_modern; + Error *local_err = NULL; + + acpi_set_cpu_present_bit(g, CPU(dev), &swtchd_to_modern); + if (swtchd_to_modern) { + /* propagate the hotplug to the modern interface */ + hotplug_handler_plug(hotplug_dev, dev, &local_err); + } else { + acpi_send_event(DEVICE(hotplug_dev), ACPI_CPU_HOTPLUG_STATUS); + } } void legacy_acpi_cpu_hotplug_init(MemoryRegion *parent, Object *owner, AcpiCpuHotplug *gpe_cpu, uint16_t base) { CPUState *cpu; + bool swtchd_to_modern; memory_region_init_io(&gpe_cpu->io, owner, &AcpiCpuHotplug_ops, gpe_cpu, "acpi-cpu-hotplug", ACPI_GPE_PROC_LEN); @@ -92,7 +104,7 @@ void legacy_acpi_cpu_hotplug_init(MemoryRegion *parent, Object *owner, gpe_cpu->device = owner; CPU_FOREACH(cpu) { - acpi_set_cpu_present_bit(gpe_cpu, cpu); + acpi_set_cpu_present_bit(gpe_cpu, cpu, &swtchd_to_modern); } } From patchwork Tue Dec 26 09:51:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880267 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Ry7kI2/v; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Szqn76Kjyz20RL for ; Tue, 26 Dec 2023 20:52:11 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI469-0001Bp-2t; Tue, 26 Dec 2023 04:51:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI466-0001BX-O5 for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:51:38 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI462-00044Q-7Z for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:51:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703584293; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ibi9cILDkS6+uCUBFarZ9oJKdZKd3xgef/MV0EFr+Q4=; b=Ry7kI2/vv1odjYkKQZekbVvlX6Sa8ZJr32J8ru/NALZcc7YWtwhin6/ly7TUFF9OMqa/iC OFRLL7VziG0XEHSLwOStb1GBcLo7i23SqKwngANsbu7sFX1K+Pt3gHujrhWdkPxs3nZuHc IbdFTdJpsCqSY3B77N3uvdMCkjvrHmk= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-219-ydAMxXNIOK-5P-BjHVZWHg-1; Tue, 26 Dec 2023 04:51:31 -0500 X-MC-Unique: ydAMxXNIOK-5P-BjHVZWHg-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-40d5bc8d13bso686055e9.2 for ; Tue, 26 Dec 2023 01:51:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703584289; x=1704189089; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=ibi9cILDkS6+uCUBFarZ9oJKdZKd3xgef/MV0EFr+Q4=; b=iWzWpIFrn9D3NaDR25KusxcS14NW7vRkLMdw68WYfyHzxi5iuHfUX/lDZZWLSSJ+oH 6ZbyjduJAzcqhVr4qTUrMxpLShqyrb6PW5ckDhT5j9bILsHpP0hsidOR+r/VY5F8vDqG 4D4hg3ml8Oi58n04NFPX1/TxrkEbNe1ErRr4xsYl4POYCMB2b9mFJLQ0+P2jzgnpsojk 3ib5kw+wm/h2FjHpyaUPvx4zrP6/NTGo0fs+8VSN+Lz1BRvsh3fDbenMM+maP1MQK077 ueJr/kU0SO8Pu7yzbdiHvtQzH7SR/OHRbQRXHqAqecch2KqIGx5rbgQ+sh83sg7lflzk F2FQ== X-Gm-Message-State: AOJu0YwBzyWKlYVDFgQN3+pGqXzMSymrtpuyYAc8W7e+uuEBXCdMaNlw lhUa7Ok6pNsZfJM/GF6BJ2CW13/tHXCDCKZl409YX7omz6mp7kGJyF5T1+AdnYmqvAZAuKujHki YN56msNVIijdmaefKlMwMXlv0Qi+KBaE0U9LUPf3luYn4/j5xmvlSqKIo0b5gmmOUO3R8KAa5pV er X-Received: by 2002:a05:600c:511a:b0:40d:3b3f:603d with SMTP id o26-20020a05600c511a00b0040d3b3f603dmr3380407wms.41.1703584289540; Tue, 26 Dec 2023 01:51:29 -0800 (PST) X-Google-Smtp-Source: AGHT+IFXnnR7Ym5Vx48pictoS+FnHH5jctBY+Q47KusuO7dmKBvyrjSaw/ALgWcihxvrZF2x6B8SgA== X-Received: by 2002:a05:600c:511a:b0:40d:3b3f:603d with SMTP id o26-20020a05600c511a00b0040d3b3f603dmr3380398wms.41.1703584289227; Tue, 26 Dec 2023 01:51:29 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id r20-20020a05600c459400b0040c3dcc36e6sm20016130wmo.47.2023.12.26.01.51.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:51:28 -0800 (PST) Date: Tue, 26 Dec 2023 04:51:25 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Dongli Zhang , Feng Li , Raphael Norwitz , Paolo Bonzini , Fam Zheng Subject: [PULL 06/21] vhost-scsi: fix usage of error_reportf_err() Message-ID: <9a557b1b226293aed3f06bd035cf87094d69ea8d.1703582625.git.mst@redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.129.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Dongli Zhang It is required to use error_report() instead of error_reportf_err(), if the prior function does not take local_err as the argument. As a result, the local_err is always NULL and segment fault may happen. vhost_scsi_start() -> vhost_scsi_set_endpoint(s) --> does not allocate local_err -> error_reportf_err() -> error_vprepend() -> g_string_append(newmsg, (*errp)->msg) --> (*errp) is NULL In addition, add ": " at the end of other error_reportf_err() logs. Fixes: 7962e432b4e4 ("vhost-user-scsi: support reconnect to backend") Signed-off-by: Dongli Zhang Message-Id: <20231214003117.43960-1-dongli.zhang@oracle.com> Reviewed-by: Feng Li Reviewed-by: Raphael Norwitz Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- hw/scsi/vhost-scsi.c | 4 ++-- hw/scsi/vhost-user-scsi.c | 3 ++- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c index 08aa7534df..6159eb6fec 100644 --- a/hw/scsi/vhost-scsi.c +++ b/hw/scsi/vhost-scsi.c @@ -91,13 +91,13 @@ static int vhost_scsi_start(VHostSCSI *s) ret = vhost_scsi_common_start(vsc, &local_err); if (ret < 0) { - error_reportf_err(local_err, "Error starting vhost-scsi"); + error_reportf_err(local_err, "Error starting vhost-scsi: "); return ret; } ret = vhost_scsi_set_endpoint(s); if (ret < 0) { - error_reportf_err(local_err, "Error setting vhost-scsi endpoint"); + error_report("Error setting vhost-scsi endpoint"); vhost_scsi_common_stop(vsc); } diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index 780f10559d..af18c4f3d3 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -83,7 +83,8 @@ static void vhost_user_scsi_set_status(VirtIODevice *vdev, uint8_t status) if (should_start) { ret = vhost_user_scsi_start(s, &local_err); if (ret < 0) { - error_reportf_err(local_err, "unable to start vhost-user-scsi: %s", + error_reportf_err(local_err, + "unable to start vhost-user-scsi: %s: ", strerror(-ret)); qemu_chr_fe_disconnect(&vs->conf.chardev); } From patchwork Tue Dec 26 09:24:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880224 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MGltQlZP; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SzqBG2gxxz1ydb for ; Tue, 26 Dec 2023 20:25:26 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3g4-0001gy-R3; Tue, 26 Dec 2023 04:24:44 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3fv-0001LI-ST for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:39 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3fu-0008I0-Dp for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582673; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=MlOqSlfgZan+WoAwKqTBR2Fe+bYpSsLlMzeyIzh7tJo=; b=MGltQlZP4Nfxw/qV8S8LebL86qxgcFX8ForvGhGp8cRULbG3OsIs/A1CEorgJE2UqWKBPf QmfW0eJwVTPgXX23Lg9udqA+9B/V956q/fA2jaJaMiV+SNPZESz+rH+/ora+MEIL06zC65 tnmYh+V52C1dNzmC7fzOCela1W/5rvM= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-241-sO7JgxYVN-Kbc81VXA6yIg-1; Tue, 26 Dec 2023 04:24:32 -0500 X-MC-Unique: sO7JgxYVN-Kbc81VXA6yIg-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-40d5bf5ca25so229815e9.0 for ; Tue, 26 Dec 2023 01:24:32 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582669; x=1704187469; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=MlOqSlfgZan+WoAwKqTBR2Fe+bYpSsLlMzeyIzh7tJo=; b=GHiMixTPF2fKgXtBz5ksY1ufqCAxW3PQyjNKdUGtV0IcHRCrv6/Kzm1TuFpdwhKtWr KpnaVQsGsfvEyi9z01OQRPImpLvVNayUjjEDjOGO6/dtv4khhzKYEgl+Nxbpk123jwYf FfrReDqW80oqyojKgmkfCeYsRDjNufxgOO+h2MCuDt7P9qbccoTsAJFluyUJrfoKZPuu luRce79EAHhINc1sSfAInit3QF2RB1F/mjCoXkxY9VyCdKJPpbriiT/jZhTgS7iN00a1 BvawuZ4rGNtr3hdclkzas+AcQfxSRiXeq68rZrFkByMKzp6Q6JPuuVab1T/7OdFTDMYv KoQg== X-Gm-Message-State: AOJu0YyVKmB2hweHSlIdMtYbjsNP4ExxyYYN8/GNth4igOvZpswnPLQM q+yL8E4XI51fHAzRB1wEmM6k9d0qsNlkidthtHtUn5B47BKnDBTO7ul1keNdRXoMtSllyUj3KSv k/HtMt++A7qZHvZ3h7nG4yJSCGayZqAzSgYS5KY0aWToK1RFTBvk0By8EJvcul7inXVOfPdCaUu cc X-Received: by 2002:a05:600c:1e1b:b0:40c:4a25:8cf9 with SMTP id ay27-20020a05600c1e1b00b0040c4a258cf9mr3886690wmb.50.1703582669547; Tue, 26 Dec 2023 01:24:29 -0800 (PST) X-Google-Smtp-Source: AGHT+IFyP2iDaERehvVnpEElQF830GO5A2ihZHyWx+DmdGnBIbL3BvtCnjHn09FG31Z4nhDxA5Q2tg== X-Received: by 2002:a05:600c:1e1b:b0:40c:4a25:8cf9 with SMTP id ay27-20020a05600c1e1b00b0040c4a258cf9mr3886682wmb.50.1703582669106; Tue, 26 Dec 2023 01:24:29 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id h9-20020a05600c350900b0040d5aca25f1sm1189620wmq.17.2023.12.26.01.24.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:24:28 -0800 (PST) Date: Tue, 26 Dec 2023 04:24:26 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , wangmeiling , Binfeng Wu , Gerd Hoffmann Subject: [PULL 07/21] Fix bugs when VM shutdown with virtio-gpu unplugged Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.129.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: wangmeiling Virtio-gpu malloc memory for the queue when it realized, but the queues was not released when it unrealized, which resulting in a memory leak. In addition, vm_change_state_handler is not cleaned up, which is related to vdev and will lead to segmentation fault when VM shutdown. Signed-off-by: wangmeiling Signed-off-by: Binfeng Wu Message-Id: <7bbbc0f3-2ad9-83ca-b39b-f976d0837daf@huawei.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- hw/display/virtio-gpu-base.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/hw/display/virtio-gpu-base.c b/hw/display/virtio-gpu-base.c index 37af256219..4fc7ef8896 100644 --- a/hw/display/virtio-gpu-base.c +++ b/hw/display/virtio-gpu-base.c @@ -251,7 +251,11 @@ void virtio_gpu_base_device_unrealize(DeviceState *qdev) { VirtIOGPUBase *g = VIRTIO_GPU_BASE(qdev); + VirtIODevice *vdev = VIRTIO_DEVICE(qdev); + virtio_del_queue(vdev, 0); + virtio_del_queue(vdev, 1); + virtio_cleanup(vdev); migrate_del_blocker(&g->migration_blocker); } From patchwork Tue Dec 26 09:24:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880230 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=cGM/qUG/; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SzqCP63MNz1ydb for ; Tue, 26 Dec 2023 20:26:25 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3g6-0001tG-5X; Tue, 26 Dec 2023 04:24:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3g1-0001e4-DK for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:42 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3fz-0008IG-Dx for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:40 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582678; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=N39XRBVP9UVXyghkAg87M/jQ4XVZeFGKXGSiwW/zl4g=; b=cGM/qUG/HErDFz6Uwl5yRoMXaNvRrbf+3ygO/kfM29qkPbQWzlXn1HsQshH8K+RQbMVT+x IjsOpL5emdpKx+D8yLAiSy/QYyIrQLvR5awR1/nnlZM1fMDOZD/49RsVlqx31mXf+GicSi draqZ+6azle79m/FjeGVxPiI43PdmTI= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-62-EmnAx7ofNDePlMvKK0t_cA-1; Tue, 26 Dec 2023 04:24:36 -0500 X-MC-Unique: EmnAx7ofNDePlMvKK0t_cA-1 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-40d5ae4cdadso2393565e9.0 for ; Tue, 26 Dec 2023 01:24:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582674; x=1704187474; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=N39XRBVP9UVXyghkAg87M/jQ4XVZeFGKXGSiwW/zl4g=; b=Rkx5gZfccvUe3ETmGYQdDJQVCa6BFbUz7nciaqUiHgFrqjBdLjlAyIWV/kD80FMq89 Ip/N3CX7Xsejkv9HeHMS0uJTzmlI76xYarhs2tIzw8vTmiWomtVAPRQsWzUsiJrSEO4h wkNHQWcd3KaAe5Lpj5ntGVHHuvJ+EHwWutF+2Sb2Lz6zE4DRV/FNBZh00xwAsyMEEctD ZnClC5jTbza2G6EUAd5YtYVVKjENFdQ60z7c3cNHLkQJRfbYxXPyEa653NPBnW/6AxRB lVJkPPYYoPzue5SKWVdVPHBKoooFZJwBTKggzKH5DFrx9jT6Q00TXB14Uuo01bmAWF6e 2TOA== X-Gm-Message-State: AOJu0YwQwk+2usNgr7IEq/dWSCKcdgNLdWF7RXDeDI7/KzvESdeVGIL7 O3SFaRG9cCNo7TQK09gwsDlU1StDi84v428uUdce3ZWoNiCN35TtAhFUaDQ0D/WQ1B6z5J3hGcS DXnFv/idXevLsFhICbU5on5NCaucrMLYlxmOo86wUUNXSUqCZfpTcTFFqL7zKCFJNHiC1Ux/kWy gJ X-Received: by 2002:a05:600c:310d:b0:40d:284a:6b6e with SMTP id g13-20020a05600c310d00b0040d284a6b6emr2951981wmo.153.1703582674283; Tue, 26 Dec 2023 01:24:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IFOzbcH/TdJGGAfLy0Kq1ZxG7ykCkkmlt35AtZ5CqzanOTXi21+ZRaKB/EymKIeOC5KrIqOoQ== X-Received: by 2002:a05:600c:310d:b0:40d:284a:6b6e with SMTP id g13-20020a05600c310d00b0040d284a6b6emr2951973wmo.153.1703582673863; Tue, 26 Dec 2023 01:24:33 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id c18-20020a5d4152000000b0033609b71825sm12088512wrq.35.2023.12.26.01.24.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:24:31 -0800 (PST) Date: Tue, 26 Dec 2023 04:24:29 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Eugenio =?utf-8?b?UMOpcmV6?= Subject: [PULL 08/21] vdpa: do not set virtio status bits if unneeded Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.133.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Eugenio Pérez Next commits will set DRIVER and ACKNOWLEDGE flags repeatedly in the case of a migration destination. Let's save ioctls with this. Signed-off-by: Eugenio Pérez Message-Id: <20231215172830.2540987-2-eperezma@redhat.com> --- hw/virtio/vhost-vdpa.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 819b2d811a..401dfa96fd 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -511,6 +511,10 @@ static int vhost_vdpa_add_status(struct vhost_dev *dev, uint8_t status) if (ret < 0) { return ret; } + if ((s & status) == status) { + /* Don't set bits already set */ + return 0; + } s |= status; From patchwork Tue Dec 26 09:24:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880222 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=eSBKNrZJ; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SzqB44FDMz20RL for ; Tue, 26 Dec 2023 20:25:16 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3g7-00023i-FL; Tue, 26 Dec 2023 04:24:47 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3g4-0001hR-Ph for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:44 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3g3-0008Ie-5G for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:44 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582682; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mLU/DtphONhg8DS+Rq3R1gZS3ALsJb4F39lfU8Eoxf0=; b=eSBKNrZJYtsrGQULuZgv0XU6oh/g+iHM2um4kNb6BXVWd92SB+TluwO/5XV/enGbF47ymX uHNj1IIdTPRDW47AxHJvZPs0yB0gHSASyxZ3Yp4Zv9acfirB5a2qRP7DS7atvORZkDlWUu C6ANXhat14q4owwLSqq4L4HfbHkeHjI= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-76-xpNZ6NTcNoy2xi8z1mjQGQ-1; Tue, 26 Dec 2023 04:24:41 -0500 X-MC-Unique: xpNZ6NTcNoy2xi8z1mjQGQ-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-40d5aef0cfdso2082615e9.3 for ; Tue, 26 Dec 2023 01:24:40 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582679; x=1704187479; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=mLU/DtphONhg8DS+Rq3R1gZS3ALsJb4F39lfU8Eoxf0=; b=D24pOYos54NUQLGul3w4ZD5O0Rp4apCXMyoztbADWyAX64vfGuFfMMGC8WWQXkUp6b +dwIfB5Vs9WahBnHH/9iqVbld3fjbbKWOtO3xtkVR5HP1T5OlSDW8QCMR7WgVQPQBT3a LsEScIf9NtpSEgRwirIqUW6BgJs5LBSthUX7k7Srll/uIcNO48FH8RBPwxbTdHFi/G9/ pAAcGDmgTdhfVt1EJfhNZYhvKOVUA7DG9ozbdXIR9sFgVRkbBSggxPyg9QMUV85rispT mtfYut5P3z8ekUgX9o3Dmbyk/HfG/C3J/+vUxfuCNaZuoKWY2K75naSJMquaiVBZ6IoV /aCQ== X-Gm-Message-State: AOJu0Yx1T8EPJHymfEZULY94eewerAE79Vln9ga8OmWFwsmvMZv/1mwo DhSic4S9Gzm1HLSDfS83SxQfNzB8ukKyvyMg8/ZtlOm6j+sSDbNwhTliYPBfSS7LqJ8PxogXydS imbLy0KW62por76DzZAyC7l260/XVhuLTM845+Ym0R5kZijPahwuprrEs6WoJVm9mHch6AhZpoO Kf X-Received: by 2002:a05:600c:1909:b0:40d:57d7:6388 with SMTP id j9-20020a05600c190900b0040d57d76388mr490077wmq.302.1703582678889; Tue, 26 Dec 2023 01:24:38 -0800 (PST) X-Google-Smtp-Source: AGHT+IFtArYLj8aR5Rhh9EzMdSeMYSHy5Q7/tYNM8auyh4R2aNYKlyjP0Rha0lb1McH21cbCmxH+Vg== X-Received: by 2002:a05:600c:1909:b0:40d:57d7:6388 with SMTP id j9-20020a05600c190900b0040d57d76388mr490071wmq.302.1703582678630; Tue, 26 Dec 2023 01:24:38 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id n33-20020a05600c502100b004030e8ff964sm28116356wmr.34.2023.12.26.01.24.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:24:36 -0800 (PST) Date: Tue, 26 Dec 2023 04:24:34 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Eugenio =?utf-8?b?UMOpcmV6?= , Jason Wang , Lei Yang Subject: [PULL 09/21] vdpa: add VhostVDPAShared Message-ID: <53ad2fa3e61f4eb6e5ee08597c488fc7056d1210.1703582625.git.mst@redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.133.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Eugenio Pérez It will hold properties shared among all vhost_vdpa instances associated with of the same device. For example, we just need one iova_tree or one memory listener for the entire device. Next patches will register the vhost_vdpa memory listener at the beginning of the VM migration at the destination. This enables QEMU to map the memory to the device before stopping the VM at the source, instead of doing while both source and destination are stopped, thus minimizing the downtime. However, the destination QEMU is unaware of which vhost_vdpa struct will register its memory_listener. If the source guest has CVQ enabled, it will be the one associated with the CVQ. Otherwise, it will be the first one. Save the memory operations related members in a common place rather than always in the first / last vhost_vdpa. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang Message-Id: <20231221174322.3130442-2-eperezma@redhat.com> Tested-by: Lei Yang Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- include/hw/virtio/vhost-vdpa.h | 5 +++++ net/vhost-vdpa.c | 24 ++++++++++++++++++++++-- 2 files changed, 27 insertions(+), 2 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 5407d54fd7..eb1a56d75a 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -30,6 +30,10 @@ typedef struct VhostVDPAHostNotifier { void *addr; } VhostVDPAHostNotifier; +/* Info shared by all vhost_vdpa device models */ +typedef struct vhost_vdpa_shared { +} VhostVDPAShared; + typedef struct vhost_vdpa { int device_fd; int index; @@ -46,6 +50,7 @@ typedef struct vhost_vdpa { bool suspended; /* IOVA mapping used by the Shadow Virtqueue */ VhostIOVATree *iova_tree; + VhostVDPAShared *shared; GPtrArray *shadow_vqs; const VhostShadowVirtqueueOps *shadow_vq_ops; void *shadow_vq_ops_opaque; diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index d0614d7954..8b661b9e6d 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -240,6 +240,10 @@ static void vhost_vdpa_cleanup(NetClientState *nc) qemu_close(s->vhost_vdpa.device_fd); s->vhost_vdpa.device_fd = -1; } + if (s->vhost_vdpa.index != 0) { + return; + } + g_free(s->vhost_vdpa.shared); } /** Dummy SetSteeringEBPF to support RSS for vhost-vdpa backend */ @@ -1661,6 +1665,7 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, bool svq, struct vhost_vdpa_iova_range iova_range, uint64_t features, + VhostVDPAShared *shared, Error **errp) { NetClientState *nc = NULL; @@ -1696,6 +1701,7 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, if (queue_pair_index == 0) { vhost_vdpa_net_valid_svq_features(features, &s->vhost_vdpa.migration_blocker); + s->vhost_vdpa.shared = g_new0(VhostVDPAShared, 1); } else if (!is_datapath) { s->cvq_cmd_out_buffer = mmap(NULL, vhost_vdpa_net_cvq_cmd_page_len(), PROT_READ | PROT_WRITE, @@ -1708,11 +1714,16 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, s->vhost_vdpa.shadow_vq_ops_opaque = s; s->cvq_isolated = cvq_isolated; } + if (queue_pair_index != 0) { + s->vhost_vdpa.shared = shared; + } + ret = vhost_vdpa_add(nc, (void *)&s->vhost_vdpa, queue_pair_index, nvqs); if (ret) { qemu_del_net_client(nc); return NULL; } + return nc; } @@ -1824,17 +1835,26 @@ int net_init_vhost_vdpa(const Netdev *netdev, const char *name, ncs = g_malloc0(sizeof(*ncs) * queue_pairs); for (i = 0; i < queue_pairs; i++) { + VhostVDPAShared *shared = NULL; + + if (i) { + shared = DO_UPCAST(VhostVDPAState, nc, ncs[0])->vhost_vdpa.shared; + } ncs[i] = net_vhost_vdpa_init(peer, TYPE_VHOST_VDPA, name, vdpa_device_fd, i, 2, true, opts->x_svq, - iova_range, features, errp); + iova_range, features, shared, errp); if (!ncs[i]) goto err; } if (has_cvq) { + VhostVDPAState *s0 = DO_UPCAST(VhostVDPAState, nc, ncs[0]); + VhostVDPAShared *shared = s0->vhost_vdpa.shared; + nc = net_vhost_vdpa_init(peer, TYPE_VHOST_VDPA, name, vdpa_device_fd, i, 1, false, - opts->x_svq, iova_range, features, errp); + opts->x_svq, iova_range, features, shared, + errp); if (!nc) goto err; } From patchwork Tue Dec 26 09:24:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880226 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=JG4RCZcl; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SzqBq5XWbz1ydb for ; Tue, 26 Dec 2023 20:25:55 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3gE-0002Qg-D1; Tue, 26 Dec 2023 04:24:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gC-0002OO-R0 for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:52 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gA-0008JF-W2 for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:52 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582690; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RbA+kUtWMbiVFNBvekZuIHQReV+HuD/UzXGFUeOqd5c=; b=JG4RCZclgIYPN8eiwBBtphjcsTTxe9m9NUvfyy39tEgAiRXPfuo+F76eY1594qIuC51qd8 7sm8ylDHqOgMDk0cZAEwhBjqZVUrm4gXwx+b4E30puEw10QJzLuAeHFiRdNPHIjG+1/BDd L55W8MUMBK8USKDQebDYbnwInMP47tg= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-250-qqK67ZCDOYSyRlbBVASV3A-1; Tue, 26 Dec 2023 04:24:48 -0500 X-MC-Unique: qqK67ZCDOYSyRlbBVASV3A-1 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-336c3c777ffso1155773f8f.2 for ; Tue, 26 Dec 2023 01:24:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582685; x=1704187485; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=RbA+kUtWMbiVFNBvekZuIHQReV+HuD/UzXGFUeOqd5c=; b=PGSyqho4kHpc/GQG/n8J/VUkcbIDcnZeZDBlvS820b9wfGfs9evFAhhG4/Ycf9VgST ihaS0b61EE/ratrYtgsjkvsKN5eTdiHEVu6bU4vgHZ0564c0CKgyvuGQAHYAZyI38xy3 DjsZMkdsWycexI2hUVUB/liAwLYfpIIRdv6tJ0xx7aFXAi0nJNj/t57hqoINkLQ2sjsp QWBL8zSCY+ZO7PCJeLPxFXYO7VF6dIF65nOVfz9mlPLBh4Si9SB3TVZ8Uyfg/4GVE5hr 3Ws70GpC8gx8LZu22za7feRNnv5F10HkE50vXSMIGH5QxBw231uAJgZUr3M1vc0Dx6Lp pvmQ== X-Gm-Message-State: AOJu0YyZIufAZsX/kgbRzPKv76XLBvuaNCoCFKhaWq/zJXglAvyTagRc AtoSVIvdfpSZHXdLIc5Kz3ux3m3rbi64G3lVkVjVzzpT2mdqzoebsYnMgyT0L9qHr+c9ywnyzAR CBA8sh1u4NJpAEZ8Pzoccwkpngb3Y2d0PUmg2mf9ZuxmLpAMJARDf4f81mSnZLVtWrPfUSKE12r ym X-Received: by 2002:a5d:51c6:0:b0:336:7d6b:de76 with SMTP id n6-20020a5d51c6000000b003367d6bde76mr4675758wrv.67.1703582684973; Tue, 26 Dec 2023 01:24:44 -0800 (PST) X-Google-Smtp-Source: AGHT+IFy+YOi8e+wxRR4V6PP016G3+jMAcU6a2RP1ulKCNKe7EHlv9qFFZNJO3itBrtlYmNBVrJT2g== X-Received: by 2002:a5d:51c6:0:b0:336:7d6b:de76 with SMTP id n6-20020a5d51c6000000b003367d6bde76mr4675750wrv.67.1703582684578; Tue, 26 Dec 2023 01:24:44 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id o10-20020adfeaca000000b0033677a4e0d6sm12149110wrn.13.2023.12.26.01.24.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:24:41 -0800 (PST) Date: Tue, 26 Dec 2023 04:24:39 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Eugenio =?utf-8?b?UMOpcmV6?= , Jason Wang , Lei Yang Subject: [PULL 10/21] vdpa: move iova tree to the shared struct Message-ID: <0a4a0fd7b5a3d93ef828dfff5be1da56e8a3b6b2.1703582625.git.mst@redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.133.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Eugenio Pérez Next patches will register the vhost_vdpa memory listener while the VM is migrating at the destination, so we can map the memory to the device before stopping the VM at the source. The main goal is to reduce the downtime. However, the destination QEMU is unaware of which vhost_vdpa device will register its memory_listener. If the source guest has CVQ enabled, it will be the CVQ device. Otherwise, it will be the first one. Move the iova tree to VhostVDPAShared so all vhost_vdpa can use it, rather than always in the first or last vhost_vdpa. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang Message-Id: <20231221174322.3130442-3-eperezma@redhat.com> Tested-by: Lei Yang Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- include/hw/virtio/vhost-vdpa.h | 4 +-- hw/virtio/vhost-vdpa.c | 19 ++++++------ net/vhost-vdpa.c | 54 +++++++++++++++------------------- 3 files changed, 35 insertions(+), 42 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index eb1a56d75a..ac036055d3 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -32,6 +32,8 @@ typedef struct VhostVDPAHostNotifier { /* Info shared by all vhost_vdpa device models */ typedef struct vhost_vdpa_shared { + /* IOVA mapping used by the Shadow Virtqueue */ + VhostIOVATree *iova_tree; } VhostVDPAShared; typedef struct vhost_vdpa { @@ -48,8 +50,6 @@ typedef struct vhost_vdpa { bool shadow_data; /* Device suspended successfully */ bool suspended; - /* IOVA mapping used by the Shadow Virtqueue */ - VhostIOVATree *iova_tree; VhostVDPAShared *shared; GPtrArray *shadow_vqs; const VhostShadowVirtqueueOps *shadow_vq_ops; diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 401dfa96fd..95a179a082 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -358,7 +358,7 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, mem_region.size = int128_get64(llsize) - 1, mem_region.perm = IOMMU_ACCESS_FLAG(true, section->readonly), - r = vhost_iova_tree_map_alloc(v->iova_tree, &mem_region); + r = vhost_iova_tree_map_alloc(v->shared->iova_tree, &mem_region); if (unlikely(r != IOVA_OK)) { error_report("Can't allocate a mapping (%d)", r); goto fail; @@ -379,7 +379,7 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, fail_map: if (v->shadow_data) { - vhost_iova_tree_remove(v->iova_tree, mem_region); + vhost_iova_tree_remove(v->shared->iova_tree, mem_region); } fail: @@ -441,13 +441,13 @@ static void vhost_vdpa_listener_region_del(MemoryListener *listener, .size = int128_get64(llsize) - 1, }; - result = vhost_iova_tree_find_iova(v->iova_tree, &mem_region); + result = vhost_iova_tree_find_iova(v->shared->iova_tree, &mem_region); if (!result) { /* The memory listener map wasn't mapped */ return; } iova = result->iova; - vhost_iova_tree_remove(v->iova_tree, *result); + vhost_iova_tree_remove(v->shared->iova_tree, *result); } vhost_vdpa_iotlb_batch_begin_once(v); /* @@ -1063,7 +1063,8 @@ static void vhost_vdpa_svq_unmap_ring(struct vhost_vdpa *v, hwaddr addr) const DMAMap needle = { .translated_addr = addr, }; - const DMAMap *result = vhost_iova_tree_find_iova(v->iova_tree, &needle); + const DMAMap *result = vhost_iova_tree_find_iova(v->shared->iova_tree, + &needle); hwaddr size; int r; @@ -1079,7 +1080,7 @@ static void vhost_vdpa_svq_unmap_ring(struct vhost_vdpa *v, hwaddr addr) return; } - vhost_iova_tree_remove(v->iova_tree, *result); + vhost_iova_tree_remove(v->shared->iova_tree, *result); } static void vhost_vdpa_svq_unmap_rings(struct vhost_dev *dev, @@ -1107,7 +1108,7 @@ static bool vhost_vdpa_svq_map_ring(struct vhost_vdpa *v, DMAMap *needle, { int r; - r = vhost_iova_tree_map_alloc(v->iova_tree, needle); + r = vhost_iova_tree_map_alloc(v->shared->iova_tree, needle); if (unlikely(r != IOVA_OK)) { error_setg(errp, "Cannot allocate iova (%d)", r); return false; @@ -1119,7 +1120,7 @@ static bool vhost_vdpa_svq_map_ring(struct vhost_vdpa *v, DMAMap *needle, needle->perm == IOMMU_RO); if (unlikely(r != 0)) { error_setg_errno(errp, -r, "Cannot map region to device"); - vhost_iova_tree_remove(v->iova_tree, *needle); + vhost_iova_tree_remove(v->shared->iova_tree, *needle); } return r == 0; @@ -1220,7 +1221,7 @@ static bool vhost_vdpa_svqs_start(struct vhost_dev *dev) goto err; } - vhost_svq_start(svq, dev->vdev, vq, v->iova_tree); + vhost_svq_start(svq, dev->vdev, vq, v->shared->iova_tree); ok = vhost_vdpa_svq_map_rings(dev, svq, &addr, &err); if (unlikely(!ok)) { goto err_map; diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 8b661b9e6d..10703e5833 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -354,8 +354,8 @@ static void vhost_vdpa_net_data_start_first(VhostVDPAState *s) migration_add_notifier(&s->migration_state, vdpa_net_migration_state_notifier); if (v->shadow_vqs_enabled) { - v->iova_tree = vhost_iova_tree_new(v->iova_range.first, - v->iova_range.last); + v->shared->iova_tree = vhost_iova_tree_new(v->iova_range.first, + v->iova_range.last); } } @@ -380,11 +380,6 @@ static int vhost_vdpa_net_data_start(NetClientState *nc) return 0; } - if (v->shadow_vqs_enabled) { - VhostVDPAState *s0 = vhost_vdpa_net_first_nc_vdpa(s); - v->iova_tree = s0->vhost_vdpa.iova_tree; - } - return 0; } @@ -417,9 +412,8 @@ static void vhost_vdpa_net_client_stop(NetClientState *nc) dev = s->vhost_vdpa.dev; if (dev->vq_index + dev->nvqs == dev->vq_index_end) { - g_clear_pointer(&s->vhost_vdpa.iova_tree, vhost_iova_tree_delete); - } else { - s->vhost_vdpa.iova_tree = NULL; + g_clear_pointer(&s->vhost_vdpa.shared->iova_tree, + vhost_iova_tree_delete); } } @@ -474,7 +468,7 @@ static int vhost_vdpa_set_address_space_id(struct vhost_vdpa *v, static void vhost_vdpa_cvq_unmap_buf(struct vhost_vdpa *v, void *addr) { - VhostIOVATree *tree = v->iova_tree; + VhostIOVATree *tree = v->shared->iova_tree; DMAMap needle = { /* * No need to specify size or to look for more translations since @@ -508,7 +502,7 @@ static int vhost_vdpa_cvq_map_buf(struct vhost_vdpa *v, void *buf, size_t size, map.translated_addr = (hwaddr)(uintptr_t)buf; map.size = size - 1; map.perm = write ? IOMMU_RW : IOMMU_RO, - r = vhost_iova_tree_map_alloc(v->iova_tree, &map); + r = vhost_iova_tree_map_alloc(v->shared->iova_tree, &map); if (unlikely(r != IOVA_OK)) { error_report("Cannot map injected element"); return r; @@ -523,7 +517,7 @@ static int vhost_vdpa_cvq_map_buf(struct vhost_vdpa *v, void *buf, size_t size, return 0; dma_map_err: - vhost_iova_tree_remove(v->iova_tree, map); + vhost_iova_tree_remove(v->shared->iova_tree, map); return r; } @@ -583,24 +577,22 @@ out: return 0; } - if (s0->vhost_vdpa.iova_tree) { - /* - * SVQ is already configured for all virtqueues. Reuse IOVA tree for - * simplicity, whether CVQ shares ASID with guest or not, because: - * - Memory listener need access to guest's memory addresses allocated - * in the IOVA tree. - * - There should be plenty of IOVA address space for both ASID not to - * worry about collisions between them. Guest's translations are - * still validated with virtio virtqueue_pop so there is no risk for - * the guest to access memory that it shouldn't. - * - * To allocate a iova tree per ASID is doable but it complicates the - * code and it is not worth it for the moment. - */ - v->iova_tree = s0->vhost_vdpa.iova_tree; - } else { - v->iova_tree = vhost_iova_tree_new(v->iova_range.first, - v->iova_range.last); + /* + * If other vhost_vdpa already have an iova_tree, reuse it for simplicity, + * whether CVQ shares ASID with guest or not, because: + * - Memory listener need access to guest's memory addresses allocated in + * the IOVA tree. + * - There should be plenty of IOVA address space for both ASID not to + * worry about collisions between them. Guest's translations are still + * validated with virtio virtqueue_pop so there is no risk for the guest + * to access memory that it shouldn't. + * + * To allocate a iova tree per ASID is doable but it complicates the code + * and it is not worth it for the moment. + */ + if (!v->shared->iova_tree) { + v->shared->iova_tree = vhost_iova_tree_new(v->iova_range.first, + v->iova_range.last); } r = vhost_vdpa_cvq_map_buf(&s->vhost_vdpa, s->cvq_cmd_out_buffer, From patchwork Tue Dec 26 09:24:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880225 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=NFAdqbtF; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SzqBJ0TNpz1ydb for ; Tue, 26 Dec 2023 20:25:28 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3gI-0002Te-6S; Tue, 26 Dec 2023 04:24:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gG-0002Sy-Fw for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:56 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gE-0008Jd-Og for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:24:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582693; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aQNgFux+8fYn065Bfp8ZbzNtSl4MOxXJa+zOn6c9qNY=; b=NFAdqbtFwrWD1vJwLcYdSjkdESoMoVxRlbBl0dzAW0jPxcc/M3GRSYjFnNFc5fwts307uW e1VJitVZSeq37qHWqsYHq53Opcj8SeHeRCHdA7FIaP7zq9ptILbDajLchZRIqzsW47of2h 7bkqRWB6d94olxB59rOx4YJfzb2RWKA= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-461-0n131OKaPFyiuz7mv5qYNQ-1; Tue, 26 Dec 2023 04:24:52 -0500 X-MC-Unique: 0n131OKaPFyiuz7mv5qYNQ-1 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-3367657a290so3237815f8f.3 for ; Tue, 26 Dec 2023 01:24:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582689; x=1704187489; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=aQNgFux+8fYn065Bfp8ZbzNtSl4MOxXJa+zOn6c9qNY=; b=BR7Sz7Eg9s5BgqnyrCBM35/v+L8q17f+5181wotFcJM/JzfOShi6VVtrj6wElC8kc5 wJ/Cwemic6PbhN2KE+TzSc0UVMsFIJJDi/QvP0MIHyx/YlrDvUrDX+sWY+2mmu6yvDj1 AU/jsDgBf1rzlyJxvERf30UUp9NXFtbOaLkrF54tS4vifXHtb/S0wcozpCgBFV2ZYdyM W4EtW37i0H/yErX7fn7QaLXvaASUo9JM8lCgj1aoJ+QCWoUsNKzIaEZSOOSBQ1EkKUrl y1Dnw/uwbwE5qjWSbZGofFngzP7zc6kzMrbo0dTaXKE8VHNnwCvR1RgReL9HjG6XlO+J 6RaA== X-Gm-Message-State: AOJu0YwhfB78NRnnT+DA/WlAi6H5u6BKuz0IlC7TJ0BT35ffL3LrAtf6 MHPfXgxGPP7d1V6lvi0eoRueqByAPlWd7Te8ZydU1sCRYrFc29QlkoWH5bUiq5c7znFt4gZSefM oVnEDgDrBSRZxbcWv8TQHY5fUUdRu7lUXUusq2iB2GFw3pyrtYCeXMuFSZCOLj/1F/YLcVZvylz i7 X-Received: by 2002:a5d:6052:0:b0:336:79df:a706 with SMTP id j18-20020a5d6052000000b0033679dfa706mr4263384wrt.99.1703582689575; Tue, 26 Dec 2023 01:24:49 -0800 (PST) X-Google-Smtp-Source: AGHT+IG9ViuaKOgbAufJfYGgo5oT+d2RG+tMce109y86/fRLzw4VZJFE8rnm4jdDJ06qmknWSMDL+g== X-Received: by 2002:a5d:6052:0:b0:336:79df:a706 with SMTP id j18-20020a5d6052000000b0033679dfa706mr4263375wrt.99.1703582689298; Tue, 26 Dec 2023 01:24:49 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id cg13-20020a5d5ccd000000b00336e6014263sm840030wrb.98.2023.12.26.01.24.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:24:46 -0800 (PST) Date: Tue, 26 Dec 2023 04:24:44 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Eugenio =?utf-8?b?UMOpcmV6?= , Jason Wang , Lei Yang Subject: [PULL 11/21] vdpa: move iova_range to vhost_vdpa_shared Message-ID: <191c3b0284586495b5aa47a2bb97987565ccba55.1703582625.git.mst@redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.129.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Eugenio Pérez Next patches will register the vhost_vdpa memory listener while the VM is migrating at the destination, so we can map the memory to the device before stopping the VM at the source. The main goal is to reduce the downtime. However, the destination QEMU is unaware of which vhost_vdpa device will register its memory_listener. If the source guest has CVQ enabled, it will be the CVQ device. Otherwise, it will be the first one. Move the iova range to VhostVDPAShared so all vhost_vdpa can use it, rather than always in the first or last vhost_vdpa. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang Message-Id: <20231221174322.3130442-4-eperezma@redhat.com> Tested-by: Lei Yang Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- include/hw/virtio/vhost-vdpa.h | 3 ++- hw/virtio/vdpa-dev.c | 5 ++++- hw/virtio/vhost-vdpa.c | 16 ++++++++++------ net/vhost-vdpa.c | 10 +++++----- 4 files changed, 21 insertions(+), 13 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index ac036055d3..8d52a7e498 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -32,6 +32,8 @@ typedef struct VhostVDPAHostNotifier { /* Info shared by all vhost_vdpa device models */ typedef struct vhost_vdpa_shared { + struct vhost_vdpa_iova_range iova_range; + /* IOVA mapping used by the Shadow Virtqueue */ VhostIOVATree *iova_tree; } VhostVDPAShared; @@ -43,7 +45,6 @@ typedef struct vhost_vdpa { bool iotlb_batch_begin_sent; uint32_t address_space_id; MemoryListener listener; - struct vhost_vdpa_iova_range iova_range; uint64_t acked_features; bool shadow_vqs_enabled; /* Vdpa must send shadow addresses as IOTLB key for data queues, not GPA */ diff --git a/hw/virtio/vdpa-dev.c b/hw/virtio/vdpa-dev.c index f22d5d5bc0..457960d28a 100644 --- a/hw/virtio/vdpa-dev.c +++ b/hw/virtio/vdpa-dev.c @@ -114,7 +114,8 @@ static void vhost_vdpa_device_realize(DeviceState *dev, Error **errp) strerror(-ret)); goto free_vqs; } - v->vdpa.iova_range = iova_range; + v->vdpa.shared = g_new0(VhostVDPAShared, 1); + v->vdpa.shared->iova_range = iova_range; ret = vhost_dev_init(&v->dev, &v->vdpa, VHOST_BACKEND_TYPE_VDPA, 0, NULL); if (ret < 0) { @@ -162,6 +163,7 @@ vhost_cleanup: vhost_dev_cleanup(&v->dev); free_vqs: g_free(vqs); + g_free(v->vdpa.shared); out: qemu_close(v->vhostfd); v->vhostfd = -1; @@ -184,6 +186,7 @@ static void vhost_vdpa_device_unrealize(DeviceState *dev) g_free(s->config); g_free(s->dev.vqs); vhost_dev_cleanup(&s->dev); + g_free(s->vdpa.shared); qemu_close(s->vhostfd); s->vhostfd = -1; } diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 95a179a082..5ff1d43ba9 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -213,10 +213,10 @@ static void vhost_vdpa_iommu_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) RCU_READ_LOCK_GUARD(); /* check if RAM section out of device range */ llend = int128_add(int128_makes64(iotlb->addr_mask), int128_makes64(iova)); - if (int128_gt(llend, int128_make64(v->iova_range.last))) { + if (int128_gt(llend, int128_make64(v->shared->iova_range.last))) { error_report("RAM section out of device range (max=0x%" PRIx64 ", end addr=0x%" PRIx64 ")", - v->iova_range.last, int128_get64(llend)); + v->shared->iova_range.last, int128_get64(llend)); return; } @@ -316,8 +316,10 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, int page_size = qemu_target_page_size(); int page_mask = -page_size; - if (vhost_vdpa_listener_skipped_section(section, v->iova_range.first, - v->iova_range.last, page_mask)) { + if (vhost_vdpa_listener_skipped_section(section, + v->shared->iova_range.first, + v->shared->iova_range.last, + page_mask)) { return; } if (memory_region_is_iommu(section->mr)) { @@ -403,8 +405,10 @@ static void vhost_vdpa_listener_region_del(MemoryListener *listener, int page_size = qemu_target_page_size(); int page_mask = -page_size; - if (vhost_vdpa_listener_skipped_section(section, v->iova_range.first, - v->iova_range.last, page_mask)) { + if (vhost_vdpa_listener_skipped_section(section, + v->shared->iova_range.first, + v->shared->iova_range.last, + page_mask)) { return; } if (memory_region_is_iommu(section->mr)) { diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 10703e5833..7be2c30ad3 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -354,8 +354,8 @@ static void vhost_vdpa_net_data_start_first(VhostVDPAState *s) migration_add_notifier(&s->migration_state, vdpa_net_migration_state_notifier); if (v->shadow_vqs_enabled) { - v->shared->iova_tree = vhost_iova_tree_new(v->iova_range.first, - v->iova_range.last); + v->shared->iova_tree = vhost_iova_tree_new(v->shared->iova_range.first, + v->shared->iova_range.last); } } @@ -591,8 +591,8 @@ out: * and it is not worth it for the moment. */ if (!v->shared->iova_tree) { - v->shared->iova_tree = vhost_iova_tree_new(v->iova_range.first, - v->iova_range.last); + v->shared->iova_tree = vhost_iova_tree_new(v->shared->iova_range.first, + v->shared->iova_range.last); } r = vhost_vdpa_cvq_map_buf(&s->vhost_vdpa, s->cvq_cmd_out_buffer, @@ -1688,12 +1688,12 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, s->always_svq = svq; s->migration_state.notify = NULL; s->vhost_vdpa.shadow_vqs_enabled = svq; - s->vhost_vdpa.iova_range = iova_range; s->vhost_vdpa.shadow_data = svq; if (queue_pair_index == 0) { vhost_vdpa_net_valid_svq_features(features, &s->vhost_vdpa.migration_blocker); s->vhost_vdpa.shared = g_new0(VhostVDPAShared, 1); + s->vhost_vdpa.shared->iova_range = iova_range; } else if (!is_datapath) { s->cvq_cmd_out_buffer = mmap(NULL, vhost_vdpa_net_cvq_cmd_page_len(), PROT_READ | PROT_WRITE, From patchwork Tue Dec 26 09:24:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880231 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ZR+dNu6o; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SzqDv5b0tz1ydb for ; Tue, 26 Dec 2023 20:27:43 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3gP-0002nP-4G; Tue, 26 Dec 2023 04:25:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gM-0002f8-Si for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:03 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gI-0008K3-Vb for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:00 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582698; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kho8Kn81JXFBrbbnbKhrQXI3g7g8qDF5iaK2v9gtOs0=; b=ZR+dNu6oTG+V/eotmO7qBAtkFEY90HAa3e4a7lS4UkWFXBj8OpyiyPPUjoEAMRTjM9LNAP HnRjmueRTlJFcuGQdWeKVZftHttXmemhsIhr7sIHBPDatU7I7sZXK1HDJ6F730byAFo0H7 c5JLcsiTqWO4xY9JWWOY6pP3NVEhmNI= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-593-P0-2xfzVN7ijvjKc_3oHGw-1; Tue, 26 Dec 2023 04:24:56 -0500 X-MC-Unique: P0-2xfzVN7ijvjKc_3oHGw-1 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-33688a38636so3151679f8f.1 for ; Tue, 26 Dec 2023 01:24:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582694; x=1704187494; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kho8Kn81JXFBrbbnbKhrQXI3g7g8qDF5iaK2v9gtOs0=; b=dMOf8vxj8qrpjcZqKr/lUQfvcWeCKCSfZJ1zkoANYzA8rwMlMHKQ8bkgHynGRudxrS 9kT/QV5RKBotu4LgAY/hsB5uev/709t1TKCBMGjH8BmbsXrLH+Tdcvri/hYdxLJVIs8A 4jJWUmz3k/WRw82pw35olK3RxZNPQqSHcsCE1XXH/8uK27Ff0TCiIcOizdmzUnmc3GZk VuPsev1Nb1heSLufyLF5RYmyqeMu/IR52PrtOj0LxXlE/MF405VnD+7CMrgwGpuCkV4q REkvNVyx1SQTarUyXfdSNkEWk67Tt96d7HhPOpUtfzZt/Kqs51n5HaSgJTYIomoiHcO2 Hh9Q== X-Gm-Message-State: AOJu0YyB9y1dL0yxN+W/sVt9CiNisc2ve4w+oXB9r3Xi0fToo0z/EvXv pveIBR4lElUd8AXMVrgxOit1244TxnktGOlH7YCIH7BByRfI9elHoNnyEdHMSms10xV8HFZq6pJ V9TjSucC2sZ/k56aRIP68VPD5LivDo4Gvwx/iDNOBq7aKVJeZCIWF+aJkS5iro786LmAAMluC/k 6O X-Received: by 2002:a05:600c:1987:b0:40d:5798:1797 with SMTP id t7-20020a05600c198700b0040d57981797mr1036205wmq.63.1703582694037; Tue, 26 Dec 2023 01:24:54 -0800 (PST) X-Google-Smtp-Source: AGHT+IFAWmUvPxc4Je98bTMHxr/II4hAz93qZvI+rO3dt2AgTAIBQQ99PHYcgWzVXK8axzSKfUm5jw== X-Received: by 2002:a05:600c:1987:b0:40d:5798:1797 with SMTP id t7-20020a05600c198700b0040d57981797mr1036197wmq.63.1703582693724; Tue, 26 Dec 2023 01:24:53 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id e25-20020adfa459000000b003365951cef9sm12261228wra.55.2023.12.26.01.24.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:24:51 -0800 (PST) Date: Tue, 26 Dec 2023 04:24:49 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Eugenio =?utf-8?b?UMOpcmV6?= , Jason Wang , Lei Yang Subject: [PULL 12/21] vdpa: move shadow_data to vhost_vdpa_shared Message-ID: <787ac82bcea98139204c26ccf2a488d2f9b54003.1703582625.git.mst@redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.129.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Eugenio Pérez Next patches will register the vhost_vdpa memory listener while the VM is migrating at the destination, so we can map the memory to the device before stopping the VM at the source. The main goal is to reduce the downtime. However, the destination QEMU is unaware of which vhost_vdpa device will register its memory_listener. If the source guest has CVQ enabled, it will be the CVQ device. Otherwise, it will be the first one. Move the shadow_data member to VhostVDPAShared so all vhost_vdpa can use it, rather than always in the first or last vhost_vdpa. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang Message-Id: <20231221174322.3130442-5-eperezma@redhat.com> Tested-by: Lei Yang Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- include/hw/virtio/vhost-vdpa.h | 5 +++-- hw/virtio/vhost-vdpa.c | 6 +++--- net/vhost-vdpa.c | 22 +++++----------------- 3 files changed, 11 insertions(+), 22 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 8d52a7e498..01e0f25e27 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -36,6 +36,9 @@ typedef struct vhost_vdpa_shared { /* IOVA mapping used by the Shadow Virtqueue */ VhostIOVATree *iova_tree; + + /* Vdpa must send shadow addresses as IOTLB key for data queues, not GPA */ + bool shadow_data; } VhostVDPAShared; typedef struct vhost_vdpa { @@ -47,8 +50,6 @@ typedef struct vhost_vdpa { MemoryListener listener; uint64_t acked_features; bool shadow_vqs_enabled; - /* Vdpa must send shadow addresses as IOTLB key for data queues, not GPA */ - bool shadow_data; /* Device suspended successfully */ bool suspended; VhostVDPAShared *shared; diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 5ff1d43ba9..97588848fc 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -353,7 +353,7 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, vaddr, section->readonly); llsize = int128_sub(llend, int128_make64(iova)); - if (v->shadow_data) { + if (v->shared->shadow_data) { int r; mem_region.translated_addr = (hwaddr)(uintptr_t)vaddr, @@ -380,7 +380,7 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, return; fail_map: - if (v->shadow_data) { + if (v->shared->shadow_data) { vhost_iova_tree_remove(v->shared->iova_tree, mem_region); } @@ -435,7 +435,7 @@ static void vhost_vdpa_listener_region_del(MemoryListener *listener, llsize = int128_sub(llend, int128_make64(iova)); - if (v->shadow_data) { + if (v->shared->shadow_data) { const DMAMap *result; const void *vaddr = memory_region_get_ram_ptr(section->mr) + section->offset_within_region + diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 7be2c30ad3..bf8e8327da 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -290,15 +290,6 @@ static ssize_t vhost_vdpa_receive(NetClientState *nc, const uint8_t *buf, return size; } -/** From any vdpa net client, get the netclient of the first queue pair */ -static VhostVDPAState *vhost_vdpa_net_first_nc_vdpa(VhostVDPAState *s) -{ - NICState *nic = qemu_get_nic(s->nc.peer); - NetClientState *nc0 = qemu_get_peer(nic->ncs, 0); - - return DO_UPCAST(VhostVDPAState, nc, nc0); -} - static void vhost_vdpa_net_log_global_enable(VhostVDPAState *s, bool enable) { struct vhost_vdpa *v = &s->vhost_vdpa; @@ -369,13 +360,12 @@ static int vhost_vdpa_net_data_start(NetClientState *nc) if (s->always_svq || migration_is_setup_or_active(migrate_get_current()->state)) { v->shadow_vqs_enabled = true; - v->shadow_data = true; } else { v->shadow_vqs_enabled = false; - v->shadow_data = false; } if (v->index == 0) { + v->shared->shadow_data = v->shadow_vqs_enabled; vhost_vdpa_net_data_start_first(s); return 0; } @@ -523,7 +513,7 @@ dma_map_err: static int vhost_vdpa_net_cvq_start(NetClientState *nc) { - VhostVDPAState *s, *s0; + VhostVDPAState *s; struct vhost_vdpa *v; int64_t cvq_group; int r; @@ -534,12 +524,10 @@ static int vhost_vdpa_net_cvq_start(NetClientState *nc) s = DO_UPCAST(VhostVDPAState, nc, nc); v = &s->vhost_vdpa; - s0 = vhost_vdpa_net_first_nc_vdpa(s); - v->shadow_data = s0->vhost_vdpa.shadow_vqs_enabled; - v->shadow_vqs_enabled = s0->vhost_vdpa.shadow_vqs_enabled; + v->shadow_vqs_enabled = v->shared->shadow_data; s->vhost_vdpa.address_space_id = VHOST_VDPA_GUEST_PA_ASID; - if (s->vhost_vdpa.shadow_data) { + if (v->shared->shadow_data) { /* SVQ is already configured for all virtqueues */ goto out; } @@ -1688,12 +1676,12 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, s->always_svq = svq; s->migration_state.notify = NULL; s->vhost_vdpa.shadow_vqs_enabled = svq; - s->vhost_vdpa.shadow_data = svq; if (queue_pair_index == 0) { vhost_vdpa_net_valid_svq_features(features, &s->vhost_vdpa.migration_blocker); s->vhost_vdpa.shared = g_new0(VhostVDPAShared, 1); s->vhost_vdpa.shared->iova_range = iova_range; + s->vhost_vdpa.shared->shadow_data = svq; } else if (!is_datapath) { s->cvq_cmd_out_buffer = mmap(NULL, vhost_vdpa_net_cvq_cmd_page_len(), PROT_READ | PROT_WRITE, From patchwork Tue Dec 26 09:24:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880229 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=hMs/tJIR; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SzqC05Z2Gz1ydb for ; Tue, 26 Dec 2023 20:26:04 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3gR-00031b-5N; Tue, 26 Dec 2023 04:25:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gP-0002sZ-OF for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:05 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gN-0008Ml-EO for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:05 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582702; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f9WQMHsSDkCkbXrOH1Cth231xwoLWiHqIfjiVylGQQQ=; b=hMs/tJIRODxpKB/vYEYfw42zb2Hl/YYMqMDH9ZtejFRIXrYQ8zaS3I0IP/Dh9nL26bwhwY FOnKuDbghRgiZ2m2s2B1n20HjGEGQC1jhNf4y+vlTj+hjYUhBxgAaDQ6Jl7Mxm+jGCDj0I HDTO6WR6ycck9riT8UCGN4keNEXzce8= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-643-qZ1BfraSOZWLAWR3QON0ZQ-1; Tue, 26 Dec 2023 04:25:01 -0500 X-MC-Unique: qZ1BfraSOZWLAWR3QON0ZQ-1 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-33688a38636so3151698f8f.1 for ; Tue, 26 Dec 2023 01:25:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582698; x=1704187498; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=f9WQMHsSDkCkbXrOH1Cth231xwoLWiHqIfjiVylGQQQ=; b=ZFcU2OBoGPc30J+sxzTmqc6jjB8yw1Yxko+Lckzl496v88atIbxfgCTv9+oEGIwAdS ZOvCERsWFsSb+uFSWN/W86dlpY7fzlJIHONksIQI+sdxZIXfzcFcq7RiwRFTG83NXEMd AmHFz2tZi8RfwhsUXtbBtSxcdvpNCZ3l5VJstw4iXyTcXi7ME0SzgBTwk2DGfGWBvaEe KY9LMe5xj0TbAf1EZAzR938l9n3eljH91+07ZLeoaErgEUePt0170XelN/LiDQZ5lTJp EV+3z7ZjWkOGbFdqvZ6/NTfwI39rAKub3lARdOWxr4IEVyry0B8cmTSD/AvAswvIBN6y wmhg== X-Gm-Message-State: AOJu0Yw0Eu6SjYKtdj671Mf6Y/3Fo78wvW6jsgCaGwhpRDHUSNCMccKg 8g1NBbJLdzphVT9/uCsrfJ/tALIyMjxBg6gxWa9tSTjvQfQVn5v9yUs86n3hLA6kAJrHunX/+Pt ObGlv4ypTU/JPM9cxapSDp94ttZez8aKRh2EB7swljXJbJdoveYxurqfD5HVKnvKnhnS7DxEkcr Cu X-Received: by 2002:a5d:4583:0:b0:336:9f70:a708 with SMTP id p3-20020a5d4583000000b003369f70a708mr2444725wrq.107.1703582698086; Tue, 26 Dec 2023 01:24:58 -0800 (PST) X-Google-Smtp-Source: AGHT+IGGIufyHfF3ErGEGbwWEFE+MnXNup9TvAxtE6sP2XqflND6l7cyjEEqIT+aAwlBOM9eUqLVYA== X-Received: by 2002:a5d:4583:0:b0:336:9f70:a708 with SMTP id p3-20020a5d4583000000b003369f70a708mr2444716wrq.107.1703582697707; Tue, 26 Dec 2023 01:24:57 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id w14-20020adfee4e000000b0033609750752sm12238175wro.8.2023.12.26.01.24.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:24:55 -0800 (PST) Date: Tue, 26 Dec 2023 04:24:53 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Eugenio =?utf-8?b?UMOpcmV6?= , Jason Wang , Lei Yang Subject: [PULL 13/21] vdpa: use vdpa shared for tracing Message-ID: <7f203ed843f6f510c4af086532998cbae1c0a52e.1703582625.git.mst@redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.129.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Eugenio Pérez By the end of this series dma_map and dma_unmap functions don't have the vdpa device for tracing. Movinge trace function to shared member one. Print it also in the vdpa initialization so log reader can relate them. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang Message-Id: <20231221174322.3130442-6-eperezma@redhat.com> Tested-by: Lei Yang Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- hw/virtio/vhost-vdpa.c | 26 ++++++++++++++------------ hw/virtio/trace-events | 14 +++++++------- 2 files changed, 21 insertions(+), 19 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 97588848fc..720cffbc08 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -101,7 +101,7 @@ int vhost_vdpa_dma_map(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, msg.iotlb.perm = readonly ? VHOST_ACCESS_RO : VHOST_ACCESS_RW; msg.iotlb.type = VHOST_IOTLB_UPDATE; - trace_vhost_vdpa_dma_map(v, fd, msg.type, msg.asid, msg.iotlb.iova, + trace_vhost_vdpa_dma_map(v->shared, fd, msg.type, msg.asid, msg.iotlb.iova, msg.iotlb.size, msg.iotlb.uaddr, msg.iotlb.perm, msg.iotlb.type); @@ -131,8 +131,8 @@ int vhost_vdpa_dma_unmap(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, msg.iotlb.size = size; msg.iotlb.type = VHOST_IOTLB_INVALIDATE; - trace_vhost_vdpa_dma_unmap(v, fd, msg.type, msg.asid, msg.iotlb.iova, - msg.iotlb.size, msg.iotlb.type); + trace_vhost_vdpa_dma_unmap(v->shared, fd, msg.type, msg.asid, + msg.iotlb.iova, msg.iotlb.size, msg.iotlb.type); if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) { error_report("failed to write, fd=%d, errno=%d (%s)", @@ -151,7 +151,8 @@ static void vhost_vdpa_listener_begin_batch(struct vhost_vdpa *v) .iotlb.type = VHOST_IOTLB_BATCH_BEGIN, }; - trace_vhost_vdpa_listener_begin_batch(v, fd, msg.type, msg.iotlb.type); + trace_vhost_vdpa_listener_begin_batch(v->shared, fd, msg.type, + msg.iotlb.type); if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) { error_report("failed to write, fd=%d, errno=%d (%s)", fd, errno, strerror(errno)); @@ -186,7 +187,7 @@ static void vhost_vdpa_listener_commit(MemoryListener *listener) msg.type = v->msg_type; msg.iotlb.type = VHOST_IOTLB_BATCH_END; - trace_vhost_vdpa_listener_commit(v, fd, msg.type, msg.iotlb.type); + trace_vhost_vdpa_listener_commit(v->shared, fd, msg.type, msg.iotlb.type); if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) { error_report("failed to write, fd=%d, errno=%d (%s)", fd, errno, strerror(errno)); @@ -329,7 +330,8 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, if (unlikely((section->offset_within_address_space & ~page_mask) != (section->offset_within_region & ~page_mask))) { - trace_vhost_vdpa_listener_region_add_unaligned(v, section->mr->name, + trace_vhost_vdpa_listener_region_add_unaligned(v->shared, + section->mr->name, section->offset_within_address_space & ~page_mask, section->offset_within_region & ~page_mask); return; @@ -349,7 +351,7 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, section->offset_within_region + (iova - section->offset_within_address_space); - trace_vhost_vdpa_listener_region_add(v, iova, int128_get64(llend), + trace_vhost_vdpa_listener_region_add(v->shared, iova, int128_get64(llend), vaddr, section->readonly); llsize = int128_sub(llend, int128_make64(iova)); @@ -417,7 +419,8 @@ static void vhost_vdpa_listener_region_del(MemoryListener *listener, if (unlikely((section->offset_within_address_space & ~page_mask) != (section->offset_within_region & ~page_mask))) { - trace_vhost_vdpa_listener_region_del_unaligned(v, section->mr->name, + trace_vhost_vdpa_listener_region_del_unaligned(v->shared, + section->mr->name, section->offset_within_address_space & ~page_mask, section->offset_within_region & ~page_mask); return; @@ -426,7 +429,7 @@ static void vhost_vdpa_listener_region_del(MemoryListener *listener, iova = ROUND_UP(section->offset_within_address_space, page_size); llend = vhost_vdpa_section_end(section, page_mask); - trace_vhost_vdpa_listener_region_del(v, iova, + trace_vhost_vdpa_listener_region_del(v->shared, iova, int128_get64(int128_sub(llend, int128_one()))); if (int128_ge(int128_make64(iova), llend)) { @@ -587,12 +590,11 @@ static void vhost_vdpa_init_svq(struct vhost_dev *hdev, struct vhost_vdpa *v) static int vhost_vdpa_init(struct vhost_dev *dev, void *opaque, Error **errp) { - struct vhost_vdpa *v; + struct vhost_vdpa *v = opaque; assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_VDPA); - trace_vhost_vdpa_init(dev, opaque); + trace_vhost_vdpa_init(dev, v->shared, opaque); int ret; - v = opaque; v->dev = dev; dev->opaque = opaque ; v->listener = vhost_vdpa_memory_listener; diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index 637cac4edf..77905d1994 100644 --- a/hw/virtio/trace-events +++ b/hw/virtio/trace-events @@ -30,16 +30,16 @@ vhost_user_write(uint32_t req, uint32_t flags) "req:%d flags:0x%"PRIx32"" vhost_user_create_notifier(int idx, void *n) "idx:%d n:%p" # vhost-vdpa.c -vhost_vdpa_dma_map(void *vdpa, int fd, uint32_t msg_type, uint32_t asid, uint64_t iova, uint64_t size, uint64_t uaddr, uint8_t perm, uint8_t type) "vdpa:%p fd: %d msg_type: %"PRIu32" asid: %"PRIu32" iova: 0x%"PRIx64" size: 0x%"PRIx64" uaddr: 0x%"PRIx64" perm: 0x%"PRIx8" type: %"PRIu8 -vhost_vdpa_dma_unmap(void *vdpa, int fd, uint32_t msg_type, uint32_t asid, uint64_t iova, uint64_t size, uint8_t type) "vdpa:%p fd: %d msg_type: %"PRIu32" asid: %"PRIu32" iova: 0x%"PRIx64" size: 0x%"PRIx64" type: %"PRIu8 -vhost_vdpa_listener_begin_batch(void *v, int fd, uint32_t msg_type, uint8_t type) "vdpa:%p fd: %d msg_type: %"PRIu32" type: %"PRIu8 -vhost_vdpa_listener_commit(void *v, int fd, uint32_t msg_type, uint8_t type) "vdpa:%p fd: %d msg_type: %"PRIu32" type: %"PRIu8 -vhost_vdpa_listener_region_add_unaligned(void *v, const char *name, uint64_t offset_as, uint64_t offset_page) "vdpa: %p region %s offset_within_address_space %"PRIu64" offset_within_region %"PRIu64 +vhost_vdpa_dma_map(void *vdpa, int fd, uint32_t msg_type, uint32_t asid, uint64_t iova, uint64_t size, uint64_t uaddr, uint8_t perm, uint8_t type) "vdpa_shared:%p fd: %d msg_type: %"PRIu32" asid: %"PRIu32" iova: 0x%"PRIx64" size: 0x%"PRIx64" uaddr: 0x%"PRIx64" perm: 0x%"PRIx8" type: %"PRIu8 +vhost_vdpa_dma_unmap(void *vdpa, int fd, uint32_t msg_type, uint32_t asid, uint64_t iova, uint64_t size, uint8_t type) "vdpa_shared:%p fd: %d msg_type: %"PRIu32" asid: %"PRIu32" iova: 0x%"PRIx64" size: 0x%"PRIx64" type: %"PRIu8 +vhost_vdpa_listener_begin_batch(void *v, int fd, uint32_t msg_type, uint8_t type) "vdpa_shared:%p fd: %d msg_type: %"PRIu32" type: %"PRIu8 +vhost_vdpa_listener_commit(void *v, int fd, uint32_t msg_type, uint8_t type) "vdpa_shared:%p fd: %d msg_type: %"PRIu32" type: %"PRIu8 +vhost_vdpa_listener_region_add_unaligned(void *v, const char *name, uint64_t offset_as, uint64_t offset_page) "vdpa_shared: %p region %s offset_within_address_space %"PRIu64" offset_within_region %"PRIu64 vhost_vdpa_listener_region_add(void *vdpa, uint64_t iova, uint64_t llend, void *vaddr, bool readonly) "vdpa: %p iova 0x%"PRIx64" llend 0x%"PRIx64" vaddr: %p read-only: %d" -vhost_vdpa_listener_region_del_unaligned(void *v, const char *name, uint64_t offset_as, uint64_t offset_page) "vdpa: %p region %s offset_within_address_space %"PRIu64" offset_within_region %"PRIu64 +vhost_vdpa_listener_region_del_unaligned(void *v, const char *name, uint64_t offset_as, uint64_t offset_page) "vdpa_shared: %p region %s offset_within_address_space %"PRIu64" offset_within_region %"PRIu64 vhost_vdpa_listener_region_del(void *vdpa, uint64_t iova, uint64_t llend) "vdpa: %p iova 0x%"PRIx64" llend 0x%"PRIx64 vhost_vdpa_add_status(void *dev, uint8_t status) "dev: %p status: 0x%"PRIx8 -vhost_vdpa_init(void *dev, void *vdpa) "dev: %p vdpa: %p" +vhost_vdpa_init(void *dev, void *s, void *vdpa) "dev: %p, common dev: %p vdpa: %p" vhost_vdpa_cleanup(void *dev, void *vdpa) "dev: %p vdpa: %p" vhost_vdpa_memslots_limit(void *dev, int ret) "dev: %p = 0x%x" vhost_vdpa_set_mem_table(void *dev, uint32_t nregions, uint32_t padding) "dev: %p nregions: %"PRIu32" padding: 0x%"PRIx32 From patchwork Tue Dec 26 09:24:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880232 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=TRiqV849; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SzqDv5h6dz20Rq for ; Tue, 26 Dec 2023 20:27:43 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3ga-0003cO-6W; Tue, 26 Dec 2023 04:25:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gY-0003Qz-53 for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:14 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gT-00004t-K2 for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:13 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582708; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=I6kn2SB5MVcdzC6da0OFpj8gZVEzPbbWYo5A8QNCIGo=; b=TRiqV8497ZcVoJtFJjFCFKIQrNBpvDGBqUGoUmS4OEnqLbBo5TF/fM2Sbm+bKZMJhET1kT uk6Xxj7UV0TLkzUFVU1X26rtxt/RhuJqtnBBgOAN52X98vTy3QDa+JC/18WreB8NhkBKsu uALHHgHvYBRFSnlhszVBeLtmozT2a4w= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-637-biOnDW4eMZKv1RorCfnZ2w-1; Tue, 26 Dec 2023 04:25:06 -0500 X-MC-Unique: biOnDW4eMZKv1RorCfnZ2w-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-40d4a423c84so33780355e9.3 for ; Tue, 26 Dec 2023 01:25:06 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582702; x=1704187502; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=I6kn2SB5MVcdzC6da0OFpj8gZVEzPbbWYo5A8QNCIGo=; b=BQUIE9GVXiDHx7RjznqVBXsLsMWC9+w4i3MzpAYuLWHJ1LQdz6++oHw3h2YOwSalrf yeD+fitS/BOc2a6WS9XKnwFew5beT2OGLzxzl8RZmdjYyE1mYiIaS25L6bo5trgM9rRG LwS6x/TaVVk1G2P80bmS4p0gxuuX/0YevHKi1AI7FuDo2g3dVDYBWQInxNCeZZ+RU/3Z butEsAJCxGy4vCY0MXbyjR7xbMfbxxr5f74AN3rhZvwXhmLFS0k9IZQN2W9VCSDxbgY2 ka5IJVU1Auj/+Tbpbamrbi7Oef6sPGUIVjowzNYpXLEEZ/TX74mosfrJRyH9ClXzIABU C9gQ== X-Gm-Message-State: AOJu0Yy1NQFq4OvlSeW3O/NXHNnIWQoQUzFkNegLeodhumJ+9pji0xhN wSTrDkdOrFTeNFaJ2shcTS54a0EGNc4LI3ABgGx4IucqE1A8msq+zIKHnPp3YRiZretU+a0GhEH ToT0ZcPsjB94z6HHIvPP/b5nov6rrAQbOAwoUAEFd0THFKQplYgunGXLgdV2IPpOhK2DzodZnaE TN X-Received: by 2002:a05:600c:1554:b0:40c:2d88:1edd with SMTP id f20-20020a05600c155400b0040c2d881eddmr2430767wmg.130.1703582702387; Tue, 26 Dec 2023 01:25:02 -0800 (PST) X-Google-Smtp-Source: AGHT+IF/NuZ+Je4MAVZeGiG08xCZr6EqAbU/dHYAqkTMteh8IK70vPj3iXLk35j09Hox+rWJIbus5g== X-Received: by 2002:a05:600c:1554:b0:40c:2d88:1edd with SMTP id f20-20020a05600c155400b0040c2d881eddmr2430760wmg.130.1703582702094; Tue, 26 Dec 2023 01:25:02 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id j20-20020a05600c191400b0040b349c91acsm12195144wmq.16.2023.12.26.01.24.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:24:59 -0800 (PST) Date: Tue, 26 Dec 2023 04:24:58 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Eugenio =?utf-8?b?UMOpcmV6?= , Jason Wang , Lei Yang Subject: [PULL 14/21] vdpa: move file descriptor to vhost_vdpa_shared Message-ID: <5c93bd6fabc2d21bead140e2dfbd5ccf45d58815.1703582625.git.mst@redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.129.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Eugenio Pérez Next patches will register the vhost_vdpa memory listener while the VM is migrating at the destination, so we can map the memory to the device before stopping the VM at the source. The main goal is to reduce the downtime. However, the destination QEMU is unaware of which vhost_vdpa device will register its memory_listener. If the source guest has CVQ enabled, it will be the CVQ device. Otherwise, it will be the first one. Move the file descriptor to VhostVDPAShared so all vhost_vdpa can use it, rather than always in the first / last vhost_vdpa. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang Message-Id: <20231221174322.3130442-7-eperezma@redhat.com> Tested-by: Lei Yang Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- include/hw/virtio/vhost-vdpa.h | 2 +- hw/virtio/vdpa-dev.c | 2 +- hw/virtio/vhost-vdpa.c | 14 +++++++------- net/vhost-vdpa.c | 11 ++++------- 4 files changed, 13 insertions(+), 16 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 01e0f25e27..796a180afa 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -32,6 +32,7 @@ typedef struct VhostVDPAHostNotifier { /* Info shared by all vhost_vdpa device models */ typedef struct vhost_vdpa_shared { + int device_fd; struct vhost_vdpa_iova_range iova_range; /* IOVA mapping used by the Shadow Virtqueue */ @@ -42,7 +43,6 @@ typedef struct vhost_vdpa_shared { } VhostVDPAShared; typedef struct vhost_vdpa { - int device_fd; int index; uint32_t msg_type; bool iotlb_batch_begin_sent; diff --git a/hw/virtio/vdpa-dev.c b/hw/virtio/vdpa-dev.c index 457960d28a..8774986571 100644 --- a/hw/virtio/vdpa-dev.c +++ b/hw/virtio/vdpa-dev.c @@ -66,7 +66,6 @@ static void vhost_vdpa_device_realize(DeviceState *dev, Error **errp) if (*errp) { return; } - v->vdpa.device_fd = v->vhostfd; v->vdev_id = vhost_vdpa_device_get_u32(v->vhostfd, VHOST_VDPA_GET_DEVICE_ID, errp); @@ -115,6 +114,7 @@ static void vhost_vdpa_device_realize(DeviceState *dev, Error **errp) goto free_vqs; } v->vdpa.shared = g_new0(VhostVDPAShared, 1); + v->vdpa.shared->device_fd = v->vhostfd; v->vdpa.shared->iova_range = iova_range; ret = vhost_dev_init(&v->dev, &v->vdpa, VHOST_BACKEND_TYPE_VDPA, 0, NULL); diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 720cffbc08..09df150ef2 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -90,7 +90,7 @@ int vhost_vdpa_dma_map(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, hwaddr size, void *vaddr, bool readonly) { struct vhost_msg_v2 msg = {}; - int fd = v->device_fd; + int fd = v->shared->device_fd; int ret = 0; msg.type = v->msg_type; @@ -122,7 +122,7 @@ int vhost_vdpa_dma_unmap(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, hwaddr size) { struct vhost_msg_v2 msg = {}; - int fd = v->device_fd; + int fd = v->shared->device_fd; int ret = 0; msg.type = v->msg_type; @@ -145,7 +145,7 @@ int vhost_vdpa_dma_unmap(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, static void vhost_vdpa_listener_begin_batch(struct vhost_vdpa *v) { - int fd = v->device_fd; + int fd = v->shared->device_fd; struct vhost_msg_v2 msg = { .type = v->msg_type, .iotlb.type = VHOST_IOTLB_BATCH_BEGIN, @@ -174,7 +174,7 @@ static void vhost_vdpa_listener_commit(MemoryListener *listener) struct vhost_vdpa *v = container_of(listener, struct vhost_vdpa, listener); struct vhost_dev *dev = v->dev; struct vhost_msg_v2 msg = {}; - int fd = v->device_fd; + int fd = v->shared->device_fd; if (!(dev->backend_cap & (0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH))) { return; @@ -499,7 +499,7 @@ static int vhost_vdpa_call(struct vhost_dev *dev, unsigned long int request, void *arg) { struct vhost_vdpa *v = dev->opaque; - int fd = v->device_fd; + int fd = v->shared->device_fd; int ret; assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_VDPA); @@ -661,7 +661,7 @@ static int vhost_vdpa_host_notifier_init(struct vhost_dev *dev, int queue_index) struct vhost_vdpa *v = dev->opaque; VirtIODevice *vdev = dev->vdev; VhostVDPAHostNotifier *n; - int fd = v->device_fd; + int fd = v->shared->device_fd; void *addr; char *name; @@ -1290,7 +1290,7 @@ static void vhost_vdpa_suspend(struct vhost_dev *dev) if (dev->backend_cap & BIT_ULL(VHOST_BACKEND_F_SUSPEND)) { trace_vhost_vdpa_suspend(dev); - r = ioctl(v->device_fd, VHOST_VDPA_SUSPEND); + r = ioctl(v->shared->device_fd, VHOST_VDPA_SUSPEND); if (unlikely(r)) { error_report("Cannot suspend: %s(%d)", g_strerror(errno), errno); } else { diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index bf8e8327da..10cf0027de 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -235,14 +235,11 @@ static void vhost_vdpa_cleanup(NetClientState *nc) vhost_net_cleanup(s->vhost_net); g_free(s->vhost_net); s->vhost_net = NULL; - } - if (s->vhost_vdpa.device_fd >= 0) { - qemu_close(s->vhost_vdpa.device_fd); - s->vhost_vdpa.device_fd = -1; } if (s->vhost_vdpa.index != 0) { return; } + qemu_close(s->vhost_vdpa.shared->device_fd); g_free(s->vhost_vdpa.shared); } @@ -448,7 +445,7 @@ static int vhost_vdpa_set_address_space_id(struct vhost_vdpa *v, }; int r; - r = ioctl(v->device_fd, VHOST_VDPA_SET_GROUP_ASID, &asid); + r = ioctl(v->shared->device_fd, VHOST_VDPA_SET_GROUP_ASID, &asid); if (unlikely(r < 0)) { error_report("Can't set vq group %u asid %u, errno=%d (%s)", asid.index, asid.num, errno, g_strerror(errno)); @@ -544,7 +541,7 @@ static int vhost_vdpa_net_cvq_start(NetClientState *nc) return 0; } - cvq_group = vhost_vdpa_get_vring_group(v->device_fd, + cvq_group = vhost_vdpa_get_vring_group(v->shared->device_fd, v->dev->vq_index_end - 1, &err); if (unlikely(cvq_group < 0)) { @@ -1671,7 +1668,6 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, qemu_set_info_str(nc, TYPE_VHOST_VDPA); s = DO_UPCAST(VhostVDPAState, nc, nc); - s->vhost_vdpa.device_fd = vdpa_device_fd; s->vhost_vdpa.index = queue_pair_index; s->always_svq = svq; s->migration_state.notify = NULL; @@ -1680,6 +1676,7 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, vhost_vdpa_net_valid_svq_features(features, &s->vhost_vdpa.migration_blocker); s->vhost_vdpa.shared = g_new0(VhostVDPAShared, 1); + s->vhost_vdpa.shared->device_fd = vdpa_device_fd; s->vhost_vdpa.shared->iova_range = iova_range; s->vhost_vdpa.shared->shadow_data = svq; } else if (!is_datapath) { From patchwork Tue Dec 26 09:25:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880237 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FDWXM6ir; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SzqF23b5qz20Rq for ; Tue, 26 Dec 2023 20:27:50 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3ga-0003bZ-1h; Tue, 26 Dec 2023 04:25:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gX-0003QT-VU for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:14 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gW-00005F-59 for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:13 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582711; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o+wiRUysJK2+tJ0AobcVOPv/z3/GSAonKn8biwMKrf4=; b=FDWXM6irPDK8Ui8pwdW3JaiCgJonwujxQj0WdwN9eeCyRQqdw2c1mvZgA58C4PyvaiOlix x1idqzvcwWTb3TBPIQN+viBxpQaK4+aQ8Z1LHhjDY9DbLC5t/QLRnHC4o0Tv9AwkPyCdZa 5t4xup/h6yoPHBRyflVfK2QLZRYRc4Q= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-152-XZmLuou9N1m0MpMhNvfUmQ-1; Tue, 26 Dec 2023 04:25:09 -0500 X-MC-Unique: XZmLuou9N1m0MpMhNvfUmQ-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-40d4a29dca7so32251735e9.0 for ; Tue, 26 Dec 2023 01:25:09 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582707; x=1704187507; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=o+wiRUysJK2+tJ0AobcVOPv/z3/GSAonKn8biwMKrf4=; b=wp8QykSSrnGJ015PjwtRng4+OAyVly087puZIVpoESpV/s4e7HFpSWYB8D+ZUD32/Y hqMPOpTV+5yviYfAYVemlJ2AFu/EPgs2aKjwpnfVzkvADBIRqPE6DCEuQrs1TmWMiijr 0xtWGHHp241smbWWOfxTJlbTI8EpXcAmmPQXxwtrtYsciJbBGm1Hj8URx8wrRDql/P7z JmZaM++xLTtVgkKXkLFDu/vtGAuYvOhajjzl8AAKkbM++myLt9hJ0N1YHf7rbDhRam6Q /ioGXEqy0MPfkpTBxsORPix19JvqZyOm7mlTh1KKPnbp5QgkY5LO9IrgPidVPesF11lG BBkA== X-Gm-Message-State: AOJu0YyRfnKeAyvidVCpp6VCfhfdvQYIpEwUl4fyOCwphwkq7jXK6+ax cqTWd0AVfwHZlVaCmAzJhOZfNcL33GIJ44DXcXmUT2iwYRhHDXoI23HTJbzhowb5ZsXNLXVFwVy JU3pY/uh3GrMurZYElI2VmKSDkXWZwE307Fy4fHed/e4V2Gm83AeKszZHHQA7Ih6hvJ2bREWPg/ YU X-Received: by 2002:a1c:7714:0:b0:40d:59be:b2e2 with SMTP id t20-20020a1c7714000000b0040d59beb2e2mr431242wmi.79.1703582707658; Tue, 26 Dec 2023 01:25:07 -0800 (PST) X-Google-Smtp-Source: AGHT+IEPQGS/Zy3GFIJi3zERaY5Q1QJURCs2zSAd5Bb30tvbJlvWKJi2N2kSxcKT3FI9u97E0+417Q== X-Received: by 2002:a1c:7714:0:b0:40d:59be:b2e2 with SMTP id t20-20020a1c7714000000b0040d59beb2e2mr431231wmi.79.1703582707379; Tue, 26 Dec 2023 01:25:07 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id k13-20020a05600c1c8d00b0040d3dc52665sm19059502wms.21.2023.12.26.01.25.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:25:04 -0800 (PST) Date: Tue, 26 Dec 2023 04:25:02 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Eugenio =?utf-8?b?UMOpcmV6?= , Jason Wang , Lei Yang Subject: [PULL 15/21] vdpa: move iotlb_batch_begin_sent to vhost_vdpa_shared Message-ID: <6b8e7650e33d3aee54d5f25159c21d97ebd2322d.1703582625.git.mst@redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.133.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Eugenio Pérez Next patches will register the vhost_vdpa memory listener while the VM is migrating at the destination, so we can map the memory to the device before stopping the VM at the source. The main goal is to reduce the downtime. However, the destination QEMU is unaware of which vhost_vdpa device will register its memory_listener. If the source guest has CVQ enabled, it will be the CVQ device. Otherwise, it will be the first one. Move the iotlb_batch_begin_sent member to VhostVDPAShared so all vhost_vdpa can use it, rather than always in the first / last vhost_vdpa. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang Message-Id: <20231221174322.3130442-8-eperezma@redhat.com> Tested-by: Lei Yang Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- include/hw/virtio/vhost-vdpa.h | 3 ++- hw/virtio/vhost-vdpa.c | 8 ++++---- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 796a180afa..05219bbcf7 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -38,6 +38,8 @@ typedef struct vhost_vdpa_shared { /* IOVA mapping used by the Shadow Virtqueue */ VhostIOVATree *iova_tree; + bool iotlb_batch_begin_sent; + /* Vdpa must send shadow addresses as IOTLB key for data queues, not GPA */ bool shadow_data; } VhostVDPAShared; @@ -45,7 +47,6 @@ typedef struct vhost_vdpa_shared { typedef struct vhost_vdpa { int index; uint32_t msg_type; - bool iotlb_batch_begin_sent; uint32_t address_space_id; MemoryListener listener; uint64_t acked_features; diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 09df150ef2..2ecaedb686 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -162,11 +162,11 @@ static void vhost_vdpa_listener_begin_batch(struct vhost_vdpa *v) static void vhost_vdpa_iotlb_batch_begin_once(struct vhost_vdpa *v) { if (v->dev->backend_cap & (0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH) && - !v->iotlb_batch_begin_sent) { + !v->shared->iotlb_batch_begin_sent) { vhost_vdpa_listener_begin_batch(v); } - v->iotlb_batch_begin_sent = true; + v->shared->iotlb_batch_begin_sent = true; } static void vhost_vdpa_listener_commit(MemoryListener *listener) @@ -180,7 +180,7 @@ static void vhost_vdpa_listener_commit(MemoryListener *listener) return; } - if (!v->iotlb_batch_begin_sent) { + if (!v->shared->iotlb_batch_begin_sent) { return; } @@ -193,7 +193,7 @@ static void vhost_vdpa_listener_commit(MemoryListener *listener) fd, errno, strerror(errno)); } - v->iotlb_batch_begin_sent = false; + v->shared->iotlb_batch_begin_sent = false; } static void vhost_vdpa_iommu_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) From patchwork Tue Dec 26 09:25:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880227 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KitDqQoz; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SzqBw58Fyz1ydb for ; Tue, 26 Dec 2023 20:26:00 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3gc-0003mY-C2; Tue, 26 Dec 2023 04:25:18 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3ga-0003fl-LF for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:16 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gY-00005k-SD for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:16 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582714; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D+Sy8LokVxiUP811D8nldNN2ZDdnJt8aAbJYeHozE3g=; b=KitDqQoz78u2vkddC9YooDf40nlD6Pskx+G/TGu9jXJl4u7SH9Jg/MhYX8t69U0jubVq2k ZlxmxEitUxyYsRVo8be0wm6zQPyDlJ/GS2HQO4uq0ONDdXHaZwq7RhE745jPkMzNEC2Yxz sbrzzdbxsAVaneduDc3YmJIv0MYtCPA= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-357-NfpOrJI6MB6OGvueCzRemg-1; Tue, 26 Dec 2023 04:25:12 -0500 X-MC-Unique: NfpOrJI6MB6OGvueCzRemg-1 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-3368abe10c5so3113216f8f.0 for ; Tue, 26 Dec 2023 01:25:12 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582711; x=1704187511; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=D+Sy8LokVxiUP811D8nldNN2ZDdnJt8aAbJYeHozE3g=; b=QxWAWIi/6YYr1AE+Hbrrzj7K5pIDt23Spu3RBvW2Nf8vSVaEI1M0Ews72RWM0Zpt0e OQYAhZy+ezUsWMoAN3QYYGx1AI5RVoxDA8sx0TEyxptLTEbStYQDiSaO2SVY3+k4XAC2 TOnbRD30Hzj52EfBBjOj3XpBiVJeMbbM1Pyltjyohus8bf5u1oWoqmvKB8lqe+GONgid B5mR75mtpFf/Xb7qadsszUct0SPtIlk+Kl3CBLb6GhVRf/gyN204BKlH42btyjzFyfza RnqP1MLp9wQjbHxHSBRWRN4LOl9VLrUX0SQG3KxNyfwr9h8dVSY3C/Qi+QzVMOGgq3ur DE7g== X-Gm-Message-State: AOJu0YzG+SyurQ3tVUxxX4TayzCdakL3KH/C1Y66ECYi4EmIN42RsktS D3jOnCYhuJAocvafCfeiArF9SxJoPmzoPdTcmuGYJfWeOTwhjHi3Qki7Z+sixRWpmHUaJFU+B8C 1dP2dvhDS5XI+R/U0xZMshj/zgWpSdB7nPgoZzVKGta11t0jAMAdkADOVqwGjhGADn6NF4z5dsY 9j X-Received: by 2002:adf:e30a:0:b0:336:7656:ee19 with SMTP id b10-20020adfe30a000000b003367656ee19mr2922257wrj.92.1703582710963; Tue, 26 Dec 2023 01:25:10 -0800 (PST) X-Google-Smtp-Source: AGHT+IED3cPVOnpCAeTIgT6HxEDadxdWPGRU88tb7gjLqILvBNvT2h829kvGs+Xnw29EQ6k/2bPEDQ== X-Received: by 2002:adf:e30a:0:b0:336:7656:ee19 with SMTP id b10-20020adfe30a000000b003367656ee19mr2922248wrj.92.1703582710563; Tue, 26 Dec 2023 01:25:10 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id g3-20020a5d4883000000b00336e32338f3sm977229wrq.70.2023.12.26.01.25.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:25:09 -0800 (PST) Date: Tue, 26 Dec 2023 04:25:07 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Eugenio =?utf-8?b?UMOpcmV6?= , Jason Wang , Lei Yang Subject: [PULL 16/21] vdpa: move backend_cap to vhost_vdpa_shared Message-ID: <75478072fdbb26ca7f704cf1da04712cf5ea3c3a.1703582625.git.mst@redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.129.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Eugenio Pérez Next patches will register the vhost_vdpa memory listener while the VM is migrating at the destination, so we can map the memory to the device before stopping the VM at the source. The main goal is to reduce the downtime. However, the destination QEMU is unaware of which vhost_vdpa device will register its memory_listener. If the source guest has CVQ enabled, it will be the CVQ device. Otherwise, it will be the first one. Move the backend_cap member to VhostVDPAShared so all vhost_vdpa can use it, rather than always in the first / last vhost_vdpa. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang Message-Id: <20231221174322.3130442-9-eperezma@redhat.com> Tested-by: Lei Yang Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- include/hw/virtio/vhost-vdpa.h | 3 +++ hw/virtio/vhost-vdpa.c | 8 +++++--- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 05219bbcf7..11ac14085a 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -38,6 +38,9 @@ typedef struct vhost_vdpa_shared { /* IOVA mapping used by the Shadow Virtqueue */ VhostIOVATree *iova_tree; + /* Copy of backend features */ + uint64_t backend_cap; + bool iotlb_batch_begin_sent; /* Vdpa must send shadow addresses as IOTLB key for data queues, not GPA */ diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 2ecaedb686..99597c3179 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -161,7 +161,7 @@ static void vhost_vdpa_listener_begin_batch(struct vhost_vdpa *v) static void vhost_vdpa_iotlb_batch_begin_once(struct vhost_vdpa *v) { - if (v->dev->backend_cap & (0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH) && + if (v->shared->backend_cap & (0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH) && !v->shared->iotlb_batch_begin_sent) { vhost_vdpa_listener_begin_batch(v); } @@ -172,11 +172,10 @@ static void vhost_vdpa_iotlb_batch_begin_once(struct vhost_vdpa *v) static void vhost_vdpa_listener_commit(MemoryListener *listener) { struct vhost_vdpa *v = container_of(listener, struct vhost_vdpa, listener); - struct vhost_dev *dev = v->dev; struct vhost_msg_v2 msg = {}; int fd = v->shared->device_fd; - if (!(dev->backend_cap & (0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH))) { + if (!(v->shared->backend_cap & (0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH))) { return; } @@ -838,6 +837,8 @@ static int vhost_vdpa_set_features(struct vhost_dev *dev, static int vhost_vdpa_set_backend_cap(struct vhost_dev *dev) { + struct vhost_vdpa *v = dev->opaque; + uint64_t features; uint64_t f = 0x1ULL << VHOST_BACKEND_F_IOTLB_MSG_V2 | 0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH | @@ -859,6 +860,7 @@ static int vhost_vdpa_set_backend_cap(struct vhost_dev *dev) } dev->backend_cap = features; + v->shared->backend_cap = features; return 0; } From patchwork Tue Dec 26 09:25:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880233 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=N4qPeSWl; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SzqDw14Pyz23dS for ; Tue, 26 Dec 2023 20:27:43 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3gh-0004Ns-Al; Tue, 26 Dec 2023 04:25:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gf-0004Cy-I4 for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:21 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gc-00006Y-OB for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582717; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5DnbzKStrGIR1igJTyuESQsjBjVqw22u5ZPR3rhCzDg=; b=N4qPeSWlBazt4J9hhCeQbvjd96Q6V5ha3mP3T8tJDmQoEKncHArNMC9ip8vEzvOmQo5X6/ UQM+v/D/PKMb4GV7S8chmTJddJSmN9/vjR+v/uAa1CCx+CoQfjNt2rxnT3y3xR7AvZfdOM EYTBvJuoMyRPIVjIRKrEgNI1SW2Pn2I= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-607-r1gcPDeOOSqQUahIlLqCng-1; Tue, 26 Dec 2023 04:25:15 -0500 X-MC-Unique: r1gcPDeOOSqQUahIlLqCng-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-40d38c09797so40899975e9.0 for ; Tue, 26 Dec 2023 01:25:15 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582714; x=1704187514; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=5DnbzKStrGIR1igJTyuESQsjBjVqw22u5ZPR3rhCzDg=; b=aYN9XmBTTPn4WYlhpB2vEB8elO1/iigbXEisG2T7+zKwUe1O/XCLar+82y0TUEW0oO kJJq0jdcNdklBKLcuiYKN/z2/guDoLwLo862J/NO94S7p95gArM53dbKDWk3LLqycron MIBJBFOPj2VxYQv+bfv5SRy6MWaSj9/I6Kdv3+gcwZkyK+j6nI8dxTpQcxMAjzYpsBXn IwT23b5Tn3E/ZfJ4pFV8f3zFLymqtCrJW+0XfZdYvNZq3gZyDem3jI3MxuCOQ+X0GZlz Cm6LmoZpe3u3YnzJjBM+DeD4LmhdX+rM9EIC11U2hIna9iKvZPIR8+LahTzcpum5YFOf M1pw== X-Gm-Message-State: AOJu0YwVw2WLjYOfhPowyK5tdnY+BJb6ogDWqAbAngeTZPN6xtX26bgZ I2UoLizpUgecKffu3YNBcjOj1zGE0KBCIE58iOPUNyIfE8U4U3I9BIYfctVxtO+z3SLbiInXwg2 XjAXojskzWr4y6lxaaXS6g2BIltTz/X/33w65Gfb/faogoTywERerERQKzKk86FzT55UdxrvjAE YC X-Received: by 2002:a05:600c:3154:b0:40c:330f:f2f9 with SMTP id h20-20020a05600c315400b0040c330ff2f9mr3948360wmo.125.1703582714038; Tue, 26 Dec 2023 01:25:14 -0800 (PST) X-Google-Smtp-Source: AGHT+IG606jeWW4cg1bSqHq4Ok9u8V3zlru8apZRp4F0L4X0hNP+6gqcb4Qswfht8FFGfU+VVko2fw== X-Received: by 2002:a05:600c:3154:b0:40c:330f:f2f9 with SMTP id h20-20020a05600c315400b0040c330ff2f9mr3948350wmo.125.1703582713680; Tue, 26 Dec 2023 01:25:13 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id fm13-20020a05600c0c0d00b0040b37f107c4sm20307608wmb.16.2023.12.26.01.25.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:25:13 -0800 (PST) Date: Tue, 26 Dec 2023 04:25:10 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Eugenio =?utf-8?b?UMOpcmV6?= , Jason Wang , Lei Yang Subject: [PULL 17/21] vdpa: remove msg type of vhost_vdpa Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.129.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Eugenio Pérez It is always VHOST_IOTLB_MSG_V2. We can always make it back per vhost_dev if needed. This change makes easier for vhost_vdpa_map and unmap not to depend on vhost_vdpa but only in VhostVDPAShared. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang Message-Id: <20231221174322.3130442-10-eperezma@redhat.com> Tested-by: Lei Yang Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- include/hw/virtio/vhost-vdpa.h | 1 - hw/virtio/vhost-vdpa.c | 9 ++++----- 2 files changed, 4 insertions(+), 6 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 11ac14085a..5bd964dac5 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -49,7 +49,6 @@ typedef struct vhost_vdpa_shared { typedef struct vhost_vdpa { int index; - uint32_t msg_type; uint32_t address_space_id; MemoryListener listener; uint64_t acked_features; diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 99597c3179..cbfcf18883 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -93,7 +93,7 @@ int vhost_vdpa_dma_map(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, int fd = v->shared->device_fd; int ret = 0; - msg.type = v->msg_type; + msg.type = VHOST_IOTLB_MSG_V2; msg.asid = asid; msg.iotlb.iova = iova; msg.iotlb.size = size; @@ -125,7 +125,7 @@ int vhost_vdpa_dma_unmap(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, int fd = v->shared->device_fd; int ret = 0; - msg.type = v->msg_type; + msg.type = VHOST_IOTLB_MSG_V2; msg.asid = asid; msg.iotlb.iova = iova; msg.iotlb.size = size; @@ -147,7 +147,7 @@ static void vhost_vdpa_listener_begin_batch(struct vhost_vdpa *v) { int fd = v->shared->device_fd; struct vhost_msg_v2 msg = { - .type = v->msg_type, + .type = VHOST_IOTLB_MSG_V2, .iotlb.type = VHOST_IOTLB_BATCH_BEGIN, }; @@ -183,7 +183,7 @@ static void vhost_vdpa_listener_commit(MemoryListener *listener) return; } - msg.type = v->msg_type; + msg.type = VHOST_IOTLB_MSG_V2; msg.iotlb.type = VHOST_IOTLB_BATCH_END; trace_vhost_vdpa_listener_commit(v->shared, fd, msg.type, msg.iotlb.type); @@ -597,7 +597,6 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void *opaque, Error **errp) v->dev = dev; dev->opaque = opaque ; v->listener = vhost_vdpa_memory_listener; - v->msg_type = VHOST_IOTLB_MSG_V2; vhost_vdpa_init_svq(dev, v); error_propagate(&dev->migration_blocker, v->migration_blocker); From patchwork Tue Dec 26 09:25:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880228 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=BBUZpdM7; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SzqBx70Kcz20Rq for ; Tue, 26 Dec 2023 20:26:01 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3gi-0004TO-GL; Tue, 26 Dec 2023 04:25:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gg-0004Gv-9l for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:22 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3ge-000091-JC for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582720; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=M5QGJ+eIeZAekNiSAn72joJJNqpG535Ykhgx4/GAM48=; b=BBUZpdM7hadzNgvKvg3ufEgx8GFbK9jurfxf0Jqiu7Ik/1YaVfv/8jDBf2lWxsbnsn6Exf Gw2LPJAyrMr6cUvVtnzTHoyT/obdeYw1LoM2OG4aTSBO1Zhv9XaLOtNG7rPVzvc4XRauVl EUZRaWOq0o1rlW/CBodsG99JJ4Y/ZRc= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-686-FnKprD_aNoiMvaYgueS5XQ-1; Tue, 26 Dec 2023 04:25:18 -0500 X-MC-Unique: FnKprD_aNoiMvaYgueS5XQ-1 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-40d2fa6b23eso47176955e9.2 for ; Tue, 26 Dec 2023 01:25:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582717; x=1704187517; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=M5QGJ+eIeZAekNiSAn72joJJNqpG535Ykhgx4/GAM48=; b=eGueebsttDmv6heCKp/HpI1aAxu9y1VfUCnLBkiBWWA8KdgNfJLc2Zl4N1RbzkNYyu ki0ubPwPBHVWp41FwFV0gHujI2o7NcBpJ7zMsTyZjbDZ5O9CgPO9151lWt+o+GyaOWMs mDMKN2sQOI8FHKZdNlXziF4e+VJEqbdtOpV7RnN7SO12V89l7NC2OTWeGBFf4PJJsmr9 xq32Pz7ne4YyNnVNfbXLGXvAthORa9UcwAAPl1J4zd4dKIuaSx7v2eOonpDBbtkdUP9b fl7dUSHDCyor86yIu3SSDx//PgUc7kTwm3GwVajKbuJmZU77Atjx1AKYX/jrlxsCT/+e APnQ== X-Gm-Message-State: AOJu0YwDMLTVE+jAV/d0FdCnu5Ja5OEL4k2rCv+85uQ9jCkEF0+Orabe W3QI/zo/bf/feVjrmDENO7ai5FzsDM9cppZgmokaPanWewHbzXwiUcsDAEO6t1WH027+17Jc5Sy vMdCgr4OcBhz0DKnwYLeiGvtAacvTXQ5p4Ov1iXbO/zSkgYvb8K+37Iq+lmK5DIjQsBen3nU8fE lJ X-Received: by 2002:a05:600c:3412:b0:40d:379a:93a2 with SMTP id y18-20020a05600c341200b0040d379a93a2mr3060056wmp.237.1703582716913; Tue, 26 Dec 2023 01:25:16 -0800 (PST) X-Google-Smtp-Source: AGHT+IE7NJIV0ZMPmyaMzls7wxGV7TFfjjRkBgGKMNMGpaCQSXbQxHU7w+cuaSj6C+58sPEluYlrwA== X-Received: by 2002:a05:600c:3412:b0:40d:379a:93a2 with SMTP id y18-20020a05600c341200b0040d379a93a2mr3060047wmp.237.1703582716497; Tue, 26 Dec 2023 01:25:16 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id h5-20020a05600c314500b0040b2a52ecaasm27801153wmo.2.2023.12.26.01.25.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:25:15 -0800 (PST) Date: Tue, 26 Dec 2023 04:25:13 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Eugenio =?utf-8?b?UMOpcmV6?= , Jason Wang , Lei Yang Subject: [PULL 18/21] vdpa: move iommu_list to vhost_vdpa_shared Message-ID: <8e0139ee409fcbedc940b61d09c022657e26bab3.1703582625.git.mst@redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.133.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Eugenio Pérez Next patches will register the vhost_vdpa memory listener while the VM is migrating at the destination, so we can map the memory to the device before stopping the VM at the source. The main goal is to reduce the downtime. However, the destination QEMU is unaware of which vhost_vdpa device will register its memory_listener. If the source guest has CVQ enabled, it will be the CVQ device. Otherwise, it will be the first one. Move the iommu_list member to VhostVDPAShared so all vhost_vdpa can use it, rather than always in the first / last vhost_vdpa. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang Message-Id: <20231221174322.3130442-11-eperezma@redhat.com> Tested-by: Lei Yang Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- include/hw/virtio/vhost-vdpa.h | 2 +- hw/virtio/vhost-vdpa.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 5bd964dac5..3880b9e7f2 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -34,6 +34,7 @@ typedef struct VhostVDPAHostNotifier { typedef struct vhost_vdpa_shared { int device_fd; struct vhost_vdpa_iova_range iova_range; + QLIST_HEAD(, vdpa_iommu) iommu_list; /* IOVA mapping used by the Shadow Virtqueue */ VhostIOVATree *iova_tree; @@ -62,7 +63,6 @@ typedef struct vhost_vdpa { struct vhost_dev *dev; Error *migration_blocker; VhostVDPAHostNotifier notifier[VIRTIO_QUEUE_MAX]; - QLIST_HEAD(, vdpa_iommu) iommu_list; IOMMUNotifier n; } VhostVDPA; diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index cbfcf18883..a2713b9f0b 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -279,7 +279,7 @@ static void vhost_vdpa_iommu_region_add(MemoryListener *listener, return; } - QLIST_INSERT_HEAD(&v->iommu_list, iommu, iommu_next); + QLIST_INSERT_HEAD(&v->shared->iommu_list, iommu, iommu_next); memory_region_iommu_replay(iommu->iommu_mr, &iommu->n); return; @@ -292,7 +292,7 @@ static void vhost_vdpa_iommu_region_del(MemoryListener *listener, struct vdpa_iommu *iommu; - QLIST_FOREACH(iommu, &v->iommu_list, iommu_next) + QLIST_FOREACH(iommu, &v->shared->iommu_list, iommu_next) { if (MEMORY_REGION(iommu->iommu_mr) == section->mr && iommu->n.start == section->offset_within_region) { From patchwork Tue Dec 26 09:25:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880234 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KaoFdhju; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SzqDw14J3z23dP for ; Tue, 26 Dec 2023 20:27:43 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3gl-0004jK-9t; Tue, 26 Dec 2023 04:25:27 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gj-0004aI-RS for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:25 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gh-0000A3-NY for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:25 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582723; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=83xRNao3wYJ+xT44DNwZkqp56Au3afhfwnreo+lS1GM=; b=KaoFdhjuTAMAWpAzrR69XuRj9nToBWrw1w62aSLFblKzrawDkfUxaK+FfQjNt14407uKXy pCSKYFcLM6q0bnymOSjS6KEH3hBW6qZsjGTLQDbt3YF0IFlSUIVXACWAty6Xl1YaHnF5Ht 4IHJFyiLSLzcDwUOMIe7GEr6nzBSjZY= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-479-7ebDJWZNPE6tPNpExdRZdg-1; Tue, 26 Dec 2023 04:25:21 -0500 X-MC-Unique: 7ebDJWZNPE6tPNpExdRZdg-1 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-33666ece1c6so3278278f8f.2 for ; Tue, 26 Dec 2023 01:25:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582720; x=1704187520; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=83xRNao3wYJ+xT44DNwZkqp56Au3afhfwnreo+lS1GM=; b=BXZIWz4WGo7K9WXwdeoL1Ro3LAfnptC006R/VzywVO9eTyh3GKX4+z3p30zuSw2fwO /0tEZOPeD9OI6GIZ6+gqYBpSMw6Zx9H5hkgJZQvdLjNH5Og+8PfkxrN4j7dTwQ5U5aAm O60UTpKl/CforDpBHLSGKhe9icm5WxjnlqOdiz0aPABgNV0r7dfX9BQSsh5dOirGpOvL kz78Lb95NZV3CTXC9HcUy1XRDpJWDQwFMgpEGNZPpUCO3ef6KGMTMMn1T8W6JZiaKUpa BBH/bfy7ZcnbgoaZ96YgNO0HpwysiWvJ5sLkRdehc8nXlhYoa+w7xJQw8uALhhOQgdBI BQ9w== X-Gm-Message-State: AOJu0Yxcrhk0HWtOVZlyHZ7YF/YCASIevbzEu0GLUPLUqgoQxWKrHgVB KJ5++2d6qVz3JbN8kpNL8UYAFV+BeC78lnGILxOzj9zPHohEz9XtYCzrwh86iwGI7QXc7rpepVU ZsyfkBoW4zv7/6yTPjxLFTpeo+XM4aQFQWxRLEJJbe6u+cjzBZLBsfrVQBl000WqteHeojwuEof Jj X-Received: by 2002:a05:6000:2a9:b0:336:c82b:dc0e with SMTP id l9-20020a05600002a900b00336c82bdc0emr764490wry.201.1703582719688; Tue, 26 Dec 2023 01:25:19 -0800 (PST) X-Google-Smtp-Source: AGHT+IFowlAp/ghY0eUmtk4TYG/aVfpUY80Y2+213mI5rbU6phxChWXYabYg1AHnFrZ7fj6Vb38n9A== X-Received: by 2002:a05:6000:2a9:b0:336:c82b:dc0e with SMTP id l9-20020a05600002a900b00336c82bdc0emr764481wry.201.1703582719307; Tue, 26 Dec 2023 01:25:19 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id a8-20020a5d53c8000000b00336905ad6d4sm9741753wrw.86.2023.12.26.01.25.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:25:18 -0800 (PST) Date: Tue, 26 Dec 2023 04:25:16 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Eugenio =?utf-8?b?UMOpcmV6?= , Jason Wang , Lei Yang Subject: [PULL 19/21] vdpa: use VhostVDPAShared in vdpa_dma_map and unmap Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.129.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Eugenio Pérez The callers only have the shared information by the end of this series. Start converting this functions. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang Message-Id: <20231221174322.3130442-12-eperezma@redhat.com> Tested-by: Lei Yang Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- include/hw/virtio/vhost-vdpa.h | 4 +-- hw/virtio/vhost-vdpa.c | 50 +++++++++++++++++----------------- net/vhost-vdpa.c | 5 ++-- 3 files changed, 30 insertions(+), 29 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 3880b9e7f2..705c754776 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -69,9 +69,9 @@ typedef struct vhost_vdpa { int vhost_vdpa_get_iova_range(int fd, struct vhost_vdpa_iova_range *iova_range); int vhost_vdpa_set_vring_ready(struct vhost_vdpa *v, unsigned idx); -int vhost_vdpa_dma_map(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, +int vhost_vdpa_dma_map(VhostVDPAShared *s, uint32_t asid, hwaddr iova, hwaddr size, void *vaddr, bool readonly); -int vhost_vdpa_dma_unmap(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, +int vhost_vdpa_dma_unmap(VhostVDPAShared *s, uint32_t asid, hwaddr iova, hwaddr size); typedef struct vdpa_iommu { diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index a2713b9f0b..ada3cc5d7c 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -86,11 +86,11 @@ static bool vhost_vdpa_listener_skipped_section(MemoryRegionSection *section, * The caller must set asid = 0 if the device does not support asid. * This is not an ABI break since it is set to 0 by the initializer anyway. */ -int vhost_vdpa_dma_map(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, +int vhost_vdpa_dma_map(VhostVDPAShared *s, uint32_t asid, hwaddr iova, hwaddr size, void *vaddr, bool readonly) { struct vhost_msg_v2 msg = {}; - int fd = v->shared->device_fd; + int fd = s->device_fd; int ret = 0; msg.type = VHOST_IOTLB_MSG_V2; @@ -101,7 +101,7 @@ int vhost_vdpa_dma_map(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, msg.iotlb.perm = readonly ? VHOST_ACCESS_RO : VHOST_ACCESS_RW; msg.iotlb.type = VHOST_IOTLB_UPDATE; - trace_vhost_vdpa_dma_map(v->shared, fd, msg.type, msg.asid, msg.iotlb.iova, + trace_vhost_vdpa_dma_map(s, fd, msg.type, msg.asid, msg.iotlb.iova, msg.iotlb.size, msg.iotlb.uaddr, msg.iotlb.perm, msg.iotlb.type); @@ -118,11 +118,11 @@ int vhost_vdpa_dma_map(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, * The caller must set asid = 0 if the device does not support asid. * This is not an ABI break since it is set to 0 by the initializer anyway. */ -int vhost_vdpa_dma_unmap(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, +int vhost_vdpa_dma_unmap(VhostVDPAShared *s, uint32_t asid, hwaddr iova, hwaddr size) { struct vhost_msg_v2 msg = {}; - int fd = v->shared->device_fd; + int fd = s->device_fd; int ret = 0; msg.type = VHOST_IOTLB_MSG_V2; @@ -131,8 +131,8 @@ int vhost_vdpa_dma_unmap(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, msg.iotlb.size = size; msg.iotlb.type = VHOST_IOTLB_INVALIDATE; - trace_vhost_vdpa_dma_unmap(v->shared, fd, msg.type, msg.asid, - msg.iotlb.iova, msg.iotlb.size, msg.iotlb.type); + trace_vhost_vdpa_dma_unmap(s, fd, msg.type, msg.asid, msg.iotlb.iova, + msg.iotlb.size, msg.iotlb.type); if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) { error_report("failed to write, fd=%d, errno=%d (%s)", @@ -143,30 +143,29 @@ int vhost_vdpa_dma_unmap(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, return ret; } -static void vhost_vdpa_listener_begin_batch(struct vhost_vdpa *v) +static void vhost_vdpa_listener_begin_batch(VhostVDPAShared *s) { - int fd = v->shared->device_fd; + int fd = s->device_fd; struct vhost_msg_v2 msg = { .type = VHOST_IOTLB_MSG_V2, .iotlb.type = VHOST_IOTLB_BATCH_BEGIN, }; - trace_vhost_vdpa_listener_begin_batch(v->shared, fd, msg.type, - msg.iotlb.type); + trace_vhost_vdpa_listener_begin_batch(s, fd, msg.type, msg.iotlb.type); if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) { error_report("failed to write, fd=%d, errno=%d (%s)", fd, errno, strerror(errno)); } } -static void vhost_vdpa_iotlb_batch_begin_once(struct vhost_vdpa *v) +static void vhost_vdpa_iotlb_batch_begin_once(VhostVDPAShared *s) { - if (v->shared->backend_cap & (0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH) && - !v->shared->iotlb_batch_begin_sent) { - vhost_vdpa_listener_begin_batch(v); + if (s->backend_cap & (0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH) && + !s->iotlb_batch_begin_sent) { + vhost_vdpa_listener_begin_batch(s); } - v->shared->iotlb_batch_begin_sent = true; + s->iotlb_batch_begin_sent = true; } static void vhost_vdpa_listener_commit(MemoryListener *listener) @@ -226,7 +225,7 @@ static void vhost_vdpa_iommu_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) if (!memory_get_xlat_addr(iotlb, &vaddr, NULL, &read_only, NULL)) { return; } - ret = vhost_vdpa_dma_map(v, VHOST_VDPA_GUEST_PA_ASID, iova, + ret = vhost_vdpa_dma_map(v->shared, VHOST_VDPA_GUEST_PA_ASID, iova, iotlb->addr_mask + 1, vaddr, read_only); if (ret) { error_report("vhost_vdpa_dma_map(%p, 0x%" HWADDR_PRIx ", " @@ -234,7 +233,7 @@ static void vhost_vdpa_iommu_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) v, iova, iotlb->addr_mask + 1, vaddr, ret); } } else { - ret = vhost_vdpa_dma_unmap(v, VHOST_VDPA_GUEST_PA_ASID, iova, + ret = vhost_vdpa_dma_unmap(v->shared, VHOST_VDPA_GUEST_PA_ASID, iova, iotlb->addr_mask + 1); if (ret) { error_report("vhost_vdpa_dma_unmap(%p, 0x%" HWADDR_PRIx ", " @@ -370,8 +369,8 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, iova = mem_region.iova; } - vhost_vdpa_iotlb_batch_begin_once(v); - ret = vhost_vdpa_dma_map(v, VHOST_VDPA_GUEST_PA_ASID, iova, + vhost_vdpa_iotlb_batch_begin_once(v->shared); + ret = vhost_vdpa_dma_map(v->shared, VHOST_VDPA_GUEST_PA_ASID, iova, int128_get64(llsize), vaddr, section->readonly); if (ret) { error_report("vhost vdpa map fail!"); @@ -455,13 +454,13 @@ static void vhost_vdpa_listener_region_del(MemoryListener *listener, iova = result->iova; vhost_iova_tree_remove(v->shared->iova_tree, *result); } - vhost_vdpa_iotlb_batch_begin_once(v); + vhost_vdpa_iotlb_batch_begin_once(v->shared); /* * The unmap ioctl doesn't accept a full 64-bit. need to check it */ if (int128_eq(llsize, int128_2_64())) { llsize = int128_rshift(llsize, 1); - ret = vhost_vdpa_dma_unmap(v, VHOST_VDPA_GUEST_PA_ASID, iova, + ret = vhost_vdpa_dma_unmap(v->shared, VHOST_VDPA_GUEST_PA_ASID, iova, int128_get64(llsize)); if (ret) { @@ -471,7 +470,7 @@ static void vhost_vdpa_listener_region_del(MemoryListener *listener, } iova += int128_get64(llsize); } - ret = vhost_vdpa_dma_unmap(v, VHOST_VDPA_GUEST_PA_ASID, iova, + ret = vhost_vdpa_dma_unmap(v->shared, VHOST_VDPA_GUEST_PA_ASID, iova, int128_get64(llsize)); if (ret) { @@ -1081,7 +1080,8 @@ static void vhost_vdpa_svq_unmap_ring(struct vhost_vdpa *v, hwaddr addr) } size = ROUND_UP(result->size, qemu_real_host_page_size()); - r = vhost_vdpa_dma_unmap(v, v->address_space_id, result->iova, size); + r = vhost_vdpa_dma_unmap(v->shared, v->address_space_id, result->iova, + size); if (unlikely(r < 0)) { error_report("Unable to unmap SVQ vring: %s (%d)", g_strerror(-r), -r); return; @@ -1121,7 +1121,7 @@ static bool vhost_vdpa_svq_map_ring(struct vhost_vdpa *v, DMAMap *needle, return false; } - r = vhost_vdpa_dma_map(v, v->address_space_id, needle->iova, + r = vhost_vdpa_dma_map(v->shared, v->address_space_id, needle->iova, needle->size + 1, (void *)(uintptr_t)needle->translated_addr, needle->perm == IOMMU_RO); diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 10cf0027de..3726ee5d67 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -471,7 +471,8 @@ static void vhost_vdpa_cvq_unmap_buf(struct vhost_vdpa *v, void *addr) return; } - r = vhost_vdpa_dma_unmap(v, v->address_space_id, map->iova, map->size + 1); + r = vhost_vdpa_dma_unmap(v->shared, v->address_space_id, map->iova, + map->size + 1); if (unlikely(r != 0)) { error_report("Device cannot unmap: %s(%d)", g_strerror(r), r); } @@ -495,7 +496,7 @@ static int vhost_vdpa_cvq_map_buf(struct vhost_vdpa *v, void *buf, size_t size, return r; } - r = vhost_vdpa_dma_map(v, v->address_space_id, map.iova, + r = vhost_vdpa_dma_map(v->shared, v->address_space_id, map.iova, vhost_vdpa_net_cvq_cmd_page_len(), buf, !write); if (unlikely(r < 0)) { goto dma_map_err; From patchwork Tue Dec 26 09:25:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880236 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=iB5B6yVM; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SzqDw2yNBz23dW for ; Tue, 26 Dec 2023 20:27:44 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3gp-00051B-N5; Tue, 26 Dec 2023 04:25:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gn-0004sO-MS for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:29 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gl-0000HP-Gw for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582726; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o0S1EGWalT9mc5fSDPitMMFcpDcP6GvPJwnq9hG5/c8=; b=iB5B6yVMC908Hx7/toOvo22cShvU4F7U41HkQvvK1Ls+xMvIepUvspzM20TECxgPozx6kx LJXOL+byy7KKXaZRfXyAadEXIrWv7HMXUl/F9VeOu0nblDpdSNmJBR3/LrCn+4AZP9G5E+ SdtXTg6NSY/AoS13cp30AyR24wl7j98= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-187-UTP067Z0OkCrCqtG7tF1pA-1; Tue, 26 Dec 2023 04:25:24 -0500 X-MC-Unique: UTP067Z0OkCrCqtG7tF1pA-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-40d54db2ab5so20914875e9.2 for ; Tue, 26 Dec 2023 01:25:24 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582723; x=1704187523; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=o0S1EGWalT9mc5fSDPitMMFcpDcP6GvPJwnq9hG5/c8=; b=gAiU4o3iQuazncZaFcnIifWUot9qmREwaHYJeeAyJEBcVXwy4DZRd0P+Q95RTBDCQt Dv8V9Cb/w7bOEQmQOQx2Aq2d6FyOQVimmqqSxWsG2lYZYqd5WJHDbCwZxDT7yrUW2keJ mmlOC1Pg5JtMMJhxxrs9PmJjpKHfjcPbU7fqyvgzTNNIym6pILmtI6HN/k9dQZO/WJxu i44i8EeZBva2VTvZsbji8gZCIzkm8x+SFsrwRb4r2dzWsQtq6cREhMI3xTSt2sjn3Klz oJnghEOPrf6C9w+SM2h/gBdMRf59qd8KHmcj6Vola7IR5/MMYTf/oQjcMZ1waJpqGKDt 0flw== X-Gm-Message-State: AOJu0Yxr+1AuGrAcb06Q+xoW92LUYbEIyLLkRaKaJJ+qXEdHrJpTYh4U kxDrGDOKU43OHSBd0KcKXoen6uiMU+dzQapDMHvLuogqiFJ9K3V+WsxxWnMXHyunojSNP929efQ s9+svn44sKYK/VV5Ux7vyezcmKGlfAnXeSLBEMe+zdXhmLLSLGQVPRmfcNirypu+WbwwDVH//29 Ln X-Received: by 2002:a05:600c:22ca:b0:40d:4a5e:e4f2 with SMTP id 10-20020a05600c22ca00b0040d4a5ee4f2mr2898476wmg.125.1703582722893; Tue, 26 Dec 2023 01:25:22 -0800 (PST) X-Google-Smtp-Source: AGHT+IElSf2jtupM7KpRYjpndKIPxpuwRtqQJBuXFGVq4b9PosqqLTpUYXIFI1fYwmVQixouZRhJHQ== X-Received: by 2002:a05:600c:22ca:b0:40d:4a5e:e4f2 with SMTP id 10-20020a05600c22ca00b0040d4a5ee4f2mr2898466wmg.125.1703582722562; Tue, 26 Dec 2023 01:25:22 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id j16-20020a05600c191000b0040c11fbe581sm20159153wmq.27.2023.12.26.01.25.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:25:21 -0800 (PST) Date: Tue, 26 Dec 2023 04:25:19 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Eugenio =?utf-8?b?UMOpcmV6?= , Jason Wang , Lei Yang Subject: [PULL 20/21] vdpa: use dev_shared in vdpa_iommu Message-ID: <878f1e7dd9acc736e7a0f5a205c278c8ae769d39.1703582625.git.mst@redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.129.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Eugenio Pérez The memory listener functions can call these too. Make vdpa_iommu work with VhostVDPAShared. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang Message-Id: <20231221174322.3130442-13-eperezma@redhat.com> Tested-by: Lei Yang Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- include/hw/virtio/vhost-vdpa.h | 2 +- hw/virtio/vhost-vdpa.c | 16 ++++++++-------- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 705c754776..2abee2164a 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -75,7 +75,7 @@ int vhost_vdpa_dma_unmap(VhostVDPAShared *s, uint32_t asid, hwaddr iova, hwaddr size); typedef struct vdpa_iommu { - struct vhost_vdpa *dev; + VhostVDPAShared *dev_shared; IOMMUMemoryRegion *iommu_mr; hwaddr iommu_offset; IOMMUNotifier n; diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index ada3cc5d7c..e6276fb1e4 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -199,7 +199,7 @@ static void vhost_vdpa_iommu_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) struct vdpa_iommu *iommu = container_of(n, struct vdpa_iommu, n); hwaddr iova = iotlb->iova + iommu->iommu_offset; - struct vhost_vdpa *v = iommu->dev; + VhostVDPAShared *s = iommu->dev_shared; void *vaddr; int ret; Int128 llend; @@ -212,10 +212,10 @@ static void vhost_vdpa_iommu_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) RCU_READ_LOCK_GUARD(); /* check if RAM section out of device range */ llend = int128_add(int128_makes64(iotlb->addr_mask), int128_makes64(iova)); - if (int128_gt(llend, int128_make64(v->shared->iova_range.last))) { + if (int128_gt(llend, int128_make64(s->iova_range.last))) { error_report("RAM section out of device range (max=0x%" PRIx64 ", end addr=0x%" PRIx64 ")", - v->shared->iova_range.last, int128_get64(llend)); + s->iova_range.last, int128_get64(llend)); return; } @@ -225,20 +225,20 @@ static void vhost_vdpa_iommu_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) if (!memory_get_xlat_addr(iotlb, &vaddr, NULL, &read_only, NULL)) { return; } - ret = vhost_vdpa_dma_map(v->shared, VHOST_VDPA_GUEST_PA_ASID, iova, + ret = vhost_vdpa_dma_map(s, VHOST_VDPA_GUEST_PA_ASID, iova, iotlb->addr_mask + 1, vaddr, read_only); if (ret) { error_report("vhost_vdpa_dma_map(%p, 0x%" HWADDR_PRIx ", " "0x%" HWADDR_PRIx ", %p) = %d (%m)", - v, iova, iotlb->addr_mask + 1, vaddr, ret); + s, iova, iotlb->addr_mask + 1, vaddr, ret); } } else { - ret = vhost_vdpa_dma_unmap(v->shared, VHOST_VDPA_GUEST_PA_ASID, iova, + ret = vhost_vdpa_dma_unmap(s, VHOST_VDPA_GUEST_PA_ASID, iova, iotlb->addr_mask + 1); if (ret) { error_report("vhost_vdpa_dma_unmap(%p, 0x%" HWADDR_PRIx ", " "0x%" HWADDR_PRIx ") = %d (%m)", - v, iova, iotlb->addr_mask + 1, ret); + s, iova, iotlb->addr_mask + 1, ret); } } } @@ -270,7 +270,7 @@ static void vhost_vdpa_iommu_region_add(MemoryListener *listener, iommu_idx); iommu->iommu_offset = section->offset_within_address_space - section->offset_within_region; - iommu->dev = v; + iommu->dev_shared = v->shared; ret = memory_region_register_iommu_notifier(section->mr, &iommu->n, NULL); if (ret) { From patchwork Tue Dec 26 09:25:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 1880235 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=QEkCwJzE; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SzqF063gQz1ydb for ; Tue, 26 Dec 2023 20:27:48 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3gs-0005Hy-M9; Tue, 26 Dec 2023 04:25:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gq-00058h-Qy for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:32 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3go-0000ID-GA for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582729; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iZD0hIOPiOh7X8EWN/o0sE45YFCMEYAc6xIQsbTilnU=; b=QEkCwJzEgaEbiV7MbxRPXEljoboTMul1HfmVmS/J537tFjKMVsRQNDZHHI4scsjF/ggHO9 rE5PEYSlYjGTVMiR6Nafx7tibzqfkf/75iGRV0FRPKVGMo67B+Op3Cf+ePgFBaLudEGlns tQVbzoeS12enR0oEi2xwFIo67TfV/SU= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-282-hadQ1UxbMoWRVRpGyaWHjw-1; Tue, 26 Dec 2023 04:25:28 -0500 X-MC-Unique: hadQ1UxbMoWRVRpGyaWHjw-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-40d51c3997fso17602715e9.0 for ; Tue, 26 Dec 2023 01:25:28 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582726; x=1704187526; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=iZD0hIOPiOh7X8EWN/o0sE45YFCMEYAc6xIQsbTilnU=; b=eSE/eUvDcCBL5Znu70luaFpthUYQyvehNN5WhvFNzz9VRolDQY20ZySDlB2Wp8ghSV 9wgn8ODVdtDWHXLj6kYPuwoYGum7i4S1n2lF4ukxn0SwLU2C2h/biX/7XaxOGAUJKQYJ LLOcS6GRwoQ+cAanoqobeBhzsW4g5wEmRSRhnk38GXCX2DGExsVhMz5bxiQ1JY6IXysS hJBvg6+LmrOU7SmB584UFanxwA8aPCWsvLxsV07YtNZ3cO26zL6BvUKYKXHk/KC0C/gS 5cJhAM7HlcCynqUcTLbijoGfGuAm/Qh5bRnvNhhfzUa0Kpxo99EQNuO5S6p+7qoEsXQu dwFA== X-Gm-Message-State: AOJu0YxT23JuDGDe1vDTsq07m7OGqE3FVlXlzRc44pLdv/vhQyiHV/I8 iE9Bp2Cuao2bZj/9U/GixqyFs9elWIuc7lYp5DjP4taMKTrf3IgiJNzf/vBwV1uE/cI48Ssks0Z 3FO9lUXkGkMxlt6VJuVKfrcjUfJlTCVVfn4IcjBGETahe3n+wKChpg3tsIZtLQDBiCS8W6OUd5N sd X-Received: by 2002:a05:600c:154c:b0:40c:3f2b:bbbb with SMTP id f12-20020a05600c154c00b0040c3f2bbbbbmr5071211wmg.75.1703582726057; Tue, 26 Dec 2023 01:25:26 -0800 (PST) X-Google-Smtp-Source: AGHT+IFUdZDqhfNDBel115mfZWm10Gb3hyveCwW7HiCvNAlaQ/i5inkbSlAo+lIPaU3LW0Y0w5Wz4g== X-Received: by 2002:a05:600c:154c:b0:40c:3f2b:bbbb with SMTP id f12-20020a05600c154c00b0040c3f2bbbbbmr5071199wmg.75.1703582725694; Tue, 26 Dec 2023 01:25:25 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id l4-20020a05600c1d0400b0040d3276ba19sm20022118wms.25.2023.12.26.01.25.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:25:25 -0800 (PST) Date: Tue, 26 Dec 2023 04:25:23 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Eugenio =?utf-8?b?UMOpcmV6?= , Jason Wang , Lei Yang Subject: [PULL 21/21] vdpa: move memory listener to vhost_vdpa_shared Message-ID: <7b67b2f0f4f7c5ec888a331af599d9daff735d60.1703582625.git.mst@redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.133.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Eugenio Pérez Next patches will register the vhost_vdpa memory listener while the VM is migrating at the destination, so we can map the memory to the device before stopping the VM at the source. The main goal is to reduce the downtime. However, the destination QEMU is unaware of which vhost_vdpa device will register its memory_listener. If the source guest has CVQ enabled, it will be the CVQ device. Otherwise, it will be the first one. Move the memory listener to a common place rather than always in the first / last vhost_vdpa. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang Message-Id: <20231221174322.3130442-14-eperezma@redhat.com> Tested-by: Lei Yang Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- include/hw/virtio/vhost-vdpa.h | 2 +- hw/virtio/vhost-vdpa.c | 84 ++++++++++++++++------------------ 2 files changed, 40 insertions(+), 46 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 2abee2164a..8f54e5edd4 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -33,6 +33,7 @@ typedef struct VhostVDPAHostNotifier { /* Info shared by all vhost_vdpa device models */ typedef struct vhost_vdpa_shared { int device_fd; + MemoryListener listener; struct vhost_vdpa_iova_range iova_range; QLIST_HEAD(, vdpa_iommu) iommu_list; @@ -51,7 +52,6 @@ typedef struct vhost_vdpa_shared { typedef struct vhost_vdpa { int index; uint32_t address_space_id; - MemoryListener listener; uint64_t acked_features; bool shadow_vqs_enabled; /* Device suspended successfully */ diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index e6276fb1e4..ddae494ca8 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -170,28 +170,28 @@ static void vhost_vdpa_iotlb_batch_begin_once(VhostVDPAShared *s) static void vhost_vdpa_listener_commit(MemoryListener *listener) { - struct vhost_vdpa *v = container_of(listener, struct vhost_vdpa, listener); + VhostVDPAShared *s = container_of(listener, VhostVDPAShared, listener); struct vhost_msg_v2 msg = {}; - int fd = v->shared->device_fd; + int fd = s->device_fd; - if (!(v->shared->backend_cap & (0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH))) { + if (!(s->backend_cap & (0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH))) { return; } - if (!v->shared->iotlb_batch_begin_sent) { + if (!s->iotlb_batch_begin_sent) { return; } msg.type = VHOST_IOTLB_MSG_V2; msg.iotlb.type = VHOST_IOTLB_BATCH_END; - trace_vhost_vdpa_listener_commit(v->shared, fd, msg.type, msg.iotlb.type); + trace_vhost_vdpa_listener_commit(s, fd, msg.type, msg.iotlb.type); if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) { error_report("failed to write, fd=%d, errno=%d (%s)", fd, errno, strerror(errno)); } - v->shared->iotlb_batch_begin_sent = false; + s->iotlb_batch_begin_sent = false; } static void vhost_vdpa_iommu_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) @@ -246,7 +246,7 @@ static void vhost_vdpa_iommu_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) static void vhost_vdpa_iommu_region_add(MemoryListener *listener, MemoryRegionSection *section) { - struct vhost_vdpa *v = container_of(listener, struct vhost_vdpa, listener); + VhostVDPAShared *s = container_of(listener, VhostVDPAShared, listener); struct vdpa_iommu *iommu; Int128 end; @@ -270,7 +270,7 @@ static void vhost_vdpa_iommu_region_add(MemoryListener *listener, iommu_idx); iommu->iommu_offset = section->offset_within_address_space - section->offset_within_region; - iommu->dev_shared = v->shared; + iommu->dev_shared = s; ret = memory_region_register_iommu_notifier(section->mr, &iommu->n, NULL); if (ret) { @@ -278,7 +278,7 @@ static void vhost_vdpa_iommu_region_add(MemoryListener *listener, return; } - QLIST_INSERT_HEAD(&v->shared->iommu_list, iommu, iommu_next); + QLIST_INSERT_HEAD(&s->iommu_list, iommu, iommu_next); memory_region_iommu_replay(iommu->iommu_mr, &iommu->n); return; @@ -287,11 +287,11 @@ static void vhost_vdpa_iommu_region_add(MemoryListener *listener, static void vhost_vdpa_iommu_region_del(MemoryListener *listener, MemoryRegionSection *section) { - struct vhost_vdpa *v = container_of(listener, struct vhost_vdpa, listener); + VhostVDPAShared *s = container_of(listener, VhostVDPAShared, listener); struct vdpa_iommu *iommu; - QLIST_FOREACH(iommu, &v->shared->iommu_list, iommu_next) + QLIST_FOREACH(iommu, &s->iommu_list, iommu_next) { if (MEMORY_REGION(iommu->iommu_mr) == section->mr && iommu->n.start == section->offset_within_region) { @@ -307,7 +307,7 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, MemoryRegionSection *section) { DMAMap mem_region = {}; - struct vhost_vdpa *v = container_of(listener, struct vhost_vdpa, listener); + VhostVDPAShared *s = container_of(listener, VhostVDPAShared, listener); hwaddr iova; Int128 llend, llsize; void *vaddr; @@ -315,10 +315,8 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, int page_size = qemu_target_page_size(); int page_mask = -page_size; - if (vhost_vdpa_listener_skipped_section(section, - v->shared->iova_range.first, - v->shared->iova_range.last, - page_mask)) { + if (vhost_vdpa_listener_skipped_section(section, s->iova_range.first, + s->iova_range.last, page_mask)) { return; } if (memory_region_is_iommu(section->mr)) { @@ -328,8 +326,7 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, if (unlikely((section->offset_within_address_space & ~page_mask) != (section->offset_within_region & ~page_mask))) { - trace_vhost_vdpa_listener_region_add_unaligned(v->shared, - section->mr->name, + trace_vhost_vdpa_listener_region_add_unaligned(s, section->mr->name, section->offset_within_address_space & ~page_mask, section->offset_within_region & ~page_mask); return; @@ -349,18 +346,18 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, section->offset_within_region + (iova - section->offset_within_address_space); - trace_vhost_vdpa_listener_region_add(v->shared, iova, int128_get64(llend), + trace_vhost_vdpa_listener_region_add(s, iova, int128_get64(llend), vaddr, section->readonly); llsize = int128_sub(llend, int128_make64(iova)); - if (v->shared->shadow_data) { + if (s->shadow_data) { int r; mem_region.translated_addr = (hwaddr)(uintptr_t)vaddr, mem_region.size = int128_get64(llsize) - 1, mem_region.perm = IOMMU_ACCESS_FLAG(true, section->readonly), - r = vhost_iova_tree_map_alloc(v->shared->iova_tree, &mem_region); + r = vhost_iova_tree_map_alloc(s->iova_tree, &mem_region); if (unlikely(r != IOVA_OK)) { error_report("Can't allocate a mapping (%d)", r); goto fail; @@ -369,8 +366,8 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, iova = mem_region.iova; } - vhost_vdpa_iotlb_batch_begin_once(v->shared); - ret = vhost_vdpa_dma_map(v->shared, VHOST_VDPA_GUEST_PA_ASID, iova, + vhost_vdpa_iotlb_batch_begin_once(s); + ret = vhost_vdpa_dma_map(s, VHOST_VDPA_GUEST_PA_ASID, iova, int128_get64(llsize), vaddr, section->readonly); if (ret) { error_report("vhost vdpa map fail!"); @@ -380,8 +377,8 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, return; fail_map: - if (v->shared->shadow_data) { - vhost_iova_tree_remove(v->shared->iova_tree, mem_region); + if (s->shadow_data) { + vhost_iova_tree_remove(s->iova_tree, mem_region); } fail: @@ -398,17 +395,15 @@ fail: static void vhost_vdpa_listener_region_del(MemoryListener *listener, MemoryRegionSection *section) { - struct vhost_vdpa *v = container_of(listener, struct vhost_vdpa, listener); + VhostVDPAShared *s = container_of(listener, VhostVDPAShared, listener); hwaddr iova; Int128 llend, llsize; int ret; int page_size = qemu_target_page_size(); int page_mask = -page_size; - if (vhost_vdpa_listener_skipped_section(section, - v->shared->iova_range.first, - v->shared->iova_range.last, - page_mask)) { + if (vhost_vdpa_listener_skipped_section(section, s->iova_range.first, + s->iova_range.last, page_mask)) { return; } if (memory_region_is_iommu(section->mr)) { @@ -417,8 +412,7 @@ static void vhost_vdpa_listener_region_del(MemoryListener *listener, if (unlikely((section->offset_within_address_space & ~page_mask) != (section->offset_within_region & ~page_mask))) { - trace_vhost_vdpa_listener_region_del_unaligned(v->shared, - section->mr->name, + trace_vhost_vdpa_listener_region_del_unaligned(s, section->mr->name, section->offset_within_address_space & ~page_mask, section->offset_within_region & ~page_mask); return; @@ -427,7 +421,7 @@ static void vhost_vdpa_listener_region_del(MemoryListener *listener, iova = ROUND_UP(section->offset_within_address_space, page_size); llend = vhost_vdpa_section_end(section, page_mask); - trace_vhost_vdpa_listener_region_del(v->shared, iova, + trace_vhost_vdpa_listener_region_del(s, iova, int128_get64(int128_sub(llend, int128_one()))); if (int128_ge(int128_make64(iova), llend)) { @@ -436,7 +430,7 @@ static void vhost_vdpa_listener_region_del(MemoryListener *listener, llsize = int128_sub(llend, int128_make64(iova)); - if (v->shared->shadow_data) { + if (s->shadow_data) { const DMAMap *result; const void *vaddr = memory_region_get_ram_ptr(section->mr) + section->offset_within_region + @@ -446,37 +440,37 @@ static void vhost_vdpa_listener_region_del(MemoryListener *listener, .size = int128_get64(llsize) - 1, }; - result = vhost_iova_tree_find_iova(v->shared->iova_tree, &mem_region); + result = vhost_iova_tree_find_iova(s->iova_tree, &mem_region); if (!result) { /* The memory listener map wasn't mapped */ return; } iova = result->iova; - vhost_iova_tree_remove(v->shared->iova_tree, *result); + vhost_iova_tree_remove(s->iova_tree, *result); } - vhost_vdpa_iotlb_batch_begin_once(v->shared); + vhost_vdpa_iotlb_batch_begin_once(s); /* * The unmap ioctl doesn't accept a full 64-bit. need to check it */ if (int128_eq(llsize, int128_2_64())) { llsize = int128_rshift(llsize, 1); - ret = vhost_vdpa_dma_unmap(v->shared, VHOST_VDPA_GUEST_PA_ASID, iova, + ret = vhost_vdpa_dma_unmap(s, VHOST_VDPA_GUEST_PA_ASID, iova, int128_get64(llsize)); if (ret) { error_report("vhost_vdpa_dma_unmap(%p, 0x%" HWADDR_PRIx ", " "0x%" HWADDR_PRIx ") = %d (%m)", - v, iova, int128_get64(llsize), ret); + s, iova, int128_get64(llsize), ret); } iova += int128_get64(llsize); } - ret = vhost_vdpa_dma_unmap(v->shared, VHOST_VDPA_GUEST_PA_ASID, iova, + ret = vhost_vdpa_dma_unmap(s, VHOST_VDPA_GUEST_PA_ASID, iova, int128_get64(llsize)); if (ret) { error_report("vhost_vdpa_dma_unmap(%p, 0x%" HWADDR_PRIx ", " "0x%" HWADDR_PRIx ") = %d (%m)", - v, iova, int128_get64(llsize), ret); + s, iova, int128_get64(llsize), ret); } memory_region_unref(section->mr); @@ -595,7 +589,7 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void *opaque, Error **errp) v->dev = dev; dev->opaque = opaque ; - v->listener = vhost_vdpa_memory_listener; + v->shared->listener = vhost_vdpa_memory_listener; vhost_vdpa_init_svq(dev, v); error_propagate(&dev->migration_blocker, v->migration_blocker); @@ -755,10 +749,10 @@ static int vhost_vdpa_cleanup(struct vhost_dev *dev) trace_vhost_vdpa_cleanup(dev, v); if (vhost_vdpa_first_dev(dev)) { ram_block_discard_disable(false); + memory_listener_unregister(&v->shared->listener); } vhost_vdpa_host_notifiers_uninit(dev, dev->nvqs); - memory_listener_unregister(&v->listener); vhost_vdpa_svq_cleanup(dev); dev->opaque = NULL; @@ -1331,7 +1325,7 @@ static int vhost_vdpa_dev_start(struct vhost_dev *dev, bool started) "IOMMU and try again"); return -1; } - memory_listener_register(&v->listener, dev->vdev->dma_as); + memory_listener_register(&v->shared->listener, dev->vdev->dma_as); return vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_DRIVER_OK); } @@ -1350,7 +1344,7 @@ static void vhost_vdpa_reset_status(struct vhost_dev *dev) vhost_vdpa_reset_device(dev); vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | VIRTIO_CONFIG_S_DRIVER); - memory_listener_unregister(&v->listener); + memory_listener_unregister(&v->shared->listener); } static int vhost_vdpa_set_log_base(struct vhost_dev *dev, uint64_t base,