From patchwork Fri Jan 19 18:13:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1888631 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=CV/2AWqZ; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TGnrY2c5yz1yPV for ; Sat, 20 Jan 2024 05:17:01 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rQtNK-0001gk-4j; Fri, 19 Jan 2024 13:13:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtN8-0001cv-KR for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:43 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtN5-0007ZP-V3 for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705688017; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7KXVoT9IraWqlZ0Y8JMlj17j6pFajLnqEWukWX7hPLs=; b=CV/2AWqZZf12VtMojhRxDgEIokTNwDBlxo9oLLM//os5McsJPwH7A976ZYlhPGIEbBondS rpDcgkKdxpKfyG78g9Mp4N6FCb1Wd6ucNWh81XqBVw+NtQFY/tIhZmhMlYpClUawAl3pYh 7atDQckhAM4QXN93xW2efJovn/oEHgw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-207-ipxZrrffOJqw720Kkp8Eqw-1; Fri, 19 Jan 2024 13:13:35 -0500 X-MC-Unique: ipxZrrffOJqw720Kkp8Eqw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BCE9F185A786; Fri, 19 Jan 2024 18:13:33 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.86]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2E5CA1B96; Fri, 19 Jan 2024 18:13:33 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 01/14] block/blklogwrites: Fix a bug when logging "write zeroes" operations. Date: Fri, 19 Jan 2024 19:13:14 +0100 Message-ID: <20240119181327.236745-2-kwolf@redhat.com> In-Reply-To: <20240119181327.236745-1-kwolf@redhat.com> References: <20240119181327.236745-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.519, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Ari Sundholm There is a bug in the blklogwrites driver pertaining to logging "write zeroes" operations, causing log corruption. This can be easily observed by setting detect-zeroes to something other than "off" for the driver. The issue is caused by a concurrency bug pertaining to the fact that "write zeroes" operations have to be logged in two parts: first the log entry metadata, then the zeroed-out region. While the log entry metadata is being written by bdrv_co_pwritev(), another operation may begin in the meanwhile and modify the state of the blklogwrites driver. This is as intended by the coroutine-driven I/O model in QEMU, of course. Unfortunately, this specific scenario is mishandled. A short example: 1. Initially, in the current operation (#1), the current log sector number in the driver state is only incremented by the number of sectors taken by the log entry metadata, after which the log entry metadata is written. The current operation yields. 2. Another operation (#2) may start while the log entry metadata is being written. It uses the current log position as the start offset for its log entry. This is in the sector right after the operation #1 log entry metadata, which is bad! 3. After bdrv_co_pwritev() returns (#1), the current log sector number is reread from the driver state in order to find out the start offset for bdrv_co_pwrite_zeroes(). This is an obvious blunder, as the offset will be the sector right after the (misplaced) operation #2 log entry, which means that the zeroed-out region begins at the wrong offset. 4. As a result of the above, the log is corrupt. Fix this by only reading the driver metadata once, computing the offsets and sizes in one go (including the optional zeroed-out region) and setting the log sector number to the appropriate value for the next operation in line. Signed-off-by: Ari Sundholm Cc: qemu-stable@nongnu.org Message-ID: <20240109184646.1128475-1-megari@gmx.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- block/blklogwrites.c | 35 ++++++++++++++++++++++++++--------- 1 file changed, 26 insertions(+), 9 deletions(-) diff --git a/block/blklogwrites.c b/block/blklogwrites.c index 7207b2e757..ba717dab4d 100644 --- a/block/blklogwrites.c +++ b/block/blklogwrites.c @@ -328,22 +328,39 @@ static void coroutine_fn GRAPH_RDLOCK blk_log_writes_co_do_log(BlkLogWritesLogReq *lr) { BDRVBlkLogWritesState *s = lr->bs->opaque; - uint64_t cur_log_offset = s->cur_log_sector << s->sectorbits; - s->nr_entries++; - s->cur_log_sector += - ROUND_UP(lr->qiov->size, s->sectorsize) >> s->sectorbits; + /* + * Determine the offsets and sizes of different parts of the entry, and + * update the state of the driver. + * + * This needs to be done in one go, before any actual I/O is done, as the + * log entry may have to be written in two parts, and the state of the + * driver may be modified by other driver operations while waiting for the + * I/O to complete. + */ + const uint64_t entry_start_sector = s->cur_log_sector; + const uint64_t entry_offset = entry_start_sector << s->sectorbits; + const uint64_t qiov_aligned_size = ROUND_UP(lr->qiov->size, s->sectorsize); + const uint64_t entry_aligned_size = qiov_aligned_size + + ROUND_UP(lr->zero_size, s->sectorsize); + const uint64_t entry_nr_sectors = entry_aligned_size >> s->sectorbits; - lr->log_ret = bdrv_co_pwritev(s->log_file, cur_log_offset, lr->qiov->size, + s->nr_entries++; + s->cur_log_sector += entry_nr_sectors; + + /* + * Write the log entry. Note that if this is a "write zeroes" operation, + * only the entry header is written here, with the zeroing being done + * separately below. + */ + lr->log_ret = bdrv_co_pwritev(s->log_file, entry_offset, lr->qiov->size, lr->qiov, 0); /* Logging for the "write zeroes" operation */ if (lr->log_ret == 0 && lr->zero_size) { - cur_log_offset = s->cur_log_sector << s->sectorbits; - s->cur_log_sector += - ROUND_UP(lr->zero_size, s->sectorsize) >> s->sectorbits; + const uint64_t zeroes_offset = entry_offset + qiov_aligned_size; - lr->log_ret = bdrv_co_pwrite_zeroes(s->log_file, cur_log_offset, + lr->log_ret = bdrv_co_pwrite_zeroes(s->log_file, zeroes_offset, lr->zero_size, 0); } From patchwork Fri Jan 19 18:13:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1888630 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=b9Khme8v; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TGnrB2QKcz1yPV for ; Sat, 20 Jan 2024 05:16:42 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rQtNK-0001gl-66; Fri, 19 Jan 2024 13:13:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtN8-0001cw-KY for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:43 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtN5-0007aG-V7 for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705688018; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5sC1mg7x2fL1b69hJeYewTvKHdoYdK1EgellCnZKA5s=; b=b9Khme8vVosBQgx2NEdlqAnml7nu47Knu1aorgxjDZCagKKrWgZwq1V3JCS1hJGoeRRhvK M0iOiRu3iPtdCU0JbI3LCj/08muXLtCZ68tiSBDejLU8Aubr6hfdv/jTaktMz3aJwVHDsc gcLVYt6oKwMq5cF5l0czAVZ21cq7hDw= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-441-DgXMf4ZdM8u35VRvKTx3Dw-1; Fri, 19 Jan 2024 13:13:35 -0500 X-MC-Unique: DgXMf4ZdM8u35VRvKTx3Dw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8DA003C11CDF; Fri, 19 Jan 2024 18:13:34 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.86]) by smtp.corp.redhat.com (Postfix) with ESMTP id F2727494; Fri, 19 Jan 2024 18:13:33 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 02/14] string-output-visitor: Fix (pseudo) struct handling Date: Fri, 19 Jan 2024 19:13:15 +0100 Message-ID: <20240119181327.236745-3-kwolf@redhat.com> In-Reply-To: <20240119181327.236745-1-kwolf@redhat.com> References: <20240119181327.236745-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.519, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Commit ff32bb53 tried to get minimal struct support into the string output visitor by just making it return "". Unfortunately, it forgot that the caller will still make more visitor calls for the content of the struct. If the struct is contained in a list, such as IOThreadVirtQueueMapping, in the better case its fields show up as separate list entries. In the worse case, it contains another list, and the string output visitor doesn't support nested lists and asserts that this doesn't happen. So as soon as the optional "vqs" field in IOThreadVirtQueueMapping is specified, we get a crash. This can be reproduced with the following command line: echo "info qtree" | ./qemu-system-x86_64 \ -object iothread,id=t0 \ -blockdev null-co,node-name=disk \ -device '{"driver": "virtio-blk-pci", "drive": "disk", "iothread-vq-mapping": [{"iothread": "t0", "vqs": [0]}]}' \ -monitor stdio Fix the problem by counting the nesting level of structs and ignoring any visitor calls for values (apart from start/end_struct) while we're not on the top level. Lists nested directly within lists remain unimplemented, as we don't currently have a use case for them. Fixes: ff32bb53476539d352653f4ed56372dced73a388 Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2069 Reported-by: Aihua Liang Signed-off-by: Kevin Wolf Message-ID: <20240109181717.42493-1-kwolf@redhat.com> Reviewed-by: Stefan Hajnoczi Signed-off-by: Kevin Wolf --- qapi/string-output-visitor.c | 46 ++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/qapi/string-output-visitor.c b/qapi/string-output-visitor.c index f0c1dea89e..5115536b15 100644 --- a/qapi/string-output-visitor.c +++ b/qapi/string-output-visitor.c @@ -65,6 +65,7 @@ struct StringOutputVisitor } range_start, range_end; GList *ranges; void *list; /* Only needed for sanity checking the caller */ + unsigned int struct_nesting; }; static StringOutputVisitor *to_sov(Visitor *v) @@ -144,6 +145,10 @@ static bool print_type_int64(Visitor *v, const char *name, int64_t *obj, StringOutputVisitor *sov = to_sov(v); GList *l; + if (sov->struct_nesting) { + return true; + } + switch (sov->list_mode) { case LM_NONE: string_output_append(sov, *obj); @@ -231,6 +236,10 @@ static bool print_type_size(Visitor *v, const char *name, uint64_t *obj, uint64_t val; char *out, *psize; + if (sov->struct_nesting) { + return true; + } + if (!sov->human) { out = g_strdup_printf("%"PRIu64, *obj); string_output_set(sov, out); @@ -250,6 +259,11 @@ static bool print_type_bool(Visitor *v, const char *name, bool *obj, Error **errp) { StringOutputVisitor *sov = to_sov(v); + + if (sov->struct_nesting) { + return true; + } + string_output_set(sov, g_strdup(*obj ? "true" : "false")); return true; } @@ -260,6 +274,10 @@ static bool print_type_str(Visitor *v, const char *name, char **obj, StringOutputVisitor *sov = to_sov(v); char *out; + if (sov->struct_nesting) { + return true; + } + if (sov->human) { out = *obj ? g_strdup_printf("\"%s\"", *obj) : g_strdup(""); } else { @@ -273,6 +291,11 @@ static bool print_type_number(Visitor *v, const char *name, double *obj, Error **errp) { StringOutputVisitor *sov = to_sov(v); + + if (sov->struct_nesting) { + return true; + } + string_output_set(sov, g_strdup_printf("%.17g", *obj)); return true; } @@ -283,6 +306,10 @@ static bool print_type_null(Visitor *v, const char *name, QNull **obj, StringOutputVisitor *sov = to_sov(v); char *out; + if (sov->struct_nesting) { + return true; + } + if (sov->human) { out = g_strdup(""); } else { @@ -295,6 +322,9 @@ static bool print_type_null(Visitor *v, const char *name, QNull **obj, static bool start_struct(Visitor *v, const char *name, void **obj, size_t size, Error **errp) { + StringOutputVisitor *sov = to_sov(v); + + sov->struct_nesting++; return true; } @@ -302,6 +332,10 @@ static void end_struct(Visitor *v, void **obj) { StringOutputVisitor *sov = to_sov(v); + if (--sov->struct_nesting) { + return; + } + /* TODO actually print struct fields */ string_output_set(sov, g_strdup("")); } @@ -312,6 +346,10 @@ start_list(Visitor *v, const char *name, GenericList **list, size_t size, { StringOutputVisitor *sov = to_sov(v); + if (sov->struct_nesting) { + return true; + } + /* we can't traverse a list in a list */ assert(sov->list_mode == LM_NONE); /* We don't support visits without a list */ @@ -329,6 +367,10 @@ static GenericList *next_list(Visitor *v, GenericList *tail, size_t size) StringOutputVisitor *sov = to_sov(v); GenericList *ret = tail->next; + if (sov->struct_nesting) { + return ret; + } + if (ret && !ret->next) { sov->list_mode = LM_END; } @@ -339,6 +381,10 @@ static void end_list(Visitor *v, void **obj) { StringOutputVisitor *sov = to_sov(v); + if (sov->struct_nesting) { + return; + } + assert(sov->list == obj); assert(sov->list_mode == LM_STARTED || sov->list_mode == LM_END || From patchwork Fri Jan 19 18:13:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1888618 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=eNKsLiHt; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TGnpZ2Ws0z1yPV for ; Sat, 20 Jan 2024 05:15:18 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rQtNJ-0001fY-3J; Fri, 19 Jan 2024 13:13:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtN9-0001cy-OE for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:44 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtN5-0007bK-Vk for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:43 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705688019; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YirToRa5ScB2Piwjcv0ToWz+Q4DVNA0SLcAoOpyrkZc=; b=eNKsLiHtorjY+z+loQ6hnZk8OVYPPURZC6Ht1/e75owzL7N/UDHknIe1O3EdnyE5/O7Z+m JzLNLa2K1ZvSf1R1t3rPxMJbU85E6wUH8DL4kHqpo5HGQwwHnFR+LGK5JpcCjJvh1z+yv8 p+89XeJ3wYdN9r48E+Zhl+1Y20+0qnA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-549-DCFs0OuLOViN9ztCO4iQWA-1; Fri, 19 Jan 2024 13:13:35 -0500 X-MC-Unique: DCFs0OuLOViN9ztCO4iQWA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5D2C886794C; Fri, 19 Jan 2024 18:13:35 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.86]) by smtp.corp.redhat.com (Postfix) with ESMTP id C3B56494; Fri, 19 Jan 2024 18:13:34 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 03/14] commit: Allow users to request only format driver names in backing file format Date: Fri, 19 Jan 2024 19:13:16 +0100 Message-ID: <20240119181327.236745-4-kwolf@redhat.com> In-Reply-To: <20240119181327.236745-1-kwolf@redhat.com> References: <20240119181327.236745-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.519, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Peter Krempa Introduce a new flag 'backing-mask-protocol' for the block-commit QMP command which instructs the internals to use 'raw' instead of the protocol driver in case when a image is used without a dummy 'raw' wrapper. The flag is designed such that it can be always asserted by management tools even when there isn't any update to backing files. The flag will be used by libvirt so that the backing images still reference the proper format even when libvirt will stop using the dummy raw driver (raw driver with no other config). Libvirt needs this so that the images stay compatible with older libvirt versions which didn't expect that a protocol driver name can appear in the backing file format field. Signed-off-by: Peter Krempa Reviewed-by: Vladimir Sementsov-Ogievskiy Message-ID: <2cb46e37093ce793ea1604abc8bbb90f4c8e434b.1701796348.git.pkrempa@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- qapi/block-core.json | 8 +++++- include/block/block-global-state.h | 3 ++- include/block/block_int-common.h | 4 ++- include/block/block_int-global-state.h | 3 +++ block.c | 37 +++++++++++++++++++++----- block/commit.c | 6 ++++- blockdev.c | 6 +++++ tests/unit/test-bdrv-drain.c | 3 ++- 8 files changed, 58 insertions(+), 12 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index ca390c5700..726145ec8a 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1810,6 +1810,11 @@ # Care should be taken when specifying the string, to specify a # valid filename or protocol. (Since 2.1) # +# @backing-mask-protocol: If true, replace any protocol mentioned in the +# 'backing file format' with 'raw', rather than storing the protocol +# name as the backing format. Can be used even when no image header +# will be updated (default false; since 9.0). +# # @speed: the maximum speed, in bytes per second # # @on-error: the action to take on an error. 'ignore' means that the @@ -1856,7 +1861,8 @@ '*base': { 'type': 'str', 'features': [ 'deprecated' ] }, '*top-node': 'str', '*top': { 'type': 'str', 'features': [ 'deprecated' ] }, - '*backing-file': 'str', '*speed': 'int', + '*backing-file': 'str', '*backing-mask-protocol': 'bool', + '*speed': 'int', '*on-error': 'BlockdevOnError', '*filter-node-name': 'str', '*auto-finalize': 'bool', '*auto-dismiss': 'bool' }, diff --git a/include/block/block-global-state.h b/include/block/block-global-state.h index 4ec0b217f0..bd7cecd1cf 100644 --- a/include/block/block-global-state.h +++ b/include/block/block-global-state.h @@ -144,7 +144,8 @@ int GRAPH_RDLOCK bdrv_make_empty(BdrvChild *c, Error **errp); void bdrv_register(BlockDriver *bdrv); int bdrv_drop_intermediate(BlockDriverState *top, BlockDriverState *base, - const char *backing_file_str); + const char *backing_file_str, + bool backing_mask_protocol); BlockDriverState * GRAPH_RDLOCK bdrv_find_overlay(BlockDriverState *active, BlockDriverState *bs); diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h index 151279d481..761276127e 100644 --- a/include/block/block_int-common.h +++ b/include/block/block_int-common.h @@ -985,7 +985,9 @@ struct BdrvChildClass { * can update its reference. */ int (*update_filename)(BdrvChild *child, BlockDriverState *new_base, - const char *filename, Error **errp); + const char *filename, + bool backing_mask_protocol, + Error **errp); bool (*change_aio_ctx)(BdrvChild *child, AioContext *ctx, GHashTable *visited, Transaction *tran, diff --git a/include/block/block_int-global-state.h b/include/block/block_int-global-state.h index ef31c58bb3..2162269df6 100644 --- a/include/block/block_int-global-state.h +++ b/include/block/block_int-global-state.h @@ -82,6 +82,8 @@ void stream_start(const char *job_id, BlockDriverState *bs, * @speed: The maximum speed, in bytes per second, or 0 for unlimited. * @on_error: The action to take upon error. * @backing_file_str: String to use as the backing file in @top's overlay + * @backing_mask_protocol: Replace potential protocol name with 'raw' in + * 'backing file format' header * @filter_node_name: The node name that should be assigned to the filter * driver that the commit job inserts into the graph above @top. NULL means * that a node name should be autogenerated. @@ -92,6 +94,7 @@ void commit_start(const char *job_id, BlockDriverState *bs, BlockDriverState *base, BlockDriverState *top, int creation_flags, int64_t speed, BlockdevOnError on_error, const char *backing_file_str, + bool backing_mask_protocol, const char *filter_node_name, Error **errp); /** * commit_active_start: diff --git a/block.c b/block.c index a097772238..30afdcbba6 100644 --- a/block.c +++ b/block.c @@ -1309,11 +1309,14 @@ static void bdrv_backing_detach(BdrvChild *c) } static int bdrv_backing_update_filename(BdrvChild *c, BlockDriverState *base, - const char *filename, Error **errp) + const char *filename, + bool backing_mask_protocol, + Error **errp) { BlockDriverState *parent = c->opaque; bool read_only = bdrv_is_read_only(parent); int ret; + const char *format_name; GLOBAL_STATE_CODE(); if (read_only) { @@ -1323,9 +1326,23 @@ static int bdrv_backing_update_filename(BdrvChild *c, BlockDriverState *base, } } - ret = bdrv_change_backing_file(parent, filename, - base->drv ? base->drv->format_name : "", - false); + if (base->drv) { + /* + * If the new base image doesn't have a format driver layer, which we + * detect by the fact that @base is a protocol driver, we record + * 'raw' as the format instead of putting the protocol name as the + * backing format + */ + if (backing_mask_protocol && base->drv->protocol_name) { + format_name = "raw"; + } else { + format_name = base->drv->format_name; + } + } else { + format_name = ""; + } + + ret = bdrv_change_backing_file(parent, filename, format_name, false); if (ret < 0) { error_setg_errno(errp, -ret, "Could not update backing file link"); } @@ -1479,10 +1496,14 @@ static void GRAPH_WRLOCK bdrv_child_cb_detach(BdrvChild *child) } static int bdrv_child_cb_update_filename(BdrvChild *c, BlockDriverState *base, - const char *filename, Error **errp) + const char *filename, + bool backing_mask_protocol, + Error **errp) { if (c->role & BDRV_CHILD_COW) { - return bdrv_backing_update_filename(c, base, filename, errp); + return bdrv_backing_update_filename(c, base, filename, + backing_mask_protocol, + errp); } return 0; } @@ -5803,7 +5824,8 @@ void bdrv_unfreeze_backing_chain(BlockDriverState *bs, BlockDriverState *base) * */ int bdrv_drop_intermediate(BlockDriverState *top, BlockDriverState *base, - const char *backing_file_str) + const char *backing_file_str, + bool backing_mask_protocol) { BlockDriverState *explicit_top = top; bool update_inherits_from; @@ -5869,6 +5891,7 @@ int bdrv_drop_intermediate(BlockDriverState *top, BlockDriverState *base, if (c->klass->update_filename) { ret = c->klass->update_filename(c, base, backing_file_str, + backing_mask_protocol, &local_err); if (ret < 0) { /* diff --git a/block/commit.c b/block/commit.c index 1dd7a65ffb..7c3fdcb0ca 100644 --- a/block/commit.c +++ b/block/commit.c @@ -42,6 +42,7 @@ typedef struct CommitBlockJob { bool base_read_only; bool chain_frozen; char *backing_file_str; + bool backing_mask_protocol; } CommitBlockJob; static int commit_prepare(Job *job) @@ -61,7 +62,8 @@ static int commit_prepare(Job *job) /* FIXME: bdrv_drop_intermediate treats total failures and partial failures * identically. Further work is needed to disambiguate these cases. */ return bdrv_drop_intermediate(s->commit_top_bs, s->base_bs, - s->backing_file_str); + s->backing_file_str, + s->backing_mask_protocol); } static void commit_abort(Job *job) @@ -254,6 +256,7 @@ void commit_start(const char *job_id, BlockDriverState *bs, BlockDriverState *base, BlockDriverState *top, int creation_flags, int64_t speed, BlockdevOnError on_error, const char *backing_file_str, + bool backing_mask_protocol, const char *filter_node_name, Error **errp) { CommitBlockJob *s; @@ -408,6 +411,7 @@ void commit_start(const char *job_id, BlockDriverState *bs, blk_set_disable_request_queuing(s->top, true); s->backing_file_str = g_strdup(backing_file_str); + s->backing_mask_protocol = backing_mask_protocol; s->on_error = on_error; trace_commit_start(bs, base, top, s); diff --git a/blockdev.c b/blockdev.c index 3a5e7222ec..292c8af3f9 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2420,6 +2420,8 @@ void qmp_block_commit(const char *job_id, const char *device, const char *top_node, const char *top, const char *backing_file, + bool has_backing_mask_protocol, + bool backing_mask_protocol, bool has_speed, int64_t speed, bool has_on_error, BlockdevOnError on_error, const char *filter_node_name, @@ -2450,6 +2452,9 @@ void qmp_block_commit(const char *job_id, const char *device, if (has_auto_dismiss && !auto_dismiss) { job_flags |= JOB_MANUAL_DISMISS; } + if (!has_backing_mask_protocol) { + backing_mask_protocol = false; + } /* Important Note: * libvirt relies on the DeviceNotFound error class in order to probe for @@ -2591,6 +2596,7 @@ void qmp_block_commit(const char *job_id, const char *device, } commit_start(job_id, bs, base_bs, top_bs, job_flags, speed, on_error, backing_file, + backing_mask_protocol, filter_node_name, &local_err); } if (local_err != NULL) { diff --git a/tests/unit/test-bdrv-drain.c b/tests/unit/test-bdrv-drain.c index 17830a69c1..666880472b 100644 --- a/tests/unit/test-bdrv-drain.c +++ b/tests/unit/test-bdrv-drain.c @@ -1591,6 +1591,7 @@ static const BlockJobDriver test_simple_job_driver = { static int drop_intermediate_poll_update_filename(BdrvChild *child, BlockDriverState *new_base, const char *filename, + bool backing_mask_protocol, Error **errp) { /* @@ -1702,7 +1703,7 @@ static void test_drop_intermediate_poll(void) job->should_complete = true; g_assert(!job_has_completed); - ret = bdrv_drop_intermediate(chain[1], chain[0], NULL); + ret = bdrv_drop_intermediate(chain[1], chain[0], NULL, false); aio_poll(qemu_get_aio_context(), false); g_assert(ret == 0); g_assert(job_has_completed); From patchwork Fri Jan 19 18:13:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1888620 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=AD+91MWF; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TGnpj6dm2z1yPV for ; Sat, 20 Jan 2024 05:15:25 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rQtNN-0001k5-CB; Fri, 19 Jan 2024 13:13:57 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtN7-0001cs-RV for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:43 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtN5-0007aL-UY for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:41 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705688018; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AFh+AqQpoSOmGs1u/gVZMucu4s6GGMQAjVb0Pg0NBuc=; b=AD+91MWFnX6o4n7N/4m9WCXC76c2yaWD201VXmJpt/Zw6C+YHEUW73sqkEXgikFeQzsFyZ syw/Z6u6CKL7nhN996CMRjHqph2cDUKjQpj3TFxMqwk8B79IWg6YeAaxBLXSN/wel95ZFi gYWZr28zzuy1BDI4mW6NBSidXbnNEZg= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-360-AE59Fc6_OWOnu-UP72imTw-1; Fri, 19 Jan 2024 13:13:36 -0500 X-MC-Unique: AE59Fc6_OWOnu-UP72imTw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2AE513C28672; Fri, 19 Jan 2024 18:13:36 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.86]) by smtp.corp.redhat.com (Postfix) with ESMTP id 91A1F494; Fri, 19 Jan 2024 18:13:35 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 04/14] stream: Allow users to request only format driver names in backing file format Date: Fri, 19 Jan 2024 19:13:17 +0100 Message-ID: <20240119181327.236745-5-kwolf@redhat.com> In-Reply-To: <20240119181327.236745-1-kwolf@redhat.com> References: <20240119181327.236745-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.519, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Peter Krempa Introduce a new flag 'backing-mask-protocol' for the block-stream QMP command which instructs the internals to use 'raw' instead of the protocol driver in case when a image is used without a dummy 'raw' wrapper. The flag is designed such that it can be always asserted by management tools even when there isn't any update to backing files. The flag will be used by libvirt so that the backing images still reference the proper format even when libvirt will stop using the dummy raw driver (raw driver with no other config). Libvirt needs this so that the images stay compatible with older libvirt versions which didn't expect that a protocol driver name can appear in the backing file format field. Signed-off-by: Peter Krempa Reviewed-by: Vladimir Sementsov-Ogievskiy Message-ID: Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- qapi/block-core.json | 9 ++++++++- include/block/block_int-global-state.h | 3 +++ block/monitor/block-hmp-cmds.c | 2 +- block/stream.c | 10 +++++++++- blockdev.c | 7 +++++++ 5 files changed, 28 insertions(+), 3 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 726145ec8a..04982bff96 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2826,6 +2826,11 @@ # Care should be taken when specifying the string, to specify a # valid filename or protocol. (Since 2.1) # +# @backing-mask-protocol: If true, replace any protocol mentioned in the +# 'backing file format' with 'raw', rather than storing the protocol +# name as the backing format. Can be used even when no image header +# will be updated (default false; since 9.0). +# # @speed: the maximum speed, in bytes per second # # @on-error: the action to take on an error (default report). 'stop' @@ -2864,7 +2869,9 @@ ## { 'command': 'block-stream', 'data': { '*job-id': 'str', 'device': 'str', '*base': 'str', - '*base-node': 'str', '*backing-file': 'str', '*bottom': 'str', + '*base-node': 'str', '*backing-file': 'str', + '*backing-mask-protocol': 'bool', + '*bottom': 'str', '*speed': 'int', '*on-error': 'BlockdevOnError', '*filter-node-name': 'str', '*auto-finalize': 'bool', '*auto-dismiss': 'bool' }, diff --git a/include/block/block_int-global-state.h b/include/block/block_int-global-state.h index 2162269df6..d2201e27f4 100644 --- a/include/block/block_int-global-state.h +++ b/include/block/block_int-global-state.h @@ -46,6 +46,8 @@ * flatten the whole backing file chain onto @bs. * @backing_file_str: The file name that will be written to @bs as the * the new backing file if the job completes. Ignored if @base is %NULL. + * @backing_mask_protocol: Replace potential protocol name with 'raw' in + * 'backing file format' header * @creation_flags: Flags that control the behavior of the Job lifetime. * See @BlockJobCreateFlags * @speed: The maximum speed, in bytes per second, or 0 for unlimited. @@ -64,6 +66,7 @@ */ void stream_start(const char *job_id, BlockDriverState *bs, BlockDriverState *base, const char *backing_file_str, + bool backing_mask_protocol, BlockDriverState *bottom, int creation_flags, int64_t speed, BlockdevOnError on_error, diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c index bdbb5cb141..d954bec6f1 100644 --- a/block/monitor/block-hmp-cmds.c +++ b/block/monitor/block-hmp-cmds.c @@ -496,7 +496,7 @@ void hmp_block_stream(Monitor *mon, const QDict *qdict) const char *base = qdict_get_try_str(qdict, "base"); int64_t speed = qdict_get_try_int(qdict, "speed", 0); - qmp_block_stream(device, device, base, NULL, NULL, NULL, + qmp_block_stream(device, device, base, NULL, NULL, false, false, NULL, qdict_haskey(qdict, "speed"), speed, true, BLOCKDEV_ON_ERROR_REPORT, NULL, false, false, false, false, &error); diff --git a/block/stream.c b/block/stream.c index 048c2d282f..7031eef12b 100644 --- a/block/stream.c +++ b/block/stream.c @@ -39,6 +39,7 @@ typedef struct StreamBlockJob { BlockDriverState *target_bs; BlockdevOnError on_error; char *backing_file_str; + bool backing_mask_protocol; bool bs_read_only; } StreamBlockJob; @@ -95,7 +96,12 @@ static int stream_prepare(Job *job) if (unfiltered_base) { base_id = s->backing_file_str ?: unfiltered_base->filename; if (unfiltered_base->drv) { - base_fmt = unfiltered_base->drv->format_name; + if (s->backing_mask_protocol && + unfiltered_base->drv->protocol_name) { + base_fmt = "raw"; + } else { + base_fmt = unfiltered_base->drv->format_name; + } } } @@ -247,6 +253,7 @@ static const BlockJobDriver stream_job_driver = { void stream_start(const char *job_id, BlockDriverState *bs, BlockDriverState *base, const char *backing_file_str, + bool backing_mask_protocol, BlockDriverState *bottom, int creation_flags, int64_t speed, BlockdevOnError on_error, @@ -398,6 +405,7 @@ void stream_start(const char *job_id, BlockDriverState *bs, s->base_overlay = base_overlay; s->above_base = above_base; s->backing_file_str = g_strdup(backing_file_str); + s->backing_mask_protocol = backing_mask_protocol; s->cor_filter_bs = cor_filter_bs; s->target_bs = bs; s->bs_read_only = bs_read_only; diff --git a/blockdev.c b/blockdev.c index 292c8af3f9..f8bb0932f8 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2278,6 +2278,8 @@ void qmp_block_stream(const char *job_id, const char *device, const char *base, const char *base_node, const char *backing_file, + bool has_backing_mask_protocol, + bool backing_mask_protocol, const char *bottom, bool has_speed, int64_t speed, bool has_on_error, BlockdevOnError on_error, @@ -2313,6 +2315,10 @@ void qmp_block_stream(const char *job_id, const char *device, return; } + if (!has_backing_mask_protocol) { + backing_mask_protocol = false; + } + if (!has_on_error) { on_error = BLOCKDEV_ON_ERROR_REPORT; } @@ -2400,6 +2406,7 @@ void qmp_block_stream(const char *job_id, const char *device, } stream_start(job_id, bs, base_bs, backing_file, + backing_mask_protocol, bottom_bs, job_flags, has_speed ? speed : 0, on_error, filter_node_name, &local_err); if (local_err) { From patchwork Fri Jan 19 18:13:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1888627 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=TcAqEGH2; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TGnqn03Dkz1yPV for ; Sat, 20 Jan 2024 05:16:21 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rQtNM-0001jZ-O7; Fri, 19 Jan 2024 13:13:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtNL-0001hq-01 for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:55 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtNI-0007hV-A1 for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705688031; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=O9voIAbYyAwKOHCB3ugMdMW49+CPAUlQc5sS8VazD/s=; b=TcAqEGH2mhgbLOeCf5U0EtjrL9HqWINZVJIrDuKBwueZ21RohXedxCo7QMz2x/gt6OfdZp XcEW/8S+EwJmtbiRPODdxXKjK5kdz7S38Hau0kjWtmdS6Nm/bjmwfpt9dtoa/a3D02n6Wi AgD6N6Fpxyq28ut/Ag4NZ7j8SB3P96w= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-313-h4HxAFeQN7ag4dygb7TWGg-1; Fri, 19 Jan 2024 13:13:37 -0500 X-MC-Unique: h4HxAFeQN7ag4dygb7TWGg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EE88910665A0; Fri, 19 Jan 2024 18:13:36 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.86]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5FDA32877; Fri, 19 Jan 2024 18:13:36 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 05/14] iotests: add filter_qmp_generated_node_ids() Date: Fri, 19 Jan 2024 19:13:18 +0100 Message-ID: <20240119181327.236745-6-kwolf@redhat.com> In-Reply-To: <20240119181327.236745-1-kwolf@redhat.com> References: <20240119181327.236745-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.519, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Stefan Hajnoczi Add a filter function for QMP responses that contain QEMU's automatically generated node ids. The ids change between runs and must be masked in the reference output. The next commit will use this new function. Signed-off-by: Stefan Hajnoczi Message-ID: <20240118144823.1497953-2-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- tests/qemu-iotests/iotests.py | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index e5c5798c71..ea48af4a7b 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -651,6 +651,13 @@ def _filter(_key, value): def filter_generated_node_ids(msg): return re.sub("#block[0-9]+", "NODE_NAME", msg) +def filter_qmp_generated_node_ids(qmsg): + def _filter(_key, value): + if is_str(value): + return filter_generated_node_ids(value) + return value + return filter_qmp(qmsg, _filter) + def filter_img_info(output: str, filename: str, drop_child_info: bool = True) -> str: lines = [] From patchwork Fri Jan 19 18:13:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1888619 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=N1w1sfeM; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TGnpd2wdyz1yPV for ; Sat, 20 Jan 2024 05:15:21 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rQtNL-0001ih-PW; Fri, 19 Jan 2024 13:13:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtNE-0001eH-Hf for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:50 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtN9-0007dZ-9R for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705688022; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xX/LqETDPHHCAR/ixbnRDJdbnZadDBuYH5MHn0GrT44=; b=N1w1sfeM0U9BsbH+d35nbC/zWJsvUnuDlpwCKGXvdge0zrdM9alVF/icESzPV/o27d8o/e Dhzu2fguRBjG/2j4RLw1vgiVfOXszr/BJf3Nke7zIqG+nD42+2owNm/K/edx9mhllOW6Ll Ndu/30HwNfHxOlmxEwO3ZICclZenLAc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-117-JIGSowUWNAq30O0xFNTt-A-1; Fri, 19 Jan 2024 13:13:38 -0500 X-MC-Unique: JIGSowUWNAq30O0xFNTt-A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D989A86794D; Fri, 19 Jan 2024 18:13:37 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.86]) by smtp.corp.redhat.com (Postfix) with ESMTP id 31002494; Fri, 19 Jan 2024 18:13:37 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 06/14] iotests: port 141 to Python for reliable QMP testing Date: Fri, 19 Jan 2024 19:13:19 +0100 Message-ID: <20240119181327.236745-7-kwolf@redhat.com> In-Reply-To: <20240119181327.236745-1-kwolf@redhat.com> References: <20240119181327.236745-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.519, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Stefan Hajnoczi The common.qemu bash functions allow tests to interact with the QMP monitor of a QEMU process. I spent two days trying to update 141 when the order of the test output changed, but found it would still fail occassionally because printf() and QMP events race with synchronous QMP communication. I gave up and ported 141 to the existing Python API for QMP tests. The Python API is less affected by the order in which QEMU prints output because it does not print all QMP traffic by default. The next commit changes the order in which QMP messages are received. Make 141 reliable first. Cc: Hanna Czenczek Signed-off-by: Stefan Hajnoczi Message-ID: <20240118144823.1497953-3-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- tests/qemu-iotests/141 | 307 ++++++++++++++++--------------------- tests/qemu-iotests/141.out | 200 ++++++------------------ 2 files changed, 176 insertions(+), 331 deletions(-) diff --git a/tests/qemu-iotests/141 b/tests/qemu-iotests/141 index a37030ee17..a7d3985a02 100755 --- a/tests/qemu-iotests/141 +++ b/tests/qemu-iotests/141 @@ -1,9 +1,12 @@ -#!/usr/bin/env bash +#!/usr/bin/env python3 # group: rw auto quick # # Test case for ejecting BDSs with block jobs still running on them # -# Copyright (C) 2016 Red Hat, Inc. +# Originally written in bash by Hanna Czenczek, ported to Python by Stefan +# Hajnoczi. +# +# Copyright Red Hat # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -19,177 +22,129 @@ # along with this program. If not, see . # -# creator -owner=hreitz@redhat.com - -seq="$(basename $0)" -echo "QA output created by $seq" - -status=1 # failure is the default! - -_cleanup() -{ - _cleanup_qemu - _cleanup_test_img - for img in "$TEST_DIR"/{b,m,o}.$IMGFMT; do - _rm_test_img "$img" - done -} -trap "_cleanup; exit \$status" 0 1 2 3 15 - -# get standard environment, filters and checks -. ./common.rc -. ./common.filter -. ./common.qemu - -# Needs backing file and backing format support -_supported_fmt qcow2 qed -_supported_proto file -_supported_os Linux - - -test_blockjob() -{ - _send_qemu_cmd $QEMU_HANDLE \ - "{'execute': 'blockdev-add', - 'arguments': { - 'node-name': 'drv0', - 'driver': '$IMGFMT', - 'file': { - 'driver': 'file', - 'filename': '$TEST_IMG' - }}}" \ - 'return' - - # If "$2" is an event, we may or may not see it before the - # {"return": {}}. Therefore, filter the {"return": {}} out both - # here and in the next command. (Naturally, if we do not see it - # here, we will see it before the next command can be executed, - # so it will appear in the next _send_qemu_cmd's output.) - _send_qemu_cmd $QEMU_HANDLE \ - "$1" \ - "$2" \ - | _filter_img_create | _filter_qmp_empty_return - - # We want this to return an error because the block job is still running - _send_qemu_cmd $QEMU_HANDLE \ - "{'execute': 'blockdev-del', - 'arguments': {'node-name': 'drv0'}}" \ - 'error' | _filter_generated_node_ids | _filter_qmp_empty_return - - _send_qemu_cmd $QEMU_HANDLE \ - "{'execute': 'block-job-cancel', - 'arguments': {'device': 'job0'}}" \ - "$3" - - _send_qemu_cmd $QEMU_HANDLE \ - "{'execute': 'blockdev-del', - 'arguments': {'node-name': 'drv0'}}" \ - 'return' -} - - -TEST_IMG="$TEST_DIR/b.$IMGFMT" _make_test_img 1M -TEST_IMG="$TEST_DIR/m.$IMGFMT" _make_test_img -b "$TEST_DIR/b.$IMGFMT" -F $IMGFMT 1M -_make_test_img -b "$TEST_DIR/m.$IMGFMT" 1M -F $IMGFMT - -_launch_qemu -nodefaults - -_send_qemu_cmd $QEMU_HANDLE \ - "{'execute': 'qmp_capabilities'}" \ - 'return' - -echo -echo '=== Testing drive-backup ===' -echo - -# drive-backup will not send BLOCK_JOB_READY by itself, and cancelling the job -# will consequently result in BLOCK_JOB_CANCELLED being emitted. - -test_blockjob \ - "{'execute': 'drive-backup', - 'arguments': {'job-id': 'job0', - 'device': 'drv0', - 'target': '$TEST_DIR/o.$IMGFMT', - 'format': '$IMGFMT', - 'sync': 'none'}}" \ - 'return' \ - '"status": "null"' - -echo -echo '=== Testing drive-mirror ===' -echo - -# drive-mirror will send BLOCK_JOB_READY basically immediately, and cancelling -# the job will consequently result in BLOCK_JOB_COMPLETED being emitted. - -test_blockjob \ - "{'execute': 'drive-mirror', - 'arguments': {'job-id': 'job0', - 'device': 'drv0', - 'target': '$TEST_DIR/o.$IMGFMT', - 'format': '$IMGFMT', - 'sync': 'none'}}" \ - 'BLOCK_JOB_READY' \ - '"status": "null"' - -echo -echo '=== Testing active block-commit ===' -echo - -# An active block-commit will send BLOCK_JOB_READY basically immediately, and -# cancelling the job will consequently result in BLOCK_JOB_COMPLETED being -# emitted. - -test_blockjob \ - "{'execute': 'block-commit', - 'arguments': {'job-id': 'job0', 'device': 'drv0'}}" \ - 'BLOCK_JOB_READY' \ - '"status": "null"' - -echo -echo '=== Testing non-active block-commit ===' -echo - -# Give block-commit something to work on, otherwise it would be done -# immediately, send a BLOCK_JOB_COMPLETED and ejecting the BDS would work just -# fine without the block job still running. - -$QEMU_IO -c 'write 0 1M' "$TEST_DIR/m.$IMGFMT" | _filter_qemu_io - -test_blockjob \ - "{'execute': 'block-commit', - 'arguments': {'job-id': 'job0', - 'device': 'drv0', - 'top': '$TEST_DIR/m.$IMGFMT', - 'speed': 1}}" \ - 'return' \ - '"status": "null"' - -echo -echo '=== Testing block-stream ===' -echo - -# Give block-stream something to work on, otherwise it would be done -# immediately, send a BLOCK_JOB_COMPLETED and ejecting the BDS would work just -# fine without the block job still running. - -$QEMU_IO -c 'write 0 1M' "$TEST_DIR/b.$IMGFMT" | _filter_qemu_io - -# With some data to stream (and @speed set to 1), block-stream will not complete -# until we send the block-job-cancel command. - -test_blockjob \ - "{'execute': 'block-stream', - 'arguments': {'job-id': 'job0', - 'device': 'drv0', - 'speed': 1}}" \ - 'return' \ - '"status": "null"' - -_cleanup_qemu - -# success, all done -echo "*** done" -rm -f $seq.full -status=0 +import iotests + +# Common filters to mask values that vary in the test output +QMP_FILTERS = [iotests.filter_qmp_testfiles, \ + iotests.filter_qmp_imgfmt] + + +class TestCase: + def __init__(self, name, vm, image_path, cancel_event): + self.name = name + self.vm = vm + self.image_path = image_path + self.cancel_event = cancel_event + + def __enter__(self): + iotests.log(f'=== Testing {self.name} ===') + self.vm.qmp_log('blockdev-add', \ + node_name='drv0', \ + driver=iotests.imgfmt, \ + file={'driver': 'file', 'filename': self.image_path}, \ + filters=QMP_FILTERS) + + def __exit__(self, *exc_details): + # This is expected to fail because the job still exists + self.vm.qmp_log('blockdev-del', node_name='drv0', \ + filters=[iotests.filter_qmp_generated_node_ids]) + + self.vm.qmp_log('block-job-cancel', device='job0') + event = self.vm.event_wait(self.cancel_event) + iotests.log(event, filters=[iotests.filter_qmp_event]) + + # This time it succeeds + self.vm.qmp_log('blockdev-del', node_name='drv0') + + # Separate test cases in output + iotests.log('') + + +def main() -> None: + with iotests.FilePath('bottom', 'middle', 'top', 'target') as \ + (bottom_path, middle_path, top_path, target_path), \ + iotests.VM() as vm: + + iotests.log('Creating bottom <- middle <- top backing file chain...') + IMAGE_SIZE='1M' + iotests.qemu_img_create('-f', iotests.imgfmt, bottom_path, IMAGE_SIZE) + iotests.qemu_img_create('-f', iotests.imgfmt, \ + '-F', iotests.imgfmt, \ + '-b', bottom_path, \ + middle_path, \ + IMAGE_SIZE) + iotests.qemu_img_create('-f', iotests.imgfmt, \ + '-F', iotests.imgfmt, \ + '-b', middle_path, \ + top_path, \ + IMAGE_SIZE) + + iotests.log('Starting VM...') + vm.add_args('-nodefaults') + vm.launch() + + # drive-backup will not send BLOCK_JOB_READY by itself, and cancelling + # the job will consequently result in BLOCK_JOB_CANCELLED being + # emitted. + with TestCase('drive-backup', vm, top_path, 'BLOCK_JOB_CANCELLED'): + vm.qmp_log('drive-backup', \ + job_id='job0', \ + device='drv0', \ + target=target_path, \ + format=iotests.imgfmt, \ + sync='none', \ + filters=QMP_FILTERS) + + # drive-mirror will send BLOCK_JOB_READY basically immediately, and + # cancelling the job will consequently result in BLOCK_JOB_COMPLETED + # being emitted. + with TestCase('drive-mirror', vm, top_path, 'BLOCK_JOB_COMPLETED'): + vm.qmp_log('drive-mirror', \ + job_id='job0', \ + device='drv0', \ + target=target_path, \ + format=iotests.imgfmt, \ + sync='none', \ + filters=QMP_FILTERS) + event = vm.event_wait('BLOCK_JOB_READY') + assert event is not None # silence mypy + iotests.log(event, filters=[iotests.filter_qmp_event]) + + # An active block-commit will send BLOCK_JOB_READY basically + # immediately, and cancelling the job will consequently result in + # BLOCK_JOB_COMPLETED being emitted. + with TestCase('active block-commit', vm, top_path, \ + 'BLOCK_JOB_COMPLETED'): + vm.qmp_log('block-commit', \ + job_id='job0', \ + device='drv0') + event = vm.event_wait('BLOCK_JOB_READY') + assert event is not None # silence mypy + iotests.log(event, filters=[iotests.filter_qmp_event]) + + # Give block-commit something to work on, otherwise it would be done + # immediately, send a BLOCK_JOB_COMPLETED and ejecting the BDS would + # work just fine without the block job still running. + iotests.qemu_io(middle_path, '-c', f'write 0 {IMAGE_SIZE}') + with TestCase('non-active block-commit', vm, top_path, \ + 'BLOCK_JOB_CANCELLED'): + vm.qmp_log('block-commit', \ + job_id='job0', \ + device='drv0', \ + top=middle_path, \ + speed=1, \ + filters=[iotests.filter_qmp_testfiles]) + + # Give block-stream something to work on, otherwise it would be done + # immediately, send a BLOCK_JOB_COMPLETED and ejecting the BDS would + # work just fine without the block job still running. + iotests.qemu_io(bottom_path, '-c', f'write 0 {IMAGE_SIZE}') + with TestCase('block-stream', vm, top_path, 'BLOCK_JOB_CANCELLED'): + vm.qmp_log('block-stream', \ + job_id='job0', \ + device='drv0', \ + speed=1) + +if __name__ == '__main__': + iotests.script_main(main, supported_fmts=['qcow2', 'qed'], + supported_protocols=['file']) diff --git a/tests/qemu-iotests/141.out b/tests/qemu-iotests/141.out index 63203d9944..91b7ba50af 100644 --- a/tests/qemu-iotests/141.out +++ b/tests/qemu-iotests/141.out @@ -1,179 +1,69 @@ -QA output created by 141 -Formatting 'TEST_DIR/b.IMGFMT', fmt=IMGFMT size=1048576 -Formatting 'TEST_DIR/m.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/b.IMGFMT backing_fmt=IMGFMT -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/m.IMGFMT backing_fmt=IMGFMT -{'execute': 'qmp_capabilities'} -{"return": {}} - +Creating bottom <- middle <- top backing file chain... +Starting VM... === Testing drive-backup === - -{'execute': 'blockdev-add', - 'arguments': { - 'node-name': 'drv0', - 'driver': 'IMGFMT', - 'file': { - 'driver': 'file', - 'filename': 'TEST_DIR/t.IMGFMT' - }}} -{"return": {}} -{'execute': 'drive-backup', -'arguments': {'job-id': 'job0', -'device': 'drv0', -'target': 'TEST_DIR/o.IMGFMT', -'format': 'IMGFMT', -'sync': 'none'}} -Formatting 'TEST_DIR/o.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.IMGFMT backing_fmt=IMGFMT -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}} -{'execute': 'blockdev-del', - 'arguments': {'node-name': 'drv0'}} +{"execute": "blockdev-add", "arguments": {"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top"}, "node-name": "drv0"}} +{"return": {}} +{"execute": "drive-backup", "arguments": {"device": "drv0", "format": "IMGFMT", "job-id": "job0", "sync": "none", "target": "TEST_DIR/PID-target"}} +{"return": {}} +{"execute": "blockdev-del", "arguments": {"node-name": "drv0"}} {"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: node is used as backing hd of 'NODE_NAME'"}} -{'execute': 'block-job-cancel', - 'arguments': {'device': 'job0'}} +{"execute": "block-job-cancel", "arguments": {"device": "job0"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "job0", "len": 1048576, "offset": 0, "speed": 0, "type": "backup"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job0"}} -{'execute': 'blockdev-del', - 'arguments': {'node-name': 'drv0'}} +{"data": {"device": "job0", "len": 1048576, "offset": 0, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"execute": "blockdev-del", "arguments": {"node-name": "drv0"}} {"return": {}} === Testing drive-mirror === - -{'execute': 'blockdev-add', - 'arguments': { - 'node-name': 'drv0', - 'driver': 'IMGFMT', - 'file': { - 'driver': 'file', - 'filename': 'TEST_DIR/t.IMGFMT' - }}} -{"return": {}} -{'execute': 'drive-mirror', -'arguments': {'job-id': 'job0', -'device': 'drv0', -'target': 'TEST_DIR/o.IMGFMT', -'format': 'IMGFMT', -'sync': 'none'}} -Formatting 'TEST_DIR/o.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.IMGFMT backing_fmt=IMGFMT -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "mirror"}} -{'execute': 'blockdev-del', - 'arguments': {'node-name': 'drv0'}} +{"execute": "blockdev-add", "arguments": {"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top"}, "node-name": "drv0"}} +{"return": {}} +{"execute": "drive-mirror", "arguments": {"device": "drv0", "format": "IMGFMT", "job-id": "job0", "sync": "none", "target": "TEST_DIR/PID-target"}} +{"return": {}} +{"data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"execute": "blockdev-del", "arguments": {"node-name": "drv0"}} {"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: block device is in use by block job: mirror"}} -{'execute': 'block-job-cancel', - 'arguments': {'device': 'job0'}} +{"execute": "block-job-cancel", "arguments": {"device": "job0"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "mirror"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job0"}} -{'execute': 'blockdev-del', - 'arguments': {'node-name': 'drv0'}} +{"data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"execute": "blockdev-del", "arguments": {"node-name": "drv0"}} {"return": {}} === Testing active block-commit === - -{'execute': 'blockdev-add', - 'arguments': { - 'node-name': 'drv0', - 'driver': 'IMGFMT', - 'file': { - 'driver': 'file', - 'filename': 'TEST_DIR/t.IMGFMT' - }}} -{"return": {}} -{'execute': 'block-commit', -'arguments': {'job-id': 'job0', 'device': 'drv0'}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}} -{'execute': 'blockdev-del', - 'arguments': {'node-name': 'drv0'}} +{"execute": "blockdev-add", "arguments": {"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top"}, "node-name": "drv0"}} +{"return": {}} +{"execute": "block-commit", "arguments": {"device": "drv0", "job-id": "job0"}} +{"return": {}} +{"data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"execute": "blockdev-del", "arguments": {"node-name": "drv0"}} {"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: block device is in use by block job: commit"}} -{'execute': 'block-job-cancel', - 'arguments': {'device': 'job0'}} +{"execute": "block-job-cancel", "arguments": {"device": "job0"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job0"}} -{'execute': 'blockdev-del', - 'arguments': {'node-name': 'drv0'}} +{"data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"execute": "blockdev-del", "arguments": {"node-name": "drv0"}} {"return": {}} === Testing non-active block-commit === - -wrote 1048576/1048576 bytes at offset 0 -1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -{'execute': 'blockdev-add', - 'arguments': { - 'node-name': 'drv0', - 'driver': 'IMGFMT', - 'file': { - 'driver': 'file', - 'filename': 'TEST_DIR/t.IMGFMT' - }}} -{"return": {}} -{'execute': 'block-commit', -'arguments': {'job-id': 'job0', -'device': 'drv0', -'top': 'TEST_DIR/m.IMGFMT', -'speed': 1}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}} -{'execute': 'blockdev-del', - 'arguments': {'node-name': 'drv0'}} +{"execute": "blockdev-add", "arguments": {"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top"}, "node-name": "drv0"}} +{"return": {}} +{"execute": "block-commit", "arguments": {"device": "drv0", "job-id": "job0", "speed": 1, "top": "TEST_DIR/PID-middle"}} +{"return": {}} +{"execute": "blockdev-del", "arguments": {"node-name": "drv0"}} {"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: block device is in use by block job: commit"}} -{'execute': 'block-job-cancel', - 'arguments': {'device': 'job0'}} +{"execute": "block-job-cancel", "arguments": {"device": "job0"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "job0", "len": 1048576, "offset": 524288, "speed": 1, "type": "commit"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job0"}} -{'execute': 'blockdev-del', - 'arguments': {'node-name': 'drv0'}} +{"data": {"device": "job0", "len": 1048576, "offset": 524288, "speed": 1, "type": "commit"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"execute": "blockdev-del", "arguments": {"node-name": "drv0"}} {"return": {}} === Testing block-stream === - -wrote 1048576/1048576 bytes at offset 0 -1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -{'execute': 'blockdev-add', - 'arguments': { - 'node-name': 'drv0', - 'driver': 'IMGFMT', - 'file': { - 'driver': 'file', - 'filename': 'TEST_DIR/t.IMGFMT' - }}} -{"return": {}} -{'execute': 'block-stream', -'arguments': {'job-id': 'job0', -'device': 'drv0', -'speed': 1}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}} -{'execute': 'blockdev-del', - 'arguments': {'node-name': 'drv0'}} +{"execute": "blockdev-add", "arguments": {"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top"}, "node-name": "drv0"}} +{"return": {}} +{"execute": "block-stream", "arguments": {"device": "drv0", "job-id": "job0", "speed": 1}} +{"return": {}} +{"execute": "blockdev-del", "arguments": {"node-name": "drv0"}} {"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: block device is in use by block job: stream"}} -{'execute': 'block-job-cancel', - 'arguments': {'device': 'job0'}} +{"execute": "block-job-cancel", "arguments": {"device": "job0"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "job0", "len": 1048576, "offset": 524288, "speed": 1, "type": "stream"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job0"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job0"}} -{'execute': 'blockdev-del', - 'arguments': {'node-name': 'drv0'}} +{"data": {"device": "job0", "len": 1048576, "offset": 524288, "speed": 1, "type": "stream"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"execute": "blockdev-del", "arguments": {"node-name": "drv0"}} {"return": {}} -*** done + From patchwork Fri Jan 19 18:13:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1888624 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=LOUt1N5v; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TGnq34bqKz1yPV for ; Sat, 20 Jan 2024 05:15:43 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rQtNQ-0001lF-5N; Fri, 19 Jan 2024 13:14:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtNF-0001eL-4V for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:50 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtN8-0007cQ-8a for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705688021; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4UYDaJQ8XPZ20R3wAuLxYn2uSRx9kMQAycDdooMBghM=; b=LOUt1N5vK5YGoKwPGjQLHb/5k86g7/mRGsw2mzPu/SN06VO07UjK7Qp/WBanpsjAHb0LJF lw21hsh9qBusIDQ8B3dPTOqxHOTmYIILJneQra9W31D1QiPr58JNxlKV9x6aPs30mQkiKS U9FrJ+BOzwkPrJmBc/tDtgo3kM47vzA= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-517-d6_VpyY_OlKC5t3WW45zrQ-1; Fri, 19 Jan 2024 13:13:39 -0500 X-MC-Unique: d6_VpyY_OlKC5t3WW45zrQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C956D3C2867D; Fri, 19 Jan 2024 18:13:38 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.86]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1F5F9494; Fri, 19 Jan 2024 18:13:38 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 07/14] monitor: only run coroutine commands in qemu_aio_context Date: Fri, 19 Jan 2024 19:13:20 +0100 Message-ID: <20240119181327.236745-8-kwolf@redhat.com> In-Reply-To: <20240119181327.236745-1-kwolf@redhat.com> References: <20240119181327.236745-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.519, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Stefan Hajnoczi monitor_qmp_dispatcher_co() runs in the iohandler AioContext that is not polled during nested event loops. The coroutine currently reschedules itself in the main loop's qemu_aio_context AioContext, which is polled during nested event loops. One known problem is that QMP device-add calls drain_call_rcu(), which temporarily drops the BQL, leading to all sorts of havoc like other vCPU threads re-entering device emulation code while another vCPU thread is waiting in device emulation code with aio_poll(). Paolo Bonzini suggested running non-coroutine QMP handlers in the iohandler AioContext. This avoids trouble with nested event loops. His original idea was to move coroutine rescheduling to monitor_qmp_dispatch(), but I resorted to moving it to qmp_dispatch() because we don't know if the QMP handler needs to run in coroutine context in monitor_qmp_dispatch(). monitor_qmp_dispatch() would have been nicer since it's associated with the monitor implementation and not as general as qmp_dispatch(), which is also used by qemu-ga. A number of qemu-iotests need updated .out files because the order of QMP events vs QMP responses has changed. Solves Issue #1933. Cc: qemu-stable@nongnu.org Fixes: 7bed89958bfbf40df9ca681cefbdca63abdde39d ("device_core: use drain_call_rcu in in qmp_device_add") Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2215192 Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2214985 Buglink: https://issues.redhat.com/browse/RHEL-17369 Signed-off-by: Stefan Hajnoczi Message-ID: <20240118144823.1497953-4-stefanha@redhat.com> Reviewed-by: Kevin Wolf Tested-by: Fiona Ebner Signed-off-by: Kevin Wolf --- monitor/qmp.c | 17 ---- qapi/qmp-dispatch.c | 24 +++++- tests/qemu-iotests/060.out | 4 +- tests/qemu-iotests/071.out | 4 +- tests/qemu-iotests/081.out | 16 ++-- tests/qemu-iotests/087.out | 12 +-- tests/qemu-iotests/108.out | 2 +- tests/qemu-iotests/109 | 4 +- tests/qemu-iotests/109.out | 78 ++++++++----------- tests/qemu-iotests/117.out | 2 +- tests/qemu-iotests/120.out | 2 +- tests/qemu-iotests/127.out | 2 +- tests/qemu-iotests/140.out | 2 +- tests/qemu-iotests/143.out | 2 +- tests/qemu-iotests/156.out | 2 +- tests/qemu-iotests/176.out | 16 ++-- tests/qemu-iotests/182.out | 2 +- tests/qemu-iotests/183.out | 4 +- tests/qemu-iotests/184.out | 32 ++++---- tests/qemu-iotests/185 | 6 +- tests/qemu-iotests/185.out | 45 +++++++++-- tests/qemu-iotests/191.out | 16 ++-- tests/qemu-iotests/195.out | 16 ++-- tests/qemu-iotests/223.out | 12 +-- tests/qemu-iotests/227.out | 32 ++++---- tests/qemu-iotests/247.out | 2 +- tests/qemu-iotests/273.out | 8 +- tests/qemu-iotests/308 | 4 +- tests/qemu-iotests/308.out | 4 +- tests/qemu-iotests/tests/file-io-error | 5 +- tests/qemu-iotests/tests/iothreads-resize.out | 2 +- tests/qemu-iotests/tests/qsd-jobs.out | 4 +- 32 files changed, 205 insertions(+), 178 deletions(-) diff --git a/monitor/qmp.c b/monitor/qmp.c index 6eee450fe4..a239945e8d 100644 --- a/monitor/qmp.c +++ b/monitor/qmp.c @@ -321,14 +321,6 @@ void coroutine_fn monitor_qmp_dispatcher_co(void *data) qemu_coroutine_yield(); } - /* - * Move the coroutine from iohandler_ctx to qemu_aio_context for - * executing the command handler so that it can make progress if it - * involves an AIO_WAIT_WHILE(). - */ - aio_co_schedule(qemu_get_aio_context(), qmp_dispatcher_co); - qemu_coroutine_yield(); - /* Process request */ if (req_obj->req) { if (trace_event_get_state(TRACE_MONITOR_QMP_CMD_IN_BAND)) { @@ -355,15 +347,6 @@ void coroutine_fn monitor_qmp_dispatcher_co(void *data) } qmp_request_free(req_obj); - - /* - * Yield and reschedule so the main loop stays responsive. - * - * Move back to iohandler_ctx so that nested event loops for - * qemu_aio_context don't start new monitor commands. - */ - aio_co_schedule(iohandler_get_aio_context(), qmp_dispatcher_co); - qemu_coroutine_yield(); } qatomic_set(&qmp_dispatcher_co, NULL); } diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c index 555528b6bb..176b549473 100644 --- a/qapi/qmp-dispatch.c +++ b/qapi/qmp-dispatch.c @@ -206,9 +206,31 @@ QDict *coroutine_mixed_fn qmp_dispatch(const QmpCommandList *cmds, QObject *requ assert(!(oob && qemu_in_coroutine())); assert(monitor_cur() == NULL); if (!!(cmd->options & QCO_COROUTINE) == qemu_in_coroutine()) { + if (qemu_in_coroutine()) { + /* + * Move the coroutine from iohandler_ctx to qemu_aio_context for + * executing the command handler so that it can make progress if it + * involves an AIO_WAIT_WHILE(). + */ + aio_co_schedule(qemu_get_aio_context(), qemu_coroutine_self()); + qemu_coroutine_yield(); + } + monitor_set_cur(qemu_coroutine_self(), cur_mon); cmd->fn(args, &ret, &err); monitor_set_cur(qemu_coroutine_self(), NULL); + + if (qemu_in_coroutine()) { + /* + * Yield and reschedule so the main loop stays responsive. + * + * Move back to iohandler_ctx so that nested event loops for + * qemu_aio_context don't start new monitor commands. + */ + aio_co_schedule(iohandler_get_aio_context(), + qemu_coroutine_self()); + qemu_coroutine_yield(); + } } else { /* * Actual context doesn't match the one the command needs. @@ -232,7 +254,7 @@ QDict *coroutine_mixed_fn qmp_dispatch(const QmpCommandList *cmds, QObject *requ .errp = &err, .co = qemu_coroutine_self(), }; - aio_bh_schedule_oneshot(qemu_get_aio_context(), do_qmp_dispatch_bh, + aio_bh_schedule_oneshot(iohandler_get_aio_context(), do_qmp_dispatch_bh, &data); qemu_coroutine_yield(); } diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out index 329977d9b9..a37bf446e9 100644 --- a/tests/qemu-iotests/060.out +++ b/tests/qemu-iotests/060.out @@ -421,8 +421,8 @@ QMP_VERSION {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_IMAGE_CORRUPTED", "data": {"device": "none0", "msg": "Preventing invalid write on metadata (overlaps with refcount table)", "offset": 65536, "node-name": "drive", "fatal": true, "size": 65536}} write failed: Input/output error {"return": ""} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Testing incoming inactive corrupted image === @@ -432,8 +432,8 @@ QMP_VERSION qcow2: Image is corrupt: L2 table offset 0x2a2a2a00 unaligned (L1 index: 0); further non-fatal corruption events will be suppressed {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_IMAGE_CORRUPTED", "data": {"device": "", "msg": "L2 table offset 0x2a2a2a00 unaligned (L1 index: 0)", "node-name": "drive", "fatal": false}} {"return": ""} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} corrupt: false *** done diff --git a/tests/qemu-iotests/071.out b/tests/qemu-iotests/071.out index bca0c02f5c..a2923b05c2 100644 --- a/tests/qemu-iotests/071.out +++ b/tests/qemu-iotests/071.out @@ -45,8 +45,8 @@ QMP_VERSION {"return": {}} read failed: Input/output error {"return": ""} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Testing blkverify on existing block device === @@ -84,9 +84,9 @@ wrote 512/512 bytes at offset 0 {"return": ""} read failed: Input/output error {"return": ""} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} QEMU_PROG: Failed to flush the L2 table cache: Input/output error QEMU_PROG: Failed to flush the refcount block cache: Input/output error +{"return": {}} *** done diff --git a/tests/qemu-iotests/081.out b/tests/qemu-iotests/081.out index 615c083549..aba85ea564 100644 --- a/tests/qemu-iotests/081.out +++ b/tests/qemu-iotests/081.out @@ -35,8 +35,8 @@ QMP_VERSION read 10485760/10485760 bytes at offset 0 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {"return": ""} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} == using quorum rewrite corrupted mode == @@ -67,8 +67,8 @@ QMP_VERSION read 10485760/10485760 bytes at offset 0 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {"return": ""} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} -- checking that the image has been corrected -- read 10485760/10485760 bytes at offset 0 @@ -106,8 +106,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} Testing: QMP_VERSION @@ -115,8 +115,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"error": {"class": "GenericError", "desc": "Cannot add a child to a quorum in blkverify mode"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} == dynamically removing a child from a quorum == @@ -125,31 +125,31 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} Testing: QMP_VERSION {"return": {}} {"return": {}} {"error": {"class": "GenericError", "desc": "The number of children cannot be lower than the vote threshold 2"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} Testing: QMP_VERSION {"return": {}} {"error": {"class": "GenericError", "desc": "blkverify=on can only be set if there are exactly two files and vote-threshold is 2"}} {"error": {"class": "GenericError", "desc": "Cannot find device='drive0-quorum' nor node-name='drive0-quorum'"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} Testing: QMP_VERSION {"return": {}} {"return": {}} {"error": {"class": "GenericError", "desc": "The number of children cannot be lower than the vote threshold 2"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} *** done diff --git a/tests/qemu-iotests/087.out b/tests/qemu-iotests/087.out index e1c23a6983..97b6d8036d 100644 --- a/tests/qemu-iotests/087.out +++ b/tests/qemu-iotests/087.out @@ -7,8 +7,8 @@ Testing: QMP_VERSION {"return": {}} {"error": {"class": "GenericError", "desc": "'node-name' must be specified for the root node"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Duplicate ID === @@ -18,8 +18,8 @@ QMP_VERSION {"return": {}} {"error": {"class": "GenericError", "desc": "node-name=disk is conflicting with a device id"}} {"error": {"class": "GenericError", "desc": "Duplicate nodes with node-name='test-node'"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === aio=native without O_DIRECT === @@ -28,8 +28,8 @@ Testing: QMP_VERSION {"return": {}} {"error": {"class": "GenericError", "desc": "aio=native was specified, but it requires cache.direct=on, which was not specified."}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Encrypted image QCow === @@ -40,8 +40,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"error": {"class": "GenericError", "desc": "Use of AES-CBC encrypted IMGFMT images is no longer supported in system emulators"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Encrypted image LUKS === @@ -52,8 +52,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Missing driver === @@ -63,7 +63,7 @@ Testing: -S QMP_VERSION {"return": {}} {"error": {"class": "GenericError", "desc": "Parameter 'driver' is missing"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} *** done diff --git a/tests/qemu-iotests/108.out b/tests/qemu-iotests/108.out index b5401d788d..b9c876b394 100644 --- a/tests/qemu-iotests/108.out +++ b/tests/qemu-iotests/108.out @@ -173,8 +173,8 @@ OK: Reftable is where we expect it {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "create"}} {"return": {}} { "execute": "quit" } -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} wrote 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) diff --git a/tests/qemu-iotests/109 b/tests/qemu-iotests/109 index e207a555f3..0fb580f9a5 100755 --- a/tests/qemu-iotests/109 +++ b/tests/qemu-iotests/109 @@ -57,13 +57,13 @@ run_qemu() _launch_qemu -drive file="${source_img}",format=raw,cache=${CACHEMODE},aio=${AIOMODE},id=src _send_qemu_cmd $QEMU_HANDLE "{ 'execute': 'qmp_capabilities' }" "return" - _send_qemu_cmd $QEMU_HANDLE \ + capture_events="$qmp_event" _send_qemu_cmd $QEMU_HANDLE \ "{'execute':'drive-mirror', 'arguments':{ 'device': 'src', 'target': '$raw_img', $qmp_format 'mode': 'existing', 'sync': 'full'}}" \ "return" - _send_qemu_cmd $QEMU_HANDLE '' "$qmp_event" + capture_events="$qmp_event JOB_STATUS_CHANGE" _wait_event $QEMU_HANDLE "$qmp_event" if test "$qmp_event" = BLOCK_JOB_ERROR; then _send_qemu_cmd $QEMU_HANDLE '' '"status": "null"' fi diff --git a/tests/qemu-iotests/109.out b/tests/qemu-iotests/109.out index 965c9a6a0a..3ae8552ff7 100644 --- a/tests/qemu-iotests/109.out +++ b/tests/qemu-iotests/109.out @@ -7,7 +7,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=SIZE { 'execute': 'qmp_capabilities' } {"return": {}} {'execute':'drive-mirror', 'arguments':{ - 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', + 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', 'mode': 'existing', 'sync': 'full'}} WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. @@ -23,8 +23,8 @@ WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed {"execute":"query-block-jobs"} {"return": []} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} read 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) { 'execute': 'qmp_capabilities' } @@ -35,12 +35,10 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 1024, "offset": 1024, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 1024, "offset": 1024, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} @@ -50,6 +48,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "src", "len": 1024, "offset": 1024, "speed": 0, "type": "mirror"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} +{"return": {}} Images are identical. === Writing a qcow2 header into raw === @@ -59,7 +58,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=SIZE { 'execute': 'qmp_capabilities' } {"return": {}} {'execute':'drive-mirror', 'arguments':{ - 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', + 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', 'mode': 'existing', 'sync': 'full'}} WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. @@ -75,8 +74,8 @@ WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed {"execute":"query-block-jobs"} {"return": []} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} read 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) { 'execute': 'qmp_capabilities' } @@ -87,12 +86,10 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 197120, "offset": 197120, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 197120, "offset": 197120, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} @@ -102,6 +99,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "src", "len": 197120, "offset": 197120, "speed": 0, "type": "mirror"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} +{"return": {}} Images are identical. === Writing a qed header into raw === @@ -111,7 +109,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=SIZE { 'execute': 'qmp_capabilities' } {"return": {}} {'execute':'drive-mirror', 'arguments':{ - 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', + 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', 'mode': 'existing', 'sync': 'full'}} WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. @@ -127,8 +125,8 @@ WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed {"execute":"query-block-jobs"} {"return": []} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} read 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) { 'execute': 'qmp_capabilities' } @@ -139,12 +137,10 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 327680, "offset": 327680, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 327680, "offset": 327680, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} @@ -154,6 +150,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "src", "len": 327680, "offset": 327680, "speed": 0, "type": "mirror"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} +{"return": {}} Images are identical. === Writing a vdi header into raw === @@ -163,7 +160,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=SIZE { 'execute': 'qmp_capabilities' } {"return": {}} {'execute':'drive-mirror', 'arguments':{ - 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', + 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', 'mode': 'existing', 'sync': 'full'}} WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. @@ -179,8 +176,8 @@ WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed {"execute":"query-block-jobs"} {"return": []} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} read 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) { 'execute': 'qmp_capabilities' } @@ -191,12 +188,10 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 1024, "offset": 1024, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 1024, "offset": 1024, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} @@ -206,6 +201,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "src", "len": 1024, "offset": 1024, "speed": 0, "type": "mirror"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} +{"return": {}} Images are identical. === Writing a vmdk header into raw === @@ -215,7 +211,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=SIZE { 'execute': 'qmp_capabilities' } {"return": {}} {'execute':'drive-mirror', 'arguments':{ - 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', + 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', 'mode': 'existing', 'sync': 'full'}} WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. @@ -231,8 +227,8 @@ WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed {"execute":"query-block-jobs"} {"return": []} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} read 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) { 'execute': 'qmp_capabilities' } @@ -243,12 +239,10 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 65536, "offset": 65536, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 65536, "offset": 65536, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} @@ -258,6 +252,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "src", "len": 65536, "offset": 65536, "speed": 0, "type": "mirror"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} +{"return": {}} Images are identical. === Writing a vpc header into raw === @@ -267,7 +262,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=SIZE { 'execute': 'qmp_capabilities' } {"return": {}} {'execute':'drive-mirror', 'arguments':{ - 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', + 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', 'mode': 'existing', 'sync': 'full'}} WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. @@ -283,8 +278,8 @@ WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed {"execute":"query-block-jobs"} {"return": []} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} read 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) { 'execute': 'qmp_capabilities' } @@ -295,12 +290,10 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 2560, "offset": 2560, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 2560, "offset": 2560, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} @@ -310,6 +303,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "src", "len": 2560, "offset": 2560, "speed": 0, "type": "mirror"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} +{"return": {}} Images are identical. === Copying sample image empty.bochs into raw === @@ -318,7 +312,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=SIZE { 'execute': 'qmp_capabilities' } {"return": {}} {'execute':'drive-mirror', 'arguments':{ - 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', + 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', 'mode': 'existing', 'sync': 'full'}} WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. @@ -334,8 +328,8 @@ WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed {"execute":"query-block-jobs"} {"return": []} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} read 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) { 'execute': 'qmp_capabilities' } @@ -346,12 +340,10 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 2560, "offset": 2560, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 2560, "offset": 2560, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} @@ -361,6 +353,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "src", "len": 2560, "offset": 2560, "speed": 0, "type": "mirror"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} +{"return": {}} Images are identical. === Copying sample image iotest-dirtylog-10G-4M.vhdx into raw === @@ -369,7 +362,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=SIZE { 'execute': 'qmp_capabilities' } {"return": {}} {'execute':'drive-mirror', 'arguments':{ - 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', + 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', 'mode': 'existing', 'sync': 'full'}} WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. @@ -385,8 +378,8 @@ WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed {"execute":"query-block-jobs"} {"return": []} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} read 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) { 'execute': 'qmp_capabilities' } @@ -397,12 +390,10 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 31457280, "offset": 31457280, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 31457280, "offset": 31457280, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} @@ -412,6 +403,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "src", "len": 31457280, "offset": 31457280, "speed": 0, "type": "mirror"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} +{"return": {}} Images are identical. === Copying sample image parallels-v1 into raw === @@ -420,7 +412,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=SIZE { 'execute': 'qmp_capabilities' } {"return": {}} {'execute':'drive-mirror', 'arguments':{ - 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', + 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', 'mode': 'existing', 'sync': 'full'}} WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. @@ -436,8 +428,8 @@ WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed {"execute":"query-block-jobs"} {"return": []} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} read 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) { 'execute': 'qmp_capabilities' } @@ -448,12 +440,10 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 327680, "offset": 327680, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 327680, "offset": 327680, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} @@ -463,6 +453,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "src", "len": 327680, "offset": 327680, "speed": 0, "type": "mirror"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} +{"return": {}} Images are identical. === Copying sample image simple-pattern.cloop into raw === @@ -471,7 +462,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=SIZE { 'execute': 'qmp_capabilities' } {"return": {}} {'execute':'drive-mirror', 'arguments':{ - 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', + 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', 'mode': 'existing', 'sync': 'full'}} WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. @@ -487,8 +478,8 @@ WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed {"execute":"query-block-jobs"} {"return": []} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} read 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) { 'execute': 'qmp_capabilities' } @@ -499,12 +490,10 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 2048, "offset": 2048, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 2048, "offset": 2048, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} @@ -514,6 +503,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "src", "len": 2048, "offset": 2048, "speed": 0, "type": "mirror"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} +{"return": {}} Images are identical. === Write legitimate MBR into raw === @@ -522,7 +512,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=SIZE { 'execute': 'qmp_capabilities' } {"return": {}} {'execute':'drive-mirror', 'arguments':{ - 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', + 'device': 'src', 'target': 'TEST_DIR/t.IMGFMT', 'mode': 'existing', 'sync': 'full'}} WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. @@ -530,12 +520,10 @@ WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 512, "offset": 512, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 512, "offset": 512, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} @@ -545,6 +533,7 @@ WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "src", "len": 512, "offset": 512, "speed": 0, "type": "mirror"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} +{"return": {}} Images are identical. { 'execute': 'qmp_capabilities' } {"return": {}} @@ -554,12 +543,10 @@ Images are identical. {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 512, "offset": 512, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 512, "offset": 512, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} @@ -569,5 +556,6 @@ Images are identical. {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "src", "len": 512, "offset": 512, "speed": 0, "type": "mirror"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} +{"return": {}} Images are identical. *** done diff --git a/tests/qemu-iotests/117.out b/tests/qemu-iotests/117.out index 735ffd25c6..1cea9e0217 100644 --- a/tests/qemu-iotests/117.out +++ b/tests/qemu-iotests/117.out @@ -18,8 +18,8 @@ wrote 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {"return": ""} { 'execute': 'quit' } -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} No errors were found on the image. read 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) diff --git a/tests/qemu-iotests/120.out b/tests/qemu-iotests/120.out index 0744c1f136..35d84a5bc5 100644 --- a/tests/qemu-iotests/120.out +++ b/tests/qemu-iotests/120.out @@ -5,8 +5,8 @@ QMP_VERSION wrote 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {"return": ""} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} read 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 65536/65536 bytes at offset 0 diff --git a/tests/qemu-iotests/127.out b/tests/qemu-iotests/127.out index 1685c4850a..dd8c4a8aa9 100644 --- a/tests/qemu-iotests/127.out +++ b/tests/qemu-iotests/127.out @@ -28,6 +28,6 @@ wrote 42/42 bytes at offset 0 { 'execute': 'quit' } {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "mirror"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "mirror"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} *** done diff --git a/tests/qemu-iotests/140.out b/tests/qemu-iotests/140.out index 312f76d5da..32866440ae 100644 --- a/tests/qemu-iotests/140.out +++ b/tests/qemu-iotests/140.out @@ -19,6 +19,6 @@ read 65536/65536 bytes at offset 0 qemu-io: can't open device nbd+unix:///drv?socket=SOCK_DIR/nbd: Requested export not available server reported: export 'drv' not present { 'execute': 'quit' } -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} *** done diff --git a/tests/qemu-iotests/143.out b/tests/qemu-iotests/143.out index 9ec5888e0e..d6afa32abc 100644 --- a/tests/qemu-iotests/143.out +++ b/tests/qemu-iotests/143.out @@ -10,6 +10,6 @@ server reported: export 'no_such_export' not present qemu-io: can't open device nbd+unix:///aa--aa1?socket=SOCK_DIR/nbd: Requested export not available server reported: export 'aa--aa...' not present { 'execute': 'quit' } -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} *** done diff --git a/tests/qemu-iotests/156.out b/tests/qemu-iotests/156.out index 4a22f0c41a..07e5e83f5d 100644 --- a/tests/qemu-iotests/156.out +++ b/tests/qemu-iotests/156.out @@ -72,8 +72,8 @@ read 65536/65536 bytes at offset 196608 {"return": ""} { 'execute': 'quit' } -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} read 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) diff --git a/tests/qemu-iotests/176.out b/tests/qemu-iotests/176.out index 9d09b60452..45e9153ef3 100644 --- a/tests/qemu-iotests/176.out +++ b/tests/qemu-iotests/176.out @@ -169,8 +169,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} wrote 196608/196608 bytes at offset 2147287040 192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 131072/131072 bytes at offset 2147352576 @@ -206,8 +206,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {"sha256": HASH}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Test pass bitmap.1 === @@ -218,8 +218,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} wrote 196608/196608 bytes at offset 2147287040 192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 131072/131072 bytes at offset 2147352576 @@ -256,8 +256,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {"sha256": HASH}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Test pass bitmap.2 === @@ -268,8 +268,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} wrote 196608/196608 bytes at offset 2147287040 192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 131072/131072 bytes at offset 2147352576 @@ -306,8 +306,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {"sha256": HASH}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Test pass bitmap.3 === @@ -318,8 +318,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} wrote 196608/196608 bytes at offset 2147287040 192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 131072/131072 bytes at offset 2147352576 @@ -353,6 +353,6 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {"sha256": HASH}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} *** done diff --git a/tests/qemu-iotests/182.out b/tests/qemu-iotests/182.out index 57f7265458..83fc1a4797 100644 --- a/tests/qemu-iotests/182.out +++ b/tests/qemu-iotests/182.out @@ -53,6 +53,6 @@ Formatting 'TEST_DIR/t.qcow2.overlay', fmt=qcow2 cluster_size=65536 extended_l2= {'execute': 'qmp_capabilities'} {"return": {}} {'execute': 'quit'} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} *** done diff --git a/tests/qemu-iotests/183.out b/tests/qemu-iotests/183.out index fd9c2e52a5..51aa41c888 100644 --- a/tests/qemu-iotests/183.out +++ b/tests/qemu-iotests/183.out @@ -53,11 +53,11 @@ wrote 65536/65536 bytes at offset 1048576 === Shut down and check image === {"execute":"quit"} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"return": {}} {"execute":"quit"} -{"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} No errors were found on the image. No errors were found on the image. wrote 65536/65536 bytes at offset 1048576 diff --git a/tests/qemu-iotests/184.out b/tests/qemu-iotests/184.out index 77e5489d65..e8f631f853 100644 --- a/tests/qemu-iotests/184.out +++ b/tests/qemu-iotests/184.out @@ -89,10 +89,6 @@ Testing: "return": [ ] } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP, @@ -104,6 +100,10 @@ Testing: "reason": "host-qmp-quit" } } +{ + "return": { + } +} == property changes in ThrottleGroup == @@ -169,10 +169,6 @@ Testing: "iops-total-max": 0 } } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP, @@ -184,6 +180,10 @@ Testing: "reason": "host-qmp-quit" } } +{ + "return": { + } +} == object creation/set errors == @@ -211,10 +211,6 @@ Testing: "desc": "bps/iops/max total values and read/write values cannot be used at the same time" } } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP, @@ -226,6 +222,10 @@ Testing: "reason": "host-qmp-quit" } } +{ + "return": { + } +} == don't specify group == @@ -247,10 +247,6 @@ Testing: "desc": "Parameter 'throttle-group' is missing" } } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP, @@ -262,6 +258,10 @@ Testing: "reason": "host-qmp-quit" } } +{ + "return": { + } +} *** done diff --git a/tests/qemu-iotests/185 b/tests/qemu-iotests/185 index 2ae0a85bbf..17489fb91c 100755 --- a/tests/qemu-iotests/185 +++ b/tests/qemu-iotests/185 @@ -344,14 +344,14 @@ wait_for_job_and_quit() { sleep 1 + # List of expected events + capture_events='BLOCK_JOB_CANCELLED JOB_STATUS_CHANGE SHUTDOWN' + _send_qemu_cmd $h \ '{"execute": "quit"}' \ 'return' - # List of expected events - capture_events='BLOCK_JOB_CANCELLED JOB_STATUS_CHANGE SHUTDOWN' _wait_event $h 'SHUTDOWN' - QEMU_EVENTS= # Ignore all JOB_STATUS_CHANGE events that came before SHUTDOWN _wait_event $h 'JOB_STATUS_CHANGE' # standby _wait_event $h 'JOB_STATUS_CHANGE' # ready _wait_event $h 'JOB_STATUS_CHANGE' # standby diff --git a/tests/qemu-iotests/185.out b/tests/qemu-iotests/185.out index 7292c26bae..6af0953c4d 100644 --- a/tests/qemu-iotests/185.out +++ b/tests/qemu-iotests/185.out @@ -40,9 +40,16 @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off comp {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} {"return": {}} { 'execute': 'quit' } -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "disk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "disk", "len": 67108864, "offset": 524288, "speed": 65536, "type": "commit"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "disk"}} +{"return": {}} === Start active commit job and exit qemu === @@ -56,9 +63,16 @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off comp {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} {"return": {}} { 'execute': 'quit' } -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "disk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "disk", "len": 4194304, "offset": 4194304, "speed": 65536, "type": "commit"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "disk"}} +{"return": {}} === Start mirror job and exit qemu === @@ -75,9 +89,16 @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 extended_l2=off {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} {"return": {}} { 'execute': 'quit' } -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "disk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "disk", "len": 4194304, "offset": 4194304, "speed": 65536, "type": "mirror"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "disk"}} +{"return": {}} === Start backup job and exit qemu === @@ -97,9 +118,16 @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 extended_l2=off {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} {"return": {}} { 'execute': 'quit' } -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "disk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "disk", "len": 67108864, "offset": 65536, "speed": 65536, "type": "backup"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "disk"}} +{"return": {}} === Start streaming job and exit qemu === @@ -112,9 +140,16 @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 extended_l2=off {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} {"return": {}} { 'execute': 'quit' } -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "disk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "disk", "len": 67108864, "offset": 524288, "speed": 65536, "type": "stream"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "disk"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "disk"}} +{"return": {}} No errors were found on the image. === Start mirror to throttled QSD and exit qemu === diff --git a/tests/qemu-iotests/191.out b/tests/qemu-iotests/191.out index ea88777374..c3309e4bc6 100644 --- a/tests/qemu-iotests/191.out +++ b/tests/qemu-iotests/191.out @@ -378,10 +378,6 @@ wrote 65536/65536 bytes at offset 1048576 ] } { 'execute': 'quit' } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP, @@ -393,6 +389,10 @@ wrote 65536/65536 bytes at offset 1048576 "reason": "host-qmp-quit" } } +{ + "return": { + } +} image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 64 MiB (67108864 bytes) @@ -796,10 +796,6 @@ wrote 65536/65536 bytes at offset 1048576 ] } { 'execute': 'quit' } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP, @@ -811,6 +807,10 @@ wrote 65536/65536 bytes at offset 1048576 "reason": "host-qmp-quit" } } +{ + "return": { + } +} image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 64 MiB (67108864 bytes) diff --git a/tests/qemu-iotests/195.out b/tests/qemu-iotests/195.out index ec84df5012..91717d302e 100644 --- a/tests/qemu-iotests/195.out +++ b/tests/qemu-iotests/195.out @@ -17,10 +17,6 @@ Testing: -drive if=none,file=TEST_DIR/t.IMGFMT,backing.node-name=mid "return": { } } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP, @@ -32,6 +28,10 @@ Testing: -drive if=none,file=TEST_DIR/t.IMGFMT,backing.node-name=mid "reason": "host-qmp-quit" } } +{ + "return": { + } +} image: TEST_DIR/t.IMGFMT.mid file format: IMGFMT @@ -55,10 +55,6 @@ Testing: -drive if=none,file=TEST_DIR/t.IMGFMT,node-name=top "return": { } } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP, @@ -70,6 +66,10 @@ Testing: -drive if=none,file=TEST_DIR/t.IMGFMT,node-name=top "reason": "host-qmp-quit" } } +{ + "return": { + } +} image: TEST_DIR/t.IMGFMT file format: IMGFMT diff --git a/tests/qemu-iotests/223.out b/tests/qemu-iotests/223.out index e5e7f42caa..5f5b42e2dc 100644 --- a/tests/qemu-iotests/223.out +++ b/tests/qemu-iotests/223.out @@ -11,8 +11,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Write part of the file under active bitmap === @@ -145,14 +145,14 @@ read 2097152/2097152 bytes at offset 2097152 {"execute":"nbd-server-remove", "arguments":{"name":"n"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "n"}} {"return": {}} {"execute":"nbd-server-remove", "arguments":{"name":"n2"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "n"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "n2"}} {"return": {}} {"execute":"nbd-server-remove", "arguments":{"name":"n2"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "n2"}} {"error": {"class": "GenericError", "desc": "Export 'n2' is not found"}} {"execute":"nbd-server-stop"} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "n3"}} @@ -267,14 +267,14 @@ read 2097152/2097152 bytes at offset 2097152 {"execute":"nbd-server-remove", "arguments":{"name":"n"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "n"}} {"return": {}} {"execute":"nbd-server-remove", "arguments":{"name":"n2"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "n"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "n2"}} {"return": {}} {"execute":"nbd-server-remove", "arguments":{"name":"n2"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "n2"}} {"error": {"class": "GenericError", "desc": "Export 'n2' is not found"}} {"execute":"nbd-server-stop"} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "n3"}} @@ -282,8 +282,8 @@ read 2097152/2097152 bytes at offset 2097152 {"execute":"nbd-server-stop"} {"error": {"class": "GenericError", "desc": "NBD server not running"}} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Use qemu-nbd as server === diff --git a/tests/qemu-iotests/227.out b/tests/qemu-iotests/227.out index a947b1a87d..d6a1d4ecb6 100644 --- a/tests/qemu-iotests/227.out +++ b/tests/qemu-iotests/227.out @@ -54,10 +54,6 @@ Testing: -drive driver=null-co,read-zeroes=on,if=virtio } ] } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP, @@ -69,6 +65,10 @@ Testing: -drive driver=null-co,read-zeroes=on,if=virtio "reason": "host-qmp-quit" } } +{ + "return": { + } +} === blockstats with -drive if=none === @@ -124,10 +124,6 @@ Testing: -drive driver=null-co,if=none } ] } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP, @@ -139,6 +135,10 @@ Testing: -drive driver=null-co,if=none "reason": "host-qmp-quit" } } +{ + "return": { + } +} === blockstats with -blockdev === @@ -155,10 +155,6 @@ Testing: -blockdev driver=null-co,node-name=null "return": [ ] } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP, @@ -170,6 +166,10 @@ Testing: -blockdev driver=null-co,node-name=null "reason": "host-qmp-quit" } } +{ + "return": { + } +} === blockstats with -blockdev and -device === @@ -226,10 +226,6 @@ Testing: -blockdev driver=null-co,read-zeroes=on,node-name=null -device virtio-b } ] } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP, @@ -241,5 +237,9 @@ Testing: -blockdev driver=null-co,read-zeroes=on,node-name=null -device virtio-b "reason": "host-qmp-quit" } } +{ + "return": { + } +} *** done diff --git a/tests/qemu-iotests/247.out b/tests/qemu-iotests/247.out index e909e83994..7d252e7fe4 100644 --- a/tests/qemu-iotests/247.out +++ b/tests/qemu-iotests/247.out @@ -17,6 +17,6 @@ QMP_VERSION {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "job0", "len": 134217728, "offset": 134217728, "speed": 0, "type": "commit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job0"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job0"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} *** done diff --git a/tests/qemu-iotests/273.out b/tests/qemu-iotests/273.out index 6a74a8138b..71843f02de 100644 --- a/tests/qemu-iotests/273.out +++ b/tests/qemu-iotests/273.out @@ -282,10 +282,6 @@ Testing: -blockdev file,node-name=base,filename=TEST_DIR/t.IMGFMT.base -blockdev ] } } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP, @@ -297,5 +293,9 @@ Testing: -blockdev file,node-name=base,filename=TEST_DIR/t.IMGFMT.base -blockdev "reason": "host-qmp-quit" } } +{ + "return": { + } +} *** done diff --git a/tests/qemu-iotests/308 b/tests/qemu-iotests/308 index de12b2b1b9..ea81dc496a 100755 --- a/tests/qemu-iotests/308 +++ b/tests/qemu-iotests/308 @@ -77,6 +77,7 @@ fuse_export_add() # $1: Export ID fuse_export_del() { + capture_events="BLOCK_EXPORT_DELETED" \ _send_qemu_cmd $QEMU_HANDLE \ "{'execute': 'block-export-del', 'arguments': { @@ -84,8 +85,7 @@ fuse_export_del() } }" \ 'return' - _send_qemu_cmd $QEMU_HANDLE \ - '' \ + _wait_event $QEMU_HANDLE \ 'BLOCK_EXPORT_DELETED' } diff --git a/tests/qemu-iotests/308.out b/tests/qemu-iotests/308.out index d5767133b1..e5e233691d 100644 --- a/tests/qemu-iotests/308.out +++ b/tests/qemu-iotests/308.out @@ -165,9 +165,9 @@ OK: Post-truncate image size is as expected === Tear down === {'execute': 'quit'} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "export-mp"}} +{"return": {}} === Compare copy with original === Images are identical. @@ -201,9 +201,9 @@ wrote 67108864/67108864 bytes at offset 0 read 67108864/67108864 bytes at offset 0 64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {'execute': 'quit'} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "export"}} +{"return": {}} read 67108864/67108864 bytes at offset 0 64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) *** done diff --git a/tests/qemu-iotests/tests/file-io-error b/tests/qemu-iotests/tests/file-io-error index 88ee5f670c..fb8db73b31 100755 --- a/tests/qemu-iotests/tests/file-io-error +++ b/tests/qemu-iotests/tests/file-io-error @@ -99,13 +99,12 @@ echo $QEMU_IO -f file -c 'write 0 64M' "$TEST_DIR/fuse-export" | _filter_qemu_io echo -_send_qemu_cmd $QEMU_HANDLE \ +capture_events=BLOCK_EXPORT_DELETED _send_qemu_cmd $QEMU_HANDLE \ "{'execute': 'block-export-del', 'arguments': {'id': 'exp0'}}" \ 'return' -_send_qemu_cmd $QEMU_HANDLE \ - '' \ +_wait_event $QEMU_HANDLE \ 'BLOCK_EXPORT_DELETED' _send_qemu_cmd $QEMU_HANDLE \ diff --git a/tests/qemu-iotests/tests/iothreads-resize.out b/tests/qemu-iotests/tests/iothreads-resize.out index 2ca5a9d964..2967ac8f0d 100644 --- a/tests/qemu-iotests/tests/iothreads-resize.out +++ b/tests/qemu-iotests/tests/iothreads-resize.out @@ -3,8 +3,8 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 QMP_VERSION {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 128 MiB (134217728 bytes) diff --git a/tests/qemu-iotests/tests/qsd-jobs.out b/tests/qemu-iotests/tests/qsd-jobs.out index c1bc9b8356..aa6b6d1aef 100644 --- a/tests/qemu-iotests/tests/qsd-jobs.out +++ b/tests/qemu-iotests/tests/qsd-jobs.out @@ -7,8 +7,8 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/ QMP_VERSION {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}} +{"return": {}} === Streaming can't get permission on base node === @@ -17,6 +17,6 @@ QMP_VERSION {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job0"}} {"error": {"class": "GenericError", "desc": "Permission conflict on node 'fmt_base': permissions 'write' are both required by an unnamed block device (uses node 'fmt_base' as 'root' child) and unshared by stream job 'job0' (uses node 'fmt_base' as 'intermediate node' child)."}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "export1"}} +{"return": {}} *** done From patchwork Fri Jan 19 18:13:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1888623 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=W3Y4R/uG; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TGnq34xDvz23fB for ; Sat, 20 Jan 2024 05:15:43 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rQtNL-0001hx-5A; Fri, 19 Jan 2024 13:13:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtNF-0001eJ-3U for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:50 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtN9-0007dd-9Z for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705688022; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IwwHTMDaIvfbN2XEyE+dY4HI0Y8yIrmO6PzexM9ZEBE=; b=W3Y4R/uGUgYTK4/PhogO7QIJX23gYwe9/iXFFk7+WRREFfz7FpL6y+3LtsaBTksQMJnQcZ 66ayrtvHjQvUbWvSyQ4JuBNnUsVAZaR/pxeYxbfClz0150sUobP0XSe7nEDRbJcFiwQagI Q0Ydo9jQBpCoR7VdG0f+UGSReI5sEzg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-119-490Nlc-oMgCJ2YxqfcF5xw-1; Fri, 19 Jan 2024 13:13:40 -0500 X-MC-Unique: 490Nlc-oMgCJ2YxqfcF5xw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E208A867942; Fri, 19 Jan 2024 18:13:39 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.86]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0AE422877; Fri, 19 Jan 2024 18:13:38 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 08/14] virtio-blk: move dataplane code into virtio-blk.c Date: Fri, 19 Jan 2024 19:13:21 +0100 Message-ID: <20240119181327.236745-9-kwolf@redhat.com> In-Reply-To: <20240119181327.236745-1-kwolf@redhat.com> References: <20240119181327.236745-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.519, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Stefan Hajnoczi The dataplane code used to be significantly different from the non-dataplane code and therefore had a separate source file. Over time the difference has gotten smaller because the I/O code paths were unified. Nowadays the distinction between the VirtIOBlock and VirtIOBlockDataPlane structs is more of an inconvenience that hinders code simplification. Move hw/block/dataplane/virtio-blk.c into hw/block/virtio-blk.c, merging VirtIOBlockDataPlane's fields into VirtIOBlock. hw/block/virtio-blk.c used VirtIOBlock->dataplane to check if virtio_blk_data_plane_create() was successful. This is not necessary because ->dataplane_started and ->dataplane_disabled can be used instead. This patch makes those changes in order to drop VirtIOBlock->dataplane. Signed-off-by: Stefan Hajnoczi Message-ID: <20240119135748.270944-2-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- hw/block/dataplane/trace.h | 1 - hw/block/dataplane/virtio-blk.h | 34 --- include/hw/virtio/virtio-blk.h | 12 +- hw/block/dataplane/virtio-blk.c | 404 -------------------------------- hw/block/virtio-blk.c | 362 ++++++++++++++++++++++++++-- hw/block/dataplane/meson.build | 1 - hw/block/dataplane/trace-events | 5 - meson.build | 1 - 8 files changed, 357 insertions(+), 463 deletions(-) delete mode 100644 hw/block/dataplane/trace.h delete mode 100644 hw/block/dataplane/virtio-blk.h delete mode 100644 hw/block/dataplane/virtio-blk.c delete mode 100644 hw/block/dataplane/trace-events diff --git a/hw/block/dataplane/trace.h b/hw/block/dataplane/trace.h deleted file mode 100644 index 240cc59834..0000000000 --- a/hw/block/dataplane/trace.h +++ /dev/null @@ -1 +0,0 @@ -#include "trace/trace-hw_block_dataplane.h" diff --git a/hw/block/dataplane/virtio-blk.h b/hw/block/dataplane/virtio-blk.h deleted file mode 100644 index 1a806fe447..0000000000 --- a/hw/block/dataplane/virtio-blk.h +++ /dev/null @@ -1,34 +0,0 @@ -/* - * Dedicated thread for virtio-blk I/O processing - * - * Copyright 2012 IBM, Corp. - * Copyright 2012 Red Hat, Inc. and/or its affiliates - * - * Authors: - * Stefan Hajnoczi - * - * This work is licensed under the terms of the GNU GPL, version 2 or later. - * See the COPYING file in the top-level directory. - * - */ - -#ifndef HW_DATAPLANE_VIRTIO_BLK_H -#define HW_DATAPLANE_VIRTIO_BLK_H - -#include "hw/virtio/virtio.h" - -typedef struct VirtIOBlockDataPlane VirtIOBlockDataPlane; - -bool virtio_blk_data_plane_create(VirtIODevice *vdev, VirtIOBlkConf *conf, - VirtIOBlockDataPlane **dataplane, - Error **errp); -void virtio_blk_data_plane_destroy(VirtIOBlockDataPlane *s); -void virtio_blk_data_plane_notify(VirtIOBlockDataPlane *s, VirtQueue *vq); - -int virtio_blk_data_plane_start(VirtIODevice *vdev); -void virtio_blk_data_plane_stop(VirtIODevice *vdev); - -void virtio_blk_data_plane_detach(VirtIOBlockDataPlane *s); -void virtio_blk_data_plane_attach(VirtIOBlockDataPlane *s); - -#endif /* HW_DATAPLANE_VIRTIO_BLK_H */ diff --git a/include/hw/virtio/virtio-blk.h b/include/hw/virtio/virtio-blk.h index 5e4091e4da..fecffdc303 100644 --- a/include/hw/virtio/virtio-blk.h +++ b/include/hw/virtio/virtio-blk.h @@ -50,8 +50,6 @@ struct VirtIOBlkConf bool x_enable_wce_if_config_wce; }; -struct VirtIOBlockDataPlane; - struct VirtIOBlockReq; struct VirtIOBlock { VirtIODevice parent_obj; @@ -64,7 +62,15 @@ struct VirtIOBlock { VMChangeStateEntry *change; bool dataplane_disabled; bool dataplane_started; - struct VirtIOBlockDataPlane *dataplane; + bool dataplane_starting; + bool dataplane_stopping; + + /* + * The AioContext for each virtqueue. The BlockDriverState will use the + * first element as its AioContext. + */ + AioContext **vq_aio_context; + uint64_t host_features; size_t config_size; BlockRAMRegistrar blk_ram_registrar; diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c deleted file mode 100644 index ba22732497..0000000000 --- a/hw/block/dataplane/virtio-blk.c +++ /dev/null @@ -1,404 +0,0 @@ -/* - * Dedicated thread for virtio-blk I/O processing - * - * Copyright 2012 IBM, Corp. - * Copyright 2012 Red Hat, Inc. and/or its affiliates - * - * Authors: - * Stefan Hajnoczi - * - * This work is licensed under the terms of the GNU GPL, version 2 or later. - * See the COPYING file in the top-level directory. - * - */ - -#include "qemu/osdep.h" -#include "qapi/error.h" -#include "trace.h" -#include "qemu/iov.h" -#include "qemu/main-loop.h" -#include "qemu/thread.h" -#include "qemu/error-report.h" -#include "hw/virtio/virtio-blk.h" -#include "virtio-blk.h" -#include "block/aio.h" -#include "hw/virtio/virtio-bus.h" -#include "qom/object_interfaces.h" - -struct VirtIOBlockDataPlane { - bool starting; - bool stopping; - - VirtIOBlkConf *conf; - VirtIODevice *vdev; - - /* - * The AioContext for each virtqueue. The BlockDriverState will use the - * first element as its AioContext. - */ - AioContext **vq_aio_context; -}; - -/* Raise an interrupt to signal guest, if necessary */ -void virtio_blk_data_plane_notify(VirtIOBlockDataPlane *s, VirtQueue *vq) -{ - virtio_notify_irqfd(s->vdev, vq); -} - -/* Generate vq:AioContext mappings from a validated iothread-vq-mapping list */ -static void -apply_vq_mapping(IOThreadVirtQueueMappingList *iothread_vq_mapping_list, - AioContext **vq_aio_context, uint16_t num_queues) -{ - IOThreadVirtQueueMappingList *node; - size_t num_iothreads = 0; - size_t cur_iothread = 0; - - for (node = iothread_vq_mapping_list; node; node = node->next) { - num_iothreads++; - } - - for (node = iothread_vq_mapping_list; node; node = node->next) { - IOThread *iothread = iothread_by_id(node->value->iothread); - AioContext *ctx = iothread_get_aio_context(iothread); - - /* Released in virtio_blk_data_plane_destroy() */ - object_ref(OBJECT(iothread)); - - if (node->value->vqs) { - uint16List *vq; - - /* Explicit vq:IOThread assignment */ - for (vq = node->value->vqs; vq; vq = vq->next) { - vq_aio_context[vq->value] = ctx; - } - } else { - /* Round-robin vq:IOThread assignment */ - for (unsigned i = cur_iothread; i < num_queues; - i += num_iothreads) { - vq_aio_context[i] = ctx; - } - } - - cur_iothread++; - } -} - -/* Context: BQL held */ -bool virtio_blk_data_plane_create(VirtIODevice *vdev, VirtIOBlkConf *conf, - VirtIOBlockDataPlane **dataplane, - Error **errp) -{ - VirtIOBlockDataPlane *s; - BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); - - *dataplane = NULL; - - if (conf->iothread || conf->iothread_vq_mapping_list) { - if (!k->set_guest_notifiers || !k->ioeventfd_assign) { - error_setg(errp, - "device is incompatible with iothread " - "(transport does not support notifiers)"); - return false; - } - if (!virtio_device_ioeventfd_enabled(vdev)) { - error_setg(errp, "ioeventfd is required for iothread"); - return false; - } - - /* If dataplane is (re-)enabled while the guest is running there could - * be block jobs that can conflict. - */ - if (blk_op_is_blocked(conf->conf.blk, BLOCK_OP_TYPE_DATAPLANE, errp)) { - error_prepend(errp, "cannot start virtio-blk dataplane: "); - return false; - } - } - /* Don't try if transport does not support notifiers. */ - if (!virtio_device_ioeventfd_enabled(vdev)) { - return false; - } - - s = g_new0(VirtIOBlockDataPlane, 1); - s->vdev = vdev; - s->conf = conf; - s->vq_aio_context = g_new(AioContext *, conf->num_queues); - - if (conf->iothread_vq_mapping_list) { - apply_vq_mapping(conf->iothread_vq_mapping_list, s->vq_aio_context, - conf->num_queues); - } else if (conf->iothread) { - AioContext *ctx = iothread_get_aio_context(conf->iothread); - for (unsigned i = 0; i < conf->num_queues; i++) { - s->vq_aio_context[i] = ctx; - } - - /* Released in virtio_blk_data_plane_destroy() */ - object_ref(OBJECT(conf->iothread)); - } else { - AioContext *ctx = qemu_get_aio_context(); - for (unsigned i = 0; i < conf->num_queues; i++) { - s->vq_aio_context[i] = ctx; - } - } - - *dataplane = s; - - return true; -} - -/* Context: BQL held */ -void virtio_blk_data_plane_destroy(VirtIOBlockDataPlane *s) -{ - VirtIOBlock *vblk; - VirtIOBlkConf *conf; - - if (!s) { - return; - } - - vblk = VIRTIO_BLK(s->vdev); - assert(!vblk->dataplane_started); - conf = s->conf; - - if (conf->iothread_vq_mapping_list) { - IOThreadVirtQueueMappingList *node; - - for (node = conf->iothread_vq_mapping_list; node; node = node->next) { - IOThread *iothread = iothread_by_id(node->value->iothread); - object_unref(OBJECT(iothread)); - } - } - - if (conf->iothread) { - object_unref(OBJECT(conf->iothread)); - } - - g_free(s->vq_aio_context); - g_free(s); -} - -/* Context: BQL held */ -int virtio_blk_data_plane_start(VirtIODevice *vdev) -{ - VirtIOBlock *vblk = VIRTIO_BLK(vdev); - VirtIOBlockDataPlane *s = vblk->dataplane; - BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vblk))); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); - unsigned i; - unsigned nvqs = s->conf->num_queues; - Error *local_err = NULL; - int r; - - if (vblk->dataplane_started || s->starting) { - return 0; - } - - s->starting = true; - - /* Set up guest notifier (irq) */ - r = k->set_guest_notifiers(qbus->parent, nvqs, true); - if (r != 0) { - error_report("virtio-blk failed to set guest notifier (%d), " - "ensure -accel kvm is set.", r); - goto fail_guest_notifiers; - } - - /* - * Batch all the host notifiers in a single transaction to avoid - * quadratic time complexity in address_space_update_ioeventfds(). - */ - memory_region_transaction_begin(); - - /* Set up virtqueue notify */ - for (i = 0; i < nvqs; i++) { - r = virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), i, true); - if (r != 0) { - int j = i; - - fprintf(stderr, "virtio-blk failed to set host notifier (%d)\n", r); - while (i--) { - virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), i, false); - } - - /* - * The transaction expects the ioeventfds to be open when it - * commits. Do it now, before the cleanup loop. - */ - memory_region_transaction_commit(); - - while (j--) { - virtio_bus_cleanup_host_notifier(VIRTIO_BUS(qbus), j); - } - goto fail_host_notifiers; - } - } - - memory_region_transaction_commit(); - - trace_virtio_blk_data_plane_start(s); - - r = blk_set_aio_context(s->conf->conf.blk, s->vq_aio_context[0], - &local_err); - if (r < 0) { - error_report_err(local_err); - goto fail_aio_context; - } - - /* - * These fields must be visible to the IOThread when it processes the - * virtqueue, otherwise it will think dataplane has not started yet. - * - * Make sure ->dataplane_started is false when blk_set_aio_context() is - * called above so that draining does not cause the host notifier to be - * detached/attached prematurely. - */ - s->starting = false; - vblk->dataplane_started = true; - smp_wmb(); /* paired with aio_notify_accept() on the read side */ - - /* Get this show started by hooking up our callbacks */ - if (!blk_in_drain(s->conf->conf.blk)) { - for (i = 0; i < nvqs; i++) { - VirtQueue *vq = virtio_get_queue(s->vdev, i); - AioContext *ctx = s->vq_aio_context[i]; - - /* Kick right away to begin processing requests already in vring */ - event_notifier_set(virtio_queue_get_host_notifier(vq)); - - virtio_queue_aio_attach_host_notifier(vq, ctx); - } - } - return 0; - - fail_aio_context: - memory_region_transaction_begin(); - - for (i = 0; i < nvqs; i++) { - virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), i, false); - } - - memory_region_transaction_commit(); - - for (i = 0; i < nvqs; i++) { - virtio_bus_cleanup_host_notifier(VIRTIO_BUS(qbus), i); - } - fail_host_notifiers: - k->set_guest_notifiers(qbus->parent, nvqs, false); - fail_guest_notifiers: - vblk->dataplane_disabled = true; - s->starting = false; - return -ENOSYS; -} - -/* Stop notifications for new requests from guest. - * - * Context: BH in IOThread - */ -static void virtio_blk_data_plane_stop_vq_bh(void *opaque) -{ - VirtQueue *vq = opaque; - EventNotifier *host_notifier = virtio_queue_get_host_notifier(vq); - - virtio_queue_aio_detach_host_notifier(vq, qemu_get_current_aio_context()); - - /* - * Test and clear notifier after disabling event, in case poll callback - * didn't have time to run. - */ - virtio_queue_host_notifier_read(host_notifier); -} - -/* Context: BQL held */ -void virtio_blk_data_plane_stop(VirtIODevice *vdev) -{ - VirtIOBlock *vblk = VIRTIO_BLK(vdev); - VirtIOBlockDataPlane *s = vblk->dataplane; - BusState *qbus = qdev_get_parent_bus(DEVICE(vblk)); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); - unsigned i; - unsigned nvqs = s->conf->num_queues; - - if (!vblk->dataplane_started || s->stopping) { - return; - } - - /* Better luck next time. */ - if (vblk->dataplane_disabled) { - vblk->dataplane_disabled = false; - vblk->dataplane_started = false; - return; - } - s->stopping = true; - trace_virtio_blk_data_plane_stop(s); - - if (!blk_in_drain(s->conf->conf.blk)) { - for (i = 0; i < nvqs; i++) { - VirtQueue *vq = virtio_get_queue(s->vdev, i); - AioContext *ctx = s->vq_aio_context[i]; - - aio_wait_bh_oneshot(ctx, virtio_blk_data_plane_stop_vq_bh, vq); - } - } - - /* - * Batch all the host notifiers in a single transaction to avoid - * quadratic time complexity in address_space_update_ioeventfds(). - */ - memory_region_transaction_begin(); - - for (i = 0; i < nvqs; i++) { - virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), i, false); - } - - /* - * The transaction expects the ioeventfds to be open when it - * commits. Do it now, before the cleanup loop. - */ - memory_region_transaction_commit(); - - for (i = 0; i < nvqs; i++) { - virtio_bus_cleanup_host_notifier(VIRTIO_BUS(qbus), i); - } - - /* - * Set ->dataplane_started to false before draining so that host notifiers - * are not detached/attached anymore. - */ - vblk->dataplane_started = false; - - /* Wait for virtio_blk_dma_restart_bh() and in flight I/O to complete */ - blk_drain(s->conf->conf.blk); - - /* - * Try to switch bs back to the QEMU main loop. If other users keep the - * BlockBackend in the iothread, that's ok - */ - blk_set_aio_context(s->conf->conf.blk, qemu_get_aio_context(), NULL); - - /* Clean up guest notifier (irq) */ - k->set_guest_notifiers(qbus->parent, nvqs, false); - - s->stopping = false; -} - -void virtio_blk_data_plane_detach(VirtIOBlockDataPlane *s) -{ - VirtIODevice *vdev = VIRTIO_DEVICE(s->vdev); - - for (uint16_t i = 0; i < s->conf->num_queues; i++) { - VirtQueue *vq = virtio_get_queue(vdev, i); - virtio_queue_aio_detach_host_notifier(vq, s->vq_aio_context[i]); - } -} - -void virtio_blk_data_plane_attach(VirtIOBlockDataPlane *s) -{ - VirtIODevice *vdev = VIRTIO_DEVICE(s->vdev); - - for (uint16_t i = 0; i < s->conf->num_queues; i++) { - VirtQueue *vq = virtio_get_queue(vdev, i); - virtio_queue_aio_attach_host_notifier(vq, s->vq_aio_context[i]); - } -} diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index b7a344ca97..510cb4248d 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -27,7 +27,6 @@ #include "sysemu/sysemu.h" #include "sysemu/runstate.h" #include "hw/virtio/virtio-blk.h" -#include "dataplane/virtio-blk.h" #include "scsi/constants.h" #ifdef __linux__ # include @@ -66,7 +65,7 @@ static void virtio_blk_req_complete(VirtIOBlockReq *req, unsigned char status) iov_discard_undo(&req->outhdr_undo); virtqueue_push(req->vq, &req->elem, req->in_len); if (s->dataplane_started && !s->dataplane_disabled) { - virtio_blk_data_plane_notify(s->dataplane, req->vq); + virtio_notify_irqfd(vdev, req->vq); } else { virtio_notify(vdev, req->vq); } @@ -1142,7 +1141,7 @@ static void virtio_blk_handle_output(VirtIODevice *vdev, VirtQueue *vq) { VirtIOBlock *s = (VirtIOBlock *)vdev; - if (s->dataplane && !s->dataplane_started) { + if (!s->dataplane_disabled && !s->dataplane_started) { /* Some guests kick before setting VIRTIO_CONFIG_S_DRIVER_OK so start * dataplane here instead of waiting for .set_status(). */ @@ -1546,16 +1545,34 @@ static void virtio_blk_resize(void *opaque) aio_bh_schedule_oneshot(qemu_get_aio_context(), virtio_resize_cb, vdev); } +static void virtio_blk_data_plane_detach(VirtIOBlock *s) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(s); + + for (uint16_t i = 0; i < s->conf.num_queues; i++) { + VirtQueue *vq = virtio_get_queue(vdev, i); + virtio_queue_aio_detach_host_notifier(vq, s->vq_aio_context[i]); + } +} + +static void virtio_blk_data_plane_attach(VirtIOBlock *s) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(s); + + for (uint16_t i = 0; i < s->conf.num_queues; i++) { + VirtQueue *vq = virtio_get_queue(vdev, i); + virtio_queue_aio_attach_host_notifier(vq, s->vq_aio_context[i]); + } +} + /* Suspend virtqueue ioeventfd processing during drain */ static void virtio_blk_drained_begin(void *opaque) { VirtIOBlock *s = opaque; - if (!s->dataplane || !s->dataplane_started) { - return; + if (s->dataplane_started) { + virtio_blk_data_plane_detach(s); } - - virtio_blk_data_plane_detach(s->dataplane); } /* Resume virtqueue ioeventfd processing after drain */ @@ -1563,11 +1580,9 @@ static void virtio_blk_drained_end(void *opaque) { VirtIOBlock *s = opaque; - if (!s->dataplane || !s->dataplane_started) { - return; + if (s->dataplane_started) { + virtio_blk_data_plane_attach(s); } - - virtio_blk_data_plane_attach(s->dataplane); } static const BlockDevOps virtio_block_ops = { @@ -1576,6 +1591,326 @@ static const BlockDevOps virtio_block_ops = { .drained_end = virtio_blk_drained_end, }; +/* Generate vq:AioContext mappings from a validated iothread-vq-mapping list */ +static void +apply_vq_mapping(IOThreadVirtQueueMappingList *iothread_vq_mapping_list, + AioContext **vq_aio_context, uint16_t num_queues) +{ + IOThreadVirtQueueMappingList *node; + size_t num_iothreads = 0; + size_t cur_iothread = 0; + + for (node = iothread_vq_mapping_list; node; node = node->next) { + num_iothreads++; + } + + for (node = iothread_vq_mapping_list; node; node = node->next) { + IOThread *iothread = iothread_by_id(node->value->iothread); + AioContext *ctx = iothread_get_aio_context(iothread); + + /* Released in virtio_blk_data_plane_destroy() */ + object_ref(OBJECT(iothread)); + + if (node->value->vqs) { + uint16List *vq; + + /* Explicit vq:IOThread assignment */ + for (vq = node->value->vqs; vq; vq = vq->next) { + vq_aio_context[vq->value] = ctx; + } + } else { + /* Round-robin vq:IOThread assignment */ + for (unsigned i = cur_iothread; i < num_queues; + i += num_iothreads) { + vq_aio_context[i] = ctx; + } + } + + cur_iothread++; + } +} + +/* Context: BQL held */ +static bool virtio_blk_data_plane_create(VirtIOBlock *s, Error **errp) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(s); + VirtIOBlkConf *conf = &s->conf; + BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + + if (conf->iothread || conf->iothread_vq_mapping_list) { + if (!k->set_guest_notifiers || !k->ioeventfd_assign) { + error_setg(errp, + "device is incompatible with iothread " + "(transport does not support notifiers)"); + return false; + } + if (!virtio_device_ioeventfd_enabled(vdev)) { + error_setg(errp, "ioeventfd is required for iothread"); + return false; + } + + /* + * If dataplane is (re-)enabled while the guest is running there could + * be block jobs that can conflict. + */ + if (blk_op_is_blocked(conf->conf.blk, BLOCK_OP_TYPE_DATAPLANE, errp)) { + error_prepend(errp, "cannot start virtio-blk dataplane: "); + return false; + } + } + /* Don't try if transport does not support notifiers. */ + if (!virtio_device_ioeventfd_enabled(vdev)) { + s->dataplane_disabled = true; + return false; + } + + s->vq_aio_context = g_new(AioContext *, conf->num_queues); + + if (conf->iothread_vq_mapping_list) { + apply_vq_mapping(conf->iothread_vq_mapping_list, s->vq_aio_context, + conf->num_queues); + } else if (conf->iothread) { + AioContext *ctx = iothread_get_aio_context(conf->iothread); + for (unsigned i = 0; i < conf->num_queues; i++) { + s->vq_aio_context[i] = ctx; + } + + /* Released in virtio_blk_data_plane_destroy() */ + object_ref(OBJECT(conf->iothread)); + } else { + AioContext *ctx = qemu_get_aio_context(); + for (unsigned i = 0; i < conf->num_queues; i++) { + s->vq_aio_context[i] = ctx; + } + } + + return true; +} + +/* Context: BQL held */ +static void virtio_blk_data_plane_destroy(VirtIOBlock *s) +{ + VirtIOBlkConf *conf = &s->conf; + + assert(!s->dataplane_started); + + if (conf->iothread_vq_mapping_list) { + IOThreadVirtQueueMappingList *node; + + for (node = conf->iothread_vq_mapping_list; node; node = node->next) { + IOThread *iothread = iothread_by_id(node->value->iothread); + object_unref(OBJECT(iothread)); + } + } + + if (conf->iothread) { + object_unref(OBJECT(conf->iothread)); + } + + g_free(s->vq_aio_context); + s->vq_aio_context = NULL; +} + +/* Context: BQL held */ +static int virtio_blk_data_plane_start(VirtIODevice *vdev) +{ + VirtIOBlock *s = VIRTIO_BLK(vdev); + BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(s))); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + unsigned i; + unsigned nvqs = s->conf.num_queues; + Error *local_err = NULL; + int r; + + if (s->dataplane_started || s->dataplane_starting) { + return 0; + } + + s->dataplane_starting = true; + + /* Set up guest notifier (irq) */ + r = k->set_guest_notifiers(qbus->parent, nvqs, true); + if (r != 0) { + error_report("virtio-blk failed to set guest notifier (%d), " + "ensure -accel kvm is set.", r); + goto fail_guest_notifiers; + } + + /* + * Batch all the host notifiers in a single transaction to avoid + * quadratic time complexity in address_space_update_ioeventfds(). + */ + memory_region_transaction_begin(); + + /* Set up virtqueue notify */ + for (i = 0; i < nvqs; i++) { + r = virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), i, true); + if (r != 0) { + int j = i; + + fprintf(stderr, "virtio-blk failed to set host notifier (%d)\n", r); + while (i--) { + virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), i, false); + } + + /* + * The transaction expects the ioeventfds to be open when it + * commits. Do it now, before the cleanup loop. + */ + memory_region_transaction_commit(); + + while (j--) { + virtio_bus_cleanup_host_notifier(VIRTIO_BUS(qbus), j); + } + goto fail_host_notifiers; + } + } + + memory_region_transaction_commit(); + + r = blk_set_aio_context(s->conf.conf.blk, s->vq_aio_context[0], + &local_err); + if (r < 0) { + error_report_err(local_err); + goto fail_aio_context; + } + + /* + * These fields must be visible to the IOThread when it processes the + * virtqueue, otherwise it will think dataplane has not started yet. + * + * Make sure ->dataplane_started is false when blk_set_aio_context() is + * called above so that draining does not cause the host notifier to be + * detached/attached prematurely. + */ + s->dataplane_starting = false; + s->dataplane_started = true; + smp_wmb(); /* paired with aio_notify_accept() on the read side */ + + /* Get this show started by hooking up our callbacks */ + if (!blk_in_drain(s->conf.conf.blk)) { + for (i = 0; i < nvqs; i++) { + VirtQueue *vq = virtio_get_queue(vdev, i); + AioContext *ctx = s->vq_aio_context[i]; + + /* Kick right away to begin processing requests already in vring */ + event_notifier_set(virtio_queue_get_host_notifier(vq)); + + virtio_queue_aio_attach_host_notifier(vq, ctx); + } + } + return 0; + + fail_aio_context: + memory_region_transaction_begin(); + + for (i = 0; i < nvqs; i++) { + virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), i, false); + } + + memory_region_transaction_commit(); + + for (i = 0; i < nvqs; i++) { + virtio_bus_cleanup_host_notifier(VIRTIO_BUS(qbus), i); + } + fail_host_notifiers: + k->set_guest_notifiers(qbus->parent, nvqs, false); + fail_guest_notifiers: + s->dataplane_disabled = true; + s->dataplane_starting = false; + return -ENOSYS; +} + +/* Stop notifications for new requests from guest. + * + * Context: BH in IOThread + */ +static void virtio_blk_data_plane_stop_vq_bh(void *opaque) +{ + VirtQueue *vq = opaque; + EventNotifier *host_notifier = virtio_queue_get_host_notifier(vq); + + virtio_queue_aio_detach_host_notifier(vq, qemu_get_current_aio_context()); + + /* + * Test and clear notifier after disabling event, in case poll callback + * didn't have time to run. + */ + virtio_queue_host_notifier_read(host_notifier); +} + +/* Context: BQL held */ +static void virtio_blk_data_plane_stop(VirtIODevice *vdev) +{ + VirtIOBlock *s = VIRTIO_BLK(vdev); + BusState *qbus = qdev_get_parent_bus(DEVICE(s)); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + unsigned i; + unsigned nvqs = s->conf.num_queues; + + if (!s->dataplane_started || s->dataplane_stopping) { + return; + } + + /* Better luck next time. */ + if (s->dataplane_disabled) { + s->dataplane_disabled = false; + s->dataplane_started = false; + return; + } + s->dataplane_stopping = true; + + if (!blk_in_drain(s->conf.conf.blk)) { + for (i = 0; i < nvqs; i++) { + VirtQueue *vq = virtio_get_queue(vdev, i); + AioContext *ctx = s->vq_aio_context[i]; + + aio_wait_bh_oneshot(ctx, virtio_blk_data_plane_stop_vq_bh, vq); + } + } + + /* + * Batch all the host notifiers in a single transaction to avoid + * quadratic time complexity in address_space_update_ioeventfds(). + */ + memory_region_transaction_begin(); + + for (i = 0; i < nvqs; i++) { + virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), i, false); + } + + /* + * The transaction expects the ioeventfds to be open when it + * commits. Do it now, before the cleanup loop. + */ + memory_region_transaction_commit(); + + for (i = 0; i < nvqs; i++) { + virtio_bus_cleanup_host_notifier(VIRTIO_BUS(qbus), i); + } + + /* + * Set ->dataplane_started to false before draining so that host notifiers + * are not detached/attached anymore. + */ + s->dataplane_started = false; + + /* Wait for virtio_blk_dma_restart_bh() and in flight I/O to complete */ + blk_drain(s->conf.conf.blk); + + /* + * Try to switch bs back to the QEMU main loop. If other users keep the + * BlockBackend in the iothread, that's ok + */ + blk_set_aio_context(s->conf.conf.blk, qemu_get_aio_context(), NULL); + + /* Clean up guest notifier (irq) */ + k->set_guest_notifiers(qbus->parent, nvqs, false); + + s->dataplane_stopping = false; +} + static void virtio_blk_device_realize(DeviceState *dev, Error **errp) { VirtIODevice *vdev = VIRTIO_DEVICE(dev); @@ -1680,7 +2015,7 @@ static void virtio_blk_device_realize(DeviceState *dev, Error **errp) virtio_add_queue(vdev, conf->queue_size, virtio_blk_handle_output); } qemu_coroutine_inc_pool_size(conf->num_queues * conf->queue_size / 2); - virtio_blk_data_plane_create(vdev, conf, &s->dataplane, &err); + virtio_blk_data_plane_create(s, &err); if (err != NULL) { error_propagate(errp, err); for (i = 0; i < conf->num_queues; i++) { @@ -1717,8 +2052,7 @@ static void virtio_blk_device_unrealize(DeviceState *dev) blk_drain(s->blk); del_boot_device_lchs(dev, "/disk@0,0"); - virtio_blk_data_plane_destroy(s->dataplane); - s->dataplane = NULL; + virtio_blk_data_plane_destroy(s); for (i = 0; i < conf->num_queues; i++) { virtio_del_queue(vdev, i); } diff --git a/hw/block/dataplane/meson.build b/hw/block/dataplane/meson.build index 025b3b061b..11a5eba2f4 100644 --- a/hw/block/dataplane/meson.build +++ b/hw/block/dataplane/meson.build @@ -1,2 +1 @@ -system_ss.add(when: 'CONFIG_VIRTIO_BLK', if_true: files('virtio-blk.c')) specific_ss.add(when: 'CONFIG_XEN_BUS', if_true: files('xen-block.c')) diff --git a/hw/block/dataplane/trace-events b/hw/block/dataplane/trace-events deleted file mode 100644 index 38fc3e7507..0000000000 --- a/hw/block/dataplane/trace-events +++ /dev/null @@ -1,5 +0,0 @@ -# See docs/devel/tracing.rst for syntax documentation. - -# virtio-blk.c -virtio_blk_data_plane_start(void *s) "dataplane %p" -virtio_blk_data_plane_stop(void *s) "dataplane %p" diff --git a/meson.build b/meson.build index d0329966f1..5ffa8c9716 100644 --- a/meson.build +++ b/meson.build @@ -3286,7 +3286,6 @@ if have_system 'hw/arm', 'hw/audio', 'hw/block', - 'hw/block/dataplane', 'hw/char', 'hw/display', 'hw/dma', From patchwork Fri Jan 19 18:13:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1888626 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=L7rjR7g5; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TGnqT0gPxz1yPV for ; Sat, 20 Jan 2024 05:16:05 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rQtNO-0001kS-5I; Fri, 19 Jan 2024 13:13:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtNF-0001eK-4D for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:50 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtN9-0007dk-Fd for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705688022; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s7qkNK5Ps/JlBaH+DYs59T5dlAG9O0lKVhHokbMulWY=; b=L7rjR7g5VxxmPZUnj+v05tKdPjzgtxlWhtmc9PoBZ6PhWRtfmsZRmh5GiyFzgK6uXG3N48 gnJwXKC9XPIu0W5Z0DiAfNLN03I5oDAjie+sK4/zhgi8uOGzFIHbonidABkggZ14/bvKsp apvLVB0n/AUu9lxEH+cHV9P23cjcFTU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-627-WwosFf_MOFKrbJh3vAcicg-1; Fri, 19 Jan 2024 13:13:41 -0500 X-MC-Unique: WwosFf_MOFKrbJh3vAcicg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B661910665A6; Fri, 19 Jan 2024 18:13:40 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.86]) by smtp.corp.redhat.com (Postfix) with ESMTP id 24E06494; Fri, 19 Jan 2024 18:13:40 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 09/14] virtio-blk: rename dataplane create/destroy functions Date: Fri, 19 Jan 2024 19:13:22 +0100 Message-ID: <20240119181327.236745-10-kwolf@redhat.com> In-Reply-To: <20240119181327.236745-1-kwolf@redhat.com> References: <20240119181327.236745-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.519, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Stefan Hajnoczi virtio_blk_data_plane_create() and virtio_blk_data_plane_destroy() are actually about s->vq_aio_context[] rather than managing dataplane-specific state. As a prerequisite to using s->vq_aio_context[] in all code paths (even when dataplane is not used), rename these functions to reflect that they just manage s->vq_aio_context and call them regardless of whether or not dataplane is in use. Note that virtio-blk supports running with -device virtio-blk-pci,ioevent=off where the vCPU thread enters the device emulation code. In this mode ioeventfd is not used for virtqueue processing. However, we still want to initialize s->vq_aio_context[] to qemu_aio_context in that case since I/O completion callbacks will be invoked in the main loop thread. Signed-off-by: Stefan Hajnoczi Message-ID: <20240119135748.270944-3-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- hw/block/virtio-blk.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index 510cb4248d..47494ebadd 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -1608,7 +1608,7 @@ apply_vq_mapping(IOThreadVirtQueueMappingList *iothread_vq_mapping_list, IOThread *iothread = iothread_by_id(node->value->iothread); AioContext *ctx = iothread_get_aio_context(iothread); - /* Released in virtio_blk_data_plane_destroy() */ + /* Released in virtio_blk_vq_aio_context_cleanup() */ object_ref(OBJECT(iothread)); if (node->value->vqs) { @@ -1631,7 +1631,7 @@ apply_vq_mapping(IOThreadVirtQueueMappingList *iothread_vq_mapping_list, } /* Context: BQL held */ -static bool virtio_blk_data_plane_create(VirtIOBlock *s, Error **errp) +static bool virtio_blk_vq_aio_context_init(VirtIOBlock *s, Error **errp) { VirtIODevice *vdev = VIRTIO_DEVICE(s); VirtIOBlkConf *conf = &s->conf; @@ -1659,11 +1659,6 @@ static bool virtio_blk_data_plane_create(VirtIOBlock *s, Error **errp) return false; } } - /* Don't try if transport does not support notifiers. */ - if (!virtio_device_ioeventfd_enabled(vdev)) { - s->dataplane_disabled = true; - return false; - } s->vq_aio_context = g_new(AioContext *, conf->num_queues); @@ -1676,7 +1671,7 @@ static bool virtio_blk_data_plane_create(VirtIOBlock *s, Error **errp) s->vq_aio_context[i] = ctx; } - /* Released in virtio_blk_data_plane_destroy() */ + /* Released in virtio_blk_vq_aio_context_cleanup() */ object_ref(OBJECT(conf->iothread)); } else { AioContext *ctx = qemu_get_aio_context(); @@ -1689,7 +1684,7 @@ static bool virtio_blk_data_plane_create(VirtIOBlock *s, Error **errp) } /* Context: BQL held */ -static void virtio_blk_data_plane_destroy(VirtIOBlock *s) +static void virtio_blk_vq_aio_context_cleanup(VirtIOBlock *s) { VirtIOBlkConf *conf = &s->conf; @@ -2015,7 +2010,13 @@ static void virtio_blk_device_realize(DeviceState *dev, Error **errp) virtio_add_queue(vdev, conf->queue_size, virtio_blk_handle_output); } qemu_coroutine_inc_pool_size(conf->num_queues * conf->queue_size / 2); - virtio_blk_data_plane_create(s, &err); + + /* Don't start dataplane if transport does not support notifiers. */ + if (!virtio_device_ioeventfd_enabled(vdev)) { + s->dataplane_disabled = true; + } + + virtio_blk_vq_aio_context_init(s, &err); if (err != NULL) { error_propagate(errp, err); for (i = 0; i < conf->num_queues; i++) { @@ -2052,7 +2053,7 @@ static void virtio_blk_device_unrealize(DeviceState *dev) blk_drain(s->blk); del_boot_device_lchs(dev, "/disk@0,0"); - virtio_blk_data_plane_destroy(s); + virtio_blk_vq_aio_context_cleanup(s); for (i = 0; i < conf->num_queues; i++) { virtio_del_queue(vdev, i); } From patchwork Fri Jan 19 18:13:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1888629 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=K8WW/pJ6; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TGnqz3ZZNz1yPV for ; Sat, 20 Jan 2024 05:16:31 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rQtNM-0001jS-EO; Fri, 19 Jan 2024 13:13:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtNF-0001eM-O5 for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:50 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtNA-0007eX-Jn for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:49 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705688023; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Nbl/qqCJEWa0LP2TGqKJo8vlByfV4ij/WjG/uLcE6Cc=; b=K8WW/pJ6nfizwfqfwgvFxH/j8Z0u4n4QBomCPfEinq6Ioy4400Wdc2550LMAmXq1rMPgzm Axd10NWPh4dDOoaPJTm1GQXujqmixg5BW1rbLNksMlo8EjULP1tj01nu/N+VKbhIWN8cn0 Ma0bE/5U//KQnwwdOyZQ5ZuGcCvffJY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-613-zh1ObvJAMk2j-cXokxe19Q-1; Fri, 19 Jan 2024 13:13:41 -0500 X-MC-Unique: zh1ObvJAMk2j-cXokxe19Q-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 88ABB185A784; Fri, 19 Jan 2024 18:13:41 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.86]) by smtp.corp.redhat.com (Postfix) with ESMTP id EEEF4494; Fri, 19 Jan 2024 18:13:40 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 10/14] virtio-blk: rename dataplane to ioeventfd Date: Fri, 19 Jan 2024 19:13:23 +0100 Message-ID: <20240119181327.236745-11-kwolf@redhat.com> In-Reply-To: <20240119181327.236745-1-kwolf@redhat.com> References: <20240119181327.236745-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.519, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Stefan Hajnoczi The dataplane code is really about using ioeventfd. It's used both for IOThreads (what we think of as dataplane) and for the core virtio-pci code's ioeventfd feature (which is enabled by default and used when no IOThread has been specified). Rename the code to reflect this. Signed-off-by: Stefan Hajnoczi Message-ID: <20240119135748.270944-4-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- include/hw/virtio/virtio-blk.h | 8 ++-- hw/block/virtio-blk.c | 78 +++++++++++++++++----------------- 2 files changed, 43 insertions(+), 43 deletions(-) diff --git a/include/hw/virtio/virtio-blk.h b/include/hw/virtio/virtio-blk.h index fecffdc303..833a9a344f 100644 --- a/include/hw/virtio/virtio-blk.h +++ b/include/hw/virtio/virtio-blk.h @@ -60,10 +60,10 @@ struct VirtIOBlock { unsigned short sector_mask; bool original_wce; VMChangeStateEntry *change; - bool dataplane_disabled; - bool dataplane_started; - bool dataplane_starting; - bool dataplane_stopping; + bool ioeventfd_disabled; + bool ioeventfd_started; + bool ioeventfd_starting; + bool ioeventfd_stopping; /* * The AioContext for each virtqueue. The BlockDriverState will use the diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index 47494ebadd..e342cb2cfb 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -64,7 +64,7 @@ static void virtio_blk_req_complete(VirtIOBlockReq *req, unsigned char status) iov_discard_undo(&req->inhdr_undo); iov_discard_undo(&req->outhdr_undo); virtqueue_push(req->vq, &req->elem, req->in_len); - if (s->dataplane_started && !s->dataplane_disabled) { + if (s->ioeventfd_started && !s->ioeventfd_disabled) { virtio_notify_irqfd(vdev, req->vq); } else { virtio_notify(vdev, req->vq); @@ -1141,12 +1141,12 @@ static void virtio_blk_handle_output(VirtIODevice *vdev, VirtQueue *vq) { VirtIOBlock *s = (VirtIOBlock *)vdev; - if (!s->dataplane_disabled && !s->dataplane_started) { + if (!s->ioeventfd_disabled && !s->ioeventfd_started) { /* Some guests kick before setting VIRTIO_CONFIG_S_DRIVER_OK so start - * dataplane here instead of waiting for .set_status(). + * ioeventfd here instead of waiting for .set_status(). */ virtio_device_start_ioeventfd(vdev); - if (!s->dataplane_disabled) { + if (!s->ioeventfd_disabled) { return; } } @@ -1213,7 +1213,7 @@ static void virtio_blk_reset(VirtIODevice *vdev) VirtIOBlockReq *req; /* Dataplane has stopped... */ - assert(!s->dataplane_started); + assert(!s->ioeventfd_started); /* ...but requests may still be in flight. */ blk_drain(s->blk); @@ -1380,7 +1380,7 @@ static void virtio_blk_set_status(VirtIODevice *vdev, uint8_t status) VirtIOBlock *s = VIRTIO_BLK(vdev); if (!(status & (VIRTIO_CONFIG_S_DRIVER | VIRTIO_CONFIG_S_DRIVER_OK))) { - assert(!s->dataplane_started); + assert(!s->ioeventfd_started); } if (!(status & VIRTIO_CONFIG_S_DRIVER_OK)) { @@ -1545,7 +1545,7 @@ static void virtio_blk_resize(void *opaque) aio_bh_schedule_oneshot(qemu_get_aio_context(), virtio_resize_cb, vdev); } -static void virtio_blk_data_plane_detach(VirtIOBlock *s) +static void virtio_blk_ioeventfd_detach(VirtIOBlock *s) { VirtIODevice *vdev = VIRTIO_DEVICE(s); @@ -1555,7 +1555,7 @@ static void virtio_blk_data_plane_detach(VirtIOBlock *s) } } -static void virtio_blk_data_plane_attach(VirtIOBlock *s) +static void virtio_blk_ioeventfd_attach(VirtIOBlock *s) { VirtIODevice *vdev = VIRTIO_DEVICE(s); @@ -1570,8 +1570,8 @@ static void virtio_blk_drained_begin(void *opaque) { VirtIOBlock *s = opaque; - if (s->dataplane_started) { - virtio_blk_data_plane_detach(s); + if (s->ioeventfd_started) { + virtio_blk_ioeventfd_detach(s); } } @@ -1580,8 +1580,8 @@ static void virtio_blk_drained_end(void *opaque) { VirtIOBlock *s = opaque; - if (s->dataplane_started) { - virtio_blk_data_plane_attach(s); + if (s->ioeventfd_started) { + virtio_blk_ioeventfd_attach(s); } } @@ -1651,11 +1651,11 @@ static bool virtio_blk_vq_aio_context_init(VirtIOBlock *s, Error **errp) } /* - * If dataplane is (re-)enabled while the guest is running there could + * If ioeventfd is (re-)enabled while the guest is running there could * be block jobs that can conflict. */ if (blk_op_is_blocked(conf->conf.blk, BLOCK_OP_TYPE_DATAPLANE, errp)) { - error_prepend(errp, "cannot start virtio-blk dataplane: "); + error_prepend(errp, "cannot start virtio-blk ioeventfd: "); return false; } } @@ -1688,7 +1688,7 @@ static void virtio_blk_vq_aio_context_cleanup(VirtIOBlock *s) { VirtIOBlkConf *conf = &s->conf; - assert(!s->dataplane_started); + assert(!s->ioeventfd_started); if (conf->iothread_vq_mapping_list) { IOThreadVirtQueueMappingList *node; @@ -1708,7 +1708,7 @@ static void virtio_blk_vq_aio_context_cleanup(VirtIOBlock *s) } /* Context: BQL held */ -static int virtio_blk_data_plane_start(VirtIODevice *vdev) +static int virtio_blk_start_ioeventfd(VirtIODevice *vdev) { VirtIOBlock *s = VIRTIO_BLK(vdev); BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(s))); @@ -1718,11 +1718,11 @@ static int virtio_blk_data_plane_start(VirtIODevice *vdev) Error *local_err = NULL; int r; - if (s->dataplane_started || s->dataplane_starting) { + if (s->ioeventfd_started || s->ioeventfd_starting) { return 0; } - s->dataplane_starting = true; + s->ioeventfd_starting = true; /* Set up guest notifier (irq) */ r = k->set_guest_notifiers(qbus->parent, nvqs, true); @@ -1773,14 +1773,14 @@ static int virtio_blk_data_plane_start(VirtIODevice *vdev) /* * These fields must be visible to the IOThread when it processes the - * virtqueue, otherwise it will think dataplane has not started yet. + * virtqueue, otherwise it will think ioeventfd has not started yet. * - * Make sure ->dataplane_started is false when blk_set_aio_context() is + * Make sure ->ioeventfd_started is false when blk_set_aio_context() is * called above so that draining does not cause the host notifier to be * detached/attached prematurely. */ - s->dataplane_starting = false; - s->dataplane_started = true; + s->ioeventfd_starting = false; + s->ioeventfd_started = true; smp_wmb(); /* paired with aio_notify_accept() on the read side */ /* Get this show started by hooking up our callbacks */ @@ -1812,8 +1812,8 @@ static int virtio_blk_data_plane_start(VirtIODevice *vdev) fail_host_notifiers: k->set_guest_notifiers(qbus->parent, nvqs, false); fail_guest_notifiers: - s->dataplane_disabled = true; - s->dataplane_starting = false; + s->ioeventfd_disabled = true; + s->ioeventfd_starting = false; return -ENOSYS; } @@ -1821,7 +1821,7 @@ static int virtio_blk_data_plane_start(VirtIODevice *vdev) * * Context: BH in IOThread */ -static void virtio_blk_data_plane_stop_vq_bh(void *opaque) +static void virtio_blk_ioeventfd_stop_vq_bh(void *opaque) { VirtQueue *vq = opaque; EventNotifier *host_notifier = virtio_queue_get_host_notifier(vq); @@ -1836,7 +1836,7 @@ static void virtio_blk_data_plane_stop_vq_bh(void *opaque) } /* Context: BQL held */ -static void virtio_blk_data_plane_stop(VirtIODevice *vdev) +static void virtio_blk_stop_ioeventfd(VirtIODevice *vdev) { VirtIOBlock *s = VIRTIO_BLK(vdev); BusState *qbus = qdev_get_parent_bus(DEVICE(s)); @@ -1844,24 +1844,24 @@ static void virtio_blk_data_plane_stop(VirtIODevice *vdev) unsigned i; unsigned nvqs = s->conf.num_queues; - if (!s->dataplane_started || s->dataplane_stopping) { + if (!s->ioeventfd_started || s->ioeventfd_stopping) { return; } /* Better luck next time. */ - if (s->dataplane_disabled) { - s->dataplane_disabled = false; - s->dataplane_started = false; + if (s->ioeventfd_disabled) { + s->ioeventfd_disabled = false; + s->ioeventfd_started = false; return; } - s->dataplane_stopping = true; + s->ioeventfd_stopping = true; if (!blk_in_drain(s->conf.conf.blk)) { for (i = 0; i < nvqs; i++) { VirtQueue *vq = virtio_get_queue(vdev, i); AioContext *ctx = s->vq_aio_context[i]; - aio_wait_bh_oneshot(ctx, virtio_blk_data_plane_stop_vq_bh, vq); + aio_wait_bh_oneshot(ctx, virtio_blk_ioeventfd_stop_vq_bh, vq); } } @@ -1886,10 +1886,10 @@ static void virtio_blk_data_plane_stop(VirtIODevice *vdev) } /* - * Set ->dataplane_started to false before draining so that host notifiers + * Set ->ioeventfd_started to false before draining so that host notifiers * are not detached/attached anymore. */ - s->dataplane_started = false; + s->ioeventfd_started = false; /* Wait for virtio_blk_dma_restart_bh() and in flight I/O to complete */ blk_drain(s->conf.conf.blk); @@ -1903,7 +1903,7 @@ static void virtio_blk_data_plane_stop(VirtIODevice *vdev) /* Clean up guest notifier (irq) */ k->set_guest_notifiers(qbus->parent, nvqs, false); - s->dataplane_stopping = false; + s->ioeventfd_stopping = false; } static void virtio_blk_device_realize(DeviceState *dev, Error **errp) @@ -2011,9 +2011,9 @@ static void virtio_blk_device_realize(DeviceState *dev, Error **errp) } qemu_coroutine_inc_pool_size(conf->num_queues * conf->queue_size / 2); - /* Don't start dataplane if transport does not support notifiers. */ + /* Don't start ioeventfd if transport does not support notifiers. */ if (!virtio_device_ioeventfd_enabled(vdev)) { - s->dataplane_disabled = true; + s->ioeventfd_disabled = true; } virtio_blk_vq_aio_context_init(s, &err); @@ -2137,8 +2137,8 @@ static void virtio_blk_class_init(ObjectClass *klass, void *data) vdc->reset = virtio_blk_reset; vdc->save = virtio_blk_save_device; vdc->load = virtio_blk_load_device; - vdc->start_ioeventfd = virtio_blk_data_plane_start; - vdc->stop_ioeventfd = virtio_blk_data_plane_stop; + vdc->start_ioeventfd = virtio_blk_start_ioeventfd; + vdc->stop_ioeventfd = virtio_blk_stop_ioeventfd; } static const TypeInfo virtio_blk_info = { From patchwork Fri Jan 19 18:13:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1888625 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=QcwqTEv/; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TGnqJ3rwhz1yPV for ; Sat, 20 Jan 2024 05:15:56 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rQtNh-0001rH-QK; Fri, 19 Jan 2024 13:14:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtNg-0001q4-2a for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:14:16 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtNe-0007uM-C9 for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:14:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705688053; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YCKVranjo32oXM1RFvDB6oB1rASnxCddPlsrDA/nAYU=; b=QcwqTEv/p8nIGH328wuozfwOh4BIY2lnN6eMmZomLdd2OTXkJkkaBTF2bOmjBIwzMRaOzC sx9MLJQAqykoO6FTsOZhYGtKoEEwVSyfB9bxOJxQy7PQRqaqyq+O96W8Ni1rk49uRiKAVp kcNU+IIYg2S0CtNuMS6x+YnZ7MEnvhA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-436-bujaNCi7PUmkBLe0szgZ1A-1; Fri, 19 Jan 2024 13:13:42 -0500 X-MC-Unique: bujaNCi7PUmkBLe0szgZ1A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 58915867943; Fri, 19 Jan 2024 18:13:42 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.86]) by smtp.corp.redhat.com (Postfix) with ESMTP id BED8B1B96; Fri, 19 Jan 2024 18:13:41 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 11/14] virtio-blk: restart s->rq reqs in vq AioContexts Date: Fri, 19 Jan 2024 19:13:24 +0100 Message-ID: <20240119181327.236745-12-kwolf@redhat.com> In-Reply-To: <20240119181327.236745-1-kwolf@redhat.com> References: <20240119181327.236745-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.519, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Stefan Hajnoczi A virtio-blk device with the iothread-vq-mapping parameter has per-virtqueue AioContexts. It is not thread-safe to process s->rq requests in the BlockBackend AioContext since that may be different from the virtqueue's AioContext to which this request belongs. The code currently races and could crash. Adapt virtio_blk_dma_restart_cb() to first split s->rq into per-vq lists and then schedule a BH each vq's AioContext as necessary. This way requests are safely processed in their vq's AioContext. Signed-off-by: Stefan Hajnoczi Message-ID: <20240119135748.270944-5-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- hw/block/virtio-blk.c | 44 ++++++++++++++++++++++++++++++++----------- 1 file changed, 33 insertions(+), 11 deletions(-) diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index e342cb2cfb..4525988d92 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -1156,16 +1156,11 @@ static void virtio_blk_handle_output(VirtIODevice *vdev, VirtQueue *vq) static void virtio_blk_dma_restart_bh(void *opaque) { - VirtIOBlock *s = opaque; + VirtIOBlockReq *req = opaque; + VirtIOBlock *s = req->dev; /* we're called with at least one request */ - VirtIOBlockReq *req; MultiReqBuffer mrb = {}; - WITH_QEMU_LOCK_GUARD(&s->rq_lock) { - req = s->rq; - s->rq = NULL; - } - while (req) { VirtIOBlockReq *next = req->next; if (virtio_blk_handle_request(req, &mrb)) { @@ -1195,16 +1190,43 @@ static void virtio_blk_dma_restart_cb(void *opaque, bool running, RunState state) { VirtIOBlock *s = opaque; + uint16_t num_queues = s->conf.num_queues; if (!running) { return; } - /* Paired with dec in virtio_blk_dma_restart_bh() */ - blk_inc_in_flight(s->conf.conf.blk); + /* Split the device-wide s->rq request list into per-vq request lists */ + g_autofree VirtIOBlockReq **vq_rq = g_new0(VirtIOBlockReq *, num_queues); + VirtIOBlockReq *rq; + + WITH_QEMU_LOCK_GUARD(&s->rq_lock) { + rq = s->rq; + s->rq = NULL; + } + + while (rq) { + VirtIOBlockReq *next = rq->next; + uint16_t idx = virtio_get_queue_index(rq->vq); + + rq->next = vq_rq[idx]; + vq_rq[idx] = rq; + rq = next; + } - aio_bh_schedule_oneshot(blk_get_aio_context(s->conf.conf.blk), - virtio_blk_dma_restart_bh, s); + /* Schedule a BH to submit the requests in each vq's AioContext */ + for (uint16_t i = 0; i < num_queues; i++) { + if (!vq_rq[i]) { + continue; + } + + /* Paired with dec in virtio_blk_dma_restart_bh() */ + blk_inc_in_flight(s->conf.conf.blk); + + aio_bh_schedule_oneshot(s->vq_aio_context[i], + virtio_blk_dma_restart_bh, + vq_rq[i]); + } } static void virtio_blk_reset(VirtIODevice *vdev) From patchwork Fri Jan 19 18:13:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1888617 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=hnZ9UIng; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TGnpX74tcz1yPV for ; Sat, 20 Jan 2024 05:15:15 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rQtNS-0001nH-DI; Fri, 19 Jan 2024 13:14:02 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtNI-0001ff-Md for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:52 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtNE-0007fn-97 for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:52 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705688026; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=g9K903/Yegnn9dUEU+6nFxa0MGx2PhUv24bahJVyoCo=; b=hnZ9UIngvyN+6TvORixpWKhVCdM49PcAfCU62QsJALcs8AGDh/R4vd8X86R/F3/OZvICmG IGaDiqrypEJ4TSmnKptLLi2Zq/MBeAOtxdxDXRj8g7hbiYIuyNNYwJdjmfFBeM0gsWSHCy +FibfUo2cAsC6UVIMoe/+n0aC72H+R8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-657-mRG-4jmUNH29YzEVrS2e9g-1; Fri, 19 Jan 2024 13:13:43 -0500 X-MC-Unique: mRG-4jmUNH29YzEVrS2e9g-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3592685A58C; Fri, 19 Jan 2024 18:13:43 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.86]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8DBD0494; Fri, 19 Jan 2024 18:13:42 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 12/14] virtio-blk: tolerate failure to set BlockBackend AioContext Date: Fri, 19 Jan 2024 19:13:25 +0100 Message-ID: <20240119181327.236745-13-kwolf@redhat.com> In-Reply-To: <20240119181327.236745-1-kwolf@redhat.com> References: <20240119181327.236745-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.519, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Stefan Hajnoczi We no longer rely on setting the AioContext since the block layer IO_CODE APIs can be called from any thread. Now it's just a hint to help block jobs and other operations co-locate themselves in a thread with the guest I/O requests. Keep going if setting the AioContext fails. Suggested-by: Kevin Wolf Signed-off-by: Stefan Hajnoczi Message-ID: <20240119135748.270944-6-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- hw/block/virtio-blk.c | 19 +++++-------------- 1 file changed, 5 insertions(+), 14 deletions(-) diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index 4525988d92..73248d15c8 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -1786,11 +1786,14 @@ static int virtio_blk_start_ioeventfd(VirtIODevice *vdev) memory_region_transaction_commit(); + /* + * Try to change the AioContext so that block jobs and other operations can + * co-locate their activity in the same AioContext. If it fails, nevermind. + */ r = blk_set_aio_context(s->conf.conf.blk, s->vq_aio_context[0], &local_err); if (r < 0) { - error_report_err(local_err); - goto fail_aio_context; + warn_report_err(local_err); } /* @@ -1819,18 +1822,6 @@ static int virtio_blk_start_ioeventfd(VirtIODevice *vdev) } return 0; - fail_aio_context: - memory_region_transaction_begin(); - - for (i = 0; i < nvqs; i++) { - virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), i, false); - } - - memory_region_transaction_commit(); - - for (i = 0; i < nvqs; i++) { - virtio_bus_cleanup_host_notifier(VIRTIO_BUS(qbus), i); - } fail_host_notifiers: k->set_guest_notifiers(qbus->parent, nvqs, false); fail_guest_notifiers: From patchwork Fri Jan 19 18:13:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1888621 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=O8YJzDOI; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TGnpk5gy0z23fB for ; Sat, 20 Jan 2024 05:15:26 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rQtNZ-0001on-Nv; Fri, 19 Jan 2024 13:14:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtNY-0001oO-8L for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:14:08 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtNW-0007jq-DO for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:14:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705688045; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8S/6JmKhF1sJKzjwL9n1KQwB67QmH+PBjRC6YVJgQIk=; b=O8YJzDOIZNLuq9Mdj4XzsD4c/k9wrOSC/ClVIVXeceBNx0eJVXx6wFIT9vlC5eAQ/OEmez ouGnZzL+Hsyz/RfodBcachKwr8xQDM+GeL7LNd3KGEqYb6a07d+x6W9/1r9yyjjRMSUWj8 hfa2ZEFWAdlBKBm3ybYQitjYfeP0J7o= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-345-HC7_So04N7qJ8bJyOMWNZw-1; Fri, 19 Jan 2024 13:13:59 -0500 X-MC-Unique: HC7_So04N7qJ8bJyOMWNZw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 04A02185A784; Fri, 19 Jan 2024 18:13:44 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.86]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6A887900; Fri, 19 Jan 2024 18:13:43 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 13/14] virtio-blk: always set ioeventfd during startup Date: Fri, 19 Jan 2024 19:13:26 +0100 Message-ID: <20240119181327.236745-14-kwolf@redhat.com> In-Reply-To: <20240119181327.236745-1-kwolf@redhat.com> References: <20240119181327.236745-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.519, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Stefan Hajnoczi When starting ioeventfd it is common practice to set the event notifier so that the ioeventfd handler is triggered to run immediately. There may be no requests waiting to be processed, but the idea is that if a request snuck in then we guarantee that it will be detected. One scenario where self-triggering the ioeventfd is necessary is when virtio_blk_handle_output() is called from a vCPU thread before the VIRTIO Device Status transitions to DRIVER_OK. In that case we need to self-trigger the ioeventfd so that the kick handled by the vCPU thread causes the vq AioContext thread to take over handling the request(s). Fixes: b6948ab01df0 ("virtio-blk: add iothread-vq-mapping parameter") Reported-by: Kevin Wolf Signed-off-by: Stefan Hajnoczi Message-ID: <20240119135748.270944-7-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- hw/block/virtio-blk.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index 73248d15c8..227d83569f 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -1809,14 +1809,14 @@ static int virtio_blk_start_ioeventfd(VirtIODevice *vdev) smp_wmb(); /* paired with aio_notify_accept() on the read side */ /* Get this show started by hooking up our callbacks */ - if (!blk_in_drain(s->conf.conf.blk)) { - for (i = 0; i < nvqs; i++) { - VirtQueue *vq = virtio_get_queue(vdev, i); - AioContext *ctx = s->vq_aio_context[i]; + for (i = 0; i < nvqs; i++) { + VirtQueue *vq = virtio_get_queue(vdev, i); + AioContext *ctx = s->vq_aio_context[i]; - /* Kick right away to begin processing requests already in vring */ - event_notifier_set(virtio_queue_get_host_notifier(vq)); + /* Kick right away to begin processing requests already in vring */ + event_notifier_set(virtio_queue_get_host_notifier(vq)); + if (!blk_in_drain(s->conf.conf.blk)) { virtio_queue_aio_attach_host_notifier(vq, ctx); } } From patchwork Fri Jan 19 18:13:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1888628 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=LG5k+D4W; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TGnqx6dyRz1yPV for ; Sat, 20 Jan 2024 05:16:29 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rQtNM-0001jf-Tl; Fri, 19 Jan 2024 13:13:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtNL-0001iB-6L for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:55 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rQtNF-0007gO-3F for qemu-devel@nongnu.org; Fri, 19 Jan 2024 13:13:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705688028; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Sj3ULLvzpdZSSoBFU8DUbG4v/mhg3sH2J0N7KsejgkQ=; b=LG5k+D4WhjYyUi1bc413hPNJOWil9Eeu7n00YNAzlR7UIN8xTyzMceQvhnv2NaRA/Oi64U 2+sOFbdW+7MbDbDANuku+oUrokwlEVTb3xCPIPn/jEyjv1VJGeDkypk8K5M53g1TIv4Zcc ybYAdrikYn5Q9kK539T7UxWaU7D6ly0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-5-hig_P3L5Ooiw46CJ8Su1bg-1; Fri, 19 Jan 2024 13:13:45 -0500 X-MC-Unique: hig_P3L5Ooiw46CJ8Su1bg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C8A3410665A2; Fri, 19 Jan 2024 18:13:44 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.86]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3A9FC1B96; Fri, 19 Jan 2024 18:13:44 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 14/14] block/blklogwrites: Protect mutable driver state with a mutex. Date: Fri, 19 Jan 2024 19:13:27 +0100 Message-ID: <20240119181327.236745-15-kwolf@redhat.com> In-Reply-To: <20240119181327.236745-1-kwolf@redhat.com> References: <20240119181327.236745-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.519, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Ari Sundholm During the review of a fix for a concurrency issue in blklogwrites, it was found that the driver needs an additional fix when enabling multiqueue, which is a new feature introduced in QEMU 9.0, as the driver state may be read and written by multiple threads at the same time, which was not the case when the driver was originally written. Fix the multi-threaded scenario by introducing a mutex to protect the mutable fields in the driver state, and always having the mutex locked by the current thread when accessing them. Also use the mutex and a CoQueue to ensure that the super block is not being written to by multiple threads concurrently and updates are properly serialized. Additionally, add the const qualifier to a few BDRVBlkLogWritesState pointer targets in contexts where the driver state is not written to. Signed-off-by: Ari Sundholm Message-ID: <20240119162913.2620245-1-ari@tuxera.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- block/blklogwrites.c | 85 +++++++++++++++++++++++++++++++++++++++----- 1 file changed, 77 insertions(+), 8 deletions(-) diff --git a/block/blklogwrites.c b/block/blklogwrites.c index ba717dab4d..399819a2a1 100644 --- a/block/blklogwrites.c +++ b/block/blklogwrites.c @@ -3,7 +3,7 @@ * * Copyright (c) 2017 Tuomas Tynkkynen * Copyright (c) 2018 Aapo Vienamo - * Copyright (c) 2018 Ari Sundholm + * Copyright (c) 2018-2024 Ari Sundholm * * This work is licensed under the terms of the GNU GPL, version 2 or later. * See the COPYING file in the top-level directory. @@ -55,9 +55,34 @@ typedef struct { BdrvChild *log_file; uint32_t sectorsize; uint32_t sectorbits; + uint64_t update_interval; + + /* + * The mutable state of the driver, consisting of the current log sector + * and the number of log entries. + * + * May be read and/or written from multiple threads, and the mutex must be + * held when accessing these fields. + */ uint64_t cur_log_sector; uint64_t nr_entries; - uint64_t update_interval; + QemuMutex mutex; + + /* + * The super block sequence number. Non-zero if a super block update is in + * progress. + * + * The mutex must be held when accessing this field. + */ + uint64_t super_update_seq; + + /* + * A coroutine-aware queue to serialize super block updates. + * + * Used with the mutex to ensure that only one thread be updating the super + * block at a time. + */ + CoQueue super_update_queue; } BDRVBlkLogWritesState; static QemuOptsList runtime_opts = { @@ -169,6 +194,9 @@ static int blk_log_writes_open(BlockDriverState *bs, QDict *options, int flags, goto fail; } + qemu_mutex_init(&s->mutex); + qemu_co_queue_init(&s->super_update_queue); + log_append = qemu_opt_get_bool(opts, "log-append", false); if (log_append) { @@ -231,6 +259,8 @@ static int blk_log_writes_open(BlockDriverState *bs, QDict *options, int flags, s->nr_entries = 0; } + s->super_update_seq = 0; + if (!blk_log_writes_sector_size_valid(log_sector_size)) { ret = -EINVAL; error_setg(errp, "Invalid log sector size %"PRIu64, log_sector_size); @@ -255,6 +285,7 @@ fail_log: bdrv_unref_child(bs, s->log_file); bdrv_graph_wrunlock(); s->log_file = NULL; + qemu_mutex_destroy(&s->mutex); } fail: qemu_opts_del(opts); @@ -269,6 +300,7 @@ static void blk_log_writes_close(BlockDriverState *bs) bdrv_unref_child(bs, s->log_file); s->log_file = NULL; bdrv_graph_wrunlock(); + qemu_mutex_destroy(&s->mutex); } static int64_t coroutine_fn GRAPH_RDLOCK @@ -295,7 +327,7 @@ static void blk_log_writes_child_perm(BlockDriverState *bs, BdrvChild *c, static void blk_log_writes_refresh_limits(BlockDriverState *bs, Error **errp) { - BDRVBlkLogWritesState *s = bs->opaque; + const BDRVBlkLogWritesState *s = bs->opaque; bs->bl.request_alignment = s->sectorsize; } @@ -338,15 +370,18 @@ blk_log_writes_co_do_log(BlkLogWritesLogReq *lr) * driver may be modified by other driver operations while waiting for the * I/O to complete. */ + qemu_mutex_lock(&s->mutex); const uint64_t entry_start_sector = s->cur_log_sector; const uint64_t entry_offset = entry_start_sector << s->sectorbits; const uint64_t qiov_aligned_size = ROUND_UP(lr->qiov->size, s->sectorsize); const uint64_t entry_aligned_size = qiov_aligned_size + ROUND_UP(lr->zero_size, s->sectorsize); const uint64_t entry_nr_sectors = entry_aligned_size >> s->sectorbits; + const uint64_t entry_seq = s->nr_entries + 1; - s->nr_entries++; + s->nr_entries = entry_seq; s->cur_log_sector += entry_nr_sectors; + qemu_mutex_unlock(&s->mutex); /* * Write the log entry. Note that if this is a "write zeroes" operation, @@ -366,17 +401,44 @@ blk_log_writes_co_do_log(BlkLogWritesLogReq *lr) /* Update super block on flush or every update interval */ if (lr->log_ret == 0 && ((lr->entry.flags & LOG_FLUSH_FLAG) - || (s->nr_entries % s->update_interval == 0))) + || (entry_seq % s->update_interval == 0))) { struct log_write_super super = { .magic = cpu_to_le64(WRITE_LOG_MAGIC), .version = cpu_to_le64(WRITE_LOG_VERSION), - .nr_entries = cpu_to_le64(s->nr_entries), + .nr_entries = const_le64(0), .sectorsize = cpu_to_le32(s->sectorsize), }; - void *zeroes = g_malloc0(s->sectorsize - sizeof(super)); + void *zeroes; QEMUIOVector qiov; + /* + * Wait if a super block update is already in progress. + * Bail out if a newer update got its turn before us. + */ + WITH_QEMU_LOCK_GUARD(&s->mutex) { + CoQueueWaitFlags wait_flags = 0; + while (s->super_update_seq) { + if (entry_seq < s->super_update_seq) { + return; + } + qemu_co_queue_wait_flags(&s->super_update_queue, + &s->mutex, wait_flags); + + /* + * In case the wait condition remains true after wakeup, + * to avoid starvation, make sure that this request is + * scheduled to rerun next by pushing it to the front of the + * queue. + */ + wait_flags = CO_QUEUE_WAIT_FRONT; + } + s->super_update_seq = entry_seq; + super.nr_entries = cpu_to_le64(s->nr_entries); + } + + zeroes = g_malloc0(s->sectorsize - sizeof(super)); + qemu_iovec_init(&qiov, 2); qemu_iovec_add(&qiov, &super, sizeof(super)); qemu_iovec_add(&qiov, zeroes, s->sectorsize - sizeof(super)); @@ -386,6 +448,13 @@ blk_log_writes_co_do_log(BlkLogWritesLogReq *lr) if (lr->log_ret == 0) { lr->log_ret = bdrv_co_flush(s->log_file->bs); } + + /* The super block has been updated. Let another request have a go. */ + qemu_mutex_lock(&s->mutex); + s->super_update_seq = 0; + (void) qemu_co_queue_next(&s->super_update_queue); + qemu_mutex_unlock(&s->mutex); + qemu_iovec_destroy(&qiov); g_free(zeroes); } @@ -405,7 +474,7 @@ blk_log_writes_co_log(BlockDriverState *bs, uint64_t offset, uint64_t bytes, { QEMUIOVector log_qiov; size_t niov = qiov ? qiov->niov : 0; - BDRVBlkLogWritesState *s = bs->opaque; + const BDRVBlkLogWritesState *s = bs->opaque; BlkLogWritesFileReq fr = { .bs = bs, .offset = offset,