[{"id":3669219,"web_url":"http://patchwork.ozlabs.org/comment/3669219/","msgid":"","list_archive_url":null,"date":"2026-03-25T17:07:40","subject":"Re: [RFC PATCH v1 05/17] vmstate: Remove vmdesc_loop","submitter":{"id":67717,"url":"http://patchwork.ozlabs.org/api/people/67717/","name":"Peter Xu","email":"peterx@redhat.com"},"content":"On Tue, Mar 24, 2026 at 04:43:20PM -0300, Fabiano Rosas wrote:\n> The vmdesc_loop variable is being used as a hacky way of writing the\n> JSON description only for the first element of a compressed\n> array. There are also a few implicit uses, such as writing the JSON\n> for the first non-compressed element after a compressed portion.\n> \n> Future work will have trouble with this. Use a simple boolean to be\n> explicit now.\n> \n> Signed-off-by: Fabiano Rosas \n> ---\n> migration/vmstate.c | 25 ++++++++++++++-----------\n> 1 file changed, 14 insertions(+), 11 deletions(-)\n> \n> diff --git a/migration/vmstate.c b/migration/vmstate.c\n> index dec9cf920b..7a12245d36 100644\n> --- a/migration/vmstate.c\n> +++ b/migration/vmstate.c\n> @@ -622,8 +622,9 @@ static bool vmstate_save_vmsd_v(QEMUFile *f, const VMStateDescription *vmsd,\n> void *first_elem = opaque + field->offset;\n> int i, n_elems = vmstate_n_elems(opaque, field);\n> int size = vmstate_size(opaque, field);\n> - JSONWriter *vmdesc_loop = vmdesc;\n> bool is_null_prev = false;\n> + bool use_vmdesc = true;\n> +\n> /*\n> * When this is enabled, it means we will always push a ptr\n> * marker first for each element saying if it's populated.\n> @@ -673,7 +674,7 @@ static bool vmstate_save_vmsd_v(QEMUFile *f, const VMStateDescription *vmsd,\n> is_null != is_null_prev) {\n> \n> is_null_prev = is_null;\n> - vmdesc_loop = vmdesc;\n> + use_vmdesc = true;\n> \n> for (int j = i + 1; j < n_elems; j++) {\n> void *elem = *(void **)(first_elem + size * j);\n> @@ -686,8 +687,13 @@ static bool vmstate_save_vmsd_v(QEMUFile *f, const VMStateDescription *vmsd,\n> }\n> }\n> \n> + if (use_dynamic_array) {\n> + use_vmdesc = true;\n> + }\n\n[1]\n\n> +\n> ok = vmstate_save_field_with_vmdesc(f, curr_elem, size, vmsd,\n> - inner_field, vmdesc_loop,\n> + inner_field,\n> + use_vmdesc ? vmdesc : NULL,\n> i, max_elems, errp);\n> \n> /* If we used a fake temp field.. free it now */\n> @@ -708,19 +714,16 @@ static bool vmstate_save_vmsd_v(QEMUFile *f, const VMStateDescription *vmsd,\n> * NOTE: do not use vmstate_size() here because we want\n> * to dump the real VMSD object now.\n> */\n> - ok = vmstate_save_field_with_vmdesc(f, curr_elem,\n> - field->size, vmsd,\n> - field, vmdesc_loop,\n> - i, max_elems, errp);\n> + ok = vmstate_save_field_with_vmdesc(\n> + f, curr_elem, field->size, vmsd, field,\n> + use_vmdesc ? vmdesc : NULL, i, max_elems, errp);\n> +\n> if (!ok) {\n> goto out;\n> }\n> }\n> \n> - /* Compressed arrays only care about the first element */\n> - if (vmdesc_loop && vmsd_can_compress(field)) {\n> - vmdesc_loop = NULL;\n> - }\n> + use_vmdesc = false;\n\nHmm...\n\nvmsd_can_compress() can return arbitrary things depending on the field\nitself, now we always do the dedup almost for everything.\n\nThe patch did add above [1] to make AUTO_ALLOC be ok, by enforce setting\nTRUE for it for every loop, but what about other cases where\nvmsd_can_compress() may return false (where the field does not support\ncompression)?\n\n> }\n> } else {\n> if (field->flags & VMS_MUST_EXIST) {\n> -- \n> 2.51.0\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=mimecast20190719 header.b=hM0sNjz0;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=google header.b=AJmRYZwG;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fgtf72msBz1xy1\n\tfor ; Thu, 26 Mar 2026 04:08:31 +1100 (AEDT)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1w5Rhu-0000wD-Og; Wed, 25 Mar 2026 13:07:50 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1w5Rht-0000vl-0X\n for qemu-devel@nongnu.org; Wed, 25 Mar 2026 13:07:49 -0400","from us-smtp-delivery-124.mimecast.com ([170.10.133.124])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1w5Rhr-0007RA-DD\n for qemu-devel@nongnu.org; Wed, 25 Mar 2026 13:07:48 -0400","from mail-qt1-f200.google.com (mail-qt1-f200.google.com\n [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS\n (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n us-mta-480-uhbpv1x5OU6QXy62RAWsqg-1; Wed, 25 Mar 2026 13:07:43 -0400","by mail-qt1-f200.google.com with SMTP id\n d75a77b69052e-5091327215dso255831cf.1\n for ; Wed, 25 Mar 2026 10:07:43 -0700 (PDT)","from x1.local ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id\n d75a77b69052e-50b9239fd9bsm2893441cf.22.2026.03.25.10.07.40\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Wed, 25 Mar 2026 10:07:41 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n s=mimecast20190719; t=1774458465;\n h=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n to:to:cc:cc:mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=N2lw7/zZDYpBP7c+P6ld1nNIZwotUhEH9R1hopqB+h8=;\n b=hM0sNjz0McwftcBgHhEe5q0XhHVuj4IcgwyPsIOKUFDSN4SWy1VwiAihIz/PR5G8TpSfdv\n knE9+3APfbGwVCTCr681z8ztUujtKtxGgUIwViMqAMOKlZkvR3ItkgHFSfWsK7raYrZvnE\n zQ6JIzfqLMnSFiOnLym0SnagTQAmHss=","v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=redhat.com; s=google; t=1774458463; x=1775063263; darn=nongnu.org;\n h=in-reply-to:content-disposition:mime-version:references:message-id\n :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;\n bh=N2lw7/zZDYpBP7c+P6ld1nNIZwotUhEH9R1hopqB+h8=;\n b=AJmRYZwGdLH2eRrizoSnzTxO59KBl2wYTvSoWNg6bauC82E4MX4aFVG9jJ8FDDH4fM\n sUs7u//8YIrx5pMg4jba/tdPpMzaLNX89ZFPY753ntzC8XnJGuELejy+e8rDH3Lqmejg\n 0Z3O14kC8vXSQH+Ap+d+T6tN5aetX6a0hRER7DWVMefx9WqjvQy3wtVTYfDqeUisWUEJ\n jfKdxeEMW7Z8zYXF6jbuDxix1Nx1F3hL49IWaE0OZAM8QDTx3Yk1Vq/ukERIcSVd8bEI\n 7XFI+0wJWcH7bmU1OOoPuqDj3l3X6K3kB/JAc8waLqhquNmdz2oQgfZXhXgnVIs7x/xm\n JUNA=="],"X-MC-Unique":"uhbpv1x5OU6QXy62RAWsqg-1","X-Mimecast-MFC-AGG-ID":"uhbpv1x5OU6QXy62RAWsqg_1774458463","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1774458463; x=1775063263;\n h=in-reply-to:content-disposition:mime-version:references:message-id\n :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc\n :subject:date:message-id:reply-to;\n bh=N2lw7/zZDYpBP7c+P6ld1nNIZwotUhEH9R1hopqB+h8=;\n b=hE5nsrvRNzM/5SQdUO/Y6nLSecL1DwHBg/zzKWKQdk/2cXvSjNzepF2JD04Hh8yRil\n JoT96StNKU1agJ2snDgCBK2AwLl1apt0X8mpzEiY+2CKOzmpjs7mpR8cHqc2J1fEsMnd\n z8QhXffTRlzb3LjN//nYWkzPjpXhNvKObtyjYsJSNutYelSOwMZbwC64wF4olYLXnm1u\n yMiB8UrdtJo33mB4zIyTBVSV3+kUKLd0et1q++uEMh+bSbUamVZs06yJmO899VtNXhC6\n SX4eWXT0135/+Vc2POQJmymY6ruJkkbt6cetkEkSggE5pJ/B/1+8UwtAYVLduKqGgIqh\n 4hPw==","X-Gm-Message-State":"AOJu0Yw6wdGjS3Bk345kre2vhZ7KvEgCq+FySGaHQGrSDXMvRAflVeHr\n Hlchj9lUzS1Itu4RAl28uKrKuqLlRj+bUModiNXFrLaKC3AhTjr683w8MmTEtQjCpGATOPRwxoJ\n iw8BGQ+pi4FCE5u3jtFe9XNRKZpNlQVu7e2Q6sGvuy9xCrdjE0ZeGolQL","X-Gm-Gg":"ATEYQzxr7vk+gp1JcCLHstqy7tGs9zhRmQEj64mfZc1m5ZIysksZ9xOT8Vxtig9m88L\n y91ht6lHfhqbXn3CQwp38n6WuulM4jfIrvpcNiJAwuKV7E7NOMZMv9BJOxaoEW9Ptvzylcz6COD\n 4l14UskufX6VvFMIcs756Yyy4b7xK36N4MKt2cDTF8xcKwXWpAhKqRNODROZXAyIFTJQKmI0wxO\n z0iTuSS8UqD09aYqshuToUtHrvSvR0/KBBAmCEVOuLRJQkysBvm5yBfdWinpIoi0MIKh9QuDM+J\n 7PEiAFpBJhshTJdZ+P8UrtauYQwX4bCXHct6g4rYIv+Xfe/hsuhAskk6alQK3fVHtFLAGIJFqyJ\n ba6imdw/RRvIELw==","X-Received":["by 2002:a05:622a:5b95:b0:509:2448:41a with SMTP id\n d75a77b69052e-50b80de0052mr59922011cf.42.1774458462736;\n Wed, 25 Mar 2026 10:07:42 -0700 (PDT)","by 2002:a05:622a:5b95:b0:509:2448:41a with SMTP id\n d75a77b69052e-50b80de0052mr59921411cf.42.1774458462154;\n Wed, 25 Mar 2026 10:07:42 -0700 (PDT)"],"Date":"Wed, 25 Mar 2026 13:07:40 -0400","From":"Peter Xu ","To":"Fabiano Rosas ","Cc":"qemu-devel@nongnu.org, Alexander Mikhalitsyn ,\n Juraj Marcin ","Subject":"Re: [RFC PATCH v1 05/17] vmstate: Remove vmdesc_loop","Message-ID":"","References":"<20260324194333.30004-1-farosas@suse.de>\n <20260324194333.30004-6-farosas@suse.de>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20260324194333.30004-6-farosas@suse.de>","Received-SPF":"pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com;\n helo=us-smtp-delivery-124.mimecast.com","X-Spam_score_int":"-20","X-Spam_score":"-2.1","X-Spam_bar":"--","X-Spam_report":"(-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001,\n DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001,\n RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,\n SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"qemu development ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}},{"id":3669260,"web_url":"http://patchwork.ozlabs.org/comment/3669260/","msgid":"<87pl4rkcdu.fsf@suse.de>","list_archive_url":null,"date":"2026-03-25T18:11:25","subject":"Re: [RFC PATCH v1 05/17] vmstate: Remove vmdesc_loop","submitter":{"id":85343,"url":"http://patchwork.ozlabs.org/api/people/85343/","name":"Fabiano Rosas","email":"farosas@suse.de"},"content":"Peter Xu writes:\n\n> On Tue, Mar 24, 2026 at 04:43:20PM -0300, Fabiano Rosas wrote:\n>> The vmdesc_loop variable is being used as a hacky way of writing the\n>> JSON description only for the first element of a compressed\n>> array. There are also a few implicit uses, such as writing the JSON\n>> for the first non-compressed element after a compressed portion.\n>> \n>> Future work will have trouble with this. Use a simple boolean to be\n>> explicit now.\n>> \n>> Signed-off-by: Fabiano Rosas \n>> ---\n>> migration/vmstate.c | 25 ++++++++++++++-----------\n>> 1 file changed, 14 insertions(+), 11 deletions(-)\n>> \n>> diff --git a/migration/vmstate.c b/migration/vmstate.c\n>> index dec9cf920b..7a12245d36 100644\n>> --- a/migration/vmstate.c\n>> +++ b/migration/vmstate.c\n>> @@ -622,8 +622,9 @@ static bool vmstate_save_vmsd_v(QEMUFile *f, const VMStateDescription *vmsd,\n>> void *first_elem = opaque + field->offset;\n>> int i, n_elems = vmstate_n_elems(opaque, field);\n>> int size = vmstate_size(opaque, field);\n>> - JSONWriter *vmdesc_loop = vmdesc;\n>> bool is_null_prev = false;\n>> + bool use_vmdesc = true;\n>> +\n>> /*\n>> * When this is enabled, it means we will always push a ptr\n>> * marker first for each element saying if it's populated.\n>> @@ -673,7 +674,7 @@ static bool vmstate_save_vmsd_v(QEMUFile *f, const VMStateDescription *vmsd,\n>> is_null != is_null_prev) {\n>> \n>> is_null_prev = is_null;\n>> - vmdesc_loop = vmdesc;\n>> + use_vmdesc = true;\n>> \n>> for (int j = i + 1; j < n_elems; j++) {\n>> void *elem = *(void **)(first_elem + size * j);\n>> @@ -686,8 +687,13 @@ static bool vmstate_save_vmsd_v(QEMUFile *f, const VMStateDescription *vmsd,\n>> }\n>> }\n>> \n>> + if (use_dynamic_array) {\n>> + use_vmdesc = true;\n>> + }\n>\n> [1]\n>\n>> +\n>> ok = vmstate_save_field_with_vmdesc(f, curr_elem, size, vmsd,\n>> - inner_field, vmdesc_loop,\n>> + inner_field,\n>> + use_vmdesc ? vmdesc : NULL,\n>> i, max_elems, errp);\n>> \n>> /* If we used a fake temp field.. free it now */\n>> @@ -708,19 +714,16 @@ static bool vmstate_save_vmsd_v(QEMUFile *f, const VMStateDescription *vmsd,\n>> * NOTE: do not use vmstate_size() here because we want\n>> * to dump the real VMSD object now.\n>> */\n>> - ok = vmstate_save_field_with_vmdesc(f, curr_elem,\n>> - field->size, vmsd,\n>> - field, vmdesc_loop,\n>> - i, max_elems, errp);\n>> + ok = vmstate_save_field_with_vmdesc(\n>> + f, curr_elem, field->size, vmsd, field,\n>> + use_vmdesc ? vmdesc : NULL, i, max_elems, errp);\n>> +\n>> if (!ok) {\n>> goto out;\n>> }\n>> }\n>> \n>> - /* Compressed arrays only care about the first element */\n>> - if (vmdesc_loop && vmsd_can_compress(field)) {\n>> - vmdesc_loop = NULL;\n>> - }\n>> + use_vmdesc = false;\n>\n> Hmm...\n>\n> vmsd_can_compress() can return arbitrary things depending on the field\n> itself, now we always do the dedup almost for everything.\n>\n> The patch did add above [1] to make AUTO_ALLOC be ok, by enforce setting\n> TRUE for it for every loop, but what about other cases where\n> vmsd_can_compress() may return false (where the field does not support\n> compression)?\n>\n\nHmm, I think you're right. But what I'm doing by setting\nuse_vmdesc=false here is just clearing it for the next iteration, where\nit should be set according to necessity. So I'd leave this down here,\nbut up there change to:\n\n if (vmdesc && vmsd_can_compress(field)) {\n if (is_null != is_null_prev) {\n use_vmdesc = true;\n ...\n }\n- }\n-\n- if (use_dynamic_array) {\n+ } else {\n use_vmdesc = true;\n }\n\nIn words: all fields should appear in the JSON, unless its part of the\nNULL portion of a compressed array. For compressed arrays only the first\nelement appears in the JSON. If a stream of NULLs is followed by a\nnon-NULL member, then that member should appear in the JSON after the\ncompressed entry.\n\nThese two catch all these corner-cases I think:\n\nPYTHON=$(pwd)/pyvenv/bin/python3.11\nQTEST_QEMU_BINARY=./qemu-system-s390x ./tests/qtest/migration-test -p\n/s390x/migration/analyze-script\n\nPYTHON=$(pwd)/pyvenv/bin/python3.11\nQTEST_QEMU_BINARY=./qemu-system-ppc64 ./tests/qtest/migration-test-p\n/ppc64/migration/analyze-script\n\n>> }\n>> } else {\n>> if (field->flags & VMS_MUST_EXIST) {\n>> -- \n>> 2.51.0\n>>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256\n header.s=susede2_rsa header.b=0zKnyFke;\n\tdkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256\n header.s=susede2_ed25519 header.b=ALcP77Mg;\n\tdkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de\n header.a=rsa-sha256 header.s=susede2_rsa header.b=0zKnyFke;\n\tdkim=neutral header.d=suse.de header.i=@suse.de header.a=ed25519-sha256\n header.s=susede2_ed25519 header.b=ALcP77Mg;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)","smtp-out2.suse.de;\n dkim=pass header.d=suse.de header.s=susede2_rsa header.b=0zKnyFke;\n dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=ALcP77Mg"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fgw3n62j0z1y1K\n\tfor ; Thu, 26 Mar 2026 05:12:20 +1100 (AEDT)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1w5Shf-00051I-5J; Wed, 25 Mar 2026 14:11:40 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1w5ShZ-00050r-OY\n for qemu-devel@nongnu.org; Wed, 25 Mar 2026 14:11:34 -0400","from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)\n (Exim 4.90_1) (envelope-from ) id 1w5ShX-0000qX-Eb\n for qemu-devel@nongnu.org; Wed, 25 Mar 2026 14:11:33 -0400","from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org\n [IPv6:2a07:de40:b281:104:10:150:64:97])\n (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest\n SHA256)\n (No client certificate requested)\n by smtp-out2.suse.de (Postfix) with ESMTPS id 799885BCF8;\n Wed, 25 Mar 2026 18:11:28 +0000 (UTC)","from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])\n (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest\n SHA256)\n (No client certificate requested)\n by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 04D7644526;\n Wed, 25 Mar 2026 18:11:27 +0000 (UTC)","from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])\n by imap1.dmz-prg2.suse.org with ESMTPSA id zyNJME8lxGmlLwAAD6G6ig\n (envelope-from ); Wed, 25 Mar 2026 18:11:27 +0000"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de;\n s=susede2_rsa;\n t=1774462288;\n h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:\n mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=6Jh3rtxKgzb5uF3D05rl3H5rlxtEEvLJmEe702E6cW0=;\n b=0zKnyFkeLOHNnFnEMSyz1xaKh6ZsSc4REVprZ234zd74422tP7Me/TocYCKVF6rzK9b5vP\n GmjPsKYAZaUMTD/G8xEdRWALWvj2CcmSaUljo/96Kt3BRBFQ3njMsUe14cqyoZGn0V7xY5\n o82pn77Lz6EXyWoRCRBPoABf3GYlG0o=","v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de;\n s=susede2_ed25519; t=1774462288;\n h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:\n mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=6Jh3rtxKgzb5uF3D05rl3H5rlxtEEvLJmEe702E6cW0=;\n b=ALcP77MghL7Btq0O/4pSrcXIGqukjxPndiSXQHsVCb1Ax5n7LWwjyMDGzpNXzMSvTGlM1w\n Ckcp0uJeSqgieQDw==","v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de;\n s=susede2_rsa;\n t=1774462288;\n h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:\n mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=6Jh3rtxKgzb5uF3D05rl3H5rlxtEEvLJmEe702E6cW0=;\n b=0zKnyFkeLOHNnFnEMSyz1xaKh6ZsSc4REVprZ234zd74422tP7Me/TocYCKVF6rzK9b5vP\n GmjPsKYAZaUMTD/G8xEdRWALWvj2CcmSaUljo/96Kt3BRBFQ3njMsUe14cqyoZGn0V7xY5\n o82pn77Lz6EXyWoRCRBPoABf3GYlG0o=","v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de;\n s=susede2_ed25519; t=1774462288;\n h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:\n mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=6Jh3rtxKgzb5uF3D05rl3H5rlxtEEvLJmEe702E6cW0=;\n b=ALcP77MghL7Btq0O/4pSrcXIGqukjxPndiSXQHsVCb1Ax5n7LWwjyMDGzpNXzMSvTGlM1w\n Ckcp0uJeSqgieQDw=="],"From":"Fabiano Rosas ","To":"Peter Xu ","Cc":"qemu-devel@nongnu.org, Alexander Mikhalitsyn ,\n Juraj Marcin ","Subject":"Re: [RFC PATCH v1 05/17] vmstate: Remove vmdesc_loop","In-Reply-To":"","References":"<20260324194333.30004-1-farosas@suse.de>\n <20260324194333.30004-6-farosas@suse.de> ","Date":"Wed, 25 Mar 2026 15:11:25 -0300","Message-ID":"<87pl4rkcdu.fsf@suse.de>","MIME-Version":"1.0","Content-Type":"text/plain","X-Spamd-Result":"default: False [-4.51 / 50.00]; BAYES_HAM(-3.00)[100.00%];\n NEURAL_HAM_LONG(-1.00)[-1.000];\n NEURAL_HAM_SHORT(-0.20)[-1.000];\n R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519];\n MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[];\n FUZZY_RATELIMITED(0.00)[rspamd.com];\n RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+];\n TO_DN_SOME(0.00)[]; ARC_NA(0.00)[]; MISSING_XM_UA(0.00)[];\n RCVD_TLS_ALL(0.00)[]; RCPT_COUNT_THREE(0.00)[4];\n FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[];\n MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2];\n TO_MATCH_ENVRCPT_ALL(0.00)[];\n DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:mid,suse.de:dkim,suse.de:email];\n DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519];\n DKIM_TRACE(0.00)[suse.de:+]","X-Rspamd-Action":"no action","X-Spam-Score":"-4.51","X-Rspamd-Server":"rspamd1.dmz-prg2.suse.org","X-Rspamd-Queue-Id":"799885BCF8","Received-SPF":"pass client-ip=2a07:de40:b251:101:10:150:64:2;\n envelope-from=farosas@suse.de; helo=smtp-out2.suse.de","X-Spam_score_int":"-20","X-Spam_score":"-2.1","X-Spam_bar":"--","X-Spam_report":"(-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,\n DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001,\n SPF_PASS=-0.001 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"qemu development ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}},{"id":3669353,"web_url":"http://patchwork.ozlabs.org/comment/3669353/","msgid":"","list_archive_url":null,"date":"2026-03-25T21:43:38","subject":"Re: [RFC PATCH v1 05/17] vmstate: Remove vmdesc_loop","submitter":{"id":67717,"url":"http://patchwork.ozlabs.org/api/people/67717/","name":"Peter Xu","email":"peterx@redhat.com"},"content":"On Wed, Mar 25, 2026 at 03:11:25PM -0300, Fabiano Rosas wrote:\n> Peter Xu writes:\n> \n> > On Tue, Mar 24, 2026 at 04:43:20PM -0300, Fabiano Rosas wrote:\n> >> The vmdesc_loop variable is being used as a hacky way of writing the\n> >> JSON description only for the first element of a compressed\n> >> array. There are also a few implicit uses, such as writing the JSON\n> >> for the first non-compressed element after a compressed portion.\n> >> \n> >> Future work will have trouble with this. Use a simple boolean to be\n> >> explicit now.\n> >> \n> >> Signed-off-by: Fabiano Rosas \n> >> ---\n> >> migration/vmstate.c | 25 ++++++++++++++-----------\n> >> 1 file changed, 14 insertions(+), 11 deletions(-)\n> >> \n> >> diff --git a/migration/vmstate.c b/migration/vmstate.c\n> >> index dec9cf920b..7a12245d36 100644\n> >> --- a/migration/vmstate.c\n> >> +++ b/migration/vmstate.c\n> >> @@ -622,8 +622,9 @@ static bool vmstate_save_vmsd_v(QEMUFile *f, const VMStateDescription *vmsd,\n> >> void *first_elem = opaque + field->offset;\n> >> int i, n_elems = vmstate_n_elems(opaque, field);\n> >> int size = vmstate_size(opaque, field);\n> >> - JSONWriter *vmdesc_loop = vmdesc;\n> >> bool is_null_prev = false;\n> >> + bool use_vmdesc = true;\n> >> +\n> >> /*\n> >> * When this is enabled, it means we will always push a ptr\n> >> * marker first for each element saying if it's populated.\n> >> @@ -673,7 +674,7 @@ static bool vmstate_save_vmsd_v(QEMUFile *f, const VMStateDescription *vmsd,\n> >> is_null != is_null_prev) {\n> >> \n> >> is_null_prev = is_null;\n> >> - vmdesc_loop = vmdesc;\n> >> + use_vmdesc = true;\n> >> \n> >> for (int j = i + 1; j < n_elems; j++) {\n> >> void *elem = *(void **)(first_elem + size * j);\n> >> @@ -686,8 +687,13 @@ static bool vmstate_save_vmsd_v(QEMUFile *f, const VMStateDescription *vmsd,\n> >> }\n> >> }\n> >> \n> >> + if (use_dynamic_array) {\n> >> + use_vmdesc = true;\n> >> + }\n> >\n> > [1]\n> >\n> >> +\n> >> ok = vmstate_save_field_with_vmdesc(f, curr_elem, size, vmsd,\n> >> - inner_field, vmdesc_loop,\n> >> + inner_field,\n> >> + use_vmdesc ? vmdesc : NULL,\n> >> i, max_elems, errp);\n> >> \n> >> /* If we used a fake temp field.. free it now */\n> >> @@ -708,19 +714,16 @@ static bool vmstate_save_vmsd_v(QEMUFile *f, const VMStateDescription *vmsd,\n> >> * NOTE: do not use vmstate_size() here because we want\n> >> * to dump the real VMSD object now.\n> >> */\n> >> - ok = vmstate_save_field_with_vmdesc(f, curr_elem,\n> >> - field->size, vmsd,\n> >> - field, vmdesc_loop,\n> >> - i, max_elems, errp);\n> >> + ok = vmstate_save_field_with_vmdesc(\n> >> + f, curr_elem, field->size, vmsd, field,\n> >> + use_vmdesc ? vmdesc : NULL, i, max_elems, errp);\n> >> +\n> >> if (!ok) {\n> >> goto out;\n> >> }\n> >> }\n> >> \n> >> - /* Compressed arrays only care about the first element */\n> >> - if (vmdesc_loop && vmsd_can_compress(field)) {\n> >> - vmdesc_loop = NULL;\n> >> - }\n> >> + use_vmdesc = false;\n> >\n> > Hmm...\n> >\n> > vmsd_can_compress() can return arbitrary things depending on the field\n> > itself, now we always do the dedup almost for everything.\n> >\n> > The patch did add above [1] to make AUTO_ALLOC be ok, by enforce setting\n> > TRUE for it for every loop, but what about other cases where\n> > vmsd_can_compress() may return false (where the field does not support\n> > compression)?\n> >\n> \n> Hmm, I think you're right. But what I'm doing by setting\n> use_vmdesc=false here is just clearing it for the next iteration, where\n> it should be set according to necessity. So I'd leave this down here,\n> but up there change to:\n> \n> if (vmdesc && vmsd_can_compress(field)) {\n> if (is_null != is_null_prev) {\n> use_vmdesc = true;\n> ...\n> }\n> - }\n> -\n> - if (use_dynamic_array) {\n> + } else {\n> use_vmdesc = true;\n> }\n\nI think this may still cause some confusion and inefficiency.\n\nFor example, for uncompressed fields, we'll update this field twice for\neach of the elements (setting true here, clearing to false at the end).\n\nThe old approach looks still better in that it only flips the pointer\nwhenever needed. Said that, it doesn't always need to be a pointer, we can\nstill use a bool.\n\n> \n> In words: all fields should appear in the JSON, unless its part of the\n> NULL portion of a compressed array. For compressed arrays only the first\n> element appears in the JSON. If a stream of NULLs is followed by a\n> non-NULL member, then that member should appear in the JSON after the\n> compressed entry.\n> \n> These two catch all these corner-cases I think:\n> \n> PYTHON=$(pwd)/pyvenv/bin/python3.11\n> QTEST_QEMU_BINARY=./qemu-system-s390x ./tests/qtest/migration-test -p\n> /s390x/migration/analyze-script\n> \n> PYTHON=$(pwd)/pyvenv/bin/python3.11\n> QTEST_QEMU_BINARY=./qemu-system-ppc64 ./tests/qtest/migration-test-p\n> /ppc64/migration/analyze-script\n> \n> >> }\n> >> } else {\n> >> if (field->flags & VMS_MUST_EXIST) {\n> >> -- \n> >> 2.51.0\n> >> \n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=mimecast20190719 header.b=hzUVOUAG;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=google header.b=cJSiQpJ1;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fh0m91v64z1y1x\n\tfor ; Thu, 26 Mar 2026 08:44:09 +1100 (AEDT)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1w5W0y-0005x2-V4; Wed, 25 Mar 2026 17:43:49 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1w5W0y-0005wt-1X\n for qemu-devel@nongnu.org; Wed, 25 Mar 2026 17:43:48 -0400","from us-smtp-delivery-124.mimecast.com ([170.10.133.124])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1w5W0w-0001V3-7E\n for qemu-devel@nongnu.org; Wed, 25 Mar 2026 17:43:47 -0400","from mail-qt1-f199.google.com (mail-qt1-f199.google.com\n [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS\n (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n us-mta-612-J6skp1AMMbmGUwcUa3Ls6g-1; Wed, 25 Mar 2026 17:43:41 -0400","by mail-qt1-f199.google.com with SMTP id\n d75a77b69052e-50b2cbe7223so12316141cf.2\n for ; Wed, 25 Mar 2026 14:43:40 -0700 (PDT)","from x1.local ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id\n d75a77b69052e-50b9237c12asm8371601cf.20.2026.03.25.14.43.38\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Wed, 25 Mar 2026 14:43:39 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n s=mimecast20190719; t=1774475024;\n h=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n to:to:cc:cc:mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=n7WQVEhwypZmULuwGej09JGKzbfGrAefaMZ0cKUE20E=;\n b=hzUVOUAGgCciBGTdCO8g0hCNOpR7wIprQIkocHIBpPOnQrgKzKOGpeA1fUYlJMs1jOspYk\n 9Fgban+Z3rbMcWRf27Q+aOztvrkFDyS5E4vK5Qo/jqgDBwSEgzJVAB+IMSMa8FunWQ98Nd\n AVs8m5yzNVqLFzMynkSKzGCbiZytKRE=","v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=redhat.com; s=google; t=1774475020; x=1775079820; darn=nongnu.org;\n h=in-reply-to:content-disposition:mime-version:references:message-id\n :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;\n bh=n7WQVEhwypZmULuwGej09JGKzbfGrAefaMZ0cKUE20E=;\n b=cJSiQpJ1Hug8FEMPyTZ3phZElmG8V1m0XMbdfME/e7ZxcMisL+dOKzBN39vdMqIwsC\n DvO7OSqIc71WzKy05nXQhZeJDVhqc3UN5OfAVuWEOjy1LQ9axVXdlPD7kIXGO20h0uDF\n TjFH8mnumlIv4Bk89mfrqgPtx3Giw/vOBr9Rxl43FC0hXTcMpCYUiczvfvu+Kbmzsp0U\n Ay/0tA9Q8cDGwL5s0KSznFeMh0iT9rPt5QjLLaNP3Ga8KEaj/zib891sE1e9or7qmaos\n bNPvA6sYRfWWV2a21DABo9/z6Eb3BosPhMy+MA+ObChLyxUg9yM65hjvSMXQ+w55nPWV\n n6gQ=="],"X-MC-Unique":"J6skp1AMMbmGUwcUa3Ls6g-1","X-Mimecast-MFC-AGG-ID":"J6skp1AMMbmGUwcUa3Ls6g_1774475020","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1774475020; x=1775079820;\n h=in-reply-to:content-disposition:mime-version:references:message-id\n :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc\n :subject:date:message-id:reply-to;\n bh=n7WQVEhwypZmULuwGej09JGKzbfGrAefaMZ0cKUE20E=;\n b=PTqU8371zGa+yRVWCzm963AkCdea04ac4ELhDFYr9XlHpptirv7P2XbuD7kbV+IdbO\n S4ebpvwwR8a0gOqQqKFB7QZwwMmMLPD72WnrY2Z9cvD1t39+3YzkFInO9biVqZvxS/Mx\n wMB9RcflikyLpypFYYBUHiu/X4a+cWrCOhJj7hwbtYQpPN2tagDtc+IHZ94H/JDiOHhO\n c6wV302wFRKw1eOK83HgtVbEcCfqQlxwVTglxdSrwijL0auv/9VP9HQPdYXTpBO3Ak8A\n bUiVOejQoM6pa5JWkeAv+KesVDV+G6swlo75McidWfXNB/2Vq8Jp+yhmWRS2jPSOcRYQ\n 1xlA==","X-Gm-Message-State":"AOJu0Yy8DgKDXb44A+F1RpF/DQph/d4wVxncDYg+qLZL+sYiITgIjOjn\n YMCk5WiWcTwHCFWsuVAW0NahUD0RHBK5Qf4C/8FzDVjswnr4clqUYVeQ1qsP0f5VE2tuRDhAh+L\n rcBob2LelfHWu3BMeisruNTQv38Mcna3K8ac4MUEp72jqyIuU94CjdudD","X-Gm-Gg":"ATEYQzx7eQLZv2qf9TracqBj8uD0qkHlNWv9mkopTkIweHJnGei5cxm7ip+3ZQPMazX\n 5RzxQ8Cies7xsgsAlfeNtLNg68cuqwtjQZMapFL66Cc4HPBjwG/imGp+ZNZoWgk/HxEMk0hMdXt\n KB4xeIjLPeSTClgHNRch3anuEaGJPtD+6xFQ2DljoRl2rpWDZ+ABYTdzUTM4N5SNjss/oqQ5uJ+\n f2Km9rKZL1M6ruHwPwjyG8xwfUSGl1A5fmSgJcHHLdqDpaMSeMjznR9A7NtKOQzTa9rzQw8vqcE\n 0DutibGAoXQnmNePNTBrHedz5g9jds23QP7UWj/Wa2fNml5Mz3Z2ZuKQZQhghT/bpgmzfKAGxGL\n Cr5jk/EkQj4Px8A==","X-Received":["by 2002:ac8:5a13:0:b0:50b:3e24:f9b8 with SMTP id\n d75a77b69052e-50b80cca3c1mr77891681cf.14.1774475020368;\n Wed, 25 Mar 2026 14:43:40 -0700 (PDT)","by 2002:ac8:5a13:0:b0:50b:3e24:f9b8 with SMTP id\n d75a77b69052e-50b80cca3c1mr77891471cf.14.1774475019841;\n Wed, 25 Mar 2026 14:43:39 -0700 (PDT)"],"Date":"Wed, 25 Mar 2026 17:43:38 -0400","From":"Peter Xu ","To":"Fabiano Rosas ","Cc":"qemu-devel@nongnu.org, Alexander Mikhalitsyn ,\n Juraj Marcin ","Subject":"Re: [RFC PATCH v1 05/17] vmstate: Remove vmdesc_loop","Message-ID":"","References":"<20260324194333.30004-1-farosas@suse.de>\n <20260324194333.30004-6-farosas@suse.de>\n <87pl4rkcdu.fsf@suse.de>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<87pl4rkcdu.fsf@suse.de>","Received-SPF":"pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com;\n helo=us-smtp-delivery-124.mimecast.com","X-Spam_score_int":"-20","X-Spam_score":"-2.1","X-Spam_bar":"--","X-Spam_report":"(-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001,\n DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001,\n RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,\n SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"qemu development ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}}]