[{"id":3347858,"web_url":"http://patchwork.ozlabs.org/comment/3347858/","msgid":"<87sew726b9.fsf@suse.de>","list_archive_url":null,"date":"2024-07-17T18:59:22","subject":"Re: [PATCH V2 04/11] migration: stop vm earlier for cpr","submitter":{"id":85343,"url":"http://patchwork.ozlabs.org/api/people/85343/","name":"Fabiano Rosas","email":"farosas@suse.de"},"content":"Steve Sistare writes:\n\n> Stop the vm earlier for cpr, to guarantee consistent device state when\n> CPR state is saved.\n>\n> Signed-off-by: Steve Sistare \n> ---\n> migration/migration.c | 22 +++++++++++++---------\n> 1 file changed, 13 insertions(+), 9 deletions(-)\n>\n> diff --git a/migration/migration.c b/migration/migration.c\n> index 0f47765..8a8e927 100644\n> --- a/migration/migration.c\n> +++ b/migration/migration.c\n> @@ -2077,6 +2077,7 @@ void qmp_migrate(const char *uri, bool has_channels,\n> MigrationState *s = migrate_get_current();\n> g_autoptr(MigrationChannel) channel = NULL;\n> MigrationAddress *addr = NULL;\n> + bool stopped = false;\n> \n> /*\n> * Having preliminary checks for uri and channel\n> @@ -2120,6 +2121,15 @@ void qmp_migrate(const char *uri, bool has_channels,\n> }\n> }\n> \n> + if (migrate_mode_is_cpr(s)) {\n> + int ret = migration_stop_vm(s, RUN_STATE_FINISH_MIGRATE);\n> + if (ret < 0) {\n> + error_setg(&local_err, \"migration_stop_vm failed, error %d\", -ret);\n> + goto out;\n> + }\n> + stopped = true;\n> + }\n> +\n> if (cpr_state_save(&local_err)) {\n> goto out;\n> }\n> @@ -2155,6 +2165,9 @@ out:\n> }\n> migrate_fd_error(s, local_err);\n> error_propagate(errp, local_err);\n> + if (stopped && runstate_is_live(s->vm_old_state)) {\n> + vm_start();\n> + }\n\nWhat about non-live states? Shouldn't this be:\n\nif (stopped) {\n vm_resume();\n}\n\n> return;\n> }\n> }\n> @@ -3738,7 +3751,6 @@ void migrate_fd_connect(MigrationState *s, Error *error_in)\n> Error *local_err = NULL;\n> uint64_t rate_limit;\n> bool resume = (s->state == MIGRATION_STATUS_POSTCOPY_RECOVER_SETUP);\n> - int ret;\n> \n> /*\n> * If there's a previous error, free it and prepare for another one.\n> @@ -3810,14 +3822,6 @@ void migrate_fd_connect(MigrationState *s, Error *error_in)\n> return;\n> }\n> \n> - if (migrate_mode_is_cpr(s)) {\n> - ret = migration_stop_vm(s, RUN_STATE_FINISH_MIGRATE);\n> - if (ret < 0) {\n> - error_setg(&local_err, \"migration_stop_vm failed, error %d\", -ret);\n> - goto fail;\n> - }\n> - }\n> -\n> if (migrate_background_snapshot()) {\n> qemu_thread_create(&s->thread, \"mig/snapshot\",\n> bg_migration_thread, s, QEMU_THREAD_JOINABLE);","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256\n header.s=susede2_rsa header.b=Aggg5Lia;\n\tdkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256\n header.s=susede2_ed25519 header.b=wB4K4x5R;\n\tdkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de\n header.a=rsa-sha256 header.s=susede2_rsa header.b=Aggg5Lia;\n\tdkim=neutral header.d=suse.de header.i=@suse.de header.a=ed25519-sha256\n header.s=susede2_ed25519 header.b=wB4K4x5R;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)","smtp-out2.suse.de;\n\tnone"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4WPQHZ28hpz20B2\n\tfor ; Thu, 18 Jul 2024 05:00:22 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1sU9sB-0002kW-FW; Wed, 17 Jul 2024 14:59:31 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1sU9sA-0002ju-1j\n for qemu-devel@nongnu.org; Wed, 17 Jul 2024 14:59:30 -0400","from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)\n (Exim 4.90_1) (envelope-from ) id 1sU9s8-00079e-Cc\n for qemu-devel@nongnu.org; Wed, 17 Jul 2024 14:59:29 -0400","from imap1.dmz-prg2.suse.org (unknown [10.150.64.97])\n (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest\n SHA256)\n (No client certificate requested)\n by smtp-out2.suse.de (Postfix) with ESMTPS id F26DD1FB95;\n Wed, 17 Jul 2024 18:59:25 +0000 (UTC)","from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])\n (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest\n SHA256)\n (No client certificate requested)\n by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 71A1E1368F;\n Wed, 17 Jul 2024 18:59:25 +0000 (UTC)","from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])\n by imap1.dmz-prg2.suse.org with ESMTPSA id sSLSDY0UmGbjUwAAD6G6ig\n (envelope-from ); Wed, 17 Jul 2024 18:59:25 +0000"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de;\n s=susede2_rsa;\n t=1721242766;\n h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:\n mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=ncSEfsOJfx2E+4pcYlKkEddskH3Z2xRKZgUBWUzYmcU=;\n b=Aggg5LiabgxsvIeJVvOilc05VXWWEkrHjNy98XnK6M0vNV/PJXLoMpd3N/9S6+elhtvbLN\n 3v1cy6Rfh8mxBXGwgIkLLCcNto3uiEXxsBeXWsLjLGh4b9kOhebzEk8iaCxmZVRKKICAOW\n Dx/EqZmraii8zTySGYl+lawPIeKz8Ag=","v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de;\n s=susede2_ed25519; t=1721242766;\n h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:\n mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=ncSEfsOJfx2E+4pcYlKkEddskH3Z2xRKZgUBWUzYmcU=;\n b=wB4K4x5Rvs1BKC0QScKXxGzJWUM1LOsDkY/AhQ+TQyllJlv3jL6X9+tkp4sA23oSA3m2bS\n Bpcr+tVxAGcwFMCQ==","v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de;\n s=susede2_rsa;\n t=1721242766;\n h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:\n mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=ncSEfsOJfx2E+4pcYlKkEddskH3Z2xRKZgUBWUzYmcU=;\n b=Aggg5LiabgxsvIeJVvOilc05VXWWEkrHjNy98XnK6M0vNV/PJXLoMpd3N/9S6+elhtvbLN\n 3v1cy6Rfh8mxBXGwgIkLLCcNto3uiEXxsBeXWsLjLGh4b9kOhebzEk8iaCxmZVRKKICAOW\n Dx/EqZmraii8zTySGYl+lawPIeKz8Ag=","v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de;\n s=susede2_ed25519; t=1721242766;\n h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:\n mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=ncSEfsOJfx2E+4pcYlKkEddskH3Z2xRKZgUBWUzYmcU=;\n b=wB4K4x5Rvs1BKC0QScKXxGzJWUM1LOsDkY/AhQ+TQyllJlv3jL6X9+tkp4sA23oSA3m2bS\n Bpcr+tVxAGcwFMCQ=="],"From":"Fabiano Rosas ","To":"Steve Sistare , qemu-devel@nongnu.org","Cc":"Peter Xu , David Hildenbrand ,\n Marcel Apfelbaum , Eduardo Habkost\n , Philippe Mathieu-Daude , Paolo\n Bonzini , \"Daniel P. Berrange\" ,\n Markus Armbruster , Steve Sistare\n ","Subject":"Re: [PATCH V2 04/11] migration: stop vm earlier for cpr","In-Reply-To":"<1719776434-435013-5-git-send-email-steven.sistare@oracle.com>","References":"<1719776434-435013-1-git-send-email-steven.sistare@oracle.com>\n <1719776434-435013-5-git-send-email-steven.sistare@oracle.com>","Date":"Wed, 17 Jul 2024 15:59:22 -0300","Message-ID":"<87sew726b9.fsf@suse.de>","MIME-Version":"1.0","Content-Type":"text/plain","X-Spamd-Result":"default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[99.99%];\n SUSPICIOUS_RECIPS(1.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000];\n NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain];\n RCPT_COUNT_SEVEN(0.00)[11]; RCVD_TLS_ALL(0.00)[];\n ARC_NA(0.00)[]; TAGGED_RCPT(0.00)[];\n RCVD_VIA_SMTP_AUTH(0.00)[]; MISSING_XM_UA(0.00)[];\n MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_ENVRCPT(0.00)[gmail.com];\n DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519];\n FROM_HAS_DN(0.00)[];\n FREEMAIL_CC(0.00)[redhat.com,gmail.com,habkost.net,linaro.org,oracle.com];\n MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[];\n RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[];\n FUZZY_BLOCKED(0.00)[rspamd.com]; TO_DN_SOME(0.00)[]","X-Spam-Score":"-2.80","Received-SPF":"pass client-ip=2a07:de40:b251:101:10:150:64:2;\n envelope-from=farosas@suse.de; helo=smtp-out2.suse.de","X-Spam_score_int":"-20","X-Spam_score":"-2.1","X-Spam_bar":"--","X-Spam_report":"(-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,\n DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,\n SPF_PASS=-0.001 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}},{"id":3349168,"web_url":"http://patchwork.ozlabs.org/comment/3349168/","msgid":"<0d1e8314-cc14-4bd2-8d80-93f6291ada1f@oracle.com>","list_archive_url":null,"date":"2024-07-20T20:00:09","subject":"Re: [PATCH V2 04/11] migration: stop vm earlier for cpr","submitter":{"id":71906,"url":"http://patchwork.ozlabs.org/api/people/71906/","name":"Steve Sistare","email":"steven.sistare@oracle.com"},"content":"On 7/17/2024 2:59 PM, Fabiano Rosas wrote:\n> Steve Sistare writes:\n> \n>> Stop the vm earlier for cpr, to guarantee consistent device state when\n>> CPR state is saved.\n>>\n>> Signed-off-by: Steve Sistare \n>> ---\n>> migration/migration.c | 22 +++++++++++++---------\n>> 1 file changed, 13 insertions(+), 9 deletions(-)\n>>\n>> diff --git a/migration/migration.c b/migration/migration.c\n>> index 0f47765..8a8e927 100644\n>> --- a/migration/migration.c\n>> +++ b/migration/migration.c\n>> @@ -2077,6 +2077,7 @@ void qmp_migrate(const char *uri, bool has_channels,\n>> MigrationState *s = migrate_get_current();\n>> g_autoptr(MigrationChannel) channel = NULL;\n>> MigrationAddress *addr = NULL;\n>> + bool stopped = false;\n>> \n>> /*\n>> * Having preliminary checks for uri and channel\n>> @@ -2120,6 +2121,15 @@ void qmp_migrate(const char *uri, bool has_channels,\n>> }\n>> }\n>> \n>> + if (migrate_mode_is_cpr(s)) {\n>> + int ret = migration_stop_vm(s, RUN_STATE_FINISH_MIGRATE);\n>> + if (ret < 0) {\n>> + error_setg(&local_err, \"migration_stop_vm failed, error %d\", -ret);\n>> + goto out;\n>> + }\n>> + stopped = true;\n>> + }\n>> +\n>> if (cpr_state_save(&local_err)) {\n>> goto out;\n>> }\n>> @@ -2155,6 +2165,9 @@ out:\n>> }\n>> migrate_fd_error(s, local_err);\n>> error_propagate(errp, local_err);\n>> + if (stopped && runstate_is_live(s->vm_old_state)) {\n>> + vm_start();\n>> + }\n> \n> What about non-live states? Shouldn't this be:\n> \n> if (stopped) {\n> vm_resume();\n> }\n\nNot quite. vm_old_state may be a stopped state, so we don't want to resume.\nHowever, I should probably restore the old stopped state here. I'll try some more\nerror recovery scenarios.\n\n- Steve\n\n> \n>> return;\n>> }\n>> }\n>> @@ -3738,7 +3751,6 @@ void migrate_fd_connect(MigrationState *s, Error *error_in)\n>> Error *local_err = NULL;\n>> uint64_t rate_limit;\n>> bool resume = (s->state == MIGRATION_STATUS_POSTCOPY_RECOVER_SETUP);\n>> - int ret;\n>> \n>> /*\n>> * If there's a previous error, free it and prepare for another one.\n>> @@ -3810,14 +3822,6 @@ void migrate_fd_connect(MigrationState *s, Error *error_in)\n>> return;\n>> }\n>> \n>> - if (migrate_mode_is_cpr(s)) {\n>> - ret = migration_stop_vm(s, RUN_STATE_FINISH_MIGRATE);\n>> - if (ret < 0) {\n>> - error_setg(&local_err, \"migration_stop_vm failed, error %d\", -ret);\n>> - goto fail;\n>> - }\n>> - }\n>> -\n>> if (migrate_background_snapshot()) {\n>> qemu_thread_create(&s->thread, \"mig/snapshot\",\n>> bg_migration_thread, s, QEMU_THREAD_JOINABLE);","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256\n header.s=corp-2023-11-20 header.b=lTeuLEgq;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com\n header.a=rsa-sha256 header.s=selector2-oracle-onmicrosoft-com\n header.b=gClUQJZo;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4WRHTk4HD0z1ySl\n\tfor ; Sun, 21 Jul 2024 06:00:42 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1sVGFn-0002I8-L7; Sat, 20 Jul 2024 16:00:27 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1sVGFk-0002H6-PI\n for qemu-devel@nongnu.org; Sat, 20 Jul 2024 16:00:25 -0400","from mx0b-00069f02.pphosted.com ([205.220.177.32])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1sVGFi-00015D-SV\n for qemu-devel@nongnu.org; Sat, 20 Jul 2024 16:00:24 -0400","from pps.filterd (m0246632.ppops.net [127.0.0.1])\n by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id\n 46KJtdLX026468;\n Sat, 20 Jul 2024 20:00:17 GMT","from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com\n (phxpaimrmta01.appoci.oracle.com [138.1.114.2])\n by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 40gktmg030-1\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);\n Sat, 20 Jul 2024 20:00:17 +0000 (GMT)","from pps.filterd\n (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1])\n by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19)\n with ESMTP id 46KIFV2c024447; Sat, 20 Jul 2024 20:00:16 GMT","from nam02-sn1-obe.outbound.protection.outlook.com\n (mail-sn1nam02lp2041.outbound.protection.outlook.com [104.47.57.41])\n by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id\n 40g3p5y91v-1\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);\n Sat, 20 Jul 2024 20:00:16 +0000","from IA1PR10MB7447.namprd10.prod.outlook.com (2603:10b6:208:44c::10)\n by PH0PR10MB4789.namprd10.prod.outlook.com (2603:10b6:510:3c::7) with\n Microsoft SMTP Server (version=TLS1_2,\n cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7784.17; Sat, 20 Jul\n 2024 20:00:13 +0000","from IA1PR10MB7447.namprd10.prod.outlook.com\n ([fe80::f2fe:d6c6:70c4:4572]) by IA1PR10MB7447.namprd10.prod.outlook.com\n ([fe80::f2fe:d6c6:70c4:4572%6]) with mapi id 15.20.7784.016; Sat, 20 Jul 2024\n 20:00:13 +0000"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=\n message-id:date:subject:to:cc:references:from:in-reply-to\n :content-type:content-transfer-encoding:mime-version; s=\n corp-2023-11-20; bh=n7AX+wiNx2T88kty9VOeN4df03iLR8GfurWksr5LyK4=; b=\n lTeuLEgqg0UIBf2nJFq0fNU7wV3T4EZk3e/HNBH1kapKorFypnv25Yg6AZ6qhngM\n w7GjtqcGoAdOeT/FX5SCScJ2iRSXvUIiRsTc11FZEgVa20QytQy06fAh1zoAURoE\n V85XtPdmzEuS9f/IxC27H3/pHuNY9qYaf/UTMD1xvuZ7bWwrv+5UVd5Oa2WciQ3e\n ud6DhJ8+/9ZRj2xsmKp7C+N/cjOb7OU1SvOjtkgFgaqj4nfAg/jHlhCM97w+opqj\n JroDCuBr13Iny1GrW7Q2vnusc2NYvGfL8d3VzX5T7PLe92DjhxF3yDKNHd7GFSXP\n FJyGkhX79IkES0NNJtEjLQ==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;\n bh=n7AX+wiNx2T88kty9VOeN4df03iLR8GfurWksr5LyK4=;\n b=gClUQJZoTWPw5ngvoFXOXNqkWvCOAwygzwStV+b0E1JKG9+HMrxVuZvy2tbfQZL1kDqsBhhgVgK/xUBnr3Pp1ZKqwqWBtZ5cbFknupheoR+FWdwCOdPT2JEYUac7dvYw/3zxCuz9csNSZs353rebCXJX4rQbJB6LbRgM/IJyhAs="],"ARC-Seal":"i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none;\n b=BiQAFdUQNzHBiTQ51fkduudwoczjNLG0RnzVH2DZCC0nEaNyALMfOomWbsqe15oBtEaXeMkdNM+QyVOGtVtlHodkPh8XfKe91AvLVEpK0gzzH4bJrSWL/7zrPi/ewqnbhUqwmY6uaPRu/1qKrBJ7SDE7jGzhGH8yvOrg41B2T/4aG3tkxOkUKCcCdnnUxRALPkdYv4l3dkbbtrVFtHnEc8vZCiKC9jTMwasJQ3YSIQGos4OU+VE1Z9x0wl7qJJ7UIIQSxoF0cgWy3s2v7WGnfH4ZTRbtNN6vj1St4akQ0q5akD8xETzFebihL205QoaQmFJtCgpSuxCb++Y95w2XZQ==","ARC-Message-Signature":"i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;\n s=arcselector10001;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;\n bh=n7AX+wiNx2T88kty9VOeN4df03iLR8GfurWksr5LyK4=;\n b=obq91KqJ0JeubGW/z6SWTQPqyXpUXc29Ug3q6Fsd7crPHnox/i0hr+xgBjjwZwLknO4dpEwsh9zWg24qj9xofAVohYtwKRyymyD0x+HUzI2fGRV5nrZGV5Km08coEP9HKIkYqJ1fI6tMQi0w0Vp4b53sHf8JIWZfiQ/AONRZOCjv7iztz+Lhm9UiNIJgbaj8g7mgdDa6Tpvdp0NGw9NXf3W6Oxp5tn9UTqfN2E0GfmGGWXZ1cl4amrNYe+KCShm7QnconwYO5LKWgaALXZr0AECC40aW7YaJi+G+mID3Nda6vcBJ59nHdp37fefkorKt+JY/Y8cK5Woq1zSn34isFQ==","ARC-Authentication-Results":"i=1; mx.microsoft.com 1; spf=pass\n smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;\n dkim=pass header.d=oracle.com; arc=none","Message-ID":"<0d1e8314-cc14-4bd2-8d80-93f6291ada1f@oracle.com>","Date":"Sat, 20 Jul 2024 16:00:09 -0400","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH V2 04/11] migration: stop vm earlier for cpr","To":"Fabiano Rosas , qemu-devel@nongnu.org","Cc":"Peter Xu , David Hildenbrand ,\n Marcel Apfelbaum ,\n Eduardo Habkost ,\n Philippe Mathieu-Daude ,\n Paolo Bonzini , \"Daniel P. Berrange\"\n , Markus Armbruster ","References":"<1719776434-435013-1-git-send-email-steven.sistare@oracle.com>\n <1719776434-435013-5-git-send-email-steven.sistare@oracle.com>\n <87sew726b9.fsf@suse.de>","Content-Language":"en-US","From":"Steven Sistare ","Organization":"Oracle Corporation","In-Reply-To":"<87sew726b9.fsf@suse.de>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","X-ClientProxiedBy":"SJ0PR03CA0230.namprd03.prod.outlook.com\n (2603:10b6:a03:39f::25) To IA1PR10MB7447.namprd10.prod.outlook.com\n (2603:10b6:208:44c::10)","MIME-Version":"1.0","X-MS-PublicTrafficType":"Email","X-MS-TrafficTypeDiagnostic":"IA1PR10MB7447:EE_|PH0PR10MB4789:EE_","X-MS-Office365-Filtering-Correlation-Id":"d63d6e4f-6bf6-4187-7e14-08dca8f69128","X-MS-Exchange-SenderADCheck":"1","X-MS-Exchange-AntiSpam-Relay":"0","X-Microsoft-Antispam":"BCL:0;ARA:13230040|366016|1800799024|7416014|376014;","X-Microsoft-Antispam-Message-Info":"=?utf-8?q?rKv5zXKPlKCZsXudFu2oPmOcRxV79Ku?=\n\t=?utf-8?q?54a24iEh6ewhW0xMRldVR4xo0VZx0xHU4lWM5u1q/e6UKFyvT5Wu+UVx+T0ynOO0B?=\n\t=?utf-8?q?yuoSVfrf7RQ6MkrjFbxI+UYlLAo1otlQ6xryHspYFxYCfLOWs7POQPqm5DJn82TNH?=\n\t=?utf-8?q?tOPxOntS2NNf8t2LM4gzZyacyy5rXkjB3QhKheCvU1dNhAQgdV3Kiv4ysuRcXzYu0?=\n\t=?utf-8?q?crzxSi2iToLkTwX2Atvpr/mHVjx08iKTvSrA9ibu6J4lEwLOw6Ghsrragx7DfYhH0?=\n\t=?utf-8?q?f975wx3rkKQga3MO3y6s98M8XHSqvUP+aJ9e+fJ24jnQdrFFd+730LirKEeMfuS51?=\n\t=?utf-8?q?rOQSZF/ZxR6vIzPrBfyM+aobdx71xUzxMeGE92dsb5keDbg19s1Ft+3RvcD74jMaG?=\n\t=?utf-8?q?izogfl57J8RD/PY4Gd8gDnK5/RLw9Y94jTPLEgkFg7NQuWwuB/fBf0mRzeZlRIDqI?=\n\t=?utf-8?q?NpPF4N9v3Zib60OAAJ5lV24Go0BD8lHbVUKbxQPVNkTJdySJIwN/HiGPllxSzV5nR?=\n\t=?utf-8?q?+NLjayrP1GKQeTOQgv9PQYCXSR9NC58eYhVARD5bY8DU6TALLiHSBq00wdMxK8+oH?=\n\t=?utf-8?q?dQ3EVH9kYFOH0p1mn7oKyhWJEUEghfNsTQqpkIndtwxsoYOLVNdAXQZQHWvl7iP0u?=\n\t=?utf-8?q?QzUfsg+u0V+yGcj2h9Qy9qmu87Wf3airZbz5mQTDfuMDyioxmjem5VuwiqIKAlA9w?=\n\t=?utf-8?q?6+A9AElJE7XZ6We2cmiOvOqvcuuTn9qC0UyOrxHHM4qjlNwbV31gCNe6xHUdZ+0D8?=\n\t=?utf-8?q?5+j5bENlOs6UxZIZ1o3JuSpshXXr1tNAKrd6TAPyWGAd6dyjFUL5ldNW/Wk7bgue5?=\n\t=?utf-8?q?Yw3VkRLnOzSAylFkbuTOOEypOdsPnWBjstAtJmrsVk59ah3cAly4mDwpGG2sNBeZC?=\n\t=?utf-8?q?ay9jbY3jNwWdkMkjfYx8rORfdhF6aeR0eelRT9XgSOyZed/ZXEVgvQL1GXIjLLOOH?=\n\t=?utf-8?q?fn73ZcKpGJG3LqjE2DTRCxoi2ujfH+pWTEC+RaV4NPtiE8TZZpJrqXofk/tQXH0ts?=\n\t=?utf-8?q?lWY6QlSQt7Bs5H6xm1N4aisObXV3r50mQBBa1WfXQHnS87gFYgcmaTqG3ScdFBPsK?=\n\t=?utf-8?q?VRI1QcZY3ZZ+3t2PzhRgTz0tbPW+RnpCeO0xNHvHgQtI5pLXsl93IDpg8+DhDF9gg?=\n\t=?utf-8?q?9s5I3goL6KLJY86uYK5CqMEkcd7Rx02N1E177b8P5OaN+rx99BqbUxNIMBMApbTk3?=\n\t=?utf-8?q?yN/WSMmeDDQmDO1UBto0KMCaD+AJrUk5ow/t2GhM6nZF+w+midcuehRxnsePmoaNR?=\n\t=?utf-8?q?pJc+x0b8Q8oGnKCoZb/bvKwuJwGPPFlA30g=3D=3D?=","X-Forefront-Antispam-Report":"CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;\n IPV:NLI; SFV:NSPM; H:IA1PR10MB7447.namprd10.prod.outlook.com; PTR:; CAT:NONE;\n SFS:(13230040)(366016)(1800799024)(7416014)(376014); DIR:OUT; SFP:1101;","X-MS-Exchange-AntiSpam-MessageData-ChunkCount":"1","X-MS-Exchange-AntiSpam-MessageData-0":"=?utf-8?q?exFnco48IkwbhFthTmPy9O7j7yl5?=\n\t=?utf-8?q?CpadmEELgfSm9gr+yMTArTD1Epeq5Qh1J2VwHYAb+4PO19PqIGz0ZMKCARA0WCOim?=\n\t=?utf-8?q?8MVWA7L+Ya4wQ64UPUfejvPJ9n2KN6xItlsVc9U5KcKxHtIlA1ku3AxIMV5YrfYGO?=\n\t=?utf-8?q?Rj2y8OdSPXBk1C2mMDhqhKOjmsExZG9qkg+N6KFtT3og4Al0/8OIbZf5vgEbUkyxS?=\n\t=?utf-8?q?/LmdIEcsOWoH/Igr1Gw1oH0rEj4+p/ZrIGsiWy0lSslTuvggEw9J4JPLjPz8VAd4q?=\n\t=?utf-8?q?P1eamriFD/GNDSZJvanlNNusd20Uo4OqA11SyD/r7Ka0mZ8j4Q3MQOEKUDC+0s3av?=\n\t=?utf-8?q?LphB3k0QMtg2sG5qLqH8GOaH11pHV0ZVWuOct8VcVpAPFoiOrMwMVcrDuFjib65yM?=\n\t=?utf-8?q?vmNxy6DLhy1Rl+1dkOYiT3A+c1qSGidMeAMa4G9mLAlDxM3UCAsKVNpFbdKZ8YL9e?=\n\t=?utf-8?q?reI+WUx13LWkMjXhV6Owmh6sekXZxLUJfuUPI1Ir+NP0NU79jHXsUbS4YL1ACcJw+?=\n\t=?utf-8?q?kp4t+BbyMHxz1PPR3O6GkDIn/EpuS4HnilatfgkYSJ8tS2Uc0mKkDWbqmINLcK/Hj?=\n\t=?utf-8?q?JbzGscdnD3N381CXTTUilac8xzEW58rgonxnoK018LHdoQhV9c8hOO01CNrI2x6IO?=\n\t=?utf-8?q?z52ANSDExlnhM1J4AMXVFAA2alTR+QIbE0FSvJEhXelcDoJ2dTGThEFcEinlP5HHb?=\n\t=?utf-8?q?B+SygkVQjMF6ZwgM2uMPWhnLoyeW1/IHldd0soydAq54NxCmRIW29FW9F0Enoc/kC?=\n\t=?utf-8?q?CsEiyz0LN4XnMo4kNIXAVqeTXfLtSxE5jSkKR60g6sa91nFwILqX2f0mjfKE80Tsb?=\n\t=?utf-8?q?irGisq62+wrXk9cpBYI6v6oiZqE0DqualdCfn4efxm8H9DKMvHC+b81tZKmkJVs/l?=\n\t=?utf-8?q?0xhfYG1wZuAbTwVDv0oGBMJGeKoT151gS4aYS203S3HiBLH+zE8IC0jjHlo03+IqM?=\n\t=?utf-8?q?TwbDC0uCa2lzjKzD7ojoolt9j17OF1ZAGHoiRs9ijaUaWNn0NmDLu7Xo3V0l4bspJ?=\n\t=?utf-8?q?zIEWCu24BNIVao5yhNPNhQNkoB6YG9O6+/SNPrq5FCWO1G9YpbLRjIdqzKGMTHaTx?=\n\t=?utf-8?q?YWPfz8xj73JER7s05Z7bBMBOfE+1ka+ymL4+2qHjSMOBSwVY+9fAbs7PP/ApGsFgE?=\n\t=?utf-8?q?iMFLsjPNA7JTn5flDAEJ2eAhPM3c0AHqVguRMjZx3h0BQ22S1cq44jK3tkYxWEC93?=\n\t=?utf-8?q?xahyAdv20f//pMuWfPjDEuhYkKiKdcsz3v3I/vU+ukaCRA+40lkrQ36xcJF9L5qd3?=\n\t=?utf-8?q?X+YrZhZTzMiNT2xMBiz/df+lGTF/UZlONYn1mvDX5hU5XXe8YJKyNKxcDC/k6mrQx?=\n\t=?utf-8?q?43WxdXwCg61nY6wzuIpACTPNuIsV6Vy44PX/FuOBuBzmmHSz89ZJgUWa6t3pyrmdN?=\n\t=?utf-8?q?FE9DxWl77hj5RqUNwth++3CpzeMQmDOPEOI7d/XN4DcuLWaO+SUqcuiAwOIn/gYC6?=\n\t=?utf-8?q?qbPYK2uFt/9/doG7664VG+LaxdiYINvVccefEChJjha/UVqlKoB/ElKc54Ufq8oS6?=\n\t=?utf-8?q?jxE7WVYeDAkmJ1QxHik56GtBUvnkgDAqdw=3D=3D?=","X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount":"1","X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0":"\n v7MEWhWPt6svronhRC2rHsB8PhpkpfR2wYKp9bExw8HF6yB590iziKXu+xtoT7zuJ2/T4oGRw4gGEUIOO3X0TnFSrUD6901wAKflxJZKJkH3Wbj7AT2IqJy4txLRYJ9T7e6CEnMStzAj0HdeLVj20XdcnAawhcVWGu5y2sJ3ZJATB972YR6TX+X3qdtfpUAsRjA+7WXZ/PJz+BrCyJ9XJVAZIzgpvIqJoOC79JyDm5Xr0y7giQ874tA0F90pJurEC1bYX46fIX5wO2y5+bgLub8xD1VSxuDxJerFMrrKHGxjNO9ZU7MPHiN2rTJXjZU09L+iChlTBMnh2Ve9HuM6mZ0fk9Hgwjkicori3fQBqgAh7JMTa2HNbg+muELcFbpnYQZtFchPR6RqN1FrICtytvOHNEw0F8NwJNbgHJ2+/bD9OYbKyBaiK1fUihuxhu9X/0kk8Y/cW02Xe3490PlWFFa2whOv3k3N8J8YfIgNuXROuIMddzhMfndWRH51QvfsxLAfyb5VkxK6+hAwbGGD9cep/ldWM5HUrB82EX0AyuY6TUmNhdwIhoBxDysSSki5EIB4j4iiCQ9SQ3+P7JTimSLU97sfh3fTMHdLPyIKlC8=","X-OriginatorOrg":"oracle.com","X-MS-Exchange-CrossTenant-Network-Message-Id":"\n d63d6e4f-6bf6-4187-7e14-08dca8f69128","X-MS-Exchange-CrossTenant-AuthSource":"IA1PR10MB7447.namprd10.prod.outlook.com","X-MS-Exchange-CrossTenant-AuthAs":"Internal","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"20 Jul 2024 20:00:13.1909 (UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"4e2c6054-71cb-48f1-bd6c-3a9705aca71b","X-MS-Exchange-CrossTenant-MailboxType":"HOSTED","X-MS-Exchange-CrossTenant-UserPrincipalName":"\n LDYlueXU77J5Tk/LDO5sIhYeizWkONwCmjjQQ9YZ5S2schy4zkoUjS6f7UiGQTvAgNZZTbe0IXqzVmogio18w4DJACHfrpiXOmBzoJ+ChXw=","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"PH0PR10MB4789","X-Proofpoint-Virus-Version":"vendor=baseguard\n engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16\n definitions=2024-07-20_17,2024-07-18_01,2024-05-17_01","X-Proofpoint-Spam-Details":"rule=notspam policy=default score=0 phishscore=0\n suspectscore=0 bulkscore=0\n mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0\n classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2407110000\n definitions=main-2407200146","X-Proofpoint-GUID":"AOB1xseEvfZj6se1CwsJXGUrqJtiQuA-","X-Proofpoint-ORIG-GUID":"AOB1xseEvfZj6se1CwsJXGUrqJtiQuA-","Received-SPF":"pass client-ip=205.220.177.32;\n envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com","X-Spam_score_int":"-27","X-Spam_score":"-2.8","X-Spam_bar":"--","X-Spam_report":"(-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,\n DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001,\n SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}},{"id":3349646,"web_url":"http://patchwork.ozlabs.org/comment/3349646/","msgid":"<87v80xzgoy.fsf@suse.de>","list_archive_url":null,"date":"2024-07-22T13:42:21","subject":"Re: [PATCH V2 04/11] migration: stop vm earlier for cpr","submitter":{"id":85343,"url":"http://patchwork.ozlabs.org/api/people/85343/","name":"Fabiano Rosas","email":"farosas@suse.de"},"content":"Steven Sistare writes:\n\n> On 7/17/2024 2:59 PM, Fabiano Rosas wrote:\n>> Steve Sistare writes:\n>> \n>>> Stop the vm earlier for cpr, to guarantee consistent device state when\n>>> CPR state is saved.\n>>>\n>>> Signed-off-by: Steve Sistare \n>>> ---\n>>> migration/migration.c | 22 +++++++++++++---------\n>>> 1 file changed, 13 insertions(+), 9 deletions(-)\n>>>\n>>> diff --git a/migration/migration.c b/migration/migration.c\n>>> index 0f47765..8a8e927 100644\n>>> --- a/migration/migration.c\n>>> +++ b/migration/migration.c\n>>> @@ -2077,6 +2077,7 @@ void qmp_migrate(const char *uri, bool has_channels,\n>>> MigrationState *s = migrate_get_current();\n>>> g_autoptr(MigrationChannel) channel = NULL;\n>>> MigrationAddress *addr = NULL;\n>>> + bool stopped = false;\n>>> \n>>> /*\n>>> * Having preliminary checks for uri and channel\n>>> @@ -2120,6 +2121,15 @@ void qmp_migrate(const char *uri, bool has_channels,\n>>> }\n>>> }\n>>> \n>>> + if (migrate_mode_is_cpr(s)) {\n>>> + int ret = migration_stop_vm(s, RUN_STATE_FINISH_MIGRATE);\n>>> + if (ret < 0) {\n>>> + error_setg(&local_err, \"migration_stop_vm failed, error %d\", -ret);\n>>> + goto out;\n>>> + }\n>>> + stopped = true;\n>>> + }\n>>> +\n>>> if (cpr_state_save(&local_err)) {\n>>> goto out;\n>>> }\n>>> @@ -2155,6 +2165,9 @@ out:\n>>> }\n>>> migrate_fd_error(s, local_err);\n>>> error_propagate(errp, local_err);\n>>> + if (stopped && runstate_is_live(s->vm_old_state)) {\n>>> + vm_start();\n>>> + }\n>> \n>> What about non-live states? Shouldn't this be:\n>> \n>> if (stopped) {\n>> vm_resume();\n>> }\n>\n> Not quite. vm_old_state may be a stopped state, so we don't want to resume.\n> However, I should probably restore the old stopped state here. I'll try some more\n> error recovery scenarios.\n\nAIUI vm_resume() does the right thing already:\n\nvoid vm_resume(RunState state)\n{\n if (runstate_is_live(state)) {\n vm_start();\n } else {\n runstate_set(state);\n }\n}\n\n>\n> - Steve\n>\n>> \n>>> return;\n>>> }\n>>> }\n>>> @@ -3738,7 +3751,6 @@ void migrate_fd_connect(MigrationState *s, Error *error_in)\n>>> Error *local_err = NULL;\n>>> uint64_t rate_limit;\n>>> bool resume = (s->state == MIGRATION_STATUS_POSTCOPY_RECOVER_SETUP);\n>>> - int ret;\n>>> \n>>> /*\n>>> * If there's a previous error, free it and prepare for another one.\n>>> @@ -3810,14 +3822,6 @@ void migrate_fd_connect(MigrationState *s, Error *error_in)\n>>> return;\n>>> }\n>>> \n>>> - if (migrate_mode_is_cpr(s)) {\n>>> - ret = migration_stop_vm(s, RUN_STATE_FINISH_MIGRATE);\n>>> - if (ret < 0) {\n>>> - error_setg(&local_err, \"migration_stop_vm failed, error %d\", -ret);\n>>> - goto fail;\n>>> - }\n>>> - }\n>>> -\n>>> if (migrate_background_snapshot()) {\n>>> qemu_thread_create(&s->thread, \"mig/snapshot\",\n>>> bg_migration_thread, s, QEMU_THREAD_JOINABLE);","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256\n header.s=susede2_rsa header.b=UB4yfN3W;\n\tdkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256\n header.s=susede2_ed25519 header.b=JoKv3f+M;\n\tdkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de\n header.a=rsa-sha256 header.s=susede2_rsa header.b=UB4yfN3W;\n\tdkim=neutral header.d=suse.de header.i=@suse.de header.a=ed25519-sha256\n header.s=susede2_ed25519 header.b=JoKv3f+M;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)","smtp-out2.suse.de;\n\tnone"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4WSM0r2LFBz1yYm\n\tfor ; Mon, 22 Jul 2024 23:42:52 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1sVtJB-00069x-9Y; Mon, 22 Jul 2024 09:42:35 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1sVtJ6-000691-EX\n for qemu-devel@nongnu.org; Mon, 22 Jul 2024 09:42:29 -0400","from smtp-out2.suse.de ([195.135.223.131])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)\n (Exim 4.90_1) (envelope-from ) id 1sVtJ4-0004RW-OA\n for qemu-devel@nongnu.org; Mon, 22 Jul 2024 09:42:28 -0400","from imap1.dmz-prg2.suse.org (unknown [10.150.64.97])\n (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest\n SHA256)\n (No client certificate requested)\n by smtp-out2.suse.de (Postfix) with ESMTPS id 115611FB6B;\n Mon, 22 Jul 2024 13:42:24 +0000 (UTC)","from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])\n (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest\n SHA256)\n (No client certificate requested)\n by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 85F9E138A7;\n Mon, 22 Jul 2024 13:42:23 +0000 (UTC)","from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])\n by imap1.dmz-prg2.suse.org with ESMTPSA id ntsoE79hnmbWPAAAD6G6ig\n (envelope-from ); Mon, 22 Jul 2024 13:42:23 +0000"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de;\n s=susede2_rsa;\n t=1721655744;\n h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:\n mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=5RXhkjT7sbCb8cmHoYzvXj9LvAdqjiPHDjQoDQ/87yk=;\n b=UB4yfN3W4KOfoTE7F//oZGHPH9TPd5pyFWjqoCJmObyeslbl2JdAe4W5+xk46nsyVrdqz8\n i1JBZCJFjrdReFzXZ01ta3YR/uc1bCv+B8SOsLci0A068/f4LUKcwbetwgp06cpCf9HZ1c\n ZnJ8DBWcm1IObdvkYWcXxtApx3zSRQk=","v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de;\n s=susede2_ed25519; t=1721655744;\n h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:\n mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=5RXhkjT7sbCb8cmHoYzvXj9LvAdqjiPHDjQoDQ/87yk=;\n b=JoKv3f+MrLQf3KsJt3suWlvPoxJ7VitV8JZq0wny3qs8uNc8hGQ6KVORrdIEKMR/oFhYOS\n K8bLeuLO5d555lBA==","v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de;\n s=susede2_rsa;\n t=1721655744;\n h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:\n mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=5RXhkjT7sbCb8cmHoYzvXj9LvAdqjiPHDjQoDQ/87yk=;\n b=UB4yfN3W4KOfoTE7F//oZGHPH9TPd5pyFWjqoCJmObyeslbl2JdAe4W5+xk46nsyVrdqz8\n i1JBZCJFjrdReFzXZ01ta3YR/uc1bCv+B8SOsLci0A068/f4LUKcwbetwgp06cpCf9HZ1c\n ZnJ8DBWcm1IObdvkYWcXxtApx3zSRQk=","v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de;\n s=susede2_ed25519; t=1721655744;\n h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:\n mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=5RXhkjT7sbCb8cmHoYzvXj9LvAdqjiPHDjQoDQ/87yk=;\n b=JoKv3f+MrLQf3KsJt3suWlvPoxJ7VitV8JZq0wny3qs8uNc8hGQ6KVORrdIEKMR/oFhYOS\n K8bLeuLO5d555lBA=="],"From":"Fabiano Rosas ","To":"Steven Sistare , qemu-devel@nongnu.org","Cc":"Peter Xu , David Hildenbrand ,\n Marcel Apfelbaum , Eduardo Habkost\n , Philippe Mathieu-Daude , Paolo\n Bonzini , \"Daniel P. Berrange\" ,\n Markus Armbruster ","Subject":"Re: [PATCH V2 04/11] migration: stop vm earlier for cpr","In-Reply-To":"<0d1e8314-cc14-4bd2-8d80-93f6291ada1f@oracle.com>","References":"<1719776434-435013-1-git-send-email-steven.sistare@oracle.com>\n <1719776434-435013-5-git-send-email-steven.sistare@oracle.com>\n <87sew726b9.fsf@suse.de> <0d1e8314-cc14-4bd2-8d80-93f6291ada1f@oracle.com>","Date":"Mon, 22 Jul 2024 10:42:21 -0300","Message-ID":"<87v80xzgoy.fsf@suse.de>","MIME-Version":"1.0","Content-Type":"text/plain","X-Spam-Score":"-2.60","X-Spamd-Result":"default: False [-2.60 / 50.00]; BAYES_HAM(-3.00)[100.00%];\n SUSPICIOUS_RECIPS(1.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000];\n MIME_GOOD(-0.10)[text/plain]; MISSING_XM_UA(0.00)[];\n TAGGED_RCPT(0.00)[]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[];\n MIME_TRACE(0.00)[0:+]; RCPT_COUNT_SEVEN(0.00)[10];\n RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[];\n FREEMAIL_ENVRCPT(0.00)[gmail.com];\n DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519];\n FROM_HAS_DN(0.00)[];\n FREEMAIL_CC(0.00)[redhat.com,gmail.com,habkost.net,linaro.org];\n TO_DN_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[];\n RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[];\n FUZZY_BLOCKED(0.00)[rspamd.com];\n DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,\n oracle.com:email]","Received-SPF":"pass client-ip=195.135.223.131; envelope-from=farosas@suse.de;\n helo=smtp-out2.suse.de","X-Spam_score_int":"-43","X-Spam_score":"-4.4","X-Spam_bar":"----","X-Spam_report":"(-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,\n DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001,\n SPF_PASS=-0.001 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}},{"id":3358087,"web_url":"http://patchwork.ozlabs.org/comment/3358087/","msgid":"<205005e6-9442-49f0-9856-619429554a33@oracle.com>","list_archive_url":null,"date":"2024-08-06T20:52:01","subject":"Re: [PATCH V2 04/11] migration: stop vm earlier for cpr","submitter":{"id":71906,"url":"http://patchwork.ozlabs.org/api/people/71906/","name":"Steve Sistare","email":"steven.sistare@oracle.com"},"content":"On 7/22/2024 9:42 AM, Fabiano Rosas wrote:\n> Steven Sistare writes:\n> \n>> On 7/17/2024 2:59 PM, Fabiano Rosas wrote:\n>>> Steve Sistare writes:\n>>>\n>>>> Stop the vm earlier for cpr, to guarantee consistent device state when\n>>>> CPR state is saved.\n>>>>\n>>>> Signed-off-by: Steve Sistare \n>>>> ---\n>>>> migration/migration.c | 22 +++++++++++++---------\n>>>> 1 file changed, 13 insertions(+), 9 deletions(-)\n>>>>\n>>>> diff --git a/migration/migration.c b/migration/migration.c\n>>>> index 0f47765..8a8e927 100644\n>>>> --- a/migration/migration.c\n>>>> +++ b/migration/migration.c\n>>>> @@ -2077,6 +2077,7 @@ void qmp_migrate(const char *uri, bool has_channels,\n>>>> MigrationState *s = migrate_get_current();\n>>>> g_autoptr(MigrationChannel) channel = NULL;\n>>>> MigrationAddress *addr = NULL;\n>>>> + bool stopped = false;\n>>>> \n>>>> /*\n>>>> * Having preliminary checks for uri and channel\n>>>> @@ -2120,6 +2121,15 @@ void qmp_migrate(const char *uri, bool has_channels,\n>>>> }\n>>>> }\n>>>> \n>>>> + if (migrate_mode_is_cpr(s)) {\n>>>> + int ret = migration_stop_vm(s, RUN_STATE_FINISH_MIGRATE);\n>>>> + if (ret < 0) {\n>>>> + error_setg(&local_err, \"migration_stop_vm failed, error %d\", -ret);\n>>>> + goto out;\n>>>> + }\n>>>> + stopped = true;\n>>>> + }\n>>>> +\n>>>> if (cpr_state_save(&local_err)) {\n>>>> goto out;\n>>>> }\n>>>> @@ -2155,6 +2165,9 @@ out:\n>>>> }\n>>>> migrate_fd_error(s, local_err);\n>>>> error_propagate(errp, local_err);\n>>>> + if (stopped && runstate_is_live(s->vm_old_state)) {\n>>>> + vm_start();\n>>>> + }\n>>>\n>>> What about non-live states? Shouldn't this be:\n>>>\n>>> if (stopped) {\n>>> vm_resume();\n>>> }\n>>\n>> Not quite. vm_old_state may be a stopped state, so we don't want to resume.\n>> However, I should probably restore the old stopped state here. I'll try some more\n>> error recovery scenarios.\n> \n> AIUI vm_resume() does the right thing already:\n> \n> void vm_resume(RunState state)\n> {\n> if (runstate_is_live(state)) {\n> vm_start();\n> } else {\n> runstate_set(state);\n> }\n> }\n\nYes, thanks, I do need to set vm_old_state if not live. It should be:\n\nout:\n ...\n if (stopped) {\n vm_resume(s->vm_old_state);\n }\n\n- Steve\n\n>>>> return;\n>>>> }\n>>>> }\n>>>> @@ -3738,7 +3751,6 @@ void migrate_fd_connect(MigrationState *s, Error *error_in)\n>>>> Error *local_err = NULL;\n>>>> uint64_t rate_limit;\n>>>> bool resume = (s->state == MIGRATION_STATUS_POSTCOPY_RECOVER_SETUP);\n>>>> - int ret;\n>>>> \n>>>> /*\n>>>> * If there's a previous error, free it and prepare for another one.\n>>>> @@ -3810,14 +3822,6 @@ void migrate_fd_connect(MigrationState *s, Error *error_in)\n>>>> return;\n>>>> }\n>>>> \n>>>> - if (migrate_mode_is_cpr(s)) {\n>>>> - ret = migration_stop_vm(s, RUN_STATE_FINISH_MIGRATE);\n>>>> - if (ret < 0) {\n>>>> - error_setg(&local_err, \"migration_stop_vm failed, error %d\", -ret);\n>>>> - goto fail;\n>>>> - }\n>>>> - }\n>>>> -\n>>>> if (migrate_background_snapshot()) {\n>>>> qemu_thread_create(&s->thread, \"mig/snapshot\",\n>>>> bg_migration_thread, s, QEMU_THREAD_JOINABLE);","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256\n header.s=corp-2023-11-20 header.b=A1Aoss5x;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com\n header.a=rsa-sha256 header.s=selector2-oracle-onmicrosoft-com\n header.b=Ro7ML2Rj;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4WdlrV4HLHz1ybS\n\tfor ; Wed, 7 Aug 2024 06:53:14 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1sbRAK-0005k3-KK; Tue, 06 Aug 2024 16:52:20 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1sbRAI-0005it-DX\n for qemu-devel@nongnu.org; Tue, 06 Aug 2024 16:52:18 -0400","from mx0b-00069f02.pphosted.com ([205.220.177.32])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1sbRAG-000152-94\n for qemu-devel@nongnu.org; Tue, 06 Aug 2024 16:52:18 -0400","from pps.filterd (m0246630.ppops.net [127.0.0.1])\n by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id\n 476Eb1vj023462;\n Tue, 6 Aug 2024 20:52:10 GMT","from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com\n (phxpaimrmta02.appoci.oracle.com [147.154.114.232])\n by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 40saye69wy-1\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);\n Tue, 06 Aug 2024 20:52:10 +0000 (GMT)","from pps.filterd\n (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1])\n by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19)\n with ESMTP id 476JgJWZ019687; Tue, 6 Aug 2024 20:52:09 GMT","from nam12-mw2-obe.outbound.protection.outlook.com\n (mail-mw2nam12lp2049.outbound.protection.outlook.com [104.47.66.49])\n by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id\n 40sb096f5s-1\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);\n Tue, 06 Aug 2024 20:52:09 +0000","from IA1PR10MB7447.namprd10.prod.outlook.com (2603:10b6:208:44c::10)\n by PH7PR10MB6579.namprd10.prod.outlook.com (2603:10b6:510:206::19)\n with Microsoft SMTP Server (version=TLS1_2,\n cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7828.22; Tue, 6 Aug\n 2024 20:52:06 +0000","from IA1PR10MB7447.namprd10.prod.outlook.com\n ([fe80::f2fe:d6c6:70c4:4572]) by IA1PR10MB7447.namprd10.prod.outlook.com\n ([fe80::f2fe:d6c6:70c4:4572%7]) with mapi id 15.20.7828.023; Tue, 6 Aug 2024\n 20:52:06 +0000"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=\n message-id:date:subject:to:cc:references:from:in-reply-to\n :content-type:content-transfer-encoding:mime-version; s=\n corp-2023-11-20; bh=nIITzeQ9v3qpO6OCGRgTrjOoxBLMfqZh37b5vaujB28=; b=\n A1Aoss5xwDVPyCmO/zPUhQCPugInenEJ2JuCAQRbwUh2M/HbH2EVV8H859UNQA3Z\n HoyuvXNaGGCUIrgC1VMO5iedZnNoOc2UENBiizizDpDK20akzCyXz0sSan7xNr9d\n aky9xlWvBE7UeKOcv99eYVBkFPiwd1rSM/lNkNTb8wQ9mfwtE0+H5fKB+WrWKHBL\n 8x/v9ET1grKpRpujeH3yPmlCVX4NzUvuhWKovYxu1zmqw3bNFnLfzQkWSSuazKaq\n NCkzlq6ChnUuP8SmKaSJR0YwIi2zcQPYuDrLz5xeIucxsQhqxEYw3/GAJgT/2aaS\n vPO3PAnny0RMmq1b4h01vQ==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;\n bh=nIITzeQ9v3qpO6OCGRgTrjOoxBLMfqZh37b5vaujB28=;\n b=Ro7ML2RjUUplp2xqgHbh2a8yJtBMK/OcWB2TNJ9/JbXUTCBy3nisIQoHvnDnsnAT6FXLCUTZudVbS56mlxdc+U82tCzoeC5ooiEK8nhguxUqxteg0c1nR34dZDhW2I5551jaCyXqkxloIihCMgnS1rlX2jnc+T0l5lWYIoXi69Q="],"ARC-Seal":"i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none;\n b=C2YuZO5PnZDJnzVY6VQ24Jk5dNYaYs7d31+WPiRA7R2uZ+D4A/B5PWRuD35mu7Ao/xbENWTLr9NySAf33KVmBVuF9JCVVsnAtXXP0K3gQ9Kx3efrhYWTkF/6fV937VkbDWcOZ3lUEEgWSWXLccZqFt2tAwiwQO1L04sDYY2e3SIdyfoyCgJeUvk2mAI68ROEnoent27z3fPTHH+0r2+erVNuxYpLkTwJ5dO383YxGRyZRGGd4IJk3RuC1XZq740xOa4uy1YVI1fgDcFsWc36OaUacAVPGTmOriHVNslt79qKArQvxlH/Z3YE4lhN+Z/s2oldodnwnKVYDn/Q1QFGeA==","ARC-Message-Signature":"i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;\n s=arcselector10001;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;\n bh=nIITzeQ9v3qpO6OCGRgTrjOoxBLMfqZh37b5vaujB28=;\n b=xaEfK/MtOGn7e/R3eRLAYDs2HFi9Blu7IDUA1/WhcF1GnKSsfEphfZhdE/Zb3y9/LqAvH3BdmzNpm6Ll2OnWu43KHKT5+Q9JNFKXC6/CYw3dpzCrku8kxbcC/hBWedfaiL4NTbuFHNoi+z0KBvHN1J268EBakaxVcFkbhERJJGJlgSxSTi4phEuFwoo0n6gs0DkMMhtkYR/T5qGeEtI++vxoILnpPqJUo4JpXawo/03RpzG4dqOT3p8JiAGeE+mx/7j/Y8ZycVAAwtuS8jFHRkliSIVG/TtbHOx9u1bbTWBfPMtFIwW/pfBqbyeLRoi2rytElugqsV8hvOBiZOYTRw==","ARC-Authentication-Results":"i=1; mx.microsoft.com 1; spf=pass\n smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;\n dkim=pass header.d=oracle.com; arc=none","Message-ID":"<205005e6-9442-49f0-9856-619429554a33@oracle.com>","Date":"Tue, 6 Aug 2024 16:52:01 -0400","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH V2 04/11] migration: stop vm earlier for cpr","To":"Fabiano Rosas , qemu-devel@nongnu.org","Cc":"Peter Xu , David Hildenbrand ,\n Marcel Apfelbaum ,\n Eduardo Habkost ,\n Philippe Mathieu-Daude ,\n Paolo Bonzini , \"Daniel P. Berrange\"\n , Markus Armbruster ","References":"<1719776434-435013-1-git-send-email-steven.sistare@oracle.com>\n <1719776434-435013-5-git-send-email-steven.sistare@oracle.com>\n <87sew726b9.fsf@suse.de> <0d1e8314-cc14-4bd2-8d80-93f6291ada1f@oracle.com>\n <87v80xzgoy.fsf@suse.de>","Content-Language":"en-US","From":"Steven Sistare ","Organization":"Oracle Corporation","In-Reply-To":"<87v80xzgoy.fsf@suse.de>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","X-ClientProxiedBy":"BN9PR03CA0947.namprd03.prod.outlook.com\n (2603:10b6:408:108::22) To IA1PR10MB7447.namprd10.prod.outlook.com\n (2603:10b6:208:44c::10)","MIME-Version":"1.0","X-MS-PublicTrafficType":"Email","X-MS-TrafficTypeDiagnostic":"IA1PR10MB7447:EE_|PH7PR10MB6579:EE_","X-MS-Office365-Filtering-Correlation-Id":"396dc04f-9d3f-40c3-0e95-08dcb659a1ad","X-MS-Exchange-SenderADCheck":"1","X-MS-Exchange-AntiSpam-Relay":"0","X-Microsoft-Antispam":"BCL:0;ARA:13230040|376014|7416014|366016|1800799024;","X-Microsoft-Antispam-Message-Info":"=?utf-8?q?9Yny4BTjkJ7jEvJMq8yv5pA3C03bgcd?=\n\t=?utf-8?q?Y2I6E1k5uk2S1Waq2S1WW2+Q4SuybJSsmhDG6I25E8PPSM4gaA04MFRRe9gHkMtFu?=\n\t=?utf-8?q?vuiuuNB4SmGNWROYqflr47YCToG6nYS4JQoA4EOzLKC/x0zkTa8ZGY/oXmAQ98UR/?=\n\t=?utf-8?q?hkYwaeQ+hY7mFhYruQQ3a8fkXr3YZHrLmYy8wFhRo2ovvtQL+iWnhh9Ehrs1bJQIN?=\n\t=?utf-8?q?noSdPAz6iS70wbxEYY0lohTPWerwORJGZXbSTKa/KviBeNmpNTyoBzUDxUFg7o6Pm?=\n\t=?utf-8?q?CJgrHbunOF9Cof49vMjtldmlM5Au6DEMRSWFuzf9cM43w2tMzZqeKzj7ALewsOe6X?=\n\t=?utf-8?q?d88ZPf25l7hp9T5HMLkhSWVdKAFi2IpWBkGvm5g178bIY04dtLiBjcWBPK+i1VVu3?=\n\t=?utf-8?q?rOvckhRSaY6rp9CIVUhJx1recnJKzVSRP3g0rF57qERmQGWF+nGqL45Px1pehpO3R?=\n\t=?utf-8?q?CvOhuVl3yhy8Ia/B5Bw3AZE13Bmj1BRXcBjy1N/U1ZgQoL2qdQmv24yzlEWSV+J3v?=\n\t=?utf-8?q?NiS+RbWs5dn5kClYbXdCL7D8bHl7Oaens4EM2Jm422JGJth7H7i9KaxzJHfqYJ6eb?=\n\t=?utf-8?q?7jpgTfDSB3EE8beDQ+UwLnOG8Mzcrd33nATLuxtpma9rlBG0dT4Un5PaaMQoppFBU?=\n\t=?utf-8?q?KphuVsMjiqRWoMdREyb2gP64Ikd67OilJknrgZJrdCauomkPhtOcNeE2vOieCstc+?=\n\t=?utf-8?q?nYF4ydOE5JzkP3AYWSuxO67g0S9XTUPPgA7lZkK9w29xOBrV+uZXWiOIrPDL57/7O?=\n\t=?utf-8?q?FXcTZz1+MdgQ3bJuv6K2us2tXTa0Jg76Jt7vrIH/D8/R2xILFSed2o2+pjKIueh0+?=\n\t=?utf-8?q?r1dPpd+uROZgYVt9p2BwXjBjWcW5aBO+ibIYHYV/1ibSUnFcL2umV6RoQVO8RNbQq?=\n\t=?utf-8?q?JMpwLtHFq215805N4fKzE2Y/lnuPahl10Y7muoRh5kcNPujWPQthFBlxslSje4cHN?=\n\t=?utf-8?q?HtcxLKQsukAQbFoNlgmyHJmo7fIwx5pWsLVwaxiq9CULDcS0j34rsHim71Hehgo88?=\n\t=?utf-8?q?bqhnvPt76XoE9oGvTPK/DmE+ciXEhq5jdQFfVMovtBOGik55NlM7w9fHogm1o6Hm6?=\n\t=?utf-8?q?3jEJNq1ZZMbniCadDECpQDVKjYFD2FCYj0X4egI3NwxAGRnGUUViCy1HFzlijhThv?=\n\t=?utf-8?q?glgycvTQyR/QAZnlETgROnu0eoZ/k/uuUyAov8Rmi9wIKAORt11EIyWnU7jxTtnJt?=\n\t=?utf-8?q?y8AjqVLxkAKDYmvLkVAVMDLa0WeQebudCxDC9akV9JdzO4jxFIWhGS1Hyr6ISJ89D?=\n\t=?utf-8?q?YkjhnZSYcqfvWnQaMwCQu91KpLirNyKBx3w=3D=3D?=","X-Forefront-Antispam-Report":"CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;\n IPV:NLI; SFV:NSPM; H:IA1PR10MB7447.namprd10.prod.outlook.com; PTR:; CAT:NONE;\n SFS:(13230040)(376014)(7416014)(366016)(1800799024); DIR:OUT; SFP:1101;","X-MS-Exchange-AntiSpam-MessageData-ChunkCount":"1","X-MS-Exchange-AntiSpam-MessageData-0":"=?utf-8?q?kHGuLpcmGkqnRkRUyZQ3t+lB+BNz?=\n\t=?utf-8?q?E2I+b+AZoQ6xNsoGO3zhkcsaHl/OaxlL+CydnAYGpMy/mdNDzqwMnGzm77JNozYJV?=\n\t=?utf-8?q?ALCiaA8llmhbHuySSymqVl2RD8j79dvmaq9xcj+Dyi887B3WDeMl9R6+XJd0XfgCe?=\n\t=?utf-8?q?n3jQBxfeStptnZt14BrHgGaPAVjKswobcbYAo87vaZnG9N9Y/2Z15Ki0Pfg4ufewq?=\n\t=?utf-8?q?u/n6w08iAVCMTUkwz7nZ6p/Oxv9I3qzmjMGu0ziaF8JWLwEImijAII66muCh6srHt?=\n\t=?utf-8?q?voZsXdoop2SCRIgMfNsn1xC+iqLU9gzMFMRXbGRSoN/aLphIBN+aEti7HbKS3aPt1?=\n\t=?utf-8?q?YEvNaD9/Pvk9N3k01IdAFfZcnWRJPQLtiJbnWHVz7jQY3SA7ltGHGtSES6HSx1P4a?=\n\t=?utf-8?q?HuCrhbsk4N79r/0WkxFmuomsLOWdfxvPN3L812T0X+F3/ENqyKFtha+9lm6nj1QbU?=\n\t=?utf-8?q?J8vC2imOBdTAp2FlZdMrbM8coGRuaeVBWZPKkb2YgonjOKWc3EDTeACfYTExGTbn2?=\n\t=?utf-8?q?LwsJPGEi2j2Fc9JCXf4rJNiJDdqfLZJ1ckfc+PILfFGyOvRBkH6bxz+qjLvaZCysH?=\n\t=?utf-8?q?KCV/5nyhwXarJJdDTYPpOaP0J/pW0ot1tfuhCvc2akfmY32yb4yY3ENi68tUAiqF5?=\n\t=?utf-8?q?Ttb8ovLyD5rbiaBkPZHxfPJpeg67rouKSnqigOppOOgzf4OgI+y3FkkpkH/y0TBd6?=\n\t=?utf-8?q?3igay5+w842I/IUnC+zb/7cQTQKInRgSR7Js6ag2zMQMus3tH0nE+Qki1ET0Qkb4P?=\n\t=?utf-8?q?uWplli2DJY/dUhxTPZf9vf8cdjnKjQlZgzIhNYF3Sx/7RAFGddwZLpyyC+cBzojvZ?=\n\t=?utf-8?q?lF+YhHu+xx8FStg2TcaA5/yVKJtU0MvV1exe38IjFw9S8zY8bE+NhmrnSR+6VV9+S?=\n\t=?utf-8?q?n7FyGpTxG6Vv3z1xYDK3eTe5BkLTs5HFzwMKH7QvloQnzhukrr5wpWgDaaaet5U4D?=\n\t=?utf-8?q?MXXKW64Ghz2Vmp4fGLgTqv7QewoZc0Tfvw++SLNoY4jwTwv7zkZnUZHcWr+KvI/Dq?=\n\t=?utf-8?q?aKkuDIHH3+YVQiLy7yuoL3x5oQ0BAnpcSjPyG6LT1WNO3DEL8CBA7wUUAIDN1i2u2?=\n\t=?utf-8?q?ZK+HJhAnmgqyTHLFuvK/Uw2FTrzEGTemloN1xdV32E3XIcN3m8b9sbAAH07CcukMJ?=\n\t=?utf-8?q?InYTwX8TKG/7VjnbwjSsztQ2iVi2g5CenHWC5ycy0VL2SlZ+oWgDSW0XJNRQrGORx?=\n\t=?utf-8?q?5OK49ofO/TEMGxal61N/0MlSzf/Bn7uwPXLJqa8WAa/a1t1kRCP91PqV9oNSO5OaO?=\n\t=?utf-8?q?Xrki5LcsDGezuXWHSHVa2RaZCE6sbR/ahwV6Pp/ctCUoO7UXz/0eZ3VX8a1fOYUha?=\n\t=?utf-8?q?v6y/xjlPapeGRpOwRVXykFF7keOh8pS990P53XH/dVuFuIrZhBBLfzASa1517daEx?=\n\t=?utf-8?q?enDb4kRmGK5Oiq3v/7gnegespNy3jRh1m9RUy9fKUxe7q0FoW3s1FFi3rIFgQ5NSe?=\n\t=?utf-8?q?LMDH0WYWiJj7jKDlnXZx3zNujzXQXUGBiQtWnSBvCVjgaHp98rZIJ24Y0ugAvAs9f?=\n\t=?utf-8?q?9rJzXbQ5K++g986fqbmy1IcvSJkQLgZJ8Q=3D=3D?=","X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount":"1","X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0":"\n /qG5NT42OzjNrfMAZ7WoQ30hm8ay6MkZhQa0fpNv/uopdXJbu0+bUK0BH2vc5vRhqVrLVmqwB97aF2zZmobCrD9swNLxN5WCKoiSxyhXzeCWwWrqOxFxphCNN6DKak9Kn6wJ9tjrYSLYA6qgCfrzSsF26skn9F0lnUaa3mfX4X+wVp6zxqR6B/212ZsmAy+MP2R6Vq+Tx2IZCyoy1U8MzEJVRXA3rOOMpct53fUs/L7L9JSWajlHzdi/SVpXpjO46PV4jt69gtfecURNs7y0P/UjB7dV0TFhIxZrgDx6DYwFROYx3BHTGdOkDv9Zy9l3U1379fQkdpMjKt1RPcmjTZjx90QuIORBtzBVyIVxXJvU2FrWD8JbxzJfjlo+/79+6r4a6y7A7AM45eVxvFTVrrPlGd5BgOhejkhmj75dT1VBq51TfoaeM1PfLDAsRgdI7CnmyYH6ZKm6zlUElrgbNrSOlvgYTm/7xcENs+sSSDZIBV6bOdWjeLbz6LSpL20UbBRBl314nYWs8H18Y/1gjvkyK6ogtjS0zgrMJotHKoQBFHCLGAaJcU5O+2fTg3Fk6+WU8Uu59v0S46RFT9kHKiARx0bYCEldHEIIGOxrerw=","X-OriginatorOrg":"oracle.com","X-MS-Exchange-CrossTenant-Network-Message-Id":"\n 396dc04f-9d3f-40c3-0e95-08dcb659a1ad","X-MS-Exchange-CrossTenant-AuthSource":"IA1PR10MB7447.namprd10.prod.outlook.com","X-MS-Exchange-CrossTenant-AuthAs":"Internal","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"06 Aug 2024 20:52:06.2269 (UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"4e2c6054-71cb-48f1-bd6c-3a9705aca71b","X-MS-Exchange-CrossTenant-MailboxType":"HOSTED","X-MS-Exchange-CrossTenant-UserPrincipalName":"\n VNUwk/0E3vxHHytpdB0y08HgyX/a8my1HGOj5y1M6Jr4RuRH/qtMO9PBgZ5Pk8KezbomAnXjcoyQleKyH2eJhpr/TpIjMca/g58QRthLd40=","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"PH7PR10MB6579","X-Proofpoint-Virus-Version":"vendor=baseguard\n engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16\n definitions=2024-08-06_17,2024-08-06_01,2024-05-17_01","X-Proofpoint-Spam-Details":"rule=notspam policy=default score=0 spamscore=0\n adultscore=0\n mlxlogscore=999 phishscore=0 bulkscore=0 mlxscore=0 malwarescore=0\n suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1\n engine=8.12.0-2407110000 definitions=main-2408060145","X-Proofpoint-ORIG-GUID":"S9BtYLNqVgJAWvzf4uUleBkFs7xJOo7E","X-Proofpoint-GUID":"S9BtYLNqVgJAWvzf4uUleBkFs7xJOo7E","Received-SPF":"pass client-ip=205.220.177.32;\n envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com","X-Spam_score_int":"-27","X-Spam_score":"-2.8","X-Spam_bar":"--","X-Spam_report":"(-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,\n DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001,\n RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001,\n RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001,\n SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}}]