[{"id":3348869,"web_url":"http://patchwork.ozlabs.org/comment/3348869/","msgid":"","list_archive_url":null,"date":"2024-07-19T16:28:09","subject":"Re: [PATCH V2 06/11] migration: fix mismatched GPAs during cpr","submitter":{"id":67717,"url":"http://patchwork.ozlabs.org/api/people/67717/","name":"Peter Xu","email":"peterx@redhat.com"},"content":"On Sun, Jun 30, 2024 at 12:40:29PM -0700, Steve Sistare wrote:\n> For new cpr modes, ramblock_is_ignored will always be true, because the\n> memory is preserved in place rather than copied. However, for an ignored\n> block, parse_ramblock currently requires that the received address of the\n> block must match the address of the statically initialized region on the\n> target. This fails for a PCI rom block, because the memory region address\n> is set when the guest writes to a BAR on the source, which does not occur\n> on the target, causing a \"Mismatched GPAs\" error during cpr migration.\n\nIs this a common fix with/without cpr mode?\n\nIt looks to me mr->addr (for these ROMs) should only be set in PCI config\nregion updates as you mentioned. But then I didn't figure out when they're\nupdated on dest in live migration: the ramblock info was sent at the\nbeginning of migration, so it doesn't even have PCI config space migrated;\nI thought the real mr->addr should be in there.\n\nI also failed to understand yet on why the mr->addr check needs to be done\nby ignore-shared only. Some explanation would be greatly helpful around\nthis area..\n\nThanks,","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=mimecast20190719 header.b=DWe8zYly;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4WQZrC6yFMz1xpQ\n\tfor ; Sat, 20 Jul 2024 02:29:14 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1sUqT1-0007xK-Jp; Fri, 19 Jul 2024 12:28:23 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1sUqSz-0007wV-Uf\n for qemu-devel@nongnu.org; Fri, 19 Jul 2024 12:28:21 -0400","from us-smtp-delivery-124.mimecast.com ([170.10.129.124])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1sUqSx-0002ds-JE\n for qemu-devel@nongnu.org; Fri, 19 Jul 2024 12:28:21 -0400","from mail-io1-f70.google.com (mail-io1-f70.google.com\n [209.85.166.70]) by relay.mimecast.com with ESMTP with STARTTLS\n (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n us-mta-343-48LfaK06NwyVdOeoFctUDw-1; Fri, 19 Jul 2024 12:28:12 -0400","by mail-io1-f70.google.com with SMTP id\n ca18e2360f4ac-8064498eb84so29261839f.1\n for ; Fri, 19 Jul 2024 09:28:12 -0700 (PDT)","from x1n (pool-99-254-121-117.cpe.net.cable.rogers.com.\n [99.254.121.117]) by smtp.gmail.com with ESMTPSA id\n 8926c6da1cb9f-4c234407d49sm379734173.179.2024.07.19.09.28.10\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Fri, 19 Jul 2024 09:28:11 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n s=mimecast20190719; t=1721406497;\n h=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n to:to:cc:cc:mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=9UuX42X/yKsA5znO4k3OK1xGFvZoJshl5haW9XBx7iQ=;\n b=DWe8zYlyhtgx+EZ0YRhjO37lURNaOD3aU/luCOrMj5/9mQJvCXfSqxPOmwfqc6TRcNitlN\n VSDSrHj+Ftd36m1X7t2I20lwDwULNt971NLaSl6U5ZPuoMUnOuv2TAO2gr9cMhiEpbwI1C\n 5+sjK3lvZ+9I3Jx+s9un0bXtjzA7J3o=","X-MC-Unique":"48LfaK06NwyVdOeoFctUDw-1","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20230601; t=1721406492; x=1722011292;\n h=in-reply-to:content-disposition:mime-version:references:message-id\n :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date\n :message-id:reply-to;\n bh=9UuX42X/yKsA5znO4k3OK1xGFvZoJshl5haW9XBx7iQ=;\n b=Bq/fL9spgAGRpmnYi5iVVEvdcZmuCP3DDrwMJrC9RnY7jVw+80Pe8hYaFlcB+ykFOm\n ylGnxChN47Svkmf27tcIoZGUBWjpaD/EyIrYYx1ntngUV38vEESEC0XNnyQetmUsjupU\n KoQJm4B38nH4gTNUg+EzIZ6kivQ8OUy3uVOkWecib+1nlbsj5kf+BMd3kSzXP5MyxWmI\n 1NRarOhebOF2ktbgFMabGwHwNpzGyLa4KBebgKG7YAG9ClJbwwo8tWtY1v+4lUbdiyAi\n RdZnZ+RO843657Q50WGTwM8jjCHbcF0ad2FAkhkm+4XnOXjEUmOLnnNM1uQgV9CvTaVF\n zFaQ==","X-Gm-Message-State":"AOJu0YxTbKvcU+q5EJvjtNNVXqgusUSHUo3Uz6nEJlf86DmchFIO+v/k\n xqMbVdKxzpeyjytLjIMujIfrlwWkOEV9yyuXCbJFBCXevJ0Z4pXMbpbi5JFHnfAnf2zhpme1HZ/\n /KfR3L85IYiaJnVJ3pevEL/rLbr8kPd15xKIf9Qt0hJ2XTCGr9Wu7","X-Received":["by 2002:a05:6602:304a:b0:7f9:3fd9:cbb with SMTP id\n ca18e2360f4ac-81aa4bd5cd5mr29559839f.1.1721406491984;\n Fri, 19 Jul 2024 09:28:11 -0700 (PDT)","by 2002:a05:6602:304a:b0:7f9:3fd9:cbb with SMTP id\n ca18e2360f4ac-81aa4bd5cd5mr29558539f.1.1721406491616;\n Fri, 19 Jul 2024 09:28:11 -0700 (PDT)"],"X-Google-Smtp-Source":"\n AGHT+IEkwogKD9mmrsCtsJNwFtA5+x+w7doEbErwVU8cygMPAigT1iyGptwxtC74tQeumw3gTVeGPQ==","Date":"Fri, 19 Jul 2024 12:28:09 -0400","From":"Peter Xu ","To":"Steve Sistare ","Cc":"qemu-devel@nongnu.org, Fabiano Rosas ,\n David Hildenbrand ,\n Marcel Apfelbaum ,\n Eduardo Habkost ,\n Philippe Mathieu-Daude ,\n Paolo Bonzini ,\n \"Daniel P. Berrange\" ,\n Markus Armbruster ","Subject":"Re: [PATCH V2 06/11] migration: fix mismatched GPAs during cpr","Message-ID":"","References":"<1719776434-435013-1-git-send-email-steven.sistare@oracle.com>\n <1719776434-435013-7-git-send-email-steven.sistare@oracle.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<1719776434-435013-7-git-send-email-steven.sistare@oracle.com>","Received-SPF":"pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com;\n helo=us-smtp-delivery-124.mimecast.com","X-Spam_score_int":"-20","X-Spam_score":"-2.1","X-Spam_bar":"--","X-Spam_report":"(-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001,\n DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001,\n SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}},{"id":3349198,"web_url":"http://patchwork.ozlabs.org/comment/3349198/","msgid":"","list_archive_url":null,"date":"2024-07-20T21:28:07","subject":"Re: [PATCH V2 06/11] migration: fix mismatched GPAs during cpr","submitter":{"id":71906,"url":"http://patchwork.ozlabs.org/api/people/71906/","name":"Steve Sistare","email":"steven.sistare@oracle.com"},"content":"On 7/19/2024 12:28 PM, Peter Xu wrote:\n> On Sun, Jun 30, 2024 at 12:40:29PM -0700, Steve Sistare wrote:\n>> For new cpr modes, ramblock_is_ignored will always be true, because the\n>> memory is preserved in place rather than copied. However, for an ignored\n>> block, parse_ramblock currently requires that the received address of the\n>> block must match the address of the statically initialized region on the\n>> target. This fails for a PCI rom block, because the memory region address\n>> is set when the guest writes to a BAR on the source, which does not occur\n>> on the target, causing a \"Mismatched GPAs\" error during cpr migration.\n> \n> Is this a common fix with/without cpr mode?\n\nIt does not occur during normal migration.\n\n> It looks to me mr->addr (for these ROMs) should only be set in PCI config\n> region updates as you mentioned. But then I didn't figure out when they're\n> updated on dest in live migration: the ramblock info was sent at the\n> beginning of migration, so it doesn't even have PCI config space migrated;\n> I thought the real mr->addr should be in there.\n> \n> I also failed to understand yet on why the mr->addr check needs to be done\n> by ignore-shared only. Some explanation would be greatly helpful around\n> this area..\n\nI will continue this thread later and explain more fully.\n\n- Steve","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256\n header.s=corp-2023-11-20 header.b=mjaPYA5F;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com\n header.a=rsa-sha256 header.s=selector2-oracle-onmicrosoft-com\n header.b=xqyNdXbV;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4WRKRN74CBz1xrQ\n\tfor ; Sun, 21 Jul 2024 07:28:48 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1sVHcu-0004HD-EK; Sat, 20 Jul 2024 17:28:24 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1sVHcs-0004G1-CN\n for qemu-devel@nongnu.org; Sat, 20 Jul 2024 17:28:22 -0400","from mx0a-00069f02.pphosted.com ([205.220.165.32])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1sVHcq-0008Qj-PY\n for qemu-devel@nongnu.org; Sat, 20 Jul 2024 17:28:22 -0400","from pps.filterd (m0246629.ppops.net [127.0.0.1])\n by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id\n 46KLGWFR002129;\n Sat, 20 Jul 2024 21:28:16 GMT","from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com\n (iadpaimrmta03.appoci.oracle.com [130.35.103.27])\n by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 40gn06g07x-1\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);\n Sat, 20 Jul 2024 21:28:16 +0000 (GMT)","from pps.filterd\n (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1])\n by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19)\n with ESMTP id 46KI0QRC027478; Sat, 20 Jul 2024 21:28:15 GMT","from nam02-dm3-obe.outbound.protection.outlook.com\n (mail-dm3nam02lp2041.outbound.protection.outlook.com [104.47.56.41])\n by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id\n 40g3p5rcr3-1\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);\n Sat, 20 Jul 2024 21:28:15 +0000","from IA1PR10MB7447.namprd10.prod.outlook.com (2603:10b6:208:44c::10)\n by DM3PR10MB7946.namprd10.prod.outlook.com (2603:10b6:0:4b::7) with\n Microsoft\n SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id\n 15.20.7784.17; Sat, 20 Jul 2024 21:28:13 +0000","from IA1PR10MB7447.namprd10.prod.outlook.com\n ([fe80::f2fe:d6c6:70c4:4572]) by IA1PR10MB7447.namprd10.prod.outlook.com\n ([fe80::f2fe:d6c6:70c4:4572%6]) with mapi id 15.20.7784.016; Sat, 20 Jul 2024\n 21:28:12 +0000"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=\n message-id:date:subject:to:cc:references:from:in-reply-to\n :content-type:content-transfer-encoding:mime-version; s=\n corp-2023-11-20; bh=Mu+wI5vJjgwz0DIzZHhL1IhFiY6sbapqnY8WoDCL7VI=; b=\n mjaPYA5FAsAojHeGDmXMfexrxxDWc0Mc0d7k9H/f1iYpCgptd2U7r8Uw5td2TlwY\n UY9J4V8JL5zMzG+sVO/CJxfkFM5DOo6wqcOo34FugVZ38yr/kuNLuVPCRh50PcN2\n tzL4SGI8H9mqisC8qn8rugqoMbswbn/vYw5VEjpXEuluEn/j+9hTK/uxTfqqCTAE\n ABD1wtdqHKRM74PGmSHMZs7th3X4pN+nqTasCykPExnPt92MQ9ViFJA6UY8N9UCJ\n I7gXgJjbKs0qNJRefafPFsRP3GVIN5mmHEfwdPtETjqtPegb8JH8EHrGfv+wPi+A\n AhOKEBPHJzhVq3JK92OEVw==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;\n bh=Mu+wI5vJjgwz0DIzZHhL1IhFiY6sbapqnY8WoDCL7VI=;\n b=xqyNdXbVg2kYXpUDIdUG/8DI1CfzrIM8q3tO5shGwGosWBKlUiatmvhUzNIRB6rjethyEXS1aGX3RXfnH4l+L48dCiFhpKoOjxEhbuH5og0MxB76seoI85KL41jUjODjNaDj/xAWqcCfQyRpMBPR5LISvqeLyLO+zEjsIZm3d/I="],"ARC-Seal":"i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none;\n b=pmWyKkB6ViVJWPin0U5Z7k1IRdDS8S9LdpE7O3NNCjQHOo1MwFCLpdqgFOh6psm9UWSPN+U4R+pE3ilZ2O5enhaH2tvGb108YU44EHvlLqXC4bWv5pGtpnh6ajL2eY+Bfs9RMwhOYM7NO08mz3YMWTdIbjcD6HnA6o97QkHaYWxBB9EAuy/GRtGGNqsH4CQ3ljQfd9R44wskSg5lXWQmhnabVArLhReiVsK35eFL0LCcE1RcgmYuF4DQUYp3Wb+eL6Xzr+7RPSyKNnRU8dlGnIIcUmdCYKD0jJiuenyYupV+wSj0jK5iZ6LHhl0NHIHGMYjKxHKQxTmwVV3PoJIpsA==","ARC-Message-Signature":"i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;\n s=arcselector10001;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;\n bh=Mu+wI5vJjgwz0DIzZHhL1IhFiY6sbapqnY8WoDCL7VI=;\n b=f6v5LPYUMGiflE9iGEEGXJ+hRm5eSVR/DDSLQaQqDpuRPIr7hbpwPOmmhSR8tARDdPS3HxRMBvgjRa0OstEB8JtoF96M0lFeuw+64dsPv+FQXd52ZI/yfQ0pggtsRlxfIcpkv/6ARHIr5EjTMLfApIv33dUoVCstEvH8oCov8kuL7IFKuiD/4BrI0VfQ7uC/6MVtMvvCUFNGjM6BDRkjgWBmYTCpU6ohrCNvWNUR3BpeX055/9fHQZuYtNAwGI5aC4t3Dr5OG8EDvo6b0G57I+QYhJyhccCmRfeZxijX52Hlnl74l9ghpmwlhha4o8bH4BKdTAKXszh1SJVgAVrBrg==","ARC-Authentication-Results":"i=1; mx.microsoft.com 1; spf=pass\n smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;\n dkim=pass header.d=oracle.com; arc=none","Message-ID":"","Date":"Sat, 20 Jul 2024 17:28:07 -0400","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH V2 06/11] migration: fix mismatched GPAs during cpr","To":"Peter Xu ","Cc":"qemu-devel@nongnu.org, Fabiano Rosas ,\n David Hildenbrand ,\n Marcel Apfelbaum ,\n Eduardo Habkost ,\n Philippe Mathieu-Daude ,\n Paolo Bonzini , \"Daniel P. Berrange\"\n , Markus Armbruster ","References":"<1719776434-435013-1-git-send-email-steven.sistare@oracle.com>\n <1719776434-435013-7-git-send-email-steven.sistare@oracle.com>\n ","Content-Language":"en-US","From":"Steven Sistare ","Organization":"Oracle Corporation","In-Reply-To":"","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","X-ClientProxiedBy":"LO2P265CA0107.GBRP265.PROD.OUTLOOK.COM\n (2603:10a6:600:c::23) To IA1PR10MB7447.namprd10.prod.outlook.com\n (2603:10b6:208:44c::10)","MIME-Version":"1.0","X-MS-PublicTrafficType":"Email","X-MS-TrafficTypeDiagnostic":"IA1PR10MB7447:EE_|DM3PR10MB7946:EE_","X-MS-Office365-Filtering-Correlation-Id":"94ca5e28-3b37-48ef-5807-08dca902dc1c","X-MS-Exchange-SenderADCheck":"1","X-MS-Exchange-AntiSpam-Relay":"0","X-Microsoft-Antispam":"BCL:0;ARA:13230040|366016|1800799024|7416014|376014;","X-Microsoft-Antispam-Message-Info":"=?utf-8?q?hHW4CdWD0TG1DGqDRW7kZGOBmuNaSqZ?=\n\t=?utf-8?q?Fa9rjO+jpwT4PpSl+8VSBCnYFpyFKHD0Ohe8M/rmBnM3NibMDXkIHm0f8hINkO1gn?=\n\t=?utf-8?q?He1mhbn6UyzqzaMI9vk50FVLQBMTdYRnjNHba8BdUT9bX4kUJKwzl+CFZm7ErNcr6?=\n\t=?utf-8?q?YbTizBQiC0QdpKUDE712fKxZ9DfmHdql+Z4R51qWs69qkl6fls3C9CY9Dg8HckH4e?=\n\t=?utf-8?q?ew3oUr/JIzm4Wf7sSIguU+4bxW4ia4tXpfukPNrH4O/B46vVtNVRhzUReuDxKT5ZJ?=\n\t=?utf-8?q?XFWR8J9kcKkiwFs2V8hQC4Cdo7Fl3crmeAZ0+HkYdlzwGyOg58/ThRN7SDiIQbsb7?=\n\t=?utf-8?q?i69UPFfo+HsFzNkTvbL+J4wX99QLsmHFlg5okQAphCRCosrgGuTE9lYn9qg1wGqCU?=\n\t=?utf-8?q?aka9+T2tsKYvDphG0NJSw1wLnOoPgQWzez5hc0a6Yh/99OtLh0NdLUqojDUtHacPb?=\n\t=?utf-8?q?goCVffDg0tshxvnzrngzEw+SsiZHeBAOV8vokEnS5utnu26C4wL8Q9NWx/bys5MPC?=\n\t=?utf-8?q?SFzi8IzQu6uZUn1I8gy8iuE26PKvvIY0o+5XR1H2urLuxpxIrN2XdGgeyNwMuT9Om?=\n\t=?utf-8?q?8iIXWPe6bmtAHI40Dl638ZsJp4KJ8pHQGML3nz4X20u9zD8g8ffngRf5Rh9Ap4pTL?=\n\t=?utf-8?q?lIKoj0a7lBza1L6OpTHzUv9/Zrp2av+FXEsyfUR77vpfiTWg9RNASlC18G0sF7dqK?=\n\t=?utf-8?q?SumFVuJJux7aouZOOnKwsRyQpAik15DXMq6doOvK5eaaUs0R09SNPJ2N58uqRlewa?=\n\t=?utf-8?q?io0pux5FXyPrRGHtI0koRcY3Lq4khlNsNcL+ojpv/Q0KknknPonSvPRfEyoayjUTY?=\n\t=?utf-8?q?MVgvQaWZM3Phk0AHkTSoxpOKJTH+uzYTFuyHLulHVIawHdiT6oNqrS8bL0ArapIey?=\n\t=?utf-8?q?bBmtSZzzDsNjua2b4xmL0n65l2bFJEzRfy/LShL+PunpmSx4Yqr1nKOYrrcG2af6/?=\n\t=?utf-8?q?rvAaAc/xvKHPnm1kVBQSHwKYu0cD6pCAIIbn7ts782tsoGU4CnOU66G2pXxRCsU1p?=\n\t=?utf-8?q?b4mwCAdmYAwP6a7kiEVs3DKasdK2rRbWPtt9SZgGobxwOwPxMQqtbO3RPa1PSFkw/?=\n\t=?utf-8?q?N7sL1/2efIlL2ATnPpj8im9B9K4UCiJXNLrLqwl4YiHhZCT4NOjB1VXJWeZi+JwB2?=\n\t=?utf-8?q?g12bcm+ndhRJtd5yyaMDUn8CHamkZdqlBdK75Oi0UDL+STzCa15U63M08Wbbu5fMJ?=\n\t=?utf-8?q?x9qJHF6ZhZWE06xMDKYvdq2CNwa+OBiCJyXR4Htma3PkzXoB8qGECx5/XguwUxwUA?=\n\t=?utf-8?q?AmlrRtZL6UUTJLklXFUlhQMpd01jD01Z8zw=3D=3D?=","X-Forefront-Antispam-Report":"CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;\n IPV:NLI; SFV:NSPM; H:IA1PR10MB7447.namprd10.prod.outlook.com; PTR:; CAT:NONE;\n SFS:(13230040)(366016)(1800799024)(7416014)(376014); DIR:OUT; SFP:1101;","X-MS-Exchange-AntiSpam-MessageData-ChunkCount":"1","X-MS-Exchange-AntiSpam-MessageData-0":"=?utf-8?q?bbP6N8YeLBN556ab5LxtcOL5ZD02?=\n\t=?utf-8?q?NMpOnT3mUgaiGCDXcYb/YaGqtUvKltK5ba4Xe5M6MqWpL51/dlKOFq+GmzFLsKvTy?=\n\t=?utf-8?q?ow8NbJskivkYmlwiDSv4mo1bn0kByH084DLiaB1D/rba5IttOwSMxaaDwgWc9mgCH?=\n\t=?utf-8?q?9Ym3e17CYObCkbZFpK46CR7m90J7YbhZq2VWhz/pvCJLiBns+TQX3OM5QbCN6uzOI?=\n\t=?utf-8?q?5UhNdl41bMfqnIkJH8tLvAvcGjTK/7u/v1oQp4AljOcl7DchY+5/5wReYNwciRDeF?=\n\t=?utf-8?q?5BvU5NoXwPafoS3Ecx20m36eG24Y8Z7tWAw+aIb0vFmEesTzDnp+9Uv7GKrN7fBtB?=\n\t=?utf-8?q?79RlzIh8QEa8HZ9oXNMBlTa/6xCL8vozCxD8k7qcBIjOt3E8S1rtmrvz+WCI/nBRF?=\n\t=?utf-8?q?YjBGHvoamSpjVHzIumeW5fmMNsDFfc1/gKi+yXiVtFLfZ12S583cSw0JqZ3zMk1Pi?=\n\t=?utf-8?q?gjt99a6Y8Z77FeqSYAxoF9JZWBirte4Qm3q1kci5A7gTs+fOTpQFENdX1SKGJGs9G?=\n\t=?utf-8?q?wFK2gJ/SStDaJ8EB5DRnlVAmm6YtvpZjr0RO1mAGhQDI8yPEZz7hKR7BRnmXWRR4f?=\n\t=?utf-8?q?snifkYZ5L2w1d/fzAmgNPVwxkIm8vzalXzeHV8Lwy9km1Qd9R78CDbX0K5weiqYgh?=\n\t=?utf-8?q?Gg752f0vas2meJhuYz+VDG+dFHmBNJEuDa//gUsNZyy1j6/d0mLeDRz7WrF1kIo1M?=\n\t=?utf-8?q?ufEn6oBISGPDPSa0qrMZjPK1YsnMFZM9dEzoMyKtmRuot6h6tC7eTIMFoqXpCO6zL?=\n\t=?utf-8?q?TA5Qd5NNc2Mb8DnPw9XzdZQUmnC3mTTi6pBtU6ZsNi2he+S6UmT8xfIAk+dkIEl23?=\n\t=?utf-8?q?ZrfSM/8xCYuOsi7wuws9uZ2LxLue8G7aFWIyVmvAFWijIaPsPKR500l5/a+EqYFED?=\n\t=?utf-8?q?N5slGde9KOpY/Hy9rKCcivBNNTEsIcxszk6XOJ5iO5rnQaVM7FdOg0vVNpp9Pxyt5?=\n\t=?utf-8?q?uEVw43+oYokKYWcVUdM0HUXNqyNmvfNl9Du2Aarz75fNGf/Bnv3FTVyEXdV5IJjoF?=\n\t=?utf-8?q?J3K25hkx2wBZD4Vnrz0lfQlfjWGV1GsxLC3As4h9BsxUB5+YqELlvjcqUq6iwA5/9?=\n\t=?utf-8?q?HIl2myjhYlJFP4xkM5LH0e3iz2jZj3QT6DOS/za4wiopyxaqceGD30/d+d/xEyIe+?=\n\t=?utf-8?q?bwbqc42mTV7ffyxtzlDlKf6oLQ6MwHPaJHdocY0yXkp7B9sjLASRo/a2fjqP4F9Cv?=\n\t=?utf-8?q?rqKR2PD3aMMwfRsZvFnTNqdzKFgDyhJykwzprGegH7NImBJ81m8yqVFT1eHm5jWKz?=\n\t=?utf-8?q?bZYUgIamngr8vVWYkC3+WbZDwJIl+NgSZT59R2/GPuQYCe1C6k12R8NyyDXxh9SRj?=\n\t=?utf-8?q?HNsUWAE4091WDlIIeI3QTV6fk18nj7s6Fl5MwXiqb96zWysZke3uRqWbg+rVBHROW?=\n\t=?utf-8?q?1ELRf7wAxG3R4Z7Gn0GnfUmG17F1UWnP0Lkjmdqq8K+rSnuiupp2mICTJIhn2up2J?=\n\t=?utf-8?q?FNsoXz/GiHMhhdAtUBUBXgidILizbkBgjwqOitfzsjYiZZgDeJRYJ17PrTN8MEJIJ?=\n\t=?utf-8?q?+gE+KXcRTqiagHfXv+yv4Kccoqr5yofrBQ=3D=3D?=","X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount":"1","X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0":"\n 21OJ8Kw08Xiz+wSfrispoC5uMuokEd93CJ89rLJCJLeET+Pin7O3XKbSyAit0/YY7ZiLnZ+OmaY+RdHXEH2L8FWHlaILqFCZ87ztf51XZnLAymyu7Q7mK7zS+OH7PmbASRxyhMvuOQswCZ4Vj9KegWWgY55EhW31bue54yL8VDiHDcznP0inmmEwiGLsg9m5nUxSLnjMrB+iXl+0Zd27+NPb41iZBhTXCexRB7t+qOLxI4xdL0Eh4HUeZQw3aE8quWPfL40OZkqqjFxVFN4YJQCX/EAX3VkrtZMekt4OJcNmMrYzNmskfjRJs8b8yN01IlrCofnmmVmfcquqmaFPE5HCXA90NUFkIEYx/ZBzNrDgmfL3TH46J5pI5ov9E1AmKSmpKAS8SxtUGKDoWZIlmZ/X2J0rehl6ZCMqji+FG44G7TnBB7WBbBrRkd4Y4muyyzKa6m98N59kXKpnuOPhnFcBe0hIID6RBirWc09x7OBsgxoNhFhuE1fRuKn2dMgVOOu6SjbIsAM7inYBAaqLG1kR/G6TXzVB7CIDj6kFtOVse44C/jnDnrlSpv9dVF7aiJLheZIZ3w2n/uDHW07S2Zl8XvFDdhg1FCx5nBigUsE=","X-OriginatorOrg":"oracle.com","X-MS-Exchange-CrossTenant-Network-Message-Id":"\n 94ca5e28-3b37-48ef-5807-08dca902dc1c","X-MS-Exchange-CrossTenant-AuthSource":"IA1PR10MB7447.namprd10.prod.outlook.com","X-MS-Exchange-CrossTenant-AuthAs":"Internal","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"20 Jul 2024 21:28:12.9216 (UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"4e2c6054-71cb-48f1-bd6c-3a9705aca71b","X-MS-Exchange-CrossTenant-MailboxType":"HOSTED","X-MS-Exchange-CrossTenant-UserPrincipalName":"\n lgln1FgwLWG9vGhPRCLvsW97EASVevLuK6VzUHDmVzguuOHw5ZgE8AqGplMOWDlY70yROQP/pulvVmy5zBPZfqWjQr8Oq6jcK6y4uav0ELw=","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"DM3PR10MB7946","X-Proofpoint-Virus-Version":"vendor=baseguard\n engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16\n definitions=2024-07-20_19,2024-07-18_01,2024-05-17_01","X-Proofpoint-Spam-Details":"rule=notspam policy=default score=0 mlxscore=0\n bulkscore=0 phishscore=0\n suspectscore=0 adultscore=0 malwarescore=0 spamscore=0 mlxlogscore=999\n classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2407110000\n definitions=main-2407200157","X-Proofpoint-GUID":"6pbHsUFVwa3c5Q2mHd0zCg091Q7Pcr_Y","X-Proofpoint-ORIG-GUID":"6pbHsUFVwa3c5Q2mHd0zCg091Q7Pcr_Y","Received-SPF":"pass client-ip=205.220.165.32;\n envelope-from=steven.sistare@oracle.com; helo=mx0a-00069f02.pphosted.com","X-Spam_score_int":"-20","X-Spam_score":"-2.1","X-Spam_bar":"--","X-Spam_report":"(-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,\n DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001,\n SPF_PASS=-0.001 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}},{"id":3358889,"web_url":"http://patchwork.ozlabs.org/comment/3358889/","msgid":"<571a4f84-693c-43d4-a43a-52a53a1091e1@oracle.com>","list_archive_url":null,"date":"2024-08-07T21:04:26","subject":"Re: [PATCH V2 06/11] migration: fix mismatched GPAs during cpr","submitter":{"id":71906,"url":"http://patchwork.ozlabs.org/api/people/71906/","name":"Steve Sistare","email":"steven.sistare@oracle.com"},"content":"On 7/19/2024 12:28 PM, Peter Xu wrote:\n> On Sun, Jun 30, 2024 at 12:40:29PM -0700, Steve Sistare wrote:\n>> For new cpr modes, ramblock_is_ignored will always be true, because the\n>> memory is preserved in place rather than copied. However, for an ignored\n>> block, parse_ramblock currently requires that the received address of the\n>> block must match the address of the statically initialized region on the\n>> target. This fails for a PCI rom block, because the memory region address\n>> is set when the guest writes to a BAR on the source, which does not occur\n>> on the target, causing a \"Mismatched GPAs\" error during cpr migration.\n> \n> Is this a common fix with/without cpr mode?\n> \n> It looks to me mr->addr (for these ROMs) should only be set in PCI config\n> region updates as you mentioned. But then I didn't figure out when they're\n> updated on dest in live migration: the ramblock info was sent at the\n> beginning of migration, so it doesn't even have PCI config space migrated;\n> I thought the real mr->addr should be in there.\n> \n> I also failed to understand yet on why the mr->addr check needs to be done\n> by ignore-shared only. Some explanation would be greatly helpful around\n> this area..\n\nThe error_report does not bite for normal migration because migrate_ram_is_ignored()\nis false for the problematic blocks, so the block->mr->addr check is not\nperformed. However, mr->addr is never fixed up in this case, which is a\nquiet potential bug, and this patch fixes that with the \"has_addr\" check.\n\nFor cpr-exec, migrate_ram_is_ignored() is true for all blocks,\nbecause we do not copy the contents over the migration stream, we preserve the\nmemory in place. So we fall into the block->mr->addr sanity check and fail\nwith the original code.\n\nI will add this to the commit message.\n\n- Steve","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256\n header.s=corp-2023-11-20 header.b=dozNQGoz;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com\n header.a=rsa-sha256 header.s=selector2-oracle-onmicrosoft-com\n header.b=br56CFpa;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4WfN483dJvz1ydk\n\tfor ; Thu, 8 Aug 2024 07:05:28 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1sbnq8-0003bC-KF; Wed, 07 Aug 2024 17:05:00 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1sbnpz-0003ZP-DR\n for qemu-devel@nongnu.org; Wed, 07 Aug 2024 17:04:53 -0400","from mx0b-00069f02.pphosted.com ([205.220.177.32])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1sbnpx-0003XD-21\n for qemu-devel@nongnu.org; Wed, 07 Aug 2024 17:04:51 -0400","from pps.filterd (m0333520.ppops.net [127.0.0.1])\n by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id\n 477L3W3M021471;\n Wed, 7 Aug 2024 21:04:37 GMT","from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com\n (phxpaimrmta02.appoci.oracle.com [147.154.114.232])\n by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 40sckcghx0-1\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);\n Wed, 07 Aug 2024 21:04:36 +0000 (GMT)","from pps.filterd\n (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1])\n by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19)\n with ESMTP id 477K2kQ0019380; Wed, 7 Aug 2024 21:04:36 GMT","from nam02-sn1-obe.outbound.protection.outlook.com\n (mail-sn1nam02lp2041.outbound.protection.outlook.com [104.47.57.41])\n by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id\n 40sb0ag3qb-1\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);\n Wed, 07 Aug 2024 21:04:35 +0000","from IA1PR10MB7447.namprd10.prod.outlook.com (2603:10b6:208:44c::10)\n by MN2PR10MB4142.namprd10.prod.outlook.com (2603:10b6:208:1d2::15)\n with Microsoft SMTP Server (version=TLS1_2,\n cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.14; Wed, 7 Aug\n 2024 21:04:33 +0000","from IA1PR10MB7447.namprd10.prod.outlook.com\n ([fe80::f2fe:d6c6:70c4:4572]) by IA1PR10MB7447.namprd10.prod.outlook.com\n ([fe80::f2fe:d6c6:70c4:4572%7]) with mapi id 15.20.7828.023; Wed, 7 Aug 2024\n 21:04:33 +0000"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=\n message-id:date:subject:to:cc:references:from:in-reply-to\n :content-type:content-transfer-encoding:mime-version; s=\n corp-2023-11-20; bh=oozn1w+FJaaXpcomAqe5jLrUYOoNhOlbTl4AkvBrk7Y=; b=\n dozNQGoz2cumhx+J0Z8xBkj99KleUa1CET4/2F1GiFeclYGFv78TRwAWJ81TvodI\n DcCxKSvOWGCnd3hj+Hc/qFGh+4kYXaNwy6KTBJujb1uMknku79cdzz5ALNobCSAx\n FHGaUVCcQ7cdF9dwOMP/ZnSv+ACmhQniJELCvrnuJwp8pwoetMwmlZaGKV4li3cc\n rfGq1vbKsneKIN/TYa6cczEK3uVcKUUSoUxwLovqPMoIjqpeU65N75hG3F1cnI/C\n 2o4AuU9zomjbSvDaa6b+sfA4KyE936woy8p51S3YqsJiKbgv5c3j+gcNdzpyEG9X\n TCGQzU8gtN3ffDauJYKbbQ==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;\n bh=oozn1w+FJaaXpcomAqe5jLrUYOoNhOlbTl4AkvBrk7Y=;\n b=br56CFpakKR6sxvyd7gE8zZjNm7DmJ7xkC/UWACXPQtyyEEqsNDZsQJ0pgn0Ic/4e3cIdkJcOq09M6i62ANrIqZR8/51fAjrOcBprxG46+qVG50JlHkpNn9T3tjU1mNe+6SrGBE8xn6/5SS7tQ1dcq9oi+fPgmkt63L1e9727UM="],"ARC-Seal":"i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none;\n b=h5WoRgYfKttvvgoY2gjTaTPfmzpjf3uUKDtUY3DpxrgXFZLyRL8J2eN5Ymh3ELZ4eLtSuIuGVeYcg1gnTmNIfJvlcB5DUrvtorFpOJUrJsf5HqHgOCsfeqIYPlFRa3RPDUrimuJ9WtxPuNysg4taDlAu5WFS0G7e3QteHZDzugF+kv4KuHWM9vBxlWRp716X18UqVABXaTnGdoakvt99+uERccKWe5YqamyGt6+RuGEUWU6FZ//QkVfIYMV3ACON6w6C4UWssc9AqC/2AQg46dlDOjlqNeJgTDntoD50Hoe479xEZapTG5XN0r+2fBoQ4AcVyM4K5xZg9LJv42AKrg==","ARC-Message-Signature":"i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;\n s=arcselector10001;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;\n bh=oozn1w+FJaaXpcomAqe5jLrUYOoNhOlbTl4AkvBrk7Y=;\n b=O0yQjMad8gMndbUawtvmh5ybGgVtz4dARSHAlL5t3198UALYgyT3CH9wR8xv4Sogqw6QCzL8USSAJJh3mfYVX+xeAJnr7yxBy9g9fhqQ9tbDCc3r+WOFpmOcoFLU3r1uwR8MvA4Ygk4cE89r0H23KTYQn4N0EE816vkUdOWb6Y6aiiejlhKIli/KSX80EEX168wQfQgs186VZaT6JkhaKJrqs8dW86qg98JjR749UYRoI9bbQygN152RHQlyrfOkJLJ37/YVx87Sdo3Bo7ZPp0yOBnXU5Qff+ctANA2HkvFjUu88RqcVfgWoh7hhUR9jDm3VYsYSMYnGPAYz7ViVxA==","ARC-Authentication-Results":"i=1; mx.microsoft.com 1; spf=pass\n smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;\n dkim=pass header.d=oracle.com; arc=none","Message-ID":"<571a4f84-693c-43d4-a43a-52a53a1091e1@oracle.com>","Date":"Wed, 7 Aug 2024 17:04:26 -0400","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH V2 06/11] migration: fix mismatched GPAs during cpr","To":"Peter Xu ","Cc":"qemu-devel@nongnu.org, Fabiano Rosas ,\n David Hildenbrand ,\n Marcel Apfelbaum ,\n Eduardo Habkost ,\n Philippe Mathieu-Daude ,\n Paolo Bonzini , \"Daniel P. Berrange\"\n , Markus Armbruster ","References":"<1719776434-435013-1-git-send-email-steven.sistare@oracle.com>\n <1719776434-435013-7-git-send-email-steven.sistare@oracle.com>\n ","Content-Language":"en-US","From":"Steven Sistare ","Organization":"Oracle Corporation","In-Reply-To":"","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","X-ClientProxiedBy":"SJ2PR07CA0004.namprd07.prod.outlook.com\n (2603:10b6:a03:505::10) To IA1PR10MB7447.namprd10.prod.outlook.com\n (2603:10b6:208:44c::10)","MIME-Version":"1.0","X-MS-PublicTrafficType":"Email","X-MS-TrafficTypeDiagnostic":"IA1PR10MB7447:EE_|MN2PR10MB4142:EE_","X-MS-Office365-Filtering-Correlation-Id":"b81a18da-db7d-44b5-7571-08dcb7248911","X-MS-Exchange-SenderADCheck":"1","X-MS-Exchange-AntiSpam-Relay":"0","X-Microsoft-Antispam":"BCL:0;ARA:13230040|7416014|376014|366016|1800799024;","X-Microsoft-Antispam-Message-Info":"=?utf-8?q?5HeZXzbrnW2D6x9IGZQx+O7scC0Uto0?=\n\t=?utf-8?q?iUweCkxmpObw1y4Bi9O6uH04SKjyCio5FclV8X8IhmTAAmMIaPnVdjlcM9XEOLnmM?=\n\t=?utf-8?q?3wkN92GxHEeEPD+P1ZPkdEOoUQHWnFfuYOU6jngdIopE8s5OcaWulF4mu/u1nb9Nx?=\n\t=?utf-8?q?qbSCexI6HAS/u8b15Q4wJqxYNcErtYpJxUPduNVM41/+0NAfy+Jeg+irk2h0uDNR6?=\n\t=?utf-8?q?P7d3UZSgN4GkYP6rokUi/wKXRnYrabt9SAVI0CihIDTgEQJxnveIfUwSrWh7XCq6h?=\n\t=?utf-8?q?N8ntnGDXBDnyOeyULvh9XEIOnu2/b8635hVqg+h/r1d/jYvFc6JJOcxSOm7/QOjzz?=\n\t=?utf-8?q?T86Dam4c6AkOKwI9pVpCbdnl78xTdM5wjnUp4Dh9YaZzWlEUneR6QMRNa32Yda0Im?=\n\t=?utf-8?q?LP7fN8RKq4Pw4w5j4wORw9oFQ5HjkR7SvTKGSa0daN/3dLMtsr3OoNqIPyLVa+6iC?=\n\t=?utf-8?q?v74UJVt6Rwmr3O2D48rbBrYaZQgXq2p+Tw0fEnAHlDei9nHTO5bTMam/e74wqtuaB?=\n\t=?utf-8?q?DVIezxoYzriAXqzheD8kerQwUNCw6uDnGt464k1gddWZNq+cKV+cwxz+7HCdQOS3D?=\n\t=?utf-8?q?O6sLc0Lg29YqWOL/iqzISfwohTr8MYVO8H7gvfcb0KCjvBP+KJx42m9DOT0hbiXgI?=\n\t=?utf-8?q?NkCS1yc67gwFJIG/POrUYArTcDq0wePmduuYkbxlBJDOhhW0h2lwzGKV6m+qKiLAB?=\n\t=?utf-8?q?VlTeEtY9uCApvgrREhNRgwBPOQ4sdjbgo1EVx3nzzbn2AAOXDcQklal+xtMyzog9j?=\n\t=?utf-8?q?G2i6gq5ilziJfWUN19E+JdrFhtkJ56r+8znwmIyd/wEoSchJ8Trwk5wgrj/z9j4vM?=\n\t=?utf-8?q?OMpDbde0E7SCEDJB5laDxwxUJv87WGbxaWxvdeQdBElOJMSOI/eZCKCgoRv46FZDw?=\n\t=?utf-8?q?kB2FsynWVUi6XDoxB8/8dfeR/cCFBOezLdaCT+yZFLef3nv9e7KTB3JzlXQEavfuK?=\n\t=?utf-8?q?6VqS3ZC1VRHE1APn/eztDW1JpdYGSmmw+jMBg3N3K9QxsiN3Ur0Znto/1VkzdEFhz?=\n\t=?utf-8?q?2TKGDN29pyMYlrk6s1veqwDccbI03McOLC5+Kv/TmZUNTrklQlYAls7tFMQpSRNBv?=\n\t=?utf-8?q?4xjywLlGxMFtaos7zu0Y9wlHARGB/PO+25u0UDjEuk4SRxmVbEMXVTuJpX0+8FSlm?=\n\t=?utf-8?q?sIuuUQKBiGw8XQe5/ZIB2bhLUmbzCIL4TkuQz5YcJRuh9h7gN7kYUolZAVquZyPBN?=\n\t=?utf-8?q?zQ8DHI6Nt0H7SvKQ9bIxdmH8zJFUVYBjR8yiqh7LdJhjEYp1Cydz+4xkI/P8MMQBq?=\n\t=?utf-8?q?dpI80kdho//gJ+fIdV2RX1mks65OJFJVjiQ=3D=3D?=","X-Forefront-Antispam-Report":"CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;\n IPV:NLI; SFV:NSPM; H:IA1PR10MB7447.namprd10.prod.outlook.com; PTR:; CAT:NONE;\n SFS:(13230040)(7416014)(376014)(366016)(1800799024); DIR:OUT; SFP:1101;","X-MS-Exchange-AntiSpam-MessageData-ChunkCount":"1","X-MS-Exchange-AntiSpam-MessageData-0":"=?utf-8?q?WyvUHTH4SE1hxoL5q8diXdOjWXYb?=\n\t=?utf-8?q?C0KaZb2zpoGgy8d512yw/XBdG9QkDyuS8joKsAIhAZI5KAtbclwncIIZWKIpOO5gj?=\n\t=?utf-8?q?C/9JrpuDKBnGXZ6v2ynyh+OEtHeNxOpQXfLtYDWPwUEWaQM8T8IuulZ/GngOHdQrg?=\n\t=?utf-8?q?nmW0SAMl1+98D4bElJ1i8i9KKPzEhi085zu+Bm7nT9KDG1FYF14NIy2C/nIGKuVHe?=\n\t=?utf-8?q?HOY0ETdQkjDQfFYYYSuFemC845C5hug8I1cc2LRud8Vfjvb42TcezKAzHOZOZYuTM?=\n\t=?utf-8?q?D7XLeox1TO4gy415XziPBJ+NlFBKa/fjM7MA0q4oiREgChd5SZU3pTIdBwLxrAxAM?=\n\t=?utf-8?q?P+/uqYRVQo4lZYNT8QYOKTPVFfDIxKLzvCYF7hVvCHcNU5MtLx3TMxvkKu0xfXLjd?=\n\t=?utf-8?q?v4c8qr//FwOteufP0UDMblpts1VWr/xz+nLwqnU3hJFca98OBv29fQdxe7+YYMGK0?=\n\t=?utf-8?q?zmqspMTIdLz2MGyu6Ftgdq8exMGnwgVYV5uZ78v9FxgK7NI26Od/gAMQQGW6y3nrv?=\n\t=?utf-8?q?rq7Wn0ZAESUjMeGae/lE0gF2ibbhtcjMvTLvfWRUJBb0mBu4H6lg1qRjFJVw5t0MZ?=\n\t=?utf-8?q?vzRauZf3xeScU7weKlGPQXCtUOu8zpFNE5xOQFGEL47kpqPMH7Z4Ba5L3HK5PPTz/?=\n\t=?utf-8?q?6aM4nJ2YeyHEORpJrydWKOBvfibu4sukq+i5IQIeuK8zOinbbw1hHbx8u2uvI2muo?=\n\t=?utf-8?q?hV8c9hxCjbB8qtosFlyCPF8HRiMw5VSfoJ7uRp6/dQWn1IEAEInvD1FBMquZZusnE?=\n\t=?utf-8?q?NLM8G2esUrcJigj+cJcpdXeI1TOjrN+7XxdvJxmI00MCONwUDCk2be7BNT5MOt1c3?=\n\t=?utf-8?q?6yclAujOIjAviOQ8tkSROYOBYSA9IKMIF746kCoSwfCd3PK8R51eegum+pIyesBBD?=\n\t=?utf-8?q?VPnlsQabL1bg8qWgW1wMNfNMMmrufaSbAIyJsomX5VBK+gJ3YUPOXzUgZVBwZe2Rf?=\n\t=?utf-8?q?Sq/+LvnYr3Bn+145+KSRcUfHTuzGmuw52Za1HxFbWX45FpdIyZCtisnxdSrPkQ4Uk?=\n\t=?utf-8?q?wNx/23ieQp7mBUAN4tUU9LrTHKS5GYxseFBrY9kBybh9rNCzta/MHaWIyyoU1DkI/?=\n\t=?utf-8?q?keeBpKI0dcgA3u2sm0k3efyLbVRmfMqYbe+XGjlemD9feO+98C4FL0raB3rO7LdTP?=\n\t=?utf-8?q?I2CWuQaaco3+8b3bejuvR1+jg0nuWyLtTQKD2iHyITNP6Jo187Jzrtj4KYmXi4NBV?=\n\t=?utf-8?q?3RW98zXQmvBIvm8Xm701kQg1lCbQGDif+nf3AAgZ64agv5Yn7N67CZbFurTwv7cF1?=\n\t=?utf-8?q?u1ENi7/NsCt5qguxFH91JN5ioQjBKH+72Ng0/V5U9sbp22qvr1F9nt3E7hurnOc8y?=\n\t=?utf-8?q?oM7r4ZyqOmqcYN11BM9PS5jvLyP0uFXg+jaPWquQbOxlUgSy8CkwInak4rQ8BFaLc?=\n\t=?utf-8?q?54GRukgGo2NRwMk+Cnik0H8Yzsc7d4Df9+gWGx4cPGeXTZbxe4rGrduZB1J9VCTHg?=\n\t=?utf-8?q?iANqfGZjPvU2oq7u10gMRF0VX0olVwglivoY7WERA/zmhzoDC/0uCEVkesiX0+FJ1?=\n\t=?utf-8?q?BmJiB9kbG5efF3LbNGalFVnDowRGGDW+vQ=3D=3D?=","X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount":"1","X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0":"\n SlnEOgdfj2bM8ry+su25jaRmtJ12ytTVKfF1cD2sJxTe2GNbs9ynVckjnyphT871G86jjTlXJ+W7AJcLHiIJYdzfsq7TXe9LXyCCjnLf42ZEdNlM5N1NUjyPiMO30O6NKVaxdqVZbs8n/MbzTFMujUJHJP5AEJMjG5POOZLtXLHk+y9XS6i/IkiBDjjbCrgBn8ES5Gw81/4ryxBpcix/pDbtW5ZgKQkEw8FJGWkTn90X06NYQzWB0qGEcwNEnM/b+FyoZN301TspN+PJ4DLgYsW+hYam/D0ynC06h1DD6XU4bQRGPxGcjouat6KooXAU4k+kBRA8hoaPTb5oLkh9wgRn8sZwGd1xAOf3QKvYHO/54shX1h3E7KxjLKUZrOTdf5ViK+YVsWKX99dGK/7hZudffUsJtsR28HxebaFYtdi2UKOgRmRJvO/kX8E9y3am7AocRpvOXfwOGSf14vK6vvWtKquB1UXLITtKka1F/ErDYEj2gCgWoiyv3v5+itWk9E/g1UXgPZwo/4mZ91FEv/fWd+whGBlswX86giW+SISLCp2oUx5oQUoytNr9h46yZbQ2AU2JZrbc316btWtz6tz2ZeFd6bQqFVyZklPcOWE=","X-OriginatorOrg":"oracle.com","X-MS-Exchange-CrossTenant-Network-Message-Id":"\n b81a18da-db7d-44b5-7571-08dcb7248911","X-MS-Exchange-CrossTenant-AuthSource":"IA1PR10MB7447.namprd10.prod.outlook.com","X-MS-Exchange-CrossTenant-AuthAs":"Internal","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"07 Aug 2024 21:04:33.0433 (UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"4e2c6054-71cb-48f1-bd6c-3a9705aca71b","X-MS-Exchange-CrossTenant-MailboxType":"HOSTED","X-MS-Exchange-CrossTenant-UserPrincipalName":"\n 9eaBtfkh8gZzMqRwrb8LG5Kz+UubMg5oW3PBeWLJ3b1hE35MUzDn6hJrh3AXcbNgwbi5A6JNz8uySTrtIJCzhDcOHHHn09oE7M5qL3k61pQ=","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"MN2PR10MB4142","X-Proofpoint-Virus-Version":"vendor=baseguard\n engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16\n definitions=2024-08-07_11,2024-08-07_01,2024-05-17_01","X-Proofpoint-Spam-Details":"rule=notspam policy=default score=0 spamscore=0\n adultscore=0\n mlxlogscore=999 phishscore=0 bulkscore=0 mlxscore=0 malwarescore=0\n suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1\n engine=8.12.0-2407110000 definitions=main-2408070146","X-Proofpoint-GUID":"8YwRQKBRQN0d9-4BvbHOqDDFRFCNQWuo","X-Proofpoint-ORIG-GUID":"8YwRQKBRQN0d9-4BvbHOqDDFRFCNQWuo","Received-SPF":"pass client-ip=205.220.177.32;\n envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com","X-Spam_score_int":"-27","X-Spam_score":"-2.8","X-Spam_bar":"--","X-Spam_report":"(-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,\n DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001,\n SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}},{"id":3361845,"web_url":"http://patchwork.ozlabs.org/comment/3361845/","msgid":"","list_archive_url":null,"date":"2024-08-13T20:43:08","subject":"Re: [PATCH V2 06/11] migration: fix mismatched GPAs during cpr","submitter":{"id":67717,"url":"http://patchwork.ozlabs.org/api/people/67717/","name":"Peter Xu","email":"peterx@redhat.com"},"content":"On Wed, Aug 07, 2024 at 05:04:26PM -0400, Steven Sistare wrote:\n> On 7/19/2024 12:28 PM, Peter Xu wrote:\n> > On Sun, Jun 30, 2024 at 12:40:29PM -0700, Steve Sistare wrote:\n> > > For new cpr modes, ramblock_is_ignored will always be true, because the\n> > > memory is preserved in place rather than copied. However, for an ignored\n> > > block, parse_ramblock currently requires that the received address of the\n> > > block must match the address of the statically initialized region on the\n> > > target. This fails for a PCI rom block, because the memory region address\n> > > is set when the guest writes to a BAR on the source, which does not occur\n> > > on the target, causing a \"Mismatched GPAs\" error during cpr migration.\n> > \n> > Is this a common fix with/without cpr mode?\n> > \n> > It looks to me mr->addr (for these ROMs) should only be set in PCI config\n> > region updates as you mentioned. But then I didn't figure out when they're\n> > updated on dest in live migration: the ramblock info was sent at the\n> > beginning of migration, so it doesn't even have PCI config space migrated;\n> > I thought the real mr->addr should be in there.\n> > \n> > I also failed to understand yet on why the mr->addr check needs to be done\n> > by ignore-shared only. Some explanation would be greatly helpful around\n> > this area..\n> \n> The error_report does not bite for normal migration because migrate_ram_is_ignored()\n> is false for the problematic blocks, so the block->mr->addr check is not\n> performed. However, mr->addr is never fixed up in this case, which is a\n> quiet potential bug, and this patch fixes that with the \"has_addr\" check.\n> \n> For cpr-exec, migrate_ram_is_ignored() is true for all blocks,\n> because we do not copy the contents over the migration stream, we preserve the\n> memory in place. So we fall into the block->mr->addr sanity check and fail\n> with the original code.\n\nOK I get your point now. However this doesn't look right, instead I start\nto question why we need to send mr->addr at all..\n\nAs I said previously, AFAIU mr->addr should only be updated when there's\nsome PCI config space updates so that it moves the MR around in the address\nspace based on how guest drivers / BIOS (?) set things up. Now after these\ndays not looking, and just started to look at this again, I think the only\nsane place to do this update is during a post_load().\n\nAnd if we start to check some of the memory_region_set_address() users,\nthat's exactly what happened..\n\n - ich9_pm_iospace_update(), update addr for ICH9LPCPMRegs.io, where\n ich9_pm_post_load() also invokes it.\n\n - pm_io_space_update(), updates PIIX4PMState.io, where\n vmstate_acpi_post_load() also invokes it.\n\nI stopped here just looking at the initial two users, it looks all sane to\nme that it only got updated there, because the update requires pci config\nspace being migrated first.\n\nIOW, I don't think having mismatched mr->addr is wrong at this stage.\nInstead, I don't see why we should send mr->addr at all in this case during\nas early as SETUP, and I don't see anything justifies the mr->addr needs to\nbe verified in parse_ramblock() since ignore-shared introduced by Yury in\ncommit fbd162e629aaf8 in 2019.\n\nWe can't drop mr->addr now when it's on-wire, but I think we should drop\nthe error report and addr check, instead of this patch.\n\nThanks,","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=mimecast20190719 header.b=dt0HaVqp;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4Wk3K342Jtz1yXl\n\tfor ; Wed, 14 Aug 2024 06:44:22 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1sdyMS-0001Yw-Iq; Tue, 13 Aug 2024 16:43:20 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1sdyMR-0001Ug-BO\n for qemu-devel@nongnu.org; Tue, 13 Aug 2024 16:43:19 -0400","from us-smtp-delivery-124.mimecast.com ([170.10.133.124])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1sdyMP-00016M-I3\n for qemu-devel@nongnu.org; Tue, 13 Aug 2024 16:43:19 -0400","from mail-qv1-f71.google.com (mail-qv1-f71.google.com\n [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS\n (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n us-mta-88-hzh0wrx6PauSf4e-d2Me6w-1; Tue, 13 Aug 2024 16:43:12 -0400","by mail-qv1-f71.google.com with SMTP id\n 6a1803df08f44-6ba92dd246fso12138356d6.0\n for ; Tue, 13 Aug 2024 13:43:12 -0700 (PDT)","from x1n (pool-99-254-121-117.cpe.net.cable.rogers.com.\n [99.254.121.117]) by smtp.gmail.com with ESMTPSA id\n 6a1803df08f44-6bd82e53a83sm37029096d6.118.2024.08.13.13.43.10\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Tue, 13 Aug 2024 13:43:11 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n s=mimecast20190719; t=1723581795;\n h=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n to:to:cc:cc:mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=Y95b4aF5TiUzwUL+WNWP1EyFDs2fFxavfLllY0yoacw=;\n b=dt0HaVqp3DLF5wNStUUlwO4imuPsQa3JAwOOtDnXaFBgCdjytIspF/fdD2xmscapQt/9mx\n kbgICJVKMGNmluNgfmUb8APLdvDLRvRyaEk7c9JRP2KWUJRDOsd0yGq2pOGYE1kwiEZyPJ\n 8B8ULi87Et7I+Nkde8Q4kRan7d0atYA=","X-MC-Unique":"hzh0wrx6PauSf4e-d2Me6w-1","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20230601; t=1723581792; x=1724186592;\n h=in-reply-to:content-disposition:mime-version:references:message-id\n :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date\n :message-id:reply-to;\n bh=Y95b4aF5TiUzwUL+WNWP1EyFDs2fFxavfLllY0yoacw=;\n b=S+osFkjXtRF8qPtKqXPM1cykGBOdlx+Uta5W7bmInmukgxAG5AnZ1XoRPVkFcI8G0d\n LnqCPsxLO1eVuUx6z3FQiK8Ut9MYV5wdlM5cYhf7JAd4m2eYQUKbho2aJLQAIIhvnamX\n 2cuPuv5op/bXH7IrOqokLtVJkwP1V19JmqaJdlONAmrTYP20bTz+K161DFc4YeP1RtRQ\n cP6NF0yMlpaywdNzVfstJqz4Z27xcbZw93gWjB+ILARwlzqw/NlUQN8pPElJpHr7C18Q\n om7OJSO/u0HJeTJb+ESKcZPtx2M7hKMTJULNbbvAU1ihrhMWmRwkuFLWuGcdrJ9Wt0Om\n WRpw==","X-Gm-Message-State":"AOJu0YxGET3fwUgkkm61XlM3cguyFjfDzWL6CVHrbgT/slI22PMX1/R5\n R8YTtnh5fUsrnB6O2NGyHXgj+TfdHljvgXYyTgL2eik+HB1x/nJXna9/T3T86OJ8+JwJGo601BM\n 3ksO223M3vKl9rXi6xLskTuhQgwhqiDtpQP2v0N+9TL9W3lew9UCG","X-Received":["by 2002:a0c:ff4a:0:b0:6bf:5d84:1318 with SMTP id\n 6a1803df08f44-6bf5d841a07mr3955826d6.0.1723581792077;\n Tue, 13 Aug 2024 13:43:12 -0700 (PDT)","by 2002:a0c:ff4a:0:b0:6bf:5d84:1318 with SMTP id\n 6a1803df08f44-6bf5d841a07mr3955596d6.0.1723581791674;\n Tue, 13 Aug 2024 13:43:11 -0700 (PDT)"],"X-Google-Smtp-Source":"\n AGHT+IEfLVXAx/uzRB1N1dJtuIf86d+N+gUEroNZFU/iuJs8QEcQ+ZrMYbYbW5XCkqS9fcprAxFrUg==","Date":"Tue, 13 Aug 2024 16:43:08 -0400","From":"Peter Xu ","To":"Steven Sistare ","Cc":"qemu-devel@nongnu.org, Fabiano Rosas ,\n David Hildenbrand ,\n Marcel Apfelbaum ,\n Eduardo Habkost ,\n Philippe Mathieu-Daude ,\n Paolo Bonzini ,\n \"Daniel P. Berrange\" ,\n Markus Armbruster ","Subject":"Re: [PATCH V2 06/11] migration: fix mismatched GPAs during cpr","Message-ID":"","References":"<1719776434-435013-1-git-send-email-steven.sistare@oracle.com>\n <1719776434-435013-7-git-send-email-steven.sistare@oracle.com>\n \n <571a4f84-693c-43d4-a43a-52a53a1091e1@oracle.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<571a4f84-693c-43d4-a43a-52a53a1091e1@oracle.com>","Received-SPF":"pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com;\n helo=us-smtp-delivery-124.mimecast.com","X-Spam_score_int":"-21","X-Spam_score":"-2.2","X-Spam_bar":"--","X-Spam_report":"(-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.125,\n DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001,\n SPF_HELO_NONE=0.001, SPF_PASS=-0.001,\n T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}},{"id":3363027,"web_url":"http://patchwork.ozlabs.org/comment/3363027/","msgid":"<5f763763-1479-4585-98ce-83fcec03b4db@oracle.com>","list_archive_url":null,"date":"2024-08-15T20:54:58","subject":"Re: [PATCH V2 06/11] migration: fix mismatched GPAs during cpr","submitter":{"id":71906,"url":"http://patchwork.ozlabs.org/api/people/71906/","name":"Steve Sistare","email":"steven.sistare@oracle.com"},"content":"On 8/13/2024 4:43 PM, Peter Xu wrote:\n> On Wed, Aug 07, 2024 at 05:04:26PM -0400, Steven Sistare wrote:\n>> On 7/19/2024 12:28 PM, Peter Xu wrote:\n>>> On Sun, Jun 30, 2024 at 12:40:29PM -0700, Steve Sistare wrote:\n>>>> For new cpr modes, ramblock_is_ignored will always be true, because the\n>>>> memory is preserved in place rather than copied. However, for an ignored\n>>>> block, parse_ramblock currently requires that the received address of the\n>>>> block must match the address of the statically initialized region on the\n>>>> target. This fails for a PCI rom block, because the memory region address\n>>>> is set when the guest writes to a BAR on the source, which does not occur\n>>>> on the target, causing a \"Mismatched GPAs\" error during cpr migration.\n>>>\n>>> Is this a common fix with/without cpr mode?\n>>>\n>>> It looks to me mr->addr (for these ROMs) should only be set in PCI config\n>>> region updates as you mentioned. But then I didn't figure out when they're\n>>> updated on dest in live migration: the ramblock info was sent at the\n>>> beginning of migration, so it doesn't even have PCI config space migrated;\n>>> I thought the real mr->addr should be in there.\n>>>\n>>> I also failed to understand yet on why the mr->addr check needs to be done\n>>> by ignore-shared only. Some explanation would be greatly helpful around\n>>> this area..\n>>\n>> The error_report does not bite for normal migration because migrate_ram_is_ignored()\n>> is false for the problematic blocks, so the block->mr->addr check is not\n>> performed. However, mr->addr is never fixed up in this case, which is a\n>> quiet potential bug, and this patch fixes that with the \"has_addr\" check.\n>>\n>> For cpr-exec, migrate_ram_is_ignored() is true for all blocks,\n>> because we do not copy the contents over the migration stream, we preserve the\n>> memory in place. So we fall into the block->mr->addr sanity check and fail\n>> with the original code.\n> \n> OK I get your point now. However this doesn't look right, instead I start\n> to question why we need to send mr->addr at all..\n> \n> As I said previously, AFAIU mr->addr should only be updated when there's\n> some PCI config space updates so that it moves the MR around in the address\n> space based on how guest drivers / BIOS (?) set things up. Now after these\n> days not looking, and just started to look at this again, I think the only\n> sane place to do this update is during a post_load().\n> \n> And if we start to check some of the memory_region_set_address() users,\n> that's exactly what happened..\n> \n> - ich9_pm_iospace_update(), update addr for ICH9LPCPMRegs.io, where\n> ich9_pm_post_load() also invokes it.\n> \n> - pm_io_space_update(), updates PIIX4PMState.io, where\n> vmstate_acpi_post_load() also invokes it.\n> \n> I stopped here just looking at the initial two users, it looks all sane to\n> me that it only got updated there, because the update requires pci config\n> space being migrated first.\n> \n> IOW, I don't think having mismatched mr->addr is wrong at this stage.\n> Instead, I don't see why we should send mr->addr at all in this case during\n> as early as SETUP, and I don't see anything justifies the mr->addr needs to\n> be verified in parse_ramblock() since ignore-shared introduced by Yury in\n> commit fbd162e629aaf8 in 2019.\n> \n> We can't drop mr->addr now when it's on-wire, but I think we should drop\n> the error report and addr check, instead of this patch.\n\nAs it turns out, my test case triggers this bug because it sets x-ignore-shared,\nbut x-ignore-shared is not needed for cpr-exec, because migrate_ram_is_ignored\nis true for all blocks when mode==cpr-exec. So, the best fix for the GPAs bug\nfor me is to stop setting x-ignore-shared. I will drop this patch.\n\nI agree that post_load is the right place to restore mr->addr, and I don't\nunderstand why commit fbd162e629aaf8 added the error report, but I am going\nto leave it as is.\n\nThanks for reviewing this.\n\n- Steve","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256\n header.s=corp-2023-11-20 header.b=f3ZtGyLJ;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com\n header.a=rsa-sha256 header.s=selector2-oracle-onmicrosoft-com\n header.b=V/D72IvP;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4WlHTZ1zdGz1yfB\n\tfor ; Fri, 16 Aug 2024 06:56:02 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1sehV9-0005rj-1L; Thu, 15 Aug 2024 16:55:19 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1sehV5-0005rE-VJ\n for qemu-devel@nongnu.org; Thu, 15 Aug 2024 16:55:16 -0400","from mx0b-00069f02.pphosted.com ([205.220.177.32])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1sehV2-0005am-O8\n for qemu-devel@nongnu.org; Thu, 15 Aug 2024 16:55:15 -0400","from pps.filterd (m0246631.ppops.net [127.0.0.1])\n by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id\n 47FJtg5K018534;\n Thu, 15 Aug 2024 20:55:07 GMT","from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com\n (iadpaimrmta03.appoci.oracle.com [130.35.103.27])\n by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 40wy033hct-1\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);\n Thu, 15 Aug 2024 20:55:07 +0000 (GMT)","from pps.filterd\n (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1])\n by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19)\n with ESMTP id 47FKJLcN003739; Thu, 15 Aug 2024 20:55:06 GMT","from nam04-bn8-obe.outbound.protection.outlook.com\n (mail-bn8nam04lp2041.outbound.protection.outlook.com [104.47.74.41])\n by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id\n 40wxncm2ns-1\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);\n Thu, 15 Aug 2024 20:55:06 +0000","from IA1PR10MB7447.namprd10.prod.outlook.com (2603:10b6:208:44c::10)\n by DS0PR10MB6126.namprd10.prod.outlook.com (2603:10b6:8:c6::20) with\n Microsoft SMTP Server (version=TLS1_2,\n cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.16; Thu, 15 Aug\n 2024 20:55:04 +0000","from IA1PR10MB7447.namprd10.prod.outlook.com\n ([fe80::f2fe:d6c6:70c4:4572]) by IA1PR10MB7447.namprd10.prod.outlook.com\n ([fe80::f2fe:d6c6:70c4:4572%7]) with mapi id 15.20.7875.016; Thu, 15 Aug 2024\n 20:55:04 +0000"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=\n message-id:date:subject:to:cc:references:from:in-reply-to\n :content-type:content-transfer-encoding:mime-version; s=\n corp-2023-11-20; bh=U6kDxdw7bz951mxvn47RgdCtFvinopTa6eD92d8VOv8=; b=\n f3ZtGyLJo7dHoXyqUqdcAoGwSoPVF/g4LYIPKHeZQJ9yi7qtnlgxo6NldC3PEypL\n zHj5HEVlyMawc9IGQre77uINQs4UP6EyFgV7bw5F7C6+ypmJaUQGpRmOmLCqaYfn\n oGx10hbm7EhoF44LsNDrTRUoNXN0NlbK1Ligzfc5NYVHbosQcH1XjHxor0qHEl88\n Y11LQBdt5+e/CHV+eMhvhuoK8rZd2BxwZLmTvl/2qvi0KBaBGZI7xBbjh/aC0Ehp\n 2Jmdc0qxEvGYGLuYRrtURGQIUFMYRzuNuH+XVM7aROwpY9BNEyYyuv+tpcFVdD/I\n h3uGHty8VBj7U84iN2evEA==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;\n bh=U6kDxdw7bz951mxvn47RgdCtFvinopTa6eD92d8VOv8=;\n b=V/D72IvPGBF/HqUuCE7d0lSYNR411kp4s7/JBgAlIfs6MTo6XU17a3b/h2Dee4k0VmhRBTvPiHttywhWDFX7aolqu330ZJAlF2emyRUBHIXF4lte0pCK93ILlTEzuaY6gkq5aRJSPj19/sGcC1mPKQcCLZn27m4bCTTTUy3+nhI="],"ARC-Seal":"i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none;\n b=NNuUsrpxGkbHJBUIwAB0/XRSaBmpkiqBNQJj3yyt88tEwtxXEsqjOZwE9zBJfnA1+xlKMmXeM+mdEFvVSixCxzPweaB4nxthyulS9OryprBejQhNJnKvof/ppxXZPWQAr+A1UQutig8n7P4iO66WiDAGsWDrtOUOJO1gvAlIesgyli39nEKJb2EuSqaEPvOfHlubVd1EX3si7GiJrV8cO3knQd3WSV/MByZzdW16we/WDqbdu3ny68mH4/Ghxi7HpA2Mc/p85hJxr6wrQGedCNlW06h6xbbc+pTNl6z+xDVRDPjwoPAHAaaVWTh55Q3MvIvDLfr/VmcP6kXKCZ6ghA==","ARC-Message-Signature":"i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;\n s=arcselector10001;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;\n bh=U6kDxdw7bz951mxvn47RgdCtFvinopTa6eD92d8VOv8=;\n b=u82o17Rz+AzLy+aPOsDn2z1AxZiif2pct8W9J91gkIUkz/J7w/nPJn0sdanzJvQ3a2+lDcObyMJp0GVN0RUp8Dupd9H4gvVejabcYe/R+AfXVZInjL43o8WdLeUdbF3ocqFvTxrcgqdy9HPysIDxknt8gK0R7q12UfWqkctYy4Deow/UTeLwaxk4ZahRf7tYdaNWWxXagUkS6RVv+T24JWrpcqFHEtN5Z7V+oxTaKIOE7S12mK3AaDKtWefQ6cvIJUMvG3jjoDeVMRu8ykewHQqqwJdOkq//WRiOw2L6YIRdDJSgarjo4pqyTR7LQ5myA/92aQ9bdfHoA7UgaqaTBw==","ARC-Authentication-Results":"i=1; mx.microsoft.com 1; spf=pass\n smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;\n dkim=pass header.d=oracle.com; arc=none","Message-ID":"<5f763763-1479-4585-98ce-83fcec03b4db@oracle.com>","Date":"Thu, 15 Aug 2024 16:54:58 -0400","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH V2 06/11] migration: fix mismatched GPAs during cpr","To":"Peter Xu ","Cc":"qemu-devel@nongnu.org, Fabiano Rosas ,\n David Hildenbrand ,\n Marcel Apfelbaum ,\n Eduardo Habkost ,\n Philippe Mathieu-Daude ,\n Paolo Bonzini , \"Daniel P. Berrange\"\n , Markus Armbruster ","References":"<1719776434-435013-1-git-send-email-steven.sistare@oracle.com>\n <1719776434-435013-7-git-send-email-steven.sistare@oracle.com>\n <571a4f84-693c-43d4-a43a-52a53a1091e1@oracle.com>\n ","Content-Language":"en-US","From":"Steven Sistare ","Organization":"Oracle Corporation","In-Reply-To":"","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","X-ClientProxiedBy":"BYAPR07CA0009.namprd07.prod.outlook.com\n (2603:10b6:a02:bc::22) To IA1PR10MB7447.namprd10.prod.outlook.com\n (2603:10b6:208:44c::10)","MIME-Version":"1.0","X-MS-PublicTrafficType":"Email","X-MS-TrafficTypeDiagnostic":"IA1PR10MB7447:EE_|DS0PR10MB6126:EE_","X-MS-Office365-Filtering-Correlation-Id":"52e0e17d-cab4-49aa-19e2-08dcbd6c898d","X-MS-Exchange-SenderADCheck":"1","X-MS-Exchange-AntiSpam-Relay":"0","X-Microsoft-Antispam":"BCL:0;ARA:13230040|366016|7416014|376014|1800799024;","X-Microsoft-Antispam-Message-Info":"=?utf-8?q?eYrzdVOEUUPQzUPwNASDoA0qT3PCslb?=\n\t=?utf-8?q?KJviBsrS5KR/Nfhv4z/gFLKnynFeAVl0GP5J5JKJ3tot6PfdSiSwbEjGXdnDk+L25?=\n\t=?utf-8?q?/gY4lKdRrMvYE1dRPeyQbVhtcafJjWh1dMsgMX/BOhCFjPHyeR3gGFi1jKpZ309pD?=\n\t=?utf-8?q?oTZrDUhJ3nsAxf948RS8ymqIK0ryAmbXNbimnab09SFTI7j2X1jh5ciP209ye6Unl?=\n\t=?utf-8?q?RBNylZtkQKR6isGmYMlL4rMf37geyOCluW4JUWBQUjA/GLTBgpxXo9Z1vfKYAdqnq?=\n\t=?utf-8?q?lyEBcTJwLsPh9hLXWn9CRlXC9+iYc5/9IaFGPgTzkJ2PWrl1FXcGnQJaZNU1q7nMP?=\n\t=?utf-8?q?u45465jX2bSYJq7WU8yLTwwy+krvq6K7Ldw341WltJv9pByQq1e93fGBpQi0DFBz4?=\n\t=?utf-8?q?sF+kAY9jvLr2jiAnk5oxcRFC33z2IwS2T7yjhqa2vipLpxhds4ULMSf28P2etdRow?=\n\t=?utf-8?q?w9RyezBsmfssxmm12FV3LwPhlKBY7lsqO8ECfREbw1eRrfWvvNazvlJWNPjPvewe9?=\n\t=?utf-8?q?RrofjjUbPUnUdFHI8BRrGJ/Evw20bP/z/D/HL2aux8ZFYW3deHQzrS+9ni8tI10bB?=\n\t=?utf-8?q?XnFam6tvZjR6Xj8SD0NV9vOVqV4g4F5lfokHq58In1xzu1+ECc1A/oOPuAFe/3MzL?=\n\t=?utf-8?q?XjK0+b6EO5xDeIB1W26Wp73FvSvaxLcNK5mVP8Y2sNHv8GApxwNUBSYQ7S74Knq5a?=\n\t=?utf-8?q?lGbYrygcpinF18c/KxDOqWZ4u1BJhbg7RiMq7AMdP1xMvt4lTKokmknRCAatncfSS?=\n\t=?utf-8?q?dJaTqo53kBV8RG3aM/JSZ94wQzx66mNVYMmVURKM6a/mzC5Og6FZCyMJOUMDIIZks?=\n\t=?utf-8?q?s7yEOfIhKlu3j8nymmY0ItCpaEAAElpbbM6BztmMYmNrHFYg5dr4u5i6lPyHfElEj?=\n\t=?utf-8?q?U14uQ6ezQrNL/oxQb/kTVqp2zxFsd+unAk3djqm55Ihu71cDy8jjrH9ZiBwAMwFR7?=\n\t=?utf-8?q?wcjbXmJlOCiipGX/VEL2t1jOOBwH8RywYT5rSAQ/4VOWJh9O3EwIvi11P5Cvnmn8a?=\n\t=?utf-8?q?tMObHf4esb1BsTG2hJVEiGLCYbuRpakH2X7eeOhcSj4h0lW5qrXGaViNudfeIBLL7?=\n\t=?utf-8?q?MfjJEgfufsSFvP/X0o1IfwV69BIGsCFIkIbJQ0/Jy/z7ck0L8mEqVBC+4uriAyhME?=\n\t=?utf-8?q?AY61txGMQi4hjnk/rlqzEN4bguiiy0a64ACL/eyUCHA2XMPx1FWjyOggUjQN1JIZR?=\n\t=?utf-8?q?QJ4u+aX3L9hy4ndmWg11Gs5Y4fS71NGs9LXBsw9dKd/HGCyLCY4ACHDL6OEMlGoxo?=\n\t=?utf-8?q?DkwFAjuOlFq7vaNzDZ7dz4sc/j9M3ZsYdRQ=3D=3D?=","X-Forefront-Antispam-Report":"CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;\n IPV:NLI; SFV:NSPM; H:IA1PR10MB7447.namprd10.prod.outlook.com; PTR:; CAT:NONE;\n SFS:(13230040)(366016)(7416014)(376014)(1800799024); DIR:OUT; SFP:1101;","X-MS-Exchange-AntiSpam-MessageData-ChunkCount":"1","X-MS-Exchange-AntiSpam-MessageData-0":"=?utf-8?q?Aqt3Q05cPO3ZMi1kuln5aQehtz9n?=\n\t=?utf-8?q?rvktXQ+l9sx1gCFXdqhYsmVeJ59W3qetbNbPFeemchO87XyJOWVTPlCpNTgQmjiGI?=\n\t=?utf-8?q?X3PtJcW7FRcARHhBeC9c+H2asviZFy+2gsljm1QkUVd5IEpLrK0Isg7tlK/4MTSgw?=\n\t=?utf-8?q?wDKmIY1jFB/fsudLHXcDGwtpETbUBNxFyiRPVjB5ldMZWANiH8F+QA3Z+g8Qni0+z?=\n\t=?utf-8?q?tBTU4u691YQvSfHiRu/piXMqqR9SqKVJJmpQAj4q80FszvPvzNej1RjPsaROxOjMa?=\n\t=?utf-8?q?ZdHblgH7OW8B3zDhtv8qKuWgRHxby515WTKB8xDPRPhbl5dqzaL0a+1uQ9xUNSE7A?=\n\t=?utf-8?q?mQfEmekbQjVyffd7hSARt1284Ij+6BfgMIJ1aJsp9V9t/H2Abev9b1ALLZRBg/eji?=\n\t=?utf-8?q?Nr0I6VJdnEuAiDEvrsNw1RNtz8uDGPUGPfYw5fEX4ppVL/xHj1Ay9BunWjulT2e+y?=\n\t=?utf-8?q?5/pqSAR04yztd7iax7AON/FJOg/afFVkhu8UJm/QdJ7yZANU4KfPOAoETcNI4veAI?=\n\t=?utf-8?q?zMNqWWBNHE+mhcDf92GI0vqS0pygTCZLUHS1o5SY03fvuPNCzLZIq0BAOhsltdMRr?=\n\t=?utf-8?q?r5VyMvEUaxrovAtRltT2QoKskithGCUn/ICKhF1tGCYyrB45M2l+xMzg3rmPymyKr?=\n\t=?utf-8?q?fdXuXJqOegqsU/Zjrz/dXnZdrwKPHyGYl3ccci6zi7MeNQMqm8CPIfB/lw8/bFzcE?=\n\t=?utf-8?q?Q2Bqe/Eg0oxLZPfyD0MMEPFTmZ0ioy7E1I5m9zcHmXYN63c/ZMsOnIvWNQn+euh6j?=\n\t=?utf-8?q?32+Bn/96j1LK8ZtJddfVVy9nEx8TKhube7TE7T1HU0eJOcebOVuzjjwRpCb8Dq92S?=\n\t=?utf-8?q?2HuPnPEFzFRZqF+IBaU5NIeUVzCci6xtdzvwZ5he4/rX3+PWraJyxHcScK20xl04N?=\n\t=?utf-8?q?AKVlvcfkvLFMia0TGmiBupxama7Bp5z0ErQRx6fIKfK81Y3cCyoBmJIAQGcxSbIK3?=\n\t=?utf-8?q?UkuLUWzs+Eo4owo3KIbUNtAqfYDKH8FieZHanjSCuJMOEhInsUpeYGFAwhQwuu0cZ?=\n\t=?utf-8?q?5jsMaRh/ke+Y9iWbAggqILI2DF59URy1kNjCfloaG85YYrlUTVEtVBltTdYG/Qgu8?=\n\t=?utf-8?q?2aH53WRct/UH0/adfYN4M25DjLbf259Bp6ttiNPlFLzQOw1s0Df7vamYQr511hjth?=\n\t=?utf-8?q?2S1o4LNjNJMaEV8/LOphK40Rvw7czy04WidmHqWwzWJBz2Y3MTE1FV/RMnl72HF9k?=\n\t=?utf-8?q?vFeVkjxYX2BGlZ2eFHPloXeYn8wp07DI+ESPa+XqEwcbL6IZKC14zS81WcowEpe9M?=\n\t=?utf-8?q?YRQs5ozqmnIu7S0dR1ZDaFTwak7eTImD2xrTjJ2vAbm0aTpUmV25unSThIWnr/PCa?=\n\t=?utf-8?q?DguF6iqGe3qkZ8GzUyKStQ//rBvYEQH/eMgofK7D1p5M29hWS5QasuarHKdDRX3/a?=\n\t=?utf-8?q?liPD709qtel1oCYbAW+tPzqwsAhRaZhRW7vq8+/3uoJ00xQwthIX17pIFWdYJjtBD?=\n\t=?utf-8?q?oyv7lWB+FLdcGRNga93ez9AGhw8mw+RC8Eukdnsg562Vd974uKfYSaCgvkluU4xMC?=\n\t=?utf-8?q?dZ9dOacw21YWq0W3Su2BBK+dNr/Z+/3E/w=3D=3D?=","X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount":"1","X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0":"\n AqhFg3HRQL1Q7m1O5o65kI4oL1F5DtWxxdr0+IdaBSciLCaAjlTHU/dKIPFSU44GmLvDo5usJqmz0v4XbAapDtckcISz2wh85FVgb1aV7V5CCOoRSxVpmaZWyZDIpNx57D1z9P5Dwr2K9WngTlj0v9HJDtvqyfWjZC7zEEC9N6EEOQmnNTpHPR46NDPFAwRlbAozTT5+0tOHXAJJIWAklPkGiItSQ8+WdLXWMkoqkFRnibS9wVwUi4Sco7ZqVQvONX9B2MK+uKChmtglFP/eMnfyvDfAg2JtxHoUdLodq70pQhscw3GKDK6X1Mnt9Sr9b5Or9peETxvfE2tdf6F+hyqdy1mEwxTO+dCOz+R7e3zSfCGAKsx+kbLrnkeP/aGCT7mK7BnwHRLVpgr3OQSewbo6cwC4uccHQ+1DhRl9uWulio8MaeBKTBfXVV3hj/WCsDQZEHM4EypQAo85ug80fkWqQfag1Pq8zQHSprDpJM2e5pPQVkrHH1fOuQ8OnRm2qp8LOcKeCrO7IDa41fN2cqyP9fJ8H2cC0EtG98p6SS43QFW283LjY6pnIs4MW5fb0welwTWQEl94yIF9GeYQjzV4cUFJ6T8X9ZEsDpyh0DU=","X-OriginatorOrg":"oracle.com","X-MS-Exchange-CrossTenant-Network-Message-Id":"\n 52e0e17d-cab4-49aa-19e2-08dcbd6c898d","X-MS-Exchange-CrossTenant-AuthSource":"IA1PR10MB7447.namprd10.prod.outlook.com","X-MS-Exchange-CrossTenant-AuthAs":"Internal","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"15 Aug 2024 20:55:04.3378 (UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"4e2c6054-71cb-48f1-bd6c-3a9705aca71b","X-MS-Exchange-CrossTenant-MailboxType":"HOSTED","X-MS-Exchange-CrossTenant-UserPrincipalName":"\n +OZPTCuRS7XdLf3Slv0gdUXz8FqmHvF18lG2Tv7oJYgpw9tyCAMm+FJ9qWjJEtBOZEAf5P80xDGRpU68dbSpxnQQGtVpAJvt3+JouEpk7So=","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"DS0PR10MB6126","X-Proofpoint-Virus-Version":"vendor=baseguard\n engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16\n definitions=2024-08-15_12,2024-08-15_01,2024-05-17_01","X-Proofpoint-Spam-Details":"rule=notspam policy=default score=0 bulkscore=0\n mlxscore=0 adultscore=0\n suspectscore=0 mlxlogscore=999 phishscore=0 spamscore=0 malwarescore=0\n classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2407110000\n definitions=main-2408150152","X-Proofpoint-GUID":"jN-fueiGuEllNwioI3UDJAySkRPbwS9s","X-Proofpoint-ORIG-GUID":"jN-fueiGuEllNwioI3UDJAySkRPbwS9s","Received-SPF":"pass client-ip=205.220.177.32;\n envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com","X-Spam_score_int":"-27","X-Spam_score":"-2.8","X-Spam_bar":"--","X-Spam_report":"(-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,\n DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001,\n SPF_HELO_NONE=0.001, SPF_PASS=-0.001,\n T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}},{"id":3363466,"web_url":"http://patchwork.ozlabs.org/comment/3363466/","msgid":"","list_archive_url":null,"date":"2024-08-16T14:43:23","subject":"Re: [PATCH V2 06/11] migration: fix mismatched GPAs during cpr","submitter":{"id":67717,"url":"http://patchwork.ozlabs.org/api/people/67717/","name":"Peter Xu","email":"peterx@redhat.com"},"content":"On Thu, Aug 15, 2024 at 04:54:58PM -0400, Steven Sistare wrote:\n> On 8/13/2024 4:43 PM, Peter Xu wrote:\n> > On Wed, Aug 07, 2024 at 05:04:26PM -0400, Steven Sistare wrote:\n> > > On 7/19/2024 12:28 PM, Peter Xu wrote:\n> > > > On Sun, Jun 30, 2024 at 12:40:29PM -0700, Steve Sistare wrote:\n> > > > > For new cpr modes, ramblock_is_ignored will always be true, because the\n> > > > > memory is preserved in place rather than copied. However, for an ignored\n> > > > > block, parse_ramblock currently requires that the received address of the\n> > > > > block must match the address of the statically initialized region on the\n> > > > > target. This fails for a PCI rom block, because the memory region address\n> > > > > is set when the guest writes to a BAR on the source, which does not occur\n> > > > > on the target, causing a \"Mismatched GPAs\" error during cpr migration.\n> > > > \n> > > > Is this a common fix with/without cpr mode?\n> > > > \n> > > > It looks to me mr->addr (for these ROMs) should only be set in PCI config\n> > > > region updates as you mentioned. But then I didn't figure out when they're\n> > > > updated on dest in live migration: the ramblock info was sent at the\n> > > > beginning of migration, so it doesn't even have PCI config space migrated;\n> > > > I thought the real mr->addr should be in there.\n> > > > \n> > > > I also failed to understand yet on why the mr->addr check needs to be done\n> > > > by ignore-shared only. Some explanation would be greatly helpful around\n> > > > this area..\n> > > \n> > > The error_report does not bite for normal migration because migrate_ram_is_ignored()\n> > > is false for the problematic blocks, so the block->mr->addr check is not\n> > > performed. However, mr->addr is never fixed up in this case, which is a\n> > > quiet potential bug, and this patch fixes that with the \"has_addr\" check.\n> > > \n> > > For cpr-exec, migrate_ram_is_ignored() is true for all blocks,\n> > > because we do not copy the contents over the migration stream, we preserve the\n> > > memory in place. So we fall into the block->mr->addr sanity check and fail\n> > > with the original code.\n> > \n> > OK I get your point now. However this doesn't look right, instead I start\n> > to question why we need to send mr->addr at all..\n> > \n> > As I said previously, AFAIU mr->addr should only be updated when there's\n> > some PCI config space updates so that it moves the MR around in the address\n> > space based on how guest drivers / BIOS (?) set things up. Now after these\n> > days not looking, and just started to look at this again, I think the only\n> > sane place to do this update is during a post_load().\n> > \n> > And if we start to check some of the memory_region_set_address() users,\n> > that's exactly what happened..\n> > \n> > - ich9_pm_iospace_update(), update addr for ICH9LPCPMRegs.io, where\n> > ich9_pm_post_load() also invokes it.\n> > \n> > - pm_io_space_update(), updates PIIX4PMState.io, where\n> > vmstate_acpi_post_load() also invokes it.\n> > \n> > I stopped here just looking at the initial two users, it looks all sane to\n> > me that it only got updated there, because the update requires pci config\n> > space being migrated first.\n> > \n> > IOW, I don't think having mismatched mr->addr is wrong at this stage.\n> > Instead, I don't see why we should send mr->addr at all in this case during\n> > as early as SETUP, and I don't see anything justifies the mr->addr needs to\n> > be verified in parse_ramblock() since ignore-shared introduced by Yury in\n> > commit fbd162e629aaf8 in 2019.\n> > \n> > We can't drop mr->addr now when it's on-wire, but I think we should drop\n> > the error report and addr check, instead of this patch.\n> \n> As it turns out, my test case triggers this bug because it sets x-ignore-shared,\n> but x-ignore-shared is not needed for cpr-exec, because migrate_ram_is_ignored\n> is true for all blocks when mode==cpr-exec. So, the best fix for the GPAs bug\n> for me is to stop setting x-ignore-shared. I will drop this patch.\n> \n> I agree that post_load is the right place to restore mr->addr, and I don't\n> understand why commit fbd162e629aaf8 added the error report, but I am going\n> to leave it as is.\n\nAh, I didn't notice that cpr special cased migrate_ram_is_ignored()..\n\nShall we stick with the old check, but always require cpr to rely on\nignore-shared?\n\nThen we replace this patch with removing the error_report, probably\ntogether with not caring about whatever is received at all.. would that be\ncleaner?\n\nThanks,","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=mimecast20190719 header.b=GyCyjgQV;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4WllBN3Yr1z1yfB\n\tfor ; Sat, 17 Aug 2024 00:44:28 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1seyB4-0006C5-8x; Fri, 16 Aug 2024 10:43:42 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1seyAv-0006BO-50\n for qemu-devel@nongnu.org; Fri, 16 Aug 2024 10:43:33 -0400","from us-smtp-delivery-124.mimecast.com ([170.10.129.124])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1seyAs-0003EW-0y\n for qemu-devel@nongnu.org; Fri, 16 Aug 2024 10:43:31 -0400","from mail-qv1-f71.google.com (mail-qv1-f71.google.com\n [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS\n (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n us-mta-433-7fothGcIMZCWbG6ee7Ss1A-1; Fri, 16 Aug 2024 10:43:26 -0400","by mail-qv1-f71.google.com with SMTP id\n 6a1803df08f44-6b7678caf7dso5880566d6.1\n for ; Fri, 16 Aug 2024 07:43:26 -0700 (PDT)","from x1n (pool-99-254-121-117.cpe.net.cable.rogers.com.\n [99.254.121.117]) by smtp.gmail.com with ESMTPSA id\n 6a1803df08f44-6bf6fdd9336sm17937786d6.23.2024.08.16.07.43.24\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Fri, 16 Aug 2024 07:43:25 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n s=mimecast20190719; t=1723819408;\n h=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n to:to:cc:cc:mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=4ySH+14f6tKskyENhcKqAqt56ugMrjuhVTnIWMiyRHc=;\n b=GyCyjgQVUW6e0aYRcFUSS/OjS4KuOum0Js2C+n+7sleZ7vkTVuxXseBTcFx2xkSJ1gEVk5\n kr2c6ndaBueFLXbQywT8OQTSDo7I4VeCdG+stntEfotBisvrRVsRzmnuxjejEGIu3cArnT\n s4fXrULzamgYYy+KxDJznsEwjGJLLQE=","X-MC-Unique":"7fothGcIMZCWbG6ee7Ss1A-1","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20230601; t=1723819406; x=1724424206;\n h=in-reply-to:content-disposition:mime-version:references:message-id\n :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date\n :message-id:reply-to;\n bh=4ySH+14f6tKskyENhcKqAqt56ugMrjuhVTnIWMiyRHc=;\n b=RSQ6Mths9T5tGMu2WQHFA4HzN0kpjgyG6beCIXe6Gb6LEn7y1DYX52b9BR7NZnBRZI\n Iy1/Bwp5+OkaVGIvYx8mKS4Ej2ERV+CRds2Bio4hHfqaSUh2YDMNBmDg5tA+AN8+H1Io\n LEw92FnqKJ5+bTQWg5xeYPRCUpaEEHbZmxZJNAeOpGhL+dJx9aW60cQXM32c6A4i+6k6\n 25Ofzpe1eyPd33QVfEOovvGKg3VMTnh3wubIaf8zTAh8MFRoZIvpqEyZK7e5H9UkpBgh\n 7sENQbF3AlSP27CXA0hr5CXzgmCnkesLz4V3o3pQ2l4LNqn9rvFBKchJy5+HqJw9SHOw\n qalw==","X-Gm-Message-State":"AOJu0Yyk56wGZZVrqQ2mvsFjy/7uvoEX0AfHJekKDNsfu59qzWv28RXx\n lgqDywp2zk5JN66v68Po/8vV/KJQ8qg/v9BxvNAV24Iqi9Qqc8gi8fNFpfwMguJCnUmhWlveOEu\n w4nK5fW69zuY2vbsenboRkwSnBrOgtklsmUIkLTbBQSh5WILuIoeo","X-Received":["by 2002:a05:6214:e8c:b0:6b4:fda5:88b6 with SMTP id\n 6a1803df08f44-6bf7cd912f4mr19740796d6.1.1723819406057;\n Fri, 16 Aug 2024 07:43:26 -0700 (PDT)","by 2002:a05:6214:e8c:b0:6b4:fda5:88b6 with SMTP id\n 6a1803df08f44-6bf7cd912f4mr19740586d6.1.1723819405592;\n Fri, 16 Aug 2024 07:43:25 -0700 (PDT)"],"X-Google-Smtp-Source":"\n AGHT+IEO/38Bybm+YA8pozloyqkzfIZdvgTZpX+9FP2dQ3YwPfcQpYSDTXaO+32TxK2ovbfyrhZUFQ==","Date":"Fri, 16 Aug 2024 10:43:23 -0400","From":"Peter Xu ","To":"Steven Sistare ","Cc":"qemu-devel@nongnu.org, Fabiano Rosas ,\n David Hildenbrand ,\n Marcel Apfelbaum ,\n Eduardo Habkost ,\n Philippe Mathieu-Daude ,\n Paolo Bonzini ,\n \"Daniel P. Berrange\" ,\n Markus Armbruster ","Subject":"Re: [PATCH V2 06/11] migration: fix mismatched GPAs during cpr","Message-ID":"","References":"<1719776434-435013-1-git-send-email-steven.sistare@oracle.com>\n <1719776434-435013-7-git-send-email-steven.sistare@oracle.com>\n \n <571a4f84-693c-43d4-a43a-52a53a1091e1@oracle.com>\n \n <5f763763-1479-4585-98ce-83fcec03b4db@oracle.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<5f763763-1479-4585-98ce-83fcec03b4db@oracle.com>","Received-SPF":"pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com;\n helo=us-smtp-delivery-124.mimecast.com","X-Spam_score_int":"-21","X-Spam_score":"-2.2","X-Spam_bar":"--","X-Spam_report":"(-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.131,\n DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001,\n SPF_HELO_NONE=0.001, SPF_PASS=-0.001,\n T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}},{"id":3363581,"web_url":"http://patchwork.ozlabs.org/comment/3363581/","msgid":"","list_archive_url":null,"date":"2024-08-16T17:10:02","subject":"Re: [PATCH V2 06/11] migration: fix mismatched GPAs during cpr","submitter":{"id":71906,"url":"http://patchwork.ozlabs.org/api/people/71906/","name":"Steve Sistare","email":"steven.sistare@oracle.com"},"content":"On 8/16/2024 10:43 AM, Peter Xu wrote:\n> On Thu, Aug 15, 2024 at 04:54:58PM -0400, Steven Sistare wrote:\n>> On 8/13/2024 4:43 PM, Peter Xu wrote:\n>>> On Wed, Aug 07, 2024 at 05:04:26PM -0400, Steven Sistare wrote:\n>>>> On 7/19/2024 12:28 PM, Peter Xu wrote:\n>>>>> On Sun, Jun 30, 2024 at 12:40:29PM -0700, Steve Sistare wrote:\n>>>>>> For new cpr modes, ramblock_is_ignored will always be true, because the\n>>>>>> memory is preserved in place rather than copied. However, for an ignored\n>>>>>> block, parse_ramblock currently requires that the received address of the\n>>>>>> block must match the address of the statically initialized region on the\n>>>>>> target. This fails for a PCI rom block, because the memory region address\n>>>>>> is set when the guest writes to a BAR on the source, which does not occur\n>>>>>> on the target, causing a \"Mismatched GPAs\" error during cpr migration.\n>>>>>\n>>>>> Is this a common fix with/without cpr mode?\n>>>>>\n>>>>> It looks to me mr->addr (for these ROMs) should only be set in PCI config\n>>>>> region updates as you mentioned. But then I didn't figure out when they're\n>>>>> updated on dest in live migration: the ramblock info was sent at the\n>>>>> beginning of migration, so it doesn't even have PCI config space migrated;\n>>>>> I thought the real mr->addr should be in there.\n>>>>>\n>>>>> I also failed to understand yet on why the mr->addr check needs to be done\n>>>>> by ignore-shared only. Some explanation would be greatly helpful around\n>>>>> this area..\n>>>>\n>>>> The error_report does not bite for normal migration because migrate_ram_is_ignored()\n>>>> is false for the problematic blocks, so the block->mr->addr check is not\n>>>> performed. However, mr->addr is never fixed up in this case, which is a\n>>>> quiet potential bug, and this patch fixes that with the \"has_addr\" check.\n>>>>\n>>>> For cpr-exec, migrate_ram_is_ignored() is true for all blocks,\n>>>> because we do not copy the contents over the migration stream, we preserve the\n>>>> memory in place. So we fall into the block->mr->addr sanity check and fail\n>>>> with the original code.\n>>>\n>>> OK I get your point now. However this doesn't look right, instead I start\n>>> to question why we need to send mr->addr at all..\n>>>\n>>> As I said previously, AFAIU mr->addr should only be updated when there's\n>>> some PCI config space updates so that it moves the MR around in the address\n>>> space based on how guest drivers / BIOS (?) set things up. Now after these\n>>> days not looking, and just started to look at this again, I think the only\n>>> sane place to do this update is during a post_load().\n>>>\n>>> And if we start to check some of the memory_region_set_address() users,\n>>> that's exactly what happened..\n>>>\n>>> - ich9_pm_iospace_update(), update addr for ICH9LPCPMRegs.io, where\n>>> ich9_pm_post_load() also invokes it.\n>>>\n>>> - pm_io_space_update(), updates PIIX4PMState.io, where\n>>> vmstate_acpi_post_load() also invokes it.\n>>>\n>>> I stopped here just looking at the initial two users, it looks all sane to\n>>> me that it only got updated there, because the update requires pci config\n>>> space being migrated first.\n>>>\n>>> IOW, I don't think having mismatched mr->addr is wrong at this stage.\n>>> Instead, I don't see why we should send mr->addr at all in this case during\n>>> as early as SETUP, and I don't see anything justifies the mr->addr needs to\n>>> be verified in parse_ramblock() since ignore-shared introduced by Yury in\n>>> commit fbd162e629aaf8 in 2019.\n>>>\n>>> We can't drop mr->addr now when it's on-wire, but I think we should drop\n>>> the error report and addr check, instead of this patch.\n>>\n>> As it turns out, my test case triggers this bug because it sets x-ignore-shared,\n>> but x-ignore-shared is not needed for cpr-exec, because migrate_ram_is_ignored\n>> is true for all blocks when mode==cpr-exec. So, the best fix for the GPAs bug\n>> for me is to stop setting x-ignore-shared. I will drop this patch.\n>>\n>> I agree that post_load is the right place to restore mr->addr, and I don't\n>> understand why commit fbd162e629aaf8 added the error report, but I am going\n>> to leave it as is.\n> \n> Ah, I didn't notice that cpr special cased migrate_ram_is_ignored()..\n> \n> Shall we stick with the old check, but always require cpr to rely on\n> ignore-shared?\n> \n> Then we replace this patch with removing the error_report, probably\n> together with not caring about whatever is received at all.. would that be\n> cleaner?\n\nmigrate_ram_is_ignored() is called in many places and must return true for\ncpr-exec/cpr-transfer, independently of migrate_ignore_shared. That logic\nmust remain as is.\n\nThe cleanest change is no change, just dropping this patch. I was just confused\nwhen I set x-ignore-shared for the test.\n\nHowever, if an unsuspecting user sets x-ignore-shared, it will trigger this error,\nso perhaps I should delete the error_report.\n\n- Steve","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256\n header.s=corp-2023-11-20 header.b=h5mWwNRm;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com\n header.a=rsa-sha256 header.s=selector2-oracle-onmicrosoft-com\n header.b=xkAAl5tM;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4WlpR00Jm1z20GF\n\tfor ; Sat, 17 Aug 2024 03:10:35 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1sf0T5-0004wf-9G; Fri, 16 Aug 2024 13:10:27 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1sf0T1-0004lo-OB\n for qemu-devel@nongnu.org; Fri, 16 Aug 2024 13:10:24 -0400","from mx0b-00069f02.pphosted.com ([205.220.177.32])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1sf0St-0007Ml-BL\n for qemu-devel@nongnu.org; Fri, 16 Aug 2024 13:10:22 -0400","from pps.filterd (m0246632.ppops.net [127.0.0.1])\n by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id\n 47GFiRPN013535;\n Fri, 16 Aug 2024 17:10:12 GMT","from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com\n (phxpaimrmta02.appoci.oracle.com [147.154.114.232])\n by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 40wyttn1xt-1\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);\n Fri, 16 Aug 2024 17:10:11 +0000 (GMT)","from pps.filterd\n (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1])\n by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19)\n with ESMTP id 47GFiAql003907; Fri, 16 Aug 2024 17:10:10 GMT","from nam10-bn7-obe.outbound.protection.outlook.com\n (mail-bn7nam10lp2045.outbound.protection.outlook.com [104.47.70.45])\n by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id\n 40wxncgwn9-1\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);\n Fri, 16 Aug 2024 17:10:10 +0000","from IA1PR10MB7447.namprd10.prod.outlook.com (2603:10b6:208:44c::10)\n by SJ2PR10MB7810.namprd10.prod.outlook.com (2603:10b6:a03:578::10)\n with Microsoft SMTP Server (version=TLS1_2,\n cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.9; Fri, 16 Aug\n 2024 17:10:07 +0000","from IA1PR10MB7447.namprd10.prod.outlook.com\n ([fe80::f2fe:d6c6:70c4:4572]) by IA1PR10MB7447.namprd10.prod.outlook.com\n ([fe80::f2fe:d6c6:70c4:4572%7]) with mapi id 15.20.7875.016; Fri, 16 Aug 2024\n 17:10:07 +0000"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=\n message-id:date:subject:to:cc:references:from:in-reply-to\n :content-type:content-transfer-encoding:mime-version; s=\n corp-2023-11-20; bh=4OvsUsY7ptVbii42cn9F70qR3LF7v2JP3P/u1Lae0A8=; b=\n h5mWwNRmUxwhiZHW6On5qnFUU5/MpAWtpYyyefrMbhjgZroN6ufwu52XWUOkdA31\n b3RKf6oHHN8yLYA0piMpIRrfvSI1tGE4J9LH7CUJJZepoxCAYVlzCtG1bCPZ40+k\n Qa9NGmOrkrpxMcpOVAAf7XWTU/hpx7+YuLblad/hZvUlp1oNJfPLycPwl0BR+yRh\n FsyMLz1mqjfKBrT0TC0VTClRhRGfNce9QqYnWnqZCIs7xHVQOKuqgbOkCWAisvMA\n JIxaRx2xg23b3QqlQpapa+JPAaGWPEloBAzs1Ltiho1+pxl5ALRlyhqVNGzt5z9w\n S01IHba/M9VH76NJz28azg==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;\n bh=4OvsUsY7ptVbii42cn9F70qR3LF7v2JP3P/u1Lae0A8=;\n b=xkAAl5tMx96a7BMNrJvmxm8k2JhuQvHNRoH/lEOJN4I5+yEI8wbGN4je6DYduJ4gXZnMwxFGlEfjy+CTo+UMPxXimqLpO+iGeFy4yEG2QFuSKf+bkK/eZdNB7JVX+0sQxASiUKDnOfmWu8c3lTVqC6qE6L43wqPFOLsg+jymBgc="],"ARC-Seal":"i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none;\n b=aFsMF6gyYWfhAJ8S2GwaPVThe3Y372QukBGk994xd4ASOu1DhHpQcb9Sx3+KrFvXa11wiK0/JDZLIwj5qDKLG2VTIRh44VdhOUkHfF/6zDN0Hl/fFvXsqeAHFYK8J27xdJBvf25ztYQpTbdh/5S8vX6eF5jO5ulHwIM6GYu86sGSBP0AIOqXwp3PXMFns1GfLP8JDfDIM/ujUyJ543H2xci+3Wgmyf8S7BBQ0RTYZTXBCS+viPlQAS3OJloasMcfyn82hI0+sNjuYJ9KHj7954gQJP1Gv/XXgS6SroyRQ6ab0SaIZipfz/RhgBP10Pt0gZWWkjvGhoI1cg9Tq6MLPg==","ARC-Message-Signature":"i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;\n s=arcselector10001;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;\n bh=4OvsUsY7ptVbii42cn9F70qR3LF7v2JP3P/u1Lae0A8=;\n b=f047HMEQZDvYUQTjacTmgfw+ebQDuZzeTmLs6UD8R12NubKA87jEfZK1w1edK6fvKVVP/0jiGC7AgegBwdkWh4aBMs00glancNSZ+sOdrGuR2PpZ6jgYVvY6dvs1pYEq6NKMhlw4qsLC90BZR/fdUKQEzlmjAVxkK+KdGi6G5ANuHy3c3aWQLfD1w5Ct444WXRfa4LnSJdK1m7tydCdmYfVb9TpYRs2JgVZ2OFGWg3m7pTsCqhJJnBysQRIuI8xMaZRSyL/HE55ikKeAQRxuaVVSafq0MPtfegetFBRhYSe+YPLp1hQOIVTV2TvCa6KoKp1RGjaBVcnjiV5yYwhWXA==","ARC-Authentication-Results":"i=1; mx.microsoft.com 1; spf=pass\n smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;\n dkim=pass header.d=oracle.com; arc=none","Message-ID":"","Date":"Fri, 16 Aug 2024 13:10:02 -0400","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH V2 06/11] migration: fix mismatched GPAs during cpr","To":"Peter Xu ","Cc":"qemu-devel@nongnu.org, Fabiano Rosas ,\n David Hildenbrand ,\n Marcel Apfelbaum ,\n Eduardo Habkost ,\n Philippe Mathieu-Daude ,\n Paolo Bonzini , \"Daniel P. Berrange\"\n , Markus Armbruster ","References":"<1719776434-435013-1-git-send-email-steven.sistare@oracle.com>\n <1719776434-435013-7-git-send-email-steven.sistare@oracle.com>\n <571a4f84-693c-43d4-a43a-52a53a1091e1@oracle.com>\n <5f763763-1479-4585-98ce-83fcec03b4db@oracle.com>\n ","Content-Language":"en-US","From":"Steven Sistare ","Organization":"Oracle Corporation","In-Reply-To":"","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","X-ClientProxiedBy":"SJ0PR13CA0096.namprd13.prod.outlook.com\n (2603:10b6:a03:2c5::11) To IA1PR10MB7447.namprd10.prod.outlook.com\n (2603:10b6:208:44c::10)","MIME-Version":"1.0","X-MS-PublicTrafficType":"Email","X-MS-TrafficTypeDiagnostic":"IA1PR10MB7447:EE_|SJ2PR10MB7810:EE_","X-MS-Office365-Filtering-Correlation-Id":"eb186ec7-8896-4af6-6d1f-08dcbe164716","X-MS-Exchange-SenderADCheck":"1","X-MS-Exchange-AntiSpam-Relay":"0","X-Microsoft-Antispam":"BCL:0;ARA:13230040|1800799024|7416014|376014|366016;","X-Microsoft-Antispam-Message-Info":"=?utf-8?q?sX+Z78bnswcCtxJCGt70EO5pKBpeqtJ?=\n\t=?utf-8?q?IxyTmvB4iUC900E18VNLRd5FTMQDJcZm+iJTtjvzCp866L2n3tvNQ1C8XP309bO68?=\n\t=?utf-8?q?wxCqeK2nQGJQUAp+t/57XfhJOVmP/L5QYlVb4sXgaJI4ap1e8jX+Odg3D+S2pUeF8?=\n\t=?utf-8?q?FOzZrQN2S5EGOMewN5aBjtP1Pj3ruM2PTFuVjfRLPRh6VBIrHVN0CKy4LMTflgrJQ?=\n\t=?utf-8?q?DvoGpfn89dK9MiQi+t8dKbkk6N6XIUcixs6Ipqf+HTPaAv++ksgz0iZM9Ieb77zI0?=\n\t=?utf-8?q?NX58lSMAT5E5qCoQ3rSSFPQNmesk2BTtwxAbFFTv9rFB33PRgYGo2rGcPhFuab135?=\n\t=?utf-8?q?kGJEOU5v9URO8ATAbcJDlbRZq/fkF57VI2WXQnv0gGqeb+SsM8Wtw/KC1moR3w/aP?=\n\t=?utf-8?q?Pz4PjqE4dVEk1KeK82v3YdQXfDxEOKAKt9oGzutAhkc7mEZf7L9Br/qQ4ttO6xSpB?=\n\t=?utf-8?q?0hvUOsUSR/0V8vMXOV6ewbpZuZfCE92dIBjpAMDuTPJhVvcxXzOItrEL0WOPsrVX6?=\n\t=?utf-8?q?WMIkg7+GukEsoFhyBvjCvQF7kFdWMEyylmvYmQ5jFQm7nhS3YNLHAgqsrZlUSgM41?=\n\t=?utf-8?q?NOVHS4B8Rp/BP8ZjAmPAKpmpVSz5x0d6G2GvyhjgMKby3zPQ4t1GeYd09DetnhIfr?=\n\t=?utf-8?q?EP0W/1L0G4XzrRmhAJ69UxR+gtfimeO+WIfZ9ANOp3Pnj7SQggHvXcLYYvEGd7gLB?=\n\t=?utf-8?q?wPn+CErDv/juGfGttEwCCnOZDtNyAnNi4Scju7fTm1uuuLj8EuhauJIWGbAkn5iSX?=\n\t=?utf-8?q?dzgydHRomRRtZ9xG7OE/HZWt45ilfhp8OF4kqCw4G7d0vaadhZi4wc6RlJBNugrw2?=\n\t=?utf-8?q?lcB09Y17Ns2gpKYr2JNnDIryp2llgH2AiquZX5gls1l+W41JyII+oroNlnWTchPJF?=\n\t=?utf-8?q?ke6fJbNeSunxYfidmJliDStvmstsHMZaAQdIdk4u6keGI5OCtAQlqOWMvjTQMjALQ?=\n\t=?utf-8?q?h9nY+Ykyys2LN0sglGhb1FIPL3VnwMUqtfok/h0DRN4VymvH+EE7VTis+8XlpgYSy?=\n\t=?utf-8?q?spXbB/m2uAjB0jBDsZWSDG/xr93Dt8TTHpB4G3izHoS0cKOKO4l33/QI6duqe2EgD?=\n\t=?utf-8?q?Z/ZOuy4SG/JfWqNXiTP96zIQsgdLYomoZl1FGiVJN4r+UP/R+Un+9YQfX537NzvKv?=\n\t=?utf-8?q?w5/YVsG8DuKoq5Vr7CekBrCe2yYqLXJF3EA3qbKioiOUQd+EovQi29ro+ISZSSJSx?=\n\t=?utf-8?q?qxDVv58Nkr7kb8PJrFTMfZGI7fUiUXpu1ylI9JQ2JATRsrPM0AjN0WVg+T0swNsLd?=\n\t=?utf-8?q?XWquYgbt1/4BxU317A7G180TraLrIV7MxzA=3D=3D?=","X-Forefront-Antispam-Report":"CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;\n IPV:NLI; SFV:NSPM; H:IA1PR10MB7447.namprd10.prod.outlook.com; PTR:; CAT:NONE;\n SFS:(13230040)(1800799024)(7416014)(376014)(366016); DIR:OUT; SFP:1101;","X-MS-Exchange-AntiSpam-MessageData-ChunkCount":"1","X-MS-Exchange-AntiSpam-MessageData-0":"=?utf-8?q?CmfdXagdjdytTx5gyh9XL17LOMl7?=\n\t=?utf-8?q?ybA3VEMEPsxmugWT+bqMYr4KPXf2QJA6fw7tWINL8L0xXbgaKEVUwWOYg9SXzMzzO?=\n\t=?utf-8?q?UIzLLs6WVE/fng8aSx9U85hZGuNaRkQvFewMJJuxtGiKPoIwlDbO5NSM3VnSGjXHM?=\n\t=?utf-8?q?vCgEYBIxczFpfQ7/hzahy+KXAXm5KP0WIxE4ZmCNY90TbjZE5ZoSHk8ALMja1HMOJ?=\n\t=?utf-8?q?bnYjz5YOIS5OURuGPbD4J5EeaRImWR5unZS29IN0CbFDoqzXKk3SX87pm8eDeshWy?=\n\t=?utf-8?q?aKQ9PYcH3EtdsWoqUtjTHLCfDMzjdOSkbLvhjYJ03z2LO2XIhfVkRYp111r+AHPEZ?=\n\t=?utf-8?q?X9vX9wbHrWVyIJ21FrPbek9ZNhQc8u7z26BuXFQk3XCzEeHd63CFfyvXUqWuqzEsH?=\n\t=?utf-8?q?iZRqax0R8kURzi1oBfdrvc12QIfsKVufVaTHcxMLnX4N/guLbNCxHetwUkk1bs+zE?=\n\t=?utf-8?q?/XXOW0vETR9fjCCVQKkJNpSmm2toXcv4qhla4W/AmpfpuuBK2DcLM8m5L2+nqxVoq?=\n\t=?utf-8?q?5p01h1eFL5/2/jEIWLn6IAhjojXvHziDs+gdu6GxAQqwHGYQhkOLKbqZkkvVq0p2p?=\n\t=?utf-8?q?SpoS/D7dtB8rCnibbRyaip8degOQimsZh2g9ocNUzhb5ZAEA8hDwrTJ2mRuvfirOi?=\n\t=?utf-8?q?TaknOiGhzGrh5i3iWY8dLRzv62Ia7Y6zZ+eFb+kJ25NVzOL/nQVRoKsbIUdaVc1T4?=\n\t=?utf-8?q?bs7f2Jny1y7azRC6evx+8ciSGg1YrOjvZ2r/1JNhk9KOIYKIb8tEDLPfO0PkwR//t?=\n\t=?utf-8?q?DjdoCtQ2uY5qN8s3ByhRwEaxv+bPipFGsISG1EY02rTMtZuTWORSxxbgZgx8tKTjg?=\n\t=?utf-8?q?B3KwXm6CLMev/8uuuTkHAS6ychpscb5RiVDsJ8LfjLc5qOben5Ra3CoCjhYX+qlDL?=\n\t=?utf-8?q?pjdadoamK0G5La8V3qBxWcNAjBE3JlthIpwwdTEL9EwY0asGeiKnqHqWFD9bDk8Qp?=\n\t=?utf-8?q?uAfDKRK6BF13Os9oNdg11k+BDethXr3iIA6oh3Kund1GAgnLHdwVUPGc4PLS0sz8H?=\n\t=?utf-8?q?+xttm8TISZiaPPeHAVMQl1YoEDnPodv5qa0U/ok5a0zYedfehxSyJKGCWG0fr0n2p?=\n\t=?utf-8?q?ljfhP8Cyq2w+GizUu10Q/EJG0uD5u3t5bIWYXuLwtQFc+FMVmrIYwf+e11AGOm7kF?=\n\t=?utf-8?q?0lkmuGXb7fCijOV6oiTk3PmIaoyKOn1wQOVfNJDe0VQyDpnKLUBK92eZ4CWgy+817?=\n\t=?utf-8?q?MAEiAXKMhQu4hDg8Pa57C3aFG4W16tEp4XqDGfvF85khVeZvnCxu1snk+QyBkwmJ/?=\n\t=?utf-8?q?P1Te8wZOSGW700mLgEd4fpKa6lJOcYpNlgncKMD7TJXBnFNrgj6zpqsddqUwiK9/Z?=\n\t=?utf-8?q?b5qnkafQSOaBuuiEGy9PlqUy1PlidqjO3/nAs9YrdLbeesGufeoI1fI9z6Ss/53Q6?=\n\t=?utf-8?q?ME/TrEd+U1yJkVbfC2VDvUDlmoPz7zvk/1K3acMV1dY3iWDOTkDbOgN09N/4yu31V?=\n\t=?utf-8?q?ukS3R64L3Re2YNDODByX22xbJhwmbTniXlveB/xyKNLSiFs5dfcXvZ0ucrOVr1CSK?=\n\t=?utf-8?q?FqiPBk8B/QtpmX9zmt4bnPdJcMbjyl3Nfw=3D=3D?=","X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount":"1","X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0":"\n MHmKqhuKtyHGDr12W9kUwFCp31F5mNVqOYlqltjnVQI10+VC9hZXH77ZLR3WVvthB4hCplKchaK7YuOYvvynXnTXX+wLQDQffYJdFS1axLVfxNVRdb7GbyjhoHsZMfCWxUWG/RzM1xuQOcJlzRqmorZFXvRYoJqCy35DiLXRoEFoseMfeOrfIZf4JfGau9r5zOba7u3sCqvD4Vg1QtKyRkV4XXVxvSS2we0p1Px5ZbJti1aKUMVF1+Qz+PfAi6yuivr5sStOGd7LknrkFQ7n44Uoh2dXRQHv+/+lAMxTKKSyZTLTd3KfLSb0YRp0tFQcDeovM4pVpn9Tg+2HbAmKCusmC/CSQ8qwBs0pYlPpRszuigaj9W7zVqgBclklF7bmjafVXQlWm5J09TTRq1RfLN/eCv8AsKN8uOnhnLyy4bt3dcr63czHU6oapzfbBMPF2msnFY6rBKmYLgRsPwmbLukTVEWUOt4wHlQxTE5jmkO9vU1ekgM0LHHGd//W130C2l+2QJ9NLdVXiIovqbmHzekiwzldsW8+FX4Vl1MDB8F2UJ2IE/LEeGKXKA+EGlSqnMGaLxPZLWyf2ekUanbYCjeqQ2IQflnKgfgtdkCih8A=","X-OriginatorOrg":"oracle.com","X-MS-Exchange-CrossTenant-Network-Message-Id":"\n eb186ec7-8896-4af6-6d1f-08dcbe164716","X-MS-Exchange-CrossTenant-AuthSource":"IA1PR10MB7447.namprd10.prod.outlook.com","X-MS-Exchange-CrossTenant-AuthAs":"Internal","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"16 Aug 2024 17:10:07.2474 (UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"4e2c6054-71cb-48f1-bd6c-3a9705aca71b","X-MS-Exchange-CrossTenant-MailboxType":"HOSTED","X-MS-Exchange-CrossTenant-UserPrincipalName":"\n AxHwCAeow1NDe7ZuKvjdYaL4qdhGUz7vwpshYbBdC8p3pr82xGhmir7l/pWQxEAx3CE5E2U+eUe7zRI44VHubiMmJSOG6xwNzKm99t95FtU=","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"SJ2PR10MB7810","X-Proofpoint-Virus-Version":"vendor=baseguard\n engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16\n definitions=2024-08-16_11,2024-08-16_01,2024-05-17_01","X-Proofpoint-Spam-Details":"rule=notspam policy=default score=0 spamscore=0\n suspectscore=0\n mlxlogscore=999 phishscore=0 bulkscore=0 malwarescore=0 mlxscore=0\n adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1\n engine=8.12.0-2407110000 definitions=main-2408160121","X-Proofpoint-ORIG-GUID":"MdplCWKJKO_HGMkHC9DBN81qWr-4XKNG","X-Proofpoint-GUID":"MdplCWKJKO_HGMkHC9DBN81qWr-4XKNG","Received-SPF":"pass client-ip=205.220.177.32;\n envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com","X-Spam_score_int":"-27","X-Spam_score":"-2.8","X-Spam_bar":"--","X-Spam_report":"(-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,\n DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001,\n SPF_HELO_NONE=0.001, SPF_PASS=-0.001,\n T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}},{"id":3365897,"web_url":"http://patchwork.ozlabs.org/comment/3365897/","msgid":"","list_archive_url":null,"date":"2024-08-21T16:57:27","subject":"Re: [PATCH V2 06/11] migration: fix mismatched GPAs during cpr","submitter":{"id":67717,"url":"http://patchwork.ozlabs.org/api/people/67717/","name":"Peter Xu","email":"peterx@redhat.com"},"content":"On Fri, Aug 16, 2024 at 01:10:02PM -0400, Steven Sistare wrote:\n> On 8/16/2024 10:43 AM, Peter Xu wrote:\n> > On Thu, Aug 15, 2024 at 04:54:58PM -0400, Steven Sistare wrote:\n> > > On 8/13/2024 4:43 PM, Peter Xu wrote:\n> > > > On Wed, Aug 07, 2024 at 05:04:26PM -0400, Steven Sistare wrote:\n> > > > > On 7/19/2024 12:28 PM, Peter Xu wrote:\n> > > > > > On Sun, Jun 30, 2024 at 12:40:29PM -0700, Steve Sistare wrote:\n> > > > > > > For new cpr modes, ramblock_is_ignored will always be true, because the\n> > > > > > > memory is preserved in place rather than copied. However, for an ignored\n> > > > > > > block, parse_ramblock currently requires that the received address of the\n> > > > > > > block must match the address of the statically initialized region on the\n> > > > > > > target. This fails for a PCI rom block, because the memory region address\n> > > > > > > is set when the guest writes to a BAR on the source, which does not occur\n> > > > > > > on the target, causing a \"Mismatched GPAs\" error during cpr migration.\n> > > > > > \n> > > > > > Is this a common fix with/without cpr mode?\n> > > > > > \n> > > > > > It looks to me mr->addr (for these ROMs) should only be set in PCI config\n> > > > > > region updates as you mentioned. But then I didn't figure out when they're\n> > > > > > updated on dest in live migration: the ramblock info was sent at the\n> > > > > > beginning of migration, so it doesn't even have PCI config space migrated;\n> > > > > > I thought the real mr->addr should be in there.\n> > > > > > \n> > > > > > I also failed to understand yet on why the mr->addr check needs to be done\n> > > > > > by ignore-shared only. Some explanation would be greatly helpful around\n> > > > > > this area..\n> > > > > \n> > > > > The error_report does not bite for normal migration because migrate_ram_is_ignored()\n> > > > > is false for the problematic blocks, so the block->mr->addr check is not\n> > > > > performed. However, mr->addr is never fixed up in this case, which is a\n> > > > > quiet potential bug, and this patch fixes that with the \"has_addr\" check.\n> > > > > \n> > > > > For cpr-exec, migrate_ram_is_ignored() is true for all blocks,\n> > > > > because we do not copy the contents over the migration stream, we preserve the\n> > > > > memory in place. So we fall into the block->mr->addr sanity check and fail\n> > > > > with the original code.\n> > > > \n> > > > OK I get your point now. However this doesn't look right, instead I start\n> > > > to question why we need to send mr->addr at all..\n> > > > \n> > > > As I said previously, AFAIU mr->addr should only be updated when there's\n> > > > some PCI config space updates so that it moves the MR around in the address\n> > > > space based on how guest drivers / BIOS (?) set things up. Now after these\n> > > > days not looking, and just started to look at this again, I think the only\n> > > > sane place to do this update is during a post_load().\n> > > > \n> > > > And if we start to check some of the memory_region_set_address() users,\n> > > > that's exactly what happened..\n> > > > \n> > > > - ich9_pm_iospace_update(), update addr for ICH9LPCPMRegs.io, where\n> > > > ich9_pm_post_load() also invokes it.\n> > > > \n> > > > - pm_io_space_update(), updates PIIX4PMState.io, where\n> > > > vmstate_acpi_post_load() also invokes it.\n> > > > \n> > > > I stopped here just looking at the initial two users, it looks all sane to\n> > > > me that it only got updated there, because the update requires pci config\n> > > > space being migrated first.\n> > > > \n> > > > IOW, I don't think having mismatched mr->addr is wrong at this stage.\n> > > > Instead, I don't see why we should send mr->addr at all in this case during\n> > > > as early as SETUP, and I don't see anything justifies the mr->addr needs to\n> > > > be verified in parse_ramblock() since ignore-shared introduced by Yury in\n> > > > commit fbd162e629aaf8 in 2019.\n> > > > \n> > > > We can't drop mr->addr now when it's on-wire, but I think we should drop\n> > > > the error report and addr check, instead of this patch.\n> > > \n> > > As it turns out, my test case triggers this bug because it sets x-ignore-shared,\n> > > but x-ignore-shared is not needed for cpr-exec, because migrate_ram_is_ignored\n> > > is true for all blocks when mode==cpr-exec. So, the best fix for the GPAs bug\n> > > for me is to stop setting x-ignore-shared. I will drop this patch.\n> > > \n> > > I agree that post_load is the right place to restore mr->addr, and I don't\n> > > understand why commit fbd162e629aaf8 added the error report, but I am going\n> > > to leave it as is.\n> > \n> > Ah, I didn't notice that cpr special cased migrate_ram_is_ignored()..\n> > \n> > Shall we stick with the old check, but always require cpr to rely on\n> > ignore-shared?\n> > \n> > Then we replace this patch with removing the error_report, probably\n> > together with not caring about whatever is received at all.. would that be\n> > cleaner?\n> \n> migrate_ram_is_ignored() is called in many places and must return true for\n> cpr-exec/cpr-transfer, independently of migrate_ignore_shared. That logic\n> must remain as is.\n\nIs this because cpr can fail some ramblock in qemu_ram_is_named_file()?\n\nIt's not obvious in this case, maybe some re-strcture would be nice. Would\nsomething like this look nicer and easier to understand?\n\n===8<===\ndiff --git a/migration/ram.c b/migration/ram.c\nindex 1e1e05e859..ace635b167 100644\n--- a/migration/ram.c\n+++ b/migration/ram.c\n@@ -214,14 +214,29 @@ static bool postcopy_preempt_active(void)\n return migrate_postcopy_preempt() && migration_in_postcopy();\n }\n \n-bool migrate_ram_is_ignored(RAMBlock *block)\n+/* Whether the destination QEMU can share the access on this ramblock? */\n+bool migrate_ram_is_shared(RAMBlock *block)\n {\n MigMode mode = migrate_mode();\n+\n+ /* Private ram is never share-able */\n+ if (!qemu_ram_is_shared(block)) {\n+ return false;\n+ }\n+\n+ /* Named file ram is always assumed to be share-able */\n+ if (qemu_ram_is_named_file(block)) {\n+ return true;\n+ }\n+\n+ /* It's a private fd, only cpr mode can share it (by sharing fd) */\n+ return (mode == MIG_MODE_CPR_EXEC) || (mode == MIG_MODE_CPR_TRANSFER);\n+}\n+\n+bool migrate_ram_is_ignored(RAMBlock *block)\n+{\n return !qemu_ram_is_migratable(block) ||\n- mode == MIG_MODE_CPR_EXEC ||\n- mode == MIG_MODE_CPR_TRANSFER ||\n- (migrate_ignore_shared() && qemu_ram_is_shared(block)\n- && qemu_ram_is_named_file(block));\n+ (migrate_ignore_shared() && migrate_ram_is_shared(block));\n }\n===8<===\n\nPlease feel free to squash this to your patch in whatever way if it looks\nreasonable to you.\n\n> \n> The cleanest change is no change, just dropping this patch. I was just confused\n> when I set x-ignore-shared for the test.\n> \n> However, if an unsuspecting user sets x-ignore-shared, it will trigger this error,\n> so perhaps I should delete the error_report.\n\nYes, feel free to send that as a separate patch if you want, since we\ndigged it this far it'll be nice we fix it even if it's not relevant now.\n\nThanks,","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=mimecast20190719 header.b=Lgj90RST;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4Wpsw109LCz1yYZ\n\tfor ; Thu, 22 Aug 2024 02:57:52 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1sgoeS-0001qR-VF; Wed, 21 Aug 2024 12:57:40 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1sgoeR-0001pf-1J\n for qemu-devel@nongnu.org; Wed, 21 Aug 2024 12:57:39 -0400","from us-smtp-delivery-124.mimecast.com ([170.10.133.124])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1sgoeO-00016O-Fw\n for qemu-devel@nongnu.org; Wed, 21 Aug 2024 12:57:38 -0400","from mail-qk1-f199.google.com (mail-qk1-f199.google.com\n [209.85.222.199]) by relay.mimecast.com with ESMTP with STARTTLS\n (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n us-mta-36-3wgfYhH6OjC6SSi2wCRLiA-1; Wed, 21 Aug 2024 12:57:33 -0400","by mail-qk1-f199.google.com with SMTP id\n af79cd13be357-7a1d4335cceso182713485a.1\n for ; Wed, 21 Aug 2024 09:57:32 -0700 (PDT)","from x1n (pool-99-254-121-117.cpe.net.cable.rogers.com.\n [99.254.121.117]) by smtp.gmail.com with ESMTPSA id\n af79cd13be357-7a4ff02505dsm644662285a.13.2024.08.21.09.57.29\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Wed, 21 Aug 2024 09:57:30 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n s=mimecast20190719; t=1724259454;\n h=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n to:to:cc:cc:mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=72nVWtKKC2sfvJi0dQHZuwXl9Lj7ZDv0etuqaJN5224=;\n b=Lgj90RSTEAy8di336b4XrPyelhI8iM/QCj7xKOqXoqUa3mB4kyJIPPd/Cdk2B62DmESp4Z\n tnVz2rqzXZcT5Hdho6TqJFYsLMxzmo6bb3MBGqvkvrVyB9mrMKmFOvekaMkm4vnTezln4S\n fR1CiFX2fXcS96UoO27viNmUZt+Rf3U=","X-MC-Unique":"3wgfYhH6OjC6SSi2wCRLiA-1","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20230601; t=1724259451; x=1724864251;\n h=in-reply-to:content-disposition:mime-version:references:message-id\n :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date\n :message-id:reply-to;\n bh=72nVWtKKC2sfvJi0dQHZuwXl9Lj7ZDv0etuqaJN5224=;\n b=GQjHoL5z1wsXR8iQjDYAWlXwDeH9GlZ5UQ2DwDe9LII2+91mW4xsu46+mrmIJyyn78\n SiScGPsqL3IkM7+BkYGJIClF75rmTvpRGcxSuj2xfB7kubrIxPfO8JCZqP2DLCa40E3b\n xLF+876zNQtGxVRA7Onqw1Vklt+YFJysSDVXBxR2WHU0hapF1qTOEFGCy0kNwJDdBgmO\n oD1DeI65rFgM+O1nPMKa4t2+zcODAK/VWY3mJTAz0D+GnBhhTs8n/TqbpwBSCVSfjFSn\n C92AlzzK61OPHJC5ykDU6WFwo7A72VA0okqgjFhnOLihIdXaX+0CYthRxA0MA+925vrA\n zJDA==","X-Gm-Message-State":"AOJu0YxJhgojk1PtqHXE6p4+iP7hRAE0L3TZMDSPlu8LhQiru/QJj3ld\n AKz0D4u9nJkOff2/5LdS8exHDybOxpvSKrPWt9vcuajXXdK2VyBfKn1fzaJwV+PzQ7kdEvi3bXz\n WRxKqXbENqdsm4glqAO4BuSlzm0vdZts66dF4xRSD4UMkJGu/dIpD","X-Received":["by 2002:a05:620a:2802:b0:79f:17f4:7154 with SMTP id\n af79cd13be357-7a67d3e2bd4mr43128385a.3.1724259451440;\n Wed, 21 Aug 2024 09:57:31 -0700 (PDT)","by 2002:a05:620a:2802:b0:79f:17f4:7154 with SMTP id\n af79cd13be357-7a67d3e2bd4mr43123885a.3.1724259450961;\n Wed, 21 Aug 2024 09:57:30 -0700 (PDT)"],"X-Google-Smtp-Source":"\n AGHT+IEaPJP7V3tmnF9VdcWHFtYulr+E6l6Ls8mJnQ6SIrJwjmiv3rgdgdeMOSeovbfJ+METZll24A==","Date":"Wed, 21 Aug 2024 12:57:27 -0400","From":"Peter Xu ","To":"Steven Sistare ","Cc":"qemu-devel@nongnu.org, Fabiano Rosas ,\n David Hildenbrand ,\n Marcel Apfelbaum ,\n Eduardo Habkost ,\n Philippe Mathieu-Daude ,\n Paolo Bonzini ,\n \"Daniel P. Berrange\" ,\n Markus Armbruster ","Subject":"Re: [PATCH V2 06/11] migration: fix mismatched GPAs during cpr","Message-ID":"","References":"<1719776434-435013-1-git-send-email-steven.sistare@oracle.com>\n <1719776434-435013-7-git-send-email-steven.sistare@oracle.com>\n \n <571a4f84-693c-43d4-a43a-52a53a1091e1@oracle.com>\n \n <5f763763-1479-4585-98ce-83fcec03b4db@oracle.com>\n \n ","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"","Received-SPF":"pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com;\n helo=us-smtp-delivery-124.mimecast.com","X-Spam_score_int":"-21","X-Spam_score":"-2.2","X-Spam_bar":"--","X-Spam_report":"(-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.138,\n DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001,\n RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,\n SPF_HELO_NONE=0.001, SPF_PASS=-0.001,\n T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}}]