From patchwork Tue Jun 12 20:56:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 928494 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amazon.com header.i=@amazon.com header.b="OKizW0Rw"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 4152L8648qz9s1B for ; Wed, 13 Jun 2018 06:58:56 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934271AbeFLU4k (ORCPT ); Tue, 12 Jun 2018 16:56:40 -0400 Received: from smtp-fw-2101.amazon.com ([72.21.196.25]:31566 "EHLO smtp-fw-2101.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933693AbeFLU4f (ORCPT ); Tue, 12 Jun 2018 16:56:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1528836995; x=1560372995; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=NpIAFvQ6O9nnG5WOiv9Pr/VSUltr4pf4KmI8tMYXerk=; b=OKizW0RwKKKLT13YiQIQKPszfkVX0m337+iyXGbVB6qdiPdkrX4JDVY0 Uh121Gge6B+RW4FuaU3DacNm7RsElkYUxlu6ULmnqvKnh84CdtGKNRgEU Dc/JvKtCRHyZlgmfDBm3Xfwgjk+fIYWNQbkyiLrt84rSJAsMznfp27aLM o=; X-IronPort-AV: E=Sophos;i="5.51,216,1526342400"; d="scan'208";a="682679273" Received: from iad6-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-2b-c7131dcf.us-west-2.amazon.com) ([10.124.125.2]) by smtp-border-fw-out-2101.iad2.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 12 Jun 2018 20:56:32 +0000 Received: from EX13MTAUWB001.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan3.pdx.amazon.com [10.236.137.198]) by email-inbound-relay-2b-c7131dcf.us-west-2.amazon.com (8.14.7/8.14.7) with ESMTP id w5CKuT1H117272 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Tue, 12 Jun 2018 20:56:29 GMT Received: from EX13D05UWB004.ant.amazon.com (10.43.161.208) by EX13MTAUWB001.ant.amazon.com (10.43.161.207) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:26 +0000 Received: from EX13MTAUWB001.ant.amazon.com (10.43.161.207) by EX13D05UWB004.ant.amazon.com (10.43.161.208) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:26 +0000 Received: from localhost (10.25.15.63) by mail-relay.amazon.com (10.43.161.249) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Tue, 12 Jun 2018 20:56:26 +0000 From: Anchal Agarwal To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [RFC PATCH 01/12] xen/manage: keep track of the on-going suspend mode Date: Tue, 12 Jun 2018 20:56:08 +0000 Message-ID: <20180612205619.28156-2-anchalag@amazon.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180612205619.28156-1-anchalag@amazon.com> References: <20180612205619.28156-1-anchalag@amazon.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Munehisa Kamata To differentiate between Xen suspend, PM suspend and PM hibernation, keep track of the on-going suspend mode by mainly using a new PM notifier. Since Xen suspend doesn't have corresponding PM event, its main logic is modfied to acquire pm_mutex and set the current mode. Note that we may see deadlock if PM suspend/hibernation is interrupted by Xen suspend. PM suspend/hibernation depends on xenwatch thread to process xenbus state transactions, but the thread will sleep to wait pm_mutex which is already held by PM suspend/hibernation context in the scenario. Though, acquirng pm_mutex is still right thing to do, and we would need to modify Xen shutdown code to avoid the issue. This will be fixed by a separate patch. Signed-off-by: Munehisa Kamata Signed-off-by: Anchal Agarwal Reviewed-by: Sebastian Biemueller Reviewed-by: Munehisa Kamata Reviewed-by: Eduardo Valentin --- drivers/xen/manage.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c index 8835065..8f9ea87 100644 --- a/drivers/xen/manage.c +++ b/drivers/xen/manage.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include @@ -39,6 +40,16 @@ enum shutdown_state { /* Ignore multiple shutdown requests. */ static enum shutdown_state shutting_down = SHUTDOWN_INVALID; +enum suspend_modes { + NO_SUSPEND = 0, + XEN_SUSPEND, + PM_SUSPEND, + PM_HIBERNATION, +}; + +/* Protected by pm_mutex */ +static enum suspend_modes suspend_mode = NO_SUSPEND; + struct suspend_info { int cancelled; }; @@ -98,6 +109,10 @@ static void do_suspend(void) int err; struct suspend_info si; + lock_system_sleep(); + + suspend_mode = XEN_SUSPEND; + shutting_down = SHUTDOWN_SUSPEND; err = freeze_processes(); @@ -161,6 +176,10 @@ static void do_suspend(void) thaw_processes(); out: shutting_down = SHUTDOWN_INVALID; + + suspend_mode = NO_SUSPEND; + + unlock_system_sleep(); } #endif /* CONFIG_HIBERNATE_CALLBACKS */ @@ -372,3 +391,42 @@ int xen_setup_shutdown_event(void) EXPORT_SYMBOL_GPL(xen_setup_shutdown_event); subsys_initcall(xen_setup_shutdown_event); + +static int xen_pm_notifier(struct notifier_block *notifier, + unsigned long pm_event, void *unused) +{ + switch (pm_event) { + case PM_SUSPEND_PREPARE: + suspend_mode = PM_SUSPEND; + break; + case PM_HIBERNATION_PREPARE: + case PM_RESTORE_PREPARE: + suspend_mode = PM_HIBERNATION; + break; + case PM_POST_SUSPEND: + case PM_POST_RESTORE: + case PM_POST_HIBERNATION: + /* Set back to the default */ + suspend_mode = NO_SUSPEND; + break; + default: + pr_warn("Receive unknown PM event 0x%lx\n", pm_event); + return -EINVAL; + } + + return 0; +}; + +static struct notifier_block xen_pm_notifier_block = { + .notifier_call = xen_pm_notifier +}; + +static int xen_setup_pm_notifier(void) +{ + if (!xen_hvm_domain()) + return -ENODEV; + + return register_pm_notifier(&xen_pm_notifier_block); +} + +subsys_initcall(xen_setup_pm_notifier); From patchwork Tue Jun 12 20:56:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 928498 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amazon.com header.i=@amazon.com header.b="p0+Os1tn"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 4152ND6879z9s1R for ; Wed, 13 Jun 2018 07:00:43 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934141AbeFLU4h (ORCPT ); Tue, 12 Jun 2018 16:56:37 -0400 Received: from smtp-fw-33001.amazon.com ([207.171.190.10]:6029 "EHLO smtp-fw-33001.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933685AbeFLU4f (ORCPT ); Tue, 12 Jun 2018 16:56:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1528836995; x=1560372995; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=0F1GfjZPIl4k7H/vkwsKAphOOqwq532rc/1rL93kMSg=; b=p0+Os1tnFnJSGqqvMjoccerrRGBaBNQdiuvTiXFKfe3Fek42U6N0adRI Ine32Q4uELpXO5qTJA3nrw8My1v3RJConGWWEMB+DSIyEX/a/hNY6b5q7 04mncv1byNXY4LaiNd7BcUqy81TEPELvK30csxb1YuNuAshQGbygu+ZFx k=; X-IronPort-AV: E=Sophos;i="5.51,216,1526342400"; d="scan'208";a="736713942" Received: from sea3-co-svc-lb6-vlan2.sea.amazon.com (HELO email-inbound-relay-2b-c7131dcf.us-west-2.amazon.com) ([10.47.22.34]) by smtp-border-fw-out-33001.sea14.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 12 Jun 2018 20:56:32 +0000 Received: from EX13MTAUWB001.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan3.pdx.amazon.com [10.236.137.198]) by email-inbound-relay-2b-c7131dcf.us-west-2.amazon.com (8.14.7/8.14.7) with ESMTP id w5CKuT1J117272 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Tue, 12 Jun 2018 20:56:32 GMT Received: from EX13D16UWB003.ant.amazon.com (10.43.161.194) by EX13MTAUWB001.ant.amazon.com (10.43.161.249) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:28 +0000 Received: from EX13MTAUEE001.ant.amazon.com (10.43.62.200) by EX13D16UWB003.ant.amazon.com (10.43.161.194) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:27 +0000 Received: from localhost (10.25.15.63) by mail-relay.amazon.com (10.43.62.226) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Tue, 12 Jun 2018 20:56:27 +0000 From: Anchal Agarwal To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [RFC PATCH 02/12] xen/manage: introduce helper function to know the on-going suspend mode Date: Tue, 12 Jun 2018 20:56:09 +0000 Message-ID: <20180612205619.28156-3-anchalag@amazon.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180612205619.28156-1-anchalag@amazon.com> References: <20180612205619.28156-1-anchalag@amazon.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Munehisa Kamata Introduce simple functions which help to know the on-going suspend mode so that other Xen-related code can behave differently according to the current suspend mode. Signed-off-by: Munehisa Kamata Signed-off-by: Anchal Agarwal Reviewed-by: Alakesh Haloi Reviewed-by: Sebastian Biemueller Reviewed-by: Munehisa Kamata Reviewed-by: Eduardo Valentin --- drivers/xen/manage.c | 15 +++++++++++++++ include/xen/xen-ops.h | 4 ++++ 2 files changed, 19 insertions(+) diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c index 8f9ea87..326631d 100644 --- a/drivers/xen/manage.c +++ b/drivers/xen/manage.c @@ -50,6 +50,21 @@ enum suspend_modes { /* Protected by pm_mutex */ static enum suspend_modes suspend_mode = NO_SUSPEND; +bool xen_suspend_mode_is_xen_suspend(void) +{ + return suspend_mode == XEN_SUSPEND; +} + +bool xen_suspend_mode_is_pm_suspend(void) +{ + return suspend_mode == PM_SUSPEND; +} + +bool xen_suspend_mode_is_pm_hibernation(void) +{ + return suspend_mode == PM_HIBERNATION; +} + struct suspend_info { int cancelled; }; diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h index fd23e42..be78f6f 100644 --- a/include/xen/xen-ops.h +++ b/include/xen/xen-ops.h @@ -39,6 +39,10 @@ u64 xen_steal_clock(int cpu); int xen_setup_shutdown_event(void); +bool xen_suspend_mode_is_xen_suspend(void); +bool xen_suspend_mode_is_pm_suspend(void); +bool xen_suspend_mode_is_pm_hibernation(void); + extern unsigned long *xen_contiguous_bitmap; #ifdef CONFIG_XEN_PV From patchwork Tue Jun 12 20:56:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 928500 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amazon.com header.i=@amazon.com header.b="DNfLe6M+"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 4152PG5PXfz9s1R for ; Wed, 13 Jun 2018 07:01:38 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934634AbeFLVB0 (ORCPT ); Tue, 12 Jun 2018 17:01:26 -0400 Received: from smtp-fw-6002.amazon.com ([52.95.49.90]:3344 "EHLO smtp-fw-6002.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933998AbeFLU4h (ORCPT ); Tue, 12 Jun 2018 16:56:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1528836996; x=1560372996; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=e5clyo9fSmITG1Y9ftbtx5PbXbkitZHATO9TO/fz9EM=; b=DNfLe6M+NBt3gP/XYxh5vw5Jw4yQSDZzo8CWEKTpS0krvI85sz8eZ3Gb d7Hdu8fBWslxwR63206GdTxqWLvT/ONP+gUEDa12ByVQRLoBU34PyzRuh 0CF8SF6lE3L5HmlR2Gd+EP3fEO1jEDtmk8uRtP9b4rgDYUErMYuUMdLwl A=; X-IronPort-AV: E=Sophos;i="5.51,216,1526342400"; d="scan'208";a="347699297" Received: from iad6-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-2b-c300ac87.us-west-2.amazon.com) ([10.124.125.6]) by smtp-border-fw-out-6002.iad6.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 12 Jun 2018 20:56:34 +0000 Received: from EX13MTAUWB001.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan3.pdx.amazon.com [10.236.137.198]) by email-inbound-relay-2b-c300ac87.us-west-2.amazon.com (8.14.7/8.14.7) with ESMTP id w5CKuVb6012052 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Tue, 12 Jun 2018 20:56:32 GMT Received: from EX13D13UWB002.ant.amazon.com (10.43.161.21) by EX13MTAUWB001.ant.amazon.com (10.43.161.207) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:29 +0000 Received: from EX13MTAUEE001.ant.amazon.com (10.43.62.200) by EX13D13UWB002.ant.amazon.com (10.43.161.21) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:29 +0000 Received: from localhost (10.25.15.63) by mail-relay.amazon.com (10.43.62.226) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Tue, 12 Jun 2018 20:56:28 +0000 From: Anchal Agarwal To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [RFC PATCH 03/12] xenbus: add freeze/thaw/restore callbacks support Date: Tue, 12 Jun 2018 20:56:10 +0000 Message-ID: <20180612205619.28156-4-anchalag@amazon.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180612205619.28156-1-anchalag@amazon.com> References: <20180612205619.28156-1-anchalag@amazon.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Munehisa Kamata Since commit b3e96c0c7562 ("xen: use freeze/restore/thaw PM events for suspend/resume/chkpt"), xenbus uses PMSG_FREEZE, PMSG_THAW and PMSG_RESTORE events for Xen suspend. However, they're actually assigned to xenbus_dev_suspend(), xenbus_dev_cancel() and xenbus_dev_resume() respectively, and only suspend and resume callbacks are supported at driver level. To support PM suspend and PM hibernation, modify the bus level PM callbacks to invoke not only device driver's suspend/resume but also freeze/thaw/restore. Note that we'll use freeze/restore callbacks even for PM suspend whereas suspend/resume callbacks are normally used in the case, becausae the existing xenbus device drivers already have suspend/resume callbacks specifically designed for Xen suspend. So we can allow the device drivers to keep the existing callbacks wihtout modification. Signed-off-by: Munehisa Kamata Signed-off-by: Anchal Agarwal Reviewed-by: Munehisa Kamata Reviewed-by: Eduardo Valentin --- drivers/xen/xenbus/xenbus_probe.c | 102 ++++++++++++++++++++++++++++++++------ include/xen/xenbus.h | 3 ++ 2 files changed, 89 insertions(+), 16 deletions(-) diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c index ec9eb4f..95b0a6d 100644 --- a/drivers/xen/xenbus/xenbus_probe.c +++ b/drivers/xen/xenbus/xenbus_probe.c @@ -49,6 +49,7 @@ #include #include #include +#include #include #include @@ -588,26 +589,47 @@ int xenbus_dev_suspend(struct device *dev) struct xenbus_driver *drv; struct xenbus_device *xdev = container_of(dev, struct xenbus_device, dev); + int (*cb)(struct xenbus_device *) = NULL; + bool xen_suspend = xen_suspend_mode_is_xen_suspend(); DPRINTK("%s", xdev->nodename); if (dev->driver == NULL) return 0; drv = to_xenbus_driver(dev->driver); - if (drv->suspend) - err = drv->suspend(xdev); - if (err) - pr_warn("suspend %s failed: %i\n", dev_name(dev), err); + + if (xen_suspend) + cb = drv->suspend; + else + cb = drv->freeze; + + if (cb) + err = cb(xdev); + + if (err) { + pr_warn("%s %s failed: %i\n", xen_suspend ? + "suspend" : "freeze", dev_name(dev), err); + return err; + } + + if (!xen_suspend) { + /* Forget otherend since this can become stale after restore */ + free_otherend_watch(xdev); + free_otherend_details(xdev); + } + return 0; } EXPORT_SYMBOL_GPL(xenbus_dev_suspend); int xenbus_dev_resume(struct device *dev) { - int err; + int err = 0; struct xenbus_driver *drv; struct xenbus_device *xdev = container_of(dev, struct xenbus_device, dev); + int (*cb)(struct xenbus_device *) = NULL; + bool xen_suspend = xen_suspend_mode_is_xen_suspend(); DPRINTK("%s", xdev->nodename); @@ -616,24 +638,34 @@ int xenbus_dev_resume(struct device *dev) drv = to_xenbus_driver(dev->driver); err = talk_to_otherend(xdev); if (err) { - pr_warn("resume (talk_to_otherend) %s failed: %i\n", + pr_warn("%s (talk_to_otherend) %s failed: %i\n", + xen_suspend ? "resume" : "restore", dev_name(dev), err); return err; } - xdev->state = XenbusStateInitialising; + if (xen_suspend) + xdev->state = XenbusStateInitialising; - if (drv->resume) { - err = drv->resume(xdev); - if (err) { - pr_warn("resume %s failed: %i\n", dev_name(dev), err); - return err; - } + if (xen_suspend) + cb = drv->resume; + else + cb = drv->restore; + + if (cb) + err = cb(xdev); + + if (err) { + pr_warn("%s %s failed: %i\n", + xen_suspend ? "resume" : "restore", + dev_name(dev), err); + return err; } err = watch_otherend(xdev); if (err) { - pr_warn("resume (watch_otherend) %s failed: %d.\n", + pr_warn("%s (watch_otherend) %s failed: %d.\n", + xen_suspend ? "resume" : "restore", dev_name(dev), err); return err; } @@ -644,8 +676,46 @@ EXPORT_SYMBOL_GPL(xenbus_dev_resume); int xenbus_dev_cancel(struct device *dev) { - /* Do nothing */ - DPRINTK("cancel"); + int err = 0; + struct xenbus_driver *drv; + struct xenbus_device *xdev + = container_of(dev, struct xenbus_device, dev); + bool xen_suspend = xen_suspend_mode_is_xen_suspend(); + + if (xen_suspend) { + /* Do nothing */ + DPRINTK("cancel"); + return 0; + } + + DPRINTK("%s", xdev->nodename); + + if (dev->driver == NULL) + return 0; + drv = to_xenbus_driver(dev->driver); + + err = talk_to_otherend(xdev); + if (err) { + pr_warn("thaw (talk_to_otherend) %s failed: %d.\n", + dev_name(dev), err); + return err; + } + + if (drv->thaw) { + err = drv->thaw(xdev); + if (err) { + pr_warn("thaw %s failed: %i\n", dev_name(dev), err); + return err; + } + } + + err = watch_otherend(xdev); + if (err) { + pr_warn("thaw (watch_otherend) %s failed: %d.\n", + dev_name(dev), err); + return err; + } + return 0; } EXPORT_SYMBOL_GPL(xenbus_dev_cancel); diff --git a/include/xen/xenbus.h b/include/xen/xenbus.h index 869c816..20261d5 100644 --- a/include/xen/xenbus.h +++ b/include/xen/xenbus.h @@ -100,6 +100,9 @@ struct xenbus_driver { int (*remove)(struct xenbus_device *dev); int (*suspend)(struct xenbus_device *dev); int (*resume)(struct xenbus_device *dev); + int (*freeze)(struct xenbus_device *dev); + int (*thaw)(struct xenbus_device *dev); + int (*restore)(struct xenbus_device *dev); int (*uevent)(struct xenbus_device *, struct kobj_uevent_env *); struct device_driver driver; int (*read_otherend_details)(struct xenbus_device *dev); From patchwork Tue Jun 12 20:56:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 928497 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amazon.com header.i=@amazon.com header.b="mEQuXVDa"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 4152N00yVzz9s1R for ; Wed, 13 Jun 2018 07:00:32 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934678AbeFLVAW (ORCPT ); Tue, 12 Jun 2018 17:00:22 -0400 Received: from smtp-fw-6002.amazon.com ([52.95.49.90]:3344 "EHLO smtp-fw-6002.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934091AbeFLU4h (ORCPT ); Tue, 12 Jun 2018 16:56:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1528836997; x=1560372997; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=wAt5tCtl8i934+pWMAzHn7TQ1fBj1MjwfTCjDanT1zU=; b=mEQuXVDav/ZQoovD3cJMr9HPly0mYofxJT0iN6hdFgNon0a9QVRfSr8i JTc0Do8NsbkpaWWjgTKJ44jHqwY5m1uXJWyiBwZLUd1UILFm9fgLcDtmL jhxeGU8RcYP4aPhFCnmn80uwPsvI6y3U/mEyCgpoAchFrba+aCVj7mxEk w=; X-IronPort-AV: E=Sophos;i="5.51,216,1526342400"; d="scan'208";a="347699295" Received: from iad6-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-2c-4e7c8266.us-west-2.amazon.com) ([10.124.125.6]) by smtp-border-fw-out-6002.iad6.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 12 Jun 2018 20:56:33 +0000 Received: from EX13MTAUWC001.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan3.pdx.amazon.com [10.236.137.198]) by email-inbound-relay-2c-4e7c8266.us-west-2.amazon.com (8.14.7/8.14.7) with ESMTP id w5CKuVEL129961 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Tue, 12 Jun 2018 20:56:31 GMT Received: from EX13D02UWC003.ant.amazon.com (10.43.162.199) by EX13MTAUWC001.ant.amazon.com (10.43.162.135) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:31 +0000 Received: from EX13MTAUEB001.ant.amazon.com (10.43.60.96) by EX13D02UWC003.ant.amazon.com (10.43.162.199) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:31 +0000 Received: from localhost (10.25.15.63) by mail-relay.amazon.com (10.43.60.129) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Tue, 12 Jun 2018 20:56:30 +0000 From: Anchal Agarwal To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [RFC PATCH 04/12] x86/xen: Introduce new function to map HYPERVISOR_shared_info on Resume Date: Tue, 12 Jun 2018 20:56:11 +0000 Message-ID: <20180612205619.28156-5-anchalag@amazon.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180612205619.28156-1-anchalag@amazon.com> References: <20180612205619.28156-1-anchalag@amazon.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Introduce a small function which re-uses shared page's PA allocated during guest initialization time in reserve_shared_info() and not allocate new page during resume flow. It also does the mapping of shared_info_page by calling xen_hvm_init_shared_info() to use the function. Signed-off-by: Anchal Agarwal Reviewed-by: Sebastian Biemueller Reviewed-by: Munehisa Kamata Reviewed-by: Eduardo Valentin CR: https://cr.amazon.com/r/8273203/ --- arch/x86/xen/enlighten_hvm.c | 7 +++++++ arch/x86/xen/xen-ops.h | 1 + 2 files changed, 8 insertions(+) diff --git a/arch/x86/xen/enlighten_hvm.c b/arch/x86/xen/enlighten_hvm.c index 754d5391d9fa..2c4bcf92a90a 100644 --- a/arch/x86/xen/enlighten_hvm.c +++ b/arch/x86/xen/enlighten_hvm.c @@ -24,6 +24,13 @@ static unsigned long shared_info_pfn; +void xen_hvm_map_shared_info(void) +{ + xen_hvm_init_shared_info(); + if (shared_info_pfn) + HYPERVISOR_shared_info = __va(PFN_PHYS(shared_info_pfn)); +} + void xen_hvm_init_shared_info(void) { struct xen_add_to_physmap xatp; diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h index f377e1820c6c..94c8a009ab35 100644 --- a/arch/x86/xen/xen-ops.h +++ b/arch/x86/xen/xen-ops.h @@ -58,6 +58,7 @@ void xen_enable_syscall(void); void xen_vcpu_restore(void); void xen_callback_vector(void); +void xen_hvm_map_shared_info(void); void xen_hvm_init_shared_info(void); void xen_unplug_emulated_devices(void); From patchwork Tue Jun 12 20:56:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 928499 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amazon.com header.i=@amazon.com header.b="DOLb/7Ig"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 4152Nj6PbVz9s3R for ; Wed, 13 Jun 2018 07:01:09 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934834AbeFLVBE (ORCPT ); Tue, 12 Jun 2018 17:01:04 -0400 Received: from smtp-fw-9102.amazon.com ([207.171.184.29]:1586 "EHLO smtp-fw-9102.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933637AbeFLU4h (ORCPT ); Tue, 12 Jun 2018 16:56:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1528836997; x=1560372997; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=aetijkW91Fp4XfXAl8frmA7sH4Dko/J8u338j2ApKUw=; b=DOLb/7IgN7/jcw4b0dlsfMpySBmoBkQnobhAVeQoUHmRlWQpUO5f0zHz 4krpSYw/1M2rVWznUnV35EP6m73X5dWpvf0PbxcbNYMjFDt7Yc6aoDwI6 XHrbuoebtXdHuYbbdZmI4GaKR2tZu+bn3il6NAqsE5WLu+mHrkbGfjd1F 0=; X-IronPort-AV: E=Sophos;i="5.51,216,1526342400"; d="scan'208";a="617032495" Received: from sea3-co-svc-lb6-vlan3.sea.amazon.com (HELO email-inbound-relay-2c-4e7c8266.us-west-2.amazon.com) ([10.47.22.38]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 12 Jun 2018 20:56:34 +0000 Received: from EX13MTAUWC001.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan3.pdx.amazon.com [10.236.137.198]) by email-inbound-relay-2c-4e7c8266.us-west-2.amazon.com (8.14.7/8.14.7) with ESMTP id w5CKuVEa129961 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Tue, 12 Jun 2018 20:56:34 GMT Received: from EX13D16UWC001.ant.amazon.com (10.43.162.117) by EX13MTAUWC001.ant.amazon.com (10.43.162.135) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:32 +0000 Received: from EX13MTAUWC001.ant.amazon.com (10.43.162.135) by EX13D16UWC001.ant.amazon.com (10.43.162.117) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:31 +0000 Received: from localhost (10.25.15.63) by mail-relay.amazon.com (10.43.162.232) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Tue, 12 Jun 2018 20:56:31 +0000 From: Anchal Agarwal To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [RFC PATCH 05/12] x86/xen: add system core suspend and resume callbacks Date: Tue, 12 Jun 2018 20:56:12 +0000 Message-ID: <20180612205619.28156-6-anchalag@amazon.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180612205619.28156-1-anchalag@amazon.com> References: <20180612205619.28156-1-anchalag@amazon.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Munehisa Kamata Add Xen PVHVM specific system core callbacks for PM suspend and hibernation support. The callbacks suspend and resume Xen primitives, like shared_info, pvclock and grant table. Note that Xen suspend can handle them in a different manner, but system core callbacks are called from the context. So if the callbacks are called from Xen suspend context, return immediately. Signed-off-by: Munehisa Kamata Signed-off-by: Anchal Agarwal Reviewed-by: Munehisa Kamata Reviewed-by: Eduardo Valentin --- arch/x86/xen/enlighten_hvm.c | 1 + arch/x86/xen/suspend.c | 53 ++++++++++++++++++++++++++++++++++++++++++++ include/xen/xen-ops.h | 2 ++ 3 files changed, 56 insertions(+) diff --git a/arch/x86/xen/enlighten_hvm.c b/arch/x86/xen/enlighten_hvm.c index d24ad16..4196a65 100644 --- a/arch/x86/xen/enlighten_hvm.c +++ b/arch/x86/xen/enlighten_hvm.c @@ -202,6 +202,7 @@ static void __init xen_hvm_guest_init(void) if (xen_feature(XENFEAT_hvm_callback_vector)) xen_have_vector_callback = 1; + xen_setup_syscore_ops(); xen_hvm_smp_init(); WARN_ON(xen_cpuhp_setup(xen_cpu_up_prepare_hvm, xen_cpu_dead_hvm)); xen_unplug_emulated_devices(); diff --git a/arch/x86/xen/suspend.c b/arch/x86/xen/suspend.c index 1d83152..784c448 100644 --- a/arch/x86/xen/suspend.c +++ b/arch/x86/xen/suspend.c @@ -2,17 +2,22 @@ #include #include #include +#include +#include #include #include +#include #include #include +#include #include #include #include #include #include +#include #include "xen-ops.h" #include "mmu.h" @@ -82,3 +87,51 @@ void xen_arch_suspend(void) on_each_cpu(xen_vcpu_notify_suspend, NULL, 1); } + +static int xen_syscore_suspend(void) +{ + struct xen_remove_from_physmap xrfp; + int ret; + + /* Xen suspend does similar stuffs in its own logic */ + if (xen_suspend_mode_is_xen_suspend()) + return 0; + + xrfp.domid = DOMID_SELF; + xrfp.gpfn = __pa(HYPERVISOR_shared_info) >> PAGE_SHIFT; + + ret = HYPERVISOR_memory_op(XENMEM_remove_from_physmap, &xrfp); + if (!ret) + HYPERVISOR_shared_info = &xen_dummy_shared_info; + + return ret; +} + +static void xen_syscore_resume(void) +{ + /* Xen suspend does similar stuffs in its own logic */ + if (xen_suspend_mode_is_xen_suspend()) + return; + + /* No need to setup vcpu_info as it's already moved off */ + xen_hvm_map_shared_info(); + + pvclock_resume(); + + gnttab_resume(); +} + +/* + * These callbacks will be called with interrupts disabled and when having only + * one CPU online. + */ +static struct syscore_ops xen_hvm_syscore_ops = { + .suspend = xen_syscore_suspend, + .resume = xen_syscore_resume +}; + +void __init xen_setup_syscore_ops(void) +{ + if (xen_hvm_domain()) + register_syscore_ops(&xen_hvm_syscore_ops); +} diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h index be78f6f..65f25bd 100644 --- a/include/xen/xen-ops.h +++ b/include/xen/xen-ops.h @@ -43,6 +43,8 @@ bool xen_suspend_mode_is_xen_suspend(void); bool xen_suspend_mode_is_pm_suspend(void); bool xen_suspend_mode_is_pm_hibernation(void); +void xen_setup_syscore_ops(void); + extern unsigned long *xen_contiguous_bitmap; #ifdef CONFIG_XEN_PV From patchwork Tue Jun 12 20:56:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 928495 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amazon.com header.i=@amazon.com header.b="RgH7n/gF"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 4152Lt2CJfz9s1R for ; Wed, 13 Jun 2018 06:59:34 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934708AbeFLU7B (ORCPT ); Tue, 12 Jun 2018 16:59:01 -0400 Received: from smtp-fw-9102.amazon.com ([207.171.184.29]:1586 "EHLO smtp-fw-9102.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934195AbeFLU4j (ORCPT ); Tue, 12 Jun 2018 16:56:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1528836998; x=1560372998; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=2y5dPCNZgeX5m6Sr6j1VUhCYPEOLggUV3bD8hHxBGF4=; b=RgH7n/gFi+Z1OVIc+/pHnT+nzoZDjq1Fv15JsABjQGNq+27UIKMIz8e+ trqbac8CPeyjuTaAtAUsmSuvIkxhFM0Svt5oT6yUCjHYK7H9AYXpLGSUT 906RR7XobXgfxbXGSoqQdresQWmbc3f0mjRnXy2L2pcWeDVq/JR9ig5Ba o=; X-IronPort-AV: E=Sophos;i="5.51,216,1526342400"; d="scan'208";a="617032505" Received: from sea3-co-svc-lb6-vlan3.sea.amazon.com (HELO email-inbound-relay-2c-579b7f5b.us-west-2.amazon.com) ([10.47.22.38]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 12 Jun 2018 20:56:38 +0000 Received: from EX13MTAUWA001.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan2.pdx.amazon.com [10.236.137.194]) by email-inbound-relay-2c-579b7f5b.us-west-2.amazon.com (8.14.7/8.14.7) with ESMTP id w5CKuWGu118041 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Tue, 12 Jun 2018 20:56:33 GMT Received: from EX13D10UWA001.ant.amazon.com (10.43.160.216) by EX13MTAUWA001.ant.amazon.com (10.43.160.118) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:32 +0000 Received: from EX13MTAUWA001.ant.amazon.com (10.43.160.58) by EX13D10UWA001.ant.amazon.com (10.43.160.216) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:31 +0000 Received: from localhost (10.25.15.63) by mail-relay.amazon.com (10.43.160.118) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Tue, 12 Jun 2018 20:56:31 +0000 From: Anchal Agarwal To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [RFC PATCH 06/12] xen-blkfront: add callbacks for PM suspend and hibernation Date: Tue, 12 Jun 2018 20:56:13 +0000 Message-ID: <20180612205619.28156-7-anchalag@amazon.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180612205619.28156-1-anchalag@amazon.com> References: <20180612205619.28156-1-anchalag@amazon.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Munehisa Kamata Add freeze and restore callbacks for PM suspend and hibernation support. The freeze handler stops a block-layer queue and disconnect the frontend from the backend while freeing ring_info and associated resources. The restore handler re-allocates ring_info and re-connect to the backedend, so the rest of the kernel can continue to use the block device transparently.Also, the handlers are used for both PM suspend and hibernation so that we can keep the existing suspend/resume callbacks for Xen suspend without modification. If a backend doesn't have commit 12ea729645ac ("xen/blkback: unmap all persistent grants when frontend gets disconnected"), the frontend may see massive amount of grant table warning when freeing resources. [ 36.852659] deferring g.e. 0xf9 (pfn 0xffffffffffffffff) [ 36.855089] xen:grant_table: WARNING: g.e. 0x112 still in use! In this case, persistent grants would need to be disabled. Ensure no reqs/rsps in rings before disconnecting. When disconnecting the frontend from the backend in blkfront_freeze(), there still may be unconsumed requests or responses in the rings, especially when the backend is backed by network-based device. If the frontend gets disconnected with such reqs/rsps remaining there, it can cause grant warnings and/or losing reqs/rsps by freeing pages afterward. This can lead resumed kernel into unrecoverable state like unexpected freeing of grant page and/or hung task due to the lost reqs or rsps. Therefore we have to ensure that there is no unconsumed requests or responses before disconnecting. Actually, the frontend just needs to wait for some amount of time so that the backend can process the requests, put responses and notify the frontend back. Timeout used here is based on some heuristic. If we somehow hit the timeout, it would mean something serious happens in the backend, the frontend will just return an error to PM core and PM suspend/hibernation will be aborted. This may be something should be fixed by the backend side, but a frontend side fix is probably still worth doing to work with broader backends. Signed-off-by: Anchal Agarwal Reviewed-by: Munehisa Kamata Reviewed-by: Eduardo Valentin --- drivers/block/xen-blkfront.c | 158 +++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 151 insertions(+), 7 deletions(-) diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c index ae00a82f350b..a223864c2220 100644 --- a/drivers/block/xen-blkfront.c +++ b/drivers/block/xen-blkfront.c @@ -46,6 +46,8 @@ #include #include #include +#include +#include #include #include @@ -78,6 +80,8 @@ enum blkif_state { BLKIF_STATE_DISCONNECTED, BLKIF_STATE_CONNECTED, BLKIF_STATE_SUSPENDED, + BLKIF_STATE_FREEZING, + BLKIF_STATE_FROZEN }; struct grant { @@ -216,6 +220,7 @@ struct blkfront_info /* Save uncomplete reqs and bios for migration. */ struct list_head requests; struct bio_list bio_list; + struct completion wait_backend_disconnected; }; static unsigned int nr_minors; @@ -262,6 +267,16 @@ static DEFINE_SPINLOCK(minor_lock); static int blkfront_setup_indirect(struct blkfront_ring_info *rinfo); static void blkfront_gather_backend_features(struct blkfront_info *info); static int negotiate_mq(struct blkfront_info *info); +static void __blkif_free(struct blkfront_info *info); + +static inline bool blkfront_ring_is_busy(struct blkif_front_ring *ring) +{ + if (RING_SIZE(ring) > RING_FREE_REQUESTS(ring) || + RING_HAS_UNCONSUMED_RESPONSES(ring)) + return true; + else + return false; +} static int get_id_from_freelist(struct blkfront_ring_info *rinfo) { @@ -996,6 +1011,7 @@ static int xlvbd_init_blk_queue(struct gendisk *gd, u16 sector_size, info->sector_size = sector_size; info->physical_sector_size = physical_sector_size; blkif_set_queue_limits(info); + init_completion(&info->wait_backend_disconnected); return 0; } @@ -1219,6 +1235,8 @@ static void xlvbd_release_gendisk(struct blkfront_info *info) /* Already hold rinfo->ring_lock. */ static inline void kick_pending_request_queues_locked(struct blkfront_ring_info *rinfo) { + if (unlikely(rinfo->dev_info->connected == BLKIF_STATE_FREEZING)) + return; if (!RING_FULL(&rinfo->ring)) blk_mq_start_stopped_hw_queues(rinfo->dev_info->rq, true); } @@ -1342,8 +1360,6 @@ static void blkif_free_ring(struct blkfront_ring_info *rinfo) static void blkif_free(struct blkfront_info *info, int suspend) { - unsigned int i; - /* Prevent new requests being issued until we fix things up. */ info->connected = suspend ? BLKIF_STATE_SUSPENDED : BLKIF_STATE_DISCONNECTED; @@ -1351,6 +1367,13 @@ static void blkif_free(struct blkfront_info *info, int suspend) if (info->rq) blk_mq_stop_hw_queues(info->rq); + __blkif_free(info); +} + +static void __blkif_free(struct blkfront_info *info) +{ + unsigned int i; + for (i = 0; i < info->nr_rings; i++) blkif_free_ring(&info->rinfo[i]); @@ -1554,8 +1577,10 @@ static irqreturn_t blkif_interrupt(int irq, void *dev_id) struct blkfront_ring_info *rinfo = (struct blkfront_ring_info *)dev_id; struct blkfront_info *info = rinfo->dev_info; - if (unlikely(info->connected != BLKIF_STATE_CONNECTED)) - return IRQ_HANDLED; + if (unlikely(info->connected != BLKIF_STATE_CONNECTED)) { + if (info->connected != BLKIF_STATE_FREEZING) + return IRQ_HANDLED; + } spin_lock_irqsave(&rinfo->ring_lock, flags); again: @@ -2005,6 +2030,7 @@ static int blkif_recover(struct blkfront_info *info) struct bio *bio; unsigned int segs; + bool frozen = info->connected == BLKIF_STATE_FROZEN; blkfront_gather_backend_features(info); /* Reset limits changed by blk_mq_update_nr_hw_queues(). */ blkif_set_queue_limits(info); @@ -2031,6 +2057,9 @@ static int blkif_recover(struct blkfront_info *info) kick_pending_request_queues(rinfo); } + if (frozen) + return 0; + list_for_each_entry_safe(req, n, &info->requests, queuelist) { /* Requeue pending requests (flush or discard) */ list_del_init(&req->queuelist); @@ -2335,6 +2364,7 @@ static void blkfront_connect(struct blkfront_info *info) return; case BLKIF_STATE_SUSPENDED: + case BLKIF_STATE_FROZEN: /* * If we are recovering from suspension, we need to wait * for the backend to announce it's features before @@ -2452,12 +2482,37 @@ static void blkback_changed(struct xenbus_device *dev, break; case XenbusStateClosed: - if (dev->state == XenbusStateClosed) + if (dev->state == XenbusStateClosed) { + if (info->connected == BLKIF_STATE_FREEZING) { + __blkif_free(info); + info->connected = BLKIF_STATE_FROZEN; + complete(&info->wait_backend_disconnected); + break; + } + + break; + } + + /* + * We may somehow receive backend's Closed again while thawing + * or restoring and it causes thawing or restoring to fail. + * Ignore such unexpected state anyway. + */ + if (info->connected == BLKIF_STATE_FROZEN && + dev->state == XenbusStateInitialised) { + dev_dbg(&dev->dev, + "ignore the backend's Closed state: %s", + dev->nodename); break; + } /* fall through */ case XenbusStateClosing: - if (info) - blkfront_closing(info); + if (info) { + if (info->connected == BLKIF_STATE_FREEZING) + xenbus_frontend_closed(dev); + else + blkfront_closing(info); + } break; } } @@ -2594,6 +2649,92 @@ static void blkif_release(struct gendisk *disk, fmode_t mode) mutex_unlock(&blkfront_mutex); } +static int blkfront_freeze(struct xenbus_device *dev) +{ + unsigned int i; + struct blkfront_info *info = dev_get_drvdata(&dev->dev); + struct blkfront_ring_info *rinfo; + struct blkif_front_ring *ring; + /* This would be reasonable timeout as used in xenbus_dev_shutdown() */ + unsigned int timeout = 5 * HZ; + int err = 0; + + info->connected = BLKIF_STATE_FREEZING; + + blk_mq_stop_hw_queues(info->rq); + + for (i = 0; i < info->nr_rings; i++) { + rinfo = &info->rinfo[i]; + + gnttab_cancel_free_callback(&rinfo->callback); + flush_work(&rinfo->work); + } + + for (i = 0; i < info->nr_rings; i++) { + spinlock_t *lock; + bool busy; + unsigned long req_timeout_ms = 25; + unsigned long ring_timeout; + + rinfo = &info->rinfo[i]; + ring = &rinfo->ring; + + lock = &rinfo->ring_lock; + + ring_timeout = jiffies + + msecs_to_jiffies(req_timeout_ms * RING_SIZE(ring)); + + do { + spin_lock_irq(lock); + busy = blkfront_ring_is_busy(ring); + spin_unlock_irq(lock); + + if (busy) + msleep(req_timeout_ms); + else + break; + } while (time_is_after_jiffies(ring_timeout)); + + /* Timed out */ + if (busy) { + xenbus_dev_error(dev, err, "the ring is still busy"); + info->connected = BLKIF_STATE_CONNECTED; + return -EBUSY; + } + } + + /* Kick the backend to disconnect */ + xenbus_switch_state(dev, XenbusStateClosing); + + /* + * We don't want to move forward before the frontend is diconnected + * from the backend cleanly. + */ + timeout = wait_for_completion_timeout(&info->wait_backend_disconnected, + timeout); + if (!timeout) { + err = -EBUSY; + xenbus_dev_error(dev, err, "Freezing timed out;" + "the device may become inconsistent state"); + } + + return err; +} + +static int blkfront_restore(struct xenbus_device *dev) +{ + struct blkfront_info *info = dev_get_drvdata(&dev->dev); + int err = 0; + + err = talk_to_blkback(dev, info); + if (err) + goto out; + blk_mq_update_nr_hw_queues(&info->tag_set, info->nr_rings); + +out: + return err; +} + static const struct block_device_operations xlvbd_block_fops = { .owner = THIS_MODULE, @@ -2616,6 +2757,9 @@ static struct xenbus_driver blkfront_driver = { .resume = blkfront_resume, .otherend_changed = blkback_changed, .is_ready = blkfront_is_ready, + .freeze = blkfront_freeze, + .thaw = blkfront_restore, + .restore = blkfront_restore }; static int __init xlblk_init(void) From patchwork Tue Jun 12 20:56:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 928496 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amazon.com header.i=@amazon.com header.b="j9++/qjP"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 4152M62Smpz9s1R for ; Wed, 13 Jun 2018 06:59:46 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934842AbeFLU7j (ORCPT ); Tue, 12 Jun 2018 16:59:39 -0400 Received: from smtp-fw-9101.amazon.com ([207.171.184.25]:65159 "EHLO smtp-fw-9101.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934156AbeFLU4i (ORCPT ); Tue, 12 Jun 2018 16:56:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1528836998; x=1560372998; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=8yVx+fhxiJVPeOdXRafxKd/h88vCdvFuGApot6cY/oE=; b=j9++/qjPqAHK6+1RGHeIf16CIuH9GR2ELyYb+YsZLxUtIHGntDGbEkk+ VKhYyaWXaZr3/RlShXHS4SRoI0Z0U7q66zYBl54HIhoP7xru8TVtCppdt dsQpsoseZHpbAOKLavyyNFt7JBbw9AF8NkxK9sKUXP7K1fTDEtEU+B4FH Q=; X-IronPort-AV: E=Sophos;i="5.51,216,1526342400"; d="scan'208";a="745800150" Received: from sea3-co-svc-lb6-vlan3.sea.amazon.com (HELO email-inbound-relay-2c-168cbb73.us-west-2.amazon.com) ([10.47.22.38]) by smtp-border-fw-out-9101.sea19.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 12 Jun 2018 20:56:36 +0000 Received: from EX13MTAUWB001.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan3.pdx.amazon.com [10.236.137.198]) by email-inbound-relay-2c-168cbb73.us-west-2.amazon.com (8.14.7/8.14.7) with ESMTP id w5CKuYMg122817 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Tue, 12 Jun 2018 20:56:34 GMT Received: from EX13D02UWB003.ant.amazon.com (10.43.161.48) by EX13MTAUWB001.ant.amazon.com (10.43.161.249) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:33 +0000 Received: from EX13MTAUEB001.ant.amazon.com (10.43.60.96) by EX13D02UWB003.ant.amazon.com (10.43.161.48) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:33 +0000 Received: from localhost (10.25.15.63) by mail-relay.amazon.com (10.43.60.129) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Tue, 12 Jun 2018 20:56:32 +0000 From: Anchal Agarwal To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [RFC PATCH 07/12] xen-netfront: add callbacks for PM suspend and hibernation support Date: Tue, 12 Jun 2018 20:56:14 +0000 Message-ID: <20180612205619.28156-8-anchalag@amazon.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180612205619.28156-1-anchalag@amazon.com> References: <20180612205619.28156-1-anchalag@amazon.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Munehisa Kamata Add freeze and restore callbacks for PM suspend and hibernation support. The freeze handler simply disconnects the frotnend from the backend and frees resources associated with queues after disabling the net_device from the system. The restore handler just changes the frontend state and let the xenbus handler to re-allocate the resources and re-connect to the backend. This can be performed transparently to the rest of the system. The handlers are used for both PM suspend and hibernation so that we can keep the existing suspend/resume callbacks for Xen suspend without modification. Freezing netfront devices is normally expected to finish within a few hundred milliseconds, but it can rarely take more than 5 seconds and hit the hard coded timeout, it would depend on backend state which may be congested and/or have complex configuration. While it's rare case, longer default timeout seems a bit more reasonable here to avoid hitting the timeout. Also, make it configurable via module parameter so that we can cover broader setups than what we know currently. Signed-off-by: Munehisa Kamata Signed-off-by: Anchal Agarwal Reviewed-by: Eduardo Valentin Reviewed-by: Munehisa Kamata --- drivers/net/xen-netfront.c | 97 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 96 insertions(+), 1 deletion(-) diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c index 4dd0668..4ea9284 100644 --- a/drivers/net/xen-netfront.c +++ b/drivers/net/xen-netfront.c @@ -43,6 +43,7 @@ #include #include #include +#include #include #include @@ -56,6 +57,12 @@ #include #include +enum netif_freeze_state { + NETIF_FREEZE_STATE_UNFROZEN, + NETIF_FREEZE_STATE_FREEZING, + NETIF_FREEZE_STATE_FROZEN, +}; + /* Module parameters */ #define MAX_QUEUES_DEFAULT 8 static unsigned int xennet_max_queues; @@ -63,6 +70,12 @@ module_param_named(max_queues, xennet_max_queues, uint, 0644); MODULE_PARM_DESC(max_queues, "Maximum number of queues per virtual interface"); +static unsigned int netfront_freeze_timeout_secs = 10; +module_param_named(freeze_timeout_secs, + netfront_freeze_timeout_secs, uint, 0644); +MODULE_PARM_DESC(freeze_timeout_secs, + "timeout when freezing netfront device in seconds"); + static const struct ethtool_ops xennet_ethtool_ops; struct netfront_cb { @@ -160,6 +173,10 @@ struct netfront_info { struct netfront_stats __percpu *tx_stats; atomic_t rx_gso_checksum_fixup; + + int freeze_state; + + struct completion wait_backend_disconnected; }; struct netfront_rx_info { @@ -723,6 +740,21 @@ static int xennet_close(struct net_device *dev) return 0; } +static int xennet_disable_interrupts(struct net_device *dev) +{ + struct netfront_info *np = netdev_priv(dev); + unsigned int num_queues = dev->real_num_tx_queues; + unsigned int i; + struct netfront_queue *queue; + + for (i = 0; i < num_queues; ++i) { + queue = &np->queues[i]; + disable_irq(queue->tx_irq); + disable_irq(queue->rx_irq); + } + return 0; +} + static void xennet_move_rx_slot(struct netfront_queue *queue, struct sk_buff *skb, grant_ref_t ref) { @@ -1296,6 +1328,8 @@ static struct net_device *xennet_create_dev(struct xenbus_device *dev) np->queues = NULL; + init_completion(&np->wait_backend_disconnected); + err = -ENOMEM; np->rx_stats = netdev_alloc_pcpu_stats(struct netfront_stats); if (np->rx_stats == NULL) @@ -1782,6 +1816,50 @@ static int xennet_create_queues(struct netfront_info *info, return 0; } +static int netfront_freeze(struct xenbus_device *dev) +{ + struct netfront_info *info = dev_get_drvdata(&dev->dev); + unsigned long timeout = netfront_freeze_timeout_secs * HZ; + int err = 0; + + xennet_disable_interrupts(info->netdev); + + netif_device_detach(info->netdev); + + info->freeze_state = NETIF_FREEZE_STATE_FREEZING; + + /* Kick the backend to disconnect */ + xenbus_switch_state(dev, XenbusStateClosing); + + /* We don't want to move forward before the frontend is diconnected + * from the backend cleanly. + */ + timeout = wait_for_completion_timeout(&info->wait_backend_disconnected, + timeout); + if (!timeout) { + err = -EBUSY; + xenbus_dev_error(dev, err, "Freezing timed out;" + "the device may become inconsistent state"); + return err; + } + + /* Tear down queues */ + xennet_disconnect_backend(info); + xennet_destroy_queues(info); + + info->freeze_state = NETIF_FREEZE_STATE_FROZEN; + + return err; +} + +static int netfront_restore(struct xenbus_device *dev) +{ + /* Kick the backend to re-connect */ + xenbus_switch_state(dev, XenbusStateInitialising); + + return 0; +} + /* Common code used when first setting up, and when resuming. */ static int talk_to_netback(struct xenbus_device *dev, struct netfront_info *info) @@ -1986,6 +2064,8 @@ static int xennet_connect(struct net_device *dev) spin_unlock_bh(&queue->rx_lock); } + np->freeze_state = NETIF_FREEZE_STATE_UNFROZEN; + return 0; } @@ -2025,11 +2105,23 @@ static void netback_changed(struct xenbus_device *dev, case XenbusStateClosed: wake_up_all(&module_unload_q); - if (dev->state == XenbusStateClosed) + if (dev->state == XenbusStateClosed) { + /* dpm context is waiting for the backend */ + if (np->freeze_state == NETIF_FREEZE_STATE_FREEZING) + complete(&np->wait_backend_disconnected); break; + } /* Missed the backend's CLOSING state -- fallthrough */ case XenbusStateClosing: wake_up_all(&module_unload_q); + /* We may see unexpected Closed or Closing from the backend. + * Just ignore it not to prevent the frontend from being + * re-connected in the case of PM suspend or hibernation. + */ + if (np->freeze_state == NETIF_FREEZE_STATE_FROZEN && + dev->state == XenbusStateInitialising) { + break; + } xenbus_frontend_closed(dev); break; } @@ -2176,6 +2268,9 @@ static struct xenbus_driver netfront_driver = { .probe = netfront_probe, .remove = xennet_remove, .resume = netfront_resume, + .freeze = netfront_freeze, + .thaw = netfront_restore, + .restore = netfront_restore, .otherend_changed = netback_changed, }; From patchwork Tue Jun 12 20:56:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 928493 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amazon.com header.i=@amazon.com header.b="BxclWojv"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 4152Kd6PrBz9s1R for ; Wed, 13 Jun 2018 06:58:29 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934412AbeFLU4u (ORCPT ); Tue, 12 Jun 2018 16:56:50 -0400 Received: from smtp-fw-2101.amazon.com ([72.21.196.25]:31588 "EHLO smtp-fw-2101.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933685AbeFLU4n (ORCPT ); Tue, 12 Jun 2018 16:56:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1528837003; x=1560373003; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=bYmNmCUIJbwg50VlZ0yhSQ9hf2/oIpgURH04cgg2wsA=; b=BxclWojvSlkmK0HuFZjTO58LF1cDecQsioipcijylgAXHfGL+qnBzIgF u9+uNhvYVMcjhXlMQOPzPg6BR8SnuNzdPz9cPYD1aOLhExFmA2Lyasyy6 8JJt/MxOKMh0ypuMNdJJId6KarXA71nrwfxZkhJM/iItlLsb3r2oq9yFe c=; X-IronPort-AV: E=Sophos;i="5.51,216,1526342400"; d="scan'208";a="682679289" Received: from iad6-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-2a-c5104f52.us-west-2.amazon.com) ([10.124.125.2]) by smtp-border-fw-out-2101.iad2.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 12 Jun 2018 20:56:41 +0000 Received: from EX13MTAUWB001.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan2.pdx.amazon.com [10.236.137.194]) by email-inbound-relay-2a-c5104f52.us-west-2.amazon.com (8.14.7/8.14.7) with ESMTP id w5CKucc6020809 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Tue, 12 Jun 2018 20:56:38 GMT Received: from EX13D07UWB004.ant.amazon.com (10.43.161.196) by EX13MTAUWB001.ant.amazon.com (10.43.161.207) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:35 +0000 Received: from EX13MTAUEA001.ant.amazon.com (10.43.61.82) by EX13D07UWB004.ant.amazon.com (10.43.161.196) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:35 +0000 Received: from localhost (10.25.15.63) by mail-relay.amazon.com (10.43.61.243) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Tue, 12 Jun 2018 20:56:34 +0000 From: Anchal Agarwal To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [RFC PATCH 08/12] xen-time-introduce-xen_-save-restore-_steal_clock Date: Tue, 12 Jun 2018 20:56:15 +0000 Message-ID: <20180612205619.28156-9-anchalag@amazon.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180612205619.28156-1-anchalag@amazon.com> References: <20180612205619.28156-1-anchalag@amazon.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Munehisa Kamata Currently, steal time accounting code in scheduler expects steal clock callback to provide monotonically increasing value. If the accounting code receives a smaller value than previous one, it uses a negative value to calculate steal time and results in incorrectly updated idle and steal time accounting. This breaks userspace tools which read /proc/stat. top - 08:05:35 up 2:12, 3 users, load average: 0.00, 0.07, 0.23 Tasks: 80 total, 1 running, 79 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,30100.0%id, 0.0%wa, 0.0%hi, 0.0%si,-1253874204672.0%st This can actually happen when a Xen PVHVM guest gets restored from hibernation, because such a restored guest is just a fresh domain from Xen perspective and the time information in runstate info starts over from scratch. This patch introduces xen_save_steal_clock() which saves current values in runstate info into per-cpu variables. Its couterpart, xen_restore_steal_clock(), sets offset if it found the current values in runstate info are smaller than previous ones. xen_steal_clock() is also modified to use the offset to ensure that scheduler only sees monotonically increasing number. Signed-off-by: Munehisa Kamata Signed-off-by: Anchal Agarwal Reviewed-by: Munehisa Kamata Reviewed-by: Eduardo Valentin --- drivers/xen/time.c | 28 +++++++++++++++++++++++++++- include/xen/xen-ops.h | 2 ++ 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/drivers/xen/time.c b/drivers/xen/time.c index 3e741cd..4756042 100644 --- a/drivers/xen/time.c +++ b/drivers/xen/time.c @@ -20,6 +20,8 @@ /* runstate info updated by Xen */ static DEFINE_PER_CPU(struct vcpu_runstate_info, xen_runstate); +static DEFINE_PER_CPU(u64, xen_prev_steal_clock); +static DEFINE_PER_CPU(u64, xen_steal_clock_offset); static DEFINE_PER_CPU(u64[4], old_runstate_time); @@ -149,7 +151,7 @@ bool xen_vcpu_stolen(int vcpu) return per_cpu(xen_runstate, vcpu).state == RUNSTATE_runnable; } -u64 xen_steal_clock(int cpu) +static u64 __xen_steal_clock(int cpu) { struct vcpu_runstate_info state; @@ -157,6 +159,30 @@ u64 xen_steal_clock(int cpu) return state.time[RUNSTATE_runnable] + state.time[RUNSTATE_offline]; } +u64 xen_steal_clock(int cpu) +{ + return __xen_steal_clock(cpu) + per_cpu(xen_steal_clock_offset, cpu); +} + +void xen_save_steal_clock(int cpu) +{ + per_cpu(xen_prev_steal_clock, cpu) = xen_steal_clock(cpu); +} + +void xen_restore_steal_clock(int cpu) +{ + u64 steal_clock = __xen_steal_clock(cpu); + + if (per_cpu(xen_prev_steal_clock, cpu) > steal_clock) { + /* Need to update the offset */ + per_cpu(xen_steal_clock_offset, cpu) = + per_cpu(xen_prev_steal_clock, cpu) - steal_clock; + } else { + /* Avoid unnecessary steal clock warp */ + per_cpu(xen_steal_clock_offset, cpu) = 0; + } +} + void xen_setup_runstate_info(int cpu) { struct vcpu_register_runstate_memory_area area; diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h index 65f25bd..10330f8 100644 --- a/include/xen/xen-ops.h +++ b/include/xen/xen-ops.h @@ -36,6 +36,8 @@ void xen_time_setup_guest(void); void xen_manage_runstate_time(int action); void xen_get_runstate_snapshot(struct vcpu_runstate_info *res); u64 xen_steal_clock(int cpu); +void xen_save_steal_clock(int cpu); +void xen_restore_steal_clock(int cpu); int xen_setup_shutdown_event(void); From patchwork Tue Jun 12 20:56:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 928489 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amazon.com header.i=@amazon.com header.b="XInkU9MP"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 4152Hv4lxRz9s1R for ; Wed, 13 Jun 2018 06:56:59 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934675AbeFLU4z (ORCPT ); Tue, 12 Jun 2018 16:56:55 -0400 Received: from smtp-fw-2101.amazon.com ([72.21.196.25]:31600 "EHLO smtp-fw-2101.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934335AbeFLU4o (ORCPT ); Tue, 12 Jun 2018 16:56:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1528837004; x=1560373004; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=Yw2zPwrO0M5CiRMiqXHX1y3V/D58hCoOhuENVMBy8i0=; b=XInkU9MPDdf3f6uxFg/qSThaABTbbrF1Jxp3imqy8PdtmZtvJSWFNS9x 10T85wAPEBsnhYb6UKCzknGMMk+5n8XKXJlB8VPItOlDlsucpgfeJ070n R04yubTSoT9dtL44assI4eR12ooWGZ58JZ4cutW5TRIJx49L4zuxxHkWU k=; X-IronPort-AV: E=Sophos;i="5.51,216,1526342400"; d="scan'208";a="682679290" Received: from iad6-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-2b-55156cd4.us-west-2.amazon.com) ([10.124.125.2]) by smtp-border-fw-out-2101.iad2.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 12 Jun 2018 20:56:42 +0000 Received: from EX13MTAUWB001.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan3.pdx.amazon.com [10.236.137.198]) by email-inbound-relay-2b-55156cd4.us-west-2.amazon.com (8.14.7/8.14.7) with ESMTP id w5CKueZc017015 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Tue, 12 Jun 2018 20:56:40 GMT Received: from EX13D02UWB002.ant.amazon.com (10.43.161.80) by EX13MTAUWB001.ant.amazon.com (10.43.161.249) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:37 +0000 Received: from EX13MTAUEA001.ant.amazon.com (10.43.61.82) by EX13D02UWB002.ant.amazon.com (10.43.161.80) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:36 +0000 Received: from localhost (10.25.15.63) by mail-relay.amazon.com (10.43.61.243) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Tue, 12 Jun 2018 20:56:35 +0000 From: Anchal Agarwal To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [RFC PATCH 09/12] x86/xen: save and restore steal clock Date: Tue, 12 Jun 2018 20:56:16 +0000 Message-ID: <20180612205619.28156-10-anchalag@amazon.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180612205619.28156-1-anchalag@amazon.com> References: <20180612205619.28156-1-anchalag@amazon.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Munehisa Kamata Save steal clock values of all present CPUs in the system core ops suspend callbacks. Also, restore a boot CPU's steal clock in the system core resume callback. For non-boot CPUs, restore after they're brought up, because runstate info for non-boot CPUs are not active until then. Signed-off-by: Munehisa Kamata Signed-off-by: Anchal Agarwal Reviewed-by: Munehisa Kamata Reviewed-by: Eduardo Valentin --- arch/x86/xen/suspend.c | 13 ++++++++++++- arch/x86/xen/time.c | 3 +++ 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/x86/xen/suspend.c b/arch/x86/xen/suspend.c index 784c448..dae0f74 100644 --- a/arch/x86/xen/suspend.c +++ b/arch/x86/xen/suspend.c @@ -91,12 +91,20 @@ void xen_arch_suspend(void) static int xen_syscore_suspend(void) { struct xen_remove_from_physmap xrfp; - int ret; + int cpu, ret; /* Xen suspend does similar stuffs in its own logic */ if (xen_suspend_mode_is_xen_suspend()) return 0; + for_each_present_cpu(cpu) { + /* + * Nonboot CPUs are already offline, but the last copy of + * runstate info is still accessible. + */ + xen_save_steal_clock(cpu); + } + xrfp.domid = DOMID_SELF; xrfp.gpfn = __pa(HYPERVISOR_shared_info) >> PAGE_SHIFT; @@ -118,6 +126,9 @@ static void xen_syscore_resume(void) pvclock_resume(); + /* Nonboot CPUs will be resumed when they're brought up */ + xen_restore_steal_clock(smp_processor_id()); + gnttab_resume(); } diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index e0f1bcf..85f8534 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -523,6 +523,9 @@ static void xen_hvm_setup_cpu_clockevents(void) { int cpu = smp_processor_id(); xen_setup_runstate_info(cpu); + if (cpu) + xen_restore_steal_clock(cpu); + /* * xen_setup_timer(cpu) - snprintf is bad in atomic context. Hence * doing it xen_hvm_cpu_notify (which gets called by smp_init during From patchwork Tue Jun 12 20:56:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 928491 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amazon.com header.i=@amazon.com header.b="OELyrn7a"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 4152Jj6fLQz9s2t for ; Wed, 13 Jun 2018 06:57:41 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934619AbeFLU4y (ORCPT ); Tue, 12 Jun 2018 16:56:54 -0400 Received: from smtp-fw-9102.amazon.com ([207.171.184.29]:1586 "EHLO smtp-fw-9102.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934338AbeFLU4o (ORCPT ); Tue, 12 Jun 2018 16:56:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1528837004; x=1560373004; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=Wq/2+Vx2/toTC1g6p4xJBEgO2q4jxfN5wL1oSCQ188U=; b=OELyrn7a5C444m/zkra/SzgYH+aJgS/L2LbGoVUcBNHckQq6p65m1GgW sfNT2APLAI1cL57ivt9LdVP1UDD6EUZaRbbUEMqpMoo8XM3AQUe0OhItj 5TPQqRN3SLyv+t5lBJTGXAYvxW/iuInEc+Ptb4p0zR2dTXJ+G9NcHWds5 Q=; X-IronPort-AV: E=Sophos;i="5.51,216,1526342400"; d="scan'208";a="617032514" Received: from sea3-co-svc-lb6-vlan3.sea.amazon.com (HELO email-inbound-relay-2a-7c3c5f85.us-west-2.amazon.com) ([10.47.22.38]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 12 Jun 2018 20:56:43 +0000 Received: from EX13MTAUWB001.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan3.pdx.amazon.com [10.236.137.198]) by email-inbound-relay-2a-7c3c5f85.us-west-2.amazon.com (8.14.7/8.14.7) with ESMTP id w5CKuewb125665 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Tue, 12 Jun 2018 20:56:41 GMT Received: from EX13D13UWB004.ant.amazon.com (10.43.161.218) by EX13MTAUWB001.ant.amazon.com (10.43.161.249) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:38 +0000 Received: from EX13MTAUEB001.ant.amazon.com (10.43.60.96) by EX13D13UWB004.ant.amazon.com (10.43.161.218) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:38 +0000 Received: from localhost (10.25.15.63) by mail-relay.amazon.com (10.43.60.129) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Tue, 12 Jun 2018 20:56:37 +0000 From: Anchal Agarwal To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [RFC PATCH 10/12] xen/events: add xen_shutdown_pirqs helper function Date: Tue, 12 Jun 2018 20:56:17 +0000 Message-ID: <20180612205619.28156-11-anchalag@amazon.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180612205619.28156-1-anchalag@amazon.com> References: <20180612205619.28156-1-anchalag@amazon.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Munehisa Kamata Add a simple helper function to "shutdown" active PIRQs, which actually closes event channels but keeps related IRQ structures intact. PM suspend/hibernation code will rely on this. Signed-off-by: Munehisa Kamata Signed-off-by: Anchal Agarwal Reviewed-by: Munehisa Kamata Reviewed-by: Eduardo Valentin --- drivers/xen/events/events_base.c | 12 ++++++++++++ include/xen/events.h | 1 + 2 files changed, 13 insertions(+) diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c index 762378f..88137c8 100644 --- a/drivers/xen/events/events_base.c +++ b/drivers/xen/events/events_base.c @@ -1581,6 +1581,18 @@ void xen_irq_resume(void) restore_pirqs(); } +void xen_shutdown_pirqs(void) +{ + struct irq_info *info; + + list_for_each_entry(info, &xen_irq_list_head, list) { + if (info->type != IRQT_PIRQ || !VALID_EVTCHN(info->evtchn)) + continue; + + shutdown_pirq(irq_get_irq_data(info->irq)); + } +} + static struct irq_chip xen_dynamic_chip __read_mostly = { .name = "xen-dyn", diff --git a/include/xen/events.h b/include/xen/events.h index c3e6bc6..e4d5ccb 100644 --- a/include/xen/events.h +++ b/include/xen/events.h @@ -70,6 +70,7 @@ static inline void notify_remote_via_evtchn(int port) void notify_remote_via_irq(int irq); void xen_irq_resume(void); +void xen_shutdown_pirqs(void); /* Clear an irq's pending state, in preparation for polling on it */ void xen_clear_irq_pending(int irq); From patchwork Tue Jun 12 20:56:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 928490 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amazon.com header.i=@amazon.com header.b="vbkH03re"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 4152JJ4HrTz9s1B for ; Wed, 13 Jun 2018 06:57:20 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933443AbeFLU45 (ORCPT ); Tue, 12 Jun 2018 16:56:57 -0400 Received: from smtp-fw-2101.amazon.com ([72.21.196.25]:31588 "EHLO smtp-fw-2101.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934350AbeFLU4r (ORCPT ); Tue, 12 Jun 2018 16:56:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1528837007; x=1560373007; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=LUwFf+Ae7TN++/Wj+Px8atzbQJwwwOGW+JAVCXV9Dic=; b=vbkH03re156tWJH7CQPVKodPhOaCFoCJooVrn9MyoAjvdOxRld3es1JW yI2+zeP8fNRL7i4qyTj+Whw6vDPwiwOAoMucGpik0UelPKhSZiHlGCvZW 35N4wI+WDv5egseEsSuN9xzp0wKIITy2bPmkFdAWNXn+gu8MokxjHsRZw U=; X-IronPort-AV: E=Sophos;i="5.51,216,1526342400"; d="scan'208";a="682679292" Received: from iad6-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-2b-baacba05.us-west-2.amazon.com) ([10.124.125.2]) by smtp-border-fw-out-2101.iad2.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 12 Jun 2018 20:56:43 +0000 Received: from EX13MTAUWA001.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan3.pdx.amazon.com [10.236.137.198]) by email-inbound-relay-2b-baacba05.us-west-2.amazon.com (8.14.7/8.14.7) with ESMTP id w5CKufg2102858 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Tue, 12 Jun 2018 20:56:41 GMT Received: from EX13D10UWA003.ant.amazon.com (10.43.160.248) by EX13MTAUWA001.ant.amazon.com (10.43.160.118) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:39 +0000 Received: from EX13MTAUWA001.ant.amazon.com (10.43.160.58) by EX13D10UWA003.ant.amazon.com (10.43.160.248) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:38 +0000 Received: from localhost (10.25.15.63) by mail-relay.amazon.com (10.43.160.118) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Tue, 12 Jun 2018 20:56:38 +0000 From: Anchal Agarwal To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [RFC PATCH 11/12] x86/xen: close event channels for PIRQs in system core suspend callback Date: Tue, 12 Jun 2018 20:56:18 +0000 Message-ID: <20180612205619.28156-12-anchalag@amazon.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180612205619.28156-1-anchalag@amazon.com> References: <20180612205619.28156-1-anchalag@amazon.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Munehisa Kamata Close event channels allocated for devices which are backed by PIRQ and still active when suspending the system core. Normally, the devices are emulated legacy devices, e.g. PS/2 keyboard, floppy controller and etc. Without this, in PM hibernation, information about the event channel remains in hibernation image, but there is no guarantee that the same event channel numbers are assigned to the devices when restoring the system. This may cause conflict like the following and prevent some devices from being restored correctly. [ 102.330821] ------------[ cut here ]------------ [ 102.333264] WARNING: CPU: 0 PID: 2324 at drivers/xen/events/events_base.c:878 bind_evtchn_to_irq+0x88/0xf0 ... [ 102.348057] Call Trace: [ 102.348057] [] dump_stack+0x63/0x84 [ 102.348057] [] __warn+0xd1/0xf0 [ 102.348057] [] warn_slowpath_null+0x1d/0x20 [ 102.348057] [] bind_evtchn_to_irq+0x88/0xf0 [ 102.348057] [] ? blkif_copy_from_grant+0xb0/0xb0 [xen_blkfront] [ 102.348057] [] bind_evtchn_to_irqhandler+0x27/0x80 [ 102.348057] [] talk_to_blkback+0x425/0xcd0 [xen_blkfront] [ 102.348057] [] ? __kmalloc+0x1ea/0x200 [ 102.348057] [] blkfront_restore+0x2d/0x60 [xen_blkfront] [ 102.348057] [] xenbus_dev_restore+0x58/0x100 [ 102.348057] [] ? xenbus_frontend_delayed_resume+0x20/0x20 [ 102.348057] [] xenbus_dev_cond_restore+0x1e/0x30 [ 102.348057] [] dpm_run_callback+0x4e/0x130 [ 102.348057] [] device_resume+0xe7/0x210 [ 102.348057] [] ? pm_dev_dbg+0x80/0x80 [ 102.348057] [] dpm_resume+0x114/0x2f0 [ 102.348057] [] hibernation_snapshot+0x15f/0x380 [ 102.348057] [] hibernate+0x183/0x290 [ 102.348057] [] state_store+0xcf/0xe0 [ 102.348057] [] kobj_attr_store+0xf/0x20 [ 102.348057] [] sysfs_kf_write+0x3a/0x50 [ 102.348057] [] kernfs_fop_write+0x10b/0x190 [ 102.348057] [] __vfs_write+0x28/0x120 [ 102.348057] [] ? rw_verify_area+0x49/0xb0 [ 102.348057] [] vfs_write+0xb2/0x1b0 [ 102.348057] [] SyS_write+0x46/0xa0 [ 102.348057] [] entry_SYSCALL_64_fastpath+0x1a/0xa9 [ 102.423005] ---[ end trace b8d6718e22e2b107 ]--- [ 102.425031] genirq: Flags mismatch irq 6. 00000000 (blkif) vs. 00000000 (floppy) Note that we don't explicitly re-allocate event channels for such devices in the resume callback. Re-allocation will occur when PM core re-enable IRQs for the devices at later point. Signed-off-by: Munehisa Kamata Signed-off-by: Anchal Agarwal Reviewed-by: Munehisa Kamata Reviewed-by: Eduardo Valentin --- arch/x86/xen/suspend.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/x86/xen/suspend.c b/arch/x86/xen/suspend.c index dae0f74..affa63d 100644 --- a/arch/x86/xen/suspend.c +++ b/arch/x86/xen/suspend.c @@ -105,6 +105,8 @@ static int xen_syscore_suspend(void) xen_save_steal_clock(cpu); } + xen_shutdown_pirqs(); + xrfp.domid = DOMID_SELF; xrfp.gpfn = __pa(HYPERVISOR_shared_info) >> PAGE_SHIFT; From patchwork Tue Jun 12 20:56:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 928492 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amazon.com header.i=@amazon.com header.b="mT6tKf8Y"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 4152K20Tdpz9s31 for ; Wed, 13 Jun 2018 06:57:58 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934517AbeFLU4w (ORCPT ); Tue, 12 Jun 2018 16:56:52 -0400 Received: from smtp-fw-6002.amazon.com ([52.95.49.90]:46135 "EHLO smtp-fw-6002.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934342AbeFLU4p (ORCPT ); Tue, 12 Jun 2018 16:56:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1528837005; x=1560373005; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=CaVNlSsvdiSt9FutpnLSEKRFgB42VECALAbNL5pFElw=; b=mT6tKf8YGb39dR1oqnseZTSoj4hq/EDdFS1ZtfKBBkV6qqTa3eRhuD2E FpdZ/dO67uEBg7Y8MJtr0RLMJ8xuxxgJ0wzTc4uTUSxzL0ySyoZDKLqry mc1avbTCefZwdS0RkIRsnYUkeIJ4kQaXU0hSocvGQj1bUldPsRLV4txSC 0=; X-IronPort-AV: E=Sophos;i="5.51,216,1526342400"; d="scan'208";a="347699308" Received: from iad6-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-2b-55156cd4.us-west-2.amazon.com) ([10.124.125.6]) by smtp-border-fw-out-6002.iad6.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 12 Jun 2018 20:56:42 +0000 Received: from EX13MTAUWB001.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan3.pdx.amazon.com [10.236.137.198]) by email-inbound-relay-2b-55156cd4.us-west-2.amazon.com (8.14.7/8.14.7) with ESMTP id w5CKueZq017015 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Tue, 12 Jun 2018 20:56:42 GMT Received: from EX13D05UWB003.ant.amazon.com (10.43.161.26) by EX13MTAUWB001.ant.amazon.com (10.43.161.207) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:39 +0000 Received: from EX13MTAUWB001.ant.amazon.com (10.43.161.207) by EX13D05UWB003.ant.amazon.com (10.43.161.26) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 12 Jun 2018 20:56:39 +0000 Received: from localhost (10.25.15.63) by mail-relay.amazon.com (10.43.161.249) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Tue, 12 Jun 2018 20:56:39 +0000 From: Anchal Agarwal To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [RFC PATCH 12/12] PM / hibernate: update the resume offset on SNAPSHOT_SET_SWAP_AREA Date: Tue, 12 Jun 2018 20:56:19 +0000 Message-ID: <20180612205619.28156-13-anchalag@amazon.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180612205619.28156-1-anchalag@amazon.com> References: <20180612205619.28156-1-anchalag@amazon.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Aleksei Besogonov The SNAPSHOT_SET_SWAP_AREA is supposed to be used to set the hibernation offset on a running kernel to enable hibernating to a swap file. However, it doesn't actually update the swsusp_resume_block variable. As a result, the hibernation fails at the last step (after all the data is written out) in the validation of the swap signature in mark_swapfiles(). Before this patch, the command line processing was the only place where swsusp_resume_block was set. Signed-off-by: Aleksei Besogonov Signed-off-by: Munehisa Kamata Signed-off-by: Anchal Agarwal Reviewed-by: Munehisa Kamata Reviewed-by: Eduardo Valentin --- kernel/power/user.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/kernel/power/user.c b/kernel/power/user.c index abd2255..b522a42 100644 --- a/kernel/power/user.c +++ b/kernel/power/user.c @@ -379,8 +379,12 @@ static long snapshot_ioctl(struct file *filp, unsigned int cmd, if (swdev) { offset = swap_area.offset; data->swap = swap_type_of(swdev, offset, NULL); - if (data->swap < 0) + if (data->swap < 0) { error = -ENODEV; + } else { + swsusp_resume_device = swdev; + swsusp_resume_block = offset; + } } else { data->swap = -1; error = -EINVAL;