From patchwork Sat Mar 3 00:20:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dexuan Cui X-Patchwork-Id: 880960 X-Patchwork-Delegate: lorenzo.pieralisi@arm.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=microsoft.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=microsoft.com header.i=@microsoft.com header.b="e9rsfjSE"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3ztRh64BB1z9s8p for ; Sat, 3 Mar 2018 11:22:30 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S935398AbeCCAUo (ORCPT ); Fri, 2 Mar 2018 19:20:44 -0500 Received: from mail-sg2apc01on0110.outbound.protection.outlook.com ([104.47.125.110]:43168 "EHLO APC01-SG2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S932890AbeCCAUk (ORCPT ); Fri, 2 Mar 2018 19:20:40 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=vxCIrSoAZrlKw9gBWMMUtwbcEj/n0Rn78xMG3DArWYc=; b=e9rsfjSEtFtUgLq4gDxjGcy7Gu4uJgEx/uRUlJqLfN7Pis0y+zw98/HCvlt5MEvr63AZmzek7IxKPMtuZ9qInfnMnkaI0kiQAO44U0864tnH1aVvUO1fa725vEwg3UWwg43XC/8gXm7Fyqxj1hLr+F5oqiYQAG6cJ1FwOIXb9Ws= Received: from KL1P15301MB0006.APCP153.PROD.OUTLOOK.COM (10.170.167.17) by KL1P15301MB0054.APCP153.PROD.OUTLOOK.COM (10.170.168.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.567.6; Sat, 3 Mar 2018 00:20:32 +0000 Received: from KL1P15301MB0006.APCP153.PROD.OUTLOOK.COM ([10.170.167.17]) by KL1P15301MB0006.APCP153.PROD.OUTLOOK.COM ([10.170.167.17]) with mapi id 15.20.0567.002; Sat, 3 Mar 2018 00:20:32 +0000 From: Dexuan Cui To: "bhelgaas@google.com" , "linux-pci@vger.kernel.org" , KY Srinivasan , Stephen Hemminger CC: "linux-kernel@vger.kernel.org" , "driverdev-devel@linuxdriverproject.org" , Haiyang Zhang , "olaf@aepfle.de" , "apw@canonical.com" , "jasowang@redhat.com" , "vkuznets@redhat.com" , "marcelo.cerri@canonical.com" , Dexuan Cui , "stable@vger.kernel.org" Subject: [PATCH 1/3] PCI: hv: fix a comment typo in _hv_pcifront_read_config() Thread-Topic: [PATCH 1/3] PCI: hv: fix a comment typo in _hv_pcifront_read_config() Thread-Index: AQHTsoVwc9zL3DkZZUigOOSbdSFRmA== Date: Sat, 3 Mar 2018 00:20:32 +0000 Message-ID: <20180303001947.20564-1-decui@microsoft.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.15.1 authentication-results: spf=none (sender IP is ) smtp.mailfrom=decui@microsoft.com; x-originating-ip: [52.168.54.252] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; KL1P15301MB0054; 7:UBceKHZAJTfPUk08AbIZlaRG05UNk/KGdpgd3m957Bb9pP55CVWaHn+KVtsnQybHlspiI47FiywiEF7xxU2aVgmQooqlAMNXdjaVO3btwmIIb11TEowPy9S7rjQvf5qAdYyf6jVGX8wju+J6XxIWkXNQTtGgtiybWKfUkZAFTugxOy7ftFY31vM9QMrCQ5rY+V2v0wDHPZPfIZTiCvGb9kCWNNjwWcrYo98xLmdxqdOGNP4ZNsUlgPqrZoaKICC7 x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: fb0993ae-ad62-4310-ee1c-08d5809c93c1 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4604075)(3008032)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603307)(7193020); SRVR:KL1P15301MB0054; x-ms-traffictypediagnostic: KL1P15301MB0054: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(28532068793085)(89211679590171)(9452136761055); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(61425038)(6040501)(2401047)(8121501046)(5005006)(10201501046)(3002001)(93006095)(93001095)(3231220)(944501244)(52105095)(6055026)(61426038)(61427038)(6041288)(20161123564045)(20161123560045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(6072148)(201708071742011); SRVR:KL1P15301MB0054; BCL:0; PCL:0; RULEID:; SRVR:KL1P15301MB0054; x-forefront-prvs: 0600F93FE1 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(39860400002)(396003)(39380400002)(346002)(366004)(376002)(189003)(199004)(5660300001)(2906002)(4326008)(6116002)(8936002)(86362001)(2501003)(8676002)(6506007)(2900100001)(6436002)(14454004)(3660700001)(86612001)(305945005)(36756003)(99286004)(316002)(97736004)(25786009)(110136005)(1511001)(102836004)(54906003)(81166006)(10090500001)(81156014)(59450400001)(3280700002)(26005)(186003)(77096007)(6512007)(6636002)(478600001)(3846002)(1076002)(22452003)(10290500003)(106356001)(66066001)(6486002)(50226002)(7416002)(68736007)(7736002)(105586002)(53936002)(22906009); DIR:OUT; SFP:1102; SCL:1; SRVR:KL1P15301MB0054; H:KL1P15301MB0006.APCP153.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: hiaYUlBokXrQkJxBhokOPzjzQS/thFsoMcZxk604eKuHPjjo7JFPla6BoWW4VOGyP81rhQVZxwX/r/tRHBsq2wv0cMn34FkrrTWiIp1rt4QisiharfUwa+KBnIH1ynYGe4oWakMYwpmPaYjj3UDfKlhLdvtnjRPpeAvAama4kao= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: fb0993ae-ad62-4310-ee1c-08d5809c93c1 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Mar 2018 00:20:32.0693 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: KL1P15301MB0054 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org No functional change. Signed-off-by: Dexuan Cui Fixes: bdd74440d9e8 ("PCI: hv: Add explicit barriers to config space access") Cc: Vitaly Kuznetsov Cc: stable@vger.kernel.org Cc: Stephen Hemminger Cc: K. Y. Srinivasan --- drivers/pci/host/pci-hyperv.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/pci/host/pci-hyperv.c b/drivers/pci/host/pci-hyperv.c index 2faf38eab785..1233300f41c6 100644 --- a/drivers/pci/host/pci-hyperv.c +++ b/drivers/pci/host/pci-hyperv.c @@ -653,7 +653,7 @@ static void _hv_pcifront_read_config(struct hv_pci_dev *hpdev, int where, break; } /* - * Make sure the write was done before we release the spinlock + * Make sure the read was done before we release the spinlock * allowing consecutive reads/writes. */ mb(); From patchwork Sat Mar 3 00:20:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dexuan Cui X-Patchwork-Id: 880958 X-Patchwork-Delegate: lorenzo.pieralisi@arm.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=microsoft.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=microsoft.com header.i=@microsoft.com header.b="Rxi4IzjO"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3ztRfN68hDz9s8p for ; Sat, 3 Mar 2018 11:21:00 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S935423AbeCCAU5 (ORCPT ); Fri, 2 Mar 2018 19:20:57 -0500 Received: from mail-sg2apc01on0106.outbound.protection.outlook.com ([104.47.125.106]:49948 "EHLO APC01-SG2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S932890AbeCCAUz (ORCPT ); Fri, 2 Mar 2018 19:20:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=k9RBo9q+bm0Fx8LnwlLFD6qT5G7PbIyBR5iD1d6FUoo=; b=Rxi4IzjOExqUEmd3l67yg/ucyxd+Wdwmw8badXxroVJjmBSsHvcglQ6N9G1Fnj055gdT122zegAFiAQqES2qVFc4pmRPBZ8YOOPKkl/gydLFz8nppdhQ4yLIROSxi11oct4R3odeHq+NM41Vuny0BDhk8C06vvhDJYiniK/76nc= Received: from KL1P15301MB0006.APCP153.PROD.OUTLOOK.COM (10.170.167.17) by KL1P15301MB0054.APCP153.PROD.OUTLOOK.COM (10.170.168.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.567.6; Sat, 3 Mar 2018 00:20:47 +0000 Received: from KL1P15301MB0006.APCP153.PROD.OUTLOOK.COM ([10.170.167.17]) by KL1P15301MB0006.APCP153.PROD.OUTLOOK.COM ([10.170.167.17]) with mapi id 15.20.0567.002; Sat, 3 Mar 2018 00:20:47 +0000 From: Dexuan Cui To: "bhelgaas@google.com" , "linux-pci@vger.kernel.org" , KY Srinivasan , Stephen Hemminger CC: "linux-kernel@vger.kernel.org" , "driverdev-devel@linuxdriverproject.org" , Haiyang Zhang , "olaf@aepfle.de" , "apw@canonical.com" , "jasowang@redhat.com" , "vkuznets@redhat.com" , "marcelo.cerri@canonical.com" , Dexuan Cui , Jack Morgenstein , "stable@vger.kernel.org" Subject: [PATCH 2/3] PCI: hv: serialize the present/eject work items Thread-Topic: [PATCH 2/3] PCI: hv: serialize the present/eject work items Thread-Index: AQHTsoV58ygEHWMoYEeyKVqO+bn/hg== Date: Sat, 3 Mar 2018 00:20:46 +0000 Message-ID: <20180303001947.20564-2-decui@microsoft.com> References: <20180303001947.20564-1-decui@microsoft.com> In-Reply-To: <20180303001947.20564-1-decui@microsoft.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.15.1 authentication-results: spf=none (sender IP is ) smtp.mailfrom=decui@microsoft.com; x-originating-ip: [52.168.54.252] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; KL1P15301MB0054; 7:wDzumdb6lZ67JknReSiPtUA1miPqv881sYQobViJf0PKUmPqwDcKlKVsCcP3KvmNhyeK4K/hQ63efKX4LW1et0eF8OUQ4kJ6Ps9QUW39X5vkYWmAohTHojOLc08AawsjpBBw1jyaw0hjSI7uPmm1ybR20HfIZkczEV64zF45Wpsy16vMj5SvrGTDrBszh+zSYPFEa+sA66FU34f4toac43RevpJXc8KzEdw2k66f83DUXPnuZQVeZD0payGa6XfL x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: b436957e-4694-4cd3-441f-08d5809c9c6c x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4604075)(3008032)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603307)(7193020); SRVR:KL1P15301MB0054; x-ms-traffictypediagnostic: KL1P15301MB0054: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(28532068793085)(89211679590171)(9452136761055); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(61425038)(6040501)(2401047)(8121501046)(5005006)(10201501046)(3002001)(93006095)(93001095)(3231220)(944501244)(52105095)(6055026)(61426038)(61427038)(6041288)(20161123564045)(20161123560045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(6072148)(201708071742011); SRVR:KL1P15301MB0054; BCL:0; PCL:0; RULEID:; SRVR:KL1P15301MB0054; x-forefront-prvs: 0600F93FE1 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(39860400002)(396003)(39380400002)(346002)(366004)(376002)(189003)(199004)(5660300001)(2906002)(4326008)(6116002)(8936002)(86362001)(2501003)(8676002)(6506007)(2900100001)(6436002)(14454004)(3660700001)(86612001)(305945005)(36756003)(99286004)(316002)(97736004)(25786009)(110136005)(1511001)(102836004)(54906003)(81166006)(10090500001)(81156014)(59450400001)(3280700002)(26005)(186003)(77096007)(6512007)(6636002)(76176011)(478600001)(2950100002)(3846002)(1076002)(22452003)(10290500003)(106356001)(66066001)(6486002)(50226002)(7416002)(68736007)(7736002)(105586002)(53936002)(22906009); DIR:OUT; SFP:1102; SCL:1; SRVR:KL1P15301MB0054; H:KL1P15301MB0006.APCP153.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: Mj9aJtiXsvrFx9T2zzJx57+enhmhSUFyZDe6/LUEhK4laVmj/Ut7W/D8tt3C0CgvIk5VG3K282CikeJenqembZN3Qd3cB16zzf66j/tFU/1yV40A5E/HbNATXgp6o7BtRhn8cWmmafFn/BdMZMDB37nzLO0qm+6ya9fGDGcEs1A= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: b436957e-4694-4cd3-441f-08d5809c9c6c X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Mar 2018 00:20:46.6174 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: KL1P15301MB0054 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org When we hot-remove the device, we first receive a PCI_EJECT message and then receive a PCI_BUS_RELATIONS message with bus_rel->device_count == 0. The first message is offloaded to hv_eject_device_work(), and the second is offloaded to pci_devices_present_work(). Both the paths can be running list_del(&hpdev->list_entry), causing general protection fault, because system_wq can run them concurrently. The patch eliminates the race condition. Signed-off-by: Dexuan Cui Tested-by: Adrian Suhov Tested-by: Chris Valean Cc: Vitaly Kuznetsov Cc: Jack Morgenstein Cc: stable@vger.kernel.org Cc: Stephen Hemminger Cc: K. Y. Srinivasan --- drivers/pci/host/pci-hyperv.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/drivers/pci/host/pci-hyperv.c b/drivers/pci/host/pci-hyperv.c index 1233300f41c6..57b1fb3ebdb9 100644 --- a/drivers/pci/host/pci-hyperv.c +++ b/drivers/pci/host/pci-hyperv.c @@ -461,6 +461,8 @@ struct hv_pcibus_device { struct retarget_msi_interrupt retarget_msi_interrupt_params; spinlock_t retarget_msi_interrupt_lock; + + struct workqueue_struct *wq; }; /* @@ -1770,7 +1772,7 @@ static void hv_pci_devices_present(struct hv_pcibus_device *hbus, spin_unlock_irqrestore(&hbus->device_list_lock, flags); get_hvpcibus(hbus); - schedule_work(&dr_wrk->wrk); + queue_work(hbus->wq, &dr_wrk->wrk); } /** @@ -1848,7 +1850,7 @@ static void hv_pci_eject_device(struct hv_pci_dev *hpdev) get_pcichild(hpdev, hv_pcidev_ref_pnp); INIT_WORK(&hpdev->wrk, hv_eject_device_work); get_hvpcibus(hpdev->hbus); - schedule_work(&hpdev->wrk); + queue_work(hpdev->hbus->wq, &hpdev->wrk); } /** @@ -2463,11 +2465,17 @@ static int hv_pci_probe(struct hv_device *hdev, spin_lock_init(&hbus->retarget_msi_interrupt_lock); sema_init(&hbus->enum_sem, 1); init_completion(&hbus->remove_event); + hbus->wq = alloc_ordered_workqueue("hv_pci_%x", 0, + hbus->sysdata.domain); + if (!hbus->wq) { + ret = -ENOMEM; + goto free_bus; + } ret = vmbus_open(hdev->channel, pci_ring_size, pci_ring_size, NULL, 0, hv_pci_onchannelcallback, hbus); if (ret) - goto free_bus; + goto destroy_wq; hv_set_drvdata(hdev, hbus); @@ -2536,6 +2544,9 @@ static int hv_pci_probe(struct hv_device *hdev, hv_free_config_window(hbus); close: vmbus_close(hdev->channel); +destroy_wq: + drain_workqueue(hbus->wq); + destroy_workqueue(hbus->wq); free_bus: free_page((unsigned long)hbus); return ret; @@ -2615,6 +2626,8 @@ static int hv_pci_remove(struct hv_device *hdev) irq_domain_free_fwnode(hbus->sysdata.fwnode); put_hvpcibus(hbus); wait_for_completion(&hbus->remove_event); + drain_workqueue(hbus->wq); + destroy_workqueue(hbus->wq); free_page((unsigned long)hbus); return 0; } From patchwork Sat Mar 3 00:20:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dexuan Cui X-Patchwork-Id: 880959 X-Patchwork-Delegate: lorenzo.pieralisi@arm.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=microsoft.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=microsoft.com header.i=@microsoft.com header.b="PMesG9Fd"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3ztRg14xcmz9s8r for ; Sat, 3 Mar 2018 11:21:33 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S935487AbeCCAVa (ORCPT ); Fri, 2 Mar 2018 19:21:30 -0500 Received: from mail-sg2apc01on0106.outbound.protection.outlook.com ([104.47.125.106]:49948 "EHLO APC01-SG2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S935421AbeCCAU5 (ORCPT ); Fri, 2 Mar 2018 19:20:57 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=p/Dc06zl5xAhA4SRL1CzqoHya6EREbhXIpwQS1qNrU0=; b=PMesG9Fd5edCRe9ZOllxZcO3KVu/bwyF4KSXX9v5OwmztEcOz4mW/oZ1y/kpkGvyWKfbTI/2iFx3JQstKAfEEudZyLxgETw941+xGELE14n1bYHCLrMLi6NVwfSFYJ7Gc9B/2OLnKhS64NAFMNa1z+vP5UQehd7GcF2XFYW3y4Y= Received: from KL1P15301MB0006.APCP153.PROD.OUTLOOK.COM (10.170.167.17) by KL1P15301MB0054.APCP153.PROD.OUTLOOK.COM (10.170.168.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.567.6; Sat, 3 Mar 2018 00:20:48 +0000 Received: from KL1P15301MB0006.APCP153.PROD.OUTLOOK.COM ([10.170.167.17]) by KL1P15301MB0006.APCP153.PROD.OUTLOOK.COM ([10.170.167.17]) with mapi id 15.20.0567.002; Sat, 3 Mar 2018 00:20:48 +0000 From: Dexuan Cui To: "bhelgaas@google.com" , "linux-pci@vger.kernel.org" , KY Srinivasan , Stephen Hemminger CC: "linux-kernel@vger.kernel.org" , "driverdev-devel@linuxdriverproject.org" , Haiyang Zhang , "olaf@aepfle.de" , "apw@canonical.com" , "jasowang@redhat.com" , "vkuznets@redhat.com" , "marcelo.cerri@canonical.com" , Dexuan Cui , "stable@vger.kernel.org" , Jack Morgenstein Subject: [PATCH 3/3] PCI: hv: fix 2 hang issues in hv_compose_msi_msg() Thread-Topic: [PATCH 3/3] PCI: hv: fix 2 hang issues in hv_compose_msi_msg() Thread-Index: AQHTsoV6L95yC1dTN0ioPU1T64sHqw== Date: Sat, 3 Mar 2018 00:20:47 +0000 Message-ID: <20180303001947.20564-3-decui@microsoft.com> References: <20180303001947.20564-1-decui@microsoft.com> In-Reply-To: <20180303001947.20564-1-decui@microsoft.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.15.1 authentication-results: spf=none (sender IP is ) smtp.mailfrom=decui@microsoft.com; x-originating-ip: [52.168.54.252] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; KL1P15301MB0054; 6:R4WaJhpG4KHsj/7ZXrbrFw/0mjcuqViw6kaSi8xIlJIAsZdHh+kz76kleTUZiWJ34AJHfcwgXUwAE7lCrRicOPNz9QOE29lG+WM6qxlYzISqkNeLAeMq6PAhq339hRXGE85w0xcnARRGnGsAuLY2r6t5NNGf2B0XDLM67m6RUXXx/tk4AViFsrWBVVGtDifICdge9h5LN+rGNA3n0TQIKaSMyRQun0ZAMJ6CA/z77/N7tX15ufp9BS4qYL+qJt/iV8lBbFo6fGGYvHvH7DDRZTOMtCo1FjaY3Z8IPB+UdRh5DtDnR63cfa2kA9RdVNzS3R69HkZfSdGSbk8Mw7hiWGvcIb057PRbn3gI8Rbc/042C+uq+Oh4HngDl7ez1PgN; 5:0GDHJTrd+yT4ST+SkbntEU/0R6MF2Rdfx+w6wOq0k7S/PC9Im0Oxd/f9aYXjt22+1Rlql+sWcWWhOw4ljj1iUJ+vCo7H/ewPLMsgEyeSrMq+aCSkr3l784ECfjc4nm+5D/sdeyzPWe4BuD87g46ydQ+5iB40Z/vkmxq/KJV4EPM=; 24:LOvUWXXDgg36Sxa1ZX9MVvXlC9kH5WCrZEmjs/TJuygdu8TFrsB+fLrHV8yStYdH07oTikPCFbFGPXrjaXMnxG7j2R4N83BQiLB2ujX9U9s=; 7:vi5aAjl+IDD8XVY4HpWDzTmgoi7ETacs4djZdJsbH3+I+t0NtvnePQh7I+ypDLWzpvClaLRn+4qkkUl4R2fCszHxoPIorIOsXYBuVVZTDg+ivsNLqoC0skPQE0iWt5X+pBZX6k7geXmWVErzJON1xrPjDy0fAS7exOj0vc+pZxeRLlzLz60SdhGogA2R+XJTRK/N2uMhoCFk0yyo15YWHARykLNWMWfNhTky3queuqFgeQazMnVUUbfZLlUWYd3I x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: c3070aba-d2f6-4335-50d9-08d5809c9d01 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4604075)(3008032)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603307)(7193020); SRVR:KL1P15301MB0054; x-ms-traffictypediagnostic: KL1P15301MB0054: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(28532068793085)(89211679590171)(9452136761055); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(61425038)(6040501)(2401047)(8121501046)(5005006)(10201501046)(3002001)(93006095)(93001095)(3231220)(944501244)(52105095)(6055026)(61426038)(61427038)(6041288)(20161123564045)(20161123560045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(6072148)(201708071742011); SRVR:KL1P15301MB0054; BCL:0; PCL:0; RULEID:; SRVR:KL1P15301MB0054; x-forefront-prvs: 0600F93FE1 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(39860400002)(396003)(39380400002)(346002)(366004)(376002)(189003)(199004)(5660300001)(2906002)(4326008)(6116002)(8936002)(86362001)(2501003)(8676002)(6506007)(2900100001)(6436002)(14454004)(3660700001)(86612001)(305945005)(36756003)(99286004)(316002)(97736004)(25786009)(110136005)(1511001)(102836004)(54906003)(81166006)(10090500001)(81156014)(59450400001)(3280700002)(26005)(186003)(77096007)(6512007)(6636002)(76176011)(478600001)(2950100002)(3846002)(1076002)(22452003)(10290500003)(106356001)(66066001)(6486002)(50226002)(7416002)(68736007)(7736002)(105586002)(53936002)(22906009); DIR:OUT; SFP:1102; SCL:1; SRVR:KL1P15301MB0054; H:KL1P15301MB0006.APCP153.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: ksyLwVvxzSFbl08WN4s2dp5vNH86iz12D5fRmPUt6Sfrs1EHXnQEotzONA2DzDyQJMyCFZiZbhVtKaiNSzKcvt5LUFzD3BwN8ZtW2BULSBMFZeGWg5sj3ZPfSBMn2iMKxNsoUdZvUyzzAEPqXQgBMDbno51INKBcQ3/hxmr3D4I= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: c3070aba-d2f6-4335-50d9-08d5809c9d01 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Mar 2018 00:20:47.5321 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: KL1P15301MB0054 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org 1. With the patch "x86/vector/msi: Switch to global reservation mode" (4900be8360), the recent v4.15 and newer kernels always hang for 1-vCPU Hyper-V VM with SR-IOV. This is because when we reach hv_compose_msi_msg() by request_irq() -> request_threaded_irq() -> __setup_irq()->irq_startup() -> __irq_startup() -> irq_domain_activate_irq() -> ... -> msi_domain_activate() -> ... -> hv_compose_msi_msg(), local irq is disabled in __setup_irq(). Fix this by polling the channel. 2. If the host is ejecting the VF device before we reach hv_compose_msi_msg(), in a UP VM, we can hang in hv_compose_msi_msg() forever, because at this time the host doesn't respond to the CREATE_INTERRUPT request. This issue also happens to old kernels like v4.14, v4.13, etc. Fix this by polling the channel for the PCI_EJECT message and hpdev->state, and by checking the PCI vendor ID. Note: actually the above issues also happen to a SMP VM, if "hbus->hdev->channel->target_cpu == smp_processor_id()" is true. Signed-off-by: Dexuan Cui Tested-by: Adrian Suhov Tested-by: Chris Valean Cc: stable@vger.kernel.org Cc: Stephen Hemminger Cc: K. Y. Srinivasan Cc: Vitaly Kuznetsov Cc: Jack Morgenstein --- drivers/pci/host/pci-hyperv.c | 58 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 57 insertions(+), 1 deletion(-) diff --git a/drivers/pci/host/pci-hyperv.c b/drivers/pci/host/pci-hyperv.c index 57b1fb3ebdb9..32d6b03cdd40 100644 --- a/drivers/pci/host/pci-hyperv.c +++ b/drivers/pci/host/pci-hyperv.c @@ -522,6 +522,8 @@ struct hv_pci_compl { s32 completion_status; }; +static void hv_pci_onchannelcallback(void *context); + /** * hv_pci_generic_compl() - Invoked for a completion packet * @context: Set up by the sender of the packet. @@ -666,6 +668,31 @@ static void _hv_pcifront_read_config(struct hv_pci_dev *hpdev, int where, } } +static u16 hv_pcifront_get_vendor_id(struct hv_pci_dev *hpdev) +{ + u16 ret; + unsigned long flags; + void __iomem *addr = hpdev->hbus->cfg_addr + CFG_PAGE_OFFSET + + PCI_VENDOR_ID; + + spin_lock_irqsave(&hpdev->hbus->config_lock, flags); + + /* Choose the function to be read. (See comment above) */ + writel(hpdev->desc.win_slot.slot, hpdev->hbus->cfg_addr); + /* Make sure the function was chosen before we start reading. */ + mb(); + /* Read from that function's config space. */ + ret = readw(addr); + /* + * mb() is not required here, because the spin_unlock_irqrestore() + * is a barrier. + */ + + spin_unlock_irqrestore(&hpdev->hbus->config_lock, flags); + + return ret; +} + /** * _hv_pcifront_write_config() - Internal PCI config write * @hpdev: The PCI driver's representation of the device @@ -1108,8 +1135,37 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) * Since this function is called with IRQ locks held, can't * do normal wait for completion; instead poll. */ - while (!try_wait_for_completion(&comp.comp_pkt.host_event)) + while (!try_wait_for_completion(&comp.comp_pkt.host_event)) { + /* 0xFFFF means an invalid PCI VENDOR ID. */ + if (hv_pcifront_get_vendor_id(hpdev) == 0xFFFF) { + dev_err_once(&hbus->hdev->device, + "the device has gone\n"); + goto free_int_desc; + } + + /* + * When the higher level interrupt code calls us with + * interrupt disabled, we must poll the channel by calling + * the channel callback directly when channel->target_cpu is + * the current CPU. When the higher level interrupt code + * calls us with interrupt enabled, let's add the + * local_bh_disable()/enable() to avoid race. + */ + local_bh_disable(); + + if (hbus->hdev->channel->target_cpu == smp_processor_id()) + hv_pci_onchannelcallback(hbus); + + local_bh_enable(); + + if (hpdev->state == hv_pcichild_ejecting) { + dev_err_once(&hbus->hdev->device, + "the device is being ejected\n"); + goto free_int_desc; + } + udelay(100); + } if (comp.comp_pkt.completion_status < 0) { dev_err(&hbus->hdev->device,