From patchwork Sun Apr 26 20:46:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marcelo Henrique Cerri X-Patchwork-Id: 1277238 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 499KjP0kZjz9sRY; Mon, 27 Apr 2020 06:47:37 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1jSoBX-0004Mb-3v; Sun, 26 Apr 2020 20:47:31 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jSoAk-0003oG-Q6 for kernel-team@lists.ubuntu.com; Sun, 26 Apr 2020 20:46:43 +0000 Received: from mail-qv1-f71.google.com ([209.85.219.71]) by youngberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jSoAg-0001ff-U2 for kernel-team@lists.ubuntu.com; Sun, 26 Apr 2020 20:46:39 +0000 Received: by mail-qv1-f71.google.com with SMTP id z14so16343495qvv.6 for ; Sun, 26 Apr 2020 13:46:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hZ2lav/mNRJB0V0sD5MU0rzpk7BYj+bc6w+P6rIWMCQ=; b=k6MUMzsEvMfBcojM/he8EPi3kYggj0y73Mp+zblkY6kwgRe2bzbBDIAZPaqhg9IPRg oKRAeaMQrOv5AfIR+6Om42SU7RmUPGiqJ9CO9+Nqein+/7PuBEwt+VF5fIVHnBNYW6HX 2VRgGjcVZalByEKbXZy8+s9+sO/9t5QBHQmgJrbmPy2ryj7oSs3KLsyLhlCpU+ctYUF8 ultOZj50geyQ3kjP5T1/mCQ9yB6Lwbn0TSCXd3lheEUmBbLoZiYWv4Wbk+V4ZWfiGdr/ DDMGkJn8bpXOW4Y9jSRkub3WvwqH4F/5MRdWkQxQn8TmaCeFrfTRmyTYrqmv3x1U3ptt rv0Q== X-Gm-Message-State: AGi0PuZUXfMkgoB44Vxyjl92vo519c/AR+u5D6JDqqMc3Q7KWy7X/G3h pvPKi0sQKbS83Z2BypVPngsM+ci+tqy668oS2wxloVgkTQp33yWl+b1qtA42PMlF2BKo9HqJiwF oBd1bYmM+4JFo3TeOOUq0KkkPtpQ84HlVtqVKVzfQ X-Received: by 2002:ad4:45ec:: with SMTP id q12mr19508825qvu.157.1587933997484; Sun, 26 Apr 2020 13:46:37 -0700 (PDT) X-Google-Smtp-Source: APiQypIzbu2ZEWB/CL3l0lCjZ1RrWd9qjm3qoEitUl8a0cJjvPxu8jnls3C0KSzUJWhrAKMeezJWQg== X-Received: by 2002:ad4:45ec:: with SMTP id q12mr19508809qvu.157.1587933997078; Sun, 26 Apr 2020 13:46:37 -0700 (PDT) Received: from gallifrey.lan ([201.82.186.200]) by smtp.gmail.com with ESMTPSA id h63sm8261654qkd.49.2020.04.26.13.46.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2020 13:46:36 -0700 (PDT) From: Marcelo Henrique Cerri To: kernel-team@lists.ubuntu.com Subject: [eoan:linux-azure][PATCH 3/4] PCI: hv: Detect and fix Hyper-V PCI domain number collision Date: Sun, 26 Apr 2020 17:46:26 -0300 Message-Id: <20200426204627.121419-4-marcelo.cerri@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200426204627.121419-1-marcelo.cerri@canonical.com> References: <20200426203646.115503-1-marcelo.cerri@canonical.com> <20200426204627.121419-1-marcelo.cerri@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Haiyang Zhang BugLink: https://bugs.launchpad.net/bugs/1867220 Currently in Azure cloud, for passthrough devices, the host sets the device instance ID's bytes 8 - 15 to a value derived from the host HWID, which is the same on all devices in a VM. So, the device instance ID's bytes 8 and 9 provided by the host are no longer unique. This affects all Azure hosts since July 2018, and can cause device passthrough to VMs to fail because the bytes 8 and 9 are used as PCI domain number. Collision of domain numbers will cause the second device with the same domain number fail to load. In the cases of collision, we will detect and find another number that is not in use. Suggested-by: Michael Kelley Signed-off-by: Haiyang Zhang Signed-off-by: Lorenzo Pieralisi Acked-by: Sasha Levin (cherry picked from commit be700103efd1050808db1cf00e52c3a2837bf802) Signed-off-by: Marcelo Henrique Cerri --- drivers/pci/controller/pci-hyperv.c | 92 +++++++++++++++++++++++++---- 1 file changed, 79 insertions(+), 13 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index 2b53976cd9f9..4caa3388692a 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -2510,6 +2510,48 @@ static void put_hvpcibus(struct hv_pcibus_device *hbus) complete(&hbus->remove_event); } +#define HVPCI_DOM_MAP_SIZE (64 * 1024) +static DECLARE_BITMAP(hvpci_dom_map, HVPCI_DOM_MAP_SIZE); + +/* + * PCI domain number 0 is used by emulated devices on Gen1 VMs, so define 0 + * as invalid for passthrough PCI devices of this driver. + */ +#define HVPCI_DOM_INVALID 0 + +/** + * hv_get_dom_num() - Get a valid PCI domain number + * Check if the PCI domain number is in use, and return another number if + * it is in use. + * + * @dom: Requested domain number + * + * return: domain number on success, HVPCI_DOM_INVALID on failure + */ +static u16 hv_get_dom_num(u16 dom) +{ + unsigned int i; + + if (test_and_set_bit(dom, hvpci_dom_map) == 0) + return dom; + + for_each_clear_bit(i, hvpci_dom_map, HVPCI_DOM_MAP_SIZE) { + if (test_and_set_bit(i, hvpci_dom_map) == 0) + return i; + } + + return HVPCI_DOM_INVALID; +} + +/** + * hv_put_dom_num() - Mark the PCI domain number as free + * @dom: Domain number to be freed + */ +static void hv_put_dom_num(u16 dom) +{ + clear_bit(dom, hvpci_dom_map); +} + /** * hv_pci_probe() - New VMBus channel probe, for a root PCI bus * @hdev: VMBus's tracking struct for this root PCI bus @@ -2521,6 +2563,7 @@ static int hv_pci_probe(struct hv_device *hdev, const struct hv_vmbus_device_id *dev_id) { struct hv_pcibus_device *hbus; + u16 dom_req, dom; int ret; /* @@ -2535,19 +2578,34 @@ static int hv_pci_probe(struct hv_device *hdev, hbus->state = hv_pcibus_init; /* - * The PCI bus "domain" is what is called "segment" in ACPI and - * other specs. Pull it from the instance ID, to get something - * unique. Bytes 8 and 9 are what is used in Windows guests, so - * do the same thing for consistency. Note that, since this code - * only runs in a Hyper-V VM, Hyper-V can (and does) guarantee - * that (1) the only domain in use for something that looks like - * a physical PCI bus (which is actually emulated by the - * hypervisor) is domain 0 and (2) there will be no overlap - * between domains derived from these instance IDs in the same - * VM. + * The PCI bus "domain" is what is called "segment" in ACPI and other + * specs. Pull it from the instance ID, to get something usually + * unique. In rare cases of collision, we will find out another number + * not in use. + * + * Note that, since this code only runs in a Hyper-V VM, Hyper-V + * together with this guest driver can guarantee that (1) The only + * domain used by Gen1 VMs for something that looks like a physical + * PCI bus (which is actually emulated by the hypervisor) is domain 0. + * (2) There will be no overlap between domains (after fixing possible + * collisions) in the same VM. */ - hbus->sysdata.domain = hdev->dev_instance.b[9] | - hdev->dev_instance.b[8] << 8; + dom_req = hdev->dev_instance.b[8] << 8 | hdev->dev_instance.b[9]; + dom = hv_get_dom_num(dom_req); + + if (dom == HVPCI_DOM_INVALID) { + dev_err(&hdev->device, + "Unable to use dom# 0x%hx or other numbers", dom_req); + ret = -EINVAL; + goto free_bus; + } + + if (dom != dom_req) + dev_info(&hdev->device, + "PCI dom# 0x%hx has collision, using 0x%hx", + dom_req, dom); + + hbus->sysdata.domain = dom; hbus->hdev = hdev; refcount_set(&hbus->remove_lock, 1); @@ -2562,7 +2620,7 @@ static int hv_pci_probe(struct hv_device *hdev, hbus->sysdata.domain); if (!hbus->wq) { ret = -ENOMEM; - goto free_bus; + goto free_dom; } ret = vmbus_open(hdev->channel, pci_ring_size, pci_ring_size, NULL, 0, @@ -2639,6 +2697,8 @@ static int hv_pci_probe(struct hv_device *hdev, vmbus_close(hdev->channel); destroy_wq: destroy_workqueue(hbus->wq); +free_dom: + hv_put_dom_num(hbus->sysdata.domain); free_bus: free_page((unsigned long)hbus); return ret; @@ -2720,6 +2780,9 @@ static int hv_pci_remove(struct hv_device *hdev) put_hvpcibus(hbus); wait_for_completion(&hbus->remove_event); destroy_workqueue(hbus->wq); + + hv_put_dom_num(hbus->sysdata.domain); + free_page((unsigned long)hbus); return 0; } @@ -2747,6 +2810,9 @@ static void __exit exit_hv_pci_drv(void) static int __init init_hv_pci_drv(void) { + /* Set the invalid domain number's bit, so it will not be used */ + set_bit(HVPCI_DOM_INVALID, hvpci_dom_map); + return vmbus_driver_register(&hv_pci_drv); }