From patchwork Wed Oct 18 13:58:17 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 827608 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="njvgkZ7i"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3yHDGS0Sgqz9t4X for ; Thu, 19 Oct 2017 01:00:16 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753534AbdJRN6a (ORCPT ); Wed, 18 Oct 2017 09:58:30 -0400 Received: from mail-wr0-f193.google.com ([209.85.128.193]:53294 "EHLO mail-wr0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753013AbdJRN62 (ORCPT ); Wed, 18 Oct 2017 09:58:28 -0400 Received: by mail-wr0-f193.google.com with SMTP id y44so5128704wry.10; Wed, 18 Oct 2017 06:58:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Y2bEKafksFT3a6FsrdA76mMSe6J+UJw6583d4sySj3Y=; b=njvgkZ7izTnuApS+ZP0xPXxBKxvoeRdQRKnV5YCplJoFHHXZ5LaJM6qfARaJa7OXsj 64ixU5IPVvNCvx3dyy+eBneaZde4Io2Ou9YeAIZU326YSWoCrDTXJASAwT8ovbBPpVOm 6lumFufDD2MtQbeuv3h5pRCIroCdjAEo1KELxPCrA5mzSGAJBuymdKuSDJmfoyTO2GQF RoF+77UOsMxnDKuXwJJ3xyzJTSpW3KTQf63vpPY3IOUOrEJY9tlTLdTFx64F062KSLM7 krfQeqffP0MqZQzgxGkRBG1ERTXcBwOt5T0q63ooAGn01sh+0qGXElb+km8V08hm0ObQ H82A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Y2bEKafksFT3a6FsrdA76mMSe6J+UJw6583d4sySj3Y=; b=QlyZfRbLinFe8o9drma2hXigxNVJPcy89dJQhsmgZwei8CoL+M46YCdmISWr44qwsN KkjV3gcMWOu3sFLHiN44Pr0BT58s19oqIHPtP4JzSgCG5/I1Vt/ctIAPlkQdc/HKq1OX 0TM0Nb4xnrnfQLslSVPz74TwXrvQ23fzlMGWMwPxZCu4y6QjEZP9oFhcRwAQiUjmeCET HoL+DMkuO8ExtNptlkymzZAClwpX9Yx0ZEj65e8BusX9u8ZaB/Tam8DSVL2QYVl8i50H 8RmSh8iyaSgAdGSefXgg7ACDsmUAfImZJgIE4BWH191mLmWU8i8RXPJG8eS70R6FqzUV BczA== X-Gm-Message-State: AMCzsaXsXK4AXpgYTFV6t1eRS/D2/sxca9prZyOMrijSjBUPxJHZPI4l O6ArmKeQdcElqL6xOn1Tt9E= X-Google-Smtp-Source: ABhQp+RZXH/RNJgB+ir/dHqOxAJixmv/LULB1mNOLPL5JcSFTH3e1wXWsPwO0wZ/mOlpabi964Hw2A== X-Received: by 10.223.148.162 with SMTP id 31mr7493961wrr.149.1508335107078; Wed, 18 Oct 2017 06:58:27 -0700 (PDT) Received: from localhost.localdomain ([2a02:908:1251:7981:4537:45bc:69b6:7f1e]) by smtp.gmail.com with ESMTPSA id o24sm15780699wmi.39.2017.10.18.06.58.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Oct 2017 06:58:26 -0700 (PDT) From: "=?UTF-8?q?Christian=20K=C3=B6nig?=" X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: helgaas@kernel.org, linux-pci@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, amd-gfx@lists.freedesktop.org Subject: [PATCH v9 1/5] PCI: add a define for the PCI resource type mask v2 Date: Wed, 18 Oct 2017 15:58:17 +0200 Message-Id: <20171018135821.3248-2-deathsimple@vodafone.de> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171018135821.3248-1-deathsimple@vodafone.de> References: <20171018135821.3248-1-deathsimple@vodafone.de> MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Christian König We use this mask multiple times in the bus setup. v2: fix some style nit picks Signed-off-by: Christian König Reviewed-by: Andy Shevchenko --- drivers/pci/pci.h | 3 +++ drivers/pci/setup-bus.c | 12 +++--------- 2 files changed, 6 insertions(+), 9 deletions(-) diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 22e061738c6f..5c475edc78c2 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -4,6 +4,9 @@ #define PCI_FIND_CAP_TTL 48 #define PCI_VSEC_ID_INTEL_TBT 0x1234 /* Thunderbolt */ +#define PCI_RES_TYPE_MASK \ + (IORESOURCE_IO | IORESOURCE_MEM | IORESOURCE_PREFETCH |\ + IORESOURCE_MEM_64) extern const unsigned char pcie_link_speed[]; diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index 958da7db9033..37450d9e1132 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1523,8 +1523,6 @@ static void pci_bridge_release_resources(struct pci_bus *bus, { struct pci_dev *dev = bus->self; struct resource *r; - unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM | - IORESOURCE_PREFETCH | IORESOURCE_MEM_64; unsigned old_flags = 0; struct resource *b_res; int idx = 1; @@ -1567,7 +1565,7 @@ static void pci_bridge_release_resources(struct pci_bus *bus, */ release_child_resources(r); if (!release_resource(r)) { - type = old_flags = r->flags & type_mask; + type = old_flags = r->flags & PCI_RES_TYPE_MASK; dev_printk(KERN_DEBUG, &dev->dev, "resource %d %pR released\n", PCI_BRIDGE_RESOURCES + idx, r); /* keep the old size */ @@ -1758,8 +1756,6 @@ void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus) enum release_type rel_type = leaf_only; LIST_HEAD(fail_head); struct pci_dev_resource *fail_res; - unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM | - IORESOURCE_PREFETCH | IORESOURCE_MEM_64; int pci_try_num = 1; enum enable_type enable_local; @@ -1818,7 +1814,7 @@ void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus) */ list_for_each_entry(fail_res, &fail_head, list) pci_bus_release_bridge_resources(fail_res->dev->bus, - fail_res->flags & type_mask, + fail_res->flags & PCI_RES_TYPE_MASK, rel_type); /* restore size and flags */ @@ -1862,8 +1858,6 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge) LIST_HEAD(fail_head); struct pci_dev_resource *fail_res; int retval; - unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM | - IORESOURCE_PREFETCH | IORESOURCE_MEM_64; again: __pci_bus_size_bridges(parent, &add_list); @@ -1889,7 +1883,7 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge) */ list_for_each_entry(fail_res, &fail_head, list) pci_bus_release_bridge_resources(fail_res->dev->bus, - fail_res->flags & type_mask, + fail_res->flags & PCI_RES_TYPE_MASK, whole_subtree); /* restore size and flags */ From patchwork Wed Oct 18 13:58:18 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 827609 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="RzMaDSQX"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3yHDGm3qm5z9t4X for ; Thu, 19 Oct 2017 01:00:32 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754349AbdJROAP (ORCPT ); Wed, 18 Oct 2017 10:00:15 -0400 Received: from mail-wr0-f193.google.com ([209.85.128.193]:54829 "EHLO mail-wr0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753458AbdJRN63 (ORCPT ); Wed, 18 Oct 2017 09:58:29 -0400 Received: by mail-wr0-f193.google.com with SMTP id o44so5122837wrf.11; Wed, 18 Oct 2017 06:58:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=l2sLUKpgXvRZcY1mRaVccP+nbfhI/wPE9taDk4Skaps=; b=RzMaDSQX5XjL2vkyvCo+axpwchUp8cVPyoAWe1OlzDnltvFCAwF5cEqBJsiqt+5iUb nhy8JU32J5geCcYQZBHB4vWvzV3m4+UJQ1ByFd/AIKya7hxPyi31z+hPbkWwDvsNrPtN BcsVlEcx2ebJq0u6Tf4YwzUje0vJr0sRYFycDasj+VxKIYmevjEC15k5jbrsJ07pMQsq 6RIEyN//ySW1UPbfLcT7XQ2FACN2njavTzc2sn4G7xqzDRIauZ4ybLGLMwuPxNQM5foz 7mHnolDUb1Uj3Gad9fF9a7BgaSzGgXvl6ZIa1TiHu3YslJSK7qQUy8YdaeBis/hlxnay fPNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=l2sLUKpgXvRZcY1mRaVccP+nbfhI/wPE9taDk4Skaps=; b=OT5XDDzmnMBDduW3SRGkgILdtNs5eDL3PSK5A1C6PzQ69qzGVBUFTbLtM3lesCu9b3 mu+7r9jpFDhLo4dPMxkL8cGoYwgzyr+SUafGyjRXuNgTicbGLJJHCyQNQ+pGXhSRHjMb IFt46809owdZPEc0TgFDIOtCQlYGuKAyE5AdbSNfCbIBxoiv0x6ZjiieL709BZGPYMdf 6P4ajzkLPR9m9wOdXOgtJnJ3rGzoL399LHR3TKGSKbzo2wMhr9AAjIjmRuZ4lBRWHH0X r9LSiFXoXEGU36ankoGawMwICun9cEseyg+HIdAvWNX6BNlxkqFR8eD4jroaIcPKpfca AVKA== X-Gm-Message-State: AMCzsaWCOkmFYG1F6jXZQgdEb+V1CprtUbwnPevoOjmoqOerhkt0YYa/ pCO8CNC4spqGjyOrGOONg2A= X-Google-Smtp-Source: ABhQp+RgHSPJPJtw8ENvZ0PdJeU8BbE7pdKz/g3/4ycgvPkU9HNUIJxXpQfLev1gfK4MlQPNYNIHow== X-Received: by 10.223.191.10 with SMTP id p10mr1710801wrh.127.1508335107941; Wed, 18 Oct 2017 06:58:27 -0700 (PDT) Received: from localhost.localdomain ([2a02:908:1251:7981:4537:45bc:69b6:7f1e]) by smtp.gmail.com with ESMTPSA id o24sm15780699wmi.39.2017.10.18.06.58.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Oct 2017 06:58:27 -0700 (PDT) From: "=?UTF-8?q?Christian=20K=C3=B6nig?=" X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: helgaas@kernel.org, linux-pci@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, amd-gfx@lists.freedesktop.org Subject: [PATCH v9 2/5] PCI: add resizeable BAR infrastructure v5 Date: Wed, 18 Oct 2017 15:58:18 +0200 Message-Id: <20171018135821.3248-3-deathsimple@vodafone.de> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171018135821.3248-1-deathsimple@vodafone.de> References: <20171018135821.3248-1-deathsimple@vodafone.de> MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Christian König Just the defines and helper functions to read the possible sizes of a BAR and update it's size. See https://pcisig.com/sites/default/files/specification_documents/ECN_Resizable-BAR_24Apr2008.pdf and PCIe r3.1, sec 7.22. This is useful for hardware with large local storage (mostly GFX) which only expose 256MB BARs initially to be compatible with 32bit systems. v2: provide read helper as well v3: improve function names, use unsigned values, add better comments. v4: move definition, improve commit message, s/bar/BAR/ v5: split out helper to find ctrl reg pos, style fixes, comment fixes, add pci_rbar_size_to_bytes as well Signed-off-by: Christian König Reviewed-by: Andy Shevchenko --- drivers/pci/pci.c | 104 ++++++++++++++++++++++++++++++++++++++++++ drivers/pci/pci.h | 8 ++++ include/uapi/linux/pci_regs.h | 11 ++++- 3 files changed, 121 insertions(+), 2 deletions(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index b4b7eab29400..3aca7393c43c 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -2957,6 +2957,110 @@ bool pci_acs_path_enabled(struct pci_dev *start, } /** + * pci_rbar_find_pos - find position of resize ctrl reg for BAR + * @dev: PCI device + * @bar: BAR to find + * + * Helper to find the postion of the ctrl register for a BAR. + * Returns -ENOTSUPP of resizeable BARs are not supported at all. + * Returns -ENOENT if not ctrl register for the BAR could be found. + */ +static int pci_rbar_find_pos(struct pci_dev *pdev, int bar) +{ + unsigned int pos, nbars; + unsigned int i; + u32 ctrl; + + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR); + if (!pos) + return -ENOTSUPP; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); + nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT; + + for (i = 0; i < nbars; ++i, pos += 8) { + int bar_idx; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); + bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >> + PCI_REBAR_CTRL_BAR_IDX_SHIFT; + if (bar_idx == bar) + return pos; + } + + return -ENOENT; +} + +/** + * pci_rbar_get_possible_sizes - get possible sizes for BAR + * @dev: PCI device + * @bar: BAR to query + * + * Get the possible sizes of a resizeable BAR as bitmask defined in the spec + * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizeable. + */ +u32 pci_rbar_get_possible_sizes(struct pci_dev *pdev, int bar) +{ + u32 cap; + int pos; + + pos = pci_rbar_find_pos(pdev, bar); + if (pos < 0) + return 0; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CAP, &cap); + return (cap & PCI_REBAR_CTRL_SIZES_MASK) >> + PCI_REBAR_CTRL_SIZES_SHIFT; +} + +/** + * pci_rbar_get_current_size - get the current size of a BAR + * @dev: PCI device + * @bar: BAR to set size to + * + * Read the size of a BAR from the resizeable BAR config. + * Returns size if found or negative error code. + */ +int pci_rbar_get_current_size(struct pci_dev *pdev, int bar) +{ + u32 ctrl; + int pos; + + pos = pci_rbar_find_pos(pdev, bar); + if (pos < 0) + return pos; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); + return (ctrl & PCI_REBAR_CTRL_BAR_SIZE_MASK) >> + PCI_REBAR_CTRL_BAR_SIZE_SHIFT; +} + +/** + * pci_rbar_set_size - set a new size for a BAR + * @dev: PCI device + * @bar: BAR to set size to + * @size: new size as defined in the spec (0=1MB, 19=512GB) + * + * Set the new size of a BAR as defined in the spec. + * Returns zero if resizing was successful, error code otherwise. + */ +int pci_rbar_set_size(struct pci_dev *pdev, int bar, int size) +{ + u32 ctrl; + int pos; + + pos = pci_rbar_find_pos(pdev, bar); + if (pos < 0) + return pos; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); + ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE_MASK; + ctrl |= size << PCI_REBAR_CTRL_BAR_SIZE_SHIFT; + pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl); + return 0; +} + +/** * pci_swizzle_interrupt_pin - swizzle INTx for device behind bridge * @dev: the PCI device * @pin: the INTx pin (1=INTA, 2=INTB, 3=INTC, 4=INTD) diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 5c475edc78c2..1681895366dc 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -368,4 +368,12 @@ int acpi_get_rc_resources(struct device *dev, const char *hid, u16 segment, struct resource *res); #endif +u32 pci_rbar_get_possible_sizes(struct pci_dev *pdev, int bar); +int pci_rbar_get_current_size(struct pci_dev *pdev, int bar); +int pci_rbar_set_size(struct pci_dev *pdev, int bar, int size); +static inline u64 pci_rbar_size_to_bytes(int size) +{ + return 1ULL << (size + 20); +} + #endif /* DRIVERS_PCI_H */ diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h index c22d3ebaca20..d6a075de6d43 100644 --- a/include/uapi/linux/pci_regs.h +++ b/include/uapi/linux/pci_regs.h @@ -943,9 +943,16 @@ #define PCI_SATA_SIZEOF_LONG 16 /* Resizable BARs */ +#define PCI_REBAR_CAP 4 /* capability register */ +#define PCI_REBAR_CTRL_SIZES_MASK (0xFFFFF << 4) /* mask for sizes */ +#define PCI_REBAR_CTRL_SIZES_SHIFT 4 /* shift for sizes */ #define PCI_REBAR_CTRL 8 /* control register */ -#define PCI_REBAR_CTRL_NBAR_MASK (7 << 5) /* mask for # bars */ -#define PCI_REBAR_CTRL_NBAR_SHIFT 5 /* shift for # bars */ +#define PCI_REBAR_CTRL_BAR_IDX_MASK (7 << 0) /* mask for BAR index */ +#define PCI_REBAR_CTRL_BAR_IDX_SHIFT 0 /* shift for BAR index */ +#define PCI_REBAR_CTRL_NBAR_MASK (7 << 5) /* mask for # BARs */ +#define PCI_REBAR_CTRL_NBAR_SHIFT 5 /* shift for # BARs */ +#define PCI_REBAR_CTRL_BAR_SIZE_MASK (0x1F << 8) /* mask for BAR size */ +#define PCI_REBAR_CTRL_BAR_SIZE_SHIFT 8 /* shift for BAR size */ /* Dynamic Power Allocation */ #define PCI_DPA_CAP 4 /* capability register */ From patchwork Wed Oct 18 13:58:19 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 827606 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="mVT7OE89"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3yHDFz6mTKz9t4X for ; Thu, 19 Oct 2017 00:59:51 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754601AbdJRN7c (ORCPT ); Wed, 18 Oct 2017 09:59:32 -0400 Received: from mail-wr0-f194.google.com ([209.85.128.194]:45687 "EHLO mail-wr0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753506AbdJRN6a (ORCPT ); Wed, 18 Oct 2017 09:58:30 -0400 Received: by mail-wr0-f194.google.com with SMTP id k7so5136126wre.2; Wed, 18 Oct 2017 06:58:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=h74dVIlA0MHTpK1ga4UUjhX1rUkHEzc2XKDK9kQLpSw=; b=mVT7OE89pK4j6mdmjRtON1qkE8Tuk7Xn7EMLkE3mihVK37YOaytABfQe87wREq0vpR J3fXDsBDOj7fMFGw65NnSyqoxEaFU14Gt2Dx+nfmfCWzPM8TD/5xnCozvdIll363h0AB cOs5JoNdVXqt9TCr0NtJhAqMqpRqVDdCUVEgaHOnX6rB4p6nzz8cLAcDasjPZhfjgCti T9GHYiQjg4qNqcq6L0cKH+5DXDSxlBfiT51S2Nllsnk4OubVisgpor0GbH13P57hLNC5 389Bge1WfUDd/mhjzGPCsjistOMdOqadG9BhGb2rm+QmYoWD/l8oiAsDPb5zGrwVup7S 7nJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=h74dVIlA0MHTpK1ga4UUjhX1rUkHEzc2XKDK9kQLpSw=; b=QRi54rAINB0KhzPrs2yq6g7fGaM5NzFyApUBJexk7CUa90HMtOc4az3I9d4i+R+HRn c/TpWHdgNkijnkiWPQ0QcYoJCFEK/aNxkT2jpNW1HY01rSztGPN99S4Bx9+QXfMWgitG +jE4SE2Bx9IJP0OqZbKNj9sICmfdxpdCSG6bRvz1J/ERWIPyunb9mmvkAaoTr7e3o/84 dXwkIIz/jkpMrZQEaVGF3A0HTj9BwdoaAmMG0NKI3XFQrkzBQOeIFntYw1gxw2J+FmPr s1mDc36b60N3AVCE3zCzMPx+Z522NVzLJQ3LRrycBWCTO6nFEOl6xtjOkOJjvB84O08L ucbQ== X-Gm-Message-State: AMCzsaUgzurqtyy8w3Lf9OO+QZcCsvjVPeLue5jVSDPVNpKYd4gTEI4F BpdyMTjJ/Onc5w4fqmF+KWY= X-Google-Smtp-Source: ABhQp+T1uL1ETpcRE4IAd0hIFcbNLFtc6cVEmSbKzfg7VmKEo1xlwWkgmM/7DNAQFh2+A+d+LHovTg== X-Received: by 10.223.142.244 with SMTP id q107mr6723280wrb.208.1508335108745; Wed, 18 Oct 2017 06:58:28 -0700 (PDT) Received: from localhost.localdomain ([2a02:908:1251:7981:4537:45bc:69b6:7f1e]) by smtp.gmail.com with ESMTPSA id o24sm15780699wmi.39.2017.10.18.06.58.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Oct 2017 06:58:28 -0700 (PDT) From: "=?UTF-8?q?Christian=20K=C3=B6nig?=" X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: helgaas@kernel.org, linux-pci@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, amd-gfx@lists.freedesktop.org Subject: [PATCH v9 3/5] PCI: add functionality for resizing resources v7 Date: Wed, 18 Oct 2017 15:58:19 +0200 Message-Id: <20171018135821.3248-4-deathsimple@vodafone.de> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171018135821.3248-1-deathsimple@vodafone.de> References: <20171018135821.3248-1-deathsimple@vodafone.de> MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Christian König This allows device drivers to request resizing their BARs. The function only tries to reprogram the windows of the bridge directly above the requesting device and only the BAR of the same type (usually mem, 64bit, prefetchable). This is done to make sure not to disturb other drivers by changing the BARs of their devices. Drivers should use the following sequence to resize their BARs: 1. Disable memory decoding of your device using the PCI cfg dword. 2. Use pci_release_resource() to release all BARs which can move during the resize. Including the one you want to resize. 3. Call pci_resize_resource() for each BAR you want to resize. 4. Call pci_assign_unassigned_bus_resources() to reassign new locations for all BARs which are not resized, but could move. 5. If everything worked as expected enable memory decoding in your device again using the PCI cfg dword. v2: rebase on changes in rbar support v3: style cleanups, fail if memory decoding is enabled or resources still allocated, resize all unused bridge BARs, drop calling pci_reenable_device v4: print resources before releasing them, style cleanups, use pci_rbar_size_to_bytes, use PCI_RES_TYPE_MASK v5: use next pointer to simplify loop v6: move reassigning resources on error to driver side v7: Document in the commit message how to use the new function. Signed-off-by: Christian König --- drivers/pci/setup-bus.c | 98 +++++++++++++++++++++++++++++++++++++++++++++++++ drivers/pci/setup-res.c | 58 +++++++++++++++++++++++++++++ include/linux/pci.h | 3 ++ 3 files changed, 159 insertions(+) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index 37450d9e1132..03af25b3eec4 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1908,6 +1908,104 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge) } EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources); +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type) +{ + struct pci_dev_resource *dev_res; + struct pci_dev *next; + LIST_HEAD(saved); + LIST_HEAD(added); + LIST_HEAD(failed); + unsigned int i; + int ret; + + /* Walk to the root hub, releasing bridge BARs when possible */ + next = bridge; + do { + bridge = next; + for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; + i++) { + struct resource *res = &bridge->resource[i]; + + if ((res->flags ^ type) & PCI_RES_TYPE_MASK) + continue; + + /* Ignore BARs which are still in use */ + if (res->child) + continue; + + ret = add_to_list(&saved, bridge, res, 0, 0); + if (ret) + goto cleanup; + + dev_info(&bridge->dev, "BAR %d: releasing %pR\n", + i, res); + + if (res->parent) + release_resource(res); + res->start = 0; + res->end = 0; + break; + } + if (i == PCI_BRIDGE_RESOURCE_END) + break; + + next = bridge->bus ? bridge->bus->self : NULL; + } while (next); + + if (list_empty(&saved)) + return -ENOENT; + + __pci_bus_size_bridges(bridge->subordinate, &added); + __pci_bridge_assign_resources(bridge, &added, &failed); + BUG_ON(!list_empty(&added)); + + if (!list_empty(&failed)) { + ret = -ENOSPC; + goto cleanup; + } + + list_for_each_entry(dev_res, &saved, list) { + /* Skip the bridge we just assigned resources for. */ + if (bridge == dev_res->dev) + continue; + + bridge = dev_res->dev; + pci_setup_bridge(bridge->subordinate); + } + + free_list(&saved); + return 0; + +cleanup: + /* restore size and flags */ + list_for_each_entry(dev_res, &failed, list) { + struct resource *res = dev_res->res; + + res->start = dev_res->start; + res->end = dev_res->end; + res->flags = dev_res->flags; + } + free_list(&failed); + + /* Revert to the old configuration */ + list_for_each_entry(dev_res, &saved, list) { + struct resource *res = dev_res->res; + + bridge = dev_res->dev; + i = res - bridge->resource; + + res->start = dev_res->start; + res->end = dev_res->end; + res->flags = dev_res->flags; + + pci_claim_resource(bridge, i); + pci_setup_bridge(bridge->subordinate); + } + free_list(&saved); + + return ret; +} + void pci_assign_unassigned_bus_resources(struct pci_bus *bus) { struct pci_dev *dev; diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c index 85774b7a316a..1d558e18270d 100644 --- a/drivers/pci/setup-res.c +++ b/drivers/pci/setup-res.c @@ -383,6 +383,64 @@ int pci_reassign_resource(struct pci_dev *dev, int resno, resource_size_t addsiz return 0; } +void pci_release_resource(struct pci_dev *dev, int resno) +{ + struct resource *res = dev->resource + resno; + + dev_info(&dev->dev, "BAR %d: releasing %pR\n", resno, res); + release_resource(res); + res->end = resource_size(res) - 1; + res->start = 0; + res->flags |= IORESOURCE_UNSET; +} +EXPORT_SYMBOL(pci_release_resource); + +int pci_resize_resource(struct pci_dev *dev, int resno, int size) +{ + struct resource *res = dev->resource + resno; + int old, ret; + u32 sizes; + u16 cmd; + + /* Make sure the resource isn't assigned before resizing it. */ + if (!(res->flags & IORESOURCE_UNSET)) + return -EBUSY; + + pci_read_config_word(dev, PCI_COMMAND, &cmd); + if (cmd & PCI_COMMAND_MEMORY) + return -EBUSY; + + sizes = pci_rbar_get_possible_sizes(dev, resno); + if (!sizes) + return -ENOTSUPP; + + if (!(sizes & BIT(size))) + return -EINVAL; + + old = pci_rbar_get_current_size(dev, resno); + if (old < 0) + return old; + + ret = pci_rbar_set_size(dev, resno, size); + if (ret) + return ret; + + res->end = res->start + pci_rbar_size_to_bytes(size) - 1; + + /* Check if the new config works by trying to assign everything. */ + ret = pci_reassign_bridge_resources(dev->bus->self, res->flags); + if (ret) + goto error_resize; + + return 0; + +error_resize: + pci_rbar_set_size(dev, resno, old); + res->end = res->start + pci_rbar_size_to_bytes(old) - 1; + return ret; +} +EXPORT_SYMBOL(pci_resize_resource); + int pci_enable_resources(struct pci_dev *dev, int mask) { u16 cmd, old_cmd; diff --git a/include/linux/pci.h b/include/linux/pci.h index a75c13673852..c21949a6ea65 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1081,6 +1081,8 @@ void pci_reset_bridge_secondary_bus(struct pci_dev *dev); void pci_update_resource(struct pci_dev *dev, int resno); int __must_check pci_assign_resource(struct pci_dev *dev, int i); int __must_check pci_reassign_resource(struct pci_dev *dev, int i, resource_size_t add_size, resource_size_t align); +void pci_release_resource(struct pci_dev *dev, int resno); +int __must_check pci_resize_resource(struct pci_dev *dev, int i, int size); int pci_select_bars(struct pci_dev *dev, unsigned long flags); bool pci_device_is_present(struct pci_dev *pdev); void pci_ignore_hotplug(struct pci_dev *dev); @@ -1159,6 +1161,7 @@ void pci_assign_unassigned_resources(void); void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge); void pci_assign_unassigned_bus_resources(struct pci_bus *bus); void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus); +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type); void pdev_enable_device(struct pci_dev *); int pci_enable_resources(struct pci_dev *, int mask); void pci_fixup_irqs(u8 (*)(struct pci_dev *, u8 *), From patchwork Wed Oct 18 13:58:20 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 827607 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="g7tz3xes"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3yHDGL3J33z9t4X for ; Thu, 19 Oct 2017 01:00:10 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754650AbdJRN7a (ORCPT ); Wed, 18 Oct 2017 09:59:30 -0400 Received: from mail-wr0-f194.google.com ([209.85.128.194]:49322 "EHLO mail-wr0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753549AbdJRN6a (ORCPT ); Wed, 18 Oct 2017 09:58:30 -0400 Received: by mail-wr0-f194.google.com with SMTP id g90so5130325wrd.6; Wed, 18 Oct 2017 06:58:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=9mvetThlt4alMNH2P1jGl+Iqkca52CLGF1BzPx+T6Ls=; b=g7tz3xesJblB3GRldwJBVAvBE7EJZAJZziegoshk8gblFUKRCYbfWFLCCO8dD+qWZd cHF8a2gfVNrGZHUfCayWwiQH6WUs2SgB8E43pvV+nd2A86hC8sNi+0ltOvwbEzjL/kFB yzizSu3/6rZPwGmLyfTy/wQRz0v/kJ2YhKZNNPMyt1X2O5pZw8RqPef3/+DCckf3WopV lgvSjYt1cca46kAUY5k5cXipcr1ShdF56G0D0EaureLvvlQ/UgsjHc5bTN3E6bcydP5h J1j63kMol51Ybru2Z2lUyTX+J2WG84mGU34JuI6gVhybniagYLRQWoKs5usao10Uq2mZ GlHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9mvetThlt4alMNH2P1jGl+Iqkca52CLGF1BzPx+T6Ls=; b=eC+1n0k9Tqo8wgnyTX0dB9dillltjEPbBssXeCTAAxQ+j6y34AxKtwJ09pq6d3ThuF yAKP+0Onq1ftdeWTfCsybiuA3fptoh/u3PGyPe7EOClXFYSM19ocSS9Oz10CsuR7YdQg aYgI3CWwTZiV4tpRyXX1l4eaNUNzMDMQnNakFOTMD9/SAU1k05zS+vW2SB3avZhB3P8/ TMKlJfxpXCxSakqo+bBqcMUSis0z4cwumFyrtg8USk8aKMPlJaGmtI5MuvoLpWaNnIY+ fSkUzEI0mPoSfVEXBVoDYH7RVnw7whCKu85K1UrVZVVcVF6vsPvwr3MLDhNSxfo4BxjH 7FsA== X-Gm-Message-State: AMCzsaXHJADH3GgG2i/qLw4Q9kFc1Ft5cWUZYe6Mi4c1tAtzapuD+5wB i3SltJlRCjkLqeJBYyH+JUc= X-Google-Smtp-Source: ABhQp+QlIyxbvMbnCGe5D11+KvX1kffSQltQC9UTYtZZAzYFjZSZzlQ27V7cNyEvIM2ESmOeagO+WA== X-Received: by 10.223.182.19 with SMTP id f19mr7376777wre.166.1508335109498; Wed, 18 Oct 2017 06:58:29 -0700 (PDT) Received: from localhost.localdomain ([2a02:908:1251:7981:4537:45bc:69b6:7f1e]) by smtp.gmail.com with ESMTPSA id o24sm15780699wmi.39.2017.10.18.06.58.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Oct 2017 06:58:29 -0700 (PDT) From: "=?UTF-8?q?Christian=20K=C3=B6nig?=" X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: helgaas@kernel.org, linux-pci@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, amd-gfx@lists.freedesktop.org Subject: [PATCH v9 4/5] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors v5 Date: Wed, 18 Oct 2017 15:58:20 +0200 Message-Id: <20171018135821.3248-5-deathsimple@vodafone.de> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171018135821.3248-1-deathsimple@vodafone.de> References: <20171018135821.3248-1-deathsimple@vodafone.de> MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Christian König Most BIOS don't enable this because of compatibility reasons. Manually enable a 64bit BAR of 64GB size so that we have enough room for PCI devices. v2: style cleanups, increase size, add resource name, set correct flags, print message that windows was added v3: add defines for all the magic numbers, style cleanups v4: add some comment that the BIOS should actually allow this using _PRS and _SRS. v5: only enable this if CONFIG_PHYS_ADDR_T_64BIT is set Signed-off-by: Christian König Reviewed-by: Andy Shevchenko Signed-off-by: Christian König Signed-off-by: Christian König --- arch/x86/pci/fixup.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c index 11e407489db0..7b6bd76713c5 100644 --- a/arch/x86/pci/fixup.c +++ b/arch/x86/pci/fixup.c @@ -618,3 +618,83 @@ static void quirk_apple_mbp_poweroff(struct pci_dev *pdev) dev_info(dev, "can't work around MacBook Pro poweroff issue\n"); } DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x8c10, quirk_apple_mbp_poweroff); + +#ifdef CONFIG_PHYS_ADDR_T_64BIT + +#define AMD_141b_MMIO_BASE(x) (0x80 + (x) * 0x8) +#define AMD_141b_MMIO_BASE_RE_MASK BIT(0) +#define AMD_141b_MMIO_BASE_WE_MASK BIT(1) +#define AMD_141b_MMIO_BASE_MMIOBASE_MASK GENMASK(31,8) + +#define AMD_141b_MMIO_LIMIT(x) (0x84 + (x) * 0x8) +#define AMD_141b_MMIO_LIMIT_MMIOLIMIT_MASK GENMASK(31,8) + +#define AMD_141b_MMIO_HIGH(x) (0x180 + (x) * 0x4) +#define AMD_141b_MMIO_HIGH_MMIOBASE_MASK GENMASK(7,0) +#define AMD_141b_MMIO_HIGH_MMIOLIMIT_SHIFT 16 +#define AMD_141b_MMIO_HIGH_MMIOLIMIT_MASK GENMASK(23,16) + +/* + * The PCI Firmware Spec, rev 3.2 notes that ACPI should optionally allow + * configuring host bridge windows using the _PRS and _SRS methods. + * + * But this is rarely implemented, so we manually enable a large 64bit BAR for + * PCIe device on AMD Family 15h (Models 30h-3fh) Processors here. + */ +static void pci_amd_enable_64bit_bar(struct pci_dev *dev) +{ + struct resource *res, *conflict; + u32 base, limit, high; + unsigned i; + + for (i = 0; i < 8; ++i) { + pci_read_config_dword(dev, AMD_141b_MMIO_BASE(i), &base); + pci_read_config_dword(dev, AMD_141b_MMIO_HIGH(i), &high); + + /* Is this slot free? */ + if (!(base & (AMD_141b_MMIO_BASE_RE_MASK | + AMD_141b_MMIO_BASE_WE_MASK))) + break; + + base >>= 8; + base |= high << 24; + + /* Abort if a slot already configures a 64bit BAR. */ + if (base > 0x10000) + return; + } + if (i == 8) + return; + + res = kzalloc(sizeof(*res), GFP_KERNEL); + if (!res) + return; + + res->name = "PCI Bus 0000:00"; + res->flags = IORESOURCE_PREFETCH | IORESOURCE_MEM | + IORESOURCE_MEM_64 | IORESOURCE_WINDOW; + res->start = 0x100000000ull; + res->end = 0xfd00000000ull - 1; + + /* Just grab the free area behind system memory for this */ + while ((conflict = request_resource_conflict(&iomem_resource, res))) + res->start = conflict->end + 1; + + dev_info(&dev->dev, "adding root bus resource %pR\n", res); + + base = ((res->start >> 8) & AMD_141b_MMIO_BASE_MMIOBASE_MASK) | + AMD_141b_MMIO_BASE_RE_MASK | AMD_141b_MMIO_BASE_WE_MASK; + limit = ((res->end + 1) >> 8) & AMD_141b_MMIO_LIMIT_MMIOLIMIT_MASK; + high = ((res->start >> 40) & AMD_141b_MMIO_HIGH_MMIOBASE_MASK) | + ((((res->end + 1) >> 40) << AMD_141b_MMIO_HIGH_MMIOLIMIT_SHIFT) + & AMD_141b_MMIO_HIGH_MMIOLIMIT_MASK); + + pci_write_config_dword(dev, AMD_141b_MMIO_HIGH(i), high); + pci_write_config_dword(dev, AMD_141b_MMIO_LIMIT(i), limit); + pci_write_config_dword(dev, AMD_141b_MMIO_BASE(i), base); + + pci_bus_add_resource(dev->bus, res, 0); +} +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x141b, pci_amd_enable_64bit_bar); + +#endif From patchwork Wed Oct 18 13:58:21 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 827605 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="c3d3eoD9"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3yHDFR6gffz9t4X for ; Thu, 19 Oct 2017 00:59:23 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754104AbdJRN6t (ORCPT ); Wed, 18 Oct 2017 09:58:49 -0400 Received: from mail-wm0-f68.google.com ([74.125.82.68]:46940 "EHLO mail-wm0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753603AbdJRN6c (ORCPT ); Wed, 18 Oct 2017 09:58:32 -0400 Received: by mail-wm0-f68.google.com with SMTP id m72so10281353wmc.1; Wed, 18 Oct 2017 06:58:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=eiz1G294DqAqysO29nTuHfuuEK4bsFbeHSzoM45Baso=; b=c3d3eoD9ByW4dav3ZBnClxNCJtqxwZPAdOChKguXamSSyvQb6AVxxoqm4c1iRLb69n BBBzOJ9fiBpNpcaNiMO3DrQ92tOpQ/Jd55Wlrp/IOCQfZpqnqi0H0gw17rdUzEApKs3u T/daa5LDNkJUtoaejWAt4+2b+ebzoQC/zpID+RUguuy0LR9D6gLmKX2qu/CMZ1xZG7Wk h42yMipm8qO6Do6uMkYX75PE369k6duVNqqdw+Fx0rp8OOAmXEu2cyRGjBXiT9gl6AQB POA4PxYHcthWtv8s5FZKiJJf228MmSyVwM2istc7cCPazp0I8zy+T+zLgWrK2FkaZnWf qR1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eiz1G294DqAqysO29nTuHfuuEK4bsFbeHSzoM45Baso=; b=IT6fuIrdawOqWNy4rX67TBXru1cc3qVdB5qO+FRZR0gOrogIsUswU5losuBN8CMZKt Jh6QvhgfIMufritIR9LAXaYbD/aH5p2bKyOVt4Q4aE0VcZ19qxkFG3Olau9+Q/PgrFqs YAhNDjPWZefmF5ei88LEqzReIodQspuDzwjnlNb8/a3u+gsv5MtcRIl8OINzqyshwDcy I9Jr5SoeJMeX6VHXfJe/basgC0BBAdz3qkIGYc71jnGVFSbXGcmRYbgmrJu1hlVLhcQ6 bbYhcmimnyZqOJifRjal9DHDTpRkOXbVAVjLyo0tk4JRQgBOplDXO5jj7kVTwia/aaWP l50A== X-Gm-Message-State: AMCzsaWwwprb2qlYNmNxTGe2KQrCemhOPu+c9CEA8eU/FFNmV2qJeEO2 rNSx011tfgWUfUC0yyUWfXppFw== X-Google-Smtp-Source: ABhQp+TYtMxXP3rCyCAMM2Pj9MgOv2ej45RL40MOvGWhACurTpEEGlBKNabqj7adE0tFiqNFrfYBzA== X-Received: by 10.28.126.208 with SMTP id z199mr6223406wmc.91.1508335110482; Wed, 18 Oct 2017 06:58:30 -0700 (PDT) Received: from localhost.localdomain ([2a02:908:1251:7981:4537:45bc:69b6:7f1e]) by smtp.gmail.com with ESMTPSA id o24sm15780699wmi.39.2017.10.18.06.58.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Oct 2017 06:58:29 -0700 (PDT) From: "=?UTF-8?q?Christian=20K=C3=B6nig?=" X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: helgaas@kernel.org, linux-pci@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, amd-gfx@lists.freedesktop.org Subject: [PATCH v9 5/5] drm/amdgpu: resize VRAM BAR for CPU access v5 Date: Wed, 18 Oct 2017 15:58:21 +0200 Message-Id: <20171018135821.3248-6-deathsimple@vodafone.de> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171018135821.3248-1-deathsimple@vodafone.de> References: <20171018135821.3248-1-deathsimple@vodafone.de> MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Christian König Try to resize BAR0 to let CPU access all of VRAM. v2: rebased, style cleanups, disable mem decode before resize, handle gmc_v9 as well, round size up to power of two. v3: handle gmc_v6 as well, release and reassign all BARs in the driver. v4: rename new function to amdgpu_device_resize_fb_bar, reenable mem decoding only if all resources are assigned. v5: reorder resource release, return -ENODEV instead of BUG_ON(). Signed-off-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 48 ++++++++++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c | 12 ++++++-- drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 13 ++++++-- drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 13 ++++++-- drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 14 ++++++--- 6 files changed, 88 insertions(+), 13 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 7ecfc5303f4f..ac4e6f6fb6d8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1850,6 +1850,7 @@ void amdgpu_ttm_placement_from_domain(struct amdgpu_bo *abo, u32 domain); bool amdgpu_ttm_bo_is_amdgpu_bo(struct ttm_buffer_object *bo); void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base); void amdgpu_gart_location(struct amdgpu_device *adev, struct amdgpu_mc *mc); +int amdgpu_device_resize_fb_bar(struct amdgpu_device *adev); void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size); int amdgpu_ttm_init(struct amdgpu_device *adev); void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 57addfe9e89b..8f2be5a36625 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -414,6 +414,9 @@ static int amdgpu_doorbell_init(struct amdgpu_device *adev) return 0; } + if (pci_resource_flags(adev->pdev, 2) & IORESOURCE_UNSET) + return -EINVAL; + /* doorbell bar mapping */ adev->doorbell.base = pci_resource_start(adev->pdev, 2); adev->doorbell.size = pci_resource_len(adev->pdev, 2); @@ -732,6 +735,51 @@ int amdgpu_fw_reserve_vram_init(struct amdgpu_device *adev) return r; } +/** + * amdgpu_device_resize_fb_bar - try to resize FB BAR + * + * @adev: amdgpu_device pointer + * + * Try to resize FB BAR to make all VRAM CPU accessible. We try very hard not + * to fail, but if any of the BARs is not accessible after the size we abort + * driver loading by returning -ENODEV. + */ +int amdgpu_device_resize_fb_bar(struct amdgpu_device *adev) +{ + u64 space_needed = roundup_pow_of_two(adev->mc.real_vram_size); + u32 rbar_size = order_base_2(((space_needed >> 20) | 1)) - 1; + u16 cmd; + int r; + + /* Disable memory decoding while we change the BAR addresses and size */ + pci_read_config_word(adev->pdev, PCI_COMMAND, &cmd); + pci_write_config_word(adev->pdev, PCI_COMMAND, + cmd & ~PCI_COMMAND_MEMORY); + + /* Free the VRAM and doorbell BAR, we most likely need to move both. */ + amdgpu_doorbell_fini(adev); + if (adev->asic_type >= CHIP_BONAIRE) + pci_release_resource(adev->pdev, 2); + + pci_release_resource(adev->pdev, 0); + + r = pci_resize_resource(adev->pdev, 0, rbar_size); + if (r == -ENOSPC) + DRM_INFO("Not enough PCI address space for a large BAR."); + else if (r && r != -ENOTSUPP) + DRM_ERROR("Problem resizing BAR0 (%d).", r); + + pci_assign_unassigned_bus_resources(adev->pdev->bus); + + /* When the doorbell or fb BAR isn't available we have no chance of + * using the device. + */ + r = amdgpu_doorbell_init(adev); + if (r || (pci_resource_flags(adev->pdev, 0) & IORESOURCE_UNSET)) + return -ENODEV; + + pci_write_config_word(adev->pdev, PCI_COMMAND, cmd); +} /* * GPU helpers function. diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c index f4603a7c8ef3..d2a43db22cff 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c @@ -283,6 +283,7 @@ static int gmc_v6_0_mc_init(struct amdgpu_device *adev) u32 tmp; int chansize, numchan; + int r; tmp = RREG32(mmMC_ARB_RAMCFG); if (tmp & (1 << 11)) { @@ -324,12 +325,17 @@ static int gmc_v6_0_mc_init(struct amdgpu_device *adev) break; } adev->mc.vram_width = numchan * chansize; - /* Could aper size report 0 ? */ - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); /* size in MB on si */ adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; + + if (!(adev->flags & AMD_IS_APU)) { + r = amdgpu_device_resize_fb_bar(adev); + if (r) + return r; + } + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); adev->mc.visible_vram_size = adev->mc.aper_size; /* set the gart size */ diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c index b0528ca9207b..583d87792820 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c @@ -322,6 +322,8 @@ static void gmc_v7_0_mc_program(struct amdgpu_device *adev) */ static int gmc_v7_0_mc_init(struct amdgpu_device *adev) { + int r; + adev->mc.vram_width = amdgpu_atombios_get_vram_width(adev); if (!adev->mc.vram_width) { u32 tmp; @@ -367,13 +369,18 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev) } adev->mc.vram_width = numchan * chansize; } - /* Could aper size report 0 ? */ - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); /* size in MB on si */ adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; + if (!(adev->flags & AMD_IS_APU)) { + r = amdgpu_device_resize_fb_bar(adev); + if (r) + return r; + } + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); + #ifdef CONFIG_X86_64 if (adev->flags & AMD_IS_APU) { adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c index f368cfe2f585..9ca5fea93ebc 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c @@ -498,6 +498,8 @@ static void gmc_v8_0_mc_program(struct amdgpu_device *adev) */ static int gmc_v8_0_mc_init(struct amdgpu_device *adev) { + int r; + adev->mc.vram_width = amdgpu_atombios_get_vram_width(adev); if (!adev->mc.vram_width) { u32 tmp; @@ -543,13 +545,18 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev) } adev->mc.vram_width = numchan * chansize; } - /* Could aper size report 0 ? */ - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); /* size in MB on si */ adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; + if (!(adev->flags & AMD_IS_APU)) { + r = amdgpu_device_resize_fb_bar(adev); + if (r) + return r; + } + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); + #ifdef CONFIG_X86_64 if (adev->flags & AMD_IS_APU) { adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c index 621699331e09..0de4dc068516 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c @@ -440,6 +440,7 @@ static int gmc_v9_0_mc_init(struct amdgpu_device *adev) { u32 tmp; int chansize, numchan; + int r; adev->mc.vram_width = amdgpu_atomfirmware_get_vram_width(adev); if (!adev->mc.vram_width) { @@ -482,17 +483,22 @@ static int gmc_v9_0_mc_init(struct amdgpu_device *adev) adev->mc.vram_width = numchan * chansize; } - /* Could aper size report 0 ? */ - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); /* size in MB on si */ adev->mc.mc_vram_size = ((adev->flags & AMD_IS_APU) ? nbio_v7_0_get_memsize(adev) : nbio_v6_1_get_memsize(adev)) * 1024ULL * 1024ULL; adev->mc.real_vram_size = adev->mc.mc_vram_size; - adev->mc.visible_vram_size = adev->mc.aper_size; + + if (!(adev->flags & AMD_IS_APU)) { + r = amdgpu_device_resize_fb_bar(adev); + if (r) + return r; + } + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); /* In case the PCI BAR is larger than the actual amount of vram */ + adev->mc.visible_vram_size = adev->mc.aper_size; if (adev->mc.visible_vram_size > adev->mc.real_vram_size) adev->mc.visible_vram_size = adev->mc.real_vram_size;