From patchwork Fri Feb 18 04:50:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "David E. Box" X-Patchwork-Id: 1594560 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=EEiM7nKW; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4K0K5t5D1Wz9sFn for ; Fri, 18 Feb 2022 15:51:14 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229983AbiBREv0 (ORCPT ); Thu, 17 Feb 2022 23:51:26 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:44518 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229972AbiBREvZ (ORCPT ); Thu, 17 Feb 2022 23:51:25 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 587CD28994; Thu, 17 Feb 2022 20:51:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645159870; x=1676695870; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TCWaxIq+2Hq1UtypOPqzOFeua9erHmtP56hr+rzFEF4=; b=EEiM7nKWCOjxIkHKDeR7cNkHEmM/P13ZkAUtH20hry3I8WImBtg1OoIx LUP+DbatQN6eKFW0b292qNglBvUM/6/F4YXwKU31v3ukEp+b9J+k47V5c lcxIvQB8G6Zc2bey8vsgykjDAWNE38szFbTaHv5xJI28ZhQx6UysuK3s7 Y0/N5q1E9Pgzx7D80MTtL1Io7m6mWfpst28i/AByDJiFZwRyuygJW9Ih+ /zwT9jccOeMtu3zvuwKAIzJ5BEA5e/W1ZALAPOEUgwdrFlrRoqcyC+qZp I4gJN68YzwiRCGMPZ/BXOqM/J1DOZdO0GnCEMrCF0dBGvbdIfNuUBe0rA g==; X-IronPort-AV: E=McAfee;i="6200,9189,10261"; a="275646328" X-IronPort-AV: E=Sophos;i="5.88,377,1635231600"; d="scan'208";a="275646328" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Feb 2022 20:51:10 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,377,1635231600"; d="scan'208";a="777890277" Received: from linux.intel.com ([10.54.29.200]) by fmsmga005.fm.intel.com with ESMTP; 17 Feb 2022 20:51:09 -0800 Received: from debox1-desk4.lan (unknown [10.251.23.8]) by linux.intel.com (Postfix) with ESMTP id 6C581580C70; Thu, 17 Feb 2022 20:51:09 -0800 (PST) From: "David E. Box" To: nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev, lorenzo.pieralisi@arm.com, hch@infradead.org, kw@linux.com, robh@kernel.org, bhelgaas@google.com, david.e.box@linux.intel.com, michael.a.bottini@linux.intel.com, rafael@kernel.org, me@adhityamohan.in Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V5 1/3] PCI/ASPM: Add pci_enable_default_link_state() Date: Thu, 17 Feb 2022 20:50:54 -0800 Message-Id: <20220218045056.333799-2-david.e.box@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220218045056.333799-1-david.e.box@linux.intel.com> References: <20220218045056.333799-1-david.e.box@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Michael Bottini Add pci_enable_default_link_state() to allow devices to change the default BIOS configured states. Clears the BIOS default settings then sets the new states and reconfigures the link under the semaphore. Also add PCIE_LINK_STATE_ALL macro for convenience for callers that want to enable all link states. Signed-off-by: Michael Bottini Signed-off-by: David E. Box --- V5 - Rename to pci_enable_default_link_state and model after pci_disable_link_state() as suggested by Bjorn. - Add helper PCIE_LINK_STATE_ALL which sets bits for all links states and clock pm. - Clarify commit language to indicate the function changes the default link states (not ASPM policy). V4 - Refactor vmd_enable_apsm() to exit early, making the lines shorter and more readable. Suggested by Christoph. V3 - No changes V2 - Use return status to print pci_info message if ASPM cannot be enabled. - Add missing static declaration, caught by lkp@intel.com drivers/pci/pcie/aspm.c | 54 +++++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 7 ++++++ 2 files changed, 61 insertions(+) diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c index a96b7424c9bc..b2752851b1ba 100644 --- a/drivers/pci/pcie/aspm.c +++ b/drivers/pci/pcie/aspm.c @@ -1132,6 +1132,60 @@ int pci_disable_link_state(struct pci_dev *pdev, int state) } EXPORT_SYMBOL(pci_disable_link_state); +/** + * pci_enable_default_link_state - Clears and sets the default device link state + * so that the link may be allowed to enter the specified states. Note that + * if the BIOS didn't grant ASPM control to the OS, this does nothing because + * we can't touch the LNKCTL register. Also note that this does not enable + * states disabled by pci_disable_link_state(). Returns 0 or a negative errno. + * + * @pdev: PCI device + * @state: Mask of ASPM link states to enable + */ +int pci_enable_default_link_state(struct pci_dev *pdev, int state) +{ + struct pcie_link_state *link = pcie_aspm_get_link(pdev); + + if (!link) + return -EINVAL; + /* + * A driver requested that ASPM be enabled on this device, but + * if we don't have permission to manage ASPM (e.g., on ACPI + * systems we have to observe the FADT ACPI_FADT_NO_ASPM bit and + * the _OSC method), we can't honor that request. + */ + if (aspm_disabled) { + pci_warn(pdev, "can't override BIOS ASPM; OS doesn't have ASPM control\n"); + return -EPERM; + } + + down_read(&pci_bus_sem); + mutex_lock(&aspm_lock); + link->aspm_default = 0; + if (state & PCIE_LINK_STATE_L0S) + link->aspm_default |= ASPM_STATE_L0S; + if (state & PCIE_LINK_STATE_L1) + /* L1 PM substates require L1 */ + link->aspm_default |= ASPM_STATE_L1 | ASPM_STATE_L1SS; + if (state & PCIE_LINK_STATE_L1_1) + link->aspm_default |= ASPM_STATE_L1_1; + if (state & PCIE_LINK_STATE_L1_2) + link->aspm_default |= ASPM_STATE_L1_2; + if (state & PCIE_LINK_STATE_L1_1_PCIPM) + link->aspm_default |= ASPM_STATE_L1_1_PCIPM; + if (state & PCIE_LINK_STATE_L1_2_PCIPM) + link->aspm_default |= ASPM_STATE_L1_2_PCIPM; + pcie_config_aspm_link(link, policy_to_aspm_state(link)); + + link->clkpm_default = (state & PCIE_LINK_STATE_CLKPM) ? 1 : 0; + pcie_set_clkpm(link, policy_to_clkpm_state(link)); + mutex_unlock(&aspm_lock); + up_read(&pci_bus_sem); + + return 0; +} +EXPORT_SYMBOL(pci_enable_default_link_state); + static int pcie_aspm_set_policy(const char *val, const struct kernel_param *kp) { diff --git a/include/linux/pci.h b/include/linux/pci.h index 8253a5413d7c..fd710afe0209 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1641,10 +1641,15 @@ extern bool pcie_ports_native; #define PCIE_LINK_STATE_L1_2 BIT(4) #define PCIE_LINK_STATE_L1_1_PCIPM BIT(5) #define PCIE_LINK_STATE_L1_2_PCIPM BIT(6) +#define PCIE_LINK_STATE_ALL (PCIE_LINK_STATE_L0S | PCIE_LINK_STATE_L1 |\ + PCIE_LINK_STATE_CLKPM | PCIE_LINK_STATE_L1_1 |\ + PCIE_LINK_STATE_L1_2 | PCIE_LINK_STATE_L1_1_PCIPM |\ + PCIE_LINK_STATE_L1_2_PCIPM) #ifdef CONFIG_PCIEASPM int pci_disable_link_state(struct pci_dev *pdev, int state); int pci_disable_link_state_locked(struct pci_dev *pdev, int state); +int pci_enable_default_link_state(struct pci_dev *pdev, int state); void pcie_no_aspm(void); bool pcie_aspm_support_enabled(void); bool pcie_aspm_enabled(struct pci_dev *pdev); @@ -1653,6 +1658,8 @@ static inline int pci_disable_link_state(struct pci_dev *pdev, int state) { return 0; } static inline int pci_disable_link_state_locked(struct pci_dev *pdev, int state) { return 0; } +static inline int pci_enable_default_link_state(struct pci_dev *pdev, int state) +{ return 0; } static inline void pcie_no_aspm(void) { } static inline bool pcie_aspm_support_enabled(void) { return false; } static inline bool pcie_aspm_enabled(struct pci_dev *pdev) { return false; } From patchwork Fri Feb 18 04:50:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "David E. Box" X-Patchwork-Id: 1594561 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=aKQqXXFp; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4K0K5v1GxCz9sFk for ; Fri, 18 Feb 2022 15:51:15 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229992AbiBREv1 (ORCPT ); Thu, 17 Feb 2022 23:51:27 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:44562 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229977AbiBREv0 (ORCPT ); Thu, 17 Feb 2022 23:51:26 -0500 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA9F82BB11; Thu, 17 Feb 2022 20:51:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645159870; x=1676695870; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=u4q792RCz+H3dOJaH3q90T0pGToJjOH3Rm2aZCHpaP8=; b=aKQqXXFp3enKmpvKrzMYp0KFkNhhDA6GxzuXjBmbFHl3OZspOzjQrcv0 TWITJGDcvwHY8i0JnGFTUQkRke5fT7z2BMcp/1lmDc5+seeFHinu6Flwl ifl4Myn7/HONC9ddMZx+BeAt5vmGPOALteE/XCIVBoLPhz59tafdHNgfv G9J8zf6FL70nLgGXBi2FQETXSzl1uARNflBpIZnr7kpBdO/s/mru5Ie5+ 1TBrpLN+VgGGCUGpVY4ed3GNZ1suozsr9pBrlCEbFjL+pPaCAp4g+g6eL zV4ZZcx6Uv2gY5msvkH33ioZk0oeAKVHJAe4zYxRlmARqVCa6B8k8UvxM A==; X-IronPort-AV: E=McAfee;i="6200,9189,10261"; a="251250339" X-IronPort-AV: E=Sophos;i="5.88,377,1635231600"; d="scan'208";a="251250339" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Feb 2022 20:51:10 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,377,1635231600"; d="scan'208";a="546102725" Received: from linux.intel.com ([10.54.29.200]) by orsmga008.jf.intel.com with ESMTP; 17 Feb 2022 20:51:10 -0800 Received: from debox1-desk4.lan (unknown [10.251.23.8]) by linux.intel.com (Postfix) with ESMTP id C5BD9580CB8; Thu, 17 Feb 2022 20:51:09 -0800 (PST) From: "David E. Box" To: nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev, lorenzo.pieralisi@arm.com, hch@infradead.org, kw@linux.com, robh@kernel.org, bhelgaas@google.com, david.e.box@linux.intel.com, michael.a.bottini@linux.intel.com, rafael@kernel.org, me@adhityamohan.in Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V5 2/3] PCI: vmd: Add vmd_device_data Date: Thu, 17 Feb 2022 20:50:55 -0800 Message-Id: <20220218045056.333799-3-david.e.box@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220218045056.333799-1-david.e.box@linux.intel.com> References: <20220218045056.333799-1-david.e.box@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Add vmd_device_data to allow adding additional info for driver data. Also refactor the PCI ID list to use PCI_VDEVICE and simplify assignments for devices that use the same driver_data. Signed-off-by: David E. Box --- V5 - New patch drivers/pci/controller/vmd.c | 58 ++++++++++++++++++++---------------- 1 file changed, 32 insertions(+), 26 deletions(-) diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c index cc166c683638..a582c351b461 100644 --- a/drivers/pci/controller/vmd.c +++ b/drivers/pci/controller/vmd.c @@ -69,6 +69,10 @@ enum vmd_features { VMD_FEAT_CAN_BYPASS_MSI_REMAP = (1 << 4), }; +struct vmd_device_data { + enum vmd_features features; +}; + static DEFINE_IDA(vmd_instance_ida); /* @@ -710,11 +714,12 @@ static void vmd_copy_host_bridge_flags(struct pci_host_bridge *root_bridge, vmd_bridge->native_dpc = root_bridge->native_dpc; } -static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features) +static int vmd_enable_domain(struct vmd_dev *vmd, struct vmd_device_data *info) { struct pci_sysdata *sd = &vmd->sysdata; struct resource *res; u32 upper_bits; + unsigned long features = info->features; unsigned long flags; LIST_HEAD(resources); resource_size_t offset[2] = {0}; @@ -881,7 +886,8 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features) static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id) { - unsigned long features = (unsigned long) id->driver_data; + struct vmd_device_data *info = (struct vmd_device_data *)id->driver_data; + unsigned long features = info->features; struct vmd_dev *vmd; int err; @@ -925,7 +931,7 @@ static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id) spin_lock_init(&vmd->cfg_lock); pci_set_drvdata(dev, vmd); - err = vmd_enable_domain(vmd, features); + err = vmd_enable_domain(vmd, info); if (err) goto out_release_instance; @@ -993,30 +999,30 @@ static int vmd_resume(struct device *dev) #endif static SIMPLE_DEV_PM_OPS(vmd_dev_pm_ops, vmd_suspend, vmd_resume); +static const struct vmd_device_data vmd_201d_data = { + .features = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP, +}; + +static const struct vmd_device_data vmd_28c0_data = { + .features = VMD_FEAT_HAS_MEMBAR_SHADOW | + VMD_FEAT_HAS_BUS_RESTRICTIONS | + VMD_FEAT_CAN_BYPASS_MSI_REMAP, +}; + +static const struct vmd_device_data vmd_467f_data = { + .features = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | + VMD_FEAT_HAS_BUS_RESTRICTIONS | + VMD_FEAT_OFFSET_FIRST_VECTOR, +}; + static const struct pci_device_id vmd_ids[] = { - {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_201D), - .driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP,}, - {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_28C0), - .driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW | - VMD_FEAT_HAS_BUS_RESTRICTIONS | - VMD_FEAT_CAN_BYPASS_MSI_REMAP,}, - {PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x467f), - .driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | - VMD_FEAT_HAS_BUS_RESTRICTIONS | - VMD_FEAT_OFFSET_FIRST_VECTOR,}, - {PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x4c3d), - .driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | - VMD_FEAT_HAS_BUS_RESTRICTIONS | - VMD_FEAT_OFFSET_FIRST_VECTOR,}, - {PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xa77f), - .driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | - VMD_FEAT_HAS_BUS_RESTRICTIONS | - VMD_FEAT_OFFSET_FIRST_VECTOR,}, - {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_9A0B), - .driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | - VMD_FEAT_HAS_BUS_RESTRICTIONS | - VMD_FEAT_OFFSET_FIRST_VECTOR,}, - {0,} + { PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_201D), (kernel_ulong_t)&vmd_201d_data }, + { PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_28C0), (kernel_ulong_t)&vmd_28c0_data }, + { PCI_VDEVICE(INTEL, 0x467f), (kernel_ulong_t)&vmd_467f_data }, + { PCI_VDEVICE(INTEL, 0x4c3d), (kernel_ulong_t)&vmd_467f_data }, + { PCI_VDEVICE(INTEL, 0xa77f), (kernel_ulong_t)&vmd_467f_data }, + { PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_9A0B), (kernel_ulong_t)&vmd_467f_data }, + { } }; MODULE_DEVICE_TABLE(pci, vmd_ids); From patchwork Fri Feb 18 04:50:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "David E. Box" X-Patchwork-Id: 1594562 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=A2kBhM/q; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4K0K5x2W3vz9sFk for ; Fri, 18 Feb 2022 15:51:17 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230007AbiBREv3 (ORCPT ); Thu, 17 Feb 2022 23:51:29 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:44590 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229986AbiBREv0 (ORCPT ); Thu, 17 Feb 2022 23:51:26 -0500 Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F2E6238D98; Thu, 17 Feb 2022 20:51:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645159870; x=1676695870; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7NweWA9u8MLHzRnDn77tVJVW+/OijRX/EBVVwLg8Lh0=; b=A2kBhM/qK8sI3luzIDXOg6H/1XIfwmw0C9J2oPsD4OP8IuEqrPHf3Wqn xYD2/KkxiBxQwzRBlNSnVSHzmGWKfnA4W4kjmfR+yLbeXgEjoxqgwwVTx h6MCTZLg1rEiqcz7fpuQYtGHWfdA9vakeVAnxdtbChQOCIzE2QbqJOxbd nUHY492B9BTym3RfOEoIGNESas90okIZR+IVzd54+4nartZC6yD0X1dvo vlGA18ysviB/lvkJ0VUq8qdPtLAFcQBgMvMKaBd7r2/EPIBUseTIeTsHO 2B86vIE4FIkfDh+uz9+VG9TVmmy1M35/1BymAzTdnOSJBt60mdpRsMHSJ Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10261"; a="311793166" X-IronPort-AV: E=Sophos;i="5.88,377,1635231600"; d="scan'208";a="311793166" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Feb 2022 20:51:10 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,377,1635231600"; d="scan'208";a="572162984" Received: from linux.intel.com ([10.54.29.200]) by orsmga001.jf.intel.com with ESMTP; 17 Feb 2022 20:51:10 -0800 Received: from debox1-desk4.lan (unknown [10.251.23.8]) by linux.intel.com (Postfix) with ESMTP id 385DD580C70; Thu, 17 Feb 2022 20:51:10 -0800 (PST) From: "David E. Box" To: nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev, lorenzo.pieralisi@arm.com, hch@infradead.org, kw@linux.com, robh@kernel.org, bhelgaas@google.com, david.e.box@linux.intel.com, michael.a.bottini@linux.intel.com, rafael@kernel.org, me@adhityamohan.in Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V5 3/3] PCI: vmd: Configure PCIe ASPM and LTR Date: Thu, 17 Feb 2022 20:50:56 -0800 Message-Id: <20220218045056.333799-4-david.e.box@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220218045056.333799-1-david.e.box@linux.intel.com> References: <20220218045056.333799-1-david.e.box@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Currently, PCIe ports reserved for VMD use are not visible to BIOS and therefore not configured to enable PCIE ASPM. Additionally, PCIE LTR values may be left unset since BIOS will set a default maximum LTR value on endpoints to ensure that misconfigured devices don't block SoC power management. Lack of this programming results in high power consumption on laptops as reported in bugzilla [1]. For currently affected products, use pci_enable_default_link_state to set the allowed link states for devices on the root ports. Also set the LTR value to the maximum value needed for the SoC. Per the VMD hardware team future products using VMD will enable BIOS configuration of these capabilities. This solution is a workaround for current products that mainly targets laptops. Support is not provided if a switch is used nor for hotplug. [1] https://bugzilla.kernel.org/show_bug.cgi?id=213717 Signed-off-by: Michael Bottini Signed-off-by: David E. Box --- V5 - Provide the LTR value as driver data. - Use DWORD for the config space write to avoid PCI WORD access bug. - Set ASPM links firsts, enabling all link states, before setting a default LTR if the capability is present - Add kernel message that VMD is setting the device LTR. V4 - Refactor vmd_enable_apsm() to exit early, making the lines shorter and more readable. Suggested by Christoph. V3 - No changes V2 - Use return status to print pci_info message if ASPM cannot be enabled. - Add missing static declaration, caught by lkp@intel.com drivers/pci/controller/vmd.c | 48 +++++++++++++++++++++++++++++++++++- 1 file changed, 47 insertions(+), 1 deletion(-) diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c index a582c351b461..eac379c80cd7 100644 --- a/drivers/pci/controller/vmd.c +++ b/drivers/pci/controller/vmd.c @@ -67,10 +67,19 @@ enum vmd_features { * interrupt handling. */ VMD_FEAT_CAN_BYPASS_MSI_REMAP = (1 << 4), + + /* + * Enable ASPM on the PCIE root ports and set the default LTR of the + * storage devices on platforms where these values are not configured by + * BIOS. This is needed for laptops, which require these settings for + * proper power management of the SoC. + */ + VMD_FEAT_BIOS_PM_QUIRK = (1 << 5), }; struct vmd_device_data { enum vmd_features features; + u16 ltr; }; static DEFINE_IDA(vmd_instance_ida); @@ -714,6 +723,38 @@ static void vmd_copy_host_bridge_flags(struct pci_host_bridge *root_bridge, vmd_bridge->native_dpc = root_bridge->native_dpc; } +/* + * Enable ASPM and LTR settings on devices that aren't configured by BIOS. + */ +static int vmd_pm_enable_quirk(struct pci_dev *pdev, void *userdata) +{ + struct vmd_device_data *info = userdata; + u32 ltr_reg; + int pos; + + if (!(info->features & VMD_FEAT_BIOS_PM_QUIRK)) + return 0; + + pci_enable_default_link_state(pdev, PCIE_LINK_STATE_ALL); + + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_LTR); + if (!pos) + return 0; + + /* + * If the LTR capability is present, set the default values to the + * maximum required by the platform to allow the deepest power + * management savings. Write this as a single DWORD where the lower word + * is the max snoop latency and the upper word is the max non-snoop + * latency. + */ + pci_info(pdev, "VMD: Setting a default LTR\n"); + ltr_reg = (info->ltr << 16) | info->ltr; + pci_write_config_dword(pdev, pos + PCI_LTR_MAX_SNOOP_LAT, ltr_reg); + + return 0; +} + static int vmd_enable_domain(struct vmd_dev *vmd, struct vmd_device_data *info) { struct pci_sysdata *sd = &vmd->sysdata; @@ -867,6 +908,8 @@ static int vmd_enable_domain(struct vmd_dev *vmd, struct vmd_device_data *info) pci_reset_bus(child->self); pci_assign_unassigned_bus_resources(vmd->bus); + pci_walk_bus(vmd->bus, vmd_pm_enable_quirk, info); + /* * VMD root buses are virtual and don't return true on pci_is_pcie() * and will fail pcie_bus_configure_settings() early. It can instead be @@ -1012,7 +1055,10 @@ static const struct vmd_device_data vmd_28c0_data = { static const struct vmd_device_data vmd_467f_data = { .features = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | VMD_FEAT_HAS_BUS_RESTRICTIONS | - VMD_FEAT_OFFSET_FIRST_VECTOR, + VMD_FEAT_OFFSET_FIRST_VECTOR | + VMD_FEAT_BIOS_PM_QUIRK, + /* 3145728 ns (LatencyScale of 1048576 ns with a LatencyValue of 3) */ + .ltr = 0x1003, }; static const struct pci_device_id vmd_ids[] = {