diff mbox series

[v7,4/5] PCI/ASPM: Add sysfs attributes for controlling ASPM link states

Message ID b1c83f8a-9bf6-eac5-82d0-cf5b90128fbf@gmail.com
State Accepted
Delegated to: Bjorn Helgaas
Headers show
Series PCI/ASPM: Add sysfs attributes for controlling ASPM | expand

Commit Message

Heiner Kallweit Oct. 5, 2019, 12:07 p.m. UTC
Background of this extension is a problem with the r8169 network driver.
Several combinations of board chipsets and network chip versions have
problems if ASPM is enabled, therefore we have to disable ASPM per default.
However especially on notebooks ASPM can provide significant power-saving,
therefore we want to give users the option to enable ASPM. With the new
sysfs attributes users can control which ASPM link-states are
enabled/disabled.

Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
---
v2:
- use a dedicated sysfs attribute per link state
- allow separate control of ASPM and PCI PM L1 sub-states
v3:
- statically allocate the attribute group
- replace snprintf with printf
- base on top of "PCI: Make pcie_downstream_port() available outside of access.c"
v4:
- add call to sysfs_update_group because is_visible callback returns false
  always at file creation time
- simplify code a little
v5:
- rebased to latest pci/next
v6:
- fix style of added documentation and extend it
- don't use term "parent" and rename function to pcie_aspm_get_link
- rename pcie_check_valid_aspm_endpoint to pcie_is_aspm_dev
- enable the sysfs files also selected other port types
- avoid usage of !!
v7:
- change group name from aspm to link_pm
- control visibility of attributes individually
---
 Documentation/ABI/testing/sysfs-bus-pci |  14 ++
 drivers/pci/pci-sysfs.c                 |   3 +
 drivers/pci/pci.h                       |   4 +
 drivers/pci/pcie/aspm.c                 | 174 ++++++++++++++++++++++++
 4 files changed, 195 insertions(+)

Comments

Bjorn Helgaas Oct. 8, 2019, 1:53 a.m. UTC | #1
On Sat, Oct 05, 2019 at 02:07:56PM +0200, Heiner Kallweit wrote:
> Background of this extension is a problem with the r8169 network driver.
> Several combinations of board chipsets and network chip versions have
> problems if ASPM is enabled, therefore we have to disable ASPM per default.
> However especially on notebooks ASPM can provide significant power-saving,
> therefore we want to give users the option to enable ASPM. With the new
> sysfs attributes users can control which ASPM link-states are
> enabled/disabled.
> 
> Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>

> +static ssize_t aspm_attr_show_common(struct device *dev,
> +				     struct device_attribute *attr,
> +				     char *buf, u8 state)
> +{
> +	struct pci_dev *pdev = to_pci_dev(dev);
> +	struct pcie_link_state *link;
> +	bool enabled;
> +
> +	link = pcie_aspm_get_link(pdev);
> +
> +	mutex_lock(&aspm_lock);
> +	enabled = link->aspm_enabled & state;
> +	mutex_unlock(&aspm_lock);

Not sure the mutex is needed; do you have a reason, or is it just
copied from existing code?  If the latter, we can just wait to see
what Rafael says.

> +	return sprintf(buf, "%d\n", enabled ? 1 : 0);
> +}
Bjorn Helgaas Nov. 21, 2019, 8:49 p.m. UTC | #2
[+cc Rafael, Mika, Wong, Hui, Rajat, Keith, LKML, original patch at [5]]

On Sat, Oct 05, 2019 at 02:07:56PM +0200, Heiner Kallweit wrote:

> +What:		/sys/bus/pci/devices/.../link_pm/clkpm
> +		/sys/bus/pci/devices/.../link_pm/l0s_aspm
> +		/sys/bus/pci/devices/.../link_pm/l1_aspm
> +		/sys/bus/pci/devices/.../link_pm/l1_1_aspm
> +		/sys/bus/pci/devices/.../link_pm/l1_2_aspm
> +		/sys/bus/pci/devices/.../link_pm/l1_1_pcipm
> +		/sys/bus/pci/devices/.../link_pm/l1_2_pcipm
> +Date:		October 2019
> +Contact:	Heiner Kallweit <hkallweit1@gmail.com>
> +Description:	If ASPM is supported for an endpoint, then these files
> +		can be used to disable or enable the individual
> +		power management states. Write y/1/on to enable,
> +		n/0/off to disable.

This is queued up for the v5.5 merge window, so if we want to tweak
anything (path names or otherwise), now is the time.

I think I might be inclined to change the directory from "link_pm" to
"link", e.g.,

  - /sys/bus/pci/devices/0000:00:1c.0/link_pm/clkpm
  + /sys/bus/pci/devices/0000:00:1c.0/link/clkpm

because there are other things that haven't been merged yet that could
go in link/ as well:

  * Mika's "link disable" control [1]
  * Dilip's link width/speed controls [2,3]

The max_link_speed, max_link_width, current_link_speed,
current_link_width files could also logically be in link/, although
they've already been merged at the top level.

Rajat's AER statistics change [4] is also coming.  Those stats aren't
link-related, so they wouldn't go in link/.  The current strawman is
an "aer_stats" directory, but I wonder if we should make a more
generic directory like "errors" that could be used for both AER and
DPC and potentially other error-related things.

For example, we could have these link-related things:

  /sys/.../0000:00:1c.0/link/clkpm            # RW ASPM stuff
  /sys/.../0000:00:1c.0/link/l0s_aspm
  /sys/.../0000:00:1c.0/link/...
  /sys/.../0000:00:1c.0/link/disable          # RW Mika
  /sys/.../0000:00:1c.0/link/speed            # RW Dilip's control
  /sys/.../0000:00:1c.0/link/width            # RW Dilip's control
  /sys/.../0000:00:1c.0/link/max_speed        # RO possible rework
  /sys/.../0000:00:1c.0/link/max_width        # RO possible rework

With these backwards compatibility symlinks:

  /sys/.../0000:00:1c.0/max_link_speed     -> link/max_speed
  /sys/.../0000:00:1c.0/current_link_speed -> link/speed

Rajat's current patch puts the AER stats here at the top level:

  /sys/.../0000:00:1c.0/aer_stats/fatal_bit4_DLP

But maybe we could push them down like this:

  /sys/.../0000:00:1c.0/errors/aer/stats/unc_04_dlp
  /sys/.../0000:00:1c.0/errors/aer/stats/unc_26_poison_tlb_blocked
  /sys/.../0000:00:1c.0/errors/aer/stats/cor_00_rx_err
  /sys/.../0000:00:1c.0/errors/aer/stats/cor_15_hdr_log_overflow

There are some AER-related things we don't have at all today that
could go here:

  /sys/.../0000:00:1c.0/errors/aer/ecrc_gen
  /sys/.../0000:00:1c.0/errors/aer/ecrc_check
  /sys/.../0000:00:1c.0/errors/aer/unc_err_status
  /sys/.../0000:00:1c.0/errors/aer/unc_err_mask
  /sys/.../0000:00:1c.0/errors/aer/unc_err_sev

And we might someday want DPC knobs like this:

  /sys/.../0000:00:1c.0/errors/dpc/status
  /sys/.../0000:00:1c.0/errors/dpc/error_source

Any thoughts?

Bjorn

[1] https://lore.kernel.org/r/20190529104942.74991-1-mika.westerberg@linux.intel.com
[2] https://lore.kernel.org/r/d8574605f8e70f41ce1e88ccfb56b63c8f85e4df.1571638827.git.eswara.kota@linux.intel.com
[3] https://lore.kernel.org/r/20191030221436.GA261632@google.com/
[4] https://lore.kernel.org/r/20190827222145.32642-2-rajatja@google.com
[5] https://lore.kernel.org/r/b1c83f8a-9bf6-eac5-82d0-cf5b90128fbf@gmail.com
Rajat Jain Nov. 21, 2019, 9:03 p.m. UTC | #3
Hi,

On Thu, Nov 21, 2019 at 12:49 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> [+cc Rafael, Mika, Wong, Hui, Rajat, Keith, LKML, original patch at [5]]
>
> On Sat, Oct 05, 2019 at 02:07:56PM +0200, Heiner Kallweit wrote:
>
> > +What:                /sys/bus/pci/devices/.../link_pm/clkpm
> > +             /sys/bus/pci/devices/.../link_pm/l0s_aspm
> > +             /sys/bus/pci/devices/.../link_pm/l1_aspm
> > +             /sys/bus/pci/devices/.../link_pm/l1_1_aspm
> > +             /sys/bus/pci/devices/.../link_pm/l1_2_aspm
> > +             /sys/bus/pci/devices/.../link_pm/l1_1_pcipm
> > +             /sys/bus/pci/devices/.../link_pm/l1_2_pcipm
> > +Date:                October 2019
> > +Contact:     Heiner Kallweit <hkallweit1@gmail.com>
> > +Description: If ASPM is supported for an endpoint, then these files
> > +             can be used to disable or enable the individual
> > +             power management states. Write y/1/on to enable,
> > +             n/0/off to disable.
>
> This is queued up for the v5.5 merge window, so if we want to tweak
> anything (path names or otherwise), now is the time.
>
> I think I might be inclined to change the directory from "link_pm" to
> "link", e.g.,
>
>   - /sys/bus/pci/devices/0000:00:1c.0/link_pm/clkpm
>   + /sys/bus/pci/devices/0000:00:1c.0/link/clkpm
>
> because there are other things that haven't been merged yet that could
> go in link/ as well:
>
>   * Mika's "link disable" control [1]
>   * Dilip's link width/speed controls [2,3]
>
> The max_link_speed, max_link_width, current_link_speed,
> current_link_width files could also logically be in link/, although
> they've already been merged at the top level.
>
> Rajat's AER statistics change [4] is also coming.  Those stats aren't
> link-related, so they wouldn't go in link/.  The current strawman is
> an "aer_stats" directory, but I wonder if we should make a more
> generic directory like "errors" that could be used for both AER and
> DPC and potentially other error-related things.

Sorry, I haven't been able to find time for it for some time. I doubt
if I'll be able to make it to 5.6 timeframe. Nevertheless...

>
> For example, we could have these link-related things:
>
>   /sys/.../0000:00:1c.0/link/clkpm            # RW ASPM stuff
>   /sys/.../0000:00:1c.0/link/l0s_aspm
>   /sys/.../0000:00:1c.0/link/...
>   /sys/.../0000:00:1c.0/link/disable          # RW Mika
>   /sys/.../0000:00:1c.0/link/speed            # RW Dilip's control
>   /sys/.../0000:00:1c.0/link/width            # RW Dilip's control
>   /sys/.../0000:00:1c.0/link/max_speed        # RO possible rework
>   /sys/.../0000:00:1c.0/link/max_width        # RO possible rework
>
> With these backwards compatibility symlinks:
>
>   /sys/.../0000:00:1c.0/max_link_speed     -> link/max_speed
>   /sys/.../0000:00:1c.0/current_link_speed -> link/speed
>
> Rajat's current patch puts the AER stats here at the top level:
>
>   /sys/.../0000:00:1c.0/aer_stats/fatal_bit4_DLP
>
> But maybe we could push them down like this:
>
>   /sys/.../0000:00:1c.0/errors/aer/stats/unc_04_dlp
>   /sys/.../0000:00:1c.0/errors/aer/stats/unc_26_poison_tlb_blocked
>   /sys/.../0000:00:1c.0/errors/aer/stats/cor_00_rx_err
>   /sys/.../0000:00:1c.0/errors/aer/stats/cor_15_hdr_log_overflow

How do we create sub-sub-sub directories in sysfs (errors/aer/stats)?
My understanding is that we can only create 1 subdirectory by using a
"named" attribute group. If we want more hierarchy, the "errors" and
the "aer" will need to be backed up by a kobject. Doable, but just
mentioning.

Overall the proposal looks like a step in the right direction to me.

Thanks & Best Regards,

Rajat

>
> There are some AER-related things we don't have at all today that
> could go here:
>
>   /sys/.../0000:00:1c.0/errors/aer/ecrc_gen
>   /sys/.../0000:00:1c.0/errors/aer/ecrc_check
>   /sys/.../0000:00:1c.0/errors/aer/unc_err_status
>   /sys/.../0000:00:1c.0/errors/aer/unc_err_mask
>   /sys/.../0000:00:1c.0/errors/aer/unc_err_sev
>
> And we might someday want DPC knobs like this:
>
>   /sys/.../0000:00:1c.0/errors/dpc/status
>   /sys/.../0000:00:1c.0/errors/dpc/error_source
>
> Any thoughts?
>
> Bjorn
>
> [1] https://lore.kernel.org/r/20190529104942.74991-1-mika.westerberg@linux.intel.com
> [2] https://lore.kernel.org/r/d8574605f8e70f41ce1e88ccfb56b63c8f85e4df.1571638827.git.eswara.kota@linux.intel.com
> [3] https://lore.kernel.org/r/20191030221436.GA261632@google.com/
> [4] https://lore.kernel.org/r/20190827222145.32642-2-rajatja@google.com
> [5] https://lore.kernel.org/r/b1c83f8a-9bf6-eac5-82d0-cf5b90128fbf@gmail.com
Greg Kroah-Hartman Nov. 21, 2019, 9:10 p.m. UTC | #4
On Thu, Nov 21, 2019 at 01:03:06PM -0800, Rajat Jain wrote:
> Hi,
> 
> On Thu, Nov 21, 2019 at 12:49 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
> >
> > [+cc Rafael, Mika, Wong, Hui, Rajat, Keith, LKML, original patch at [5]]
> >
> > On Sat, Oct 05, 2019 at 02:07:56PM +0200, Heiner Kallweit wrote:
> >
> > > +What:                /sys/bus/pci/devices/.../link_pm/clkpm
> > > +             /sys/bus/pci/devices/.../link_pm/l0s_aspm
> > > +             /sys/bus/pci/devices/.../link_pm/l1_aspm
> > > +             /sys/bus/pci/devices/.../link_pm/l1_1_aspm
> > > +             /sys/bus/pci/devices/.../link_pm/l1_2_aspm
> > > +             /sys/bus/pci/devices/.../link_pm/l1_1_pcipm
> > > +             /sys/bus/pci/devices/.../link_pm/l1_2_pcipm
> > > +Date:                October 2019
> > > +Contact:     Heiner Kallweit <hkallweit1@gmail.com>
> > > +Description: If ASPM is supported for an endpoint, then these files
> > > +             can be used to disable or enable the individual
> > > +             power management states. Write y/1/on to enable,
> > > +             n/0/off to disable.
> >
> > This is queued up for the v5.5 merge window, so if we want to tweak
> > anything (path names or otherwise), now is the time.
> >
> > I think I might be inclined to change the directory from "link_pm" to
> > "link", e.g.,
> >
> >   - /sys/bus/pci/devices/0000:00:1c.0/link_pm/clkpm
> >   + /sys/bus/pci/devices/0000:00:1c.0/link/clkpm
> >
> > because there are other things that haven't been merged yet that could
> > go in link/ as well:
> >
> >   * Mika's "link disable" control [1]
> >   * Dilip's link width/speed controls [2,3]
> >
> > The max_link_speed, max_link_width, current_link_speed,
> > current_link_width files could also logically be in link/, although
> > they've already been merged at the top level.
> >
> > Rajat's AER statistics change [4] is also coming.  Those stats aren't
> > link-related, so they wouldn't go in link/.  The current strawman is
> > an "aer_stats" directory, but I wonder if we should make a more
> > generic directory like "errors" that could be used for both AER and
> > DPC and potentially other error-related things.
> 
> Sorry, I haven't been able to find time for it for some time. I doubt
> if I'll be able to make it to 5.6 timeframe. Nevertheless...
> 
> >
> > For example, we could have these link-related things:
> >
> >   /sys/.../0000:00:1c.0/link/clkpm            # RW ASPM stuff
> >   /sys/.../0000:00:1c.0/link/l0s_aspm
> >   /sys/.../0000:00:1c.0/link/...
> >   /sys/.../0000:00:1c.0/link/disable          # RW Mika
> >   /sys/.../0000:00:1c.0/link/speed            # RW Dilip's control
> >   /sys/.../0000:00:1c.0/link/width            # RW Dilip's control
> >   /sys/.../0000:00:1c.0/link/max_speed        # RO possible rework
> >   /sys/.../0000:00:1c.0/link/max_width        # RO possible rework
> >
> > With these backwards compatibility symlinks:
> >
> >   /sys/.../0000:00:1c.0/max_link_speed     -> link/max_speed
> >   /sys/.../0000:00:1c.0/current_link_speed -> link/speed
> >
> > Rajat's current patch puts the AER stats here at the top level:
> >
> >   /sys/.../0000:00:1c.0/aer_stats/fatal_bit4_DLP
> >
> > But maybe we could push them down like this:
> >
> >   /sys/.../0000:00:1c.0/errors/aer/stats/unc_04_dlp
> >   /sys/.../0000:00:1c.0/errors/aer/stats/unc_26_poison_tlb_blocked
> >   /sys/.../0000:00:1c.0/errors/aer/stats/cor_00_rx_err
> >   /sys/.../0000:00:1c.0/errors/aer/stats/cor_15_hdr_log_overflow
> 
> How do we create sub-sub-sub directories in sysfs (errors/aer/stats)?

You should not.

> My understanding is that we can only create 1 subdirectory by using a
> "named" attribute group. If we want more hierarchy, the "errors" and
> the "aer" will need to be backed up by a kobject. Doable, but just
> mentioning.

Not doable, you break userspace tools as they will not "see" those
directories or attributes.

Keep it only 1 deep if at all possible please.

thanks,

greg k-h
Bjorn Helgaas Nov. 21, 2019, 11:04 p.m. UTC | #5
[-cc Wong, Hui]

On Thu, Nov 21, 2019 at 10:10:17PM +0100, Greg KH wrote:
> On Thu, Nov 21, 2019 at 01:03:06PM -0800, Rajat Jain wrote:
> > On Thu, Nov 21, 2019 at 12:49 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
> > > On Sat, Oct 05, 2019 at 02:07:56PM +0200, Heiner Kallweit wrote:
> > >
> > > > +What:                /sys/bus/pci/devices/.../link_pm/clkpm
> > > > +             /sys/bus/pci/devices/.../link_pm/l0s_aspm
> > > > +             /sys/bus/pci/devices/.../link_pm/l1_aspm
> > > > +             /sys/bus/pci/devices/.../link_pm/l1_1_aspm
> > > > +             /sys/bus/pci/devices/.../link_pm/l1_2_aspm
> > > > +             /sys/bus/pci/devices/.../link_pm/l1_1_pcipm
> > > > +             /sys/bus/pci/devices/.../link_pm/l1_2_pcipm
> > > > +Date:                October 2019
> > > > +Contact:     Heiner Kallweit <hkallweit1@gmail.com>
> > > > +Description: If ASPM is supported for an endpoint, then these files
> > > > +             can be used to disable or enable the individual
> > > > +             power management states. Write y/1/on to enable,
> > > > +             n/0/off to disable.
> > >
> > > This is queued up for the v5.5 merge window, so if we want to tweak
> > > anything (path names or otherwise), now is the time.
> > >
> > > I think I might be inclined to change the directory from "link_pm" to
> > > "link", e.g.,
> > >
> > >   - /sys/bus/pci/devices/0000:00:1c.0/link_pm/clkpm
> > >   + /sys/bus/pci/devices/0000:00:1c.0/link/clkpm
> > >
> > > because there are other things that haven't been merged yet that could
> > > go in link/ as well:
> > >
> > >   * Mika's "link disable" control [1]
> > >   * Dilip's link width/speed controls [2,3]
> > >
> > > The max_link_speed, max_link_width, current_link_speed,
> > > current_link_width files could also logically be in link/, although
> > > they've already been merged at the top level.
> > >
> > > Rajat's AER statistics change [4] is also coming.  Those stats aren't
> > > link-related, so they wouldn't go in link/.  The current strawman is
> > > an "aer_stats" directory, but I wonder if we should make a more
> > > generic directory like "errors" that could be used for both AER and
> > > DPC and potentially other error-related things.
> > 
> > Sorry, I haven't been able to find time for it for some time. I doubt
> > if I'll be able to make it to 5.6 timeframe. Nevertheless...
> > 
> > > For example, we could have these link-related things:
> > >
> > >   /sys/.../0000:00:1c.0/link/clkpm            # RW ASPM stuff
> > >   /sys/.../0000:00:1c.0/link/l0s_aspm
> > >   /sys/.../0000:00:1c.0/link/...
> > >   /sys/.../0000:00:1c.0/link/disable          # RW Mika
> > >   /sys/.../0000:00:1c.0/link/speed            # RW Dilip's control
> > >   /sys/.../0000:00:1c.0/link/width            # RW Dilip's control
> > >   /sys/.../0000:00:1c.0/link/max_speed        # RO possible rework
> > >   /sys/.../0000:00:1c.0/link/max_width        # RO possible rework
> > >
> > > With these backwards compatibility symlinks:
> > >
> > >   /sys/.../0000:00:1c.0/max_link_speed     -> link/max_speed
> > >   /sys/.../0000:00:1c.0/current_link_speed -> link/speed
> > >
> > > Rajat's current patch puts the AER stats here at the top level:
> > >
> > >   /sys/.../0000:00:1c.0/aer_stats/fatal_bit4_DLP
> > >
> > > But maybe we could push them down like this:
> > >
> > >   /sys/.../0000:00:1c.0/errors/aer/stats/unc_04_dlp
> > >   /sys/.../0000:00:1c.0/errors/aer/stats/unc_26_poison_tlb_blocked
> > >   /sys/.../0000:00:1c.0/errors/aer/stats/cor_00_rx_err
> > >   /sys/.../0000:00:1c.0/errors/aer/stats/cor_15_hdr_log_overflow
> > 
> > How do we create sub-sub-sub directories in sysfs (errors/aer/stats)?
> 
> You should not.
> 
> > My understanding is that we can only create 1 subdirectory by using a
> > "named" attribute group. If we want more hierarchy, the "errors" and
> > the "aer" will need to be backed up by a kobject. Doable, but just
> > mentioning.
> 
> Not doable, you break userspace tools as they will not "see" those
> directories or attributes.
>
> Keep it only 1 deep if at all possible please.

Oh, that's good to know, thanks!  I guess we'll have to think more
about the error stuff.

What sort of tools would this break?  There are no AER tools since the
AER stats sysfs files don't exist yet, so I assume there are some
generic sysfs tools or libraries.

Incidentally,
https://www.kernel.org/doc/html/latest/admin-guide/sysfs-rules.html
suggests that maybe we should be documenting these files with
/sys/devices paths instead of the symlinks in /sys/bus/pci/devices/,
e.g.,

  diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci
  -What:		/sys/bus/pci/devices/.../msi_bus
  -What:		/sys/bus/pci/devices/.../msi_irqs/
  -What:		/sys/bus/pci/devices/.../msi_irqs/<N>
  +What:		/sys/devices/pci*/.../msi_bus
  +What:		/sys/devices/pci*/.../msi_irqs/
  +What:		/sys/devices/pci*/.../msi_irqs/<N>
Greg Kroah-Hartman Nov. 24, 2019, 5:02 p.m. UTC | #6
On Thu, Nov 21, 2019 at 05:04:11PM -0600, Bjorn Helgaas wrote:
> What sort of tools would this break?  There are no AER tools since the
> AER stats sysfs files don't exist yet, so I assume there are some
> generic sysfs tools or libraries.

Any normal tool that looks at sysfs attributes, like udev and friends.
They will "miss" the uevent for the subdirs and not know how to
associate anything with the "parent" struct device.

> Incidentally,
> https://www.kernel.org/doc/html/latest/admin-guide/sysfs-rules.html
> suggests that maybe we should be documenting these files with
> /sys/devices paths instead of the symlinks in /sys/bus/pci/devices/,
> e.g.,
> 
>   diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci
>   -What:		/sys/bus/pci/devices/.../msi_bus
>   -What:		/sys/bus/pci/devices/.../msi_irqs/
>   -What:		/sys/bus/pci/devices/.../msi_irqs/<N>
>   +What:		/sys/devices/pci*/.../msi_bus
>   +What:		/sys/devices/pci*/.../msi_irqs/
>   +What:		/sys/devices/pci*/.../msi_irqs/<N>

Either is fine, but yes, the second one is nicer.

thanks,

greg k-h
diff mbox series

Patch

diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci
index 8bfee557e..c86c8ab00 100644
--- a/Documentation/ABI/testing/sysfs-bus-pci
+++ b/Documentation/ABI/testing/sysfs-bus-pci
@@ -347,3 +347,17 @@  Description:
 		If the device has any Peer-to-Peer memory registered, this
 	        file contains a '1' if the memory has been published for
 		use outside the driver that owns the device.
+
+What:		/sys/bus/pci/devices/.../link_pm/clkpm
+		/sys/bus/pci/devices/.../link_pm/l0s_aspm
+		/sys/bus/pci/devices/.../link_pm/l1_aspm
+		/sys/bus/pci/devices/.../link_pm/l1_1_aspm
+		/sys/bus/pci/devices/.../link_pm/l1_2_aspm
+		/sys/bus/pci/devices/.../link_pm/l1_1_pcipm
+		/sys/bus/pci/devices/.../link_pm/l1_2_pcipm
+Date:		October 2019
+Contact:	Heiner Kallweit <hkallweit1@gmail.com>
+Description:	If ASPM is supported for an endpoint, then these files
+		can be used to disable or enable the individual
+		power management states. Write y/1/on to enable,
+		n/0/off to disable.
diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c
index 793412954..0e76c02e0 100644
--- a/drivers/pci/pci-sysfs.c
+++ b/drivers/pci/pci-sysfs.c
@@ -1587,6 +1587,9 @@  static const struct attribute_group *pci_dev_attr_groups[] = {
 	&pcie_dev_attr_group,
 #ifdef CONFIG_PCIEAER
 	&aer_stats_attr_group,
+#endif
+#ifdef CONFIG_PCIEASPM
+	&aspm_ctrl_attr_group,
 #endif
 	NULL,
 };
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 3f6947ee3..b2cd21e8c 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -667,4 +667,8 @@  static inline int pci_acpi_program_hp_params(struct pci_dev *dev)
 }
 #endif
 
+#ifdef CONFIG_PCIEASPM
+extern const struct attribute_group aspm_ctrl_attr_group;
+#endif
+
 #endif /* DRIVERS_PCI_H */
diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
index 91cfea673..05ea02abf 100644
--- a/drivers/pci/pcie/aspm.c
+++ b/drivers/pci/pcie/aspm.c
@@ -899,6 +899,14 @@  static struct pcie_link_state *alloc_pcie_link_state(struct pci_dev *pdev)
 	return link;
 }
 
+static void pcie_aspm_update_sysfs_visibility(struct pci_dev *pdev)
+{
+	struct pci_dev *child;
+
+	list_for_each_entry(child, &pdev->subordinate->devices, bus_list)
+		sysfs_update_group(&child->dev.kobj, &aspm_ctrl_attr_group);
+}
+
 /*
  * pcie_aspm_init_link_state: Initiate PCI express link state.
  * It is called after the pcie and its children devices are scanned.
@@ -960,6 +968,9 @@  void pcie_aspm_init_link_state(struct pci_dev *pdev)
 		pcie_set_clkpm(link, policy_to_clkpm_state(link));
 	}
 
+	/* Update visibility of ASPM sysfs attributes */
+	pcie_aspm_update_sysfs_visibility(pdev);
+
 unlock:
 	mutex_unlock(&aspm_lock);
 out:
@@ -1315,6 +1326,169 @@  void pcie_aspm_remove_sysfs_dev_files(struct pci_dev *pdev)
 }
 #endif
 
+static ssize_t aspm_attr_show_common(struct device *dev,
+				     struct device_attribute *attr,
+				     char *buf, u8 state)
+{
+	struct pci_dev *pdev = to_pci_dev(dev);
+	struct pcie_link_state *link;
+	bool enabled;
+
+	link = pcie_aspm_get_link(pdev);
+
+	mutex_lock(&aspm_lock);
+	enabled = link->aspm_enabled & state;
+	mutex_unlock(&aspm_lock);
+
+	return sprintf(buf, "%d\n", enabled ? 1 : 0);
+}
+
+static ssize_t aspm_attr_store_common(struct device *dev,
+				      struct device_attribute *attr,
+				      const char *buf, size_t len, u8 state)
+{
+	struct pci_dev *pdev = to_pci_dev(dev);
+	struct pcie_link_state *link;
+	bool state_enable;
+
+	link = pcie_aspm_get_link(pdev);
+
+	if (strtobool(buf, &state_enable) < 0)
+		return -EINVAL;
+
+	down_read(&pci_bus_sem);
+	mutex_lock(&aspm_lock);
+
+	if (state_enable) {
+		link->aspm_disable &= ~state;
+		/* need to enable L1 for sub-states */
+		if (state & ASPM_STATE_L1SS)
+			link->aspm_disable &= ~ASPM_STATE_L1;
+	} else {
+		link->aspm_disable |= state;
+	}
+
+	pcie_config_aspm_link(link, policy_to_aspm_state(link));
+
+	mutex_unlock(&aspm_lock);
+	up_read(&pci_bus_sem);
+
+	return len;
+}
+
+#define ASPM_ATTR(_f, _s)						\
+static ssize_t _f##_show(struct device *dev,				\
+			 struct device_attribute *attr, char *buf)	\
+{ return aspm_attr_show_common(dev, attr, buf, ASPM_STATE_##_s); }	\
+									\
+static ssize_t _f##_store(struct device *dev,				\
+			  struct device_attribute *attr,		\
+			  const char *buf, size_t len)			\
+{ return aspm_attr_store_common(dev, attr, buf, len, ASPM_STATE_##_s); }
+
+ASPM_ATTR(l0s_aspm, L0S)
+ASPM_ATTR(l1_aspm, L1)
+ASPM_ATTR(l1_1_aspm, L1_1)
+ASPM_ATTR(l1_2_aspm, L1_2)
+ASPM_ATTR(l1_1_pcipm, L1_1_PCIPM)
+ASPM_ATTR(l1_2_pcipm, L1_2_PCIPM)
+
+static ssize_t clkpm_show(struct device *dev,
+			  struct device_attribute *attr, char *buf)
+{
+	struct pci_dev *pdev = to_pci_dev(dev);
+	struct pcie_link_state *link;
+	int val;
+
+	link = pcie_aspm_get_link(pdev);
+
+	mutex_lock(&aspm_lock);
+	val = link->clkpm_enabled;
+	mutex_unlock(&aspm_lock);
+
+	return sprintf(buf, "%d\n", val);
+}
+
+static ssize_t clkpm_store(struct device *dev,
+			   struct device_attribute *attr,
+			   const char *buf, size_t len)
+{
+	struct pci_dev *pdev = to_pci_dev(dev);
+	struct pcie_link_state *link;
+	bool state_enable;
+
+	link = pcie_aspm_get_link(pdev);
+
+	if (strtobool(buf, &state_enable) < 0)
+		return -EINVAL;
+
+	down_read(&pci_bus_sem);
+	mutex_lock(&aspm_lock);
+
+	link->clkpm_disable = !state_enable;
+	pcie_set_clkpm(link, policy_to_clkpm_state(link));
+
+	mutex_unlock(&aspm_lock);
+	up_read(&pci_bus_sem);
+
+	return len;
+}
+
+static DEVICE_ATTR_RW(clkpm);
+static DEVICE_ATTR_RW(l0s_aspm);
+static DEVICE_ATTR_RW(l1_aspm);
+static DEVICE_ATTR_RW(l1_1_aspm);
+static DEVICE_ATTR_RW(l1_2_aspm);
+static DEVICE_ATTR_RW(l1_1_pcipm);
+static DEVICE_ATTR_RW(l1_2_pcipm);
+
+static struct attribute *aspm_ctrl_attrs[] = {
+	&dev_attr_clkpm.attr,
+	&dev_attr_l0s_aspm.attr,
+	&dev_attr_l1_aspm.attr,
+	&dev_attr_l1_1_aspm.attr,
+	&dev_attr_l1_2_aspm.attr,
+	&dev_attr_l1_1_pcipm.attr,
+	&dev_attr_l1_2_pcipm.attr,
+	NULL
+};
+
+static umode_t aspm_ctrl_attrs_are_visible(struct kobject *kobj,
+					   struct attribute *a, int n)
+{
+	struct device *dev = kobj_to_dev(kobj);
+	struct pci_dev *pdev = to_pci_dev(dev);
+	struct pcie_link_state *link = NULL;
+	static const u8 aspm_state_map[] = {
+		ASPM_STATE_L0S,
+		ASPM_STATE_L1,
+		ASPM_STATE_L1_1,
+		ASPM_STATE_L1_2,
+		ASPM_STATE_L1_1_PCIPM,
+		ASPM_STATE_L1_2_PCIPM,
+	};
+
+	if (aspm_disabled)
+		return 0;
+
+	if (pci_is_pcie(pdev))
+		link = pcie_aspm_get_link(pdev);
+
+	if (!link)
+		return 0;
+
+	if (n)
+		return link->aspm_capable & aspm_state_map[n - 1] ? a->mode : 0;
+	else
+		return link->clkpm_capable ? a->mode : 0;
+}
+
+const struct attribute_group aspm_ctrl_attr_group = {
+	.name = "link_pm",
+	.attrs = aspm_ctrl_attrs,
+	.is_visible = aspm_ctrl_attrs_are_visible,
+};
+
 static int __init pcie_aspm_disable(char *str)
 {
 	if (!strcmp(str, "off")) {