From patchwork Wed Apr 11 16:06:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Puthukattukaran X-Patchwork-Id: 897268 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.b="cgPsKyho"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40LpqL0wHCz9s27 for ; Thu, 12 Apr 2018 02:08:14 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753317AbeDKQIM (ORCPT ); Wed, 11 Apr 2018 12:08:12 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:49716 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753040AbeDKQIM (ORCPT ); Wed, 11 Apr 2018 12:08:12 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w3BG6ZLa173637; Wed, 11 Apr 2018 16:08:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=to : cc : from : subject : message-id : date : mime-version : content-type : content-transfer-encoding; s=corp-2017-10-26; bh=pB5FzvV872Be7bY6hVzulJmms4rr/gxJRS0gv5cBVsE=; b=cgPsKyhoszWrbmFAUkvb/hyregYG/3y8bkOkiUInj6xv6HR765kwuGNoszp0z2yU4fv/ QWMq/x0r3LTC7aLeIhmFKDnpoRY7UkeAuMhbeCvUKC4iuJlVuZw0788+Ytma2RRql2hl GelyT58GHk2Fjw5ZOdXj1O98q+oUTjTVa97ocQOTpVHDqtsELQSSInTbtYVBKRT/rvvc pzzFUuK3wWn6vwM5opDYQHrBUlBBs7MOZ/dpQcY/kMV8t/8Ff7CUbt04IckAnSjdnZGg 1P+zoK7LpgzGgQX/9sTopLZcXc8nlF4L0UAecIKZrXNlvpJ8Vi6k0AI5EpMESk3cgmOT 3w== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2130.oracle.com with ESMTP id 2h6ne7g2cw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 11 Apr 2018 16:08:08 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w3BG87df024055 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 11 Apr 2018 16:08:08 GMT Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w3BG87jh000356; Wed, 11 Apr 2018 16:08:07 GMT Received: from anastasis.us.oracle.com (/10.152.36.80) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 11 Apr 2018 09:08:07 -0700 To: "linux-pci@vger.kernel.org" Cc: Sinan Kaya , Bjorn Helgaas From: James Puthukattukaran Subject: [PATCH 2/2] PCI: Workaround ACS hw bug for IDT switch Message-ID: <2e297847-323a-09b6-f42e-e179a9306b91@oracle.com> Date: Wed, 11 Apr 2018 12:06:21 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 Content-Language: en-US X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8860 signatures=668698 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=699 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1804110150 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org This patch implements the hw workaround found in the IDT switch. The IDT switch incorrectly flags an ACS source violation on a read config request to an end point device on the completion (IDT 89H32H8G3-YC, errata #36) even though the PCI Express spec states that completions are never affected by ACS source violation (PCI Spec 3.1, Section 6.12.1.1). Here's the specific copy of the errata text "Item #36 - Downstream port applies ACS Source Validation to Completions Section 6.12.1.1 of the PCI Express Base Specification 3.1 states that completions are never affected by ACS Source Validation. However, completions received by a downstream port of the PCIe switch from a device that has not yet captured a PCIe bus number are incorrectly dropped by ACS source validation by the switch downstream port. Workaround: Issue a CfgWr1 to the downstream device before issuing the first CfgRd1 to the device. This allows the downstream device to capture its bus number; ACS source validation no longer stops completions from being forwarded by the downstream port. It has been observed that Microsoft Windows implements this workaround already; however, some versions of Linux and other operating systems may not. " The suggested workaround by IDT is to issue a configuration write to the downstream device before issuing the first config read. This allows the downstream device to capture its bus number, thus avoiding the ACS violation on the completion. In order to make sure that the device is ready for config accesses, we do what is currently done in making config reads till it succeeds and then do the config write as specified by the errata. However, to avoid hitting the errata issue when doing config reads, we disable ACS SV around this process. The patch does the following - 1. Disable ACS source violation if enabled. 2. Wait for config space access to become available by reading vendor id 3. Do a config write to the end point (errata workaround) 4. Enable ACS source validation (if it was enabled to begin with) --- drivers/pci/quirks.c | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) --- diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index bf423e3..932e0ff 100644 --- a/drivers/pci/quirks.c +++ b/drivers/pci/quirks.c @@ -4738,13 +4738,51 @@ static void quirk_gpu_hda(struct pci_dev *hda) DECLARE_PCI_FIXUP_CLASS_FINAL(PCI_VENDOR_ID_NVIDIA, PCI_ANY_ID, PCI_CLASS_MULTIMEDIA_HD_AUDIO, 8, quirk_gpu_hda); +static int pci_idt_acs_quirk(struct pci_bus *bus, int devfn, int enable, + bool found) +{ + int pos; + u16 cap; + u16 ctrl; + int retval; + struct pci_dev *dev = bus->self; + + + /* Write 0 to the devfn device under the PCIE switch (bus->self) + * as part of forcing the devfn number to latch with the device below + */ + if (found) + pci_bus_write_config_word(bus, devfn, PCI_VENDOR_ID, 0); + + + /* Enable/disable ACS SV feature (based on enable flag) */ + pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ACS); + if (!pos) + return -ENODEV; + + pci_read_config_word(dev, pos + PCI_ACS_CAP, &cap); + + if (!(cap & PCI_ACS_SV)) + return -ENODEV; + pci_read_config_word(dev, pos + PCI_ACS_CTRL, &ctrl); + + retval = !!(ctrl & cap & PCI_ACS_SV); + if (enable) + ctrl |= (cap & PCI_ACS_SV); + else + ctrl &= ~(cap & PCI_ACS_SV); + + pci_write_config_word(dev, pos + PCI_ACS_CTRL, ctrl); + return retval; +} static const struct pci_dev_acs_quirk{ u16 vendor; u16 device; int (*acs_quirk)(struct pci_bus *bus, int devfn, int enable, bool found); } pci_dev_acs_quirks[] = { + { PCI_VENDOR_ID_IDT, 0x80b5, pci_idt_acs_quirk}, {0} };