From patchwork Tue Apr 19 10:28:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 1618800 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=glv/vOuT; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4KjKl60HTvz9sG0 for ; Tue, 19 Apr 2022 20:28:18 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242263AbiDSKa4 (ORCPT ); Tue, 19 Apr 2022 06:30:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57276 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242200AbiDSKaz (ORCPT ); Tue, 19 Apr 2022 06:30:55 -0400 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85263275F6; Tue, 19 Apr 2022 03:28:13 -0700 (PDT) Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23J9Aexp004777; Tue, 19 Apr 2022 10:28:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=QnPfcT3aQm4wRRbd1y2TwgeqmbD666U2uW+Vxfg7yaQ=; b=glv/vOuT58kVk8IeD+SxAWgb5AZgUx1RNZyO1Pvbog+G7M1iNrMqS4fDdLFI0N9xxTOA gVsv/R7aSAY3kn+Pz3bfdLxyocumpeAAOzFObd4D6Rb/A8Nhht6eHg9do7A+sZtlzGhc TKXpssa4f73Pl744ZwpTEq0Psp8n5001SBtj00Y/NNiH3NGmBzfWmO75IJlr8gcD5++c WmH36Yb3bYz3RqZbsCvCgXlD0F+v0IMAZftc4c4ke4N/DjWMZZeU6fxLNE5BaEiAmr1R SenPYh2ugq4Dz6bxeL+I36/8iYoBs5DhDr+vh+7chUFlLKtYGgs56tQ+6wXYtAy7sQsB Cg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3fg7ekva0m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 19 Apr 2022 10:28:09 +0000 Received: from m0098417.ppops.net (m0098417.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 23J9vo4v023136; Tue, 19 Apr 2022 10:28:09 GMT Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com [169.51.49.99]) by mx0a-001b2d01.pphosted.com with ESMTP id 3fg7ekv9yx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 19 Apr 2022 10:28:09 +0000 Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1]) by ppma04ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 23JACxiA026438; Tue, 19 Apr 2022 10:28:07 GMT Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by ppma04ams.nl.ibm.com with ESMTP id 3ffne945vx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 19 Apr 2022 10:28:07 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 23JAS4CB37224850 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 19 Apr 2022 10:28:04 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4F05811C04C; Tue, 19 Apr 2022 10:28:04 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0F27911C050; Tue, 19 Apr 2022 10:28:04 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 19 Apr 2022 10:28:03 +0000 (GMT) From: Niklas Schnelle To: Bjorn Helgaas , Jan Kiszka , Matthew Rosato , Pierre Morel Cc: linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-s390@vger.kernel.org, linux-pci@vger.kernel.org Subject: [PATCH v3 1/4] PCI: Clean up pci_scan_slot() Date: Tue, 19 Apr 2022 12:28:00 +0200 Message-Id: <20220419102803.3430139-2-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220419102803.3430139-1-schnelle@linux.ibm.com> References: <20220419102803.3430139-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: t8czAsQle4o1PotRlKGhmVkxOf1_Dlo3 X-Proofpoint-GUID: MB_ON1VMkQx_KoETrl42A61be9hStEkS X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-04-19_03,2022-04-15_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 lowpriorityscore=0 mlxscore=0 bulkscore=0 adultscore=0 spamscore=0 mlxlogscore=999 impostorscore=0 phishscore=0 suspectscore=0 clxscore=1015 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204190052 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org While determining the next PCI function is factored out of pci_scan_slot() into next_fn() the former still handles the first function as a special case duplicating the code from the scan loop and splitting the condition that the first function exits from it being multifunction which is tested in next_fn(). Furthermore the non ARI branch of next_fn() mixes the case that multifunction devices may have non-contiguous function ranges and dev may thus be NULL with the multifunction requirement. It also signals that no further functions need to be scanned by returning 0 which is a valid function number. Improve upon this by moving all conditions for having to scan for more functions into next_fn() and make them obvious and commented. By changing next_fn() to return -ENODEV instead of 0 when there is no next function we can then handle the initial function inside the loop and deduplicate the shared handling. No functional change is intended. Signed-off-by: Niklas Schnelle --- drivers/pci/probe.c | 41 +++++++++++++++++++---------------------- 1 file changed, 19 insertions(+), 22 deletions(-) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 17a969942d37..389aa1f9cb2c 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2579,33 +2579,35 @@ struct pci_dev *pci_scan_single_device(struct pci_bus *bus, int devfn) } EXPORT_SYMBOL(pci_scan_single_device); -static unsigned int next_fn(struct pci_bus *bus, struct pci_dev *dev, - unsigned int fn) +static int next_fn(struct pci_bus *bus, struct pci_dev *dev, int fn) { int pos; u16 cap = 0; unsigned int next_fn; - if (pci_ari_enabled(bus)) { - if (!dev) - return 0; + if (dev && pci_ari_enabled(bus)) { pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ARI); if (!pos) - return 0; + return -ENODEV; pci_read_config_word(dev, pos + PCI_ARI_CAP, &cap); next_fn = PCI_ARI_CAP_NFN(cap); if (next_fn <= fn) - return 0; /* protect against malformed list */ + return -ENODEV; /* protect against malformed list */ return next_fn; } - /* dev may be NULL for non-contiguous multifunction devices */ - if (!dev || dev->multifunction) - return (fn + 1) % 8; - - return 0; + /* only multifunction devices may have more functions */ + if (dev && !dev->multifunction) + return -ENODEV; + /* + * A function 0 is required but multifunction devices may + * be non-contiguous so dev can be NULL otherwise. + */ + if (!fn && !dev) + return -ENODEV; + return (fn <= 6) ? fn + 1 : -ENODEV; } static int only_one_child(struct pci_bus *bus) @@ -2643,24 +2645,19 @@ static int only_one_child(struct pci_bus *bus) */ int pci_scan_slot(struct pci_bus *bus, int devfn) { - unsigned int fn, nr = 0; - struct pci_dev *dev; + int fn, nr = 0; + struct pci_dev *dev = NULL; if (only_one_child(bus) && (devfn > 0)) return 0; /* Already scanned the entire slot */ - dev = pci_scan_single_device(bus, devfn); - if (!dev) - return 0; - if (!pci_dev_is_added(dev)) - nr++; - - for (fn = next_fn(bus, dev, 0); fn > 0; fn = next_fn(bus, dev, fn)) { + for (fn = 0; fn >= 0; fn = next_fn(bus, dev, fn)) { dev = pci_scan_single_device(bus, devfn + fn); if (dev) { if (!pci_dev_is_added(dev)) nr++; - dev->multifunction = 1; + if (nr > 1) + dev->multifunction = 1; } } From patchwork Tue Apr 19 10:28:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 1618804 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=EtJcqmuf; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4KjKl86Bqhz9sG6 for ; Tue, 19 Apr 2022 20:28:20 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350683AbiDSKbA (ORCPT ); Tue, 19 Apr 2022 06:31:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57312 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350666AbiDSKa5 (ORCPT ); Tue, 19 Apr 2022 06:30:57 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D492426562; Tue, 19 Apr 2022 03:28:15 -0700 (PDT) Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23JA9qvB026300; Tue, 19 Apr 2022 10:28:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : mime-version; s=pp1; bh=+4dOxYOPtwtBuZjYxuIWrW63gioildK1M39qmgldJ6Q=; b=EtJcqmufjstz0jdcFcN6QgMClA0INXt50tVzWt0pHa0g2sWvxa9+jX7f8vQ7SfsnxeHv oPhHPT9Aak+aWY3RhFDvnPc4cqZLR6W20XsNLbdDh0TgC4QyKldZWcX5EAd5iiPhsnSi jpGyMhHqfL6UtMbJvCZ1i8FZppJiK//NhE67Ownm9PWbkcr5ujDCYEPgsWJahoFByZns SiylaNSYFbTZK+3bBT3VyfOjL2T5mq3HKq53sVketqeO6RLe+f8+8ihjC3cZVt1IU1vC 7WSSaeAbMzY4xvCqY80ZjnZJKl2QDz1wNVjssbc45KxADj463s99Se4zf/QvVK5zrzSv 5Q== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3fg7vntw6e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 19 Apr 2022 10:28:11 +0000 Received: from m0098393.ppops.net (m0098393.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 23JAQDGj016758; Tue, 19 Apr 2022 10:28:10 GMT Received: from ppma04fra.de.ibm.com (6a.4a.5195.ip4.static.sl-reverse.com [149.81.74.106]) by mx0a-001b2d01.pphosted.com with ESMTP id 3fg7vntw5n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 19 Apr 2022 10:28:10 +0000 Received: from pps.filterd (ppma04fra.de.ibm.com [127.0.0.1]) by ppma04fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 23JADdOp020327; Tue, 19 Apr 2022 10:28:08 GMT Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by ppma04fra.de.ibm.com with ESMTP id 3ffvt9ay63-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 19 Apr 2022 10:28:07 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 23JAS4VV37224854 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 19 Apr 2022 10:28:04 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 99BED11C054; Tue, 19 Apr 2022 10:28:04 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 59CB911C052; Tue, 19 Apr 2022 10:28:04 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 19 Apr 2022 10:28:04 +0000 (GMT) From: Niklas Schnelle To: Bjorn Helgaas , Jan Kiszka , Matthew Rosato , Pierre Morel Cc: linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-s390@vger.kernel.org, linux-pci@vger.kernel.org Subject: [PATCH v3 2/4] PCI: Move jailhouse's isolated function handling to pci_scan_slot() Date: Tue, 19 Apr 2022 12:28:01 +0200 Message-Id: <20220419102803.3430139-3-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220419102803.3430139-1-schnelle@linux.ibm.com> References: <20220419102803.3430139-1-schnelle@linux.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: M_0DY4D98XC3juVwNZx3D0se8wf5xJxh X-Proofpoint-GUID: ImOh5p38FNgg00uQQBt7JgAEERB4B23Y X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-04-19_04,2022-04-15_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 mlxscore=0 phishscore=0 suspectscore=0 bulkscore=0 lowpriorityscore=0 spamscore=0 priorityscore=1501 impostorscore=0 mlxlogscore=999 adultscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204190052 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org The special case of the jailhouse hypervisor passing through individual PCI functions handles scanning for PCI functions even if function 0 does not exist. Currently this is done with an extra loop duplicating the one in pci_scan_slot(). By incorporating the check for jailhouse_paravirt() into next_fn() we can instead do this as part of the normal pci_scan_slot(). The only functional change is that we now call pcie_aspm_init_link_state() for these functions but this already happened if function 0 was passed through and should not be a problem. Link: https://lore.kernel.org/linux-pci/20220408224514.GA353445@bhelgaas/ Suggested-by: Bjorn Helgaas Signed-off-by: Niklas Schnelle --- drivers/pci/probe.c | 28 +++++++--------------------- 1 file changed, 7 insertions(+), 21 deletions(-) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 389aa1f9cb2c..a1e8f1e14c3d 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2602,10 +2602,11 @@ static int next_fn(struct pci_bus *bus, struct pci_dev *dev, int fn) if (dev && !dev->multifunction) return -ENODEV; /* - * A function 0 is required but multifunction devices may - * be non-contiguous so dev can be NULL otherwise. + * Usually a function 0 is required but the jailhouse hypervisor may + * pass individual functions. For non-contiguous multifunction devices + * some functions may also be missing. */ - if (!fn && !dev) + if (!fn && !dev && !jailhouse_paravirt()) return -ENODEV; return (fn <= 6) ? fn + 1 : -ENODEV; } @@ -2855,29 +2856,14 @@ static unsigned int pci_scan_child_bus_extend(struct pci_bus *bus, { unsigned int used_buses, normal_bridges = 0, hotplug_bridges = 0; unsigned int start = bus->busn_res.start; - unsigned int devfn, fn, cmax, max = start; + unsigned int devfn, cmax, max = start; struct pci_dev *dev; - int nr_devs; dev_dbg(&bus->dev, "scanning bus\n"); /* Go find them, Rover! */ - for (devfn = 0; devfn < 256; devfn += 8) { - nr_devs = pci_scan_slot(bus, devfn); - - /* - * The Jailhouse hypervisor may pass individual functions of a - * multi-function device to a guest without passing function 0. - * Look for them as well. - */ - if (jailhouse_paravirt() && nr_devs == 0) { - for (fn = 1; fn < 8; fn++) { - dev = pci_scan_single_device(bus, devfn + fn); - if (dev) - dev->multifunction = 1; - } - } - } + for (devfn = 0; devfn < 256; devfn += 8) + pci_scan_slot(bus, devfn); /* Reserve buses for SR-IOV capability */ used_buses = pci_iov_bus_range(bus); From patchwork Tue Apr 19 10:28:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 1618801 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=kI3gMNQU; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4KjKl63zP5z9sG6 for ; Tue, 19 Apr 2022 20:28:18 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350663AbiDSKa5 (ORCPT ); Tue, 19 Apr 2022 06:30:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242212AbiDSKa4 (ORCPT ); Tue, 19 Apr 2022 06:30:56 -0400 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D6D627FC9; Tue, 19 Apr 2022 03:28:13 -0700 (PDT) Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23J8RrNB004745; Tue, 19 Apr 2022 10:28:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=Gq2oKIc1RzWTzoNaDxTmQ2QjMaScxj7ajJ62N5KdPA8=; b=kI3gMNQU7BweAePKs8NojIdNG3CdZZn3x7IZSo1PvbEUnpBTAL9CvZnb8piu1xVNXnNs ohuK6LRPobMUZKJbhrwkzsgjXdz7Q4yYrGWpffHKZ9wgsOxqlNZqfR7z4Mq3xNMffPZx /RGKqtZlCXp6J7wueapszCGD9fE5zLDksmEYCU5zfYNRAMFDOQiwL1D6JI2NF0GgUNwV Gmegi5W3/wP7xfsnOfcXF2mldTZo2kB6xZ1CP05p3M+I1Z4j4EWHYbcP7cFBlkBz+v35 k9zRdgetCOU5kTHFcn4vrSEyi1QuxeAnImJJeRSMDctT3hWjTrjXRfGwdsUFgwom5MBS iQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3fg7ekva0x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 19 Apr 2022 10:28:10 +0000 Received: from m0098417.ppops.net (m0098417.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 23JA9QDE006964; Tue, 19 Apr 2022 10:28:10 GMT Received: from ppma02fra.de.ibm.com (47.49.7a9f.ip4.static.sl-reverse.com [159.122.73.71]) by mx0a-001b2d01.pphosted.com with ESMTP id 3fg7ekva0a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 19 Apr 2022 10:28:10 +0000 Received: from pps.filterd (ppma02fra.de.ibm.com [127.0.0.1]) by ppma02fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 23JACvwW007462; Tue, 19 Apr 2022 10:28:08 GMT Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by ppma02fra.de.ibm.com with ESMTP id 3fgu6u1t4d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 19 Apr 2022 10:28:08 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 23JAS5gb17957370 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 19 Apr 2022 10:28:05 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DADAB11C052; Tue, 19 Apr 2022 10:28:04 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A4C6711C058; Tue, 19 Apr 2022 10:28:04 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 19 Apr 2022 10:28:04 +0000 (GMT) From: Niklas Schnelle To: Bjorn Helgaas , Jan Kiszka , Matthew Rosato , Pierre Morel Cc: linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-s390@vger.kernel.org, linux-pci@vger.kernel.org Subject: [PATCH v3 3/4] PCI: Extend isolated function probing to s390 Date: Tue, 19 Apr 2022 12:28:02 +0200 Message-Id: <20220419102803.3430139-4-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220419102803.3430139-1-schnelle@linux.ibm.com> References: <20220419102803.3430139-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: k2VqLELu8Q9weWumHrKuTUxMV4bR4NiC X-Proofpoint-GUID: OCethT7X0KOLKk3tuW0XMH0rl49aOvks X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-04-19_03,2022-04-15_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 lowpriorityscore=0 mlxscore=0 bulkscore=0 adultscore=0 spamscore=0 mlxlogscore=832 impostorscore=0 phishscore=0 suspectscore=0 clxscore=1015 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204190052 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Like the jailhouse hypervisor s390's PCI architecture allows passing isolated PCI functions to an OS instance. As of now this is was not utilized even with multi-function support as the s390 PCI code makes sure that only virtual PCI busses including a function with devfn 0 are presented to the PCI subsystem. A subsequent change will remove this restriction. Allow probing such functions by replacing the existing check for jailhouse_paravirt() with a new hypervisor_isolated_pci_functions() helper. Signed-off-by: Niklas Schnelle --- drivers/pci/probe.c | 8 ++++---- include/linux/hypervisor.h | 9 +++++++++ 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index a1e8f1e14c3d..09b46746d48e 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2602,11 +2602,11 @@ static int next_fn(struct pci_bus *bus, struct pci_dev *dev, int fn) if (dev && !dev->multifunction) return -ENODEV; /* - * Usually a function 0 is required but the jailhouse hypervisor may - * pass individual functions. For non-contiguous multifunction devices - * some functions may also be missing. + * Usually a function 0 is required but some hypervisors may pass + * individual functions. For non-contiguous multifunction devices some + * functions may also be missing. */ - if (!fn && !dev && !jailhouse_paravirt()) + if (!fn && !dev && !hypervisor_isolated_pci_functions()) return -ENODEV; return (fn <= 6) ? fn + 1 : -ENODEV; } diff --git a/include/linux/hypervisor.h b/include/linux/hypervisor.h index fc08b433c856..52abd459f9a3 100644 --- a/include/linux/hypervisor.h +++ b/include/linux/hypervisor.h @@ -32,4 +32,13 @@ static inline bool jailhouse_paravirt(void) #endif /* !CONFIG_X86 */ +static inline bool hypervisor_isolated_pci_functions(void) +{ + if (IS_ENABLED(CONFIG_S390)) + return true; + else + return jailhouse_paravirt(); +} + + #endif /* __LINUX_HYPEVISOR_H */ From patchwork Tue Apr 19 10:28:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 1618803 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=sEUnVZkI; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4KjKl83sF9z9sG0 for ; Tue, 19 Apr 2022 20:28:20 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350675AbiDSKa7 (ORCPT ); Tue, 19 Apr 2022 06:30:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241657AbiDSKa4 (ORCPT ); Tue, 19 Apr 2022 06:30:56 -0400 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B1CE527FD8; Tue, 19 Apr 2022 03:28:14 -0700 (PDT) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23J9fHuW012606; Tue, 19 Apr 2022 10:28:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=VKd5vFUiEKfuRcGuSWKwzFHHhQa5vtGA/lkq5v9/Zc8=; b=sEUnVZkIkPMlllNZC2XWKnso6jwozB5AM7HwdX45blmuj8VfmI21TLZorsoi07S805yY 4dyCHlrd2DtUAGRNZZ/SCr1mk02Yb7mppyMUCV22nFCpZCySfOpN6X36xYwUjCTSig3W NnKmIt92y+10a1u93af+wyI4zmKD21jHYCaGxt93fyJVi+vtG/jOJU01g3mNzlkoVMAu JCMGb8qv+Ca4vPIGlDNJug4PuE8fZolLs/QMBbjrD/bRsLfLV8/zZvNbToq/i61MaH5x 639Y6CxG3gGwiD0f8EhLXlXioEhFZaVwYRLp8Fjixu0oEGyah5qTvd/7KILIbE9SY5/e TQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3fg7kac1fy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 19 Apr 2022 10:28:11 +0000 Received: from m0098421.ppops.net (m0098421.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 23JAAM7n019279; Tue, 19 Apr 2022 10:28:10 GMT Received: from ppma02fra.de.ibm.com (47.49.7a9f.ip4.static.sl-reverse.com [159.122.73.71]) by mx0a-001b2d01.pphosted.com with ESMTP id 3fg7kac1f9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 19 Apr 2022 10:28:10 +0000 Received: from pps.filterd (ppma02fra.de.ibm.com [127.0.0.1]) by ppma02fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 23JADEwZ008048; Tue, 19 Apr 2022 10:28:08 GMT Received: from b06cxnps3074.portsmouth.uk.ibm.com (d06relay09.portsmouth.uk.ibm.com [9.149.109.194]) by ppma02fra.de.ibm.com with ESMTP id 3fgu6u1t4e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 19 Apr 2022 10:28:08 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 23JAS50h34406764 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 19 Apr 2022 10:28:05 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3AA5211C050; Tue, 19 Apr 2022 10:28:05 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EF95211C04C; Tue, 19 Apr 2022 10:28:04 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 19 Apr 2022 10:28:04 +0000 (GMT) From: Niklas Schnelle To: Bjorn Helgaas , Jan Kiszka , Matthew Rosato , Pierre Morel Cc: linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-s390@vger.kernel.org, linux-pci@vger.kernel.org Subject: [PATCH v3 4/4] s390/pci: allow zPCI zbus without a function zero Date: Tue, 19 Apr 2022 12:28:03 +0200 Message-Id: <20220419102803.3430139-5-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220419102803.3430139-1-schnelle@linux.ibm.com> References: <20220419102803.3430139-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: -rzEf-AB7HzwKD_71u8p8jSlD4WLw3Kt X-Proofpoint-GUID: A3dutt-W29B1NJv68v56XhGKsahfINAY X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-04-19_04,2022-04-15_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 spamscore=0 priorityscore=1501 adultscore=0 malwarescore=0 suspectscore=0 phishscore=0 mlxscore=0 mlxlogscore=902 bulkscore=0 lowpriorityscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204190056 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Currently the zPCI code block PCI bus creation and probing of a zPCI zbus unless there is a PCI function with devfn 0. This is always the case for the PCI functions with hidden RID but may keep PCI functions from a multi-function PCI device with RID information invisible until the function 0 becomes visible. Worse as a PCI bus is necessary to even present a PCI hotplug slot even that remains invisible. With the probing of these so called isolated PCI functions enabled for s390 in common code this restriction is no longer necessary. On network cards with multiple ports and a PF per port this also allows using each port on its own while still providing the physical PCI topology information in the devfn needed to associate VFs with their parent PF. Signed-off-by: Niklas Schnelle --- arch/s390/pci/pci_bus.c | 82 ++++++++++------------------------------- 1 file changed, 20 insertions(+), 62 deletions(-) diff --git a/arch/s390/pci/pci_bus.c b/arch/s390/pci/pci_bus.c index 5d77acbd1c87..6a8da1b742ae 100644 --- a/arch/s390/pci/pci_bus.c +++ b/arch/s390/pci/pci_bus.c @@ -145,9 +145,6 @@ int zpci_bus_scan_bus(struct zpci_bus *zbus) struct zpci_dev *zdev; int devfn, rc, ret = 0; - if (!zbus->function[0]) - return 0; - for (devfn = 0; devfn < ZPCI_FUNCTIONS_PER_BUS; devfn++) { zdev = zbus->function[devfn]; if (zdev && zdev->state == ZPCI_FN_STATE_CONFIGURED) { @@ -184,26 +181,26 @@ void zpci_bus_scan_busses(void) /* zpci_bus_create_pci_bus - Create the PCI bus associated with this zbus * @zbus: the zbus holding the zdevices - * @f0: function 0 of the bus + * @fr: PCI root function that will determine the bus's domain, and bus speeed * @ops: the pci operations * - * Function zero is taken as a parameter as this is used to determine the - * domain, multifunction property and maximum bus speed of the entire bus. + * The PCI function @fr determines the domain (its UID), multifunction property + * and maximum bus speed of the entire bus. * * Return: 0 on success, an error code otherwise */ -static int zpci_bus_create_pci_bus(struct zpci_bus *zbus, struct zpci_dev *f0, struct pci_ops *ops) +static int zpci_bus_create_pci_bus(struct zpci_bus *zbus, struct zpci_dev *fr, struct pci_ops *ops) { struct pci_bus *bus; int domain; - domain = zpci_alloc_domain((u16)f0->uid); + domain = zpci_alloc_domain((u16)fr->uid); if (domain < 0) return domain; zbus->domain_nr = domain; - zbus->multifunction = f0->rid_available; - zbus->max_bus_speed = f0->max_bus_speed; + zbus->multifunction = fr->rid_available; + zbus->max_bus_speed = fr->max_bus_speed; /* * Note that the zbus->resources are taken over and zbus->resources @@ -303,47 +300,6 @@ void pcibios_bus_add_device(struct pci_dev *pdev) } } -/* zpci_bus_create_hotplug_slots - Add hotplug slot(s) for device added to bus - * @zdev: the zPCI device that was newly added - * - * Add the hotplug slot(s) for the newly added PCI function. Normally this is - * simply the slot for the function itself. If however we are adding the - * function 0 on a zbus, it might be that we already registered functions on - * that zbus but could not create their hotplug slots yet so add those now too. - * - * Return: 0 on success, an error code otherwise - */ -static int zpci_bus_create_hotplug_slots(struct zpci_dev *zdev) -{ - struct zpci_bus *zbus = zdev->zbus; - int devfn, rc = 0; - - rc = zpci_init_slot(zdev); - if (rc) - return rc; - zdev->has_hp_slot = 1; - - if (zdev->devfn == 0 && zbus->multifunction) { - /* Now that function 0 is there we can finally create the - * hotplug slots for those functions with devfn != 0 that have - * been parked in zbus->function[] waiting for us to be able to - * create the PCI bus. - */ - for (devfn = 1; devfn < ZPCI_FUNCTIONS_PER_BUS; devfn++) { - zdev = zbus->function[devfn]; - if (zdev && !zdev->has_hp_slot) { - rc = zpci_init_slot(zdev); - if (rc) - return rc; - zdev->has_hp_slot = 1; - } - } - - } - - return rc; -} - static int zpci_bus_add_device(struct zpci_bus *zbus, struct zpci_dev *zdev) { int rc = -EINVAL; @@ -352,21 +308,19 @@ static int zpci_bus_add_device(struct zpci_bus *zbus, struct zpci_dev *zdev) pr_err("devfn %04x is already assigned\n", zdev->devfn); return rc; } + zdev->zbus = zbus; zbus->function[zdev->devfn] = zdev; zpci_nb_devices++; - if (zbus->bus) { - if (zbus->multifunction && !zdev->rid_available) { - WARN_ONCE(1, "rid_available not set for multifunction\n"); - goto error; - } - - zpci_bus_create_hotplug_slots(zdev); - } else { - /* Hotplug slot will be created once function 0 appears */ - zbus->multifunction = 1; + if (zbus->multifunction && !zdev->rid_available) { + WARN_ONCE(1, "rid_available not set for multifunction\n"); + goto error; } + rc = zpci_init_slot(zdev); + if (rc) + goto error; + zdev->has_hp_slot = 1; return 0; @@ -400,7 +354,11 @@ int zpci_bus_device_register(struct zpci_dev *zdev, struct pci_ops *ops) return -ENOMEM; } - if (zdev->devfn == 0) { + if (!zbus->bus) { + /* The UID of the first PCI function registered with a zpci_bus + * is used as the domain number for that bus. Currently there + * is exactly one zpci_bus per domain. + */ rc = zpci_bus_create_pci_bus(zbus, zdev, ops); if (rc) goto error;