From patchwork Wed Nov 3 13:56:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Heimes X-Patchwork-Id: 1550287 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=iRVfKKTB; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HkpJ01ZFGz9sXS for ; Thu, 4 Nov 2021 00:57:51 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1miGlq-0000ZR-TF; Wed, 03 Nov 2021 13:57:42 +0000 Received: from smtp-relay-canonical-1.internal ([10.131.114.174] helo=smtp-relay-canonical-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1miGlo-0000Yr-AU for kernel-team@lists.ubuntu.com; Wed, 03 Nov 2021 13:57:40 +0000 Received: from T570.fritz.box (p54abb560.dip0.t-ipconnect.de [84.171.181.96]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id 209193F1CD for ; Wed, 3 Nov 2021 13:57:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1635947860; bh=4F+d0/Ihkn3MJ9gLbjbf8N4rJq5CWuAWg8In/m4iMRY=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iRVfKKTBW1sbxK8Qm775RR4v1Wt62w9nlOV//t59EMGkqMT+k2F+42WOQsCeq8tXG hISCol+ftNAe1Fj/pQ2oDeFP6zTOyvMhyiY7Axsbl/cBXjPFRZ77W+kMZc+x6zEtkU 08GZE5o1v/rVSZ40aGPAOkAJdF2tzaDv2ECBcPF9PIlpne1TPNITgFMuDtKxaA0N2C 03JGQlmyGw0Fz0qxe1zg4JYiWoiW8Slix+5rN3yqugGPguWYqoLmouz1H+nmTtfazv LfXhilmWSPEJiK81oc53CRCv22F90sK5b+LRH217b012GKCZBroMZIMh3Vzn1i1Szh oUFbDX67kRZkA== From: frank.heimes@canonical.com To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 1/3] s390/pci: fix leak of PCI device structure Date: Wed, 3 Nov 2021 14:56:35 +0100 Message-Id: <20211103135637.2220305-2-frank.heimes@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211103135637.2220305-1-frank.heimes@canonical.com> References: <20211103135637.2220305-1-frank.heimes@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Niklas Schnelle BugLink: https://bugs.launchpad.net/bugs/1943464 In commit 05bc1be6db4b2 ("s390/pci: create zPCI bus") we removed the pci_dev_put() call matching the earlier pci_get_slot() done as part of __zpci_event_availability(). This was based on the wrong understanding that the device_put() done as part of pci_destroy_device() would counter the pci_get_slot() when it only counters the initial reference. This same understanding and existing bad example also lead to not doing a pci_dev_put() in zpci_remove_device(). Since releasing the PCI devices, unlike releasing the PCI slot, does not print any debug message for testing I added one in pci_release_dev(). This revealed that we are indeed leaking the PCI device on PCI hotunplug. Further testing also revealed another missing pci_dev_put() in disable_slot(). Fix this by adding the missing pci_dev_put() in disable_slot() and fix zpci_remove_device() with the correct pci_dev_put() calls. Also instead of calling pci_get_slot() in __zpci_event_availability() to determine if a PCI device is registered and then doing the same again in zpci_remove_device() do this once in zpci_remove_device() which makes sure that the pdev in __zpci_event_availability() is only used for the result of pci_scan_single_device() which does not need a reference count decremnt as its ownership goes to the PCI bus. Also move the check if zdev->zbus->bus is set into zpci_remove_device() since it may be that we're removing a device with devfn != 0 which never had a PCI bus. So we can still set the pdev->error_state to indicate that the device is not usable anymore, add a flag to set the error state. Fixes: 05bc1be6db4b2 ("s390/pci: create zPCI bus") Cc: # 5.8+: e1bff843cde6 s390/pci: remove superfluous zdev->zbus check Cc: # 5.8+: ba764dd703fe s390/pci: refactor zpci_create_device() Cc: # 5.8+ Reviewed-by: Matthew Rosato Signed-off-by: Niklas Schnelle Signed-off-by: Heiko Carstens (backported from commit 0b13525c20febcfecccf6fc1db5969727401317d) Signed-off-by: Frank Heimes --- arch/s390/include/asm/pci.h | 4 ++-- arch/s390/pci/pci.c | 28 ++++++++++++++++++++++++---- arch/s390/pci/pci_event.c | 18 ++++++------------ drivers/pci/hotplug/s390_pci_hpc.c | 3 ++- 4 files changed, 34 insertions(+), 19 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index ea0daab3eb80..5bcac3ac872d 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -201,8 +201,8 @@ extern unsigned int s390_pci_no_rid; Prototypes ----------------------------------------------------------------------------- */ /* Base stuff */ -int zpci_create_device(struct zpci_dev *); -void zpci_remove_device(struct zpci_dev *zdev); +int zpci_create_device(struct zpci_dev *zdev); +void zpci_remove_device(struct zpci_dev *zdev, bool set_error); int zpci_enable_device(struct zpci_dev *); int zpci_disable_device(struct zpci_dev *); int zpci_register_ioat(struct zpci_dev *, u8, u64, u64, u64); diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c index 55b7a8a48940..5e9b6b2185c9 100644 --- a/arch/s390/pci/pci.c +++ b/arch/s390/pci/pci.c @@ -717,16 +717,36 @@ int zpci_disable_device(struct zpci_dev *zdev) } EXPORT_SYMBOL_GPL(zpci_disable_device); -void zpci_remove_device(struct zpci_dev *zdev) +/* zpci_remove_device - Removes the given zdev from the PCI core + * @zdev: the zdev to be removed from the PCI core + * @set_error: if true the device's error state is set to permanent failure + * + * Sets a zPCI device to a configured but offline state; the zPCI + * device is still accessible through its hotplug slot and the zPCI + * API but is removed from the common code PCI bus, making it + * no longer available to drivers. + */ +void zpci_remove_device(struct zpci_dev *zdev, bool set_error) { struct zpci_bus *zbus = zdev->zbus; struct pci_dev *pdev; + if (!zdev->zbus->bus) + return; + pdev = pci_get_slot(zbus->bus, zdev->devfn); if (pdev) { - if (pdev->is_virtfn) - return zpci_remove_virtfn(pdev, zdev->vfn); + if (set_error) + pdev->error_state = pci_channel_io_perm_failure; + if (pdev->is_virtfn) { + zpci_remove_virtfn(pdev, zdev->vfn); + /* balance pci_get_slot */ + pci_dev_put(pdev); + return; + } pci_stop_and_remove_bus_device_locked(pdev); + /* balance pci_get_slot */ + pci_dev_put(pdev); } } @@ -775,7 +795,7 @@ void zpci_release_device(struct kref *kref) struct zpci_dev *zdev = container_of(kref, struct zpci_dev, kref); if (zdev->zbus->bus) - zpci_remove_device(zdev); + zpci_remove_device(zdev, false); switch (zdev->state) { case ZPCI_FN_STATE_ONLINE: diff --git a/arch/s390/pci/pci_event.c b/arch/s390/pci/pci_event.c index 40a569b548cc..c5b7980ff937 100644 --- a/arch/s390/pci/pci_event.c +++ b/arch/s390/pci/pci_event.c @@ -76,13 +76,10 @@ void zpci_event_error(void *data) static void __zpci_event_availability(struct zpci_ccdf_avail *ccdf) { struct zpci_dev *zdev = get_zdev_by_fid(ccdf->fid); - struct pci_dev *pdev = NULL; enum zpci_state state; + struct pci_dev *pdev; int ret; - if (zdev && zdev->zbus && zdev->zbus->bus) - pdev = pci_get_slot(zdev->zbus->bus, zdev->devfn); - zpci_err("avail CCDF:\n"); zpci_err_hex(ccdf, sizeof(*ccdf)); @@ -124,8 +121,7 @@ static void __zpci_event_availability(struct zpci_ccdf_avail *ccdf) case 0x0303: /* Deconfiguration requested */ if (!zdev) break; - if (pdev) - zpci_remove_device(zdev); + zpci_remove_device(zdev, false); ret = zpci_disable_device(zdev); if (ret) @@ -140,12 +136,10 @@ static void __zpci_event_availability(struct zpci_ccdf_avail *ccdf) case 0x0304: /* Configured -> Standby|Reserved */ if (!zdev) break; - if (pdev) { - /* Give the driver a hint that the function is - * already unusable. */ - pdev->error_state = pci_channel_io_perm_failure; - zpci_remove_device(zdev); - } + /* Give the driver a hint that the function is + * already unusable. + */ + zpci_remove_device(zdev, true); zdev->fh = ccdf->fh; zpci_disable_device(zdev); diff --git a/drivers/pci/hotplug/s390_pci_hpc.c b/drivers/pci/hotplug/s390_pci_hpc.c index c9e790c74051..a047c421debe 100644 --- a/drivers/pci/hotplug/s390_pci_hpc.c +++ b/drivers/pci/hotplug/s390_pci_hpc.c @@ -93,8 +93,9 @@ static int disable_slot(struct hotplug_slot *hotplug_slot) pci_dev_put(pdev); return -EBUSY; } + pci_dev_put(pdev); - zpci_remove_device(zdev); + zpci_remove_device(zdev, false); rc = zpci_disable_device(zdev); if (rc) From patchwork Wed Nov 3 13:56:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Heimes X-Patchwork-Id: 1550284 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=sZxL2LxX; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HkpHz6Brsz9sRK for ; Thu, 4 Nov 2021 00:57:51 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1miGls-0000Zr-2i; Wed, 03 Nov 2021 13:57:44 +0000 Received: from smtp-relay-canonical-1.internal ([10.131.114.174] helo=smtp-relay-canonical-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1miGlo-0000Yz-JS for kernel-team@lists.ubuntu.com; Wed, 03 Nov 2021 13:57:40 +0000 Received: from T570.fritz.box (p54abb560.dip0.t-ipconnect.de [84.171.181.96]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id 4ED0D3F168 for ; Wed, 3 Nov 2021 13:57:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1635947860; bh=wD7uw/I3Sok9L1STEHCaNBtwI6V+Ba1Si9K2CIhxQLk=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=sZxL2LxXdrVHDfb8ETGS2WR9c/789FQDdTHseMsKVj7LADhGiOCYmicSITgzzCpzv FqpmdLNt7p7hHrCkO/XfCv3t8q0kuXAlRYdDW3KTs5fE4/wa0q8o5EHnDp56fYyCeH t8EDxA+1QWvsSQPf/33ar7IzgNfKvAW6bJMPiv2DR7Yi6NeG5a+3VE/DEpPQysImil DrzJMShnVX/DwreUu4YyQ7uSOsYXJUWSWIDQIL8lZH5dHdEM38PhUeI4WwEA3i/a6n 4Nqq9qpLnCwA+z20ExfGz3IKm2Kw/fR3oCKgGNk6VsXWH6UGnaQGFBtEHnVUDjyxj6 KHhpeQACtxmpQ== From: frank.heimes@canonical.com To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 2/3] s390/pci: fix use after free of zpci_dev Date: Wed, 3 Nov 2021 14:56:36 +0100 Message-Id: <20211103135637.2220305-3-frank.heimes@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211103135637.2220305-1-frank.heimes@canonical.com> References: <20211103135637.2220305-1-frank.heimes@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Niklas Schnelle BugLink: https://bugs.launchpad.net/bugs/1943464 The struct pci_dev uses reference counting but zPCI assumed erroneously that the last reference would always be the local reference after calling pci_stop_and_remove_bus_device(). This is usually the case but not how reference counting works and thus inherently fragile. In fact one case where this causes a NULL pointer dereference when on an SRIOV device the function 0 was hot unplugged before another function of the same multi-function device. In this case the second function's pdev->sriov->dev reference keeps the struct pci_dev of function 0 alive even after the unplug. This bug was previously hidden by the fact that we were leaking the struct pci_dev which in turn means that it always outlived the struct zpci_dev. This was fixed in commit 0b13525c20fe ("s390/pci: fix leak of PCI device structure") exposing the broken behavior. Fix this by accounting for the long living reference a struct pci_dev has to its underlying struct zpci_dev via the zbus->function[] array and only release that in pcibios_release_device() ensuring that the struct pci_dev is not left with a dangling reference. This is a minimal fix in the future it would probably better to use fine grained reference counting for struct zpci_dev. Fixes: 05bc1be6db4b2 ("s390/pci: create zPCI bus") Cc: stable@vger.kernel.org Reviewed-by: Matthew Rosato Signed-off-by: Niklas Schnelle Signed-off-by: Vasily Gorbik (backported from commit 2a671f77ee49f3e78997b77fdee139467ff6a598) Signed-off-by: Frank Heimes --- arch/s390/pci/pci.c | 6 ++++++ arch/s390/pci/pci_bus.h | 5 +++++ 2 files changed, 11 insertions(+) diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c index 5e9b6b2185c9..5e2fe6d682b7 100644 --- a/arch/s390/pci/pci.c +++ b/arch/s390/pci/pci.c @@ -550,9 +550,12 @@ static void zpci_cleanup_bus_resources(struct zpci_dev *zdev) int pcibios_add_device(struct pci_dev *pdev) { + struct zpci_dev *zdev = to_zpci(pdev); struct resource *res; int i; + /* The pdev has a reference to the zdev via its bus */ + zpci_zdev_get(zdev); if (pdev->is_physfn) pdev->no_vf_scan = 1; @@ -572,7 +575,10 @@ int pcibios_add_device(struct pci_dev *pdev) void pcibios_release_device(struct pci_dev *pdev) { + struct zpci_dev *zdev = to_zpci(pdev); + zpci_unmap_resources(pdev); + zpci_zdev_put(zdev); } int pcibios_enable_device(struct pci_dev *pdev, int mask) diff --git a/arch/s390/pci/pci_bus.h b/arch/s390/pci/pci_bus.h index 4972433df458..8808ff0835b8 100644 --- a/arch/s390/pci/pci_bus.h +++ b/arch/s390/pci/pci_bus.h @@ -17,6 +17,11 @@ static inline void zpci_zdev_put(struct zpci_dev *zdev) kref_put(&zdev->kref, zpci_release_device); } +static inline void zpci_zdev_get(struct zpci_dev *zdev) +{ + kref_get(&zdev->kref); +} + int zpci_alloc_domain(int domain); void zpci_free_domain(int domain); int zpci_setup_bus_resources(struct zpci_dev *zdev, From patchwork Wed Nov 3 13:56:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Heimes X-Patchwork-Id: 1550286 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=S+5xsENz; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HkpJ01gKJz9sfG for ; Thu, 4 Nov 2021 00:57:51 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1miGls-0000a6-9q; Wed, 03 Nov 2021 13:57:44 +0000 Received: from smtp-relay-canonical-1.internal ([10.131.114.174] helo=smtp-relay-canonical-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1miGlo-0000Z6-Tn for kernel-team@lists.ubuntu.com; Wed, 03 Nov 2021 13:57:40 +0000 Received: from T570.fritz.box (p54abb560.dip0.t-ipconnect.de [84.171.181.96]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id 9501D3F1CD for ; Wed, 3 Nov 2021 13:57:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1635947860; bh=HsztNUQzeHYpN0vNMmpWVD4vMg1Kx83lNee60UDm/Ro=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=S+5xsENzhYsPCfamFAijddUr0C4qpxxIII/a4VNZvA9/JD7L6baiV75pA3BD4LgCq 83rTFYiB3mlBL9vocN3USHZplI17hDkLbFvANJFcEmjyhnX83jfLG2niukz5Ou6bVA bVKYUr1eKmxhK/nYzfkIlY1ssdVSf7e36jDA3+9XR5EUF+pDA32zPiPwQlbwQLMvZu 54o/wl2NIBYvahZHLoSAN0s38uKIRszfdzX2xodIKTt7DkGXhQa5qehCH1vlDdaPBx BfyAKisscNf0l5BkOHlcBSjbntNeovBU4avw+l8nz8sXUVwTHXNj3K5ZvKYKFRs5KH 7N4IRvtrSOdZg== From: frank.heimes@canonical.com To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 3/3] s390/pci: fix zpci_zdev_put() on reserve Date: Wed, 3 Nov 2021 14:56:37 +0100 Message-Id: <20211103135637.2220305-4-frank.heimes@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211103135637.2220305-1-frank.heimes@canonical.com> References: <20211103135637.2220305-1-frank.heimes@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Niklas Schnelle BugLink: https://bugs.launchpad.net/bugs/1943464 Since commit 2a671f77ee49 ("s390/pci: fix use after free of zpci_dev") the reference count of a zpci_dev is incremented between pcibios_add_device() and pcibios_release_device() which was supposed to prevent the zpci_dev from being freed while the common PCI code has access to it. It was missed however that the handling of zPCI availability events assumed that once zpci_zdev_put() was called no later availability event would still see the device. With the previously mentioned commit however this assumption no longer holds and we must make sure that we only drop the initial long-lived reference the zPCI subsystem holds exactly once. Do so by introducing a zpci_device_reserved() function that handles when a device is reserved. Here we make sure the zpci_dev will not be considered for further events by removing it from the zpci_list. This also means that the device actually stays in the ZPCI_FN_STATE_RESERVED state between the time we know it has been reserved and the final reference going away. We thus need to consider it a real state instead of just a conceptual state after the removal. The final cleanup of PCI resources, removal from zbus, and destruction of the IOMMU stays in zpci_release_device() to make sure holders of the reference do see valid data until the release. Fixes: 2a671f77ee49 ("s390/pci: fix use after free of zpci_dev") Cc: stable@vger.kernel.org Signed-off-by: Niklas Schnelle Signed-off-by: Vasily Gorbik (backported from commit a46044a92add6a400f4dada7b943b30221f7cc80) Signed-off-by: Frank Heimes --- arch/s390/include/asm/pci.h | 3 ++ arch/s390/pci/pci.c | 45 ++++++++++++++++++++++++++---- arch/s390/pci/pci_event.c | 4 +-- drivers/pci/hotplug/s390_pci_hpc.c | 9 +----- 4 files changed, 46 insertions(+), 15 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 5bcac3ac872d..74177c11611a 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -205,6 +205,9 @@ int zpci_create_device(struct zpci_dev *zdev); void zpci_remove_device(struct zpci_dev *zdev, bool set_error); int zpci_enable_device(struct zpci_dev *); int zpci_disable_device(struct zpci_dev *); +void zpci_device_reserved(struct zpci_dev *zdev); +bool zpci_is_device_configured(struct zpci_dev *zdev); + int zpci_register_ioat(struct zpci_dev *, u8, u64, u64, u64); int zpci_unregister_ioat(struct zpci_dev *, u8); void zpci_remove_reserved_devices(void); diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c index 5e2fe6d682b7..563c9138d876 100644 --- a/arch/s390/pci/pci.c +++ b/arch/s390/pci/pci.c @@ -90,7 +90,7 @@ void zpci_remove_reserved_devices(void) spin_unlock(&zpci_list_lock); list_for_each_entry_safe(zdev, tmp, &remove, entry) - zpci_zdev_put(zdev); + zpci_device_reserved(zdev); } int pci_domain_nr(struct pci_bus *bus) @@ -723,6 +723,14 @@ int zpci_disable_device(struct zpci_dev *zdev) } EXPORT_SYMBOL_GPL(zpci_disable_device); +bool zpci_is_device_configured(struct zpci_dev *zdev) +{ + enum zpci_state state = zdev->state; + + return state != ZPCI_FN_STATE_RESERVED && + state != ZPCI_FN_STATE_STANDBY; +} + /* zpci_remove_device - Removes the given zdev from the PCI core * @zdev: the zdev to be removed from the PCI core * @set_error: if true the device's error state is set to permanent failure @@ -796,6 +804,31 @@ int zpci_create_device(struct zpci_dev *zdev) return rc; } +/** + * zpci_device_reserved() - Mark device as resverved + * @zdev: the zpci_dev that was reserved + * + * Handle the case that a given zPCI function was reserved by another system. + * After a call to this function the zpci_dev can not be found via + * get_zdev_by_fid() anymore but may still be accessible via existing + * references though it will not be functional anymore. + */ +void zpci_device_reserved(struct zpci_dev *zdev) +{ + if (zdev->has_hp_slot) + zpci_exit_slot(zdev); + /* + * Remove device from zpci_list as it is going away. This also + * makes sure we ignore subsequent zPCI events for this device. + */ + spin_lock(&zpci_list_lock); + list_del(&zdev->entry); + spin_unlock(&zpci_list_lock); + zdev->state = ZPCI_FN_STATE_RESERVED; + zpci_dbg(3, "rsv fid:%x\n", zdev->fid); + zpci_zdev_put(zdev); +} + void zpci_release_device(struct kref *kref) { struct zpci_dev *zdev = container_of(kref, struct zpci_dev, kref); @@ -811,6 +844,12 @@ void zpci_release_device(struct kref *kref) case ZPCI_FN_STATE_STANDBY: if (zdev->has_hp_slot) zpci_exit_slot(zdev); + spin_lock(&zpci_list_lock); + list_del(&zdev->entry); + spin_unlock(&zpci_list_lock); + zpci_dbg(3, "rsv fid:%x\n", zdev->fid); + fallthrough; + case ZPCI_FN_STATE_RESERVED: zpci_cleanup_bus_resources(zdev); zpci_bus_device_unregister(zdev); zpci_destroy_iommu(zdev); @@ -818,10 +857,6 @@ void zpci_release_device(struct kref *kref) default: break; } - - spin_lock(&zpci_list_lock); - list_del(&zdev->entry); - spin_unlock(&zpci_list_lock); zpci_dbg(3, "rem fid:%x\n", zdev->fid); kfree(zdev); } diff --git a/arch/s390/pci/pci_event.c b/arch/s390/pci/pci_event.c index c5b7980ff937..bdcf288cf8cc 100644 --- a/arch/s390/pci/pci_event.c +++ b/arch/s390/pci/pci_event.c @@ -146,7 +146,7 @@ static void __zpci_event_availability(struct zpci_ccdf_avail *ccdf) zdev->state = ZPCI_FN_STATE_STANDBY; if (!clp_get_state(ccdf->fid, &state) && state == ZPCI_FN_STATE_RESERVED) { - zpci_zdev_put(zdev); + zpci_device_reserved(zdev); } break; case 0x0306: /* 0x308 or 0x302 for multiple devices */ @@ -155,7 +155,7 @@ static void __zpci_event_availability(struct zpci_ccdf_avail *ccdf) case 0x0308: /* Standby -> Reserved */ if (!zdev) break; - zpci_zdev_put(zdev); + zpci_device_reserved(zdev); break; default: break; diff --git a/drivers/pci/hotplug/s390_pci_hpc.c b/drivers/pci/hotplug/s390_pci_hpc.c index a047c421debe..93174f503464 100644 --- a/drivers/pci/hotplug/s390_pci_hpc.c +++ b/drivers/pci/hotplug/s390_pci_hpc.c @@ -109,14 +109,7 @@ static int get_power_status(struct hotplug_slot *hotplug_slot, u8 *value) struct zpci_dev *zdev = container_of(hotplug_slot, struct zpci_dev, hotplug_slot); - switch (zdev->state) { - case ZPCI_FN_STATE_STANDBY: - *value = 0; - break; - default: - *value = 1; - break; - } + *value = zpci_is_device_configured(zdev) ? 1 : 0; return 0; }