From patchwork Tue Aug 9 18:00:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: William McVicker X-Patchwork-Id: 1665025 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20210112 header.b=LT0t1bsN; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4M2LWD3Rzgz9s5W for ; Wed, 10 Aug 2022 04:02:16 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245667AbiHISCN (ORCPT ); Tue, 9 Aug 2022 14:02:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39258 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344368AbiHISBr (ORCPT ); Tue, 9 Aug 2022 14:01:47 -0400 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 837CD26AF4 for ; Tue, 9 Aug 2022 11:01:03 -0700 (PDT) Received: by mail-pg1-x549.google.com with SMTP id r74-20020a632b4d000000b0041bc393913eso4695818pgr.10 for ; Tue, 09 Aug 2022 11:01:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:from:to:cc; bh=JMRo2ixMHb3JOzogb+sGCIpZ2H1yxkZ6X1ZhcAfiIwI=; b=LT0t1bsNi8ymDwfrg5OhBBNiiN6VxO9/fwn7AvNHMR/6BvRV9oxpUACqkfuuJu5N07 JwWFYys2GuTgOEr3E7lIRNpoQ6NyHZFOZxsupWY+Y1Yr0XRfNylilp/WjbfFgu6T30E4 l2HD4L+hAzGNBKuezkfRGtO1GGi2Pjgk0Hg2kDN5c4U1otuRHvXorqAcsqsrW5CBXtI7 3/Krs46z/tZ622sMN7iocrknfmHuHLF8ci5dTAVb4E6VpTzsrTR/ncbGmlo/BX8kQYRK n76LkKXT29nExez07OJW/SsqIMRyYJMpkzei6bpllpoKHHNqEj95xWjZLLTnXFdvHbDP bVAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:x-gm-message-state:from:to:cc; bh=JMRo2ixMHb3JOzogb+sGCIpZ2H1yxkZ6X1ZhcAfiIwI=; b=JLq6q9cmRIY4U8DFcFmJV2y8m77tBSuRAqlWD5xbyuOeEA5ES6to2MGNByuXzeEG4d d+4rTgQ1XrLnm/7mJyWRJOGAE6VarAot8e8h8OLqSSFtsfzfHd9sYtSjGmwQVWNYSevj 1fKVEgo1j+eYA2AGO/PHSn8OT1ONr7GNkt7VmJhErYZvNxVDBbkvawhPXQmmM2XhtnrC vV/DuofIEZ6m08funFEOVH9riDKq/t6kF60fGYbZHgOu3hCrSogNqJCqaz+joAQpt0Or LEkwH+51h89bq4HEASOG1Vzt0VNsBybHuFQ3vAuCr6KOywwkZZiETFcUizaDeQSyd/MP 87iA== X-Gm-Message-State: ACgBeo3YdRnQ8Drbe/eNp2mrIQNBhNzLGFMW+ENcqGpRHGi1fGA3G/o2 ji9dhEwai8m8Q89bSQQ41VxV+NuySLe7nW+KO3s= X-Google-Smtp-Source: AA6agR7Cc5DUEBvfdyA5rq0jFSyZ+4r/tBcU9vcCcTtzlfHE3q1eOu8BjALWVEzjrnX+XR7J3lm6bhmIuXswNRqDLM4= X-Received: from wmcvicker.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5ebe]) (user=willmcvicker job=sendgmr) by 2002:a17:902:cece:b0:16e:ea30:6287 with SMTP id d14-20020a170902cece00b0016eea306287mr24265895plg.104.1660068062910; Tue, 09 Aug 2022 11:01:02 -0700 (PDT) Date: Tue, 9 Aug 2022 18:00:49 +0000 In-Reply-To: <20220809180051.1063653-1-willmcvicker@google.com> Message-Id: <20220809180051.1063653-2-willmcvicker@google.com> Mime-Version: 1.0 References: <20220809180051.1063653-1-willmcvicker@google.com> X-Mailer: git-send-email 2.37.1.559.g78731f0fdb-goog Subject: [PATCH v1 1/2] PCI: dwc: drop dependency on ZONE_DMA32 From: Will McVicker To: Jingoo Han , Gustavo Pimentel , Lorenzo Pieralisi , Rob Herring , " =?utf-8?q?Krzysztof_Wilczy=C5=84ski?= " , Bjorn Helgaas , Will McVicker Cc: kernel-team@android.com, Vidya Sagar , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-8.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,HK_RANDOM_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Re-work the msi_msg DMA allocation logic to use dma_alloc_coherent() which uses the coherent DMA mask to try and return an allocation within the DMA mask limits. This allows kernel configurations that disable ZONE_DMA32 to continue supporting a 32-bit DMA mask. Without this patch, the PCIe host device will fail to probe when ZONE_DMA32 is disabled. Fixes: 35797e672ff0 ("PCI: dwc: Fix MSI msi_msg DMA mapping") Signed-off-by: Will McVicker --- .../pci/controller/dwc/pcie-designware-host.c | 28 +++++++++---------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c index 7746f94a715f..0cfc3c098f13 100644 --- a/drivers/pci/controller/dwc/pcie-designware-host.c +++ b/drivers/pci/controller/dwc/pcie-designware-host.c @@ -272,9 +272,9 @@ static void dw_pcie_free_msi(struct dw_pcie_rp *pp) struct dw_pcie *pci = to_dw_pcie_from_pp(pp); struct device *dev = pci->dev; - dma_unmap_page(dev, pp->msi_data, PAGE_SIZE, DMA_FROM_DEVICE); - if (pp->msi_page) - __free_page(pp->msi_page); + dma_free_coherent(dev, PAGE_SIZE, pp->msi_page, pp->msi_data); + pp->msi_data = 0; + pp->msi_page = NULL; } } @@ -375,22 +375,22 @@ static int dw_pcie_msi_host_init(struct dw_pcie_rp *pp) dw_chained_msi_isr, pp); } - ret = dma_set_mask(dev, DMA_BIT_MASK(32)); + ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32)); if (ret) dev_warn(dev, "Failed to set DMA mask to 32-bit. Devices with only 32-bit MSI support may not work properly\n"); - pp->msi_page = alloc_page(GFP_DMA32); - pp->msi_data = dma_map_page(dev, pp->msi_page, 0, - PAGE_SIZE, DMA_FROM_DEVICE); - ret = dma_mapping_error(dev, pp->msi_data); - if (ret) { - dev_err(pci->dev, "Failed to map MSI data\n"); - __free_page(pp->msi_page); - pp->msi_page = NULL; + pp->msi_page = dma_alloc_coherent(dev, PAGE_SIZE, &pp->msi_data, + GFP_KERNEL); + if (!pp->msi_page || dma_mapping_error(dev, pp->msi_data)) { + dev_err(dev, "Failed to alloc and map MSI data\n"); + if (pp->msi_page) { + dma_free_coherent(dev, PAGE_SIZE, pp->msi_page, + pp->msi_data); + pp->msi_page = NULL; + } pp->msi_data = 0; dw_pcie_free_msi(pp); - - return ret; + return -ENOMEM; } return 0; From patchwork Tue Aug 9 18:00:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: William McVicker X-Patchwork-Id: 1665026 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20210112 header.b=E0OZstRO; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4M2LWR5PYfz9s5W for ; Wed, 10 Aug 2022 04:02:27 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245075AbiHISCU (ORCPT ); Tue, 9 Aug 2022 14:02:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34306 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344329AbiHISBx (ORCPT ); Tue, 9 Aug 2022 14:01:53 -0400 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 678A4DB5 for ; Tue, 9 Aug 2022 11:01:08 -0700 (PDT) Received: by mail-pj1-x104a.google.com with SMTP id b8-20020a17090a010800b001f1f4fc8178so5091042pjb.8 for ; Tue, 09 Aug 2022 11:01:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:from:to:cc; bh=0naQsNDIr7/tAK4RQoashO+D0OWwjfQP9uh6N2f59FQ=; b=E0OZstROSh47tpxJpKX8HBf3moLKJy6IFfy5NC6Hdzk0TlaJw8ObdAjRoyJkZOub/n 65iscEokmC+ZvKgUCnkydveYOtQ6dpB5plLMkvBOr7wP6ZWHsTfCS6f/gQYq9nN2YXVj MBV/LRFKcVeiqThCoaxjac2rfymZGfHhvJTTe/tVwqOQ+ziqt6HWSOii2XsUBNyOd+Ys cgE3EZHeickYunSaJVvj4WRSHSpRssiPMyL2uHKkwkDjzcJshWYOENq1rVPncvwZzWYX VyyfaLgtIxFWEFRG0HEecXYX3KwnuVm21ycKciBHsAQeGwSdiEYeGlJuwzfrV2uUZ0gO 4Spg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:x-gm-message-state:from:to:cc; bh=0naQsNDIr7/tAK4RQoashO+D0OWwjfQP9uh6N2f59FQ=; b=i2L80hjlgsb3s03nFiYuERV1/h/5yNFjsZFeddx72H4njoZQwzSwunXciG76afSzRS AAb6qLb1u1SwoZ2naa+TYm4YUq0aPqFJW3nZVkGtq1zOHeiVr9Kq9CHz2FMus/L+HMBf IXIqYrSs5+8qnbv7vzUs22u04g8TPRiFsNwcIW05b6ekQkFQfsZNLmQqanCGFFlbllQ1 rYWw4H3Jndk2jBz5icUgT7r4O+PtYjTlOuMDiiFeBmym5WHC99E92Ws6kNPmHuQe7Go4 /kTAm8VesOr+UPsQP9YpVb2IEFdBzHNbpu3h+mbCzjC+2RyeFehI3qAYhezsJUGxS+VO /wsQ== X-Gm-Message-State: ACgBeo0BNQIdtUBIaZ3tHYaefgZhfHD39A19AOG/Da6oTocqyaBwJgsS EVrbmlH5QqjnJHIEFZMqHgNwAK6tQa3wPaI4sS8= X-Google-Smtp-Source: AA6agR48VRt/dvuYwc+vflV/Z+atvVVKkGsHFrM7H1GhF17Pfeyb5NTYc06R9UeQBXtLcDO2b7iYAntU0oxxR0aEKVM= X-Received: from wmcvicker.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5ebe]) (user=willmcvicker job=sendgmr) by 2002:a17:903:234d:b0:16f:3e9:491b with SMTP id c13-20020a170903234d00b0016f03e9491bmr24194019plh.89.1660068067582; Tue, 09 Aug 2022 11:01:07 -0700 (PDT) Date: Tue, 9 Aug 2022 18:00:50 +0000 In-Reply-To: <20220809180051.1063653-1-willmcvicker@google.com> Message-Id: <20220809180051.1063653-3-willmcvicker@google.com> Mime-Version: 1.0 References: <20220809180051.1063653-1-willmcvicker@google.com> X-Mailer: git-send-email 2.37.1.559.g78731f0fdb-goog Subject: [PATCH v1 2/2] PCI: dwc: add support for 64-bit MSI target address From: Will McVicker To: Jingoo Han , Gustavo Pimentel , Lorenzo Pieralisi , Rob Herring , " =?utf-8?q?Krzysztof_Wilczy=C5=84ski?= " , Bjorn Helgaas , Will McVicker Cc: kernel-team@android.com, Vidya Sagar , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-8.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,HK_RANDOM_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Since not all devices require a 32-bit MSI address, add support to the PCIe host driver to allow setting the DMA mask to 64-bits. This allows kernels to disable ZONE_DMA32 and bounce buffering (swiotlb) without risking not being able to get a 32-bit address during DMA allocation. Basically, in the slim chance that there are no 32-bit allocations available, the current PCIe host driver will fail to allocate the msi_msg page due to a DMA address overflow (seen in [1]). With this patch, the PCIe driver can advertise 64-bit support via it's MSI capabilities to hint to the PCIe host driver to set the DMA mask to 64-bits. [1] https://lore.kernel.org/all/Yo0soniFborDl7+C@google.com/ Signed-off-by: Will McVicker Reported-by: kernel test robot Reported-by: kernel test robot --- drivers/pci/controller/dwc/pcie-designware-host.c | 14 ++++++++++++-- drivers/pci/controller/dwc/pcie-designware.c | 9 +++++++++ drivers/pci/controller/dwc/pcie-designware.h | 6 ++++++ 3 files changed, 27 insertions(+), 2 deletions(-) diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c index 0cfc3c098f13..630615719236 100644 --- a/drivers/pci/controller/dwc/pcie-designware-host.c +++ b/drivers/pci/controller/dwc/pcie-designware-host.c @@ -338,6 +338,8 @@ static int dw_pcie_msi_host_init(struct dw_pcie_rp *pp) struct platform_device *pdev = to_platform_device(dev); int ret; u32 ctrl, num_ctrls; + bool msi_64b = false; + u16 msi_capabilities; for (ctrl = 0; ctrl < MAX_MSI_CTRLS; ctrl++) pp->irq_mask[ctrl] = ~0; @@ -375,9 +377,17 @@ static int dw_pcie_msi_host_init(struct dw_pcie_rp *pp) dw_chained_msi_isr, pp); } - ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32)); + msi_capabilities = dw_pcie_msi_capabilities(pci); + if (msi_capabilities & PCI_MSI_FLAGS_ENABLE) + msi_64b = msi_capabilities & PCI_MSI_FLAGS_64BIT ? true : false; + + dev_dbg(dev, "Setting MSI DMA mask to %s-bit.\n", + msi_64b ? "64" : "32"); + ret = dma_set_mask_and_coherent(dev, msi_64b ? + DMA_BIT_MASK(64) : DMA_BIT_MASK(32)); if (ret) - dev_warn(dev, "Failed to set DMA mask to 32-bit. Devices with only 32-bit MSI support may not work properly\n"); + dev_warn(dev, "Failed to set DMA mask to %s-bit.\n", + msi_64b ? "64" : "32"); pp->msi_page = dma_alloc_coherent(dev, PAGE_SIZE, &pp->msi_data, GFP_KERNEL); diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c index c6725c519a47..8ed402307d7f 100644 --- a/drivers/pci/controller/dwc/pcie-designware.c +++ b/drivers/pci/controller/dwc/pcie-designware.c @@ -82,6 +82,15 @@ u8 dw_pcie_find_capability(struct dw_pcie *pci, u8 cap) } EXPORT_SYMBOL_GPL(dw_pcie_find_capability); +u16 dw_pcie_msi_capabilities(struct dw_pcie *pci) +{ + u8 offset; + + offset = dw_pcie_find_capability(pci, PCI_CAP_ID_MSI); + return dw_pcie_readw_dbi(pci, offset + PCI_MSI_FLAGS); +} +EXPORT_SYMBOL_GPL(dw_pcie_msi_capabilities); + static u16 dw_pcie_find_next_ext_capability(struct dw_pcie *pci, u16 start, u8 cap) { diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h index 09b887093a84..70a251c8f72b 100644 --- a/drivers/pci/controller/dwc/pcie-designware.h +++ b/drivers/pci/controller/dwc/pcie-designware.h @@ -432,6 +432,7 @@ void dw_pcie_host_deinit(struct dw_pcie_rp *pp); int dw_pcie_allocate_domains(struct dw_pcie_rp *pp); void __iomem *dw_pcie_own_conf_map_bus(struct pci_bus *bus, unsigned int devfn, int where); +u16 dw_pcie_msi_capabilities(struct dw_pcie *pci); #else static inline irqreturn_t dw_handle_msi_irq(struct dw_pcie_rp *pp) { @@ -462,6 +463,11 @@ static inline void __iomem *dw_pcie_own_conf_map_bus(struct pci_bus *bus, { return NULL; } + +static inline u16 dw_pcie_msi_capabilities(struct dw_pcie *pci) +{ + return 0; +} #endif #ifdef CONFIG_PCIE_DW_EP