From patchwork Wed Jul 13 12:56:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Gardner X-Patchwork-Id: 1655988 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=EIO4vEaC; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Ljd1t6yjtz9s1l for ; Wed, 13 Jul 2022 22:57:22 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1oBbvQ-0006FF-QE; Wed, 13 Jul 2022 12:57:08 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1oBbvP-0006EU-Ep for kernel-team@lists.ubuntu.com; Wed, 13 Jul 2022 12:57:07 +0000 Received: from mail-pj1-f70.google.com (mail-pj1-f70.google.com [209.85.216.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id E513D3F6FC for ; Wed, 13 Jul 2022 12:57:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1657717026; bh=2uLKHH0dBpu+LViN39syt09q4wA3GsczM4zd4sM6ucE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EIO4vEaCxGMytLJZqH1YidLcrXCGcaeoWaXynVooRfW7LAProsAKq4rW/J6qqSHQ9 k7eiyl3uvSMY4h7a1+i93H8IWsmgcztlrnHXduSsITqLKzQXvpeHxfhJHnd/7WL/jG DKki+f05umuNttu2hVWGbBE2jtkyAmMucv8GbITd4Pn07sTin6hC2+voqqjgWQ+I7l eIKAdG5eg/iqNIgaco0q9UaOYXVRF0dmd340GnleqeQ+uTxiPdbupK01Mr9htjBxmK EZWOgekWQAxsCs7iB2Hpti47JYVSmb2NgcnL4efb/7odOPPsMeKkTHCkRnrsqPUbXX ILFzKuTX8IvOA== Received: by mail-pj1-f70.google.com with SMTP id f91-20020a17090a706400b001ef890a44e0so5440140pjk.9 for ; Wed, 13 Jul 2022 05:57:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2uLKHH0dBpu+LViN39syt09q4wA3GsczM4zd4sM6ucE=; b=jmDRkkzb8hZxPhNHdbMjIK28xWBgToZR0z2C3iUPhJRQBWqn/gPVBEzrTisFuHxi3M YuUt5xpYss3RrsYJk+eLwiOjE9+sKn/JcE3V7M2lQQdhDnYj5mXpQBzMo7b8vPmBQz0l nNl0aI4L28ST/fZNj4rk3Z7eKpTQ1Qx/x1gdiA/Cr+3IVeVS69+XrTM1WnZdCRcbpkAu ZnvGmbykLTuW8jhp9Ei568Yx2mMG29aasFAWoxGLJG2oC32YUnM2ZfIae/RiL8ObDCTb sjEYgMB61Q8xk+fdzqFeUdv38I622POZP9yp/0qKRAn5iYzcOsjFdnNAQWJtuJNTXRXp mD9A== X-Gm-Message-State: AJIora/2LsNBc778xvNn5IJK64wXXCPEfhf/HBWNPQIi2UFfrYA49Dfa lmC55yEVCtAiJ6fVV5W6M4TzMv4MDGBWhECYT4jgCx5TkIyE0nrV1aFbXWBkevriCECSbXoSpM8 sa8Ydpl/mcbLBY7sr5793Vb1BjsuAhPOfTfVmIHgtaw== X-Received: by 2002:a17:903:234c:b0:16c:3dba:37fa with SMTP id c12-20020a170903234c00b0016c3dba37famr3100850plh.89.1657717025128; Wed, 13 Jul 2022 05:57:05 -0700 (PDT) X-Google-Smtp-Source: AGRyM1sm3kMGX2Yd1Z4ohT7bk7mCkZrNDli7q733zbA8zazYJFUgQtmscWn+exY5On7jjoxfwojxAQ== X-Received: by 2002:a17:903:234c:b0:16c:3dba:37fa with SMTP id c12-20020a170903234c00b0016c3dba37famr3100839plh.89.1657717024879; Wed, 13 Jul 2022 05:57:04 -0700 (PDT) Received: from localhost.localdomain ([69.163.84.166]) by smtp.gmail.com with ESMTPSA id k91-20020a17090a14e400b001ec798b0dc4sm1551135pja.38.2022.07.13.05.57.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:57:04 -0700 (PDT) From: Tim Gardner To: kernel-team@lists.ubuntu.com Subject: [PATCH 01/11][jammy/linux-azure] genirq/msi: Guard sysfs code Date: Wed, 13 Jul 2022 06:56:39 -0600 Message-Id: <20220713125649.24063-6-tim.gardner@canonical.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713125649.24063-1-tim.gardner@canonical.com> References: <20220713125649.24063-1-tim.gardner@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Thomas Gleixner BugLink: https://bugs.launchpad.net/bugs/1981577 No point in building unused code when CONFIG_SYSFS=n. Signed-off-by: Thomas Gleixner Tested-by: Juergen Gross Reviewed-by: Jason Gunthorpe Reviewed-by: Greg Kroah-Hartman Link: https://lore.kernel.org/r/20211206210223.985907940@linutronix.de (cherry picked from commit 1197528aaea79ed4909aba695d18fdecc5387a36) Signed-off-by: Tim Gardner --- include/linux/msi.h | 10 ++++++++++ kernel/irq/msi.c | 2 ++ 2 files changed, 12 insertions(+) diff --git a/include/linux/msi.h b/include/linux/msi.h index e616f94c7c58..d43b9469c88b 100644 --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -239,9 +239,19 @@ void __pci_write_msi_msg(struct msi_desc *entry, struct msi_msg *msg); void pci_msi_mask_irq(struct irq_data *data); void pci_msi_unmask_irq(struct irq_data *data); +#ifdef CONFIG_SYSFS const struct attribute_group **msi_populate_sysfs(struct device *dev); void msi_destroy_sysfs(struct device *dev, const struct attribute_group **msi_irq_groups); +#else +static inline const struct attribute_group **msi_populate_sysfs(struct device *dev) +{ + return NULL; +} +static inline void msi_destroy_sysfs(struct device *dev, const struct attribute_group **msi_irq_groups) +{ +} +#endif /* * The arch hooks to setup up msi irqs. Default functions are implemented diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c index 7f350ae59c5f..a8a0daeb22f5 100644 --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -72,6 +72,7 @@ void get_cached_msi_msg(unsigned int irq, struct msi_msg *msg) } EXPORT_SYMBOL_GPL(get_cached_msi_msg); +#ifdef CONFIG_SYSFS static ssize_t msi_mode_show(struct device *dev, struct device_attribute *attr, char *buf) { @@ -204,6 +205,7 @@ void msi_destroy_sysfs(struct device *dev, const struct attribute_group **msi_ir kfree(msi_irq_groups); } } +#endif #ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN static inline void irq_chip_write_msi_msg(struct irq_data *data, From patchwork Wed Jul 13 12:56:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Gardner X-Patchwork-Id: 1655987 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=RvGb7H6G; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Ljd1t6Hvkz9s09 for ; Wed, 13 Jul 2022 22:57:22 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1oBbvO-0006Dk-GX; Wed, 13 Jul 2022 12:57:06 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1oBbvM-0006Ac-80 for kernel-team@lists.ubuntu.com; Wed, 13 Jul 2022 12:57:04 +0000 Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id D9DBE3F6FC for ; Wed, 13 Jul 2022 12:57:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1657717023; bh=GOuyPYh3NGNVgV16urBWqz8mWzXAlAMMkjk1HGh8xP8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RvGb7H6GUtnTsbaqutN8mmOigYiAuxIH9BXrWsFnwe8/gpsM/zd2IJJSA86rw2N+g Gu90LWSesOQCvHuwYwt9tW6wH4cNlnJpps/HGI8c1m14Vbsor90Pi1zXR0lCCZMY0G jQ8W7+JUm6Z97hbC2AvPmv15QNhmCEiblrNQmX4gsaaXLDtvhTaArRUpt4Vj7PV71X 1CJBrTjeLMmp+aKtmVI2xYb0Hn0oEW3ux4Cz/IK4wvF4ABSS2pAfDQ1A6wArP88WR+ hkNeLmitpXBVJHdO766V3iaD124/DkA4p5BJgX4hU8sGG70lUEISuZE2+zCR3KUvOA uKzDHyBEZMhZA== Received: by mail-pg1-f197.google.com with SMTP id t37-20020a635365000000b0041965b8ec8eso2643293pgl.0 for ; Wed, 13 Jul 2022 05:57:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GOuyPYh3NGNVgV16urBWqz8mWzXAlAMMkjk1HGh8xP8=; b=JVxDETNMyWNMmw32J0MhTdDbDGB2Sp4OQK7IlN9Mq+3wnoGnZQE/b+mqa4ftJWTGZJ xpHa+g8+A/qu7Pzkb3ra4CLNJOYh3CXJiUO38edId1zDPZUi8GiCxUaZrg/eY8iUsNh3 Z4yOiLJglMr9dJRpRgn0xScaf3z1gEvlS7Rd2Be4Vj6Ft8lbzSFbJOBMgbkliT1/3Psu Z+RPnz6v7aIb28b+FoYhVyJzpe1h9CayLBu37Q09ttMFUefziodgPB63ZL7SN2YUREyw +2NXckRcpHtTxSk+wNQYnAd2APc7gbFPG75tQXxGmyMaJyPAljCjOQeQoKCdiJrhvGTg /LyQ== X-Gm-Message-State: AJIora+9nE1CgdXQOvAZkNYiwj3FF4IXFe1o9QW8HGqlutQOsY7pt4zX Yy6A8TvGggJt4rKB1CIrKyTLYFskqBdiQ+WUecXP61MuQFwPnOQTgGiPVWvov9HFOtdQhqx3+xu hTn486j9ejer/r/CyvMihzcKywx27SVvKciaofp9aIw== X-Received: by 2002:a17:903:1c7:b0:16c:4f00:65df with SMTP id e7-20020a17090301c700b0016c4f0065dfmr3021556plh.158.1657717021554; Wed, 13 Jul 2022 05:57:01 -0700 (PDT) X-Google-Smtp-Source: AGRyM1t/Qqn6WCZVDnSbkxTOEkzwWt/dK2wdJWDM3kAcX+V5Mzc1TukCuzQ2bsVyHm2RSfGsZ3uFEQ== X-Received: by 2002:a17:903:1c7:b0:16c:4f00:65df with SMTP id e7-20020a17090301c700b0016c4f0065dfmr3021532plh.158.1657717021259; Wed, 13 Jul 2022 05:57:01 -0700 (PDT) Received: from localhost.localdomain ([69.163.84.166]) by smtp.gmail.com with ESMTPSA id k91-20020a17090a14e400b001ec798b0dc4sm1551135pja.38.2022.07.13.05.57.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:57:00 -0700 (PDT) From: Tim Gardner To: kernel-team@lists.ubuntu.com Subject: [PATCH 2/4][Focal/linux-azure] PCI: hv: Fix hv_arch_irq_unmask() for multi-MSI Date: Wed, 13 Jul 2022 06:56:36 -0600 Message-Id: <20220713125649.24063-3-tim.gardner@canonical.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713125649.24063-1-tim.gardner@canonical.com> References: <20220713125649.24063-1-tim.gardner@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Jeffrey Hugo BugLink: https://bugs.launchpad.net/bugs/1981577 In the multi-MSI case, hv_arch_irq_unmask() will only operate on the first MSI of the N allocated. This is because only the first msi_desc is cached and it is shared by all the MSIs of the multi-MSI block. This means that hv_arch_irq_unmask() gets the correct address, but the wrong data (always 0). This can break MSIs. Lets assume MSI0 is vector 34 on CPU0, and MSI1 is vector 33 on CPU0. hv_arch_irq_unmask() is called on MSI0. It uses a hypercall to configure the MSI address and data (0) to vector 34 of CPU0. This is correct. Then hv_arch_irq_unmask is called on MSI1. It uses another hypercall to configure the MSI address and data (0) to vector 33 of CPU0. This is wrong, and results in both MSI0 and MSI1 being routed to vector 33. Linux will observe extra instances of MSI1 and no instances of MSI0 despite the endpoint device behaving correctly. For the multi-MSI case, we need unique address and data info for each MSI, but the cached msi_desc does not provide that. However, that information can be gotten from the int_desc cached in the chip_data by compose_msi_msg(). Fix the multi-MSI case to use that cached information instead. Since hv_set_msi_entry_from_desc() is no longer applicable, remove it. Signed-off-by: Jeffrey Hugo Reviewed-by: Michael Kelley Link: https://lore.kernel.org/r/1651068453-29588-1-git-send-email-quic_jhugo@quicinc.com Signed-off-by: Wei Liu (backported from commit 455880dfe292a2bdd3b4ad6a107299fce610e64b) [rtg - hv_arch_irq_unmask() does not yet exist, add changes to hv_irq_unmask() instead. ] Signed-off-by: Tim Gardner --- drivers/pci/controller/pci-hyperv.c | 20 ++++---------------- 1 file changed, 4 insertions(+), 16 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index 7d38cda1cacf..24ef7f76ff30 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -609,13 +609,6 @@ static unsigned int hv_msi_get_int_vector(struct irq_data *data) return cfg->vector; } -static void hv_set_msi_entry_from_desc(union hv_msi_entry *msi_entry, - struct msi_desc *msi_desc) -{ - msi_entry->address.as_uint32 = msi_desc->msg.address_lo; - msi_entry->data.as_uint32 = msi_desc->msg.data; -} - static int hv_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec, msi_alloc_info_t *info) { @@ -665,14 +658,6 @@ static unsigned int hv_msi_get_int_vector(struct irq_data *irqd) return irqd->parent_data->hwirq; } -static void hv_set_msi_entry_from_desc(union hv_msi_entry *msi_entry, - struct msi_desc *msi_desc) -{ - msi_entry->address = ((u64)msi_desc->msg.address_hi << 32) | - msi_desc->msg.address_lo; - msi_entry->data = msi_desc->msg.data; -} - /* * @nr_bm_irqs: Indicates the number of IRQs that were allocated from * the bitmap. @@ -1483,6 +1468,7 @@ static void hv_irq_unmask(struct irq_data *data) { struct msi_desc *msi_desc = irq_data_get_msi_desc(data); struct hv_retarget_device_interrupt *params; + struct tran_int_desc *int_desc; struct hv_pcibus_device *hbus; struct cpumask *dest; cpumask_var_t tmp; @@ -1497,6 +1483,7 @@ static void hv_irq_unmask(struct irq_data *data) pdev = msi_desc_to_pci_dev(msi_desc); pbus = pdev->bus; hbus = container_of(pbus->sysdata, struct hv_pcibus_device, sysdata); + int_desc = data->chip_data; spin_lock_irqsave(&hbus->retarget_msi_interrupt_lock, flags); @@ -1504,7 +1491,8 @@ static void hv_irq_unmask(struct irq_data *data) memset(params, 0, sizeof(*params)); params->partition_id = HV_PARTITION_ID_SELF; params->int_entry.source = HV_INTERRUPT_SOURCE_MSI; - hv_set_msi_entry_from_desc(¶ms->int_entry.msi_entry, msi_desc); + params->int_entry.msi_entry.address.as_uint32 = int_desc->address & 0xffffffff; + params->int_entry.msi_entry.data.as_uint32 = int_desc->data; params->device_id = (hbus->hdev->dev_instance.b[5] << 24) | (hbus->hdev->dev_instance.b[4] << 16) | (hbus->hdev->dev_instance.b[7] << 8) | From patchwork Wed Jul 13 12:56:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Gardner X-Patchwork-Id: 1655991 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=CyZ6Tuxz; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Ljd1w0h7Yz9sFx for ; Wed, 13 Jul 2022 22:57:23 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1oBbvU-0006IA-7y; Wed, 13 Jul 2022 12:57:12 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1oBbvP-0006Eb-PT for kernel-team@lists.ubuntu.com; Wed, 13 Jul 2022 12:57:07 +0000 Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id D928A3F0C5 for ; Wed, 13 Jul 2022 12:57:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1657717025; bh=JBZBA6ykHKxHsqCaNlQRwvjIruOY8I+YUJjOY9fuvfE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=CyZ6Tuxz1issVX/c5EVUZZi1Jg5nfIC3uE5TxZypzydhzsGFUpiyxZ+4kPGivQwUj tEmOIuVcoHXgcmDryJUenF5IbyFt6aAIRA/85KB67EdojRhA0mY8vRwRckSWjf/wcm A0shDzCp72uM5ycPbF2dYFDJ2hux5AYiia24A/30Y4IAvhllDVbb6P1CpOCHHHe+8v iJb9Gc/zh3njiAOYdXNTJFIsBVvpIvGdHL3unxyy+ryZCQI/gVxAtpxPb0zw3Rc0U0 w2SEep15Rdq8nmljO0XRXGKnEqBdesojGzD8+37tKx1CuhTwkmTfAUV+NvI3qQbqy3 eJheuJkaMq+Vg== Received: by mail-pg1-f199.google.com with SMTP id 31-20020a63125f000000b00419a2da53bdso384368pgs.8 for ; Wed, 13 Jul 2022 05:57:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JBZBA6ykHKxHsqCaNlQRwvjIruOY8I+YUJjOY9fuvfE=; b=jxun5DRh2sQJRHJBUQeU2jODTtrPTi90b2nksyeENqtt+dAaNryToE8rsNVPZ2Vmcj QPYbiITCjf1Hgmg8bGr3d5kKXh5f78gG+F8FeWGDKQJGsDcqA0MyvZEsDZwQidr6Qk6U 9NC7x4BrjfU9rPgv+/bfhKXYrE77MyVR4g5ivFnmWxnQ3whJ6ur9EyxjeoBsAbQgt4tO Y86+z0XzNb6khXsyoLVoryqD4kfjbCRCjhID/80UK6CAj+UfBpqcKQ26ozdqFBykeXad UNaTXE46sfcJzX4dM94F4PzMkncauFW9r7eVyq/OIH4Os89hFVhlkgbz1gf0VXEs2fbx V4Jg== X-Gm-Message-State: AJIora+p3WRDUyFBYK0ulYrtQT4lvsO+d1kIoudsTvR7oUtOZimPVh2D wOIObY83EsSXRoHSWrtutvSxK4jPTpUhYfPSYAzG+YUrB/NO10IbjZUtUKw8yTDx+thhpbEULlO i32nC4hC8C/PvVpkEE1MD8fPtzNtdYFpovwcm8V+8ag== X-Received: by 2002:a17:902:cece:b0:16c:3683:8835 with SMTP id d14-20020a170902cece00b0016c36838835mr3300502plg.104.1657717023016; Wed, 13 Jul 2022 05:57:03 -0700 (PDT) X-Google-Smtp-Source: AGRyM1sqLWSh1RhAjwHJWjGjegpBj3SXggsMBMPxshkIdY3cngIdPn4hswBEv5+eFZAkXTLzKcbvQg== X-Received: by 2002:a17:902:cece:b0:16c:3683:8835 with SMTP id d14-20020a170902cece00b0016c36838835mr3300482plg.104.1657717022756; Wed, 13 Jul 2022 05:57:02 -0700 (PDT) Received: from localhost.localdomain ([69.163.84.166]) by smtp.gmail.com with ESMTPSA id k91-20020a17090a14e400b001ec798b0dc4sm1551135pja.38.2022.07.13.05.57.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:57:01 -0700 (PDT) From: Tim Gardner To: kernel-team@lists.ubuntu.com Subject: [PATCH 3/4][Focal/linux-azure] PCI: hv: Reuse existing IRTE allocation in compose_msi_msg() Date: Wed, 13 Jul 2022 06:56:37 -0600 Message-Id: <20220713125649.24063-4-tim.gardner@canonical.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713125649.24063-1-tim.gardner@canonical.com> References: <20220713125649.24063-1-tim.gardner@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Jeffrey Hugo BugLink: https://bugs.launchpad.net/bugs/1981577 Currently if compose_msi_msg() is called multiple times, it will free any previous IRTE allocation, and generate a new allocation. While nothing prevents this from occurring, it is extraneous when Linux could just reuse the existing allocation and avoid a bunch of overhead. However, when future IRTE allocations operate on blocks of MSIs instead of a single line, freeing the allocation will impact all of the lines. This could cause an issue where an allocation of N MSIs occurs, then some of the lines are retargeted, and finally the allocation is freed/reallocated. The freeing of the allocation removes all of the configuration for the entire block, which requires all the lines to be retargeted, which might not happen since some lines might already be unmasked/active. Signed-off-by: Jeffrey Hugo Reviewed-by: Dexuan Cui Tested-by: Dexuan Cui Tested-by: Michael Kelley Link: https://lore.kernel.org/r/1652282582-21595-1-git-send-email-quic_jhugo@quicinc.com Signed-off-by: Wei Liu (cherry picked from commit b4b77778ecc5bfbd4e77de1b2fd5c1dd3c655f1f) Signed-off-by: Tim Gardner --- drivers/pci/controller/pci-hyperv.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index 24ef7f76ff30..04165db6d385 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -1693,6 +1693,15 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) u32 size; int ret; + /* Reuse the previous allocation */ + if (data->chip_data) { + int_desc = data->chip_data; + msg->address_hi = int_desc->address >> 32; + msg->address_lo = int_desc->address & 0xffffffff; + msg->data = int_desc->data; + return; + } + pdev = msi_desc_to_pci_dev(irq_data_get_msi_desc(data)); dest = irq_data_get_effective_affinity_mask(data); pbus = pdev->bus; @@ -1702,13 +1711,6 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) if (!hpdev) goto return_null_message; - /* Free any previous message that might have already been composed. */ - if (data->chip_data) { - int_desc = data->chip_data; - data->chip_data = NULL; - hv_int_desc_free(hpdev, int_desc); - } - int_desc = kzalloc(sizeof(*int_desc), GFP_ATOMIC); if (!int_desc) goto drop_reference; From patchwork Wed Jul 13 12:56:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Gardner X-Patchwork-Id: 1655994 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=GOetHovz; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Ljd1y3q1xz9s1l for ; Wed, 13 Jul 2022 22:57:26 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1oBbvU-0006IV-JG; Wed, 13 Jul 2022 12:57:12 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1oBbvQ-0006Ec-Cd for kernel-team@lists.ubuntu.com; Wed, 13 Jul 2022 12:57:08 +0000 Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 663883F0CE for ; Wed, 13 Jul 2022 12:57:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1657717026; bh=EFJNdM5m4nLqybBJRx084hHrsmw9Q30vFGuwV2m1Aaw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GOetHovzeW88Q39uxZ23U9d8J6CFSJjk7eCYk0gjT+XLCSQXlrwBv0AjNKdl/51Dl PSWYktNAanQtkdxTAuw7lUhxVDh2XZx5wuYHiH7caBfwHThWB0RRN9J5aYXjh/Q9at eh33vqYBg8q7O7WOzyLeY+z87oackkBHXIAVsZxf5s/EuJEXhno8OhppdnmUWj/bo4 s758y+g2n/J4weIoGRwHeDpODtyGpP52GBIbns0yxhbPv6uznuE8K6Hn+3wKCkeo4c b+0KHcFKWap7Q9rLC/8rw6iUJ3R4/ka+0NONIBtZca4PeoHJHzLB+1Arlp/Aw5gPPh Ainlr6VJlAW3w== Received: by mail-pf1-f198.google.com with SMTP id c21-20020a624e15000000b0052abf43401fso3871567pfb.14 for ; Wed, 13 Jul 2022 05:57:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EFJNdM5m4nLqybBJRx084hHrsmw9Q30vFGuwV2m1Aaw=; b=4p2ilskz3pyy849+9HxROq6I64UPpxajfMadndH6aGB3cq/t/TJFGZJ284nBc+UG8s z0UVUUeV5xBheFXOo/2O6z3HSS37wRIffRSmRBJ7u8W32AUKehVnNnX9CDBHYEgipdmG noTzsX0wYm7m1Soi2GoYESmj6ZO4Yb/8h4wou4RMsSnjWyoNErLMes9Y8NzWxJvkqkH9 qoBGimq3hIhVOYXlfF+/p4P2ZkJwxqUhDpozqlediWphz0o24MvoORS/GnNYcSW5XdNm a8eV7hoEH1QY6Ytk5eKTmKXXhk1j0WZwzofvodNB+5t1DvBRh9eJMn/BejJBP4Q0LVFj ll3w== X-Gm-Message-State: AJIora8WkXHywkh+p3xvxoSsoHnIVyzvYsOUCqTnwijDXjD1s6qLZ2dC 5ap3pzRsSNUaT9fBWTw4mWUGCEsY8gfCYpFFDfhQ78DTj5UQwKfTAG6BKTppjbqXcRxOEPZ8Fcg 2qBh/HgUX+t1px0OvgBOeotY62Y6uZlm4PWZ9I6xStg== X-Received: by 2002:a17:902:9348:b0:167:8e92:272f with SMTP id g8-20020a170902934800b001678e92272fmr2977051plp.77.1657717024315; Wed, 13 Jul 2022 05:57:04 -0700 (PDT) X-Google-Smtp-Source: AGRyM1t/JIxz7Fm4ixwu5/4UJmdeuiqRF4mUhnaLrsKgDB58gmC6mQPVhxixZmp8DtWlmtELWGAVjw== X-Received: by 2002:a17:902:9348:b0:167:8e92:272f with SMTP id g8-20020a170902934800b001678e92272fmr2977033plp.77.1657717023933; Wed, 13 Jul 2022 05:57:03 -0700 (PDT) Received: from localhost.localdomain ([69.163.84.166]) by smtp.gmail.com with ESMTPSA id k91-20020a17090a14e400b001ec798b0dc4sm1551135pja.38.2022.07.13.05.57.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:57:03 -0700 (PDT) From: Tim Gardner To: kernel-team@lists.ubuntu.com Subject: [PATCH 4/4][Focal/linux-azure] PCI: hv: Fix interrupt mapping for multi-MSI Date: Wed, 13 Jul 2022 06:56:38 -0600 Message-Id: <20220713125649.24063-5-tim.gardner@canonical.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713125649.24063-1-tim.gardner@canonical.com> References: <20220713125649.24063-1-tim.gardner@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Jeffrey Hugo BugLink: https://bugs.launchpad.net/bugs/1981577 According to Dexuan, the hypervisor folks beleive that multi-msi allocations are not correct. compose_msi_msg() will allocate multi-msi one by one. However, multi-msi is a block of related MSIs, with alignment requirements. In order for the hypervisor to allocate properly aligned and consecutive entries in the IOMMU Interrupt Remapping Table, there should be a single mapping request that requests all of the multi-msi vectors in one shot. Dexuan suggests detecting the multi-msi case and composing a single request related to the first MSI. Then for the other MSIs in the same block, use the cached information. This appears to be viable, so do it. Suggested-by: Dexuan Cui Signed-off-by: Jeffrey Hugo Reviewed-by: Dexuan Cui Tested-by: Michael Kelley Link: https://lore.kernel.org/r/1652282599-21643-1-git-send-email-quic_jhugo@quicinc.com Signed-off-by: Wei Liu (backported from commit a2bad844a67b1c7740bda63e87453baf63c3a7f7) [rtg - msi_desc attributes are in a different place in the structure ] Signed-off-by: Tim Gardner --- drivers/pci/controller/pci-hyperv.c | 67 ++++++++++++++++++++++++----- 1 file changed, 57 insertions(+), 10 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index 04165db6d385..6ce774de3525 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -1398,6 +1398,10 @@ static void hv_int_desc_free(struct hv_pci_dev *hpdev, u8 buffer[sizeof(struct pci_delete_interrupt)]; } ctxt; + if (!int_desc->vector_count) { + kfree(int_desc); + return; + } memset(&ctxt, 0, sizeof(ctxt)); int_pkt = (struct pci_delete_interrupt *)&ctxt.pkt.message; int_pkt->message_type.type = @@ -1491,8 +1495,15 @@ static void hv_irq_unmask(struct irq_data *data) memset(params, 0, sizeof(*params)); params->partition_id = HV_PARTITION_ID_SELF; params->int_entry.source = HV_INTERRUPT_SOURCE_MSI; +#ifdef CONFIG_X86 params->int_entry.msi_entry.address.as_uint32 = int_desc->address & 0xffffffff; params->int_entry.msi_entry.data.as_uint32 = int_desc->data; +#elif CONFIG_ARM64 + params->int_entry.msi_entry.address = int_desc->address & 0xffffffff; + params->int_entry.msi_entry.data = int_desc->data; +#else +#error Unsupported architecture +#endif params->device_id = (hbus->hdev->dev_instance.b[5] << 24) | (hbus->hdev->dev_instance.b[4] << 16) | (hbus->hdev->dev_instance.b[7] << 8) | @@ -1595,12 +1606,12 @@ static void hv_pci_compose_compl(void *context, struct pci_response *resp, static u32 hv_compose_msi_req_v1( struct pci_create_interrupt *int_pkt, struct cpumask *affinity, - u32 slot, u8 vector) + u32 slot, u8 vector, u8 vector_count) { int_pkt->message_type.type = PCI_CREATE_INTERRUPT_MESSAGE; int_pkt->wslot.slot = slot; int_pkt->int_desc.vector = vector; - int_pkt->int_desc.vector_count = 1; + int_pkt->int_desc.vector_count = vector_count; int_pkt->int_desc.delivery_mode = DELIVERY_MODE; /* @@ -1623,14 +1634,14 @@ static int hv_compose_msi_req_get_cpu(struct cpumask *affinity) static u32 hv_compose_msi_req_v2( struct pci_create_interrupt2 *int_pkt, struct cpumask *affinity, - u32 slot, u8 vector) + u32 slot, u8 vector, u8 vector_count) { int cpu; int_pkt->message_type.type = PCI_CREATE_INTERRUPT_MESSAGE2; int_pkt->wslot.slot = slot; int_pkt->int_desc.vector = vector; - int_pkt->int_desc.vector_count = 1; + int_pkt->int_desc.vector_count = vector_count; int_pkt->int_desc.delivery_mode = DELIVERY_MODE; cpu = hv_compose_msi_req_get_cpu(affinity); int_pkt->int_desc.processor_array[0] = @@ -1642,7 +1653,7 @@ static u32 hv_compose_msi_req_v2( static u32 hv_compose_msi_req_v3( struct pci_create_interrupt3 *int_pkt, struct cpumask *affinity, - u32 slot, u32 vector) + u32 slot, u32 vector, u8 vector_count) { int cpu; @@ -1650,7 +1661,7 @@ static u32 hv_compose_msi_req_v3( int_pkt->wslot.slot = slot; int_pkt->int_desc.vector = vector; int_pkt->int_desc.reserved = 0; - int_pkt->int_desc.vector_count = 1; + int_pkt->int_desc.vector_count = vector_count; int_pkt->int_desc.delivery_mode = DELIVERY_MODE; cpu = hv_compose_msi_req_get_cpu(affinity); int_pkt->int_desc.processor_array[0] = @@ -1681,6 +1692,8 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) struct cpumask *dest; struct compose_comp_ctxt comp; struct tran_int_desc *int_desc; + struct msi_desc *msi_desc; + u8 vector, vector_count; struct { struct pci_packet pci_pkt; union { @@ -1702,7 +1715,8 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) return; } - pdev = msi_desc_to_pci_dev(irq_data_get_msi_desc(data)); + msi_desc = irq_data_get_msi_desc(data); + pdev = msi_desc_to_pci_dev(msi_desc); dest = irq_data_get_effective_affinity_mask(data); pbus = pdev->bus; hbus = container_of(pbus->sysdata, struct hv_pcibus_device, sysdata); @@ -1715,6 +1729,36 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) if (!int_desc) goto drop_reference; + if (!msi_desc->msi_attrib.is_msix && msi_desc->nvec_used > 1) { + /* + * If this is not the first MSI of Multi MSI, we already have + * a mapping. Can exit early. + */ + if (msi_desc->irq != data->irq) { + data->chip_data = int_desc; + int_desc->address = msi_desc->msg.address_lo | + (u64)msi_desc->msg.address_hi << 32; + int_desc->data = msi_desc->msg.data + + (data->irq - msi_desc->irq); + msg->address_hi = msi_desc->msg.address_hi; + msg->address_lo = msi_desc->msg.address_lo; + msg->data = int_desc->data; + put_pcichild(hpdev); + return; + } + /* + * The vector we select here is a dummy value. The correct + * value gets sent to the hypervisor in unmask(). This needs + * to be aligned with the count, and also not zero. Multi-msi + * is powers of 2 up to 32, so 32 will always work here. + */ + vector = 32; + vector_count = msi_desc->nvec_used; + } else { + vector = hv_msi_get_int_vector(data); + vector_count = 1; + } + memset(&ctxt, 0, sizeof(ctxt)); init_completion(&comp.comp_pkt.host_event); ctxt.pci_pkt.completion_func = hv_pci_compose_compl; @@ -1725,7 +1769,8 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) size = hv_compose_msi_req_v1(&ctxt.int_pkts.v1, dest, hpdev->desc.win_slot.slot, - hv_msi_get_int_vector(data)); + vector, + vector_count); break; case PCI_PROTOCOL_VERSION_1_2: @@ -1733,14 +1778,16 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) size = hv_compose_msi_req_v2(&ctxt.int_pkts.v2, dest, hpdev->desc.win_slot.slot, - hv_msi_get_int_vector(data)); + vector, + vector_count); break; case PCI_PROTOCOL_VERSION_1_4: size = hv_compose_msi_req_v3(&ctxt.int_pkts.v3, dest, hpdev->desc.win_slot.slot, - hv_msi_get_int_vector(data)); + vector, + vector_count); break; default: From patchwork Wed Jul 13 12:56:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Gardner X-Patchwork-Id: 1655995 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=W5nXJAB8; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Ljd246H8lz9ryY for ; Wed, 13 Jul 2022 22:57:32 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1oBbvZ-0006O6-BO; Wed, 13 Jul 2022 12:57:17 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1oBbvS-0006GY-Nl for kernel-team@lists.ubuntu.com; Wed, 13 Jul 2022 12:57:10 +0000 Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id DFBA63F6FC for ; Wed, 13 Jul 2022 12:57:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1657717029; bh=p+inD7LrHLhwqB+b9ZnGazDhOuCSLtaEbH4SwSmCyCM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=W5nXJAB8hh/NNk0Zn+HlveDw1s1Ziv4KbuNPpM7q1vXcRVhRIEeXA4zt1MCxAKSRb +8muV08ZH5vhx97weKEkuD3xV1hSsByCmk++0s18X/zO4nQWzdpfb/KYytkuCzm/4J t68PnSiGqM03CdEPc+DKdE5BEynEaMINUtvRoqR5qU0RnstKzcAfc2uFSAWoCIXa1y dqrPAHz/SCpMS7SAQLrZjTO5gKRMFsgiYsh+mllgCS2f5QmsNWNQGwxm2q22DreThk RtJt+hYkr2TC9UPwgBlLVHpr3yg4YGphn08J5hHMrAHKy9/u42bMqSJ9LlxjFqSu+U r7k7ne9rP/0EQ== Received: by mail-pf1-f199.google.com with SMTP id r17-20020a056a00217100b0052ab8271e11so3985484pff.22 for ; Wed, 13 Jul 2022 05:57:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=p+inD7LrHLhwqB+b9ZnGazDhOuCSLtaEbH4SwSmCyCM=; b=nxmaOjxNiDQem6G7fp20eubI5g9O5tMtk0n/ziiY3IVXWTWAW5HQzLF1BVE0uMOtTJ W91i/JIb7Nf8ICOS3UB9n+/PEGoJfuF6lxuWeKnRuC5RicgMSrJl3LMrwzMqjyBFyN/d aPpQqRjowRdG8E5cRWc99AzgPCv997iN0fQ3BMJe+jZOE+6Ex58Q/4RmpkM7sSpssyeX WcKGTYm94Vlz1jzO5kOi/e1EpFCMNOL9xCtH5RoZ/b1vBF/gKZzE6HkStFbbswiSPSEm 1aWJR/VSjXPNfe5M33ZN4G3odJ0WLCAz59bTLCr7T6qan8X4PsR6fho+BEdOl8AaYgrH 1LFA== X-Gm-Message-State: AJIora98hxeLeaxJIqIRZah9uZ6j7Y5AuHIgAegfZoTk9OnqRRYomsUc 94FyUb1xiL9xkenUGHrURMPNxa1n1Je/fAsZ4RWV6QH5BKf6PWDsr+u1RvfURuxMElP4SLO8DFr KO1XepIQfEnQGY8eVbm+cwj2NBc7/ujP5RgBXdSPfGg== X-Received: by 2002:a17:902:b114:b0:16c:4a04:67b7 with SMTP id q20-20020a170902b11400b0016c4a0467b7mr3083341plr.103.1657717029224; Wed, 13 Jul 2022 05:57:09 -0700 (PDT) X-Google-Smtp-Source: AGRyM1spSAH7L+vquyEx7H/A0ijDobMNUsV3VI7fqsqZz5HekKrRb/R2vuzMfm2hzYziDtc3UuNvYw== X-Received: by 2002:a17:902:b114:b0:16c:4a04:67b7 with SMTP id q20-20020a170902b11400b0016c4a0467b7mr3083326plr.103.1657717028992; Wed, 13 Jul 2022 05:57:08 -0700 (PDT) Received: from localhost.localdomain ([69.163.84.166]) by smtp.gmail.com with ESMTPSA id k91-20020a17090a14e400b001ec798b0dc4sm1551135pja.38.2022.07.13.05.57.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:57:08 -0700 (PDT) From: Tim Gardner To: kernel-team@lists.ubuntu.com Subject: [PATCH 05/11][jammy/linux-azure] PCI/MSI: Remove msi_desc_to_pci_sysdata() Date: Wed, 13 Jul 2022 06:56:43 -0600 Message-Id: <20220713125649.24063-10-tim.gardner@canonical.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713125649.24063-1-tim.gardner@canonical.com> References: <20220713125649.24063-1-tim.gardner@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Thomas Gleixner BugLink: https://bugs.launchpad.net/bugs/1981577 Last user is gone long ago. Signed-off-by: Thomas Gleixner Tested-by: Juergen Gross Reviewed-by: Jason Gunthorpe Acked-by: Bjorn Helgaas Link: https://lore.kernel.org/r/20211206210224.210768199@linutronix.de (cherry picked from commit ade044a3d0f0389e4f916337c505550acc3fd011) Signed-off-by: Tim Gardner --- drivers/pci/msi.c | 8 -------- include/linux/msi.h | 5 ----- 2 files changed, 13 deletions(-) diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c index c9a7877285f5..46b12f5f29b6 100644 --- a/drivers/pci/msi.c +++ b/drivers/pci/msi.c @@ -1266,14 +1266,6 @@ struct pci_dev *msi_desc_to_pci_dev(struct msi_desc *desc) } EXPORT_SYMBOL(msi_desc_to_pci_dev); -void *msi_desc_to_pci_sysdata(struct msi_desc *desc) -{ - struct pci_dev *dev = msi_desc_to_pci_dev(desc); - - return dev->bus->sysdata; -} -EXPORT_SYMBOL_GPL(msi_desc_to_pci_sysdata); - #ifdef CONFIG_PCI_MSI_IRQ_DOMAIN /** * pci_msi_domain_write_msg - Helper to write MSI message to PCI config space diff --git a/include/linux/msi.h b/include/linux/msi.h index d7b143a79cb4..ac6fec105edc 100644 --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -218,13 +218,8 @@ static inline void msi_desc_set_iommu_cookie(struct msi_desc *desc, for_each_msi_entry((desc), &(pdev)->dev) struct pci_dev *msi_desc_to_pci_dev(struct msi_desc *desc); -void *msi_desc_to_pci_sysdata(struct msi_desc *desc); void pci_write_msi_msg(unsigned int irq, struct msi_msg *msg); #else /* CONFIG_PCI_MSI */ -static inline void *msi_desc_to_pci_sysdata(struct msi_desc *desc) -{ - return NULL; -} static inline void pci_write_msi_msg(unsigned int irq, struct msi_msg *msg) { } From patchwork Wed Jul 13 12:56:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Gardner X-Patchwork-Id: 1655997 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=CwEQWDuH; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Ljd2G3zKLz9ryY for ; Wed, 13 Jul 2022 22:57:42 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1oBbvj-0006cK-6b; Wed, 13 Jul 2022 12:57:27 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1oBbvW-0006J0-Ew for kernel-team@lists.ubuntu.com; Wed, 13 Jul 2022 12:57:14 +0000 Received: from mail-pj1-f69.google.com (mail-pj1-f69.google.com [209.85.216.69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 934633F0C5 for ; Wed, 13 Jul 2022 12:57:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1657717032; bh=VQnXBR48sf2yqDgLF9NG4mu+lS9TcVcgPtszu4XzFs8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=CwEQWDuHlj+hIQ4cTjpegLbH9fwwudjOSXB/5ObSFzYKfpGML3M/WNszvsTi2M2FO DIvW3raQHSKVRIyNvThnCB0Lr7SYCQtGN/VksSRPfgm45n0usGC3RNsOCELkyuVwjB bKYR+6QkN8l2hbW7G6kj4iavo8x86jIMm1/T/K7p/ncVZ7trraQ26rcFRzjifkPGK+ sL2ed+wvL4JRjCCNkAMkuG10jKp6UOQDg7G4smx3m56PXbbIZIGu5v/xc6RYAJ82yO AZhXvkdrHL5ez9CO1hHJU6wk5AU0djUN1YmrQy1DTbJdZMjaslXHfkL04jU/eCpjne sp1U5m35Ds0cg== Received: by mail-pj1-f69.google.com with SMTP id mi13-20020a17090b4b4d00b001ef9759d26aso2303518pjb.0 for ; Wed, 13 Jul 2022 05:57:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VQnXBR48sf2yqDgLF9NG4mu+lS9TcVcgPtszu4XzFs8=; b=d9oCNyzG48cHz7JSap6a287ZCT3BNB9wepQg0c6EpuvgCtzwGsYKFtHUvtVoLKKkJC vUvLxgDxBIVc3e3bEL1vs1h3yPxr+v4Mlk8jKRwF3faQEIn+V6eKorPJclHyG0tz9q2Y 3AnaTo+Lqzc5RHpk1kkZfesiHH+yZwbGz7NZ0yEFFyN8ZV59jmiAyJtb8OH4Qm/XsE9Q ax4VKKUxjubRNFvZRcMPTAM8EBntGpGSlcs54t2gUGzWeWpOmGH0ciEFQZN8FBsfK3bJ 3139ml2/kA0cjahv77EQxKfUhylqdyGqRsZyBXmz1geNtr/zPK5zORggGZpc6JHSytbi F2kQ== X-Gm-Message-State: AJIora+3MAXkVk52AcO8zpxH3c2NjZ4VZE8D4TLbaxQ90MyQjjixb+gX XY/sij8EWBeLEKagkyU3dpWd4YLk8JiKHlCEotClKb4Loyr1CQMuqnyoWOjCa1Z79E5sUYUW884 YQJoQqP0yx5DRw28s5PHcsdP4Fpari0tKxgwOCeDqbA== X-Received: by 2002:a17:902:d488:b0:16c:59ce:5000 with SMTP id c8-20020a170902d48800b0016c59ce5000mr3026476plg.134.1657717030628; Wed, 13 Jul 2022 05:57:10 -0700 (PDT) X-Google-Smtp-Source: AGRyM1si14w2zSWY4yplSBsEebXwNrdmI2N9DhUZx+a+8aZdVTG1tFVbFC2bTfRIeBuKQACf6HkbWQ== X-Received: by 2002:a17:902:d488:b0:16c:59ce:5000 with SMTP id c8-20020a170902d48800b0016c59ce5000mr3026442plg.134.1657717030127; Wed, 13 Jul 2022 05:57:10 -0700 (PDT) Received: from localhost.localdomain ([69.163.84.166]) by smtp.gmail.com with ESMTPSA id k91-20020a17090a14e400b001ec798b0dc4sm1551135pja.38.2022.07.13.05.57.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:57:09 -0700 (PDT) From: Tim Gardner To: kernel-team@lists.ubuntu.com Subject: [PATCH 06/11][jammy/linux-azure] genirq/msi, treewide: Use a named struct for PCI/MSI attributes Date: Wed, 13 Jul 2022 06:56:44 -0600 Message-Id: <20220713125649.24063-11-tim.gardner@canonical.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713125649.24063-1-tim.gardner@canonical.com> References: <20220713125649.24063-1-tim.gardner@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Thomas Gleixner BugLink: https://bugs.launchpad.net/bugs/1981577 The unnamed struct sucks and is in the way of further cleanups. Stick the PCI related MSI data into a real data structure and cleanup all users. No functional change. Signed-off-by: Thomas Gleixner Tested-by: Juergen Gross Reviewed-by: Jason Gunthorpe Reviewed-by: Greg Kroah-Hartman Acked-by: Kalle Valo Link: https://lore.kernel.org/r/20211206210224.374863119@linutronix.de (cherry picked from commit e58f2259b91c02974c20db7b28d39d810a21249b) Signed-off-by: Tim Gardner --- arch/powerpc/platforms/cell/axon_msi.c | 2 +- arch/powerpc/platforms/powernv/pci-ioda.c | 4 +- arch/powerpc/platforms/pseries/msi.c | 6 +- arch/sparc/kernel/pci_msi.c | 4 +- arch/x86/kernel/apic/msi.c | 2 +- arch/x86/pci/xen.c | 6 +- drivers/net/wireless/ath/ath11k/pci.c | 2 +- drivers/pci/msi.c | 116 +++++++++++----------- drivers/pci/xen-pcifront.c | 2 +- include/linux/msi.h | 84 ++++++++-------- kernel/irq/msi.c | 4 +- 11 files changed, 115 insertions(+), 117 deletions(-) diff --git a/arch/powerpc/platforms/cell/axon_msi.c b/arch/powerpc/platforms/cell/axon_msi.c index 82335e364c44..79e21128518c 100644 --- a/arch/powerpc/platforms/cell/axon_msi.c +++ b/arch/powerpc/platforms/cell/axon_msi.c @@ -212,7 +212,7 @@ static int setup_msi_msg_address(struct pci_dev *dev, struct msi_msg *msg) entry = first_pci_msi_entry(dev); for (; dn; dn = of_get_next_parent(dn)) { - if (entry->msi_attrib.is_64) { + if (entry->pci.msi_attrib.is_64) { prop = of_get_property(dn, "msi-address-64", &len); if (prop) break; diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c index 3dd35c327d1c..d23159a1fb78 100644 --- a/arch/powerpc/platforms/powernv/pci-ioda.c +++ b/arch/powerpc/platforms/powernv/pci-ioda.c @@ -2154,10 +2154,10 @@ static void pnv_msi_compose_msg(struct irq_data *d, struct msi_msg *msg) int rc; rc = __pnv_pci_ioda_msi_setup(phb, pdev, d->hwirq, - entry->msi_attrib.is_64, msg); + entry->pci.msi_attrib.is_64, msg); if (rc) dev_err(&pdev->dev, "Failed to setup %s-bit MSI #%ld : %d\n", - entry->msi_attrib.is_64 ? "64" : "32", d->hwirq, rc); + entry->pci.msi_attrib.is_64 ? "64" : "32", d->hwirq, rc); } /* diff --git a/arch/powerpc/platforms/pseries/msi.c b/arch/powerpc/platforms/pseries/msi.c index 8627362f613e..8e287204eeae 100644 --- a/arch/powerpc/platforms/pseries/msi.c +++ b/arch/powerpc/platforms/pseries/msi.c @@ -332,7 +332,7 @@ static int check_msix_entries(struct pci_dev *pdev) expected = 0; for_each_pci_msi_entry(entry, pdev) { - if (entry->msi_attrib.entry_nr != expected) { + if (entry->pci.msi_attrib.entry_nr != expected) { pr_debug("rtas_msi: bad MSI-X entries.\n"); return -EINVAL; } @@ -449,7 +449,7 @@ static int pseries_msi_ops_prepare(struct irq_domain *domain, struct device *dev { struct pci_dev *pdev = to_pci_dev(dev); struct msi_desc *desc = first_pci_msi_entry(pdev); - int type = desc->msi_attrib.is_msix ? PCI_CAP_ID_MSIX : PCI_CAP_ID_MSI; + int type = desc->pci.msi_attrib.is_msix ? PCI_CAP_ID_MSIX : PCI_CAP_ID_MSI; return rtas_prepare_msi_irqs(pdev, nvec, type, arg); } @@ -580,7 +580,7 @@ static int pseries_irq_domain_alloc(struct irq_domain *domain, unsigned int virq int hwirq; int i, ret; - hwirq = rtas_query_irq_number(pci_get_pdn(pdev), desc->msi_attrib.entry_nr); + hwirq = rtas_query_irq_number(pci_get_pdn(pdev), desc->pci.msi_attrib.entry_nr); if (hwirq < 0) { dev_err(&pdev->dev, "Failed to query HW IRQ: %d\n", hwirq); return hwirq; diff --git a/arch/sparc/kernel/pci_msi.c b/arch/sparc/kernel/pci_msi.c index fb5899cbfa51..9ed11985768e 100644 --- a/arch/sparc/kernel/pci_msi.c +++ b/arch/sparc/kernel/pci_msi.c @@ -146,13 +146,13 @@ static int sparc64_setup_msi_irq(unsigned int *irq_p, msiqid = pick_msiq(pbm); err = ops->msi_setup(pbm, msiqid, msi, - (entry->msi_attrib.is_64 ? 1 : 0)); + (entry->pci.msi_attrib.is_64 ? 1 : 0)); if (err) goto out_msi_free; pbm->msi_irq_table[msi - pbm->msi_first] = *irq_p; - if (entry->msi_attrib.is_64) { + if (entry->pci.msi_attrib.is_64) { msg.address_hi = pbm->msi64_start >> 32; msg.address_lo = pbm->msi64_start & 0xffffffff; } else { diff --git a/arch/x86/kernel/apic/msi.c b/arch/x86/kernel/apic/msi.c index dbacb9ec8843..1656477e4169 100644 --- a/arch/x86/kernel/apic/msi.c +++ b/arch/x86/kernel/apic/msi.c @@ -163,7 +163,7 @@ int pci_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec, struct msi_desc *desc = first_pci_msi_entry(pdev); init_irq_alloc_info(arg, NULL); - if (desc->msi_attrib.is_msix) { + if (desc->pci.msi_attrib.is_msix) { arg->type = X86_IRQ_ALLOC_TYPE_PCI_MSIX; } else { arg->type = X86_IRQ_ALLOC_TYPE_PCI_MSI; diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c index f153e9ab8c96..8e75babdc96f 100644 --- a/arch/x86/pci/xen.c +++ b/arch/x86/pci/xen.c @@ -305,7 +305,7 @@ static int xen_initdom_setup_msi_irqs(struct pci_dev *dev, int nvec, int type) return -EINVAL; map_irq.table_base = pci_resource_start(dev, bir); - map_irq.entry_nr = msidesc->msi_attrib.entry_nr; + map_irq.entry_nr = msidesc->pci.msi_attrib.entry_nr; } ret = -EINVAL; @@ -397,7 +397,7 @@ static void xen_pv_teardown_msi_irqs(struct pci_dev *dev) { struct msi_desc *msidesc = first_pci_msi_entry(dev); - if (msidesc->msi_attrib.is_msix) + if (msidesc->pci.msi_attrib.is_msix) xen_pci_frontend_disable_msix(dev); else xen_pci_frontend_disable_msi(dev); @@ -413,7 +413,7 @@ static int xen_msi_domain_alloc_irqs(struct irq_domain *domain, if (WARN_ON_ONCE(!dev_is_pci(dev))) return -EINVAL; - if (first_msi_entry(dev)->msi_attrib.is_msix) + if (first_msi_entry(dev)->pci.msi_attrib.is_msix) type = PCI_CAP_ID_MSIX; else type = PCI_CAP_ID_MSI; diff --git a/drivers/net/wireless/ath/ath11k/pci.c b/drivers/net/wireless/ath/ath11k/pci.c index 251145b4e2ba..e66b08815911 100644 --- a/drivers/net/wireless/ath/ath11k/pci.c +++ b/drivers/net/wireless/ath/ath11k/pci.c @@ -889,7 +889,7 @@ static int ath11k_pci_enable_msi(struct ath11k_pci *ab_pci) } ab_pci->msi_ep_base_data = msi_desc->msg.data; - if (msi_desc->msi_attrib.is_64) + if (msi_desc->pci.msi_attrib.is_64) set_bit(ATH11K_PCI_FLAG_IS_MSI_64, &ab_pci->flags); ath11k_dbg(ab, ATH11K_DBG_PCI, "msi base data is %d\n", ab_pci->msi_ep_base_data); diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c index 46b12f5f29b6..f69c00a0242b 100644 --- a/drivers/pci/msi.c +++ b/drivers/pci/msi.c @@ -138,9 +138,9 @@ void __weak arch_restore_msi_irqs(struct pci_dev *dev) static inline __attribute_const__ u32 msi_multi_mask(struct msi_desc *desc) { /* Don't shift by >= width of type */ - if (desc->msi_attrib.multi_cap >= 5) + if (desc->pci.msi_attrib.multi_cap >= 5) return 0xffffffff; - return (1 << (1 << desc->msi_attrib.multi_cap)) - 1; + return (1 << (1 << desc->pci.msi_attrib.multi_cap)) - 1; } static noinline void pci_msi_update_mask(struct msi_desc *desc, u32 clear, u32 set) @@ -148,14 +148,14 @@ static noinline void pci_msi_update_mask(struct msi_desc *desc, u32 clear, u32 s raw_spinlock_t *lock = &desc->dev->msi_lock; unsigned long flags; - if (!desc->msi_attrib.can_mask) + if (!desc->pci.msi_attrib.can_mask) return; raw_spin_lock_irqsave(lock, flags); - desc->msi_mask &= ~clear; - desc->msi_mask |= set; - pci_write_config_dword(msi_desc_to_pci_dev(desc), desc->mask_pos, - desc->msi_mask); + desc->pci.msi_mask &= ~clear; + desc->pci.msi_mask |= set; + pci_write_config_dword(msi_desc_to_pci_dev(desc), desc->pci.mask_pos, + desc->pci.msi_mask); raw_spin_unlock_irqrestore(lock, flags); } @@ -171,7 +171,7 @@ static inline void pci_msi_unmask(struct msi_desc *desc, u32 mask) static inline void __iomem *pci_msix_desc_addr(struct msi_desc *desc) { - return desc->mask_base + desc->msi_attrib.entry_nr * PCI_MSIX_ENTRY_SIZE; + return desc->pci.mask_base + desc->pci.msi_attrib.entry_nr * PCI_MSIX_ENTRY_SIZE; } /* @@ -184,27 +184,27 @@ static void pci_msix_write_vector_ctrl(struct msi_desc *desc, u32 ctrl) { void __iomem *desc_addr = pci_msix_desc_addr(desc); - if (desc->msi_attrib.can_mask) + if (desc->pci.msi_attrib.can_mask) writel(ctrl, desc_addr + PCI_MSIX_ENTRY_VECTOR_CTRL); } static inline void pci_msix_mask(struct msi_desc *desc) { - desc->msix_ctrl |= PCI_MSIX_ENTRY_CTRL_MASKBIT; - pci_msix_write_vector_ctrl(desc, desc->msix_ctrl); + desc->pci.msix_ctrl |= PCI_MSIX_ENTRY_CTRL_MASKBIT; + pci_msix_write_vector_ctrl(desc, desc->pci.msix_ctrl); /* Flush write to device */ - readl(desc->mask_base); + readl(desc->pci.mask_base); } static inline void pci_msix_unmask(struct msi_desc *desc) { - desc->msix_ctrl &= ~PCI_MSIX_ENTRY_CTRL_MASKBIT; - pci_msix_write_vector_ctrl(desc, desc->msix_ctrl); + desc->pci.msix_ctrl &= ~PCI_MSIX_ENTRY_CTRL_MASKBIT; + pci_msix_write_vector_ctrl(desc, desc->pci.msix_ctrl); } static void __pci_msi_mask_desc(struct msi_desc *desc, u32 mask) { - if (desc->msi_attrib.is_msix) + if (desc->pci.msi_attrib.is_msix) pci_msix_mask(desc); else pci_msi_mask(desc, mask); @@ -212,7 +212,7 @@ static void __pci_msi_mask_desc(struct msi_desc *desc, u32 mask) static void __pci_msi_unmask_desc(struct msi_desc *desc, u32 mask) { - if (desc->msi_attrib.is_msix) + if (desc->pci.msi_attrib.is_msix) pci_msix_unmask(desc); else pci_msi_unmask(desc, mask); @@ -256,10 +256,10 @@ void __pci_read_msi_msg(struct msi_desc *entry, struct msi_msg *msg) BUG_ON(dev->current_state != PCI_D0); - if (entry->msi_attrib.is_msix) { + if (entry->pci.msi_attrib.is_msix) { void __iomem *base = pci_msix_desc_addr(entry); - if (WARN_ON_ONCE(entry->msi_attrib.is_virtual)) + if (WARN_ON_ONCE(entry->pci.msi_attrib.is_virtual)) return; msg->address_lo = readl(base + PCI_MSIX_ENTRY_LOWER_ADDR); @@ -271,7 +271,7 @@ void __pci_read_msi_msg(struct msi_desc *entry, struct msi_msg *msg) pci_read_config_dword(dev, pos + PCI_MSI_ADDRESS_LO, &msg->address_lo); - if (entry->msi_attrib.is_64) { + if (entry->pci.msi_attrib.is_64) { pci_read_config_dword(dev, pos + PCI_MSI_ADDRESS_HI, &msg->address_hi); pci_read_config_word(dev, pos + PCI_MSI_DATA_64, &data); @@ -289,12 +289,12 @@ void __pci_write_msi_msg(struct msi_desc *entry, struct msi_msg *msg) if (dev->current_state != PCI_D0 || pci_dev_is_disconnected(dev)) { /* Don't touch the hardware now */ - } else if (entry->msi_attrib.is_msix) { + } else if (entry->pci.msi_attrib.is_msix) { void __iomem *base = pci_msix_desc_addr(entry); - u32 ctrl = entry->msix_ctrl; + u32 ctrl = entry->pci.msix_ctrl; bool unmasked = !(ctrl & PCI_MSIX_ENTRY_CTRL_MASKBIT); - if (entry->msi_attrib.is_virtual) + if (entry->pci.msi_attrib.is_virtual) goto skip; /* @@ -323,12 +323,12 @@ void __pci_write_msi_msg(struct msi_desc *entry, struct msi_msg *msg) pci_read_config_word(dev, pos + PCI_MSI_FLAGS, &msgctl); msgctl &= ~PCI_MSI_FLAGS_QSIZE; - msgctl |= entry->msi_attrib.multiple << 4; + msgctl |= entry->pci.msi_attrib.multiple << 4; pci_write_config_word(dev, pos + PCI_MSI_FLAGS, msgctl); pci_write_config_dword(dev, pos + PCI_MSI_ADDRESS_LO, msg->address_lo); - if (entry->msi_attrib.is_64) { + if (entry->pci.msi_attrib.is_64) { pci_write_config_dword(dev, pos + PCI_MSI_ADDRESS_HI, msg->address_hi); pci_write_config_word(dev, pos + PCI_MSI_DATA_64, @@ -376,9 +376,9 @@ static void free_msi_irqs(struct pci_dev *dev) pci_msi_teardown_msi_irqs(dev); list_for_each_entry_safe(entry, tmp, msi_list, list) { - if (entry->msi_attrib.is_msix) { + if (entry->pci.msi_attrib.is_msix) { if (list_is_last(&entry->list, msi_list)) - iounmap(entry->mask_base); + iounmap(entry->pci.mask_base); } list_del(&entry->list); @@ -420,7 +420,7 @@ static void __pci_restore_msi_state(struct pci_dev *dev) pci_read_config_word(dev, dev->msi_cap + PCI_MSI_FLAGS, &control); pci_msi_update_mask(entry, 0, 0); control &= ~PCI_MSI_FLAGS_QSIZE; - control |= (entry->msi_attrib.multiple << 4) | PCI_MSI_FLAGS_ENABLE; + control |= (entry->pci.msi_attrib.multiple << 4) | PCI_MSI_FLAGS_ENABLE; pci_write_config_word(dev, dev->msi_cap + PCI_MSI_FLAGS, control); } @@ -449,7 +449,7 @@ static void __pci_restore_msix_state(struct pci_dev *dev) arch_restore_msi_irqs(dev); for_each_pci_msi_entry(entry, dev) - pci_msix_write_vector_ctrl(entry, entry->msix_ctrl); + pci_msix_write_vector_ctrl(entry, entry->pci.msix_ctrl); pci_msix_clear_and_set_ctrl(dev, PCI_MSIX_FLAGS_MASKALL, 0); } @@ -481,24 +481,24 @@ msi_setup_entry(struct pci_dev *dev, int nvec, struct irq_affinity *affd) if (dev->dev_flags & PCI_DEV_FLAGS_HAS_MSI_MASKING) control |= PCI_MSI_FLAGS_MASKBIT; - entry->msi_attrib.is_msix = 0; - entry->msi_attrib.is_64 = !!(control & PCI_MSI_FLAGS_64BIT); - entry->msi_attrib.is_virtual = 0; - entry->msi_attrib.entry_nr = 0; - entry->msi_attrib.can_mask = !pci_msi_ignore_mask && + entry->pci.msi_attrib.is_msix = 0; + entry->pci.msi_attrib.is_64 = !!(control & PCI_MSI_FLAGS_64BIT); + entry->pci.msi_attrib.is_virtual = 0; + entry->pci.msi_attrib.entry_nr = 0; + entry->pci.msi_attrib.can_mask = !pci_msi_ignore_mask && !!(control & PCI_MSI_FLAGS_MASKBIT); - entry->msi_attrib.default_irq = dev->irq; /* Save IOAPIC IRQ */ - entry->msi_attrib.multi_cap = (control & PCI_MSI_FLAGS_QMASK) >> 1; - entry->msi_attrib.multiple = ilog2(__roundup_pow_of_two(nvec)); + entry->pci.msi_attrib.default_irq = dev->irq; /* Save IOAPIC IRQ */ + entry->pci.msi_attrib.multi_cap = (control & PCI_MSI_FLAGS_QMASK) >> 1; + entry->pci.msi_attrib.multiple = ilog2(__roundup_pow_of_two(nvec)); if (control & PCI_MSI_FLAGS_64BIT) - entry->mask_pos = dev->msi_cap + PCI_MSI_MASK_64; + entry->pci.mask_pos = dev->msi_cap + PCI_MSI_MASK_64; else - entry->mask_pos = dev->msi_cap + PCI_MSI_MASK_32; + entry->pci.mask_pos = dev->msi_cap + PCI_MSI_MASK_32; /* Save the initial mask status */ - if (entry->msi_attrib.can_mask) - pci_read_config_dword(dev, entry->mask_pos, &entry->msi_mask); + if (entry->pci.msi_attrib.can_mask) + pci_read_config_dword(dev, entry->pci.mask_pos, &entry->pci.msi_mask); out: kfree(masks); @@ -629,26 +629,26 @@ static int msix_setup_entries(struct pci_dev *dev, void __iomem *base, goto out; } - entry->msi_attrib.is_msix = 1; - entry->msi_attrib.is_64 = 1; + entry->pci.msi_attrib.is_msix = 1; + entry->pci.msi_attrib.is_64 = 1; if (entries) - entry->msi_attrib.entry_nr = entries[i].entry; + entry->pci.msi_attrib.entry_nr = entries[i].entry; else - entry->msi_attrib.entry_nr = i; + entry->pci.msi_attrib.entry_nr = i; - entry->msi_attrib.is_virtual = - entry->msi_attrib.entry_nr >= vec_count; + entry->pci.msi_attrib.is_virtual = + entry->pci.msi_attrib.entry_nr >= vec_count; - entry->msi_attrib.can_mask = !pci_msi_ignore_mask && - !entry->msi_attrib.is_virtual; + entry->pci.msi_attrib.can_mask = !pci_msi_ignore_mask && + !entry->pci.msi_attrib.is_virtual; - entry->msi_attrib.default_irq = dev->irq; - entry->mask_base = base; + entry->pci.msi_attrib.default_irq = dev->irq; + entry->pci.mask_base = base; - if (entry->msi_attrib.can_mask) { + if (entry->pci.msi_attrib.can_mask) { addr = pci_msix_desc_addr(entry); - entry->msix_ctrl = readl(addr + PCI_MSIX_ENTRY_VECTOR_CTRL); + entry->pci.msix_ctrl = readl(addr + PCI_MSIX_ENTRY_VECTOR_CTRL); } list_add_tail(&entry->list, dev_to_msi_list(&dev->dev)); @@ -873,7 +873,7 @@ static void pci_msi_shutdown(struct pci_dev *dev) pci_msi_unmask(desc, msi_multi_mask(desc)); /* Restore dev->irq to its default pin-assertion IRQ */ - dev->irq = desc->msi_attrib.default_irq; + dev->irq = desc->pci.msi_attrib.default_irq; pcibios_alloc_irq(dev); } @@ -1202,7 +1202,7 @@ int pci_irq_vector(struct pci_dev *dev, unsigned int nr) struct msi_desc *entry; for_each_pci_msi_entry(entry, dev) { - if (entry->msi_attrib.entry_nr == nr) + if (entry->pci.msi_attrib.entry_nr == nr) return entry->irq; } WARN_ON_ONCE(1); @@ -1241,7 +1241,7 @@ const struct cpumask *pci_irq_get_affinity(struct pci_dev *dev, int nr) struct msi_desc *entry; for_each_pci_msi_entry(entry, dev) { - if (entry->msi_attrib.entry_nr == nr) + if (entry->pci.msi_attrib.entry_nr == nr) return &entry->affinity->mask; } WARN_ON_ONCE(1); @@ -1294,14 +1294,14 @@ static irq_hw_number_t pci_msi_domain_calc_hwirq(struct msi_desc *desc) { struct pci_dev *dev = msi_desc_to_pci_dev(desc); - return (irq_hw_number_t)desc->msi_attrib.entry_nr | + return (irq_hw_number_t)desc->pci.msi_attrib.entry_nr | pci_dev_id(dev) << 11 | (pci_domain_nr(dev->bus) & 0xFFFFFFFF) << 27; } static inline bool pci_msi_desc_is_multi_msi(struct msi_desc *desc) { - return !desc->msi_attrib.is_msix && desc->nvec_used > 1; + return !desc->pci.msi_attrib.is_msix && desc->nvec_used > 1; } /** @@ -1325,7 +1325,7 @@ int pci_msi_domain_check_cap(struct irq_domain *domain, if (pci_msi_desc_is_multi_msi(desc) && !(info->flags & MSI_FLAG_MULTI_PCI_MSI)) return 1; - else if (desc->msi_attrib.is_msix && !(info->flags & MSI_FLAG_PCI_MSIX)) + else if (desc->pci.msi_attrib.is_msix && !(info->flags & MSI_FLAG_PCI_MSIX)) return -ENOTSUPP; return 0; diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c index 2156c632524d..7229b32472cd 100644 --- a/drivers/pci/xen-pcifront.c +++ b/drivers/pci/xen-pcifront.c @@ -263,7 +263,7 @@ static int pci_frontend_enable_msix(struct pci_dev *dev, i = 0; for_each_pci_msi_entry(entry, dev) { - op.msix_entries[i].entry = entry->msi_attrib.entry_nr; + op.msix_entries[i].entry = entry->pci.msi_attrib.entry_nr; /* Vector is useless at this point. */ op.msix_entries[i].vector = -1; i++; diff --git a/include/linux/msi.h b/include/linux/msi.h index ac6fec105edc..7e5c13f4e41b 100644 --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -68,6 +68,42 @@ static inline void get_cached_msi_msg(unsigned int irq, struct msi_msg *msg) typedef void (*irq_write_msi_msg_t)(struct msi_desc *desc, struct msi_msg *msg); +/** + * pci_msi_desc - PCI/MSI specific MSI descriptor data + * + * @msi_mask: [PCI MSI] MSI cached mask bits + * @msix_ctrl: [PCI MSI-X] MSI-X cached per vector control bits + * @is_msix: [PCI MSI/X] True if MSI-X + * @multiple: [PCI MSI/X] log2 num of messages allocated + * @multi_cap: [PCI MSI/X] log2 num of messages supported + * @can_mask: [PCI MSI/X] Masking supported? + * @is_64: [PCI MSI/X] Address size: 0=32bit 1=64bit + * @entry_nr: [PCI MSI/X] Entry which is described by this descriptor + * @default_irq:[PCI MSI/X] The default pre-assigned non-MSI irq + * @mask_pos: [PCI MSI] Mask register position + * @mask_base: [PCI MSI-X] Mask register base address + */ +struct pci_msi_desc { + union { + u32 msi_mask; + u32 msix_ctrl; + }; + struct { + u8 is_msix : 1; + u8 multiple : 3; + u8 multi_cap : 3; + u8 can_mask : 1; + u8 is_64 : 1; + u8 is_virtual : 1; + u16 entry_nr; + unsigned default_irq; + } msi_attrib; + union { + u8 mask_pos; + void __iomem *mask_base; + }; +}; + /** * platform_msi_desc - Platform device specific msi descriptor data * @msi_priv_data: Pointer to platform private data @@ -107,17 +143,7 @@ struct ti_sci_inta_msi_desc { * address or data changes * @write_msi_msg_data: Data parameter for the callback. * - * @msi_mask: [PCI MSI] MSI cached mask bits - * @msix_ctrl: [PCI MSI-X] MSI-X cached per vector control bits - * @is_msix: [PCI MSI/X] True if MSI-X - * @multiple: [PCI MSI/X] log2 num of messages allocated - * @multi_cap: [PCI MSI/X] log2 num of messages supported - * @maskbit: [PCI MSI/X] Mask-Pending bit supported? - * @is_64: [PCI MSI/X] Address size: 0=32bit 1=64bit - * @entry_nr: [PCI MSI/X] Entry which is described by this descriptor - * @default_irq:[PCI MSI/X] The default pre-assigned non-MSI irq - * @mask_pos: [PCI MSI] Mask register position - * @mask_base: [PCI MSI-X] Mask register base address + * @pci: [PCI] PCI speficic msi descriptor data * @platform: [platform] Platform device specific msi descriptor data * @fsl_mc: [fsl-mc] FSL MC device specific msi descriptor data * @inta: [INTA] TISCI based INTA specific msi descriptor data @@ -138,38 +164,10 @@ struct msi_desc { void *write_msi_msg_data; union { - /* PCI MSI/X specific data */ - struct { - union { - u32 msi_mask; - u32 msix_ctrl; - }; - struct { - u8 is_msix : 1; - u8 multiple : 3; - u8 multi_cap : 3; - u8 can_mask : 1; - u8 is_64 : 1; - u8 is_virtual : 1; - u16 entry_nr; - unsigned default_irq; - } msi_attrib; - union { - u8 mask_pos; - void __iomem *mask_base; - }; - }; - - /* - * Non PCI variants add their data structure here. New - * entries need to use a named structure. We want - * proper name spaces for this. The PCI part is - * anonymous for now as it would require an immediate - * tree wide cleanup. - */ - struct platform_msi_desc platform; - struct fsl_mc_msi_desc fsl_mc; - struct ti_sci_inta_msi_desc inta; + struct pci_msi_desc pci; + struct platform_msi_desc platform; + struct fsl_mc_msi_desc fsl_mc; + struct ti_sci_inta_msi_desc inta; }; }; diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c index 6718bab1bde3..7d78d8aff076 100644 --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -91,7 +91,7 @@ static ssize_t msi_mode_show(struct device *dev, struct device_attribute *attr, return -ENODEV; if (dev_is_pci(dev)) - is_msix = entry->msi_attrib.is_msix; + is_msix = entry->pci.msi_attrib.is_msix; return sysfs_emit(buf, "%s\n", is_msix ? "msix" : "msi"); } @@ -535,7 +535,7 @@ static bool msi_check_reservation_mode(struct irq_domain *domain, * masking and MSI does so when the can_mask attribute is set. */ desc = first_msi_entry(dev); - return desc->msi_attrib.is_msix || desc->msi_attrib.can_mask; + return desc->pci.msi_attrib.is_msix || desc->pci.msi_attrib.can_mask; } int __msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev, From patchwork Wed Jul 13 12:56:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Gardner X-Patchwork-Id: 1655998 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=UwqD08f9; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Ljd2L1VDBz9s1l for ; Wed, 13 Jul 2022 22:57:46 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1oBbvo-0006nG-TK; Wed, 13 Jul 2022 12:57:33 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1oBbvY-0006LG-EP for kernel-team@lists.ubuntu.com; Wed, 13 Jul 2022 12:57:16 +0000 Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id F36593F0CB for ; Wed, 13 Jul 2022 12:57:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1657717033; bh=mNFP4p4RpTwnkxcPgxbSZsoD/PDDbtvd6iejmFVlgyk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UwqD08f9NTzThvJ9oUlPxXkVzxUkmI7+jlXAhehBaPpG3KIhBtLn/GHt9w73k5Bkd yAwVfsGoRhCdmUY7h/v2szRaRz5F6vjyEc0THsXMu3EC/GLGYrZQRRwwUB88voo4FE 4kYRTkXSYSKemVGUR/nUulCb554bFwuaW04tWbwT+7MwtpQZrLutmfPrcBHBKQ9CWY Dg25DDRzfOOJANIyghz6N8DBTgDoArwSEASjOB5BZqdm8cvfELrne2l0tce10615PN 0o2a+KS52Xjhm2Z4B2o52iiOxK6yUhdBFEJWyBaWawEFTUN9E3ck/y7rIa7/U+oCDO k3oKSjGP0xt8A== Received: by mail-pg1-f197.google.com with SMTP id 9-20020a631449000000b00412b1418c79so5139471pgu.2 for ; Wed, 13 Jul 2022 05:57:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mNFP4p4RpTwnkxcPgxbSZsoD/PDDbtvd6iejmFVlgyk=; b=Lq4W/Dt49x0Pp3J3BJIhrpWqlzFzcNx9J2a6TKm4MIHc1L5JSA9FBX/gJPtEHAaKQV 0Wrjtk6kB4LeslPKMOUnlblGxlsgRxUp/9NhOhKFtxLc0opZk0iuqoSOyhCL0GuonISp 67wTdi3zFUhdq9oFN8NX/Nm2yCNz7lU63CORoTlAKpTNW850W+glkeCwz3MpB2oslMgp RbIkHmqeAS8eZEj2I+WWLQyEy4op0KkbGoshhwLJRa3AUCj0ebEoLlifRKsq9WVCAMY0 3K8gxsxQ65Zz3O4s6S/+oFQCRc6YAJ+AuBDpQ86hTwVBU0yi+9sKw8JPEoBlsksoj6vH fH3g== X-Gm-Message-State: AJIora8qd+3M/C5+nkbiQ4+EJh84f1yDONZNKWF9n00iPdHOsJh6UNWP roUxmIIvgiS/sOdC5G2Gq8iAjuVL8gHSQOMN5iZahY2a+8Wef+0BshW8Wu2cMmFr2xMA2yYp3ab WWb62iD0wRq8sdxu1NKIA1NAlY9EFVyOzqoQJXr2HQA== X-Received: by 2002:a17:903:32c2:b0:16c:3c8d:3807 with SMTP id i2-20020a17090332c200b0016c3c8d3807mr3279079plr.173.1657717031279; Wed, 13 Jul 2022 05:57:11 -0700 (PDT) X-Google-Smtp-Source: AGRyM1tGC5A7RveD2i17rJZZ7mMApHkj4zjoSElTtVpbOV+mJNAjUWucwIW0BZiW6JGKgLz8Ki3yUQ== X-Received: by 2002:a17:903:32c2:b0:16c:3c8d:3807 with SMTP id i2-20020a17090332c200b0016c3c8d3807mr3279068plr.173.1657717031016; Wed, 13 Jul 2022 05:57:11 -0700 (PDT) Received: from localhost.localdomain ([69.163.84.166]) by smtp.gmail.com with ESMTPSA id k91-20020a17090a14e400b001ec798b0dc4sm1551135pja.38.2022.07.13.05.57.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:57:10 -0700 (PDT) From: Tim Gardner To: kernel-team@lists.ubuntu.com Subject: [PATCH 07/11][jammy/linux-azure] PCI: hv: Fix multi-MSI to allow more than one MSI vector Date: Wed, 13 Jul 2022 06:56:45 -0600 Message-Id: <20220713125649.24063-12-tim.gardner@canonical.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713125649.24063-1-tim.gardner@canonical.com> References: <20220713125649.24063-1-tim.gardner@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Jeffrey Hugo BugLink: https://bugs.launchpad.net/bugs/1981577 If the allocation of multiple MSI vectors for multi-MSI fails in the core PCI framework, the framework will retry the allocation as a single MSI vector, assuming that meets the min_vecs specified by the requesting driver. Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR domain to implement that for x86. The VECTOR domain does not support multi-MSI, so the alloc will always fail and fallback to a single MSI allocation. In short, Hyper-V advertises a capability it does not implement. Hyper-V can support multi-MSI because it coordinates with the hypervisor to map the MSIs in the IOMMU's interrupt remapper, which is something the VECTOR domain does not have. Therefore the fix is simple - copy what the x86 IOMMU drivers (AMD/Intel-IR) do by removing X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's pci_msi_prepare(). Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs") Signed-off-by: Jeffrey Hugo Reviewed-by: Dexuan Cui Link: https://lore.kernel.org/r/1649856981-14649-1-git-send-email-quic_jhugo@quicinc.com Signed-off-by: Wei Liu (cherry picked from commit 08e61e861a0e47e5e1a3fb78406afd6b0cea6b6d) Signed-off-by: Tim Gardner --- drivers/pci/controller/pci-hyperv.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index aa164c094e5b..f639d6b10753 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -615,7 +615,16 @@ static void hv_set_msi_entry_from_desc(union hv_msi_entry *msi_entry, static int hv_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec, msi_alloc_info_t *info) { - return pci_msi_prepare(domain, dev, nvec, info); + int ret = pci_msi_prepare(domain, dev, nvec, info); + + /* + * By using the interrupt remapper in the hypervisor IOMMU, contiguous + * CPU vectors is not needed for multi-MSI + */ + if (info->type == X86_IRQ_ALLOC_TYPE_PCI_MSI) + info->flags &= ~X86_IRQ_ALLOC_CONTIGUOUS_VECTORS; + + return ret; } #elif defined(CONFIG_ARM64) /* From patchwork Wed Jul 13 12:56:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Gardner X-Patchwork-Id: 1655999 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=doEGzRfV; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Ljd2Q6d6vz9ryY for ; Wed, 13 Jul 2022 22:57:50 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1oBbvu-00070c-Sw; Wed, 13 Jul 2022 12:57:38 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1oBbvW-0006Js-7M for kernel-team@lists.ubuntu.com; Wed, 13 Jul 2022 12:57:14 +0000 Received: from mail-pj1-f69.google.com (mail-pj1-f69.google.com [209.85.216.69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id CB50B3F6FC for ; Wed, 13 Jul 2022 12:57:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1657717033; bh=T603wHALU+5kJGm90zpGmWmQrGi4WBVfmXe/4cflEiA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=doEGzRfVVxgQh585eq8beAZV6ilTu6GZKvWhphvFV3uotxV8DwAGZyjBDNClKXHhc jzhSMG9R+6iNOpMuNTHsicpJv+du8lybatrDRVmSM878REShkwsZO5nGKfcbZ4r+yb RQlfXNEPDK04UtPf8JqxKu9M/8wUlYJxfaO7pyv2zRdXCrGwDJ7gAm3+nuQkE2sq+2 66JkCa0CCZoqa9fr8nHmtKbodI1RbzeZC4zxEhioXOk6pR+dW9Mv1oacOk4O3z/V63 GKdCRdmdTJkuILBqRnjcypmIXGW81TfaIKDbg/ALKxl2HOrVh57ABNKwq1ch+2FM7H z68pXJ7FD6+wQ== Received: by mail-pj1-f69.google.com with SMTP id cu12-20020a17090afa8c00b001efdd3bac65so5442882pjb.7 for ; Wed, 13 Jul 2022 05:57:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=T603wHALU+5kJGm90zpGmWmQrGi4WBVfmXe/4cflEiA=; b=UmlWSGcfajzm/oFw8E5QQxch5qwBpwSxMUBBd6r6ZKV9jlfEfS5uMKU+ug65vzlA2w GMwKA79R8NxwWoiB0lXS9u1Mrh5nNgnI7aI3DbP21IfiPIYauyu7c/6VCbbsn5MeTL/w iUEs6YlHoK3NqrF8HUughtSB9ryU5uLs+a+52L6Ffay4MqTG01xY6HlCuzmSoysqEf8W YRsN018ai09Yt2oJ0deNBZP8BlBith2wdr5AyCdec1lLNprxtriQdtgI2Kq4U8ZRHh61 H2XSD2jW5BDWpmWnMOnq9u5q+Zay94KUY/43jxDpDrq3PPHeg5r7El8n2icPEhegDbmS 1Kiw== X-Gm-Message-State: AJIora+XaegJqZybu1KIBV+covDCFwQ2E+sjoOrXftD8DIUPHsQovZ9f V2rfdOuVxrWH/AlfBwLV+2r2jBMZm/5x6ANz5YBzQyG2iH4Gb0XkzQYA0SxKM1+aid+h061nQ4y 9NCTQoQxGUhIcLYuWoyyPhNiIX6PUVpV1kOZXksOmHQ== X-Received: by 2002:a17:902:ef4f:b0:16b:8744:6c5f with SMTP id e15-20020a170902ef4f00b0016b87446c5fmr3153219plx.60.1657717032814; Wed, 13 Jul 2022 05:57:12 -0700 (PDT) X-Google-Smtp-Source: AGRyM1s2ME0vzqN0aL8EAvoNmRzElTWzqTzN146jhTuJhStOC7br7IAB45AIKXgmlnZwrNtBG9jpJg== X-Received: by 2002:a17:902:ef4f:b0:16b:8744:6c5f with SMTP id e15-20020a170902ef4f00b0016b87446c5fmr3153191plx.60.1657717032350; Wed, 13 Jul 2022 05:57:12 -0700 (PDT) Received: from localhost.localdomain ([69.163.84.166]) by smtp.gmail.com with ESMTPSA id k91-20020a17090a14e400b001ec798b0dc4sm1551135pja.38.2022.07.13.05.57.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:57:11 -0700 (PDT) From: Tim Gardner To: kernel-team@lists.ubuntu.com Subject: [PATCH 08/11][jammy/linux-azure] PCI: hv: Avoid the retarget interrupt hypercall in irq_unmask() on ARM64 Date: Wed, 13 Jul 2022 06:56:46 -0600 Message-Id: <20220713125649.24063-13-tim.gardner@canonical.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713125649.24063-1-tim.gardner@canonical.com> References: <20220713125649.24063-1-tim.gardner@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Boqun Feng BugLink: https://bugs.launchpad.net/bugs/1981577 On ARM64 Hyper-V guests, SPIs are used for the interrupts of virtual PCI devices, and SPIs can be managed directly via GICD registers. Therefore the retarget interrupt hypercall is not needed on ARM64. An arch-specific interface hv_arch_irq_unmask() is introduced to handle the architecture level differences on this. For x86, the behavior remains unchanged, while for ARM64 no hypercall is invoked when unmasking an irq for virtual PCI devices. Link: https://lore.kernel.org/r/20220217034525.1687678-1-boqun.feng@gmail.com Signed-off-by: Boqun Feng Signed-off-by: Lorenzo Pieralisi Reviewed-by: Michael Kelley (cherry picked from commit d06957d7a6929e6a4aa959cb59d66f0c095fc974) Signed-off-by: Tim Gardner --- drivers/pci/controller/pci-hyperv.c | 233 +++++++++++++++------------- 1 file changed, 122 insertions(+), 111 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index f639d6b10753..2244338a56ba 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -626,6 +626,121 @@ static int hv_msi_prepare(struct irq_domain *domain, struct device *dev, return ret; } + +/** + * hv_arch_irq_unmask() - "Unmask" the IRQ by setting its current + * affinity. + * @data: Describes the IRQ + * + * Build new a destination for the MSI and make a hypercall to + * update the Interrupt Redirection Table. "Device Logical ID" + * is built out of this PCI bus's instance GUID and the function + * number of the device. + */ +static void hv_arch_irq_unmask(struct irq_data *data) +{ + struct msi_desc *msi_desc = irq_data_get_msi_desc(data); + struct hv_retarget_device_interrupt *params; + struct hv_pcibus_device *hbus; + struct cpumask *dest; + cpumask_var_t tmp; + struct pci_bus *pbus; + struct pci_dev *pdev; + unsigned long flags; + u32 var_size = 0; + int cpu, nr_bank; + u64 res; + + dest = irq_data_get_effective_affinity_mask(data); + pdev = msi_desc_to_pci_dev(msi_desc); + pbus = pdev->bus; + hbus = container_of(pbus->sysdata, struct hv_pcibus_device, sysdata); + + spin_lock_irqsave(&hbus->retarget_msi_interrupt_lock, flags); + + params = &hbus->retarget_msi_interrupt_params; + memset(params, 0, sizeof(*params)); + params->partition_id = HV_PARTITION_ID_SELF; + params->int_entry.source = HV_INTERRUPT_SOURCE_MSI; + hv_set_msi_entry_from_desc(¶ms->int_entry.msi_entry, msi_desc); + params->device_id = (hbus->hdev->dev_instance.b[5] << 24) | + (hbus->hdev->dev_instance.b[4] << 16) | + (hbus->hdev->dev_instance.b[7] << 8) | + (hbus->hdev->dev_instance.b[6] & 0xf8) | + PCI_FUNC(pdev->devfn); + params->int_target.vector = hv_msi_get_int_vector(data); + + /* + * Honoring apic->delivery_mode set to APIC_DELIVERY_MODE_FIXED by + * setting the HV_DEVICE_INTERRUPT_TARGET_MULTICAST flag results in a + * spurious interrupt storm. Not doing so does not seem to have a + * negative effect (yet?). + */ + + if (hbus->protocol_version >= PCI_PROTOCOL_VERSION_1_2) { + /* + * PCI_PROTOCOL_VERSION_1_2 supports the VP_SET version of the + * HVCALL_RETARGET_INTERRUPT hypercall, which also coincides + * with >64 VP support. + * ms_hyperv.hints & HV_X64_EX_PROCESSOR_MASKS_RECOMMENDED + * is not sufficient for this hypercall. + */ + params->int_target.flags |= + HV_DEVICE_INTERRUPT_TARGET_PROCESSOR_SET; + + if (!alloc_cpumask_var(&tmp, GFP_ATOMIC)) { + res = 1; + goto exit_unlock; + } + + cpumask_and(tmp, dest, cpu_online_mask); + nr_bank = cpumask_to_vpset(¶ms->int_target.vp_set, tmp); + free_cpumask_var(tmp); + + if (nr_bank <= 0) { + res = 1; + goto exit_unlock; + } + + /* + * var-sized hypercall, var-size starts after vp_mask (thus + * vp_set.format does not count, but vp_set.valid_bank_mask + * does). + */ + var_size = 1 + nr_bank; + } else { + for_each_cpu_and(cpu, dest, cpu_online_mask) { + params->int_target.vp_mask |= + (1ULL << hv_cpu_number_to_vp_number(cpu)); + } + } + + res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT | (var_size << 17), + params, NULL); + +exit_unlock: + spin_unlock_irqrestore(&hbus->retarget_msi_interrupt_lock, flags); + + /* + * During hibernation, when a CPU is offlined, the kernel tries + * to move the interrupt to the remaining CPUs that haven't + * been offlined yet. In this case, the below hv_do_hypercall() + * always fails since the vmbus channel has been closed: + * refer to cpu_disable_common() -> fixup_irqs() -> + * irq_migrate_all_off_this_cpu() -> migrate_one_irq(). + * + * Suppress the error message for hibernation because the failure + * during hibernation does not matter (at this time all the devices + * have been frozen). Note: the correct affinity info is still updated + * into the irqdata data structure in migrate_one_irq() -> + * irq_do_set_affinity() -> hv_set_affinity(), so later when the VM + * resumes, hv_pci_restore_msi_state() is able to correctly restore + * the interrupt with the correct affinity. + */ + if (!hv_result_success(res) && hbus->state != hv_pcibus_removing) + dev_err(&hbus->hdev->device, + "%s() failed: %#llx", __func__, res); +} #elif defined(CONFIG_ARM64) /* * SPI vectors to use for vPCI; arch SPIs range is [32, 1019], but leaving a bit @@ -849,6 +964,12 @@ static struct irq_domain *hv_pci_get_root_domain(void) { return hv_msi_gic_irq_domain; } + +/* + * SPIs are used for interrupts of PCI devices and SPIs is managed via GICD + * registers which Hyper-V already supports, so no hypercall needed. + */ +static void hv_arch_irq_unmask(struct irq_data *data) { } #endif /* CONFIG_ARM64 */ /** @@ -1466,119 +1587,9 @@ static void hv_irq_mask(struct irq_data *data) irq_chip_mask_parent(data); } -/** - * hv_irq_unmask() - "Unmask" the IRQ by setting its current - * affinity. - * @data: Describes the IRQ - * - * Build new a destination for the MSI and make a hypercall to - * update the Interrupt Redirection Table. "Device Logical ID" - * is built out of this PCI bus's instance GUID and the function - * number of the device. - */ static void hv_irq_unmask(struct irq_data *data) { - struct msi_desc *msi_desc = irq_data_get_msi_desc(data); - struct hv_retarget_device_interrupt *params; - struct hv_pcibus_device *hbus; - struct cpumask *dest; - cpumask_var_t tmp; - struct pci_bus *pbus; - struct pci_dev *pdev; - unsigned long flags; - u32 var_size = 0; - int cpu, nr_bank; - u64 res; - - dest = irq_data_get_effective_affinity_mask(data); - pdev = msi_desc_to_pci_dev(msi_desc); - pbus = pdev->bus; - hbus = container_of(pbus->sysdata, struct hv_pcibus_device, sysdata); - - spin_lock_irqsave(&hbus->retarget_msi_interrupt_lock, flags); - - params = &hbus->retarget_msi_interrupt_params; - memset(params, 0, sizeof(*params)); - params->partition_id = HV_PARTITION_ID_SELF; - params->int_entry.source = HV_INTERRUPT_SOURCE_MSI; - hv_set_msi_entry_from_desc(¶ms->int_entry.msi_entry, msi_desc); - params->device_id = (hbus->hdev->dev_instance.b[5] << 24) | - (hbus->hdev->dev_instance.b[4] << 16) | - (hbus->hdev->dev_instance.b[7] << 8) | - (hbus->hdev->dev_instance.b[6] & 0xf8) | - PCI_FUNC(pdev->devfn); - params->int_target.vector = hv_msi_get_int_vector(data); - - /* - * Honoring apic->delivery_mode set to APIC_DELIVERY_MODE_FIXED by - * setting the HV_DEVICE_INTERRUPT_TARGET_MULTICAST flag results in a - * spurious interrupt storm. Not doing so does not seem to have a - * negative effect (yet?). - */ - - if (hbus->protocol_version >= PCI_PROTOCOL_VERSION_1_2) { - /* - * PCI_PROTOCOL_VERSION_1_2 supports the VP_SET version of the - * HVCALL_RETARGET_INTERRUPT hypercall, which also coincides - * with >64 VP support. - * ms_hyperv.hints & HV_X64_EX_PROCESSOR_MASKS_RECOMMENDED - * is not sufficient for this hypercall. - */ - params->int_target.flags |= - HV_DEVICE_INTERRUPT_TARGET_PROCESSOR_SET; - - if (!alloc_cpumask_var(&tmp, GFP_ATOMIC)) { - res = 1; - goto exit_unlock; - } - - cpumask_and(tmp, dest, cpu_online_mask); - nr_bank = cpumask_to_vpset(¶ms->int_target.vp_set, tmp); - free_cpumask_var(tmp); - - if (nr_bank <= 0) { - res = 1; - goto exit_unlock; - } - - /* - * var-sized hypercall, var-size starts after vp_mask (thus - * vp_set.format does not count, but vp_set.valid_bank_mask - * does). - */ - var_size = 1 + nr_bank; - } else { - for_each_cpu_and(cpu, dest, cpu_online_mask) { - params->int_target.vp_mask |= - (1ULL << hv_cpu_number_to_vp_number(cpu)); - } - } - - res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT | (var_size << 17), - params, NULL); - -exit_unlock: - spin_unlock_irqrestore(&hbus->retarget_msi_interrupt_lock, flags); - - /* - * During hibernation, when a CPU is offlined, the kernel tries - * to move the interrupt to the remaining CPUs that haven't - * been offlined yet. In this case, the below hv_do_hypercall() - * always fails since the vmbus channel has been closed: - * refer to cpu_disable_common() -> fixup_irqs() -> - * irq_migrate_all_off_this_cpu() -> migrate_one_irq(). - * - * Suppress the error message for hibernation because the failure - * during hibernation does not matter (at this time all the devices - * have been frozen). Note: the correct affinity info is still updated - * into the irqdata data structure in migrate_one_irq() -> - * irq_do_set_affinity() -> hv_set_affinity(), so later when the VM - * resumes, hv_pci_restore_msi_state() is able to correctly restore - * the interrupt with the correct affinity. - */ - if (!hv_result_success(res) && hbus->state != hv_pcibus_removing) - dev_err(&hbus->hdev->device, - "%s() failed: %#llx", __func__, res); + hv_arch_irq_unmask(data); if (data->parent_data->chip->irq_unmask) irq_chip_unmask_parent(data); From patchwork Wed Jul 13 12:56:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Gardner X-Patchwork-Id: 1656001 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=Xoo4riQH; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Ljd2Z0xPPz9ryY for ; Wed, 13 Jul 2022 22:57:58 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1oBbw2-0007Hm-9F; Wed, 13 Jul 2022 12:57:46 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1oBbvg-0006Ue-S7 for kernel-team@lists.ubuntu.com; Wed, 13 Jul 2022 12:57:24 +0000 Received: from mail-pj1-f72.google.com (mail-pj1-f72.google.com [209.85.216.72]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 06E343FA41 for ; Wed, 13 Jul 2022 12:57:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1657717041; bh=OnNU/4WCrGHDVM30GM6xhwGIRItplorDh29BbMws+q8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Xoo4riQHG/IXmlVZ12kiXlvFCOiQdhIL2isCt431CrPiV9IaiKe71TVvTRyAxI/k2 kT+BoCf24H8WOeXFEG56iqFRoCIPzA7VgAZmVFxxerGuCDJs0NaFYFbMsP5PHl6iNa AtjJcMQDohmtb7QqgopncZG3ALdMBIrzly6JKssSVk5DWobucPjT8M0ZrU6d7/A4jr IuG+rY/g5W+xNqVhA9/XdopyLY7ay4uD6wYm+vgt+H1+A/69rZswjYUAxVOowj+6qO 320LcP8tdJmYoy8I+7QI9iOCK+ptDJhZW3u7ER3+ZrjCVxNbWCg1FfpEi8riTdj/vY JHFDy8pcCMoAA== Received: by mail-pj1-f72.google.com with SMTP id i15-20020a17090a2a0f00b001ef826b921dso1477206pjd.5 for ; Wed, 13 Jul 2022 05:57:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OnNU/4WCrGHDVM30GM6xhwGIRItplorDh29BbMws+q8=; b=uEs8uMHvj7W1Q/Y3ttxkFXpuRlIlDLujjrztAhGtDUMNaPFBsc9RhTC45lVXQOd5LA etCmczHuf5WXMpDDXwxJSy3zP+eGfeEkMPa2qCKDJCT587GOL0svgSZtTGdF5mXZuEAF XhHsx0nhgMcaG0Ea9CMrzcCHPwAaMlYI5x1gkyFYJx0X69Ja01p5FTLbE0tRFDnls2hw APzcuzQjUfLAoE/HrytlhWlvtflHzzobWOVowcDwJMbe1As6KiTKAaZbhv0wk4HjCLIj 1dR9pBgpQ+LZvoiLv/m6Nxs/WiMN3KFhHmg5JLZahSrrjIgJlo4W1v1Zmn4qiIYUiyJ7 WnQg== X-Gm-Message-State: AJIora/rooly6eiDCh1z6j2yyTUwZ7gpOOtr3zhlPSA/99zNRwOoEPQ+ 5mnjOkTajXA4KmSP5nAuPv3Q/717ZlKeyCTANPC8BsBWJWSEZgcoNHgOgkN1X/AQ+E2iXlqUgo1 mNcQTZrOSp251WfW3y2KvFXmM8ZadR1Dk1TpMHzhlXQ== X-Received: by 2002:a17:90a:e7c8:b0:1f0:c82:c89d with SMTP id kb8-20020a17090ae7c800b001f00c82c89dmr3745027pjb.108.1657717033732; Wed, 13 Jul 2022 05:57:13 -0700 (PDT) X-Google-Smtp-Source: AGRyM1sg61urkUNs8w0KA2ctMxM+NxuBewqDa85XuR9PWIY6orOrubElO17RLTt5sby7jG6sbr+k/g== X-Received: by 2002:a17:90a:e7c8:b0:1f0:c82:c89d with SMTP id kb8-20020a17090ae7c800b001f00c82c89dmr3744995pjb.108.1657717033468; Wed, 13 Jul 2022 05:57:13 -0700 (PDT) Received: from localhost.localdomain ([69.163.84.166]) by smtp.gmail.com with ESMTPSA id k91-20020a17090a14e400b001ec798b0dc4sm1551135pja.38.2022.07.13.05.57.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:57:12 -0700 (PDT) From: Tim Gardner To: kernel-team@lists.ubuntu.com Subject: [PATCH 09/11][jammy/linux-azure] PCI: hv: Fix hv_arch_irq_unmask() for multi-MSI Date: Wed, 13 Jul 2022 06:56:47 -0600 Message-Id: <20220713125649.24063-14-tim.gardner@canonical.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713125649.24063-1-tim.gardner@canonical.com> References: <20220713125649.24063-1-tim.gardner@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Jeffrey Hugo BugLink: https://bugs.launchpad.net/bugs/1981577 In the multi-MSI case, hv_arch_irq_unmask() will only operate on the first MSI of the N allocated. This is because only the first msi_desc is cached and it is shared by all the MSIs of the multi-MSI block. This means that hv_arch_irq_unmask() gets the correct address, but the wrong data (always 0). This can break MSIs. Lets assume MSI0 is vector 34 on CPU0, and MSI1 is vector 33 on CPU0. hv_arch_irq_unmask() is called on MSI0. It uses a hypercall to configure the MSI address and data (0) to vector 34 of CPU0. This is correct. Then hv_arch_irq_unmask is called on MSI1. It uses another hypercall to configure the MSI address and data (0) to vector 33 of CPU0. This is wrong, and results in both MSI0 and MSI1 being routed to vector 33. Linux will observe extra instances of MSI1 and no instances of MSI0 despite the endpoint device behaving correctly. For the multi-MSI case, we need unique address and data info for each MSI, but the cached msi_desc does not provide that. However, that information can be gotten from the int_desc cached in the chip_data by compose_msi_msg(). Fix the multi-MSI case to use that cached information instead. Since hv_set_msi_entry_from_desc() is no longer applicable, remove it. Signed-off-by: Jeffrey Hugo Reviewed-by: Michael Kelley Link: https://lore.kernel.org/r/1651068453-29588-1-git-send-email-quic_jhugo@quicinc.com Signed-off-by: Wei Liu (cherry picked from commit 455880dfe292a2bdd3b4ad6a107299fce610e64b) Signed-off-by: Tim Gardner --- drivers/pci/controller/pci-hyperv.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index 2244338a56ba..317d8d8e52e9 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -605,13 +605,6 @@ static unsigned int hv_msi_get_int_vector(struct irq_data *data) return cfg->vector; } -static void hv_set_msi_entry_from_desc(union hv_msi_entry *msi_entry, - struct msi_desc *msi_desc) -{ - msi_entry->address.as_uint32 = msi_desc->msg.address_lo; - msi_entry->data.as_uint32 = msi_desc->msg.data; -} - static int hv_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec, msi_alloc_info_t *info) { @@ -641,6 +634,7 @@ static void hv_arch_irq_unmask(struct irq_data *data) { struct msi_desc *msi_desc = irq_data_get_msi_desc(data); struct hv_retarget_device_interrupt *params; + struct tran_int_desc *int_desc; struct hv_pcibus_device *hbus; struct cpumask *dest; cpumask_var_t tmp; @@ -655,6 +649,7 @@ static void hv_arch_irq_unmask(struct irq_data *data) pdev = msi_desc_to_pci_dev(msi_desc); pbus = pdev->bus; hbus = container_of(pbus->sysdata, struct hv_pcibus_device, sysdata); + int_desc = data->chip_data; spin_lock_irqsave(&hbus->retarget_msi_interrupt_lock, flags); @@ -662,7 +657,8 @@ static void hv_arch_irq_unmask(struct irq_data *data) memset(params, 0, sizeof(*params)); params->partition_id = HV_PARTITION_ID_SELF; params->int_entry.source = HV_INTERRUPT_SOURCE_MSI; - hv_set_msi_entry_from_desc(¶ms->int_entry.msi_entry, msi_desc); + params->int_entry.msi_entry.address.as_uint32 = int_desc->address & 0xffffffff; + params->int_entry.msi_entry.data.as_uint32 = int_desc->data; params->device_id = (hbus->hdev->dev_instance.b[5] << 24) | (hbus->hdev->dev_instance.b[4] << 16) | (hbus->hdev->dev_instance.b[7] << 8) | From patchwork Wed Jul 13 12:56:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Gardner X-Patchwork-Id: 1656000 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=OY4HJcfp; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Ljd2V24zcz9ryY for ; Wed, 13 Jul 2022 22:57:54 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1oBbvy-0007AL-U0; Wed, 13 Jul 2022 12:57:42 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1oBbva-0006Ny-52 for kernel-team@lists.ubuntu.com; Wed, 13 Jul 2022 12:57:18 +0000 Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id B27B53F0C5 for ; Wed, 13 Jul 2022 12:57:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1657717036; bh=IjYDlEWRsVuVk6/h9l8F75lMwrJl/ajSfKVNogouIWk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=OY4HJcfpA/knBMOOQfkyp6z7FBtSs/HPpD0bF3PW67Kfg4osEiF+7t5LVHQqzkTR7 4IHNJ8gry3W7hCh0LK1N99huVt7FZRYEOqzgdEZ3e+XThwl0ixJhvQzfMvrwjoasYM rrJtKRuIJ6ejRqhf1jqJD8dvlALB79/XreTvul2cPWiu0duCilIMfuU2tYJIOCISnK 5Lvpqpdy1CJFayTJd3oGyuQr/ssUQaASvt+nCpwYu72YTbvvbDLyrZ0KLEHe+MpzGQ VgRvNu1C05v6KcU6K/gRNzq0HiA6y6CYNH8CgzwuQcsGT5dznucsa2YhNaPcRmsqpx vGOXlvB3z7N/A== Received: by mail-pf1-f199.google.com with SMTP id d18-20020aa78692000000b0052abaa9a6bbso3896937pfo.2 for ; Wed, 13 Jul 2022 05:57:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IjYDlEWRsVuVk6/h9l8F75lMwrJl/ajSfKVNogouIWk=; b=Im/b9C1+AhCxqi25da3JQOpoukPjXQ42zpPrFAUyGWT2nPihmZn1CvaVxtYDog7E3x yd25uBS/SBH8s4keTyuUawdpvHYqk4O/HQA4+rXr4v89DAhCgtdnOAYihcR7aPrveCtW r6lug6aal3GnDmbE+YvqWgcRXtupuXI7f+/PvBbZbNTtbEW/Sn6HGizE0Xj7udkC8Vt1 ME/Y8/shhuyH1+KD1tfDHIVOm8lFbo7237GPWCyYLavC/jzIXj43E5qRrfvJk62BHl89 zN6/yUDfgjh3y+jK0u8EVDV+42Rb56xOi+hnpHP8Miap7XRXUno17A29PtxZojXUpyl2 xI3Q== X-Gm-Message-State: AJIora/jKurGnf3s25RL1CaBk+jpjuxwZjQb+vkM0ljAUjcr72OHh4j3 tBrFvhgOvZ5bk7gGZfnOst0MRV/2EKcnYFFN64mtkETQxL+mRX8/GZ/P0t6qVPSJJ2ssqyXJuNp s01UEBXPWbJCN9lTutbSy0eWcuCmL98tx3tXXskO+Dw== X-Received: by 2002:a62:b60d:0:b0:528:99aa:da09 with SMTP id j13-20020a62b60d000000b0052899aada09mr3248535pff.86.1657717034846; Wed, 13 Jul 2022 05:57:14 -0700 (PDT) X-Google-Smtp-Source: AGRyM1t6L5Vwxh0buY+Ea+6LTX/5DXlXBVXzxutJFgRcT5eWKwz0FXTbAjoBQPB/MZq+HPwHu6DyIg== X-Received: by 2002:a62:b60d:0:b0:528:99aa:da09 with SMTP id j13-20020a62b60d000000b0052899aada09mr3248520pff.86.1657717034571; Wed, 13 Jul 2022 05:57:14 -0700 (PDT) Received: from localhost.localdomain ([69.163.84.166]) by smtp.gmail.com with ESMTPSA id k91-20020a17090a14e400b001ec798b0dc4sm1551135pja.38.2022.07.13.05.57.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:57:13 -0700 (PDT) From: Tim Gardner To: kernel-team@lists.ubuntu.com Subject: [PATCH 10/11][jammy/linux-azure] PCI: hv: Reuse existing IRTE allocation in compose_msi_msg() Date: Wed, 13 Jul 2022 06:56:48 -0600 Message-Id: <20220713125649.24063-15-tim.gardner@canonical.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713125649.24063-1-tim.gardner@canonical.com> References: <20220713125649.24063-1-tim.gardner@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Jeffrey Hugo BugLink: https://bugs.launchpad.net/bugs/1981577 Currently if compose_msi_msg() is called multiple times, it will free any previous IRTE allocation, and generate a new allocation. While nothing prevents this from occurring, it is extraneous when Linux could just reuse the existing allocation and avoid a bunch of overhead. However, when future IRTE allocations operate on blocks of MSIs instead of a single line, freeing the allocation will impact all of the lines. This could cause an issue where an allocation of N MSIs occurs, then some of the lines are retargeted, and finally the allocation is freed/reallocated. The freeing of the allocation removes all of the configuration for the entire block, which requires all the lines to be retargeted, which might not happen since some lines might already be unmasked/active. Signed-off-by: Jeffrey Hugo Reviewed-by: Dexuan Cui Tested-by: Dexuan Cui Tested-by: Michael Kelley Link: https://lore.kernel.org/r/1652282582-21595-1-git-send-email-quic_jhugo@quicinc.com Signed-off-by: Wei Liu (cherry picked from commit b4b77778ecc5bfbd4e77de1b2fd5c1dd3c655f1f) Signed-off-by: Tim Gardner --- drivers/pci/controller/pci-hyperv.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index 317d8d8e52e9..db8e2a570494 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -1709,6 +1709,15 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) u32 size; int ret; + /* Reuse the previous allocation */ + if (data->chip_data) { + int_desc = data->chip_data; + msg->address_hi = int_desc->address >> 32; + msg->address_lo = int_desc->address & 0xffffffff; + msg->data = int_desc->data; + return; + } + pdev = msi_desc_to_pci_dev(irq_data_get_msi_desc(data)); dest = irq_data_get_effective_affinity_mask(data); pbus = pdev->bus; @@ -1718,13 +1727,6 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) if (!hpdev) goto return_null_message; - /* Free any previous message that might have already been composed. */ - if (data->chip_data) { - int_desc = data->chip_data; - data->chip_data = NULL; - hv_int_desc_free(hpdev, int_desc); - } - int_desc = kzalloc(sizeof(*int_desc), GFP_ATOMIC); if (!int_desc) goto drop_reference; From patchwork Wed Jul 13 12:56:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Gardner X-Patchwork-Id: 1656002 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=PHnnjJj8; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Ljd2Z1T2nz9s1l for ; Wed, 13 Jul 2022 22:57:58 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1oBbw1-0007Em-EU; Wed, 13 Jul 2022 12:57:45 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1oBbvf-0006TV-Ay for kernel-team@lists.ubuntu.com; Wed, 13 Jul 2022 12:57:23 +0000 Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 01C7A3F0CB for ; Wed, 13 Jul 2022 12:57:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1657717037; bh=pOSQkYq/Q55E/ENe7jvyUgwKZkt8QrarxouvPuvUMrY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=PHnnjJj8I+1G3izpgVHWctfTKxIt6ajq5FBamcACGPcTCi2o7UB/KD9a/FfWTaJvV j8LA1niN1s5grWdjbJ++pgYAu2652MtLZtV53bYYLbn5tlziT3zrmfuaabLyWYMTke qGPkBKqKhXsCFs3RGGYT5s8lPuaTdEakvMpqgCbKLtR3oEQjqkQI3Oj9w5CbwJK7FF 3dpYRi/x8XVXjfikoT1QfvPLhGTT0vs+vi8EFu/0AekPumTiFY0/LmkKJrIT2YeM00 lXvhqS5O4rs3Xd8IGKM9T4/kvji+GSKiaiht9qLGEsALhOcvZdSe8GbltbuEnvvupK b4FkV8rzglNdw== Received: by mail-pf1-f198.google.com with SMTP id c21-20020a624e15000000b0052abf43401fso3871837pfb.14 for ; Wed, 13 Jul 2022 05:57:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pOSQkYq/Q55E/ENe7jvyUgwKZkt8QrarxouvPuvUMrY=; b=TQYRCUO2YDqFPzgE9EvKsEHb3IOH8WVScEmv2GCdPW0Gf+1DsChBjM1r6Z9TdUDFeg tnMrbu1W/m0JdqB9syvgRwUZPFIxFefnp+tDZ/AufwCq753XXxlsDBR4lUhUYU95lgoe CTZVI8Efizij94zHoU3zzofmMXcWDA+I9PdgYEzAG93jQHrq+qRhdkwrADrVvS89uymJ yY8UdBQXfADUc1YNpZ2dIZBPev1s8qitajItULCRDMXJBuEasXHL1/bwO/vsXZ2+2UPS 88onnVraFKH2L5UciAt8jtQsLs/xEgVumrzFbr63chJ4CiQZPKXOrgu0QdNyaWVef/8K ezhw== X-Gm-Message-State: AJIora/dKZN+kNACLMaVnKiEkAppQnHaGpbQSbcqEQDtcsJd79+N61iS 2BBV8ssUd6RrW4TBn+tzyoiH3EVI3Pn9XdDNw2/UfzoUZ7yZPMZ5PXmZD6UZtjWXcUYjdEZmGiA ZKjy1kQ+BBvKTN8suWN7gZL2U1aVKHrQgmWaazbR+bQ== X-Received: by 2002:a17:902:e5d1:b0:16b:ece4:79e6 with SMTP id u17-20020a170902e5d100b0016bece479e6mr3134273plf.83.1657717036222; Wed, 13 Jul 2022 05:57:16 -0700 (PDT) X-Google-Smtp-Source: AGRyM1syz0pay/BVTYZzEGniIKPB2sq8dPtQ46UDblTTWF0JJfK9VFSqrfhmDpjwekoqWpRzGqE8sw== X-Received: by 2002:a17:902:e5d1:b0:16b:ece4:79e6 with SMTP id u17-20020a170902e5d100b0016bece479e6mr3134253plf.83.1657717035898; Wed, 13 Jul 2022 05:57:15 -0700 (PDT) Received: from localhost.localdomain ([69.163.84.166]) by smtp.gmail.com with ESMTPSA id k91-20020a17090a14e400b001ec798b0dc4sm1551135pja.38.2022.07.13.05.57.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:57:15 -0700 (PDT) From: Tim Gardner To: kernel-team@lists.ubuntu.com Subject: [PATCH 11/11][jammy/linux-azure] PCI: hv: Fix interrupt mapping for multi-MSI Date: Wed, 13 Jul 2022 06:56:49 -0600 Message-Id: <20220713125649.24063-16-tim.gardner@canonical.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713125649.24063-1-tim.gardner@canonical.com> References: <20220713125649.24063-1-tim.gardner@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Jeffrey Hugo BugLink: https://bugs.launchpad.net/bugs/1981577 According to Dexuan, the hypervisor folks beleive that multi-msi allocations are not correct. compose_msi_msg() will allocate multi-msi one by one. However, multi-msi is a block of related MSIs, with alignment requirements. In order for the hypervisor to allocate properly aligned and consecutive entries in the IOMMU Interrupt Remapping Table, there should be a single mapping request that requests all of the multi-msi vectors in one shot. Dexuan suggests detecting the multi-msi case and composing a single request related to the first MSI. Then for the other MSIs in the same block, use the cached information. This appears to be viable, so do it. Suggested-by: Dexuan Cui Signed-off-by: Jeffrey Hugo Reviewed-by: Dexuan Cui Tested-by: Michael Kelley Link: https://lore.kernel.org/r/1652282599-21643-1-git-send-email-quic_jhugo@quicinc.com Signed-off-by: Wei Liu (cherry picked from commit a2bad844a67b1c7740bda63e87453baf63c3a7f7) Signed-off-by: Tim Gardner --- drivers/pci/controller/pci-hyperv.c | 60 ++++++++++++++++++++++++----- 1 file changed, 50 insertions(+), 10 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index db8e2a570494..7dfaf869ea1d 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -1527,6 +1527,10 @@ static void hv_int_desc_free(struct hv_pci_dev *hpdev, u8 buffer[sizeof(struct pci_delete_interrupt)]; } ctxt; + if (!int_desc->vector_count) { + kfree(int_desc); + return; + } memset(&ctxt, 0, sizeof(ctxt)); int_pkt = (struct pci_delete_interrupt *)&ctxt.pkt.message; int_pkt->message_type.type = @@ -1611,12 +1615,12 @@ static void hv_pci_compose_compl(void *context, struct pci_response *resp, static u32 hv_compose_msi_req_v1( struct pci_create_interrupt *int_pkt, struct cpumask *affinity, - u32 slot, u8 vector) + u32 slot, u8 vector, u8 vector_count) { int_pkt->message_type.type = PCI_CREATE_INTERRUPT_MESSAGE; int_pkt->wslot.slot = slot; int_pkt->int_desc.vector = vector; - int_pkt->int_desc.vector_count = 1; + int_pkt->int_desc.vector_count = vector_count; int_pkt->int_desc.delivery_mode = DELIVERY_MODE; /* @@ -1639,14 +1643,14 @@ static int hv_compose_msi_req_get_cpu(struct cpumask *affinity) static u32 hv_compose_msi_req_v2( struct pci_create_interrupt2 *int_pkt, struct cpumask *affinity, - u32 slot, u8 vector) + u32 slot, u8 vector, u8 vector_count) { int cpu; int_pkt->message_type.type = PCI_CREATE_INTERRUPT_MESSAGE2; int_pkt->wslot.slot = slot; int_pkt->int_desc.vector = vector; - int_pkt->int_desc.vector_count = 1; + int_pkt->int_desc.vector_count = vector_count; int_pkt->int_desc.delivery_mode = DELIVERY_MODE; cpu = hv_compose_msi_req_get_cpu(affinity); int_pkt->int_desc.processor_array[0] = @@ -1658,7 +1662,7 @@ static u32 hv_compose_msi_req_v2( static u32 hv_compose_msi_req_v3( struct pci_create_interrupt3 *int_pkt, struct cpumask *affinity, - u32 slot, u32 vector) + u32 slot, u32 vector, u8 vector_count) { int cpu; @@ -1666,7 +1670,7 @@ static u32 hv_compose_msi_req_v3( int_pkt->wslot.slot = slot; int_pkt->int_desc.vector = vector; int_pkt->int_desc.reserved = 0; - int_pkt->int_desc.vector_count = 1; + int_pkt->int_desc.vector_count = vector_count; int_pkt->int_desc.delivery_mode = DELIVERY_MODE; cpu = hv_compose_msi_req_get_cpu(affinity); int_pkt->int_desc.processor_array[0] = @@ -1697,6 +1701,8 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) struct cpumask *dest; struct compose_comp_ctxt comp; struct tran_int_desc *int_desc; + struct msi_desc *msi_desc; + u8 vector, vector_count; struct { struct pci_packet pci_pkt; union { @@ -1718,7 +1724,8 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) return; } - pdev = msi_desc_to_pci_dev(irq_data_get_msi_desc(data)); + msi_desc = irq_data_get_msi_desc(data); + pdev = msi_desc_to_pci_dev(msi_desc); dest = irq_data_get_effective_affinity_mask(data); pbus = pdev->bus; hbus = container_of(pbus->sysdata, struct hv_pcibus_device, sysdata); @@ -1731,6 +1738,36 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) if (!int_desc) goto drop_reference; + if (!msi_desc->pci.msi_attrib.is_msix && msi_desc->nvec_used > 1) { + /* + * If this is not the first MSI of Multi MSI, we already have + * a mapping. Can exit early. + */ + if (msi_desc->irq != data->irq) { + data->chip_data = int_desc; + int_desc->address = msi_desc->msg.address_lo | + (u64)msi_desc->msg.address_hi << 32; + int_desc->data = msi_desc->msg.data + + (data->irq - msi_desc->irq); + msg->address_hi = msi_desc->msg.address_hi; + msg->address_lo = msi_desc->msg.address_lo; + msg->data = int_desc->data; + put_pcichild(hpdev); + return; + } + /* + * The vector we select here is a dummy value. The correct + * value gets sent to the hypervisor in unmask(). This needs + * to be aligned with the count, and also not zero. Multi-msi + * is powers of 2 up to 32, so 32 will always work here. + */ + vector = 32; + vector_count = msi_desc->nvec_used; + } else { + vector = hv_msi_get_int_vector(data); + vector_count = 1; + } + memset(&ctxt, 0, sizeof(ctxt)); init_completion(&comp.comp_pkt.host_event); ctxt.pci_pkt.completion_func = hv_pci_compose_compl; @@ -1741,7 +1778,8 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) size = hv_compose_msi_req_v1(&ctxt.int_pkts.v1, dest, hpdev->desc.win_slot.slot, - hv_msi_get_int_vector(data)); + vector, + vector_count); break; case PCI_PROTOCOL_VERSION_1_2: @@ -1749,14 +1787,16 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) size = hv_compose_msi_req_v2(&ctxt.int_pkts.v2, dest, hpdev->desc.win_slot.slot, - hv_msi_get_int_vector(data)); + vector, + vector_count); break; case PCI_PROTOCOL_VERSION_1_4: size = hv_compose_msi_req_v3(&ctxt.int_pkts.v3, dest, hpdev->desc.win_slot.slot, - hv_msi_get_int_vector(data)); + vector, + vector_count); break; default: