From patchwork Tue Mar 6 19:29:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander H Duyck X-Patchwork-Id: 882199 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="D5d4+Za+"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zwn1t57fzz9sgq for ; Wed, 7 Mar 2018 06:30:58 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933229AbeCFT3q (ORCPT ); Tue, 6 Mar 2018 14:29:46 -0500 Received: from mail-io0-f196.google.com ([209.85.223.196]:41779 "EHLO mail-io0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753851AbeCFT3n (ORCPT ); Tue, 6 Mar 2018 14:29:43 -0500 Received: by mail-io0-f196.google.com with SMTP id q24so218514ioh.8; Tue, 06 Mar 2018 11:29:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=1fM5uRqqiFdWRpppIMODoqapUkIE1/t+jZTZNsmPETo=; b=D5d4+Za+IzmF6D6ehoNaIsJ8+PxDybINsbGigBVNXJHafgHMxdVLd4/wH9sryF2pYh 6T2WZ/NGLPNE1FhF33/p26P/SqsABaN2C/eFyaJk2ePnja10HjNuBA3RvVvZpZSPKMmf XWylsM029nD0F+lCDiQyuLCaGQc9ujhWzD0L3SLeACYDc5ZZiuXO9bE079voWi0rif8r Tgr+w4yC4V37rtD+h95awyWnwAS8gtVyPpBvBrqmUgEhBukgxYudZ03HE/1k5NvS4K+N DzChvlKHyYeqIIHS4VUXu1ohVvUrQnXBm+dnJ94TeQmfWXwZTTJ8SKG3V05J0XGKpqJu eulg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=1fM5uRqqiFdWRpppIMODoqapUkIE1/t+jZTZNsmPETo=; b=c+RIZQLmBunyW7Y/qbSz9UHm1vZMDB6Ni871WpwfkjVAGmE4ZGttDBKu9ww5jIZbbF pCNZLgp91uDDR5EdLM0SeKaZQfoiseQtPzyVATBNYa2NbluA8uBWpLN+b9CEkKJxvamF jos2LmCWF9QmEywsuN0UdtmQbM26gQtrW3OcUwKzBLvu7TAwRwjjjCMDevIVbM7YOWrx nbAEDP7/V+cTWp+BiYtBZSvUfV6CSApUwxX/RZ+4/vMKEKATY25WqqXUWxdrhwkqCaji AzYmTULCeCrdYS3j3N7a2Nnb+mKB1haOM4VOECmkejJwu57cao/Pcmyu6Bri/tbMSpr8 kyLw== X-Gm-Message-State: APf1xPDzt3haDGL1jgGFPqDPTphYPjOs2wx2ROVlEunF8ce4LOD14LgU MXtyCKGrXAEQrDW1kppJDS8= X-Google-Smtp-Source: AG47ELt2QduZ5WJ1zm04dOfwPAZwfuwd38083DDz/Bw6Uyf0xxAAQStHXgcXImw19d3dgjIhVOZSbw== X-Received: by 10.107.102.13 with SMTP id a13mr22929586ioc.61.1520364582782; Tue, 06 Mar 2018 11:29:42 -0800 (PST) Received: from localhost.localdomain ([2001:470:b:9c3:9e5c:8eff:fe4f:f2d0]) by smtp.gmail.com with ESMTPSA id p194sm7285849itp.34.2018.03.06.11.29.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Mar 2018 11:29:42 -0800 (PST) Subject: [pci PATCH v3 1/3] pci-iov: Add support for unmanaged SR-IOV From: Alexander Duyck To: bhelgaas@google.com, alexander.h.duyck@intel.com, linux-pci@vger.kernel.org Cc: virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org, netdev@vger.kernel.org, dan.daly@intel.com, linux-kernel@vger.kernel.org, mheyne@amazon.de, liang-min.wang@intel.com, mark.d.rustad@intel.com, dwmw2@infradead.org, dwmw@amazon.co.uk Date: Tue, 06 Mar 2018 11:29:40 -0800 Message-ID: <20180306192915.3153.86465.stgit@localhost.localdomain> In-Reply-To: <20180306192423.3153.42741.stgit@localhost.localdomain> References: <20180306192423.3153.42741.stgit@localhost.localdomain> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Alexander Duyck This patch is meant to add some basic functionality to support for SR-IOV on devices when the VFs are not managed by the kernel. The functions provided here can be used by drivers such as vfio-pci and virtio to enable SR-IOV on devices that are either managed by userspace, or by some sort of firmware entity respectively. A new sysfs value called sriov_unmanaged_autoprobe has been added. This value is used as the drivers_autoprobe setting of the VFs when they are being managed by an external entity such as userspace or device firmware instead of being managed by the kernel. One side effect of this change is that the sriov_drivers_autoprobe and sriov_unmanaged_autoprobe will only apply their updates when SR-IOV is disabled. Attempts to update them when SR-IOV is in use will only update the local value and will not update sriov->autoprobe. Signed-off-by: Alexander Duyck --- v3: Updated documentation to better explain sriov_unmanaged_autoprobe use Documentation/ABI/testing/sysfs-bus-pci | 24 ++++++++++++++++++++ drivers/pci/iov.c | 37 +++++++++++++++++++++++++++++++ drivers/pci/pci-driver.c | 2 +- drivers/pci/pci-sysfs.c | 29 ++++++++++++++++++++++++ drivers/pci/pci.h | 4 +++ include/linux/pci.h | 1 + 6 files changed, 95 insertions(+), 2 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci index 44d4b2be92fd..d9d20611fc91 100644 --- a/Documentation/ABI/testing/sysfs-bus-pci +++ b/Documentation/ABI/testing/sysfs-bus-pci @@ -323,3 +323,27 @@ Description: This is similar to /sys/bus/pci/drivers_autoprobe, but affects only the VFs associated with a specific PF. + +What: /sys/bus/pci/devices/.../sriov_unmanaged_autoprobe +Date: March 2018 +Contact: Alexander Duyck +Description: + This file is associated with the PF of a device that + supports SR-IOV. It determines whether newly-enabled VFs + are immediately bound to a driver when the PF driver does + not manage the VFs itself. It initially contains 0, which + means the kernel will not automatically bind VFs to a driver. + If an application writes 1 to the file before enabling VFs, + the kernel will bind VFs to a compatible driver immediately + after they are enabled. + + Currently the use of this setting is limited to drivers that + make use of pci_sriov_configure_unmanaged. Examples might + include drivers such as virtio_net which could expose a PCI + function that resembles a VF with the extra SR-IOV related + bits, or a PF attached to a vfio interface which is being + managed by userspace instead of the kernel directly. + + This overrides /sys/bus/pci/devices/.../sriov_drivers_autoprobe + when a PF driver does not provide functionality to manage the + VFs when SR-IOV is enabled. diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c index 677924ae0350..3dcec1fa86bd 100644 --- a/drivers/pci/iov.c +++ b/drivers/pci/iov.c @@ -446,6 +446,7 @@ static int sriov_init(struct pci_dev *dev, int pos) pci_read_config_word(dev, pos + PCI_SRIOV_VF_DID, &iov->vf_device); iov->pgsz = pgsz; iov->self = dev; + iov->autoprobe = true; iov->drivers_autoprobe = true; pci_read_config_dword(dev, pos + PCI_SRIOV_CAP, &iov->cap); pci_read_config_byte(dev, pos + PCI_SRIOV_FUNC_LINK, &iov->link); @@ -683,6 +684,9 @@ int pci_enable_sriov(struct pci_dev *dev, int nr_virtfn) if (!dev->is_physfn) return -ENOSYS; + /* Update autoprobe setting to reflect managed device */ + dev->sriov->autoprobe = dev->sriov->drivers_autoprobe; + return sriov_enable(dev, nr_virtfn); } EXPORT_SYMBOL_GPL(pci_enable_sriov); @@ -807,3 +811,36 @@ int pci_sriov_get_totalvfs(struct pci_dev *dev) return dev->sriov->total_VFs; } EXPORT_SYMBOL_GPL(pci_sriov_get_totalvfs); + +/** + * pci_sriov_configure_unmanaged - helper to configure unmanaged SR-IOV + * @dev: the PCI device + * @nr_virtfn: number of virtual functions to enable, 0 to disable + * + * Used to provide generic enable/disable SR-IOV option for devices + * that do not manage the VFs generated by their driver, or have no + * driver present. + */ +int pci_sriov_configure_unmanaged(struct pci_dev *dev, int nr_virtfn) +{ + int err; + + might_sleep(); + + if (!dev->is_physfn) + return -ENODEV; + + if (!nr_virtfn) { + sriov_disable(dev); + + return 0; + } + + /* Update autoprobe setting to reflect unmanaged device */ + dev->sriov->autoprobe = dev->sriov->unmanaged_autoprobe; + + err = sriov_enable(dev, nr_virtfn); + + return err ? err : nr_virtfn; +} +EXPORT_SYMBOL_GPL(pci_sriov_configure_unmanaged); diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index 3bed6beda051..2cc68dff6130 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -398,7 +398,7 @@ void __weak pcibios_free_irq(struct pci_dev *dev) #ifdef CONFIG_PCI_IOV static inline bool pci_device_can_probe(struct pci_dev *pdev) { - return (!pdev->is_virtfn || pdev->physfn->sriov->drivers_autoprobe); + return (!pdev->is_virtfn || pdev->physfn->sriov->autoprobe); } #else static inline bool pci_device_can_probe(struct pci_dev *pdev) diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c index eb6bee8724cc..6f78fa73e317 100644 --- a/drivers/pci/pci-sysfs.c +++ b/drivers/pci/pci-sysfs.c @@ -710,6 +710,30 @@ static ssize_t sriov_drivers_autoprobe_store(struct device *dev, return count; } +static ssize_t sriov_unmanaged_autoprobe_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct pci_dev *pdev = to_pci_dev(dev); + + return sprintf(buf, "%u\n", pdev->sriov->unmanaged_autoprobe); +} + +static ssize_t sriov_unmanaged_autoprobe_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct pci_dev *pdev = to_pci_dev(dev); + bool unmanaged_autoprobe; + + if (kstrtobool(buf, &unmanaged_autoprobe) < 0) + return -EINVAL; + + pdev->sriov->unmanaged_autoprobe = unmanaged_autoprobe; + + return count; +} + static struct device_attribute sriov_totalvfs_attr = __ATTR_RO(sriov_totalvfs); static struct device_attribute sriov_numvfs_attr = __ATTR(sriov_numvfs, (S_IRUGO|S_IWUSR|S_IWGRP), @@ -720,6 +744,10 @@ static ssize_t sriov_drivers_autoprobe_store(struct device *dev, static struct device_attribute sriov_drivers_autoprobe_attr = __ATTR(sriov_drivers_autoprobe, (S_IRUGO|S_IWUSR|S_IWGRP), sriov_drivers_autoprobe_show, sriov_drivers_autoprobe_store); +static struct device_attribute sriov_unmanaged_autoprobe_attr = + __ATTR(sriov_unmanaged_autoprobe, (S_IRUGO|S_IWUSR|S_IWGRP), + sriov_unmanaged_autoprobe_show, + sriov_unmanaged_autoprobe_store); #endif /* CONFIG_PCI_IOV */ static ssize_t driver_override_store(struct device *dev, @@ -1789,6 +1817,7 @@ static umode_t pcie_dev_attrs_are_visible(struct kobject *kobj, &sriov_stride_attr.attr, &sriov_vf_device_attr.attr, &sriov_drivers_autoprobe_attr.attr, + &sriov_unmanaged_autoprobe_attr.attr, NULL, }; diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index fcd81911b127..b5f8b034f02d 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -272,7 +272,9 @@ struct pci_sriov { struct pci_dev *dev; /* Lowest numbered PF */ struct pci_dev *self; /* This PF */ resource_size_t barsz[PCI_SRIOV_NUM_BARS]; /* VF BAR size */ - bool drivers_autoprobe; /* Auto probing of VFs by driver */ + bool autoprobe; /* Auto probing of VFs by VF driver */ + bool drivers_autoprobe; /* "" managed by PF driver */ + bool unmanaged_autoprobe; /* "" unmanaged by kernel */ }; /* pci_dev priv_flags */ diff --git a/include/linux/pci.h b/include/linux/pci.h index 024a1beda008..553860a08131 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1953,6 +1953,7 @@ static inline void pci_mmcfg_late_init(void) { } int pci_vfs_assigned(struct pci_dev *dev); int pci_sriov_set_totalvfs(struct pci_dev *dev, u16 numvfs); int pci_sriov_get_totalvfs(struct pci_dev *dev); +int pci_sriov_configure_unmanaged(struct pci_dev *dev, int nr_virtfn); resource_size_t pci_iov_resource_size(struct pci_dev *dev, int resno); void pci_vf_drivers_autoprobe(struct pci_dev *dev, bool probe); #else From patchwork Tue Mar 6 19:29:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander H Duyck X-Patchwork-Id: 882195 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Vmfje/RY"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zwn190t3bz9skm for ; Wed, 7 Mar 2018 06:30:21 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754052AbeCFT35 (ORCPT ); Tue, 6 Mar 2018 14:29:57 -0500 Received: from mail-io0-f196.google.com ([209.85.223.196]:45494 "EHLO mail-io0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754030AbeCFT3x (ORCPT ); Tue, 6 Mar 2018 14:29:53 -0500 Received: by mail-io0-f196.google.com with SMTP id m22so206318iob.12; Tue, 06 Mar 2018 11:29:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=3OGGZ1C4FnkyldqOvS6sTBiiwnMOR617LQEXPowyiC4=; b=Vmfje/RYcoKG6KAJA5NJIVU1rb4R1/J+Y3qpP9iRgXnRB4rMB8fPEOkLDhZ1sHFvvH JinogLlCm0/+dJL2J/SCwlzmehdcoT8EvMckgm1lGfWrm2yYQiw9aZCl5nnB5TIKjUXw 56E3kHD3e8HPamu93WZ5GyLFaB/baylZqzHrF3++FWQIm+gMpX+i4B+GHH3HnpHg0zOL CTJrJQH5fBv63TKL3LOPb7G2yJGrZJvNSMbjhyzCzJwUJklMNwaRMlz1KVGGGvTLfe3C lK9CJr3y5gG3QmrDsFG9lYURAAysElvH0NG/nFljA5PTO3W38931cR0iDR8mVsFiD5xA aXlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=3OGGZ1C4FnkyldqOvS6sTBiiwnMOR617LQEXPowyiC4=; b=a14m2DOGQCebrSDPX2pcjrUX9iMk6gvlsPQAQEsjRA5nSyBXs4SY5ccKDn6OAmL/Jy hYs4BcgyyEtlCHaggBHpBggqgNvEuk/UmnNBD4ksJUv980OVsUDj78MuvtbR/bRKf6uw jzsRdyiFr/QW3B/9X/s4cFvkM1C2pdj5Pqxq2T05T+ofpryLNalE31vDOy/U1FNEfKL/ CesNhd8WoKhlpwtKRETtCb47MvEBmxYc/VpVLLIzJMuRbZ4ILRl9TGD1WZNj/uS7KeRe WdxCxk2wb0hybljbtZxRfDzkkYdPetIhgHsGPz//ci3jHYZxB26lE+1u95mpDCNQNAv3 sm0A== X-Gm-Message-State: APf1xPBVSuInV75ejMCjqFUaARtNSr6hl8YlUc1ScI6uJYn5HKmf7l2d 47fvNc9N3Hlo5ycECMHrvtE= X-Google-Smtp-Source: AG47ELt10uiDrKYg7H4yjwiFkDLzvntpOT/jJRuzJH+hNP6805MYi6HB+PKKl6rZKCTl4Ea0T+HaZA== X-Received: by 10.107.181.197 with SMTP id e188mr23643799iof.243.1520364592231; Tue, 06 Mar 2018 11:29:52 -0800 (PST) Received: from localhost.localdomain ([2001:470:b:9c3:9e5c:8eff:fe4f:f2d0]) by smtp.gmail.com with ESMTPSA id r6sm6769049ioe.85.2018.03.06.11.29.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Mar 2018 11:29:51 -0800 (PST) Subject: [pci PATCH v3 2/3] vfio: Add support for unmanaged or userspace managed SR-IOV From: Alexander Duyck To: bhelgaas@google.com, alexander.h.duyck@intel.com, linux-pci@vger.kernel.org Cc: virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org, netdev@vger.kernel.org, dan.daly@intel.com, linux-kernel@vger.kernel.org, mheyne@amazon.de, liang-min.wang@intel.com, mark.d.rustad@intel.com, dwmw2@infradead.org, dwmw@amazon.co.uk Date: Tue, 06 Mar 2018 11:29:50 -0800 Message-ID: <20180306192947.3153.20106.stgit@localhost.localdomain> In-Reply-To: <20180306192423.3153.42741.stgit@localhost.localdomain> References: <20180306192423.3153.42741.stgit@localhost.localdomain> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Alexander Duyck This patch is meant to allow assignment of an SR-IOV enabled PF, as in VFs have been generated, with vfio-pci. My understanding is the primary use case for this is something like DPDK running the PF while the VFs are all assigned to guests. A secondary effect of this is that it provides an interface through which it would be possible to enable SR-IOV on drivers that may not have a physical function that actually manages the device. Enabling SR-IOV should be pretty straight forward. As long as there are no userspace processes currently controlling the interface the number of VFs can be changed, and VFs will be generated without drivers being loaded on the host. Once the userspace process begins controlling the interface the number of VFs cannot be updated via the sysfs until the control is released. Note the VFs will have drivers load on them in the host if the sriov_unmanaged_autoprobe is updated to a value of 1. However the behavior of the VFs in such a setup cannot be guaranteed as the PF will not be available until the userspace process starts and begins to manage the device. For now I am leaving the value as locked when the PF is being controlled from userspace as a form of synchronization. Basically this way we cannot have the number of VFs change out from under the process so it should not require any notification framework, and the configuration can just be read out via configuration space accesses. Signed-off-by: Alexander Duyck --- drivers/vfio/pci/vfio_pci.c | 59 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index b0f759476900..8025d7336071 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -1224,6 +1224,8 @@ static void vfio_pci_remove(struct pci_dev *pdev) VGA_RSRC_LEGACY_IO | VGA_RSRC_LEGACY_MEM); } + pci_disable_sriov(pdev); + if (!disable_idle_d3) pci_set_power_state(pdev, PCI_D0); } @@ -1260,12 +1262,69 @@ static pci_ers_result_t vfio_pci_aer_err_detected(struct pci_dev *pdev, .error_detected = vfio_pci_aer_err_detected, }; +#ifdef CONFIG_PCI_IOV +static int vfio_pci_sriov_configure(struct pci_dev *pdev, int nr_virtfn) +{ + struct vfio_pci_device *vdev; + struct vfio_device *device; + int err; + + device = vfio_device_get_from_dev(&pdev->dev); + if (device == NULL) + return -ENODEV; + + vdev = vfio_device_data(device); + if (vdev == NULL) { + vfio_device_put(device); + return -ENODEV; + } + + /* + * If a userspace process is already using this device just return + * busy and don't allow for any changes. + */ + if (vdev->refcnt) { + pci_warn(pdev, + "PF is currently in use, blocked until released by user\n"); + return -EBUSY; + } + + err = pci_sriov_configure_unmanaged(pdev, nr_virtfn); + if (err <= 0) + return err; + + /* + * We are now leaving VFs in the control of some unknown PF entity. + * + * Best case is a well behaved userspace PF is expected and any VMs + * that the VFs will be assigned to are dependent on the userspace + * entity anyway. An example being NFV where maybe the PF is acting + * as an accelerated interface for a firewall or switch. + * + * Worst case is somebody really messed up and just enabled SR-IOV + * on a device they were planning to assign to a VM somwhere. + * + * In either case it is probably best for us to set the taint flag + * and warn the user since this could get really ugly really quick + * if this wasn't what they were planning to do. + */ + add_taint(TAINT_USER, LOCKDEP_STILL_OK); + pci_warn(pdev, + "Adding kernel taint for vfio-pci now managing SR-IOV PF device\n"); + + return nr_virtfn; +} +#endif /* CONFIG_PCI_IOV */ + static struct pci_driver vfio_pci_driver = { .name = "vfio-pci", .id_table = NULL, /* only dynamic ids */ .probe = vfio_pci_probe, .remove = vfio_pci_remove, .err_handler = &vfio_err_handlers, +#ifdef CONFIG_PCI_IOV + .sriov_configure = vfio_pci_sriov_configure, +#endif }; struct vfio_devices { From patchwork Tue Mar 6 19:30:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander H Duyck X-Patchwork-Id: 882197 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="AtqU4oXl"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zwn1f5mjQz9sgt for ; Wed, 7 Mar 2018 06:30:46 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754085AbeCFTaY (ORCPT ); Tue, 6 Mar 2018 14:30:24 -0500 Received: from mail-it0-f66.google.com ([209.85.214.66]:54699 "EHLO mail-it0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753466AbeCFTaU (ORCPT ); Tue, 6 Mar 2018 14:30:20 -0500 Received: by mail-it0-f66.google.com with SMTP id c11so240919ith.4; Tue, 06 Mar 2018 11:30:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=CuLv+rs5mWy18or5XaHfw+elVLJSsYvA3+3YSK4r3rg=; b=AtqU4oXlp14dq8zYensoWMSZquvyvLj95ddpv5pKnWPtxMwaWBj8sY4t6R+DnPsHKE LKOg4ptgV1gwnk/wggXlQOtY3eoLOp6D5yUzB1cO9xjftXO2ky4xnqpWWzGDWfpE6cTt 3UTkMXW78oU+awFFbnZarVb3aaLkbMk4tNtgnvHkgNdpIWkGcnG7Qkegd2z9YOL9M/V1 IWhmVhBr140lQ8oPH9ZG9xDIFARiHgztZQTEG80D4Dj1XN6TRh1dtljAGhPXmwijuaZG xXw0ysxYxvPx1Od4ckbguz5tyUNL2q0JoGgxQvkvcBe1aQHUxIvVky8OHkM5LJi0CAnX SyVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=CuLv+rs5mWy18or5XaHfw+elVLJSsYvA3+3YSK4r3rg=; b=cYKmlWtKJVinQDMmDfOgwLlbNSKU/dNxPvaauTXGfQ+rwcarcQjdYZ58lpfKAzSQql Pj6UxPtuBWxTa/kgoAkLKnzUrrKk+5H7v+7c8KNIgbRVNYn5dg03ALlQpcUBy5khyOl3 PK+D3L8kVSavJwT+PTMdqj2Vl456v98Gnc5eaY7XMUWDj+WgVT192ZDSBb3Nrqgef944 Fo0JcsS+u/2eVMneuw6VSVV/k0MzVoXlSWHAGx2LwkZHScZwZZwOUsxHyoDY8zShnjd6 ugsAM8XkIwlkMCcXf6KwMLtKRfDdlYVHwtDi/7eKiBgdP+jNbi2stGrtV3yHDXU3p87Q kvUg== X-Gm-Message-State: AElRT7GV8ox021l9XOQB2RvX55l84yEAY+BOJfSrmJr9zA5AHVZEQuWl 4kM4eF/HKdB4cZN2K7NDH+c= X-Google-Smtp-Source: AG47ELsO7zHOYB5OcA9UsX8MGwO2Qkj2x3nRO7i04UpJKkDeVr+E6R0tyAzPbEhCsWgjRA2EIpo91w== X-Received: by 10.36.53.197 with SMTP id k188mr20138612ita.30.1520364619461; Tue, 06 Mar 2018 11:30:19 -0800 (PST) Received: from localhost.localdomain ([2001:470:b:9c3:9e5c:8eff:fe4f:f2d0]) by smtp.gmail.com with ESMTPSA id v198sm7281006ita.29.2018.03.06.11.30.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Mar 2018 11:30:18 -0800 (PST) Subject: [pci PATCH v3 3/3] virtio_pci: Add support for unmanaged SR-IOV on virtio_pci devices From: Alexander Duyck To: bhelgaas@google.com, alexander.h.duyck@intel.com, linux-pci@vger.kernel.org Cc: virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org, netdev@vger.kernel.org, dan.daly@intel.com, linux-kernel@vger.kernel.org, mheyne@amazon.de, liang-min.wang@intel.com, mark.d.rustad@intel.com, dwmw2@infradead.org, dwmw@amazon.co.uk Date: Tue, 06 Mar 2018 11:30:17 -0800 Message-ID: <20180306192957.3153.60365.stgit@localhost.localdomain> In-Reply-To: <20180306192423.3153.42741.stgit@localhost.localdomain> References: <20180306192423.3153.42741.stgit@localhost.localdomain> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Alexander Duyck Hardware-realized virtio_pci devices can implement SR-IOV, so this patch enables its use. The device in question is an upcoming Intel NIC that implements both a virtio_net PF and virtio_net VFs. These are hardware realizations of what has been up to now been a software interface. The device in question has the following 4-part PCI IDs: PF: vendor: 1af4 device: 1041 subvendor: 8086 subdevice: 15fe VF: vendor: 1af4 device: 1041 subvendor: 8086 subdevice: 05fe The patch currently needs no check for device ID, because the callback will never be made for devices that do not assert the capability or when run on a platform incapable of SR-IOV. One reason for this patch is because the hardware requires the vendor ID of a VF to be the same as the vendor ID of the PF that created it. So it seemed logical to simply have a fully-functioning virtio_net PF create the VFs. This patch makes that possible. Signed-off-by: Mark Rustad Signed-off-by: Alexander Duyck --- drivers/virtio/virtio_pci_common.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c index 48d4d1cf1cb6..ca1549393255 100644 --- a/drivers/virtio/virtio_pci_common.c +++ b/drivers/virtio/virtio_pci_common.c @@ -584,6 +584,7 @@ static void virtio_pci_remove(struct pci_dev *pci_dev) else virtio_pci_modern_remove(vp_dev); + pci_disable_sriov(pci_dev); pci_disable_device(pci_dev); put_device(dev); } @@ -596,6 +597,9 @@ static void virtio_pci_remove(struct pci_dev *pci_dev) #ifdef CONFIG_PM_SLEEP .driver.pm = &virtio_pci_pm_ops, #endif +#ifdef CONFIG_PCI_IOV + .sriov_configure = pci_sriov_configure_unmanaged, +#endif }; module_pci_driver(virtio_pci_driver);