[net-next,7/7] nfp: allow application firmware to limit number of SR-IOV VFs

Some application firmware projects may choose to limit the number
of VFs available below what is specified in PCI capability to be
able to reuse the PCIe interface resources.  There may also be
projects which use cases don't require SR-IOV support at all and
therefore don't want to spend time implementing/testing it.

Check nfd_vf_cfg_max_vfs firmware symbol to see if application
firmware is reporting how many VFs it supports.  This mechanism
is an opt-in, if symbol is not present we will only look at the
PCI capability values.

Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
---
 drivers/net/ethernet/netronome/nfp/nfp_main.c | 23 +++++++++++++++++++++++
 drivers/net/ethernet/netronome/nfp/nfp_main.h |  2 ++
 2 files changed, 25 insertions(+)

> +static void nfp_pcie_sriov_read_nfd_limit(struct nfp_pf *pf)
> +{
> +	int err;
> +
> +	pf->limit_vfs = nfp_rtsym_read_le(pf->cpp, "nfd_vf_cfg_max_vfs",
> &err);
> +	if (!err)
> +		return;
> +
> +	pf->limit_vfs = ~0;
> +	/* Allow any setting for backwards compatibility if symbol not found
> */
> +	if (err != -ENOENT)
> +		nfp_warn(pf->cpp, "Warning: VF limit read failed: %d\n", err);
> +}
> +

If you're only going to use this under CONFIG_PCI_IOV,
perhaps put this underneath the ifdef as well?

>  static int nfp_pcie_sriov_enable(struct pci_dev *pdev, int num_vfs)
>  {
>  #ifdef CONFIG_PCI_IOV
>  	struct nfp_pf *pf = pci_get_drvdata(pdev);
>  	int err;
> 
> +	if (num_vfs > pf->limit_vfs) {
> +		nfp_info(pf->cpp, "Firmware limits number of VFs to %u\n",
> +			 pf->limit_vfs);
> +		return -EINVAL;
> +	}
> +
>  	err = pci_enable_sriov(pdev, num_vfs);
>  	if (err) {
>  		dev_warn(&pdev->dev, "Failed to enable PCI sriov: %d\n",
> err);
> @@ -333,6 +354,8 @@ static int nfp_pci_probe(struct pci_dev *pdev,
>  	if (err)
>  		goto err_cpp_free;
> 
> +	nfp_pcie_sriov_read_nfd_limit(pf);
> +
>  	err = nfp_net_pci_probe(pf);
>  	if (err)
>  		goto err_fw_unload;
> diff --git a/drivers/net/ethernet/netronome/nfp/nfp_main.h
> b/drivers/net/ethernet/netronome/nfp/nfp_main.h
> index 6c40fa322da3..39105d0435e9 100644
> --- a/drivers/net/ethernet/netronome/nfp/nfp_main.h
> +++ b/drivers/net/ethernet/netronome/nfp/nfp_main.h
> @@ -59,6 +59,7 @@ struct nfp_eth_table;
>   * @tx_area:		Pointer to the CPP area for the TX queues
>   * @rx_area:		Pointer to the CPP area for the FL/RX queues
>   * @irq_entries:	Array of MSI-X entries for all ports
> + * @limit_vfs:		Number of VFs supported by firmware (~0 for PCI
> limit)
>   * @num_vfs:		Number of SR-IOV VFs enabled
>   * @fw_loaded:		Is the firmware loaded?
>   * @eth_tbl:		NSP ETH table
> @@ -77,6 +78,7 @@ struct nfp_pf {
> 
>  	struct msix_entry *irq_entries;
> 
> +	unsigned int limit_vfs;
>  	unsigned int num_vfs;
> 
>  	bool fw_loaded;
> --
> 2.11.0

On Sun, Feb 19, 2017 at 8:00 AM, Mintz, Yuval <Yuval.Mintz@cavium.com> wrote:
>> +static void nfp_pcie_sriov_read_nfd_limit(struct nfp_pf *pf)
>> +{
>> +     int err;
>> +
>> +     pf->limit_vfs = nfp_rtsym_read_le(pf->cpp, "nfd_vf_cfg_max_vfs",
>> &err);
>> +     if (!err)
>> +             return;
>> +
>> +     pf->limit_vfs = ~0;
>> +     /* Allow any setting for backwards compatibility if symbol not found
>> */
>> +     if (err != -ENOENT)
>> +             nfp_warn(pf->cpp, "Warning: VF limit read failed: %d\n", err);
>> +}
>> +
>
> If you're only going to use this under CONFIG_PCI_IOV,
> perhaps put this underneath the ifdef as well?

Sure, I have a slight preference to not use ifdefs unless necessary,
but I guess the body of this function could be under an ifdef without
making things too ugly..  Thanks :)  Testing almost done, v2 coming
shortly.