[Xen-devel,v4,18/22] xen/pci: Pay attention to PCI_MSIX_TABLE_OFFSET

On Thu, Apr 25, 2013 at 10:40:57AM +0100, Jan Beulich wrote:
> >>> On 24.04.13 at 18:34, Bjorn Helgaas <bhelgaas@google.com> wrote:
> > On Mon, Apr 22, 2013 at 5:12 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> >> The MSI-X Table structure may be at a non-zero offset into the
> >> device BAR, and we should account for that.
> 
> NAK: The base is just being use to pass to the hypervisor, which
> then takes care to add the offset.

Thanks for noticing this.  I updated the patch to just add a comment to
avert future confusion, and refreshed the subsequent xen patches as below.

> Recent hypervisors will actually
> only consume this to issue a warning if not matching what gets
> read from the corresponding BAR. Earlier hypervisors used this
> instead of reading the BAR.

Note that pci_resource_start() gives you a CPU address, and what you
read from the BAR is a PCI bus address.  These are not in the same
address space and can't be directly compared.  I assume the
hypervisors take that into account and do the appropriate
conversions?

Bjorn


commit 34f394aad6706a42fb69922ed184c918ec9f9f81
Author: Bjorn Helgaas <bhelgaas@google.com>
Date:   Thu Apr 18 15:02:34 2013 -0600

    xen/pci: Comment on unusual PCI_MSIX_TABLE usage
    
    To find the MSI-X Table structure, you look at the BAR specified
    by PCI_MSIX_FLAGS_BIRMASK, then apply the PCI_MSIX_TABLE_OFFSET to
    the address from the BAR.  So most readers of PCI_MSIX_TABLE use
    PCI_MSIX_TABLE_OFFSET, but in xen's case, the hypervisor takes
    care of applying the offset, so we don't need to do it here.
    
    Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
    CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
    CC: Jan Beulich <JBeulich@suse.com>

--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

>>> On 25.04.13 at 18:42, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Thu, Apr 25, 2013 at 10:40:57AM +0100, Jan Beulich wrote:
>> Recent hypervisors will actually
>> only consume this to issue a warning if not matching what gets
>> read from the corresponding BAR. Earlier hypervisors used this
>> instead of reading the BAR.
> 
> Note that pci_resource_start() gives you a CPU address, and what you
> read from the BAR is a PCI bus address.  These are not in the same
> address space and can't be directly compared.  I assume the
> hypervisors take that into account and do the appropriate
> conversions?

I suppose Xen has never been run on a system where the two
would differ, and it's quite likely that it would break on such
systems. Question is - are there any x86-based systems where
this is the case?

Jan

--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Fri, Apr 26, 2013 at 1:16 AM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 25.04.13 at 18:42, Bjorn Helgaas <bhelgaas@google.com> wrote:
>> On Thu, Apr 25, 2013 at 10:40:57AM +0100, Jan Beulich wrote:
>>> Recent hypervisors will actually
>>> only consume this to issue a warning if not matching what gets
>>> read from the corresponding BAR. Earlier hypervisors used this
>>> instead of reading the BAR.
>>
>> Note that pci_resource_start() gives you a CPU address, and what you
>> read from the BAR is a PCI bus address.  These are not in the same
>> address space and can't be directly compared.  I assume the
>> hypervisors take that into account and do the appropriate
>> conversions?
>
> I suppose Xen has never been run on a system where the two
> would differ, and it's quite likely that it would break on such
> systems. Question is - are there any x86-based systems where
> this is the case?

I haven't personally seen one, but it looks like they're coming, based
on this patch we merged last year:

commit b4873931cc8c934a9893d5962bde97aca23be983
Author: Mike Yoknis <mike.yoknis@hp.com>
Date:   Wed Nov 7 15:52:20 2012 -0700

    x86/PCI: Allow x86 platforms to use translation offsets

    The memory range descriptors in the _CRS control method contain an address
    translation offset for host bridges.  This value is used to translate
    addresses across the bridge.  The support to use _TRA values is present for
    other architectures but not for X86 platforms.

    For existing X86 platforms the _TRA value is zero.  Non-zero _TRA values
    are expected on future X86 platforms.  This change will register that value
    with the resource.

    Signed-off-by: Mike Yoknis <mike.yoknis@hp.com>
    Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html