[RFC,1/2] dma-mapping: Clean up dma_set_*mask() hooks

Arch-specific implementions for dma_set_{coherent_,}mask() currently
rely on an inconsistent mix of arch-defined Kconfig symbols and macro
overrides. Now that we have a nice centralised home for DMA API gubbins,
let's consolidate these loose ends under consistent config options.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
---

Here's hoping the buildbot comes by to point out what I've inevitably
missed, although I did check a cursory cross-compile of ppc64_defconfig
to iron out the obvious howlers.

The motivation here is that I'm looking at adding set_mask overrides
for arm64, and having discovered a bit of a mess it seemed prudent to
clean up before ingraining it any more.

Robin.


 arch/arm/Kconfig                       | 3 ---
 arch/powerpc/Kconfig                   | 4 +---
 arch/powerpc/include/asm/dma-mapping.h | 3 ---
 include/linux/dma-mapping.h            | 4 +++-
 kernel/dma/Kconfig                     | 6 ++++++
 5 files changed, 10 insertions(+), 10 deletions(-)

On Wed, Jul 04, 2018 at 06:50:11PM +0100, Robin Murphy wrote:
> Arch-specific implementions for dma_set_{coherent_,}mask() currently
> rely on an inconsistent mix of arch-defined Kconfig symbols and macro
> overrides. Now that we have a nice centralised home for DMA API gubbins,
> let's consolidate these loose ends under consistent config options.
> 
> Signed-off-by: Robin Murphy <robin.murphy@arm.com>
> ---
> 
> Here's hoping the buildbot comes by to point out what I've inevitably
> missed, although I did check a cursory cross-compile of ppc64_defconfig
> to iron out the obvious howlers.

The patch looks sensible to me, although I was hoping to get rid of these
hooks in this or the next merge window as they are a horrible bad idea.

> The motivation here is that I'm looking at adding set_mask overrides
> for arm64, and having discovered a bit of a mess it seemed prudent to
> clean up before ingraining it any more.

What are you trying to do?  I really don't want to see more users of
the hooks as they are are a horribly bad idea.

On 05/07/18 20:37, Christoph Hellwig wrote:
> On Wed, Jul 04, 2018 at 06:50:11PM +0100, Robin Murphy wrote:
>> Arch-specific implementions for dma_set_{coherent_,}mask() currently
>> rely on an inconsistent mix of arch-defined Kconfig symbols and macro
>> overrides. Now that we have a nice centralised home for DMA API gubbins,
>> let's consolidate these loose ends under consistent config options.
>>
>> Signed-off-by: Robin Murphy <robin.murphy@arm.com>
>> ---
>>
>> Here's hoping the buildbot comes by to point out what I've inevitably
>> missed, although I did check a cursory cross-compile of ppc64_defconfig
>> to iron out the obvious howlers.
> 
> The patch looks sensible to me, although I was hoping to get rid of these
> hooks in this or the next merge window as they are a horrible bad idea.
> 
>> The motivation here is that I'm looking at adding set_mask overrides
>> for arm64, and having discovered a bit of a mess it seemed prudent to
>> clean up before ingraining it any more.
> 
> What are you trying to do?  I really don't want to see more users of
> the hooks as they are are a horribly bad idea.

I really need to fix the ongoing problem we have where, due to funky 
integrations, devices suffer some downstream addressing limit (described 
by DT dma-ranges or ACPI IORT/_DMA) which we carefully set up in 
dma_configure(), but then just gets lost when the driver probes and 
innocently calls dma_set_mask() with something wider. I think it's 
effectively the generalised case of the VIA 32-bit quirk, if I 
understand that one correctly.

The approach that seemed to me to be safest is largely based on the one 
proposed in a thread from ages ago[1]; namely to make dma_configure() 
better at distinguishing firmware-specified masks from bus defaults, 
capture the firmware mask in dev->archdata during arch_setup_dma_ops(), 
then use the custom set_mask routines to ensure any subsequent updates 
never exceed that. It doesn't seem possible to make this work robustly 
without storing *some* additional per-device data, and for that archdata 
is a lesser evil than struct device itself. Plus even though it's not 
actually an arch-specific issue it feels like there's such a risk of 
breaking other platforms that I'm reticent to even try handling it 
entirely in generic code.

Robin.

[1] 
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1306507.html

On Fri, Jul 06, 2018 at 03:20:34PM +0100, Robin Murphy wrote:
>> What are you trying to do?  I really don't want to see more users of
>> the hooks as they are are a horribly bad idea.
>
> I really need to fix the ongoing problem we have where, due to funky 
> integrations, devices suffer some downstream addressing limit (described by 
> DT dma-ranges or ACPI IORT/_DMA) which we carefully set up in 
> dma_configure(), but then just gets lost when the driver probes and 
> innocently calls dma_set_mask() with something wider. I think it's 
> effectively the generalised case of the VIA 32-bit quirk, if I understand 
> that one correctly.

I'd much rather fix this in generic code.  How funky are your limitations?
In fact when I did the 32-bit quirk (which will also be used by a Xiling
PCIe root port usable on a lot of architectures) I did initially consider
adding a bus_dma_mask or similar to struct device, but opted for the
most simple implementation for now.  I'd be happy to chanfe this.

Especially these days where busses and IP blocks are generally not tied
to a specific cpu instruction set I really believe that having any
more architecture code than absolutely required is a bad idea.

> The approach that seemed to me to be safest is largely based on the one 
> proposed in a thread from ages ago[1]; namely to make dma_configure() 
> better at distinguishing firmware-specified masks from bus defaults, 
> capture the firmware mask in dev->archdata during arch_setup_dma_ops(), 
> then use the custom set_mask routines to ensure any subsequent updates 
> never exceed that. It doesn't seem possible to make this work robustly 
> without storing *some* additional per-device data, and for that archdata is 
> a lesser evil than struct device itself. Plus even though it's not actually 
> an arch-specific issue it feels like there's such a risk of breaking other 
> platforms that I'm reticent to even try handling it entirely in generic 
> code.

My plan for a few merge windows from now is that dma_mask and
coherent_mask are 100% in device control and dma_set_mask will never
fail.  It will be up to the dma ops to make sure things are addressible.

On 08/07/18 16:07, Christoph Hellwig wrote:
> On Fri, Jul 06, 2018 at 03:20:34PM +0100, Robin Murphy wrote:
>>> What are you trying to do?  I really don't want to see more users of
>>> the hooks as they are are a horribly bad idea.
>>
>> I really need to fix the ongoing problem we have where, due to funky
>> integrations, devices suffer some downstream addressing limit (described by
>> DT dma-ranges or ACPI IORT/_DMA) which we carefully set up in
>> dma_configure(), but then just gets lost when the driver probes and
>> innocently calls dma_set_mask() with something wider. I think it's
>> effectively the generalised case of the VIA 32-bit quirk, if I understand
>> that one correctly.
> 
> I'd much rather fix this in generic code.  How funky are your limitations?
> In fact when I did the 32-bit quirk (which will also be used by a Xiling
> PCIe root port usable on a lot of architectures) I did initially consider
> adding a bus_dma_mask or similar to struct device, but opted for the
> most simple implementation for now.  I'd be happy to chanfe this.
> 
> Especially these days where busses and IP blocks are generally not tied
> to a specific cpu instruction set I really believe that having any
> more architecture code than absolutely required is a bad idea.

Oh, for sure, the generic fix would be the longer-term goal, this was 
just an expedient compromise because I want to get *something* landed 
for 4.19. Since in practice this is predominantly affecting arm64, doing 
the arch-specific fix to appease affected customers then working to 
generalise it afterwards seemed to carry the lowest risk.

That said, I think I can see a relatively safe and clean alternative 
approach based on converting dma_32bit_limit to a mask, so I'll spin 
some patches around that idea ASAP to continue the discussion.

>> The approach that seemed to me to be safest is largely based on the one
>> proposed in a thread from ages ago[1]; namely to make dma_configure()
>> better at distinguishing firmware-specified masks from bus defaults,
>> capture the firmware mask in dev->archdata during arch_setup_dma_ops(),
>> then use the custom set_mask routines to ensure any subsequent updates
>> never exceed that. It doesn't seem possible to make this work robustly
>> without storing *some* additional per-device data, and for that archdata is
>> a lesser evil than struct device itself. Plus even though it's not actually
>> an arch-specific issue it feels like there's such a risk of breaking other
>> platforms that I'm reticent to even try handling it entirely in generic
>> code.
> 
> My plan for a few merge windows from now is that dma_mask and
> coherent_mask are 100% in device control and dma_set_mask will never
> fail.  It will be up to the dma ops to make sure things are addressible.

It's entirely possible to plug an old PCI soundcard via a bridge adapter 
into a modern board where the card's 24-bit DMA mask reaches nothing but 
the SoC's boot flash, and no IOMMU is available (e.g. some of the 
smaller NXP Layercape stuff); I still think there should be an error in 
such rare cases when DMA is utterly impossible, but otherwise I agree it 
would be much nicer for drivers to just provide their preferred mask and 
let the ops massage it as necessary.

Robin.

On Mon, Jul 09, 2018 at 03:53:50PM +0100, Robin Murphy wrote:
> Oh, for sure, the generic fix would be the longer-term goal, this was just 
> an expedient compromise because I want to get *something* landed for 4.19. 
> Since in practice this is predominantly affecting arm64, doing the 
> arch-specific fix to appease affected customers then working to generalise 
> it afterwards seemed to carry the lowest risk.
>
> That said, I think I can see a relatively safe and clean alternative 
> approach based on converting dma_32bit_limit to a mask, so I'll spin some 
> patches around that idea ASAP to continue the discussion.

Great!  I really want to sort out this area as soon as possible, and
introducing more arch specific code isn't really helping with that.  In
fact my whole drive to consolidate code is based on the fact that
I want to fix issues like this in one code base instead of 20 slightly
different ones.

FYI, this is the Xilinx/RISC-V use case for the 32-bit limit,
which I'll need to respin a bit based on linux-pci feedback:

http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/xilinx-dma-32

> It's entirely possible to plug an old PCI soundcard via a bridge adapter 
> into a modern board where the card's 24-bit DMA mask reaches nothing but 
> the SoC's boot flash, and no IOMMU is available (e.g. some of the smaller 
> NXP Layercape stuff); I still think there should be an error in such rare 
> cases when DMA is utterly impossible, but otherwise I agree it would be 
> much nicer for drivers to just provide their preferred mask and let the ops 
> massage it as necessary.

Ok, I guess we need still need to keep the return value for that.