| Submitter | Ira Snyder |
|---|---|
| Date | Feb. 28, 2011, 9:11 p.m. |
| Message ID | <20110228211103.GB31428@ovro.caltech.edu> |
| Download | mbox | patch |
| Permalink | /patch/84866/ |
| State | Not Applicable |
| Headers | show |
Comments
Hi Ira, On 02/28/2011 11:11 PM, Ira W. Snyder wrote: > On Mon, Feb 28, 2011 at 10:15:49PM +0200, Felix Radensky wrote: >> Hi Ira, >> >>> Thank you very much Felix. The dmesg output shows that the controller >>> never got an interrupt for the second transaction. The patch below has >>> extra debugging information that may help determine why this happens. >>> Please apply it and re-run the test. >>> >>> The last section of dmesg (after "Freeing unused kernel memory") is all >>> I need. >>> >> Attached relevant dmesg portion. >> > Ok, try this patch on top of the last one. > > It looks like you used the dmatest module in multi-channel mode last > time. One channel makes it easier to debug: > > modprobe dmatest max_channels=1 threads_per_chan=2 iterations=1 > > Thanks for your help in debugging this. Hopefully this is the last > patch to test. :) > > Ira > Looks like this was not the last one. The test still fails, see below __dma_request_channel: success (dma0chan0) of:fsl-elo-dma ffe0c300.dma: chan0: LD ef380000 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef380060 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3800c0 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef380120 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef380180 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3801e0 allocated of:fsl-elo-dma ffe0c300.dma: chan0: assign cookies: start=0 of:fsl-elo-dma ffe0c300.dma: chan0: assign cookies: end=6 of:fsl-elo-dma ffe0c300.dma: chan0: idle, starting controller dmatest: Started 2 threads using dma0chan0 of:fsl-elo-dma ffe0c300.dma: chan0: irq: stat = 0x8 of:fsl-elo-dma ffe0c300.dma: chan0: irq: End-of-link INT of:fsl-elo-dma ffe0c300.dma: chan0: irq: Exit of:fsl-elo-dma ffe0c300.dma: chan0: tasklet entry of:fsl-elo-dma ffe0c300.dma: chan0: completed_cookie=6 of:fsl-elo-dma ffe0c300.dma: chan0: no pending LDs of:fsl-elo-dma ffe0c300.dma: chan0: LD ef380000 callback of:fsl-elo-dma ffe0c300.dma: chan0: LD ef380000 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef380060 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3800c0 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef380120 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef380180 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3801e0 free of:fsl-elo-dma ffe0c300.dma: chan0: tasklet exit dma0chan0-copy0: verifying source buffer... dma0chan0-copy0: verifying dest buffer... dma0chan0-copy0: #0: No errors with src_off=0xf92 dst_off=0xf46 len=0x14ad dma0chan0-copy0: terminating after 1 tests, 0 failures (status 0) of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3801e0 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef380180 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef380120 allocated of:fsl-elo-dma ffe0c300.dma: chan0: assign cookies: start=6 of:fsl-elo-dma ffe0c300.dma: chan0: assign cookies: end=9 of:fsl-elo-dma ffe0c300.dma: chan0: idle, starting controller dma0chan0-copy1: #0: test timed out dma0chan0-copy1: terminating after 1 tests, 1 failures (status 0) Felix.
On Mon, Feb 28, 2011 at 11:27:40PM +0200, Felix Radensky wrote: > Hi Ira, > > On 02/28/2011 11:11 PM, Ira W. Snyder wrote: > > On Mon, Feb 28, 2011 at 10:15:49PM +0200, Felix Radensky wrote: > >> Hi Ira, > >> > >>> Thank you very much Felix. The dmesg output shows that the controller > >>> never got an interrupt for the second transaction. The patch below has > >>> extra debugging information that may help determine why this happens. > >>> Please apply it and re-run the test. > >>> > >>> The last section of dmesg (after "Freeing unused kernel memory") is all > >>> I need. > >>> > >> Attached relevant dmesg portion. > >> > > Ok, try this patch on top of the last one. > > > > It looks like you used the dmatest module in multi-channel mode last > > time. One channel makes it easier to debug: > > > > modprobe dmatest max_channels=1 threads_per_chan=2 iterations=1 > > > > Thanks for your help in debugging this. Hopefully this is the last > > patch to test. :) > > > > Ira > > > > Looks like this was not the last one. The test still fails, see below > From this log, it looks like the DMA controller is not generating an interrupt after the second chain is started. The first chain is finished before the second thread runs and starts its chain. The end-of-segments interrupt is completely missing. The part is not behaving as the datasheet explains it should. Are you sure you applied the patch and rebuilt the kernel? (Just checking to be sure. I'm very appreciative of the amount of help you've given me debugging this!) Can you run this for me: modprobe dmatest max_channels=1 threads_per_chan=1 iterations=4 Thanks again, Ira
Hi Ira, On 03/01/2011 02:21 AM, Ira W. Snyder wrote: > On Mon, Feb 28, 2011 at 11:27:40PM +0200, Felix Radensky wrote: >> Hi Ira, >> >> On 02/28/2011 11:11 PM, Ira W. Snyder wrote: >>> On Mon, Feb 28, 2011 at 10:15:49PM +0200, Felix Radensky wrote: >>>> Hi Ira, >>>> >>>>> Thank you very much Felix. The dmesg output shows that the controller >>>>> never got an interrupt for the second transaction. The patch below has >>>>> extra debugging information that may help determine why this happens. >>>>> Please apply it and re-run the test. >>>>> >>>>> The last section of dmesg (after "Freeing unused kernel memory") is all >>>>> I need. >>>>> >>>> Attached relevant dmesg portion. >>>> >>> Ok, try this patch on top of the last one. >>> >>> It looks like you used the dmatest module in multi-channel mode last >>> time. One channel makes it easier to debug: >>> >>> modprobe dmatest max_channels=1 threads_per_chan=2 iterations=1 >>> >>> Thanks for your help in debugging this. Hopefully this is the last >>> patch to test. :) >>> >>> Ira >>> >> Looks like this was not the last one. The test still fails, see below >> > From this log, it looks like the DMA controller is not generating an > interrupt after the second chain is started. The first chain is finished > before the second thread runs and starts its chain. > > The end-of-segments interrupt is completely missing. The part is not > behaving as the datasheet explains it should. Are you sure you applied > the patch and rebuilt the kernel? (Just checking to be sure. I'm very > appreciative of the amount of help you've given me debugging this!) I've double checked and I'm sure I've applied your patch and rebuilt the kernel. > Can you run this for me: > > modprobe dmatest max_channels=1 threads_per_chan=1 iterations=4 > > Thanks again, > Ira See below. __dma_request_channel: success (dma0chan0) dmatest: Started 1 threads using dma0chan0 of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7000 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7060 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a70c0 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7120 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7180 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a71e0 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7240 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a72a0 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7300 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7360 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a73c0 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7420 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7480 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a74e0 allocated of:fsl-elo-dma ffe0c300.dma: chan0: assign cookies: start=0 of:fsl-elo-dma ffe0c300.dma: chan0: assign cookies: end=14 of:fsl-elo-dma ffe0c300.dma: chan0: idle, starting controller of:fsl-elo-dma ffe0c300.dma: chan0: irq: stat = 0x8 of:fsl-elo-dma ffe0c300.dma: chan0: irq: End-of-link INT of:fsl-elo-dma ffe0c300.dma: chan0: irq: Exit of:fsl-elo-dma ffe0c300.dma: chan0: tasklet entry of:fsl-elo-dma ffe0c300.dma: chan0: completed_cookie=14 of:fsl-elo-dma ffe0c300.dma: chan0: no pending LDs of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7000 callback of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7000 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7060 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a70c0 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7120 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7180 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a71e0 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7240 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a72a0 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7300 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7360 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a73c0 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7420 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7480 free of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a74e0 free of:fsl-elo-dma ffe0c300.dma: chan0: tasklet exit dma0chan0-copy0: verifying source buffer... dma0chan0-copy0: verifying dest buffer... dma0chan0-copy0: #0: No errors with src_off=0x43e dst_off=0x3a5 len=0x3605 of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a74e0 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7480 allocated of:fsl-elo-dma ffe0c300.dma: chan0: assign cookies: start=14 of:fsl-elo-dma ffe0c300.dma: chan0: assign cookies: end=16 of:fsl-elo-dma ffe0c300.dma: chan0: idle, starting controller dma0chan0-copy0: #1: test timed out of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7420 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a73c0 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7360 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7300 allocated of:fsl-elo-dma ffe0c300.dma: chan0: assign cookies: start=16 of:fsl-elo-dma ffe0c300.dma: chan0: assign cookies: end=20 of:fsl-elo-dma ffe0c300.dma: chan0: DMA controller still busy dma0chan0-copy0: #2: test timed out of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a72a0 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7240 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a71e0 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7180 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7120 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a70c0 allocated of:fsl-elo-dma ffe0c300.dma: chan0: LD ef3a7060 allocated of:fsl-elo-dma ffe0c300.dma: chan0: assign cookies: start=20 of:fsl-elo-dma ffe0c300.dma: chan0: assign cookies: end=27 of:fsl-elo-dma ffe0c300.dma: chan0: DMA controller still busy dma0chan0-copy0: #3: test timed out dma0chan0-copy0: terminating after 4 tests, 3 failures (status 0) Felix.
Hi Ira, On 03/01/2011 02:21 AM, Ira W. Snyder wrote: > On Mon, Feb 28, 2011 at 11:27:40PM +0200, Felix Radensky wrote: >> Hi Ira, >> >> On 02/28/2011 11:11 PM, Ira W. Snyder wrote: >>> On Mon, Feb 28, 2011 at 10:15:49PM +0200, Felix Radensky wrote: >>>> Hi Ira, >>>> >>>>> Thank you very much Felix. The dmesg output shows that the controller >>>>> never got an interrupt for the second transaction. The patch below has >>>>> extra debugging information that may help determine why this happens. >>>>> Please apply it and re-run the test. >>>>> >>>>> The last section of dmesg (after "Freeing unused kernel memory") is all >>>>> I need. >>>>> >>>> Attached relevant dmesg portion. >>>> >>> Ok, try this patch on top of the last one. >>> >>> It looks like you used the dmatest module in multi-channel mode last >>> time. One channel makes it easier to debug: >>> >>> modprobe dmatest max_channels=1 threads_per_chan=2 iterations=1 >>> >>> Thanks for your help in debugging this. Hopefully this is the last >>> patch to test. :) >>> >>> Ira >>> >> Looks like this was not the last one. The test still fails, see below >> > From this log, it looks like the DMA controller is not generating an > interrupt after the second chain is started. The first chain is finished > before the second thread runs and starts its chain. > > The end-of-segments interrupt is completely missing. The part is not > behaving as the datasheet explains it should. Are you sure you applied > the patch and rebuilt the kernel? (Just checking to be sure. I'm very > appreciative of the amount of help you've given me debugging this!) > > Can you run this for me: > > modprobe dmatest max_channels=1 threads_per_chan=1 iterations=4 > > Thanks again, > Ira Without your patches applied the output of the test above looks like this: __dma_request_channel: success (dma0chan0) dmatest: Started 1 threads using dma0chan0 of:fsl-elo-dma ffe0c300.dma: irq: channel 0, stat = 0xa of:fsl-elo-dma ffe0c300.dma: irq: End-of-segments INT of:fsl-elo-dma ffe0c300.dma: irq: clndar 0x2f34d000, nlndar 0x100000000 of:fsl-elo-dma ffe0c300.dma: irq: End-of-link INT of:fsl-elo-dma ffe0c300.dma: no pending LDs of:fsl-elo-dma ffe0c300.dma: irq: Exit of:fsl-elo-dma ffe0c300.dma: chan completed_cookie = 1 of:fsl-elo-dma ffe0c300.dma: LD ef34d000 callback of:fsl-elo-dma ffe0c300.dma: chan completed_cookie = 1 dma0chan0-copy0: verifying source buffer... dma0chan0-copy0: verifying dest buffer... dma0chan0-copy0: #0: No errors with src_off=0xb90 dst_off=0x101c len=0x2aea of:fsl-elo-dma ffe0c300.dma: irq: channel 0, stat = 0xa of:fsl-elo-dma ffe0c300.dma: irq: End-of-segments INT of:fsl-elo-dma ffe0c300.dma: irq: clndar 0x2f34d000, nlndar 0x100000000 of:fsl-elo-dma ffe0c300.dma: irq: End-of-link INT of:fsl-elo-dma ffe0c300.dma: no pending LDs of:fsl-elo-dma ffe0c300.dma: irq: Exit of:fsl-elo-dma ffe0c300.dma: chan completed_cookie = 2 of:fsl-elo-dma ffe0c300.dma: LD ef34d000 callback of:fsl-elo-dma ffe0c300.dma: chan completed_cookie = 2 dma0chan0-copy0: verifying source buffer... dma0chan0-copy0: verifying dest buffer... dma0chan0-copy0: #1: No errors with src_off=0x74a dst_off=0x54d len=0x35f5 of:fsl-elo-dma ffe0c300.dma: irq: channel 0, stat = 0xa of:fsl-elo-dma ffe0c300.dma: irq: End-of-segments INT of:fsl-elo-dma ffe0c300.dma: irq: clndar 0x2f34d000, nlndar 0x100000000 of:fsl-elo-dma ffe0c300.dma: irq: End-of-link INT of:fsl-elo-dma ffe0c300.dma: no pending LDs of:fsl-elo-dma ffe0c300.dma: irq: Exit of:fsl-elo-dma ffe0c300.dma: chan completed_cookie = 3 of:fsl-elo-dma ffe0c300.dma: LD ef34d000 callback of:fsl-elo-dma ffe0c300.dma: chan completed_cookie = 3 dma0chan0-copy0: verifying source buffer... dma0chan0-copy0: verifying dest buffer... dma0chan0-copy0: #2: No errors with src_off=0x2ad dst_off=0x557 len=0x35e1 of:fsl-elo-dma ffe0c300.dma: irq: channel 0, stat = 0xa of:fsl-elo-dma ffe0c300.dma: irq: End-of-segments INT of:fsl-elo-dma ffe0c300.dma: irq: clndar 0x2f34d000, nlndar 0x100000000 of:fsl-elo-dma ffe0c300.dma: irq: End-of-link INT of:fsl-elo-dma ffe0c300.dma: no pending LDs of:fsl-elo-dma ffe0c300.dma: irq: Exit of:fsl-elo-dma ffe0c300.dma: chan completed_cookie = 4 of:fsl-elo-dma ffe0c300.dma: LD ef34d000 callback of:fsl-elo-dma ffe0c300.dma: chan completed_cookie = 4 dma0chan0-copy0: verifying source buffer... dma0chan0-copy0: verifying dest buffer... dma0chan0-copy0: #3: No errors with src_off=0x0 dst_off=0x0 len=0x4000 dma0chan0-copy0: terminating after 4 tests, 0 failures (status 0) Felix.
On Tue, Mar 01, 2011 at 07:52:39AM +0200, Felix Radensky wrote: > Hi Ira, > > On 03/01/2011 02:21 AM, Ira W. Snyder wrote: > > On Mon, Feb 28, 2011 at 11:27:40PM +0200, Felix Radensky wrote: > >> Hi Ira, > >> > >> On 02/28/2011 11:11 PM, Ira W. Snyder wrote: > >>> On Mon, Feb 28, 2011 at 10:15:49PM +0200, Felix Radensky wrote: > >>>> Hi Ira, > >>>> > >>>>> Thank you very much Felix. The dmesg output shows that the controller > >>>>> never got an interrupt for the second transaction. The patch below has > >>>>> extra debugging information that may help determine why this happens. > >>>>> Please apply it and re-run the test. > >>>>> > >>>>> The last section of dmesg (after "Freeing unused kernel memory") is all > >>>>> I need. > >>>>> > >>>> Attached relevant dmesg portion. > >>>> > >>> Ok, try this patch on top of the last one. > >>> > >>> It looks like you used the dmatest module in multi-channel mode last > >>> time. One channel makes it easier to debug: > >>> > >>> modprobe dmatest max_channels=1 threads_per_chan=2 iterations=1 > >>> > >>> Thanks for your help in debugging this. Hopefully this is the last > >>> patch to test. :) > >>> > >>> Ira > >>> > >> Looks like this was not the last one. The test still fails, see below > >> > > From this log, it looks like the DMA controller is not generating an > > interrupt after the second chain is started. The first chain is finished > > before the second thread runs and starts its chain. > > > > The end-of-segments interrupt is completely missing. The part is not > > behaving as the datasheet explains it should. Are you sure you applied > > the patch and rebuilt the kernel? (Just checking to be sure. I'm very > > appreciative of the amount of help you've given me debugging this!) > > > > Can you run this for me: > > > > modprobe dmatest max_channels=1 threads_per_chan=1 iterations=4 > > > > Thanks again, > > Ira > > Without your patches applied the output of the test above looks > like this: > Thanks, this is exactly what I was going to ask for next. :) I really don't understand why the P2020 DMA controller isn't behaving nicely after my patches. Can you run a git bisect to figure out which patch in the series causes the problems. It should take three or four build + test cycles to narrow down which patch breaks the driver. When it is finished, send me the output of git bisect. Like this (assuming you have two branches: master and fsldma, where fsldma is master + my patches): # setup the bisect git bisect start git bisect bad fsldma git bisect good master # build and test the kernel using the same test as before: modprobe dmatest max_channels=1 threads_per_chan=1 iterations=4 # if the test passes: git bisect good # if the test fails: git bisect bad # now build + test again, then mark that good or bad. Repeat until # finished. I really appreciate your help in testing this. You've been great at providing everything I've asked for. Thanks, Ira
Patch
diff --git a/drivers/dma/fsldma.c b/drivers/dma/fsldma.c index b82b76e..e4d9d17 100644 --- a/drivers/dma/fsldma.c +++ b/drivers/dma/fsldma.c @@ -141,7 +141,7 @@ static void set_ld_eol(struct fsldma_chan *chan, struct fsl_desc_sw *desc) u64 snoop_bits; snoop_bits = ((chan->feature & FSL_DMA_IP_MASK) == FSL_DMA_IP_83XX) - ? FSL_DMA_SNEN : 0; + ? FSL_DMA_SNEN : (u64)(0x8); desc->hw.next_ln_addr = CPU_TO_DMA(chan, DMA_TO_CPU(chan, desc->hw.next_ln_addr, 64) | FSL_DMA_EOL @@ -165,7 +165,6 @@ static void dma_init(struct fsldma_chan *chan) * BWC - Bandwidth sharing among channels */ DMA_OUT(chan, &chan->regs->mr, FSL_DMA_MR_BWC - | FSL_DMA_MR_EOSIE | FSL_DMA_MR_EOLSIE | FSL_DMA_MR_EIE | FSL_DMA_MR_EOLNIE, 32); break; case FSL_DMA_IP_83XX: