mbox

[PULL,00/18] Block layer patches

Message ID 1474984441-28516-1-git-send-email-kwolf@redhat.com
State New
Headers show

Pull-request

git://repo.or.cz/qemu/kevin.git tags/for-upstream

Message

Kevin Wolf Sept. 27, 2016, 1:53 p.m. UTC
The following changes since commit 7cfdc02dae0d2ff58c897496cfdbbafc0eda0f3f:

  Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2016-09-26 19:47:00 +0100)

are available in the git repository at:


  git://repo.or.cz/qemu/kevin.git tags/for-upstream

for you to fetch changes up to 3b856cebe5e93547852c156ca2119d075e62aed7:

  coroutine: reduce stack size to 60kB (2016-09-27 14:05:21 +0200)

----------------------------------------------------------------
Block layer patches

----------------------------------------------------------------
John Snow (3):
      block: reintroduce bdrv_flush_all
      qemu: use bdrv_flush_all for vm_stop et al
      block-backend: remove blk_flush_all

Kevin Wolf (8):
      block: Fix error path in qmp_blockdev_change_medium()
      block: Drop aio/cache consistency check from qmp_blockdev_add()
      block/qapi: Use separate options type for curl driver
      block/qapi: Move 'aio' option to file driver
      block: Parse 'detect-zeroes' in bdrv_open_common()
      block: Use 'detect-zeroes' option for 'blockdev-change-medium'
      block: Move 'discard' option to bdrv_open_common()
      block: Remove qemu_root_bds_opts

Peter Lieven (7):
      oslib-posix: add helpers for stack alloc and free
      coroutine-sigaltstack: rename coroutine struct appropriately
      coroutine: add a macro for the coroutine stack size
      coroutine-ucontext: use helper for allocating stack memory
      coroutine-sigaltstack: use helper for allocating stack memory
      oslib-posix: add a configure switch to debug stack usage
      coroutine: reduce stack size to 60kB

 block.c                        |  50 +++++++++++++++++-
 block/block-backend.c          |  31 ++----------
 block/io.c                     |  25 +++++++++
 block/raw-posix.c              |  44 +++++++++-------
 block/raw-win32.c              |  56 +++++++++++++++++++--
 blockdev.c                     | 112 +++--------------------------------------
 configure                      |  19 +++++++
 cpus.c                         |   4 +-
 hw/i386/xen/xen_platform.c     |   2 -
 hw/ide/piix.c                  |   4 ++
 include/block/block.h          |   2 +
 include/qemu/coroutine_int.h   |   2 +
 include/sysemu/block-backend.h |   3 +-
 include/sysemu/os-posix.h      |  27 ++++++++++
 qapi/block-core.json           |  31 ++++++++----
 tests/qemu-iotests/087         |   4 +-
 tests/qemu-iotests/087.out     |   2 +-
 util/coroutine-sigaltstack.c   |  25 ++++-----
 util/coroutine-ucontext.c      |  11 ++--
 util/coroutine-win32.c         |   2 +-
 util/oslib-posix.c             |  77 ++++++++++++++++++++++++++++
 21 files changed, 342 insertions(+), 191 deletions(-)

Comments

Peter Maydell Sept. 27, 2016, 7:42 p.m. UTC | #1
On 27 September 2016 at 06:53, Kevin Wolf <kwolf@redhat.com> wrote:
> The following changes since commit 7cfdc02dae0d2ff58c897496cfdbbafc0eda0f3f:
>
>   Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2016-09-26 19:47:00 +0100)
>
> are available in the git repository at:
>
>
>   git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to 3b856cebe5e93547852c156ca2119d075e62aed7:
>
>   coroutine: reduce stack size to 60kB (2016-09-27 14:05:21 +0200)
>
> ----------------------------------------------------------------
> Block layer patches
>
> ----------------------------------------------------------------

I see 'make check' failures on x86-64 host, clang Linux:

  /i386/ahci/migrate/ncq/simple:                                       OK
  /i386/ahci/migrate/ncq/halted:                                       OK
  /i386/ahci/cdrom/dma/single:                                         OK
  /i386/ahci/cdrom/dma/multi:                                          OK
  /i386/ahci/cdrom/pio/single:
Broken pipe
FAIL
GTester: last random seed: R02Sa8f729848b07c3b3e5ee67368f9d0350
(pid=10590)
  /i386/ahci/cdrom/pio/multi:
Broken pipe
FAIL
GTester: last random seed: R02Se85704e04bbd382223983c878723b811
(pid=10598)
FAIL: tests/ahci-test
TEST: tests/hd-geo-test... (pid=10601)
  /i386/hd-geo/ide/none:                                               OK


thanks
-- PMM
Kevin Wolf Sept. 28, 2016, 9:37 a.m. UTC | #2
Am 27.09.2016 um 21:42 hat Peter Maydell geschrieben:
> On 27 September 2016 at 06:53, Kevin Wolf <kwolf@redhat.com> wrote:
> > The following changes since commit 7cfdc02dae0d2ff58c897496cfdbbafc0eda0f3f:
> >
> >   Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2016-09-26 19:47:00 +0100)
> >
> > are available in the git repository at:
> >
> >
> >   git://repo.or.cz/qemu/kevin.git tags/for-upstream
> >
> > for you to fetch changes up to 3b856cebe5e93547852c156ca2119d075e62aed7:
> >
> >   coroutine: reduce stack size to 60kB (2016-09-27 14:05:21 +0200)
> >
> > ----------------------------------------------------------------
> > Block layer patches
> >
> > ----------------------------------------------------------------
> 
> I see 'make check' failures on x86-64 host, clang Linux:
> 
>   /i386/ahci/migrate/ncq/simple:                                       OK
>   /i386/ahci/migrate/ncq/halted:                                       OK
>   /i386/ahci/cdrom/dma/single:                                         OK
>   /i386/ahci/cdrom/dma/multi:                                          OK
>   /i386/ahci/cdrom/pio/single:
> Broken pipe
> FAIL
> GTester: last random seed: R02Sa8f729848b07c3b3e5ee67368f9d0350
> (pid=10590)
>   /i386/ahci/cdrom/pio/multi:
> Broken pipe
> FAIL
> GTester: last random seed: R02Se85704e04bbd382223983c878723b811
> (pid=10598)
> FAIL: tests/ahci-test
> TEST: tests/hd-geo-test... (pid=10601)
>   /i386/hd-geo/ide/none:                                               OK

I asked on IRC, but as you don't seem to be around at the moment, I'll
keep things on the list instead.

Can you tell me the exact 'configure' and 'make check' command lines
you're using? I always run the tests with clang/Linux on x86_64 before
sending a pull request, and I repeated it now, but the tests pass for
me.

This is what I'm doing (yesterday I built all targets, today only
i386-softmmu because that seems to be what fails for you):

$ clang --version
clang version 3.4.2 (tags/RELEASE_34/dot2-final)
Target: x86_64-redhat-linux-gnu
Thread model: posix
$ ../qemu/configure --cc=clang --host-cc=clang --cxx=clang++ --target-list=i386-softmmu
$ make -j4
$ make check

Kevin
Peter Maydell Sept. 28, 2016, 2:52 p.m. UTC | #3
On 28 September 2016 at 02:37, Kevin Wolf <kwolf@redhat.com> wrote:
> [what clang config?]

This is with Ubuntu Xenial's clang. configure args are

'--cc=clang' '--cxx=clang++' '--enable-gtk'
'--extra-cflags=-fsanitize=undefined -Werror'

and the build is just 'make all -j8 && make check' in the build
directory.

thanks
-- PMM
Peter Maydell Sept. 28, 2016, 7:03 p.m. UTC | #4
On 28 September 2016 at 02:37, Kevin Wolf <kwolf@redhat.com> wrote:
> Am 27.09.2016 um 21:42 hat Peter Maydell geschrieben:
>> On 27 September 2016 at 06:53, Kevin Wolf <kwolf@redhat.com> wrote:
>> > The following changes since commit 7cfdc02dae0d2ff58c897496cfdbbafc0eda0f3f:
>> >
>> >   Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2016-09-26 19:47:00 +0100)
>> >
>> > are available in the git repository at:
>> >
>> >
>> >   git://repo.or.cz/qemu/kevin.git tags/for-upstream
>> >
>> > for you to fetch changes up to 3b856cebe5e93547852c156ca2119d075e62aed7:
>> >
>> >   coroutine: reduce stack size to 60kB (2016-09-27 14:05:21 +0200)
>> >
>> > ----------------------------------------------------------------
>> > Block layer patches
>> >
>> > ----------------------------------------------------------------
>>
>> I see 'make check' failures on x86-64 host, clang Linux:
>>
>>   /i386/ahci/migrate/ncq/simple:                                       OK
>>   /i386/ahci/migrate/ncq/halted:                                       OK
>>   /i386/ahci/cdrom/dma/single:                                         OK
>>   /i386/ahci/cdrom/dma/multi:                                          OK
>>   /i386/ahci/cdrom/pio/single:
>> Broken pipe
>> FAIL
>> GTester: last random seed: R02Sa8f729848b07c3b3e5ee67368f9d0350
>> (pid=10590)
>>   /i386/ahci/cdrom/pio/multi:
>> Broken pipe
>> FAIL
>> GTester: last random seed: R02Se85704e04bbd382223983c878723b811
>> (pid=10598)
>> FAIL: tests/ahci-test
>> TEST: tests/hd-geo-test... (pid=10601)
>>   /i386/hd-geo/ide/none:                                               OK
>
> I asked on IRC, but as you don't seem to be around at the moment, I'll
> keep things on the list instead.

I got a gdb backtrace:

Thread 1 "qemu-system-i38" received signal SIGSEGV, Segmentation fault.
0x00005555561dea15 in address_space_translate (as=0x55555a46bfc0,
addr=1106048, xlat=0x7ffff7e0d050, plen=0x7ffff7e0d058,
    is_write=false) at /home/petmay01/linaro/qemu-for-merges/exec.c:423
423     {


Backtrace suggests we've run out of stack due to some infinite
recursion:

#0  0x00005555561dea15 in address_space_translate (as=0x55555a46bfc0,
addr=1106048, xlat=0x7ffff7e0d050, plen=0x7ffff7e0d058,
is_write=false) at /home/petmay01/linaro/qemu-for-merges/exec.c:423
#1  0x00005555561edeab in address_space_map (as=<optimised out>,
addr=1106048, plen=<optimised out>, is_write=false)
    at /home/petmay01/linaro/qemu-for-merges/exec.c:2909
#2  0x0000555556840b9b in ahci_populate_sglist (as=0x55555a46bfc0,
addr=1106048, dir=DMA_DIRECTION_TO_DEVICE, len=<optimised out>)
    at /home/petmay01/linaro/qemu-for-merges/include/sysemu/dma.h:135
#3  0x0000555556840b9b in ahci_populate_sglist (ad=<optimised out>,
sglist=<optimised out>, cmd=<optimised out>, limit=<optimised out>,
offset=1592) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:863
#4  0x0000555556844de4 in ahci_dma_prepare_buf (dma=0x55555a475b48,
limit=<optimised out>)
    at /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1366
#5  0x000055555684354c in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1295
#6  0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#7  0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#8  0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#9  0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#10 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#11 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#12 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#13 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#14 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#15 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#16 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#17 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#18 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#19 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#20 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#21 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#22 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#23 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#24 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#25 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#26 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#27 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#28 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#29 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#30 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#31 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#32 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#33 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#34 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#35 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#36 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#37 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#38 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#39 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#40 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#41 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
/home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318

[skip a lot of repeated stack frames]

#393 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#394 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#395 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#396 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#397 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#398 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#399 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#400 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#401 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#402 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#403 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
#404 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
#405 0x0000555556809cfc in ide_buffered_readv_cb
(opaque=0x5555594f57e0, ret=<optimised out>)
    at /home/petmay01/linaro/qemu-for-merges/hw/ide/core.c:605
#406 0x0000555556df7f73 in blk_aio_complete (acb=0x55555a4387c0) at
/home/petmay01/linaro/qemu-for-merges/block/block-backend.c:943
#407 0x0000555556f676f1 in coroutine_trampoline (i0=<optimised out>,
i1=<optimised out>)
    at /home/petmay01/linaro/qemu-for-merges/util/coroutine-ucontext.c:79
#408 0x00007fffdca05590 in __start_context () at /lib/x86_64-linux-gnu/libc.so.6
#409 0x00007fffffffc318 in  ()
#410 0x0000000000000000 in  ()


thanks
-- PMM
Kevin Wolf Sept. 29, 2016, 10:25 a.m. UTC | #5
Am 28.09.2016 um 21:03 hat Peter Maydell geschrieben:
> On 28 September 2016 at 02:37, Kevin Wolf <kwolf@redhat.com> wrote:
> > Am 27.09.2016 um 21:42 hat Peter Maydell geschrieben:
> >> On 27 September 2016 at 06:53, Kevin Wolf <kwolf@redhat.com> wrote:
> >> > The following changes since commit 7cfdc02dae0d2ff58c897496cfdbbafc0eda0f3f:
> >> >
> >> >   Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2016-09-26 19:47:00 +0100)
> >> >
> >> > are available in the git repository at:
> >> >
> >> >
> >> >   git://repo.or.cz/qemu/kevin.git tags/for-upstream
> >> >
> >> > for you to fetch changes up to 3b856cebe5e93547852c156ca2119d075e62aed7:
> >> >
> >> >   coroutine: reduce stack size to 60kB (2016-09-27 14:05:21 +0200)
> >> >
> >> > ----------------------------------------------------------------
> >> > Block layer patches
> >> >
> >> > ----------------------------------------------------------------
> >>
> >> I see 'make check' failures on x86-64 host, clang Linux:
> >>
> >>   /i386/ahci/migrate/ncq/simple:                                       OK
> >>   /i386/ahci/migrate/ncq/halted:                                       OK
> >>   /i386/ahci/cdrom/dma/single:                                         OK
> >>   /i386/ahci/cdrom/dma/multi:                                          OK
> >>   /i386/ahci/cdrom/pio/single:
> >> Broken pipe
> >> FAIL
> >> GTester: last random seed: R02Sa8f729848b07c3b3e5ee67368f9d0350
> >> (pid=10590)
> >>   /i386/ahci/cdrom/pio/multi:
> >> Broken pipe
> >> FAIL
> >> GTester: last random seed: R02Se85704e04bbd382223983c878723b811
> >> (pid=10598)
> >> FAIL: tests/ahci-test
> >> TEST: tests/hd-geo-test... (pid=10601)
> >>   /i386/hd-geo/ide/none:                                               OK
> >
> > I asked on IRC, but as you don't seem to be around at the moment, I'll
> > keep things on the list instead.
> 
> I got a gdb backtrace:
> 
> Thread 1 "qemu-system-i38" received signal SIGSEGV, Segmentation fault.
> 0x00005555561dea15 in address_space_translate (as=0x55555a46bfc0,
> addr=1106048, xlat=0x7ffff7e0d050, plen=0x7ffff7e0d058,
>     is_write=false) at /home/petmay01/linaro/qemu-for-merges/exec.c:423
> 423     {
> 
> 
> Backtrace suggests we've run out of stack due to some infinite
> recursion:

Thanks, Peter, this is useful.

The series contains a patch that reduces the coroutine stack size, so I
guess it's not quite infinite, but pretty deep recursion anyway. I will
drop that final patch that reduces the stack size and hope that the rest
will pass your testing (I tried some more to reproduce it, but I still
didn't manage to).

John, can you have a look at the IDE code and check whether we can get
rid of the deep recursion? It seems that the test issues a large request
that is then split into many small requests. But it should be possible
to do this iteratively rather than recursively.

Kevin

> #0  0x00005555561dea15 in address_space_translate (as=0x55555a46bfc0,
> addr=1106048, xlat=0x7ffff7e0d050, plen=0x7ffff7e0d058,
> is_write=false) at /home/petmay01/linaro/qemu-for-merges/exec.c:423
> #1  0x00005555561edeab in address_space_map (as=<optimised out>,
> addr=1106048, plen=<optimised out>, is_write=false)
>     at /home/petmay01/linaro/qemu-for-merges/exec.c:2909
> #2  0x0000555556840b9b in ahci_populate_sglist (as=0x55555a46bfc0,
> addr=1106048, dir=DMA_DIRECTION_TO_DEVICE, len=<optimised out>)
>     at /home/petmay01/linaro/qemu-for-merges/include/sysemu/dma.h:135
> #3  0x0000555556840b9b in ahci_populate_sglist (ad=<optimised out>,
> sglist=<optimised out>, cmd=<optimised out>, limit=<optimised out>,
> offset=1592) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:863
> #4  0x0000555556844de4 in ahci_dma_prepare_buf (dma=0x55555a475b48,
> limit=<optimised out>)
>     at /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1366
> #5  0x000055555684354c in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1295
> #6  0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #7  0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #8  0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #9  0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #10 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #11 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #12 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #13 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #14 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #15 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #16 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #17 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #18 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #19 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #20 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #21 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #22 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #23 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #24 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #25 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #26 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #27 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #28 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #29 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #30 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #31 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #32 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #33 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #34 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #35 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #36 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #37 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #38 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #39 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #40 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #41 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>) at
> /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> 
> [skip a lot of repeated stack frames]
> 
> #393 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #394 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #395 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #396 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #397 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #398 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #399 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #400 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #401 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #402 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #403 0x0000555556843662 in ahci_start_transfer (dma=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/ahci.c:1318
> #404 0x00005555568250cb in ide_atapi_cmd_reply_end (s=<optimised out>)
> at /home/petmay01/linaro/qemu-for-merges/hw/ide/atapi.c:324
> #405 0x0000555556809cfc in ide_buffered_readv_cb
> (opaque=0x5555594f57e0, ret=<optimised out>)
>     at /home/petmay01/linaro/qemu-for-merges/hw/ide/core.c:605
> #406 0x0000555556df7f73 in blk_aio_complete (acb=0x55555a4387c0) at
> /home/petmay01/linaro/qemu-for-merges/block/block-backend.c:943
> #407 0x0000555556f676f1 in coroutine_trampoline (i0=<optimised out>,
> i1=<optimised out>)
>     at /home/petmay01/linaro/qemu-for-merges/util/coroutine-ucontext.c:79
> #408 0x00007fffdca05590 in __start_context () at /lib/x86_64-linux-gnu/libc.so.6
> #409 0x00007fffffffc318 in  ()
> #410 0x0000000000000000 in  ()
> 
> 
> thanks
> -- PMM
John Snow Sept. 29, 2016, 5:02 p.m. UTC | #6
On 09/29/2016 06:25 AM, Kevin Wolf wrote:
> John, can you have a look at the IDE code and check whether we can get
> rid of the deep recursion? It seems that the test issues a large request
> that is then split into many small requests. But it should be possible
> to do this iteratively rather than recursively.

http://wiki.qemu.org/Features/IDE

"Rewrite ide_atapi_cmd_reply_end to be fully non-recursive; and ideally 
less confusing."

Guess I have to float this one to the top.

--js
Peter Maydell Sept. 29, 2016, 5:18 p.m. UTC | #7
On 29 September 2016 at 03:25, Kevin Wolf <kwolf@redhat.com> wrote:
> The series contains a patch that reduces the coroutine stack size, so I
> guess it's not quite infinite, but pretty deep recursion anyway. I will
> drop that final patch that reduces the stack size and hope that the rest
> will pass your testing (I tried some more to reproduce it, but I still
> didn't manage to).

Ah, I see. I suspect the clang build (since it has the sanitizer
enabled) is a worst-case for stack usage.

Is it possible for a guest to issue a sufficiently large
request that it blows the stack, or is that capped
somehow?

thanks
-- PMM
Paolo Bonzini Sept. 29, 2016, 6:17 p.m. UTC | #8
On 29/09/2016 19:02, John Snow wrote:
> 
> 
> On 09/29/2016 06:25 AM, Kevin Wolf wrote:
>> John, can you have a look at the IDE code and check whether we can get
>> rid of the deep recursion? It seems that the test issues a large request
>> that is then split into many small requests. But it should be possible
>> to do this iteratively rather than recursively.
> 
> http://wiki.qemu.org/Features/IDE
> 
> "Rewrite ide_atapi_cmd_reply_end to be fully non-recursive; and ideally
> less confusing."
> 
> Guess I have to float this one to the top.

Would it be enough to hide the call to s->bus->dma->ops->start_transfer
behind a bottom half?

Paolo
John Snow Sept. 29, 2016, 6:19 p.m. UTC | #9
On 09/29/2016 02:17 PM, Paolo Bonzini wrote:
>
>
> On 29/09/2016 19:02, John Snow wrote:
>>
>>
>> On 09/29/2016 06:25 AM, Kevin Wolf wrote:
>>> John, can you have a look at the IDE code and check whether we can get
>>> rid of the deep recursion? It seems that the test issues a large request
>>> that is then split into many small requests. But it should be possible
>>> to do this iteratively rather than recursively.
>>
>> http://wiki.qemu.org/Features/IDE
>>
>> "Rewrite ide_atapi_cmd_reply_end to be fully non-recursive; and ideally
>> less confusing."
>>
>> Guess I have to float this one to the top.
>
> Would it be enough to hide the call to s->bus->dma->ops->start_transfer
> behind a bottom half?
>
> Paolo
>

Probably the simplest way, even if not the prettiest.
John Snow Sept. 29, 2016, 6:19 p.m. UTC | #10
On 09/29/2016 01:18 PM, Peter Maydell wrote:
> On 29 September 2016 at 03:25, Kevin Wolf <kwolf@redhat.com> wrote:
>> The series contains a patch that reduces the coroutine stack size, so I
>> guess it's not quite infinite, but pretty deep recursion anyway. I will
>> drop that final patch that reduces the stack size and hope that the rest
>> will pass your testing (I tried some more to reproduce it, but I still
>> didn't manage to).
>
> Ah, I see. I suspect the clang build (since it has the sanitizer
> enabled) is a worst-case for stack usage.
>
> Is it possible for a guest to issue a sufficiently large
> request that it blows the stack, or is that capped
> somehow?
>
> thanks
> -- PMM
>

If the qtest can blow the stack, so can a guest. In practice it does not 
happen that guests send requests so large, but mum's the word on a 
naughty guest.

--js