mbox

[PULL,00/29] Block layer patches

Message ID 20230223185146.306454-1-kwolf@redhat.com
State New
Headers show

Pull-request

https://repo.or.cz/qemu/kevin.git tags/for-upstream

Message

Kevin Wolf Feb. 23, 2023, 6:51 p.m. UTC
The following changes since commit 79b677d658d3d35e1e776826ac4abb28cdce69b8:

  Merge tag 'net-pull-request' of https://github.com/jasowang/qemu into staging (2023-02-21 11:28:31 +0000)

are available in the Git repository at:

  https://repo.or.cz/qemu/kevin.git tags/for-upstream

for you to fetch changes up to 0f385a2420d2c3f8ae7ed65fbe2712027664059e:

  block/rbd: Add support for layered encryption (2023-02-23 19:49:35 +0100)

----------------------------------------------------------------
Block layer patches

- Lock the graph, part 2 (BlockDriver callbacks)
- virtio-scsi: fix SCSIDevice hot unplug with IOThread
- rbd: Add support for layered encryption

----------------------------------------------------------------
Emanuele Giuseppe Esposito (5):
      block/qed: add missing graph rdlock in qed_need_check_timer_entry
      block: Mark bdrv_co_flush() and callers GRAPH_RDLOCK
      block: Mark bdrv_co_pdiscard() and callers GRAPH_RDLOCK
      block: Mark bdrv_co_copy_range() GRAPH_RDLOCK
      block: Mark bdrv_co_is_inserted() and callers GRAPH_RDLOCK

Kevin Wolf (18):
      block: Make bdrv_can_set_read_only() static
      mirror: Fix access of uninitialised fields during start
      block: Mark bdrv_co_truncate() and callers GRAPH_RDLOCK
      block: Mark bdrv_co_block_status() and callers GRAPH_RDLOCK
      block: Mark bdrv_co_ioctl() and callers GRAPH_RDLOCK
      block: Mark bdrv_co_pwrite_zeroes() and callers GRAPH_RDLOCK
      block: Mark read/write in block/io.c GRAPH_RDLOCK
      block: Mark public read/write functions GRAPH_RDLOCK
      block: Mark bdrv_co_pwrite_sync() and callers GRAPH_RDLOCK
      block: Mark bdrv_co_do_pwrite_zeroes() GRAPH_RDLOCK
      block: Mark preadv_snapshot/snapshot_block_status GRAPH_RDLOCK
      block: Mark bdrv_co_create() and callers GRAPH_RDLOCK
      block: Mark bdrv_co_io_(un)plug() and callers GRAPH_RDLOCK
      block: Mark bdrv_co_eject/lock_medium() and callers GRAPH_RDLOCK
      block: Mark bdrv_(un)register_buf() GRAPH_RDLOCK
      block: Mark bdrv_co_delete_file() and callers GRAPH_RDLOCK
      block: Mark bdrv_*_dirty_bitmap() and callers GRAPH_RDLOCK
      block: Mark bdrv_co_refresh_total_sectors() and callers GRAPH_RDLOCK

Or Ozeri (3):
      block/rbd: Remove redundant stack variable passphrase_len
      block/rbd: Add luks-any encryption opening option
      block/rbd: Add support for layered encryption

Stefan Hajnoczi (3):
      scsi: protect req->aiocb with AioContext lock
      dma-helpers: prevent dma_blk_cb() vs dma_aio_cancel() race
      virtio-scsi: reset SCSI devices from main loop thread

 qapi/block-core.json               |  27 +++++-
 block/coroutines.h                 |   2 +-
 block/qcow2.h                      |  27 ++++--
 block/qed.h                        |  45 +++++----
 include/block/block-copy.h         |   6 +-
 include/block/block-global-state.h |  14 +--
 include/block/block-io.h           | 110 ++++++++++++----------
 include/block/block_int-common.h   | 173 ++++++++++++++++++----------------
 include/block/block_int-io.h       |  53 ++++++-----
 include/block/dirty-bitmap.h       |  12 +--
 include/hw/virtio/virtio-scsi.h    |  11 ++-
 include/sysemu/block-backend-io.h  |   7 +-
 block.c                            |  12 ++-
 block/backup.c                     |   3 +
 block/blkdebug.c                   |  19 ++--
 block/blklogwrites.c               |  35 ++++---
 block/blkreplay.c                  |  24 +++--
 block/blkverify.c                  |   5 +-
 block/block-backend.c              |  39 +++++---
 block/block-copy.c                 |  32 ++++---
 block/bochs.c                      |   2 +-
 block/commit.c                     |   5 +-
 block/copy-before-write.c          |  33 ++++---
 block/copy-on-read.c               |  44 +++++----
 block/create.c                     |   9 +-
 block/crypto.c                     |  16 ++--
 block/dirty-bitmap.c               |   2 +
 block/file-posix.c                 |  27 +++---
 block/file-win32.c                 |   7 +-
 block/filter-compress.c            |  36 ++++---
 block/io.c                         | 108 +++++++++++++--------
 block/iscsi.c                      |  28 +++---
 block/mirror.c                     |  59 ++++++++----
 block/parallels.c                  |  33 +++----
 block/preallocate.c                |  38 ++++----
 block/qcow.c                       |  46 +++++----
 block/qcow2-cluster.c              |  17 ++--
 block/qcow2.c                      | 136 +++++++++++++++------------
 block/qed-check.c                  |   3 +-
 block/qed-table.c                  |  10 +-
 block/qed.c                        | 101 ++++++++++----------
 block/quorum.c                     |  62 +++++++-----
 block/raw-format.c                 |  76 ++++++++-------
 block/rbd.c                        | 188 ++++++++++++++++++++++++++++++++++---
 block/replication.c                |  18 ++--
 block/snapshot-access.c            |   8 +-
 block/stream.c                     |  40 ++++----
 block/throttle.c                   |  36 ++++---
 block/vdi.c                        |  11 +--
 block/vhdx.c                       |  18 ++--
 block/vmdk.c                       | 132 ++++++++++++--------------
 block/vpc.c                        |  11 +--
 hw/scsi/scsi-disk.c                |  23 +++--
 hw/scsi/scsi-generic.c             |  11 ++-
 hw/scsi/virtio-scsi.c              | 169 ++++++++++++++++++++++++++-------
 qemu-img.c                         |   8 +-
 softmmu/dma-helpers.c              |  12 ++-
 tests/unit/test-bdrv-drain.c       |  20 ++--
 tests/unit/test-block-iothread.c   |   3 +-
 59 files changed, 1355 insertions(+), 907 deletions(-)

Comments

Peter Maydell Feb. 24, 2023, 6:50 p.m. UTC | #1
On Thu, 23 Feb 2023 at 18:51, Kevin Wolf <kwolf@redhat.com> wrote:
>
> The following changes since commit 79b677d658d3d35e1e776826ac4abb28cdce69b8:
>
>   Merge tag 'net-pull-request' of https://github.com/jasowang/qemu into staging (2023-02-21 11:28:31 +0000)
>
> are available in the Git repository at:
>
>   https://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to 0f385a2420d2c3f8ae7ed65fbe2712027664059e:
>
>   block/rbd: Add support for layered encryption (2023-02-23 19:49:35 +0100)
>
> ----------------------------------------------------------------
> Block layer patches
>
> - Lock the graph, part 2 (BlockDriver callbacks)
> - virtio-scsi: fix SCSIDevice hot unplug with IOThread
> - rbd: Add support for layered encryption
>
> ----------------------------------------------------------------


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/8.0
for any user-visible changes.

-- PMM
Philippe Mathieu-Daudé Feb. 24, 2023, 9:35 p.m. UTC | #2
Hi,

On 24/2/23 19:50, Peter Maydell wrote:
> On Thu, 23 Feb 2023 at 18:51, Kevin Wolf <kwolf@redhat.com> wrote:
>>
>> The following changes since commit 79b677d658d3d35e1e776826ac4abb28cdce69b8:
>>
>>    Merge tag 'net-pull-request' of https://github.com/jasowang/qemu into staging (2023-02-21 11:28:31 +0000)
>>
>> are available in the Git repository at:
>>
>>    https://repo.or.cz/qemu/kevin.git tags/for-upstream
>>
>> for you to fetch changes up to 0f385a2420d2c3f8ae7ed65fbe2712027664059e:
>>
>>    block/rbd: Add support for layered encryption (2023-02-23 19:49:35 +0100)
>>
>> ----------------------------------------------------------------
>> Block layer patches
>>
>> - Lock the graph, part 2 (BlockDriver callbacks)
>> - virtio-scsi: fix SCSIDevice hot unplug with IOThread
>> - rbd: Add support for layered encryption
>>
>> ----------------------------------------------------------------
> 
> 
> Applied, thanks.

Configuring with --extra-cflags=-ggdb, on

C compiler for the host machine: clang (clang 14.0.0 "Apple clang 
version 14.0.0 (clang-1400.0.29.202)")
C linker for the host machine: clang ld64 820.1
Host machine cpu family: aarch64
Host machine cpu: aarch64
...
     CFLAGS                       : -ggdb -g -O2

I'm getting:

../../block/io.c:182:38: warning: reading variable 'bdrv_aio_preadv' 
requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
                                 drv->bdrv_aio_preadv ||
                                      ^
../../block/io.c:997:14: warning: reading variable 'bdrv_aio_preadv' 
requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
     if (drv->bdrv_aio_preadv) {
              ^
../../block/io.c:1003:20: warning: reading variable 'bdrv_aio_preadv' 
requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
         acb = drv->bdrv_aio_preadv(bs, offset, bytes, qiov, flags,
                    ^
../../block/io.c:1076:14: warning: reading variable 'bdrv_aio_pwritev' 
requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
     if (drv->bdrv_aio_pwritev) {
              ^
../../block/io.c:1082:20: warning: reading variable 'bdrv_aio_pwritev' 
requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
         acb = drv->bdrv_aio_pwritev(bs, offset, bytes, qiov, flags,
                    ^
../../block/io.c:2899:25: warning: reading variable 'bdrv_aio_flush' 
requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
     } else if (bs->drv->bdrv_aio_flush) {
                         ^
../../block/io.c:2905:24: warning: reading variable 'bdrv_aio_flush' 
requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
         acb = bs->drv->bdrv_aio_flush(bs, bdrv_co_io_em_complete, &co);
                        ^
../../block/io.c:2991:49: warning: reading variable 'bdrv_aio_pdiscard' 
requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
     if (!bs->drv->bdrv_co_pdiscard && !bs->drv->bdrv_aio_pdiscard) {
                                                 ^
../../block/io.c:3058:28: warning: reading variable 'bdrv_aio_pdiscard' 
requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
             acb = bs->drv->bdrv_aio_pdiscard(bs, offset, num,
                            ^
../../block/io.c:3094:24: warning: reading variable 'bdrv_aio_ioctl' 
requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
     if (!drv || (!drv->bdrv_aio_ioctl && !drv->bdrv_co_ioctl)) {
                        ^
../../block/io.c:3102:20: warning: reading variable 'bdrv_aio_ioctl' 
requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
         acb = drv->bdrv_aio_ioctl(bs, req, buf, bdrv_co_io_em_complete, 
&co);
                    ^
11 warnings generated.
Thomas Huth Feb. 27, 2023, 9:12 a.m. UTC | #3
On 24/02/2023 22.35, Philippe Mathieu-Daudé wrote:
> Hi,
> 
> On 24/2/23 19:50, Peter Maydell wrote:
>> On Thu, 23 Feb 2023 at 18:51, Kevin Wolf <kwolf@redhat.com> wrote:
>>>
>>> The following changes since commit 79b677d658d3d35e1e776826ac4abb28cdce69b8:
>>>
>>>    Merge tag 'net-pull-request' of https://github.com/jasowang/qemu into 
>>> staging (2023-02-21 11:28:31 +0000)
>>>
>>> are available in the Git repository at:
>>>
>>>    https://repo.or.cz/qemu/kevin.git tags/for-upstream
>>>
>>> for you to fetch changes up to 0f385a2420d2c3f8ae7ed65fbe2712027664059e:
>>>
>>>    block/rbd: Add support for layered encryption (2023-02-23 19:49:35 +0100)
>>>
>>> ----------------------------------------------------------------
>>> Block layer patches
>>>
>>> - Lock the graph, part 2 (BlockDriver callbacks)
>>> - virtio-scsi: fix SCSIDevice hot unplug with IOThread
>>> - rbd: Add support for layered encryption
>>>
>>> ----------------------------------------------------------------
>>
>>
>> Applied, thanks.
> 
> Configuring with --extra-cflags=-ggdb, on
> 
> C compiler for the host machine: clang (clang 14.0.0 "Apple clang version 
> 14.0.0 (clang-1400.0.29.202)")
> C linker for the host machine: clang ld64 820.1
> Host machine cpu family: aarch64
> Host machine cpu: aarch64
> ...
>      CFLAGS                       : -ggdb -g -O2
> 
> I'm getting:
> 
> ../../block/io.c:182:38: warning: reading variable 'bdrv_aio_preadv' 
> requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
>                                  drv->bdrv_aio_preadv ||
>                                       ^
> ../../block/io.c:997:14: warning: reading variable 'bdrv_aio_preadv' 
> requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
>      if (drv->bdrv_aio_preadv) {
>               ^
> ../../block/io.c:1003:20: warning: reading variable 'bdrv_aio_preadv' 
> requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
>          acb = drv->bdrv_aio_preadv(bs, offset, bytes, qiov, flags,
>                     ^
> ../../block/io.c:1076:14: warning: reading variable 'bdrv_aio_pwritev' 
> requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
>      if (drv->bdrv_aio_pwritev) {
>               ^
> ../../block/io.c:1082:20: warning: reading variable 'bdrv_aio_pwritev' 
> requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
>          acb = drv->bdrv_aio_pwritev(bs, offset, bytes, qiov, flags,
>                     ^
> ../../block/io.c:2899:25: warning: reading variable 'bdrv_aio_flush' 
> requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
>      } else if (bs->drv->bdrv_aio_flush) {
>                          ^
> ../../block/io.c:2905:24: warning: reading variable 'bdrv_aio_flush' 
> requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
>          acb = bs->drv->bdrv_aio_flush(bs, bdrv_co_io_em_complete, &co);
>                         ^
> ../../block/io.c:2991:49: warning: reading variable 'bdrv_aio_pdiscard' 
> requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
>      if (!bs->drv->bdrv_co_pdiscard && !bs->drv->bdrv_aio_pdiscard) {
>                                                  ^
> ../../block/io.c:3058:28: warning: reading variable 'bdrv_aio_pdiscard' 
> requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
>              acb = bs->drv->bdrv_aio_pdiscard(bs, offset, num,
>                             ^
> ../../block/io.c:3094:24: warning: reading variable 'bdrv_aio_ioctl' 
> requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
>      if (!drv || (!drv->bdrv_aio_ioctl && !drv->bdrv_co_ioctl)) {
>                         ^
> ../../block/io.c:3102:20: warning: reading variable 'bdrv_aio_ioctl' 
> requires holding mutex 'graph_lock' [-Wthread-safety-analysis]
>          acb = drv->bdrv_aio_ioctl(bs, req, buf, bdrv_co_io_em_complete, &co);
>                     ^
> 11 warnings generated.

This can also reproduced in the gitlab-ci:

  https://gitlab.com/thuth/qemu/-/jobs/3837884040#L2862

Peter, in case you also have a github account, could you maybe enable the 
Cirrus-CI for your gitlab repo like it is explained here:

  .gitlab-ci.d/cirrus/README.rst

?

  Thanks,
   Thomas
Peter Maydell Feb. 27, 2023, 11:22 a.m. UTC | #4
On Mon, 27 Feb 2023 at 09:12, Thomas Huth <thuth@redhat.com> wrote:
> Peter, in case you also have a github account, could you maybe enable the
> Cirrus-CI for your gitlab repo like it is explained here:
>
>   .gitlab-ci.d/cirrus/README.rst

Sorry, no. The Cirrus-CI signup wants more permissions on
my github account than I am willing to give it.

thanks
-- PMM