diff mbox series

util/async: Add memory barrier to aio_ctx_prepare

Message ID 20200402011803.1270-1-fangying1@huawei.com
State New
Headers show
Series util/async: Add memory barrier to aio_ctx_prepare | expand

Commit Message

fangying April 2, 2020, 1:18 a.m. UTC
Qemu main thread is found to hang up in the mainloop when doing
image format convert on aarch64 platform and it is highly
reproduceable by executing test using:

qemu-img convert -f qcow2 -O qcow2 origin.qcow2 converted.qcow2

This mysterious hang can be explained by a race condition between
the main thread and an io worker thread. There can be a chance that
the last worker thread has called aio_bh_schedule_oneshot and it is
checking against notify_me to deliver a notfiy event. At the same
time, the main thread is calling aio_ctx_prepare however it first
calls qemu_timeout_ns_to_ms, thus the worker thread did not see
notify_me as true and did not send a notify event. The time line
can be shown in the following way:

 Main Thread
 ------------------------------------------------
 aio_ctx_prepare
    atomic_or(&ctx->notify_me, 1);
    /* out of order execution goes here */
    *timeout = qemu_timeout_ns_to_ms(aio_compute_timeout(ctx));

 Worker Thread
 -----------------------------------------------
 aio_bh_schedule_oneshot -> aio_bh_enqueue
    aio_notify
	smp_mb();
	if (ctx->notify_me) {   /* worker thread checks notify_me here */
	    event_notifier_set(&ctx->notifier);
	    atomic_mb_set(&ctx->notified, true);
	}

Normal VM runtime is not affected by this hang since there is always some
timer timeout or subsequent io worker come and notify the main thead.
To fix this problem, a memory barrier is added to aio_ctx_prepare and
it is proved to have the hang fixed in our test.

This hang is not observed on the x86 platform however it can be easily
reproduced on the aarch64 platform, thus it is architecture related.
Not sure if this is revelant to Commit eabc977973103527bbb8fed69c91cfaa6691f8ab

Signed-off-by: Ying Fang <fangying1@huawei.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
---
 util/async.c | 1 +
 1 file changed, 1 insertion(+)

Comments

no-reply@patchew.org April 2, 2020, 3:29 a.m. UTC | #1
Patchew URL: https://patchew.org/QEMU/20200402011803.1270-1-fangying1@huawei.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [PATCH] util/async: Add memory barrier to aio_ctx_prepare
Message-id: 20200402011803.1270-1-fangying1@huawei.com
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag]         patchew/20200402024431.1629-1-fangying1@huawei.com -> patchew/20200402024431.1629-1-fangying1@huawei.com
Switched to a new branch 'test'
06a5d59 util/async: Add memory barrier to aio_ctx_prepare

=== OUTPUT BEGIN ===
ERROR: memory barrier without comment
#60: FILE: util/async.c:254:
+    smp_mb();

total: 1 errors, 0 warnings, 7 lines checked

Commit 06a5d59d541d (util/async: Add memory barrier to aio_ctx_prepare) has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20200402011803.1270-1-fangying1@huawei.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
diff mbox series

Patch

diff --git a/util/async.c b/util/async.c
index b94518b948..a51fffc20d 100644
--- a/util/async.c
+++ b/util/async.c
@@ -251,6 +251,7 @@  aio_ctx_prepare(GSource *source, gint    *timeout)
 
     atomic_or(&ctx->notify_me, 1);
 
+    smp_mb();
     /* We assume there is no timeout already supplied */
     *timeout = qemu_timeout_ns_to_ms(aio_compute_timeout(ctx));