From patchwork Fri Feb 23 23:51:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877337 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7Tn2QZjz9sW0 for ; Sat, 24 Feb 2018 10:58:36 +1100 (AEDT) Received: from localhost ([::1]:47896 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNEX-0002ps-FQ for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 18:58:33 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55317) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8M-0005sZ-R8 for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:12 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8J-0005uV-MN for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:10 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:44370 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8E-0005hU-CA; Fri, 23 Feb 2018 18:52:02 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C1258EAEBE; Fri, 23 Feb 2018 23:51:51 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8A39610B0F55; Fri, 23 Feb 2018 23:51:51 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:22 -0500 Message-Id: <20180223235142.21501-2-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Fri, 23 Feb 2018 23:51:51 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Fri, 23 Feb 2018 23:51:51 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 01/21] blockjobs: fix set-speed kick X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" If speed is '0' it's not actually "less than" the previous speed. Kick the job in this case too. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf --- blockjob.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blockjob.c b/blockjob.c index 3f52f29f75..24833ef30f 100644 --- a/blockjob.c +++ b/blockjob.c @@ -499,7 +499,7 @@ void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) } job->speed = speed; - if (speed <= old_speed) { + if (speed && speed <= old_speed) { return; } From patchwork Fri Feb 23 23:51:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877344 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7bc6Ndbz9sW0 for ; Sat, 24 Feb 2018 11:03:39 +1100 (AEDT) Received: from localhost ([::1]:47933 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNJR-0006wh-JQ for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 19:03:37 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55474) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8S-0005zP-0k for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8O-00063a-9n for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:16 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33880 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8E-0005iJ-NV; Fri, 23 Feb 2018 18:52:02 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 001BE4085010; Fri, 23 Feb 2018 23:51:51 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id C352810AF9FC; Fri, 23 Feb 2018 23:51:51 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:23 -0500 Message-Id: <20180223235142.21501-3-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:52 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:52 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 02/21] blockjobs: model single jobs as transactions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" model all independent jobs as single job transactions. It's one less case we have to worry about when we add more states to the transition machine. This way, we can just treat all job lifetimes exactly the same. This helps tighten assertions of the STM graph and removes some conditionals that would have been needed in the coming commits adding a more explicit job lifetime management API. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf --- block/backup.c | 3 +-- block/commit.c | 2 +- block/mirror.c | 2 +- block/stream.c | 2 +- blockjob.c | 25 ++++++++++++++++--------- include/block/blockjob_int.h | 3 ++- tests/test-bdrv-drain.c | 4 ++-- tests/test-blockjob-txn.c | 19 +++++++------------ tests/test-blockjob.c | 2 +- 9 files changed, 32 insertions(+), 30 deletions(-) diff --git a/block/backup.c b/block/backup.c index 4a16a37229..7e254dabff 100644 --- a/block/backup.c +++ b/block/backup.c @@ -621,7 +621,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, } /* job->common.len is fixed, so we can't allow resize */ - job = block_job_create(job_id, &backup_job_driver, bs, + job = block_job_create(job_id, &backup_job_driver, txn, bs, BLK_PERM_CONSISTENT_READ, BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE | BLK_PERM_WRITE_UNCHANGED | BLK_PERM_GRAPH_MOD, @@ -677,7 +677,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL, &error_abort); job->common.len = len; - block_job_txn_add_job(txn, &job->common); return &job->common; diff --git a/block/commit.c b/block/commit.c index bb6c904704..9682158ee7 100644 --- a/block/commit.c +++ b/block/commit.c @@ -289,7 +289,7 @@ void commit_start(const char *job_id, BlockDriverState *bs, return; } - s = block_job_create(job_id, &commit_job_driver, bs, 0, BLK_PERM_ALL, + s = block_job_create(job_id, &commit_job_driver, NULL, bs, 0, BLK_PERM_ALL, speed, BLOCK_JOB_DEFAULT, NULL, NULL, errp); if (!s) { return; diff --git a/block/mirror.c b/block/mirror.c index c9badc1203..6bab7cfdd8 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1166,7 +1166,7 @@ static void mirror_start_job(const char *job_id, BlockDriverState *bs, } /* Make sure that the source is not resized while the job is running */ - s = block_job_create(job_id, driver, mirror_top_bs, + s = block_job_create(job_id, driver, NULL, mirror_top_bs, BLK_PERM_CONSISTENT_READ, BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGED | BLK_PERM_WRITE | BLK_PERM_GRAPH_MOD, speed, diff --git a/block/stream.c b/block/stream.c index 499cdacdb0..f3b53f49e2 100644 --- a/block/stream.c +++ b/block/stream.c @@ -244,7 +244,7 @@ void stream_start(const char *job_id, BlockDriverState *bs, /* Prevent concurrent jobs trying to modify the graph structure here, we * already have our own plans. Also don't allow resize as the image size is * queried only at the job start and then cached. */ - s = block_job_create(job_id, &stream_job_driver, bs, + s = block_job_create(job_id, &stream_job_driver, NULL, bs, BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGED | BLK_PERM_GRAPH_MOD, BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGED | diff --git a/blockjob.c b/blockjob.c index 24833ef30f..7ba3683ee3 100644 --- a/blockjob.c +++ b/blockjob.c @@ -357,10 +357,8 @@ static void block_job_completed_single(BlockJob *job) } } - if (job->txn) { - QLIST_REMOVE(job, txn_list); - block_job_txn_unref(job->txn); - } + QLIST_REMOVE(job, txn_list); + block_job_txn_unref(job->txn); block_job_unref(job); } @@ -647,7 +645,7 @@ static void block_job_event_completed(BlockJob *job, const char *msg) */ void *block_job_create(const char *job_id, const BlockJobDriver *driver, - BlockDriverState *bs, uint64_t perm, + BlockJobTxn *txn, BlockDriverState *bs, uint64_t perm, uint64_t shared_perm, int64_t speed, int flags, BlockCompletionFunc *cb, void *opaque, Error **errp) { @@ -729,6 +727,17 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, return NULL; } } + + /* Single jobs are modeled as single-job transactions for sake of + * consolidating the job management logic */ + if (!txn) { + txn = block_job_txn_new(); + block_job_txn_add_job(txn, job); + block_job_txn_unref(txn); + } else { + block_job_txn_add_job(txn, job); + } + return job; } @@ -752,13 +761,11 @@ void block_job_early_fail(BlockJob *job) void block_job_completed(BlockJob *job, int ret) { + assert(job && job->txn && !job->completed); assert(blk_bs(job->blk)->job == job); - assert(!job->completed); job->completed = true; job->ret = ret; - if (!job->txn) { - block_job_completed_single(job); - } else if (ret < 0 || block_job_is_cancelled(job)) { + if (ret < 0 || block_job_is_cancelled(job)) { block_job_completed_txn_abort(job); } else { block_job_completed_txn_success(job); diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h index c9b23b0cc9..becaae74c2 100644 --- a/include/block/blockjob_int.h +++ b/include/block/blockjob_int.h @@ -115,6 +115,7 @@ struct BlockJobDriver { * @job_id: The id of the newly-created job, or %NULL to have one * generated automatically. * @job_type: The class object for the newly-created job. + * @txn: The transaction this job belongs to, if any. %NULL otherwise. * @bs: The block * @perm, @shared_perm: Permissions to request for @bs * @speed: The maximum speed, in bytes per second, or 0 for unlimited. @@ -132,7 +133,7 @@ struct BlockJobDriver { * called from a wrapper that is specific to the job type. */ void *block_job_create(const char *job_id, const BlockJobDriver *driver, - BlockDriverState *bs, uint64_t perm, + BlockJobTxn *txn, BlockDriverState *bs, uint64_t perm, uint64_t shared_perm, int64_t speed, int flags, BlockCompletionFunc *cb, void *opaque, Error **errp); diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c index d760e2b243..a7eecf1c1e 100644 --- a/tests/test-bdrv-drain.c +++ b/tests/test-bdrv-drain.c @@ -541,8 +541,8 @@ static void test_blockjob_common(enum drain_type drain_type) blk_target = blk_new(BLK_PERM_ALL, BLK_PERM_ALL); blk_insert_bs(blk_target, target, &error_abort); - job = block_job_create("job0", &test_job_driver, src, 0, BLK_PERM_ALL, 0, - 0, NULL, NULL, &error_abort); + job = block_job_create("job0", &test_job_driver, NULL, src, 0, BLK_PERM_ALL, + 0, 0, NULL, NULL, &error_abort); block_job_add_bdrv(job, "target", target, 0, BLK_PERM_ALL, &error_abort); block_job_start(job); diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c index 3591c9617f..34f09ef8c1 100644 --- a/tests/test-blockjob-txn.c +++ b/tests/test-blockjob-txn.c @@ -87,7 +87,7 @@ static const BlockJobDriver test_block_job_driver = { */ static BlockJob *test_block_job_start(unsigned int iterations, bool use_timer, - int rc, int *result) + int rc, int *result, BlockJobTxn *txn) { BlockDriverState *bs; TestBlockJob *s; @@ -101,7 +101,7 @@ static BlockJob *test_block_job_start(unsigned int iterations, g_assert_nonnull(bs); snprintf(job_id, sizeof(job_id), "job%u", counter++); - s = block_job_create(job_id, &test_block_job_driver, bs, + s = block_job_create(job_id, &test_block_job_driver, txn, bs, 0, BLK_PERM_ALL, 0, BLOCK_JOB_DEFAULT, test_block_job_cb, data, &error_abort); s->iterations = iterations; @@ -120,8 +120,7 @@ static void test_single_job(int expected) int result = -EINPROGRESS; txn = block_job_txn_new(); - job = test_block_job_start(1, true, expected, &result); - block_job_txn_add_job(txn, job); + job = test_block_job_start(1, true, expected, &result, txn); block_job_start(job); if (expected == -ECANCELED) { @@ -160,10 +159,8 @@ static void test_pair_jobs(int expected1, int expected2) int result2 = -EINPROGRESS; txn = block_job_txn_new(); - job1 = test_block_job_start(1, true, expected1, &result1); - block_job_txn_add_job(txn, job1); - job2 = test_block_job_start(2, true, expected2, &result2); - block_job_txn_add_job(txn, job2); + job1 = test_block_job_start(1, true, expected1, &result1, txn); + job2 = test_block_job_start(2, true, expected2, &result2, txn); block_job_start(job1); block_job_start(job2); @@ -224,10 +221,8 @@ static void test_pair_jobs_fail_cancel_race(void) int result2 = -EINPROGRESS; txn = block_job_txn_new(); - job1 = test_block_job_start(1, true, -ECANCELED, &result1); - block_job_txn_add_job(txn, job1); - job2 = test_block_job_start(2, false, 0, &result2); - block_job_txn_add_job(txn, job2); + job1 = test_block_job_start(1, true, -ECANCELED, &result1, txn); + job2 = test_block_job_start(2, false, 0, &result2, txn); block_job_start(job1); block_job_start(job2); diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c index 23bdf1a932..599e28d732 100644 --- a/tests/test-blockjob.c +++ b/tests/test-blockjob.c @@ -30,7 +30,7 @@ static BlockJob *do_test_id(BlockBackend *blk, const char *id, BlockJob *job; Error *errp = NULL; - job = block_job_create(id, &test_block_job_driver, blk_bs(blk), + job = block_job_create(id, &test_block_job_driver, NULL, blk_bs(blk), 0, BLK_PERM_ALL, 0, BLOCK_JOB_DEFAULT, block_job_cb, NULL, &errp); if (should_succeed) { From patchwork Fri Feb 23 23:51:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877338 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7V23NXRz9sW0 for ; Sat, 24 Feb 2018 10:58:50 +1100 (AEDT) Received: from localhost ([::1]:47902 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNEm-00034e-Ir for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 18:58:48 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55392) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8P-0005vW-4W for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8N-00061C-0S for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:13 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33882 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8E-0005is-RY; Fri, 23 Feb 2018 18:52:03 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3608E400186C; Fri, 23 Feb 2018 23:51:52 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 05A4610A9DC0; Fri, 23 Feb 2018 23:51:52 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:24 -0500 Message-Id: <20180223235142.21501-4-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:52 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:52 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 03/21] blockjobs: add manual property X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This property will be used to opt-in to the new BlockJobs workflow that allows a tighter, more explicit control over transitions from one runstate to another. While we're here, fix up the documentation for block_job_create a little bit. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf --- blockjob.c | 1 + include/block/blockjob.h | 10 ++++++++++ include/block/blockjob_int.h | 4 +++- 3 files changed, 14 insertions(+), 1 deletion(-) diff --git a/blockjob.c b/blockjob.c index 7ba3683ee3..47468331ec 100644 --- a/blockjob.c +++ b/blockjob.c @@ -700,6 +700,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, job->paused = true; job->pause_count = 1; job->refcnt = 1; + job->manual = (flags & BLOCK_JOB_MANUAL); aio_timer_init(qemu_get_aio_context(), &job->sleep_timer, QEMU_CLOCK_REALTIME, SCALE_NS, block_job_sleep_timer_cb, job); diff --git a/include/block/blockjob.h b/include/block/blockjob.h index 00403d9482..8ffabdcbc4 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -141,14 +141,24 @@ typedef struct BlockJob { */ QEMUTimer sleep_timer; + /** + * Set to true when the management API has requested manual job + * management semantics. + */ + bool manual; + /** Non-NULL if this job is part of a transaction */ BlockJobTxn *txn; QLIST_ENTRY(BlockJob) txn_list; } BlockJob; typedef enum BlockJobCreateFlags { + /* Default behavior */ BLOCK_JOB_DEFAULT = 0x00, + /* BlockJob is not QMP-created and should not send QMP events */ BLOCK_JOB_INTERNAL = 0x01, + /* BlockJob requests manual job management steps. */ + BLOCK_JOB_MANUAL = 0x02, } BlockJobCreateFlags; /** diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h index becaae74c2..259d49b32a 100644 --- a/include/block/blockjob_int.h +++ b/include/block/blockjob_int.h @@ -114,11 +114,13 @@ struct BlockJobDriver { * block_job_create: * @job_id: The id of the newly-created job, or %NULL to have one * generated automatically. - * @job_type: The class object for the newly-created job. + * @driver: The class object for the newly-created job. * @txn: The transaction this job belongs to, if any. %NULL otherwise. * @bs: The block * @perm, @shared_perm: Permissions to request for @bs * @speed: The maximum speed, in bytes per second, or 0 for unlimited. + * @flags: Creation flags for the Block Job. + * See @BlockJobCreateFlags * @cb: Completion function for the job. * @opaque: Opaque pointer value passed to @cb. * @errp: Error object. From patchwork Fri Feb 23 23:51:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877358 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7qj4Shpz9sWB for ; Sat, 24 Feb 2018 11:14:08 +1100 (AEDT) Received: from localhost ([::1]:47999 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNTZ-0006gd-Pb for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 19:14:05 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55536) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8V-00062S-IN for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8S-0006Aj-AU for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:19 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33884 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8F-0005jS-1D; Fri, 23 Feb 2018 18:52:03 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6C6F24040856; Fri, 23 Feb 2018 23:51:52 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3BEBB10A9DC0; Fri, 23 Feb 2018 23:51:52 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:25 -0500 Message-Id: <20180223235142.21501-5-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:52 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:52 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 04/21] blockjobs: add status enum X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We're about to add several new states, and booleans are becoming unwieldly and difficult to reason about. It would help to have a more explicit bookkeeping of the state of blockjobs. To this end, add a new "status" field and add our existing states in a redundant manner alongside the bools they are replacing: UNDEFINED: Placeholder, default state. Not currently visible to QMP unless changes occur in the future to allow creating jobs without starting them via QMP. CREATED: replaces !!job->co && paused && !busy RUNNING: replaces effectively (!paused && busy) PAUSED: Nearly redundant with info->paused, which shows pause_count. This reports the actual status of the job, which almost always matches the paused request status. It differs in that it is strictly only true when the job has actually gone dormant. READY: replaces job->ready. STANDBY: Paused, but job->ready is true. New state additions in coming commits will not be quite so redundant: WAITING: Waiting on transaction. This job has finished all the work it can until the transaction converges, fails, or is canceled. PENDING: Pending authorization from user. This job has finished all the work it can until the job or transaction is finalized via block_job_finalize. This implies the transaction has converged and left the WAITING phase. ABORTING: Job has encountered an error condition and is in the process of aborting. CONCLUDED: Job has ceased all operations and has a return code available for query and may be dismissed via block_job_dismiss. NULL: Job has been dismissed and (should) be destroyed. Should never be visible to QMP. Some of these states appear somewhat superfluous, but it helps define the expected flow of a job; so some of the states wind up being synchronous empty transitions. Importantly, jobs can be in only one of these states at any given time, which helps code and external users alike reason about the current condition of a job unambiguously. Signed-off-by: John Snow Reviewed-by: Kevin Wolf --- blockjob.c | 9 +++++++++ include/block/blockjob.h | 7 +++++-- qapi/block-core.json | 31 ++++++++++++++++++++++++++++++- tests/qemu-iotests/109.out | 24 ++++++++++++------------ 4 files changed, 56 insertions(+), 15 deletions(-) diff --git a/blockjob.c b/blockjob.c index 47468331ec..1be9c20cff 100644 --- a/blockjob.c +++ b/blockjob.c @@ -320,6 +320,7 @@ void block_job_start(BlockJob *job) job->pause_count--; job->busy = true; job->paused = false; + job->status = BLOCK_JOB_STATUS_RUNNING; bdrv_coroutine_enter(blk_bs(job->blk), job->co); } @@ -598,6 +599,7 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) info->speed = job->speed; info->io_status = job->iostatus; info->ready = job->ready; + info->status = job->status; return info; } @@ -701,6 +703,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, job->pause_count = 1; job->refcnt = 1; job->manual = (flags & BLOCK_JOB_MANUAL); + job->status = BLOCK_JOB_STATUS_CREATED; aio_timer_init(qemu_get_aio_context(), &job->sleep_timer, QEMU_CLOCK_REALTIME, SCALE_NS, block_job_sleep_timer_cb, job); @@ -814,9 +817,14 @@ void coroutine_fn block_job_pause_point(BlockJob *job) } if (block_job_should_pause(job) && !block_job_is_cancelled(job)) { + BlockJobStatus status = job->status; + job->status = status == BLOCK_JOB_STATUS_READY ? \ + BLOCK_JOB_STATUS_STANDBY : \ + BLOCK_JOB_STATUS_PAUSED; job->paused = true; block_job_do_yield(job, -1); job->paused = false; + job->status = status; } if (job->driver->resume) { @@ -922,6 +930,7 @@ void block_job_iostatus_reset(BlockJob *job) void block_job_event_ready(BlockJob *job) { + job->status = BLOCK_JOB_STATUS_READY; job->ready = true; if (block_job_is_internal(job)) { diff --git a/include/block/blockjob.h b/include/block/blockjob.h index 8ffabdcbc4..e254359d6b 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -143,10 +143,13 @@ typedef struct BlockJob { /** * Set to true when the management API has requested manual job - * management semantics. + * management semantics. See @BlockJobStatus for details. */ bool manual; + /** Current state; See @BlockJobStatus for details. */ + BlockJobStatus status; + /** Non-NULL if this job is part of a transaction */ BlockJobTxn *txn; QLIST_ENTRY(BlockJob) txn_list; @@ -157,7 +160,7 @@ typedef enum BlockJobCreateFlags { BLOCK_JOB_DEFAULT = 0x00, /* BlockJob is not QMP-created and should not send QMP events */ BLOCK_JOB_INTERNAL = 0x01, - /* BlockJob requests manual job management steps. */ + /* BlockJob requests manual job management steps. See @BlockJobStatus. */ BLOCK_JOB_MANUAL = 0x02, } BlockJobCreateFlags; diff --git a/qapi/block-core.json b/qapi/block-core.json index 5c5921bfb7..deddee616a 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -955,6 +955,32 @@ { 'enum': 'BlockJobType', 'data': ['commit', 'stream', 'mirror', 'backup'] } +## +# @BlockJobStatus: +# +# Indicates the present state of a given blockjob in its lifetime. +# +# @undefined: Erroneous, default state. Should not ever be visible. +# +# @created: The job has been created, but not yet started. +# +# @running: The job is currently running. +# +# @paused: The job is running, but paused. The pause may be requested by +# either the QMP user or by internal processes. +# +# @ready: The job is running, but is ready for the user to signal completion. +# This is used for long-running jobs like mirror that are designed to +# run indefinitely. +# +# @standby: The job is ready, but paused. This is nearly identical to @paused. +# The job may return to @ready or otherwise be canceled. +# +# Since: 2.12 +## +{ 'enum': 'BlockJobStatus', + 'data': ['undefined', 'created', 'running', 'paused', 'ready', 'standby'] } + ## # @BlockJobInfo: # @@ -981,12 +1007,15 @@ # # @ready: true if the job may be completed (since 2.2) # +# @status: Current job state/status (since 2.12) +# # Since: 1.1 ## { 'struct': 'BlockJobInfo', 'data': {'type': 'str', 'device': 'str', 'len': 'int', 'offset': 'int', 'busy': 'bool', 'paused': 'bool', 'speed': 'int', - 'io-status': 'BlockDeviceIoStatus', 'ready': 'bool'} } + 'io-status': 'BlockDeviceIoStatus', 'ready': 'bool', + 'status': 'BlockJobStatus' } } ## # @query-block-jobs: diff --git a/tests/qemu-iotests/109.out b/tests/qemu-iotests/109.out index c189e2833d..d288f2eef6 100644 --- a/tests/qemu-iotests/109.out +++ b/tests/qemu-iotests/109.out @@ -19,7 +19,7 @@ read 65536/65536 bytes at offset 0 {"return": {}} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 1024, "offset": 1024, "speed": 0, "type": "mirror"}} -{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 1024, "offset": 1024, "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 1024, "offset": 1024, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 1024, "offset": 1024, "speed": 0, "type": "mirror"}} @@ -45,7 +45,7 @@ read 65536/65536 bytes at offset 0 {"return": {}} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 197120, "offset": 197120, "speed": 0, "type": "mirror"}} -{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 197120, "offset": 197120, "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 197120, "offset": 197120, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 197120, "offset": 197120, "speed": 0, "type": "mirror"}} @@ -71,7 +71,7 @@ read 65536/65536 bytes at offset 0 {"return": {}} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 327680, "offset": 327680, "speed": 0, "type": "mirror"}} -{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 327680, "offset": 327680, "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 327680, "offset": 327680, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 327680, "offset": 327680, "speed": 0, "type": "mirror"}} @@ -97,7 +97,7 @@ read 65536/65536 bytes at offset 0 {"return": {}} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 1024, "offset": 1024, "speed": 0, "type": "mirror"}} -{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 1024, "offset": 1024, "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 1024, "offset": 1024, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 1024, "offset": 1024, "speed": 0, "type": "mirror"}} @@ -123,7 +123,7 @@ read 65536/65536 bytes at offset 0 {"return": {}} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 65536, "offset": 65536, "speed": 0, "type": "mirror"}} -{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 65536, "offset": 65536, "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 65536, "offset": 65536, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 65536, "offset": 65536, "speed": 0, "type": "mirror"}} @@ -149,7 +149,7 @@ read 65536/65536 bytes at offset 0 {"return": {}} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 2560, "offset": 2560, "speed": 0, "type": "mirror"}} -{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 2560, "offset": 2560, "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 2560, "offset": 2560, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 2560, "offset": 2560, "speed": 0, "type": "mirror"}} @@ -174,7 +174,7 @@ read 65536/65536 bytes at offset 0 {"return": {}} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 2560, "offset": 2560, "speed": 0, "type": "mirror"}} -{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 2560, "offset": 2560, "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 2560, "offset": 2560, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 2560, "offset": 2560, "speed": 0, "type": "mirror"}} @@ -199,7 +199,7 @@ read 65536/65536 bytes at offset 0 {"return": {}} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 31457280, "offset": 31457280, "speed": 0, "type": "mirror"}} -{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 31457280, "offset": 31457280, "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 31457280, "offset": 31457280, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 31457280, "offset": 31457280, "speed": 0, "type": "mirror"}} @@ -224,7 +224,7 @@ read 65536/65536 bytes at offset 0 {"return": {}} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 327680, "offset": 327680, "speed": 0, "type": "mirror"}} -{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 327680, "offset": 327680, "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 327680, "offset": 327680, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 327680, "offset": 327680, "speed": 0, "type": "mirror"}} @@ -249,7 +249,7 @@ read 65536/65536 bytes at offset 0 {"return": {}} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 2048, "offset": 2048, "speed": 0, "type": "mirror"}} -{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 2048, "offset": 2048, "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 2048, "offset": 2048, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 2048, "offset": 2048, "speed": 0, "type": "mirror"}} @@ -265,7 +265,7 @@ Automatically detecting the format is dangerous for raw images, write operations Specify the 'raw' format explicitly to remove the restrictions. {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 512, "offset": 512, "speed": 0, "type": "mirror"}} -{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 512, "offset": 512, "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 512, "offset": 512, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 512, "offset": 512, "speed": 0, "type": "mirror"}} @@ -274,7 +274,7 @@ Images are identical. {"return": {}} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 512, "offset": 512, "speed": 0, "type": "mirror"}} -{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 512, "offset": 512, "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"io-status": "ok", "device": "src", "busy": false, "len": 512, "offset": 512, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 512, "offset": 512, "speed": 0, "type": "mirror"}} From patchwork Fri Feb 23 23:51:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877340 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7XV11LJz9sW0 for ; Sat, 24 Feb 2018 11:00:58 +1100 (AEDT) Received: from localhost ([::1]:47918 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNGq-0004zB-4f for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 19:00:56 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55300) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8M-0005rs-5i for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8I-0005s2-BO for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:10 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33863 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8A-0005Vf-K9; Fri, 23 Feb 2018 18:51:58 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A3518404085A; Fri, 23 Feb 2018 23:51:52 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 726C610AF9ED; Fri, 23 Feb 2018 23:51:52 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:26 -0500 Message-Id: <20180223235142.21501-6-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:52 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:52 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 05/21] blockjobs: add state transition table X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" The state transition table has mostly been implied. We're about to make it a bit more complex, so let's make the STM explicit instead. Perform state transitions with a function that for now just asserts the transition is appropriate. Transitions: Undefined -> Created: During job initialization. Created -> Running: Once the job is started. Jobs cannot transition from "Created" to "Paused" directly, but will instead synchronously transition to running to paused immediately. Running -> Paused: Normal workflow for pauses. Running -> Ready: Normal workflow for jobs reaching their sync point. (e.g. mirror) Ready -> Standby: Normal workflow for pausing ready jobs. Paused -> Running: Normal resume. Standby -> Ready: Resume of a Standby job. +---------+ |UNDEFINED| +--+------+ | +--v----+ |CREATED| +--+----+ | +--v----+ +------+ |RUNNING<----->PAUSED| +--+----+ +------+ | +--v--+ +-------+ |READY<------->STANDBY| +-----+ +-------+ Notably, there is no state presently defined as of this commit that deals with a job after the "running" or "ready" states, so this table will be adjusted alongside the commits that introduce those states. Signed-off-by: John Snow --- block/trace-events | 3 +++ blockjob.c | 42 ++++++++++++++++++++++++++++++++++++------ 2 files changed, 39 insertions(+), 6 deletions(-) diff --git a/block/trace-events b/block/trace-events index 02dd80ff0c..b75a0c8409 100644 --- a/block/trace-events +++ b/block/trace-events @@ -4,6 +4,9 @@ bdrv_open_common(void *bs, const char *filename, int flags, const char *format_name) "bs %p filename \"%s\" flags 0x%x format_name \"%s\"" bdrv_lock_medium(void *bs, bool locked) "bs %p locked %d" +# blockjob.c +block_job_state_transition(void *job, int ret, const char *legal, const char *s0, const char *s1) "job %p (ret: %d) attempting %s transition (%s-->%s)" + # block/block-backend.c blk_co_preadv(void *blk, void *bs, int64_t offset, unsigned int bytes, int flags) "blk %p bs %p offset %"PRId64" bytes %u flags 0x%x" blk_co_pwritev(void *blk, void *bs, int64_t offset, unsigned int bytes, int flags) "blk %p bs %p offset %"PRId64" bytes %u flags 0x%x" diff --git a/blockjob.c b/blockjob.c index 1be9c20cff..d745b3bb69 100644 --- a/blockjob.c +++ b/blockjob.c @@ -28,6 +28,7 @@ #include "block/block.h" #include "block/blockjob_int.h" #include "block/block_int.h" +#include "block/trace.h" #include "sysemu/block-backend.h" #include "qapi/error.h" #include "qapi/qmp/qerror.h" @@ -41,6 +42,34 @@ * block_job_enter. */ static QemuMutex block_job_mutex; +/* BlockJob State Transition Table */ +bool BlockJobSTT[BLOCK_JOB_STATUS__MAX][BLOCK_JOB_STATUS__MAX] = { + /* U, C, R, P, Y, S */ + /* U: */ [BLOCK_JOB_STATUS_UNDEFINED] = {0, 1, 0, 0, 0, 0}, + /* C: */ [BLOCK_JOB_STATUS_CREATED] = {0, 0, 1, 0, 0, 0}, + /* R: */ [BLOCK_JOB_STATUS_RUNNING] = {0, 0, 0, 1, 1, 0}, + /* P: */ [BLOCK_JOB_STATUS_PAUSED] = {0, 0, 1, 0, 0, 0}, + /* Y: */ [BLOCK_JOB_STATUS_READY] = {0, 0, 0, 0, 0, 1}, + /* S: */ [BLOCK_JOB_STATUS_STANDBY] = {0, 0, 0, 0, 1, 0}, +}; + +static void block_job_state_transition(BlockJob *job, BlockJobStatus s1) +{ + BlockJobStatus s0 = job->status; + if (s0 == s1) { + return; + } + assert(s1 >= 0 && s1 <= BLOCK_JOB_STATUS__MAX); + trace_block_job_state_transition(job, job->ret, BlockJobSTT[s0][s1] ? + "allowed" : "disallowed", + qapi_enum_lookup(&BlockJobStatus_lookup, + s0), + qapi_enum_lookup(&BlockJobStatus_lookup, + s1)); + assert(BlockJobSTT[s0][s1]); + job->status = s1; +} + static void block_job_lock(void) { qemu_mutex_lock(&block_job_mutex); @@ -320,7 +349,7 @@ void block_job_start(BlockJob *job) job->pause_count--; job->busy = true; job->paused = false; - job->status = BLOCK_JOB_STATUS_RUNNING; + block_job_state_transition(job, BLOCK_JOB_STATUS_RUNNING); bdrv_coroutine_enter(blk_bs(job->blk), job->co); } @@ -704,6 +733,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, job->refcnt = 1; job->manual = (flags & BLOCK_JOB_MANUAL); job->status = BLOCK_JOB_STATUS_CREATED; + block_job_state_transition(job, BLOCK_JOB_STATUS_CREATED); aio_timer_init(qemu_get_aio_context(), &job->sleep_timer, QEMU_CLOCK_REALTIME, SCALE_NS, block_job_sleep_timer_cb, job); @@ -818,13 +848,13 @@ void coroutine_fn block_job_pause_point(BlockJob *job) if (block_job_should_pause(job) && !block_job_is_cancelled(job)) { BlockJobStatus status = job->status; - job->status = status == BLOCK_JOB_STATUS_READY ? \ - BLOCK_JOB_STATUS_STANDBY : \ - BLOCK_JOB_STATUS_PAUSED; + block_job_state_transition(job, status == BLOCK_JOB_STATUS_READY ? \ + BLOCK_JOB_STATUS_STANDBY : \ + BLOCK_JOB_STATUS_PAUSED); job->paused = true; block_job_do_yield(job, -1); job->paused = false; - job->status = status; + block_job_state_transition(job, status); } if (job->driver->resume) { @@ -930,7 +960,7 @@ void block_job_iostatus_reset(BlockJob *job) void block_job_event_ready(BlockJob *job) { - job->status = BLOCK_JOB_STATUS_READY; + block_job_state_transition(job, BLOCK_JOB_STATUS_READY); job->ready = true; if (block_job_is_internal(job)) { From patchwork Fri Feb 23 23:51:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877339 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7VN0S0Kz9sW0 for ; Sat, 24 Feb 2018 10:59:08 +1100 (AEDT) Received: from localhost ([::1]:47907 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNF4-0003OK-5W for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 18:59:06 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55253) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8K-0005pY-QI for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:12 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8H-0005qJ-Qs for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:08 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:44364 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8A-0005Vg-K8; Fri, 23 Feb 2018 18:51:58 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D95C2EB6ED; Fri, 23 Feb 2018 23:51:52 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id A915F1049479; Fri, 23 Feb 2018 23:51:52 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:27 -0500 Message-Id: <20180223235142.21501-7-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Fri, 23 Feb 2018 23:51:52 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Fri, 23 Feb 2018 23:51:52 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 06/21] iotests: add pause_wait X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Split out the pause command into the actual pause and the wait. Not every usage presently needs to resubmit a pause request. The intent with the next commit will be to explicitly disallow redundant or meaningless pause/resume requests, so the tests need to become more judicious to reflect that. Signed-off-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Eric Blake --- tests/qemu-iotests/030 | 6 ++---- tests/qemu-iotests/055 | 17 ++++++----------- tests/qemu-iotests/iotests.py | 12 ++++++++---- 3 files changed, 16 insertions(+), 19 deletions(-) diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030 index 457984b8e9..251883226c 100755 --- a/tests/qemu-iotests/030 +++ b/tests/qemu-iotests/030 @@ -86,11 +86,9 @@ class TestSingleDrive(iotests.QMPTestCase): result = self.vm.qmp('block-stream', device='drive0') self.assert_qmp(result, 'return', {}) - result = self.vm.qmp('block-job-pause', device='drive0') - self.assert_qmp(result, 'return', {}) - + self.pause_job('drive0', wait=False) self.vm.resume_drive('drive0') - self.pause_job('drive0') + self.pause_wait('drive0') result = self.vm.qmp('query-block-jobs') offset = self.dictpath(result, 'return[0]/offset') diff --git a/tests/qemu-iotests/055 b/tests/qemu-iotests/055 index 8a5d9fd269..3437c11507 100755 --- a/tests/qemu-iotests/055 +++ b/tests/qemu-iotests/055 @@ -86,11 +86,9 @@ class TestSingleDrive(iotests.QMPTestCase): target=target, sync='full') self.assert_qmp(result, 'return', {}) - result = self.vm.qmp('block-job-pause', device='drive0') - self.assert_qmp(result, 'return', {}) - + self.pause_job('drive0', wait=False) self.vm.resume_drive('drive0') - self.pause_job('drive0') + self.pause_wait('drive0') result = self.vm.qmp('query-block-jobs') offset = self.dictpath(result, 'return[0]/offset') @@ -303,13 +301,12 @@ class TestSingleTransaction(iotests.QMPTestCase): ]) self.assert_qmp(result, 'return', {}) - result = self.vm.qmp('block-job-pause', device='drive0') - self.assert_qmp(result, 'return', {}) + self.pause_job('drive0', wait=False) result = self.vm.qmp('block-job-set-speed', device='drive0', speed=0) self.assert_qmp(result, 'return', {}) - self.pause_job('drive0') + self.pause_wait('drive0') result = self.vm.qmp('query-block-jobs') offset = self.dictpath(result, 'return[0]/offset') @@ -534,11 +531,9 @@ class TestDriveCompression(iotests.QMPTestCase): result = self.vm.qmp(cmd, device='drive0', sync='full', compress=True, **args) self.assert_qmp(result, 'return', {}) - result = self.vm.qmp('block-job-pause', device='drive0') - self.assert_qmp(result, 'return', {}) - + self.pause_job('drive0', wait=False) self.vm.resume_drive('drive0') - self.pause_job('drive0') + self.pause_wait('drive0') result = self.vm.qmp('query-block-jobs') offset = self.dictpath(result, 'return[0]/offset') diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 1bcc9ca57d..5303bbc8e2 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -473,10 +473,7 @@ class QMPTestCase(unittest.TestCase): event = self.wait_until_completed(drive=drive) self.assert_qmp(event, 'data/type', 'mirror') - def pause_job(self, job_id='job0'): - result = self.vm.qmp('block-job-pause', device=job_id) - self.assert_qmp(result, 'return', {}) - + def pause_wait(self, job_id='job0'): with Timeout(1, "Timeout waiting for job to pause"): while True: result = self.vm.qmp('query-block-jobs') @@ -484,6 +481,13 @@ class QMPTestCase(unittest.TestCase): if job['device'] == job_id and job['paused'] == True and job['busy'] == False: return job + def pause_job(self, job_id='job0', wait=True): + result = self.vm.qmp('block-job-pause', device=job_id) + self.assert_qmp(result, 'return', {}) + if wait: + return self.pause_wait(job_id) + return result + def notrun(reason): '''Skip this test suite''' From patchwork Fri Feb 23 23:51:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877356 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7lp5spKz9sW8 for ; Sat, 24 Feb 2018 11:10:46 +1100 (AEDT) Received: from localhost ([::1]:47977 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNQK-0004A2-SY for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 19:10:44 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55385) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8O-0005v3-R0 for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8J-0005uM-LO for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:12 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:44365 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8A-0005Vc-KG; Fri, 23 Feb 2018 18:51:58 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 21E82EAEB8; Fri, 23 Feb 2018 23:51:53 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id E018C1049479; Fri, 23 Feb 2018 23:51:52 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:28 -0500 Message-Id: <20180223235142.21501-8-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Fri, 23 Feb 2018 23:51:53 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Fri, 23 Feb 2018 23:51:53 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 07/21] blockjobs: add block_job_verb permission table X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Which commands ("verbs") are appropriate for jobs in which state is also somewhat burdensome to keep track of. As of this commit, it looks rather useless, but begins to look more interesting the more states we add to the STM table. A recurring theme is that no verb will apply to an 'undefined' job. Further, it's not presently possible to restrict the "pause" or "resume" verbs any more than they are in this commit because of the asynchronous nature of how jobs enter the PAUSED state; justifications for some seemingly erroneous applications are given below. Reviewed-by: Kevin Wolf Reviewed-by: Eric Blake ===== Verbs ===== Cancel: Any state except undefined. Pause: Any state except undefined; 'created': Requests that the job pauses as it starts. 'running': Normal usage. (PAUSED) 'paused': The job may be paused for internal reasons, but the user may wish to force an indefinite user-pause, so this is allowed. 'ready': Normal usage. (STANDBY) 'standby': Same logic as above. Resume: Any state except undefined; 'created': Will lift a user's pause-on-start request. 'running': Will lift a pause request before it takes effect. 'paused': Normal usage. 'ready': Will lift a pause request before it takes effect. 'standby': Normal usage. Set-speed: Any state except undefined, though ready may not be meaningful. Complete: Only a 'ready' job may accept a complete request. ======= Changes ======= (1) To facilitate "nice" error checking, all five major block-job verb interfaces in blockjob.c now support an errp parameter: - block_job_user_cancel is added as a new interface. - block_job_user_pause gains an errp paramter - block_job_user_resume gains an errp parameter - block_job_set_speed already had an errp parameter. - block_job_complete already had an errp parameter. (2) block-job-pause and block-job-resume will no longer no-op when trying to pause an already paused job, or trying to resume a job that isn't paused. These functions will now report that they did not perform the action requested because it was not possible. iotests have been adjusted to address this new behavior. (3) block-job-complete doesn't worry about checking !block_job_started, because the permission table guards against this. (4) test-bdrv-drain's job implementation needs to announce that it is 'ready' now, in order to be completed. Signed-off-by: John Snow --- block/trace-events | 1 + blockdev.c | 10 +++---- blockjob.c | 71 ++++++++++++++++++++++++++++++++++++++++++------ include/block/blockjob.h | 13 +++++++-- qapi/block-core.json | 20 ++++++++++++++ tests/test-bdrv-drain.c | 1 + 6 files changed, 100 insertions(+), 16 deletions(-) diff --git a/block/trace-events b/block/trace-events index b75a0c8409..3fe89f7ea6 100644 --- a/block/trace-events +++ b/block/trace-events @@ -6,6 +6,7 @@ bdrv_lock_medium(void *bs, bool locked) "bs %p locked %d" # blockjob.c block_job_state_transition(void *job, int ret, const char *legal, const char *s0, const char *s1) "job %p (ret: %d) attempting %s transition (%s-->%s)" +block_job_apply_verb(void *job, const char *state, const char *verb, const char *legal) "job %p in state %s; applying verb %s (%s)" # block/block-backend.c blk_co_preadv(void *blk, void *bs, int64_t offset, unsigned int bytes, int flags) "blk %p bs %p offset %"PRId64" bytes %u flags 0x%x" diff --git a/blockdev.c b/blockdev.c index 3fb1ca803c..cba935a0a6 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3805,7 +3805,7 @@ void qmp_block_job_cancel(const char *device, } trace_qmp_block_job_cancel(job); - block_job_cancel(job); + block_job_user_cancel(job, errp); out: aio_context_release(aio_context); } @@ -3815,12 +3815,12 @@ void qmp_block_job_pause(const char *device, Error **errp) AioContext *aio_context; BlockJob *job = find_block_job(device, &aio_context, errp); - if (!job || block_job_user_paused(job)) { + if (!job) { return; } trace_qmp_block_job_pause(job); - block_job_user_pause(job); + block_job_user_pause(job, errp); aio_context_release(aio_context); } @@ -3829,12 +3829,12 @@ void qmp_block_job_resume(const char *device, Error **errp) AioContext *aio_context; BlockJob *job = find_block_job(device, &aio_context, errp); - if (!job || !block_job_user_paused(job)) { + if (!job) { return; } trace_qmp_block_job_resume(job); - block_job_user_resume(job); + block_job_user_resume(job, errp); aio_context_release(aio_context); } diff --git a/blockjob.c b/blockjob.c index d745b3bb69..4e424fef72 100644 --- a/blockjob.c +++ b/blockjob.c @@ -53,6 +53,15 @@ bool BlockJobSTT[BLOCK_JOB_STATUS__MAX][BLOCK_JOB_STATUS__MAX] = { /* S: */ [BLOCK_JOB_STATUS_STANDBY] = {0, 0, 0, 0, 1, 0}, }; +bool BlockJobVerbTable[BLOCK_JOB_VERB__MAX][BLOCK_JOB_STATUS__MAX] = { + /* U, C, R, P, Y, S */ + [BLOCK_JOB_VERB_CANCEL] = {0, 1, 1, 1, 1, 1}, + [BLOCK_JOB_VERB_PAUSE] = {0, 1, 1, 1, 1, 1}, + [BLOCK_JOB_VERB_RESUME] = {0, 1, 1, 1, 1, 1}, + [BLOCK_JOB_VERB_SET_SPEED] = {0, 1, 1, 1, 1, 1}, + [BLOCK_JOB_VERB_COMPLETE] = {0, 0, 0, 0, 1, 0}, +}; + static void block_job_state_transition(BlockJob *job, BlockJobStatus s1) { BlockJobStatus s0 = job->status; @@ -70,6 +79,23 @@ static void block_job_state_transition(BlockJob *job, BlockJobStatus s1) job->status = s1; } +static int block_job_apply_verb(BlockJob *job, BlockJobVerb bv, Error **errp) +{ + assert(bv >= 0 && bv <= BLOCK_JOB_VERB__MAX); + trace_block_job_apply_verb(job, qapi_enum_lookup(&BlockJobStatus_lookup, + job->status), + qapi_enum_lookup(&BlockJobVerb_lookup, bv), + BlockJobVerbTable[bv][job->status] ? + "allowed" : "prohibited"); + if (BlockJobVerbTable[bv][job->status]) { + return 0; + } + error_setg(errp, "Job '%s' in state '%s' cannot accept command verb '%s'", + job->id, qapi_enum_lookup(&BlockJobStatus_lookup, job->status), + qapi_enum_lookup(&BlockJobVerb_lookup, bv)); + return -EPERM; +} + static void block_job_lock(void) { qemu_mutex_lock(&block_job_mutex); @@ -520,6 +546,9 @@ void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) error_setg(errp, QERR_UNSUPPORTED); return; } + if (block_job_apply_verb(job, BLOCK_JOB_VERB_SET_SPEED, errp)) { + return; + } job->driver->set_speed(job, speed, &local_err); if (local_err) { error_propagate(errp, local_err); @@ -539,8 +568,10 @@ void block_job_complete(BlockJob *job, Error **errp) { /* Should not be reachable via external interface for internal jobs */ assert(job->id); - if (job->pause_count || job->cancelled || - !block_job_started(job) || !job->driver->complete) { + if (block_job_apply_verb(job, BLOCK_JOB_VERB_COMPLETE, errp)) { + return; + } + if (job->pause_count || job->cancelled || !job->driver->complete) { error_setg(errp, "The active block job '%s' cannot be completed", job->id); return; @@ -549,8 +580,15 @@ void block_job_complete(BlockJob *job, Error **errp) job->driver->complete(job, errp); } -void block_job_user_pause(BlockJob *job) +void block_job_user_pause(BlockJob *job, Error **errp) { + if (block_job_apply_verb(job, BLOCK_JOB_VERB_PAUSE, errp)) { + return; + } + if (job->user_paused) { + error_setg(errp, "Job is already paused"); + return; + } job->user_paused = true; block_job_pause(job); } @@ -560,13 +598,19 @@ bool block_job_user_paused(BlockJob *job) return job->user_paused; } -void block_job_user_resume(BlockJob *job) +void block_job_user_resume(BlockJob *job, Error **errp) { - if (job && job->user_paused && job->pause_count > 0) { - block_job_iostatus_reset(job); - job->user_paused = false; - block_job_resume(job); + assert(job); + if (!job->user_paused || job->pause_count <= 0) { + error_setg(errp, "Can't resume a job that was not paused"); + return; } + if (block_job_apply_verb(job, BLOCK_JOB_VERB_RESUME, errp)) { + return; + } + block_job_iostatus_reset(job); + job->user_paused = false; + block_job_resume(job); } void block_job_cancel(BlockJob *job) @@ -579,6 +623,14 @@ void block_job_cancel(BlockJob *job) } } +void block_job_user_cancel(BlockJob *job, Error **errp) +{ + if (block_job_apply_verb(job, BLOCK_JOB_VERB_CANCEL, errp)) { + return; + } + block_job_cancel(job); +} + /* A wrapper around block_job_cancel() taking an Error ** parameter so it may be * used with block_job_finish_sync() without the need for (rather nasty) * function pointer casts there. */ @@ -1004,8 +1056,9 @@ BlockErrorAction block_job_error_action(BlockJob *job, BlockdevOnError on_err, action, &error_abort); } if (action == BLOCK_ERROR_ACTION_STOP) { + block_job_pause(job); /* make the pause user visible, which will be resumed from QMP. */ - block_job_user_pause(job); + job->user_paused = true; block_job_iostatus_set_err(job, error); } return action; diff --git a/include/block/blockjob.h b/include/block/blockjob.h index e254359d6b..403a168073 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -260,7 +260,7 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp); * Asynchronously pause the specified job. * Do not allow a resume until a matching call to block_job_user_resume. */ -void block_job_user_pause(BlockJob *job); +void block_job_user_pause(BlockJob *job, Error **errp); /** * block_job_paused: @@ -277,7 +277,16 @@ bool block_job_user_paused(BlockJob *job); * Resume the specified job. * Must be paired with a preceding block_job_user_pause. */ -void block_job_user_resume(BlockJob *job); +void block_job_user_resume(BlockJob *job, Error **errp); + +/** + * block_job_user_cancel: + * @job: The job to be cancelled. + * + * Cancels the specified job, but may refuse to do so if the + * operation isn't currently meaningful. + */ +void block_job_user_cancel(BlockJob *job, Error **errp); /** * block_job_cancel_sync: diff --git a/qapi/block-core.json b/qapi/block-core.json index deddee616a..11659496c5 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -955,6 +955,26 @@ { 'enum': 'BlockJobType', 'data': ['commit', 'stream', 'mirror', 'backup'] } +## +# @BlockJobVerb: +# +# Represents command verbs that can be applied to a blockjob. +# +# @cancel: see @block-job-cancel +# +# @pause: see @block-job-pause +# +# @resume: see @block-job-resume +# +# @set-speed: see @block-job-set-speed +# +# @complete: see @block-job-complete +# +# Since: 2.12 +## +{ 'enum': 'BlockJobVerb', + 'data': ['cancel', 'pause', 'resume', 'set-speed', 'complete' ] } + ## # @BlockJobStatus: # diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c index a7eecf1c1e..7673de1062 100644 --- a/tests/test-bdrv-drain.c +++ b/tests/test-bdrv-drain.c @@ -505,6 +505,7 @@ static void coroutine_fn test_job_start(void *opaque) { TestBlockJob *s = opaque; + block_job_event_ready(&s->common); while (!s->should_complete) { block_job_sleep_ns(&s->common, 100000); } From patchwork Fri Feb 23 23:51:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877342 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7Y464wGz9sW0 for ; Sat, 24 Feb 2018 11:01:28 +1100 (AEDT) Received: from localhost ([::1]:47923 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNHK-0005Kk-TF for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 19:01:26 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55289) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8L-0005rD-Sb for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:12 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8H-0005qo-VX for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:09 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33862 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8A-0005Vb-KD; Fri, 23 Feb 2018 18:51:58 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5896D407246C; Fri, 23 Feb 2018 23:51:53 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 27E751049479; Fri, 23 Feb 2018 23:51:53 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:29 -0500 Message-Id: <20180223235142.21501-9-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:53 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:53 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 08/21] blockjobs: add ABORTING state X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Add a new state ABORTING. This makes transitions from normative states to error states explicit in the STM, and serves as a disambiguation for which states may complete normally when normal end-states (CONCLUDED) are added in future commits. Notably, Paused/Standby jobs do not transition directly to aborting, as they must wake up first and cooperate in their cancellation. Transitions: Running -> Aborting: can be cancelled or encounter an error Ready -> Aborting: can be cancelled or encounter an error Verbs: None. The job must finish cleaning itself up and report its final status. +---------+ |UNDEFINED| +--+------+ | +--v----+ |CREATED| +--+----+ | +--v----+ +------+ +---------+RUNNING<----->PAUSED| | +--+----+ +------+ | | | +--v--+ +-------+ +---------+READY<------->STANDBY| | +-----+ +-------+ | +--v-----+ |ABORTING| +--------+ Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf --- blockjob.c | 31 ++++++++++++++++++------------- qapi/block-core.json | 7 ++++++- 2 files changed, 24 insertions(+), 14 deletions(-) diff --git a/blockjob.c b/blockjob.c index 4e424fef72..4c3fcda46c 100644 --- a/blockjob.c +++ b/blockjob.c @@ -44,22 +44,23 @@ static QemuMutex block_job_mutex; /* BlockJob State Transition Table */ bool BlockJobSTT[BLOCK_JOB_STATUS__MAX][BLOCK_JOB_STATUS__MAX] = { - /* U, C, R, P, Y, S */ - /* U: */ [BLOCK_JOB_STATUS_UNDEFINED] = {0, 1, 0, 0, 0, 0}, - /* C: */ [BLOCK_JOB_STATUS_CREATED] = {0, 0, 1, 0, 0, 0}, - /* R: */ [BLOCK_JOB_STATUS_RUNNING] = {0, 0, 0, 1, 1, 0}, - /* P: */ [BLOCK_JOB_STATUS_PAUSED] = {0, 0, 1, 0, 0, 0}, - /* Y: */ [BLOCK_JOB_STATUS_READY] = {0, 0, 0, 0, 0, 1}, - /* S: */ [BLOCK_JOB_STATUS_STANDBY] = {0, 0, 0, 0, 1, 0}, + /* U, C, R, P, Y, S, X */ + /* U: */ [BLOCK_JOB_STATUS_UNDEFINED] = {0, 1, 0, 0, 0, 0, 0}, + /* C: */ [BLOCK_JOB_STATUS_CREATED] = {0, 0, 1, 0, 0, 0, 0}, + /* R: */ [BLOCK_JOB_STATUS_RUNNING] = {0, 0, 0, 1, 1, 0, 1}, + /* P: */ [BLOCK_JOB_STATUS_PAUSED] = {0, 0, 1, 0, 0, 0, 0}, + /* Y: */ [BLOCK_JOB_STATUS_READY] = {0, 0, 0, 0, 0, 1, 1}, + /* S: */ [BLOCK_JOB_STATUS_STANDBY] = {0, 0, 0, 0, 1, 0, 0}, + /* X: */ [BLOCK_JOB_STATUS_ABORTING] = {0, 0, 0, 0, 0, 0, 0}, }; bool BlockJobVerbTable[BLOCK_JOB_VERB__MAX][BLOCK_JOB_STATUS__MAX] = { - /* U, C, R, P, Y, S */ - [BLOCK_JOB_VERB_CANCEL] = {0, 1, 1, 1, 1, 1}, - [BLOCK_JOB_VERB_PAUSE] = {0, 1, 1, 1, 1, 1}, - [BLOCK_JOB_VERB_RESUME] = {0, 1, 1, 1, 1, 1}, - [BLOCK_JOB_VERB_SET_SPEED] = {0, 1, 1, 1, 1, 1}, - [BLOCK_JOB_VERB_COMPLETE] = {0, 0, 0, 0, 1, 0}, + /* U, C, R, P, Y, S, X */ + [BLOCK_JOB_VERB_CANCEL] = {0, 1, 1, 1, 1, 1, 0}, + [BLOCK_JOB_VERB_PAUSE] = {0, 1, 1, 1, 1, 1, 0}, + [BLOCK_JOB_VERB_RESUME] = {0, 1, 1, 1, 1, 1, 0}, + [BLOCK_JOB_VERB_SET_SPEED] = {0, 1, 1, 1, 1, 1, 0}, + [BLOCK_JOB_VERB_COMPLETE] = {0, 0, 0, 0, 1, 0, 0}, }; static void block_job_state_transition(BlockJob *job, BlockJobStatus s1) @@ -383,6 +384,10 @@ static void block_job_completed_single(BlockJob *job) { assert(job->completed); + if (job->ret || block_job_is_cancelled(job)) { + block_job_state_transition(job, BLOCK_JOB_STATUS_ABORTING); + } + if (!job->ret) { if (job->driver->commit) { job->driver->commit(job); diff --git a/qapi/block-core.json b/qapi/block-core.json index 11659496c5..3f7d559fc0 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -996,10 +996,15 @@ # @standby: The job is ready, but paused. This is nearly identical to @paused. # The job may return to @ready or otherwise be canceled. # +# @aborting: The job is in the process of being aborted, and will finish with +# an error. The job will afterwards report that it is @concluded. +# This status may not be visible to the management process. +# # Since: 2.12 ## { 'enum': 'BlockJobStatus', - 'data': ['undefined', 'created', 'running', 'paused', 'ready', 'standby'] } + 'data': ['undefined', 'created', 'running', 'paused', 'ready', 'standby', + 'aborting' ] } ## # @BlockJobInfo: From patchwork Fri Feb 23 23:51:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877351 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7jR3JCvz9sW8 for ; Sat, 24 Feb 2018 11:08:42 +1100 (AEDT) Received: from localhost ([::1]:47962 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNOJ-0002d3-Mp for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 19:08:39 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55313) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8M-0005sR-Pj for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8I-0005tA-Tq for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:10 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33870 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8B-0005ZJ-2o; Fri, 23 Feb 2018 18:51:59 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8F42840744D0; Fri, 23 Feb 2018 23:51:53 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5EB991049475; Fri, 23 Feb 2018 23:51:53 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:30 -0500 Message-Id: <20180223235142.21501-10-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:53 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:53 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 09/21] blockjobs: add CONCLUDED state X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" add a new state "CONCLUDED" that identifies a job that has ceased all operations. The wording was chosen to avoid any phrasing that might imply success, error, or cancellation. The task has simply ceased all operation and can never again perform any work. ("finished", "done", and "completed" might all imply success.) Transitions: Running -> Concluded: normal completion Ready -> Concluded: normal completion Aborting -> Concluded: error and cancellations Verbs: None as of this commit. (a future commit adds 'dismiss') +---------+ |UNDEFINED| +--+------+ | +--v----+ |CREATED| +--+----+ | +--v----+ +------+ +---------+RUNNING<----->PAUSED| | +--+-+--+ +------+ | | | | | +------------------+ | | | | +--v--+ +-------+ | +---------+READY<------->STANDBY| | | +--+--+ +-------+ | | | | +--v-----+ +--v------+ | |ABORTING+--->CONCLUDED<-------------+ +--------+ +---------+ Signed-off-by: John Snow Reviewed-by: Eric Blake --- blockjob.c | 43 ++++++++++++++++++++++++++++--------------- qapi/block-core.json | 5 ++++- 2 files changed, 32 insertions(+), 16 deletions(-) diff --git a/blockjob.c b/blockjob.c index 4c3fcda46c..93b0a36306 100644 --- a/blockjob.c +++ b/blockjob.c @@ -44,23 +44,24 @@ static QemuMutex block_job_mutex; /* BlockJob State Transition Table */ bool BlockJobSTT[BLOCK_JOB_STATUS__MAX][BLOCK_JOB_STATUS__MAX] = { - /* U, C, R, P, Y, S, X */ - /* U: */ [BLOCK_JOB_STATUS_UNDEFINED] = {0, 1, 0, 0, 0, 0, 0}, - /* C: */ [BLOCK_JOB_STATUS_CREATED] = {0, 0, 1, 0, 0, 0, 0}, - /* R: */ [BLOCK_JOB_STATUS_RUNNING] = {0, 0, 0, 1, 1, 0, 1}, - /* P: */ [BLOCK_JOB_STATUS_PAUSED] = {0, 0, 1, 0, 0, 0, 0}, - /* Y: */ [BLOCK_JOB_STATUS_READY] = {0, 0, 0, 0, 0, 1, 1}, - /* S: */ [BLOCK_JOB_STATUS_STANDBY] = {0, 0, 0, 0, 1, 0, 0}, - /* X: */ [BLOCK_JOB_STATUS_ABORTING] = {0, 0, 0, 0, 0, 0, 0}, + /* U, C, R, P, Y, S, X, E */ + /* U: */ [BLOCK_JOB_STATUS_UNDEFINED] = {0, 1, 0, 0, 0, 0, 0, 0}, + /* C: */ [BLOCK_JOB_STATUS_CREATED] = {0, 0, 1, 0, 0, 0, 0, 0}, + /* R: */ [BLOCK_JOB_STATUS_RUNNING] = {0, 0, 0, 1, 1, 0, 1, 1}, + /* P: */ [BLOCK_JOB_STATUS_PAUSED] = {0, 0, 1, 0, 0, 0, 0, 0}, + /* Y: */ [BLOCK_JOB_STATUS_READY] = {0, 0, 0, 0, 0, 1, 1, 1}, + /* S: */ [BLOCK_JOB_STATUS_STANDBY] = {0, 0, 0, 0, 1, 0, 0, 0}, + /* X: */ [BLOCK_JOB_STATUS_ABORTING] = {0, 0, 0, 0, 0, 0, 0, 1}, + /* E: */ [BLOCK_JOB_STATUS_CONCLUDED] = {0, 0, 0, 0, 0, 0, 0, 0}, }; bool BlockJobVerbTable[BLOCK_JOB_VERB__MAX][BLOCK_JOB_STATUS__MAX] = { - /* U, C, R, P, Y, S, X */ - [BLOCK_JOB_VERB_CANCEL] = {0, 1, 1, 1, 1, 1, 0}, - [BLOCK_JOB_VERB_PAUSE] = {0, 1, 1, 1, 1, 1, 0}, - [BLOCK_JOB_VERB_RESUME] = {0, 1, 1, 1, 1, 1, 0}, - [BLOCK_JOB_VERB_SET_SPEED] = {0, 1, 1, 1, 1, 1, 0}, - [BLOCK_JOB_VERB_COMPLETE] = {0, 0, 0, 0, 1, 0, 0}, + /* U, C, R, P, Y, S, X, E */ + [BLOCK_JOB_VERB_CANCEL] = {0, 1, 1, 1, 1, 1, 0, 0}, + [BLOCK_JOB_VERB_PAUSE] = {0, 1, 1, 1, 1, 1, 0, 0}, + [BLOCK_JOB_VERB_RESUME] = {0, 1, 1, 1, 1, 1, 0, 0}, + [BLOCK_JOB_VERB_SET_SPEED] = {0, 1, 1, 1, 1, 1, 0, 0}, + [BLOCK_JOB_VERB_COMPLETE] = {0, 0, 0, 0, 1, 0, 0, 0}, }; static void block_job_state_transition(BlockJob *job, BlockJobStatus s1) @@ -114,6 +115,7 @@ static void __attribute__((__constructor__)) block_job_init(void) static void block_job_event_cancelled(BlockJob *job); static void block_job_event_completed(BlockJob *job, const char *msg); +static void block_job_event_concluded(BlockJob *job); static void block_job_enter_cond(BlockJob *job, bool(*fn)(BlockJob *job)); /* Transactional group of block jobs */ @@ -420,6 +422,7 @@ static void block_job_completed_single(BlockJob *job) QLIST_REMOVE(job, txn_list); block_job_txn_unref(job->txn); + block_job_event_concluded(job); block_job_unref(job); } @@ -620,7 +623,9 @@ void block_job_user_resume(BlockJob *job, Error **errp) void block_job_cancel(BlockJob *job) { - if (block_job_started(job)) { + if (job->status == BLOCK_JOB_STATUS_CONCLUDED) { + return; + } else if (block_job_started(job)) { block_job_cancel_async(job); block_job_enter(job); } else { @@ -727,6 +732,14 @@ static void block_job_event_completed(BlockJob *job, const char *msg) &error_abort); } +static void block_job_event_concluded(BlockJob *job) +{ + if (block_job_is_internal(job) || !job->manual) { + return; + } + block_job_state_transition(job, BLOCK_JOB_STATUS_CONCLUDED); +} + /* * API for block job drivers and the block layer. These functions are * declared in blockjob_int.h. diff --git a/qapi/block-core.json b/qapi/block-core.json index 3f7d559fc0..aeb9b1937b 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1000,11 +1000,14 @@ # an error. The job will afterwards report that it is @concluded. # This status may not be visible to the management process. # +# @concluded: The job has finished all work. If manual was set to true, the job +# will remain in the query list until it is dismissed. +# # Since: 2.12 ## { 'enum': 'BlockJobStatus', 'data': ['undefined', 'created', 'running', 'paused', 'ready', 'standby', - 'aborting' ] } + 'aborting', 'concluded' ] } ## # @BlockJobInfo: From patchwork Fri Feb 23 23:51:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877334 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7Qp45CBz9sW0 for ; Sat, 24 Feb 2018 10:56:02 +1100 (AEDT) Received: from localhost ([::1]:47883 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNC4-0000RZ-KE for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 18:56:00 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55268) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8L-0005q2-2E for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:12 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8H-0005qe-Ts for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:09 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33450 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8A-0005Vi-KC; Fri, 23 Feb 2018 18:51:58 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C7FBB84221; Fri, 23 Feb 2018 23:51:53 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 959D31049475; Fri, 23 Feb 2018 23:51:53 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:31 -0500 Message-Id: <20180223235142.21501-11-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Fri, 23 Feb 2018 23:51:53 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Fri, 23 Feb 2018 23:51:53 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 10/21] blockjobs: add NULL state X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Add a new state that specifically demarcates when we begin to permanently demolish a job after it has performed all work. This makes the transition explicit in the STM table and highlights conditions under which a job may be demolished. Transitions: Created -> Null: Early failure event before the job is started Concluded -> Null: Standard transition. Verbs: None. This should not ever be visible to the monitor. +---------+ |UNDEFINED| +--+------+ | +--v----+ |CREATED+------------------+ +--+----+ | | | +--v----+ +------+ | +---------+RUNNING<----->PAUSED| | | +--+-+--+ +------+ | | | | | | | +------------------+ | | | | | | +--v--+ +-------+ | | +---------+READY<------->STANDBY| | | | +--+--+ +-------+ | | | | | | +--v-----+ +--v------+ | | |ABORTING+--->CONCLUDED<-------------+ | +--------+ +--+------+ | | | +--v-+ | |NULL<---------------------+ +----+ Signed-off-by: John Snow Reviewed-by: Eric Blake --- blockjob.c | 35 +++++++++++++++++------------------ qapi/block-core.json | 5 ++++- 2 files changed, 21 insertions(+), 19 deletions(-) diff --git a/blockjob.c b/blockjob.c index 93b0a36306..7b5c4063cf 100644 --- a/blockjob.c +++ b/blockjob.c @@ -44,24 +44,25 @@ static QemuMutex block_job_mutex; /* BlockJob State Transition Table */ bool BlockJobSTT[BLOCK_JOB_STATUS__MAX][BLOCK_JOB_STATUS__MAX] = { - /* U, C, R, P, Y, S, X, E */ - /* U: */ [BLOCK_JOB_STATUS_UNDEFINED] = {0, 1, 0, 0, 0, 0, 0, 0}, - /* C: */ [BLOCK_JOB_STATUS_CREATED] = {0, 0, 1, 0, 0, 0, 0, 0}, - /* R: */ [BLOCK_JOB_STATUS_RUNNING] = {0, 0, 0, 1, 1, 0, 1, 1}, - /* P: */ [BLOCK_JOB_STATUS_PAUSED] = {0, 0, 1, 0, 0, 0, 0, 0}, - /* Y: */ [BLOCK_JOB_STATUS_READY] = {0, 0, 0, 0, 0, 1, 1, 1}, - /* S: */ [BLOCK_JOB_STATUS_STANDBY] = {0, 0, 0, 0, 1, 0, 0, 0}, - /* X: */ [BLOCK_JOB_STATUS_ABORTING] = {0, 0, 0, 0, 0, 0, 0, 1}, - /* E: */ [BLOCK_JOB_STATUS_CONCLUDED] = {0, 0, 0, 0, 0, 0, 0, 0}, + /* U, C, R, P, Y, S, X, E, N */ + /* U: */ [BLOCK_JOB_STATUS_UNDEFINED] = {0, 1, 0, 0, 0, 0, 0, 0, 0}, + /* C: */ [BLOCK_JOB_STATUS_CREATED] = {0, 0, 1, 0, 0, 0, 0, 0, 1}, + /* R: */ [BLOCK_JOB_STATUS_RUNNING] = {0, 0, 0, 1, 1, 0, 1, 1, 0}, + /* P: */ [BLOCK_JOB_STATUS_PAUSED] = {0, 0, 1, 0, 0, 0, 0, 0, 0}, + /* Y: */ [BLOCK_JOB_STATUS_READY] = {0, 0, 0, 0, 0, 1, 1, 1, 0}, + /* S: */ [BLOCK_JOB_STATUS_STANDBY] = {0, 0, 0, 0, 1, 0, 0, 0, 0}, + /* X: */ [BLOCK_JOB_STATUS_ABORTING] = {0, 0, 0, 0, 0, 0, 0, 1, 0}, + /* E: */ [BLOCK_JOB_STATUS_CONCLUDED] = {0, 0, 0, 0, 0, 0, 0, 0, 1}, + /* N: */ [BLOCK_JOB_STATUS_NULL] = {0, 0, 0, 0, 0, 0, 0, 0, 0}, }; bool BlockJobVerbTable[BLOCK_JOB_VERB__MAX][BLOCK_JOB_STATUS__MAX] = { - /* U, C, R, P, Y, S, X, E */ - [BLOCK_JOB_VERB_CANCEL] = {0, 1, 1, 1, 1, 1, 0, 0}, - [BLOCK_JOB_VERB_PAUSE] = {0, 1, 1, 1, 1, 1, 0, 0}, - [BLOCK_JOB_VERB_RESUME] = {0, 1, 1, 1, 1, 1, 0, 0}, - [BLOCK_JOB_VERB_SET_SPEED] = {0, 1, 1, 1, 1, 1, 0, 0}, - [BLOCK_JOB_VERB_COMPLETE] = {0, 0, 0, 0, 1, 0, 0, 0}, + /* U, C, R, P, Y, S, X, E, N */ + [BLOCK_JOB_VERB_CANCEL] = {0, 1, 1, 1, 1, 1, 0, 0, 0}, + [BLOCK_JOB_VERB_PAUSE] = {0, 1, 1, 1, 1, 1, 0, 0, 0}, + [BLOCK_JOB_VERB_RESUME] = {0, 1, 1, 1, 1, 1, 0, 0, 0}, + [BLOCK_JOB_VERB_SET_SPEED] = {0, 1, 1, 1, 1, 1, 0, 0, 0}, + [BLOCK_JOB_VERB_COMPLETE] = {0, 0, 0, 0, 1, 0, 0, 0, 0}, }; static void block_job_state_transition(BlockJob *job, BlockJobStatus s1) @@ -423,6 +424,7 @@ static void block_job_completed_single(BlockJob *job) QLIST_REMOVE(job, txn_list); block_job_txn_unref(job->txn); block_job_event_concluded(job); + block_job_state_transition(job, BLOCK_JOB_STATUS_NULL); block_job_unref(job); } @@ -734,9 +736,6 @@ static void block_job_event_completed(BlockJob *job, const char *msg) static void block_job_event_concluded(BlockJob *job) { - if (block_job_is_internal(job) || !job->manual) { - return; - } block_job_state_transition(job, BLOCK_JOB_STATUS_CONCLUDED); } diff --git a/qapi/block-core.json b/qapi/block-core.json index aeb9b1937b..578c0c91ca 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1003,11 +1003,14 @@ # @concluded: The job has finished all work. If manual was set to true, the job # will remain in the query list until it is dismissed. # +# @null: The job is in the process of being dismantled. This state should not +# ever be visible externally. +# # Since: 2.12 ## { 'enum': 'BlockJobStatus', 'data': ['undefined', 'created', 'running', 'paused', 'ready', 'standby', - 'aborting', 'concluded' ] } + 'aborting', 'concluded', 'null' ] } ## # @BlockJobInfo: From patchwork Fri Feb 23 23:51:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877333 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7Ql4MrRz9sW0 for ; Sat, 24 Feb 2018 10:55:59 +1100 (AEDT) Received: from localhost ([::1]:47881 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNC1-0000NB-MM for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 18:55:57 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55457) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8R-0005yp-Bj for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:18 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8P-000668-Ia for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:15 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:44372 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8F-0005l0-Is; Fri, 23 Feb 2018 18:52:03 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0C37CEB6F0; Fri, 23 Feb 2018 23:51:54 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id CDBAC104947D; Fri, 23 Feb 2018 23:51:53 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:32 -0500 Message-Id: <20180223235142.21501-12-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Fri, 23 Feb 2018 23:51:54 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Fri, 23 Feb 2018 23:51:54 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 11/21] blockjobs: add block_job_dismiss X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" For jobs that have reached their CONCLUDED state, prior to having their last reference put down (meaning jobs that have completed successfully, unsuccessfully, or have been canceled), allow the user to dismiss the job's lingering status report via block-job-dismiss. This gives management APIs the chance to conclusively determine if a job failed or succeeded, even if the event broadcast was missed. Note that jobs do not yet linger in any such state, they are freed immediately upon reaching this previously-unnamed state. such a state is added immediately in the next commit. Verbs: Dismiss: operates on CONCLUDED jobs only. Signed-off-by: John Snow Reviewed-by: Eric Blake --- block/trace-events | 1 + blockdev.c | 14 ++++++++++++++ blockjob.c | 34 ++++++++++++++++++++++++++++++++-- include/block/blockjob.h | 9 +++++++++ qapi/block-core.json | 24 +++++++++++++++++++++++- 5 files changed, 79 insertions(+), 3 deletions(-) diff --git a/block/trace-events b/block/trace-events index 3fe89f7ea6..266afd9e99 100644 --- a/block/trace-events +++ b/block/trace-events @@ -50,6 +50,7 @@ qmp_block_job_cancel(void *job) "job %p" qmp_block_job_pause(void *job) "job %p" qmp_block_job_resume(void *job) "job %p" qmp_block_job_complete(void *job) "job %p" +qmp_block_job_dismiss(void *job) "job %p" qmp_block_stream(void *bs, void *job) "bs %p job %p" # block/file-win32.c diff --git a/blockdev.c b/blockdev.c index cba935a0a6..3180130782 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3852,6 +3852,20 @@ void qmp_block_job_complete(const char *device, Error **errp) aio_context_release(aio_context); } +void qmp_block_job_dismiss(const char *id, Error **errp) +{ + AioContext *aio_context; + BlockJob *job = find_block_job(id, &aio_context, errp); + + if (!job) { + return; + } + + trace_qmp_block_job_dismiss(job); + block_job_dismiss(&job, errp); + aio_context_release(aio_context); +} + void qmp_change_backing_file(const char *device, const char *image_node_name, const char *backing_file, diff --git a/blockjob.c b/blockjob.c index 7b5c4063cf..4d29391673 100644 --- a/blockjob.c +++ b/blockjob.c @@ -63,6 +63,7 @@ bool BlockJobVerbTable[BLOCK_JOB_VERB__MAX][BLOCK_JOB_STATUS__MAX] = { [BLOCK_JOB_VERB_RESUME] = {0, 1, 1, 1, 1, 1, 0, 0, 0}, [BLOCK_JOB_VERB_SET_SPEED] = {0, 1, 1, 1, 1, 1, 0, 0, 0}, [BLOCK_JOB_VERB_COMPLETE] = {0, 0, 0, 0, 1, 0, 0, 0, 0}, + [BLOCK_JOB_VERB_DISMISS] = {0, 0, 0, 0, 0, 0, 0, 1, 0}, }; static void block_job_state_transition(BlockJob *job, BlockJobStatus s1) @@ -424,7 +425,6 @@ static void block_job_completed_single(BlockJob *job) QLIST_REMOVE(job, txn_list); block_job_txn_unref(job->txn); block_job_event_concluded(job); - block_job_state_transition(job, BLOCK_JOB_STATUS_NULL); block_job_unref(job); } @@ -441,6 +441,13 @@ static void block_job_cancel_async(BlockJob *job) job->cancelled = true; } +static void block_job_do_dismiss(BlockJob *job) +{ + assert(job); + block_job_state_transition(job, BLOCK_JOB_STATUS_NULL); + block_job_unref(job); +} + static int block_job_finish_sync(BlockJob *job, void (*finish)(BlockJob *, Error **errp), Error **errp) @@ -590,6 +597,19 @@ void block_job_complete(BlockJob *job, Error **errp) job->driver->complete(job, errp); } +void block_job_dismiss(BlockJob **jobptr, Error **errp) +{ + BlockJob *job = *jobptr; + /* similarly to _complete, this is QMP-interface only. */ + assert(job->id); + if (block_job_apply_verb(job, BLOCK_JOB_VERB_DISMISS, errp)) { + return; + } + + block_job_do_dismiss(job); + *jobptr = NULL; +} + void block_job_user_pause(BlockJob *job, Error **errp) { if (block_job_apply_verb(job, BLOCK_JOB_VERB_PAUSE, errp)) { @@ -626,7 +646,7 @@ void block_job_user_resume(BlockJob *job, Error **errp) void block_job_cancel(BlockJob *job) { if (job->status == BLOCK_JOB_STATUS_CONCLUDED) { - return; + block_job_do_dismiss(job); } else if (block_job_started(job)) { block_job_cancel_async(job); block_job_enter(job); @@ -737,6 +757,10 @@ static void block_job_event_completed(BlockJob *job, const char *msg) static void block_job_event_concluded(BlockJob *job) { block_job_state_transition(job, BLOCK_JOB_STATUS_CONCLUDED); + /* for pre-2.12 style jobs, automatically destroy */ + if (!job->manual) { + block_job_do_dismiss(job); + } } /* @@ -841,6 +865,9 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, block_job_txn_add_job(txn, job); } + /* For the expanded job control STM, grab an extra + * reference for finalize() to put down */ + block_job_ref(job); return job; } @@ -859,6 +886,9 @@ void block_job_pause_all(void) void block_job_early_fail(BlockJob *job) { + /* One for creation, one for finalize() */ + assert(job->status == BLOCK_JOB_STATUS_CREATED); + block_job_unref(job); block_job_unref(job); } diff --git a/include/block/blockjob.h b/include/block/blockjob.h index 403a168073..9bb0be1b13 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -245,6 +245,15 @@ void block_job_cancel(BlockJob *job); */ void block_job_complete(BlockJob *job, Error **errp); +/** + * block_job_dismiss: + * @job: The job to be dismissed. + * @errp: Error object. + * + * Remove a concluded job from the query list. + */ +void block_job_dismiss(BlockJob **job, Error **errp); + /** * block_job_query: * @job: The job to get information about. diff --git a/qapi/block-core.json b/qapi/block-core.json index 578c0c91ca..b42796a6a0 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -970,10 +970,12 @@ # # @complete: see @block-job-complete # +# @dismiss: see @block-job-dismiss +# # Since: 2.12 ## { 'enum': 'BlockJobVerb', - 'data': ['cancel', 'pause', 'resume', 'set-speed', 'complete' ] } + 'data': ['cancel', 'pause', 'resume', 'set-speed', 'complete', 'dismiss' ] } ## # @BlockJobStatus: @@ -2243,6 +2245,26 @@ ## { 'command': 'block-job-complete', 'data': { 'device': 'str' } } +## +# @block-job-dismiss: +# +# For jobs that have already concluded, remove them from the block-job-query +# list. This command only needs to be run for jobs which were started with +# QEMU 2.12+ job lifetime management semantics. +# +# This command will refuse to operate on any job that has not yet reached +# its terminal state, BLOCK_JOB_STATUS_CONCLUDED. For jobs that make use of +# BLOCK_JOB_READY event, block-job-cancel or block-job-complete will still need +# to be used as appropriate. +# +# @id: The job identifier. +# +# Returns: Nothing on success +# +# Since: 2.12 +## +{ 'command': 'block-job-dismiss', 'data': { 'id': 'str' } } + ## # @BlockdevDiscardOptions: # From patchwork Fri Feb 23 23:51:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877332 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7N22N23z9sW0 for ; Sat, 24 Feb 2018 10:53:38 +1100 (AEDT) Received: from localhost ([::1]:47856 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN9k-0006Yx-Er for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 18:53:36 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55275) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8L-0005qO-A6 for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:12 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8I-0005rj-8w for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:09 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33868 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8B-0005ZL-2K; Fri, 23 Feb 2018 18:51:59 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 430E340744D1; Fri, 23 Feb 2018 23:51:54 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 119CE104947B; Fri, 23 Feb 2018 23:51:54 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:33 -0500 Message-Id: <20180223235142.21501-13-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:54 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:54 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 12/21] blockjobs: ensure abort is called for cancelled jobs X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Presently, even if a job is canceled post-completion as a result of a failing peer in a transaction, it will still call .commit because nothing has updated or changed its return code. The reason why this does not cause problems currently is because backup's implementation of .commit checks for cancellation itself. I'd like to simplify this contract: (1) Abort is called if the job/transaction fails (2) Commit is called if the job/transaction succeeds To this end: A job's return code, if 0, will be forcibly set as -ECANCELED if that job has already concluded. Remove the now redundant check in the backup job implementation. We need to check for cancellation in both block_job_completed AND block_job_completed_single, because jobs may be cancelled between those two calls; for instance in transactions. The check in block_job_completed could be removed, but there's no point in starting to attempt to succeed a transaction that we know in advance will fail. This does NOT affect mirror jobs that are "canceled" during their synchronous phase. The mirror job itself forcibly sets the canceled property to false prior to ceding control, so such cases will invoke the "commit" callback. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf --- block/backup.c | 2 +- block/trace-events | 1 + blockjob.c | 19 +++++++++++++++---- 3 files changed, 17 insertions(+), 5 deletions(-) diff --git a/block/backup.c b/block/backup.c index 7e254dabff..453cd62c24 100644 --- a/block/backup.c +++ b/block/backup.c @@ -206,7 +206,7 @@ static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret) BdrvDirtyBitmap *bm; BlockDriverState *bs = blk_bs(job->common.blk); - if (ret < 0 || block_job_is_cancelled(&job->common)) { + if (ret < 0) { /* Merge the successor back into the parent, delete nothing. */ bm = bdrv_reclaim_dirty_bitmap(bs, job->sync_bitmap, NULL); assert(bm); diff --git a/block/trace-events b/block/trace-events index 266afd9e99..5e531e0310 100644 --- a/block/trace-events +++ b/block/trace-events @@ -5,6 +5,7 @@ bdrv_open_common(void *bs, const char *filename, int flags, const char *format_n bdrv_lock_medium(void *bs, bool locked) "bs %p locked %d" # blockjob.c +block_job_completed(void *job, int ret, int jret) "job %p ret %d corrected ret %d" block_job_state_transition(void *job, int ret, const char *legal, const char *s0, const char *s1) "job %p (ret: %d) attempting %s transition (%s-->%s)" block_job_apply_verb(void *job, const char *state, const char *verb, const char *legal) "job %p in state %s; applying verb %s (%s)" diff --git a/blockjob.c b/blockjob.c index 4d29391673..ef17dea004 100644 --- a/blockjob.c +++ b/blockjob.c @@ -384,13 +384,22 @@ void block_job_start(BlockJob *job) bdrv_coroutine_enter(blk_bs(job->blk), job->co); } +static void block_job_update_rc(BlockJob *job) +{ + if (!job->ret && block_job_is_cancelled(job)) { + job->ret = -ECANCELED; + } + if (job->ret) { + block_job_state_transition(job, BLOCK_JOB_STATUS_ABORTING); + } +} + static void block_job_completed_single(BlockJob *job) { assert(job->completed); - if (job->ret || block_job_is_cancelled(job)) { - block_job_state_transition(job, BLOCK_JOB_STATUS_ABORTING); - } + /* Ensure abort is called for late-transactional failures */ + block_job_update_rc(job); if (!job->ret) { if (job->driver->commit) { @@ -898,7 +907,9 @@ void block_job_completed(BlockJob *job, int ret) assert(blk_bs(job->blk)->job == job); job->completed = true; job->ret = ret; - if (ret < 0 || block_job_is_cancelled(job)) { + block_job_update_rc(job); + trace_block_job_completed(job, ret, job->ret); + if (job->ret) { block_job_completed_txn_abort(job); } else { block_job_completed_txn_success(job); From patchwork Fri Feb 23 23:51:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877336 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7RF0BdFz9sW0 for ; Sat, 24 Feb 2018 10:56:25 +1100 (AEDT) Received: from localhost ([::1]:47887 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNCR-0000jg-36 for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 18:56:23 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55172) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8H-0005mD-W1 for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:08 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8F-0005m9-Rr for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:05 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33872 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8B-0005ZW-3r; Fri, 23 Feb 2018 18:51:59 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 796D04078039; Fri, 23 Feb 2018 23:51:54 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 48E08104947B; Fri, 23 Feb 2018 23:51:54 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:34 -0500 Message-Id: <20180223235142.21501-14-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:54 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:54 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 13/21] blockjobs: add commit, abort, clean helpers X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" The completed_single function is getting a little mucked up with checking to see which callbacks exist, so let's factor them out. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf --- blockjob.c | 35 ++++++++++++++++++++++++++--------- 1 file changed, 26 insertions(+), 9 deletions(-) diff --git a/blockjob.c b/blockjob.c index ef17dea004..431ce9c220 100644 --- a/blockjob.c +++ b/blockjob.c @@ -394,6 +394,29 @@ static void block_job_update_rc(BlockJob *job) } } +static void block_job_commit(BlockJob *job) +{ + assert(!job->ret); + if (job->driver->commit) { + job->driver->commit(job); + } +} + +static void block_job_abort(BlockJob *job) +{ + assert(job->ret); + if (job->driver->abort) { + job->driver->abort(job); + } +} + +static void block_job_clean(BlockJob *job) +{ + if (job->driver->clean) { + job->driver->clean(job); + } +} + static void block_job_completed_single(BlockJob *job) { assert(job->completed); @@ -402,17 +425,11 @@ static void block_job_completed_single(BlockJob *job) block_job_update_rc(job); if (!job->ret) { - if (job->driver->commit) { - job->driver->commit(job); - } + block_job_commit(job); } else { - if (job->driver->abort) { - job->driver->abort(job); - } - } - if (job->driver->clean) { - job->driver->clean(job); + block_job_abort(job); } + block_job_clean(job); if (job->cb) { job->cb(job->opaque, job->ret); From patchwork Fri Feb 23 23:51:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877345 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7by0jF6z9sW0 for ; Sat, 24 Feb 2018 11:03:58 +1100 (AEDT) Received: from localhost ([::1]:47935 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNJk-0007Ji-5x for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 19:03:56 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55323) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8N-0005sm-2B for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8K-0005wW-Qx for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:10 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33454 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8F-0005ky-Je; Fri, 23 Feb 2018 18:52:03 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B50F7A1BD3; Fri, 23 Feb 2018 23:51:54 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 80061104947B; Fri, 23 Feb 2018 23:51:54 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:35 -0500 Message-Id: <20180223235142.21501-15-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Fri, 23 Feb 2018 23:51:54 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Fri, 23 Feb 2018 23:51:54 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 14/21] blockjobs: add block_job_txn_apply function X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Simply apply a function transaction-wide. A few more uses of this in forthcoming patches. Signed-off-by: John Snow Reviewed-by: Eric Blake --- blockjob.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/blockjob.c b/blockjob.c index 431ce9c220..8f02c03880 100644 --- a/blockjob.c +++ b/blockjob.c @@ -467,6 +467,19 @@ static void block_job_cancel_async(BlockJob *job) job->cancelled = true; } +static void block_job_txn_apply(BlockJobTxn *txn, void fn(BlockJob *)) +{ + AioContext *ctx; + BlockJob *job, *next; + + QLIST_FOREACH_SAFE(job, &txn->jobs, txn_list, next) { + ctx = blk_get_aio_context(job->blk); + aio_context_acquire(ctx); + fn(job); + aio_context_release(ctx); + } +} + static void block_job_do_dismiss(BlockJob *job) { assert(job); @@ -552,9 +565,8 @@ static void block_job_completed_txn_abort(BlockJob *job) static void block_job_completed_txn_success(BlockJob *job) { - AioContext *ctx; BlockJobTxn *txn = job->txn; - BlockJob *other_job, *next; + BlockJob *other_job; /* * Successful completion, see if there are other running jobs in this * txn. @@ -565,13 +577,7 @@ static void block_job_completed_txn_success(BlockJob *job) } } /* We are the last completed job, commit the transaction. */ - QLIST_FOREACH_SAFE(other_job, &txn->jobs, txn_list, next) { - ctx = blk_get_aio_context(other_job->blk); - aio_context_acquire(ctx); - assert(other_job->ret == 0); - block_job_completed_single(other_job); - aio_context_release(ctx); - } + block_job_txn_apply(txn, block_job_completed_single); } /* Assumes the block_job_mutex is held */ From patchwork Fri Feb 23 23:51:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877346 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7fJ1DRRz9sW7 for ; Sat, 24 Feb 2018 11:06:00 +1100 (AEDT) Received: from localhost ([::1]:47947 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNLi-0000W9-8M for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 19:05:58 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55441) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8Q-0005xq-P2 for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:17 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8N-000625-CT for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:14 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:44374 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8F-0005kz-Ks; Fri, 23 Feb 2018 18:52:03 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E8EE6EB710; Fri, 23 Feb 2018 23:51:54 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id B8491104947E; Fri, 23 Feb 2018 23:51:54 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:36 -0500 Message-Id: <20180223235142.21501-16-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Fri, 23 Feb 2018 23:51:54 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Fri, 23 Feb 2018 23:51:54 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 15/21] blockjobs: add prepare callback X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Some jobs upon finalization may need to perform some work that can still fail. If these jobs are part of a transaction, it's important that these callbacks fail the entire transaction. We allow for a new callback in addition to commit/abort/clean that allows us the opportunity to have fairly late-breaking failures in the transactional process. The expected flow is: - All jobs in a transaction converge to the WAITING state (added in a forthcoming commit) - All jobs prepare to call either commit/abort - If any job fails, is canceled, or fails preparation, all jobs call their .abort callback. - All jobs enter the PENDING state, awaiting manual intervention (also added in a forthcoming commit) - block-job-finalize is issued by the user/management layer - All jobs call their commit callbacks. Signed-off-by: John Snow --- blockjob.c | 34 +++++++++++++++++++++++++++++++--- include/block/blockjob_int.h | 10 ++++++++++ 2 files changed, 41 insertions(+), 3 deletions(-) diff --git a/blockjob.c b/blockjob.c index 8f02c03880..1c010ec100 100644 --- a/blockjob.c +++ b/blockjob.c @@ -394,6 +394,18 @@ static void block_job_update_rc(BlockJob *job) } } +static int block_job_prepare(BlockJob *job) +{ + if (job->ret) { + goto out; + } + if (job->driver->prepare) { + job->ret = job->driver->prepare(job); + } + out: + return job->ret; +} + static void block_job_commit(BlockJob *job) { assert(!job->ret); @@ -417,7 +429,7 @@ static void block_job_clean(BlockJob *job) } } -static void block_job_completed_single(BlockJob *job) +static int block_job_completed_single(BlockJob *job) { assert(job->completed); @@ -452,6 +464,7 @@ static void block_job_completed_single(BlockJob *job) block_job_txn_unref(job->txn); block_job_event_concluded(job); block_job_unref(job); + return 0; } static void block_job_cancel_async(BlockJob *job) @@ -467,17 +480,22 @@ static void block_job_cancel_async(BlockJob *job) job->cancelled = true; } -static void block_job_txn_apply(BlockJobTxn *txn, void fn(BlockJob *)) +static int block_job_txn_apply(BlockJobTxn *txn, int fn(BlockJob *)) { AioContext *ctx; BlockJob *job, *next; + int rc; QLIST_FOREACH_SAFE(job, &txn->jobs, txn_list, next) { ctx = blk_get_aio_context(job->blk); aio_context_acquire(ctx); - fn(job); + rc = fn(job); aio_context_release(ctx); + if (rc) { + break; + } } + return rc; } static void block_job_do_dismiss(BlockJob *job) @@ -567,6 +585,8 @@ static void block_job_completed_txn_success(BlockJob *job) { BlockJobTxn *txn = job->txn; BlockJob *other_job; + int rc = 0; + /* * Successful completion, see if there are other running jobs in this * txn. @@ -576,6 +596,14 @@ static void block_job_completed_txn_success(BlockJob *job) return; } } + + /* Jobs may require some prep-work to complete without failure */ + rc = block_job_txn_apply(txn, block_job_prepare); + if (rc) { + block_job_completed_txn_abort(job); + return; + } + /* We are the last completed job, commit the transaction. */ block_job_txn_apply(txn, block_job_completed_single); } diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h index 259d49b32a..642adce68b 100644 --- a/include/block/blockjob_int.h +++ b/include/block/blockjob_int.h @@ -53,6 +53,16 @@ struct BlockJobDriver { */ void (*complete)(BlockJob *job, Error **errp); + /** + * If the callback is not NULL, prepare will be invoked when all the jobs + * belonging to the same transaction complete; or upon this job's completion + * if it is not in a transaction. + * + * This callback will not be invoked if the job has already failed. + * If it fails, abort and then clean will be called. + */ + int (*prepare)(BlockJob *job); + /** * If the callback is not NULL, it will be invoked when all the jobs * belonging to the same transaction complete; or upon this job's From patchwork Fri Feb 23 23:51:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877347 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7fh0XNnz9sW7 for ; Sat, 24 Feb 2018 11:06:20 +1100 (AEDT) Received: from localhost ([::1]:47953 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNM2-0000lt-3x for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 19:06:18 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55281) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8L-0005r1-ML for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8I-0005rd-7f for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:09 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33874 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8B-0005ZZ-4z; Fri, 23 Feb 2018 18:51:59 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2C514407806F; Fri, 23 Feb 2018 23:51:55 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id F01F0104947E; Fri, 23 Feb 2018 23:51:54 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:37 -0500 Message-Id: <20180223235142.21501-17-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:55 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:55 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 16/21] blockjobs: add waiting status X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" For jobs that are stuck waiting on others in a transaction, it would be nice to know that they are no longer "running" in that sense, but instead are waiting on other jobs in the transaction. Jobs that are "waiting" in this sense cannot be meaningfully altered any longer as they have left their running loop. The only meaningful user verb for jobs in this state is "cancel," which will cancel the whole transaction, too. Transitions: Running -> Waiting: Normal transition. Ready -> Waiting: Normal transition. Waiting -> Aborting: Transactional cancellation. Waiting -> Concluded: Normal transition. Removed Transitions: Running -> Concluded: Jobs must go to WAITING first. Ready -> Concluded: Jobs must go to WAITING fisrt. Verbs: Cancel: Can be applied to WAITING jobs. +---------+ |UNDEFINED| +--+------+ | +--v----+ |CREATED+-----------------+ +--+----+ | | | +--v----+ +------+ | +---------+RUNNING<----->PAUSED| | | +--+-+--+ +------+ | | | | | | | +------------------+ | | | | | | +--v--+ +-------+ | | +---------+READY<------->STANDBY| | | | +--+--+ +-------+ | | | | | | | +--v----+ | | +---------+WAITING<---------------+ | | +--+----+ | | | | +--+-----+ +--v------+ | |ABORTING+--->CONCLUDED| | +--------+ +--+------+ | | | +--v-+ | |NULL<--------------------+ +----+ Signed-off-by: John Snow --- blockjob.c | 37 ++++++++++++++++++++----------------- qapi/block-core.json | 29 ++++++++++++++++++++++++++++- 2 files changed, 48 insertions(+), 18 deletions(-) diff --git a/blockjob.c b/blockjob.c index 1c010ec100..4aed86fc6b 100644 --- a/blockjob.c +++ b/blockjob.c @@ -44,26 +44,27 @@ static QemuMutex block_job_mutex; /* BlockJob State Transition Table */ bool BlockJobSTT[BLOCK_JOB_STATUS__MAX][BLOCK_JOB_STATUS__MAX] = { - /* U, C, R, P, Y, S, X, E, N */ - /* U: */ [BLOCK_JOB_STATUS_UNDEFINED] = {0, 1, 0, 0, 0, 0, 0, 0, 0}, - /* C: */ [BLOCK_JOB_STATUS_CREATED] = {0, 0, 1, 0, 0, 0, 0, 0, 1}, - /* R: */ [BLOCK_JOB_STATUS_RUNNING] = {0, 0, 0, 1, 1, 0, 1, 1, 0}, - /* P: */ [BLOCK_JOB_STATUS_PAUSED] = {0, 0, 1, 0, 0, 0, 0, 0, 0}, - /* Y: */ [BLOCK_JOB_STATUS_READY] = {0, 0, 0, 0, 0, 1, 1, 1, 0}, - /* S: */ [BLOCK_JOB_STATUS_STANDBY] = {0, 0, 0, 0, 1, 0, 0, 0, 0}, - /* X: */ [BLOCK_JOB_STATUS_ABORTING] = {0, 0, 0, 0, 0, 0, 0, 1, 0}, - /* E: */ [BLOCK_JOB_STATUS_CONCLUDED] = {0, 0, 0, 0, 0, 0, 0, 0, 1}, - /* N: */ [BLOCK_JOB_STATUS_NULL] = {0, 0, 0, 0, 0, 0, 0, 0, 0}, + /* U, C, R, P, Y, S, W, X, E, N */ + /* U: */ [BLOCK_JOB_STATUS_UNDEFINED] = {0, 1, 0, 0, 0, 0, 0, 0, 0, 0}, + /* C: */ [BLOCK_JOB_STATUS_CREATED] = {0, 0, 1, 0, 0, 0, 0, 0, 0, 1}, + /* R: */ [BLOCK_JOB_STATUS_RUNNING] = {0, 0, 0, 1, 1, 0, 1, 1, 0, 0}, + /* P: */ [BLOCK_JOB_STATUS_PAUSED] = {0, 0, 1, 0, 0, 0, 0, 0, 0, 0}, + /* Y: */ [BLOCK_JOB_STATUS_READY] = {0, 0, 0, 0, 0, 1, 1, 1, 0, 0}, + /* S: */ [BLOCK_JOB_STATUS_STANDBY] = {0, 0, 0, 0, 1, 0, 0, 0, 0, 0}, + /* W: */ [BLOCK_JOB_STATUS_WAITING] = {0, 0, 0, 0, 0, 0, 0, 1, 1, 0}, + /* X: */ [BLOCK_JOB_STATUS_ABORTING] = {0, 0, 0, 0, 0, 0, 0, 0, 1, 0}, + /* E: */ [BLOCK_JOB_STATUS_CONCLUDED] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 1}, + /* N: */ [BLOCK_JOB_STATUS_NULL] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, }; bool BlockJobVerbTable[BLOCK_JOB_VERB__MAX][BLOCK_JOB_STATUS__MAX] = { - /* U, C, R, P, Y, S, X, E, N */ - [BLOCK_JOB_VERB_CANCEL] = {0, 1, 1, 1, 1, 1, 0, 0, 0}, - [BLOCK_JOB_VERB_PAUSE] = {0, 1, 1, 1, 1, 1, 0, 0, 0}, - [BLOCK_JOB_VERB_RESUME] = {0, 1, 1, 1, 1, 1, 0, 0, 0}, - [BLOCK_JOB_VERB_SET_SPEED] = {0, 1, 1, 1, 1, 1, 0, 0, 0}, - [BLOCK_JOB_VERB_COMPLETE] = {0, 0, 0, 0, 1, 0, 0, 0, 0}, - [BLOCK_JOB_VERB_DISMISS] = {0, 0, 0, 0, 0, 0, 0, 1, 0}, + /* U, C, R, P, Y, S, W, X, E, N */ + [BLOCK_JOB_VERB_CANCEL] = {0, 1, 1, 1, 1, 1, 1, 0, 0, 0}, + [BLOCK_JOB_VERB_PAUSE] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0}, + [BLOCK_JOB_VERB_RESUME] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0}, + [BLOCK_JOB_VERB_SET_SPEED] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0}, + [BLOCK_JOB_VERB_COMPLETE] = {0, 0, 0, 0, 1, 0, 0, 0, 0, 0}, + [BLOCK_JOB_VERB_DISMISS] = {0, 0, 0, 0, 0, 0, 0, 0, 1, 0}, }; static void block_job_state_transition(BlockJob *job, BlockJobStatus s1) @@ -587,6 +588,8 @@ static void block_job_completed_txn_success(BlockJob *job) BlockJob *other_job; int rc = 0; + block_job_state_transition(job, BLOCK_JOB_STATUS_WAITING); + /* * Successful completion, see if there are other running jobs in this * txn. diff --git a/qapi/block-core.json b/qapi/block-core.json index b42796a6a0..3005aac7e2 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -998,6 +998,10 @@ # @standby: The job is ready, but paused. This is nearly identical to @paused. # The job may return to @ready or otherwise be canceled. # +# @waiting: The job is waiting for other jobs in the transaction to converge +# to the waiting state. This status will likely not be visible for +# the last job in a transaction. +# # @aborting: The job is in the process of being aborted, and will finish with # an error. The job will afterwards report that it is @concluded. # This status may not be visible to the management process. @@ -1012,7 +1016,7 @@ ## { 'enum': 'BlockJobStatus', 'data': ['undefined', 'created', 'running', 'paused', 'ready', 'standby', - 'aborting', 'concluded', 'null' ] } + 'waiting', 'aborting', 'concluded', 'null' ] } ## # @BlockJobInfo: @@ -3934,6 +3938,29 @@ 'offset': 'int', 'speed' : 'int' } } +## +# @BLOCK_JOB_WAITING: +# +# Emitted when a block job that is part of a transction has stopped work and is +# waiting for other jobs in the transaction to reach the same state. +# +# @type: job type +# +# @id: The job identifier. +# +# Since: 2.12 +# +# Example: +# +# <- { "event": "BLOCK_JOB_WAITING", +# "data": { "id": "drive0", "type": "mirror" }, +# "timestamp": { "seconds": 1265044230, "microseconds": 450486 } } +# +## +{ 'event': 'BLOCK_JOB_WAITING', + 'data': { 'type' : 'BlockJobType', + 'id' : 'str' } } + ## # @PreallocMode: # From patchwork Fri Feb 23 23:51:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877357 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7np3Ykzz9sW8 for ; Sat, 24 Feb 2018 11:12:30 +1100 (AEDT) Received: from localhost ([::1]:47994 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNS0-0005na-Ji for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 19:12:28 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55480) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8S-0005zg-9P for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8P-000664-JQ for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:16 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33453 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8F-0005ku-Kz; Fri, 23 Feb 2018 18:52:03 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 685017D85D; Fri, 23 Feb 2018 23:51:55 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 328831049482; Fri, 23 Feb 2018 23:51:55 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:38 -0500 Message-Id: <20180223235142.21501-18-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Fri, 23 Feb 2018 23:51:55 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Fri, 23 Feb 2018 23:51:55 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 17/21] blockjobs: add PENDING status and event X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" For jobs utilizing the new manual workflow, we intend to prohibit them from modifying the block graph until the management layer provides an explicit ACK via block-job-finalize to move the process forward. To distinguish this runstate from "ready" or "waiting," we add a new "pending" event. For now, the transition from PENDING to CONCLUDED/ABORTING is automatic, but a future commit will add the explicit block-job-finalize step. Transitions: Waiting -> Pending: Normal transition. Pending -> Concluded: Normal transition. Pending -> Aborting: Late transactional failures and cancellations. Removed Transitions: Waiting -> Concluded: Jobs must go to PENDING first. Verbs: Cancel: Can be applied to a pending job. +---------+ |UNDEFINED| +--+------+ | +--v----+ |CREATED+-----------------+ +--+----+ | | | +--+----+ +------+ | +---------+RUNNING<----->PAUSED| | | +--+-+--+ +------+ | | | | | | | +------------------+ | | | | | | +--v--+ +-------+ | | +---------+READY<------->STANDBY| | | | +--+--+ +-------+ | | | | | | | +--v----+ | | +---------+WAITING+---------------+ | | +--+----+ | | | | | +--v----+ | +---------+PENDING| | | +--+----+ | | | | +--v-----+ +--v------+ | |ABORTING+--->CONCLUDED| | +--------+ +--+------+ | | | +--v-+ | |NULL+--------------------+ +----+ Signed-off-by: John Snow Reviewed-by: Kevin Wolf --- blockjob.c | 66 +++++++++++++++++++++++++++++++++------------------- qapi/block-core.json | 31 +++++++++++++++++++++++- 2 files changed, 72 insertions(+), 25 deletions(-) diff --git a/blockjob.c b/blockjob.c index 4aed86fc6b..23b4b99fd4 100644 --- a/blockjob.c +++ b/blockjob.c @@ -44,27 +44,28 @@ static QemuMutex block_job_mutex; /* BlockJob State Transition Table */ bool BlockJobSTT[BLOCK_JOB_STATUS__MAX][BLOCK_JOB_STATUS__MAX] = { - /* U, C, R, P, Y, S, W, X, E, N */ - /* U: */ [BLOCK_JOB_STATUS_UNDEFINED] = {0, 1, 0, 0, 0, 0, 0, 0, 0, 0}, - /* C: */ [BLOCK_JOB_STATUS_CREATED] = {0, 0, 1, 0, 0, 0, 0, 0, 0, 1}, - /* R: */ [BLOCK_JOB_STATUS_RUNNING] = {0, 0, 0, 1, 1, 0, 1, 1, 0, 0}, - /* P: */ [BLOCK_JOB_STATUS_PAUSED] = {0, 0, 1, 0, 0, 0, 0, 0, 0, 0}, - /* Y: */ [BLOCK_JOB_STATUS_READY] = {0, 0, 0, 0, 0, 1, 1, 1, 0, 0}, - /* S: */ [BLOCK_JOB_STATUS_STANDBY] = {0, 0, 0, 0, 1, 0, 0, 0, 0, 0}, - /* W: */ [BLOCK_JOB_STATUS_WAITING] = {0, 0, 0, 0, 0, 0, 0, 1, 1, 0}, - /* X: */ [BLOCK_JOB_STATUS_ABORTING] = {0, 0, 0, 0, 0, 0, 0, 0, 1, 0}, - /* E: */ [BLOCK_JOB_STATUS_CONCLUDED] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 1}, - /* N: */ [BLOCK_JOB_STATUS_NULL] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, + /* U, C, R, P, Y, S, W, D, X, E, N */ + /* U: */ [BLOCK_JOB_STATUS_UNDEFINED] = {0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0}, + /* C: */ [BLOCK_JOB_STATUS_CREATED] = {0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1}, + /* R: */ [BLOCK_JOB_STATUS_RUNNING] = {0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0}, + /* P: */ [BLOCK_JOB_STATUS_PAUSED] = {0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0}, + /* Y: */ [BLOCK_JOB_STATUS_READY] = {0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0}, + /* S: */ [BLOCK_JOB_STATUS_STANDBY] = {0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0}, + /* W: */ [BLOCK_JOB_STATUS_WAITING] = {0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0}, + /* D: */ [BLOCK_JOB_STATUS_PENDING] = {0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0}, + /* X: */ [BLOCK_JOB_STATUS_ABORTING] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0}, + /* E: */ [BLOCK_JOB_STATUS_CONCLUDED] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1}, + /* N: */ [BLOCK_JOB_STATUS_NULL] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, }; bool BlockJobVerbTable[BLOCK_JOB_VERB__MAX][BLOCK_JOB_STATUS__MAX] = { - /* U, C, R, P, Y, S, W, X, E, N */ - [BLOCK_JOB_VERB_CANCEL] = {0, 1, 1, 1, 1, 1, 1, 0, 0, 0}, - [BLOCK_JOB_VERB_PAUSE] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0}, - [BLOCK_JOB_VERB_RESUME] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0}, - [BLOCK_JOB_VERB_SET_SPEED] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0}, - [BLOCK_JOB_VERB_COMPLETE] = {0, 0, 0, 0, 1, 0, 0, 0, 0, 0}, - [BLOCK_JOB_VERB_DISMISS] = {0, 0, 0, 0, 0, 0, 0, 0, 1, 0}, + /* U, C, R, P, Y, S, W, D, X, E, N */ + [BLOCK_JOB_VERB_CANCEL] = {0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0}, + [BLOCK_JOB_VERB_PAUSE] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0}, + [BLOCK_JOB_VERB_RESUME] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0}, + [BLOCK_JOB_VERB_SET_SPEED] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0}, + [BLOCK_JOB_VERB_COMPLETE] = {0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0}, + [BLOCK_JOB_VERB_DISMISS] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0}, }; static void block_job_state_transition(BlockJob *job, BlockJobStatus s1) @@ -119,6 +120,7 @@ static void __attribute__((__constructor__)) block_job_init(void) static void block_job_event_cancelled(BlockJob *job); static void block_job_event_completed(BlockJob *job, const char *msg); static void block_job_event_concluded(BlockJob *job); +static int block_job_event_pending(BlockJob *job); static void block_job_enter_cond(BlockJob *job, bool(*fn)(BlockJob *job)); /* Transactional group of block jobs */ @@ -481,17 +483,21 @@ static void block_job_cancel_async(BlockJob *job) job->cancelled = true; } -static int block_job_txn_apply(BlockJobTxn *txn, int fn(BlockJob *)) +static int block_job_txn_apply(BlockJobTxn *txn, int fn(BlockJob *), bool lock) { AioContext *ctx; BlockJob *job, *next; int rc; QLIST_FOREACH_SAFE(job, &txn->jobs, txn_list, next) { - ctx = blk_get_aio_context(job->blk); - aio_context_acquire(ctx); + if (lock) { + ctx = blk_get_aio_context(job->blk); + aio_context_acquire(ctx); + } rc = fn(job); - aio_context_release(ctx); + if (lock) { + aio_context_release(ctx); + } if (rc) { break; } @@ -601,14 +607,15 @@ static void block_job_completed_txn_success(BlockJob *job) } /* Jobs may require some prep-work to complete without failure */ - rc = block_job_txn_apply(txn, block_job_prepare); + rc = block_job_txn_apply(txn, block_job_prepare, true); if (rc) { block_job_completed_txn_abort(job); return; } /* We are the last completed job, commit the transaction. */ - block_job_txn_apply(txn, block_job_completed_single); + block_job_txn_apply(txn, block_job_event_pending, false); + block_job_txn_apply(txn, block_job_completed_single, true); } /* Assumes the block_job_mutex is held */ @@ -817,6 +824,17 @@ static void block_job_event_completed(BlockJob *job, const char *msg) &error_abort); } +static int block_job_event_pending(BlockJob *job) +{ + block_job_state_transition(job, BLOCK_JOB_STATUS_PENDING); + if (job->manual && !block_job_is_internal(job)) { + qapi_event_send_block_job_pending(job->driver->job_type, + job->id, + &error_abort); + } + return 0; +} + static void block_job_event_concluded(BlockJob *job) { block_job_state_transition(job, BLOCK_JOB_STATUS_CONCLUDED); diff --git a/qapi/block-core.json b/qapi/block-core.json index 3005aac7e2..acf9e56876 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1002,6 +1002,11 @@ # to the waiting state. This status will likely not be visible for # the last job in a transaction. # +# @pending: The job has finished its work, but has finalization steps that it +# needs to make prior to completing. These changes may require +# manual intervention by the management process if manual was set +# to true. +# # @aborting: The job is in the process of being aborted, and will finish with # an error. The job will afterwards report that it is @concluded. # This status may not be visible to the management process. @@ -1016,7 +1021,7 @@ ## { 'enum': 'BlockJobStatus', 'data': ['undefined', 'created', 'running', 'paused', 'ready', 'standby', - 'waiting', 'aborting', 'concluded', 'null' ] } + 'waiting', 'pending', 'aborting', 'concluded', 'null' ] } ## # @BlockJobInfo: @@ -3961,6 +3966,30 @@ 'data': { 'type' : 'BlockJobType', 'id' : 'str' } } +## +# @BLOCK_JOB_PENDING: +# +# Emitted when a block job is awaiting explicit authorization to finalize graph +# changes via @block-job-finalize. If this job is part of a transaction, it will +# not emit this event until the transaction has converged first. +# +# @type: job type +# +# @id: The job identifier. +# +# Since: 2.12 +# +# Example: +# +# <- { "event": "BLOCK_JOB_WAITING", +# "data": { "device": "drive0", "type": "mirror" }, +# "timestamp": { "seconds": 1265044230, "microseconds": 450486 } } +# +## +{ 'event': 'BLOCK_JOB_PENDING', + 'data': { 'type' : 'BlockJobType', + 'id' : 'str' } } + ## # @PreallocMode: # From patchwork Fri Feb 23 23:51:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877343 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7bT5mkWz9sW0 for ; Sat, 24 Feb 2018 11:03:33 +1100 (AEDT) Received: from localhost ([::1]:47932 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNJL-0006ur-So for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 19:03:31 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55431) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8Q-0005xI-9J for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8K-0005wO-Oz for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:14 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:44368 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8B-0005ZO-3V; Fri, 23 Feb 2018 18:51:59 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9FFD8EB715; Fri, 23 Feb 2018 23:51:55 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6F185104947D; Fri, 23 Feb 2018 23:51:55 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:39 -0500 Message-Id: <20180223235142.21501-19-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Fri, 23 Feb 2018 23:51:55 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Fri, 23 Feb 2018 23:51:55 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 18/21] blockjobs: add block-job-finalize X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Instead of automatically transitioning from PENDING to CONCLUDED, gate the .prepare() and .commit() phases behind an explicit acknowledgement provided by the QMP monitor if manual completion mode has been requested. This allows us to perform graph changes in prepare and/or commit so that graph changes do not occur autonomously without knowledge of the controlling management layer. Transactions that have reached the "PENDING" state together can all be moved to invoke their finalization methods by issuing block_job_finalize to any one job in the transaction. Jobs in a transaction with mixed job->manual settings will remain stuck in the "WAITING" state until block_job_finalize is authored on the job(s) that have reached the "PENDING" state. These jobs are not allowed to progress because other jobs in the transaction may still fail during their preparation phase during finalization, so these jobs must remain in the WAITING phase until success is guaranteed. These jobs will then automatically dismiss themselves, but jobs that had the manual property set will remain at CONCLUDED as normal. Signed-off-by: John Snow Reviewed-by: Eric Blake --- block/trace-events | 1 + blockdev.c | 14 ++++++++++ blockjob.c | 69 +++++++++++++++++++++++++++++++++++++----------- include/block/blockjob.h | 17 ++++++++++++ qapi/block-core.json | 23 +++++++++++++++- 5 files changed, 108 insertions(+), 16 deletions(-) diff --git a/block/trace-events b/block/trace-events index 5e531e0310..a81b66ff36 100644 --- a/block/trace-events +++ b/block/trace-events @@ -51,6 +51,7 @@ qmp_block_job_cancel(void *job) "job %p" qmp_block_job_pause(void *job) "job %p" qmp_block_job_resume(void *job) "job %p" qmp_block_job_complete(void *job) "job %p" +qmp_block_job_finalize(void *job) "job %p" qmp_block_job_dismiss(void *job) "job %p" qmp_block_stream(void *bs, void *job) "bs %p job %p" diff --git a/blockdev.c b/blockdev.c index 3180130782..05fd421cdc 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3852,6 +3852,20 @@ void qmp_block_job_complete(const char *device, Error **errp) aio_context_release(aio_context); } +void qmp_block_job_finalize(const char *id, Error **errp) +{ + AioContext *aio_context; + BlockJob *job = find_block_job(id, &aio_context, errp); + + if (!job) { + return; + } + + trace_qmp_block_job_finalize(job); + block_job_finalize(job, errp); + aio_context_release(aio_context); +} + void qmp_block_job_dismiss(const char *id, Error **errp) { AioContext *aio_context; diff --git a/blockjob.c b/blockjob.c index 23b4b99fd4..f9e8a64261 100644 --- a/blockjob.c +++ b/blockjob.c @@ -65,14 +65,15 @@ bool BlockJobVerbTable[BLOCK_JOB_VERB__MAX][BLOCK_JOB_STATUS__MAX] = { [BLOCK_JOB_VERB_RESUME] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0}, [BLOCK_JOB_VERB_SET_SPEED] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0}, [BLOCK_JOB_VERB_COMPLETE] = {0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0}, + [BLOCK_JOB_VERB_FINALIZE] = {0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0}, [BLOCK_JOB_VERB_DISMISS] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0}, }; -static void block_job_state_transition(BlockJob *job, BlockJobStatus s1) +static bool block_job_state_transition(BlockJob *job, BlockJobStatus s1) { BlockJobStatus s0 = job->status; if (s0 == s1) { - return; + return false; } assert(s1 >= 0 && s1 <= BLOCK_JOB_STATUS__MAX); trace_block_job_state_transition(job, job->ret, BlockJobSTT[s0][s1] ? @@ -83,6 +84,7 @@ static void block_job_state_transition(BlockJob *job, BlockJobStatus s1) s1)); assert(BlockJobSTT[s0][s1]); job->status = s1; + return true; } static int block_job_apply_verb(BlockJob *job, BlockJobVerb bv, Error **errp) @@ -432,7 +434,7 @@ static void block_job_clean(BlockJob *job) } } -static int block_job_completed_single(BlockJob *job) +static int block_job_finalize_single(BlockJob *job) { assert(job->completed); @@ -581,18 +583,44 @@ static void block_job_completed_txn_abort(BlockJob *job) assert(other_job->cancelled); block_job_finish_sync(other_job, NULL, NULL); } - block_job_completed_single(other_job); + block_job_finalize_single(other_job); aio_context_release(ctx); } block_job_txn_unref(txn); } +static int block_job_is_manual(BlockJob *job) +{ + return job->manual; +} + +static void block_job_do_finalize(BlockJob *job) +{ + int rc; + assert(job && job->txn); + + /* For jobs set !job->manual, transition to pending synchronously now */ + block_job_txn_apply(job->txn, block_job_event_pending, false); + + /* prepare the transaction to complete */ + rc = block_job_txn_apply(job->txn, block_job_prepare, true); + if (rc) { + block_job_completed_txn_abort(job); + } else { + block_job_txn_apply(job->txn, block_job_finalize_single, true); + } +} + +static int block_job_pending_conditional(BlockJob *job) +{ + return job->manual ? block_job_event_pending(job) : 0; +} + static void block_job_completed_txn_success(BlockJob *job) { BlockJobTxn *txn = job->txn; BlockJob *other_job; - int rc = 0; block_job_state_transition(job, BLOCK_JOB_STATUS_WAITING); @@ -606,16 +634,15 @@ static void block_job_completed_txn_success(BlockJob *job) } } - /* Jobs may require some prep-work to complete without failure */ - rc = block_job_txn_apply(txn, block_job_prepare, true); - if (rc) { - block_job_completed_txn_abort(job); - return; - } + /* For jobs with (job->manual), transition to the PENDING state. + * jobs with !job->manual are left WAITING (on their pending comrades). */ + block_job_txn_apply(txn, block_job_pending_conditional, false); - /* We are the last completed job, commit the transaction. */ - block_job_txn_apply(txn, block_job_event_pending, false); - block_job_txn_apply(txn, block_job_completed_single, true); + /* Transactions with any manual jobs must await finalization. + * do_finalize will handle lingering WAITING->PENDING transitions. */ + if (!block_job_txn_apply(txn, block_job_is_manual, false)) { + block_job_do_finalize(job); + } } /* Assumes the block_job_mutex is held */ @@ -667,6 +694,15 @@ void block_job_complete(BlockJob *job, Error **errp) job->driver->complete(job, errp); } +void block_job_finalize(BlockJob *job, Error **errp) +{ + assert(job && job->id && job->txn); + if (block_job_apply_verb(job, BLOCK_JOB_VERB_FINALIZE, errp)) { + return; + } + block_job_do_finalize(job); +} + void block_job_dismiss(BlockJob **jobptr, Error **errp) { BlockJob *job = *jobptr; @@ -826,7 +862,10 @@ static void block_job_event_completed(BlockJob *job, const char *msg) static int block_job_event_pending(BlockJob *job) { - block_job_state_transition(job, BLOCK_JOB_STATUS_PENDING); + /* If we're already pending, don't re-announce */ + if (!block_job_state_transition(job, BLOCK_JOB_STATUS_PENDING)) { + return 0; + } if (job->manual && !block_job_is_internal(job)) { qapi_event_send_block_job_pending(job->driver->job_type, job->id, diff --git a/include/block/blockjob.h b/include/block/blockjob.h index 9bb0be1b13..e09064c342 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -245,6 +245,23 @@ void block_job_cancel(BlockJob *job); */ void block_job_complete(BlockJob *job, Error **errp); + +/** + * block_job_finalize: + * @job: The job to fully commit and finish. + * @errp: Error object. + * + * For jobs that have finished their work and are pending + * awaiting explicit acknowledgement to commit their work, + * This will commit that work. + * + * FIXME: Make the below statement universally true: + * For jobs that support the manual workflow mode, all graph + * changes that occur as a result will occur after this command + * and before a successful reply. + */ +void block_job_finalize(BlockJob *job, Error **errp); + /** * block_job_dismiss: * @job: The job to be dismissed. diff --git a/qapi/block-core.json b/qapi/block-core.json index acf9e56876..549c6c02d8 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -972,10 +972,13 @@ # # @dismiss: see @block-job-dismiss # +# @finalize: see @block-job-finalize +# # Since: 2.12 ## { 'enum': 'BlockJobVerb', - 'data': ['cancel', 'pause', 'resume', 'set-speed', 'complete', 'dismiss' ] } + 'data': ['cancel', 'pause', 'resume', 'set-speed', 'complete', 'dismiss', + 'finalize' ] } ## # @BlockJobStatus: @@ -2274,6 +2277,24 @@ ## { 'command': 'block-job-dismiss', 'data': { 'id': 'str' } } +## +# @block-job-finalize: +# +# Once a job that has manual=true reaches the pending state, it can be +# instructed to finalize any graph changes and do any necessary cleanup +# via this command. +# For jobs in a transaction, instructing one job to finalize will force +# ALL jobs in the transaction to finalize, so it is only necessary to instruct +# a single member job to finalize. +# +# @id: The job identifier. +# +# Returns: Nothing on success +# +# Since: 2.12 +## +{ 'command': 'block-job-finalize', 'data': { 'id': 'str' } } + ## # @BlockdevDiscardOptions: # From patchwork Fri Feb 23 23:51:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877335 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7R75BTNz9sW0 for ; Sat, 24 Feb 2018 10:56:19 +1100 (AEDT) Received: from localhost ([::1]:47886 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNCL-0000ff-PS for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 18:56:17 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55277) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8L-0005qh-FS for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8H-0005pW-Hv for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:09 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33876 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8B-0005ai-JB; Fri, 23 Feb 2018 18:51:59 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D77304072467; Fri, 23 Feb 2018 23:51:55 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id A64D11049475; Fri, 23 Feb 2018 23:51:55 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:40 -0500 Message-Id: <20180223235142.21501-20-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:55 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:55 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 19/21] blockjobs: Expose manual property X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Expose the "manual" property via QAPI for the backup-related jobs. As of this commit, this allows the management API to request the "concluded" and "dismiss" semantics for backup jobs. Signed-off-by: John Snow --- blockdev.c | 19 ++++++++++++++++--- qapi/block-core.json | 32 ++++++++++++++++++++++++++------ 2 files changed, 42 insertions(+), 9 deletions(-) diff --git a/blockdev.c b/blockdev.c index 05fd421cdc..2eddb0e726 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3260,7 +3260,7 @@ static BlockJob *do_drive_backup(DriveBackup *backup, BlockJobTxn *txn, AioContext *aio_context; QDict *options = NULL; Error *local_err = NULL; - int flags; + int flags, job_flags = BLOCK_JOB_DEFAULT; int64_t size; bool set_backing_hd = false; @@ -3279,6 +3279,9 @@ static BlockJob *do_drive_backup(DriveBackup *backup, BlockJobTxn *txn, if (!backup->has_job_id) { backup->job_id = NULL; } + if (!backup->has_manual) { + backup->manual = false; + } if (!backup->has_compress) { backup->compress = false; } @@ -3370,11 +3373,14 @@ static BlockJob *do_drive_backup(DriveBackup *backup, BlockJobTxn *txn, goto out; } } + if (backup->manual) { + job_flags |= BLOCK_JOB_MANUAL; + } job = backup_job_create(backup->job_id, bs, target_bs, backup->speed, backup->sync, bmap, backup->compress, backup->on_source_error, backup->on_target_error, - BLOCK_JOB_DEFAULT, NULL, NULL, txn, &local_err); + job_flags, NULL, NULL, txn, &local_err); bdrv_unref(target_bs); if (local_err != NULL) { error_propagate(errp, local_err); @@ -3409,6 +3415,7 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, BlockJobTxn *txn, Error *local_err = NULL; AioContext *aio_context; BlockJob *job = NULL; + int job_flags = BLOCK_JOB_DEFAULT; if (!backup->has_speed) { backup->speed = 0; @@ -3422,6 +3429,9 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, BlockJobTxn *txn, if (!backup->has_job_id) { backup->job_id = NULL; } + if (!backup->has_manual) { + backup->manual = false; + } if (!backup->has_compress) { backup->compress = false; } @@ -3450,10 +3460,13 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, BlockJobTxn *txn, goto out; } } + if (backup->manual) { + job_flags |= BLOCK_JOB_MANUAL; + } job = backup_job_create(backup->job_id, bs, target_bs, backup->speed, backup->sync, NULL, backup->compress, backup->on_source_error, backup->on_target_error, - BLOCK_JOB_DEFAULT, NULL, NULL, txn, &local_err); + job_flags, NULL, NULL, txn, &local_err); if (local_err != NULL) { error_propagate(errp, local_err); } diff --git a/qapi/block-core.json b/qapi/block-core.json index 549c6c02d8..7b3af93682 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1177,6 +1177,16 @@ # @job-id: identifier for the newly-created block job. If # omitted, the device name will be used. (Since 2.7) # +# @manual: True to use an expanded, more explicit job control flow. +# Jobs may transition from a running state to a pending state, +# where they must be instructed to complete manually via +# block-job-finalize. +# Jobs belonging to a transaction must either all or all not use this +# setting. Once a transaction reaches a pending state, issuing the +# finalize command to any one job in the transaction is sufficient +# to finalize the entire transaction. +# (Since 2.12) +# # @device: the device name or node-name of a root node which should be copied. # # @target: the target of the new image. If the file exists, or if it @@ -1217,9 +1227,10 @@ # Since: 1.6 ## { 'struct': 'DriveBackup', - 'data': { '*job-id': 'str', 'device': 'str', 'target': 'str', - '*format': 'str', 'sync': 'MirrorSyncMode', '*mode': 'NewImageMode', - '*speed': 'int', '*bitmap': 'str', '*compress': 'bool', + 'data': { '*job-id': 'str', '*manual': 'bool', 'device': 'str', + 'target': 'str', '*format': 'str', 'sync': 'MirrorSyncMode', + '*mode': 'NewImageMode', '*speed': 'int', + '*bitmap': 'str', '*compress': 'bool', '*on-source-error': 'BlockdevOnError', '*on-target-error': 'BlockdevOnError' } } @@ -1229,6 +1240,16 @@ # @job-id: identifier for the newly-created block job. If # omitted, the device name will be used. (Since 2.7) # +# @manual: True to use an expanded, more explicit job control flow. +# Jobs may transition from a running state to a pending state, +# where they must be instructed to complete manually via +# block-job-finalize. +# Jobs belonging to a transaction must either all or all not use this +# setting. Once a transaction reaches a pending state, issuing the +# finalize command to any one job in the transaction is sufficient +# to finalize the entire transaction. +# (Since 2.12) +# # @device: the device name or node-name of a root node which should be copied. # # @target: the device name or node-name of the backup target node. @@ -1258,9 +1279,8 @@ # Since: 2.3 ## { 'struct': 'BlockdevBackup', - 'data': { '*job-id': 'str', 'device': 'str', 'target': 'str', - 'sync': 'MirrorSyncMode', - '*speed': 'int', + 'data': { '*job-id': 'str', '*manual': 'bool', 'device': 'str', + 'target': 'str', 'sync': 'MirrorSyncMode', '*speed': 'int', '*compress': 'bool', '*on-source-error': 'BlockdevOnError', '*on-target-error': 'BlockdevOnError' } } From patchwork Fri Feb 23 23:51:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877331 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7Mh2SDXz9sW0 for ; Sat, 24 Feb 2018 10:53:19 +1100 (AEDT) Received: from localhost ([::1]:47851 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN9Q-0006Fu-SZ for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 18:53:16 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55382) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8O-0005uw-Ov for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8K-0005vP-2T for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:12 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33878 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8B-0005aj-Jj; Fri, 23 Feb 2018 18:51:59 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1B94E4078073; Fri, 23 Feb 2018 23:51:56 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id DD0DD1049475; Fri, 23 Feb 2018 23:51:55 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:41 -0500 Message-Id: <20180223235142.21501-21-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:56 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 23 Feb 2018 23:51:56 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 20/21] iotests: test manual job dismissal X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: John Snow Reviewed-by: Eric Blake --- tests/qemu-iotests/056 | 195 +++++++++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/056.out | 4 +- 2 files changed, 197 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/056 b/tests/qemu-iotests/056 index 04f2c3c841..bc21ba9af8 100755 --- a/tests/qemu-iotests/056 +++ b/tests/qemu-iotests/056 @@ -29,6 +29,26 @@ backing_img = os.path.join(iotests.test_dir, 'backing.img') test_img = os.path.join(iotests.test_dir, 'test.img') target_img = os.path.join(iotests.test_dir, 'target.img') +def img_create(img, fmt=iotests.imgfmt, size='64M', **kwargs): + fullname = os.path.join(iotests.test_dir, '%s.%s' % (img, fmt)) + optargs = [] + for k,v in kwargs.iteritems(): + optargs = optargs + ['-o', '%s=%s' % (k,v)] + args = ['create', '-f', fmt] + optargs + [fullname, size] + iotests.qemu_img(*args) + return fullname + +def try_remove(img): + try: + os.remove(img) + except OSError: + pass + +def io_write_patterns(img, patterns): + for pattern in patterns: + iotests.qemu_io('-c', 'write -P%s %s %s' % pattern, img) + + class TestSyncModesNoneAndTop(iotests.QMPTestCase): image_len = 64 * 1024 * 1024 # MB @@ -108,5 +128,180 @@ class TestBeforeWriteNotifier(iotests.QMPTestCase): event = self.cancel_and_wait() self.assert_qmp(event, 'data/type', 'backup') +class BackupTest(iotests.QMPTestCase): + def setUp(self): + self.vm = iotests.VM() + self.test_img = img_create('test') + self.dest_img = img_create('dest') + self.vm.add_drive(self.test_img) + self.vm.launch() + + def tearDown(self): + self.vm.shutdown() + try_remove(self.test_img) + try_remove(self.dest_img) + + def hmp_io_writes(self, drive, patterns): + for pattern in patterns: + self.vm.hmp_qemu_io(drive, 'write -P%s %s %s' % pattern) + self.vm.hmp_qemu_io(drive, 'flush') + + def qmp_job_pending_wait(self, device): + event = self.vm.event_wait(name="BLOCK_JOB_PENDING", + match={'data': {'id': device}}) + self.assertNotEqual(event, None) + res = self.vm.qmp("block-job-finalize", id=device) + self.assert_qmp(res, 'return', {}) + + def qmp_backup_and_wait(self, cmd='drive-backup', serror=None, + aerror=None, **kwargs): + if not self.qmp_backup(cmd, serror, **kwargs): + return False + if 'manual' in kwargs and kwargs['manual']: + self.qmp_job_pending_wait(kwargs['device']) + return self.qmp_backup_wait(kwargs['device'], aerror) + + def qmp_backup(self, cmd='drive-backup', + error=None, **kwargs): + self.assertTrue('device' in kwargs) + res = self.vm.qmp(cmd, **kwargs) + if error: + self.assert_qmp(res, 'error/desc', error) + return False + self.assert_qmp(res, 'return', {}) + return True + + def qmp_backup_wait(self, device, error=None): + event = self.vm.event_wait(name="BLOCK_JOB_COMPLETED", + match={'data': {'device': device}}) + self.assertNotEqual(event, None) + try: + failure = self.dictpath(event, 'data/error') + except AssertionError: + # Backup succeeded. + self.assert_qmp(event, 'data/offset', event['data']['len']) + return True + else: + # Failure. + self.assert_qmp(event, 'data/error', qerror) + return False + + def test_dismiss_false(self): + res = self.vm.qmp('query-block-jobs') + self.assert_qmp(res, 'return', []) + self.qmp_backup_and_wait(device='drive0', format=iotests.imgfmt, + sync='full', target=self.dest_img, manual=False) + res = self.vm.qmp('query-block-jobs') + self.assert_qmp(res, 'return', []) + + def test_dismiss_true(self): + res = self.vm.qmp('query-block-jobs') + self.assert_qmp(res, 'return', []) + self.qmp_backup_and_wait(device='drive0', format=iotests.imgfmt, + sync='full', target=self.dest_img, manual=True) + res = self.vm.qmp('query-block-jobs') + self.assert_qmp(res, 'return[0]/status', 'concluded') + res = self.vm.qmp('block-job-dismiss', id='drive0') + self.assert_qmp(res, 'return', {}) + res = self.vm.qmp('query-block-jobs') + self.assert_qmp(res, 'return', []) + + def test_dismiss_bad_id(self): + res = self.vm.qmp('query-block-jobs') + self.assert_qmp(res, 'return', []) + res = self.vm.qmp('block-job-dismiss', id='foobar') + self.assert_qmp(res, 'error/class', 'DeviceNotActive') + + def test_dismiss_collision(self): + res = self.vm.qmp('query-block-jobs') + self.assert_qmp(res, 'return', []) + self.qmp_backup_and_wait(device='drive0', format=iotests.imgfmt, + sync='full', target=self.dest_img, manual=True) + res = self.vm.qmp('query-block-jobs') + self.assert_qmp(res, 'return[0]/status', 'concluded') + # Leave zombie job un-dismissed, observe a failure: + res = self.qmp_backup_and_wait(serror='Need a root block node', + device='drive0', format=iotests.imgfmt, + sync='full', target=self.dest_img, + manual=True) + self.assertEqual(res, False) + # OK, dismiss the zombie. + res = self.vm.qmp('block-job-dismiss', id='drive0') + self.assert_qmp(res, 'return', {}) + res = self.vm.qmp('query-block-jobs') + self.assert_qmp(res, 'return', []) + # Ensure it's really gone. + self.qmp_backup_and_wait(device='drive0', format=iotests.imgfmt, + sync='full', target=self.dest_img, manual=True) + + def dismissal_failure(self, manual_opt): + res = self.vm.qmp('query-block-jobs') + self.assert_qmp(res, 'return', []) + # Give blkdebug something to chew on + self.hmp_io_writes('drive0', + (('0x9a', 0, 512), + ('0x55', '8M', '352k'), + ('0x78', '15872k', '1M'))) + # Add destination node via blkdebug + res = self.vm.qmp('blockdev-add', + node_name='target0', + driver=iotests.imgfmt, + file={ + 'driver': 'blkdebug', + 'image': { + 'driver': 'file', + 'filename': self.dest_img + }, + 'inject-error': [{ + 'event': 'write_aio', + 'errno': 5, + 'immediately': False, + 'once': True + }], + }) + self.assert_qmp(res, 'return', {}) + + res = self.qmp_backup(cmd='blockdev-backup', + device='drive0', target='target0', + on_target_error='stop', + sync='full', + manual=manual_opt) + self.assertTrue(res) + event = self.vm.event_wait(name="BLOCK_JOB_ERROR", + match={'data': {'device': 'drive0'}}) + self.assertNotEqual(event, None) + # OK, job should be wedged + res = self.vm.qmp('query-block-jobs') + self.assert_qmp(res, 'return[0]/status', 'paused') + res = self.vm.qmp('block-job-dismiss', id='drive0') + self.assert_qmp(res, 'error/desc', + "Job 'drive0' in state 'paused' cannot accept" + " command verb 'dismiss'") + res = self.vm.qmp('query-block-jobs') + self.assert_qmp(res, 'return[0]/status', 'paused') + # OK, unstick job and move forward. + res = self.vm.qmp('block-job-resume', device='drive0') + self.assert_qmp(res, 'return', {}) + if manual_opt: + # Job is now PENDING. Wait and finalize it. + self.qmp_job_pending_wait('drive0') + # And now we need to wait for it to conclude; + res = self.qmp_backup_wait(device='drive0') + self.assertTrue(res) + if manual_opt: + # Job should now be languishing: + res = self.vm.qmp('query-block-jobs') + self.assert_qmp(res, 'return[0]/status', 'concluded') + res = self.vm.qmp('block-job-dismiss', id='drive0') + self.assert_qmp(res, 'return', {}) + res = self.vm.qmp('query-block-jobs') + self.assert_qmp(res, 'return', []) + + def test_dismiss_premature(self): + self.dismissal_failure(True) + + def test_dismiss_erroneous(self): + self.dismissal_failure(False) + if __name__ == '__main__': iotests.main(supported_fmts=['qcow2', 'qed']) diff --git a/tests/qemu-iotests/056.out b/tests/qemu-iotests/056.out index 8d7e996700..dae404e278 100644 --- a/tests/qemu-iotests/056.out +++ b/tests/qemu-iotests/056.out @@ -1,5 +1,5 @@ -... +......... ---------------------------------------------------------------------- -Ran 3 tests +Ran 9 tests OK From patchwork Fri Feb 23 23:51:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 877341 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zp7Xf62l6z9sW0 for ; Sat, 24 Feb 2018 11:01:06 +1100 (AEDT) Received: from localhost ([::1]:47920 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epNGy-00054w-Tx for incoming@patchwork.ozlabs.org; Fri, 23 Feb 2018 19:01:05 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55449) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1epN8Q-0005yC-VG for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1epN8N-00062g-NX for qemu-devel@nongnu.org; Fri, 23 Feb 2018 18:52:14 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33452 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1epN8F-0005kw-Gy; Fri, 23 Feb 2018 18:52:03 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5221FA27F1; Fri, 23 Feb 2018 23:51:56 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-231.bos.redhat.com [10.18.17.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 20C32104947C; Fri, 23 Feb 2018 23:51:56 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org Date: Fri, 23 Feb 2018 18:51:42 -0500 Message-Id: <20180223235142.21501-22-jsnow@redhat.com> In-Reply-To: <20180223235142.21501-1-jsnow@redhat.com> References: <20180223235142.21501-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Fri, 23 Feb 2018 23:51:56 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Fri, 23 Feb 2018 23:51:56 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jsnow@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v4 21/21] blockjobs: add manual_mgmt option to transactions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, John Snow , pkrempa@redhat.com, jtc@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This allows us to easily force the option for all jobs belonging to a transaction to ensure consistency with how all those jobs will be handled. This is purely a convenience. Signed-off-by: John Snow --- blockdev.c | 7 ++++++- blockjob.c | 10 +++++++--- include/block/blockjob.h | 5 ++++- qapi/transaction.json | 3 ++- tests/test-blockjob-txn.c | 6 +++--- 5 files changed, 22 insertions(+), 9 deletions(-) diff --git a/blockdev.c b/blockdev.c index 2eddb0e726..34181c41c2 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2225,6 +2225,11 @@ static TransactionProperties *get_transaction_properties( props->completion_mode = ACTION_COMPLETION_MODE_INDIVIDUAL; } + if (!props->has_manual_mgmt) { + props->has_manual_mgmt = true; + props->manual_mgmt = false; + } + return props; } @@ -2250,7 +2255,7 @@ void qmp_transaction(TransactionActionList *dev_list, */ props = get_transaction_properties(props); if (props->completion_mode != ACTION_COMPLETION_MODE_INDIVIDUAL) { - block_job_txn = block_job_txn_new(); + block_job_txn = block_job_txn_new(props->manual_mgmt); } /* drain all i/o before any operations */ diff --git a/blockjob.c b/blockjob.c index f9e8a64261..eaaa2aea65 100644 --- a/blockjob.c +++ b/blockjob.c @@ -136,6 +136,9 @@ struct BlockJobTxn { /* Reference count */ int refcnt; + + /* Participating jobs must use manual completion */ + bool manual; }; static QLIST_HEAD(, BlockJob) block_jobs = QLIST_HEAD_INITIALIZER(block_jobs); @@ -176,11 +179,12 @@ BlockJob *block_job_get(const char *id) return NULL; } -BlockJobTxn *block_job_txn_new(void) +BlockJobTxn *block_job_txn_new(bool manual_mgmt) { BlockJobTxn *txn = g_new0(BlockJobTxn, 1); QLIST_INIT(&txn->jobs); txn->refcnt = 1; + txn->manual = manual_mgmt; return txn; } @@ -944,7 +948,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, job->paused = true; job->pause_count = 1; job->refcnt = 1; - job->manual = (flags & BLOCK_JOB_MANUAL); + job->manual = (flags & BLOCK_JOB_MANUAL) || (txn && txn->manual); job->status = BLOCK_JOB_STATUS_CREATED; block_job_state_transition(job, BLOCK_JOB_STATUS_CREATED); aio_timer_init(qemu_get_aio_context(), &job->sleep_timer, @@ -978,7 +982,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, /* Single jobs are modeled as single-job transactions for sake of * consolidating the job management logic */ if (!txn) { - txn = block_job_txn_new(); + txn = block_job_txn_new(false); block_job_txn_add_job(txn, job); block_job_txn_unref(txn); } else { diff --git a/include/block/blockjob.h b/include/block/blockjob.h index e09064c342..f3d026f13d 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -371,8 +371,11 @@ void block_job_iostatus_reset(BlockJob *job); * All jobs in the transaction either complete successfully or fail/cancel as a * group. Jobs wait for each other before completing. Cancelling one job * cancels all jobs in the transaction. + * + * @manual_mgmt: whether or not jobs that belong to this transaction will be + * forced to use 2.12+ job management semantics */ -BlockJobTxn *block_job_txn_new(void); +BlockJobTxn *block_job_txn_new(bool manual_mgmt); /** * block_job_ref: diff --git a/qapi/transaction.json b/qapi/transaction.json index bd312792da..9611758cb6 100644 --- a/qapi/transaction.json +++ b/qapi/transaction.json @@ -79,7 +79,8 @@ ## { 'struct': 'TransactionProperties', 'data': { - '*completion-mode': 'ActionCompletionMode' + '*completion-mode': 'ActionCompletionMode', + '*manual-mgmt': 'bool' } } diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c index 34f09ef8c1..2d84f9a41e 100644 --- a/tests/test-blockjob-txn.c +++ b/tests/test-blockjob-txn.c @@ -119,7 +119,7 @@ static void test_single_job(int expected) BlockJobTxn *txn; int result = -EINPROGRESS; - txn = block_job_txn_new(); + txn = block_job_txn_new(false); job = test_block_job_start(1, true, expected, &result, txn); block_job_start(job); @@ -158,7 +158,7 @@ static void test_pair_jobs(int expected1, int expected2) int result1 = -EINPROGRESS; int result2 = -EINPROGRESS; - txn = block_job_txn_new(); + txn = block_job_txn_new(false); job1 = test_block_job_start(1, true, expected1, &result1, txn); job2 = test_block_job_start(2, true, expected2, &result2, txn); block_job_start(job1); @@ -220,7 +220,7 @@ static void test_pair_jobs_fail_cancel_race(void) int result1 = -EINPROGRESS; int result2 = -EINPROGRESS; - txn = block_job_txn_new(); + txn = block_job_txn_new(false); job1 = test_block_job_start(1, true, -ECANCELED, &result1, txn); job2 = test_block_job_start(2, false, 0, &result2, txn); block_job_start(job1);