[{"id":1761065,"web_url":"http://patchwork.ozlabs.org/comment/1761065/","msgid":"","list_archive_url":null,"date":"2017-08-31T14:37:25","subject":"Re: [Qemu-devel] [PATCH v2] io: add new qio_channel_{readv, writev, \n\tread, write}_all functions","submitter":{"id":6591,"url":"http://patchwork.ozlabs.org/api/people/6591/","name":"Eric Blake","email":"eblake@redhat.com"},"content":"On 08/31/2017 04:46 AM, Daniel P. Berrange wrote:\n> These functions wait until they are able to read / write the full\n> requested data buffer(s).\n> \n> Signed-off-by: Daniel P. Berrange \n> ---\n> \nReviewed-by: Eric Blake ","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx10.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx10.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=eblake@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xjlNM1485z9s7G\n\tfor ;\n\tFri, 1 Sep 2017 00:38:11 +1000 (AEST)","from localhost ([::1]:56111 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dnQbh-0001jj-8d\n\tfor incoming@patchwork.ozlabs.org; Thu, 31 Aug 2017 10:38:09 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:47585)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dnQb6-0001iK-IE\n\tfor qemu-devel@nongnu.org; Thu, 31 Aug 2017 10:37:33 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dnQb1-0004Pt-TS\n\tfor qemu-devel@nongnu.org; Thu, 31 Aug 2017 10:37:32 -0400","from mx1.redhat.com ([209.132.183.28]:39880)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1dnQb1-0004PZ-MW\n\tfor qemu-devel@nongnu.org; Thu, 31 Aug 2017 10:37:27 -0400","from smtp.corp.redhat.com\n\t(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id B0BBF6146E\n\tfor ; Thu, 31 Aug 2017 14:37:26 +0000 (UTC)","from [10.10.122.186] (ovpn-122-186.rdu2.redhat.com [10.10.122.186])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id 02DBDB5150;\n\tThu, 31 Aug 2017 14:37:25 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com B0BBF6146E","To":"\"Daniel P. Berrange\" , qemu-devel@nongnu.org","References":"<20170831094643.22567-1-berrange@redhat.com>","From":"Eric Blake ","Openpgp":"url=http://people.redhat.com/eblake/eblake.gpg","Organization":"Red Hat, Inc.","Message-ID":"","Date":"Thu, 31 Aug 2017 09:37:25 -0500","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170831094643.22567-1-berrange@redhat.com>","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\";\n\tboundary=\"utaCKOFmbfNR5odCNBN7FH1Ks2sHo9f16\"","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.12","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.39]);\n\tThu, 31 Aug 2017 14:37:26 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","X-Content-Filtered-By":"Mailman/MimeDel 2.1.21","Subject":"Re: [Qemu-devel] [PATCH v2] io: add new qio_channel_{readv, writev, \n\tread, write}_all functions","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Paolo Bonzini , Juan Quintela ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1763554,"web_url":"http://patchwork.ozlabs.org/comment/1763554/","msgid":"<334b0d5c-aff2-ca40-b02a-4c95e822c53c@redhat.com>","list_archive_url":null,"date":"2017-09-05T18:25:50","subject":"Re: [Qemu-devel] [PATCH v2] io: add new qio_channel_{readv, writev, \n\tread, write}_all functions","submitter":{"id":6591,"url":"http://patchwork.ozlabs.org/api/people/6591/","name":"Eric Blake","email":"eblake@redhat.com"},"content":"On 08/31/2017 04:46 AM, Daniel P. Berrange wrote:\n> These functions wait until they are able to read / write the full\n> requested data buffer(s).\n> \n> Signed-off-by: Daniel P. Berrange \n> ---\n> \n> Changed in v2:\n> \n> - Remove 'coroutine_fn' annotation (Stefan)\n> - Fix docs typos (Eric)\n> - Remove bogus error overwriting (Eric)\n> \n> include/io/channel.h | 90 +++++++++++++++++++++++++++++++++++++++\n> io/channel.c | 94 +++++++++++++++++++++++++++++++++++++++++\n> tests/io-channel-helpers.c | 102 ++++-----------------------------------------\n> 3 files changed, 193 insertions(+), 93 deletions(-)\n\nAlready applied, but just now noticing two things:\n\n> \n> diff --git a/include/io/channel.h b/include/io/channel.h\n> index 54f3dc252f..8f25893c45 100644\n> --- a/include/io/channel.h\n> +++ b/include/io/channel.h\n> @@ -269,6 +269,58 @@ ssize_t qio_channel_writev_full(QIOChannel *ioc,\n> Error **errp);\n> \n> /**\n> + * qio_channel_readv_all:\n> + * @ioc: the channel object\n> + * @iov: the array of memory regions to read data into\n> + * @niov: the length of the @iov array\n> + * @errp: pointer to a NULL-initialized error object\n> + *\n> + * Read data from the IO channel, storing it in the\n> + * memory regions referenced by @iov. Each element\n> + * in the @iov will be fully populated with data\n> + * before the next one is used. The @niov parameter\n> + * specifies the total number of elements in @iov.\n> + *\n> + * The function will wait for all requested data\n> + * to be read, yielding from the current coroutine\n> + * if required.\n\n[1] This states we yield...\n\n> + *\n> + * If end-of-file occurs before all requested data\n> + * has been read, an error will be reported.\n\n[2] nbd_read_eof() can't use this function, because it wants to\ndistinguish between early EOF (no data at all - server quit at a sane\npoint in the protocol, so the client doesn't need to report an error but\njust start clean shutdown) and short read (EOF encountered after at\nleast one byte was read - server quit mid-message and client must report\nthe error). But we don't want all callers to have to check for this\ncorner-case. Obvious solution: add qio_channel_readv_all_eof() (I've\ngot the patch ready to submit, and can take it through my NBD tree since\nI'm also patch nbd to take advantage of these new functions).\n\n> +int qio_channel_readv_all(QIOChannel *ioc,\n\n> + *\n> + * The function will wait for all requested data\n> + * to be written, yielding from the current coroutine\n> + * if required.\n\n[1] again\n\n> +++ b/io/channel.c\n> @@ -22,6 +22,7 @@\n> #include \"io/channel.h\"\n> #include \"qapi/error.h\"\n> #include \"qemu/main-loop.h\"\n> +#include \"qemu/iov.h\"\n> \n> bool qio_channel_has_feature(QIOChannel *ioc,\n> QIOChannelFeature feature)\n> @@ -85,6 +86,79 @@ ssize_t qio_channel_writev_full(QIOChannel *ioc,\n> }\n> \n> \n> +\n> +int qio_channel_readv_all(QIOChannel *ioc,\n> + const struct iovec *iov,\n> + size_t niov,\n> + Error **errp)\n> +{\n> + int ret = -1;\n> + struct iovec *local_iov = g_new(struct iovec, niov);\n> + struct iovec *local_iov_head = local_iov;\n> + unsigned int nlocal_iov = niov;\n> +\n> + nlocal_iov = iov_copy(local_iov, nlocal_iov,\n> + iov, niov,\n> + 0, iov_size(iov, niov));\n> +\n> + while (nlocal_iov > 0) {\n> + ssize_t len;\n> + len = qio_channel_readv(ioc, local_iov, nlocal_iov, errp);\n> + if (len == QIO_CHANNEL_ERR_BLOCK) {\n> + qio_channel_wait(ioc, G_IO_IN);\n\n[1] Wait a minute. qio_channel_wait() spawns a NEW coroutine, rather\nthan yielding the current coroutine. nbd_rwv() was using\nqio_channel_yield() here (after asserting that QIO_CHANNEL_ERR_BLOCK is\nonly possible for a non-blocking channel). Keeping the wait in place\nbreaks iotests (things hang), while s/wait/yield/ lets NBD get through\niotests, but then breaks check-tests/test-io/channel-socket. So I think\nwe have to make the code choose between the two conditions according to\nwhether it is currently in a coroutine. I'll propose that patch, and\nsee what you say about it :)\n\n> + continue;\n> + } else if (len < 0) {\n> + goto cleanup;\n> + } else if (len == 0) {\n> + error_setg(errp,\n> + \"Unexpected end-of-file before all bytes were read\");\n\n[2] This is where the special-casing for early EOF vs. short read fits.\n\n\n> +int qio_channel_writev_all(QIOChannel *ioc,\n> + const struct iovec *iov,\n> + size_t niov,\n> + Error **errp)\n> +{\n> + int ret = -1;\n> + struct iovec *local_iov = g_new(struct iovec, niov);\n> + struct iovec *local_iov_head = local_iov;\n> + unsigned int nlocal_iov = niov;\n> +\n> + nlocal_iov = iov_copy(local_iov, nlocal_iov,\n> + iov, niov,\n> + 0, iov_size(iov, niov));\n> +\n> + while (nlocal_iov > 0) {\n> + ssize_t len;\n> + len = qio_channel_writev(ioc, local_iov, nlocal_iov, errp);\n> + if (len == QIO_CHANNEL_ERR_BLOCK) {\n> + qio_channel_wait(ioc, G_IO_OUT);\n\n[1] again","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx08.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx08.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=eblake@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xmwCf16Mrz9sNV\n\tfor ;\n\tWed, 6 Sep 2017 04:26:37 +1000 (AEST)","from localhost ([::1]:60555 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dpIYW-0003bP-0d\n\tfor incoming@patchwork.ozlabs.org; Tue, 05 Sep 2017 14:26:36 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:40513)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dpIXv-0003Xe-KJ\n\tfor qemu-devel@nongnu.org; Tue, 05 Sep 2017 14:26:04 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dpIXq-00013J-Fn\n\tfor qemu-devel@nongnu.org; Tue, 05 Sep 2017 14:25:59 -0400","from mx1.redhat.com ([209.132.183.28]:41484)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1dpIXq-00011t-66\n\tfor qemu-devel@nongnu.org; Tue, 05 Sep 2017 14:25:54 -0400","from smtp.corp.redhat.com\n\t(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 70112C0753D6\n\tfor ; Tue, 5 Sep 2017 18:25:52 +0000 (UTC)","from [10.10.120.228] (ovpn-120-228.rdu2.redhat.com [10.10.120.228])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id B0B5D7D7BD;\n\tTue, 5 Sep 2017 18:25:51 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 70112C0753D6","To":"\"Daniel P. Berrange\" , qemu-devel@nongnu.org","References":"<20170831094643.22567-1-berrange@redhat.com>","From":"Eric Blake ","Openpgp":"url=http://people.redhat.com/eblake/eblake.gpg","Organization":"Red Hat, Inc.","Message-ID":"<334b0d5c-aff2-ca40-b02a-4c95e822c53c@redhat.com>","Date":"Tue, 5 Sep 2017 13:25:50 -0500","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170831094643.22567-1-berrange@redhat.com>","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\";\n\tboundary=\"9IJBX8Gub3C3t0VGIVh4vqrNde7mQh2PK\"","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.11","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.32]);\n\tTue, 05 Sep 2017 18:25:52 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","X-Content-Filtered-By":"Mailman/MimeDel 2.1.21","Subject":"Re: [Qemu-devel] [PATCH v2] io: add new qio_channel_{readv, writev, \n\tread, write}_all functions","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Paolo Bonzini , Juan Quintela ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}}]