From patchwork Thu Aug 9 12:30:39 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yoshihiro YUNOMAE X-Patchwork-Id: 176069 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id E0FD82C00D1 for ; Thu, 9 Aug 2012 22:31:10 +1000 (EST) Received: from localhost ([::1]:58937 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SzRtM-0007Hf-Ti for incoming@patchwork.ozlabs.org; Thu, 09 Aug 2012 08:31:08 -0400 Received: from eggs.gnu.org ([208.118.235.92]:56497) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SzRtA-000781-BQ for qemu-devel@nongnu.org; Thu, 09 Aug 2012 08:30:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1SzRt8-0004Dw-QT for qemu-devel@nongnu.org; Thu, 09 Aug 2012 08:30:56 -0400 Received: from mail4.hitachi.co.jp ([133.145.228.5]:52420) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SzRt8-0004Bo-5Y for qemu-devel@nongnu.org; Thu, 09 Aug 2012 08:30:54 -0400 Received: from mlsv1.hitachi.co.jp (unknown [133.144.234.166]) by mail4.hitachi.co.jp (Postfix) with ESMTP id 833E133CC4; Thu, 9 Aug 2012 21:30:52 +0900 (JST) Received: from mfilter06.hitachi.co.jp by mlsv1.hitachi.co.jp (8.13.1/8.13.1) id q79CUqDo024342; Thu, 9 Aug 2012 21:30:52 +0900 Received: from vshuts4.hitachi.co.jp (vshuts4.hitachi.co.jp [10.201.6.80]) by mfilter06.hitachi.co.jp (Switch-3.3.4/Switch-3.3.4) with ESMTP id q79CUpNx025881; Thu, 9 Aug 2012 21:30:51 +0900 X-AuditID: b753bd60-90ac1ba0000047ca-2a-5023ad7afe8d Received: from hsdlmain.sdl.hitachi.co.jp (unknown [133.144.14.194]) by vshuts4.hitachi.co.jp (Symantec Mail Security) with ESMTP id A0FE92043CD; Thu, 9 Aug 2012 21:30:50 +0900 (JST) Received: from hsdlvgate2.sdl.hitachi.co.jp by hsdlmain.sdl.hitachi.co.jp (8.13.8/3.7W11021512) id q79CUov0010297; Thu, 9 Aug 2012 21:30:50 +0900 X-AuditID: b753bd60-90ac1ba0000047ca-2a-5023ad7afe8d Received: from sdl99w.sdl.hitachi.co.jp (sdl99w.sdl.hitachi.co.jp [133.144.14.250]) by hsdlvgate2.sdl.hitachi.co.jp (Symantec Mail Security) with ESMTP id 18C7C236561; Thu, 9 Aug 2012 21:30:50 +0900 (JST) Received: from maild.sdl.hitachi.co.jp (sdl99d.sdl.hitachi.co.jp [133.144.14.199]) by sdl99w.sdl.hitachi.co.jp (Postfix) with ESMTP id AC0F053C158; Thu, 9 Aug 2012 21:30:50 +0900 (JST) Received: from ltc189.sdl.hitachi.co.jp (unknown [10.232.28.60]) by maild.sdl.hitachi.co.jp (Postfix) with ESMTP id 111FE495B83; Thu, 9 Aug 2012 21:30:50 +0900 (JST) From: Yoshihiro YUNOMAE To: linux-kernel@vger.kernel.org Date: Thu, 09 Aug 2012 21:30:39 +0900 Message-ID: <20120809123039.8542.18593.stgit@ltc189.sdl.hitachi.co.jp> In-Reply-To: <20120809123029.8542.38311.stgit@ltc189.sdl.hitachi.co.jp> References: <20120809123029.8542.38311.stgit@ltc189.sdl.hitachi.co.jp> User-Agent: StGIT/0.14.3 MIME-Version: 1.0 X-Brightmail-Tracker: AAAAAA== X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4-2.6 X-Received-From: 133.145.228.5 Cc: Herbert Xu , Arnd Bergmann , Frederic Weisbecker , Rusty Russell , qemu-devel@nongnu.org, Borislav Petkov , virtualization@lists.linux-foundation.org, Masami Hiramatsu , "Franch Ch. Eigler" , Ingo Molnar , Mathieu Desnoyers , Steven Rostedt , Anthony Liguori , Greg Kroah-Hartman , Amit Shah , yrl.pp-manager.tt@hitachi.com Subject: [Qemu-devel] [PATCH V2 1/6] virtio/console: Add splice_write support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Masami Hiramatsu Enable to use splice_write from pipe to virtio-console port. This steals pages from pipe and directly send it to host. Note that this may accelerate only the guest to host path. Changes in v2: - Use GFP_KERNEL instead of GFP_ATOMIC in syscall context function. Signed-off-by: Masami Hiramatsu --- drivers/char/virtio_console.c | 136 +++++++++++++++++++++++++++++++++++++++-- 1 files changed, 128 insertions(+), 8 deletions(-) diff --git a/drivers/char/virtio_console.c b/drivers/char/virtio_console.c index cdf2f54..730816c 100644 --- a/drivers/char/virtio_console.c +++ b/drivers/char/virtio_console.c @@ -24,6 +24,8 @@ #include #include #include +#include +#include #include #include #include @@ -227,6 +229,7 @@ struct port { bool guest_connected; }; +#define MAX_SPLICE_PAGES 32 /* This is the very early arch-specified put chars function. */ static int (*early_put_chars)(u32, const char *, int); @@ -474,26 +477,52 @@ static ssize_t send_control_msg(struct port *port, unsigned int event, return 0; } +struct buffer_token { + union { + void *buf; + struct scatterlist *sg; + } u; + bool sgpages; +}; + +static void reclaim_sg_pages(struct scatterlist *sg) +{ + int i; + struct page *page; + + for (i = 0; i < MAX_SPLICE_PAGES; i++) { + page = sg_page(&sg[i]); + if (!page) + break; + put_page(page); + } + kfree(sg); +} + /* Callers must take the port->outvq_lock */ static void reclaim_consumed_buffers(struct port *port) { - void *buf; + struct buffer_token *tok; unsigned int len; if (!port->portdev) { /* Device has been unplugged. vqs are already gone. */ return; } - while ((buf = virtqueue_get_buf(port->out_vq, &len))) { - kfree(buf); + while ((tok = virtqueue_get_buf(port->out_vq, &len))) { + if (tok->sgpages) + reclaim_sg_pages(tok->u.sg); + else + kfree(tok->u.buf); + kfree(tok); port->outvq_full = false; } } -static ssize_t send_buf(struct port *port, void *in_buf, size_t in_count, - bool nonblock) +static ssize_t __send_to_port(struct port *port, struct scatterlist *sg, + int nents, size_t in_count, + struct buffer_token *tok, bool nonblock) { - struct scatterlist sg[1]; struct virtqueue *out_vq; ssize_t ret; unsigned long flags; @@ -505,8 +534,7 @@ static ssize_t send_buf(struct port *port, void *in_buf, size_t in_count, reclaim_consumed_buffers(port); - sg_init_one(sg, in_buf, in_count); - ret = virtqueue_add_buf(out_vq, sg, 1, 0, in_buf, GFP_ATOMIC); + ret = virtqueue_add_buf(out_vq, sg, nents, 0, tok, GFP_ATOMIC); /* Tell Host to go! */ virtqueue_kick(out_vq); @@ -544,6 +572,37 @@ done: return in_count; } +static ssize_t send_buf(struct port *port, void *in_buf, size_t in_count, + bool nonblock) +{ + struct scatterlist sg[1]; + struct buffer_token *tok; + + tok = kmalloc(sizeof(*tok), GFP_ATOMIC); + if (!tok) + return -ENOMEM; + tok->sgpages = false; + tok->u.buf = in_buf; + + sg_init_one(sg, in_buf, in_count); + + return __send_to_port(port, sg, 1, in_count, tok, nonblock); +} + +static ssize_t send_pages(struct port *port, struct scatterlist *sg, int nents, + size_t in_count, bool nonblock) +{ + struct buffer_token *tok; + + tok = kmalloc(sizeof(*tok), GFP_ATOMIC); + if (!tok) + return -ENOMEM; + tok->sgpages = true; + tok->u.sg = sg; + + return __send_to_port(port, sg, nents, in_count, tok, nonblock); +} + /* * Give out the data that's requested from the buffer that we have * queued up. @@ -725,6 +784,66 @@ out: return ret; } +struct sg_list { + unsigned int n; + size_t len; + struct scatterlist *sg; +}; + +static int pipe_to_sg(struct pipe_inode_info *pipe, struct pipe_buffer *buf, + struct splice_desc *sd) +{ + struct sg_list *sgl = sd->u.data; + unsigned int len = 0; + + if (sgl->n == MAX_SPLICE_PAGES) + return 0; + + /* Try lock this page */ + if (buf->ops->steal(pipe, buf) == 0) { + /* Get reference and unlock page for moving */ + get_page(buf->page); + unlock_page(buf->page); + + len = min(buf->len, sd->len); + sg_set_page(&(sgl->sg[sgl->n]), buf->page, len, buf->offset); + sgl->n++; + sgl->len += len; + } + + return len; +} + +/* Faster zero-copy write by splicing */ +static ssize_t port_fops_splice_write(struct pipe_inode_info *pipe, + struct file *filp, loff_t *ppos, + size_t len, unsigned int flags) +{ + struct port *port = filp->private_data; + struct sg_list sgl; + ssize_t ret; + struct splice_desc sd = { + .total_len = len, + .flags = flags, + .pos = *ppos, + .u.data = &sgl, + }; + + sgl.n = 0; + sgl.len = 0; + sgl.sg = kmalloc(sizeof(struct scatterlist) * MAX_SPLICE_PAGES, + GFP_KERNEL); + if (unlikely(!sgl.sg)) + return -ENOMEM; + + sg_init_table(sgl.sg, MAX_SPLICE_PAGES); + ret = __splice_from_pipe(pipe, &sd, pipe_to_sg); + if (likely(ret > 0)) + ret = send_pages(port, sgl.sg, sgl.n, sgl.len, true); + + return ret; +} + static unsigned int port_fops_poll(struct file *filp, poll_table *wait) { struct port *port; @@ -856,6 +975,7 @@ static const struct file_operations port_fops = { .open = port_fops_open, .read = port_fops_read, .write = port_fops_write, + .splice_write = port_fops_splice_write, .poll = port_fops_poll, .release = port_fops_release, .fasync = port_fops_fasync,