From patchwork Mon Sep 14 05:49:20 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yuanhan Liu X-Patchwork-Id: 517272 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id B175F140909 for ; Mon, 14 Sep 2015 15:49:13 +1000 (AEST) Received: from localhost ([::1]:38514 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZbMdb-0007Ld-Sl for incoming@patchwork.ozlabs.org; Mon, 14 Sep 2015 01:49:11 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37250) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZbMdD-0006iB-Ux for qemu-devel@nongnu.org; Mon, 14 Sep 2015 01:48:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZbMd8-0004T5-Uy for qemu-devel@nongnu.org; Mon, 14 Sep 2015 01:48:47 -0400 Received: from mga11.intel.com ([192.55.52.93]:7862) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZbMd8-0004Sb-LD for qemu-devel@nongnu.org; Mon, 14 Sep 2015 01:48:42 -0400 Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP; 13 Sep 2015 22:48:40 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.17,526,1437462000"; d="scan'208";a="561185252" Received: from yliu-dev.sh.intel.com (HELO yliu-dev) ([10.239.66.60]) by FMSMGA003.fm.intel.com with ESMTP; 13 Sep 2015 22:48:40 -0700 Date: Mon, 14 Sep 2015 13:49:20 +0800 From: Yuanhan Liu To: Jason Wang Message-ID: <20150914054920.GA2339@yliu-dev.sh.intel.com> References: <1441697927-16456-1-git-send-email-yuanhan.liu@linux.intel.com> <1441697927-16456-6-git-send-email-yuanhan.liu@linux.intel.com> <55F0F593.9030408@redhat.com> <20150910035702.GP2925@yliu-dev.sh.intel.com> <55F10B08.1060707@redhat.com> <20150910051735.GQ2925@yliu-dev.sh.intel.com> <55F11A9E.8040303@redhat.com> <20150910061825.GR2925@yliu-dev.sh.intel.com> <55F1290A.1050208@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <55F1290A.1050208@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.93 Cc: changchun.ouyang@intel.com, qemu-devel@nongnu.org, mst@redhat.com Subject: Re: [Qemu-devel] [PATCH 5/7] vhost_net: move vhost_net_set_vq_index ahead at vhost_net_init X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org On Thu, Sep 10, 2015 at 02:54:02PM +0800, Jason Wang wrote: > > > On 09/10/2015 02:18 PM, Yuanhan Liu wrote: > > On Thu, Sep 10, 2015 at 01:52:30PM +0800, Jason Wang wrote: > >> > >> On 09/10/2015 01:17 PM, Yuanhan Liu wrote: > >>> On Thu, Sep 10, 2015 at 12:46:00PM +0800, Jason Wang wrote: > >>>>> > >>>>> On 09/10/2015 11:57 AM, Yuanhan Liu wrote: > >>>>>>> On Thu, Sep 10, 2015 at 11:14:27AM +0800, Jason Wang wrote: > >>>>>>>>> On 09/08/2015 03:38 PM, Yuanhan Liu wrote: > >>>>>>>>>>> So that we could use the `vq_index' as well in the vhost_net_init > >>>>>>>>>>> stage, which is required when adding vhost-user multiple-queue support, > >>>>>>>>>>> where we need the vq_index to indicate which queue pair we are gonna > >>>>>>>>>>> initiate. > >>>>>>>>>>> > >>>>>>>>>>> vhost-user has no multiple queue support yet, hence no queue_index set > >>>>>>>>>>> before. Here is a quick set to 0 at net_vhost_user_init() stage, and it > >>>>>>>>>>> will be set properly soon in the next patch. > >>>>>>>>>>> > >>>>>>>>>>> Signed-off-by: Yuanhan Liu > >>>>>>>>>>> --- > >>>>>>>>>>> hw/net/vhost_net.c | 16 +++++++--------- > >>>>>>>>>>> net/vhost-user.c | 1 + > >>>>>>>>>>> 2 files changed, 8 insertions(+), 9 deletions(-) > >>>>>>>>>>> > >>>>>>>>>>> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c > >>>>>>>>>>> index f9441e9..141b557 100644 > >>>>>>>>>>> --- a/hw/net/vhost_net.c > >>>>>>>>>>> +++ b/hw/net/vhost_net.c > >>>>>>>>>>> @@ -138,6 +138,11 @@ static int vhost_net_get_fd(NetClientState *backend) > >>>>>>>>>>> } > >>>>>>>>>>> } > >>>>>>>>>>> > >>>>>>>>>>> +static void vhost_net_set_vq_index(struct vhost_net *net, int vq_index) > >>>>>>>>>>> +{ > >>>>>>>>>>> + net->dev.vq_index = vq_index; > >>>>>>>>>>> +} > >>>>>>>>>>> + > >>>>>>>>>>> struct vhost_net *vhost_net_init(VhostNetOptions *options) > >>>>>>>>>>> { > >>>>>>>>>>> int r; > >>>>>>>>>>> @@ -167,6 +172,8 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options) > >>>>>>>>>>> } > >>>>>>>>>>> net->nc = options->net_backend; > >>>>>>>>>>> > >>>>>>>>>>> + vhost_net_set_vq_index(net, net->nc->queue_index * 2); > >>>>>>>>>>> + > >>>>>>>>> This breaks vhost kernel multiqueue since queue_index was not > >>>>>>>>> initialized at this time. > >>>>>>> Right, thanks for pointing it out. > >>>>>>> > >>>>>>>>> We do this in set_netdev() instead of setting > >>>>>>>>> it in each kind of netdev. > >>>>>>> Can we move it to net_init_tap() for setting the right queue_index > >>>>>>> for each nc? > >>>>>>> > >>>>>>> Or, can we call vhost_net_set_vq_index twice, one at vhost_net_init(for > >>>>>>> vhost-user mq support), another one at vhost_net_start(for vhost kernel > >>>>>>> mq support)? > >>>>>>> > >>>>>>> Or, do you have better ideas? > >>>>> I think setting queue_index in net_init_tap() looks ok. > >>> Good to know. > >>> > >>>>> But a question > >>>>> is that why need we do this at so early stage? ( Even before its peers > >>>>> is connected.) > >>> For vhost-user multiple queues support, we will invoke vhost_net_init() > >>> N times for each queue pair, and hence we need to distinguish which > >>> queue it is while sending messages like VHOST_SET_VRING_CALL for > >>> initializing corresponding queue pair. > >>> > >>> Does that make sense to you? > >>> > >> Not sure. Since current codes works for vhost-kernel. (vhost_net_init() > >> was also called N times). We don't want to break existed vhost-kernel > >> API when developing multiqueue. For each virtqueue TX/RX pair, we have > >> one vhost net device and it has no knowledge for the others (which was > >> hide by qemu). So VHOST_SET_VRING_CALL works without any change here. > >> > >> For the case here, since you still have multiple instances of vhost_net > >> structure. Maybe the vhost-user backend can distinguish form this? > > Yeah, I guess that's the difference between vhost-user and vhost-kernel. > > Vhost-kernel opens a char device(/dev/vhost-net) for each vhost_dev, > > hence it's distinguishable. But for vhost-user, all vhost_dev share one > > char device(a socket) for communication, hence, it's not distinguishable. > > How about using individual socket in this case? This seems can also > minimize the changes of backend. > > > > > I was thinking maybe we could export vhost_net_set_vq_index() and invoke > > it at net/vhost-user.c, so that we break nothing, and in the meantime, > > it keeps the logic inside vhost-user. > > > > What do you think? > > > > --yliu > > > > Sounds work. Then I believe you will need to set queue_index in > vhost_user initialization code? Nah, it will not work, as vhost_net_set_vq_index() needs a vhost_net as it's parameter: you can't do that before vhost_net_init(), but it's useless to do that after vhost_net_init(). My bad for not being aware it in the first time. How about following then? Thanks. --yliu diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index e2985bc..7197c3f 100644 --- a/hw/net/vhost_net.c +++ b/hw/net/vhost_net.c @@ -148,6 +148,7 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options) fprintf(stderr, "vhost-net requires net backend to be setup\n"); goto fail; } + net->nc = options->net_backend; net->dev.max_queues = 1; @@ -164,8 +165,10 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options) net->dev.backend_features = 0; net->dev.protocol_features = 0; net->backend = -1; + + /* vhost-user needs vq_index to initiate a specific queue pair */ + net->dev.vq_index = net->nc->queue_index * 2; } - net->nc = options->net_backend; net->dev.nvqs = 2; net->dev.vqs = net->vqs;