From patchwork Thu Jun 25 11:01:29 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thibaut Collet X-Patchwork-Id: 488414 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id E5BD61402C3 for ; Thu, 25 Jun 2015 21:01:52 +1000 (AEST) Received: from localhost ([::1]:54968 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z84uk-00012d-R3 for incoming@patchwork.ozlabs.org; Thu, 25 Jun 2015 07:01:50 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60524) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z84uU-0000m5-CC for qemu-devel@nongnu.org; Thu, 25 Jun 2015 07:01:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z84uQ-0001ML-SI for qemu-devel@nongnu.org; Thu, 25 Jun 2015 07:01:34 -0400 Received: from mail-ig0-f177.google.com ([209.85.213.177]:37795) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z84uQ-0001Lw-N5 for qemu-devel@nongnu.org; Thu, 25 Jun 2015 07:01:30 -0400 Received: by igblr2 with SMTP id lr2so52680149igb.0 for ; Thu, 25 Jun 2015 04:01:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=Po4pvjfr0YY39x7Zcpg19K9MgPPWnhBp4/4SeOw1Cgo=; b=lojLnew6EsU/7v8MyVXQ7M4alzMo13GWF6X/4B5+f9VovsaucnJQg3Aw9XGZR+IFRU 9EOiFuHs7DpO8KLFkdsekNV04yqlQYpEuUj0CXFv6vRjLRMO3PNVVHdHxdbFQiiZOIqF cV38ujsVB9vlMjHkw1zNU/73NwOKzYsw3kxeU5mrHTacVG71X+gJhCnUjz3XZLoRKyL7 7VIMblKRnuY/P07JfU5DOXehWoBUBt2bUFEKkP7r5X1d4S6ooEkHORFfcvgfQy2YO6L7 lZTxhMrp6DbmzD7jtYQLXYIc9DLJd6MQz1PZ4c6HsaPFrmpui1WqAGDUFa3dNRmBz0ha 1ElA== X-Gm-Message-State: ALoCoQm/JtzCn2FJNxkIcAdQnRhidWPBTohZeSVdYifLvpFj76D+F67xESenPJqw8+xO937wIgKa MIME-Version: 1.0 X-Received: by 10.42.90.138 with SMTP id k10mr19674711icm.86.1435230089738; Thu, 25 Jun 2015 04:01:29 -0700 (PDT) Received: by 10.64.58.136 with HTTP; Thu, 25 Jun 2015 04:01:29 -0700 (PDT) In-Reply-To: <558BD0F7.9070306@redhat.com> References: <557E8211.1050606@redhat.com> <20150615103617-mutt-send-email-mst@redhat.com> <557FB435.1030202@redhat.com> <557FD8C4.8050809@redhat.com> <5588C081.1040800@redhat.com> <20150623073410-mutt-send-email-mst@redhat.com> <558A6AD3.2090005@redhat.com> <20150624130113-mutt-send-email-mst@redhat.com> <558BD0F7.9070306@redhat.com> Date: Thu, 25 Jun 2015 13:01:29 +0200 Message-ID: From: Thibaut Collet To: Jason Wang X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.85.213.177 Cc: qemu-devel , Stefan Hajnoczi , "Michael S. Tsirkin" Subject: Re: [Qemu-devel] [PATCH v3 2/2] vhost user: Add RARP injection for legacy guest X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org On Thu, Jun 25, 2015 at 11:59 AM, Jason Wang wrote: > > > > On 06/24/2015 07:05 PM, Michael S. Tsirkin wrote: > > On Wed, Jun 24, 2015 at 04:31:15PM +0800, Jason Wang wrote: > >> > >> On 06/23/2015 01:49 PM, Michael S. Tsirkin wrote: > >>> On Tue, Jun 23, 2015 at 10:12:17AM +0800, Jason Wang wrote: > >>>>> > >>>>> On 06/18/2015 11:16 PM, Thibaut Collet wrote: > >>>>>>> On Tue, Jun 16, 2015 at 10:05 AM, Jason Wang wrote: > >>>>>>>>> On 06/16/2015 03:24 PM, Thibaut Collet wrote: > >>>>>>>>>>> If my understanding is correct, on a resume operation, we have the > >>>>>>>>>>> following callback trace: > >>>>>>>>>>> 1. virtio_pci_restore function that calls all restore call back of > >>>>>>>>>>> virtio devices > >>>>>>>>>>> 2. virtnet_restore that calls try_fill_recv function for each virtual queues > >>>>>>>>>>> 3. try_fill_recv function kicks the virtual queue (through > >>>>>>>>>>> virtqueue_kick function) > >>>>>>>>> Yes, but this happens only after pm resume not migration. Migration is > >>>>>>>>> totally transparent to guest. > >>>>>>>>> > >>>>>>> Hi Jason, > >>>>>>> > >>>>>>> After a deeper look in the migration code of QEMU a resume event is > >>>>>>> always sent when the live migration is finished. > >>>>>>> On a live migration we have the following callback trace: > >>>>>>> 1. The VM on the new host is set to the state RUN_STATE_INMIGRATE, the > >>>>>>> autostart boolean to 1 and calls the qemu_start_incoming_migration > >>>>>>> function (see function main of vl.c) > >>>>>>> ..... > >>>>>>> 2. call of process_incoming_migration function in > >>>>>>> migration/migration.c file whatever the way to do the live migration > >>>>>>> (tcp:, fd:, unix:, exec: ...) > >>>>>>> 3. call of process_incoming_migration_co function in migration/migration.c > >>>>>>> 4. call of vm_start function in vl.c (otherwise the migrated VM stay > >>>>>>> in the pause state, the autostart boolean is set to 1 by the main > >>>>>>> function in vl.c) > >>>>>>> 5. call of vm_start function that sets the VM is the RUN_STATE_RUNNING state. > >>>>>>> 6. call of qapi_event_send_resume function that ends a resume event to the VM > >>>>> AFAIK, this function sends resume event to qemu monitor not VM. > >>>>> > >>>>>>> So when a live migration is ended: > >>>>>>> 1. a resume event is sent to the guest > >>>>>>> 2. On the reception of this resume event the virtual queue are kicked > >>>>>>> by the guest > >>>>>>> 3. Backend vhost user catches this kick and can emit a RARP to guest > >>>>>>> that does not support GUEST_ANNOUNCE > >>>>>>> > >>>>>>> This solution, as solution based on detection of DRIVER_OK status > >>>>>>> suggested by Michael, allows backend to send the RARP to legacy guest > >>>>>>> without involving QEMU and add ioctl to vhost-user. > >>>>> A question here is did vhost-user code pass status to the backend? If > >>>>> not, how can userspace backend detect DRIVER_OK? > >>> Sorry, I must have been unclear. > >>> vhost core calls VHOST_NET_SET_BACKEND on DRIVER_OK. > >>> Unfortunately vhost user currently translates it to VHOST_USER_NONE. > >> Looks like VHOST_NET_SET_BACKEND was only used for tap backend. > >> > >>> As a work around, I think kicking ioeventfds once you get > >>> VHOST_NET_SET_BACKEND will work. > >> Maybe just a eventfd_set() in vhost_net_start(). But is this > >> "workaround" elegant enough to be documented? Is it better to do this > >> explicitly with a new feature? > > If you are going to do this anyway, there are a couple of other changes > > we should do, in particular, decide what we want to do with control vq. > > > > If I understand correctly, you mean VIRTIO_NET_CTRL_MQ and > VIRTIO_NET_CTRL_GUEST_OFFLOADS? Looks like both of these were broken. > Need more thought, maybe new kinds of requests. > > Are there any objections to add VHOST_NET_SET_BACKEND support to vhost user with a patch like that: hw/net/vhost_net.c | 8 ++++++++ hw/virtio/vhost-user.c | 10 +++++++++- 2 files changed, 17 insertions(+), 1 deletion(-) static VhostUserRequest vhost_user_request_translate(unsigned long int request) @@ -287,6 +289,12 @@ static int vhost_user_call(struct vhost_dev *dev, unsigned long int request, msg.u64 |= VHOST_USER_VRING_NOFD_MASK; } break; + + case VHOST_NET_SET_BACKEND: + memcpy(&msg.file, arg, sizeof(struct vhost_vring_state)); + msg.size = sizeof(m.state); + break; + default: error_report("vhost-user trying to send unhandled ioctl"); return -1; This message will be sent when guest is ready and can be used by vhost user backend to send RARP to legacy guest. This solution avoids to add new message and has no impact on control vq. diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index 907e002..7a008c0 100644 --- a/hw/net/vhost_net.c +++ b/hw/net/vhost_net.c @@ -234,6 +234,14 @@ static int vhost_net_start_one(struct vhost_net *net, goto fail; } } + } else if (net->nc->info->type == NET_CLIENT_OPTIONS_KIND_VHOST_USER) { + file.fd = 0; + for (file.index = 0; file.index < net->dev.nvqs; ++file.index) { + const VhostOps *vhost_ops = net->dev.vhost_ops; + int r = vhost_ops->vhost_call(&net->dev, VHOST_NET_SET_BACKEND, + &file); + assert(r >= 0); + } } return 0; fail: diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index d6f2163..32c6bd9 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -41,6 +41,7 @@ typedef enum VhostUserRequest { VHOST_USER_SET_VRING_KICK = 12, VHOST_USER_SET_VRING_CALL = 13, VHOST_USER_SET_VRING_ERR = 14, + VHOST_USER_NET_SET_BACKEND = 15, VHOST_USER_MAX } VhostUserRequest; @@ -104,7 +105,8 @@ static unsigned long int ioctl_to_vhost_user_request[VHOST_USER_MAX] = { VHOST_GET_VRING_BASE, /* VHOST_USER_GET_VRING_BASE */ VHOST_SET_VRING_KICK, /* VHOST_USER_SET_VRING_KICK */ VHOST_SET_VRING_CALL, /* VHOST_USER_SET_VRING_CALL */ - VHOST_SET_VRING_ERR /* VHOST_USER_SET_VRING_ERR */ + VHOST_SET_VRING_ERR, /* VHOST_USER_SET_VRING_ERR */ + VHOST_NET_SET_BACKEND /* VHOST_USER_NET_SET_BACKEND */ };