From patchwork Tue May 11 17:26:33 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 52302 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id CADD3B7DA4 for ; Wed, 12 May 2010 03:31:50 +1000 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932294Ab0EKRan (ORCPT ); Tue, 11 May 2010 13:30:43 -0400 Received: from mx1.redhat.com ([209.132.183.28]:17573 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932195Ab0EKRak (ORCPT ); Tue, 11 May 2010 13:30:40 -0400 Received: from int-mx08.intmail.prod.int.phx2.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o4BHUWQW024282 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 11 May 2010 13:30:32 -0400 Received: from redhat.com (dhcp-0-94.tlv.redhat.com [10.35.0.94]) by int-mx08.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with SMTP id o4BHUSYc015560; Tue, 11 May 2010 13:30:29 -0400 Date: Tue, 11 May 2010 20:26:33 +0300 From: "Michael S. Tsirkin" To: "Michael S. Tsirkin" , Juan Quintela , Rusty Russell , "David S. Miller" , "Paul E. McKenney" , kvm@vger.kernel.org, virtualization@lists.osdl.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH RFC] vhost: fix barrier pairing Message-ID: <20100511172633.GA9091@redhat.com> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.5.19 (2009-01-05) X-Scanned-By: MIMEDefang 2.67 on 10.5.11.21 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org According to memory-barriers.txt, an smp memory barrier should always be paired with another smp memory barrier, and I quote "a lack of appropriate pairing is almost certainly an error". In case of vhost, failure to flush out used index update before looking at the interrupt disable flag could result in missed interrupts, resulting in networking hang under stress. This might happen when flags read bypasses used index write. So we see interrupts disabled and do not interrupt, at the same time guest writes flags value to enable interrupt, reads an old used index value, thinks that used ring is empty and waits for interrupt. Note: the barrier we pair with here is in drivers/virtio/virtio_ring.c, function vring_enable_cb. Signed-off-by: Michael S. Tsirkin --- Dave, I think this is needed in 2.6.34, I'll send a pull request after doing some more testing. Rusty, Juan, could you take a look as well please? Thanks! drivers/vhost/vhost.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index e69d238..14fa2f5 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -1035,7 +1035,10 @@ int vhost_add_used(struct vhost_virtqueue *vq, unsigned int head, int len) /* This actually signals the guest, using eventfd. */ void vhost_signal(struct vhost_dev *dev, struct vhost_virtqueue *vq) { - __u16 flags = 0; + __u16 flags; + /* Flush out used index updates. */ + smp_mb(); + if (get_user(flags, &vq->avail->flags)) { vq_err(vq, "Failed to get flags"); return;