From patchwork Tue Mar 22 11:36:50 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 87894 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id C2DF9B6F77 for ; Tue, 22 Mar 2011 22:37:19 +1100 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755530Ab1CVLhO (ORCPT ); Tue, 22 Mar 2011 07:37:14 -0400 Received: from mx1.redhat.com ([209.132.183.28]:24994 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754220Ab1CVLhM (ORCPT ); Tue, 22 Mar 2011 07:37:12 -0400 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id p2MBb36r008321 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 22 Mar 2011 07:37:03 -0400 Received: from redhat.com (dhcp-1-35.tlv.redhat.com [10.35.1.35]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with SMTP id p2MBb0Ks019526; Tue, 22 Mar 2011 07:37:01 -0400 Date: Tue, 22 Mar 2011 13:36:50 +0200 From: "Michael S. Tsirkin" To: Shirley Ma Cc: Herbert Xu , rusty@rustcorp.com.au, davem@davemloft.net, kvm@vger.kernel.org, netdev@vger.kernel.org Subject: Re: [PATCH 2/2] virtio_net: remove send completion interrupts and avoid TX queue overrun through packet drop Message-ID: <20110322113649.GA17071@redhat.com> References: <20110318133311.GA20623@gondor.apana.org.au> <1300498915.3441.21.camel@localhost.localdomain> <1300730587.3441.24.camel@localhost.localdomain> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <1300730587.3441.24.camel@localhost.localdomain> User-Agent: Mutt/1.5.21 (2010-09-15) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Mon, Mar 21, 2011 at 11:03:07AM -0700, Shirley Ma wrote: > On Fri, 2011-03-18 at 18:41 -0700, Shirley Ma wrote: > > > > + /* Drop packet instead of stop queue for better > > performance > > > */ > > > > > > I would like to see some justification as to why this is the right > > > way to go and not just papering over the real problem. > > > > Fair. KVM guest virtio_net TX queue stop/restart is pretty expensive, > > which involves: > > > > 1. Guest enable callback: one memory barrier, interrupt flag set > > Missed this cost: for history reason, it also involves a guest exit from > I/O write (PCI_QUEUE_NOTIFY). OK, after some research, it looks like the reason was the tx timer that qemu used to use. So the hack of avoiding the add_buf call will avoid this kick and so break these hosts. I guess we can add a feature bit to detect a new host and so avoid the kick. We are running low on feature bits unfortunately, but just fo testing, could you quantify the difference that this makes using the following patch: diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index cc2f73e..6106017 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -185,11 +185,6 @@ int virtqueue_add_buf_gfp(struct virtqueue *_vq, if (vq->num_free < out + in) { pr_debug("Can't add buf len %i - avail = %i\n", out + in, vq->num_free); - /* FIXME: for historical reasons, we force a notify here if - * there are outgoing parts to the buffer. Presumably the - * host should service the ring ASAP. */ - if (out) - vq->notify(&vq->vq); END_USE(vq); return -ENOSPC; }