From patchwork Fri Jan 14 06:54:15 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 78864 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 693B8B7043 for ; Fri, 14 Jan 2011 17:54:53 +1100 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752829Ab1ANGyt (ORCPT ); Fri, 14 Jan 2011 01:54:49 -0500 Received: from mx1.redhat.com ([209.132.183.28]:50729 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752479Ab1ANGyr (ORCPT ); Fri, 14 Jan 2011 01:54:47 -0500 Received: from int-mx12.intmail.prod.int.phx2.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0E6sY1x011395 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 14 Jan 2011 01:54:34 -0500 Received: from redhat.com (vpn-200-19.tlv.redhat.com [10.35.200.19]) by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id p0E6sT4I025234; Fri, 14 Jan 2011 01:54:30 -0500 Date: Fri, 14 Jan 2011 08:54:15 +0200 From: "Michael S. Tsirkin" To: Simon Horman Cc: Jesse Gross , Eric Dumazet , Rusty Russell , virtualization@lists.linux-foundation.org, dev@openvswitch.org, virtualization@lists.osdl.org, netdev@vger.kernel.org, kvm@vger.kernel.org Subject: Re: Flow Control and Port Mirroring Revisited Message-ID: <20110114065415.GA30300@redhat.com> References: <1294309362.3074.11.camel@edumazet-laptop> <20110106124439.GA17004@verge.net.au> <20110107012356.GA1257@verge.net.au> <20110110093155.GB13420@verge.net.au> <20110113064718.GA17905@verge.net.au> <20110113234135.GC8426@verge.net.au> <20110114045818.GA29738@redhat.com> <20110114063528.GB10957@verge.net.au> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20110114063528.GB10957@verge.net.au> User-Agent: Mutt/1.5.21 (2010-09-15) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Fri, Jan 14, 2011 at 03:35:28PM +0900, Simon Horman wrote: > On Fri, Jan 14, 2011 at 06:58:18AM +0200, Michael S. Tsirkin wrote: > > On Fri, Jan 14, 2011 at 08:41:36AM +0900, Simon Horman wrote: > > > On Thu, Jan 13, 2011 at 10:45:38AM -0500, Jesse Gross wrote: > > > > On Thu, Jan 13, 2011 at 1:47 AM, Simon Horman wrote: > > > > > On Mon, Jan 10, 2011 at 06:31:55PM +0900, Simon Horman wrote: > > > > >> On Fri, Jan 07, 2011 at 10:23:58AM +0900, Simon Horman wrote: > > > > >> > On Thu, Jan 06, 2011 at 05:38:01PM -0500, Jesse Gross wrote: > > > > >> > > > > > >> > [ snip ] > > > > >> > > > > > > >> > > I know that everyone likes a nice netperf result but I agree with > > > > >> > > Michael that this probably isn't the right question to be asking.  I > > > > >> > > don't think that socket buffers are a real solution to the flow > > > > >> > > control problem: they happen to provide that functionality but it's > > > > >> > > more of a side effect than anything.  It's just that the amount of > > > > >> > > memory consumed by packets in the queue(s) doesn't really have any > > > > >> > > implicit meaning for flow control (think multiple physical adapters, > > > > >> > > all with the same speed instead of a virtual device and a physical > > > > >> > > device with wildly different speeds).  The analog in the physical > > > > >> > > world that you're looking for would be Ethernet flow control. > > > > >> > > Obviously, if the question is limiting CPU or memory consumption then > > > > >> > > that's a different story. > > > > >> > > > > > >> > Point taken. I will see if I can control CPU (and thus memory) consumption > > > > >> > using cgroups and/or tc. > > > > >> > > > > >> I have found that I can successfully control the throughput using > > > > >> the following techniques > > > > >> > > > > >> 1) Place a tc egress filter on dummy0 > > > > >> > > > > >> 2) Use ovs-ofctl to add a flow that sends skbs to dummy0 and then eth1, > > > > >>    this is effectively the same as one of my hacks to the datapath > > > > >>    that I mentioned in an earlier mail. The result is that eth1 > > > > >>    "paces" the connection. > > > > This is actually a bug. This means that one slow connection will affect > > fast ones. I intend to change the default for qemu to sndbuf=0 : this > > will fix it but break your "pacing". So pls do not count on this > > behaviour. > > Do you have a patch I could test? You can (and users already can) just run qemu with sndbuf=0. But if you like, below. > > > > > Further to this, I wonder if there is any interest in providing > > > > > a method to switch the action order - using ovs-ofctl is a hack imho - > > > > > and/or switching the default action order for mirroring. > > > > > > > > I'm not sure that there is a way to do this that is correct in the > > > > generic case. It's possible that the destination could be a VM while > > > > packets are being mirrored to a physical device or we could be > > > > multicasting or some other arbitrarily complex scenario. Just think > > > > of what a physical switch would do if it has ports with two different > > > > speeds. > > > > > > Yes, I have considered that case. And I agree that perhaps there > > > is no sensible default. But perhaps we could make it configurable somehow? > > > > The fix is at the application level. Run netperf with -b and -w flags to > > limit the speed to a sensible value. > > Perhaps I should have stated my goals more clearly. > I'm interested in situations where I don't control the application. Well an application that streams UDP without any throttling at the application level will break on a physical network, right? So I am not sure why should one try to make it work on the virtual one. But let's assume that you do want to throttle the guest for reasons such as QOS. The proper approach seems to be to throttle the sender, not have a dummy throttled receiver "pacing" it. Place the qemu process in the correct net_cls cgroup, set the class id and apply a rate limit? --- -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/net/tap-linux.c b/net/tap-linux.c index f7aa904..0dbcdd4 100644 --- a/net/tap-linux.c +++ b/net/tap-linux.c @@ -87,7 +87,7 @@ int tap_open(char *ifname, int ifname_size, int *vnet_hdr, int vnet_hdr_required * Ethernet NICs generally have txqueuelen=1000, so 1Mb is * a good default, given a 1500 byte MTU. */ -#define TAP_DEFAULT_SNDBUF 1024*1024 +#define TAP_DEFAULT_SNDBUF 0 int tap_set_sndbuf(int fd, QemuOpts *opts) {