From patchwork Tue May 28 20:15:08 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shawn Bohrer X-Patchwork-Id: 246955 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id DE6952C030A for ; Wed, 29 May 2013 06:15:25 +1000 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755937Ab3E1UPU (ORCPT ); Tue, 28 May 2013 16:15:20 -0400 Received: from mail-oa0-f49.google.com ([209.85.219.49]:35388 "EHLO mail-oa0-f49.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755587Ab3E1UPR (ORCPT ); Tue, 28 May 2013 16:15:17 -0400 Received: by mail-oa0-f49.google.com with SMTP id k14so10724553oag.36 for ; Tue, 28 May 2013 13:15:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=9UDuWWkdXcYLeQ+po26gz37RVkGZ7bpnDcv7Sgm2Qm4=; b=Dl3y0GH+5/qgS89lTGrwxeSUPTLRie9SznR50G3wb+0+YHUKddDVy5t7cx0j3YQ3C0 +Fpm9uKSJ1qAocamAQBhcSDVOcPwbzed0aLVdoxDzR4COi8lvAwfLbWoxKZKh4MRr1RJ LUn0LUSm2/vYTpTDlfaslnlrB6WlwL9svQBAh34A85rqqyFY/O5VKS+GWtUD/SFoBcft J5ENNUqr5/hBD5Frpb3tVq/5trSP5KagsGG/eMPBH0nsVBuygQ75PN0ITViUCtaylJoo rKhCFqObnUhHp8aUIjAS1rI71r+ZyIlwepT3MD446jMVnKL3nkU0R5bFQUBwf3hGTCaV YzwQ== X-Received: by 10.182.46.230 with SMTP id y6mr3145107obm.79.1369772116418; Tue, 28 May 2013 13:15:16 -0700 (PDT) Received: from sbohrermbp13-local.rgmadvisors.com ([173.227.92.65]) by mx.google.com with ESMTPSA id i2sm6709049obz.11.2013.05.28.13.15.15 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 28 May 2013 13:15:15 -0700 (PDT) Date: Tue, 28 May 2013 15:15:08 -0500 From: Shawn Bohrer To: Or Gerlitz Cc: netdev@vger.kernel.org, Hadar Hen Zion , Amir Vadai Subject: Re: 3.10.0-rc2 mlx4 not receiving packets for some multicast groups Message-ID: <20130528201508.GA6409@sbohrermbp13-local.rgmadvisors.com> References: <20130524154931.GA9245@sbohrermbp13-local.rgmadvisors.com> <20130524163446.GC9245@sbohrermbp13-local.rgmadvisors.com> <20130525151347.GB25744@lintop.rgmadvisors.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20130525151347.GB25744@lintop.rgmadvisors.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Sat, May 25, 2013 at 10:13:47AM -0500, Shawn Bohrer wrote: > On Sat, May 25, 2013 at 06:41:05AM +0300, Or Gerlitz wrote: > > On Fri, May 24, 2013 at 7:34 PM, Shawn Bohrer wrote: > > > On Fri, May 24, 2013 at 10:49:31AM -0500, Shawn Bohrer wrote: > > > > I just started testing the 3.10 kernel, previously we were on 3.4 so > > > > there is a fairly large jump. I've additionally applied the following > > > > four patches to the 3.10.0-rc2 kernel that I'm testing: > > > > > > > > https://patchwork.kernel.org/patch/2484651/ > > > > https://patchwork.kernel.org/patch/2484671/ > > > > https://patchwork.kernel.org/patch/2484681/ > > > > https://patchwork.kernel.org/patch/2484641/ > > > > > > >> I don't know if those patches are related to my issues or not but I > > >> plan on trying to reproduce without them soon. > > > > > I've reverted the four patches above from my test kernel and still see > > > the issue so they don't appear to be the cause. > > > > Hi Shawn, > > > > So 3.4 works, 3.10-rc2 breaks? its indeed a fairly large gap, maybe > > try to bisec that? just to make sure, did use touch any mlx4 > > non-default config? specifically did you turn DMFS (Device Managed > > Flow Steering) on using the set the mlx4_core module param of > > log_num_mgm_entry_size or you were using B0 steering (the default)? > > Initially my goal is to sanity check 3.10 before I start playing with > the knobs, so I haven't explicitly changed any new mlx4 settings yet. > We do however set some non-default values but I'm doing that on both > kernels: > > mlx4_core log_num_vlan=7 > mlx4_en pfctx=0xff pfcrx=0xff Naturally I was wrong and we set more than the above non-default values. We additionally set high_rate_steer=1 on mlx4_core. As you may know this parameter isn't currently available in the upstream driver, so I've been carrying the following patch in my 3.4 and 3.10 trees: --- drivers/net/ethernet/mellanox/mlx4/main.c | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c index 0d32a82..7808e4a 100644 --- a/drivers/net/ethernet/mellanox/mlx4/main.c +++ b/drivers/net/ethernet/mellanox/mlx4/main.c @@ -71,6 +71,11 @@ static int msi_x = 1; module_param(msi_x, int, 0444); MODULE_PARM_DESC(msi_x, "attempt to use MSI-X if nonzero"); +static int high_rate_steer; +module_param(high_rate_steer, int, 0444); +MODULE_PARM_DESC(high_rate_steer, "Enable steering mode for higher packet rate" + " (default off)"); + #else /* CONFIG_PCI_MSI */ #define msi_x (0) @@ -288,6 +293,11 @@ static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) if (mlx4_is_mfunc(dev)) dev->caps.flags &= ~MLX4_DEV_CAP_FLAG_SENSE_SUPPORT; + if (high_rate_steer && !mlx4_is_mfunc(dev)) { + dev->caps.flags &= ~MLX4_DEV_CAP_FLAG_VEP_UC_STEER; + dev->caps.flags &= ~MLX4_DEV_CAP_FLAG_VEP_MC_STEER; + } + dev->caps.log_num_macs = log_num_mac; dev->caps.log_num_vlans = MLX4_LOG_NUM_VLANS; dev->caps.log_num_prios = use_prio ? 3 : 0;