From patchwork Wed Sep 23 09:44:01 2009 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: hyl X-Patchwork-Id: 34122 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.176.167]) by ozlabs.org (Postfix) with ESMTP id C704CB7B70 for ; Wed, 23 Sep 2009 19:44:04 +1000 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751519AbZIWJn7 (ORCPT ); Wed, 23 Sep 2009 05:43:59 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1751210AbZIWJn7 (ORCPT ); Wed, 23 Sep 2009 05:43:59 -0400 Received: from mail-qy0-f188.google.com ([209.85.221.188]:34014 "EHLO mail-qy0-f188.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750911AbZIWJn6 (ORCPT ); Wed, 23 Sep 2009 05:43:58 -0400 X-Greylist: delayed 7420 seconds by postgrey-1.27 at vger.kernel.org; Wed, 23 Sep 2009 05:43:57 EDT Received: by qyk26 with SMTP id 26so458517qyk.5 for ; Wed, 23 Sep 2009 02:44:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:cc:content-type; bh=JHwGdzqnpbFRnxzWH//Mv8d2fIdUtYh6RlIGYQX85Yw=; b=h+33bjhZ6SWMvzc06M7jVgbGpIK2703Yy1sKmkfr7emQWB7X9OnbAzl5kCcoG2oCIP DFJqB2gs0UyhxHPkUZLXg4xE34hMsQjkeTAsum7o4QDBzKJ8HtI6W3Bo2Aio3u0nSWM4 VPdTXc66d5OG7/KdNPwgeUdu+2Y7WDzTT1Z0I= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type; b=s1j+ZSd3oUIb96v8uLfHYSPlIo8hwACitdoMQSoLXDAmt8plEHjFoVwkupa13qlvqC 9Q663Di4DpqLbIRyyDc4FbkU+w5Uy/5oqEXRtgF9PK1lIFpS+JkipM+ofcnnHChDP2fD Rda7qSmUTQkP3HJVMQaXYTwSxOmrg4owrawt0= MIME-Version: 1.0 Received: by 10.229.36.195 with SMTP id u3mr788185qcd.61.1253699041586; Wed, 23 Sep 2009 02:44:01 -0700 (PDT) Date: Wed, 23 Sep 2009 17:44:01 +0800 Message-ID: <505766fa0909230244r3737379s1748a8f7c181f366@mail.gmail.com> Subject: Re: Linux guest domain with two vnets , the dirty quick fix From: hyl To: sparclinux@vger.kernel.org Cc: linux-kernel@vger.kernel.org, David Miller Sender: sparclinux-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org 1. add the debug code: struct ldc_channel *lp = dev_id; @@ -892,9 +894,10 @@ handshake_complete: } out: + spin_lock_irqsave(&event_lock, flags);//serialize all events spin_unlock_irqrestore(&lp->lock, flags); - send_events(lp, event_mask); + spin_unlock_irqrestore(&event_lock, flags); return IRQ_HANDLED; } --- a/drivers/net/sunvnet.c +++ b/drivers/net/sunvnet.c @@ -540,6 +540,11 @@ static void vnet_event(void *arg, int event) } if (err == 0) break; + { + long stackdiff; + viodbg(DATA, "** vio:%llx, vio->lp:%llx, stack:%llx\n", + &vio, &vio->lp, &stackdiff); + } viodbg(DATA, "TAG [%02x:%02x:%04x:%08x]\n", msgbuf.tag.type, msgbuf.tag.stype, 2. the output from the below output, we see a re-enter to vnet_event happend. the vio-hardwre report 2 interrupt for same packet/or msg( why?) vio: ID[0] vnet_walk_rx start[00000009] end[00000008] vio: ID[0] vio_walk_rx_one desc[02:00:0000003c:00000001:842800:800] vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:845000:800] vio: ID[0] ** vio:fffff8003f9cafa8, vio->lp:fffff8003ebf0008, stack:fffff8003f9cafa0 << same ldc vio vio: ID[0] ** vio:fffff8003f9cafa8, vio->lp:fffff8003ebf0008, stack:fffff8003f9ccc << with diffrent run stack afa0 vio: ID[0] TAG [02:01:0042:0d4becc7] vio: ID[0] TAG [02:01:0042:0d4becc7] ........... vio: ID[0] vnet_walk_rx start[0000000a] end[00000009] vio: ID[0] vio_walk_rx_one desc[02:00:0000003c:00000001:845000:800] vio: ID[0] vio_walk_rx_one desc[02:00:0000003c:00000001:845000:800] vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:847800:800] vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:847800:800] vio: ID[0] ** vio:fffff8003f9cb588, vio->lp:fffff8003ebf0008, stack:fffff8003f9cb580 vio: ID[0] ** vio:fffff8003f9cb588, vio->lp:fffff8003ebf0008, stack:fffff8003f9ccc 3. where need protection the reason is : static irqreturn_t ldc_rx(int irq, void *dev_id){ out: spin_unlock_irqrestore(&lp->lock, flags); << here run with out any lock send_events(lp, event_mask); 4. the dirty fix, welcome fain grained lock implementation suggestion: From f24f76cc7b9bba04fcbe0406d8a1ada959c0bc4a Mon Sep 17 00:00:00 2001 From: Yongli He Date: Wed, 23 Sep 2009 17:42:16 +0800 Subject: [PATCH] Sun ldom vnet driver dead lock static irqreturn_t ldc_rx(int irq, void *dev_id){ ... out: spin_unlock_irqrestore(&lp->lock, flags); << here run with out any lock send_events(lp, event_mask); //if 2 vnet attach to same vswitch, ldom //will report 2 irq then lead to dead lock //on the vio->lock or lp->lock } this version fix just serialize all the event. Signed-off-by: Yongli He --- arch/sparc64/kernel/ldc.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/arch/sparc64/kernel/ldc.c b/arch/sparc64/kernel/ldc.c index a6b75cd..8d7c9b5 100644 --- a/arch/sparc64/kernel/ldc.c +++ b/arch/sparc64/kernel/ldc.c @@ -785,6 +785,8 @@ static void send_events(struct ldc_channel *lp, unsigned int event_mask) lp->cfg.event(lp->event_arg, LDC_EVENT_DATA_READY); } +static spinlock_t event_lock=__SPIN_LOCK_UNLOCKED("ldc_event lock"); + static irqreturn_t ldc_rx(int irq, void *dev_id) {