From patchwork Wed Aug 30 16:56:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Gardner X-Patchwork-Id: 1827870 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=eKbNtL0s; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=185.125.189.65; helo=lists.ubuntu.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=patchwork.ozlabs.org) Received: from lists.ubuntu.com (lists.ubuntu.com [185.125.189.65]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RbVpC1j9Dz1ygM for ; Thu, 31 Aug 2023 02:57:23 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=lists.ubuntu.com) by lists.ubuntu.com with esmtp (Exim 4.86_2) (envelope-from ) id 1qbOV6-0004Vr-PV; Wed, 30 Aug 2023 16:57:05 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by lists.ubuntu.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1qbOUI-00042t-1h for kernel-team@lists.ubuntu.com; Wed, 30 Aug 2023 16:56:15 +0000 Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 6672C41BF7 for ; Wed, 30 Aug 2023 16:56:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1693414572; bh=ly4cvawGbs7bTg5jyP8pWosXGDm7JzI7DedbTgq8yFc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eKbNtL0sbDpQuWpbYY6oIuDpUM3s5+sz/llVic1kg6O7xjj+AN55IhYRZHC19yi6P XF1gbyFWhOuz/sycvG8nM8yp9rvmGpJjg6Xdk3rhOAs3TdWoT2VeYZHcrztdx7ZgCM xLp9LsSUeFMozS2mBWkxB7Tw8izvPwW0gPZ4NbSg7CrgBzTCLzOeZs6QYAihkO9FW5 5tPNghe989yDjLZgQA9eZT0jZte5rF0X2e+UngL8UNqK40eAupS8xabHP8ODMkXeHZ lSsiTqM3Dnd6AF2cEE0Edb4y/nssvF7n78m+nGp7cQmxU7THI5jASuxMamkKQoVdEq NWh2TrJqg93BA== Received: by mail-io1-f72.google.com with SMTP id ca18e2360f4ac-77b257b9909so124339f.0 for ; Wed, 30 Aug 2023 09:56:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693414571; x=1694019371; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ly4cvawGbs7bTg5jyP8pWosXGDm7JzI7DedbTgq8yFc=; b=IObCtlaC12/3wM7dJ1KaQ1Btn5pJBqYuopnx8y50a2DITTzTa4bvMg0+Cik/vmUgmm U6JxkVKlNXPpqjsfmBS6UIZdKVducYSEniaWzulvhDbZy6DiynkH2G88O97QG2JrbTZ0 L88ZzKM2fsdTx5CY2lVmt4Ay9rX7xiGuk7qMKdpWZe8VCHFIrHZbZm7obTLMbpEL9dg7 rpoTIcIhgLHqLRvyomxsxb/iRsVEanhNGTSxiiiQM+tyYIk8BNpMQB8ylqw5JzsTdRha 79qFE8RpnQeLM5pYA2F5vAb4F7dTdbODSRPmCQaFhqJboo6yEFKZn4jprycIm+9yJzjM u9Tg== X-Gm-Message-State: AOJu0YywY/gJXgxUOgADgklHV/tunWzRC/YFG+C/06MQYLea0Xcfc70X 0KaYhsZdahmNbJwGMTgGkptDvhyS8Vu5mgq/Ash8tKnS+faMh7a2Q/lG1FjvTy/tkhnfl4w+kiB /9E6x8UAdlsbX1nbO5/ZW66j6gTZqnYlu7QyyBKaWIySqmUtyHg== X-Received: by 2002:a5d:8a15:0:b0:795:12b0:c2a1 with SMTP id w21-20020a5d8a15000000b0079512b0c2a1mr3591348iod.10.1693414571074; Wed, 30 Aug 2023 09:56:11 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHAS0PW/nBBC742+NXCaW9dgtyRre2srZh8EcPzyWXFKB4dBRd2VWtxX3mxa4P1I2N3UzJwXA== X-Received: by 2002:a5d:8a15:0:b0:795:12b0:c2a1 with SMTP id w21-20020a5d8a15000000b0079512b0c2a1mr3591336iod.10.1693414570744; Wed, 30 Aug 2023 09:56:10 -0700 (PDT) Received: from smtp.gmail.com (174-045-099-030.res.spectrum.com. [174.45.99.30]) by smtp.gmail.com with ESMTPSA id n23-20020a056602221700b0078714764ca0sm4005758ion.40.2023.08.30.09.56.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 30 Aug 2023 09:56:10 -0700 (PDT) From: Tim Gardner To: kernel-team@lists.ubuntu.com Subject: [PATCH][lunar/jammy linux-azure] net: mana: Fix MANA VF unload when hardware is unresponsive Date: Wed, 30 Aug 2023 10:56:04 -0600 Message-Id: <20230830165605.976845-2-tim.gardner@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230830165605.976845-1-tim.gardner@canonical.com> References: <20230830165605.976845-1-tim.gardner@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Souradeep Chakrabarti BugLink: https://bugs.launchpad.net/bugs/2033531 When unloading the MANA driver, mana_dealloc_queues() waits for the MANA hardware to complete any inflight packets and set the pending send count to zero. But if the hardware has failed, mana_dealloc_queues() could wait forever. Fix this by adding a timeout to the wait. Set the timeout to 120 seconds, which is a somewhat arbitrary value that is more than long enough for functional hardware to complete any sends. Cc: stable@vger.kernel.org Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)") Signed-off-by: Souradeep Chakrabarti Link: https://lore.kernel.org/r/1691576525-24271-1-git-send-email-schakrabarti@linux.microsoft.com Signed-off-by: Jakub Kicinski (cherry picked from commit a7dfeda6fdeccab4c7c3dce9a72c4262b9530c80) Signed-off-by: Tim Gardner --- drivers/net/ethernet/microsoft/mana/mana_en.c | 37 +++++++++++++++++-- 1 file changed, 33 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c index 146b4f99a5d8..de2844bdd36a 100644 --- a/drivers/net/ethernet/microsoft/mana/mana_en.c +++ b/drivers/net/ethernet/microsoft/mana/mana_en.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include @@ -2348,9 +2349,12 @@ int mana_attach(struct net_device *ndev) static int mana_dealloc_queues(struct net_device *ndev) { struct mana_port_context *apc = netdev_priv(ndev); + unsigned long timeout = jiffies + 120 * HZ; struct gdma_dev *gd = apc->ac->gdma_dev; struct mana_txq *txq; + struct sk_buff *skb; int i, err; + u32 tsleep; if (apc->port_is_up) return -EINVAL; @@ -2366,15 +2370,40 @@ static int mana_dealloc_queues(struct net_device *ndev) * to false, but it doesn't matter since mana_start_xmit() drops any * new packets due to apc->port_is_up being false. * - * Drain all the in-flight TX packets + * Drain all the in-flight TX packets. + * A timeout of 120 seconds for all the queues is used. + * This will break the while loop when h/w is not responding. + * This value of 120 has been decided here considering max + * number of queues. */ + for (i = 0; i < apc->num_queues; i++) { txq = &apc->tx_qp[i].txq; - - while (atomic_read(&txq->pending_sends) > 0) - usleep_range(1000, 2000); + tsleep = 1000; + while (atomic_read(&txq->pending_sends) > 0 && + time_before(jiffies, timeout)) { + usleep_range(tsleep, tsleep + 1000); + tsleep <<= 1; + } + if (atomic_read(&txq->pending_sends)) { + err = pcie_flr(to_pci_dev(gd->gdma_context->dev)); + if (err) { + netdev_err(ndev, "flr failed %d with %d pkts pending in txq %u\n", + err, atomic_read(&txq->pending_sends), + txq->gdma_txq_id); + } + break; + } } + for (i = 0; i < apc->num_queues; i++) { + txq = &apc->tx_qp[i].txq; + while ((skb = skb_dequeue(&txq->pending_skbs))) { + mana_unmap_skb(skb, apc); + dev_kfree_skb_any(skb); + } + atomic_set(&txq->pending_sends, 0); + } /* We're 100% sure the queues can no longer be woken up, because * we're sure now mana_poll_tx_cq() can't be running. */