From patchwork Thu Jan 31 02:47:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li RongQing X-Patchwork-Id: 1033895 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=baidu.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43ql6S0hCHz9s4s for ; Thu, 31 Jan 2019 13:48:31 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 9FC5F3BC7; Thu, 31 Jan 2019 02:48:27 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id B75E53BBF for ; Thu, 31 Jan 2019 02:47:07 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from tc-sys-mailedm01.tc.baidu.com (mx134-tc.baidu.com [61.135.168.134]) by smtp1.linuxfoundation.org (Postfix) with ESMTP id 2AF298D for ; Thu, 31 Jan 2019 02:47:07 +0000 (UTC) Received: from localhost (cp01-cos-dev01.cp01.baidu.com [10.92.119.46]) by tc-sys-mailedm01.tc.baidu.com (Postfix) with ESMTP id E3F1F2040055 for ; Thu, 31 Jan 2019 10:47:03 +0800 (CST) From: Li RongQing To: dev@openvswitch.org Date: Thu, 31 Jan 2019 10:47:03 +0800 Message-Id: <1548902823-18719-1-git-send-email-lirongqing@baidu.com> X-Mailer: git-send-email 1.7.1 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH] netdev-dpdk: shrink critical region under tx_q[qid].tx_lock X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org netdev_dpdk_filter_packet_len() does not need to be protected by tx_q[].tx_lock, and tx_q[].tx_lock can not protect it too, same to netdev_dpdk_qos_run so move them out of this lock to improve the scalability Signed-off-by: Li RongQing --- lib/netdev-dpdk.c | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 4bf0ca9e8..bf4918e2c 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -2333,15 +2333,15 @@ __netdev_dpdk_vhost_send(struct netdev *netdev, int qid, goto out; } - rte_spinlock_lock(&dev->tx_q[qid].tx_lock); - cnt = netdev_dpdk_filter_packet_len(dev, cur_pkts, cnt); /* Check has QoS has been configured for the netdev */ cnt = netdev_dpdk_qos_run(dev, cur_pkts, cnt, true); dropped = total_pkts - cnt; + rte_spinlock_lock(&dev->tx_q[qid].tx_lock); + + int vhost_qid = qid * VIRTIO_QNUM + VIRTIO_RXQ; do { - int vhost_qid = qid * VIRTIO_QNUM + VIRTIO_RXQ; unsigned int tx_pkts; tx_pkts = rte_vhost_enqueue_burst(vid, vhost_qid, cur_pkts, cnt); @@ -2462,15 +2462,20 @@ netdev_dpdk_send__(struct netdev_dpdk *dev, int qid, return; } - if (OVS_UNLIKELY(concurrent_txq)) { - qid = qid % dev->up.n_txq; - rte_spinlock_lock(&dev->tx_q[qid].tx_lock); - } - if (OVS_UNLIKELY(batch->packets[0]->source != DPBUF_DPDK)) { struct netdev *netdev = &dev->up; + if (OVS_UNLIKELY(concurrent_txq)) { + qid = qid % dev->up.n_txq; + rte_spinlock_lock(&dev->tx_q[qid].tx_lock); + } + dpdk_do_tx_copy(netdev, qid, batch); + + if (OVS_UNLIKELY(concurrent_txq)) { + rte_spinlock_unlock(&dev->tx_q[qid].tx_lock); + } + dp_packet_delete_batch(batch, true); } else { int tx_cnt, dropped; @@ -2481,8 +2486,17 @@ netdev_dpdk_send__(struct netdev_dpdk *dev, int qid, tx_cnt = netdev_dpdk_qos_run(dev, pkts, tx_cnt, true); dropped = batch_cnt - tx_cnt; + if (OVS_UNLIKELY(concurrent_txq)) { + qid = qid % dev->up.n_txq; + rte_spinlock_lock(&dev->tx_q[qid].tx_lock); + } + dropped += netdev_dpdk_eth_tx_burst(dev, qid, pkts, tx_cnt); + if (OVS_UNLIKELY(concurrent_txq)) { + rte_spinlock_unlock(&dev->tx_q[qid].tx_lock); + } + if (OVS_UNLIKELY(dropped)) { rte_spinlock_lock(&dev->stats_lock); dev->stats.tx_dropped += dropped; @@ -2490,9 +2504,6 @@ netdev_dpdk_send__(struct netdev_dpdk *dev, int qid, } } - if (OVS_UNLIKELY(concurrent_txq)) { - rte_spinlock_unlock(&dev->tx_q[qid].tx_lock); - } } static int