From patchwork Tue Jul 25 10:27:48 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gao Zhenyu X-Patchwork-Id: 793297 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="cgYjw5pw"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3xGvbT47w4z9s4q for ; Tue, 25 Jul 2017 20:28:36 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 1CE2C900; Tue, 25 Jul 2017 10:28:22 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 617C08E3 for ; Tue, 25 Jul 2017 10:28:20 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pf0-f194.google.com (mail-pf0-f194.google.com [209.85.192.194]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id F3E773CF for ; Tue, 25 Jul 2017 10:28:19 +0000 (UTC) Received: by mail-pf0-f194.google.com with SMTP id 1so4524893pfi.3 for ; Tue, 25 Jul 2017 03:28:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=rDAnWRovKZBXk5Ytl6hGvdDsQ3vRlTUUlUb00hxZWsY=; b=cgYjw5pwovyHNGwVzZycsynYBxrU8wsHixykjgqGM+HuvxOaFlqa3ch8T1Jot4G4FO 83x56RZ3235hkgrL/0g7q+QHsiNfMJRe6IhiZhFDjrJxnR/aSYFzFl7cbiadcR2BEK2u YPnVtOnR7dilYZw46P1LMNwAu1IWhewUYjFgDPTn4HSg78UZl3CcnlrAmnz9cSSrUANT IjrgUO/q9NMvrYn53DxcxXmrsfJk1QjsyQTYXvLrIe0riLGxSd2WH+dh2VOaiXpmY6xG F3YPLHaZb3fwDAKC2NYgTo8lh33gGr8AGFbNfma6s+ADZ74v2oXIZszylKNbfR/Pffc0 Af4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=rDAnWRovKZBXk5Ytl6hGvdDsQ3vRlTUUlUb00hxZWsY=; b=cx9UxEBa1OstbKh0z0Uhpj75yLIcYkWhxhvPLDhcVzlJlpHtoxBqbp3//rHqiP/BPz H08oTEl75MyyHMaXshdE7Vpd23CzeZSIFh8CVP50NYKJMJdEGSopNVDE6jygHuEc0CRM 2l11QdXoiOFrSBxA+EvV1x42uzS2QW+vXZE5c0ygo1lRZOwqI65mQPUDKFfm68FuaxI6 od041cUzFpLtiiwJtQk+2ZpY09EveWybnhOKmlllwkObzWMNd8jy47iDj0dcy+Nx2eoI xH1DYqGDTTRapZmeEW6BjSCnyOOqnx3iejGlb6n9PgTJ95j5JjdAweSsn/xjv8Yg+UzA XiEg== X-Gm-Message-State: AIVw110NxYg+NcbWWUKYtVpyPNVuFP0aDxQeE2SB6uwONffkmd1KwlmR HTU7rGssRb0afA== X-Received: by 10.99.6.209 with SMTP id 200mr18493563pgg.227.1500978499569; Tue, 25 Jul 2017 03:28:19 -0700 (PDT) Received: from vultr.guest ([45.77.33.88]) by smtp.gmail.com with ESMTPSA id i67sm22914243pfc.122.2017.07.25.03.28.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Jul 2017 03:28:18 -0700 (PDT) From: Zhenyu Gao To: blp@ovn.org, sugesh.chandran@intel.com, dev@openvswitch.org Date: Tue, 25 Jul 2017 10:27:48 +0000 Message-Id: <20170725102748.501-1-sysugaozhenyu@gmail.com> X-Mailer: git-send-email 2.11.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE, RCVD_IN_SORBS_SPAM autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v1] netdev-dpdk: Execute QoS Checking before copying to mbuf X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org In dpdk_do_tx_copy function, all packets were copied to mbuf first, but QoS checking may drop some of them. Move the QoS checking in front of copying data to mbuf, it helps to reduce useless copy. Signed-off-by: Zhenyu Gao --- lib/netdev-dpdk.c | 55 ++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 36 insertions(+), 19 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index ea17b97..bb8bd8f 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -258,7 +258,7 @@ struct dpdk_qos_ops { * For all QoS implementations it should always be non-null. */ int (*qos_run)(struct qos_conf *qos_conf, struct rte_mbuf **pkts, - int pkt_cnt); + int pkt_cnt, bool may_steal); }; /* dpdk_qos_ops for each type of user space QoS implementation */ @@ -1438,7 +1438,8 @@ netdev_dpdk_policer_pkt_handle(struct rte_meter_srtcm *meter, static int netdev_dpdk_policer_run(struct rte_meter_srtcm *meter, - struct rte_mbuf **pkts, int pkt_cnt) + struct rte_mbuf **pkts, int pkt_cnt, + bool may_steal) { int i = 0; int cnt = 0; @@ -1454,7 +1455,9 @@ netdev_dpdk_policer_run(struct rte_meter_srtcm *meter, } cnt++; } else { - rte_pktmbuf_free(pkt); + if (may_steal) { + rte_pktmbuf_free(pkt); + } } } @@ -1463,12 +1466,13 @@ netdev_dpdk_policer_run(struct rte_meter_srtcm *meter, static int ingress_policer_run(struct ingress_policer *policer, struct rte_mbuf **pkts, - int pkt_cnt) + int pkt_cnt, bool may_steal) { int cnt = 0; rte_spinlock_lock(&policer->policer_lock); - cnt = netdev_dpdk_policer_run(&policer->in_policer, pkts, pkt_cnt); + cnt = netdev_dpdk_policer_run(&policer->in_policer, pkts, + pkt_cnt, may_steal); rte_spinlock_unlock(&policer->policer_lock); return cnt; @@ -1572,7 +1576,7 @@ netdev_dpdk_vhost_rxq_recv(struct netdev_rxq *rxq, dropped = nb_rx; nb_rx = ingress_policer_run(policer, (struct rte_mbuf **) batch->packets, - nb_rx); + nb_rx, true); dropped -= nb_rx; } @@ -1609,7 +1613,7 @@ netdev_dpdk_rxq_recv(struct netdev_rxq *rxq, struct dp_packet_batch *batch) dropped = nb_rx; nb_rx = ingress_policer_run(policer, (struct rte_mbuf **) batch->packets, - nb_rx); + nb_rx, true); dropped -= nb_rx; } @@ -1627,13 +1631,13 @@ netdev_dpdk_rxq_recv(struct netdev_rxq *rxq, struct dp_packet_batch *batch) static inline int netdev_dpdk_qos_run(struct netdev_dpdk *dev, struct rte_mbuf **pkts, - int cnt) + int cnt, bool may_steal) { struct qos_conf *qos_conf = ovsrcu_get(struct qos_conf *, &dev->qos_conf); if (qos_conf) { rte_spinlock_lock(&qos_conf->lock); - cnt = qos_conf->ops->qos_run(qos_conf, pkts, cnt); + cnt = qos_conf->ops->qos_run(qos_conf, pkts, cnt, may_steal); rte_spinlock_unlock(&qos_conf->lock); } @@ -1707,7 +1711,7 @@ __netdev_dpdk_vhost_send(struct netdev *netdev, int qid, cnt = netdev_dpdk_filter_packet_len(dev, cur_pkts, cnt); /* Check has QoS has been configured for the netdev */ - cnt = netdev_dpdk_qos_run(dev, cur_pkts, cnt); + cnt = netdev_dpdk_qos_run(dev, cur_pkts, cnt, true); dropped = total_pkts - cnt; do { @@ -1753,10 +1757,22 @@ dpdk_do_tx_copy(struct netdev *netdev, int qid, struct dp_packet_batch *batch) #endif struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); struct rte_mbuf *pkts[PKT_ARRAY_SIZE]; + int txcnt_eth = batch->count; int dropped = 0; int newcnt = 0; int i; + if (dev->type != DPDK_DEV_VHOST) { + /* Check if QoS has been configured for this netdev. */ + txcnt_eth = netdev_dpdk_qos_run(dev, + (struct rte_mbuf **)batch->packets, + txcnt_eth, false); + if (txcnt_eth == 0) { + dropped = batch->count; + goto out; + } + } + dp_packet_batch_apply_cutlen(batch); for (i = 0; i < batch->count; i++) { @@ -1785,21 +1801,20 @@ dpdk_do_tx_copy(struct netdev *netdev, int qid, struct dp_packet_batch *batch) rte_pktmbuf_pkt_len(pkts[newcnt]) = size; newcnt++; + if (dev->type != DPDK_DEV_VHOST && newcnt >= txcnt_eth) { + dropped += batch->count - i - 1; + break; + } } if (dev->type == DPDK_DEV_VHOST) { __netdev_dpdk_vhost_send(netdev, qid, (struct dp_packet **) pkts, newcnt); } else { - unsigned int qos_pkts = newcnt; - - /* Check if QoS has been configured for this netdev. */ - newcnt = netdev_dpdk_qos_run(dev, pkts, newcnt); - - dropped += qos_pkts - newcnt; dropped += netdev_dpdk_eth_tx_burst(dev, qid, pkts, newcnt); } +out: if (OVS_UNLIKELY(dropped)) { rte_spinlock_lock(&dev->stats_lock); dev->stats.tx_dropped += dropped; @@ -1852,7 +1867,7 @@ netdev_dpdk_send__(struct netdev_dpdk *dev, int qid, dp_packet_batch_apply_cutlen(batch); cnt = netdev_dpdk_filter_packet_len(dev, pkts, cnt); - cnt = netdev_dpdk_qos_run(dev, pkts, cnt); + cnt = netdev_dpdk_qos_run(dev, pkts, cnt, true); dropped = batch->count - cnt; dropped += netdev_dpdk_eth_tx_burst(dev, qid, pkts, cnt); @@ -3061,13 +3076,15 @@ egress_policer_qos_is_equal(const struct qos_conf *conf, } static int -egress_policer_run(struct qos_conf *conf, struct rte_mbuf **pkts, int pkt_cnt) +egress_policer_run(struct qos_conf *conf, struct rte_mbuf **pkts, int pkt_cnt, + bool may_steal) { int cnt = 0; struct egress_policer *policer = CONTAINER_OF(conf, struct egress_policer, qos_conf); - cnt = netdev_dpdk_policer_run(&policer->egress_meter, pkts, pkt_cnt); + cnt = netdev_dpdk_policer_run(&policer->egress_meter, pkts, + pkt_cnt, may_steal); return cnt; }