From patchwork Mon Jan 15 10:20:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Maximets X-Patchwork-Id: 860742 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=samsung.com header.i=@samsung.com header.b="Cz/D6qul"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zKqCw46tQz9s9Y for ; Mon, 15 Jan 2018 21:22:20 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 0E0B3EB6; Mon, 15 Jan 2018 10:21:19 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 483A4EB1 for ; Mon, 15 Jan 2018 10:21:18 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id B413614E for ; Mon, 15 Jan 2018 10:21:17 +0000 (UTC) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20180115102115euoutp025631ae090cb3113f382d736aff52edda~J9JY02i7T0208202082euoutp02d; Mon, 15 Jan 2018 10:21:15 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20180115102115euoutp025631ae090cb3113f382d736aff52edda~J9JY02i7T0208202082euoutp02d DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1516011675; bh=zCQRHTfTbn6yAQdPtiLZFjOa1qwqPNQ0Y00qbik+C/8=; h=From:To:Cc:Subject:Date:In-reply-to:References:From; b=Cz/D6qulBWW5XH3icZiKAm7SJqJwlFXws/4XZ/PHLai9NMhT/rDboLHGjbdXaJqzw NPBOiQVrO+oehdMA8dCzlsCAUCaAdLokaSAcyEIK8u291UN9VVc1NKwwXFY5tCNrDQ op5nqnSbC0RA02huT/OzC8enB6T3eE2PnxY9UdtY= Received: from eusmges2.samsung.com (unknown [203.254.199.241]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20180115102114eucas1p29a12d38330a03dfca775a0bb4113dc27~J9JYK8LfJ0890708907eucas1p2l; Mon, 15 Jan 2018 10:21:14 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges2.samsung.com (EUCPMTA) with SMTP id 9A.A0.12907.A908C5A5; Mon, 15 Jan 2018 10:21:14 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20180115102114eucas1p1368de6d50059f1c885ddf02acfade43e~J9JXk9ZIq2458524585eucas1p1S; Mon, 15 Jan 2018 10:21:14 +0000 (GMT) X-AuditID: cbfec7f1-f793a6d00000326b-53-5a5c809a8cb1 Received: from eusync4.samsung.com ( [203.254.199.214]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id C5.BD.20118.A908C5A5; Mon, 15 Jan 2018 10:21:14 +0000 (GMT) Received: from imaximets.rnd.samsung.ru ([106.109.129.180]) by eusync4.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0P2L00JTGE31YI80@eusync4.samsung.com>; Mon, 15 Jan 2018 10:21:14 +0000 (GMT) From: Ilya Maximets To: ovs-dev@openvswitch.org Date: Mon, 15 Jan 2018 13:20:52 +0300 Message-id: <1516011655-15669-3-git-send-email-i.maximets@samsung.com> X-Mailer: git-send-email 2.7.4 In-reply-to: <1516011655-15669-1-git-send-email-i.maximets@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrHIsWRmVeSWpSXmKPExsWy7djPc7qzGmKiDJ6vU7FY/YvTYuczZYsd a5cyWrT0z2S2uHPlJ5vFtM+32S2utP9kt9j48CyrxZHvpxkt1h76wG4x99NzRgduj19fr7J5 LN7zksnj2c3/jB7v9wG5fVtWMQawRnHZpKTmZJalFunbJXBlXP+mV/BBtaLraWAD4wmZLkZO DgkBE4ldZ34zQdhiEhfurWcDsYUEljJKPFwn1sXIBWR/ZpR4cPs7UIIDrOH7OwOI+DJGiev7 F7FAOM1MEu1zLjKDdLMJ6EicWn2EEcQWEZCWeN37hhXEZhbYyiyxsE8NxBYWcJHofvaDFWQo i4CqxLpZmiBhXgE3ic9nvrFAHCQncfNcJ9hITgF3iY3PVjGC7JIQWMMmMeHvS1aIIheJxzcW skHYwhKvjm9hh7BlJC5P7maBaGhmlGhYdQmqewKjxJfm5VA/20ucunmVCeI6PolJ26YzQ7zJ K9HRJgRR4iHxat8rqAWOEhePLGCF+HgWo8SH1a3MExilFzAyrGIUSS0tzk1PLTbSK07MLS7N S9dLzs/dxAiM5dP/jn/cwfj+hNUhRgEORiUe3ohl0VFCrIllxZW5hxglOJiVRHgbg2OihHhT EiurUovy44tKc1KLDzFKc7AoifPaRrVFCgmkJ5akZqemFqQWwWSZODilGhiPpWour/joZ5Hx w2C990zNA4/eGMxk2vD/xJne6hyf9pbZT6Y9cRb6nFBSscr8j6xwN0Oym2ps14m1jP38HNcn Z1nd+xei+0Zrv9y7C0nxRo+POPy2aa9Qee4qYf6YJ81lgvEevePuElNCW0//tU5SXjSnJXe+ mLBo62mPK9H72ya9kg6oua/EUpyRaKjFXFScCADohUeh4QIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrELMWRmVeSWpSXmKPExsVy+t/xa7qzGmKiDA7f5LZY/YvTYuczZYsd a5cyWrT0z2S2uHPlJ5vFtM+32S2utP9kt9j48CyrxZHvpxkt1h76wG4x99NzRgduj19fr7J5 LN7zksnj2c3/jB7v9wG5fVtWMQawRnHZpKTmZJalFunbJXBlXP+mV/BBtaLraWAD4wmZLkYO DgkBE4nv7wy6GDmBTDGJC/fWs4HYQgJLGCXW76rsYuQCsluZJO7vnM4CkmAT0JE4tfoII4gt IiAt8br3DStIEbPAdmaJk60TwYqEBVwkup/9YAVZwCKgKrFuliZImFfATeLzmW8sEMvkJG6e 62QGsTkF3CU2PlvFCLHYTeLU9NdsExh5FzAyrGIUSS0tzk3PLTbSK07MLS7NS9dLzs/dxAgM uW3Hfm7Zwdj1LvgQowAHoxIPb8Sy6Cgh1sSy4srcQ4wSHMxKIryNwTFRQrwpiZVVqUX58UWl OanFhxilOViUxHl796yOFBJITyxJzU5NLUgtgskycXBKNTDufO26KEuU5Wk3v8zMqpWnbotM DPab/cD+7pLzjFLzNonzvCi0EPnErLOOY+q38NgK5SbTqffM1R8JydczyMfeZInd8/NA75E4 UZlpUU6BpaLN+7psXz/95JpwYelLNsavmTxpjjNv1c6f8Mvs/JPz9+qni1yJ+79LcFVDC/9m acfi7u1TmlWVWIozEg21mIuKEwHdvGrHNQIAAA== X-CMS-MailID: 20180115102114eucas1p1368de6d50059f1c885ddf02acfade43e X-Msg-Generator: CA CMS-TYPE: 201P X-CMS-RootMailID: 20180115102114eucas1p1368de6d50059f1c885ddf02acfade43e X-RootMTR: 20180115102114eucas1p1368de6d50059f1c885ddf02acfade43e References: <1516011655-15669-1-git-send-email-i.maximets@samsung.com> X-Spam-Status: No, score=-7.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Heetae Ahn , Ilya Maximets Subject: [ovs-dev] [PATCH v11 2/5] dpif-netdev: Count cycles on per-rxq basis. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Upcoming time-based output batching will allow to collect in a single output batch packets from different RX queues. Lets keep the list of RX queues for each output packet and collect cycles for them on send. Tested-by: Jan Scheurich Acked-by: Jan Scheurich Signed-off-by: Ilya Maximets --- lib/dpif-netdev.c | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 61f301e..4979356 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -497,6 +497,7 @@ struct tx_port { long long last_used; struct hmap_node node; struct dp_packet_batch output_pkts; + struct dp_netdev_rxq *output_pkts_rxqs[NETDEV_MAX_BURST]; }; /* A set of properties for the current processing loop that is not directly @@ -506,6 +507,8 @@ struct tx_port { struct dp_netdev_pmd_thread_ctx { /* Latest measured time. See 'pmd_thread_ctx_time_update()'. */ long long now; + /* RX queue from which last packet was received. */ + struct dp_netdev_rxq *last_rxq; }; /* PMD: Poll modes drivers. PMD accesses devices via polling to eliminate @@ -3151,9 +3154,14 @@ static void dp_netdev_pmd_flush_output_on_port(struct dp_netdev_pmd_thread *pmd, struct tx_port *p) { + int i; int tx_qid; int output_cnt; bool dynamic_txqs; + struct cycle_timer timer; + uint64_t cycles; + + cycle_timer_start(&pmd->perf_stats, &timer); dynamic_txqs = p->port->dynamic_txqs; if (dynamic_txqs) { @@ -3163,12 +3171,23 @@ dp_netdev_pmd_flush_output_on_port(struct dp_netdev_pmd_thread *pmd, } output_cnt = dp_packet_batch_size(&p->output_pkts); + ovs_assert(output_cnt > 0); netdev_send(p->port->netdev, tx_qid, &p->output_pkts, dynamic_txqs); dp_packet_batch_init(&p->output_pkts); pmd_perf_update_counter(&pmd->perf_stats, PMD_STAT_SENT_PKTS, output_cnt); pmd_perf_update_counter(&pmd->perf_stats, PMD_STAT_SENT_BATCHES, 1); + + /* Distribute send cycles evenly among transmitted packets and assign to + * their respective rx queues. */ + cycles = cycle_timer_stop(&pmd->perf_stats, &timer) / output_cnt; + for (i = 0; i < output_cnt; i++) { + if (p->output_pkts_rxqs[i]) { + dp_netdev_rxq_add_cycles(p->output_pkts_rxqs[i], + RXQ_CYCLES_PROC_CURR, cycles); + } + } } static void @@ -3192,10 +3211,14 @@ dp_netdev_process_rxq_port(struct dp_netdev_pmd_thread *pmd, struct cycle_timer timer; int error; int batch_cnt = 0; + uint64_t cycles; /* Measure duration for polling and processing rx burst. */ cycle_timer_start(&pmd->perf_stats, &timer); + + pmd->ctx.last_rxq = rxq; dp_packet_batch_init(&batch); + error = netdev_rxq_recv(rxq->rx, &batch); if (!error) { /* At least one packet received. */ @@ -3204,12 +3227,12 @@ dp_netdev_process_rxq_port(struct dp_netdev_pmd_thread *pmd, batch_cnt = batch.count; dp_netdev_input(pmd, &batch, port_no); - dp_netdev_pmd_flush_output_packets(pmd); /* Assign processing cycles to rx queue. */ - uint64_t cycles = cycle_timer_stop(&pmd->perf_stats, &timer); + cycles = cycle_timer_stop(&pmd->perf_stats, &timer); dp_netdev_rxq_add_cycles(rxq, RXQ_CYCLES_PROC_CURR, cycles); + dp_netdev_pmd_flush_output_packets(pmd); } else { /* Discard cycles. */ cycle_timer_stop(&pmd->perf_stats, &timer); @@ -3221,6 +3244,8 @@ dp_netdev_process_rxq_port(struct dp_netdev_pmd_thread *pmd, } } + pmd->ctx.last_rxq = NULL; + return batch_cnt; } @@ -4508,6 +4533,7 @@ dp_netdev_configure_pmd(struct dp_netdev_pmd_thread *pmd, struct dp_netdev *dp, ovs_mutex_init(&pmd->port_mutex); cmap_init(&pmd->flow_table); cmap_init(&pmd->classifiers); + pmd->ctx.last_rxq = NULL; pmd_thread_ctx_time_update(pmd); pmd->next_optimization = pmd->ctx.now + DPCLS_OPTIMIZATION_INTERVAL; pmd->rxq_next_cycle_store = pmd->ctx.now + PMD_RXQ_INTERVAL_LEN; @@ -5385,6 +5411,8 @@ dp_execute_cb(void *aux_, struct dp_packet_batch *packets_, dp_netdev_pmd_flush_output_on_port(pmd, p); } DP_PACKET_BATCH_FOR_EACH (packet, packets_) { + p->output_pkts_rxqs[dp_packet_batch_size(&p->output_pkts)] = + pmd->ctx.last_rxq; dp_packet_batch_add(&p->output_pkts, packet); } return;