From patchwork Fri Feb 16 20:53:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Billy O'Mahony X-Patchwork-Id: 874653 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zjljR4X4Jz9t2l for ; Sat, 17 Feb 2018 07:53:31 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id D50EC11F0; Fri, 16 Feb 2018 20:53:28 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id D6B2F11DE for ; Fri, 16 Feb 2018 20:53:26 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 59826124 for ; Fri, 16 Feb 2018 20:53:26 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Feb 2018 12:53:25 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,520,1511856000"; d="scan'208";a="28653657" Received: from sivswdev01.ir.intel.com (HELO localhost.localdomain) ([10.237.217.45]) by orsmga003.jf.intel.com with ESMTP; 16 Feb 2018 12:53:24 -0800 From: Billy O'Mahony To: dev@openvswitch.org Date: Fri, 16 Feb 2018 20:53:19 +0000 Message-Id: <1518814401-21311-1-git-send-email-billy.o.mahony@intel.com> X-Mailer: git-send-email 1.7.0.7 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [RFC 0/2] Ingress Scheduling X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org This patch set implements the 'preferential read' part of the feature of ingress scheduling described at OvS 2017 Fall Conference https://www.slideshare.net/LF_OpenvSwitch/lfovs17ingress-scheduling-82280320. It allows configuration to specify an ingress priority for and entire interface. This protects traffic on higher priority interfaces from loss and latency as PMDs get overloaded. Results so far a are very promising; For a uniform traffic distribution as total offered load increases loss starts on the lowest priority port first and the highest priority port last. When using four physical ports with each port forwarded to one of the other ports. The following packets loss is seen. The EMC was bypassed in this case and a small delay loop was added to each packet to simulate more realistic per packet processing cost of 1000cycles approx. Port dpdk_0 dpdk_1 dpdk_2 dpdk_3 Traffic Dist. 25% 25% 25% 25% Priority 0 1 2 3 n_rxq 8 8 8 8 Total Load Kpps Loss Rate Per Port (Kpps) 2110 0 0 0 0 2120 5 0 0 0 2920 676 0 0 0 2930 677 5 0 0 3510 854 411 0 0 3520 860 415 3 0 4390 1083 716 348 0 4400 1088 720 354 1 Even in the case where most traffic is on the priority port this remains the case: Port dpdk_0 dpdk_1 dpdk_2 dpdk_3 Traffic Dist. 10% 20% 30% 40% Priority 0 1 2 3 n_rxq 8 8 8 8 Total Load Kpps Loss Rate Per Port (Kpps) 2400 0 0 0 0 2410 5 0 0 0 2720 225 5 0 0 2880 252 121 9 0 3030 269 176 82 3 4000 369 377 384 392 5000 471 580 691 801 The latency characteristics of the traffic on the higher priority ports is also protected. Port dpdk_0 dpdk_1 dpdk_2 dpdk_3 Traffic Dist. 10% 20% 30% 40% Priority 0 1 2 3 n_rxq 8 8 8 8 Total dpdk0 dpdk1 dpdk2 dpdk3 Load Kpps 2400 113 122 120 125 2410 36117 571 577 560 2720 323242 14424 3265 3235 2880 391404 33508 10075 4600 3030 412597 35450 17061 7965 4000 414729 36070 17740 11106 5000 416801 36445 18233 11653 Some General setup notes: Fortville. (X710 DA4. firmware-version: 6.01 0x800034af 1.1747.0) Intel(R) Xeon(R) CPU E5-2695 v3 @ 2.30GHz One pmd Port fwding port 0<->1, 2 <-> 3 Frame 64B, UDP 221 streams per port. OvS base - 4c80644 http://github.com/istokes/ovs dpdk_merge_2_9. Added 600cycles approx pkt processing in order to bring per packet cost to ~1000 cycles. DPDK v17.11.1 Billy O'Mahony (2): ingress scheduling: schema and docs ingress scheduling: Provide per interface ingress priority Documentation/howto/dpdk.rst | 18 ++++++++++++ include/openvswitch/ofp-parse.h | 3 ++ lib/dpif-netdev.c | 47 +++++++++++++++++++++--------- lib/netdev-bsd.c | 1 + lib/netdev-dpdk.c | 64 +++++++++++++++++++++++++++++++++++++++-- lib/netdev-dummy.c | 1 + lib/netdev-linux.c | 1 + lib/netdev-provider.h | 11 ++++++- lib/netdev-vport.c | 1 + lib/netdev.c | 23 +++++++++++++++ lib/netdev.h | 2 ++ vswitchd/bridge.c | 2 ++ vswitchd/vswitch.ovsschema | 9 ++++-- vswitchd/vswitch.xml | 40 ++++++++++++++++++++++++++ 14 files changed, 205 insertions(+), 18 deletions(-)