From patchwork Fri Nov 17 10:26:02 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 838944 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ydZKv3Y9hz9ryk for ; Fri, 17 Nov 2017 21:36:50 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 70C7699F; Fri, 17 Nov 2017 10:36:47 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 70CF998C for ; Fri, 17 Nov 2017 10:36:46 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 6CD4E1AD for ; Fri, 17 Nov 2017 10:36:45 +0000 (UTC) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Nov 2017 02:36:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.44,408,1505804400"; d="scan'208";a="8725562" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by orsmga002.jf.intel.com with ESMTP; 17 Nov 2017 02:36:42 -0800 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Fri, 17 Nov 2017 10:26:02 +0000 Message-Id: <1510914362-103044-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 X-Spam-Status: No, score=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH] packets: Prefetch the packet metadata in cacheline1. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org pkt_metadata_prefetch_init() is used to prefetch the packet metadata before initializing the metadata in pkt_metadata_init(). This is done for every packet in userspace datapath and is performance critical. Commit 99fc16c0 prefetches only cachline0 and cacheline2 as the metadata part of respective cachelines will be initialized by pkt_metadata_init(). However in VXLAN case when popping the vxlan header, netdev_vxlan_pop_header() invokes pkt_metadata_init_tnl() which zeroes out metadata part of cacheline1 that wasn't prefetched earlier and causes performance degradation. By prefetching cacheline1, 9% performance improvement is observed. CC: Ben Pfaff Fixes: 99fc16c0 ("Reorganize the pkt_metadata structure.") Signed-off-by: Bhanuprakash Bodireddy --- lib/packets.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/lib/packets.h b/lib/packets.h index 461f488..2e8c0f1 100644 --- a/lib/packets.h +++ b/lib/packets.h @@ -159,7 +159,8 @@ pkt_metadata_init(struct pkt_metadata *md, odp_port_t port) } /* This function prefetches the cachelines touched by pkt_metadata_init() - * For performance reasons the two functions should be kept in sync. */ + * and pkt_metadata_init_tnl(). For performance reasons the two functions + * should be kept in sync. */ static inline void pkt_metadata_prefetch_init(struct pkt_metadata *md) { @@ -167,6 +168,10 @@ pkt_metadata_prefetch_init(struct pkt_metadata *md) * be initialized later in pkt_metadata_init(). */ OVS_PREFETCH(md->cacheline0); + /* Prefetch cacheline1 as members of this cacheline will be zeroed out + * in pkt_metadata_init_tnl(). */ + OVS_PREFETCH(md->cacheline1); + /* Prefetch cachline2 as ip_dst & ipv6_dst fields will be initialized. */ OVS_PREFETCH(md->cacheline2); }