From patchwork Sun Oct 1 07:57:34 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 820237 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3y4dFg4wCTz9sRq for ; Sun, 1 Oct 2017 19:07:49 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 5C12C49F; Sun, 1 Oct 2017 08:07:44 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 1CE3F82 for ; Sun, 1 Oct 2017 08:07:43 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 33139CE for ; Sun, 1 Oct 2017 08:07:42 +0000 (UTC) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga104.jf.intel.com with ESMTP; 01 Oct 2017 01:07:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.42,463,1500966000"; d="scan'208"; a="1020308545" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by orsmga003.jf.intel.com with ESMTP; 01 Oct 2017 01:07:40 -0700 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Sun, 1 Oct 2017 08:57:34 +0100 Message-Id: <1506844660-4902-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 X-Spam-Status: No, score=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH 1/7] ccmap: Use PADDED_MEMBERS macro in ccmap_impl structure. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Instead of explicitly adding the pad bytes to force the structure an exact multiple of cacheline size, let the PADDED_MEMBERS macro do the job. Signed-off-by: Bhanuprakash Bodireddy --- lib/ccmap.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/lib/ccmap.c b/lib/ccmap.c index 08359b5..a460833 100644 --- a/lib/ccmap.c +++ b/lib/ccmap.c @@ -102,16 +102,15 @@ BUILD_ASSERT_DECL(sizeof(struct ccmap_bucket) == CACHE_LINE_SIZE); /* The implementation of a concurrent hash map. */ struct ccmap_impl { - unsigned int n_unique; /* Number of in-use nodes. */ - unsigned int n; /* Number of hashes inserted. */ - unsigned int max_n; /* Max nodes before enlarging. */ - unsigned int min_n; /* Min nodes before shrinking. */ - uint32_t mask; /* Number of 'buckets', minus one. */ - uint32_t basis; /* Basis for rehashing client's hash values. */ - - /* Padding to make ccmap_impl exactly one cache line long. */ - uint8_t pad[CACHE_LINE_SIZE - sizeof(unsigned int) * 6]; - + PADDED_MEMBERS(CACHE_LINE_SIZE, + unsigned int n_unique; /* Number of in-use nodes. */ + unsigned int n; /* Number of hashes inserted. */ + unsigned int max_n; /* Max nodes before enlarging. */ + unsigned int min_n; /* Min nodes before shrinking. */ + uint32_t mask; /* Number of 'buckets', minus one. */ + uint32_t basis; /* Basis for rehashing client's + hash values. */ + ); struct ccmap_bucket buckets[]; }; BUILD_ASSERT_DECL(sizeof(struct ccmap_impl) == CACHE_LINE_SIZE); From patchwork Sun Oct 1 07:57:35 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 820238 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3y4dGP2lnqz9sRq for ; Sun, 1 Oct 2017 19:08:29 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 5E84C723; Sun, 1 Oct 2017 08:07:45 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 63A7D82 for ; Sun, 1 Oct 2017 08:07:43 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 21175CE for ; Sun, 1 Oct 2017 08:07:43 +0000 (UTC) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga104.jf.intel.com with ESMTP; 01 Oct 2017 01:07:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.42,463,1500966000"; d="scan'208"; a="1020308548" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by orsmga003.jf.intel.com with ESMTP; 01 Oct 2017 01:07:41 -0700 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Sun, 1 Oct 2017 08:57:35 +0100 Message-Id: <1506844660-4902-2-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 In-Reply-To: <1506844660-4902-1-git-send-email-bhanuprakash.bodireddy@intel.com> References: <1506844660-4902-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Spam-Status: No, score=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH 2/7] cmap: Use PADDED_MEMBERS_CACHELINE_MARKER in cmap_impl. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Instead of explicitly adding the pad bytes to force the structure an exact multiple of cacheline size, let the macro do the job. This way the pad bytes will be auto adjusted when the new members get introduced in to the structure. Signed-off-by: Bhanuprakash Bodireddy --- lib/cmap.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/lib/cmap.c b/lib/cmap.c index 8c7312d..35decea 100644 --- a/lib/cmap.c +++ b/lib/cmap.c @@ -165,13 +165,18 @@ BUILD_ASSERT_DECL(sizeof(struct cmap_bucket) == CACHE_LINE_SIZE); /* The implementation of a concurrent hash map. */ struct cmap_impl { - unsigned int n; /* Number of in-use elements. */ - unsigned int max_n; /* Max elements before enlarging. */ - unsigned int min_n; /* Min elements before shrinking. */ - uint32_t mask; /* Number of 'buckets', minus one. */ - uint32_t basis; /* Basis for rehashing client's hash values. */ - uint8_t pad[CACHE_LINE_SIZE - 4 * 5]; /* Pad to end of cache line. */ - struct cmap_bucket buckets[1]; + PADDED_MEMBERS_CACHELINE_MARKER(CACHE_LINE_SIZE, cacheline0, + unsigned int n; /* Number of in-use elements. */ + unsigned int max_n; /* Max elements before enlarging. */ + unsigned int min_n; /* Min elements before shrinking. */ + uint32_t mask; /* Number of 'buckets', minus one. */ + uint32_t basis; /* Basis for rehashing client's + hash values. */ + ); + + PADDED_MEMBERS_CACHELINE_MARKER(CACHE_LINE_SIZE, cacheline1, + struct cmap_bucket buckets[1]; + ); }; BUILD_ASSERT_DECL(sizeof(struct cmap_impl) == CACHE_LINE_SIZE * 2); From patchwork Sun Oct 1 07:57:36 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 820239 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3y4dHC6gczz9t2M for ; Sun, 1 Oct 2017 19:09:11 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 5A3D43EE; Sun, 1 Oct 2017 08:07:47 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id EEDE682 for ; Sun, 1 Oct 2017 08:07:43 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 6CB50CE for ; Sun, 1 Oct 2017 08:07:43 +0000 (UTC) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga104.jf.intel.com with ESMTP; 01 Oct 2017 01:07:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.42,463,1500966000"; d="scan'208"; a="1020308549" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by orsmga003.jf.intel.com with ESMTP; 01 Oct 2017 01:07:42 -0700 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Sun, 1 Oct 2017 08:57:36 +0100 Message-Id: <1506844660-4902-3-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 In-Reply-To: <1506844660-4902-1-git-send-email-bhanuprakash.bodireddy@intel.com> References: <1506844660-4902-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Spam-Status: No, score=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH 3/7] dpif_netdev: Refactor dp_netdev_pmd_thread structure. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org This commit introduces below changes to dp_netdev_pmd_thread structure. - Mark cachelines and in this process reorder few members to avoid holes. - Align emc_cache to a cacheline. - Maintain the grouping of related member variables. - Add comment on the information on pad bytes whereever appropriate so that new member variables may be introduced to fill the holes in future. Below is how the structure looks with this commit. Member size OVS_CACHE_LINE_MARKER cacheline0; struct dp_netdev * dp; 8 struct cmap_node node; 8 pthread_cond_t cond; 48 OVS_CACHE_LINE_MARKER cacheline1; struct ovs_mutex cond_mutex; 48 pthread_t thread; 8 unsigned int core_id; 4 int numa_id; 4 OVS_CACHE_LINE_MARKER cacheline2; struct emc_cache flow_cache; 4849672 ###cachelineX: 64 bytes, 0 pad bytes#### struct cmap flow_table; 8 .... ###cachelineY: 59 bytes, 5 pad bytes#### struct dp_netdev_pmd_stats stats 40 .... ###cachelineZ: 48 bytes, 16 pad bytes### struct ovs_mutex port_mutex; 48 .... This change also improve the performance marginally. Signed-off-by: Bhanuprakash Bodireddy --- lib/dpif-netdev.c | 160 +++++++++++++++++++++++++++++++----------------------- 1 file changed, 91 insertions(+), 69 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index d5eb830..4cd0edf 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -547,18 +547,31 @@ struct tx_port { * actions in either case. * */ struct dp_netdev_pmd_thread { - struct dp_netdev *dp; - struct ovs_refcount ref_cnt; /* Every reference must be refcount'ed. */ - struct cmap_node node; /* In 'dp->poll_threads'. */ - - pthread_cond_t cond; /* For synchronizing pmd thread reload. */ - struct ovs_mutex cond_mutex; /* Mutex for condition variable. */ + PADDED_MEMBERS_CACHELINE_MARKER(CACHE_LINE_SIZE, cacheline0, + struct dp_netdev *dp; + struct cmap_node node; /* In 'dp->poll_threads'. */ + pthread_cond_t cond; /* For synchronizing pmd thread + reload. */ + ); + + PADDED_MEMBERS_CACHELINE_MARKER(CACHE_LINE_SIZE, cacheline1, + struct ovs_mutex cond_mutex; /* Mutex for condition variable. */ + pthread_t thread; + unsigned core_id; /* CPU core id of this pmd thread. */ + int numa_id; /* numa node id of this pmd thread. */ + ); /* Per thread exact-match cache. Note, the instance for cpu core * NON_PMD_CORE_ID can be accessed by multiple threads, and thusly * need to be protected by 'non_pmd_mutex'. Every other instance * will only be accessed by its own pmd thread. */ - struct emc_cache flow_cache; + OVS_ALIGNED_VAR(CACHE_LINE_SIZE) struct emc_cache flow_cache; + struct ovs_refcount ref_cnt; /* Every reference must be refcount'ed. */ + + /* Queue id used by this pmd thread to send packets on all netdevs if + * XPS disabled for this netdev. All static_tx_qid's are unique and less + * than 'cmap_count(dp->poll_threads)'. */ + uint32_t static_tx_qid; /* Flow-Table and classifiers * @@ -567,68 +580,77 @@ struct dp_netdev_pmd_thread { * 'flow_mutex'. */ struct ovs_mutex flow_mutex; - struct cmap flow_table OVS_GUARDED; /* Flow table. */ - - /* One classifier per in_port polled by the pmd */ - struct cmap classifiers; - /* Periodically sort subtable vectors according to hit frequencies */ - long long int next_optimization; - /* End of the next time interval for which processing cycles - are stored for each polled rxq. */ - long long int rxq_interval; - - /* Statistics. */ - struct dp_netdev_pmd_stats stats; - - /* Cycles counters */ - struct dp_netdev_pmd_cycles cycles; - - /* Used to count cicles. See 'cycles_counter_end()' */ - unsigned long long last_cycles; - - struct latch exit_latch; /* For terminating the pmd thread. */ - struct seq *reload_seq; - uint64_t last_reload_seq; - atomic_bool reload; /* Do we need to reload ports? */ - pthread_t thread; - unsigned core_id; /* CPU core id of this pmd thread. */ - int numa_id; /* numa node id of this pmd thread. */ - bool isolated; - - /* Queue id used by this pmd thread to send packets on all netdevs if - * XPS disabled for this netdev. All static_tx_qid's are unique and less - * than 'cmap_count(dp->poll_threads)'. */ - uint32_t static_tx_qid; - - struct ovs_mutex port_mutex; /* Mutex for 'poll_list' and 'tx_ports'. */ - /* List of rx queues to poll. */ - struct hmap poll_list OVS_GUARDED; - /* Map of 'tx_port's used for transmission. Written by the main thread, - * read by the pmd thread. */ - struct hmap tx_ports OVS_GUARDED; - - /* These are thread-local copies of 'tx_ports'. One contains only tunnel - * ports (that support push_tunnel/pop_tunnel), the other contains ports - * with at least one txq (that support send). A port can be in both. - * - * There are two separate maps to make sure that we don't try to execute - * OUTPUT on a device which has 0 txqs or PUSH/POP on a non-tunnel device. - * - * The instances for cpu core NON_PMD_CORE_ID can be accessed by multiple - * threads, and thusly need to be protected by 'non_pmd_mutex'. Every - * other instance will only be accessed by its own pmd thread. */ - struct hmap tnl_port_cache; - struct hmap send_port_cache; - - /* Only a pmd thread can write on its own 'cycles' and 'stats'. - * The main thread keeps 'stats_zero' and 'cycles_zero' as base - * values and subtracts them from 'stats' and 'cycles' before - * reporting to the user */ - unsigned long long stats_zero[DP_N_STATS]; - uint64_t cycles_zero[PMD_N_CYCLES]; - - /* Set to true if the pmd thread needs to be reloaded. */ - bool need_reload; + PADDED_MEMBERS(CACHE_LINE_SIZE, + struct cmap flow_table OVS_GUARDED; /* Flow table. */ + + /* One classifier per in_port polled by the pmd */ + struct cmap classifiers; + /* Periodically sort subtable vectors according to hit frequencies */ + long long int next_optimization; + /* End of the next time interval for which processing cycles + are stored for each polled rxq. */ + long long int rxq_interval; + + /* Cycles counters */ + struct dp_netdev_pmd_cycles cycles; + + /* Used to count cycles. See 'cycles_counter_end()'. */ + unsigned long long last_cycles; + struct latch exit_latch; /* For terminating the pmd thread. */ + ); + + PADDED_MEMBERS(CACHE_LINE_SIZE, + /* Statistics. */ + struct dp_netdev_pmd_stats stats; + + struct seq *reload_seq; + uint64_t last_reload_seq; + atomic_bool reload; /* Do we need to reload ports? */ + bool isolated; + + /* Set to true if the pmd thread needs to be reloaded. */ + bool need_reload; + /* 5 pad bytes. */ + ); + + PADDED_MEMBERS(CACHE_LINE_SIZE, + struct ovs_mutex port_mutex; /* Mutex for 'poll_list' + and 'tx_ports'. */ + /* 16 pad bytes. */ + ); + PADDED_MEMBERS(CACHE_LINE_SIZE, + /* List of rx queues to poll. */ + struct hmap poll_list OVS_GUARDED; + /* Map of 'tx_port's used for transmission. Written by the main + * thread, read by the pmd thread. */ + struct hmap tx_ports OVS_GUARDED; + ); + PADDED_MEMBERS(CACHE_LINE_SIZE, + /* These are thread-local copies of 'tx_ports'. One contains only + * tunnel ports (that support push_tunnel/pop_tunnel), the other + * contains ports with at least one txq (that support send). + * A port can be in both. + * + * There are two separate maps to make sure that we don't try to + * execute OUTPUT on a device which has 0 txqs or PUSH/POP on a + * non-tunnel device. + * + * The instances for cpu core NON_PMD_CORE_ID can be accessed by + * multiple threads and thusly need to be protected by 'non_pmd_mutex'. + * Every other instance will only be accessed by its own pmd thread. */ + struct hmap tnl_port_cache; + struct hmap send_port_cache; + ); + + PADDED_MEMBERS(CACHE_LINE_SIZE, + /* Only a pmd thread can write on its own 'cycles' and 'stats'. + * The main thread keeps 'stats_zero' and 'cycles_zero' as base + * values and subtracts them from 'stats' and 'cycles' before + * reporting to the user */ + unsigned long long stats_zero[DP_N_STATS]; + uint64_t cycles_zero[PMD_N_CYCLES]; + /* 8 pad bytes. */ + ); }; /* Interface to netdev-based datapath. */ From patchwork Sun Oct 1 07:57:37 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 820242 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3y4dKD1Rnzz9t2M for ; Sun, 1 Oct 2017 19:10:56 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 2AAAF97A; Sun, 1 Oct 2017 08:08:21 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 5EECB305 for ; Sun, 1 Oct 2017 08:08:18 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id C0FE5171 for ; Sun, 1 Oct 2017 08:08:16 +0000 (UTC) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga105.fm.intel.com with ESMTP; 01 Oct 2017 01:08:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.42,463,1500966000"; d="scan'208"; a="1020308550" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by orsmga003.jf.intel.com with ESMTP; 01 Oct 2017 01:07:43 -0700 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Sun, 1 Oct 2017 08:57:37 +0100 Message-Id: <1506844660-4902-4-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 In-Reply-To: <1506844660-4902-1-git-send-email-bhanuprakash.bodireddy@intel.com> References: <1506844660-4902-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Spam-Status: No, score=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH 4/7] netdev-dpdk: Refactor netdev_dpdk structure. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org This commit introduces below changes to netdev_dpdk structure. - Mark cachelines and reorder few member variables. - Maintain the grouping of related member variables. - Add comment on the information on pad bytes where ever appropriate, so new members can be introduced in the future to fill the gaps. Below is how this structure looks with this commit. Member size OVS_CACHE_LINE_MARKER cacheline0; dpdk_port_t port_id; 1 bool attached; 1 ... OVS_CACHE_LINE_MARKER cacheline1; struct ovs_mutex; 48 struct dpdk_mp *dpdk_mp; 8 ... Signed-off-by: Bhanuprakash Bodireddy --- lib/netdev-dpdk.c | 158 +++++++++++++++++++++++++++++------------------------- 1 file changed, 84 insertions(+), 74 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index c60f46f..d999807 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -350,80 +350,90 @@ enum dpdk_hw_ol_features { }; struct netdev_dpdk { - struct netdev up; - dpdk_port_t port_id; - int max_packet_len; - enum dpdk_dev_type type; - - struct dpdk_tx_queue *tx_q; - - struct ovs_mutex mutex OVS_ACQ_AFTER(dpdk_mutex); - - struct dpdk_mp *dpdk_mp; - int mtu; - int socket_id; - int buf_size; - struct netdev_stats stats; - /* Protects stats */ - rte_spinlock_t stats_lock; - - struct eth_addr hwaddr; - enum netdev_flags flags; - - struct rte_eth_link link; - int link_reset_cnt; - - /* virtio identifier for vhost devices */ - ovsrcu_index vid; - - /* True if vHost device is 'up' and has been reconfigured at least once */ - bool vhost_reconfigured; - - /* Identifier used to distinguish vhost devices from each other. */ - char vhost_id[PATH_MAX]; - - /* Device arguments for dpdk ports */ - char *devargs; - - /* If true, device was attached by rte_eth_dev_attach(). */ - bool attached; - - /* In dpdk_list. */ - struct ovs_list list_node OVS_GUARDED_BY(dpdk_mutex); - - /* QoS configuration and lock for the device */ - OVSRCU_TYPE(struct qos_conf *) qos_conf; - - /* The following properties cannot be changed when a device is running, - * so we remember the request and update them next time - * netdev_dpdk*_reconfigure() is called */ - int requested_mtu; - int requested_n_txq; - int requested_n_rxq; - int requested_rxq_size; - int requested_txq_size; - - /* Number of rx/tx descriptors for physical devices */ - int rxq_size; - int txq_size; - - /* Socket ID detected when vHost device is brought up */ - int requested_socket_id; - - /* Denotes whether vHost port is client/server mode */ - uint64_t vhost_driver_flags; - - /* Ingress Policer */ - OVSRCU_TYPE(struct ingress_policer *) ingress_policer; - uint32_t policer_rate; - uint32_t policer_burst; - - /* DPDK-ETH Flow control */ - struct rte_eth_fc_conf fc_conf; - - /* DPDK-ETH hardware offload features, - * from the enum set 'dpdk_hw_ol_features' */ - uint32_t hw_ol_features; + PADDED_MEMBERS_CACHELINE_MARKER(CACHE_LINE_SIZE, cacheline0, + dpdk_port_t port_id; + + /* If true, device was attached by rte_eth_dev_attach(). */ + bool attached; + struct eth_addr hwaddr; + int mtu; + int socket_id; + int buf_size; + int max_packet_len; + enum dpdk_dev_type type; + enum netdev_flags flags; + char *devargs; /* Device arguments for dpdk ports */ + struct dpdk_tx_queue *tx_q; + struct rte_eth_link link; + int link_reset_cnt; + /* 4 pad bytes here. */ + ); + + PADDED_MEMBERS_CACHELINE_MARKER(CACHE_LINE_SIZE, cacheline1, + struct ovs_mutex mutex OVS_ACQ_AFTER(dpdk_mutex); + struct dpdk_mp *dpdk_mp; + + /* virtio identifier for vhost devices */ + ovsrcu_index vid; + + /* True if vHost device is 'up' and has been reconfigured at least once */ + bool vhost_reconfigured; + /* 3 pad bytes here. */ + ); + + PADDED_MEMBERS(CACHE_LINE_SIZE, + /* Identifier used to distinguish vhost devices from each other. */ + char vhost_id[PATH_MAX]; + ); + + PADDED_MEMBERS(CACHE_LINE_SIZE, + struct netdev up; + /* In dpdk_list. */ + struct ovs_list list_node OVS_GUARDED_BY(dpdk_mutex); + + /* QoS configuration and lock for the device */ + OVSRCU_TYPE(struct qos_conf *) qos_conf; + + /* Ingress Policer */ + OVSRCU_TYPE(struct ingress_policer *) ingress_policer; + uint32_t policer_rate; + uint32_t policer_burst; + ); + + PADDED_MEMBERS(CACHE_LINE_SIZE, + struct netdev_stats stats; + /* Protects stats */ + rte_spinlock_t stats_lock; + /* 44 pad bytes here. */ + ); + + PADDED_MEMBERS(CACHE_LINE_SIZE, + /* The following properties cannot be changed when a device is running, + * so we remember the request and update them next time + * netdev_dpdk*_reconfigure() is called */ + int requested_mtu; + int requested_n_txq; + int requested_n_rxq; + int requested_rxq_size; + int requested_txq_size; + + /* Number of rx/tx descriptors for physical devices */ + int rxq_size; + int txq_size; + + /* Socket ID detected when vHost device is brought up */ + int requested_socket_id; + + /* Denotes whether vHost port is client/server mode */ + uint64_t vhost_driver_flags; + + /* DPDK-ETH Flow control */ + struct rte_eth_fc_conf fc_conf; + + /* DPDK-ETH hardware offload features, + * from the enum set 'dpdk_hw_ol_features' */ + uint32_t hw_ol_features; + ); }; struct netdev_rxq_dpdk { From patchwork Sun Oct 1 07:57:38 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 820241 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3y4dJb16Tcz9t2M for ; Sun, 1 Oct 2017 19:10:23 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 277B1958; Sun, 1 Oct 2017 08:08:20 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 36D9D305 for ; Sun, 1 Oct 2017 08:08:18 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 3B1D9CE for ; Sun, 1 Oct 2017 08:08:16 +0000 (UTC) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga105.fm.intel.com with ESMTP; 01 Oct 2017 01:08:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.42,463,1500966000"; d="scan'208"; a="1020308553" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by orsmga003.jf.intel.com with ESMTP; 01 Oct 2017 01:07:43 -0700 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Sun, 1 Oct 2017 08:57:38 +0100 Message-Id: <1506844660-4902-5-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 In-Reply-To: <1506844660-4902-1-git-send-email-bhanuprakash.bodireddy@intel.com> References: <1506844660-4902-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Spam-Status: No, score=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH 5/7] ofproto: Remove break after OVS_NOT_REACHED. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org The break statement would never be executed as OVS_NOT_REACHED() internally invokes abort() and causes process termination. Signed-off-by: Bhanuprakash Bodireddy --- ofproto/ofproto-dpif-ipfix.c | 1 - ofproto/ofproto-dpif-xlate.c | 2 -- 2 files changed, 3 deletions(-) diff --git a/ofproto/ofproto-dpif-ipfix.c b/ofproto/ofproto-dpif-ipfix.c index 472c272..538faff 100644 --- a/ofproto/ofproto-dpif-ipfix.c +++ b/ofproto/ofproto-dpif-ipfix.c @@ -1314,7 +1314,6 @@ ipfix_def_options_template_fields(enum ipfix_options_template opt_tmpl_type, case NUM_IPFIX_OPTIONS_TEMPLATE: default: OVS_NOT_REACHED(); - break; } return 0; diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c index d320d57..0a71855 100644 --- a/ofproto/ofproto-dpif-xlate.c +++ b/ofproto/ofproto-dpif-xlate.c @@ -3229,7 +3229,6 @@ propagate_tunnel_data_to_flow(struct xlate_ctx *ctx, struct eth_addr dmac, case __OVS_VPORT_TYPE_MAX: default: OVS_NOT_REACHED(); - break; } /* * Update base_flow first followed by flow as the dst_flow gets modified @@ -5951,7 +5950,6 @@ xlate_generic_encap_action(struct xlate_ctx *ctx, default: /* New packet type was checked during decoding. */ OVS_NOT_REACHED(); - break; } if (!ctx->error) { From patchwork Sun Oct 1 07:57:39 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 820240 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3y4dHz6v4cz9t2M for ; Sun, 1 Oct 2017 19:09:51 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 369D29BA; Sun, 1 Oct 2017 08:08:19 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id BA5AD305 for ; Sun, 1 Oct 2017 08:08:17 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 771AC171 for ; Sun, 1 Oct 2017 08:08:15 +0000 (UTC) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga105.fm.intel.com with ESMTP; 01 Oct 2017 01:08:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.42,463,1500966000"; d="scan'208"; a="1020308554" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by orsmga003.jf.intel.com with ESMTP; 01 Oct 2017 01:07:44 -0700 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Sun, 1 Oct 2017 08:57:39 +0100 Message-Id: <1506844660-4902-6-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 In-Reply-To: <1506844660-4902-1-git-send-email-bhanuprakash.bodireddy@intel.com> References: <1506844660-4902-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Spam-Status: No, score=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH 6/7] ofp-actions: Remove break after OVS_NOT_REACHED. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org The break statement would never be executed as OVS_NOT_REACHED() internally invokes abort() and causes process termination. Signed-off-by: Bhanuprakash Bodireddy --- lib/ofp-actions.c | 1 - 1 file changed, 1 deletion(-) diff --git a/lib/ofp-actions.c b/lib/ofp-actions.c index 71eb70c..432b469 100644 --- a/lib/ofp-actions.c +++ b/lib/ofp-actions.c @@ -5482,7 +5482,6 @@ parse_UNROLL_XLATE(char *arg OVS_UNUSED, enum ofputil_protocol *usable_protocols OVS_UNUSED) { OVS_NOT_REACHED(); - return NULL; } static void From patchwork Sun Oct 1 07:57:40 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 820243 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3y4dKd6KRPz9t2M for ; Sun, 1 Oct 2017 19:11:17 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id F36519F8; Sun, 1 Oct 2017 08:08:21 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 8BBA096F for ; Sun, 1 Oct 2017 08:08:18 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 2F6F11AE for ; Sun, 1 Oct 2017 08:08:18 +0000 (UTC) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga105.fm.intel.com with ESMTP; 01 Oct 2017 01:08:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.42,463,1500966000"; d="scan'208"; a="1020308556" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by orsmga003.jf.intel.com with ESMTP; 01 Oct 2017 01:07:45 -0700 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Sun, 1 Oct 2017 08:57:40 +0100 Message-Id: <1506844660-4902-7-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 In-Reply-To: <1506844660-4902-1-git-send-email-bhanuprakash.bodireddy@intel.com> References: <1506844660-4902-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Spam-Status: No, score=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH 7/7] ovsdb: Remove break after OVS_NOT_REACHED. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org The break statement would never be executed as OVS_NOT_REACHED() internally invokes abort() and causes process termination. Signed-off-by: Bhanuprakash Bodireddy --- ovsdb/replication.c | 1 - 1 file changed, 1 deletion(-) diff --git a/ovsdb/replication.c b/ovsdb/replication.c index 304212d..8dd1181 100644 --- a/ovsdb/replication.c +++ b/ovsdb/replication.c @@ -874,7 +874,6 @@ replication_status(void) break; default: OVS_NOT_REACHED(); - break; } } else { ds_put_format(&ds, "not connected to %s", sync_from);