From patchwork Fri Dec 1 15:44:27 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Maximets X-Patchwork-Id: 843552 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=samsung.com header.i=@samsung.com header.b="OeDH4JLb"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ypJWs5B5gz9ryk for ; Sat, 2 Dec 2017 02:45:45 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 09333C14; Fri, 1 Dec 2017 15:44:52 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id F0B9ABF2 for ; Fri, 1 Dec 2017 15:44:50 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 140A74C5 for ; Fri, 1 Dec 2017 15:44:48 +0000 (UTC) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20171201154445euoutp02273f7f4c756e0c116140d6575cb0c14a~8Nh-k8awg3158431584euoutp02k; Fri, 1 Dec 2017 15:44:45 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20171201154445euoutp02273f7f4c756e0c116140d6575cb0c14a~8Nh-k8awg3158431584euoutp02k DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1512143085; bh=090YdnurHgCx7WWVWnPzKOAUXDQGEx8fXhgLrv1ve0w=; h=From:To:Cc:Subject:Date:In-reply-to:References:From; b=OeDH4JLboTaMGuK4fMSBRsuVuygVSWgUGP2eMduRBJx6Mcn4Bw5BXuQ5jYwgOdjYx htMEsQZt/i2xIDKV3/TD15A1bULj3W4NvTq+wvo1/xk5YY9qwMhTn037HhgoYYdkWA uy90qbmk4xgCvGGQBcDuwsIEZrGyB3B/hOgpryrw= Received: from eusmges5.samsung.com (unknown [203.254.199.245]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20171201154443eucas1p278c09dc1dbe2cf10dbb873214d464fd9~8Nh_IPJQb1819718197eucas1p28; Fri, 1 Dec 2017 15:44:43 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges5.samsung.com (EUCPMTA) with SMTP id 48.FD.12743.BE8712A5; Fri, 1 Dec 2017 15:44:43 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20171201154443eucas1p2d360bc11b0554074efe3c0dd0e0dd771~8Nh9YAmQ21816418164eucas1p23; Fri, 1 Dec 2017 15:44:43 +0000 (GMT) X-AuditID: cbfec7f5-f79d06d0000031c7-47-5a2178eb54c3 Received: from eusync3.samsung.com ( [203.254.199.213]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 1C.72.18832.BE8712A5; Fri, 1 Dec 2017 15:44:43 +0000 (GMT) Received: from imaximets.rnd.samsung.ru ([106.109.129.180]) by eusync3.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0P0A00BWSH2B5A50@eusync3.samsung.com>; Fri, 01 Dec 2017 15:44:43 +0000 (GMT) From: Ilya Maximets To: ovs-dev@openvswitch.org, Bhanuprakash Bodireddy Date: Fri, 01 Dec 2017 18:44:27 +0300 Message-id: <1512143073-22347-2-git-send-email-i.maximets@samsung.com> X-Mailer: git-send-email 2.7.4 In-reply-to: <1512143073-22347-1-git-send-email-i.maximets@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrFIsWRmVeSWpSXmKPExsWy7djPc7qvKxSjDGY0aVms/sVpsfOZskVL /0xmiztXfrJZTPt8m93iSvtPdouND8+yWhz5fprRYu2hD+wWcz89Z3Tg8vj19Sqbx+I9L5k8 nt38z+jxfh+Q27dlFWMAaxSXTUpqTmZZapG+XQJXxrV/m9kKNuhUdH6xaGDsk+9i5OSQEDCR OLHmOiOELSZx4d56ti5GLg4hgaWMEu/vbmEHSQgJfGaUWHzTFqbhVtsUJoiiZYwSJ39uZoQo amaS2P1UD8RmE9CROLX6CFhcRCBC4sH3WewgDcwCp5kk3mxaxNzFyMEhLBAtsemeNkgNi4Cq xIvpD8DqeQXcJPqfPWCDWCYncfNcJ1g5p4C7xJHf4iBjJAQWsEmsP/GbCaLGRWLusb0sELaw xKvjEEdLCMhIdHYcZIJoaGaUaFh1iRHCmcAo8aV5OVS3vcSpm1fBbGYBPolJ26aDbZMQ4JXo aBOCKPGQ6Hk5gxXCdpToaoeYIyQwi1Hi9MWpbBMYpRcwMqxiFEktLc5NTy021StOzC0uzUvX S87P3cQIjOHT/45/3cG49JjVIUYBDkYlHl6GcMUoIdbEsuLK3EOMEhzMSiK8WSVAId6UxMqq 1KL8+KLSnNTiQ4zSHCxK4ry2UW2RQgLpiSWp2ampBalFMFkmDk6pBsbm96lsM2MnfViya15K t9ejp6nnYgWNDRTqVoleFNrOb3jfc0moidaZA7suP9AXubl0wt8FBrY6jHy32HltBeuZCi46 b4l1jo++/t9T4p3ZsZ37Vy7b3LPA56XV5xl3JQ4J3dqrwPg/5p9/TPOfOYG98ovWBFYl7X3P fGV1gMglhjRl7nPT0wOUWIozEg21mIuKEwGe9bPV3QIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrALMWRmVeSWpSXmKPExsVy+t/xq7qvKxSjDE6+Z7NY/YvTYuczZYuW /pnMFneu/GSzmPb5NrvFlfaf7BYbH55ltTjy/TSjxdpDH9gt5n56zujA5fHr61U2j8V7XjJ5 PLv5n9Hj/T4gt2/LKsYA1igum5TUnMyy1CJ9uwSujGv/NrMVbNCp6Pxi0cDYJ9/FyMkhIWAi cattChOELSZx4d56ti5GLg4hgSWMEg9Xt7JAOK1MEhOb/rKBVLEJ6EicWn2EEcQWEYiQaJmz nhGkiFngLJNE6+njYEXCAtES9988ZQexWQRUJV5MfwDWwCvgJtH/7AEbxDo5iZvnOpm7GDk4 OAXcJY78FgcJCwGV7N+9g3ECI+8CRoZVjCKppcW56bnFhnrFibnFpXnpesn5uZsYgeG27djP zTsYL20MPsQowMGoxMO7IkQxSog1say4MvcQowQHs5IIb1YJUIg3JbGyKrUoP76oNCe1+BCj NAeLkjhv757VkUIC6YklqdmpqQWpRTBZJg5OqQbG/iY1eR8dzwKxwpepUYlH5vzmLYg7GC1V fzFH74XPYgnBmPtpv2LiPs/J+tTm8IS50MEnXeyaarVXQs7hyP4vHPemqqi/a9ik9ii18lyf Vncwz5RtMy0zmkr1TzzSPWxYLzD9Ae/24/l57RpqlzKK18gbl73vZvzG8zL0SbyzGbfH4mmx TUosxRmJhlrMRcWJAKZDr48zAgAA X-CMS-MailID: 20171201154443eucas1p2d360bc11b0554074efe3c0dd0e0dd771 X-Msg-Generator: CA CMS-TYPE: 201P X-CMS-RootMailID: 20171201154443eucas1p2d360bc11b0554074efe3c0dd0e0dd771 X-RootMTR: 20171201154443eucas1p2d360bc11b0554074efe3c0dd0e0dd771 References: <1512143073-22347-1-git-send-email-i.maximets@samsung.com> X-Spam-Status: No, score=-7.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Heetae Ahn , Ilya Maximets Subject: [ovs-dev] [PATCH v6 1/7] dpif-netdev: Refactor PMD thread structure for further extension. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org This is preparation for 'struct dp_netdev_pmd_thread' modification in upcoming commits. Needed to avoid reordering and regrouping while replacing old and adding new members. Signed-off-by: Bhanuprakash Bodireddy Co-authored-by: Bhanuprakash Bodireddy Signed-off-by: Ilya Maximets --- lib/dpif-netdev.c | 65 ++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 40 insertions(+), 25 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 0a62630..7a7c6ce 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -549,29 +549,22 @@ struct tx_port { struct dp_netdev_pmd_thread { PADDED_MEMBERS_CACHELINE_MARKER(CACHE_LINE_SIZE, cacheline0, struct dp_netdev *dp; - struct cmap_node node; /* In 'dp->poll_threads'. */ - pthread_cond_t cond; /* For synchronizing pmd thread - reload. */ + struct cmap_node node; /* In 'dp->poll_threads'. */ + pthread_cond_t cond; /* For synchronizing pmd thread reload. */ ); PADDED_MEMBERS_CACHELINE_MARKER(CACHE_LINE_SIZE, cacheline1, struct ovs_mutex cond_mutex; /* Mutex for condition variable. */ pthread_t thread; - unsigned core_id; /* CPU core id of this pmd thread. */ - int numa_id; /* numa node id of this pmd thread. */ ); /* Per thread exact-match cache. Note, the instance for cpu core * NON_PMD_CORE_ID can be accessed by multiple threads, and thusly * need to be protected by 'non_pmd_mutex'. Every other instance * will only be accessed by its own pmd thread. */ - OVS_ALIGNED_VAR(CACHE_LINE_SIZE) struct emc_cache flow_cache; - struct ovs_refcount ref_cnt; /* Every reference must be refcount'ed. */ - - /* Queue id used by this pmd thread to send packets on all netdevs if - * XPS disabled for this netdev. All static_tx_qid's are unique and less - * than 'cmap_count(dp->poll_threads)'. */ - uint32_t static_tx_qid; + PADDED_MEMBERS(CACHE_LINE_SIZE, + OVS_ALIGNED_VAR(CACHE_LINE_SIZE) struct emc_cache flow_cache; + ); /* Flow-Table and classifiers * @@ -579,7 +572,10 @@ struct dp_netdev_pmd_thread { * changes to 'classifiers' must be made while still holding the * 'flow_mutex'. */ - struct ovs_mutex flow_mutex; + PADDED_MEMBERS(CACHE_LINE_SIZE, + struct ovs_mutex flow_mutex; + /* 8 pad bytes. */ + ); PADDED_MEMBERS(CACHE_LINE_SIZE, struct cmap flow_table OVS_GUARDED; /* Flow table. */ @@ -596,35 +592,50 @@ struct dp_netdev_pmd_thread { /* Used to count cycles. See 'cycles_counter_end()'. */ unsigned long long last_cycles; - struct latch exit_latch; /* For terminating the pmd thread. */ - ); + /* 8 pad bytes. */ + ); PADDED_MEMBERS(CACHE_LINE_SIZE, /* Statistics. */ struct dp_netdev_pmd_stats stats; + /* 8 pad bytes. */ + ); + PADDED_MEMBERS(CACHE_LINE_SIZE, + struct latch exit_latch; /* For terminating the pmd thread. */ struct seq *reload_seq; uint64_t last_reload_seq; - atomic_bool reload; /* Do we need to reload ports? */ - bool isolated; - + atomic_bool reload; /* Do we need to reload ports? */ /* Set to true if the pmd thread needs to be reloaded. */ bool need_reload; - /* 5 pad bytes. */ + bool isolated; + + struct ovs_refcount ref_cnt; /* Every reference must be refcount'ed. */ + + /* Queue id used by this pmd thread to send packets on all netdevs if + * XPS disabled for this netdev. All static_tx_qid's are unique and + * less than 'cmap_count(dp->poll_threads)'. */ + uint32_t static_tx_qid; + + unsigned core_id; /* CPU core id of this pmd thread. */ + int numa_id; /* numa node id of this pmd thread. */ + + /* 20 pad bytes. */ ); PADDED_MEMBERS(CACHE_LINE_SIZE, - struct ovs_mutex port_mutex; /* Mutex for 'poll_list' - and 'tx_ports'. */ - /* 16 pad bytes. */ + /* Mutex for 'poll_list' and 'tx_ports'. */ + struct ovs_mutex port_mutex; ); + PADDED_MEMBERS(CACHE_LINE_SIZE, /* List of rx queues to poll. */ struct hmap poll_list OVS_GUARDED; - /* Map of 'tx_port's used for transmission. Written by the main - * thread, read by the pmd thread. */ + /* Map of 'tx_port's used for transmission. + * Written by the main thread, read by the pmd thread. */ struct hmap tx_ports OVS_GUARDED; ); + PADDED_MEMBERS(CACHE_LINE_SIZE, /* These are thread-local copies of 'tx_ports'. One contains only * tunnel ports (that support push_tunnel/pop_tunnel), the other @@ -648,9 +659,13 @@ struct dp_netdev_pmd_thread { * values and subtracts them from 'stats' and 'cycles' before * reporting to the user */ unsigned long long stats_zero[DP_N_STATS]; - uint64_t cycles_zero[PMD_N_CYCLES]; /* 8 pad bytes. */ ); + + PADDED_MEMBERS(CACHE_LINE_SIZE, + uint64_t cycles_zero[PMD_N_CYCLES]; + /* 48 pad bytes. */ + ); }; /* Interface to netdev-based datapath. */