From patchwork Sat Feb 2 15:03:55 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [v2,net-next] inet: Get critical word in first 64bit of cache line Date: Sat, 02 Feb 2013 05:03:55 -0000 From: Eric Dumazet X-Patchwork-Id: 217667 Message-Id: <1359817435.30177.70.camel@edumazet-glaptop> To: ling.ma.program@gmail.com, David Miller Cc: netdev , Maciej =?UTF-8?Q?=C5=BBenczykowski?= From: Ma Ling In order to reduce memory latency when last level cache miss occurs, modern CPUs i.e. x86 and arm introduced Critical Word First(CWF) or Early Restart(ER) to get data ASAP. For CWF if critical word is first member in cache line, memory feed CPU with critical word, then fill others data in cache line one by one, otherwise after critical word it must cost more cycle to fill the remaining cache line. For Early First CPU will restart until critical word in cache line reaches. Hash value is critical word, so in this patch we place it as first member in cache line (sock address is cache-line aligned), and it is also good for Early Restart platform as well . [edumazet: respin on net-next after commit ce43b03e8889] Signed-off-by: Ma Ling Signed-off-by: Eric Dumazet Cc: Maciej Żenczykowski --- include/net/sock.h | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/include/net/sock.h b/include/net/sock.h index a340ab4..efabd9a 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -131,12 +131,12 @@ typedef __u64 __bitwise __addrpair; /** * struct sock_common - minimal network layer representation of sockets - * @skc_daddr: Foreign IPv4 addr - * @skc_rcv_saddr: Bound local IPv4 addr * @skc_hash: hash value used with various protocol lookup tables * @skc_u16hashes: two u16 hash values used by UDP lookup tables * @skc_dport: placeholder for inet_dport/tw_dport * @skc_num: placeholder for inet_num/tw_num + * @skc_daddr: Foreign IPv4 addr + * @skc_rcv_saddr: Bound local IPv4 addr * @skc_family: network address family * @skc_state: Connection state * @skc_reuse: %SO_REUSEADDR setting @@ -153,18 +153,10 @@ typedef __u64 __bitwise __addrpair; * * This is the minimal network layer representation of sockets, the header * for struct sock and struct inet_timewait_sock. + * Order of first fields is critical for __inet_lookup_established() : + * skc_hash, skc_portpair, skc_addrpair */ struct sock_common { - /* skc_daddr and skc_rcv_saddr must be grouped on a 8 bytes aligned - * address on 64bit arches : cf INET_MATCH() and INET_TW_MATCH() - */ - union { - __addrpair skc_addrpair; - struct { - __be32 skc_daddr; - __be32 skc_rcv_saddr; - }; - }; union { unsigned int skc_hash; __u16 skc_u16hashes[2]; @@ -178,6 +170,16 @@ struct sock_common { }; }; + /* skc_daddr and skc_rcv_saddr must be grouped on a 8 bytes aligned + * address on 64bit arches : cf INET_MATCH() and INET_TW_MATCH() + */ + union { + __addrpair skc_addrpair; + struct { + __be32 skc_daddr; + __be32 skc_rcv_saddr; + }; + }; unsigned short skc_family; volatile unsigned char skc_state; unsigned char skc_reuse:4;