{"id":808534,"url":"http://patchwork.ozlabs.org/api/1.0/patches/808534/?format=json","project":{"id":7,"url":"http://patchwork.ozlabs.org/api/1.0/projects/7/?format=json","name":"Linux network development","link_name":"netdev","list_id":"netdev.vger.kernel.org","list_email":"netdev@vger.kernel.org","web_url":null,"scm_url":null,"webscm_url":null},"msgid":"<20170901062713.1842249-4-kafai@fb.com>","date":"2017-09-01T06:27:13","name":"[net-next,3/3] bpf: Only set node->ref = 1 if it has not been set","commit_ref":null,"pull_url":null,"state":"accepted","archived":true,"hash":"babcdf06c1be0810be672679fdadc09eb689d2e5","submitter":{"id":64907,"url":"http://patchwork.ozlabs.org/api/1.0/people/64907/?format=json","name":"Martin KaFai Lau","email":"kafai@fb.com"},"delegate":{"id":34,"url":"http://patchwork.ozlabs.org/api/1.0/users/34/?format=json","username":"davem","first_name":"David","last_name":"Miller","email":"davem@davemloft.net"},"mbox":"http://patchwork.ozlabs.org/project/netdev/patch/20170901062713.1842249-4-kafai@fb.com/mbox/","series":[{"id":953,"url":"http://patchwork.ozlabs.org/api/1.0/series/953/?format=json","date":"2017-09-01T06:27:10","name":"bpf: Improve LRU map lookup performance","version":1,"mbox":"http://patchwork.ozlabs.org/series/953/mbox/"}],"check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/808534/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (1024-bit key;\n\tunprotected) header.d=fb.com header.i=@fb.com header.b=\"eGVye1+u\";\n\tdkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xk8RY2Rqlz9s72\n\tfor ;\n\tFri, 1 Sep 2017 16:27:21 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751435AbdIAG1T (ORCPT );\n\tFri, 1 Sep 2017 02:27:19 -0400","from mx0a-00082601.pphosted.com ([67.231.145.42]:59520 \"EHLO\n\tmx0a-00082601.pphosted.com\" rhost-flags-OK-OK-OK-OK)\n\tby vger.kernel.org with ESMTP id S1751022AbdIAG1R (ORCPT\n\t); Fri, 1 Sep 2017 02:27:17 -0400","from pps.filterd (m0109334.ppops.net [127.0.0.1])\n\tby mx0a-00082601.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id\n\tv816RHPD024820\n\tfor ; Thu, 31 Aug 2017 23:27:17 -0700","from mail.thefacebook.com ([199.201.64.23])\n\tby mx0a-00082601.pphosted.com with ESMTP id 2cq17wr9sy-2\n\t(version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT)\n\tfor ; Thu, 31 Aug 2017 23:27:17 -0700","from mx-out.facebook.com (192.168.52.123) by\n\tPRN-CHUB13.TheFacebook.com (192.168.16.23) with Microsoft SMTP Server\n\tid 14.3.319.2; Thu, 31 Aug 2017 23:27:14 -0700","by devbig738.prn1.facebook.com (Postfix, from userid 6611) id\n\tCA7C645005A9; Thu, 31 Aug 2017 23:27:13 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com;\n\th=from : to : cc : subject\n\t: date : message-id : in-reply-to : references : mime-version :\n\tcontent-type; s=facebook;\n\tbh=wtqhc9Juj2ibGCfH8u8YtgWDTmYguOMo6/t6FeEPwus=; \n\tb=eGVye1+ufj7Y/xMyZy6na0QCb30rBVkKHHTwmoyns2bCLKXDpmfY2e8IUMsvxjGmy25j\n\tRndYxaIA9OFLs9LeAHgPAB7kIyZ4eibOT1CZOlu4qSuvmLQJSADBgxcqyV7QML9YGhUk\n\t2Xo0EzwiBWudHoHPsrufW90UjHBZ5Vyc0bQ= ","X-ThriftRelayHost":"devbig738.prn1.facebook.com","Smtp-Origin-Hostprefix":"devbig","From":"Martin KaFai Lau ","Smtp-Origin-Hostname":"devbig738.prn1.facebook.com","To":"","CC":"Alexei Starovoitov ,\n\tDaniel Borkmann , ","Smtp-Origin-Cluster":"prn1c29","Subject":"[PATCH net-next 3/3] bpf: Only set node->ref = 1 if it has not been\n\tset","Date":"Thu, 31 Aug 2017 23:27:13 -0700","Message-ID":"<20170901062713.1842249-4-kafai@fb.com>","X-Mailer":"git-send-email 2.9.5","In-Reply-To":"<20170901062713.1842249-1-kafai@fb.com>","References":"<20170901062713.1842249-1-kafai@fb.com>","X-FB-Internal":["Safe","Safe"],"MIME-Version":"1.0","Content-Type":"text/plain","X-Proofpoint-Spam-Reason":"safe","X-Proofpoint-Virus-Version":"vendor=fsecure engine=2.50.10432:, ,\n\tdefinitions=2017-09-01_02:, , signatures=0","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"},"content":"This patch writes 'node->ref = 1' only if node->ref is 0.\nThe number of lookups/s for a ~1M entries LRU map increased by\n~30% (260097 to 343313).\n\nOther writes on 'node->ref = 0' is not changed. In those cases, the\nsame cache line has to be changed anyway.\n\nFirst column: Size of the LRU hash\nSecond column: Number of lookups/s\n\nBefore:\n> echo \"$((2**20+1)): $(./map_perf_test 1024 1 $((2**20+1)) 10000000 | awk '{print $3}')\"\n1048577: 260097\n\nAfter:\n> echo \"$((2**20+1)): $(./map_perf_test 1024 1 $((2**20+1)) 10000000 | awk '{print $3}')\"\n1048577: 343313\n\nSigned-off-by: Martin KaFai Lau \n---\n kernel/bpf/bpf_lru_list.h | 3 ++-\n kernel/bpf/hashtab.c | 7 ++++++-\n 2 files changed, 8 insertions(+), 2 deletions(-)","diff":"diff --git a/kernel/bpf/bpf_lru_list.h b/kernel/bpf/bpf_lru_list.h\nindex 5c35a98d02bf..7d4f89b7cb84 100644\n--- a/kernel/bpf/bpf_lru_list.h\n+++ b/kernel/bpf/bpf_lru_list.h\n@@ -69,7 +69,8 @@ static inline void bpf_lru_node_set_ref(struct bpf_lru_node *node)\n \t/* ref is an approximation on access frequency. It does not\n \t * have to be very accurate. Hence, no protection is used.\n \t */\n-\tnode->ref = 1;\n+\tif (!node->ref)\n+\t\tnode->ref = 1;\n }\n \n int bpf_lru_init(struct bpf_lru *lru, bool percpu, u32 hash_offset,\ndiff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c\nindex 682f4543fefa..431126f31ea3 100644\n--- a/kernel/bpf/hashtab.c\n+++ b/kernel/bpf/hashtab.c\n@@ -519,9 +519,14 @@ static u32 htab_lru_map_gen_lookup(struct bpf_map *map,\n {\n \tstruct bpf_insn *insn = insn_buf;\n \tconst int ret = BPF_REG_0;\n+\tconst int ref_reg = BPF_REG_1;\n \n \t*insn++ = BPF_EMIT_CALL((u64 (*)(u64, u64, u64, u64, u64))__htab_map_lookup_elem);\n-\t*insn++ = BPF_JMP_IMM(BPF_JEQ, ret, 0, 2);\n+\t*insn++ = BPF_JMP_IMM(BPF_JEQ, ret, 0, 4);\n+\t*insn++ = BPF_LDX_MEM(BPF_B, ref_reg, ret,\n+\t\t\t offsetof(struct htab_elem, lru_node) +\n+\t\t\t offsetof(struct bpf_lru_node, ref));\n+\t*insn++ = BPF_JMP_IMM(BPF_JNE, ref_reg, 0, 1);\n \t*insn++ = BPF_ST_MEM(BPF_B, ret,\n \t\t\t offsetof(struct htab_elem, lru_node) +\n \t\t\t offsetof(struct bpf_lru_node, ref),\n","prefixes":["net-next","3/3"]}