From patchwork Wed Jan 2 09:36:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vakul Garg X-Patchwork-Id: 1019944 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=nxp.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=nxp.com header.i=@nxp.com header.b="iaT2Hqsn"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 43V5YF60NCz9s7T for ; Wed, 2 Jan 2019 20:37:05 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727631AbfABJhE (ORCPT ); Wed, 2 Jan 2019 04:37:04 -0500 Received: from mail-eopbgr130055.outbound.protection.outlook.com ([40.107.13.55]:48832 "EHLO EUR01-HE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726736AbfABJhE (ORCPT ); Wed, 2 Jan 2019 04:37:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PFkmbluLAbDTUyUOFuddMZm9E/m5slWBcK2Kp4e+kD8=; b=iaT2HqsnWD5UH3ZhD2DoUiVECcMB/eEOOwbEwaNBwlBUVYFTVZiKMBfhHcT9i4RWs2J7B1xLLM8ZbJ4UcwzUv3gzzUhlamskDt3sw8j8fNbpJX4iYf40XoEtIk/oNshuVjLRTSM64ul1x+tl/9bGdq0MeMZj5BhnoqEOe8BuRS8= Received: from DB7PR04MB4252.eurprd04.prod.outlook.com (52.135.131.26) by DB7PR04MB4412.eurprd04.prod.outlook.com (52.135.137.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1495.6; Wed, 2 Jan 2019 09:36:59 +0000 Received: from DB7PR04MB4252.eurprd04.prod.outlook.com ([fe80::ddb0:738d:4c88:b3ed]) by DB7PR04MB4252.eurprd04.prod.outlook.com ([fe80::ddb0:738d:4c88:b3ed%3]) with mapi id 15.20.1471.019; Wed, 2 Jan 2019 09:36:59 +0000 From: Vakul Garg To: "netdev@vger.kernel.org" CC: "john.fastabend@gmail.com" , "daniel@iogearbox.net" , "davejwatson@fb.com" , "davem@davemloft.net" , Vakul Garg Subject: [PATCH net-next] Optimize sk_msg_clone() by data merge to end dst sg entry Thread-Topic: [PATCH net-next] Optimize sk_msg_clone() by data merge to end dst sg entry Thread-Index: AQHUon61k1PUcqBsQUi6ZyNg/VZYDQ== Date: Wed, 2 Jan 2019 09:36:59 +0000 Message-ID: <20190102093447.29832-1-vakul.garg@nxp.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BM1PR0101CA0053.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:19::15) To DB7PR04MB4252.eurprd04.prod.outlook.com (2603:10a6:5:27::26) authentication-results: spf=none (sender IP is ) smtp.mailfrom=vakul.garg@nxp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [14.143.30.134] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB7PR04MB4412; 6:IpZQot3QnD5kWifaykA/iRI5T4btwtmZ4pt66SbTaaIbdyAxgSmTcfrvcjZI5Kg461K+RswMkoAPeYvX7HvN7g7XwbGxqtDkRDwNgCo/OZ2WU17vTcwtFAC0Ew3NF4Kxx8UGNmMbQ54LXvaQnwnd+TDov85UtdUmADwxMI7XcxlLTgwmQMe9ERJ8lcQAR0z6kvNIGL9pXcJVKOOL915F1J1v9BULF7x7ogwxyNm1M//Bdhj0Wdp+MQ6cyLVQp7aelfSv2Uybc30g0sgbzy7IjNjt42sr4UKoD/XZGrdvX5Jye1senjELS42Ozx/KTkgacN3lCkzznxgkEAGCerMg4UhVGfZRSID+OCz2XIW2UTLXSqn6eG9q4jBy+94Ec7qy3plNx2aXasGn3TxUYjAOP/ZiPhLDfUAJvowbEaXDHw+gGS9+EwTVGz10sVVkEAOdZqggnft7avBocPkgv+dkSA==; 5:q+e11q55hT6YdhXhqQTbn/0cWDsU587n/tAeWquJA9ARLNYBSpRE01GPD7QQ7TtgN7VPLWUZe9P5sLs2DN2LKbuSL6IG8O5bH8JFn0wPHBrdnUJoJlOnFQ7Fo7Nj2G121tGQZb3mtO/dXkODEZhHM7y3gmzWnmeIEf4F4ytZHVUPtBRTMdzC0XB57Lh1PtKv/ZxoXhnpC8cA3w8z4nDK7g==; 7:p8hFJvaBiY/lLd9nflrNDLCL3h2PFxvtmlIBuhzvyX+pQLzW8hdAcDZ5mbT8zMrmZsUEJBdMbfc6DZcnHMS5CFfApRZoAtsJKOoBMZgHr+jSt3EHRIHO2GJ4+mRwStkS90pKP8XXSPCyaj6JqWlkeQ== x-ms-office365-filtering-correlation-id: 025538f7-4f8f-4edd-c2f5-08d67095d7f0 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600109)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:DB7PR04MB4412; x-ms-traffictypediagnostic: DB7PR04MB4412: x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(3230021)(908002)(999002)(5005026)(6040522)(8220060)(2401047)(8121501046)(93006095)(93001095)(10201501046)(3231475)(944501520)(52105112)(3002001)(6055026)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123560045)(20161123558120)(20161123562045)(201708071742011)(7699051)(76991095); SRVR:DB7PR04MB4412; BCL:0; PCL:0; RULEID:; SRVR:DB7PR04MB4412; x-forefront-prvs: 0905A6B2C7 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(39860400002)(136003)(396003)(376002)(346002)(189003)(199004)(25786009)(39060400002)(4326008)(2501003)(105586002)(6486002)(5640700003)(6512007)(6116002)(3846002)(86362001)(5660300001)(6436002)(2351001)(71190400001)(53936002)(106356001)(6916009)(78486014)(71200400001)(256004)(36756003)(6506007)(386003)(102836004)(1076003)(55236004)(52116002)(26005)(7736002)(1730700003)(81156014)(81166006)(478600001)(8676002)(316002)(14454004)(8936002)(44832011)(486006)(186003)(99286004)(2616005)(476003)(97736004)(2906002)(305945005)(66066001)(54906003)(68736007); DIR:OUT; SFP:1101; SCL:1; SRVR:DB7PR04MB4412; H:DB7PR04MB4252.eurprd04.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: wLfJ8NHdx0Xu4cYlPRFo5bV6ElctVS+fsGf/i1ul7QUjd7t2bq+yhKejfn7UHJSRyVhtn5H3Wfo821H3sv+j1gW1AifXh8+OPdihMzyoXTov5IF9zF6/PPCoImpVg0xMDw5GtqvOQbcmHWjngXzkpl++rDklomCr/koOYG8qhK1nbEKWppGxd5uq9HuziCanTAMXX8dMzzXsWkkZwhYFvqPg5kIcyNDuFY90h2Qwt5nRCiFfwY2YGbddSG5B+qSTg/0vQYTlNv0sjCSrtYCeiZit29KVqapOP1ZqaZkvfxymvcMeLpFICsMwP7SYCjX5 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 025538f7-4f8f-4edd-c2f5-08d67095d7f0 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Jan 2019 09:36:59.8571 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR04MB4412 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Function sk_msg_clone has been modified to merge the data from source sg entry to destination sg entry if the cloned data resides in same page and is contiguous to the end entry of destination sk_msg. This improves kernel tls throughput to the tune of 10%. When the user space tls application calls sendmsg() with MSG_MORE, it leads to calling sk_msg_clone() with new data being cloned placed continuous to previously cloned data. Without this optimization, a new SG entry in the destination sk_msg i.e. rec->msg_plaintext in tls_clone_plaintext_msg() gets used. This leads to exhaustion of sg entries in rec->msg_plaintext even before a full 16K of allowable record data is accumulated. Hence we lose oppurtunity to encrypt and send a full 16K record. With this patch, the kernel tls can accumulate full 16K of record data irrespective of the size of data passed in sendmsg() with MSG_MORE. Signed-off-by: Vakul Garg --- net/core/skmsg.c | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/net/core/skmsg.c b/net/core/skmsg.c index 26458876d763..f15393ab7fe1 100644 --- a/net/core/skmsg.c +++ b/net/core/skmsg.c @@ -78,11 +78,9 @@ int sk_msg_clone(struct sock *sk, struct sk_msg *dst, struct sk_msg *src, { int i = src->sg.start; struct scatterlist *sge = sk_msg_elem(src, i); + struct scatterlist *sgd = NULL; u32 sge_len, sge_off; - if (sk_msg_full(dst)) - return -ENOSPC; - while (off) { if (sge->length > off) break; @@ -94,16 +92,27 @@ int sk_msg_clone(struct sock *sk, struct sk_msg *dst, struct sk_msg *src, } while (len) { - if (sk_msg_full(dst)) - return -ENOSPC; - sge_len = sge->length - off; - sge_off = sge->offset + off; if (sge_len > len) sge_len = len; + + if (dst->sg.end) + sgd = sk_msg_elem(dst, dst->sg.end - 1); + + if (sgd && + (sg_page(sge) == sg_page(sgd)) && + (sg_virt(sge) + off == sg_virt(sgd) + sgd->length)) { + sgd->length += sge_len; + dst->sg.size += sge_len; + } else if (!sk_msg_full(dst)) { + sge_off = sge->offset + off; + sk_msg_page_add(dst, sg_page(sge), sge_len, sge_off); + } else { + return -ENOSPC; + } + off = 0; len -= sge_len; - sk_msg_page_add(dst, sg_page(sge), sge_len, sge_off); sk_mem_charge(sk, sge_len); sk_msg_iter_var_next(i); if (i == src->sg.end && len)