From patchwork Sat Dec 21 01:03:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yifeng Sun X-Patchwork-Id: 1214397 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="iIrbGgoV"; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 47fnSH1jsmz9sR1 for ; Sat, 21 Dec 2019 12:03:59 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 9940988A5B; Sat, 21 Dec 2019 01:03:57 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kRq-5Q63fDqA; Sat, 21 Dec 2019 01:03:54 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 15FFE8890B; Sat, 21 Dec 2019 01:03:54 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id EE371C1796; Sat, 21 Dec 2019 01:03:53 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 494B6C077D for ; Sat, 21 Dec 2019 01:03:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 4230888393 for ; Sat, 21 Dec 2019 01:03:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4xOGKja1CP-1 for ; Sat, 21 Dec 2019 01:03:50 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pf1-f193.google.com (mail-pf1-f193.google.com [209.85.210.193]) by whitealder.osuosl.org (Postfix) with ESMTPS id 2C64288392 for ; Sat, 21 Dec 2019 01:03:50 +0000 (UTC) Received: by mail-pf1-f193.google.com with SMTP id w62so6138236pfw.8 for ; Fri, 20 Dec 2019 17:03:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=XUBP4tAVMO/c5+KMxZ9xnbqxYB+Xs3zz90tGm1xUMBE=; b=iIrbGgoV6J76YF7IcuWrE/r/yt9dO1WUEy3t4cLdxEVtkTl8G/2nLMNkskdSeTBtRz Jga3MPGxLuIrhk+sKWj+3fWl66f4ESaQ0GzoQNMF8kXdV7T0IfSXm6+sR7U8/OSkRLMj UpVdYZcVjDjT2JyD2A5h8dPJBvl1c/kYNGk8FV6iTpLm2m1AaqPXxkWjIUGcWsVvGorm ntHNiCiIEi5SFBJ76QQZEvnGB1TmC0tJnrs7doPujwVZ7H9FQUw6b47WjuPWDXZ1EnHY JDsF8sDqtZfXt1dIR61kHv5HN82m22hmbGBsITO1PFCQJPIHEi4Gvj0uw79jufAdWUBK CwFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=XUBP4tAVMO/c5+KMxZ9xnbqxYB+Xs3zz90tGm1xUMBE=; b=Mgo+B3dK5u/qUBudr88AVNVkD9mg2S9cb8L5XxIp7nhppozHFJtMnd4CJHKmZzrF1U a2O5OgPGnawgj8Zbev/mZFfuEgoD2rU/EuY5mwx4jxgKLOiCh/7HZhooMWspjU0WJHG1 rkOKIvO4HSzqF6TuQx1VeRsAt3y53YdHmZu4eMQ6erK4qv45TwLsRtWFra+1dNJolVyJ VZbc3WEOySaYRHgRQd/IRq7K+C+V6Ps9/hpzCt/TcPp1yh4I13BzBxeTsx6c9I8iSgOT IBPY5ATSmlicaSQD/HXbz1hyLMD5q00mT3BXv4C4LMX/3Nj2/N07eyDqfJeadSrWARig x8qQ== X-Gm-Message-State: APjAAAUqgIxXvDRV9EhiGRPA4Efd1S4wxQ7aNoGyKzE/t68I/cCo5esy B04TBkaXB3UzF3NnNvyFEsMRAmLf X-Google-Smtp-Source: APXvYqxpfXoNjpQ1E4dfnKN+IUkJQqyE41NeughVcRKcCml33eoQj4o3AAsf75Iv34BLMb2ayaU0WQ== X-Received: by 2002:aa7:95a9:: with SMTP id a9mr19044964pfk.15.1576890229585; Fri, 20 Dec 2019 17:03:49 -0800 (PST) Received: from kern417.eng.vmware.com ([66.170.99.1]) by smtp.gmail.com with ESMTPSA id j38sm12824931pgj.27.2019.12.20.17.03.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 20 Dec 2019 17:03:49 -0800 (PST) From: Yifeng Sun To: dev@openvswitch.org Date: Fri, 20 Dec 2019 17:03:43 -0800 Message-Id: <1576890223-24011-1-git-send-email-pkusunyifeng@gmail.com> X-Mailer: git-send-email 2.7.4 Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH] afxdp: Reduce afxdp's batch size to match kernel's xdp batch size X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" William reported that there is iperf TCP issue between two afxdp ports: [ 3] local 10.1.1.2 port 40384 connected with 10.1.1.1 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 1.0 sec 17.0 MBytes 143 Mbits/sec [ 3] 1.0- 2.0 sec 9.62 MBytes 80.7 Mbits/sec [ 3] 2.0- 3.0 sec 6.75 MBytes 56.6 Mbits/sec [ 3] 3.0- 4.0 sec 11.0 MBytes 92.3 Mbits/sec [ 3] 5.0- 6.0 sec 0.00 Bytes 0.00 bits/sec [ 3] 6.0- 7.0 sec 0.00 Bytes 0.00 bits/sec [ 3] 7.0- 8.0 sec 0.00 Bytes 0.00 bits/sec [ 3] 8.0- 9.0 sec 0.00 Bytes 0.00 bits/sec [ 3] 9.0-10.0 sec 0.00 Bytes 0.00 bits/sec [ 3] 10.0-11.0 sec 0.00 Bytes 0.00 bits/sec The reason is, currently, netdev-afxdp's batch size is 32 while kernel's xdp batch size is only 16. This can result in exhausting of sock wmem if netdev-afxdp keeps sending large number of packets. Later on, when ARP expires at one side of TCP connection, ARP packets can be delayed or even dropped because sock wmen is already full. This patch fixes this issue by reducing netdev-afxdp's batch size so as to match kernel's xdp batch size. Now iperf TCP works correctly. [ 3] local 10.1.1.2 port 57770 connected with 10.1.1.1 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 1.0 sec 262 MBytes 2.20 Gbits/sec [ 3] 1.0- 2.0 sec 299 MBytes 2.51 Gbits/sec [ 3] 2.0- 3.0 sec 271 MBytes 2.27 Gbits/sec [ 3] 3.0- 4.0 sec 247 MBytes 2.07 Gbits/sec [ 3] 4.0- 5.0 sec 290 MBytes 2.43 Gbits/sec [ 3] 5.0- 6.0 sec 292 MBytes 2.45 Gbits/sec [ 3] 6.0- 7.0 sec 223 MBytes 1.87 Gbits/sec [ 3] 7.0- 8.0 sec 243 MBytes 2.04 Gbits/sec [ 3] 8.0- 9.0 sec 234 MBytes 1.97 Gbits/sec [ 3] 9.0-10.0 sec 238 MBytes 2.00 Gbits/sec Reported-by: William Tu Reported-at: https://mail.openvswitch.org/pipermail/ovs-dev/2019-November/365076.html Signed-off-by: Yifeng Sun --- lib/netdev-afxdp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/netdev-afxdp.c b/lib/netdev-afxdp.c index 58365ed483e3..38bbbeb055cc 100644 --- a/lib/netdev-afxdp.c +++ b/lib/netdev-afxdp.c @@ -82,7 +82,7 @@ static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20); * enough for most corner cases. */ #define NUM_FRAMES (4 * (PROD_NUM_DESCS + CONS_NUM_DESCS)) -#define BATCH_SIZE NETDEV_MAX_BURST +#define BATCH_SIZE 16 BUILD_ASSERT_DECL(IS_POW2(NUM_FRAMES)); BUILD_ASSERT_DECL(PROD_NUM_DESCS == CONS_NUM_DESCS);