From patchwork Sat Jul 21 18:03:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 947401 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="KPJZWCOy"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41Xwf65VpSz9s29 for ; Sun, 22 Jul 2018 04:05:34 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728377AbeGUS7H (ORCPT ); Sat, 21 Jul 2018 14:59:07 -0400 Received: from mail-pl0-f65.google.com ([209.85.160.65]:35051 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728302AbeGUS7G (ORCPT ); Sat, 21 Jul 2018 14:59:06 -0400 Received: by mail-pl0-f65.google.com with SMTP id w3-v6so6508059plq.2 for ; Sat, 21 Jul 2018 11:05:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=BkHtJrGhTv+n45V+wRH+0FHm6+I51mrAfgPRffMIkDg=; b=KPJZWCOyvOe0POiP0O97H0B3WyFYsy6Ex+irKIW23WVBz8ehQIlTMrPPjdDLulgx6P dW/+IcZoG7gPur0Qx8cW6p6jb515uJyHJXuU+UI6ud52owv4wwqIDoMbFwuibLZFZqMh JGupDK5pDdX9+Kfnl5/uMOs9f3npu069WxsA5AbAMBWl/fOxUlu25s96lUJ6wZvAfKpt PjSZ2S9hXJPIn1/+1FSo+69Igc2ewpppnFZSKL+mgW6hT77SG5dpR/Rwnn1Z2IMCq5UY IERywHHki1O5atHWQUzZZNU8/PpFMVifJFqexQEoVuAkAJw0br5+4byHjHSfBxg06F7J vHbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=BkHtJrGhTv+n45V+wRH+0FHm6+I51mrAfgPRffMIkDg=; b=ac92MedejjXn6MqUA8lfKBhLtY/g+wChnUdFqcW2r8uXWXE4Qehap72OBX3lTcVtP1 IkgyJZ77eJSTmVcfwI9Wbk4iBHIfF19xpWgW68ndu+ol9S4NC0I5h1iz3orjl+tPVMiH LTmv2vt8uEB0IL78nXrAqHckzYB2jr2MtucRcoalA3dRLERre+I7imb7Y4UTLC9F2Ox1 1/mcxIvduyaxnCtyvnjTT1GkqNsD5YbsZDp/srTqWhZy1db/VpKwksfObf/J+iMXjaQl Pj6zZWcMyjMfSFhSLw6YNkYFrx1jfOYDjKoW+IUvK7oV9AXJnMR7iVfRqMiyNjLUsxAL H3+g== X-Gm-Message-State: AOUpUlEmLOf7e8Cp4sqShN4j3ryeIavTNBV3VeRqhZ4CdcyHrM4jGTY4 Uz75/nK1Ucl8ppyiegqknBOpuw6q X-Google-Smtp-Source: AAOMgpdsw+yzMtIXNlJ3tGYdZHuGuH9b7baDMxBUpqb0lj9eEKUxNZtMa18zDGhpB6BxMuP80TFHxA== X-Received: by 2002:a17:902:7482:: with SMTP id h2-v6mr6737213pll.185.1532196332599; Sat, 21 Jul 2018 11:05:32 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([36.102.208.98]) by smtp.gmail.com with ESMTPSA id l71-v6sm7063679pgd.70.2018.07.21.11.05.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Jul 2018 11:05:32 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: jasowang@redhat.com Cc: mst@redhat.com, makita.toshiaki@lab.ntt.co.jp, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Tonghao Zhang Subject: [PATCH net-next v6 1/4] net: vhost: lock the vqs one by one Date: Sat, 21 Jul 2018 11:03:59 -0700 Message-Id: <1532196242-2998-2-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1532196242-2998-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1532196242-2998-1-git-send-email-xiangxia.m.yue@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tonghao Zhang This patch changes the way that lock all vqs at the same, to lock them one by one. It will be used for next patch to avoid the deadlock. Signed-off-by: Tonghao Zhang Acked-by: Jason Wang Signed-off-by: Jason Wang --- drivers/vhost/vhost.c | 24 +++++++----------------- 1 file changed, 7 insertions(+), 17 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index a502f1a..a1c06e7 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -294,8 +294,11 @@ static void vhost_vq_meta_reset(struct vhost_dev *d) { int i; - for (i = 0; i < d->nvqs; ++i) + for (i = 0; i < d->nvqs; ++i) { + mutex_lock(&d->vqs[i]->mutex); __vhost_vq_meta_reset(d->vqs[i]); + mutex_unlock(&d->vqs[i]->mutex); + } } static void vhost_vq_reset(struct vhost_dev *dev, @@ -890,20 +893,6 @@ static inline void __user *__vhost_get_user(struct vhost_virtqueue *vq, #define vhost_get_used(vq, x, ptr) \ vhost_get_user(vq, x, ptr, VHOST_ADDR_USED) -static void vhost_dev_lock_vqs(struct vhost_dev *d) -{ - int i = 0; - for (i = 0; i < d->nvqs; ++i) - mutex_lock_nested(&d->vqs[i]->mutex, i); -} - -static void vhost_dev_unlock_vqs(struct vhost_dev *d) -{ - int i = 0; - for (i = 0; i < d->nvqs; ++i) - mutex_unlock(&d->vqs[i]->mutex); -} - static int vhost_new_umem_range(struct vhost_umem *umem, u64 start, u64 size, u64 end, u64 userspace_addr, int perm) @@ -953,7 +942,10 @@ static void vhost_iotlb_notify_vq(struct vhost_dev *d, if (msg->iova <= vq_msg->iova && msg->iova + msg->size - 1 > vq_msg->iova && vq_msg->type == VHOST_IOTLB_MISS) { + mutex_lock(&node->vq->mutex); vhost_poll_queue(&node->vq->poll); + mutex_unlock(&node->vq->mutex); + list_del(&node->node); kfree(node); } @@ -985,7 +977,6 @@ static int vhost_process_iotlb_msg(struct vhost_dev *dev, int ret = 0; mutex_lock(&dev->mutex); - vhost_dev_lock_vqs(dev); switch (msg->type) { case VHOST_IOTLB_UPDATE: if (!dev->iotlb) { @@ -1019,7 +1010,6 @@ static int vhost_process_iotlb_msg(struct vhost_dev *dev, break; } - vhost_dev_unlock_vqs(dev); mutex_unlock(&dev->mutex); return ret; From patchwork Sat Jul 21 18:04:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 947403 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Np3Uf38U"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41Xwf95M5Jz9s4s for ; Sun, 22 Jul 2018 04:05:37 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728302AbeGUS7K (ORCPT ); Sat, 21 Jul 2018 14:59:10 -0400 Received: from mail-pl0-f67.google.com ([209.85.160.67]:36168 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728068AbeGUS7J (ORCPT ); Sat, 21 Jul 2018 14:59:09 -0400 Received: by mail-pl0-f67.google.com with SMTP id e11-v6so6507759plb.3 for ; Sat, 21 Jul 2018 11:05:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=WfNlH7MD4766tSpdqr+7VWReovyiYb37tOAWwj5vtoA=; b=Np3Uf38U0D6BV9d2mt+2ysjM/qQtzT5k+NnhyNfcvI9dY6te1VnK0zlQr1hxWh26DQ wfkUcTi4vt8uc7HDOhdMrTa0S/6KGIG8eMI4HlMwaEEXTejkCSjQg3y6t0z80VVTB0B5 C9waYDopDYIjwpdQVKnn4AlYhJajZyH60INO6fsQxvdEW8ErJVj8UPNeSEy0cOLuNsc+ r8fRCQgFTYkBHqRZTjq8Cqx8RVbEkQW9wpklBRbvDh0OtZXZd8BLP7ZLXOgFCWWEJRP6 Y8gsDgGJuuwbUKNAMVdOjZ9C4WyYaMcYat6sR3LqQm4wiQCUxM59NaqZe7zsdy9xIZc0 2rlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=WfNlH7MD4766tSpdqr+7VWReovyiYb37tOAWwj5vtoA=; b=rdswjTUzNRejsyZXP2XwpvUVrdhqepc71nYc+TjFF8UrGy1q/M9xrVhQhM92puKCjs zP2NVPoctpeeQVNti1AkAQOofgj99PYWhWFZeJAeWiEGWE1u+a61jZzT2ktbczqxQfcu /ZsVfN8DU1PBWaC3WjROEz5Nl4jSq9na9DmA+Q6ThjmYKAYd5GFO7tk26FxfLZeKgk4r SnvkSAyjh4DzJlR06ugKXdVhX9Crnw1tGs6ubCzRtxIoR/wSGiifJpLEZ3PWgVNuurD8 gcthVZFfnC5PT5aaP6aBAtPdpOaEs5IEx4K46CZvKbYmTVLkTpKGO6GjAwwiUOgZQue1 /GSg== X-Gm-Message-State: AOUpUlHqxZuk4FePjJJVsp8GFCRWFMWtQi8vn5KqPV2FdIfLCHQtuaue aVSU8grKDiMTofkhEo4Ys/o= X-Google-Smtp-Source: AAOMgpfLCc2wqXVc43QnswU3NtULcMwj1/HrfX+Cwmf5/l5bNA/gMDkvCoxCwtggEOkSiIQXA52QPg== X-Received: by 2002:a17:902:7781:: with SMTP id o1-v6mr6601295pll.48.1532196335011; Sat, 21 Jul 2018 11:05:35 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([36.102.208.98]) by smtp.gmail.com with ESMTPSA id l71-v6sm7063679pgd.70.2018.07.21.11.05.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Jul 2018 11:05:34 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: jasowang@redhat.com Cc: mst@redhat.com, makita.toshiaki@lab.ntt.co.jp, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Tonghao Zhang Subject: [PATCH net-next v6 2/4] net: vhost: replace magic number of lock annotation Date: Sat, 21 Jul 2018 11:04:00 -0700 Message-Id: <1532196242-2998-3-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1532196242-2998-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1532196242-2998-1-git-send-email-xiangxia.m.yue@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tonghao Zhang Use the VHOST_NET_VQ_XXX as a subclass for mutex_lock_nested. Signed-off-by: Tonghao Zhang Acked-by: Jason Wang --- drivers/vhost/net.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index b224036..321264c 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -490,7 +490,7 @@ static void handle_tx(struct vhost_net *net) bool zcopy, zcopy_used; int sent_pkts = 0; - mutex_lock(&vq->mutex); + mutex_lock_nested(&vq->mutex, VHOST_NET_VQ_TX); sock = vq->private_data; if (!sock) goto out; @@ -667,7 +667,7 @@ static int vhost_net_rx_peek_head_len(struct vhost_net *net, struct sock *sk, /* Flush batched heads first */ vhost_rx_signal_used(rnvq); /* Both tx vq and rx socket were polled here */ - mutex_lock_nested(&tvq->mutex, 1); + mutex_lock_nested(&tvq->mutex, VHOST_NET_VQ_TX); vhost_disable_notify(&net->dev, tvq); preempt_disable(); @@ -809,7 +809,7 @@ static void handle_rx(struct vhost_net *net) __virtio16 num_buffers; int recv_pkts = 0; - mutex_lock_nested(&vq->mutex, 0); + mutex_lock_nested(&vq->mutex, VHOST_NET_VQ_RX); sock = vq->private_data; if (!sock) goto out; From patchwork Sat Jul 21 18:04:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 947404 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="b0/LAb0a"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41XwfC6VvXz9s29 for ; Sun, 22 Jul 2018 04:05:39 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728395AbeGUS7M (ORCPT ); Sat, 21 Jul 2018 14:59:12 -0400 Received: from mail-pl0-f65.google.com ([209.85.160.65]:37136 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728068AbeGUS7L (ORCPT ); Sat, 21 Jul 2018 14:59:11 -0400 Received: by mail-pl0-f65.google.com with SMTP id 31-v6so6523359plc.4 for ; Sat, 21 Jul 2018 11:05:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=eX0KNIUYkTNZUBj77kzJHKozoz4Pp3eD19rs5JvR5HA=; b=b0/LAb0a7FOoECkrwS+YmKYy2zbBXZwR0gUJcPcJJDMI7eyFno2+0EGpJYuU8ugDfZ axXysNnu9hp97I4IvzDBwZ7nZVPrpYM3iT+88TfRZncSfHtDXUehkgLrSOpw44X1tl7M A44XZRwkZLLvGT4UD0FSSD0CM4IAb0726wem6gWSBYj4E2RjT41Dc77LuFF9CzWIpt7A LXmuIn8D6eeYaUH0qAGY5vjAGwZ6p1BmiFfw0oNJ7a5sln6nFvbJtGggYQm2rtsLk/wE GdEGNr4pwrLbKYNOXtNF7a2wKzOaxHVfBrYpdkYLk9nNqpssUpdXI35YvaTecMA7cKIC QkwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=eX0KNIUYkTNZUBj77kzJHKozoz4Pp3eD19rs5JvR5HA=; b=ROL06FduKewjWcKEpCR47UxGHtNYwq7JsIa4qGzwPrSSGeg1mwXnEcNjKmJnNrzYIh J5K366Y9UpD8WVl3o0kkgLbO8GCj+X8GHfl7WSAJvb1RY/zbM1hv5xpbik8WJm6MyEmj dqgoHx3MMzNA5j7bgcyYRthSKzZGqfGBwHYcAxQyGMpaEpjo6gx6t9Jwjmd7MN4+TbhL QcwysYyrrcNKd3PebXvr9yf2IjkrPLW5cPtfSdZLPkB6eoyRvQwsPfxFTQVPUNbKhu26 V/e3/vu1FGKiThO3p5wJPhiQH9mHQQwR0cJViPjaYG/RMw0wP5HZx4pyQU6VTStgdJXm vXyw== X-Gm-Message-State: AOUpUlEW02d/5cDTbmNMrSYKqQvcDzzmWZezItHMEVYtMwybbWthmJm0 TUHRzJR8Tn5SEXgCy/sWR8o= X-Google-Smtp-Source: AAOMgpeLEt8Ctl0qTuLSvM6A1d6YMzsibXApXtXhwuNNBomZff19FxoFeS8ZszBSnOBywn8wWF+aKg== X-Received: by 2002:a17:902:262:: with SMTP id 89-v6mr6587929plc.221.1532196337408; Sat, 21 Jul 2018 11:05:37 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([36.102.208.98]) by smtp.gmail.com with ESMTPSA id l71-v6sm7063679pgd.70.2018.07.21.11.05.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Jul 2018 11:05:36 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: jasowang@redhat.com Cc: mst@redhat.com, makita.toshiaki@lab.ntt.co.jp, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Tonghao Zhang Subject: [PATCH net-next v6 3/4] net: vhost: factor out busy polling logic to vhost_net_busy_poll() Date: Sat, 21 Jul 2018 11:04:01 -0700 Message-Id: <1532196242-2998-4-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1532196242-2998-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1532196242-2998-1-git-send-email-xiangxia.m.yue@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tonghao Zhang Factor out generic busy polling logic and will be used for in tx path in the next patch. And with the patch, qemu can set differently the busyloop_timeout for rx queue. Signed-off-by: Tonghao Zhang --- drivers/vhost/net.c | 114 ++++++++++++++++++++++++++++++++++------------------ 1 file changed, 74 insertions(+), 40 deletions(-) diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index 321264c..2dc937e 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -428,6 +428,78 @@ static int vhost_net_enable_vq(struct vhost_net *n, return vhost_poll_start(poll, sock->file); } +static int sk_has_rx_data(struct sock *sk) +{ + struct socket *sock = sk->sk_socket; + + if (sock->ops->peek_len) + return sock->ops->peek_len(sock); + + return skb_queue_empty(&sk->sk_receive_queue); +} + +static void vhost_net_busy_poll_vq_check(struct vhost_net *net, + struct vhost_virtqueue *rvq, + struct vhost_virtqueue *tvq, + bool rx) +{ + struct socket *sock = rvq->private_data; + + if (rx) { + if (!vhost_vq_avail_empty(&net->dev, tvq)) { + vhost_poll_queue(&tvq->poll); + } else if (unlikely(vhost_enable_notify(&net->dev, tvq))) { + vhost_disable_notify(&net->dev, tvq); + vhost_poll_queue(&tvq->poll); + } + } else if ((sock && sk_has_rx_data(sock->sk)) && + !vhost_vq_avail_empty(&net->dev, rvq)) { + vhost_poll_queue(&rvq->poll); + } +} + +static void vhost_net_busy_poll(struct vhost_net *net, + struct vhost_virtqueue *rvq, + struct vhost_virtqueue *tvq, + bool *busyloop_intr, + bool rx) +{ + unsigned long busyloop_timeout; + unsigned long endtime; + struct socket *sock; + struct vhost_virtqueue *vq = rx ? tvq : rvq; + + mutex_lock_nested(&vq->mutex, rx ? VHOST_NET_VQ_TX: VHOST_NET_VQ_RX); + vhost_disable_notify(&net->dev, vq); + sock = rvq->private_data; + + busyloop_timeout = rx ? rvq->busyloop_timeout: + tvq->busyloop_timeout; + + preempt_disable(); + endtime = busy_clock() + busyloop_timeout; + + while (vhost_can_busy_poll(endtime)) { + if (vhost_has_work(&net->dev)) { + *busyloop_intr = true; + break; + } + + if ((sock && sk_has_rx_data(sock->sk) && + !vhost_vq_avail_empty(&net->dev, rvq)) || + !vhost_vq_avail_empty(&net->dev, tvq)) + break; + + cpu_relax(); + } + + preempt_enable(); + + vhost_net_busy_poll_vq_check(net, rvq, tvq, rx); + + mutex_unlock(&vq->mutex); +} + static int vhost_net_tx_get_vq_desc(struct vhost_net *net, struct vhost_virtqueue *vq, struct iovec iov[], unsigned int iov_size, @@ -631,16 +703,6 @@ static int peek_head_len(struct vhost_net_virtqueue *rvq, struct sock *sk) return len; } -static int sk_has_rx_data(struct sock *sk) -{ - struct socket *sock = sk->sk_socket; - - if (sock->ops->peek_len) - return sock->ops->peek_len(sock); - - return skb_queue_empty(&sk->sk_receive_queue); -} - static void vhost_rx_signal_used(struct vhost_net_virtqueue *nvq) { struct vhost_virtqueue *vq = &nvq->vq; @@ -660,41 +722,13 @@ static int vhost_net_rx_peek_head_len(struct vhost_net *net, struct sock *sk, struct vhost_net_virtqueue *tnvq = &net->vqs[VHOST_NET_VQ_TX]; struct vhost_virtqueue *rvq = &rnvq->vq; struct vhost_virtqueue *tvq = &tnvq->vq; - unsigned long uninitialized_var(endtime); int len = peek_head_len(rnvq, sk); - if (!len && tvq->busyloop_timeout) { + if (!len && rvq->busyloop_timeout) { /* Flush batched heads first */ vhost_rx_signal_used(rnvq); /* Both tx vq and rx socket were polled here */ - mutex_lock_nested(&tvq->mutex, VHOST_NET_VQ_TX); - vhost_disable_notify(&net->dev, tvq); - - preempt_disable(); - endtime = busy_clock() + tvq->busyloop_timeout; - - while (vhost_can_busy_poll(endtime)) { - if (vhost_has_work(&net->dev)) { - *busyloop_intr = true; - break; - } - if ((sk_has_rx_data(sk) && - !vhost_vq_avail_empty(&net->dev, rvq)) || - !vhost_vq_avail_empty(&net->dev, tvq)) - break; - cpu_relax(); - } - - preempt_enable(); - - if (!vhost_vq_avail_empty(&net->dev, tvq)) { - vhost_poll_queue(&tvq->poll); - } else if (unlikely(vhost_enable_notify(&net->dev, tvq))) { - vhost_disable_notify(&net->dev, tvq); - vhost_poll_queue(&tvq->poll); - } - - mutex_unlock(&tvq->mutex); + vhost_net_busy_poll(net, rvq, tvq, busyloop_intr, true); len = peek_head_len(rnvq, sk); } From patchwork Sat Jul 21 18:04:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 947405 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Wudr6W50"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41XwfG1LlTz9s29 for ; Sun, 22 Jul 2018 04:05:42 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728418AbeGUS7O (ORCPT ); Sat, 21 Jul 2018 14:59:14 -0400 Received: from mail-pg1-f193.google.com ([209.85.215.193]:39893 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728068AbeGUS7O (ORCPT ); Sat, 21 Jul 2018 14:59:14 -0400 Received: by mail-pg1-f193.google.com with SMTP id g2-v6so9344627pgs.6 for ; Sat, 21 Jul 2018 11:05:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=xwCfOzA8GWxUM/VcWZ//sLoLyZDT3gUi7utdPfd7H+8=; b=Wudr6W50VP7R3O4L5wAw7Vs9IAGIIYX8qV2tx3fLt71F0nE2kNIUGuST0ODmbfQv2o rm4yI24hiJHCRmDHzxeQ+2e/DmsCa66Dp1FSOSv2zFYI5bUFUsLrm0bD/9svv7SEkXdu yciZmNgFmn8JvlaAb74Mwxpz3fILFK02J3TE773FbVchst4sWRQnpMZ9zexbv8ywJsWT TakUHGl0CAn5VowhQZhjTAoEsJhewLquXpEnkA4vaRGA2nPOipm+8VxcuGRlu/dDewaQ xaTMjfxxjA5XMSlJErFxY6d0pXyptUhmF68dNwnnkiq67AXePjbsY7mUtvOMSr/HYwI6 auXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=xwCfOzA8GWxUM/VcWZ//sLoLyZDT3gUi7utdPfd7H+8=; b=AkD3uJqiA1/suVDEuCzGyrL4XIVMFFRusdwY4pnYYpSKA79IRtJJZOm/2X8/RJKjN3 YaOq7CwrQgF4hw8MQCOJAFo8BajaqUKmDfph5cE47MUXcuA9h3SijbUNjGqoQVYrkaHF ZNQtL8GiZ8lPOqV41yiEqrAa6aBN/IRFlt/0QyW/XPiaWP/i6PzUGPgLRLtVv6oT5hvH hz+68ccjtce0rPc4CUbjzB7jM9wQelonHtGJN+5R8F97Qbrex53aHGu9wCuMeFKlKZtM 3P14RdpVg44wSiQ6ogOPAILm/e7JOI7jiv/3cYCIEQZV0UAre8LGq4RCrVddovFv6Ggx 0Zfg== X-Gm-Message-State: AOUpUlGIoUvB0Jlslgdd5UqGOr7aaQiKjxKOfnXWdTTfS7dlfPVM+phE +SU3+lQu+OgGsK9HxVkPFWA= X-Google-Smtp-Source: AAOMgpex8H2MOgwd0WrnJR+hWrO95F8bXCcTA9L6l15IaYbm4HcmAYwb96TPAFv1Np8I7MtsQtumqQ== X-Received: by 2002:a62:198e:: with SMTP id 136-v6mr6852983pfz.103.1532196339897; Sat, 21 Jul 2018 11:05:39 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([36.102.208.98]) by smtp.gmail.com with ESMTPSA id l71-v6sm7063679pgd.70.2018.07.21.11.05.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Jul 2018 11:05:39 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: jasowang@redhat.com Cc: mst@redhat.com, makita.toshiaki@lab.ntt.co.jp, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Tonghao Zhang Subject: [PATCH net-next v6 4/4] net: vhost: add rx busy polling in tx path Date: Sat, 21 Jul 2018 11:04:02 -0700 Message-Id: <1532196242-2998-5-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1532196242-2998-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1532196242-2998-1-git-send-email-xiangxia.m.yue@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tonghao Zhang This patch improves the guest receive performance. On the handle_tx side, we poll the sock receive queue at the same time. handle_rx do that in the same way. We set the poll-us=100us and use the netperf to test throughput and mean latency. When running the tests, the vhost-net kthread of that VM, is alway 100% CPU. The commands are shown as below. Rx performance is greatly improved by this patch. There is not notable performance change on tx with this series though. This patch is useful for bi-directional traffic. netperf -H IP -t TCP_STREAM -l 20 -- -O "THROUGHPUT, THROUGHPUT_UNITS, MEAN_LATENCY" Topology: [Host] ->linux bridge -> tap vhost-net ->[Guest] TCP_STREAM: * Without the patch: 19842.95 Mbps, 6.50 us mean latency * With the patch: 37373.10 Mbps, 3.43 us mean latency Signed-off-by: Tonghao Zhang --- drivers/vhost/net.c | 30 +++++++++++------------------- 1 file changed, 11 insertions(+), 19 deletions(-) diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index 2dc937e..a562828 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -501,29 +501,21 @@ static void vhost_net_busy_poll(struct vhost_net *net, } static int vhost_net_tx_get_vq_desc(struct vhost_net *net, - struct vhost_virtqueue *vq, + struct vhost_virtqueue *tvq, struct iovec iov[], unsigned int iov_size, unsigned int *out_num, unsigned int *in_num, bool *busyloop_intr) { - unsigned long uninitialized_var(endtime); - int r = vhost_get_vq_desc(vq, vq->iov, ARRAY_SIZE(vq->iov), - out_num, in_num, NULL, NULL); - - if (r == vq->num && vq->busyloop_timeout) { - preempt_disable(); - endtime = busy_clock() + vq->busyloop_timeout; - while (vhost_can_busy_poll(endtime)) { - if (vhost_has_work(vq->dev)) { - *busyloop_intr = true; - break; - } - if (!vhost_vq_avail_empty(vq->dev, vq)) - break; - cpu_relax(); - } - preempt_enable(); - r = vhost_get_vq_desc(vq, vq->iov, ARRAY_SIZE(vq->iov), + struct vhost_net_virtqueue *rnvq = &net->vqs[VHOST_NET_VQ_RX]; + struct vhost_virtqueue *rvq = &rnvq->vq; + + int r = vhost_get_vq_desc(tvq, tvq->iov, ARRAY_SIZE(tvq->iov), + out_num, in_num, NULL, NULL); + + if (r == tvq->num && tvq->busyloop_timeout) { + vhost_net_busy_poll(net, rvq, tvq, busyloop_intr, false); + + r = vhost_get_vq_desc(tvq, tvq->iov, ARRAY_SIZE(tvq->iov), out_num, in_num, NULL, NULL); }