From patchwork Fri Aug 10 00:28:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Caleb Raitto X-Patchwork-Id: 955899 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="QeDFn3Nq"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41mmFZ5sfnz9s4v for ; Fri, 10 Aug 2018 10:28:50 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727239AbeHJC4H (ORCPT ); Thu, 9 Aug 2018 22:56:07 -0400 Received: from mail-pl0-f67.google.com ([209.85.160.67]:36536 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727123AbeHJC4G (ORCPT ); Thu, 9 Aug 2018 22:56:06 -0400 Received: by mail-pl0-f67.google.com with SMTP id e11-v6so3255215plb.3; Thu, 09 Aug 2018 17:28:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QAjlmUccx/9CZ/tnsqcSMw/dMfY0KMFs4ShzzElOYnE=; b=QeDFn3NqWUi294f5K+/kyMihKoUj0zB+reWyuH3oLPDab7mQ6f40qjGKn1gnP5qJ3Z IP3GWPSRhuEe/1q3R1EM5j3X3jvrVW36DbfTKjgWKTJJfI2NSBVd4/9G1QDxZDu6ob9p N6leByoKEoz7TnmvTESWe+1IOF/pqO8Vf7141dK6jtEMvj2827eOIuKZ7KoSeTpLLF6i S/vKHP3djFNkEUIacPKXdDMgdfHj2CScNrDe7g7kThKc54yYASsqoKfJzZz5/X1bH62z Q4W4XyD0v6kTxlJHxtHqpmmeSxA3UD8qws4pmC/lUb2EbqS0dSt6QE2GaxAIVijiWALJ TVQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QAjlmUccx/9CZ/tnsqcSMw/dMfY0KMFs4ShzzElOYnE=; b=mOn4TqPQtxUMnTrTew/XVCIYzdWpxLxw2/8vTR3sTiF6E+ohvlR1LE7W9/IG1xhQXd 5PKo3j9kk1+ujlF76O70r3zG9ULAojUm71EFEYcmwnLdXL+5aVzWyCGum8TleP03kgW2 BG49Mu3k5F8w4zLF5eJrahLTHZpv16YqJ0lcYcF0Ei1MlCIf5t9zftCtGKp+diOACFvX o4zId053FUusFl4ukA80zrJwEWYXYpCX8Ua9XyL/wcpOw6wtPqmgp2MuJ1nKOJP+NQ80 eonCsn1JUTGtFY/K1T1g4zOS9lm0bJ5smLp/fhO2/NUhTrZSZQH2sYeRG9iSRghfQpPU bNbQ== X-Gm-Message-State: AOUpUlF/STkcFZcGf0UvXF/sv7lXud7ljGojscAmF1iqNoeTkEanGpgV Hr/pwa9odOqy+ZIg6JZGD/kRG6SbBAM= X-Google-Smtp-Source: AA+uWPwlndLDiu+3j2sTPM6o54jorPurovsnf2gYz9vV3xqieltzpuv94QdkjkgXYjzoyGCH5upoZA== X-Received: by 2002:a17:902:2e01:: with SMTP id q1-v6mr3966478plb.40.1533860925670; Thu, 09 Aug 2018 17:28:45 -0700 (PDT) Received: from caraitto-dev.sea.corp.google.com ([2620:0:1009:11:84ae:3ba4:ea4b:7d69]) by smtp.gmail.com with ESMTPSA id z8-v6sm9794910pfe.163.2018.08.09.17.28.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 17:28:45 -0700 (PDT) From: Caleb Raitto To: herbert@gondor.apana.org.au, mst@redhat.com, davem@davemloft.net Cc: arei.gonglei@huawei.com, jasowang@redhat.com, netdev@vger.kernel.org, linux-crypto@vger.kernel.org, Caleb Raitto Subject: [PATCH net-next 1/2] virtio_net: Make vp_set_vq_affinity() take a mask. Date: Thu, 9 Aug 2018 17:28:39 -0700 Message-Id: <20180810002840.186635-2-caleb.raitto@gmail.com> X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: <20180810002840.186635-1-caleb.raitto@gmail.com> References: <20180810002840.186635-1-caleb.raitto@gmail.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Caleb Raitto Make vp_set_vq_affinity() take a cpumask instead of taking a single CPU. If there are fewer queues than cores, queue affinity should be able to map to multiple cores. Link: https://patchwork.ozlabs.org/patch/948149/ Suggested-by: Willem de Bruijn Signed-off-by: Caleb Raitto Acked-by: Gongle --- drivers/crypto/virtio/virtio_crypto_core.c | 4 ++-- drivers/net/virtio_net.c | 8 ++++---- drivers/virtio/virtio_pci_common.c | 7 +++---- drivers/virtio/virtio_pci_common.h | 2 +- include/linux/virtio_config.h | 7 ++++--- 5 files changed, 14 insertions(+), 14 deletions(-) diff --git a/drivers/crypto/virtio/virtio_crypto_core.c b/drivers/crypto/virtio/virtio_crypto_core.c index 83326986c113..7c7198553699 100644 --- a/drivers/crypto/virtio/virtio_crypto_core.c +++ b/drivers/crypto/virtio/virtio_crypto_core.c @@ -146,7 +146,7 @@ static void virtcrypto_clean_affinity(struct virtio_crypto *vi, long hcpu) if (vi->affinity_hint_set) { for (i = 0; i < vi->max_data_queues; i++) - virtqueue_set_affinity(vi->data_vq[i].vq, -1); + virtqueue_set_affinity(vi->data_vq[i].vq, NULL); vi->affinity_hint_set = false; } @@ -173,7 +173,7 @@ static void virtcrypto_set_affinity(struct virtio_crypto *vcrypto) * */ for_each_online_cpu(cpu) { - virtqueue_set_affinity(vcrypto->data_vq[i].vq, cpu); + virtqueue_set_affinity(vcrypto->data_vq[i].vq, cpumask_of(cpu)); if (++i >= vcrypto->max_data_queues) break; } diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 39a7f4452587..43fabc0eb4d2 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -1878,8 +1878,8 @@ static void virtnet_clean_affinity(struct virtnet_info *vi, long hcpu) if (vi->affinity_hint_set) { for (i = 0; i < vi->max_queue_pairs; i++) { - virtqueue_set_affinity(vi->rq[i].vq, -1); - virtqueue_set_affinity(vi->sq[i].vq, -1); + virtqueue_set_affinity(vi->rq[i].vq, NULL); + virtqueue_set_affinity(vi->sq[i].vq, NULL); } vi->affinity_hint_set = false; @@ -1905,8 +1905,8 @@ static void virtnet_set_affinity(struct virtnet_info *vi) for_each_online_cpu(cpu) { const unsigned long *mask = cpumask_bits(cpumask_of(cpu)); - virtqueue_set_affinity(vi->rq[i].vq, cpu); - virtqueue_set_affinity(vi->sq[i].vq, cpu); + virtqueue_set_affinity(vi->rq[i].vq, cpumask_of(cpu)); + virtqueue_set_affinity(vi->sq[i].vq, cpumask_of(cpu)); __netif_set_xps_queue(vi->dev, mask, i, false); i++; } diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c index 705aebd74e56..465a6f5142cc 100644 --- a/drivers/virtio/virtio_pci_common.c +++ b/drivers/virtio/virtio_pci_common.c @@ -421,7 +421,7 @@ const char *vp_bus_name(struct virtio_device *vdev) * - OR over all affinities for shared MSI * - ignore the affinity request if we're using INTX */ -int vp_set_vq_affinity(struct virtqueue *vq, int cpu) +int vp_set_vq_affinity(struct virtqueue *vq, const struct cpumask *cpu_mask) { struct virtio_device *vdev = vq->vdev; struct virtio_pci_device *vp_dev = to_vp_device(vdev); @@ -435,11 +435,10 @@ int vp_set_vq_affinity(struct virtqueue *vq, int cpu) if (vp_dev->msix_enabled) { mask = vp_dev->msix_affinity_masks[info->msix_vector]; irq = pci_irq_vector(vp_dev->pci_dev, info->msix_vector); - if (cpu == -1) + if (!cpu_mask) irq_set_affinity_hint(irq, NULL); else { - cpumask_clear(mask); - cpumask_set_cpu(cpu, mask); + cpumask_copy(mask, cpu_mask); irq_set_affinity_hint(irq, mask); } } diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h index 135ee3cf7175..02271002c2f3 100644 --- a/drivers/virtio/virtio_pci_common.h +++ b/drivers/virtio/virtio_pci_common.h @@ -141,7 +141,7 @@ const char *vp_bus_name(struct virtio_device *vdev); * - OR over all affinities for shared MSI * - ignore the affinity request if we're using INTX */ -int vp_set_vq_affinity(struct virtqueue *vq, int cpu); +int vp_set_vq_affinity(struct virtqueue *vq, const struct cpumask *cpu_mask); const struct cpumask *vp_get_vq_affinity(struct virtio_device *vdev, int index); diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h index 5559a2d31c46..32baf8e26735 100644 --- a/include/linux/virtio_config.h +++ b/include/linux/virtio_config.h @@ -79,7 +79,8 @@ struct virtio_config_ops { u64 (*get_features)(struct virtio_device *vdev); int (*finalize_features)(struct virtio_device *vdev); const char *(*bus_name)(struct virtio_device *vdev); - int (*set_vq_affinity)(struct virtqueue *vq, int cpu); + int (*set_vq_affinity)(struct virtqueue *vq, + const struct cpumask *cpu_mask); const struct cpumask *(*get_vq_affinity)(struct virtio_device *vdev, int index); }; @@ -236,11 +237,11 @@ const char *virtio_bus_name(struct virtio_device *vdev) * */ static inline -int virtqueue_set_affinity(struct virtqueue *vq, int cpu) +int virtqueue_set_affinity(struct virtqueue *vq, const struct cpumask *cpu_mask) { struct virtio_device *vdev = vq->vdev; if (vdev->config->set_vq_affinity) - return vdev->config->set_vq_affinity(vq, cpu); + return vdev->config->set_vq_affinity(vq, cpu_mask); return 0; } From patchwork Fri Aug 10 00:28:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Caleb Raitto X-Patchwork-Id: 955900 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="N2T4c5wl"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41mmFd0tYjz9ryt for ; Fri, 10 Aug 2018 10:28:53 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727258AbeHJC4K (ORCPT ); Thu, 9 Aug 2018 22:56:10 -0400 Received: from mail-pl0-f68.google.com ([209.85.160.68]:36538 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726953AbeHJC4H (ORCPT ); Thu, 9 Aug 2018 22:56:07 -0400 Received: by mail-pl0-f68.google.com with SMTP id e11-v6so3255222plb.3; Thu, 09 Aug 2018 17:28:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=R4zvkUtkvgzhoodHheXYne5bbf8kIXUSpP2h/1QzVmw=; b=N2T4c5wluUGFgkAsGxaXdOq+0FbZlfWmnUgFbRpnwyrRIjETk6T2pvhvdIQKif1dvO 0/wINHKEFqJX0Du8fOmc4yUTFZngnbXsQjZwnBOhYQWfiPL2dSgwDrRsDt80UbBi/A8k /KH+URcNiTrbmzrlCL4Tq7TxeC77/8SAwf50/e5NYsQfbcnivNJ9hUT3jRSCd/T7FXjb a/c+KV0jJjrDnEEcEapNTo04DtSdHiMdQXW5/Vw8evXp0SsF6CoRjQiQucoms/fzE7zY zKMWCws+nILTBFZYFbrdoLVOvFjyQoUlrxMdaXcA+u+3m6TTVkA96I15Q4pXcaAw8Wok Gn5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=R4zvkUtkvgzhoodHheXYne5bbf8kIXUSpP2h/1QzVmw=; b=FuAgY3hkjWNaBEh65vu8NVEx4GY74fL5Rto7zRPRCL8cT/cnIS4Zsst3KxKRKr9gLf +sXpxup+cd5rHI8LeXV5HI1V5OVLC5ZTO0RyPQahn5DxaeXejTa+bRGVSiZBVguOpxjQ GPy2LGuU9m91TqkNAfSQK6dP/T72dFLYOuGhre+hcOylIYdYoFHUBJ7FJBsPsbO/aEQl RMMwu6oBSMHpC4B2FIF0QfE0vWORpmkARMKE4n91kY8DC+aONJ1an+A43FxHi+g5Wk9T VzffAvr9SS0QfMgcR/iLrCZIbhEvRoiccWvvxY4GgSqeE2VuLNUbP6sDJmQPEY9/oEoP p+cg== X-Gm-Message-State: AOUpUlFM/bnTPou+YGY2ljKL2mIcwGSANra5CztC6SSPMxjgmEbLzhUT q26ZfaLxnsV56msdWFSXDMXKR+0a7GI= X-Google-Smtp-Source: AA+uWPzmn/e3L78Q8K6jQgrpkpHJc6+M8ID/F0Eh/oHpZu22ggqs5klK2Xuj/R777fvbbdXq3/16bQ== X-Received: by 2002:a17:902:24c7:: with SMTP id l7-v6mr3933923plg.170.1533860926653; Thu, 09 Aug 2018 17:28:46 -0700 (PDT) Received: from caraitto-dev.sea.corp.google.com ([2620:0:1009:11:84ae:3ba4:ea4b:7d69]) by smtp.gmail.com with ESMTPSA id z8-v6sm9794910pfe.163.2018.08.09.17.28.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 17:28:45 -0700 (PDT) From: Caleb Raitto To: herbert@gondor.apana.org.au, mst@redhat.com, davem@davemloft.net Cc: arei.gonglei@huawei.com, jasowang@redhat.com, netdev@vger.kernel.org, linux-crypto@vger.kernel.org, Caleb Raitto , Willem de Bruijn Subject: [PATCH net-next 2/2] virtio_net: Stripe queue affinities across cores. Date: Thu, 9 Aug 2018 17:28:40 -0700 Message-Id: <20180810002840.186635-3-caleb.raitto@gmail.com> X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: <20180810002840.186635-1-caleb.raitto@gmail.com> References: <20180810002840.186635-1-caleb.raitto@gmail.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Caleb Raitto Always set the affinity hint, even if #cpu != #vq. Handle the case where #cpu > #vq (including when #cpu % #vq != 0) and when #vq > #cpu (including when #vq % #cpu != 0). Signed-off-by: Caleb Raitto Signed-off-by: Willem de Bruijn Acked-by: Jon Olson --- drivers/net/virtio_net.c | 42 ++++++++++++++++++++++++++-------------- 1 file changed, 27 insertions(+), 15 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 43fabc0eb4d2..eb00ae6ee475 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -31,6 +31,7 @@ #include #include #include +#include #include #include #include @@ -1888,30 +1889,41 @@ static void virtnet_clean_affinity(struct virtnet_info *vi, long hcpu) static void virtnet_set_affinity(struct virtnet_info *vi) { - int i; - int cpu; + cpumask_var_t mask; + int stragglers; + int group_size; + int i, j, cpu; + int num_cpu; + int stride; - /* In multiqueue mode, when the number of cpu is equal to the number of - * queue pairs, we let the queue pairs to be private to one cpu by - * setting the affinity hint to eliminate the contention. - */ - if (vi->curr_queue_pairs == 1 || - vi->max_queue_pairs != num_online_cpus()) { + if (!zalloc_cpumask_var(&mask, GFP_KERNEL)) { virtnet_clean_affinity(vi, -1); return; } - i = 0; - for_each_online_cpu(cpu) { - const unsigned long *mask = cpumask_bits(cpumask_of(cpu)); + num_cpu = num_online_cpus(); + stride = max_t(int, num_cpu / vi->curr_queue_pairs, 1); + stragglers = num_cpu >= vi->curr_queue_pairs ? + num_cpu % vi->curr_queue_pairs : + 0; + cpu = cpumask_next(-1, cpu_online_mask); - virtqueue_set_affinity(vi->rq[i].vq, cpumask_of(cpu)); - virtqueue_set_affinity(vi->sq[i].vq, cpumask_of(cpu)); - __netif_set_xps_queue(vi->dev, mask, i, false); - i++; + for (i = 0; i < vi->curr_queue_pairs; i++) { + group_size = stride + (i < stragglers ? 1 : 0); + + for (j = 0; j < group_size; j++) { + cpumask_set_cpu(cpu, mask); + cpu = cpumask_next_wrap(cpu, cpu_online_mask, + nr_cpu_ids, false); + } + virtqueue_set_affinity(vi->rq[i].vq, mask); + virtqueue_set_affinity(vi->sq[i].vq, mask); + __netif_set_xps_queue(vi->dev, cpumask_bits(mask), i, false); + cpumask_clear(mask); } vi->affinity_hint_set = true; + free_cpumask_var(mask); } static int virtnet_cpu_online(unsigned int cpu, struct hlist_node *node)