From patchwork Mon Nov 2 12:48:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: William Breathitt Gray X-Patchwork-Id: 1392220 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CPt5p5V4zz9sWS; Mon, 2 Nov 2020 23:49:18 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1kZZGq-00061r-RX; Mon, 02 Nov 2020 12:49:12 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1kZZGn-0005yV-BI for kernel-team@lists.ubuntu.com; Mon, 02 Nov 2020 12:49:09 +0000 Received: from mail-qk1-f199.google.com ([209.85.222.199]) by youngberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1kZZGm-00043n-R4 for kernel-team@lists.ubuntu.com; Mon, 02 Nov 2020 12:49:09 +0000 Received: by mail-qk1-f199.google.com with SMTP id r129so7339414qkd.5 for ; Mon, 02 Nov 2020 04:49:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JA0ndzQkDG8d46e9gh6YIHrmIrd12YKPRXVShYNh/Ss=; b=ZZNgnu9F704jopVmxk9FZjSxj1TGSXUDlTF7GxeDsdIUhb3FvDt/DwN7bb6OOHFGCT zqyvgZxJNmvTNlL42rNTyeY1dYi+jFFwVudSUK8Io7hOccRsHD7udPQxDFCXje8WAJL7 654Qz4f9LKj0k5/hvhTglqwhbIhlk92jZCSTq+2grWb3juIEPHhgLMII8hbrQtuNDUr/ 33PWmZOFAGutgOT/yeGRFAxAS236Ndo2Inpt8jP7Y0qHogxLtx+84DasXWvZ2VO5Got/ b+ijfiHiYVymimJyupevMbdfkdeKA6KuPZjgncRJ4yJP4+yZGF6uLShM2z9FTSTj+UmS Ueaw== X-Gm-Message-State: AOAM5311CIuLBanvJLYN78StLT419Cfk5WV+F7dLZOx7VP5t4UmRgbDn tAxL4tl0Vm6XfsjJM9eZpJq23XzVjAOp6hNy178rbCJwLoCIus6BqfPfweSZwTjuYtWvLvOo3nL xmftMqQ6nlWafCN6XfDUjz8JbJzHO20P67qoklaIrVQ== X-Received: by 2002:ae9:ebca:: with SMTP id b193mr2312457qkg.235.1604321347506; Mon, 02 Nov 2020 04:49:07 -0800 (PST) X-Google-Smtp-Source: ABdhPJzPEIs6SvF9rZIoJq0MxnOrsHXfwPnmTbt8oK5yqTEM1GMjzXuuV7ozIA8eLojSzOfTztinmA== X-Received: by 2002:ae9:ebca:: with SMTP id b193mr2312439qkg.235.1604321347224; Mon, 02 Nov 2020 04:49:07 -0800 (PST) Received: from localhost.localdomain (072-189-064-225.res.spectrum.com. [72.189.64.225]) by smtp.gmail.com with ESMTPSA id q7sm7666201qtd.49.2020.11.02.04.49.06 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Nov 2020 04:49:06 -0800 (PST) From: William Breathitt Gray To: kernel-team@lists.ubuntu.com Subject: [SRU][B:linux-azure-4.15][PATCH 08/40] tun/tap: use ptr_ring instead of skb_array Date: Mon, 2 Nov 2020 07:48:24 -0500 Message-Id: <20201102124856.4659-9-william.gray@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201102124856.4659-1-william.gray@canonical.com> References: <20201102124856.4659-1-william.gray@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Jason Wang BugLink: https://bugs.launchpad.net/bugs/1877654 This patch switches to use ptr_ring instead of skb_array. This will be used to enqueue different types of pointers by encoding type into lower bits. Signed-off-by: Jason Wang Signed-off-by: David S. Miller (backported from commit 5990a30510ed1c37a769d3a035ad2d030b843528) [ vilhelmgray: context adjustments ] [ vilhelmgray: use ptr_ring_resize() in tun_attach() ] [ vilhelmgray: use ptr_ring_init() in tun_chr_open() ] Signed-off-by: William Breathitt Gray --- debian.azure-4.15/config/config.common.ubuntu | 1 + drivers/net/tap.c | 42 ++++++++++-------- drivers/net/tun.c | 44 ++++++++++--------- drivers/vhost/net.c | 39 ++++++++-------- include/linux/if_tap.h | 6 +-- include/linux/if_tun.h | 4 +- 6 files changed, 72 insertions(+), 64 deletions(-) diff --git a/debian.azure-4.15/config/config.common.ubuntu b/debian.azure-4.15/config/config.common.ubuntu index fbb0a87534ec..5d866c752057 100644 --- a/debian.azure-4.15/config/config.common.ubuntu +++ b/debian.azure-4.15/config/config.common.ubuntu @@ -3402,6 +3402,7 @@ CONFIG_PAGE_COUNTER=y # CONFIG_PAGE_EXTENSION is not set # CONFIG_PAGE_OWNER is not set # CONFIG_PAGE_POISONING is not set +CONFIG_PAGE_POOL=y CONFIG_PAGE_TABLE_ISOLATION=y # CONFIG_PANASONIC_LAPTOP is not set CONFIG_PANEL=m diff --git a/drivers/net/tap.c b/drivers/net/tap.c index f8b44d395c2f..658235e0204f 100644 --- a/drivers/net/tap.c +++ b/drivers/net/tap.c @@ -330,6 +330,9 @@ rx_handler_result_t tap_handle_frame(struct sk_buff **pskb) if (!q) return RX_HANDLER_PASS; + if (__ptr_ring_full(&q->ring)) + goto drop; + skb_push(skb, ETH_HLEN); /* Apply the forward feature mask so that we perform segmentation @@ -345,7 +348,7 @@ rx_handler_result_t tap_handle_frame(struct sk_buff **pskb) goto drop; if (!segs) { - if (skb_array_produce(&q->skb_array, skb)) + if (ptr_ring_produce(&q->ring, skb)) goto drop; goto wake_up; } @@ -355,7 +358,7 @@ rx_handler_result_t tap_handle_frame(struct sk_buff **pskb) struct sk_buff *nskb = segs->next; segs->next = NULL; - if (skb_array_produce(&q->skb_array, segs)) { + if (ptr_ring_produce(&q->ring, segs)) { kfree_skb(segs); kfree_skb_list(nskb); break; @@ -372,7 +375,7 @@ rx_handler_result_t tap_handle_frame(struct sk_buff **pskb) !(features & NETIF_F_CSUM_MASK) && skb_checksum_help(skb)) goto drop; - if (skb_array_produce(&q->skb_array, skb)) + if (ptr_ring_produce(&q->ring, skb)) goto drop; } @@ -494,7 +497,7 @@ static void tap_sock_destruct(struct sock *sk) { struct tap_queue *q = container_of(sk, struct tap_queue, sk); - skb_array_cleanup(&q->skb_array); + ptr_ring_cleanup(&q->ring, __skb_array_destroy_skb); } static int tap_open(struct inode *inode, struct file *file) @@ -514,7 +517,7 @@ static int tap_open(struct inode *inode, struct file *file) &tap_proto, 0); if (!q) goto err; - if (skb_array_init(&q->skb_array, tap->dev->tx_queue_len, GFP_KERNEL)) { + if (ptr_ring_init(&q->ring, tap->dev->tx_queue_len, GFP_KERNEL)) { sk_free(&q->sk); goto err; } @@ -543,7 +546,7 @@ static int tap_open(struct inode *inode, struct file *file) err = tap_set_queue(tap, file, q); if (err) { - /* tap_sock_destruct() will take care of freeing skb_array */ + /* tap_sock_destruct() will take care of freeing ptr_ring */ goto err_put; } @@ -580,7 +583,7 @@ static unsigned int tap_poll(struct file *file, poll_table *wait) mask = 0; poll_wait(file, &q->wq.wait, wait); - if (!skb_array_empty(&q->skb_array)) + if (!ptr_ring_empty(&q->ring)) mask |= POLLIN | POLLRDNORM; if (sock_writeable(&q->sk) || @@ -844,7 +847,7 @@ static ssize_t tap_do_read(struct tap_queue *q, TASK_INTERRUPTIBLE); /* Read frames from the queue */ - skb = skb_array_consume(&q->skb_array); + skb = ptr_ring_consume(&q->ring); if (skb) break; if (noblock) { @@ -1176,7 +1179,7 @@ static int tap_peek_len(struct socket *sock) { struct tap_queue *q = container_of(sock, struct tap_queue, sock); - return skb_array_peek_len(&q->skb_array); + return PTR_RING_PEEK_CALL(&q->ring, __skb_array_len_with_tag); } /* Ops structure to mimic raw sockets with tun */ @@ -1202,7 +1205,7 @@ struct socket *tap_get_socket(struct file *file) } EXPORT_SYMBOL_GPL(tap_get_socket); -struct skb_array *tap_get_skb_array(struct file *file) +struct ptr_ring *tap_get_ptr_ring(struct file *file) { struct tap_queue *q; @@ -1211,29 +1214,30 @@ struct skb_array *tap_get_skb_array(struct file *file) q = file->private_data; if (!q) return ERR_PTR(-EBADFD); - return &q->skb_array; + return &q->ring; } -EXPORT_SYMBOL_GPL(tap_get_skb_array); +EXPORT_SYMBOL_GPL(tap_get_ptr_ring); int tap_queue_resize(struct tap_dev *tap) { struct net_device *dev = tap->dev; struct tap_queue *q; - struct skb_array **arrays; + struct ptr_ring **rings; int n = tap->numqueues; int ret, i = 0; - arrays = kmalloc_array(n, sizeof(*arrays), GFP_KERNEL); - if (!arrays) + rings = kmalloc_array(n, sizeof(*rings), GFP_KERNEL); + if (!rings) return -ENOMEM; list_for_each_entry(q, &tap->queue_list, next) - arrays[i++] = &q->skb_array; + rings[i++] = &q->ring; - ret = skb_array_resize_multiple(arrays, n, - dev->tx_queue_len, GFP_KERNEL); + ret = ptr_ring_resize_multiple(rings, n, + dev->tx_queue_len, GFP_KERNEL, + __skb_array_destroy_skb); - kfree(arrays); + kfree(rings); return ret; } EXPORT_SYMBOL_GPL(tap_queue_resize); diff --git a/drivers/net/tun.c b/drivers/net/tun.c index 2434b2bd902f..d1f2f25a748f 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -180,7 +180,7 @@ struct tun_file { struct mutex napi_mutex; /* Protects access to the above napi */ struct list_head next; struct tun_struct *detached; - struct skb_array tx_array; + struct ptr_ring tx_ring; struct xdp_rxq_info xdp_rxq; }; @@ -606,7 +606,7 @@ static void tun_queue_purge(struct tun_file *tfile) { struct sk_buff *skb; - while ((skb = skb_array_consume(&tfile->tx_array)) != NULL) + while ((skb = ptr_ring_consume(&tfile->tx_ring)) != NULL) kfree_skb(skb); skb_queue_purge(&tfile->sk.sk_write_queue); @@ -660,7 +660,8 @@ static void __tun_detach(struct tun_file *tfile, bool clean) unregister_netdevice(tun->dev); } if (tun) { - skb_array_cleanup(&tfile->tx_array); + ptr_ring_cleanup(&tfile->tx_ring, + __skb_array_destroy_skb); xdp_rxq_info_unreg(&tfile->xdp_rxq); } sock_put(&tfile->sk); @@ -758,7 +759,7 @@ static int tun_attach(struct tun_struct *tun, struct file *file, } if (!tfile->detached && - skb_array_resize(&tfile->tx_array, dev->tx_queue_len, GFP_KERNEL)) { + ptr_ring_resize(&tfile->tx_ring, dev->tx_queue_len, GFP_KERNEL, __skb_array_destroy_skb)) { err = -ENOMEM; goto out; } @@ -1012,7 +1013,7 @@ static netdev_tx_t tun_net_xmit(struct sk_buff *skb, struct net_device *dev) nf_reset(skb); - if (skb_array_produce(&tfile->tx_array, skb)) + if (ptr_ring_produce(&tfile->tx_ring, skb)) goto drop; /* Notify and wake up reader process */ @@ -1302,7 +1303,7 @@ static unsigned int tun_chr_poll(struct file *file, poll_table *wait) poll_wait(file, sk_sleep(sk), wait); - if (!skb_array_empty(&tfile->tx_array)) + if (!ptr_ring_empty(&tfile->tx_ring)) mask |= POLLIN | POLLRDNORM; /* Make sure SOCKWQ_ASYNC_NOSPACE is set if not writable to @@ -1981,7 +1982,7 @@ static struct sk_buff *tun_ring_recv(struct tun_file *tfile, int noblock, struct sk_buff *skb = NULL; int error = 0; - skb = skb_array_consume(&tfile->tx_array); + skb = ptr_ring_consume(&tfile->tx_ring); if (skb) goto out; if (noblock) { @@ -1993,7 +1994,7 @@ static struct sk_buff *tun_ring_recv(struct tun_file *tfile, int noblock, while (1) { set_current_state(TASK_INTERRUPTIBLE); - skb = skb_array_consume(&tfile->tx_array); + skb = ptr_ring_consume(&tfile->tx_ring); if (skb) break; if (signal_pending(current)) { @@ -2191,7 +2192,7 @@ static int tun_peek_len(struct socket *sock) if (!tun) return 0; - ret = skb_array_peek_len(&tfile->tx_array); + ret = PTR_RING_PEEK_CALL(&tfile->tx_ring, __skb_array_len_with_tag); tun_put(tun); return ret; @@ -2921,7 +2922,7 @@ static int tun_chr_open(struct inode *inode, struct file * file) &tun_proto, 0); if (!tfile) return -ENOMEM; - if (skb_array_init(&tfile->tx_array, 0, GFP_KERNEL)) { + if (ptr_ring_init(&tfile->tx_ring, 0, GFP_KERNEL)) { sk_free(&tfile->sk); return -ENOMEM; } @@ -3094,25 +3095,26 @@ static int tun_queue_resize(struct tun_struct *tun) { struct net_device *dev = tun->dev; struct tun_file *tfile; - struct skb_array **arrays; + struct ptr_ring **rings; int n = tun->numqueues + tun->numdisabled; int ret, i; - arrays = kmalloc_array(n, sizeof(*arrays), GFP_KERNEL); - if (!arrays) + rings = kmalloc_array(n, sizeof(*rings), GFP_KERNEL); + if (!rings) return -ENOMEM; for (i = 0; i < tun->numqueues; i++) { tfile = rtnl_dereference(tun->tfiles[i]); - arrays[i] = &tfile->tx_array; + rings[i] = &tfile->tx_ring; } list_for_each_entry(tfile, &tun->disabled, next) - arrays[i++] = &tfile->tx_array; + rings[i++] = &tfile->tx_ring; - ret = skb_array_resize_multiple(arrays, n, - dev->tx_queue_len, GFP_KERNEL); + ret = ptr_ring_resize_multiple(rings, n, + dev->tx_queue_len, GFP_KERNEL, + __skb_array_destroy_skb); - kfree(arrays); + kfree(rings); return ret; } @@ -3207,7 +3209,7 @@ struct socket *tun_get_socket(struct file *file) } EXPORT_SYMBOL_GPL(tun_get_socket); -struct skb_array *tun_get_skb_array(struct file *file) +struct ptr_ring *tun_get_tx_ring(struct file *file) { struct tun_file *tfile; @@ -3216,9 +3218,9 @@ struct skb_array *tun_get_skb_array(struct file *file) tfile = file->private_data; if (!tfile) return ERR_PTR(-EBADFD); - return &tfile->tx_array; + return &tfile->tx_ring; } -EXPORT_SYMBOL_GPL(tun_get_skb_array); +EXPORT_SYMBOL_GPL(tun_get_tx_ring); module_init(tun_init); module_exit(tun_cleanup); diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index 17606d9688c7..0f1e31ee29b3 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -95,7 +95,7 @@ struct vhost_net_ubuf_ref { #define VHOST_RX_BATCH 64 struct vhost_net_buf { - struct sk_buff **queue; + void **queue; int tail; int head; }; @@ -114,7 +114,7 @@ struct vhost_net_virtqueue { /* Reference counting for outstanding ubufs. * Protected by vq mutex. Writers must also take device mutex. */ struct vhost_net_ubuf_ref *ubufs; - struct skb_array *rx_array; + struct ptr_ring *rx_ring; struct vhost_net_buf rxq; }; @@ -164,7 +164,7 @@ static int vhost_net_buf_produce(struct vhost_net_virtqueue *nvq) struct vhost_net_buf *rxq = &nvq->rxq; rxq->head = 0; - rxq->tail = skb_array_consume_batched(nvq->rx_array, rxq->queue, + rxq->tail = ptr_ring_consume_batched(nvq->rx_ring, rxq->queue, VHOST_RX_BATCH); return rxq->tail; } @@ -173,9 +173,10 @@ static void vhost_net_buf_unproduce(struct vhost_net_virtqueue *nvq) { struct vhost_net_buf *rxq = &nvq->rxq; - if (nvq->rx_array && !vhost_net_buf_is_empty(rxq)) { - skb_array_unconsume(nvq->rx_array, rxq->queue + rxq->head, - vhost_net_buf_get_size(rxq)); + if (nvq->rx_ring && !vhost_net_buf_is_empty(rxq)) { + ptr_ring_unconsume(nvq->rx_ring, rxq->queue + rxq->head, + vhost_net_buf_get_size(rxq), + __skb_array_destroy_skb); rxq->head = rxq->tail = 0; } } @@ -593,7 +594,7 @@ static int peek_head_len(struct vhost_net_virtqueue *rvq, struct sock *sk) int len = 0; unsigned long flags; - if (rvq->rx_array) + if (rvq->rx_ring) return vhost_net_buf_peek(rvq); spin_lock_irqsave(&sk->sk_receive_queue.lock, flags); @@ -806,7 +807,7 @@ static void handle_rx(struct vhost_net *net) * they refilled. */ goto out; } - if (nvq->rx_array) + if (nvq->rx_ring) msg.msg_control = vhost_net_buf_consume(&nvq->rxq); /* On overrun, truncate and discard */ if (unlikely(headcount > UIO_MAXIOV)) { @@ -910,7 +911,7 @@ static int vhost_net_open(struct inode *inode, struct file *f) struct vhost_net *n; struct vhost_dev *dev; struct vhost_virtqueue **vqs; - struct sk_buff **queue; + void **queue; int i; n = kvmalloc(sizeof *n, GFP_KERNEL | __GFP_RETRY_MAYFAIL); @@ -922,7 +923,7 @@ static int vhost_net_open(struct inode *inode, struct file *f) return -ENOMEM; } - queue = kmalloc_array(VHOST_RX_BATCH, sizeof(struct sk_buff *), + queue = kmalloc_array(VHOST_RX_BATCH, sizeof(void *), GFP_KERNEL); if (!queue) { kfree(vqs); @@ -1052,23 +1053,23 @@ static struct socket *get_raw_socket(int fd) return ERR_PTR(r); } -static struct skb_array *get_tap_skb_array(int fd) +static struct ptr_ring *get_tap_ptr_ring(int fd) { - struct skb_array *array; + struct ptr_ring *ring; struct file *file = fget(fd); if (!file) return NULL; - array = tun_get_skb_array(file); - if (!IS_ERR(array)) + ring = tun_get_tx_ring(file); + if (!IS_ERR(ring)) goto out; - array = tap_get_skb_array(file); - if (!IS_ERR(array)) + ring = tap_get_ptr_ring(file); + if (!IS_ERR(ring)) goto out; - array = NULL; + ring = NULL; out: fput(file); - return array; + return ring; } static struct socket *get_tap_socket(int fd) @@ -1149,7 +1150,7 @@ static long vhost_net_set_backend(struct vhost_net *n, unsigned index, int fd) vq->private_data = sock; vhost_net_buf_unproduce(nvq); if (index == VHOST_NET_VQ_RX) - nvq->rx_array = get_tap_skb_array(fd); + nvq->rx_ring = get_tap_ptr_ring(fd); r = vhost_vq_init_access(vq); if (r) goto err_used; diff --git a/include/linux/if_tap.h b/include/linux/if_tap.h index 3ecef57c31e3..8e66866c11be 100644 --- a/include/linux/if_tap.h +++ b/include/linux/if_tap.h @@ -4,7 +4,7 @@ #if IS_ENABLED(CONFIG_TAP) struct socket *tap_get_socket(struct file *); -struct skb_array *tap_get_skb_array(struct file *file); +struct ptr_ring *tap_get_ptr_ring(struct file *file); #else #include #include @@ -14,7 +14,7 @@ static inline struct socket *tap_get_socket(struct file *f) { return ERR_PTR(-EINVAL); } -static inline struct skb_array *tap_get_skb_array(struct file *f) +static inline struct ptr_ring *tap_get_ptr_ring(struct file *f) { return ERR_PTR(-EINVAL); } @@ -70,7 +70,7 @@ struct tap_queue { u16 queue_index; bool enabled; struct list_head next; - struct skb_array skb_array; + struct ptr_ring ring; }; rx_handler_result_t tap_handle_frame(struct sk_buff **pskb); diff --git a/include/linux/if_tun.h b/include/linux/if_tun.h index bf9bdf42d577..bdee9b83baf6 100644 --- a/include/linux/if_tun.h +++ b/include/linux/if_tun.h @@ -19,7 +19,7 @@ #if defined(CONFIG_TUN) || defined(CONFIG_TUN_MODULE) struct socket *tun_get_socket(struct file *); -struct skb_array *tun_get_skb_array(struct file *file); +struct ptr_ring *tun_get_tx_ring(struct file *file); #else #include #include @@ -29,7 +29,7 @@ static inline struct socket *tun_get_socket(struct file *f) { return ERR_PTR(-EINVAL); } -static inline struct skb_array *tun_get_skb_array(struct file *f) +static inline struct ptr_ring *tun_get_tx_ring(struct file *f) { return ERR_PTR(-EINVAL); }