{"id":814648,"url":"http://patchwork.ozlabs.org/api/patches/814648/?format=json","web_url":"http://patchwork.ozlabs.org/project/netdev/patch/1505651798-106642-3-git-send-email-robert.hu@linux.intel.com/","project":{"id":7,"url":"http://patchwork.ozlabs.org/api/projects/7/?format=json","name":"Linux network development","link_name":"netdev","list_id":"netdev.vger.kernel.org","list_email":"netdev@vger.kernel.org","web_url":null,"scm_url":null,"webscm_url":null,"list_archive_url":"","list_archive_url_format":"","commit_url_format":""},"msgid":"<1505651798-106642-3-git-send-email-robert.hu@linux.intel.com>","list_archive_url":null,"date":"2017-09-17T12:36:38","name":"[pktgen,script,v2,2/2] Add pktgen script: pktgen_sample06_numa_awared_queue_irq_affinity.sh","commit_ref":null,"pull_url":null,"state":"not-applicable","archived":true,"hash":"06dec015130bd03bf5a1d592314a4c86b38e495a","submitter":{"id":72226,"url":"http://patchwork.ozlabs.org/api/people/72226/?format=json","name":"Robert Hoo","email":"robert.hu@linux.intel.com"},"delegate":{"id":34,"url":"http://patchwork.ozlabs.org/api/users/34/?format=json","username":"davem","first_name":"David","last_name":"Miller","email":"davem@davemloft.net"},"mbox":"http://patchwork.ozlabs.org/project/netdev/patch/1505651798-106642-3-git-send-email-robert.hu@linux.intel.com/mbox/","series":[{"id":3512,"url":"http://patchwork.ozlabs.org/api/series/3512/?format=json","web_url":"http://patchwork.ozlabs.org/project/netdev/list/?series=3512","date":"2017-09-17T12:36:36","name":"Add a pktgen sample script of NUMA awareness","version":2,"mbox":"http://patchwork.ozlabs.org/series/3512/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/814648/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/814648/checks/","tags":{},"related":[],"headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xw7td0NVQz9s81\n\tfor ;\n\tSun, 17 Sep 2017 22:36:57 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751445AbdIQMgu (ORCPT );\n\tSun, 17 Sep 2017 08:36:50 -0400","from mga04.intel.com ([192.55.52.120]:56558 \"EHLO mga04.intel.com\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1751378AbdIQMgt (ORCPT );\n\tSun, 17 Sep 2017 08:36:49 -0400","from fmsmga004.fm.intel.com ([10.253.24.48])\n\tby fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t17 Sep 2017 05:36:49 -0700","from sqa-gate.sh.intel.com (HELO kvm-host.tsp.org)\n\t([10.239.48.212])\n\tby fmsmga004.fm.intel.com with ESMTP; 17 Sep 2017 05:36:47 -0700"],"X-ExtLoop1":"1","X-IronPort-AV":"E=Sophos;i=\"5.42,407,1500966000\"; d=\"scan'208\";a=\"312616200\"","From":"Robert Hoo ","To":"brouer@redhat.com, davem@davemloft.net, tariqt@mellanox.com,\n\tkyle.leet@gmail.com","Cc":"netdev@vger.kernel.org, robert.hu@intel.com","Subject":"[pktgen script v2 2/2] Add pktgen script:\n\tpktgen_sample06_numa_awared_queue_irq_affinity.sh","Date":"Sun, 17 Sep 2017 20:36:38 +0800","Message-Id":"<1505651798-106642-3-git-send-email-robert.hu@linux.intel.com>","X-Mailer":"git-send-email 1.8.3.1","In-Reply-To":"<1505651798-106642-1-git-send-email-robert.hu@linux.intel.com>","References":"<1505651798-106642-1-git-send-email-robert.hu@linux.intel.com>","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"},"content":"From: Robert Hoo \n\nThis script simply does:\nDetect $DEV's NUMA node belonging.\nBind each thread (processor of NUMA locality) with each $DEV queue's\nirq affinity, 1:1 mapping.\nHow many '-t' threads input determines how many queues will be\nutilized.\nIf '-f' designates first cpu id, then offset in the NUMA node's cpu\nlist.\n\nSigned-off-by: Robert Hoo \n---\n ...tgen_sample06_numa_awared_queue_irq_affinity.sh | 97 ++++++++++++++++++++++\n 1 file changed, 97 insertions(+)\n create mode 100755 pktgen/pktgen_sample06_numa_awared_queue_irq_affinity.sh","diff":"diff --git a/pktgen/pktgen_sample06_numa_awared_queue_irq_affinity.sh b/pktgen/pktgen_sample06_numa_awared_queue_irq_affinity.sh\nnew file mode 100755\nindex 0000000..52da0f4\n--- /dev/null\n+++ b/pktgen/pktgen_sample06_numa_awared_queue_irq_affinity.sh\n@@ -0,0 +1,97 @@\n+#!/bin/bash\n+#\n+# Multiqueue: Using pktgen threads for sending on multiple CPUs\n+# * adding devices to kernel threads which are in the same NUMA node\n+# * bound devices queue's irq affinity to the threads, 1:1 mapping\n+# * notice the naming scheme for keeping device names unique\n+# * nameing scheme: dev@thread_number\n+# * flow variation via random UDP source port\n+#\n+basedir=`dirname $0`\n+source ${basedir}/functions.sh\n+root_check_run_with_sudo \"$@\"\n+#\n+# Required param: -i dev in $DEV\n+source ${basedir}/parameters.sh\n+\n+# Base Config\n+DELAY=\"0\" # Zero means max speed\n+COUNT=\"20000000\" # Zero means indefinitely\n+[ -z \"$CLONE_SKB\" ] && CLONE_SKB=\"0\"\n+\n+# Flow variation random source port between min and max\n+UDP_MIN=9\n+UDP_MAX=109\n+\n+node=`get_iface_node $DEV`\n+irq_array=(`get_iface_irqs $DEV`)\n+cpu_array=(`get_node_cpus $node`)\n+\n+[ $THREADS -gt ${#irq_array[*]} -o $THREADS -gt ${#cpu_array[*]} ] && \\\n+\terr 1 \"Thread number $THREADS exceeds: min (${#irq_array[*]},${#cpu_array[*]})\"\n+\n+# (example of setting default params in your script)\n+if [ -z \"$DEST_IP\" ]; then\n+ [ -z \"$IP6\" ] && DEST_IP=\"198.18.0.42\" || DEST_IP=\"FD00::1\"\n+fi\n+[ -z \"$DST_MAC\" ] && DST_MAC=\"90:e2:ba:ff:ff:ff\"\n+\n+# General cleanup everything since last run\n+pg_ctrl \"reset\"\n+\n+# Threads are specified with parameter -t value in $THREADS\n+for ((i = 0; i < $THREADS; i++)); do\n+ # The device name is extended with @name, using thread number to\n+ # make then unique, but any name will do.\n+ # Set the queue's irq affinity to this $thread (processor)\n+ # if '-f' is designated, offset cpu id\n+ thread=${cpu_array[$((i+F_THREAD))]}\n+ dev=${DEV}@${thread}\n+ echo $thread > /proc/irq/${irq_array[$i]}/smp_affinity_list\n+ info \"irq ${irq_array[$i]} is set affinity to `cat /proc/irq/${irq_array[$i]}/smp_affinity_list`\"\n+\n+ # Add remove all other devices and add_device $dev to thread\n+ pg_thread $thread \"rem_device_all\"\n+ pg_thread $thread \"add_device\" $dev\n+\n+ # select queue and bind the queue and $dev in 1:1 relationship\n+ queue_num=$i\n+ info \"queue number is $queue_num\"\n+ pg_set $dev \"queue_map_min $queue_num\"\n+ pg_set $dev \"queue_map_max $queue_num\"\n+\n+ # Notice config queue to map to cpu (mirrors smp_processor_id())\n+ # It is beneficial to map IRQ /proc/irq/*/smp_affinity 1:1 to CPU number\n+ pg_set $dev \"flag QUEUE_MAP_CPU\"\n+\n+ # Base config of dev\n+ pg_set $dev \"count $COUNT\"\n+ pg_set $dev \"clone_skb $CLONE_SKB\"\n+ pg_set $dev \"pkt_size $PKT_SIZE\"\n+ pg_set $dev \"delay $DELAY\"\n+\n+ # Flag example disabling timestamping\n+ pg_set $dev \"flag NO_TIMESTAMP\"\n+\n+ # Destination\n+ pg_set $dev \"dst_mac $DST_MAC\"\n+ pg_set $dev \"dst$IP6 $DEST_IP\"\n+\n+ # Setup random UDP port src range\n+ pg_set $dev \"flag UDPSRC_RND\"\n+ pg_set $dev \"udp_src_min $UDP_MIN\"\n+ pg_set $dev \"udp_src_max $UDP_MAX\"\n+done\n+\n+# start_run\n+echo \"Running... ctrl^C to stop\" >&2\n+pg_ctrl \"start\"\n+echo \"Done\" >&2\n+\n+# Print results\n+for ((i = 0; i < $THREADS; i++)); do\n+ thread=${cpu_array[$((i+F_THREAD))]}\n+ dev=${DEV}@${thread}\n+ echo \"Device: $dev\"\n+ cat /proc/net/pktgen/$dev | grep -A2 \"Result:\"\n+done\n","prefixes":["pktgen","script","v2","2/2"]}