[{"id":1770302,"web_url":"http://patchwork.ozlabs.org/comment/1770302/","msgid":"<20170918.093436.1069699729581966538.davem@davemloft.net>","list_archive_url":null,"date":"2017-09-18T16:34:36","subject":"Re: [PATCH RFC V1 net-next 0/6] Time based packet transmission","submitter":{"id":15,"url":"http://patchwork.ozlabs.org/api/people/15/","name":"David Miller","email":"davem@davemloft.net"},"content":"From: Richard Cochran \nDate: Mon, 18 Sep 2017 09:41:15 +0200\n\n> - The driver does not handle out of order packets. If user space\n> sends a packet with an earlier Tx time, then the code should stop\n> the queue, reshuffle the descriptors accordingly, and then\n> restart the queue.\n\nThe user should simply be not allowed to do this.\n\nOnce the packet is in the device queue, that's it. You cannot insert\na new packet to be transmitted before an already hw queued packet,\nperiod.\n\nAny out of order request should be rejected with an error.\n\nI'd say the same is true for requests to send packets timed\nin the past.","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xws7H3tSGz9s72\n\tfor ;\n\tTue, 19 Sep 2017 02:35:23 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S932586AbdIRQen (ORCPT );\n\tMon, 18 Sep 2017 12:34:43 -0400","from shards.monkeyblade.net ([184.105.139.130]:43842 \"EHLO\n\tshards.monkeyblade.net\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1755934AbdIRQem (ORCPT\n\t); Mon, 18 Sep 2017 12:34:42 -0400","from localhost (74-93-104-98-Washington.hfc.comcastbusiness.net\n\t[74.93.104.98]) (using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(Client did not present a certificate)\n\t(Authenticated sender: davem-davemloft)\n\tby shards.monkeyblade.net (Postfix) with ESMTPSA id C6A4B101FF141;\n\tMon, 18 Sep 2017 09:34:38 -0700 (PDT)"],"Date":"Mon, 18 Sep 2017 09:34:36 -0700 (PDT)","Message-Id":"<20170918.093436.1069699729581966538.davem@davemloft.net>","To":"rcochran@linutronix.de","Cc":"netdev@vger.kernel.org, linux-kernel@vger.kernel.org,\n\tintel-wired-lan@lists.osuosl.org, andre.guedes@intel.com,\n\tanna-maria@linutronix.de, henrik@austad.us,\n\tjesus.sanchez-palencia@intel.com, john.stultz@linaro.org,\n\ttglx@linutronix.de, vinicius.gomes@intel.com","Subject":"Re: [PATCH RFC V1 net-next 0/6] Time based packet transmission","From":"David Miller ","In-Reply-To":"","References":"","X-Mailer":"Mew version 6.7 on Emacs 25.2 / Mule 6.0 (HANACHIRUSATO)","Mime-Version":"1.0","Content-Type":"Text/Plain; charset=us-ascii","Content-Transfer-Encoding":"7bit","X-Greylist":"Sender succeeded SMTP AUTH, not delayed by\n\tmilter-greylist-4.5.12 (shards.monkeyblade.net\n\t[149.20.54.216]); Mon, 18 Sep 2017 09:34:39 -0700 (PDT)","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1771096,"web_url":"http://patchwork.ozlabs.org/comment/1771096/","msgid":"<20170919144302.GB4347@localhost>","list_archive_url":null,"date":"2017-09-19T14:43:02","subject":"Re: [PATCH RFC V1 net-next 0/6] Time based packet transmission","submitter":{"id":17040,"url":"http://patchwork.ozlabs.org/api/people/17040/","name":"Miroslav Lichvar","email":"mlichvar@redhat.com"},"content":"On Mon, Sep 18, 2017 at 09:41:15AM +0200, Richard Cochran wrote:\n> This series is an early RFC that introduces a new socket option\n> allowing time based transmission of packets. This option will be\n> useful in implementing various real time protocols over Ethernet,\n> including but not limited to P802.1Qbv, which is currently finding\n> its way into 802.1Q.\n\nIf I understand it correctly, this also allows us to make a PTP/NTP\n\"one-step\" clock with HW that doesn't support it directly.\n\n> * Open questions about SO_TXTIME semantics\n> \n> - What should the kernel do if the dialed Tx time is in the past?\n> Should the packet be sent ASAP, or should we throw an error?\n\nDropping the packet with an error would make more sense to me.\n\n> - What should the timescale be for the dialed Tx time? Should the\n> kernel select UTC when using the SW Qdisc and the HW time\n> otherwise? Or should the socket option include a clockid_t?\n\nI think for applications that don't (want to) bind their socket to a\nspecific interface it would be useful if the cmsg specified clockid_t\nor maybe if_index. If the packet would be sent using a different\nPHC/interface, it should be dropped.\n\n> | | plain preempt_rt | so_txtime | txtime @ 250 us |\n> |---------+------------------+---------------+-----------------|\n> | min: | +1.940800e+04 | +4.720000e+02 | +4.720000e+02 |\n> | max: | +7.556000e+04 | +5.680000e+02 | +5.760000e+02 |\n> | pk-pk: | +5.615200e+04 | +9.600000e+01 | +1.040000e+02 |\n> | mean: | +3.292776e+04 | +5.072274e+02 | +5.073602e+02 |\n> | stddev: | +6.514709e+03 | +1.310849e+01 | +1.507144e+01 |\n> | count: | 600000 | 600000 | 2400000 |\n> \n> Using so_txtime, the peak to peak jitter is about 100 nanoseconds,\n\nNice!","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ext-mx04.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx04.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=mlichvar@redhat.com"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xxQbW4wj4z9s7m\n\tfor ;\n\tWed, 20 Sep 2017 00:43:19 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751729AbdISOnJ (ORCPT );\n\tTue, 19 Sep 2017 10:43:09 -0400","from mx1.redhat.com ([209.132.183.28]:33202 \"EHLO mx1.redhat.com\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1750892AbdISOnH (ORCPT );\n\tTue, 19 Sep 2017 10:43:07 -0400","from smtp.corp.redhat.com\n\t(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id C21AF800A4;\n\tTue, 19 Sep 2017 14:43:06 +0000 (UTC)","from localhost (unknown [10.43.2.117])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 1744160F89;\n\tTue, 19 Sep 2017 14:43:03 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com C21AF800A4","Date":"Tue, 19 Sep 2017 16:43:02 +0200","From":"Miroslav Lichvar ","To":"Richard Cochran ","Cc":"netdev@vger.kernel.org, Andre Guedes ,\n\tHenrik Austad , linux-kernel@vger.kernel.org,\n\tJesus Sanchez-Palencia ,\n\tintel-wired-lan@lists.osuosl.org, John Stultz ,\n\tThomas Gleixner ,\n\tAnna-Maria Gleixner ,\n\tDavid Miller ","Subject":"Re: [PATCH RFC V1 net-next 0/6] Time based packet transmission","Message-ID":"<20170919144302.GB4347@localhost>","References":"","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.12","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.28]);\n\tTue, 19 Sep 2017 14:43:07 +0000 (UTC)","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1771210,"web_url":"http://patchwork.ozlabs.org/comment/1771210/","msgid":"<20170919164615.mfv77pxuuyqc4zq4@localhost>","list_archive_url":null,"date":"2017-09-19T16:46:15","subject":"Re: [PATCH RFC V1 net-next 0/6] Time based packet transmission","submitter":{"id":4009,"url":"http://patchwork.ozlabs.org/api/people/4009/","name":"Richard Cochran","email":"richardcochran@gmail.com"},"content":"On Tue, Sep 19, 2017 at 04:43:02PM +0200, Miroslav Lichvar wrote:\n> If I understand it correctly, this also allows us to make a PTP/NTP\n> \"one-step\" clock with HW that doesn't support it directly.\n\nCool, yeah, I hadn't thought of that, but it would work...\n\nThanks,\nRichard","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"kJrMSSUU\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xxTL65cMfz9sPs\n\tfor ;\n\tWed, 20 Sep 2017 02:46:54 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751450AbdISQqi (ORCPT );\n\tTue, 19 Sep 2017 12:46:38 -0400","from mail-wr0-f179.google.com ([209.85.128.179]:54834 \"EHLO\n\tmail-wr0-f179.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1750973AbdISQqh (ORCPT\n\t); Tue, 19 Sep 2017 12:46:37 -0400","by mail-wr0-f179.google.com with SMTP id g29so151684wrg.11;\n\tTue, 19 Sep 2017 09:46:36 -0700 (PDT)","from localhost (213-225-0-7.nat.highway.a1.net. [213.225.0.7])\n\tby smtp.gmail.com with ESMTPSA id\n\tu1sm8905109wrd.95.2017.09.19.09.46.20\n\t(version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);\n\tTue, 19 Sep 2017 09:46:34 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=gmail.com; s=20161025;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=5HQC3O8X5r5BVRhVN2CkWOUoDlbLAKo1rpo9B4jbS2k=;\n\tb=kJrMSSUUg7aoAceXe7vwFuhSkd74JlCfBAzZRXrOslqHYF7d5yIFEH28piPTgDqK52\n\tnr0olcEK5bOIALn7uiOMSc2R+1GjrIGB9eVlU40er+Eap5DVti6KorfJRT8BIgUk/yfd\n\tI41FLrPiU8ruAM9NHLeAiIxzqa18sRl2TbefECR8gYB/ROWG7ERYqpm1b9PV5GAzWG86\n\t3ffy/tLiWsjNiJMwNaOPAp2qjkkgSuCJVYj7tgVl2FhQslHO7d2E7UeWgYvmguu2Ogfj\n\tsld24q7lKBt9/GkW1wmX3ITDeD8vBAVqaNyuDR9nVoMlYmPIuFLg7DtrdaZGw8nXAuJk\n\tJ8gA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=5HQC3O8X5r5BVRhVN2CkWOUoDlbLAKo1rpo9B4jbS2k=;\n\tb=uWFlnGNrUB4qTrwkN1MSjP+KCh3UWxcMvBXA7CrMAU6H2Qb9psdQpYM53peEwCAkMV\n\t4dW9eSzS345+Q6aQ5WojHCubzGBkc7vErT3DE2mYBzKQAGNQHT2QLM8O5Pltz8yKxDju\n\tU07A5Jd711Ea2zmcb0Q6VbQ3ARhdMqtAJigi2ZNcvh55C7FGnPSbWNW3EIxFpcroyyAd\n\tscUjK8oRJTYrGHxqVvZIC5gQsaZ/0u/DBikHvlY2SsIMsgnEAj2beYVCFa05iCf3En5I\n\tq3vMUpedZI5u4wCKBDQn1xVDE+1wnOTzp/KmtTbE+RMR+Mi9CMSVjYcUDskexKruSXsS\n\t9XWQ==","X-Gm-Message-State":"AHPjjUhehi8dZPPqFcf8DEk4RSm/CGysJpgE5B1gfJGUaXVTk3op0vmm\n\tcToSqb60eJJZTRcPc8m7rq0=","X-Google-Smtp-Source":"AOwi7QAQ/tGcdG3vkUkBpEYGkWjH6Dq06S2aK2v3iDo1PgJlUcBihXPDPADxpIuHft1/dfkyH4B5ww==","X-Received":"by 10.223.142.82 with SMTP id n76mr2172577wrb.272.1505839595849; \n\tTue, 19 Sep 2017 09:46:35 -0700 (PDT)","Date":"Tue, 19 Sep 2017 18:46:15 +0200","From":"Richard Cochran ","To":"Miroslav Lichvar ","Cc":"Richard Cochran , netdev@vger.kernel.org,\n\tAndre Guedes ,\n\tHenrik Austad , linux-kernel@vger.kernel.org,\n\tJesus Sanchez-Palencia ,\n\tintel-wired-lan@lists.osuosl.org, John Stultz ,\n\tThomas Gleixner ,\n\tAnna-Maria Gleixner ,\n\tDavid Miller ","Subject":"Re: [PATCH RFC V1 net-next 0/6] Time based packet transmission","Message-ID":"<20170919164615.mfv77pxuuyqc4zq4@localhost>","References":"\n\t<20170919144302.GB4347@localhost>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20170919144302.GB4347@localhost>","User-Agent":"NeoMutt/20170113 (1.7.2)","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1772109,"web_url":"http://patchwork.ozlabs.org/comment/1772109/","msgid":"<20170920173533.32537-1-levipearson@gmail.com>","list_archive_url":null,"date":"2017-09-20T17:35:33","subject":"Re: [PATCH RFC V1 net-next 0/6] Time based packet transmission","submitter":{"id":72396,"url":"http://patchwork.ozlabs.org/api/people/72396/","name":"Levi Pearson","email":"levipearson@gmail.com"},"content":"> This series is an early RFC that introduces a new socket option\n> allowing time based transmission of packets. This option will be\n> useful in implementing various real time protocols over Ethernet,\n> including but not limited to P802.1Qbv, which is currently finding\n> its way into 802.1Q.\n> \n> * Open questions about SO_TXTIME semantics\n> \n> - What should the kernel do if the dialed Tx time is in the past?\n> Should the packet be sent ASAP, or should we throw an error?\n\nBased on the i210 and latest NXP/Freescale FEC launch time behavior,\nthe hardware timestamps work over 1-second windows corresponding to\nthe time elapsed since the last PTP second began. When considering the\nhead-of-queue frame, the launch time is compared to the elapsed time\ncounter and if the elapsed time is between exactly the launch time and\nhalf a second after the launch time, it is launched. If you enqueue a\nframe with a scheduled launch time that ends up more than half a second\nlate, it is considered by the hardware to be scheduled *in the future*\nat the offset belonging to the next second after the 1-second window\nwraps around.\n\nSo *slightly* late (<<.5sec late) frames could be scheduled as normal,\nbut approaching .5sec late frames would have to either be dropped or \nhave their schedule changed to avoid blocking the queue for a large\nfraction of a second.\n\nI don't like the idea of changing the scheduled time, and anything that\nis close to half a second late is most likely useless. But it is also\nreasonable to let barely-late frames go out ASAP--in the case of a Qav-\nshaped stream, the bunching would get smoothed out downstream. A timed\nlaunch schedule need not be used as an exact time, but a \"don't send\nbefore time X\" flag. Both are useful in different circumstances.\n\nA configurable parameter for allowable lateness, with the upper bound\nset by the driver based on the hardware capabilities, seems ideal.\nBarring that, I would suggest dropping frames with already-missed\nlaunch times.\n\n> \n> - Should the kernel inform the user if it detects a missed deadline,\n> via the error queue for example?\n\nI think some sort of counter for mis-scheduled/late-delivered frames\nwould be in keeping with the general 802.1 error handling strategy.\n\n> \n> - What should the timescale be for the dialed Tx time? Should the\n> kernel select UTC when using the SW Qdisc and the HW time\n> otherwise? Or should the socket option include a clockid_t?\n\nWhen I implemented something like this, I left it relative to the HW\ntime for the sake of simplicity, but I don't have a strong opinion.\n\n> \n> * Things todo\n> \n> - Design a Qdisc for purpose of configuring SO_TXTIME. There should\n> be one option to dial HW offloading or SW best effort.\n\nYou seem focused on Qbv, but there is another aspect of the endpoint\nrequirements for Qav that this would provide a perfect use case for. A\nbridge can treat all traffic in a Qav-shaped class equally, but an\nendpoint must essentially run one credit-based shaper per active stream\nfeeding into the class--this is because a stream must adhere to its\nframes-per-interval promise in its t-spec, and when the observation\ninterval is not an even multiple of the sample rate, it will occasionally\nhave an observation interval with no frame. This leaves extra bandwidth\nin the class reservation, but it cannot be used by any other stream if\nit would cause more than one frame per interval to be sent!\n\nEven if a stream is not explicitly scheduled in userspace, a per-stream\nQdisc could apply a rough launch time that the class Qdisc (or hardware\nshaping) would use to ensure the frames-per-interval aspect of the\nreservation for the stream is adhered to. For example, each observation\ninterval could be assigned a launch time, and all streams would get a\nnumber of frames corresponding to their frames-per-interval reservation\nassigned that same launch time before being put into the class queue.\nThe i210's shaper would then only consider the current interval's set \nof frames ready to launch, and spread them evenly with its hardware\ncredit-based shaping.\n\nFor industrial and automotive control applications, a Qbv Qdisc based on\nSO_TXTIME would be very interesting, but pro and automotive media uses\nwill most likely continue to use SRP + Qav, and these are becoming\nincreasingly common uses as you can see by the growing support for Qav in\nautomotive chips.\n\n> - Implement the SW best effort variant. Here is my back of the\n> napkin sketch. Each interface has its own timerqueue keeping the\n> TXTIME packets in order and a FIFO for all other traffic. A guard\n> window starts at the earliest deadline minus the maximum MTU minus\n> a configurable fudge factor. The Qdisc uses a hrtimer to transmit\n> the next packet in the timerqueue. During the guard window, all\n> other traffic is defered unless the next packet can be transmitted\n> before the guard window expires.\n\nThis sounds plausible to me.\n\n> \n> * Current limitations\n> \n> - The driver does not handle out of order packets. If user space\n> sends a packet with an earlier Tx time, then the code should stop\n> the queue, reshuffle the descriptors accordingly, and then\n> restart the queue.\n\nYou might store the last scheduled timestamp in the driver private struct\nand drop any frame with a timestamp not greater or equal to the last one.\n\n> \n> - The driver does not correctly queue up packets in the distant\n> future. The i210 has a limited time window of +/- 0.5 seconds.\n> Packets with a Tx time greater than that should be deferred in\n> order to enqueue them later on.\n\nThe limit is not half a second in the future, but half a second from the\nprevious scheduled frame if one is enqueued. Another use case for the last\nscheduled frame field. There are definitely cases that might need to be\ndeferred though.\n\n> \n> * Performance measurements\n> \n> 1. Prepared a PC and the Device Under Test (DUT) each with an Intel\n> i210 card connected with a crossover cable.\n> 2. The DUT was a Pentium(R) D CPU 2.80GHz running PREEMPT_RT\n> 4.9.40-rt30 with about 50 usec maximum latency under cyclictest.\n> 3. Synchronized the DUT's PHC to the PC's PHC using ptp4l.\n> 4. Synchronized the DUT's system clock to its PHC using phc2sys.\n> 5. Started netperf to produce some network load.\n> 6. Measured the arrival time of the packets at the PC's PHC using\n> hardware time stamping.\n> \n> I ran ten minute tests both with and without using the so_txtime\n> option, with a period was 1 millisecond. I then repeated the\n> so_txtime case but with a 250 microsecond period. The measured\n> offset from the expected period (in nanoseconds) is shown in the\n> following table.\n> \n> | | plain preempt_rt | so_txtime | txtime @ 250 us |\n> |---------+------------------+---------------+-----------------|\n> | min: | +1.940800e+04 | +4.720000e+02 | +4.720000e+02 |\n> | max: | +7.556000e+04 | +5.680000e+02 | +5.760000e+02 |\n> | pk-pk: | +5.615200e+04 | +9.600000e+01 | +1.040000e+02 |\n> | mean: | +3.292776e+04 | +5.072274e+02 | +5.073602e+02 |\n> | stddev: | +6.514709e+03 | +1.310849e+01 | +1.507144e+01 |\n> | count: | 600000 | 600000 | 2400000 |\n> \n> Using so_txtime, the peak to peak jitter is about 100 nanoseconds,\n> independent of the period. In contrast, plain preempt_rt shows a\n> jitter of of 56 microseconds. The average delay of 507 nanoseconds\n> when using so_txtime is explained by the documented input and output\n> delays on the i210 cards.\n> \n> The test program is appended, below. If anyone is interested in\n> reproducing this test, I can provide helper scripts.\n> \n> Thanks,\n> Richard\n> \n\n< most of test program snipped >\n\n> \n> \t/*\n> \t * We specify the transmission time in the CMSG.\n> \t */\n> \tif (use_so_txtime) {\n> \t\tmsg.msg_control = u.buf;\n> \t\tmsg.msg_controllen = sizeof(u.buf);\n> \t\tcmsg = CMSG_FIRSTHDR(&msg);\n> \t\tcmsg->cmsg_level = SOL_SOCKET;\n> \t\tcmsg->cmsg_type = SO_TXTIME;\n> \t\tcmsg->cmsg_len = CMSG_LEN(sizeof(__u64));\n> \t\t*((__u64 *) CMSG_DATA(cmsg)) = txtime;\n> \t}\n> \tcnt = sendmsg(fd, &msg, 0);\n\nAn interesting use case I have explored is to increase efficiency by batching\ntransmissions with sendmmsg. This is attractive when getting large chunks of\naudio data from ALSA and scheduling them for transmit all at once.\n\nAnyway, I am wholly in favor of this proposal--in fact, it is very similar to\na patch set I shared with Eric Mann and others at Intel in early Dec 2016 with\nthe intention to get some early feedback before submitting here. I never heard\nback and got busy with other things. I only mention this since you said\nelsewhere that you got this idea from Eric Mann yourself, and I am curious\nwhether Eric and I came up with it independently (which I would not be\nsurprised at).\n\n\nLevi","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"ncsM+Ogs\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xy6Nb2X7Pz9s0g\n\tfor ;\n\tThu, 21 Sep 2017 03:36:15 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751666AbdITRfs (ORCPT );\n\tWed, 20 Sep 2017 13:35:48 -0400","from mail-pf0-f170.google.com ([209.85.192.170]:45161 \"EHLO\n\tmail-pf0-f170.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1751378AbdITRfr (ORCPT\n\t); Wed, 20 Sep 2017 13:35:47 -0400","by mail-pf0-f170.google.com with SMTP id z84so1876741pfi.2;\n\tWed, 20 Sep 2017 10:35:46 -0700 (PDT)","from brute.local (c-98-202-130-141.hsd1.ut.comcast.net.\n\t[98.202.130.141]) by smtp.gmail.com with ESMTPSA id\n\ts4sm8106651pgr.32.2017.09.20.10.35.43\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tWed, 20 Sep 2017 10:35:44 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=gmail.com; s=20161025;\n\th=from:to:cc:subject:date:message-id:in-reply-to:references;\n\tbh=pbekhRM+O7M6E4B9wrOifCuk0G4F6iVBFtKqi2wGuB4=;\n\tb=ncsM+OgsASrojReNLAFrRhpLAk9tKqsGJiGBI4z88/pdh08zxOmD8rDRYW/WMEkxfl\n\txeFCm94pgIRsemmVV98t+sajJNqDN/83orW54k3U/rfRHkUJk9tZxablH3CDtgqSEdZt\n\th7NrRhV2KhqZX8A3zGhz7GtDCtFEVBx0+w8q4XHNKYDPbpz/wNMMmnxTzITvzLyTvPW0\n\terVC8jXjE1yvjbbHC5EbmkLbxH3n/+QUUySCIvV487HzvRym1w+oRsD4m70OxL1D+qLL\n\tppt2LEfCNUvffWgTGmrpzhkOTQp39bsQjSqvnh75g7ssSJzSeyTq2PYMyZqXO92xtB0A\n\t+EUQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references;\n\tbh=pbekhRM+O7M6E4B9wrOifCuk0G4F6iVBFtKqi2wGuB4=;\n\tb=XTwRwPxrwO3NqSQeqW1q4YcQwmR0IYLXqhJ6YZd5kGhz+OlN/FzlDVQ6zrDE5PYFhP\n\trFCRpXHr7wOekHg3dBgGQdd2LSQA7xmb1DHdA7TULzm9hFxmc8MXsVWue7L2Eu4GYYBY\n\tky9fNDjRr+zzQaErTzu5TFUcZfNk4+E9/cZLLkJdfYtxztHA48rzE5eoY+QNx+O6ssX3\n\tRzITT9wdVxKISvcRcO//SOB4KL6kSjX9yVcAU5pXP4goVnXPwe5swDNt9htUZNyCTz1P\n\tIXfln30ONtA3rIq2kHMroYazV6zDO1FkVZyHFW0T4sxs38boXpDkuwop5vv4EDRN/2wq\n\t8cvQ==","X-Gm-Message-State":"AHPjjUj/r2mv2EFU2Fyt8TyrOxQD9aw+EA/ZkSgNkD4NOGY32LwkcU1P\n\tyk4oZwNMeJxZcwscA/+4CvI=","X-Google-Smtp-Source":"AOwi7QBiDVitUJ4AqQeEgyQFIrYtCiIRWOEEtPGOUJXywkk+qiGBBLSj3xNzmnZ1UT2arJLU7Zfj7w==","X-Received":"by 10.98.147.219 with SMTP id r88mr2950177pfk.0.1505928945991;\n\tWed, 20 Sep 2017 10:35:45 -0700 (PDT)","From":"levipearson@gmail.com","To":"rcochran@linutronix.de","Cc":"netdev@vger.kernel.org, linux-kernel@vger.kernel.org,\n\tintel-wired-lan@lists.osuosl.org, vinicius.gomes@intel.com,\n\tandre.guedes@intel.com, john.stultz@linaro.org,\n\tjesus.sanchez-palencia@intel.com, henrik@austad.us,\n\ttglx@linutronix.de, anna-maria@linutronix.de, davem@davemloft.net","Subject":"Re: [PATCH RFC V1 net-next 0/6] Time based packet transmission","Date":"Wed, 20 Sep 2017 11:35:33 -0600","Message-Id":"<20170920173533.32537-1-levipearson@gmail.com>","X-Mailer":"git-send-email 2.14.1","In-Reply-To":"","References":"","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1772181,"web_url":"http://patchwork.ozlabs.org/comment/1772181/","msgid":"<20170920201153.d3o3h62y2snuacef@localhost>","list_archive_url":null,"date":"2017-09-20T20:11:53","subject":"Re: [PATCH RFC V1 net-next 0/6] Time based packet transmission","submitter":{"id":4009,"url":"http://patchwork.ozlabs.org/api/people/4009/","name":"Richard Cochran","email":"richardcochran@gmail.com"},"content":"On Wed, Sep 20, 2017 at 11:35:33AM -0600, levipearson@gmail.com wrote:\n> Anyway, I am wholly in favor of this proposal--in fact, it is very similar to\n> a patch set I shared with Eric Mann and others at Intel in early Dec 2016 with\n> the intention to get some early feedback before submitting here. I never heard\n> back and got busy with other things. I only mention this since you said\n> elsewhere that you got this idea from Eric Mann yourself, and I am curious\n> whether Eric and I came up with it independently (which I would not be\n> surprised at).\n\nWell, I actually thought of placing the Tx time in a CMSG all by\nmyself, but later I found Eric's talk from 2012,\n\n https://linuxplumbers.ubicast.tv/videos/linux-network-enabling-requirements-for-audiovideo-bridging-avb/\n\nand so I wanted to give him credit.\n\nThanks,\nRichard","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"b40AzQU4\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xy9rc3GHvz9s7v\n\tfor ;\n\tThu, 21 Sep 2017 06:12:16 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751499AbdITUMA (ORCPT );\n\tWed, 20 Sep 2017 16:12:00 -0400","from mail-wr0-f173.google.com ([209.85.128.173]:47744 \"EHLO\n\tmail-wr0-f173.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1750938AbdITUL7 (ORCPT\n\t); Wed, 20 Sep 2017 16:11:59 -0400","by mail-wr0-f173.google.com with SMTP id k20so3049844wre.4;\n\tWed, 20 Sep 2017 13:11:58 -0700 (PDT)","from localhost (213-225-0-7.nat.highway.a1.net. [213.225.0.7])\n\tby smtp.gmail.com with ESMTPSA id m64sm40966wmb.0.2017.09.20.13.11.55\n\t(version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);\n\tWed, 20 Sep 2017 13:11:56 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=gmail.com; s=20161025;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=v/aO1hmGflPz448aIZkWq2UxLuTYVQcfeN20Ilis9Wk=;\n\tb=b40AzQU49OdqooR0Ehm/v8BeFsN+if26zh5fdMcuwCxFQKn4/KD0ytPjkkyBZAveU7\n\tMNq7lpnLwoqsy8RNNaieUoP3yVKQoq8fznlwH7q4u9EuKtjVigfJ5yTQXEEkzrLK64S9\n\tBgDSeDz+rV8R4Ud3FeFfj3GLu1OMHYz1vkMQMpTomvypgszwtshcj8Mn4Q9hOGAni6F9\n\tbj41FKO7qgM9/euYv+aRnRElzy9nUHYUzUAiuXuylYQPyj6YpbLIBXzjYagFOfK8Cplv\n\tAuoACsBeXInvBfJAfl4U8OWy5+mpXBdWOalOOrYJ4Z8WqJ+sLT7qSOV9fAZs6iZqthyr\n\t4lKA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=v/aO1hmGflPz448aIZkWq2UxLuTYVQcfeN20Ilis9Wk=;\n\tb=Evw5kk14S8PNwg/WfbowSt4WS4AQZUPbSa/g5E36dJ21ooknQ9HvmidEGrbrRPp9dc\n\tVX1g6lU/HzzmkoNuhYh5D7qagxcObeyn/pIWtJH8ne5L2mxieuZ3XBysaMaitZbQPVbt\n\tp+ops4TfSzMLvATcs4uz50ABOcx2r+tVuyzYWPNZdffFbH38vxUHJKrH2M2q/nVR57Pj\n\tlsHIj7KdkHUj141BrVmontAz2ci+Ttx80poQsFasBrPiISHAIPVqXBLk8nnXQX2jsMS1\n\t7PO8mA6KWw9K8pofjDYmwRJQkklL9gCfC+PIpho86pJFkj/eKE8b1FZGibgyYxFBvchC\n\tV6Jw==","X-Gm-Message-State":"AHPjjUjMTVId1V81QWlADIo0YrjHHbWMTm59/kcO9O7PnwVzpWk4lz+A\n\taxqzlNZUNkADK3o43W3PvyI=","X-Google-Smtp-Source":"AOwi7QDpTlmVinFt0SymQu9NlHsV6QS8L+1LujSYO4cCu7xzdzDxn0o7xNjFD9TNfrYg0u/ktdwd/g==","X-Received":"by 10.223.142.199 with SMTP id q65mr6180173wrb.110.1505938317825;\n\tWed, 20 Sep 2017 13:11:57 -0700 (PDT)","Date":"Wed, 20 Sep 2017 22:11:53 +0200","From":"Richard Cochran ","To":"levipearson@gmail.com","Cc":"rcochran@linutronix.de, netdev@vger.kernel.org,\n\tlinux-kernel@vger.kernel.org, intel-wired-lan@lists.osuosl.org,\n\tvinicius.gomes@intel.com, andre.guedes@intel.com,\n\tjohn.stultz@linaro.org, jesus.sanchez-palencia@intel.com,\n\thenrik@austad.us, tglx@linutronix.de, anna-maria@linutronix.de,\n\tdavem@davemloft.net","Subject":"Re: [PATCH RFC V1 net-next 0/6] Time based packet transmission","Message-ID":"<20170920201153.d3o3h62y2snuacef@localhost>","References":"\n\t<20170920173533.32537-1-levipearson@gmail.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20170920173533.32537-1-levipearson@gmail.com>","User-Agent":"NeoMutt/20170113 (1.7.2)","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1790014,"web_url":"http://patchwork.ozlabs.org/comment/1790014/","msgid":"<743a4550-7344-5e73-bf6d-6ec368263ad9@intel.com>","list_archive_url":null,"date":"2017-10-18T22:18:55","subject":"Re: [PATCH RFC V1 net-next 0/6] Time based packet transmission","submitter":{"id":72279,"url":"http://patchwork.ozlabs.org/api/people/72279/","name":"Jesus Sanchez-Palencia","email":"jesus.sanchez-palencia@intel.com"},"content":"Hi Richard,\n\n\nOn 09/18/2017 12:41 AM, Richard Cochran wrote:\n> This series is an early RFC that introduces a new socket option\n> allowing time based transmission of packets. This option will be\n> useful in implementing various real time protocols over Ethernet,\n> including but not limited to P802.1Qbv, which is currently finding\n> its way into 802.1Q.\n> \n> * Open questions about SO_TXTIME semantics\n> \n> - What should the kernel do if the dialed Tx time is in the past?\n> Should the packet be sent ASAP, or should we throw an error?\n> \n> - Should the kernel inform the user if it detects a missed deadline,\n> via the error queue for example?\n> \n> - What should the timescale be for the dialed Tx time? Should the\n> kernel select UTC when using the SW Qdisc and the HW time\n> otherwise? Or should the socket option include a clockid_t?\n> \n> * Things todo\n> \n> - Design a Qdisc for purpose of configuring SO_TXTIME. There should\n> be one option to dial HW offloading or SW best effort.\n> \n> - Implement the SW best effort variant. Here is my back of the\n> napkin sketch. Each interface has its own timerqueue keeping the\n> TXTIME packets in order and a FIFO for all other traffic. A guard\n> window starts at the earliest deadline minus the maximum MTU minus\n> a configurable fudge factor. The Qdisc uses a hrtimer to transmit\n> the next packet in the timerqueue. During the guard window, all\n> other traffic is defered unless the next packet can be transmitted\n> before the guard window expires.\n\n\nEven for HW offloading this timerqueue could be used for enforcing that packets\nare always sorted by their launch time when they get enqueued into the\nnetdevice. Of course, assuming that this would be something that we'd like to\nprovide from within the kernel.\n\n\n\n> \n> * Current limitations\n> \n> - The driver does not handle out of order packets. If user space\n> sends a packet with an earlier Tx time, then the code should stop\n> the queue, reshuffle the descriptors accordingly, and then\n> restart the queue.\n\n\nWouldn't be an issue if the above was provided.\n\n\n\n> \n> - The driver does not correctly queue up packets in the distant\n> future. The i210 has a limited time window of +/- 0.5 seconds.\n> Packets with a Tx time greater than that should be deferred in\n> order to enqueue them later on.\n> \n> * Performance measurements\n> \n> 1. Prepared a PC and the Device Under Test (DUT) each with an Intel\n> i210 card connected with a crossover cable.\n> 2. The DUT was a Pentium(R) D CPU 2.80GHz running PREEMPT_RT\n> 4.9.40-rt30 with about 50 usec maximum latency under cyclictest.\n> 3. Synchronized the DUT's PHC to the PC's PHC using ptp4l.\n> 4. Synchronized the DUT's system clock to its PHC using phc2sys.\n> 5. Started netperf to produce some network load.\n> 6. Measured the arrival time of the packets at the PC's PHC using\n> hardware time stamping.\n> \n> I ran ten minute tests both with and without using the so_txtime\n> option, with a period was 1 millisecond. I then repeated the\n> so_txtime case but with a 250 microsecond period. The measured\n> offset from the expected period (in nanoseconds) is shown in the\n> following table.\n> \n> | | plain preempt_rt | so_txtime | txtime @ 250 us |\n> |---------+------------------+---------------+-----------------|\n> | min: | +1.940800e+04 | +4.720000e+02 | +4.720000e+02 |\n> | max: | +7.556000e+04 | +5.680000e+02 | +5.760000e+02 |\n> | pk-pk: | +5.615200e+04 | +9.600000e+01 | +1.040000e+02 |\n> | mean: | +3.292776e+04 | +5.072274e+02 | +5.073602e+02 |\n> | stddev: | +6.514709e+03 | +1.310849e+01 | +1.507144e+01 |\n> | count: | 600000 | 600000 | 2400000 |\n> \n> Using so_txtime, the peak to peak jitter is about 100 nanoseconds,\n> independent of the period. In contrast, plain preempt_rt shows a\n> jitter of of 56 microseconds. The average delay of 507 nanoseconds\n> when using so_txtime is explained by the documented input and output\n> delays on the i210 cards.\n\n\nThis is great. Just out of curiosity, were you using vlans on your tests?\n\n\n> \n> The test program is appended, below. If anyone is interested in\n> reproducing this test, I can provide helper scripts.\n\n\nI might try to reproduce them soon. I would appreciate if you could provide me\nwith the scripts, please.\n\n\nThanks,\nJesus\n\n\n\n\n> \n> Thanks,\n> Richard\n> \n> \n> Richard Cochran (6):\n> net: Add a new socket option for a future transmit time.\n> net: skbuff: Add a field to support time based transmission.\n> net: ipv4: raw: Hook into time based transmission.\n> net: ipv4: udp: Hook into time based transmission.\n> net: packet: Hook into time based transmission.\n> net: igb: Implement time based transmission.\n> \n> arch/alpha/include/uapi/asm/socket.h | 3 ++\n> arch/frv/include/uapi/asm/socket.h | 3 ++\n> arch/ia64/include/uapi/asm/socket.h | 3 ++\n> arch/m32r/include/uapi/asm/socket.h | 3 ++\n> arch/mips/include/uapi/asm/socket.h | 3 ++\n> arch/mn10300/include/uapi/asm/socket.h | 3 ++\n> arch/parisc/include/uapi/asm/socket.h | 3 ++\n> arch/powerpc/include/uapi/asm/socket.h | 3 ++\n> arch/s390/include/uapi/asm/socket.h | 3 ++\n> arch/sparc/include/uapi/asm/socket.h | 3 ++\n> arch/xtensa/include/uapi/asm/socket.h | 3 ++\n> drivers/net/ethernet/intel/igb/e1000_82575.h | 1 +\n> drivers/net/ethernet/intel/igb/e1000_defines.h | 68 +++++++++++++++++++++++++-\n> drivers/net/ethernet/intel/igb/e1000_regs.h | 5 ++\n> drivers/net/ethernet/intel/igb/igb.h | 3 +-\n> drivers/net/ethernet/intel/igb/igb_main.c | 68 +++++++++++++++++++++++---\n> include/linux/skbuff.h | 2 +\n> include/net/sock.h | 2 +\n> include/uapi/asm-generic/socket.h | 3 ++\n> net/core/sock.c | 12 +++++\n> net/ipv4/raw.c | 2 +\n> net/ipv4/udp.c | 5 +-\n> net/packet/af_packet.c | 6 +++\n> 23 files changed, 200 insertions(+), 10 deletions(-)\n>","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yHRVf6dhMz9s82\n\tfor ;\n\tThu, 19 Oct 2017 09:26:34 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751108AbdJRW0X (ORCPT );\n\tWed, 18 Oct 2017 18:26:23 -0400","from mga14.intel.com ([192.55.52.115]:48417 \"EHLO mga14.intel.com\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1750920AbdJRW0V (ORCPT );\n\tWed, 18 Oct 2017 18:26:21 -0400","from orsmga004.jf.intel.com ([10.7.209.38])\n\tby fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t18 Oct 2017 15:26:21 -0700","from darjeeling.jf.intel.com (HELO [10.7.159.56]) ([10.7.159.56])\n\tby orsmga004.jf.intel.com with ESMTP; 18 Oct 2017 15:26:20 -0700"],"X-ExtLoop1":"1","X-IronPort-AV":"E=Sophos;i=\"5.43,398,1503385200\"; d=\"scan'208\";a=\"139811453\"","Subject":"Re: [PATCH RFC V1 net-next 0/6] Time based packet transmission","To":"Richard Cochran ","Cc":"netdev@vger.kernel.org, linux-kernel@vger.kernel.org,\n\tintel-wired-lan@lists.osuosl.org, Andre Guedes ,\n\tAnna-Maria Gleixner ,\n\tDavid Miller , Henrik Austad ,\n\tJohn Stultz ,\n\tThomas Gleixner ,\n\tVinicius Costa Gomes ,\n\t\"Briano, Ivan\" ,\n\tLevi Pearson ","References":"","From":"Jesus Sanchez-Palencia ","Message-ID":"<743a4550-7344-5e73-bf6d-6ec368263ad9@intel.com>","Date":"Wed, 18 Oct 2017 15:18:55 -0700","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1790903,"web_url":"http://patchwork.ozlabs.org/comment/1790903/","msgid":"<20171019204409.vmkhii2vocum43l6@localhost>","list_archive_url":null,"date":"2017-10-19T20:44:09","subject":"Re: [PATCH RFC V1 net-next 0/6] Time based packet transmission","submitter":{"id":4009,"url":"http://patchwork.ozlabs.org/api/people/4009/","name":"Richard Cochran","email":"richardcochran@gmail.com"},"content":"On Wed, Oct 18, 2017 at 03:18:55PM -0700, Jesus Sanchez-Palencia wrote:\n> This is great. Just out of curiosity, were you using vlans on your tests?\n\nNo, just raw packets. VLAN tags could be added trivially (in the\nprogram), but that naturally avoids the kernel's VLAN code.\n\n> I might try to reproduce them soon. I would appreciate if you could provide me\n> with the scripts, please.\n\nOk, will do.\n\nThanks,\nRichard","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"RDkHb5Gv\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yJ1BR0MNGz9t5x\n\tfor ;\n\tFri, 20 Oct 2017 07:44:31 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1753728AbdJSUoT (ORCPT );\n\tThu, 19 Oct 2017 16:44:19 -0400","from mail-pf0-f177.google.com ([209.85.192.177]:53732 \"EHLO\n\tmail-pf0-f177.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1751855AbdJSUoR (ORCPT\n\t); Thu, 19 Oct 2017 16:44:17 -0400","by mail-pf0-f177.google.com with SMTP id t188so7693139pfd.10;\n\tThu, 19 Oct 2017 13:44:17 -0700 (PDT)","from localhost ([12.247.123.130]) by smtp.gmail.com with ESMTPSA id\n\ta1sm23767167pgu.47.2017.10.19.13.44.16\n\t(version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);\n\tThu, 19 Oct 2017 13:44:16 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=gmail.com; s=20161025;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=neF5hKRsY8OHNFOAHQS3TgxKeg1advnKImhR4RPU0PQ=;\n\tb=RDkHb5Gv7t3tudlcjNfrPgmBXwyrPRoc7yp6nxbkcDHIAZ7eU8g4kbHT32c7aXaox+\n\tpfE3jeSFRh3mEUgooQnb49ZhrDs513dQOrJV9H9qaihtOGAbbvfJX4a/CPQE72jeHDgk\n\t031mzJnq+6S+W07Yy1d9PULPkw740S8GGRv6IeH7Lbzn6b/4JxTF5sdleQf7u2rOSItm\n\t75jaqwNpbxWAFHNcbsqQAE/tLsEFk33qEma9Njhw+tQ4W9YUkL3QYhXyELmZ1aKJrUqX\n\tvcKPezmKQBGIOR/0eNjbUygzuXzhzHLaj2LmzYIZtYOxxkZEWBKT6ki9Bf0mB0BJKdZ1\n\tM1UQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=neF5hKRsY8OHNFOAHQS3TgxKeg1advnKImhR4RPU0PQ=;\n\tb=U75mCmzFDf/Yc751ImrvcqAIMBCwBwmmAAJM9ec0pTBxUjjW4kHWeXxDBaKh6BJ0qf\n\t0lDlBbXtRA6wY4UhLKC5WatNccLusxd7plpB2YO56B7hciUNb7asxanbppH+M0FPrhcv\n\tyH0Qewi1v46qt902xQpdRuS7bPPDyEJWMWL2XumM0DDYqV4fRkgyYZiw0q50I9a5+lIC\n\teRlVDzbuSSM5pUccoLJszEUbZSYWokZ+Puvpax5OMvNhlUNlN1zSc/kNMT7M2fJtYowD\n\tvtXCBnHd6dWiSrNo7M/2EvdNSBhnsDilwxoCYWfheajuLkvUzHqd1RyHgr13qgGJunV7\n\tELZQ==","X-Gm-Message-State":"AMCzsaU9m/QQ0LegVQS3W/ed5IhMDxQSncOVUkkUIP8tNwTTDFeBsoAg\n\tBkErnGhNLVCY4iGaq8nWbaw=","X-Google-Smtp-Source":"ABhQp+RWP0s+i+DvIxgZQIAnoyYwW8HmkIFdtZ8pNQoMlMvsA5aBStqoToz3R8KC60JgJOu1QGGlyQ==","X-Received":"by 10.84.168.129 with SMTP id f1mr2370375plb.71.1508445857178;\n\tThu, 19 Oct 2017 13:44:17 -0700 (PDT)","Date":"Thu, 19 Oct 2017 13:44:09 -0700","From":"Richard Cochran ","To":"Jesus Sanchez-Palencia ","Cc":"Richard Cochran , netdev@vger.kernel.org,\n\tlinux-kernel@vger.kernel.org, intel-wired-lan@lists.osuosl.org,\n\tAndre Guedes ,\n\tAnna-Maria Gleixner ,\n\tDavid Miller , Henrik Austad ,\n\tJohn Stultz ,\n\tThomas Gleixner ,\n\tVinicius Costa Gomes ,\n\t\"Briano, Ivan\" ,\n\tLevi Pearson ","Subject":"Re: [PATCH RFC V1 net-next 0/6] Time based packet transmission","Message-ID":"<20171019204409.vmkhii2vocum43l6@localhost>","References":"\n\t<743a4550-7344-5e73-bf6d-6ec368263ad9@intel.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<743a4550-7344-5e73-bf6d-6ec368263ad9@intel.com>","User-Agent":"NeoMutt/20170113 (1.7.2)","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1816492,"web_url":"http://patchwork.ozlabs.org/comment/1816492/","msgid":"<87609kj2f4.fsf@intel.com>","list_archive_url":null,"date":"2017-12-05T21:22:07","subject":"Re: [PATCH RFC V1 net-next 0/6] Time based packet transmission","submitter":{"id":72272,"url":"http://patchwork.ozlabs.org/api/people/72272/","name":"Vinicius Costa Gomes","email":"vinicius.gomes@intel.com"},"content":"Hi David,\n\nDavid Miller writes:\n\n> From: Richard Cochran \n> Date: Mon, 18 Sep 2017 09:41:15 +0200\n>\n>> - The driver does not handle out of order packets. If user space\n>> sends a packet with an earlier Tx time, then the code should stop\n>> the queue, reshuffle the descriptors accordingly, and then\n>> restart the queue.\n>\n> The user should simply be not allowed to do this.\n>\n> Once the packet is in the device queue, that's it. You cannot insert\n> a new packet to be transmitted before an already hw queued packet,\n> period.\n>\n> Any out of order request should be rejected with an error.\n\nJust to clarify, I agree that after after the packet is enqueued to the\nHW, there's no going back, in another words, we must never enqueue\nanything to the HW with a timestamp earlier than the last enqueued\npacket.\n\nBut re-ordering packets at the Qdisc level is, I think, necessary: two\napplications (one (A) with period of 50us and the other (B) of 100us),\nif it happens that (B) enqueues its packet before (A), I think, we would\nhave a problem.\n\nThe problem is deciding for how long we should keep packets in the Qdisc\nqueue. In the implementation we are working on, this is left for the\nuser to decide.\n\nOr do you have a reason for not doing *any* kind of re-ordering?\n\n>\n> I'd say the same is true for requests to send packets timed\n> in the past.\n\n+1\n\n\nCheers,\n--\nVinicius","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yrvpg2CYCz9s9Y\n\tfor ;\n\tWed, 6 Dec 2017 08:22:35 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751630AbdLEVWJ (ORCPT );\n\tTue, 5 Dec 2017 16:22:09 -0500","from mga01.intel.com ([192.55.52.88]:50268 \"EHLO mga01.intel.com\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1751066AbdLEVWI (ORCPT );\n\tTue, 5 Dec 2017 16:22:08 -0500","from fmsmga006.fm.intel.com ([10.253.24.20])\n\tby fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t05 Dec 2017 13:22:08 -0800","from ellie.jf.intel.com (HELO ellie) ([10.24.13.63])\n\tby fmsmga006.fm.intel.com with ESMTP; 05 Dec 2017 13:22:07 -0800"],"X-ExtLoop1":"1","X-IronPort-AV":"E=Sophos;i=\"5.45,365,1508828400\"; d=\"scan'208\";a=\"184384524\"","From":"Vinicius Costa Gomes ","To":"David Miller , rcochran@linutronix.de","Cc":"netdev@vger.kernel.org, linux-kernel@vger.kernel.org,\n\tintel-wired-lan@lists.osuosl.org, andre.guedes@intel.com,\n\tanna-maria@linutronix.de, henrik@austad.us,\n\tjesus.sanchez-palencia@intel.com, john.stultz@linaro.org,\n\ttglx@linutronix.de","Subject":"Re: [PATCH RFC V1 net-next 0/6] Time based packet transmission","In-Reply-To":"<20170918.093436.1069699729581966538.davem@davemloft.net>","References":"\n\t<20170918.093436.1069699729581966538.davem@davemloft.net>","User-Agent":"Notmuch/0.21 (http://notmuchmail.org) Emacs/25.3.1\n\t(x86_64-pc-linux-gnu)","Date":"Tue, 05 Dec 2017 13:22:07 -0800","Message-ID":"<87609kj2f4.fsf@intel.com>","MIME-Version":"1.0","Content-Type":"text/plain","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}}]