diff mbox

fixing hw timestamping in igb

Message ID 9D02360A-8016-4BCB-B14A-DAFE22EF558E@halon.se
State Awaiting Upstream, archived
Delegated to: David Miller
Headers show

Commit Message

Anders Berggren Feb. 3, 2011, 10:39 a.m. UTC
Hardware timestamping for Intel 82580 didn't work in either 2.6.36 or 2.6.37. Comparing it to Intel's igb-2.4.12 I found that the timecounter_init clock/counter initialization was done too early. 

Anders Berggren
Halon Security

lab-slang-1:~# diff -u linux-2.6.37/drivers/net/igb/igb_main.c linux/drivers/net/igb/igb_main.c

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Comments

Kirsher, Jeffrey T Feb. 3, 2011, 11:23 p.m. UTC | #1
On Thu, 2011-02-03 at 02:39 -0800, Anders Berggren wrote:
> Hardware timestamping for Intel 82580 didn't work in either 2.6.36 or 2.6.37. Comparing it to Intel's igb-2.4.12 I found that the timecounter_init clock/counter initialization was done too early. 
> 
> Anders Berggren
> Halon Security

Thanks Anders, I will add this patch to my queue of igb patches.
Anders Berggren Feb. 8, 2011, 3:25 p.m. UTC | #2
On Feb 4, 2011, at 12:23 AM, Jeff Kirsher wrote:

> Thanks Anders, I will add this patch to my queue of igb patches.

Another question which I asked John Ronciak earlier; is there any work on TX timestamping for IPv6? In 2.6.37 sock_tx_timestamp() is only called in IPv4 UDP (net/ipv4/udp.c) and raw sockets (net/packet/af_packet.c).

Would it be meaningful if I tried to add it to ipv6/ip6_output.c as Marcus D. Leech suggested in http://kerneltrap.org/mailarchive/linux-netdev/2009/11/10/6260604 and http://kerneltrap.org/mailarchive/linux-netdev/2009/11/11/6260643 --
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Ronciak, John Feb. 8, 2011, 4:27 p.m. UTC | #3
> -----Original Message-----
> From: Anders Berggren [mailto:anders@halon.se]
> Sent: Tuesday, February 08, 2011 7:25 AM
> To: Kirsher, Jeffrey T
> Cc: Ronciak, John; e1000-devel@lists.sourceforge.net;
> netdev@vger.kernel.org
> Subject: Re: [E1000-devel] [PATCH] fixing hw timestamping in igb
> 
> On Feb 4, 2011, at 12:23 AM, Jeff Kirsher wrote:
> 
> > Thanks Anders, I will add this patch to my queue of igb patches.
> 
> Another question which I asked John Ronciak earlier; is there any work
> on TX timestamping for IPv6? In 2.6.37 sock_tx_timestamp() is only
> called in IPv4 UDP (net/ipv4/udp.c) and raw sockets
> (net/packet/af_packet.c).
> 
> Would it be meaningful if I tried to add it to ipv6/ip6_output.c as
> Marcus D. Leech suggested in http://kerneltrap.org/mailarchive/linux-
> netdev/2009/11/10/6260604 and http://kerneltrap.org/mailarchive/linux-
> netdev/2009/11/11/6260643
Anders,

The question of IPv6 support for TX timestamping is still under discussion.  We are trying to understand the use case for it as well as how it would be used.  We have had no customers asking for this type of support (at least not yet).  If there is kernel work going on regarding it then maybe this needs to be looked at closer.

Other than being a science project, what is your use case for the TX timestamping in IPv6?

Cheers,
John
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Anders Berggren Feb. 9, 2011, 8:24 a.m. UTC | #4
On Feb 8, 2011, at 5:27 PM, Ronciak, John wrote:

> The question of IPv6 support for TX timestamping is still under discussion.  We are trying to understand the use case for it as well as how it would be used.  We have had no customers asking for this type of support (at least not yet).  If there is kernel work going on regarding it then maybe this needs to be looked at closer. Other than being a science project, what is your use case for the TX timestamping in IPv6?

We're developing a "hardware/kernel timestamp ping" for Linux in C. It's part of a complete SLA platform (appliance based on Debian with XMLRPC APIs). It's both a science project (Chalmers University, Gothenburg) and a commercial project (Tele2).

Commercial SLA appliances from Cisco and Juniper typically have a accuracy of XXX microseconds. Our hardware ping, using Intel 82580 NICs, have an accuracy of 8 nanoseconds. That is VERY good.

The reason for supporting IPv6 is, well, because it's the future ;) --
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Joe Perches Feb. 9, 2011, 8:56 a.m. UTC | #5
On Wed, 2011-02-09 at 09:24 +0100, Anders Berggren wrote:
> Our hardware ping, using Intel 82580 NICs,
> have an accuracy of 8 nanoseconds.

Perhaps you mean 8 nanosecond resolution?
Is documentation available for this claim?


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Anders Berggren Feb. 9, 2011, 9:27 a.m. UTC | #6
On Feb 9, 2011, at 9:56 AM, Joe Perches wrote:

> On Wed, 2011-02-09 at 09:24 +0100, Anders Berggren wrote:
>> Our hardware ping, using Intel 82580 NICs,
>> have an accuracy of 8 nanoseconds.
> 
> Perhaps you mean 8 nanosecond resolution?
> Is documentation available for this claim?

Well, both. 8 ns is the accuracy when performing RTT (round-trip time) measurements using our hardware ping tool. We tested it by connecting a 10 m (European meters ;) CAT6 copper and measuring the jitter. The RTT is always 1240 ns or 1248 ns, hence the accuracy of 8 ns.

$ sudo ./probed -c 10.10.10.3 -i eth2 -p 666
SLA-NG probed 0.1
probed: Binding port 666
probed: Using hardware timestamps
probed: client: ::ffff:10.10.10.3: Connecting to port 666
probed: client: ::ffff:10.10.10.3: Connected
Response    1 from 0 in 1240 ns
Response    2 from 0 in 1248 ns
Response    3 from 0 in 1248 ns
Response    4 from 0 in 1248 ns
^C
4 ok, 0 ts err, 0 lost pong, 0 timeout, 0 dup, 0.000000% loss
max: 1248 ns, avg: 1246 ns, min: 1240 ns

This is our tool: 

$ ./probed 
SLA-NG probed 0.1
usage: probed [-saqd] [-c addr] [-t type] [-i iface] [-p port] [-f file]

	          MODES OF OPERATION
	-c addr   Client mode: PING 'addr', fetch UDP timestamps
	-s        Server mode: respond to PING, send UDP timestamps
	-d        Daemon mode: both server and client, output to pipe

	          OPTIONS
	-k        Create timestamps in kernel driver instead of hardware
	-u        Create timestamps in userland instead of hardware
	-i iface  Network interface used for hardware timestamping
	-p port   UDP port, both source and destination
	-w usecs  Client mode wait time between PINGs, in microseconds
	-v        Output more debugging
	-q        Be quiet, log error to syslog only
	-f file   Path to configuration file

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Anders Berggren Feb. 9, 2011, 9:51 a.m. UTC | #7
On Feb 9, 2011, at 9:56 AM, Joe Perches wrote:

> Perhaps you mean 8 nanosecond resolution?
> Is documentation available for this claim?

To clarify; we haven't proven this either mathematically not in a laboratory (looking at actual data transmissions). Also, it's the jitter accuracy we're concerned with. We have simply assumed that the jitter of two independent servers (exchanging timestamps) over a very short cable is an approximation of jitter accuracy.

As you say; if the resolution is 8 ns, the accuracy is of course worse than 8 ns. So far, we have however assumed the accuracy to be within the same order of magnitude.--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

--- linux-2.6.37/drivers/net/igb/igb_main.c	2011-02-03 10:02:53.000000000 +0100
+++ linux/drivers/net/igb/igb_main.c	2011-02-03 10:12:40.000000000 +0100
@@ -98,6 +98,7 @@ 
 static void igb_setup_mrqc(struct igb_adapter *);
 static int igb_probe(struct pci_dev *, const struct pci_device_id *);
 static void __devexit igb_remove(struct pci_dev *pdev);
+static void igb_init_hw_timer(struct igb_adapter *adapter);
 static int igb_sw_init(struct igb_adapter *);
 static int igb_open(struct net_device *);
 static int igb_close(struct net_device *);
@@ -1987,6 +1988,10 @@ 
 	}
 
 #endif
+
+	/* do hw tstamp init after resetting */ 
+	igb_init_hw_timer(adapter);
+
 	dev_info(&pdev->dev, "Intel(R) Gigabit Ethernet Network Connection\n");
 	/* print bus type/speed/width info */
 	dev_info(&pdev->dev, "%s: (PCIe:%s:%s) %pM\n",
@@ -2301,7 +2306,6 @@ 
 		return -ENOMEM;
 	}
 
-	igb_init_hw_timer(adapter);
 	igb_probe_vfs(adapter);
 
 	/* Explicitly disable IRQ since the NIC can be in any state. */