| Message ID | MW5PR22MB2988A0A2054E7D7160C6D1B6B2EE2@MW5PR22MB2988.namprd22.prod.outlook.com |
|---|---|
| State | Rejected |
| Delegated to: | Ilya Maximets |
| Headers | show |
| Series | [ovs-dev] netdev: Padding runt packet on VXLAN and DPDK ports. | expand |
| Context | Check | Description |
|---|---|---|
| ovsrobot/apply-robot | warning | apply and check: warning |
| ovsrobot/github-robot-_Build_and_Test | fail | github build: failed |
| ovsrobot/intel-ovs-compilation | fail | test: fail |
From: robot@bytheb.org
Test-Label: github-robot: Build and Test
Test-Status: fail
http://patchwork.ozlabs.org/api/patches/1936643/
_github build: failed_
Build URL: https://github.com/ovsrobot/ovs/actions/runs/9135321447
Build Logs:
-----------------------Summary of failed steps-----------------------
"osx clang --disable-ssl" failed at step build
----------------------End summary of failed steps--------------------
-------------------------------BEGIN LOGS----------------------------
####################################################################################
#### [Begin job log] "osx clang --disable-ssl" at step build
####################################################################################
####################################################################################
#### [End job log] "osx clang --disable-ssl" at step build
####################################################################################
--------------------------------END LOGS-----------------------------
On 5/18/24 00:55, Kumar, Rohit via dev wrote: > The basic idea behind this patch is to pad the runt packet before vxlan > encapsulation and on a DPDK port. > > topology: > IXIA <----> SWITCH <--vxlan--> OVS <--dpdk--> HOST > > The host sends a runt packet (54 bytes) with a size less than 64 bytes > to the OVS. The OVS receives this runt packet and sends it further down > the VXLAN tunnel (original packet 54 + 50 VXLAN = 104 B total packet size) > to a physical switch. At the switch, after decapsulation, the original > packet size is 54B and it then sends the packet on to the ixia. However, > the switch adds 2B of padding with random content (not zero as RFC 1042 > mentions) and due to this random byte padding, ixia drops the packet. > > Switch vendors claim to be compliant with both RFC 1042 and RFC 2119 to > fix it in the switch. > > RFC 1042 defines: > “IEEE 802 packets may have a minimum size restriction. When > necessary, the data field should be padded (with octets of zero) > to meet the IEEE 802 minimum frame size requirements. This > padding is not part of the IP datagram and is not included in the > total length field of the IP header.” > > RFC 2119 defines: > "SHOULD This word, or the adjective "RECOMMENDED", mean that there > may exist valid reasons in particular circumstances to ignore a > particular item, but the full implications must be understood and > carefully weighed before choosing a different course." > > So, a fix is needed in the OVS. The OVS is connected to the switch on a > VXLAN port and to the host on a DPDK port. The padding fix is applied to > both netdev types. I tested the fix in the same setup and below are the > captures after padding on both VXLAN and DPDK ports from OVS UT. > > VXLAN (46B zero pad): > 0000 aa 55 aa 66 00 00 aa 55 aa 55 00 00 08 00 45 00 .U.f...U.U....E. > 0010 00 60 00 00 40 00 40 11 26 81 0a 00 00 02 0a 00 .`..@.@.&....... > 0020 00 0b 9b 77 12 b5 00 4c 00 00 08 00 00 00 00 00 ...w...L........ > 0030 7b 00 50 54 00 00 00 0a 50 54 00 00 00 09 12 34 {.PT....PT.....4 > 0040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 0060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .............. > > DPDK (16B zero pad): > 0000 ff ff ff ff ff ff 5e fb 90 96 7d b7 08 06 00 01 ......^...}..... > 0010 08 00 06 04 00 01 5e fb 90 96 7d b7 0a 01 01 01 ......^...}..... > 0020 00 00 00 00 00 00 0a 01 01 02 00 00 00 00 00 00 ................ > 0030 00 00 00 00 00 00 00 00 00 00 00 00 ............ > > Signed-off-by: Rohit Kumar <rokumar@idirect.net> Hi, Rohit. Thanks for the patch! Though I don't think OVS should add padding to packets. RFC 1042 is talking about transmission to ethernet networks. In your example above OVS sends 104 byte long packet to the ethernet link, so no padding is needed. If the 'SWITCH' in your setup sends incorectly padded packet after decapsulation, that sounds like an issue of that switch. Is this patch a workaround for a hardware switch issue? Also, we even had a request in the past to strip all the existing padding before encapsulation, because older Linux kernel versions had issues processing padded inner packets after decapsulation. Adding the padding will also introduce a difference in behavior between Linux kenrel datapath and the userspace datapath, since Linux kernel doesn't pad packet before encapsulation. Having a difference between datapath implementations is not good. For the part where we add padding before sending to a DPDK interface, this also doesn't seem right. Normally it's a job of a driver to properly pad packets before transmitting them. OVS doesn't know what the requirements for the particular network it is connected to or even if it is an ethernet or some other type of the interface. So, it can't make a decision on padding, only driver can do that. Best regards, Ilya Maximets.
Hi Ilya, Thanks for your comments. Yes, it's the hardware switch that doesn't do zero padding. Unfortunately, the switch vendor claims that the padding is added after the FCS of the original packet, and there is no way around it. And if it's a requirement to zero pad, then RFC 1042 should use the word "must". As for the "valid reason in special circumstances", the reason is that generating zero padding would have greatly complicated their switch architecture, resulting in additional power consumption and latency. So OVS has to do it. For the other part, where zero padding is to be added to the DPDK interface, no PMD except a few (Atomic Rules, Broadcom, HiSilicon PMDs) do this in the driver, and we use Intel, so that's the reason for doing it in OVS. Regards, Rohit Kumar
On 5/21/24 13:29, Kumar, Rohit wrote: > Hi Ilya, > > Thanks for your comments. > > Yes, it's the hardware switch that doesn't do zero padding. Unfortunately, > the switch vendor claims that the padding is added after the FCS of the > original packet, and there is no way around it. And if it's a requirement > to zero pad, then RFC 1042 should use the word "must". As for the "valid > reason in special circumstances", the reason is that generating zero padding > would have greatly complicated their switch architecture, resulting in > additional power consumption and latency. So OVS has to do it. This is not a good argument for OVS to do that. We can't workaround every possible thing hardware vendors implement differently. Adding extra padding into tunnel packets beside performance impact will break communication with older Linux kernels, so we can't do that. Also, as I already said, this will introduce the difference between Userpsace and Kernel datapaths, i.e your setup will still not work if you use Linux kernel tunnels for encapsulation. Same is likely true for other operating systems. > > For the other part, where zero padding is to be added to the DPDK interface, > no PMD except a few (Atomic Rules, Broadcom, HiSilicon PMDs) do this in the > driver, and we use Intel, so that's the reason for doing it in OVS. Most modern NICs support padding on the hardware level. Intel supports it by enabling IXGBE_HLREG0_TXPADEN for ixgbe cards for example. So, even if the padding is not performed in software it doesn't mean it's not performed. Doing this in software will be a waste of CPU resources in most cases. Best regards, Ilya Maximets. > > Regards, > Rohit Kumar > ---------------------------------------------------------------------------- > *From:* Ilya Maximets <i.maximets@ovn.org> > *Sent:* Monday, May 20, 2024 11:30 PM > *To:* Kumar, Rohit <rokumar@idirect.net>; dev@openvswitch.org <dev@openvswitch.org> > *Cc:* i.maximets@ovn.org <i.maximets@ovn.org> > *Subject:* Re: [ovs-dev] [PATCH] netdev: Padding runt packet on VXLAN and DPDK ports. > > ***WARNING! THIS EMAIL ORIGINATES FROM OUTSIDE ST ENGINEERING IDIRECT.*** > > On 5/18/24 00:55, Kumar, Rohit via dev wrote: >> The basic idea behind this patch is to pad the runt packet before vxlan >> encapsulation and on a DPDK port. >> >> topology: >> IXIA <----> SWITCH <--vxlan--> OVS <--dpdk--> HOST >> >> The host sends a runt packet (54 bytes) with a size less than 64 bytes >> to the OVS. The OVS receives this runt packet and sends it further down >> the VXLAN tunnel (original packet 54 + 50 VXLAN = 104 B total packet size) >> to a physical switch. At the switch, after decapsulation, the original >> packet size is 54B and it then sends the packet on to the ixia. However, >> the switch adds 2B of padding with random content (not zero as RFC 1042 >> mentions) and due to this random byte padding, ixia drops the packet. >> >> Switch vendors claim to be compliant with both RFC 1042 and RFC 2119 to >> fix it in the switch. >> >> RFC 1042 defines: >> “IEEE 802 packets may have a minimum size restriction. When >> necessary, the data field should be padded (with octets of zero) >> to meet the IEEE 802 minimum frame size requirements. This >> padding is not part of the IP datagram and is not included in the >> total length field of the IP header.” >> >> RFC 2119 defines: >> "SHOULD This word, or the adjective "RECOMMENDED", mean that there >> may exist valid reasons in particular circumstances to ignore a >> particular item, but the full implications must be understood and >> carefully weighed before choosing a different course." >> >> So, a fix is needed in the OVS. The OVS is connected to the switch on a >> VXLAN port and to the host on a DPDK port. The padding fix is applied to >> both netdev types. I tested the fix in the same setup and below are the >> captures after padding on both VXLAN and DPDK ports from OVS UT. >> >> VXLAN (46B zero pad): >> 0000 aa 55 aa 66 00 00 aa 55 aa 55 00 00 08 00 45 00 .U.f...U.U....E. >> 0010 00 60 00 00 40 00 40 11 26 81 0a 00 00 02 0a 00 .`..@.@.&....... >> 0020 00 0b 9b 77 12 b5 00 4c 00 00 08 00 00 00 00 00 ...w...L........ >> 0030 7b 00 50 54 00 00 00 0a 50 54 00 00 00 09 12 34 {.PT....PT.....4 >> 0040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ >> 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ >> 0060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .............. >> >> DPDK (16B zero pad): >> 0000 ff ff ff ff ff ff 5e fb 90 96 7d b7 08 06 00 01 ......^...}..... >> 0010 08 00 06 04 00 01 5e fb 90 96 7d b7 0a 01 01 01 ......^...}..... >> 0020 00 00 00 00 00 00 0a 01 01 02 00 00 00 00 00 00 ................ >> 0030 00 00 00 00 00 00 00 00 00 00 00 00 ............ >> >> Signed-off-by: Rohit Kumar <rokumar@idirect.net> > > Hi, Rohit. Thanks for the patch! > > Though I don't think OVS should add padding to packets. RFC 1042 is talking > about transmission to ethernet networks. In your example above OVS sends 104 > byte long packet to the ethernet link, so no padding is needed. If the > 'SWITCH' in your setup sends incorectly padded packet after decapsulation, > that sounds like an issue of that switch. Is this patch a workaround for > a hardware switch issue? > > Also, we even had a request in the past to strip all the existing padding before > encapsulation, because older Linux kernel versions had issues processing padded > inner packets after decapsulation. > > Adding the padding will also introduce a difference in behavior between Linux > kenrel datapath and the userspace datapath, since Linux kernel doesn't pad packet > before encapsulation. Having a difference between datapath implementations is > not good. > > For the part where we add padding before sending to a DPDK interface, this > also doesn't seem right. Normally it's a job of a driver to properly pad > packets before transmitting them. OVS doesn't know what the requirements > for the particular network it is connected to or even if it is an ethernet > or some other type of the interface. So, it can't make a decision on padding, > only driver can do that. > > Best regards, Ilya Maximets.
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 7b84c858e..441c7b2ea 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -26,6 +26,7 @@ #include <sys/socket.h> #include <sys/stat.h> #include <linux/if.h> +#include <linux/if_ether.h> #include <rte_bus.h> #include <rte_config.h> @@ -2755,6 +2756,8 @@ netdev_dpdk_eth_tx_burst(struct netdev_dpdk *dev, int qid, { uint32_t nb_tx = 0; uint16_t nb_tx_prep = cnt; + uint8_t pad = 0; + char *data = NULL; nb_tx_prep = rte_eth_tx_prepare(dev->port_id, qid, pkts, cnt); if (nb_tx_prep != cnt) { @@ -2765,6 +2768,24 @@ netdev_dpdk_eth_tx_burst(struct netdev_dpdk *dev, int qid, "First invalid packet", pkts[nb_tx_prep]); } + while (nb_tx != nb_tx_prep) { + /* adjust the packet to minimum ethernet frame len of 60 bytes */ + if (OVS_UNLIKELY (rte_pktmbuf_pkt_len (pkts[nb_tx]) < ETH_ZLEN)) { + pad = ETH_ZLEN - rte_pktmbuf_pkt_len (pkts[nb_tx]); + data = rte_pktmbuf_append (pkts[nb_tx], pad); + if (OVS_UNLIKELY (data == NULL)) { + VLOG_WARN_RL(&rl, "Mbuf appending failed to pad %d bytes", pad); + /* This packet has error, can't send it, so just free it */ + rte_pktmbuf_free(pkts[nb_tx]); + nb_tx_prep -= 1; + continue; + } + memset (data, 0, pad); + } + nb_tx += 1; + } + + nb_tx = 0; while (nb_tx != nb_tx_prep) { uint32_t ret; diff --git a/lib/netdev.c b/lib/netdev.c index f2d921ed6..97d3711fb 100644 --- a/lib/netdev.c +++ b/lib/netdev.c @@ -24,6 +24,7 @@ #include <stdlib.h> #include <string.h> #include <unistd.h> +#include <linux/if_ether.h> #ifndef _WIN32 #include <ifaddrs.h> @@ -1007,8 +1008,16 @@ netdev_push_header(const struct netdev *netdev, { struct dp_packet *packet; size_t i, size = dp_packet_batch_size(batch); + uint8_t pad = 0; DP_PACKET_BATCH_REFILL_FOR_EACH (i, size, packet, batch) { + /* adjust the packet to minimum ethernet frame len of 60 bytes */ + if (OVS_UNLIKELY (data->tnl_type == OVS_VPORT_TYPE_VXLAN + && dp_packet_is_eth (packet) + && dp_packet_size (packet) < ETH_ZLEN)) { + pad = ETH_ZLEN - dp_packet_size (packet); + dp_packet_put_zeros (packet, pad); + } if (OVS_UNLIKELY(data->tnl_type != OVS_VPORT_TYPE_GENEVE && data->tnl_type != OVS_VPORT_TYPE_VXLAN && dp_packet_hwol_is_tso(packet))) { diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at index 1c97bf777..09533f035 100644 --- a/tests/system-dpdk.at +++ b/tests/system-dpdk.at @@ -863,3 +863,60 @@ AT_CHECK([ovs-vsctl del-port br10 p1], [], [stdout], [stderr]) OVS_DPDK_STOP_VSWITCHD AT_CLEANUP dnl -------------------------------------------------------------------------- + +dnl -------------------------------------------------------------------------- +dnl Add, ping and capture on a standard DPDK PHY port to check the padding. +AT_SETUP([OVS-DPDK - DPDK port padding]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_PHY_SKIP() +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev]) + +dnl create empty pcap files +AT_CHECK([tcpdump -n -w /tmp/rx.pcap -c 1], [], [stdout], [stderr]) +AT_CHECK([tcpdump -n -w /tmp/tx.pcap -c 1], [], [stdout], [stderr]) + +dnl Add libpcap-based DPDK phy port +AT_CHECK([ovs-vsctl add-port br0 p8 -- set Interface p8 type=dpdk ofport_request=8 options:dpdk-devargs=$(cat DPDK_PCI_ADDR) options:dpdk-devargs="eth_pcap0,rx_pcap=/tmp/rx.pcap,tx_pcap=/tmp/tx.pcap"], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl show], [], [stdout]) + +dnl Set up namespace +AT_CHECK([ip netns add at_ns0], []) +on_exit 'ip netns del at_ns0' + +dnl Create a0 and afdp-a0 peer ports +AT_CHECK([ip link add a0 type veth peer name afdp-a0], []) +AT_CHECK([ip link set dev afdp-a0 up], []) + +dnl Add a0 to namespace +AT_CHECK([ip link set a0 netns at_ns0], []) + +dnl Add afdp_a0 to namespace +AT_CHECK([ovs-vsctl add-port br0 afdp-a0], []) +AT_CHECK([ovs-vsctl show], [], [stdout]) + +AT_CHECK([ovs-ofctl add-flow br0 'table=0,in_port=1 action=output:8'], []) +AT_CHECK([ovs-ofctl dump-flows br0], [], [stdout]) + +AT_CHECK([ip netns exec at_ns0 ip addr add 10.1.1.1/24 dev a0], []) +AT_CHECK([ip netns exec at_ns0 ip link set a0 up], []) +dnl ping unknown IP for ARP request of 42 bytes. +AT_CHECK([ip netns exec at_ns0 ping 10.1.1.2 -c 1 2>&1 || true], [], [stdout], [stderr]) + +dnl Output and capture on DPDK phy port from a afdp_a0 peer port. +dnl Checking that the packet transmitted from DPDK phy port is correctly padded. +OVS_WAIT_UNTIL([test `ovs-pcap /tmp/tx.pcap | grep -E "000000000000000000000000000000000000" | wc -l` -ge 3]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br0 p8], []) +AT_CHECK([ovs-vsctl del-port br0 afdp-a0], []) +AT_CHECK([ovs-vsctl del-br br0], []) +AT_CHECK([ip netns del at_ns0], []) +AT_CHECK([rm -f /tmp/rx.pcap], []) +AT_CHECK([rm -f /tmp/tx.pcap], []) +dnl OVS_DPDK_STOP_VSWITCHD +AT_CLEANUP +dnl -------------------------------------------------------------------------- diff --git a/tests/tunnel-push-pop.at b/tests/tunnel-push-pop.at index 508737c53..4b6619f38 100644 --- a/tests/tunnel-push-pop.at +++ b/tests/tunnel-push-pop.at @@ -1252,3 +1252,54 @@ hash(l4(0)),recirc(0x2) OVS_VSWITCHD_STOP AT_CLEANUP + +AT_SETUP([tunnel_push_pop - VXLAN port padding]) + +dnl Create bridge that has a MAC address. +OVS_VSWITCHD_START([set bridge br0 datapath_type=dummy dnl + -- set Interface br0 other-config:hwaddr=aa:55:aa:55:00:00]) +AT_CHECK([ovs-appctl vlog/set dpif_netdev:dbg]) +AT_CHECK([ovs-vsctl add-port br0 p8 dnl + -- set Interface p8 type=dummy ofport_request=8]) + +dnl Create another bridge. +AT_CHECK([ovs-vsctl add-br ovs-tun0 -- set bridge ovs-tun0 datapath_type=dummy]) + +dnl Add VXLAN port to this bridge. +AT_CHECK([ovs-vsctl add-port ovs-tun0 tun0 dnl + -- set int tun0 type=vxlan \ + options:remote_ip=10.0.0.11 options:key=123 ofport_request=2\ + -- add-port ovs-tun0 p7 dnl + -- set interface p7 type=dummy ofport_request=7]) + +dnl Set IP address and route for br0. +AT_CHECK([ovs-appctl netdev-dummy/ip4addr br0 10.0.0.2/24], [0], [OK +]) +AT_CHECK([ovs-appctl ovs/route/add 10.0.0.11/24 br0], [0], [OK +]) + +dnl Send an ARP reply to port p8 on br0, so that packets will be forwarded +dnl to learned port. +AT_CHECK([ovs-ofctl add-flow br0 action=normal]) + +AT_CHECK([ovs-appctl netdev-dummy/receive p8 'in_port(8),dnl + eth(src=aa:55:aa:66:00:00,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),dnl + arp(sip=10.0.0.11,tip=10.0.0.2,op=2,sha=aa:55:aa:66:00:00,tha=00:00:00:00:00:00)']) + +AT_CHECK([ovs-vsctl -- set Interface p8 options:tx_pcap=p8.pcap]) +packet=50540000000a505400000009123 +dnl Source port is based on a packet hash, so it may differ depending on the +dnl compiler flags and CPU type. Masked with '....'. +encap=aa55aa660000aa55aa55000008004500003200004000401134060a0000020a00000b....12bf001e0000080000000004c400 + +dnl Output to tunnel from a ovs-tun0 internal port. +dnl Checking that the packet arrived and it was correctly encapsulated. +AT_CHECK([ovs-ofctl add-flow ovs-tun0 "in_port=LOCAL,actions=output:2"]) +AT_CHECK([ovs-appctl netdev-dummy/receive ovs-tun0 "${packet}4"]) +OVS_WAIT_UNTIL([test `ovs-pcap p8.pcap | grep -E "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000" | wc -l` -ge 1]) +Sending again to exercise the non-miss upcall path. +AT_CHECK([ovs-appctl netdev-dummy/receive ovs-tun0 "${packet}4"]) +OVS_WAIT_UNTIL([test `ovs-pcap p8.pcap | grep -E "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000" | wc -l` -ge 2]) + +OVS_VSWITCHD_STOP +AT_CLEANUP
The basic idea behind this patch is to pad the runt packet before vxlan encapsulation and on a DPDK port. topology: IXIA <----> SWITCH <--vxlan--> OVS <--dpdk--> HOST The host sends a runt packet (54 bytes) with a size less than 64 bytes to the OVS. The OVS receives this runt packet and sends it further down the VXLAN tunnel (original packet 54 + 50 VXLAN = 104 B total packet size) to a physical switch. At the switch, after decapsulation, the original packet size is 54B and it then sends the packet on to the ixia. However, the switch adds 2B of padding with random content (not zero as RFC 1042 mentions) and due to this random byte padding, ixia drops the packet. Switch vendors claim to be compliant with both RFC 1042 and RFC 2119 to fix it in the switch. RFC 1042 defines: “IEEE 802 packets may have a minimum size restriction. When necessary, the data field should be padded (with octets of zero) to meet the IEEE 802 minimum frame size requirements. This padding is not part of the IP datagram and is not included in the total length field of the IP header.” RFC 2119 defines: "SHOULD This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course." So, a fix is needed in the OVS. The OVS is connected to the switch on a VXLAN port and to the host on a DPDK port. The padding fix is applied to both netdev types. I tested the fix in the same setup and below are the captures after padding on both VXLAN and DPDK ports from OVS UT. VXLAN (46B zero pad): 0000 aa 55 aa 66 00 00 aa 55 aa 55 00 00 08 00 45 00 .U.f...U.U....E. 0010 00 60 00 00 40 00 40 11 26 81 0a 00 00 02 0a 00 .`..@.@.&....... 0020 00 0b 9b 77 12 b5 00 4c 00 00 08 00 00 00 00 00 ...w...L........ 0030 7b 00 50 54 00 00 00 0a 50 54 00 00 00 09 12 34 {.PT....PT.....4 0040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .............. DPDK (16B zero pad): 0000 ff ff ff ff ff ff 5e fb 90 96 7d b7 08 06 00 01 ......^...}..... 0010 08 00 06 04 00 01 5e fb 90 96 7d b7 0a 01 01 01 ......^...}..... 0020 00 00 00 00 00 00 0a 01 01 02 00 00 00 00 00 00 ................ 0030 00 00 00 00 00 00 00 00 00 00 00 00 ............ Signed-off-by: Rohit Kumar <rokumar@idirect.net> This electronic message and any files transmitted with it contains information from ST Engineering iDirect, which may be privileged, proprietary and/or confidential. It is intended solely for the use of the individual or entity to whom they are addressed. If you are not the original recipient or the person responsible for delivering the email to the intended recipient, be advised that you have received this email in error, and that any use, dissemination, forwarding, printing, or copying of this email is strictly prohibited. If you received this email in error, please delete it and immediately notify the sender.