| Message ID | 1489516202-29401-1-git-send-email-kriskend@linux.vnet.ibm.com |
|---|---|
| State | Deferred |
| Headers | show |
Recreate and error info: Hit a new issue with OVS after updating to the Ubuntu 4.8 kernel from the Ubuntu 4.4 kernel. Iperf was used to send traffic between client VMs over VXLAN. The traffic did still flow, but every packet had to go to user space due to the flow creation failures, which drastically impacted performance and cpu utilization. When using VXLAN, the following error is showing up in dmesg openvswitch: netlink: Unknown IP tunnel attribute 14 Also there are tons of these errors in the openvswitch log 2017-03-01T15:50:47.860Z|00018|dpif(handler164)|WARN|system@ovs-system: failed to put[create] (Invalid argument) ufid:2d1a9aeb-7b24-4235-a208-a01f98237e60 recirc_id(0),dp_hash(0/0),skb_pri Debug showed that this attribute, OVS_TUNNEL_KEY_ATTR_PAD, was being seen in the switch statement in method, static int ip_tun_from_nlattr, in flow_netlink.c . Because there is no case for this attribute, the default is hit and returns an error. The issue was first seen using the packages in the Ubuntu 4.8 kernel, which is OVS 2.5. OVS 2.6 and 2.6.1 were also tried with the kernel packages and the same issue was seen. Tried building OVS 2.7 and loading the openvswitch-datapath-dkms_2.7.0-1_all.deb that got built but the issue persisted. The proposed patch seems to eliminate the error messages and also fixed the segmentation and performance issues that were seen. On 3/14/2017 1:30 PM, Kris Murphy wrote: > Added a case for OVS_TUNNEL_KEY_ATTR_PAD to the switch statement > in ip_tun_from_nlattr in order to prevent the default case > returning an error. > Signed-off-by: Kris Murphy <kriskend@linux.vnet.ibm.com> > --- > datapath/flow_netlink.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/datapath/flow_netlink.c b/datapath/flow_netlink.c > index 0e38dd2..1e08ed3 100644 > --- a/datapath/flow_netlink.c > +++ b/datapath/flow_netlink.c > @@ -651,6 +651,8 @@ static int ip_tun_from_nlattr(const struct nlattr *attr, > tun_flags |= TUNNEL_VXLAN_OPT; > opts_type = type; > break; > + case OVS_TUNNEL_KEY_ATTR_PAD: > + break; > default: > OVS_NLERR(log, "Unknown IP tunnel attribute %d", > type);
On 14 March 2017 at 11:30, Kris Murphy <kriskend@linux.vnet.ibm.com> wrote: > Recreate and error info: > > Hit a new issue with OVS after updating to the Ubuntu 4.8 kernel from the > Ubuntu 4.4 kernel. > > Iperf was used to send traffic between client VMs over VXLAN. The traffic > did still flow, but every packet had to go to user space due to the flow > creation failures, which drastically impacted performance and cpu > utilization. > > When using VXLAN, the following error is showing up in dmesg > > openvswitch: netlink: Unknown IP tunnel attribute 14 > > Also there are tons of these errors in the openvswitch log > > 2017-03-01T15:50:47.860Z|00018|dpif(handler164)|WARN|system@ovs-system: > failed to put[create] (Invalid argument) > ufid:2d1a9aeb-7b24-4235-a208-a01f98237e60 recirc_id(0),dp_hash(0/0),skb_pri > > Debug showed that this attribute, OVS_TUNNEL_KEY_ATTR_PAD, was being seen in > the switch statement in method, static int ip_tun_from_nlattr, in > flow_netlink.c . Because there is no case for this attribute, the default > is hit and returns an error. > > The issue was first seen using the packages in the Ubuntu 4.8 kernel, which > is OVS 2.5. OVS 2.6 and 2.6.1 were also tried with the kernel packages and > the same issue was seen. Tried building OVS 2.7 and loading the > openvswitch-datapath-dkms_2.7.0-1_all.deb that got built but the issue > persisted. The proposed patch seems to eliminate the error messages and > also fixed the segmentation and performance issues that were seen. Thanks for the report! Would you mind preparing this against the linux "net" tree and sending to netdev mailinglist, since that's the upstream for Linux datapath changes? You can use this tag in the commit message, as well: Fixes: b46f6ded906e ("libnl: nla_put_be64(): align on a 64-bit area")
diff --git a/datapath/flow_netlink.c b/datapath/flow_netlink.c index 0e38dd2..1e08ed3 100644 --- a/datapath/flow_netlink.c +++ b/datapath/flow_netlink.c @@ -651,6 +651,8 @@ static int ip_tun_from_nlattr(const struct nlattr *attr, tun_flags |= TUNNEL_VXLAN_OPT; opts_type = type; break; + case OVS_TUNNEL_KEY_ATTR_PAD: + break; default: OVS_NLERR(log, "Unknown IP tunnel attribute %d", type);
Added a case for OVS_TUNNEL_KEY_ATTR_PAD to the switch statement in ip_tun_from_nlattr in order to prevent the default case returning an error. Signed-off-by: Kris Murphy <kriskend@linux.vnet.ibm.com> --- datapath/flow_netlink.c | 2 ++ 1 file changed, 2 insertions(+)