| Message ID | 1554159893-29704-1-git-send-email-si-wei.liu@oracle.com |
|---|---|
| State | Superseded |
| Delegated to: | David Miller |
| Headers | show |
| Series | [net,v5] failover: allow name change on IFF_UP slave interfaces | expand |
On 4/1/2019 4:04 PM, Si-Wei Liu wrote: > When a netdev appears through hot plug then gets enslaved by a failover > master that is already up and running, the slave will be opened > right away after getting enslaved. Today there's a race that userspace > (udev) may fail to rename the slave if the kernel (net_failover) > opens the slave earlier than when the userspace rename happens. > Unlike bond or team, the primary slave of failover can't be renamed by > userspace ahead of time, since the kernel initiated auto-enslavement is > unable to, or rather, is never meant to be synchronized with the rename > request from userspace. > > As the failover slave interfaces are not designed to be operated > directly by userspace apps: IP configuration, filter rules with > regard to network traffic passing and etc., should all be done on master > interface. In general, userspace apps only care about the > name of master interface, while slave names are less important as long > as admin users can see reliable names that may carry > other information describing the netdev. For e.g., they can infer that > "ens3nsby" is a standby slave of "ens3", while for a > name like "eth0" they can't tell which master it belongs to. > > Historically the name of IFF_UP interface can't be changed because > there might be admin script or management software that is already > relying on such behavior and assumes that the slave name can't be > changed once UP. But failover is special: with the in-kernel > auto-enslavement mechanism, the userspace expectation for device > enumeration and bring-up order is already broken. Previously initramfs > and various userspace config tools were modified to bypass failover > slaves because of auto-enslavement and duplicate MAC address. Similarly, > in case that users care about seeing reliable slave name, the new type > of failover slaves needs to be taken care of specifically in userspace > anyway. > > It's less risky to lift up the rename restriction on failover slave > which is already UP. Although it's possible this change may potentially > break userspace component (most likely configuration scripts or > management software) that assumes slave name can't be changed while > UP, it's relatively a limited and controllable set among all userspace > components, which can be fixed specifically to listen for the rename > and/or link down/up events on failover slaves. Userspace component > interacting with slaves is expected to be changed to operate on failover > master interface instead, as the failover slave is dynamic in nature > which may come and go at any point. The goal is to make the role of > failover slaves less relevant, and userspace components should only > deal with failover master in the long run. > > Fixes: 30c8bd5aa8b2 ("net: Introduce generic failover module") > Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com> > Reviewed-by: Liran Alon <liran.alon@oracle.com> Acked-by: Sridhar Samudrala <sridhar.samudrala@intel.com> > > -- > v1 -> v2: > - Drop configurable module parameter (Sridhar) > > v2 -> v3: > - Drop additional IFF_SLAVE_RENAME_OK flag (Sridhar) > - Send down and up events around rename (Michael S. Tsirkin) > > v3 -> v4: > - Simplify notification to be sent (Stephen Hemminger) > > v4 -> v5: > - Sync up code with latest net-next (Sridhar) > - Use proper structure initialization (Stephen, Jiri) > --- > net/core/dev.c | 25 ++++++++++++++++++++++++- > 1 file changed, 24 insertions(+), 1 deletion(-) > > diff --git a/net/core/dev.c b/net/core/dev.c > index 9823b77..b694184 100644 > --- a/net/core/dev.c > +++ b/net/core/dev.c > @@ -1185,7 +1185,21 @@ int dev_change_name(struct net_device *dev, const char *newname) > BUG_ON(!dev_net(dev)); > > net = dev_net(dev); > - if (dev->flags & IFF_UP) > + > + /* Allow failover slave to rename even when > + * it is up and running. > + * > + * Failover slaves are special, since userspace > + * might rename the slave after the interface > + * has been brought up and running due to > + * auto-enslavement. > + * > + * Failover users don't actually care about slave > + * name change, as they are only expected to operate > + * on master interface directly. > + */ > + if (dev->flags & IFF_UP && > + likely(!(dev->priv_flags & IFF_FAILOVER_SLAVE))) > return -EBUSY; > > write_seqcount_begin(&devnet_rename_seq); > @@ -1232,6 +1246,15 @@ int dev_change_name(struct net_device *dev, const char *newname) > hlist_add_head_rcu(&dev->name_hlist, dev_name_hash(net, dev->name)); > write_unlock_bh(&dev_base_lock); > > + if (unlikely(dev->flags & IFF_UP)) { > + struct netdev_notifier_change_info change_info = { > + .info.dev = dev, > + }; > + > + call_netdevice_notifiers_info(NETDEV_CHANGE, > + &change_info.info); > + } > + > ret = call_netdevice_notifiers(NETDEV_CHANGENAME, dev); > ret = notifier_to_errno(ret); > >
On Mon, Apr 01, 2019 at 07:04:53PM -0400, Si-Wei Liu wrote: > When a netdev appears through hot plug then gets enslaved by a failover > master that is already up and running, the slave will be opened > right away after getting enslaved. Today there's a race that userspace > (udev) may fail to rename the slave if the kernel (net_failover) > opens the slave earlier than when the userspace rename happens. > Unlike bond or team, the primary slave of failover can't be renamed by > userspace ahead of time, since the kernel initiated auto-enslavement is > unable to, or rather, is never meant to be synchronized with the rename > request from userspace. > > As the failover slave interfaces are not designed to be operated > directly by userspace apps: IP configuration, filter rules with > regard to network traffic passing and etc., should all be done on master > interface. In general, userspace apps only care about the > name of master interface, while slave names are less important as long > as admin users can see reliable names that may carry > other information describing the netdev. For e.g., they can infer that > "ens3nsby" is a standby slave of "ens3", while for a > name like "eth0" they can't tell which master it belongs to. > > Historically the name of IFF_UP interface can't be changed because > there might be admin script or management software that is already > relying on such behavior and assumes that the slave name can't be > changed once UP. But failover is special: with the in-kernel > auto-enslavement mechanism, the userspace expectation for device > enumeration and bring-up order is already broken. Previously initramfs > and various userspace config tools were modified to bypass failover > slaves because of auto-enslavement and duplicate MAC address. Similarly, > in case that users care about seeing reliable slave name, the new type > of failover slaves needs to be taken care of specifically in userspace > anyway. > > It's less risky to lift up the rename restriction on failover slave > which is already UP. Although it's possible this change may potentially > break userspace component (most likely configuration scripts or > management software) that assumes slave name can't be changed while > UP, it's relatively a limited and controllable set among all userspace > components, which can be fixed specifically to listen for the rename > and/or link down/up events on failover slaves. Userspace component > interacting with slaves is expected to be changed to operate on failover > master interface instead, as the failover slave is dynamic in nature > which may come and go at any point. The goal is to make the role of > failover slaves less relevant, and userspace components should only > deal with failover master in the long run. > > Fixes: 30c8bd5aa8b2 ("net: Introduce generic failover module") > Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com> > Reviewed-by: Liran Alon <liran.alon@oracle.com> > > -- > v1 -> v2: > - Drop configurable module parameter (Sridhar) > > v2 -> v3: > - Drop additional IFF_SLAVE_RENAME_OK flag (Sridhar) > - Send down and up events around rename (Michael S. Tsirkin) > > v3 -> v4: > - Simplify notification to be sent (Stephen Hemminger) > > v4 -> v5: > - Sync up code with latest net-next (Sridhar) > - Use proper structure initialization (Stephen, Jiri) > --- Acked-by: Michael S. Tsirkin <mst@redhat.com> > net/core/dev.c | 25 ++++++++++++++++++++++++- > 1 file changed, 24 insertions(+), 1 deletion(-) > > diff --git a/net/core/dev.c b/net/core/dev.c > index 9823b77..b694184 100644 > --- a/net/core/dev.c > +++ b/net/core/dev.c > @@ -1185,7 +1185,21 @@ int dev_change_name(struct net_device *dev, const char *newname) > BUG_ON(!dev_net(dev)); > > net = dev_net(dev); > - if (dev->flags & IFF_UP) > + > + /* Allow failover slave to rename even when > + * it is up and running. > + * > + * Failover slaves are special, since userspace > + * might rename the slave after the interface > + * has been brought up and running due to > + * auto-enslavement. > + * > + * Failover users don't actually care about slave > + * name change, as they are only expected to operate > + * on master interface directly. > + */ > + if (dev->flags & IFF_UP && > + likely(!(dev->priv_flags & IFF_FAILOVER_SLAVE))) > return -EBUSY; > > write_seqcount_begin(&devnet_rename_seq); > @@ -1232,6 +1246,15 @@ int dev_change_name(struct net_device *dev, const char *newname) > hlist_add_head_rcu(&dev->name_hlist, dev_name_hash(net, dev->name)); > write_unlock_bh(&dev_base_lock); > > + if (unlikely(dev->flags & IFF_UP)) { > + struct netdev_notifier_change_info change_info = { > + .info.dev = dev, > + }; > + > + call_netdevice_notifiers_info(NETDEV_CHANGE, > + &change_info.info); > + } > + > ret = call_netdevice_notifiers(NETDEV_CHANGENAME, dev); > ret = notifier_to_errno(ret); > > -- > 1.8.3.1
On Mon, 1 Apr 2019 19:04:53 -0400 Si-Wei Liu <si-wei.liu@oracle.com> wrote: > + if (dev->flags & IFF_UP && > + likely(!(dev->priv_flags & IFF_FAILOVER_SLAVE))) Why is property limited to failover slave, it would make sense for netvsc as well. Why not make it a flag like live address change?
On 4/2/2019 2:53 PM, Stephen Hemminger wrote: > On Mon, 1 Apr 2019 19:04:53 -0400 > Si-Wei Liu <si-wei.liu@oracle.com> wrote: > >> + if (dev->flags & IFF_UP && >> + likely(!(dev->priv_flags & IFF_FAILOVER_SLAVE))) > Why is property limited to failover slave, it would make sense for netvsc > as well. Why not make it a flag like live address change? Well, netvsc today is still taking the delayed approach meaning that it is incompatible yet with this live name change flag if need be. ;-) I thought Sridhar did not like to introduce an additional IFF_SLAVE_RENAME_OK flag given that failover slave is the only consumer for the time being. Even though I can get it back, patch is needed for netvsc to remove the VF takeover delay IMHO. Sridhar, what do you think we revive the IFF_SLAVE_RENAME_OK flag which allows netvsc to be used later on? Or maybe, IFF_LIVE_RENAME_OK for a better name? -Siwei
On Tue, 2 Apr 2019 15:23:29 -0700 si-wei liu <si-wei.liu@oracle.com> wrote: > On 4/2/2019 2:53 PM, Stephen Hemminger wrote: > > On Mon, 1 Apr 2019 19:04:53 -0400 > > Si-Wei Liu <si-wei.liu@oracle.com> wrote: > > > >> + if (dev->flags & IFF_UP && > >> + likely(!(dev->priv_flags & IFF_FAILOVER_SLAVE))) > > Why is property limited to failover slave, it would make sense for netvsc > > as well. Why not make it a flag like live address change? > Well, netvsc today is still taking the delayed approach meaning that it > is incompatible yet with this live name change flag if need be. ;-) > > I thought Sridhar did not like to introduce an additional > IFF_SLAVE_RENAME_OK flag given that failover slave is the only consumer > for the time being. Even though I can get it back, patch is needed for > netvsc to remove the VF takeover delay IMHO. > > Sridhar, what do you think we revive the IFF_SLAVE_RENAME_OK flag which > allows netvsc to be used later on? Or maybe, IFF_LIVE_RENAME_OK for a > better name? > > -Siwei I would name it IFF_LIVE_NAME_CHANGE to match IFF_LIVE_ADDR_CHANGE there is no reason its use should be restricted to SLAVE devices.
On 4/2/2019 8:14 PM, Stephen Hemminger wrote: > On Tue, 2 Apr 2019 15:23:29 -0700 > si-wei liu <si-wei.liu@oracle.com> wrote: > >> On 4/2/2019 2:53 PM, Stephen Hemminger wrote: >>> On Mon, 1 Apr 2019 19:04:53 -0400 >>> Si-Wei Liu <si-wei.liu@oracle.com> wrote: >>> >>>> + if (dev->flags & IFF_UP && >>>> + likely(!(dev->priv_flags & IFF_FAILOVER_SLAVE))) >>> Why is property limited to failover slave, it would make sense for netvsc >>> as well. Why not make it a flag like live address change? >> Well, netvsc today is still taking the delayed approach meaning that it >> is incompatible yet with this live name change flag if need be. ;-) >> >> I thought Sridhar did not like to introduce an additional >> IFF_SLAVE_RENAME_OK flag given that failover slave is the only consumer >> for the time being. Even though I can get it back, patch is needed for >> netvsc to remove the VF takeover delay IMHO. >> >> Sridhar, what do you think we revive the IFF_SLAVE_RENAME_OK flag which >> allows netvsc to be used later on? Or maybe, IFF_LIVE_RENAME_OK for a >> better name? >> >> -Siwei > > I would name it IFF_LIVE_NAME_CHANGE to match IFF_LIVE_ADDR_CHANGE > there is no reason its use should be restricted to SLAVE devices. > Stephen, May be you should consider moving netvsc to use the net_failover driver now?
On Tue, 2 Apr 2019 22:22:18 -0700 "Samudrala, Sridhar" <sridhar.samudrala@intel.com> wrote: > On 4/2/2019 8:14 PM, Stephen Hemminger wrote: > > On Tue, 2 Apr 2019 15:23:29 -0700 > > si-wei liu <si-wei.liu@oracle.com> wrote: > > > >> On 4/2/2019 2:53 PM, Stephen Hemminger wrote: > >>> On Mon, 1 Apr 2019 19:04:53 -0400 > >>> Si-Wei Liu <si-wei.liu@oracle.com> wrote: > >>> > >>>> + if (dev->flags & IFF_UP && > >>>> + likely(!(dev->priv_flags & IFF_FAILOVER_SLAVE))) > >>> Why is property limited to failover slave, it would make sense for netvsc > >>> as well. Why not make it a flag like live address change? > >> Well, netvsc today is still taking the delayed approach meaning that it > >> is incompatible yet with this live name change flag if need be. ;-) > >> > >> I thought Sridhar did not like to introduce an additional > >> IFF_SLAVE_RENAME_OK flag given that failover slave is the only consumer > >> for the time being. Even though I can get it back, patch is needed for > >> netvsc to remove the VF takeover delay IMHO. > >> > >> Sridhar, what do you think we revive the IFF_SLAVE_RENAME_OK flag which > >> allows netvsc to be used later on? Or maybe, IFF_LIVE_RENAME_OK for a > >> better name? > >> > >> -Siwei > > > > I would name it IFF_LIVE_NAME_CHANGE to match IFF_LIVE_ADDR_CHANGE > > there is no reason its use should be restricted to SLAVE devices. > > > Stephen, > May be you should consider moving netvsc to use the net_failover driver now? > NO Why would I waste time doing that when there is a working and cleaner solution that is working across 4 OS's and three versions of five major distributions?
diff --git a/net/core/dev.c b/net/core/dev.c index 9823b77..b694184 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -1185,7 +1185,21 @@ int dev_change_name(struct net_device *dev, const char *newname) BUG_ON(!dev_net(dev)); net = dev_net(dev); - if (dev->flags & IFF_UP) + + /* Allow failover slave to rename even when + * it is up and running. + * + * Failover slaves are special, since userspace + * might rename the slave after the interface + * has been brought up and running due to + * auto-enslavement. + * + * Failover users don't actually care about slave + * name change, as they are only expected to operate + * on master interface directly. + */ + if (dev->flags & IFF_UP && + likely(!(dev->priv_flags & IFF_FAILOVER_SLAVE))) return -EBUSY; write_seqcount_begin(&devnet_rename_seq); @@ -1232,6 +1246,15 @@ int dev_change_name(struct net_device *dev, const char *newname) hlist_add_head_rcu(&dev->name_hlist, dev_name_hash(net, dev->name)); write_unlock_bh(&dev_base_lock); + if (unlikely(dev->flags & IFF_UP)) { + struct netdev_notifier_change_info change_info = { + .info.dev = dev, + }; + + call_netdevice_notifiers_info(NETDEV_CHANGE, + &change_info.info); + } + ret = call_netdevice_notifiers(NETDEV_CHANGENAME, dev); ret = notifier_to_errno(ret);
When a netdev appears through hot plug then gets enslaved by a failover master that is already up and running, the slave will be opened right away after getting enslaved. Today there's a race that userspace (udev) may fail to rename the slave if the kernel (net_failover) opens the slave earlier than when the userspace rename happens. Unlike bond or team, the primary slave of failover can't be renamed by userspace ahead of time, since the kernel initiated auto-enslavement is unable to, or rather, is never meant to be synchronized with the rename request from userspace. As the failover slave interfaces are not designed to be operated directly by userspace apps: IP configuration, filter rules with regard to network traffic passing and etc., should all be done on master interface. In general, userspace apps only care about the name of master interface, while slave names are less important as long as admin users can see reliable names that may carry other information describing the netdev. For e.g., they can infer that "ens3nsby" is a standby slave of "ens3", while for a name like "eth0" they can't tell which master it belongs to. Historically the name of IFF_UP interface can't be changed because there might be admin script or management software that is already relying on such behavior and assumes that the slave name can't be changed once UP. But failover is special: with the in-kernel auto-enslavement mechanism, the userspace expectation for device enumeration and bring-up order is already broken. Previously initramfs and various userspace config tools were modified to bypass failover slaves because of auto-enslavement and duplicate MAC address. Similarly, in case that users care about seeing reliable slave name, the new type of failover slaves needs to be taken care of specifically in userspace anyway. It's less risky to lift up the rename restriction on failover slave which is already UP. Although it's possible this change may potentially break userspace component (most likely configuration scripts or management software) that assumes slave name can't be changed while UP, it's relatively a limited and controllable set among all userspace components, which can be fixed specifically to listen for the rename and/or link down/up events on failover slaves. Userspace component interacting with slaves is expected to be changed to operate on failover master interface instead, as the failover slave is dynamic in nature which may come and go at any point. The goal is to make the role of failover slaves less relevant, and userspace components should only deal with failover master in the long run. Fixes: 30c8bd5aa8b2 ("net: Introduce generic failover module") Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com> Reviewed-by: Liran Alon <liran.alon@oracle.com> -- v1 -> v2: - Drop configurable module parameter (Sridhar) v2 -> v3: - Drop additional IFF_SLAVE_RENAME_OK flag (Sridhar) - Send down and up events around rename (Michael S. Tsirkin) v3 -> v4: - Simplify notification to be sent (Stephen Hemminger) v4 -> v5: - Sync up code with latest net-next (Sridhar) - Use proper structure initialization (Stephen, Jiri) --- net/core/dev.c | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-)