[ovs-dev] netdev: Retry getting interfaces on inconsistent dumps from kernel

Message ID 20180712061843.46010-1-dalvarez@redhat.com
State Superseded
Headers show
Series
  • [ovs-dev] netdev: Retry getting interfaces on inconsistent dumps from kernel
Related show

Commit Message

Daniel Alvarez Sanchez July 12, 2018, 6:18 a.m.
This patch in glibc [0] is fixing a bug where we may be getting
inconsistent dumps from the kernel when listing interfaces due to
a race condition.

This could happen if we try to retrieve them while interfaces are
being added/removed from the system at the same time.
For systems running against old glibc versions, this patch is retrying
the operation up to 3 times and then proceeding by logging a
warning.

Note that 3 times should be enough to not delay the operation much
and since it's unlikely that we hit the race condition 3 times in
a row. Still, if this happened, this patch is not changing the
current behavior.

[0] https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c1f86a33ca32e26a9d6e29fc961e5ecb5e2e5eb4

Signed-off-by: Daniel Alvarez <dalvarez@redhat.com>
Signed-off-by: Jiri Benc <jbenc@redhat.com>
---
 lib/netdev.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

Comments

Jiri Benc July 12, 2018, 6:44 a.m. | #1
On Thu, 12 Jul 2018 08:18:43 +0200, Daniel Alvarez wrote:
> This patch in glibc [0] is fixing a bug where we may be getting
> inconsistent dumps from the kernel when listing interfaces due to
> a race condition.
> 
> This could happen if we try to retrieve them while interfaces are
> being added/removed from the system at the same time.
> For systems running against old glibc versions, this patch is retrying
> the operation up to 3 times and then proceeding by logging a
> warning.
> 
> Note that 3 times should be enough to not delay the operation much
> and since it's unlikely that we hit the race condition 3 times in
> a row. Still, if this happened, this patch is not changing the
> current behavior.
> 
> [0] https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c1f86a33ca32e26a9d6e29fc961e5ecb5e2e5eb4
> 
> Signed-off-by: Daniel Alvarez <dalvarez@redhat.com>
> Signed-off-by: Jiri Benc <jbenc@redhat.com>

Co-authored-by: Jiri Benc <jbenc@redhat.com>

is better in this case :-)

Thanks, Daniel, for following up on this!

 Jiri
0-day Robot July 12, 2018, 6:55 a.m. | #2
Bleep bloop.  Greetings Daniel Alvarez, I am a robot and I have tried out your patch.
Thanks for your contribution.

I encountered some error that I wasn't expecting.  See the details below.


checkpatch:
ERROR: Too many signoffs; are you missing Co-authored-by lines?
WARNING: Line is 80 characters long (recommended limit is 79)
#57 FILE: lib/netdev.c:2061:
                 * address addition which may cause one of the returned ifa_name

Lines checked: 73, Warnings: 1, Errors: 1


Please check this out.  If you feel there has been an error, please email aconole@bytheb.org

Thanks,
0-day Robot

Patch

diff --git a/lib/netdev.c b/lib/netdev.c
index 82ffeb901..448d6e661 100644
--- a/lib/netdev.c
+++ b/lib/netdev.c
@@ -2039,19 +2039,35 @@  netdev_get_addrs(const char dev[], struct in6_addr **paddr,
     struct in6_addr *addr_array, *mask_array;
     const struct ifaddrs *ifa;
     int cnt = 0, i = 0;
+    int retries = 3;
 
     ovs_mutex_lock(&if_addr_list_lock);
     if (!if_addr_list) {
         int err;
 
+retry:
         err = getifaddrs(&if_addr_list);
         if (err) {
             ovs_mutex_unlock(&if_addr_list_lock);
             return -err;
         }
+        retries--;
     }
 
     for (ifa = if_addr_list; ifa; ifa = ifa->ifa_next) {
+        if (!ifa->ifa_name) {
+            if (retries) {
+                /* Older versions of glibc have a bug on race condition with
+                 * address addition which may cause one of the returned ifa_name
+                 * values to be NULL. In such case, we know that we've got an
+                 * inconsistent dump. Retry but beware of an endless loop. */
+                freeifaddrs(if_addr_list);
+                goto retry;
+            } else {
+                VLOG_WARN("Proceeding with an inconsistent dump of "
+                          "interfaces from the kernel. Some may be missing");
+            }
+        }
         if (ifa->ifa_addr && ifa->ifa_name && ifa->ifa_netmask) {
             int family;