[ovs-dev] socket-util: Introduce emulation and wrapper for recvmmsg().
diff mbox series

Message ID 20191217203836.2208039-1-blp@ovn.org
State New
Headers show
Series
  • [ovs-dev] socket-util: Introduce emulation and wrapper for recvmmsg().
Related show

Commit Message

Ben Pfaff Dec. 17, 2019, 8:38 p.m. UTC
Not every system will have recvmmsg(), so introduce compatibility code
that will allow it to be used blindly from the rest of the tree.

This assumes that recvmmsg() and sendmmsg() are either both present or
both absent in system libraries and headers.

CC: Yi Yang <yangyi01@inspur.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
---
I haven't actually tested this!

 include/sparse/sys/socket.h |  7 ++++-
 lib/socket-util.c           | 56 +++++++++++++++++++++++++++++++++++++
 lib/socket-util.h           | 24 +++++++++-------
 3 files changed, 76 insertions(+), 11 deletions(-)

Comments

0-day Robot Dec. 17, 2019, 8:59 p.m. UTC | #1
Bleep bloop.  Greetings Ben Pfaff, I am a robot and I have tried out your patch.
Thanks for your contribution.

I encountered some error that I wasn't expecting.  See the details below.


checkpatch:
WARNING: Comment with 'xxx' marker
#65 FILE: lib/socket-util.c:1293:
    ovs_assert(!timeout);       /* XXX not emulated */

WARNING: Comment with 'xxx' marker
#99 FILE: lib/socket-util.c:1327:
    ovs_assert(!timeout);       /* XXX not emulated */

Lines checked: 165, Warnings: 2, Errors: 0


Please check this out.  If you feel there has been an error, please email aconole@redhat.com

Thanks,
0-day Robot
Aaron Conole Dec. 17, 2019, 9:51 p.m. UTC | #2
Ben Pfaff <blp@ovn.org> writes:

> Not every system will have recvmmsg(), so introduce compatibility code
> that will allow it to be used blindly from the rest of the tree.
>
> This assumes that recvmmsg() and sendmmsg() are either both present or
> both absent in system libraries and headers.
>
> CC: Yi Yang <yangyi01@inspur.com>
> Signed-off-by: Ben Pfaff <blp@ovn.org>
> ---
> I haven't actually tested this!

I have trouble understanding the motivation for this patch.  Will
there be an attempt to rewrite netlink-socket or netdev-linux to use
recvmmsg?  Simply adding this function with no new callers seems like
introducing dead code.  I'd expect the next version to include some
in-tree user.

>  include/sparse/sys/socket.h |  7 ++++-
>  lib/socket-util.c           | 56 +++++++++++++++++++++++++++++++++++++
>  lib/socket-util.h           | 24 +++++++++-------
>  3 files changed, 76 insertions(+), 11 deletions(-)
>
> diff --git a/include/sparse/sys/socket.h b/include/sparse/sys/socket.h
> index 4178f57e2bda..6ff245ae939b 100644
> --- a/include/sparse/sys/socket.h
> +++ b/include/sparse/sys/socket.h
> @@ -27,6 +27,7 @@
>  
>  typedef unsigned short int sa_family_t;
>  typedef __socklen_t socklen_t;
> +struct timespec;
>  
>  struct sockaddr {
>      sa_family_t sa_family;
> @@ -126,7 +127,8 @@ enum {
>      MSG_PEEK,
>      MSG_TRUNC,
>      MSG_WAITALL,
> -    MSG_DONTWAIT
> +    MSG_DONTWAIT,
> +    MSG_WAITFORONE
>  };
>  
>  enum {
> @@ -171,4 +173,7 @@ int sockatmark(int);
>  int socket(int, int, int);
>  int socketpair(int, int, int, int[2]);
>  
> +int sendmmsg(int, struct mmsghdr *, unsigned int, int);
> +int recvmmsg(int, struct mmsghdr *, unsigned int, int, struct timespec *);
> +
>  #endif /* <sys/socket.h> for sparse */
> diff --git a/lib/socket-util.c b/lib/socket-util.c
> index 6b7378de934b..f6f6f3b0a33f 100644
> --- a/lib/socket-util.c
> +++ b/lib/socket-util.c
> @@ -1283,3 +1283,59 @@ wrap_sendmmsg(int fd, struct mmsghdr *msgs, unsigned int n, unsigned int flags)
>  }
>  #endif
>  #endif
> +
> +#ifndef _WIN32 /* Avoid using recvmsg on Windows entirely. */
> +static int
> +emulate_recvmmsg(int fd, struct mmsghdr *msgs, unsigned int n,
> +                 int flags, struct timespec *timeout OVS_UNUSED)
> +{
> +    ovs_assert(!timeout);       /* XXX not emulated */

Why not?  A really simple way of implementing it (albeit not a terribly
resource friendly one) would be to call recvmsg() with
  (flags | MSG_DONTWAIT)

Then decrement time, and if there's still time left in the timeout,
retry.  Alternately, setup an alarm.  There may be other methods to
achieve it.  The point being that we probably can accommodate this
timeout.

Otherwise, it should at least be documented that the behavior isn't the
same.  In that case, I'd prefer calling it 'ovs_recvmmsg' rather than
simply masquerading the function away.  That way, reading the code (and
authoring new code) the difference is obvious (and we could even
just... eliminate timeout as a parameter).

> +    bool waitforone = flags & MSG_WAITFORONE;
> +    flags &= ~MSG_WAITFORONE;
> +
> +    for (unsigned int i = 0; i < n; i++) {
> +        ssize_t retval = recvmsg(fd, &msgs[i].msg_hdr, flags);
> +        if (retval < 0) {
> +            return i ? i : retval;
> +        }
> +        msgs[i].msg_len = retval;
> +
> +        if (waitforone) {
> +            flags |= MSG_DONTWAIT;
> +        }
> +    }
> +    return n;
> +}
> +
> +#ifndef HAVE_SENDMMSG
> +int
> +recvmmsg(int fd, struct mmsghdr *msgs, unsigned int n,
> +         int flags, struct timespec *timeout)
> +{
> +    return emulate_recvmmsg(fd, msgs, n, flags, timeout);
> +}
> +#else
> +/* recvmmsg was redefined in lib/socket-util.c, should undef recvmmsg here

Shouldn't that read lib/socket-util.h ?  That also applies to the
sendmmsg comment ;)

> + * to avoid recursion */
> +#undef recvmmsg
> +int
> +wrap_recvmmsg(int fd, struct mmsghdr *msgs, unsigned int n,
> +              int flags, struct timespec *timeout)
> +{
> +    ovs_assert(!timeout);       /* XXX not emulated */

This doesn't make sense.  recvmmsg(..., NULL); is perfectly valid, and
this is supposed to be a wrapper?  On systems where it is valid to call,
why can't we use it?

> +    static bool recvmmsg_broken = false;
> +    if (!recvmmsg_broken) {
> +        int save_errno = errno;
> +        int retval = recvmmsg(fd, msgs, n, flags, timeout);
> +        if (retval >= 0 || errno != ENOSYS) {

It's strange, no matter what retval is, we would return here.  The only
case we really care about is errno, yes?  I'm not asking for any change
at this line, it's just an observation.

> +            return retval;
> +        }
> +        recvmmsg_broken = true;
> +        errno = save_errno;
> +    }
> +    return emulate_recvmmsg(fd, msgs, n, flags, timeout);
> +}
> +#endif
> +#endif
> diff --git a/lib/socket-util.h b/lib/socket-util.h
> index a65433d90738..71bd68926aaa 100644
> --- a/lib/socket-util.h
> +++ b/lib/socket-util.h
> @@ -104,19 +104,20 @@ int make_unix_socket(int style, bool nonblock,
>                       const char *bind_path, const char *connect_path);
>  int get_unix_name_len(const struct sockaddr_un *sun, socklen_t sun_len);
>  
> -/* Universal sendmmsg support.
> +/* Universal sendmmsg and recvmmsg support.
>   *
> - * Some platforms, such as new enough Linux and FreeBSD, support sendmmsg, but
> - * other platforms (or older ones) do not.  We add the following infrastructure
> - * to allow all code to use sendmmsg, regardless of platform support:
> + * Some platforms, such as new enough Linux and FreeBSD, support sendmmsg and
> + * recvmmsg, but other platforms (or older ones) do not.  We add the following
> + * infrastructure to allow all code to use sendmmsg and recvmmsg, regardless of
> + * platform support:
>   *
> - *   - For platforms that lack sendmmsg entirely, we emulate it.
> + * - For platforms that lack these functions entirely, we emulate them.
>   *
> - *   - Some platforms have sendmmsg() in the C library but not in the kernel.
> - *     For example, this is true if a Linux system has a newer glibc with an
> - *     old kernel.  To compensate, even if sendmmsg() appears to be available,
> - *     we still wrap it with a handler that uses our emulation if sendmmsg()
> - *     returns ENOSYS.
> + * - Some platforms have sendmmsg() and recvmmsg() in the C library but not in
> + *   the kernel.  For example, this is true if a Linux system has a newer glibc
> + *   with an old kernel.  To compensate, even if these functions appear to be
> + *   available, we still wrap them with handlers that uses our emulation if the
> + *   underlying function returns ENOSYS.
>   */
>  #ifndef HAVE_STRUCT_MMSGHDR_MSG_LEN
>  struct mmsghdr {
> @@ -126,9 +127,12 @@ struct mmsghdr {
>  #endif
>  #ifndef HAVE_SENDMMSG
>  int sendmmsg(int, struct mmsghdr *, unsigned int, unsigned int);
> +int recvmmsg(int, struct mmsghdr *, unsigned int, int, struct timespec *);
>  #else
>  #define sendmmsg wrap_sendmmsg
>  int wrap_sendmmsg(int, struct mmsghdr *, unsigned int, unsigned int);
> +#define recvmmsg wrap_recvmmsg
> +int wrap_recvmmsg(int, struct mmsghdr *, unsigned int, int, struct timespec *);
>  #endif
>  
>  /* Helpers for calling ioctl() on an AF_INET socket. */
Ben Pfaff Dec. 17, 2019, 10:02 p.m. UTC | #3
On Tue, Dec 17, 2019 at 04:51:35PM -0500, Aaron Conole wrote:
> Ben Pfaff <blp@ovn.org> writes:
> 
> > Not every system will have recvmmsg(), so introduce compatibility code
> > that will allow it to be used blindly from the rest of the tree.
> >
> > This assumes that recvmmsg() and sendmmsg() are either both present or
> > both absent in system libraries and headers.
> >
> > CC: Yi Yang <yangyi01@inspur.com>
> > Signed-off-by: Ben Pfaff <blp@ovn.org>
> > ---
> > I haven't actually tested this!
> 
> I have trouble understanding the motivation for this patch.  Will
> there be an attempt to rewrite netlink-socket or netdev-linux to use
> recvmmsg?  Simply adding this function with no new callers seems like
> introducing dead code.  I'd expect the next version to include some
> in-tree user.

This is relevant to Yi Yang's patch:
http://patchwork.ozlabs.org/patch/1204939/

I should have referred to it or put it in his thread, sorry.
Ben Pfaff Dec. 20, 2019, 8:51 p.m. UTC | #4
On Fri, Dec 20, 2019 at 01:25:29AM +0000, Yi Yang (杨�D)-云服务集团 wrote:
> Current ovs matser has included sendmmsg declaration in
> include/sparse/sys/socket.h

I believe you are mistaken.

> int sendmmsg(int, struct mmsghdr *, unsigned int, unsigned int);
> 
> I saw  "+^L" in your patch.

Yes, OVS uses page breaks to separate logical sections of code.  The
coding style document mentions this.

> --- a/lib/socket-util.c
> +++ b/lib/socket-util.c
> @@ -1283,3 +1283,59 @@ wrap_sendmmsg(int fd, struct mmsghdr *msgs, unsigned
> int n, unsigned int flags)
>  }
>  #endif
>  #endif
> +^L
> +#ifndef _WIN32 /* Avoid using recvmsg on Windows entirely. */
> 
> +#undef recvmmsg
> +int
> +wrap_recvmmsg(int fd, struct mmsghdr *msgs, unsigned int n,
> +              int flags, struct timespec *timeout)
> +{
> +    ovs_assert(!timeout);       /* XXX not emulated */
> +
> +    static bool recvmmsg_broken = false;
> +    if (!recvmmsg_broken) {
> +        int save_errno = errno;
> +        int retval = recvmmsg(fd, msgs, n, flags, timeout);
> +        if (retval >= 0 || errno != ENOSYS) {
> +            return retval;
> +        }
> +        recvmmsg_broken = true;
> +        errno = save_errno;
> +    }
> +    return emulate_recvmmsg(fd, msgs, n, flags, timeout);
> +}
> +#endif
> 
> I don't understand why call recvmmsg here although we have known recvmmsg
> isn't defined,

Can you explain that comment?  I don't believe that the code tries to
call recvmmsg() when it is not defined.  The code inside #ifndef
HAVE_SENDMMSG only uses emulate_recvmmsg(), which itself only calls
recvmsg(), not recvmmsg().

> I don't think "static bool recvmmsg_broken" is thread-safe. 

It is thread-safe enough, because it is merely an optimization: if the
value is wrong, then at most the code gets a little bit slower.

> I think we can completely remove the below part if we do know recvmmsg
> isn't defined (I think autoconf can detect it very precisely, we
> needn't to do runtime check for this)
> +    static bool recvmmsg_broken = false;
> +    if (!recvmmsg_broken) {
> +        int save_errno = errno;
> +        int retval = recvmmsg(fd, msgs, n, flags, timeout);
> +        if (retval >= 0 || errno != ENOSYS) {
> +            return retval;
> +        }
> +        recvmmsg_broken = true;
> +        errno = save_errno;
> +    }

There are three cases:

1. The C library does not have recvmmsg().  Then we cannot call it at
   all.  In this case, HAVE_SENDMMSG is false and the "#ifndef
   HAVE_SENDMMSG" fork will use emulate_recvmmsg().

2. The C library has recvmmsg() but the kernel does not, because it is
   too old.  Then wrap_recvmmsg() will receive an ENOSYS error from the
   kernel, call emulate_recvmmsg(), and set recvmmsg_broken so that
   future calls don't have to bother going into the kernel at all.

3. The C library and the kernel both have recvmmsg().  Then
   wrap_recvmmsg() will call recvmmsg() and either succeed or get back
   some error other than ENOSYS.  recvmmsg_broken will remain false, and
   all future calls to recvmmsg() will also take the kernel fast path.

Autoconf cannot distinguish cases 2 and 3, nor can anything that runs at
build time, because there is no way to guess whether the runtime kernel
matches the build time kernel.
Yi Yang (杨燚)-云服务集团 Dec. 23, 2019, 12:22 a.m. UTC | #5
Ben, socket.h in master does include sendmmsg

https://github.com/openvswitch/ovs/blob/master/include/sparse/sys/socket.h#L165

Per your explanation, I understood why you call recvmsg there, so I don't have other comments.

As William explained in his RFC patch, I think TPACKET_V3 is the best way to fix this. I tried af_packet to use veth in OVS DPDK, it's performance is 2 times more than my patch, about 4Gbps, for my patch, veth performance is about 1.47Gbps, af_packet just used TPACKET_V2, TPACKET_V3 should be much better than TPACKET_V2 per William's explanation.

-----邮件原件-----
发件人: Ben Pfaff [mailto:blp@ovn.org] 
发送时间: 2019年12月21日 4:51
收件人: Yi Yang (杨燚)-云服务集团 <yangyi01@inspur.com>
抄送: dev@openvswitch.org
主题: Re: 答复: [PATCH] socket-util: Introduce emulation and wrapper for recvmmsg().

On Fri, Dec 20, 2019 at 01:25:29AM +0000, Yi Yang (杨 D)-云服务集团 wrote:
> Current ovs matser has included sendmmsg declaration in 
> include/sparse/sys/socket.h

I believe you are mistaken.

> int sendmmsg(int, struct mmsghdr *, unsigned int, unsigned int);
> 
> I saw  "+^L" in your patch.

Yes, OVS uses page breaks to separate logical sections of code.  The coding style document mentions this.

> --- a/lib/socket-util.c
> +++ b/lib/socket-util.c
> @@ -1283,3 +1283,59 @@ wrap_sendmmsg(int fd, struct mmsghdr *msgs, 
> unsigned int n, unsigned int flags)  }  #endif  #endif
> +^L
> +#ifndef _WIN32 /* Avoid using recvmsg on Windows entirely. */
> 
> +#undef recvmmsg
> +int
> +wrap_recvmmsg(int fd, struct mmsghdr *msgs, unsigned int n,
> +              int flags, struct timespec *timeout) {
> +    ovs_assert(!timeout);       /* XXX not emulated */
> +
> +    static bool recvmmsg_broken = false;
> +    if (!recvmmsg_broken) {
> +        int save_errno = errno;
> +        int retval = recvmmsg(fd, msgs, n, flags, timeout);
> +        if (retval >= 0 || errno != ENOSYS) {
> +            return retval;
> +        }
> +        recvmmsg_broken = true;
> +        errno = save_errno;
> +    }
> +    return emulate_recvmmsg(fd, msgs, n, flags, timeout); } #endif
> 
> I don't understand why call recvmmsg here although we have known 
> recvmmsg isn't defined,

Can you explain that comment?  I don't believe that the code tries to call recvmmsg() when it is not defined.  The code inside #ifndef HAVE_SENDMMSG only uses emulate_recvmmsg(), which itself only calls recvmsg(), not recvmmsg().

> I don't think "static bool recvmmsg_broken" is thread-safe. 

It is thread-safe enough, because it is merely an optimization: if the value is wrong, then at most the code gets a little bit slower.

> I think we can completely remove the below part if we do know recvmmsg 
> isn't defined (I think autoconf can detect it very precisely, we 
> needn't to do runtime check for this)
> +    static bool recvmmsg_broken = false;
> +    if (!recvmmsg_broken) {
> +        int save_errno = errno;
> +        int retval = recvmmsg(fd, msgs, n, flags, timeout);
> +        if (retval >= 0 || errno != ENOSYS) {
> +            return retval;
> +        }
> +        recvmmsg_broken = true;
> +        errno = save_errno;
> +    }

There are three cases:

1. The C library does not have recvmmsg().  Then we cannot call it at
   all.  In this case, HAVE_SENDMMSG is false and the "#ifndef
   HAVE_SENDMMSG" fork will use emulate_recvmmsg().

2. The C library has recvmmsg() but the kernel does not, because it is
   too old.  Then wrap_recvmmsg() will receive an ENOSYS error from the
   kernel, call emulate_recvmmsg(), and set recvmmsg_broken so that
   future calls don't have to bother going into the kernel at all.

3. The C library and the kernel both have recvmmsg().  Then
   wrap_recvmmsg() will call recvmmsg() and either succeed or get back
   some error other than ENOSYS.  recvmmsg_broken will remain false, and
   all future calls to recvmmsg() will also take the kernel fast path.

Autoconf cannot distinguish cases 2 and 3, nor can anything that runs at build time, because there is no way to guess whether the runtime kernel matches the build time kernel.
Ben Pfaff Jan. 7, 2020, 8:10 p.m. UTC | #6
On Mon, Dec 23, 2019 at 12:22:52AM +0000, Yi Yang (杨燚)-云服务集团 wrote:
> Ben, socket.h in master does include sendmmsg
> 
> https://github.com/openvswitch/ovs/blob/master/include/sparse/sys/socket.h#L165
> 
> Per your explanation, I understood why you call recvmsg there, so I don't have other comments.
> 
> As William explained in his RFC patch, I think TPACKET_V3 is the best way to fix this. I tried af_packet to use veth in OVS DPDK, it's performance is 2 times more than my patch, about 4Gbps, for my patch, veth performance is about 1.47Gbps, af_packet just used TPACKET_V2, TPACKET_V3 should be much better than TPACKET_V2 per William's explanation.

OK.  Do you want to continue working to use recvmmsg() in OVS?  Or do
you want to withdraw the idea in favor of TPACKET_V3?  The possible
advantage of recvmmsg() is that it's going to be available pretty much
everywhere, whereas TPACKET_V3 is a more recent addition to Linux.
Yi Yang (杨燚)-云服务集团 Jan. 8, 2020, 12:27 a.m. UTC | #7
Ben, I think the patch using recvmmsg is ready for merge if you want, basically 4.15 or later kernels can support TPACKET_V3, I'm not sure if recvmmsg and TPACKET_V3 can coexist, do you mean we can use config HAVE_ TPACKET_V3/2 to build different version for different kernel?

-----邮件原件-----
发件人: Ben Pfaff [mailto:blp@ovn.org] 
发送时间: 2020年1月8日 4:11
收件人: Yi Yang (杨燚)-云服务集团 <yangyi01@inspur.com>
抄送: dev@openvswitch.org
主题: Re: 答复: 答复: [PATCH] socket-util: Introduce emulation and wrapper for recvmmsg().

On Mon, Dec 23, 2019 at 12:22:52AM +0000, Yi Yang (杨燚)-云服务集团 wrote:
> Ben, socket.h in master does include sendmmsg
> 
> https://github.com/openvswitch/ovs/blob/master/include/sparse/sys/sock
> et.h#L165
> 
> Per your explanation, I understood why you call recvmsg there, so I don't have other comments.
> 
> As William explained in his RFC patch, I think TPACKET_V3 is the best way to fix this. I tried af_packet to use veth in OVS DPDK, it's performance is 2 times more than my patch, about 4Gbps, for my patch, veth performance is about 1.47Gbps, af_packet just used TPACKET_V2, TPACKET_V3 should be much better than TPACKET_V2 per William's explanation.

OK.  Do you want to continue working to use recvmmsg() in OVS?  Or do you want to withdraw the idea in favor of TPACKET_V3?  The possible advantage of recvmmsg() is that it's going to be available pretty much everywhere, whereas TPACKET_V3 is a more recent addition to Linux.
Ben Pfaff Jan. 9, 2020, 5:51 p.m. UTC | #8
OK.  I applied both patches to master.  Thank you!

On Wed, Jan 08, 2020 at 12:27:31AM +0000, Yi Yang (杨燚)-云服务集团 wrote:
> Ben, I think the patch using recvmmsg is ready for merge if you want, basically 4.15 or later kernels can support TPACKET_V3, I'm not sure if recvmmsg and TPACKET_V3 can coexist, do you mean we can use config HAVE_ TPACKET_V3/2 to build different version for different kernel?
> 
> -----邮件原件-----
> 发件人: Ben Pfaff [mailto:blp@ovn.org] 
> 发送时间: 2020年1月8日 4:11
> 收件人: Yi Yang (杨燚)-云服务集团 <yangyi01@inspur.com>
> 抄送: dev@openvswitch.org
> 主题: Re: 答复: 答复: [PATCH] socket-util: Introduce emulation and wrapper for recvmmsg().
> 
> On Mon, Dec 23, 2019 at 12:22:52AM +0000, Yi Yang (杨燚)-云服务集团 wrote:
> > Ben, socket.h in master does include sendmmsg
> > 
> > https://github.com/openvswitch/ovs/blob/master/include/sparse/sys/sock
> > et.h#L165
> > 
> > Per your explanation, I understood why you call recvmsg there, so I don't have other comments.
> > 
> > As William explained in his RFC patch, I think TPACKET_V3 is the best way to fix this. I tried af_packet to use veth in OVS DPDK, it's performance is 2 times more than my patch, about 4Gbps, for my patch, veth performance is about 1.47Gbps, af_packet just used TPACKET_V2, TPACKET_V3 should be much better than TPACKET_V2 per William's explanation.
> 
> OK.  Do you want to continue working to use recvmmsg() in OVS?  Or do you want to withdraw the idea in favor of TPACKET_V3?  The possible advantage of recvmmsg() is that it's going to be available pretty much everywhere, whereas TPACKET_V3 is a more recent addition to Linux.
Ilya Maximets Jan. 10, 2020, 5:02 p.m. UTC | #9
> OK.  I applied both patches to master.  Thank you!

Hi, Ben.

These patches broke OSX build:

https://travis-ci.org/openvswitch/ovs/jobs/634863662

lib/socket-util.c:1294:31: error: use of undeclared identifier 'MSG_WAITFORONE'
    bool waitforone = flags & MSG_WAITFORONE;
                              ^
lib/socket-util.c:1295:15: error: use of undeclared identifier 'MSG_WAITFORONE'
    flags &= ~MSG_WAITFORONE;
              ^

Best regards, Ilya Maximets.
Ben Pfaff Jan. 13, 2020, 6:03 p.m. UTC | #10
On Fri, Jan 10, 2020 at 06:02:29PM +0100, Ilya Maximets wrote:
> > OK.  I applied both patches to master.  Thank you!
> 
> Hi, Ben.
> 
> These patches broke OSX build:
> 
> https://travis-ci.org/openvswitch/ovs/jobs/634863662
> 
> lib/socket-util.c:1294:31: error: use of undeclared identifier 'MSG_WAITFORONE'
>     bool waitforone = flags & MSG_WAITFORONE;
>                               ^
> lib/socket-util.c:1295:15: error: use of undeclared identifier 'MSG_WAITFORONE'
>     flags &= ~MSG_WAITFORONE;

Oops.

I sent a fix:
http://patchwork.ozlabs.org/patch/1222256/

Patch
diff mbox series

diff --git a/include/sparse/sys/socket.h b/include/sparse/sys/socket.h
index 4178f57e2bda..6ff245ae939b 100644
--- a/include/sparse/sys/socket.h
+++ b/include/sparse/sys/socket.h
@@ -27,6 +27,7 @@ 
 
 typedef unsigned short int sa_family_t;
 typedef __socklen_t socklen_t;
+struct timespec;
 
 struct sockaddr {
     sa_family_t sa_family;
@@ -126,7 +127,8 @@  enum {
     MSG_PEEK,
     MSG_TRUNC,
     MSG_WAITALL,
-    MSG_DONTWAIT
+    MSG_DONTWAIT,
+    MSG_WAITFORONE
 };
 
 enum {
@@ -171,4 +173,7 @@  int sockatmark(int);
 int socket(int, int, int);
 int socketpair(int, int, int, int[2]);
 
+int sendmmsg(int, struct mmsghdr *, unsigned int, int);
+int recvmmsg(int, struct mmsghdr *, unsigned int, int, struct timespec *);
+
 #endif /* <sys/socket.h> for sparse */
diff --git a/lib/socket-util.c b/lib/socket-util.c
index 6b7378de934b..f6f6f3b0a33f 100644
--- a/lib/socket-util.c
+++ b/lib/socket-util.c
@@ -1283,3 +1283,59 @@  wrap_sendmmsg(int fd, struct mmsghdr *msgs, unsigned int n, unsigned int flags)
 }
 #endif
 #endif
+
+#ifndef _WIN32 /* Avoid using recvmsg on Windows entirely. */
+static int
+emulate_recvmmsg(int fd, struct mmsghdr *msgs, unsigned int n,
+                 int flags, struct timespec *timeout OVS_UNUSED)
+{
+    ovs_assert(!timeout);       /* XXX not emulated */
+
+    bool waitforone = flags & MSG_WAITFORONE;
+    flags &= ~MSG_WAITFORONE;
+
+    for (unsigned int i = 0; i < n; i++) {
+        ssize_t retval = recvmsg(fd, &msgs[i].msg_hdr, flags);
+        if (retval < 0) {
+            return i ? i : retval;
+        }
+        msgs[i].msg_len = retval;
+
+        if (waitforone) {
+            flags |= MSG_DONTWAIT;
+        }
+    }
+    return n;
+}
+
+#ifndef HAVE_SENDMMSG
+int
+recvmmsg(int fd, struct mmsghdr *msgs, unsigned int n,
+         int flags, struct timespec *timeout)
+{
+    return emulate_recvmmsg(fd, msgs, n, flags, timeout);
+}
+#else
+/* recvmmsg was redefined in lib/socket-util.c, should undef recvmmsg here
+ * to avoid recursion */
+#undef recvmmsg
+int
+wrap_recvmmsg(int fd, struct mmsghdr *msgs, unsigned int n,
+              int flags, struct timespec *timeout)
+{
+    ovs_assert(!timeout);       /* XXX not emulated */
+
+    static bool recvmmsg_broken = false;
+    if (!recvmmsg_broken) {
+        int save_errno = errno;
+        int retval = recvmmsg(fd, msgs, n, flags, timeout);
+        if (retval >= 0 || errno != ENOSYS) {
+            return retval;
+        }
+        recvmmsg_broken = true;
+        errno = save_errno;
+    }
+    return emulate_recvmmsg(fd, msgs, n, flags, timeout);
+}
+#endif
+#endif
diff --git a/lib/socket-util.h b/lib/socket-util.h
index a65433d90738..71bd68926aaa 100644
--- a/lib/socket-util.h
+++ b/lib/socket-util.h
@@ -104,19 +104,20 @@  int make_unix_socket(int style, bool nonblock,
                      const char *bind_path, const char *connect_path);
 int get_unix_name_len(const struct sockaddr_un *sun, socklen_t sun_len);
 
-/* Universal sendmmsg support.
+/* Universal sendmmsg and recvmmsg support.
  *
- * Some platforms, such as new enough Linux and FreeBSD, support sendmmsg, but
- * other platforms (or older ones) do not.  We add the following infrastructure
- * to allow all code to use sendmmsg, regardless of platform support:
+ * Some platforms, such as new enough Linux and FreeBSD, support sendmmsg and
+ * recvmmsg, but other platforms (or older ones) do not.  We add the following
+ * infrastructure to allow all code to use sendmmsg and recvmmsg, regardless of
+ * platform support:
  *
- *   - For platforms that lack sendmmsg entirely, we emulate it.
+ * - For platforms that lack these functions entirely, we emulate them.
  *
- *   - Some platforms have sendmmsg() in the C library but not in the kernel.
- *     For example, this is true if a Linux system has a newer glibc with an
- *     old kernel.  To compensate, even if sendmmsg() appears to be available,
- *     we still wrap it with a handler that uses our emulation if sendmmsg()
- *     returns ENOSYS.
+ * - Some platforms have sendmmsg() and recvmmsg() in the C library but not in
+ *   the kernel.  For example, this is true if a Linux system has a newer glibc
+ *   with an old kernel.  To compensate, even if these functions appear to be
+ *   available, we still wrap them with handlers that uses our emulation if the
+ *   underlying function returns ENOSYS.
  */
 #ifndef HAVE_STRUCT_MMSGHDR_MSG_LEN
 struct mmsghdr {
@@ -126,9 +127,12 @@  struct mmsghdr {
 #endif
 #ifndef HAVE_SENDMMSG
 int sendmmsg(int, struct mmsghdr *, unsigned int, unsigned int);
+int recvmmsg(int, struct mmsghdr *, unsigned int, int, struct timespec *);
 #else
 #define sendmmsg wrap_sendmmsg
 int wrap_sendmmsg(int, struct mmsghdr *, unsigned int, unsigned int);
+#define recvmmsg wrap_recvmmsg
+int wrap_recvmmsg(int, struct mmsghdr *, unsigned int, int, struct timespec *);
 #endif
 
 /* Helpers for calling ioctl() on an AF_INET socket. */