| Message ID | 20090124224930.GA4456@localhost.localdomain |
|---|---|
| State | Changes Requested, archived |
| Delegated to: | David Miller |
| Headers | show |
On 24-01-2009 23:49, Vegard Nossum wrote: > Hi, > > This survives basic testing here, but I don't know what that counts for > when I couldn't reproduce the lockdep report in the first place. Please > review. > > > Vegard > > > From cc8bcd1c4fd219a31d6d191aefa4b4b57dadb9b0 Mon Sep 17 00:00:00 2001 > From: Vegard Nossum <vegard.nossum@gmail.com> > Date: Sat, 24 Jan 2009 22:44:16 +0100 > Subject: [PATCH] net: fix setsockopt() locking errors > MIME-Version: 1.0 > Content-Type: text/plain; charset=utf-8 > Content-Transfer-Encoding: 8bit > > Martin MOKREJ. <mmokrejs@ribosome.natur.cuni.cz> reported: >> ======================================================= >> [ INFO: possible circular locking dependency detected ] >> 2.6.29-rc2-git1 #1 >> ------------------------------------------------------- >> tcpdump/3734 is trying to acquire lock: >> (&mm->mmap_sem){----}, at: [<c1053294>] might_fault+0x30/0x6b >> >> but task is already holding lock: >> (sk_lock-AF_PACKET){--..}, at: [<c12798c8>] sock_setsockopt+0x12b/0x4a4 >> >> which lock already depends on the new lock. > > It turns out that sock_setsockopt() is calling copy_from_user() while > holding the lock on the socket. I guess it has been like this for some time, so it would be nice to mention what scenario happens here, or IOW what exactly needs to get these locks in reverse order. > We fix it by splitting the ioctl code > so that one switch handles the ioctls that have their own code for > reading from userspace, and one switch handles the cases that require > no additional reading. > > Reported-by: Martin MOKREJ. <mmokrejs@ribosome.natur.cuni.cz> > Signed-off-by: Vegard Nossum <vegard.nossum@gmail.com> > --- > net/core/sock.c | 134 +++++++++++++++++++++++++++++++++++------------------- > 1 files changed, 87 insertions(+), 47 deletions(-) > > diff --git a/net/core/sock.c b/net/core/sock.c > index f3a0d08..6bd618d 100644 > --- a/net/core/sock.c > +++ b/net/core/sock.c > @@ -424,6 +424,80 @@ out: > return ret; > } > > +static int sock_linger(struct sock *sk, char __user *optval, int optlen) ... > +static int sock_set_rcvtimeo(struct sock *sk, char __user *optval, int optlen) > +{ > + int ret; > + long rcvtimeo; > + > + ret = sock_set_timeout(&rcvtimeo, optval, optlen); A check for error is needed here and below. > + > + lock_sock(sk); > + sk->sk_rcvtimeo = rcvtimeo; > + release_sock(sk); > + > + return ret; > +} > + > +static int sock_set_sndtimeo(struct sock *sk, char __user *optval, int optlen) > +{ > + int ret; > + long sndtimeo; > + > + ret = sock_set_timeout(&sndtimeo, optval, optlen); > + > + lock_sock(sk); > + sk->sk_sndtimeo = sndtimeo; > + release_sock(sk); > + > + return ret; > +} ... Jarek P. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Jarek Poplawski <jarkao2@gmail.com> Date: Mon, 26 Jan 2009 11:50:12 +0000 > On 24-01-2009 23:49, Vegard Nossum wrote: > > +static int sock_linger(struct sock *sk, char __user *optval, int optlen) > ... > > +static int sock_set_rcvtimeo(struct sock *sk, char __user *optval, int optlen) > > +{ > > + int ret; > > + long rcvtimeo; > > + > > + ret = sock_set_timeout(&rcvtimeo, optval, optlen); > > A check for error is needed here and below. Right, you cannot continue and update the socket state if this sock_set_timeout() call returns an error. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
The patch really did not help: http://bugzilla.kernel.org/show_bug.cgi?id=12515#c5 Martin Jarek Poplawski wrote: > On 24-01-2009 23:49, Vegard Nossum wrote: >> Hi, >> >> This survives basic testing here, but I don't know what that counts for >> when I couldn't reproduce the lockdep report in the first place. Please >> review. >> >> >> Vegard >> >> >> From cc8bcd1c4fd219a31d6d191aefa4b4b57dadb9b0 Mon Sep 17 00:00:00 2001 >> From: Vegard Nossum <vegard.nossum@gmail.com> >> Date: Sat, 24 Jan 2009 22:44:16 +0100 >> Subject: [PATCH] net: fix setsockopt() locking errors >> MIME-Version: 1.0 >> Content-Type: text/plain; charset=utf-8 >> Content-Transfer-Encoding: 8bit >> >> Martin MOKREJ. <mmokrejs@ribosome.natur.cuni.cz> reported: >>> ======================================================= >>> [ INFO: possible circular locking dependency detected ] >>> 2.6.29-rc2-git1 #1 >>> ------------------------------------------------------- >>> tcpdump/3734 is trying to acquire lock: >>> (&mm->mmap_sem){----}, at: [<c1053294>] might_fault+0x30/0x6b >>> >>> but task is already holding lock: >>> (sk_lock-AF_PACKET){--..}, at: [<c12798c8>] sock_setsockopt+0x12b/0x4a4 >>> >>> which lock already depends on the new lock. >> It turns out that sock_setsockopt() is calling copy_from_user() while >> holding the lock on the socket. > > I guess it has been like this for some time, so it would be nice to > mention what scenario happens here, or IOW what exactly needs to get > these locks in reverse order. > >> We fix it by splitting the ioctl code >> so that one switch handles the ioctls that have their own code for >> reading from userspace, and one switch handles the cases that require >> no additional reading. >> >> Reported-by: Martin MOKREJ. <mmokrejs@ribosome.natur.cuni.cz> >> Signed-off-by: Vegard Nossum <vegard.nossum@gmail.com> >> --- >> net/core/sock.c | 134 +++++++++++++++++++++++++++++++++++------------------- >> 1 files changed, 87 insertions(+), 47 deletions(-) >> >> diff --git a/net/core/sock.c b/net/core/sock.c >> index f3a0d08..6bd618d 100644 >> --- a/net/core/sock.c >> +++ b/net/core/sock.c >> @@ -424,6 +424,80 @@ out: >> return ret; >> } >> >> +static int sock_linger(struct sock *sk, char __user *optval, int optlen) > ... >> +static int sock_set_rcvtimeo(struct sock *sk, char __user *optval, int optlen) >> +{ >> + int ret; >> + long rcvtimeo; >> + >> + ret = sock_set_timeout(&rcvtimeo, optval, optlen); > > A check for error is needed here and below. > [cut] -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Jan 26, 2009 at 10:30:30PM +0100, Martin MOKREJŠ wrote: > The patch really did not help: > http://bugzilla.kernel.org/show_bug.cgi?id=12515#c5 > Martin Actually, there is a little change: the warning triggerd in another place (sock_setsockopt() -> sk_attach_filter()). So we could go deeper with these changes, but I'm not sure this is the right way to fix. It looks like the scenario is very old, but probably wasn't reported (maybe there is some lockdep improvement): A) sys_mmap2() -> mm->mmap_sem -> packet_mmap() -> sk_lock B) sock_setsockopt() -> sk_lock -> copy_from_user() -> mm->mmap_sem packet_mmap() (net/packet/af_packet.c) seems to be the only place in net to implement mmap method, and using this lock order btw. On the other hand copy_from_user() could be more popular under sk_lock, and I'm not sure these changes are necessary. Since I don't know enough neither sock/packet nor sys_mmap, I guess some advice would be precious. It looks like Peter Zijlstra solved similar problems in nfs, so I CC him. Thanks, Jarek P. > > Jarek Poplawski wrote: > > On 24-01-2009 23:49, Vegard Nossum wrote: > >> Hi, > >> > >> This survives basic testing here, but I don't know what that counts for > >> when I couldn't reproduce the lockdep report in the first place. Please > >> review. > >> > >> > >> Vegard > >> > >> > >> From cc8bcd1c4fd219a31d6d191aefa4b4b57dadb9b0 Mon Sep 17 00:00:00 2001 > >> From: Vegard Nossum <vegard.nossum@gmail.com> > >> Date: Sat, 24 Jan 2009 22:44:16 +0100 > >> Subject: [PATCH] net: fix setsockopt() locking errors > >> MIME-Version: 1.0 > >> Content-Type: text/plain; charset=utf-8 > >> Content-Transfer-Encoding: 8bit > >> > >> Martin MOKREJ. <mmokrejs@ribosome.natur.cuni.cz> reported: > >>> ======================================================= > >>> [ INFO: possible circular locking dependency detected ] > >>> 2.6.29-rc2-git1 #1 > >>> ------------------------------------------------------- > >>> tcpdump/3734 is trying to acquire lock: > >>> (&mm->mmap_sem){----}, at: [<c1053294>] might_fault+0x30/0x6b > >>> > >>> but task is already holding lock: > >>> (sk_lock-AF_PACKET){--..}, at: [<c12798c8>] sock_setsockopt+0x12b/0x4a4 > >>> > >>> which lock already depends on the new lock. > >> It turns out that sock_setsockopt() is calling copy_from_user() while > >> holding the lock on the socket. > > > > I guess it has been like this for some time, so it would be nice to > > mention what scenario happens here, or IOW what exactly needs to get > > these locks in reverse order. > > > >> We fix it by splitting the ioctl code > >> so that one switch handles the ioctls that have their own code for > >> reading from userspace, and one switch handles the cases that require > >> no additional reading. > >> > >> Reported-by: Martin MOKREJ. <mmokrejs@ribosome.natur.cuni.cz> > >> Signed-off-by: Vegard Nossum <vegard.nossum@gmail.com> > >> --- > >> net/core/sock.c | 134 +++++++++++++++++++++++++++++++++++------------------- > >> 1 files changed, 87 insertions(+), 47 deletions(-) > >> > >> diff --git a/net/core/sock.c b/net/core/sock.c > >> index f3a0d08..6bd618d 100644 > >> --- a/net/core/sock.c > >> +++ b/net/core/sock.c > >> @@ -424,6 +424,80 @@ out: > >> return ret; > >> } > >> > >> +static int sock_linger(struct sock *sk, char __user *optval, int optlen) > > ... > >> +static int sock_set_rcvtimeo(struct sock *sk, char __user *optval, int optlen) > >> +{ > >> + int ret; > >> + long rcvtimeo; > >> + > >> + ret = sock_set_timeout(&rcvtimeo, optval, optlen); > > > > A check for error is needed here and below. > > > [cut] -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, 2009-01-27 at 08:45 +0000, Jarek Poplawski wrote: > On Mon, Jan 26, 2009 at 10:30:30PM +0100, Martin MOKREJŠ wrote: > > The patch really did not help: > > http://bugzilla.kernel.org/show_bug.cgi?id=12515#c5 > > Martin > > Actually, there is a little change: the warning triggerd in another > place (sock_setsockopt() -> sk_attach_filter()). So we could go deeper > with these changes, but I'm not sure this is the right way to fix. > > It looks like the scenario is very old, but probably wasn't reported > (maybe there is some lockdep improvement): Yes, they likely are very old, and yes we added a lockdep annotation to copy_to/from_user() to catch these. > A) sys_mmap2() -> mm->mmap_sem -> packet_mmap() -> sk_lock > B) sock_setsockopt() -> sk_lock -> copy_from_user() -> mm->mmap_sem > > packet_mmap() (net/packet/af_packet.c) seems to be the only place in > net to implement mmap method, and using this lock order btw. On the > other hand copy_from_user() could be more popular under sk_lock, and > I'm not sure these changes are necessary. > > Since I don't know enough neither sock/packet nor sys_mmap, I guess > some advice would be precious. It looks like Peter Zijlstra solved > similar problems in nfs, so I CC him. The NFS/sunrpc case was special in that it did copy_to/from_kernel, that is, it never actually touched user memory -- we taught the might_fault() annotation about that. Can't you simply do the copy_from_user() before you take the sk_lock? -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Jan 27, 2009 at 09:52:49AM +0100, Peter Zijlstra wrote: > On Tue, 2009-01-27 at 08:45 +0000, Jarek Poplawski wrote: > > On Mon, Jan 26, 2009 at 10:30:30PM +0100, Martin MOKREJŠ wrote: > > > The patch really did not help: > > > http://bugzilla.kernel.org/show_bug.cgi?id=12515#c5 > > > Martin > > > > Actually, there is a little change: the warning triggerd in another > > place (sock_setsockopt() -> sk_attach_filter()). So we could go deeper > > with these changes, but I'm not sure this is the right way to fix. > > > > It looks like the scenario is very old, but probably wasn't reported > > (maybe there is some lockdep improvement): > > Yes, they likely are very old, and yes we added a lockdep annotation to > copy_to/from_user() to catch these. > > > A) sys_mmap2() -> mm->mmap_sem -> packet_mmap() -> sk_lock > > B) sock_setsockopt() -> sk_lock -> copy_from_user() -> mm->mmap_sem > > > > packet_mmap() (net/packet/af_packet.c) seems to be the only place in > > net to implement mmap method, and using this lock order btw. On the > > other hand copy_from_user() could be more popular under sk_lock, and > > I'm not sure these changes are necessary. > > > > Since I don't know enough neither sock/packet nor sys_mmap, I guess > > some advice would be precious. It looks like Peter Zijlstra solved > > similar problems in nfs, so I CC him. > > The NFS/sunrpc case was special in that it did copy_to/from_kernel, that > is, it never actually touched user memory -- we taught the might_fault() > annotation about that. > > Can't you simply do the copy_from_user() before you take the sk_lock? > Since it's really needed, and Vegard started doing it like this, I guess he will try to add the missing pieces. Thanks again, Jarek P. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Jan 26, 2009 at 9:33 PM, David Miller <davem@davemloft.net> wrote: > From: Jarek Poplawski <jarkao2@gmail.com> > Date: Mon, 26 Jan 2009 11:50:12 +0000 > >> On 24-01-2009 23:49, Vegard Nossum wrote: >> > +static int sock_linger(struct sock *sk, char __user *optval, int optlen) >> ... >> > +static int sock_set_rcvtimeo(struct sock *sk, char __user *optval, int optlen) >> > +{ >> > + int ret; >> > + long rcvtimeo; >> > + >> > + ret = sock_set_timeout(&rcvtimeo, optval, optlen); >> >> A check for error is needed here and below. > > Right, you cannot continue and update the socket state if this > sock_set_timeout() call returns an error. > Argh, thanks both. What a stupid mistake. What is the point in fixing something if the fix introduces a different kind of error? :-( I will fix the patch and also check for other places (like that attach_filter) that need to be fixed. Vegard
diff --git a/net/core/sock.c b/net/core/sock.c index f3a0d08..6bd618d 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -424,6 +424,80 @@ out: return ret; } +static int sock_linger(struct sock *sk, char __user *optval, int optlen) +{ + struct linger ling; + + if (optlen < sizeof(ling)) + return -EINVAL; /* 1003.1g */ + if (copy_from_user(&ling, optval, sizeof(ling))) + return -EFAULT; + + lock_sock(sk); + + if (!ling.l_onoff) + sock_reset_flag(sk, SOCK_LINGER); + else { +#if (BITS_PER_LONG == 32) + if ((unsigned int) ling.l_linger >= MAX_SCHEDULE_TIMEOUT/HZ) + sk->sk_lingertime = MAX_SCHEDULE_TIMEOUT; + else +#endif + sk->sk_lingertime = (unsigned int) ling.l_linger * HZ; + sock_set_flag(sk, SOCK_LINGER); + } + + release_sock(sk); + + return 0; +} + +static int sock_set_rcvtimeo(struct sock *sk, char __user *optval, int optlen) +{ + int ret; + long rcvtimeo; + + ret = sock_set_timeout(&rcvtimeo, optval, optlen); + + lock_sock(sk); + sk->sk_rcvtimeo = rcvtimeo; + release_sock(sk); + + return ret; +} + +static int sock_set_sndtimeo(struct sock *sk, char __user *optval, int optlen) +{ + int ret; + long sndtimeo; + + ret = sock_set_timeout(&sndtimeo, optval, optlen); + + lock_sock(sk); + sk->sk_sndtimeo = sndtimeo; + release_sock(sk); + + return ret; +} + +static int sock_attach_filter(struct sock *sk, char __user *optval, int optlen) +{ + int ret; + struct sock_fprog fprog; + + if (optlen != sizeof(struct sock_fprog)) + return -EINVAL; + + if (copy_from_user(&fprog, optval, sizeof(fprog))) + return -EFAULT; + + lock_sock(sk); + ret = sk_attach_filter(&fprog, sk); + release_sock(sk); + + return ret; +} + static inline void sock_valbool_flag(struct sock *sk, int bit, int valbool) { if (valbool) @@ -440,18 +514,27 @@ static inline void sock_valbool_flag(struct sock *sk, int bit, int valbool) int sock_setsockopt(struct socket *sock, int level, int optname, char __user *optval, int optlen) { - struct sock *sk=sock->sk; + struct sock *sk = sock->sk; int val; int valbool; - struct linger ling; int ret = 0; /* - * Options without arguments + * Options with special locking requirements */ - if (optname == SO_BINDTODEVICE) + switch (optname) { + case SO_BINDTODEVICE: return sock_bindtodevice(sk, optval, optlen); + case SO_LINGER: + return sock_linger(sk, optval, optlen); + case SO_RCVTIMEO: + return sock_set_rcvtimeo(sk, optval, optlen); + case SO_SNDTIMEO: + return sock_set_sndtimeo(sk, optval, optlen); + case SO_ATTACH_FILTER: + return sock_attach_filter(sk, optval, optlen); + } if (optlen < sizeof(int)) return -EINVAL; @@ -573,28 +656,6 @@ set_rcvbuf: ret = -EPERM; break; - case SO_LINGER: - if (optlen < sizeof(ling)) { - ret = -EINVAL; /* 1003.1g */ - break; - } - if (copy_from_user(&ling,optval,sizeof(ling))) { - ret = -EFAULT; - break; - } - if (!ling.l_onoff) - sock_reset_flag(sk, SOCK_LINGER); - else { -#if (BITS_PER_LONG == 32) - if ((unsigned int)ling.l_linger >= MAX_SCHEDULE_TIMEOUT/HZ) - sk->sk_lingertime = MAX_SCHEDULE_TIMEOUT; - else -#endif - sk->sk_lingertime = (unsigned int)ling.l_linger * HZ; - sock_set_flag(sk, SOCK_LINGER); - } - break; - case SO_BSDCOMPAT: sock_warn_obsolete_bsdism("setsockopt"); break; @@ -627,27 +688,6 @@ set_rcvbuf: sk->sk_rcvlowat = val ? : 1; break; - case SO_RCVTIMEO: - ret = sock_set_timeout(&sk->sk_rcvtimeo, optval, optlen); - break; - - case SO_SNDTIMEO: - ret = sock_set_timeout(&sk->sk_sndtimeo, optval, optlen); - break; - - case SO_ATTACH_FILTER: - ret = -EINVAL; - if (optlen == sizeof(struct sock_fprog)) { - struct sock_fprog fprog; - - ret = -EFAULT; - if (copy_from_user(&fprog, optval, sizeof(fprog))) - break; - - ret = sk_attach_filter(&fprog, sk); - } - break; - case SO_DETACH_FILTER: ret = sk_detach_filter(sk); break;