| Message ID | 20190904201933.10736-11-cyphar@cyphar.com |
|---|---|
| State | Not Applicable |
| Delegated to: | David Miller |
| Headers | show |
| Series | namei: openat2(2) path resolution restrictions | expand |
On Wed, Sep 4, 2019 at 1:23 PM Aleksa Sarai <cyphar@cyphar.com> wrote: > > This patch allows for LOOKUP_BENEATH and LOOKUP_IN_ROOT to safely permit > ".." resolution (in the case of LOOKUP_BENEATH the resolution will still > fail if ".." resolution would resolve a path outside of the root -- > while LOOKUP_IN_ROOT will chroot(2)-style scope it). Magic-link jumps > are still disallowed entirely because now they could result in > inconsistent behaviour if resolution encounters a subsequent ".."[*]. This is the only patch in the series that makes me go "umm". Why is it ok to re-initialize m_seq, which is used by other things too? I think it's because we're out of RCU lookup, but there's no comment about it, and it looks iffy to me. I'd rather have a separate sequence count that doesn't have two users with different lifetime rules. But even apart from that, I think from a "patch continuity" standpoint it would be better to introduce the sequence counts as just an error condition first - iow, not have the "path_is_under()" check, but just return -EXDEV if the sequence number doesn't match. So you'd have three stages: 1) ".." always returns -EXDEV 2) ".." returns -EXDEV if there was a concurrent rename/mount 3) ".." returns -EXDEV if there was a concurrent rename/mount and we reset the sequence numbers and check if you escaped. becasue the sequence number reset really does make me go "hmm", plus I get this nagging little feeling in the back of my head that you can cause nasty O(n^2) lookup cost behavior with deep paths, lots of "..", and repeated path_is_under() calls. So (1) sounds safe. (2) sounds simple. And (3) is where I think subtle things start happening. Also, I'm not 100% convinced that (3) is needed at all. I think the retry could be done in user space instead, which needs to have a fallback anyway. Yes? No? Linus
On Wed, Sep 4, 2019 at 2:09 PM Linus Torvalds <torvalds@linux-foundation.org> wrote: > > So you'd have three stages: > > 1) ".." always returns -EXDEV > > 2) ".." returns -EXDEV if there was a concurrent rename/mount > > 3) ".." returns -EXDEV if there was a concurrent rename/mount and we > reset the sequence numbers and check if you escaped. In fact, I wonder if this should return -EAGAIN instead - to say that "retrying may work". Because then: > Also, I'm not 100% convinced that (3) is needed at all. I think the > retry could be done in user space instead, which needs to have a > fallback anyway. Yes? No? Any user mode fallback would want to know whether it's a final error or whether simply re-trying might make it work again. I think that re-try case is valid for any of the possible "races happened, we can't guarantee that it's safe", and retrying inside the kernel (or doing that re-validation) could have latency issues. Maybe ".." is the only such case. I can't think of any other ones in your series, but at least conceptually they could happen. For example, we've had people who wanted pathname lookup without any IO happening, because if you have to wait for IO you could want to use another thread etc if you're doing some server in user space.. Linus
On Wed, Sep 4, 2019 at 2:35 PM Linus Torvalds <torvalds@linux-foundation.org> wrote: > > On Wed, Sep 4, 2019 at 2:09 PM Linus Torvalds > <torvalds@linux-foundation.org> wrote: > > > > So you'd have three stages: > > > > 1) ".." always returns -EXDEV > > > > 2) ".." returns -EXDEV if there was a concurrent rename/mount > > > > 3) ".." returns -EXDEV if there was a concurrent rename/mount and we > > reset the sequence numbers and check if you escaped. > > In fact, I wonder if this should return -EAGAIN instead - to say that > "retrying may work". And here "this" was meant to be "case 2" - I was moving the quoted text around and didn't fix my wording, so now it is ambiguous or implies #3, which would be crazy. Sorry for the confusion, Linus
On 2019-09-04, Linus Torvalds <torvalds@linux-foundation.org> wrote: > On Wed, Sep 4, 2019 at 1:23 PM Aleksa Sarai <cyphar@cyphar.com> wrote: > > This patch allows for LOOKUP_BENEATH and LOOKUP_IN_ROOT to safely permit > > ".." resolution (in the case of LOOKUP_BENEATH the resolution will still > > fail if ".." resolution would resolve a path outside of the root -- > > while LOOKUP_IN_ROOT will chroot(2)-style scope it). Magic-link jumps > > are still disallowed entirely because now they could result in > > inconsistent behaviour if resolution encounters a subsequent ".."[*]. > > This is the only patch in the series that makes me go "umm". > > Why is it ok to re-initialize m_seq, which is used by other things > too? I think it's because we're out of RCU lookup, but there's no > comment about it, and it looks iffy to me. I'd rather have a separate > sequence count that doesn't have two users with different lifetime > rules. Yeah, the reasoning was that it's because we're out of RCU lookup and if we didn't re-grab ->m_seq we'd hit path_is_under() on every subsequent ".." (even though we've checked that it's safe). But yes, I should've used a different field to avoid confusion (and stop it looking unnecessarily dodgy). I will fix that. > But even apart from that, I think from a "patch continuity" standpoint > it would be better to introduce the sequence counts as just an error > condition first - iow, not have the "path_is_under()" check, but just > return -EXDEV if the sequence number doesn't match. Ack, will do. > So you'd have three stages: > > 1) ".." always returns -EXDEV > > 2) ".." returns -EXDEV if there was a concurrent rename/mount > > 3) ".." returns -EXDEV if there was a concurrent rename/mount and we > reset the sequence numbers and check if you escaped. > > becasue the sequence number reset really does make me go "hmm", plus I > get this nagging little feeling in the back of my head that you can > cause nasty O(n^2) lookup cost behavior with deep paths, lots of "..", > and repeated path_is_under() calls. The reason for doing the concurrent-{rename,mount} checks was to try to avoid the O(n^2) in most cases, but you're right that if you have an attacker that is spamming renames (or you're on a box with a lot of renames and/or mounts going on *anywhere*) you will hit an O(n^2) here (more pedantically, O(m*n) but who's counting?). Unfortunately, I'm not sure what the best solution would be for this one. If -EAGAIN retries are on the table, we could limit how many times we're willing to do path_is_under() and then just return -EAGAIN. > So (1) sounds safe. (2) sounds simple. And (3) is where I think subtle > things start happening. > > Also, I'm not 100% convinced that (3) is needed at all. I think the > retry could be done in user space instead, which needs to have a > fallback anyway. Yes? No? Hinting to userspace to do a retry (with -EAGAIN as you mention in your other mail) wouldn't be a bad thing at all, though you'd almost certainly get quite a few spurious -EAGAINs -- &{mount,rename}_lock are global for the entire machine, after all. But if the only significant roadblock is that (3) seems a bit too hairy, I would be quite happy with landing (2) as a first step (with -EAGAIN).
On Wed, Sep 4, 2019 at 2:49 PM Aleksa Sarai <cyphar@cyphar.com> wrote: > > Hinting to userspace to do a retry (with -EAGAIN as you mention in your > other mail) wouldn't be a bad thing at all, though you'd almost > certainly get quite a few spurious -EAGAINs -- &{mount,rename}_lock are > global for the entire machine, after all. I'd hope that we have some future (possibly very long-term) alternative that is not quite system-global, but yes, right now they are. Which is one reason I'd rather see EAGAIN in user space - yes, it probably makes it even easier to trigger, but it also means that user space might be able to do something about it when it does trigger. For example, maybe user space can first just use an untrusted path as-is, and if it gets EAGAIN or EXDEV, it may be that user space can simplify the path (ie turn "xyz/.../abc" into just "abc". And even if user space doesn't do anything like that, I suspect a performance problem is going to be a whole lot easier to debug and report when somebody ends up seeing excessive retries happening. As a developer you'll see it in profiles or in system call traces, rather than it resulting in very odd possible slowdowns for the kernel. And yeah, it would probably be best to then at least delay doing option 3 indefinitely, just to make sure user space knows about and actually has a test-case for that EAGAIN happening. Linus
Linus Torvalds <torvalds@linux-foundation.org> wrote: > > Hinting to userspace to do a retry (with -EAGAIN as you mention in your > > other mail) wouldn't be a bad thing at all, though you'd almost > > certainly get quite a few spurious -EAGAINs -- &{mount,rename}_lock are > > global for the entire machine, after all. > > I'd hope that we have some future (possibly very long-term) > alternative that is not quite system-global, but yes, right now they > are. It ought to be reasonably easy to make them per-sb at least, I think. We don't allow cross-super rename, right? David
On Wed, Sep 4, 2019 at 3:31 PM David Howells <dhowells@redhat.com> wrote: > > It ought to be reasonably easy to make them per-sb at least, I think. We > don't allow cross-super rename, right? Right now the sequence count handling very much depends on it being a global entity on the reader side, at least. And while the rename sequence count could (and probably should) be per-sb, the same is very much not true of the mount one. So the rename seqcount is likely easier to fix than the mount one, but neither of them are entirely trivial, afaik. Linus
On Wed, Sep 04, 2019 at 03:38:20PM -0700, Linus Torvalds wrote: > On Wed, Sep 4, 2019 at 3:31 PM David Howells <dhowells@redhat.com> wrote: > > > > It ought to be reasonably easy to make them per-sb at least, I think. We > > don't allow cross-super rename, right? > > Right now the sequence count handling very much depends on it being a > global entity on the reader side, at least. > > And while the rename sequence count could (and probably should) be > per-sb, the same is very much not true of the mount one. Huh? That will cost us having to have a per-superblock dentry hash table; recall that lockless lockup can give false negatives if something gets moved from chain to chain, and rename_lock is first and foremost used to catch those and retry. If we split it on per-superblock basis, we can't have dentries from different superblocks in the same chain anymore...
On Wed, Sep 4, 2019 at 4:29 PM Al Viro <viro@zeniv.linux.org.uk> wrote: > > On Wed, Sep 04, 2019 at 03:38:20PM -0700, Linus Torvalds wrote: > > On Wed, Sep 4, 2019 at 3:31 PM David Howells <dhowells@redhat.com> wrote: > > > > > > It ought to be reasonably easy to make them per-sb at least, I think. We > > > don't allow cross-super rename, right? > > > > Right now the sequence count handling very much depends on it being a > > global entity on the reader side, at least. > > > > And while the rename sequence count could (and probably should) be > > per-sb, the same is very much not true of the mount one. > > Huh? That will cost us having to have a per-superblock dentry > hash table; recall that lockless lockup can give false negatives > if something gets moved from chain to chain, and rename_lock is > first and foremost used to catch those and retry. If we split > it on per-superblock basis, we can't have dentries from different > superblocks in the same chain anymore... That's exactly the "very much depends on it being a global entity on the reader side" thing. I'm not convinced that's the _only_ way to handle things. Maybe a combination of (wild handwaving) per-hashqueue sequence count and some clever scheme for pathname handling could work. I've not personally seen a load where the global rename lock has been a problem (very few things really do a lot of renames), but system-wide locks do make me nervous. We have other (and worse) ones. tasklist_lock comes to mind. Linus
diff --git a/fs/namei.c b/fs/namei.c index 0352d275bd13..fd1eb5ce8baa 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -491,7 +491,7 @@ struct nameidata { struct path root; struct inode *inode; /* path.dentry.d_inode */ unsigned int flags; - unsigned seq, m_seq; + unsigned seq, m_seq, r_seq; int last_type; unsigned depth; int total_link_count; @@ -1758,22 +1758,36 @@ static inline int handle_dots(struct nameidata *nd, int type) if (type == LAST_DOTDOT) { int error = 0; - /* - * LOOKUP_BENEATH resolving ".." is not currently safe -- races - * can cause our parent to have moved outside of the root and - * us to skip over it. - */ - if (unlikely(nd->flags & (LOOKUP_BENEATH | LOOKUP_IN_ROOT))) - return -EXDEV; if (!nd->root.mnt) { error = set_root(nd); if (error) return error; } - if (nd->flags & LOOKUP_RCU) { - return follow_dotdot_rcu(nd); - } else - return follow_dotdot(nd); + if (nd->flags & LOOKUP_RCU) + error = follow_dotdot_rcu(nd); + else + error = follow_dotdot(nd); + if (error) + return error; + + if (unlikely(nd->flags & (LOOKUP_BENEATH | LOOKUP_IN_ROOT))) { + bool m_retry = read_seqretry(&mount_lock, nd->m_seq); + bool r_retry = read_seqretry(&rename_lock, nd->r_seq); + + /* + * Don't bother checking unless there's a racing + * rename(2) or MS_MOVE. + */ + if (likely(!m_retry && !r_retry)) + return 0; + + if (m_retry && !(nd->flags & LOOKUP_RCU)) + nd->m_seq = read_seqbegin(&mount_lock); + if (r_retry) + nd->r_seq = read_seqbegin(&rename_lock); + if (!path_is_under(&nd->path, &nd->root)) + return -EXDEV; + } } return 0; } @@ -2245,6 +2259,11 @@ static const char *path_init(struct nameidata *nd, unsigned flags) nd->last_type = LAST_ROOT; /* if there are only slashes... */ nd->flags = flags | LOOKUP_JUMPED | LOOKUP_PARENT; nd->depth = 0; + + nd->m_seq = read_seqbegin(&mount_lock); + if (flags & (LOOKUP_BENEATH | LOOKUP_IN_ROOT)) + nd->r_seq = read_seqbegin(&rename_lock); + if (flags & LOOKUP_ROOT) { struct dentry *root = nd->root.dentry; struct inode *inode = root->d_inode; @@ -2266,8 +2285,6 @@ static const char *path_init(struct nameidata *nd, unsigned flags) nd->path.mnt = NULL; nd->path.dentry = NULL; - nd->m_seq = read_seqbegin(&mount_lock); - /* LOOKUP_IN_ROOT treats absolute paths as being relative-to-dirfd. */ if (flags & LOOKUP_IN_ROOT) while (*s == '/')
This patch allows for LOOKUP_BENEATH and LOOKUP_IN_ROOT to safely permit ".." resolution (in the case of LOOKUP_BENEATH the resolution will still fail if ".." resolution would resolve a path outside of the root -- while LOOKUP_IN_ROOT will chroot(2)-style scope it). Magic-link jumps are still disallowed entirely because now they could result in inconsistent behaviour if resolution encounters a subsequent ".."[*]. The need for this patch is explained by observing there is a fairly easy-to-exploit race condition with chroot(2) (and thus by extension LOOKUP_IN_ROOT and LOOKUP_BENEATH if ".." is allowed) where a rename(2) of a path can be used to "skip over" nd->root and thus escape to the filesystem above nd->root. thread1 [attacker]: for (;;) renameat2(AT_FDCWD, "/a/b/c", AT_FDCWD, "/a/d", RENAME_EXCHANGE); thread2 [victim]: for (;;) openat2(dirb, "b/c/../../etc/shadow", { .flags = O_PATH, .resolve = RESOLVE_IN_ROOT } ); With fairly significant regularity, thread2 will resolve to "/etc/shadow" rather than "/a/b/etc/shadow". There is also a similar (though somewhat more privileged) attack using MS_MOVE. With this patch, such cases will be detected *during* ".." resolution (which is the weak point of chroot(2) -- since walking *into* a subdirectory tautologically cannot result in you walking *outside* nd->root -- except through a bind-mount or magic-link). By detecting this at ".." resolution (rather than checking only at the end of the entire resolution) we can both correct escapes by jumping back to the root (in the case of LOOKUP_IN_ROOT), as well as avoid revealing to attackers the structure of the filesystem outside of the root (through timing attacks for instance). In order to avoid a quadratic lookup with each ".." entry, we only activate the slow path if a write through &rename_lock or &mount_lock has occurred during path resolution (&rename_lock and &mount_lock are re-taken to further optimise the lookup). Since the primary attack being protected against is MS_MOVE or rename(2), not doing additional checks unless a mount or rename have occurred avoids making the common case slow. The use of path_is_under() here might seem suspect, but on further inspection of the most important race (a path was *inside* the root but is now *outside*), there appears to be no attack potential: * If path_is_under() occurs before the rename, then the path will be resolved -- however the path was originally inside the root and thus there is no escape (and to userspace it'd look like the rename occurred after the path was resolved). If path_is_under() occurs afterwards, the resolution is blocked. * Subsequent ".." jumps are guaranteed to check path_is_under() -- by construction, &rename_lock or &mount_lock must have been taken by the attacker after path_is_under() returned in the victim. Thus ".." will not be able to escape from the previously-inside-root path. * Walking down in the moved path is still safe since the entire subtree was moved (either by rename(2) or MS_MOVE) and because (as discussed above) walking down is safe. A variant of the above attack is included in the selftests for openat2(2) later in this patch series. I've run this test on several machines for several days and no instances of a breakout were detected. While this is not concrete proof that this is safe, when combined with the above argument it should lend some trustworthiness to this construction. [*] It may be acceptable in the future to do a path_is_under() check after resolving a magic-link and permit resolution if the nd_jump_link() result is still within the dirfd. However this seems unlikely to be a feature that people *really* need* -- it can be added later if it turns out a lot of people want it. Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Jann Horn <jannh@google.com> Cc: Kees Cook <keescook@chromium.org> Signed-off-by: Aleksa Sarai <cyphar@cyphar.com> --- fs/namei.c | 45 +++++++++++++++++++++++++++++++-------------- 1 file changed, 31 insertions(+), 14 deletions(-)