diff mbox series

[v4] flock.2: add CIFS details

Message ID 20210304095026.782-1-aaptel@suse.com
State New
Headers show
Series [v4] flock.2: add CIFS details | expand

Commit Message

Aurélien Aptel March 4, 2021, 9:50 a.m. UTC
From: Aurelien Aptel <aaptel@suse.com>

Similarly to NFS, CIFS flock() locks behave differently than the
standard. Document those differences.

Here is the rendered text:

CIFS details
  In  Linux kernels up to 5.4, flock() is not propagated over SMB. A file
  with such locks will not appear locked for remote clients.

  Since Linux 5.5, flock() locks are emulated with SMB  byte-range  locks
  on  the  entire  file.  Similarly  to NFS, this means that fcntl(2) and
  flock() locks interact with one another. Another important  side-effect
  is  that  the  locks are not advisory anymore: a write on a locked file
  will always fail with EACCES.   This difference originates from the de-
  sign of locks in the SMB protocol, which provides mandatory locking se-
  mantics. The nobrl mount option (see mount.cifs(8)) turns off  fnctl(2)
  and  flock() lock propagation to remote clients and makes flock() locks
  advisory again.

Signed-off-by: Aurelien Aptel <aaptel@suse.com>
Reviewed-By: Tom Talpey <tom@talpey.com>
---
 man2/flock.2 | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

Comments

Alejandro Colomar \(man-pages\) March 10, 2021, 7:07 p.m. UTC | #1
On 3/4/21 10:50 AM, Aurélien Aptel wrote:
> From: Aurelien Aptel <aaptel@suse.com>
> 
> Similarly to NFS, CIFS flock() locks behave differently than the
> standard. Document those differences.
> 
> Here is the rendered text:
> 
> CIFS details
>    In  Linux kernels up to 5.4, flock() is not propagated over SMB. A file
>    with such locks will not appear locked for remote clients.
> 
>    Since Linux 5.5, flock() locks are emulated with SMB  byte-range  locks
>    on  the  entire  file.  Similarly  to NFS, this means that fcntl(2) and
>    flock() locks interact with one another. Another important  side-effect
>    is  that  the  locks are not advisory anymore: a write on a locked file
>    will always fail with EACCES.   This difference originates from the de-
>    sign of locks in the SMB protocol, which provides mandatory locking se-
>    mantics. The nobrl mount option (see mount.cifs(8)) turns off  fnctl(2)
>    and  flock() lock propagation to remote clients and makes flock() locks
>    advisory again.
> 
> Signed-off-by: Aurelien Aptel <aaptel@suse.com>
> Reviewed-By: Tom Talpey <tom@talpey.com>
> ---
[...]
> +This difference originates from the design of locks in the SMB
> +protocol, which provides mandatory locking semantics. The
> +.I nobrl

I agree with Tom.  It's much easier to read if you just say that 'nobrl' 
torns off the non-locale behaviour, and acts as 5.4 and earlier kernels. 
  Unless there's any subtlety that makes it different.  Is there any?

BTW, you should use "semantic newlines":

$ man 7 man-pages |sed -n '/semantic newlines/,/^$/p'
    Use semantic newlines
        In the source of a manual page,  new  sentences  should  be
        started  on new lines, and long sentences should split into
        lines at clause breaks (commas, semicolons, colons, and  so
        on).   This  convention,  sometimes known as "semantic new-
        lines", makes it easier to see the effect of patches, which
        often  operate at the level of individual sentences or sen-
        tence clauses.


Thanks,

Alex

> +mount option (see
> +.BR mount.cifs (8))
> +turns off
> +.BR fnctl (2)
> +and
> +.BR flock ()
> +lock propagation to remote clients and makes
> +.BR flock ()
> +locks advisory again.
>   .SH SEE ALSO
>   .BR flock (1),
>   .BR close (2),
>
Aurélien Aptel March 11, 2021, 10:11 a.m. UTC | #2
"Alejandro Colomar (man-pages)" <alx.manpages@gmail.com> writes:
> I agree with Tom.  It's much easier to read if you just say that 'nobrl' 
> torns off the non-locale behaviour, and acts as 5.4 and earlier kernels. 
>   Unless there's any subtlety that makes it different.  Is there any?

nobrl also makes fnctl() locks local.
In 5.4 and earlier kernel, flock() is local but fnctl() isn't.

> BTW, you should use "semantic newlines":

Ok, I'll redo once we agree on the text.

Cheers,
Tom Talpey March 11, 2021, 4:21 p.m. UTC | #3
On 3/11/2021 5:11 AM, Aurélien Aptel wrote:
> "Alejandro Colomar (man-pages)" <alx.manpages@gmail.com> writes:
>> I agree with Tom.  It's much easier to read if you just say that 'nobrl'
>> torns off the non-locale behaviour, and acts as 5.4 and earlier kernels.
>>    Unless there's any subtlety that makes it different.  Is there any?
> 
> nobrl also makes fnctl() locks local.
> In 5.4 and earlier kernel, flock() is local but fnctl() isn't.
> 
>> BTW, you should use "semantic newlines":
> 
> Ok, I'll redo once we agree on the text.

I wonder if it's best to leave the nobrl details to the mount.cifs
manpage, and just make a reference from here.

Another advantage of putting this in a cifs.ko-specific manpage
is that it would be significantly easier to maintain. The details
of a 5.4-to-5.5 transition are going to fade over time, and the
APIs in fcntl(2)/flock(2) really aren't driving that.

If not, it's going to be messy... Aurélien is this correct?

cifs.ko flock()
- local in <= 5.4
- remote by default in >= 5.5
- local if nobrl in >= 5.5

cifs.ko fcntl()
- remote by default in X.Y
- local if nobrl in X.Y

Not sure what the value(s) of X.Y actually might be.

It seems odd that "nobrl" means "handle locking locally, and never
send to server". I mean, there is always byte-range locking, right?

Are there any other options or configurations that alter this?

Tom.
Aurélien Aptel March 11, 2021, 5:13 p.m. UTC | #4
Tom Talpey <tom@talpey.com> writes:

> On 3/11/2021 5:11 AM, Aurélien Aptel wrote:
>> "Alejandro Colomar (man-pages)" <alx.manpages@gmail.com> writes:
>>> I agree with Tom.  It's much easier to read if you just say that 'nobrl'
>>> torns off the non-locale behaviour, and acts as 5.4 and earlier kernels.
>>>    Unless there's any subtlety that makes it different.  Is there any?
>> 
>> nobrl also makes fnctl() locks local.
>> In 5.4 and earlier kernel, flock() is local but fnctl() isn't.
>> 
>>> BTW, you should use "semantic newlines":
>> 
>> Ok, I'll redo once we agree on the text.
>
> I wonder if it's best to leave the nobrl details to the mount.cifs
> manpage, and just make a reference from here.
>
> Another advantage of putting this in a cifs.ko-specific manpage
> is that it would be significantly easier to maintain. The details
> of a 5.4-to-5.5 transition are going to fade over time, and the
> APIs in fcntl(2)/flock(2) really aren't driving that.

I was trying to write in the same style as the NFS details just above (see
existing man page). Give basic overview of the issues.

> If not, it's going to be messy... Aurélien is this correct?
>
> cifs.ko flock()
> - local in <= 5.4
> - remote by default in >= 5.5
> - local if nobrl in >= 5.5
>
> cifs.ko fcntl()
> - remote by default in X.Y
> - local if nobrl in X.Y

Correct.

> Not sure what the value(s) of X.Y actually might be.

AFAIK fcntl() was always remote by default.
And nobrl was added in 2.6.15 (15 years ago). I wouldn't bother
mentionning X.Y, it's already complex enough as it is.

> It seems odd that "nobrl" means "handle locking locally, and never
> send to server". I mean, there is always byte-range locking, right?

Yes the option name can be confusing. Byte-range locking is always
possible, but with "nobrl" it's local-only.

> Are there any other options or configurations that alter this?

I've taken another long look at the cifs.ko and samba code. There are
many knobs that would make an accurate matrix table pretty big.

* If the mount point is done on an SMB1+UNIX Extensions
  connection, locking becomes advisory. Unless
  forcemandatorylock option is passed. This will eventually be
  implemented for SMB3 posix extensions as well (I've started a
  thread on the samba-technical mailing list).
* If cifs.ko can get guarantees (via oplocks or leases) that it is the
  only user of a file, it caches read/writes but also locking
  locally. If the lease is broke then it will send the locks.
  The levels of caching cifs.ko can do can be changed with the cache
  mount option.

Cheers,
Tom Talpey March 11, 2021, 5:29 p.m. UTC | #5
On 3/11/2021 12:13 PM, Aurélien Aptel wrote:
> Tom Talpey <tom@talpey.com> writes:
> 
>> On 3/11/2021 5:11 AM, Aurélien Aptel wrote:
>>> "Alejandro Colomar (man-pages)" <alx.manpages@gmail.com> writes:
>>>> I agree with Tom.  It's much easier to read if you just say that 'nobrl'
>>>> torns off the non-locale behaviour, and acts as 5.4 and earlier kernels.
>>>>     Unless there's any subtlety that makes it different.  Is there any?
>>>
>>> nobrl also makes fnctl() locks local.
>>> In 5.4 and earlier kernel, flock() is local but fnctl() isn't.
>>>
>>>> BTW, you should use "semantic newlines":
>>>
>>> Ok, I'll redo once we agree on the text.
>>
>> I wonder if it's best to leave the nobrl details to the mount.cifs
>> manpage, and just make a reference from here.
>>
>> Another advantage of putting this in a cifs.ko-specific manpage
>> is that it would be significantly easier to maintain. The details
>> of a 5.4-to-5.5 transition are going to fade over time, and the
>> APIs in fcntl(2)/flock(2) really aren't driving that.
> 
> I was trying to write in the same style as the NFS details just above (see
> existing man page). Give basic overview of the issues.
> 
>> If not, it's going to be messy... Aurélien is this correct?
>>
>> cifs.ko flock()
>> - local in <= 5.4
>> - remote by default in >= 5.5
>> - local if nobrl in >= 5.5
>>
>> cifs.ko fcntl()
>> - remote by default in X.Y
>> - local if nobrl in X.Y
> 
> Correct.
> 
>> Not sure what the value(s) of X.Y actually might be.
> 
> AFAIK fcntl() was always remote by default.
> And nobrl was added in 2.6.15 (15 years ago). I wouldn't bother
> mentionning X.Y, it's already complex enough as it is.
> 
>> It seems odd that "nobrl" means "handle locking locally, and never
>> send to server". I mean, there is always byte-range locking, right?
> 
> Yes the option name can be confusing. Byte-range locking is always
> possible, but with "nobrl" it's local-only.
> 
>> Are there any other options or configurations that alter this?
> 
> I've taken another long look at the cifs.ko and samba code. There are
> many knobs that would make an accurate matrix table pretty big.

I vote for simplicity! Especially on the fcntl(2) page in question.
Totally agree on not mentioning 2.6.x in a current manpage.

> * If the mount point is done on an SMB1+UNIX Extensions
>    connection, locking becomes advisory. Unless
>    forcemandatorylock option is passed. This will eventually be
>    implemented for SMB3 posix extensions as well (I've started a
>    thread on the samba-technical mailing list).

NO SMB1 discussion! Any manpage update should not add discussion of
an obsolete nonsecure deprecated protocol, and should definitely not
passively encourage its consideration.

> * If cifs.ko can get guarantees (via oplocks or leases) that it is the
>    only user of a file, it caches read/writes but also locking
>    locally. If the lease is broke then it will send the locks.
>    The levels of caching cifs.ko can do can be changed with the cache
>    mount option.

I think this is relevant to some sort of smb(7) manpage, but is much
too detailed and subtle for a paragraph summary in fcntl(2).

To be more clear, my updated thinking is to mostly drop the details
in the closing sentence:

> The nobrl mount option (see mount.cifs(8)) turns off  fnctl(2)
>   and  flock() lock propagation to remote clients and makes flock() locks
>   advisory again.

and simply state (perhaps)

  "Remote and mandatory locking semantics may vary with SMB protocol,
   mount options and server type. See mount.cifs(8) for additional
   information."

Tom.
Aurélien Aptel March 11, 2021, 5:45 p.m. UTC | #6
Tom Talpey <tom@talpey.com> writes:
> and simply state (perhaps)
>
>   "Remote and mandatory locking semantics may vary with SMB protocol,
>    mount options and server type. See mount.cifs(8) for additional
>    information."

This would be the complete addition to the man page? I feel like we
should at least say it is *likely* that:
- locks will be mandatory
- flock() is emulated via fnctl() and so they interact with each other

Which are the 2 aspects that really diverges from the expected behaviour
of flock() and likely to hit people in the wild. Mentionning this will
send people trying to debug their app in the right direction.

Cheers,
Matthew Wilcox March 11, 2021, 6:01 p.m. UTC | #7
On Thu, Mar 04, 2021 at 10:50:26AM +0100, Aurélien Aptel wrote:
>   mantics. The nobrl mount option (see mount.cifs(8)) turns off  fnctl(2)

s/fnctl/fcntl/
Tom Talpey March 11, 2021, 8:42 p.m. UTC | #8
On 3/11/2021 12:45 PM, Aurélien Aptel wrote:
> Tom Talpey <tom@talpey.com> writes:
>> and simply state (perhaps)
>>
>>    "Remote and mandatory locking semantics may vary with SMB protocol,
>>     mount options and server type. See mount.cifs(8) for additional
>>     information."
> 
> This would be the complete addition to the man page? I feel like we

Only replacing the last sentence, which I quoted earlier.

> should at least say it is *likely* that:
> - locks will be mandatory
> - flock() is emulated via fnctl() and so they interact with each other
> 
> Which are the 2 aspects that really diverges from the expected behaviour
> of flock() and likely to hit people in the wild. Mentionning this will
> send people trying to debug their app in the right direction.

Ok, and agreed. SMB lock semantics are certainly important to describe.

Tom.
Aurélien Aptel March 11, 2021, 10:39 p.m. UTC | #9
Ok, then I agree with your last paragraph. Here's the current version, with semantic newlines:

 In Linux kernels up to 5.4, flock() is not propagated over SMB.
 A file with such locks will not appear locked for remote clients.
 
 Since Linux 5.5, flock() locks are emulated with SMB byte-range locks on the entire file.
 Similarly to NFS, this means that fcntl(2) and flock() locks interact with one another.
 Another important side-effect is that the locks are not advisory anymore:
 a write on a locked file will always fail with EACCES.
 This difference originates from the design of locks in the SMB protocol, which provides mandatory locking semantics.
 
 Remote and mandatory locking semantics may vary with SMB protocol, mount options and server type.
 See mount.cifs(8) for additional information.

Cheers,
Pavel Shilovsky March 15, 2021, 6:05 p.m. UTC | #10
чт, 11 мар. 2021 г. в 14:41, Aurélien Aptel <aaptel@suse.com>:


Hi Aurelien,

>
> Ok, then I agree with your last paragraph. Here's the current version, with semantic newlines:
>
>  In Linux kernels up to 5.4, flock() is not propagated over SMB.
>  A file with such locks will not appear locked for remote clients.
>
>  Since Linux 5.5, flock() locks are emulated with SMB byte-range locks on the entire file.
>  Similarly to NFS, this means that fcntl(2) and flock() locks interact with one another.
>  Another important side-effect is that the locks are not advisory anymore:
>  a write on a locked file will always fail with EACCES.

It is not only about writing to a locked file. It is also about any IO
against a locked file if such a file is locked through another file
handle. Right?

--
Best regards,
Pavel Shilovsky
Aurélien Aptel March 16, 2021, 10:42 a.m. UTC | #11
Pavel Shilovsky <piastryyy@gmail.com> writes:
> It is not only about writing to a locked file. It is also about any IO
> against a locked file if such a file is locked through another file
> handle. Right?

Yes that was implied, the write was a simple example to illustrate. I'll
update to make it more generic:

  Another important side-effect is that the locks are not advisory anymore:
  any IO on a locked file will always fail with EACCES,
  even when done from a separate file descriptor.

If you have comments please provide direct text suggestion to save time.

Cheers,
Pavel Shilovsky March 16, 2021, 5:39 p.m. UTC | #12
Sure. Thanks!

I would put more details from
https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-lockfileex
:

"""
  Another important side-effect is that the locks are not advisory anymore:
  any IO on an exclusively locked file will always fail with EACCES
  when done from a separate file descriptor; write calls on
  a file locked for shared access will fail with EACCES when done
  from any file descriptor including the one used to lock the file.
"""

Thoughts?

--
Best regards,
Pavel Shilovsky

вт, 16 мар. 2021 г. в 03:42, Aurélien Aptel <aaptel@suse.com>:
>
> Pavel Shilovsky <piastryyy@gmail.com> writes:
> > It is not only about writing to a locked file. It is also about any IO
> > against a locked file if such a file is locked through another file
> > handle. Right?
>
> Yes that was implied, the write was a simple example to illustrate. I'll
> update to make it more generic:
>
>   Another important side-effect is that the locks are not advisory anymore:
>   any IO on a locked file will always fail with EACCES,
>   even when done from a separate file descriptor.
>
> If you have comments please provide direct text suggestion to save time.
>
> Cheers,
> --
> Aurélien Aptel / SUSE Labs Samba Team
> GPG: 1839 CB5F 9F5B FB9B AA97  8C99 03C8 A49B 521B D5D3
> SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg, DE
> GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 247165 (AG München)
>
Tom Talpey March 16, 2021, 7:42 p.m. UTC | #13
On 3/16/2021 1:39 PM, Pavel Shilovsky wrote:
> Sure. Thanks!
> 
> I would put more details from
> https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-lockfileex
> :
> 
> """
>    Another important side-effect is that the locks are not advisory anymore:
>    any IO on an exclusively locked file will always fail with EACCES
>    when done from a separate file descriptor; write calls on
>    a file locked for shared access will fail with EACCES when done
>    from any file descriptor including the one used to lock the file.
> """
> 
> Thoughts?

I think it'll be important to define what "exclusive" and "shared"
mean from a Linux/POSIX API perspective, and that will get into dragon
territory. I don't think it's a good idea to attempt that in this
manpage. It is best to leave Windows semantics, and interop with
Windows clients, out of it.

IOW, I personally prefer Aurélien's simple version for now.

Tom.

> 
> --
> Best regards,
> Pavel Shilovsky
> 
> вт, 16 мар. 2021 г. в 03:42, Aurélien Aptel <aaptel@suse.com>:
>>
>> Pavel Shilovsky <piastryyy@gmail.com> writes:
>>> It is not only about writing to a locked file. It is also about any IO
>>> against a locked file if such a file is locked through another file
>>> handle. Right?
>>
>> Yes that was implied, the write was a simple example to illustrate. I'll
>> update to make it more generic:
>>
>>    Another important side-effect is that the locks are not advisory anymore:
>>    any IO on a locked file will always fail with EACCES,
>>    even when done from a separate file descriptor.
>>
>> If you have comments please provide direct text suggestion to save time.
>>
>> Cheers,
>> --
>> Aurélien Aptel / SUSE Labs Samba Team
>> GPG: 1839 CB5F 9F5B FB9B AA97  8C99 03C8 A49B 521B D5D3
>> SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg, DE
>> GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 247165 (AG München)
>>
>
Pavel Shilovsky March 16, 2021, 11:04 p.m. UTC | #14
Make sense to make it simpler. Then I would just propose a minor fix -
to remove "even" on the last line because IO from the same file
descriptor is allowed.

"""
  Another important side-effect is that the locks are not advisory anymore:
  any IO on a locked file will always fail with EACCES,
  when done from a separate file descriptor.
"""
--
Best regards,
Pavel Shilovsky

вт, 16 мар. 2021 г. в 12:42, Tom Talpey <tom@talpey.com>:
>
> On 3/16/2021 1:39 PM, Pavel Shilovsky wrote:
> > Sure. Thanks!
> >
> > I would put more details from
> > https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-lockfileex
> > :
> >
> > """
> >    Another important side-effect is that the locks are not advisory anymore:
> >    any IO on an exclusively locked file will always fail with EACCES
> >    when done from a separate file descriptor; write calls on
> >    a file locked for shared access will fail with EACCES when done
> >    from any file descriptor including the one used to lock the file.
> > """
> >
> > Thoughts?
>
> I think it'll be important to define what "exclusive" and "shared"
> mean from a Linux/POSIX API perspective, and that will get into dragon
> territory. I don't think it's a good idea to attempt that in this
> manpage. It is best to leave Windows semantics, and interop with
> Windows clients, out of it.
>
> IOW, I personally prefer Aurélien's simple version for now.
>
> Tom.
>
> >
> > --
> > Best regards,
> > Pavel Shilovsky
> >
> > вт, 16 мар. 2021 г. в 03:42, Aurélien Aptel <aaptel@suse.com>:
> >>
> >> Pavel Shilovsky <piastryyy@gmail.com> writes:
> >>> It is not only about writing to a locked file. It is also about any IO
> >>> against a locked file if such a file is locked through another file
> >>> handle. Right?
> >>
> >> Yes that was implied, the write was a simple example to illustrate. I'll
> >> update to make it more generic:
> >>
> >>    Another important side-effect is that the locks are not advisory anymore:
> >>    any IO on a locked file will always fail with EACCES,
> >>    even when done from a separate file descriptor.
> >>
> >> If you have comments please provide direct text suggestion to save time.
> >>
> >> Cheers,
> >> --
> >> Aurélien Aptel / SUSE Labs Samba Team
> >> GPG: 1839 CB5F 9F5B FB9B AA97  8C99 03C8 A49B 521B D5D3
> >> SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg, DE
> >> GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 247165 (AG München)
> >>
> >
diff mbox series

Patch

diff --git a/man2/flock.2 b/man2/flock.2
index 61d4b5396..d447eac9c 100644
--- a/man2/flock.2
+++ b/man2/flock.2
@@ -239,6 +239,35 @@  see the discussion of the
 .I "local_lock"
 option in
 .BR nfs (5).
+.SS CIFS details
+In Linux kernels up to 5.4,
+.BR flock ()
+is not propagated over SMB. A file with such locks will not appear
+locked for remote clients.
+.PP
+Since Linux 5.5,
+.BR flock ()
+locks are emulated with SMB byte-range locks on the entire
+file. Similarly to NFS, this means that
+.BR fcntl (2)
+and
+.BR flock ()
+locks interact with one another. Another important side-effect is that
+the locks are not advisory anymore: a write on a locked file will
+always fail with
+.BR EACCES .
+This difference originates from the design of locks in the SMB
+protocol, which provides mandatory locking semantics. The
+.I nobrl
+mount option (see
+.BR mount.cifs (8))
+turns off
+.BR fnctl (2)
+and
+.BR flock ()
+lock propagation to remote clients and makes
+.BR flock ()
+locks advisory again.
 .SH SEE ALSO
 .BR flock (1),
 .BR close (2),