smb2quota.py: Userspace helper to display quota information for the Linux SMB client file system (CIFS)

Signed-off-by: Kenneth D'souza <kdsouza@redhat.com>
Signed-off-by: Ronnie Sahlberg <lsahlber@redhat.com>
---
 smb2quota.py | 172 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 172 insertions(+)
 create mode 100755 smb2quota.py

Merged. thanks.
--
Best regards,
Pavel Shilovsky

пн, 23 сент. 2019 г. в 21:56, Kenneth D'souza <kdsouza@redhat.com>:
>
> Signed-off-by: Kenneth D'souza <kdsouza@redhat.com>
> Signed-off-by: Ronnie Sahlberg <lsahlber@redhat.com>
> ---
>  smb2quota.py | 172 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 172 insertions(+)
>  create mode 100755 smb2quota.py
>
> diff --git a/smb2quota.py b/smb2quota.py
> new file mode 100755
> index 0000000..11d98db
> --- /dev/null
> +++ b/smb2quota.py
> @@ -0,0 +1,172 @@
> +#!/usr/bin/env python
> +# coding: utf-8
> +#
> +# smb2quota is a cmdline tool to display quota information for the
> +# Linux SMB client file system (CIFS)
> +#
> +# Copyright (C) Ronnie Sahlberg (lsahlberg@redhat.com) 2019
> +# Copyright (C) Kenneth D'souza (kdsouza@redhat.com) 2019
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 2 of the License, or
> +# (at your option) any later version.
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write to the Free Software
> +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
> +
> +
> +import array
> +import fcntl
> +import os
> +import struct
> +import sys
> +import argparse
> +
> +CIFS_QUERY_INFO = 0xc018cf07
> +BBOLD = '\033[1;30;47m'   # Bold black text with white background.
> +ENDC = '\033[m'   # Rest to defaults
> +
> +
> +def usage():
> +    print("Usage: %s [-h] <options>  <filename>" % sys.argv[0])
> +    print("Try 'smb2quota -h' for more information")
> +    sys.exit()
> +
> +
> +class SID:
> +    def __init__(self, buf):
> +        self.sub_authority_count = buf[1]
> +        self.buffer = buf[:8 + self.sub_authority_count * 4]
> +        self.revision = self.buffer[0]
> +        if self.revision != 1:
> +            raise ValueError('SID Revision %d not supported' % self.revision)
> +        self.identifier_authority = 0
> +        for x in self.buffer[2:8]:
> +            self.identifier_authority = self.identifier_authority * 256 + x
> +        self.sub_authority = []
> +        for i in range(self.sub_authority_count):
> +            self.sub_authority.append(struct.unpack_from('<I', self.buffer, 8 + 4 * i)[0])
> +
> +    def __str__(self):
> +        s = "S-%u-%u" % (self.revision, self.identifier_authority)
> +        for x in self.sub_authority:
> +            s += '-%u' % x
> +        return s
> +
> +
> +def convert(num):  # Convert bytes to closest human readable UNIT.
> +    for unit in ['', 'kiB', 'MiB', 'GiB', 'TiB', 'PiB', 'EiB']:
> +        if abs(num) < 1024.0:
> +            return "%3.1f %s" % (num, unit)
> +        num /= 1024.0
> +
> +
> +class QuotaEntry:
> +    def __init__(self, buf, flag):
> +        sl = struct.unpack_from('<I', buf, 4)[0]
> +        self.sid = SID(buf[40:40 + sl])
> +        self.used = struct.unpack_from('<Q', buf, 16)[0]
> +        self.thr = struct.unpack_from('<Q', buf, 24)[0]
> +        self.lim = struct.unpack_from('<Q', buf, 32)[0]
> +        self.a = (convert(self.used))
> +        self.b = (convert(self.thr))
> +        self.c = (convert(self.lim))
> +        self.flag = flag
> +
> +    def __str__(self):
> +        if self.flag == 0:
> +            s = " %-11s | %-11s | %-13s | %s" % (self.a, self.c, self.b, self.sid)
> +        elif self.flag == 1:
> +            s = "%s,%d,%d,%d" % (self.sid, self.used, self.lim, self.thr)
> +        else:
> +            s = 'SID:%s\n' % self.sid
> +            s += 'Quota Used:%d\n' % self.used
> +            if self.thr == 0xffffffffffffffff:
> +                s += 'Quota Threshold Limit:NO_WARNING_THRESHOLD\n'
> +            else:
> +                s += 'Quota Threshold Limit:%d\n' % self.thr
> +            if self.lim == 0xffffffffffffffff:
> +                s += 'Quota Limit:NO_LIMIT\n'
> +            else:
> +                s += 'Quota Limit:%d\n' % self.lim
> +        return s
> +
> +
> +class Quota:
> +    def __init__(self, buf, flag):
> +        self.quota = []
> +        s = struct.unpack_from('<I', buf, 0)[0]
> +        while s:
> +            qe = QuotaEntry(buf[0:s], flag)
> +            self.quota.append(qe)
> +            buf = buf[s:]
> +            a = s
> +            s = struct.unpack_from('<I', buf, 0)[0]
> +            if s == 0:
> +                s = a   # Use the last value of s and process it.
> +                qe = QuotaEntry(buf[0:s], flag)
> +                self.quota.append(qe)
> +                break
> +
> +    def __str__(self):
> +        s = ''
> +        for q in self.quota:
> +            s += '%s\n' % q
> +        return s
> +
> +
> +def parser_check(path, flag):
> +    titleused = "Amount Used"
> +    titlelim = "Quota Limit"
> +    titlethr = "Warning Level"
> +    titlesid = "SID"
> +    buf = array.array('B', [0] * 16384)
> +    struct.pack_into('<I', buf, 0, 4)  # InfoType: Quota
> +    struct.pack_into('<I', buf, 16, 16384)  # InputBufferLength
> +    struct.pack_into('<I', buf, 20, 16)  # OutputBufferLength
> +    struct.pack_into('b', buf, 24, 0)  # return single
> +    struct.pack_into('b', buf, 25, 1)  # return single
> +    try:
> +        f = os.open(path, os.O_RDONLY)
> +        fcntl.ioctl(f, CIFS_QUERY_INFO, buf, 1)
> +        os.close(f)
> +        if flag == 0:
> +            print(BBOLD + " %-7s | %-7s | %-7s | %s " + ENDC) % (titleused, titlelim, titlethr, titlesid)
> +        q = Quota(buf[24:24 + struct.unpack_from('<I', buf, 16)[0]], flag)
> +        print(q)
> +    except IOError as reason:
> +        print("ioctl failed: %s" % reason)
> +    except OSError as reason:
> +        print("ioctl failed: %s" % reason)
> +
> +
> +def main():
> +    if len(sys.argv) < 2:
> +        usage()
> +
> +    parser = argparse.ArgumentParser(description="Please specify an action to perform.", prog="smb2quota")
> +    parser.add_argument("-tabular", "-t", metavar="", help="Print quota information in tabular format")
> +    parser.add_argument("-csv", "-c", metavar="", help="Print quota information in csv format")
> +    parser.add_argument("-list", "-l", metavar="", help="Print quota information in list format")
> +    args = parser.parse_args()
> +
> +    if args.tabular:
> +        path = args.tabular
> +        parser_check(path, 0)
> +
> +    if args.csv:
> +        path = args.csv
> +        parser_check(path, 1)
> +
> +    if args.list:
> +        path = args.list
> +        parser_check(path, 2)
> +
> +
> +if __name__ == "__main__":
> +    main()
> --
> 2.21.0
>

We now have 2 user scripts in the repo but the configure script doesn't
know about them: they are not installed. If they are considered dev
tools or experimental that's fine otherwise we should consider
installing them on make install. I just wanted to point this out.

A bit late but some comments on the script.

Kenneth D'souza <kdsouza@redhat.com> writes:
> +#!/usr/bin/env python

We should script for python3 at this point I think. Python2 is on the
way out.

> +def usage():
> +    print("Usage: %s [-h] <options>  <filename>" % sys.argv[0])
> +    print("Try 'smb2quota -h' for more information")
> +    sys.exit()

argparse already generates usage message from its conf.

> +
> +def parser_check(path, flag):
> +    titleused = "Amount Used"
> +    titlelim = "Quota Limit"
> +    titlethr = "Warning Level"
> +    titlesid = "SID"
> +    buf = array.array('B', [0] * 16384)
> +    struct.pack_into('<I', buf, 0, 4)  # InfoType: Quota
> +    struct.pack_into('<I', buf, 16, 16384)  # InputBufferLength
> +    struct.pack_into('<I', buf, 20, 16)  # OutputBufferLength
> +    struct.pack_into('b', buf, 24, 0)  # return single
> +    struct.pack_into('b', buf, 25, 1)  # return single
> +    try:
> +        f = os.open(path, os.O_RDONLY)
> +        fcntl.ioctl(f, CIFS_QUERY_INFO, buf, 1)
> +        os.close(f)
> +        if flag == 0:
> +            print(BBOLD + " %-7s | %-7s | %-7s | %s " + ENDC) % (titleused, titlelim, titlethr, titlesid)
> +        q = Quota(buf[24:24 + struct.unpack_from('<I', buf, 16)[0]], flag)
> +        print(q)
> +    except IOError as reason:
> +        print("ioctl failed: %s" % reason)
> +    except OSError as reason:
> +        print("ioctl failed: %s" % reason)
> +
> +
> +def main():
> +    if len(sys.argv) < 2:
> +        usage()
> +
> +    parser = argparse.ArgumentParser(description="Please specify an action to perform.", prog="smb2quota")

description is used to generate a useful help/usage message. Maybe use
"tool to display quota information for the Linux SMB client file system (CIFS)"

> +    parser.add_argument("-tabular", "-t", metavar="", help="Print quota information in tabular format")
> +    parser.add_argument("-csv", "-c", metavar="", help="Print quota information in csv format")
> +    parser.add_argument("-list", "-l", metavar="", help="Print quota information in list format")

* I think we should use 2 dashes (i.e. "--tabular") for long options.
* For boolean flags, action="store_true" is what you want
* You can put let argparse know about the path arg, and then use args.path

    parser.add_argument("--tabular", "-t", action="store_true", help="Print quota information in tabular format")
    parser.add_argument("--csv", "-c", action="store_true", help="Print quota information in csv format")
    parser.add_argument("--list", "-l", action="store_true", help="Print quota information in list format")
    parser.add_argument("path", help="Path to query")

Cheers,

Can you try the new updated patch?
Some of the suggested changes have been incorporated.
Pavel picked up the new changes from my github repo.
https://github.com/piastry/cifs-utils/commits/next

I will send a follow-on patch to work on the remaining changes.

On Wed, Oct 9, 2019 at 6:43 PM Aurélien Aptel <aaptel@suse.com> wrote:
>
>
> We now have 2 user scripts in the repo but the configure script doesn't
> know about them: they are not installed. If they are considered dev
> tools or experimental that's fine otherwise we should consider
> installing them on make install. I just wanted to point this out.
>
> A bit late but some comments on the script.
>
> Kenneth D'souza <kdsouza@redhat.com> writes:
> > +#!/usr/bin/env python
>
> We should script for python3 at this point I think. Python2 is on the
> way out.
>
> > +def usage():
> > +    print("Usage: %s [-h] <options>  <filename>" % sys.argv[0])
> > +    print("Try 'smb2quota -h' for more information")
> > +    sys.exit()
>
> argparse already generates usage message from its conf.
>
> > +
> > +def parser_check(path, flag):
> > +    titleused = "Amount Used"
> > +    titlelim = "Quota Limit"
> > +    titlethr = "Warning Level"
> > +    titlesid = "SID"
> > +    buf = array.array('B', [0] * 16384)
> > +    struct.pack_into('<I', buf, 0, 4)  # InfoType: Quota
> > +    struct.pack_into('<I', buf, 16, 16384)  # InputBufferLength
> > +    struct.pack_into('<I', buf, 20, 16)  # OutputBufferLength
> > +    struct.pack_into('b', buf, 24, 0)  # return single
> > +    struct.pack_into('b', buf, 25, 1)  # return single
> > +    try:
> > +        f = os.open(path, os.O_RDONLY)
> > +        fcntl.ioctl(f, CIFS_QUERY_INFO, buf, 1)
> > +        os.close(f)
> > +        if flag == 0:
> > +            print(BBOLD + " %-7s | %-7s | %-7s | %s " + ENDC) % (titleused, titlelim, titlethr, titlesid)
> > +        q = Quota(buf[24:24 + struct.unpack_from('<I', buf, 16)[0]], flag)
> > +        print(q)
> > +    except IOError as reason:
> > +        print("ioctl failed: %s" % reason)
> > +    except OSError as reason:
> > +        print("ioctl failed: %s" % reason)
> > +
> > +
> > +def main():
> > +    if len(sys.argv) < 2:
> > +        usage()
> > +
> > +    parser = argparse.ArgumentParser(description="Please specify an action to perform.", prog="smb2quota")
>
> description is used to generate a useful help/usage message. Maybe use
> "tool to display quota information for the Linux SMB client file system (CIFS)"
>
> > +    parser.add_argument("-tabular", "-t", metavar="", help="Print quota information in tabular format")
> > +    parser.add_argument("-csv", "-c", metavar="", help="Print quota information in csv format")
> > +    parser.add_argument("-list", "-l", metavar="", help="Print quota information in list format")
>
> * I think we should use 2 dashes (i.e. "--tabular") for long options.
> * For boolean flags, action="store_true" is what you want
> * You can put let argparse know about the path arg, and then use args.path
>
>     parser.add_argument("--tabular", "-t", action="store_true", help="Print quota information in tabular format")
>     parser.add_argument("--csv", "-c", action="store_true", help="Print quota information in csv format")
>     parser.add_argument("--list", "-l", action="store_true", help="Print quota information in list format")
>     parser.add_argument("path", help="Path to query")
>
> Cheers,
> --
> Aurélien Aptel / SUSE Labs Samba Team
> GPG: 1839 CB5F 9F5B FB9B AA97  8C99 03C8 A49B 521B D5D3
> SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg, DE
> GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 247165 (AG München)

A general comment on the script I have which I should have mentionned in
my first email:

Do we really need a separate script for this? I like that we have a
simple python example do this kind of stuff as an example and reference,
I think it's useful, but installing it makes it yet another utility with an
unconventionnal name instead of a new smbinfo subcommand.

For reminders, we already ship:

- mount.cifs
- cifs.upcall
- cifscreds
- pam_cifscreds
- smbinfo
- getcifsacl
- setcifsacl
- cifs.idmap
- secdesc-ui.py (not installed)
- smb2quota.py (not installed)

On Thu, Oct 10, 2019 at 12:33 AM Aurélien Aptel <aaptel@suse.com> wrote:
>
>
> A general comment on the script I have which I should have mentionned in
> my first email:
>
> Do we really need a separate script for this? I like that we have a
> simple python example do this kind of stuff as an example and reference,
> I think it's useful, but installing it makes it yet another utility with an
> unconventionnal name instead of a new smbinfo subcommand.

I think it would be good to have these tools as part of the actual install.
They are in python so they are imho much more useful to the target users
(sysadmins) than a utility written in C.
(I kind of regret that smbinfo is in C, it should have been python too:-( )

Maybe we just need to decide on a naming prefix for these utilities
and then there shouldn't be
a problem to add many small useful tools.
The nice thing with small python tools is that it is so easy to tweak
them to specific usecases.

I think they should be installed but perhaps be called:
smb2-secdesc-ui.py
smb2-quota.py

regards
ronnie

>
> For reminders, we already ship:
>
> - mount.cifs
> - cifs.upcall
> - cifscreds
> - pam_cifscreds
> - smbinfo
> - getcifsacl
> - setcifsacl
> - cifs.idmap
> - secdesc-ui.py (not installed)
> - smb2quota.py (not installed)
>
> --
> Aurélien Aptel / SUSE Labs Samba Team
> GPG: 1839 CB5F 9F5B FB9B AA97  8C99 03C8 A49B 521B D5D3
> SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg, DE
> GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 247165 (AG München)

or smb3-     …. since "smb3" means more secure ...

On Wed, Oct 9, 2019 at 4:29 PM ronnie sahlberg <ronniesahlberg@gmail.com> wrote:
>
> On Thu, Oct 10, 2019 at 12:33 AM Aurélien Aptel <aaptel@suse.com> wrote:
> >
> >
> > A general comment on the script I have which I should have mentionned in
> > my first email:
> >
> > Do we really need a separate script for this? I like that we have a
> > simple python example do this kind of stuff as an example and reference,
> > I think it's useful, but installing it makes it yet another utility with an
> > unconventionnal name instead of a new smbinfo subcommand.
>
> I think it would be good to have these tools as part of the actual install.
> They are in python so they are imho much more useful to the target users
> (sysadmins) than a utility written in C.
> (I kind of regret that smbinfo is in C, it should have been python too:-( )
>
> Maybe we just need to decide on a naming prefix for these utilities
> and then there shouldn't be
> a problem to add many small useful tools.
> The nice thing with small python tools is that it is so easy to tweak
> them to specific usecases.
>
> I think they should be installed but perhaps be called:
> smb2-secdesc-ui.py
> smb2-quota.py
>
> regards
> ronnie
>
> >
> > For reminders, we already ship:
> >
> > - mount.cifs
> > - cifs.upcall
> > - cifscreds
> > - pam_cifscreds
> > - smbinfo
> > - getcifsacl
> > - setcifsacl
> > - cifs.idmap
> > - secdesc-ui.py (not installed)
> > - smb2quota.py (not installed)
> >
> > --
> > Aurélien Aptel / SUSE Labs Samba Team
> > GPG: 1839 CB5F 9F5B FB9B AA97  8C99 03C8 A49B 521B D5D3
> > SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg, DE
> > GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 247165 (AG München)

"ronnie sahlberg" <ronniesahlberg@gmail.com> writes:
> I think it would be good to have these tools as part of the actual install.
> They are in python so they are imho much more useful to the target users
> (sysadmins) than a utility written in C.
> (I kind of regret that smbinfo is in C, it should have been python
> too:-( )

I completely agree, we could rewrite smbinfo in python.

> Maybe we just need to decide on a naming prefix for these utilities
> and then there shouldn't be
> a problem to add many small useful tools.

We can also make the C code call the python script for now (or vice
versa, while smbinfo gets rewritten).

> The nice thing with small python tools is that it is so easy to tweak
> them to specific usecases.

+100

Cheers,

Hi everyone,

Sorry for the delay in responses. I agree with the idea to re-write
smbinfo in python since it gives more flexibility. And yes, we will
need installers for python utilities in order to properly ship those.

I am going to cut off the next release at the beginning of next week.
The "next" branch is updated to have everything that I have got for
now, except smbinfo re-write in python that I would like to postpone
for the next release.

https://github.com/piastry/cifs-utils/commits/next

If any patch is missed or If someone has any WIP patches that need to
be in the next release, please let me know.

--
Best regards,
Pavel Shilovsky

чт, 10 окт. 2019 г. в 00:22, Aurélien Aptel <aaptel@suse.com>:
>
> "ronnie sahlberg" <ronniesahlberg@gmail.com> writes:
> > I think it would be good to have these tools as part of the actual install.
> > They are in python so they are imho much more useful to the target users
> > (sysadmins) than a utility written in C.
> > (I kind of regret that smbinfo is in C, it should have been python
> > too:-( )
>
> I completely agree, we could rewrite smbinfo in python.
>
> > Maybe we just need to decide on a naming prefix for these utilities
> > and then there shouldn't be
> > a problem to add many small useful tools.
>
> We can also make the C code call the python script for now (or vice
> versa, while smbinfo gets rewritten).
>
> > The nice thing with small python tools is that it is so easy to tweak
> > them to specific usecases.
>
> +100
>
> Cheers,
> --
> Aurélien Aptel / SUSE Labs Samba Team
> GPG: 1839 CB5F 9F5B FB9B AA97  8C99 03C8 A49B 521B D5D3
> SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg, DE
> GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 247165 (AG München)

The patch is pretty small.
You might need to massage it a little bit since I recall that we
renamed the smb2quota util
but that is not yet in github

On Fri, Dec 13, 2019 at 10:23 AM Pavel Shilovsky <piastryyy@gmail.com> wrote:
>
> Hi everyone,
>
> Sorry for the delay in responses. I agree with the idea to re-write
> smbinfo in python since it gives more flexibility. And yes, we will
> need installers for python utilities in order to properly ship those.
>
> I am going to cut off the next release at the beginning of next week.
> The "next" branch is updated to have everything that I have got for
> now, except smbinfo re-write in python that I would like to postpone
> for the next release.
>
> https://github.com/piastry/cifs-utils/commits/next
>
> If any patch is missed or If someone has any WIP patches that need to
> be in the next release, please let me know.
>
> --
> Best regards,
> Pavel Shilovsky
>
> чт, 10 окт. 2019 г. в 00:22, Aurélien Aptel <aaptel@suse.com>:
> >
> > "ronnie sahlberg" <ronniesahlberg@gmail.com> writes:
> > > I think it would be good to have these tools as part of the actual install.
> > > They are in python so they are imho much more useful to the target users
> > > (sysadmins) than a utility written in C.
> > > (I kind of regret that smbinfo is in C, it should have been python
> > > too:-( )
> >
> > I completely agree, we could rewrite smbinfo in python.
> >
> > > Maybe we just need to decide on a naming prefix for these utilities
> > > and then there shouldn't be
> > > a problem to add many small useful tools.
> >
> > We can also make the C code call the python script for now (or vice
> > versa, while smbinfo gets rewritten).
> >
> > > The nice thing with small python tools is that it is so easy to tweak
> > > them to specific usecases.
> >
> > +100
> >
> > Cheers,
> > --
> > Aurélien Aptel / SUSE Labs Samba Team
> > GPG: 1839 CB5F 9F5B FB9B AA97  8C99 03C8 A49B 521B D5D3
> > SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg, DE
> > GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 247165 (AG München)

чт, 12 дек. 2019 г. в 16:34, ronnie sahlberg <ronniesahlberg@gmail.com>:
>
> The patch is pretty small.
> You might need to massage it a little bit since I recall that we
> renamed the smb2quota util
> but that is not yet in github
>

Thanks. Merged:

https://github.com/piastry/cifs-utils/commit/c10c217b117ae6417be29ca4a0b901b98566898b

--
Best regards,
Pavel Shilovsky