Patchwork [v7,2/6] qapi: Introduce add-fd, remove-fd, query-fdsets

login
register
mail settings
Submitter Corey Bryant
Date Aug. 7, 2012, 3:58 p.m.
Message ID <1344355108-14786-3-git-send-email-coreyb@linux.vnet.ibm.com>
Download mbox | patch
Permalink /patch/175732/
State New
Headers show

Comments

Corey Bryant - Aug. 7, 2012, 3:58 p.m.
This patch adds support that enables passing of file descriptors
to the QEMU monitor where they will be stored in specified file
descriptor sets.

A file descriptor set can be used by a client like libvirt to
store file descriptors for the same file.  This allows the
client to open a file with different access modes (O_RDWR,
O_WRONLY, O_RDONLY) and add/remove the passed fds to/from an fd
set as needed.  This will allow QEMU to (in a later patch in this
series) "open" and "reopen" the same file by dup()ing the fd in
the fd set that corresponds to the file, where the fd has the
matching access mode flag that QEMU requests.

The new QMP commands are:
  add-fd: Add a file descriptor to an fd set
  remove-fd: Remove a file descriptor from an fd set
  query-fdsets: Return information describing all fd sets

Note: These commands are not compatible with the existing getfd
and closefd QMP commands.

Signed-off-by: Corey Bryant <coreyb@linux.vnet.ibm.com>
---
v5:
 -This patch is new in v5 and replaces the pass-fd QMP command
  from v4.
 -By grouping fds in fd sets, we ease managability with an fd
  set per file, addressing concerns raised in v4 about handling
  "reopens" and preventing fd leakage. (eblake@redhat.com,
  kwolf@redhat.com, dberrange@redhat.com)

v6
 -Make @fd optional for remove-fd (eblake@redhat.com)
 -Make @fdset-id optional for add-fd (eblake@redhat.com)

v7:
 -Share fd sets among all monitor connections (kwolf@redhat.com)
 -Added mon_refcount to keep track of monitor connection count.

 monitor.c        |  167 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qapi-schema.json |  110 +++++++++++++++++++++++++++++++++++
 qerror.c         |    4 ++
 qerror.h         |    3 +
 qmp-commands.hx  |  131 ++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 415 insertions(+)
Eric Blake - Aug. 7, 2012, 4:49 p.m.
On 08/07/2012 09:58 AM, Corey Bryant wrote:
> This patch adds support that enables passing of file descriptors
> to the QEMU monitor where they will be stored in specified file
> descriptor sets.
> 
> A file descriptor set can be used by a client like libvirt to
> store file descriptors for the same file.  This allows the
> client to open a file with different access modes (O_RDWR,
> O_WRONLY, O_RDONLY) and add/remove the passed fds to/from an fd
> set as needed.  This will allow QEMU to (in a later patch in this
> series) "open" and "reopen" the same file by dup()ing the fd in
> the fd set that corresponds to the file, where the fd has the
> matching access mode flag that QEMU requests.
> 
> The new QMP commands are:
>   add-fd: Add a file descriptor to an fd set
>   remove-fd: Remove a file descriptor from an fd set
>   query-fdsets: Return information describing all fd sets
> 

> +
> +# @AddfdInfo:
> +#
> +# Information about a file descriptor that was added to an fd set.
> +#
> +# @fdset_id: The ID of the fd set that @fd was added to.
> +#
> +# @fd: The file descriptor that was received via SCM rights and
> +#      added to the fd set.
> +#
> +# Since: 1.2.0

We're not very consistent on '1.2' vs. '1.2.0' in since listings, but
that's probably worth a global cleanup closer to hard freeze.

> +##
> +{ 'type': 'AddfdInfo', 'data': {'fdset_id': 'int', 'fd': 'int'} }

This is a new command, so s/fdset_id/fdset-id/

> +
> +##
> +# @add-fd:
> +#
> +# Add a file descriptor, that was passed via SCM rights, to an fd set.
> +#
> +# @fdset_id: #optional The ID of the fd set to add the file descriptor to.
> +#
> +# Returns: @AddfdInfo on success
> +#          If file descriptor was not received, FdNotSupplied
> +#          If @fdset_id does not exist, FdSetNotFound
> +#
> +# Notes: The list of fd sets is shared by all monitor connections.
> +#
> +#        If @fdset_id is not specified, a new fd set will be created.
> +#
> +# Since: 1.2.0
> +##
> +{ 'command': 'add-fd', 'data': {'*fdset_id': 'int'},

Again, s/fdset_id/fdset-id/

> +  'returns': 'AddfdInfo' }
> +
> +##
> +# @remove-fd:
> +#
> +# Remove a file descriptor from an fd set.
> +#
> +# @fdset_id: The ID of the fd set that the file descriptor belongs to.
> +#
> +# @fd: #optional The file descriptor that is to be removed.
> +#
> +# Returns: Nothing on success
> +#          If @fdset_id or @fd is not found, FdNotFound
> +#
> +# Since: 1.2.0
> +#
> +# Notes: The list of fd sets is shared by all monitor connections.
> +#
> +#        File descriptors that are removed:
> +#        o will not be closed until the reference count corresponding
> +#          to @fdset_id reaches zero.
> +#        o will not be available for use after successful completion
> +#          of the remove-fd command.
> +#
> +#        If @fd is not specified, all file descriptors in @fdset_id
> +#        will be removed.
> +##
> +{ 'command': 'remove-fd', 'data': {'fdset_id': 'int', '*fd': 'int'} }

And again.

> +
> +##
> +# @FdsetFdInfo:
> +#
> +# Information about a file descriptor that belongs to an fd set.
> +#
> +# @fd: The file descriptor value.
> +#
> +# @removed: If true, the remove-fd command has been issued for this fd.
> +#
> +# Since: 1.2.0
> +##
> +{ 'type': 'FdsetFdInfo', 'data': {'fd': 'int', 'removed': 'bool'} }

Is it worth providing any additional information?  For example, knowing
whether the fd is O_RDONLY, O_WRONLY, or O_RDWR might be beneficial to
management apps trying to discover what fds are already present after a
reconnection, in order to decide whether to close them without having to
resort to /proc/$qemupid/fdinfo/nnn lookups.  It might even be worth
marking such information optional, present only when 'removed':false.

> +
> +##
> +# @FdsetInfo:
> +#
> +# Information about an fd set.
> +#
> +# @fdset_id: The ID of the fd set.
> +#
> +# @refcount: A count of the number of outstanding dup() references to
> +#            this fd set.
> +#
> +# @in_use: If true, a monitor is connected and has access to this fd set.
> +#
> +# @fds: A list of file descriptors that belong to this fd set.
> +#
> +# Since: 1.2.0
> +##
> +{ 'type': 'FdsetInfo',
> +  'data': {'fdset_id': 'int', 'refcount': 'int', 'in_use': 'bool',
> +           'fds': ['FdsetFdInfo']} }

s/fdset_id/fdset-id/; s/in_use/in-use/

> +
> +##
> +# @query-fdsets:
> +#
> +# Return information describing all fd sets.
> +#
> +# Returns: A list of @FdsetInfo
> +#
> +# Since: 1.2.0
> +#
> +# Notes: The list of fd sets is shared by all monitor connections.
> +#
> +#        File descriptors are not closed until @refcount is zero,
> +#        and either @in_use is false or @removed is true.
> +#
> +##
> +{ 'command': 'query-fdsets', 'returns': ['FdsetInfo'] }
> diff --git a/qerror.c b/qerror.c
> index 92c4eff..63a0aa1 100644
> --- a/qerror.c
> +++ b/qerror.c
> @@ -148,6 +148,10 @@ static const QErrorStringTable qerror_table[] = {
>          .desc      = "No file descriptor supplied via SCM_RIGHTS",
>      },
>      {
> +        .error_fmt = QERR_FDSET_NOT_FOUND,
> +        .desc      = "File descriptor set with ID '%(id)' not found",
> +    },

Conflicts with Luiz's error cleanups.  I'm not sure whether libvirt
needs to know this specific error, so I'd rather leave it generic for now.

> +++ b/qmp-commands.hx
> @@ -926,6 +926,137 @@ Example:
>  
>  EQMP
>  
> +     {
> +        .name       = "add-fd",
> +        .args_type  = "fdset_id:i?",
> +        .params     = "add-fd fdset_id",

More fallout from s/_/-/ in the QMP naming, throughout this file.
Corey Bryant - Aug. 7, 2012, 5:07 p.m.
On 08/07/2012 12:49 PM, Eric Blake wrote:
> On 08/07/2012 09:58 AM, Corey Bryant wrote:
>> This patch adds support that enables passing of file descriptors
>> to the QEMU monitor where they will be stored in specified file
>> descriptor sets.
>>
>> A file descriptor set can be used by a client like libvirt to
>> store file descriptors for the same file.  This allows the
>> client to open a file with different access modes (O_RDWR,
>> O_WRONLY, O_RDONLY) and add/remove the passed fds to/from an fd
>> set as needed.  This will allow QEMU to (in a later patch in this
>> series) "open" and "reopen" the same file by dup()ing the fd in
>> the fd set that corresponds to the file, where the fd has the
>> matching access mode flag that QEMU requests.
>>
>> The new QMP commands are:
>>    add-fd: Add a file descriptor to an fd set
>>    remove-fd: Remove a file descriptor from an fd set
>>    query-fdsets: Return information describing all fd sets
>>
>
>> +
>> +# @AddfdInfo:
>> +#
>> +# Information about a file descriptor that was added to an fd set.
>> +#
>> +# @fdset_id: The ID of the fd set that @fd was added to.
>> +#
>> +# @fd: The file descriptor that was received via SCM rights and
>> +#      added to the fd set.
>> +#
>> +# Since: 1.2.0
>
> We're not very consistent on '1.2' vs. '1.2.0' in since listings, but
> that's probably worth a global cleanup closer to hard freeze.
>

I'll make a note of it.  Or does Luiz usually do a cleanup?

>> +##
>> +{ 'type': 'AddfdInfo', 'data': {'fdset_id': 'int', 'fd': 'int'} }
>
> This is a new command, so s/fdset_id/fdset-id/
>

Ok

>> +
>> +##
>> +# @add-fd:
>> +#
>> +# Add a file descriptor, that was passed via SCM rights, to an fd set.
>> +#
>> +# @fdset_id: #optional The ID of the fd set to add the file descriptor to.
>> +#
>> +# Returns: @AddfdInfo on success
>> +#          If file descriptor was not received, FdNotSupplied
>> +#          If @fdset_id does not exist, FdSetNotFound
>> +#
>> +# Notes: The list of fd sets is shared by all monitor connections.
>> +#
>> +#        If @fdset_id is not specified, a new fd set will be created.
>> +#
>> +# Since: 1.2.0
>> +##
>> +{ 'command': 'add-fd', 'data': {'*fdset_id': 'int'},
>
> Again, s/fdset_id/fdset-id/
>

Ok

>> +  'returns': 'AddfdInfo' }
>> +
>> +##
>> +# @remove-fd:
>> +#
>> +# Remove a file descriptor from an fd set.
>> +#
>> +# @fdset_id: The ID of the fd set that the file descriptor belongs to.
>> +#
>> +# @fd: #optional The file descriptor that is to be removed.
>> +#
>> +# Returns: Nothing on success
>> +#          If @fdset_id or @fd is not found, FdNotFound
>> +#
>> +# Since: 1.2.0
>> +#
>> +# Notes: The list of fd sets is shared by all monitor connections.
>> +#
>> +#        File descriptors that are removed:
>> +#        o will not be closed until the reference count corresponding
>> +#          to @fdset_id reaches zero.
>> +#        o will not be available for use after successful completion
>> +#          of the remove-fd command.
>> +#
>> +#        If @fd is not specified, all file descriptors in @fdset_id
>> +#        will be removed.
>> +##
>> +{ 'command': 'remove-fd', 'data': {'fdset_id': 'int', '*fd': 'int'} }
>
> And again.
>

Ok

>> +
>> +##
>> +# @FdsetFdInfo:
>> +#
>> +# Information about a file descriptor that belongs to an fd set.
>> +#
>> +# @fd: The file descriptor value.
>> +#
>> +# @removed: If true, the remove-fd command has been issued for this fd.
>> +#
>> +# Since: 1.2.0
>> +##
>> +{ 'type': 'FdsetFdInfo', 'data': {'fd': 'int', 'removed': 'bool'} }
>
> Is it worth providing any additional information?  For example, knowing
> whether the fd is O_RDONLY, O_WRONLY, or O_RDWR might be beneficial to
> management apps trying to discover what fds are already present after a
> reconnection, in order to decide whether to close them without having to
> resort to /proc/$qemupid/fdinfo/nnn lookups.  It might even be worth
> marking such information optional, present only when 'removed':false.
>

It makes sense but I'd like to limit the new functionality at this point 
so that I can get this support into QEMU 1.2.  Can this be added as a 
follow up patch?

>> +
>> +##
>> +# @FdsetInfo:
>> +#
>> +# Information about an fd set.
>> +#
>> +# @fdset_id: The ID of the fd set.
>> +#
>> +# @refcount: A count of the number of outstanding dup() references to
>> +#            this fd set.
>> +#
>> +# @in_use: If true, a monitor is connected and has access to this fd set.
>> +#
>> +# @fds: A list of file descriptors that belong to this fd set.
>> +#
>> +# Since: 1.2.0
>> +##
>> +{ 'type': 'FdsetInfo',
>> +  'data': {'fdset_id': 'int', 'refcount': 'int', 'in_use': 'bool',
>> +           'fds': ['FdsetFdInfo']} }
>
> s/fdset_id/fdset-id/; s/in_use/in-use/
>

Ok

>> +
>> +##
>> +# @query-fdsets:
>> +#
>> +# Return information describing all fd sets.
>> +#
>> +# Returns: A list of @FdsetInfo
>> +#
>> +# Since: 1.2.0
>> +#
>> +# Notes: The list of fd sets is shared by all monitor connections.
>> +#
>> +#        File descriptors are not closed until @refcount is zero,
>> +#        and either @in_use is false or @removed is true.
>> +#
>> +##
>> +{ 'command': 'query-fdsets', 'returns': ['FdsetInfo'] }
>> diff --git a/qerror.c b/qerror.c
>> index 92c4eff..63a0aa1 100644
>> --- a/qerror.c
>> +++ b/qerror.c
>> @@ -148,6 +148,10 @@ static const QErrorStringTable qerror_table[] = {
>>           .desc      = "No file descriptor supplied via SCM_RIGHTS",
>>       },
>>       {
>> +        .error_fmt = QERR_FDSET_NOT_FOUND,
>> +        .desc      = "File descriptor set with ID '%(id)' not found",
>> +    },
>
> Conflicts with Luiz's error cleanups.  I'm not sure whether libvirt
> needs to know this specific error, so I'd rather leave it generic for now.
>

Ok.  I'll try to use something more generic.

>> +++ b/qmp-commands.hx
>> @@ -926,6 +926,137 @@ Example:
>>
>>   EQMP
>>
>> +     {
>> +        .name       = "add-fd",
>> +        .args_type  = "fdset_id:i?",
>> +        .params     = "add-fd fdset_id",
>
> More fallout from s/_/-/ in the QMP naming, throughout this file.
>

Ok
Eric Blake - Aug. 7, 2012, 10:16 p.m.
On 08/07/2012 11:07 AM, Corey Bryant wrote:

>>> +#
>>> +# Since: 1.2.0
>>
>> We're not very consistent on '1.2' vs. '1.2.0' in since listings, but
>> that's probably worth a global cleanup closer to hard freeze.
>>
> 
> I'll make a note of it.  Or does Luiz usually do a cleanup?

No idea.

>>> +##
>>> +{ 'type': 'FdsetFdInfo', 'data': {'fd': 'int', 'removed': 'bool'} }
>>
>> Is it worth providing any additional information?  For example, knowing
>> whether the fd is O_RDONLY, O_WRONLY, or O_RDWR might be beneficial to
>> management apps trying to discover what fds are already present after a
>> reconnection, in order to decide whether to close them without having to
>> resort to /proc/$qemupid/fdinfo/nnn lookups.  It might even be worth
>> marking such information optional, present only when 'removed':false.
>>
> 
> It makes sense but I'd like to limit the new functionality at this point
> so that I can get this support into QEMU 1.2.  Can this be added as a
> follow up patch?

I'm personally okay with the idea of adding additional output fields in
later releases, but I know that has been questioned in the past, so you
may want to get buy-in from Luiz or Anthony.  I guess the other thing is
to see what libvirt would actually find useful, once I complete some
counterpart libvirt patches.  If libvirt can get by without any extra
information and without needing to hack things from procfs, then it's
not worth you spending the effort coding something that will be ignored;
conversely, if a piece of info is so important that I end up hacking
procfs anyways, that says we have a hole in QMP.  I'm okay waiting for now.
Corey Bryant - Aug. 8, 2012, 7:07 p.m.
On 08/07/2012 06:16 PM, Eric Blake wrote:
> On 08/07/2012 11:07 AM, Corey Bryant wrote:
>
>>>> +#
>>>> +# Since: 1.2.0
>>>
>>> We're not very consistent on '1.2' vs. '1.2.0' in since listings, but
>>> that's probably worth a global cleanup closer to hard freeze.
>>>
>>
>> I'll make a note of it.  Or does Luiz usually do a cleanup?
>
> No idea.
>

Luiz, were you planning to take a pass at cleaning up the "since" 
release?  If not let me know and I can submit a patch.  Just let me know 
which to use, '1.2' or '1.2.0'.

>>>> +##
>>>> +{ 'type': 'FdsetFdInfo', 'data': {'fd': 'int', 'removed': 'bool'} }
>>>
>>> Is it worth providing any additional information?  For example, knowing
>>> whether the fd is O_RDONLY, O_WRONLY, or O_RDWR might be beneficial to
>>> management apps trying to discover what fds are already present after a
>>> reconnection, in order to decide whether to close them without having to
>>> resort to /proc/$qemupid/fdinfo/nnn lookups.  It might even be worth
>>> marking such information optional, present only when 'removed':false.
>>>
>>
>> It makes sense but I'd like to limit the new functionality at this point
>> so that I can get this support into QEMU 1.2.  Can this be added as a
>> follow up patch?
>
> I'm personally okay with the idea of adding additional output fields in
> later releases, but I know that has been questioned in the past, so you
> may want to get buy-in from Luiz or Anthony.  I guess the other thing is
> to see what libvirt would actually find useful, once I complete some
> counterpart libvirt patches.  If libvirt can get by without any extra
> information and without needing to hack things from procfs, then it's
> not worth you spending the effort coding something that will be ignored;
> conversely, if a piece of info is so important that I end up hacking
> procfs anyways, that says we have a hole in QMP.  I'm okay waiting for now.
>

Assuming the list of to-do's for the next patch version remains minimal, 
I'll go ahead and add support to return the access mode flags  from 
query-fdsets.  Also, I'm going to remove in-use from the returned data, 
because it is always going to be true.
Luiz Capitulino - Aug. 8, 2012, 8:48 p.m.
On Wed, 08 Aug 2012 15:07:02 -0400
Corey Bryant <coreyb@linux.vnet.ibm.com> wrote:

> 
> 
> On 08/07/2012 06:16 PM, Eric Blake wrote:
> > On 08/07/2012 11:07 AM, Corey Bryant wrote:
> >
> >>>> +#
> >>>> +# Since: 1.2.0
> >>>
> >>> We're not very consistent on '1.2' vs. '1.2.0' in since listings, but
> >>> that's probably worth a global cleanup closer to hard freeze.
> >>>
> >>
> >> I'll make a note of it.  Or does Luiz usually do a cleanup?
> >
> > No idea.
> >
> 
> Luiz, were you planning to take a pass at cleaning up the "since" 
> release?  If not let me know and I can submit a patch.  Just let me know 
> which to use, '1.2' or '1.2.0'.

I'd appreciate it if you submit a patch. I guess we should use 1.2.0.

> 
> >>>> +##
> >>>> +{ 'type': 'FdsetFdInfo', 'data': {'fd': 'int', 'removed': 'bool'} }
> >>>
> >>> Is it worth providing any additional information?  For example, knowing
> >>> whether the fd is O_RDONLY, O_WRONLY, or O_RDWR might be beneficial to
> >>> management apps trying to discover what fds are already present after a
> >>> reconnection, in order to decide whether to close them without having to
> >>> resort to /proc/$qemupid/fdinfo/nnn lookups.  It might even be worth
> >>> marking such information optional, present only when 'removed':false.
> >>>
> >>
> >> It makes sense but I'd like to limit the new functionality at this point
> >> so that I can get this support into QEMU 1.2.  Can this be added as a
> >> follow up patch?
> >
> > I'm personally okay with the idea of adding additional output fields in
> > later releases, but I know that has been questioned in the past, so you
> > may want to get buy-in from Luiz or Anthony.  I guess the other thing is
> > to see what libvirt would actually find useful, once I complete some
> > counterpart libvirt patches.  If libvirt can get by without any extra
> > information and without needing to hack things from procfs, then it's
> > not worth you spending the effort coding something that will be ignored;
> > conversely, if a piece of info is so important that I end up hacking
> > procfs anyways, that says we have a hole in QMP.  I'm okay waiting for now.
> >
> 
> Assuming the list of to-do's for the next patch version remains minimal, 
> I'll go ahead and add support to return the access mode flags  from 
> query-fdsets.  Also, I'm going to remove in-use from the returned data, 
> because it is always going to be true.
>
Corey Bryant - Aug. 8, 2012, 8:52 p.m.
On 08/08/2012 04:48 PM, Luiz Capitulino wrote:
> On Wed, 08 Aug 2012 15:07:02 -0400
> Corey Bryant <coreyb@linux.vnet.ibm.com> wrote:
>
>>
>>
>> On 08/07/2012 06:16 PM, Eric Blake wrote:
>>> On 08/07/2012 11:07 AM, Corey Bryant wrote:
>>>
>>>>>> +#
>>>>>> +# Since: 1.2.0
>>>>>
>>>>> We're not very consistent on '1.2' vs. '1.2.0' in since listings, but
>>>>> that's probably worth a global cleanup closer to hard freeze.
>>>>>
>>>>
>>>> I'll make a note of it.  Or does Luiz usually do a cleanup?
>>>
>>> No idea.
>>>
>>
>> Luiz, were you planning to take a pass at cleaning up the "since"
>> release?  If not let me know and I can submit a patch.  Just let me know
>> which to use, '1.2' or '1.2.0'.
>
> I'd appreciate it if you submit a patch. I guess we should use 1.2.0.
>

Sure I'll do that.

Btw, should I be using error_set(errp, ERROR_CLASS_GENERIC_ERROR, "...") 
and have a dependency on your patch series?  Or should I stick with the 
old error messages?
Luiz Capitulino - Aug. 8, 2012, 9:13 p.m.
On Wed, 08 Aug 2012 16:52:41 -0400
Corey Bryant <coreyb@linux.vnet.ibm.com> wrote:

> 
> 
> On 08/08/2012 04:48 PM, Luiz Capitulino wrote:
> > On Wed, 08 Aug 2012 15:07:02 -0400
> > Corey Bryant <coreyb@linux.vnet.ibm.com> wrote:
> >
> >>
> >>
> >> On 08/07/2012 06:16 PM, Eric Blake wrote:
> >>> On 08/07/2012 11:07 AM, Corey Bryant wrote:
> >>>
> >>>>>> +#
> >>>>>> +# Since: 1.2.0
> >>>>>
> >>>>> We're not very consistent on '1.2' vs. '1.2.0' in since listings, but
> >>>>> that's probably worth a global cleanup closer to hard freeze.
> >>>>>
> >>>>
> >>>> I'll make a note of it.  Or does Luiz usually do a cleanup?
> >>>
> >>> No idea.
> >>>
> >>
> >> Luiz, were you planning to take a pass at cleaning up the "since"
> >> release?  If not let me know and I can submit a patch.  Just let me know
> >> which to use, '1.2' or '1.2.0'.
> >
> > I'd appreciate it if you submit a patch. I guess we should use 1.2.0.
> >
> 
> Sure I'll do that.
> 
> Btw, should I be using error_set(errp, ERROR_CLASS_GENERIC_ERROR, "...") 
> and have a dependency on your patch series?  Or should I stick with the 
> old error messages?

That would preferable, yes. Specially if you're adding new errors.

But I'm not exactly sure when my series will be merged, this depends on
what review will bring.
Corey Bryant - Aug. 8, 2012, 9:18 p.m.
On 08/08/2012 05:13 PM, Luiz Capitulino wrote:
> On Wed, 08 Aug 2012 16:52:41 -0400
> Corey Bryant <coreyb@linux.vnet.ibm.com> wrote:
>
>>
>>
>> On 08/08/2012 04:48 PM, Luiz Capitulino wrote:
>>> On Wed, 08 Aug 2012 15:07:02 -0400
>>> Corey Bryant <coreyb@linux.vnet.ibm.com> wrote:
>>>
>>>>
>>>>
>>>> On 08/07/2012 06:16 PM, Eric Blake wrote:
>>>>> On 08/07/2012 11:07 AM, Corey Bryant wrote:
>>>>>
>>>>>>>> +#
>>>>>>>> +# Since: 1.2.0
>>>>>>>
>>>>>>> We're not very consistent on '1.2' vs. '1.2.0' in since listings, but
>>>>>>> that's probably worth a global cleanup closer to hard freeze.
>>>>>>>
>>>>>>
>>>>>> I'll make a note of it.  Or does Luiz usually do a cleanup?
>>>>>
>>>>> No idea.
>>>>>
>>>>
>>>> Luiz, were you planning to take a pass at cleaning up the "since"
>>>> release?  If not let me know and I can submit a patch.  Just let me know
>>>> which to use, '1.2' or '1.2.0'.
>>>
>>> I'd appreciate it if you submit a patch. I guess we should use 1.2.0.
>>>
>>
>> Sure I'll do that.
>>
>> Btw, should I be using error_set(errp, ERROR_CLASS_GENERIC_ERROR, "...")
>> and have a dependency on your patch series?  Or should I stick with the
>> old error messages?
>
> That would preferable, yes. Specially if you're adding new errors.
>
> But I'm not exactly sure when my series will be merged, this depends on
> what review will bring.
>

Are you planning on getting in QEMU 1.2?
Luiz Capitulino - Aug. 9, 2012, 12:38 a.m.
On Wed, 08 Aug 2012 17:18:33 -0400
Corey Bryant <coreyb@linux.vnet.ibm.com> wrote:

> 
> 
> On 08/08/2012 05:13 PM, Luiz Capitulino wrote:
> > On Wed, 08 Aug 2012 16:52:41 -0400
> > Corey Bryant <coreyb@linux.vnet.ibm.com> wrote:
> >
> >>
> >>
> >> On 08/08/2012 04:48 PM, Luiz Capitulino wrote:
> >>> On Wed, 08 Aug 2012 15:07:02 -0400
> >>> Corey Bryant <coreyb@linux.vnet.ibm.com> wrote:
> >>>
> >>>>
> >>>>
> >>>> On 08/07/2012 06:16 PM, Eric Blake wrote:
> >>>>> On 08/07/2012 11:07 AM, Corey Bryant wrote:
> >>>>>
> >>>>>>>> +#
> >>>>>>>> +# Since: 1.2.0
> >>>>>>>
> >>>>>>> We're not very consistent on '1.2' vs. '1.2.0' in since listings, but
> >>>>>>> that's probably worth a global cleanup closer to hard freeze.
> >>>>>>>
> >>>>>>
> >>>>>> I'll make a note of it.  Or does Luiz usually do a cleanup?
> >>>>>
> >>>>> No idea.
> >>>>>
> >>>>
> >>>> Luiz, were you planning to take a pass at cleaning up the "since"
> >>>> release?  If not let me know and I can submit a patch.  Just let me know
> >>>> which to use, '1.2' or '1.2.0'.
> >>>
> >>> I'd appreciate it if you submit a patch. I guess we should use 1.2.0.
> >>>
> >>
> >> Sure I'll do that.
> >>
> >> Btw, should I be using error_set(errp, ERROR_CLASS_GENERIC_ERROR, "...")
> >> and have a dependency on your patch series?  Or should I stick with the
> >> old error messages?
> >
> > That would preferable, yes. Specially if you're adding new errors.
> >
> > But I'm not exactly sure when my series will be merged, this depends on
> > what review will bring.
> >
> 
> Are you planning on getting in QEMU 1.2?

Absolutely.
Stefan Hajnoczi - Aug. 9, 2012, 9:04 a.m.
On Tue, Aug 7, 2012 at 4:58 PM, Corey Bryant <coreyb@linux.vnet.ibm.com> wrote:
> +##
> +# @FdsetFdInfo:
> +#
> +# Information about a file descriptor that belongs to an fd set.
> +#
> +# @fd: The file descriptor value.
> +#
> +# @removed: If true, the remove-fd command has been issued for this fd.
> +#
> +# Since: 1.2.0
> +##
> +{ 'type': 'FdsetFdInfo', 'data': {'fd': 'int', 'removed': 'bool'} }

I'm not sure if the removed field is useful, especially since
remove-fd is idempotent (you can keep querying fds and then marking
them removed again until they finally close).  The reason I suggest
minimizing the schema is so that we can change implementation details
later without having to synthesize this state.

> +##
> +# @FdsetInfo:
> +#
> +# Information about an fd set.
> +#
> +# @fdset_id: The ID of the fd set.
> +#
> +# @refcount: A count of the number of outstanding dup() references to
> +#            this fd set.
> +#
> +# @in_use: If true, a monitor is connected and has access to this fd set.
> +#
> +# @fds: A list of file descriptors that belong to this fd set.
> +#
> +# Since: 1.2.0
> +##
> +{ 'type': 'FdsetInfo',
> +  'data': {'fdset_id': 'int', 'refcount': 'int', 'in_use': 'bool',
> +           'fds': ['FdsetFdInfo']} }

Why are refcount and in_use exposed?  How will applications use them?
This seems like internal state to me.

Should add-fd allow the application to associate an opaque string with
the fdset?  This could be used to recover after crash.  Otherwise the
application needs to store the fdset_id -> filename mapping in a file
on disk and hope it was safely stored before crash.  It seems like the
best place to keep this info is with the fdset.

Stefan
Kevin Wolf - Aug. 9, 2012, 10:11 a.m.
Am 07.08.2012 18:49, schrieb Eric Blake:
> On 08/07/2012 09:58 AM, Corey Bryant wrote:
>> This patch adds support that enables passing of file descriptors
>> to the QEMU monitor where they will be stored in specified file
>> descriptor sets.
>>
>> A file descriptor set can be used by a client like libvirt to
>> store file descriptors for the same file.  This allows the
>> client to open a file with different access modes (O_RDWR,
>> O_WRONLY, O_RDONLY) and add/remove the passed fds to/from an fd
>> set as needed.  This will allow QEMU to (in a later patch in this
>> series) "open" and "reopen" the same file by dup()ing the fd in
>> the fd set that corresponds to the file, where the fd has the
>> matching access mode flag that QEMU requests.
>>
>> The new QMP commands are:
>>   add-fd: Add a file descriptor to an fd set
>>   remove-fd: Remove a file descriptor from an fd set
>>   query-fdsets: Return information describing all fd sets
>>
> 
>> +
>> +# @AddfdInfo:
>> +#
>> +# Information about a file descriptor that was added to an fd set.
>> +#
>> +# @fdset_id: The ID of the fd set that @fd was added to.
>> +#
>> +# @fd: The file descriptor that was received via SCM rights and
>> +#      added to the fd set.
>> +#
>> +# Since: 1.2.0
> 
> We're not very consistent on '1.2' vs. '1.2.0' in since listings, but
> that's probably worth a global cleanup closer to hard freeze.
> 
>> +##
>> +{ 'type': 'AddfdInfo', 'data': {'fdset_id': 'int', 'fd': 'int'} }
> 
> This is a new command, so s/fdset_id/fdset-id/
> 
>> +
>> +##
>> +# @add-fd:
>> +#
>> +# Add a file descriptor, that was passed via SCM rights, to an fd set.
>> +#
>> +# @fdset_id: #optional The ID of the fd set to add the file descriptor to.
>> +#
>> +# Returns: @AddfdInfo on success
>> +#          If file descriptor was not received, FdNotSupplied
>> +#          If @fdset_id does not exist, FdSetNotFound
>> +#
>> +# Notes: The list of fd sets is shared by all monitor connections.
>> +#
>> +#        If @fdset_id is not specified, a new fd set will be created.
>> +#
>> +# Since: 1.2.0
>> +##
>> +{ 'command': 'add-fd', 'data': {'*fdset_id': 'int'},
> 
> Again, s/fdset_id/fdset-id/
> 
>> +  'returns': 'AddfdInfo' }
>> +
>> +##
>> +# @remove-fd:
>> +#
>> +# Remove a file descriptor from an fd set.
>> +#
>> +# @fdset_id: The ID of the fd set that the file descriptor belongs to.
>> +#
>> +# @fd: #optional The file descriptor that is to be removed.
>> +#
>> +# Returns: Nothing on success
>> +#          If @fdset_id or @fd is not found, FdNotFound
>> +#
>> +# Since: 1.2.0
>> +#
>> +# Notes: The list of fd sets is shared by all monitor connections.
>> +#
>> +#        File descriptors that are removed:
>> +#        o will not be closed until the reference count corresponding
>> +#          to @fdset_id reaches zero.
>> +#        o will not be available for use after successful completion
>> +#          of the remove-fd command.
>> +#
>> +#        If @fd is not specified, all file descriptors in @fdset_id
>> +#        will be removed.
>> +##
>> +{ 'command': 'remove-fd', 'data': {'fdset_id': 'int', '*fd': 'int'} }
> 
> And again.
> 
>> +
>> +##
>> +# @FdsetFdInfo:
>> +#
>> +# Information about a file descriptor that belongs to an fd set.
>> +#
>> +# @fd: The file descriptor value.
>> +#
>> +# @removed: If true, the remove-fd command has been issued for this fd.
>> +#
>> +# Since: 1.2.0
>> +##
>> +{ 'type': 'FdsetFdInfo', 'data': {'fd': 'int', 'removed': 'bool'} }
> 
> Is it worth providing any additional information?  For example, knowing
> whether the fd is O_RDONLY, O_WRONLY, or O_RDWR might be beneficial to
> management apps trying to discover what fds are already present after a
> reconnection, in order to decide whether to close them without having to
> resort to /proc/$qemupid/fdinfo/nnn lookups.  It might even be worth
> marking such information optional, present only when 'removed':false.

Why do we even include removed=true file descriptors in query-fdsets?
Shouldn't they appear to be, well, removed from a clients POV?

The problem with adding flags is the same as with errno numbers: How to
do it in a platform independent way? The management application might
run on a different OS than qemu, so a numeric 'flags' field could have
an entirely different meaning there.

We could add bools for some flags and an enum for
O_RDONLY/O_WRONLY/O_RDWR, but it's probably better to wait until we know
which of them we really need.

Kevin
Eric Blake - Aug. 9, 2012, 1:06 p.m.
On 08/09/2012 03:04 AM, Stefan Hajnoczi wrote:
> On Tue, Aug 7, 2012 at 4:58 PM, Corey Bryant <coreyb@linux.vnet.ibm.com> wrote:
>> +##
>> +# @FdsetFdInfo:
>> +#
>> +# Information about a file descriptor that belongs to an fd set.
>> +#
>> +# @fd: The file descriptor value.
>> +#
>> +# @removed: If true, the remove-fd command has been issued for this fd.
>> +#
>> +# Since: 1.2.0
>> +##
>> +{ 'type': 'FdsetFdInfo', 'data': {'fd': 'int', 'removed': 'bool'} }
> 
> I'm not sure if the removed field is useful, especially since
> remove-fd is idempotent (you can keep querying fds and then marking
> them removed again until they finally close).  The reason I suggest
> minimizing the schema is so that we can change implementation details
> later without having to synthesize this state.

Pretty much agreed - rather than listing 'removed', omitting the fd by
default will match the monitors expectations ("why are you telling me
about an fd I told you to remove?").  The only reason I could see for
keeping it would be for debug purposes, but that would be debugging of
qemu, not the application connected to the monitor, at which point gdb
debugging is probably better.

>> +{ 'type': 'FdsetInfo',
>> +  'data': {'fdset_id': 'int', 'refcount': 'int', 'in_use': 'bool',
>> +           'fds': ['FdsetFdInfo']} }
> 
> Why are refcount and in_use exposed?  How will applications use them?
> This seems like internal state to me.

refcount _might_ be useful for knowing whether qemu is actively using
the fdset.  For example, in the sequence:

add-fd nnn
drive-add

if libvirtd crashes after sending drive-add but before getting a
response, then on restart it has to figure out if the drive-add took or
failed.  A non-zero refcount means it took.  But then again, so does a
query-block.  I like the approach of being minimal until we prove we
need it, and can't right now think of anything where having a refcount
would tell libvirt anything new that it can't already learn from
somewhere else.

> 
> Should add-fd allow the application to associate an opaque string with
> the fdset?  This could be used to recover after crash.  Otherwise the
> application needs to store the fdset_id -> filename mapping in a file
> on disk and hope it was safely stored before crash.  It seems like the
> best place to keep this info is with the fdset.

Very nice idea!  In fact, even nicer than returning a markup of O_RDONLY
- if the management app cares about knowing details on an fd, such as
whether it is read-only, then an opaque string tied to the fd in the
fdset becomes very useful.  The string needs to be optional on add-fd.
(Libvirt might even use an xml-like string like "<fd
file='/path/to/file' mode='O_RDONLY'/>", but of course, being an opaque
string, qemu doesn't have to care what the string contains.)
Corey Bryant - Aug. 9, 2012, 1:23 p.m.
On 08/09/2012 09:06 AM, Eric Blake wrote:
> On 08/09/2012 03:04 AM, Stefan Hajnoczi wrote:
>> On Tue, Aug 7, 2012 at 4:58 PM, Corey Bryant <coreyb@linux.vnet.ibm.com> wrote:
>>> +##
>>> +# @FdsetFdInfo:
>>> +#
>>> +# Information about a file descriptor that belongs to an fd set.
>>> +#
>>> +# @fd: The file descriptor value.
>>> +#
>>> +# @removed: If true, the remove-fd command has been issued for this fd.
>>> +#
>>> +# Since: 1.2.0
>>> +##
>>> +{ 'type': 'FdsetFdInfo', 'data': {'fd': 'int', 'removed': 'bool'} }
>>
>> I'm not sure if the removed field is useful, especially since
>> remove-fd is idempotent (you can keep querying fds and then marking
>> them removed again until they finally close).  The reason I suggest
>> minimizing the schema is so that we can change implementation details
>> later without having to synthesize this state.
>
> Pretty much agreed - rather than listing 'removed', omitting the fd by
> default will match the monitors expectations ("why are you telling me
> about an fd I told you to remove?").  The only reason I could see for
> keeping it would be for debug purposes, but that would be debugging of
> qemu, not the application connected to the monitor, at which point gdb
> debugging is probably better.
>

Thanks for the input!  I'll go ahead and drop removed fd's from the 
query-fdsets output.

>>> +{ 'type': 'FdsetInfo',
>>> +  'data': {'fdset_id': 'int', 'refcount': 'int', 'in_use': 'bool',
>>> +           'fds': ['FdsetFdInfo']} }
>>
>> Why are refcount and in_use exposed?  How will applications use them?
>> This seems like internal state to me.
>
> refcount _might_ be useful for knowing whether qemu is actively using
> the fdset.  For example, in the sequence:
>
> add-fd nnn
> drive-add
>
> if libvirtd crashes after sending drive-add but before getting a
> response, then on restart it has to figure out if the drive-add took or
> failed.  A non-zero refcount means it took.  But then again, so does a
> query-block.  I like the approach of being minimal until we prove we
> need it, and can't right now think of anything where having a refcount
> would tell libvirt anything new that it can't already learn from
> somewhere else.
>

I'll also drop both in_use and refcount. I was already planning on 
dropping in_use because at this point it's always true anyway.

>>
>> Should add-fd allow the application to associate an opaque string with
>> the fdset?  This could be used to recover after crash.  Otherwise the
>> application needs to store the fdset_id -> filename mapping in a file
>> on disk and hope it was safely stored before crash.  It seems like the
>> best place to keep this info is with the fdset.
>
> Very nice idea!  In fact, even nicer than returning a markup of O_RDONLY
> - if the management app cares about knowing details on an fd, such as
> whether it is read-only, then an opaque string tied to the fd in the
> fdset becomes very useful.  The string needs to be optional on add-fd.
> (Libvirt might even use an xml-like string like "<fd
> file='/path/to/file' mode='O_RDONLY'/>", but of course, being an opaque
> string, qemu doesn't have to care what the string contains.)
>

Yes this makes a lot of sense.  I'll add the opaque string support. 
Since the client can put the access mode in the opaque string then I 
won't add support to QEMU to return the access-mode for each fd on 
query-fdsets.
Corey Bryant - Aug. 9, 2012, 1:27 p.m.
On 08/09/2012 06:11 AM, Kevin Wolf wrote:
> Am 07.08.2012 18:49, schrieb Eric Blake:
>> On 08/07/2012 09:58 AM, Corey Bryant wrote:
>>> This patch adds support that enables passing of file descriptors
>>> to the QEMU monitor where they will be stored in specified file
>>> descriptor sets.
>>>
>>> A file descriptor set can be used by a client like libvirt to
>>> store file descriptors for the same file.  This allows the
>>> client to open a file with different access modes (O_RDWR,
>>> O_WRONLY, O_RDONLY) and add/remove the passed fds to/from an fd
>>> set as needed.  This will allow QEMU to (in a later patch in this
>>> series) "open" and "reopen" the same file by dup()ing the fd in
>>> the fd set that corresponds to the file, where the fd has the
>>> matching access mode flag that QEMU requests.
>>>
>>> The new QMP commands are:
>>>    add-fd: Add a file descriptor to an fd set
>>>    remove-fd: Remove a file descriptor from an fd set
>>>    query-fdsets: Return information describing all fd sets
>>>
>>
>>> +
>>> +# @AddfdInfo:
>>> +#
>>> +# Information about a file descriptor that was added to an fd set.
>>> +#
>>> +# @fdset_id: The ID of the fd set that @fd was added to.
>>> +#
>>> +# @fd: The file descriptor that was received via SCM rights and
>>> +#      added to the fd set.
>>> +#
>>> +# Since: 1.2.0
>>
>> We're not very consistent on '1.2' vs. '1.2.0' in since listings, but
>> that's probably worth a global cleanup closer to hard freeze.
>>
>>> +##
>>> +{ 'type': 'AddfdInfo', 'data': {'fdset_id': 'int', 'fd': 'int'} }
>>
>> This is a new command, so s/fdset_id/fdset-id/
>>
>>> +
>>> +##
>>> +# @add-fd:
>>> +#
>>> +# Add a file descriptor, that was passed via SCM rights, to an fd set.
>>> +#
>>> +# @fdset_id: #optional The ID of the fd set to add the file descriptor to.
>>> +#
>>> +# Returns: @AddfdInfo on success
>>> +#          If file descriptor was not received, FdNotSupplied
>>> +#          If @fdset_id does not exist, FdSetNotFound
>>> +#
>>> +# Notes: The list of fd sets is shared by all monitor connections.
>>> +#
>>> +#        If @fdset_id is not specified, a new fd set will be created.
>>> +#
>>> +# Since: 1.2.0
>>> +##
>>> +{ 'command': 'add-fd', 'data': {'*fdset_id': 'int'},
>>
>> Again, s/fdset_id/fdset-id/
>>
>>> +  'returns': 'AddfdInfo' }
>>> +
>>> +##
>>> +# @remove-fd:
>>> +#
>>> +# Remove a file descriptor from an fd set.
>>> +#
>>> +# @fdset_id: The ID of the fd set that the file descriptor belongs to.
>>> +#
>>> +# @fd: #optional The file descriptor that is to be removed.
>>> +#
>>> +# Returns: Nothing on success
>>> +#          If @fdset_id or @fd is not found, FdNotFound
>>> +#
>>> +# Since: 1.2.0
>>> +#
>>> +# Notes: The list of fd sets is shared by all monitor connections.
>>> +#
>>> +#        File descriptors that are removed:
>>> +#        o will not be closed until the reference count corresponding
>>> +#          to @fdset_id reaches zero.
>>> +#        o will not be available for use after successful completion
>>> +#          of the remove-fd command.
>>> +#
>>> +#        If @fd is not specified, all file descriptors in @fdset_id
>>> +#        will be removed.
>>> +##
>>> +{ 'command': 'remove-fd', 'data': {'fdset_id': 'int', '*fd': 'int'} }
>>
>> And again.
>>
>>> +
>>> +##
>>> +# @FdsetFdInfo:
>>> +#
>>> +# Information about a file descriptor that belongs to an fd set.
>>> +#
>>> +# @fd: The file descriptor value.
>>> +#
>>> +# @removed: If true, the remove-fd command has been issued for this fd.
>>> +#
>>> +# Since: 1.2.0
>>> +##
>>> +{ 'type': 'FdsetFdInfo', 'data': {'fd': 'int', 'removed': 'bool'} }
>>
>> Is it worth providing any additional information?  For example, knowing
>> whether the fd is O_RDONLY, O_WRONLY, or O_RDWR might be beneficial to
>> management apps trying to discover what fds are already present after a
>> reconnection, in order to decide whether to close them without having to
>> resort to /proc/$qemupid/fdinfo/nnn lookups.  It might even be worth
>> marking such information optional, present only when 'removed':false.
>
> Why do we even include removed=true file descriptors in query-fdsets?
> Shouldn't they appear to be, well, removed from a clients POV?
>
> The problem with adding flags is the same as with errno numbers: How to
> do it in a platform independent way? The management application might
> run on a different OS than qemu, so a numeric 'flags' field could have
> an entirely different meaning there.
>
> We could add bools for some flags and an enum for
> O_RDONLY/O_WRONLY/O_RDWR, but it's probably better to wait until we know
> which of them we really need.

Based on the other email thread that's going on in parallel to this, I 
think these issues are resolved.

Patch

diff --git a/monitor.c b/monitor.c
index 49dccfe..04b86b7 100644
--- a/monitor.c
+++ b/monitor.c
@@ -140,6 +140,23 @@  struct mon_fd_t {
     QLIST_ENTRY(mon_fd_t) next;
 };
 
+/* file descriptor associated with a file descriptor set */
+typedef struct mon_fdset_fd_t mon_fdset_fd_t;
+struct mon_fdset_fd_t {
+    int fd;
+    bool removed;
+    QLIST_ENTRY(mon_fdset_fd_t) next;
+};
+
+/* file descriptor set containing fds passed via SCM_RIGHTS */
+typedef struct mon_fdset_t mon_fdset_t;
+struct mon_fdset_t {
+    int64_t id;
+    int refcount;
+    QLIST_HEAD(, mon_fdset_fd_t) fds;
+    QLIST_ENTRY(mon_fdset_t) next;
+};
+
 typedef struct MonitorControl {
     QObject *id;
     JSONMessageParser parser;
@@ -211,6 +228,8 @@  static inline int mon_print_count_get(const Monitor *mon) { return 0; }
 #define QMP_ACCEPT_UNKNOWNS 1
 
 static QLIST_HEAD(mon_list, Monitor) mon_list;
+static QLIST_HEAD(mon_fdsets, mon_fdset_t) mon_fdsets;
+static int mon_refcount;
 
 static mon_cmd_t mon_cmds[];
 static mon_cmd_t info_cmds[];
@@ -2389,6 +2408,154 @@  int monitor_get_fd(Monitor *mon, const char *fdname)
     return -1;
 }
 
+static void monitor_fdset_cleanup(mon_fdset_t *mon_fdset)
+{
+    mon_fdset_fd_t *mon_fdset_fd;
+    mon_fdset_fd_t *mon_fdset_fd_next;
+
+    if (mon_fdset->refcount != 0) {
+        return;
+    }
+
+    QLIST_FOREACH_SAFE(mon_fdset_fd, &mon_fdset->fds, next, mon_fdset_fd_next) {
+        if (mon_refcount == 0 || mon_fdset_fd->removed) {
+            close(mon_fdset_fd->fd);
+            QLIST_REMOVE(mon_fdset_fd, next);
+            g_free(mon_fdset_fd);
+        }
+    }
+
+    if (QLIST_EMPTY(&mon_fdset->fds)) {
+        QLIST_REMOVE(mon_fdset, next);
+        g_free(mon_fdset);
+    }
+}
+
+AddfdInfo *qmp_add_fd(bool has_fdset_id, int64_t fdset_id, Error **errp)
+{
+    int fd;
+    Monitor *mon = cur_mon;
+    mon_fdset_t *mon_fdset;
+    mon_fdset_fd_t *mon_fdset_fd;
+    AddfdInfo *fdinfo;
+
+    fd = qemu_chr_fe_get_msgfd(mon->chr);
+    if (fd == -1) {
+        qerror_report(QERR_FD_NOT_SUPPLIED);
+        return NULL;
+    }
+
+    if (has_fdset_id) {
+        QLIST_FOREACH(mon_fdset, &mon_fdsets, next) {
+            if (mon_fdset->id == fdset_id) {
+                break;
+            }
+        }
+        if (mon_fdset == NULL) {
+            qerror_report(QERR_FDSET_NOT_FOUND, fdset_id);
+            return NULL;
+        }
+    } else {
+        int64_t fdset_id_prev = -1;
+        mon_fdset_t *mon_fdset_cur = QLIST_FIRST(&mon_fdsets);
+
+        /* Use first available fdset ID */
+        QLIST_FOREACH(mon_fdset, &mon_fdsets, next) {
+            mon_fdset_cur = mon_fdset;
+            if (fdset_id_prev == mon_fdset_cur->id - 1) {
+                fdset_id_prev = mon_fdset_cur->id;
+                continue;
+            }
+            break;
+        }
+
+        mon_fdset = g_malloc0(sizeof(*mon_fdset));
+        mon_fdset->id = fdset_id_prev + 1;
+        mon_fdset->refcount = 0;
+
+        /* The fdset list is ordered by fdset ID */
+        if (mon_fdset->id == 0) {
+            QLIST_INSERT_HEAD(&mon_fdsets, mon_fdset, next);
+        } else if (mon_fdset->id < mon_fdset_cur->id) {
+            QLIST_INSERT_BEFORE(mon_fdset_cur, mon_fdset, next);
+        } else {
+            QLIST_INSERT_AFTER(mon_fdset_cur, mon_fdset, next);
+        }
+    }
+
+    mon_fdset_fd = g_malloc0(sizeof(*mon_fdset_fd));
+    mon_fdset_fd->fd = fd;
+    mon_fdset_fd->removed = false;
+    QLIST_INSERT_HEAD(&mon_fdset->fds, mon_fdset_fd, next);
+
+    fdinfo = g_malloc0(sizeof(*fdinfo));
+    fdinfo->fdset_id = mon_fdset->id;
+    fdinfo->fd = mon_fdset_fd->fd;
+
+    return fdinfo;
+}
+
+void qmp_remove_fd(int64_t fdset_id, bool has_fd, int64_t fd, Error **errp)
+{
+    mon_fdset_t *mon_fdset;
+    mon_fdset_fd_t *mon_fdset_fd;
+    char fd_str[20];
+
+    QLIST_FOREACH(mon_fdset, &mon_fdsets, next) {
+        if (mon_fdset->id != fdset_id) {
+            continue;
+        }
+        QLIST_FOREACH(mon_fdset_fd, &mon_fdset->fds, next) {
+            if (has_fd && mon_fdset_fd->fd != fd) {
+                continue;
+            }
+            mon_fdset_fd->removed = true;
+            if (has_fd) {
+                break;
+            }
+        }
+        monitor_fdset_cleanup(mon_fdset);
+        return;
+    }
+    snprintf(fd_str, sizeof(fd_str), "%ld", fd);
+    qerror_report(QERR_FD_NOT_FOUND, fd_str);
+}
+
+FdsetInfoList *qmp_query_fdsets(Error **errp)
+{
+    mon_fdset_t *mon_fdset;
+    mon_fdset_fd_t *mon_fdset_fd;
+    FdsetInfoList *fdset_list = NULL;
+
+    QLIST_FOREACH(mon_fdset, &mon_fdsets, next) {
+        FdsetInfoList *fdset_info = g_malloc0(sizeof(*fdset_info));
+        FdsetFdInfoList *fdsetfd_list = NULL;
+
+        fdset_info->value = g_malloc0(sizeof(*fdset_info->value));
+        fdset_info->value->fdset_id = mon_fdset->id;
+        fdset_info->value->refcount = mon_fdset->refcount;
+        fdset_info->value->in_use = mon_refcount > 0 ? true : false;
+
+        QLIST_FOREACH(mon_fdset_fd, &mon_fdset->fds, next) {
+            FdsetFdInfoList *fdsetfd_info = g_malloc0(sizeof(*fdsetfd_info));
+
+            fdsetfd_info->value = g_malloc0(sizeof(*fdsetfd_info->value));
+            fdsetfd_info->value->fd = mon_fdset_fd->fd;
+            fdsetfd_info->value->removed = mon_fdset_fd->removed;
+
+            fdsetfd_info->next = fdsetfd_list;
+            fdsetfd_list = fdsetfd_info;
+        }
+
+        fdset_info->value->fds = fdsetfd_list;
+
+        fdset_info->next = fdset_list;
+        fdset_list = fdset_info;
+    }
+
+    return fdset_list;
+}
+
 /* mon_cmds and info_cmds would be sorted at runtime */
 static mon_cmd_t mon_cmds[] = {
 #include "hmp-commands.h"
diff --git a/qapi-schema.json b/qapi-schema.json
index cddf63a..1a21bf8 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2200,3 +2200,113 @@ 
 # Since: 0.14.0
 ##
 { 'command': 'closefd', 'data': {'fdname': 'str'} }
+
+# @AddfdInfo:
+#
+# Information about a file descriptor that was added to an fd set.
+#
+# @fdset_id: The ID of the fd set that @fd was added to.
+#
+# @fd: The file descriptor that was received via SCM rights and
+#      added to the fd set.
+#
+# Since: 1.2.0
+##
+{ 'type': 'AddfdInfo', 'data': {'fdset_id': 'int', 'fd': 'int'} }
+
+##
+# @add-fd:
+#
+# Add a file descriptor, that was passed via SCM rights, to an fd set.
+#
+# @fdset_id: #optional The ID of the fd set to add the file descriptor to.
+#
+# Returns: @AddfdInfo on success
+#          If file descriptor was not received, FdNotSupplied
+#          If @fdset_id does not exist, FdSetNotFound
+#
+# Notes: The list of fd sets is shared by all monitor connections.
+#
+#        If @fdset_id is not specified, a new fd set will be created.
+#
+# Since: 1.2.0
+##
+{ 'command': 'add-fd', 'data': {'*fdset_id': 'int'},
+  'returns': 'AddfdInfo' }
+
+##
+# @remove-fd:
+#
+# Remove a file descriptor from an fd set.
+#
+# @fdset_id: The ID of the fd set that the file descriptor belongs to.
+#
+# @fd: #optional The file descriptor that is to be removed.
+#
+# Returns: Nothing on success
+#          If @fdset_id or @fd is not found, FdNotFound
+#
+# Since: 1.2.0
+#
+# Notes: The list of fd sets is shared by all monitor connections.
+#
+#        File descriptors that are removed:
+#        o will not be closed until the reference count corresponding
+#          to @fdset_id reaches zero.
+#        o will not be available for use after successful completion
+#          of the remove-fd command.
+#
+#        If @fd is not specified, all file descriptors in @fdset_id
+#        will be removed.
+##
+{ 'command': 'remove-fd', 'data': {'fdset_id': 'int', '*fd': 'int'} }
+
+##
+# @FdsetFdInfo:
+#
+# Information about a file descriptor that belongs to an fd set.
+#
+# @fd: The file descriptor value.
+#
+# @removed: If true, the remove-fd command has been issued for this fd.
+#
+# Since: 1.2.0
+##
+{ 'type': 'FdsetFdInfo', 'data': {'fd': 'int', 'removed': 'bool'} }
+
+##
+# @FdsetInfo:
+#
+# Information about an fd set.
+#
+# @fdset_id: The ID of the fd set.
+#
+# @refcount: A count of the number of outstanding dup() references to
+#            this fd set.
+#
+# @in_use: If true, a monitor is connected and has access to this fd set.
+#
+# @fds: A list of file descriptors that belong to this fd set.
+#
+# Since: 1.2.0
+##
+{ 'type': 'FdsetInfo',
+  'data': {'fdset_id': 'int', 'refcount': 'int', 'in_use': 'bool',
+           'fds': ['FdsetFdInfo']} }
+
+##
+# @query-fdsets:
+#
+# Return information describing all fd sets.
+#
+# Returns: A list of @FdsetInfo
+#
+# Since: 1.2.0
+#
+# Notes: The list of fd sets is shared by all monitor connections.
+#
+#        File descriptors are not closed until @refcount is zero,
+#        and either @in_use is false or @removed is true.
+#
+##
+{ 'command': 'query-fdsets', 'returns': ['FdsetInfo'] }
diff --git a/qerror.c b/qerror.c
index 92c4eff..63a0aa1 100644
--- a/qerror.c
+++ b/qerror.c
@@ -148,6 +148,10 @@  static const QErrorStringTable qerror_table[] = {
         .desc      = "No file descriptor supplied via SCM_RIGHTS",
     },
     {
+        .error_fmt = QERR_FDSET_NOT_FOUND,
+        .desc      = "File descriptor set with ID '%(id)' not found",
+    },
+    {
         .error_fmt = QERR_FEATURE_DISABLED,
         .desc      = "The feature '%(name)' is not enabled",
     },
diff --git a/qerror.h b/qerror.h
index b4c8758..b908d3f 100644
--- a/qerror.h
+++ b/qerror.h
@@ -133,6 +133,9 @@  QError *qobject_to_qerror(const QObject *obj);
 #define QERR_FD_NOT_SUPPLIED \
     "{ 'class': 'FdNotSupplied', 'data': {} }"
 
+#define QERR_FDSET_NOT_FOUND \
+    "{ 'class': 'FdSetNotFound', 'data': { 'id': %ld } }"
+
 #define QERR_FEATURE_DISABLED \
     "{ 'class': 'FeatureDisabled', 'data': { 'name': %s } }"
 
diff --git a/qmp-commands.hx b/qmp-commands.hx
index ac46638..3c243d8 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -926,6 +926,137 @@  Example:
 
 EQMP
 
+     {
+        .name       = "add-fd",
+        .args_type  = "fdset_id:i?",
+        .params     = "add-fd fdset_id",
+        .help       = "Add a file descriptor, that was passed via SCM rights, to an fd set",
+        .mhandler.cmd_new = qmp_marshal_input_add_fd,
+    },
+
+SQMP
+add-fd
+-------
+
+Add a file descriptor, that was passed via SCM rights, to an fd set.
+
+Arguments:
+
+- "fdset_id": The ID of the fd set to add the file descriptor to.
+              (json-int, optional)
+
+Return a json-object with the following information:
+
+- "fdset_id": The ID of the fd set that the fd was added to. (json-int)
+- "fd": The file descriptor that was received via SCM rights and added to the
+        fd set. (json-int)
+
+Example:
+
+-> { "execute": "add-fd", "arguments": { "fdset_id": 1 } }
+<- { "return": { "fdset_id": 1, "fd": 3 } }
+
+Notes:
+
+(1) The list of fd sets is shared by all monitor connections.
+(2) If "fdset_id" is not specified, a new fd set will be created.
+
+EQMP
+
+     {
+        .name       = "remove-fd",
+        .args_type  = "fdset_id:i,fd:i?",
+        .params     = "remove-fd fdset_id fd",
+        .help       = "Remove a file descriptor from an fd set",
+        .mhandler.cmd_new = qmp_marshal_input_remove_fd,
+    },
+
+SQMP
+remove-fd
+---------
+
+Remove a file descriptor from an fd set.
+
+Arguments:
+
+- "fdset_id": The ID of the fd set that the file descriptor belongs to.
+              (json-int)
+- "fd": The file descriptor that is to be removed. (json-int, optional)
+
+Example:
+
+-> { "execute": "remove-fd", "arguments": { "fdset_id": 1, "fd": 3 } }
+<- { "return": {} }
+
+Notes:
+
+(1) The list of fd sets is shared by all monitor connections.
+(2) File descriptors that are removed:
+    (a) will not be closed until the reference count corresponding to fdset_id
+        reaches zero.
+    (b) will not be available for use after successful completion of the
+        remove-fd command.
+(3) If "fd" is not specified, all file descriptors in "fdset_id" will be
+    removed.
+
+EQMP
+
+    {
+        .name       = "query-fdsets",
+        .args_type  = "",
+        .help       = "Return information describing all fd sets",
+        .mhandler.cmd_new = qmp_marshal_input_query_fdsets,
+    },
+
+SQMP
+query-fdsets
+-------------
+
+Return information describing all fd sets.
+
+Arguments: None
+
+Example:
+
+-> { "execute": "query-fdsets" }
+<- { "return": [
+       {
+         "fdset_id": 1
+         "refcount": 0,
+         "in_use": true,
+         "fds": [
+           {
+             "fd": 21,
+             "removed": false
+           },
+           {
+             "fd": 23,
+             "removed": false
+           }
+         ],
+       },
+       {
+         "fdset_id": 2
+         "refcount": 0,
+         "in_use": true,
+         "fds": [
+           {
+             "fd": 22,
+             "removed": false
+           }
+         ],
+       }
+     ]
+   }
+
+Notes:
+
+(1) The list of fd sets is shared by all monitor connections.
+(2) File descriptors are not closed until refcount is zero, and
+    either in_use is false or removed is true.
+
+EQMP
+
     {
         .name       = "block_passwd",
         .args_type  = "device:B,password:s",