[1/3] qemu-char: Introduce Memory driver

This driver handles in-memory chardev operations. That's, all writes
to this driver are stored in an internal buffer and it doesn't talk
to the external world in any way.

Right now it's very simple: it supports only writes. But it can be
easily extended to support more operations.

This is going to be used by the monitor's "HMP passthrough via QMP"
feature, which needs to run monitor handlers without a backing
device.

Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
---
 qemu-char.c |   66 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qemu-char.h |    6 +++++
 2 files changed, 72 insertions(+), 0 deletions(-)

Luiz Capitulino <lcapitulino@redhat.com> writes:

> This driver handles in-memory chardev operations. That's, all writes
> to this driver are stored in an internal buffer and it doesn't talk
> to the external world in any way.
>
> Right now it's very simple: it supports only writes. But it can be
> easily extended to support more operations.
>
> This is going to be used by the monitor's "HMP passthrough via QMP"
> feature, which needs to run monitor handlers without a backing
> device.
>
> Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
> ---
>  qemu-char.c |   66 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  qemu-char.h |    6 +++++
>  2 files changed, 72 insertions(+), 0 deletions(-)
>
> diff --git a/qemu-char.c b/qemu-char.c
> index 88997f9..896df14 100644
> --- a/qemu-char.c
> +++ b/qemu-char.c
> @@ -2275,6 +2275,72 @@ static CharDriverState *qemu_chr_open_socket(QemuOpts *opts)
>      return NULL;
>  }
>  
> +/***********************************************************/
> +/* Memory chardev */
> +typedef struct {
> +    size_t outbuf_size;
> +    size_t outbuf_capacity;
> +    uint8_t *outbuf;
> +} MemoryDriver;
> +
> +static int mem_chr_write(CharDriverState *chr, const uint8_t *buf, int len)
> +{
> +    MemoryDriver *d = chr->opaque;
> +
> +    /* TODO: the QString implementation has the same code, we should
> +     * introduce a generic way to do this in cutils.c */
> +    if (d->outbuf_capacity < d->outbuf_size + len) {
> +        /* grown outbuf */

Used to say "grow" (sans n) here.  Intentional change?

> +        d->outbuf_capacity += len;
> +        d->outbuf_capacity *= 2;
> +        d->outbuf = qemu_realloc(d->outbuf, d->outbuf_capacity);
> +    }
> +
> +    memcpy(d->outbuf + d->outbuf_size, buf, len);
> +    d->outbuf_size += len;
> +
> +    return len;
> +}
> +
> +#define DEFAULT_BUF_SIZE 4096

It's the *initial* buffer size, isn't it?

Doubt it's worth a #define (there's just one user), but that's a matter
of taste.

> +
> +void qemu_chr_init_mem(CharDriverState *chr)
> +{
> +    MemoryDriver *d;
> +
> +    d = qemu_malloc(sizeof(*d));
> +    d->outbuf_size = 0;
> +    d->outbuf_capacity = DEFAULT_BUF_SIZE;
> +    d->outbuf = qemu_mallocz(d->outbuf_capacity);
> +
> +    memset(chr, 0, sizeof(*chr));
> +    chr->opaque = d;
> +    chr->chr_write = mem_chr_write;
> +}
> +
> +/* assumes the stored data is a string */

What else could it be?  Worrying about embedded '\0's?

> +QString *qemu_chr_mem_to_qs(CharDriverState *chr)
> +{
> +    MemoryDriver *d = chr->opaque;
> +
> +    if (d->outbuf_size == 0) {
> +        return NULL;
> +    }

Did you forget to change this?  We agreed to return an empty QString
when chr contains an empty string.

> +
> +    return qstring_from_substr((char *) d->outbuf, 0, d->outbuf_size - 1);
> +}
> +
> +/* NOTE: this driver can not be closed with qemu_chr_close()! */
> +void qemu_chr_close_mem(CharDriverState *chr)
> +{
> +    MemoryDriver *d = chr->opaque;
> +
> +    qemu_free(d->outbuf);
> +    qemu_free(chr->opaque);
> +    chr->opaque = NULL;
> +    chr->chr_write = NULL;
> +}
> +

Unlike normal character drivers, this one can't be closed with
qemu_chr_close().  It probably explodes if you try.  Please add a
suitable assertion to qemu_chr_close() to document the fact, and to
ensure misuse fails in a controlled, obvious manner.

>  QemuOpts *qemu_chr_parse_compat(const char *label, const char *filename)
>  {
>      char host[65], port[33], width[8], height[8];
> diff --git a/qemu-char.h b/qemu-char.h
> index 18ad12b..c4e55b4 100644
> --- a/qemu-char.h
> +++ b/qemu-char.h
> @@ -6,6 +6,7 @@
>  #include "qemu-option.h"
>  #include "qemu-config.h"
>  #include "qobject.h"
> +#include "qstring.h"
>  
>  /* character device */
>  
> @@ -100,6 +101,11 @@ CharDriverState *qemu_chr_open_eventfd(int eventfd);
>  
>  extern int term_escape_char;
>  
> +/* memory chardev */
> +void qemu_chr_init_mem(CharDriverState *chr);
> +void qemu_chr_close_mem(CharDriverState *chr);
> +QString *qemu_chr_mem_to_qs(CharDriverState *chr);
> +
>  /* async I/O support */
>  
>  int qemu_set_fd_handler2(int fd,

On Thu, 11 Nov 2010 16:30:26 +0100
Markus Armbruster <armbru@redhat.com> wrote:

> Luiz Capitulino <lcapitulino@redhat.com> writes:
> 
> > This driver handles in-memory chardev operations. That's, all writes
> > to this driver are stored in an internal buffer and it doesn't talk
> > to the external world in any way.
> >
> > Right now it's very simple: it supports only writes. But it can be
> > easily extended to support more operations.
> >
> > This is going to be used by the monitor's "HMP passthrough via QMP"
> > feature, which needs to run monitor handlers without a backing
> > device.
> >
> > Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
> > ---
> >  qemu-char.c |   66 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  qemu-char.h |    6 +++++
> >  2 files changed, 72 insertions(+), 0 deletions(-)
> >
> > diff --git a/qemu-char.c b/qemu-char.c
> > index 88997f9..896df14 100644
> > --- a/qemu-char.c
> > +++ b/qemu-char.c
> > @@ -2275,6 +2275,72 @@ static CharDriverState *qemu_chr_open_socket(QemuOpts *opts)
> >      return NULL;
> >  }
> >  
> > +/***********************************************************/
> > +/* Memory chardev */
> > +typedef struct {
> > +    size_t outbuf_size;
> > +    size_t outbuf_capacity;
> > +    uint8_t *outbuf;
> > +} MemoryDriver;
> > +
> > +static int mem_chr_write(CharDriverState *chr, const uint8_t *buf, int len)
> > +{
> > +    MemoryDriver *d = chr->opaque;
> > +
> > +    /* TODO: the QString implementation has the same code, we should
> > +     * introduce a generic way to do this in cutils.c */
> > +    if (d->outbuf_capacity < d->outbuf_size + len) {
> > +        /* grown outbuf */
> 
> Used to say "grow" (sans n) here.  Intentional change?

Hum, no. I think I've squashed an older commit while rebasing (but this seems
to be the only problem).

> 
> > +        d->outbuf_capacity += len;
> > +        d->outbuf_capacity *= 2;
> > +        d->outbuf = qemu_realloc(d->outbuf, d->outbuf_capacity);
> > +    }
> > +
> > +    memcpy(d->outbuf + d->outbuf_size, buf, len);
> > +    d->outbuf_size += len;
> > +
> > +    return len;
> > +}
> > +
> > +#define DEFAULT_BUF_SIZE 4096
> 
> It's the *initial* buffer size, isn't it?

Yes.

> Doubt it's worth a #define (there's just one user), but that's a matter
> of taste.
> 
> > +
> > +void qemu_chr_init_mem(CharDriverState *chr)
> > +{
> > +    MemoryDriver *d;
> > +
> > +    d = qemu_malloc(sizeof(*d));
> > +    d->outbuf_size = 0;
> > +    d->outbuf_capacity = DEFAULT_BUF_SIZE;
> > +    d->outbuf = qemu_mallocz(d->outbuf_capacity);
> > +
> > +    memset(chr, 0, sizeof(*chr));
> > +    chr->opaque = d;
> > +    chr->chr_write = mem_chr_write;
> > +}
> > +
> > +/* assumes the stored data is a string */
> 
> What else could it be?  Worrying about embedded '\0's?

Yes, as the driver itself doesn't interpret the contents of its
buffer.

> > +QString *qemu_chr_mem_to_qs(CharDriverState *chr)
> > +{
> > +    MemoryDriver *d = chr->opaque;
> > +
> > +    if (d->outbuf_size == 0) {
> > +        return NULL;
> > +    }
> 
> Did you forget to change this?  We agreed to return an empty QString
> when chr contains an empty string.

I've changed my mind and forgot to mention it: I thought that we would
need to return NULL on error conditions, but turns out that this function
never fails.

So, I do think it's better to let it that way for two reasons:

 1. An empty has at least the '\0' character, but in this case the buffer
    is really empty

 2. Returning an empty string for this case will add unneeded complexity
    to the caller, ie. checking if the QString's length is 0 and decref'ing it

> 
> > +
> > +    return qstring_from_substr((char *) d->outbuf, 0, d->outbuf_size - 1);
> > +}
> > +
> > +/* NOTE: this driver can not be closed with qemu_chr_close()! */
> > +void qemu_chr_close_mem(CharDriverState *chr)
> > +{
> > +    MemoryDriver *d = chr->opaque;
> > +
> > +    qemu_free(d->outbuf);
> > +    qemu_free(chr->opaque);
> > +    chr->opaque = NULL;
> > +    chr->chr_write = NULL;
> > +}
> > +
> 
> Unlike normal character drivers, this one can't be closed with
> qemu_chr_close().  It probably explodes if you try.  Please add a
> suitable assertion to qemu_chr_close() to document the fact, and to
> ensure misuse fails in a controlled, obvious manner.

Ah forgot, but that can be done as a separate patch, so if I don't respin
this series I'll send an additional patch for that.

> 
> >  QemuOpts *qemu_chr_parse_compat(const char *label, const char *filename)
> >  {
> >      char host[65], port[33], width[8], height[8];
> > diff --git a/qemu-char.h b/qemu-char.h
> > index 18ad12b..c4e55b4 100644
> > --- a/qemu-char.h
> > +++ b/qemu-char.h
> > @@ -6,6 +6,7 @@
> >  #include "qemu-option.h"
> >  #include "qemu-config.h"
> >  #include "qobject.h"
> > +#include "qstring.h"
> >  
> >  /* character device */
> >  
> > @@ -100,6 +101,11 @@ CharDriverState *qemu_chr_open_eventfd(int eventfd);
> >  
> >  extern int term_escape_char;
> >  
> > +/* memory chardev */
> > +void qemu_chr_init_mem(CharDriverState *chr);
> > +void qemu_chr_close_mem(CharDriverState *chr);
> > +QString *qemu_chr_mem_to_qs(CharDriverState *chr);
> > +
> >  /* async I/O support */
> >  
> >  int qemu_set_fd_handler2(int fd,
>

Luiz Capitulino <lcapitulino@redhat.com> writes:

> On Thu, 11 Nov 2010 16:30:26 +0100
> Markus Armbruster <armbru@redhat.com> wrote:
>
>> Luiz Capitulino <lcapitulino@redhat.com> writes:
>> 
>> > This driver handles in-memory chardev operations. That's, all writes
>> > to this driver are stored in an internal buffer and it doesn't talk
>> > to the external world in any way.
>> >
>> > Right now it's very simple: it supports only writes. But it can be
>> > easily extended to support more operations.
>> >
>> > This is going to be used by the monitor's "HMP passthrough via QMP"
>> > feature, which needs to run monitor handlers without a backing
>> > device.
>> >
>> > Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
>> > ---
>> >  qemu-char.c |   66 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >  qemu-char.h |    6 +++++
>> >  2 files changed, 72 insertions(+), 0 deletions(-)
>> >
>> > diff --git a/qemu-char.c b/qemu-char.c
>> > index 88997f9..896df14 100644
>> > --- a/qemu-char.c
>> > +++ b/qemu-char.c
>> > @@ -2275,6 +2275,72 @@ static CharDriverState *qemu_chr_open_socket(QemuOpts *opts)
>> >      return NULL;
>> >  }
>> >  
>> > +/***********************************************************/
>> > +/* Memory chardev */
>> > +typedef struct {
>> > +    size_t outbuf_size;
>> > +    size_t outbuf_capacity;
>> > +    uint8_t *outbuf;
>> > +} MemoryDriver;
>> > +
>> > +static int mem_chr_write(CharDriverState *chr, const uint8_t *buf, int len)
>> > +{
>> > +    MemoryDriver *d = chr->opaque;
>> > +
>> > +    /* TODO: the QString implementation has the same code, we should
>> > +     * introduce a generic way to do this in cutils.c */
>> > +    if (d->outbuf_capacity < d->outbuf_size + len) {
>> > +        /* grown outbuf */
>> 
>> Used to say "grow" (sans n) here.  Intentional change?
>
> Hum, no. I think I've squashed an older commit while rebasing (but this seems
> to be the only problem).
>
>> 
>> > +        d->outbuf_capacity += len;
>> > +        d->outbuf_capacity *= 2;
>> > +        d->outbuf = qemu_realloc(d->outbuf, d->outbuf_capacity);
>> > +    }
>> > +
>> > +    memcpy(d->outbuf + d->outbuf_size, buf, len);
>> > +    d->outbuf_size += len;
>> > +
>> > +    return len;
>> > +}
>> > +
>> > +#define DEFAULT_BUF_SIZE 4096
>> 
>> It's the *initial* buffer size, isn't it?
>
> Yes.

Could we make the name reflect that then?

>> Doubt it's worth a #define (there's just one user), but that's a matter
>> of taste.
>> 
>> > +
>> > +void qemu_chr_init_mem(CharDriverState *chr)
>> > +{
>> > +    MemoryDriver *d;
>> > +
>> > +    d = qemu_malloc(sizeof(*d));
>> > +    d->outbuf_size = 0;
>> > +    d->outbuf_capacity = DEFAULT_BUF_SIZE;
>> > +    d->outbuf = qemu_mallocz(d->outbuf_capacity);
>> > +
>> > +    memset(chr, 0, sizeof(*chr));
>> > +    chr->opaque = d;
>> > +    chr->chr_write = mem_chr_write;
>> > +}
>> > +
>> > +/* assumes the stored data is a string */
>> 
>> What else could it be?  Worrying about embedded '\0's?
>
> Yes, as the driver itself doesn't interpret the contents of its
> buffer.

What happens if there are embedded '\0's?

>> > +QString *qemu_chr_mem_to_qs(CharDriverState *chr)
>> > +{
>> > +    MemoryDriver *d = chr->opaque;
>> > +
>> > +    if (d->outbuf_size == 0) {
>> > +        return NULL;
>> > +    }
>> 
>> Did you forget to change this?  We agreed to return an empty QString
>> when chr contains an empty string.
>
> I've changed my mind and forgot to mention it: I thought that we would
> need to return NULL on error conditions, but turns out that this function
> never fails.
>
> So, I do think it's better to let it that way for two reasons:
>
>  1. An empty has at least the '\0' character, but in this case the buffer
>     is really empty

qstring_from_substr() copies the contents of the buffer (any length
works, including 0), then appends a '\0'.  I'm afraid I don't get the
problem here...

>  2. Returning an empty string for this case will add unneeded complexity
>     to the caller, ie. checking if the QString's length is 0 and decref'ing it

I strongly recommend not to screw up the interface of a generally useful
function like qemu_chr_mem_to_qs() just to make its initial user
marginally simpler.

If you decide not to follow my recommendation, please document the
unusual mapping of empty string to null pointer in a function comment.

>> > +
>> > +    return qstring_from_substr((char *) d->outbuf, 0, d->outbuf_size - 1);
>> > +}
>> > +
>> > +/* NOTE: this driver can not be closed with qemu_chr_close()! */
>> > +void qemu_chr_close_mem(CharDriverState *chr)
>> > +{
>> > +    MemoryDriver *d = chr->opaque;
>> > +
>> > +    qemu_free(d->outbuf);
>> > +    qemu_free(chr->opaque);
>> > +    chr->opaque = NULL;
>> > +    chr->chr_write = NULL;
>> > +}
>> > +
>> 
>> Unlike normal character drivers, this one can't be closed with
>> qemu_chr_close().  It probably explodes if you try.  Please add a
>> suitable assertion to qemu_chr_close() to document the fact, and to
>> ensure misuse fails in a controlled, obvious manner.
>
> Ah forgot, but that can be done as a separate patch, so if I don't respin
> this series I'll send an additional patch for that.

Okay.

>> 
>> >  QemuOpts *qemu_chr_parse_compat(const char *label, const char *filename)
>> >  {
>> >      char host[65], port[33], width[8], height[8];
>> > diff --git a/qemu-char.h b/qemu-char.h
>> > index 18ad12b..c4e55b4 100644
>> > --- a/qemu-char.h
>> > +++ b/qemu-char.h
>> > @@ -6,6 +6,7 @@
>> >  #include "qemu-option.h"
>> >  #include "qemu-config.h"
>> >  #include "qobject.h"
>> > +#include "qstring.h"
>> >  
>> >  /* character device */
>> >  
>> > @@ -100,6 +101,11 @@ CharDriverState *qemu_chr_open_eventfd(int eventfd);
>> >  
>> >  extern int term_escape_char;
>> >  
>> > +/* memory chardev */
>> > +void qemu_chr_init_mem(CharDriverState *chr);
>> > +void qemu_chr_close_mem(CharDriverState *chr);
>> > +QString *qemu_chr_mem_to_qs(CharDriverState *chr);
>> > +
>> >  /* async I/O support */
>> >  
>> >  int qemu_set_fd_handler2(int fd,
>>

On Thu, 11 Nov 2010 17:32:06 +0100
Markus Armbruster <armbru@redhat.com> wrote:

> Luiz Capitulino <lcapitulino@redhat.com> writes:
> 
> > On Thu, 11 Nov 2010 16:30:26 +0100
> > Markus Armbruster <armbru@redhat.com> wrote:
> >
> >> Luiz Capitulino <lcapitulino@redhat.com> writes:
> >> 
> >> > This driver handles in-memory chardev operations. That's, all writes
> >> > to this driver are stored in an internal buffer and it doesn't talk
> >> > to the external world in any way.
> >> >
> >> > Right now it's very simple: it supports only writes. But it can be
> >> > easily extended to support more operations.
> >> >
> >> > This is going to be used by the monitor's "HMP passthrough via QMP"
> >> > feature, which needs to run monitor handlers without a backing
> >> > device.
> >> >
> >> > Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
> >> > ---
> >> >  qemu-char.c |   66 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> >  qemu-char.h |    6 +++++
> >> >  2 files changed, 72 insertions(+), 0 deletions(-)
> >> >
> >> > diff --git a/qemu-char.c b/qemu-char.c
> >> > index 88997f9..896df14 100644
> >> > --- a/qemu-char.c
> >> > +++ b/qemu-char.c
> >> > @@ -2275,6 +2275,72 @@ static CharDriverState *qemu_chr_open_socket(QemuOpts *opts)
> >> >      return NULL;
> >> >  }
> >> >  
> >> > +/***********************************************************/
> >> > +/* Memory chardev */
> >> > +typedef struct {
> >> > +    size_t outbuf_size;
> >> > +    size_t outbuf_capacity;
> >> > +    uint8_t *outbuf;
> >> > +} MemoryDriver;
> >> > +
> >> > +static int mem_chr_write(CharDriverState *chr, const uint8_t *buf, int len)
> >> > +{
> >> > +    MemoryDriver *d = chr->opaque;
> >> > +
> >> > +    /* TODO: the QString implementation has the same code, we should
> >> > +     * introduce a generic way to do this in cutils.c */
> >> > +    if (d->outbuf_capacity < d->outbuf_size + len) {
> >> > +        /* grown outbuf */
> >> 
> >> Used to say "grow" (sans n) here.  Intentional change?
> >
> > Hum, no. I think I've squashed an older commit while rebasing (but this seems
> > to be the only problem).
> >
> >> 
> >> > +        d->outbuf_capacity += len;
> >> > +        d->outbuf_capacity *= 2;
> >> > +        d->outbuf = qemu_realloc(d->outbuf, d->outbuf_capacity);
> >> > +    }
> >> > +
> >> > +    memcpy(d->outbuf + d->outbuf_size, buf, len);
> >> > +    d->outbuf_size += len;
> >> > +
> >> > +    return len;
> >> > +}
> >> > +
> >> > +#define DEFAULT_BUF_SIZE 4096
> >> 
> >> It's the *initial* buffer size, isn't it?
> >
> > Yes.
> 
> Could we make the name reflect that then?
> 
> >> Doubt it's worth a #define (there's just one user), but that's a matter
> >> of taste.
> >> 
> >> > +
> >> > +void qemu_chr_init_mem(CharDriverState *chr)
> >> > +{
> >> > +    MemoryDriver *d;
> >> > +
> >> > +    d = qemu_malloc(sizeof(*d));
> >> > +    d->outbuf_size = 0;
> >> > +    d->outbuf_capacity = DEFAULT_BUF_SIZE;
> >> > +    d->outbuf = qemu_mallocz(d->outbuf_capacity);
> >> > +
> >> > +    memset(chr, 0, sizeof(*chr));
> >> > +    chr->opaque = d;
> >> > +    chr->chr_write = mem_chr_write;
> >> > +}
> >> > +
> >> > +/* assumes the stored data is a string */
> >> 
> >> What else could it be?  Worrying about embedded '\0's?
> >
> > Yes, as the driver itself doesn't interpret the contents of its
> > buffer.
> 
> What happens if there are embedded '\0's?

The string will be shorter than expected? And what if it contains
non-printable characters?

It's just a cautionary comment to help the user identify such problems, I think
we're making a whole argument about a quite minor thing.

> 
> >> > +QString *qemu_chr_mem_to_qs(CharDriverState *chr)
> >> > +{
> >> > +    MemoryDriver *d = chr->opaque;
> >> > +
> >> > +    if (d->outbuf_size == 0) {
> >> > +        return NULL;
> >> > +    }
> >> 
> >> Did you forget to change this?  We agreed to return an empty QString
> >> when chr contains an empty string.
> >
> > I've changed my mind and forgot to mention it: I thought that we would
> > need to return NULL on error conditions, but turns out that this function
> > never fails.
> >
> > So, I do think it's better to let it that way for two reasons:
> >
> >  1. An empty has at least the '\0' character, but in this case the buffer
> >     is really empty
> 
> qstring_from_substr() copies the contents of the buffer (any length
> works, including 0), then appends a '\0'.  I'm afraid I don't get the
> problem here...
> 
> >  2. Returning an empty string for this case will add unneeded complexity
> >     to the caller, ie. checking if the QString's length is 0 and decref'ing it
> 
> I strongly recommend not to screw up the interface of a generally useful
> function like qemu_chr_mem_to_qs() just to make its initial user
> marginally simpler.

Okay, found a different way of doing this that should make us both happy.

Very exciting changes in v3!

> 
> If you decide not to follow my recommendation, please document the
> unusual mapping of empty string to null pointer in a function comment.
> 
> >> > +
> >> > +    return qstring_from_substr((char *) d->outbuf, 0, d->outbuf_size - 1);
> >> > +}
> >> > +
> >> > +/* NOTE: this driver can not be closed with qemu_chr_close()! */
> >> > +void qemu_chr_close_mem(CharDriverState *chr)
> >> > +{
> >> > +    MemoryDriver *d = chr->opaque;
> >> > +
> >> > +    qemu_free(d->outbuf);
> >> > +    qemu_free(chr->opaque);
> >> > +    chr->opaque = NULL;
> >> > +    chr->chr_write = NULL;
> >> > +}
> >> > +
> >> 
> >> Unlike normal character drivers, this one can't be closed with
> >> qemu_chr_close().  It probably explodes if you try.  Please add a
> >> suitable assertion to qemu_chr_close() to document the fact, and to
> >> ensure misuse fails in a controlled, obvious manner.
> >
> > Ah forgot, but that can be done as a separate patch, so if I don't respin
> > this series I'll send an additional patch for that.
> 
> Okay.

Btw, what should we assert() for? We're going to have to access QTAILQ
member I guess.

> 
> >> 
> >> >  QemuOpts *qemu_chr_parse_compat(const char *label, const char *filename)
> >> >  {
> >> >      char host[65], port[33], width[8], height[8];
> >> > diff --git a/qemu-char.h b/qemu-char.h
> >> > index 18ad12b..c4e55b4 100644
> >> > --- a/qemu-char.h
> >> > +++ b/qemu-char.h
> >> > @@ -6,6 +6,7 @@
> >> >  #include "qemu-option.h"
> >> >  #include "qemu-config.h"
> >> >  #include "qobject.h"
> >> > +#include "qstring.h"
> >> >  
> >> >  /* character device */
> >> >  
> >> > @@ -100,6 +101,11 @@ CharDriverState *qemu_chr_open_eventfd(int eventfd);
> >> >  
> >> >  extern int term_escape_char;
> >> >  
> >> > +/* memory chardev */
> >> > +void qemu_chr_init_mem(CharDriverState *chr);
> >> > +void qemu_chr_close_mem(CharDriverState *chr);
> >> > +QString *qemu_chr_mem_to_qs(CharDriverState *chr);
> >> > +
> >> >  /* async I/O support */
> >> >  
> >> >  int qemu_set_fd_handler2(int fd,
> >> 
>

Luiz Capitulino <lcapitulino@redhat.com> writes:

> On Thu, 11 Nov 2010 17:32:06 +0100
> Markus Armbruster <armbru@redhat.com> wrote:
>
>> Luiz Capitulino <lcapitulino@redhat.com> writes:
>> 
>> > On Thu, 11 Nov 2010 16:30:26 +0100
>> > Markus Armbruster <armbru@redhat.com> wrote:
>> >
>> >> Luiz Capitulino <lcapitulino@redhat.com> writes:
>> >> 
>> >> > This driver handles in-memory chardev operations. That's, all writes
>> >> > to this driver are stored in an internal buffer and it doesn't talk
>> >> > to the external world in any way.
>> >> >
>> >> > Right now it's very simple: it supports only writes. But it can be
>> >> > easily extended to support more operations.
>> >> >
>> >> > This is going to be used by the monitor's "HMP passthrough via QMP"
>> >> > feature, which needs to run monitor handlers without a backing
>> >> > device.
>> >> >
>> >> > Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
>> >> > ---
>> >> >  qemu-char.c |   66 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >> >  qemu-char.h |    6 +++++
>> >> >  2 files changed, 72 insertions(+), 0 deletions(-)
>> >> >
>> >> > diff --git a/qemu-char.c b/qemu-char.c
>> >> > index 88997f9..896df14 100644
>> >> > --- a/qemu-char.c
>> >> > +++ b/qemu-char.c
>> >> > @@ -2275,6 +2275,72 @@ static CharDriverState *qemu_chr_open_socket(QemuOpts *opts)
>> >> >      return NULL;
>> >> >  }
>> >> >  
>> >> > +/***********************************************************/
>> >> > +/* Memory chardev */
>> >> > +typedef struct {
>> >> > +    size_t outbuf_size;
>> >> > +    size_t outbuf_capacity;
>> >> > +    uint8_t *outbuf;
>> >> > +} MemoryDriver;
>> >> > +
>> >> > +static int mem_chr_write(CharDriverState *chr, const uint8_t *buf, int len)
>> >> > +{
>> >> > +    MemoryDriver *d = chr->opaque;
>> >> > +
>> >> > +    /* TODO: the QString implementation has the same code, we should
>> >> > +     * introduce a generic way to do this in cutils.c */
>> >> > +    if (d->outbuf_capacity < d->outbuf_size + len) {
>> >> > +        /* grown outbuf */
>> >> 
>> >> Used to say "grow" (sans n) here.  Intentional change?
>> >
>> > Hum, no. I think I've squashed an older commit while rebasing (but this seems
>> > to be the only problem).
>> >
>> >> 
>> >> > +        d->outbuf_capacity += len;
>> >> > +        d->outbuf_capacity *= 2;
>> >> > +        d->outbuf = qemu_realloc(d->outbuf, d->outbuf_capacity);
>> >> > +    }
>> >> > +
>> >> > +    memcpy(d->outbuf + d->outbuf_size, buf, len);
>> >> > +    d->outbuf_size += len;
>> >> > +
>> >> > +    return len;
>> >> > +}
>> >> > +
>> >> > +#define DEFAULT_BUF_SIZE 4096
>> >> 
>> >> It's the *initial* buffer size, isn't it?
>> >
>> > Yes.
>> 
>> Could we make the name reflect that then?
>> 
>> >> Doubt it's worth a #define (there's just one user), but that's a matter
>> >> of taste.
>> >> 
>> >> > +
>> >> > +void qemu_chr_init_mem(CharDriverState *chr)
>> >> > +{
>> >> > +    MemoryDriver *d;
>> >> > +
>> >> > +    d = qemu_malloc(sizeof(*d));
>> >> > +    d->outbuf_size = 0;
>> >> > +    d->outbuf_capacity = DEFAULT_BUF_SIZE;
>> >> > +    d->outbuf = qemu_mallocz(d->outbuf_capacity);
>> >> > +
>> >> > +    memset(chr, 0, sizeof(*chr));
>> >> > +    chr->opaque = d;
>> >> > +    chr->chr_write = mem_chr_write;
>> >> > +}
>> >> > +
>> >> > +/* assumes the stored data is a string */
>> >> 
>> >> What else could it be?  Worrying about embedded '\0's?
>> >
>> > Yes, as the driver itself doesn't interpret the contents of its
>> > buffer.
>> 
>> What happens if there are embedded '\0's?
>
> The string will be shorter than expected? And what if it contains
> non-printable characters?
>
> It's just a cautionary comment to help the user identify such problems, I think
> we're making a whole argument about a quite minor thing.

When I see "assumes X" in a function comment, I immediately ask "and
what happens when !X?"  The default answer is "it explodes, so don't do
that".  That answer is wrong here.  Therefore, I find the comment
misleading.

Let's figure out what really happens.  The human command's output is
sent to the client as a JSON string (response object member return).
JSON strings can consist of Unicode characters, "except for the
characters that must be escaped: quotation mark, reverse solidus, and
the control characters (U+0000 through U+001F)" (RFC 4627, section 2.5).

Do we escape these characters?  Where in the code?

>> >> > +QString *qemu_chr_mem_to_qs(CharDriverState *chr)
>> >> > +{
>> >> > +    MemoryDriver *d = chr->opaque;
>> >> > +
>> >> > +    if (d->outbuf_size == 0) {
>> >> > +        return NULL;
>> >> > +    }
>> >> 
>> >> Did you forget to change this?  We agreed to return an empty QString
>> >> when chr contains an empty string.
>> >
>> > I've changed my mind and forgot to mention it: I thought that we would
>> > need to return NULL on error conditions, but turns out that this function
>> > never fails.
>> >
>> > So, I do think it's better to let it that way for two reasons:
>> >
>> >  1. An empty has at least the '\0' character, but in this case the buffer
>> >     is really empty
>> 
>> qstring_from_substr() copies the contents of the buffer (any length
>> works, including 0), then appends a '\0'.  I'm afraid I don't get the
>> problem here...
>> 
>> >  2. Returning an empty string for this case will add unneeded complexity
>> >     to the caller, ie. checking if the QString's length is 0 and decref'ing it
>> 
>> I strongly recommend not to screw up the interface of a generally useful
>> function like qemu_chr_mem_to_qs() just to make its initial user
>> marginally simpler.
>
> Okay, found a different way of doing this that should make us both happy.
>
> Very exciting changes in v3!
>
>> 
>> If you decide not to follow my recommendation, please document the
>> unusual mapping of empty string to null pointer in a function comment.
>> 
>> >> > +
>> >> > +    return qstring_from_substr((char *) d->outbuf, 0, d->outbuf_size - 1);
>> >> > +}
>> >> > +
>> >> > +/* NOTE: this driver can not be closed with qemu_chr_close()! */
>> >> > +void qemu_chr_close_mem(CharDriverState *chr)
>> >> > +{
>> >> > +    MemoryDriver *d = chr->opaque;
>> >> > +
>> >> > +    qemu_free(d->outbuf);
>> >> > +    qemu_free(chr->opaque);
>> >> > +    chr->opaque = NULL;
>> >> > +    chr->chr_write = NULL;
>> >> > +}
>> >> > +
>> >> 
>> >> Unlike normal character drivers, this one can't be closed with
>> >> qemu_chr_close().  It probably explodes if you try.  Please add a
>> >> suitable assertion to qemu_chr_close() to document the fact, and to
>> >> ensure misuse fails in a controlled, obvious manner.
>> >
>> > Ah forgot, but that can be done as a separate patch, so if I don't respin
>> > this series I'll send an additional patch for that.
>> 
>> Okay.
>
> Btw, what should we assert() for? We're going to have to access QTAILQ
> member I guess.

Fair question.

Semantically, we assert that close is safe and does the job.

For your memory driver, it's not safe, because the QTAILQ_REMOVE() is
safe only when chr is in chardevs, which it isn't.  And it doesn't do
the job, because it doesn't free resources.

We can detect the "not safe" condition: search chardevs for chr.  Might
want to put it in a function qemu_chr_is_internal().

If we don't want to search, we can add a flag that reflects "is in
chardevs".

Taking a step back: "external" character devices are in chardevs, and
are to be closed with qemu_chr_close().  "internal" ones are not, and
are to be closed differently.

[...]

On Fri, 12 Nov 2010 11:16:54 +0100
Markus Armbruster <armbru@redhat.com> wrote:

> Luiz Capitulino <lcapitulino@redhat.com> writes:
> 
> > On Thu, 11 Nov 2010 17:32:06 +0100
> > Markus Armbruster <armbru@redhat.com> wrote:
> >
> >> Luiz Capitulino <lcapitulino@redhat.com> writes:
> >> 
> >> > On Thu, 11 Nov 2010 16:30:26 +0100
> >> > Markus Armbruster <armbru@redhat.com> wrote:
> >> >
> >> >> Luiz Capitulino <lcapitulino@redhat.com> writes:
> >> >> 
> >> >> > This driver handles in-memory chardev operations. That's, all writes
> >> >> > to this driver are stored in an internal buffer and it doesn't talk
> >> >> > to the external world in any way.
> >> >> >
> >> >> > Right now it's very simple: it supports only writes. But it can be
> >> >> > easily extended to support more operations.
> >> >> >
> >> >> > This is going to be used by the monitor's "HMP passthrough via QMP"
> >> >> > feature, which needs to run monitor handlers without a backing
> >> >> > device.
> >> >> >
> >> >> > Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
> >> >> > ---
> >> >> >  qemu-char.c |   66 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> >> >  qemu-char.h |    6 +++++
> >> >> >  2 files changed, 72 insertions(+), 0 deletions(-)
> >> >> >
> >> >> > diff --git a/qemu-char.c b/qemu-char.c
> >> >> > index 88997f9..896df14 100644
> >> >> > --- a/qemu-char.c
> >> >> > +++ b/qemu-char.c
> >> >> > @@ -2275,6 +2275,72 @@ static CharDriverState *qemu_chr_open_socket(QemuOpts *opts)
> >> >> >      return NULL;
> >> >> >  }
> >> >> >  
> >> >> > +/***********************************************************/
> >> >> > +/* Memory chardev */
> >> >> > +typedef struct {
> >> >> > +    size_t outbuf_size;
> >> >> > +    size_t outbuf_capacity;
> >> >> > +    uint8_t *outbuf;
> >> >> > +} MemoryDriver;
> >> >> > +
> >> >> > +static int mem_chr_write(CharDriverState *chr, const uint8_t *buf, int len)
> >> >> > +{
> >> >> > +    MemoryDriver *d = chr->opaque;
> >> >> > +
> >> >> > +    /* TODO: the QString implementation has the same code, we should
> >> >> > +     * introduce a generic way to do this in cutils.c */
> >> >> > +    if (d->outbuf_capacity < d->outbuf_size + len) {
> >> >> > +        /* grown outbuf */
> >> >> 
> >> >> Used to say "grow" (sans n) here.  Intentional change?
> >> >
> >> > Hum, no. I think I've squashed an older commit while rebasing (but this seems
> >> > to be the only problem).
> >> >
> >> >> 
> >> >> > +        d->outbuf_capacity += len;
> >> >> > +        d->outbuf_capacity *= 2;
> >> >> > +        d->outbuf = qemu_realloc(d->outbuf, d->outbuf_capacity);
> >> >> > +    }
> >> >> > +
> >> >> > +    memcpy(d->outbuf + d->outbuf_size, buf, len);
> >> >> > +    d->outbuf_size += len;
> >> >> > +
> >> >> > +    return len;
> >> >> > +}
> >> >> > +
> >> >> > +#define DEFAULT_BUF_SIZE 4096
> >> >> 
> >> >> It's the *initial* buffer size, isn't it?
> >> >
> >> > Yes.
> >> 
> >> Could we make the name reflect that then?
> >> 
> >> >> Doubt it's worth a #define (there's just one user), but that's a matter
> >> >> of taste.
> >> >> 
> >> >> > +
> >> >> > +void qemu_chr_init_mem(CharDriverState *chr)
> >> >> > +{
> >> >> > +    MemoryDriver *d;
> >> >> > +
> >> >> > +    d = qemu_malloc(sizeof(*d));
> >> >> > +    d->outbuf_size = 0;
> >> >> > +    d->outbuf_capacity = DEFAULT_BUF_SIZE;
> >> >> > +    d->outbuf = qemu_mallocz(d->outbuf_capacity);
> >> >> > +
> >> >> > +    memset(chr, 0, sizeof(*chr));
> >> >> > +    chr->opaque = d;
> >> >> > +    chr->chr_write = mem_chr_write;
> >> >> > +}
> >> >> > +
> >> >> > +/* assumes the stored data is a string */
> >> >> 
> >> >> What else could it be?  Worrying about embedded '\0's?
> >> >
> >> > Yes, as the driver itself doesn't interpret the contents of its
> >> > buffer.
> >> 
> >> What happens if there are embedded '\0's?
> >
> > The string will be shorter than expected? And what if it contains
> > non-printable characters?
> >
> > It's just a cautionary comment to help the user identify such problems, I think
> > we're making a whole argument about a quite minor thing.
> 
> When I see "assumes X" in a function comment, I immediately ask "and
> what happens when !X?"  The default answer is "it explodes, so don't do
> that".  That answer is wrong here.  Therefore, I find the comment
> misleading.

That's how you interpret it, my interpretation is that I might not get
the expected behavior.

> Let's figure out what really happens.  The human command's output is
> sent to the client as a JSON string (response object member return).
> JSON strings can consist of Unicode characters, "except for the
> characters that must be escaped: quotation mark, reverse solidus, and
> the control characters (U+0000 through U+001F)" (RFC 4627, section 2.5).
> 
> Do we escape these characters?  Where in the code?

Should be in the json parser, but qemu_chr_mem_to_qs() doesn't assume its
users (and it obviously shouldn't).

> 
> >> >> > +QString *qemu_chr_mem_to_qs(CharDriverState *chr)
> >> >> > +{
> >> >> > +    MemoryDriver *d = chr->opaque;
> >> >> > +
> >> >> > +    if (d->outbuf_size == 0) {
> >> >> > +        return NULL;
> >> >> > +    }
> >> >> 
> >> >> Did you forget to change this?  We agreed to return an empty QString
> >> >> when chr contains an empty string.
> >> >
> >> > I've changed my mind and forgot to mention it: I thought that we would
> >> > need to return NULL on error conditions, but turns out that this function
> >> > never fails.
> >> >
> >> > So, I do think it's better to let it that way for two reasons:
> >> >
> >> >  1. An empty has at least the '\0' character, but in this case the buffer
> >> >     is really empty
> >> 
> >> qstring_from_substr() copies the contents of the buffer (any length
> >> works, including 0), then appends a '\0'.  I'm afraid I don't get the
> >> problem here...
> >> 
> >> >  2. Returning an empty string for this case will add unneeded complexity
> >> >     to the caller, ie. checking if the QString's length is 0 and decref'ing it
> >> 
> >> I strongly recommend not to screw up the interface of a generally useful
> >> function like qemu_chr_mem_to_qs() just to make its initial user
> >> marginally simpler.
> >
> > Okay, found a different way of doing this that should make us both happy.
> >
> > Very exciting changes in v3!
> >
> >> 
> >> If you decide not to follow my recommendation, please document the
> >> unusual mapping of empty string to null pointer in a function comment.
> >> 
> >> >> > +
> >> >> > +    return qstring_from_substr((char *) d->outbuf, 0, d->outbuf_size - 1);
> >> >> > +}
> >> >> > +
> >> >> > +/* NOTE: this driver can not be closed with qemu_chr_close()! */
> >> >> > +void qemu_chr_close_mem(CharDriverState *chr)
> >> >> > +{
> >> >> > +    MemoryDriver *d = chr->opaque;
> >> >> > +
> >> >> > +    qemu_free(d->outbuf);
> >> >> > +    qemu_free(chr->opaque);
> >> >> > +    chr->opaque = NULL;
> >> >> > +    chr->chr_write = NULL;
> >> >> > +}
> >> >> > +
> >> >> 
> >> >> Unlike normal character drivers, this one can't be closed with
> >> >> qemu_chr_close().  It probably explodes if you try.  Please add a
> >> >> suitable assertion to qemu_chr_close() to document the fact, and to
> >> >> ensure misuse fails in a controlled, obvious manner.
> >> >
> >> > Ah forgot, but that can be done as a separate patch, so if I don't respin
> >> > this series I'll send an additional patch for that.
> >> 
> >> Okay.
> >
> > Btw, what should we assert() for? We're going to have to access QTAILQ
> > member I guess.
> 
> Fair question.
> 
> Semantically, we assert that close is safe and does the job.
> 
> For your memory driver, it's not safe, because the QTAILQ_REMOVE() is
> safe only when chr is in chardevs, which it isn't.  And it doesn't do
> the job, because it doesn't free resources.
> 
> We can detect the "not safe" condition: search chardevs for chr.  Might
> want to put it in a function qemu_chr_is_internal().
> 
> If we don't want to search, we can add a flag that reflects "is in
> chardevs".
> 
> Taking a step back: "external" character devices are in chardevs, and
> are to be closed with qemu_chr_close().  "internal" ones are not, and
> are to be closed differently.
> 
> [...]
>

Luiz Capitulino <lcapitulino@redhat.com> writes:

> On Fri, 12 Nov 2010 11:16:54 +0100
> Markus Armbruster <armbru@redhat.com> wrote:
>
>> Luiz Capitulino <lcapitulino@redhat.com> writes:
>> 
>> > On Thu, 11 Nov 2010 17:32:06 +0100
>> > Markus Armbruster <armbru@redhat.com> wrote:
>> >
>> >> Luiz Capitulino <lcapitulino@redhat.com> writes:
>> >> 
>> >> > On Thu, 11 Nov 2010 16:30:26 +0100
>> >> > Markus Armbruster <armbru@redhat.com> wrote:
>> >> >
>> >> >> Luiz Capitulino <lcapitulino@redhat.com> writes:
>> >> >> 
>> >> >> > This driver handles in-memory chardev operations. That's, all writes
>> >> >> > to this driver are stored in an internal buffer and it doesn't talk
>> >> >> > to the external world in any way.
>> >> >> >
>> >> >> > Right now it's very simple: it supports only writes. But it can be
>> >> >> > easily extended to support more operations.
>> >> >> >
>> >> >> > This is going to be used by the monitor's "HMP passthrough via QMP"
>> >> >> > feature, which needs to run monitor handlers without a backing
>> >> >> > device.
>> >> >> >
>> >> >> > Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
>> >> >> > ---
>> >> >> >  qemu-char.c |   66 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >> >> >  qemu-char.h |    6 +++++
>> >> >> >  2 files changed, 72 insertions(+), 0 deletions(-)
>> >> >> >
>> >> >> > diff --git a/qemu-char.c b/qemu-char.c
>> >> >> > index 88997f9..896df14 100644
>> >> >> > --- a/qemu-char.c
>> >> >> > +++ b/qemu-char.c
>> >> >> > @@ -2275,6 +2275,72 @@ static CharDriverState *qemu_chr_open_socket(QemuOpts *opts)
>> >> >> >      return NULL;
>> >> >> >  }
>> >> >> >  
>> >> >> > +/***********************************************************/
>> >> >> > +/* Memory chardev */
>> >> >> > +typedef struct {
>> >> >> > +    size_t outbuf_size;
>> >> >> > +    size_t outbuf_capacity;
>> >> >> > +    uint8_t *outbuf;
>> >> >> > +} MemoryDriver;
>> >> >> > +
>> >> >> > +static int mem_chr_write(CharDriverState *chr, const uint8_t *buf, int len)
>> >> >> > +{
>> >> >> > +    MemoryDriver *d = chr->opaque;
>> >> >> > +
>> >> >> > +    /* TODO: the QString implementation has the same code, we should
>> >> >> > +     * introduce a generic way to do this in cutils.c */
>> >> >> > +    if (d->outbuf_capacity < d->outbuf_size + len) {
>> >> >> > +        /* grown outbuf */
>> >> >> 
>> >> >> Used to say "grow" (sans n) here.  Intentional change?
>> >> >
>> >> > Hum, no. I think I've squashed an older commit while rebasing (but this seems
>> >> > to be the only problem).
>> >> >
>> >> >> 
>> >> >> > +        d->outbuf_capacity += len;
>> >> >> > +        d->outbuf_capacity *= 2;
>> >> >> > +        d->outbuf = qemu_realloc(d->outbuf, d->outbuf_capacity);
>> >> >> > +    }
>> >> >> > +
>> >> >> > +    memcpy(d->outbuf + d->outbuf_size, buf, len);
>> >> >> > +    d->outbuf_size += len;
>> >> >> > +
>> >> >> > +    return len;
>> >> >> > +}
>> >> >> > +
>> >> >> > +#define DEFAULT_BUF_SIZE 4096
>> >> >> 
>> >> >> It's the *initial* buffer size, isn't it?
>> >> >
>> >> > Yes.
>> >> 
>> >> Could we make the name reflect that then?
>> >> 
>> >> >> Doubt it's worth a #define (there's just one user), but that's a matter
>> >> >> of taste.
>> >> >> 
>> >> >> > +
>> >> >> > +void qemu_chr_init_mem(CharDriverState *chr)
>> >> >> > +{
>> >> >> > +    MemoryDriver *d;
>> >> >> > +
>> >> >> > +    d = qemu_malloc(sizeof(*d));
>> >> >> > +    d->outbuf_size = 0;
>> >> >> > +    d->outbuf_capacity = DEFAULT_BUF_SIZE;
>> >> >> > +    d->outbuf = qemu_mallocz(d->outbuf_capacity);
>> >> >> > +
>> >> >> > +    memset(chr, 0, sizeof(*chr));
>> >> >> > +    chr->opaque = d;
>> >> >> > +    chr->chr_write = mem_chr_write;
>> >> >> > +}
>> >> >> > +
>> >> >> > +/* assumes the stored data is a string */
>> >> >> 
>> >> >> What else could it be?  Worrying about embedded '\0's?
>> >> >
>> >> > Yes, as the driver itself doesn't interpret the contents of its
>> >> > buffer.
>> >> 
>> >> What happens if there are embedded '\0's?
>> >
>> > The string will be shorter than expected? And what if it contains
>> > non-printable characters?
>> >
>> > It's just a cautionary comment to help the user identify such problems, I think
>> > we're making a whole argument about a quite minor thing.
>> 
>> When I see "assumes X" in a function comment, I immediately ask "and
>> what happens when !X?"  The default answer is "it explodes, so don't do
>> that".  That answer is wrong here.  Therefore, I find the comment
>> misleading.
>
> That's how you interpret it, my interpretation is that I might not get
> the expected behavior.

Actually, this function works just fine for embedded '\0's (I tested
it): it returns the correct QString, with full length and '\0' embedded.

Only later, when we attempt to put that QString on the wire do we screw
up, in to_json().  It fails to consider the length, and stops at the
first 0.  In fact, there's not even a way to get the length of a
QString!  There's only qstring_get_str().  I'd call that an API bug.
You might call it a restriction instead ;)

If anything needs a comment, it's qobject_to_json().  But I think that
one needs a bug fix instead.

Alternatively, we could document that QString and its users can't cope
with embedded '\0'.

>> Let's figure out what really happens.  The human command's output is
>> sent to the client as a JSON string (response object member return).
>> JSON strings can consist of Unicode characters, "except for the
>> characters that must be escaped: quotation mark, reverse solidus, and
>> the control characters (U+0000 through U+001F)" (RFC 4627, section 2.5).
>> 
>> Do we escape these characters?  Where in the code?
>
> Should be in the json parser, but qemu_chr_mem_to_qs() doesn't assume its
> users (and it obviously shouldn't).

It's in to_json().

[...]

On Fri, 12 Nov 2010 16:54:14 +0100
Markus Armbruster <armbru@redhat.com> wrote:

> Luiz Capitulino <lcapitulino@redhat.com> writes:
> 
> > On Fri, 12 Nov 2010 11:16:54 +0100
> > Markus Armbruster <armbru@redhat.com> wrote:
> >
> >> Luiz Capitulino <lcapitulino@redhat.com> writes:
> >> 
> >> > On Thu, 11 Nov 2010 17:32:06 +0100
> >> > Markus Armbruster <armbru@redhat.com> wrote:
> >> >
> >> >> Luiz Capitulino <lcapitulino@redhat.com> writes:
> >> >> 
> >> >> > On Thu, 11 Nov 2010 16:30:26 +0100
> >> >> > Markus Armbruster <armbru@redhat.com> wrote:
> >> >> >
> >> >> >> Luiz Capitulino <lcapitulino@redhat.com> writes:
> >> >> >> 
> >> >> >> > This driver handles in-memory chardev operations. That's, all writes
> >> >> >> > to this driver are stored in an internal buffer and it doesn't talk
> >> >> >> > to the external world in any way.
> >> >> >> >
> >> >> >> > Right now it's very simple: it supports only writes. But it can be
> >> >> >> > easily extended to support more operations.
> >> >> >> >
> >> >> >> > This is going to be used by the monitor's "HMP passthrough via QMP"
> >> >> >> > feature, which needs to run monitor handlers without a backing
> >> >> >> > device.
> >> >> >> >
> >> >> >> > Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
> >> >> >> > ---
> >> >> >> >  qemu-char.c |   66 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> >> >> >  qemu-char.h |    6 +++++
> >> >> >> >  2 files changed, 72 insertions(+), 0 deletions(-)
> >> >> >> >
> >> >> >> > diff --git a/qemu-char.c b/qemu-char.c
> >> >> >> > index 88997f9..896df14 100644
> >> >> >> > --- a/qemu-char.c
> >> >> >> > +++ b/qemu-char.c
> >> >> >> > @@ -2275,6 +2275,72 @@ static CharDriverState *qemu_chr_open_socket(QemuOpts *opts)
> >> >> >> >      return NULL;
> >> >> >> >  }
> >> >> >> >  
> >> >> >> > +/***********************************************************/
> >> >> >> > +/* Memory chardev */
> >> >> >> > +typedef struct {
> >> >> >> > +    size_t outbuf_size;
> >> >> >> > +    size_t outbuf_capacity;
> >> >> >> > +    uint8_t *outbuf;
> >> >> >> > +} MemoryDriver;
> >> >> >> > +
> >> >> >> > +static int mem_chr_write(CharDriverState *chr, const uint8_t *buf, int len)
> >> >> >> > +{
> >> >> >> > +    MemoryDriver *d = chr->opaque;
> >> >> >> > +
> >> >> >> > +    /* TODO: the QString implementation has the same code, we should
> >> >> >> > +     * introduce a generic way to do this in cutils.c */
> >> >> >> > +    if (d->outbuf_capacity < d->outbuf_size + len) {
> >> >> >> > +        /* grown outbuf */
> >> >> >> 
> >> >> >> Used to say "grow" (sans n) here.  Intentional change?
> >> >> >
> >> >> > Hum, no. I think I've squashed an older commit while rebasing (but this seems
> >> >> > to be the only problem).
> >> >> >
> >> >> >> 
> >> >> >> > +        d->outbuf_capacity += len;
> >> >> >> > +        d->outbuf_capacity *= 2;
> >> >> >> > +        d->outbuf = qemu_realloc(d->outbuf, d->outbuf_capacity);
> >> >> >> > +    }
> >> >> >> > +
> >> >> >> > +    memcpy(d->outbuf + d->outbuf_size, buf, len);
> >> >> >> > +    d->outbuf_size += len;
> >> >> >> > +
> >> >> >> > +    return len;
> >> >> >> > +}
> >> >> >> > +
> >> >> >> > +#define DEFAULT_BUF_SIZE 4096
> >> >> >> 
> >> >> >> It's the *initial* buffer size, isn't it?
> >> >> >
> >> >> > Yes.
> >> >> 
> >> >> Could we make the name reflect that then?
> >> >> 
> >> >> >> Doubt it's worth a #define (there's just one user), but that's a matter
> >> >> >> of taste.
> >> >> >> 
> >> >> >> > +
> >> >> >> > +void qemu_chr_init_mem(CharDriverState *chr)
> >> >> >> > +{
> >> >> >> > +    MemoryDriver *d;
> >> >> >> > +
> >> >> >> > +    d = qemu_malloc(sizeof(*d));
> >> >> >> > +    d->outbuf_size = 0;
> >> >> >> > +    d->outbuf_capacity = DEFAULT_BUF_SIZE;
> >> >> >> > +    d->outbuf = qemu_mallocz(d->outbuf_capacity);
> >> >> >> > +
> >> >> >> > +    memset(chr, 0, sizeof(*chr));
> >> >> >> > +    chr->opaque = d;
> >> >> >> > +    chr->chr_write = mem_chr_write;
> >> >> >> > +}
> >> >> >> > +
> >> >> >> > +/* assumes the stored data is a string */
> >> >> >> 
> >> >> >> What else could it be?  Worrying about embedded '\0's?
> >> >> >
> >> >> > Yes, as the driver itself doesn't interpret the contents of its
> >> >> > buffer.
> >> >> 
> >> >> What happens if there are embedded '\0's?
> >> >
> >> > The string will be shorter than expected? And what if it contains
> >> > non-printable characters?
> >> >
> >> > It's just a cautionary comment to help the user identify such problems, I think
> >> > we're making a whole argument about a quite minor thing.
> >> 
> >> When I see "assumes X" in a function comment, I immediately ask "and
> >> what happens when !X?"  The default answer is "it explodes, so don't do
> >> that".  That answer is wrong here.  Therefore, I find the comment
> >> misleading.
> >
> > That's how you interpret it, my interpretation is that I might not get
> > the expected behavior.
> 
> Actually, this function works just fine for embedded '\0's (I tested
> it): it returns the correct QString, with full length and '\0' embedded.

Good.

> Only later, when we attempt to put that QString on the wire do we screw
> up, in to_json().  It fails to consider the length, and stops at the
> first 0.  In fact, there's not even a way to get the length of a
> QString!  There's only qstring_get_str().  I'd call that an API bug.
> You might call it a restriction instead ;)

Whatever it is, let's do what has to be done: just add it.

> If anything needs a comment, it's qobject_to_json().  But I think that
> one needs a bug fix instead.

Care to send a patch then?

> Alternatively, we could document that QString and its users can't cope
> with embedded '\0'.

That depend on QString users, doesn't it?

> 
> >> Let's figure out what really happens.  The human command's output is
> >> sent to the client as a JSON string (response object member return).
> >> JSON strings can consist of Unicode characters, "except for the
> >> characters that must be escaped: quotation mark, reverse solidus, and
> >> the control characters (U+0000 through U+001F)" (RFC 4627, section 2.5).
> >> 
> >> Do we escape these characters?  Where in the code?
> >
> > Should be in the json parser, but qemu_chr_mem_to_qs() doesn't assume its
> > users (and it obviously shouldn't).
> 
> It's in to_json().
> 
> [...]
>