[ovs-dev,2/2,v2] Work around Python/C JSON unicode differences

The OVS C-based JSON parser operates on bytes, so the parser_feed
function returns the number of bytes that are processed. The pure
Python JSON parser currently operates on unicode, so it expects
that Parser.feed() returns a number of characters. This difference
leads to parsing errors when unicode characters are passed to the
C JSON parser from Python.

Signed-off-by: Terry Wilson <twilson@redhat.com>
---
 python/ovs/jsonrpc.py | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

On Tue, Oct 9, 2018 at 5:33 PM Terry Wilson <twilson@redhat.com> wrote:
>
> The OVS C-based JSON parser operates on bytes, so the parser_feed
> function returns the number of bytes that are processed. The pure
> Python JSON parser currently operates on unicode, so it expects
> that Parser.feed() returns a number of characters. This difference
> leads to parsing errors when unicode characters are passed to the
> C JSON parser from Python.
>
> Signed-off-by: Terry Wilson <twilson@redhat.com>
> ---
>  python/ovs/jsonrpc.py | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/python/ovs/jsonrpc.py b/python/ovs/jsonrpc.py
> index 4873cff..1323ba7 100644
> --- a/python/ovs/jsonrpc.py
> +++ b/python/ovs/jsonrpc.py
> @@ -272,7 +272,8 @@ class Connection(object):
>                  # data, so we convert it here as soon as possible.
>                  if data and not error:
>                      try:
> -                        data = decoder.decode(data)
> +                        if six.PY3 or ovs.json.PARSER == ovs.json.PARSER_PY:
> +                            data = decoder.decode(data)
>                      except UnicodeError:
>                          error = errno.EILSEQ
>                  if error:
> @@ -298,7 +299,11 @@ class Connection(object):
>              else:
>                  if self.parser is None:
>                      self.parser = ovs.json.Parser()
> -                self.input = self.input[self.parser.feed(self.input):]
> +                if six.PY3 and ovs.json.PARSER == ovs.json.PARSER_C:
> +                    self.input = self.input.encode('utf-8')[
> +                        self.parser.feed(self.input):].decode()
> +                else:
> +                    self.input = self.input[self.parser.feed(self.input):]
>                  if self.parser.is_done():
>                      msg = self.__process_msg()
>                      if msg:
> --
> 1.8.3.1
>
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Acked-By: Lucas Alvares Gomes <lucasagomes@gmail.com>

On Tue, Oct 09, 2018 at 05:45:22PM +0100, Lucas Alvares Gomes wrote:
> On Tue, Oct 9, 2018 at 5:33 PM Terry Wilson <twilson@redhat.com> wrote:
> >
> > The OVS C-based JSON parser operates on bytes, so the parser_feed
> > function returns the number of bytes that are processed. The pure
> > Python JSON parser currently operates on unicode, so it expects
> > that Parser.feed() returns a number of characters. This difference
> > leads to parsing errors when unicode characters are passed to the
> > C JSON parser from Python.
> >
> > Signed-off-by: Terry Wilson <twilson@redhat.com>
> > ---
> >  python/ovs/jsonrpc.py | 9 +++++++--
> >  1 file changed, 7 insertions(+), 2 deletions(-)
> >
> > diff --git a/python/ovs/jsonrpc.py b/python/ovs/jsonrpc.py
> > index 4873cff..1323ba7 100644
> > --- a/python/ovs/jsonrpc.py
> > +++ b/python/ovs/jsonrpc.py
> > @@ -272,7 +272,8 @@ class Connection(object):
> >                  # data, so we convert it here as soon as possible.
> >                  if data and not error:
> >                      try:
> > -                        data = decoder.decode(data)
> > +                        if six.PY3 or ovs.json.PARSER == ovs.json.PARSER_PY:
> > +                            data = decoder.decode(data)
> >                      except UnicodeError:
> >                          error = errno.EILSEQ
> >                  if error:
> > @@ -298,7 +299,11 @@ class Connection(object):
> >              else:
> >                  if self.parser is None:
> >                      self.parser = ovs.json.Parser()
> > -                self.input = self.input[self.parser.feed(self.input):]
> > +                if six.PY3 and ovs.json.PARSER == ovs.json.PARSER_C:
> > +                    self.input = self.input.encode('utf-8')[
> > +                        self.parser.feed(self.input):].decode()
> > +                else:
> > +                    self.input = self.input[self.parser.feed(self.input):]
> >                  if self.parser.is_done():
> >                      msg = self.__process_msg()
> >                      if msg:
> > --
> > 1.8.3.1
> >
> > _______________________________________________
> > dev mailing list
> > dev@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> 
> Acked-By: Lucas Alvares Gomes <lucasagomes@gmail.com>

Thanks, Terry (and Lucas).  I applied these to master.