Patchwork [RFC] fix crashes with vmware driver

login
register
mail settings
Submitter Serge Hallyn
Date March 2, 2012, 9:11 p.m.
Message ID <20120302211122.GA9652@vostro.hallyn.com>
Download mbox | patch
Permalink /patch/144359/
State New
Headers show

Comments

Serge Hallyn - March 2, 2012, 9:11 p.m.
Hi,

I don't know where the best place to catch this would be, but
with vnc and vmware_vga it's possible to get set_bit called on
a negative index, crashing qemu.  See

https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/918791

for details.  This patch prevents that.  It's possible this
should be caught earlier, but this patch works for me.

Signed-off-by: Serge Hallyn <serge.hallyn@canonical.com>
---
 vmware_vga.c |   16 ++++++++++++++++
 1 file changed, 16 insertions(+)
Ryan Harper - March 2, 2012, 9:20 p.m.
* Serge Hallyn <serge.hallyn@canonical.com> [2012-03-02 15:13]:
> Hi,
> 
> I don't know where the best place to catch this would be, but
> with vnc and vmware_vga it's possible to get set_bit called on
> a negative index, crashing qemu.  See
> 
> https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/918791
> 
> for details.  This patch prevents that.  It's possible this
> should be caught earlier, but this patch works for me.
> 
> Signed-off-by: Serge Hallyn <serge.hallyn@canonical.com>
> ---
>  vmware_vga.c |   16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
> 
> Index: qemu-kvm-1.0+noroms/hw/vmware_vga.c
> ===================================================================
> --- qemu-kvm-1.0+noroms.orig/hw/vmware_vga.c	2012-03-01 16:19:23.280571798 -0600
> +++ qemu-kvm-1.0+noroms/hw/vmware_vga.c	2012-03-01 16:27:27.910975006 -0600
> @@ -298,6 +298,22 @@
>      uint8_t *src;
>      uint8_t *dst;
> 
> +    if (x < 0) {
> +        fprintf(stderr, "%s: update x was < 0 (%d, w %d)\n",
> +                        __FUNCTION__, x, w);
> +        w += x;
> +	if (w < 0)
> +		return;
> +        x = 0;
> +    }
> +    if (y < 0) {
> +        fprintf(stderr, "%s: update y was < 0 (%d, h %d)\n",
> +                        __FUNCTION__, y, h);
> +        h += y;
> +	if (h < 0)
> +		return;
> +        y = 0;
> +    }

Looks like it has mixed spaces and tabs.

CODING_STYLE wants {} on all if's 



>      if (x + w > s->width) {
>          fprintf(stderr, "%s: update width too large x: %d, w: %d\n",
>                          __FUNCTION__, x, w);
Serge E. Hallyn - March 5, 2012, 7:33 p.m.
Thanks.  As there's been no substantial feedback, I'll resend with
those changes.

-serge

Quoting Ryan Harper (ryanh@us.ibm.com):
> * Serge Hallyn <serge.hallyn@canonical.com> [2012-03-02 15:13]:
> > Hi,
> > 
> > I don't know where the best place to catch this would be, but
> > with vnc and vmware_vga it's possible to get set_bit called on
> > a negative index, crashing qemu.  See
> > 
> > https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/918791
> > 
> > for details.  This patch prevents that.  It's possible this
> > should be caught earlier, but this patch works for me.
> > 
> > Signed-off-by: Serge Hallyn <serge.hallyn@canonical.com>
> > ---
> >  vmware_vga.c |   16 ++++++++++++++++
> >  1 file changed, 16 insertions(+)
> > 
> > Index: qemu-kvm-1.0+noroms/hw/vmware_vga.c
> > ===================================================================
> > --- qemu-kvm-1.0+noroms.orig/hw/vmware_vga.c	2012-03-01 16:19:23.280571798 -0600
> > +++ qemu-kvm-1.0+noroms/hw/vmware_vga.c	2012-03-01 16:27:27.910975006 -0600
> > @@ -298,6 +298,22 @@
> >      uint8_t *src;
> >      uint8_t *dst;
> > 
> > +    if (x < 0) {
> > +        fprintf(stderr, "%s: update x was < 0 (%d, w %d)\n",
> > +                        __FUNCTION__, x, w);
> > +        w += x;
> > +	if (w < 0)
> > +		return;
> > +        x = 0;
> > +    }
> > +    if (y < 0) {
> > +        fprintf(stderr, "%s: update y was < 0 (%d, h %d)\n",
> > +                        __FUNCTION__, y, h);
> > +        h += y;
> > +	if (h < 0)
> > +		return;
> > +        y = 0;
> > +    }
> 
> Looks like it has mixed spaces and tabs.
> 
> CODING_STYLE wants {} on all if's 
> 
> 
> 
> >      if (x + w > s->width) {
> >          fprintf(stderr, "%s: update width too large x: %d, w: %d\n",
> >                          __FUNCTION__, x, w);
> 
> -- 
> Ryan Harper
> Software Engineer; Linux Technology Center
> IBM Corp., Austin, Tx
> ryanh@us.ibm.com
>

Patch

Index: qemu-kvm-1.0+noroms/hw/vmware_vga.c
===================================================================
--- qemu-kvm-1.0+noroms.orig/hw/vmware_vga.c	2012-03-01 16:19:23.280571798 -0600
+++ qemu-kvm-1.0+noroms/hw/vmware_vga.c	2012-03-01 16:27:27.910975006 -0600
@@ -298,6 +298,22 @@ 
     uint8_t *src;
     uint8_t *dst;
 
+    if (x < 0) {
+        fprintf(stderr, "%s: update x was < 0 (%d, w %d)\n",
+                        __FUNCTION__, x, w);
+        w += x;
+	if (w < 0)
+		return;
+        x = 0;
+    }
+    if (y < 0) {
+        fprintf(stderr, "%s: update y was < 0 (%d, h %d)\n",
+                        __FUNCTION__, y, h);
+        h += y;
+	if (h < 0)
+		return;
+        y = 0;
+    }
     if (x + w > s->width) {
         fprintf(stderr, "%s: update width too large x: %d, w: %d\n",
                         __FUNCTION__, x, w);