Patchwork vmware_vga: Don't crash on too-big DEFINE_CURSOR command

login
register
mail settings
Submitter Roland Dreier
Date Dec. 17, 2009, 9:02 p.m.
Message ID <ada7hsl5n6o.fsf@roland-alpha.cisco.com>
Download mbox | patch
Permalink /patch/41369/
State New
Headers show

Comments

Roland Dreier - Dec. 17, 2009, 9:02 p.m.
QEMU crashes with vmware_vga when running a Linux guest with the latest
X.org vmware video driver if QEMU is using SDL for video output.  In
this case, vmware_vga advertises cursor acceleration to the guest, and
the crash comes when the guest does a DEFINE_CURSOR command with a 64x64
32bpp cursor.  This request overruns the image[] array in struct
vmsvga_cursor_definition_s and QEMU ends up segfaulting because of
memory corruption caused by writing past the end of the array.

Fix this by enlarging the image[] array to be able to hold 4096 32-bit
pixels so we don't fail for the case of 64*64*32bpp, and also add error
checking to avoid a crash if an even bigger request is sent by a guest.

Signed-off-by: Roland Dreier <rolandd@cisco.com>
---
 hw/vmware_vga.c |   11 ++++++++++-
 1 files changed, 10 insertions(+), 1 deletions(-)

Patch

diff --git a/hw/vmware_vga.c b/hw/vmware_vga.c
index f3e3749..d253a2e 100644
--- a/hw/vmware_vga.c
+++ b/hw/vmware_vga.c
@@ -462,7 +462,7 @@  struct vmsvga_cursor_definition_s {
     int hot_x;
     int hot_y;
     uint32_t mask[1024];
-    uint32_t image[1024];
+    uint32_t image[4096];	/* allow for 64x64 32bpp cursor */
 };
 
 #define SVGA_BITMAP_SIZE(w, h)		((((w) + 31) >> 5) * (h))
@@ -557,6 +557,15 @@  static void vmsvga_fifo_run(struct vmsvga_state_s *s)
             cursor.height = y = vmsvga_fifo_read(s);
             vmsvga_fifo_read(s);
             cursor.bpp = vmsvga_fifo_read(s);
+
+	    if (SVGA_BITMAP_SIZE(x, y) > sizeof cursor.mask / sizeof (uint32_t) ||
+		SVGA_PIXMAP_SIZE(x, y, cursor.bpp) > sizeof cursor.image / sizeof (uint32_t)) {
+		    fprintf(stderr, "%s: DEFINE_CSURSOR too large x: %d, y: %d bpp: %d\n",
+			    __FUNCTION__, x, y, cursor.bpp);
+		    args = SVGA_BITMAP_SIZE(x, y) + SVGA_PIXMAP_SIZE(x, y, cursor.bpp);
+		    goto badcmd;
+	    }
+
             for (args = 0; args < SVGA_BITMAP_SIZE(x, y); args ++)
                 cursor.mask[args] = vmsvga_fifo_read_raw(s);
             for (args = 0; args < SVGA_PIXMAP_SIZE(x, y, cursor.bpp); args ++)