From patchwork Thu Dec 17 22:27:12 2009 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roland Dreier X-Patchwork-Id: 41370 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [199.232.76.165]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 679B9B6F0E for ; Fri, 18 Dec 2009 12:57:58 +1100 (EST) Received: from localhost ([127.0.0.1]:54355 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1NLS6N-0001VJ-LU for incoming@patchwork.ozlabs.org; Thu, 17 Dec 2009 20:57:55 -0500 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1NLOob-0002L3-2x for qemu-devel@nongnu.org; Thu, 17 Dec 2009 17:27:21 -0500 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1NLOoV-0002Jp-M6 for qemu-devel@nongnu.org; Thu, 17 Dec 2009 17:27:20 -0500 Received: from [199.232.76.173] (port=51118 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1NLOoV-0002Jm-Gu for qemu-devel@nongnu.org; Thu, 17 Dec 2009 17:27:15 -0500 Received: from sj-iport-1.cisco.com ([171.71.176.70]:45854) by monty-python.gnu.org with esmtps (TLS-1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.60) (envelope-from ) id 1NLOoV-0005eH-1K for qemu-devel@nongnu.org; Thu, 17 Dec 2009 17:27:15 -0500 Authentication-Results: sj-iport-1.cisco.com; dkim=neutral (message not signed) header.i=none X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApoEAIs+KkurRN+J/2dsb2JhbAC+cpcthC0E X-IronPort-AV: E=Sophos;i="4.47,415,1257120000"; d="scan'208";a="281081757" Received: from sj-core-3.cisco.com ([171.68.223.137]) by sj-iport-1.cisco.com with ESMTP; 17 Dec 2009 22:27:12 +0000 Received: from xbh-sjc-221.amer.cisco.com (xbh-sjc-221.cisco.com [128.107.191.63]) by sj-core-3.cisco.com (8.13.8/8.14.3) with ESMTP id nBHMRCdB013714; Thu, 17 Dec 2009 22:27:12 GMT Received: from xfe-sjc-212.amer.cisco.com ([171.70.151.187]) by xbh-sjc-221.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.3959); Thu, 17 Dec 2009 14:27:13 -0800 Received: from roland-alpha.cisco.com ([10.33.42.9]) by xfe-sjc-212.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.3959); Thu, 17 Dec 2009 14:27:12 -0800 Received: by roland-alpha.cisco.com (Postfix, from userid 33217) id 406901FD0B; Thu, 17 Dec 2009 14:27:12 -0800 (PST) From: Roland Dreier To: anthony@codemonkey.ws User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1 (gnu/linux) X-Message-Flag: Warning: May contain useful information Date: Thu, 17 Dec 2009 14:27:12 -0800 Message-ID: MIME-Version: 1.0 X-OriginalArrivalTime: 17 Dec 2009 22:27:12.0478 (UTC) FILETIME=[14250FE0:01CA7F68] X-detected-operating-system: by monty-python.gnu.org: Genre and OS details not recognized. X-Mailman-Approved-At: Thu, 17 Dec 2009 20:30:12 -0500 Cc: qemu-devel@nongnu.org Subject: [Qemu-devel] [PATCH to consider for 0.12] vmware_vga: Don't crash on too-big DEFINE_CURSOR command X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Hi Anthony -- just sent this patch to qemu-devel (although I don't see it in archives yet). Anyway I realize it is really really late given your release timeframe but I think the risk of this pretty minimal, and the patch fixes a crash in a pretty reasonable config (running a modern Linux distro with the fastest guest video adapter). So please consider this for 0.12. Another possibility would be to just take the part of the patch that bumps the array size in the structure, since that seems to have essentially 0 risk and fixes the crash in the case I've seen. Thanks, Roland === QEMU crashes with vmware_vga when running a Linux guest with the latest X.org vmware video driver if QEMU is using SDL for video output. In this case, vmware_vga advertises cursor acceleration to the guest, and the crash comes when the guest does a DEFINE_CURSOR command with a 64x64 32bpp cursor. This request overruns the image[] array in struct vmsvga_cursor_definition_s and QEMU ends up segfaulting because of memory corruption caused by writing past the end of the array. Fix this by enlarging the image[] array to be able to hold 4096 32-bit pixels so we don't fail for the case of 64*64*32bpp, and also add error checking to avoid a crash if an even bigger request is sent by a guest. Signed-off-by: Roland Dreier --- hw/vmware_vga.c | 11 ++++++++++- 1 files changed, 10 insertions(+), 1 deletions(-) diff --git a/hw/vmware_vga.c b/hw/vmware_vga.c index f3e3749..d253a2e 100644 --- a/hw/vmware_vga.c +++ b/hw/vmware_vga.c @@ -462,7 +462,7 @@ struct vmsvga_cursor_definition_s { int hot_x; int hot_y; uint32_t mask[1024]; - uint32_t image[1024]; + uint32_t image[4096]; /* allow for 64x64 32bpp cursor */ }; #define SVGA_BITMAP_SIZE(w, h) ((((w) + 31) >> 5) * (h)) @@ -557,6 +557,15 @@ static void vmsvga_fifo_run(struct vmsvga_state_s *s) cursor.height = y = vmsvga_fifo_read(s); vmsvga_fifo_read(s); cursor.bpp = vmsvga_fifo_read(s); + + if (SVGA_BITMAP_SIZE(x, y) > sizeof cursor.mask / sizeof (uint32_t) || + SVGA_PIXMAP_SIZE(x, y, cursor.bpp) > sizeof cursor.image / sizeof (uint32_t)) { + fprintf(stderr, "%s: DEFINE_CSURSOR too large x: %d, y: %d bpp: %d\n", + __FUNCTION__, x, y, cursor.bpp); + args = SVGA_BITMAP_SIZE(x, y) + SVGA_PIXMAP_SIZE(x, y, cursor.bpp); + goto badcmd; + } + for (args = 0; args < SVGA_BITMAP_SIZE(x, y); args ++) cursor.mask[args] = vmsvga_fifo_read_raw(s); for (args = 0; args < SVGA_PIXMAP_SIZE(x, y, cursor.bpp); args ++)