From patchwork Tue Mar 25 17:15:30 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kamal Mostafa X-Patchwork-Id: 333555 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) by ozlabs.org (Postfix) with ESMTP id A5E5B140087 for ; Wed, 26 Mar 2014 04:18:13 +1100 (EST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.76) (envelope-from ) id 1WSUzJ-0006pL-3L; Tue, 25 Mar 2014 17:18:09 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtp (Exim 4.76) (envelope-from ) id 1WSUyS-0006UK-35 for kernel-team@lists.ubuntu.com; Tue, 25 Mar 2014 17:17:16 +0000 Received: from c-67-160-228-185.hsd1.ca.comcast.net ([67.160.228.185] helo=fourier) by youngberry.canonical.com with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1WSUyR-0000pV-AW; Tue, 25 Mar 2014 17:17:15 +0000 Received: from kamal by fourier with local (Exim 4.80) (envelope-from ) id 1WSUyP-0001ML-Ic; Tue, 25 Mar 2014 10:17:13 -0700 From: Kamal Mostafa To: linux-kernel@vger.kernel.org, stable@vger.kernel.org, kernel-team@lists.ubuntu.com Subject: [PATCH 3.8 17/81] drm/vmwgfx: Fix possible integer overflow Date: Tue, 25 Mar 2014 10:15:30 -0700 Message-Id: <1395767794-5012-18-git-send-email-kamal@canonical.com> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1395767794-5012-1-git-send-email-kamal@canonical.com> References: <1395767794-5012-1-git-send-email-kamal@canonical.com> X-Extended-Stable: 3.8 Cc: Thomas Hellstrom , Kamal Mostafa X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.14 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: kernel-team-bounces@lists.ubuntu.com 3.8.13.20 -stable review patch. If anyone has any objections, please let me know. ------------------ From: Thomas Hellstrom commit b055211d941eb6cb7b285be580cf6e51a01d2f44 upstream. Reported-by: Brian Paul Signed-off-by: Thomas Hellstrom Reviewed-by: Jakob Bornecrantz Signed-off-by: Kamal Mostafa --- drivers/gpu/drm/vmwgfx/svga3d_surfacedefs.h | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/vmwgfx/svga3d_surfacedefs.h b/drivers/gpu/drm/vmwgfx/svga3d_surfacedefs.h index 8369c3b..ef33850 100644 --- a/drivers/gpu/drm/vmwgfx/svga3d_surfacedefs.h +++ b/drivers/gpu/drm/vmwgfx/svga3d_surfacedefs.h @@ -38,8 +38,11 @@ #define DIV_ROUND_UP(x, y) (((x) + (y) - 1) / (y)) #define max_t(type, x, y) ((x) > (y) ? (x) : (y)) +#define min_t(type, x, y) ((x) < (y) ? (x) : (y)) #define surf_size_struct SVGA3dSize #define u32 uint32 +#define u64 uint64_t +#define U32_MAX ((u32)~0U) #endif /* __KERNEL__ */ @@ -704,8 +707,8 @@ static const struct svga3d_surface_desc svga3d_surface_descs[] = { static inline u32 clamped_umul32(u32 a, u32 b) { - uint64_t tmp = (uint64_t) a*b; - return (tmp > (uint64_t) ((u32) -1)) ? (u32) -1 : tmp; + u64 tmp = (u64) a*b; + return (tmp > (u64) U32_MAX) ? U32_MAX : tmp; } static inline const struct svga3d_surface_desc * @@ -834,7 +837,7 @@ svga3dsurface_get_serialized_size(SVGA3dSurfaceFormat format, bool cubemap) { const struct svga3d_surface_desc *desc = svga3dsurface_get_desc(format); - u32 total_size = 0; + u64 total_size = 0; u32 mip; for (mip = 0; mip < num_mip_levels; mip++) { @@ -847,7 +850,7 @@ svga3dsurface_get_serialized_size(SVGA3dSurfaceFormat format, if (cubemap) total_size *= SVGA3D_MAX_SURFACE_FACES; - return total_size; + return (u32) min_t(u64, total_size, (u64) U32_MAX); }