Patchwork Fix SIGFPE for vnc display of width/height = 1

login
register
mail settings
Submitter Chris Webb
Date March 8, 2010, 2:34 p.m.
Message ID <20100308143449.GA4084@arachsys.com>
Download mbox | patch
Permalink /patch/47117/
State New
Headers show

Comments

Chris Webb - March 8, 2010, 2:34 p.m.
During boot, the screen gets resized to height 1 and a mouse click at this
point will cause a division by zero when calculating the absolute pointer
position from the pixel (x, y). Return a click in the middle of the screen
instead in this case.

Signed-off-by: Chris Webb <chris@arachsys.com>
---
 vnc.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)
Chris Webb - March 8, 2010, 6:15 p.m.
Chris Webb <chris@arachsys.com> writes:

> During boot, the screen gets resized to height 1 and a mouse click at this
> point will cause a division by zero when calculating the absolute pointer
> position from the pixel (x, y). Return a click in the middle of the screen
> instead in this case.

I think this probably ought to be a candidate for 0.12-stable too. We're
seeing these crashes for real from time-to-time so it's not just a
theoretical problem.

Cheers,

Chris.
Anthony Liguori - March 17, 2010, 3:59 p.m.
On 03/08/2010 08:34 AM, Chris Webb wrote:
> During boot, the screen gets resized to height 1 and a mouse click at this
> point will cause a division by zero when calculating the absolute pointer
> position from the pixel (x, y). Return a click in the middle of the screen
> instead in this case.
>
> Signed-off-by: Chris Webb<chris@arachsys.com>
>    
Applied.  Thanks.

Regards,

Anthony Liguori
> ---
>   vnc.c |    6 ++++--
>   1 files changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/vnc.c b/vnc.c
> index 01353a9..676a707 100644
> --- a/vnc.c
> +++ b/vnc.c
> @@ -1457,8 +1457,10 @@ static void pointer_event(VncState *vs, int button_mask, int x, int y)
>           dz = 1;
>
>       if (vs->absolute) {
> -        kbd_mouse_event(x * 0x7FFF / (ds_get_width(vs->ds) - 1),
> -                        y * 0x7FFF / (ds_get_height(vs->ds) - 1),
> +        kbd_mouse_event(ds_get_width(vs->ds)>  1 ?
> +                          x * 0x7FFF / (ds_get_width(vs->ds) - 1) : 0x4000,
> +                        ds_get_height(vs->ds)>  1 ?
> +                          y * 0x7FFF / (ds_get_height(vs->ds) - 1) : 0x4000,
>                           dz, buttons);
>       } else if (vnc_has_feature(vs, VNC_FEATURE_POINTER_TYPE_CHANGE)) {
>           x -= 0x7FFF;
>
Alexander Graf - March 17, 2010, 4:55 p.m.
Anthony Liguori wrote:
> On 03/08/2010 08:34 AM, Chris Webb wrote:
>> During boot, the screen gets resized to height 1 and a mouse click at
>> this
>> point will cause a division by zero when calculating the absolute
>> pointer
>> position from the pixel (x, y). Return a click in the middle of the
>> screen
>> instead in this case.
>>
>> Signed-off-by: Chris Webb<chris@arachsys.com>
>>    
> Applied.  Thanks.

Also queued it to stable?


Alex

Patch

diff --git a/vnc.c b/vnc.c
index 01353a9..676a707 100644
--- a/vnc.c
+++ b/vnc.c
@@ -1457,8 +1457,10 @@  static void pointer_event(VncState *vs, int button_mask, int x, int y)
         dz = 1;
 
     if (vs->absolute) {
-        kbd_mouse_event(x * 0x7FFF / (ds_get_width(vs->ds) - 1),
-                        y * 0x7FFF / (ds_get_height(vs->ds) - 1),
+        kbd_mouse_event(ds_get_width(vs->ds) > 1 ?
+                          x * 0x7FFF / (ds_get_width(vs->ds) - 1) : 0x4000,
+                        ds_get_height(vs->ds) > 1 ?
+                          y * 0x7FFF / (ds_get_height(vs->ds) - 1) : 0x4000,
                         dz, buttons);
     } else if (vnc_has_feature(vs, VNC_FEATURE_POINTER_TYPE_CHANGE)) {
         x -= 0x7FFF;