| Message ID | 20180205114938.15784-2-berrange@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | [1/4] ui: avoid risk of 32-bit int overflow in VNC buffer check | expand |
On 02/05/18 12:49, Daniel P. Berrangé wrote: > For very large framebuffers, it is theoretically possible for the result > of 'vs->throttle_output_offset * VNC_THROTTLE_OUTPUT_LIMIT_SCALE' to > exceed the size of a 32-bit int. For this to happen in practice, the > video RAM would have to be set to a large enough value, which is not > likely today. None the less we can be paranoid against future growth by > using division instead of multiplication when checking the limits. > > Reported-by: Laszlo Ersek <lersek@redhat.com> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> > --- > ui/vnc.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/ui/vnc.c b/ui/vnc.c > index 93731accb6..e14e524764 100644 > --- a/ui/vnc.c > +++ b/ui/vnc.c > @@ -1572,8 +1572,8 @@ void vnc_write(VncState *vs, const void *data, size_t len) > * handshake, or from the job thread's VncState clone > */ > if (vs->throttle_output_offset != 0 && > - vs->output.offset > (vs->throttle_output_offset * > - VNC_THROTTLE_OUTPUT_LIMIT_SCALE)) { > + (vs->output.offset / VNC_THROTTLE_OUTPUT_LIMIT_SCALE) > > + vs->throttle_output_offset) { > trace_vnc_client_output_limit(vs, vs->ioc, vs->output.offset, > vs->throttle_output_offset); > vnc_disconnect_start(vs); > Reviewed-by: Laszlo Ersek <lersek@redhat.com>
On 02/05/2018 08:49 AM, Daniel P. Berrangé wrote: > For very large framebuffers, it is theoretically possible for the result > of 'vs->throttle_output_offset * VNC_THROTTLE_OUTPUT_LIMIT_SCALE' to > exceed the size of a 32-bit int. For this to happen in practice, the > video RAM would have to be set to a large enough value, which is not > likely today. None the less we can be paranoid against future growth by > using division instead of multiplication when checking the limits. > > Reported-by: Laszlo Ersek <lersek@redhat.com> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> > --- > ui/vnc.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/ui/vnc.c b/ui/vnc.c > index 93731accb6..e14e524764 100644 > --- a/ui/vnc.c > +++ b/ui/vnc.c > @@ -1572,8 +1572,8 @@ void vnc_write(VncState *vs, const void *data, size_t len) > * handshake, or from the job thread's VncState clone > */ > if (vs->throttle_output_offset != 0 && > - vs->output.offset > (vs->throttle_output_offset * > - VNC_THROTTLE_OUTPUT_LIMIT_SCALE)) { > + (vs->output.offset / VNC_THROTTLE_OUTPUT_LIMIT_SCALE) > > + vs->throttle_output_offset) { > trace_vnc_client_output_limit(vs, vs->ioc, vs->output.offset, > vs->throttle_output_offset); > vnc_disconnect_start(vs); >
diff --git a/ui/vnc.c b/ui/vnc.c index 93731accb6..e14e524764 100644 --- a/ui/vnc.c +++ b/ui/vnc.c @@ -1572,8 +1572,8 @@ void vnc_write(VncState *vs, const void *data, size_t len) * handshake, or from the job thread's VncState clone */ if (vs->throttle_output_offset != 0 && - vs->output.offset > (vs->throttle_output_offset * - VNC_THROTTLE_OUTPUT_LIMIT_SCALE)) { + (vs->output.offset / VNC_THROTTLE_OUTPUT_LIMIT_SCALE) > + vs->throttle_output_offset) { trace_vnc_client_output_limit(vs, vs->ioc, vs->output.offset, vs->throttle_output_offset); vnc_disconnect_start(vs);
For very large framebuffers, it is theoretically possible for the result of 'vs->throttle_output_offset * VNC_THROTTLE_OUTPUT_LIMIT_SCALE' to exceed the size of a 32-bit int. For this to happen in practice, the video RAM would have to be set to a large enough value, which is not likely today. None the less we can be paranoid against future growth by using division instead of multiplication when checking the limits. Reported-by: Laszlo Ersek <lersek@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- ui/vnc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)