Patchwork [v2] ui/vnc: VA API based H.264 encoding for VNC

login
register
mail settings
Submitter David Verbeiren
Date Feb. 13, 2013, 10:45 a.m.
Message ID <1360752304-6746-1-git-send-email-david.verbeiren@intel.com>
Download mbox | patch
Permalink /patch/220115/
State New
Headers show

Comments

David Verbeiren - Feb. 13, 2013, 10:45 a.m.
This patch implements H.264 encoding of the VNC framebuffer updates
using hardware acceleration through the VA API.

This is experimental support to let the community explore the possibilities
offered by the potential bandwidth and latency reductions that H.264
encoding allows. This may be particularly useful for use cases such as
online gaming, hosted desktops, hosted set top boxes...
This patch provides the VNC server side support. Corresponding VNC
client side support is required. To this end, we are also contributing
patches to the gtk-vnc and libvncserver/libvncclient projects which can be
used to test this experimental feature.
See instructions below for how to build a test VNC client.

In case multiple regions are updated, only the first framebuffer
update message of the batch carries the H.264 frame data.
Subsequent update framebuffer messages will contain only the
coordinates and size of the other updated regions.

This is backwards compatible with standard VNC clients thanks to
the encoding scheme negotiation included in VNC. If the client doesn't
support H.264 encoding, the server will fall back to one of the usual
VNC encodings.

Instructions/Requirements:
* Currently only works with libva 1.0: use branch "v1.0-branch" for libva
and intel-driver. Those can be built as follows:
   cd libva
   git checkout v1.0-branch
   ./autogen.sh
   make
   sudo make install
   cd ..
   git clone git://anongit.freedesktop.org/vaapi/intel-driver
   cd intel-driver
   git checkout v1.0-branch
   ./autogen.sh
   make
   sudo make install
* A graphical environment must be running as the v1.0-branch of VA API
does not support headless operation.
* When using Intel integrated graphics, hardware encoding support requires
a 2nd generation (or later) i3, i5 or i7 processor ("Sandy Bridge" or
later), or similar, with enabled Intel(R) HD graphics.
See http://intellinuxgraphics.org/h264.html for details.

Instructions for building and using a gtk-vnc test client:
* Get gtk-vnc project
   git clone git://git.gnome.org/gtk-vnc
   cd gtk-vnc
   git checkout a4f1d1912090d5
* Download and apply (git apply <patch_file>) patch from:
   https://mail.gnome.org/archives/gtk-vnc-list/2013-February/msg00000.html
* Build
   ./autogen.sh --with-libva
   make -j4
* Run the client, for example:
   ./examples/gvncviewer <vm_host>:1

Instructions for building and using a libvncclient test client:
* Get LibVNCServer project
   git clone git://libvncserver.git.sourceforge.net/gitroot/libvncserver/libvncserver
   cd libvncserver
   git checkout 55bdab02574e3ac
* Download and apply (git apply <patch_file>) following two patches
  in sequence:
   http://sourceforge.net/mailarchive/message.php?msg_id=30323804
   http://sourceforge.net/mailarchive/message.php?msg_id=30327573
* Build:
   ./autogen.sh --with-libva
   make -j4
* Run the client, for example:
   ./client_examples/gtkvncviewer <vm_host>:5901

Signed-off-by: David Verbeiren <david.verbeiren@intel.com>
---
 Changes for v1->v2:
  * No more statics; most moved into the VncDisplayH264 struct.
  * All variable declarations at top of funcs
  * VA encoder init now performed as part of set_encodings() so we can
    fallback to another encoding in case VA init fails.
  * configure script now defaults to libva=""
  * (no code change) "VA H.264" RFB encoding type number is now registered
    with IANA at http://www.iana.org/assignments/rfb/rfb.xml

 Also note that you can now use clients based on either libvncclient
 or gtk-vnc as a patch for the latter project was also submitted.


 configure         |   39 ++++
 ui/Makefile.objs  |    1 +
 ui/vnc-enc-h264.c |  616 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 ui/vnc-enc-h264.h |   74 +++++++
 ui/vnc-jobs.c     |    6 +
 ui/vnc.c          |   26 +++
 ui/vnc.h          |   18 ++
 7 files changed, 780 insertions(+)
 create mode 100644 ui/vnc-enc-h264.c
 create mode 100644 ui/vnc-enc-h264.h
Blue Swirl - Feb. 13, 2013, 9:16 p.m.
On Wed, Feb 13, 2013 at 10:45 AM, David Verbeiren
<david.verbeiren@intel.com> wrote:
> This patch implements H.264 encoding of the VNC framebuffer updates
> using hardware acceleration through the VA API.
>
> This is experimental support to let the community explore the possibilities
> offered by the potential bandwidth and latency reductions that H.264
> encoding allows. This may be particularly useful for use cases such as
> online gaming, hosted desktops, hosted set top boxes...
> This patch provides the VNC server side support. Corresponding VNC
> client side support is required. To this end, we are also contributing
> patches to the gtk-vnc and libvncserver/libvncclient projects which can be
> used to test this experimental feature.
> See instructions below for how to build a test VNC client.
>
> In case multiple regions are updated, only the first framebuffer
> update message of the batch carries the H.264 frame data.
> Subsequent update framebuffer messages will contain only the
> coordinates and size of the other updated regions.
>
> This is backwards compatible with standard VNC clients thanks to
> the encoding scheme negotiation included in VNC. If the client doesn't
> support H.264 encoding, the server will fall back to one of the usual
> VNC encodings.
>
> Instructions/Requirements:
> * Currently only works with libva 1.0: use branch "v1.0-branch" for libva
> and intel-driver. Those can be built as follows:
>    cd libva
>    git checkout v1.0-branch
>    ./autogen.sh
>    make
>    sudo make install
>    cd ..
>    git clone git://anongit.freedesktop.org/vaapi/intel-driver
>    cd intel-driver
>    git checkout v1.0-branch
>    ./autogen.sh
>    make
>    sudo make install
> * A graphical environment must be running as the v1.0-branch of VA API
> does not support headless operation.
> * When using Intel integrated graphics, hardware encoding support requires
> a 2nd generation (or later) i3, i5 or i7 processor ("Sandy Bridge" or
> later), or similar, with enabled Intel(R) HD graphics.
> See http://intellinuxgraphics.org/h264.html for details.
>
> Instructions for building and using a gtk-vnc test client:
> * Get gtk-vnc project
>    git clone git://git.gnome.org/gtk-vnc
>    cd gtk-vnc
>    git checkout a4f1d1912090d5
> * Download and apply (git apply <patch_file>) patch from:
>    https://mail.gnome.org/archives/gtk-vnc-list/2013-February/msg00000.html
> * Build
>    ./autogen.sh --with-libva
>    make -j4
> * Run the client, for example:
>    ./examples/gvncviewer <vm_host>:1
>
> Instructions for building and using a libvncclient test client:
> * Get LibVNCServer project
>    git clone git://libvncserver.git.sourceforge.net/gitroot/libvncserver/libvncserver
>    cd libvncserver
>    git checkout 55bdab02574e3ac
> * Download and apply (git apply <patch_file>) following two patches
>   in sequence:
>    http://sourceforge.net/mailarchive/message.php?msg_id=30323804
>    http://sourceforge.net/mailarchive/message.php?msg_id=30327573
> * Build:
>    ./autogen.sh --with-libva
>    make -j4
> * Run the client, for example:
>    ./client_examples/gtkvncviewer <vm_host>:5901
>
> Signed-off-by: David Verbeiren <david.verbeiren@intel.com>
> ---
>  Changes for v1->v2:
>   * No more statics; most moved into the VncDisplayH264 struct.
>   * All variable declarations at top of funcs
>   * VA encoder init now performed as part of set_encodings() so we can
>     fallback to another encoding in case VA init fails.
>   * configure script now defaults to libva=""
>   * (no code change) "VA H.264" RFB encoding type number is now registered
>     with IANA at http://www.iana.org/assignments/rfb/rfb.xml
>
>  Also note that you can now use clients based on either libvncclient
>  or gtk-vnc as a patch for the latter project was also submitted.
>
>
>  configure         |   39 ++++
>  ui/Makefile.objs  |    1 +
>  ui/vnc-enc-h264.c |  616 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  ui/vnc-enc-h264.h |   74 +++++++
>  ui/vnc-jobs.c     |    6 +
>  ui/vnc.c          |   26 +++
>  ui/vnc.h          |   18 ++
>  7 files changed, 780 insertions(+)
>  create mode 100644 ui/vnc-enc-h264.c
>  create mode 100644 ui/vnc-enc-h264.h
>
> diff --git a/configure b/configure
> index 8789324..d742a6c 100755
> --- a/configure
> +++ b/configure
> @@ -213,6 +213,7 @@ pie=""
>  zero_malloc=""
>  trace_backend="nop"
>  trace_file="trace"
> +libva=""
>  spice=""
>  rbd=""
>  smartcard_nss=""
> @@ -771,6 +772,10 @@ for opt do
>    ;;
>    --enable-spice) spice="yes"
>    ;;
> +  --disable-libva) libva="no"
> +  ;;
> +  --enable-libva) libva="yes"
> +  ;;
>    --disable-libiscsi) libiscsi="no"
>    ;;
>    --enable-libiscsi) libiscsi="yes"
> @@ -1129,6 +1134,8 @@ echo "  --with-trace-file=NAME   Full PATH,NAME of file to store traces"
>  echo "                           Default:trace-<pid>"
>  echo "  --disable-spice          disable spice"
>  echo "  --enable-spice           enable spice"
> +echo "  --disable-libva          disable H.264 encoding with libva"
> +echo "  --enable-libva           enable H.264 encoding with libva"
>  echo "  --enable-rbd             enable building the rados block device (rbd)"
>  echo "  --disable-libiscsi       disable iscsi support"
>  echo "  --enable-libiscsi        enable iscsi support"
> @@ -2842,6 +2849,33 @@ EOF
>    fi
>  fi
>
> +##########################################
> +# libva probe
> +if test "$libva" != "no" ; then
> +  cat > $TMPC << EOF
> +#include <X11/Xlib.h>
> +#include <va/va.h>
> +#include <va/va_x11.h>
> +int main(void) { int major_ver, minor_ver;
> + Display *win_display = (Display *)XOpenDisplay(":0.0");
> + VADisplay va_dpy = vaGetDisplay(win_display);
> + vaInitialize(va_dpy, &major_ver, &minor_ver); return 0;
> +}
> +EOF
> +libva_cflags="$($pkg_config --cflags libva 2>/dev/null) $($pkg_config --cflags libva-x11 2>/dev/null) $($pkg_config --cflags x11 2>/dev/null)"
> +libva_libs="$($pkg_config --libs libva 2>/dev/null) $($pkg_config --libs libva-x11 2>/dev/null) $($pkg_config --libs x11 2>/dev/null)"
> +  if $pkg_config --atleast-version=0.26 libva >/dev/null 2>&1 && \
> +     compile_prog "$libva_cflags" "$libva_libs" ; then
> +    libva="yes"
> +    libs_softmmu="$libs_softmmu $libva_libs"
> +  else
> +    if test "$libva" = "yes" ; then
> +      feature_not_found "libva"
> +    fi
> +    libva="no"
> +  fi
> +fi
> +
>  # check for libcacard for smartcard support
>  smartcard_cflags=""
>  # TODO - what's the minimal nss version we support?
> @@ -3336,6 +3370,7 @@ echo "xfsctl support    $xfs"
>  echo "nss used          $smartcard_nss"
>  echo "usb net redir     $usb_redir"
>  echo "OpenGL support    $opengl"
> +echo "libva support     $libva"
>  echo "libiscsi support  $libiscsi"
>  echo "build guest agent $guest_agent"
>  echo "seccomp support   $seccomp"
> @@ -3626,6 +3661,10 @@ if test "$spice" = "yes" ; then
>    echo "CONFIG_SPICE=y" >> $config_host_mak
>  fi
>
> +if test "$libva" = "yes" ; then
> +  echo "CONFIG_VNC_LIBVA=y" >> $config_host_mak
> +fi
> +
>  if test "$smartcard_nss" = "yes" ; then
>    echo "CONFIG_SMARTCARD_NSS=y" >> $config_host_mak
>    echo "libcacard_libs=$libcacard_libs" >> $config_host_mak
> diff --git a/ui/Makefile.objs b/ui/Makefile.objs
> index d9db073..6b1bd1a 100644
> --- a/ui/Makefile.objs
> +++ b/ui/Makefile.objs
> @@ -5,6 +5,7 @@ vnc-obj-y += vnc-enc-zrle.o
>  vnc-obj-$(CONFIG_VNC_TLS) += vnc-tls.o vnc-auth-vencrypt.o
>  vnc-obj-$(CONFIG_VNC_SASL) += vnc-auth-sasl.o
>  vnc-obj-$(CONFIG_VNC_WS) += vnc-ws.o
> +vnc-obj-$(CONFIG_VNC_LIBVA) += vnc-enc-h264.o
>  vnc-obj-y += vnc-jobs.o
>
>  common-obj-y += keymaps.o console.o cursor.o input.o qemu-pixman.o
> diff --git a/ui/vnc-enc-h264.c b/ui/vnc-enc-h264.c
> new file mode 100644
> index 0000000..da9bcf0
> --- /dev/null
> +++ b/ui/vnc-enc-h264.c
> @@ -0,0 +1,616 @@
> +/*
> + * QEMU VNC display driver: VAAPI based H.264 encoding
> + *
> + * Copyright (c) 2012 Intel Corporation.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the
> + * "Software"), to deal in the Software without restriction, including
> + * without limitation the rights to use, copy, modify, merge, publish,
> + * distribute, sub license, and/or sell copies of the Software, and to
> + * permit persons to whom the Software is furnished to do so, subject to
> + * the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the
> + * next paragraph) shall be included in all copies or substantial portions
> + * of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
> + * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
> + * IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
> + * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
> + * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
> + * USE OR OTHER DEALINGS IN THE SOFTWARE.
> + */
> +
> +#include "vnc.h"
> +
> +#define CHECK_VASTATUS(va_status, func) do {                                   \
> +    if (va_status != VA_STATUS_SUCCESS) {                                      \
> +        fprintf(stderr, "%s:%s:%d failed (0x%x),exit\n", __func__, func,       \
> +                __LINE__, va_status);                                          \
> +        exit(1);                                                               \
> +    } else {                                                                   \
> +        /* fprintf(stderr, "%s:%s:%d success\n", __func__, func, __LINE__); */ \
> +    }                                                                          \
> +} while (0);
> +
> +#define CHECK_VASTATUS_RET(va_status, func, retval) do {                       \
> +    if (va_status != VA_STATUS_SUCCESS) {                                      \
> +        fprintf(stderr, "%s:%s:%d failed (0x%x),exit\n", __func__, func,       \
> +                __LINE__, va_status);                                          \
> +        return retval;                                                         \
> +    } else {                                                                   \
> +        /* fprintf(stderr, "%s:%s:%d success\n", __func__, func, __LINE__); */ \
> +    }                                                                          \
> +} while (0);
> +
> +
> +/* Forward declarations */
> +static int h264_encoder_reinit(VncDisplayH264 *h264, int w, int h, int pitch,
> +                               int depth);
> +static void h264_encoder_close(VncDisplayH264 *h264);
> +
> +static int h264_create_context(VncDisplayH264 *h264, int width, int height,
> +                               int pitch, int depth);
> +static void h264_release_context(VncDisplayH264 *h264);
> +
> +static void h264_prepare_input(VncDisplayH264 *h264,
> +                               const unsigned char *const in_buf,
> +                               const bool is_intra, const bool is_last);
> +static int h264_get_coded_buf_size(VncDisplayH264 *h264);
> +
> +static void h264_encode_frame(VncDisplayH264 *h264,
> +                              const unsigned char *const in_buf, const int last,
> +                              const bool is_intra, int *out_length,
> +                              unsigned char **out_buf);
> +static void rgba_to_surface(const VncDisplayH264 *h264,
> +                            const unsigned char *const inbuf,
> +                            VASurfaceID surface_id);
> +static int get_buffer_length(const unsigned char * const buffer,
> +                             int buffer_length);
> +
> +
> +int vnc_h264_encoder_init(VncDisplayH264 *h264)
> +{
> +    VAStatus va_status;
> +    int va_major_ver, va_minor_ver;
> +    VAEntrypoint entrypoints[5];
> +    int num_entrypoints, entrypoint;
> +    VAConfigAttrib attrib[2];
> +    int i;
> +
> +    if (h264->initialized) {
> +        return 0; /* Already initialized */
> +    }
> +
> +    /* Initialize current frame data */
> +    h264->cur_frame.data = NULL;
> +    h264->cur_frame.length = -1;
> +    h264->cur_frame.is_intra = -1;
> +    h264->cur_frame.count = -1;
> +    h264->cur_frame.width = -1;
> +    h264->cur_frame.height = -1;
> +
> +    /* Open local X11 display and VA display */
> +    h264->x11_dpy = XOpenDisplay(":0.0");

This assumes that the first display is always used. Why can't you use $DISPLAY?

> +    if (!h264->x11_dpy) {
> +        fprintf(stderr, "Could not open display :0.0 for VA encoding\n");
> +        return -1;
> +    }
> +    h264->va_dpy = vaGetDisplay(h264->x11_dpy);
> +    va_status = vaInitialize(h264->va_dpy, &va_major_ver, &va_minor_ver);
> +    CHECK_VASTATUS_RET(va_status, "vaInitialize", -1);
> +
> +    /* Check for Slice entrypoint */
> +    vaQueryConfigEntrypoints(h264->va_dpy, VAProfileH264Baseline, entrypoints,
> +                             &num_entrypoints);
> +
> +    for (entrypoint = 0; entrypoint < num_entrypoints; entrypoint++) {
> +        if (entrypoints[entrypoint] == VAEntrypointEncSlice) {
> +            break;
> +        }
> +    }
> +
> +    if (entrypoint == num_entrypoints) {
> +        VNC_DEBUG("Could not find Encoder Slice entrypoint\n");
> +        h264_encoder_close(h264);
> +        return -1;
> +    }
> +
> +    /* Set encode pipeline configuration */
> +    attrib[0].type = VAConfigAttribRTFormat;
> +    attrib[1].type = VAConfigAttribRateControl;
> +    vaGetConfigAttributes(h264->va_dpy, VAProfileH264Baseline,
> +                          VAEntrypointEncSlice, &attrib[0], 2);
> +
> +    if ((attrib[0].value & VA_RT_FORMAT_YUV420) == 0) {
> +        VNC_DEBUG("YUV420 format is not supported by VA encoder\n");
> +        h264_encoder_close(h264);
> +        return -1;
> +    }
> +    if ((attrib[1].value & VA_RC_VBR) == 0) {
> +        VNC_DEBUG("VBR mode is not supported by VA encoder\n");
> +        h264_encoder_close(h264);
> +        return -1;
> +    }
> +
> +    attrib[0].value = VA_RT_FORMAT_YUV420;
> +    attrib[1].value = VA_RC_VBR;
> +    va_status = vaCreateConfig(h264->va_dpy, VAProfileH264Baseline,
> +                               VAEntrypointEncSlice, &attrib[0], 2,
> +                               &h264->config);
> +    CHECK_VASTATUS_RET(va_status, "vaCreateConfig", -1);
> +
> +    /* Invalidate context */
> +    h264->context = VA_INVALID_ID;
> +    h264->seq_param = VA_INVALID_ID;
> +    h264->pic_param = VA_INVALID_ID;
> +    h264->slice_param = VA_INVALID_ID;
> +    h264->coded_buf = VA_INVALID_ID;
> +
> +    for (i = 0; i < SID_NUMBER; ++i) {
> +        h264->surface_ids[i] = VA_INVALID_ID;
> +    }
> +
> +    /* Misc */
> +    h264->frame_count = 0;
> +    h264->initialized = true;
> +
> +    return 0;
> +}
> +
> +static int h264_encoder_reinit(VncDisplayH264 *h264, int w, int h, int pitch,
> +                               int depth)
> +{
> +    h264_release_context(h264);
> +
> +    return h264_create_context(h264, w, h, pitch, depth);
> +}
> +
> +static void h264_encoder_close(VncDisplayH264 *h264)
> +{
> +    if (h264->config != VA_INVALID_ID) {
> +        vaDestroyConfig(h264->va_dpy, h264->config);
> +        h264->config = VA_INVALID_ID;
> +    }
> +    if (h264->va_dpy) {
> +        vaTerminate(h264->va_dpy);
> +        h264->va_dpy = NULL;
> +    }
> +    if (h264->x11_dpy) {
> +        XCloseDisplay(h264->x11_dpy);
> +        h264->x11_dpy = NULL;
> +    }
> +}
> +
> +int vnc_h264_send_framebuffer_update(VncState *vs, int ch_x,
> +                                     int ch_y, int ch_w, int ch_h)
> +{
> +    int frame_w, frame_h;
> +    int pitch, depth;
> +    VncDisplayH264 *h264 = &vs->vd->h264;
> +    VncH264Frame *cur_frame = &h264->cur_frame;
> +    int bytes_to_send;
> +
> +    VNC_DEBUG("%s: x=%d; y=%d; w=%d; h=%d\n", __func__, ch_x, ch_y,
> +              ch_w, ch_h);
> +
> +    frame_w = ds_get_width(vs->ds);
> +    frame_h = ds_get_height(vs->ds);
> +
> +    /* first call for this frame: perform encoding */
> +    if ((cur_frame->data == NULL) ||
> +        (cur_frame->count != h264->frame_count)) {
> +
> +        if (!h264->va_dpy || (h264->config == VA_INVALID_ID)) {
> +            fprintf(stderr, "VA H264 encoding selected but "
> +                            "H264 encoder not initialized!\n");
> +            return 0;
> +        }
> +
> +        if ((frame_w != vs->vd->h264.cur_frame.width) ||
> +            (frame_h != vs->vd->h264.cur_frame.height)) {
> +            pitch = ds_get_linesize(vs->ds);
> +            depth = ds_get_bytes_per_pixel(vs->ds);
> +
> +            VNC_DEBUG("%s: Resolution change: (%dx%d) => (%dx%d) "
> +                      "(depth: %d, pitch: %d)\n", __func__,
> +                      cur_frame->width, cur_frame->height,
> +                      frame_w, frame_h, depth, pitch);
> +
> +            if (h264_encoder_reinit(h264, frame_w, frame_h,
> +                                    pitch, depth) != 0) {
> +                fprintf(stderr, "Error re-initializing the VA encoder!\n");
> +                VNC_DEBUG("Error re-initializing the VA encoder\n");
> +                return 0;
> +            }
> +
> +            /* send full screen update on framebuffer resize/init */
> +            h264->force_intra = true;
> +
> +            cur_frame->width = frame_w;
> +            cur_frame->height = frame_h;
> +        }
> +
> +        /* Encode the frame and get the encoded frame and its size */
> +        cur_frame->is_intra = h264->force_intra ||
> +                                 ((h264->frame_count % 30) == 0);
> +        h264_encode_frame(h264, vnc_server_fb_ptr(vs->vd, 0, 0), 0,
> +                          cur_frame->is_intra,
> +                          &cur_frame->length, &cur_frame->data);
> +
> +        cur_frame->count = h264->frame_count;
> +        h264->force_intra = false;
> +    }
> +
> +    /* Send framebuffer update header for this rectangle */
> +    vnc_framebuffer_update(vs, ch_x, ch_y, ch_w, ch_h, VNC_ENCODING_VA_H264);
> +
> +    /* Send frame contents only once. Multiple updates for the same frame
> +     * are sent when there are multiple changed rectangles in the same
> +     * framebuffer update run (see vnc_update_client() in vnc.c). */
> +    bytes_to_send =
> +        (vs->h264_last_sent_frame != cur_frame->count) ? cur_frame->length : 0;
> +
> +    /* VNC VA H264 Header: frame data size | frame_type | w | h | data */
> +    vnc_write_s32(vs, bytes_to_send);
> +    vnc_write_s32(vs, cur_frame->is_intra ? 2 : 0);
> +    vnc_write_s32(vs, frame_w);
> +    vnc_write_s32(vs, frame_h);
> +    vnc_write(vs, cur_frame->data, bytes_to_send);
> +
> +    vs->h264_last_sent_frame = cur_frame->count;
> +
> +    return 1;
> +}
> +
> +
> +static int h264_get_coded_buf_size(VncDisplayH264 *h264)
> +{
> +    return h264->pic_width * h264->pic_height * 1.5;
> +}
> +
> +static int h264_create_context(VncDisplayH264 *h264, int width, int height,
> +                               int pitch, int depth)
> +{
> +    VAStatus va_status;
> +    unsigned int coded_buf_size;
> +    int i;
> +
> +    VNC_DEBUG("%s: width=%d; height=%d; pitch=%d; depth=%d\n", __func__,
> +              width, height, pitch, depth);
> +
> +    if (h264->context != VA_INVALID_ID) {
> +        VNC_DEBUG("%s: context already exists, doing nothing\n", __func__);
> +        return 0;
> +    }
> +
> +    /* Set picture related parameters */
> +    h264->pic_width = width;
> +    h264->pic_height = height;
> +    h264->pic_depth = depth;
> +    h264->pic_pitch = pitch;
> +
> +    /* Create context */
> +    va_status = vaCreateContext(h264->va_dpy, h264->config, width, height,
> +                                VA_PROGRESSIVE, 0, 0, &h264->context);
> +    CHECK_VASTATUS_RET(va_status, "vaCreateContext", -1);
> +
> +    /* Sequence parameters */
> +    VAEncSequenceParameterBufferH264 seq_h264 = {0};
> +    seq_h264.level_idc = 30;
> +    seq_h264.picture_width_in_mbs = (width + 15) / 16;
> +    seq_h264.picture_height_in_mbs = (height + 15) / 16;
> +    seq_h264.bits_per_second = 384 * 1000;
> +    seq_h264.initial_qp = 26;
> +    seq_h264.min_qp = 3;
> +
> +    va_status = vaCreateBuffer(h264->va_dpy, h264->context,
> +                               VAEncSequenceParameterBufferType,
> +                               sizeof(seq_h264), 1, &seq_h264,
> +                               &h264->seq_param);
> +    CHECK_VASTATUS_RET(va_status, "vaCreateBuffer(seq_param)", -1);
> +
> +    /* Surfaces */
> +    va_status = vaCreateSurfaces(h264->va_dpy, width, height,
> +                                 VA_RT_FORMAT_YUV420, SID_NUMBER,
> +                                 &h264->surface_ids[0]);
> +    CHECK_VASTATUS_RET(va_status, "vaCreateSurfaces", -1);
> +    VNC_DEBUG("%s: Created surfaces:", __func__);
> +    for (i = 0; i < SID_NUMBER; ++i) {
> +        VNC_DEBUG(" [#%d -> 0x%x]", i, h264->surface_ids[i]);
> +    }
> +    VNC_DEBUG("\n");
> +
> +    /* Coded buffer */
> +    coded_buf_size = h264_get_coded_buf_size(h264);
> +    va_status = vaCreateBuffer(h264->va_dpy, h264->context,
> +                               VAEncCodedBufferType, coded_buf_size, 1, NULL,
> +                               &h264->coded_buf);
> +    CHECK_VASTATUS_RET(va_status, "vaCreateBuffer(coded_buf)", -1);
> +    VNC_DEBUG("%s: created coded_buf: id=0x%x, size=%d\n", __func__,
> +              h264->coded_buf, coded_buf_size);
> +
> +    return 0;
> +}
> +
> +static void h264_release_context(VncDisplayH264 *h264)
> +{
> +    VAStatus va_status;
> +    VASurfaceStatus surface_status;
> +    int i;
> +
> +    if (h264->context == VA_INVALID_ID) {
> +        return;
> +    }
> +
> +    /* Sync all surfaces */
> +    VNC_DEBUG("%s: Syncing surfaces: ", __func__);
> +    for (i = 0; i < SID_NUMBER; ++i) {
> +        if (h264->surface_ids[i] != VA_INVALID_ID) {
> +            VNC_DEBUG("[#%d", i);
> +
> +            va_status = vaSyncSurface(h264->va_dpy, h264->surface_ids[i]);
> +            CHECK_VASTATUS(va_status, "vaSyncSurface");
> +
> +            surface_status = (VASurfaceStatus)0;
> +            va_status = vaQuerySurfaceStatus(h264->va_dpy, h264->surface_ids[i],
> +                                             &surface_status);
> +            CHECK_VASTATUS(va_status, "vaQuerySurfaceStatus");
> +
> +            VNC_DEBUG(" -> 0x%x] ", surface_status);
> +        }
> +    }
> +    VNC_DEBUG("done!\n");
> +
> +    /* Release parameter buffers */
> +    VNC_DEBUG("%s: seq_param = 0x%x\n", __func__, h264->seq_param);
> +    if (h264->seq_param != VA_INVALID_ID) {
> +        va_status = vaDestroyBuffer(h264->va_dpy, h264->seq_param);
> +        CHECK_VASTATUS(va_status, "vaDestroyBuffer(seq_param)");
> +        h264->seq_param = VA_INVALID_ID;
> +    }
> +
> +    VNC_DEBUG("%s: pic_param = 0x%x\n", __func__, h264->pic_param);
> +    if (h264->pic_param != VA_INVALID_ID) {
> +        va_status = vaDestroyBuffer(h264->va_dpy, h264->pic_param);
> +        CHECK_VASTATUS(va_status, "vaDestroyBuffer(pic_param)");
> +        h264->pic_param = VA_INVALID_ID;
> +    }
> +
> +    VNC_DEBUG("%s: slice_param = 0x%x\n", __func__, h264->slice_param);
> +    if (h264->slice_param != VA_INVALID_ID) {
> +        va_status = vaDestroyBuffer(h264->va_dpy, h264->slice_param);
> +        CHECK_VASTATUS(va_status, "vaDestroyBuffer(slice_param)");
> +        h264->slice_param = VA_INVALID_ID;
> +    }
> +
> +    VNC_DEBUG("%s: coded_buf = 0x%x\n", __func__, h264->coded_buf);
> +    if (h264->coded_buf != VA_INVALID_ID) {
> +        va_status = vaDestroyBuffer(h264->va_dpy, h264->coded_buf);
> +        CHECK_VASTATUS(va_status, "vaDestroyBuffer(coded_buf)");
> +        h264->coded_buf = VA_INVALID_ID;
> +    }
> +
> +    /* Release surfaces */
> +    if (h264->surface_ids[0] != VA_INVALID_ID) {
> +        va_status = vaDestroySurfaces(h264->va_dpy, &h264->surface_ids[0],
> +                                      SID_NUMBER);
> +        CHECK_VASTATUS(va_status, "vaDestroySurfaces");
> +    }
> +    for (i = 0; i < SID_NUMBER; ++i) {
> +        h264->surface_ids[i] = VA_INVALID_ID;
> +    }
> +
> +    /* Release context */
> +    va_status = vaDestroyContext(h264->va_dpy, h264->context);
> +    CHECK_VASTATUS(va_status, "vaDestroyContext");
> +    h264->context = VA_INVALID_ID;
> +}
> +
> +
> +static void h264_encode_frame(VncDisplayH264 *h264,
> +                              const unsigned char *const in_buf, const int last,
> +                              const bool is_intra, int *out_length,
> +                              unsigned char **out_buf)
> +{
> +    VAStatus va_status;
> +    VASurfaceStatus surface_status;
> +    VACodedBufferSegment *coded_buf_segment = NULL;
> +    int coded_buf_size;
> +
> +    va_status = vaBeginPicture(h264->va_dpy, h264->context,
> +                               h264->surface_ids[SID_INPUT_PICTURE]);
> +    CHECK_VASTATUS(va_status, "vaBeginPicture");
> +
> +    /* Set parameters and put picture on the input picture surface */
> +    h264_prepare_input(h264, in_buf, is_intra, last);
> +
> +    /* Sync input surface */
> +    va_status = vaEndPicture(h264->va_dpy, h264->context);
> +    CHECK_VASTATUS(va_status, "vaEndPicture");
> +
> +    va_status = vaSyncSurface(h264->va_dpy,
> +                              h264->surface_ids[SID_INPUT_PICTURE]);
> +    CHECK_VASTATUS(va_status, "vaSyncSurface");
> +
> +    surface_status = (VASurfaceStatus)0;
> +    va_status = vaQuerySurfaceStatus(h264->va_dpy,
> +                                     h264->surface_ids[SID_INPUT_PICTURE],
> +                                     &surface_status);
> +    CHECK_VASTATUS(va_status, "vaQuerySurfaceStatus");
> +
> +    /* Get H.264 encoded data */
> +    if (h264->coded_buf != VA_INVALID_ID) {
> +        va_status = vaUnmapBuffer(h264->va_dpy, h264->coded_buf);
> +        CHECK_VASTATUS(va_status, "vaUnmapBuffer(coded_buf)");
> +    }
> +
> +    va_status = vaMapBuffer(h264->va_dpy, h264->coded_buf,
> +                            (void **)(&coded_buf_segment));
> +    CHECK_VASTATUS(va_status, "vaMapBuffer");
> +
> +    *out_buf = (unsigned char *)coded_buf_segment->buf;
> +
> +    if (coded_buf_segment->next) {
> +        VNC_DEBUG("%s: Unsupported: multiple segments detected. "
> +                  "Skipping next segments\n", __func__);
> +    }
> +
> +    coded_buf_size = h264_get_coded_buf_size(h264);
> +
> +    *out_length = get_buffer_length(*out_buf, coded_buf_size);
> +}
> +
> +
> +static void h264_prepare_input(VncDisplayH264 *h264,
> +                               const unsigned char *const in_buf,
> +                               const bool is_intra, const bool is_last)
> +{
> +    VAStatus va_status;
> +    VAEncPictureParameterBufferH264 pic_h264;
> +    VAEncSliceParameterBuffer slice_h264;
> +    VACodedBufferSegment *coded_buf_segment = NULL;
> +    VABufferID tempID;
> +
> +    VNC_DEBUG("%s: in_buf=%p; is_intra=%d; is_last=%d\n", __func__, in_buf,
> +              is_intra, is_last);
> +
> +    /* Set sequence parameters */
> +    va_status = vaRenderPicture(h264->va_dpy, h264->context,
> +                                &h264->seq_param, 1);
> +    CHECK_VASTATUS(va_status, "vaRenderPicture(seq_param)");
> +
> +    /* Set picture parameters */
> +    pic_h264.reference_picture = h264->surface_ids[SID_REFERENCE_PICTURE];
> +    pic_h264.reconstructed_picture = h264->surface_ids[SID_RECON_PICTURE];
> +    pic_h264.coded_buf = h264->coded_buf;
> +    pic_h264.picture_width = h264->pic_width;
> +    pic_h264.picture_height = h264->pic_height;
> +    pic_h264.last_picture = is_last;
> +
> +    if (h264->pic_param != VA_INVALID_ID) {
> +        va_status = vaDestroyBuffer(h264->va_dpy, h264->pic_param);
> +        CHECK_VASTATUS(va_status, "vaDestroyBuffer(pic_parameter)");
> +    }
> +
> +    va_status = vaCreateBuffer(h264->va_dpy, h264->context,
> +                               VAEncPictureParameterBufferType,
> +                               sizeof(pic_h264), 1, &pic_h264,
> +                               &h264->pic_param);
> +    CHECK_VASTATUS(va_status, "vaCreateBuffer(pic_parameter)");
> +
> +    va_status = vaRenderPicture(h264->va_dpy, h264->context,
> +                                &h264->pic_param, 1);
> +    CHECK_VASTATUS(va_status, "vaRenderPicture(pic_parameter)");
> +
> +    /* Allocate and zero out coded buffer segment.
> +     * See also get_buffer_length(). */
> +    va_status = vaMapBuffer(h264->va_dpy, h264->coded_buf,
> +                            (void **)(&coded_buf_segment));
> +    CHECK_VASTATUS(va_status, "vaMapBuffer(coded_buf_segment)");
> +
> +    /* FIXME: Is this (still) needed? */
> +    memset((unsigned char *)coded_buf_segment->buf, 0, coded_buf_segment->size);
> +
> +    va_status = vaUnmapBuffer(h264->va_dpy, h264->coded_buf);
> +    CHECK_VASTATUS(va_status, "vaUnmapBuffer(coded_buf)");
> +
> +    /* Set slice parameters */
> +    slice_h264.start_row_number = 0;
> +    slice_h264.slice_height = h264->pic_height / 16;    /* Measured in MB */
> +    slice_h264.slice_flags.bits.is_intra = is_intra;
> +    slice_h264.slice_flags.bits.disable_deblocking_filter_idc = 0;
> +
> +    if (h264->slice_param != VA_INVALID_ID) {
> +        va_status = vaDestroyBuffer(h264->va_dpy, h264->slice_param);
> +        CHECK_VASTATUS(va_status, "vaDestroyBuffer(slice_parameter)");
> +    }
> +
> +    va_status = vaCreateBuffer(h264->va_dpy, h264->context,
> +                               VAEncSliceParameterBufferType,
> +                               sizeof(slice_h264), 1, &slice_h264,
> +                               &h264->slice_param);
> +    CHECK_VASTATUS(va_status, "vaCreateBuffer(slice_parameter)");
> +
> +    va_status = vaRenderPicture(h264->va_dpy, h264->context,
> +                                &h264->slice_param, 1);
> +    CHECK_VASTATUS(va_status, "vaRenderPicture(slice_parameter)");
> +
> +    /* Copy RGBA input buffer to VA surface */
> +    rgba_to_surface(h264, in_buf, h264->surface_ids[SID_INPUT_PICTURE]);
> +
> +    /* Prepare for next picture */
> +    tempID = h264->surface_ids[SID_RECON_PICTURE];
> +    h264->surface_ids[SID_RECON_PICTURE] =
> +                                      h264->surface_ids[SID_REFERENCE_PICTURE];
> +    h264->surface_ids[SID_REFERENCE_PICTURE] = tempID;
> +}
> +
> +static void rgba_to_surface(const VncDisplayH264 *h264,
> +                            const unsigned char *const inbuf,
> +                            VASurfaceID surface_id)
> +{
> +    VAStatus va_status;
> +    VAImage image;
> +    void *pbuffer = NULL;
> +    const unsigned char *psrc = inbuf;
> +    const unsigned char *s;
> +    unsigned char *pdst = NULL;
> +    unsigned char *dst_y, *dst_uv;
> +    unsigned char *dst_uv_line = NULL;
> +    int i, j;
> +
> +    VNC_DEBUG("%s: inbuf=%p; width=%d; height=%d; frame=%ld\n", __func__,
> +              inbuf, h264->pic_width, h264->pic_height, h264->frame_count);
> +
> +    va_status = vaDeriveImage(h264->va_dpy, surface_id, &image);
> +    CHECK_VASTATUS(va_status, "vaDeriveImage");
> +
> +    va_status = vaMapBuffer(h264->va_dpy, image.buf, &pbuffer);
> +    CHECK_VASTATUS(va_status, "vaMapBuffer(image.buf)");
> +
> +    pdst = (unsigned char *)pbuffer;
> +    dst_uv_line = pdst + image.offsets[1];
> +
> +    /* RGBA => NV12 */
> +    for (i = 0; i < h264->pic_height; ++i) {
> +        dst_y = (pdst + image.offsets[0]) + i*image.pitches[0];
> +        dst_uv = dst_uv_line;
> +        s = psrc;
> +        for (j = 0; j < h264->pic_width; j++) {
> +            *dst_y++ = ((66*s[0] + 129*s[1] + 25*s[2] + 128) >> 8) + 16;

Please add spaces around operators, scripts/checkpatch.pl may find
problems like this. The magic constants should be replaced by enums or
#defines.

> +            /* NV12 means subsampling by 2 in x and y */
> +            if ((0 == i%2) && (0 == j%2)) {

This order is not common in QEMU, please reverse.

> +                *dst_uv++ = ((112*s[0] - 94*s[1] - 18*s[2] + 128) >> 8) + 128;
> +                *dst_uv++ = ((-38*s[0] - 74*s[1] + 112*s[2] + 128) >> 8) + 128;
> +            }
> +            s += h264->pic_depth;
> +        }
> +        psrc += h264->pic_pitch;
> +        if (0 == i%2) {
> +            dst_uv_line += image.pitches[1];
> +        }
> +    }
> +
> +    va_status = vaUnmapBuffer(h264->va_dpy, image.buf);
> +    CHECK_VASTATUS(va_status, "vaUnmapBuffer(image.buf)");
> +
> +    va_status = vaDestroyImage(h264->va_dpy, image.image_id);
> +    CHECK_VASTATUS(va_status, "vaDestroyImage");
> +}
> +
> +static int get_buffer_length(const unsigned char * const buffer,
> +                             int buffer_length)
> +{
> +    int i;
> +    for (i = buffer_length - 1; i >= 0; i--) {
> +        if (buffer[i]) {
> +            break;
> +        }
> +    }
> +
> +    return i + 1;
> +}
> diff --git a/ui/vnc-enc-h264.h b/ui/vnc-enc-h264.h
> new file mode 100644
> index 0000000..0047d9b
> --- /dev/null
> +++ b/ui/vnc-enc-h264.h
> @@ -0,0 +1,74 @@
> +/*
> + * QEMU VNC display driver: VAAPI based H.264 encoding
> + *
> + * Copyright (c) 2012 Intel Corporation.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the
> + * "Software"), to deal in the Software without restriction, including
> + * without limitation the rights to use, copy, modify, merge, publish,
> + * distribute, sub license, and/or sell copies of the Software, and to
> + * permit persons to whom the Software is furnished to do so, subject to
> + * the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the
> + * next paragraph) shall be included in all copies or substantial portions
> + * of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
> + * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
> + * IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
> + * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
> + * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
> + * USE OR OTHER DEALINGS IN THE SOFTWARE.
> + */
> +
> +#ifndef __QEMU_VNC_ENC_H264_H__
> +#define __QEMU_VNC_ENC_H264_H__
> +
> +#include <X11/Xlib.h>
> +#include <va/va_x11.h>
> +
> +#define SID_NUMBER                              3
> +#define SID_INPUT_PICTURE                       0
> +#define SID_REFERENCE_PICTURE                   1
> +#define SID_RECON_PICTURE                       2
> +
> +typedef struct VncH264Frame {
> +    unsigned char *data;
> +    int length;
> +    int is_intra;

The name suggests that the variable should be bool.

> +    unsigned long count;
> +    int width;
> +    int height;
> +} VncH264Frame;
> +
> +typedef struct VncDisplayH264 {
> +    /* Encoding context related */
> +    Display *x11_dpy;
> +    VADisplay va_dpy;
> +    VAConfigID config;
> +    VAContextID context;
> +    VASurfaceID surface_ids[SID_NUMBER];
> +    VABufferID seq_param;       /* Sequence parameters */
> +    VABufferID pic_param;       /* Picture parameters */
> +    VABufferID slice_param;     /* Slice parameters */
> +    VABufferID coded_buf;       /* Output buffer, encoded data */
> +
> +    /* Picture related */
> +    int pic_width;
> +    int pic_height;
> +    int pic_depth;
> +    int pic_pitch;
> +
> +    /* Data for last encoded frame, e.g. to use it for multiple clients */
> +    VncH264Frame cur_frame;
> +
> +    /* Misc */
> +    bool initialized;
> +    unsigned long frame_count;
> +    bool force_intra;
> +} VncDisplayH264;
> +
> +#endif /* __QEMU_VNC_ENC_H264_H__ */
> diff --git a/ui/vnc-jobs.c b/ui/vnc-jobs.c
> index 0bfc0c5..ad02e37 100644
> --- a/ui/vnc-jobs.c
> +++ b/ui/vnc-jobs.c
> @@ -194,6 +194,9 @@ static void vnc_async_encoding_start(VncState *orig, VncState *local)
>      local->hextile = orig->hextile;
>      local->zrle = orig->zrle;
>      local->output =  queue->buffer;
> +#ifdef CONFIG_VNC_LIBVA
> +    local->h264_last_sent_frame = orig->h264_last_sent_frame;
> +#endif
>      local->csock = -1; /* Don't do any network work on this thread */
>
>      buffer_reset(&local->output);
> @@ -206,6 +209,9 @@ static void vnc_async_encoding_end(VncState *orig, VncState *local)
>      orig->hextile = local->hextile;
>      orig->zrle = local->zrle;
>      orig->lossy_rect = local->lossy_rect;
> +#ifdef CONFIG_VNC_LIBVA
> +    orig->h264_last_sent_frame = local->h264_last_sent_frame;
> +#endif
>
>      queue->buffer = local->output;
>  }
> diff --git a/ui/vnc.c b/ui/vnc.c
> index ff4e2ae..d977563 100644
> --- a/ui/vnc.c
> +++ b/ui/vnc.c
> @@ -713,6 +713,11 @@ int vnc_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
>          case VNC_ENCODING_ZYWRLE:
>              n = vnc_zywrle_send_framebuffer_update(vs, x, y, w, h);
>              break;
> +#ifdef CONFIG_VNC_LIBVA
> +        case VNC_ENCODING_VA_H264:
> +            n = vnc_h264_send_framebuffer_update(vs, x, y, w, h);
> +            break;
> +#endif
>          default:
>              vnc_framebuffer_update(vs, x, y, w, h, VNC_ENCODING_RAW);
>              n = vnc_raw_send_framebuffer_update(vs, x, y, w, h);
> @@ -1864,6 +1869,16 @@ static void set_encodings(VncState *vs, int32_t *encodings, size_t n_encodings)
>              vs->features |= VNC_FEATURE_ZYWRLE_MASK;
>              vs->vnc_encoding = enc;
>              break;
> +#ifdef CONFIG_VNC_LIBVA
> +        case VNC_ENCODING_VA_H264:
> +            if (vnc_h264_encoder_init(&vs->vd->h264) == 0) {
> +                vs->features |= VNC_FEATURE_VA_H264_MASK;
> +                vs->vnc_encoding = enc;
> +            } else {
> +                fprintf(stderr, "Failed to initialize VA H264 encoder\n");
> +            }
> +            break;
> +#endif
>          case VNC_ENCODING_DESKTOPRESIZE:
>              vs->features |= VNC_FEATURE_RESIZE_MASK;
>              break;
> @@ -2659,6 +2674,12 @@ static void vnc_refresh(void *opaque)
>      has_dirty = vnc_refresh_server_surface(vd);
>      vnc_unlock_display(vd);
>
> +#ifdef CONFIG_VNC_LIBVA
> +    if (has_dirty) {
> +        ++vd->h264.frame_count;
> +    }
> +#endif
> +
>      QTAILQ_FOREACH_SAFE(vs, &vd->clients, next, vn) {
>          rects += vnc_update_client(vs, has_dirty);
>          /* vs might be free()ed here */
> @@ -2770,6 +2791,11 @@ void vnc_init_state(VncState *vs)
>
>      QTAILQ_INSERT_HEAD(&vd->clients, vs, next);
>
> +#ifdef CONFIG_VNC_LIBVA
> +    vs->h264_last_sent_frame = vd->h264.frame_count - 1;
> +    vd->h264.force_intra = true;
> +#endif
> +
>      vga_hw_update();
>
>      vnc_write(vs, "RFB 003.008\n", 12);
> diff --git a/ui/vnc.h b/ui/vnc.h
> index 45d7686..89c5027 100644
> --- a/ui/vnc.h
> +++ b/ui/vnc.h
> @@ -102,6 +102,9 @@ typedef struct VncDisplay VncDisplay;
>  #ifdef CONFIG_VNC_WS
>  #include "vnc-ws.h"
>  #endif
> +#ifdef CONFIG_VNC_LIBVA
> +#include "vnc-enc-h264.h"
> +#endif
>
>  struct VncRectStat
>  {
> @@ -175,6 +178,9 @@ struct VncDisplay
>  #ifdef CONFIG_VNC_SASL
>      VncDisplaySASL sasl;
>  #endif
> +#ifdef CONFIG_VNC_LIBVA
> +    VncDisplayH264 h264;
> +#endif
>  };
>
>  typedef struct VncTight {
> @@ -320,6 +326,10 @@ struct VncState
>      VncZrle zrle;
>      VncZywrle zywrle;
>
> +#ifdef CONFIG_VNC_LIBVA
> +    unsigned long h264_last_sent_frame;
> +#endif
> +
>      Notifier mouse_mode_notifier;
>
>      QTAILQ_ENTRY(VncState) next;
> @@ -375,6 +385,7 @@ enum {
>  #define VNC_ENCODING_TRLE                 0x0000000f
>  #define VNC_ENCODING_ZRLE                 0x00000010
>  #define VNC_ENCODING_ZYWRLE               0x00000011
> +#define VNC_ENCODING_VA_H264              0x48323634
>  #define VNC_ENCODING_COMPRESSLEVEL0       0xFFFFFF00 /* -256 */
>  #define VNC_ENCODING_QUALITYLEVEL0        0xFFFFFFE0 /* -32  */
>  #define VNC_ENCODING_XCURSOR              0xFFFFFF10 /* -240 */
> @@ -424,6 +435,7 @@ enum {
>  #define VNC_FEATURE_TIGHT_PNG                8
>  #define VNC_FEATURE_ZRLE                     9
>  #define VNC_FEATURE_ZYWRLE                  10
> +#define VNC_FEATURE_VA_H264                 11
>
>  #define VNC_FEATURE_RESIZE_MASK              (1 << VNC_FEATURE_RESIZE)
>  #define VNC_FEATURE_HEXTILE_MASK             (1 << VNC_FEATURE_HEXTILE)
> @@ -436,6 +448,7 @@ enum {
>  #define VNC_FEATURE_TIGHT_PNG_MASK           (1 << VNC_FEATURE_TIGHT_PNG)
>  #define VNC_FEATURE_ZRLE_MASK                (1 << VNC_FEATURE_ZRLE)
>  #define VNC_FEATURE_ZYWRLE_MASK              (1 << VNC_FEATURE_ZYWRLE)
> +#define VNC_FEATURE_VA_H264_MASK             (1 << VNC_FEATURE_VA_H264)
>
>
>  /* Client -> Server message IDs */
> @@ -581,4 +594,9 @@ int vnc_zrle_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
>  int vnc_zywrle_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
>  void vnc_zrle_clear(VncState *vs);
>
> +#ifdef CONFIG_VNC_LIBVA
> +int vnc_h264_encoder_init(VncDisplayH264 *h264);
> +int vnc_h264_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
> +#endif
> +
>  #endif /* __QEMU_VNC_H */
> --
> 1.7.9.5
>
> Intel Corporation NV/SA
> Kings Square, Veldkant 31
> 2550 Kontich
> RPM (Bruxelles) 0415.497.718.
> Citibank, Brussels, account 570/1031255/09
>
> This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
>
>
Daniel P. Berrange - Feb. 21, 2013, 3:22 p.m.
On Wed, Feb 13, 2013 at 11:45:04AM +0100, David Verbeiren wrote:
> This patch implements H.264 encoding of the VNC framebuffer updates
> using hardware acceleration through the VA API.
> 
> This is experimental support to let the community explore the possibilities
> offered by the potential bandwidth and latency reductions that H.264
> encoding allows. This may be particularly useful for use cases such as
> online gaming, hosted desktops, hosted set top boxes...
> This patch provides the VNC server side support. Corresponding VNC
> client side support is required. To this end, we are also contributing
> patches to the gtk-vnc and libvncserver/libvncclient projects which can be
> used to test this experimental feature.

I just mentioned on the gtk-vnc list, that I'd like to see the
wire protocol for this encoding format documented, so we can
review whether the proposed implementations are actually
correctly written.

Daniel.
Anthony Liguori - Feb. 21, 2013, 4:18 p.m.
"Daniel P. Berrange" <berrange@redhat.com> writes:

> On Wed, Feb 13, 2013 at 11:45:04AM +0100, David Verbeiren wrote:
>> This patch implements H.264 encoding of the VNC framebuffer updates
>> using hardware acceleration through the VA API.
>> 
>> This is experimental support to let the community explore the possibilities
>> offered by the potential bandwidth and latency reductions that H.264
>> encoding allows. This may be particularly useful for use cases such as
>> online gaming, hosted desktops, hosted set top boxes...
>> This patch provides the VNC server side support. Corresponding VNC
>> client side support is required. To this end, we are also contributing
>> patches to the gtk-vnc and libvncserver/libvncclient projects which can be
>> used to test this experimental feature.
>
> I just mentioned on the gtk-vnc list, that I'd like to see the
> wire protocol for this encoding format documented, so we can
> review whether the proposed implementations are actually
> correctly written.

Ack.

Regards,

Antony Liguori

>
> Daniel.
> -- 
> |: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org              -o-             http://virt-manager.org :|
> |: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
> |: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
David Verbeiren - Feb. 22, 2013, 2:31 p.m.
On Wed, Feb 13, 2013 at 22:16, Blue Swirl <blauwirbel@gmail.com> wrote:
>> +    /* RGBA => NV12 */

>> +    for (i = 0; i < h264->pic_height; ++i) {

>> +        dst_y = (pdst + image.offsets[0]) + i*image.pitches[0];

>> +        dst_uv = dst_uv_line;

>> +        s = psrc;

>> +        for (j = 0; j < h264->pic_width; j++) {

>> +            *dst_y++ = ((66*s[0] + 129*s[1] + 25*s[2] + 128) >> 8) + 16;

>

> Please add spaces around operators, scripts/checkpatch.pl may find

> problems like this. The magic constants should be replaced by enums or

> #defines.

>

>> +            /* NV12 means subsampling by 2 in x and y */

>> +            if ((0 == i%2) && (0 == j%2)) {

>> +                *dst_uv++ = ((112*s[0] - 94*s[1] - 18*s[2] + 128) >> 8) + 128;

>> +                *dst_uv++ = ((-38*s[0] - 74*s[1] + 112*s[2] + 128) >> 8) + 128;


Instead of creating 9 artificial names for the factors, I'd rather create the
following 3 macros:
	#define RGB_TO_Y(r, g, b) ((( 66 * r + 129 * g +  25 * b + 128) >> 8) + 16)
	#define RGB_TO_U(r, g, b) (((112 * r -  94 * g -  18 * b + 128) >> 8) + 128)
	#define RGB_TO_V(r, g, b) (((-38 * r -  74 * g + 112 * b + 128) >> 8) + 128)

I think that will make the code more readable and will make it obvious
what the magic numbers are for. Would that be ok?

I'll follow your recommendation for the other items you brought up.

-David
Intel Corporation NV/SA
Kings Square, Veldkant 31
2550 Kontich
RPM (Bruxelles) 0415.497.718. 
Citibank, Brussels, account 570/1031255/09

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Blue Swirl - Feb. 23, 2013, 4:16 p.m.
On Fri, Feb 22, 2013 at 2:31 PM, Verbeiren, David
<david.verbeiren@intel.com> wrote:
> On Wed, Feb 13, 2013 at 22:16, Blue Swirl <blauwirbel@gmail.com> wrote:
>>> +    /* RGBA => NV12 */
>>> +    for (i = 0; i < h264->pic_height; ++i) {
>>> +        dst_y = (pdst + image.offsets[0]) + i*image.pitches[0];
>>> +        dst_uv = dst_uv_line;
>>> +        s = psrc;
>>> +        for (j = 0; j < h264->pic_width; j++) {
>>> +            *dst_y++ = ((66*s[0] + 129*s[1] + 25*s[2] + 128) >> 8) + 16;
>>
>> Please add spaces around operators, scripts/checkpatch.pl may find
>> problems like this. The magic constants should be replaced by enums or
>> #defines.
>>
>>> +            /* NV12 means subsampling by 2 in x and y */
>>> +            if ((0 == i%2) && (0 == j%2)) {
>>> +                *dst_uv++ = ((112*s[0] - 94*s[1] - 18*s[2] + 128) >> 8) + 128;
>>> +                *dst_uv++ = ((-38*s[0] - 74*s[1] + 112*s[2] + 128) >> 8) + 128;
>
> Instead of creating 9 artificial names for the factors, I'd rather create the
> following 3 macros:
>         #define RGB_TO_Y(r, g, b) ((( 66 * r + 129 * g +  25 * b + 128) >> 8) + 16)
>         #define RGB_TO_U(r, g, b) (((112 * r -  94 * g -  18 * b + 128) >> 8) + 128)
>         #define RGB_TO_V(r, g, b) (((-38 * r -  74 * g + 112 * b + 128) >> 8) + 128)
>
> I think that will make the code more readable and will make it obvious
> what the magic numbers are for. Would that be ok?

That's even better. Inline functions are preferred to macros though.

>
> I'll follow your recommendation for the other items you brought up.
>
> -David
> Intel Corporation NV/SA
> Kings Square, Veldkant 31
> 2550 Kontich
> RPM (Bruxelles) 0415.497.718.
> Citibank, Brussels, account 570/1031255/09
>
> This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

Patch

diff --git a/configure b/configure
index 8789324..d742a6c 100755
--- a/configure
+++ b/configure
@@ -213,6 +213,7 @@  pie=""
 zero_malloc=""
 trace_backend="nop"
 trace_file="trace"
+libva=""
 spice=""
 rbd=""
 smartcard_nss=""
@@ -771,6 +772,10 @@  for opt do
   ;;
   --enable-spice) spice="yes"
   ;;
+  --disable-libva) libva="no"
+  ;;
+  --enable-libva) libva="yes"
+  ;;
   --disable-libiscsi) libiscsi="no"
   ;;
   --enable-libiscsi) libiscsi="yes"
@@ -1129,6 +1134,8 @@  echo "  --with-trace-file=NAME   Full PATH,NAME of file to store traces"
 echo "                           Default:trace-<pid>"
 echo "  --disable-spice          disable spice"
 echo "  --enable-spice           enable spice"
+echo "  --disable-libva          disable H.264 encoding with libva"
+echo "  --enable-libva           enable H.264 encoding with libva"
 echo "  --enable-rbd             enable building the rados block device (rbd)"
 echo "  --disable-libiscsi       disable iscsi support"
 echo "  --enable-libiscsi        enable iscsi support"
@@ -2842,6 +2849,33 @@  EOF
   fi
 fi
 
+##########################################
+# libva probe
+if test "$libva" != "no" ; then
+  cat > $TMPC << EOF
+#include <X11/Xlib.h>
+#include <va/va.h>
+#include <va/va_x11.h>
+int main(void) { int major_ver, minor_ver;
+ Display *win_display = (Display *)XOpenDisplay(":0.0");
+ VADisplay va_dpy = vaGetDisplay(win_display);
+ vaInitialize(va_dpy, &major_ver, &minor_ver); return 0;
+}
+EOF
+libva_cflags="$($pkg_config --cflags libva 2>/dev/null) $($pkg_config --cflags libva-x11 2>/dev/null) $($pkg_config --cflags x11 2>/dev/null)"
+libva_libs="$($pkg_config --libs libva 2>/dev/null) $($pkg_config --libs libva-x11 2>/dev/null) $($pkg_config --libs x11 2>/dev/null)"
+  if $pkg_config --atleast-version=0.26 libva >/dev/null 2>&1 && \
+     compile_prog "$libva_cflags" "$libva_libs" ; then
+    libva="yes"
+    libs_softmmu="$libs_softmmu $libva_libs"
+  else
+    if test "$libva" = "yes" ; then
+      feature_not_found "libva"
+    fi
+    libva="no"
+  fi
+fi
+
 # check for libcacard for smartcard support
 smartcard_cflags=""
 # TODO - what's the minimal nss version we support?
@@ -3336,6 +3370,7 @@  echo "xfsctl support    $xfs"
 echo "nss used          $smartcard_nss"
 echo "usb net redir     $usb_redir"
 echo "OpenGL support    $opengl"
+echo "libva support     $libva"
 echo "libiscsi support  $libiscsi"
 echo "build guest agent $guest_agent"
 echo "seccomp support   $seccomp"
@@ -3626,6 +3661,10 @@  if test "$spice" = "yes" ; then
   echo "CONFIG_SPICE=y" >> $config_host_mak
 fi
 
+if test "$libva" = "yes" ; then
+  echo "CONFIG_VNC_LIBVA=y" >> $config_host_mak
+fi
+
 if test "$smartcard_nss" = "yes" ; then
   echo "CONFIG_SMARTCARD_NSS=y" >> $config_host_mak
   echo "libcacard_libs=$libcacard_libs" >> $config_host_mak
diff --git a/ui/Makefile.objs b/ui/Makefile.objs
index d9db073..6b1bd1a 100644
--- a/ui/Makefile.objs
+++ b/ui/Makefile.objs
@@ -5,6 +5,7 @@  vnc-obj-y += vnc-enc-zrle.o
 vnc-obj-$(CONFIG_VNC_TLS) += vnc-tls.o vnc-auth-vencrypt.o
 vnc-obj-$(CONFIG_VNC_SASL) += vnc-auth-sasl.o
 vnc-obj-$(CONFIG_VNC_WS) += vnc-ws.o
+vnc-obj-$(CONFIG_VNC_LIBVA) += vnc-enc-h264.o
 vnc-obj-y += vnc-jobs.o
 
 common-obj-y += keymaps.o console.o cursor.o input.o qemu-pixman.o
diff --git a/ui/vnc-enc-h264.c b/ui/vnc-enc-h264.c
new file mode 100644
index 0000000..da9bcf0
--- /dev/null
+++ b/ui/vnc-enc-h264.c
@@ -0,0 +1,616 @@ 
+/*
+ * QEMU VNC display driver: VAAPI based H.264 encoding
+ *
+ * Copyright (c) 2012 Intel Corporation.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sub license, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the
+ * next paragraph) shall be included in all copies or substantial portions
+ * of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
+ * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+ * IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
+ * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+ * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
+ * USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+#include "vnc.h"
+
+#define CHECK_VASTATUS(va_status, func) do {                                   \
+    if (va_status != VA_STATUS_SUCCESS) {                                      \
+        fprintf(stderr, "%s:%s:%d failed (0x%x),exit\n", __func__, func,       \
+                __LINE__, va_status);                                          \
+        exit(1);                                                               \
+    } else {                                                                   \
+        /* fprintf(stderr, "%s:%s:%d success\n", __func__, func, __LINE__); */ \
+    }                                                                          \
+} while (0);
+
+#define CHECK_VASTATUS_RET(va_status, func, retval) do {                       \
+    if (va_status != VA_STATUS_SUCCESS) {                                      \
+        fprintf(stderr, "%s:%s:%d failed (0x%x),exit\n", __func__, func,       \
+                __LINE__, va_status);                                          \
+        return retval;                                                         \
+    } else {                                                                   \
+        /* fprintf(stderr, "%s:%s:%d success\n", __func__, func, __LINE__); */ \
+    }                                                                          \
+} while (0);
+
+
+/* Forward declarations */
+static int h264_encoder_reinit(VncDisplayH264 *h264, int w, int h, int pitch,
+                               int depth);
+static void h264_encoder_close(VncDisplayH264 *h264);
+
+static int h264_create_context(VncDisplayH264 *h264, int width, int height,
+                               int pitch, int depth);
+static void h264_release_context(VncDisplayH264 *h264);
+
+static void h264_prepare_input(VncDisplayH264 *h264,
+                               const unsigned char *const in_buf,
+                               const bool is_intra, const bool is_last);
+static int h264_get_coded_buf_size(VncDisplayH264 *h264);
+
+static void h264_encode_frame(VncDisplayH264 *h264,
+                              const unsigned char *const in_buf, const int last,
+                              const bool is_intra, int *out_length,
+                              unsigned char **out_buf);
+static void rgba_to_surface(const VncDisplayH264 *h264,
+                            const unsigned char *const inbuf,
+                            VASurfaceID surface_id);
+static int get_buffer_length(const unsigned char * const buffer,
+                             int buffer_length);
+
+
+int vnc_h264_encoder_init(VncDisplayH264 *h264)
+{
+    VAStatus va_status;
+    int va_major_ver, va_minor_ver;
+    VAEntrypoint entrypoints[5];
+    int num_entrypoints, entrypoint;
+    VAConfigAttrib attrib[2];
+    int i;
+
+    if (h264->initialized) {
+        return 0; /* Already initialized */
+    }
+
+    /* Initialize current frame data */
+    h264->cur_frame.data = NULL;
+    h264->cur_frame.length = -1;
+    h264->cur_frame.is_intra = -1;
+    h264->cur_frame.count = -1;
+    h264->cur_frame.width = -1;
+    h264->cur_frame.height = -1;
+
+    /* Open local X11 display and VA display */
+    h264->x11_dpy = XOpenDisplay(":0.0");
+    if (!h264->x11_dpy) {
+        fprintf(stderr, "Could not open display :0.0 for VA encoding\n");
+        return -1;
+    }
+    h264->va_dpy = vaGetDisplay(h264->x11_dpy);
+    va_status = vaInitialize(h264->va_dpy, &va_major_ver, &va_minor_ver);
+    CHECK_VASTATUS_RET(va_status, "vaInitialize", -1);
+
+    /* Check for Slice entrypoint */
+    vaQueryConfigEntrypoints(h264->va_dpy, VAProfileH264Baseline, entrypoints,
+                             &num_entrypoints);
+
+    for (entrypoint = 0; entrypoint < num_entrypoints; entrypoint++) {
+        if (entrypoints[entrypoint] == VAEntrypointEncSlice) {
+            break;
+        }
+    }
+
+    if (entrypoint == num_entrypoints) {
+        VNC_DEBUG("Could not find Encoder Slice entrypoint\n");
+        h264_encoder_close(h264);
+        return -1;
+    }
+
+    /* Set encode pipeline configuration */
+    attrib[0].type = VAConfigAttribRTFormat;
+    attrib[1].type = VAConfigAttribRateControl;
+    vaGetConfigAttributes(h264->va_dpy, VAProfileH264Baseline,
+                          VAEntrypointEncSlice, &attrib[0], 2);
+
+    if ((attrib[0].value & VA_RT_FORMAT_YUV420) == 0) {
+        VNC_DEBUG("YUV420 format is not supported by VA encoder\n");
+        h264_encoder_close(h264);
+        return -1;
+    }
+    if ((attrib[1].value & VA_RC_VBR) == 0) {
+        VNC_DEBUG("VBR mode is not supported by VA encoder\n");
+        h264_encoder_close(h264);
+        return -1;
+    }
+
+    attrib[0].value = VA_RT_FORMAT_YUV420;
+    attrib[1].value = VA_RC_VBR;
+    va_status = vaCreateConfig(h264->va_dpy, VAProfileH264Baseline,
+                               VAEntrypointEncSlice, &attrib[0], 2,
+                               &h264->config);
+    CHECK_VASTATUS_RET(va_status, "vaCreateConfig", -1);
+
+    /* Invalidate context */
+    h264->context = VA_INVALID_ID;
+    h264->seq_param = VA_INVALID_ID;
+    h264->pic_param = VA_INVALID_ID;
+    h264->slice_param = VA_INVALID_ID;
+    h264->coded_buf = VA_INVALID_ID;
+
+    for (i = 0; i < SID_NUMBER; ++i) {
+        h264->surface_ids[i] = VA_INVALID_ID;
+    }
+
+    /* Misc */
+    h264->frame_count = 0;
+    h264->initialized = true;
+
+    return 0;
+}
+
+static int h264_encoder_reinit(VncDisplayH264 *h264, int w, int h, int pitch,
+                               int depth)
+{
+    h264_release_context(h264);
+
+    return h264_create_context(h264, w, h, pitch, depth);
+}
+
+static void h264_encoder_close(VncDisplayH264 *h264)
+{
+    if (h264->config != VA_INVALID_ID) {
+        vaDestroyConfig(h264->va_dpy, h264->config);
+        h264->config = VA_INVALID_ID;
+    }
+    if (h264->va_dpy) {
+        vaTerminate(h264->va_dpy);
+        h264->va_dpy = NULL;
+    }
+    if (h264->x11_dpy) {
+        XCloseDisplay(h264->x11_dpy);
+        h264->x11_dpy = NULL;
+    }
+}
+
+int vnc_h264_send_framebuffer_update(VncState *vs, int ch_x,
+                                     int ch_y, int ch_w, int ch_h)
+{
+    int frame_w, frame_h;
+    int pitch, depth;
+    VncDisplayH264 *h264 = &vs->vd->h264;
+    VncH264Frame *cur_frame = &h264->cur_frame;
+    int bytes_to_send;
+
+    VNC_DEBUG("%s: x=%d; y=%d; w=%d; h=%d\n", __func__, ch_x, ch_y,
+              ch_w, ch_h);
+
+    frame_w = ds_get_width(vs->ds);
+    frame_h = ds_get_height(vs->ds);
+
+    /* first call for this frame: perform encoding */
+    if ((cur_frame->data == NULL) ||
+        (cur_frame->count != h264->frame_count)) {
+
+        if (!h264->va_dpy || (h264->config == VA_INVALID_ID)) {
+            fprintf(stderr, "VA H264 encoding selected but "
+                            "H264 encoder not initialized!\n");
+            return 0;
+        }
+
+        if ((frame_w != vs->vd->h264.cur_frame.width) ||
+            (frame_h != vs->vd->h264.cur_frame.height)) {
+            pitch = ds_get_linesize(vs->ds);
+            depth = ds_get_bytes_per_pixel(vs->ds);
+
+            VNC_DEBUG("%s: Resolution change: (%dx%d) => (%dx%d) "
+                      "(depth: %d, pitch: %d)\n", __func__,
+                      cur_frame->width, cur_frame->height,
+                      frame_w, frame_h, depth, pitch);
+
+            if (h264_encoder_reinit(h264, frame_w, frame_h,
+                                    pitch, depth) != 0) {
+                fprintf(stderr, "Error re-initializing the VA encoder!\n");
+                VNC_DEBUG("Error re-initializing the VA encoder\n");
+                return 0;
+            }
+
+            /* send full screen update on framebuffer resize/init */
+            h264->force_intra = true;
+
+            cur_frame->width = frame_w;
+            cur_frame->height = frame_h;
+        }
+
+        /* Encode the frame and get the encoded frame and its size */
+        cur_frame->is_intra = h264->force_intra ||
+                                 ((h264->frame_count % 30) == 0);
+        h264_encode_frame(h264, vnc_server_fb_ptr(vs->vd, 0, 0), 0,
+                          cur_frame->is_intra,
+                          &cur_frame->length, &cur_frame->data);
+
+        cur_frame->count = h264->frame_count;
+        h264->force_intra = false;
+    }
+
+    /* Send framebuffer update header for this rectangle */
+    vnc_framebuffer_update(vs, ch_x, ch_y, ch_w, ch_h, VNC_ENCODING_VA_H264);
+
+    /* Send frame contents only once. Multiple updates for the same frame
+     * are sent when there are multiple changed rectangles in the same
+     * framebuffer update run (see vnc_update_client() in vnc.c). */
+    bytes_to_send =
+        (vs->h264_last_sent_frame != cur_frame->count) ? cur_frame->length : 0;
+
+    /* VNC VA H264 Header: frame data size | frame_type | w | h | data */
+    vnc_write_s32(vs, bytes_to_send);
+    vnc_write_s32(vs, cur_frame->is_intra ? 2 : 0);
+    vnc_write_s32(vs, frame_w);
+    vnc_write_s32(vs, frame_h);
+    vnc_write(vs, cur_frame->data, bytes_to_send);
+
+    vs->h264_last_sent_frame = cur_frame->count;
+
+    return 1;
+}
+
+
+static int h264_get_coded_buf_size(VncDisplayH264 *h264)
+{
+    return h264->pic_width * h264->pic_height * 1.5;
+}
+
+static int h264_create_context(VncDisplayH264 *h264, int width, int height,
+                               int pitch, int depth)
+{
+    VAStatus va_status;
+    unsigned int coded_buf_size;
+    int i;
+
+    VNC_DEBUG("%s: width=%d; height=%d; pitch=%d; depth=%d\n", __func__,
+              width, height, pitch, depth);
+
+    if (h264->context != VA_INVALID_ID) {
+        VNC_DEBUG("%s: context already exists, doing nothing\n", __func__);
+        return 0;
+    }
+
+    /* Set picture related parameters */
+    h264->pic_width = width;
+    h264->pic_height = height;
+    h264->pic_depth = depth;
+    h264->pic_pitch = pitch;
+
+    /* Create context */
+    va_status = vaCreateContext(h264->va_dpy, h264->config, width, height,
+                                VA_PROGRESSIVE, 0, 0, &h264->context);
+    CHECK_VASTATUS_RET(va_status, "vaCreateContext", -1);
+
+    /* Sequence parameters */
+    VAEncSequenceParameterBufferH264 seq_h264 = {0};
+    seq_h264.level_idc = 30;
+    seq_h264.picture_width_in_mbs = (width + 15) / 16;
+    seq_h264.picture_height_in_mbs = (height + 15) / 16;
+    seq_h264.bits_per_second = 384 * 1000;
+    seq_h264.initial_qp = 26;
+    seq_h264.min_qp = 3;
+
+    va_status = vaCreateBuffer(h264->va_dpy, h264->context,
+                               VAEncSequenceParameterBufferType,
+                               sizeof(seq_h264), 1, &seq_h264,
+                               &h264->seq_param);
+    CHECK_VASTATUS_RET(va_status, "vaCreateBuffer(seq_param)", -1);
+
+    /* Surfaces */
+    va_status = vaCreateSurfaces(h264->va_dpy, width, height,
+                                 VA_RT_FORMAT_YUV420, SID_NUMBER,
+                                 &h264->surface_ids[0]);
+    CHECK_VASTATUS_RET(va_status, "vaCreateSurfaces", -1);
+    VNC_DEBUG("%s: Created surfaces:", __func__);
+    for (i = 0; i < SID_NUMBER; ++i) {
+        VNC_DEBUG(" [#%d -> 0x%x]", i, h264->surface_ids[i]);
+    }
+    VNC_DEBUG("\n");
+
+    /* Coded buffer */
+    coded_buf_size = h264_get_coded_buf_size(h264);
+    va_status = vaCreateBuffer(h264->va_dpy, h264->context,
+                               VAEncCodedBufferType, coded_buf_size, 1, NULL,
+                               &h264->coded_buf);
+    CHECK_VASTATUS_RET(va_status, "vaCreateBuffer(coded_buf)", -1);
+    VNC_DEBUG("%s: created coded_buf: id=0x%x, size=%d\n", __func__,
+              h264->coded_buf, coded_buf_size);
+
+    return 0;
+}
+
+static void h264_release_context(VncDisplayH264 *h264)
+{
+    VAStatus va_status;
+    VASurfaceStatus surface_status;
+    int i;
+
+    if (h264->context == VA_INVALID_ID) {
+        return;
+    }
+
+    /* Sync all surfaces */
+    VNC_DEBUG("%s: Syncing surfaces: ", __func__);
+    for (i = 0; i < SID_NUMBER; ++i) {
+        if (h264->surface_ids[i] != VA_INVALID_ID) {
+            VNC_DEBUG("[#%d", i);
+
+            va_status = vaSyncSurface(h264->va_dpy, h264->surface_ids[i]);
+            CHECK_VASTATUS(va_status, "vaSyncSurface");
+
+            surface_status = (VASurfaceStatus)0;
+            va_status = vaQuerySurfaceStatus(h264->va_dpy, h264->surface_ids[i],
+                                             &surface_status);
+            CHECK_VASTATUS(va_status, "vaQuerySurfaceStatus");
+
+            VNC_DEBUG(" -> 0x%x] ", surface_status);
+        }
+    }
+    VNC_DEBUG("done!\n");
+
+    /* Release parameter buffers */
+    VNC_DEBUG("%s: seq_param = 0x%x\n", __func__, h264->seq_param);
+    if (h264->seq_param != VA_INVALID_ID) {
+        va_status = vaDestroyBuffer(h264->va_dpy, h264->seq_param);
+        CHECK_VASTATUS(va_status, "vaDestroyBuffer(seq_param)");
+        h264->seq_param = VA_INVALID_ID;
+    }
+
+    VNC_DEBUG("%s: pic_param = 0x%x\n", __func__, h264->pic_param);
+    if (h264->pic_param != VA_INVALID_ID) {
+        va_status = vaDestroyBuffer(h264->va_dpy, h264->pic_param);
+        CHECK_VASTATUS(va_status, "vaDestroyBuffer(pic_param)");
+        h264->pic_param = VA_INVALID_ID;
+    }
+
+    VNC_DEBUG("%s: slice_param = 0x%x\n", __func__, h264->slice_param);
+    if (h264->slice_param != VA_INVALID_ID) {
+        va_status = vaDestroyBuffer(h264->va_dpy, h264->slice_param);
+        CHECK_VASTATUS(va_status, "vaDestroyBuffer(slice_param)");
+        h264->slice_param = VA_INVALID_ID;
+    }
+
+    VNC_DEBUG("%s: coded_buf = 0x%x\n", __func__, h264->coded_buf);
+    if (h264->coded_buf != VA_INVALID_ID) {
+        va_status = vaDestroyBuffer(h264->va_dpy, h264->coded_buf);
+        CHECK_VASTATUS(va_status, "vaDestroyBuffer(coded_buf)");
+        h264->coded_buf = VA_INVALID_ID;
+    }
+
+    /* Release surfaces */
+    if (h264->surface_ids[0] != VA_INVALID_ID) {
+        va_status = vaDestroySurfaces(h264->va_dpy, &h264->surface_ids[0],
+                                      SID_NUMBER);
+        CHECK_VASTATUS(va_status, "vaDestroySurfaces");
+    }
+    for (i = 0; i < SID_NUMBER; ++i) {
+        h264->surface_ids[i] = VA_INVALID_ID;
+    }
+
+    /* Release context */
+    va_status = vaDestroyContext(h264->va_dpy, h264->context);
+    CHECK_VASTATUS(va_status, "vaDestroyContext");
+    h264->context = VA_INVALID_ID;
+}
+
+
+static void h264_encode_frame(VncDisplayH264 *h264,
+                              const unsigned char *const in_buf, const int last,
+                              const bool is_intra, int *out_length,
+                              unsigned char **out_buf)
+{
+    VAStatus va_status;
+    VASurfaceStatus surface_status;
+    VACodedBufferSegment *coded_buf_segment = NULL;
+    int coded_buf_size;
+
+    va_status = vaBeginPicture(h264->va_dpy, h264->context,
+                               h264->surface_ids[SID_INPUT_PICTURE]);
+    CHECK_VASTATUS(va_status, "vaBeginPicture");
+
+    /* Set parameters and put picture on the input picture surface */
+    h264_prepare_input(h264, in_buf, is_intra, last);
+
+    /* Sync input surface */
+    va_status = vaEndPicture(h264->va_dpy, h264->context);
+    CHECK_VASTATUS(va_status, "vaEndPicture");
+
+    va_status = vaSyncSurface(h264->va_dpy,
+                              h264->surface_ids[SID_INPUT_PICTURE]);
+    CHECK_VASTATUS(va_status, "vaSyncSurface");
+
+    surface_status = (VASurfaceStatus)0;
+    va_status = vaQuerySurfaceStatus(h264->va_dpy,
+                                     h264->surface_ids[SID_INPUT_PICTURE],
+                                     &surface_status);
+    CHECK_VASTATUS(va_status, "vaQuerySurfaceStatus");
+
+    /* Get H.264 encoded data */
+    if (h264->coded_buf != VA_INVALID_ID) {
+        va_status = vaUnmapBuffer(h264->va_dpy, h264->coded_buf);
+        CHECK_VASTATUS(va_status, "vaUnmapBuffer(coded_buf)");
+    }
+
+    va_status = vaMapBuffer(h264->va_dpy, h264->coded_buf,
+                            (void **)(&coded_buf_segment));
+    CHECK_VASTATUS(va_status, "vaMapBuffer");
+
+    *out_buf = (unsigned char *)coded_buf_segment->buf;
+
+    if (coded_buf_segment->next) {
+        VNC_DEBUG("%s: Unsupported: multiple segments detected. "
+                  "Skipping next segments\n", __func__);
+    }
+
+    coded_buf_size = h264_get_coded_buf_size(h264);
+
+    *out_length = get_buffer_length(*out_buf, coded_buf_size);
+}
+
+
+static void h264_prepare_input(VncDisplayH264 *h264,
+                               const unsigned char *const in_buf,
+                               const bool is_intra, const bool is_last)
+{
+    VAStatus va_status;
+    VAEncPictureParameterBufferH264 pic_h264;
+    VAEncSliceParameterBuffer slice_h264;
+    VACodedBufferSegment *coded_buf_segment = NULL;
+    VABufferID tempID;
+
+    VNC_DEBUG("%s: in_buf=%p; is_intra=%d; is_last=%d\n", __func__, in_buf,
+              is_intra, is_last);
+
+    /* Set sequence parameters */
+    va_status = vaRenderPicture(h264->va_dpy, h264->context,
+                                &h264->seq_param, 1);
+    CHECK_VASTATUS(va_status, "vaRenderPicture(seq_param)");
+
+    /* Set picture parameters */
+    pic_h264.reference_picture = h264->surface_ids[SID_REFERENCE_PICTURE];
+    pic_h264.reconstructed_picture = h264->surface_ids[SID_RECON_PICTURE];
+    pic_h264.coded_buf = h264->coded_buf;
+    pic_h264.picture_width = h264->pic_width;
+    pic_h264.picture_height = h264->pic_height;
+    pic_h264.last_picture = is_last;
+
+    if (h264->pic_param != VA_INVALID_ID) {
+        va_status = vaDestroyBuffer(h264->va_dpy, h264->pic_param);
+        CHECK_VASTATUS(va_status, "vaDestroyBuffer(pic_parameter)");
+    }
+
+    va_status = vaCreateBuffer(h264->va_dpy, h264->context,
+                               VAEncPictureParameterBufferType,
+                               sizeof(pic_h264), 1, &pic_h264,
+                               &h264->pic_param);
+    CHECK_VASTATUS(va_status, "vaCreateBuffer(pic_parameter)");
+
+    va_status = vaRenderPicture(h264->va_dpy, h264->context,
+                                &h264->pic_param, 1);
+    CHECK_VASTATUS(va_status, "vaRenderPicture(pic_parameter)");
+
+    /* Allocate and zero out coded buffer segment.
+     * See also get_buffer_length(). */
+    va_status = vaMapBuffer(h264->va_dpy, h264->coded_buf,
+                            (void **)(&coded_buf_segment));
+    CHECK_VASTATUS(va_status, "vaMapBuffer(coded_buf_segment)");
+
+    /* FIXME: Is this (still) needed? */
+    memset((unsigned char *)coded_buf_segment->buf, 0, coded_buf_segment->size);
+
+    va_status = vaUnmapBuffer(h264->va_dpy, h264->coded_buf);
+    CHECK_VASTATUS(va_status, "vaUnmapBuffer(coded_buf)");
+
+    /* Set slice parameters */
+    slice_h264.start_row_number = 0;
+    slice_h264.slice_height = h264->pic_height / 16;    /* Measured in MB */
+    slice_h264.slice_flags.bits.is_intra = is_intra;
+    slice_h264.slice_flags.bits.disable_deblocking_filter_idc = 0;
+
+    if (h264->slice_param != VA_INVALID_ID) {
+        va_status = vaDestroyBuffer(h264->va_dpy, h264->slice_param);
+        CHECK_VASTATUS(va_status, "vaDestroyBuffer(slice_parameter)");
+    }
+
+    va_status = vaCreateBuffer(h264->va_dpy, h264->context,
+                               VAEncSliceParameterBufferType,
+                               sizeof(slice_h264), 1, &slice_h264,
+                               &h264->slice_param);
+    CHECK_VASTATUS(va_status, "vaCreateBuffer(slice_parameter)");
+
+    va_status = vaRenderPicture(h264->va_dpy, h264->context,
+                                &h264->slice_param, 1);
+    CHECK_VASTATUS(va_status, "vaRenderPicture(slice_parameter)");
+
+    /* Copy RGBA input buffer to VA surface */
+    rgba_to_surface(h264, in_buf, h264->surface_ids[SID_INPUT_PICTURE]);
+
+    /* Prepare for next picture */
+    tempID = h264->surface_ids[SID_RECON_PICTURE];
+    h264->surface_ids[SID_RECON_PICTURE] =
+                                      h264->surface_ids[SID_REFERENCE_PICTURE];
+    h264->surface_ids[SID_REFERENCE_PICTURE] = tempID;
+}
+
+static void rgba_to_surface(const VncDisplayH264 *h264,
+                            const unsigned char *const inbuf,
+                            VASurfaceID surface_id)
+{
+    VAStatus va_status;
+    VAImage image;
+    void *pbuffer = NULL;
+    const unsigned char *psrc = inbuf;
+    const unsigned char *s;
+    unsigned char *pdst = NULL;
+    unsigned char *dst_y, *dst_uv;
+    unsigned char *dst_uv_line = NULL;
+    int i, j;
+
+    VNC_DEBUG("%s: inbuf=%p; width=%d; height=%d; frame=%ld\n", __func__,
+              inbuf, h264->pic_width, h264->pic_height, h264->frame_count);
+
+    va_status = vaDeriveImage(h264->va_dpy, surface_id, &image);
+    CHECK_VASTATUS(va_status, "vaDeriveImage");
+
+    va_status = vaMapBuffer(h264->va_dpy, image.buf, &pbuffer);
+    CHECK_VASTATUS(va_status, "vaMapBuffer(image.buf)");
+
+    pdst = (unsigned char *)pbuffer;
+    dst_uv_line = pdst + image.offsets[1];
+
+    /* RGBA => NV12 */
+    for (i = 0; i < h264->pic_height; ++i) {
+        dst_y = (pdst + image.offsets[0]) + i*image.pitches[0];
+        dst_uv = dst_uv_line;
+        s = psrc;
+        for (j = 0; j < h264->pic_width; j++) {
+            *dst_y++ = ((66*s[0] + 129*s[1] + 25*s[2] + 128) >> 8) + 16;
+            /* NV12 means subsampling by 2 in x and y */
+            if ((0 == i%2) && (0 == j%2)) {
+                *dst_uv++ = ((112*s[0] - 94*s[1] - 18*s[2] + 128) >> 8) + 128;
+                *dst_uv++ = ((-38*s[0] - 74*s[1] + 112*s[2] + 128) >> 8) + 128;
+            }
+            s += h264->pic_depth;
+        }
+        psrc += h264->pic_pitch;
+        if (0 == i%2) {
+            dst_uv_line += image.pitches[1];
+        }
+    }
+
+    va_status = vaUnmapBuffer(h264->va_dpy, image.buf);
+    CHECK_VASTATUS(va_status, "vaUnmapBuffer(image.buf)");
+
+    va_status = vaDestroyImage(h264->va_dpy, image.image_id);
+    CHECK_VASTATUS(va_status, "vaDestroyImage");
+}
+
+static int get_buffer_length(const unsigned char * const buffer,
+                             int buffer_length)
+{
+    int i;
+    for (i = buffer_length - 1; i >= 0; i--) {
+        if (buffer[i]) {
+            break;
+        }
+    }
+
+    return i + 1;
+}
diff --git a/ui/vnc-enc-h264.h b/ui/vnc-enc-h264.h
new file mode 100644
index 0000000..0047d9b
--- /dev/null
+++ b/ui/vnc-enc-h264.h
@@ -0,0 +1,74 @@ 
+/*
+ * QEMU VNC display driver: VAAPI based H.264 encoding
+ *
+ * Copyright (c) 2012 Intel Corporation.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sub license, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the
+ * next paragraph) shall be included in all copies or substantial portions
+ * of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
+ * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+ * IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
+ * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+ * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
+ * USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef __QEMU_VNC_ENC_H264_H__
+#define __QEMU_VNC_ENC_H264_H__
+
+#include <X11/Xlib.h>
+#include <va/va_x11.h>
+
+#define SID_NUMBER                              3
+#define SID_INPUT_PICTURE                       0
+#define SID_REFERENCE_PICTURE                   1
+#define SID_RECON_PICTURE                       2
+
+typedef struct VncH264Frame {
+    unsigned char *data;
+    int length;
+    int is_intra;
+    unsigned long count;
+    int width;
+    int height;
+} VncH264Frame;
+
+typedef struct VncDisplayH264 {
+    /* Encoding context related */
+    Display *x11_dpy;
+    VADisplay va_dpy;
+    VAConfigID config;
+    VAContextID context;
+    VASurfaceID surface_ids[SID_NUMBER];
+    VABufferID seq_param;       /* Sequence parameters */
+    VABufferID pic_param;       /* Picture parameters */
+    VABufferID slice_param;     /* Slice parameters */
+    VABufferID coded_buf;       /* Output buffer, encoded data */
+
+    /* Picture related */
+    int pic_width;
+    int pic_height;
+    int pic_depth;
+    int pic_pitch;
+
+    /* Data for last encoded frame, e.g. to use it for multiple clients */
+    VncH264Frame cur_frame;
+
+    /* Misc */
+    bool initialized;
+    unsigned long frame_count;
+    bool force_intra;
+} VncDisplayH264;
+
+#endif /* __QEMU_VNC_ENC_H264_H__ */
diff --git a/ui/vnc-jobs.c b/ui/vnc-jobs.c
index 0bfc0c5..ad02e37 100644
--- a/ui/vnc-jobs.c
+++ b/ui/vnc-jobs.c
@@ -194,6 +194,9 @@  static void vnc_async_encoding_start(VncState *orig, VncState *local)
     local->hextile = orig->hextile;
     local->zrle = orig->zrle;
     local->output =  queue->buffer;
+#ifdef CONFIG_VNC_LIBVA
+    local->h264_last_sent_frame = orig->h264_last_sent_frame;
+#endif
     local->csock = -1; /* Don't do any network work on this thread */
 
     buffer_reset(&local->output);
@@ -206,6 +209,9 @@  static void vnc_async_encoding_end(VncState *orig, VncState *local)
     orig->hextile = local->hextile;
     orig->zrle = local->zrle;
     orig->lossy_rect = local->lossy_rect;
+#ifdef CONFIG_VNC_LIBVA
+    orig->h264_last_sent_frame = local->h264_last_sent_frame;
+#endif
 
     queue->buffer = local->output;
 }
diff --git a/ui/vnc.c b/ui/vnc.c
index ff4e2ae..d977563 100644
--- a/ui/vnc.c
+++ b/ui/vnc.c
@@ -713,6 +713,11 @@  int vnc_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
         case VNC_ENCODING_ZYWRLE:
             n = vnc_zywrle_send_framebuffer_update(vs, x, y, w, h);
             break;
+#ifdef CONFIG_VNC_LIBVA
+        case VNC_ENCODING_VA_H264:
+            n = vnc_h264_send_framebuffer_update(vs, x, y, w, h);
+            break;
+#endif
         default:
             vnc_framebuffer_update(vs, x, y, w, h, VNC_ENCODING_RAW);
             n = vnc_raw_send_framebuffer_update(vs, x, y, w, h);
@@ -1864,6 +1869,16 @@  static void set_encodings(VncState *vs, int32_t *encodings, size_t n_encodings)
             vs->features |= VNC_FEATURE_ZYWRLE_MASK;
             vs->vnc_encoding = enc;
             break;
+#ifdef CONFIG_VNC_LIBVA
+        case VNC_ENCODING_VA_H264:
+            if (vnc_h264_encoder_init(&vs->vd->h264) == 0) {
+                vs->features |= VNC_FEATURE_VA_H264_MASK;
+                vs->vnc_encoding = enc;
+            } else {
+                fprintf(stderr, "Failed to initialize VA H264 encoder\n");
+            }
+            break;
+#endif
         case VNC_ENCODING_DESKTOPRESIZE:
             vs->features |= VNC_FEATURE_RESIZE_MASK;
             break;
@@ -2659,6 +2674,12 @@  static void vnc_refresh(void *opaque)
     has_dirty = vnc_refresh_server_surface(vd);
     vnc_unlock_display(vd);
 
+#ifdef CONFIG_VNC_LIBVA
+    if (has_dirty) {
+        ++vd->h264.frame_count;
+    }
+#endif
+
     QTAILQ_FOREACH_SAFE(vs, &vd->clients, next, vn) {
         rects += vnc_update_client(vs, has_dirty);
         /* vs might be free()ed here */
@@ -2770,6 +2791,11 @@  void vnc_init_state(VncState *vs)
 
     QTAILQ_INSERT_HEAD(&vd->clients, vs, next);
 
+#ifdef CONFIG_VNC_LIBVA
+    vs->h264_last_sent_frame = vd->h264.frame_count - 1;
+    vd->h264.force_intra = true;
+#endif
+
     vga_hw_update();
 
     vnc_write(vs, "RFB 003.008\n", 12);
diff --git a/ui/vnc.h b/ui/vnc.h
index 45d7686..89c5027 100644
--- a/ui/vnc.h
+++ b/ui/vnc.h
@@ -102,6 +102,9 @@  typedef struct VncDisplay VncDisplay;
 #ifdef CONFIG_VNC_WS
 #include "vnc-ws.h"
 #endif
+#ifdef CONFIG_VNC_LIBVA
+#include "vnc-enc-h264.h"
+#endif
 
 struct VncRectStat
 {
@@ -175,6 +178,9 @@  struct VncDisplay
 #ifdef CONFIG_VNC_SASL
     VncDisplaySASL sasl;
 #endif
+#ifdef CONFIG_VNC_LIBVA
+    VncDisplayH264 h264;
+#endif
 };
 
 typedef struct VncTight {
@@ -320,6 +326,10 @@  struct VncState
     VncZrle zrle;
     VncZywrle zywrle;
 
+#ifdef CONFIG_VNC_LIBVA
+    unsigned long h264_last_sent_frame;
+#endif
+
     Notifier mouse_mode_notifier;
 
     QTAILQ_ENTRY(VncState) next;
@@ -375,6 +385,7 @@  enum {
 #define VNC_ENCODING_TRLE                 0x0000000f
 #define VNC_ENCODING_ZRLE                 0x00000010
 #define VNC_ENCODING_ZYWRLE               0x00000011
+#define VNC_ENCODING_VA_H264              0x48323634
 #define VNC_ENCODING_COMPRESSLEVEL0       0xFFFFFF00 /* -256 */
 #define VNC_ENCODING_QUALITYLEVEL0        0xFFFFFFE0 /* -32  */
 #define VNC_ENCODING_XCURSOR              0xFFFFFF10 /* -240 */
@@ -424,6 +435,7 @@  enum {
 #define VNC_FEATURE_TIGHT_PNG                8
 #define VNC_FEATURE_ZRLE                     9
 #define VNC_FEATURE_ZYWRLE                  10
+#define VNC_FEATURE_VA_H264                 11
 
 #define VNC_FEATURE_RESIZE_MASK              (1 << VNC_FEATURE_RESIZE)
 #define VNC_FEATURE_HEXTILE_MASK             (1 << VNC_FEATURE_HEXTILE)
@@ -436,6 +448,7 @@  enum {
 #define VNC_FEATURE_TIGHT_PNG_MASK           (1 << VNC_FEATURE_TIGHT_PNG)
 #define VNC_FEATURE_ZRLE_MASK                (1 << VNC_FEATURE_ZRLE)
 #define VNC_FEATURE_ZYWRLE_MASK              (1 << VNC_FEATURE_ZYWRLE)
+#define VNC_FEATURE_VA_H264_MASK             (1 << VNC_FEATURE_VA_H264)
 
 
 /* Client -> Server message IDs */
@@ -581,4 +594,9 @@  int vnc_zrle_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
 int vnc_zywrle_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
 void vnc_zrle_clear(VncState *vs);
 
+#ifdef CONFIG_VNC_LIBVA
+int vnc_h264_encoder_init(VncDisplayH264 *h264);
+int vnc_h264_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
+#endif
+
 #endif /* __QEMU_VNC_H */