[RFC] File-backed memory maps

Hello,

This patch makes QEMU optionally use file-backed mmaps for the
ram_alloc'ed memory from hw/pc.c and hw/vga.c. I've only tested on
Linux, but I haven't knowingly used Linux specific features.

It means that

  qemu ... -sharedmaps prefix:vga,bios

makes two files named "prefix.vga" and "prefix.bios" which contain the
respective memory regions. The maps I've named are 640k, before_4g,
after_4g, vga, bios and option_rom.

I'm mainly interested in the idea of moving the VNC server into its own
process. It would listen for connections as usual and then send
framebuffer updates from the file. Doing that also requires a
side-channel for communicating graphics mode updates and peripheral
input between QEMU and the VNC server. (Something like "-mouse
<char-dev-spec> -keyboard <char-dev-spec>", perhaps?)

This would make it (relatively) easy for people to create other
interfaces to a VM, like RDP, or an MPEG stream.

To be honest, I'm not sure how useful the mappings other than vga could
be.

I also considered mmaping a framebuffer in a known format (e.g. RGBA)
and swapping/reformating the VGA buffer into that periodically. It
would probably be another display type, and could make it even easier
for other processes. OTOH, it would introduce additional latency which
could be an issue for some?

But mainly I didn't do that because I don't know QEMU's innards well
enough to do that yet. :-)

Signed-off-by: Nathan Baum <nathan@parenthephobia.org.uk>

Nathan Baum wrote:
> makes two files named "prefix.vga" and "prefix.bios" which contain the
> respective memory regions. The maps I've named are 640k, before_4g,
> after_4g, vga, bios and option_rom.
> 
> I'm mainly interested in the idea of moving the VNC server into its own
> process. It would listen for connections as usual and then send
> framebuffer updates from the file. Doing that also requires a
> side-channel for communicating graphics mode updates and peripheral
> input between QEMU and the VNC server. (Something like "-mouse
> <char-dev-spec> -keyboard <char-dev-spec>", perhaps?)

Are there any cache coherency issues?

On architectures other than x86, sometimes data written by one process
is not visible to another process mapping the same file, until the
writing process flushes it's cache for those pages.  Whether that's
necessary depends on the address that pages are mapped to.  Afaik,
normally Linux chooses an address where this issue is avoided, but if
you specify it with MAP_FIXED (or whatever KVM does to map pages),
then there's cache coherency to deal with.

-- Jamie

On Fri, 2009-09-18 at 14:59 +0100, Jamie Lokier wrote:
> Nathan Baum wrote:
> > makes two files named "prefix.vga" and "prefix.bios" which contain the
> > respective memory regions. The maps I've named are 640k, before_4g,
> > after_4g, vga, bios and option_rom.
> > 
> > I'm mainly interested in the idea of moving the VNC server into its own
> > process. It would listen for connections as usual and then send
> > framebuffer updates from the file. Doing that also requires a
> > side-channel for communicating graphics mode updates and peripheral
> > input between QEMU and the VNC server. (Something like "-mouse
> > <char-dev-spec> -keyboard <char-dev-spec>", perhaps?)
> 
> Are there any cache coherency issues?

Hmm. I expect so.

I'm not sure what the consequences of this are, or whether cache
decoherence itself is the major issue. Even if cache coherence is
guaranteed, the contents of VRAM can change whilst the "display server"
is processing it, likely with undesirable results, unless it were double
buffered.

If, upon a dpy_update, VRAM was blitted to "prefix.vga" which we then
msync(), a display server could be sure of always seeing its copy of the
VRAM in a consistent state. It seems to me that might as well be
implemented as the new display type I mentioned later in my post, rather
than a double-buffered version of this more general memory sharing
feature (who wants to be blitting 4GB of RAM around anyway?). I assume
that with that arrangement, cache decoherence wouldn't be an issue on
any platform.

I'm not sure that would leave this patch with much to do. :-)

> On architectures other than x86, sometimes data written by one process
> is not visible to another process mapping the same file, until the
> writing process flushes it's cache for those pages.  Whether that's
> necessary depends on the address that pages are mapped to.  Afaik,
> normally Linux chooses an address where this issue is avoided, but if
> you specify it with MAP_FIXED (or whatever KVM does to map pages),
> then there's cache coherency to deal with.

> -- Jamie

Nathan Baum wrote:
> I'm mainly interested in the idea of moving the VNC server into its own
> process. It would listen for connections as usual and then send
> framebuffer updates from the file. Doing that also requires a
> side-channel for communicating graphics mode updates and peripheral
> input between QEMU and the VNC server. (Something like "-mouse
> <char-dev-spec> -keyboard <char-dev-spec>", perhaps?)
>   

I think the preferred way to do this would be to introduce a shared 
memory encoding to VNC.  You could then implement or whatever as a VNC 
client that used this shared memory transport.

You can't always share VGA remote because sometimes it's in planar 
mode.  However, in the cases where memory is proper ram, you could 
potentially share that memory directly resulting in zero copies.

I think the ideal thing to do would be to share a file descriptor that 
was mmap()'able.  Sharing sys v IPC keys is also a possibility and would 
better integrate with XShmImage.

Regards,

Anthony Liguori

Nathan Baum wrote:
> > On architectures other than x86, sometimes data written by one process
> > is not visible to another process mapping the same file, until the
> > writing process flushes it's cache for those pages.  Whether that's
> > necessary depends on the address that pages are mapped to.  Afaik,
> > normally Linux chooses an address where this issue is avoided, but if
> > you specify it with MAP_FIXED (or whatever KVM does to map pages),
> > then there's cache coherency to deal with.

I missed out something important: Not only does the writing process
have to clean it's cache (write dirty cachelines); the reading process
has to invalidate it's cache too.

At the system call level, that means the reading process must call
msync().  Until it does that, it may still see some of the data
change, but it's not guaranteed to see all the changes.

If the writer's copying from QEMU's VRAM to a copy, during that copy,
the reading process will see something uncertain; probably not very
useful for a VNC server.  They would need to coordinate this.  Not
sure if that buys anything over whatever KVM has to do already :-)

RDP server add-ons etc. are a good idea, imho, but perhaps can be
implemented as a shared library loaded into qemu?

-- Jamie