From patchwork Wed Sep 28 13:05:36 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Llu=C3=ADs_Vilanova?= X-Patchwork-Id: 676248 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3skdqV41vVz9s5w for ; Wed, 28 Sep 2016 23:30:14 +1000 (AEST) Received: from localhost ([::1]:58863 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bpEw7-0005zc-By for incoming@patchwork.ozlabs.org; Wed, 28 Sep 2016 09:30:11 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34530) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bpEdI-0003nG-Oo for qemu-devel@nongnu.org; Wed, 28 Sep 2016 09:10:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bpEdD-0000Qg-FM for qemu-devel@nongnu.org; Wed, 28 Sep 2016 09:10:43 -0400 Received: from roura.ac.upc.es ([147.83.33.10]:57526) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bpEdD-0000NU-3d for qemu-devel@nongnu.org; Wed, 28 Sep 2016 09:10:39 -0400 Received: from gw-2.ac.upc.es (gw-2.ac.upc.es [147.83.30.8]) by roura.ac.upc.es (8.13.8/8.13.8) with ESMTP id u8SD5aND032470; Wed, 28 Sep 2016 15:05:36 +0200 Received: from localhost (unknown [84.88.51.85]) by gw-2.ac.upc.es (Postfix) with ESMTPSA id 4B343149; Wed, 28 Sep 2016 15:05:36 +0200 (CEST) From: =?utf-8?b?TGx1w61z?= Vilanova To: qemu-devel@nongnu.org Date: Wed, 28 Sep 2016 15:05:36 +0200 Message-Id: <147506793596.18187.14194318363118441846.stgit@fimbulvetr.bsc.es> X-Mailer: git-send-email 2.9.3 In-Reply-To: <147506793053.18187.11460694747365824870.stgit@fimbulvetr.bsc.es> References: <147506793053.18187.11460694747365824870.stgit@fimbulvetr.bsc.es> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-MIME-Autoconverted: from 8bit to quoted-printable by roura.ac.upc.es id u8SD5aND032470 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 147.83.33.10 Subject: [Qemu-devel] [PATCH v3 1/6] hypertrace: Add documentation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Stefan Hajnoczi , Luiz Capitulino Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Lluís Vilanova --- docs/hypertrace.txt | 232 +++++++++++++++++++++++++++++++++++++++++++++++++++ docs/tracing.txt | 3 + 2 files changed, 235 insertions(+) create mode 100644 docs/hypertrace.txt diff --git a/docs/hypertrace.txt b/docs/hypertrace.txt new file mode 100644 index 0000000..871a824 --- /dev/null +++ b/docs/hypertrace.txt @@ -0,0 +1,232 @@ += Hypertrace channel = + +Copyright (C) 2016 Lluís Vilanova + +This work is licensed under the terms of the GNU GPL, version 2 or later. +See the COPYING file in the top-level directory. + + +The hypertrace channel allows guest code to emit events in QEMU (the host) using +its tracing infrastructure (see "docs/trace.txt"). This works in both 'system' +and 'user' modes. Therefore, hypertrace is to tracing what hypercalls are to +system calls. + +The hypertrace channel can be used for various purposes: + +* Efficient selective event tracing guided by guest code semantics (see the + example in "Quick guide" below). The example shows how to use this to identify + "regions of interest" in your guest code. It then uses these regions to trace + QEMU's behaviour during their execution (the example traces memory accesses), + without paying the price of tracing events outside the interest regions. + +* Mark "progress points" in guest code (e.g., processed client requests, + scheduled processes, etc), so that they can be easily traced and correlated + between QEMU's various tracing events and the guest's own tracing + infrastructure (e.g., Linux's tracepoints). + +* You can also use regions of interest and progress points on the guest code to + time the performance of new TCG optimizations. Each hypertrace event comes + with a host timestamp, making it easy to compare the host execution times of + interesting guest code. + +Hypertrace highlights: + +* Works with 'system' and 'user' mode. + +* Minimal setup for the guest; QEMU provides support guest code libraries to + make 'system' mode with Linux and 'user' mode work out of the box. + +* Independent of guest architecture; the guest code uses accesses to special + memory regions, as opposed to redefining instruction semantics. + +* Negligible guest overhead; emitting a hypertrace event requires a single guest + memory access, making it as unobtrusive as possible. + +Warning: The hypertrace channel in 'system' mode is presented as a PCI device, +and thus will only be available on systems with support for PCI. You can get the +list of guests with PCI support with 'grep pci.mak default-configs/*'. + + +== Quick guide == + +This shows an example of using the hypertrace channel to trace the guest memory +accesses only in a specific guest code region, which is identified by calls to +the hypertrace channel. + +We are going to trace memory accesses to disk using QEMU's "log" backend, and +will use QEMU's "dtrace" (SystemTap) backend to ensure memory accesses are only +traced in the guest code region of interest. The first time the guest code +invokes the hypertrace channel, we will start tracing the +"guest_mem_before_exec" event using dtrace, and then will disable it the second +time around. + +Tracing is done with "log" because it is more efficient than using "dtrace" in +high-volume events like memory accesses. If the event rate is low, you can +ignore the "log" backend and simply use "dtrace"'s printf, while still using the +hypertrace events to identify guest semantics. + +1. Set the tracing backends and number of arguments for the hypertrace events: + + mkdir /tmp/qemu-build + cd /tmp/qemu-build + /path/to/qemu-source/configure \ + --enable-trace-backends=dtrace,log \ + --with-hypertrace-args=4 \ + --prefix=/tmp/qemu-install + make -j install + +2. Enable the event "guest_mem_before_exec": + + sed -i -e 's/disable vcpu tcg guest_mem_before/vcpu tcg guest_mem_before/g' /path/to/qemu-source/trace-events + +3. Compile the guest support code: + + make -C /tmp/qemu-build/x86_64-linux-user/hypertrace/guest/user + make -C /tmp/qemu-build/x86_64-softmmu/hypertrace/guest/user + make -C /tmp/qemu-build/x86_64-softmmu/hypertrace/guest/linux-module + + If you need to cross-compile the guest library, set the 'CC' variable: + + make -C /tmp/qemu-build/mipsel-linux-user/hypertrace/guest/user CC=mipsel-gnu-linux-gcc + +3. Create a guest application using "qemu-hypertrace.h" to interact with the + hypertrace channel: + + cat > /tmp/my-hypertrace.c <<\EOF + #include + #include + #include + #include + #include + + + int main(int argc, char **argv) + { + char *base = NULL; + if (argc > 1) { + base = argv[1]; + } + + /* In 'user' mode this path must be the same we will use to start QEMU. */ + if (qemu_hypertrace_init(base) != 0) { + perror("error: qemu_hypertrace_init"); + abort(); + } + + /* Set additional event arguments */ + uint64_t client = 0; + uint64_t *data = qemu_hypertrace_data(client); + data[0] = 0xcafe; + data[1] = 0xdead; + data[2] = 0xbeef; + + /* Emit event to start tracing */ + qemu_hypertrace(client, 1); + + /* Computation in between */ + printf("Some computation...\n"); + + /* Emit event to stop tracing */ + qemu_hypertrace(client, 0); + } + EOF + + gcc -o /tmp/my-hypertrace-user /tmp/my-hypertrace.c \ + /tmp/qemu-build/x86_64-linux-user/hypertrace/guest/user/libqemu-hypertrace-guest.a \ + -I/tmp/qemu-install/include -lpthread + + gcc -o /tmp/my-hypertrace-softmmu /tmp/my-hypertrace.c \ + /tmp/qemu-build/x86_64-softmmu/hypertrace/guest/user/libqemu-hypertrace-guest.a \ + -I/tmp/qemu-install/include -lpthread + +4. Create a SystemTap script to control event tracing: + + cat > /tmp/my-hypertrace-script.stp <<\EOF + #!/usr/bin/env stap + + %{ + #include + %} + + function enable_mem:long() + %{ + /* Tell QEMU's monitor to enable tracing */ + char *argv[4] = {"/bin/sh", "-c", "echo 'trace-event guest_mem_before_exec on' | telnet localhost 1234", NULL}; + printk(KERN_ERR "enable\n"); + call_usermodehelper(argv[0], argv, NULL, UMH_WAIT_EXEC); + /* Wait for changes to apply */ + msleep(1000); + printk(KERN_ERR "enabled\n"); + STAP_RETURN(0); + %} + + function disable_mem:long() + %{ + char *argv[4] = {"/bin/sh", "-c", "echo 'trace-event guest_mem_before_exec off' | telnet localhost 1234", NULL}; + printk(KERN_ERR "disable\n"); + call_usermodehelper(argv[0], argv, NULL, UMH_WAIT_EXEC); + msleep(1000); + printk(KERN_ERR "disabled\n"); + STAP_RETURN(0); + %} + + probe process("/tmp/qemu-install/bin/qemu-*").mark("guest_hypertrace") + { + if ($arg0 == 1) { + enable_mem() + } else if ($arg0 == 0) { + disable_mem() + } + } + EOF + +4. Run a guest system with access to QEMU's hypertrace: + + stap -g /tmp/my-hypertrace-script.stp -c \ + '/tmp/qemu-install/bin/qemu-system-x86_64 \ + -device hypertrace \ + -monitor tcp:localhost:1234,server,nowait \ + -trace enable=guest_hypertrace -D /dev/stdout \ + ...' + + And inside the VM: + + sudo /tmp/my-hypertrace-softmmu + + The result will be something like this: + + VNC server running on ::1:5900 + 23071@1473096085.744211:guest_hypertrace cpu=0x5602e1f49c10 arg1=0x000000000000cafe arg2=0x000000000000babe arg3=0x000000000000dead arg4=0x000000000000beef + 23071@1473096085.745763:guest_mem_before_trans cpu=0x5602e1f49c10 info=19 + 23071@1473096085.745907:guest_mem_before_trans cpu=0x5602e1f49c10 info=3 + 23071@1473096085.752368:guest_mem_before_trans cpu=0x5602e1f49c10 info=3 + 23071@1473096085.752384:guest_mem_before_trans cpu=0x5602e1f49c10 info=19 + 23071@1473096086.756117:guest_hypertrace cpu=0x5602e1f49c10 arg1=0x000000000000cafe arg2=0x000000000000babe arg3=0x000000000000dead arg4=0x000000000000beef + +Similarly, you can enable hypertrace when running standalone guest applications: + + /tmp/qemu-install/bin/qemu-x86_64 \ + -hypertrace /tmp/hypertrace \ + -trace enable=guest* -D /dev/stdout \ + /tmp/my-hypertrace-user /tmp/hypertrace + +You can also use hypertrace inside the Linux's kernel code with the provided +guest module (see "/tmp/qemu-install/include/linux/qemu-hypertrace.h"): + + sudo insmod /tmp/qemu-build/x86_64-softmmu/hypertrace/guest/linux-module/qemu-hypertrace.ko + + +== Details == + +To make it more efficient in terms of guest and host time, hypertrace provides +two different memory areas (channels). + +The control channel is used by the guest to tell QEMU that new data is ready to +be processed in the data channel. Writes to the control channel are intercepted +by QEMU, which emits the "hypertrace" tracing event. + +The data channel is a regular memory buffer used by the guest to write +additional event arguments before raising the event through the control channel. + +Both channels accept different "per-client offsets" to enable multiple guest +threads or CPUs to use the hypertrace channel without having to synchronize. diff --git a/docs/tracing.txt b/docs/tracing.txt index e62444c..97d6efd 100644 --- a/docs/tracing.txt +++ b/docs/tracing.txt @@ -5,6 +5,9 @@ This document describes the tracing infrastructure in QEMU and how to use it for debugging, profiling, and observing execution. +See "docs/hypertrace.txt" to correlate guest tracing events with those in the +QEMU host. + == Quickstart == 1. Build with the 'simple' trace backend: