[{"id":1768655,"web_url":"http://patchwork.ozlabs.org/comment/1768655/","msgid":"","list_archive_url":null,"date":"2017-09-14T14:41:38","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":5111,"url":"http://patchwork.ozlabs.org/api/people/5111/","name":"Peter Maydell","email":"peter.maydell@linaro.org"},"content":"On 13 September 2017 at 10:57, Lluís Vilanova wrote:\n> Signed-off-by: Lluís Vilanova \n> ---\n> MAINTAINERS | 6 ++\n> docs/instrument.txt | 173 +++++++++++++++++++++++++++++++++++++++++++++++++++\n> 2 files changed, 179 insertions(+)\n> create mode 100644 docs/instrument.txt\n>\n> diff --git a/MAINTAINERS b/MAINTAINERS\n> index 36eeb42d19..fb0eaee06a 100644\n> --- a/MAINTAINERS\n> +++ b/MAINTAINERS\n> @@ -1486,6 +1486,12 @@ F: scripts/tracetool/\n> F: docs/tracing.txt\n> T: git git://github.com/stefanha/qemu.git tracing\n>\n> +Event instrumentation\n> +M: Lluís Vilanova \n> +M: Stefan Hajnoczi \n> +S: Maintained\n> +F: docs/instrument.txt\n> +\n> TPM\n> S: Orphan\n> F: tpm.c\n> diff --git a/docs/instrument.txt b/docs/instrument.txt\n> new file mode 100644\n> index 0000000000..24a0d21fc7\n> --- /dev/null\n> +++ b/docs/instrument.txt\n> @@ -0,0 +1,173 @@\n> += Event instrumentation =\n> +\n> +== Introduction ==\n> +\n> +Event instrumentation allows users to execute their own host-native code on a\n> +set of pre-defined events provided by QEMU. QEMU also exposes other\n> +functionality to peek/poke at the guest state (e.g., memory or registers), as\n> +well as interacting with tracing events. For those familiar with the term, this\n> +provides dynamic binary instrumentation, works on all QEMU-supported\n> +architectures, as well as works in both 'user' (standalone application) and\n> +'system' (full-system emulation) modes.\n> +\n> +Look at the headers installed by QEMU on the \"qemu-instr\" directory for further\n> +information beyond this document.\n> +\n> +\n> +== Loading an instrumentation library ==\n> +\n> +Instrumentation code can be bundled into a dynamic library, which can be later\n> +loaded into QEMU:\n> +\n> +* Using the command-line \"-instr\" argument.\n> +\n> +* Using the \"instr-load\" and \"instr-unload\" commands in the HMP and QMP\n> + interfaces.\n> +\n> +\n> +== Example ==\n> +\n> +1. Configure QEMU with event instrumentation:\n> +\n> + # instrument guest_cpu_enter and guest_mem_before\n> + mkdir -p /path/to/qemu-build\n> + cd /path/to/qemu-build\n> + /path/to/qemu-source/configure \\\n> + --enable-instrument \\\n> + --prefix=/path/to/qemu-install\n\nIdeally instrumentation should be cost-free in the case where\nwe're not using it, so we can default it to enabled.\n\n> +\n> +2. Build and install QEMU:\n> +\n> + make install\n> +\n> +3. Create the \"Makefile\" to build the instrumentation library:\n> +\n> + mkdir -p /tmp/my-instrument\n> +\n> + cat > /tmp/my-instrument/Makefile < + QEMU_PATH=/tmp/qemu-install/\n> +\n> + CFLAGS += -g\n> + CFLAGS += -O3\n> + CFLAGS += -Werror -Wall\n> + CFLAGS += -I$(QEMU_PATH)/include\n\nPlugins shouldn't have or need access to all of the QEMU source\ntree or its include files. We want to be able to provide them\nwith one header file which defines all they need (and all they\nget), under a suitably non-restrictive license like 2-clause-BSD.\n\n> +\n> + all: libtrace-instrument.la\n> +\n> + libtrace-instrument.la: instrument.lo\n> + libtool --mode=link --tag=CC $(CC) -module -rpath /usr/local/lib -o $@ $^\n\n-rpath ?\n\n> +\n> + %.lo: %.c\n> + libtool --mode=compile --tag=CC $(CC) $(CFLAGS) -c $^\n> +\n> + clean:\n> + $(RM) -f *.o *.so *.lo\n> + $(RM) -Rf .libs\n> + EOF\n> +\n> +4. Write your instrumentation library:\n> +\n> + cat > /tmp/my-instrument/instrument.c < + #include \n> + #include \n> +\n> + #include /* manipulate events */\n> + #include /* manipulate tracing */\n> +\n> + /* the address for the memory access is not known at translation time */\n> + void guest_mem_before_trans(QICPU vcpu_trans, QITCGv_cpu vcpu_exec,\n> + QITCGv vaddr, QIMemInfo info)\n> + {\n> + printf(\"%s: %p %p %p %d %d %d %d\\n\", __func__, vcpu_trans, vcpu_exec, vaddr,\n> + 1 << info.size_shift, info.sign_extend, info.endianness, info.store);\n> + if (info.store) {\n> + /* generate at execution time only for memory writes */\n> + qi_event_gen_guest_mem_before_exec(vcpu_exec, vaddr, info);\n> + }\n> + }\n> +\n> + /* called when QEMU executes a memory access */\n> + void guest_mem_before_exec(QICPU vcpu, uint64_t vaddr, QIMemInfo info)\n> + {\n> + if (info.store) {\n> + /* if called by TCG code, we'll only get writes (see above) */\n> + printf(\"%s: %p %lx %d %d %d %d\\n\", __func__, vcpu, vaddr,\n> + 1 << info.size_shift, info.sign_extend, info.endianness, info.store);\n> + }\n> + }\n\nThis looks like it's exposing too much implementation detail.\nWe should just provide an API for \"hook to be called for\nmemory writes\" which gets all the information when it\nis called. I don't think we should expose any kind of\n\"this hook is called at translation time\" at all.\n\nI guess the API docs are in doc comments in a header somewhere,\nso I'll go look in the other patches.\n\nthanks\n-- PMM","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"cDkUbT/a\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtMHz0LRHz9sPk\n\tfor ;\n\tFri, 15 Sep 2017 01:04:15 +1000 (AEST)","from localhost ([::1]:48303 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dsVgb-0005bp-7x\n\tfor incoming@patchwork.ozlabs.org; Thu, 14 Sep 2017 11:04:13 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:47330)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dsVL7-0006jC-SP\n\tfor qemu-devel@nongnu.org; Thu, 14 Sep 2017 10:42:07 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dsVL6-0003J9-7x\n\tfor qemu-devel@nongnu.org; Thu, 14 Sep 2017 10:42:01 -0400","from mail-wr0-x236.google.com ([2a00:1450:400c:c0c::236]:53726)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1dsVL5-0003Hz-Ul\n\tfor qemu-devel@nongnu.org; Thu, 14 Sep 2017 10:42:00 -0400","by mail-wr0-x236.google.com with SMTP id l22so528547wrc.10\n\tfor ; Thu, 14 Sep 2017 07:41:59 -0700 (PDT)","by 10.223.139.215 with HTTP; Thu, 14 Sep 2017 07:41:38 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc:content-transfer-encoding;\n\tbh=n3/gm/UQvC7GTf/bKHahyojnXzLmWWSg2HHqdeA2qr0=;\n\tb=cDkUbT/a8AZngAzTdEWsOYaQiPDl2vh8vG3gFe1X/hq8oMKSH2mAgFwbB5qNnCQd4l\n\tJKXhHZAsuqn5DJKDJmM9P3wnfu7ncvFs/l/nb7f5obaFI3yoSe8zAGTmVE38XXr3Ee8a\n\tUrOy+aIkLHZh41C6p6MO3iNR4uYt67LGW4b9E=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc:content-transfer-encoding;\n\tbh=n3/gm/UQvC7GTf/bKHahyojnXzLmWWSg2HHqdeA2qr0=;\n\tb=D8Bp++v+M4cx3eE+GoFysyX8b9gbuNEgw54PRQNa3UIOr5nIEucbHozW6UFaCkcoNQ\n\toaJ3mO3krVN/5w3S4U/qTE3VF+eNJjgBj2sTxqVkIHoMgyjzwvPNQVHQWqbcny2TtvIv\n\tMCjPv/2y9ZpioI9e/jJdBxsX54/UuAs92rlZav5fiF08PttPHARCc1y8dOGB5ilkBmQB\n\tNW0GYjwwWlvJN77X2o5cgAI+NbhifCWR+hCIUv52HgcPWmfRTR70V65gtCHx93JaCLaW\n\tAfIf+O9l6L5nyaG3RrLFdkcjqz7cgEB9+Y/zRunVVrUM+32p9W0eJsElgpzaqWKgiVGK\n\torrw==","X-Gm-Message-State":"AHPjjUj2XxqCar1amYNVnGvRuiLmUDGSA+Cor1mIYTob/9GFgTKzvwVF\n\tH+jmauFod8Vln97UPFcoaC5fUwZQaUqitAs/N4nR8w==","X-Google-Smtp-Source":"ADKCNb4z5mU+9zRFIeuXUl97uWT8MLJwsC+29YNYFHSHP8N0KPAashj3865C+jXeaNbPqY+huWsGBGwnZzaO4ycnNd8=","X-Received":"by 10.223.134.174 with SMTP id 43mr20441127wrx.173.1505400118603;\n\tThu, 14 Sep 2017 07:41:58 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<150529666493.10902.14830445134051381968.stgit@frigg.lan>","References":"<150529642278.10902.18234057937634437857.stgit@frigg.lan>\n\t<150529666493.10902.14830445134051381968.stgit@frigg.lan>","From":"Peter Maydell ","Date":"Thu, 14 Sep 2017 15:41:38 +0100","Message-ID":"","To":"=?utf-8?q?Llu=C3=ADs_Vilanova?= ","Content-Type":"text/plain; charset=\"UTF-8\"","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c0c::236","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"\"Emilio G. Cota\" ,\n\tQEMU Developers , \n\tStefan Hajnoczi ,\n\tMarkus Armbruster ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1769196,"web_url":"http://patchwork.ozlabs.org/comment/1769196/","msgid":"<87poasgjyh.fsf@frigg.lan>","list_archive_url":null,"date":"2017-09-15T13:39:02","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":9099,"url":"http://patchwork.ozlabs.org/api/people/9099/","name":"Lluís Vilanova","email":"vilanova@ac.upc.edu"},"content":"Peter Maydell writes:\n\n> On 13 September 2017 at 10:57, Lluís Vilanova wrote:\n>> Signed-off-by: Lluís Vilanova \n>> ---\n>> MAINTAINERS | 6 ++\n>> docs/instrument.txt | 173 +++++++++++++++++++++++++++++++++++++++++++++++++++\n>> 2 files changed, 179 insertions(+)\n>> create mode 100644 docs/instrument.txt\n>> \n>> diff --git a/MAINTAINERS b/MAINTAINERS\n>> index 36eeb42d19..fb0eaee06a 100644\n>> --- a/MAINTAINERS\n>> +++ b/MAINTAINERS\n>> @@ -1486,6 +1486,12 @@ F: scripts/tracetool/\n>> F: docs/tracing.txt\n>> T: git git://github.com/stefanha/qemu.git tracing\n>> \n>> +Event instrumentation\n>> +M: Lluís Vilanova \n>> +M: Stefan Hajnoczi \n>> +S: Maintained\n>> +F: docs/instrument.txt\n>> +\n>> TPM\n>> S: Orphan\n>> F: tpm.c\n>> diff --git a/docs/instrument.txt b/docs/instrument.txt\n>> new file mode 100644\n>> index 0000000000..24a0d21fc7\n>> --- /dev/null\n>> +++ b/docs/instrument.txt\n>> @@ -0,0 +1,173 @@\n>> += Event instrumentation =\n>> +\n>> +== Introduction ==\n>> +\n>> +Event instrumentation allows users to execute their own host-native code on a\n>> +set of pre-defined events provided by QEMU. QEMU also exposes other\n>> +functionality to peek/poke at the guest state (e.g., memory or registers), as\n>> +well as interacting with tracing events. For those familiar with the term, this\n>> +provides dynamic binary instrumentation, works on all QEMU-supported\n>> +architectures, as well as works in both 'user' (standalone application) and\n>> +'system' (full-system emulation) modes.\n>> +\n>> +Look at the headers installed by QEMU on the \"qemu-instr\" directory for further\n>> +information beyond this document.\n>> +\n>> +\n>> +== Loading an instrumentation library ==\n>> +\n>> +Instrumentation code can be bundled into a dynamic library, which can be later\n>> +loaded into QEMU:\n>> +\n>> +* Using the command-line \"-instr\" argument.\n>> +\n>> +* Using the \"instr-load\" and \"instr-unload\" commands in the HMP and QMP\n>> + interfaces.\n>> +\n>> +\n>> +== Example ==\n>> +\n>> +1. Configure QEMU with event instrumentation:\n>> +\n>> + # instrument guest_cpu_enter and guest_mem_before\n>> + mkdir -p /path/to/qemu-build\n>> + cd /path/to/qemu-build\n>> + /path/to/qemu-source/configure \\\n>> + --enable-instrument \\\n>> + --prefix=/path/to/qemu-install\n\n> Ideally instrumentation should be cost-free in the case where\n> we're not using it, so we can default it to enabled.\n\nI wasn't sure if everyone would want it enabled by default on the first release,\nbut I can easily change that.\n\n\n>> +\n>> +2. Build and install QEMU:\n>> +\n>> + make install\n>> +\n>> +3. Create the \"Makefile\" to build the instrumentation library:\n>> +\n>> + mkdir -p /tmp/my-instrument\n>> +\n>> + cat > /tmp/my-instrument/Makefile <> + QEMU_PATH=/tmp/qemu-install/\n>> +\n>> + CFLAGS += -g\n>> + CFLAGS += -O3\n>> + CFLAGS += -Werror -Wall\n>> + CFLAGS += -I$(QEMU_PATH)/include\n\n> Plugins shouldn't have or need access to all of the QEMU source\n> tree or its include files. We want to be able to provide them\n> with one header file which defines all they need (and all they\n> get), under a suitably non-restrictive license like 2-clause-BSD.\n\nVariable QEMU_PATH refers to the *installation* path of QEMU.\n\nI can change the API headers to use some other license.\n\n\n>> +\n>> + all: libtrace-instrument.la\n>> +\n>> + libtrace-instrument.la: instrument.lo\n>> + libtool --mode=link --tag=CC $(CC) -module -rpath /usr/local/lib -o $@ $^\n\n> -rpath ?\n\nI couldn't make libtool to generate a .so file without it. I can change the\nexample to directly use gcc instead of libtool.\n\n\n>> +\n>> + %.lo: %.c\n>> + libtool --mode=compile --tag=CC $(CC) $(CFLAGS) -c $^\n>> +\n>> + clean:\n>> + $(RM) -f *.o *.so *.lo\n>> + $(RM) -Rf .libs\n>> + EOF\n>> +\n>> +4. Write your instrumentation library:\n>> +\n>> + cat > /tmp/my-instrument/instrument.c <> + #include \n>> + #include \n>> +\n>> + #include /* manipulate events */\n>> + #include /* manipulate tracing */\n>> +\n>> + /* the address for the memory access is not known at translation time */\n>> + void guest_mem_before_trans(QICPU vcpu_trans, QITCGv_cpu vcpu_exec,\n>> + QITCGv vaddr, QIMemInfo info)\n>> + {\n>> + printf(\"%s: %p %p %p %d %d %d %d\\n\", __func__, vcpu_trans, vcpu_exec, vaddr,\n>> + 1 << info.size_shift, info.sign_extend, info.endianness, info.store);\n>> + if (info.store) {\n>> + /* generate at execution time only for memory writes */\n>> + qi_event_gen_guest_mem_before_exec(vcpu_exec, vaddr, info);\n>> + }\n>> + }\n>> +\n>> + /* called when QEMU executes a memory access */\n>> + void guest_mem_before_exec(QICPU vcpu, uint64_t vaddr, QIMemInfo info)\n>> + {\n>> + if (info.store) {\n>> + /* if called by TCG code, we'll only get writes (see above) */\n>> + printf(\"%s: %p %lx %d %d %d %d\\n\", __func__, vcpu, vaddr,\n>> + 1 << info.size_shift, info.sign_extend, info.endianness, info.store);\n>> + }\n>> + }\n\n> This looks like it's exposing too much implementation detail.\n> We should just provide an API for \"hook to be called for\n> memory writes\" which gets all the information when it\n> is called. I don't think we should expose any kind of\n> \"this hook is called at translation time\" at all.\n\nThe differentiation between translation-time and execution-time is key to\nperform certain analysis efficiently.\n\nThe simplest example is probably a BBL trace (e.g., as used by SimPoint\n[1]). For each BBL (or, rather, TB) that you translate, you collect what\ninstructions are in it (addresses, length, opcodes, etc). Then, at execution\ntime you only need to collect the starting address of each executed BBL/TB,\nsince its information was already collected before at translation time\n(therefore making it more efficient).\n\n[1] https://cseweb.ucsd.edu/~calder/simpoint/\n\n\n> I guess the API docs are in doc comments in a header somewhere,\n> so I'll go look in the other patches.\n\nYes, you can look at headers in instrument/qemu-instr/.\n\n\nThanks,\n Lluis","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtxMz4pfBz9sRm\n\tfor ;\n\tFri, 15 Sep 2017 23:39:43 +1000 (AEST)","from localhost ([::1]:53432 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dsqqL-0000dI-Qb\n\tfor incoming@patchwork.ozlabs.org; Fri, 15 Sep 2017 09:39:41 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:48466)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dsqq2-0000d3-VQ\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 09:39:24 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dsqpz-0003z8-RB\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 09:39:23 -0400","from roura.ac.upc.edu ([147.83.33.10]:33411 helo=roura.ac.upc.es)\n\tby eggs.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dsqpz-0003tW-C6\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 09:39:19 -0400","from correu-1.ac.upc.es (correu-1.ac.upc.es [147.83.30.91])\n\tby roura.ac.upc.es (8.13.8/8.13.8) with ESMTP id v8FDd9Ok018149;\n\tFri, 15 Sep 2017 15:39:09 +0200","from localhost (unknown [132.68.50.201])\n\tby correu-1.ac.upc.es (Postfix) with ESMTPSA id 21DC31E7;\n\tFri, 15 Sep 2017 15:39:04 +0200 (CEST)"],"From":"=?utf-8?q?Llu=C3=ADs_Vilanova?= ","To":"Peter Maydell ","References":"<150529642278.10902.18234057937634437857.stgit@frigg.lan>\n\t<150529666493.10902.14830445134051381968.stgit@frigg.lan>\n\t","Mail-Followup-To":"Peter Maydell , \"Emilio G. Cota\"\n\t, QEMU Developers ,\n\tStefan\n\tHajnoczi , Markus Armbruster ","Date":"Fri, 15 Sep 2017 16:39:02 +0300","In-Reply-To":"\n\t(Peter Maydell's message of \"Thu, 14 Sep 2017 15:41:38 +0100\")","Message-ID":"<87poasgjyh.fsf@frigg.lan>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux)","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.6.x [fuzzy]","X-Received-From":"147.83.33.10","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"\"Emilio G. Cota\" ,\n\tQEMU Developers , \n\tStefan Hajnoczi ,\n\tMarkus Armbruster ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770206,"web_url":"http://patchwork.ozlabs.org/comment/1770206/","msgid":"<20170918143308.GA4141@stefanha-x1.localdomain>","list_archive_url":null,"date":"2017-09-18T14:33:08","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":2747,"url":"http://patchwork.ozlabs.org/api/people/2747/","name":"Stefan Hajnoczi","email":"stefanha@gmail.com"},"content":"On Wed, Sep 13, 2017 at 12:57:45PM +0300, Lluís Vilanova wrote:\n> + /* mandatory initialization function */\n> + int main(int argc, const char **argv)\n\nMost shared library plugin interfaces I have seen do not use \"main()\" as\nthe entry point. Instead they use a unique name that allows the host\napplication to differentiate between share library files that are valid\nplugins (e.g. \"qemu_instr_init\") and non-plugin shared libraries.\n\nStable plugin APIs usually have a versioning or feature detection\nscheme. Versioning is simple: the host application passes a major/minor\nversion number to the init function.\n\nOf course the dynamic linker already enforces compatibility somewhat: if\na plugin uses a newer API than available in the host application then\nthere will be a missing symbol/linker error.\n\nSo what versioning strategy should we follow? The simplest would be to\ndepend 100% on the dynamic linker with no explicit checks inside the\nplugin or QEMU. In that case the API/ABI need to follow some rules like\nthis (not sure how oudated this information is):\nhttp://plan99.net/~mike/writing-shared-libraries.html\n\n> + {\n> + int i;\n> + printf(\"init!\\n\");\n> + printf(\" argc :: %d\\n\", argc);\n> + for (i = 0; i < argc; i++) {\n> + printf(\" -> %s\\n\", argv[i]);\n> + }\n> + \n> + qi_set_fini(fini, NULL);\n> + \n> + /* instrument and trace events */\n> + QITraceEvent *ev;\n> + \n> + qi_event_set_guest_cpu_enter(guest_cpu_enter);\n> + ev = qi_trace_event_name(\"guest_cpu_enter\");\n> + assert(ev);\n> + qi_trace_event_set_state_dynamic(ev, true);\n> + \n> + qi_event_set_guest_mem_before_trans(guest_mem_before_trans);\n> + ev = qi_trace_event_name(\"guest_mem_before_trans\");\n> + assert(ev);\n> + qi_trace_event_set_state_dynamic(ev, true);\n> + \n> + qi_event_set_guest_mem_before_exec(guest_mem_before_exec);\n> + ev = qi_trace_event_name(\"guest_mem_before_exec\");\n> + assert(ev);\n> + qi_trace_event_set_state_dynamic(ev, true);\n\nWhy are trace events being enabled in this example?\n\nI would expect qi_event_set_guest_cpu_enter(guest_cpu_enter) to\nimmediately enable the callback. The user shouldn't need to use tracing\nto receive callbacks.\n\nqi_event_set_guest_cpu_enter(NULL) should disable the callback.\n\nStefan","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"nVYILFuU\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xwpRP2RGTz9s7F\n\tfor ;\n\tTue, 19 Sep 2017 00:34:08 +1000 (AEST)","from localhost ([::1]:37000 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dtx7e-0001w8-EQ\n\tfor incoming@patchwork.ozlabs.org; Mon, 18 Sep 2017 10:34:06 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:53238)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dtx6t-0001pY-H1\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 10:33:20 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dtx6n-0002d0-Qc\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 10:33:19 -0400","from mail-wr0-x22e.google.com ([2a00:1450:400c:c0c::22e]:50638)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from ) id 1dtx6n-0002cc-KD\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 10:33:13 -0400","by mail-wr0-x22e.google.com with SMTP id w12so662293wrc.7\n\tfor ; Mon, 18 Sep 2017 07:33:13 -0700 (PDT)","from localhost ([51.15.41.238]) by smtp.gmail.com with ESMTPSA id\n\tm64sm4370031wmb.0.2017.09.18.07.33.11\n\t(version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);\n\tMon, 18 Sep 2017 07:33:11 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:content-transfer-encoding:in-reply-to\n\t:user-agent; bh=ICmN9aV3euZPLbs5odMAU1cxSVEg7uE0VQVwd9HG9i0=;\n\tb=nVYILFuUAfXKH2M142Bh880Qk6iO7Gm+oeZIjqwwgb6m4rEw80+tRVN8Wh5Bn28sag\n\tn3IUTTY4tDFM6qoSdr+NUD3G3LAN1j/QhGpqnwUob765K9N0nJZGPVfjjwth/p1Bny7N\n\tSwCiH/fol6x07TmZWz74f1jzVpY3HS9bT3PeS0HiMB7E75wt160gYGEJ5D8A2xF8xm1O\n\tc1dCZCkd/FCt16eH+Rrq/o9sZmvwYJm70eN4qR9LCKm6wKt/AcVErVIsHRu+x32LDfaU\n\toEJ5XSJtgbfCO5OaM7A1waS5wfKC0A8NzKLlVcS5SI6lqLyM3y6t2xeXUuGgqeJp5hsI\n\tbQZg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:content-transfer-encoding\n\t:in-reply-to:user-agent;\n\tbh=ICmN9aV3euZPLbs5odMAU1cxSVEg7uE0VQVwd9HG9i0=;\n\tb=fFnWquTNvZc7FzqIrExb/JO50WnD42i6igsLYa8DVy5gmD0wvhbeDfTS2GXypjexWz\n\tQhZSeGmcdyJrh9hntYYLDOSApfODLlfxoQWISdj+tIl+RJYuVytnd3I/6EILfktvgLY8\n\t4oPCH5a+aiQTD9rPhckLo5uDyrJxFaQhPUtTN514634W6icVVOG1WqYqlgA40kMNOCTk\n\tYeiDlfLHgxoKHKgFtvzDKSwkwmnIB5CFGTuUqvkrPNpO7K/4clZLaauoIlRVVC2otNNa\n\tjUYYbzfHL0C+jEylDfX5cEB8q7A4ymWiWcBB8IRNOQSZoBZnleTqcxRC49buyjP4pNeN\n\tYOBQ==","X-Gm-Message-State":"AHPjjUj5kKV0SEPz+tw3LJUW5YtguhCX2kxrfHBEFxvRlzO9pqft2haE\n\t+dVUCWPSMa/bjA==","X-Google-Smtp-Source":"ADKCNb73E1oRRzBCUMAVRHMKSGLxMUvHzNom6VRB/8bWb+VHt1pI0Ttp4c9sJmHS0r0cmC2jb2m7WA==","X-Received":"by 10.223.177.211 with SMTP id r19mr30067633wra.2.1505745192418; \n\tMon, 18 Sep 2017 07:33:12 -0700 (PDT)","Date":"Mon, 18 Sep 2017 15:33:08 +0100","From":"Stefan Hajnoczi ","To":"=?iso-8859-1?q?Llu=EDs?= Vilanova ","Message-ID":"<20170918143308.GA4141@stefanha-x1.localdomain>","References":"<150529642278.10902.18234057937634437857.stgit@frigg.lan>\n\t<150529666493.10902.14830445134051381968.stgit@frigg.lan>","MIME-Version":"1.0","Content-Type":"text/plain; charset=iso-8859-1","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<150529666493.10902.14830445134051381968.stgit@frigg.lan>","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c0c::22e","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"\"Emilio G. Cota\" , qemu-devel@nongnu.org,\n\tStefan Hajnoczi ,\n\tMarkus Armbruster ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770209,"web_url":"http://patchwork.ozlabs.org/comment/1770209/","msgid":"<20170918144021.GC4141@stefanha-x1.localdomain>","list_archive_url":null,"date":"2017-09-18T14:40:21","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":2747,"url":"http://patchwork.ozlabs.org/api/people/2747/","name":"Stefan Hajnoczi","email":"stefanha@gmail.com"},"content":"On Wed, Sep 13, 2017 at 12:57:45PM +0300, Lluís Vilanova wrote:\n> +Event instrumentation\n> +M: Lluís Vilanova \n> +M: Stefan Hajnoczi \n> +S: Maintained\n> +F: docs/instrument.txt\n\nThanks for including me but I'm afraid I don't have time to co-maintain\nthe instrumentation API.\n\nYou could maintain it yourself. Maintainers send signed pull requests\nto Peter Maydell to get commits merged into qemu.git/master.\n\nMaybe someone else would like to co-maintain it with you?\n\nStefan","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"g27FQqFW\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xwpcB2NyPz9s7F\n\tfor ;\n\tTue, 19 Sep 2017 00:41:46 +1000 (AEST)","from localhost ([::1]:37038 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dtxF2-00072J-Da\n\tfor incoming@patchwork.ozlabs.org; Mon, 18 Sep 2017 10:41:44 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:56360)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dtxDo-0006bm-Mv\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 10:40:32 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dtxDk-00068y-Mj\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 10:40:28 -0400","from mail-wr0-x235.google.com ([2a00:1450:400c:c0c::235]:54859)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from ) id 1dtxDk-00068c-GN\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 10:40:24 -0400","by mail-wr0-x235.google.com with SMTP id g29so676481wrg.11\n\tfor ; Mon, 18 Sep 2017 07:40:24 -0700 (PDT)","from localhost ([51.15.41.238]) by smtp.gmail.com with ESMTPSA id\n\tq188sm7944832wmb.43.2017.09.18.07.40.22\n\t(version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);\n\tMon, 18 Sep 2017 07:40:22 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:content-transfer-encoding:in-reply-to\n\t:user-agent; bh=Liar5kWqCVW6XQNuXIv/1rAQmQ2bnxqZoJMR0UqpHJM=;\n\tb=g27FQqFW8+v3hy2G1xHA0oH+oYot4kJ56vRQR4P93n6st3jpQEfzf9SBzlHEL3V0aJ\n\tTK6Hza8ep3feAfuq+mLDEhW4nI06t3QuDs0FTWsy8ZCs4RlL/CMw4NR3CD8vSNu8Wxzp\n\tnCcCJDffVr7c8x45vPmooX7jfX1dQhvX+cG21IGJkW9h4xAD2IpDVVub3h6F6HjsNLaC\n\t2/pfHtF+/nPzkueaPjWqhWXgslGTBVhtrvxATnpJkNxmN8wuCqtCJW6njK/x/U4oFRl7\n\tdr+lzpavLpRtVXyrocPN97NAYHkSBGs7cQdqbibCx42h/6N+1ef0phuXYSJnP4nfi3cg\n\tzu7Q==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:content-transfer-encoding\n\t:in-reply-to:user-agent;\n\tbh=Liar5kWqCVW6XQNuXIv/1rAQmQ2bnxqZoJMR0UqpHJM=;\n\tb=LNaGVuYHBRaJF3TpZmn2WkfVPKHejdhUvjfc6zlktVXsXEb/XGz5pPYEiOOIZCvJ47\n\tBlZDvkR7+123DK17i8iQQehYt4K1sI4zoDSAMpQcXPbpIc6QlQlscgTyNvbhYxJhJcTJ\n\tv9AXoa7QqHcWdYnoXwfPtZR9+PwJzLMlGiassLzqq9eUMqiu7u8IdujyChQ02JYGpK1Y\n\t5kgyy9kNOj/pYOsZrxuQXMRUg2frA+Scx05G5SIl202Ymz1v81gjgsCYrtay44bUv5WA\n\t+7XSMmyV0p97a496KAna48VmdnIBEdYbUwgJyMhYkPCl+V1CYW/84TogPtgX1meo9F26\n\tt2eQ==","X-Gm-Message-State":"AHPjjUjISXaD2L+xa4I4rccPzXk8bCTgjdXdDFjxxjCeaCRvzNywF+gU\n\tiTDE+eHCltbMTA==","X-Google-Smtp-Source":"ADKCNb5s5VkyjnkQWemHNR9pJyfUrFy2edsTQ8KNMI2/LV+UZpyTcm2mnZw/Mc4OEEdjIh2frml4ag==","X-Received":"by 10.223.185.5 with SMTP id k5mr24378005wrf.189.1505745623411; \n\tMon, 18 Sep 2017 07:40:23 -0700 (PDT)","Date":"Mon, 18 Sep 2017 15:40:21 +0100","From":"Stefan Hajnoczi ","To":"=?iso-8859-1?q?Llu=EDs?= Vilanova ","Message-ID":"<20170918144021.GC4141@stefanha-x1.localdomain>","References":"<150529642278.10902.18234057937634437857.stgit@frigg.lan>\n\t<150529666493.10902.14830445134051381968.stgit@frigg.lan>","MIME-Version":"1.0","Content-Type":"text/plain; charset=iso-8859-1","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<150529666493.10902.14830445134051381968.stgit@frigg.lan>","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c0c::235","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"\"Emilio G. Cota\" , qemu-devel@nongnu.org,\n\tStefan Hajnoczi ,\n\tMarkus Armbruster ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770212,"web_url":"http://patchwork.ozlabs.org/comment/1770212/","msgid":"","list_archive_url":null,"date":"2017-09-18T14:41:38","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":5111,"url":"http://patchwork.ozlabs.org/api/people/5111/","name":"Peter Maydell","email":"peter.maydell@linaro.org"},"content":"On 15 September 2017 at 14:39, Lluís Vilanova wrote:\n> Peter Maydell writes:\n>> This looks like it's exposing too much implementation detail.\n>> We should just provide an API for \"hook to be called for\n>> memory writes\" which gets all the information when it\n>> is called. I don't think we should expose any kind of\n>> \"this hook is called at translation time\" at all.\n>\n> The differentiation between translation-time and execution-time is key to\n> perform certain analysis efficiently.\n\nIt's also exposing internal QEMU implementation detail.\nWhat if in future we decide to switch from our current\nsetup to always interpreting guest instructions as a\nfirst pass with JITting done only in the background for\nhot code?\n\nSticking to instrumentation events that correspond exactly to guest\nexecution events means they won't break or expose internals.\n\nthanks\n-- PMM","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org; dkim=pass (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"j9+rWn01\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xwphC6SqWz9s7F\n\tfor ;\n\tTue, 19 Sep 2017 00:45:15 +1000 (AEST)","from localhost ([::1]:37056 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dtxIP-0001q5-QQ\n\tfor incoming@patchwork.ozlabs.org; Mon, 18 Sep 2017 10:45:13 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:56985)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dtxFJ-0007sy-8m\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 10:42:07 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dtxFI-0006oK-Ay\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 10:42:01 -0400","from mail-wr0-x231.google.com ([2a00:1450:400c:c0c::231]:52262)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1dtxFI-0006mz-55\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 10:42:00 -0400","by mail-wr0-x231.google.com with SMTP id c23so679005wrg.9\n\tfor ; Mon, 18 Sep 2017 07:42:00 -0700 (PDT)","by 10.223.139.215 with HTTP; Mon, 18 Sep 2017 07:41:38 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:content-transfer-encoding;\n\tbh=JmUbAi3ZgJaU1etj7upqKy4VGgxhdiA5GHPlhbFZX2M=;\n\tb=j9+rWn01QxY7SAfbLlahqHPuhl3WgC0oOVl1d4KzUyfbzSSQySXlaMjBB1nGjqrzBb\n\t6i0SNgd6bMX7NixKJm0q2C2xd5E8Tq9eD6h8aGq0ZlT5rB11HfVy01h3X7Eu4MIjLFKe\n\ttxCoPBgBYJadqLwBaw8JwT5Cmjr8scQtQfJxs=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:content-transfer-encoding;\n\tbh=JmUbAi3ZgJaU1etj7upqKy4VGgxhdiA5GHPlhbFZX2M=;\n\tb=oiio3UsPIwIlZYsZk/BENzAwyTMNjE93yMn2M4oX3Xnmc6Dk1d2pDvLejUNXXSA2v3\n\t7D3vuq1GP6r7P78I/AOSWVLTZMkeLeyvGiw4v+78DoQVoc+EhkKWpbOjQVJmuLnZ0gwP\n\twjopcOo/Y9Xvkzsu1Zuvf6MB8YIAzQiRrysuWWMWnTnLNHEqza5zmrmj88DeeN/j9yIo\n\tmrUAPPB5pZW4q0sDFr4yYZg2hwkKEGVFHmzIMmgtzoyuI2BKX5i5bCGvflxTJLzP61Aw\n\tYqVqVLlsjBciixn8bvFIl38TmvyTvHAbu3N1eL5E0GqQJWggFPAXMP4vCe+u6mJzox2P\n\tghwA==","X-Gm-Message-State":"AHPjjUjL+16wXF6NBs9SinV1CKarmEoaJuDLRh2a2J0+K/MbjkzddmOe\n\t02HPLYSxrcb327jsSWP32hbFVPDni3qHyq4CDyuRuQ==","X-Google-Smtp-Source":"ADKCNb5TiqQHsTvTJXmCc0ehQNpdhJpGbbFMNs155bIi9AoV3XtN7ncfJ7SDHgol+MntNbs4jAI92XdrYNRUkIi3Gbc=","X-Received":"by 10.223.175.100 with SMTP id\n\tz91mr30756847wrc.177.1505745718981; \n\tMon, 18 Sep 2017 07:41:58 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<87poasgjyh.fsf@frigg.lan>","References":"<150529642278.10902.18234057937634437857.stgit@frigg.lan>\n\t<150529666493.10902.14830445134051381968.stgit@frigg.lan>\n\t\n\t<87poasgjyh.fsf@frigg.lan>","From":"Peter Maydell ","Date":"Mon, 18 Sep 2017 15:41:38 +0100","Message-ID":"","To":"Peter Maydell ,\n\t\"Emilio G. Cota\" , \n\tQEMU Developers ,\n\tStefan Hajnoczi , \n\tMarkus Armbruster ","Content-Type":"text/plain; charset=\"UTF-8\"","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c0c::231","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770334,"web_url":"http://patchwork.ozlabs.org/comment/1770334/","msgid":"<87d16o53xr.fsf@frigg.lan>","list_archive_url":null,"date":"2017-09-18T17:09:36","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":9099,"url":"http://patchwork.ozlabs.org/api/people/9099/","name":"Lluís Vilanova","email":"vilanova@ac.upc.edu"},"content":"Peter Maydell writes:\n\n> On 15 September 2017 at 14:39, Lluís Vilanova wrote:\n>> Peter Maydell writes:\n>>> This looks like it's exposing too much implementation detail.\n>>> We should just provide an API for \"hook to be called for\n>>> memory writes\" which gets all the information when it\n>>> is called. I don't think we should expose any kind of\n>>> \"this hook is called at translation time\" at all.\n>> \n>> The differentiation between translation-time and execution-time is key to\n>> perform certain analysis efficiently.\n\n> It's also exposing internal QEMU implementation detail.\n> What if in future we decide to switch from our current\n> setup to always interpreting guest instructions as a\n> first pass with JITting done only in the background for\n> hot code?\n\nTCI still has a separation of translation-time (translate.c) and execution-time\n(interpreting the TCG opcodes), and I don't think that's gonna go away anytime\nsoon.\n\nEven if it did, I think there still will be a translation/execution separation\neasy enough to hook into (even if it's a \"fake\" one for the cold-path\ninterpreted instructions).\n\n\n> Sticking to instrumentation events that correspond exactly to guest\n> execution events means they won't break or expose internals.\n\nIt also means we won't be able to \"conditionally\" instrument instructions (e.g.,\nbased on their opcode, address range, etc.).\n\nOf course we can add the translation/execution differentiation later if we find\nit necessary for performance, but I would rather avoid leaving \"historical\"\ninstrumentation points behind on the API.\n\nWhat are the use-cases you're aiming for?\n\n\nCheers!\n Lluis","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xwsvz6pX2z9s7M\n\tfor ;\n\tTue, 19 Sep 2017 03:10:37 +1000 (AEST)","from localhost ([::1]:37956 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dtzZ5-0007KX-BW\n\tfor incoming@patchwork.ozlabs.org; Mon, 18 Sep 2017 13:10:35 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:35632)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dtzYY-0007Dw-83\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 13:10:08 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dtzYT-0003EZ-70\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 13:10:02 -0400","from roura.ac.upc.es ([147.83.33.10]:48616)\n\tby eggs.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dtzYS-0003Bw-RM\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 13:09:57 -0400","from correu-2.ac.upc.es (correu-2.ac.upc.es [147.83.30.92])\n\tby roura.ac.upc.es (8.13.8/8.13.8) with ESMTP id v8IH9hPi005947;\n\tMon, 18 Sep 2017 19:09:43 +0200","from localhost (unknown [31.210.187.58])\n\tby correu-2.ac.upc.es (Postfix) with ESMTPSA id 1E73221D;\n\tMon, 18 Sep 2017 19:09:38 +0200 (CEST)"],"From":"=?utf-8?q?Llu=C3=ADs_Vilanova?= ","To":"Peter Maydell ","References":"<150529642278.10902.18234057937634437857.stgit@frigg.lan>\n\t<150529666493.10902.14830445134051381968.stgit@frigg.lan>\n\t\n\t<87poasgjyh.fsf@frigg.lan>\n\t","Mail-Followup-To":"Peter Maydell , \"Emilio G. Cota\"\n\t, QEMU Developers ,\n\tStefan\n\tHajnoczi , Markus Armbruster ","Date":"Mon, 18 Sep 2017 20:09:36 +0300","In-Reply-To":"\n\t(Peter Maydell's message of \"Mon, 18 Sep 2017 15:41:38 +0100\")","Message-ID":"<87d16o53xr.fsf@frigg.lan>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux)","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.6.x [fuzzy]","X-Received-From":"147.83.33.10","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"\"Emilio G. Cota\" ,\n\tQEMU Developers , \n\tStefan Hajnoczi ,\n\tMarkus Armbruster ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770363,"web_url":"http://patchwork.ozlabs.org/comment/1770363/","msgid":"","list_archive_url":null,"date":"2017-09-18T17:42:55","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":5111,"url":"http://patchwork.ozlabs.org/api/people/5111/","name":"Peter Maydell","email":"peter.maydell@linaro.org"},"content":"On 18 September 2017 at 18:09, Lluís Vilanova wrote:\n> Peter Maydell writes:\n>> It's also exposing internal QEMU implementation detail.\n>> What if in future we decide to switch from our current\n>> setup to always interpreting guest instructions as a\n>> first pass with JITting done only in the background for\n>> hot code?\n>\n> TCI still has a separation of translation-time (translate.c) and execution-time\n> (interpreting the TCG opcodes), and I don't think that's gonna go away anytime\n> soon.\n\nI didn't mean TCI, which is nothing like what you'd use for\nthis if you did it (TCI is slower than just JITting.)\n\n> Even if it did, I think there still will be a translation/execution separation\n> easy enough to hook into (even if it's a \"fake\" one for the cold-path\n> interpreted instructions).\n\nBut what would it mean? You don't have basic blocks any more.\n\n>> Sticking to instrumentation events that correspond exactly to guest\n>> execution events means they won't break or expose internals.\n>\n> It also means we won't be able to \"conditionally\" instrument instructions (e.g.,\n> based on their opcode, address range, etc.).\n\nYou can still do that, it's just less efficient (your\ncondition-check happens in the callout to the instrumentation\nplugin). We can add \"filter\" options later if we need them\n(which I would rather do than have translate-time callbacks).\n\n> Of course we can add the translation/execution differentiation later if we find\n> it necessary for performance, but I would rather avoid leaving \"historical\"\n> instrumentation points behind on the API.\n>\n> What are the use-cases you're aiming for?\n\n* I want to be able to point the small stream of people who come\ninto qemu-devel asking \"how do I trace all my guest's memory\naccesses\" at a clean API for it.\n\n* I want to be able to have less ugly and confusing tracing\nthan our current -d output (and perhaps emit tracing in formats\nthat other analysis tools want as input)\n\n* I want to keep this initial tracing API simple enough that\nwe can agree on it and get a first working useful version.\n\nthanks\n-- PMM","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org; dkim=pass (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"RzbB22RS\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xwtf56G48z9s7c\n\tfor ;\n\tTue, 19 Sep 2017 03:43:40 +1000 (AEST)","from localhost ([::1]:38050 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1du054-0003Jo-HD\n\tfor incoming@patchwork.ozlabs.org; Mon, 18 Sep 2017 13:43:38 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:47738)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1du04l-0003JT-Ts\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 13:43:21 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1du04k-0003cA-Sc\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 13:43:20 -0400","from mail-wm0-x235.google.com ([2a00:1450:400c:c09::235]:46488)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1du04k-0003ZN-LB\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 13:43:18 -0400","by mail-wm0-x235.google.com with SMTP id i189so4827944wmf.1\n\tfor ; Mon, 18 Sep 2017 10:43:17 -0700 (PDT)","by 10.223.139.215 with HTTP; Mon, 18 Sep 2017 10:42:55 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:content-transfer-encoding;\n\tbh=b6MGfBtUAxgL8aUW0JzcrDZ7RucKsOR9Dm+GsQj/5FM=;\n\tb=RzbB22RSLnDomPmFC98jyJGQBHutkOJe7Wuf5wcRpuVEBxRdmuCGakLn3V3xXLlJR9\n\t/mD7IReP2BcXc/SSeo5vkzCozU9+oPmkWzLvjsovZ/MuxW/VmyNczajawWYNf29X1ahY\n\tv1STcIiJssTVl7xTNBsg/OAAXwgLhWBu0m8r8=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:content-transfer-encoding;\n\tbh=b6MGfBtUAxgL8aUW0JzcrDZ7RucKsOR9Dm+GsQj/5FM=;\n\tb=SFWwl0RFVZQ5AoSqbzf3fA2CX8ogCFyE5ndXdqCv9c5uwZMg4XWt9pAYghZjUS9SnK\n\tqnL2zppXSL84+cLduRGpjjzYdJ594+Dx+5ZjcDtzQ4UwAJKWf+F+1N850jZErGlOPpwp\n\tv+7YOIIwJhRpO8Z+hnu7HxzpVU4jeQNf6s5vr3L+JMK53YtlaXr9VU9z1Mi7OFa3vyK0\n\tnvJTbC5DfFhShRPznucku0krQOUx3EweMnsztTkej967BLx0v7a2n36Upj0o00l2Wq9R\n\tx3xUTbQdi5IqBO5Ev5g9TK3lMpMaYjFZAdei+Iy3qLqo2cdGYAUlQcJEjU28OfmIykzU\n\tO5ng==","X-Gm-Message-State":"AHPjjUiGzEzNhwU/n54ha+erRDOifquf+da2dj9Ms4+pPlXmb1uQ8Q2V\n\tZ8cHtBYBVkEVeWyemqGXfeILb+3cYDtrhMI5DR/HTA==","X-Google-Smtp-Source":"AOwi7QCTwBYSvKuSP+oA+5QQvgMltmaXaVekx24VI0P+Ll4JXrvyCC5gF2xPnUZAHuE3wRytGbLGPEO+ZaB3iGQZQI4=","X-Received":"by 10.28.52.81 with SMTP id b78mr9557768wma.11.1505756595933;\n\tMon, 18 Sep 2017 10:43:15 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<87d16o53xr.fsf@frigg.lan>","References":"<150529642278.10902.18234057937634437857.stgit@frigg.lan>\n\t<150529666493.10902.14830445134051381968.stgit@frigg.lan>\n\t\n\t<87poasgjyh.fsf@frigg.lan>\n\t\n\t<87d16o53xr.fsf@frigg.lan>","From":"Peter Maydell ","Date":"Mon, 18 Sep 2017 18:42:55 +0100","Message-ID":"","To":"Peter Maydell ,\n\t\"Emilio G. Cota\" , \n\tQEMU Developers ,\n\tStefan Hajnoczi , \n\tMarkus Armbruster ","Content-Type":"text/plain; charset=\"UTF-8\"","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c09::235","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1771028,"web_url":"http://patchwork.ozlabs.org/comment/1771028/","msgid":"","list_archive_url":null,"date":"2017-09-19T13:09:07","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":5111,"url":"http://patchwork.ozlabs.org/api/people/5111/","name":"Peter Maydell","email":"peter.maydell@linaro.org"},"content":"On 18 September 2017 at 18:09, Lluís Vilanova wrote:\n> TCI still has a separation of translation-time (translate.c) and execution-time\n> (interpreting the TCG opcodes), and I don't think that's gonna go away anytime\n> soon.\n>\n> Even if it did, I think there still will be a translation/execution separation\n> easy enough to hook into (even if it's a \"fake\" one for the cold-path\n> interpreted instructions).\n\nAs a slightly more immediate and practical example, I'm currently\nimplementing the v8M \"SG\" instruction. This is a somewhat weird\ncorner-case of an instruction (it's the only instruction you can\nexecute in non-secure state from a code region that's secure).\nI'm implementing it in the exception-handling code path: if we\ndetect \"NS execute from S memory\" we throw a QEMU internal exception,\nand in the cpu_do_interrupt code we either (a) identify that this\nis the SG instruction and execute it or (b) generate the right guest\nCPU exception.\n\nThat's definitely executing an instruction, and there's no translation\ntime for it...\n\nthanks\n-- PMM","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org; dkim=pass (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"JZ3vDpqp\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxPD86VYcz9rxl\n\tfor ;\n\tTue, 19 Sep 2017 23:41:28 +1000 (AEST)","from localhost ([::1]:42963 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duImE-0003me-1x\n\tfor incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 09:41:26 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:59285)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duIHN-0003vF-TE\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 09:09:37 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duIHJ-0008QY-QY\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 09:09:33 -0400","from mail-wm0-x22f.google.com ([2a00:1450:400c:c09::22f]:47736)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1duIHJ-0008Pn-JE\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 09:09:29 -0400","by mail-wm0-x22f.google.com with SMTP id 13so5108805wmq.2\n\tfor ; Tue, 19 Sep 2017 06:09:29 -0700 (PDT)","by 10.223.139.215 with HTTP; Tue, 19 Sep 2017 06:09:07 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:content-transfer-encoding;\n\tbh=TfB2J2qd0NM5lNWIQAiC6rKnXSL69HKxSHqMKPa+SQI=;\n\tb=JZ3vDpqpvPyuhSVcEkqPUX/QTmW/Yt2voAoIZ9bDcOIkSWyAlKwFqmc6cCvumkcy6q\n\tv/xg2TRF7F7LV/Vsusfn2lreZshJyOW9UnsSWmWpPrZxbJ4QWMWw6o5xOcDhJqMBgPIo\n\tCRvKS6hs9+X2JqlmsR+c0rII7ovH3jwLX3vmo=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:content-transfer-encoding;\n\tbh=TfB2J2qd0NM5lNWIQAiC6rKnXSL69HKxSHqMKPa+SQI=;\n\tb=iHMhVOrZdSHoFCkymNE4GQ9UxYC9AAaZQXGNLGHZiQJA6NCxhBcHZhPgfqTk/k8PzL\n\teWmjtf1b8jGszf9i8wfZ/mazY5codZlC474clojHSBM9yb9zZ9CiOQ9rKG8eDIUfk5zq\n\tBJPOLNH0zdXC3dR7sVzPxwLcgZ5A9S9QGC+p3P2DJ3YmM94xuPmfhgT4yIOY8AO1wxFe\n\tA6rqwPYwxx9hl3Pl5pru6EW4OBT1+HZboc5KChWLckN8ftmNKcBS30AvCywna61NqleG\n\tg642v/MGCjATDS042MI7bdmvS8/1Eid+7VNExoSAWBYVGbzdYCOcuQX/FFNKyMiROhT8\n\tBNKA==","X-Gm-Message-State":"AHPjjUiib2L8SI/Mxhf5UqsCOlL/S7jiYgf0Cwu+xka8lI8ryxjPiC7+\n\tMRKVbZNFGHQ9Y6S8/sS2+uUPbsfSd9kXVVvxe/0D+w==","X-Google-Smtp-Source":"AOwi7QCJkkmDwBCIUHFnJvLCmcLISVykoEIxeK5fmcNb4diN9IdaOY9KHrxTEV9QvsfkMZEQwJiVE7/OIYIxrv/iyCc=","X-Received":"by 10.28.166.135 with SMTP id p129mr1093000wme.147.1505826568589;\n\tTue, 19 Sep 2017 06:09:28 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<87d16o53xr.fsf@frigg.lan>","References":"<150529642278.10902.18234057937634437857.stgit@frigg.lan>\n\t<150529666493.10902.14830445134051381968.stgit@frigg.lan>\n\t\n\t<87poasgjyh.fsf@frigg.lan>\n\t\n\t<87d16o53xr.fsf@frigg.lan>","From":"Peter Maydell ","Date":"Tue, 19 Sep 2017 14:09:07 +0100","Message-ID":"","To":"Peter Maydell ,\n\t\"Emilio G. Cota\" , \n\tQEMU Developers ,\n\tStefan Hajnoczi , \n\tMarkus Armbruster ","Content-Type":"text/plain; charset=\"UTF-8\"","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c09::22f","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1771043,"web_url":"http://patchwork.ozlabs.org/comment/1771043/","msgid":"<20170919135019.GA10773@flamenco>","list_archive_url":null,"date":"2017-09-19T13:50:19","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":65690,"url":"http://patchwork.ozlabs.org/api/people/65690/","name":"Emilio Cota","email":"cota@braap.org"},"content":"On Mon, Sep 18, 2017 at 18:42:55 +0100, Peter Maydell wrote:\n> On 18 September 2017 at 18:09, Lluís Vilanova wrote:\n> > It also means we won't be able to \"conditionally\" instrument instructions (e.g.,\n> > based on their opcode, address range, etc.).\n> \n> You can still do that, it's just less efficient (your\n> condition-check happens in the callout to the instrumentation\n> plugin). We can add \"filter\" options later if we need them\n> (which I would rather do than have translate-time callbacks).\n\nMy initial reaction to not having translation-time hooks was that it'd\nbe too slow. However, thinking about it a bit more I suspect it might\nnot matter much. Other tools such as Pin/DynamoRIO have these hooks,\nand in their case it makes sense because one can choose to, instead of\nhaving one callback per executed instruction, get a callback per\nexecuted \"trace\" (a trace is a chain of TBs). Crucially, these tools do\nnot go through an intermediate IR, and as a result are about 10X faster\nthan QEMU.\n\nTherefore, given that QEMU is comparatively slow, we might find that\nper-executed-instruction callbacks do not end up slowing things down\nsignificantly.\n\nI like the idea of discussing an API alone, but there is value in having\nan implementation to go with it, so that we can explore the perf\ntrade-offs involved. I was busy with other things the last 10 days or so,\nbut by the end of this week I hope I'll be able to share some\nperf numbers on the per-TB vs. per-instruction callback issue.\n\nThanks,\n\n\t\tEmilio","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org; dkim=pass (1024-bit key;\n\tunprotected) header.d=braap.org header.i=@braap.org\n\theader.b=\"Bw54ch04\"; dkim=pass (2048-bit key;\n\tunprotected) header.d=messagingengine.com\n\theader.i=@messagingengine.com header.b=\"KEbf4ZDk\"; \n\tdkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxPWk68GFz9s7B\n\tfor ;\n\tTue, 19 Sep 2017 23:54:58 +1000 (AEST)","from localhost ([::1]:43071 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duIzJ-0005bY-0W\n\tfor incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 09:54:57 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:59687)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duIux-00030a-6a\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 09:50:32 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duIus-0006pP-HG\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 09:50:27 -0400","from out1-smtp.messagingengine.com ([66.111.4.25]:43853)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1duIus-0006nt-8w\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 09:50:22 -0400","from compute4.internal (compute4.nyi.internal [10.202.2.44])\n\tby mailout.nyi.internal (Postfix) with ESMTP id 1F2FB20DE5;\n\tTue, 19 Sep 2017 09:50:20 -0400 (EDT)","from frontend1 ([10.202.2.160])\n\tby compute4.internal (MEProxy); Tue, 19 Sep 2017 09:50:20 -0400","from localhost (flamenco.cs.columbia.edu [128.59.20.216])\n\tby mail.messagingengine.com (Postfix) with ESMTPA id BECD57E8F0;\n\tTue, 19 Sep 2017 09:50:19 -0400 (EDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h=cc\n\t:content-transfer-encoding:content-type:date:from:in-reply-to\n\t:message-id:mime-version:references:subject:to:x-me-sender\n\t:x-me-sender:x-sasl-enc:x-sasl-enc; s=mesmtp; bh=cCjAfsz7Oa7VtaI\n\tJIFiTNCKjhVqCwPCiHnhxt7806/U=; b=Bw54ch04f+64tm9T8Eqx9Uw2oz5htTR\n\tKRgjhRKphAF0L+20EABGbpxry5T+ACBhJlIP/DtN2Za7+qo71W3fcQztUjUYkKVF\n\tY7p5eqKIDLPLBhg97xd4ftDOy9MezPR7UcjDLdLvlcwxRdXvVV7D6d0Lgm9CvKe8\n\t1daN3SNj3nHM=","v=1; a=rsa-sha256; c=relaxed/relaxed; d=\n\tmessagingengine.com; h=cc:content-transfer-encoding:content-type\n\t:date:from:in-reply-to:message-id:mime-version:references\n\t:subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s=\n\tfm1; bh=cCjAfsz7Oa7VtaIJIFiTNCKjhVqCwPCiHnhxt7806/U=; b=KEbf4ZDk\n\t9SlhFQrviKDVX79gImmZ1QOX428ajucjgvJW9cJCQjMSRI5V5ACwsXACDV491QmS\n\tDmNzCMtql7oNqALIgEXn5CxW8FhXqESdbMPFpzmNAiVKJdogs5wBSb+NQlYRvPHT\n\tW/6ekcbvAzsgzGQSw1uCXXq48CtHFd8UDM2YllsmpXyFLAtjSqiIHi5TXZfZ0AF2\n\tQ+oawBtjGgEed6cYWS41Ijv/L+IOHrspe7J2cH48l28TdxEMUfksHGaJX3qQe/4I\n\tZ+eMx+Z2jMB8fcTPwNI43jC2XiFHvLsXG3W80iY9TT4fETCkOTclcpIuJULU5SS6\n\tLtUGYQ3KS3M/AA=="],"X-ME-Sender":"","X-Sasl-enc":"kZXDKT7wCFJAIBRPeAfjuxfk9s5OG421+Gif4dLg1xXO 1505829019","Date":"Tue, 19 Sep 2017 09:50:19 -0400","From":"\"Emilio G. Cota\" ","To":"Peter Maydell ","Message-ID":"<20170919135019.GA10773@flamenco>","References":"<150529642278.10902.18234057937634437857.stgit@frigg.lan>\n\t<150529666493.10902.14830445134051381968.stgit@frigg.lan>\n\t\n\t<87poasgjyh.fsf@frigg.lan>\n\t\n\t<87d16o53xr.fsf@frigg.lan>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=iso-8859-1","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"66.111.4.25","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"QEMU Developers ,\n\tStefan Hajnoczi ,\n\tMarkus Armbruster ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1774897,"web_url":"http://patchwork.ozlabs.org/comment/1774897/","msgid":"<87o9pywt8k.fsf@frigg.lan>","list_archive_url":null,"date":"2017-09-25T18:03:39","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":9099,"url":"http://patchwork.ozlabs.org/api/people/9099/","name":"Lluís Vilanova","email":"vilanova@ac.upc.edu"},"content":"First, sorry for the late response; I was away for a few days.\n\n\nPeter Maydell writes:\n\n> On 18 September 2017 at 18:09, Lluís Vilanova wrote:\n>> Peter Maydell writes:\n>>> It's also exposing internal QEMU implementation detail.\n>>> What if in future we decide to switch from our current\n>>> setup to always interpreting guest instructions as a\n>>> first pass with JITting done only in the background for\n>>> hot code?\n>> \n>> TCI still has a separation of translation-time (translate.c) and execution-time\n>> (interpreting the TCG opcodes), and I don't think that's gonna go away anytime\n>> soon.\n\n> I didn't mean TCI, which is nothing like what you'd use for\n> this if you did it (TCI is slower than just JITting.)\n\nMy point is that even on the cold path you need to decode a guest instruction\n(equivalent to translating) and emulate it on the spot (equivalent to\nexecuting).\n\n\n>> Even if it did, I think there still will be a translation/execution separation\n>> easy enough to hook into (even if it's a \"fake\" one for the cold-path\n>> interpreted instructions).\n\n> But what would it mean? You don't have basic blocks any more.\n\nEvery instruction emulated on the spot can be seen as a newly translated block\n(of one instruction only), which is executed immediately after.\n\n\n>>> Sticking to instrumentation events that correspond exactly to guest\n>>> execution events means they won't break or expose internals.\n>> \n>> It also means we won't be able to \"conditionally\" instrument instructions (e.g.,\n>> based on their opcode, address range, etc.).\n\n> You can still do that, it's just less efficient (your\n> condition-check happens in the callout to the instrumentation\n> plugin). We can add \"filter\" options later if we need them\n> (which I would rather do than have translate-time callbacks).\n\nBefore answering, a short summary of when knowing about translate/execute makes\na difference:\n\n* Record some information only once when an instruction is translated, instead\n of recording it on every executed instruction (e.g., a study of opcode\n distribution, which you can get from a file of per-TB opcodes - generated at\n translation time - and a list of executed TBs - generated at execution time\n -). The translate/execute separation makes this run faster *and* produces much\n smaller files with the recorded info.\n\n Other typical examples that benefit from this are writing a simulator that\n feeds off a stream of instruction information (a common reason why people want\n to trace memory accesses and information of executed instructions).\n\n* Conditionally instrumenting instructions.\n\nAdding filtering to the instrumentation API would only solve the second point,\nbut not the first one.\n\nNow, do we need/want to support the first point?\n\n\n>> Of course we can add the translation/execution differentiation later if we find\n>> it necessary for performance, but I would rather avoid leaving \"historical\"\n>> instrumentation points behind on the API.\n>> \n>> What are the use-cases you're aiming for?\n\n> * I want to be able to point the small stream of people who come\n> into qemu-devel asking \"how do I trace all my guest's memory\n> accesses\" at a clean API for it.\n\n> * I want to be able to have less ugly and confusing tracing\n> than our current -d output (and perhaps emit tracing in formats\n> that other analysis tools want as input)\n\n> * I want to keep this initial tracing API simple enough that\n> we can agree on it and get a first working useful version.\n\nFair enough.\n\nI know it's not exactly the same we're discussing, but the plot in [1] compares\na few different ways to trace memory accesses on SPEC benchmarks:\n\n* First bar is using a Intel's tool called PIN [2].\n* Second is calling into an instrumentation function on every executed memory\n access in QEMU.\n* Third is embedding the hot path of writing the memory access info to an array\n into the TCG opcode stream (more or less equivalent to supporting filtering;\n when the array is full, a user's callback is called - cold path -)\n* Fourth bar can be ignored.\n\nThis was working on a much older version of instrumentation for QEMU, but I can\nimplement something that does the first use-case point above and some filtering\nexample (second use-case point) to see what's the performance difference.\n\n[1] https://filetea.me/n3wy9WwyCCZR72E9OWXHArHDw\n[2] https://software.intel.com/en-us/articles/pin-a-dynamic-binary-instrumentation-tool\n\n\nThanks!\n Lluis","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y1Bmt1fsXz9sRq\n\tfor ;\n\tTue, 26 Sep 2017 04:04:29 +1000 (AEST)","from localhost ([::1]:43690 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dwXk3-0005SG-RB\n\tfor incoming@patchwork.ozlabs.org; Mon, 25 Sep 2017 14:04:27 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:46744)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dwXjd-0005Ry-KI\n\tfor qemu-devel@nongnu.org; Mon, 25 Sep 2017 14:04:03 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dwXja-0002RC-Eq\n\tfor qemu-devel@nongnu.org; Mon, 25 Sep 2017 14:04:01 -0400","from roura.ac.upc.es ([147.83.33.10]:41609)\n\tby eggs.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dwXjZ-0002PW-VF\n\tfor qemu-devel@nongnu.org; Mon, 25 Sep 2017 14:03:58 -0400","from correu-1.ac.upc.es (correu-1.ac.upc.es [147.83.30.91])\n\tby roura.ac.upc.es (8.13.8/8.13.8) with ESMTP id v8PI3kvB024225;\n\tMon, 25 Sep 2017 20:03:46 +0200","from localhost (unknown [132.68.50.201])\n\tby correu-1.ac.upc.es (Postfix) with ESMTPSA id F160E147;\n\tMon, 25 Sep 2017 20:03:40 +0200 (CEST)"],"From":"=?utf-8?q?Llu=C3=ADs_Vilanova?= ","To":"Peter Maydell ","References":"<150529642278.10902.18234057937634437857.stgit@frigg.lan>\n\t<150529666493.10902.14830445134051381968.stgit@frigg.lan>\n\t\n\t<87poasgjyh.fsf@frigg.lan>\n\t\n\t<87d16o53xr.fsf@frigg.lan>\n\t","Mail-Followup-To":"Peter Maydell , \"Emilio G. Cota\"\n\t, QEMU Developers ,\n\tStefan\n\tHajnoczi , Markus Armbruster ","Date":"Mon, 25 Sep 2017 21:03:39 +0300","In-Reply-To":"\n\t(Peter Maydell's message of \"Mon, 18 Sep 2017 18:42:55 +0100\")","Message-ID":"<87o9pywt8k.fsf@frigg.lan>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux)","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.6.x [fuzzy]","X-Received-From":"147.83.33.10","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"\"Emilio G. Cota\" ,\n\tQEMU Developers , \n\tStefan Hajnoczi ,\n\tMarkus Armbruster ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1774946,"web_url":"http://patchwork.ozlabs.org/comment/1774946/","msgid":"<20170925194202.GA19618@flamenco>","list_archive_url":null,"date":"2017-09-25T19:42:02","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":65690,"url":"http://patchwork.ozlabs.org/api/people/65690/","name":"Emilio Cota","email":"cota@braap.org"},"content":"On Mon, Sep 25, 2017 at 21:03:39 +0300, Lluís Vilanova wrote:\n> I know it's not exactly the same we're discussing, but the plot in [1] compares\n> a few different ways to trace memory accesses on SPEC benchmarks:\n> \n> * First bar is using a Intel's tool called PIN [2].\n> * Second is calling into an instrumentation function on every executed memory\n> access in QEMU.\n> * Third is embedding the hot path of writing the memory access info to an array\n> into the TCG opcode stream (more or less equivalent to supporting filtering;\n> when the array is full, a user's callback is called - cold path -)\n> * Fourth bar can be ignored.\n> \n> This was working on a much older version of instrumentation for QEMU, but I can\n> implement something that does the first use-case point above and some filtering\n> example (second use-case point) to see what's the performance difference.\n> \n> [1] https://filetea.me/n3wy9WwyCCZR72E9OWXHArHDw\n\nInteresting! Unfortunately, this URL gives me a 404.\n\n\t\tE.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org; dkim=pass (1024-bit key;\n\tunprotected) header.d=braap.org header.i=@braap.org\n\theader.b=\"Zrnh9qOO\"; dkim=pass (2048-bit key;\n\tunprotected) header.d=messagingengine.com\n\theader.i=@messagingengine.com header.b=\"BP0SEPtr\"; \n\tdkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y1Dy10Wy3z9s76\n\tfor ;\n\tTue, 26 Sep 2017 05:42:33 +1000 (AEST)","from localhost ([::1]:43992 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dwZGx-0005LW-0R\n\tfor incoming@patchwork.ozlabs.org; Mon, 25 Sep 2017 15:42:31 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:48873)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dwZGZ-0005LP-3u\n\tfor qemu-devel@nongnu.org; Mon, 25 Sep 2017 15:42:08 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dwZGW-0005km-0A\n\tfor qemu-devel@nongnu.org; Mon, 25 Sep 2017 15:42:07 -0400","from out1-smtp.messagingengine.com ([66.111.4.25]:56309)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1dwZGV-0005kY-QP\n\tfor qemu-devel@nongnu.org; Mon, 25 Sep 2017 15:42:03 -0400","from compute4.internal (compute4.nyi.internal [10.202.2.44])\n\tby mailout.nyi.internal (Postfix) with ESMTP id F185A22152;\n\tMon, 25 Sep 2017 15:42:02 -0400 (EDT)","from frontend2 ([10.202.2.161])\n\tby compute4.internal (MEProxy); Mon, 25 Sep 2017 15:42:02 -0400","from localhost (flamenco.cs.columbia.edu [128.59.20.216])\n\tby mail.messagingengine.com (Postfix) with ESMTPA id A53DD247DD;\n\tMon, 25 Sep 2017 15:42:02 -0400 (EDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h=\n\tcontent-transfer-encoding:content-type:date:from:in-reply-to\n\t:message-id:mime-version:references:subject:to:x-me-sender\n\t:x-me-sender:x-sasl-enc:x-sasl-enc; s=mesmtp; bh=YKtQZl4x58Be7um\n\tWa0PcrWck72Ye74uz3ejsU+hyXVM=; b=Zrnh9qOOb/QGkMdY0+f7dUH/JuFmrnq\n\tlyiu7/CBA7JbIPEC557D8ih6MeM6S5f5K5F7yzwfSszAkS/JIcq8Y1YSKtHxhbuY\n\txuRGq7m4cF6s7qYAt9Oh+joMVvt7uy841EgtCZlx1rEH7SD1vagsC1lI88z0CheK\n\tsXc3Ly3GRsF0=","v=1; a=rsa-sha256; c=relaxed/relaxed; d=\n\tmessagingengine.com; h=content-transfer-encoding:content-type\n\t:date:from:in-reply-to:message-id:mime-version:references\n\t:subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s=\n\tfm1; bh=YKtQZl4x58Be7umWa0PcrWck72Ye74uz3ejsU+hyXVM=; b=BP0SEPtr\n\thUhjZ2qX9JFWNbQYqEV8/VM3N9PNQez6Ce6QTLFlP3Lra82odlLfYDovyd9llik2\n\tXeiYKZCHE3A8UZCzrym01yi/wskZttiE0WuCHxUrBChPM3AOfl3zdjyu492vxJY4\n\t6FnUT+YI6bn42ECF1TdbH30TIvxnUyOmlEa8zegJv3F3gnZ+LJyR6b/9JqRwFUlk\n\tZcV2/wqXdRWvD9GH2pWlnP2D9xsU7M8Np2mWFBpzXil9Oja9NGuXI6RPbXLkQd5i\n\tK2yO6rGhfLkKF7PyuOrjZiA3m5KYQEfjZLqpoaMPcdLW0ObnmzdeffBbb5z55ggB\n\ts7Wx0YBhMpxY6g=="],"X-ME-Sender":"","X-Sasl-enc":"++X1wZO+IxyUwUEgGObCNT7zfoHN9meBmb9gfVLvKh+Q 1506368522","Date":"Mon, 25 Sep 2017 15:42:02 -0400","From":"\"Emilio G. Cota\" ","To":"Peter Maydell ,\n\tQEMU Developers ,\n\tStefan Hajnoczi ,\n\tMarkus Armbruster ","Message-ID":"<20170925194202.GA19618@flamenco>","References":"<150529642278.10902.18234057937634437857.stgit@frigg.lan>\n\t<150529666493.10902.14830445134051381968.stgit@frigg.lan>\n\t\n\t<87poasgjyh.fsf@frigg.lan>\n\t\n\t<87d16o53xr.fsf@frigg.lan>\n\t\n\t<87o9pywt8k.fsf@frigg.lan>","MIME-Version":"1.0","Content-Type":"text/plain; charset=iso-8859-1","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<87o9pywt8k.fsf@frigg.lan>","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"66.111.4.25","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1775679,"web_url":"http://patchwork.ozlabs.org/comment/1775679/","msgid":"<87bmlxtnft.fsf@frigg.lan>","list_archive_url":null,"date":"2017-09-26T16:49:26","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":9099,"url":"http://patchwork.ozlabs.org/api/people/9099/","name":"Lluís Vilanova","email":"vilanova@ac.upc.edu"},"content":"Emilio G Cota writes:\n\n> On Mon, Sep 25, 2017 at 21:03:39 +0300, Lluís Vilanova wrote:\n>> I know it's not exactly the same we're discussing, but the plot in [1] compares\n>> a few different ways to trace memory accesses on SPEC benchmarks:\n>> \n>> * First bar is using a Intel's tool called PIN [2].\n>> * Second is calling into an instrumentation function on every executed memory\n>> access in QEMU.\n>> * Third is embedding the hot path of writing the memory access info to an array\n>> into the TCG opcode stream (more or less equivalent to supporting filtering;\n>> when the array is full, a user's callback is called - cold path -)\n>> * Fourth bar can be ignored.\n>> \n>> This was working on a much older version of instrumentation for QEMU, but I can\n>> implement something that does the first use-case point above and some filtering\n>> example (second use-case point) to see what's the performance difference.\n>> \n>> [1] https://filetea.me/n3wy9WwyCCZR72E9OWXHArHDw\n\n> Interesting! Unfortunately, this URL gives me a 404.\n\nOk, I've uploade it somewhere else:\n\n https://people.gso.ac.upc.edu/vilanova/mtrace.pdf\n\nThere's also another one that simply counts the number of memory accesses, using\nthe same three approaches:\n\n https://people.gso.ac.upc.edu/vilanova/mcount.pdf\n\nCheers,\n Lluis","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y1n4h42fyz9s7f\n\tfor ;\n\tWed, 27 Sep 2017 02:50:12 +1000 (AEST)","from localhost ([::1]:50310 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dwt3i-0004FK-NM\n\tfor incoming@patchwork.ozlabs.org; Tue, 26 Sep 2017 12:50:10 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:46089)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dwt3H-0004Dl-4J\n\tfor qemu-devel@nongnu.org; Tue, 26 Sep 2017 12:49:44 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dwt3E-00072x-0v\n\tfor qemu-devel@nongnu.org; Tue, 26 Sep 2017 12:49:43 -0400","from roura.ac.upc.es ([147.83.33.10]:38270)\n\tby eggs.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dwt3D-000720-M0\n\tfor qemu-devel@nongnu.org; Tue, 26 Sep 2017 12:49:39 -0400","from correu-1.ac.upc.es (correu-1.ac.upc.es [147.83.30.91])\n\tby roura.ac.upc.es (8.13.8/8.13.8) with ESMTP id v8QGnXsO024425;\n\tTue, 26 Sep 2017 18:49:33 +0200","from localhost (unknown [132.68.50.201])\n\tby correu-1.ac.upc.es (Postfix) with ESMTPSA id A1501136E;\n\tTue, 26 Sep 2017 18:49:27 +0200 (CEST)"],"From":"=?utf-8?q?Llu=C3=ADs_Vilanova?= ","To":"\"Emilio G. Cota\" ","References":"<150529642278.10902.18234057937634437857.stgit@frigg.lan>\n\t<150529666493.10902.14830445134051381968.stgit@frigg.lan>\n\t\n\t<87poasgjyh.fsf@frigg.lan>\n\t\n\t<87d16o53xr.fsf@frigg.lan>\n\t\n\t<87o9pywt8k.fsf@frigg.lan> <20170925194202.GA19618@flamenco>","Mail-Followup-To":"\"Emilio G. Cota\" , Peter Maydell\n\t, QEMU Developers , \n\tStefan Hajnoczi , Markus Armbruster\n\t","Date":"Tue, 26 Sep 2017 19:49:26 +0300","In-Reply-To":"<20170925194202.GA19618@flamenco> (Emilio G. Cota's message of\n\t\"Mon, 25 Sep 2017 15:42:02 -0400\")","Message-ID":"<87bmlxtnft.fsf@frigg.lan>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux)","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.6.x [fuzzy]","X-Received-From":"147.83.33.10","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Peter Maydell ,\n\tQEMU Developers ,\n\tStefan Hajnoczi ,\n\tMarkus Armbruster ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1780179,"web_url":"http://patchwork.ozlabs.org/comment/1780179/","msgid":"<8760bu333n.fsf@frigg.lan>","list_archive_url":null,"date":"2017-10-04T23:28:12","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":9099,"url":"http://patchwork.ozlabs.org/api/people/9099/","name":"Lluís Vilanova","email":"vilanova@ac.upc.edu"},"content":"Emilio G Cota writes:\n\n> On Sat, Sep 30, 2017 at 00:46:33 +0300, Lluís Vilanova wrote:\n>> Emilio G Cota writes:\n>> > I'm not sure I understand this concept of filtering. Are you saying that in\n>> > the first case, all memory accesses are instrumented, and then in the\n>> > \"access helper\" we only call the user's callback if it's a memory write?\n>> > And in the second case, we simply just generate a \"write helper\" instead\n>> > of an \"access helper\". Am I understanding this correctly?\n>> \n>> In the previous case (no filtering), the user callback is always called when a\n>> memory access is *executed*, and the user then checks if the access mode is a\n>> write to decide whether to increment a counter.\n>> \n>> In this case (with filtering), a user callback is called when a memory access is\n>> *translated*, and if the access mode is a write, the user generates a call to a\n>> second callback that is executed every time a memory access is executed (only\n>> that it is only generated for memory writes, the ones we care about).\n>> \n>> Is this clearer?\n\n> I get it now, thanks!\n\n>> > FWIW my experiments so far show similar numbers for instrumenting each\n>> > instruction (haven't done the per-tb yet). The difference is that I'm\n>> > exposing to instrumenters a copy of the guest instructions (const void *data,\n>> > size_t size). These copies are kept around until TB's are flushed.\n>> > Luckily there seems to be very little overhead in keeping these around,\n>> > apart from the memory overhead -- but in terms of performance, the\n>> > necessary allocations do not induce significant overhead.\n>> \n>> To keep this use-case simpler, I added the memory access API I posted in this\n>> series, where instrumenters can read guest memory (more general than passing a\n>> copy of the current instruction).\n\n> I see some potential problems with this:\n> 1. Instrumenters' accesses could generate exceptions. I presume we'd want to avoid\n> this, or leave it as a debug-only kind of option.\n\nThe API takes care of telling you if the access could be performed\nsuccessfully. If you access the instruction's memory representation at\ntranslation time, it should be able to perform the access, since QEMU's\ntranslation loop just had to do so in order to access that instruction (I should\ncheck what happens in the corner case where another guest CPU changes the page\ntable, since I'm not sure if the address translation functions I'm using in QEMU\nwill use the per-vCPU TLB cache or always traverse the page table).\n\n\n> 2. Instrumenters won't know where the end of an instruction (for variable-length\n> ISAs) or of a TB is (TB != basic block). For instructions one could have a loop\n> where we read byte-by-byte and pass it to the decoder, something similar to\n> what we have in the capstone code recently posted to the list (v4). For TBs,\n> we really should have a way to delimit the length of the TB. This is further\n> complicated if we want instrumentation to be inserted *before* a TB is\n> translated.\n\n> Some thoughts on the latter problem: if we want a tb_trans_pre callback, like\n> Pin/DynamoRIO provide, instead of doing two passes (one to delimit the TB and\n> call the tb_trans_pre callback, to then generate the translated TB), we could:\n> - have a tb_trans_pre callback. This callback inserts an exec-time callback\n> with a user-defined pointer (let's call it **tb_info). The callback has\n> no arguments, perhaps just the pc.\n> - have a tb_trans_post callback. This one passes a copy of the guest\n> instructions. The instrumenter then can allocate whatever data structure\n> to represent the TB (*tb_info), and copies this pointer to **tb_info, so\n> that at execution time, we can obtain tb_info _before_ the TB is executed.\n> After the callback returns, the copy of the guest instructions can be freed.\n> This has two disadvantages:\n> - We have an extra dereference to find tb_info\n> - If it turns out that the TB should not be instrumented, we have generated\n> a callback for nothing.\n\nThat's precisely one of the reasons why I proposed adding instrumentation points\nbefore and after events happen (e.g., instrument right after translating an\ninstruction, where you know its size).\n\nWhat you propose is actually a broader issue, how to allow instrumentors to pass\ntheir own data to execution-time functions \"after the fact\". For this, I\nimplemented \"promises\", a kind of generalization of what gen_icount() does (you\npass a value to the execution-time callback that is computed later during\ntranslation-time).\n\n\nCheers,\n Lluis","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y6sYL47Grz9t2Z\n\tfor ;\n\tThu, 5 Oct 2017 10:29:07 +1100 (AEDT)","from localhost ([::1]:37176 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dzt66-0003pC-Bk\n\tfor incoming@patchwork.ozlabs.org; Wed, 04 Oct 2017 19:29:02 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:56091)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dzt5k-0003p5-Uk\n\tfor qemu-devel@nongnu.org; Wed, 04 Oct 2017 19:28:42 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dzt5f-0004Fc-Ou\n\tfor qemu-devel@nongnu.org; Wed, 04 Oct 2017 19:28:40 -0400","from roura.ac.upc.edu ([147.83.33.10]:35742 helo=roura.ac.upc.es)\n\tby eggs.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dzt5f-0004Cy-9x\n\tfor qemu-devel@nongnu.org; Wed, 04 Oct 2017 19:28:35 -0400","from correu-2.ac.upc.es (correu-2.ac.upc.es [147.83.30.92])\n\tby roura.ac.upc.es (8.13.8/8.13.8) with ESMTP id v94NSKiG023970;\n\tThu, 5 Oct 2017 01:28:25 +0200","from localhost (unknown [31.210.187.58])\n\tby correu-2.ac.upc.es (Postfix) with ESMTPSA id D8069155;\n\tThu, 5 Oct 2017 01:28:14 +0200 (CEST)"],"From":"=?utf-8?q?Llu=C3=ADs_Vilanova?= ","To":"\"Emilio G. Cota\" ","References":"<150529666493.10902.14830445134051381968.stgit@frigg.lan>\n\t\n\t<87poasgjyh.fsf@frigg.lan>\n\t\n\t<87d16o53xr.fsf@frigg.lan>\n\t\n\t<87o9pywt8k.fsf@frigg.lan> <87shf5zlty.fsf@frigg.lan>\n\t<20170929175943.GA25038@flamenco> <87vak1w53a.fsf@frigg.lan>\n\t<20170930180941.GA22048@flamenco>","Mail-Followup-To":"\"Emilio G. Cota\" , Peter Maydell\n\t, QEMU Developers , \n\tStefan Hajnoczi , Markus Armbruster\n\t","Date":"Thu, 05 Oct 2017 02:28:12 +0300","In-Reply-To":"<20170930180941.GA22048@flamenco> (Emilio G. Cota's message of\n\t\"Sat, 30 Sep 2017 14:09:41 -0400\")","Message-ID":"<8760bu333n.fsf@frigg.lan>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux)","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.6.x [fuzzy]","X-Received-From":"147.83.33.10","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Peter Maydell ,\n\tQEMU Developers ,\n\tStefan Hajnoczi ,\n\tMarkus Armbruster ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1780233,"web_url":"http://patchwork.ozlabs.org/comment/1780233/","msgid":"<20171005005043.GA20425@flamenco>","list_archive_url":null,"date":"2017-10-05T00:50:43","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":65690,"url":"http://patchwork.ozlabs.org/api/people/65690/","name":"Emilio Cota","email":"cota@braap.org"},"content":"On Thu, Oct 05, 2017 at 02:28:12 +0300, Lluís Vilanova wrote:\n> Emilio G Cota writes:\n> > I see some potential problems with this:\n> > 1. Instrumenters' accesses could generate exceptions. I presume we'd want to avoid\n> > this, or leave it as a debug-only kind of option.\n> \n> The API takes care of telling you if the access could be performed\n> successfully. If you access the instruction's memory representation at\n> translation time, it should be able to perform the access, since QEMU's\n> translation loop just had to do so in order to access that instruction (I should\n> check what happens in the corner case where another guest CPU changes the page\n> table, since I'm not sure if the address translation functions I'm using in QEMU\n> will use the per-vCPU TLB cache or always traverse the page table).\n\nThat was my concern, I'd rather just perform the read once, that is, the read(s)\ndone by ops->insn_translate.\n\n> > 2. Instrumenters won't know where the end of an instruction (for variable-length\n> > ISAs) or of a TB is (TB != basic block). For instructions one could have a loop\n> > where we read byte-by-byte and pass it to the decoder, something similar to\n> > what we have in the capstone code recently posted to the list (v4). For TBs,\n> > we really should have a way to delimit the length of the TB. This is further\n> > complicated if we want instrumentation to be inserted *before* a TB is\n> > translated.\n> \n> > Some thoughts on the latter problem: if we want a tb_trans_pre callback, like\n> > Pin/DynamoRIO provide, instead of doing two passes (one to delimit the TB and\n> > call the tb_trans_pre callback, to then generate the translated TB), we could:\n> > - have a tb_trans_pre callback. This callback inserts an exec-time callback\n> > with a user-defined pointer (let's call it **tb_info). The callback has\n> > no arguments, perhaps just the pc.\n> > - have a tb_trans_post callback. This one passes a copy of the guest\n> > instructions. The instrumenter then can allocate whatever data structure\n> > to represent the TB (*tb_info), and copies this pointer to **tb_info, so\n> > that at execution time, we can obtain tb_info _before_ the TB is executed.\n> > After the callback returns, the copy of the guest instructions can be freed.\n> > This has two disadvantages:\n> > - We have an extra dereference to find tb_info\n> > - If it turns out that the TB should not be instrumented, we have generated\n> > a callback for nothing.\n> \n> That's precisely one of the reasons why I proposed adding instrumentation points\n> before and after events happen (e.g., instrument right after translating an\n> instruction, where you know its size).\n> \n> What you propose is actually a broader issue, how to allow instrumentors to pass\n> their own data to execution-time functions \"after the fact\". For this, I\n> implemented \"promises\", a kind of generalization of what gen_icount() does (you\n> pass a value to the execution-time callback that is computed later during\n> translation-time).\n\nI see. I implemented what I suggested above, i.e. tb_trans_cb\n(i.e. post_trans) passes an opaque descriptor of the TB (which can\nbe iterated over insn by insn) and the return value (void *) of this\ncb will be passed by tb_exec_cb (i.e. pre_exec). Perf-wise this\nis pretty OK, turns out even if we don't end up caring about the\nTB, the additional per-TB helper (which might not end up calling\na callback) does not introduce significant overhead at execution time.\n\nThe major hurdle I found is what to do when removing a plugin,\nso that we avoid flushing potentially all translated code. What I ended up\ndoing is adding a per-TB list of \"plugin_tb\" descriptors, which\ntrack these user pointers so that (1) each plugin gets the right\npointer, and (2) if n_plugins > 1, we still have a single helper\nthat dispatches the callbacks instead of n_plugin helpers.\n\nIf I understand correctly, with promises we directly generate\na callback, which has the promise(s) as one (or more) of its\narguments. This is neat and very flexible. However, it forces us to\nretranslate the TB when the plugin is removed (if we're lazy we could\nflush all TBs), and if we have several plugins, we end up with one\nhelper per callback, instead of a single one.\nNone of this is a huge deal though, I just think is worth considering.\n\nAlso, I'm not sure Peter and others would be happy with allowing\nplugin code to generate arbitrary callbacks (IIRC arbitrary code\nhas already been ruled out). So perhaps a more restrictive option\nlike what I suggested above would be more palatable.\n\nCheers,\n\n\t\tEmilio","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org; dkim=pass (1024-bit key;\n\tunprotected) header.d=braap.org header.i=@braap.org\n\theader.b=\"wpUwhqxk\"; dkim=pass (2048-bit key;\n\tunprotected) header.d=messagingengine.com\n\theader.i=@messagingengine.com header.b=\"Jo6vmaOQ\"; \n\tdkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y6vNL43Lvz9t3R\n\tfor ;\n\tThu, 5 Oct 2017 11:51:29 +1100 (AEDT)","from localhost ([::1]:37323 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dzuNo-0006jB-Gq\n\tfor incoming@patchwork.ozlabs.org; Wed, 04 Oct 2017 20:51:24 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:41478)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dzuNG-0006hw-OF\n\tfor qemu-devel@nongnu.org; Wed, 04 Oct 2017 20:50:51 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dzuND-00088v-MB\n\tfor qemu-devel@nongnu.org; Wed, 04 Oct 2017 20:50:50 -0400","from out1-smtp.messagingengine.com ([66.111.4.25]:43301)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1dzuND-00088M-Cz\n\tfor qemu-devel@nongnu.org; Wed, 04 Oct 2017 20:50:47 -0400","from compute4.internal (compute4.nyi.internal [10.202.2.44])\n\tby mailout.nyi.internal (Postfix) with ESMTP id 50CC120E8C;\n\tWed, 4 Oct 2017 20:50:44 -0400 (EDT)","from frontend2 ([10.202.2.161])\n\tby compute4.internal (MEProxy); Wed, 04 Oct 2017 20:50:44 -0400","from localhost (flamenco.cs.columbia.edu [128.59.20.216])\n\tby mail.messagingengine.com (Postfix) with ESMTPA id 127B624724;\n\tWed, 4 Oct 2017 20:50:44 -0400 (EDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h=\n\tcontent-transfer-encoding:content-type:date:from:in-reply-to\n\t:message-id:mime-version:references:subject:to:x-me-sender\n\t:x-me-sender:x-sasl-enc:x-sasl-enc; s=mesmtp; bh=c48gXlHAvsHZGK6\n\te8widH6gzTB/4DiedjgRjX7wxBRI=; b=wpUwhqxksqASC0Ni1dlRU3U8P4hk50/\n\tkXr05SXzoAhe+b43mD5qb7DlUv3HUORQzZocxqMaAKgi6FX1xDw/N8UQmN5JA2jI\n\t0Y7FeFlgIWXrHYnlJo/xImVYpCztZvjzhSU8eqDv/6QQAnYRKBDEAGSMqD6X/gHJ\n\tOqoAdm2LqsLc=","v=1; a=rsa-sha256; c=relaxed/relaxed; d=\n\tmessagingengine.com; h=content-transfer-encoding:content-type\n\t:date:from:in-reply-to:message-id:mime-version:references\n\t:subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s=\n\tfm1; bh=c48gXlHAvsHZGK6e8widH6gzTB/4DiedjgRjX7wxBRI=; b=Jo6vmaOQ\n\tjvTupeu8bltXfJkBtnKfLWyaeiTbG43Mx8ZLXI3dN2fjTW/AGBf9VLrmloXML4Kp\n\tN5Rg6E8yvr23gFn897KyGeYMZNSEI32cs7j7TupUYD00z2YCUYJLu945knPnA3mF\n\tk9xObfVFZWgG90FWP5EU4O1ywKXwBaAberDKz/4SmuLffm8aKKAcX0tFstz3zrud\n\tfwKBIcamxq9FIytcjP8t5OzCHb7umZ0XGdsXxXJL8aa3BmegLSYpeeLWNP8KTBU5\n\tKVvWzOuFxbv7A62V+ZjmcyZOwKmVlzXKD+3tzwQ/ULO53bDhPHwrJip9lDHCHD+5\n\tZqHhg6CiQtPTVA=="],"X-ME-Sender":"","X-Sasl-enc":"6FFxxDu9xsCpqw6I8j5imqWTKPvh47AxlrIG+duvYdSw 1507164644","Date":"Wed, 4 Oct 2017 20:50:43 -0400","From":"\"Emilio G. Cota\" ","To":"Peter Maydell ,\n\tQEMU Developers ,\n\tStefan Hajnoczi ,\n\tMarkus Armbruster ","Message-ID":"<20171005005043.GA20425@flamenco>","References":"<87poasgjyh.fsf@frigg.lan>\n\t\n\t<87d16o53xr.fsf@frigg.lan>\n\t\n\t<87o9pywt8k.fsf@frigg.lan> <87shf5zlty.fsf@frigg.lan>\n\t<20170929175943.GA25038@flamenco> <87vak1w53a.fsf@frigg.lan>\n\t<20170930180941.GA22048@flamenco> <8760bu333n.fsf@frigg.lan>","MIME-Version":"1.0","Content-Type":"text/plain; charset=iso-8859-1","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<8760bu333n.fsf@frigg.lan>","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"66.111.4.25","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1781713,"web_url":"http://patchwork.ozlabs.org/comment/1781713/","msgid":"<87vajsl3h7.fsf@frigg.lan>","list_archive_url":null,"date":"2017-10-06T15:07:16","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":9099,"url":"http://patchwork.ozlabs.org/api/people/9099/","name":"Lluís Vilanova","email":"vilanova@ac.upc.edu"},"content":"Emilio G Cota writes:\n\n> On Thu, Oct 05, 2017 at 02:28:12 +0300, Lluís Vilanova wrote:\n>> Emilio G Cota writes:\n>> > I see some potential problems with this:\n>> > 1. Instrumenters' accesses could generate exceptions. I presume we'd want to avoid\n>> > this, or leave it as a debug-only kind of option.\n>> \n>> The API takes care of telling you if the access could be performed\n>> successfully. If you access the instruction's memory representation at\n>> translation time, it should be able to perform the access, since QEMU's\n>> translation loop just had to do so in order to access that instruction (I should\n>> check what happens in the corner case where another guest CPU changes the page\n>> table, since I'm not sure if the address translation functions I'm using in QEMU\n>> will use the per-vCPU TLB cache or always traverse the page table).\n\n> That was my concern, I'd rather just perform the read once, that is, the read(s)\n> done by ops->insn_translate.\n\nIf your concern is on performance, that should not be an issue, since you'd be\nusing the memory peek functions at translation-time. Furthermore, since others\nsuggested having memory peek anyway, that's a nicer way (to me) to compose APIs\n(and is less complex to implement).\n\n\n>> > 2. Instrumenters won't know where the end of an instruction (for variable-length\n>> > ISAs) or of a TB is (TB != basic block). For instructions one could have a loop\n>> > where we read byte-by-byte and pass it to the decoder, something similar to\n>> > what we have in the capstone code recently posted to the list (v4). For TBs,\n>> > we really should have a way to delimit the length of the TB. This is further\n>> > complicated if we want instrumentation to be inserted *before* a TB is\n>> > translated.\n>> \n>> > Some thoughts on the latter problem: if we want a tb_trans_pre callback, like\n>> > Pin/DynamoRIO provide, instead of doing two passes (one to delimit the TB and\n>> > call the tb_trans_pre callback, to then generate the translated TB), we could:\n>> > - have a tb_trans_pre callback. This callback inserts an exec-time callback\n>> > with a user-defined pointer (let's call it **tb_info). The callback has\n>> > no arguments, perhaps just the pc.\n>> > - have a tb_trans_post callback. This one passes a copy of the guest\n>> > instructions. The instrumenter then can allocate whatever data structure\n>> > to represent the TB (*tb_info), and copies this pointer to **tb_info, so\n>> > that at execution time, we can obtain tb_info _before_ the TB is executed.\n>> > After the callback returns, the copy of the guest instructions can be freed.\n>> > This has two disadvantages:\n>> > - We have an extra dereference to find tb_info\n>> > - If it turns out that the TB should not be instrumented, we have generated\n>> > a callback for nothing.\n>> \n>> That's precisely one of the reasons why I proposed adding instrumentation points\n>> before and after events happen (e.g., instrument right after translating an\n>> instruction, where you know its size).\n>> \n>> What you propose is actually a broader issue, how to allow instrumentors to pass\n>> their own data to execution-time functions \"after the fact\". For this, I\n>> implemented \"promises\", a kind of generalization of what gen_icount() does (you\n>> pass a value to the execution-time callback that is computed later during\n>> translation-time).\n\n> I see. I implemented what I suggested above, i.e. tb_trans_cb\n> (i.e. post_trans) passes an opaque descriptor of the TB (which can\n> be iterated over insn by insn) and the return value (void *) of this\n> cb will be passed by tb_exec_cb (i.e. pre_exec). Perf-wise this\n> is pretty OK, turns out even if we don't end up caring about the\n> TB, the additional per-TB helper (which might not end up calling\n> a callback) does not introduce significant overhead at execution time.\n\nSo you build this structure after translating the whole TB, and the user can\niterate it to check the translated instructions. This is closer to other\nexisting tools: you iterate the structure and then decide which/how to\ninstrument instructions, memory accesses, etc within it.\n\nMy only concern is that this is much more complex than the simpler API I propose\n(you must build the informational structures, generate calls to every possible\ninstrumentation call, which will be optimized-out by TCG if the user decides not\nto use them, and overall pay in performance for any unused functionality),\nwhereas your approach can be implemented on top of it.\n\n\n> The major hurdle I found is what to do when removing a plugin,\n> so that we avoid flushing potentially all translated code. What I ended up\n> doing is adding a per-TB list of \"plugin_tb\" descriptors, which\n> track these user pointers so that (1) each plugin gets the right\n> pointer, and (2) if n_plugins > 1, we still have a single helper\n> that dispatches the callbacks instead of n_plugin helpers.\n\nI'm not sure it's worth optimising for the multiple plugin case.\n\n\n> If I understand correctly, with promises we directly generate\n> a callback, which has the promise(s) as one (or more) of its\n> arguments. This is neat and very flexible. However, it forces us to\n> retranslate the TB when the plugin is removed (if we're lazy we could\n> flush all TBs), and if we have several plugins, we end up with one\n> helper per callback, instead of a single one.\n> None of this is a huge deal though, I just think is worth considering.\n\nYes, that happens seldomly enough that the flush cost is negligible.\n\n> Also, I'm not sure Peter and others would be happy with allowing\n> plugin code to generate arbitrary callbacks (IIRC arbitrary code\n> has already been ruled out). So perhaps a more restrictive option\n> like what I suggested above would be more palatable.\n\nAFAIU, arbitrary access to QEMU's structures was ruled out, but not generating\ncalls to arbitrary user functions.\n\n\nCheers,\n Lluis","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y7tLG3yKZz9sMN\n\tfor ;\n\tSat, 7 Oct 2017 02:08:06 +1100 (AEDT)","from localhost ([::1]:45395 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e0UEN-00080H-U5\n\tfor incoming@patchwork.ozlabs.org; Fri, 06 Oct 2017 11:08:03 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:36784)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e0UDy-0007w4-LG\n\tfor qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:07:39 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e0UDu-0008Sf-GD\n\tfor qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:07:38 -0400","from roura.ac.upc.edu ([147.83.33.10]:34496 helo=roura.ac.upc.es)\n\tby eggs.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e0UDt-0008Kz-Sv\n\tfor qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:07:34 -0400","from correu-2.ac.upc.es (correu-2.ac.upc.es [147.83.30.92])\n\tby roura.ac.upc.es (8.13.8/8.13.8) with ESMTP id v96F7NU7013705;\n\tFri, 6 Oct 2017 17:07:23 +0200","from localhost (unknown [132.68.53.29])\n\tby correu-2.ac.upc.es (Postfix) with ESMTPSA id 5EA5F2F7;\n\tFri, 6 Oct 2017 17:07:18 +0200 (CEST)"],"From":"=?utf-8?q?Llu=C3=ADs_Vilanova?= ","To":"\"Emilio G. Cota\" ","References":"<87poasgjyh.fsf@frigg.lan>\n\t\n\t<87d16o53xr.fsf@frigg.lan>\n\t\n\t<87o9pywt8k.fsf@frigg.lan> <87shf5zlty.fsf@frigg.lan>\n\t<20170929175943.GA25038@flamenco> <87vak1w53a.fsf@frigg.lan>\n\t<20170930180941.GA22048@flamenco> <8760bu333n.fsf@frigg.lan>\n\t<20171005005043.GA20425@flamenco>","Mail-Followup-To":"\"Emilio G. Cota\" , Peter Maydell\n\t, QEMU Developers , \n\tStefan Hajnoczi , Markus Armbruster\n\t","Date":"Fri, 06 Oct 2017 18:07:16 +0300","In-Reply-To":"<20171005005043.GA20425@flamenco> (Emilio G. Cota's message of\n\t\"Wed, 4 Oct 2017 20:50:43 -0400\")","Message-ID":"<87vajsl3h7.fsf@frigg.lan>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux)","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.6.x [fuzzy]","X-Received-From":"147.83.33.10","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Peter Maydell ,\n\tQEMU Developers ,\n\tStefan Hajnoczi ,\n\tMarkus Armbruster ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1781829,"web_url":"http://patchwork.ozlabs.org/comment/1781829/","msgid":"<20171006175929.GA28281@flamenco>","list_archive_url":null,"date":"2017-10-06T17:59:29","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":65690,"url":"http://patchwork.ozlabs.org/api/people/65690/","name":"Emilio Cota","email":"cota@braap.org"},"content":"On Fri, Oct 06, 2017 at 18:07:16 +0300, Lluís Vilanova wrote:\n> Emilio G Cota writes:\n> > On Thu, Oct 05, 2017 at 02:28:12 +0300, Lluís Vilanova wrote:\n> >> The API takes care of telling you if the access could be performed\n> >> successfully. If you access the instruction's memory representation at\n> >> translation time, it should be able to perform the access, since QEMU's\n> >> translation loop just had to do so in order to access that instruction (I should\n> >> check what happens in the corner case where another guest CPU changes the page\n> >> table, since I'm not sure if the address translation functions I'm using in QEMU\n> >> will use the per-vCPU TLB cache or always traverse the page table).\n> \n> > That was my concern, I'd rather just perform the read once, that is, the read(s)\n> > done by ops->insn_translate.\n> \n> If your concern is on performance, that should not be an issue, since you'd be\n> using the memory peek functions at translation-time. Furthermore, since others\n> suggested having memory peek anyway, that's a nicer way (to me) to compose APIs\n> (and is less complex to implement).\n\nMy concern was the same as yours, correctness -- what happens if something\nchanges between the two reads? Because the two reads should always return\nthe same thing.\n\n> > I see. I implemented what I suggested above, i.e. tb_trans_cb\n> > (i.e. post_trans) passes an opaque descriptor of the TB (which can\n> > be iterated over insn by insn) and the return value (void *) of this\n> > cb will be passed by tb_exec_cb (i.e. pre_exec). Perf-wise this\n> > is pretty OK, turns out even if we don't end up caring about the\n> > TB, the additional per-TB helper (which might not end up calling\n> > a callback) does not introduce significant overhead at execution time.\n> \n> So you build this structure after translating the whole TB, and the user can\n> iterate it to check the translated instructions. This is closer to other\n> existing tools: you iterate the structure and then decide which/how to\n> instrument instructions, memory accesses, etc within it.\n\nCorrect. I suspect they went with this design because it makes sense to\ndo this preprocessing once, instead of having each plugin do it\nthemselves. I'm not sure how much we should care about supporting multiple\nplugins, but the impression I get from DynamoRIO is that it seems important\nto users.\n\n> My only concern is that this is much more complex than the simpler API I propose\n> (you must build the informational structures, generate calls to every possible\n> instrumentation call, which will be optimized-out by TCG if the user decides not\n> to use them, and overall pay in performance for any unused functionality),\n> whereas your approach can be implemented on top of it.\n\nIt's pretty simple; tr_ops->translate_insn has to copy each insn.\nFor instance, on aarch64 (disas_a64 is called from tr_translate_insn):\n\n-static void disas_a64_insn(CPUARMState *env, DisasContext *s)\n+static void disas_a64_insn(CPUARMState *env, DisasContext *s, struct qemu_plugin_insn *q_insn)\n {\n uint32_t insn;\n\n insn = arm_ldl_code(env, s->pc, s->sctlr_b);\n+ if (q_insn) {\n+ qemu_plugin_insn_append(q_insn, &insn, sizeof(insn));\n+ }\n\nIt takes some memory though (we duplicate the guest code), but perf-wise this\nisn't a big deal (an empty callback on every TB execution incurs only a 10-15%\nperf slowdown).\n\nI don't understand the part where you say that the instrumentation call can\nbe optimized out. Is there a special value of a \"TCG promise\" (at tb_trans_post\ntime) that removes the previously generated callback (at tb_trans_pre time)?\nOtherwise I don't see how selective TB instrumentation can work at tb_trans_pre\ntime.\n\nThanks,\n\n\t\tE.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org; dkim=pass (1024-bit key;\n\tunprotected) header.d=braap.org header.i=@braap.org\n\theader.b=\"tjGDZSuI\"; dkim=pass (2048-bit key;\n\tunprotected) header.d=messagingengine.com\n\theader.i=@messagingengine.com header.b=\"gvxRrXMU\"; \n\tdkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y7y8h0Vfgz9t41\n\tfor ;\n\tSat, 7 Oct 2017 05:00:02 +1100 (AEDT)","from localhost ([::1]:46630 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e0Wul-0004OV-UR\n\tfor incoming@patchwork.ozlabs.org; Fri, 06 Oct 2017 13:59:59 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:46084)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e0WuM-0004Ny-4g\n\tfor qemu-devel@nongnu.org; Fri, 06 Oct 2017 13:59:35 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e0WuJ-0003Gn-3w\n\tfor qemu-devel@nongnu.org; Fri, 06 Oct 2017 13:59:34 -0400","from out1-smtp.messagingengine.com ([66.111.4.25]:36437)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1e0WuI-0003GV-UF\n\tfor qemu-devel@nongnu.org; Fri, 06 Oct 2017 13:59:31 -0400","from compute4.internal (compute4.nyi.internal [10.202.2.44])\n\tby mailout.nyi.internal (Postfix) with ESMTP id 2AA9720BB7;\n\tFri, 6 Oct 2017 13:59:30 -0400 (EDT)","from frontend2 ([10.202.2.161])\n\tby compute4.internal (MEProxy); Fri, 06 Oct 2017 13:59:30 -0400","from localhost (flamenco.cs.columbia.edu [128.59.20.216])\n\tby mail.messagingengine.com (Postfix) with ESMTPA id CFECC24009;\n\tFri, 6 Oct 2017 13:59:29 -0400 (EDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h=\n\tcontent-transfer-encoding:content-type:date:from:in-reply-to\n\t:message-id:mime-version:references:subject:to:x-me-sender\n\t:x-me-sender:x-sasl-enc:x-sasl-enc; s=mesmtp; bh=j83bgp94/7dKOLP\n\tb3l3AffF00hDu/bzDok6zomTc+O0=; b=tjGDZSuI3vmcTpnt2Jk9NesbzHmvwhD\n\tyMr62OkSgs28/lUWiJfj38IXBQ7zTThU/Cx4MjgevmFY8C+eJAxc/DJ7Yoz5SqQm\n\tdBYApNk4iyoltfGmRMpdQ3Xg4yzk71DQzVE4C6PdjctnREg9NJXz6BUoyFnrBZU9\n\tZpzjtjGPojk8=","v=1; a=rsa-sha256; c=relaxed/relaxed; d=\n\tmessagingengine.com; h=content-transfer-encoding:content-type\n\t:date:from:in-reply-to:message-id:mime-version:references\n\t:subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s=\n\tfm1; bh=j83bgp94/7dKOLPb3l3AffF00hDu/bzDok6zomTc+O0=; b=gvxRrXMU\n\tvDARv/62PKDjBr+TG7zIVbY6rvxjOgRKTIZAE7V1RiMQgV/t0r715fHyXjv32Fjk\n\tbUcLTd4vjfSoevmj4i175L6Dw/5QW5QQ8GmQG9KTUjDKUTciR8BpfbuwBVXNFOR0\n\tqv4CjK+M33Wzs6/AjNEmBLoiDuf5zEImNQKsQSW8rAN4M1CLP9EHnZxsbei9eZs2\n\t63/pDfcGo2XAdO+fH1pxiNB7c6FQGJAqMCIJzA3eBxccTK6ODPH3c4SrcnD/MhSw\n\tjScFAoVwD2qmzwLs1T3o3vTtpOOEXj5qMuklV75AEkrDHtnDXcBhPO1NXx9ZUdz7\n\t2VhLmpm3YzJ/NQ=="],"X-ME-Sender":"","X-Sasl-enc":"ALt092jwy6n8g+q2o3WvtabDYngN4/Qq6k6hKAtuz1BY 1507312769","Date":"Fri, 6 Oct 2017 13:59:29 -0400","From":"\"Emilio G. Cota\" ","To":"Peter Maydell ,\n\tQEMU Developers ,\n\tStefan Hajnoczi ,\n\tMarkus Armbruster ","Message-ID":"<20171006175929.GA28281@flamenco>","References":"<87d16o53xr.fsf@frigg.lan>\n\t\n\t<87o9pywt8k.fsf@frigg.lan> <87shf5zlty.fsf@frigg.lan>\n\t<20170929175943.GA25038@flamenco> <87vak1w53a.fsf@frigg.lan>\n\t<20170930180941.GA22048@flamenco> <8760bu333n.fsf@frigg.lan>\n\t<20171005005043.GA20425@flamenco> <87vajsl3h7.fsf@frigg.lan>","MIME-Version":"1.0","Content-Type":"text/plain; charset=iso-8859-1","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<87vajsl3h7.fsf@frigg.lan>","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"66.111.4.25","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1787043,"web_url":"http://patchwork.ozlabs.org/comment/1787043/","msgid":"<87tvz08jc3.fsf@frigg.lan>","list_archive_url":null,"date":"2017-10-15T16:30:20","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":9099,"url":"http://patchwork.ozlabs.org/api/people/9099/","name":"Lluís Vilanova","email":"vilanova@ac.upc.edu"},"content":"Emilio G Cota writes:\n\n> On Fri, Oct 06, 2017 at 18:07:16 +0300, Lluís Vilanova wrote:\n>> Emilio G Cota writes:\n>> > On Thu, Oct 05, 2017 at 02:28:12 +0300, Lluís Vilanova wrote:\n>> >> The API takes care of telling you if the access could be performed\n>> >> successfully. If you access the instruction's memory representation at\n>> >> translation time, it should be able to perform the access, since QEMU's\n>> >> translation loop just had to do so in order to access that instruction (I should\n>> >> check what happens in the corner case where another guest CPU changes the page\n>> >> table, since I'm not sure if the address translation functions I'm using in QEMU\n>> >> will use the per-vCPU TLB cache or always traverse the page table).\n>> \n>> > That was my concern, I'd rather just perform the read once, that is, the read(s)\n>> > done by ops->insn_translate.\n>> \n>> If your concern is on performance, that should not be an issue, since you'd be\n>> using the memory peek functions at translation-time. Furthermore, since others\n>> suggested having memory peek anyway, that's a nicer way (to me) to compose APIs\n>> (and is less complex to implement).\n\n> My concern was the same as yours, correctness -- what happens if something\n> changes between the two reads? Because the two reads should always return\n> the same thing.\n\nThinking about it, shouldn't this always be the same given QEMU's TLB/page table\nconsistency assurances? Otherwise, QEMU could read bytes from different physical\npages while translating an instruction from the same virtual page.\n\nTherefore, this leads me to believe it is safe to use the memory read operations\nduring translation to let instrumentation libraries know what exactly they are\ndealing with.\n\n\n\n>> > I see. I implemented what I suggested above, i.e. tb_trans_cb\n>> > (i.e. post_trans) passes an opaque descriptor of the TB (which can\n>> > be iterated over insn by insn) and the return value (void *) of this\n>> > cb will be passed by tb_exec_cb (i.e. pre_exec). Perf-wise this\n>> > is pretty OK, turns out even if we don't end up caring about the\n>> > TB, the additional per-TB helper (which might not end up calling\n>> > a callback) does not introduce significant overhead at execution time.\n>> \n>> So you build this structure after translating the whole TB, and the user can\n>> iterate it to check the translated instructions. This is closer to other\n>> existing tools: you iterate the structure and then decide which/how to\n>> instrument instructions, memory accesses, etc within it.\n\n> Correct. I suspect they went with this design because it makes sense to\n> do this preprocessing once, instead of having each plugin do it\n> themselves. I'm not sure how much we should care about supporting multiple\n> plugins, but the impression I get from DynamoRIO is that it seems important\n> to users.\n\nIf that can be built into a \"helper\" instrumentation library / header that\nothers can use, I would rather keep this functionality outside QEMU.\n\n\n>> My only concern is that this is much more complex than the simpler API I propose\n>> (you must build the informational structures, generate calls to every possible\n>> instrumentation call, which will be optimized-out by TCG if the user decides not\n>> to use them, and overall pay in performance for any unused functionality),\n>> whereas your approach can be implemented on top of it.\n\n> It's pretty simple; tr_ops->translate_insn has to copy each insn.\n> For instance, on aarch64 (disas_a64 is called from tr_translate_insn):\n\n> -static void disas_a64_insn(CPUARMState *env, DisasContext *s)\n> +static void disas_a64_insn(CPUARMState *env, DisasContext *s, struct qemu_plugin_insn *q_insn)\n> {\n> uint32_t insn;\n\n> insn = arm_ldl_code(env, s->pc, s->sctlr_b);\n> + if (q_insn) {\n> + qemu_plugin_insn_append(q_insn, &insn, sizeof(insn));\n> + }\n\n> It takes some memory though (we duplicate the guest code), but perf-wise this\n> isn't a big deal (an empty callback on every TB execution incurs only a 10-15%\n> perf slowdown).\n\n> I don't understand the part where you say that the instrumentation call can\n> be optimized out. Is there a special value of a \"TCG promise\" (at tb_trans_post\n> time) that removes the previously generated callback (at tb_trans_pre time)?\n> Otherwise I don't see how selective TB instrumentation can work at tb_trans_pre\n> time.\n\nWith the approach you propose, the instrumentation library is only called at\ntranslation-time after the whole TB has been generated. If the instrumentor now\ndecides to instrument each instruction, this means QEMU must inject an\ninstrumentation call to every instruction while translating the TB *just in\ncase* the instrumentor will need it later. In case the instrumentor decides some\ninstructions don't need to be instrumented, now QEMU needs to eliminate them\nfrom the TB before generating the host code (in order to eliminate unnecessary\noverheads).\n\nHope it's clearer now.\n\n\nThanks!\n Lluis","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yFRql6tJ3z9t3B\n\tfor ;\n\tMon, 16 Oct 2017 03:34:27 +1100 (AEDT)","from localhost ([::1]:57821 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e3lru-0007uU-4N\n\tfor incoming@patchwork.ozlabs.org; Sun, 15 Oct 2017 12:34:26 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:34612)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e3loG-0005kM-85\n\tfor qemu-devel@nongnu.org; Sun, 15 Oct 2017 12:30:41 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e3loB-0000lJ-9r\n\tfor qemu-devel@nongnu.org; Sun, 15 Oct 2017 12:30:40 -0400","from roura.ac.upc.edu ([147.83.33.10]:35069 helo=roura.ac.upc.es)\n\tby eggs.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e3loA-0000jp-Ra\n\tfor qemu-devel@nongnu.org; Sun, 15 Oct 2017 12:30:35 -0400","from correu-2.ac.upc.es (correu-2.ac.upc.es [147.83.30.92])\n\tby roura.ac.upc.es (8.13.8/8.13.8) with ESMTP id v9FGUQ0T016768;\n\tSun, 15 Oct 2017 18:30:27 +0200","from localhost (unknown [31.210.187.58])\n\tby correu-2.ac.upc.es (Postfix) with ESMTPSA id 5922A155;\n\tSun, 15 Oct 2017 18:30:21 +0200 (CEST)"],"From":"=?utf-8?q?Llu=C3=ADs_Vilanova?= ","To":"\"Emilio G. Cota\" ","References":"<87d16o53xr.fsf@frigg.lan>\n\t\n\t<87o9pywt8k.fsf@frigg.lan> <87shf5zlty.fsf@frigg.lan>\n\t<20170929175943.GA25038@flamenco> <87vak1w53a.fsf@frigg.lan>\n\t<20170930180941.GA22048@flamenco> <8760bu333n.fsf@frigg.lan>\n\t<20171005005043.GA20425@flamenco> <87vajsl3h7.fsf@frigg.lan>\n\t<20171006175929.GA28281@flamenco>","Mail-Followup-To":"\"Emilio G. Cota\" , Peter Maydell\n\t, QEMU Developers , \n\tStefan Hajnoczi , Markus Armbruster\n\t","Date":"Sun, 15 Oct 2017 19:30:20 +0300","In-Reply-To":"<20171006175929.GA28281@flamenco> (Emilio G. Cota's message of\n\t\"Fri, 6 Oct 2017 13:59:29 -0400\")","Message-ID":"<87tvz08jc3.fsf@frigg.lan>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux)","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.6.x [fuzzy]","X-Received-From":"147.83.33.10","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Peter Maydell ,\n\tQEMU Developers ,\n\tStefan Hajnoczi ,\n\tMarkus Armbruster ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1787046,"web_url":"http://patchwork.ozlabs.org/comment/1787046/","msgid":"","list_archive_url":null,"date":"2017-10-15T16:47:21","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":5111,"url":"http://patchwork.ozlabs.org/api/people/5111/","name":"Peter Maydell","email":"peter.maydell@linaro.org"},"content":"On 15 October 2017 at 17:30, Lluís Vilanova wrote:\n> Thinking about it, shouldn't this always be the same given QEMU's TLB/page table\n> consistency assurances?\n\nWhat TLB/page table consistency assurances? For ARM at least\nwe will only update (ie flush) the TLB when the guest next\nexecutes a relevant TLB maintenance instruction. So a\nmisbehaving guest can set things up so the page table\nis completely different from what's in QEMU's TLB if it\nwants. This all falls in the realms of architecturally\nunpredictable behaviour for the guest -- whether you\nwant the instrumentation to be confused as well is a\ndifferent question...\n\nthanks\n-- PMM","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org; dkim=pass (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"LNlKGGQG\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yFS7x2FXCz9t3B\n\tfor ;\n\tMon, 16 Oct 2017 03:48:29 +1100 (AEDT)","from localhost ([::1]:57851 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e3m5T-0001og-Dk\n\tfor incoming@patchwork.ozlabs.org; Sun, 15 Oct 2017 12:48:27 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:37401)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e3m4m-0001nU-Mz\n\tfor qemu-devel@nongnu.org; Sun, 15 Oct 2017 12:47:47 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e3m4l-0002kv-UH\n\tfor qemu-devel@nongnu.org; Sun, 15 Oct 2017 12:47:44 -0400","from mail-wm0-x22a.google.com ([2a00:1450:400c:c09::22a]:45980)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1e3m4l-0002kO-Nb\n\tfor qemu-devel@nongnu.org; Sun, 15 Oct 2017 12:47:43 -0400","by mail-wm0-x22a.google.com with SMTP id q124so29503618wmb.0\n\tfor ; Sun, 15 Oct 2017 09:47:43 -0700 (PDT)","by 10.223.139.195 with HTTP; Sun, 15 Oct 2017 09:47:21 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:content-transfer-encoding;\n\tbh=BU4q+lp4kCug5OiDQ1GR/bfFgqMyC9r7/eC6c9vp/uw=;\n\tb=LNlKGGQGiXwKj0iLYVzqNHHx5kmLSgyQjn4iQoAA2vF3dSIAcowFjnxYjSuuw67BjS\n\t4BWt5idNy0n7NOb82ZuUFVVqwhxB35PrjYFxrHwq5X7K64SGnrsrfI+avbB/zZjWplTH\n\tEnxaCv2QyHCN7u/bWOshFwWdFNOisxHeFBHp0=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:content-transfer-encoding;\n\tbh=BU4q+lp4kCug5OiDQ1GR/bfFgqMyC9r7/eC6c9vp/uw=;\n\tb=rpeUxwgM17skboOjXbc+w7Z3B3NYnhx9gImci/TGUQxViYPxJs2lBD2YttcjkAU1Ss\n\thA1ntXp+f+ay1TzQwUgZUy81CoDDcCQ6V7nX9ZShkddFgidxPak5HOJ/gSQdRsWHEKoL\n\tqX8ByDPGtTMO6n7qZDsivglRLFq89lKJRiUmul+NsCkb1wpVi9xMDjZS0lXlVVz8LTb+\n\tx5I1q7gpnFohdOy8PTXDP2eEHqhkbRPbweW9lDe7YvELq/rX2I2NKP7kKxSWNpW2V/n0\n\tnxqMBHBmpa8UrJXSCpkxLLMDI3u7fOqluXv8gGFOMa89dE7MDBGWQzma1gAO3sMQJly7\n\tExoA==","X-Gm-Message-State":"AMCzsaUUKKL4Xg7XD2DQDEvf05gx3Wk4FCXMO4TFNsJrgIU4U1l6a/+n\n\tSWZJJCpcGVuLux0HgCj/yS3atDP0aR4LS8Pc0SdBCQ==","X-Google-Smtp-Source":"ABhQp+R6UZhKIlPmey3/dO2dZN6sJcstks4h5UrpxCqJgg4k5kT9qbMeTDQHWlcGmdTO6pw6xj4Z+sO78iLMEDMYPgo=","X-Received":"by 10.28.23.3 with SMTP id 3mr5346234wmx.62.1508086062536;\n\tSun, 15 Oct 2017 09:47:42 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<87tvz08jc3.fsf@frigg.lan>","References":"<87d16o53xr.fsf@frigg.lan>\n\t\n\t<87o9pywt8k.fsf@frigg.lan> <87shf5zlty.fsf@frigg.lan>\n\t<20170929175943.GA25038@flamenco>\n\t<87vak1w53a.fsf@frigg.lan> <20170930180941.GA22048@flamenco>\n\t<8760bu333n.fsf@frigg.lan> <20171005005043.GA20425@flamenco>\n\t<87vajsl3h7.fsf@frigg.lan> <20171006175929.GA28281@flamenco>\n\t<87tvz08jc3.fsf@frigg.lan>","From":"Peter Maydell ","Date":"Sun, 15 Oct 2017 17:47:21 +0100","Message-ID":"","To":"\"Emilio G. Cota\" ,\n\tPeter Maydell , \n\tQEMU Developers ,\n\tStefan Hajnoczi , \n\tMarkus Armbruster ","Content-Type":"text/plain; charset=\"UTF-8\"","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c09::22a","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1791931,"web_url":"http://patchwork.ozlabs.org/comment/1791931/","msgid":"<877evo8ukl.fsf@frigg.lan>","list_archive_url":null,"date":"2017-10-21T14:05:46","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":9099,"url":"http://patchwork.ozlabs.org/api/people/9099/","name":"Lluís Vilanova","email":"vilanova@ac.upc.edu"},"content":"Peter Maydell writes:\n\n> On 15 October 2017 at 17:30, Lluís Vilanova wrote:\n>> Thinking about it, shouldn't this always be the same given QEMU's TLB/page table\n>> consistency assurances?\n\n> What TLB/page table consistency assurances? For ARM at least\n> we will only update (ie flush) the TLB when the guest next\n> executes a relevant TLB maintenance instruction. So a\n> misbehaving guest can set things up so the page table\n> is completely different from what's in QEMU's TLB if it\n> wants. This all falls in the realms of architecturally\n> unpredictable behaviour for the guest -- whether you\n> want the instrumentation to be confused as well is a\n> different question...\n\nI meant that if the contents of a virtual memory page change while QEMU is\ntranslating an instruction, it must be able to detect that and act accordingly\nfor correctness.\n\nHaving that in mind, the same should hold true when an instrumentor reads a\npage's contents during translation (e.g., to gather information on opcodes).\n\n\nCheers,\n Lluis","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yK4GR5Kc0z9t3C\n\tfor ;\n\tSun, 22 Oct 2017 01:06:38 +1100 (AEDT)","from localhost ([::1]:57952 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e5uQ6-0002LK-AT\n\tfor incoming@patchwork.ozlabs.org; Sat, 21 Oct 2017 10:06:34 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:33861)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e5uPh-0002LF-05\n\tfor qemu-devel@nongnu.org; Sat, 21 Oct 2017 10:06:09 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e5uPd-0005R5-QJ\n\tfor qemu-devel@nongnu.org; Sat, 21 Oct 2017 10:06:08 -0400","from roura.ac.upc.es ([147.83.33.10]:42483)\n\tby eggs.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e5uPd-0005QP-Fm\n\tfor qemu-devel@nongnu.org; Sat, 21 Oct 2017 10:06:05 -0400","from correu-1.ac.upc.es (correu-1.ac.upc.es [147.83.30.91])\n\tby roura.ac.upc.es (8.13.8/8.13.8) with ESMTP id v9LE5sj5012277;\n\tSat, 21 Oct 2017 16:05:54 +0200","from localhost (unknown [31.210.187.58])\n\tby correu-1.ac.upc.es (Postfix) with ESMTPSA id 72F4821D;\n\tSat, 21 Oct 2017 16:05:48 +0200 (CEST)"],"From":"=?utf-8?q?Llu=C3=ADs_Vilanova?= ","To":"Peter Maydell ","References":"<87d16o53xr.fsf@frigg.lan>\n\t\n\t<87o9pywt8k.fsf@frigg.lan> <87shf5zlty.fsf@frigg.lan>\n\t<20170929175943.GA25038@flamenco> <87vak1w53a.fsf@frigg.lan>\n\t<20170930180941.GA22048@flamenco> <8760bu333n.fsf@frigg.lan>\n\t<20171005005043.GA20425@flamenco> <87vajsl3h7.fsf@frigg.lan>\n\t<20171006175929.GA28281@flamenco> <87tvz08jc3.fsf@frigg.lan>\n\t","Mail-Followup-To":"Peter Maydell , \"Emilio G. Cota\"\n\t, QEMU Developers ,\n\tStefan\n\tHajnoczi , Markus Armbruster ","Date":"Sat, 21 Oct 2017 17:05:46 +0300","In-Reply-To":"\n\t(Peter Maydell's message of \"Sun, 15 Oct 2017 17:47:21 +0100\")","Message-ID":"<877evo8ukl.fsf@frigg.lan>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux)","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.6.x [fuzzy]","X-Received-From":"147.83.33.10","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"\"Emilio G. Cota\" ,\n\tQEMU Developers , \n\tStefan Hajnoczi ,\n\tMarkus Armbruster ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1791969,"web_url":"http://patchwork.ozlabs.org/comment/1791969/","msgid":"","list_archive_url":null,"date":"2017-10-21T16:56:22","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":5111,"url":"http://patchwork.ozlabs.org/api/people/5111/","name":"Peter Maydell","email":"peter.maydell@linaro.org"},"content":"On 21 October 2017 at 15:05, Lluís Vilanova wrote:\n> Peter Maydell writes:\n>\n>> On 15 October 2017 at 17:30, Lluís Vilanova wrote:\n>>> Thinking about it, shouldn't this always be the same given QEMU's TLB/page table\n>>> consistency assurances?\n>\n>> What TLB/page table consistency assurances? For ARM at least\n>> we will only update (ie flush) the TLB when the guest next\n>> executes a relevant TLB maintenance instruction. So a\n>> misbehaving guest can set things up so the page table\n>> is completely different from what's in QEMU's TLB if it\n>> wants. This all falls in the realms of architecturally\n>> unpredictable behaviour for the guest -- whether you\n>> want the instrumentation to be confused as well is a\n>> different question...\n>\n> I meant that if the contents of a virtual memory page change while QEMU is\n> translating an instruction, it must be able to detect that and act accordingly\n> for correctness.\n\nThat's an interesting corner case, actually. Traditionally\nit simply couldn't happen because we were strictly single\nthreaded and so if we were translating then we weren't\nrunning guest code. We did need to handle \"writes mean we\nmust invalidate an already produced translation\", but not\n\"invalidate one we're halfway through and haven't put in\nour data structures yet\". Did we get that right in the MTTCG\ndesign? How does it work?\n\n(Did we produce a summary of the MTTCG design anywhere?\nI didn't follow the development in detail as it was going\non, but it would be useful to understand the final result.)\n\nIn any case, the only assurance we provide over QEMU as a\nwhole is that if the guest writes to a physical address then\nwe don't keep hold of a now-duff translation for that physaddr.\nWe don't guarantee the same thing for guest changes of\nthe vaddr-to-physaddr mapping -- instead we let the target\nspecific code deal with this by invalidating QEMU's TLB\nwhen the guest code does TLB invalidate ops.\n\n> Having that in mind, the same should hold true when an instrumentor reads a\n> page's contents during translation (e.g., to gather information on opcodes).\n\nBasically I don't think we actually have very strong\nguarantees here, and that's another reason for not\nproviding instrumentation callbacks at translate time.\n\nthanks\n-- PMM","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org; dkim=pass (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"f29WmHzJ\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yK83K5lbNz9sCZ\n\tfor ;\n\tSun, 22 Oct 2017 03:57:16 +1100 (AEDT)","from localhost ([::1]:58528 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e5x5G-0004i7-3n\n\tfor incoming@patchwork.ozlabs.org; Sat, 21 Oct 2017 12:57:14 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:32773)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e5x4p-0004hp-F9\n\tfor qemu-devel@nongnu.org; Sat, 21 Oct 2017 12:56:48 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e5x4o-0008A5-Fg\n\tfor qemu-devel@nongnu.org; Sat, 21 Oct 2017 12:56:47 -0400","from mail-wr0-x22f.google.com ([2a00:1450:400c:c0c::22f]:49721)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1e5x4o-000895-8o\n\tfor qemu-devel@nongnu.org; Sat, 21 Oct 2017 12:56:46 -0400","by mail-wr0-x22f.google.com with SMTP id g90so13677127wrd.6\n\tfor ; Sat, 21 Oct 2017 09:56:44 -0700 (PDT)","by 10.223.161.5 with HTTP; Sat, 21 Oct 2017 09:56:22 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:content-transfer-encoding;\n\tbh=B7II1XIb3KvYUGl7XzfCmopOE4WD9L+mlbhd6NZJMh8=;\n\tb=f29WmHzJvwbqqpa84lPE6dYhfaOhkMQYYeXxhAN+veBlrSFNUERJfu94AZMSxA1qgm\n\tujqYNYxS4J4s9glt/YF6rKgg1QORwEIw7AF0U0TSpK07sbOv9zOjYDioLLT4MRHPtADK\n\ttgoXGYvtN2YqLI7QYQlZvYu5PpqauS4geiNqA=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:content-transfer-encoding;\n\tbh=B7II1XIb3KvYUGl7XzfCmopOE4WD9L+mlbhd6NZJMh8=;\n\tb=mF+1zwZTDgbRf/Zjwo52IIaBoam3r0bf9thdRhlEbwV9kgB5dUmCiuLizKKsPtzgNw\n\tCcCkpv8QTMNlYa5PRbeX1zujQTuRJlDTqhvVEpyuEb+Ij2hN+porETzC9GPRwQqgv/QH\n\tcVEeb0mGqPfBWe8ZcNv4KjGMVBNAlrUE/7NiJ1PYumKAzqlkDjRCvKb/swvtg+W9ekVq\n\tb05UjeqSfvyo10mY3Oe41TGJxvPwlAAi1FKZCdU8sxUN06GvqtCBXWHT9hy/M1ktOd++\n\tMwof8eC08WGMX8fcoF3gkqn9ZDpG+Ji9C84wuRP0ECx0uYigTBr0OmFQrRL2HWheb5yi\n\t0PJw==","X-Gm-Message-State":"AMCzsaWNvvrdNDPJYVTAUx56jxAG3CRqp83Icw5knR/5ta5mce/S4Tzr\n\tZuSJqlmXr5QrnmZijzMrhez2YqpFez3Ere3SGn22DQ==","X-Google-Smtp-Source":"ABhQp+TAGEWv3IfC2od8hRNoFpcnB0hY/78zg7Xzs2J2xpJAXOoDIbIHSoquW2jaWIn0wnCnkV6crdqpUqZI04VgToY=","X-Received":"by 10.223.157.11 with SMTP id k11mr1435548wre.281.1508605003568; \n\tSat, 21 Oct 2017 09:56:43 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<877evo8ukl.fsf@frigg.lan>","References":"<87d16o53xr.fsf@frigg.lan>\n\t\n\t<87o9pywt8k.fsf@frigg.lan> <87shf5zlty.fsf@frigg.lan>\n\t<20170929175943.GA25038@flamenco>\n\t<87vak1w53a.fsf@frigg.lan> <20170930180941.GA22048@flamenco>\n\t<8760bu333n.fsf@frigg.lan> <20171005005043.GA20425@flamenco>\n\t<87vajsl3h7.fsf@frigg.lan> <20171006175929.GA28281@flamenco>\n\t<87tvz08jc3.fsf@frigg.lan>\n\t\n\t<877evo8ukl.fsf@frigg.lan>","From":"Peter Maydell ","Date":"Sat, 21 Oct 2017 17:56:22 +0100","Message-ID":"","To":"Peter Maydell ,\n\t\"Emilio G. Cota\" , \n\tQEMU Developers ,\n\tStefan Hajnoczi , \n\tMarkus Armbruster ","Content-Type":"text/plain; charset=\"UTF-8\"","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c0c::22f","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1791972,"web_url":"http://patchwork.ozlabs.org/comment/1791972/","msgid":"<877evojugq.fsf@linaro.org>","list_archive_url":null,"date":"2017-10-21T17:12:37","subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","submitter":{"id":39532,"url":"http://patchwork.ozlabs.org/api/people/39532/","name":"Alex Bennée","email":"alex.bennee@linaro.org"},"content":"Peter Maydell writes:\n\n> On 21 October 2017 at 15:05, Lluís Vilanova wrote:\n>> Peter Maydell writes:\n>>\n>>> On 15 October 2017 at 17:30, Lluís Vilanova wrote:\n>>>> Thinking about it, shouldn't this always be the same given QEMU's TLB/page table\n>>>> consistency assurances?\n>>\n>>> What TLB/page table consistency assurances? For ARM at least\n>>> we will only update (ie flush) the TLB when the guest next\n>>> executes a relevant TLB maintenance instruction. So a\n>>> misbehaving guest can set things up so the page table\n>>> is completely different from what's in QEMU's TLB if it\n>>> wants. This all falls in the realms of architecturally\n>>> unpredictable behaviour for the guest -- whether you\n>>> want the instrumentation to be confused as well is a\n>>> different question...\n>>\n>> I meant that if the contents of a virtual memory page change while QEMU is\n>> translating an instruction, it must be able to detect that and act accordingly\n>> for correctness.\n>\n> That's an interesting corner case, actually. Traditionally\n> it simply couldn't happen because we were strictly single\n> threaded and so if we were translating then we weren't\n> running guest code. We did need to handle \"writes mean we\n> must invalidate an already produced translation\", but not\n> \"invalidate one we're halfway through and haven't put in\n> our data structures yet\". Did we get that right in the MTTCG\n> design? How does it work?\n\nIt's currently protected by locks, as you need to grab tb_lock/mmap_lock\nto call:\n\n void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end,\n int is_cpu_write_access)\n\nSo no new blocks can be created until you've complete your invalidation\n- or you are serialised until the block currently being translated is\ncompleted. At which point the block will be immediately marked as\ninvalid and not be called again.\n\n>\n> (Did we produce a summary of the MTTCG design anywhere?\n> I didn't follow the development in detail as it was going\n> on, but it would be useful to understand the final result.)\n\nSure, it's in:\n\n docs/devel/multi-thread-tcg.txt\n\n>\n> In any case, the only assurance we provide over QEMU as a\n> whole is that if the guest writes to a physical address then\n> we don't keep hold of a now-duff translation for that physaddr.\n> We don't guarantee the same thing for guest changes of\n> the vaddr-to-physaddr mapping -- instead we let the target\n> specific code deal with this by invalidating QEMU's TLB\n> when the guest code does TLB invalidate ops.\n>\n>> Having that in mind, the same should hold true when an instrumentor reads a\n>> page's contents during translation (e.g., to gather information on opcodes).\n>\n> Basically I don't think we actually have very strong\n> guarantees here, and that's another reason for not\n> providing instrumentation callbacks at translate time.\n>\n> thanks\n> -- PMM\n\n\n--\nAlex Bennée","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org; dkim=pass (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"Icf+tH6x\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yK8Pm3t92z9t2S\n\tfor ;\n\tSun, 22 Oct 2017 04:13:15 +1100 (AEDT)","from localhost ([::1]:58565 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e5xKg-0007KU-Rj\n\tfor incoming@patchwork.ozlabs.org; Sat, 21 Oct 2017 13:13:10 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:34931)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e5xKI-0007KP-6f\n\tfor qemu-devel@nongnu.org; Sat, 21 Oct 2017 13:12:47 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e5xKE-0007gA-2w\n\tfor qemu-devel@nongnu.org; Sat, 21 Oct 2017 13:12:46 -0400","from mail-wr0-x22f.google.com ([2a00:1450:400c:c0c::22f]:44885)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1e5xKD-0007ey-Sh\n\tfor qemu-devel@nongnu.org; Sat, 21 Oct 2017 13:12:42 -0400","by mail-wr0-x22f.google.com with SMTP id z55so7644301wrz.1\n\tfor ; Sat, 21 Oct 2017 10:12:40 -0700 (PDT)","from zen.linaro.local ([81.128.185.34])\n\tby smtp.gmail.com with ESMTPSA id\n\tm25sm1369795wmi.7.2017.10.21.10.12.38\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tSat, 21 Oct 2017 10:12:38 -0700 (PDT)","from zen (localhost [127.0.0.1])\n\tby zen.linaro.local (Postfix) with ESMTPS id 981A63E009F;\n\tSat, 21 Oct 2017 18:12:37 +0100 (BST)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=references:user-agent:from:to:cc:subject:in-reply-to:date\n\t:message-id:mime-version:content-transfer-encoding;\n\tbh=G2JeSqBAnlgWIH6V8khkE6jzsVdcRg7ZwuS/fbvUXOc=;\n\tb=Icf+tH6xTwWYaJnRddPpE7Eoja9jTnSwPMMbHP2fYxpYDU5gB+yJWfbSDx6nsmq9Es\n\ts3lzuqQqgM4PwVU9eGsDDySCcJsaq5L6hzyhTgQw8MN2xtYApr7uczKa2Z6cOvya4Oyz\n\tuLpPRwRaNFs167/0vxWGADffh41VDV3jresDo=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:references:user-agent:from:to:cc:subject\n\t:in-reply-to:date:message-id:mime-version:content-transfer-encoding; \n\tbh=G2JeSqBAnlgWIH6V8khkE6jzsVdcRg7ZwuS/fbvUXOc=;\n\tb=Scp6YpF5MEwk17aEJu8ClODjOCaeXYbD0IG1bwjirY9+U5qWtcNiYBZ1mx/0GJmdpn\n\tFKFZoeXn90d2GUOnOEa4lOQ5s4H9zV8JksxLdrtJ1srVabWFwvBXeJuLJHUpECPtBNte\n\t+QKHkUXx3Gqde/sbcp4TAhQ0ZlouWBgQ2bcaBeVwxpiYBocRZfAO3JnGfXzluI8ca8mP\n\tXgMdwOklZm5Zzk6ik18HgfsDIMcd49a6CciIYvrmRqBkXEGSyDo2aOZ3HMAUTHF4a+QG\n\tRm1QpbyBMn7MJPmzYPQi/c6ZAbaKYPBK9aGB3MLleehG9xnFEAzJ+wD/6t31sKIPbDom\n\tzojw==","X-Gm-Message-State":"AMCzsaUMjeaYBi3fPjlXz3M5NHqSyfm291g0uAVxgE0PbL422LJk6nv5\n\tggGKI6oMVcBsGw9E/s3ByNWcSw==","X-Google-Smtp-Source":"ABhQp+T5POJH6r2VKhrQC7otjgb6IKXF/qSFlBjRn/Sht7y8rHNDzYqRK586XiU3Mg4O+4MqJofUSw==","X-Received":"by 10.223.146.101 with SMTP id 92mr6651428wrj.21.1508605959426; \n\tSat, 21 Oct 2017 10:12:39 -0700 (PDT)","References":"<87d16o53xr.fsf@frigg.lan>\n\t\n\t<87o9pywt8k.fsf@frigg.lan> <87shf5zlty.fsf@frigg.lan>\n\t<20170929175943.GA25038@flamenco> <87vak1w53a.fsf@frigg.lan>\n\t<20170930180941.GA22048@flamenco> <8760bu333n.fsf@frigg.lan>\n\t<20171005005043.GA20425@flamenco> <87vajsl3h7.fsf@frigg.lan>\n\t<20171006175929.GA28281@flamenco> <87tvz08jc3.fsf@frigg.lan>\n\t\n\t<877evo8ukl.fsf@frigg.lan>\n\t","User-agent":"mu4e 0.9.19; emacs 26.0.90","From":"Alex =?utf-8?q?Benn=C3=A9e?= ","To":"Peter Maydell ","In-reply-to":"","Date":"Sat, 21 Oct 2017 18:12:37 +0100","Message-ID":"<877evojugq.fsf@linaro.org>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c0c::22f","Subject":"Re: [Qemu-devel] [PATCH v6 01/22] instrument: Add documentation","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"\"Emilio G. Cota\" ,\n\tQEMU Developers , \n\tStefan Hajnoczi ,\n\tMarkus Armbruster ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}}]