From patchwork Tue Jan 10 10:44:46 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Harsh Prateek Bora X-Patchwork-Id: 135200 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [140.186.70.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 806FFB6FBA for ; Tue, 10 Jan 2012 21:45:10 +1100 (EST) Received: from localhost ([::1]:36847 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RkZCU-0005DT-Us for incoming@patchwork.ozlabs.org; Tue, 10 Jan 2012 05:45:06 -0500 Received: from eggs.gnu.org ([140.186.70.92]:44952) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RkZCK-0005Cl-VQ for qemu-devel@nongnu.org; Tue, 10 Jan 2012 05:45:01 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RkZCJ-0007ux-7z for qemu-devel@nongnu.org; Tue, 10 Jan 2012 05:44:56 -0500 Received: from e28smtp09.in.ibm.com ([122.248.162.9]:58270) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RkZCI-0007uh-AY for qemu-devel@nongnu.org; Tue, 10 Jan 2012 05:44:55 -0500 Received: from /spool/local by e28smtp09.in.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 10 Jan 2012 16:14:50 +0530 Received: from d28relay04.in.ibm.com (9.184.220.61) by e28smtp09.in.ibm.com (192.168.1.139) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 10 Jan 2012 16:14:49 +0530 Received: from d28av04.in.ibm.com (d28av04.in.ibm.com [9.184.220.66]) by d28relay04.in.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q0AAilmR4317312 for ; Tue, 10 Jan 2012 16:14:48 +0530 Received: from d28av04.in.ibm.com (loopback [127.0.0.1]) by d28av04.in.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q0AAikEv020381 for ; Tue, 10 Jan 2012 21:44:47 +1100 Received: from harshbora.in.ibm.com ([9.124.46.126]) by d28av04.in.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q0AAik8M020366; Tue, 10 Jan 2012 21:44:46 +1100 Message-ID: <4F0C169E.7080903@linux.vnet.ibm.com> Date: Tue, 10 Jan 2012 16:14:46 +0530 From: Harsh Bora User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.10) Gecko/20100621 Fedora/3.0.5-1.fc13 Thunderbird/3.0.5 MIME-Version: 1.0 To: Mathieu Desnoyers References: <1326123980-6038-1-git-send-email-harsh@linux.vnet.ibm.com> <20120109160105.GA2444@Krystal> <4F0B3E0E.3040505@linux.vnet.ibm.com> <20120110001453.GA8203@Krystal> <4F0BE091.3020704@linux.vnet.ibm.com> <20120110071701.GB785@Krystal> <4F0BFF97.6040508@linux.vnet.ibm.com> In-Reply-To: <4F0BFF97.6040508@linux.vnet.ibm.com> x-cbid: 12011010-2674-0000-0000-000002EAD2AD X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 122.248.162.9 Cc: stefanha@linux.vnet.ibm.com, qemu-devel@nongnu.org, lttng-dev@lists.lttng.org, Mathieu Desnoyers , aneesh.kumar@linux.vnet.ibm.com, vilanova@ac.upc.edu Subject: Re: [Qemu-devel] [lttng-dev] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings. X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org On 01/10/2012 02:36 PM, Harsh Bora wrote: > On 01/10/2012 12:47 PM, Mathieu Desnoyers wrote: >> * Harsh Bora (harsh@linux.vnet.ibm.com) wrote: >>> On 01/10/2012 05:44 AM, Mathieu Desnoyers wrote: >>>> * Harsh Bora (harsh@linux.vnet.ibm.com) wrote: >>>>> On 01/09/2012 09:31 PM, Mathieu Desnoyers wrote: >>>>>> * Harsh Prateek Bora (harsh@linux.vnet.ibm.com) wrote: >>>>>>> Existing simple trace can log upto 6 args per trace event and >>>>>>> does not >>>>>>> support strings in trace record format. Introducing new trace >>>>>>> format as >>>>>>> discussed earlier on list to support variable number/size of >>>>>>> arguments. >>>>>>> (Ref: >>>>>>> http://lists.gnu.org/archive/html/qemu-devel/2011-11/msg03426.html) >>>>>>> >>>>>>> Basic testing of this patch is successful. Stress testing not yet >>>>>>> done. >>>>>>> >>>>>>> Apply patches, then run: >>>>>>> >>>>>>> make distclean >>>>>>> ./configure with --enable-trace-backend=simple >>>>>>> make >>>>>>> sudo make install >>>>>>> >>>>>>> Sample tracelog showing strings support: >>>>>>> [harsh@harshbora v9fs]$ scripts/simpletrace.py trace-events >>>>>>> trace-23261 >>>>>>> v9fs_version 0.000 tag=65535 id=100 msize=8192 version=9P2000.L >>>>>>> v9fs_version_return 17.530 tag=65535 id=100 msize=8192 >>>>>>> version=9P2000.L >>>>>>> v9fs_attach 180.121 tag=1 id=104 fid=0 afid=18446744073709551615 >>>>>>> uname=nobody aname= >>>>>>> >>>>>>> >>>>>>> Note: LTTng ust backend is broken in upstream qemu, therefore >>>>>>> tracetool.py >>>>>>> doesnt support ust backend as of now. IIUC, ust's trace event >>>>>>> APIs are under >>>>>>> development and not yet stable. >>>>>> >>>>>> Hi, >>>>>> >>>>>> FYI, the LTTng-UST TRACEPOINT_EVENT API is very much stable as of >>>>>> now. >>>>>> Even though we are still in LTTng-UST 2.0 prereleases, the fact >>>>>> that we >>>>>> started the round of discussions on this API last summer makes us >>>>>> confident that from this point on we should not have to change it. >>>>>> >>>>>> Moreover, I would like to know if the old UST 0.x (0.16 is the >>>>>> latest) >>>>>> is broken wrt qemu, or if this is just for LTTng-2.0 UST support ? >>>>>> UST 0.x instrumentation is not supposed to have broken wrt qemu. >>>>>> >>>>> >>>>> Hi, >>>>> Thanks for an early response. I had tried building with ust 0.16 >>>>> and it >>>>> gives compilation errors, specially for trace events with 'void' >>>>> argument: >>>>> >>>>> CC osdep.o >>>>> In file included from osdep.c:49: >>>>> trace.h: In function ‘__trace_ust_slavio_misc_update_irq_raise’: >>>>> trace.h:277: error: ‘void’ must be the only parameter >>>>> trace.h:277: error: expected expression before ‘)’ token >>>>> trace.h:277: error: too many arguments to function ‘(void (*)(void >>>>> *))__tp_it_func’ >>>>> trace.h: At top level: >>>>> trace.h:277: error: ‘void’ must be the only parameter >>>>> trace.h:277: error: ‘void’ must be the only parameter >>>>> In file included from osdep.c:49: >>>>> trace.h: In function ‘__trace_ust_slavio_misc_update_irq_lower’: >>>>> trace.h:280: error: ‘void’ must be the only parameter >>>>> trace.h:280: error: expected expression before ‘)’ token >>>>> trace.h:280: error: too many arguments to function ‘(void (*)(void >>>>> *))__tp_it_func’ >>>>> >>>>> >>>>> I am not sure which interface is supposed to be used for void >>>>> arguments >>>>> in ust 0.16. >>>> >>>> Looking at scripts/tracetool: >>>> >>>> linetoh_ust() >>>> { >>>> local name args argnames >>>> name=$(get_name "$1") >>>> args=$(get_args "$1") >>>> argnames=$(get_argnames "$1", ",") >>>> >>>> cat<>>> DECLARE_TRACE(ust_$name, TP_PROTO($args), TP_ARGS($argnames)); >>>> #define trace_$name trace_ust_$name >>>> EOF >>>> } >>>> >>>> for those tracepoints with argument "void", DECLARE_TRACE_NOARGS should >>>> be used for UST 0.16. Similar for: >>>> >>>> DEFINE_TRACE(ust_$name); -> DEFINE_TRACE_NOARGS(ust_$name); >>>> >>> >>> I had tried with _NOARGS variants initially by manually changing the >>> auto-generated code like this: >>> >>> In trace.h: >>> >>> DECLARE_TRACE_NOARGS(ust_slavio_misc_update_irq_raise); >>> #define trace_slavio_misc_update_irq_raise >>> trace_ust_slavio_misc_update_irq_raise >>> >>> In trace.c: >>> >>> DEFINE_TRACE_NOARGS(ust_slavio_misc_update_irq_raise); >>> >>> static void ust_slavio_misc_update_irq_raise_probe() >>> { >>> trace_mark(ust, slavio_misc_update_irq_raise); >>> } >>> >>> >>> However, it still gave error like this: >>> >>> [harsh@harshbora v9fs]$ make >>> CC osdep.o >>> cc1: warnings being treated as errors >>> In file included from osdep.c:49: >>> trace.h:277: error: data definition has no type or storage class >>> trace.h:277: error: type defaults to ‘int’ in declaration of >>> ‘DECLARE_TRACE_NOARGS’ >>> trace.h:277: error: parameter names (without types) in function >>> declaration >>> >>> >>> It will be great if you can provide a sample code (to be auto-generated) >>> required for a trace-event with void argument. >> >> Ah, right, now I remember that I only left DEFINE_TRACE and DECLARE_TRACE >> as backward compatibility in UST 0.x, and did only add the "_NOARGS" to >> the backward compatibility macros with an added underscore prefix. >> >> If you really just want to have compatibility with UST 0.16, changing the >> minimum of things compared with what you currently have, I would >> recommend to use: >> >> in trace.h: >> _DECLARE_TRACEPOINT_NOARGS(ust_slavio_misc_update_irq_raise) >> #define trace_slavio_misc_update_irq_raise >> trace_ust_slavio_misc_update_irq_raise >> >> in trace.c: >> DEFINE_TRACE(ust_slavio_misc_update_irq_raise); >> >> static void ust_slavio_misc_update_irq_raise_probe(void) >> { >> trace_mark(ust, slavio_misc_update_irq_raise, UST_MARKER_NOARGS); >> } >> >> Let me know how this works for you, > > Thanks for the help, I wrote the python code to auto-generate the same > for events with void arg. However, after doing make, it says: > > trace.o:(__ust_markers+0x4a08): undefined reference to > `__ust_marker_empty_function' > trace.o:(__ust_markers+0x4a60): undefined reference to > `ust_marker_probe_cb' > trace.o:(__ust_markers+0x4a68): undefined reference to > `__ust_marker_empty_function' > trace.o:(__ust_markers+0x4ac0): undefined reference to > `ust_marker_probe_cb' > > I have tested using both: > 1) ust 0.15 with urcu 0.6. > 2) ust 0.16 with urcu 0.6.6 > > Any idea, why this error ? Ok, I got it (Thanks Aneesh). It required this change in configure: Harsh > > - Harsh > >> >> Best regards, >> >> Mathieu >> >>> >>> - Harsh >>> >>>>> Moreover, if ust 2.0 uses different interfaces, we might >>>>> want to use the latest one. >>>> >>>> Note that this kind of special-case won't be needed with LTTng-UST 2.0 >>>> TRACEPOINT_EVENT. In place of DECLARE_TRACE, one would use: >>>> >>>> TRACEPOINT_EVENT(qemu_kvm, $name, >>>> TP_ARGS($args), >>>> TP_FIELDS() >>>> ) >>>> >>>> Note that I notice that some care will need to be taken to generate the >>>> TP_FIELDS() from your existing trace-events file, an example: >>>> >>>> g_realloc(void *ptr, size_t size, void *newptr) >>>> >>>> would have to be translated to: >>>> >>>> TRACE_EVENT(qemu_kvm, g_realloc, >>>> TP_ARGS(void *, ptr, size_t, size, void *, newptr), >>>> TP_FIELDS( >>>> ctf_integer_hex(void *, ptr, ptr) >>>> ctf_integer(size_t, size, size) >>>> ctf_integer_hex(void *, newptr, newptr) >>>> ) >>>> ) >>>> >>>> Note that the bright side is that the tracepoint probe does not need to >>>> be hand-coded anymore, and there is no need to use the markers anymore >>>> neither, which makes the tracer much faster. >>>> >>>> For most of your fields (using %p, %d style format strings), you should >>>> use ctf_integer or ctf_integer_hex (the latter lets the trace viewer >>>> know that the data should be printed as hexadecimal). >>>> You will likely need to detect the %s format strings you have there and >>>> translate them into ctf_string(field, field) too. You can have a >>>> look at >>>> lttng-ust tests/hello/*.[ch] for examples. >>> >>> Thanks for the info, I will look into it later. >>>> >>>> The call which would have looked like trace_qemu_kvm_g_realloc() in UST >>>> 0.x should now be done with: >>>> >>>> tracepoint(qemu_kvm, g_realloc, ptr, size, newptr); >>>> >>>> This is needed to (very soon) add support for sdt.h in LTTng-UST >>>> 2.0, so >>>> systemtap and gdb can hook into tracepoints declared by lttng-ust 2.0. >>>> >>>> Best regards, >>>> >>>> Mathieu >>>> >>>>> >>>>> regards, >>>>> Harsh >>>>> >>>>>> Best regards, >>>>>> >>>>>> Mathieu >>>>>> >>>>>>> >>>>>>> Version History: >>>>>>> >>>>>>> v2: >>>>>>> - Updated tracetool.py to support nop, stderr, dtrace backend >>>>>>> >>>>>>> v1: >>>>>>> - Working protoype with tracetool.py converted only for >>>>>>> simpletrace backend >>>>>>> >>>>>>> Harsh Prateek Bora (4): >>>>>>> Converting tracetool.sh to tracetool.py >>>>>>> Makefile and configure changes for tracetool.py >>>>>>> simpletrace-v2: Handle variable number/size of elements per trace >>>>>>> record. >>>>>>> simpletrace.py: updated log reader script to handle new log format >>>>>>> >>>>>>> Makefile.objs | 6 +- >>>>>>> Makefile.target | 10 +- >>>>>>> configure | 4 +- >>>>>>> monitor.c | 2 +- >>>>>>> scripts/simpletrace.py | 110 ++++++++++- >>>>>>> scripts/tracetool.py | 505 >>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++ >>>>>>> trace/simple.c | 178 ++++++----------- >>>>>>> trace/simple.h | 31 +++- >>>>>>> 8 files changed, 702 insertions(+), 144 deletions(-) >>>>>>> create mode 100755 scripts/tracetool.py >>>>>>> >>>>>> >>>>> >>>> >>> >>> >>> _______________________________________________ >>> lttng-dev mailing list >>> lttng-dev@lists.lttng.org >>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >> > diff --git a/configure b/configure index 0ea8634..40a18d8 100755 --- a/configure +++ b/configure @@ -2495,7 +2495,8 @@ if test "$trace_backend" = "ust"; then int main(void) { return 0; } EOF if compile_prog "" "" ; then - LIBS="-lust $LIBS" + LIBS="-lust -lurcu-bp $LIBS" + libs_qga+="-lust -lurcu-bp" else regards,