From patchwork Tue Jan 10 10:59:37 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harsh Prateek Bora X-Patchwork-Id: 135207 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [140.186.70.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 3F04CB6FC3 for ; Tue, 10 Jan 2012 22:16:03 +1100 (EST) Received: from localhost ([::1]:39735 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RkZRj-0001KC-EP for incoming@patchwork.ozlabs.org; Tue, 10 Jan 2012 06:00:51 -0500 Received: from eggs.gnu.org ([140.186.70.92]:45390) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RkZQs-00085s-Vv for qemu-devel@nongnu.org; Tue, 10 Jan 2012 06:00:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RkZQp-0001kR-Rt for qemu-devel@nongnu.org; Tue, 10 Jan 2012 05:59:58 -0500 Received: from e28smtp08.in.ibm.com ([122.248.162.8]:55818) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RkZQo-0001jP-Cc for qemu-devel@nongnu.org; Tue, 10 Jan 2012 05:59:55 -0500 Received: from /spool/local by e28smtp08.in.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 10 Jan 2012 16:29:46 +0530 Received: from d28relay01.in.ibm.com (9.184.220.58) by e28smtp08.in.ibm.com (192.168.1.138) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 10 Jan 2012 16:29:41 +0530 Received: from d28av02.in.ibm.com (d28av02.in.ibm.com [9.184.220.64]) by d28relay01.in.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q0AAxeYS4386894 for ; Tue, 10 Jan 2012 16:29:41 +0530 Received: from d28av02.in.ibm.com (loopback [127.0.0.1]) by d28av02.in.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q0AAxepa009863 for ; Tue, 10 Jan 2012 21:59:40 +1100 Received: from harshbora.in.ibm.com ([9.124.35.77]) by d28av02.in.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q0AAxeOv009855; Tue, 10 Jan 2012 21:59:40 +1100 From: Harsh Prateek Bora To: qemu-devel@nongnu.org Date: Tue, 10 Jan 2012 16:29:37 +0530 Message-Id: <1326193179-19677-2-git-send-email-harsh@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.1.1 In-Reply-To: <1326193179-19677-1-git-send-email-harsh@linux.vnet.ibm.com> References: <1326193179-19677-1-git-send-email-harsh@linux.vnet.ibm.com> x-cbid: 12011010-2000-0000-0000-000005FD31FB X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 122.248.162.8 Cc: aneesh.kumar@linux.vnet.ibm.com, mathieu.desnoyers@efficios.com, vilanova@ac.upc.edu, stefanha@linux.vnet.ibm.com Subject: [Qemu-devel] [RFC PATCH v3 1/3] Converting tracetool.sh to tracetool.py X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Signed-off-by: Harsh Prateek Bora --- Makefile.objs | 6 +- Makefile.target | 10 +- configure | 7 +- scripts/tracetool | 643 -------------------------------------------------- scripts/tracetool.py | 585 +++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 597 insertions(+), 654 deletions(-) delete mode 100755 scripts/tracetool create mode 100755 scripts/tracetool.py diff --git a/Makefile.objs b/Makefile.objs index 804bc3c..984034a 100644 --- a/Makefile.objs +++ b/Makefile.objs @@ -344,12 +344,12 @@ else trace.h: trace.h-timestamp endif trace.h-timestamp: $(SRC_PATH)/trace-events $(BUILD_DIR)/config-host.mak - $(call quiet-command,sh $(SRC_PATH)/scripts/tracetool --$(TRACE_BACKEND) -h < $< > $@," GEN trace.h") + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/tracetool.py --backend=$(TRACE_BACKEND) -h < $< > $@," GEN trace.h") @cmp -s $@ trace.h || cp $@ trace.h trace.c: trace.c-timestamp trace.c-timestamp: $(SRC_PATH)/trace-events $(BUILD_DIR)/config-host.mak - $(call quiet-command,sh $(SRC_PATH)/scripts/tracetool --$(TRACE_BACKEND) -c < $< > $@," GEN trace.c") + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/tracetool.py --backend=$(TRACE_BACKEND) -c < $< > $@," GEN trace.c") @cmp -s $@ trace.c || cp $@ trace.c trace.o: trace.c $(GENERATED_HEADERS) @@ -362,7 +362,7 @@ trace-dtrace.h: trace-dtrace.dtrace # rule file. So we use '.dtrace' instead trace-dtrace.dtrace: trace-dtrace.dtrace-timestamp trace-dtrace.dtrace-timestamp: $(SRC_PATH)/trace-events $(BUILD_DIR)/config-host.mak - $(call quiet-command,sh $(SRC_PATH)/scripts/tracetool --$(TRACE_BACKEND) -d < $< > $@," GEN trace-dtrace.dtrace") + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/tracetool.py --backend=$(TRACE_BACKEND) -d < $< > $@," GEN trace-dtrace.dtrace") @cmp -s $@ trace-dtrace.dtrace || cp $@ trace-dtrace.dtrace trace-dtrace.o: trace-dtrace.dtrace $(GENERATED_HEADERS) diff --git a/Makefile.target b/Makefile.target index 1e90df7..07b21d1 100644 --- a/Makefile.target +++ b/Makefile.target @@ -50,11 +50,11 @@ TARGET_TYPE=system endif $(QEMU_PROG).stp: - $(call quiet-command,sh $(SRC_PATH)/scripts/tracetool \ - --$(TRACE_BACKEND) \ - --binary $(bindir)/$(QEMU_PROG) \ - --target-arch $(TARGET_ARCH) \ - --target-type $(TARGET_TYPE) \ + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/tracetool.py \ + --backend=$(TRACE_BACKEND) \ + --binary=$(bindir)/$(QEMU_PROG) \ + --target-arch=$(TARGET_ARCH) \ + --target-type=$(TARGET_TYPE) \ --stap < $(SRC_PATH)/trace-events > $(QEMU_PROG).stp," GEN $(QEMU_PROG).stp") else stap: diff --git a/configure b/configure index 19e8394..40a18d8 100755 --- a/configure +++ b/configure @@ -1040,7 +1040,7 @@ echo " --disable-docs disable documentation build" echo " --disable-vhost-net disable vhost-net acceleration support" echo " --enable-vhost-net enable vhost-net acceleration support" echo " --enable-trace-backend=B Set trace backend" -echo " Available backends:" $("$source_path"/scripts/tracetool --list-backends) +echo " Available backends:" $("$source_path"/scripts/tracetool.py --list-backends) echo " --with-trace-file=NAME Full PATH,NAME of file to store traces" echo " Default:trace-" echo " --disable-spice disable spice" @@ -2477,7 +2477,7 @@ fi ########################################## # check if trace backend exists -sh "$source_path/scripts/tracetool" "--$trace_backend" --check-backend > /dev/null 2> /dev/null +$python "$source_path/scripts/tracetool.py" "--backend=$trace_backend" --check-backend > /dev/null 2> /dev/null if test "$?" -ne 0 ; then echo echo "Error: invalid trace backend" @@ -2495,7 +2495,8 @@ if test "$trace_backend" = "ust"; then int main(void) { return 0; } EOF if compile_prog "" "" ; then - LIBS="-lust $LIBS" + LIBS="-lust -lurcu-bp $LIBS" + libs_qga+="-lust -lurcu-bp" else echo echo "Error: Trace backend 'ust' missing libust header files" diff --git a/scripts/tracetool b/scripts/tracetool deleted file mode 100755 index 4c9951d..0000000 --- a/scripts/tracetool +++ /dev/null @@ -1,643 +0,0 @@ -#!/bin/sh -# -# Code generator for trace events -# -# Copyright IBM, Corp. 2010 -# -# This work is licensed under the terms of the GNU GPL, version 2. See -# the COPYING file in the top-level directory. - -# Disable pathname expansion, makes processing text with '*' characters simpler -set -f - -usage() -{ - cat >&2 < return 0 if property is present, or 1 otherwise -has_property() -{ - local props prop - props=${1%%\(*} - props=${props% *} - for prop in $props; do - if [ "$prop" = "$2" ]; then - return 0 - fi - done - return 1 -} - -# Get the argument list of a trace event, including types and names -get_args() -{ - local args - args=${1#*\(} - args=${args%%\)*} - echo "$args" -} - -# Get the argument name list of a trace event -get_argnames() -{ - local nfields field name sep - nfields=0 - sep="$2" - for field in $(get_args "$1"); do - nfields=$((nfields + 1)) - - # Drop pointer star - field=${field#\*} - - # Only argument names have commas at the end - name=${field%,} - test "$field" = "$name" && continue - - printf "%s%s " $name $sep - done - - # Last argument name - if [ "$nfields" -gt 1 ] - then - printf "%s" "$name" - fi -} - -# Get the number of arguments to a trace event -get_argc() -{ - local name argc - argc=0 - for name in $(get_argnames "$1", ","); do - argc=$((argc + 1)) - done - echo $argc -} - -# Get the format string including double quotes for a trace event -get_fmt() -{ - puts "${1#*)}" -} - -linetoh_begin_nop() -{ - return -} - -linetoh_nop() -{ - local name args - name=$(get_name "$1") - args=$(get_args "$1") - - # Define an empty function for the trace event - cat < -#include "trace/stderr.h" - -extern TraceEvent trace_list[]; -EOF - - stderr_event_num=0 -} - -linetoh_stderr() -{ - local name args argnames argc fmt - name=$(get_name "$1") - args=$(get_args "$1") - argnames=$(get_argnames "$1" ",") - argc=$(get_argc "$1") - fmt=$(get_fmt "$1") - - if [ "$argc" -gt 0 ]; then - argnames=", $argnames" - fi - - cat <" - ust_clean_namespace -} - -linetoh_ust() -{ - local name args argnames - name=$(get_name "$1") - args=$(get_args "$1") - argnames=$(get_argnames "$1", ",") - - cat < -$(ust_clean_namespace) -#include "trace.h" -EOF -} - -linetoc_ust() -{ - local name args argnames fmt - name=$(get_name "$1") - args=$(get_args "$1") - argnames=$(get_argnames "$1", ",") - [ -z "$argnames" ] || argnames=", $argnames" - fmt=$(get_fmt "$1") - - cat < 1: + str.append(name) + return ''.join(str) + else: + return '' + +def get_argc(line): + argc = 0 + argnames = get_argnames(line) + if argnames: + for name in argnames.split(','): + argc = argc + 1 + return argc + +def get_fmt(line, sep=')'): + event, sep, fmt = line.partition(sep) + return fmt + +def calc_sizeofargs(line): + args = get_args(line) + argc = get_argc(line) + strtype = ('const char*', 'char*', 'const char *', 'char *') + str = [] + newstr = "" + if argc > 0: + str = args.split(',') + for elem in str: + if elem.lstrip().startswith(strtype): #strings + type, sep, var = elem.rpartition('*') + newstr = newstr+"4 + strlen("+var.lstrip()+") + " + #elif '*' in elem: + # newstr = newstr + "4 + " # pointer vars + else: + #type, sep, var = elem.rpartition(' ') + #newstr = newstr+"sizeof("+type.lstrip()+") + " + newstr = newstr + '8 + ' + newstr = newstr + '0' # for last + + return newstr + + +def trace_h_begin(): + print '''#ifndef TRACE_H +#define TRACE_H + +/* This file is autogenerated by tracetool, do not edit. */ + +#include "qemu-common.h"''' + return + +def trace_h_end(): + print '#endif /* TRACE_H */' + return + +def trace_c_begin(): + print '/* This file is autogenerated by tracetool, do not edit. */' + return + +def trace_c_end(): + # nop, required for trace_gen + return + +def nop_h(events): + print + for event in events: + print '''static inline void trace_%(name)s(%(args)s) +{ +} +''' % { + 'name': event.name, + 'args': event.args +} + return + +def nop_c(events): + # nop, reqd for converters + return + + +def simple_h(events): + print '#include "trace/simple.h"' + print + for event in events: + print 'void trace_%(name)s(%(args)s);' % { + 'name': event.name, + 'args': event.args +} + print + print '#define NR_TRACE_EVENTS %d' % (event.num + 1) + print 'extern TraceEvent trace_list[NR_TRACE_EVENTS];' + + return + +def is_string(arg): + strtype = ('const char*', 'char*', 'const char *', 'char *') + if arg.lstrip().startswith(strtype): + return True + else: + return False + +def simple_c(events): + rec_off = 0 + print '#include "trace.h"' + print '#include "trace/simple.h"' + print + print 'TraceEvent trace_list[] = {' + print + eventlist = list(events) + for event in eventlist: + print '{.tp_name = "%(name)s", .state=0},' % { + 'name': event.name +} + print + print '};' + print + for event in eventlist: + argc = event.argc + print '''void trace_%(name)s(%(args)s) +{ + unsigned int tbuf_idx, rec_off; + uint64_t var64 __attribute__ ((unused)); + uint64_t pvar64 __attribute__ ((unused)); + uint32_t slen __attribute__ ((unused)); + + if (!trace_list[%(event_id)s].state) { + return; + } +''' % { + 'name': event.name, + 'args': event.args, + 'event_id': event.num, +} + print ''' + tbuf_idx = trace_alloc_record(%(event_id)s, %(sizestr)s); + rec_off = (tbuf_idx + ST_V2_REC_HDR_LEN) %% TRACE_BUF_LEN; /* seek record header */ +''' % {'event_id': event.num, 'sizestr': event.sizestr,} + + if argc > 0: + str = event.arglist + for elem in str: + if is_string(elem): # if string + type, sep, var = elem.rpartition('*') + print ''' + slen = strlen(%(var)s); + write_to_buffer(rec_off, (uint8_t*)&slen, sizeof(slen)); + rec_off += sizeof(slen);''' % { + 'var': var.lstrip() +} + print ''' + write_to_buffer(rec_off, (uint8_t*)%(var)s, slen); + rec_off += slen;''' % { + 'var': var.lstrip() +} + elif '*' in elem: # pointer var (not string) + type, sep, var = elem.rpartition('*') + print ''' + pvar64 = (uint64_t)(uint64_t*)%(var)s; + write_to_buffer(rec_off, (uint8_t*)&pvar64, sizeof(uint64_t)); + rec_off += sizeof(uint64_t);''' % { + 'var': var.lstrip() +} + else: # primitive data type + type, sep, var = elem.rpartition(' ') + print ''' + var64 = (uint64_t)%(var)s; + write_to_buffer(rec_off, (uint8_t*)&var64, sizeof(uint64_t)); + rec_off += sizeof(uint64_t);''' % { + 'var': var.lstrip() +} + print ''' + trace_mark_record_complete(tbuf_idx);''' + print '}' + print + + return + +def stderr_h(events): + print '''#include +#include "trace/stderr.h" + +extern TraceEvent trace_list[];''' + for event in events: + argnames = event.argnames + if event.argc > 0: + argnames = ', ' + event.argnames + else: + argnames = '' + print ''' +static inline void trace_%(name)s(%(args)s) +{ + if (trace_list[%(event_num)s].state != 0) { + fprintf(stderr, "%(name)s " %(fmt)s "\\n" %(argnames)s); + } +}''' % { + 'name': event.name, + 'args': event.args, + 'event_num': event.num, + 'fmt': event.fmt.rstrip('\n'), + 'argnames': argnames +} + print + print '#define NR_TRACE_EVENTS %d' % (event.num + 1) + +def stderr_c(events): + print '''#include "trace.h" + +TraceEvent trace_list[] = { +''' + for event in events: + print '{.tp_name = "%(name)s", .state=0},' % { + 'name': event.name +} + print + print '};' + +def ust_h(events): + print '''#include +#undef mutex_lock +#undef mutex_unlock +#undef inline +#undef wmb''' + + for event in events: + if event.argc > 0: + print ''' +DECLARE_TRACE(ust_%(name)s, TP_PROTO(%(args)s), TP_ARGS(%(argnames)s)); +#define trace_%(name)s trace_ust_%(name)s''' % { + 'name': event.name, + 'args': event.args, + 'argnames': event.argnames +} + else: + print ''' +_DECLARE_TRACEPOINT_NOARGS(ust_%(name)s); +#define trace_%(name)s trace_ust_%(name)s''' % { + 'name': event.name, +} + print + return + +def ust_c(events): + print '''#include +#undef mutex_lock +#undef mutex_unlock +#undef inline +#undef wmb +#include "trace.h"''' + eventlist = list(events) + for event in eventlist: + argnames = event.argnames + if event.argc > 0: + argnames = ', ' + event.argnames + print ''' +DEFINE_TRACE(ust_%(name)s); + +static void ust_%(name)s_probe(%(args)s) +{ + trace_mark(ust, %(name)s, %(fmt)s%(argnames)s); +}''' % { + 'name': event.name, + 'args': event.args, + 'fmt': event.fmt.rstrip('\n'), + 'argnames': argnames +} + else: + print ''' +DEFINE_TRACE(ust_%(name)s); + +static void ust_%(name)s_probe(%(args)s) +{ + trace_mark(ust, %(name)s, UST_MARKER_NOARGS); +}''' % { + 'name': event.name, + 'args': event.args, +} + + # register probes + print ''' +static void __attribute__((constructor)) trace_init(void) +{''' + for event in eventlist: + print ' register_trace_ust_%(name)s(ust_%(name)s_probe);' % { + 'name': event.name +} + print '}' + + return + +def dtrace_h(events): + print '#include "trace-dtrace.h"' + print + for event in events: + print '''static inline void trace_%(name)s(%(args)s) { + if (QEMU_%(uppername)s_ENABLED()) { + QEMU_%(uppername)s(%(argnames)s); + } +} +''' % { + 'name': event.name, + 'args': event.args, + 'uppername': event.name.upper(), + 'argnames': event.argnames, +} + +def dtrace_c(events): + return # No need for function definitions in dtrace backend + +def dtrace_d(events): + print 'provider qemu {' + for event in events: + args = event.args + + # DTrace provider syntax expects foo() for empty + # params, not foo(void) + if args == 'void': + args = '' + + # Define prototype for probe arguments + print ''' + probe %(name)s(%(args)s);''' % { + 'name': event.name, + 'args': args +} + print + print '};' + return + +def dtrace_stp(events): + for event in events: + # Define prototype for probe arguments + print ''' +probe %(probeprefix)s.%(name)s = process("%(binary)s").mark("%(name)s") +{''' % { + 'probeprefix': probeprefix, + 'name': event.name, + 'binary': binary +} + i = 1 + if event.argc > 0: + for arg in event.argnames.split(','): + # 'limit' is a reserved keyword + if arg == 'limit': + arg = '_limit' + print ' %s = $arg%d;' % (arg.lstrip(), i) + i += 1 + print '}' + print + return + +def trace_stap_begin(): + global probeprefix + if backend != "dtrace": + print 'SystemTAP tapset generator not applicable to %s backend' % backend + sys.exit(1) + if binary == "": + print '--binary is required for SystemTAP tapset generator' + sys.exit(1) + if ((probeprefix == "") and (targettype == "")): + print '--target-type is required for SystemTAP tapset generator' + sys.exit(1) + if ((probeprefix == "") and (targetarch == "")): + print '--target-arch is required for SystemTAP tapset generator' + sys.exit(1) + if probeprefix == "": + probeprefix = 'qemu.' + targettype + '.' + targetarch + print '/* This file is autogenerated by tracetool, do not edit. */' + return + +def trace_stap_end(): + return #nop, reqd for trace_gen + +def trace_d_begin(): + if backend != 'dtrace': + print 'DTrace probe generator not applicable to %s backend' % backend + sys.exit(1) + print '/* This file is autogenerated by tracetool, do not edit. */' + +def trace_d_end(): + return #nop, reqd for trace_gen + + +# Registry of backends and their converter functions +converters = { + 'simple': { + 'h': simple_h, + 'c': simple_c, + }, + + 'nop': { + 'h': nop_h, + 'c': nop_c, + }, + + 'stderr': { + 'h': stderr_h, + 'c': stderr_c, + }, + + 'dtrace': { + 'h': dtrace_h, + 'c': dtrace_c, + 'd': dtrace_d, + 'stap': dtrace_stp + }, + + 'ust': { + 'h': ust_h, + 'c': ust_c, + }, + +} + +# Trace file header and footer code generators +trace_gen = { + 'h': { + 'begin': trace_h_begin, + 'end': trace_h_end, + }, + 'c': { + 'begin': trace_c_begin, + 'end': trace_c_end, + }, + 'd': { + 'begin': trace_d_begin, + 'end': trace_d_end, + }, + 'stap': { + 'begin': trace_stap_begin, + 'end': trace_stap_end, + }, +} + +# A trace event +class Event(object): + def __init__(self, num, line): + self.num = num + self.args = get_args(line) + self.arglist = self.args.split(',') + self.name = get_name(line) + self.argc = get_argc(line) + self.argnames = get_argnames(line) + self.sizestr = calc_sizeofargs(line) + self.fmt = get_fmt(line) + +# Generator that yields Event objects given a trace-events file object +def read_events(fobj): + event_num = 0 + for line in fobj: + if not line.strip(): + continue + if line.lstrip().startswith('#'): + continue + yield Event(event_num, line) + event_num += 1 + +backend = "" +output = "" +binary = "" +targettype = "" +targetarch = "" +probeprefix = "" + +def main(): + global backend, output, binary, targettype, targetarch, probeprefix + supported_backends = ["simple", "nop", "stderr", "dtrace", "ust"] + short_options = "hcd" + long_options = ["stap", "backend=", "binary=", "target-arch=", "target-type=", "probe-prefix=", "list-backends", "check-backend"] + try: + opts, args = getopt.getopt(sys.argv[1:], short_options, long_options) + except getopt.GetoptError, err: + # print help information and exit: + print str(err) # will print something like "option -a not recognized" + usage() + sys.exit(2) + for opt, arg in opts: + if opt == '-h': + output = 'h' + elif opt == '-c': + output = 'c' + elif opt == '-d': + output = 'd' + elif opt == '--stap': + output = 'stap' + elif opt == '--backend': + backend = arg + elif opt == '--binary': + binary = arg + elif opt == '--target-arch': + targetarch = arg + elif opt == '--target-type': + targettype = arg + elif opt == '--probe-prefix': + probeprefix = arg + elif opt == '--list-backends': + print 'simple, nop, stderr, dtrace' + sys.exit(0) + elif opt == "--check-backend": + if any(backend in s for s in supported_backends): + sys.exit(0) + else: + sys.exit(1) + else: + #assert False, "unhandled option" + print "unhandled option: ", opt + usage() + + if backend == "" or output == "": + usage() + sys.exit(0) + + events = read_events(sys.stdin) + trace_gen[output]['begin']() + converters[backend][output](events) + trace_gen[output]['end']() + return + +if __name__ == "__main__": + main() +