From patchwork Fri Aug  1 03:12:31 2014
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Patchwork-Submitter: Sanidhya Kashyap <sanidhya.iiith@gmail.com>
X-Patchwork-Id: 375518
Return-Path: <qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org>
X-Original-To: incoming@patchwork.ozlabs.org
Delivered-To: patchwork-incoming@bilbo.ozlabs.org
Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])
	(using TLSv1 with cipher AES256-SHA (256/256 bits))
	(No client certificate requested)
	by ozlabs.org (Postfix) with ESMTPS id A634B14013E
	for <incoming@patchwork.ozlabs.org>;
	Fri,  1 Aug 2014 13:17:00 +1000 (EST)
Received: from localhost ([::1]:59548 helo=lists.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.71) (envelope-from
	<qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org>)
	id 1XD3L0-0003o4-Pu
	for incoming@patchwork.ozlabs.org; Thu, 31 Jul 2014 23:16:58 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:35892)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sanidhya.iiith@gmail.com>) id 1XD3I3-0008Ju-3p
	for qemu-devel@nongnu.org; Thu, 31 Jul 2014 23:14:01 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sanidhya.iiith@gmail.com>) id 1XD3Hw-0000e2-5x
	for qemu-devel@nongnu.org; Thu, 31 Jul 2014 23:13:55 -0400
Received: from mail-pd0-x22c.google.com ([2607:f8b0:400e:c02::22c]:33355)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sanidhya.iiith@gmail.com>) id 1XD3Hv-0000dU-QG
	for qemu-devel@nongnu.org; Thu, 31 Jul 2014 23:13:48 -0400
Received: by mail-pd0-f172.google.com with SMTP id ft15so4666269pdb.17
	for <qemu-devel@nongnu.org>; Thu, 31 Jul 2014 20:13:47 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=from:to:cc:subject:date:message-id:in-reply-to:references;
	bh=xyifjcMhAUFfuIgKE19zh10kPnguy21UNAphDwn8VSI=;
	b=wGGnRrTR11HC4A73hCP+GDDZRJ26q6dKqsi1+ElVgdUgjbVYzC6zSFAF4sg5p+o91F
	2fCjg9pHtJsH+KogaH/qRHpSLFC2YTGOUf44QtKiNovX/hF5boU7gOtUGYyAfYlYB0YE
	ByCdeEnWE78Q1TvkiIGUMr5YAwdzi3R/MUCWgLwyHffxm0iRqx+Ocj2wQ/IVD2MnIWZ6
	sMWCoWZcEShDhV+HezuuxXQ0/KtthcJ0XoYXIC3SZJeru06KvZQyykyEvKRLX5w2/G2W
	QhAL9PtJxLAcA+AT5YlztK7TBvuLOquTmxJ5UTNCYkWs1bISSScVIf7nX5Mv31gh3U0B
	k/lA==
X-Received: by 10.70.52.1 with SMTP id p1mr2596995pdo.111.1406862826960;
	Thu, 31 Jul 2014 20:13:46 -0700 (PDT)
Received: from localhost.localdomain ([106.219.174.182])
	by mx.google.com with ESMTPSA id
	u7sm10711518pdj.34.2014.07.31.20.13.40 for <multiple recipients>
	(version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
	Thu, 31 Jul 2014 20:13:46 -0700 (PDT)
From: Sanidhya Kashyap <sanidhya.iiith@gmail.com>
To: qemu list <qemu-devel@nongnu.org>
Date: Fri,  1 Aug 2014 08:42:31 +0530
Message-Id: <1406862751-24008-7-git-send-email-sanidhya.iiith@gmail.com>
X-Mailer: git-send-email 1.9.3
In-Reply-To: <1406862751-24008-1-git-send-email-sanidhya.iiith@gmail.com>
References: <1406862751-24008-1-git-send-email-sanidhya.iiith@gmail.com>
X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address
	(bad octet value).
X-Received-From: 2607:f8b0:400e:c02::22c
Cc: Sanidhya Kashyap <sanidhya.iiith@gmail.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Juan Quintela <quintela@redhat.com>
Subject: [Qemu-devel] [PATCH v5 6/6] BitmapLog: python script for extracting
	bitmap from a binary file
X-BeenThere: qemu-devel@nongnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org
Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org

The file not only extracts the bitmap from the file but also draws the figure
if required. Currently, figure is drawn for all the bitmaps. Later, I'll make
the change to draw for different blocks.

The picture is drawn by generating a matrix of 0s and 1s from the bitmap. The
dimensions are calculated on the basis of total bitmap pages which is represented
as sqrt(total pages) X (sqrt(total pages) + 1). The white parts indicate non dirtied
region while the black - dirtied region.

The python code requires some libraries such as numpy, pylab and math to generate
the images.

Signed-off-by: Sanidhya Kashyap <sanidhya.iiith@gmail.com>
---
 scripts/extract-bitmap.py | 144 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 144 insertions(+)
 create mode 100755 scripts/extract-bitmap.py

diff --git a/scripts/extract-bitmap.py b/scripts/extract-bitmap.py
new file mode 100755
index 0000000..942deca
--- /dev/null
+++ b/scripts/extract-bitmap.py
@@ -0,0 +1,144 @@
+#!/usr/bin/python
+# This python script helps in extracting the dirty bitmap present
+# in the file after executing the log-dirty-bitmap command either
+# from the qmp or hmp interface. This file only processes binary
+# file obtained via command.
+#
+# Copyright (C) 2014 Sanidhya Kashyap <sanidhya.iiith@gmail.com>
+#
+# Authors:
+#       Sanidhya Kashyap
+#
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+
+import struct
+import argparse
+from functools import partial
+from math import sqrt
+from numpy import array
+from pylab import figure,imshow,show,gray
+
+long_bytes = 8
+byte_size = 8
+int_bytes = 4
+complete_bitmap_list = []
+block_list = []
+
+def get_unsigned_long_integer(value):
+	return struct.unpack('<Q', value)[0]
+
+def get_long_integer(value):
+	return struct.unpack('<q', value)[0]
+
+def get_integer(value):
+	return struct.unpack('<i', value)[0]
+
+def get_char(value):
+	return struct.unpack('<c', value)[0]
+
+def get_string(value, length):
+	name = struct.unpack('<'+str(length)+'s', value)[0]
+	for i in range(len(name)):
+		if name[i] == '\x00':
+			return name[:i]
+
+def dec2bin(decimal):
+    bin_value = bin(decimal)[2:]
+    if len(bin_value) < long_bytes * byte_size:
+        add_zeroes = long_bytes * byte_size - len(bin_value)
+        for i in range(add_zeroes):
+            bin_value += "0"
+    return str(bin_value)
+
+def get_bitmap_length(ram_bitmap_pages):
+    bitmap_length = ram_bitmap_pages / (long_bytes * byte_size)
+    if ram_bitmap_pages % (long_bytes * byte_size) != 0:
+        bitmap_length += 1
+    return bitmap_length
+
+def dump_ram_block_info(infile):
+    total_blocks = get_integer(infile.read(int_bytes))
+    for i in range(total_blocks):
+        block_name_length = get_integer(infile.read(int_bytes))
+        block_name = get_string(infile.read(block_name_length), block_name_length)
+        block_offset = get_unsigned_long_integer(infile.read(long_bytes))
+        block_length = get_unsigned_long_integer(infile.read(long_bytes))
+        block_list.append(dict(name=block_name, offset=block_offset, length=block_length))
+
+def generate_images():
+    r = 0
+    for list in complete_bitmap_list:
+        all_digits = ""
+        for element in list:
+            all_digits += dec2bin(element)
+        l = len(all_digits)
+        sqrtvalue = int(sqrt(l))
+        for i in range(sqrtvalue * (sqrtvalue+1)-l):
+            all_digits+="0"
+
+        v = []
+        l = len(all_digits)
+        for i in range(sqrtvalue+1):
+            v1 = []
+            for j in range(sqrtvalue):
+                v1.append(int(all_digits[i*sqrtvalue+j]))
+            v.append(v1)
+
+        im_array = array(v)
+        figure(r)
+        imshow(im_array, cmap=gray())
+        r += 1
+    show()
+
+def dump_bitmap(infile, draw):
+    marker = 'M'
+    count = 0
+    value = ' '
+    current_ram_bitmap_pages = 0
+    prev_ram_bitmap_pages = 0
+    while True:
+        if len(value) == 0  or marker != 'M':
+            print "issue with the dump"
+            return
+        bitmap_page_raw_value = infile.read(long_bytes)
+        if not bitmap_page_raw_value:
+            break
+        current_ram_bitmap_pages = get_long_integer(bitmap_page_raw_value)
+        if current_ram_bitmap_pages != prev_ram_bitmap_pages:
+            prev_ram_bitmap_pages = current_ram_bitmap_pages
+            dump_ram_block_info(infile)
+
+        bitmap_length = get_bitmap_length(current_ram_bitmap_pages)
+        bitmap_list = []
+        bitmap_raw_value = infile.read(long_bytes * bitmap_length)
+        if not bitmap_raw_value:
+            break
+        count+=1
+        for i in range(bitmap_length):
+            mark = i * long_bytes
+            bitmap_list.append((get_unsigned_long_integer(bitmap_raw_value[mark:mark+long_bytes])))
+        complete_bitmap_list.append(bitmap_list)
+        value = infile.read(1)
+        marker = get_char(value)
+    if draw is True:
+        generate_images()
+    else:
+        print complete_bitmap_list
+
+def main():
+    extracter = argparse.ArgumentParser(description='Extract dirty bitmap from binary file.')
+    extracter.add_argument('infile', help='Input file to extract the bitmap')
+    extracter.add_argument('-d', action='store_true', dest='draw', default=False,
+            help='Draw a black and white image of the processed dirty bitmap')
+    args = extracter.parse_args()
+    print 'The filename is {}'.format(args.infile)
+
+    infile = open(format(args.infile), 'rb')
+
+    dump_bitmap(infile, args.draw);
+
+    infile.close()
+
+if __name__ == '__main__':
+    main()