Patchwork [10/11] Add XBZRLE statistics

login
register
mail settings
Submitter Juan Quintela
Date Aug. 1, 2012, 6:01 p.m.
Message ID <1343844079-2949-11-git-send-email-quintela@redhat.com>
Download mbox | patch
Permalink /patch/174558/
State New
Headers show

Comments

Juan Quintela - Aug. 1, 2012, 6:01 p.m.
From: Orit Wasserman <owasserm@redhat.com>

Signed-off-by: Benoit Hudzia <benoit.hudzia@sap.com>
Signed-off-by: Petter Svard <petters@cs.umu.se>
Signed-off-by: Aidan Shribman <aidan.shribman@sap.com>
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 arch_init.c      | 28 ++++++++++++++++++++++++++++
 hmp.c            | 13 +++++++++++++
 migration.c      | 17 +++++++++++++++++
 migration.h      |  4 ++++
 qapi-schema.json | 28 +++++++++++++++++++++++++++-
 qmp-commands.hx  | 36 +++++++++++++++++++++++++++++++++++-
 6 files changed, 124 insertions(+), 2 deletions(-)
Eric Blake - Aug. 1, 2012, 7:08 p.m.
On 08/01/2012 12:01 PM, Juan Quintela wrote:
> From: Orit Wasserman <owasserm@redhat.com>
> 
> Signed-off-by: Benoit Hudzia <benoit.hudzia@sap.com>
> Signed-off-by: Petter Svard <petters@cs.umu.se>
> Signed-off-by: Aidan Shribman <aidan.shribman@sap.com>
> Signed-off-by: Orit Wasserman <owasserm@redhat.com>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  arch_init.c      | 28 ++++++++++++++++++++++++++++
>  hmp.c            | 13 +++++++++++++
>  migration.c      | 17 +++++++++++++++++
>  migration.h      |  4 ++++
>  qapi-schema.json | 28 +++++++++++++++++++++++++++-
>  qmp-commands.hx  | 36 +++++++++++++++++++++++++++++++++++-
>  6 files changed, 124 insertions(+), 2 deletions(-)
> 

>  ##
> +# @XBZRLECacheStats
> +#
> +# Detailed XBZRLE migration cache statistics
> +#
> +# @cache-size: XBZRLE cache size
> +#
> +# @xbzrle-bytes: amount of bytes already transferred to the target VM
> +#
> +# @xbzrle-pages: amount of pages transferred to the target VM
> +#
> +# @xbzrle-cache-miss: @optional, number of cache miss
> +#
> +# @xbzrle-overflow: number of overflows
> +#
> +# Since: 1.2
> +##
> +{ 'type': 'XBZRLECacheStats',
> +  'data': {'cache-size': 'int', 'xbzrle-bytes': 'int', 'xbzrle-pages': 'int',
> +           'xbzrle-cache-miss': 'int', 'xbzrle-overflow': 'int' } }

Given that this struct already has xbzrle in the name, do all the fields
need to repeat that information?

> +++ b/qmp-commands.hx
> @@ -2128,7 +2128,17 @@ The main json-object contains the following:
>           - "transferred": amount transferred (json-int)
>           - "remaining": amount remaining (json-int)
>           - "total": total (json-int)
> -
> +- "xbzrle-cache": only present if XBZRLE is active.
> +  It is a json-object with the following XBZRLE information:
> +         - "cache-size": XBZRLE cache size
> +         - "xbzrle-bytes": total XBZRLE bytes transferred, only present if
> +                           status is "active" or "completed"
> +         - "xbzrle-pages": number of XBZRLE compressed pages, only present if
> +                           status is "active" or "completed"
> +         - "cache-miss": number of cache misses, only present if

s/cache-miss/xbzrle-cache-miss/ (unless you heed my earlier complaint,
then s/xbzrle-bytes/bytes/ instead)

> +                           status is "active" or "completed"
> +         - "overflow": number of XBZRLE overflows, only present if

s/overflow/xbzrle-overflow/

> +                           status is "active" or "completed"

Given that the overall 'xbzrle-cache' struct is already documented as
present when XBZRLE is active, do you need to also document that the
subfields are present when active or completed?

>  Examples:
> 
>  1. Before the first migration
> @@ -2196,6 +2206,30 @@ Examples:
>        }
>     }
> 
> +6. Migration is being performed and XBZRLE is active:
> +
> +-> { "execute": "query-migrate" }
> +<- {
> +      "return":{
> +         "status":"active",
> +         "capabilities" : [ { "capability": "xbzrle", "state" : true } ],
> +         "ram":{
> +            "total":1057024,
> +            "remaining":1053304,
> +            "transferred":3720,
> +            "duplicate": 10,
> +            "normal" : 3333

Inconsistent spacing around these ':'.

> +         },
> +         "cache":{

s/cache/xbzrle-cache/

> +            "cache-size": 1024

s/ 1024/1024,/

> +            "xbzrle-transferred":20971520,
> +            "xbzrle-pages":2444343,
> +            "xbzrle-cache-miss":2244,
> +            "xbzrle-overflow":34434

and if you take my earlier comments, these lines would change too.
Orit Wasserman - Aug. 2, 2012, 11:32 a.m.
On 08/01/2012 10:08 PM, Eric Blake wrote:
> On 08/01/2012 12:01 PM, Juan Quintela wrote:
>> From: Orit Wasserman <owasserm@redhat.com>
>>
>> Signed-off-by: Benoit Hudzia <benoit.hudzia@sap.com>
>> Signed-off-by: Petter Svard <petters@cs.umu.se>
>> Signed-off-by: Aidan Shribman <aidan.shribman@sap.com>
>> Signed-off-by: Orit Wasserman <owasserm@redhat.com>
>> Signed-off-by: Juan Quintela <quintela@redhat.com>
>> ---
>>  arch_init.c      | 28 ++++++++++++++++++++++++++++
>>  hmp.c            | 13 +++++++++++++
>>  migration.c      | 17 +++++++++++++++++
>>  migration.h      |  4 ++++
>>  qapi-schema.json | 28 +++++++++++++++++++++++++++-
>>  qmp-commands.hx  | 36 +++++++++++++++++++++++++++++++++++-
>>  6 files changed, 124 insertions(+), 2 deletions(-)
>>
> 
>>  ##
>> +# @XBZRLECacheStats
>> +#
>> +# Detailed XBZRLE migration cache statistics
>> +#
>> +# @cache-size: XBZRLE cache size
>> +#
>> +# @xbzrle-bytes: amount of bytes already transferred to the target VM
>> +#
>> +# @xbzrle-pages: amount of pages transferred to the target VM
>> +#
>> +# @xbzrle-cache-miss: @optional, number of cache miss
>> +#
>> +# @xbzrle-overflow: number of overflows
>> +#
>> +# Since: 1.2
>> +##
>> +{ 'type': 'XBZRLECacheStats',
>> +  'data': {'cache-size': 'int', 'xbzrle-bytes': 'int', 'xbzrle-pages': 'int',
>> +           'xbzrle-cache-miss': 'int', 'xbzrle-overflow': 'int' } }
> 
> Given that this struct already has xbzrle in the name, do all the fields
> need to repeat that information?
I will remove it.
> 
>> +++ b/qmp-commands.hx
>> @@ -2128,7 +2128,17 @@ The main json-object contains the following:
>>           - "transferred": amount transferred (json-int)
>>           - "remaining": amount remaining (json-int)
>>           - "total": total (json-int)
>> -
>> +- "xbzrle-cache": only present if XBZRLE is active.
>> +  It is a json-object with the following XBZRLE information:
>> +         - "cache-size": XBZRLE cache size
>> +         - "xbzrle-bytes": total XBZRLE bytes transferred, only present if
>> +                           status is "active" or "completed"
>> +         - "xbzrle-pages": number of XBZRLE compressed pages, only present if
>> +                           status is "active" or "completed"
>> +         - "cache-miss": number of cache misses, only present if
> 
> s/cache-miss/xbzrle-cache-miss/ (unless you heed my earlier complaint,
> then s/xbzrle-bytes/bytes/ instead)
> 
>> +                           status is "active" or "completed"
>> +         - "overflow": number of XBZRLE overflows, only present if
> 
> s/overflow/xbzrle-overflow/
> 
>> +                           status is "active" or "completed"
> 
> Given that the overall 'xbzrle-cache' struct is already documented as
> present when XBZRLE is active, do you need to also document that the
> subfields are present when active or completed?
I will remove the comment
Orit
> 
>>  Examples:
>>
>>  1. Before the first migration
>> @@ -2196,6 +2206,30 @@ Examples:
>>        }
>>     }
>>
>> +6. Migration is being performed and XBZRLE is active:
>> +
>> +-> { "execute": "query-migrate" }
>> +<- {
>> +      "return":{
>> +         "status":"active",
>> +         "capabilities" : [ { "capability": "xbzrle", "state" : true } ],
>> +         "ram":{
>> +            "total":1057024,
>> +            "remaining":1053304,
>> +            "transferred":3720,
>> +            "duplicate": 10,
>> +            "normal" : 3333
> 
> Inconsistent spacing around these ':'.
> 
>> +         },
>> +         "cache":{
> 
> s/cache/xbzrle-cache/
> 
>> +            "cache-size": 1024
> 
> s/ 1024/1024,/
> 
>> +            "xbzrle-transferred":20971520,
>> +            "xbzrle-pages":2444343,
>> +            "xbzrle-cache-miss":2244,
>> +            "xbzrle-overflow":34434
> 
> and if you take my earlier comments, these lines would change too.
>

Patch

diff --git a/arch_init.c b/arch_init.c
index 882448e..b55f47b 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -205,7 +205,11 @@  int64_t xbzrle_cache_resize(int64_t new_size)
 typedef struct AccountingInfo {
     uint64_t dup_pages;
     uint64_t norm_pages;
+    uint64_t xbzrle_bytes;
+    uint64_t xbzrle_pages;
+    uint64_t xbzrle_cache_miss;
     uint64_t iterations;
+    uint64_t xbzrle_overflows;
 } AccountingInfo;

 static AccountingInfo acct_info;
@@ -235,6 +239,26 @@  uint64_t norm_mig_pages_transferred(void)
     return acct_info.norm_pages;
 }

+uint64_t xbzrle_mig_bytes_transferred(void)
+{
+    return acct_info.xbzrle_bytes;
+}
+
+uint64_t xbzrle_mig_pages_transferred(void)
+{
+    return acct_info.xbzrle_pages;
+}
+
+uint64_t xbzrle_mig_pages_cache_miss(void)
+{
+    return acct_info.xbzrle_cache_miss;
+}
+
+uint64_t xbzrle_mig_pages_overflow(void)
+{
+    return acct_info.xbzrle_overflows;
+}
+
 static void save_block_hdr(QEMUFile *f, RAMBlock *block, ram_addr_t offset,
         int cont, int flag)
 {
@@ -259,6 +283,7 @@  static int save_xbzrle_page(QEMUFile *f, uint8_t *current_data,
     if (!cache_is_cached(XBZRLE.cache, current_addr)) {
         cache_insert(XBZRLE.cache, current_addr,
                      g_memdup(current_data, TARGET_PAGE_SIZE));
+        acct_info.xbzrle_cache_miss++;
         return -1;
     }

@@ -276,6 +301,7 @@  static int save_xbzrle_page(QEMUFile *f, uint8_t *current_data,
         return 0;
     } else if (encoded_len == -1) {
         DPRINTF("Overflow\n");
+        acct_info.xbzrle_overflows++;
         /* update data in the cache */
         memcpy(prev_cached_page, current_data, TARGET_PAGE_SIZE);
         return -1;
@@ -290,6 +316,8 @@  static int save_xbzrle_page(QEMUFile *f, uint8_t *current_data,
     qemu_put_be16(f, encoded_len);
     qemu_put_buffer(f, XBZRLE.encoded_buf, encoded_len);
     bytes_sent = encoded_len + 1 + 2;
+    acct_info.xbzrle_pages++;
+    acct_info.xbzrle_bytes += bytes_sent;

     return bytes_sent;
 }
diff --git a/hmp.c b/hmp.c
index f19a63b..a05145a 100644
--- a/hmp.c
+++ b/hmp.c
@@ -177,6 +177,19 @@  void hmp_info_migrate(Monitor *mon)
                        info->disk->total >> 10);
     }

+    if (info->has_xbzrle_cache) {
+        monitor_printf(mon, "cache size: %" PRIu64 " bytes\n",
+                       info->xbzrle_cache->cache_size);
+        monitor_printf(mon, "xbzrle transferred: %" PRIu64 " kbytes\n",
+                       info->xbzrle_cache->xbzrle_bytes >> 10);
+        monitor_printf(mon, "xbzrle pages: %" PRIu64 " pages\n",
+                       info->xbzrle_cache->xbzrle_pages);
+        monitor_printf(mon, "xbzrle cache miss: %" PRIu64 "\n",
+                       info->xbzrle_cache->xbzrle_cache_miss);
+        monitor_printf(mon, "xbzrle overflow : %" PRIu64 "\n",
+                       info->xbzrle_cache->xbzrle_overflow);
+    }
+
     qapi_free_MigrationInfo(info);
     qapi_free_MigrationCapabilityStatusList(caps);
 }
diff --git a/migration.c b/migration.c
index fef06b3..15f0898 100644
--- a/migration.c
+++ b/migration.c
@@ -141,6 +141,19 @@  MigrationCapabilityStatusList *qmp_query_migrate_capabilities(Error **errp)
     return head;
 }

+static void get_xbzrle_cache_stats(MigrationInfo *info)
+{
+    if (migrate_use_xbzrle()) {
+        info->has_xbzrle_cache = true;
+        info->xbzrle_cache = g_malloc0(sizeof(*info->xbzrle_cache));
+        info->xbzrle_cache->cache_size = migrate_xbzrle_cache_size();
+        info->xbzrle_cache->xbzrle_bytes = xbzrle_mig_bytes_transferred();
+        info->xbzrle_cache->xbzrle_pages = xbzrle_mig_pages_transferred();
+        info->xbzrle_cache->xbzrle_cache_miss = xbzrle_mig_pages_cache_miss();
+        info->xbzrle_cache->xbzrle_overflow = xbzrle_mig_pages_overflow();
+    }
+}
+
 MigrationInfo *qmp_query_migrate(Error **errp)
 {
     MigrationInfo *info = g_malloc0(sizeof(*info));
@@ -172,8 +185,12 @@  MigrationInfo *qmp_query_migrate(Error **errp)
             info->disk->remaining = blk_mig_bytes_remaining();
             info->disk->total = blk_mig_bytes_total();
         }
+
+        get_xbzrle_cache_stats(info);
         break;
     case MIG_STATE_COMPLETED:
+        get_xbzrle_cache_stats(info);
+
         info->has_status = true;
         info->status = g_strdup("completed");

diff --git a/migration.h b/migration.h
index e4a7cd7..a9852fc 100644
--- a/migration.h
+++ b/migration.h
@@ -91,6 +91,10 @@  uint64_t dup_mig_bytes_transferred(void);
 uint64_t dup_mig_pages_transferred(void);
 uint64_t norm_mig_bytes_transferred(void);
 uint64_t norm_mig_pages_transferred(void);
+uint64_t xbzrle_mig_bytes_transferred(void);
+uint64_t xbzrle_mig_pages_transferred(void);
+uint64_t xbzrle_mig_pages_overflow(void);
+uint64_t xbzrle_mig_pages_cache_miss(void);

 /**
  * @migrate_add_blocker - prevent migration from proceeding
diff --git a/qapi-schema.json b/qapi-schema.json
index 5cbcdb4..f4b5a93 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -278,6 +278,27 @@ 
            'normal-bytes': 'int' } }

 ##
+# @XBZRLECacheStats
+#
+# Detailed XBZRLE migration cache statistics
+#
+# @cache-size: XBZRLE cache size
+#
+# @xbzrle-bytes: amount of bytes already transferred to the target VM
+#
+# @xbzrle-pages: amount of pages transferred to the target VM
+#
+# @xbzrle-cache-miss: @optional, number of cache miss
+#
+# @xbzrle-overflow: number of overflows
+#
+# Since: 1.2
+##
+{ 'type': 'XBZRLECacheStats',
+  'data': {'cache-size': 'int', 'xbzrle-bytes': 'int', 'xbzrle-pages': 'int',
+           'xbzrle-cache-miss': 'int', 'xbzrle-overflow': 'int' } }
+
+##
 # @MigrationInfo
 #
 # Information about current migration process.
@@ -295,11 +316,16 @@ 
 #        status, only returned if status is 'active' and it is a block
 #        migration
 #
+# @xbzrle-cache: #optional @XBZRLECacheStats containing detailed XBZRLE
+#                migration statistics, only returned if XBZRLE feature is on and
+#                status is 'active' or 'completed' (since 1.2)
+#
 # Since: 0.14.0
 ##
 { 'type': 'MigrationInfo',
   'data': {'*status': 'str', '*ram': 'MigrationStats',
-           '*disk': 'MigrationStats'} }
+           '*disk': 'MigrationStats',
+           '*xbzrle-cache': 'XBZRLECacheStats'} }

 ##
 # @query-migrate
diff --git a/qmp-commands.hx b/qmp-commands.hx
index f7ae7e6..a5b35b0 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -2128,7 +2128,17 @@  The main json-object contains the following:
          - "transferred": amount transferred (json-int)
          - "remaining": amount remaining (json-int)
          - "total": total (json-int)
-
+- "xbzrle-cache": only present if XBZRLE is active.
+  It is a json-object with the following XBZRLE information:
+         - "cache-size": XBZRLE cache size
+         - "xbzrle-bytes": total XBZRLE bytes transferred, only present if
+                           status is "active" or "completed"
+         - "xbzrle-pages": number of XBZRLE compressed pages, only present if
+                           status is "active" or "completed"
+         - "cache-miss": number of cache misses, only present if
+                           status is "active" or "completed"
+         - "overflow": number of XBZRLE overflows, only present if
+                           status is "active" or "completed"
 Examples:

 1. Before the first migration
@@ -2196,6 +2206,30 @@  Examples:
       }
    }

+6. Migration is being performed and XBZRLE is active:
+
+-> { "execute": "query-migrate" }
+<- {
+      "return":{
+         "status":"active",
+         "capabilities" : [ { "capability": "xbzrle", "state" : true } ],
+         "ram":{
+            "total":1057024,
+            "remaining":1053304,
+            "transferred":3720,
+            "duplicate": 10,
+            "normal" : 3333
+         },
+         "cache":{
+            "cache-size": 1024
+            "xbzrle-transferred":20971520,
+            "xbzrle-pages":2444343,
+            "xbzrle-cache-miss":2244,
+            "xbzrle-overflow":34434
+         }
+      }
+   }
+
 EQMP

     {