Patchwork [RFC,V5,20/36] qcow2: Drop hash for a given cluster when dedup makes refcount > 2^16/2.

login
register
mail settings
Submitter Benoît Canet
Date Jan. 16, 2013, 4:24 p.m.
Message ID <1358353497-5292-21-git-send-email-benoit@irqsave.net>
Download mbox | patch
Permalink /patch/213004/
State New
Headers show

Comments

Benoît Canet - Jan. 16, 2013, 4:24 p.m.
A new physical cluster with the same hash value will be used for further
occurence of this hash.
---
 block/qcow2-dedup.c    |   32 ++++++++++++++++++++++++++++++++
 block/qcow2-refcount.c |    3 +++
 block/qcow2.h          |    4 ++++
 3 files changed, 39 insertions(+)
Benoît Canet - Jan. 21, 2013, 11:51 a.m.
> You are hardcoding to a width of 16 bits; however, version 3 makes the
> refcount field variable-sized:
>
>          96 -  99:  refcount_order
>                     Describes the width of a reference count block entry
> (width
>                     in bits = 1 << refcount_order). For version 2
> images, the
>                     order is always assumed to be 4 (i.e. the width is
> 16 bits).

Currently the qcow2 code doesn't support anything but refcount_order == 4.

In qcow2.c qcow_open there is:
        be32_to_cpus(&header.refcount_order);
to get the qcow2 order followed by:
    /* Check support for various header values */
    if (header.refcount_order != 4) {
        report_unsupported(bs, "%d bit reference counts",
                           1 << header.refcount_order);
        ret = -ENOTSUP;
        goto fail;
    }

I guess the code doesn't need any special handling for now.

> Hmm, what happens if refcount_order is 0 to disable reference counting?
>  That setting is valid for creating a qcow2 file that can't be used for
> internal snapshots.  But it also interferes with dedup; so you probably
> want to add some additional requirements in the spec (patch 1/36) that
> when dedup is in use, refcount_order must be a minimum value (or require
> that it be exactly 4, for a width of 16 bits).

I'll do that.

Regards

Benoît

Patch

diff --git a/block/qcow2-dedup.c b/block/qcow2-dedup.c
index 25ecefa..9eba773 100644
--- a/block/qcow2-dedup.c
+++ b/block/qcow2-dedup.c
@@ -941,3 +941,35 @@  int qcow2_dedup_store_new_hashes(BlockDriverState *bs,
 
     return ret;
 }
+
+/* Force to use a new physical cluster and QCowHashNode when the refcount pass
+ * 2^16/2.
+ *
+ * @cluster_index: the index of the physical cluster
+ */
+void qcow2_dedup_refcount_half_max_reached(BlockDriverState *bs,
+                                           uint64_t cluster_index)
+{
+    BDRVQcowState *s = bs->opaque;
+    QCowHashNode *hash_node;
+    uint64_t physical_sect = cluster_index * s->cluster_sectors;
+
+    hash_node =  g_tree_lookup(s->dedup_tree_by_sect, &physical_sect);
+
+    if (!hash_node) {
+        return;
+    }
+
+    /* mark this hash so we won't load it anymore at startup after writing it */
+    hash_node->first_logical_sect |= QCOW_FLAG_HALF_MAX_REFCOUNT;
+
+    /* write to disk */
+    qcow2_dedup_read_write_hash(bs,
+                                &hash_node->hash,
+                                &hash_node->first_logical_sect,
+                                hash_node->physical_sect,
+                                true);
+
+    /* remove the QCowHashNode from ram so we won't use it anymore for dedup */
+    qcow2_remove_hash_node(bs, hash_node);
+}
diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
index b1ad112..ac396c4 100644
--- a/block/qcow2-refcount.c
+++ b/block/qcow2-refcount.c
@@ -489,6 +489,9 @@  int QEMU_WARN_UNUSED_RESULT update_refcount(BlockDriverState *bs,
             ret = -EINVAL;
             goto fail;
         }
+        if (s->has_dedup && deduplication && refcount >= 0xFFFF/2) {
+            qcow2_dedup_refcount_half_max_reached(bs, cluster_index);
+        }
         if (refcount == 0 && cluster_index < s->free_cluster_index) {
             s->free_cluster_index = cluster_index;
         }
diff --git a/block/qcow2.h b/block/qcow2.h
index 5c126be..ba10ed0 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -65,6 +65,8 @@ 
 #define DEFAULT_CLUSTER_SIZE 65536
 
 #define HASH_LENGTH 32
+/* indicate that this cluster refcount has reached its maximum value */
+#define QCOW_FLAG_HALF_MAX_REFCOUNT (1LL << 61)
 /* indicate that the hash structure is empty and miss offset */
 #define QCOW_FLAG_EMPTY   (1LL << 62)
 /* indicate that the cluster for this hash has QCOW_OFLAG_COPIED on disk */
@@ -499,5 +501,7 @@  int qcow2_dedup_store_new_hashes(BlockDriverState *bs,
                                  int count,
                                  uint64_t logical_sect,
                                  uint64_t physical_sect);
+void qcow2_dedup_refcount_half_max_reached(BlockDriverState *bs,
+                                           uint64_t cluster_index);
 
 #endif