From patchwork Fri Nov 23 08:02:32 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robert Wang X-Patchwork-Id: 201250 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 0CD0C2C008C for ; Fri, 23 Nov 2012 19:03:22 +1100 (EST) Received: from localhost ([::1]:43311 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TboEK-0000Af-1j for incoming@patchwork.ozlabs.org; Fri, 23 Nov 2012 03:03:20 -0500 Received: from eggs.gnu.org ([208.118.235.92]:54661) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TboDx-0008Vj-7G for qemu-devel@nongnu.org; Fri, 23 Nov 2012 03:03:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TboDv-00066x-Aj for qemu-devel@nongnu.org; Fri, 23 Nov 2012 03:02:57 -0500 Received: from mail-ia0-f173.google.com ([209.85.210.173]:52375) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TboDv-00066n-34 for qemu-devel@nongnu.org; Fri, 23 Nov 2012 03:02:55 -0500 Received: by mail-ia0-f173.google.com with SMTP id w21so4843565iac.4 for ; Fri, 23 Nov 2012 00:02:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=ER50hmdPfLKrxMdYlaGIlEgzxtfMP7ZvnvW3/xdkfa0=; b=uIgmxy7byGMB6wLfDUUCrg38RXWly2v8dr4BGtuJNUGgqEAMhaTcYvkiB7uJlBHhNr DCp+UuNNIF1SgkPQ2XVgLGyNTHV/+L+2kT1i7rTWY0LBan+JVTHNnXWp94E3nWWLiq3B JdP2MM0RwHnmvZU8WhENMPbv+JUkmJh9IVG1Er7D3GV5JAjJo3FQs6LiGFlzby8FCJFO k7DJWJdU6uAQcVOc9UNCu74/r7G+D9DK1JaamiUvyWoXxva73D3JhHPaf9RYng2bASQk /rNtuWE8abhYHLMh1O1bnzJRF67Ge/FmRAC8gKNfkb5vleUc7gvPgbQvvdvSRcF00pWx 7ZbQ== Received: by 10.50.106.227 with SMTP id gx3mr5579706igb.10.1353657774803; Fri, 23 Nov 2012 00:02:54 -0800 (PST) Received: from localhost.localdomain ([202.108.130.153]) by mx.google.com with ESMTPS id rd10sm3898632igb.1.2012.11.23.00.02.52 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 23 Nov 2012 00:02:54 -0800 (PST) From: Dong Xu Wang To: qemu-devel@nongnu.org Date: Fri, 23 Nov 2012 16:02:32 +0800 Message-Id: <1353657757-21424-2-git-send-email-wdongxu@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1353657757-21424-1-git-send-email-wdongxu@linux.vnet.ibm.com> References: <1353657757-21424-1-git-send-email-wdongxu@linux.vnet.ibm.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 209.85.210.173 Cc: kwolf@redhat.com, Dong Xu Wang Subject: [Qemu-devel] [PATCH V16 1/6] docs: document for add-cow file format X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Document for add-cow format, the usage and spec of add-cow are introduced. Signed-off-by: Dong Xu Wang --- docs/specs/add-cow.txt | 154 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 154 insertions(+), 0 deletions(-) create mode 100644 docs/specs/add-cow.txt diff --git a/docs/specs/add-cow.txt b/docs/specs/add-cow.txt new file mode 100644 index 0000000..24e9a11 --- /dev/null +++ b/docs/specs/add-cow.txt @@ -0,0 +1,154 @@ +== General == + +The raw file format does not support backing files or copy on write feature. +The add-cow image format makes it possible to use backing files with a raw +image by keeping a separate .add-cow metadata file. Once all sectors +have been written into the raw image it is safe to discard the .add-cow +and backing files, then we can use the raw image directly. + +An example usage of add-cow would look like:: +(ubuntu.img is a disk image which has an installed OS.) + 1) Create a raw image with the same size of ubuntu.img + qemu-img create -f raw test.raw 8G + 2) Create an add-cow image which will store dirty bitmap + qemu-img create -f add-cow test.add-cow \ + -o backing_file=ubuntu.img,image_file=test.raw + 3) Run qemu with add-cow image + qemu -drive if=virtio,file=test.add-cow + +test.raw may be larger than ubuntu.img, in that case, the size of test.add-cow +will be calculated from the size of test.raw. + +image_fmt can be omitted, in that case image_fmt should be set as "raw". +backing_fmt can also be omitted, add-cow should do a probe operation and determine +what the backing file's format is. + +=Specification= + +The file format looks like this: + + +---------------+-------------------------------+ + | Header | COW bitmap | + +---------------+-------------------------------+ + +All numbers in add-cow are stored in Little Endian byte order. + +== Header == + +The Header is included in the first bytes: +(HEADER_SIZE is defined in 44-47 bytes.) + Byte 0 - 3: magic + add-cow magic string ("ACOW"). + + 4 - 7: version + Version number (only valid value is 1 now). + + 8 - 11: backing file name offset + Offset in the add-cow file at which the backing file + name is stored (NB: The string is not NUL-terminated). + If backing file name does NOT exist, this field will be + 0. Must be between 80 and [HEADER_SIZE - 2](a file name + must be at least 1 byte). + + 12 - 15: backing file name size + Length of the backing file name in bytes. It will be 0 + if the backing file name offset is 0. If backing file + name offset is non-zero, then it must be non-zero. Must + be less than [HEADER_SIZE - 80] to fit in the reserved + part of the header. Backing file name offset + size + must be no more than HEADER_SIZE. + + 16 - 19: image file name offset + Offset in the add-cow file at which the image file name + is stored (NB: The string is not NUL-terminated). It + must be between 80 and [HEADER_SIZE - 2]. Image file + name size + offset must be no more than HEADER_SIZE. + + 20 - 23: image file name size + Length of the image file name in bytes. + Must be less than [HEADER_SIZE - 80] to fit in the reserved + part of the header. + + 24 - 27: cluster bits + Number of bits that are used for addressing an offset + within a cluster (1 << cluster_bits is the cluster size). + Must not be less than 9 (i.e. 512 byte clusters). + + Note: qemu as of today has an implementation limit of 2 MB + as the maximum cluster size and won't be able to open images + with larger cluster sizes. + + 28 - 35: features + Bitmask of features. If a feature bit is set but not recognized, + the add-cow file should be dropped. They are not used in v1. + + Bits 0-63: Reserved (set to 0) + + 36 - 43: compatible features + Bitmask of compatible features. An implementation can + safely ignore any unknown bits that are set. + Bit 0: All allocated bit. If this bit is set then + backing file and COW bitmap will not be used, + and can read from or write to image file directly. + + Bits 1-63: Reserved (set to 0) + + 44 - 47: HEADER_SIZE + The header field is variable-sized. This field indicates + how many bytes will be used to store add-cow header. + In add-cow v1, it is fixed to 4096. + + 48 - 63: backing file format + Format of backing file. It will be filled with 0 if + backing file name offset is 0. If backing file name + offset is non-empty, it must be non-empty. It is coded + in free-form ASCII, and is not NUL-terminated. Zero + padded on the right. + + 64 - 79: image file format + Format of image file. It must be non-empty. It is coded + in free-form ASCII, and is not NUL-terminated. Zero + padded on the right. + + 80 - [HEADER_SIZE - 1]: + It is used to make sure COW bitmap field starts at the + HEADER_SIZE byte, backing file name and image file name + will be stored here. The bytes that are not pointing to + backing file and image file names must be set to 0. + +== COW bitmap == + +The "COW bitmap" field starts at offset HEADER_SIZE, stores a bitmap related to +backing file and image file. It is tracking whether the sector in image file +is allocated or not. + +Each bit in the bitmap tracks one cluster's status. For example, if cluster +bit is 16, then each bit tracks one cluster, (1 << 16) = 65536 bytes. The +image file size is rounded up to cluster size (where any bytes in the +last cluster that do not fit in the image are ignored), then if the +number of clusters is not a multiple of 8, then remaining bits in the +bitmap will be set to 0. + +The size of bitmap is calculated according to virtual size of image file, and +the size of bitmap should be multiple of add-cow file's cluster size, the bits +not used will be set to 0. Within each byte, the least significant bit covers +the first cluster. Bit orders in one byte look like: + +----+----+----+----+----+----+----+----+ + | b7 | b6 | b5 | b4 | b3 | b2 | b1 | b0 | + +----+----+----+----+----+----+----+----+ + +If the bit is 0, it indicates the sector has not been allocated in image file, +data should be loaded from backing file while reading; if the bit is 1, it +indicates the related sector has been dirty, should be loaded from image file +while reading. Writing to a sector causes the corresponding bit to be set to 1. +If there is no backing file, or if the image file is larger than the backing +file and the offset is beyond the end of the backing file, then the data should +be read as all zero bytes instead. + +If raw image is not an even multiple of cluster bytes, bits that correspond to +bytes beyond the raw file size in add-cow must be written as 0 and must be +ignored when reading. + +Image file name and backing file name must NOT be the same, we prevent this +while creating add-cow files via qemu-img. If image file name and backing file +name are the same, the add-cow image must be treated as invalid.