From patchwork Mon Oct 7 16:04:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1172888 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46n55P0CVfz9sPJ for ; Tue, 8 Oct 2019 03:09:11 +1100 (AEDT) Received: from localhost ([::1]:47000 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iHVZL-0005T8-98 for incoming@patchwork.ozlabs.org; Mon, 07 Oct 2019 12:09:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54747) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iHVVN-0001d9-Hh for qemu-devel@nongnu.org; Mon, 07 Oct 2019 12:05:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iHVVL-0006Xa-W8 for qemu-devel@nongnu.org; Mon, 07 Oct 2019 12:05:01 -0400 Received: from relay.sw.ru ([185.231.240.75]:56606) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iHVVH-0006SH-Ef; Mon, 07 Oct 2019 12:04:55 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.2) (envelope-from ) id 1iHVVD-0003oB-Uk; Mon, 07 Oct 2019 19:04:52 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v7 1/2] docs: improve qcow2 spec about extending image header Date: Mon, 7 Oct 2019 19:04:50 +0300 Message-Id: <20191007160451.27334-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191007160451.27334-1-vsementsov@virtuozzo.com> References: <20191007160451.27334-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, den@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, dplotnikov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Make it more obvious how to add new fields to the version 3 header and how to interpret them. Signed-off-by: Vladimir Sementsov-Ogievskiy --- docs/interop/qcow2.txt | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index af5711e533..3f2855593f 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -79,9 +79,9 @@ The first cluster of a qcow2 image contains the file header: Offset into the image file at which the snapshot table starts. Must be aligned to a cluster boundary. -If the version is 3 or higher, the header has the following additional fields. -For version 2, the values are assumed to be zero, unless specified otherwise -in the description of a field. +For version 2, header is always 72 bytes length and finishes here. +For version 3 or higher the header length is at least 104 bytes and has at +least next five fields, up to the @header_length field. 72 - 79: incompatible_features Bitmask of incompatible features. An implementation must @@ -165,6 +165,26 @@ in the description of a field. Length of the header structure in bytes. For version 2 images, the length is always assumed to be 72 bytes. +Additional fields (version 3 and higher) + +The following fields of the header are optional: if software don't know how to +interpret the field, it may safely ignore it. Still the field must be kept as is +when rewriting the image. @header_length must be bound to the end of one of +these fields (or to @header_length field end itself, to be 104 bytes). +This definition implies the following: +1. Software may support some of these optional fields and ignore the others, + which means that features may be backported to downstream Qemu independently. +2. Software may check @header_length, if it knows optional fields specification + enough (knows about the field which exceeds @header_length). +3. If @header_length is higher than the highest field end that software knows, + it should assume that additional fields are correct, @header_length is + correct and keep @header_length and additional unknown fields as is on + rewriting the image. +3. If we want to add incompatible field (or a field, for which some its values + would be incompatible), it must be accompanied by incompatible feature bit. + + < ... No additional fields in the header currently ... > + Directly after the image header, optional sections called header extensions can be stored. Each extension has a structure like the following: From patchwork Mon Oct 7 16:04:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1172890 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46n58G6hDWz9sP7 for ; Tue, 8 Oct 2019 03:11:42 +1100 (AEDT) Received: from localhost ([::1]:47050 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iHVbn-0007sP-Sy for incoming@patchwork.ozlabs.org; Mon, 07 Oct 2019 12:11:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54743) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iHVVN-0001cT-3I for qemu-devel@nongnu.org; Mon, 07 Oct 2019 12:05:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iHVVL-0006XK-L4 for qemu-devel@nongnu.org; Mon, 07 Oct 2019 12:05:00 -0400 Received: from relay.sw.ru ([185.231.240.75]:56598) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iHVVH-0006SG-7I; Mon, 07 Oct 2019 12:04:55 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.2) (envelope-from ) id 1iHVVE-0003oB-8Y; Mon, 07 Oct 2019 19:04:52 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v7 2/2] docs: qcow2: introduce compression type feature Date: Mon, 7 Oct 2019 19:04:51 +0300 Message-Id: <20191007160451.27334-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191007160451.27334-1-vsementsov@virtuozzo.com> References: <20191007160451.27334-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, den@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, dplotnikov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" The patch add new additional field to qcow2 header: compression_type, which specifies compression type. If field is absent or zero, default compression type is set: ZLIB, which corresponds to current behavior. New compression type (ZSTD) is to be added in further commit. Suggested-by: Denis Plotnikov Signed-off-by: Vladimir Sementsov-Ogievskiy --- docs/interop/qcow2.txt | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index 3f2855593f..82ec0845d8 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -109,6 +109,12 @@ least next five fields, up to the @header_length field. An External Data File Name header extension may be present if this bit is set. + Bit 3: Compression type bit. If this bit is set, + non-default compression is used for compressed + clusters. In this case, @header_length must + be at least 108 and @compression_type field + must be non-zero. + Bits 3-63: Reserved (set to 0) 80 - 87: compatible_features @@ -183,7 +189,18 @@ This definition implies the following: 3. If we want to add incompatible field (or a field, for which some its values would be incompatible), it must be accompanied by incompatible feature bit. - < ... No additional fields in the header currently ... > + 104: compression_type + Defines the compression method used for compressed clusters. + A single compression type is applied to all compressed image + clusters. + If incompatible compression type bit is set: the field must + exist (i.e. @header_length >= 105) and must be non-zero ( + which means non-zlib compression type) + If incompatible compression type bit is unset: the field + may not exist (if @header_length < 105) or it must be zero + (which means zlib). + Available compression type values: + 0: zlib Directly after the image header, optional sections called header extensions can be stored. Each extension has a structure like the following: