diff mbox

No change in userland tools after resizing qcow2 image

Message ID CADsSMSdfSFRBnwW94mMjwRoz5KOUY5P8CeZqDd5-VQBazD9TqA@mail.gmail.com
State New
Headers show

Commit Message

Taeha Kim July 19, 2015, 9:24 a.m. UTC
Hello,


There is no change in userland tools after resizing qcow2 image except
file utility.

For example when resize qcow2 image, the "file" utility is detectable
increased size.
However, the "ls", “stat”, and “du” utility still don't know how many
change size of image is changed.
The following patch enables to let userland tools - ls, du, stat -
know and apply changed size after resizing qcow2 image created by the
qemu-img tool.
Currently, “file” utility can only know without this patch.

In addtion, "file" utility sometimes don't know qcow2 image format's
version as follows.

$ file 100G.qcow2
100G.qcow2: QEMU QCOW Image (Unknown)


So, user can't have any information about qcow2 image size.

=====================


$ ./qemu/qemu-img --version
qemu-img version 2.3.91, Copyright (c) 2004-2008 Fabrice Bellard

The how-to-reproduce is as follows with three reproduction.

First let’s say that we create a qcow2 image using qemu-img tool as
follows. There is still no problem.

$ ./qemu/qemu-img create -f qcow2 -o preallocation=metadata 100G.qcow2 100G
Formatting '100G.qcow2', fmt=qcow2 size=107374182400 encryption=off
cluster_size=65536 preallocation='metadata' lazy_refcounts=off
refcount_bits=16

$ ./qemu/qemu-img check 100G.qcow2
No errors were found on the image.
1638400/1638400 = 100.00% allocated, 0.00% fragmented, 0.00% compressed clusters
Image end offset: 107390828544

$ ls -l 100G.qcow2
-rw-r--r--.   1 devjames devjames 107390828544 Jul 15 09:55 100G.qcow2

$ stat 100G.qcow2
  File: ‘100G.qcow2’
  Size: 107390828544    Blocks: 32408          IO Block: 4096   regular file
Device: fd00h/64768d    Inode: 5778245         Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/devjames)   Gid: ( 1000/devjames)
Context: unconfined_u:object_r:user_home_t:s0
Access: 2015-07-15 09:55:17.269620129 +0900
Modify: 2015-07-15 09:55:17.269620129 +0900
Change: 2015-07-15 09:55:17.269620129 +0900
 Birth: -

$ du -bb 100G.qcow2
107390828544    100G.qcow2

$ file 100G.qcow2
100G.qcow2: QEMU QCOW Image (v3), 107374182400 bytes

But, from now on there is a problem.

Second let’s say we resize the qcow2 image size from 100G to 200G
using current qemu-img without the patch.

$ ./qemu/qemu-img resize 100G.qcow2 200G
Image resized.

$ ./qemu/qemu-img check 100G.qcow2
No errors were found on the image.
1638400/3276800 = 50.00% allocated, 0.00% fragmented, 0.00% compressed clusters
Image end offset: 107390894080

$ ls -l 100G.qcow2
-rw-r--r--. 1 devjames devjames 107390832128 Jul 15 10:02 100G.qcow2

$ stat 100G.qcow2
  File: ‘100G.qcow2’
  Size: 107390832128    Blocks: 32416          IO Block: 4096   regular file
Device: fd00h/64768d    Inode: 5778245         Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/devjames)   Gid: ( 1000/devjames)
Context: unconfined_u:object_r:user_home_t:s0
Access: 2015-07-15 10:02:04.842425479 +0900
Modify: 2015-07-15 10:02:04.854425682 +0900
Change: 2015-07-15 10:02:04.854425682 +0900
 Birth: -

$ du -bb 100G.qcow2
107390832128    100G.qcow2

$ file 100G.qcow2
100G.qcow2: QEMU QCOW Image (v3), 214748364800 bytes

We can see that “ls”, “stat”, and “du” utilities don’t know change
size of qcow2 image except “file” one.

Third let’s say we resize the qcow2 image size from 100G to 200G using
qemu-img with the patch.

$ ./qemu/qemu-img resize 100G.qcow2 200G
Image resized.

$ ./qemu/qemu-img check 100G.qcow2
No errors were found on the image.
1638400/3276800 = 50.00% allocated, 0.00% fragmented, 0.00% compressed clusters
Image end offset: 107390894080

$ ls -l 100G.qcow2
-rw-r--r--. 1 devjames devjames 214748364800 Jul 15 10:08 100G.qcow2

$ stat 100G.qcow2
  File: ‘100G.qcow2’
  Size: 214748364800    Blocks: 32416          IO Block: 4096   regular file
Device: fd00h/64768d    Inode: 5778245         Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/devjames)   Gid: ( 1000/devjames)
Context: unconfined_u:object_r:user_home_t:s0
Access: 2015-07-15 10:04:37.595018501 +0900
Modify: 2015-07-15 10:08:01.622484709 +0900
Change: 2015-07-15 10:08:01.622484709 +0900
 Birth: -

$ du -bb 100G.qcow2
14748364800    100G.qcow2

$ file 100G.qcow2
100G.qcow2: QEMU QCOW Image (v3), 214748364800 bytes

Now we can see above all four utilities know change size of qcow2 image.
So I made the patch above for the "qemu-img" tool so that "ls",
“stat”, and “du” utility can be applied increased size of the qcow2
image file.

I have created a VM using qcow2 image after patching using this patch.
The 100G.qcow2 has resized 200G is detected and available.
$ qemu-kvm -smp 4 -m 4096 100G.qcow2 -cdrom
../iso/Fedora-Live-Workstation-x86_64-21-5.iso

Thanks.



Sincerely yours,
Taeha Kim
Senior Engineer, Network Functions Virtualization Part, Samsung Electronics

Comments

John Snow July 20, 2015, 4:47 p.m. UTC | #1
On 07/19/2015 05:24 AM, Taeha Kim wrote:
> Hello,
> 
> 
> There is no change in userland tools after resizing qcow2 image except
> file utility.
> 
> For example when resize qcow2 image, the "file" utility is detectable
> increased size.
> However, the "ls", “stat”, and “du” utility still don't know how many
> change size of image is changed.
> The following patch enables to let userland tools - ls, du, stat -
> know and apply changed size after resizing qcow2 image created by the
> qemu-img tool.
> Currently, “file” utility can only know without this patch.
> 
> In addtion, "file" utility sometimes don't know qcow2 image format's
> version as follows.
> 
> $ file 100G.qcow2
> 100G.qcow2: QEMU QCOW Image (Unknown)
> 
> 
> So, user can't have any information about qcow2 image size.
> 
> ======================
> Signed-off-by: Taeha Kim <kthguru@gmail.com>
> diff --git a/block.c b/block.c
> index d088ee0..93427f8 100644
> --- a/block.c
> +++ b/block.c
> @@ -2529,6 +2529,9 @@ int bdrv_truncate(BlockDriverState *bs, int64_t offset)
>         if (bs->blk) {
>             blk_dev_resize_cb(bs->blk);
>         }
> +       if (ret == 0) {
> +           ret = truncate(bs->filename, offset);
> +       }
>     }
>     return ret;
> }
> =====================
> 
> 
> $ ./qemu/qemu-img --version
> qemu-img version 2.3.91, Copyright (c) 2004-2008 Fabrice Bellard
> 
> The how-to-reproduce is as follows with three reproduction.
> 
> First let’s say that we create a qcow2 image using qemu-img tool as
> follows. There is still no problem.
> 
> $ ./qemu/qemu-img create -f qcow2 -o preallocation=metadata 100G.qcow2 100G
> Formatting '100G.qcow2', fmt=qcow2 size=107374182400 encryption=off
> cluster_size=65536 preallocation='metadata' lazy_refcounts=off
> refcount_bits=16
> 
> $ ./qemu/qemu-img check 100G.qcow2
> No errors were found on the image.
> 1638400/1638400 = 100.00% allocated, 0.00% fragmented, 0.00% compressed clusters
> Image end offset: 107390828544
> 
> $ ls -l 100G.qcow2
> -rw-r--r--.   1 devjames devjames 107390828544 Jul 15 09:55 100G.qcow2
> 
> $ stat 100G.qcow2
>   File: ‘100G.qcow2’
>   Size: 107390828544    Blocks: 32408          IO Block: 4096   regular file
> Device: fd00h/64768d    Inode: 5778245         Links: 1
> Access: (0644/-rw-r--r--)  Uid: ( 1000/devjames)   Gid: ( 1000/devjames)
> Context: unconfined_u:object_r:user_home_t:s0
> Access: 2015-07-15 09:55:17.269620129 +0900
> Modify: 2015-07-15 09:55:17.269620129 +0900
> Change: 2015-07-15 09:55:17.269620129 +0900
>  Birth: -
> 
> $ du -bb 100G.qcow2
> 107390828544    100G.qcow2
> 
> $ file 100G.qcow2
> 100G.qcow2: QEMU QCOW Image (v3), 107374182400 bytes
> 
> But, from now on there is a problem.
> 
> Second let’s say we resize the qcow2 image size from 100G to 200G
> using current qemu-img without the patch.
> 
> $ ./qemu/qemu-img resize 100G.qcow2 200G
> Image resized.
> 
> $ ./qemu/qemu-img check 100G.qcow2
> No errors were found on the image.
> 1638400/3276800 = 50.00% allocated, 0.00% fragmented, 0.00% compressed clusters
> Image end offset: 107390894080
> 
> $ ls -l 100G.qcow2
> -rw-r--r--. 1 devjames devjames 107390832128 Jul 15 10:02 100G.qcow2
> 
> $ stat 100G.qcow2
>   File: ‘100G.qcow2’
>   Size: 107390832128    Blocks: 32416          IO Block: 4096   regular file
> Device: fd00h/64768d    Inode: 5778245         Links: 1
> Access: (0644/-rw-r--r--)  Uid: ( 1000/devjames)   Gid: ( 1000/devjames)
> Context: unconfined_u:object_r:user_home_t:s0
> Access: 2015-07-15 10:02:04.842425479 +0900
> Modify: 2015-07-15 10:02:04.854425682 +0900
> Change: 2015-07-15 10:02:04.854425682 +0900
>  Birth: -
> 
> $ du -bb 100G.qcow2
> 107390832128    100G.qcow2
> 
> $ file 100G.qcow2
> 100G.qcow2: QEMU QCOW Image (v3), 214748364800 bytes
> 
> We can see that “ls”, “stat”, and “du” utilities don’t know change
> size of qcow2 image except “file” one.
> 
> Third let’s say we resize the qcow2 image size from 100G to 200G using
> qemu-img with the patch.
> 
> $ ./qemu/qemu-img resize 100G.qcow2 200G
> Image resized.
> 
> $ ./qemu/qemu-img check 100G.qcow2
> No errors were found on the image.
> 1638400/3276800 = 50.00% allocated, 0.00% fragmented, 0.00% compressed clusters
> Image end offset: 107390894080
> 
> $ ls -l 100G.qcow2
> -rw-r--r--. 1 devjames devjames 214748364800 Jul 15 10:08 100G.qcow2
> 
> $ stat 100G.qcow2
>   File: ‘100G.qcow2’
>   Size: 214748364800    Blocks: 32416          IO Block: 4096   regular file
> Device: fd00h/64768d    Inode: 5778245         Links: 1
> Access: (0644/-rw-r--r--)  Uid: ( 1000/devjames)   Gid: ( 1000/devjames)
> Context: unconfined_u:object_r:user_home_t:s0
> Access: 2015-07-15 10:04:37.595018501 +0900
> Modify: 2015-07-15 10:08:01.622484709 +0900
> Change: 2015-07-15 10:08:01.622484709 +0900
>  Birth: -
> 
> $ du -bb 100G.qcow2
> 14748364800    100G.qcow2
> 
> $ file 100G.qcow2
> 100G.qcow2: QEMU QCOW Image (v3), 214748364800 bytes
> 
> Now we can see above all four utilities know change size of qcow2 image.
> So I made the patch above for the "qemu-img" tool so that "ls",
> “stat”, and “du” utility can be applied increased size of the qcow2
> image file.
> 

It seems to me that the real issue here is that the resize command you
are issuing doesn't respect your original pre-allocation desires.

When changing the size of the qcow2 file, it generally just updates the
headers and doesn't pre-allocate those storage blocks. The actual size
of the .qcow2 image doesn't take up disk space for empty blocks. That's
part of its design. The reason 'file' can tell the "virtual" size of the
disk is because it is reading the header to do so.

ls and friends are telling you the size of the actual file, as you've
noticed.

Changing the resize mechanisms to automatically truncate to achieve
pre-allocation is wrong, I'm afraid.

I think if anything, we'd want to allow the qemu-img tool to take
options for the resize, concerning pre-allocation strategy.

--js

> I have created a VM using qcow2 image after patching using this patch.
> The 100G.qcow2 has resized 200G is detected and available.
> $ qemu-kvm -smp 4 -m 4096 100G.qcow2 -cdrom
> ../iso/Fedora-Live-Workstation-x86_64-21-5.iso
> 
> Thanks.
> 
> 
> 
> Sincerely yours,
> Taeha Kim
> Senior Engineer, Network Functions Virtualization Part, Samsung Electronics
>
김태하 July 21, 2015, 1:56 a.m. UTC | #2
> On 07/19/2015 05:24 AM, Taeha Kim wrote:
> > Hello,
> >
> >
> > There is no change in userland tools after resizing qcow2 image except
> > file utility.
> >
> > For example when resize qcow2 image, the "file" utility is detectable
> > increased size.
> > However, the "ls", “stat”, and “du” utility still don't know how many
> > change size of image is changed.
> > The following patch enables to let userland tools - ls, du, stat -
> > know and apply changed size after resizing qcow2 image created by the
> > qemu-img tool.
> > Currently, “file” utility can only know without this patch.
> >
> > In addtion, "file" utility sometimes don't know qcow2 image format's
> > version as follows.
> >
> > $ file 100G.qcow2
> > 100G.qcow2: QEMU QCOW Image (Unknown)
> >
> >
> > So, user can't have any information about qcow2 image size.
> >
> > ======================
> > Signed-off-by: Taeha Kim <kthguru@gmail.com> diff --git a/block.c
> > b/block.c index d088ee0..93427f8 100644
> > --- a/block.c
> > +++ b/block.c
> > @@ -2529,6 +2529,9 @@ int bdrv_truncate(BlockDriverState *bs, int64_t
> offset)
> >         if (bs->blk) {
> >             blk_dev_resize_cb(bs->blk);
> >         }
> > +       if (ret == 0) {
> > +           ret = truncate(bs->filename, offset);
> > +       }
> >     }
> >     return ret;
> > }
> > =====================
> >
> >
> > $ ./qemu/qemu-img --version
> > qemu-img version 2.3.91, Copyright (c) 2004-2008 Fabrice Bellard
> >
> > The how-to-reproduce is as follows with three reproduction.
> >
> > First let’s say that we create a qcow2 image using qemu-img tool as
> > follows. There is still no problem.
> >
> > $ ./qemu/qemu-img create -f qcow2 -o preallocation=metadata 100G.qcow2
> > 100G Formatting '100G.qcow2', fmt=qcow2 size=107374182400
> > encryption=off
> > cluster_size=65536 preallocation='metadata' lazy_refcounts=off
> > refcount_bits=16
> >
> > $ ./qemu/qemu-img check 100G.qcow2
> > No errors were found on the image.
> > 1638400/1638400 = 100.00% allocated, 0.00% fragmented, 0.00%
> > compressed clusters Image end offset: 107390828544
> >
> > $ ls -l 100G.qcow2
> > -rw-r--r--.   1 devjames devjames 107390828544 Jul 15 09:55 100G.qcow2
> >
> > $ stat 100G.qcow2
> >   File: ‘100G.qcow2’
> >   Size: 107390828544    Blocks: 32408          IO Block: 4096   regular file
> > Device: fd00h/64768d    Inode: 5778245         Links: 1
> > Access: (0644/-rw-r--r--)  Uid: ( 1000/devjames)   Gid: ( 1000/devjames)
> > Context: unconfined_u:object_r:user_home_t:s0
> > Access: 2015-07-15 09:55:17.269620129 +0900
> > Modify: 2015-07-15 09:55:17.269620129 +0900
> > Change: 2015-07-15 09:55:17.269620129 +0900
> >  Birth: -
> >
> > $ du -bb 100G.qcow2
> > 107390828544    100G.qcow2
> >
> > $ file 100G.qcow2
> > 100G.qcow2: QEMU QCOW Image (v3), 107374182400 bytes
> >
> > But, from now on there is a problem.
> >
> > Second let’s say we resize the qcow2 image size from 100G to 200G
> > using current qemu-img without the patch.
> >
> > $ ./qemu/qemu-img resize 100G.qcow2 200G Image resized.
> >
> > $ ./qemu/qemu-img check 100G.qcow2
> > No errors were found on the image.
> > 1638400/3276800 = 50.00% allocated, 0.00% fragmented, 0.00% compressed
> > clusters Image end offset: 107390894080
> >
> > $ ls -l 100G.qcow2
> > -rw-r--r--. 1 devjames devjames 107390832128 Jul 15 10:02 100G.qcow2
> >
> > $ stat 100G.qcow2
> >   File: ‘100G.qcow2’
> >   Size: 107390832128    Blocks: 32416          IO Block: 4096   regular file
> > Device: fd00h/64768d    Inode: 5778245         Links: 1
> > Access: (0644/-rw-r--r--)  Uid: ( 1000/devjames)   Gid: ( 1000/devjames)
> > Context: unconfined_u:object_r:user_home_t:s0
> > Access: 2015-07-15 10:02:04.842425479 +0900
> > Modify: 2015-07-15 10:02:04.854425682 +0900
> > Change: 2015-07-15 10:02:04.854425682 +0900
> >  Birth: -
> >
> > $ du -bb 100G.qcow2
> > 107390832128    100G.qcow2
> >
> > $ file 100G.qcow2
> > 100G.qcow2: QEMU QCOW Image (v3), 214748364800 bytes
> >
> > We can see that “ls”, “stat”, and “du” utilities don’t know change
> > size of qcow2 image except “file” one.
> >
> > Third let’s say we resize the qcow2 image size from 100G to 200G using
> > qemu-img with the patch.
> >
> > $ ./qemu/qemu-img resize 100G.qcow2 200G Image resized.
> >
> > $ ./qemu/qemu-img check 100G.qcow2
> > No errors were found on the image.
> > 1638400/3276800 = 50.00% allocated, 0.00% fragmented, 0.00% compressed
> > clusters Image end offset: 107390894080
> >
> > $ ls -l 100G.qcow2
> > -rw-r--r--. 1 devjames devjames 214748364800 Jul 15 10:08 100G.qcow2
> >
> > $ stat 100G.qcow2
> >   File: ‘100G.qcow2’
> >   Size: 214748364800    Blocks: 32416          IO Block: 4096   regular file
> > Device: fd00h/64768d    Inode: 5778245         Links: 1
> > Access: (0644/-rw-r--r--)  Uid: ( 1000/devjames)   Gid: ( 1000/devjames)
> > Context: unconfined_u:object_r:user_home_t:s0
> > Access: 2015-07-15 10:04:37.595018501 +0900
> > Modify: 2015-07-15 10:08:01.622484709 +0900
> > Change: 2015-07-15 10:08:01.622484709 +0900
> >  Birth: -
> >
> > $ du -bb 100G.qcow2
> > 14748364800    100G.qcow2
> >
> > $ file 100G.qcow2
> > 100G.qcow2: QEMU QCOW Image (v3), 214748364800 bytes
> >
> > Now we can see above all four utilities know change size of qcow2 image.
> > So I made the patch above for the "qemu-img" tool so that "ls",
> > “stat”, and “du” utility can be applied increased size of the qcow2
> > image file.
> >
> 

Hello,

Sorry for late response. At last, I found how to add ">" when the email is replied in outlook. :)

> It seems to me that the real issue here is that the resize command you are
> issuing doesn't respect your original pre-allocation desires.
> 

You're right. At first I thought that the change of resizing with pre-allocation needs to be applied in user space.
And the "preallocation" option doesn't have full, for example, "preallocation=full".
I expected that this will enable the qcow2 image to fill increased size fully when preallocated image is resized.

> When changing the size of the qcow2 file, it generally just updates the
> headers and doesn't pre-allocate those storage blocks. The actual size of
> the .qcow2 image doesn't take up disk space for empty blocks. That's part
> of its design. The reason 'file' can tell the "virtual" size of the disk
> is because it is reading the header to do so.
> 

If it updates only headers, I think the "preallocation=metadata" is right. But user still cannot know the exact size of the qcow2 image without the qemu-img tool.
If people who has only qcow2 images will guess or need many time to know the image size. Because currently qemu-img doesn't support shrinking, perhaps the user will have slack space unintentionally.

> ls and friends are telling you the size of the actual file, as you've
> noticed.
> 
> Changing the resize mechanisms to automatically truncate to achieve pre-
> allocation is wrong, I'm afraid.

I agree with you. Could you tell me your opinion about adding the resize mechanisms iff use "preallocation=full" or "preallocation=data"?
Of course, new patch will have other approach with internal functions related to qcow2 structure, not just simple userspace API.

> 
> I think if anything, we'd want to allow the qemu-img tool to take options
> for the resize, concerning pre-allocation strategy.

I didn't understand what you were saying. Could you please explain to me one more time?
What is exactly pre-allocation strategy? You mean that current resize function cannot change any more, or have to keep "as-is"?

> 
> --js
> 

Thank you very much.

Sincerely,
Taeha


> > I have created a VM using qcow2 image after patching using this patch.
> > The 100G.qcow2 has resized 200G is detected and available.
> > $ qemu-kvm -smp 4 -m 4096 100G.qcow2 -cdrom
> > ../iso/Fedora-Live-Workstation-x86_64-21-5.iso
> >
> > Thanks.
> >
> >
> >
> > Sincerely yours,
> > Taeha Kim
> > Senior Engineer, Network Functions Virtualization Part, Samsung
> > Electronics
> >
John Snow July 29, 2015, 7:40 p.m. UTC | #3
On 07/20/2015 09:56 PM, 김태하 wrote:
>> On 07/19/2015 05:24 AM, Taeha Kim wrote:
>>> Hello,
>>>
>>>
>>> There is no change in userland tools after resizing qcow2 image except
>>> file utility.
>>>
>>> For example when resize qcow2 image, the "file" utility is detectable
>>> increased size.
>>> However, the "ls", “stat”, and “du” utility still don't know how many
>>> change size of image is changed.
>>> The following patch enables to let userland tools - ls, du, stat -
>>> know and apply changed size after resizing qcow2 image created by the
>>> qemu-img tool.
>>> Currently, “file” utility can only know without this patch.
>>>
>>> In addtion, "file" utility sometimes don't know qcow2 image format's
>>> version as follows.
>>>
>>> $ file 100G.qcow2
>>> 100G.qcow2: QEMU QCOW Image (Unknown)
>>>
>>>
>>> So, user can't have any information about qcow2 image size.
>>>
>>> ======================
>>> Signed-off-by: Taeha Kim <kthguru@gmail.com> diff --git a/block.c
>>> b/block.c index d088ee0..93427f8 100644
>>> --- a/block.c
>>> +++ b/block.c
>>> @@ -2529,6 +2529,9 @@ int bdrv_truncate(BlockDriverState *bs, int64_t
>> offset)
>>>         if (bs->blk) {
>>>             blk_dev_resize_cb(bs->blk);
>>>         }
>>> +       if (ret == 0) {
>>> +           ret = truncate(bs->filename, offset);
>>> +       }
>>>     }
>>>     return ret;
>>> }
>>> =====================
>>>
>>>
>>> $ ./qemu/qemu-img --version
>>> qemu-img version 2.3.91, Copyright (c) 2004-2008 Fabrice Bellard
>>>
>>> The how-to-reproduce is as follows with three reproduction.
>>>
>>> First let’s say that we create a qcow2 image using qemu-img tool as
>>> follows. There is still no problem.
>>>
>>> $ ./qemu/qemu-img create -f qcow2 -o preallocation=metadata 100G.qcow2
>>> 100G Formatting '100G.qcow2', fmt=qcow2 size=107374182400
>>> encryption=off
>>> cluster_size=65536 preallocation='metadata' lazy_refcounts=off
>>> refcount_bits=16
>>>
>>> $ ./qemu/qemu-img check 100G.qcow2
>>> No errors were found on the image.
>>> 1638400/1638400 = 100.00% allocated, 0.00% fragmented, 0.00%
>>> compressed clusters Image end offset: 107390828544
>>>
>>> $ ls -l 100G.qcow2
>>> -rw-r--r--.   1 devjames devjames 107390828544 Jul 15 09:55 100G.qcow2
>>>
>>> $ stat 100G.qcow2
>>>   File: ‘100G.qcow2’
>>>   Size: 107390828544    Blocks: 32408          IO Block: 4096   regular file
>>> Device: fd00h/64768d    Inode: 5778245         Links: 1
>>> Access: (0644/-rw-r--r--)  Uid: ( 1000/devjames)   Gid: ( 1000/devjames)
>>> Context: unconfined_u:object_r:user_home_t:s0
>>> Access: 2015-07-15 09:55:17.269620129 +0900
>>> Modify: 2015-07-15 09:55:17.269620129 +0900
>>> Change: 2015-07-15 09:55:17.269620129 +0900
>>>  Birth: -
>>>
>>> $ du -bb 100G.qcow2
>>> 107390828544    100G.qcow2
>>>
>>> $ file 100G.qcow2
>>> 100G.qcow2: QEMU QCOW Image (v3), 107374182400 bytes
>>>
>>> But, from now on there is a problem.
>>>
>>> Second let’s say we resize the qcow2 image size from 100G to 200G
>>> using current qemu-img without the patch.
>>>
>>> $ ./qemu/qemu-img resize 100G.qcow2 200G Image resized.
>>>
>>> $ ./qemu/qemu-img check 100G.qcow2
>>> No errors were found on the image.
>>> 1638400/3276800 = 50.00% allocated, 0.00% fragmented, 0.00% compressed
>>> clusters Image end offset: 107390894080
>>>
>>> $ ls -l 100G.qcow2
>>> -rw-r--r--. 1 devjames devjames 107390832128 Jul 15 10:02 100G.qcow2
>>>
>>> $ stat 100G.qcow2
>>>   File: ‘100G.qcow2’
>>>   Size: 107390832128    Blocks: 32416          IO Block: 4096   regular file
>>> Device: fd00h/64768d    Inode: 5778245         Links: 1
>>> Access: (0644/-rw-r--r--)  Uid: ( 1000/devjames)   Gid: ( 1000/devjames)
>>> Context: unconfined_u:object_r:user_home_t:s0
>>> Access: 2015-07-15 10:02:04.842425479 +0900
>>> Modify: 2015-07-15 10:02:04.854425682 +0900
>>> Change: 2015-07-15 10:02:04.854425682 +0900
>>>  Birth: -
>>>
>>> $ du -bb 100G.qcow2
>>> 107390832128    100G.qcow2
>>>
>>> $ file 100G.qcow2
>>> 100G.qcow2: QEMU QCOW Image (v3), 214748364800 bytes
>>>
>>> We can see that “ls”, “stat”, and “du” utilities don’t know change
>>> size of qcow2 image except “file” one.
>>>
>>> Third let’s say we resize the qcow2 image size from 100G to 200G using
>>> qemu-img with the patch.
>>>
>>> $ ./qemu/qemu-img resize 100G.qcow2 200G Image resized.
>>>
>>> $ ./qemu/qemu-img check 100G.qcow2
>>> No errors were found on the image.
>>> 1638400/3276800 = 50.00% allocated, 0.00% fragmented, 0.00% compressed
>>> clusters Image end offset: 107390894080
>>>
>>> $ ls -l 100G.qcow2
>>> -rw-r--r--. 1 devjames devjames 214748364800 Jul 15 10:08 100G.qcow2
>>>
>>> $ stat 100G.qcow2
>>>   File: ‘100G.qcow2’
>>>   Size: 214748364800    Blocks: 32416          IO Block: 4096   regular file
>>> Device: fd00h/64768d    Inode: 5778245         Links: 1
>>> Access: (0644/-rw-r--r--)  Uid: ( 1000/devjames)   Gid: ( 1000/devjames)
>>> Context: unconfined_u:object_r:user_home_t:s0
>>> Access: 2015-07-15 10:04:37.595018501 +0900
>>> Modify: 2015-07-15 10:08:01.622484709 +0900
>>> Change: 2015-07-15 10:08:01.622484709 +0900
>>>  Birth: -
>>>
>>> $ du -bb 100G.qcow2
>>> 14748364800    100G.qcow2
>>>
>>> $ file 100G.qcow2
>>> 100G.qcow2: QEMU QCOW Image (v3), 214748364800 bytes
>>>
>>> Now we can see above all four utilities know change size of qcow2 image.
>>> So I made the patch above for the "qemu-img" tool so that "ls",
>>> “stat”, and “du” utility can be applied increased size of the qcow2
>>> image file.
>>>
>>
> 
> Hello,
> 
> Sorry for late response. At last, I found how to add ">" when the email is replied in outlook. :)
> 
>> It seems to me that the real issue here is that the resize command you are
>> issuing doesn't respect your original pre-allocation desires.
>>
> 
> You're right. At first I thought that the change of resizing with pre-allocation needs to be applied in user space.
> And the "preallocation" option doesn't have full, for example, "preallocation=full".
> I expected that this will enable the qcow2 image to fill increased size fully when preallocated image is resized.
> 
>> When changing the size of the qcow2 file, it generally just updates the
>> headers and doesn't pre-allocate those storage blocks. The actual size of
>> the .qcow2 image doesn't take up disk space for empty blocks. That's part
>> of its design. The reason 'file' can tell the "virtual" size of the disk
>> is because it is reading the header to do so.
>>
> 
> If it updates only headers, I think the "preallocation=metadata" is right. But user still cannot know the exact size of the qcow2 image without the qemu-img tool.
> If people who has only qcow2 images will guess or need many time to know the image size. Because currently qemu-img doesn't support shrinking, perhaps the user will have slack space unintentionally.
> 
>> ls and friends are telling you the size of the actual file, as you've
>> noticed.
>>
>> Changing the resize mechanisms to automatically truncate to achieve pre-
>> allocation is wrong, I'm afraid.
> 
> I agree with you. Could you tell me your opinion about adding the resize mechanisms iff use "preallocation=full" or "preallocation=data"?
> Of course, new patch will have other approach with internal functions related to qcow2 structure, not just simple userspace API.
> 
>>
>> I think if anything, we'd want to allow the qemu-img tool to take options
>> for the resize, concerning pre-allocation strategy.
> 
> I didn't understand what you were saying. Could you please explain to me one more time?
> What is exactly pre-allocation strategy? You mean that current resize function cannot change any more, or have to keep "as-is"?
> 

What I mean to say is that the qemu-img tool doesn't currently appear to
accept arguments for its resize function. That is, there's no way to say
e.g.

qemu-img resize -o preallocation=metadata 100G.qcow2 200G

because there is no valid "-o" flag for "resize" currently.

If it is your goal to resize a preallocated image to a new preallocated
image using the qemu-img tool, I think changes to qemu-img are what we want.

>>
>> --js
>>
> 
> Thank you very much.
> 
> Sincerely,
> Taeha
> 
> 
>>> I have created a VM using qcow2 image after patching using this patch.
>>> The 100G.qcow2 has resized 200G is detected and available.
>>> $ qemu-kvm -smp 4 -m 4096 100G.qcow2 -cdrom
>>> ../iso/Fedora-Live-Workstation-x86_64-21-5.iso
>>>
>>> Thanks.
>>>
>>>
>>>
>>> Sincerely yours,
>>> Taeha Kim
>>> Senior Engineer, Network Functions Virtualization Part, Samsung
>>> Electronics
>>>
>
diff mbox

Patch

======================
Signed-off-by: Taeha Kim <kthguru@gmail.com>
diff --git a/block.c b/block.c
index d088ee0..93427f8 100644
--- a/block.c
+++ b/block.c
@@ -2529,6 +2529,9 @@  int bdrv_truncate(BlockDriverState *bs, int64_t offset)
        if (bs->blk) {
            blk_dev_resize_cb(bs->blk);
        }
+       if (ret == 0) {
+           ret = truncate(bs->filename, offset);
+       }
    }
    return ret;
}