Patchwork [2/4] e2image: perform in-place move

login
register
mail settings
Submitter Phillip Susi
Date Dec. 16, 2013, 4:33 p.m.
Message ID <1387211599-25789-1-git-send-email-psusi@ubuntu.com>
Download mbox | patch
Permalink /patch/301763/
State Superseded
Headers show

Comments

Phillip Susi - Dec. 16, 2013, 4:33 p.m.
If given at least one offset and only one file, assume source
and dest are the same, and do an in place move.

Signed-off-by: Phillip Susi <psusi@ubuntu.com>
---
 misc/e2image.8.in |  3 +++
 misc/e2image.c    | 67 +++++++++++++++++++++++++++++++++++++++++++++++++------
 2 files changed, 63 insertions(+), 7 deletions(-)
Theodore Ts'o - Dec. 25, 2013, 4:58 a.m.
On Mon, Dec 16, 2013 at 11:33:19AM -0500, Phillip Susi wrote:
> If given at least one offset and only one file, assume source
> and dest are the same, and do an in place move.
> 
> Signed-off-by: Phillip Susi <psusi@ubuntu.com>

Thanks, applied.

				- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Theodore Ts'o - Dec. 26, 2013, 4:48 a.m.
On Tue, Dec 24, 2013 at 11:58:26PM -0500, Theodore Ts'o wrote:
> On Mon, Dec 16, 2013 at 11:33:19AM -0500, Phillip Susi wrote:
> > If given at least one offset and only one file, assume source
> > and dest are the same, and do an in place move.
> > 
> > Signed-off-by: Phillip Susi <psusi@ubuntu.com>
> 
> Thanks, applied.

Sigh, I've had to revert this change since it is causing test
failures.  **Please** run the regression test suite before you submit
patches.

					- Ted

Contents of i_e2image.failed:

e2image -r i_e2image/image1024.orig _image.raw
e2image 1.42.8 (20-Jun-2013)
e2image -Q i_e2image/image1024.orig _image.qcow2
e2image 1.42.8 (20-Jun-2013)
e2image -r _image.qcow2 _image.qcow2.raw
e2image 1.42.8 (20-Jun-2013)
e2image -r i_e2image/image2048.orig _image.raw
e2image 1.42.8 (20-Jun-2013)
e2image -Q i_e2image/image2048.orig _image.qcow2
e2image 1.42.8 (20-Jun-2013)
e2image -r _image.qcow2 _image.qcow2.raw
e2image 1.42.8 (20-Jun-2013)
e2image -r i_e2image/image4096.orig _image.raw
e2image 1.42.8 (20-Jun-2013)
e2image -Q i_e2image/image4096.orig _image.qcow2
e2image 1.42.8 (20-Jun-2013)
e2image -r _image.qcow2 _image.qcow2.raw
e2image 1.42.8 (20-Jun-2013)
md5sums:
i_e2image/image1024.orig
d34914e0da07bdae80ab02288118fae2  image1024.orig
bbef4e50d7237546c7d9c521d3df5b68  _image.raw
1d4eb39452bed097dcd2c5bcd57180e6  _image.qcow2
bbef4e50d7237546c7d9c521d3df5b68  _image.qcow2.raw
i_e2image/image2048.orig
aa9f702de181188f2a6d2c5158686c09  image2048.orig
e097e571c809ec5418f060e2a2ec1737  _image.raw
52dba396b0fc338b99defcb086058d80  _image.qcow2
e097e571c809ec5418f060e2a2ec1737  _image.qcow2.raw
i_e2image/image4096.orig
1d3e7f15b2ce9ca07aa23c32951c5176  image4096.orig
ebaddd9f5a62610d789adbbec4ee7f46  _image.raw
a72165eaffb6900889108fbd05002822  _image.qcow2
ebaddd9f5a62610d789adbbec4ee7f46  _image.qcow2.raw

8,10c8,10
< e6f8410d0690ef551bee0c2c0c642d8c  _image.raw
< dbbd9aa97c6c946b9122586bbd2a325a  _image.qcow2
< e6f8410d0690ef551bee0c2c0c642d8c  _image.qcow2.raw
---
> e097e571c809ec5418f060e2a2ec1737  _image.raw
> 52dba396b0fc338b99defcb086058d80  _image.qcow2
> e097e571c809ec5418f060e2a2ec1737  _image.qcow2.raw
13,15c13,15
< 734119dd8f240a33704139f8cdd8127c  _image.raw
< 85fdbf5a8451b24b36ab82a02196deb9  _image.qcow2
< 734119dd8f240a33704139f8cdd8127c  _image.qcow2.raw
---
> ebaddd9f5a62610d789adbbec4ee7f46  _image.raw
> a72165eaffb6900889108fbd05002822  _image.qcow2
> ebaddd9f5a62610d789adbbec4ee7f46  _image.qcow2.raw
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Theodore Ts'o - Dec. 26, 2013, 5:24 a.m.
On Wed, Dec 25, 2013 at 11:48:57PM -0500, Theodore Ts'o wrote:
> Sigh, I've had to revert this change since it is causing test
> failures.  **Please** run the regression test suite before you submit
> patches.

I figured out the problem.  The issue is that you changed e2image to
no longer truncate the output file (i.e., you removed O_TRUNC).

The test failure was caused by the fact that previously written data
blocks that were in unallocated blocks weren't getting over written,
and so this would cause the md5 checksums to break.  I can work around
this in the test by deleting the output files, but I think I'll also
modify your comment to avoid the truncate only when trying to create a
raw image dump.  For qcow2 and normal e2image files, it's better to
keep the existing behavior of truncating the output when we first open
it.

This change of your also turned up a bug which I had to fix up.  It
could happen today when people used e2image to copy a file system to
another block device:

commit aa2c743331f392e49d5b168b5e3d8c8a8084b8b4
Author: Theodore Ts'o <tytso@mit.edu>
Date:   Wed Dec 25 16:33:52 2013 -0500

    e2image: only skip writing zero blocks when writing to a new file
    
    The e2image progam was originally intended to create image files.
    However, some people have started using e2image to copy a file system
    from one block device to another, since it is more efficient than
    using dd because it only copies the blocks which are in use.  If we
    are doing this, however, we must not skip writing blocks which are all
    zero in the source device, since they may not be zero in the
    destination device.
    
    Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
    Cc: Phillip Susi <psusi@ubuntu.com>

						- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/misc/e2image.8.in b/misc/e2image.8.in
index 86d3dfc..3d3aa18 100644
--- a/misc/e2image.8.in
+++ b/misc/e2image.8.in
@@ -227,6 +227,9 @@  MiB for a partition table with:
 \	\fBe2image -arO 1048576 /dev/sda1 img\fR
 .br
 .PP
+If you specify at least one offset, and only one file, an in-place
+move will be performed, allowing you to safely move the filesystem
+from one offset to another.
 .SH AUTHOR
 .B e2image
 was written by Theodore Ts'o (tytso@mit.edu).
diff --git a/misc/e2image.c b/misc/e2image.c
index f7f2e90..2dcdb2e 100644
--- a/misc/e2image.c
+++ b/misc/e2image.c
@@ -56,6 +56,7 @@  char all_data;
 char output_is_blk;
 /* writing to blk device: don't skip zeroed blocks */
 blk64_t source_offset, dest_offset;
+char move_mode;
 
 static void lseek_error_and_exit(int errnum)
 {
@@ -488,6 +489,9 @@  static void output_meta_data_blocks(ext2_filsys fs, int fd)
 	blk64_t		blk;
 	char		*buf, *zero_buf;
 	int		sparse = 0;
+	blk64_t		start = 0;
+	blk64_t		distance = 0;
+	blk64_t		end = ext2fs_blocks_count(fs->super);
 
 	retval = ext2fs_get_mem(fs->blocksize, &buf);
 	if (retval) {
@@ -499,7 +503,19 @@  static void output_meta_data_blocks(ext2_filsys fs, int fd)
 		com_err(program_name, retval, "while allocating buffer");
 		exit(1);
 	}
-	for (blk = 0; blk < ext2fs_blocks_count(fs->super); blk++) {
+	/* when doing an in place move to the right, you can't start
+	   at the beginning or you will overwrite data, so instead
+	   divide the fs up into distance size chunks and write them
+	   in reverse. */
+	if (move_mode && dest_offset > source_offset) {
+		distance = (dest_offset - source_offset) / fs->blocksize;
+		if (distance < ext2fs_blocks_count(fs->super))
+			start = ext2fs_blocks_count(fs->super) - distance;
+	}
+more_blocks:
+	if (distance)
+		ext2fs_llseek (fd, (start * fs->blocksize) + dest_offset, SEEK_SET);
+	for (blk = start; blk < end; blk++) {
 		if ((blk >= fs->super->s_first_data_block) &&
 		    ext2fs_test_block_bitmap2(meta_block_map, blk)) {
 			retval = io_channel_read_blk64(fs->io, blk, 1, buf);
@@ -528,9 +544,31 @@  static void output_meta_data_blocks(ext2_filsys fs, int fd)
 			}
 		}
 	}
+	if (distance && start) {
+		if (start < distance) {
+			end = start;
+			start = 0;
+		} else {
+			end -= distance;
+			start -= distance;
+			if (end < distance) {
+				/* past overlap, do rest in one go */
+				end = start;
+				start = 0;
+			}
+		}
+		sparse = 0;
+		goto more_blocks;
+	}
 #ifdef HAVE_FTRUNCATE64
 	if (sparse) {
-		ext2_loff_t offset = ext2fs_llseek(fd, sparse, SEEK_CUR);
+		ext2_loff_t offset;
+		if (distance)
+			offset = ext2fs_llseek(
+				fd,
+				fs->blocksize * ext2fs_blocks_count(fs->super) + dest_offset,
+				SEEK_SET);
+		else offset = ext2fs_llseek(fd, sparse, SEEK_CUR);
 
 		if (offset < 0)
 			lseek_error_and_exit(errno);
@@ -538,7 +576,7 @@  static void output_meta_data_blocks(ext2_filsys fs, int fd)
 			write_block(fd, zero_buf, -1, 1, -1);
 	}
 #else
-	if (sparse)
+	if (sparse && !distance)
 		write_block(fd, zero_buf, sparse-1, 1, -1);
 #endif
 	ext2fs_free_mem(&zero_buf);
@@ -1300,7 +1338,10 @@  int main (int argc, char ** argv)
 		default:
 			usage();
 		}
-	if (optind != argc - 2 )
+	if (optind == argc - 1 &&
+	    (source_offset || dest_offset))
+		    move_mode = 1;
+	else if (optind != argc - 2 )
 		usage();
 
 	if (all_data && !img_type) {
@@ -1313,8 +1354,20 @@  int main (int argc, char ** argv)
 			"Offsets are only allowed with raw images.");
 		exit(1);
 	}
+	if (move_mode && img_type != E2IMAGE_RAW) {
+		com_err(program_name, 0,
+			"Move mode is only allowed with raw images.");
+		exit(1);
+	}
+	if (move_mode && !all_data) {
+		com_err(program_name, 0,
+			"Move mode requires all data mode.");
+		exit(1);
+	}
 	device_name = argv[optind];
-	image_fn = argv[optind+1];
+	if (move_mode)
+		image_fn = device_name;
+	else image_fn = argv[optind+1];
 
 	if (flags & E2IMAGE_INSTALL_FLAG) {
 		install_image(device_name, image_fn, img_type);
@@ -1344,10 +1397,10 @@  skip_device:
 	if (strcmp(image_fn, "-") == 0)
 		fd = 1;
 	else {
-		fd = ext2fs_open_file(image_fn, O_CREAT|O_TRUNC|O_WRONLY, 0600);
+		fd = ext2fs_open_file(image_fn, O_CREAT|O_WRONLY, 0600);
 		if (fd < 0) {
 			com_err(program_name, errno,
-				_("while trying to open %s"), argv[optind+1]);
+				_("while trying to open %s"), image_fn);
 			exit(1);
 		}
 	}