| Message ID | 20190820164616.4072-1-dplotnikov@virtuozzo.com |
|---|---|
| State | New |
| Headers | show |
| Series | [v10] qemu-io: add pattern file for write command | expand |
On 8/20/19 11:46 AM, Denis Plotnikov wrote: > The patch allows to provide a pattern file for write > command. There was no similar ability before. > > Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> > --- > @@ -983,8 +1057,9 @@ static int write_f(BlockBackend *blk, int argc, char **argv) > /* Some compilers get confused and warn if this is not initialized. */ > int64_t total = 0; > int pattern = 0xcd; > + const char *file_name = NULL; > > - while ((c = getopt(argc, argv, "bcCfnpP:quz")) != -1) { > + while ((c = getopt(argc, argv, "bcCfnpP:quzs:")) != -1) { This one looks odd (I would have preserved ordering by sticking s: between q and u). But a maintainer could fix that. > switch (c) { > case 'b': > bflag = true; > @@ -1020,6 +1095,10 @@ static int write_f(BlockBackend *blk, int argc, char **argv) > case 'z': > zflag = true; > break; > + case 's': > + sflag = true; > + file_name = optarg; > + break; Likewise, sorting the cases in the same order as the getopt() listing helps in finding code during later edits. > @@ -1088,7 +1168,14 @@ static int write_f(BlockBackend *blk, int argc, char **argv) > } > > if (!zflag) { > - buf = qemu_io_alloc(blk, count, pattern); > + if (sflag) { > + buf = qemu_io_alloc_from_file(blk, count, file_name); > + if (!buf) { > + return -EINVAL; > + } > + } else { > + buf = qemu_io_alloc(blk, count, pattern); > + } Pre-existing, but it is odd that qemu_io_alloc() exit()s rather than returning NULL on huge allocation requests that can't be met. (Then again, we have an early exit on any length > 2G, and 2G allocations tend to succeed on modern development machines). Perhaps it would be nice to teach qemu-io to use blk_try_blockalign for more graceful handling even on 32-bit platforms, but that's not the problem of your patch. Option ordering is minor enough that I'm fine giving: Reviewed-by: Eric Blake <eblake@redhat.com> Now, to figure out which maintainer should take it. Perhaps you want to add a patch 2/1 that adds an iotest using this new mode, to a) ensure it doesn't regress, and b) makes it reasonable to take in through the iotest tree.
On 20.08.19 19:24, Eric Blake wrote: > On 8/20/19 11:46 AM, Denis Plotnikov wrote: >> The patch allows to provide a pattern file for write >> command. There was no similar ability before. >> >> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> >> --- > >> @@ -983,8 +1057,9 @@ static int write_f(BlockBackend *blk, int argc, char **argv) >> /* Some compilers get confused and warn if this is not initialized. */ >> int64_t total = 0; >> int pattern = 0xcd; >> + const char *file_name = NULL; >> >> - while ((c = getopt(argc, argv, "bcCfnpP:quz")) != -1) { >> + while ((c = getopt(argc, argv, "bcCfnpP:quzs:")) != -1) { > > This one looks odd (I would have preserved ordering by sticking s: > between q and u). But a maintainer could fix that. > >> switch (c) { >> case 'b': >> bflag = true; >> @@ -1020,6 +1095,10 @@ static int write_f(BlockBackend *blk, int argc, char **argv) >> case 'z': >> zflag = true; >> break; >> + case 's': >> + sflag = true; >> + file_name = optarg; >> + break; > > Likewise, sorting the cases in the same order as the getopt() listing > helps in finding code during later edits. But it is in order of the getopt() listing. ;-) >> @@ -1088,7 +1168,14 @@ static int write_f(BlockBackend *blk, int argc, char **argv) >> } >> >> if (!zflag) { >> - buf = qemu_io_alloc(blk, count, pattern); >> + if (sflag) { >> + buf = qemu_io_alloc_from_file(blk, count, file_name); >> + if (!buf) { >> + return -EINVAL; >> + } >> + } else { >> + buf = qemu_io_alloc(blk, count, pattern); >> + } > > Pre-existing, but it is odd that qemu_io_alloc() exit()s rather than > returning NULL on huge allocation requests that can't be met. (Then > again, we have an early exit on any length > 2G, and 2G allocations tend > to succeed on modern development machines). Perhaps it would be nice to > teach qemu-io to use blk_try_blockalign for more graceful handling even > on 32-bit platforms, but that's not the problem of your patch. Then again, this is qemu-io. Printing an error instead of just aborting doesn’t really help anyone. Also, the code would be wrong without an early exit on a length > INT_MAX. (Because pattern_len is an int, so the result of fread() might overflow otherwise, which would be bad.) (I just noticed that fread() might do a short read, but let’s just ignore this at this point.) > Option ordering is minor enough that I'm fine giving: > > Reviewed-by: Eric Blake <eblake@redhat.com> > > Now, to figure out which maintainer should take it. Perhaps you want to > add a patch 2/1 that adds an iotest using this new mode, to a) ensure it > doesn't regress, and b) makes it reasonable to take in through the > iotest tree. Adding a test does not seem to bad of an idea, but I don’t see how that would clarify things. Both qemu-io and the iotests are part of the block layer core: $ scripts/get_maintainer.pl -f qemu-io-cmds.c Kevin Wolf <kwolf@redhat.com> (supporter:Block layer core) Max Reitz <mreitz@redhat.com> (supporter:Block layer core) qemu-block@nongnu.org (open list:Block layer core) qemu-devel@nongnu.org (open list:All patches CC here) $ scripts/get_maintainer.pl -f tests/qemu-iotests Kevin Wolf <kwolf@redhat.com> (supporter:Block layer core) Max Reitz <mreitz@redhat.com> (supporter:Block layer core) qemu-block@nongnu.org (open list:Block layer core) qemu-devel@nongnu.org (open list:All patches CC here) So we only need to figure out whether it should be Kevin or me to take it; but Kevin is on PTO, so that decision is simple. :-) Therefor, I’ve changed the optstring (and switch case) order to be alphabetical, and applied the patch to my block branch: https://git.xanclic.moe/XanClic/qemu/commits/branch/block Thanks for the patch and the review, Max (I wouldn’t mind an iotest, but well. qemu-io itself is a testing utility, so I don’t deem it important to test it.)
On 8/20/19 1:24 PM, Eric Blake wrote: > On 8/20/19 11:46 AM, Denis Plotnikov wrote: >> The patch allows to provide a pattern file for write >> command. There was no similar ability before. >> >> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> >> --- > >> @@ -983,8 +1057,9 @@ static int write_f(BlockBackend *blk, int argc, char **argv) >> /* Some compilers get confused and warn if this is not initialized. */ >> int64_t total = 0; >> int pattern = 0xcd; >> + const char *file_name = NULL; >> >> - while ((c = getopt(argc, argv, "bcCfnpP:quz")) != -1) { >> + while ((c = getopt(argc, argv, "bcCfnpP:quzs:")) != -1) { > > This one looks odd (I would have preserved ordering by sticking s: > between q and u). But a maintainer could fix that. > >> switch (c) { >> case 'b': >> bflag = true; >> @@ -1020,6 +1095,10 @@ static int write_f(BlockBackend *blk, int argc, char **argv) >> case 'z': >> zflag = true; >> break; >> + case 's': >> + sflag = true; >> + file_name = optarg; >> + break; > > Likewise, sorting the cases in the same order as the getopt() listing > helps in finding code during later edits. > >> @@ -1088,7 +1168,14 @@ static int write_f(BlockBackend *blk, int argc, char **argv) >> } >> >> if (!zflag) { >> - buf = qemu_io_alloc(blk, count, pattern); >> + if (sflag) { >> + buf = qemu_io_alloc_from_file(blk, count, file_name); >> + if (!buf) { >> + return -EINVAL; >> + } >> + } else { >> + buf = qemu_io_alloc(blk, count, pattern); >> + } > > Pre-existing, but it is odd that qemu_io_alloc() exit()s rather than > returning NULL on huge allocation requests that can't be met. (Then > again, we have an early exit on any length > 2G, and 2G allocations tend > to succeed on modern development machines). Perhaps it would be nice to > teach qemu-io to use blk_try_blockalign for more graceful handling even > on 32-bit platforms, but that's not the problem of your patch. > > Option ordering is minor enough that I'm fine giving: > > Reviewed-by: Eric Blake <eblake@redhat.com> > > Now, to figure out which maintainer should take it. Perhaps you want to > add a patch 2/1 that adds an iotest using this new mode, to a) ensure it > doesn't regress, and b) makes it reasonable to take in through the > iotest tree. > Yes, this is a good idea. I'm sure over time we'll pick up uses of pattern writing that will strengthen the the regression testing of the feature, but for now a simple test case will help ensure it. (It'll also help "document" how to use the feature for other test writers.) Thanks!
diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c index 09750a23ce..f411811d95 100644 --- a/qemu-io-cmds.c +++ b/qemu-io-cmds.c @@ -351,6 +351,79 @@ static void qemu_io_free(void *p) qemu_vfree(p); } +/* + * qemu_io_alloc_from_file() + * + * Allocates the buffer and populates it with the content of the given file + * up to @len bytes. If the file length is less than @len, then the buffer + * is populated with the file content cyclically. + * + * @blk - the block backend where the buffer content is going to be written to + * @len - the buffer length + * @file_name - the file to read the content from + * + * Returns: the buffer pointer on success + * NULL on error + */ +static void *qemu_io_alloc_from_file(BlockBackend *blk, size_t len, + const char *file_name) +{ + char *buf, *buf_origin; + FILE *f = fopen(file_name, "r"); + int pattern_len; + + if (!f) { + perror(file_name); + return NULL; + } + + if (qemuio_misalign) { + len += MISALIGN_OFFSET; + } + + buf_origin = buf = blk_blockalign(blk, len); + + if (qemuio_misalign) { + buf_origin += MISALIGN_OFFSET; + buf += MISALIGN_OFFSET; + len -= MISALIGN_OFFSET; + } + + pattern_len = fread(buf_origin, 1, len, f); + + if (ferror(f)) { + perror(file_name); + goto error; + } + + if (pattern_len == 0) { + fprintf(stderr, "%s: file is empty\n", file_name); + goto error; + } + + fclose(f); + + if (len > pattern_len) { + len -= pattern_len; + buf += pattern_len; + + while (len > 0) { + size_t len_to_copy = MIN(pattern_len, len); + + memcpy(buf, buf_origin, len_to_copy); + + len -= len_to_copy; + buf += len_to_copy; + } + } + + return buf_origin; + +error: + qemu_io_free(buf_origin); + return NULL; +} + static void dump_buffer(const void *buffer, int64_t offset, int64_t len) { uint64_t i; @@ -949,6 +1022,7 @@ static void write_help(void) " -n, -- with -z, don't allow slow fallback\n" " -p, -- ignored for backwards compatibility\n" " -P, -- use different pattern to fill file\n" +" -s, -- use a pattern file to fill the write buffer\n" " -C, -- report statistics in a machine parsable format\n" " -q, -- quiet mode, do not show I/O statistics\n" " -u, -- with -z, allow unmapping\n" @@ -965,7 +1039,7 @@ static const cmdinfo_t write_cmd = { .perm = BLK_PERM_WRITE, .argmin = 2, .argmax = -1, - .args = "[-bcCfnquz] [-P pattern] off len", + .args = "[-bcCfnquz] [-P pattern | -s source_file] off len", .oneline = "writes a number of bytes at a specified offset", .help = write_help, }; @@ -974,7 +1048,7 @@ static int write_f(BlockBackend *blk, int argc, char **argv) { struct timeval t1, t2; bool Cflag = false, qflag = false, bflag = false; - bool Pflag = false, zflag = false, cflag = false; + bool Pflag = false, zflag = false, cflag = false, sflag = false; int flags = 0; int c, cnt, ret; char *buf = NULL; @@ -983,8 +1057,9 @@ static int write_f(BlockBackend *blk, int argc, char **argv) /* Some compilers get confused and warn if this is not initialized. */ int64_t total = 0; int pattern = 0xcd; + const char *file_name = NULL; - while ((c = getopt(argc, argv, "bcCfnpP:quz")) != -1) { + while ((c = getopt(argc, argv, "bcCfnpP:quzs:")) != -1) { switch (c) { case 'b': bflag = true; @@ -1020,6 +1095,10 @@ static int write_f(BlockBackend *blk, int argc, char **argv) case 'z': zflag = true; break; + case 's': + sflag = true; + file_name = optarg; + break; default: qemuio_command_usage(&write_cmd); return -EINVAL; @@ -1051,8 +1130,9 @@ static int write_f(BlockBackend *blk, int argc, char **argv) return -EINVAL; } - if (zflag && Pflag) { - printf("-z and -P cannot be specified at the same time\n"); + if (zflag + Pflag + sflag > 1) { + printf("Only one of -z, -P, and -s " + "can be specified at the same time\n"); return -EINVAL; } @@ -1088,7 +1168,14 @@ static int write_f(BlockBackend *blk, int argc, char **argv) } if (!zflag) { - buf = qemu_io_alloc(blk, count, pattern); + if (sflag) { + buf = qemu_io_alloc_from_file(blk, count, file_name); + if (!buf) { + return -EINVAL; + } + } else { + buf = qemu_io_alloc(blk, count, pattern); + } } gettimeofday(&t1, NULL);
The patch allows to provide a pattern file for write command. There was no similar ability before. Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> --- v10: * fix overflow [Max] * remove casting to bool [Max, Eric] * fix wording [Max] v9: * replace flag cast to int with bool [Eric] * fix the error message [Eric] * use qemu_io_free instead of qemu_vfree [Eric] * add function description [Eric] v8: fix according to Max's comments * get rid of unnecessary buffer for the pattern * buffer allocation just in bytes * take into account the missalign offset * don't copy file name * changed char* to const char* in input params v7: * fix variable naming * make code more readable * extend help for write command v6: * the pattern file is read once to reduce io v5: * file name initiated with null to make compilers happy v4: * missing signed-off clause added v3: * missing file closing added * exclusive flags processing changed * buffer void* converted to char* to fix pointer arithmetics * file reading error processing added --- qemu-io-cmds.c | 99 +++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 93 insertions(+), 6 deletions(-)