| Message ID | 20200525100801.13859-4-vsementsov@virtuozzo.com |
|---|---|
| State | New |
| Headers | show |
| Series | coroutines: generate wrapper code | expand |
On 5/25/20 5:07 AM, Vladimir Sementsov-Ogievskiy wrote: > We have a very frequent pattern of creating coroutine from function > with several arguments: > > - create structure to pack parameters > - create _entry function to call original function taking parameters > from struct > - do different magic to handle completion: set ret to NOT_DONE or > EINPROGRESS, use separate bool for void functions > - fill the struct and create coroutine from _entry function and this > struct as a parameter > - do coroutine enter and BDRV_POLL_WHILE loop > > Let's reduce code duplication. Here: > > Functional part (BDRV_POLL_WHILE loop, aio_wait_kick()) moved to > (non-generated) block/block-gen.h > > Mechanical part (arguments packing, different kind of needed wrappers) > are generated from template by scripts/coroutine-wrapper.py to > resulting file block/block-gen.c > > Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> > --- > @@ -175,6 +177,10 @@ generated-files-y += $(TRACE_SOURCES) > generated-files-y += $(BUILD_DIR)/trace-events-all > generated-files-y += .git-submodule-status > > +COROUTINE_HEADERS = include/block/block.h block/coroutines.h > +block/block-gen.c: $(COROUTINE_HEADERS) $(SRC_PATH)/scripts/coroutine-wrapper.py > + $(call quiet-command, cat $(COROUTINE_HEADERS) | $(SRC_PATH)/scripts/coroutine-wrapper.py > $@,"GEN","$(TARGET_DIR)$@") > + Not VPATH-friendly; I posted a proposed fixup! separately. > trace-group-name = $(shell dirname $1 | sed -e 's/[^a-zA-Z0-9]/_/g') > > tracetool-y = $(SRC_PATH)/scripts/tracetool.py > diff --git a/Makefile.objs b/Makefile.objs > index 99774cfd25..8cb20f94c3 100644 > --- a/Makefile.objs > +++ b/Makefile.objs > @@ -14,7 +14,7 @@ chardev-obj-y = chardev/ > authz-obj-y = authz/ > > block-obj-y = block/ block/monitor/ nbd/ scsi/ > -block-obj-y += block.o blockjob.o job.o > +block-obj-y += block.o blockjob.o job.o block/block-gen.o It may be cleaner to add this in block/Makefile.objs rather than in top-level Makefile.objs. In fact, rearranging your mail a bit... > diff --git a/block/Makefile.objs b/block/Makefile.objs > index 3635b6b4c1..05e4d033c1 100644 > --- a/block/Makefile.objs > +++ b/block/Makefile.objs > @@ -45,6 +45,7 @@ block-obj-y += crypto.o > block-obj-y += aio_task.o > block-obj-y += backup-top.o > block-obj-y += filter-compress.o > +block-obj-y += block-gen.o > common-obj-y += monitor/ > > block-obj-y += stream.o ...you did just that. Dropping the change to top-level Makefile.objs seems to make no difference to a correct build. > +++ b/block/block-gen.h > @@ -0,0 +1,55 @@ > +/* > + * Block layer I/O functions Is this still the best one-line summary for this file? Especially since... > + > +/* This function is called at the end of generated coroutine entries. */ > +static inline void bdrv_poll_co__on_exit(void) > +{ > + aio_wait_kick(); > +} > + > +/* Base structure for argument packing structures */ > +typedef struct BdrvPollCo { > + BlockDriverState *bs; > + bool in_progress; > + int ret; > + Coroutine *co; /* Keep pointer here for debugging */ > +} BdrvPollCo; > + > +static inline int bdrv_poll_co(BdrvPollCo *s) > +{ > + assert(!qemu_in_coroutine()); > + > + bdrv_coroutine_enter(s->bs, s->co); > + BDRV_POLL_WHILE(s->bs, s->in_progress); > + > + return s->ret; > +} This part looks fine. > +++ b/include/block/generated-co-wrapper.h > @@ -0,0 +1,35 @@ > +/* > + * Block layer I/O functions ...you repeat it here? > +/* > + * generated_co_wrapper > + * Function specifier, which does nothing but marking functions to be > + * generated by scripts/coroutine-wrapper.py > + */ > +#define generated_co_wrapper > + > +#endif /* BLOCK_GENERATED_CO_WRAPPER_H */ Not sure if a separate header was needed for this, but I guess it doesn't hurt. I might have just used block_int.h. > diff --git a/block.c b/block.c > index 7f06e82880..c1132ab323 100644 > --- a/block.c > +++ b/block.c > @@ -4640,43 +4640,6 @@ int coroutine_fn bdrv_co_check(BlockDriverState *bs, > return bs->drv->bdrv_co_check(bs, res, fix); > } > > -typedef struct CheckCo { > - BlockDriverState *bs; > - BdrvCheckResult *res; > - BdrvCheckMode fix; > - int ret; > -} CheckCo; This patch is doing two things - introducing a new generator script that scans the code for generated_co_wrapper tags, _and_ adds the tags in as many places as possible. It makes for a big patch. Better might have been to introduce the script and the concept of a tag in one patch but not actually tag any new functions (so the generated file is basically empty, but you prove the build works and can audit the script without being bogged down by the mechanical changes), then do a separate patch with adding the tags and deleting the code now covered by the generator (which will be mostly mechanical). > +++ b/scripts/coroutine-wrapper.py > @@ -0,0 +1,168 @@ > +#!/usr/bin/env python3 My python review skills are weak, so you'll probably want another reviewer here (although I _can_ state that I checked the generated block/block-gen.c file and it makes sense). > +import re > +from typing import List, Iterator > + > +header = '''/* > + * File is generated by scripts/coroutine-wrapper.py > + */ It's worth also generating a short copyright blurb into the generated file. > + > +#include "qemu/osdep.h" > +#include "block/coroutines.h" > +#include "block/block-gen.h"''' > + > +template = """ Why ''' above and """ here? While python accepts both forms, consistency is desirable. I have no idea what pylint thinks of your code, though, so I may be completely overlooking idiomatic styles. > + > +/* > + * Wrappers for $name$ > + */ > + > +typedef struct $struct_name$ { > + BdrvPollCo poll_state; > + $fields$ > +} $struct_name$; > + > +static void coroutine_fn $name$_entry(void *opaque) > +{ > + $struct_name$ *s = opaque; > + > + $assign_ret$$name$($args_from_s$); Creates a long line. Doesn't matter since this is generated code, so I don't know if it's worth trying to coerce python into generating something that is wrapped nicely. > + > + s->poll_state.in_progress = false; > + > + bdrv_poll_co__on_exit(); That function only calls aio_wait_kick(). Why did you not want to call that directly here, instead of going through such a one-line wrapper function? > +} > + > +$ret_type$ $wrapper_name$($args_def$) > +{ > + if (qemu_in_coroutine()) { > + $do_return$$name$($arg_names$); Would it be any simpler to teach bdrv_co_invalidate_cache() to return a value, so that you can eliminate the need for $do_return$ and just hard-code 'return ' here? The .bdrv_co_invalidate_cache driver callback can still return void, and just have the wrapper in block.c return 0. But if you want to do that, it would be a separate prerequisite patch. > + } else { > + $struct_name$ s = { > + .poll_state.bs = $bs$, > + .poll_state.in_progress = true, > + > + $initializers$ > + }; > + > + s.poll_state.co = qemu_coroutine_create($name$_entry, &s); > + > + $do_return$bdrv_poll_co(&s.poll_state); Another spot impacted by that one outlier. > + } > +}""" > + > +# We want to use python string.format() formatter, which uses curly brackets > +# as separators. But it's not comfortable with C. So, we used dollars instead, > +# and now is the time to escape curly brackets and convert dollars. > +template = template.replace('{', '{{').replace('}', '}}') > +template = re.sub(r'\$(\w+)\$', r'{\1}', template) I'll leave it to others to decide if your approach has a more idiomatic python solution. I'm used to the qapi generator using code like this in script/qapi/visit.py: def gen_visit_members_decl(name): return mcgen(''' void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp); ''', c_name=c_name(name)) which has the same end goal of inserting named tags into a format string, but which uses %(tag) for tags rather than your novel approach of $tag$ with post-processing. > + > + > +class ParamDecl: > + param_re = re.compile(r'(?P<decl>' > + r'(?P<type>.*[ *])' > + r'(?P<name>[a-z][a-z0-9_]*)' > + r')') > + > + def __init__(self, param_decl: str) -> None: > + m = self.param_re.match(param_decl.strip()) > + self.decl = m.group('decl') > + self.type = m.group('type') > + self.name = m.group('name') > + > + > +class FuncDecl: > + def __init__(self, return_type: str, name: str, args: str) -> None: > + self.return_type = return_type.strip() > + self.name = name.strip() > + self.args: List[ParamDecl] = [] > + self.args = [ParamDecl(arg) for arg in args.split(',')] > + > + def get_args_decl(self) -> str: > + return ', '.join(arg.decl for arg in self.args) > + > + def get_arg_names(self) -> str: > + return ', '.join(arg.name for arg in self.args) > + > + def gen_struct_fields(self) -> str: > + return '\n '.join(f'{arg.decl};' for arg in self.args) > + > + def gen_struct_initializers(self, indent: int) -> str: > + sep = '\n' + ' ' * indent > + return sep.join(f'.{a.name} = {a.name},' for a in self.args) > + > + > +# Match wrappers declaration, with generated_co_wrapper mark s/declaration, with/declared with a/ > +func_decl_re = re.compile(r'^(?P<return_type>(int|void))' > + r'\s*generated_co_wrapper\s*' > + r'(?P<wrapper_name>[a-z][a-z0-9_]*)' > + r'\((?P<args>[^)]*)\);$', re.MULTILINE) > + > + > +def func_decl_iter(text: str) -> Iterator: > + for m in func_decl_re.finditer(text): > + yield FuncDecl(return_type=m.group('return_type'), > + name=m.group('wrapper_name'), > + args=m.group('args')) > + > + > +def struct_name(func_name: str) -> str: > + """some_function_name -> SomeFunctionName""" > + words = func_name.split('_') > + words = [w[0].upper() + w[1:] for w in words] > + return ''.join(words) > + > + > +def make_wrapper(func: FuncDecl) -> str: > + assert func.name.startswith('bdrv_') > + co_name = 'bdrv_co_' + func.name[5:] > + > + has_ret = func.return_type != 'void' > + > + params = { > + 'name': co_name, > + 'do_return': 'return ' if has_ret else '', > + 'assign_ret': 's->poll_state.ret = ' if has_ret else '', See my comments above about possibly normalizing the one outlier of bdrv_co_invalidate_cache first. > + 'struct_name': struct_name(co_name), > + 'wrapper_name': func.name, > + 'ret_type': func.return_type, > + 'args_def': func.get_args_decl(), > + 'arg_names': func.get_arg_names(), > + 'fields': func.gen_struct_fields(), > + 'initializers': func.gen_struct_initializers(12), 12 looks like a magic number. > + 'args_from_s': ', '.join(f's->{a.name}' for a in func.args), > + } > + > + if func.args[0].type == 'BlockDriverState *': > + params['bs'] = 'bs' > + else: > + assert func.args[0].type == 'BdrvChild *' > + params['bs'] = 'child->bs' > + > + return template.format(**params) > + > + > +if __name__ == '__main__': > + import sys > + > + print(header) > + for func in func_decl_iter(sys.stdin.read()): > + print(make_wrapper(func)) >
27.05.2020 00:30, Eric Blake wrote: > On 5/25/20 5:07 AM, Vladimir Sementsov-Ogievskiy wrote: >> We have a very frequent pattern of creating coroutine from function >> with several arguments: >> >> - create structure to pack parameters >> - create _entry function to call original function taking parameters >> from struct >> - do different magic to handle completion: set ret to NOT_DONE or >> EINPROGRESS, use separate bool for void functions >> - fill the struct and create coroutine from _entry function and this >> struct as a parameter >> - do coroutine enter and BDRV_POLL_WHILE loop >> >> Let's reduce code duplication. Here: >> >> Functional part (BDRV_POLL_WHILE loop, aio_wait_kick()) moved to >> (non-generated) block/block-gen.h >> >> Mechanical part (arguments packing, different kind of needed wrappers) >> are generated from template by scripts/coroutine-wrapper.py to >> resulting file block/block-gen.c >> >> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> >> --- > >> @@ -175,6 +177,10 @@ generated-files-y += $(TRACE_SOURCES) >> generated-files-y += $(BUILD_DIR)/trace-events-all >> generated-files-y += .git-submodule-status >> +COROUTINE_HEADERS = include/block/block.h block/coroutines.h >> +block/block-gen.c: $(COROUTINE_HEADERS) $(SRC_PATH)/scripts/coroutine-wrapper.py >> + $(call quiet-command, cat $(COROUTINE_HEADERS) | $(SRC_PATH)/scripts/coroutine-wrapper.py > $@,"GEN","$(TARGET_DIR)$@") >> + > > Not VPATH-friendly; I posted a proposed fixup! separately. Thanks! > >> trace-group-name = $(shell dirname $1 | sed -e 's/[^a-zA-Z0-9]/_/g') >> tracetool-y = $(SRC_PATH)/scripts/tracetool.py >> diff --git a/Makefile.objs b/Makefile.objs >> index 99774cfd25..8cb20f94c3 100644 >> --- a/Makefile.objs >> +++ b/Makefile.objs >> @@ -14,7 +14,7 @@ chardev-obj-y = chardev/ >> authz-obj-y = authz/ >> block-obj-y = block/ block/monitor/ nbd/ scsi/ >> -block-obj-y += block.o blockjob.o job.o >> +block-obj-y += block.o blockjob.o job.o block/block-gen.o > > It may be cleaner to add this in block/Makefile.objs rather than in top-level Makefile.objs. In fact, rearranging your mail a bit... > > > diff --git a/block/Makefile.objs b/block/Makefile.objs > > index 3635b6b4c1..05e4d033c1 100644 > > --- a/block/Makefile.objs > > +++ b/block/Makefile.objs > > @@ -45,6 +45,7 @@ block-obj-y += crypto.o > > block-obj-y += aio_task.o > > block-obj-y += backup-top.o > > block-obj-y += filter-compress.o > > +block-obj-y += block-gen.o > > common-obj-y += monitor/ > > > > block-obj-y += stream.o > > ...you did just that. Dropping the change to top-level Makefile.objs seems to make no difference to a correct build. Oops, yes. > >> +++ b/block/block-gen.h >> @@ -0,0 +1,55 @@ >> +/* >> + * Block layer I/O functions > > Is this still the best one-line summary for this file? Especially since... Yes, it's outdated. Maybe Block coroutine wrapping core, used by auto-generated block/block-gen.c > >> + >> +/* This function is called at the end of generated coroutine entries. */ >> +static inline void bdrv_poll_co__on_exit(void) >> +{ >> + aio_wait_kick(); >> +} >> + >> +/* Base structure for argument packing structures */ >> +typedef struct BdrvPollCo { >> + BlockDriverState *bs; >> + bool in_progress; >> + int ret; >> + Coroutine *co; /* Keep pointer here for debugging */ >> +} BdrvPollCo; >> + >> +static inline int bdrv_poll_co(BdrvPollCo *s) >> +{ >> + assert(!qemu_in_coroutine()); >> + >> + bdrv_coroutine_enter(s->bs, s->co); >> + BDRV_POLL_WHILE(s->bs, s->in_progress); >> + >> + return s->ret; >> +} > > This part looks fine. > > >> +++ b/include/block/generated-co-wrapper.h >> @@ -0,0 +1,35 @@ >> +/* >> + * Block layer I/O functions > > ...you repeat it here? > >> +/* >> + * generated_co_wrapper >> + * Function specifier, which does nothing but marking functions to be >> + * generated by scripts/coroutine-wrapper.py >> + */ >> +#define generated_co_wrapper >> + >> +#endif /* BLOCK_GENERATED_CO_WRAPPER_H */ > > Not sure if a separate header was needed for this, but I guess it doesn't hurt. I might have just used block_int.h. Than we'll have to include block_int.h in block.h.. Is it OK? This is the reason why I decided to keep it separate. > >> diff --git a/block.c b/block.c >> index 7f06e82880..c1132ab323 100644 >> --- a/block.c >> +++ b/block.c >> @@ -4640,43 +4640,6 @@ int coroutine_fn bdrv_co_check(BlockDriverState *bs, >> return bs->drv->bdrv_co_check(bs, res, fix); >> } >> -typedef struct CheckCo { >> - BlockDriverState *bs; >> - BdrvCheckResult *res; >> - BdrvCheckMode fix; >> - int ret; >> -} CheckCo; > > This patch is doing two things - introducing a new generator script that scans the code for generated_co_wrapper tags, _and_ adds the tags in as many places as possible. It makes for a big patch. Better might have been to introduce the script and the concept of a tag in one patch but not actually tag any new functions (so the generated file is basically empty, but you prove the build works and can audit the script without being bogged down by the mechanical changes), then do a separate patch with adding the tags and deleting the code now covered by the generator (which will be mostly mechanical). No problem, will do. > >> +++ b/scripts/coroutine-wrapper.py >> @@ -0,0 +1,168 @@ >> +#!/usr/bin/env python3 > > My python review skills are weak, so you'll probably want another reviewer here (although I _can_ state that I checked the generated block/block-gen.c file and it makes sense). > > >> +import re >> +from typing import List, Iterator >> + >> +header = '''/* >> + * File is generated by scripts/coroutine-wrapper.py >> + */ > > It's worth also generating a short copyright blurb into the generated file. > >> + >> +#include "qemu/osdep.h" >> +#include "block/coroutines.h" >> +#include "block/block-gen.h"''' >> + >> +template = """ > > Why ''' above and """ here? While python accepts both forms, consistency is desirable. I have no idea what pylint thinks of your code, though, so I may be completely overlooking idiomatic styles. ''' is used just to not interfer with " around include files.. So I tend to generally use """ for multiline strings, but the above is and exception just for internal " characters. > >> + >> +/* >> + * Wrappers for $name$ >> + */ >> + >> +typedef struct $struct_name$ { >> + BdrvPollCo poll_state; >> + $fields$ >> +} $struct_name$; >> + >> +static void coroutine_fn $name$_entry(void *opaque) >> +{ >> + $struct_name$ *s = opaque; >> + >> + $assign_ret$$name$($args_from_s$); > > Creates a long line. Doesn't matter since this is generated code, so I don't know if it's worth trying to coerce python into generating something that is wrapped nicely. > >> + >> + s->poll_state.in_progress = false; >> + >> + bdrv_poll_co__on_exit(); > > That function only calls aio_wait_kick(). Why did you not want to call that directly here, instead of going through such a one-line wrapper function? To keep polling logic out of generated code, in block/block-gen.h, like bdrv_poll_co. Does it worth doing - I'm not sure. Up to maintainers. > >> +} >> + >> +$ret_type$ $wrapper_name$($args_def$) >> +{ >> + if (qemu_in_coroutine()) { >> + $do_return$$name$($arg_names$); > > Would it be any simpler to teach bdrv_co_invalidate_cache() to return a value, so that you can eliminate the need for $do_return$ and just hard-code 'return ' here? The .bdrv_co_invalidate_cache driver callback can still return void, and just have the wrapper in block.c return 0. But if you want to do that, it would be a separate prerequisite patch. Hmm. I thought, that it's not a good reason to add a always-success return value.. But stop! bdrv_invalidate_cache is a void functions with errp argument, so it makes sense to add a return value anyway. Will do. > >> + } else { >> + $struct_name$ s = { >> + .poll_state.bs = $bs$, >> + .poll_state.in_progress = true, >> + >> + $initializers$ >> + }; >> + >> + s.poll_state.co = qemu_coroutine_create($name$_entry, &s); >> + >> + $do_return$bdrv_poll_co(&s.poll_state); > > Another spot impacted by that one outlier. > >> + } >> +}""" >> + >> +# We want to use python string.format() formatter, which uses curly brackets >> +# as separators. But it's not comfortable with C. So, we used dollars instead, >> +# and now is the time to escape curly brackets and convert dollars. >> +template = template.replace('{', '{{').replace('}', '}}') >> +template = re.sub(r'\$(\w+)\$', r'{\1}', template) > > I'll leave it to others to decide if your approach has a more idiomatic python solution. I'm used to the qapi generator using code like this in script/qapi/visit.py: > > def gen_visit_members_decl(name): > return mcgen(''' > > void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp); > ''', > c_name=c_name(name)) > > which has the same end goal of inserting named tags into a format string, but which uses %(tag) for tags rather than your novel approach of $tag$ with post-processing. Note, that actually not %(tag) but %(tag)s %(name)s notation comes from % interpolation operator.. As I understand, in modern python using .format(), f-strings and templates is preferable. But I agree, that introducing new hand-made notation is not a great idea. I think, I'll move to string.Template or f-strings. > >> + >> + >> +class ParamDecl: >> + param_re = re.compile(r'(?P<decl>' >> + r'(?P<type>.*[ *])' >> + r'(?P<name>[a-z][a-z0-9_]*)' >> + r')') >> + >> + def __init__(self, param_decl: str) -> None: >> + m = self.param_re.match(param_decl.strip()) >> + self.decl = m.group('decl') >> + self.type = m.group('type') >> + self.name = m.group('name') >> + >> + >> +class FuncDecl: >> + def __init__(self, return_type: str, name: str, args: str) -> None: >> + self.return_type = return_type.strip() >> + self.name = name.strip() >> + self.args: List[ParamDecl] = [] >> + self.args = [ParamDecl(arg) for arg in args.split(',')] >> + >> + def get_args_decl(self) -> str: >> + return ', '.join(arg.decl for arg in self.args) >> + >> + def get_arg_names(self) -> str: >> + return ', '.join(arg.name for arg in self.args) >> + >> + def gen_struct_fields(self) -> str: >> + return '\n '.join(f'{arg.decl};' for arg in self.args) >> + >> + def gen_struct_initializers(self, indent: int) -> str: >> + sep = '\n' + ' ' * indent >> + return sep.join(f'.{a.name} = {a.name},' for a in self.args) >> + >> + >> +# Match wrappers declaration, with generated_co_wrapper mark > > s/declaration, with/declared with a/ > >> +func_decl_re = re.compile(r'^(?P<return_type>(int|void))' >> + r'\s*generated_co_wrapper\s*' >> + r'(?P<wrapper_name>[a-z][a-z0-9_]*)' >> + r'\((?P<args>[^)]*)\);$', re.MULTILINE) >> + >> + >> +def func_decl_iter(text: str) -> Iterator: >> + for m in func_decl_re.finditer(text): >> + yield FuncDecl(return_type=m.group('return_type'), >> + name=m.group('wrapper_name'), >> + args=m.group('args')) >> + >> + >> +def struct_name(func_name: str) -> str: >> + """some_function_name -> SomeFunctionName""" >> + words = func_name.split('_') >> + words = [w[0].upper() + w[1:] for w in words] >> + return ''.join(words) >> + >> + >> +def make_wrapper(func: FuncDecl) -> str: >> + assert func.name.startswith('bdrv_') >> + co_name = 'bdrv_co_' + func.name[5:] >> + >> + has_ret = func.return_type != 'void' >> + >> + params = { >> + 'name': co_name, >> + 'do_return': 'return ' if has_ret else '', >> + 'assign_ret': 's->poll_state.ret = ' if has_ret else '', > > See my comments above about possibly normalizing the one outlier of bdrv_co_invalidate_cache first. > >> + 'struct_name': struct_name(co_name), >> + 'wrapper_name': func.name, >> + 'ret_type': func.return_type, >> + 'args_def': func.get_args_decl(), >> + 'arg_names': func.get_arg_names(), >> + 'fields': func.gen_struct_fields(), >> + 'initializers': func.gen_struct_initializers(12), > > 12 looks like a magic number. > >> + 'args_from_s': ', '.join(f's->{a.name}' for a in func.args), >> + } >> + >> + if func.args[0].type == 'BlockDriverState *': >> + params['bs'] = 'bs' >> + else: >> + assert func.args[0].type == 'BdrvChild *' >> + params['bs'] = 'child->bs' >> + >> + return template.format(**params) >> + >> + >> +if __name__ == '__main__': >> + import sys >> + >> + print(header) >> + for func in func_decl_iter(sys.stdin.read()): >> + print(make_wrapper(func)) >> >
diff --git a/Makefile b/Makefile index 40e4f7677b..ec15b8ea89 100644 --- a/Makefile +++ b/Makefile @@ -159,6 +159,8 @@ generated-files-$(CONFIG_TRACE_UST) += trace-ust-all.c generated-files-y += module_block.h +generated-files-y += block/block-gen.c + TRACE_HEADERS = trace-root.h $(trace-events-subdirs:%=%/trace.h) TRACE_SOURCES = trace-root.c $(trace-events-subdirs:%=%/trace.c) TRACE_DTRACE = @@ -175,6 +177,10 @@ generated-files-y += $(TRACE_SOURCES) generated-files-y += $(BUILD_DIR)/trace-events-all generated-files-y += .git-submodule-status +COROUTINE_HEADERS = include/block/block.h block/coroutines.h +block/block-gen.c: $(COROUTINE_HEADERS) $(SRC_PATH)/scripts/coroutine-wrapper.py + $(call quiet-command, cat $(COROUTINE_HEADERS) | $(SRC_PATH)/scripts/coroutine-wrapper.py > $@,"GEN","$(TARGET_DIR)$@") + trace-group-name = $(shell dirname $1 | sed -e 's/[^a-zA-Z0-9]/_/g') tracetool-y = $(SRC_PATH)/scripts/tracetool.py diff --git a/Makefile.objs b/Makefile.objs index 99774cfd25..8cb20f94c3 100644 --- a/Makefile.objs +++ b/Makefile.objs @@ -14,7 +14,7 @@ chardev-obj-y = chardev/ authz-obj-y = authz/ block-obj-y = block/ block/monitor/ nbd/ scsi/ -block-obj-y += block.o blockjob.o job.o +block-obj-y += block.o blockjob.o job.o block/block-gen.o block-obj-y += qemu-io-cmds.o block-obj-$(CONFIG_REPLICATION) += replication.o diff --git a/block/block-gen.h b/block/block-gen.h new file mode 100644 index 0000000000..11a44247b7 --- /dev/null +++ b/block/block-gen.h @@ -0,0 +1,55 @@ +/* + * Block layer I/O functions + * + * Copyright (c) 2003 Fabrice Bellard + * Copyright (c) 2020 Virtuozzo International GmbH + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#ifndef BLOCK_BLOCK_GEN_H +#define BLOCK_BLOCK_GEN_H + +#include "block/block_int.h" + +/* This function is called at the end of generated coroutine entries. */ +static inline void bdrv_poll_co__on_exit(void) +{ + aio_wait_kick(); +} + +/* Base structure for argument packing structures */ +typedef struct BdrvPollCo { + BlockDriverState *bs; + bool in_progress; + int ret; + Coroutine *co; /* Keep pointer here for debugging */ +} BdrvPollCo; + +static inline int bdrv_poll_co(BdrvPollCo *s) +{ + assert(!qemu_in_coroutine()); + + bdrv_coroutine_enter(s->bs, s->co); + BDRV_POLL_WHILE(s->bs, s->in_progress); + + return s->ret; +} + +#endif /* BLOCK_BLOCK_GEN_H */ diff --git a/block/coroutines.h b/block/coroutines.h index 74278cfef2..e2047697d6 100644 --- a/block/coroutines.h +++ b/block/coroutines.h @@ -26,6 +26,7 @@ #define BLOCK_COROUTINES_INT_H #include "block/block_int.h" +#include "block/generated-co-wrapper.h" int coroutine_fn bdrv_co_check(BlockDriverState *bs, BdrvCheckResult *res, BdrvCheckMode fix); @@ -34,7 +35,7 @@ void coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp); int coroutine_fn bdrv_co_prwv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov, bool is_write, BdrvRequestFlags flags); -int +int generated_co_wrapper bdrv_prwv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov, bool is_write, BdrvRequestFlags flags); @@ -47,7 +48,7 @@ bdrv_co_common_block_status_above(BlockDriverState *bs, int64_t *pnum, int64_t *map, BlockDriverState **file); -int +int generated_co_wrapper bdrv_common_block_status_above(BlockDriverState *bs, BlockDriverState *base, bool want_zero, @@ -60,7 +61,7 @@ bdrv_common_block_status_above(BlockDriverState *bs, int coroutine_fn bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos, bool is_read); -int +int generated_co_wrapper bdrv_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos, bool is_read); diff --git a/include/block/block.h b/include/block/block.h index 25e299605e..aed6ffcc4f 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -9,6 +9,7 @@ #include "block/dirty-bitmap.h" #include "block/blockjob.h" #include "qemu/hbitmap.h" +#include "block/generated-co-wrapper.h" /* block.c */ typedef struct BlockDriver BlockDriver; @@ -398,8 +399,9 @@ void bdrv_refresh_filename(BlockDriverState *bs); int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, bool exact, PreallocMode prealloc, BdrvRequestFlags flags, Error **errp); -int bdrv_truncate(BdrvChild *child, int64_t offset, bool exact, - PreallocMode prealloc, BdrvRequestFlags flags, Error **errp); +int generated_co_wrapper +bdrv_truncate(BdrvChild *child, int64_t offset, bool exact, + PreallocMode prealloc, BdrvRequestFlags flags, Error **errp); int64_t bdrv_nb_sectors(BlockDriverState *bs); int64_t bdrv_getlength(BlockDriverState *bs); @@ -441,7 +443,8 @@ typedef enum { BDRV_FIX_ERRORS = 2, } BdrvCheckMode; -int bdrv_check(BlockDriverState *bs, BdrvCheckResult *res, BdrvCheckMode fix); +int generated_co_wrapper bdrv_check(BlockDriverState *bs, BdrvCheckResult *res, + BdrvCheckMode fix); /* The units of offset and total_work_size may be chosen arbitrarily by the * block driver; total_work_size may change during the course of the amendment @@ -464,12 +467,13 @@ void bdrv_aio_cancel_async(BlockAIOCB *acb); int bdrv_co_ioctl(BlockDriverState *bs, int req, void *buf); /* Invalidate any cached metadata used by image formats */ -void bdrv_invalidate_cache(BlockDriverState *bs, Error **errp); +void generated_co_wrapper bdrv_invalidate_cache(BlockDriverState *bs, + Error **errp); void bdrv_invalidate_cache_all(Error **errp); int bdrv_inactivate_all(void); /* Ensure contents are flushed to disk. */ -int bdrv_flush(BlockDriverState *bs); +int generated_co_wrapper bdrv_flush(BlockDriverState *bs); int coroutine_fn bdrv_co_flush(BlockDriverState *bs); int bdrv_flush_all(void); void bdrv_close_all(void); @@ -484,7 +488,8 @@ void bdrv_drain_all(void); AIO_WAIT_WHILE(bdrv_get_aio_context(bs_), \ cond); }) -int bdrv_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes); +int generated_co_wrapper bdrv_pdiscard(BdrvChild *child, int64_t offset, + int64_t bytes); int bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes); int bdrv_has_zero_init_1(BlockDriverState *bs); int bdrv_has_zero_init(BlockDriverState *bs); diff --git a/include/block/generated-co-wrapper.h b/include/block/generated-co-wrapper.h new file mode 100644 index 0000000000..62c6e053ba --- /dev/null +++ b/include/block/generated-co-wrapper.h @@ -0,0 +1,35 @@ +/* + * Block layer I/O functions + * + * Copyright (c) 2020 Virtuozzo International GmbH + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#ifndef BLOCK_GENERATED_CO_WRAPPER_H +#define BLOCK_GENERATED_CO_WRAPPER_H + +/* + * generated_co_wrapper + * Function specifier, which does nothing but marking functions to be + * generated by scripts/coroutine-wrapper.py + */ +#define generated_co_wrapper + +#endif /* BLOCK_GENERATED_CO_WRAPPER_H */ diff --git a/block.c b/block.c index 7f06e82880..c1132ab323 100644 --- a/block.c +++ b/block.c @@ -4640,43 +4640,6 @@ int coroutine_fn bdrv_co_check(BlockDriverState *bs, return bs->drv->bdrv_co_check(bs, res, fix); } -typedef struct CheckCo { - BlockDriverState *bs; - BdrvCheckResult *res; - BdrvCheckMode fix; - int ret; -} CheckCo; - -static void coroutine_fn bdrv_check_co_entry(void *opaque) -{ - CheckCo *cco = opaque; - cco->ret = bdrv_co_check(cco->bs, cco->res, cco->fix); - aio_wait_kick(); -} - -int bdrv_check(BlockDriverState *bs, - BdrvCheckResult *res, BdrvCheckMode fix) -{ - Coroutine *co; - CheckCo cco = { - .bs = bs, - .res = res, - .ret = -EINPROGRESS, - .fix = fix, - }; - - if (qemu_in_coroutine()) { - /* Fast-path if already in coroutine context */ - bdrv_check_co_entry(&cco); - } else { - co = qemu_coroutine_create(bdrv_check_co_entry, &cco); - bdrv_coroutine_enter(bs, co); - BDRV_POLL_WHILE(bs, cco.ret == -EINPROGRESS); - } - - return cco.ret; -} - /* * Return values: * 0 - success @@ -5721,39 +5684,6 @@ void coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp) } } -typedef struct InvalidateCacheCo { - BlockDriverState *bs; - Error **errp; - bool done; -} InvalidateCacheCo; - -static void coroutine_fn bdrv_invalidate_cache_co_entry(void *opaque) -{ - InvalidateCacheCo *ico = opaque; - bdrv_co_invalidate_cache(ico->bs, ico->errp); - ico->done = true; - aio_wait_kick(); -} - -void bdrv_invalidate_cache(BlockDriverState *bs, Error **errp) -{ - Coroutine *co; - InvalidateCacheCo ico = { - .bs = bs, - .done = false, - .errp = errp - }; - - if (qemu_in_coroutine()) { - /* Fast-path if already in coroutine context */ - bdrv_invalidate_cache_co_entry(&ico); - } else { - co = qemu_coroutine_create(bdrv_invalidate_cache_co_entry, &ico); - bdrv_coroutine_enter(bs, co); - BDRV_POLL_WHILE(bs, !ico.done); - } -} - void bdrv_invalidate_cache_all(Error **errp) { BlockDriverState *bs; diff --git a/block/io.c b/block/io.c index f5b6ce3bf6..f9700cc897 100644 --- a/block/io.c +++ b/block/io.c @@ -892,15 +892,6 @@ static int bdrv_check_byte_request(BlockDriverState *bs, int64_t offset, return 0; } -typedef struct RwCo { - BdrvChild *child; - int64_t offset; - QEMUIOVector *qiov; - bool is_write; - int ret; - BdrvRequestFlags flags; -} RwCo; - int coroutine_fn bdrv_co_prwv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov, bool is_write, BdrvRequestFlags flags) @@ -912,43 +903,6 @@ int coroutine_fn bdrv_co_prwv(BdrvChild *child, int64_t offset, } } -static void coroutine_fn bdrv_rw_co_entry(void *opaque) -{ - RwCo *rwco = opaque; - - rwco->ret = bdrv_co_prwv(rwco->child, rwco->offset, rwco->qiov, - rwco->is_write, rwco->flags); - aio_wait_kick(); -} - -/* - * Process a vectored synchronous request using coroutines - */ -int bdrv_prwv(BdrvChild *child, int64_t offset, - QEMUIOVector *qiov, bool is_write, - BdrvRequestFlags flags) -{ - Coroutine *co; - RwCo rwco = { - .child = child, - .offset = offset, - .qiov = qiov, - .is_write = is_write, - .ret = NOT_DONE, - .flags = flags, - }; - - if (qemu_in_coroutine()) { - /* Fast-path if already in coroutine context */ - bdrv_rw_co_entry(&rwco); - } else { - co = qemu_coroutine_create(bdrv_rw_co_entry, &rwco); - bdrv_coroutine_enter(child->bs, co); - BDRV_POLL_WHILE(child->bs, rwco.ret == NOT_DONE); - } - return rwco.ret; -} - int bdrv_pwrite_zeroes(BdrvChild *child, int64_t offset, int bytes, BdrvRequestFlags flags) { @@ -2223,20 +2177,6 @@ int bdrv_flush_all(void) return result; } - -typedef struct BdrvCoBlockStatusData { - BlockDriverState *bs; - BlockDriverState *base; - bool want_zero; - int64_t offset; - int64_t bytes; - int64_t *pnum; - int64_t *map; - BlockDriverState **file; - int ret; - bool done; -} BdrvCoBlockStatusData; - int coroutine_fn bdrv_co_block_status_from_file(BlockDriverState *bs, bool want_zero, int64_t offset, @@ -2488,56 +2428,6 @@ bdrv_co_common_block_status_above(BlockDriverState *bs, return ret; } -/* Coroutine wrapper for bdrv_block_status_above() */ -static void coroutine_fn bdrv_block_status_above_co_entry(void *opaque) -{ - BdrvCoBlockStatusData *data = opaque; - - data->ret = bdrv_co_common_block_status_above(data->bs, data->base, - data->want_zero, - data->offset, data->bytes, - data->pnum, data->map, - data->file); - data->done = true; - aio_wait_kick(); -} - -/* - * Synchronous wrapper around bdrv_co_block_status_above(). - * - * See bdrv_co_block_status_above() for details. - */ -int bdrv_common_block_status_above(BlockDriverState *bs, - BlockDriverState *base, - bool want_zero, int64_t offset, - int64_t bytes, int64_t *pnum, - int64_t *map, - BlockDriverState **file) -{ - Coroutine *co; - BdrvCoBlockStatusData data = { - .bs = bs, - .base = base, - .want_zero = want_zero, - .offset = offset, - .bytes = bytes, - .pnum = pnum, - .map = map, - .file = file, - .done = false, - }; - - if (qemu_in_coroutine()) { - /* Fast-path if already in coroutine context */ - bdrv_block_status_above_co_entry(&data); - } else { - co = qemu_coroutine_create(bdrv_block_status_above_co_entry, &data); - bdrv_coroutine_enter(bs, co); - BDRV_POLL_WHILE(bs, !data.done); - } - return data.ret; -} - int bdrv_block_status_above(BlockDriverState *bs, BlockDriverState *base, int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file) @@ -2631,14 +2521,6 @@ int bdrv_is_allocated_above(BlockDriverState *top, return 0; } -typedef struct BdrvVmstateCo { - BlockDriverState *bs; - QEMUIOVector *qiov; - int64_t pos; - bool is_read; - int ret; -} BdrvVmstateCo; - int coroutine_fn bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos, bool is_read) @@ -2664,34 +2546,6 @@ bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos, return ret; } -static void coroutine_fn bdrv_co_rw_vmstate_entry(void *opaque) -{ - BdrvVmstateCo *co = opaque; - co->ret = bdrv_co_rw_vmstate(co->bs, co->qiov, co->pos, co->is_read); - aio_wait_kick(); -} - -int bdrv_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos, - bool is_read) -{ - if (qemu_in_coroutine()) { - return bdrv_co_rw_vmstate(bs, qiov, pos, is_read); - } else { - BdrvVmstateCo data = { - .bs = bs, - .qiov = qiov, - .pos = pos, - .is_read = is_read, - .ret = -EINPROGRESS, - }; - Coroutine *co = qemu_coroutine_create(bdrv_co_rw_vmstate_entry, &data); - - bdrv_coroutine_enter(bs, co); - BDRV_POLL_WHILE(bs, data.ret == -EINPROGRESS); - return data.ret; - } -} - int bdrv_save_vmstate(BlockDriverState *bs, const uint8_t *buf, int64_t pos, int size) { @@ -2767,20 +2621,6 @@ void bdrv_aio_cancel_async(BlockAIOCB *acb) /**************************************************************/ /* Coroutine block device emulation */ -typedef struct FlushCo { - BlockDriverState *bs; - int ret; -} FlushCo; - - -static void coroutine_fn bdrv_flush_co_entry(void *opaque) -{ - FlushCo *rwco = opaque; - - rwco->ret = bdrv_co_flush(rwco->bs); - aio_wait_kick(); -} - int coroutine_fn bdrv_co_flush(BlockDriverState *bs) { int current_gen; @@ -2893,40 +2733,6 @@ early_exit: return ret; } -int bdrv_flush(BlockDriverState *bs) -{ - Coroutine *co; - FlushCo flush_co = { - .bs = bs, - .ret = NOT_DONE, - }; - - if (qemu_in_coroutine()) { - /* Fast-path if already in coroutine context */ - bdrv_flush_co_entry(&flush_co); - } else { - co = qemu_coroutine_create(bdrv_flush_co_entry, &flush_co); - bdrv_coroutine_enter(bs, co); - BDRV_POLL_WHILE(bs, flush_co.ret == NOT_DONE); - } - - return flush_co.ret; -} - -typedef struct DiscardCo { - BdrvChild *child; - int64_t offset; - int64_t bytes; - int ret; -} DiscardCo; -static void coroutine_fn bdrv_pdiscard_co_entry(void *opaque) -{ - DiscardCo *rwco = opaque; - - rwco->ret = bdrv_co_pdiscard(rwco->child, rwco->offset, rwco->bytes); - aio_wait_kick(); -} - int coroutine_fn bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes) { @@ -3041,28 +2847,6 @@ out: return ret; } -int bdrv_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes) -{ - Coroutine *co; - DiscardCo rwco = { - .child = child, - .offset = offset, - .bytes = bytes, - .ret = NOT_DONE, - }; - - if (qemu_in_coroutine()) { - /* Fast-path if already in coroutine context */ - bdrv_pdiscard_co_entry(&rwco); - } else { - co = qemu_coroutine_create(bdrv_pdiscard_co_entry, &rwco); - bdrv_coroutine_enter(child->bs, co); - BDRV_POLL_WHILE(child->bs, rwco.ret == NOT_DONE); - } - - return rwco.ret; -} - int bdrv_co_ioctl(BlockDriverState *bs, int req, void *buf) { BlockDriver *drv = bs->drv; @@ -3460,47 +3244,3 @@ out: return ret; } - -typedef struct TruncateCo { - BdrvChild *child; - int64_t offset; - bool exact; - PreallocMode prealloc; - BdrvRequestFlags flags; - Error **errp; - int ret; -} TruncateCo; - -static void coroutine_fn bdrv_truncate_co_entry(void *opaque) -{ - TruncateCo *tco = opaque; - tco->ret = bdrv_co_truncate(tco->child, tco->offset, tco->exact, - tco->prealloc, tco->flags, tco->errp); - aio_wait_kick(); -} - -int bdrv_truncate(BdrvChild *child, int64_t offset, bool exact, - PreallocMode prealloc, BdrvRequestFlags flags, Error **errp) -{ - Coroutine *co; - TruncateCo tco = { - .child = child, - .offset = offset, - .exact = exact, - .prealloc = prealloc, - .flags = flags, - .errp = errp, - .ret = NOT_DONE, - }; - - if (qemu_in_coroutine()) { - /* Fast-path if already in coroutine context */ - bdrv_truncate_co_entry(&tco); - } else { - co = qemu_coroutine_create(bdrv_truncate_co_entry, &tco); - bdrv_coroutine_enter(child->bs, co); - BDRV_POLL_WHILE(child->bs, tco.ret == NOT_DONE); - } - - return tco.ret; -} diff --git a/block/Makefile.objs b/block/Makefile.objs index 3635b6b4c1..05e4d033c1 100644 --- a/block/Makefile.objs +++ b/block/Makefile.objs @@ -45,6 +45,7 @@ block-obj-y += crypto.o block-obj-y += aio_task.o block-obj-y += backup-top.o block-obj-y += filter-compress.o +block-obj-y += block-gen.o common-obj-y += monitor/ block-obj-y += stream.o diff --git a/scripts/coroutine-wrapper.py b/scripts/coroutine-wrapper.py new file mode 100755 index 0000000000..72115f7dd8 --- /dev/null +++ b/scripts/coroutine-wrapper.py @@ -0,0 +1,168 @@ +#!/usr/bin/env python3 +# +# Generate coroutine wrappers for block subsystem. +# +# Copyright (c) 2020 Virtuozzo International GmbH. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +import re +from typing import List, Iterator + +header = '''/* + * File is generated by scripts/coroutine-wrapper.py + */ + +#include "qemu/osdep.h" +#include "block/coroutines.h" +#include "block/block-gen.h"''' + +template = """ + +/* + * Wrappers for $name$ + */ + +typedef struct $struct_name$ { + BdrvPollCo poll_state; + $fields$ +} $struct_name$; + +static void coroutine_fn $name$_entry(void *opaque) +{ + $struct_name$ *s = opaque; + + $assign_ret$$name$($args_from_s$); + + s->poll_state.in_progress = false; + + bdrv_poll_co__on_exit(); +} + +$ret_type$ $wrapper_name$($args_def$) +{ + if (qemu_in_coroutine()) { + $do_return$$name$($arg_names$); + } else { + $struct_name$ s = { + .poll_state.bs = $bs$, + .poll_state.in_progress = true, + + $initializers$ + }; + + s.poll_state.co = qemu_coroutine_create($name$_entry, &s); + + $do_return$bdrv_poll_co(&s.poll_state); + } +}""" + +# We want to use python string.format() formatter, which uses curly brackets +# as separators. But it's not comfortable with C. So, we used dollars instead, +# and now is the time to escape curly brackets and convert dollars. +template = template.replace('{', '{{').replace('}', '}}') +template = re.sub(r'\$(\w+)\$', r'{\1}', template) + + +class ParamDecl: + param_re = re.compile(r'(?P<decl>' + r'(?P<type>.*[ *])' + r'(?P<name>[a-z][a-z0-9_]*)' + r')') + + def __init__(self, param_decl: str) -> None: + m = self.param_re.match(param_decl.strip()) + self.decl = m.group('decl') + self.type = m.group('type') + self.name = m.group('name') + + +class FuncDecl: + def __init__(self, return_type: str, name: str, args: str) -> None: + self.return_type = return_type.strip() + self.name = name.strip() + self.args: List[ParamDecl] = [] + self.args = [ParamDecl(arg) for arg in args.split(',')] + + def get_args_decl(self) -> str: + return ', '.join(arg.decl for arg in self.args) + + def get_arg_names(self) -> str: + return ', '.join(arg.name for arg in self.args) + + def gen_struct_fields(self) -> str: + return '\n '.join(f'{arg.decl};' for arg in self.args) + + def gen_struct_initializers(self, indent: int) -> str: + sep = '\n' + ' ' * indent + return sep.join(f'.{a.name} = {a.name},' for a in self.args) + + +# Match wrappers declaration, with generated_co_wrapper mark +func_decl_re = re.compile(r'^(?P<return_type>(int|void))' + r'\s*generated_co_wrapper\s*' + r'(?P<wrapper_name>[a-z][a-z0-9_]*)' + r'\((?P<args>[^)]*)\);$', re.MULTILINE) + + +def func_decl_iter(text: str) -> Iterator: + for m in func_decl_re.finditer(text): + yield FuncDecl(return_type=m.group('return_type'), + name=m.group('wrapper_name'), + args=m.group('args')) + + +def struct_name(func_name: str) -> str: + """some_function_name -> SomeFunctionName""" + words = func_name.split('_') + words = [w[0].upper() + w[1:] for w in words] + return ''.join(words) + + +def make_wrapper(func: FuncDecl) -> str: + assert func.name.startswith('bdrv_') + co_name = 'bdrv_co_' + func.name[5:] + + has_ret = func.return_type != 'void' + + params = { + 'name': co_name, + 'do_return': 'return ' if has_ret else '', + 'assign_ret': 's->poll_state.ret = ' if has_ret else '', + 'struct_name': struct_name(co_name), + 'wrapper_name': func.name, + 'ret_type': func.return_type, + 'args_def': func.get_args_decl(), + 'arg_names': func.get_arg_names(), + 'fields': func.gen_struct_fields(), + 'initializers': func.gen_struct_initializers(12), + 'args_from_s': ', '.join(f's->{a.name}' for a in func.args), + } + + if func.args[0].type == 'BlockDriverState *': + params['bs'] = 'bs' + else: + assert func.args[0].type == 'BdrvChild *' + params['bs'] = 'child->bs' + + return template.format(**params) + + +if __name__ == '__main__': + import sys + + print(header) + for func in func_decl_iter(sys.stdin.read()): + print(make_wrapper(func))
We have a very frequent pattern of creating coroutine from function with several arguments: - create structure to pack parameters - create _entry function to call original function taking parameters from struct - do different magic to handle completion: set ret to NOT_DONE or EINPROGRESS, use separate bool for void functions - fill the struct and create coroutine from _entry function and this struct as a parameter - do coroutine enter and BDRV_POLL_WHILE loop Let's reduce code duplication. Here: Functional part (BDRV_POLL_WHILE loop, aio_wait_kick()) moved to (non-generated) block/block-gen.h Mechanical part (arguments packing, different kind of needed wrappers) are generated from template by scripts/coroutine-wrapper.py to resulting file block/block-gen.c Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> --- Makefile | 6 + Makefile.objs | 2 +- block/block-gen.h | 55 ++++++ block/coroutines.h | 7 +- include/block/block.h | 17 +- include/block/generated-co-wrapper.h | 35 ++++ block.c | 70 -------- block/io.c | 260 --------------------------- block/Makefile.objs | 1 + scripts/coroutine-wrapper.py | 168 +++++++++++++++++ 10 files changed, 281 insertions(+), 340 deletions(-) create mode 100644 block/block-gen.h create mode 100644 include/block/generated-co-wrapper.h create mode 100755 scripts/coroutine-wrapper.py