| Message ID | 20191016170905.8325-5-vsementsov@virtuozzo.com |
|---|---|
| State | New |
| Headers | show |
| Series | block-copy: memory limit | expand |
On 16.10.19 19:09, Vladimir Sementsov-Ogievskiy wrote: > Introduce an API for some shared splittable resource, like memory. > It's going to be used by backup. Backup uses both read/write io and > copy_range. copy_range may consume memory implictly, so the new API is > abstract: it doesn't allocate any real memory by but only hands out -by > tickets. > > The idea is that we have some total amount of something and callers > should wait in coroutine queue if there is not enough of the resource > at the moment. > > Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> > --- > include/qemu/co-shared-resource.h | 71 +++++++++++++++++++++++++++++ > util/qemu-co-shared-resource.c | 76 +++++++++++++++++++++++++++++++ > util/Makefile.objs | 1 + > 3 files changed, 148 insertions(+) > create mode 100644 include/qemu/co-shared-resource.h > create mode 100644 util/qemu-co-shared-resource.c > > diff --git a/include/qemu/co-shared-resource.h b/include/qemu/co-shared-resource.h > new file mode 100644 > index 0000000000..04c9c3d5be > --- /dev/null > +++ b/include/qemu/co-shared-resource.h > @@ -0,0 +1,71 @@ > +/* > + * Helper functionality for distributing a fixed total amount of > + * an abstract resource among multiple coroutines. > + * > + * Copyright (c) 2019 Virtuozzo International GmbH > + * > + * Permission is hereby granted, free of charge, to any person obtaining a copy > + * of this software and associated documentation files (the "Software"), to deal > + * in the Software without restriction, including without limitation the rights > + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell > + * copies of the Software, and to permit persons to whom the Software is > + * furnished to do so, subject to the following conditions: > + * > + * The above copyright notice and this permission notice shall be included in > + * all copies or substantial portions of the Software. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR > + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, > + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL > + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER > + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, > + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN > + * THE SOFTWARE. > + */ > + > +#ifndef QEMU_CO_SHARED_AMOUNT_H > +#define QEMU_CO_SHARED_AMOUNT_H You should probably adjust these include guards to match the new filename. With those bits fixed: Reviewed-by: Max Reitz <mreitz@redhat.com>
22.10.2019 13:23, Max Reitz wrote: > On 16.10.19 19:09, Vladimir Sementsov-Ogievskiy wrote: >> Introduce an API for some shared splittable resource, like memory. >> It's going to be used by backup. Backup uses both read/write io and >> copy_range. copy_range may consume memory implictly, so the new API is >> abstract: it doesn't allocate any real memory by but only hands out > > -by > >> tickets. >> >> The idea is that we have some total amount of something and callers >> should wait in coroutine queue if there is not enough of the resource >> at the moment. >> >> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> >> --- >> include/qemu/co-shared-resource.h | 71 +++++++++++++++++++++++++++++ >> util/qemu-co-shared-resource.c | 76 +++++++++++++++++++++++++++++++ >> util/Makefile.objs | 1 + >> 3 files changed, 148 insertions(+) >> create mode 100644 include/qemu/co-shared-resource.h >> create mode 100644 util/qemu-co-shared-resource.c >> >> diff --git a/include/qemu/co-shared-resource.h b/include/qemu/co-shared-resource.h >> new file mode 100644 >> index 0000000000..04c9c3d5be >> --- /dev/null >> +++ b/include/qemu/co-shared-resource.h >> @@ -0,0 +1,71 @@ >> +/* >> + * Helper functionality for distributing a fixed total amount of >> + * an abstract resource among multiple coroutines. >> + * >> + * Copyright (c) 2019 Virtuozzo International GmbH >> + * >> + * Permission is hereby granted, free of charge, to any person obtaining a copy >> + * of this software and associated documentation files (the "Software"), to deal >> + * in the Software without restriction, including without limitation the rights >> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell >> + * copies of the Software, and to permit persons to whom the Software is >> + * furnished to do so, subject to the following conditions: >> + * >> + * The above copyright notice and this permission notice shall be included in >> + * all copies or substantial portions of the Software. >> + * >> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR >> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, >> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL >> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER >> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, >> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN >> + * THE SOFTWARE. >> + */ >> + >> +#ifndef QEMU_CO_SHARED_AMOUNT_H >> +#define QEMU_CO_SHARED_AMOUNT_H > > You should probably adjust these include guards to match the new filename. > > With those bits fixed: > > Reviewed-by: Max Reitz <mreitz@redhat.com> > Thank for reviewing! I'll fix it and resend.
diff --git a/include/qemu/co-shared-resource.h b/include/qemu/co-shared-resource.h new file mode 100644 index 0000000000..04c9c3d5be --- /dev/null +++ b/include/qemu/co-shared-resource.h @@ -0,0 +1,71 @@ +/* + * Helper functionality for distributing a fixed total amount of + * an abstract resource among multiple coroutines. + * + * Copyright (c) 2019 Virtuozzo International GmbH + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#ifndef QEMU_CO_SHARED_AMOUNT_H +#define QEMU_CO_SHARED_AMOUNT_H + + +typedef struct SharedResource SharedResource; + +/* + * Create SharedResource structure + * + * @total: total amount of some resource to be shared between clients + * + * Note: this API is not thread-safe. + */ +SharedResource *shres_create(uint64_t total); + +/* + * Release SharedResource structure + * + * This function may only be called once everything allocated by all + * clients has been deallocated. + */ +void shres_destroy(SharedResource *s); + +/* + * Try to allocate an amount of @n. Return true on success, and false + * if there is too little left of the collective resource to fulfill + * the request. + */ +bool co_try_get_from_shres(SharedResource *s, uint64_t n); + +/* + * Allocate an amount of @n, and, if necessary, yield until + * that becomes possible. + */ +void coroutine_fn co_get_from_shres(SharedResource *s, uint64_t n); + +/* + * Deallocate an amount of @n. The total amount allocated by a caller + * does not need to be deallocated/released with a single call, but may + * be split over several calls. For example, get(4), get(3), and then + * put(5), put(2). + */ +void coroutine_fn co_put_to_shres(SharedResource *s, uint64_t n); + + +#endif /* QEMU_CO_SHARED_AMOUNT_H */ diff --git a/util/qemu-co-shared-resource.c b/util/qemu-co-shared-resource.c new file mode 100644 index 0000000000..1c83cd9d29 --- /dev/null +++ b/util/qemu-co-shared-resource.c @@ -0,0 +1,76 @@ +/* + * Helper functionality for distributing a fixed total amount of + * an abstract resource among multiple coroutines. + * + * Copyright (c) 2019 Virtuozzo International GmbH + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#include "qemu/osdep.h" +#include "qemu/coroutine.h" +#include "qemu/co-shared-resource.h" + +struct SharedResource { + uint64_t total; + uint64_t available; + + CoQueue queue; +}; + +SharedResource *shres_create(uint64_t total) +{ + SharedResource *s = g_new0(SharedResource, 1); + + s->total = s->available = total; + qemu_co_queue_init(&s->queue); + + return s; +} + +void shres_destroy(SharedResource *s) +{ + assert(s->available == s->total); + g_free(s); +} + +bool co_try_get_from_shres(SharedResource *s, uint64_t n) +{ + if (s->available >= n) { + s->available -= n; + return true; + } + + return false; +} + +void coroutine_fn co_get_from_shres(SharedResource *s, uint64_t n) +{ + assert(n <= s->total); + while (!co_try_get_from_shres(s, n)) { + qemu_co_queue_wait(&s->queue, NULL); + } +} + +void coroutine_fn co_put_to_shres(SharedResource *s, uint64_t n) +{ + assert(s->total - s->available >= n); + s->available += n; + qemu_co_queue_restart_all(&s->queue); +} diff --git a/util/Makefile.objs b/util/Makefile.objs index 41bf59d127..df124af1c5 100644 --- a/util/Makefile.objs +++ b/util/Makefile.objs @@ -37,6 +37,7 @@ util-obj-y += rcu.o util-obj-$(CONFIG_MEMBARRIER) += sys_membarrier.o util-obj-y += qemu-coroutine.o qemu-coroutine-lock.o qemu-coroutine-io.o util-obj-y += qemu-coroutine-sleep.o +util-obj-y += qemu-co-shared-resource.o util-obj-y += coroutine-$(CONFIG_COROUTINE_BACKEND).o util-obj-y += buffer.o util-obj-y += timed-average.o
Introduce an API for some shared splittable resource, like memory. It's going to be used by backup. Backup uses both read/write io and copy_range. copy_range may consume memory implictly, so the new API is abstract: it doesn't allocate any real memory by but only hands out tickets. The idea is that we have some total amount of something and callers should wait in coroutine queue if there is not enough of the resource at the moment. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> --- include/qemu/co-shared-resource.h | 71 +++++++++++++++++++++++++++++ util/qemu-co-shared-resource.c | 76 +++++++++++++++++++++++++++++++ util/Makefile.objs | 1 + 3 files changed, 148 insertions(+) create mode 100644 include/qemu/co-shared-resource.h create mode 100644 util/qemu-co-shared-resource.c