From patchwork Tue Jul 26 09:21:15 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 106812 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [140.186.70.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 35E49B70C5 for ; Tue, 26 Jul 2011 19:21:55 +1000 (EST) Received: from localhost ([::1]:43547 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Qldpm-0003X4-1I for incoming@patchwork.ozlabs.org; Tue, 26 Jul 2011 05:21:50 -0400 Received: from eggs.gnu.org ([140.186.70.92]:36036) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QldpN-0002zm-Cv for qemu-devel@nongnu.org; Tue, 26 Jul 2011 05:21:32 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QldpK-0000Cs-Lc for qemu-devel@nongnu.org; Tue, 26 Jul 2011 05:21:25 -0400 Received: from mtagate7.uk.ibm.com ([194.196.100.167]:60661) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QldpK-0000CH-3F for qemu-devel@nongnu.org; Tue, 26 Jul 2011 05:21:22 -0400 Received: from d06nrmr1707.portsmouth.uk.ibm.com (d06nrmr1707.portsmouth.uk.ibm.com [9.149.39.225]) by mtagate7.uk.ibm.com (8.13.1/8.13.1) with ESMTP id p6Q9LKU4016108 for ; Tue, 26 Jul 2011 09:21:20 GMT Received: from d06av06.portsmouth.uk.ibm.com (d06av06.portsmouth.uk.ibm.com [9.149.37.217]) by d06nrmr1707.portsmouth.uk.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id p6Q9LKar2125928 for ; Tue, 26 Jul 2011 10:21:20 +0100 Received: from d06av06.portsmouth.uk.ibm.com (loopback [127.0.0.1]) by d06av06.portsmouth.uk.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id p6Q9LK2B016525 for ; Tue, 26 Jul 2011 03:21:20 -0600 Received: from stefanha-thinkpad.manchester-maybrook.uk.ibm.com (dyn-9-174-219-30.manchester-maybrook.uk.ibm.com [9.174.219.30]) by d06av06.portsmouth.uk.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id p6Q9LIgl016463; Tue, 26 Jul 2011 03:21:19 -0600 From: Stefan Hajnoczi To: Date: Tue, 26 Jul 2011 10:21:15 +0100 Message-Id: <1311672077-4592-4-git-send-email-stefanha@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.5.4 In-Reply-To: <1311672077-4592-1-git-send-email-stefanha@linux.vnet.ibm.com> References: <1311672077-4592-1-git-send-email-stefanha@linux.vnet.ibm.com> MIME-Version: 1.0 X-MIME-Autoconverted: from 8bit to quoted-printable by mtagate7.uk.ibm.com id p6Q9LKU4016108 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6, seldom 2.4 (older, 4) X-Received-From: 194.196.100.167 Cc: Kevin Wolf , Blue Swirl , Anthony Liguori , Venkateswararao Jujjuri , Stefan Hajnoczi Subject: [Qemu-devel] [PATCH v8 3/5] coroutine: add ucontext and win32 implementations X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Kevin Wolf This coroutines implementation is based on the gtk-vnc implementation written by Anthony Liguori but it has been significantly rewritten by Kevin Wolf to use setjmp()/longjmp() instead of the more expensive swapcontext() and by Paolo Bonzini for Windows Fibers support. Darwin has makecontext(3) but getcontext(3) is stubbed out to return ENOTSUP. Andreas Färber debugged this and contributed the ./configure test which solves the issue for Darwin/ppc64 (and ppc) v10.5. Signed-off-by: Kevin Wolf Signed-off-by: Stefan Hajnoczi --- Makefile.objs | 8 ++- configure | 18 ++++ coroutine-ucontext.c | 230 ++++++++++++++++++++++++++++++++++++++++++++++++++ coroutine-win32.c | 91 ++++++++++++++++++++ 4 files changed, 346 insertions(+), 1 deletions(-) create mode 100644 coroutine-ucontext.c create mode 100644 coroutine-win32.c diff --git a/Makefile.objs b/Makefile.objs index db907c4..5679e1f 100644 --- a/Makefile.objs +++ b/Makefile.objs @@ -12,7 +12,13 @@ oslib-obj-$(CONFIG_POSIX) += oslib-posix.o qemu-thread-posix.o ####################################################################### # coroutines -coroutine-obj-y = qemu-coroutine.o coroutine-gthread.o +coroutine-obj-y = qemu-coroutine.o +ifeq ($(CONFIG_UCONTEXT_COROUTINE),y) +coroutine-obj-$(CONFIG_POSIX) += coroutine-ucontext.o +else +coroutine-obj-$(CONFIG_POSIX) += coroutine-gthread.o +endif +coroutine-obj-$(CONFIG_WIN32) += coroutine-win32.o ####################################################################### # block-obj-y is code used by both qemu system emulation and qemu-img diff --git a/configure b/configure index 5150372..a4d7921 100755 --- a/configure +++ b/configure @@ -2499,6 +2499,20 @@ if test "$trace_backend" = "dtrace"; then fi ########################################## +# check if we have makecontext + +ucontext_coroutine=no +if test "$darwin" != "yes"; then + cat > $TMPC << EOF +#include +int main(void) { makecontext(0, 0, 0); } +EOF + if compile_prog "" "" ; then + ucontext_coroutine=yes + fi +fi + +########################################## # End of CC checks # After here, no more $cc or $ld runs @@ -2970,6 +2984,10 @@ if test "$rbd" = "yes" ; then echo "CONFIG_RBD=y" >> $config_host_mak fi +if test "$ucontext_coroutine" = "yes" ; then + echo "CONFIG_UCONTEXT_COROUTINE=y" >> $config_host_mak +fi + # USB host support case "$usb" in linux) diff --git a/coroutine-ucontext.c b/coroutine-ucontext.c new file mode 100644 index 0000000..41c2379 --- /dev/null +++ b/coroutine-ucontext.c @@ -0,0 +1,230 @@ +/* + * ucontext coroutine initialization code + * + * Copyright (C) 2006 Anthony Liguori + * Copyright (C) 2011 Kevin Wolf + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.0 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, see . + */ + +/* XXX Is there a nicer way to disable glibc's stack check for longjmp? */ +#ifdef _FORTIFY_SOURCE +#undef _FORTIFY_SOURCE +#endif +#include +#include +#include +#include +#include +#include "qemu-common.h" +#include "qemu-coroutine-int.h" + +enum { + /* Maximum free pool size prevents holding too many freed coroutines */ + POOL_MAX_SIZE = 64, +}; + +typedef struct { + Coroutine base; + void *stack; + jmp_buf env; +} CoroutineUContext; + +/** + * Per-thread coroutine bookkeeping + */ +typedef struct { + /** Currently executing coroutine */ + Coroutine *current; + + /** Free list to speed up creation */ + QLIST_HEAD(, Coroutine) pool; + unsigned int pool_size; + + /** The default coroutine */ + CoroutineUContext leader; +} CoroutineThreadState; + +static pthread_key_t thread_state_key; + +/* + * va_args to makecontext() must be type 'int', so passing + * the pointer we need may require several int args. This + * union is a quick hack to let us do that + */ +union cc_arg { + void *p; + int i[2]; +}; + +static CoroutineThreadState *coroutine_get_thread_state(void) +{ + CoroutineThreadState *s = pthread_getspecific(thread_state_key); + + if (!s) { + s = qemu_mallocz(sizeof(*s)); + s->current = &s->leader.base; + QLIST_INIT(&s->pool); + pthread_setspecific(thread_state_key, s); + } + return s; +} + +static void qemu_coroutine_thread_cleanup(void *opaque) +{ + CoroutineThreadState *s = opaque; + Coroutine *co; + Coroutine *tmp; + + QLIST_FOREACH_SAFE(co, &s->pool, pool_next, tmp) { + qemu_free(DO_UPCAST(CoroutineUContext, base, co)->stack); + qemu_free(co); + } + qemu_free(s); +} + +static void __attribute__((constructor)) coroutine_init(void) +{ + int ret; + + ret = pthread_key_create(&thread_state_key, qemu_coroutine_thread_cleanup); + if (ret != 0) { + fprintf(stderr, "unable to create leader key: %s\n", strerror(errno)); + abort(); + } +} + +static void coroutine_trampoline(int i0, int i1) +{ + union cc_arg arg; + CoroutineUContext *self; + Coroutine *co; + + arg.i[0] = i0; + arg.i[1] = i1; + self = arg.p; + co = &self->base; + + /* Initialize longjmp environment and switch back the caller */ + if (!setjmp(self->env)) { + longjmp(*(jmp_buf *)co->entry_arg, 1); + } + + while (true) { + co->entry(co->entry_arg); + qemu_coroutine_switch(co, co->caller, COROUTINE_TERMINATE); + } +} + +static Coroutine *coroutine_new(void) +{ + const size_t stack_size = 1 << 20; + CoroutineUContext *co; + ucontext_t old_uc, uc; + jmp_buf old_env; + union cc_arg arg; + + /* The ucontext functions preserve signal masks which incurs a system call + * overhead. setjmp()/longjmp() does not preserve signal masks but only + * works on the current stack. Since we need a way to create and switch to + * a new stack, use the ucontext functions for that but setjmp()/longjmp() + * for everything else. + */ + + if (getcontext(&uc) == -1) { + abort(); + } + + co = qemu_mallocz(sizeof(*co)); + co->stack = qemu_malloc(stack_size); + co->base.entry_arg = &old_env; /* stash away our jmp_buf */ + + uc.uc_link = &old_uc; + uc.uc_stack.ss_sp = co->stack; + uc.uc_stack.ss_size = stack_size; + uc.uc_stack.ss_flags = 0; + + arg.p = co; + + makecontext(&uc, (void (*)(void))coroutine_trampoline, + 2, arg.i[0], arg.i[1]); + + /* swapcontext() in, longjmp() back out */ + if (!setjmp(old_env)) { + swapcontext(&old_uc, &uc); + } + return &co->base; +} + +Coroutine *qemu_coroutine_new(void) +{ + CoroutineThreadState *s = coroutine_get_thread_state(); + Coroutine *co; + + co = QLIST_FIRST(&s->pool); + if (co) { + QLIST_REMOVE(co, pool_next); + s->pool_size--; + } else { + co = coroutine_new(); + } + return co; +} + +void qemu_coroutine_delete(Coroutine *co_) +{ + CoroutineThreadState *s = coroutine_get_thread_state(); + CoroutineUContext *co = DO_UPCAST(CoroutineUContext, base, co_); + + if (s->pool_size < POOL_MAX_SIZE) { + QLIST_INSERT_HEAD(&s->pool, &co->base, pool_next); + co->base.caller = NULL; + s->pool_size++; + return; + } + + qemu_free(co->stack); + qemu_free(co); +} + +CoroutineAction qemu_coroutine_switch(Coroutine *from_, Coroutine *to_, + CoroutineAction action) +{ + CoroutineUContext *from = DO_UPCAST(CoroutineUContext, base, from_); + CoroutineUContext *to = DO_UPCAST(CoroutineUContext, base, to_); + CoroutineThreadState *s = coroutine_get_thread_state(); + int ret; + + s->current = to_; + + ret = setjmp(from->env); + if (ret == 0) { + longjmp(to->env, action); + } + return ret; +} + +Coroutine *qemu_coroutine_self(void) +{ + CoroutineThreadState *s = coroutine_get_thread_state(); + + return s->current; +} + +bool qemu_in_coroutine(void) +{ + CoroutineThreadState *s = pthread_getspecific(thread_state_key); + + return s && s->current->caller; +} diff --git a/coroutine-win32.c b/coroutine-win32.c new file mode 100644 index 0000000..a13d1d1 --- /dev/null +++ b/coroutine-win32.c @@ -0,0 +1,91 @@ +/* + * Win32 coroutine initialization code + * + * Copyright (c) 2011 Kevin Wolf + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#include "qemu-common.h" +#include "qemu-coroutine-int.h" + +typedef struct { + Coroutine base; + + LPVOID fiber; + CoroutineAction action; +} CoroutineWin32; + +static __thread CoroutineWin32 leader; +static __thread Coroutine *current; + +CoroutineAction qemu_coroutine_switch(Coroutine *from_, Coroutine *to_, + CoroutineAction action) +{ + CoroutineWin32 *from = DO_UPCAST(CoroutineWin32, base, from_); + CoroutineWin32 *to = DO_UPCAST(CoroutineWin32, base, to_); + + current = to_; + + to->action = action; + SwitchToFiber(to->fiber); + return from->action; +} + +static void CALLBACK coroutine_trampoline(void *co_) +{ + Coroutine *co = co_; + + while (true) { + co->entry(co->entry_arg); + qemu_coroutine_switch(co, co->caller, COROUTINE_TERMINATE); + } +} + +Coroutine *qemu_coroutine_new(void) +{ + const size_t stack_size = 1 << 20; + CoroutineWin32 *co; + + co = qemu_mallocz(sizeof(*co)); + co->fiber = CreateFiber(stack_size, coroutine_trampoline, &co->base); + return &co->base; +} + +void qemu_coroutine_delete(Coroutine *co_) +{ + CoroutineWin32 *co = DO_UPCAST(CoroutineWin32, base, co_); + + DeleteFiber(co->fiber); + qemu_free(co); +} + +Coroutine *qemu_coroutine_self(void) +{ + if (!current) { + current = &leader.base; + leader.fiber = ConvertThreadToFiber(NULL); + } + return current; +} + +bool qemu_in_coroutine(void) +{ + return current && current->caller; +}