Patchwork [4/7] add win32 qemu-thread implementation

login
register
mail settings
Submitter Paolo Bonzini
Date Feb. 10, 2011, 5:37 p.m.
Message ID <1297359464-9789-5-git-send-email-pbonzini@redhat.com>
Download mbox | patch
Permalink /patch/82633/
State New
Headers show

Comments

Paolo Bonzini - Feb. 10, 2011, 5:37 p.m.
For now, qemu_cond_timedwait and qemu_mutex_timedlock are left as
POSIX-only functions.  They can be removed later, once the patches
that remove their uses are in.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Cc: Stefan Weil <weil@mail.berlios.de>
Cc: Blue Swirl <blauwirbel@gmail.com>
---
 Makefile.objs                        |    4 +-
 qemu-thread.c => qemu-thread-posix.c |    0
 qemu-thread-posix.h                  |   18 +++
 qemu-thread-win32.c                  |  272 ++++++++++++++++++++++++++++++++++
 qemu-thread-win32.h                  |   22 +++
 qemu-thread.h                        |   27 ++--
 6 files changed, 326 insertions(+), 17 deletions(-)
 rename qemu-thread.c => qemu-thread-posix.c (100%)
 create mode 100644 qemu-thread-posix.h
 create mode 100644 qemu-thread-win32.c
 create mode 100644 qemu-thread-win32.h
Stefan Weil - Feb. 10, 2011, 7:46 p.m.
Am 10.02.2011 18:37, schrieb Paolo Bonzini:
> For now, qemu_cond_timedwait and qemu_mutex_timedlock are left as
> POSIX-only functions. They can be removed later, once the patches
> that remove their uses are in.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Stefan Weil <weil@mail.berlios.de>
> Cc: Blue Swirl <blauwirbel@gmail.com>
> ---
> Makefile.objs | 4 +-
> qemu-thread.c => qemu-thread-posix.c | 0
> qemu-thread-posix.h | 18 +++
> qemu-thread-win32.c | 272 ++++++++++++++++++++++++++++++++++
> qemu-thread-win32.h | 22 +++
> qemu-thread.h | 27 ++--
> 6 files changed, 326 insertions(+), 17 deletions(-)
> rename qemu-thread.c => qemu-thread-posix.c (100%)
> create mode 100644 qemu-thread-posix.h
> create mode 100644 qemu-thread-win32.c
> create mode 100644 qemu-thread-win32.h
>

What about using MinGW's pthread support?
http://sourceforge.net/projects/mingw/files/MinGW/pthreads-w32/

I used it for the threaded vnc server in QEMU, but I don't
have more practical experience with it.

Regards,
Stefan
Paolo Bonzini - Feb. 11, 2011, 12:28 p.m.
On 02/10/2011 08:46 PM, Stefan Weil wrote:
> Am 10.02.2011 18:37, schrieb Paolo Bonzini:
>> For now, qemu_cond_timedwait and qemu_mutex_timedlock are left as
>> POSIX-only functions. They can be removed later, once the patches
>> that remove their uses are in.
>>
>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>> Cc: Stefan Weil <weil@mail.berlios.de>
>> Cc: Blue Swirl <blauwirbel@gmail.com>
>> ---
>> Makefile.objs | 4 +-
>> qemu-thread.c => qemu-thread-posix.c | 0
>> qemu-thread-posix.h | 18 +++
>> qemu-thread-win32.c | 272 ++++++++++++++++++++++++++++++++++
>> qemu-thread-win32.h | 22 +++
>> qemu-thread.h | 27 ++--
>> 6 files changed, 326 insertions(+), 17 deletions(-)
>> rename qemu-thread.c => qemu-thread-posix.c (100%)
>> create mode 100644 qemu-thread-posix.h
>> create mode 100644 qemu-thread-win32.c
>> create mode 100644 qemu-thread-win32.h
>>
>
> What about using MinGW's pthread support?
> http://sourceforge.net/projects/mingw/files/MinGW/pthreads-w32/
>
> I used it for the threaded vnc server in QEMU, but I don't
> have more practical experience with it.

1) For threads, we'd need to do the same in order to keep the HANDLE for 
iothread IPIs (SuspendThread/ResumeThread).  We could use nonportable 
functions from pthread-w32, but if for example in the future QEMU 
switched to glib these wouldn't be available.  So QEMU would need a 
separate Win32 implementation anyway sooner or later.

2) For condvars, we can make some more assumptions and make it 
simpler/faster.

3) It has some namespace pollution issues.  These wouldn't be a problem 
for QEMU at the moment, however.

Paolo
Blue Swirl - Feb. 20, 2011, 9:11 p.m.
On Fri, Feb 11, 2011 at 2:28 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> On 02/10/2011 08:46 PM, Stefan Weil wrote:
>>
>> Am 10.02.2011 18:37, schrieb Paolo Bonzini:
>>>
>>> For now, qemu_cond_timedwait and qemu_mutex_timedlock are left as
>>> POSIX-only functions. They can be removed later, once the patches
>>> that remove their uses are in.
>>>
>>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>>> Cc: Stefan Weil <weil@mail.berlios.de>
>>> Cc: Blue Swirl <blauwirbel@gmail.com>
>>> ---
>>> Makefile.objs | 4 +-
>>> qemu-thread.c => qemu-thread-posix.c | 0
>>> qemu-thread-posix.h | 18 +++
>>> qemu-thread-win32.c | 272 ++++++++++++++++++++++++++++++++++
>>> qemu-thread-win32.h | 22 +++
>>> qemu-thread.h | 27 ++--
>>> 6 files changed, 326 insertions(+), 17 deletions(-)
>>> rename qemu-thread.c => qemu-thread-posix.c (100%)
>>> create mode 100644 qemu-thread-posix.h
>>> create mode 100644 qemu-thread-win32.c
>>> create mode 100644 qemu-thread-win32.h
>>>
>>
>> What about using MinGW's pthread support?
>> http://sourceforge.net/projects/mingw/files/MinGW/pthreads-w32/
>>
>> I used it for the threaded vnc server in QEMU, but I don't
>> have more practical experience with it.
>
> 1) For threads, we'd need to do the same in order to keep the HANDLE for
> iothread IPIs (SuspendThread/ResumeThread).  We could use nonportable
> functions from pthread-w32, but if for example in the future QEMU switched
> to glib these wouldn't be available.  So QEMU would need a separate Win32
> implementation anyway sooner or later.
>
> 2) For condvars, we can make some more assumptions and make it
> simpler/faster.
>
> 3) It has some namespace pollution issues.  These wouldn't be a problem for
> QEMU at the moment, however.

This patch no longer applies, please rebase.

Patch

diff --git a/Makefile.objs b/Makefile.objs
index 353b1a8..19c31fc 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -140,8 +140,8 @@  endif
 common-obj-y += $(addprefix ui/, $(ui-obj-y))
 
 common-obj-y += iov.o acl.o
-common-obj-$(CONFIG_THREAD) += qemu-thread.o
-common-obj-$(CONFIG_IOTHREAD) += compatfd.o
+common-obj-$(CONFIG_POSIX) += qemu-thread-posix.o compatfd.o
+common-obj-$(CONFIG_WIN32) += qemu-thread-win32.o
 common-obj-y += notify.o event_notifier.o
 common-obj-y += qemu-timer.o qemu-timer-common.o
 
diff --git a/qemu-thread.c b/qemu-thread-posix.c
similarity index 100%
rename from qemu-thread.c
rename to qemu-thread-posix.c
diff --git a/qemu-thread-posix.h b/qemu-thread-posix.h
new file mode 100644
index 0000000..7af371c
--- /dev/null
+++ b/qemu-thread-posix.h
@@ -0,0 +1,18 @@ 
+#ifndef __QEMU_THREAD_POSIX_H
+#define __QEMU_THREAD_POSIX_H 1
+#include "pthread.h"
+
+struct QemuMutex {
+    pthread_mutex_t lock;
+};
+
+struct QemuCond {
+    pthread_cond_t cond;
+};
+
+struct QemuThread {
+    pthread_t thread;
+};
+
+void qemu_thread_signal(QemuThread *thread, int sig);
+#endif
diff --git a/qemu-thread-win32.c b/qemu-thread-win32.c
new file mode 100644
index 0000000..0465a9a
--- /dev/null
+++ b/qemu-thread-win32.c
@@ -0,0 +1,272 @@ 
+/*
+ * Win32 implementation for mutex/cond/thread functions
+ *
+ * Copyright Red Hat, Inc. 2010
+ *
+ * Author:
+ *  Paolo Bonzini <pbonzini@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+#include "qemu-common.h"
+#include "qemu-thread.h"
+#include <process.h>
+#include <assert.h>
+#include <limits.h>
+
+static void error_exit(int err, const char *msg)
+{
+    char *pstr;
+
+    FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_ALLOCATE_BUFFER,
+                  NULL, err, 0, (LPTSTR) &pstr, 2, NULL);
+    fprintf(stderr, "qemu: %s: %s\n", msg, pstr);
+    LocalFree(pstr);
+    exit(1);
+}
+
+void qemu_mutex_init(QemuMutex *mutex)
+{
+    mutex->owner = 0;
+    InitializeCriticalSection(&mutex->lock);
+}
+
+void qemu_mutex_lock(QemuMutex *mutex)
+{
+    EnterCriticalSection(&mutex->lock);
+
+    /* Win32 CRITICAL_SECTIONs are recursive.  Assert that we're not
+     * using them as such.
+     */
+    assert (mutex->owner == 0);
+    mutex->owner = GetCurrentThreadId();
+}
+
+int qemu_mutex_trylock(QemuMutex *mutex)
+{
+    int owned;
+
+    owned = TryEnterCriticalSection(&mutex->lock);
+    if (owned) {
+        assert (mutex->owner == 0);
+        mutex->owner = GetCurrentThreadId();
+    }
+    return !owned;
+}
+
+void qemu_mutex_unlock(QemuMutex *mutex)
+{
+    assert (mutex->owner == GetCurrentThreadId());
+    mutex->owner = 0;
+    LeaveCriticalSection(&mutex->lock);
+}
+
+void qemu_cond_init(QemuCond *cond)
+{
+    cond->waiters = 0;
+    cond->was_broadcast = 0;
+
+    cond->sema = CreateSemaphore(NULL, 0, LONG_MAX, NULL);
+    if (!cond->sema) {
+        error_exit(GetLastError(), __func__);
+    }
+    cond->continue_broadcast = CreateEvent(NULL,    /* security */
+                                           FALSE,   /* auto-reset */
+                                           FALSE,   /* not signaled */
+                                           NULL);   /* name */
+    if (!cond->continue_broadcast) {
+        error_exit(GetLastError(), __func__);
+    }
+}
+
+void qemu_cond_signal(QemuCond *cond)
+{
+    /*
+     * Signal only when there are waiters.  cond->waiters is
+     * incremented by pthread_cond_wait under the external lock,
+     * so we are safe about that.
+     *
+     * Waiting threads decrement it outside the external lock, but
+     * only if another thread is executing pthread_cond_broadcast and
+     * has the mutex.  So, it also cannot be decremented concurrently
+     * with this particular access.
+     */
+    if (cond->waiters > 0) {
+        cond->waiters--;
+        if (!ReleaseSemaphore(cond->sema, 1, NULL)) {
+            error_exit(GetLastError(), __func__);
+        }
+    }
+}
+
+void qemu_cond_broadcast(QemuCond *cond)
+{
+    BOOLEAN result;
+    /*
+     * As in pthread_cond_signal, access to cond->waiters and
+     * cond->was_broadcast is locked via the external mutex.
+     */
+    if (cond->waiters == 0) {
+        return;
+    }
+
+    /*
+     * As an optimization, when there is exactly one waiter
+     * broadcast is the same as signal.
+     */
+    if (cond->waiters == 1) {
+        qemu_cond_signal(cond);
+        return;
+    }
+
+    cond->was_broadcast = 1;
+    result = ReleaseSemaphore(cond->sema, cond->waiters, NULL);
+    if (!result) {
+        error_exit(GetLastError(), __func__);
+    }
+
+    /*
+     * At this point all waiters continue. Each one takes its
+     * slice of the semaphore. Now it's our turn to wait: Since
+     * the external mutex is held, no thread can leave cond_wait,
+     * yet. For this reason, we can be sure that no thread gets
+     * a chance to eat *more* than one slice. OTOH, it means
+     * that the last waiter must send us a wake-up.
+     */
+    WaitForSingleObject(cond->continue_broadcast, INFINITE);
+    cond->was_broadcast = 0;
+}
+
+void qemu_cond_wait(QemuCond *cond, QemuMutex *mutex)
+{
+    /*
+     * This access is protected under the mutex.
+     */
+    cond->waiters++;
+
+    /*
+     * Unlock external mutex and wait for signal.
+     * NOTE: we've held mutex locked long enough to increment
+     * waiters count above, so there's no problem with
+     * leaving mutex unlocked before we wait on semaphore.
+     */
+    qemu_mutex_unlock(mutex);
+    WaitForSingleObject(cond->sema, INFINITE);
+
+    /* If the condvar was broadcast, waiters must rendez-vous with
+     * the broadcasting thread and let it continue.  For cond_signal
+     * we do not have to do that because only one signal was added
+     * to the semaphore (also, cond_signal will decrement num_waiters
+     * itself).
+     *
+     * Yes, this has heavy contention and triggers thundering herd.
+     * So goes life.
+     */
+    if (cond->was_broadcast) {
+        /*
+         * Decrease waiters count.  The mutex is not taken, so we have
+         * to do this atomically.
+         *
+         * cond_broadcast was issued while mutex was held, so all
+         * waiters contend for the mutex at the end of this function
+         * until the broadcasting thread relinquishes it.  To ensure
+         * each waiter consumes exactly one slice of the semaphore,
+         * the broadcasting thread stops until it is told by the last
+         * waiter that it can go on.
+         */
+        if (InterlockedDecrement(&cond->waiters) == 0) {
+            SetEvent(cond->continue_broadcast);
+        }
+    }
+    /* lock external mutex again */
+    qemu_mutex_lock(mutex);
+}
+
+struct qemu_thread_data {
+    QemuThread *thread;
+    void *(*start_routine)(void*);
+    void *arg;
+};
+
+static int qemu_thread_tls_index = TLS_OUT_OF_INDEXES;
+
+static unsigned __stdcall win32_start_routine(void *arg)
+{
+    struct qemu_thread_data data = *(struct qemu_thread_data *) arg;
+    QemuThread *thread = data.thread;
+
+    free(arg);
+    TlsSetValue(qemu_thread_tls_index, thread);
+
+    /*
+     * Use DuplicateHandle instead of assigning thread->thread in the
+     * creating thread to avoid races.  It's simpler this way than with
+     * synchronization.
+     */
+    DuplicateHandle(GetCurrentProcess(), GetCurrentThread(),
+                    GetCurrentProcess(), &thread->thread,
+                    0, FALSE, DUPLICATE_SAME_ACCESS);
+
+    qemu_thread_exit(data.start_routine(data.arg));
+}
+
+void qemu_thread_exit(void *arg)
+{
+    QemuThread *thread = TlsGetValue(qemu_thread_tls_index);
+    thread->ret = arg;
+    CloseHandle(thread->thread);
+    thread->thread = NULL;
+    ExitThread(0);
+}
+
+static inline void qemu_thread_init(void)
+{
+    if (qemu_thread_tls_index == TLS_OUT_OF_INDEXES) {
+        qemu_thread_tls_index = TlsAlloc();
+        if (qemu_thread_tls_index == TLS_OUT_OF_INDEXES) {
+            error_exit(ERROR_NO_SYSTEM_RESOURCES, __func__);
+        }
+    }
+}
+
+
+void qemu_thread_create(QemuThread *thread,
+                       void *(*start_routine)(void*),
+                       void *arg)
+{
+    HANDLE hThread;
+
+    struct qemu_thread_data *data;
+    qemu_thread_init ();
+    data = qemu_malloc(sizeof *data);
+    data->thread = thread;
+    data->start_routine = start_routine;
+    data->arg = arg;
+
+    hThread = (HANDLE) _beginthreadex(NULL, 0, win32_start_routine,
+                                      data, 0, NULL);
+    if (!hThread) {
+        error_exit(GetLastError(), __func__);
+    }
+    CloseHandle(hThread);
+}
+
+void qemu_thread_self(QemuThread *thread)
+{
+    if (!thread->thread) {
+        /* In the main thread of the process.  Initialize the QemuThread
+           pointer in TLS, and use the dummy GetCurrentThread handle as
+           the identifier for qemu_thread_is_current.  */
+        qemu_thread_init ();
+        TlsSetValue(qemu_thread_tls_index, thread);
+        thread->thread = GetCurrentThread();
+    }
+}
+
+int qemu_thread_is_current(QemuThread *thread)
+{
+    QemuThread *this_thread = TlsGetValue(qemu_thread_tls_index);
+    return this_thread->thread == thread->thread;
+}
diff --git a/qemu-thread-win32.h b/qemu-thread-win32.h
new file mode 100644
index 0000000..d7d7d81
--- /dev/null
+++ b/qemu-thread-win32.h
@@ -0,0 +1,22 @@ 
+#ifndef __QEMU_THREAD_WIN32_H
+#define __QEMU_THREAD_WIN32_H 1
+#include "windows.h"
+
+struct QemuMutex {
+    CRITICAL_SECTION lock;
+    LONG owner;
+};
+
+struct QemuCond {
+    LONG waiters;
+    int was_broadcast;
+    HANDLE sema;
+    HANDLE continue_broadcast;
+};
+
+struct QemuThread {
+    HANDLE thread;
+    void *ret;
+};
+
+#endif
diff --git a/qemu-thread.h b/qemu-thread.h
index 19bb30c..a088eba 100644
--- a/qemu-thread.h
+++ b/qemu-thread.h
@@ -1,24 +1,16 @@ 
 #ifndef __QEMU_THREAD_H
 #define __QEMU_THREAD_H 1
-#include "semaphore.h"
-#include "pthread.h"
-
-struct QemuMutex {
-    pthread_mutex_t lock;
-};
-
-struct QemuCond {
-    pthread_cond_t cond;
-};
-
-struct QemuThread {
-    pthread_t thread;
-};
 
 typedef struct QemuMutex QemuMutex;
 typedef struct QemuCond QemuCond;
 typedef struct QemuThread QemuThread;
 
+#ifdef _WIN32
+#include "qemu-thread-win32.h"
+#else
+#include "qemu-thread-posix.h"
+#endif
+
 void qemu_mutex_init(QemuMutex *mutex);
 void qemu_mutex_destroy(QemuMutex *mutex);
 void qemu_mutex_lock(QemuMutex *mutex);
@@ -28,6 +20,12 @@  void qemu_mutex_unlock(QemuMutex *mutex);
 
 void qemu_cond_init(QemuCond *cond);
 void qemu_cond_destroy(QemuCond *cond);
+
+/*
+ * IMPORTANT: The implementation does not guarantee that qemu_cond_signal
+ * and qemu_cond_broadcast can be called except while the same mutex is
+ * held as in the corresponding qemu_cond_wait calls!
+ */
 void qemu_cond_signal(QemuCond *cond);
 void qemu_cond_broadcast(QemuCond *cond);
 void qemu_cond_wait(QemuCond *cond, QemuMutex *mutex);
@@ -36,7 +34,6 @@  int qemu_cond_timedwait(QemuCond *cond, QemuMutex *mutex, uint64_t msecs);
 void qemu_thread_create(QemuThread *thread,
                        void *(*start_routine)(void*),
                        void *arg);
-void qemu_thread_signal(QemuThread *thread, int sig);
 void qemu_thread_self(QemuThread *thread);
 int qemu_thread_equal(QemuThread *thread1, QemuThread *thread2);
 void qemu_thread_exit(void *retval);