[v2] semaphore: fix a hangup problem under load on NetBSD hosts.

Fix following bugs in "fallback implementation of counting semaphores
with mutex+condvar" added in c166cb72f1676855816340666c3b618beef4b976:
 - waiting threads are not restarted properly if more than one threads
   are waiting unblock signals in qemu_sem_timedwait()
 - possible missing pthread_cond_signal(3) calls when waiting threads
   are returned by ETIMEDOUT
 - fix an uninitialized variable
The problem is analyzed by and fix is provided by Noriyuki Soda.

Also put additional cleanup suggested by Laszlo Ersek:
 - make QemuSemaphore.count unsigned (it won't be negative)
 - check a return value of in pthread_cond_wait() in qemu_sem_wait()

Signed-off-by: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
---

 v2:
 - make QemuSemaphore.count unsigned (it won't be negative)
 - also eliminate checks for negative count values
 - check a return value of in pthread_cond_wait() in qemu_sem_wait()

 include/qemu/thread-posix.h |  2 +-
 util/qemu-thread-posix.c    | 26 +++++++++++++++-----------
 2 files changed, 16 insertions(+), 12 deletions(-)

On 07/02/13 18:54, Izumi Tsutsui wrote:
> Fix following bugs in "fallback implementation of counting semaphores
> with mutex+condvar" added in c166cb72f1676855816340666c3b618beef4b976:
>  - waiting threads are not restarted properly if more than one threads
>    are waiting unblock signals in qemu_sem_timedwait()
>  - possible missing pthread_cond_signal(3) calls when waiting threads
>    are returned by ETIMEDOUT
>  - fix an uninitialized variable
> The problem is analyzed by and fix is provided by Noriyuki Soda.
> 
> Also put additional cleanup suggested by Laszlo Ersek:
>  - make QemuSemaphore.count unsigned (it won't be negative)
>  - check a return value of in pthread_cond_wait() in qemu_sem_wait()
> 
> Signed-off-by: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
> Reviewed-by: Laszlo Ersek <lersek@redhat.com>
> ---
> 
>  v2:
>  - make QemuSemaphore.count unsigned (it won't be negative)
>  - also eliminate checks for negative count values
>  - check a return value of in pthread_cond_wait() in qemu_sem_wait()

I've compared this patch against v1. I can see one problem near the end:

> 
>  include/qemu/thread-posix.h |  2 +-
>  util/qemu-thread-posix.c    | 26 +++++++++++++++-----------
>  2 files changed, 16 insertions(+), 12 deletions(-)
> 
> diff --git a/include/qemu/thread-posix.h b/include/qemu/thread-posix.h
> index 0f30dcc..361566a 100644
> --- a/include/qemu/thread-posix.h
> +++ b/include/qemu/thread-posix.h
> @@ -15,7 +15,7 @@ struct QemuSemaphore {
>  #if defined(__APPLE__) || defined(__NetBSD__)
>      pthread_mutex_t lock;
>      pthread_cond_t cond;
> -    int count;
> +    unsigned int count;
>  #else
>      sem_t sem;
>  #endif
> diff --git a/util/qemu-thread-posix.c b/util/qemu-thread-posix.c
> index 4489abf..44f6f30 100644
> --- a/util/qemu-thread-posix.c
> +++ b/util/qemu-thread-posix.c
> @@ -170,12 +170,11 @@ void qemu_sem_post(QemuSemaphore *sem)
>  
>  #if defined(__APPLE__) || defined(__NetBSD__)
>      pthread_mutex_lock(&sem->lock);
> -    if (sem->count == INT_MAX) {
> +    if (sem->count == UINT_MAX) {
>          rc = EINVAL;
> -    } else if (sem->count++ < 0) {
> -        rc = pthread_cond_signal(&sem->cond);
>      } else {
> -        rc = 0;
> +        sem->count++;
> +        rc = pthread_cond_signal(&sem->cond);
>      }
>      pthread_mutex_unlock(&sem->lock);
>      if (rc != 0) {
> @@ -207,19 +206,21 @@ int qemu_sem_timedwait(QemuSemaphore *sem, int ms)
>      struct timespec ts;
>  
>  #if defined(__APPLE__) || defined(__NetBSD__)
> +    rc = 0;
>      compute_abs_deadline(&ts, ms);
>      pthread_mutex_lock(&sem->lock);
> -    --sem->count;
> -    while (sem->count < 0) {
> +    while (sem->count == 0) {
>          rc = pthread_cond_timedwait(&sem->cond, &sem->lock, &ts);
>          if (rc == ETIMEDOUT) {
> -            ++sem->count;
>              break;
>          }
>          if (rc != 0) {
>              error_exit(rc, __func__);
>          }
>      }
> +    if (rc != ETIMEDOUT) {
> +        --sem->count;
> +    }
>      pthread_mutex_unlock(&sem->lock);
>      return (rc == ETIMEDOUT ? -1 : 0);
>  #else
> @@ -249,16 +250,19 @@ int qemu_sem_timedwait(QemuSemaphore *sem, int ms)
>  
>  void qemu_sem_wait(QemuSemaphore *sem)
>  {
> +    int rc;
> +
>  #if defined(__APPLE__) || defined(__NetBSD__)
>      pthread_mutex_lock(&sem->lock);
> -    --sem->count;
> -    while (sem->count < 0) {
> +    while (sem->count == 0) {
>          pthread_cond_wait(&sem->cond, &sem->lock);
> +        if (rc != 0) {
> +            error_exit(rc, __func__);
> +        }

You forgot to store the retval of pthread_cond_wait() in "rc", the
(rc!=0) check refers to an indeterminate value.

Otherwise it's good IMO.

Thanks!
Laszlo

>      }
> +    --sem->count;
>      pthread_mutex_unlock(&sem->lock);
>  #else
> -    int rc;
> -
>      do {
>          rc = sem_wait(&sem->sem);
>      } while (rc == -1 && errno == EINTR);
>

Laszlo Ersek wrote:

> On 07/02/13 18:54, Izumi Tsutsui wrote:
 :
> > @@ -249,16 +250,19 @@ int qemu_sem_timedwait(QemuSemaphore *sem, int ms)
> >  
> >  void qemu_sem_wait(QemuSemaphore *sem)
> >  {
> > +    int rc;
> > +
> >  #if defined(__APPLE__) || defined(__NetBSD__)
> >      pthread_mutex_lock(&sem->lock);
> > -    --sem->count;
> > -    while (sem->count < 0) {
> > +    while (sem->count == 0) {
> >          pthread_cond_wait(&sem->cond, &sem->lock);
> > +        if (rc != 0) {
> > +            error_exit(rc, __func__);
> > +        }
> 
> You forgot to store the retval of pthread_cond_wait() in "rc", the
> (rc!=0) check refers to an indeterminate value.

Yes, I was a bit hasty and probably missed gcc warnings.
(there is no qemu_sem_wait() callers currently)

I'll post v3 one.

Thanks,
---
Izumi Tsutsui