我有3个需要同步的进程.进程一做某事然后唤醒进程二并休眠,这会做一些事情然后唤醒进程三并休眠,这会做一些事情并唤醒进程并休眠.整个循环定时运行大约25hz(在我的“真实”应用程序中触发进程2之前,由外部同步进入进程1).我使用sem_post来触发(唤醒)每个进程,并使用sem_timedwait()来等待触发器.

这一切都成功地工作了几个小时.但是在某个随机时间(通常在两到四个小时之后),其中一个进程在sem_timedwait()中开始超时,即使我确定使用sem_post()触发了信号量.为了证明这一点,我甚至在超时后立即使用sem_getvalue(),并且值为1,因此应该触发timedwait.

请参阅以下代码：

#include <stdio.h>
#include <time.h>
#include <string.h>
#include <errno.h>
#include <semaphore.h>

sem_t trigger_sem1, trigger_sem2, trigger_sem3;

// The main thread process.  Called three times with a different num arg - 1, 2 or 3.
void *thread(void *arg)
{
  int num = (int) arg;
  sem_t *wait, *trigger;
  int val, retval;
  struct timespec ts;
  struct timeval tv;

  switch (num)
    {
      case 1:
        wait = &trigger_sem1;
        trigger = &trigger_sem2;
        break;
      case 2:
        wait = &trigger_sem2;
        trigger = &trigger_sem3;
        break;
      case 3:
        wait = &trigger_sem3;
        trigger = &trigger_sem1;
        break;
    }

  while (1)
    {
      // The first thread delays by 40ms to time the whole loop.  
      // This is an external sync in the real app.
      if (num == 1)   
        usleep(40000);

      // print sem value before we wait.  If this is 1, sem_timedwait() will
      // return immediately, otherwise it will block until sem_post() is called on this sem. 
      sem_getvalue(wait, &val);
      printf("sem%d wait sync sem%d. val before %d\n", num, num, val);

          // get current time and add half a second for timeout.
      gettimeofday(&tv, NULL);
      ts.tv_sec = tv.tv_sec;
      ts.tv_nsec = (tv.tv_usec + 500000);    // add half a second
      if (ts.tv_nsec > 1000000)
        {
          ts.tv_sec++;
          ts.tv_nsec -= 1000000;
        }
      ts.tv_nsec *= 1000;    /* convert to nanosecs */

      retval = sem_timedwait(wait, &ts);
      if (retval == -1)
        {
          // timed out.  Print value of sem now.  This should be 0, otherwise sem_timedwait
          // would have woken before timeout (unless the sem_post happened between the 
          // timeout and this call to sem_getvalue).
          sem_getvalue(wait, &val);
          printf("!!!!!!    sem%d sem_timedwait failed: %s, val now %d\n", 
            num, strerror(errno), val);
        }
      else
        printf("sem%d wakeup.\n", num);

        // get value of semaphore to trigger.  If it's 1, don't post as it has already been 
        // triggered and sem_timedwait on this sem *should* not block.
      sem_getvalue(trigger, &val);
      if (val <= 0)
        {
          printf("sem%d send sync sem%d. val before %d\n", num, (num == 3 ? 1 : num+1), val);
          sem_post(trigger);
        }
      else
        printf("!! sem%d not sending sync, val %d\n", num, val);
    }
}



int main(int argc, char *argv[])
{
  pthread_t t1, t2, t3;

   // create semaphores.  val of sem1 is 1 to trigger straight away and start the whole ball rolling.
  if (sem_init(&trigger_sem1, 0, 1) == -1)
    perror("Error creating trigger_listman semaphore");
  if (sem_init(&trigger_sem2, 0, 0) == -1)
    perror("Error creating trigger_comms semaphore");
  if (sem_init(&trigger_sem3, 0, 0) == -1)
    perror("Error creating trigger_vws semaphore");

  pthread_create(&t1, NULL, thread, (void *) 1);
  pthread_create(&t2, NULL, thread, (void *) 2);
  pthread_create(&t3, NULL, thread, (void *) 3);

  pthread_join(t1, NULL);
  pthread_join(t2, NULL);
  pthread_join(t3, NULL);
}

当程序正确运行时(在开始时和随机但很长时间后)打印以下输出. sem1的值在thread1等待40ms之前始终为1,此时sem3已触发它,因此它会立即唤醒.其他两个线程一直等到从前一个线程收到信号量.

[...]
sem1 wait sync sem1. val before 1
sem1 wakeup.
sem1 send sync sem2. val before 0
sem2 wakeup.
sem2 send sync sem3. val before 0
sem2 wait sync sem2. val before 0
sem3 wakeup.
sem3 send sync sem1. val before 0
sem3 wait sync sem3. val before 0
sem1 wait sync sem1. val before 1
sem1 wakeup.
sem1 send sync sem2. val before 0
[...]

但是,几个小时后,其中一个线程开始超时.我可以从输出中看到信号量被触发,当我在超时后打印该值时,它是1.所以sem_timedwait应该在超时之前很好地唤醒.在超时之后我永远不会期望信号量的值为1,除非在超时之后但在我调用sem_getvalue之前触发发生非常罕见的场合(几乎肯定从不但可能).

此外,一旦它开始失败,该信号量上的每个sem_timedwait()也会以相同的方式失败.请参阅以下输出,我已对其进行了编号：

01  sem3 wait sync sem3. val before 0
02  sem1 wakeup.
03  sem1 send sync sem2. val before 0
04  sem2 wakeup.
05  sem2 send sync sem3. val before 0
06  sem2 wait sync sem2. val before 0
07  sem1 wait sync sem1. val before 0
08  !!!!!!    sem3 sem_timedwait failed: Connection timed out, val now 1
09  sem3 send sync sem1. val before 0
10  sem3 wait sync sem3. val before 1
11  sem3 wakeup.
12  !! sem3 not sending sync, val 1
13  sem3 wait sync sem3. val before 0
14  sem1 wakeup.
[...]

在第1行,线程3(我在printf中混淆地称为sem3)等待触发sem3.在第5行,thread2为sem3调用sem_post.但是,第8行显示sem3超时,但信号量的值为1. thread3然后触发sem1并再次等待(10).但是,因为该值已经为1,所以它会立即唤醒.它不再发送sem1,因为这一切都发生在对thread1给出控制之前,然后它再次等待(val现在为0)并且sem1唤醒.这现在重复,sem3总是超时并显示值为1.

所以,我的问题是为什么sem3超时,即使信号量已被触发且值明显为1？我永远不会期望在输出中看到第08行.如果它超时(因为,假设线程2已经崩溃或花费的时间太长),则该值应为0.为什么它在进入此状态之前首先工作3或4小时？

我尝试过使用三个独立程序进行类似测试,通过共享内存进行通信,而不是同一程序中的三个线程.这更类似于我的真实世界应用程序.结果和输出是一样的.问题确实出现在信号量(特别是sem_timedwait调用)中,而不是与pthreads有关.

我也尝试过更短和更长的延迟,以及完全消除延迟,结果与上述类似.没有任何延迟,它有时会在几分钟而不是几小时后开始产生错误.这当然意味着可以更快地再现问题.

这是使用Ubuntu 9.4和内核2.6.28.相同的程序在Redhat和Fedora上运行正常,但我现在正试图移植到Ubuntu.我也试过使用Ubuntu 9.10,没有任何区别.

谢谢你的建议,
贾尔斯

解决方法:

问题似乎来自传递无效的超时参数.

至少在我的机器上,第一次失败不是ETIMEDOUT,而是：

!!!!!! sem2 sem_timedwait failed：参数无效,val现在为0

现在,如果我写：

  if (ts.tv_nsec >= 1000000)

(注意添加=)然后它工作正常.这是另一个问题,为什么信号量的状态(大概)会消失,以便在后续尝试中超时,或者直接在sem_wait上直接阻塞.看起来像libc或内核中的错误.