Mutex, Cond and Thread questions (pthread, linux)

Hello All,

The scenario of my problem:
I have 3 threads with one mutex, one cond and a variable to share with.
The first thread is a controller thread which changes the variable incrementally while the rest two threads will compete against each other for its turn to read this variable. The purpose for this setup is so that the two working threads have a unique number after doing a read. For the controller thread, wait() will be called after changing the variable.

My question:
Assuming worker thread 1 wins the competition for resource and completes its read, calls signal() and releases the mutex, will the controller thread automatically awakes and locks the mutex?? or will it have to compete with thread 2 for this mutex after awakening??

Any help is appreciated.
Thanks!

Hey,

when saying that you use a mutex, do you mean the mutex associated with the condition variable or do you use an additional mutex.

According to the POSIX standard, pthread_cond_signal does:

Quote:

---

"The pthread_cond_signal() function shall unblock at least one of the threads that are blocked on the specified condition variable cond (if any threads are blocked on cond)."

---

Furthermore:

Quote:

---

"If more than one thread is blocked on a condition variable, the scheduling policy shall determine the order in which threads are unblocked. When each thread unblocked as a result of a pthread_cond_broadcast() or pthread_cond_signal() returns from its call to pthread_cond_wait() or pthread_cond_timedwait(), the thread shall own the mutex with which it called pthread_cond_wait() or pthread_cond_timedwait(). The thread(s) that are unblocked shall contend for the mutex according to the scheduling policy (if applicable), and as if each had called pthread_mutex_lock()."

---

Though, you would need to define an appropriate scheduling policy, if that's possible for your scenario. Alternatively, if you would like to guarantee that the controling thread is waked up after reading the value from one of your worker threads, you could also use 2 distinct condition variables protected by a single mutex (if I though correctly ;-))

Hope that helps,
- Andi -

Your method has a race condition because the threads have no way of knowing whether the variable's value has been incremented. It basically gets right to the point of what you are saying -- when a worker grabs a value then signals the condition, the other worker may acquire the mutex first and then try to get a variable which hasn't been incremented yet. The result is both workers have the same value, which is bad.

The problem is the controller itself, which should not exist. The threads should increment the value of the variable themselves, then signal the condvar. Each thread, upon waking from the cond_wait(), will find a fresh value in the variable.

Quote:

---

Originally Posted by brewbuck

The problem is the controller itself, which should not exist. The threads should increment the value of the variable themselves, then signal the condvar. Each thread, upon waking from the cond_wait(), will find a fresh value in the variable.

---

This method will not work if the "incrementing variable" is anything but a counter: eg, if the control thread is producing non-contiguous unique values for consumption by the other threads, then a separate controller must exist.

The problem is making the producer/controller wait while the consumer/workers work. To do this you need one global condition and one global mutex; the producer needs the lock at each iteration of it's task, but only the consumers can unlock it. The consumers all wait on the same condition, which is signalled once each time by the producer -- then one consumer consumes and unlocks the lock so the producer can continue:

Code:

---

#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>

pthread_cond_t Cond = PTHREAD_COND_INITIALIZER;
pthread_mutex_t Lock = PTHREAD_MUTEX_INITIALIZER;
int X;

void *consumer(void *id) {

        pthread_mutex_t local = PTHREAD_MUTEX_INITIALIZER;
        pthread_mutex_lock(&local);
        while (1) {
                pthread_cond_wait(&Cond,&local);
                copy = X;    /* get the variable */
                pthread_mutex_unlock(&Lock);
                printf("%p: %d\n",id,copy);                     
        }
}


void *producer() {
        int i;
        for (i=0;i<100;i++) {
                printf("%d ",i);
                pthread_mutex_lock(&Lock);
                X = rand();
                pthread_cond_signal(&Cond);
        }
        return NULL;
}


int main() {
        pthread_t p,a,b;

        setvbuf(stdout,NULL,_IONBF,0);

        pthread_create(&a,NULL,consumer,(void*)0xA);
        pthread_create(&b,NULL,consumer,(void*)0xB);
        /* maybe: sleep(1) */
        pthread_create(&p,NULL,producer,NULL);
        pthread_join(p,NULL);

        return 0;
}

---

You can actually replace the mutex with a boolean flag set by the producer and unset by the consumer; the producer waits with a while(Flag) -- but this is active whereas the mutex is passive.

Thanks very much for the help and information guys!!

http://www.opengroup.org/onlinepubs/...cond_timedwait

Quote:

---

When a thread waits on a condition variable, having specified a particular mutex to either the pthread_cond_timedwait() or the pthread_cond_wait() operation, a dynamic binding is formed between that mutex and condition variable that remains in effect as long as at least one thread is blocked on the condition variable. During this time, the effect of an attempt by any thread to wait on that condition variable using a different mutex is undefined.

---

So a condition-mutex pair should always be used together. A third component is always needed as well - a "predicate condition", which handles any spurious wakeups.

>> You can actually replace the mutex with a boolean flag set by the producer and unset by the consumer
Access to the same memory location by multiple threads, where at least one thread is writing, must be "synchronized" using one of the functions listed in 4.10 of the Posix standard: http://www.opengroup.org/onlinepubs/...html#tag_04_10

gg

Quote:

---

Originally Posted by MK27

This method will not work if the "incrementing variable" is anything but a counter: eg, if the control thread is producing non-contiguous unique values for consumption by the other threads, then a separate controller must exist

---

I would try to avoid something like that, as it implies an extra context switch per effective thread switch, i.e. increases context switch overhead by 100%. If the sole purpose of the values is to serve as a unique cookie, I wouldn't use a controller

Quote:

---

Originally Posted by Codeplug

So a condition-mutex pair should always be used together. A third component is always needed as well - a "predicate condition", which handles any spurious wakeups.

---

Hmmm. So what is the proper way to use the condition here? That paragraph actually contains an oxymoron, making the standard impossible to obey:

Quote:

---

When a thread waits on a condition variable, having specified a particular mutex to either the pthread_cond_timedwait() or the pthread_cond_wait() operation, a dynamic binding is formed between that mutex and condition variable that remains in effect as long as at least one thread is blocked on the condition variable. During this time, the effect of an attempt by any thread to wait on that condition variable using a different mutex is undefined. Once all waiting threads have been unblocked (as by the pthread_cond_broadcast() operation), the next wait operation on that condition variable shall form a new dynamic binding with the mutex specified by that wait operation.

---

Okay, if a thread must first acquire a specific mutex to wait on a condition and a condition must always be associated with the same mutex, there can only be one thread waiting on the condition at a time. No other thread will be able to wait on the condition because it cannot aquire the mutex because "They shall be called with mutex locked by the calling thread or undefined behavior results.". That says to me: you can only have ONE thread wait on a particular condition at a time. One mutex. One condition. One thread. Also nb, "Upon successful return, the mutex shall have been locked and shall be owned by the calling thread." One mutex, one condition, one thread.

This makes the command pthread_cond_broadcast (wake all, not just one) thread completely meaningless, and the rest of the discussion about multiple threads waiting on a condition also meaningless. Why discuss how the standard applies to something that must be the result of undefined behavior? (This is what I meant by oxymoronic). It also means there is no purpose to the mutex -- wait_cond should just include it. If the mutex and the condition must always be the same, why separate them this way?

Anyway, I suppose a more compliant version looks like this:

Code:

---

pthread_cond_t Cond = PTHREAD_COND_INITIALIZER;
pthread_mutex_t Cnlk = PTHREAD_MUTEX_INITIALIZER; 
void *consumer(void *id) {
        int copy;
       
        while (1) {
                pthread_mutex_lock(&Cnlk);
                pthread_cond_wait(&Cond,&Cnlk);
                pthread_mutex_unlock(&Cnlk);
                copy = X;
                pthread_mutex_unlock(&Lock);
                printf("%p: %d\n",id,copy);
        }
}

---

This makes it impossible for more than one thread to wait on the condition at at time, which despite all of it's discussion to the contrary is what the specification asks for (one mutex, one condition = one thread only).

Anyway, any clarification you can add here would be great, vis the purpose of cond_broadcast.

Quote:

---

Originally Posted by Codeplug

>> You can actually replace the mutex with a boolean flag set by the producer and unset by the consumer
Access to the same memory location by multiple threads, where at least one thread is writing, must be "synchronized" using one of the functions listed in 4.10 of the Posix standard: General Concepts

---

Yeah, you've said that before and I think in the meantime I seem to have decided that it only applies if more than one thread is writing, mostly because I've been misled by this tutorial:

Linux Tutorial: POSIX Threads

Which does exactly that. Altho the author at least admits: "Note: Race conditions abound with this example because count is used as the condition and can't be locked in the while statement without causing deadlock. I'll work on a cleaner example but it is an example of a condition variable. " :p

In fact, I did test this* and at least on my system it is obviously impossible to misread a variable during writing -- reads/writes occur consecutively and never concurrently. But of course the fact that undefined behavior may work does not make it any less undefined....

* one thread alternates writing either 4 or 33033 to a global int; the other threads just keep checking the number. If it is ever NOT 4 or 33033, there's an error. The error never occurs.

Quote:

---

Originally Posted by MK27

Okay, if a thread must first acquire a specific mutex to wait on a condition and a condition must always be associated with the same mutex, there can only be one thread waiting on the condition at a time. No other thread will be able to wait on the condition because it cannot aquire the mutex because "They shall be called with mutex locked by the calling thread or undefined behavior results.". That says to me: you can only have ONE thread wait on a particular condition at a time. One mutex. One condition. One thread. Also nb, "Upon successful return, the mutex shall have been locked and shall be owned by the calling thread." One mutex, one condition, one thread.

---

No, that isn't how it works. pthread_cond_wait() unlocks the mutex for you when it begins waiting, and then re-acquires it when it returns. The mutex must be locked before you call pthread_cond_wait(), but the mutex does NOT remain locked DURING the wait. If what you said was true it would be ridiculous.

>> Anyway, I suppose a more compliant version looks like this:
You need a predicate condition so you can tell the difference between a real signal and a spurious wakeup.

You can use a second condition for the producer to wait on until the last thing produced has been consumed:

Code:

---

#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>

pthread_cond_t DataReady = PTHREAD_COND_INITIALIZER;
int bDataReady = 0;

pthread_cond_t DataConsumed = PTHREAD_COND_INITIALIZER;
int bDataConsumed = 0;

pthread_mutex_t Lock = PTHREAD_MUTEX_INITIALIZER;
int Data;

void* consumer(void *p)
{
    int myid = (int)p;
    while (1)
    {
        pthread_mutex_lock(&Lock);

        while (!bDataReady)
            pthread_cond_wait(&DataReady, &Lock);

        printf("%d: consumed %d\n", myid, Data);
        fflush(stdout);

        bDataConsumed = 1;
        pthread_cond_signal(&DataConsumed);
       
        bDataReady = 0;
        pthread_mutex_unlock(&Lock);
    }

    return NULL;
}

void* producer(void *p)
{
    int i;

    pthread_mutex_lock(&Lock);

    for (i = 0; i < 20; ++i)
    {
        Data = i;
       
        bDataReady = 1;
        pthread_cond_signal(&DataReady);

        bDataConsumed = 0;
        while (!bDataConsumed)
            pthread_cond_wait(&DataConsumed, &Lock);
    }

    bDataReady = 0;
    pthread_mutex_unlock(&Lock);
    return NULL;
}

int main()
{
    pthread_t p,a,b;

    pthread_create(&a, NULL, consumer, (void*)1);
    pthread_create(&b, NULL, consumer, (void*)2);
    pthread_create(&p, NULL, producer, NULL);
    pthread_join(p, NULL);

    return 0;
}

---

You can remove the need for the producer to have to wait for consumption by producing into some sort of container (like a linked list etc). Then the consumer predicate would be "while container empty, wait".

gg

Grrr...slightly more (maybe) confused now :p

What strikes me about that is this:

Code:

---

    pthread_mutex_lock(&Lock);

    for (i = 0; i < 20; ++i)
    {
        Data = i;
       
        bDataReady = 1;
        pthread_cond_signal(&DataReady);

---

Must mean that by signalling DataReady, the lock on the associated mutex is given to to the waking thread???? Otherwise, the signal is pointless -- Lock is held by the producer to start with, and never unlocked by it.

This is not clear from the man page/spec ("if predictable scheduling behavior is required, then that mutex shall be locked by the thread calling pthread_cond_signal()") which IMO implies that it's going to keep the lock altho obviously that is not the case.

WRT "a predicate condition so you can tell the difference between a real signal and a spurious wakeup":

1) wazza "spurious wakeup"?
2) how does this help? It implies the "spurious wakeup" cannot be concurrent to the actual signal call.

>> Must mean that by signalling DataReady, the lock on the associated mutex is given to to the waking thread?
Calling signal or broadcast doesn't affect the lock associated with the condition variable. Lock ownership is "allowed to be passed" when the producer releases the lock - via the "pthread_cond_wait(&DataConsumed, &Lock)" call.

Keep in mind that calling pthread_cond_wait() is an implicit pthread_mutex_unlock() on the second parameter.

>> if predictable scheduling behavior is required, then that mutex shall be locked by the thread calling pthread_cond_signal()
By doing this, any thread that wants to wait or signal the condition must first lock the mutex. It's the locking of a mutex that has "predictable scheduling behavior" (depending on the scheduling policy).
http://www.opengroup.org/onlinepubs/...l#tag_02_08_04

Spurious wakeups are discussed in the rationale section here:
http://www.opengroup.org/onlinepubs/...#tag_03_515_08

So basically, calling signal may wake up more than 1 thread - and that's allowed for efficiency sake.

gg

I am new to it !
But #10th reply makes sense . Yes, pthread_cond_wait() when returns, it will ensure that the mutex which was associated with this cond variable, should be in locked state, even in case of error it ensures the same.

Though I have a small question in #10 , why do we need a signal to be sent by consumer thread. the requirement can get satisfied if only producer sends a signal, both consumers can simply wait at pthread_cond_wait(&condvar,&mutex)
Isnt that

>> ... why do we need a signal to be sent by consumer thread.
Because there is only a single variable to hold the data to be consumed. What if the producer thread produces two times in a row without any consumption? The first value produced is lost once it's overwritten by the second value produced. So you need two conditions to prevent this type of loss: "data ready for consumption", and "data consumed, produce another".

An alternative to this is to use a Queue data-structure to hold the data being produced. Then you could get away with using a single condition: "Queue not empty".

gg

I was thinking of satisfying the requirement only , But Yes what u mentioned is the efficient and better way i.e to send a signal by a consumer.

Thanks for clarifying it.