producer / consumer question

Hi,

my consumer waits for a producer putting things in a vector as in the pseudo code below:

Code:

---

while(true)
{
  lock mutex;
  wait_for_condition;
  // ok, the condition was signaled by the producer
  unlock mutex;
    // the producer can now put new things in
    // and they are going to be proper handled because of the while-loop
  while(vector != empty)
  {
      lock mutex;
      pop element;
      unlock mutex;
      consume;
  }
  // now this place is interesting:
  // the while loop is ended so all things are consumed
  // but exactly at this point a new thing is produced and put in the vector
  // and the condition is signaled
  // but the consumer does not wait yet
  // so I guess the consumer is starving until a second thing is produced
}

---

I guess I got something wrong regarding the concept of producer/consumer. how can I avoid the problem mentioned in the comment above?

Thank you in advance!

Quote:

---

Originally Posted by pheres

I guess I got something wrong regarding the concept of producer/consumer. how can I avoid the problem mentioned in the comment above?

---

Your use of the condition is not correct. More like:

Code:

---

lock mutex;
while(true)
{
  wait for condition;
  while(vector != empty)
  {
      pop element;
      unlock mutex;
      consume;
      lock mutex;
  }
}
unlock mutex;

---

You unlock/lock in the inner loop is not strictly necessary, but it prevents the producer from blocking while the consumer is processing. You could eliminate both problems by replacing that inner loop with:

Code:

---

if(vector != empty)
{
    newvec = vector;
    vector.clear();
    unlock mutex;
    consume_all(newvec);
    lock mutex;
}

---

In other words, deque ALL the requests at once, and process them serially, without repeatedly locking/unlocking the original vector as you do so.

Quote:

---

Originally Posted by brewbuck

In other words, deque ALL the requests at once, and process them serially, without repeatedly locking/unlocking the original vector as you do so.

---

So I end up with:

Code:

---

lock mutex;
while(true)
{
  wait for condition;
  if(vector != empty)
  {
      newvec = vector;
      vector.clear();
      unlock mutex;
      consume_all(newvec);
      lock mutex;
  }
}
unlock mutex;

---

but what if a thing is produced between consume_all and lock? wouldn't it be better to remove the unlock/lock from the inner if?

Quote:

---

Originally Posted by pheres

So I end up with:

Code:

---

lock mutex;
while(true)
{
  wait for condition;
  if(vector != empty)
  {
      newvec = vector;
      vector.clear();
      unlock mutex;
      consume_all(newvec);
      lock mutex;
  }
}
unlock mutex;

---

but what if a thing is produced between consume_all and lock? wouldn't it be better to remove the unlock/lock from the inner if?

---

Since the data being processed is the "newvec", the producer is free do fill in vector, and as long as the "condition" you wait for isn't transient (in which case you have a race-condition in the condition itself, and you should fix that), you should be fine.

--
Mats

Quote:

---

Originally Posted by pheres

but what if a thing is produced between consume_all and lock?

---

Because the lock is unlocked, the producer would simply stick that thing on the vector, and you'd see it the next time around the loop.

Quote:

---

Originally Posted by matsp

Since the data being processed is the "newvec", the producer is free do fill in vector, and as long as the "condition" you wait for isn't transient (in which case you have a race-condition in the condition itself, and you should fix that), you should be fine.

---

It's impossible for the condition to be transient (in the sense you mean) because the "wait for condition" statement silently implies an atomic unlock/lock pair surrounding the condition wait. That's why you don't just wait for the condition -- you also check it after you've waited. But if you find that the condition is true, it REMAINS true, because the mutex is locked, and so the state can't change.

Quote:

---

Originally Posted by pheres

but what if a thing is produced between consume_all and lock? wouldn't it be better to remove the unlock/lock from the inner if?

---

Yikes. Actually, there IS a problem here. Removing the unlock/lock solves it, but not optimally.

If a new element is added to the vector while consume_all is processing, the producer will signal the condition at that point. But a condition change can be caught only while waiting for it -- since the consumer is in the middle of consuming, it won't catch the condition.

This means it won't dequeue the next event (or events, if the producer is producing fast enough) as quickly as it should.

Simple fix. Turn the inner "if" back into a "while." I'm dumb.

you solved it. thank you!

Quote:

---

Originally Posted by pheres

you solved it. thank you!

---

I'm embarrassed though :) I've written essentially exactly this code, many times. Guess I just had a brain fart.