Thread: producer / consumer question

  1. #1
    Registered User
    Join Date
    Nov 2006
    Posts
    519

    producer / consumer question

    Hi,

    my consumer waits for a producer putting things in a vector as in the pseudo code below:

    Code:
    while(true)
    {
       lock mutex;
       wait_for_condition;
       // ok, the condition was signaled by the producer
       unlock mutex;
        // the producer can now put new things in
        // and they are going to be proper handled because of the while-loop
       while(vector != empty)
       {
          lock mutex;
          pop element;
          unlock mutex;
          consume;
       }
       // now this place is interesting:
       // the while loop is ended so all things are consumed
       // but exactly at this point a new thing is produced and put in the vector
       // and the condition is signaled
       // but the consumer does not wait yet
       // so I guess the consumer is starving until a second thing is produced
    }
    I guess I got something wrong regarding the concept of producer/consumer. how can I avoid the problem mentioned in the comment above?

    Thank you in advance!

  2. #2
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    Quote Originally Posted by pheres View Post
    I guess I got something wrong regarding the concept of producer/consumer. how can I avoid the problem mentioned in the comment above?
    Your use of the condition is not correct. More like:

    Code:
    lock mutex;
    while(true)
    {
       wait for condition;
       while(vector != empty)
       {
          pop element;
          unlock mutex;
          consume;
          lock mutex;
       }
    }
    unlock mutex;
    You unlock/lock in the inner loop is not strictly necessary, but it prevents the producer from blocking while the consumer is processing. You could eliminate both problems by replacing that inner loop with:

    Code:
    if(vector != empty)
    {
        newvec = vector;
        vector.clear();
        unlock mutex;
        consume_all(newvec);
        lock mutex;
    }
    In other words, deque ALL the requests at once, and process them serially, without repeatedly locking/unlocking the original vector as you do so.

  3. #3
    Registered User
    Join Date
    Nov 2006
    Posts
    519
    Quote Originally Posted by brewbuck View Post


    In other words, deque ALL the requests at once, and process them serially, without repeatedly locking/unlocking the original vector as you do so.

    So I end up with:


    Code:
    lock mutex;
    while(true)
    {
       wait for condition;
       if(vector != empty)
       {
           newvec = vector;
           vector.clear();
           unlock mutex;
           consume_all(newvec);
           lock mutex;
       }
    }
    unlock mutex;
    but what if a thing is produced between consume_all and lock? wouldn't it be better to remove the unlock/lock from the inner if?

  4. #4
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by pheres View Post
    So I end up with:


    Code:
    lock mutex;
    while(true)
    {
       wait for condition;
       if(vector != empty)
       {
           newvec = vector;
           vector.clear();
           unlock mutex;
           consume_all(newvec);
           lock mutex;
       }
    }
    unlock mutex;
    but what if a thing is produced between consume_all and lock? wouldn't it be better to remove the unlock/lock from the inner if?
    Since the data being processed is the "newvec", the producer is free do fill in vector, and as long as the "condition" you wait for isn't transient (in which case you have a race-condition in the condition itself, and you should fix that), you should be fine.

    --
    Mats

  5. #5
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    Quote Originally Posted by pheres View Post
    but what if a thing is produced between consume_all and lock?
    Because the lock is unlocked, the producer would simply stick that thing on the vector, and you'd see it the next time around the loop.

  6. #6
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    Quote Originally Posted by matsp View Post
    Since the data being processed is the "newvec", the producer is free do fill in vector, and as long as the "condition" you wait for isn't transient (in which case you have a race-condition in the condition itself, and you should fix that), you should be fine.
    It's impossible for the condition to be transient (in the sense you mean) because the "wait for condition" statement silently implies an atomic unlock/lock pair surrounding the condition wait. That's why you don't just wait for the condition -- you also check it after you've waited. But if you find that the condition is true, it REMAINS true, because the mutex is locked, and so the state can't change.

  7. #7
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    Quote Originally Posted by pheres View Post
    but what if a thing is produced between consume_all and lock? wouldn't it be better to remove the unlock/lock from the inner if?
    Yikes. Actually, there IS a problem here. Removing the unlock/lock solves it, but not optimally.

    If a new element is added to the vector while consume_all is processing, the producer will signal the condition at that point. But a condition change can be caught only while waiting for it -- since the consumer is in the middle of consuming, it won't catch the condition.

    This means it won't dequeue the next event (or events, if the producer is producing fast enough) as quickly as it should.

    Simple fix. Turn the inner "if" back into a "while." I'm dumb.

  8. #8
    Registered User
    Join Date
    Nov 2006
    Posts
    519
    you solved it. thank you!

  9. #9
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    Quote Originally Posted by pheres View Post
    you solved it. thank you!
    I'm embarrassed though I've written essentially exactly this code, many times. Guess I just had a brain fart.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. pthread producer consumer problem
    By Amit Chikorde in forum C Programming
    Replies: 1
    Last Post: 01-17-2007, 07:39 PM
  2. Alice....
    By Lurker in forum A Brief History of Cprogramming.com
    Replies: 16
    Last Post: 06-20-2005, 02:51 PM
  3. Debugging question
    By o_0 in forum C Programming
    Replies: 9
    Last Post: 10-10-2004, 05:51 PM
  4. Question...
    By TechWins in forum A Brief History of Cprogramming.com
    Replies: 16
    Last Post: 07-28-2003, 09:47 PM
  5. Question, question!
    By oskilian in forum A Brief History of Cprogramming.com
    Replies: 5
    Last Post: 12-24-2001, 01:47 AM