Thread: ThreadPool question

  1. #1
    Registered User
    Join Date
    May 2003
    Posts
    1,619

    ThreadPool question

    OK here's a question:

    I have a program that batch manipulates a few thousand images (cropping and resizing). I use ThreadPool and have the worker threads do all the image processing, and that works fine.

    My main question is how to signal that the work is complete -- that is, some way I know that every thread in the queue has finished. Ideally I'd like a particular thread to wait until every worker thread was finished, and then resume execution. Effectively my program has a GUI thread, a thread that recurses over folders seeking images, and then worker threads which process the images that are found. I'd like that second thread (the recursing thread) to suspend once it's fully recursed, and not continue execution until all worker threads were done.

    Now I know I could use a ManualResetEvent[], and then WaitForAll on it -- this is how a lot of example code works. This isn't all that great for me because:

    1. That array could be tens of thousands of entries (maybe a problem, maybe not, I dunno),

    2. My worker threads are created before I know a final count of how many worker threads there will be. As such, I wouldn't know how large of an array to make.
    You ever try a pink golf ball, Wally? Why, the wind shear on a pink ball alone can take the head clean off a 90 pound midget at 300 yards.

  2. #2
    Registered User
    Join Date
    May 2003
    Posts
    1,619
    OK I think I came up with a solution, I guess you can evaluate how it works:


    * I create a private static ManualResetEvent, and a static accessor to retrieve it.

    * I create a private static counter of threads.

    * In my PoolQueue() function, which adds a thread to the pool queue, just prior to queueing the thread, it does:
    Code:
                Interlocked.Increment(ref workers);
                mre.Reset();
    * My entire worker thread function is in a try... finally, with the finally being:

    Code:
                finally
                {
                    Interlocked.Decrement(ref workers);
                    if (workers <= 0) mre.Set();
                }
    So then, after my directory traversal thread finishes adding worker threads, it can just wait for this event.
    You ever try a pink golf ball, Wally? Why, the wind shear on a pink ball alone can take the head clean off a 90 pound midget at 300 yards.

  3. #3
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Seems like a problem that you'd have to solve a different way.

    I would probably implement another thread:
    - GUI Thread
    - Recursive Find-Files Thread
    - Processing Manager thread
    - Worker Thread(s)

    The find-files thread builds a linked list, and the Process Manager thread takes items in the linked list, assigns them to a worker thread, and when there are no available worker threads, waits for the first event to finish, then starts another worker thread off with the next image. That way you only need one event per worker thread, easy to count and no problems with the limits [if any] of the number of events needed.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Alice....
    By Lurker in forum A Brief History of Cprogramming.com
    Replies: 16
    Last Post: 06-20-2005, 02:51 PM
  2. Debugging question
    By o_0 in forum C Programming
    Replies: 9
    Last Post: 10-10-2004, 05:51 PM
  3. Question about pointers #2
    By maxhavoc in forum C++ Programming
    Replies: 28
    Last Post: 06-21-2004, 12:52 PM
  4. Question...
    By TechWins in forum A Brief History of Cprogramming.com
    Replies: 16
    Last Post: 07-28-2003, 09:47 PM
  5. Question, question!
    By oskilian in forum A Brief History of Cprogramming.com
    Replies: 5
    Last Post: 12-24-2001, 01:47 AM