Thread: my questions on Boost

Originally Posted by manasij7479

That isn't very true..
The parts that aren't supported yet are not very comonly used.
(The concurrency section in the Status site is misleading as it speaks of the core extensions needed, not the libraries.... AFAIK)
Your original program, for example can easily be implemented with std::threads.

thank you verymuch then im going to test it and see if it is any better

Originally Posted by phantomotap

O_o

For your example you have several options but at a primitive level the situation probably calls for a fairly standard "work queue" with a "reader/writer strategy" acting as "results" storage. Those fiddly bits that you are trying to find support for are never going to be native things because those fiddly bits are almost always need to be specially crafted.

Look at your "how do I know if a job completed successfully" example. A lot of threading libraries don't have an answer for that because the notion of result is unique to the problem. However, it is pretty trivial to add locking around a shared container such as a map and store results of any kind within that map.

As for not having any examples... you are clearly looking in the wrong place. Don't look for "Boost thread examples", "wxWidget thread examples", or anything like that. The "API" is just window dressing. Great programmers have already distilled out the core details from thousands of strategies, labelled them with a shared jargon, and thoroughly illustrated their use and implementation. What you should be looking for is "concurrency patterns" or similar. You may not find real code using a specific "API", but you'll certainly get a better idea of how to use threads and serialization primitives.

Soma

thank you Soma , i really appreciate that .
so i'll be reading C++ Concurrency in Action Practical Multithreading suggested by one the boost community members .
hope that book get me going

Sample code demonstrating a shared producer/consumer Q, and exit notification: Boost thread request queue design question

gg

i really appreciate that

Welcome. ^_^

Can you post a link to such a resource ?

Well...

or very complicated

probably not.

Simple patterns allow for simple expressions of those patterns. I could provide a link to patterns such as those, but those are the ones that are easy to find. (Looking for "readers writer locking" will give you tons of examples in dozens of languages using dozens of different "API".)

The more complex patterns, such as lock-free queues, have complex expressions. You can't change that. (One popular implementation of an unfortunately invalid lock-free linked list was thought to be correct even after three years of peer review.) Heck, some of the patterns I've used have proofs that I literally can't follow let alone verify.

That said, Intel, Microsoft, and IBM all have or have had a large number of "blog" (I hate that ........ing word!) entries about this stuff.

Those blogs aren't necessarily dedicated to concurrency patterns specifically but do often discuss real solutions to those problems.

[Edit]
Oh, and I actually get the majority of the articles I read through ACM so I can't provide them.
[/Edit]

[Edit]
I can't remember the name of the "blog" but a regular at the forum often posts links to a blog largely focusing on parallelism and concurrency where a lot of the examples are given in C++ using the "pthread" library.

Maybe someone will find that link and post it here.
[/Edit]

Soma

Parallel Programming – Blogs - Intel® Software Network

A lot of your issues seem very theoretical -- you have developed a bunch of concerns about threading without having done any threaded programming. :/

For the same reason, the library you use really does not matter at this point, because you just don't have much perspective on its value. IMO, you should just start with whatever the native interface is for your preferred OS. That will be the simplest choice, and the closest to the metal, because anything else has to go thru there. Once you understand better, you can start shopping around. Until then, you are like a ten year old watching car ads on TV (no offense).

What you should focus on is implementing standard concurrency/threading exercises -- the dining philosophers, the producer-consumer model, etc. Threading has a lot of limitations related to what manasij7479 called "the C++ way of thinking" in post #11. You generally can't have your cake and eat it too. Rather than looking around for an API that promises you everything (with a bunch of caveats), try to think/learn about how threading actually works at a fundamental level and how you can work with that to accomplish what you want elegantly and effectively.

Maybe you have heard the zen thing to do with an analogy of (the experiential) mind to water, whereby your "knowledge" is fixation on a rock in a fast flowing stream. I don't know if that helps, lol, or if you've ever been in a fast flowing stream with rocks, but if you fixate on them, you have fixated on exactly the one thing that will get you killed and ignored what is all around you, which is the only possible path to safety .

Anyway, let's consider (and try to answer my questions to about your questions, that might help clarify):

i needed a mechanism to make sure how a specific message is dealt with , was it successful or not ?

In what form is the message? Let's say it is an object in a queue. In what sense could its "having been dealt" be successful, or not? These are aspects of the object, not the thread or the threading implementation. These concerns apply equally well to a single threaded application (and there's your answer, more or less).

how could i know about the status of threads being created in the process ?

When you write a for(i) loop, how can you know about the status of i?

how could i impose some priority on threads , so that for example a writer thread has a more priority towards the reader threads

Why is this necessary? You seem to have fixated on a potential design issue without any actual design. Again, try a simple producer-consumer thing, this problem is totally illusory IMO.

how could i terminate a thread if the need arises ?

The best way for a thread to terminate is for the thread to terminate itself. Much like a function or a process. Threads can be implemented in an object oriented way, but (like OO itself), they are still very procedural.

how could i manage the concurrency of the application if i didnt know how many threads are working or blocked or what ever

At what point do you think you will end up not knowing how many threads are working, and why?

Basically, you need to take a step back and do simple things without thinking too much about more complex things until you get there. It is not as if you are at a point in your life where the devil has appeared and said, "Aha! Multi-threading! NOW CHOOSE:"

a) std::thread
b) pthreads
c) Boost::Threads
d) TBB
e) Elysia's pick of the week

And if you choose wrong right now, you will be damned to hell FOREVER!!!

great , thanks everyone , specially you mk27, that post of yours really helped me get my thoughts straight and organized again
(by the way , i now know that boost has a future feature which can be used to find about the success or failure of thread ; actually i need to create packages , then pass these packages to the threads i want, and then their results can be inferred and thus one can know what happened to a thread job , there is also a barrier class which seems really promising to me .
for priority in threads ,one can use conditional variable and thats again a very fine way of doing stuff . --)
so i'll do as told here and will be back if i had any more questions
again thank you every one , i appreciate every single second you spent in this topic helping me , i really do

Originally Posted by Masterx

]
(by the way , i now know that boost has a future feature which can be used to find about the success or failure of thread ; actually i need to create packages , then pass these packages to the threads i want, and then their results can be inferred and thus one can know what happened to a thread job , there is also a barrier class which seems really promising to me .

Combined with another feature called 'promise', it is used to return values from the thread when ready... not just a boolean pass/fail.

for priority in threads ,one can use conditional variable and thats again a very fine way of doing stuff . --)

I'm curious how you'll manage that ..
Can you post a small example?

Originally Posted by manasij7479

Combined with another feature called 'promise', it is used to return values from the thread when ready... not just a boolean pass/fail.


I'm curious how you'll manage that ..
Can you post a small example?

i saw an example demonstrating such behavior : here it is :

Code:

#include <boost/thread.hpp>
#include <iostream>
#include <vector>
#include <cstdlib>
#include <ctime>

boost::mutex mutex;
boost::condition_variable_any cond;
std::vector<int> random_numbers;

void fill()
{
  std::srand(static_cast<unsigned int>(std::time(0)));
  for (int i = 0; i < 3; ++i)
  {
    boost::unique_lock<boost::mutex> lock(mutex);
    random_numbers.push_back(std::rand());
    cond.notify_all();
    cond.wait(mutex);
  }
}

void print()
{
  std::size_t next_size = 1;
  for (int i = 0; i < 3; ++i)
  {
    boost::unique_lock<boost::mutex> lock(mutex);
    while (random_numbers.size() != next_size)
      cond.wait(mutex);
    std::cout << random_numbers.back() << std::endl;
    ++next_size;
    cond.notify_all();
  }
}

int main()
{
  boost::thread t1(fill);
  boost::thread t2(print);
  t1.join();
  t2.join();
}

i modified the previous example , but i dont know why it gets locked!

Code:

#define BOOST_THREAD_USE_LIB
#include <iostream>
#include <boost/thread.hpp>
#include <vector>
using namespace std;

vector<int> someSharedResource;
boost::mutex mutex;
boost::condition_variable_any condition;
void Writer()
{


    for (int i = 0; i < 10 ; i++)
        {
           boost::unique_lock<boost::mutex> locker(mutex);
           someSharedResource.push_back(i);
          // cout<<boost::this_thread::get_id()<<"\t"<<endl;
           condition.notify_all();
           condition.wait(mutex);
        }

}
void Reader()
{
    int counted_items=0;
   for( int i =0;i<10;i++)
        {
               boost::unique_lock<boost::mutex> locker(mutex);
               condition.wait(mutex);
                cout<<someSharedResource.back()<<"\t"<<boost::this_thread::get_id()<<endl;
                counted_items++;
                condition.notify_all();
        }


}
int main()
{
  boost::thread t1(Writer);
  boost::thread t2(Reader);
  t1.join();
  t2.join();
}

is it not that , the mutex is not released until the writer function , gets passed the wait(mutex) method ? and after that waits for the mutex ?
and inside reader function , when it is awakened by the notify_all() method by writer function , it waits until writer releases the mutex and then it owns it and there it executes .
so why do i fell into a locked down of some kind .
if i run this example couple of times , some times it works fine and some times it just hangs at the beginning ( getting locked ? ) whats wrong with it . ?

The Writer could reach the first notify_all() call before the Reader reaches the first wait(). The problem is that in this case, the notification is lost.
Also, doesn't condition_variable has spurious wakeups? Basically, you can't use a condition_variable as the simple back-and-forth semaphore as you're trying to do. There's a reason why the original version has the loop.

In fact, on my system your version never runs, which shows you what difference a scheduler will make. It will seem to run "correctly" if I reverse lines 40 and 41 -- but you should never depend on the threads starting in a particular order even if statistically, it seems like they always will, which they might with something this simple.

Originally Posted by Masterx

is it not that , the mutex is not released until the writer function , gets passed the wait(mutex) method ? and after that waits for the mutex ?

Yes, but let's examine what that means. Presuming Writer() runs first (which it could, and, statistically on my system, does) and executes this section:

Code:

           boost::unique_lock<boost::mutex> locker(mutex);
           someSharedResource.push_back(i);
          // cout<<boost::this_thread::get_id()<<"\t"<<endl;
           condition.notify_all();  
/** NB: that notify_all() did not notify anyone, because no one was waiting **/
           condition.wait(mutex);

So, the lock is now open and Writer() is waiting on the condition. Remember, at this point Reader() has not run, or else it is still waiting for the lock at this point:

Code:

               boost::unique_lock<boost::mutex> locker(mutex);

But now Writer() has released the lock by calling wait(), Reader() can aquire the lock and the first thing it does:

Code:

               condition.wait(mutex);

Now both threads are waiting on the condition. Forever....

The difference between your code and the example is:

Code:

	while (random_numbers.size() != next_size)
		cond.wait(mutex);

Notice that this is inside a locked section, because access to random_numbers (a shared resource) must be restricted. Also notice that's a while and not an if, altho in the example, "if" would work just as well, until more complications are added. IMO the most robust condition would actually be while (size < 1), not != 1. That would be important if the example were less contrived.

Anyway, we are in a locked section, so it is safe for Reader to do whatever at this point. However, if there is nothing to do, that is because Writer() (aka, fill()) is waiting here:

Code:

    boost::unique_lock<boost::mutex> lock(mutex);

Not here:

Code:

    cond.wait(mutex);

Therefore, if Reader() (aka. print) calls wait, Writer will lock, add a number, and notify_all (waking Reader).

However, if there is something to do (based on the check of random_numbers), then the situation is the other way around: Writer() has just added a number and is at cond_wait. In that case, Reader() should not call wait(), it should perform the queued task and call condition_notify to wake Writer(), taking Reader back to the beginning of its loop, where it waits for the lock here again:

Code:

    boost::unique_lock<boost::mutex> lock(mutex);

That's how water flows Now a nit-pick note:

for priority in threads ,one can use conditional variable and thats again a very fine way of doing stuff

Maybe it's just me, but I would not call this "prioritizing", I'd call it "synchronizing". Prioritizing would have more to do with threads performing different tasks that do not depend on one another (therefore, do not need to be synchronized) but where one task is considered more important. Eg, if you had a real time application with a thread that cleans something up when enough processor time is available, that thread would be lower priority -- if there are not sufficient resources to do both things, the real time critical thread(s) should get priority.

>> i saw an example demonstrating such behavior : here it is :
The code in post #24 is broken as well, since fill() waits on a condition variable without a predicate loop. Unfortunately, lots of broken code on the net to choose from.

gg

Originally Posted by Codeplug

The code in post #24 is broken as well, since fill() waits on a condition variable without a predicate loop.

Not really -- the predicate is implicit; if "broken" means something that doesn't work properly, or something prone to malfunction, the logic of that example as is is not broken.

If print() does these two things in sequence:

Code:

   cond.notify_all(); // <--- end of for()
   boost::unique_lock<boost::mutex> lock(mutex); // <-- beginning of for

It's acquired the lock before fill() could do anything. Based on the API, that can't actually happen because notify_all() undoes the wait(mutex) in fill meaning the fill thread must now have mutex. But even if it did, the next bit:

Code:

    while (random_numbers.size() != next_size)
      cond.wait(mutex);

Means that since fill() didn't do anything, mutex is released and fill() will acquire it.

Ie, the predicate condition of condition.wait(mutex) in fill is the inverse, while (random_numbers.size() == next_size). That could be made explicit, but you'd have to make next_size global.

With that in mind, the lock(mutex) declaration in fill() actually belongs before the for loop, not inside it. I also notice main() doesn't have a return value, so:

Code:

    #include <boost/thread.hpp>
    #include <iostream>
    #include <vector>
    #include <cstdlib>
    #include <ctime>
     
    boost::mutex mutex;
    boost::condition_variable_any cond;
    std::vector<int> random_numbers;
    std::size_t next_size = 1;
     
    void fill()
    {
      std::srand(static_cast<unsigned int>(std::time(0)));
      boost::unique_lock<boost::mutex> lock(mutex);
      for (int i = 0; i < 3; ++i)
      {
        random_numbers.push_back(std::rand());
        cond.notify_all();
        while (random_numbers.size() == next_size)
          cond.wait(mutex);
      }
    }
     
    void print()
    {
      for (int i = 0; i < 3; ++i)
      {
        boost::unique_lock<boost::mutex> lock(mutex);
        while (random_numbers.size() != next_size)
          cond.wait(mutex);
        std::cout << random_numbers.back() << std::endl;
        ++next_size;
        cond.notify_all();
      }
    }
     
    int main()
    {
      boost::thread t1(fill);
      boost::thread t2(print);
      t1.join();
      t2.join();
      return 0;
    }

The wait in fill() now has an explicit predicate, but it is redundant.

thank you guys . i read this page and after trying it out , if i face any further problem , i'll post back
(MK27 thank you again )