Thread: First venture into multithreading

Originally Posted by Cactus_Hugger

605 threads? In one process? Christ. That's 3 times the thread count for my entire system, and six times the leading process on my system. Why do you need some many threads?

Believe it or not, it was over 1000 threads about 6 months ago. Some pieces of code got rewritten and we were able to architect it in such a way that it required fewer threads.

The system is basically a giant data recorder. It can probably be rewritten in such a way that it would only require about 100-200 threads, but the nature of projects that evolve over several years is that there is always a lot of things that you would do differently.

Originally Posted by Salem

I wonder how much time is actually spent in "switch thread context and work out the highest priority thread to run next" when you have hundreds of them.

It depends on the threading implementation. Using LinuxThreads, this was pretty bad. Using NPTL, the overhead is negligible.

Originally Posted by bithub

In simple projects I would agree with you 100%. Try working on an application that has close to 1 million lines of code, and see how often you run into problems that require you do things like:
- share locks between objects
- Have large individual lock coverages
- Generic callbacks while obtaining a lock which can cause deadlocks across modules.

A current line count on my project is 912015 lines of code, and a cat /proc/pid/status | grep Threads shows 605 threads atm. The last 3 years of my life has been spent debugging multi-threaded problems, and I don't feel like I've overstated the problem one bit

I have personally worked on very large applications. In code this big there tends to be other issues than just the treading model. In my experience threading issues have less to do with the size of the code, as with the complexity of the threading model. In the case of the application you worked on, it sounds like it may fall just a weeeee bit outside of the KISS rule I would hesitate to condemn threading based on this example. Anything can be abused.

I never condemned threading. My only assertion was, "Even well designed threaded programs that are written by experts are a pain to debug". It is my belief that you should only create a thread if it is the only way to elegantly solve the problem at hand.

Threading is hard. I find that even people who consider themselves to be experts do not fully understand the implications of concurrency. Take the following contrived example:

Code:

int value;
int value_saved = 0;

void save_value(int new_value)
{
    value = new_value;
    int value_saved = 1;
}

void print_value()
{
    if(value_saved)
        printf("The value is %d\n", value);
    else
        printf("No value has been saved\n");
}

Let's say 1 thread can call save_value() at the same time another thread can call print_value(). If you ask people to modify the code to make it thread safe, and yet still run as fast as possible, most people (including experts) will get it wrong.

Originally Posted by bithub

Let's say 1 thread can call save_value() at the same time another thread can call print_value(). If you ask people to modify the code to make it thread safe, and yet still run as fast as possible, most people (including experts) will get it wrong.

This is a well known "race case". I would guess most programmers who have had some basic training in thread programming will do this correctly. If you use a mutex of some sort it will work. However if you don't, the fail in this case probably isn't that bad.

Even with single statements that read and write shared data you need to do something to make things work 100% correctly because you don't know what low level instructions the compiler is generating. Your issue seems to be "still run as fast as possible". If you if you want to take risks and sort cuts you may run in to problems and your program becomes machine or complier dependant.

However to get it to actually work reliably is easy if you take the accepted strategy.

This is a well known "race case". I would guess most programmers who have had some basic training in thread programming will do this correctly.

There is no race condition in the above code, and no mutex is needed to fix any synchronization issues (but a mutex can be used).

However this has noting to do with the discussion at hand which is probably why I wasn't looking for errors like this

The example was just to illustrate that threading is hard, and should be avoided if possible. I believe that relates to the discussion at hand.

This kind of error can happen in non threaded code.

What error?

Also there is a race case even after you fix it. It's just not a serious one.

No, there's not (unless you are extrapolating beyond the given example).

Anyways, for anyone that's interested, the fixed code looks like:

Code:

volatile int value;
volatile int value_saved = 0;

void save_value(int new_value)
{
    value = new_value;
    barrier(); // memory barrier to prevent reordering of the store operations
    int value_saved = 1;
}

void print_value()
{
    if(value_saved)
        printf("The value is %d\n", value);
    else
        printf("No value has been saved\n");
}

It can also be fixed by putting a mutex around the value assignment since on most mutex implementations, a memory barrier is performed. This would add the unneeded overhead of mutual exclusion though.

Originally Posted by bithub

The example was just to illustrate that threading is hard, and should be avoided if possible. I believe that relates to the discussion at hand.

What error?

No, there's not (unless you are extrapolating beyond the given example).

Anyways, for anyone that's interested, the fixed code looks like:

Code:

volatile int value;
volatile int value_saved = 0;

void save_value(int new_value)
{
    value = new_value;
    barrier(); // memory barrier to prevent reordering of the store operations
    int value_saved = 1;
}

void print_value()
{
    if(value_saved)
        printf("The value is %d\n", value);
    else
        printf("No value has been saved\n");
}

It can also be fixed by putting a mutex around the value assignment since on most mutex implementations, a memory barrier is performed. This would add the unneeded overhead of mutual exclusion though.

Unless volatile has more magic properties than I thought, I don't see how that makes the newly-declared int value_saved inside save_value visible to print_value (which will see the file-scope variable above that isn't changed no matter what).

Originally Posted by bithub

void save_value(int new_value)

Code:

{
    value = new_value;
    barrier(); // memory barrier to prevent reordering of the store operations
    int value_saved = 1;
}

So what exactly does setting this local variable do?

Originally Posted by SyntaxError

So what exactly does setting this local variable do?

That's what I get for typing code directly into the cboard text box

Originally Posted by bithub

That's what I get for typing code directly into the cboard text box

In any case I wouldn't count on putting a routine call guaranteeing anything. What if the complier determines it does nothing and removes it. What if the compiler rearranges code anyway because it thinks it won't make a difference? Even if you can force the complier to do things in the correct order I still maintain that writing convoluted code generally gets you into trouble more frequently. If you do things by the book you will get fewer problems. If you do stuff like this I'm not surprised you think that thread programming is hard. I will admit thread programming adds complexity as does many other aspects of programming. However, as your mistake pointed out, very simple non threaded code can still have bugs.

However, as your mistake pointed out, very simple non threaded code can still have bugs.

Now you're just being silly. My mistake was due to re-arranging code in a message board text box.

In any case I wouldn't count on putting a routine call guaranteeing anything.

What?

What if the complier determines it does nothing and removes it.

Removes what? What is it in this sentence?

What if the compiler rearranges code anyway because it thinks it won't make a difference?

What if it does? This is something you need to account for when doing multi-threaded programming. You either need to account for this, or just place a mutex over the area in question. There's no way around it.

If you do stuff like this I'm not surprised you think that thread programming is hard.

You think that this was a convoluted example?? It was about as simple as you can get. Granted it was contrived to illustrate a point, but it's not hard to imagine that sort of paradigm existing in production code anywhere.

If you do things by the book you will get fewer problems.

What book?