Spawn on fly vs pre allocate threads

For a realtime application, where performance matters. What is the general view of getting better performance. Using threads spawned on the fly (pthread_create), or preallocate group of threads with signalling (cond/locking) ?

background 120Hz graphics update, heavy data processing, but too much data and programming complexity to use GPU/OpenCL. Have a fixed number of parallel jobs (6). In one cycle, peices of work can be forked off, while others remain stage based.

Any views would be good to digest.

Try both. Test. Benchmark.

It's not the sort of question that has an "answer" except in very, very specific circumstances (which you haven't given and probably don't have without showing us the code, by which time you've written the program and could Test, Benchmark, etc.).

P.S. What do you honestly think you are you doing that needs 120Hz updates while processing data and deliberately NOT using the GPU?

You might want to clarify what you actually mean by "realtime".

My definition of realtime is the formal one: that the complete system has failed to meet operational requirements if any process runs outside its specified time intervals (i.e. no process can occur before its designated start time, and that process must have completed before its designated deadline). Consider design of a pacemaker where the patient might suffer physical consequences or even death if the heart is stimulated either too early or too late.

From my perspective, your question is (best case) underspecified or (worst case) meaningless. Which makes me think your interpretation of "realtime" is different to mine.

If we take my notion of realtime out of the picture, relative performance of the two schemes you describe is system (OS and hardware) dependent. Starting threads introduces overhead. There is also overhead of signalling that affects the thread that waits for or receives the signal.

Quote:

---

Originally Posted by ledow

It's not the sort of question that has an "answer" except in very, very specific circumstances (which you haven't given and probably don't have without showing us the code, by which time you've written the program and could Test, Benchmark, etc.).

---

Firstly, I realise that my question is ill-posed. I am more interested in drawing upon others experiences, methodology or approach. I am not looking for a magic answer, just your view or personal experience.

Quote:

---

Originally Posted by ledow

Try both. Test. Benchmark.

---

I agree entirely.

Are there any tools to evaluate where per-thread performance can be measured, specifically the overhead of spawning threads vs waiting on signal (assume Linux dev env)?

I can understand that there are subtle points of the signal/lock procedures. I am not an expert in this.

Quote:

---

Originally Posted by ledow

P.S. What do you honestly think you are you doing that needs 120Hz updates while processing data and deliberately NOT using the GPU?

---

My problem is not mappable onto a SIMD parallel architecture like that of a GPU. Of course you realise that GPU is not an "answer" to all heavy computational problems.

Quote:

---

Originally Posted by grumpy

You might want to clarify what you actually mean by "realtime".

My definition of realtime is the formal one: that the complete system has failed to meet operational requirements if any process runs outside its specified time intervals (i.e. no process can occur before its designated start time, and that process must have completed before its designated deadline). Consider design of a pacemaker where the patient might suffer physical consequences or even death if the heart is stimulated either too early or too late.

From my perspective, your question is (best case) underspecified or (worst case) meaningless. Which makes me think your interpretation of "realtime" is different to mine.

If we take my notion of realtime out of the picture, relative performance of the two schemes you describe is system (OS and hardware) dependent. Starting threads introduces overhead. There is also overhead of signalling that affects the thread that waits for or receives the signal.

---

Nothing wrong with conflict in what is realtime. From my view, 120Hz is needed for crisp refresh rate when using stereo vision. You could say this is soft-realtime for which the deadline missed is not critical (appears laggy).

So being OS and hardware dependent, what could you say about your personal experience with either? Do you believe that pthread cond wait has been more convenient/useful than spawning threads?

I realised that by starting this thread I intended to obtain some experience or recommendations on advanced threading in C. I am more interested in the process of how performance and optimisation is approached, rather than how to solve the specific problem at hand. It's kind of like designing software, then implementation. One does not simply jump in and solve the problem. There are (I hope) some software engineering practices that developers use as part of their workflow. A workflow for a bigger-than-a-hack project is useful.

Apologies if I had come off snooty.

Obviously spawning a thread every time you need one is going to be more expensive than reusing them. It also encourages you to think about the problem at the WRONG level. You have a bunch of tasks you want to do in parallel, the fact that threads may be involved is an implementation detail. Your question is sort of like asking whether you should use some particular machine instruction or another one. Chances are you are asking the question at too low a level.

Go get Intel TBB, solve your problem and move forward instead of spending the next two years learning about every obscure threading bug known to man.

Quote:

---

Originally Posted by brewbuck

Go get Intel TBB, solve your problem and move forward instead of spending the next two years learning about every obscure threading bug known to man.

---

I have the impression that Intel TBB is a C++ library, not a C library.

Quote:

---

Originally Posted by brewbuck

Obviously spawning a thread every time you need one is going to be more expensive than reusing them.

---

Is this obvious? In ideal circumstances, keeping threads suspended and waiting for signal incurs zero computation cost. In addition, the time to activate that thread will have zero delay. If it is obvious, it be proven for any system.

Quote:

---

Originally Posted by brewbuck

It also encourages you to think about the problem at the WRONG level. You have a bunch of tasks you want to do in parallel, the fact that threads may be involved is an implementation detail. Your question is sort of like asking whether you should use some particular machine instruction or another one. Chances are you are asking the question at too low a level.

Go get Intel TBB, solve your problem and move forward instead of spending the next two years learning about every obscure threading bug known to man.

---

Thank you for referencing Intel TBB. It looks like they are addressing issues much closer to what I am interested in.

FYI I wouldn't actively discourage anyone to spend time learning to better their knowledge. If they have the time, point them in the right direction. Perhaps a warning is more appropriate.

Quote:

---

Originally Posted by laserlight

I have the impression that Intel TBB is a C++ library, not a C library.

---

This is true. The border of using C++ compiler for mostly C code had already been crossed anyways :)

Quote:

---

Originally Posted by lordmule

Nothing wrong with conflict in what is realtime. From my view, 120Hz is needed for crisp refresh rate when using stereo vision. You could say this is soft-realtime for which the deadline missed is not critical (appears laggy).

---

There is plenty wrong with using terms to mean something different from what others expect. It is like asking a question in english, and expecting a native japanese speaker to give you a useful answer. No matter how intelligent people might be, it is counter-productive (or, at best, inefficient) to speak to them using language and meanings they are not familiar with.

Quote:

---

Originally Posted by lordmule

So being OS and hardware dependent, what could you say about your personal experience with either? Do you believe that pthread cond wait has been more convenient/useful than spawning threads?

---

My personal experience is that there are trade-offs. There are some circumstances where spawning threads is more convenient/useful than waiting on conditions. And other circumstances where the reverse is true. There are also alternatives that sometimes surpass either technique (and sometimes not).

Such is the nature of engineering trade-off. There are often no absolute answers. What may be considered better on one machine (computer, operating system, operating system version, etc) may be woefully deficient on another.

It all comes down to requirements. Sometimes to meet highly specific requirements, it is necessary to be selective of hardware, operating system, as well as in your software design. If, for example, your timing requirements are relaxed, the software will run as required on a wider range of hardware.

Quote:

---

Originally Posted by lordmule

I realised that by starting this thread I intended to obtain some experience or recommendations on advanced threading in C. I am more interested in the process of how performance and optimisation is approached, rather than how to solve the specific problem at hand. It's kind of like designing software, then implementation. One does not simply jump in and solve the problem. There are (I hope) some software engineering practices that developers use as part of their workflow. A workflow for a bigger-than-a-hack project is useful.

---

Good luck on that. I've lost count of the number of researchers, and number of PhD projects, that seek to define some universal approach to capturing requirements, translating those requirements into software, etc etc. Every time someone comes up with their perceived panacea, some other researcher or some real-world practitioner comes up with a counter-example that demonstrates their approach does not always work.

The same is true of advanced threading. Firstly, there is formally no threading in C. Threading is outside the scope of the C standard. All threading relies on some library that is specified outside the C standard (for example, the IEEE POSIX specification). Second, even if you settle on using some threading API (be it POSIX, win32 threading functions, or whatever) you are in the realm of functionality that is highly operating system dependent. Different operating systems optimise different attributes that affect the overhead of things like process creation, thread creation within threads, creating and triggering semaphores, etc. So the relative performance of (say) creating a thread pool versus creating a new thread every time one is needed are highly operating system dependent.

And then you get into hardware dependence: how the CPU, memory system, etc is optimised around some performance parameters rather than others.

Quote:

---

Originally Posted by grumpy

There is plenty wrong with using terms to mean something different from what others expect. It is like asking a question in english, and expecting a native japanese speaker to give you a useful answer. No matter how intelligent people might be, it is counter-productive (or, at best, inefficient) to speak to them using language and meanings they are not familiar with.


My personal experience is that there are trade-offs. There are some circumstances where spawning threads is more convenient/useful than waiting on conditions. And other circumstances where the reverse is true. There are also alternatives that sometimes surpass either technique (and sometimes not).

Such is the nature of engineering trade-off. There are often no absolute answers. What may be considered better on one machine (computer, operating system, operating system version, etc) may be woefully deficient on another.

It all comes down to requirements. Sometimes to meet highly specific requirements, it is necessary to be selective of hardware, operating system, as well as in your software design. If, for example, your timing requirements are relaxed, the software will run as required on a wider range of hardware.


Good luck on that. I've lost count of the number of researchers, and number of PhD projects, that seek to define some universal approach to capturing requirements, translating those requirements into software, etc etc. Every time someone comes up with their perceived panacea, some other researcher or some real-world practitioner comes up with a counter-example that demonstrates their approach does not always work.

The same is true of advanced threading. Firstly, there is formally no threading in C. Threading is outside the scope of the C standard. All threading relies on some library that is specified outside the C standard (for example, the IEEE POSIX specification). Second, even if you settle on using some threading API (be it POSIX, win32 threading functions, or whatever) you are in the realm of functionality that is highly operating system dependent. Different operating systems optimise different attributes that affect the overhead of things like process creation, thread creation within threads, creating and triggering semaphores, etc. So the relative performance of (say) creating a thread pool versus creating a new thread every time one is needed are highly operating system dependent.

And then you get into hardware dependence: how the CPU, memory system, etc is optimised around some performance parameters rather than others.

---

Wow. I can see why you are called grumpy.
- Realtime has no strict meaning. It is context dependent. (Japanese speaking vs English? lol)
- I have already understood that trade off exists. Nobody's solution, or approach is applicable to my own. I am merely surveying the landscape, not finding the definitive and only path.
grumpy, while you appear to have good experience and knowledge about you, little has been presented with reference to threads.


Reading through the questions and answers of this FAQ proved very useful: comp.programming.threads

Quote:

---

Originally Posted by lordmule

Wow. I can see why you are called grumpy.

---

And I can see why you're named after a beast of burden.

Quote:

---

Originally Posted by lordmule

- Realtime has no strict meaning. It is context dependent. (Japanese speaking vs English? lol)

---

Wrong. The term realtime (in the context of computing systems) has meaning that is specified in a number of international standards.

You need to do your homework.

Quote:

---

Originally Posted by lordmule

- I have already understood that trade off exists. Nobody's solution, or approach is applicable to my own. I am merely surveying the landscape, not finding the definitive and only path.
grumpy, while you appear to have good experience and knowledge about you, little has been presented with reference to threads.

---

You asked a general question, and you have received general answers. And the answers were appropriate to the question you asked.

C11 has threading, BTW. Don't know of any implementations yet though.

gg