PDA

View Full Version : Maximum concurrency



CornedBee
11-18-2007, 12:04 PM
Is there a convenient API call that tells me the maximum system concurrency? (I.e. maximum number of threads that can actually be executed at the same time - CPU count, core count, that stuff.)

matsp
11-18-2007, 12:30 PM
It's probably a little bit awkward to get the info there, but /proc/cpuinfo contains the information you are after.

--
Mats

CornedBee
11-18-2007, 12:43 PM
Something along the lines of

cat /proc/cpuinfo | grep vendor_id | wc -l
?
The grep is so that I get one line for every info block.

Yeah, possible, but as you say, awkward.


I need this to size a thread pool. (One thread per CPU - it's a CPU-centric thing, no I/O at all.) There has to be some function for it.

matsp
11-18-2007, 01:13 PM
Yes, that looks about right.

Here's a link that shows different lines for doing either including or excluding hyperthreads - scroll down about halfway.

http://www.wcurtispreston.com/phpBB2/viewtopic.php?p=173683&sid=2e5b663603332c668eb46ead795fa879

--
Mats

CornedBee
11-18-2007, 01:22 PM
Well, thanks. I found GetCPUCount in the wxWidgets source, and they parse /proc/cpuinfo too.

Codeplug
11-18-2007, 02:16 PM
There's also "sysconf(_SC_NPROCESSORS_CONF)".

You may want to use "sched_getaffinity()" since that'll tell you what your process is allowed to run on.

gg

CornedBee
11-18-2007, 03:02 PM
Thanks, these look interesting.

amit_sahrawat
11-22-2007, 03:44 AM
There's an entry in /proc/sys/kernel/threads-max, may be that is what is needed ?

Regards,
Amit Sahrawat

matsp
11-22-2007, 03:50 AM
There's an entry in /proc/sys/kernel/threads-max, may be that is what is needed ?

Regards,
Amit Sahrawat

I would have thought that's the "max number of threads ALLOWED", which is slightly different than "the number of threads that won't share a processor".

Edit: googline that /proc thingy, it shows that "regular" vaues in that is 2047 or bigger - so not really the "number of threads you may want to start without competing for processors", but rather a system-wide limit for how many CAN EXIST in the system [and it's changeable, if you wish to increase/decrease it to allow more or save some kernel memory space respectively].

--
Mats

CornedBee
11-22-2007, 03:54 AM
Yeah, I doubt that 2^30 threads are really going to be useful ;)

After examining the problem I'm currently working on more closely, I've realized that threads won't be an improvement after all. Thanks anyway.

matsp
11-22-2007, 03:57 AM
Yeah, I doubt that 2^30 threads are really going to be useful ;)

2^30, eh? That's quite some number - I guess this is a 64-bit system with plenty of memory, then?



After examining the problem I'm currently working on more closely, I've realized that threads won't be an improvement after all. Thanks anyway.

A fairly common thing - it's often easy to say "It will be faster if we multithread it", only to find that when you ACTUALLY try to do that, it doesn't gain nearly as much as you thought, because it's too reliant on previous results produced by another thread or some such. Converting a single threaded app to multithreads is often not as easy as people think when they first look at multithreading [not saying that you didn't know this].

--
Mats

CornedBee
11-22-2007, 04:38 AM
2^30, eh? That's quite some number - I guess this is a 64-bit system with plenty of memory, then?
It is, but it's also a brain fart. 2^14 is the number I meant.



A fairly common thing - it's often easy to say "It will be faster if we multithread it"

Actually, my thought was, "Hey, I can split this loop into blocks for different threads, and they won't need any synchronization at all."

Problem is that it's an inner loop, and the threads must wait for each other at the end of each iteration of the outer loop, and since the inner loop ought to be pretty tight, even this simple sync cost might still outweigh the gain.

New plan is to SIMD the hell out of it.

pheres
11-22-2007, 10:09 AM
Problem is that it's an inner loop, and the threads must wait for each other at the end of each iteration of the outer loop, and since the inner loop ought to be pretty tight, even this simple sync cost might still outweigh the gain.Sounds like a job for openMP?


New plan is to SIMD the hell out of it.Which instruction set are you going to use for this?

CornedBee
11-22-2007, 10:49 AM
Sounds like a job for openMP?
I don't see how OpenMP could avoid synchronization.


Which instruction set are you going to use for this?
Integer SSE.

Just some background info. This is for a university class in low-level optimization. We have a pretty much given algorithm and we're supposed to make it as fast as possible. Target architecture is an Intel Core2 Duo.

pheres
11-22-2007, 11:45 AM
I don't see how OpenMP could avoid synchronization.It doesn't. I just wondered if you considered using it (beacuse GCC seems to support it since 4.2) and why/why not before you came up with the SIMD idea.



Integer SSE.Are there GCC extensions like for MSVC or do you port to assembler? I'm just asking out of general interest.

CornedBee
11-23-2007, 06:55 AM
I hadn't considered OpenMP because I've had practically no exposure to it.

GCC doesn't have any intrinsics like the Intel and MS compilers have. I'll have to write that part in Assembly.