Maximum concurrency

This is a discussion on Maximum concurrency within the Linux Programming forums, part of the Platform Specific Boards category; Is there a convenient API call that tells me the maximum system concurrency? (I.e. maximum number of threads that can ...

  1. #1
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893

    Maximum concurrency

    Is there a convenient API call that tells me the maximum system concurrency? (I.e. maximum number of threads that can actually be executed at the same time - CPU count, core count, that stuff.)
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  2. #2
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    It's probably a little bit awkward to get the info there, but /proc/cpuinfo contains the information you are after.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  3. #3
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    Something along the lines of
    Code:
    cat /proc/cpuinfo | grep vendor_id | wc -l
    ?
    The grep is so that I get one line for every info block.

    Yeah, possible, but as you say, awkward.


    I need this to size a thread pool. (One thread per CPU - it's a CPU-centric thing, no I/O at all.) There has to be some function for it.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  4. #4
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Yes, that looks about right.

    Here's a link that shows different lines for doing either including or excluding hyperthreads - scroll down about halfway.

    http://www.wcurtispreston.com/phpBB2...b46ead795fa879

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  5. #5
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    Well, thanks. I found GetCPUCount in the wxWidgets source, and they parse /proc/cpuinfo too.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  6. #6
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,645
    There's also "sysconf(_SC_NPROCESSORS_CONF)".

    You may want to use "sched_getaffinity()" since that'll tell you what your process is allowed to run on.

    gg

  7. #7
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    Thanks, these look interesting.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  8. #8
    Registered User
    Join Date
    Nov 2007
    Posts
    15
    There's an entry in /proc/sys/kernel/threads-max, may be that is what is needed ?

    Regards,
    Amit Sahrawat

  9. #9
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by amit_sahrawat View Post
    There's an entry in /proc/sys/kernel/threads-max, may be that is what is needed ?

    Regards,
    Amit Sahrawat
    I would have thought that's the "max number of threads ALLOWED", which is slightly different than "the number of threads that won't share a processor".

    Edit: googline that /proc thingy, it shows that "regular" vaues in that is 2047 or bigger - so not really the "number of threads you may want to start without competing for processors", but rather a system-wide limit for how many CAN EXIST in the system [and it's changeable, if you wish to increase/decrease it to allow more or save some kernel memory space respectively].

    --
    Mats
    Last edited by matsp; 11-22-2007 at 02:53 AM.
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  10. #10
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    Yeah, I doubt that 2^30 threads are really going to be useful

    After examining the problem I'm currently working on more closely, I've realized that threads won't be an improvement after all. Thanks anyway.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  11. #11
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by CornedBee View Post
    Yeah, I doubt that 2^30 threads are really going to be useful
    2^30, eh? That's quite some number - I guess this is a 64-bit system with plenty of memory, then?

    After examining the problem I'm currently working on more closely, I've realized that threads won't be an improvement after all. Thanks anyway.
    A fairly common thing - it's often easy to say "It will be faster if we multithread it", only to find that when you ACTUALLY try to do that, it doesn't gain nearly as much as you thought, because it's too reliant on previous results produced by another thread or some such. Converting a single threaded app to multithreads is often not as easy as people think when they first look at multithreading [not saying that you didn't know this].

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  12. #12
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    Quote Originally Posted by matsp View Post
    2^30, eh? That's quite some number - I guess this is a 64-bit system with plenty of memory, then?
    It is, but it's also a brain fart. 2^14 is the number I meant.


    A fairly common thing - it's often easy to say "It will be faster if we multithread it"
    Actually, my thought was, "Hey, I can split this loop into blocks for different threads, and they won't need any synchronization at all."

    Problem is that it's an inner loop, and the threads must wait for each other at the end of each iteration of the outer loop, and since the inner loop ought to be pretty tight, even this simple sync cost might still outweigh the gain.

    New plan is to SIMD the hell out of it.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  13. #13
    Registered User
    Join Date
    Nov 2006
    Posts
    519
    Quote Originally Posted by CornedBee View Post
    Problem is that it's an inner loop, and the threads must wait for each other at the end of each iteration of the outer loop, and since the inner loop ought to be pretty tight, even this simple sync cost might still outweigh the gain.
    Sounds like a job for openMP?

    Quote Originally Posted by CornedBee View Post
    New plan is to SIMD the hell out of it.
    Which instruction set are you going to use for this?

  14. #14
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    Quote Originally Posted by pheres View Post
    Sounds like a job for openMP?
    I don't see how OpenMP could avoid synchronization.

    Which instruction set are you going to use for this?
    Integer SSE.

    Just some background info. This is for a university class in low-level optimization. We have a pretty much given algorithm and we're supposed to make it as fast as possible. Target architecture is an Intel Core2 Duo.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  15. #15
    Registered User
    Join Date
    Nov 2006
    Posts
    519
    Quote Originally Posted by CornedBee View Post
    I don't see how OpenMP could avoid synchronization.
    It doesn't. I just wondered if you considered using it (beacuse GCC seems to support it since 4.2) and why/why not before you came up with the SIMD idea.

    Quote Originally Posted by CornedBee View Post
    Integer SSE.
    Are there GCC extensions like for MSVC or do you port to assembler? I'm just asking out of general interest.

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. HELP!!!!emergency Problem~expert please help
    By unknowppl in forum C++ Programming
    Replies: 9
    Last Post: 08-21-2008, 06:41 PM
  2. HELP!!!!emergency ~expert please help
    By unknowppl in forum C Programming
    Replies: 1
    Last Post: 08-19-2008, 07:35 AM
  3. Displaying Minimum and Maximum values from input
    By mgardnertech in forum C Programming
    Replies: 1
    Last Post: 06-29-2008, 08:47 PM
  4. C++ Binary Tree Maximum Value function
    By krizam in forum C++ Programming
    Replies: 4
    Last Post: 12-05-2005, 08:52 AM
  5. maximum number
    By drdodirty2002 in forum C Programming
    Replies: 7
    Last Post: 05-08-2003, 05:33 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21