How difficult is parallel programming in C?

This is a discussion on How difficult is parallel programming in C? within the C Programming forums, part of the General Programming Boards category; I know some graduate physics students who are writing research programs in C. These programs involve massive calculations. I am ...

  1. #1
    Registered User
    Join Date
    May 2006
    Posts
    57

    How difficult is parallel programming in C?

    I know some graduate physics students who are writing research programs in C. These programs involve massive calculations.

    I am trying to choose a configuration for a new workstation for these students. They tell me, however, that they don't want a multicore processor because it is too difficult to write a program utilizing more than one processor, they don't want to write parallel.

    This seems like a problem to me because I don't know any workstation config that doesn't use multiple processors. Is it really that hard to write a C program to utilize 2, 4 or 8 processors? (I've never tired it)

    Thanks

  2. #2
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,669
    Owning a multi-core processor, and writing code to utilize more than one core are unrelated. A single threaded C/C++ program will always run on just one core, regardless of how many cores you have.

    The OS and other applications may appreciate the extra cores, even if the programmers don't.

    gg

  3. #3
    Registered User
    Join Date
    Jul 2009
    Posts
    6
    The C language, as far as I know, doesn't have any statement or anything that can help you learn parallel programming.

    That does not mean you can't do parallel computing from C, but you have to use a library, for example. Useful keywords : threads, OpenMP, CUDA, etc. CUDA could be fun I guess.

    Or maybe start there :
    Parallel programming model - Wikipedia, the free encyclopedia

  4. #4
    Registered User
    Join Date
    May 2006
    Posts
    57
    These are physics students, they really don't want to get into heavy programming. And I don't know if I have the time to help them.

    They do run some of these programs on supercomputers, but they have someone there to utilize the 100+ parallel processors there.

    I'll at least take a look at some libraries.

  5. #5
    Registered User
    Join Date
    Sep 2004
    Location
    California
    Posts
    3,246
    One thing to keep in mind: If their application is doing a bunch of calculations where each calculation relies on the result of the previous calculation, it can be pretty tough to refactor it such that parrallel programming provides any performance gains.

  6. #6
    C++まいる!Cをこわせ! Elysia's Avatar
    Join Date
    Oct 2007
    Posts
    22,602
    Parallel programming can both be easy and difficult.
    If everything is done by its own, all different tasks, so to speak, then it's rather straight-forward. But if calculations depends on other calculations which depends on some other calculation, then it's going to grow complex pretty fast because you have to synchronize.
    And that opens up the possibility of bugs and race conditions which are hard to debug.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  7. #7
    Registered User
    Join Date
    May 2006
    Posts
    57
    Apparently at the supercomputer center there were lots of experts helping them run their programs on multiple processors, plus they used libraries. But they want to do some work on their own on a local workstation.

    Also their professor was complaining that sometimes those special libraries were not adequate all the time.

    Like I said before, I'll do some snooping around (I've done some programming before) but I think it will be too hard for those students to do it on their own.

  8. #8
    Registered User
    Join Date
    Oct 2001
    Posts
    2,129
    If all you want is some speedup from the cores, and not necessarily a certain amount of speedup in a certain way, you might want to use an auto-parallelizing compiler. I don't think GCC has that option, but Sun's SunCC does. Check out the -xautopar option.

  9. #9
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,708
    >> Also their professor was complaining that sometimes those special libraries were not adequate all the time.

    Well, threading libraries are fairly standard (eg: pthreads) so I wouldn't necessarily call them "special". If they really can't handle it, though, I'd look into robwhit's suggestion.
    Code:
    #include <cmath>
    #include <complex>
    bool euler_flip(bool value)
    {
        return std::pow
        (
            std::complex<float>(std::exp(1.0)), 
            std::complex<float>(0, 1) 
            * std::complex<float>(std::atan(1.0)
            *(1 << (value + 2)))
        ).real() < 0;
    }

  10. #10
    C++11 User Tux0r's Avatar
    Join Date
    Nov 2008
    Location
    Sweden
    Posts
    135
    Is there a list of stuff to follow, like use const etc

  11. #11
    Registered User
    Join Date
    Dec 2006
    Location
    Canada
    Posts
    3,183
    Most primitive kind of parallel processing, without needing to change the program - use the OS scheduler!

    Can they run multiple instances of their program to work on different problems? (or different parts of a problem) Can they split the input into independent sections?

  12. #12
    Registered User
    Join Date
    Jun 2005
    Posts
    6,290
    Yeah, OK. These physics students want to run a lot of calculations, but have a "it's just software" mentality and don't understand the mechanics of running their software

    If the only hardware they can obtain has a multi-core processor the software will still only execute on one processor/core unless it is already designed in parts that can be run in parallel. That will mean one processor/core flat out, and the others idling. The output will be the same. If it isn't that means the software is broken in some way.

    Other things they need to consider (some of which others have already said in this thread, some not) are;

    1) The best opportunities for parallelising will come when there are no inter-dependencies between sets of calculations. If any of the output from run A needs to be fed input into run B, the opportunities for parallelising run A and run B are constrained.

    2) A lot of physics-based simulations dump output at the end of a run, so that means no opportunity for parallelisation.

    3) Physicists are required to actually understand the physics. If they do they should be able to describe the process that they are simulating on paper, and understand where the interdependencies of the process are. That bit of paper may be large, but it will probably be the best thing they can provide to the software specialists who implement the code.

    4) If physicists are worrying about executing code on their own computer, that probably means they have limited understanding of the physics. Building that understanding would be the best application of their time. If they understand the physics, they can better describe things so the simulation can be implemented in software - or they can solve a problem another way.

    5) I've seen horror stories in situations like this. For example, one physics PhD student spending all his time coordinating with computing centres for spare processing cycles, and setting up batch jobs to run the program with select inputs on different supercomputers. After receiving (literally!) truck loads of output, he noticed some anomalies in outputs but didn't understand them. Six months later (with more runs on the various supercomputers) he isolated the problem to a bug in some code he had modified. He then had to fix the code and run most of the computations again from scratch. His PhD project turned out to take seven years.

  13. #13
    Registered User
    Join Date
    May 2006
    Posts
    57
    I've discussed this issue with the professor, and have been told the main graduate student in discussion is quite competent in writing code. So apparently he can write multithreaded code, but doesn't want to in this situation. (They write their own code, they don't have THAT much money). I've heard them complain a great deal about the difficulties they are have been running into over the years working with parallel systems at supercomputer centers (even with people helping them). Maybe they are facing some of the difficulties you guys are pointing out.

    I've explained the inefficiency that running a single thread program would cause, particularly with the basic hardware designs they have today, and that it would be so much faster multi-threaded, but I'm being ignored. So hopefully they know what they are doing.

    I will bring up all your points to them, however. It might help.

    I'm guessing the programs will run rather slow (on just one of the 4 cores in a quad core processor, and not even using the other processor) but will have access to a huge amount of memory for neccessary data. Also it is possible some other physics students might be new at this and would want to avoid writing parallel code.

    I too can see all kinds of trouble a physics programs can run into, considering how gigantic their calculations are. There are 'soft errors' caused by cosmic rays flipping bits, one or two errors a month. Then there are round off errors caused by that crazy 'floating point representation' in computer storage. And of course the infamous bug in the code. I am a bit concerned about bugs, since they are physicists first, programmers second, and even dedicated programmers have all kinds of problems with bugs.

    I'm trying to design the computer workstation to take into account these problems (except for bugs), but this is my first workstation! Wish me luck.

  14. #14
    C++まいる!Cをこわせ! Elysia's Avatar
    Join Date
    Oct 2007
    Posts
    22,602
    Quote Originally Posted by darsunt View Post
    Then there are round off errors caused by that crazy 'floating point representation' in computer storage.
    This is true, and frankly, the natives types are unless IMHO.
    Thankfully, there are libraries out there with unlimited precision. But it comes at the price of extra computing power, of course.

    Well, good luck anyway. Parallelism is not easy.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  15. #15
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    Quote Originally Posted by darsunt View Post
    I've discussed this issue with the professor, and have been told the main graduate student in discussion is quite competent in writing code. So apparently he can write multithreaded code, but doesn't want to in this situation. (They write their own code, they don't have THAT much money). I've heard them complain a great deal about the difficulties they are have been running into over the years working with parallel systems at supercomputer centers (even with people helping them). Maybe they are facing some of the difficulties you guys are pointing out.
    I think you are little hardware crazied with your feeling that a single core processor is wrong to use no matter what, but here's a thought in support of your idea: They can write single threaded programs and run them in parallel to greater effect on a multi core system, since if you run two labour intensive things at the same time, the OS will give them a processor each. Unless there is only one thing they actually ever need to do at one time, which seems unlikely.


    I too can see all kinds of trouble a physics programs can run into, considering how gigantic their calculations are. There are 'soft errors' caused by cosmic rays flipping bits, one or two errors a month.
    Wow. I guess nothing will prevent that but it is worth noting that the other advantage to running single threaded apps on a multi-core system is that the OS will always have resources available to it if they are leaving something running for weeks, which might make it all more stable.
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Parallel Port to USB controller and outb()
    By coletek in forum Linux Programming
    Replies: 1
    Last Post: 06-05-2009, 06:57 AM
  2. Serial to Parallel
    By ssharish2005 in forum Tech Board
    Replies: 11
    Last Post: 09-10-2007, 01:11 PM
  3. Parallel port programming
    By h3ro in forum Windows Programming
    Replies: 6
    Last Post: 08-08-2007, 11:14 AM
  4. Segmentation Fault - Trying to access parallel port
    By tvsinesperanto in forum C Programming
    Replies: 3
    Last Post: 05-24-2006, 03:28 AM
  5. Help needed: Output to Parallel port.
    By Ingsy in forum C Programming
    Replies: 4
    Last Post: 10-10-2001, 12:06 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21