Thread: Help with high CPU usage

  1. #1
    Registered User
    Join Date
    Nov 2007
    Posts
    4

    Help with high CPU usage

    Hi all

    I wrote a program that continuously extracting specific format files to proper records for post processing, but the usage of CPU is keeping high, usually 99%.
    So the extracting process was slowly and slowly, when I use sleep inside a loop, the CPU is down, but the extract performance wasn't improved because of sleeping.

    I wonder what caused CPU keeping so high? Does my extract algorithm bad?
    I use gprof to profile the program, nothing really helps. I was struggled what could I do, Please someone help me out.


    Thanks

    Black Thought

  2. #2
    Registered User
    Join Date
    Apr 2007
    Location
    Sydney, Australia
    Posts
    217
    Any program that has no blocking operations or no sleep() command will probably use 100% cpu (or 50%). Also why would you think the performance of the extract algorithm would IMPROVE when you add a sleep command? It would definately slow down.
    Last edited by 39ster; 07-20-2008 at 09:49 PM.

  3. #3
    Registered User
    Join Date
    Nov 2007
    Posts
    4
    Hi 39ster.

    In your words, no matter how good the algorithm is, a non-blocking program will exhaust the entire CPU, right?

  4. #4
    Registered User
    Join Date
    Apr 2007
    Location
    Sydney, Australia
    Posts
    217
    Try this program:
    Code:
    int main(int argc, char* argv[])
    {
        while(1){}
        return 0;
    }
    So for a GUI program it would use "GetMessage" in the main loop, which pauses until it receives a windows message. A server program might use "select" which pauses until at least one socket has data to receive. A game might limit the FPS using Sleep(). If you don't want it to use 99% cpu than you need to use a blocking operation in the main loop.
    Last edited by 39ster; 07-20-2008 at 09:57 PM.

  5. #5
    int x = *((int *) NULL); Cactus_Hugger's Avatar
    Join Date
    Jul 2003
    Location
    Banks of the River Styx
    Posts
    902
    Quote Originally Posted by 51dunk View Post
    Hi 39ster.

    In your words, no matter how good the algorithm is, a non-blocking program will exhaust the entire CPU, right?
    Yes, basically. Exhausting the CPU is not by itself a bad thing. Exhausting the CPU and accomplishing nothing is.

    A better algorithm will finish its work quicker. I'd rather have 100% cpu for 2 seconds and the program be complete than 50% cpu for 2 minutes for the same result.

    Now, if you need to wait on, say, more data to become available, don't have the program enter a while(1) loop - waiting for some condition to occur is when a program should sleep. (And then, wake when the condition is met.)
    long time; /* know C? */
    Unprecedented performance: Nothing ever ran this slow before.
    Any sufficiently advanced bug is indistinguishable from a feature.
    Real Programmers confuse Halloween and Christmas, because dec 25 == oct 31.
    The best way to accelerate an IBM is at 9.8 m/s/s.
    recursion (re - cur' - zhun) n. 1. (see recursion)

  6. #6
    Algorithm Dissector iMalc's Avatar
    Join Date
    Dec 2005
    Location
    New Zealand
    Posts
    6,318
    Your code does what is called Busy waiting.
    My homepage
    Advice: Take only as directed - If symptoms persist, please see your debugger

    Linus Torvalds: "But it clearly is the only right way. The fact that everybody else does it some other way only means that they are wrong"

  7. #7
    Registered User
    Join Date
    Nov 2007
    Posts
    4
    Thanks everybody for replying.

    My program is an extracting program, it will extract one or more 100MB+ files to 1,000,000+ records .
    So it isn't a one-two seconds operation, and I've read "Busy Waiting", yes, I have a lot of "while" loops, but I think everything is necessary.

    I can't find anything helpful except sleep() which will slow down my program the other way.

  8. #8
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    You may want to use some profiling on your application and see where it spends time. It sounds like you do a bit of file-IO and some string handing [I'm just guessing]. It is quite often possible to find more efficient methods. However, that's only of some importance if 2 seconds of CPU usage is considered too much. Otherwise, just leave it as it is.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  9. #9
    Registered User
    Join Date
    Nov 2007
    Posts
    4
    Hello mats

    Yes, I've use GProf to profile my program, there are some functions that was called million times, so I said before I wonder if it might be problem of bad algorithm. I believe that 2 seconds of CPU usage is not critical, but my program always run a couple of minutes or hours, so high CPU usage will kill the system.

    Black Thought

  10. #10
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Just because a function is called a million times doesn't make it bad in itself.

    What are those functions doing, and can they be done in a different way more efficiently?

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  11. #11
    Registered User
    Join Date
    Jul 2008
    Posts
    29
    Sounds like an infinite loop to me.....just sayin. But if that isnt it and you arent busy waiting (which pretty much is an infinite loop) the only thing that you can do is try and optimize your algorithm

  12. #12
    Algorithm Dissector iMalc's Avatar
    Join Date
    Dec 2005
    Location
    New Zealand
    Posts
    6,318
    Quote Originally Posted by chacham15 View Post
    Sounds like an infinite loop to me.....just sayin. But if that isnt it and you arent busy waiting (which pretty much is an infinite loop) the only thing that you can do is try and optimize your algorithm
    You're absolutely right. It may not actually be busy waiting here. The initial description said it was "continually" doing something, but it may be that what it really does is a fixed number of iterations, taking a long time overall. That is not busy waiting, just plain old high CPU usage (bottleneck).

    In general decompression algorithms are normally very fast, and the bottleneck is on the disk IO. If the bottleneck in your decompression is CPU usage rather than IO, then that suggests that there is some horrible inefficiency in your decompressor. To help with that we'd need to know the algorithm used, or even have access to portions of the source code.
    My homepage
    Advice: Take only as directed - If symptoms persist, please see your debugger

    Linus Torvalds: "But it clearly is the only right way. The fact that everybody else does it some other way only means that they are wrong"

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. questions on multiple thread programming
    By lehe in forum C Programming
    Replies: 11
    Last Post: 03-27-2009, 07:44 AM
  2. Net cpu usage of pthreads?!
    By mynickmynick in forum C++ Programming
    Replies: 15
    Last Post: 09-26-2008, 07:59 AM
  3. Calculating CPU Usage
    By vitaliy in forum Linux Programming
    Replies: 3
    Last Post: 08-21-2005, 09:38 AM
  4. fseek high cpu usage?
    By chunlee in forum C Programming
    Replies: 2
    Last Post: 02-19-2005, 07:27 AM
  5. CPU Usage so high
    By X PaYnE X in forum Windows Programming
    Replies: 9
    Last Post: 12-21-2003, 03:07 AM