Thread: A huge page faults problem

  1. #1
    In the Land of Diddly-Doo g4j31a5's Avatar
    Join Date
    Jul 2006
    Posts
    476

    A huge page faults problem

    My current project was running hideously slow after a few hours or so. When I checked the memory consumption in the task manager, it showed a pretty steady memory fluctuation. But when I expanded the view, it showed that its page fault was 4 gigs. Is this some kind of memory leak? Or is this a whole different matter? Thanks in advance.
    ERROR: Brain not found. Please insert a new brain!

    “Do nothing which is of no use.” - Miyamoto Musashi.

  2. #2
    Algorithm Dissector iMalc's Avatar
    Join Date
    Dec 2005
    Location
    New Zealand
    Posts
    6,318
    Do you mean that the Page Fault Delta was in the Gigabytes?
    That doesn't indicate a memory leak. On the contrary, it indicates that you program is using chunks of memory from all over the place. If those chunks were leaked, your app wouldn't be accessing them.
    Consider using Sysinternal's "Process Explorer" rather than Task Manager, btw.
    My homepage
    Advice: Take only as directed - If symptoms persist, please see your debugger

    Linus Torvalds: "But it clearly is the only right way. The fact that everybody else does it some other way only means that they are wrong"

  3. #3
    In the Land of Diddly-Doo g4j31a5's Avatar
    Join Date
    Jul 2006
    Posts
    476
    Quote Originally Posted by iMalc View Post
    Do you mean that the Page Fault Delta was in the Gigabytes?
    That doesn't indicate a memory leak. On the contrary, it indicates that you program is using chunks of memory from all over the place. If those chunks were leaked, your app wouldn't be accessing them.
    Consider using Sysinternal's "Process Explorer" rather than Task Manager, btw.
    Not the Delta but the Page Fault itself. But the Page Fault Delta always at 6 bytes even if I didn't do anything. It will go up if I do something within my application, but even if I didn't do anything it would add 6 bytes constantly.

    About the Process Explorer, I'll give it a shot. Thanks.
    ERROR: Brain not found. Please insert a new brain!

    “Do nothing which is of no use.” - Miyamoto Musashi.

  4. #4
    Registered User
    Join Date
    Jan 2008
    Posts
    290
    The Page Faults column in Windows Task Manager tells you how many times your process has encountered a page fault. It is a COUNT, not a measurement in bytes.

    http://en.wikipedia.org/wiki/Page_fault

  5. #5
    Algorithm Dissector iMalc's Avatar
    Join Date
    Dec 2005
    Location
    New Zealand
    Posts
    6,318
    If you app runs slower and slower, then leave it runing for a reasonably long time and then attach the debugger and hit break. There's a good chance it will break in the place where the time is being spent.
    It's probably the result of using a wrong Big-Oh notation algorithm somewhere, causing you to (for example) have to search through a list of ever increasing length.
    My homepage
    Advice: Take only as directed - If symptoms persist, please see your debugger

    Linus Torvalds: "But it clearly is the only right way. The fact that everybody else does it some other way only means that they are wrong"

  6. #6
    In the Land of Diddly-Doo g4j31a5's Avatar
    Join Date
    Jul 2006
    Posts
    476
    @arpsmack

    OIC, didn't know that. Thanks. But IMHO that many counts of page faults are not healthy, right?

    @iMalc

    I'll try that. Although I'm not quite sure it will point to the real problem. FYI, I'm using a third party engine and it uses its own C-like scripting language and most of the processes (eg. object instantiations, logics, etc. ) are using that scripting. It's created on lex/yacc, AFAIK. That's also what's driving me crazy right now. Because I can debug the source code but not the scripting. I can't put any breaks in the script. And any break in the source code won't get me much information of which script uses that particular line.

    On a second thought, maybe it is a mem leak after all. The memory (and page faults) is always increasing after all. Unfortunately object instantiations and deletions are handled via scripting so I can't debug them quite well. Oooh... The agony...

    Thanks anyway.

    BTW, sorry for the late reply.
    ERROR: Brain not found. Please insert a new brain!

    “Do nothing which is of no use.” - Miyamoto Musashi.

  7. #7
    Registered User
    Join Date
    Jan 2008
    Posts
    290
    Its possible to write wrapper functions around your memory allocation routines to keep track of allocations and related deallocations. I've seen a couple open source projects that do this and I suspect that its probably fairly common for large projects (although I don't really have the experience required to make a statement like that).

    Keep in mind what iMalc said though. Page faults don't indicate a memory leak. The ever increasing memory does though (unless it's supposed to be increasing).

  8. #8
    In the Land of Diddly-Doo g4j31a5's Avatar
    Join Date
    Jul 2006
    Posts
    476
    Actually, the engine has already did that wrapping. I've tried using it to determine the leak. But that is source code wise, not script wise. So even if I know which line in the source was leaking, I can't get the particular scripting that was the real culprit. Thanks anyway. Still working my @$$ off as of now...
    ERROR: Brain not found. Please insert a new brain!

    “Do nothing which is of no use.” - Miyamoto Musashi.

  9. #9
    Registered User
    Join Date
    Jan 2008
    Posts
    290
    Wait, so the scripting language already provides wrapping facilities to detect memory leaks, but they can't pinpoint exactly where the allocation occurred? Well, then what the hell is the point?

    Scripting Language: "You have a memory leak somewhere in this script, uh hyuk!"
    You: "Thanks for the tip, @$^@$%@."

    And you have NO way of writing your own additional wrappers that would accomplish this?

  10. #10
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    A profiler can help you find bottlenecks in your program.
    There are also several tools to help detect memory leaks.
    You can also use smart pointers & co to avoid new/delete problems altogether.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  11. #11
    In the Land of Diddly-Doo g4j31a5's Avatar
    Join Date
    Jul 2006
    Posts
    476
    @arpsmack: Well, not quite like that. It can recognize the exact line & the source .cpp file of the leak but not in which part of the script that call that line. E.g.:


    A leak in: file "leak.cpp" line 13:

    1: MyObject(name){ //constructor
    ...
    13: mMyName = new char[strlen(name)]; //mMyName is a member variable with a type of "char *"
    In the script (it was C++ like BTW), maybe there are a few lines of scripting that use that line like this:

    15: new MyObject(obj1);
    25: new MyObject(obj2);
    35: new MyObject(obj3);
    Now, I know that it is leaking in MyObject because its constructor is leaking. But which one of those objects are the one leaking? I may have already deleted, for example, obj1 & 2 but forgot the third one, but how do I know that? They were all using the same leaking line.

    I've tried to debug it step by step but ended up in the lex/yacc part of the scripting library. Gah, it's so frustating.

    @Elysia

    Actually there's already a built-in profiler (along with the memory manager & leak detector). I've tried it before and it's pretty good actually. You can toggle only a few lines of code with a "profileStart(name);" and "profileEnd(name)" commands that act like a braccet to check the time of individual lines. I still use it to profile my app. For the tools to detect leaks, I doubt that they will also recognize the scripting part.

    About the smart pointers, I don't quite believe in anything that's "automatic" like smart pointers, atExit(), etc. I feel much more secure to handle things traditionally. The problem is the library is managing memory in its own way and to change them all is a pain in the ass. Also there's a problem with the scripting like I mentioned earlier.

    BTW, after looking at the leaking parts of the code, I notice that most (if not all) of them are basically allocations of chars. So the class would have a "char *" member variable that's defined and allocated from a member method like in my example earlier. And that's from the engine's source code itself.
    ERROR: Brain not found. Please insert a new brain!

    “Do nothing which is of no use.” - Miyamoto Musashi.

  12. #12
    Registered User
    Join Date
    Jan 2008
    Posts
    290
    This might sound like a stupid question but I feel I must ask: you are remembering to delete mMyName in the destructor right?

  13. #13
    In the Land of Diddly-Doo g4j31a5's Avatar
    Join Date
    Jul 2006
    Posts
    476
    Quote Originally Posted by arpsmack View Post
    This might sound like a stupid question but I feel I must ask: you are remembering to delete mMyName in the destructor right?
    I think I have because not all of the instances are flagged as leaks.
    ERROR: Brain not found. Please insert a new brain!

    “Do nothing which is of no use.” - Miyamoto Musashi.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Bin packing problem....
    By 81N4RY_DR460N in forum C++ Programming
    Replies: 0
    Last Post: 08-01-2005, 05:20 AM
  2. Words and lines count problem
    By emo in forum C Programming
    Replies: 1
    Last Post: 07-12-2005, 03:36 PM
  3. half ADT (nested struct) problem...
    By CyC|OpS in forum C Programming
    Replies: 1
    Last Post: 10-26-2002, 08:37 AM
  4. binary tree problem - help needed
    By sanju in forum C Programming
    Replies: 4
    Last Post: 10-16-2002, 05:18 AM
  5. huge string-pointer problem
    By MKashlev in forum C++ Programming
    Replies: 3
    Last Post: 09-16-2002, 02:54 PM