Thread: stdout -> ram?

  1. #1
    Registered User
    Join Date
    May 2004
    Posts
    73

    stdout -> ram?

    Hey all, I have an interesting predicament. In my debugging, I need to print out a massive number of debug statements, unbuffered (so i know where exactly im crashing), on every frame of my real-time program.

    These printouts are slowing me way, way down. I'd imagine the reason stdio is so slow is that it has to write to disk. Is there any way to mount stdio into the ram? I've seen other things mount pieces of the hard drive onto ram for super-fast access, but I haven't a clue how to go about it for stdout.

  2. #2
    Registered User
    Join Date
    Nov 2005
    Posts
    673
    I am not sure I understand your question completely. If I were to do it, I would build a vector containing the debug information/statements, then periodically write to files ( say every 1000 elements ). Then have an functions that is forced to run at program termination ( say a class destructor ) write the remainder of debug information. This would minimize overhead during runtime, but by what amount im not sure.

    You may even optimize more by combining the debug statements into a single write call. ( batching )

  3. #3
    Registered User
    Join Date
    May 2004
    Posts
    73
    I can't be buffering things like that though. My program crashes, and the last few statements are vital to figuring out where the crash happened.

  4. #4
    Registered User
    Join Date
    Nov 2005
    Posts
    673
    Have you considered using a second process to handle the debug information, and just send the information from the debugging application. So, even after the crash the information is not lost?

  5. #5
    Registered User
    Join Date
    May 2010
    Posts
    4,632
    You might want to write to stderr instead of stdout, stderr is not buffered, where stdout is buffered.

    Jim

  6. #6
    Registered User
    Join Date
    May 2004
    Posts
    73
    @jim: wouldn't that be slower? I need both speed and non-buffering.

    @Raigne: that's a good idea. i'll give it a shot, thanks!

  7. #7
    Registered User
    Join Date
    May 2010
    Posts
    4,632
    It should not be much slower and once you have your problems fixed you can always revert to stdout.

    Edit: You should also be running the program in debug mode so when it does crash you can see where it is crashing and the state of the local variables.

    Jim
    Last edited by jimblumberg; 04-25-2011 at 04:52 PM.

  8. #8
    Frequently Quite Prolix dwks's Avatar
    Join Date
    Apr 2005
    Location
    Canada
    Posts
    8,057
    The name for mounting a filesystem in your ram is a ramdisk. If you google e.g. "ramdisk linux" you can find out how to create them, it's not too hard if you have a little bit of experience with Linux. (Never tried on other operating systems.)

    You can also, as Raigne suggested, use another process to collect the debugging messages. If you're on a UNIX-like system you could use memory mapping to allow both processes to read/write the same memory. Memory mapping is extremely fast and when one program dies the other can keep reading from it.

    You may be interested in this: Multithreaded app logging
    dwk

    Seek and ye shall find. quaere et invenies.

    "Simplicity does not precede complexity, but follows it." -- Alan Perlis
    "Testing can only prove the presence of bugs, not their absence." -- Edsger Dijkstra
    "The only real mistake is the one from which we learn nothing." -- John Powell


    Other boards: DaniWeb, TPS
    Unofficial Wiki FAQ: cpwiki.sf.net

    My website: http://dwks.theprogrammingsite.com/
    Projects: codeform, xuni, atlantis, nort, etc.

  9. #9
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    Is printing messages really the only way you can isolate the crashing line of code? You mention real-time, so maybe you have some constraints, but it seems like you ought to be able to just use a debugger. (By the way, writing to disk is a great way to cause your "real time" program to no longer be real time)

    If you really do need to log massive amount of information, then you can't get away from the fact that this is going to take some time. I'd be surprised if every call to stdio resulted in a write to the physical disk. The OS is standing between you and the disk and it should be buffering stuff for you, even if you're flushing every line of output.

    It may be useful to know that printf() has a lot more overhead than fputs(). If you don't need to format things (e.g. you're just printing a simple message like "I made it"), then calling fputs() instead of printf() will save some time.

    Another alternative is to install a crash handler, and from within the crash handler, flush your output streams. This will let you run the streams fully buffered. It's not foolproof, because once your program has crashed (or is in the process of crashing), pretty much anything could be corrupted, but you could try it.

    EDIT: Multi-threaded C runtimes use locking to serialize calls to the stdio routines. If your program is not threaded, this is just a big waste of time. Depending on your runtime, there may be "unlocked" versions of the stdio calls available which should run faster (but will do wrong things if called from multiple threads simultaneously)
    Last edited by brewbuck; 04-25-2011 at 05:06 PM.
    Code:
    //try
    //{
    	if (a) do { f( b); } while(1);
    	else   do { f(!b); } while(1);
    //}

  10. #10
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,659
    This seems to be basically the same question as Making different threads use different stdout's
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  11. #11
    Algorithm Dissector iMalc's Avatar
    Join Date
    Dec 2005
    Location
    New Zealand
    Posts
    6,318
    You could set up a logging system using shared memory to log the errors to shared memory so that they are accessable to a separate application.
    We had considered this for our main product not so long ago, but it never got past prototype stage before a method of bringing the existing method of logging up to sufficient speed was found. No I do not have and could not share code for it.
    My homepage
    Advice: Take only as directed - If symptoms persist, please see your debugger

    Linus Torvalds: "But it clearly is the only right way. The fact that everybody else does it some other way only means that they are wrong"

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Size of stdout
    By omGeeK in forum C Programming
    Replies: 1
    Last Post: 03-20-2011, 11:28 AM
  2. stdout with JNI
    By GoLuM83 in forum C Programming
    Replies: 4
    Last Post: 12-14-2006, 01:27 PM
  3. using stdout
    By rebok in forum C Programming
    Replies: 4
    Last Post: 11-05-2004, 06:19 PM
  4. redirecting stdout
    By gregulator in forum C Programming
    Replies: 2
    Last Post: 04-22-2004, 10:07 AM
  5. stdout
    By Unregistered in forum C Programming
    Replies: 1
    Last Post: 03-25-2002, 05:43 AM