stdout -> ram?

Hey all, I have an interesting predicament. In my debugging, I need to print out a massive number of debug statements, unbuffered (so i know where exactly im crashing), on every frame of my real-time program.

These printouts are slowing me way, way down. I'd imagine the reason stdio is so slow is that it has to write to disk. Is there any way to mount stdio into the ram? I've seen other things mount pieces of the hard drive onto ram for super-fast access, but I haven't a clue how to go about it for stdout.

I am not sure I understand your question completely. If I were to do it, I would build a vector containing the debug information/statements, then periodically write to files ( say every 1000 elements ). Then have an functions that is forced to run at program termination ( say a class destructor ) write the remainder of debug information. This would minimize overhead during runtime, but by what amount im not sure.

You may even optimize more by combining the debug statements into a single write call. ( batching )

I can't be buffering things like that though. My program crashes, and the last few statements are vital to figuring out where the crash happened.

Have you considered using a second process to handle the debug information, and just send the information from the debugging application. So, even after the crash the information is not lost?

You might want to write to stderr instead of stdout, stderr is not buffered, where stdout is buffered.

Jim

@jim: wouldn't that be slower? I need both speed and non-buffering.

@Raigne: that's a good idea. i'll give it a shot, thanks!

It should not be much slower and once you have your problems fixed you can always revert to stdout.

Edit: You should also be running the program in debug mode so when it does crash you can see where it is crashing and the state of the local variables.

Jim

The name for mounting a filesystem in your ram is a ramdisk. If you google e.g. "ramdisk linux" you can find out how to create them, it's not too hard if you have a little bit of experience with Linux. (Never tried on other operating systems.)

You can also, as Raigne suggested, use another process to collect the debugging messages. If you're on a UNIX-like system you could use memory mapping to allow both processes to read/write the same memory. Memory mapping is extremely fast and when one program dies the other can keep reading from it.

You may be interested in this: http://cboard.cprogramming.com/linux...p-logging.html

Is printing messages really the only way you can isolate the crashing line of code? You mention real-time, so maybe you have some constraints, but it seems like you ought to be able to just use a debugger. (By the way, writing to disk is a great way to cause your "real time" program to no longer be real time)

If you really do need to log massive amount of information, then you can't get away from the fact that this is going to take some time. I'd be surprised if every call to stdio resulted in a write to the physical disk. The OS is standing between you and the disk and it should be buffering stuff for you, even if you're flushing every line of output.

It may be useful to know that printf() has a lot more overhead than fputs(). If you don't need to format things (e.g. you're just printing a simple message like "I made it"), then calling fputs() instead of printf() will save some time.

Another alternative is to install a crash handler, and from within the crash handler, flush your output streams. This will let you run the streams fully buffered. It's not foolproof, because once your program has crashed (or is in the process of crashing), pretty much anything could be corrupted, but you could try it.

EDIT: Multi-threaded C runtimes use locking to serialize calls to the stdio routines. If your program is not threaded, this is just a big waste of time. Depending on your runtime, there may be "unlocked" versions of the stdio calls available which should run faster (but will do wrong things if called from multiple threads simultaneously)

This seems to be basically the same question as http://cboard.cprogramming.com/cplus...t-stdouts.html

You could set up a logging system using shared memory to log the errors to shared memory so that they are accessable to a separate application.
We had considered this for our main product not so long ago, but it never got past prototype stage before a method of bringing the existing method of logging up to sufficient speed was found. No I do not have and could not share code for it.