Thread: What's the fastest way to write a file?

I imagine that functions which do not do any type parsing would be faster than those which do, such as fprintf(). So fputs() might be a good idea. Though cout.operator<<(const char *) might be just as fast.

Those are just the standard C and C++ functions. I haven't considered the Windows-specific ones.

How important is this speed to your program? Are you willing to sacrifice portablity for a few milliseconds' execution speed? Why do you need fast I/O?

As already said, the fastest way is to use system calls.

WriteFile() for Windows. write() for Unix/Linux.

If there is not such a good function, I have to consider about printing the data into a string, and printing the string out when the program ends. Will that save a lot of time?

It might save some time. Would it be difficult to program?

It sounds like your program might be taking a lot of time to open files over and over again. You might be able to save lots of time by only opening the file once. Assuming that you're only writing to one file.

How much data are you writing, and how long does your program take to run?

Sorry for late response, I have just been too absorbed in programming.

It happens to all of us.

I need to run the program for thousands of times, and one run may takes around 40 mins. The output file size, as I could estimate now, is roughly 20~200k.

so any suggestions for optimization?

Wow, yes, then definitely go for whatever speed increases you can get away with.

But one run, generating a 20-200KB file, takes 40 minutes? You must either have a very slow computer, or the program must do a lot of extra work. How much of that time is actually spent on file I/O?

I suggest using a profiler to see which portions of your code are taking excessive amounts of time. With GCC, you can use gprof.

Code:

$ gcc -pg hello.c -o hello.exe
$ ./hello.exe
$ gprof hello.exe > gprof_output1

You should be able to figure out the output from gprof. If not, do some searching or post it here.

Originally Posted by pingpangpang

Sorry for late response, I have just been too absorbed in programming.

I need to run the program for thousands of times, and one run may takes around 40 mins. The output file size, as I could estimate now, is roughly 20~200k.

so any suggestions for optimization?

Taking that long has nothing to do with the method used to write to file.
The reality is, you have no idea why it takes so long and are just picking one thing and asking how to speed up that one thing. That wont get you anywhere.

You would have to make available a lot of the code that is so slow. Right now we can't do squat to help you.

> The output file size, as I could estimate now, is roughly 20~200k.
If it takes 40 minutes to produce 200K of output, then I/O is NOT your problem.

Try writing a program which just generates 200K of output using cout or printf, and see how long it takes. You'll barely be able to get your finger off the return key before it's done.

Which compiler are you using?

If it's gcc based, then look into something called 'gprof', which is a profiler.
This will tell you exactly where the code is spending the most time, and those are the areas you need to focus on.

I need to run the program for thousands of times, and one run may takes around 40 mins.

hmm thousands = at least 2

2000 *40min / 60min/hr / 24h/day = wtf! 55.55 days, hope you got a snickers :-)