Faster Output to Disk

This is a discussion on Faster Output to Disk within the C++ Programming forums, part of the General Programming Boards category; Right now I have a program that loops, and if given the right conditions writes to disk. Because of this ...

  1. #1
    Registered User
    Join Date
    Sep 2011
    Posts
    117

    Faster Output to Disk

    Right now I have a program that loops, and if given the right conditions writes to disk.

    Because of this it write a lot of small files, and currently I am naming the files with std::sprintf and writing to that file with std:fstream.

    Without calling this write to disk function it takes 4 seconds to finish, but with it (written on a ram disk) it takes 24 seconds, 6 times as long!

    While this doesn't matter that much in certain situations it runs much longer (2 minutes without writing) and could be much longer.

    Would there be any other faster way in order to make multiple small files?

    Thanks!

    Code:
            char filename[100];
    
            std::sprintf(filename, "%05.0lf.txt", numberofFiles);
            std::ofstream ofs(filename);
    
            for(int o = 0; o<100; o++)
            {
    
                if( outputClean[o][0][0] != '\0')
                    ofs << "abcde    " << outputClean[o][1] << "  " << outputClean[o][0][0] << endl;
            }
    
            for(int o = 0; o < 100; o++)
            {
                if(outputClean[o][0][0] != '\0')
                {
                    ofs << "fghij    ";
                    for(int i = 1; i < 10; i++)
                    {
                        if(outputClean[o][i][0] != '0')
                            ofs << outputClean[o][i] << "    ";
                    }
    
                    ofs << "\n";
                }
            }
    
            ofs << "END";
    
            numberofFiles++;
    My Ctrl+S addiction gets in the way when using Code Blocks...

  2. #2
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,484
    If the desired end result is a lot of small files, then the answer is no.

    Sure, you could speed up THIS program by say writing one large file, but then you would need to spend time waiting for another program to split the large file up into a number of small files.

    If you have no care at all about portability, then use the lowest level platform API calls for doing the same things (open, write, close).
    It might buy you some improvement, but chances are it will still be closer to 24 than 4.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Why is one faster than the other?
    By phlook in forum C Programming
    Replies: 6
    Last Post: 04-02-2009, 01:27 PM
  2. Disk failure in 3 disk RAID0 & 5?
    By cpjust in forum Tech Board
    Replies: 12
    Last Post: 12-22-2008, 09:09 AM
  3. Which is faster?
    By Yarin in forum C++ Programming
    Replies: 4
    Last Post: 08-22-2008, 05:20 PM
  4. Threads to keep the CPU faster than the disk?
    By matthew180 in forum C Programming
    Replies: 4
    Last Post: 06-06-2007, 03:23 PM
  5. how much faster is ASM output vs. printf(),cout,echo,etc...
    By Unregistered in forum A Brief History of Cprogramming.com
    Replies: 5
    Last Post: 06-04-2002, 04:54 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21