Thread: I/O Rates - MBs per second - what's good?

Threaded View

Previous Post Previous Post   Next Post Next Post
  1. #1
    Jack of many languages Dino's Avatar
    Join Date
    Nov 2007
    Location
    Chappell Hill, Texas
    Posts
    2,337

    I/O Rates - MBs per second - what's good?

    I've written a C++ program to convert data from one data format to another. My benchmark program reads 136MB of binary data, converts each record and writes it to a text file.

    My first cut ran for 42 seconds. Not too impressive. I reviewed my logic and realized I was passing whole structures between high-use functions instead of pointers (or references). I changed that and the process time dropped to 21 seconds. Better.

    To read the file, I was using istream.read(), but I was writing each line (of text) with ostream << text << endl. I rewrote it to buffer the data up and then do a ostream.write() and now the elapsed time is down to 11 seconds. For the numbers of records I'm processing, these are the rates:

    Input Records = 1,066,766
    Input file size: 136 MB
    Output text file size: 184.1MB

    Code:
    elapsed: 42 seconds 
    rate: 136MB / 42 = 3.238 MB per second 
    
    elapsed: 21 seconds 
    rate: 136 MB / 21 = 6.476 MB  per second 
    
    elapsed: 12 seconds
    rate: 136 MB / 11 = 12.363 MB per second.
    This is single threading.

    If I remove the conversion logic, and just read the binary data, don't touch or move it, don't also write a 184 MB text file, and then write it back out from the same input buffer, it takes 2 seconds, for an I/O rate of 68MB per second.

    Would you say the 12 MB per second rate I've reached is reasonable?

    Thanks, Todd
    Last edited by Dino; 01-14-2008 at 09:41 AM. Reason: typo

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Good resources for maths and electronics
    By nickname_changed in forum A Brief History of Cprogramming.com
    Replies: 8
    Last Post: 12-22-2004, 04:23 PM
  2. Good C++ books for a begginer
    By Rare177 in forum C++ Programming
    Replies: 13
    Last Post: 06-22-2004, 04:30 PM
  3. Good sound storage method
    By VirtualAce in forum Game Programming
    Replies: 1
    Last Post: 10-14-2003, 05:18 AM
  4. linked list recursive function spaghetti
    By ... in forum C++ Programming
    Replies: 4
    Last Post: 09-02-2003, 02:53 PM
  5. Using Windows File I/O functions
    By SMurf in forum Windows Programming
    Replies: 2
    Last Post: 10-05-2001, 05:36 AM