Thread: Loading large files with c++

  1. #1

    Loading large files with c++

    I am writing a log parser which does what i want it to do (not some other parser which never does what i want)

    I was wondering, i am going to load the file into a llist (a record for each line) so i can perform regexps on the text - what would be better, loading the whole file (200+mbs) into the llist or breaking it up into bits ?

  2. #2
    Registered User
    Join Date
    Aug 2001
    Depends on how much memory you have available. With a logfile that large, you might want to do it one line at a time; while you will incur some overhead with multiple disk reads, I imagine that would be pretty small in comparison to the overhead of having that much data resident in memory. Besides, at 200 MB+, it's going to take a while no matter which way you do it.


Popular pages Recent additions subscribe to a feed

Similar Threads

  1. fwrite for large files
    By ccoolgoose in forum C Programming
    Replies: 11
    Last Post: 06-01-2009, 09:38 PM
  2. Reading large complicated data files
    By dodzy in forum C Programming
    Replies: 16
    Last Post: 05-17-2006, 04:57 PM
  3. I Need To Know Some Things That I Can Put Into A Batch File
    By TheRealNapster in forum A Brief History of
    Replies: 1
    Last Post: 10-20-2003, 08:12 PM
  4. Replies: 0
    Last Post: 07-12-2002, 01:40 PM
  5. Loading all files in a directory
    By steve_i83 in forum Windows Programming
    Replies: 2
    Last Post: 08-26-2001, 02:18 PM