Loading large files with c++

This is a discussion on Loading large files with c++ within the Linux Programming forums, part of the Platform Specific Boards category; I am writing a log parser which does what i want it to do (not some other parser which never ...

  1. #1
    Unregistered
    Guest

    Loading large files with c++

    I am writing a log parser which does what i want it to do (not some other parser which never does what i want)

    I was wondering, i am going to load the file into a llist (a record for each line) so i can perform regexps on the text - what would be better, loading the whole file (200+mbs) into the llist or breaking it up into bits ?

  2. #2
    Registered User
    Join Date
    Aug 2001
    Posts
    202
    Depends on how much memory you have available. With a logfile that large, you might want to do it one line at a time; while you will incur some overhead with multiple disk reads, I imagine that would be pretty small in comparison to the overhead of having that much data resident in memory. Besides, at 200 MB+, it's going to take a while no matter which way you do it.

    starX
    www.axisoftime.com

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. fwrite for large files
    By ccoolgoose in forum C Programming
    Replies: 11
    Last Post: 06-01-2009, 10:38 PM
  2. Reading large complicated data files
    By dodzy in forum C Programming
    Replies: 16
    Last Post: 05-17-2006, 05:57 PM
  3. I Need To Know Some Things That I Can Put Into A Batch File
    By TheRealNapster in forum A Brief History of Cprogramming.com
    Replies: 1
    Last Post: 10-20-2003, 09:12 PM
  4. Replies: 0
    Last Post: 07-12-2002, 02:40 PM
  5. Loading all files in a directory
    By steve_i83 in forum Windows Programming
    Replies: 2
    Last Post: 08-26-2001, 03:18 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21