Loading large files with c++ [Archive] - C Board

PDA

View Full Version : Loading large files with c++


Unregistered
03-14-2002, 10:55 AM
I am writing a log parser which does what i want it to do (not some other parser which never does what i want)

I was wondering, i am going to load the file into a llist (a record for each line) so i can perform regexps on the text - what would be better, loading the whole file (200+mbs) into the llist or breaking it up into bits ?

starX
03-18-2002, 12:26 AM
Depends on how much memory you have available. With a logfile that large, you might want to do it one line at a time; while you will incur some overhead with multiple disk reads, I imagine that would be pretty small in comparison to the overhead of having that much data resident in memory. Besides, at 200 MB+, it's going to take a while no matter which way you do it.

starX
www.axisoftime.com