Don't be hesitant to suggest somethingNot to be argumentative, but...
The files I am reading are around 5mb - 40mb, however, I want to read them all in together using threads. I was thinking because of this that doing a giant memory write would be a bad choice.Define "quite large"... especially in a machine with 8 or 16gb of memory...
I commonly load 10 and 15 megabyte files into memory in one pop without problems.