Search:

Okay. Once I moved to a single mapped file, I think I changed the bottleneck. I now try commenting out the "write out to files" portion of the operation, and I scale much better (to 20 or 30...

Thanks for the response. I can try your load test tomorrow.

Real files may be as small as ~1MB, or as large as gigabytes. In the real program doing this, the code will need to run on 32-bit and...

thanks for the reply. From looking at the numbers in the table, we can see that I can get down at least to about the half-second range for the computation. By comparing that to the ~10s and ~5s...

Hey, thanks for the reply. I think you may have something there. If my chunks are typically in the kilobytes, is this still a likely candidate for the problem?

Also, I should note, when I run...

Hi all -

Here's some context:

I have files of some large-ish size consisting of chunks. I need to
load each chunk, do some work (call it munging) on it, and write the
munged chunk to a file....