Printable View
Hi,
I am working on a problem has to do with manipulating large files. Essentially, the program will open up any give files and append, insert, and delete data from the file at random position.
What is the most efficient solution?
I have tried memory mapping, but it does not work as expect. Memory mapping works only when the program change a byte in the file. It does not work for inserting and delete bytes from the file at random position.
Thanks,
Kuphryn
You can't insert data into a file without rewriting the entire file.
> What is the most efficient solution?
Maintain a separate index file to track insert, append, delete operations. Manipulating the ordering of the index file will be much more efficient than manipulating the data files.
From the point of view of the data file, insert and append are either
- overwrite of any deleted record
- append to file if there are no deleted record.
Nothing immediately happens to the data file on deletion, it's simply a marker in the index file.
If, at the end of all your operations, you want to rewrite a normalised data file (which can be accesses without the index), then that's easy to do as well.
Okay. Thanks.
Kuphryn