The best way to delete a record from a file?

Hi ALL,

Lets say I have a file name "cars.bin".
Each record in the file is a struct (made from int, char [20], int).
Lets say I have 10000 records,
struct 1, struct 2, struct 3, struct ....., struct 10000.

Now, If I want to delete record #400, how should I do it??
The only thing I could think of is to:
1. Create a new file
2. Copy all the records from old file to new file (except of the #400)
3. Delete old file
4. Rename new file to old file name.

This seem like a slow poor solution, is there any other way to do it?

Many thanks
Salvador

Since all the structures are the same size, you could open the file, seek to structure 10,000, copy it to structure #400 position, then resize the file to be smaller by one structure.

This would be very fast.

Todd

You could have a marker that says "deleted" in each record [e.g. the first int being a negative number, just as an example]. Then when you are reading the records into your application, check the "deleted" marker and skip those that are deleted.

If you want to, you could have a "compress" mechanism, so that when you exit the applicaiton, if you counted more than X% deleted records, THEN you rewrite the whole file without the deleted records.

Or, if you want to re-use deleted records, you could keep a linked list of records that have been deleted, e.g. using the second int to indicate which record is the next free one, so when you add new records, you overwrite the first record in that list.

--
Mats

hummm... I probably would copy struct401 to struct400 then struct402 to struct140 and so on...
I have no much experience with handling files but if the structs were linked with pointers is could be easier and faster. Unfurtunately in the case of files i don't think this is like this. :(

Quote:

---

Originally Posted by Todd Burch

Since all the structures are the same size, you could open the file, seek to structure 10,000, copy it to structure #400 position, then resize the file to be smaller by one structure.

This would be very fast.

Todd

---

Although resizing files is only supported by some OS's.

--
Mats

This dilemma is why databases were invented.

Yes, and most databases don't ACTUALLY delete things, they just mark it as "deleted", and re-use as needed.

--
Mats

Cheers!

How do I resize a file?

Quote:

---

Since all the structures are the same size, you could open the file, seek to structure 10,000, copy it to structure #400 position, then resize the file to be smaller by one structure.

This would be very fast.

Todd

---

I don't get exactly how this would work. Would you not need to move the structure instead of copy it?

Say each entry is a line of text, then to actually delete it you would have to copy all the following text over it, then resize the file. Or am I missing something here?

Personally I like matsp's suggestion, which sounds easy to do.

On unix, you would use truncate().

Quote:

---

Originally Posted by mike_g

I don't get exactly how this would work. Would you not need to move the structure instead of copy it?

Say each entry is a line of text, then to actually delete it you would have to copy all the following text over it, then resize the file. Or am I missing something here?

Personally I like matsp's suggestion, which sounds easy to do.

---

move==copy. Same thing.

Oh yeah, actually I get it now (i think). You could just swap the entry to be deleted with the last entry then truncate it. So yeah both ways would be fast :)

I will adapt mat's suggestion too if the file is sorted. (thank you)
and Todd's if not (thank you too)
I still don't understand how can I resize a file size on Windows....

On Windows, _chsize().

Quote:

---

Originally Posted by mike_g

Oh yeah, actually I get it now (i think). You could just swap the entry to be deleted with the last entry then truncate it. So yeah both ways would be fast :)

---

But then the records would be out-of-order. Record 1 ... 400, 900, 401 ... 899.
The best and easiest way is to just use a flag to indicate if the record is valid or deleted. You can then write new data there later or just move data down if the records needs to be in order.
Again, if they don't need to be in order, mike_g's suggestion works, as well.

Yeah I realized that afterwards, but dident bother posting about it. So, after all, I still don't get how 'moving' is meant to work the same way as 'copying' here ???

Just copy and truncate the end of the file and you've basically "moved" the data. Or you can just zero it out. Either way.

Yeah, but then you have two copies and by truncating you just remove the copy you just made o_0

The way I see it is that to physically 'move' the data you need to remove the space where the original data was stored at. The only way I know how to do this is by overwriting the rest of the file from the place where the data was removed.

Quote:

---

Originally Posted by mike_g

Yeah, but then you have two copies and by truncating you just remove the copy you just made o_0

The way I see it is that to physically 'move' the data you need to remove the space where the original data was stored at. The only way I know how to do this is by overwriting the rest of the file from the place where the data was removed.

---

If the old data is past the end of file and therefore not accessible, it is in essence, deleted.

Yeah I get that part, but it does not explain the process of moving it.

On the contrary, it does.
Let's say we have 5 records, 1, 2, 3, 4 and 5. We delete record 3. So we copy the from record 4 and 5 downwards, so we overwrite record 3. So we copy record 4 and 5 and it thus becomes:
1, 2, 4, 5, 5. Then we truncate the end to remove the extra 5 and it becomes 1, 2, 4, 5. Makes sense?

Lol, well I knew that too! I just explained it as the only way i saw this being possible. With a large database thats getting updated all the time thats going to be a hell of a lot slower than just marking a character to signal an entry as deleted. Then you would only need to rewrite the data once in a while instead.

Indeed, and that's why the delete flag is preferable!

Who said they needed to be in order? Who said they were in order to start with?

None, but then again, no one said they do not need to.
So it's just to pick an applicable solution.

For most things in a computer: Move = Copy + Delete
Unless you want to physically cut a piece of the hard drive surface away and glue it onto a new location on the platter; then it would truly be considered a Move. ;)