Freak data loss

[idelovski]

My clients have experienced last friday strange loss of data from my application.

The app works with almost 200 files, but at any given moment only 20 are actually open. So, they were working in it, at some point there was a surge in power, the lights blinked once, the screen turned off, but computer continued to work. As they tried to save, application reported an error and on relaunch all the documents were missing.

They called me and I connected over remote access and this is what I found: 36 files (that should have had content from the documents they entered since March) were 4K in size (i.e. empty, only few bytes in use, all zeroes). These files had all creation date March 1st and modification date Apr 15, 17:20. The rest of the files, never used as it seems, had both creation and modification at Mach 1st. Those files that were never used were 4K in size too, but their content was not zeroed out. They had proper header content at the beginning.

There is a file that keeps record of changes/transactions, but that file is empy too.

!?!#?%!@!

What could have happened? I have the same application on several hundred other places and never, ever anything similar to that ever occured. On power loss, maybe one or two records would disappear sometimes, most of the times everything would be just fine. This makes no sense to me. Or maybe it should? I have this app on Mac OSX and Windows and I never experienced anything similar to this.

Is it something related to hardware? Maybe some strange drive configuration, something in OS, some strange settings (XP SP2). They don't have a fresh backup, of course. Bad luck?

[Mario F.]

It would help if you described the file handling steps you use as well as the opening method(s). As well as the language.

From your description however, I'm guessing you are using buffers. And probably you were caught in freak accident in which the buffer was zeroed out due to the power surge and then flushed to disk, when users tried to save or as part of its normal operation (buffer became full, your code uses explicit flush, etc...).

[idelovski]

It would help if you described the file handling steps you use as well as the opening method(s). As well as the language.

Straight C, files are opened with regular CreateFile() call, after every batch, transaction files involved in it are flushed, but this usually involves from two to maybe four or five files at any given scenario that I can think of. There is no need to flush 30 files at the same time.

Other calls that are used: WriteFile(), ReadFile() and CloseHandle().

[CommonTater]

Straight C, files are opened with regular CreateFile() call, after every batch, transaction files involved in it are flushed, but this usually involves from two to maybe four or five files at any given scenario that I can think of. There is no need to flush 30 files at the same time.

Other calls that are used: WriteFile(), ReadFile() and CloseHandle().

I take it these are Windows calls... Check the Options in your CreateFile() calls... if you do not have either FILE_FLAG_WRITE_THROUGH or FILE_FLAG_NO_BUFFERING set, the data will buffer in memory. If the power surge scrambles memory... guess what?

There's also a possibility that although the computer kept running it may not have been running in an organized manner and simply truncated the files.

Frankly if your software tends to do this, it might be a good idea to create backup files (they can be hidden) just in case...
Or you could have a sit down with your customer and talk about uninterruptable power supplies, which should give them enough time to shut down properly.

[cpjust]

Or you could have a sit down with your customer and talk about uninterruptable power supplies, which should give them enough time to shut down properly.

And also about doing regular backups.

[idelovski]

Everybody knows they should make backups. Nothing new in that. My real question was if this was technically possible, or better yet, if anyone here experienced something similar?

[idelovski]

Well, few days ago I added one little extra information here, but it got lost as cboard had database problem on its own. Ironic or not, here it is again - my clients mailed me that they were not using defragmentation on their computers. It was something else. I just hope they'll never lose data again and i sincerely hope cboard never loses posts again.

[CommonTater]

Well, few days ago I added one little extra information here, but it got lost as cboard had database problem on its own. Ironic or not, here it is again - my clients mailed me that they were not using defragmentation on their computers. It was something else. I just hope they'll never lose data again and i sincerely hope cboard never loses posts again.

Well, then the only real suggestion I can offer is a software update...
1) keep hidden backups of your files that can be used to restore to the last file close...
2) if you are writing files "whole cloth" to disk, write to a temp file and test for success. If it worked delete the original and rename the replacement.

There are many strategies that can prevent this kind of thing.
The big pain is identifying the cause... which you may never do.

[idelovski]

As long as this is a single case I'm ok. If it becomes epidemic, then I'm in trouble.