Undelete utilities

To what extent will they work?
My IDE (CodeBlocks) has just eaten my 4000+ line C file, reducing it to absolutely nothing (officially 0 bytes). From what I've heard on the subject, deleting a file from the FS is manageable for undelete utilities, however if you write all zeroes to the file, the chance of retreiving it is slim to none. Since this is essentially what happened in my case, I'm thinking my prospects are very slim here. I've already tried ntfsundelete.com, which seemed proficient at finding and undeleting every other file, including files I had deleted years ago, except the file in question.
So is it worth it to spend the money on another utility, or do they essentially all yield the same results? What is the prognosis for my situation here?

Trashign it and rewriting it are two different things. Under what circumstances did CodeBlock murder this file? I'd report it fast if it was a recent revision. I don't know how to help, but would like to hear the diagnosis from the team on the issue.

It very much depends on how the tool works: Many times, the new file is created as a temporary file, then the old file renamed to make a backup, and finally the temporary file is renamed to the original name. So edit foo.c -> save to temp.aaa -> rename foo.c to foo.bak -> rename temp.aaa to foo.c.

But if the editor renames the original file first and then writes the new file as "foo.c" and it's got some rubbish instead of your file-content, then that's pretty much it.

Undelete can only fix when a file is deleted, not when the existing content is replaced by something else in the existing place.

--
Mats

Quote:

---

however if you write all zeroes to the file, the chance of retreiving it is slim to none

---

Unless you have some of those nifty tools used in computer forensics, or if it has been repeatedly rewritten with garbage and then zeroed, or so I have heard. Unfortunately, I have no idea how to get them or how exactly do they work, though my guess is that they are just sensitive enough to read traces of the overwritten data and guess the previous data from those traces.

It will not help you now, but perhaps you should have used version control and taken backups?

I was wondering about that too, because I know of the free ones that quasi-work and the one's you pay for (perhaps better?) and then I know there are the ones that only FBI crime labs get, which I would love to have right about now.

The guys over at the codeblocks forums say that the file writing process works as indigo suggested, but after I posed some doubts, there may have been some old code that instead directly overwrote the file. You can follow this topic here. I was using a Nightly build, released a couple of months ago. As they suggested, I tried looking for the temp file used as a backup between file writing (.cbTemp or .backup) but neither were found either in tact or via the undelete utility.

I save often, and since I don't expect my IDE to eat my file (crash maybe, but not empty the file completely) I tend not to keep so many "backups". I had a standing backup from the early versions of my program as I was adding new features in case they did not work and I needed to revert back, so unfortunately the only backup I have is so old it is virtually unusable...

Quote:

---

I save often, and since I don't expect my IDE to eat my file (crash maybe, but not empty the file completely) I tend not to keep so many "backups". I had a standing backup from the early versions of my program as I was adding new features in case they did not work and I needed to revert back, so unfortunately the only backup I have is so old it is virtually unusable...

---

Even if your IDE does not "eat" your file, you might accidentally do it yourself :p In fact, a version control system is especially handy when you are adding new features as it makes reverting to older (and working) versions easier, and you could save/commit/record versions often and save space by not having a standalone backup copy each time you want to save the current version.

Quote:

---

Originally Posted by laserlight

Even if your IDE does not "eat" your file, you might accidentally do it yourself :p In fact, a version control system is especially handy when you are adding new features as it makes reverting to older (and working) versions easier, and you could save/commit/record versions often and save space by not having a standalone backup copy each time you want to save the current version.

---

Not to mention that you could revert only PARTS of a change (ok, so you probably need revert all, then get the change out as a diff and then apply parts back in as a patch or such).

--
Mats

Quote:

---

Originally Posted by matsp

Not to mention that you could revert only PARTS of a change (ok, so you probably need revert all, then get the change out as a diff and then apply parts back in as a patch or such).

---

rcs, although considered "primitive" by most people, makes it easy to revert only certain parts of a change.

cvs was originally built on top of rcs, by the way.

From my brief encounters with cvs in the past, I came away thinking you needed a file server for this or ftp of some sort, is this correct? I guess its worth learning how to set it up correctly, though, at the moment, I haven't much reason to...

Edit: I have another question:
If I can obtain a copy of the executable from a recent version of my application, is it worth attempting to reverse engineer it? Exactly how accurate to the original can I expect the resulting code to be?

Quote:

---

Originally Posted by @nthony

From my brief encounters with cvs in the past, I came away thinking you needed a file server for this or ftp of some sort, is this correct? I guess its worth learning how to set it up correctly, though, at the moment, I haven't much reason to...

---

If you can ssh to a machine, you can run CVS from it.

Quote:

---

Originally Posted by @nthony

If I can obtain a copy of the executable from a recent version of my application, is it worth attempting to reverse engineer it? Exactly how accurate to the original can I expect the resulting code to be?

---

what optimizations did you compile it with? did you use the debug setting? if you did, the code should be mostly there in the exe.

You won't be able to get back to C from your EXE.

If your file has been trashed you won't be able to recover it if you have done any other operations on your drive.

Essentially the data that was your file may still be on the disk per se, but the filesystem says your file is now 0 bytes. The way to get past this is to remove the end of file descriptor but you will need to learn up on NTFS to do this. But since your file is now zero bytes all of the other data past the end of file descriptor can be overwritten by other processes since it has been declared as 'available'.

I give you a 10% chance of getting your file back. Undelete only fixes files that were deleted at one time - not files that had their data scrambled or removed.

And why do you have 4000 lines of code in one file?

That was all of my front end code (GUI and User configuration), I have another 4000+ line file with all of my back-end code, which luckily is still intact, but is rendered useless without the front-end to drive it. Unfortunately I did compile with full optimizations on and no debug code, so I guess reversing the .exe will be of little use then?

Indeed, I was thinking about what would happen if I simply "read" past what windows said was the end of the file, but I would not know how to do this. Also if some files have been created since then, would that essentially garauntee that the space free'd up by the emptying of my file is no probably taken again? Are there any utilities that can alter NTFS file descriptors as you say?

diskprobe
This will allow you to scan the disk sector by sector for your file fragments. There's even a search mode if you can remember fragments of your code. With any luck, most of the lost file will be in one or two consecutive runs of sectors on disk. Once found, you can copy the sectors to a real file on another disk (don't write to the one you're diagnosing).

You should be able to find the program buried somewhere on your XP installation CD.

You should be aware that the more you've used this disk since the files were lost, the less your chance of recovery.

As for version control, I've used http://tortoisesvn.tigris.org/ recently. It integrates nicely into windows explorer (lots of nice right-click context menu functionality).

The reason for the low chances of recovery are that the disk is always being accessed while inside of Windows. Most of the swap file is probably in the same place on the disk every time - hence why Windows tends to destroy hard drives where the swap file is. But Salem is right in that the more you use your computer the more other programs can use those sectors that contain your file data.

You will need to do a recovery immediately. Back in the DOS days this wasn't such an issue because only 1 task was running (save for TSRs) and the disk was not accessed nearly as much. Inside of Windows there are a plethora of tasks and modules running all asking for access to the drive at one point or another. If Windows has granted any of these processes access to the sectors that were your file, you will lose the data.

You don't need a fileserver for CVS/SVN/whatever. You can run it locally. Or you can use the direct filesystem access that both provide. It's all pretty easy, and absolutely wonderful. Nowadays, whenever I start a project, the first thing I do is creating an SVN repository for it.

Though you might want to try git as an alternative.

Quote:

---

Though you might want to try git as an alternative.

---

Be warned that there is less Windows support for git compared to CVS or Subversion (or Bazaar, for that matter).

How new is your hard drive? How full is it? When did you last defragment it?

If you have an older drive or it's very nearly full or very fragmented, a problem could have occured with your hardware, instead of with Code::Blocks. Files become crosslinked etc quite easily when they're in 3,000 fragments . . . that happened to me recently to an entire folder, though fortunately I had a very recent backup of the files in question. I've deleted 8% of the files on the hard drive and had no further problems. I should probably defragment it sooner or later too.

Have you done anything unusual recently to your hard drive? With an old version of a live Linux CD, any file I accessed on the partition tended to get crosslinked or truncated. And any folder I created would be perfectly visible in the Linux, even across reboots, but completely invisible in Linux.

Or, of course, it could just have been Code::Blocks messing up.

The longer you leave it, the less likely you'll be able to recover the file. If you tried an undelete utility immediately after you discovered that it was gone, you might have had a chance at recovering it; but by now, if you've done other work etc then it's almost certainly gone.

Quote:

---

Originally Posted by @nthony

To what extent will they work?
My IDE (CodeBlocks) has just eaten my 4000+ line C file, reducing it to absolutely nothing (officially 0 bytes). From what I've heard on the subject, deleting a file from the FS is manageable for undelete utilities, however if you write all zeroes to the file, the chance of retreiving it is slim to none. Since this is essentially what happened in my case, I'm thinking my prospects are very slim here. I've already tried ntfsundelete.com, which seemed proficient at finding and undeleting every other file, including files I had deleted years ago, except the file in question.
So is it worth it to spend the money on another utility, or do they essentially all yield the same results? What is the prognosis for my situation here?

---

I'd go download a Knoppix image and boot off the CD. Then grep the hard drive for some key string which you know occurs in your file. With any luck, the position on disk where the file resides will pop up, and you can copy the raw bits off to some safe location. Unless the file itself has been trashed -- but the fact that it's 0 bytes seems to indicate that the directory entry has been replaced. It's quite possible that your file is intact on the drive somewhere.

Is there any way I can "lock" the sector/blocks where the directory/file is located from further use?
Also, I've used grep with files before, but how would I "grep" my harddrive in terms of raw bytes? Also are there any concerns I should worry about when mounting my NTFS drive with Knoppix?

Quote:

---

Originally Posted by @nthony

Is there any way I can "lock" the sector/blocks where the directory/file is located from further use?

---

Not without some special utility. Basically, you'd allocate those blocks to a "dummy" file to prevent them from being used by something else. Similar to actually undeleting it, actually.

Quote:

---

Also, I've used grep with files before, but how would I "grep" my harddrive in terms of raw bytes? Also are there any concerns I should worry about when mounting my NTFS drive with Knoppix?

---

You don't have to mount it at all. Just grep the raw partition. Your drive will be designated by a file either /dev/hdX or /dev/sdX, where X is a partition number. For instance, if your drive is IDE and it's the primary drive, the partitions will be labeled /dev/hda1, /dev/hda2, etc. So...

Code:

---

$ grep -b "something distinctive" /dev/hda1

---

This will search the first partition of the primary IDE disk for occurrences of "something distinctive," and print out the byte position of the match, if found. Then, you can use the "dd" utility to extract the raw data.

Try the grep first -- if it works, I'll explain the exact usage of dd.

EDIT: In the meantime, don't do ANYTHING on your box. The most recently deleted files are the best candidates to be overwritten by new activity.

I happen to have a Ubuntu disk on hand, can I use the grep utility from this as well?

Quote:

---

Originally Posted by @nthony

I happen to have a Ubuntu disk on hand, can I use the grep utility from this as well?

---

Sure... You should be able to boot off that to the installer. When it comes up, don't do anything. Just hit Alt-F2. You should be at a root prompt.

Window's encoding (1252 I think my file was) won't intefere much with grep will it (its still ASCII right?)

Quote:

---

Originally Posted by @nthony

Window's encoding (1252 I think my file was) won't intefere much with grep will it (its still ASCII right?)

---

The lower 7 bits of all Latin Windows encodings are identical to ASCII. So yeah, it'll work fine.

Ok, so I'm here via Ubuntu's live CD, still amazed at how much can actually be accomplished with just 1 gig of memory and a 700 mb CD, but mostly because I'm afraid to boot into windows and risk changing the state of the FS (though in all honesty, I've been using it 2 days now since the incident occured).
After grepping the primary partition for about 3.5 hours now (I guess that's normal for 120 GB), I've gotten mixed results:
Of the 4 or 5 patterns searched for that I remembered best from the lost file, 2 came back as matches. However, grep did not report info on where the matches came from (I've tried -A,B,C,H), the only thing that it returned was "Binary file /dev/sda2 matches" (no offset information either). Maybe this is because those options do not apply to binary files or maybe the grep utility on the live CD differs, I'm not entirely sure. I am pretty sure though that the search terms were exclusive to that file, but I cannot be certain without seeing the surrounding bytes.
So now my question would be, how can I find out where grep is finding the matches and what are the surrounding byte values?
And also, is it possible that my file has been partially destroyed, explaining why grep found some terms, but not all?
And of course, the most important, if this actually does turn out to be parts of my lost file, how can I reclaim them? I mean, it can't be just as easy as copying a series of blocks right (i.e. what happens if the file was not contiguous)?
Trying not to get my hopes up too high though, I don't think I could bear getting shot down again...

"grep -b" should give you a "byte-offset", according to this:
http://linux.die.net/man/1/grep

You can do "man grep" in your ubuntu to see if it's there - it may be that you have a reduced set of grep functionality if the bootable CD uses something like "busybox" as the shell (busybox is a small shell with LOTS of stuff built in as opposed to external tools).

--
Mats

Yep, I was using brewbuck's instructions from the previous post, in addition to the other switches I mentioned (and prefixed with "sudo", to be completely verbose). Man gave a grep page with full listings, but I guess only some switches are actually implemented on the live CD then? (though $SHELL is /bin/bash)

Try using "find / -name grep -print" to see if there's a "real" grep somewhere in your filesystem (could try "ls -l /bin/grep" or some such first). Watch out for symbolic links - you could have grep linked to busybox for example, in which case busybox will look at argv[0] to decide what to do.

--
Mats

Eh, find takes forever. Use
which grep
to find out the exact path of the grep binary found by the PATH mechanism.

/bin/grep
unless running it through "sudo" changes things...

Having found some likely block offsets using "grep -b", use the "dd" command to transfer that part of the disk to another file on another disk, where you should be able to inspect it.

that's the one trouble I'm having though, getting the offset...

Code:

---

sudo grep -b "GetDlgItem(hwnd, IDC_GRIPPIE)" /dev/sda2

---

That's the exact command I'm using... am I doing anything wrong there?

Quote:

---

Originally Posted by @nthony

Yep, I was using brewbuck's instructions from the previous post, in addition to the other switches I mentioned (and prefixed with "sudo", to be completely verbose). Man gave a grep page with full listings, but I guess only some switches are actually implemented on the live CD then? (though $SHELL is /bin/bash)

---

Trick grep into thinking the drive is ASCII by passing the -a option. This way it will actually behave as expected.

If you're booted from Ubuntu, then your grep is a "real grep."

Okay that seems to have triggered something, though I'm not sure what. A large amount of text is dumped into the terminal window and some lines of this text are prefixed with a number and colon:

Quote:

---

Originally Posted by grep

69761224820:#define WC_CHILD_WND_ADD_ADV "AddAdvChildWnd"

---

is that the byte offset?
Excitingly, some of the text I recognize as belonging to the lost file, however many of the lines are garbled and some lines that follow each other did not in the actual file. What does this mean... is this resultant from forcing some binary to display as ASCII, or is my file not stored contiguously (in which case, how would I ever locate the other parts?), or is it just mangled beyond rescue?

Also, what would be my next step to retreive it, from the man on dd, I've worked out I'll need to do something like: "dd if=/dev/sda2" count=(my file size, ~500kb) ibs=1 skip=(byte offset of file??)"

Yes, that big number is your "byte offset".

Garbled order may indicate that you are seeing blocks in "backwards order", e.g. blocks later on in the filesystem are lines of previous sections of file - which is fine. But it could also be other data/files that just happen to contain your file...

you probably want dd to look something like this:

Code:

---

dd if=/dev/sda2 skip={byteoffset} bs=4K count={some small number} >> somefile-not-on-sda2

---

--
Mats

You'll probably want to round down the block offset to some multiple of say 512. It depends how far you remember your search pattern was from the beginning of the file.

Does 1 byte = 1 block?
My file was ~250kb, so if I start 250kb before the offset and read 500kb, that should cover it to be safe right?

Quote:

---

Originally Posted by @nthony

Does 1 byte = 1 block?
My file was ~250kb, so if I start 250kb before the offset and read 500kb, that should cover it to be safe right?

---

A disk-block is (in almost all cases) 512byte. I wouldn't necessarily expect the file to be in consecutive blocks - it could be, but it may also not be.

--
Mats

Meaning that I'd have to hunt down every block where parts of the file were stored in? (that would make it near impossible to salvage the entire file...)

p.s. mkdosfs combined with prolonged exposure to 60Hz refresh rate is giiving me a severe headache:

Code:

---

sudo mkdosfs /media/disk

---

is what I'm using to format my floppy disk, but mkdosfs simply returns "unable to open /media/disk"...

Quote:

---

Originally Posted by @nthony

Meaning that I'd have to hunt down every block where parts of the file were stored in? (that would make it near impossible to salvage the entire file...)

---

Just save a HUGE chunk (e.g. a few MB) around where your file appears to be, load it up in some suitable editor and remove and re-order the text until it resembles the original. It is likely that large portions are in consecutive sectors, but you'd be quite lucky to have ALL in one consecutive chunk.

Quote:

---

p.s. mkdosfs combined with prolonged exposure to 60Hz refresh rate if giiving me a severe headache:

Code:

---

mkdosfs /media/disk

---

is what I'm using to format my floppy disk, but mkdosfs simply returns "unable to open /media/disk"...

---

Did you also try /dev/fd0 or some such? [That probably won't work if it's a USB floppy]

--
Mats

Unless your file system was nearly full, or horribly fragmented to begin with, chances are that most of the file will exist in a small number (hopefully just the one) of consecutive blocks.

The other problem being that if you used the disk since the file was lost, that some parts of the file may already have been lost for good.

Yea, put me down for the latter. At the time I was in such shock I completely forgot about trying to recover it and continued to use the computer for about a day afterwards...
I'll try what mat said incase the file was scattered across blocks, but I guess I won't be using a floppy for that then. I have a 64 MB USB key, but something tells me using it with Ubuntu is going to prove tricky, anybody know how I would go about doing this?

Quote:

---

Originally Posted by @nthony

Yea, put me down for the latter. At the time I was in such shock I completely forgot about trying to recover it and continued to use the computer for about a day afterwards...
I'll try what mat said incase the file was scattered across blocks, but I guess I won't be using a floppy for that then. I have a 64 MB USB key, but something tells me using it with Ubuntu is going to prove tricky, anybody know how I would go about doing this?

---

Recent versions of Linux handles USB keys quite well - I was surprised when I tried it lately and I didn't heve to do anything other than figure out what device it was.

Alternatively, use emacs to edit the file - there should be an emacs on the ubuntu.

--
Mats

You're right, I didn't think Ubuntu had PnP detection for USB keys, but as soon as I plugged it in, it brought up "/media/USB MEMORY"
I'm transfering data to it now, but dd is taking quite a while. Here's the command I'm using:

Code:

---

sudo dd if=/dev/sda2 skip=119648000000 bs=4000 count=2000 >> /media/"USB MEMORY"/part1

---

I have quite a lot of work to do stilll as I noticed offsets occur at 65, 66, 69, 75, 92, 118, and 119 GB. I'll have to investigate each range in small increments... this could take me a very long time.

and after all that:

Quote:

---

0+0 records in
0+0 records out
0 bytes (0 B) copied, 2640.14 seconds, 0.0 kB/s

---

I don't get it, what went wrong?

maybe you need:

Code:

---

"/media/USB MEMORY/part1"

---

It didn't report any errors with the file/path, the file even showed up on my USB key, although completely empty. I've modified the command as salem suggested with arguments that are more byte-friendly (and replaced the stdout redirector with an explcit of=) and tried again:

Code:

---

sudo dd if=/dev/sda2 bs=4096 count=7813 skip=118383000024 of=/media/"USB MEMORY"/file01

---

*crosses fingers*

edit: I think I know what is going wrong, I've used the byte offset where it wants a block offset... sigh (ctrl+c, start-over).

Code:

---

sudo dd if=/dev/sda2 bs=4096 count=7813 skip=231216797 of=/media/"USB MEMORY"/file01

---

I tried arbitrarly smaller numbers (i.e bs=1 count=1953 and skip=10240, and this seems to make it happy now, apparently it doesn't like a bs value of 4K...

So now I'm seeing what definitely is parts of my file, large parts still in tact, but large parts seemingly missing or mangled. Oddest of all though is that certain sections of the file are repeated again and again... wierd... well I've got a lot to sift through to see if any of it is still worth recovering but its better than being completely helpless. Thanks a lot to everyone who helped here, I'll let you all know how it turns out.

Quote:

---

Originally Posted by @nthony

edit: I think I know what is going wrong, I've used the byte offset where it wants a block offset... sigh (ctrl+c, start-over).

Code:

---

sudo dd if=/dev/sda2 bs=4096 count=7813 skip=231216797 of=/media/"USB MEMORY"/file01

---

---

Another thing to consider is that some of the numbers involved here are very large -- larger than a 32 bit integer. I wouldn't be surprised if something somewhere is choking because of that. Although I'd expect "dd" to handle it appropriately, being that it is a heavily used core utility.

I'm glad you've been able to recover at least some of your work.

Update:
After spending the last 3 days parsing through approx. 75mb of mangled, misplaced, and run-together binary and text, I was able to pull from the wreckage significant portions of my original code (I believe the fragments of code came from the remnants of temporary files created by CodeBlocks before saving, which would explain why there were so many repeated code fragments). I would put the retrieval success of my work somewhere between 80-87% of its original state before the crash. I still have some work to do to get it back up to its previous state, but all things considered, THIS WAS LARGELY IN PART A HUGE SUCCESS. And I would like to extend my deepest gratitude and thanks to those of you who have provided so much help for me in this thread. On top of everything else, this was also an unexpected, but much appreciated learning experience for myself; I gained a great deal more knowledge on the structure of internal data storage, as well as methods to ensure that it stays stored (which I will begin deploying immediately!).
Thanks again everyone!