Reverse-Engineering socket data

[filler_bunny]

Hi all.

We have this lovely application at work that was written in house some time ago, features no documentation and no source code and the original author left before I got there.

We now have to update this app, but before we can we have to decipher what is going on between the application and the server.

The application (client) uses a mix of UDP and TCP sockets and will run on a windows machine. Anyone got ideas on the best way to reverse engineer this? i.e. tools, ideas, experiences etc?

Cheers,

FB

[adrianxw]

If you've got no source code, I doubt you are going to update it. To decompile an .exe may give you some assembler, but you'll have a hell of a job going any further than that. So what you need to do is replace whichever part you need to change.

If the client and server give you the information, you can try and log all the responses to all the outgoing messages, and try to build up the protocol. If they won't do it for you, you'll need to sniff the connection and try and work it out that way.

Whatever way forward, there is no guarantee you have caught every possibility, and it may well be one of the overlooked situations will cause a critical failure.

If the app is mission critical, forget it. Work out what you want and re-do it, remembering the documentation and the source code!

[filler_bunny]

If you've got no source code, I doubt you are going to update it.

Absolutely, I should have been more specific.

The component (application) will be replaced - it is not a large component and its behaviour is pretty predictable. I have done some port sniffing and can see that at least some of the data is plain text. But I am unsure about the rest. I can guarentee it is not encrypted, but looks like garbage so I presume it is probably binary data?? I really don't know much about network programming - but I am about to learn.

If the app is mission critical, forget it. Work out what you want and re-do it, remembering the documentation and the source code!

Agreed - this is a good lesson to anyone who doesn't believe in documentation! This is exactly what we are trying to achieve - we are consolidating our software (a lot of nawk tools etc), and documenting those tools we can with the source we have. But this particular program while not mission critical is very useful.

The protocol doesn't seem to be that difficult since it really just sends a request to a server then displays the result (which is a simple text based financial report). I'm just having difficulty understanding what I am seeing and how best to go about deciphering it.

[adrianxw]

Is it so simple that you can send a 1 line report saying something like "This is a 1" and see if the message says "This is a ", and the hex/octal/binary, whatever you use, for the interger 1? Or something along those lines so you can readily identify and correlate the "garbage" with a single number.

You can then check out the encoding systems for floating points etc. on your system, which I'm guessing is a *nix, (you said nawk).

[Xei]

Use something called Etherreal. It may work.

[filler_bunny]

Thanks guys

I have had a few lucky breaks - for instance, nothing is encrypted, including passwords (rather disturbing), so using a packet sniffer it has been easy to determine what is contained within the packets - I am now going to write a couple of apps to test my assumptions.

Thanks all - you've given me the right push in the correct direction.

Bit worried about that lack of encryption on the passwords though - that programmer should be shot...

[edit]
Xei - your right, Ethereal does work - bit buggy on the windows version but does the job. Otherwise Sniffit on *nix is pretty good I have discovered.
[/edit]

You need your network sniffer and a systematic way of sending (hopefully) simple requests from client to server. By slowly vaying the information in the requests, you should be able to figure out what the delta's are in the requests, and from that what it means.

Cheers - that's a great idea. I better get started!