If you're so blessed as to be using unix or linux, then try the file command
Short of reading the whole file and checking every byte, the best you can do is get a probable answer by checking say the first 256 bytes.
$ file hello.c
hello.c: ASCII C program text
$ file a.out
a.out: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), not stripped
$ file hello.o
hello.o: ELF 32-bit LSB relocatable, Intel 80386, version 1 (SYSV), not stripped
For a text file, you would expect all the characters to be printable, and at least one newline within the first 256 chars.