You need all the exe's because gcc/g++ calls them in background for you.
For example,
ld.exe is the linker
as.exe is the assembler
ar.exe is the archiver (for making libraries)
gdb.exe is the GNU Debugger
gprof.exe is the GNU profiler (tells you where your program spends its time)
gcov.exe is a test coverage program (tells you what lines of your code are actually run)
make.exe is a build system (makes building multi-file projects easier)
Those are the ones I know... I'm sure you can Google the rest up if you wanted to.

