Been doing a little Googling and looking for some kind of benchmarking list for executables from various C compilers... Specifically I'm trying to find out if there are pronounced speed differences between say... VC++ (in C mode) and GCC (in C99 mode) and Pelles C...
Anyone got some information they can share?
EDIT... I don't care about time to compile... I'm after the speed of the executables the compilers produce.
Last edited by CommonTater; 11-17-2011 at 09:24 PM.
I'm not sure...but I've heard that Intel's Compilers' Executables are unmatched....but only on Intel's chips; because of 'undocumented' optimizations .
Also, the native tools on each platform possibly has an edge.
Last edited by manasij7479; 11-17-2011 at 09:37 PM.
Used to be Intel was #1, MS (VS 6.0) was second, with the rest, not too far behind.. Have not heard of any comparison with Pelles C.
New name to me is Clang, which is at or near the top spot.
The person you need to ask about this, is Jim over on www.talkchess.com. He is the noted best at producing the fastest executables, and many chess programmers go to him for his help. His work is simply sensational.
The way to get the fastest code is to choose the best algorithms and data structures.
No compiler can possibly save a dumb programmer who chooses bubble sort over quicksort when they have millions of records to sort.
Or one who loads entire files into memory because it can be done with a single API call, without realising the possible knock-on effect of things like swapping.
If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
If at first you don't succeed, try writing your phone number on the exam paper.
3rd for the Intel C compiler. I actually did a comparison of this a while ago:
Hazudra Fodder: gcc vs. LLVM vs. icc (GNU C comp. vs. LLVM vs. Intel C++ comp.)
Basically, icc is godly and LLVM is like 10% faster than GCC.
Also, if you're compiling on Windows, GCC (either mingw or cygwin) will always lose. Especially mingw since it uses the old MSVCRT.dll as its C library.
If you're using Linux or do use Linux, you can try out Intel's compiler suite for free (non-commercial use):
Non-Commercial Software Download - Intel® Software Network
Oh come on Salem... This is me asking a question, not an opportunity to take swipes at me.Originally Posted by Salem
Last edited by CommonTater; 11-18-2011 at 07:04 AM.
Depends on what you're building. Here GCC SPEC2000/2006 Performace Tracking LLVM trails GCC in most of SPEC2000. Check the last few links on the left, scroll down to the performance charts.
Clang & LLVM has lots of good advertising going for it. Some of it even lines up with reality. But like GCC, it's free so there's no reason not to try it and see how well it does compared to other free alternatives.
Last edited by KCfromNC; 11-18-2011 at 08:58 AM.
Thank you for an actual answer...
Very interesting... I wonder how that translates to Windows systems...
SPEC2000 does a pretty decent job of being library & OS independent, so the differences you're seeing are mostly due to generated code quality. No guarantee that this code has anything to do with the code you're writing, though, so the results may or may not mean anything.
But since GCC and LLVM are both free, the best way to get an answer is download them and try them out. You can get GCC as part of cygwin or MinGW. LLVM you'll have to build from source. Intel has a 30 day evaluation for Windows.
I assume you realise that different compilers excel in different areas. Code snippet A might run faster on compiler 1, but code snippet B runs faster on compiler 2.
I've even heard that it can be as simple as changing your loop type from a 'while' to a 'for' as a certain compiler out there optimises for loops more heavily. Stupid, I know.
If you spend enough time profiling and optimising your code for one compiler then you may have only tailored it to run fast when compiled using just that compiler. Swap compilers and repeat the process of profiling and optimising and then it may run slower when compiled with the first compiler.
My homepage
Advice: Take only as directed - If symptoms persist, please see your debugger
Linus Torvalds: "But it clearly is the only right way. The fact that everybody else does it some other way only means that they are wrong"
You assume correctly, I do understand that. I don't expect any one compiler to be the fastest/best/smallest at everything... However, I was wondering if something new was on the horizong or if one particular compiler stood out far enough that I should know about it.
This is mostly curiosity, I'm always game to try something different/better/newer ... I'm just tired of going off on "snipe hunts" and accomplishing nothing but wasted time.
That is not why. The reason why is that the compiler uses a piece of code to check whether the given processor supports SSE. On non-Intel systems, this check fails, and the optimized code path is not used. This was discovered years ago, AMD sued Intel, won the lawsuit, yet Intel continues to cripple their binaries on non-Intel systems -- they've simply gotten sneakier about it, so they can claim it was just a "bug" in their compiler. Oops. Yeah right.
There are numerous documented hacks out there which, when applies to a binary compiled with the Intel compiler, cause it to run faster on AMD systems.
Code://try //{ if (a) do { f( b); } while(1); else do { f(!b); } while(1); //}
> Oh come on Salem... This is me asking a question, not an opportunity to take swipes at me.
Well comparing compilers is just a ........ing contest.
First of all, this "Intel is best" is only actually relevant if you have an intel processor. If you're using something else - and there are MANY different processors out there, it's really a moot point.
Second - it's not like one compiler is like 10x faster than another on ALL code (not just the self-select benchmarks). Most of the time, it's +/-10% - which is getting down into the realms of micro-optimisation territory. That is, spending more and more time for less and less effect.
Third, aggressive optimisation tends to find bugs in both the compiler AND the user generated code. The user code isn't a problem (unless the user can't program, and therefore doesn't understand the nature of the problem facing them -> blame the compiler). But bugs in your chosen tool chain are a real PITA to deal with (BTDTGTTS).
> Hazudra Fodder: gcc vs. LLVM vs. icc (GNU C comp. vs. LLVM vs. Intel C++ comp.)
Ohh - one run in the sub-second category - now that's a solid evaluation of the facts if ever I saw one.
You need multiple runs of code taking at LEAST 10 seconds (ideally longer) before you can start to tease out the effects of the OS and other variables which the program has NO control over.
A random fart in the scheduler, and the code goes from "best" to "worst".
If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
If at first you don't succeed, try writing your phone number on the exam paper.
Here's a (rather old, January 2011) comparison of GCC, LLVM, and MSVC running a ray tracer.
Compilers status: Visual C++, GCC and LLVM (updated)
According to that, LLVM < GCC < MSVC. I wonder if LLVM has closed the gap in the last 10 months, being the new kid on the block.
I think it all gets somewhat subsumed by system wide considerations anyway... it's always an interesting challenge trying to get a benchmark away from all the OS scheduling, slow memory, network delays, random crap -- but in the end those things obviously affect your real system.
Perhaps one compiler might pull out in front when a new architecture or some new instructions appear.... but I think the value of benchmarking is as much to make sure one compiler hasn't epically screwed up and dropped performance on the floor. Although one of the big deals about LLVM is that it's great for writing optimisation passes, so who knows, maybe I'm wrong and suddenly a superduperfast compiler will emerge.
