Benchmarking c compilers

What would be an appropriate simple function to compare the relative execution speed of different c compilers. I've used the following, which shows clear differences between gcc, Pelles C and Borland, for example but doesn't necessarily evaluate all compiler optimizations.

Code:

#include <stdio.h> 
#include <math.h>
#include <time.h>
 
#define NRUNS 1000
 
int main()
{
	int h=0,i,j,k;
	double d=0,pi = 4.0*atan(1.0);
	time_t start,end;
	printf("Starting now ...");
	start=clock();
	for (i=0;i<NRUNS;i++)
	{	
		for (j=0;j<NRUNS;j++)
		{
			for (k=0;k<NRUNS;k++)
			{
				d+=sqrt(pi);
				h++;
			}
		}
	}
	end=clock();
	printf(" done. \n");
	printf("h: %d \n", h);
	printf("d: %f \n", d);
	printf("Time (seconds): %15.3f \n", (double)(end-start)/CLOCKS_PER_SEC);
	return 0;
}

I sure wouldn't use this.

All you're really testing is sqrt(). That's a very small part of most programs. I'm sure there are benchmark programs that do a host of tests for this purpose.

Also, you're not using the high speed timer available for Windows and Linux.

I like the way you're thinking about testing these different compilers, however!

Also, if the compiler is at least a little bit clever, it will move this out of the inner loop, and then collapse the three loops into nothing.

Code:

				d+=sqrt(pi);
				h++;

After all, just doing h = NRUNS * NRUNS * NRUNS after the loop, and d = (NRUNS * NRUNS * NRUNS) * sqrt(pi) will provide the d and h values.

Also, assuming the compiler DIDN'T optimize everything, sqrt() is a fairly long operation in relation to the rest of the code, so it will be "most of the time inside your loop". On top of that, different C libraries and/or compiler switches may produce different code for sqrt() - anything from 26 or 100 clock-cycles on a modern x86 processor. This is of course meaningfull if your code does a lot of sqrt() calculations, but not so useful if your code is doing mostly other calculations.

--
Mats

I'd get the source code to some real application and compile that under different compilers. Anything less than a few thousand lines of code isn't going to exercise the compiler nearly enough for good results.

Thanks

Of course I agree with all that was posted but perhaps I should have stated my intentions more clearly. I'm writing a small (< 100 lines) packing simulation program which is cpu intensive and one run takes ~27 hours (compiled using a BASIC compiler). The math it includes is very simple (nearest neighbor algo). The prog can easily be converted to C and I'm in the need for speed. I can't afford Intel so I'm evaluating a number of free alternatives using the code listing provided. A few minutes difference in performance doesn't bother me but if I can shave a few hours off, well ...

Originally Posted by freevryheid

I can't afford Intel

linux version of Intel compiler is free for personal use.

Originally Posted by freevryheid

Thanks

Of course I agree with all that was posted but perhaps I should have stated my intentions more clearly. I'm writing a small (< 100 lines) packing simulation program which is cpu intensive and one run takes ~27 hours (compiled using a BASIC compiler). The math it includes is very simple (nearest neighbor algo). The prog can easily be converted to C and I'm in the need for speed. I can't afford Intel so I'm evaluating a number of free alternatives using the code listing provided. A few minutes difference in performance doesn't bother me but if I can shave a few hours off, well ...

A packing simulation? A 27 hour run-time packing simulation?

You post that baby up here and I'll bet you a dollar to a donut that we can chew that little baby right on down in run time!

Your OS is Windows or Linux?