Thread: Benchmarking c compilers

  1. #1
    Registered User
    Join Date
    Jan 2009
    Posts
    8

    Benchmarking c compilers

    What would be an appropriate simple function to compare the relative execution speed of different c compilers. I've used the following, which shows clear differences between gcc, Pelles C and Borland, for example but doesn't necessarily evaluate all compiler optimizations.

    Code:
    
    #include <stdio.h> 
    #include <math.h>
    #include <time.h>
     
    #define NRUNS 1000
     
    int main()
    {
    	int h=0,i,j,k;
    	double d=0,pi = 4.0*atan(1.0);
    	time_t start,end;
    	printf("Starting now ...");
    	start=clock();
    	for (i=0;i<NRUNS;i++)
    	{	
    		for (j=0;j<NRUNS;j++)
    		{
    			for (k=0;k<NRUNS;k++)
    			{
    				d+=sqrt(pi);
    				h++;
    			}
    		}
    	}
    	end=clock();
    	printf(" done. \n");
    	printf("h: %d \n", h);
    	printf("d: %f \n", d);
    	printf("Time (seconds): %15.3f \n", (double)(end-start)/CLOCKS_PER_SEC);
    	return 0;
    }

  2. #2
    Registered User
    Join Date
    Sep 2006
    Posts
    8,868
    I sure wouldn't use this.

    All you're really testing is sqrt(). That's a very small part of most programs. I'm sure there are benchmark programs that do a host of tests for this purpose.

    Also, you're not using the high speed timer available for Windows and Linux.

    I like the way you're thinking about testing these different compilers, however!

  3. #3
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Also, if the compiler is at least a little bit clever, it will move this out of the inner loop, and then collapse the three loops into nothing.
    Code:
    				d+=sqrt(pi);
    				h++;
    After all, just doing h = NRUNS * NRUNS * NRUNS after the loop, and d = (NRUNS * NRUNS * NRUNS) * sqrt(pi) will provide the d and h values.

    Also, assuming the compiler DIDN'T optimize everything, sqrt() is a fairly long operation in relation to the rest of the code, so it will be "most of the time inside your loop". On top of that, different C libraries and/or compiler switches may produce different code for sqrt() - anything from 26 or 100 clock-cycles on a modern x86 processor. This is of course meaningfull if your code does a lot of sqrt() calculations, but not so useful if your code is doing mostly other calculations.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  4. #4
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    I'd get the source code to some real application and compile that under different compilers. Anything less than a few thousand lines of code isn't going to exercise the compiler nearly enough for good results.
    Code:
    //try
    //{
    	if (a) do { f( b); } while(1);
    	else   do { f(!b); } while(1);
    //}

  5. #5
    Registered User
    Join Date
    Jan 2009
    Posts
    8
    Thanks

    Of course I agree with all that was posted but perhaps I should have stated my intentions more clearly. I'm writing a small (< 100 lines) packing simulation program which is cpu intensive and one run takes ~27 hours (compiled using a BASIC compiler). The math it includes is very simple (nearest neighbor algo). The prog can easily be converted to C and I'm in the need for speed. I can't afford Intel so I'm evaluating a number of free alternatives using the code listing provided. A few minutes difference in performance doesn't bother me but if I can shave a few hours off, well ...
    Last edited by freevryheid; 01-23-2009 at 02:59 PM. Reason: just how small?

  6. #6
    Hurry Slowly vart's Avatar
    Join Date
    Oct 2006
    Location
    Rishon LeZion, Israel
    Posts
    6,788
    Quote Originally Posted by freevryheid View Post
    I can't afford Intel
    linux version of Intel compiler is free for personal use.
    All problems in computer science can be solved by another level of indirection,
    except for the problem of too many layers of indirection.
    – David J. Wheeler

  7. #7
    Registered User
    Join Date
    Sep 2006
    Posts
    8,868
    Quote Originally Posted by freevryheid View Post
    Thanks

    Of course I agree with all that was posted but perhaps I should have stated my intentions more clearly. I'm writing a small (< 100 lines) packing simulation program which is cpu intensive and one run takes ~27 hours (compiled using a BASIC compiler). The math it includes is very simple (nearest neighbor algo). The prog can easily be converted to C and I'm in the need for speed. I can't afford Intel so I'm evaluating a number of free alternatives using the code listing provided. A few minutes difference in performance doesn't bother me but if I can shave a few hours off, well ...
    A packing simulation? A 27 hour run-time packing simulation?

    You post that baby up here and I'll bet you a dollar to a donut that we can chew that little baby right on down in run time!

    Your OS is Windows or Linux?

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. C++ Builder Comparison
    By ryanlcs in forum Tech Board
    Replies: 14
    Last Post: 08-20-2006, 09:56 AM
  2. Other compilers
    By OldSchool in forum C++ Programming
    Replies: 6
    Last Post: 06-10-2006, 09:15 PM
  3. Is It Possible To Install Multiple Compilers?
    By mishna_toreh in forum C Programming
    Replies: 3
    Last Post: 05-13-2005, 07:32 AM
  4. Compilers for Windows
    By LegendsEnd in forum Windows Programming
    Replies: 2
    Last Post: 03-26-2004, 08:03 AM
  5. Compilers, Compilers, Compilers
    By Stan100 in forum C++ Programming
    Replies: 11
    Last Post: 11-08-2002, 04:21 PM