Running a console program with maximum speed

arno-nyme · 05-26-2008, 01:06 PM

Hello,

I am programming a "agent based" computer simulation in gcc. It basically does vector operations and gives as output some text files. Its an console application. It is written for scientific purposes. I would like to find a way to running this program dedicating the complete (or like 80%) of my processing power to it. File is attached. (code is messy I am not a professional programmer)

Some more details for the curious:It is basically a simulation of object oriented agent, which trade, produce and consume in a network. On to of that they change there trading partners, to get a competitive advantage

Thank you very much

Davoud

**Salem** · 05-26-2008, 01:29 PM

Well how long does it take to run as it is?

I see a lot of console and file I/O, and that kind of stuff just doesn't use all the CPU time no matter how much you "optimise" it.

Elysia · 05-26-2008, 01:35 PM

As for the I/O, try using buffering approaches. Buffering can help a lot with file I/O. However, it would be platform specific.
And that indentation could use some work too. I found it mostly confusing.

arno-nyme · 05-26-2008, 01:55 PM

Every round takes 20 to 30 second, so for 200 rounds, 100 minutes. Which is a problem as I have to try different configurations.

I do not know what buffering is, could you give me a hint where to start?

I do not think that it is only a problem of the i/o operations, they are only in the reporting. When I increased some thinking about the future of the agents, which increases the number of vector operations, but not I/O operations as thinking of the agents does not get reported processing time wen up drastically.

Elysia · 05-26-2008, 01:56 PM

I'm not familiar with Linux, sorry.
Another good idea is to try a profiler to find the bottlenecks in your code. Then you can start optimizing.

**Salem** · 05-26-2008, 02:22 PM

> vector <int> concatinate(vector <int> a,vector <int>b)
Look into returning the result by reference (in some way). But I've just noticed that you never call it anyway. For the sake of people reading the code, prune out all the dead wood.

> void sort(vector<int> & mir,vector<float> vec)
I was going to say this is horrible as well (look at std::sort in algorithm), but then I found this isn't used either.

arno-nyme · 05-26-2008, 03:42 PM

yes I purged the stuff.

matsp · 05-26-2008, 03:51 PM

Use either gprof or oprofile to find out where your code spends its time. That way, you are not spending time optimizing someones guess of what takes time.

--
Mats

matsp · 05-26-2008, 04:47 PM

One thought that strikes me after running gprof on the code is that you seem to maintain multiple deque's with the same data. E.g.

Code:

	    while ((_Price_List<Price_List[Auction_Color].end()) && (_Price_Response<Price_Response.end())){
		if (*_Price_List<*_Price_Response) {                //  This is verz cricial
		    Price_Help[Auction_Color].push_back(*_Price_List);
		    Name_Help [Auction_Color].push_back(*_Name_List);
		    _Price_List++;
		    _Name_List++;
		} else {
		    Price_Help[Auction_Color].push_back(*_Price_Response);
		    Name_Help [Auction_Color].push_back(friends[i]);
		    _Price_Response++;
		}
	    }
	    while (_Price_List<Price_List[Auction_Color].end()) {
		Price_Help[Auction_Color].push_back(*_Price_List);
		Name_Help[Auction_Color].push_back(*_Name_List);
		_Price_List++;
		_Name_List++;
	    }
	    while (_Price_Response<Price_Response.end()) {
		Price_Help[Auction_Color].push_back(*_Price_Response);
		Name_Help[Auction_Color].push_back(friends[i]);
		_Price_Response++;
	    }

I think it would create a bit less overhead if you had a struct/class containing the price and the name, and store that in a single deque. This will reduce the overhead of push_back() and iterator updates.

Here's the first five functions of gprof-output (running only 5 iterations):

Code:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls   s/call   s/call  name    
 10.88      5.86     5.86 117668390     0.00     0.00  cAgent::_Production(int*, float&)
  9.23     10.82     4.97    48064     0.00     0.00  cAgent::Ask_Prices()
  5.96     14.04     3.21 58874421     0.00     0.00  cAgent::MPX(int, int*, int)
  5.30     16.89     2.85 387271859     0.00     0.00  std::_Deque_iterator<float, float&, float*>::_Deque_iterator(std::_Deque_iterator<float, float&, float*> const&)
  5.17     19.66     2.78 470673560     0.00     0.00  powf

Since _production is one of the top functions, and it calls pow() which is number 5 on the list, would it be possible to pre-calculate this calculaiton of pow():

Code:

void cAgent::_Production(int *Colors, float &y)
{
    int i;
    y=productivity;
    for (i = 0; i < Number_of_Different_Colors; i++) {
	y=y*pow(Colors[i],p[i]);
	Colors[i]=1;
    }
}

You may gain some speed in thise type of thing:

Code:

    for (i = 0; i < Number_of_Different_Colors; i++){
	lColors[i]=i_Colors[i];
	iColors[i]=i_Colors[i];
    }

by using memcpy:

Code:

    memcpy(lColors, i_Colors, sizeof(lColors));
    memcpy(iColors, i_Colors, sizeof(iColors));

Edit: I ran the last change, and MPX() went from being third on the list to about tenth place in time taken. There may be similar cases in other places - I only scanned the code briefly.

--
Mats

matsp · 05-26-2008, 05:26 PM

Code:

    int   lColors[Number_of_Different_Colors];
    for (i = 0; i < Number_of_Different_Colors; i++)  
	lColors[i]=Colors[i];

at the end of "Buy" seems to be completely unnecessary.

--
Mats

matsp · 05-26-2008, 05:38 PM

And I can confirm that using a single struct, like this:

Code:

struct pricename
{
    float price;
    int name;
};
...
    deque <pricename> Price_List[Number_of_Different_Colors];
...

and removing all references to deque<int> Name_List[] reduces the runtime by approximately 25% (Went from around 52 seconds to around 39 seconds)

There are likely other possible gains. The biggest hit is still _Production(), which uses up about 13% of the total runtime. But add to that about 25% of the calls to powf, so it's more like 16% of the total time.

Just iterating through the deque using the pricename iterator takes about 5% of the runtime.

--
Mats

arno-nyme · 05-27-2008, 02:54 AM

mats,

thank you for all the effort you put!

I have put the memcp.

The structure is a bit difficult, As you have tried it out already could you send me the code?
My mail schumanpatriotes@gmail.com

As you seam to be interested in this program, I attach an auxilliary porgram, which converts the output, to .dot files, which than can be converted using http://www.graphviz.org/ to JPG. Gladly I have no speed issues with that one. If you are interested I can send you the paper, for this model

Davoud

**CornedBee** · 05-27-2008, 03:04 AM

You do realize that names starting with two underscores or an underscore and a capital are reserved for the compiler, right?

arno-nyme · 05-27-2008, 03:12 AM

I did not realize that underscores are reserved. Does it lead to trouble or is it just a question od style. I am not really a programmer, I just write a msters thesis

Davoud

ATTACHMENT: JPG - generator

matsp · 05-27-2008, 03:32 AM

Quote:

Originally Posted by arno-nyme

mats,

thank you for all the effort you put!

I have put the memcp.

The structure is a bit difficult, As you have tried it out already could you send me the code?
My mail schumanpatriotes@gmail.com

As you seam to be interested in this program, I attach an auxilliary porgram, which converts the output, to .dot files, which than can be converted using http://www.graphviz.org/ to JPG. Gladly I have no speed issues with that one. If you are interested I can send you the paper, for this model

Davoud

I think it's worth you doing the structure change, as it's actually your work. I did it quickly, I may have done it wrong (I don't think so), and it's on my home machine which I'm not at right now.

One thought: Since pow() is so frequently used, perhaps switching to -ffast-math and using basic functionality instead of the complexities of "pow" (it does all sorts of quite complicated stuff).

I wrote this:

Code:

static inline float mypowf(float x, float y)
{
    return exp(log(x) * y);
}

In my little test-app, that almost halves the time used by pow (which is essentially all that the app does, so halving the time of the whole test-code).

--
Mats

05-26-2008, 01:29 PM	#2
Salem and the hat of vanishing Join Date: Aug 2001 Location: The edge of the known universe Posts: 21,214	Well how long does it take to run as it is? I see a lot of console and file I/O, and that kind of stuff just doesn't use all the CPU time no matter how much you "optimise" it. __________________ If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut. Up to 8Mb PlusNet broadband from only £5.99 a month!
Salem is offline

05-26-2008, 01:55 PM	#4
arno-nyme Registered User Join Date: May 2008 Posts: 10	Every round takes 20 to 30 second, so for 200 rounds, 100 minutes. Which is a problem as I have to try different configurations. I do not know what buffering is, could you give me a hint where to start? I do not think that it is only a problem of the i/o operations, they are only in the reporting. When I increased some thinking about the future of the agents, which increases the number of vector operations, but not I/O operations as thinking of the agents does not get reported processing time wen up drastically.
arno-nyme is offline

05-26-2008, 02:22 PM	#6
Salem and the hat of vanishing Join Date: Aug 2001 Location: The edge of the known universe Posts: 21,214	> vector <int> concatinate(vector <int> a,vector <int>b) Look into returning the result by reference (in some way). But I've just noticed that you never call it anyway. For the sake of people reading the code, prune out all the dead wood. > void sort(vector<int> & mir,vector<float> vec) I was going to say this is horrible as well (look at std::sort in algorithm), but then I found this isn't used either. __________________ If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut. Up to 8Mb PlusNet broadband from only £5.99 a month!
Salem is offline

05-26-2008, 03:42 PM	#7
arno-nyme Registered User Join Date: May 2008 Posts: 10	yes I purged the stuff.
arno-nyme is offline

05-26-2008, 03:51 PM	#8
matsp Kernel hacker Join Date: Jul 2007 Location: Farncombe, Surrey, England Posts: 15,686	Use either gprof or oprofile to find out where your code spends its time. That way, you are not spending time optimizing someones guess of what takes time. -- Mats __________________ Compilers can produce warnings - make the compiler programmers happy: Use them! Please don't PM me for help - and no, I don't do help over instant messengers.
matsp is offline

05-26-2008, 05:26 PM	#10
matsp Kernel hacker Join Date: Jul 2007 Location: Farncombe, Surrey, England Posts: 15,686	Code: int lColors[Number_of_Different_Colors]; for (i = 0; i < Number_of_Different_Colors; i++) lColors[i]=Colors[i]; at the end of "Buy" seems to be completely unnecessary. -- Mats __________________ Compilers can produce warnings - make the compiler programmers happy: Use them! Please don't PM me for help - and no, I don't do help over instant messengers.
matsp is offline

05-26-2008, 05:38 PM	#11
matsp Kernel hacker Join Date: Jul 2007 Location: Farncombe, Surrey, England Posts: 15,686	And I can confirm that using a single struct, like this: Code: struct pricename { float price; int name; }; ... deque <pricename> Price_List[Number_of_Different_Colors]; ... and removing all references to deque<int> Name_List[] reduces the runtime by approximately 25% (Went from around 52 seconds to around 39 seconds) There are likely other possible gains. The biggest hit is still _Production(), which uses up about 13% of the total runtime. But add to that about 25% of the calls to powf, so it's more like 16% of the total time. Just iterating through the deque using the pricename iterator takes about 5% of the runtime. -- Mats __________________ Compilers can produce warnings - make the compiler programmers happy: Use them! Please don't PM me for help - and no, I don't do help over instant messengers.
matsp is offline

05-27-2008, 02:54 AM	#12
arno-nyme Registered User Join Date: May 2008 Posts: 10	mats, thank you for all the effort you put! I have put the memcp. The structure is a bit difficult, As you have tried it out already could you send me the code? My mail schumanpatriotes@gmail.com As you seam to be interested in this program, I attach an auxilliary porgram, which converts the output, to .dot files, which than can be converted using http://www.graphviz.org/ to JPG. Gladly I have no speed issues with that one. If you are interested I can send you the paper, for this model Davoud
arno-nyme is offline

05-27-2008, 03:04 AM	#13
CornedBee Cat without Hat Join Date: Apr 2003 Posts: 8,439	You do realize that names starting with two underscores or an underscore and a capital are reserved for the compiler, right? __________________ All the buzzt! CornedBee "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code." - Flon's Law
CornedBee is offline

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Running a program	dingolay	C Programming	4	10-01-2006 09:03 PM
Running a console application	maxorator	C++ Programming	4	10-03-2005 04:23 AM
Trying to output to a directory other than the root of where the program is running	Howie17	C++ Programming	3	09-05-2005 08:57 AM
Get the PID of a console program	BianConiglio	Windows Programming	6	05-24-2004 05:24 AM
C++ console program using Ms Visual Studio .NET	matheo917	C++ Programming	4	09-05-2002 11:16 PM