• 06-23-2011
Chris_1980
Dear All.

Maybe this is a little bit irrelative question for this forum but maybe someone has experience on this. I would like to minimize a function of the form (A*x-b)^2 ( A matrix x vector b vector) where x the parameters are a lot. So far I am using genetic algorithm which works but gives a lot of local minima. Do you have to suggest a stochastic method which could work better in finding global minima.

Many thanks,
Chris
• 06-23-2011
m37h0d
• 06-24-2011
grumpy
If, by (A*x-b)^2, you mean the square of the length of Ax-b, then you might try some basic geometry/algebra. That allows you to evaluate the solution exactly, without any searching.
• 06-24-2011
m37h0d
he's doing a least squares problem.
• 06-25-2011
grumpy
Quote:

Originally Posted by m37h0d
he's doing a least squares problem.

That is my point.
• 06-25-2011
tabstop
Quote:

Originally Posted by grumpy
That is my point.

I think you're assuming that A is square and non-singular. If A is nonsquare or singular, then there is no "analytic" answer, hence the need to do minimization. (EDIT: I suppose I should add that still in some of these situations there is still an analytic solution to the least-squares problem, although I don't believe it's all.)

For the OP: I've never worked with genetic algorithms here, so I've not got direct advice for you, unfortunately. My hand-wavy suggestion is that, if you are often getting stuck in a "rut", I would guess that either you've got some very good local minima (is the geometry very steep around these points, making it difficult to get out?) or that something weird is going on with mutations, that aren't allowing them to get "far enough" away.
• 06-26-2011
grumpy
Look up pseudo-inverse, specifically the Moore-Penrose pseudo-inverse (which is a particular pseudo-inverse that is unique for any matrix A). Techniques to compute it include QR decomposition, singular value decomposition, etc.

One of the most common applications of the pseudo-inverse is computing a "best" (least-squares) fit for exactly this problem.