Could you please describe in more detail what you are trying to accomplish and what the problem is. It seems as if this is something that would be easy for you therefore I feel I don't quite understand what is really going on. Otherwise, I have written code to make something look in the direction of something (I had a monster's head look at the player regardless of where the player was, looked like something from the exorcist). It's just a matter of computing pitch and yaw. You just do basic trig to compute the angles, then build the orientation matrix from that. If you want I can post code.
edit:
umm, this is the only code I can find that performs these tasks. Note that the 'fast' functions are from when I experimented with implementing my own square roots, it wasn't practical but this code works. It also is with OpenGL right handed coordinate system with negative z into the screen, therefore you adjust as necessary. also the code is terribly inefficient but as i said it does work.
Code:
float FastVecToPitch(Vector3 *Vec)
{
Vector3 copy = *Vec;
copy.FastEnsureUnitLength();
return RAD2DEG(asinf(copy.y));
}
float FastVecToYaw(Vector3 *Vec,Vector3 *Origin)
{
Vector3 copy = *Vec;
if (copy.y)
{
copy.y = 0;
copy.Fast_Normalize();
}
else
{
copy.EnsureUnitLength();
}
if(Origin->y)
{
Origin->y = 0;
Origin->Fast_Normalize();
}
else
{
copy.EnsureUnitLength();
}
return RAD2DEG(acosf(DotProduct(©,Origin)));
}
You have to be careful how you use it, i.e, you have to negate the angle from VecToYaw in certain instances.
Code:
if(pTarget->mPosition.x > this->mPosition.x)
mYaw = -mYaw;
I believe to solve this problem, use tan, but check for division by zero.