This is a cool feature of flight sims and here is how you do it.



Code:
void CCamera::GetViewMatrix(D3DXMATRIX *outMatrix)
{
    D3DXVec3Normalize(&Look,&Look);
  
    D3DXVec3Cross(&Up,&Look,&Right);
    D3DXVec3Normalize(&Up,&Up);

    D3DXVec3Cross(&Right,&Up,&Look);
    D3DXVec3Normalize(&Right,&Right);

    D3DXVECTOR3 vecPos=Pos;
  
    float x=-D3DXVec3Dot(&Right,&vecPos);
    float y=-D3DXVec3Dot(&Up,&vecPos);
    float z=-D3DXVec3Dot(&Look,&vecPos);
  
    if (m_iMode==FLYBY)
    {
      D3DXMatrixLookAtLH(outMatrix,&Pos,&m_spTargetObject->GetOrientControl()->GetPosition(),
                         &Up);
    
    }
    else
    {
    
      (*outMatrix)(0,0)=Right.x;
      (*outMatrix)(0,1)=Up.x;
      (*outMatrix)(0,2)=Look.x;
      (*outMatrix)(0,3)=0.0f;

      (*outMatrix)(1,0)=Right.y;
      (*outMatrix)(1,1)=Up.y;
      (*outMatrix)(1,2)=Look.y;
      (*outMatrix)(1,3)=0.0f;

      (*outMatrix)(2,0)=Right.z;
      (*outMatrix)(2,1)=Up.z;
      (*outMatrix)(2,2)=Look.z;
      (*outMatrix)(2,3)=0.0f;

      (*outMatrix)(3,0)=x;
      (*outMatrix)(3,1)=y;
      (*outMatrix)(3,2)=z;
      (*outMatrix)(3,3)=1.0f;
    }
}

The first time you enter flyby mode you need to set the camera a certain distance ahead of the object -> some distance along its look vector. You may also want to translate left/right along the object's right vector and maybe up/down along it's up vector for some random positioning of your camera. Then you set a boolean indicating you don't need to do this again.

Now in the camera class you test for the mode and if it's FLYBY then you create a look at matrix from your camera to the object and pass in your camera's up vector.

Now in my game engine it's possible to fly the ship just like you would in MS Flight Sim in control tower view.

Very nice.