Hi, all. I am developing a 6DOF camera class for use with OpenGL. It "mostly" works, but sometimes a "roll" occurs after a sufficient (> 90 degree) yaw or pitch. Here's the camera structure:

Code:
typedef struct Camera3D
{
    Vector3D u;    // the "right" direction
    Vector3D v;    // the "up" direction
    Vector3D n;    // the "forward" direction
    Vector3D p;    // eye position
    Matrix3D Mv;   // view transformation matrix
    Matrix3D Mp;   // perspective transformation matrix
    float    rx;   // current rotation about x axis
    float    ry;   // current rotation about y axis
    float    rz;   // current rotation about z axis
}
Camera3D;
And here's the code for yaw:

Code:
void
cam3D_yaw(Camera3D* eye, float angle)
{
    Matrix3D r;
    mat3D_set_to_identity(r);
    mat3D_rotx(r, eye->rx);
    mat3D_roty(r, eye->ry);
    mat3D_rotz(r, eye->rz);
    mat3D_roty(r, angle);
    mat3D_rotz(r, -eye->rz);
    mat3D_roty(r, -eye->ry);
    mat3D_rotx(r, -eye->rx);
    mat3D_xfrm(r, eye->n);
    vec3D_norm(eye->n);
    vec3D_dup(eye->u, eye->n);
    vec3D_cross(eye->u, eye->v);
    eye->ry += angle;
    update(eye);
}
The basic approach in yaw is to make the N vector coincide with the Z axis, then rotate it about Y, and keep U orthogonal to N.

I have tried many approaches, this one seems to work the best. I think I am confused about transforming between "view" space and "world" space. The camera vectors N, U, and V are in "view" space, correct? Also, to "yaw" is logically equivalent to rotating the N "look" axis about the V "up" axis? I think the problem is that I am rotating N about Y, not about V, and I know of no way to rotate N about an arbitrary axis, other than the matrix transform I already have:

Code:
static void
update(Camera3D* eye)
{
    /* Normalize u, v, n */
    vec3D_norm(eye->u);
    vec3D_norm(eye->v);
    vec3D_norm(eye->n);
    
    /* Set up transform */
    eye->Mv[0] = eye->u[0];
    eye->Mv[1] = eye->v[0];
    eye->Mv[2] = eye->n[0];
    eye->Mv[3] = 0.0;
    eye->Mv[4] = eye->u[1];
    eye->Mv[5] = eye->v[1];
    eye->Mv[6] = eye->n[1];
    eye->Mv[7] = 0.0;
    eye->Mv[8] = eye->u[2];
    eye->Mv[9] = eye->v[2];
    eye->Mv[10] = eye->n[2];
    eye->Mv[11] = 0.0;
    eye->Mv[12] = -vec3D_dot(eye->u, eye->p);
    eye->Mv[13] = -vec3D_dot(eye->v, eye->p);
    eye->Mv[14] = -vec3D_dot(eye->n, eye->p);
    eye->Mv[15] = 1.0;
    
#ifdef DEBUG
    printf("eye location: (%f, %f, %f)\n", eye->p[0], eye->p[1], eye->p[2]);
    printf("right vector: (%f, %f, %f)\n", eye->u[0], eye->u[1], eye->u[2]);
    printf("up vector:    (%f, %f, %f)\n", eye->v[0], eye->v[1], eye->v[2]);
    printf("look vector:  (%f, %f, %f)\n", eye->n[0], eye->n[1], eye->n[2]);
#endif
}
So I guess the final question is how to rotate N about V, and in what space this should be done. Note that I do not have time to implement a quaternion class and would like to use what I already have.