I'm writing a space game and I'm using a huge sphere centered around the player as the backdrop. A skybox didn't quite look right so I switched to a sphere. Here is my problem.

When you fly straight up or straight down and attempt to move left or right - the craft spins. This is fine on planet earth since it is how aircraft really act. However in space this is not good. I want the craft to move left and right at any angle of rotation. In other words I don't want the x rotation mapped over the y rotation at any point in time. All rotations need to be separate. However with my current design, and no I'm not yet using quaternions, it doesn't work.

Do I need to use the matrix for rotation about an arbitrary axis for all rotations and dump the x,y,z rotational matrices?

I'm sorta stumped.