But that's just it, what do you really think happens when you say 'you are the one rotating'. You are rotating your view vector and changing your position, and unless you perform the transformations manually you probably pass
1) the position of the camera
2) the view vector (in worldspace)
3) the up vector
and then it uses that to rotate the world around you before drawing! The glu library supplies gluLookAt and I know I"ve seen similar in d3d code.
EDIT: ohh I think I somewhat might have mis understood what you were saying, the total effect is different right? I believe I was confusing what you said with something different.