Rotating a point using a quaternion, then using an inverse quaternion to rotate back?

I have a function that takes a unit quaternion <w,x,y,z> and a point <x,y,z> and spits out the rotated 3d point.

void rotate(const float *quaternion, const float *point, float *newpoint)

This code works OK:

void rotate(quaternion, point, newpoint)

I would now like to rotate the point back to where it was originally. I understand this is possible using the inverse of the quaternion (which for a unit quaternion is equivalent to the conjugate, right?):

This is how I do this this:

inverse_quaternion[4];

inverse_quaternion[0] = quaternion[0]; //w

inverse_quaternion[1] = -quaternion[1]; //x

inverse_quaternion[2] = -quaternion[2]; //y

inverse_quaternion[3] = -quaternion[3]; //z

void rotate(inverse_quaternion, rotated_point, final3dpoint)

Unfortunately this is not doing what I expect. What am I missing?