For that you'll need two speed variables speedX and speedY, or perhaps a vector with two components.
Well, change the direction. IE if it hits the right side, it is now travelling in the original y, and the inverse x. You can figure out the other sides using similar logic. So keep some speed, and on update set x += speed * dx and y += speed * dy.
But most of the better implementations of pong use something more than that. You could for instance do it using some trig as mentioned above.
If you want to reflect the particle against a plane, then you'll have to use the get the dot product between your vector and the normal and calculate the new direction. Something like -
R = V - (2 * N * dot(V, N));