SDL has a cleaner design in my opinion, especially the newly released SDL2.
Also if you are doing something sophisticated with the graphics, consider using OpenGL on top of these libraries.
That...