Which API should a hobbyist start with?

I've always been interested in being able to do some graphics programming. I am no professional programmer, yet, but I'd really like to be able to do some 2D/3D programming to animate the problems I might solve with programming.

I am really interested in math, and a lot of my time programming goes into solving mathematical problems or doing something math based.

This is why I come to you guys for help; is there some API I should be looking at that isn't insanely complex, but will still allow me to do 2D or 3D graphics with enough power?

Should I simply go for direct3d? Or OpenGL?

Any input will be greatly appreciated, thanks.