The video card hardware essentially contains the graphics algorithms that were learned in the DOS days and even before that. The algorithms are in the hardware and are extremely fast. Rasterization, filtering, perspective correct texture mapping, anti-aliasing, vertex shaders, pixel shaders, etc, are all done on the GPU. You could never hope to achieve the effects and quality of current games without having specialized hardware. It is possible but not practical nor feasible. Anything done in software can be done in hardware but the hardware will always be faster. So the same paradigm exists between hardware and software. Essentially anything done in hardware can be done in software albeit a bit slower.
The modern video card is probably as complex as the CPU and I would guess it processes faster or perhaps is more parallel in execution. Not to mention it does not have to service the entire computer system nor perform any interrupts, etc. So it has less to do and yet also has a bit more to do than the CPU. A pixel shader essentially means that a small program is running per pixel for all pixels of some rendered object. A vertex shaders means that a small program is running for each vertex of some rendered object. Vertex and pixel shaders are normally paired. That is a lot of instructions flying around under the hood and yet the video card barely suffers any performance hit while using shaders. It's actually quite amazing what we take for granted as the modern video card. It is northing short of amazing what it is really doing at the hardware level. We've come a long way from the old video cards.