VS 2005 Express has some good and some bad sides. It has many, many very useful features. However, to use this features you will need to configure it properly and you will end up one day asking for help here when all you did was chosing the wrong feature. Other compilers don't even have those features and you will not get frustrated by chosing the wrong one.
From my personal experience with the full versions of Visual Studio, I'd say that it's the best IDE for C/C++ programming around. The free version will likely be the best free IDE around, though I have not used the other free IDE's extensivly.
Short version: yes, VC Express is a great tool. As any great tool, it's not easy to use, you will need to learn both, coding C and using VC. But it's worth using and worth learning.
I do support the book suggestion. Get a good book. It helps a lot.
Starting out with C you will code a lot of unimpressive commandline ( aka DOS Window ) programs. Though VC Express will let you drag and drop a windows interface, you will never learn to properly make it work behind the fancy graphics if you don't do the boring commandline excercises. Don't expect to have anything to show off to your friends in the first year. C is powerful, but not easy. And a good book helps a lot in deciding what to learn next, because the glitter stuff that makes your friends go "aaaaah" is not what you need to master C. Though I'm sounding like a broken record here, you will need to do the boring excercises. All of them. By the book ( or online tutorial ).