I've considered what you're talking about. It does seem like fun to dig into a grittier, more challenging OS. On the other hand, it seems like cutting off your nose to spite your face if you have to take a massive hit to your daily productivity, stress levels, etc. to do it. All Linux is Linux. You can dig deeper and learn more even if you have the shiniest GUI interface covering your system. That's the way I see it. When I'm in the mood to have things "just work", I often get that from Ubuntu. On the other hand, nothing prevents me from trying to do some systems programming or something.
I'm not saying either is right or wrong, I just think you can have it both ways if you have the mind to.
