Would a portfolio consisting only of games be a bad way to go?
Nope - as long as you're smart about it. The skills you pick up making games can be very useful in lots of areas other than games. I know several programmers who skipped college, had experience primarily in games and landed good jobs. One of them worked in games, the other two ended up using their knowledge of graphics to build graphing / data visualization software at different companies.

Since you currently have a job that (I assume) meets your basic financial needs, my advice to you is not to give up. I would guess that you will find what you're looking for in less time than it would take you go through college. Also be careful however, to never be arrogant and make sure you work on your professional and people skills. In hindsight, I found that college was a colossal waste of time for me in terms of technical skills, but it's value for me was in the things I didn't know I didn't know. I learnt a lot about how to be creative, how to jump through hoops (you might avoid having professors make you do it, but you probably won't avoid having managers make you do it), how to communicate with my team better, where to find good literature in my field, etc. Being active in online communities, open-sourcing little projects on Github, attending local tech / user groups, etc... are all very valuable in my opinion.

edit:

Or at least tips on what they look for when hiring a new programmer
Every person and every company is different, so do your homework on what they're looking for, but also be yourself. Personally, when I interview I want to see that in general you're well-informed about the current state of the industry, that you're good at general problem-solving and coding (rarely specific languages), and that you can communicate well (i.e. that you're upfront about what you do and don't know, that you can explain how you're thinking through a problem, etc.).