Amen. I didn't pick up on pen-and-paper RPG's until college, so I missed out on that experience.
I still play D&D weekly, and dabble in a couple other systems. All of my "gamer friends" have decent jobs; most of us graduated college; about half of us are married (to other gamers, though some of the "gamer wives" have picked up the dice in self-defense, I think), and a few have started families.
It hasn't lead to any great works of art from us, but D&D has lead to various pieces of fiction (Writing for fun! Who knew that our comp. sci. guy would do that?), as well as campaigns focused on story and character, as opposed to "hack, slash, and loot!" The character backgrounds we write up vary from shallow to ten-page soap operas. It also is a way to meet new people, which as a shy introvert, I value.
And if people want to call me a geek or a dork for my pass time, so be it. IMHO, people spend too much time evading their inner dork instead of embracing it.