How, exactly, are gays being granted special rights? They are asking for the right heterosexual people already enjoy, to marry the person they want to share their life with. I guess it was pretty offensive when women got special rights to be able to vote, making them more special than anyone else (except men, who could already vote, of course).
I don't think the act of stopping a course of discrimination against someone, on any grounds, is on par with the act of conferring special rights.
If we said, gay people can get married now, and also, their marriage licenses will be free, and also, the justice of the peace has to move them to the front of the line and marry them before anyone else, and also, they get $1,000 marriage bonus, then yeah, we would be giving them special rights and I would object.
But refraining from denying them a right? nah, sorry. Not special.