I don't see why women should have a form of privacy from men but not from each other unless there is something particular about men and women being different in a fundamental way (which is what the gay marriage opponents argue) that we intuitively acknowledge.

You do bring up a very important point that needs to be addressed. It's the real crux of the overall argument which people are loathe to go into because it opens up a pandora's box of questions.

We already have a separate but equal class system: The legal identities of Male and Female. This class system is seldom analyzed, and we have a long history of legal precedent that enshrines it into law and encodes it into our very culture, to the point that we feel we can "intuitively" tell the difference between these classes, that this difference is right and good, morally and legally as it should be.

Segregation of the sexes is being challenged in many areas, and not just by same sex marriage, but also by social phenomena such as transgender law, legal sex changes, and increasing acceptance of androgynous people and lifestyles that we see in GLBT communities.

The defeat of the Equal Rights Amendment was an attempt to abolish the separate but equal class system. It failed largely because society simply wasn't ready for the idea that sex differences are not definitive of personhood (imho).

The question of same sex marriage is tied up in this idea as well... and its a question that begs answering. Are the rights of marriage necessarily sexed? Or have they been arbitrarily sexed by tradition. If so, should we/can we adapt marriage to reflect the movement towards gender neutrality we now see in modern society?

To answer your question... you don't have the "right" to go to the women's locker room because publically you aren't recognized as a woman, you have no legal standing as a woman to sue for entry, and society has reserved certain things as either only for men or only for women. Sometimes segregation is legally mandated (like birth certificates, the draft), sometimes it's just an unofficial social code that you must obey or people will get pissed off (like Jim Crow, and yes men's and women's restrooms).

Same sex marriage is just another challenge to the idea that the sexes MUST stay put in their proper places or else all is lost. We really hated being challenged on race and still do. But as a society, we seem to hate challenges to sex roles more than any other kind.