Actually, part of his story is likely true.

He probably did move to a new city, but in reality, he is lonely and has low self esteem. And he probably has an attractive female neighbor, but doesn't have the self confidence to approach her. So he has concocted a fantasy about her and her friends. And there is probably a really attractive friend (the one who mysteriously has his mail) he uses for his extra fantasies.

Regardless of the length of his member, its his ego and self esteem which are the real issue.

As for actually turning away women, apparently the man is over 20 years old, right? But he has never turned a woman away? Not even an unattractive woman? Why does he lack this basic skill of "just say no"?

Yes, of course this is a fantasy. A grown man, even if he is above average, would know how to address this matter.