Isn't it likely that the type of person who would respond to Slate's questionnaire is exactly the type of person who would respond to being tagged in the 25 things chain? I would suspect that that would skew the data significantly, probably pushing the respond time data toward the low end, but I am not a statistician.

The biases in something like this are numerous and the author surely accepts that there are plenty of ways to challenge the sample. But there's another kind of bias - the people that are annoyed by the meme and have no intention of participating but are eager to voice that. That would apply to a respondent like me.

spackle:

The biases in something like this are numerous and the author surely accepts that there are plenty of ways to challenge the sample. But there's another kind of bias - the people that are annoyed by the meme and have no intention of participating but are eager to voice that. That would apply to a respondent like me.

Is that another standard of bias or an entirely different "disease"?

I suppose only people with intense or extreme opinions would bother to respond to any questionnaire (and people who are really, really bored). I would hope that all statistical analysis would account for this, but the way polls and surveys are touted as some form of revealed truth gives me doubts.

For what it's worth, many of the people who responded noted that they had no intention of writing their own list, but still noted when they were first tagged. So while it's definitely possible that a selection bias is present, I don't think our sample only accounts for those eager to participate in the meme.

I suppose only people with intense or extreme opinions would bother to respond to any questionnaire

Yes, and that's why the poll would be skewed. It's like the reviews on Amazon - they're not really meaningful, because a large number of people simply wouldn't feel strongly enough to bother posting a review, so the results you get won't be realistic. The squeaky wheel may get the grease, but that doesn't mean its squeaking conveys anything of real importance.