Concise explanation of statistical errors
by
Adam
11/23/2007, 11:02 PM #
Given a response variable (y) and two control variables (x1, x2). Assume the following model:
y = a1*x1 + a2*x2 + a3*(x1*x2)
Suppose your statistical test show that a1, a2, and a3 all significantly different than zero. The next appropriate step is to test for differences between x1 at a constant value of x2. That way one can determine if a really large value of a3 is artificially inflating the values of a1 and a2. This is basic statistics.
Examining the study: Let x1 be gender and x2 be fancy/nonfancy drink. Y is wait time.
Many people have suggested that women order extremely fancy drinks which take a long time to prepare. That would suggest a3 could be very large. Therefore the authors should test for the effect of gender at a fixed level of fancy drink (say non-fancy).
If the women are still served more slowly then men then one can make the case that a1 is significant and there is a gender bias. On the other hand, if the times are the same, then a3 is likely large and one can credibly assume that women order fancier drinks.
In the paper the authors didn't include a gender and drink type interaction term in their model (why I wonder?). Nor did they compare the wait times for men and women at a fixed level of fanciness.
But you can! Look at figure 1 in the paper. Focus on the histogram of wait times for men and women ordering non-fancy drinks. Even in the absence of a statistical test it's pretty obvious that men and women are served equally quickly.
Therefore you can conclude that a1 is rather small. What the authors of the study have actually proved is that a3 is very big.
How about that! Women order very fancy drinks at coffee shops, drinks which take a long time to make. We can thank the students of Middlebury College and Professor Whatever for proving it to us.