I was happily surprised to pull up Slate yesterday and discover that my paper with a group of students had been written up by Mr. Hartford. I've been reading through the comments here and thought that I'd address a handful of them. Many of the others are already addressed in the paper, if you're so inclined.

First, "better latte than never" cracked me up.

Second, I don't think that people who know me would classify me as anything like a crazed feminist looking for proof for conspiracies against women in every corner. This was a student-designed project by a class with both men and women in it and none of us were sure what the outcome would be. To those who argue that 20 second extra wait isn't worth getting upset about, I agree that 20 seconds isn't a very long time. But that doesn't mean that the differential is not important. Despite anecdotal evidence that it exists, there is very little evidence on discrimination in small ticket consumer markets. Our finding that women wait about 25 percent longer than men is significant and robust to a large variety of specifications. (In response to a comment, we include multiple models to see if we can "kill" the significant differential with the addition of progressively stricter controls. We find that we can't.) The big deal isn't really that women might wait longer for coffee, but that this suggests that maybe women are waiting longer in other "little" markets like this. These cumulative effects (if they exist—the study mostly motivates looking at the issue) are likely important. In other words, I'm not up in arms about 20 seconds, but knowing whether there is routine discrimination of this sort is important. If women frequently wait 25 percent longer for service, that's a bigger deal. And, as Mr. Hartford points out, understanding why the difference persists despite "competition" is also important.

I've been really interested in the alternative "non-discriminatory" explanations put forth here. We consider a few in the paper. This may represent employees wishing to inflict a cost on women, but we think that that is the least compelling of the possibilities. It may also be employees wanting to chat with women more then men. Or it may be that employees expect women to be "easier" customers who are less likely to complain. It may also be that women are worse tippers. Economists would call all of these "discrimination," although each represents a different type. (The latter two would be "statistical discrimination," or profiling based on expectations about the behavior of women.) I thought that the point that this may represent employees taking more care with a woman's order was a great one; we hadn't considered it. As a note, we did control for things like type of payment and we didn't start recording time until after the customer completed the order.

On to a few technical points. The paper clearly (and admittedly) is based on a research project designed and conducted by undergraduate students for a class at Middlebury College on how to define, identify, and measure discrimination in labor and consumer markets. Because of limited participants and time, we could not design a larger study that would have provided larger sample size or greater geographical coverage. That said, I see no reason to expect that Boston and/or these particular shops are somehow the site of more discriminatory treatment than any other locations. And a sample of 277 transactions is large enough to allow for a variety of controls for factors other than gender that might have affected wait times. I think that the paper is fairly clear about the possible limitations of the study due to sampling technique and power issues. We are also careful to point out what unobserved factors may continue to bias the result and interpret the models with interactions as suggestive (but not as proving) that there is more here than simple differences in drink orders. We only identify coefficients as significant if they have a p-value below 0.05.

As for inter-coder reliability, most tests for this would be inappropriate given that each enumerator coded different coffee shops at different time. Instead, we include a model with both enumerator and coffee shop fixed effects to account for systematic differences across observers or stores. This model would account for systematic biases by certain enumerators in certain stores and for differences due to store types. The gender differential remains significant in this model.

In regards to the enumerator measures of customer "appearance," this is not a variable of interest unto itself, but an additional control in case we found that some groups of customers in the stores tended to be dressed more nicely than others. We standardized the measure by enumerator to account for difference in mean and variance by observer. In any event, it doesn't seem to matter much.

It's great that you wrote in...thanks for your "better latte than never" critique!

I am curious, however, just what exactly it is that you think really does explain your findings. Coffee shops are staffed primarily by young people like your students. Not only that, but they tend, as others have written, to be multiply-pierced, gay, or otherwise to be of a liberal persuasion. Are these likely candidates for misogyny? Even on this small scale?

And it seems extraordinarily difficult to go about your workday trying to take a few extra seconds for some orders and not for others. That would be an art in itself. Why bother?

I can tell you that I feel discriminated against by the female employees in places like Right Start and Pottery Barn kids; not so much in the sense of slow service, but rather in the sense that they are rushing me through my experience to try to get me out of there. I'm not sure I really am discriminated against, but there you are. I also feel that way when interacting with my Kindergarten and preschool-aged kid's teachers, who give my wife continual eye contact, but barely acknowledge me. I like the teachers very much, and doubt it is overt discrimination. But if you did an "eye contact" study I think you would find hard data to support my claim.

Anyway...thanks again for getting in on the discussion.

All well and good, but the primary flaw in this work is that the principle determinant of wait time, drink complexity, is poorly coded. This is not addressed in your response. In addition, I'm not sure the statistical tests applied were appropriate.

Consider the non-fancy drinks. The orders in this set should be most equivalent - there are many possible fancy drinks but only a few non-fancy ones. Therefore this set best reduces drink complexity as a source of error.

Looking at this data, which was provided in figure 1, it seems the difference in wait times was minimal. Most of the difference between the male and female means is due to 4 (?) orders with wait times over 150 seconds. Without those 4 orders, the wait time for women is substantially less than for men. I tried to estimate the underlying numbers (assigning the average value for each bin to every observation in that bin - including the 4 long waits) and apply a Wilcoxon ranksum test for equal medians. The resulting p-value was .88. I realize this is largely BS -- I don't have the real underlying data. Perhaps you could run the same analysis and report the p-value?

In general however, it seems that for the most controlled comparison there is no significant difference. It might have been a better idea to pick a typical drink often ordered by both men and women and examine the wait times for those groups, ignoring special orders.

If persisting with the regression method of analysis, it might be a good idea to toss the insignificant terms in the model and re-evaluate the model with only the significant terms.

This is getting good and buried in a sea of ranting, but I'll quickly respond to Adam anyway. The issue of drink complexity is a serious one and we address it throughout the paper and in the conclusion. We acknowledge throughout that it could be biasing the results. however, we address the issue in several ways and feel that the overall evidence suggests that drink complexity is not driving all the results. First, we point out that although sample sizes are too small to get much power, the differential indicates longer wait times for other minority groups as well. Second, we try adding more thorough controls for order by breaking them down by latte, cap, etc. The results remain the same. However, we recognize that this is not fully satisfactory since it was impossible to reliably overhear and quickly record every detail of the order. As an additional check, we note that the coefficients in models with interaction terms suggest that the gender differential changes with gender composition of employees and with line length. Neither interaction term is significant (p-values of 0.31 and 0.13). They are large in magnitude, so we're probably suffering from low sample size. We don't place too much weight on them, but conclude that this evidence also suggests that it seems like the differential might vary in ways that couldn't be explained by drink complexity.

As for outliers, you can have the data if you want. (Email me at work-- I'm easy enough to find.) We also ran the analysis with the bottom 1% and top 1% outliers removed as well as only with top outliers removed and the result remains.

I agree that a paired audit study would be nice- we didn't have the resources or good pairs of auditors to conduct one. But we still find these results to be suggestive.

Finally, removed non-significant variables is generally viewed as poor practice and indicative of data mining. In our case, it would make the result way more significant rather than less.

Thank you for posting Ms Myers. But if as you say, that your study is robust and significant, then why not do another to follow up to find the actual reason for the delay you found?

Not all of the criticism on the forum is ranting, or commentary on the worthiness of the study but some of it is actually pointed criticism. I will give my own:

It seems to me that your analysis of the reason for the delay is flawed simply because it, like the posters here, is not based on empirical observation.

That is, after observing the delay that women have you go on to argue the reasons without making further observations. Having established a delay, why not follow up with observations of interactions between orderer and server with respect to men and women. The variables could be for example, time taken to order, the amount of chat between orderer and waiter and so on.

For example, several posters, including myself, several coffee bar patrons who were women, and baristas noted anecdotaly that men tend to order succinctly and forcefully where as women are more likely to be chatty and friendly and that these interactions last into the preparation of the drink.

Another poster noted that the distribution of non-fancy drinks were exponential and of fancy drinks bell curve as would be expected from the number of steps taken to prepare each drink. And further, the number of fast served orderers is about the same. This would imply that a similar process is happening to a majority of both men and women.

Yet I note that while there are very few men (less than 5% for men of non-fancy drinks) at the long wait, there are few but significantly large (12% for non-fancy drinks) of females at the long wait. This would imply that there are some women who take significantly longer, skewing the wait. That is, that the wait is not necessarily a female vs male problem but a problem generated by certain women with certain ordering interactions. At least this is the implication to me.

Am I right? I don't know. But this argument and many of the others here in the forum are no better than the arguments presented for the cause of the delay in your study because they are simply not based on data.

By the study design you present in the paper and your explanation of how it was made here in your post, it would seem that a follow up study would be relatively easy to do by you and your students.

And if you really wanted to defend your point then you would do such a study.