Zeynep Tufekci in Medium:
I’ve taught “introduction to research methods” to undergraduate students for many years, and they would sometimes ask me why they should care about all this “method stuff”, besides having a required class for a sociology major out of the way. I would always tell them, without understanding research methods, you cannot understand how to judge what you see.
The Hollaback video shows us exactly why.
The Hollaback video also shows why “data” without theory can be so misleading—and how the same data can fit multiple theories. Since all data collection involves some form of data selection (even the biggest dataset has selection going into what gets included, from what source), and since data selection is always a research method, there is always a need for understanding methods.
First, let’s list all the hypothesis compatible with the “data”, this video:
Hypothesis 1- Men of color are disproportionately more likely to catcall, especially to a white, conventionally attractive female.
Hypothesis 2- All men are equally likely to catcall but the makers of the video were biased, consciously or unconsciously, against black men (and edited out men of other races on purpose.)
2.a Consciously: they are racists and are playing to the “white women endangered by black men” trope — which has a long and ugly history, hence the concern raised by many over the past week.
2.b Unconsciously: There is a methodological twist to the research which creates this outcome.
2.c. Both 2.a. and 2.b are true.
Hypothesis 3- It’s a spurious correlation: there is some other reason that caused these two events to go together
The important methodological point is that the video, without further reflection, can support all three wildly incompatible propositions. In other words, if you just look at the video, you can believe any three, and you will likely choose whichever fits your existing conclusions and prejudices.
Let’s start with 3, the easiest to dismiss.
A spurious correlation occurs when a third, unrelated variable, causes a change in other variables, which then seem like they are causally connected even though they are not. And since the human brain is a narrative writing machine, seeing A and B together makes people write stories that tie A and B. A silly, but correct, example is the correlation between ice cream and murder: during months when more people eat ice cream, there are more murders. This is not because popsicles are good murder devices. This spurious correlation caused by a confounding variable: the season.
More here.