Nate Silver and Justin Wolfers are having a friendly blog-dispute about momentum in political polling. Nate and Justin each make good points but are also missing parts of the picture. These questions relate to my own research so I thought I'd discuss them here.
There ain't no mo'
Nate led off the discussion by writing that pundits are always talking about “momentum” in the polls:
Turn on the news or read through much of the analysis put out by some of our friends, and you're likely to hear a lot of talk about “momentum”: the term is used about 60 times per day by major media outlets in conjunction with articles about polling.
When people say a particular candidate has momentum, what they are implying is that present trends are likely to perpetuate themselves into the future. Say, for instance, that a candidate trailed by 10 points in a poll three weeks ago — and now a new poll comes out showing the candidate down by just 5 points. It will frequently be said that this candidate “has the momentum”, “is gaining ground,” “is closing his deficit,” or something similar.
Each of these phrases are in the present tense. They create the impression that — if the candidate has gone from being 10 points down to 5 points down, then by next week, he'll have closed his deficit further: perhaps he'll even be ahead!
But, as Nate points out, this ain't actually happening:
Say that a candidate has improved her position in the polls from August to September. Is her position more likely than not to improve further from September to October? . . . this is not what we see at all . . . Sometimes, a candidate who has gained ground in the polls continues to do so; otherwise, the trend reverses itself, or the race simply flatlines. . . . There is also no sign of momentum we look at the change in polling between other periods. . . . In general elections, the direction in which polls have moved is not predictive of the direction in which they will move. [italics added]
I like Nate's analysis. It's very much in the Bill James style, but with graphs.
Consider the time scale
Enter Justin Wolfers, who writes that Nate is all wrong, that there is momentum in political polling.
Justin argues that Nate made a mistake by using the same data in his “before” and “after” comparison.