Abstract We evaluated the predictions of politicians, journalists, and pundits (collectively, “prognosticators”) over a 16-month period surrounding the 2008 United States federal elections. We sought to test the accuracy of these predictions and understand what makes individuals unusually good or bad at prognostication. We evaluated a random sample of Meet the Press, This Week, and Face the Nation transcripts and printed columns from the most prominent American prognosticators. Ultimately, we determined that some prognosticators are significantly better than others, and there are characteristics that significantly impact a prognosticator’s predictive power.
After finding which characteristics make an accurate prognosticator, we ranked prognosticators based on outcomes. We found that a number of individuals in our sample, including Paul Krugman, Maureen Dowd, Ed Rendell, Chuck Schumer, Nancy Pelosi, and Kathleen Parker were better than a coin flip (sometimes, substantially so.) A number of prognosticators were frequently inaccurate, including Cal Thomas, Lindsey Graham, and Carl Levin. Our paper explores the reasons for these differences and attempts to evaluate the purpose of prognosticators in the media in light of their varying degrees of accuracy.
From the conclusion:
Overall, our results indicate that most prognosticators are not very accurate predictors, but only two are worse than a coin flip (with statistical significance). Few offer reliably accurate predictions, but even fewer are wrong more than half of the time – most barely hover above the dreaded “ugly line.” This should be startling, considering the number of Americans who rely on these prognosticators for their supposedly superior knowledge of the political environment.
Given this sad reality, who should you listen to? Good predictors tend to be liberal and are not lawyers. More rigorous study can confirm our findings, especially the question of whether partisanship has an impact on an individual’s ability to make accurate predictions.