Gary Gutting in the New York Times:
Public policy debates often involve appeals to results of work in social sciences like economics and sociology. For example, in his State of the Union address this year, President Obama cited a recent high-profile study to support his emphasis on evaluating teachers by their students’ test scores. The study purportedly shows that students with teachers who raise their standardized test scores are “more likely to attend college, earn higher salaries, live in better neighborhoods and save more for retirement.”
How much authority should we give to such work in our policy decisions? The question is important because media reports often seem to assume that any result presented as “scientific” has a claim to our serious attention. But this is hardly a reasonable view. There is considerable distance between, say, the confidence we should place in astronomers’ calculations of eclipses and a small marketing study suggesting that consumers prefer laundry soap in blue boxes.
A rational assessment of a scientific result must first take account of the broader context of the particular science involved. Where does the result lie on the continuum from preliminary studies, designed to suggest further directions of research, to maximally supported conclusions of the science? In physics, for example, there is the difference between early calculations positing the Higgs boson and what we hope will soon be the final experimental proof that it actually exists.
More here.