John Allen Paulos in the New York Times Book Review:
Bayes’s theorem, named after the 18th-century Presbyterian minister Thomas Bayes, addresses this selfsame essential task: How should we modify our beliefs in the light of additional information? Do we cling to old assumptions long after they’ve become untenable, or abandon them too readily at the first whisper of doubt? Bayesian reasoning promises to bring our views gradually into line with reality and so has become an invaluable tool for scientists of all sorts and, indeed, for anyone who wants, putting it grandiloquently, to sync up with the universe. If you are not thinking like a Bayesian, perhaps you should be.
At its core, Bayes’s theorem depends upon an ingenious turnabout: If you want to assess the strength of your hypothesis given the evidence, you must also assess the strength of the evidence given your hypothesis. In the face of uncertainty, a Bayesian asks three questions: How confident am I in the truth of my initial belief? On the assumption that my original belief is true, how confident am I that the new evidence is accurate? And whether or not my original belief is true, how confident am I that the new evidence is accurate?
More here.