Carl Zimmer in Nautilus (illustration by Jeffrey Alan Love):
People often think that the job of scientists is to prove a hypothesis is true—the existence of electrons, for example, or the ability of a drug to cure cancer. But very often, scientists do the reverse: They set out to disprove a hypothesis.
It took many decades for scientists to develop this method, but one afternoon in the early 1920s looms large in its history. At an agricultural research station in England, three scientists took a break for tea. A statistician named Ronald Fisher poured a cup and offered it to his colleague,Muriel Bristol.
Bristol declined it. She much preferred the taste of a cup into which the milk had been poured first.
“Nonsense,” Fisher reportedly said. “Surely it makes no difference.”
But Bristol was adamant. She maintained that she could tell the difference.
The third scientist in the conversation, William Roach, suggested that they run an experiment. (This may have actually been a moment of scientific flirtation: Roach and Bristol married in 1923.) But how to test Bristol’s claim? The simplest thing that Fisher and Roach could have done was pour a cup of tea out of her sight, hand it to her to sip, and then let her guess how it was prepared.
If Bristol got the answer right, however, that would not necessarily be proof that she had an eerie perception of tea. With a 50 percent chance of being right, she might easily answer correctly by chance alone.
Several years later, in his 1935 book The Design of Experiments, Fisher described how to test such a claim. Instead of trying to prove that Bristol could tell the difference between the cups of tea, he would try to reject the hypothesis that her choices were random. “We may speak of this hypothesis as the ‘null hypothesis,’ ” Fisher wrote. “The null hypothesis is never proved or established, but is possibly disproved, in the course of experimentation. Every experiment may be said to exist only in order to give the facts a chance of disproving the null hypothesis.”
More here.