Hugo Mercier in Psychology Today:
Reasoning is funny. Not the act of reasoning-although it can also be fun-but the cognitive skill that allows us to figure out math problems and decide what computer to buy. We can agree that reasoning is responsible, at least in part, for some of the greatest achievements of humankind, from calculus to the International Space Station. Yet we should also agree that people can reason their way to asinine beliefs — we are poisoned by the souls of aliens dropped in volcanoes millions years ago — and disastrous decisions — Napoleon's ‘invasion' of Russia. And it's not that some people are smart while others are dumb: Isaac Newton spent more time bent on alchemy than on mathematics and, for their many faults, neither L. Ron Hubbard nor Napoleon were dunces.
Psychologists interested in reasoning cannot expect to recreate such dramatic events as the creation of calculus or the decision to attack Russia in the lab. Instead, they must work with ‘toy models,' simple problems that people can try to solve in less than half an hour before going back to their routine. The most commonly used of these problems is known as the Wason Selection Task — after its creator the pioneering psychologist Peter Wason. Many of you may already be familiar with it, but if you aren't, you can try to solve the problem yourself – it's laid out in figure one. Take your time.
Did you answer A, or A and 4? If yes, you are in good company, comfortably sitting with the large majority of the thousands of people who've faced the Wason Selection Task. But psychologists would not be nearly as interested in a problem that most people get right: the correct answer, arrived at by about 10% of participants, is A and 7. Why 7? Because if there's a vowel and the other side of the 7 card, then the rule is false. Why not 4? Because even if there's a consonant on the other side of the 4 card, the rule can still be true: it doesn't say that a consonant must have an odd number on the other side. Here's a nice toy model of reasoning's failures: a simple task leading to abysmal performance.
Now for the toy model of reasoning's successes. It is called the Wason Selection Task — after its creator the pioneering psychologist Peter Wason. It is depicted in figure one. A large majority of people manages to find the correct solution, showing how efficient reasoning can be at solving logical problems.
Wait. How can the very same problem illustrate both reasoning's failures and its successes? The task is the same. The people solving it are the same. What's the catch? The context. In the first toy model, people face the problem on their own. In the second they solve it in groups.
More here.