Power of Positive Thinking Skews Mindfulness Studies

Anna Nowogrodzki in Scientific American:

There’s a little too much wishful thinking about mindfulness, and it is skewing how researchers report their studies of the technique. Researchers at McGill University in Montreal, Canada, analysed 124 published trials of mindfulness as a mental-health treatment, and found that scientists reported positive findings 60% more often than is statistically likely. The team also examined another 21 trials that were registered with databases such as ClinicalTrials.gov; of these, 62% were unpublished 30 months after they finished. The findings—reported in PLoS ONE on April 8— hint that negative results are going unpublished. Mindfulness is the practice of being aware of thoughts and feelings without judging them good or bad. Mental-health treatments that focus on this method include mindfulness-based stress reduction—an 8-week group-based programme that includes yoga and daily meditation—and mindfulness-based cognitive therapy.

A bias toward publishing studies that find the technique to be effective withholds important information from mental-health clinicians and patients, says Christopher Ferguson, a psychologist at Stetson University in Florida, who was not involved in the study. “I think this is a very important finding,” he adds. “We’ll invest a lot of social and financial capital in these issues, and a lot of that can be misplaced unless we have good data.” For the 124 trials, the researchers calculated the probability that a trial with that sample size could detect the result reported. Experiments with smaller sample sizes are more affected by chance and thus worse at detecting statistically significant positive results. The scientists’ calculations suggested that 66 of 124 trials would have positive results. Instead, 108 trials had positive results. And none of the 21 registered trials adequately specified which of the variables they tracked would be the main one used to evaluate success.

More here.