Don't Trust Scientific Studies -- They're Probably Wrong

I've been saying this for a long time.

The New Scientist has a devastating piece on the validity of published scientific research, titled "Most Scientific Papers are Probably Wrong."

Most published scientific research papers are wrong, according to a new analysis. Assuming that the new paper is itself correct, problems with experimental and statistical methods mean that there is less than a 50% chance that the results of any randomly chosen scientific paper are true.

John Ioannidis, an epidemiologist at the University of Ioannina School of Medicine in Greece, says that small sample sizes, poor study design, researcher bias, and selective reporting and other problems combine to make most research findings false. But even large, well-designed studies are not always right, meaning that scientists and the public have to be wary of reported findings.

"We should accept that most research findings will be refuted. Some will be replicated and validated. The replication process is more important than the first discovery," Ioannidis says....

Solomon Snyder, senior editor at the Proceedings of the National Academy of Sciences, and a neuroscientist at Johns Hopkins Medical School in Baltimore, US, says most working scientists understand the limitations of published research.

"When I read the literature, I'm not reading it to find proof like a textbook. I'm reading to get ideas.

Read the whole thing here.

But that is not how scientific research is presented to the public. In the current political climate "science says" is considered to trump all other sources of information -- it is considered to be more important than religious or moral or even ethical considerations and definitely more so than mere political ones. For decades social and political activists have produced "studies," many of them irreplicable, "proving" whatever point they want to establish. This naive faith in the superiority of credentialed scientific authority and the efficacy of peer review is a dangerous thing and should be strongly challenged. Remember that the greatest horrors of the past century -- Marxism and Nazi race theory both claimed the mantle of "science."

For earlier posts on the collapse of scientific credibility see here, here, and here.