Are most scientific studies screwed up?

[Insp. Clue!So?]

http://volokh.com/posts/1189863895.shtml

From the blog:

[Jonathan Adler, September 15, 2007 at 9:44am] Trackbacks
Are Most Scientific Studies Sloppy?

This week's "Science Journal" column in the W$J (link for subscribers) reports on the interesting and provocative research of John Ioannidis, who argues that the results of most published scientific studies are wrong.

Dr. Ioannidis is an epidemiologist who studies research methods at the University of Ioannina School of Medicine in Greece and Tufts University in Medford, Mass. In a series of influential analytical reports, he has documented how, in thousands of peer-reviewed research papers published every year, there may be so much less than meets the eye.

These flawed findings, for the most part, stem not from fraud or formal misconduct, but from more mundane misbehavior: miscalculation, poor study design or self-serving data analysis. "There is an increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims," Dr. Ioannidis said. "A new claim about a research finding is more likely to be false than true." . . .

Statistically speaking, science suffers from an excess of significance. Overeager researchers often tinker too much with the statistical variables of their analysis to coax any meaningful insight from their data sets. "People are messing around with the data to find anything that seems significant, to show they have found something that is new and unusual," Dr. Ioannidis said.

He further argues that only a fraction of incorrect studies are ever corrected or retracted, meaning that there are many published studies still "on the books" that support erroneous findings. It's an interesting and provocative thesis.