Tomorrows issue of Nature has an interesting interview with Uri Simonsohn, a researcher that has been investigating several cases of data fabrication in psychology by requesting the raw data and looking for statistical anomalies. In the interview, he explains that
The basic idea is to see if the data are too close to the theoretical prediction, or if multiple estimates are too similar to each other. I looked at several papers by Smeesters and asked him for the raw data, which he sent. I did some additional analyses on those and the results looked less likely. I’ll be submitting a paper on the method this week.
He goes on to state
I don’t know how systemic the crime is. What’s systemic is the lack of defences. Social psychology — and science in general — doesn’t have sufficient mechanisms for preventing fraud. I doubt that fabrication is any worse in psychology than in other fields. But I’m worried by how easy it was for me to come across these people.
This fits with some recent studies on fabrication and results that can be explained with low statistical power alone that I had highlighted previously. While my sympathy is with all honest scientists whose results may now falsely be reduced in credibility, I think it’s very good that people seem to become increasingly aware of the problem (or is the problem itself increasing?) and try to act against it. Ultimately, however, I fear that no control will be able to prevent scientific fraud when we do not maintain a level of scientific ethics in the community that precludes the idea in the first place. As I said before, too much pressure and too much of an “output-oriented culture” might do more harm than good on the long run to achieve this goal.
Addition 8.7: Some follow-up links: two posts by Ed Young, and another ScienceInsider post.
theoretical ecology 