The inadvertent? image-duplication in the human clone paper still creates some unrest, maybe less so with regard to what this means for the core results of the study (it seems most people think that those will hold) than with regard to the thoroughness of the peer-review and the ability to detect irregularities in high-profile publications (see recent news features in Science and Nature 1, 2, 3).
I don’t know if the image duplication should have been obvious, but the debate about the responsibility of the reviewers seems a bit exaggerated to me. As a reviewer, I see it as my job to judge the scientific importance and validity of the study that is presented to me, but I would usually assume that results are reported truthfully and correct. If it’s really necessary to systematically check papers for fraud or careless mistakes, I guess journals should install appropriate checks. And after all, as long as this was just a small mistake with no bearing on the results, we can easily correct this online nowadays, so no harm done.
What I found interesting about the story though is that the image duplications was first mentioned on a website (PubPeer) that offers the possibility to write anonymous comments in response to published scientific paper. PubPeer states that they try to provide an independent, anonymous platform for post-publication peer-review. I browsed a bit on the site, and it seems to me that there are a few quite good comments out there, although at the moment practically only on biology papers, I couldn’t find any comments on an ecology paper so far.
If widely used, such a post-publication comment system could probably really have a huge potential. There must be a wide array of useful hints and comments on any paper (not only about formal errors), but at the moment this information is scattered across personal communications, journal clubs, blogs and internet forums, even though some Journals such as PLOS have quite good comment functions already. Making this discussion available seems really interesting. One small caveat is that I didn’t find the “forum-type” website of PubPeer ideal, it certainly serves it’s purpose for small comments, but I think it would also be great if you could make more formal replies, with proper typesetting, the option to reveal your name, and permanent ID and histories for all comments, so that comments become citable. Also, it would of course be very interesting to have the pre-publication reviews there as well, along with the response of the authors. Technically, all of this would be no problem – a slightly modified stack-exchange website would already do the trick, but it probably requires a big change of thinking so that this really becomes widely used.
theoretical ecology 
PubPeer has taken off basically because there is an active community out there on the lookout for image manipulations in cell & molecular papers. I’d be surprised if PubPeer were to grow into the sort of general-purpose post-publication commenting system you suggest.
And the fact that you don’t see it as your job as a reviewer to check for this sort of carelessness or misconduct on the part of authors just shows that you aren’t a cell or molecular biologist! 🙂 Image manipulation (inadvertent or otherwise) clearly is sufficiently common in those fields, and caught sufficiently often by readers, that journals ought to have systematic procedures to check for it (and indeed, many do), *and* referees ought to be on the lookout for it. But in ecology, sloppiness or misconduct is either rare, or else takes forms that are much harder to detect than image manipulation. So yeah, in ecology we can’t really expect referees to catch most authorial screw ups or misconduct, nor can we really expect journals to systematically check for those problems.
LikeLike
Hi Jeremy,
I guess you’re right that there’s nothing to suggest post-publication review will take off any time soon in ecology, either at PubPeer or elsewhere, but I still think it would be a great thing if we had such a system running and used – I’m sure a lot of people could make interesting comments, either on problems starting from inappropriate data sources to statistical methods up to unsupported conclusions, or simply congratulating the authors on a great study.
About the responsibility of the reviewers: if there’s a good chance that deliberate manipulations are taking place, yes, reviewers should look out for that, although I still think mechanical tasks should be taken over by the journals, it doesn’t make sense to waste the time of a scientific reviewer on what could be done by a computer program or an editorial assistant.
Sloppy errors such as mislabeled figures, however, are in my opinion firstly the responsibility of the authors, secondly those of the journals and only then those of the reviewers. That’s why I find it funny the reviewers and the journal are blamed, but the authors are to a much lesser extend. They knew they probably had the most exciting result of their career – you would think they would take care to select the correct images for their figures (I’m assuming here that this was indeed an honest error, but I find this rather probably, they would have to have really good nerves to submit a fake on a study that will foreseeably immediately be replicated around the world).
LikeLike
Just to further depress you:
http://dynamicecology.wordpress.com/2012/10/11/in-praise-of-pre-publication-peer-review-because-post-publication-review-is-an-utter-failure/
🙂
Re: blame for publication of mislabeled figures, duplicated images, etc., I’d say there’s plenty of blame to go round. Like you, I’m flabbergasted that authors of high profile papers in particular could ever do something as sloppy as submitting a paper with the wrong images in it. Which is why, like many folks, I suspect such “mistakes” are mostly intentional. Workers in certain areas of molecular biology are moving in the direction of cyclists in the Tour de France–presumed guilty of cheating until proven innocent. And not entirely without reason…
LikeLike
hehe, I was aware of your post, so I’m only mildly depressed 🙂
Well, I don’t think post-publication review should completely replace pre-publication review (it might though to some extend), but I definitely think it’s a valuable addition to any level of pre-publication reviewing – given your recent post http://dynamicecology.wordpress.com/2013/06/06/on-ignoring-criticisms/ , wouldn’t you think it’s useful to write “WARNING: see Mayfield & Levine (2010)” under a few papers 😉 ? I think the option to directly engage with published papers might make the rebuttal of an idea a lot more efficient than just putting it out there where it can savely be ignored.
LikeLike
As I said in the comments on that old post, it’s worth a try, sure. I’m just skeptical that post-publication review will ever take off, or that it will do much good if it does.
Here’s another line of evidence on how the sort of post-publication review you’re hoping for is unlikely to take off. A fair bit of science blogging is people summarizing recently-published papers. In my experience, the vast majority of such posts add very little original commentary, and when they do it’s usually either positive, or focuses on possible next steps. So even the rare people who are inclined to take the time to make post-publication comments on the recent literature hardly ever say anything critical.
LikeLike
That’s true, but I think this is also because many people tend to choose articles they like to blog about (at least I do), and because many bloggers are rather on the young side and should probably be a bit careful to offend random people until they have a more permanent position … if you want critique though, you might like my next post waiting in the pipeline 😉
LikeLike