Proceedings of Peerage of Science – complementing the journal landscape, or yet another new OA journal?

Entry site of Proceedings of Peerage of Science

Entry site of Proceedings of Peerage of Science

There is no consolidation on the journal front in sight. Rather, new (open access) journals are popping up all over the place. While I’d be happy to dismiss them all on the grounds of adding to the paper flood, it seems that we need these new players to get some changes going in the complacent system of established publishers; because at least some of those newcomers deliver real innovations while established journals seem unable to move. For example, I LOVE the option of PeerJ to comment on particular parts of a paper, this is GREAT, why can’t we have this everywhere?

In this spirit, I was looking at the latest addition to the journal ecosystem: Peerage of Science has create a new outlet, called “Proceedings of Peerage of Science” (henceforth ProcPoS). Under “Why another journal?”, the website states that

ProcPoS is different from other scientific journals. It does not publish new results, nor reviews of a particular topic, nor meta-analyses.

Instead, ProcPoS takes aim at already published research. ProcPoS publishes Commentaries, only. It is a platform for proper post-publication peer review.

In fact, ProcPoS is not publishing any commentary, but only material from reviews that have been written through Peerage of Science EDIT: this was a wrong interpretation of me, ProcPoS is publishing both reviews that have been written for Peerage of Science and commentary that has been submitted externally to Peerage of Science.

The main entry site looks all flashy and modern (see above), so no complaints there, and articles layout (see below) is fine, nothing special, but readable. The articles are OA and published with a business model that they call “Public Patronage Open Access”. I found the explanation on the website rather cryptic, but it seems you don’t have to pay for publishing, and you might even get some money back from people (patrons) that sponsor the article. Looking at the articles (see below), “patron” seems very much an euphemism for “advertiser” if you ask me, but OK, who cares if that makes the whole thing free. The way I understand it if someone (or a community) cares about the ads, they could always remove the ads by becoming a patron themselves.

So, is this what progress looks like? Unlike some other voices, I’m generally in favor of post-publication peer review. I don’t think it’s a replacement for per-publication screening, so I would say keep a pre-publication review for quality, but at the same time I think we urgently need to evaluate and consolidate more thoroughly what is written and measured instead of piling up “novelties”. I also think that it is generally useful to have reviews openly published, regardless of pre or post publication. In that sense, ProcPoS seems to be on the right track.

However, I also see a number of major problems with this new journal. First of all, the whole point of making reviews available is a) to draw attention to potential problems with a paper, and b) to get a discussion going. From that, I think fundamental requirements are a) to have links from the original publication to the comments b) to be able to comment on comments. Both points, as far as I understand, are not available through ProcPoS. Plus, it seems to me that publishing only selected reviews doesn’t make sense, why not make them all available?

In conclusion, the funding model is interesting, and in general it can’t be bad if people can make sensible comments about published research in a citable way. In an ideal world, I think the community would be better served by publishing comments and open reviews directly with the original publication, and allowing additional comments afterwards, which is supported by more and more journals. And, we also have already PubPeer, Publons or the arXiv as alternative outlets for commentary. But then again, beware of the Nirvana fallacy – as long as this doesn’t work perfect either, why not try out new things?

Hence, I wish ProcPoS all the best, while hoping it will become obsolete soon because publishers will integrate its functions in the existing journals.

Article layout of Proceedings of Peerage of Science

Article layout with “patronage” of Proceedings of Peerage of Science

16 thoughts on “Proceedings of Peerage of Science – complementing the journal landscape, or yet another new OA journal?

  1. Thanks for a pretty accurate and encouraging posting. A few clarifications though to prevent misunderstanding:

    “ProcPoS is not publishing any commentary, but only material from reviews that have been written through Peerage of Science”.
    In fact, ProcPoS does consider publishing comments to any published paper. However, the submission system is such that you can not submit to ProcPoS directly but you will have to submit your commentary to Peerage of Science for peer review first. In the Peerage of Science system you should alert the editor of ProcPoS (by simply clicking alert ProcPoS button). That is all. If your commentary is considered publishable after peer review in Peerage of Science, you will get a publishing offer from ProcPoS. The good thing is that you may also get publishing offers from other journals participating in Peerage of Science.

    “From that, I think fundamental requirements are a) to have links from the original publication to the comments b) to be able to comment on comments.”
    You are correct, and these are indeed functionalities we are currently building into ProcPos. Point a) is not entirely dependent on ProcPoS as it requires collaboration from the original publishers, but the other way is soon possible i.e. from the commentary there will be a link to the original articles. Point b) is already possible by having a comment on the comment peer reviewd through Peerage of Science.

    Finally, Public Patron Licensing systm is indeed an experiment aiming to integrate best of both worlds i.e. open access with no article processing fees. Anyone who feels they benefit from the contents of the article are requested to pay a fair fee. Fair fee is determined by the reader so fair is what ever you feel is fair. The actual licensing text can be found from http://www.peerageofscience.org/proceedings/PP_license.pdf.

    Janne Kotiaho
    ProcPos Editor-in-Chief
    Co-founder of Peerage of Science

    • Hi,

      apologies for the wrong information about which articles are considered, rereading the website it was actually pretty clearly stated. My fault – I corrected this in the article.

      About the commenting: OK, formal comments are of course possible, but if you ask me it seems a bit complicated to get a technical discussion going that way. I would find an option for commenting directly on articles useful, specially considering the post-publication peer review nature of this journal. I also would find it interesting to publish the reviews alongside with the commentaries.

      The funding model is innovative indeed, although I think if we would scale this up, it would be in the end the same science money that is also paying the conventional OA publishing fees, because I can’t see companies acting as patrons on a large enough scale (I mean hundreds of million Euro) to make this applicable for a large share of publications. But maybe I’m wrong, maybe companies would more readily become article patrons than funding research, good to try this out.

  2. H Florian,

    Interesting summary. Admittedly, I haven’t checked recently, but have PoS dealt with <a href="http://www.scilogs.com/theoretically_speaking/peerage-of-science-a-publishing-revolution/"my concerns from a couple of years ago, that ProcPoS would essentially be redundant?

    I think you touch on a similar issue above, but they appear to conflate pre- and post-publication peer review with this system.

    If the pre-publication peer review has value, authors will make changes, and the published article will no longer reflect the problems highlighted during the PoS review process. If the pre-pub peer review only highlights trivial, irrelevant or incorrect ‘problems’ with a submitted article, I don’t really want to read it in ProcPoS.

    Like I say, I may be out of date with their format, but it sounds like you’ve given it a good look – any thoughts?

    • Hi Mike,

      see my corrections above, besides publishing reviews, it is also possible to submit a normal “response” to a published article, so that would be a regular “post-publication” interaction. But sure, quite a few of the articles will probably originate from pre-publication peer review, so your question is valid.

      I don’t think it’s possible to make some general statements about the format at the moment, I mean, the format is about two pages of text, the question how the authors are going to use it, and at the moment there are only a few articles online, so I feel it’s a too small samplesize to pass judgment on the general content.

      That being sad, I guess your feeling might be quite accurate, but I’m not sure if that really makes the articles completely redundant. I want to take out one example, the article http://www.peerageofscience.org/proceedings/2014_e3/ . In this text, the authors basically recapitulates the review process, what they suggested, what the authors did, and what they didn’t, and gives his opinion about the final paper. Is that redundant? Yes and no.

      Yes, if we consider that an alternative would be that the whole review process could be open in the first place, so that we could read the reviews, responses and final recommendations first hand. There is little in the article that could not be drawn from having access to the review process.

      No, because the review is not open in POS and many other journals. And having a closed review as a baseline, it is actually quite interesting to hear the account of the reviewer. If there would be no interest in this, I should also not support open peer review.

      So, I’m swaying here … I would personally favor open pre-publication review in the first place … but in case a journal doesn’t want to do this, this model might be a good option to provide a reduced and maybe a bit more polite/distorted version of what has been going on behind the scenes.

      • I wish to draw attention to the need to critically evaluate Mr. Jeffrey Beall’s blog at http://www.scholarlyoa.com. Although I am of the belief that that blog brings considerable awareness about the open access movement, I find that there are serious flaws that need to be seriously addressed. The predatory nature of open access journals affects all academics, and thus we need correctly factual, unbiased, balanced and transparent criticism of the Beall blog. Some points worth pondering may be found at Retraction Watch:
        http://retractionwatch.com/2014/01/20/jeffrey-beall-scores-a-retraction/#comment-74829
        (this notice has been cloned on multiple blog sites to increase awareness)

        • I have accepted this comment for the moment, but I fail to see how this relates to the post or the comments made so far … please explain what you mean, in particular with respect to the topics of this post, or this will get erased

          • PPS was introduced as a possible redundant OA journal. For what reason is not immediately clear. However, it is evident from Beall’s site that several new OA publishers explore the OA model simply as an easy way to make profit. Therefore, is this a possibility, i.e., that PPS was born simply to serve as a revenue source? In that sense, what properties, if any, would make this new OA journal “predatory”, or “exploratory”, based on the Beall criteria for predatory journals? Once such possibilities exist, then the link, I believe, is quite clear. So, it would be interesting to get a more in-deoth analysis about several aspects of this new OA journal that is being critiqued, other than its apparent redundancy. Hope this clarifies the need for comment.

          • Hi,

            thanks for the clarification. In my opinion, predation doesn’t seem to be an issue here. Peerage of Science and Janne Kotiaho who commented above, despite being commercial, are known to the community and certainly have the reputation of operating their enterprise on sound scientific standards (at least I haven’t heard anything else).

            And after all, you don’t pay, and it’s OA, so who would be the prey?

        • I’m not sure that it’s correct to characterise Beall as unbiased… the blog is very closely related to an opinion piece he published (commissioned?) in Science, which is not an OA journal, and this arguably represents a direct conflict of interest. There’s been a lot of discussion about this case, and Beall has been open and helpful throughout, as far as I can tell, but the issue of balance and bias is not as neat as might originally have been thought.

          There is nothing inherently wrong in establishing a journal as a revenue source. In some cases (e.g., for scholarly society journals) this is a laudable goal, that can help fund important research that may otherwise lack funding, or support student attendance/prizes at conferences.

          However, “double dipping” (the sort of thing where commercial publishers charge a subscription fee to readers and accept a fee from authors to make their research open access, often without reducing the general subscription fee) is seen as less reasonable.

          Where does “Science” (the journal) fit into this? It’s the major scientific magazine/journal of the “World’s Largest General Scientific Society, but they seem to be double dipping… Still no conflict of interest?

      • Yeah – apologies to Janne and the PoS crew for mischaracterising ProcPoS (our earlier comments crossed before they were posted – he answered some of my concerns before I’d even asked them!). The opportunity to publish original commentary articles is definitely valuable, which is probably why many journals do so already.

        As for making peer review completely open, I certainly see some value here. I think open, anonymous peer-review could be an interesting avenue to explore.

        However, I also think the arguments I made about redundancy in ProcPoS above also apply here. I have limited time, so do I really want to read a rehash of how an article was improved during the review process? How many reviews of your own work do you think would be really valuable to publish (for a general audience) along with the final MS?

        • I guess you’re right, most people won’t be interested, but I also don’t see a downside of publishing reviews alongside with the paper if this doesn’t create additional costs. It may be useful for some at least to get the accounts of a few people that have read a paper in depth and thought about it. It may even add value if, like in ProcPoS, they update their review after the last revision to a final assessment of the corrected paper, so that you don’t have to go through comments such as “there is an s missing line line 28, biodiverity”.

          In stats, it’s quite common that they invite people to give short comments about new papers and publish them alongside the paper, I quite like that idea.

  3. A simple rehash of how an article was improved is not very valuable scientifically, true.

    But think, for example, how valuable it can be for a brilliant but thus far un-recognized, un-funded young scientist (perhaps with repressive lack of opportunities, say, in a developing country) to be able to show, in a peer-reviewed publication of her own, that it was *her* comments and insight that led to the improvements in someone else’s paper. That young scientist thus gets the recognition she deserves, and will be better positioned to do great things for all of us in the future.

    And once she is famous, you’d be curious to follow her latest commentaries, since the material she deems worthy of her reviewing time is probably interesting to anyone it that field.

    Another important function of ProcPoS commentaries is to allow peer-reviewers point out what the author did *not* do! It is currently immensely frustrating to peer review a paper, point out the gaping holes in methods or logic… and then a year later find it has been published, unaltered, in another journal (which then refuses to consider your critical letter as it would make them look bad).

    Hopefully, in the future authors who entertain an idea of publishing a less than solid paper, also have to take into account that it is likely to be deservedly bashed to smithereens in a ProcPoS commentary. And this is not limited to papers originally peer reviewed within Peerage of Science – ProcPoS would have been in ideal platform a few years ago, if Rosie Redfield wanted a formal, peer-reviewed yet quick way to assault the arsenic life paper.

    Janne Seppänen
    ProcPoS Executive Editor

    • Hi Janne,

      I think I’m more sympathetic to your 2nd point in a real world scenario, though your 1st point remains important in an ideal world.

      The difficulty with publishing signed reviews/critiques, is the inherent danger of pissing off more experienced researchers (who may be directly or indirectly involved with decisions that affect one’s future employment) or direct competitors (who will affect your ability to publish by reviewing your work, and therefore employment). This remains one of the strongest arguments for anonymous peer-review (in my opinion): it gives reviewers freedom to say exactly what they think without fear of repercussions.

      Of course, some reviewers abuse anonymity by writing poor reviews – too short, too unconstructive, too irrelevant, “not how I would do it” #sixwordpeerreview, but I remain to be convinced that anywhere near the majority of your young, insightful reviewers would be willing to be so insightful if their names were attached to the reviews. Rosie Redfield doesn’t fall into this category.

      Just to be clear, none of the above should be interpreted as being against ProcPoS, I simply think that there are still serious blocks against completely open (i.e., signed, published) peer review for a vital group of scientists.

  4. Mike, I would like to emphasize that pre-publication peer reviews (where anonymity is desirable for reasons stated below, yet everyone’s own choice) should not be confused with public commentaries (where commenter must take responsibility and do it under their own name). ProcPoS is a journal, PoS is a service. They are separate things, and operate at opposite sides of the DOI-boundary.

    But more generally, beyond our little attempts to build things:

    What you say about the need to prevent reprisal is correct, and half the reason why peer reviews in Peerage of Science are by default always triple-blind anonymous. An equally important reason to promote triple-blind anonymity in peer reviews is that one’s personal or institutional prestige or academic ranking should not matter in peer reviewing, only the merits of the arguments. In the traditional system, reviewer’s name may (and I suspect regularly does) bias editorial decisions. You’ll hear the-powers-that-be (= cadres of senior academics = editors) argue that they need reviewer names to have “context”, but in reality they need the names to weight reviewer verdicts by seniority. That is wrong, and hampers the progress of science.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s