Risks of shifting to post-publication review

Question

I've heard some compelling arguments from open science advocates for a system of reviewing scientific research outputs that focuses almost exclusively on post-publication peer review. The idea is that research would initially be published in free and open venues, with little to no lag time resulting from pre-publication review (pre-pub review, if any, is focused on weeding out large and obvious flaws and not on assessing perceived significance or impact). Once published, social tools similar to those in place at Amazon and StackOverflow (reviewing, commenting, reputation systems, etc.) would be used by the community for more in-depth assessment and critique of research outputs. Scientists could get credit for post-publication reviews, which (if openly accessible along with the article) are valuable scientific contributions themselves. On the other hand, junior scientists afraid of unjust retribution for being critical of a senior scientist's research could post reviews anonymously/pseudonymously, protecting their professional reputation but still making an important contribution to the scientific discourse.

I've seen two primary arguments against this kind of system: 1) that pre-publication review is an important quality control mechanism, and 2) that it ensures that the published scientific literature actually gets read. Regarding this second point, many raise the point that many research articles go uncited (and potentially unread by anyone other than the authors and reviewers). It could be tragic if a major scientific breakthrough existed in the literature, but was never widely acknowledged because the paper never got any attention and post-publication review.

Do these risks give real cause for concern? Is there some level of quality control that can only be done pre-publication? Do we risk missing out on important research by not requiring rigorous review of all published material? And are there other risks not mentioned here that would make a shift from primarily pre-pub review to primarily post-pub review problematic?

This post has been migrated from the Open Science private beta at StackExchange (A51.SE)

Comments

To keep the benefit of point 1), quality control, any place for doing post-publication reviewing would need to be properly (community?) moderated, to avoid it getting flooded by crackpots and cranks. Indeed I have heard that PhysicsForums tried to do something like this but they have given up, because the part dedicated to reviewing was quickly flooded by bad-quality stuff ...

This post has been migrated from the Open Science private beta at StackExchange (A51.SE)

---

If you think that this thread should be migrated to Academia or another SE site because the OpenScience beta is closing, please edit the list of questions shortlisted for the migration [here](http://meta.openscience.stackexchange.com/questions/73/).

This post has been migrated from the Open Science private beta at StackExchange (A51.SE)

Answer 1

Given the availability of social platform, I would very much appreciate including post-pub review already, alongside the usual pre-pub. Just think about the amount of papers researchers read during their work: if they can gain reputation by leaving a post-pub review on all of them, and if the review can influence the authors', I guess the quality standard of upcoming publications will definitely rise (as in: no more rush to publishing). Then, I already start to imagine "distilled" science Journals, where the top rank post-reviewed articles will end up.

A risk related to moving to post-pub only is that we could end up with an unmanageable publication deluge in which the "noise" makes it difficult to identify high quality content. So I tend to confirm both (1) and (2) since three reviewers reading an article are much better than nobody and quality (at least on form and clarity) should be definitely addressed pre-pub.

Long story short: we should start post-pub alongside with pre-pub and ensure a good balance of both.

This post has been migrated from the Open Science private beta at StackExchange (A51.SE)