I've heard some compelling arguments from open science advocates for a system of reviewing scientific research outputs that focuses almost exclusively on post-publication peer review. The idea is that research would initially be published in free and open venues, with little to no lag time resulting from pre-publication review (pre-pub review, if any, is focused on weeding out large and obvious flaws and not on assessing perceived significance or impact). Once published, social tools similar to those in place at Amazon and StackOverflow (reviewing, commenting, reputation systems, etc.) would be used by the community for more in-depth assessment and critique of research outputs. Scientists could get credit for post-publication reviews, which (if openly accessible along with the article) are valuable scientific contributions themselves. On the other hand, junior scientists afraid of unjust retribution for being critical of a senior scientist's research could post reviews anonymously/pseudonymously, protecting their professional reputation but still making an important contribution to the scientific discourse.
I've seen two primary arguments against this kind of system: 1) that pre-publication review is an important quality control mechanism, and 2) that it ensures that the published scientific literature actually gets read. Regarding this second point, many raise the point that many research articles go uncited (and potentially unread by anyone other than the authors and reviewers). It could be tragic if a major scientific breakthrough existed in the literature, but was never widely acknowledged because the paper never got any attention and post-publication review.
Do these risks give real cause for concern? Is there some level of quality control that can only be done pre-publication? Do we risk missing out on important research by not requiring rigorous review of all published material? And are there other risks not mentioned here that would make a shift from primarily pre-pub review to primarily post-pub review problematic?
This post has been migrated from the Open Science private beta at StackExchange (A51.SE)