Post

My radical suggestion for peer review: Move from nominal pre-publication review to explicit post-publication review. The original reason for peer review was that journal pages are a limited resource, so we need a filter before publication. That no longer makes sense with digital publishing. 1/

With the internet making publishing ~free, the limited resource is human attention, not publishing pages. It doesn’t make sense to use human reviewer time as a filter. And the filter is not exactly working well. 2/

So my suggestion is that we should review papers after they have been published – give the community some time to digest the paper before we formally evaluate it. Only allocate human reviewer time to papers that gain some attention first. E.g. only review papers once they start getting cited. 4/

Is something like this a perfect system? No. Is it better than the status quo? Maybe. It puts the emphasis on filtering what deserves reviewer attention, rather than wasting reviewer attention to filter what goes on a website. Authors should make a case that their work should be reviewed. 5/

What should happen to conferences like NeurIPS/ICML/ICLR? I think the attitude should be that anyone can present their work – subject to some very lightweight review and a strict one-per-registered-attendee limit. That way the focus is on the actual conference, not the associated review process. 6/

Yes, let's use all available human resources to review papers by Google, Meta and other big companies and top US universities only, and push the rest to /dev/null. Let's make it even more a popularity contest where some of the contestant have a PR budget and others don't.