In my last blog I argued that carrying out a completely transparent public evaluation of research results – Public Post-Publication Peer Review (4PR) – is the best way to ensure scientific quality. I strongly believe that and in some ways started the publishing platform ScienceOpen as an experiment to test this hypothesis. But what happens in the extreme case when a manuscript is submitted that can be perceived as outside of the scientific discourse – “crackpot” or “pseudoscience” theories from perpetual motion to parapsychology abound. A whole list of pseudoscience topics can be found here. It is easy to reject the papers on the taxonomy of unicorns, but there are some fields of alternative medicine for example where the lines are not so clearly drawn. Politically charged fields such as climate change or genetically modified foods can also present a grey zone where legitimate research and industry-sponsored propaganda can be difficult to distinguish. In principal one could think about two options of an editorial workflow to cope with those submissions.
In our Public Post-Publication Peer Review model we still require a first editorial check that should filter out the clearly non-scientific work written by non-scientists outside of any norms of scientific communication. Papers that then pass this check should be evaluated by the scientific community in a transparent and public way. But in some cases the editors doing a first editorial check may be unsure whether a paper is sound science and may call on one or more experts to give their opinion – a pre-publication peer review. If these experts give the paper a very poor review, what should happen next? ScienceOpen is committed to providing an open platform for discussion – is it then against our principles to begin rejecting some manuscripts based on a pre-publication peer review process? One editorial policy could simply be for ScienceOpen to reserve the right to reject all papers which do not meet some minimum standards of scientific communication. This sounds straightforward but if defining those standards in a formal and transparent way seems to be almost impossible. Without such a formal and transparent reasoning, the rejection could be perceived as arbitrary. and so violate the principle of public post publication peer review as such. It would protect the reputation of the platform and of the scientific community. There could also be public health considerations if papers make false claims about the health benefits of certain products which should not be underestimated. So rejection is an attractive option.
Another editorial policy could be to inform authors about the poor reviews and should they choose to publish their work anyway to publish the paper with the poor reviews attached. This would it be in line with the Public Post-Publication Peer Review model and a platform model of publishing where curating of content is kept to a minimum. Because the internet has made it so easy to “publish” ideas in the meaning of “make public” we can assume that there is an open access journal somewhere with such low standards that will happily publish the findings on alien DNA even without any peer review at all. The “peer reviewed” Journal of Cryptozoology would probably be happy to publish that unicorn taxonomy paper. And of course an editorial check is not going to pick up every case of pseudoscience or fraudulent science so the post publication evaluation is absolutely necessary. This is a principled but risky approach.
The ScienceOpen platform will begin publishing in January 2014 and is in the process of fine-tuning its editorial policy. While the problem of “crackpot” submissions will not come up often, it is important to have transparent policies in place before confronted with a climate change denial paper. Because it is so easy to make ideas public in our digital networked world, this is a difficult problem that will require a concerted effort by the entire scientific community.