I recently had a paper accepted which I co-authored as part of a group of just PhD students. Our article “researcher degrees of freedom in the psychology of religion” (available via PsyArXiv) will be in an open science special issue of the International Journal for the Psychology of Religion. When we saw the call for papers, we wanted to contribute, but we did not just want to rehash the same “here’s a description of open science” article adapted for every sub-field of psychology. After originally considering a simulation paper for how researcher degrees of freedom can increase error rates, we focused on whether open science had actually permeated the psychology of religion sub-field. We specifically focused on researcher degrees of freedom that we could detect in published research. We conducted a systematic review of all the articles published in three journals across 2017 and audited whether they had included any open science practices. We found pretty much no evidence that open science practices had made their way into psychology of religion research, so we felt justified in outlining how they could incorporate these practices in future to mitigate researcher degrees of freedom. It was the best publishing experience I’ve had up to yet, so I wanted to highlight a few positives we experienced which I felt added value to the final article given there is still scepticism over the benefit of preprints and open science.

We originally submitted the article in December 2018 and finally had it accepted in August 2019 after two revisions. If we followed the traditional approach and just left it in the journal’s inbox, we would have had just over eight months of dead space. However, as open science advocates, we posted a preprint of the article and updated it after each revision. In addition to the exposure from the approximately 250 downloads, we had one interested reader email us to say they liked the paper. However, they thought one of the sections was out of date and provided some information we were able to use to update the section when we revised the paper. Without posting a preprint, the only people who could view the paper would be the editor and two reviewers. Opening access with a preprint provided us with the opportunity to further improve the paper based on peer feedback.

In addition to posting our paper as a preprint, we cited several preprints. In the eight months between submission and acceptance, one of these preprints was also accepted for publication. Posting these papers as preprints allows their contents to be communicated more rapidly than the traditional publishing process allows. Providing you cast a critical eye over their contents (as you should published articles), I think this allows science to develop much more efficiently instead of waiting for the veil to be taken off when the article is finally published.

Turning to open science practices, we tried to practice what we preached as best we could. For the systematic review element of the article, we preregistered a protocol to outline how we planned to audit the articles and included the coding scheme we were going to use. We then shared the final coding scheme and data/code for calculating inter-rater reliability. One of the dilemmas when you preregister your study is whose responsibility it is to compare the protocol to the final article. In our paper, we argue reviewers and editors are best placed as gatekeepers of the publishing process, despite taking more resources from voluntary labour. In an ideal world, this is factored into the publishing process if you submit a registered report. However, one of the real benefits we had was one of the reviewers closely compared our preregistration protocol to the manuscript and highlighted some areas they thought were inconsistent or not specified. We were able to address these and highlight any deviations in the revised manuscript. They also highlighted some rounding errors in our statistics. It will be the first time I’ll genuinely write “we would like to thank the two anonymous reviewers” instead of murmuring it through gritted teeth.

To conclude, I’m a huge advocate of preprints and open science. I think it provides the opportunity for science to develop more efficiently and adds value at each stage of the research and publishing process. However, you still hear horror stories, such as the pretty egregious response from an editor shared on Twitter below who found out an author had posted a preprint of the article they submitted. My parting advice is to make sure you check the Sherpa/Romeo database if you plan to submit a preprint. This will tell you whether the journal allows pre- and post-prints, so you can decide on whether to submit to that particular journal.