In scholarship, there’s been increasing conversation over the past decade about a ‘Reproducibility crisis.’ Empirical findings in notable research papers have not been able to be replicated in follow-up studies. In one telling 2008 study from the Reproducibility Project, only 39 out of 100 studies published in prominent psychology journals were successfully replicated.
Way back in 2005, John Ioannidis published a prescient article titled ‘Why Most Published Research Findings Are False.’ He summarised his findings as follows:
Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.
Sadly, the situation seems even worse nearly two decades later. Just today, Retraction Watch have reported on a record number of over 200 retractions from one researcher, relaying that: “Joachim Boldt, has now been credited with 210 retractions – making him the first author (to our knowledge) with more than 200 retractions to his name.” His papers have been retracted on the basis of “suspicious data.”
What can be done? Naturally on this blog, we advocate adopting Open Research Practices. As we concluded in our previous post about Ethics in Research, ‘Essentially, good practice in Open Research amounts to communicating findings accurately and honestly and properly acknowledging the works of others.’ Open Research is a necessary counterbalance to the reproducibility crisis. On our UWL webpages, we’ve listed 7 Open Research Steps that should help keep your works open, transparent and reproducible, thereby enhancing academic credibility. They’re condensed slightly below but check out the webpage linked above for further links and resources:
- ‘Pre-register’ your finalised research design, either by uploading it to the UWL repository or elsewhere. Consider publishing your finalised design as a ‘registered report’ in a relevant journal. Journals which will accept Registered Reports will agree to publish a final paper, with ‘significant’ outcomes or not, if you follow the plan.
- Complete a Data Management Plan which includes how you are going to make your data sets available through a data repository, confidentiality permitting.
- Make sure your datasets are structured and labelled so that they are ‘Findable, Accessible, Interoperable and Reuseable’.
- Structure your papers so that titles and abstracts are illuminating, and keywords are prominent thereby making your paper easily discoverable through databases.
- Consider uploading a pre-print (a paper yet to be peer-reviewed) to a pre-print server and asking for feedback.
- Make sure your published papers are open access either through the UWL repository or through the expanding opportunities to make your paper open ‘in situ’ in the journal’s own website.
- Engage with Open Peer Review and teach Open Research practices if you are a teacher or research supervisor.
