Code-sharing policies are associated with increased reproducibility potential of ecological findings

Alfredo Sánchez-Tójar , Aya Bezine, Marija Purgar , Antica Culina

Software code (e.g., analytical code) is increasingly recognised as an important research output because it improves transparency, collaboration, and research credibility. Many scientific journals have introduced code-sharing policies; however, surveys have shown alarmingly low compliance with these policies. In this study, we expanded on a recent survey of ecological journals with code-sharing policies by investigating sharing practices in a comparable set of ecological journals without code-sharing policies. Our aims were to estimate code- and data-sharing rates, assess key reproducibility-boosting features, such as the reporting of software versioning, and compare reproducibility potential between journals with and without a code-sharing policy. We reviewed a random sample of 314 articles published between 2015 and 2019 in 12 ecological journals without a code-sharing policy. Only 15 articles (4.8%) provided analytical code, with the percentage nearly tripling over time (2015-2016:2.5%, 2018-2019:7.0%). Data sharing was higher than code-sharing (2015-2016:31.0%, 2018-2019:43.3%), yet only eight articles (2.5%) shared both code and data. Compared to a comparative sample of 346 articles from 14 ecological journals with a code-sharing policy, journals without a code-sharing policy showed 5.6 times lower code-sharing, 2.1 times lower data-sharing, and 8.1 times lower reproducibility potential. Despite these differences, the key reproducibility-boosting features of the two journal types were similar. Approximately 90% of all articles reported the analytical software used; however, for journals with and without a code-sharing policy, the software version was often missing (49.8% and 36.1% of articles, respectively), and exclusively proprietary (i.e., non-free) software was used in 16.7% and 23.5%  of articles, respectively. Our study suggests that journals with a code-sharing policy have greater reproducibility potential than those without. Code-sharing policies are likely to be a necessary but insufficient step towards increasing reproducibility. Journals should prioritize the adoption of explicit, easy-to-find, and strict code-sharing policies to facilitate researcher compliance and implement mechanisms such as checklists to ensure compliance.