Elevating the importance of Risk of Bias assessment for ecology and evolution

Antica Culina, Dugald Foster, Matthew Grainger, Rose E O'Dea, Alfredo Sánchez-Tójar , Oliver L. Pescott , Rob James Boyd, Julia Koricheva, Shinichi Nakagawa, Gavin Stewart

Systematic reviews and meta-analyses are key evidence synthesis methods informing research and policy. An assessment of the Risk of Bias (RoB) in included studies is normally considered an essential component of these. However, RoB assessment is rare in ecology and evolutionary biology (EEB), and tools from other fields are seldom adopted. To identify reasons for this limited uptake, we surveyed 232 ecologists and evolutionary biologists with evidence synthesis experience. Only 28 were familiar with the RoB concept, and 46 confused it with publication bias. Just 10 had conducted a RoB assessment, most of whom found it challenging. We also reviewed the guidelines of 275 journals publishing EEB research. Out of 209 journals that likely solicit evidence synthesis, only five referenced guidelines for conducting evidence synthesis, and thus including of RoB assessment. To elevate the status of RoB assessment we recommend: (1) recognising RoB as essential to reliable evidence synthesis, with journals promoting clear guidelines and policies; (2) improving reporting standards for primary studies to enable RoB evaluation; and (3) developing RoB tools that accommodate EEB research and training researchers in their implementation. To start, we propose four RoB questions researchers should consider when conducting evidence synthesis or designing primary studies.e