Rob James Boyd, Gary Powney, Fiona Burns, Alain Danet, François Duchenne, Matthew Grainger, Susan Jarvis, Gabrielle Martin, Erlend Birkeland Nilsen , Emmanuelle Porcher

1. Aggregated species occurrence and abundance data from disparate sources are increasingly accessible to ecologists for the analysis of temporal trends in biodiversity. However, sampling biases relevant to any given research question are often poorly explored and infrequently reported; this has the potential to undermine statistical inference. In other disciplines, but particularly medicine, researchers are frequently required to complete “risk-of-bias” assessments to expose and document the potential for biases to undermine inference. The huge growth in available data, and recent controversies surrounding their use to infer temporal trends, indicate that similar tools are urgently needed in ecology.
2. We introduce ROBITT, a structured tool for assessing the “Risk-Of-Bias In studies of Temporal Trends in ecology”. ROBITT has a similar format to its counterparts in other disciplines: it comprises signalling questions designed to elicit information on the potential for bias in key study domains. In answering these, users will define their inferential goal(s) and relevant statistical population. This information is used to assess potential sampling biases across domains relevant to the research question (e.g. geography, taxonomy, environment), and how these vary through time. If assessments indicate likely sampling biases, then the user must explain what mitigating action will be taken.
3. Everything that users need to complete a ROBITT assessment is provided: the tool, a guidance document, and a worked example. Following other disciplines, the tool and guidance document were developed through a consensus-forming process across experts working in relevant areas of ecology and evidence synthesis.
4. We propose that researchers should be strongly encouraged to include a ROBITT assessment as supplementary information when publishing studies of biodiversity trends. This will help researchers to structure their thinking, clearly acknowledge potential sampling issues, and provides an opportunity to describe data checks that might otherwise not be reported. ROBITT will also enable reviewers, editors, and readers to establish whether research conclusions are supported given a particular dataset combined with some analytical approach. In turn, it should strengthen evidence-based policy and practice, reduce differing interpretations of data, and provide a clearer picture of the uncertainties associated with our understanding of ecological reality.