Rethinking success and quality in ecological forecasting

Victoria L Boult, Sandro Bertolino, Adam Devenish, Luke C Evans , Cara A. Gallagher , Valentin Journe, Gerbrand Koren, Joe Frederick Langley , Solène Marion, Maria Paniw, James Pearce-Higgins, Nigel Yoccoz, Emily Simmonds

Ecological forecasts are increasingly required to inform science, policy and action in the face of rapid environmental and societal change. However, differences in how developers and end-users judge the value of ecological forecasts may limit their utility and uptake in real-world decision-making. Here, we use a participatory approach to propose a novel, holistic framework for the evaluation of ecological forecasts along two intersecting axes: quality and success. Quality is defined by the defensible development and predictive performance of ecological forecasts, while success captures the usefulness of a forecast in achieving its intended aim. We propose twelve shared criteria for evaluating quality and success spanning the forecasting process, set-up, and outputs while capturing forecast development, communication, performance, and operationality. Together, these criteria provide a common basis for the co-development of ecological forecasts, and better connect science with forecast-informed decision-making.