Do we need all of this research synthesis in ecology and conservation? A highly subjective criticism of reviews and meta-analyses

Jacopo Cerri

Research synthesis, which includes systematic reviews and meta-analyses, provides applied ecologists and conservationists with big pictures of ecological processes and the effectiveness of conservation interventions. It can therefore hugely benefit applied ecology and conservation science.
However, while in some areas of ecology and the conservation sciences study design, measurements and statistical analyses have been standardized, in many fast-evolving fields of research they change relentlessly. This raises the question on the extent to which studies, which are inherently different in their design, methods and analyses can be compared. Even worse: is pooling together these studies actually misleading?
I believe four sources of heterogeneity to be particularly troublesome for research synthesis in many areas of ecology and conservation: i) the lack of well-agreed measures, ii) nonprobability sampling, iii) scarce background information about observational units and iv) limited control over treatment.
Considering the ongoing increase in research synthesis in ecology and conservation, there is a need to discuss these issues and figure out how to mitigate them. Incentives in favor of publishing research synthesis are too high to hope that scientists will self-regulate. Instead, I believe that research synthesis should: i) be published only after some years and a second peer-review, ii) consider only studies whose data can be re-analyzed in the future, iii) be periodically assessed and retracted, if deemed to be obsolete.