This is a Preprint and has not been peer reviewed. This is version 3 of this Preprint.
Downloads
Authors
Abstract
Uncovering general rules enhances the predictive capabilities in ecology and evolution. Meta-analytic approaches play a critical role in this endeavour, examining the extent to which phenomena can be replicated, generalized, and transferred. However, ecologists and evolutionary biologists have largely overlooked the role of meta-analytic heterogeneity in informing generality. To reform this situation, we introduce a pluralistic approach aimed at quantifying and stratifying various heterogeneity metrics, such as I^2, CV, M, and predictive distribution. These metrics offer complementary information, revealing the source, magnitude, and visual representation of heterogeneity. Our analysis of 512 meta-analyses demonstrates that heterogeneity is, on average, ten times larger than statistical noise, contributing to 91% of the observed variance (median I2 = 91%). This amount of heterogeneity is nearly twice the size of the meta-analytic mean effect (median CV = 1.8, M = 0.6), indicating substantial total heterogeneity in ecology and evolution. Surprisingly, in half of the cases, focal effects could generalize across studies even with high total heterogeneity by controlling for within-study variation. Our synthesis also visualises empirical distributions of various heterogeneity metrics, potentially serving as new benchmarks for informed interpretation. Our proposed pluralistic approach will accelerate the future quest for general rules via meta-analyses.
DOI
https://doi.org/10.32942/X2RG7X
Subjects
Ecology and Evolutionary Biology, Statistics and Probability
Keywords
Generaliability, Transferrability, Replicability, heterogeneity, Variation
Dates
Published: 2023-11-24 17:51
Last Updated: 2023-11-28 03:39
Older Versions
License
CC-By Attribution-NonCommercial-NoDerivatives 4.0 International
Additional Metadata
Language:
English
Conflict of interest statement:
None
Data and Code Availability Statement:
https://github.com/Yefeng0920/heterogeneity_ecoevo/tree/main
There are no comments or no comments have been made public for this article.