Measuring biological generality in meta-analysis: a pluralistic approach to heterogeneity quantification and stratification

This is a Preprint and has not been peer reviewed. This is version 3 of this Preprint.

Add a Comment

You must log in to post a comment.


There are no comments or no comments have been made public for this article.


Download Preprint


Yefeng Yang, Daniel W.A. Noble, Rebecca Spake, Alistair M Senior, Malgorzata Lagisz, Shinichi Nakagawa


Uncovering general rules enhances the predictive capabilities in ecology and evolution. Meta-analytic approaches play a critical role in this endeavour, examining the extent to which phenomena can be replicated, generalized, and transferred. However, ecologists and evolutionary biologists have largely overlooked the role of meta-analytic heterogeneity in informing generality. To reform this situation, we introduce a pluralistic approach aimed at quantifying and stratifying various heterogeneity metrics, such as I^2, CV, M, and predictive distribution. These metrics offer complementary information, revealing the source, magnitude, and visual representation of heterogeneity. Our analysis of 512 meta-analyses demonstrates that heterogeneity is, on average, ten times larger than statistical noise, contributing to 91% of the observed variance (median I2 = 91%). This amount of heterogeneity is nearly twice the size of the meta-analytic mean effect (median CV = 1.8, M = 0.6), indicating substantial total heterogeneity in ecology and evolution. Surprisingly, in half of the cases, focal effects could generalize across studies even with high total heterogeneity by controlling for within-study variation. Our synthesis also visualises empirical distributions of various heterogeneity metrics, potentially serving as new benchmarks for informed interpretation. Our proposed pluralistic approach will accelerate the future quest for general rules via meta-analyses.



Ecology and Evolutionary Biology, Statistics and Probability


Generaliability, Transferrability, Replicability, heterogeneity, Variation


Published: 2023-11-24 13:21

Last Updated: 2023-11-27 23:09

Older Versions

CC-By Attribution-NonCommercial-NoDerivatives 4.0 International

Additional Metadata


Conflict of interest statement:

Data and Code Availability Statement: