Measuring biological generality in meta-analysis: a pluralistic approach to heterogeneity quantification and stratification

This is a Preprint and has not been peer reviewed. This is version 3 of this Preprint.

Add a Comment

You must log in to post a comment.


Comments

There are no comments or no comments have been made public for this article.

Downloads

Download Preprint

Authors

Yefeng Yang, Daniel W.A. Noble, Rebecca Spake, Alistair M Senior, Malgorzata Lagisz, Shinichi Nakagawa

Abstract

Uncovering general rules enhances the predictive capabilities in ecology and evolution. Meta-analytic approaches play a critical role in this endeavour, examining the extent to which phenomena can be replicated, generalized, and transferred. However, ecologists and evolutionary biologists have largely overlooked the role of meta-analytic heterogeneity in informing generality. To reform this situation, we introduce a pluralistic approach aimed at quantifying and stratifying various heterogeneity metrics, such as I^2, CV, M, and predictive distribution. These metrics offer complementary information, revealing the source, magnitude, and visual representation of heterogeneity. Our analysis of 512 meta-analyses demonstrates that heterogeneity is, on average, ten times larger than statistical noise, contributing to 91% of the observed variance (median I2 = 91%). This amount of heterogeneity is nearly twice the size of the meta-analytic mean effect (median CV = 1.8, M = 0.6), indicating substantial total heterogeneity in ecology and evolution. Surprisingly, in half of the cases, focal effects could generalize across studies even with high total heterogeneity by controlling for within-study variation. Our synthesis also visualises empirical distributions of various heterogeneity metrics, potentially serving as new benchmarks for informed interpretation. Our proposed pluralistic approach will accelerate the future quest for general rules via meta-analyses.

DOI

https://doi.org/10.32942/X2RG7X

Subjects

Ecology and Evolutionary Biology, Statistics and Probability

Keywords

Generaliability, Transferrability, Replicability, heterogeneity, Variation

Dates

Published: 2023-11-24 13:21

Last Updated: 2023-11-27 23:09

Older Versions
License

CC-By Attribution-NonCommercial-NoDerivatives 4.0 International

Additional Metadata

Language:
English

Conflict of interest statement:
None

Data and Code Availability Statement:
https://github.com/Yefeng0920/heterogeneity_ecoevo/tree/main