Ungeneralizable generalizations? A meta-meta-analysis of the influence of taxonomic bias on the study of behavior.

Ned A Dochtermann , MA Sekhar, Shinichi Nakagawa

Meta-analysis is a powerful tool for synthesizing behavioral research and identifying general patterns. However, are the conclusions we draw from these analyses truly representative across animal groups? Alternatively, are our conclusions shaped by taxonomic biases in the underlying research? For example, in animal behavior, vertebrates are often overrepresented in the research we conduct. This taxonomic imbalance raises concerns about the validity of generalizations drawn in the field, especially from meta-analyses. To examine this issue, we examined the meta-analyses published in Animal Behaviour, Behavioral Ecology, and Behavioral Ecology and Sociobiology from 2000 – 2024. We then conducted a “meta-meta-analysis” to calculate the degree to which overall effects in prior meta-analytical results may have been mis-estimated due to taxonomic bias. We found that taxonomic biases in the primary research strongly influence effect size estimates in meta-analyses and can lead to improper inferences and generalizations. On average, meta-analytical averages are mis-estimated by ~35% (p << 0.01) and significance changes in about 25% of instances when sampling is taxonomically representative. Because meta-analyses aggregate data, they propagate the biases present in an area of research, leading to potentially incorrect generalizations. Addressing this taxonomic bias is critical to generalizations that describe the true richness of animal behavior.