Pietro Pollo, Malgorzata Lagisz , Renato Chaves Macedo-Rego, Ayumi Mizuno , Yefeng Yang, Shinichi Nakagawa

Meta-analyses are powerful synthesis tools that are popular in ecology and evolution due to the rapidly growing literature of this field. Although the usefulness of meta-analyses depends on their reliability, such as the precision of individual and mean effect sizes, attempts to reproduce meta-analyses’ results remain rare in ecology and evolution. Here, we assess the reliability of 41 meta-analyses on sexual signals by evaluating the reproducibility and replicability of their results. We attempted to (1) reproduce meta-analyses’ mean effect sizes using the datasets they provided, (2) reproduce meta-analyses’ effect sizes by re-extracting 5,703 effect sizes from 246 empirical studies they used as sources, (3) assess the extent of relevant data missed by original meta-analyses, and (4) replicate meta-analyses’ mean effect sizes after incorporating re-extracted and relevant missing data. We found many discrepancies between meta-analyses’ reported results and those generated by our analyses for all reproducibility and replicability attempts. Nonetheless, we argue that the meta-analyses we evaluated are largely reproducible and replicable because the differences we found were small in magnitude, leaving the original interpretation of these meta-analyses’ results unchanged. Still, we highlight issues we observed in these meta-analyses that affected their reliability, providing recommendations to ameliorate them.