Audio embeddings identify novel sounds and track soundscape change within a tropical restoration landscape

Jared De Silva , Cristina Banks-Leite

Passive acoustic monitoring (PAM) is increasingly being used over conventional biodiversity surveys because it is more cost-effective and implementable at a landscape level. However, the cheaper sensors that removed the barriers to PAM have now shifted the challenge from data collection to interpretation. Acoustic indices address this by condensing each recording into single summary statistics, but their relationship to biodiversity varies widely across studies. The alternative, acoustic classifiers, name what is calling. However, they are mainly trained on bird call, so exclude the insects and frogs that dominate tropical nocturnal soundscapes and serve as powerful bioindicators. At most tropical restoration sites, the non-avian vocalising community has not been catalogued, and therefore its contribution to changing soundscapes cannot be quantified meaningfully. Here, we built a pipeline using Perch v2 audio embeddings to identify novel sounds and to measure the dissimilarity of the non-avian community between reforested pasture and forest. We tested it on >3,880 hours of audio recorded at reforestation sites in the Amazon over three consecutive years. Using an agile modelling approach, we built acoustic classifiers for fifteen frog and insect sounds (sonotypes) and derived a dissimilarity in sonotype composition between pasture and forest. We also used the embeddings as an acoustic index to quantify dissimilarity between the whole night-time soundscapes of pasture and forest. Both dissimilarities showed reforested pasture soundscapes diverging from their baseline and becoming more similar to forest after two years. Two forest-associated sonotypes became more prevalent over time, while one pasture-associated sonotype declined, highlighting the sonotypes driving species turnover. Only five sonotypes could be matched to a candidate frog species from public databases, likely due to limited archive coverage. Our pipeline requires no reference library or fine-tuning, and therefore any restoration projects with recorded audio can measure change in their uncatalogued vocalising communities.