MetAnoDe: Metabarcoding Anomaly Detection using an LSTM-CNN deep neural network ensemble

Alexander Keller 

Amplicon metabarcoding has emerged as a critical tool in ecology and other scientific disciplines, facilitating species identification in mixed samples for biodiversity monitoring, community and microbiome analysis, dietary studies, and understanding species interactions. However, challenges arise from errors and artifacts introduced during sampling and laboratory processes such as polymerase chain reaction (PCR) and sequencing. Manual inspection is impractical due to the vast number of sequences, necessitating rapid algorithms for data cleanup. Thorough bioinformatic processing can reduce such errors through removal of low-quality or non-target sequences using quality-, abundance-, and alignment-based approaches. However, in practice, some anomalous sequences evade detection, while valid sequences may also be incorrectly removed.

Deep neural networks offer a promising complementary solution to alignment-based methods by recognizing complex DNA sequence patterns. This study introduces MetAnoDe (Metabarcoding Anomaly Detection), an alignment-free tool that classifies sequences of a defined amplicon as valid, anomalous (accumulated substitutions or indels, chimeras) or off-target, using a convolutional neural network (CNN) and a recurrent long short-term memory (LSTM) network combined into one model. Pre-trained models are provided for 13 markers across bacteria, fungi, plants and animals, and models for further markers can be trained with the same workflow. To evaluate MetAnoDe independently of its training data, I simulated mock communities for all markers, including pseudogene-like divergent copies, chimeras and off-target sequences, and processed them with a standard workflow, so that the origin of every resulting sequence was known. Added after de novo chimera removal and taxonomic filtering, MetAnoDe reduced the proportion of retained anomalous sequences from 22.5-58.1% to 10.5-25.1%, while additionally removing 0.2 to 3.6% of valid sequences. It detected most divergent copies and many off-target sequences missed by taxonomic filtering, whereas chimeras were detected mainly already by de novo chimera removal. MetAnoDe is therefore suited as an additional quality-control layer after standard processing.