Lara S. Burchardt, Yana van de Sande, Mounia Kehy, Marco Gamba, Andrea Ravignani, Wim Pouw

Biological structures are defined by elements like bones and cartilage, and elastic elements like muscles and membranes. Computer vision advances have enabled automatic tracking of moving animal skeletal poses. Such developments put us on the verge of gaining insights in complex dynamics otherwise studied in more static terms (e.g., images). However, the elastic soft-tissues of organisms, like the nose of Elephant seals, or the buccal sac of frogs, have been poorly studied
and no computer vision methods have been proposed. This leaves major gaps in different areas in biology. In the area of primatology, most critically, the function of air sacs is widely debated and many questions exist about their role in communication and human language evolution. Moving towards the dynamic study of soft-tissue elastic structures, we present a toolkit for the automated tracking of semi-circular elastic structures in biological video data. The toolkit contains
unsupervised computer vision tools (using Hough transform) and supervised deep learning (by adapting Python’s Deeplabcut) methodology to track inflation of laryngeal air sacs or other
biological spherical objects (e.g., gular cavities). Confirming the value of elastic kinematic analysis we show that air sac inflation correlates with acoustic markers that likely inform about
body size. Finally, we present a pre-processed audiovisual-kinematic dataset of 7+ hours of closeup audiovisual recordings of Siamang (Symphalangus syndactylus) singing. This toolkit
revitalizes the study of non-skeletal morphological structures in a wide range of animals.