A computer vision toolkit for the dynamic study of air sacs in Siamang with a general application to the study of elastic kinematics in other animals

This is a Preprint and has not been peer reviewed. This is version 1 of this Preprint.

Add a Comment

You must log in to post a comment.


There are no comments or no comments have been made public for this article.


Download Preprint


Lara S. Burchardt, Yana van de Sande, Mounia Kehy, Marco Gamba, Andrea Ravignani, Wim Pouw


Biological structures are defined by elements like bones and cartilage, and elastic elements like muscles and membranes. Computer vision advances have enabled automatic tracking of moving animal skeletal poses. Such developments put us on the verge of gaining insights in complex dynamics otherwise studied in more static terms (e.g., images). However, the elastic soft-tissues of organisms, like the nose of Elephant seals, or the buccal sac of frogs, have been poorly studied
and no computer vision methods have been proposed. This leaves major gaps in different areas in biology. In the area of primatology, most critically, the function of air sacs is widely debated and many questions exist about their role in communication and human language evolution. Moving towards the dynamic study of soft-tissue elastic structures, we present a toolkit for the automated tracking of semi-circular elastic structures in biological video data. The toolkit contains
unsupervised computer vision tools (using Hough transform) and supervised deep learning (by adapting Python’s Deeplabcut) methodology to track inflation of laryngeal air sacs or other
biological spherical objects (e.g., gular cavities). Confirming the value of elastic kinematic analysis we show that air sac inflation correlates with acoustic markers that likely inform about
body size. Finally, we present a pre-processed audiovisual-kinematic dataset of 7+ hours of closeup audiovisual recordings of Siamang (Symphalangus syndactylus) singing. This toolkit
revitalizes the study of non-skeletal morphological structures in a wide range of animals.




Animal Sciences, Biology, Neuroscience and Neurobiology


Siamang, air sac, kinematics, acoustics, vocalization, singing, computer vision, animal body tracking


Published: 2023-10-14 16:03


CC BY Attribution 4.0 International

Additional Metadata


Conflict of interest statement:

Data and Code Availability Statement:
The computational tools can be found in our Github repository (https://github.com/WimPouw/AirSacTracker/tree/main). The code and data for reproducing the kinematic-acoustic analyses can also be found on our github page (https://github.com/WimPouw/AirSacTracker/tree/main/project). The open dataset can be found on the Donders repository (https://doi.org/10.34973/6apg-q804).