Evaluating AI agents on the IUCN Red List assessor exam: Tradeoffs and opportunities for AI assistants in extinction risk assessments

Shane Weisz , Michael Winston Dales, Sadiq Jaffer, Steven Bachman, Anil Madhavapeddy

The IUCN Red List of Threatened Species is the world's most widely used system for assessing the global extinction risk of species, underpinning conservation policy and action worldwide. However, due to funding constraints and limited assessor capacity, the Red List faces significant gaps in species coverage (less than 20% of plant species have been assessed) and challenges in keeping assessments up to date (over 30% are 10+ years old). Recent advances in AI open new opportunities for helping assessors increase assessment throughput and quality, but also raise concerns about trust and reliability, data security, cost, and environmental impact.

As a starting point for considering these tradeoffs, this paper evaluates the performance of AI agents on the official IUCN Red List assessor training exam. On a benchmark of 101 exam questions, we show that several frontier models from Anthropic and OpenAI reliably clear the exam's 75% pass mark. The top-performing model, Anthropic's Claude Opus 5, reaches 96% mean accuracy, increasing to 99% with majority vote over five attempts. Notably, the pass mark is now also reached by open-weight models running on local hardware: Alibaba's Qwen3.8 Flash, self-hosted on a single desktop Mac Studio, attains 81% mean accuracy (87% majority vote). This points to viable hosting alternatives where data security or sustainability concerns preclude sending data through external providers' data centres. Whilst the exam serves only as a capability floor, these results strengthen the case for thoughtful use of AI assistants in assessors' workflows, such as quality and consistency checks on draft assessments.