Dieter Lukas , Kelsey McCune, Aaron Blaisdell, Zoe Johnson-Ulrich, Maggie MacPherson, Benjamin M Seitz, August Sevchik, Corina J Logan 

Environments can change suddenly and unpredictably, so animals might benefit from being able to flexibly adapt their behavior through learning new associations. Reversal learning experiments, where individuals initially learn that a reward is associated with a specific cue before the reward is switched to a different cue, thus forcing individuals to reverse their learned associations, have long been used to investigate differences in behavioral flexibility among individuals and species. Here, we apply and expand newly developed Bayesian reinforcement learning models to gain additional insights into how individuals might dynamically adapt their behavioral flexibility if they experience repeated reversals in which cue is associated with a reward. Using data from simulations and great tailed grackles (Quiscalus mexicanus), we find that two parameters, the association updating rate, which reflects how much individuals weigh the most recent information relative to previously learned associations, and the sensitivity to learned associations, which reflects whether individuals no longer explore alternative options after having formed associations, are sufficient to explain the different strategies individuals display during the experiment. Individuals gain rewards more consistently if they have a higher association updating rate, because they learned that cues are reliable and they therefore can gain the reward consistently during one phase. The sensitivities to learned associations plays a role for the grackles who experienced a series of reversals, where individuals with lower sensitivities are better able to explore the alternative option after a switch. The grackles who experienced the serial reversal adapted their behavioral flexibility through two different strategies. Some individuals showed more exploration such that they can quickly change to the alternative option after a switch even if they continue to occasionally choose the unrewarded option. Others stick to the previously learned associations such that they take longer to change after a switch, but, once they have reversed their associations consistently, choose the correct option. These strategies the grackles exhibited at the end of the reversal learning experiment also relate to their performance on multi-option puzzle boxes where there are different behaviors required to access rewards. Grackles with intermediate strategies solved fewer options to access the rewards than grackles with either of the extreme strategies, and they took longer to attempt a new option. Our approach offers new insights into how individuals react to uncertainty and changes in their environment, in particular showing that they can adapt their behavioral flexibility in response to their experiences.