Gelada monkeys groom each other. Photo by Rieke T-bo.
Gelada monkeys groom each other. Photo by Rieke T-bo.
We take a modern, AI-first approach to studying animal communication—leveraging advances in large-scale models, multimodal learning, and vast datasets to unlock the hidden languages of the natural world.
Our research operates in a virtuous cycle: AI facilitates scientific discovery in animal communication, and those discoveries refine our AI models with greater capabilities.
Trained on massive data sets across human speech and music, bioacoustic data and environmental sounds, our models learn foundational representations of bioacoustic signals.
This makes them generalizable across tasks and species. Whether studying birds or whales or jumping spiders, our large animal language models help researchers automatically detect, classify, and find new answers in massive unlabeled ethology datasets.
NatureLM-Audio is the world’s first large audio-language model for animal sounds. Trained on a vast and diverse dataset spanning human speech, music, and bioacoustics, it brings powerful AI capabilities to the study of animal communication.
With NatureLM-Audio, researchers can:
NatureLM exhibits few-shot learning capabilities at inference time, allowing it to generalize to new tasks without needing to be retrained. Instead of needing to fine-tune the model on a new dataset, users can prompt it with just a few examples of a new task, and the model can generate meaningful responses based on existing knowledge.
NatureLM-Audio also demonstrates emergent properties like the ability to count the number of individuals in a recording, identify distress and affiliative calls, and classify new vocalizations—without explicit training. It also shows promising domain transfer from human speech to animal communication, supporting our hypothesis that shared representations in AI can help decode animal languages.
We are committed to open science and will release NatureLM-audio in 2025.
Using our AI models and tools, we conduct deep dive experiments to understand the communication systems and diverse intelligences of species across the Tree of Life.
Task- or Species-Specific: We select and solve bioacoustic tasks and build tools for the study of specific species that will generalize to other species and improve our core models.
Decode Questions Only AI Can Answer: We design large-scale decode projects that seek to answer questions that have been impossible to ask because of the size and complexity of the data sets.
We apply AI principles to core bioacoustic tasks like detection, classification, and dataset processing, ensuring that scientists have powerful, scalable tools to accelerate their work:
We’ve also developed the first benchmarks for applying AI to animal communication research. These benchmarks provide researchers with high-quality datasets and rigorous evaluation metrics, ensuring that AI-driven insights into animal vocalizations and behavior are both reliable and reproducible.
This work can’t be done alone. We aim to accelerate the entire field by providing access to our models and benchmarks. Researchers and conservationists can explore, test, and contribute to our growing ecosystem of tools.
