Earth Species Project is the new frontier of interspecies understanding. We decode animal communication with advanced AI to illuminate the diverse intelligences on earth.
Our vision is a relationship with the rest of nature that allows the diversity of life to thrive.
With exponential advances in AI and Large Language Models, decoding animal communication is no longer a question of if—it’s when. We’re building the first-ever large language models models designed to analyze data from species across the Tree of Life. By listening more deeply, we can unlock a new relationship with the rest of nature.
Language is the lens through which we seek new and useful perspectives on nature. Using our AI models and tools, we conduct deep dive experiments to understand the communication systems and diverse intelligences of species across the Tree of Life. In partnership with leading biologists around the world, we’re currently studying the languages of crows, beluga whales, elephants, and more.
NatureLM-audio is the world’s first and most capable large audio-language model for bioacoustics. It’s trained on large data sets and learns foundational representations of bioacoustic signals. That makes it generalizable about task and species.
Whether studying birds or whales or jumping spiders, our large animal language models help researchers automatically detect, classify, and find new answers in massive unlabeled ethology datasets.
Early results show positive domain transfer from human speech to animal communication, reinforcing our hypothesis that AI can decode the shared structures of language across species.
Playbacks are a common technique used to study animal vocalizations, involving the experimental presentation of stimuli to animals (usually recorded calls played back to them) to build an understanding of their physiological and cognitive abilities. With current playback tools, biologists are limited in their ability to manipulate the vocalizations in ways that will establish or change their meaning, and their exploratory power is limited. Senior AI research scientist Jen-Yu Liu is exploring whether it is possible to train AI models to generate new vocalizations in a way that allows us to solve for a particular research question or task.
Jen-yu has been working on generating calls for a number of species, including chiff-chaffs (Phylloscopus collybita) and humpback whales (Megaptera novaeangliae). A new project is now allowing us to test this work through interactive playback experiments with captive zebra finches (Taeniopygia guttata) in partnership with research scientist Dr. Logan James at the University of McGill. Providing researchers with the ability to control various aspects of the vocalization production process will greatly expand the exploratory and explanatory power of bioacoustics research, and is an important step on our Roadmap to Decode.Given that we may not understand the meaning of the novel vocalizations being generated by the model, there are important ethical considerations related to potentially interfering with animals and their culture. For this reason, we are beginning this research only with captive populations and working exclusively with scientists who follow strict ethical protocols.
Senior AI Research Scientists Benjamin Hoffman, Maddie Cusimano and Jen-Yu Liu are working to map the vocal repertoires of two species of crow. The first one, the Hawaiian crow, is notable for its natural ability to use foraging tools as well as its precarious conservation status – the species sadly became extinct in the wild in 2002 and currently only survives in captivity. With Professor Christian Rutz and collaborators we are investigating how the species’ vocal repertoire has changed over time in two captive breeding populations, to inform ongoing reintroduction efforts. The second species, the carrion crow, is abundant across its European range, but has attracted attention with its unusually plastic social behavior, with groups in some populations breeding cooperatively. We are working with Professors Daniela Canestrari, Vittorio Baglione and Christian Rutz to analyze field recordings to understand the role of acoustic communication in group coordination. Mapping vocal repertoires can help uncover cultural and behavioral complexity, which in some cases has important implications for planning effective conservation strategies.
“Ever since I was a child, I’ve dreamed of understanding what animals are saying. How wonderful that is now a real possibility.”
