Using AI to Understand Animal Communication

New funding from Paul G Allen Family Foundation to support development of technologies that will reconnect us to nature ‍

Human beings are part of nature. But the vast majority of what is happening in this complex, interconnected natural world we inhabit is often beyond our ability to perceive - let alone comprehend. This disconnection from the world around us arguably represents one of the biggest obstacles humanity faces in tackling the biodiversity and climate crises. 

Fortunately the exponential progress we’re seeing in AI offers new ways of looking at the world -  extending our ability to perceive through technology, and finding patterns in data that will help to make sense of what is happening around us.

At Earth Species Project we are focused on harnessing these new developments in AI to gain a better understanding of animal communication because we believe it offers a unique and valuable way to connect to the rest of nature. ‍

We’re thrilled to announce that we have just received significant new funding of $1.2 million from the Paul G Allen Family Foundation that will support these efforts.‍

“It’s been exciting over the past year to see the growing interest in the ability of AI research to unlock a more profound understanding of animal communication and help solve long-standing challenges in conservation biology,” said Katie Zacarian, ESP CEO. “The willingness of the Paul G Allen Family Foundation to invest in this kind of fundamental research is critical in allowing us to experiment and to build the cross-disciplinary partnerships needed to make progress on this challenge. Their support will help move what might have been considered science fiction three or four years ago firmly in the realm of the possible.”

The grant funding will support our work to develop multimodal foundation models (the equivalent of ChatGPT and Dall-E like models for the animal domain) that can superpower analysis of not just animal vocalizations but also corresponding movements and behaviors. These new models will offer powerful general tools to address the challenges of understanding animal communication, starting with the fundamentals:  denoising and source separation, and automatically detecting and classifying vocalizations in what are often thousands of hours of recordings.

The funding will also help us to answer important questions about the data needed to train machine learning models for the animal domain. While human language models like ChatGPT have been trained on vast amounts of data from the internet, there are fundamental challenges in gathering and interpreting animal communication data. We will be working with our partners to test different approaches that will allow us to build effective data pipelines. And, importantly, we will also be making the data and our models available to other research teams to catalyze broader progress.

We believe this work will significantly accelerate animal communication, behavioral ecology, and machine learning research, and if successful this could dramatically support conservation efforts. Imagine a world where we could understand the communications of whales or dolphins to the point where it could help us avert stranding events, or where we could use our deep understanding of elephant culture to create more effective conservation corridors. 

"We’re in the midst of a computation revolution,” said Gabriel A. Miller, technology director on behalf of the Paul G. Allen Family Foundation. “ESP will electrify the conservation community with curated data sets, benchmarks, and AI tools that lead to a better understanding of animal behavior and communication.” 

In the future, it may even be possible to use AI to help monitor the health of entire ecosystems through the analysis of bioacoustic recordings - comparing, for example, the sounds of a healthy coral reef to one that is degraded, or being able to measure the recovery of a tropical rainforest after a mudslide, a forest fire or intensive agriculture.

The potential implications are only just beginning to be understood, but they offer huge promise for rebalancing our relationship with the rest of nature.

‍