Sound with a vision: What our planet is telling us

The crushing groan of a glacier. The reverberating explosion of a thunderstorm. A midnight choir of tree frogs in the rain forest. These are the immersive sounds that researchers at Purdue University’s Center for Global Soundscapes are inspired by. And for good reason: By listening to our planet, we can discover things we can’t see. “I’m on a mission to record the Earth,” relates Bryan C. Pijanowski PhD, Professor of Landscape and Soundscape Ecology at the university. “And I use that information to ask questions like: what’s the status of the planet right now?” Using AI to unravel the sounds of our ecosystems could be the next step in preserving them for future generations.

The sounds of science

Seeing is only part of believing

To feed the growing world population, we need to double food production in the next 30 years.
If you wanted to understand Earth solely based on the research we’ve compiled about our home planet, you might come to the conclusion that sight was humanity’s only sense.
One of the great achievements of the Age of Discovery was the mapping of the globe—with surprising accuracy as early as the 1500s. Once we were able to look at our planet from space, we began mapping it via satellite. And now, in the Information Age, those maps have become ever more detailed, rich, and navigable.

But sound is just as important as sight—maybe even more so. “Sound is the first sense we develop when we’re born,” says Professor Pijanowski. “And it’s the last sense that the medical profession says we lose before we die. Sound was present even before light in the universe.”
So when we talk about using sound as a means to study our planet, it makes sense on an almost primordial level.

“Sounds are intensely emotional to us. We can hear a song that we haven’t listened to in a long time, and all of a sudden, it triggers an emotion,” Pijanowski relates. In the same way, it’s the sense that might be able to best describe what’s going on in our world, and what might be changing.

Listening to the data

Turning the world into a recording studio

Photo: Bryan Pijanowski. Ugala Research Station, Kigoma, Tanzania.

Listening to an entire planet is an ambitious goal. But how do you even approach it? First you have to record it. That means installing sensors to capture audio in each of the planet’s major biomes—from tropical rainforests to arid deserts.

To do this, Pijanowski and team rely on battery powered audio recorders. “These recorders are at the very beginning of our data pipeline. They’re portable—we sometimes mount them on trees—and they can go a year before they need new batteries,” Pijanowski explains. “They record in stereo at 48,000 samples per second, so when you listen on headphones, it sounds like you’re there.”
Since the program’s inception in 2014, the Center for Global Soundscapes has recorded a continuous year of audio in 80% of Earth’s major terrestrial biomes. In the process, the center has trained more than 100 researchers in the use of its analytical tools, and enlisted more than 2,600 citizen scientists to post recordings using Purdue’s Soundscape Recorder app.

Decoding the messages of nature

Preserving sounds as a first step to ecosystem conservation

Photo: Bryan Pijanowski. Ulu Temburong National Park. Brunei.

Think there’s nothing going on in the endless expanse of Nebraska’s grasslands? A listen to the cacophony of cranes taking flight on a summer afternoon might change your mind. It’s just one of myriad examples of how the Soundscapes project is bridging the gap between perception and reality.

But in order to collect the amount of data that can deliver preservation-focused research, Pijanowski and team have to travel to remote locations to collect the audio. “One of my first expeditions was to Borneo. It’s a place where there have been many great scientific discoveries,” Pijanowski explains. “I wanted to go there as a 19th century kind of explorer with 21st century tools.”

So he set up acoustic sensors all over a pristine, paleotropical rainforest, and spent a night in the treetops—just listening. And he worked with indigenous peoples to study how they use sound as an information source.

“Oftentimes, indigenous peoples don’t have access to information on high-tech devices, so they listen to nature—and nature tells them what’s happening. What’s changing.” Pijanowski says. “I’m using technology and sound to understand patterns that might suggest solutions to how we can preserve this diverse, beautiful planet for future generations.”

From the wild to the data center

Sometimes the edge is beyond reach

Photo: Swapan Sarker. Sundarbans East Wildlife Sanctuary, Bangladesh.

When you’re hundreds of miles away from the nearest city, and you need to move two terabytes of data from point A to point B, there’s only one way to do it.
“We do sneaker net,” Pijanowski says. “We don’t have the technology to grab the data online. So we have to go and get the data, bring it back to the lab and begin to process it.”
But once the data is in the lab, it’s a different story. “We’re using very sophisticated AI tools to probe the data,” Pijanowski explains. “We sit down with an audio file and as we listen, we’ll score the database—labelling what we’re hearing—to try to teach the deep learning algorithm what is in the file.”
The amount of information in these files is enormous, and it’s not just biological. “We have information not just from the ecological environment, but also the geophysical environment. We can listen to thunder, which is really cool—I’ve listened to thunder all over the world and I can tell you stories about the differences of thunder.”
But the amount of data is massive, and it’s growing all the time. “The data pipeline is becoming extremely difficult to manage,” Pijanowski says. “We’ve got about a petabyte of data right now—four million files from more than 600 locations around the world.”
It was time to bring in enterprise level technology to achieve Pijanowski’s goal of preserving and understanding what the world sounds like.

From ingestion to visualization

Data libraries that give the user a window in time

To solve its data pipeline problems, Soundscapes began a conversation with HPE. “We have these massive data sets, and we’re examining different ways to support different phases of analysis. HPE has the hardware, software, and peopleware to help us do something new with it,” Pijanowski explains.
The team needed a place to consolidate all its field data to perform calculations and analysis. By harnessing the power of HPE Edgeline Converged Systems and HPE ProLiant servers on the front end of the process, Soundscapes and HPE built an environment to usher the data from ingestion to visualization via HPE Apollo Systems.
The audio files are loaded onto a server, and then distributed for processing using a combination of Apache Hadoop, Spark and Kafka, before landing in a MongoDB distributed database, and finally moving to Tableau for data visualization.
“We’re just beginning to explore how we can use these big data visualization tools—and customize some of our own tools—so we can show how a place changes over the course of a year,” Pijanowski says. “And we’re building it to allow us to look at it and listen to it all in three minute segments.”

Accelerating research, preserving the future

Hidden insights into healthier ecosystems

Now the team at Soundscapes is armed with the computational power to extract multitudes of data points from its growing sound library, and it’s already revealing new truths about ecosystems.
During a study in Arizona for the National Park Service, the team researched the site of a recent wildfire. “They wanted to understand what this fire did when it swept through the park, and how the ecosystem is recovering from it,” Pijanowski explains.
One of the things they learned from studying the sound collected from 50 sensors was that the nocturnal wildlife community had recovered almost immediately, while the diurnal animal population was still struggling. “If we didn’t have sound recorders out there, we would have missed this—we would have missed the potential pathway to a solution,” Pijanowski says.
Having a solid technology environment in the Purdue data center enables the research to happen faster, and more reliably. “Technology is helping us accelerate our research in new ways. It allows us to get good quality data in the field, and it allows us to analyze the data and create these visualizations in a way that can benefit society.”
Pijanowski sees Purdue’s partnership with HPE as a natural collaboration for discovery. “It’s a good fit for us, because HPE understands the social and technical relevance of what we’re trying to do,” he relates. “As an educator, that’s really what I want to do—I want to promote the knowledge and the discovery in a way that can help this marvelous planet. Just open up your ears. You’re going to be amazed by all the different sounds that exist here.”