Microsoft app uses 3D audio to help blind people navigate
March 27, 2018
Microsoft has developed a free app that can help blind or visually impaired people navigate around cities. Called Soundscape, it helps iPhone users explore the world around them through a 3D audio experience.
The app enriches perception of surroundings as the user walks, helping them get where you want to go. Rushing between classes, going for coffee with a friend or touring a new city with the family are all common experiences that could be more rewarding and cause less anxiety for nearly 300 million people with sight loss worldwide.
The technology lets users set audio beacons at familiar destinations and landmarks, and then use 3D audio cues to improve their ambient awareness as they make their way to the destination. By connecting a stereo headset, users can explore the outdoor world with more independence. With 3D audio, the sounds are perceived as coming from the point of interest, so the user can build a mental image of what’s around from the sounds in the environment and the Soundscape app.
“I use My Location in Soundscape, which tells me what’s around me,” said Shona Black, a guide dog owner. “It’s amazing, it tells me all the street names that I didn’t know before. One of the most useful things is that you can put in points of interest. I’ve put in a tower that is at the end of my street, which is really handy, and that’s where I head towards for home.”
Soundscape can run in the background while the owner uses other programmes, and was designed to be used in addition to other navigation support methods such as guide dogs and canes. As the user walks, Soundscape helps them stay aware of where they are by calling out roads and landmarks as they pass them.
If at any point the user is unsure of where they are or which road to choose, they hold the phone flat in their hand with the top edge facing the direction of interest and use the buttons at the bottom of the screen to locate nearby roads, orientate themselves to familiar landmarks and explore the places ahead.
“If you’re living with sight loss, getting around towns and cities can be daunting,” said Tom Wright, chief executive of the charity Guide Dogs. “Choosing where to go and what to do is an impossible dream. Soundscape will change this for many people. We worked with Microsoft to put people with sight loss at the heart of the development of Soundscape. This close collaboration has resulted in an app that makes exploring towns and cities a more enjoyable experience for those with sight loss, enabling people to make spontaneous choices about where they go and what they do.”
The project began in 2014 when Amos Miller, a Microsoft product strategist and researcher in the Microsoft Research Enable team, felt that independent mobility for visually impaired people could be greatly improved with technology. He was born with a genetic eye disease that later led to complete sight loss. Inspired by fatherhood, he sought to understand how the sense of independence, presence and empowerment might be heightened and thereby motivate many more people who are blind or have low vision to leave their home on their own.
“When someone can relate to their environment in a natural and intuitive way, it changes their relationship with their world,” said Miller. “It is that relationship that brings about a sense of meaning and connection, which is so important in life.”
To explore the problem, he brought together Microsoft researchers with experts from Guide Dogs UK, who help thousands of people who are blind or have low vision by providing them with specially trained dogs and mobility training. With growing support, the Soundscape research project was established, and began to reimagine how technology could enhance independence and mobility.
“While cities have a long way to go to become truly accessible for everyone, technology helps play a significant role for people with disabilities,” said Chris Yates, a lead mobility instructor at Guide Dogs UK. “Soundscape helps add more insight and information in addition to traditional mobility resources, like guide dogs and walking canes, that can help people gain confidence in how they get around their city.”
For each step in the development process, gaining feedback has helped bring the research project to life. In San Francisco, Microsoft worked LightHouse for the Blind, a non-profit organisation that promotes independence, equality and self-reliance of people who are blind or have low vision.
“Obstacle avoidance is not the problem, we have a dog, a cane and our blindness skills for that,” said Erin Lauridsen, access technology director at LightHouse for the Blind. “The gap is knowing where things are and being able to decide what’s of interest. At LightHouse, we know that accessible technology can play a huge role in helping more people be productive, gain confidence and become self-reliant. Soundscape addresses a crucial need – a tool for efficiently exploring your surroundings. That is a game-changer.”