Researchers at Hebrew University in Jerusalem, Israel have developed a new, more intuitive way for blind people to perceive and interact with objects in the environment around them. EyeMusic is a sensory substitution device (SSD) that converts visual input into audio output, and many have been attempted before. According to a study just published in journal Restorative Neurology and Neuroscience, EyeMusic is an effective system that users can quickly learn to use and that may one day lead to an effective interface for blind people.
The trick for making a good SSD is in finding a mapping from visuals to sound that makes it easy to interpret what one is hearing. EyeMusic does this by representing colors using different sounds (white = vocals, blue = trumpet, red = reggae organ) and brighter objects are made to sound louder. The system scans the image left to right and plays the audio representation along with the scan. So objects on the left side sound first and ones on the right sound last. For examples, here’s a page with images and audio clips representing them that EyeMusic created.
Abstract in Restorative Neurology and Neuroscience: Fast, accurate reaching movements with a visual-to-auditory sensory substitution device.