People that have to use sign language to communicate with others can feel like foreigners in their own land. Very few random strangers can read sign language, but a new technology out of University of California, Los Angeles may make all of us be able to understand someone speaking using signs.
The UCLA team developed gloves that use stretchable sensors, that can detect when individual fingers are bent and the hand motions that are produced, to interpret each sign as it is displayed by the wearer.
“Our hope is that this opens up an easy way for people who use sign language to communicate directly with non-signers without needing someone else to translate for them,” said Jun Chen, the principal investigator of the research which was just published in journal Nature Electronics, in a press release. “In addition, we hope it can help more people learn sign language themselves.”
The gloves actually send their measurements of finger and hand movements to a smartphone app that does the interpreting and reading out of the words and phrases that are being communicated. The system, which was trained using a machine learning algorithm to recognize American Sign Language (ASL), is able to interpret about one word per second from a vocabulary of 660 signs, but the researchers expect to speed this up and expand the number of words that it can recognize.
One of the important developments of the system, over previously developed ones (see flashbacks below), is that it is small and light enough to be used in everyday life. It has all the electronics on a small circuit board on the back of the glove and relies on ubiquitous smartphones for the data processing. Moreover, all the electronics, including the flexible sensors, are very cheap and can be manufactured in large numbers.
Here’s a video demonstration of the new glove in action:
Study in Nature Electronics: Sign-to-speech translation using machine-learning-assisted stretchable sensor arrays
Flashbacks: A Sign Language Interpreter Glove for Your Smartphone; Low Cost Glove Translates Sign Language, May Be Used to Practice Surgery in Virtual Reality; EnableTalk, a Synchronous Interpreter for Sign Language; “Lorm” Glove Allows Deaf-Blind People to Text With Their Thumbs (And Fingers and Palms Too)
Via: UCLA