Microsoft has released an iPhone app for blind people and those with significantly decreased vision. Seeing AI, as the app is called, does its magic when the user points the smartphone’s camera at something and it reads out what it’s seeing. For example, pointing it at a person, it tries to identify if it’s someone you know, otherwise specifying basic characteristics of the person, such as gender, and whether they’re smiling or sad.
There are six “channels,” including products, person, short text, currency, documents, and scene, that the user can select from to help the app better understand what it’s looking at and the type of description it provides. The products channel can help the user find where a UPC barcode is on a product and then uses that to call out what it is. The experimental scene channel is a work in progress, and hopefully Microsoft will be able to compress the one thousand words that a picture is worth into much fewer.
As far as processing, the heavy duty computer vision algorithms that power the app actually run on the cloud, so the app can quickly read out what it’s seeing.
Here’s a video from Microsoft presenting Seeing AI:
And here’s a demo of the scene channel:
Here’s the link to download the Seeing AI app for the iPhone…