Seeing AI

Last modified
Wednesday, December 20, 2017
Category

Description of App

Seeing AI is a free app that narrates the world around you. Designed for the blind and low vision community, this ongoing research project harnesses the power of AI to open up the visual world and describe nearby people, text and objects.

Optimized for use with VoiceOver, the app enables you to recognize:
• Short Text - Speaks text as soon as it appears in front of the camera.
• Documents - Provides audio guidance to capture a printed page, and recognizes the text, along with its original formatting.
• Products - Scans barcodes, using audio beeps to guide you; hear the name, and package information when available. (works with iPhone 6 and later)
• People - Saves people’s faces so you can recognize them, and get an estimate of their age, gender, and emotions.
• Scenes (early preview) - Hear an overall description of the scene captured.
• Currency - Recognizes currency notes.
• Color - Identifies color.
• Handwriting - Reads handwritten text.
• Light - Generates an audible tone corresponding to the brightness in the surroundings.
• Images in other apps - Just tap “Share” and “Recognize with Seeing AI” to describe images from Mail, Photos, Twitter, and more.

Seeing AI is built to help you achieve more by leveraging the power of the cloud and artificial intelligence. As the research progresses, more channels may be added.

Check out tutorials with this YouTube playlist: http://aka.ms/SeeingAIPlaylist

Visit http://SeeingAI.com for more details

Available in 35 countries including the U.S., Canada, European Union, U.K., Australia, India, Hong Kong, New Zealand, Singapore. Coming soon to more countries.
Languages supported – English
Supported on iPhone 5C, 5S and later; best performance with iPhone 6S and later models.

Questions, feedback or feature requests? Email us at seeingai @ microsoft.com.

Version

2.0

Free or Paid

Free

Apple Watch Support

No

Device(s) App Was Tested On

iPhone

iOS Version

11.2.1

Accessibility Comments

This app is very accessible

VoiceOver Performance

VoiceOver reads all page elements.

Button Labeling

Most buttons are clearly labeled.

Usability

The app is fully accessible with VoiceOver and is easy to navigate and use.

Other Comments

This is an amazing app! I was pleasantly surprised to learn about this app from mainstream tech sites this morning.

Recommendations

49 people have recommended this app

Most recently recommended by Tarja 1 month ago

Options

Comments

Submitted by JeffB on Monday, July 17, 2017

Does anyone know if bar codes are universal or have any overlap? When I scanned the bar code on a spray can of sunscreen it seemed to think it was Bunratty Botcheen which turns out to be a type of Irish moonshine. I did submit this under feedback but was just wondering if there could be some overlap when it came to bar codes?

Submitted by Troy B on Tuesday, July 18, 2017

It would be nice if somebody could do a walk-through podcast on this site for seeing AI. Somebody did a podcast on CoolBlindTech, but I've listened to it and didn't get much out of it because the person is talking as she's flicking through the different areas on her phone so it's hard to hear what's going on.

Submitted by Sean Terry on Tuesday, July 18, 2017

Hi all, I just spoke with one of the app developers on Twitter. They said that a fix for iOS 11 beta will be coming soon. I also suggested a nice feature that would really help out. What do you think about this! A prescription barcode reader! There are a few of these, but I think it would be very useful for this app and especially iOS to have something like this!

I searched for it and could not find.; Where I can find it? Did search typing blind cool tech and nothing about it.

Submitted by Darrell Bowles on Wednesday, July 19, 2017

Apple vis has just posted a recording that I did. Please go have a listen, and feedback is welcome!

Submitted by Troy B on Thursday, July 20, 2017

In reply to by Darrell Bowles

I agree with the comment that perhaps the voiceover rate needs to be a bit slower, but I got a lot more out of this podcast than I did the one from CoolBlindTech and for that I'm greatful.

Submitted by Raul on Thursday, July 20, 2017

How is the heating and battery draining on version 1.1? I can't download it yet

Heating and battery draining issues are still present in version 1.1. I don't think that this app will be my primary app unless they solve these issues. How much my iPhone 7 gets heated up wile using the app is scary. However, we should all keep in mind that we are still in the early versions of the app. I am sure things will get better.

Submitted by Deborah Armstrong on Thursday, July 20, 2017

Took a photo of my husband yesterday in a restaurant. It said "man sitting at table indoors using tablet", he was actually surfing the internet on his phone. Just amazing! Tried reading all the tags that came with a shirt I bought and after some positioning difficulty, it worked as well as KNFB reader. But do remember KNFB works in airplane mode and this does not. This also needs the document positioned on a dark background, like a piece of black construction paper, to more accurately determine the document edges; KNFB is less picky.

But most amazing is the short text it grabs with video. I panned around the restaurant, capturing all the little advertising circulars scattered on our table. I point it at a random pile of papers and it reads a surprisingly large amount of visible text.
I also noticed that it kind of decides on its own when to turn the flash on and off which might be what is causing the battery and heating. When I turned more light on in my office, (powerful flourescents located above my desk) to the point that it made me uncomfortable, the app didn't keep activating the flash. So if you can take your item somewhere brighter to identify or read, it might save your battery.

I had the same experience with a box of Keeblers Club Crackers. On the side of the box SeeingAi thinks it sees a bar code and there is probably one there but not a real one. So I continue looking and found the real one on the bottom of the box. So I suggest to keep scanning to see if there are more than one bar code.
I also used the short text channel while riding in an Uber by holding the camera pointing out the window to see what text I can pick up. Fascinating what it can pick up. Like the name of Condo complex in my neighborhood.
It will be interesting to see if it can read the street sign names.

Submitted by Justin on Thursday, July 20, 2017

Hi,
I here's what i think on your heating/battery issues. Remember, this app is probably using a lot of processing power/also it is using the camera and the flash. This, in my experience can cause the phone to heat up slightly. It shouldn't get to the point where the phone gets too hot to hold. If it gets to a point where it gets too warm, then the device should automatically shut down.

Submitted by Remy on Thursday, July 20, 2017

There is a setting in settings for the light to automatically come on or not.

Also, it's interesting that quicktext will read some signs and advertisements, but when I get it to read an envelope or small bit of tinier text, it often can't manage so well. Still fantastic, but surprising.

Submitted by Orko on Saturday, July 22, 2017

This is a nice app, but since processing appears to be cloud based it won't replace KNFB Reader for privacy.

I'd also recommend that you get permission before taking people's picture with this app since the picture will eventually be put through facial recognition.

Submitted by LaBoheme on Saturday, July 22, 2017

"I'd also recommend that you get permission before taking people's picture with this app since the picture will eventually be put through facial recognition."

face recognition is done completely on your phone, nothing is sent to the cloud. try turn on airplane mode, then you can go from building a face profile then recognize the face, you can achieve all of this without any network connection.

Submitted by Raul on Saturday, July 22, 2017

@Justin , I disabled the automanage lighting and the phone heats and the app drains battery the same way. I also tried the new version 1.1 and nothing changes. The short text option is awesome

Submitted by Matt on Saturday, July 22, 2017

I would agree with some comments on here. I love the app but I don't like that it's fully cloud based. Even the bar code reader is cloud based. I confirmed this by turning on airplane mode and then attempting to scan a barcode. Also, I have scanned multiple documents with the document scan feature, and have attained very poor results. I get much better OCR recognition with Prizmo Go and I do with this app. Hopefully this will be worked on in future updates, and it would be nice if the app included a built-in OCR like Prismo go and KNFB reader do

Submitted by Dave Nason on Saturday, July 22, 2017

Member of the AppleVis Editorial Team

As someone who cannot yet get the app as it’s not in my country, can I ask, does the app allow you to import documents and images for OCR or for scene recognition, as KNFB READER does for example, or is it only done live?
Also is there any product recognition other than the barcode scanning, for example like TapTapSee does?
Regarding the point above about the cloud, I would imagine the barcode recognition has to be off device because the databases it’s searching, by their very nature, have to be online.
That’s not to say that there aren’t legitimate privacy questions about the app though, for example around facial recognition. This app is a testing ground for AI research after all.

Submitted by Dawn 👩🏻‍🦯 on Sunday, July 23, 2017

Yes it can import images from your photos app on your device. Just make sure you go into the 2nd more section, and toggle regignize with seeing ai to on. Then whenever you'd like to get a discription for an image, you can go to your photos app, select the photo you want to see, then select the recignize with seeing ai. I hope it's expanded to other countries in the near future. I don't mean to rub people's noses in it, but I feel very fortunate that I could get the app in mine. Hope this helps!

Submitted by Deborah Armstrong on Sunday, July 23, 2017

Its ability to scan bar codes is similar to store scanners, that are omnidirectional Digit-Eyes and other apps need the camera to point right at the code. The short text is great for scanning bookshelves. Walking around our living room, it can read spines if I position the camera with care. If you read many documents with it, purchase a thin black scarf and use it the way photographers use a black drape to enhance contrast. I found if I spread it under the document it detected the edges best. Black construction paper or a black rubber mat also works, but a scarf is portable.

Submitted by falcon wings on Monday, July 24, 2017

Hi,
I have read that one should be able to have a picture recognised with Seeing AI through the share options but need to turn it on first.
I have tried the share option on the photos app just like someone else here, but don't see a option to turn Seeing AI on.
There are 2 more buttons,one with copy slideshare,print etc,the other with Whatsapp,Icloud drive,Dropbox etc with toggles to switch them of and on.
But nothing like that for the Seeing AI. what am I missing?
Using Iphone 6 plus on IOS 10
ideas?

Submitted by Darrell Bowles on Monday, July 24, 2017

Hello all,
I was just informed, that the hot device is caused by the flashlight being turned on automatically, due to the lighting. As far as privacy goes, it's on the internet, as is most of our digital life is, this is, as someone pointed out, a testing ground for AI, so please keep that in mmind. As far as document scanning goes, in my podcast, while one of the documents tested was a glossy piece of paper, it did maintain the formatting, and I found my hands shook less because I wasn't conserned about having to capture the image my self. To the person who had poor results, did you try a different background? Also, give the ap some time. I wrote a review for this ap for a job assignment, and one of the points I stressed, is this is version 1.1, so it's kind of in the ground breaking stage. Glad to help, and I look forward to more comments!

Submitted by Greg Wocher on Monday, July 24, 2017

Hello,
Is it possible to scan multi page documents with this app? If not this is one area where KNFB still has a bit of a leg up.

Regards,
Greg Wocher

Submitted by Troy B on Monday, July 24, 2017

I just scanned a big box of rice crispy treats and the app actually started beeping on the complete opposite side from where the bar code was. Has anybody else experienced this?

Submitted by Justin on Monday, July 24, 2017

Lol yep!
I found that happened the other day. Lol, I was trying to get a box scanned with the bar code feature and somehow, my shirt that i was wearing at the time must of had numbers or such on it, cause it was making the beeping sound even though there wasn't a box there in the first place. I just wanted to see what happened if i took it away from the box.
I did a successful scan the other day of some instant potatoes, my god I was very impressed with how great it was. It found the code, and it was on the top of the box.
This app just keeps getting better and better!

Submitted by ellzagirliegirl on Tuesday, August 1, 2017

Such a shame this isn't available in the UK yet... I can just tell how useful this app would be for me as a 14 year old school student. The ability to ready documents and short texts (let alone recognise faces and products) would be a life saver for me :)!

Submitted by George on Saturday, August 12, 2017

Hello to all the community I can't understand why Seeing ai has been delayed being available in the UK Apple Store does anyone know when it will be available in the UK please let me know thank you to everyone

Hello James I also live in the UK and wonder when this app is going to be available to us it doesn't make any sense that it wasn't available to us when it was launched in the United States regards

Submitted by chris R on Saturday, August 12, 2017

I'm in the uk and using this app all the time and it works great here. Not really had any problems with anything that might explain why it's not officially released here yet except for some barcodes that are UK specific. It's quite easy to obtain this app through the Appstore but I won't give instruction here incase it violates something, just google how to do it.

Submitted by Eileen on Sunday, August 13, 2017

There are many possible reasons why this app is only available in a small number of countries at the moment.

For example:

They may want to check that the hardware and technology behind this service is scalable.

They may want to have a limited roll-out so that they can resolve any initial problems whilst there is still a relatively small user base.

They may not have translations in place for all languages.

There may be licensing agreements that limit or restrict how and where a technology used by the app can be used.

And probably many other possible reasons.

So, lets not give the developers a hard time because the app isn't currently available in your country. I am sure that Microsoft will want to make it as widely available as possible, as soon as possible.

Submitted by George on Sunday, August 13, 2017

Many thanks I have now got Seeing ai thanks Chris

Submitted by Dawn 👩🏻‍🦯 on Monday, August 14, 2017

Hello all!

I have found a pretty neat use for Seeing AI. I have found that it does a good job reading memes! I've done some limited testing with random memes from various FB. pages. My results are that it just reads the text in memes. Which is fine because really that's all you need. I think it disscribed the photos in 1 or 2 cases as well as read text.

I also have a very funny story that has to do with this app! I have 2 photos of myself holding a Calves shirt. Well, every time I find a photos app like Braigo or Seeing AI., I run those 2 or at least 1 of those 2 photos through the app just for giggles and out of pure curiousity! Well, I ran 1 of those photos through and it read the text. That's not the funny part. Seeing AI. said that I was a 19-year-old male looking happy! haha lol Obviously, I'm not a man lol! I know it's because my hair was pulled back, but I still got a kick out of it!! And I thought you all might as well! Anyway, I just figured I'd share this experience and also the little tidbit about memes. I'll keep testing memes from random pages on Facebook.

Submitted by Will on Saturday, September 23, 2017

hi how will i get this app as still not in uk? assume all that doesn't work yet are uk barcodes? svague how still not out, rediculous really.

Hi, its best if you google how to do it as I don’t wish to give specific instructions here. It’s pretty straightforward though.

Submitted by Will on Saturday, September 23, 2017

what doesn't x∂ then in the uk? assume just not recognising barcodes?

Submitted by AnonyMouse on Monday, October 9, 2017

Member of the AppleVis Editorial Team

What's New in Version 1.3

• Product channel optimized for iOS 11 - still works on iOS 10 as before.
• Added support for 3D Touch, providing a convenient way to launch the app with a channel pre-selected.
• It is now possible to share/save the original image of a document.
• Various minor bug fixes.

Submitted by ellzagirliegirl on Wednesday, November 15, 2017

Got this app about half an hour ago as it's finally out in the UK. First thing I tried was the face recognition - as that's the feature I was most excited about.

Now I'm fully aware that I look older than my years (14) but it came back with: "50 year old man with black here and a beard looking sad". The only part of that which is true was the sad face I was pulling at the camera. I'm a: "14 year old girl with brown hair" haha!

Submitted by Peter Verhoeven on Wednesday, December 20, 2017

I listen to one of the Youtube videos. In one video you can share photos in the photo App with Seeing AI. If I select a photo in my photo App and select Share I do not see the option to share it with Seeing AI. I have a iPhone 5S with IOS 11.1. Hope somebody can help to make this work!
BTW: I downoaded the app from the Dutch appstore.

Submitted by Faruk on Thursday, December 21, 2017

Hello,

For some reason, I cannot launch the app after updating it to V2.0. Everything was working perfectly before the update but now, every time I try to open the app, it sends me back to the home screen. I deleted and reinstaled the app but it didn't work. I tried everything short of factory reset.
Has anyone experienced the same issue? If so, how did you solve it? I am using an iPhone 7 running iOS 10.3.3.

To send a photo to Seeing AI go to photos, choose your album and photo and double tap on share. There you have two lists, the first has Facebook, Twitter and so on, and the second one has print, copy and other things. Each list has a More button at the end, pres the second one. There you can check and uncheck things that you want to appear on that list, including seeing ai.
After that it should appear on the second list.

Hi ashandarei, this is Anirudh from the Seeing AI Team. Thanks for reporting this. A handful of users on the older iOS have noticed an issue in starting the app. We have published an update yesterday, so give it a try. Also, highly recommended to update your phone to iOS 11.