Seeing AI

Last modified
Wednesday, December 20, 2017
Category

Description of App

Seeing AI is a free app that narrates the world around you. Designed for the blind and low vision community, this ongoing research project harnesses the power of AI to open up the visual world and describe nearby people, text and objects.

Optimized for use with VoiceOver, the app enables you to recognize:
• Short Text - Speaks text as soon as it appears in front of the camera.
• Documents - Provides audio guidance to capture a printed page, and recognizes the text, along with its original formatting.
• Products - Scans barcodes, using audio beeps to guide you; hear the name, and package information when available. (works with iPhone 6 and later)
• People - Saves people’s faces so you can recognize them, and get an estimate of their age, gender, and emotions.
• Scenes (early preview) - Hear an overall description of the scene captured.
• Currency - Recognizes currency notes.
• Color - Identifies color.
• Handwriting - Reads handwritten text.
• Light - Generates an audible tone corresponding to the brightness in the surroundings.
• Images in other apps - Just tap “Share” and “Recognize with Seeing AI” to describe images from Mail, Photos, Twitter, and more.

Seeing AI is built to help you achieve more by leveraging the power of the cloud and artificial intelligence. As the research progresses, more channels may be added.

Check out tutorials with this YouTube playlist: http://aka.ms/SeeingAIPlaylist

Visit http://SeeingAI.com for more details

Available in 35 countries including the U.S., Canada, European Union, U.K., Australia, India, Hong Kong, New Zealand, Singapore. Coming soon to more countries.
Languages supported – English
Supported on iPhone 5C, 5S and later; best performance with iPhone 6S and later models.

Questions, feedback or feature requests? Email us at seeingai @ microsoft.com.

Version

2.0

Free or Paid

Free

Apple Watch Support

No

Device(s) App Was Tested On

iPhone

Accessibility Comments

This app is very accessible

iOS Version

11.2.1

VoiceOver Performance

VoiceOver reads all page elements.

Button Labeling

Most buttons are clearly labeled.

Usability

The app is fully accessible with VoiceOver and is easy to navigate and use.

Other Comments

This is an amazing app! I was pleasantly surprised to learn about this app from mainstream tech sites this morning.

Recommendations

44 people have recommended this app

Most recently recommended by Kevan 4 months 2 weeks ago

Options

Comments

Submitted by Erion on Wednesday, July 12, 2017

The fact that one can read a print book live is quite impressive, but that's not everything. This application is truly amazing.

Submitted by Oliver Kennett on Wednesday, July 12, 2017

GRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR!!!!!!! Turin was from the UK!!!

Submitted by Joseph on Wednesday, July 12, 2017

I just got it. And wow! Very, very nice. This one and taptapsee are probably gonna be my go-to things.

Submitted by jrjolley (not verified) on Wednesday, July 12, 2017

It'll be interesting to see how this app works out. I question it being free though but we'll see. It's a bit surprising that it's not available worldwide, doesn't look good in 2017 when we're all mobile all of the time.

Submitted by jrjolley (not verified) on Wednesday, July 12, 2017

In reply to by Carlos Taylor

Don't you think it looks a bit wierd that MS launch this in just a few countries? Surely if this is all just AI, why not just release globally - seems strange but we'll see what happens.I think Prizmo's using MS OCR for the cloud OCR option, I used to use it when it was project oxford.

Submitted by JeffB on Wednesday, July 12, 2017

This is great wow! I'm impressed!

Submitted by Toonhead on Wednesday, July 12, 2017

I hate to rain on everyone's parade here but so far, I ain't impressed. I tried, for about 15 minutes to get it to recognize different bar codes, and after following the instructions in the help, my phone beeped just as the instructions said it would, but that's all it did, and it heated up my phone to the point where I thought it was going to shut off but luckily, it didn't. Then again I am not very good with using my camera so it could certainly be operator error. To all of you having good luck with the app, yay! But for me, it's about as useless as a paperweight, at this point anyways.

Submitted by Oliver Kennett on Wednesday, July 12, 2017

If it's an AI they'd have to have the infer structure to support how ever many requests, maybe they're not confident that it could cope yet. And it will be free, lucky us, we're perfect gynnypigs for object recognition and sight free interaction. Go us!

Submitted by Kevan on Wednesday, July 12, 2017

This is excellent! I played around with it for a few minutes and I'm very impressed. I especially like making funny faces at the camera and getting it to recognize me. Only thing is it ages me around 10 years, LOL.

I'd update the AppleVis app listing to include the full description from the appstore.

Submitted by JeffB on Wednesday, July 12, 2017

How close are you holding the phone? When I got it to read the barcode I held it about a half a foot or a foot away from the object. Then I turned the object slowly in 1 direction.

Submitted by Carlos Taylor on Wednesday, July 12, 2017

In reply to by Kevan

Yes, I plan to update the apps description when I get home. I was in the car leaving town when I posted it this morning. I was so excited about this app that I wanted to get the word out as soon as possible. I will update the app description later this evening when I get home.

Submitted by Carlos Taylor on Wednesday, July 12, 2017

I will say that scanning barcodes is a feature I haven't tried yet. I will do so later when I get home. However, taking pictures of your surroundings, reading text off of a printed page, and recognizing people, all works great here. It is amazing how it recognizes text in a live video stream. This is great as it can start reading much quicker and doesn't rely on you to snap the picture as it can do so automatically when using the document mode.

Submitted by Dawn on Wednesday, July 12, 2017

Hi! I downloaded this as soon as I heard about it! I really love it! I can't wait to play with it and try it in different circumstances! That being said, I do have a couple questions.

1. How do you change the camera from back to front facing in the app? Is that even something you can do in the Seeing AI. app, or do you have to go to the camera app and change it?

2. I'm having an issue with the product channel. I select it, but then I can't take a pic and scan the barcode. Any help would be appreciated! I can't wait until someone does a podcast on this app! I listened to the tutorial videos and found them helpful but I couldn't find the solutions to the issues I'm having.

Thans for any help in advance! Also, many thanks for sharing this app! I'm glad Microsoft made this, and I'm looking forward to the updates to this app and what knew channels that are added!

Submitted by Greg Wocher on Wednesday, July 12, 2017

Hello,
When you are scanning barcodes there is not a take picture button or scan button. All you need to do is move the product under the camera and it starts scanning automatically. The app will start beeping when you get close to the barcode. I believe the faster the beep the closer to the barcode you are. I really am liking this app. It is working much better for me than digit eyes does.

Regards,
Greg Wocher

Submitted by Remy on Wednesday, July 12, 2017

Okay, maybe not, but this thing is all sorts of amazing so far. So far I've only used facial recognition and the live text reader. I'm extremely impressed with the live text reader. Obviously no private documents unfortunately, but wow, everything else! It's not super accurate all the time yet, but I think a part of that is because you need to hold the camera very still. I could even read bits of a computer screen with it, as well as random notes and pieces of paper around the office. Considering this is 1.0, I am highly impressed.

Submitted by JeffB on Wednesday, July 12, 2017

I watched a video that said you can find out what pictures are from Facebook and other things like that. However I did not see Scan with Seeing AI when I picked a photo from messages. Has anyone had any luck doing this? If not I guess its something coming soon.

Submitted by Carlos Taylor on Wednesday, July 12, 2017

In reply to by JeffB

I opened my photos app and shared a few pics to seeing AI. I was amazed with the descriptions I got! Not sure how to make use of Facebook photos.

Submitted by JeffB on Wednesday, July 12, 2017

I can't find that option either in the photos app.

Submitted by Mattie SV on Wednesday, July 12, 2017

Hello all!!!

I am really loving this app so far however it really heats up my phone and it gets really hot and the battery drains fast!!! However overall I really love it!!! I really love the short text mode I really love when it reads live instead of me having to take a picture!!! So far it is really great and I'm looking forward to it getting better and better!!!

Submitted by Carlos Taylor on Wednesday, July 12, 2017

In reply to by JeffB

Locate a picture in the photos app. Double tap the picture. You should find share in the bottom left corner of the screen. Seeing AI should appear as one of the share options.

Submitted by Troy B on Thursday, July 13, 2017

I've just installed this app, but when I try to take a picture of myself using the frontfacing camera the app never sees my face when I snap the picture. Where is the camera on the front of an iPhone 7?

Submitted by Chris on Thursday, July 13, 2017

Hi,

I can't find the product category. When I swipe up on the slider, I have short text, document, person and scene. Why isn't the product category showing up? I'm using an iPhone 5S running iOS 10.3.2.

Submitted by smhy on Thursday, July 13, 2017

I also cannot find the extension for this app in the share sheet. I'm on an iPhone 6 running 10.3.1. But I do have the product channel.

Submitted by Blinken223 on Thursday, July 13, 2017

As instructed earlier, open the Photos app, select a photo, select the Share option near the bottom, but in order to have it appear, go at the bottom of the list, select the More option. From here, find the option to Open in SeeingAI, and turn it on. It seems to be turned off by default. You can even reorder the list of options on the Share Sheet this way. I've positioned it higher on the list of options. I'm guessing that this may be the same in other apps, you may have to enable it in other Share Sheets. HTH!

Submitted by Oliver Kennett on Thursday, July 13, 2017

Seems to crash when I try and get it to look at scenes or a picture, this may be due to using it in the UK.

For those having problems with the photos app, it's in the more part of the second section of sharing, along with print, airplay etc, you have to switch it on.

Submitted by JeffB on Thursday, July 13, 2017

I found it thanks for the help!

Submitted by jesseleivo on Thursday, July 13, 2017

This is really cool for a free app. But it just sucks that it is not available in the Finland app store! GRRRRRRRRRRRRRRRRRRRRRRR!

Submitted by Holger Fiallo on Thursday, July 13, 2017

Please will be nice if someone did a podcast. For a strange reason I can not get the idea of taking a picture. Back and front camera. I am good with technology but taking photos is hard to get for me.

Submitted by LaBoheme on Friday, July 14, 2017

so what is your favorite channel? mine is facial recognition. it seems to work pretty good, it can also recognize face in a photo. right now it only works with physical photos, i.e. paper photos, and not pictures on your phone.

Submitted by DMNagel on Friday, July 14, 2017

Who is to blame when an app is not in all countries? Is it Apple or is it The developer?

Submitted by Oliver Kennett on Friday, July 14, 2017

It's the choice of the developer. It's the first iteration of the app and as it uses off phone processing, I'd assume that they're wanting to scale up load cautiously. It would be beneficial for it to be launched all over the world as raises the profile of MS, so there must be a good reason why it's not outside the states just yet. Don't get me wrong, massively annoying, but, as most of us are apple advocates, we can hardly argue we've got a vested interest in this. It's free and therefore a unilateral transaction...

There are two more buttons in the share sheet, it's the second one and Seeing AI is right at the bottom of the list.

Submitted by Blinken223 on Friday, July 14, 2017

If some are looking for a podcast that talks about this app, Cool Blind Tech has just released an episode this week with Jessica Rickers going through how to use the app. As for availability, it appears to be available in North America as here in Canada, we had it.

Submitted by DPinWI on Friday, July 14, 2017

I read in the description that the app can identify currency. I assume this would be in the Product channel, but had no luck. I tried the Short Text, and i could read some words, but still no luck with identifying the bill. I tried Canadian and American bills.

Anyone else?

Submitted by mendi on Friday, July 14, 2017

I used it successfully at first and had a great time snapping pics of my team at work. But then, I tried it later that evening and/or yesterday. The app would say the verbiage about one face and where it was in the frame, but then when I snapped a pic it came back and said no people were detected. I'm baffled by that, as it said it saw a face but then when the pic is snapped nothing? Nonetheless, I do think this app has great potential, and I am looking forward to doing more with it!

Submitted by DPinWI on Friday, July 14, 2017

On some of the screens, I hear reference to a help video. While the app is simple and easy to navigate, I am interested in the video. however, I have yet to get it to play. Has anyone had success?

Submitted by Jesse Anderson on Friday, July 14, 2017

I wondered about this too the other day, and tried the various channels. I learned via Twitter that currency identification isn't yet in the current build but coming soon.

Submitted by Mattie SV on Friday, July 14, 2017

Hello!

I have successfully watched all of the videos! Most of the time it does work when I push the link for play video but if not if you have a 3D Touch capable iPhone I would 3D Touch on the link and push start playback or sometimes it will just start playing on its own unfortunately I have no experiences with non 3D Touch iPhones I hope this helps!

Submitted by Justin on Friday, July 14, 2017

Hi,
I've successfully gotten all the vids to play on my end. You select the category you want, find the help button, then there should be a button for playing the vid. Hit that, then your default video controls should appear, hit start playback and the video should automatically start playback.

Submitted by DPinWI on Friday, July 14, 2017

Thanks for the ideas about the videos. I have an iPhone 7, but did not try 3d touch. I will give that a go. All other forms of tapping, split tapping, and pass through tapping did not work for me.

Submitted by Toonhead on Friday, July 14, 2017

Here's exactly how to access the video tutorials, no 3d touch required. 1. With seeing AI open, double tap on the menu button. 2. Next, double tap on the help icon. 3. on the next screen, there'll be some text describing what Seeing AI is, and if you keep swiping you'll find a link that says "play the video tutorial. This launches the one that gives an overview of the entire app. Double tap on that link, and next, double-tap on the start playback button. You can do this for each editional video tutorial in the app. Just remember that there's a play video tutorial link for each video. I hope this helps!

Submitted by Troy B on Friday, July 14, 2017

I haven't tried much with this app yet, but I did try to take a picture of my surroundings while outdoors earlier and it told me it looked like a flock of birds sitting on something wood. Well it was half right because it was a wooden deck, but the flock of birds was actually a bunch of flowers.

I may not use that part of the app a lot, but I do think it'll be a great addition to the other apps on my phone.

Submitted by Remy on Friday, July 14, 2017

That is one feature I wish readers like this would be able to get better about. I can't tell you how many times I want to read a block of text on my 32 inch HDTV only and end up having various degrees of success. This one sort of does it, but even still, not very well.

Submitted by Will on Sunday, July 16, 2017

Hi all, i am sure it will come to the uk but we won't know when and it is a bit strange that it's not world-wide you would have thought a company like microsoft could handle requests. also i hope it comes to android to

Submitted by Jordan on Monday, July 17, 2017

In reply to by Erion

I heard about this app right about the time I was leaving for the NFB convention and was going to wait to get it. Well, it was during the convention that I ended up downloading it and it is now my go to app when I need to figure things out.

Submitted by Becca on Monday, July 17, 2017

This app is great! I was able to read my computer screen and the description was pretty accurate. Great app!

Submitted by Troy B on Monday, July 17, 2017

I just went to the product channel in seeing AI and scanned the side of a can that I knew had cashew nuts in it and I found the bar code easily and the app identified the product, but then I got a box of rice and although the bar code was detected and I believe that the code was in good focus given how fast the app was beeping the product was never identified. Is this just due to the work that still needs to be done with the app?

I will say that I've tried 2 or 3 different bar code readers in the past, both on the iPhone and otherwise, and given how easily I just found the bar codes on these 2 products I just tried seeing AI is by far the easiest thing I've used for finding bar codes and I'm very excited to test the app further.