I was sitting at Dish network a couple of years ago waiting for another call to come over the Sling TV customer service line, as that was my job. I had a lot of time between calls. I'm one of those people who is quite creative, but don't know how to code, so all these thoughts just hang around my mind, in less they are music or other audio. I thought of JAWS implementing a new feature, that would be, so to speak, scripting for dummies, or, the myself, the general JAWS user. I would call it JAWS AI learning. You would go into an inaccessible app, you would perform a keystroke, JAWS would take some time and label the apps various controls, using OCR to discover the text which corresponded to each control, and let the user know the type of control. After this machine learning was completed, JAWS would take the end user through a wizard, the list of controls to allow the end user to see if the machine learning had been accurate, and allow the end user to make any necessary changes. IF the end user had a sighted person with them JAWS would show where the control in question was on the screen for even better scripting accuracy. Once a person had finished the app could be quite accessible. I had never written to FS about this because, I feel as the general public, my thoughts wouldn't gain any traction. Now a couple of years later, in iOS we have a some what similar feature, that will only get better in time. I personally couldn't be more excited.
I tried an inaccessible app this morning, Pacemaker with the Voiceover recognition enabled. This app would not do anything at all before. I have had it for a few years. With this new feature, it wasn't totally usable but I could at least get somewhere.
after iOS got image recognition in Voiceover other screen readers also picked this up. Now, could my idea come to a reality in another mainstream screen reader? maybe so. We all, as blind people spend so much time with software that isn't accessible. Jobs aren't available for us because of this, as one of the main holdbacks. If this did come to the main stream computing world, this would open a lot of doors! Again, Apple, way to go! You guys always innovate, and I am always truly so thankful for it! I hope someone who has some clout in the accessibility world sees this post, or builds off of these machine learning ideas in the future.