My Top Ten Accessibility Wishes for OS X 10.10

Member of the AppleVis Editorial Team

My Top Ten Accessibility Wishes for OSX 10.10

With WWDC coming up on June 2, we are very probably about to find out what iOS8 and OS X 10.10 will contain. AppleVis already has an article about our hopes for iOS 8 and Jonathan Mosen wrote up an iOS8 wish list of his own that is well worth a read, but no one has yet put forth a list of their hopes and dreams for OS X 10.10. I therefore thought I'd write down what I want to see in Apple's next Mac operating system. Note that these are not in any particular order.

Fix the Little Things

Currently, there are a number of small but annoying bugs floating around Mavericks. Pressing left or right arrow in text will sometimes jump several characters instead of one, Nuance voices have odd pronunciations for certain words and do other strange things, using Activities with certain voices can cause lag, and so on. While not show-stoppers in themselves, they all add up to make the experience of using a Mac less polished for VoiceOver users than it is for sighted users.

Bigger Problems

VoiceOver does have its share of small bugs, but there are a few more serious ones that really should have been addressed a long time ago.

  1. When you are reading items in a table, and new items are added, VoiceOver informs you "x rows added". This normally makes sense, unless you are reading longer items or you are getting a lot of added rows. Both mail and YoruFukurou, the Twitter client of choice for most visually impaired Mac users, have table-based interfaces. So, you are reading a tweet when, halfway through it, VoiceOver stops and says "1 row added". You start to read the tweet again, and it happens again. The problem is made worse if you use the message preview in Mail to read a message. You wade through all the intro to the message (see below), start reading the preview, and suddenly: "1 row added". You now have to start all over again. Worse, if you are using shift with up or down arrow to select multiple emails and a new one comes in, you are back at the bottom of your selection. That is, if you were using shift-up and had highlighted ten messages, then a new one arrives, VoiceOver will say "1 row added. 10 rows selected." and place you back on the first message you highlighted. Shift-up will therefore de-select your first message, instead of continuing to select. How long would the sighted world use an app that blanked the screen and displayed "1 new item" every time it retrieved a new item?
  2. If you pair your Mac with bluetooth speakers, and have VoiceOver set to use the system's default audio output, you get VoiceOver speaking through your bluetooth audio device. The problem is that, when you turn your bluetooth device off or move out of range, VoiceOver is silent. System audio continues to come through your speakers, or whatever the Mac falls back to when the bluetooth device is not available, but VoiceOver does not. You can sometimes get speech back by restarting the computer, but that is a ridiculous step to have to take. VoiceOver should fall back to the same audio the Mac does, not go completely silent.

The Mail App

Mail is a good app, and it does what it should do. The problem for VoiceOver users is that it is not as customizable as it could be, and some core features don't work consistently or are hard to use. Here are some examples:

  • there is no way to customize the order of columns. For each message, you have to hear sender, subject, date received, message preview, unread status, mailbox. If you are on a conversation, you are treated to the very verbose "x message collapsed disclosure triangle" message before you even hear the sender and the rest of the information. My ideal setup for Mail columns would be unread status, subject, conversation count (without the "disclosure triangle"), sender, preview. I can't do that, though, unless I switch to Classic View, which has its own problems that prevent me from using it. Another person's ideal layout will be completely different, and there should be a way to set up the column order to suit your needs. The need for this was made very clear when, in the first release of Mavericks, the conversation count and unread status were both moved to the end of the message readout, a move that caused so much negative feedback that Apple adjusted the order in 10.9.1.
  • The preview pane can only be closed with careful use of the mouse, or mouse keys. When not closed, any message you arrow to is marked as read automatically, even if you did not actually read it. The flip side of this is that you can press vo-j on a message to jump to the preview pane and read the content, then vo-j again to jump back to the message list. Some people love the pane, some people hate it, yet there is no toggle or other easy way of showing or hiding it. There used to be a simple Applescript command to do this, but that is broken in Mavericks, returning an error.
  • Speaking of VO-j, that command has gotten unreliable for many Mavericks users, at least in the Mail app. It sometimes jumps to the message content you want, but it just as often moves somewhere else, or does nothing at all.
  • Pressing enter to open a message used to be immediate. In 10.9, though, it can take up to five seconds, at least on my machine.

Better VoiceOver Control Through Cocoa

Cocoa is the programming framework on which all Mac apps are built. Its iOS counterpart, Cocoa Touch, offers some very useful tools to let iOS apps speak text using VoiceOver, test if VoiceOver is running, and more. As far as I can tell, Cocoa offers none of these, though it should. Games could announce when it is your turn, tell you which card you flipped over, or speak information like the current score with a keystroke; a scanning app could tell you when the scan is complete; a wordprocessor could speak a word count without making you change focus or dismiss a popup; you get the idea.

iBooks Needs An Accessibility Overhaul

iBooks was touted as a hot new feature in OS X Mavericks, letting you read books on your Mac or iOS device and sync bookmarks, reading positions, notes, and more across all your devices. It was a great idea, especially for the visually impaired community. Until now there has been no solid way to read .epub or .pdf files on the Mac, but iBooks supports both formats. Finally, we thought, a way to read on the Mac!

It was not to be. When iBooks debuted, it was with mostly unlabeled buttons, most ways of finding books inaccessible, and the reading of a book a subpar experience. With each incremental Mavericks update we hoped for changes, but the major problems in iBooks still persist. Apple just needs to sit down with a few proficient VoiceOver users, both students and professionals, and fix iBooks. It has the potential to be a huge asset, but right now it is barely usable. Yes, there are tricks and workarounds for some features, but Apple is known for their "it just works" philosophy, and iBooks just… doesn't work, not with VoiceOver at least.

iCloud Syncing of VoiceOver Preferences

For years, VoiceOver has had the ability to export its preferences to a file. You could use that file to set up any instance of VoiceOver just how you want it, even using a thumb drive so that, when you and your drive leave, VO reverts to how it was before you arrived. While this is a fine system, it would be even better with iCloud. Imagine getting a new Mac, and using the default VO settings only as long as it takes you to sign into iCloud. Once you do, your preferences are pulled from the cloud, and before you know it, VoiceOver is set exactly how you like it. No thumb drives, no installing a service like Dropbox and copying your preferences file from there, just the sign-in you'd do anyway and you're done.

Fix Xcode

Xcode is accessible, of course, but it has odd problems for VoiceOver users.

  • Using VoiceOver, there is no easy way to drag and drop UI elements when connecting them to code.
  • Xcode often becomes sluggish.
  • Xcode sometimes crashes, but according to some on a programming email list, I am the only one experiencing this. Given the frequent slow-down, I am forced to conclude that VoiceOver is negatively affecting Xcode in some way.
  • The things you drag when connecting UI elements to code are "unknown" objects. This becomes particularly annoying when you realize they are also the indicators for whether or not a connection is active; they should really be treated like checkboxes.
  • There should be ways of jumping focus around. To get to the line of code for an error you are examining in the Issue Navigator is currently five to six steps, when a single keystroke to focus on the line would be so much easier.

I'm missing others, but the point is that Xcode is usable, but not efficiently so. It also seems to not play nicely with VoiceOver and can slow down or crash completely for no apparent reason.

Keystrokes to Check Basic Information

Most screen readers I've seen include keystrokes to check the time and date and, in some cases, the battery level. While I realize that one person's important information is another's wasted keystroke, it seems like VoiceOver should have a built-in way of speaking the time/date and battery status. I am well aware that enabling the Keyboard Commander will do some of this, but I have two problems with the Applescript approach:

  • You have to know what you're doing. A new user can get by on just a few commands (vo-arrows, spacebar, and vo-h). Just to check the time, though, you need to know to enable a commander and then know how it works. This feels to me like an intermediate user feature, not a basic one.
  • It is slow. On my Mac Mini, which uses a mechanical hard drive, speaking the time takes up to five seconds. On a Macbook it is faster, but still not immediate.

I am also aware of the Status Menus, where you can find the time, date, battery level, wifi, and so on. However, getting there can be a relatively long process, at least compared to simply pressing a keystroke.

I debated including this since I know people will point out that it is already possible. Hopefully I have made it clear that my problem is more with how tedious, slow, or advanced the current methods are; I am not saying it is impossible, but compared to other screen readers, it is a lot to go through just to check the time or see if you should plug in your Macbook.

Separate Some VoiceOver Components from System Upgrades

Much of VoiceOver is tied deeply into OS X, as it has to be to provide the unparalleled access it does. It can speak in recovery modes, during system upgrades and installations, and so on, and it is hooked into all parts of the system at all times. The only problem with this approach is that it means no VoiceOver updates without OS X updates. This cannot be helped in some cases, but there are aspects of VoiceOver that could, I feel, be divorced from the operating system.

One example that comes immediately to mind is that of braille display drivers. These are simply protocols for how to talk to different displays, so when an update breaks a driver, Apple should be able to just issue a minor patch via the App Store to fix the problem. Right now, users must wait until the next minor or major OS update, yet Apple pushes security fixes, camera compatibility updates, firmware fixes, and other minor patches all the time. Why could the same not be done for parts of VoiceOver?

Small Improvements

Once the bugs are worked out, it would be nice to have, among other things:

  • support for regular expressions in the pronunciation dictionary
  • differentiation of pronunciation dictionary entries based on voice
  • a faster maximum speed for voices
  • a way to dynamically toggle sound ducking, such as with a Commander
  • extended Applescript support for VoiceOver, opening up the possibility of spoken VO commands when combined with spoken workflows in Mavericks and above
  • a customizable punctuation set, similar to NVDA's punctuation management
  • a hotkey to force all sound through the internal speakers, or cycle through sound sources one at a time (see the bluetooth audio bug above)

Conclusion

VoiceOver is a powerful, robust, and useful screen reader. The above are ways I feel it could be improved even more, though some items I would classify as critical.

What do you think? Did I miss things? Am i being too picky? What are the fixes and features you hope to see in the next release of OS X? Let me know in the comments!

Blog Tags: 

27 Comments

#1 I'm with You On the Battery Thing

Hello Mehgcap. I'm with you on the battery thing. I purchased the Battery Monitor app a few months ago, and although it works great for me you'd think Apple would've thought to include a hotkey for checking the battery status and such. I only got my Mac at the end of last year and therefore I'm probably not as experienced as most on this site. But I also think Mail needs some improvements. I think that's all on my end.

#2 Mail

I can definitely agree with you 100% in terms of mail. Another issue I've seen is Mavericks with Gmail. Gmail recently altered their iMap servers to be much more diverse, as well as offering support for all the different labels that you can see on the GMail website. Taking a Mac out of the box and instantly getting your Gmail set up with the mail application is literally impossible. I had to fiddle around with labels on the site -- not to mention how navigation was almost an annoyance (several times I had to use the mouse to find things, not the VO cursor, and even though it was still bad). The website is a GMail issue, not Apple, however. Mac just needs to obtain a better integration with GMail itself. I would've expected a fix to this in 10.9.3, but none was to be seen.
It's a severe turn-off having to use Outlook on a Windows virtual machine just to reliably receive mail in an exemplary fassion that supports my workflow.
I'd like to see more compatibility, as right now, it's fairly possible to get around the labels issue; I can receive my mail on iOS just fine.

#3 `

You managed to nail just about everything, so I don't have much to add here... except maybe language switching on webpages. If VO on iOS can do this, I realy don't see why the mac one couldn't. It's nice that you have activities, which make switching between multilingual synthesisers quick, but I still wouldn't mind having language detection on webpages. Speaking of multilingual voices, I don't understand why, even though Vocalizer Expressive supports pitch change natively with all variants, and VO uses it if you change the pitch, when it changes the pitch for EG capital letters, deleted characters or announcing detectors, the pitch change is very artificial and sounds just horrible.

#4 iMessages

Mehgcap, This was an excellent post. I agree with all of your suggestions. I'd like to add that iMessage should automatically read messages as they are received if you're in the iMessage window. I enjoy typing my iMessages on my Mac, but I prefer reading them on my iPhone. The Mac should work as well as the phone.

#5 Agreed, with 1 addition

I agree with what's above. I *especially agree with increasing maximum speech. I've been using synthetic speech in one way or another since I was very young, and how slow the maximum speech is in both iOS and OSX drives me absolutely batty, and interferes ith productivity. The faster I can use my speech, the faster I can read things, and get them done.
My other request for a fix is a minor one. VO+J in iMessage, if you have multiple accounts, doesn't take you to the list of conversations. You have to do it manually. Very annoying. Hopefully these things, along with the several bug reports I've sent in, will get some attention in the next update.

#6 Web browsing

Hey all,

Does anyone else get the issue where the keys for quick-nav mode just stop working after a while? I'm not sure if this is an OSX issue, or just especially for me as I use a Mac Mini without a screen and so have had other issues caused by this anyway. Table navigation on webpages is something else I'm yet to get my head around, even after reading all of the help info I could find. Perhaps I'm simply a slow learner, but I ended up setting up Bootcamp just so I could reliably search the internet as a fall back option.

#7 Addendum About Mail

Hi everyone. I just thought of something else in Mail that needs to be fixed, and I'm wondering if it's just me or if others have this problem too. Sometimes when deleting a message, it won't do so until I exit Mail. When I go back into Mail the message which I attempted to delete during my previous session will most likely have been moved to Trash. However, there've been times when these messages just lingered around and haven't gone to Trash at all.

#8 The mail bug has everythign

The mail bug has everythign to do wiht the server. This is not an apple issue. I shoudl know as I have done some pattern recognision on this nasty mail server issue.

#9 I do agree with everything

I do agree with everything posted. This shows a mature view of the system and also of the screen reader.

I would add one more thing and highlight one point posted by another person ...

1- Fix the slowness of the speech server. In IOS, a more limiited system running on a more restrict hardware this slowness does not exist and the source code is known by Apple ...... so I don't know why it is still that slow on Mac OS.
2- The Mac should work as well as the phone. It should ... it does not. Although I can understand that the cash is generated by IOS devices, the codebase between most part of both systems is similar, if not exactly the same. I think that if Apple implements something on one of these systems, porting it to the other shouldn't be hard, so again if the Mac OS worked like IOS in softwares such as ibooks and in voice aspects it should be great.
The other aspect to be noticed is scriptability ... if Apple cant do everything then let us do what we can, specially in making apps more usable. I think that xcode and garageband would be two good examples of things we can fix if Apple opens voiceover scriptability so that we could code workflows that would run incredbly well.

#10 I think the speach surver is

I think the speach surver is fine for me. it is plenty fast. Dunno what you mean by slow. I get thigns done and voices are very responsive.

#11 I just sent apple

I just sent apple accessibility this blog post. maybe this will gje ttheir attention as well. I encourage all of you to do the same with different words of corse. lol! but at least link to the post.

#12 The amount of time a key

The amount of time a key press or other commands will take to speak.

This issue is more perceived bby folks doing programming or terminal use and significantly reduce productivity on several areas. Pressing a key and taking 0.5 milisegonds to hear feedback is very different than pressing a key abnd waiting ten miliseconds or more to hear the feedback.
This is one of the reazons why voiceover is considered slow. It has nothing to do with the reading speed, this is something the synthezizer will solve but the delay between pressing an arrow or making a ciommand and the speech starts is something that needs to be addressed quick mainly because such delay does not exist on ios devices.

#13 Speech Delay

I use terminal on a regular basis, as well as some elements of XCode and get no delay whatsoever in terms of speech.
This could probably be a processor/memory thing.

#14 That'swhat I'm thinking as

That'swhat I'm thinking as well. I got a lotof them on my 4 year old mac book. It took like 15 minutes to start so yeah there you go. lol! The speach server is just awsome and constently getting better and getter.

#15 In addition to web browsing

Continueing the conversation to the person who mentioned web browsing, I don't know about any here but for me there a few things that I find really strange and some that are annoying too.

To provide my examples: this is with using Safari and Voiceover of course.

List, of 1 item, 2 item, 3 item. You get the idea?
I have found this more apparent of late on webpages. I don't know about others but I seem to get it all the time mainly when navigating by headings.

A couple of somewhat over spoken by Voiceover relating to links the 2nd of my two being more of an issue for me are:
Voiceover says: "internal link." For me at least a bit too much for my liking on webpages. By this I mean the word, internal. Unless I'm missing the purpose of what that word symbolises e.g. interacting with a particular link that it announces that presence on, I fail to see how it comes in handy for people who just want the link and not something before it. Unless of course it contains relevant information.
Moving forward to my 2nd point that I would really appreciate the accessibility team to look into has to do with links being reported of the contence of there meanings.
If that didn't make sense let me explain below. Before I do though, I should point that this isn't as much of an issue with Google Crome.

If you are reading an article say from Wikipedia and by the way, this is a source where you can experience this a lot. for whatever you search for.
In this example, I'll use Eric Clapton to illustrate what I mean.

So you pull up the Wikipedia page for him and a 1st example is: CBE Note: If you are just using the VO keys to move through the page or say all Voiceover will say "commander of the most excellent order of the british empire." In fairness, this sounds grate because it means one doesn't have to find out what that means but even then, I don't think that wording is strictly true because is one becomes an OBE then Voiceover says "Order of the british empire."
Why not just have for CBE "commander of the british empire." instead of including "of the most excellent order of the."

Of course, I don't spend all day looking for such people with titles but to just say a couple more so you get what I am trying to point out here. again, using the Eric Clapton page as our example.
Hard Rock, Hard Rock
Rock and Roll, Rock and Roll.
Chesse, Chesse Records
Richmond, Richmond, London

So with these examples, I hope you can see what I am referring too. It seems that, while Voiceover is pertentually repeating links and sometimes providing more information throughout the process, e.g. Richmond, Richmond, London. One finding out that Richmond is, you guessed it London. However, for me at least and I'd be interested to know what others think here I am becoming more tired of this sort of behaviour with links. I'd rather not here the extra verbiage and click on the link to find out where "Richmond." is myself without having Voiceover telling me that it is in London.

Again, I'd just like to say that when you use Crome none of this extra verbiage is spoken after links including no "internal links."

Conclusion
By no means am I bashing Apple here. I just want to say that I am providing my feedback in a professional and honest manor in order for Voiceover to become more less repetitive when using Safari.

Thank you for reading.

#16 Apple Accessibility, and VO with Links

Member of the AppleVis Editorial Team

First, thanks to all of you for wanting to provide feedback to Apple. Doing so is the only way to make our wishes and bug reports heard. I've have sent Apple the link to, and text of, this blog post. They responded with a promise to send it to the appropriate people. Instead of sending the whole thing again, I encourage everyone to pick the top items that they want to see fixed (I included those in my email to them) so they can see where the users' priorities are. That is, feel free to email them the link, but don't forget to explain your personal views on the matters as well.

As to links, "internal" tells you that the link jumps to a different location on the current page. For instance, any sort of "skip navigation" link is internal, since it does not open a new page but rather moves somewhere else on the already-opened one.

The extra verbosity on many links is indeed frustrating. I believe what VO is doing is reading the title attribute of the link. If you are reading this comment with VO, here we have a link that will be spoken normally. While this link will have extra wording at the end. The only difference between the two is that the second has a title and the first does not. My personal view is that the title should be part of a link's help tag, but that's just me.

#17 iMessages

Yes I also iMessages should be announced automatically.

#18 Link verbosity

Club AppleVis Member

Thanks for the explanation about link titles and how VoiceOver treats them. I was also wondering why I sometimes heard some redundancy when reading a webpage, especially on Wikipedia.

I do agree that the extra verbosity should not be spoken automatically at the end of the link text, but should only be spoken as a help tag. The only exception I would make is if the link does not have any normal text, such as a linked image or something similar.

#19 notification senter

I would like a few things to be more streamlined and refined. One of them being the notification senter. for me, the only way I've been able to load the notification senter is by turning off my trackpad and doing a two fingers swipe left from the right hand side of the trackpad. and getting rid of it is not as simple as hitting the escape key or doing a command q. One had to swipe over to the close button and double tap it to get rid of it. I don't think this is very efficient and wished there was a more quicker way of doing this.
Has anyone else noticed this or know if there is a easier way to working with the notification senter?
the other thing I would like introduced to voice over is being able to review system announcements more specifically, being able to review the last message spoken. this would come in handy when using Skype and being able to see who the last person logged in.
And finally, I would like a greater apple earpod integration with mac OS X. Although we are able to use the mic on the earpod with the mac, I would like certain functionality to also be integrated such as using the remote to skip ahead and backwards, as well as pause and resume play.

#20 Some Answers

Member of the AppleVis Editorial Team

First, you can access the Notification Center through the Status Menus (vo-m twice). For even easier access, go to System Preferences, Keyboard, Shortcuts. Choose "mission control" from the shortcuts categories table, then assign a shortcut to "show Notification Center" in the second table.

Second, VoiceOver already has a "repeat last phrase" command, it is vo-z (control-option-z). Sorry if this is not what you meant.

#21 Vo shift c is wht you want

Vo shift c is wht you want for reviewing and as stated by the prior poster you can set up a keyboard short cut for the notification center.

Take care.

#22 Vo-z, no shift

Member of the AppleVis Editorial Team

Vo-z repeats the last spoken phrase. Vo-shift-z saves it to the desktop as an audio file. Useful to some, but not if all you want to do is hear the last thing VoiceOver spoke.

#23 Another VO quirk

Hi! I think your list covered almost everything, but there are 2 things I think need fixing.

#1: Pitch changes need to become optional like they are in IOS. I don't like the way the pitch shifting is implemented, and there isn't a way to disable it as far as I know. I understand that many find that type of thing helpful, but for those of us who don't find it helpful, or deem it down right distracting, it should be something we can toggle on and off.

The second thing I'd like to see fixed is the way things in finder lists are read. Back in the Snow Leopard days, which is when I first got my MacBook, reading a list view was almost the same as in windows, although I wish all folders would be at the top and files listed under them rather than everything just being in alphabetical order. However, I understand that that particular aspect isn't VO related, so I won't complain about it here. What I don't like is that voiceOver has become far too vocal when reading through a list view. It used to just say the file name, extention and date. If it was a folder, it would indicate that last or at least in a way that wasn't annoying. Then, in Lion and Mountain lion, something changed, and VO said folder or whatever right after it read the name of stuff. I could demonstrate all of this if people don't understand. I found the saying of folder a bit annoying, but eventually got over it. Now, in Maverics, it's realy gotten out of control, especially if you're in the home finder window. Not only does it always say folder, but now it says disclosure triangle after everything. So, for example, you hit down arrow and it says something like "downloads folder disclosier triangle" Is all that truly necessary? What happened to something useful like "Downloads 6/1/2013 folder" If the folder bit is at the end, you can interrupt and skip past it if you choose. This new way is just getting worse and worse, and again, I can't find any VO settings to change to make it less verbose.

So, those are the changes I'd most like to see, but your list is really great too.

#24 Change your Finder settings

Member of the AppleVis Editorial Team

It sounds like your Finder settings are giving you more than you'd like. See my recent Finder article, specifically the section on changing view settings. Once in your view options, turn off kind, date, and anything else you don't want. I'd also recommend using List View, which has become my personal favorite, as well as setting VO's verbosity to medium instead of high. That should take care of your Finder problems.

#25 Wait. I'm confused. Hitting

Club AppleVis Member

Wait. I'm confused. Hitting Escape gets rid of Notification Center for me.

Also, I think you can pause/play with the EarPods remote. You just click the middle part of the button, since the top and bottom edges control volume. You can also click and hold to call Siri. Skip forward a track is a quick double-click. Fast-forward is double-click and hold. Skip back a track is triple-click and rewind is triple-click and hold. However, fast-forward and rewind do not work on the Mac. Am not sure how to do it with VoiceOver. Siri/dictation doesn't work with the remote, but that should be no problem with a hardware keyboard.

You can also control phone calls. Clicking once is answer/end, or swap if another call is coming in. Click and hold while the phone is ringing rejects the call.

Feel free to correct if I'm wrong.

Edit: I just sent an e-mail about iBooks to Apple yesterday. That's very high on my priority list.

#26 nice list

Hi, just a couple comments on this list, some of this can be worked around. In mail, if you want to avoid rows being added while you are selecting messages, you can disable the automatic checking for new messages. Also, regarding automatic speech in applications, this is already possible. Applications can access Apple's TTS voices directly, they don't even need to use voiceover to do it.

#27 Not the same

Member of the AppleVis Editorial Team

Using speech in OS X pas is definitely possible, through NSSpeechSynthesizer. The thing is, that isn't VO speech. Using VO directly lets your app speak with all the speech preferences (voice, rate, number format, punctuation, etc) the user wants. Detecting VO is also important, so you could use VO speech or fall back to system speech if VO is not on.

As to mail, that is certainly an option. I'm not a fan of disabling convenient features just to get around accessibility-specific problems, but it is better than nothing for now.