My Top Ten Accessibility Wishes for OSX 10.10
With WWDC coming up on June 2, we are very probably about to find out what iOS8 and OS X 10.10 will contain. AppleVis already has an article about our hopes for iOS 8 and Jonathan Mosen wrote up an iOS8 wish list of his own that is well worth a read, but no one has yet put forth a list of their hopes and dreams for OS X 10.10. I therefore thought I'd write down what I want to see in Apple's next Mac operating system. Note that these are not in any particular order.
Fix the Little Things
Currently, there are a number of small but annoying bugs floating around Mavericks. Pressing left or right arrow in text will sometimes jump several characters instead of one, Nuance voices have odd pronunciations for certain words and do other strange things, using Activities with certain voices can cause lag, and so on. While not show-stoppers in themselves, they all add up to make the experience of using a Mac less polished for VoiceOver users than it is for sighted users.
VoiceOver does have its share of small bugs, but there are a few more serious ones that really should have been addressed a long time ago.
- When you are reading items in a table, and new items are added, VoiceOver informs you "x rows added". This normally makes sense, unless you are reading longer items or you are getting a lot of added rows. Both mail and YoruFukurou, the Twitter client of choice for most visually impaired Mac users, have table-based interfaces. So, you are reading a tweet when, halfway through it, VoiceOver stops and says "1 row added". You start to read the tweet again, and it happens again. The problem is made worse if you use the message preview in Mail to read a message. You wade through all the intro to the message (see below), start reading the preview, and suddenly: "1 row added". You now have to start all over again. Worse, if you are using shift with up or down arrow to select multiple emails and a new one comes in, you are back at the bottom of your selection. That is, if you were using shift-up and had highlighted ten messages, then a new one arrives, VoiceOver will say "1 row added. 10 rows selected." and place you back on the first message you highlighted. Shift-up will therefore de-select your first message, instead of continuing to select. How long would the sighted world use an app that blanked the screen and displayed "1 new item" every time it retrieved a new item?
- If you pair your Mac with bluetooth speakers, and have VoiceOver set to use the system's default audio output, you get VoiceOver speaking through your bluetooth audio device. The problem is that, when you turn your bluetooth device off or move out of range, VoiceOver is silent. System audio continues to come through your speakers, or whatever the Mac falls back to when the bluetooth device is not available, but VoiceOver does not. You can sometimes get speech back by restarting the computer, but that is a ridiculous step to have to take. VoiceOver should fall back to the same audio the Mac does, not go completely silent.
The Mail App
Mail is a good app, and it does what it should do. The problem for VoiceOver users is that it is not as customizable as it could be, and some core features don't work consistently or are hard to use. Here are some examples:
- there is no way to customize the order of columns. For each message, you have to hear sender, subject, date received, message preview, unread status, mailbox. If you are on a conversation, you are treated to the very verbose "x message collapsed disclosure triangle" message before you even hear the sender and the rest of the information. My ideal setup for Mail columns would be unread status, subject, conversation count (without the "disclosure triangle"), sender, preview. I can't do that, though, unless I switch to Classic View, which has its own problems that prevent me from using it. Another person's ideal layout will be completely different, and there should be a way to set up the column order to suit your needs. The need for this was made very clear when, in the first release of Mavericks, the conversation count and unread status were both moved to the end of the message readout, a move that caused so much negative feedback that Apple adjusted the order in 10.9.1.
- The preview pane can only be closed with careful use of the mouse, or mouse keys. When not closed, any message you arrow to is marked as read automatically, even if you did not actually read it. The flip side of this is that you can press vo-j on a message to jump to the preview pane and read the content, then vo-j again to jump back to the message list. Some people love the pane, some people hate it, yet there is no toggle or other easy way of showing or hiding it. There used to be a simple Applescript command to do this, but that is broken in Mavericks, returning an error.
- Speaking of VO-j, that command has gotten unreliable for many Mavericks users, at least in the Mail app. It sometimes jumps to the message content you want, but it just as often moves somewhere else, or does nothing at all.
- Pressing enter to open a message used to be immediate. In 10.9, though, it can take up to five seconds, at least on my machine.
Better VoiceOver Control Through Cocoa
Cocoa is the programming framework on which all Mac apps are built. Its iOS counterpart, Cocoa Touch, offers some very useful tools to let iOS apps speak text using VoiceOver, test if VoiceOver is running, and more. As far as I can tell, Cocoa offers none of these, though it should. Games could announce when it is your turn, tell you which card you flipped over, or speak information like the current score with a keystroke; a scanning app could tell you when the scan is complete; a wordprocessor could speak a word count without making you change focus or dismiss a popup; you get the idea.
iBooks Needs An Accessibility Overhaul
iBooks was touted as a hot new feature in OS X Mavericks, letting you read books on your Mac or iOS device and sync bookmarks, reading positions, notes, and more across all your devices. It was a great idea, especially for the visually impaired community. Until now there has been no solid way to read .epub or .pdf files on the Mac, but iBooks supports both formats. Finally, we thought, a way to read on the Mac!
It was not to be. When iBooks debuted, it was with mostly unlabeled buttons, most ways of finding books inaccessible, and the reading of a book a subpar experience. With each incremental Mavericks update we hoped for changes, but the major problems in iBooks still persist. Apple just needs to sit down with a few proficient VoiceOver users, both students and professionals, and fix iBooks. It has the potential to be a huge asset, but right now it is barely usable. Yes, there are tricks and workarounds for some features, but Apple is known for their "it just works" philosophy, and iBooks just… doesn't work, not with VoiceOver at least.
iCloud Syncing of VoiceOver Preferences
For years, VoiceOver has had the ability to export its preferences to a file. You could use that file to set up any instance of VoiceOver just how you want it, even using a thumb drive so that, when you and your drive leave, VO reverts to how it was before you arrived. While this is a fine system, it would be even better with iCloud. Imagine getting a new Mac, and using the default VO settings only as long as it takes you to sign into iCloud. Once you do, your preferences are pulled from the cloud, and before you know it, VoiceOver is set exactly how you like it. No thumb drives, no installing a service like Dropbox and copying your preferences file from there, just the sign-in you'd do anyway and you're done.
Xcode is accessible, of course, but it has odd problems for VoiceOver users.
- Using VoiceOver, there is no easy way to drag and drop UI elements when connecting them to code.
- Xcode often becomes sluggish.
- Xcode sometimes crashes, but according to some on a programming email list, I am the only one experiencing this. Given the frequent slow-down, I am forced to conclude that VoiceOver is negatively affecting Xcode in some way.
- The things you drag when connecting UI elements to code are "unknown" objects. This becomes particularly annoying when you realize they are also the indicators for whether or not a connection is active; they should really be treated like checkboxes.
- There should be ways of jumping focus around. To get to the line of code for an error you are examining in the Issue Navigator is currently five to six steps, when a single keystroke to focus on the line would be so much easier.
I'm missing others, but the point is that Xcode is usable, but not efficiently so. It also seems to not play nicely with VoiceOver and can slow down or crash completely for no apparent reason.
Keystrokes to Check Basic Information
Most screen readers I've seen include keystrokes to check the time and date and, in some cases, the battery level. While I realize that one person's important information is another's wasted keystroke, it seems like VoiceOver should have a built-in way of speaking the time/date and battery status. I am well aware that enabling the Keyboard Commander will do some of this, but I have two problems with the Applescript approach:
- You have to know what you're doing. A new user can get by on just a few commands (vo-arrows, spacebar, and vo-h). Just to check the time, though, you need to know to enable a commander and then know how it works. This feels to me like an intermediate user feature, not a basic one.
- It is slow. On my Mac Mini, which uses a mechanical hard drive, speaking the time takes up to five seconds. On a Macbook it is faster, but still not immediate.
I am also aware of the Status Menus, where you can find the time, date, battery level, wifi, and so on. However, getting there can be a relatively long process, at least compared to simply pressing a keystroke.
I debated including this since I know people will point out that it is already possible. Hopefully I have made it clear that my problem is more with how tedious, slow, or advanced the current methods are; I am not saying it is impossible, but compared to other screen readers, it is a lot to go through just to check the time or see if you should plug in your Macbook.
Separate Some VoiceOver Components from System Upgrades
Much of VoiceOver is tied deeply into OS X, as it has to be to provide the unparalleled access it does. It can speak in recovery modes, during system upgrades and installations, and so on, and it is hooked into all parts of the system at all times. The only problem with this approach is that it means no VoiceOver updates without OS X updates. This cannot be helped in some cases, but there are aspects of VoiceOver that could, I feel, be divorced from the operating system.
One example that comes immediately to mind is that of braille display drivers. These are simply protocols for how to talk to different displays, so when an update breaks a driver, Apple should be able to just issue a minor patch via the App Store to fix the problem. Right now, users must wait until the next minor or major OS update, yet Apple pushes security fixes, camera compatibility updates, firmware fixes, and other minor patches all the time. Why could the same not be done for parts of VoiceOver?
Once the bugs are worked out, it would be nice to have, among other things:
- support for regular expressions in the pronunciation dictionary
- differentiation of pronunciation dictionary entries based on voice
- a faster maximum speed for voices
- a way to dynamically toggle sound ducking, such as with a Commander
- extended Applescript support for VoiceOver, opening up the possibility of spoken VO commands when combined with spoken workflows in Mavericks and above
- a customizable punctuation set, similar to NVDA's punctuation management
- a hotkey to force all sound through the internal speakers, or cycle through sound sources one at a time (see the bluetooth audio bug above)
VoiceOver is a powerful, robust, and useful screen reader. The above are ways I feel it could be improved even more, though some items I would classify as critical.
What do you think? Did I miss things? Am i being too picky? What are the fixes and features you hope to see in the next release of OS X? Let me know in the comments!