Designing a new equation-editing app - what should I focus on?

Thanks for taking a stab at this. I've given this problem some careful thought over the past few years.

One of the big drawbacks to reading in-line ASCII equations is not hearing how the equation is supposed to sound when read aloud.

e = MC^2 should be read as e equals mc squared. I would love the ability to move past each of the coefficients and know where in the equation it is relative to the other coefficients. This is an easy one as it all fits on one line. C squared is its own coefficient and a fairly easy one to grasp.

For more complex equations, we start running into issues:

V = f/ᴧ: read aloud as V equals f over lambda. Is there a way to communicate the spatial relationship of f over lambda relative to V?

Another one:

ps = T-D/W*V: Read aloud as: P sub s equals T minus D over W times V. (This is a formula for measuring energy states in jet fighters.)

Question: In this formula, is the quantity T minus D to be divided by W or, is D over W to be subtracted from T?

I have found it helpful to use the term, "the quantity" to describe what is in brackets. For example, the quantity X squared minus 1 times the quantity X squared plus 1. This lets me know the terms of the quantity are self-contained. Inspecting by character could reveal brackets or parens.

Where T minus D is all over W, I would like to hear VoiceOver say something like: "P sub s equals the quantity T minus D over W……. In the case where the term "D over W" is to be subtracted from T, VoiceOver would have to distinguish this by saying, "V equals T minus the fraction D over W…".
The next part of the equation becomes tricky. If VoiceOver says, "Times V, my expectation is that the fraction T minus D over W is to be multiplied by V. If I hear divided by WV, I would know W is to be multiplied by V and that WV is its own term.

VoiceOver should be able to handle exponents with the terms like, "to the power of ~" and subscripts with the term "sub ~".

There is a part of this program that would definitely benefit from having someone who knows the linguistics of math or science to develop the vocabulary which would dictate how VoiceOver would speak the equation. If this is laid out in a concise way, there could be some type of NLP interface that would allow someone to dictate the formula.

I would also look at using the Apple Pencil to write in equations and have them translated to a usable form on iPad. Some of us with sight loss know how to write out the equation. Perhaps there is an affordance to do this in a stepwise fashion.

It would also be helpful for those who know Nemeth braille code to key formulae in with refreshable braille displays.

Always happy to add more thoughts.

First, as you probably know, blind people would have difficulty reading equations and expressions that appear in the 2D format that sighted people are used to. Any equations or expressions I have worked with using speech or braile are format in a 1D, linear format.

Secondly, it can be difficult for a blind person to understand the logical relationships in long and/or complex expressions. One method of coping with this is to have a gesture or keyboard shortcut for easily moving from, say, a left parenthesis to the corresponding right parentheses and vice versa.

Probably the best way of representing complex expressions to the blind is how most screen readers are dealing with MathML expressions these days. using mathML and a screen reader one can navigate from sub expression to sub expression, up and down an expression at a time, or navigate within an expression. In this way logicaly connected content is kept together and the blind person can have a better understanding of the overview of the expression.

I have also made use of assigning variables to sub expressions. For example a complex expression might have many sub expressions within it. If the larger expression is written in terms of numberbed variables, the user can then query each of the numbered variables separately to see the details of each sub expression but yet still have a good overview of the larger expression and how it is put together.

Hope that feedback helps. As a retired research physicist who is blind, I know that blind people can have rewarding careers in STEM and like to see work such as yours that makes tools like this readily available to those considering careers in STEM.

Keep up the good work and keep us posted on progress.

--Pete
P.S. If you are eventually interested in adding the capability of actually solving equations and performing symbolic math, look into some of the CAS open source projects that could make this possible. Maxima is a great example that works on PC's and I believe people might have worked on porting such tools to mobile devices.

CJ,

One more suggestion I have that could help make this app useful and productive to the blind is to include a complete set of keyboard shortcuts that can be used to perform important functions such as moving from parentheses to parentheses, navigating expressions and sub expressions, etc. Gestures can be kind of limited and accidental touches of the screen can easily change focus to unexpected places. Even sighted users like using keyboard shortcuts to make their work more efficient.

--Pete

Hey, I was looking through some old posts about doing math on a Mac and ran into yours. Did you ever get to finish the app?
I'd love to try it or test it now that I have a Mac.

Thanks for the update. I'm a software engineering student, but new on Mac. I started learning iOS development so in future I would be open for collaboration.