Stop Waiting for AI to Tell You What to See. Start Exploring It Yourself.

Great minds think alike! I was updating this as you made that comment :).

Hi Ming,
Thank you for your suggestion! I understand you're looking for a feature that can capture your computer screen and read out game menus when navigating with arrow keys.
This is a challenging request because this app is built as a progressive web app (PWA) that runs in your browser. Web browsers have significant security restrictions around screen capture - they can't directly access another application's screen content or intercept keyboard events from other programs like games. This is by design to protect user privacy and security.
Why we chose a web app approach:
• Universal accessibility: Works on any device (iPhone, Android, Windows, Mac) without separate downloads or app store approvals
• Instant updates: Everyone gets new features immediately without reinstalling
• No storage concerns: Doesn't take up device space like native apps
• Cross-platform: One codebase works everywhere
• Easier to maintain: We can focus on building features rather than managing multiple platform-specific versions
The technical challenges for your use case:
• Web apps are sandboxed and can't access content outside the browser tab for security reasons
• Intercepting arrow key presses from games would require system-level permissions that browsers don't have
• Reading game menus would need the game developer to provide accessibility APIs, which is outside our control
Game accessibility is ultimately dependent on game developers implementing accessible features in their games. Unfortunately, this isn't something a web app can bridge due to browser security limitations.

Live Camera Mode is great - drag your finger around, explore what's in front of you, get detailed descriptions. But what if you just want to walk somewhere without touching your phone?
That's the whole point of AI Navigator.
Three-finger swipe down → just start talking.
No dragging. No tapping. Just conversation.
"Take me to the bathroom." Guides you step-by-step with live distance updates
"What's in front of me?" Describes everything ahead without you lifting a finger
"Where's the exit?" Spots it, tells you exactly where and how far
It reads building directory maps automatically, remembers every location, and keeps track of where you've been. Multi-floor buildings? No problem - it'll guide you to the elevator, tell you which floor to go to, and pick up navigation when you arrive.
It's proactive too. Spots spills, moving people, stairs - warns you before you get there.
Different tools for different moments. Sometimes you want to explore with your hands. Sometimes you just want to get somewhere. Now you can do both.
Testing it now. Updates coming soon. 🚀

PS: Goes without saying but to go completely handsfree you would need some sort of mount :).

I'm building those features now so when Meta hands out that nice little API I can work to see how to implement. I'm staying 5 steps ahead with getting the infrastructure in place and we can go from there :).

Screen sharing in a browser is not a system-level capture mechanism, and it cannot be used to directly interpret or interact with the contents of other running applications such as games as Ming is requesting.

On desktop systems, a web app yes can request temporary access to capture a specific browser tab, application window, or monitor using the browser’s built-in screen sharing permission prompt. That access is strictly limited to what the user manually selects, only remains active during that session, and provides nothing more than a raw video feed. It however does not expose the actual interface structure, menu hierarchy, UI elements, focus states, or internal game data. In other words, it can show pixels, but it cannot understand what those pixels represent in a reliable, semantic way.

In addition to that, a web app cannot intercept or listen to arrow-key input that is being handled by another application. The browser only receives keyboard input while it is the active, focused window. Once focus shifts to a game, the browser and any web app running inside it immediately lose access to those key events. This makes real-time “menu tracking” based on arrow keys technically impossible in a standard web environment.

On mobile platforms, especially iOS, the restrictions are even more severe. Safari and Progressive Web Apps cannot capture or analyze the screen contents of other apps at all. They also cannot run continuous background capture or monitoring processes. These limitations are enforced at the operating system level to protect user privacy, security, and data isolation between apps.

While some platforms demonstrate controlled screen capture in specific environments, those implementations operate within tightly constrained ecosystems and do not equate to universal, system-wide access for third-party web applications. A typical web app does not gain the ability to observe other programs or interpret their user interfaces simply by using screen sharing.

For a feature that reads game menus dynamically while navigating with arrow keys, the only technically valid approaches are:

• Native system-level accessibility services provided by the operating system
• Direct accessibility APIs implemented by the game developer
• Or a dedicated desktop application with elevated permissions and OS-level hooks

These capabilities are intentionally blocked from browsers and PWAs to prevent surveillance, input capture, and unauthorized data access across applications.

Thanks for letting me know. I don’t have an android so I’m counting on my android friends to yell at me if there is any bugs. I’ll fix that with the next update :).

I’ll definitely look into this for you 😊.

Hi Ming,
What feature were you using? Was it live camera explorer or photo explorer?
Did you double tap on an object and then swipe left to zoom in? Were you able to do the tutorial or did it pop up for you at all?

I need more details in order to help you better 😊.
Also, yes it does need a camera to work.

Hey everyone! 🎉
You're Amazing!
The app is temporarily down, but here's the thing - you've all been using it SO MUCH that we've completely blown through our integration limits! This is honestly the best problem to have. It means you love what we've built! 🙌
What's happening:
Your enthusiasm has pushed us past our usage caps way faster than expected. The sheer volume of activity shows how valuable this tool is to the community.
What's next:
To keep things running smoothly for everyone, we're rolling out subscriptions sooner than planned. This ensures reliable, uninterrupted service and lets us scale properly to handle all your usage.
Thank you for being such incredible early adopters! Your support means everything, and this "problem" is actually proof we're building something people truly need. 💪
Stay tuned - we'll be back up very soon!

All the features should be working again. Also the subscription price will be lowered to $4.99 USD. I will be putting this in shortly.

Oh absolutely. Their will be both monthly and annual. I have to just build those products and because we are moving everything to subscription I have to get a legal TOS as well so those are going to be my main focus to implement within the next few days.

Those are some awesome suggestions my friend. I really love the prompt idea especially. I’m gonna look to see how to implement that. Thank you so much again and don’t hesitate to throw your suggestions at me 😊.

Hello everyone,
I wanted to share an important update about Vision AI Assistant and explain the changes we're implementing today.
Why We're Moving to Subscriptions
When I launched Vision AI Assistant, I never imagined the response would be this incredible. You've been using the app so extensively that we had hit our AI service limits far sooner than anticipated. This is genuinely the best problem to have - it means you're finding real value in what we've built together.
However, the reality is that every AI vision analysis, every voice interaction, and every real-time navigation feature costs money to run. The current usage has made it clear that to keep the service reliable and available for everyone, we need a sustainable business model.
What's Changing
Starting today, Vision AI Assistant requires a $4.99 USD/month subscription to access the full app. This is not a decision I made lightly, but it's necessary to ensure:
• Reliable service for all users without interruptions
• Continued development of new features you've been requesting
• Priority support so I can help you when you need it
• Sustainable infrastructure that can scale with our growing community
Why No Yearly Plans Yet?
I know some of you would prefer an annual subscription option, and I promise it's coming. However, I'm intentionally waiting before offering yearly plans because:
1.
Scalability Testing: I need to see how the infrastructure performs at scale over the next few months. Vision AI is resource-intensive, and I want to make sure I can confidently provide reliable service for an entire year before asking for that commitment.
2.
Service Guarantee: If I take your money for a full year, I want to be absolutely certain I can deliver consistent, high-quality service for those 12 months. Right now, we're still optimizing costs and infrastructure to ensure long-term sustainability.
3.
Fair Pricing: Once I understand the true costs at scale, I'll be able to offer a yearly plan that provides real savings while ensuring the service remains viable.
My Commitment to You
I'm committed to making Vision AI Assistant the best accessibility tool it can be. Your subscription directly funds:
• Server costs and AI processing
• Ongoing feature development
• Quality improvements and bug fixes
• My time to provide support and implement your feedback
Thank you for being part of this journey. Your enthusiasm and feedback have been incredible, and I'm excited to continue building something truly useful for the blind and visually impaired community.
If you have any questions or concerns about this change, please don't hesitate to reach out through the "Contact Developer" feature in the app or give me a call at:
1-866-825-6177
[ext]: 0
It is a toll-free number so you can contact me from anywhere without any charges.

With gratitude,
Stephen.

All right, so I had to make the app free for now. Having some bit of technical difficulties getting subscriptions implemented. I’m putting this on the back burner for today lol.

Yeah I’m looking at it now. I’m sending it back too you. I got rid of stripe this morning so I don’t know why the app is still holding onto it so I might have to pull it off-line for a moment.

I was actually trying to implement the 24 hour trial, but I think by doing so I might’ve messed up the web hook somehow and it is looking like that is messing with stripe. Needless to say I’m rolling back these subscriptions and just going to keep it free until I can figure out this stripe situation and what’s going on with it.

Tip jar here:
https://www.paypal.com/ncp/payment/8RUHTTVFBJDCQ

Hey everyone.
I know there’s a lot of comments here so I will try to get to them ASAP but I wanted to jump on here to let everyone know that we have reached our integration quota.
Right now all features will be very limited or completely down until the billing cycle. Yes I had moved it up to $500 a month on my end, but you folks are phenomenal and have been using it so much we have to wait till it resets as my monthly budget for this project is over extended at the $500 a month. Hang tight, enjoy your holidays and will be back online soon.

We officially have a GoFundMe!
https://www.gofundme.com/f/vision-ai-assistant-empowering-independent-exploration

All app services have been restored.

Hi Diego,
Thank you for your detailed feedback! Let me address each of the issues you mentioned.
Tap and Hold Opening Menu in Photo Explorer:
I've been trying to reproduce this issue but haven't been able to trigger it on my end. This sounds like it might be a conflict with VoiceOver or another system accessibility feature intercepting the touch gestures. Please make sure VoiceOver is completely turned off, it can interfere with the app's gesture handling and cause a regular tap to be interpreted as a long press. If VoiceOver is definitely disabled and this is still happening, please let me know so I can investigate the touch event handling.
AI Embellishments in Photo Exploration:
I hear your feedback about the embellishments. I want to explain why they're included: many users have told me they really appreciate the detailed, descriptive language because it helps them not just understand what's in the photo, but actually visualize and connect with it emotionally. When exploring a Christmas tree photo, for example, users have said they want to feel immersed in the scene, to "feel" the warmth and texture through the descriptions, not just get a basic list of objects. It's about creating an experience that's engaging and meaningful, like having a friend describe the photo with enthusiasm rather than reading a catalog.
That said, I understand everyone has different preferences. Some people want concise facts, others want rich descriptions. I'm considering adding a setting that lets you choose between "detailed" and "concise" description modes so you can customize it to your preference.
Book Reader - Portuguese Language and Major Update Coming:
You're absolutely right about the Portuguese issue. The current book reader only works in English. I was working on adding Portuguese support to the existing system however I've scrapped that and I'm developing a completely redesigned book reader that I just finished yesterday and is currently in testing. This should solve all the problems you had mentioned.
This new version is called Interactive Live Text Reader, and it works completely differently. Instead of capture-analyze-display, it works like the Live Camera Explorer mode. The camera stays on continuously, and you drag your finger across the screen to read words in real-time as you touch them. It's like your finger is directly touching the words on the page through the camera. You get instant feedback, no capture-retry loops, and a much more natural reading experience.
The reason I'm mentioning this is because I'm implementing Portuguese language support in this new version using a completely different architectural approach. Instead of trying to retrofit Portuguese into the old capture-based system, I'm building the language support into the new live reader from the ground up. This will make Portuguese work properly across all the real-time OCR processing, text-to-speech output, UI announcements, and gesture instructions.
The new Live Text Reader includes:
• Real-time word reading: Drag your finger across the screen, words are spoken as you touch them
• Double-tap to spell: Double-tap any word to hear it spelled out letter by letter
• Two-finger double-tap: Reads the entire page automatically from top to bottom
• Automatic page change detection: When you turn the page, it detects the new page number and clears the old text automatically
• End-of-page detection: Announces when your finger reaches the bottom of the page
• No capture-retry loops: The camera is always on, so you get immediate feedback on positioning
I'm finishing testing now and expect to release it within the next few days. The Portuguese language integration is being built into this new system, so when it launches, Portuguese users will get the full experience. Proper OCR recognition of Portuguese text with all accent marks, all spoken feedback in Portuguese, and all UI instructions in Portuguese or what ever language you have the app set to.
Book Reader Freezing Issue:
Your description of the freezing issue is exactly what I've heard from other users and is why I'm doing a rebuild. When the OCR can't detect text, it gets stuck asking you to turn the page without actually helping you understand what's wrong. Your suggestion for better alerts is excellent. It should say things like "No text detected, try moving closer" or "Image is too blurry, hold steadier" instead of just looping on "turn to next page" however,
the new Live Text Reader completely eliminates this problem because there's no capture-fail-retry loop. The camera is always on, you're always seeing what it sees, and you get immediate feedback by dragging your finger. If text isn't being detected, you know right away because you're not hearing any words, and you can adjust your camera angle, lighting, or distance in real-time. It's a fundamentally better approach to the problem.
Thank you for taking the time to report these specific issues. This kind of detailed feedback is exactly what I need to keep improving the app!

This is not an easy message to write.

As of today, Vision AI Assistant may be facing closure as early as January 8 , pending the outcome of a critical meeting scheduled for January 7. While no final decision has been made yet, it’s important to be transparent about where things currently stand.

Vision AI Assistant was built with a clear purpose: to give blind and low-vision users more agency, more independence, and a more exploratory way to understand the world around them. From day one, this project was driven by lived experience, accessibility-first design, and a belief that assistive technology should empower curiosity, not limit it.

Unfortunately, the reality behind the scenes has been far more complex.

The ongoing backend and infrastructure costs required to keep Vision AI Assistant running have continued to rise well beyond what was originally projected. Despite careful optimization, personal financial investment, and countless unpaid hours poured into development, testing, and support, the platform has reached a point where it is no longer financially sustainable under its current model.

At this time, I can no longer justify injecting additional personal capital into the project without a clear and viable path forward. This decision is not about lack of belief in the idea, the community, or the technology itself, it is purely about sustainability and long-term feasibility.

The meeting on January 7 will determine whether there is any realistic alternative: restructuring, outside support, or a last-minute solution that could allow Vision AI Assistant to continue in some form. If no such path emerges, the service may officially shut down on January 8.

To everyone who tested the platform, shared feedback, reported bugs, advocated for it, or simply believed in what Vision AI Assistant was trying to do, thank you. This project existed because of a community that cares deeply about accessibility, innovation, and doing things differently.

No matter the outcome, the work done here was not wasted. The ideas, lessons, and conversations sparked by Vision AI Assistant will continue to influence what comes next, even if this chapter comes to a close.

Further updates will be shared as soon as a final decision is made following the January 7 meeting.

I can’t thank you enough for your kind words. Getting this to where it is has taken more time than I would like to admit. The biggest challenge with the subscription is properly implementing stripe on the back end for subscriptions. I had to pull the subscriptions originally because it was charging people double then not giving them access to the app. Testing that is going to take quite a bit of time and maybe a few months. In order to try to keep the lights on for this project and to keep it funded, I had set up a go fund me. I really wanna keep this project moving forward, but I obviously can’t afford to take hours away from my full-time job unless this app becomes self-sufficient. Your support would be greatly appreciated. The GoFundMe link is below.
https://www.gofundme.com/f/vision-ai-assistant-empowering-independent-exploration

I want to share an important update about the future of Vision AI Assistant and how we’re working to keep the service running.

Until now, the app has been supported largely through myself and community donations. To create a more stable and transparent path forward, we’re going to try something new. On a limited basis, we’ll begin acknowledging individuals, companies, and organizations that financially sponsor the project by featuring them on the login screen. Placement will only be available to those who have made a direct financial contribution to keeping the service operational.

In some cases, a sponsor may also include a link to their service or website. Every organization will be carefully vetted in advance. This includes accessible employers who actively hire blind and low-vision people, as well as businesses, services, and organizations that genuinely support accessibility, autonomy, and independence. This is not an open advertising marketplace and it is not pay-to-appear without standards. Nothing will be displayed that compromises trust, accessibility, or the overall user experience.

This is a new approach for us, and we’re moving forward thoughtfully. The goal is sustainability without exploitation—finding a way to extend the life of the service without shifting costs onto users or introducing intrusive advertising.

I wanted to be transparent with the community about what we’re trying and why. Vision AI Assistant exists because of this community, and this experiment is about giving the project a realistic chance to continue, improve, and remain available to the people who rely on it.
If you are an organization please give us a call at:
1866-825-6177
To discuss further.
If you are an individual, please ensure to donate through our GoFundMe link at:
https://www.gofundme.com/f/vision-ai-assistant-empowering-independent-exploration
If you prefer to remain anonymous on the app, please reach out and let me know.
Thank you all so much.