New AI app for describing images and video: PiccyBot

I'm already subscribed, but would pay again for this. it'd be an wonderful thing to have especially if we could refine it to work fairly decent.

Firstly, great news about the Mac app - thank you. I was initially unsure I was downloading the right thing as the App Store suggested it wasn't a Mac app, but I installed it and opened it up and it seems to work well.

On the Mac, my main use case for this sort of thing is getting images on the clipboard described. But being on my work computer, I don't really want the automatically detection of clipboard on. I noticed there was an option to turn it off. Is there another way I can paste the image in? I tried but I only ever managed to get the last text I copied into the text box as opposed to the last image.

Whilst the feature is fantastic, I'm still a little unsure about how much I like the automatic option on the phone either. For example, I could be in an app and share an image to PiccyBot. So the app opens and immediately I'm prompted if I want to paste in the text I happened to have copied to clipboard before. It's a minor thing but I think I would prefer to be in control of when it happens. Is the only other way to do this in iOS as per the original version? I never really got the hang of it because I don't think it was really set for VoiceOver.

I would quite like an easy way to paste on my own terms in both applications.

And going back to the Mac, the other thing I would really, really like is a way to have the image described locally without going to the cloud. I'm a bit reluctant to put work images in here - I probably will if I have to, but I would really rather have something that was all local in those cases. Please correct me if I am wrong but I don't think an option like this exists now, so can it maybe be added as a feature request?

Thanks again for continuing to work on this. The amount of new features that have come in since I first subscribed is incredible.

Works as well as the Guided frames feature in Google pixel devices. It's come out really well for a first implimentation. but it'd be really nice and useful if it can be developed as a dedicated photography tool for the visually impaired with detailed instructions as to frame and capture pictures, using both cameras and not just portraits, but also of landscapes etc. Maybe one will have to surrender some of the creative autonomy to the AI in such a tool, but I'd be fine with that.

I feel that'd be great. The way I see it, the primary function of PiccyBot is image/video recognition and the primary function of the new app should be photography. Like I said in my last comment, I'm really looking forward to that app becoming an actual photography tool for the blind camera user. That would be empowering for so many at so many levels.

Hello Martijn and the PiccyBot community,
Thank you for developing such a fantastic and useful app! I find PiccyBot's image and video description features incredibly helpful.
I have a suggestion for a feature that would greatly enhance the experience for taking photos of people:
Could you please consider adding guidance for using the rear camera that also incorporates the ability to detect and count multiple faces?
Currently, using the rear camera for photos of people, especially groups, can be challenging. Adding audio guidance (like "Move left," "Two faces detected," or "Closer," "Further away") would make it much easier to frame the shot correctly and ensure everyone is in view before taking the picture.
This would be a game-changer for group photos and is a feature many users would appreciate.
Thanks again for all your hard work!

Yes thanks so much for continuing to improve the app.
I agree that having this kind of feature for the rear camera too would be great. And in both cases, having it work for both single and multiple faces or subjects.
If you were to go down the road of spinning it off into a separate app, I wonder could you designate it as an app that can be launched with the camera control button?
Dave

@Martijn you continue to be one of the first to bring these promissed excitements to the community! All the best for the good work!

I'm super excited to get my hands on this!
Having alternatives to meta AI will be a welcome change for many of us.
I do have a suggestion for the mobile app.
Would it be possible for the guided selfie mode to have the option to start a countdown and automatically capture selfies?
This was one of my favorite features of selfieX before it died.

Thanks again for the update, Martijn. I love seeing all these new things appear in PiccyBot and can't wait to give this a go.

I'm looking forward to seeing this evolve, as more tools become available. :-)

For everyone I mean?

That sounds amazing. Can you just clarify what is going on?

I think you are opening up PiccyBot as normal. Is it then sitting there listening out for a voice command, which you can speak through the microphone in the glasses? And so at that point it takes a picture and does its thing?

So if I was going out and about, could I just leave PiccyBot running and then talk to either meta or PiccyBot as needed? Does PiccyBot need to be in the foreground? Does it matter if the phone is locked?

Anyway really excited by this. I love how this app always seem to be ahead of the pack with new features, and genuinely useful ones at that.

I found a bug in the app: tapping in the subscription page the price of the different purchaase options with VoiceOver on doesn't activate the relative purchase option, I had to turn off VoiceOver and try tapping the right spot. Secondly a question: I sent a video and the app described it in a text description which the voice I chose read it sequentially without the audio of the video underneath it. Would it be possible to have the video and the audio playing at the same time and the audio is played and generating respecting the silences of the video, like it's done in a real audio description for films or TV shows? Meaning, it tells what happens when it really happens in the video.
Lastly, could we have more voice options for more languages? OpenAI voices such as Fable, Onix etc. are very good in English, but struggle with other languages like Italian, while ElevenLabs voices are much better.

When I'm in ask more to ask follow up questions, the app hangs. I'm noticing this on the latest update and this seems to happen quite frequently across the different LLM engines.