AI-powered Audio Description: seeking feedback from the community

The app itself is currently in development. Right now, I'm interested in feedback on the quality of the audio description itself. You can send me a video, and I will send back the audio description for it.

Thanks for pointing that out!

If this project ever turns into a full public app, my plan is to keep the pricing significantly lower.

I wanted to compare the quality with ViddyScribe directly, but as far as I understand, creating standard audio description without pauses requires a paid subscription. If anyone has an example of a video processed with ViddyScribe, I’d be curious to check it out and compare the results with my tool.

I have decided to release my program, Intellica, with a few preliminary warnings. You can download it using this link:
https://pub-45231abf13e3432aba06f2fed23b44d9.r2.dev/Intellica-RU-Portable.zip
First of all, there is almost no help documentation yet, and the small part that does exist has not been translated into English. I am currently working
on this. Therefore, if you have any questions, you can ask me right here or write to my email. Additionally, right inside the application, there is a Contact
Support button. You can write to me through it, and there is an Attach logs checkbox there. If you check it, your logs will be sent to me, which will make
it much easier for me to figure out the cause of any problem you might encounter.
Generally speaking, this program is an aggregator of various neural network services. Beside audio description, which I will focus on in this guide, there
are many other features if anyone is interested. You can chat with text models and recognize PDF documents, including those with complex structures and
mathematical formulas. So, if you or someone you know is involved in mathematics, this is a very convenient way to get documents in LaTeX format. There
is also audio recognition for transcribing speech to text, image generation, speech synthesis using voices from Gemini and ElevenLabs, music generation
via Suno, Google Lyria, and ElevenLabs, as well as sound effects generation from ElevenLabs.
Let me explain how the subscription and balance system works. To use the program, you need both an active subscription and some funds on your balance.
The subscription costs 3 dollars, and it is intended to cover server costs and provide a small income to motivate me to develop the project further if
it reaches a significant number of users. You can top up your balance with any amount, and this money is spent directly on requests to the AI models. Each
service has its own price. Unfortunately, there is no centralized price table yet, so when trying a new service, I advise starting with small volumes to
estimate the cost. For example, upload 3 to 5 pages of a PDF document, or just a few minutes of a video. I understand this is a drawback and I will work
on it, but the project has grown quite large, and since I am working on it alone, I have not had time to do everything I wanted yet.
Regarding payment methods available today, PayPal works. There is an automatic payment option via PayPal in the program interface, where you just select the balance amount and subscription duration and it does everything automatically. Another option is transferring money to a Kazakhstan bank card, which is no problem at all if that works from your country. Finally, there is also a crypto wallet available for payments. Additionally, you can use your own API keys for Gemini, including for video processing if you have enough quota and limits on them.
Now let me show you how to create an audio description. First, you need to register in the program by entering your name, email, and password. The email
is not verified at the moment, so please type it correctly so that you can receive password recovery emails if you ever forget your password. Upon registration,
you receive 1 dollar for testing and a one-month subscription. You will hardly be able to make a full audio description for 1 dollar, maybe just try a
one-minute clip, but it is enough to test other services like chatting or PDF recognition.
In the main window, press the Tab key until you reach the service categories, which include Chat, Voice Speech, Music Audio, Graphics Video, Search, and
Utilities. We need the Graphics Video category. You can also open the audio description service directly by pressing the shortcut Control Shift D.
Once opened, press the Load file from computer button, which opens a standard file selection dialog, and choose your video. Next, move to the Audio description
instruction edit box. This is for additional information for the model. I usually write the title of the movie and list the names of the main characters
here. This helps the model not to get confused with names as much.
Then, you can choose the model in the combo box: Gemini 4, Gemini 3.8 Flash, or Gemini 3.7. Currently, the best one is Gemini 4, although it is the most
expensive of the three. I choose Gemini 4. After that, go to the Audio description language combo box and select your language. Then press the Start button.
If you check the read-only Status edit box, it will say Analyzing video. Wait for some time while the video is being processed.
When the work is finished, you will hear a sound signal, and the built-in player will open. Here, the commentary is read automatically by your screen reader.
From here, we can already save the audio track or the video, but first, let us do one more thing. The examples I sent you earlier were made using the Improve
audio description with AI button. This is an additional pass through the model after the initial processing to adjust the placement of the audio descriptions
so they overlap less with the dialogues, and to fix any confusion that is obvious from the text context. At this stage, the video itself is no longer viewed
by the AI; it only improves the text and timing of the commentary.
When you press the Improve audio description with AI button, a warning appears stating that this may cost several dollars. For a full-length movie of an
hour and a half, it really can cost a few dollars because the model processes a very long text of dialogues. Because of this, I set a restriction so that
you can only use this feature if you have more than 5 dollars on your balance, to prevent a situation where you run out of money mid-process and lose the
result. If you are willing to sacrifice a bit of quality to save money, you can skip this step and save the audio description as it is.
Inside the improvement dialog, there is also an edit box where you can paste additional context if you know something specific about the movie. For example,
if you have a script describing what happens at each minute, the model can cross-reference it. Or, if you noticed in the initial result that a character's
name was mixed up, you can write an instruction here clarifying the character's real name. You can also just leave it blank, which I often do because extra
information is not always easy to find and it increases the cost, even though it helps a lot. Press the Start button and wait for the sound signal again.
When it is ready, we can move on to saving the file to the computer. Press the Save button. First, press the Select file path button to choose where your
file will be saved, and select the format, such as MP3 or FLAC.
For speech synthesis, we have three options in the synthesizer combo box. The first is SAPI 5, which uses the synthesizers installed on your computer and
is completely free. The second option is Edge TTS from Microsoft. It is also free, so you can try it, but it can be unstable and sometimes fails to synthesize
certain segments. The third option is Gemini TTS. It has the most realistic voices, and this is the one I used in the video examples I shared with you,
but it is a paid option. Also, Gemini TTS has a quirk where it sometimes refuses to synthesize certain phrases. I programmed a feature so that if it encounters
such a phrase, it will prompt you to rephrase it. Ironically, since I added this fix, I have not encountered the problem once, so I hope it works as intended,
but please let me know if any issues arise.
Next, you can select the specific voice and the speech rate. Then there is the Pause original video checkbox. If you used the AI improvement step, you
generally do not need to check this, as the commentary already fits into the pauses quite well. However, if you skipped the improvement and the description
overlaps heavily with the dialogue, you can check this box, and the video will pause while the commentary is being read.
If you check the Save with video checkbox, the program will save the full video file. If you leave it unchecked, it will save only the audio track, or
you can choose to save solely the audio description track without the original movie sound if you want to edit and mix it manually in a third-party editor
later.
I usually check the Duck original audio checkbox, which lowers the volume of the movie while the audio description is being read. You can also set the
ducking level. In the examples I sent you, it was set to the default value of 50 percent. Finally, press the Start or Save button at the bottom. In the
status field, you will see messages like Synthesizing audio segment as the progress goes on, and once it finishes, your file will be ready.
Finally, for those who want to use the general assistant and chat features, the program has a set of global hotkeys. Pressing Control Shift F1 toggles
the visibility of the program window. Control Shift F2 starts or stops voice recording with automatic message sending, while Control Shift F6 toggles voice
recording without sending it automatically. Control Shift F5 sends the currently composed message in the chat. Control Shift F7 captures a screenshot,
and Control Shift F11 captures a screenshot and sends it immediately. Control Shift F9 stops the active UI tool, and Control Shift F12 starts a new chat.
I'd like to emphasize that upon registration, you get one dollar on your balance. You can just spend it in the app, completely free of charge.