Each episode is only 20 minutes long, so I thought that open whisper would be great to translate from French to English.
However. It failed dismally. Constant, regurgitation of repeated sentences. Throughout entire episodes open whisper used “him” instead of “her” and many other instances of misspelling. It would fail if there was music playing in the background.
I extracted the audio from the videos into small .wav format and .mp3 format but both failed.
I spent over a week trying to create suitable subtitles to no avail.
Whisper struggles with non-english languages and background noise - you might get better results using the larger models (medium/large) with a lower temperature setting to reduce the hallucinations and repetitions your experiencing.
I have used Open-whisper and Fast-whisper to do subtitles.
Open whisper is easy to set up and install locally. I tried various models.
Recently I tried to do the French series En Therapie (In Therapy) which has 35 short episodes.
https://www.arte.tv/fr/videos/RC-020578/en-therapie/
Each episode is only 20 minutes long, so I thought that open whisper would be great to translate from French to English.
However. It failed dismally. Constant, regurgitation of repeated sentences. Throughout entire episodes open whisper used “him” instead of “her” and many other instances of misspelling. It would fail if there was music playing in the background.
I extracted the audio from the videos into small .wav format and .mp3 format but both failed.
I spent over a week trying to create suitable subtitles to no avail.
Whisper struggles with non-english languages and background noise - you might get better results using the larger models (medium/large) with a lower temperature setting to reduce the hallucinations and repetitions your experiencing.
hey MysteriousSophon21
I did use the larger and medium models with Open whisper and Fast whisper.
I did not consider the lower termperature settings
Thank you