Inspiration
I love watching movies in different languages. However, sometimes I am left to rue the absence of subtitles. This project is dedicated to provide the functionality to generate subtitles in the language of your choice.
What it does
Our workflow first fetches from the user the video in mp4 format. Using OpenAI Whisper API/CLI, we then transcribe the video in its original language. Once the transcription is obtained in SRT (subtitle format), we go on to translate it to the user's target language. We can then upload the procured translated SRT file as a subtitle file for the video.
How we built it
We followed the instructions found on the Whisper GitHub page. Using the Linux command line, we installed the necessary packages. This was followed by invoking the Whisper API/CLI to transcribe the video saved on the local file system after which it was translated. The final SRT file generated was then uploaded to the video to display the subtitles.
Challenges we ran into
The API/CLI is very slow ... It takes almost a day to generate the transcription ... This will be improved in the future as OpenAI improves their NLP model.
Accomplishments that we're proud of
One video successfully finally has subtitles!
What we learned
How to use OpenAI Whisper and also about subtitle formats.
What's next for Missing Subtitles ???
We need a mechanism to make this faster. Also, we can automate this entire workflow by using AWS. Specifically, we would allow the upload of the video to an S3 bucket. This will trigger a Lambda/Fargate to start the processing. Whisper has an equivalent python API SDK for its CLI commands. We can use that implementation in python for the AWS compute services.
Log in or sign up for Devpost to join the conversation.