Inspiration
During the hackathon's welcome ceremony, we were inspired by the sign language translator. We realized that during many events, a sign language translator is not available, so a real-time transcriber could be extremely helpful for both deaf audiences and the general population.
What it does
Hosts can create an event, and watchers can join by using a unique event ID. Panscriber streams and transcribes the video in real-time, translating to a language of each watcher's choice.
How we built it
We used Flask for the backend, and socket.io for streaming audio and subtitles between hosts and watchers. We used Materialize for styling.
Challenges we ran into
We had quite a bit of trouble getting audio formats to work with each other, since the MediaStream Recording API only outputs webm files, and Google Cloud Speech-to-Text does not work with webm. However, we resolved this by using the SpeechRecognition web API instead.
Accomplishments that we're proud of
Learning a lot about video streaming.
What we learned
We learned a lot about WebRTC and video streaming, as well as various Speech-to-Text APIs.
Log in or sign up for Devpost to join the conversation.