Linguify

Multilingual Educational Video Generator

Comment

Inspiration

IVC’s diverse student body includes international students, ESL learners, transfer students, and more, many of whom face language barriers, difficulty accessing academic content, or feeling disconnected from the community. Linguify directly tackles these challenges.

What it does

Linguify is an inclusive learning app that breaks down language barriers by letting users explore any subject through engaging, conversational videos between two of their favorite characters. With 24 custom trained multilingual voice models, Linguify makes education accessible and entertaining for people of all backgrounds, no subtitles needed.

How we built it

1) Initially, all screens were designed in Figma before development. 2) Frontend: React.js and HTML/CSS. 3) Backend: Python and FastAPI 4) User prompt is converted into a script between two characters using OpenAI API. 5) Trained 24 unique voice models on 4+ hours of voice lines including KIM CHAEWON, CAILLOU, and PETER GRIFFIN using the ElevenLabs Eleven Multilingual v2 model. 6) Compiled background footage, captions, voice lines, speaker PNGs, and background audio using MoviePy, a Python library that is able to render videos without a standard video editor.

Challenges we ran into

We found it difficult to train all 24 voice models. We spent around 6+ hours scouring YouTube to find raw voice lines for each of the speakers. It was also difficult to clone the accent of the speakers, with most of them having American accents. We also struggled with rendering times with MoviePy and ElevenLabs API, with some videos taking as long as 5-6 minutes to complete.

Accomplishments that we're proud of

We’re proud to have generated a complete video from nothing but plain text, bringing ideas to life from scratch. We’re also proud of how quickly we built a working prototype, going from concept to demo in just two days.

What we learned

For this hackathon, we switched languages and ended up working with languages that we were uncomfortable with. Justin was comfortable with Javascript, and Lucas was comfortable with Python. We decided to switch, as we thought that it would be a good learning experience for both of us.

What's next for Linguify

We would like to cut down the rendering times of the videos by optimizing the video creation algorithm that we have. In addition, we would like to add visual aids such as diagrams and graphs to our educational videos to make learning easier.

Built With

Share this project:

Updates