Lingly

Sample firebase preview
Project structure
Basic backend setup

Inspiration

Since we were all raised by immigrant parents, we all know the challenges of communicating with relatives that speak another language, and how this distance makes it seem like your own family members are complete strangers. Thus, by creating Lingly, our team hopes to eliminate this struggle and make it so people from diverse cultures and backgrounds can all communicate and build lasting relationships with ease.

What it does

Our project allows users to effectively and efficiently communicate with others speaking different languages automatically. For example, when you text your friend in English but they only know how to speak Chinese, our project will automatically translate the English text to Chinese, displaying the Chinese translated text to your friend while displaying the original English message to yourself.

How we built it

The front end for this project is created using React and Typescript, with Tailwind for styling. We use Firebase for data retrieval/storage, and a custom API through Flask. Our custom API implements Google Cloud Translation and ElevenLabs, while also facilitating communication between our frontend and the API.

Challenges we ran into

Time was definitely our biggest issue. We would have liked to add a lot more features to our project, such as transcribing audio recordings to your native language and even video calling with deepfake technology, but unfortunately ran out of time towards the end. A major frontend issue was making the sizes of all the screen elements work for all screen sizes because we struggled to make them relative to the user. On the backend, adding APIs and integrating them was a huge struggle because we got lots of 400 and 403 errors from invalid API keys. When it came to collaboration in general, we ran into several merge conflicts due to unclear communication, thus causing some code to be temporarily misplaced or even lost.

Accomplishments that we're proud of

We’re definitely proud with the amount of progress we got through in 24 hours. We achieved our main goal for this project overall, which was to build a convenient and attractive chat app that allows people with different native languages to instantly understand what the other person sends them. Our UI is also quite dynamic because there is an option to toggle between light and dark mode, and this applies to every screen. Although we struggled through them a lot, we felt quite proud after we got the API for translating text into our project, since it was our coolest feature.

What we learned

Through this project, we all learned new APIs such as Google Cloud’s translation and ElevenLabs that we hadn’t known before. We also learned how to effectively work together. At the beginning of the hackathon, we drew clear diagrams of our frontend and backend and effectively split up tasks in order, making our work very efficient at first. However, as time went on, there were less things to work on, therefore making it harder to split up tasks without more than one person working on the same file and causing merge conflicts. Additionally, we felt quite burnt out after coding for several hours straight, so we learned that it’s better to take a breather once in a while if struggling on some really annoying bug, since you can reapproach the issue with a fresh mind.

What's next for Lingly

We definitely hope to extend our project to a lot more people. As of now, we only have the option to translate 20 different languages, but we hope to be able to increase the amount of supported languages to as many as possible. Another thing that we were working on but didn’t have time to finish was audio translation. Essentially, a user would record themselves saying something, which will get converted to text using some speech-to-text API, so it can be translated and transformed into a new audio clip with the user’s deepfaked voice saying the same thing but in the translated language.