Substitutor

Inspiration

Learning a foreign language is extremely time-consuming and often requires dedicated periods of time set aside to solely focus on working towards this goal. We wanted a more efficient solution that intrudes in our daily lives as little as possible. A large portion of our day is already spent browsing the internet and reading all its text. We came up with a way to seamlessly incorporate language-learning into this daily routine.

What it does

Substitutor accesses text that is displayed on the user’s screen during their browsing experience. It intelligibly substitutes certain words from English to a language that the user wants to learn. The user can be comfortably absorbed in their reading experience until they come across a term or phrase from another language. If they recognize the term, then exposure to this term further reinforces memory retention. Else, they can derive the meaning of the foreign term(s) by using contextual cues—a strategy proven to have a profound effect on increasing the ability to learn new words (İlter, 2019). Then, they can hover over a term to reveal the English translation.

How we built it

To build Substitutor, we used JavaScript, OpenAIs ChatGPT api, HTML, and CSS. The HTML, CSS, and JavaScript were used to style our pop-up and define on-click events such as saving a users selected difficulty and language to the browsers cache. From there we used the saved data from the cache, text information from the webpage, and OpenAI’s GPT api to intelligently select words to translate that would still allow the sentence to make sense from the context. After translating the text, we used a service that overwrites the English text on the current page with the translated text. We also added a feature to allow the user to see the untranslated English version of the word by simply hovering over it.

Challenges we ran into

To build Substitutor, we started with a boilerplate repository for chrome extensions, which used Webpack and TypeScript, to construct our extension. However, Webpack started to become a development bottleneck as its bundling gave rise to issues when we attempted to run our service workers and background functions. As such, we chose to build the extension without boilerplate. We also had trouble having a one shoe fits all for all the different DOM elements. Thereafter, we attempted to leverage open AI’s Chat GPT. Finding a prompt that gives deterministic outputs was initially tricky. Furthermore, the ability to translate a variable amount of queried text depending upon the users chosen difficulty also created troubling design choices that required quick workarounds.

Accomplishments that we're proud of

We overcame multiple roadblocks through intensive research and workarounds. Most notably, we created a clean chrome extension (we have never done this before) and also allowed for a real time manipulation of the text that the user sees and interacts with. We also added last minute features such as connecting to a pronunciation api to also learn the pronunciation if users are curious

What we learned

We learned the significance of forward planning. When we had our main idea, a lot of features came into our mind, especially future directions that might not have been tractable in a limited time frame. These features distracted us from identifying and focusing on our MVP, which made us overlook some major flaws that we did not previously consider.

What's next for Substitutor

We have 4 main plans for our product in the future.

Add a text-to-speech feature for the foreign vocabulary where the audio pronunciation would play upon a click.
Implement range of difficulty levels with narrower increments to facilitate more progressive learning, and allow for autonomous adjustments in level.
Build an analytical dashboard which offers users insights into their learning progress, like the number of words they've learned, their most frequently visited translations, or words they struggle with.
Gradually expand to support more languages such as French, German and Chinese.