ImagePlex

Inspiration

Create an application that gamifies and makes learning languages accessible

What it does

User uploads a picture of an object and the word gets translated into another language

How we built it

There are 3 components in the tech stack. They are as follows:

1. Back end + web development using Django and Python
2. Object Recognition using YOLO Detector in PyTorch.
3. Machine Translation with Natural Language Processing by training a Transformer based LLMs. ## Challenges we ran into We ran into multiple challenges while creating this project.
4. Environment setup was one of the biggest challenge in a time crunch as it is not easy to get everything integrated together in a short time span.

What we learned

1. Learnt how to work and deploy Deep Learning models post training. For example, conversion of the model to ONNX to deploy it on web.
2. Learnt git in a shared environment. This made it fun and exciting.
3. Learnt the integration of multiple tech stack together.

Accomplishments that we're proud of

1. Successfully created a working demo for submission.

What's next for ImagePlex

1. Add more functionality to handle complicated objects and translation of written text on the image.
2. Creating real-time translations and deploying on Android/IOS with the help of mobile developers.

Built With

• css3
• django
• html5
• opencv
• python
• pytorch
• spacy