ImLing

Inspiration

This idea stemmed from our common interest in image recognition and language learning. We are a team of visual learners who learn best when we are able to make connections between text and the physical world. We realized that existing solutions for language learning fail to take into account a key component of learning new languages -- visual connections. We decided to call it ImLing for ImageLinguistics!

What it does

This app captures snapshots of the user's surroundings and identifies the main object in the image, and displays the user's selected first language and language they want to learn.

How I built it

Android Studio was the IDE we used to implement the Firebase API into this project. This part was responsible for the image recognition and translation features. The user interface was developed and enhanced using Sketchware.

Challenges I ran into

None of us were experienced with Java or Kotlin before this hackathon, or Android Studio for the matter. It took us some time to learn the key syntax needed to code the app. We also ran into issues with setting up the Firebase API. We needed to persistently consult various mentors as well as online sources to resolve them. Moreover, we encountered various setup issues with Android Studio. We had difficulties developing the UI and incorporating the camera into the app.

Accomplishments that we're proud of

We are proud that we learned many skills during this weekend and made an app that could illustrate our idea. Previously, we had never used APIs, but we have learned to implement them with this project!

What I learned

We learned how to use some features of Firebase and became more familiar with Android Studio. We learned how to incorporate camera usage and image recognition into mobile apps.

What's next for ImLing

Next steps for ImLing include:

Extend the variety of languages that are supported by the app
Increase the number of objects that are identified through image recognition in each captured frame
Extend beyond objects and account for situation-based phrases that can help people start conversations
Replace the image capturing with AR so that the object labeling can be live and more convenient