Inspiration
We saw the many potential uses of having an app that tells you the meanings of sign language. In general, most are not aware of the meanings of ASL and rarely require the opportunity to learn or practice it. Unlike traditional language, where text can be used to be translated relatively easily, sign language involves images that are not perfectly consistent. Thus, it was a solid candidate for us in using object recognition.
What it does
The ASL Reader is able to recognize sign languages of the ASL alphabet, using a webcam as reference, it looks for hands within a given image and outputs the letter that it thinks it sees to the user.
How we built it
Using Resideo's object recognization, the Reader is built upon the dataset of hundreds of images of hands making symbols. These images were used to train a model that recognizes a sign through some form of media (images, webcam feed, or youtube videoes).
Challenges we ran into
Projects involving machine learning often require many files and high processing powers to train the model. Throughout the Reader's creation, we struggled to train the models within reasonable time as our laptaps do not have GPUs. We ended up using Google Colaboratory to train our models which gave us results in reasonable time.
Accomplishments that we're proud of
As four newcomers to Hackathons, we are proud that we delve into such a complex and interesting project that stretched our understanding of machine learning and its usage and capabilities. Though it proved difficult in practice, we believe that we have learned much in the field of machine learning throughout this project.
What we learned
Throughout our research for what we would choose as our project, we discovered that there were numerous datasets out there that covers subjects of completely different categories, each bringing possibility of complex projects.
What's next for ASL Reader
Though our skills are limited, we wish to improve our capabilities as programmers to unlock more potential possibilities of machine learning and object recognition, possibly one day brining an ASL reader much more capabable than its current state.
Log in or sign up for Devpost to join the conversation.