Inspiration
SignMate was inspired by the need to make communication more inclusive for the 466 million people worldwide living with disabling hearing loss. We wanted to create a tool that would help bridge the communication gap by enabling people to learn American Sign Language (ASL) in an interactive and engaging way.
What it does
SignMate is a web app that generates random ASL videos of common English words. The user attempts to identify the word shown in the video by typing it out. If successful, they receive positive feedback and can continue with a new set of videos. If unsuccessful, the app provides a failure warning, highlighting the missed words and encouraging improvement.
How we built it
We built SignMate using React for the front-end and Node.js for the back-end, ensuring smooth interaction and functionality. For the ASL dataset, we used a Kaggle dataset, which provided a variety of ASL images and videos for common English words. The app uses these resources to generate random ASL videos and evaluate user input.
Challenges we ran into
One of the major challenges we faced was ensuring the accuracy of user input in matching the ASL gestures with the correct words. Additionally, working with video data from Kaggle required optimization for quick loading times while maintaining video quality.
Accomplishments that we're proud of
We are proud of building an interactive web app that not only serves as a learning tool but also fosters inclusivity. The app provides immediate feedback, which helps users improve their ASL skills. It was exciting to integrate the ASL video dataset and create a smooth user experience.
What we learned
Through this project, we gained valuable experience in working with large datasets, optimizing video content for web apps, and handling real-time user interactions. We also learned about the importance of accessibility and inclusivity in digital applications.
What's next for Sign Mate
Next, we plan to expand the app’s features by incorporating more advanced ASL vocabulary, improving the dataset to include more diverse signs, and possibly adding speech recognition to allow for a more immersive learning experience. We also aim to enhance the mobile version for greater accessibility.
Log in or sign up for Devpost to join the conversation.