Inspiration

Communication is a fundamental human connection, yet millions of Deaf and hard-of-hearing individuals face barriers due to the lack of widespread sign language knowledge. Our inspiration for sAIgn comes from the desire to bridge this gap—whether it's for a parent, a friend, or simply to make the world a more inclusive place. We wanted to create a fun and engaging way for anyone to learn ASL and break communication barriers.

What it does

sAIgn is an interactive sign language learning platform that gamifies the learning experience. Users can progress through structured lessons, track their achievements with XP and levels, unlock trophies, and compete on a leaderboard. The app also integrates AI-powered sign recognition, allowing learners to practice and receive feedback on their signs in real time.

How we built it

Frontend: Built using React, Next.js, and Tailwind CSS for a clean, responsive, and dynamic UI.

Backend: We developed two separate backends—one using Node.js and Express.js to handle user data, XP, trophies, and lessons, and another in Python, which integrates Google's Hand Gesturing API and GROQ LLM for sign detection.

Database: MongoDB stores user progress, levels, and achievements.

Security: Bcrypt is used to hash passwords before storage for added security.

Deployment: The platform is being deployed on GoDaddy to ensure accessibility.

Challenges we ran into

One of the biggest challenges was learning AI for the first time and integrating it into our system. Working with hand gesture recognition APIs was a completely new experience, requiring us to experiment with different models and improve accuracy. Additionally, building a real-time, responsive experience with two different backends was a complex task. We also had to ensure that user progress was stored correctly and trophies were awarded at the right milestones.

Accomplishments that we're proud of

Successfully integrating AI-powered hand gesture recognition for ASL learning. Creating an engaging XP and leveling system to keep users motivated. Implementing a secure authentication system to protect user data. Designing a leaderboard and trophy system to make learning more interactive and competitive. Learning new technologies and overcoming AI-related challenges as a team.

What we learned

How to work with AI and machine learning models for hand gesture detection. The importance of gamification in making learning engaging. How to build and manage a full-stack application with multiple backends. The value of real-time feedback and progress tracking in education. The challenges and best practices of deploying a web application.

What's next for sAIgn

Improving AI accuracy for detecting hand gestures and sign recognition. Adding real-time video chat for users to practice sign language with each other. Expanding lesson content with more advanced ASL topics and interactive exercises. Developing a mobile app for on-the-go learning. Partnering with ASL educators to create verified lessons and certification programs. Enhancing community features to connect learners with native signers and mentors.