SignMeUp

Providing an ISL learnig app for DHH community

Comment

SignMeUp

Inspiration

The idea for SignMeUp was born out of a desire to bridge the communication gap between the deaf community and the hearing world. Despite technological advancements, there is a lack of user-friendly platforms that teach Indian Sign Language (ISL) effectively. We wanted to create a solution that not only promotes inclusivity but also empowers users to learn ISL in an engaging and accessible way.

What it does

SignMeUp is an interactive app designed to teach users Indian Sign Language (ISL).

  • Advanced Dictionary: Users can explore an extensive library of ISL gestures, complete with visuals and descriptions.
  • Machine Learning Integration: The app uses ML models to recognize and verify gestures, allowing users to practice and receive feedback in real time.
  • Gamified Learning: Inspired by platforms like Duolingo, SignMeUp incorporates levels, challenges, and rewards to keep users motivated.
  • Accessibility First: A clean and intuitive interface ensures that users of all ages and abilities can navigate the app effortlessly.

How we built it

  • Frontend: Developed using Swift for iOS, ensuring a smooth and visually appealing user experience.
  • Backend: Leveraged iOS CoreML to integrate machine learning models that recognize ISL gestures.
  • Database: A secure cloud-based database stores gesture data and user progress.
  • Design: Focused on accessibility with a primary blue theme for clarity and inclusiveness.

Challenges we ran into

  1. Dataset for ISL: Collecting and preparing an extensive dataset of ISL gestures to train the ML model was time-consuming.
  2. Gesture Recognition Accuracy: Fine-tuning the ML model to ensure high accuracy in recognizing hand gestures.
  3. Time Constraints: Developing a robust app within the limited timeframe of the hackathon.
  4. User Experience: Designing an interface that is engaging yet simple for a wide demographic.

Accomplishments that we're proud of

  • Successfully integrated a functional ML model for real-time gesture recognition.
  • Built an extensive ISL dictionary with visual aids and descriptions.
  • Designed a gamified learning experience that keeps users engaged.
  • Developed an inclusive app that can make a meaningful impact in bridging communication barriers.

What we learned

  • The importance of accessibility and inclusivity in app design.
  • The challenges of training and deploying ML models in a mobile app.
  • Effective time management and collaboration during a hackathon.
  • Deeper insights into the needs of the deaf community and how technology can address them.

What's next for SignMeUp

  • Expanded Database: Add more gestures and phrases to the ISL dictionary.
  • Cross-Platform Support: Develop an Android version to reach a broader audience.
  • Voice-to-Sign Translation: Implement real-time voice-to-sign conversion for seamless communication.
  • Community Features: Enable users to connect, share learning experiences, and practice together.
  • Freemium Model: Introduce premium features, such as advanced lessons and certifications, while keeping the basic app free.

Built With

Share this project:

Updates