Math Ninja

"Imagine if every time you sliced a fruit in the air with just your hands tracked by AI-powered hand recognition, you also leveled up your math skills. Welcome to MathNinja."

Comment

🌟 Inspiration

Everyone knows Fruit Ninja, the iconic game where you slice fruits like a ninja. 🍉⚔️
But here’s the twist: what if instead of just improving reflexes, the game also sharpened your brain?
What if it combined the thrill of slicing with learning?

That question gave birth to Math Ninja 🥷 where AI-powered hand-tracking meets math learning.
Instead of tapping a screen or dragging a mouse, you wave your hands in the air 🖐️, and our system uses MediaPipe AI/ML models to trace your hands and fingers through a simple webcam.

No controllers. No sensors. Just you, your math skills, and your ninja reflexes.

What It Does

At its core, Math Ninja is a gesture-controlled educational arcade game:

  • A math equation appears (e.g., 9 − 4 = ?).
  • Several fruits fly across the screen, each carrying a potential answer.
  • You slice the fruit in the air with your hands ✋ using webcam-based gesture recognition (powered by MediaPipe AI).
  • ✅ Correct slice → Score increases.
  • ❌ Wrong slice → Lose a life.
  • ⏱️ You get 60 seconds per round with 3–4 lives.

Extras that make it addictive:

  • 3 difficulty levels → The math gets harder as you level up.
  • Interactive sounds → Web Audio API generates satisfying “slice” and “explosion” effects in real time.
  • Achievements & Roadmap → Track milestones (e.g., first 100 points, streaks, speed bonuses).
  • Leaderboard → Global top 3 players fetched dynamically from MongoDB.

🛠️ How We Built It

Frontend (React + AI/ML + Web APIs)

  • React Components manage game flow:

    • GameScreen.jsx: Runs the equation + fruit spawning + gesture detection.
    • GameOverScreen.jsx: Displays score summary and sends data to backend.
    • MenuScreen.jsx: Entry point with difficulty selection.
    • LeaderboardScreen.jsx: Fetches & displays top 3 players.
    • AchievementsScreen.jsx: Shows player progress & unlocked milestones.

  • AI/ML Integration (MediaPipe)

    • Detects hands & finger gestures via webcam in real time.
    • Translates hand swipes into slicing actions.
    • No extra hardware → accessible with just a webcam.

  • Interactive Audio (Web Audio API)

    • Generates real-time slicing, smashing, and reward sounds.
    • Creates a fully immersive arcade vibe.

  • Authentication & Progress

    • SignIn.jsx + SignUp.jsx with Firebase Auth.
    • User data synced with MongoDB for persistent progress tracking.

⚙️ Backend (Flask + FastAPI + MongoDB + Firebase)

  • FastAPI/Flask Layer

    • Connects React frontend with the database.
    • Validates Firebase authentication tokens for security.
    • Handles score submissions, leaderboard queries, and achievements.

  • MongoDB

    • Stores users, scores, achievements, and leaderboard.
    • Schema optimized for fast updates after each game.
    • Example collections:
      json users: { userId, email, displayName, achievements[] } scores: { userId, score, timestamp }

  • Firebase Services

    • Auth → Secure sign-in/sign-up.
    • Ensures each user has a unique identity linked to MongoDB.

🚧 Challenges We Faced

  • Real-time Gesture Control → Calibrating MediaPipe’s hand-tracking to respond fast enough for a game environment.
  • Cross-Stack Integration → Making React ↔ Flask ↔ Firebase ↔ MongoDB sync smoothly without lag.
  • Leaderboard Consistency → Handling async updates so scores appear in real time without duplicates.
  • Latency Minimization → Balancing AI hand detection + game rendering at high FPS.

🏆 Accomplishments We’re Proud Of

  • Built a hands-free AI-powered game where you slice fruits with gestures in real time.
  • Seamlessly integrated React, Flask/FastAPI, MongoDB, Firebase, and MediaPipe AI.
  • Added leaderboards, achievements, adaptive difficulty, and immersive sounds.
  • Created a platform that’s fun, educational, and hardware-light, playable by kids using just a webcam.

What We Learned

  • AI/ML in Games → Applying MediaPipe to turn a regular game into a functional, gesture-based interactive system.
  • Full-Stack Integration → Managing token validation, API contracts, and cross-origin requests.
  • Gamification of Learning → How to make math fun without losing its educational value.
  • Performance Optimization → Keeping gesture detection + game rendering smooth in the browser.

🚀 What’s Next for Math Ninja

  • More Math Topics → Fractions, algebra, and geometry challenges.
  • Multiplayer Mode → Real-time gesture battles with friends.
  • Power-Ups & Special Fruits → Freeze time, double score, or penalty bombs.
  • Mobile/Tablet Deployment → AR-style slicing with mobile cameras.
  • Analytics Dashboard → Progress insights for teachers & parents.

Why It Matters

Math Ninja shows that learning doesn’t have to be boring.
By combining AI-powered hand-tracking, gamification, and math, we built an immersive, educational, and accessible experience.

Children can now:

  • Learn math interactively 🧮
  • Play using just a webcam 🎥 (compatible with most devices, so no VR/AR needed)
  • Compete globally via leaderboards 🌍

Welcome to the future of AI-powered learning through play.

Share this project:

Updates