CharAIdes

Starting Screen
Game Play
Game Play
Game Over
Leaderboard

Inspiration

We wanted to create a game that combines physical activity with AI, making classic charades more interactive and engaging. Traditional charades require a human judge to determine if a performance matches the word. We thought: what if AI could take that role in real-time, making the game playable anywhere, even solo or online?

What it does

CharAIdes lets a user act out a word in front of a webcam while an AI guesses the action. The AI evaluates the performance using image analysis and provides instant feedback. Each round has a time limit, and the user progresses through multiple rounds, earning points when the AI correctly identifies the action.

Real-time webcam feed capture
AI guesses actions based on visual input
Scorekeeping over multiple rounds
Timer for each round

How we built it

Frontend: React with components for the start screen, game screen, webcam feed, and end screen. State management handles rounds, score, and AI guesses.
Backend: FastAPI server handling image uploads, integrating with Gemini API for generative AI image analysis.
AI Integration: Gemini API interprets webcam images and returns a predicted action. We process the response to check against a curated word list.

Challenges we ran into

Timer vs animation desync: naive interval updates or CSS width transitions produced drift. Fix: rAF + transform scaleX driven by real ms.
Debouncing AI API calls: To avoid overwhelming Gemini API, we limited requests to once every few seconds.
Managing re-renders: Updating words or AI guesses initially caused the webcam to reload; solved using useCallback and separating state to prevent unnecessary remounts.
AI guess strictness: Gemini was too lenient initially, sometimes predicting words outside our curated list. We refined prompts to balance strictness and flexibility.

Accomplishments that we're proud of

Seamless integration of generative AI to interpret human gestures in real-time.
Smooth, responsive game experience without noticeable flickering or camera reloads.
Dynamic word and round management, supporting multiple rounds with scoring.

What we learned

Integrating real-time webcam input with AI models requires careful handling of state and rendering to avoid UX glitches.
Generative AI can be used effectively beyond text—here, it successfully interprets human motion.
Game design benefits from decoupling frontend state from backend AI calls to maintain smooth interaction.
Small optimizations, like API call throttling and canvas pre-sizing, significantly improve responsiveness.

What's next for AI Charades

Expand the word list with more nuanced gestures and actions.
Multiplayer mode where users compete or cooperate while AI scores performances.
Experiment with multimodal prompts, combining audio cues with visual gestures for richer gameplay.