kinemo

A party game where your body is the controller!

Comment

Inspiration

We started off making a boxing motion tracker with the intention of creating real-time AI feedback. It slowly evolved into feeling more like a game, so we decided to work towards that instead.

Eventually, we realized we could turn this into something bigger. Then the idea for a whole console-like experience with multiple games was born.

What it does

Our project is a motion-controlled gaming platform that only needs a webcam. Two players can compete in three different active games. The main menu is controlled with hand gestures, letting players choose between Boxing, Flying Fruits, or Reaction Time.

In Boxing, players can punch, kick, or weave. Each player starts with 12 hearts. A punch takes away 1 heart, and a kick takes away 3. If the opponent weaves within 0.75 seconds, they avoid damage, and the attacker is stunned for 1.25 seconds. If the weave happens between 0.75 and 1.25 seconds, no one takes damage or gets stunned. To stop spamming, each attack has a 0.5-second cooldown.

In Flying Fruits, players have 30 seconds to slice as many fruits as possible. Moving your hand quickly through a fruit earns +1 point, while slicing a bomb loses 5 points. The angle of the slice is also shown in the animation.

In Reaction Time, players race to raise both hands after a green light appears. If a player moves too early or during the red light, they “cheat,” and the other player gets a point.

How we built it

Our gaming platform was developed in VS Code with Python, OpenCV, MediaPipe, Pygame, and NumPy. Python is used on the backend to handle application management. OpenCV is used to process the video webcams in real time, which is then piped into Google's MediaPipe framework for the pose estimation and body tracking for all of the games. Afterwards, we used NumPy for position analysis and velocity/acceleration calculations. Finally, Pygame helped us with the audio and visuals.

Challenges we ran into

  • Picking up different movements without interference from background noise.
  • Problems with our original hand tracking system until we found a great library for hand tracking. Since it was more of a general-purpose module that controlled the computer's cursor, we had to integrate it to work within our window and user interface.

Accomplishments that we're proud of

After finishing the app, we took it around to the other hackers so they could try the games and provide us with feedback. I can confidently say that our greatest accomplishment of the weekend was seeing people's eyes light up as they faced off against their friends in our game. We had an absolute blast touring Kinemo around, and so did others.

Our makeshift team is incredibly proud of what we accomplished in under 48 hours, and we look forward to becoming great friends and participating in future hackathons together.

What's next for Kinemo

  • More games! Some ideas we had were a game like Just Dance or basketball.
  • Adding hand control priority. Basically, we want to give one player the "magic touch" for controlling the user interface. Revamping the UI.

Built With

Share this project:

Updates