-
-
-
Empty Hit!
-
Miss!
-
Hit!
What it does
Welcome to ZotTaiko!
Here’s the beginning of your adventure in rhythm games. Are you hungry for ants? Well we have good news for you! In this game, your hands become the snout of the anteater, in this incredibly fun and interactive rhythmic drum game!
Feel the joy of gobbling up ants to earn points with the music! Make a zot sign whenever the “ant” comes close, and you’d be in for a delicious treat.
Inspiration
The inspiration for ZotTaiko comes from the game Taiko no Tatsujin. With a strong love for games, especially creative formats that appear in Eastern arcades, we wanted to bring this game to UCI students. We have added one more unique element to our school’s gaming culture.
Inspired by Xbox’s Kinect Cursor (hand recognition), we wanted to merge the traditional Taiko game with an excitement in our unique gameplay. We picked up on the “zot”, our school’s specialty. Tapping rhythm became a creative idea of eating ants, through making the zot sign with your hands.
We hope players would find themselves immersed in the joy of the hunt, and of zotting perfectly to the musical rhythm. Let ZotTaiko take you on a journey, where every beat makes you more proud of our culture!
Accomplishments that we're proud of
Community.
During our data collection journey, we spoke with eager, new, young, enthusiastic people who are about to start their journey at UC Irvine, as well as those who have graduated and accumulated their memories and knowledge here in this beautiful community. What ties all of us together is the hand sign — the synchronous “zot” that really shines through as UCI's identity.
We met a family whose proud daughter starts here for biomedical engineering in the Fall. While collecting our zot-sign data, her father asked us what courses she should be taking, and about the housing options—how the dorm life is, how easy meeting new people is, etc. We tried our best to show that our friendliness and enthusiasm in this project made her more sure of coming to UCI. Sometimes, remembering that we belong to this community by just chilling and playing such an amusing game (a popular arcade game) makes us so sure of our missions as programmers, who use code to create joy.
A micro version of this unity and tight-knit feeling shows in our small group of 4. As beginners, we really tried to take advantage of this opportunity to learn something together as a team, through extensions of our undergrad lower-division Python curriculum. Our common ground is the language, but with different fortes that shine through — ML, design, game dev, asset research & integration. Welcoming someone not from UCI, working in grass fields, showing them the culture… We have made inclusivity our priority.
And we want to include you too! That means, all of you are welcome to download and play this game on your own computers.
We are proud of our teamwork and technical accomplishments; we got over 200 photos in our dataset, built and trained the random forest model successfully with 82% accuracy, found the integral Mediapipe library, interpreted and parsed the OSU beat maps, synced the tempo and note generation, integrated the model with Pygame with multithreading, following OOP paradigms.
So ZotTaiko was born — a game that relied on all of you, every single unique hand, to make this booth worth wondering, worth relaxing our minds and playing together, to start conversations.
How we built it
Gesture Pack.
- Over 200 data points (photos) were collected from hackathon participants to people in Aldrich Park. The mediapipe library was used to segment the knuckle data. A 3D-spaced tree with each knuckle a coordinate, and fingers edges between the coordinates.
- We trained a Random Forest network, with accuracy 82%, stored with joblib into a pkl file.
- We wrote an identifier that partitions the screen into two parts. Hands were identified in each part, in real time.
Taiko Gameplay.
- The main game loop checks the shared data structure for recognized gestures.
- Based on the gestures, corresponding actions are performed in the game.
- The game logic and display updates are handled in the main thread to ensure smooth operation.
- Music playing is integrated.
Assets.
- Procreate used for assets, ants, background, drum and all visual features are custom designed & UCI themed.
Challenges we ran into + What we learned
We tried a CNN approach but it did not work as well (58% accuracy, due to backgrounds). Instead, we used the preexisting knuckle-extraction tool mediapipe. It eliminated the need for a uniform background and allowed for real time detection.
Our understanding of multithreading was learned on the fly. We realized the problem of the impossibility of running two open windows at once, so we learned to use multithreading from scratch and gradually built up a real-time update system. Implemented custom classes to store the finger data, accessed in both threads.
What's next for Zottaiko
We hope we can add more features, more songs, and a two player system! Playing this game with your friend would help us achieve our purpose of using this game to connect our students better. We would also love more data on the “zot” hand signs, so the classification becomes even more accurate, more useful, and more inspiring for future hackers.
Log in or sign up for Devpost to join the conversation.