Inspiration
Our inspiration stemmed from our shared passion as software engineering students for combining technology with physical activity. We noticed that while tech-related hobbies often keep us seated for long periods, we yearn for fun activities that get us moving and physically active. That thought led us to create Duo Move, an interactive game that not only encourages movement but also utilizes cutting-edge AI technology to provide a fun, immersive experience.
What it does
Our project uses a Raspberry Pi with an integrated RPI5-Hailo8L AI chip to capture real-time video from a connected camera. The AI detects players' joints and body parts, which are then used as inputs to control the game.
How we built it
We set up a client-server architecture where the Raspberry Pi processes video data using AI and sends real-time motion information to the game client via a custom WebSocket communication. The Raspberry Pi acts as a local network as well. The server side runs a Flask-based backend that captures the video feed and performs pose estimation, while the client side, built with JavaScript, renders the game and updates based on the received data. We integrated frameworks like gstreamer for streaming and used Flask to manage the data flow.
Challenges we ran into
One of the key challenges we faced was getting the live video feed from the Raspberry Pi to render properly in a web browser. The second challenge involved mapping the coordinate system of the real-world camera feed to match the game's coordinate system, ensuring that player movements were accurately represented in the gameplay. It was also challenging to find documentation on the newer or niche technologies we worked with.
Accomplishments that we're proud of
We're immensely proud of being able to incorporate AI into fun solutions to problems of not getting active while playing with a friend. We're particularly happy about the seamless integration of the joint detections and being able to display it on screen!
What we learned
We learned various things in regards to the tech stack. This project was the first time some of the members used a Raspberry Pi for a project and some frameworks, like gstreamer to make an AI pipeline and Flask, was learned on the fly to ensure us delivering on this software.
What's next for Duo Move
Adding more game modes, adding player profiles and expanding the player base.
Log in or sign up for Devpost to join the conversation.