JustChi

"Just Dance. Tai Chi. VR. Elderly."

Inspiration

The idea behind JustChi stemmed from a desire to improve the physical and mental well-being of seniors through accessible and engaging technology. Since everyone on the team has a relative who would benefit from the gentle exercise that is Tai Chi, we thought it would be a wonderful idea to prototype a futuristic VR game targeting the growing senior population. By combining the ancient practice of Tai Chi, known for its health benefits such as improved balance, flexibility, and stress reduction, with modern Virtual Reality (VR) and Computer Vision technology, JustChi aims to provide a unique and immersive experience that is both beneficial and enjoyable for its users.

What it does

Think "Just Dance" but instead of high-intensity dance, our users are immersed in relaxing and peaceful VR environments where they can engage in the ancient Chinese art of Tai Chi. Once set up on the Meta Quest 2, our users can select from a variety of Tai Chi styles and soothing background environments. They are then sent to these environments where they can follow along with our prerecorded Tai Chi "master" through a game-like experience. Along the way, we provide supportive feedback and calculate the accuracy of their moves, using it to display their Tai Chi score in real-time. Through Just Chi, anyone can learn Tai Chi and reap a myriad of health benefits while being educated on ancient Chinese culture.

How we built it

We used an open-source computer vision framework called MediaPipe to determine our user's pose by tracking keypoints (and their coordinates) displayed along the user's body, storing and transmitting these coordinates through a Flask server. We then ran these coordinates through our pose comparison Python algorithm, which compares the coordinates of the user's keypoints with the keypoints of our Tai Chi "master" using linear algebra techniques. Then, we used Figma, HTML, CSS, JavaScript, and a VR development framework called A-Frame to design our VR UI/UX and environment that is displayed to the user through the Meta Quest headset.

Challenges we ran into

Initially, it was difficult to find a way to track keypoints while interfacing with Unity (a game engine we originally thought about using). However, we eventually found A-Frame and never looked back. It was incredibly challenging to write the pose comparison algorithm because we had trouble syncing the Tai Chi "master" recording with the user in order to compare their keypoint coordinates. Even once we implemented several different versions of the algorithm, it wasn't especially accurate at times. It was also difficult for us to design the UI/UX and environment for the Meta Quest. It was all of our first time working with VR so it was quite the learning curve for us.

Accomplishments that we're proud of

We're proud of our functional pose comparison algorithm which we finally got to work after almost an entire day of programming, testing, and debugging. We're also very proud that we were able to have a working, enjoyable, and impactful VR game done in less than 2 days. Additionally, we are proud that we were able to find A-Frame and learn a VR web development framework that was previously completely unknown to us. Finally, we are so proud that we were able to spend the weekend with so many passionate, talented, and friendly people.

What we learned

We learned a lot about VR development frameworks such as A-Frame, the different kinds of computer vision frameworks such as MediaPipe, the amazing art of Tai Chi, and vector similarity algorithms. Particularly, we learned that there are other VR development platforms out there (besides Unity), how to use MediaPipe, different forms of Tai Chi styles (Chen, Yang, Hao, Wu, Chen, etc), and how we can vectorize information and use linear algebra to determine similarity. Most importantly, we learned more about how prevalent depression and health issues are among the elderly, motivating us to create a fitness program that combats these issues: JustChi.

What's next for JustChi

Our roadmap is filled with exciting developments, from introducing multiplayer capabilities to integrating heart rate monitoring for health and fitness insights. We're committed to expanding JustChi's content, offering new Tai Chi styles, and creating more immersive and personalized environments. The potential for JustChi to make a significant impact is boundless, and we're thrilled to continue this journey of innovation and wellness.