core-control

CV View of the Target Footpads
Pose Estimation
In App Video/Photo Capture
Another Screen

Inspiration

rock climbing - dassit

What it does

This game trains your hanging core control for climbers. Hang on a pull-up bar and carefully kick targets on a wall in front of you. Our software determines the quality of your kick using OpenPose and OpenCV.

How We Built It

We used OpenPose's software to track many body points from images or video. From a Flask server with a GPU, we run this software and listen for requests. A user can stream images to our server through our iOS app and our server returns feedback to the user.

Challenges We Ran Into

CV is hard, and we weren't able to complete our original idea of giving feedback to the user regarding the form and style of their kick. We would have done this through 3d-reconstruction of the paths of the 25 body joints that are tracked by OpenPose.

We also were very new to iOS development and had never used Swift before...trying to build an app that streamed live video to a web server was rough.

Accomplishments That We're Proud Of

We successfully determined whether or not a person kicks a target through computer vision analysis alone. We also learned a lot about OpenPose, CV, Swift, and iOS development.

What We Learned

We learned more about server communications, Swift, iOS app development, and computer vision software.

What's next for core-control

This idea could easily generalize to other kinds of workout-motions, such as push-ups and handstands, where CV could help give instant feedback to the user about proper form and technique.