SquatMoar

An app that uses computer vision to help you squat more.

Comment

Inspiration

Our entire group likes working out and since the squat is such an important exercise, we thought it would be useful to develop a squat form analyzer. Such an app could warn beginner lifters (us) if we were to make common form mistakes. While we do recognize that the perfect squat varies from individual to individual, there are some easy signs such as excessive knee travel, back bending, and not hitting depth that are generally considered bad form.

What it does

It takes a video of a person squatting, analyzes it by tracking key features about the person's posture, and provides feedback about their form.

How we built it

We built an Android application at the front end, which handles filming the video and selecting features to track. The video is then sent to the backend for processing using Google Firebase Storage and Real-Time-Database. The backend uses a Python implementation of OpenCV to analyze the video and then sends it back to the user's mobile device.

Challenges we ran into

The most challenging part was deciding on how to approach the computer vision part. At first, we attempted to use a pre-trained convolutional neural network (openPose), which proved not to be a good fit for our application as it doesn't provide great profile pose estimation. Then we tried OpenCV's "Oriented FAST and Rotated BRIEF" (ORB) feature detection as an attempt to solve the issue as an optical flow problem. In the end, we used a CSRT tracker (again OpenCV) to track user-determined features. To increase the reliability of features, we attached stickers to major body parts - this is an easy task to do in the gym.

Accomplishments that we're proud of

We're proud that we were able to finish such a complex project within such a short time. Not only did we manage to develop a successful feature tracking program, but we were also able to make it usable in an Android app.

What we learned

We learned that computer vision is VERY hard. Robots are far from taking over the world.

What's next for SquatMoar

Deploy in a gym. Also, analyze more exercises: deadlift, bench press.

Built With

Share this project:

Updates