Jumping Jarvis!

Inspiration

Our inspiration came after talking with each other about how we were so out of shape because of quarantine.

What it does

Our app’s main functionality revolves around the PoseNet pose tracking model. It allows us to count the number of reps that a person does for different exercises (we implemented it for jumping jacks and squats). It also keeps track of each workout and automatically calculates calorie burn using the height and weight of the user. There is also functionality to track how many days in a row a user has done a workout.

How we built it

Our app uses a Python Flask backend, with API endpoints to handle user and data storage. The frontend uses the React Javascript framework. We used also used JWT token-based authentication. We used the Tensorflow Posenet model to do pose estimation.

Challenges we ran into

The main challenge we ran into was getting the rep counter to accurately count reps. We originally thought that the PoseNet library would handle the pose classification for us, but as it turns out we had to implement algorithms of our own to keep track of the progression states of each transformation. We had to implement multiple debugging features to see where the problems were and ultimately had to change our initial method to track the reps. We also figured out that lighting played a huge role in the accuracy of the pose tracking.

Accomplishments we are proud of

Our main feeling of accomplishment came after we refined our pose classification algorithms and were finally able to have an accurate count of the number of exercise reps the user did.

What we learned

Our team was fairly unfamiliar with React, so we had a steep learning curve in that regard. We also learned more about how to interface with libraries like Tensorflow.

What’s next for Jumping Jarvis

Features that we would implement would be the ability to track more types of exercises, a leaderboard, and more game-ified features like tokens to buy insurance for your streak.