Inspiration
Golf is a tough sport that takes a lot of time and resources to master. What if you can't afford a professional coach who can analyze your technique and give you helpful guidance? We want to make golfing more accessible, by softening the difficulty curve, making it easier for new players to improve at a rapid rate. By leveraging the power of AI, we are able to optimize the sports performance of our clients, by giving accurate analyses of weak points, finding multiple avenues of continuous improvement.
What it does
Using our own custom-built and trained AI model, our application accepts a video of a client demonstrating their swinging technique, identifying any weak points in their stance and posture which may be hindering optimal and more pleasant golfing results.
How we built it
Starting with an incomplete dataset from Kaggle, with labeled video footage as "bad" golfing posture and technique, we started our journey with not much to work with. Using the early hours of the event, as a team, we brainstormed the steps we would have to take to tackle this ambitious project. Using a linear work flowchart, we broke down our journey into 6 steps:
- Obtain the video data we need to determine what technique is bad and what technique is optimal.
- Parameterize the data to capture the posture of a player, into important "landmarks" using Google's MediaPipe, and using a data manipulation technique we call the "shuffle", we turned our sparse data set into real usable data, identifying numerous metrics for technique analysis.
- Once we had our metrics, we worked to load our data and started working on our neural network, optimizing our parameters, and getting our calculated technique scores.
- Using the scores from the model, we used Google's Gemini API to generate thorough feedback about the user's technique, and ways they could improve it.
- Needing somewhere to display the results, and allowing a user to upload a video to be analyzed, we had to develop a user friendly front-end experience. We opted to use Next.js to complete this step.
- Finally, we wrote our documentation, improved our code sanity, and compiled our hard work into Devpost.
Challenges we ran into
During this event, we ran into many different challenges, which greatly tested our mental fortitude, adaptability, and teamwork. This was a dynamic project with a myriad of moving parts.
- Our first major challenge was data collection. Finding datasets with usable video footage from around the internet was a difficult task, as we had to look in a few places to fully flesh out our dataset.
- Our second major challenge was parameterizing the dataset into relevant posing/posture data. Integrating Google's MediaPipe was unsurprisingly a challenge, and finding the proper "bins" or groups for our parameters took a lengthy group discussion.
- Our third major challenge was determining the right shuffling algorithm to generate a more granular data set, for better predictions. Not only was this technically challenging, it also required careful consideration of golfing biomechanics. Further, our videos had different lengths, so we had to find ways to normalize our data.
- Our fourth and last major challenge was getting the neural network to work.
Accomplishments that we're proud of
Turning what scarce data we had into something that could train and power an AI prediction model.
What we learned
We discovered that working with incomplete datasets can be a blessing in disguise. The limited data forced us to be creative, leading to us developing our “shuffle” technique to generate a rich, granular dataset from what scarce inputs we had.
What's next for Golfmate
Our vision is to help people learn how to play sports and develop new passions with the power of AI. We plan to add user accounts, allowing a user to save their scores, generating performance reports showing their growth over time. Users will be able to observe where their strong points are and where their weak points are, and how they can leverage and improve them respectively. Further, we want to increase the spread of our impact, and apply similar methods to other sports, such as basketball, soccer, tennis, and even gymnastics.
Log in or sign up for Devpost to join the conversation.