GridEye

Heatmap indicating a QB option run play.
Heatmap indicating a quick out-route completion.
Heatmap showing a quick slant completion.
Color converted starting lineup to identify player position.
Heatmap indicating a run play.
Player identification system highlighting player positions.
User interface to upload plays and receive strategic analysis.
Grid lines identified with color conversions and Gaussian blur.
Front page of the website we developed to market our full product.
Diagram on our developed website showcasing the features and process of GridEye.
Mathematically perspective adjusted footage to account for inconsistent camera perspectives.
GridEye logo, designed to include aspects of gridiron, a football, and an eye, capturing the themes of knowledge embodied by AI.

Inspiration

Watching and analyzing football game film can take weeks, with each game requiring hours of review and every snap needing detailed breakdowns. Coordinators and analysts spend significant time manually identifying plays, tracking player movements, and recognizing opponent tendencies. A team’s time is their most valuable asset, and oftentimes too much time is spent pouring over hours of film instead of developing their players. We developed GridEye to automate this process, providing a faster, more comprehensive, and more efficient solution for teams at all levels.

What It Does

GridEye uses AI and computer vision to rapidly analyze game film. It classifies offensive and defensive plays, tracks patterns across multiple games, and generates custom strategic insights for each opponent. By mapping player movements with vector paths, GridEye visualizes how plays develop and provides AI-driven game plan recommendations to help teams adjust their strategy. Whether preparing for an upcoming opponent or refining team execution, GridEye offers custom data-backed insights to gain a competitive edge.

How We Built It

We developed GridEye using computer vision frameworks such as OpenCV and AI models like YOLO v10, ResNET, Tesseract to recognize formations, player movements, and field position. The system processes game footage, identifying player positions and utilizing context from the field to infer camera perspective. GridEye tracks player positions relative to the current frame using the YOLO v10 algorithm. Before making inferences from each frame, the data is preprocessed by converting to the LAB/HSV color spaces to enhance brightness, contrast, and saturation, as well as applying a Gaussian blur to denoise. Then, the data is given to the model, and player position data is extracted from the inferences.

However, as the camera pans and zooms, the positions of the players will shift around. To correct for camera perspective movement, scale and translational information from the field is required. This is inferred from the field lines, which are always visible to the tree camera in the box and always oriented vertically from the tree camera’s perspective. These lines are found by converting the current frame to grayscale, applying a Gaussian blur to denoise, and then using the Canny edge detection algorithm and Hough transforms to find straight line segments. These line segments are then merged by proximity, both translationally and angularly. Finally, lines greater than 45 degrees from the vertical are culled to ensure only the field lines remain visible. From the start frame, the central two field lines are tracked for translational and scaling conversions from the current frame into the start frame, allowing for the greatest flexibility in camera framing. These translational and scaling conversions are used to correct the current frame’s player position data into the start frame’s coordinates. From this, GridEye annotates the starting formation with player movements in the form of a heat map.

After collecting player movement data, GridEye employs GPT 4o for analysis of the play. The model was tuned first by system prompting and context reinforcement. The system prompt describes the format of the heat maps that the model will see and the format of the analysis that the model will complete. The context reinforcement ensures that the model will make accurate reads on the formations and plays that it sees, correcting for any initial mistakes. Finally, the GPT 4o model is enhanced with prompt engineering, ensuring the model stays on the topic at hand and the data is presented in an optimal way for the model to understand. The model’s analysis output includes play type, movement analysis, as well as a predicted outcome for each team.

To deliver a seamless user experience, we built GridEye’s frontend using Svelte for a fast and reactive UI. Users can upload their game film directly through the web interface, which then communicates with our Flask-based backend to process video frames and extract play data. Within seconds, the AI-generated play breakdown is displayed on the dashboard, allowing users to see tracked player movements, play classifications, and AI-driven strategic recommendations in an intuitive format. The Flask backend efficiently manages video processing and AI inference, ensuring low-latency, real-time analysis while keeping the system scalable for future enhancements.

Challenges We Ran Into

One of the biggest challenges we faced was camera perspective variability—since game film is often recorded from inconsistent angles with unpredictable zoom levels, accurately tracking player movement required stabilizing the field grid. Extracting reliable reference points through line detection and perspective correction was crucial to ensuring accurate positional data. Initially, we attempted background masking to isolate players, but as the camera perspective shifted, this approach became unreliable. To address this, we implemented YOLO v10 for player tracking, ensuring precise identification regardless of movement or perspective changes.

Fine-tuning our YOLO v10 model and reinforcement learning pipeline to classify plays correctly proved difficult, as football formations and strategies often feature subtle variations that can be challenging for AI to distinguish. Additionally, processing large video files in real-time while maintaining efficiency was another major hurdle. We had to optimize frame extraction, parallel processing, and API communication to ensure near-instantaneous analysis.

Despite these challenges, we successfully built a fully functional AI-powered sports analysis tool in just a few days. GridEye showcases the potential of computer vision and AI in transforming game strategy, helping teams gain an unmatched competitive edge through data-driven insights.

Accomplishments

We successfully developed and deployed a fully functional end-to-end AI-powered play analysis tool capable of transforming hours of game footage into meaningful strategic insights. Our advanced computer vision pipeline accurately stabilizes inconsistent camera perspectives, a crucial challenge in automated sports analysis. Additionally, we built an AI model capable of classifying plays with precision, recognizing offensive and defensive formations, and providing intelligent counter-strategy recommendations.

Beyond AI-driven insights, we created an intuitive real-time web interface using Svelte and Flask, allowing users to effortlessly upload footage and receive instant breakdowns. Our system efficiently processes large video files while maintaining low latency, ensuring usability even in high-performance coaching environments. We also successfully integrated GPT 4o into our pipeline, enhancing our AI’s ability to analyze motion patterns and contextualize play execution. Most importantly, we built a system that is currently usable, scalable, and adaptable, making it applicable for various levels of play, from professional leagues to high school teams.

Future Vision

We aim to expand GridEye to support multiple sports, bringing AI-powered strategy to a wider range of athletic competition. We also aim to improve GridEye’s adaptability for environments where camera angles and film quality vary, such as high school and amateur games. Enhancing our AI training dataset will improve recognition accuracy across different team styles and formations. Additionally, we plan to integrate real-time analysis for in-game adjustments, allowing coaches to make instant, data-driven decisions. In the future, with a presence in multiple sports and a highly adaptable, foolproof model, GridEye is poised to become a driving force in the evolution of sports competition, transforming how teams analyze and strategize.