ScoutR

Inspiration

Up to now, modern sports analytics has allowed teams to quantify optimal shooting zones, when and when not to utilize a coach's challenge, the optimal rest intervals for a star player, and so on. But what if team chemistry could be quantified? Team dynamics between players plays a pivotal role in their success and by preemptively identifying great team chemistry as well as subpar team chemistry, teams will have an advantage on and off the court. Teams will be able to take informed action towards optimal team chemistry.

We were especially inspired by Wednesday's talk from Warriors VP of Technology, Daniel Brusilovsky, who mentioned that scouts when looking at prospects particularly key in on what they do between plays - how they react to a benching, how they interact with their teammates, etc. Warriors staff thus have to be there in person, not just watching tape, to be able to best evaluate a potential new player.

ScoutR helps bring EQ to the sports IQ category by helping measure the previously immeasurable, leveraging AI/ML. Highlight great team chemistry, shore up lackluster team dynamics.

What it does

Detects a given player's nonverbal cues to determine a strong or weak team chemistry score between other players on his/her team.

How we built it

With the limited amount of time, we knew we couldn't train a body language ML model from scratch - sourcing the "in-between moments" of game footage would also be a challenge so we keyed in on specific, readily available context rich instances to test our implementation such as the 2018 Warriors - Cavs Game 1 Final that had a crucial Cavs miscue that forced the game to overtime (https://youtu.be/ST3B4ZzOaUM?si=_iXvRlW4Uhd_hwhb&t=109).

We spliced different segments of the full length video to still frames using OpenCV. We then cycled through different LLMs to get a sampling of that model's ability (or inability) to properly read the context of a situation, playing close attention to its ability to read specific body language cues such as frustration, confusion, anger, joy/happiness, cohesion, disappointment and so on.

For the end-user that "body language score" for a given input context would output a further distilled team chemistry score. A cumulative score card would be generated (imagine a leaderboard) to further break down the teammates that have the highest team chemistry (think Klay - Steph Splash Brothers (tm)), and the lowest team chemistry (think Draymond - Poole).

Challenges we ran into

Ironically, our own team dynamics could have been better. We tried to add to our two man team, walking through two potential teammates, at separate instances, what the idea was and talking through how they could possibly help - front end work as a quick example. Unfortunately, neither team member we explored bringing on stayed through the hackathon to contribute and help with our laundry list of work items.

Also as mentioned, because this doesn't exist in the wild / hasn't really been thought of to any great extent, we spent a considerable amount of time parsing through good "in-between plays" game video footage to sample / use.

Accomplishments that we're proud of

Modern CV (Computer Vision) is amazing. With just a base level of knowledge and limited time, we were able to get going a rough prototype of a snapshot tool giving the context behind a given situation.