ValorSync

Build Your Ultimate VCT Squad with Precision and Synergy

Comment

About the Project

Inspiration

The inspiration behind this project stems from the challenge of optimizing team composition in competitive games like Valorant, where player roles, agents, and strategies play a critical part in success. With Generative AI advancing rapidly, we saw an opportunity to leverage LLMs to make data-driven suggestions for optimal team formation, ensuring better coordination and strategy alignment in competitive settings.

ValorSync: Valorant Team Composition Suggestion Chatbot

Project Overview

In this project, we built a Valorant team suggestion chatbot designed to assist in forming the most effective team compositions for competitive gameplay. The chatbot analyzes player performance data from past matches and suggests optimal team compositions by factoring in key performance indicators and player roles. The core of the project integrates Large Language Models (LLMs), specifically fine-tuned with agent data from popular sources like Fandom, to provide reasoning for agent selections in line with user-provided requirements.

Data Sources

  • Game Data: Match data in .csv format stored in an S3 bucket. This data includes game-level statistics such as timestamps, player actions (kills, damage), agent roles, and team performance. It also includes:

    • Event name
    • Round numbers
    • Players' actions (damage, kills)
    • Teams involved (attacking, defending)
    • The result of each round (winning team)

  • Mapping Data: External files that map player, agent, and team IDs from the game data to standard formats.

Methodology

The workflow of the project can be broken down into two key phases:

Phase 1: Data Preprocessing and Player Performance Analysis

  1. Data Preprocessing: Game-wise data from the S3 bucket is retrieved using APIs and processed to extract meaningful statistics. This includes player performance metrics such as total kills, damage contribution, kill success rate, and headshot percentage.

  2. Player Performance Metrics: Two sets of 13-dimensional performance vectors are computed for each player based on their roles in attacking and defending rounds. These vectors encapsulate various player performance metrics including:

    • Damage contribution percentage
    • Kill success rate
    • Initial kill success percentage
    • Round-based total kills and deaths
    • Other contextual statistics such as roles and game modes

Phase 2: Player-wise Performance Scoring and Role Assignment

  1. Player Scoring: A weighted scoring system is applied based on player performance vectors. The scoring function is tuned based on user requirements and role-specific attributes. This allows the system to rank players based on their contributions in both offensive and defensive scenarios.

Example scoring formula: player_score = attacking × (w₁ × damage_contribution%) + (w₂ × kill_success%) + ... + defensive × (w₃ × defensive_metrics)

Each weight (w₁, w₂, w₃, ...) represents a user-defined or scenario-based importance factor.

  1. Role Assignment: The top-performing players are sorted role-wise, and the chatbot suggests agents that fit their strengths. Player-agent pairings are optimized for team balance and cohesiveness.

LLM-based Query Handling

  • The user inputs requirements (e.g., team balance, specific agent preferences, or role counts).
  • The chatbot generates role-wise suggestions using LLM models (LLaMA 3.2 on AWS Bedrock), which have been fine-tuned on Fandom data. The model reasons out the best agent combinations for each player, ensuring both cohesiveness and strategic alignment with user preferences.
  • The chatbot outputs the suggested agents and explains the reasoning behind each decision, providing additional insights for optimal team synergy.

AWS Usage

LLM Integration

  • Deployed AWS Bedrock with a fine-tuned LLaMA 3.2 model, which analyzes performance data and recommends team compositions by weighing the best player combinations for specific roles (e.g., Dualist, Initiator, etc.).

Coupling Different Pipeline Components using AWS

  • To facilitate real-time synchronization between data fetching and processing, we utilize AWS Simple Queue Service (SQS) as a message queue. Given that data fetching is significantly faster than processing, the data-fetching unit asynchronously pushes fetched data to the SQS queue. Meanwhile, the processing unit polls the queue and processes data, outputting results as small CSV files to an Amazon S3 bucket. The generation pipeline subsequently retrieves these processed files from S3, ensuring an efficient and decoupled data flow across the pipeline.

Handling Different Prompts and Overcoming Challenges

Handling League-Specific Data

  • The data processing pipeline converts unstructured game data into structured, player-wise performance scores in CSV format. A separate CSV is generated for each league (e.g., Game Changers, VCT International). When a user query is received, the specified league is identified and the corresponding CSV is loaded via a custom function call.

Handling Cross-Region Queries and Mixed-Gender Team Submissions

  • Using an LLM JSON engine, we identify user-specified filters, such as the desired region, from the query input. Extracted filter values are then applied to the CSV to isolate relevant data efficiently.

Handling Rising Star Team Submission

  • We use a custom scoring model that assigns a deterministic score for each role based on weighted parameters. To reduce bias toward only highly scored players, we predict each player’s score movement through statistical analysis. If a player’s current score is low but shows an upward trend over time, we use historical data to project potential growth. Currently, due to limited data, we calculate this using the mean of the first-order difference, but with more data, we plan to implement more advanced time-series models for prediction.

Determining Best Maps for a Team and IGL

  • A fine-tuned Llama model, trained on blog and fandom data, is used to analyze team synergy and identify maps with the highest win rates for each player, maximizing team performance. The ideal IGL is selected by calculating an aggregated weighted score across relevant metrics, ensuring strategic alignment with the team’s strengths.

Handling Player Replacements

  • Top players are selected for each role, agents are assigned, and if any players are unavailable, the system automatically selects the next best players and reassigns agents to maximize team synergy.

Reasoning Behind Role/Agent Assignment and Statistics Driven Pipeline

  • Our process encompasses multiple checkpoints, transitioning from role assignment to agent allocation and strategic discussions. We evaluate player performance across 13 parameters, mathematically modeling each player into a score by assigning weights based on user requirements and role types. This modeling clarifies the effectiveness of role counts and the rationale behind specific weight assignments. Ultimately, a fine-tuned Llama Model determines agent selection based on their respective abilities.

Challenges we ran into

  • Data Processing: Efficiently managing large volumes of in-game event data for quick analysis was a challenge, especially with real-time player performance monitoring.
  • LLM Fine-Tuning: Tuning the LLaMA 3.2 model to understand game-specific strategies and align recommendations with competitive play styles was complex.
  • Metrics Weighting: Properly balancing different player performance metrics to ensure optimal team recommendations required significant experimentation.

Accomplishments that we're proud of

  • Successfully integrating an LLM to generate real-time team composition suggestions using player performance data.
  • Efficiently processing and analyzing large game datasets to create player performance vectors that accurately reflect in-game actions and roles.
  • Fine-tuning an LLM for a competitive gaming use case, where optimal team composition can directly influence game outcomes.

What we learned

We learned how to:

  • Analyze and process large-scale game data to generate useful insights on player performance.
  • Integrate AWS services such as S3, Lambda, and Bedrock to build scalable, AI-driven solutions.
  • Fine-tune large language models for highly specific, data-driven recommendations in a dynamic and fast-paced environment like competitive gaming.

What's Next for ValorSync

  • Expansion to Other Games: Extend the platform to support more competitive games beyond Valorant by generalizing the performance metrics.
  • Player Behavior Prediction: Incorporate predictive analytics to suggest not just team compositions but potential player behavior patterns and strategies based on historical data. This could leverage time series models for prediction to enhance accuracy in forecasting player performance trends over time.
  • Mobile App: Build a mobile version of the platform, making it easier for teams to get composition suggestions and performance insights on the go.

Built With

Share this project:

Updates