Gigabrain

Gigabrain uses a similar approach as DeepStack - A deep reinforcement learning algorithm to play information-asymmetry games.

Comment

Inspiration

The project was inspired by the Nash Equilibrium in Game Theory, in a state where no players can exploit the others or be exploited. During our research, we saw a paper regarding building an AI which randomly generates cards, uses Counterfactual Regrets recursive algorithm to solve for counterfactual values, and feed those to a 7-layer full-connected neural network as the final algorithm to solve for Texa's Hold'em games. In our games, the problem has a smaller problem space, indicating we do not need such large scale of data to train our model, which seems viable through days of coding, and we determined to do that approach.

What it does

Overview

The model is inspired by DeepStack model. It takes feature vectors as inputs, and outputs the number of possible hole card combinations.

Procedure

The agent first converts the current poker state into feature vectors, with one representing the normalized pot size, the other one representing the community cards. Then, it initializes actions and action probabilities, and feeds the features into the neural network.

How we built it

Our workflow began with thorough research to build a solid foundational understanding of the problem domain. Once we had a general conceptual framework, we studied relevant open-source implementations to extract architectural insights and best practices. We then proceeded to develop our own system from scratch, dividing responsibilities efficiently: one of us focused on implementing the core system logic, while the other handled model training and reinforcement learning components. After completing the initial prototype, we entered an optimization phase to refine our architecture and improve overall performance.

Challenges we ran into

During development, we encountered several technical hurdles, most notably memory leakage (which once exceeded 300GB of ram while running) and unpredictably long data generation times (which is normal). These issues significantly impeded our development progress. Nevertheless, through persistent debugging, extensive research from multiple sources, and assistance from peers, we managed to resolve these problems and establish a stable environment for training and evaluation.

Accomplishments that we're proud of

This project marked multiple milestones:

  • Our first hands-on experience building an AI system with practical utility.
  • First-time use of high-performance computing infrastructure, including supercomputers.
  • First foray into training and applying reinforcement learning models in a game-theoretic context. These accomplishments expanded both our technical skills and our understanding of applied AI systems.

What we learned

We developed a strong grasp of reinforcement learning pipelines, gained conceptual and practical insights into unsupervised learning, and deepened our understanding of game theory as it applies to decision-making in adversarial environments.

What's next for Gigabrain

Moving forward, we aim to iteratively refine and enhance our current system, codenamed Gigabrain, with the long-term goal of reaching a level of complexity and robustness comparable to **DeepStack. Once polished, we intend to open-source the project to share our work with the broader AI and research community.

Built With

Share this project:

Updates

posted an update —

Project Summary

Approach

Our workflow began with thorough research to build a solid foundational understanding of the problem domain. Once we had a general conceptual framework, we studied relevant open-source implementations to extract architectural insights and best practices. We then proceeded to develop our own system from scratch, dividing responsibilities efficiently: one of us focused on implementing the core system logic, while the other handled model training and reinforcement learning components. After completing the initial prototype, we entered an optimization phase to refine our architecture and improve overall performance.

Challenges

During development, we encountered several technical hurdles, most notably memory leakage and unpredictably long data generation times. These issues significantly impeded our development progress. Nevertheless, through persistent debugging, extensive research from multiple sources, and assistance from peers, we managed to resolve these problems and establish a stable environment for training and evaluation.

Accomplishments

This project marked multiple milestones:

  • Our first hands-on experience building an AI system with practical utility.
  • First-time use of high-performance computing infrastructure, including supercomputers.
  • First foray into training and applying reinforcement learning models in a game-theoretic context.

These accomplishments expanded both our technical skills and our understanding of applied AI systems.

Lessons Learned

We developed a strong grasp of reinforcement learning pipelines, gained conceptual and practical insights into unsupervised learning, and deepened our understanding of game theory as it applies to decision-making in adversarial environments.

Next Steps

Moving forward, we aim to iteratively refine and enhance our current system, codenamed Gigabrain, with the long-term goal of reaching a level of complexity and robustness comparable to DeepStack. Once polished, we intend to open-source the project to share our work with the broader AI and research community.

Log in or sign up for Devpost to join the conversation.