A comprehensive Pong game implementation featuring both traditional gameplay and reinforcement learning capabilities. The system is built with a modular architecture that separates game logic, UI components, and AI training.

• Overview
• Architecture
• Installation
• Usage
• Game Modes
• Reinforcement Learning
• Testing
• File Structure
• Technical Details

This project implements a complete Pong game system with the following features:

• Traditional Pong Gameplay: Human vs Human and Human vs AI modes
• Reinforcement Learning: AI agents trained using PPO (Proximal Policy Optimization)
• Modular Architecture: Clean separation of game logic, UI, and AI components
• Multiple AI Types: Scripted AI, trained RL agents, and random agents
• Comprehensive Testing: Unit tests for all components
• Visualization Tools: Real-time game visualization with training metrics

The system follows a modular design with clear separation of concerns:

1. Game Logic (game_logic.py): Pure game mechanics without UI dependencies
2. Environment (pong_env.py): Gymnasium-compatible RL environment
3. UI Components: Multiple UI implementations for different use cases
4. Training System (train_rl_agent.py): Complete RL training pipeline
5. Testing Framework: Comprehensive unit tests

• Single Responsibility: Each module has a specific purpose
• Dependency Inversion: High-level modules don't depend on low-level details
• Testability: All components can be tested independently
• Extensibility: Easy to add new features and AI types

• Python 3.8+
• Pygame
• Stable Baselines3
• Gymnasium
• NumPy

# Install dependencies
pip install pygame stable-baselines3 gymnasium numpy --break-system-packages

# Clone or download the project
cd pong

# Start basic human vs human game
python3 main.py --mode human_vs_human

# Start human vs AI game
python3 main.py --mode human_vs_ai

# Start AI vs AI game
python3 main.py --mode ai_vs_ai

# Train a new RL agent
python3 train_rl_agent.py --mode train

# Test a trained agent
python3 train_rl_agent.py --mode test

# Visualize training with UI
python3 train_rl_agent.py --mode test-ui

• Classic Pong gameplay
• Two human players
• Controls: Z/S (left paddle), Arrow keys (right paddle)

• Human player vs AI opponent
• Human controls right paddle (arrow keys)
• AI controls left paddle (scripted or trained)

• Both paddles controlled by AI
• Can use different AI types for each paddle
• Useful for training and evaluation

• Shows random agent vs scripted AI
• Demonstrates RL environment without training
• Good for understanding the system

• Action Space: 3 discrete actions (0=nothing, 1=up, 2=down)
• Observation Space: 5 normalized values [ball_x, ball_y, paddle_y, ball_vel_x, ball_vel_y]
• Reward System:
  • +1.0 for successful paddle hits
  • +10.0 for winning a point
  • -10.0 for losing a point
  • -0.001 per timestep (encourages fast play)

• Algorithm: PPO (Proximal Policy Optimization)
• Network: MLP with [64, 64] architecture
• Learning Rate: 4e-4
• Batch Size: 64
• Steps per Update: 2048
• Epochs per Update: 10
• Gamma: 0.99 (discount factor)

• pong_agent_final.zip: Final trained model
• best_model/: Best model during training
• checkpoints/: Regular training checkpoints
• logs/: Training logs and metrics
• tensorboard_logs/: TensorBoard visualization data

# Run all unit tests
python3 -m pytest tests/

# Run specific test categories
python3 tests/test_game_logic.py
python3 tests/test_rl_integration.py
python3 tests/test_ui_components.py

• Game Logic: Physics, collision detection, scoring
• RL Environment: Observation space, reward calculation, termination conditions
• UI Components: Rendering, event handling, game modes
• Training Pipeline: Model loading, training callbacks, evaluation

pong/
├── game_logic.py          # Core game mechanics
├── pong_env.py            # RL environment
├── pong_ui.py             # Basic UI for human players
├── pong_rl_ui.py          # UI with RL agent support
├── demo_rl_ui.py          # Demo UI with random agent
├── train_rl_agent.py      # RL training pipeline
├── main.py                # Main entry point
├── tests/                 # Unit tests
│   ├── test_game_logic.py
│   ├── test_rl_integration.py
│   ├── test_ui_components.py
│   └── ...
├── best_model/            # Best trained models
├── checkpoints/           # Training checkpoints
├── logs/                  # Training logs
├── tensorboard_logs/      # TensorBoard data
├── README.md              # This file
└── README_RL.md           # Detailed RL documentation

from game_logic import PongGame

# Create game instance
game = PongGame(width=600, height=400)

# Update game state
game.update()

# Control paddles
game.set_paddle1_velocity(velocity)
game.set_paddle2_velocity(velocity)

# Get game state
state = game.get_game_state()

from pong_env import PongEnv

# Create environment
env = PongEnv(opponent_model_path="path/to/model.zip")

# Standard gym interface
obs, info = env.reset()
obs, reward, terminated, truncated, info = env.step(action)

from train_rl_agent import train_agent, test_agent

# Train agent
train_agent(opponent_model_path="path/to/opponent.zip")

# Test agent
test_agent(model_path="path/to/model.zip")

• Training Time: ~2-4 hours for 100M steps
• Memory Usage: ~2GB during training
• Game Performance: 60 FPS with Pygame
• Model Size: ~1-2MB per trained model

When contributing to this project:

1. Follow the existing code structure and naming conventions
2. Add unit tests for new features
3. Update documentation for any API changes
4. Use English for all code comments and documentation
5. Ensure all tests pass before submitting changes

This project is open source and available under the MIT License.