AI-Powered Algorithm Performance Analysis and Optimization

Code Optimizer is an intelligent tool that automatically generates alternative implementations of your algorithms, tests them rigorously, and provides detailed performance analysis. It leverages LLMs (GPT-4, Claude) to explore different algorithmic approaches and empirically validates their performance characteristics.

• AI-Powered Alternatives: Automatically generates 2-3 alternative implementations using different algorithms and data structures
• Performance Testing: Compares execution time across all solutions with microsecond precision
• Strategic Stress Testing: AI generates strategically-chosen test inputs (primes, highly composite numbers, edge cases) to expose algorithmic differences and worst-case scenarios
• Intelligent Test Selection: Each stress test includes a rationale explaining why it was chosen (e.g., "prime number to test worst case", "highly composite to test divisor-heavy case")
• Complexity Analysis: AI-powered Big O analysis with empirical validation
• Beautiful Reports: Generates detailed markdown and HTML reports with performance insights and test strategy explanations
• Correctness Validation: Ensures all alternatives pass your original test cases before performance comparison

flowchart TD
    A[Load Problem JSON] --> B{Validate<br/>Original Solution}
    B -->|Pass| C[Generate Alternatives<br/>via LLM]
    B -->|Fail| Z[Stop: Fix Solution]
    C --> D[Test All Solutions<br/>Basic Test Cases]
    D --> E[Stress Test<br/>Large Inputs]
    E --> F[AI Analysis<br/>Complexity & Trade-offs]
    F --> G[Generate Reports<br/>Markdown + HTML]

    style A fill:#e1f5ff,stroke:#01579b,stroke-width:2px,color:#000
    style C fill:#fff3e0,stroke:#e65100,stroke-width:2px,color:#000
    style F fill:#f3e5f5,stroke:#4a148c,stroke-width:2px,color:#000
    style G fill:#e8f5e9,stroke:#1b5e20,stroke-width:2px,color:#000
    style Z fill:#ffebee,stroke:#b71c1c,stroke-width:2px,color:#000

1. Load & Validate: Parse problem definition and verify your solution works correctly
2. Generate Alternatives: LLM creates 2 alternative implementations using different approaches
3. Basic Testing: Execute all solutions against test cases, measure timing
4. Strategic Stress Testing: AI generates smart test inputs (primes, composite numbers, edge cases) with rationale explanations (5-second timeout per test)
5. AI Analysis: Deep complexity analysis comparing time/space trade-offs with empirical stress test data
6. Report Generation: Create detailed markdown and HTML reports with test strategy explanations

• Python 3.13+
• uv (recommended) or pip
• OpenAI API key (for GPT-4o-mini)
• Anthropic API key (optional, for Claude models)

Option 1: Using uv (Recommended)

# Clone the repository
git clone https://github.com/yourusername/code-optimizer.git
cd code-optimizer

# Initialize uv and sync dependencies
uv sync

# Create .env file with your API keys
cat > .env << EOF
OPENAI_API_KEY=sk-your-key-here
ANTHROPIC_API_KEY=sk-ant-your-key-here
EOF

Option 2: Using pip

# Clone the repository
git clone https://github.com/yourusername/code-optimizer.git
cd code-optimizer

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
pip install anthropic openai python-dotenv markdown

# Create .env file with your API keys
cat > .env << EOF
OPENAI_API_KEY=sk-your-key-here
ANTHROPIC_API_KEY=sk-ant-your-key-here
EOF

python main.py problem_divisors.json

This will:

1. Validate the original solution
2. Generate 2 alternative implementations
3. Test all solutions
4. Run stress tests with large inputs
5. Generate AI-powered analysis
6. Create reports in reports/ directory

Create a JSON file describing your problem:

{
  "description": "Your problem description here...",
  "examples": [
    {
      "input": 15,
      "expected_output": [3, 5]
    },
    {
      "args": [10, 20],
      "expected_output": 30
    }
  ],
  "solution_python": "def your_function(param):\n    return result"
}

Supported formats:

• Single argument: {"input": value, "expected_output": result}
• Multiple arguments: {"args": [val1, val2], "expected_output": result}
• No arguments: {"expected_output": result}

Code Optimizer

Loading: problem_divisors.json
Loaded: divisors() with 7 tests

======================================================================
STEP 1: Validating Your Solution
======================================================================

Passed: 7/7
Avg time: 0.0053 ms

======================================================================
STEP 2: Generating Alternative Solutions
======================================================================

Model: gpt-4o-mini
Generated 2 alternatives

======================================================================
STEP 3: Testing Alternative Solutions
======================================================================

--- Testing Alternative 1 ---
Passed: 7/7
Avg time: 0.0043 ms

--- Testing Alternative 2 ---
Passed: 7/7
Avg time: 0.0061 ms

======================================================================
STEP 4: Performance Comparison
======================================================================

Solution             Passed     Avg Time (ms)   Status
------------------------------------------------------------
original             7/7        0.0053          Pass
alternative_1        7/7        0.0043          Pass
alternative_2        7/7        0.0061          Pass

Fastest Working Solution: alternative_1
Time: 0.0043 ms
Speedup: 1.23x faster than original

======================================================================
STEP 5: Stress Testing (Large Inputs)
======================================================================

Generated stress tests:
  • Size 100: 97 is a prime number to test worst-case scenario
  • Size 1000: 840 is highly composite with many divisors
  • Size 10000: 10007 is a large prime for stress testing

Solution             Size       Time (ms)      Status
------------------------------------------------------------
original             100        1.93           Pass
original             1000       1.43           Pass
original             10000      4.40           Pass
alternative_1        100        1.69           Pass
alternative_1        1000       1.07           Pass
alternative_1        10000      0.71           Pass (6.2x faster!)

Reports are saved in the reports/ directory (or custom directory via CLI):

• Markdown Report (divisors_20251029_140254.md): Complete analysis with tables
• HTML Report (divisors_20251029_140254.html): Styled, shareable report

Report Contents:

• Problem description
• All solution implementations
• Basic test results table
• Test Strategy: AI-generated rationale for each stress test input
• Stress test performance data with explanations
• AI-powered complexity analysis with empirical validation
• Final recommendations

View Example Report

Problem: Find all divisors of an integer (excluding 1 and itself)

def divisors(integer):
    return [i for i in range(2, integer) if integer % i == 0] or f'{integer} is prime'

def divisors(integer):
    result = []
    for i in range(2, int(integer**0.5) + 1):
        if integer % i == 0:
            result.append(i)
            if i != integer // i:
                result.append(integer // i)
    return sorted(result) if result else f'{integer} is prime'

The AI strategically chose these inputs to expose algorithmic differences:

• Small inputs (n=100): Both algorithms perform similarly
• Medium inputs (n=1000): O(√n) starts showing advantage with composite numbers
• Large inputs (n=10000): O(√n) dramatically outperforms O(n), especially with primes
• Strategic testing: AI-chosen inputs (primes vs composites) revealed performance characteristics that simple sequential testing would miss

Speedup: Alternative 1 is 6.2x faster on large inputs!

python main.py problem.json my_reports/

Edit main.py and change the model parameter:

# Use GPT-4
response = llm.generate_alternatives(
    description=data["description"],
    original_solution=data["solution_python"],
    function_name=function_name,
    model="gpt-4"  # Change here
)

# Use Claude
model="claude-3-5-sonnet-20241022"

In main.py:193, modify:

stress_sizes = [100, 1000, 10000]  # Customize these values

In main.py:196:

timeout_seconds = 5  # Adjust timeout (in seconds)

code-optimizer/
├── main.py              # Main workflow orchestrator
├── execute.py           # Code execution & testing engine
├── llm.py               # LLM integration (OpenAI/Anthropic)
├── report.py            # Report generation (MD/HTML)
├── utils.py             # JSON parsing & data conversion
├── problem_*.json       # Problem definitions
├── reports/             # Generated reports
└── test_*.py            # Development test files

Create a .env file:

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...  # Optional

Requires Python 3.13+ (specified in pyproject.toml)

The project will be increasingly reformulated to handle input in more general ways and include a front-end interface.

Planned Features:

• Web Interface: Browser-based UI for easier interaction
• More Input Formats: Support for file uploads, code snippets, direct paste
• Language Support: JavaScript, Java, C++, Go implementations
• Custom Test Generation: User-defined stress test patterns
• Benchmarking Suite: Compare against standard library implementations
• Visualization: Interactive charts for performance comparison
• Database Storage: Track optimization history over time
• API Endpoint: RESTful API for integration with other tools

Contributions are welcome! Feel free to open issues or submit pull requests.

MIT License - feel free to use this project for personal or commercial purposes.

• Powered by OpenAI GPT-4 and Anthropic Claude
• Built with Python 3.13+
• Inspired by competitive programming and algorithm optimization challenges

Made with care for developers who care about performance