The first open-source LLM benchmark for Performance Marketing.

Developed by hawky.ai — evaluating how well LLMs reason, plan, and act in real-world performance marketing scenarios across Meta Ads and Google Ads.

PM-AGI measures LLM performance across four critical dimensions of performance marketing:

git clone https://github.com/Hawky-ai/pm-AGI
cd pm-agi-benchmark
pip install -r requirements.txt

# Evaluate using OpenAI
python evaluate.py --model gpt-4o --provider openai --api-key YOUR_KEY

# Evaluate using Anthropic
python evaluate.py --model claude-opus-4-6 --provider anthropic --api-key YOUR_KEY

# Evaluate using any OpenAI-compatible endpoint
python evaluate.py --model llama-3.3-70b --provider openai-compatible --base-url YOUR_URL --api-key YOUR_KEY

# Run only a specific category
python evaluate.py --model gpt-4o --provider openai --category meta_ads

Each question in benchmark/dataset.json follows this schema:

{
  "id": "meta_001",
  "category": "meta_ads",
  "subcategory": "bidding_strategy",
  "difficulty": "medium",
  "type": "mcq",
  "question": "...",
  "options": {"A": "...", "B": "...", "C": "...", "D": "..."},
  "answer": "B",
  "explanation": "...",
  "tags": ["roas", "value_optimization"]
}

Question Types:

• mcq — Multiple choice, single correct answer
• open_ended — Free-form answer evaluated against key criteria
• action_based — Scenario with expected actions/reasoning

Overall Score = weighted average across all categories.

1. Run evaluation: python evaluate.py --model YOUR_MODEL --provider YOUR_PROVIDER
2. A result file is saved to results/YOUR_MODEL_results.json
3. Open a PR to this repo adding your result file, OR submit via the HF Space form.

Visit the live leaderboard: huggingface.co/spaces/hawky-ai/pm-agi-leaderboard

We welcome contributions to the benchmark dataset! See CONTRIBUTING.md.

• Adding new questions: follow the schema above and open a PR to benchmark/dataset.json
• New categories (TikTok Ads, LinkedIn Ads coming soon)
• Translations

@misc{pmagi2025,
  title={PM-AGI: A Performance Marketing Benchmark for Large Language Models},
  author={hawky.ai},
  year={2025},
  url={https://github.com/Hawky-ai/pm-AGI}
}

MIT License — see LICENSE.

Built with ❤️ by the hawky.ai team.