AI-powered code review service with RAG (Retrieval-Augmented Generation) for contextual analysis. Integrates with GitHub webhooks to automatically review pull requests using Google's Gemini 2.5 Pro model.

Why Observability Matters: As AI transitions from standalone tools to embedded infrastructure, operational concerns become critical:

• Inference at scale rivals training in complexity and cost
• Value is shifting to deployment, evaluation, and integration
• Latency, reliability, and cost control outweigh marginal benchmark gains
• Sustainable, long-term scalable systems require visibility

This service demonstrates production-grade AI with built-in Datadog observability.

• Automated PR Reviews: Triggered via GitHub webhooks on pull request events
• RAG-Powered Context: Uses FAISS vector search to provide relevant codebase context
• Security Scanning: Detects secrets, PII, and potential prompt injection
• Datadog Observability: Comprehensive metrics for LLM costs, latency, and RAG performance
• Multi-file Support: Reviews Python, JavaScript, TypeScript, Go, Java, and more

                                    ┌─────────────┐
                                    │   GitHub    │
                                    │  Webhook    │
                                    └──────┬──────┘
                                           │
                                           ▼
┌──────────────────────────────────────────────────────────────────┐
│                         Cloud Run                                 │
│  ┌─────────┐    ┌──────────┐    ┌─────────┐    ┌─────────────┐   │
│  │ FastAPI │───▶│ Security │───▶│   RAG   │───▶│ Gemini LLM  │   │
│  │         │    │   Scan   │    │ Context │    │ (Vertex AI) │   │
│  └─────────┘    └──────────┘    └─────────┘    └─────────────┘   │
│       │              │               │               │            │
│       └──────────────┴───────────────┴───────────────┘            │
│                              │                                    │
└──────────────────────────────┼────────────────────────────────────┘
                               │
              ┌────────────────┼────────────────┐
              ▼                ▼                ▼
        ┌──────────┐    ┌──────────┐    ┌──────────┐
        │ Datadog  │    │  GitHub  │    │  FAISS   │
        │ Metrics  │    │ PR API   │    │  Index   │
        └──────────┘    └──────────┘    └──────────┘

• Python 3.11+
• GCP Project with Vertex AI enabled
• GitHub Personal Access Token
• Datadog API keys

1. Clone and setup environment

   python -m venv venv
   source venv/bin/activate
   pip install -r requirements.txt

2. Configure environment

   cp .env.example .env
   # Edit .env with your credentials

3. Run locally

   uvicorn app.main:app --host 0.0.0.0 --port 8000

4. Test the service

   curl http://localhost:8000/health

All configuration is managed through environment variables. See .env.example for the full list.

gcloud config set project YOUR_PROJECT_ID

echo -n "your-dd-api-key" | gcloud secrets create dd-api-key --data-file=-
echo -n "your-dd-app-key" | gcloud secrets create dd-app-key --data-file=-
echo -n "your-github-token" | gcloud secrets create github-token --data-file=-
echo -n "your-webhook-secret" | gcloud secrets create github-webhook-secret --data-file=-

export GCP_PROJECT=your-project-id
export GCP_LOCATION=us-central1
./deploy.sh

1. Go to your repository → Settings → Webhooks → Add webhook
2. Payload URL: https://YOUR-CLOUD-RUN-URL/webhook/github
3. Content type: application/json
4. Secret: Same value as github-webhook-secret
5. Events: Select "Pull requests"

Import the dashboard configuration from datadog/dashboard.json to visualize:

• Overview KPIs: Total reviews, avg cost, success rate, errors
• Pipeline Performance: LLM duration, RAG retrieval latency
• Cost Analysis: Per-review and cumulative costs by repository
• Token Metrics: Prompt vs completion tokens, context utilization
• RAG Metrics: Retrieval quality, chunks retrieved, indexing stats
• Security & Errors: Security events by type, error rates

.
├── app/
│   ├── main.py           # FastAPI application & endpoints
│   ├── config.py         # Configuration management
│   ├── models.py         # Pydantic models
│   ├── metrics.py        # Datadog metrics (HTTP API)
│   ├── security.py       # Security scanning
│   ├── github_client.py  # GitHub API integration
│   └── rag/
│       ├── embeddings.py # Vertex AI embeddings
│       ├── code_parser.py# Code chunking
│       ├── indexer.py    # FAISS index building
│       └── retriever.py  # Context retrieval
├── datadog/
│   └── dashboard.json    # Datadog dashboard config
├── deploy.sh             # Cloud Run deployment script
├── Dockerfile            # Container configuration
├── requirements.txt      # Python dependencies
└── .env.example          # Environment template

MIT