Multi-agent NL-to-SQL copilot with governed analytics, audit trails, and multi-warehouse support

Nexus-Hive turns natural-language business questions into audited SQL, executes them safely against a warehouse, and returns chart-ready answers with a full agent trace. Every step -- translation, policy check, execution, visualization -- is independently inspectable, testable, and governed.

Why this matters: Most NL-to-SQL tools generate SQL and run it. Nexus-Hive inserts a policy engine, audit trail, and retry loop between generation and execution -- making it safe for environments where data access requires governance.

flowchart TB
    User["Business User / Analyst"]
    User -->|"Natural language question"| API["FastAPI Runtime\n/api/ask"]

    subgraph AgentPipeline["LangGraph Multi-Agent Pipeline"]
        direction TB
        T["Agent 1: Translator\nNL --> SQL via Ollama/LLM\n+ heuristic fallback"]
        E["Agent 2: Executor\nPolicy engine + read-only SQL execution"]
        V["Agent 3: Visualizer\nChart.js config generation"]
        T --> E
        E -->|"error + retry budget > 0"| T
        E -->|"success"| V
    end

    API --> AgentPipeline

    subgraph Governance["Governance Layer"]
        direction LR
        PE["Policy Engine\nDeny / Review / Allow"]
        AT["Audit Trail\nJSONL + query tags"]
        GE["Gold Eval Suite\nSQL quality scoring"]
    end

    E --- PE
    E --- AT

    subgraph Warehouses["Warehouse Adapters"]
        direction LR
        SQLite["SQLite\n(demo, active by default)"]
        SF["Snowflake\n(live when configured)"]
        DB["Databricks\n(live when configured)"]
    end

    E --> Warehouses
    V --> UI["Frontend\nChart.js + agent trace viewer"]

The pipeline is built with LangGraph as a compiled state graph. Each agent is a node with typed state (AgentState), connected by edges with conditional routing. When the Executor rejects SQL (policy denial or execution error), the graph routes back to the Translator for correction -- up to 3 retries -- without re-running the Visualizer.

git clone https://github.com/KIM3310/Nexus-Hive.git && cd Nexus-Hive
make install                 # creates venv, installs deps
python3 seed_db.py           # seeds 10k enterprise sales records into SQLite
make run                     # starts uvicorn on http://localhost:8000

Open http://localhost:8000 for the frontend, or http://localhost:8000/docs for Swagger UI.

docker compose up app
# With live LLM inference via Ollama:
docker compose --profile with-ollama up
docker exec nexus-hive-ollama ollama pull phi3

make verify   # runs lint + pytest + smoke test against a live server

All adapters implement the same 5-method interface (get_schema, run_scalar_query, fetch_date_window, build_table_profiles, execute_sql_preview) and return results in an identical format. The Executor node and Visualizer node never branch on adapter type. See ADR-002 for the design rationale.

# Ask a question (returns stream URL for agent trace)
curl -X POST http://localhost:8000/api/ask \
  -H "Content-Type: application/json" \
  -d '{"question": "Show total net revenue by region"}'

# Stream agent trace (SSE)
curl -N "http://localhost:8000/api/stream?q=Show+total+net+revenue+by+region&rid=REQUEST_ID"

# Policy check -- preview SQL before execution
curl -X POST http://localhost:8000/api/policy/check \
  -H "Content-Type: application/json" \
  -d '{"sql": "SELECT region_name, SUM(net_revenue) FROM sales GROUP BY region_name", "role": "analyst"}'

# Runtime metadata
curl http://localhost:8000/api/meta

# Run gold eval suite
curl http://localhost:8000/api/evals/nl2sql-gold/run

Interactive docs: http://localhost:8000/docs (Swagger) | http://localhost:8000/redoc (ReDoc)

From pytest 8.3.5 on Python 3.11 -- make verify (lint + test + smoke)

pip install -e ".[snowflake]"
export SNOWFLAKE_ACCOUNT=your_account.us-east-1
export SNOWFLAKE_USER=your_username
export SNOWFLAKE_PASSWORD=your_password
export SNOWFLAKE_DATABASE=ANALYTICS

pip install -e ".[databricks]"
export DATABRICKS_HOST=https://your-workspace.cloud.databricks.com
export DATABRICKS_AUTH_TYPE=databricks-cli
export DATABRICKS_WAREHOUSE_ID=your_sql_warehouse_id
python scripts/bootstrap_databricks_demo.py

Kubernetes

kubectl apply -f infra/k8s/namespace.yaml
kubectl apply -f infra/k8s/

GCP Cloud Run -- Terraform configs in infra/terraform/. See infra/terraform/README.md for variables and outputs.

Docker

docker compose up app

Production-grade overlays for regulated and high-scale environments.

A full Helm chart lives at helm/nexus-hive/ with three value overlays:

# Default / dev-staging
helm upgrade --install nexus-hive helm/nexus-hive \
  -n analytics --create-namespace

# Production: HPA, PDB, NetworkPolicy, ServiceMonitor, external secrets, HA spread
helm upgrade --install nexus-hive helm/nexus-hive \
  -n analytics --create-namespace \
  -f helm/nexus-hive/values.yaml \
  -f helm/nexus-hive/values-prod.yaml \
  --set image.tag=0.2.0

# Air-gapped: Ollama sidecar, no external LLM, internal registry only
helm upgrade --install nexus-hive helm/nexus-hive \
  -n analytics --create-namespace \
  -f helm/nexus-hive/values.yaml \
  -f helm/nexus-hive/values-airgap.yaml

See helm/nexus-hive/README.md for the full values reference.

infra/k8s/production/ ships a hardened namespace baseline: Pod Security Standards (restricted), ResourceQuota, LimitRange, default-deny NetworkPolicy, and scoped allow policies for ingress and warehouse egress.

infra/k8s/observability/ ships a ServiceMonitor and PrometheusRule set with SLO-grade alerts (error rate, p99 latency, circuit breaker, audit-write stall).

infra/terraform/ provisions Cloud Run v2 with Secret Manager, Workload Identity, VPC connector, uptime checks, and log-based alerting. See infra/terraform/README.md for the variable and output tables.

Five step-by-step runbooks live at docs/runbooks/:

Narrative case studies written as SE-style handouts. Composites drawn from common patterns; numbers illustrative.

Each story walks through the problem, the 90- or 180-day rollout, the policy-tuning decisions, and the specific lessons that fed back into the product. Designed for SE discovery + pre-sales handoff.

benchmarks/ contains reproducible performance and accuracy tests.

• benchmarks/load_test.js - k6 script, 50 VUs x 5 min with governed question mix, thresholds gated on p95/p99 latency and checks pass rate.
• benchmarks/agent_accuracy_eval.py - runs the gold eval suite N times and renders a per-case accuracy chart.
• benchmarks/sample_results.json - realistic run output.

See benchmarks/README.md for the full usage and CI integration recipe.

Nexus-Hive/
  main.py                    # FastAPI entrypoint (thin, delegates to modules)
  config.py                  # Centralized config, constants, metric definitions
  graph/
    nodes.py                 # LangGraph agent nodes + graph builder
  policy/
    engine.py                # Policy evaluation, query tags, heuristic inference
    audit.py                 # Audit trail writer
    governance.py            # Governance surfaces (scorecard, warehouse brief)
  routes/
    ask.py                   # /api/ask + /api/stream (SSE agent trace)
    health_meta.py           # /health, /api/meta
    warehouse.py             # Warehouse adapter endpoints
    auth.py                  # Operator auth session endpoints
  services/
    build_helpers.py         # Runtime brief, meta builders
    streaming.py             # SSE streaming for agent trace
    openai_helpers.py        # Optional OpenAI integration
  warehouse_adapter.py       # Adapter base class + SQLite + registry
  snowflake_adapter.py       # Live Snowflake adapter
  databricks_adapter.py      # Live Databricks adapter
  circuit_breaker.py         # Ollama circuit breaker
  security.py                # HMAC sessions, token auth, RBAC
  seed_db.py                 # 10k-row synthetic dataset generator
  tests/                     # 8 test modules, 80+ test cases
  infra/
    k8s/                     # Kubernetes manifests
    terraform/               # GCP Cloud Run Terraform configs
  frontend/                  # Chart.js + agent trace viewer
  docs/
    adr/                     # Architecture Decision Records
    solution-architecture.md
    discovery-guide.md

MIT