Late-interaction retrieval for on-prem AI. One node. CPU or GPU. MaxSim is the truth scorer.

ColBERT-quality retrieval is table-stakes for serious RAG, and the production options force a choice you should not have to make.

• Managed SaaS — fast to start, hard to control, your data leaves the box.
• Distributed clusters — strong recall, expensive to operate.
• Offline benchmarks — great numbers, no API, no WAL, no recovery.

Most "production" stacks treat MaxSim as an optional rerank stage and lose its signal under aggressive shortlisting. Most engines that ship operationally drop late interaction entirely.

voyager-index is a multi-vector native retrieval engine built around MaxSim as the final scorer — and engineered so a single machine can serve it.

• One-node deployment. No control plane, no orchestration tax.
• One contract across CPU and GPU. Rust SIMD on CPU, Triton on GPU.
• Quantized fast paths. FP16, INT8, FP8, ROQ-4, all reranked back to float truth.
• Late-interaction native. ColBERT, ColPali, ColQwen out of the box.
• Database semantics. WAL, checkpoint, crash recovery, scroll, retrieve.
• Optional graph lane. The Latence sidecar augments first-stage retrieval — never required.

pip install "voyager-index[full,gpu]"   # drop ,gpu on CPU-only hosts
voyager-index-server                    # OpenAPI at http://127.0.0.1:8080/docs

Python:

import numpy as np
from voyager_index import Index

rng = np.random.default_rng(7)
docs = [rng.normal(size=(16, 128)).astype("float32") for _ in range(32)]
query = rng.normal(size=(16, 128)).astype("float32")

idx = Index("demo", dim=128, engine="shard", n_shards=32,
            k_candidates=256, compression="fp16")
idx.add(docs, ids=list(range(len(docs))))
print(idx.search(query, k=5)[0])

HTTP (base64 vector payloads, fp8 GPU scoring, ColBANDIT pruning):

import numpy as np, requests
from voyager_index import encode_vector_payload

q = np.random.default_rng(7).normal(size=(16, 128)).astype("float32")
r = requests.post(
    "http://127.0.0.1:8080/collections/demo/search",
    json={"vectors": encode_vector_payload(q, dtype="float16"), "top_k": 5,
          "quantization_mode": "fp8", "use_colbandit": True},
    timeout=30,
)
print(r.json()["results"][0])

Docker:

docker build -f deploy/reference-api/Dockerfile -t voyager-index .
docker run -p 8080:8080 -v "$(pwd)/data:/data" voyager-index

• Routing — LEMUR proxy router + FAISS MIPS shortlist, optional ColBANDIT query-time pruning.
• Scoring — Triton MaxSim and fused Rust MaxSim, INT8 / FP8 / ROQ-4 with float rerank.
• Storage — safetensors shards, memory-mapped CPU, GPU-resident corpus mode.
• Hybrid — BM25 + dense fusion via RRF or Tabu Search refinement.
• Multimodal — text (ColBERT), images (ColPali / ColQwen), preprocessing for PDF / DOCX / XLSX.
• Operations — WAL, checkpoint, crash recovery, scroll, retrieve, multi-worker FastAPI.
• Optional graph lane — Latence sidecar for graph-aware rescue and provenance, additive to the OSS path.
• Optional groundedness lane — Latence Trace premium sidecar for post-generation hallucination scoring against retrieved chunk_ids or raw context. Calibrated green/amber/red risk band, NLI peer with cross-encoder premise reranking, atomic-claim decomposition, retrieval-coverage observability, response chunking, multilingual EN+DE, three Pareto-optimal profiles, ~118 ms p95 end-to-end with NLI on. Commercial license; runs as a separate process, additive to the OSS retrieval path. See the Groundedness sidecar guide and latence.ai for access.

Encoder: lightonai/GTE-ModernColBERT-v1. CPU lane uses 8 native Rust workers.

GPU P95 stays under 6 ms across every dataset. The full per-dataset head-to-head against next-plaid (same model, H100, encoding included), methodology, and caveats live in docs/benchmarks.md.

query (token / patch embeddings)
  → LEMUR routing MLP → FAISS ANN → candidate IDs
  → optional BM25 fusion · centroid pruning · ColBANDIT
  → exact MaxSim   (Rust SIMD CPU  |  Triton FP16/INT8/FP8/ROQ-4 GPU)
  → optional Latence graph augmentation
  → top-K (or packed context)

Three execution modes share the same collection format and API contract: CPU exact (mmap → Rust fused), GPU streamed (CPU → GPU → Triton), and GPU corpus (fully VRAM-resident). Start with CPU, add GPU when latency matters.

• Quickstart · Installation · Install from source
• Reference API Tutorial · HTTP endpoint reference · Python API
• Shard Engine Guide · Max-Performance Guide · Scaling Guide
• Latence Graph Sidecar Guide · Groundedness Sidecar Guide · Enterprise Control Plane
• Benchmarks And Methodology · Production Notes
• Full Feature Cookbook

• File a bug: bug report template
• Request a feature: feature request template
• Open a PR: pull request template
• Contributing guide: CONTRIBUTING.md
• Security policy: SECURITY.md
• Release process: RELEASING.md
• Code of Conduct: CODE_OF_CONDUCT.md

Apache-2.0. See LICENSE.