This directory contains everything needed to reproduce the benchmark results from the HAT paper.

# Run all benchmarks
./run_all_benchmarks.sh

# Run abbreviated version (faster)
./run_all_benchmarks.sh --quick

Test file: tests/phase31_hat_vs_hnsw.rs

Compares HAT against HNSW on hierarchically-structured data (AI conversation patterns).

Expected Results:

Key finding: HAT achieves 30% higher recall while building 70x faster.

Test file: tests/phase32_real_embeddings.rs

Tests HAT with production embedding sizes.

Expected Results:

Test file: tests/phase33_persistence.rs

Validates serialization/deserialization correctness and performance.

Expected Results:

Test file: tests/phase42_attention_state.rs

Tests the attention state serialization format.

Expected Results:

• All 9 tests pass
• Role types roundtrip correctly
• Metadata preserved
• KV cache support working

Script: examples/demo_hat_memory.py

Full integration with sentence-transformers and optional LLM.

Expected Results:

# HAT vs HNSW
cargo test --test phase31_hat_vs_hnsw -- --nocapture

# Real embeddings
cargo test --test phase32_real_embeddings -- --nocapture

# Persistence
cargo test --test phase33_persistence -- --nocapture

# Attention state
cargo test --test phase42_attention_state -- --nocapture

# Setup
python3 -m venv venv
source venv/bin/activate
pip install maturin pytest sentence-transformers

# Build extension
maturin develop --features python

# Run tests
pytest python/tests/ -v

# Run demo
python examples/demo_hat_memory.py

• Minimum: 4GB RAM, any modern CPU
• Recommended: 8GB RAM for large-scale tests
• Storage: ~2GB for full benchmark suite

1. Recall@k: Percentage of true nearest neighbors found

   • HAT target: 100% on hierarchical data
   • HNSW baseline: ~70% on hierarchical data

2. Build Time: Time to construct the index

   • HAT target: <100ms for 1000 points
   • Should be 50-100x faster than HNSW

3. Query Latency: Time per query

   • HAT target: <5ms
   • Acceptable to be 2-3x slower than HNSW (recall matters more)

4. Throughput: Serialization/deserialization speed

   • Target: 100+ MB/s

The benchmarks validate the paper's claims if:

1. HAT recall@10 ≥ 99% on hierarchical data
2. HAT recall significantly exceeds HNSW on hierarchical data
3. HAT builds faster than HNSW
4. Persistence preserves 100% recall
5. Python bindings pass all tests
6. End-to-end demo achieves ≥95% retrieval accuracy

# Update Rust
rustup update

# Clean build
cargo clean && cargo build --release

# Ensure venv is activated
source venv/bin/activate

# Rebuild extension
maturin develop --features python --release

For large-scale tests, ensure sufficient RAM:

# Check available memory
free -h

# Run with limited parallelism
RAYON_NUM_THREADS=2 cargo test --test phase31_hat_vs_hnsw

Results are saved to benchmarks/results/:

results/
  benchmark_results_YYYYMMDD_HHMMSS.txt  # Full output

If you use these benchmarks, please cite:

@article{hat2026,
  title={Hierarchical Attention Tree: Extending LLM Context Through Structural Memory},
  author={AI Research Lab},
  year={2026}
}