A complete, single-file technical documentation for the SHL Assessment Recommendation System, fully rewritten based on the detailed reference document you shared. This README covers the entire architecture, end-to-end pipeline, algorithms, mathematical formulations, evaluation results, and deployment overview.

The SHL Assessment Recommendation System is a Retrieval-Augmented Generation (RAG)–based application that intelligently recommends SHL assessments for recruiters based on natural language queries.

Recruiters often express complex, multi-dimensional requirements such as:

"Java developer who collaborates with business teams"

This query contains:

• ✅ Technical requirement (Java)
• ✅ Soft skills requirement (collaboration)
• ✅ Personality traits (teamwork)
• ✅ Implicit cognitive skill expectations (problem-solving)

Traditional keyword search cannot handle this complexity. This system addresses the challenge using a 3-stage technical pipeline:

1. Query Classification (LLM-powered)
2. Hybrid Retrieval (Dense + Sparse)
3. Three-Factor Re-ranking
4. Optional LLM Contextual Recommendation Engine

• HTML pages scraped using BeautifulSoup

• Cleaned to extract:

  • name
  • category
  • description
  • duration
  • test types
  • remote testing
  • adaptive IRT flag

Original: 800 chars
Chunks: ~500 chars + overlapping 100 chars

• OpenAI ada-002
• 1536-dim vectors
• Normalized to unit length

The system converts free-text queries into probabilities across 8 SHL test categories:

P(type | query) = {p_A, p_B, p_C, p_D, p_E, p_K, p_P, p_S}
∑ p_i = 1.0

Example:

"Java developer who collaborates with business teams"

Output:

• K (Knowledge): 0.55
• C (Competencies): 0.25
• P (Personality): 0.15
• A (Ability): 0.05

This distribution is used later in the ranking stage.

Retrieval combines two complementary search mechanisms:

Uses cosine similarity between query embedding and document embedding:

sim(q, d) = (q · d) / (|q| |d|)

Pros:

• Understands synonyms
• Understands semantics

Captures lexical signals based on term frequency and rare-term weighting.

BM25(D, Q) = Σ IDF(qi) * [...]

Pros:

• Exact keyword matching
• Strong for exact terms

score_hybrid = α * score_dense + (1 - α) * score_sparse

Default:

α = 0.7

Interpretation:

• 70% semantic intent
• 30% keyword alignment

Initial retrieval may be misleading due to:

• generic descriptions
• incorrect test types
• keyword-only matches

Final ranking uses three weighted factors:

S_final = 0.5 S_similarity + 0.3 S_description + 0.2 S_type

Uses normalized hybrid retrieval score.

Uses cosine similarity between query and full description embedding.

Uses classification probabilities from Stage 1.

S_type = avg(P(t) for t in candidate.test_types)

Why this works:

• Description matching promotes content relevance
• Type alignment ensures category correctness
• Retrieval similarity captures semantic meaning

After re-ranking, an LLM analyzes the top candidates.

• Improves diversity of results
• Removes irrelevant matches
• Generates justifications
• Produces structured output

{
  "recommendations": [
    {
      "name": "Python (New)",
      "url": "https://...",
      "justification": "Directly assesses Python programming knowledge..."
    }
  ],
  "explanation": "These assessments balance technical and competency needs."
}

• 3 real queries with manually labeled relevant assessments

• 83% of relevant assessments appear in top-5
• No further improvement at top-10 → strong ranking consistency
• 2/3 queries achieve perfect recall (100%)

✅ Strong semantic understanding ✅ Accurate hybrid retrieval ✅ Intelligent re-ranking ✅ High recall in top-5 ✅ Diverse and contextual recommendations

⚠ Small evaluation set (3 queries) ⚠ Need more real-world query coverage

• Expand evaluation to 20–30 queries
• Target 90%+ recall
• Add fuzzy keyword matching
• Improve chunking strategy
• Add usage analytics

The SHL Assessment Recommendation System successfully combines semantic embeddings, lexical search, structured query classification, multi-factor ranking, and LLM reasoning to deliver:

✅ 83% recall in top-5 ✅ 67% perfect recall rate ✅ +15% improvement over previous version ✅ Reliable and context-aware recommendations

This README contains everything needed to understand, maintain, and further enhance the system.