A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.

Fully neural approach for text chunking

🍱 semantic-chunking ⇢ semantically create chunks from large document for passing to LLM workflows

RAG boilerplate with semantic/propositional chunking, hybrid search (BM25 + dense), LLM reranking, query enhancement agents, CrewAI orchestration, Qdrant vector search, Redis/Mongo sessioning, Celery ingestion pipeline, Gradio UI, and an evaluation suite (Hit-Rate, MRR, hybrid configs).

Rust CLI implementing the Recursive Language Model (RLM) pattern for Claude Code. Process documents 100x larger than context windows through intelligent chunking, SQLite persistence, and recursive sub-LLM orchestration.

Token-aware, LangChain-compatible semantic chunker with PDF, markdown, and layout support

🍶 llm-distillery ⇢ use LLMs to run map-reduce summarization tasks on large documents until a target token size is met.

Semantic Chunking is a Python library for segmenting text into meaningful chunks using embeddings from Sentence Transformers.

Advanced semantic text chunking with custom structural markers, whole-text coherence preservation, and flexible token management. Features async processing, LangChain integration, and dynamic drift detection. Ideal for RAG systems, augmented text processing, and domain-specific document analysis.

A research framework tA research framework to evaluate how document parsing quality determines downstream RAG performance.o evaluate how document parsing quality de

Advanced local-first RAG system powered by Ollama and LangGraph. Optimized for high-performance sLLM orchestration featuring adaptive intent routing, semantic chunking, intelligent hybrid search (FAISS + BM25), and real-time thought streaming. Includes integrated PDF analysis and secure vector caching.

Sementic chunking algorithm in (mostly) Go

A hands-on guide to RAG techniques using LangGraph.

treechunk é uma biblioteca TypeScript para segmentação semântica de código JS/TS baseada em AST. Extrai funções, classes, métodos e exports em blocos coerentes com preservação de contexto, otimizada para RAG, embeddings, busca de código e análise de repositórios em larga escala.

Cutting-edge semantic text processing system that uses hierarchical clustering and advanced language models to automatically organize and summarize large volumes of text.

A modular RAG pipeline for automated document processing using Semantic Chunking and Qdrant Vector Database.

A Sidecar service for applications that need vector database functionality to augment their LLMs. This service provides embeddings and retrieval capabilities by abstracting embeddings generation (LiteLLM) and vector storage and search (Qdrant).

All in One-Solution for converting documents to finetune LLMs

A high-performance Retrieval-Augmented Generation pipeline for technical Q&A workloads. Combines hybrid retrieval (dense + BM25), query expansion, Reciprocal Rank Fusion (RRF), and cross-encoder re-ranking to improve retrieval precision and answer grounding. Evaluated with Ragas, showing measurable gains in context recall and faithfulness.

Chomper - Chomp through any document. MCP server for parsing 36+ file formats with semantic chunking & TOON token optimization for Claude and AI systems.