Everyone summarizes WhatsApp.
LangRAG goes deeper.
_{(With Beeper! )}

⭐ The newsletter solution chosen by AI community leaders since 2024 ⭐

• Overview
  • Why
  • How
  • What
• Chosen By Leading AI-Engineering Communities
  • LangTalks
  • API: AI Protocols Israel
  • n8n Israel
• How It Works
  • Pipeline Overview
  • Beeper: The Secret Sauce
  • Single Chat Analysis
  • Multi-Chat Consolidation
  • What Gives This Solution an Edge
  • Tech Stack
• Setup
  • Getting Started
  • User Interfaces
  • API Examples
  • Configuration Reference
  • Adding Your Own WhatsApp Group
• Be a Friend
  • Improving LangRAG
  • Supporting LangRAG

If you're an AI engineer, you're dealing with FOMO.
So community discussions are the best place for you to be up to speed, learn what's worth your attention, and sharpen your knowledge by discussing real-world challenges with peers.
AI practitioners sharing nuanced, battle-tested insights and perspectives.
The problem: the messages are too valuable to miss, but too many to keep up with.

LangRAG uses Beeper for the crucial first phase: extracting WhatsApp messages along with rich metadata.
Then, LangRAG runs through async LangGraph pipelines that process group-chats in parallel, and generates structured newsletters. The analysis settings and output destinations are deeply configurable.
Currently works with OpenAI or Anthropic as the LLM provider.
Cost-optimized with Batch API (50% savings) and local SLM pre-filtering (15-30% additional savings).
Full observability via Langfuse, Prometheus, and Grafana.

Generating newsletters since November 2024 across lively communities containing 12+ active WhatsApp groups with more than 8,000 distinct members.

Israel's largest AI-engineering community.

Founded by Lee Twito & Gal Peretz

Join the community 🍕🍻

The go-to community for MCP, A2A, MCP-UI, and emerging AI protocol standards.

Founded by Gilad Shoham, Leon Melamud & Adir Duchan

Join the community 🍕🍻

Israel's workflow automation community.

Founded by Elay Guez, Gilad Shoham & Leon Melamud

Join the community 🍕🍻

View static diagram

Most WhatsApp export tools lose reply metadata - the link between a reply and the specific message it responds to. Without it, discussion-separation algorithms must fall back on timestamp proximity and content similarity, which is inherently error-prone. A group member might continue a discussion hours or even a full day later, replying to a message from yesterday - but with dozens of unrelated messages in between, there's no reliable way to reconstruct that thread from timestamps alone.

Beeper solves this by bridging WhatsApp to the Matrix protocol. Matrix exposes the m.in_reply_to field on every reply event, giving LangRAG explicit reply chains rather than guesses. The separate_discussions node uses these chains as its primary signal for grouping messages into topical discussions, with LLM-based content analysis as a secondary signal for standalone messages.

Better data, better results.

Each chat runs through the SingleChatAnalyzer - a LangGraph pipeline which would be used as a subgraph in multi-chat. These are its stages:

Pulls messages from the selected groups, using a Beeper/Matrix bridge. Preserves metadata for reply threading. Includes automatic decryption using a hybrid decryption manager with three configurable strategies: persistent session store, server-side key backup, and manual key export.

A local Phi3 model (via Ollama) classifies each message as KEEP, FILTER, or UNCERTAIN. Reduces downstream LLM API calls by 15-30%. Fail-soft: if the SLM is unavailable, the pipeline continues without filtering. UNCERTAIN messages always pass through to the LLM pipeline.

Parse timestamps, sender names, media attachments, and URLs. Pure transformation | no LLM calls. Structures raw Matrix events into a normalized message format.

Normalize all messages to English using gpt-4o-mini in batch mode. Messages already in English are skipped. This ensures consistent language for downstream analysis regardless of the source language.

Group messages into topical discussions. Uses reply-to chains (from Beeper's m.relates_to) as the primary signal, with LLM-based content analysis as a secondary signal for standalone messages. Outputs a structured list of discussions, each with its constituent messages.

Score discussions by importance and diversity. Uses a discussion ranker subgraph that combines multi-factor scoring with Maximal Marginal Relevance (MMR) re-ranking. Includes anti-repetition checks against previous newsletters (embedding cosine similarity + LLM validation) | reducing repeated content by up to 80%.

Create the newsletter content from the top-ranked discussions. Uses format plugins (langtalks_format, mcp_israel_format) to produce output as JSON, Markdown, and HTML. Each format defines its own structure, section ordering, and editorial style.

Extract URLs from the original messages and optionally perform web searches to find additional relevant links. Inserts markdown hyperlinks into the newsletter content non-destructively | existing text is preserved, links are woven in contextually.

When processing multiple WhatsApp groups, the ParallelOrchestrator coordinates the full pipeline. The orchestrator dispatches N chats concurrently using LangGraph's Send API, each running the full 8-stage SingleChatAnalyzer independently. After all chats complete, the consolidation stages begin:

Collect all discussions from all chats into a unified pool. Partial success is allowed | individual chat failures don't block others.

Identify and merge discussions about the same topic across different groups (embedding cosine similarity with configurable thresholds: strict, moderate, aggressive).

Score and rank the merged discussion pool using the same multi-factor scoring system, now operating cross-chat.

Apply Maximal Marginal Relevance re-ranking to balance quality with diversity across the consolidated discussion pool.

Create a single unified newsletter from the top-ranked discussions.

For supported formats, the pipeline supports a two-phase execution. Phase 1 runs through consolidation and ranking, then pauses | presenting the ranked discussions to a human editor via the Web UI. The editor reviews, reorders, or deselects discussions. Phase 2 generates the newsletter using only the human-approved discussions.

Final translation to the target language (if it's different than the normalized English which was used for analysis).

Format the newsletter output into multiple structures: JSON, Markdown, and HTML. Each format plugin defines its own structure, section ordering, and editorial style.

The output handler dispatches to multiple destinations:

• Local files | JSON, Markdown, HTML saved to output/ directory
• Email | Via SendGrid, Gmail, or SMTP2GO
• LinkedIn | Draft post via n8n webhook
• Webhook | Custom HTTP endpoint

• Docker and Docker Compose
• OpenAI/Anthropic API key
• Beeper account with WhatsApp bridge configured

LangRAG reads WhatsApp messages through Beeper, which bridges WhatsApp to the Matrix protocol.

See the full setup guide: docs/setup/SETUP_BEEPER.md

Quick version:

1. Create a Beeper account and link your WhatsApp
2. Export your E2E encryption keys from the Beeper Web UI
3. Place the exported keys file at ./secrets/exported_keys/element-keys.txt
4. Set BEEPER_EMAIL, BEEPER_PASSWORD, and BEEPER_EXPORT_PASSWORD in .env

git clone https://github.com/eladlaor/langrag.git
cd langrag

cp .env.example .env
# Edit .env | at minimum set: OPENAI_API_KEY or ANTHROPIC_API_KEY, BEEPER_USERNAME, BEEPER_PASSWORD

# Generate required secrets
echo "LANGFUSE_AUTH_SECRET=$(openssl rand -base64 32)" >> .env
echo "LANGFUSE_SALT=$(openssl rand -base64 32)" >> .env
echo "LANGFUSE_DB_PASSWORD=$(openssl rand -base64 32)" >> .env

See Configuration Reference for the full list of parameters and environment variables.

docker compose up -d

That's it.

# Single chat newsletter
curl -X POST "http://localhost:8000/api/generate_periodic_newsletter" \
  -H "Content-Type: application/json" \
  -d '{
    "start_date": "2025-10-01",
    "end_date": "2025-10-14",
    "data_source_name": "langtalks",
    "whatsapp_chat_names_to_include": ["LangTalks Community"],
    "desired_language_for_summary": "english",
    "summary_format": "langtalks_format",
    "consolidate_chats": false
  }'

# Multi-chat consolidated newsletter
curl -X POST "http://localhost:8000/api/generate_periodic_newsletter" \
  -H "Content-Type: application/json" \
  -d '{
    "start_date": "2025-10-01",
    "end_date": "2025-10-14",
    "data_source_name": "mcp_israel",
    "whatsapp_chat_names_to_include": ["MCP Israel", "MCP Israel #2"],
    "consolidate_chats": true
  }'

Output is written to output/<source>_<start_date>_to_<end_date>/ with per_chat/ and consolidated/ subdirectories.

Add "create_linkedin_draft": true to any newsletter API request to automatically create a LinkedIn draft post from the consolidated newsletter. Requires a one-time n8n + LinkedIn OAuth setup.

See: docs/setup/SETUP_LINKEDIN_WEBHOOK.md

See .env.example for the full list. Key variables:

Adding a community takes a few constant definitions and a Docker rebuild:

1. Backend | Add the community to src/constants.py (data source name, chat list, format mapping)
2. Frontend | Add the community to ui/frontend/src/constants/index.ts

Note: Chat names are case-sensitive and must match Beeper/Matrix room names exactly.

Feel free to connect if you'd like me to hook this up for you. If you're leading an AI-engineering community, I'll be happy to add it to the system's periodic runs, and generate custom-format newsletters for your community.

Feature request? Optimization suggestion? Bug report? Open an issue. Thanks!

If you're feeling generous, please consider clicking that "Star" thingy at the top ⭐