A production-ready Retrieval-Augmented Generation (RAG) chatbot system that allows you to chat with your documents using local LLMs. Upload files, YouTube videos, or web pages, and have intelligent conversations with the content.

• Features
• Architecture
• Tech Stack
• Installation
• Quick Start
• API Reference
• Project Structure
• Usage Examples
• Testing
• Troubleshooting
• Performance Tips
• Contributing

• 📄 Multi-Source Document Upload - PDF, TXT, Markdown, YouTube, Web pages
• 🧠 Intelligent Conversation - Context-aware with conversation history
• 🔍 Semantic Search - Vector-based retrieval with source attribution
• 🚀 Automatic Pipeline - One-call upload to storage
• 💾 Session Management - Multiple concurrent conversations

• 🏗️ Modular Architecture - Clean separation of concerns
• 🔌 RESTful API - Easy frontend integration
• 🎯 Local LLM - Privacy-focused with Ollama
• 📊 Monitoring - Built-in stats and health checks
• 🛡️ Error Handling - Comprehensive error management

Client → Flask API → FileManager/ChatSession → DocumentProcessor/LLMManager 
                                            → VectorStore → Ollama

Data Flow:

1. Upload: Document → Extract → Chunk → Embed → Store
2. Chat: Question → Retrieve Context → LLM → Answer with Sources

• Python 3.8+
• Ollama
• FFmpeg (for YouTube support)

# 1. Install Ollama
# macOS: brew install ollama
# Linux: curl -fsSL https://ollama.ai/install.sh | sh
# Windows: Download from https://ollama.com/download

# 2. Start Ollama and pull models
ollama serve
ollama pull llama3.2
ollama pull nomic-embed-text

# 3. Install FFmpeg
# macOS: brew install ffmpeg
# Linux: sudo apt-get install ffmpeg

# 4. Clone and setup project
git clone https://github.com/Rnamrata/llm-chatbot.git
cd llm-chatbot
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install -r requirements.txt

# 5. Create directories
mkdir -p uploads/media chroma_db

flask==3.0.0
flask-cors==4.0.0
langchain==0.1.0
langchain-community==0.0.10
chromadb==0.4.22
openai-whisper==20231117
yt-dlp==2023.12.30
PyPDF2==3.0.1
requests==2.31.0
numpy==1.24.3

# Terminal 1: Start Ollama
ollama serve

# Terminal 2: Start Flask Server
cd llm-chatbot
source .venv/bin/activate
python main.py

# Terminal 3: Test
python test_system.py

Server will be available at: http://localhost:5001

Base URL: http://localhost:5001

Upload Success:

{
  "success": true,
  "chunks_created": 45,
  "filename": "document.pdf"
}

Chat Response:

{
  "success": true,
  "answer": "Machine learning is...",
  "sources": [...],
  "num_sources": 5,
  "session_id": "abc-123"
}

llm-chatbot/
├── main.py                     # Flask app entry point
├── requirements.txt            # Dependencies
├── README.md                   # Documentation
│
├── src/modules/
│   ├── file_manager.py         # Upload & processing
│   ├── document_processor.py   # Text extraction & chunking
│   ├── vector_store_and_embedding.py  # Vector operations
│   ├── llm_manager.py          # LLM management
│   └── chat_session.py         # Session management
│
├── test/
│   └── test_system.py          # Test suite
│
├── uploads/                    # Uploaded files
│   └── media/                  # YouTube audio
│
└── chroma_db/                  # Vector database

import requests

BASE_URL = 'http://localhost:5001'

# Upload PDF
with open('document.pdf', 'rb') as f:
    requests.post(f'{BASE_URL}/upload/file', files={'file': f})

# Create session
response = requests.post(f'{BASE_URL}/chat/new')
session_id = response.json()['session_id']

# Ask question
response = requests.post(f'{BASE_URL}/chat', json={
    'question': 'What is this document about?',
    'session_id': session_id
})
print(response.json()['answer'])

# Upload YouTube video
requests.post(f'{BASE_URL}/upload/youtube', 
    json={'url': 'https://www.youtube.com/watch?v=...'})

# Chat with transcription
response = requests.post(f'{BASE_URL}/chat',
    json={'question': 'Summarize the video'})

# Upload web page
curl -X POST http://localhost:5001/upload/web \
  -H "Content-Type: application/json" \
  -d '{"url": "https://en.wikipedia.org/wiki/Python_(programming_language)"}'

# Chat
curl -X POST http://localhost:5001/chat \
  -H "Content-Type: application/json" \
  -d '{"question": "What is Python?"}'

# Get stats
curl http://localhost:5001/stats

python test_system.py

Use Postman, cURL, or Python requests to test endpoints. See Usage Examples.

1. Choose Right Model

   • Fast: phi3
   • Balanced: llama3.2:1b
   • Best: llama3.2 (default)

2. Optimize Chunk Size

   # In document_processor.py
   chunk_size=500,  # Smaller = faster
   chunk_overlap=50

3. Reduce Retrieved Chunks

   {"question": "...", "k": 3}  # Default is 5

4. Regular Cleanup

   # Clean up old sessions daily
   curl -X POST http://localhost:5001/chat/cleanup \
     -H "Content-Type: application/json" \
     -d '{"inactive_hours": 24}'

Planned Features:

• User authentication & authorization
• Database persistence (PostgreSQL)
• Frontend UI (React/Vue)
• More file formats (DOCX, PPTX, CSV)
• Streaming responses
• Document update/deletion
• Export conversations
• Docker containerization
• Analytics dashboard

Contributions welcome! Please:

1. Fork the repository
2. Create feature branch (git checkout -b feature/AmazingFeature)
3. Commit changes (git commit -m 'Add feature')
4. Push to branch (git push origin feature/AmazingFeature)
5. Open Pull Request

Code Style:

• Follow PEP 8
• Add docstrings
• Include type hints
• Write tests

Namrata Roy

• GitHub: @Rnamrata
• Email: [email protected]

• LangChain - RAG framework
• Ollama - Local LLM
• ChromaDB - Vector database
• Flask - Web framework
• Whisper - Transcription

Built using Python, LangChain, and Ollama