Distributed document processing system using Docling workers.
⚠️ Work in Progress: This project is under active development. The Python BullMQ library is not yet production-ready. Use at your own risk.
┌─────────────┐ ┌───────────────┐ ┌─────────────────┐
│ Hono API │───▶ │ Message Queue │───▶ │ Docling Workers │
│ (Bun) │ │ (Redis/BullMQ │ │ (Scalable) │
└─────────────┘ └───────────────┘ └─────────────────┘
│ │
▼ ▼
┌───────────────┐ ┌─────────────────┐
│ Job Status │ │ Object │
│ Storage │ │ Storage │
│ (Redis) │ │ (MinIO) │
└───────────────┘ └─────────────────┘- API: Hono + Bun
- Message Queue: BullMQ + Redis
- Workers: Python + Docling
- Storage: MinIO (S3-compatible)
- Infrastructure: Docker Compose
- Bun (https://bun.sh)
- Python 3.11+
- Docker & Docker Compose
- Clone and setup everything:
git clone https://github.com/albisserAdrian/parsly
cd parsly
bun run setupThis installs Bun dependencies and creates a Python virtual environment with all packages.
- Start infrastructure services:
bun run setup:docker- Copy environment files:
cp packages/api/.env.example packages/api/.env
cp packages/worker/.env.example packages/worker/.env- Run development servers:
bun run devThis starts both the API and worker in parallel.