This Cloudflare Worker is responsible for serving llms.txt and related markdown documentation files directly from Cloudflare R2 storage, ensuring high availability and zero load on the main application backend for these resources.
- Reliability: It serves content even if the main Fly.io backend is down.
- Performance: Files are served from Cloudflare's edge network, closer to the user/crawler.
- Efficiency: Offloads static file serving from the application server.
The worker intercepts requests to divemap.gr for specific files:
llms.txtdive-sites.mddive-routes.mddiving-centers.mddives.md
When a request matches one of these files:
- The worker checks the
divemap-prodR2 bucket for the corresponding file in thellm_content/folder. - If found, it serves the file with aggressive caching headers:
Cache-Control: public, max-age=86400, stale-while-revalidate=86400, stale-if-error=86400
- If not found (or on error), it falls back to fetching from the origin server (the main website).
The worker project is located in divemap-llm-worker/late-moon-cc3c (the default name generated by Wrangler).
src/index.ts: The main worker logic.wrangler.jsonc: Cloudflare configuration (routes, bindings).
- Node.js installed
npminstalled- Access to the Cloudflare account
cd divemap-llm-worker/late-moon-cc3c
npm installTo deploy updates to the worker:
cd divemap-llm-worker/late-moon-cc3c
npx wrangler deployThis command will:
- Build the TypeScript code.
- Upload the worker to Cloudflare.
- Update the route configurations.
The actual content (llms.txt, *.md) is generated by the backend task backend/generate_static_content.py. This script runs daily (or manually) and uploads the fresh content to the divemap-prod R2 bucket. The worker then serves whatever is in that bucket.
To force a content update:
- Run the generation script in the backend.
- The worker will serve the new content once the cache expires (24h) or immediately if the cache is purged.