Tutorial: Your first unsurf

This tutorial walks you through the complete unsurf loop — Scout → Worker → Heal — against a live website. By the end you will have a deployed instance, a captured API with typed schemas, and an AI agent connected over MCP.

Every step shows the exact command and its expected output. Nothing is left to guesswork.

1 — Deploy

You need a Cloudflare account on the Workers Paid plan (required for Browser Rendering) and Bun installed.

1. Clone and install

   git clone https://github.com/acoyfellow/unsurf
   cd unsurf
   bun install

2. Set your Alchemy password

   echo 'ALCHEMY_PASSWORD=choose-a-secure-password' > .env

3. Deploy

   CLOUDFLARE_API_TOKEN=your-token bun run deploy

   Alchemy creates the D1 database, R2 bucket, and Browser Rendering binding automatically. When it finishes you will see:

   Published unsurf (0.2.0)
   https://unsurf.YOUR-SUBDOMAIN.workers.dev

4. Save your URL

   Every command below uses $UNSURF_URL. Set it now:

   export UNSURF_URL="https://unsurf.YOUR-SUBDOMAIN.workers.dev"

Verify the deploy:

curl $UNSURF_URL

{
  "name": "unsurf",
  "version": "0.2.0",
  "description": "Turn any website into a typed API",
  "tools": ["scout", "worker", "heal"],
  "mcp": "/mcp",
  "docs": "https://unsurf.coey.dev"
}

2 — Scout: capture an API

Scouting launches a headless browser on Cloudflare’s edge, navigates to a URL, and captures every API call the page makes. For each endpoint it normalises the URL pattern (/posts/1 → /posts/:id), infers a JSON Schema from the response body, and generates an OpenAPI 3.1 spec.

Point it at JSONPlaceholder:

curl -X POST $UNSURF_URL/tools/scout \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://jsonplaceholder.typicode.com",
    "task": "discover all API endpoints"
  }'

Expected response (your IDs will differ):

{
  "siteId": "site_m5xq2a_7hk3p1",
  "endpointCount": 6,
  "pathId": "path_m5xq2a_9bc4d2",
  "openApiSpec": {
    "openapi": "3.1.0",
    "info": {
      "title": "jsonplaceholder.typicode.com",
      "version": "1.0.0"
    },
    "paths": {
      "/posts": {
        "get": {
          "operationId": "getPosts",
          "responses": {
            "200": {
              "description": "Successful response",
              "content": {
                "application/json": {
                  "schema": {
                    "type": "array",
                    "items": {
                      "type": "object",
                      "properties": {
                        "userId": { "type": "integer" },
                        "id": { "type": "integer" },
                        "title": { "type": "string" },
                        "body": { "type": "string" }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      },
      "/users": {
        "get": { "...":  "(same structure — inferred from captured traffic)" }
      },
      "/comments": {
        "get": { "...":  "(same structure)" }
      }
    }
  }
}

Here is what just happened:

1. A headless Chrome navigated to jsonplaceholder.typicode.com.
2. unsurf watched every XHR/fetch request via Chrome DevTools Protocol — the same mechanism behind your browser’s Network tab.
3. Each endpoint was grouped by method + normalised URL pattern, and a JSON Schema was inferred from the response body.
4. Everything was persisted to D1 and R2, and an OpenAPI 3.1 spec was generated.

Save the pathId from your response — you will use it for every remaining step:

export PATH_ID="path_m5xq2a_9bc4d2"   # ← paste yours here

Tip

The endpointCount tells you how many distinct API calls the page made on load. JSONPlaceholder’s homepage typically fires requests to /posts, /users, /comments, and a few others.

Here is the domain model for a captured endpoint — every field was populated automatically by the scout:

CapturedEndpoint — what the scout stores for each endpoint View source →

import { Schema } from "effect";

const HttpMethod = Schema.Literal("GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS");

export class CapturedEndpoint extends Schema.Class<CapturedEndpoint>("CapturedEndpoint")({
	id: Schema.String,
	siteId: Schema.String,
	method: HttpMethod,
	pathPattern: Schema.String,
	requestSchema: Schema.optionalWith(Schema.Unknown, { as: "Option" }),
	responseSchema: Schema.optionalWith(Schema.Unknown, { as: "Option" }),
	sampleCount: Schema.Number,
	firstSeenAt: Schema.String,
	lastSeenAt: Schema.String,
}) {}

3 — Worker: replay without a browser

The scout gave you a pathId that points to captured endpoints. The Worker tool replays those API calls directly over HTTP — no browser, no rendering. Responses come back in milliseconds.

curl -X POST $UNSURF_URL/tools/worker \
  -H "Content-Type: application/json" \
  -d "{\"pathId\": \"$PATH_ID\"}"

Expected response:

{
  "success": true,
  "response": [
    {
      "userId": 1,
      "id": 1,
      "title": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",
      "body": "quia et suscipit\nsuscipit recusandae ..."
    },
    {
      "userId": 1,
      "id": 2,
      "title": "qui est esse",
      "body": "est rerum tempore vitae\nsequi sint nihil ..."
    }
  ]
}

That data came straight from JSONPlaceholder’s API. unsurf looked up the stored path, found the first GET endpoint, and issued a plain fetch. No browser was involved.

You can also pass data to fill path parameters or provide a request body:

curl -X POST $UNSURF_URL/tools/worker \
  -H "Content-Type: application/json" \
  -d "{\"pathId\": \"$PATH_ID\", \"data\": {\"id\": 3}}"

If the captured endpoint pattern contains :id, Worker substitutes it with the value you provide.

4 — Heal: recover from a broken path

Websites change. Endpoints move, response shapes shift, URLs get rewritten. When a Worker replay fails, Heal fixes it automatically.

The heal sequence:

1. Retry the Worker call with exponential backoff (handles transient errors).
2. If retries fail, mark the path as broken.
3. Re-scout the original URL and task to capture fresh endpoints.
4. Retry Worker with the new path.
5. If it works, mark the original path active and return the newPathId.

Call heal with your working path to see it in action:

curl -X POST $UNSURF_URL/tools/heal \
  -H "Content-Type: application/json" \
  -d "{\"pathId\": \"$PATH_ID\", \"error\": \"simulated failure for tutorial\"}"

Because JSONPlaceholder is stable, the retry succeeds immediately — no re-scout needed:

{
  "healed": true
}

If the endpoint had genuinely broken, Heal would have re-scouted and returned a new path:

{
  "healed": true,
  "newPathId": "path_m5xq3b_4ef8g9"
}

Caution

Heal launches a browser to re-scout, so it is slower than Worker. An agent should call Heal only when Worker returns a failure — not on every request.

The path domain model tracks health across the full cycle:

ScoutedPath — status, failCount, and healCount track path health View source →

import { Schema } from "effect";

export class PathStep extends Schema.Class<PathStep>("PathStep")({
	action: Schema.Literal("navigate", "click", "fill", "submit", "wait"),
	selector: Schema.optional(Schema.String),
	value: Schema.optional(Schema.String),
	url: Schema.optional(Schema.String),
}) {}

const PathStatus = Schema.Literal("active", "broken", "healing");

export class ScoutedPath extends Schema.Class<ScoutedPath>("ScoutedPath")({
	id: Schema.String,
	siteId: Schema.String,
	task: Schema.String,
	steps: Schema.Array(PathStep),
	endpointIds: Schema.Array(Schema.String),
	status: PathStatus,
	createdAt: Schema.String,
	lastUsedAt: Schema.optionalWith(Schema.String, { as: "Option" }),
	failCount: Schema.Number,
	healCount: Schema.Number,
}) {}

5 — Connect an AI agent over MCP

unsrf exposes Scout, Worker, and Heal over a single MCP endpoint. Any MCP-compatible client can connect and use all three tools with no extra configuration.

Claude Desktop

Add this to ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "unsurf": {
      "type": "streamable-http",
      "url": "https://unsurf.YOUR-SUBDOMAIN.workers.dev/mcp"
    }
  }
}

Cursor

Add this to .cursor/mcp.json in your project root:

{
  "mcpServers": {
    "unsurf": {
      "type": "streamable-http",
      "url": "https://unsurf.YOUR-SUBDOMAIN.workers.dev/mcp"
    }
  }
}

Any MCP client

Point your client at the streamable-http endpoint:

https://unsurf.YOUR-SUBDOMAIN.workers.dev/mcp

Once connected, the agent sees three tools:

Tool	Description the agent reads	Input
scout	Explore a website and capture every API call. Returns endpoints with inferred schemas and an OpenAPI spec.	url, task
worker	Replay a scouted API path directly — no browser needed.	pathId, optional data
heal	Fix a broken path. Retries with backoff, then re-scouts and patches if needed.	pathId, optional error

Try asking Claude:

Scout jsonplaceholder.typicode.com and find all the API endpoints. Then fetch the list of users.

Claude will call scout, read the pathId from the result, then call worker — the same two steps you did with curl. If the worker call ever fails, Claude will call heal with the error message and get back a working path.

Note

The agent never needs to know about headless browsers, schema inference, or OpenAPI generation. It sees three tools, calls them in sequence, and gets structured data back.

What you learned

You completed the full unsurf loop:

1. Deployed an unsurf instance to Cloudflare — D1, R2, and Browser Rendering provisioned automatically.
2. Scouted JSONPlaceholder — a headless browser captured API calls and generated typed schemas + an OpenAPI spec.
3. Replayed a captured endpoint with Worker — direct HTTP, no browser, millisecond response times.
4. Healed a path — unsurf retried with backoff and would have re-scouted if the endpoint had genuinely broken.
5. Connected an AI agent over MCP — three tools, one endpoint, the agent handles the rest.

The core pattern: Scout turns a website into a typed API. Worker replays it fast. Heal keeps it working when sites change. An agent chains all three without human intervention.

Next steps:

• How to scout a website — targeting specific pages, multi-step navigation, and the agent-scout mode.
• How to replay a captured API — passing data, choosing endpoints, and handling errors.
• How to heal a broken path — understanding the retry policy and when to re-scout.
• MCP Server — authentication, rate limiting, and production configuration.