Web Extraction API for AI Agents

Open Source · Free Tier Available · Production Ready

Your AI agent just unlocked the internet.

Structured, AI‑ready content from any URL. One API call.

Start Extracting Free → How It Works ↓

No credit card · ~2,000 fetches/month free

API live · 99.9% uptime · npm ↗ · GitHub ↗

99.9% uptime SLA

97.6% protected success

<650ms avg latency

7 MCP tools

30-day rolling averages · Tested across 500+ domains · methodology ↗

Works with

Claude GPT-4o Cursor LangChain Python Node.js

HOW IT WORKS

Three lines. Live in minutes.

Install

One command. Works everywhere — Node.js or Python. No dependencies to configure.

# Node.js npm install -g webpeel # Python pip install webpeel

Fetch

Pass any URL. WebPeel auto-detects what's needed — HTTP, headless browser, or stealth mode.

# Pipe directly to your LLM webpeel "https://example.com" --content-only | llm "summarize" # See engine decisions live webpeel "https://complex-spa.com" --progress # Process local HTML via stdin cat page.html | webpeel --stdin # Batch 50 agent queries in parallel curl api.webpeel.dev/v1/agent/batch

Integrate

Clean markdown, metadata, structured data — ready for your agent in milliseconds.

from webpeel import peel # Python page = peel('stripe.com') // or Node.js: const page = await peel('stripe.com')

FEATURES

Everything your AI agent needs to browse.

Extract structured content from anywhere. Everything you need for production AI agents — from simple fetches to full research pipelines.

🔒 stripe.com

✓ 2,847 tokens extracted

Zero-Config Extraction

JS rendering, Cloudflare bypass, paywall handling — all automatic. Just pass the URL.

POST /v1/agent
Searching 47 sources…
→ 3 synthesis reports
→ Key facts extracted

🧠 Research Agent

Built-in deep research agent synthesizes info from 47+ sources into structured insights.

{

"schema": "{ price, rating }",

"price": "$29.99",

"rating": 4.8,

"in_stock": true

}

📊 Structured JSON

Extract structured data with JSON schemas. Define what you want, get clean JSON back.

vercel.com

Develop. Preview. Ship.

Full-page · custom viewport · device emulation

▶ screenshot captured

📸 Screenshot API

Full-page screenshots with device emulation. Capture what users see, pixel perfect.

GET /v1/search?q=AI+trends
27+ sources · ranked by trust
★★★ Official · ★★ Verified
→ Credibility-ranked results

🔍 Web Search

Built-in search across 27+ sources with credibility ranking. Official & verified sources float to top.

POST /v1/fetch
actions: [
  { click: "#login" },
  { type: "[email protected]" },
  { wait: 1000 }
]
✓ Extracted after login

🤖 Browser Actions

Click, type, scroll, wait — automate any page interaction before extraction.

webpeel monitor "site.com/price"
Watching for changes…
+ $29.99 → $24.99
Diff delivered · 2 changes

📡 Content Monitoring

Track any page for changes. Get diffs when content updates. Watch mode built in.

webpeel "youtube.com/watch?v=…"
Extracting transcript…
4,210 tokens · timestamped
Auto-detect · 50+ languages

🎬 YouTube Transcripts

Full video transcripts with timestamps. Auto-detects language, 50+ languages.

webpeel crawl "docs.stripe.com"
depth: 3 · dedup · rate-limited
✓ 87 / 100 pages crawled
→ 412k tokens extracted

🌍 Crawl Entire Sites

Crawl full websites with depth control, dedup, and rate limiting. Up to 100 pages.

WHY WEBPEEL

Other tools fetch pages.
WebPeel reads them.

Any page. Any protection. Auto-handled — so your agent always gets through.

Handles JS rendering, SPAs, Cloudflare
32 domain extractors for structured data
7 MCP tools for Claude, Cursor, GPT
Firecrawl-compatible API

Before (raw HTML)

<div class="wrapper"> <nav>...50 links...</nav> <aside>...ads, popups...</aside> <script>...tracking...</script> <article>The actual content</article> </div>

↓ webpeel --readable

After (WebPeel output)

# The Verge — Tech & Science **AI is reshaping how we search the web** Google's new AI Overviews summarize results using Gemini. Critics worry about publisher traffic. Advertisers are watching closely. > "The biggest shift in search since 2009." *2,847 tokens · rendered in 412ms*

Content completeness

WebPeel

98%

Firecrawl

82%

raw fetch

56%

Protected site success

WebPeel

97.6%

Firecrawl

65%

raw fetch

35%

WORKS WITH ● Claude ● Cursor ● GPT-4o ● LangChain ● Windsurf ● OpenClaw

PRICING

Start free. Scale when ready.

Every plan includes MCP tools, extraction & search. Scale up when your agent needs more.

Free

$0/month

500 fetches/week (~2,000/month)

All 7 MCP tools
Full browser rendering + JS
Readability extraction
Screenshot API
Web search (27+ sources)
YouTube transcripts

Get Started Free →

Common questions.

WebPeel extracts more content (98% vs 82% completeness), has better protected site success (97.6% vs 65%), includes 7 MCP tools vs ~6, and has a built-in deep research agent. It's also fully open source and offers a Firecrawl-compatible API for easy migration.

Free tier: requests return a 429 status with a Retry-After header. No charges, no service interruption — limits reset every Monday at 00:00 UTC. Pro and Max plans have higher limits and you can upgrade instantly from the dashboard. We never charge overages without explicit opt-in.

WebPeel auto-escalates through three levels: simple HTTP fetch, headless browser rendering, and advanced stealth mode. It handles JavaScript-heavy pages, dynamic content, and protected sites with a 97.6% content retrieval success rate across 500+ URLs.

Yes. 500 fetches per week, no credit card required. The free tier includes all features — browser rendering, MCP tools, search, extraction, everything. Weekly limits reset every Monday.

Yes. WebPeel is open source. Install the CLI with npm install -g webpeel and run your own MCP server locally for unlimited usage.

WebPeel is a tool — responsible use is on the operator. By default, the CLI respects robots.txt. We recommend only fetching pages you have permission to access, using reasonable rate limits, and complying with each site's terms of service.

Markdown (default), plain text, raw HTML, and structured JSON via schema extraction. You can use JSON Schema for precise field extraction, built-in CSS schemas for common page types, or LLM extraction with your own API key.

WebPeel offers a Firecrawl-compatible API endpoint — change your base URL and you're done. The /v1/scrape and /v1/crawl endpoints accept the same request format. Most migrations take under 5 minutes. See the documentation for details.