Web Scraping API

GET /api/scrape $0.01 per call USDC on Base · x402

Fetch any public web page and return clean, readable Markdown with navigation, ads, and boilerplate stripped. For agents that need article text or documentation as Markdown.

Parameters

Name	In		Description
`url`	query	required	Public http(s) URL of the page to fetch and convert to markdown

Example request

curl "https://api.webbersites.com/api/scrape?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FMarkdown"
# first call returns 402 + payment requirements; an x402 client pays and retries automatically

Example response

{
    "url": "https://example.com",
    "title": "Example",
    "word_count": 120,
    "markdown": "# Example\n\nClean readable text..."
}

MCP tool: get_scrape — via npx -y webbersites-x402-mcp (local, key stays on your machine) or the remote endpoint https://api.webbersites.com/mcp.

How payment works

There is no signup and no API key. Call the endpoint; it replies 402 Payment Required with machine-readable payment requirements. Your client signs a USDC transfer authorization (EIP-3009, gasless) and retries with the X-PAYMENT header — @x402/fetch does this automatically. See the overview for a working snippet.