webforai platform

webforai platform is a hosted crawl→Markdown API built on this library. It takes a URL (or a whole site) and gives you back the same clean Markdown htmlToMarkdown produces, without you running the browsers, proxies and queues yourself.

It is fully open source and self-hostable — the entire service is a single Cloudflare Worker in apps/platform of this repository.

curl -X POST https://platform.webforai.dev/v1/scrape \
  -H "Authorization: Bearer wfa_..." \
  -H "content-type: application/json" \
  -d '{ "url": "https://example.com/article" }'

Try it without an account on the live demo, or read the full API reference.

Quickstart

Create an account at platform.webforai.dev — no card required, 500 credits per month are free.
Mint an API key on the dashboard. Keys look like wfa_... and are shown once at creation.
Send the first request — with curl (above), the TypeScript client, or the CLI. The playground runs the same request from the browser and prints the equivalent curl.

Clients

TypeScript — webforai/platform, a subpath of the webforai package: typed, dependency-free, with job polling and result paging built in.

import { createPlatformClient } from "webforai/platform";
 
const platform = createPlatformClient({ apiKey: process.env.WEBFORAI_API_KEY });
const { markdown } = await platform.scrape({ url: "https://example.com", engine: "browser" });

CLI — npx webforai uses the platform as a loader:

WEBFORAI_API_KEY=wfa_... npx webforai https://example.com --engine browser --json

Anything that speaks HTTP — the API reference documents every route; authentication is one Authorization: Bearer wfa_... header.

Capabilities

Scrape — a single URL → Markdown + metadata. Synchronous, returns in one request.
Batch — a list of URLs (≤100 per job) → Markdown per URL. Asynchronous.
Crawl — a seed URL → recursive same-origin crawl with depth/page limits and include/exclude path patterns → Markdown per page. Asynchronous.

Asynchronous jobs run the exact same core logic on Cloudflare Workflows, so every page gets durable execution and retries. Results are retained for 7 days and read back through the jobs API.

Engines

Every request picks how the HTML is acquired. Engines differ in what they can render, whether they egress through a proxy, and what they cost in credits.

engine	how it fetches	proxy	screenshot	credits
`fetch`	Workers `fetch()`	no	no	1
`browser`	Browser rendering	no	yes	5
`proxy-fetch`	Plain fetch through a rotating proxy	yes	no	2
`proxy-browser`	Browser rendering behind the rotating proxy	yes	yes	5

Use fetch for static HTML, the proxy-* engines when a site blocks datacenter IPs or you need a specific region, and either browser engine when the content only exists after JavaScript runs.

Options

screenshot — PNG of the rendered page, stored in R2 and returned as an expiring URL. Browser engines only (browser, proxy-browser).
rehostImages — downloads the images referenced by the resulting Markdown, re-uploads them to R2 with a TTL, and rewrites the Markdown to point at those copies.
convert — extraction options passed straight through to webforai (extractor preset, frontmatter, baseUrl).

Accounts and pricing

Accounts and API keys live on the dashboard; requests authenticate with Authorization: Bearer wfa_.... Usage is metered in credits with 500 credits per month free — see Billing for the full schedule.

Self-hosting

Nothing here is proprietary. Deploy your own instance on your own Cloudflare account — D1, KV, R2, Workflows and Browser Rendering bindings, plus your own rotating-proxy and Stripe credentials. The setup steps live in the apps/platform README.