Crawlbase
/crawlbase
Crawlbase (formerly ProxyCrawl) is an all-in-one platform for scraping and large-scale crawling, with a smart proxy layer that handles rotation, rendering and anti-bot defenses behind the scenes. Alongside its scraping API it offers ready-made extractors for popular sites, clean markdown output, and a managed crawl queue that takes on the heavy lifting of broad, asynchronous data collection. Its pay-for-success model — you’re billed only for results actually delivered — makes it an economical workhorse for production data pipelines.
Call Crawlbase through the gateway
Same paths, parameters and responses as Crawlbase's native API — auxiliar injects the upstream key server-side and debits the provider's metered cost from your credits.
curl https://api.auxiliar.ai/crawlbase/<native-path> \ -H "Authorization: Bearer $AUXILIAR_API_KEY"
Exact endpoints and parameters: docs.auxiliar.ai/crawlbase. An empty balance returns HTTP 402 — no silent degradation; see how billing behaves.
Choose if
You want a managed crawl queue and pay-for-success by domain complexity.
Avoid if
You want predictable behavior — two distinct tokens (Normal vs JS) and a polymorphic body field.
Evals
How Crawlbase performed on the curator-fleet web-access evals — the real measured numbers behind each composite, ranked best-first. Every provider runs the identical task corpus; each scorecard carries its run date. Latest run: 2026-06-22.