> ## Documentation Index > Fetch the complete documentation index at: https://deepline.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Firecrawl Batch Scrape: Inputs, Cost & CLI Example > Scraping a known list of URLs in parallel. Includes inputs, outputs, pricing notes, Deepline CLI examples, and GTM automation guidance. ## Run in Enrichment Spreadsheet Use this function as a column step in `deepline enrich`. ```bash theme={null} deepline enrich --input leads.csv --output leads.enriched.csv --with 'result=firecrawl_batch_scrape:{"urls":"{{urls}}"}' --json ``` Map payload values to spreadsheet columns with `{{column_name}}` placeholders. ## Input Schema | Name | Type | Required | Default | Description | | --------------------------- | --------- | -------- | ------- | ------------------------------------------------------------------------------------------------------ | | `payload.urls` | `array` | Yes | | URLs to scrape | | `payload.scrapeOptions` | `object` | No | | Scrape options applied to all URLs | | `payload.webhook` | `object` | No | | Webhook for completion notification | | `payload.maxConcurrency` | `integer` | No | | Max concurrent scrapes | | `payload.ignoreInvalidURLs` | `boolean` | No | | Skip invalid URLs instead of failing (default true) | | `payload.zeroDataRetention` | `boolean` | No | | Do not store any data (default false) | | `payload.timeoutMs` | `integer` | No | | Poll timeout in ms (default 30000; larger values are clamped to the Deepline async sync-wait ceiling). |
Show raw input schema ### Input JSON Schema ```json theme={null} { "type": "object", "description": "Scrape multiple URLs in batch (async with polling).", "properties": { "urls": { "type": "array", "description": "URLs to scrape", "items": { "type": "string" } }, "scrapeOptions": { "type": "object", "description": "Scrape options applied to all URLs", "properties": { "formats": { "type": "array", "description": "Output formats (default [{\"type\":\"markdown\"}])", "items": { "anyOf": [ { "type": "string", "enum": [ "markdown", "summary", "html", "rawHtml", "links", "images", "screenshot", "json", "changeTracking", "branding" ] }, { "type": "object", "properties": { "type": { "type": "string", "const": "markdown" } }, "required": [ "type" ], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "const": "summary" } }, "required": [ "type" ], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "const": "html" } }, "required": [ "type" ], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "const": "rawHtml" } }, "required": [ "type" ], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "const": "links" } }, "required": [ "type" ], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "const": "images" } }, "required": [ "type" ], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "const": "screenshot" }, "fullPage": { "type": "boolean", "description": "Capture full page screenshot" }, "quality": { "type": "integer", "description": "Screenshot quality 0-100" }, "viewport": { "type": "object", "description": "Custom viewport size", "properties": { "width": { "type": "integer" }, "height": { "type": "integer" } }, "additionalProperties": false } }, "required": [ "type" ], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "const": "json" }, "schema": { "type": "object", "description": "JSON schema for structured output", "additionalProperties": {} }, "prompt": { "type": "string", "description": "Natural language extraction instructions" } }, "required": [ "type" ], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "const": "changeTracking" }, "modes": { "type": "array", "description": "Change tracking modes", "items": { "type": "string", "enum": [ "git-diff", "json" ] } }, "schema": { "type": "object", "description": "Schema for tracked changes", "additionalProperties": {} }, "prompt": { "type": "string", "description": "Instructions for change tracking" }, "tag": { "type": "string", "description": "Tag for this change tracking snapshot" } }, "required": [ "type" ], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "const": "branding" } }, "required": [ "type" ], "additionalProperties": false } ] } }, "onlyMainContent": { "type": "boolean", "description": "Only return main content, filtering navs/footers" }, "includeTags": { "type": "array", "description": "Only include these HTML tags", "items": { "type": "string" } }, "excludeTags": { "type": "array", "description": "Exclude these HTML tags", "items": { "type": "string" } }, "headers": { "type": "object", "description": "Custom HTTP headers", "additionalProperties": { "type": "string" } }, "waitFor": { "type": "number", "description": "Wait ms after page load before scraping" }, "mobile": { "type": "boolean", "description": "Use mobile user agent" }, "timeout": { "type": "number", "description": "Request timeout in ms (default 30000, max 300000)" }, "maxAge": { "type": "integer", "description": "Max cache age in ms (default 172800000)" }, "storeInCache": { "type": "boolean", "description": "Store result in cache (default true)" }, "parsers": { "type": "array", "description": "Document parsers (e.g. PDF)", "items": { "type": "object", "properties": { "type": { "type": "string", "description": "Parser type", "const": "pdf" }, "mode": { "type": "string", "description": "PDF parsing mode", "enum": [ "fast", "auto", "ocr" ] }, "maxPages": { "type": "integer", "description": "Max pages to parse (1-10000)" } }, "required": [ "type" ], "additionalProperties": false } }, "actions": { "type": "array", "description": "Browser actions to execute before scraping", "items": { "type": "object", "properties": { "type": { "type": "string", "description": "Browser action type", "enum": [ "wait", "click", "write", "press", "screenshot", "scroll", "pdf", "scrape", "executeJavascript" ] }, "selector": { "type": "string", "description": "CSS selector for the element" }, "milliseconds": { "type": "number", "description": "Wait time in milliseconds" }, "text": { "type": "string", "description": "Text to write" }, "key": { "type": "string", "description": "Key to press" }, "direction": { "type": "string", "description": "Scroll direction", "enum": [ "up", "down" ] }, "amount": { "type": "number", "description": "Scroll amount in pixels" }, "script": { "type": "string", "description": "JavaScript to execute" }, "fullPage": { "type": "boolean", "description": "Full-page screenshot" }, "format": { "type": "string", "description": "PDF page format", "enum": [ "A0", "A1", "A2", "A3", "A4", "A5", "A6", "Letter", "Legal", "Tabloid", "Ledger" ] }, "landscape": { "type": "boolean", "description": "PDF landscape orientation" }, "scale": { "type": "number", "description": "PDF scale factor (0.1-2)" } }, "required": [ "type" ], "additionalProperties": false } }, "location": { "type": "object", "description": "Geographic location settings", "properties": { "country": { "type": "string", "description": "Country code (e.g. \"US\")" }, "languages": { "type": "array", "description": "Language codes", "items": { "type": "string" } } }, "additionalProperties": false }, "proxy": { "type": "string", "description": "Proxy mode", "enum": [ "basic", "enhanced", "auto" ] }, "blockAds": { "type": "boolean", "description": "Block ads during scraping" }, "removeBase64Images": { "type": "boolean", "description": "Strip base64 images from output" }, "skipTlsVerification": { "type": "boolean", "description": "Skip TLS certificate verification" } }, "additionalProperties": false }, "webhook": { "type": "object", "description": "Webhook for completion notification", "properties": { "url": { "type": "string", "description": "Webhook URL for notifications" }, "headers": { "type": "object", "description": "Custom headers for webhook requests", "additionalProperties": { "type": "string" } }, "metadata": { "type": "object", "description": "Metadata to include in webhook payloads", "additionalProperties": {} }, "events": { "type": "array", "description": "Events to trigger webhook (default all)", "items": { "type": "string", "enum": [ "completed", "page", "failed", "started" ] } } }, "required": [ "url" ], "additionalProperties": false }, "maxConcurrency": { "type": "integer", "description": "Max concurrent scrapes" }, "ignoreInvalidURLs": { "type": "boolean", "description": "Skip invalid URLs instead of failing (default true)" }, "zeroDataRetention": { "type": "boolean", "description": "Do not store any data (default false)" }, "timeoutMs": { "type": "integer", "description": "Poll timeout in ms (default 30000; larger values are clamped to the Deepline async sync-wait ceiling)." } }, "required": [ "urls" ], "additionalProperties": false } ```
## Output Schema | Name | Type | Required | Default | Description | | ------------- | -------- | -------- | ------- | ---------------------------------------------- | | `result.data` | `object` | Yes | | Provider response payload. | | `result.meta` | `record` | No | | Additional response metadata (status, paging). |
Show raw output schema ### Output JSON Schema ```json theme={null} { "type": "object", "description": "Batch scrape job result.", "properties": { "data": { "type": "object", "description": "Provider response payload.", "properties": { "status": { "type": "string" }, "total": { "type": "number" }, "completed": { "type": "number" }, "data": { "type": "array", "items": { "type": "object", "additionalProperties": {} } } }, "required": [ "status" ], "additionalProperties": false }, "meta": { "type": "object", "description": "Additional response metadata (status, paging).", "additionalProperties": {} } }, "required": [ "data" ], "additionalProperties": false } ```
## Advanced: Direct CLI Use direct execution for single payload debugging. ```bash theme={null} deepline tools execute firecrawl_batch_scrape --payload '{ "urls": "array" }' --json ``` ### CLI flags | Flag | Description | | --------------------------- | --------------------------------------------------- | | `--json` | Print machine-readable output. | | `--wait` | Wait for terminal provider status when supported. | | `--debug` | Enable wait mode with additional status/log output. | | `--wait-timeout SECONDS` | Max seconds to wait in wait mode. | | `--poll-interval SECONDS` | Polling interval in seconds during wait mode. | | `--timeout SECONDS` | Request timeout in seconds. | | `--connect-timeout SECONDS` | Connection timeout in seconds. | ## Provider API Context
Show provider reference (optional) ```md theme={null} ```
## Cost * Pricing model: `per_page` (per page). * Estimated Deepline credits: `0.02` per pricing unit. * Billing mode: `post_deduct`.