> ## Documentation Index
> Fetch the complete documentation index at: https://deepline.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Cloudflare Crawl: Inputs, Cost & CLI Example

> Crawling websites with browser rendering and extracting content as markdown, HTML, or structured JSON. Includes inputs, outputs, and CLI examples.

## Run in Enrichment Spreadsheet

<Info>
  Use this function as a column step in `deepline enrich`.
</Info>

```bash theme={null}
deepline enrich --input leads.csv --output leads.enriched.csv --with 'result=cloudflare_crawl:{"url":"{{url}}"}' --json
```

<Tip>
  Map payload values to spreadsheet columns with `{{column_name}}` placeholders.
</Tip>

## Input Schema

| Name                    | Type                             | Required | Default        | Description                                                                            |
| ----------------------- | -------------------------------- | -------- | -------------- | -------------------------------------------------------------------------------------- |
| `payload.url`           | `string`                         | Yes      |                | Starting URL to crawl                                                                  |
| `payload.accountId`     | `string`                         | No       |                | Cloudflare account ID. Falls back to CLOUDFLARE\_ACCOUNT\_ID env var                   |
| `payload.limit`         | `integer`                        | No       | `10`           | Max pages to crawl (default 10, max 100,000)                                           |
| `payload.depth`         | `integer`                        | No       |                | Max link depth from starting URL (default/max 100,000)                                 |
| `payload.source`        | `"all" \| "sitemaps" \| "links"` | No       |                | URL discovery source: all (default), sitemaps, or links                                |
| `payload.formats`       | `array`                          | No       | `["markdown"]` | Content formats to return (default \["markdown"]). JSON uses Workers AI for extraction |
| `payload.render`        | `boolean`                        | No       | `true`         | Execute JavaScript via headless browser (default true). false = fast HTML fetch        |
| `payload.jsonOptions`   | `object`                         | No       |                | AI extraction config when formats includes "json"                                      |
| `payload.maxAge`        | `integer`                        | No       |                | Cache TTL in seconds (default 86400, max 604800)                                       |
| `payload.modifiedSince` | `integer`                        | No       |                | Unix timestamp — only crawl pages modified since this time                             |
| `payload.options`       | `object`                         | No       |                | URL filtering options. excludePatterns takes priority over includePatterns             |
| `payload.timeoutMs`     | `integer`                        | No       | `300000`       | Client-side poll timeout in ms (default 300000). Not sent to Cloudflare API            |

### Allowed values

| Field            | Allowed values             |
| ---------------- | -------------------------- |
| `payload.source` | `all`, `sitemaps`, `links` |

<details>
  <summary>Show raw input schema</summary>

  ### Input JSON Schema

  ```json theme={null}
  {
    "type": "object",
    "description": "Cloudflare Browser Rendering crawl input.",
    "properties": {
      "url": {
        "type": "string",
        "description": "Starting URL to crawl"
      },
      "accountId": {
        "type": "string",
        "description": "Cloudflare account ID. Falls back to CLOUDFLARE_ACCOUNT_ID env var"
      },
      "limit": {
        "type": "integer",
        "description": "Max pages to crawl (default 10, max 100,000)",
        "default": 10
      },
      "depth": {
        "type": "integer",
        "description": "Max link depth from starting URL (default/max 100,000)"
      },
      "source": {
        "type": "string",
        "description": "URL discovery source: all (default), sitemaps, or links",
        "enum": [
          "all",
          "sitemaps",
          "links"
        ]
      },
      "formats": {
        "type": "array",
        "description": "Content formats to return (default [\"markdown\"]). JSON uses Workers AI for extraction",
        "default": [
          "markdown"
        ],
        "items": {
          "type": "string",
          "enum": [
            "html",
            "markdown",
            "json"
          ]
        }
      },
      "render": {
        "type": "boolean",
        "description": "Execute JavaScript via headless browser (default true). false = fast HTML fetch",
        "default": true
      },
      "jsonOptions": {
        "type": "object",
        "description": "AI extraction config when formats includes \"json\"",
        "properties": {
          "prompt": {
            "type": "string"
          },
          "response_format": {},
          "custom_ai": {}
        },
        "additionalProperties": false
      },
      "maxAge": {
        "type": "integer",
        "description": "Cache TTL in seconds (default 86400, max 604800)"
      },
      "modifiedSince": {
        "type": "integer",
        "description": "Unix timestamp — only crawl pages modified since this time"
      },
      "options": {
        "type": "object",
        "description": "URL filtering options. excludePatterns takes priority over includePatterns",
        "properties": {
          "includePatterns": {
            "type": "array",
            "items": {
              "type": "string"
            }
          },
          "excludePatterns": {
            "type": "array",
            "items": {
              "type": "string"
            }
          },
          "includeExternalLinks": {
            "type": "boolean"
          },
          "includeSubdomains": {
            "type": "boolean"
          }
        },
        "additionalProperties": false
      },
      "timeoutMs": {
        "type": "integer",
        "description": "Client-side poll timeout in ms (default 300000). Not sent to Cloudflare API",
        "default": 300000
      }
    },
    "required": [
      "url"
    ],
    "additionalProperties": false
  }
  ```
</details>

## Output Schema

| Name          | Type     | Required | Default | Description                                    |
| ------------- | -------- | -------- | ------- | ---------------------------------------------- |
| `result.data` | `object` | Yes      |         | Provider response payload.                     |
| `result.meta` | `record` | No       |         | Additional response metadata (status, paging). |

<details>
  <summary>Show raw output schema</summary>

  ### Output JSON Schema

  ```json theme={null}
  {
    "type": "object",
    "description": "Cloudflare Browser Rendering crawl result.",
    "properties": {
      "data": {
        "type": "object",
        "description": "Provider response payload.",
        "properties": {
          "jobId": {
            "type": "string"
          },
          "status": {
            "type": "string"
          },
          "browserSecondsUsed": {
            "type": "number"
          },
          "total": {
            "type": "number"
          },
          "finished": {
            "type": "number"
          },
          "records": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "url": {
                  "type": "string"
                },
                "status": {
                  "type": "string"
                },
                "markdown": {
                  "type": "string"
                },
                "html": {
                  "type": "string"
                },
                "json": {},
                "metadata": {
                  "type": "object",
                  "additionalProperties": {}
                }
              },
              "required": [
                "url"
              ],
              "additionalProperties": false
            }
          },
          "cursor": {
            "anyOf": [
              {
                "type": "number"
              },
              {
                "type": "string"
              },
              {}
            ]
          },
          "timedOut": {
            "type": "boolean"
          }
        },
        "required": [
          "jobId",
          "status",
          "browserSecondsUsed",
          "total",
          "finished",
          "records",
          "cursor",
          "timedOut"
        ],
        "additionalProperties": false
      },
      "meta": {
        "type": "object",
        "description": "Additional response metadata (status, paging).",
        "additionalProperties": {}
      }
    },
    "required": [
      "data"
    ],
    "additionalProperties": false
  }
  ```
</details>

## Advanced: Direct CLI

<Info>
  Use direct execution for single payload debugging.
</Info>

```bash theme={null}
deepline tools execute cloudflare_crawl --payload '{
  "url": "string"
}' --json
```

### CLI flags

| Flag                        | Description                                         |
| --------------------------- | --------------------------------------------------- |
| `--json`                    | Print machine-readable output.                      |
| `--wait`                    | Wait for terminal provider status when supported.   |
| `--debug`                   | Enable wait mode with additional status/log output. |
| `--wait-timeout SECONDS`    | Max seconds to wait in wait mode.                   |
| `--poll-interval SECONDS`   | Polling interval in seconds during wait mode.       |
| `--timeout SECONDS`         | Request timeout in seconds.                         |
| `--connect-timeout SECONDS` | Connection timeout in seconds.                      |

## Cost

* Pricing model: `provider_usage` (provider usage).
* Estimated Deepline credits: `0.01` per pricing unit.
* Billing mode: `post_deduct`.