> ## Documentation Index
> Fetch the complete documentation index at: https://deepline.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Vercel AI Gateway: Ai Evaluate

> Typed evaluation of shared state with Vercel AI SDK evaluation models such as TypeSafe Jev. Includes inputs, outputs, SDK V2 examples, and Deepline cost.

Typed evaluation of shared state with Vercel AI SDK evaluation models such as TypeSafe Jev.

<Info>
  Tool ID: `ai_evaluate`
</Info>

## Run this action

Use the TypeScript SDK for a single call. Put `ctx.tools.execute(...)` inside a Play when the call should be durable, scheduled, or run across a CSV.

```ts theme={null}
import { Deepline } from 'deepline';

const deepline = await Deepline.connect();
const result = await deepline.tools.execute(
  'ai_evaluate',
  {
    "state": "California",
    "questions": {}
  },
);

console.log(result.toolResponse.raw);
```

### CLI

```bash theme={null}
deepline tools execute ai_evaluate --input '{
  "state": "California",
  "questions": {}
}' --json
```

## Example response

The SDK exposes this shape at `result.toolResponse.raw`. Values below are representative.

```json theme={null}
{
  "status": "completed",
  "result": {
    "answers": {},
    "response": {
      "modelId": "model_123",
      "timestamp": "example"
    }
  },
  "usage": {
    "inputTokens": 123,
    "outputTokens": 123,
    "totalTokens": 123
  },
  "output": "example",
  "extracted_json": {}
}
```

Use `deepline tools get ai_evaluate --json` for the latest machine-readable contract.

## Input reference

| Name                      | Type      | Required | Default             | Details                                                                                                                                         |
| ------------------------- | --------- | -------- | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `payload.model`           | `string`  | No       | `"typesafe-ai/jev"` | Provider-prefixed Vercel AI Gateway evaluation model id, for example "typesafe-ai/jev". Pattern: `^[a-z0-9][a-z0-9._-]*/[a-z0-9][a-z0-9._-]*$`. |
| `payload.state`           | `unknown` | Yes      | —                   | —                                                                                                                                               |
| `payload.questions`       | `record`  | Yes      | —                   | —                                                                                                                                               |
| `payload.providerOptions` | `record`  | No       | —                   | AI SDK providerOptions passthrough, namespaced by provider.                                                                                     |

<details>
  <summary>Show raw input schema</summary>

  ### Input JSON Schema

  ```json theme={null}
  {
    "type": "object",
    "properties": {
      "model": {
        "type": "string",
        "description": "Provider-prefixed Vercel AI Gateway evaluation model id, for example \"typesafe-ai/jev\".",
        "default": "typesafe-ai/jev",
        "pattern": "^[a-z0-9][a-z0-9._-]*/[a-z0-9][a-z0-9._-]*$"
      },
      "state": {},
      "questions": {
        "type": "object",
        "additionalProperties": {
          "type": "object",
          "properties": {
            "type": {
              "type": "string",
              "enum": [
                "choice",
                "score",
                "boolean"
              ]
            },
            "instructions": {},
            "criteria": {}
          },
          "required": [
            "type",
            "instructions"
          ],
          "additionalProperties": false
        }
      },
      "providerOptions": {
        "type": "object",
        "description": "AI SDK providerOptions passthrough, namespaced by provider.",
        "additionalProperties": {}
      }
    },
    "required": [
      "state",
      "questions"
    ],
    "additionalProperties": false
  }
  ```
</details>

## Output reference

| Name                               | Type          | Required | Default | Details |
| ---------------------------------- | ------------- | -------- | ------- | ------- |
| `result.status`                    | `"completed"` | Yes      | —       | —       |
| `result.result`                    | `object`      | Yes      | —       | —       |
| `result.result.answers`            | `record`      | Yes      | —       | —       |
| `result.result.rounding`           | `record`      | No       | —       | —       |
| `result.result.response`           | `object`      | Yes      | —       | —       |
| `result.result.response.modelId`   | `string`      | Yes      | —       | —       |
| `result.result.response.timestamp` | `string`      | Yes      | —       | —       |
| `result.usage`                     | `object`      | Yes      | —       | —       |
| `result.usage.inputTokens`         | `integer`     | Yes      | —       | —       |
| `result.usage.outputTokens`        | `integer`     | Yes      | —       | —       |
| `result.usage.totalTokens`         | `integer`     | Yes      | —       | —       |
| `result.meta`                      | `object`      | No       | —       | —       |
| `result.meta.model`                | `string`      | Yes      | —       | —       |
| `result.output`                    | `string`      | Yes      | —       | —       |
| `result.extracted_json`            | `record`      | Yes      | —       | —       |

<details>
  <summary>Show raw output schema</summary>

  ### Output JSON Schema

  ```json theme={null}
  {
    "type": "object",
    "properties": {
      "status": {
        "type": "string",
        "const": "completed"
      },
      "result": {
        "type": "object",
        "properties": {
          "answers": {
            "type": "object",
            "additionalProperties": {
              "type": "object",
              "additionalProperties": {}
            }
          },
          "rounding": {
            "type": "object",
            "additionalProperties": {
              "type": "integer"
            }
          },
          "response": {
            "type": "object",
            "properties": {
              "modelId": {
                "type": "string"
              },
              "timestamp": {
                "type": "string"
              }
            },
            "required": [
              "modelId",
              "timestamp"
            ],
            "additionalProperties": false
          }
        },
        "required": [
          "answers",
          "response"
        ],
        "additionalProperties": false
      },
      "usage": {
        "type": "object",
        "properties": {
          "inputTokens": {
            "type": "integer"
          },
          "outputTokens": {
            "type": "integer"
          },
          "totalTokens": {
            "type": "integer"
          }
        },
        "required": [
          "inputTokens",
          "outputTokens",
          "totalTokens"
        ],
        "additionalProperties": false
      },
      "meta": {
        "type": "object",
        "properties": {
          "model": {
            "type": "string"
          }
        },
        "required": [
          "model"
        ],
        "additionalProperties": false
      },
      "output": {
        "type": "string"
      },
      "extracted_json": {
        "type": "object",
        "additionalProperties": {}
      }
    },
    "required": [
      "status",
      "result",
      "usage",
      "output",
      "extracted_json"
    ],
    "additionalProperties": false
  }
  ```
</details>

## Deepline cost

* Pricing model: `provider_usage` (provider usage).
* Estimated Deepline credits: `0` per pricing unit.

## Related documentation

* [Vercel AI Gateway provider guide](/docs/providers/deeplineagent/guide)
* [SDK V2 quickstart](/docs/sdk-v2/quickstart)
* [SDK reference](/docs/sdk-v2/sdk-reference)
* [Run tools across a CSV](/docs/sdk-v2/batch-csv)