> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trainlyai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Testing API

> Create test suites, add test cases, run evaluations, and retrieve results.

## Create a Test Suite

<CodeGroup>
  ```bash cURL theme={null}
  curl -X POST https://api.trainlyai.com/v1/{project_id}/testing/suites \
    -H "Authorization: Bearer tk_your_api_key" \
    -H "Content-Type: application/json" \
    -d '{
      "name": "Translation Quality",
      "description": "Verify translation accuracy across languages.",
      "tags": ["translation", "regression"]
    }'
  ```
</CodeGroup>

`POST /v1/{project_id}/testing/suites`

### Request Body

| Parameter     | Type          | Required | Description                          |
| ------------- | ------------- | -------- | ------------------------------------ |
| `name`        | string        | Yes      | Suite name                           |
| `description` | string        | No       | Description of what this suite tests |
| `tags`        | list\[string] | No       | Tags for organizing suites           |

### Response

```json theme={null}
{
  "suite_id": "ste_a1b2c3d4",
  "name": "Translation Quality",
  "created_at": "2026-04-07T14:00:00Z"
}
```

***

## List Test Suites

<CodeGroup>
  ```bash cURL theme={null}
  curl "https://api.trainlyai.com/v1/{project_id}/testing/suites" \
    -H "Authorization: Bearer tk_your_api_key"
  ```
</CodeGroup>

`GET /v1/{project_id}/testing/suites`

### Response

```json theme={null}
[
  {
    "suite_id": "ste_a1b2c3d4",
    "name": "Translation Quality",
    "description": "Verify translation accuracy across languages.",
    "tags": ["translation", "regression"],
    "case_count": 12,
    "created_at": "2026-04-07T14:00:00Z"
  }
]
```

***

## Add a Test Case

<CodeGroup>
  ```bash cURL theme={null}
  curl -X POST https://api.trainlyai.com/v1/{project_id}/testing/suites/ste_a1b2c3d4/cases \
    -H "Authorization: Bearer tk_your_api_key" \
    -H "Content-Type: application/json" \
    -d '{
      "query": "Translate to French: Good morning",
      "expected_answer": "Bonjour",
      "category": "french",
      "assertions": [
        { "type": "contains", "value": "Bonjour" },
        { "type": "max_latency_ms", "value": 2000 }
      ]
    }'
  ```
</CodeGroup>

`POST /v1/{project_id}/testing/suites/{suite_id}/cases`

### Request Body

| Parameter         | Type          | Required | Description                        |
| ----------------- | ------------- | -------- | ---------------------------------- |
| `query`           | string        | Yes      | Input prompt for the test case     |
| `expected_answer` | string        | Yes      | Ground-truth expected output       |
| `category`        | string        | No       | Category label for grouping cases  |
| `assertions`      | list\[object] | No       | Assertions to evaluate (see below) |

### Assertion Types

| Type             | Value  | Description                              |
| ---------------- | ------ | ---------------------------------------- |
| `contains`       | string | Output must contain the value            |
| `not_contains`   | string | Output must not contain the value        |
| `equals`         | string | Output must exactly match the value      |
| `max_latency_ms` | number | Response must complete within this time  |
| `min_score`      | number | LLM judge score must meet this threshold |

### Response

```json theme={null}
{
  "case_id": "tsc_x9y8z7",
  "suite_id": "ste_a1b2c3d4",
  "created_at": "2026-04-07T14:05:00Z"
}
```

***

## Run a Test Suite

<CodeGroup>
  ```bash cURL theme={null}
  curl -X POST https://api.trainlyai.com/v1/{project_id}/testing/suites/ste_a1b2c3d4/run \
    -H "Authorization: Bearer tk_your_api_key"
  ```
</CodeGroup>

`POST /v1/{project_id}/testing/suites/{suite_id}/run`

### Response

```json theme={null}
{
  "run_id": "run_m3n4o5p6",
  "suite_id": "ste_a1b2c3d4",
  "status": "running",
  "started_at": "2026-04-07T14:10:00Z"
}
```

***

## Get Run Results

<CodeGroup>
  ```bash cURL theme={null}
  curl "https://api.trainlyai.com/v1/{project_id}/testing/runs/run_m3n4o5p6/results" \
    -H "Authorization: Bearer tk_your_api_key"
  ```
</CodeGroup>

`GET /v1/{project_id}/testing/runs/{run_id}/results`

### Response

```json theme={null}
{
  "run_id": "run_m3n4o5p6",
  "suite_id": "ste_a1b2c3d4",
  "status": "completed",
  "passed": 11,
  "failed": 1,
  "total": 12,
  "pass_rate": 0.9167,
  "results": [
    {
      "case_id": "tsc_x9y8z7",
      "query": "Translate to French: Good morning",
      "expected_answer": "Bonjour",
      "actual_output": "Bonjour",
      "passed": true,
      "latency_ms": 580.2,
      "assertions": [
        { "type": "contains", "value": "Bonjour", "passed": true },
        { "type": "max_latency_ms", "value": 2000, "passed": true }
      ]
    }
  ],
  "completed_at": "2026-04-07T14:10:47Z"
}
```