> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trainlyai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Core Concepts

> Understand traces, spans, sessions, scores, and versions in Trainly.

# Core Concepts

Trainly's observability model is built on five primitives.

## Traces

A **trace** is a single recorded AI interaction — one input in, one output out. Every trace captures:

| Field         | Description                                            |
| ------------- | ------------------------------------------------------ |
| `input`       | The prompt or user message                             |
| `output`      | The AI response                                        |
| `model`       | Which LLM was used (e.g., `gpt-4o`)                    |
| `latency_ms`  | Response time in milliseconds                          |
| `token_usage` | Prompt and completion token counts                     |
| `cost`        | Estimated cost in USD                                  |
| `tags`        | Labels for filtering (e.g., `["production", "agent"]`) |
| `metadata`    | Arbitrary key-value pairs                              |
| `status`      | `success` or `error`                                   |

```python theme={null}
@client.observe(model="gpt-4o", tags=["production"])
def my_function(input):
    return llm_call(input)
```

## Spans

A **span** is a sub-unit of work within a trace. Use spans to instrument multi-step pipelines — each step gets its own timing, attributes, and output.

```python theme={null}
with client.observe.span("retrieval", kind="retrieval") as s:
    docs = search(query)
    s.set_output(f"Found {len(docs)} documents")
    s.set_attribute("doc_count", len(docs))
```

Span kinds: `chain`, `retrieval`, `tool`, `agent`, `llm`, `embedding`.

## Sessions

A **session** groups multiple traces into a single agent run. All traces within a session share a `session_id` for end-to-end correlation.

```python theme={null}
with client.observe.agent_session() as session_id:
    plan = plan_step(query)       # trace 1
    result = execute(plan)        # trace 2
    answer = synthesize(result)   # trace 3
# All 3 traces linked by session_id
```

## Scores

**Scores** attach evaluation metrics to traces. Use them for human feedback, automated checks, or LLM-as-judge evaluation.

```python theme={null}
# Manual score (0-1 scale)
client.score(trace_id="tr_...", name="relevance", value=0.95)

# LLM judge score
client.score_with_judge(
    scorer_slug="helpfulness",
    trace_id="tr_...",
    input="What is X?",
    output="X is..."
)
```

## Versions

**Versions** tag your pipeline configuration with semver strings. Publish a version, compare performance across versions, and rollback instantly.

```python theme={null}
client.versions.publish("1.2.0", description="Added retry logic")
# Later...
client.versions.rollback(version_id="ver_...")
```

## Projects

A **project** is an isolated environment for traces. Each project has its own API key (`tk_`), project ID (`proj_`), and trace store. Use separate projects for different apps, environments, or teams.