Scoring API

Score a Trace

curl -X POST https://api.trainlyai.com/v1/{project_id}/traces/trc_9f8e7d6c5b4a/scores \
  -H "Authorization: Bearer tk_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "accuracy",
    "value": 0.95,
    "comment": "Correct translation with natural phrasing."
  }'

POST /v1/{project_id}/traces/{trace_id}/scores

Request Body

Parameter	Type	Required	Description
`name`	string	Yes	Score name (e.g. `"accuracy"`, `"relevance"`, `"toxicity"`)
`value`	float	Yes	Score between `0` and `1`
`comment`	string	No	Human-readable justification

Response

{
  "success": true
}

LLM Judge Scoring

Run an automated evaluation using a built-in or custom LLM judge scorer.

curl -X POST https://api.trainlyai.com/v1/{project_id}/score-with-judge \
  -H "Authorization: Bearer tk_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "scorer_slug": "correctness",
    "trace_id": "trc_9f8e7d6c5b4a",
    "input": "Translate to French: Hello world",
    "output": "Bonjour le monde",
    "expected_output": "Bonjour le monde"
  }'

POST /v1/{project_id}/score-with-judge

Request Body

Parameter	Type	Required	Description
`scorer_slug`	string	Yes	Identifier for the scorer (e.g. `"correctness"`, `"faithfulness"`, `"toxicity"`)
`trace_id`	string	No	Link the score to an existing trace
`input`	string	No	The original input/prompt
`output`	string	No	The model output to evaluate
`expected_output`	string	No	Ground-truth expected output for comparison

Response

{
  "score": 0.98,
  "reasoning": "The output is an accurate and natural French translation of the input. It matches the expected output exactly.",
  "scorer": "correctness"
}

Overview

APIs

Score a Trace

Request Body

Response

LLM Judge Scoring

Request Body

Response

​Score a Trace

​Request Body

​Response

​LLM Judge Scoring

​Request Body

​Response

Score a Trace

Request Body

Response

LLM Judge Scoring

Request Body

Response