Skip to main content

​
Score a Trace

curl -X POST https://api.trainlyai.com/v1/{project_id}/traces/trc_9f8e7d6c5b4a/scores \
  -H "Authorization: Bearer tk_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "accuracy",
    "value": 0.95,
    "comment": "Correct translation with natural phrasing."
  }'
POST /v1/{project_id}/traces/{trace_id}/scores

​
Request Body

ParameterTypeRequiredDescription
namestringYesScore name (e.g. "accuracy", "relevance", "toxicity")
valuefloatYesScore between 0 and 1
commentstringNoHuman-readable justification

​
Response

{
  "success": true
}

​
LLM Judge Scoring

Run an automated evaluation using a built-in or custom LLM judge scorer. 
curl -X POST https://api.trainlyai.com/v1/{project_id}/score-with-judge \
  -H "Authorization: Bearer tk_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "scorer_slug": "correctness",
    "trace_id": "trc_9f8e7d6c5b4a",
    "input": "Translate to French: Hello world",
    "output": "Bonjour le monde",
    "expected_output": "Bonjour le monde"
  }'
POST /v1/{project_id}/score-with-judge

​
Request Body

ParameterTypeRequiredDescription
scorer_slugstringYesIdentifier for the scorer (e.g. "correctness", "faithfulness", "toxicity")
trace_idstringNoLink the score to an existing trace
inputstringNoThe original input/prompt
outputstringNoThe model output to evaluate
expected_outputstringNoGround-truth expected output for comparison

​
Response

{
  "score": 0.98,
  "reasoning": "The output is an accurate and natural French translation of the input. It matches the expected output exactly.",
  "scorer": "correctness"
}