> ## Documentation Index
> Fetch the complete documentation index at: https://docs.halios.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluation concepts

> Key terms and mental models used in Halios evaluations.

The core evaluation flow in Halios follows a simple progression:

```text theme={null}
Scenario → fresh trials → traces → checks → results
```

A **scenario** is the reusable test. A **trace** is the execution evidence from one trial. Running the scenario again produces a fresh trace against the current version of your agent.

***

## Core terms

### Scenario

The reusable test situation you want to evaluate. It defines the initial user goal, persona, constraints, and multi-turn conversation arc.

### Trial

One fresh execution generated from a scenario. Instead of replaying a saved transcript, Halios simulates a live user interaction with your agent for each trial.

### Trace

The execution evidence emitted by your agent during a trial. Contains the full conversational turn history, model calls, tool arguments, retrieval context, and latency metrics.

### Check

One specific invariant you want to verify about your agent's behavior (e.g. answer groundedness, regex format match, policy compliance, or tool argument schema).

### Eval suite

The collection of scenarios, checks, and configuration defined for an agent or workflow (stored in your repository under `.halios/`).

### Run

One complete execution pass of an eval suite against the current agent implementation, recording trial traces, check evaluations, and pass rates.

### pass\@k

Repeating the same scenario $k$ times (e.g. 3 or 5 trials) to measure consistency. A scenario passes only if all $k$ trials succeed, ensuring that passing was not just random chance.

***

<CardGroup cols={2}>
  <Card title="Run your first eval" icon="play" href="/getting-started/run-your-first-eval">
    Get started with your coding agent in minutes.
  </Card>

  <Card title="CLI reference" icon="square-terminal" href="/reference/cli">
    Command reference for automation and CI pipelines.
  </Card>
</CardGroup>
