> ## Documentation Index
> Fetch the complete documentation index at: https://docs.halios.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Run your first eval

> How to set up and run an evaluation suite for your AI agent using your coding agent.

This guide walks you through setting up an evaluation suite for your AI agent. Your coding agent acts as the operator, while Halios serves as the evaluation runtime underneath.

***

## 1. Switch to your agent repository root

Ensure your IDE or terminal workspace is open at the root of your agent repository.

***

## 2. Prompt your coding agent

You can let your coding agent install the skill automatically, or install it manually first:

### Option A: All-in-one prompt

Ask your coding agent to install the skill and set up evaluations in one step:

```text theme={null}
Install Halios skill from github.com/HaliosAI/halios and set up evals for this agent.
```

### Option B: Install manually first

Add the skill to your coding agent:

```bash theme={null}
npx skills add HaliosAI/halios --skill halios
```

Then prompt your coding agent:

```text theme={null}
Set up evals for this agent using halios.
```

***

## 3. What happens under the hood

> Depending on your coding agent's permissions and configuration, it may ask for confirmation before making changes or running commands. You remain in control of what gets created and executed.

<Steps>
  <Step title="Inspect repository">
    The coding agent examines your codebase to understand the agent entrypoint, tools, system prompts, and critical user workflows.
  </Step>

  <Step title="Connect Halios">
    On first use, your coding agent helps authenticate your workspace with Halios. The skill and CLI handle communication with the Halios runtime.
  </Step>

  <Step title="Create evaluation artifacts">
    The coding agent creates test scenarios, checks, and configuration files (`.halios/`) alongside your application.
  </Step>

  <Step title="Configure telemetry">
    Halios uses OpenTelemetry-compatible traces. Existing instrumentation is reused, or your coding agent helps add standard auto-instrumentation (such as OpenLLMetry / Traceloop).
  </Step>

  <Step title="Simulate user interactions with agent">
    Halios generates multi-turn conversations based on the evaluation scenarios to produce realistic traces of agent-user interaction patterns. The coding agent creates a lightweight adapter so Halios can communicate with your agent during simulations.
  </Step>

  <Step title="Run evaluations">
    Halios scores the generated traces against your checks (deterministic rules, LLM judges, and reliability thresholds).
  </Step>
</Steps>

***

## 4. Review results

Your coding agent can summarize the run directly in your chat, or you can open the Halios dashboard to inspect:

* **Scenario results and pass rates** across repeated trials
* **Full multi-turn traces** with user and assistant turns
* **Tool calls, arguments, and return values**
* **Check evaluations and failure explanations**

***

<CardGroup cols={2}>
  <Card title="Prompting guide" icon="messages-square" href="/prompts/prompting-guide">
    Learn how to direct your coding agent to build and maintain your eval suite.
  </Card>

  <Card title="Evaluation concepts" icon="boxes" href="/concepts/evaluation-concepts">
    Explore the core mental model and key evaluation terms.
  </Card>
</CardGroup>