# Run and rank

Source: https://redcrown.ai/docs/guides/run-and-rank/

Run an experiment spec and read the ranked report.

## 1. Run

Execute a saved experiment spec against all candidate models. Raise concurrency for large token benchmarks to finish faster.

**UI**

Open **Experiments** under the Advanced menu. Select a saved experiment and click **Run**. The ranked report appears when the run completes.

**CLI**

```
redcrown eval exp.json --concurrency 10
```

Raising `--concurrency` runs candidates in parallel. Useful for many-candidate token benchmarks where sequential execution would take too long.

**MCP**

Create then run an experiment in two calls:

```
{
  "experiment_id": "..."
}
```

First call `create_experiment` with your spec, then call `run_experiment` with the returned `experiment_id`.

## 2. Reproducibility

Run a candidate multiple times to measure run-to-run quality variance. The report includes a mean and standard deviation confidence band.

**UI**

Not available via the UI. Use the CLI or MCP for repeat runs.

**CLI**

```
redcrown eval exp.json --repeat 5
```

Each candidate runs 5 times. The report shows the mean quality score and run-to-run standard deviation. Note that `--repeat 5` multiplies spend by 5.

**MCP**

Set `repeat_n` in the experiment definition when calling `create_experiment`. The resulting report includes the mean and standard deviation for each candidate.

## 3. Read the winner

The ranked report identifies the cheapest candidate that clears your quality bar. Every other candidate is ranked below it with its score and cost.

**UI**

The ranked report highlights the winner row in green. The table shows quality score, cost, latency, and savings vs. the incumbent for each candidate.

**CLI**

The ranked report prints to the terminal at the end of the run. Add `--report-json` to save it:

```
redcrown eval exp.json --report-json out.json
```

**MCP**

Call `get_report` with the run id to fetch the full ranked report:

```
{
  "run_id": "..."
}
```

Last verified 2026-07-02.