# Concepts

Source: https://redcrown.ai/docs/concepts/

The objects RedCrown works with, and what "prove" means.

## Prove, not deploy

RedCrown fans one task across many models and configurations, scores the outputs against your own ground truth on cost, quality, and latency, and names the cheapest configuration that clears your quality bar. The verb is prove.

## The object model

- **Experiment** , a task definition: the objective, the quality metric and bar, the candidate models, and the dataset.
- **Run** , one execution of an experiment that produces a ranked report.
- **Report** , the ranked result: per-candidate cost, quality, latency percentiles, and the winner.
- **Receipts** , per-item evidence (the input, each candidate's output, the score) behind the ranking.
- **Provenance** , who ran it, when, the dataset hash, the model versions, and the methodology version.
- **Proof link** , a no-login page that shares a run's report and receipts.
- **Attestation** , a downloadable, cryptographically signed, tamper-evident version of the proof (see [Signed attestation](../guides/signed-attestation/)).

## How the surfaces relate

The same objects are reachable from the app UI, the `redcrown` CLI, and the MCP tools. See the [CLI](../reference/cli/), [MCP](../reference/mcp/), and [UI](../reference/ui/) reference for the full mapping.

Last verified 2026-07-02.

---

# Authentication

Source: https://redcrown.ai/docs/guides/auth/

Sign in from the app, the CLI, and agents.

## 1. App

Sign in to the web app with your email and password. The app maintains your session in the browser automatically.

**UI**

Go to **Login** at `/login`. Enter your email and password and click **Sign in**. If you do not have an account yet, click **Sign up** and check your inbox for a confirmation link before signing in.

**CLI**

Not available via the CLI. Use the UI to sign in to the app, or see the CLI device login step below.

**MCP**

On the first tool call from a connected agent, RedCrown triggers OAuth browser consent. Your agent's MCP client will open a browser page where you approve access. The token is stored and reused for subsequent calls from that client.

## 2. CLI device login

The CLI uses an OAuth device flow so you can authorize from any browser, even if the CLI runs on a remote machine.

**UI**

Not available via the UI. The CLI logs in from your terminal.

**CLI**

Run `redcrown login` to start a device-flow session. The CLI prints a short code and opens a verification page in your browser. Approve access there and the CLI stores credentials at `~/.redcrown/credentials.json`.

```
redcrown login
```

To sign out and remove stored credentials:

```
redcrown logout
```

For non-interactive use (CI, scripts), set **`REDCROWN_TOKEN`** to a valid access token and the CLI will use it without prompting for a browser flow.

**MCP**

Not applicable to MCP. See the headless agents step below for running agents without a browser.

## 3. Headless agents

Agents running in CI or other headless environments can authenticate without a browser by exchanging credentials for a dual-audience token via the first-party `cli-token` route.

**UI**

Not available via the UI. Headless token exchange is done from the CLI or directly via the API.

**CLI**

A first-party email and password **cli-token** exchange yields a dual-audience token that authorizes both the REST API and `mcp.redcrown.ai`, so an agent can drive MCP tools with no browser consent step. Set the resulting token as `REDCROWN_TOKEN` in your environment before running the agent.

**MCP**

Present the dual-audience bearer token obtained via the cli-token exchange as the `Authorization: Bearer` header when connecting to `mcp.redcrown.ai`. No browser consent step is needed. The token carries the MCP audience alongside the API audience, so every tool call is authorized for the full session lifetime of the token.

Last verified 2026-07-02.

---

# Benchmarks

Source: https://redcrown.ai/docs/guides/benchmarks/

Run RedCrown's public weekly benchmarks, or a pinned subset of your own.

## 1. Build a pinned subset

Pin exactly the items you want to evaluate by passing an IDs file. This ensures every re-run uses the same inputs so results are comparable over time.

**UI**

Benchmarks run from the CLI. View public benchmark results at [redcrown.ai/benchmarks/](https://redcrown.ai/benchmarks/).

**CLI**

Build a pinned experiment from the PriMock57 medical-transcription corpus. The `--ids-file` flag pins exactly those items:

```
redcrown build-dataset primock57 --ids-file subset.txt --out exp.json
```

`subset.txt` is a newline-delimited list of item IDs. An empty file raises an error loudly. The output `exp.json` is a validated experiment spec ready for `redcrown eval`.

**MCP**

Not available via MCP. Use the CLI above to build benchmark datasets.

## 2. Run it

Evaluate the pinned experiment across all candidates. Use concurrency to finish large benchmark runs in minutes rather than hours.

**UI**

View the results of RedCrown's public benchmarks at [redcrown.ai/benchmarks/](https://redcrown.ai/benchmarks/). The page shows the ranked table, winner, savings over the incumbent, and methodology for each benchmark.

**CLI**

```
redcrown eval exp.json --concurrency 10
```

The ranked report prints to the terminal when the run completes. Add `--report-json out.json` to write the full receipts to a file, and `redcrown push out.json --proof-link` to share the results.

**MCP**

Not available via MCP. Use the CLI above to run benchmark experiments.

Last verified 2026-07-02.

---

# Bring your own data

Source: https://redcrown.ai/docs/guides/bring-your-own-data/

Turn your CSV, JSONL, or media files into a scored dataset.

## 1. From a CSV or JSONL

Your data needs two columns: one for the model input and one for the reference output RedCrown scores against.

**UI**

On **Prove** (`/`), paste examples directly into the task input. For a file, use the **Upload** screen (`/advanced/upload`) to import a results file as a ranked run.

**CLI**

From a CSV:

```
redcrown build-dataset --from-csv data.csv --task extract --input-col input --reference-col reference --out task.json
```

From a JSONL file, replace `--from-csv` with `--from-jsonl` and point to your `.jsonl` file.

**MCP**

Pass examples inline in the `examples` argument to `scaffold_experiment` or `prove_task`. No separate dataset build step is needed.

## 2. Audio and images

For transcription or vision tasks, reference media by URL. RedCrown fetches each file at eval time.

**UI**

On **Prove**, attach media files to your examples using the media attachment control. Supported types include audio and images.

**CLI**

After building a dataset with per-item audio references, pass the base URL for your hosted clips:

```
redcrown eval task.json --audio-base https://host/clips/
```

**MCP**

Include input URLs directly in the `examples` array. RedCrown resolves the URLs during the run.

## 3. A public corpus

RedCrown ships two built-in public corpora for benchmarking and reproducibility.

**UI**

Not available via the UI. Use the CLI for public corpora.

**CLI**

Build from the PriMock57 clinical transcription corpus:

```
redcrown build-dataset primock57 --out exp.json
```

Build from an MMLU-Pro reasoning subset:

```
redcrown build-dataset mmlu-pro --out exp.json
```

**MCP**

Not available via MCP. Use the CLI commands above to build corpus datasets.

Last verified 2026-07-02.

---

# Connect providers

Source: https://redcrown.ai/docs/guides/connect-providers/

Bring your own provider keys so runs use your accounts and rates.

## 1. Provider API keys

RedCrown stores your keys encrypted at rest and resolves them per run. Keys are scoped to your workspace and never shared across workspaces.

**UI**

Go to **Connect** (`/connect`). Choose a provider from the list, paste your API key, and click **Save**. The key is stored encrypted and is never shown again after saving.

**CLI**

Set the provider API key as an environment variable in your shell before running `redcrown eval`. Keys never leave your machine during a local eval.

```
export OPENAI_API_KEY=sk-...
redcrown eval task.json
```

See the [CLI reference](../../reference/cli/) for the full list of accepted environment variable names per provider.

**MCP**

Call `list_models` with no arguments to see which providers are currently connected for your workspace:

```
{}
```

The response lists every available model grouped by provider. A provider that appears here has a connected key.

## 2. AWS Bedrock

Bedrock requires an AWS access key, secret, and region. RedCrown stores the three fields as an encrypted credential blob.

**UI**

Go to **Connect** (`/connect`). Select **AWS Bedrock**, then enter your **Access key ID**, **Secret access key**, and **Region** (for example, `us-east-1`). Click **Save**.

**CLI**

The standard AWS environment and credential chain applies. Set the environment variables or configure a profile with the AWS CLI before running `redcrown eval`:

```
export AWS_ACCESS_KEY_ID=AKIA...
export AWS_SECRET_ACCESS_KEY=...
export AWS_DEFAULT_REGION=us-east-1
redcrown eval task.json
```

**MCP**

Not available via MCP. Connect Bedrock in the app.

## 3. A custom endpoint

A custom HTTP endpoint lets you register any authenticated API that follows an OpenAI-compatible or plain request/response shape as a candidate model.

**UI**

Go to **Connect** (`/connect`). Select **Custom endpoint** and fill in the **Base URL**, the **Auth header** (for example, `Authorization: Bearer ...`), and the **Body style** (`openai` or `raw`). Click **Save**.

**CLI**

Set `REDCROWN_CUSTOM_HTTP` to a JSON object with the connection details to run a `custom_http` candidate locally:

```
export REDCROWN_CUSTOM_HTTP='{"base_url":"https://my-model.example.com","auth_header":"Bearer my-token","body_style":"openai"}'
redcrown eval task.json
```

**MCP**

Not available via MCP. Use the UI or the CLI environment variable above.

Last verified 2026-07-02.

---

# Cost simulator

Source: https://redcrown.ai/docs/guides/cost-simulator/

Estimate hosted and self-hosted cost for a model and token counts.

## 1. Estimate

Enter a model ID and your expected monthly token volume. The simulator returns projected hosted-API cost and self-hosted GPU cost per cloud provider.

**UI**

Open **Advanced** and navigate to **Simulator** (`/advanced/simulator`). Pick a model from the dropdown, enter your expected input and output token counts, then click **Estimate**. The results show projected hosted-API cost and self-hosted cost for AWS, Azure, and GCP.

**CLI**

Not available via the CLI. Use the app or MCP.

**MCP**

Call `simulate_cost` with a model ID and token counts:

```
{
  "model": "gpt-4o-mini",
  "input_tokens": 500,
  "output_tokens": 200,
  "monthly_volume": 100000
}
```

The response includes **projected hosted-API cost** and **self-hosted cost per cloud** (AWS, Azure, GCP) at the given monthly volume.

Last verified 2026-07-02.

---

# Run a Face-Off

Source: https://redcrown.ai/docs/guides/face-off/

Paste a plain-language task, run it across a default set of top models, and get a rubric-ranked verdict with a shareable proof page.

## 1. Open Face-Off

Go to [app.redcrown.ai/faceoff](https://app.redcrown.ai/faceoff) or click **Face-Off** in the app nav. You can also reach it from the call-to-action on the home screen.

Face-Off is available in the UI only. There is no dedicated `redcrown` CLI command or MCP tool for it today; the guide below covers the UI flow.

## 2. Paste your task

Type or paste the plain-language instruction you would send any LLM. For example: *Write a Python function that merges two sorted lists.* No labeled examples or ground-truth answers are required.

Face-Off uses an AI judge to score outputs, so the task can be anything a language model can respond to: code generation, summarization, rewriting, question answering, and so on.

## 3. Select models

A default set of top models is pre-selected: Qwen3 235B, Llama 4 Maverick, DeepSeek V4 Flash, and GPT-4o mini, all accessed via OpenRouter. Deselect any model you do not want to include.

**Provider key required.** Face-Off runs route through OpenRouter, so you need an OpenRouter key connected to your workspace before running. Go to **Advanced** then **Models and keys** to add one. Without a key, runs pin to the keyless demo provider (stub outputs) and the result cannot be shared as a proof.

## 4. Choose who judges

Select one of three options under **Who judges this?**

- **AI only** , the AI judge scores each output immediately and the ranked report is ready as soon as the run finishes.
- **AI + human** , the AI judge runs first, then a blind reviewer link is minted so a human expert can vote on the contested cases.
- **Human only** , no AI scoring; a blind reviewer link is minted and the final verdict waits for human votes.
For AI + human and human only, the existing magic-link reviewer portal is used. See [Reviewer portal](../reviewer-portal/) for details on sending and managing reviewer links.

## 5. Set a consistency band (optional)

Enter a repeat count to run each model N times on the same task. RedCrown reports the mean score and run-to-run standard deviation for each candidate, giving you a consistency band instead of a single-trial result. Higher N means more API calls and proportionally higher spend.

## 6. Contribute to the model index (optional)

Tick **Help build a public model index** to share anonymized win and score signals from this run. Your prompts, inputs, and outputs stay private, only the aggregate result (which model won, per-model scores, and a coarse task category) is recorded. It is off by default and opt-in per run.

## 7. Run and read the verdict

Click **Run Face-Off**. RedCrown fans the task across every selected model and the AI judge scores each output against three criteria: correctness, completeness, and follows-instructions. Each criterion is scored 1 to 10.

**How the winner is picked.** The report ranks candidates by score, then breaks ties by cost (cheapest that clears the quality bar wins). This is a model judgment, not ground truth. Because there is no reference answer, the AI judge can disagree with a human expert, and different runs may score differently if the model is non-deterministic. Treat the verdict as a strong signal, not a certified measurement.

To get ground-truth-backed results, bring labeled examples using [Prove a task](../prove-a-task/) or [Bring your own data](../bring-your-own-data/).

## 8. Share a proof

After the run completes, click **Share proof** on the ranked report. RedCrown mints a no-login page at `app.redcrown.ai/proof/<token>` with the full ranked table, per-item receipts, and provenance. Anyone with the link can read it without an account.

Proof minting requires a real provider key (the demo provider cannot produce a shareable proof). See [Share a proof](../share-a-proof/) for more on proof pages and how to revoke them.

Last verified 2026-07-05.

---

# Import external results

Source: https://redcrown.ai/docs/guides/import-external-results/

Turn an eval you already ran in any harness into a ranked, shareable RedCrown run.

## 1. Prepare the results JSON

RedCrown expects an aggregate results object with one entry per candidate. The required top-level fields are `name`, `objective`, `quality_metric`, `quality_bar`, `step`, and `candidates`.

**UI**

Go to **Upload** (`/advanced/upload`). Paste your results JSON directly into the text area or load it from a file using **Load file**.

**CLI**

Prepare a `results.json` file in the upload schema. The required fields are `name`, `objective`, `quality_metric`, `quality_bar`, `step`, and `candidates` (an array of per-candidate aggregate scores). See the [CLI reference](../../reference/cli/) for the full schema.

**MCP**

Pass the same object shape to `import_results`. The required fields are `name`, `objective`, `quality_metric`, `quality_bar`, `step`, and `candidates`:

```
{
  "name": "My external eval",
  "objective": "cheapest",
  "quality_metric": "exact_match",
  "quality_bar": 0.8,
  "step": "generate",
  "candidates": [...]
}
```

## 2. Import

Once your results are imported, RedCrown ranks them, applies the same integrity checks as a native run, and makes the run shareable via a proof link.

**UI**

After loading your results on **Upload** (`/advanced/upload`), click **Import**. The ranked run report appears immediately.

**CLI**

```
redcrown import-results results.json
```

The command uploads the results to your workspace and prints the run ID and a link to the ranked report in the app.

**MCP**

Call `import_results` with the full results object. The response includes the **experiment ID** and **run ID**:

```
{
  "name": "My external eval",
  "objective": "cheapest",
  "quality_metric": "exact_match",
  "quality_bar": 0.8,
  "step": "generate",
  "candidates": [...]
}
```

Last verified 2026-07-02.

---

# Inspect runs and reports

Source: https://redcrown.ai/docs/guides/inspect-runs-and-reports/

List runs and open their ranked reports and receipts.

## 1. List

Get a list of your runs to find the one you want to inspect.

**UI**

Open **Proofs** in the main nav. All your runs appear as cards with the experiment name, date, winner, and a link to the ranked report.

**CLI**

Not available via the CLI. Use the app or MCP to list runs.

**MCP**

Call `list_experiments` to retrieve your runs:

```
{}
```

## 2. Open a report

The ranked report shows each candidate's quality score, cost, latency, and savings. Expand any candidate to read the per-item receipts.

**UI**

Click a run in **Proofs** to open the ranked report. Click any candidate row to expand the receipts drill-down, which shows the model's output and score for each input item.

**CLI**

Write the full receipts to a local file when you run an eval:

```
redcrown eval task.json --report-json out.json
```

Open `out.json` to inspect per-item outputs and scores.

**MCP**

Fetch a run summary with `get_run` or the full ranked report with `get_report`:

```
{
  "run_id": "..."
}
```

Last verified 2026-07-02.

---

# Live proxy and collect

Source: https://redcrown.ai/docs/guides/live-proxy-and-collect/

Capture live traffic and shadow-replay it across candidate models.

## 1. Set up the proxy

Create a proxied endpoint. RedCrown forwards real traffic to your incumbent model and captures a sampled fraction for shadow replay.

**UI**

Open **Advanced** and navigate to **Live traffic** (`/advanced/live-traffic`). Click **New endpoint**, enter your upstream URL and candidate models, then click **Create**. RedCrown returns an endpoint key and a forward URL to point your application at.

**CLI**

Not available via the CLI. Use the app or MCP.

**MCP**

Call `setup_proxy` with your upstream URL and the candidate models to shadow:

```
{
  "name": "production-transcription",
  "upstream_url": "https://your-model-endpoint.example.com/v1/transcribe",
  "candidates": ["whisper-large-v3", "nova-2-medical"]
}
```

The response includes the **endpoint key** and the **forward URL**. Update your application to send traffic to the forward URL instead of the upstream directly.

## 2. List endpoints

Review all proxied endpoints in your workspace, including their capture counts and sampling rates.

**UI**

The **Live traffic** page (`/advanced/live-traffic`) lists all endpoints with their status, capture count, and daily budget remaining.

**CLI**

Not available via the CLI. Use the app or MCP.

**MCP**

Call `list_proxies` with no arguments to retrieve all endpoints in the current workspace:

```
{}
```

Each entry includes the endpoint ID, name, upstream URL, capture count, and sampling rate.

## 3. Replay captures

Fan captured traffic across every candidate model and store the scored outputs for comparison or reviewer sign-off.

**UI**

On the **Live traffic** page, click **Replay** on an endpoint. RedCrown fans the captured inputs across all candidate models and links the results to the endpoint.

**CLI**

Not available via the CLI. Use the app or MCP.

**MCP**

Call `replay_captures` with the endpoint ID:

```
{
  "endpoint_id": "<endpoint-id>"
}
```

RedCrown decrypts each stored capture, fans it across the non-incumbent candidates, scores relative to the incumbent, and persists one result row per capture per candidate.

Last verified 2026-07-02.

---

# Prove a task

Source: https://redcrown.ai/docs/guides/prove-a-task/

Run every candidate model on your task and prove the cheapest one that clears your bar.

## 1. Describe the task

Give RedCrown a plain-language task and a few input/output examples. It uses these to build a scored experiment.

**UI**

Open **Prove** (`/`). Paste your task description and a few examples, then click **Use my data**. If you are signed out, you are redirected to log in and then back to Prove.

**CLI**

Build a dataset from a CSV with your inputs and reference outputs:

```
redcrown build-dataset --from-csv examples.csv --task summarize --input-col text --reference-col summary --out task.json
```

**MCP**

Call `scaffold_experiment` with a plain-language task and examples to get a validated experiment spec, or pass them directly to `prove_task` and let it handle scaffolding automatically:

```
{
  "task": "Summarize a customer support ticket in one sentence.",
  "examples": [
    {"input": "Ticket text here...", "output": "One-sentence summary."}
  ]
}
```

## 2. Run and rank

RedCrown fans your task across every candidate model, scores outputs against your examples, and ranks by the cheapest model that clears the quality bar.

**UI**

After clicking **Use my data**, Prove runs the experiment automatically and displays the ranked report inline. The winner row is highlighted in green.

**CLI**

```
redcrown eval task.json --report-json out.json
```

The ranked report prints to the terminal. `--report-json` also writes the full receipts to `out.json` for sharing or inspection.

**MCP**

Call `prove_task` with your task and examples. It runs the full experiment and returns the ranked winner and savings in one call:

```
{
  "task": "Summarize a customer support ticket in one sentence.",
  "examples": [
    {"input": "Ticket text here...", "output": "One-sentence summary."}
  ]
}
```

## 3. Share

Mint a shareable, no-login proof link so stakeholders can read the ranked report and receipts without an account.

**UI**

On the ranked report, click **Share proof**. RedCrown mints a public link at `app.redcrown.ai/proof/<token>`.

**CLI**

```
redcrown push out.json --proof-link
```

The proof URL is printed to the terminal after a successful push.

**MCP**

The `prove_task` response includes a `proof_url` field. Pass it to stakeholders directly; no additional call is needed.

Last verified 2026-07-02.

---

# Reviewer portal

Source: https://redcrown.ai/docs/guides/reviewer-portal/

Send a run to a non-technical domain expert for a blinded verdict.

## 1. Create a session and invite

A review session selects the contested cases from a run and sends a magic-link invite to an expert. Model names are hidden from the reviewer.

**UI**

Open a completed run report and click **Send for review**. RedCrown seeds the contested cases automatically. Copy the invite link and share it with your expert.

**CLI**

Not available via the CLI. Use the app or MCP.

**MCP**

Call `create_review` with the run ID and an accept threshold, then call `invite_reviewer` to mint a magic-link token:

```
{
  "run_id": "<run-id>",
  "accept_threshold": 0.8
}
```

Then invite:

```
{
  "session_id": "<session-id>",
  "label": "Dr. Smith"
}
```

`invite_reviewer` returns the one-time token. The reviewer link is `app.redcrown.ai/review/<token>`.

## 2. The expert reviews (blinded)

The reviewer sees Option A and Option B without model names. They vote on each case, optionally adding a confidence level and severity flag.

**UI**

The reviewer opens `app.redcrown.ai/review/<token>`. No account is needed. Each case shows the incumbent output and a challenger output side by side with word-diff highlighting. The reviewer selects **Accept A**, **Accept B**, or **Flag** for each case.

**CLI**

Not available via the CLI. Use the app or MCP.

**MCP**

Call `get_review_examples` with the session ID to retrieve the blinded cases and any existing votes:

```
{
  "session_id": "<session-id>"
}
```

Each example includes the incumbent output, the challenger output, and the reviewer's vote if already cast.

## 3. Read the decision

Once votes are in, the decision rollup un-blinds the candidates and names the recommended model based on accept rate and cost.

**UI**

Open the un-blinded decision rollup at `app.redcrown.ai/reviews/<sessionId>`. The recommended candidate is shown at the top, followed by a ranked table of accept rates, mean cost, and divergence per candidate.

**CLI**

Not available via the CLI. Use the app or MCP.

**MCP**

Call `get_decision_report` with the session ID:

```
{
  "session_id": "<session-id>"
}
```

The response includes the recommended candidate, accept rate per candidate, mean cost, mean divergence, and cross-reviewer agreement.

Last verified 2026-07-02.

---

# Run and rank

Source: https://redcrown.ai/docs/guides/run-and-rank/

Run an experiment spec and read the ranked report.

## 1. Run

Execute a saved experiment spec against all candidate models. Raise concurrency for large token benchmarks to finish faster.

**UI**

Open **Experiments** under the Advanced menu. Select a saved experiment and click **Run**. The ranked report appears when the run completes.

**CLI**

```
redcrown eval exp.json --concurrency 10
```

Raising `--concurrency` runs candidates in parallel. Useful for many-candidate token benchmarks where sequential execution would take too long.

**MCP**

Create then run an experiment in two calls:

```
{
  "experiment_id": "..."
}
```

First call `create_experiment` with your spec, then call `run_experiment` with the returned `experiment_id`.

## 2. Reproducibility

Run a candidate multiple times to measure run-to-run quality variance. The report includes a mean and standard deviation confidence band.

**UI**

Not available via the UI. Use the CLI or MCP for repeat runs.

**CLI**

```
redcrown eval exp.json --repeat 5
```

Each candidate runs 5 times. The report shows the mean quality score and run-to-run standard deviation. Note that `--repeat 5` multiplies spend by 5.

**MCP**

Set `repeat_n` in the experiment definition when calling `create_experiment`. The resulting report includes the mean and standard deviation for each candidate.

## 3. Read the winner

The ranked report identifies the cheapest candidate that clears your quality bar. Every other candidate is ranked below it with its score and cost.

**UI**

The ranked report highlights the winner row in green. The table shows quality score, cost, latency, and savings vs. the incumbent for each candidate.

**CLI**

The ranked report prints to the terminal at the end of the run. Add `--report-json` to save it:

```
redcrown eval exp.json --report-json out.json
```

**MCP**

Call `get_report` with the run id to fetch the full ranked report:

```
{
  "run_id": "..."
}
```

Last verified 2026-07-02.

---

# Share a proof

Source: https://redcrown.ai/docs/guides/share-a-proof/

Mint a no-login proof page from a completed run.

## 1. Mint

A proof link is a public, no-login page at `app.redcrown.ai/proof/<token>` that shows the ranked report, winner, receipts, and provenance for a completed run. Anyone with the link can read it; no account is required.

**UI**

Open the ranked report for a completed run and click **Share proof**. RedCrown mints the link and copies it to your clipboard.

**CLI**

```
redcrown push out.json --proof-link
```

The proof URL is printed to the terminal after a successful push.

**MCP**

The `prove_task` response includes a `proof_url` field. Pass it to stakeholders directly; no additional call is needed.

## 2. Keep confidential outputs local

By default, `redcrown push` uploads per-item receipts alongside the ranked report. If your individual outputs are confidential, strip them before uploading so only aggregate scores are shared.

**UI**

Not available via the UI. Use the CLI flag to strip receipts before upload.

**CLI**

```
redcrown push out.json --no-receipts --proof-link
```

The `--no-receipts` flag strips per-item receipts client-side before upload. Individual model outputs never leave your machine.

**MCP**

Not available via MCP. Use the CLI flag above to strip receipts before upload.

Last verified 2026-07-02.

---

# Signed attestation

Source: https://redcrown.ai/docs/guides/signed-attestation/

Download a tamper-evident, independently verifiable signed proof.

## 1. Download

An attestation proves authenticity, integrity, and provenance. It does not by itself prove the input data was representative.

**UI**

Open a proof page at `app.redcrown.ai/proof/<token>` and click **Download signed attestation**. The file is saved as `<run-id>-attestation.json`.

**CLI**

```
redcrown attest <run-id>
```

Writes `<run-id>-attestation.json` to the current directory. The file contains the signed run report and the RedCrown public key URL used to verify it.

**MCP**

Not available via MCP. Fetch the public attestation route directly or use the CLI above.

## 2. Verify offline

Verification checks the RedCrown RS256 signature against the published key. The check is fully offline once you have the attestation file and a copy of the key.

**UI**

Not available via the UI. Verify with the CLI.

**CLI**

```
redcrown verify <run-id>-attestation.json
```

Checks the RedCrown signature against the published key. Add `--key <jwks>` to pin a specific key URL for air-gapped verification.

**MCP**

Not available via MCP. Use the CLI above to verify attestation files.

Last verified 2026-07-02.

---

# Try a sample

Source: https://redcrown.ai/docs/guides/try-a-sample/

See a proof in one step with a public sample, no data, no keys, no account.

## 1. Pick a sample

Choose one of the built-in sample tasks. RedCrown runs it against a set of public models and returns a ranked proof.

**UI**

Open **Prove** (`/`). Under the task input, choose a sample task from the sample picker. No login is required.

**CLI**

```
redcrown eval --sample summarize
```

The `--sample` flag accepts `extract`, `classify`, or `summarize`.

**MCP**

Call `try_sample`. The `task` argument is optional and defaults to `transcribe`:

```
{
  "task": "transcribe"
}
```

## 2. Read the proof

The ranked report shows each candidate model, its score, cost, and whether it clears the quality bar. The winner is the cheapest model that clears.

**UI**

The ranked report renders inline on the Prove page. Each row shows the model name, quality score, cost, and a winner badge on the cheapest model that clears the bar.

**CLI**

The ranked report prints to the terminal. Add `--report-json out.json` to save the full receipts to a file:

```
redcrown eval --sample summarize --report-json out.json
```

**MCP**

`try_sample` returns a `proof_url` pointing to the public ranked report. Share it or open it directly.

Last verified 2026-07-02.

---

# Workspaces and billing

Source: https://redcrown.ai/docs/guides/workspaces-and-billing/

Organize work into workspaces and manage your plan.

## 1. Workspaces

A workspace groups experiments, runs, and credentials under one organization. You can be a member of multiple workspaces and switch between them in the app.

**UI**

Open **Workspace** at `/advanced/workspace`. From here you can rename the organization, invite members by token, and change member roles. Use the **org switcher** in the top nav to move between workspaces.

**CLI**

Target a specific workspace when pushing results by passing `--org`:

```
redcrown push out.json --org <org-id>
```

The `<org-id>` is the UUID shown on the Workspace settings screen. Omit `--org` to push to your default personal workspace.

**MCP**

Not available via MCP. Manage workspaces in the app.

## 2. Billing

RedCrown meters on **proven workloads** (active experiments). Each plan tier sets a limit on how many you can keep active at once.

**UI**

Open **Billing** at `/advanced/billing` to see your current plan, usage against your workload limit, and how many hosted proof links you have minted. Click **Upgrade** to open the Stripe-hosted checkout and choose a plan tier.

**CLI**

Not available via the CLI. Use the UI to manage your plan.

**MCP**

Not available via MCP. Use the UI to manage your plan.

Last verified 2026-07-02.

---

# RedCrown documentation

Source: https://redcrown.ai/docs/

RedCrown proves, on your own data, which model configuration is cheapest at equal-or-better quality. These docs show every capability three ways: in the app UI, with the `redcrown` CLI, and via the MCP tools.

## Start here

- [Quickstart](quickstart/) , prove one task end to end.
- [Concepts](concepts/) , experiment, run, report, receipts, provenance, proof, attestation.

## Guides

- [Prove a task](guides/prove-a-task/)
- [Try a sample](guides/try-a-sample/)
- [Run a Face-Off](guides/face-off/)
- [Bring your own data](guides/bring-your-own-data/)
- [Connect providers and credentials](guides/connect-providers/)
- [Run and rank an experiment](guides/run-and-rank/)
- [Inspect runs and reports](guides/inspect-runs-and-reports/)
- [Import external eval results](guides/import-external-results/)
- [Share a proof](guides/share-a-proof/)
- [Signed attestation](guides/signed-attestation/)
- [Reviewer portal](guides/reviewer-portal/)
- [Live proxy and collect mode](guides/live-proxy-and-collect/)
- [Cost simulator](guides/cost-simulator/)
- [Benchmarks](guides/benchmarks/)
- [Workspaces and billing](guides/workspaces-and-billing/)
- [Authentication](guides/auth/)

## Reference

- [CLI commands](reference/cli/)
- [MCP tools](reference/mcp/)
- [UI screens](reference/ui/)

## For agents

An LLM can read [/llms.txt](/llms.txt) (the index) or [/llms-full.txt](/llms-full.txt) (the whole corpus), and any page has a Markdown twin at its `index.md`.

Last verified 2026-07-04.

---

# Quickstart

Source: https://redcrown.ai/docs/quickstart/

Prove one task end to end: run a set of models on your data, rank the cheapest one that clears your quality bar, and share a proof. Pick a surface above, or compare all three.

## 1. Describe the task and give a few examples

A task is a plain-language instruction plus a few example inputs (and, if you have them, the correct outputs to score against).

**No labeled data?** Set `reference_source: "incumbent"` to score every challenger against your current model's own output. You prove which config is cheaper at equal-or-better quality without labeling a thing: the incumbent's output is the reference.

**UI**

Open [app.redcrown.ai](https://app.redcrown.ai), land on **Prove**, paste your task and a few example rows, then choose "Use my data".

**CLI**

```
redcrown build-dataset --from-csv examples.csv --task summarize \
  --input-col text --reference-col summary --out task.json
```

**MCP**

```
prove_task({
  "task": "Summarize each support ticket in one sentence.",
  "examples": [{"input": "...", "output": "..."}]
})
```

## 2. Run every candidate and rank them

RedCrown fans the task across the candidate models, scores each against your references, and ranks by the cheapest that clears the bar.

**UI**

The Prove screen runs the candidates and shows the ranked report with cost, quality, and receipts.

**CLI**

```
redcrown eval task.json --report-json out.json
```

**MCP**

`prove_task` runs the eval and returns the ranked winner and savings in one call. For a demo with no data, use `try_sample`.

## 3. Share a proof

Turn the ranked run into a no-login proof page anyone can open.

**UI**

Click **Share proof** on the report to mint a link.

**CLI**

```
redcrown push out.json --proof-link
```

**MCP**

`prove_task` returns a `proof_url` directly.

Last verified 2026-07-02. New here? Read [Concepts](../concepts/) next.

---

# CLI reference

Source: https://redcrown.ai/docs/reference/cli/

Every redcrown subcommand and flag.

## `redcrown attest`

Download a signed attestation for a completed run

| Flag | Description | Default |
| --- | --- | --- |
| `run_id` |  | `` |
| `--out` | output file (default <run_id>-attestation.json) | `` |
| `--api-url` |  | `https://api.redcrown.ai` |

## `redcrown build-dataset`

Build an experiment JSON from a public corpus (primock57, mmlu-pro)

| Flag | Description | Default |
| --- | --- | --- |
| `source` | public corpus source (omit when using --from-csv/--from-jsonl) | `` |
| `--out` | path to write the experiment JSON | `` |
| `--limit` | cap the number of (audio, reference) items | `` |
| `--corpus-dir` | path to an existing corpus checkout (skips cloning) | `` |
| `--quality-bar` | quality bar a candidate must clear; default = the dataset's published bar (mmlu-pro 0.75), NOT 0.0 | `` |
| `--monthly-minutes` | projected monthly audio minutes for the $/mo column | `20280` |
| `--no-clone` | do not clone; require --corpus-dir to point at a checkout | `False` |
| `--ids-file` | path to a newline-separated list of clip ids; builds exactly those items (the pinned benchmark subset). Applied after --limit, so do not combine them | `` |
| `--from-csv` | build from a CSV with input + reference columns (text tasks) | `` |
| `--from-jsonl` | build from a JSONL with input + reference fields (text tasks) | `` |
| `--task` | task kind for --from-csv/--from-jsonl | `` |
| `--input-col` | input column/field name (default 'input') | `input` |
| `--reference-col` | reference column/field name (default 'reference') | `reference` |

## `redcrown eval`

Run an experiment JSON locally and print the report

| Flag | Description | Default |
| --- | --- | --- |
| `path` | path to an experiment JSON file | `` |
| `--sample` | run a bundled offline sample with the demo provider (no keys) | `` |
| `--report-json` | also write the full results (receipts included) as an upload-schema JSON for redcrown push | `` |
| `--audio-base` | base URL for per-item audio (item_audio = base + id + .mp3) | `` |
| `--concurrency` | number of provider calls to run at once (default 1, sequential; raise it for many-candidate token benchmarks) | `1` |
| `--repeat` | run each candidate N times and report a run-to-run quality stdev (default: the experiment's repeat_n, or 1). Note: this makes N times the API calls, so N times the eval spend. Tip: set params.seed on a candidate for a provider that supports seeded sampling. | `` |

## `redcrown import-results`

Upload a locally-run eval's aggregated results JSON into RedCrown

| Flag | Description | Default |
| --- | --- | --- |
| `path` | path to a results JSON file (the upload schema) | `` |
| `--api-url` | RedCrown API base URL | `https://api.redcrown.ai` |
| `--token` | access token (or set REDCROWN_TOKEN); your RedCrown/Supabase bearer token | `` |

## `redcrown login`

Log in to RedCrown (device flow; opens a code page)

| Flag | Description | Default |
| --- | --- | --- |
| `--api-url` | RedCrown API base URL | `https://api.redcrown.ai` |

## `redcrown logout`

Delete the stored RedCrown credentials

## `redcrown push`

Push a local eval's results JSON to RedCrown as a cloud run

| Flag | Description | Default |
| --- | --- | --- |
| `path` | results JSON from `redcrown eval --report-json` | `` |
| `--api-url` |  | `https://api.redcrown.ai` |
| `--app-url` |  | `https://app.redcrown.ai` |
| `--org` | workspace (org) id; default personal org | `` |
| `--experiment` | append the run to this standing experiment id | `` |
| `--no-receipts` | strip per-item receipts/references before upload (confidential outputs never leave this machine) | `False` |
| `--proof-link` | mint and print a shareable proof link for the pushed run | `False` |

## `redcrown verify`

Verify a RedCrown attestation file offline

| Flag | Description | Default |
| --- | --- | --- |
| `path` |  | `` |
| `--key` | a JWKS file to verify against (offline); default: fetch the file's key_url | `` |
Generated from source. Last verified 2026-07-02.

---

# MCP reference

Source: https://redcrown.ai/docs/reference/mcp/

Every RedCrown MCP tool at mcp.redcrown.ai.

## Core tools

| Tool | Description |
| --- | --- |
| `prove_task` | Benchmark every model on a task and return the cheapest one that's good enough, with a shareable proof link. Pass a plain-language task and a few examples ({input, output?}); leave the output blank to rank against the model you use now. For a no-input demo, use try_sample. |
| `try_sample` | Run a benchmark on a public sample dataset and return a shareable proof, no input, no keys, no account. Great for a quick demo. Optional task (default: transcribe). |
| `import_results` | Turn an eval you already ran (in any harness or the redcrown CLI) into a ranked, shareable run. Pass the aggregate results object (name, objective, quality_metric, quality_bar, step, candidates[, references]). Returns the experiment and run ids. |
| `get_run` | Get an experiment run and its ranked report by run id. |
| `get_report` | Alias of get_run: fetch the ranked proof report for a run id. |

## Advanced tools

| Tool | Description |
| --- | --- |
| `list_models` | List available providers and models, including free no-key models. |
| `list_experiments` | List the caller's experiments. |
| `create_experiment` | Create an experiment from a full definition (name, objective, quality_metric, quality_bar, reference_source, allowed_providers, pipeline, dataset). Most callers should use prove_task instead. |
| `run_experiment` | Queue a run of an experiment by id; returns the run id to poll with get_report. |
| `scaffold_experiment` | Turn a plain-language task into a valid experiment spec ready for create_experiment. (prove_task does this for you.) |
| `simulate_cost` | Estimate projected hosted-API and self-hosted (per cloud) cost for a model and token counts. |
| `setup_proxy` | Create a proxied endpoint that captures live traffic for shadow evaluation. Returns the endpoint key and forward URL. |
| `list_proxies` | List all proxied endpoints for the caller's workspace. |
| `replay_captures` | Replay captured traffic through the shadow eval engine for a proxied endpoint. |
| `create_review` | Create a human-review session for a proxied endpoint's captures. |
| `invite_reviewer` | Invite a reviewer to a review session. Returns a one-time token; the link is https://app.redcrown.ai/review/<token>. |
| `get_review_examples` | Get the candidate outputs for a captured request, ready for human review. |
| `get_decision_report` | Get the aggregated decision report for a review session, including reviewer verdicts and the recommended winner. |
Generated from source. Last verified 2026-07-02.

---

# UI reference

Source: https://redcrown.ai/docs/reference/ui/

The app.redcrown.ai screens and what each one does. The primary nav is Prove, Proofs, Connect, and an Advanced menu.

| Screen | Path | What it does |
| --- | --- | --- |
| Prove | `/` | Paste a task and examples (or pick a sample) and prove the cheapest good-enough model. |
| Proofs | `/proofs` | Your runs and their ranked reports; mint or open a proof link. |
| Connect | `/connect` | Add provider API keys (BYO), AWS Bedrock, or a custom HTTP endpoint. |
| Models | `/advanced/models` | Model registry and connection hub. |
| Live traffic | `/advanced/live-traffic` | Proxy endpoints, capture, shadow replay, and reviewer sessions. |
| Simulator | `/advanced/simulator` | Estimate hosted and self-hosted cost for a model and token counts. |
| Upload | `/advanced/upload` | Import an eval you ran elsewhere as a ranked, shareable run. |
| Workspace | `/advanced/workspace` | Organizations, members, and invites. |
| Billing | `/advanced/billing` | Plan and usage. |
| Proof (public) | `/proof/:token` | No-login ranked proof report with receipts, provenance, and attestation download. |
| Reviewer (public) | `/review/:token` | No-login blinded reviewer portal for a domain expert. |
Last verified 2026-07-02.