# Reviewer portal

Source: https://redcrown.ai/docs/guides/reviewer-portal/

Send a run to a non-technical domain expert for a blinded verdict.

## 1. Create a session and invite

A review session selects the contested cases from a run and sends a magic-link invite to an expert. Model names are hidden from the reviewer.

**UI**

Open a completed run report and click **Send for review**. RedCrown seeds the contested cases automatically. Copy the invite link and share it with your expert.

**CLI**

Not available via the CLI. Use the app or MCP.

**MCP**

Call `create_review` with the run ID and an accept threshold, then call `invite_reviewer` to mint a magic-link token:

```
{
  "run_id": "<run-id>",
  "accept_threshold": 0.8
}
```

Then invite:

```
{
  "session_id": "<session-id>",
  "label": "Dr. Smith"
}
```

`invite_reviewer` returns the one-time token. The reviewer link is `app.redcrown.ai/review/<token>`.

## 2. The expert reviews (blinded)

The reviewer sees Option A and Option B without model names. They vote on each case, optionally adding a confidence level and severity flag.

**UI**

The reviewer opens `app.redcrown.ai/review/<token>`. No account is needed. Each case shows the incumbent output and a challenger output side by side with word-diff highlighting. The reviewer selects **Accept A**, **Accept B**, or **Flag** for each case.

**CLI**

Not available via the CLI. Use the app or MCP.

**MCP**

Call `get_review_examples` with the session ID to retrieve the blinded cases and any existing votes:

```
{
  "session_id": "<session-id>"
}
```

Each example includes the incumbent output, the challenger output, and the reviewer's vote if already cast.

## 3. Read the decision

Once votes are in, the decision rollup un-blinds the candidates and names the recommended model based on accept rate and cost.

**UI**

Open the un-blinded decision rollup at `app.redcrown.ai/reviews/<sessionId>`. The recommended candidate is shown at the top, followed by a ranked table of accept rates, mean cost, and divergence per candidate.

**CLI**

Not available via the CLI. Use the app or MCP.

**MCP**

Call `get_decision_report` with the session ID:

```
{
  "session_id": "<session-id>"
}
```

The response includes the recommended candidate, accept rate per candidate, mean cost, mean divergence, and cross-reviewer agreement.

Last verified 2026-07-02.