# Start from what you already have Source: https://redcrown.ai/docs/guides/start-from-what-you-have/ You do not need a new experiment to get a first record. Three things you probably have today are enough: a journal your harness already writes, a set of production traces, or a rules list your client wrote. Each section below ends in one command. Every command on this page runs offline, with no account and no provider key. Install the CLI with `pip install redcrown` (Python 3.11 or newer). ## 1. A journal you already keep A journal is one directory per run. Each run directory holds a `meta.json` and one or more `verdicts*.jsonl` files. Each line of a verdicts file is one case. A case carries a `case_id`, an optional written `status`, an optional `elapsed_ms`, and a list of `checks`. Each check has a `name`, a `status` of `ok`, `fail`, or `unmeasured`, and an optional one-line `evidence`. A journal that wrote a legacy `dimensions` object (name to `true`, `false`, or `null`) is read the same way, without a rewrite. This is a three-row journal written by hand, in `runs/2026-09-16-a/`: ``` runs/2026-09-16-a/meta.json { "corpus_version": "corpus-v3", "harness_sha": "unknown", "target": {"commit": "a1b2c3d", "prompt_hash": "p9"}, "opened_at": 0 } runs/2026-09-16-a/verdicts.jsonl {"case_id": "pass-case", "status": "PASS", "elapsed_ms": 1, "checks": [{"name": "persisted", "status": "ok"}]} {"case_id": "fail-case", "status": "FAIL", "elapsed_ms": 1, "checks": [{"name": "persisted", "status": "fail"}]} {"case_id": "blocked-case", "status": "BLOCKED", "elapsed_ms": 1, "checks": [{"name": "persisted", "status": "unmeasured"}]} ``` **UI** Not available in the app. A journal is read on the machine that wrote it. The record reaches the app in step 4, with `redcrown push`. **CLI** ``` redcrown harness report 2026-09-16-a --runs runs ``` The real output: ``` PASS 1 of 2 graded (1 of 2 counting partial), 1 not graded, 3 cases (no group) PASS 1 FAIL 1 PARTIAL 0 BLOCKED 1 ERROR 0 (2 of 3 graded) checks, worst first: persisted 50.0% (1 of 2), 1 unmeasured ``` `--runs` is the journal directory, and the first argument is the run subdirectory. The summary prints two denominators: strict (`PASS` plus `FAIL`) and with partial. The blocked case shows as not graded. It is never a pass and never a fail. The command exits 0. It exits 1 when the run directory does not exist. **MCP** Not available via MCP. Use the CLI above. ## 2. Production traces A trace is what your product already produced: an input, the output the model returned, and the answer that was correct. Put the inputs and correct answers in a CSV and the outputs in a JSONL file. RedCrown scores the outputs with a deterministic metric. This path makes no model call and needs no provider key. The result carries a declared configuration, not a witnessed one: RedCrown scored what you supplied and does not claim to know how it was generated. `items.csv` needs an `item_id` column (or `id`), an `input` column, and a `reference` column. `outputs.jsonl` needs one line per output with `candidate_id`, `item_id`, and `output` (or `error`). `candidate_label`, `cost_usd`, and `latency_ms` are optional. These are the two files used below, two rows each: ``` items.csv item_id,input,reference t-1001,"Ticket: printer on floor 2 is offline again",hardware t-1002,"Ticket: cannot log in after the password reset",access outputs.jsonl {"candidate_id": "prod", "candidate_label": "production, week 37", "item_id": "t-1001", "output": "hardware"} {"candidate_id": "prod", "candidate_label": "production, week 37", "item_id": "t-1002", "output": "billing"} ``` **UI** Open **Advanced → Upload** and use the **Score supplied outputs** section. Click **Choose CSV or JSONL**, map the columns, pick the metric, then click **Score outputs**. Pick **New feature** to create an active AI feature. That uses a slot on your plan. Pick an existing feature in **Add to feature** to add a version instead. The existing feature must be one you created with **Score supplied outputs**. Adding a version uses no new slot. **CLI** ``` redcrown score --items-csv items.csv --outputs-jsonl outputs.jsonl \ --metric exact_match --evidence-json evidence.json --report-json scored.json ``` The real output: ``` Experiment f534e02e-f873-4a6e-9a1a-ca7c8cbd9c8d objective=quality bar=0.0 baseline=labels Step outputs: candidate quality $/min $/mo cost latency flags prod 0.500 n/a n/a unknown unknown baseline, WINNER wrote private raw evidence; it may contain prompts, outputs, references, and other customer data. Keep it private. wrote scored results to scored.json; push to the cloud with: redcrown push scored.json ``` One of two outputs matched its reference, so quality is 0.500. Cost and latency read `unknown` because the traces carried neither. `--metric` accepts `exact_match`, `similarity` (the default), `json_field_accuracy`, and `f1`. `f1` against references that are not JSON objects is refused before scoring. `--evidence-json` writes the private raw report that `redcrown check` reads; it can contain your data, so do not publish it. `--report-json` writes the upload format for `redcrown push`. **MCP** Not available via MCP. `import_results` records aggregate results, with receipts only when the payload supplies them, and does not produce an evaluation identity. Use the CLI or the UI above. ## 3. A rules list your client wrote A client states rules in plain words. Each rule becomes one named check. The harness grades every case on every check and writes one line of evidence a reader can verify. Three rules and the check names they become: - "The order lands in the database with the right customer." becomes `order_persisted`. - "The reply names the pickup date." becomes `pickup_date_named`. - "No price is quoted when the rate is unknown." becomes `no_price_when_rate_unknown`. A rule that did not apply to a case is `unmeasured` on that case, with the reason in the evidence line. Here the third rule did not apply to the third order because the rate table had a price. This is the journal, one line per case, with no written status: ``` runs/2026-09-17-rules/verdicts.jsonl {"case_id": "order-017", "group": "intake", "elapsed_ms": 2140, "checks": [{"name": "order_persisted", "status": "ok", "evidence": "orders row 8812, customer 4410"}, {"name": "pickup_date_named", "status": "ok", "evidence": "reply: pickup Thursday, September 18"}, {"name": "no_price_when_rate_unknown", "status": "ok", "evidence": "no dollar amount in the reply"}]} {"case_id": "order-018", "group": "intake", "elapsed_ms": 1980, "checks": [{"name": "order_persisted", "status": "ok", "evidence": "orders row 8813, customer 4410"}, {"name": "pickup_date_named", "status": "fail", "evidence": "reply: we will confirm the pickup soon"}, {"name": "no_price_when_rate_unknown", "status": "ok", "evidence": "no dollar amount in the reply"}]} {"case_id": "order-019", "group": "intake", "elapsed_ms": 2305, "checks": [{"name": "order_persisted", "status": "ok", "evidence": "orders row 8814, customer 2207"}, {"name": "pickup_date_named", "status": "ok", "evidence": "reply: pickup Friday, September 19"}, {"name": "no_price_when_rate_unknown", "status": "unmeasured", "evidence": "rate table had a price, so the rule did not apply"}]} ``` When a row carries no written status, RedCrown derives one from its checks, in this order: an error is `ERROR`; any fail is `FAIL`; nothing measured is `BLOCKED`; something measured and the rest unmeasured is `PARTIAL`; otherwise `PASS`. An unmeasured check is never a pass or a fail. A status the harness wrote is authoritative, because the harness knew things the row cannot show, such as a case for a feature the product has not built yet. A grading override is a rulings file, passed with `--rulings`. It flips one named check on one case, records who ruled and why on that check, and keeps the raw status on the row. The journal on disk is never rewritten. **UI** Not available in the app. After `redcrown push`, the report view shows every check on every case, worst first. **CLI** ``` redcrown harness report 2026-09-17-rules --runs runs ``` The real output: ``` PASS 1 of 2 graded (1 of 3 counting partial), 1 not graded, 3 cases intake PASS 1 FAIL 1 PARTIAL 1 BLOCKED 0 ERROR 0 (2 of 3 graded) checks, worst first: pickup_date_named 66.7% (2 of 3) no_price_when_rate_unknown 100.0% (2 of 2), 1 unmeasured order_persisted 100.0% (3 of 3) ``` The three rows became `PASS`, `FAIL`, and `PARTIAL`. The `group` field gives the per-group line. A check's pass rate counts measured checks only, and the unmeasured count is printed beside it. To check the run against the client's written requirements, put them in a policy file and pass it with `--policy`. Needs redcrown 0.1.23 or later. This `policy.json` asks for at least 3 cases and an `ok` on `order_persisted` and `pickup_date_named` on every case: ``` { "schema_version": 1, "kind": "harness", "requirements": { "minimum_cases": 3, "required_checks": [{"names": ["order_persisted", "pickup_date_named"]}] } } redcrown harness report 2026-09-17-rules --runs runs --policy policy.json ``` The command prints the same summary, then this block, and exits 2 because `order-018` failed a required check: ``` redcrown harness policy: fail (exit 2) minimum_cases: pass; observed 3; threshold 3 required_checks[0]: fail; reason required_check_failed; 1 of 3; cases order-018 ``` **MCP** Not available via MCP. Use the CLI above. ## 4. Compare, export, push A second run of the same corpus goes in a second run directory. Here `runs/2026-09-17-b/` holds the same three cases as step 1, after a change that made `pass-case` fail its `persisted` check. The compare is check by check on the shared case ids. A measured check that becomes unmeasured is inconclusive, never a regression. A case on one side only is listed, never counted. **UI** After both runs are pushed to one experiment, the report view of the later run shows **Compared with an earlier run**, with a base-run selector and the same per-check table. **CLI** ``` redcrown harness compare 2026-09-16-a 2026-09-17-b --runs runs ``` The real output: ``` cases: regressed 1, fixed 0, inconclusive 0, newly measured 0, unchanged 2, only in base 0, only in head 0 persisted regressed 1 fixed 0 inconclusive 0 newly measured 0 regressed (1): pass-case persisted: ok -> fail exit 1 ``` The command exits 1 on any regression, so a CI step stops there. It exits 0 when nothing regressed. Two runs on different corpus versions are refused with exit 2, because a pass rate is only comparable within one corpus; `--allow-corpus-drift` compares the shared case ids anyway and prints the drift. `--format markdown` and `--format json` give the same result in those forms. **MCP** Not available via MCP. Use the CLI above. Export turns one run into the upload payload. `--name` is the experiment name shown on the record and `--label` names the system under test at one version. **UI** Not available in the app. The export is a file on your machine. **CLI** ``` redcrown harness export 2026-09-17-b --runs runs \ --name "Order intake regression" --label "order-intake@d4e5f6a" --out upload.json ``` The real output: ``` wrote upload.json: 3 cases ``` The payload has `quality_metric` `"checks"`, one candidate, and one item per case. Each item carries its `checks` and its `harness_status`. The candidate's `source_revision` reads `d4e5f6a@corpus-v3`, the target commit and the corpus version from `meta.json`. Quality is 1.0 for `PASS`, 0.0 for `FAIL`, and null for anything else, because an unmeasured case is not a score. Cost is null when any case's cost is unknown. **MCP** Not available via MCP. Use the CLI above. Push sends the payload to RedCrown as one run. This is the one command on this page that needs an account. Run `redcrown login` once first. Push the first run with no `--experiment`; it prints `pushed run (experiment )`. Push every later run with `--experiment ` so the runs share one experiment and the app can compare them. **UI** The run lands in **History**. Its report view lists every case with its checks, worst first, and compares the run with the earlier run of the same experiment. **CLI** ``` redcrown push upload.json --experiment ``` On success the command prints `pushed run (experiment )`. Without a login it prints `error: not logged in` and names the three ways to authenticate. `--no-receipts` strips the per-item receipts before upload, so outputs and evidence lines never leave your machine. `--proof-link` also mints a public proof link and prints it as `proof link: https://app.redcrown.ai/proof/`. A run with an `ERROR` case is pushed and visible in the app, but the proof link is refused with the report's own caveat, quoted in full, until every case ran; `--publish-anyway` overrides that and the override is recorded. A `BLOCKED` or `PARTIAL` case does not refuse the link: the page discloses the unmeasured count beside the pass rate, because no re-run scores a check the product has not built. **MCP** Not available via MCP. Use the CLI above. A coding agent that runs this loop for you can read the [Harness guide for coding agents](../harness-for-agents/). Every command and exit code on that page is the same as here. Last verified 2026-09-23.