Repro tests — turning a bug report into a failing test

Generate a test that fails the way a bug report describes, review it before anything is saved, and let a later green run stand as evidence of the fix.

"Checkout breaks when the discount is exactly zero" is a sentence. It cannot be run, it cannot be gated on, and in six weeks nobody will know whether it is still true.

A repro test is that sentence as a test that fails the way the report describes. Vera writes one from an issue, and the whole design turns on an inversion worth stating before anything else:

A repro that runs red is the success. A repro that passes on its first run has proved nothing, and Vera says so loudly rather than showing you a green tick.

There is no plan gate on this. It needs an AI key or your onboarding grant, the ai:use permission to propose, and issues:manage + tests:manage + tests:run to approve. Free workspaces included.

Where it starts

Open an issue at /issues. Any issue can carry a repro, but one filed from a failed run is far better grounded — Vera walks back through that run to find the step that failed, the page it was on, and the error it produced.

The starting page is derived in that order: the last goto before the failure, then the project's base URL, then whatever you type. Whichever it used is shown to you, labelled, rather than left implicit.

If none of those yields a page, the proposal refuses with insufficient context. It does not pick a plausible URL and generate against it.

Propose

Propose a repro composes a defect brief — the issue text, the failing run's step and error, tails of its console and network logs — and asks the model for one test. Alongside the brief it sends your project's reusable flows and your approved business rules, so the repro can reuse a login flow rather than reinventing it.

Three optional controls:

Control	What it changes
Page URL	Overrides the derived starting page
Extra guidance	Free text handed to the generator — up to 2,000 characters
Depth	Single page extracts one page and writes against it. Multi-page journey walks the app live, bounded by your plan's journey caps

The run-as role defaults to whatever the failing test used; you can override it, or ask for a logged-out start.

Nothing is saved. The response is steps to read, plus the grounding it used and every bound it applied — whether or not the bound bit. If your plan's journey caps stopped the walk, that is in the response rather than left to be inferred from a short test.

A proposal is not cheap and not instant: it drives a browser. Expect tens of seconds, up to four minutes.

Approve

Approve, save & run takes the steps you are looking at — including any you edited — and does three things:

Writes the test, tagged regression, marked as AI-authored.
Links it to the issue, so the issue and its evidence travel together.
Runs it once.

That run produces the verdict:

Repro status	What it means
Reproduced	It ran red. You now have the bug under test.
Not reproduced	It passed first time. Flagged, not hidden — either the repro is wrong or the bug is not where the report said.
Passing	It was red and has since gone green. Evidence the fix landed.
Not run	Saved and linked, but this deployment could not run it inline — see below.

Only a repro observed reproduced can ever become passing. A test that never failed cannot prove a fix, and Vera will not let it pretend to.

Nothing auto-closes the issue. The chip changes; the status does not. A green run is evidence, and the decision that a defect is done belongs to a person.

When it is saved but not run

If the project targets a local agent, or the deployment runs tests through a worker queue, the test is written and linked but not executed inline. You get not run, the reason, and the call to make (POST /api/tests/:id/run) — rather than a verdict Vera did not observe.

The same applies if the run outlives the request's three-minute deadline: you get the run id and a pending, never a guess. pending is a transport state and is never stored as the issue's repro status.

One repro per issue

Approving twice is refused unless you explicitly ask to replace. The link is taken atomically, so two people approving at the same moment cannot both win — the loser's test is deleted rather than left orphaned.

A test is capped at 100 steps, its name at 100 characters and its description at 500. A truncation is reported, not applied quietly.

It lands in the review queue

An approved repro is an AI-written test, so on a workspace that requires review it lands pending like any other — and its first run is stamped as the recording a reviewer watches.

That matters more here than elsewhere, because a repro's first run is supposed to be red. Letting it straight into service would put a deliberately-failing test into every schedule and CI gate the project has.

So: propose → approve → review → it starts gating. A reviewer looking at this one will see "the generator's run did not pass", which is the normal and correct outcome, not a reason to reject.

From an agent

Two tools, matching the two halves:

generate_repro_test — proposes, persists nothing. It blocks while a browser works; raise your client's tool timeout rather than firing it twice at one issue.
approve_repro_test — saves, links, runs. Requires confirm: true, and replace: true to take over an existing link.

Read the returned verdict rather than inferring one: pending and not run are real answers and neither means the bug did not reproduce.

There is also a /vera:repro command in the Claude Code plugin.

Limits worth knowing

One proposal per call. No shortlist to pick from.
Nothing is persisted at propose time, so a client that loses the response loses the steps. Approve from the response you are holding.
A named environment that does not exist is a 400, never a silent fallback to the project defaults.
If an approved step uses wait-for-email, the Pro email-inbox feature is required — that one check is conditional on the steps, not on the feature.
The repro-passing transition is a chip on the issue. It sends no notification.