vera check — what did I change, and is it still covered?

vera check reads your diff, maps it onto the tests and rules of the linked Vera project, and tells you three things:

1. which tests to run for this change,
2. which rules the change touches that nothing adequately tests (gaps),
3. which files no join could place at all (unmapped — see §4, and read it before you believe anything on this page).

With --run it then runs exactly that set. With --strict it can fail the build on a gap. Both are off by default, and this document explains when to turn them on.

npx vera-agent check                 # read-only: map and report
npx vera-agent check --run           # …then run the impacted tests (cloud)
npx vera-agent check --run --local   # …run them on THIS machine instead
npx vera-agent check --run --strict  # …and fail the build on an uncovered rule

Everything here is deterministic — code-level joins and heuristics, no AI. It needs no BYOK key and burns no AI credits.

Write vera-agent, not vera: the published package installs both bins, but npx vera fetches an unrelated squatted package, so the short alias is safe only once vera-agent is installed.

1. What it reads

Changed files are the union of four sources, because a developer who has not committed yet is the main user:

git diff --name-only <base>...HEAD   +   staged   +   unstaged   +   untracked

--files a,b,c skips git entirely and maps exactly those paths.

<base>...HEAD is three dots — "what this branch added since it diverged", not "how this branch differs from the base right now" — so commits that landed on the base after your branch point are correctly excluded.

Renames are reported as both paths (--no-renames is deliberately on): the source path is the interesting half, because a moved route leaves tests pointing at the old one.

How the base is chosen, and the one way that bites in CI

--base defaults to origin/HEAD, then origin/main, then origin/master — the first that resolves wins. Two cases, and they behave differently:

That second row is the CI trap. On a shallow checkout (actions/checkout defaults to fetch-depth: 1) there is no origin/main and there are no uncommitted changes, so an un---base'd vera check examines zero files, reports nothing impacted, and exits 0 — a green that proves nothing. In CI, always pass --base explicitly and fetch enough history for it to resolve (§9). An explicit base that is missing fails loudly, which is the behavior you want from a gate.

Two more conditions are reported rather than hidden: a repo with no commits yet (untracked files only), and any git command that fails (its files are declared absent from the report instead of being silently omitted).

Monorepos — the frame of reference

The server matches your diff against a project whose files may live in a subdirectory. git diff emits paths from the repo root (apps/web/app/…), so a project rooted at apps/web needs that prefix stripped before anything can line up.

That prefix is a project setting, not a CLI flag: open the project's Tests page → project menu → Repo path, and set e.g. apps/web. Leave it empty when the repo root is the app. .. is rejected rather than normalized away.

Note that .vera/config.json also carries a repoPathPrefix, written by vera init. It is not what the impact join uses — check sends paths as git reports them and the server normalizes with the project setting, so that both sides of the evidence join use literally the same function. If your monorepo project reports everything as unmapped, the project setting is the thing to check.

vera init also writes repoPathPrefix into the committed .vera/config.json, but the authority is the server-side setting — that is what strips the prefix from your paths and from rule evidence at map time. Set one and not the other and nothing maps, which looks identical to "you have no coverage". When that exact shape appears (a warm map, zero impacted tests, and changed files under the declared prefix) the CLI asks the map one extra read-only question with the prefix stripped; if that version matches tests, it says so and names the setting to change. It proves the mismatch rather than guessing at it, and never fires on the happy path.

2. The three joins

They are applied in this order, and the order is a claim about which one carries the weight:

(a) is primary because (b) is empty on most projects. The rule join depends on tests being linked to rules (ruleIds), and only the authoring paths write that — a fresh or imported project has none. A map built on the rule join alone would answer "impacted: 0" for every diff, which is the failure mode this design exists to avoid.

(a) Route heuristic, in detail

Two tiers, and every evidence line prints which one it came from:

• high — a framework route tree. The file sits under an app/, pages/ or routes/ directory, so the framework itself defines the mapping: Next.js App and Pages routers, SvelteKit, Remix flat routes, and plain Express/Rails-style route modules. Route groups (marketing), parallel slots @modal, private _folders and Remix's pathless _layout segments carry no URL segment and are dropped; every dynamic dialect — [id], [...slug], [[...slug]], {id}, {{var}}, :id, $id, a trailing ?, and a * wildcard — collapses to one placeholder, so orders/[id]/page.tsx and goto /orders/42 match. Under app/ only reserved filenames count (page, +page, route, +server, layout, index, _index, error, loading, template, default), so app/components/Button.tsx invents no route — a heuristic without that rule would claim /components/button and be confidently wrong.
• guess — a filename. No route tree, so the file's own name (minus decorating suffixes) and its non-generic parent directory become candidates: OrdersService.ts ⇒ /orders. It exists because most repos are not route-tree-shaped and an empty report helps nobody. It is a hint, not a claim.

On the test side, the URL-bearing steps are exactly goto, api-request and mock-route. extract is not among them — an ExtractStep has no url field, so a test that only extracts is invisible to this join.

resolveFlows runs first. A flow-factored test hides its goto behind a use-flow step, so flows are expanded before any URL is read — otherwise the projects that have been tidied up the most would be exactly the ones the map goes blind on. If a flow cannot be expanded (missing, or nested past the depth limit) the test is still matched on its unexpanded steps and the degradation is named in warnings.

Printing a framework fact and a filename guess identically is the fastest way to stop believing the output, so they never are: app/orders/page.tsx ⇒ /orders is a fact about Next.js, OrdersService.ts ⇒ /orders is a guess, and a test whose evidence is all guesses is additionally labelled guess only.

(b) Rule evidence, in detail

Only approved rules participate — a proposed rule is not the specification yet, and getApprovedRules is where that human gate lives.

Paths are normalized by the same function at write time (when discovery stores a rule, stripping the discovery root or the throwaway GitHub snapshot directory) and at read time (on your changed files, stripping the project's Repo path). Being literally the same function is the only reason the join can work. Legacy rows whose absolute prefix is unrecoverable are rescued by a longest-common-suffix fallback.

Rules with no file path are excluded and counted, never silently zeroed. OpenAPI-derived rules are dropped up front by source; Postman-derived rules carry the same endpoint-only evidence and drop out the same way, and ticket-derived rules cite a ticket key rather than a file. The Map coverage block reports how many, so a rule book that cannot participate says so instead of contributing an invisible zero.

This is a reporting limit, not a coverage limit: a ticket-derived rule is still covered by any test that declares it (tests.ruleIds) — what it cannot do is be reached backwards from a changed file.

Two consequences worth internalizing, because they bound what --strict can ever tell you:

• Only rules matched by this join can be reported as a gap or as partial. The route heuristic never contributes a rule. A rule whose evidence paths are not in your diff is simply absent from the report — not "fine".
• Therefore a rule with no file evidence at all can never be a gap, no matter how central it is.

(c) Always-run tag

Tests tagged always-run are reported in their own section and are never mixed into "impacted". "The map found these" and "you told me to always run these" are different claims, and blurring them would inflate the map's apparent hit rate.

They are also not executed by --run unless you pass --with-always-run.

3. Map coverage — the number to watch

Map coverage — 12 of 40 tests mapped (30%)
    by route: 11   by rule link: 3   (a test can be both)
    approved rules: 8 — 6 carry file evidence the diff can match, 2 do not

mapCoverage measures the project, not your diff — it is the same number whatever you changed. 12 of 40 is the honest headline: 28 of your tests are invisible to this map today. It climbs as tests gain URL-bearing steps and rule links, and until it is meaningfully above zero, --strict is not for you (§8).

4. Honest limits — read this part

Unmapped is not the same as safe.

A file that no join could place may still be covered by a test the map cannot see. The map is a heuristic over URLs and cited file paths; it is not a coverage prover. An empty impacted-test list means the map is blind here, never that this change is safe to ship. Both check and affected say so in their own output, and this is the single most important sentence on the page.

Specifically, here is what the map cannot see:

• A test that reaches the page by clicking, not by goto. Log in, navigate through the UI, assert — a perfectly good test with one URL in it, and the route join can only match that one.
• Backend and shared code with no route of its own. A pricing module, a queue consumer, a migration. Tier 2 will guess a route from the filename, and the guess is usually wrong; absent a rule citing the file, it is unmapped.
• Anything your rules do not cite. The rule join sees evidence[].path and nothing else. Rules imported from OpenAPI or Postman cite endpoints, not files, and can never match a diff.
• Rules nothing declares. Gap severity is computed from the tests whose ruleIds include the rule. That column is written only by the paths where a human or an agent authored the test — importers, repro and seed paths carry no rule reference to write — so a project can have good tests and still show every matched rule as uncovered.
• Deleted files. They appear in the diff and are matched like any other path; a route you removed can therefore still "impact" the test that used to cover it. That is usually what you want, but it is a match on a path, not on a page.

And two shape limits:

• Heuristics are heuristics. A guess line is a hint. Read the confidence column before you treat the list as authoritative.
• A cold start is reported as a cold start (§6) — you get the one command that fixes it, not an empty table that reads like a clean bill of health.

5. gaps vs partial — and why the difference is load-bearing

When a changed file matches a rule's evidence, that rule's protection is derived from the tests that declare it. Four possible verdicts:

insensitive sorts as worse than uncovered on purpose: an uncovered rule at least looks like work to do, while an insensitive one looks like protection and is not.

--strict never fails on partial. An ordinary covering test that is not dataset-backed is the healthy state of almost every real project — earning covered requires a dataset and a passing sensitivity check. If partial counted, this gate would be red on a healthy repo from day one, and a gate that is red on a healthy repo gets || true'd in week two. At that point you have a gate that does nothing and a team that has learned to ignore it, which is strictly worse than never having shipped it. So partial is reported as its own, weaker bucket and never changes the exit code.

6. Cold start — what it looks like, and the one command

The map is cold when the project has no tests at all, or when no test is reachable by either join (no URL-bearing step, no approved-rule link). That is the likely state of your first run, so it is handled before anything else and prints a fix rather than a table:

Cold start — the impact map cannot answer for this project yet.

  40 tests exist, but 0 of them are mapped:
    · 0 have a URL-bearing step (goto / api-request / mock-route) — the primary join is blind without one
    · 0 are linked to an approved rule (0 approved rules in this project)

  The one command that fixes it:

      vera init --import all

  The tests you have carry no URL, so no changed file can be matched to them.
  Imported specs bring their own `goto`s — the one thing the route join needs.
  Deterministic, no AI, no credits. If the repo has nothing importable it says so.

  Rules are the second join: approve a rule book at <host>/projects/<id>/rules
  and link tests to rules, and `vera check` starts reporting gaps as well as tests.

There are two branches — "no tests at all" and "tests exist but none is mapped" — and both name exactly one runnable command, vera init --import all.

Read-only vera check still exits 0 on a cold map: it produced an honest report. --run on the same project exits 2, because it verified nothing (§9).

7. vera affected <file> — the reverse lookup

npx vera-agent affected src/lib/pricing.ts

Same joins, one file, inverted question: which tests cover this? It exists because that is the question you ask with a file open, and answering it should not require an editor plugin. Same --url / --project / --token / --json flags as check; it exits 0 on a report, 1 on usage or an unlinked repo, and 2 when the report could not be fetched.

8. --run — two honest modes

There is no free ride here: running the impacted tests means executing them somewhere, and the two places behave differently.

(a) Default — server-side

npx vera-agent check --run

The impacted test ids go to POST /api/v1/projects/:id/run in one bounded request, using the same vera_… token vera init already provisioned.

It runs cloud-side regardless of the project's run target. Even if the project (or the selected environment) is configured with runTarget: agent, this path executes in the Vera server process. That means it cannot reach a host that is private to your machine — http://localhost:3000 will not resolve. The CLI prints this sentence on every run rather than letting you discover it as a wall of connection-refused failures.

Every run is a real run: it is persisted with artifacts, it rescores the test's flakiness, it can prove a repro test green, and it fires your notification rules — the same reporting any CI-triggered run gets.

• Bounded at 50 tests per run, and the bound refuses. A larger impacted set is not trimmed to the first 50; nothing runs, and the CLI tells you the count and what to do (--base <closer ref>, or run the project as a whole). A "pass" built from 50 of 84 tests is exactly the answer you must not get from a merge gate. The server enforces the same limit (400 too_many_test_ids), so a hand-rolled client cannot get the trimmed answer either.
• --env <name> selects an environment profile; --timeout <s> caps the wait (default 1800). A per-PR preview URL is not a flag here yet — use an environment profile.
• A test the server did not report on is listed as NOT RUN, never dropped: a shorter list must not read as a complete one.

(b) --local — on this machine

npx vera-agent check --run --local

The same tests, executed in this process through the local runner (the same path as vera-agent verify), so they reach http://localhost:3000 and anything else your machine can see. The control plane still creates every run row and holds the artifacts, so local and cloud runs share one history.

This needs the agent credential — vera_agent_…, from vera-agent login — which is a different credential from the vera_… CLI token used to read the map. They sit behind different URL fences on the server, so presenting the wrong one is a bare 401.

If this machine is not enrolled, or is enrolled against a different deployment, or has no Playwright browser, the command exits 1 and tells you what to install or run — and it checks that before fetching the map, so you learn in a second rather than after a full round trip. It never falls back to draining the agent queue: a drain claims any unpinned job in the org, and an empty queue exits 0 — a green built from nothing at all.

9. --strict — opt in, later, on purpose

--strict makes the command exit 4 when the diff touches a rule with no adequate coverage (a gap — §5).

It is off by default, and you should leave it off until this project has measured non-zero map coverage. Run vera check on a few real branches first and watch the Map coverage line. If it says 0 of 40 tests mapped, --strict would be red on every pull request from day one — and a gate that is red on a healthy repo gets || true'd in week two, at which point you have a gate that does nothing and a team that has learned to ignore it.

Turn it on when:

1. mapCoverage is meaningfully above zero, and
2. your tests are actually linked to rules (an unlinked rule book means every matched rule reports uncovered), and
3. the current gap list is one you intend to keep at zero.

--strict never fails on partial — see §5 for why that is deliberate and what it costs. Only uncovered and insensitive rules count as gaps.

The --strict verdict line is printed whether or not the flag is on, so the behavior of the gate is legible from a run that did not trip it.

10. Exit codes

These are vera check's. (vera affected uses the same 0 / 1 / 2 subset.)

Precedence when several apply: 3 › 2 › 4 › 0. A proven break outranks "I could not answer", which outranks "you have no proof for this rule". The output lists all findings regardless of which code is returned.

Note the asymmetry in code 2: read-only vera check on a cold project exits 0 (it produced an honest report), while --run on the same project exits 2 — because it verified nothing, and a gate that verified nothing must not return the exit code of a passing gate.

11. In CI

The PR-gate shape is vera check --run --strict, with --strict added only once §9 applies:

- uses: actions/checkout@v4
  with:
    fetch-depth: 0 # `--base origin/<base_ref>` must resolve — see §1

- name: Impacted tests
  run: npx vera-agent check --base origin/${{ github.base_ref }} --run
  env:
    VERA_API_URL: ${{ vars.VERA_API_URL }}
    VERA_PROJECT_ID: ${{ vars.VERA_PROJECT_ID }}
    VERA_TOKEN: ${{ secrets.VERA_TOKEN }}

fetch-depth: 0 is load-bearing. actions/checkout defaults to a shallow clone (fetch-depth: 1), which fetches no base branch. None of origin/HEAD, origin/main or origin/master then resolve, and a CI worktree is clean — so zero files reach the impact map. The map is warm, so this is not a cold start; it would simply report "nothing impacted".

vera check --run refuses that state with exit 2 rather than exiting 0: zero files because no base resolved means the diff was never computed, and the absence of an answer must not be reported as the answer "no". Zero files with a base that did resolve is a genuinely clean tree and still exits 0. If you cannot deepen the clone, name the base explicitly with --base.

These three variable names are the CLI's, and they are not the ones the vera-run action or the shell gate use (VERA_URL / VERA_PROJECT / api-url / project — see CI integration). Copying one snippet's env block into the other is the mistake to avoid.

--json emits one document on stdout (progress and warnings go to stderr) carrying the full report, the per-test run results, exitCode and exitReason, plus the CLI-side context the server cannot know: which base was used, whether it was explicit, and the per-source file counts. That is what a bot should read rather than scraping the human output.

The PR comment itself does not exist yet. Posting this report back onto the pull request as a sticky, pre-triaged comment is planned (ENGAGEMENT_PLAN G10) and is not shipped. Today vera check is a step that passes or fails and prints to the job log; anything richer, you build on --json.

Plan requirement: /api/v1 needs the Pro api-access feature. A free cloud workspace gets 401 token_rejected on both the impact map and the run endpoint; the MCP tools on /api/mcp keep working. The CLI says exactly this instead of printing a bare 401. Self-hosted and local installs have no billing — every feature is on.

12. vera watch --affected — the same map, on every save

npx vera-agent watch src --affected

The save loop. On every debounced save it re-reads your diff, maps it, and runs exactly the impacted tests on this machine — the same selection vera check would make, executed the same way --run --local executes it (literally the same function; two copies would eventually disagree about one diff).

It runs locally on purpose. A queue round trip is a lease long-poll away from an answer, and the queue cannot reach the http://localhost:3000 where the code you just saved is running.

What changed always comes from git, never from the watcher: the watcher only decides when to look, and the file set is the same four-source union vera check uses. So the loop and the PR gate cannot disagree about the same diff.

Debounce and concurrency. A burst of saves (a formatter, a multi-file refactor) coalesces into one pass. A save arriving during a pass is queued, not interleaved: exactly one follow-up pass runs afterwards, however many saves landed, and it re-reads git at that point so it evaluates the newest tree. Dropping those saves would leave your latest edit unverified behind a green from the previous one; cancelling the in-flight run would litter history with cancelled runs and can livelock under a fast save cadence.

A cold map is announced on every pass, not silently skipped — a watch loop that quietly does nothing looks exactly like a passing one. The full cold-start banner prints once per session; after that every pass carries a one-line COLD MAP notice, and an empty selection from a cold map is refused rather than reported as "nothing to do".

It replaces the watch tag — and resolves what that tag meant

The watch tag carried two meanings that cannot share one flat, org-wide namespace:

1. "Include this test in save-loop runs" — a membership marker, intent about how a test is run.
2. "An org-wide tag selector" — --tag <anything> resolves every test in the org carrying that tag, across every project.

A save loop is per-developer, per-repo, per-directory; meaning 2 is org-wide, so a colleague's watch-tagged test ran on your laptop when you saved an unrelated file. And because watch sat in the same namespace as smoke or checkout, anyone tagging a test watch for a human reason enlisted it in everyone's save loop. Neither meaning varies with what you changed.

Nothing has been taken away:

Only the magic default is deprecated, and it is deprecated out loud rather than changed underneath you.

See also

• docs/getting-started.md — vera init, the config files, minting the token.
• docs/local-agent.md — enrolment (vera-agent login) and everything else the local agent does.
• docs/ci-integration.md — triggering whole-project runs from CI over the same token.
• docs/mcp.md — check_coverage, the same report over MCP, for an agent that would rather ask than shell out.