Proof of coverage

What has to be true before Vera calls a test coverage.

Generating a test is easy, and getting easier every month. Knowing whether the one you generated proves anything is the part that does not come free with a better model. This is every check Vera makes, and every thing it refuses to count.

in
scenarios the model generated
01
written against the real element map
− selectors that do not exist on the page
02
past the value guard
− no interaction · no assertion · empty selector
03
survived the negative control
− passes both ways ⇒ insensitive
04
counted as coverage
= all the board is allowed to call green

A diagram, not a measurement. The widths show that every gate removes something and names what — a page arguing against invented numbers does not get to decorate itself with any.

01

It is written against your real page

Before a word is generated, Vera loads the page and maps every interactive element to a selector it has checked is unique — preferring test-ids and attributes over text, which can resolve to a child node. The model writes against that map, so it cannot assert on a button that isn’t there, and the run that follows proves each scenario in a live browser rather than trusting the output.

02

Empty tests never reach you

A scenario has to do something and then claim something. One with no real interaction, no assertion, or a selector-bearing step left blank is flagged before it is ever offered to you — because “loads the page, sees the logo” passes on almost any page, including one where the app did the wrong thing.

03

The negative control: it has to be able to fail

The check that actually answers the question. Vera re-runs the test demanding an outcome your app cannot produce — a data table has its pass/reject column flipped, and any other test has one assertion’s expected value falsified. Every one of those controls must fail. If the test passes anyway, its assertions are not wired to your app at all, and Vera records the verdict insensitive against the test.

Only the expectation is ever mutated — never the selector. A control that fails does so because the app disagreed, not because the test could no longer find the button. It stays an explicit action rather than an automatic step after generation, because each control is a real browser run and silently tripling the cost of generating a suite would be a worse surprise than a button.

04

Coverage that refuses to flatter you

The coverage map derives its status at read time from the tests that declare each rule, so it cannot go stale. A rule is covered only when a proven-sensitive test has actually run against it. A rule whose only tests are insensitive is not covered, not partial, and not quietly averaged away — it gets its own bucket, ranked below uncovered, because an uncovered rule at least looks like work to do.

Where the expectations come from

A page can only ever tell you what is on the screen. It can never tell you what the screen is supposed to say — so Vera does not try to infer it. Each project keeps a rule book of what the app must do, and every rule carries the source it was read from. Generation reads approved rules and nothing else, so a rule nobody has accepted cannot end up quietly defining your coverage.

AI proposes · a human approves · only approved rules are ever generated against

Not all evidence is worth the same

Here is the trap, stated plainly, because it is the question a careful engineer asks about this whole idea. Suppose the requirement was “premium customers get 20% off” and the code shipped 10%. A rule read out of that code says 10%. A test generated from that rule asserts 10%. The negative control flips the expectation, the test fails as it should, and Vera marks the assertion sensitive — correctly, and uselessly. Sensitivity is not correctness. It proves the test is wired to your app; it cannot prove the expectation was right.

Which is why the rule book records provenance, and shows it on every rule. A rule read from your implementation agrees with your implementation by construction — that is all it can ever do. A rule transcribed from something written before the code, by a person, is the only kind that can contradict a running app. So Vera reads acceptance criteria straight off your Jira, Linear or GitHub ticket — deterministically, quoting the author’s own words rather than paraphrasing them — and labels every rule with where it came from, so approving one is a decision made with the relevant fact in view.

Ticket acceptance criteria · an API contract · typed by a personcan disagree with your app · written independently of the code
Read from your source · inferred by AIagrees with the code by construction · flagged as such at review time
a ticket is not a guess · Vera will not invent rules out of free prose

Permissions are held to the same standard

A permission rule varies the actor, not the input, so it is checked by running the same test once per role with a per-role expectation. The trap is scoring a “this role must be denied” cell as satisfied whenever the run failed — a selector timeout, a network blip or a typo would all read as correctly denied. So a denial has to be observed: a 401, a 403, or a refusal message your project can name itself. Anything else is inconclusive, which is a third outcome on purpose — not a pass to celebrate, not a failure to fix, but a result that needs a human.

adminmanagerviewer Delete an order ✓ allowedran and passed ✗ deniedobserved a 403 ? inconclusivethe step timed out — never scored as a denial

What Vera will not call coverage

covered partial uncovered insensitive

The four buckets the board reports. Insensitive ranks below uncovered deliberately: an uncovered rule at least looks like work to do, while a rule covered by a test that cannot fail looks finished. Widths illustrate the buckets, not a measured project.

A test that passes whatever the app doesverdict insensitive · excluded from coverage, listed as its own bucket
A run that failed for an unknown reason, on a “must be denied” roleoutcome inconclusive · never scored as a denial
A generated scenario that interacts with nothing, or asserts nothingflagged by the value guard before it is offered
A business rule the AI proposed but nobody approvedinvisible to generation, and not in the denominator
A repair that a model was confident aboutdiscarded unless the test actually goes green on re-run

Green is not one thing

A suite that reports “all passing” is telling you less than it sounds like. A test can pass because your app is fine, because it failed and a retry saved it, or because a selector went stale and self-heal rewrote it mid-run. Those are three different facts about your release, and collapsing them into one green tick is how a pipeline quietly stops meaning anything.

Healing only ever rewrites the locator, never the expectation — the assertion still had to hold against your app, which is why a healed pass is not a failure. What it can hide is a semantic change: if a Delete customer button disappears and the healer binds the step to a neighbouring control, a broad assertion can still pass while the interaction the test was written to exercise is gone. So Vera reports it separately, names the rewritten selector before and after, and lets you gate on it — teams typically fail on healed runs for a release branch and warn on feature branches.

PASSdid what it says, first time
PASS ON RETRYfailed, then passed — flakyopt-in gate
PASS ON HEALa stale selector was rewritten mid-runopt-in gate
INSENSITIVEpasses whatever the app doesnever counted as coverage
FAILyour app disagreedblocks the merge
quarantined failures never gate · fail-on-flaky · fail-on-healed

Straight answers

The questions you were going to ask anyway.

Written the way they actually get asked, not the way we would like them asked. The answers are the mechanism rather than the marketing — and five of them concede a real limit, because a reader who finds one himself stops believing the rest.

Why not just use a model

Claude can drive a browser now. Why do I need you at all?

For looking at a page, you do not — an agent with a browser is genuinely better at that, and cheaper. Use it.

What that session is not is a gate. It is ephemeral, unauthenticated unless you walk it through a login every time, and it leaves no record. Nobody is prompting it at 2am when a bad merge lands. Vera runs the same flows credentialed, as a chosen role, against a chosen environment, on a schedule and in CI — and keeps the result, so there is something to compare against next week.

It is also not either/or. Vera ships an MCP server, so you can ask Claude to run the checkout suite against staging, read the failure, and open the issue — it just does it through the thing that keeps the record.

Could I not just ask a coding agent to write Playwright tests for me?

You can, today, and it will hand you something plausible. Then you own the part that is actually hard: running it on every pull request as the right user, against the right environment, on real browsers, with credentials that work; keeping it green when the DOM shifts next sprint; and knowing whether any of it proves something.

Writing the first draft is the cheapest step in the job. Vera is the rest of it.

This is a wrapper around a model.

The generation feature is a wrapper, deliberately and thinly, so you can swap the model the day a better one ships. We would rather be a thin wrapper than a stale prompt.

The rest is not. Turn AI off entirely and Vera still records flows, runs them on three browser engines, holds a browser pool, seals credentials, keeps video and traces, diffs screenshots against baselines, runs on a schedule, gates pull requests, and exports back to code. The model writes first drafts. It does none of that.

Will the next model not be good enough to make your guardrails pointless?

A better model makes a generated test wrong less often. It does not tell you whether this test is wrong — that is a different kind of question, and no amount of model quality answers it, because the model grading its own output is the problem, not the solution.

The negative control is an experiment, not a prompt: run the test against data it should reject and see whether it fails. That check gets cheaper as models improve, and never redundant. If anything, the better generation gets, the more tests you will have and the less any human will read them.

Why trust what it generates

How do you know a generated test covers the use case, and not just the page?

Because the use cases are written down. Vera keeps a per-project rule book of what your app is supposed to do — each rule proposed from your source with file-and-line evidence, and approved only by a person. Coverage is then measured against those rules, not against how many tests got generated.

The distinction matters because a page can only ever tell you what is on the screen. It can never tell you what the screen is supposed to say. That is the one thing no amount of DOM reading solves, and it is why the rule book is human-approved rather than inferred.

What stops a generated test from passing while testing nothing?

A negative control. Vera re-runs the test demanding an outcome your app cannot produce — a data table has its pass/reject column flipped, and any other test has one assertion's expected value falsified. Every one of those controls must fail. If the test passes anyway, its assertions are not connected to your app at all, and Vera records the verdict insensitive.

An insensitive test is then refused as coverage. It gets its own bucket on the coverage map, ranked below uncovered on purpose: an uncovered rule at least looks like work to do, while a rule covered by a test that cannot fail looks finished and is not.

How do I know the negative control is not theatre?

Because it is a real browser run, not a static check, and you see each control, which assertion it falsified, and whether it failed. A control that passes is displayed as the failure it is. Only the expectation is mutated — never the selector — so a control that fails does so because the app disagreed, not because the test could no longer find the button.

Here is its limit, since you will find it anyway: an assertion that only checks something is present has no expectation to falsify. A test built entirely from those is reported unknown, not sensitive — Vera will not award it a clean bill of health it did not earn, and tells you to assert an expected message, value, URL or status instead.

Self-healing sounds like it hides regressions. If a selector changed because we broke the page, you just papered over it.

Healing replaces the locator, never the expectation. The step is retried with a new selector and the same assertion, and that assertion still has to pass — if you broke the page, it fails and the run stays red. An expectation like a URL check is deliberately never re-derived from the live page; only a locator is. That asymmetry is the entire safety property.

Where you are right: healing can mask a deliberate removal — a button you meant to delete gets found again under a new name. So heals are capped at three per run, each one is recorded on the report with the old selector and the new one, and an in-run heal never rewrites your stored test. It gets the run past a renamed class; it does not quietly edit your suite.

locatorbutton.submit-btn[data-testid=checkout-submit]replaced
assertionurl matches **/successuntouched — still has to pass
An AI editing my test suite unattended is a non-starter.

Then leave it off, which is where it already is. Auto-repair is opt-in per project, needs a paid plan and your own AI key, and only ever touches saved tests.

Switched on, it escalates heal, then suggest-a-fix, then diagnose-by-reading-your-source — and a fix persists only after the test has actually gone green on a re-run. Nothing is accepted because a model sounded confident. Every change lands in an append-only audit log and is revertible.

AI-generated tests will be flaky and I will drown in false alarms.

Some will be, and so are hand-written ones — flake is a property of end-to-end testing, not of who wrote the test. Pretending otherwise is how suites get abandoned.

So Vera measures it instead. Every test carries a 0–100 flake score, a pass-on-retry is flagged rather than hidden, and quarantine lets a flaky test keep running and recording while it stops blocking the pipeline. The GitHub Action fails a pull request only on real failures; quarantined and passed-on-retry surface as warnings. The goal is not zero flake. It is that red keeps meaning something.

What happens when the AI gets a step wrong?

You get an editor, not an export folder. Every generated step lands in the visual builder, so you fix the one that is wrong and move on — no code, no regenerating the suite, no waiting for a better prompt.

When Vera repairs something itself, the fix only persists after the test has actually gone green. Nothing is accepted because a model sounded confident, and every change is recorded and revertible.

No-code means I hit a ceiling in week two.

There is a ceiling, and it is worth knowing where it is. Below it: 27 step types, if/else and loops with element, URL and variable conditions, project and per-environment variables, reusable flows shared across tests, HTTP calls with JSON path extraction, and values captured from one page and used on the next.

Above it: if a test needs arbitrary code in the middle of it, Vera is the wrong shape and you should not fight that. Export the test as runnable Playwright and take it with you — that path is there precisely so hitting the ceiling costs you a click rather than a rewrite.

Why pay for it

I already pay for Claude and Cursor. Why add another bill?

You pay them to write code. Neither of them remembers that checkout worked yesterday, notices that it stopped working this morning, or blocks the merge that broke it.

And you are not paying twice for the model: AI in Vera runs on your existing Anthropic, OpenAI or Google key. We resell no tokens and meter no credits, so the subscription buys the part your assistant does not do.

I bring my own AI key. So what am I actually paying you for?

Everything that is not the model. Free is real and generous enough to answer "does this work on my app" — unlimited tests and runs, the editor, the recorder, AI generation on your key, the local agent, site audits, video and failure traces, and shareable reports.

You pay when it has to run without you: scheduled runs, environments with their own URLs and sealed credentials, the CI API and pull-request gating, quarantine, visual regression, alerting. And you pay more when more than a couple of people depend on it: custom roles, an audit log, the coverage map, cross-project chains. That is the bill — operations, not intelligence.

Playwright is free and my team already knows it.

So is the compiler, and teams still buy build systems. The cost of end-to-end testing has never been the runner.

It is that the person who knows which flows matter — support, QA, the founder — usually cannot write the code, and the engineers who can have shipping to do. It is that a suite decays every sprint until someone declares bankruptcy on it. If your team is genuinely maintaining a healthy Playwright suite today, keep it, and point Vera at the flows it does not cover.

My team could build this in a sprint.

You could build a Playwright runner in a sprint. Then you own a browser pool that does not exhaust the box, artifact storage with retention, a step schema, retries and flake scoring, environments and sealed credentials, screenshot baselines with an approve/reject review, a live progress stream, role-based access, an audit log, and an interface a non-engineer will actually use.

That is not a sprint. It is a product, with an on-call rotation. If all you need is Playwright in CI, build it — genuinely, that is the right call for some teams. Vera earns its price when the people who know what to test cannot write the tests.

Why bet on you

I don't want AI writing my tests. Is Vera any use to me?

Yes, and you never have to switch it on. Record a flow by clicking through your app, import a suite you already have from Cypress, Playwright, WebdriverIO, Selenium, TestCafe, Postman, a HAR capture or an OpenAPI spec, or build the steps by hand in the editor.

Nothing in the recorder, the builder, the runner, the scheduler, the CI action or the reports calls a model. Even self-repair is off until a project opts in. AI is a feature here, not a dependency.

What does Vera do with my code, my credentials and my data?

AI runs on your own Anthropic, OpenAI or Google key. Keys, login credentials and captured sessions are sealed before they touch the database and are write-only over the API — they can be set, never read back. We resell no tokens and meter no credits.

When Vera reads your repository to propose config or rules, it reads: it can open, grep and glob files, and it cannot write one or run a command. And the local agent executes tests on your machine, next to localhost — no tunnel into your network, no database access.

We have 400 Cypress tests. The migration is the real cost, not your tool.

Correct, so do not migrate. That is the honest answer and it is also our advice: your Cypress suite already works, and converting it buys you nothing but risk.

Point Vera at the flows you never got round to covering — the ones that keep breaking in production because nobody had time to write them. If you do want a head start, the importers turn Cypress, Playwright, WebdriverIO, Selenium, TestCafe, Postman, HAR captures and OpenAPI specs into editable steps. Be clear-eyed about what that gives you: the steps, not a passing suite. You will still run them and fix what breaks.

If a test passes and we ship the bug anyway, what was I paying for?

Then the test did not cover that case, and no tool on earth can promise you otherwise. Anything claiming to guarantee you will never ship a bug is selling you something.

What Vera does is narrow the gap and refuse to hide it: coverage is measured against a rule book you approved rather than against a test count, a test that cannot fail is not counted, a denial that was not observed is not scored, and an uncovered rule is shown as uncovered. You will still ship bugs. You should be able to see, afterwards, exactly which rule nobody had a sensitive test for.

You are a small company. Why would I put my release process on you?

It is a fair thing to price in, and the mitigations are concrete rather than reassuring words. Every test exports to runnable Playwright, WebdriverIO, pytest or TestCafe code, and every run to JUnit XML, so your suite leaves in a form your team maintains by hand — no proprietary format to reverse-engineer. The local agent already executes those tests on your own machines rather than ours.

What that adds up to: if we disappear, you lose the dashboard and the history, not the tests. That is the most any vendor can honestly offer, and rather more than most do.

Get started

Describe what to test — and let Vera build it, run it, and keep it green.

Point Vera at your app, write one sentence, and watch real tests generate and pass in seconds. Free plan, your own AI key, no credit card.