Vera

Run Vera end-to-end tests in GitLab CI

Vera ships a GitLab CI job you copy into .gitlab-ci.yml. It starts a run through the Vera CI API, gates the pipeline on blocking failures only, and publishes per-step JUnit results to the merge request. It needs nothing but curl and a POSIX shell.

What comes across

  • The job needs only curl and a POSIX shell — no Node, no jq, no install step.
  • Per-step results are published as JUnit XML, which GitLab shows in the merge request test widget.
  • A dotenv report (VERA_STATUS, VERA_PASSED and the rest) hands the result to later jobs.
  • Set VERA_PREVIEW_URL to $CI_ENVIRONMENT_URL to test the review app your deploy job created.
  • A failed quarantined test passes the job with a warning; only a real, non-quarantined failure blocks.
  • A test that passed only on retry is a warning by default. Set VERA_FAIL_ON_FLAKY to true to make it fail the pipeline.
  • GitLab and Bitbucket run the same gate script, embedded verbatim, and a test in Vera executes it beside the GitHub Action against canned responses and fails if their exit codes differ.

What does not

  • The passed-on-heal opt-in never fires today: a CI-started run has no selector healer attached, so healed is always 0 on this path. The flag is kept so it takes effect the day CI runs can heal.
  • The CI API needs the Pro plan's api-access feature on Vera's cloud; a free workspace's token is refused there. Self-hosted installs have no billing and hold everything.
  • The template runs a fixed set: one test, one suite or the whole project. Running only the tests a change touches is a separate command, vera check.

How to set it up

  1. Mint a token under Settings → CI Tokens. The vera_… value is shown once, so store it as a secret straight away.
  2. Add VERA_TOKEN under Settings → CI/CD → Variables, masked (and protected if your merge request branches are).
  3. Copy ci/gitlab/.gitlab-ci.yml into your .gitlab-ci.yml, or include it.
  4. Set VERA_URL and VERA_PROJECT in the job's variables block.
  5. Leave allow_failure off: the script decides what blocks, and a non-blocking job would hide real failures too.

Questions

What does the Vera GitLab job need installed?

Only curl and a POSIX shell. The template uses a small alpine/curl image, and any image with curl works.

Do Vera results show up in the GitLab merge request?

Yes. The job writes JUnit XML that GitLab renders in the merge request test widget, one entry per step.

Does a flaky test fail the GitLab pipeline?

Not by default. A test that passed only on retry is a warning. Set VERA_FAIL_ON_FLAKY to true to make it fail.

Read more

Start free