The CI-rider (vera-agent run --drain)

Run your queued Vera tests on your own CI runners — scheduled iOS/Android/web regression that executes on a GitHub-hosted machine and reports back into your Vera workspace. The cloud stays the control plane (dashboard, history, AI); the browser (or simulator/emulator) runs on the runner you already pay GitHub for.

This is a different path from docs/ci-integration.md. That one triggers runs from CI over an API token and the cloud executes them. The CI-rider goes the other way: it boots vera-agent on the runner and the runner executes the queued jobs locally. Use the CI-rider when you want the run to happen on a macOS/Linux runner you control — the only place you get an iOS simulator or a fresh Android emulator for free.

1. What run --drain does

vera-agent run --drain is the batch shape of the agent. It leases and executes queued runs continuously, then exits on its own — no Ctrl+C needed, which is what makes it CI-safe:

• Exits when the queue is empty. "Empty" means --idle-exits consecutive no-work long-polls (default 2, ~25s each ⇒ ~50s of quiet). A lease that returns work resets the counter.
• Exits when --timeout <seconds> elapses. 0 or absent means no cap. The timeout is checked between leases, so a run already in flight always finishes before the process exits — you never strand a half-done run.
• Exit code reflects the aggregate result:
  • 0 — every executed run passed, or nothing was queued.
  • 1 — at least one run failed (fails the CI job).
  • 2 — a fatal agent error (bad/again-revoked token, handshake failure, or Chromium could not be installed). Nothing ran.
  • Cancelled runs (a drain/timeout that interrupted execution) are graceful stops and do not fail the job.

As each run finishes it prints a compact run: <test> — <status> (<ms>ms) line, and a final Drain aggregate: N run(s) — P passed, F failed, C cancelled → exit X line — CI logs are the intended reader.

# Drain the queue, capped at 20 minutes:
vera-agent run --drain --timeout 1200

# No time cap, exit as soon as the queue goes quiet for ~75s:
vera-agent run --drain --idle-exits 3

VERA_API_URL / VERA_AGENT_TOKEN are read from the environment exactly like the other agent commands (see docs/local-agent.md §3).

2. GitHub Action — vera-agent-ci

A composite action lives in this repo at .github/actions/vera-agent-ci. It sets up Node, installs vera-agent from npm, optionally installs Playwright Chromium, and runs the drain with your token supplied as a secret (never echoed).

Inputs

api-url, timeout, idle-exits and both versions are validated before anything installs: versions must be an exact version (or a dist-tag), the two numbers must be non-negative integers, and api-url must be https:// (or http://localhost for a self-hosted runner on the same machine, so a token is never sent in clear). A bad value fails on the first step rather than behaving oddly later.

Why the versions are pinned. This job holds your agent token, which can lease and execute runs in your workspace. With a mutable latest, a future npm publish would run there with no change on your side; actions/setup-node is pinned to a commit SHA for the same reason. Override deliberately if you accept that trade — agent-version: latest still works. Bump the default when you upgrade.

Setup

1. Mint an agent token. In the dashboard, open Settings → Agents, create an agent, and copy the vera_agent_… token (shown once).
2. Store it as a repo secret. Settings → Secrets and variables → Actions → New repository secret, e.g. VERA_AGENT_TOKEN. Store your API root as a secret or variable too (VERA_URL).
3. Add a workflow. Reference the action by owner/repo/.github/actions/vera-agent-ci@ref (or vendor the directory and use ./.github/actions/vera-agent-ci):

# .github/workflows/vera-regression.yml
name: Vera regression
on:
  schedule:
    - cron: '0 7 * * *' # 07:00 UTC daily
  workflow_dispatch: {}

jobs:
  drain:
    # macos-* gives you an iOS simulator; ubuntu-* is cheaper for web/Android.
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      # Queue the runs you want executed — e.g. hit your Vera API to enqueue a
      # suite against an environment whose run target is "agent", or rely on a
      # Vera scheduled run that queues jobs this drain will pick up.
      - uses: your-org/vera/.github/actions/vera-agent-ci@main
        with:
          api-url: ${{ secrets.VERA_URL }}
          agent-token: ${{ secrets.VERA_AGENT_TOKEN }}
          timeout: '1200'
          # runs-on: macos-14 for iOS; install-playwright: 'false' if you only run mobile

The queue must have jobs waiting (or arriving within the idle window) for the drain to do work. Point a Vera environment at the agent run target, or use a Vera scheduled run that enqueues jobs on the same cadence as this workflow. An empty queue simply exits 0.

Do not commit a workflow that drains automatically into this repo — it would start pulling jobs off whatever queue its token points at. The snippet above is a template for your repo.

3. Choosing the runner: macos-* vs ubuntu-*

Web runs work today. Mobile execution (iOS simulator / Android emulator device targets) lands with the mobile phases; the runner choice above is what you will select then. For web-only regression, stay on ubuntu-* — it is ~10× cheaper than macOS (see below).

4. Cost math

The runner minutes are billed to your GitHub account, not Vera's. A representative iOS run:

• macOS runners bill at $0.062/min.
• A 20-minute iOS run ⇒ 20 × $0.062 = ~$1.24 per run on your GitHub bill.
• Public repos run GitHub-hosted minutes for free — scheduled iOS/web regression on a public repo costs nothing.
• Private repos include 2,000 free minutes/month, but macOS is billed at 10× the minute rate, so that budget is ≈ 200 macOS minutes/month — about ten 20-minute iOS runs before per-minute charges begin. Linux (ubuntu-*) minutes count 1×, so web/Android runs stretch the free tier ~10× further.

Rules of thumb:

• Web-only regression → ubuntu-latest, effectively free within the monthly Linux allotment.
• iOS regression → macos-*, ~$1.24 per 20-minute run (free on public repos).
• Keep --timeout tight (e.g. 1200) so a stuck queue can't burn minutes.

See also

• docs/local-agent.md — the full vera-agent guide (enrollment, start, configuration, environment targeting, troubleshooting).
• docs/ci-integration.md — the cloud-execution CI path (trigger runs from CI over an API token; Vera runs them), including JUnit XML export (GET /api/v1/runs/:id/junit and the suite-run variant) for rendering per-step results in the PR checks UI.