Test engines — browse guides

Test engines (Playwright, WebDriver & Appium)

Choosing an execution engine, the capability matrix across the three, how selectors translate, known limitations, and how to set up a Selenium grid.

Vera runs every test through an engine-agnostic executor that talks to the browser only through a small driver seam (VeraPage/VeraSession/VeraDriver in @vera/runner). Three backends implement that seam:

  • Playwright (default) — the in-process driver. Full-featured: video, traces, console/network capture, accessibility audits, visual regression.
  • WebDriver — a bare W3C WebDriver client (via the webdriver npm package, WDIO's protocol client — no WDIO test-runner) that drives a remote Selenium server or grid. This is the door to Selenium grids today and to device clouds (BrowserStack, LambdaTest) — all remote W3C endpoints reached the same way.
  • Appium — the native-mobile flavor of the same W3C client. It dials an Appium server (UiAutomator2 for Android, XCUITest for iOS) and drives a native app instead of a web page. It shares the WebDriver machinery but resolves native locators and speaks the native tap/swipe/launchApp verbs. See Native mobile (Appium) below.

The WebDriver engine is a paid feature (multi-engine, Team plan and above; local/self-host installs resolve to enterprise, so everything is on). Selecting webdriver on a lower plan is refused with the standard upgrade prompt.

Choosing the engine

Resolution order (highest wins), decided server-side at run time:

  1. The selected environment's engine (Environments → Engine).
  2. The test config's engine (Run settings → Engine on the test builder).
  3. playwright (the default).

The Selenium server URL is resolved the same way: the environment's seleniumUrl, else the test config's seleniumUrl. A webdriver run with no Selenium URL configured fails at launch with a clear error.

Queue / agent execution note: an environment-level engine override is not yet threaded into the queue/agent job payloads — those paths re-read the persisted test.config. Set the engine on the test config to run webdriver through the queue or a vera-agent. In-process (synchronous) runs honor both.

Capability matrix

CapabilityPlaywrightWebDriver
Step execution (click/fill/select/check/hover/press/wait/goto)
Assertions (visible/hidden/text/url/title/value)
extract steps (text / attribute / input value / URL)
Screenshots✅ (full-page + viewport)✅ (viewport only — fullPage degrades)
Self-heal (AI selector repair)✅ (rich element map)✅ (raw DOM snapshot via getPageSource)
api-request steps✅ (browser request context)⚠️ fetch with forwarded same-origin cookies (see below)
Video recording
Playwright traces
Console / network capture (failure report)
axe accessibility audits
AI design review❌ (no screenshots pipeline change; review still runs on captured shots)
Visual regression diffs✅ (from viewport screenshots)
UI-heuristic asserts (assertToast/assertModal/assertValidationError/waitForLoadingComplete)❌ — fail with a clear "only supported on the Playwright engine" message
Device emulation (device)❌ (window is sized to the viewport only)
Real-Chrome mode / consent-overlay dismissal / cursor overlay❌ (skipped)
Captured auth session (storageState) injection❌ (not injected — see limitations)

When a capability is off, Vera reports its absence — it does not pretend. A WebDriver run simply has no video/trace/console artifacts.

Selector translation

Vera selectors are CSS by default, but tests (and the recorder) also emit some Playwright-specific forms. On the WebDriver engine they are mapped to W3C locators:

Vera selectorWebDriver strategy
#id, .class, [attr=x], …CSS selector (as-is)
data-testid=fooCSS [data-testid="foo"]
text=FooXPath .//*[normalize-space(text())='Foo' or contains(normalize-space(.), 'Foo')]
//…, (//…), xpath=…, ./…XPath
role=button[name="Submit"]XPath union approximation — button / link / heading / textbox only (see below)
role=<other> (checkbox, radio, combobox, slider, …)❌ not supported — clear error listing the supported roles
label=…❌ not supported — clear error (use CSS/XPath/data-testid)
a >> b (chained)❌ not supported — clear error

text= divergence: the XPath approximates Playwright's substring, whitespace- normalized match but is case-sensitive (XPath 1.0 has no case-insensitive contains). A text= selector that relied on case-insensitivity may not match on WebDriver — prefer a CSS or data-testid selector for cross-engine tests.

role= support (E10.5): the recorder and the AI generator emit ARIA role locators (role=button[name="Submit"]) as a top-tier, refactor-resilient selector. On Playwright these run natively (its own role engine). On WebDriver they are approximated as an XPath 1.0 union for a practical subset of roles — button, link, heading, textbox — matching the accessible name against the element's normalized text, aria-label, title, @value (buttons), or an associated <label> (textboxes, via label[@for]input[@id]). Divergences vs Playwright: the name match is case-sensitive (XPath 1.0 limitation) and exact (no substring); only the bare role=<role> and a single exact role=<role>[name="…"] are translatable — flagged/regex names ([name="x"i], [name=/re/]) and other role attributes ([checked], [level=2], [pressed]) throw a clear error, as do roles outside the supported subset. Prefer data-testid for cross-engine tests where the label is unstable.

api-request on WebDriver

WebDriver has no request-context API. Vera's WebDriver driver issues api-request steps with plain fetch, forwarding the session's cookies for same-origin URLs (read via the W3C Get All Cookies command) so a logged-in role's session applies. For a cross-origin URL it sends only the step's own headers. This is a close-but-not-identical match to Playwright's context-request semantics.

Known limitations (initial WebDriver support)

  • No captured-session (storageState) or httpCredentials injection — a WebDriver session starts fresh. Log in via steps, or use an engine-appropriate auth flow. (Playwright's api-login capture does not transfer.)
  • Screenshots are viewport-only; fullPage is ignored.
  • No device emulation; the window is sized to the run's viewport.
  • One window per session (multi-viewport passes reuse the same session).

Native mobile (Appium)

The Appium engine drives a real native app (.apk/.ipa) on an emulator, simulator, or a device cloud — not a mobile web page. It is the native-mobile flavor of the WebDriver family: the same W3C client and self-heal machinery, but pointed at an Appium server with UiAutomator2 (Android) or XCUITest (iOS) drivers.

When it is selected

The Appium driver is chosen whenever a run's config carries a native mobile target (TestConfig.mobile — a MobileTarget with a platform, optional deviceProfile, and the uploaded app artifact). A native target takes precedence over the engine field: a mobile test is inherently a W3C run and never executes on Playwright. Absent mobile, engine selection is unchanged (Playwright vs WebDriver as above).

The Appium/hub URL reuses the same seleniumUrl resolution (environment → test config); a native run with no server URL fails at launch with a clear error. Native mobile is a paid feature — device clouds are gated by mobile-cloud (Team+); see the device-clouds doc.

Capabilities in a native session

CapabilityNative (Appium)
click / fill (via the native locator ladder)
getText / getAttribute / getInputValue✅ (getInputValue reads text on Android, value on iOS)
exists / waitForSelector✅ (walks the native ladder)
tap (element via ladder or absolute x/y)✅ (native touch)
swipe (from → to, durationMs)✅ (native touch pointer)
launchApp(appId)mobile: activateApp (modern Appium 2); a bare relaunch with no appId errors — the package/bundle id is required
Screenshots✅ (device screen; fullPage ignored)
Self-heal (AI selector repair)✅ — snapshot is the Appium page-source XML (getPageSource)
goto, selectOption, check, uncheck, hover, press❌ — no faithful native meaning; honest error
waitForUrl, currentUrl, title, api-request❌ — there is no URL/HTTP context in a native app; honest error
Video / traces / console-network / axe❌ (all capture flags false; nativeMobile: true)

Web-only verbs throw a clear "not supported in a native app session" error rather than approximating — honest failure beats a wrong-ish mapping.

Native selector ladder

Vera selectors resolve to an ordered list of native W3C strategies, walked first-match-wins in the tier order accessibility-id ≻ resource-id ≻ (iOS) predicate ≻ xpath:

Vera selectorNative resolution
accessibility-id=Fooaccessibility id = Foo
id=Fooid (Android resource-id / iOS name) = Foo
predicate=…-ios predicate string (iOS only; Android ⇒ clear error)
uiautomator=…-android uiautomator (Android only; iOS ⇒ clear error)
//…, (//…), xpath=…, ./…xpath
[data-testid="x"]accessibility id = x (the recorder's top tier ⇒ content-desc / accessibility id)
#fooaccessibility id = foo, then id = foo
.class, a > b, [name=…], other CSS❌ no native analog — clear error
bare string Foo (no prefix)the full ladder: accessibility-id → resource-id → (iOS) predicate name/label/value == "Foo" → xpath attribute match

Capabilities & device selection

Capabilities are built from the MobileTarget (buildAppiumCapabilities):

  • Android ⇒ { platformName: 'Android', 'appium:automationName': 'UiAutomator2' }
  • iOS ⇒ { platformName: 'iOS', 'appium:automationName': 'XCUITest' }
  • appium:deviceName = the target's deviceProfile, else a platform default.

The resolved app (appium:app — a local .apk/.ipa path or a device-cloud vendor app id like bs://…/lt://…) and device-cloud vendor options (bstack:options / lt:options) are merged in as extra capabilities server-side (the run-flow that owns the app artifact + sealed cloud credentials). Merge order is base-then-extra, so a vendor appium:app wins over a local path.

Known limitations (initial Appium support)

  • A bare launchApp (no appId) errors: the app auto-launches at session start, and a relaunch needs the app's package/bundle id (the MobileTarget holds a Vera artifact id, not a runtime bundle id). Pass an explicit appId.
  • No captured web-session injection, no URL/HTTP verbs, no video/trace capture.
  • Screenshots are the device screen; fullPage is ignored.

Two ways to reach an Appium server

A mobile run needs a reachable Appium server. There are two paths, chosen by the run target (see the local-agent doc):

  • Local agent (E8.3) — a vera-agent on the developer's machine drives a local Appium server against a local emulator / USB device or simulator. The agent advertises its native-mobile capabilities on the /hello handshake ({ mobileAndroid, mobileIos }), and the control plane routes a mobile job only to an agent that advertises the job's platform. The agent downloads the uploaded .apk/.ipa over the authenticated agent API, passes its local path as appium:app, and probes/spawns Appium on 127.0.0.1:4723. Run vera-agent doctor to verify the toolchain (adb/emulator/Appium + drivers, Xcode/simulators on macOS) with actionable fix-its. This path is local-Appium only — no device cloud. For iOS the agent resolves the test's deviceProfile to a simulator and boots it; iOS requires your Mac or your device-cloud account — see the iOS native testing guide.
  • Device cloud (E8.5, server-side) — the server resolves a cloud session (BrowserStack / LambdaTest) and merges the vendor caps + app id itself. This is the mobile-cloud (Team+) paid path and never involves an agent.

Selenium setup guide

The WebDriver engine needs a reachable Selenium (or grid) server. Point Vera at its base URL (per environment, or on the test config).

# Standalone Chrome (includes chromedriver + a noVNC viewer on :7900)
docker run -d --name selenium -p 4444:4444 -p 7900:7900 --shm-size=2g \
  selenium/standalone-chrome:latest

Set the Selenium URL to http://localhost:4444 (the W3C endpoint is at the root on modern images; older Selenium 3 grids use http://localhost:4444/wd/hub).

For Firefox use selenium/standalone-firefox. For a scalable grid, run selenium/hub plus one or more selenium/node-chrome containers and point Vera at the hub URL.

Option B — selenium-standalone (npm)

npx selenium-standalone install
npx selenium-standalone start
# → http://localhost:4444/wd/hub

Option C — plain chromedriver

npx --yes chromedriver --port=9515
# → http://localhost:9515

chromedriver speaks W3C directly (single browser, no grid). Good for a quick local check; use Selenium for parallelism.

Configuring the URL in Vera

  • Per environment: Project → Environments → pick/add an environment → set Engine to WebDriver / Selenium and fill the Selenium server URL.
  • Per test: the test builder's Run settingsEngine. (Set the URL on an environment; the environment's engine + URL win at run time.)

Private / internal grids

Selenium grids usually live on an internal network. Vera's SSRF guard therefore allows private/loopback hosts for the Selenium URL (only cloud-metadata and unspecified addresses stay blocked), even when QA_BLOCK_PRIVATE_HOSTS=true.


Why WebDriver (and not others)

Evaluated backends for the "second engine" (plan §8.3):

OptionVerdict
Selenium / W3C WebDriver✅ Adopted — the industry-standard remote protocol; one client reaches local Selenium, grids, and device clouds.
WebdriverIO (WDIO)✅ Its bare webdriver protocol client is exactly what we use (no WDIO test-runner) — same door as Selenium.
TestCafe❌ Import/export interop only — its proxy-injection execution model doesn't fit the driver seam.