TesterArmy e2e and Independent Validation With Katalon True Platform

TesterArmy shipped something good last week. e2e puts natural-language agent steps and exact assertions in the same TypeScript test, replays recorded actions without paying for the model again, and runs the same suite on web and mobile. That is a hard combination to get right, and the team got it right in an open-source 0.17 release.

I lead product for Katalon True Platform, so I read the launch with professional interest and a stake in the outcome. We work on a close cousin of the same problem, for banks, insurers, and healthcare companies whose suites run to thousands of cases across many teams. At that scale, running the test stops being the hard part. Proving the result is. The answer that keeps holding up is independent validation: whoever writes or runs a test should never be the only one grading it.

This is a public-docs read as of 6 October 2026. The e2e and True integration I sketch near the end is a proposal and does not exist yet.

Launch week on X

TesterArmy co-founder Oskar (@o_kwasniewski) announced e2e on 1 October. Five days later the post sat at 1.1M views, and two days after launch he posted a star chart heading for 2k. Deserved.

What TesterArmy got right

e2e 0.17.0 mixes agent steps with locators, deterministic assertions, fixtures, hooks, retries, and CI support. It runs on Chromium, Firefox, WebKit, physical mobile devices, and APIs, and emits JSON, JUnit, and Markdown. With its assertion API, an agent can find its way through the UI while an exact check decides pass or fail. That split is the right instinct.

The replay cache is the other strong call. Agent actions are recorded and replayed without model calls, while agent.assert, agent.waitFor, and agent.extract always run live. Strict mode fails a stale recording with REPLAY_STALE before the model can take the step over, which is the kind of loud failure a CI pipeline needs.

One public trial on a small Astro site (e2e 0.16.0) saw median runtime drop from 25.1s to 13.5s and model calls from 5 to 1 on cached runs, with both injected defects caught. A Playwright control caught both too and ran faster, and the author's verdict was "adapt". Early, encouraging, and clear about its limits.

Independent validation

The best pushback in launch week came from davert, author of CodeceptJS and Codeception, now building Testomat.io. He asked how anyone decides which checks belong in agent.assert() and which in expect(), and argued for manual test cases as the blueprint. The card in his post says it in a few words: a test written from code "repeats the bug"; a test written from approved intent "catches the bug".

He is pointing at the oldest rule in quality engineering, and AI makes it urgent again. If the same agent reads the code, writes the test, runs it, and judges the result, a misunderstanding passes every step. Each check needs a different owner:

e2e already covers the middle three well: the test code, the runner, and exact assertions that keep the model out of the verdict. The other two, approved intent and an independent record behind the release decision, live outside any single framework. That is the layer we build True Platform for.

What changes at enterprise scale

A small team can keep all five owners in one person's head. The engineer who writes the test also seeds the data, reads the CI failure, and decides whether to ship. In a bank or a hospital network those jobs belong to different people, often in different time zones, and each handoff needs a record that outlives the CI log.

Three things get harder. First, intent: thousands of existing manual cases carry domain knowledge nobody wrote down, so someone has to sort which are runnable and resolve ambiguous expectations before an agent turns them into code. Second, data: a refund test depends on payment method, subscription state, and permissions, and the result has to carry the dataset version and row so a reviewer can reproduce a failure.

Third, the meaning of a result across runs and frameworks: which tests were expected, which were selected and completed, which passed only after retry, and which failed because of the environment or the model provider. The e2e Playwright migration guide is candid about its own gaps here too, including native HTML reporting and pixel-diff snapshots.

e2e as a runner in the True flow

The natural fit is e2e as one more execution lane under a shared record. Engineers keep writing e2e tests in their repo and running them in CI. QA keeps reviewed, versioned test cases and requirement links in True, next to Run with AI, Katalon Studio, Playwright, and manual runs. True's Report Uploader already accepts JUnit, which e2e emits, and the MCP server can link those cases to ALM-synced requirements.

The adapter's real job is preserving what the runner saw. In the e2e 0.17 JUnit implementation, a retry-pass becomes a passing testcase with a flaky: note, and an interrupted test becomes <skipped> with an interrupted: message. That is a sensible choice for a generic format. An independent record should keep flaky and interrupted as states of their own.

Completeness needs its own check too. The CLI docs say a run that stops before its tests leaves the previous report files in place, and reporter failures never change the exit code. Reading back what was stored and reconciling it against what was expected to run is what turns an upload into validation.

Regulated teams: BFSI and healthcare

In banking, insurance, and healthcare, independent validation is a requirement. Auditors and regulators ask, months after the release, which requirement a test proved, who approved the expected behavior, which test version ran on which build, and who signed off. With AI steps, they add which model made the judgment and what path the agent took.

Katalon builds True Platform for these teams. The banking and financial services page describes every edit, run, and result logged for audit and regulatory examination, traceability from each test to a requirement or Jira ticket, and SSO, SCIM, and RBAC. The healthcare page frames the same controls around HIPAA, HITECH, and keeping PHI out of test environments, with insurance and public sector versions alongside. Katalon's security page lists ISO/IEC 27001, 27017 and 42001 certification and a SOC 2 Type II attestation, with more in the trust center. A vendor certification covers the vendor's own controls. The validation process still belongs to the customer, and imported results only inherit an audit trail if the adapter keeps case links, attempts, and evidence intact.

A pilot worth running

Take one important journey with several data variants, run it through e2e, record it in True, and break it on purpose:

Credit where it is due: TesterArmy made agentic testing feel like ordinary engineering, which is the hardest part to get right. The next step for the industry is making those results trustworthy to people who never saw the code. e2e as the runner and Katalon True Platform as the independent record is a combination I would like to see proven.