When a self-healing test repairs a locator, a green result does not prove the test still checks the behavior its author intended. Luthfi Ferdian argues that some short-lived checks are better treated as throwaway automation: keep the readable test case, let an AI agent exercise it in a browser, and have a person review the results and evidence. Turn only recurring or important scenarios into reviewed, deterministic tests.
Why a self-healed test can be green and still be wrong
A locator repair can keep a test running by finding an element that seems plausible. But if the repaired locator points somewhere other than the intended control, the test may pass without checking the original scenario. That is the risk Ferdian highlights in his September 30, 2026 article, “Stop Healing Your Tests: Why Throwaway Automation Fits the AI Era”.
That is an argument for reviewing an automatic repair as a proposed change, not treating it as proof that the test remains valid. It is not evidence that every self-healing product produces false positives. The practical question is whether the changed test still identifies the right page element and asserts the intended outcome.
Ferdian’s alternative is to make the human-readable test case—the preconditions, actions, expected outcomes, and evidence to capture—the durable asset. An agent can carry out selected browser checks; a person evaluates what happened. As he puts it, “You can’t break a script that doesn’t exist.”
#1 Best Overall
When throwaway automation is a reasonable fit
Agent-run checks are candidates for scenarios with a limited useful life: a release-specific check, an exploration during a refactor or migration, or a reproduction of a newly reported bug. These are possible uses, not proven universal advantages over maintained automation.
Use the following questions to decide whether a case is suitable:
- How long will it matter? A one-off investigation is different from a behavior that must be checked every release.
- How consequential is failure? Core flows and audit or compliance evidence call for durable, repeatable controls. Applying that principle to financial, access-control, or other high-consequence behavior means setting a particularly high bar for assertion and review.
- Can the result be stated objectively? Write expected outcomes that distinguish success from a workaround or a merely plausible page state.
- Can someone review the evidence? Steps and captured outputs should let a human judge whether the expected result occurred.
- Can the environment be controlled? Supply seeded accounts, test data, staging constraints, and relevant environment details.
- Does the economics make sense? Compare the cost of writing and maintaining a script with the uncertainty and recurring cost of agent runs. Ferdian gives no measured break-even point, so this remains a team judgment rather than a quantified rule.
These axes help frame the choice; they are not a validated scoring model.
Write the test case before asking an agent to run it
An agent cannot reliably infer missing setup or the intended meaning of a vague instruction. The test case should state what must be true before the run, what to do, what counts as passing or failing, and what evidence to return.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Example: check a promotion in the cart
Ferdian’s illustrative case—not a report of an executed test—can be expressed as follows:
- Preconditions: A standard user is logged in and the cart is empty.
- Action: Add the promotion SKU, then open the cart.
- Expected results: A promotion banner and discount text appear; no error toast appears; the mobile layout has no overlap.
- Evidence: Request step-by-step observations, a PASS or FAIL for each expected result, and a screenshot.
Keep each result separate. A banner appearing should not conceal a missing discount, an error toast, or a layout defect. If the test cannot say what counts as an overlap or which discount text is expected, clarify that before relying on the agent’s judgment.
Use browser evidence carefully
Playwright’s MCP introduction describes a browser-control server that uses structured accessibility snapshots for page interaction and provides screenshot tools. The Playwright MCP repository also documents a headless mode. A screenshot can help a reviewer assess visual layout; an accessibility snapshot is the structured representation used for interaction. Neither makes the agent’s interpretation correct, nor establishes that a run will be deterministic.
Ask for observations tied to each expected result, then inspect the actions and evidence rather than accepting a summary alone. Ferdian warns that an agent may work around a broken flow and claim success. Do not keep rerunning a failing check until it turns green: investigate the failure, the instructions, and the environment instead.
Keep deterministic automation where repeatability matters
Ferdian does not propose agent runs as merge gates or substitutes for a large regression suite. He points to non-deterministic outcomes, slower runs than compiled scripts, and token costs as limitations. A thousand-test regression run, a core user flow, or an audit trail that needs durable repeatability is a poor fit for a disposable agent check.
Best Value
Choose based on the job the check must do, not on whether a tool can perform it. A human-reviewed observation may be enough for a temporary investigation; a gate that must give the same dependable signal on repeated runs needs maintained, explicit assertions.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Promote valuable scenarios into reviewed Playwright tests
If a temporary case recurs, catches meaningful defects, or protects critical behavior, convert it into a small deterministic test. Ferdian’s example promotes the cart scenario into Playwright code with explicit API setup and an assertion on a test identifier. The durable test should encode the intended behavior directly; preserving an agent transcript is not the same as creating a reliable regression check.
Ferdian’s rule of thumb is: “if you wouldn’t urgently fix a script when it breaks, don’t promote it.” Promotion creates a maintenance commitment, so reserve it for checks whose repeated value justifies that work. His broader point is that “Your value as an SDET isn’t how many scripts you maintain.”
Free tools Windows power users keep installed
One-click scans. No signup required.
The article and the cited Playwright documentation do not provide measured comparisons of false-positive rates, maintenance hours, speed, token cost, or defect detection for this workflow. Treat the decision as a risk and lifecycle judgment, not a quantified performance claim.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
