Recommended Free Tools
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Playwright tests that fail intermittently on CI are showing a symptom, not naming a cause. Start by preserving the failure report and trace, then use the evidence to check test isolation, timing, and runner capacity. Retries, more workers, and longer timeouts can change what you observe, but none is a diagnosis by itself.
Why do Playwright tests pass locally but fail in CI?
CI runs can differ from a developer’s machine in available CPU and memory, execution speed, and the state or ordering of test data. A test that relies on another test’s setup, shared mutable state, or a timing assumption may therefore fail only under CI conditions. The exact cause depends on the test code and runner; the word “timeout” alone does not establish that the timeout setting is too short.
Playwright recommends that tests be independent, each with its own state and data. Independence makes failures easier to reproduce and prevents one test’s failure from cascading into others. Review the suite for shared cookies, local or session storage, test data, or other state that can make one test depend on execution order. See Playwright’s best practices.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
How do I debug a flaky Playwright test?
1. Preserve the report and trace
Keep the failing CI run’s HTML report and trace artifact before rerunning or changing settings. A common setup is to collect a trace on the first retry; if retries are disabled, consider retaining traces on failure instead. Tracing every test can add runtime and storage overhead, so target collection to failures or retries. Playwright’s Trace Viewer documentation explains how to inspect traces.
#1 Best Overall
npx playwright show-trace path/to/trace.zipPlaywright’s CI guidance recommends collecting traces on the first retry. The trace can show action timing, DOM snapshots, and network requests, helping you relate the failed action to what the page and network were doing at that moment.
2. Read the first failure against the trace
Identify the failed assertion or action and its locator. Then compare the action duration with the DOM snapshot and network activity around it. Ask whether the page reached the expected user-visible state, whether navigation or data arrived later than expected, or whether the runner appears constrained. Treat those as hypotheses to verify, not conclusions inferred from a timeout message.
3. Check whether the test is independent
Confirm that the test creates or resets the data and browser state it needs. Look for shared mutable state and dependencies on test order, including cookies and browser storage. If a test passes only after another test runs, that dependency is a more direct reliability problem than a missing retry.
4. Check runner capacity and parallelism
Compare the configured worker count with the CPU and memory available to the CI job. Playwright recommends setting workers to 1 in CI to prioritize stability and reproducibility. Its CI guidance also warns that setting workers above the detected core count can lead to unnecessary timeouts and failures. One worker is a stability-oriented starting point, not a guarantee against flakes. For more parallel throughput, consider distributing work across CI jobs with sharding rather than simply increasing concurrency on one constrained agent. See Playwright’s CI guide.
Should I increase the Playwright timeout?
Only when the failure evidence shows that the operation legitimately needs more time. Playwright’s default test timeout is 30 seconds, according to its current timeout documentation. A longer limit can be appropriate for a genuinely slower operation, but it can also make a broken state take longer to report. Playwright’s timeout guidance cautions that flaky tests often need a solution beyond changing low-level timeouts.
Use the trace to distinguish a slow but progressing operation from a test waiting for the wrong state, missing data, or an action that never completes. Change the relevant action, navigation, or test timeout only when that evidence supports it. If setting a global CI timeout, keep it comfortably below the outer job timeout so Playwright can stop and report before CI terminates the job; consult the CI guide for the applicable configuration.
How many Playwright workers should I use in CI?
Start with one worker when stability and reproducibility matter most, as Playwright recommends for CI. Increase the count only after checking the runner’s actual capacity and observing how the suite behaves under that concurrency. More workers can improve throughput, but they also increase resource demand and may expose contention. If the goal is parallelism across machines, sharding is an alternative to packing more work onto a single agent.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →What should retries tell you?
Playwright classifies a test that fails initially but passes on retry as flaky. Retries are disabled by default. A retry-passing test is useful evidence that the failure is intermittent; it is not proof the underlying defect is fixed. Use retries to capture diagnostic artifacts and identify patterns, rather than treating a green final result as a clean test.
Playwright added failOnFlakyTests in v1.52. When enabled, it can make CI exit unsuccessfully when tests are classified as flaky, keeping retry-passing failures visible. Check the installed Playwright version before adding this version-specific option. Details are in the TestConfig API documentation.
A CI configuration to collect useful evidence
Playwright’s configuration guide shows a pattern combining CI-only retries, one worker in CI, an HTML report, and a trace on the first retry:
import { defineConfig } from '@playwright/test';
export default defineConfig({
retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 1 : undefined,
reporter: 'html',
use: {
trace: 'on-first-retry',
},
});This is an example to adapt to your suite and runner, not a universal prescription. The retry count and worker policy should serve your debugging and capacity needs. For teams ready to make intermittent failures actionable in CI, the version-dependent flaky-test gate can be configured separately:
export default defineConfig({
failOnFlakyTests: !!process.env.CI,
});See Playwright’s configuration documentation for the configuration options and the retry documentation for retry behavior.
Choose the fix that matches the evidence
| Change | What it helps with | Trade-off or risk |
|---|---|---|
| Trace failures or first retries | Shows action timing, DOM state, and network activity for diagnosis. | Tracing adds runtime and storage overhead, especially when enabled for every test. |
| Use one CI worker | Prioritizes stability and reproducibility on a runner. | May reduce throughput on a runner capable of handling more concurrency. |
| Shard across CI jobs | Adds parallelism across machines. | Requires splitting work across jobs rather than relying on more workers on one agent. |
| Enable retries | Reveals tests that fail initially but pass on retry and can trigger first-retry traces. | Can hide accumulating flakes if retry-passing tests are treated as healthy. |
| Increase a timeout | Allows a demonstrably slow operation more time to finish. | Can delay failure reporting without addressing a wrong state, dependency, or resource problem. |
These options address different problems. Prefer the change that explains the observed failure, and retain enough reporting to tell whether it improved the underlying test or only changed how the failure appears. For command-line options and report handling, see Playwright’s Command line documentation.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
