For a quick count, run Playwright with a built-in reporter such as list, dot, or html. For CI scripts, write a JSON report or use a custom reporter. To count logical tests rather than retry attempts, classify each test using its full retry history: passed on the first run is passed; failed first and passed on a retry is flaky; failed on the first run and every retry is failed.
Choose the right way to count test results
The best method depends on whether you need a count for a person reading the terminal, a machine consuming CI output, or custom retry-aware logic.
| Need | Use | Important limitation |
|---|---|---|
| Quick human-readable summary | list, dot, or html reporter |
Display output is for people; do not treat parsing terminal text as a stable machine interface. |
| CI artifact or script input | JSON reporter with outputFile |
JSON structure can vary by Playwright version; inspect output from the installed version before hard-coding a parser. |
| Exact custom logic or metrics | Custom Reporter, aggregating completed attempts in onTestEnd |
Group retries as attempts belonging to the same logical test; do not count each callback as a separate test. |
Understand passed, failed, and flaky
Playwright’s retry-aware classifications depend on what happened to one logical test across attempts. The official definitions are documented in Playwright’s retry documentation.
- Passed: the test passed on its first run.
- Flaky: the first run failed, but a retry passed.
- Failed: the first run failed and all retries failed.
Retries are off by default. Enable them with the CLI option --retries=N or the retries configuration option. Without retries, a first-run failure cannot become a flaky test in that run.
#1 Best Overall
- CRISP CLARITY: This 23.8″ Philips V line monitor delivers crisp Full HD 1920x1080 visuals. Enjoy movies, shows and videos with remarkable detail
- INCREDIBLE CONTRAST: The VA panel produces brighter whites and deeper blacks. You get true-to-life images and more gradients with 16.7 million colors
- THE PERFECT VIEW: The 178/178 degree extra wide viewing angle prevents the shifting of colors when viewed from an offset angle, so you always get consistent colors
- WORK SEAMLESSLY: This sleek monitor is virtually bezel-free on three sides, so the screen looks even bigger for the viewer. This minimalistic design also allows for seamless multi-monitor setups that enhance your workflow and boost productivity
- A BETTER READING EXPERIENCE: For busy office workers, EasyRead mode provides a more paper-like experience for when viewing lengthy documents
Get a quick count from a built-in reporter
List reporter
Run npx playwright test --reporter=list for detailed test-by-test output and a final summary. This is useful when diagnosing a run locally or in a readable CI log.
Dot reporter
Run npx playwright test --reporter=dot for compact progress output. The reporter uses symbols to distinguish outcomes, including · for passed, F for failed, × for retrying, ± for passed on retry (flaky), T for timed out, and ° for skipped. Read the summary for the actual totals; symbols on the progress line represent run activity and attempts, not necessarily a count of unique logical tests.
HTML reporter
Use the HTML reporter when you want a browsable report rather than just terminal output. Built-in reporters are useful for inspection, but for automation prefer JSON output or the Reporter API over scraping formatted text.
Write JSON output for CI
Configure a readable reporter and JSON output together. Playwright documents using multiple reporters and the JSON reporter’s outputFile option.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Rank #2
- CRISP CLARITY: This 22 inch class (21.5″ viewable) Philips V line monitor delivers crisp Full HD 1920x1080 visuals. Enjoy movies, shows and videos with remarkable detail
- 100HZ FAST REFRESH RATE: 100Hz brings your favorite movies and video games to life. Stream, binge, and play effortlessly
- SMOOTH ACTION WITH ADAPTIVE-SYNC: Adaptive-Sync technology ensures fluid action sequences and rapid response time. Every frame will be rendered smoothly with crystal clarity and without stutter
- INCREDIBLE CONTRAST: The VA panel produces brighter whites and deeper blacks. You get true-to-life images and more gradients with 16.7 million colors
- THE PERFECT VIEW: The 178/178 degree extra wide viewing angle prevents the shifting of colors when viewed from an offset angle, so you always get consistent colors
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [
['list'],
['json', { outputFile: 'test-results.json' }],
],
});Run the suite as usual with npx playwright test. The terminal receives the list output, while the JSON report is written to test-results.json. Add that file to the CI artifacts or pass it to an aggregation step after the test command completes.
Do not assume a particular JSON nesting or field path without checking the report generated by your installed Playwright version. The report is the right input for a script, but its exact structure is version-sensitive. Keep a small fixture from the version used in CI and test your parser when upgrading Playwright.
Count logical tests with a custom Reporter
A custom reporter is the most direct option when you need application-specific totals or a defined treatment for unusual statuses. Playwright calls onTestEnd(test, result) after a test attempt finishes and its TestResult is complete. The result exposes a status and sequential retry number. See the Reporter API and TestResult API.
The core approach is to collect attempts by stable test identity, then classify the group once the run is over. This TypeScript illustrates the classification logic; wire record into your reporter’s onTestEnd and supply a stable test ID from the Playwright test object for your installed version.
Rank #3
- Clear visuals. Fluid motion: A 144Hz refresh rate and 1ms MPRT deliver smooth, tear‑free motion across work, gaming, and streaming for clearer, more fluid viewing.
- Eye comfort: TÜV Rheinland 3‑star* certification reduces harmful blue light while preserving stunning color quality without compromise. *TÜV Rheinland 3-star eye comfort certification.
- Wide viewing angle: Get consistent views across a wide 178° /178° viewing angle.
- In-Plane Switching (IPS): See excellent color accuracy and consistency across wide viewing angles with In-plane Switching (IPS) technology.
- Ultra-thin bezels: Maximize your viewing experience with thin bezels.
type Attempt = { status: string; retry: number };
const attempts = new Map<string, Attempt[]>();
function record(testId: string, result: Attempt) {
const list = attempts.get(testId) ?? [];
list.push(result);
attempts.set(testId, list);
}
function classify(list: Attempt[]) {
const first = list.find(a => a.retry === 0) ?? list[0];
const retriedPass = list.some(a => a.retry > 0 && a.status === 'passed');
if (first?.status === 'passed') return 'passed';
if (first?.status === 'failed' && retriedPass) return 'flaky';
if (first?.status === 'failed' && list.every(a => a.status === 'failed')) return 'failed';
return 'other';
}Here other is deliberate. A test may time out, be skipped, be interrupted, or have another status; silently folding those outcomes into failed or passed would make the totals misleading. Decide and document how your consumer should report each status.
Build the reporter lifecycle around the run
- In
onTestEnd(test, result), record the test identity,result.retry, andresult.status. This callback is per completed attempt. - After all tests have ended, group attempts by logical test identity. Ensure identity distinguishes genuinely separate tests, including cases produced by projects or repeat-each, if those should count separately.
- Apply the retry-aware definitions to each group, then emit totals for passed, flaky, failed, and any other statuses you preserve.
- When publishing metrics, label the scope: for example, one project, one shard, or the merged run. Avoid adding counts from overlapping reports.
The grouping pattern is an implementation approach based on the documented lifecycle and retry model, not a promise that every Playwright version or project setup uses the same identity field. Confirm identity handling against your installed version and suite configuration.
Define the scope before comparing counts
Counts are easy to misread when one report mixes multiple projects, shards, repeated executions, or retries. Decide what one “test” means for the metric before aggregating.
- Retries: an additional attempt is not a new logical test. Classify the sequence rather than incrementing a test total for every
onTestEnd. - Projects: the same test file may run under multiple projects. Decide whether your number means test-project executions or unique test definitions.
- Shards: shard reports cover portions of a run. Combine non-overlapping shards once, and report whether totals are per shard or merged.
- Repeat-each: repeated executions create multiple outcomes for a test. Decide whether each repetition is its own counted execution or whether you aggregate them under one definition.
- Nonstandard statuses: retain timed-out, skipped, interrupted, and version-specific states separately unless your team has an explicit mapping.
Playwright’s CLI documentation and configuration documentation describe test execution options. For reproducible dashboards, record the project, shard, repeat, and retry scope alongside the counts.
Rank #4
- CURVED FOR ENHANCED ENGAGEMENT: An immersive viewing experience with a curved monitor that wraps more closely around your field of vision; It creates a wider view, enhancing depth perception and minimizing peripheral distraction
- SMOOTH PERFORMANCE FOR SEAMLESS CONTENT: Stay in the action when playing games, watching videos, or working on creative projects; The 100Hz refresh rate reduces lag and motion blur so you don't miss a thing in fast-paced moments¹
- MORE GAMING POWER: Gain the edge with optimizable game settings; Color and image contrast can be adjusted to see scenes more vividly and spot enemies hiding in the dark; Game Mode adjusts any game to fill the screen so you can view every detail²
- KEEP IT EASY ON THE EYES: Care for your eyes and stay comfortable, even during long sessions; Advanced eye comfort technology certified by TÜV reduces eye strain by minimizing blue light and reducing irritating screen flicker²
- INCREASED VERSATILITY: Connect to more; Plug devices straight into your monitor for increased flexibility, making your computing environment even more convenient
Troubleshoot unexpected totals
There are no flaky tests
Retries are disabled by default. Set retries in the Playwright configuration or pass --retries=N to enable them. A test must fail first and pass on a retry to be classified flaky.
The total is larger than the number of tests you expected
You may be counting attempts instead of logical tests, or counting executions across multiple projects, shards, or repeat-each runs. Group retries and state the counting scope explicitly.
JSON parsing breaks after an upgrade
Do not assume the same nesting in every version. Generate a report with the installed version, inspect its structure, and update and test the parser against that version before relying on it in CI.
Timed-out or skipped tests disappear from the three totals
Passed, failed, and flaky are not an exhaustive classification of every possible result. Preserve other statuses in a separate bucket or state an explicit mapping; do not silently label them failed.
Recommended Free Tools
Best Value
- 【INTEGRATED SPEAKERS】Whether you're at work or in the midst of an intense gaming session, our built-in speakers provide rich and seamless audio, all while keeping your desk clutter-free.
- 【EASY ON THE EYES】 Protect your eyes and enhance your comfort with Blue-Light Shift technology. This feature reduces harmful blue light emissions from your screen, helping to alleviate eye strain during long hours of use and promoting healthier viewing habits.
- 【WIDEN YOUR PERSPECTIVE】Our sleek minimal bezel design ensures undivided attention. The nearly bezel-free display seamlessly connects in a dual monitor arrangement, delivering an unobstructed view that lets you focus on more at once, completely distraction-free.
The terminal symbols do not match your test count
The dot reporter shows progress and retries as well as outcomes. Use the final reporter summary for a human-readable result, or aggregate completed attempts by logical test for a machine count.
Keep CI counts reliable and affordable
- Reliability: prefer structured JSON or the Reporter API over parsing presentation-oriented output.
- Version changes: pin or deliberately upgrade the Playwright version, and validate JSON parsing and test identity handling whenever it changes.
- Scope: attach run, project, and shard identifiers to metrics so parallel jobs are not double-counted.
- Retries: show flaky counts separately from hard failures. Increasing retries can make a run report more recovered tests, but it does not make the initial failures disappear from reliability analysis.
- Cost: these reporting choices are Playwright configuration and code; CI cost is driven by your runner and execution setup. The cited Playwright material does not establish a universal runtime or cost impact for enabling retries.
Or skip the browser setup
If your Playwright work also needs screenshots of pages, ScreenshotNeo provides a screenshot API and MCP server. A screenshot does not replace Playwright test-result reporting; it can remove browser-capture setup for separate screenshot tasks.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpSee the ScreenshotNeo documentation for the request options. Cookie banners, popups, and chat widgets are removed before the shot; bot checks, blank pages, and failed loads are never billed. Its MCP server lets AI agents take screenshots. The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000.
Sign up free for ScreenshotNeo.
Frequently Asked Questions
Does Playwright show a flaky count without retries?
No. Retries are disabled by default, and a test can only be classified flaky when it fails initially and passes on a retry.
Should retries be included in the total number of tests?
No, not when reporting logical-test counts. Treat retries as additional attempts belonging to the same test.
Can I use these counts across multiple Playwright versions?
The retry definitions are documented, but JSON shape and implementation details can be version-sensitive. Validate report parsing and test identity against the version your CI runs.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
