Flaky Test Detective review
A focused CLI for Playwright and JUnit teams managing flaky-test patterns and quarantine.
Reviewed by iTechGuides Editors · Editorial team · Updated Oct 2026
Flaky Test Detective is an open-source TypeScript command-line tool for QA and development teams that need to identify and manage unreliable automated tests. It analyzes results across multiple runs, calculates flakiness scores, detects timing, network, and ordering patterns, and supports threshold-based quarantine tagging. The tool is aimed at projects using Playwright or JUnit, rather than teams seeking broad framework coverage or a hosted testing service.
Its strongest fit is a focused, repository-based workflow. The published feature set includes Playwright JSON and JUnit XML result parsing with automatic format detection, while GitHub Actions is listed as an integration. Results can be presented through console, JSON, HTML, or GitHub pull-request reporters, giving teams several ways to surface findings in development and CI processes. Trend tracking uses SQLite-backed storage, with JSON fallback storage available for simpler setups. Teams should expect to connect the CLI to their existing test-result pipeline rather than adopt a separate hosted workspace.
The standout workflow is quarantine management: configurable thresholds can automatically tag tests, and the quarantine command produces patterns that can be used to exclude identified tests from Playwright runs. That makes the tool suitable for teams that want a measurable way to separate recurring flakes from ordinary failures without abandoning their current test frameworks. Its narrower framework support and CLI focus make it less suitable for organizations running other test ecosystems, seeking hosted deployment, or requiring formal issue-tracker linking. Choose Flaky Test Detective for open-source Playwright and JUnit projects that value historical scoring, pattern detection, and repository-centered reporting.
Flaky Test Detective pros and cons
- Where it wins
- Scores flakiness across historical test runs
- Detects timing, network, and ordering failure patterns
- Reports through console, JSON, HTML, and GitHub pull requests
- Where it doesn't
- Supports only Playwright and JUnit result formats
- Requires CLI-based integration with existing test workflows
- Does not document formal issue-tracker linking
Flaky Test Detective fact sheet, pricing and score →
Advertiser disclosure: iTechGuides is reader-supported. Vendors can pay for top positions in our rankings and for a place on other products' pages, and we may earn a commission when you click some links. Bought places stand at the number bought; this page and our disclosure say so. Paying vendors may get a fuller, more detailed review from our editors, and that review can change their score. How we rank.
Last updated · How we research and update