The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →A flaky test sometimes passes and sometimes fails despite effectively unchanged code and inputs. Treat that inconsistency as a defect to diagnose—not as a harmless failed run. Find the uncontrolled state, timing, dependency, or execution condition; make it deterministic; and use retries or quarantine only as visible, temporary mitigation.
What makes a test flaky?
A test is nondeterministic when it passes on some runs and fails on others without a noticeable change in the code, tests, or environment. The failure weakens the test suite’s signal: a genuine regression can be obscured when developers learn to dismiss intermittent failures. A pass after a rerun shows that the result was intermittent; it does not establish that the test or the software is correct.
Common causes include shared or stale state, incomplete setup or cleanup, order dependence, uncontrolled time, asynchronous races, external services, and insufficient runner resources. These causes can overlap. For example, a slow runner may expose a race that is difficult to reproduce locally.
How to investigate a flaky test
1. Record the failure and confirm its pattern
Capture the code revision, test identity, environment, and complete failure details, including logs and timing. Rerun the suspect test independently rather than immediately rerunning the entire suite. Compare the failing and passing runs, and check whether the result changes when the test runs alone or in a different order.
#1 Best Overall
- Read Before You Buy — No Video Output: These adapters support charging and USB 2.0 data transfer, but cannot transmit video signals. Except for standard USB webcams (which use USB data only), they are not compatible with HDMI/DisplayPort cables, video-capable USB-C hubs, or docking stations with video output.
- Convert USB-A Ports to USB-C: Designed to connect USB-C earphones, cables, flash drives, card readers, and other USB-C accessories to standard USB-A ports. Plug-and-play with no drivers or software required.
- Aluminum Alloy Housing: Built with a sturdy aluminum alloy shell that aids in heat dissipation and protects against daily wear and scratches. Designed to maintain a stable and secure connection.
- Compact & Travel-Friendly: The ultra-compact design allows the adapter to stay plugged into your device without blocking adjacent ports or adding bulk, reducing wear and tear on your original USB ports.
- 12-Month Warranty: Backed by a 12-month manufacturer warranty for peace of mind. Designed to meet strict quality control standards for reliable everyday performance.
A test that passes on retry is evidence of intermittency, not evidence that the original failure can be ignored. Google’s guidance on test flakiness likewise treats inconsistent outcomes as a reliability problem that can hide real regressions.
2. Check state, fixtures, and test order
Make sure each test starts from a known state and that setup and teardown complete on both success and failure. Look for hidden shared fixtures, singletons, static variables, database rows, files, caches, and dependence on a test that ran earlier. A useful diagnostic is to run the suspect test before and after related tests, then run it repeatedly in isolation.
Isolation should make tests safe to run in different sequences. Rebuilding a known starting state is often easier to reason about than relying on cleanup, although larger fixtures can take longer to create. Choose based on the cost of setup and the risk that cleanup will leave residue; do not assume that a cleanup hook always runs after a crash or forced termination.
Rank #2
- 5-in-1 USB-C Hub: Experience comprehensive connectivity featuring a Power Delivery input, two USB-A 2.0 ports, a USB-A 3.0 port, and an HDMI port. (Note: The USB-C power delivery input port is only for connecting an external wall charger to power your laptop and cannot power peripheral devices.)
- 90W Pass-Through Charging: Achieve optimal charging with 90W pass-through power to your laptop, supported by a total input of 100W, with the hub reserving 10W for operational efficiency. (Note: Wall charger not included.)
- Quick Data Transfers: Accelerate your productivity with rapid data transfers using a high-speed 5Gbps USB 3.0 port and two 480Mbps USB 2.0 ports.
- 4K HDMI Display: Enhance your visual experience with a hub capable of delivering 4K resolution at 30Hz in both mirror and extend modes. Please note that this hub is compatible with MacBook (macOS 12 and newer), Windows 10 and 11, ChromeOS, and laptops equipped with DP Alt Mode and Power Delivery. Note: This device is not compatible with Linux.
- What You Get: Anker USB-C Hub (5-in-1, 4K HDMI), welcome guide, 18-month warranty, and our friendly customer service.
3. Control clocks and asynchronous work
Tests that read the wall clock can cross a date, minute, or daylight-saving boundary, or disagree with the timestamps in their fixtures. Put clock access behind a controllable seam and freeze or seed it for the test. Keep the time assumptions in fixtures consistent with the controlled clock.
For asynchronous behavior, wait for a specific application state and set a timeout that produces useful diagnostics. A fixed sleep does not prove that the required state has been reached: a short delay can fail on a slower run, while a long delay wastes time. Google’s triage guidance warns against arbitrary delays for these reasons. Prefer an observable condition, such as an expected state transition, completed job, or rendered element, and include the last observed state in timeout failures.
4. Control dependencies without losing useful coverage
Remote services and third-party systems add behavior and timing that a test may not control. For stable regression coverage, substitute a test double when the test does not need the real service. This improves repeatability but reduces direct end-to-end fidelity: the test proves behavior against the double, not against every detail of the real integration.
Rank #3
- Sleek 7-in-1 USB-C Hub: Features an HDMI port, two USB-A 3.0 ports, and a USB-C data port, each providing 5Gbps transfer speeds. It also includes a USB-C PD input port for charging up to 100W and dual SD and TF card slots, all in a compact design.
- Flawless 4K@60Hz Video with HDMI: Delivers exceptional clarity and smoothness with its 4K@60Hz HDMI port, making it ideal for high-definition presentations and entertainment. (Note: Only the HDMI port supports video projection; the USB-C port is for data transfer only.)
- Double Up on Efficiency: The two USB-A 3.0 ports and a USB-C port support a fast 5Gbps data rate, significantly boosting your transfer speeds and improving productivity.
- Fast and Reliable 85W Charging: Offers high-capacity, speedy charging for laptops up to 85W, so you spend less time tethered to an outlet and more time being productive.
- What You Get: Anker USB-C Hub (7-in-1), welcome guide, 18-month warranty, and our friendly customer service.
Where the integration matters, pair the repeatable tests with appropriate contract or integration checks that compare the double’s important behavior with the real interaction. Do not make every test depend on a remote service simply to retain fidelity, and do not treat a double as proof that the real service behaves identically.
5. Inspect the runner and environment
Compare the runner’s environment and resource allocation across failing and passing runs. Check logs for resource pressure, incomplete initialization, and assumptions that differ between local and CI execution. A system under test that receives insufficient resources may behave differently under load; missing setup can also make a run depend on whatever state happened to exist.
Free tools Windows power users keep installed
One-click scans. No signup required.
Make required setup explicit and keep execution environments as controlled as practical. Google’s testing guidance notes that hermetic environments are generally less prone to flakiness. That is a direction, not a guarantee: a hermetic test can still have races or faulty state management.
Rank #4
- Dual Converters, Infinite Potential：Includes 2× USB C male to USB A female adapters and 2× USB A male to USB C female adapters. Perfect for a wide range of uses—tablets with Bluetooth keyboards, expand USB ports on macbook, and more. Two different converters for all your daily needs
- Next-Level 10Gbps & 3A Charging: No more slow 480Mbps, this usb to usb c adapter has a transfer speed of up to 10Gbps, allowing you to do more transferring in less time. This usb adapter fits both USB A and USB C charger, supporting up to 3A fast charging
- Upgraded Exquisite Craftsmanship: With an aluminum alloy housing and metal connector, the usbc to usb adapter is extremely durable and sturdy. Rigorously tested to withstand more than 10,000 times of plugging and unplugging, ensuring long-lasting performance
- Broad Compatible: The usb c to usb adapter widely supports all USB C/ USB A devices like laptops, tablets, cellphones, car chargers, and phone chargers. Such as compatible with MacBook Pro/Air 2023/2022, Thunderbolt 4/3 Devices,Apple MagSafe Watch 9/8/7/SE/Ultra, iPad Pro 2022/2021, Samsung Galaxy S23/S20/S10, and iPhone 17/16/15 Pro. Plug and play
- Please Note: To reach 10Gbps speed, keep the cable under 3.3 ft. For USB A Male to USB C adapters, try flipping the USB C connector. USB C Male to USB A adapters support bidirectional 10Gbps transfer within 3.3 ft
Choose a repair that matches the cause
| Decision | Trade-off | Use when |
|---|---|---|
| Rebuild a fixture or clean up existing state | Rebuilding gives a known start but may cost more setup time; cleanup can be faster but may leave residue or fail to run. | Choose the simplest approach that reliably isolates the test, considering fixture size and cleanup failure modes. |
| Use a test double or the real dependency | A double improves control and repeatability but reduces direct fidelity to the production interaction; a real dependency tests more of the integration but adds uncontrolled behavior and timing. | Use doubles for stable regression coverage and add contract or integration checks where the real interaction needs validation. |
| Fail immediately, retry, or quarantine | Fail-fast preserves the immediate diagnostic signal; retries or quarantine can reduce workflow disruption but may conceal the underlying defect. | Use retries to gather evidence or maintain continuity temporarily, and keep any quarantine visible, owned, and time-bounded. |
Use retries and quarantine as temporary controls
A retry can help identify an intermittent result or keep a workflow moving while an investigation is underway. It cannot make a nondeterministic test reliable. Record retry outcomes separately from clean first-run results, track the failure, and assign an owner to find and fix its cause.
If a test must be quarantined to protect the main suite’s signal, keep it visible, set a review or repair deadline, and preserve a route for running it. Martin Fowler’s guidance on non-deterministic tests warns against allowing quarantine to become abandonment. A growing retry count or an indefinite quarantine is a reason to prioritize repair, not a measure of stability.
Troubleshoot common symptoms
- Fails only after another test: inspect shared fixtures, global state, database rows, and cleanup; run the suspect test in alternate orders.
- Fails near a time boundary: control the clock and align fixture timestamps rather than relying on the machine’s current time.
- Fails more often on slower or busy runners: inspect resource allocation and setup completion, then synchronize on a meaningful state with a diagnostic timeout.
- Fails only when a remote service is slow or unavailable: isolate stable regression tests from that dependency and retain targeted integration checks where needed.
- Passes on retry but remains unexplained: keep the incident visible and investigate the original failure; a retry is not a fix.
Or skip the browser setup
If browser screenshots are part of your test workflow, ScreenshotNeo can capture a page with one GET request instead of requiring you to set up a browser capture path. This does not diagnose flaky tests or replace application assertions; it provides a screenshot capture option.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Best Value
- 5-in-1 Connectivity: Equipped with a 4K HDMI port, a 5 Gbps USB-C data port, two 5 Gbps USB-A ports, and a USB C 100W PD-IN port. Note: The USB C 100W PD-IN port supports only charging and does not support data transfer devices such as headphones or speakers.
- Powerful Pass-Through Charging: Supports up to 85W pass-through charging so you can power up your laptop while you use the hub. Note: Pass-through charging requires a charger (not included). Note: To achieve full power for iPad, we recommend using a 45W wall charger.
- Transfer Files in Seconds: Move files to and from your laptop at speeds of up to 5 Gbps via the USB-C and USB-A data ports. Note: The USB C 5Gbps Data port does not support video output.
- HD Display: Connect to the HDMI port to stream or mirror content to an external monitor in resolutions of up to 4K@30Hz. Note: The USB-C ports do not support video output.
- What You Get: Anker 332 USB-C Hub (5-in-1), welcome guide, our worry-free 18-month warranty, and friendly customer service.
For API parameters and options, see the ScreenshotNeo documentation.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp- Cookie and consent banners are accepted, and more than 60 known consent platforms, newsletter popups, and chat widgets are removed before capture; each step can be turned off.
- Bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed; response headers report the page verdict and billing status.
- An MCP server provides
take_screenshot,get_page_info, andcapture_pdftools for AI agents and MCP clients. - The Free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000 screenshots.
Sign up for ScreenshotNeo’s free plan to get 1,000 screenshots a month with no card.
How much flakiness is there?
Google reported that about 1.5% of its test runs were flaky and almost 16% of its tests had some level of flakiness in a 2016 report. Those are historical figures about Google’s test corpus, not current measurements or industry-wide estimates. Google also described around 4.2 million tests on its continuous integration system in a 2017 article; that figure is historical and specific to Google.
These figures illustrate that intermittent failures can matter even at large scale, but they should not be used as a benchmark for another organization. Track your own flaky outcomes over time using consistent definitions and report retries separately from first-run results.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Frequently Asked Questions
Is a flaky test always a bug in the test?
Not necessarily. The inconsistency may come from the test, the system under test, an external dependency, or the execution environment. The investigation should identify which condition is uncontrolled.
Can I use a fixed sleep if a test is timing out?
A fixed delay can be a temporary diagnostic aid, but it is not a durable synchronization strategy. Prefer waiting for the specific state the test requires, with a timeout that reports useful context.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
