Recommended Free Tools
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
An AI gateway can become slower in several different places: its own queue and policy checks, the DNS/TCP/TLS connection to an upstream provider, the provider’s queue, model time to first token (TTFT), or token generation. The fastest way to find the cause is to measure each stage separately, then segment the measurements by model, service tier, project, region, consumer, and request size.
Do not diagnose this from one average response time. Compare a known-good baseline with current P50, P75, P95, and P99 request duration and TTFT, check retries and HTTP 429 responses, verify cache-hit behavior, and correlate gateway records with provider request IDs.
Where AI-gateway latency is introduced
A request’s total duration is the sum of several intervals. A slow total does not tell you which interval changed, and TTFT is not the same signal as completion time.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problems| Stage | What to measure | Typical clues |
|---|---|---|
| Gateway ingress and policy | Queue wait, authentication, routing, transformations, logging overhead | All providers slow for the same gateway slice; queue depth or concurrency rises. |
| DNS, TCP and TLS | DNS lookup, connection establishment, TLS handshake, pooled-connection reuse | Connect timeouts, TLS errors, stale pools or a regional network-only increase. |
| Provider queue | Time from outbound request to provider acceptance or first-byte response | Gateway work is stable, but one provider, model or tier has a longer wait. |
| Model TTFT | Outbound request to first generated token | Larger uncached prompts, more reasoning, or provider-side contention raise TTFT. |
| Token generation | First token to final token, output-token count and token velocity | Longer outputs or lower generation speed increase total time while TTFT stays stable. |
Graph TTFT and total request time independently. A TTFT regression points toward queuing, prompt processing or reasoning; a stable TTFT with a longer completion usually means more output tokens or slower token generation.
#1 Best Overall
- Powerful Turbo Fan:CAGIWIRU Electric Air Duster have reached speed up to 130000RPM,Can efficient remove dust and debris.With LED light and Three Adjustable Speed to meet with different cleaning tasks.
- With Attached USB C Fasting Charging cable.Charing 3 hours,Enjoy up to 240 minutes of use on the lowest setting, For third speed can use for 25mins.with four charging options to meet with your needs.
- Wide Application: attached 5 different nozzles,making it suitable for a variety of scenes, such as car,pet, pc, keyboards, and other electronic devices. It also serves for office and home clean duster.
- Handy Design & Portable:Cordless air duster electric nozzle is Light weight and small size, design and comfortable to hold and space saving, convenient to carry around. You can put it in your room or inside the car without taking up space.
- Economical &Reusable: Built durable materials with a long-lasting battery,our CAGIWIRU air duster it's a sustainable alternative to disposable air cans.
1. Define the regression with percentiles
Start by writing down what “slower” means. Choose a period that was known to be healthy and compare it with the affected period using the same definitions and clock source.
- Record request duration and TTFT at P50, P75, P95 and P99 for both periods. Include request counts so a percentile based on a tiny sample is not mistaken for a broad regression.
- Keep successful, timed-out and retried requests identifiable. A final response time that excludes failed attempts can make the user experience look better than it is.
- Plot the distributions over time rather than only a single number. A rising P99 with a flat P50 is a tail problem; a rise across every percentile is more likely systemic.
- Compare the same endpoint and request class. Changes in prompt length, output limits or streaming mode can otherwise look like an infrastructure change.
OpenAI’s Help Center troubleshooting guidance says averages can hide real-user impact and recommends reviewing P50, P75 and P95 percentiles. P99 is also useful for exposing the requests most affected by queueing, retries or network failures.
2. Filter the population before changing settings
Mixed traffic can make a localized issue appear global. OpenAI’s guidance states, “Always filter to a single model,” and notes that meaningful troubleshooting requires filtering.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchRun separate views for:
- One model and model version at a time.
- One service tier, such as standard versus a priority tier, when your provider exposes that distinction.
- One project, API key group or application.
- One gateway region, availability zone or egress location.
- One consumer, tenant or route.
- Similar prompt and output-size bands.
If only one project has a higher P95, a global gateway change is unlikely. If every project in one region changes together, inspect regional egress, DNS and connection pools before tuning model parameters.
Rank #2
- Screen Cleaner is Your Screen’s New Bestfriend! - EVEO is proud to present a screen cleaner that’s perfectly suited for your TV. Meet your new screen cleaner, computer screen cleaner, laptop cleaner, iPad cleaner, Macbook,tv cleaner or any other electronic device. Our screen cleaner spray and wipe along with the included plush microfiber cloth, easily removes dust, fingerprints & other smudges on the screen’s surface. Clean effectively for a streak-free screen
- Gentle on your Screens - Our screen cleaner spray and wipes is compatible to be used even for the most sensitive LCD, LED, CRT, and OLED screens in proper use. Be at ease knowing that this product is compatible with all the major brands: LG, SONY, Samsung, Sharp, iPad, iPhone, Android screens, Kindle, PC monitors - we’ve got you covered! Can be used as LCD screen cleaner, tv cleaner, smart tv screen cleaner, and more
- Premium Super Soft Plush Microfiber Cloth - Included in this efficient screen cleaning kit. This screen cleaner spray and wipe has our signature plush microfiber cloth specially designed to comply with screens and monitors and leave them streak-free. Has been shown to effectively absorbs stubborn stains, zero streaks, and no fiber shedding. Always use this together with our screen cleaner for the best results
- Easy-to-Use Screen Cleaner Kit + microfiber cleaning cloth - The EVEO Screen Cleaning Kit will easily be your new favorite tool when it comes to taking care of the things you love most, quick and easy solution for a streak-free screen. Simply screw on the sprayer gun on the screen cleaner bottle. Spray the cleaning liquid onto the microfiber cloth, then wipe the screen until desired cleanliness & shine is achieved.
- Quality Screen Care Kit Worth Purchasing - Enjoy a convenient Screen Cleaner anytime, anywhere you need it. Your satisfaction is our top priority. We stand behind the quality of our products and that’s why, you shouldn’t be worrying at all. We manufacture the Best Screen Cleaner. and can be used for laptop cleaning kit, laptop screen cleaner, tv screen cleaner for smart tv. MacBook screen cleaner, monitor cleaner or screen cleaner spray for electronic devices with microfiber cleaning cloth
3. Check rate limits, queues and concurrency
Traffic controls can add waiting time even when model generation has not changed. Inspect HTTP 429 counts, provider and gateway rate-limit counters, queue depth, in-flight concurrency and burst size on the same timeline as latency.
What a 429 pattern means
A 429 can be returned immediately, or it can trigger client or gateway retry logic. The immediate error may be fast, while the eventual successful response is slow because it includes one or more waits. Count both the 429 and the attempts that followed it.
What queue growth means
A queue that grows with concurrency indicates work is arriving faster than the gateway or provider can admit it. Compare arrival rate, completion rate and queue wait. A sudden burst can create a tail-latency increase without changing the median until the backlog persists.
Cloudflare limit to verify
Cloudflare documents a Unified Billing limit of 200 requests per 60 seconds per gateway in 2026; exceeding that limit returns HTTP 429. Confirm that the limit applies to your account and traffic path, then check whether multiple applications share the same gateway budget.
Rank #3
- 【Large size】：Measuring 800 x 300 mm/ 31. 5×12 inches, which is wider and taller than others. Large mouse pad can perfectly fit your keyboard and mouse of any size, and there is also ample space for peripherals such as phones, tablets, etc.
- 【LED backlight mouse pad】：Blade Hawks RGB mouse pad provides 7 static modes and 7 dynamic modes will give you the options you are looking for. Power-off memory function remembers the last lighting mode you selected. The super glow fiber Chroma will give you a colorful gaming setup that you want.
- 【Anti-slip rubber base】：Made from Natural rubber, this gaming mousepad will firmly grip your desktop, allowing you to enjoy the ultimate precision in the games you are most passionate about. It also comes with a texture that enhances this ability even further.
- 【Smooth and waterproof surface】: Micro-textured cloth surface, for extremely precise mouse control and a smooth movement. When liquid splashes on the gaming mouse pad, it forms water droplets and slides down. Will not delay your work or gameplay.
- 【Plug and play】: This mouse and keyboard pad is powered by USB, plug and play, no driver required. One button control light modes. Press one time to change the lighting mode, press twice to change the brightness, press and hold about 3 seconds turn ON/OFF.
4. Account for retries and timeout behavior
Retries often explain why a gateway appears slow even when each individual upstream attempt is normal. For every request, record the attempt number, start and end time, retry reason, backoff delay, timeout deadline and final outcome.
Bound the retry budget
Use exponential backoff with jitter, a maximum attempt count and a request-wide deadline. Independent clients, workers and gateways that all retry the same operation can overlap and multiply load. A retry should not restart after the user-visible deadline has already expired.
Interpret timeout settings
A timeout can return an error or trigger a retry when a provider takes too long to respond. A short connect timeout is appropriate for an unreachable network path; a read timeout must allow the expected TTFT and generation window for the selected model. Keep connect, first-byte and total-deadline values distinct so one slow phase does not consume an unlimited budget.
Cloudflare retry limits to verify
Cloudflare’s AI Gateway documentation describes automatic retries with a maximum of five retry attempts and retry delays up to 60 seconds. Treat those as documented product limits, not as a target configuration. Record whether your own client adds another retry layer.
Rank #4
- 【Softer and More Comfortable】Vaydeer wrist rest has unique diamond pattern, which is the combination of softness and aesthetics. The materials of wrist rest are improved into higher quality memory foam and covered with silky smooth lycra. The computer wrist rest makes you as comfortable and cushiony as like rest your wrists on clouds.
- 【Ergonomic Wrist Saver】The wrist rests for keyboard and mouse comes with a 17.32×3.15×0.83 inch keyboard wrist pad and a 5.94×3.15×0.83 inch mouse wrist support. Based on ergonomic design, the unique concave shape is the perfect fit for your wrist joints. The wrist rest pad fits most computer keyboards and laptops, improve hand and wrist posture, release your wrist and arm stress.
- 【Non-Slip Rubber Bottom】Featuring an anti-skid silicone base on the bottom, this wrist keyboard support stays firmly in place on your desk, preventing the padding from sliding around, ensuring stable and consistent wrist support during extended computer sessions.
- 【Better Experience & Pain Relief】Our keyboard arm rest is beneficial to alleviate the soreness caused by direct contact and friction between your arm and a hard desk surface, reducing the risk of wrist fatigue or carpal tunnel. The soft texture of memory foam can evenly distribute the pressure around your wrists and provide good support with just enough give.
- 【Helpful in Multiple Scenarios】Whether you're working, studying, writing, typing, gaming, this keyboard and mouse rest combo is an essential accessory to add comfort and support to your hands and wrists. It’s also a great gift for men, women, family, friend, coworker, gamer, teacher, etc.
5. Verify cache hits, keys and streaming behavior
A cache miss sends work to the provider and can increase both TTFT and total duration. A cache hit may return quickly, but only when the request is valid to reuse and the cache key is stable.
- Confirm caching is enabled for the route and environment where the regression occurs.
- Check the cacheable method, response status and request-size eligibility.
- Compare hit and miss rates by model, project and region. A key change or a header that varies per request can turn nearly identical prompts into misses.
- Inspect authorization, content-type and custom headers used in the cache key. Remove only headers that are safe to ignore; never share a response across users when it contains private data.
- Verify how streaming is handled. Cloudflare states that streaming responses are not cached by default, so a streaming route can show zero cache hits even when a non-streaming route is cacheable.
Cloudflare documents a 25 MB maximum cacheable request size and a maximum cache TTL of one month in 2026. Check the current account and product documentation before relying on either value.
6. Separate network delay from provider delay
Correlate each gateway request with the upstream provider request ID, if one is returned. Build a timing breakdown that includes DNS, TCP connect, TLS handshake, time waiting for an available pooled connection, upload, provider wait, TTFT and download.
Free tools Windows power users keep installed
One-click scans. No signup required.
When the provider has a matching request
If the provider dashboard or logs show the request, compare its server-side timing with your gateway timing. A small provider duration and a large gateway duration points to connection setup, proxy buffering, response handling or retries in your path. A large provider-side wait with a matching request points toward provider queueing, model work or service-tier capacity.
Best Value
- 【No Drill Mounted】Cable management tray can be assembled in as little as 3 minutes or less-simply. Side clips easily clampe the edge of desk, pass through the holes on the sides, and secure excess cables with baskets. Desk cable management supports inward or outward installation to meet your needs in different scenarios, and provides you with great convenience when collecting and organizing wires.
- 【Rubber Pad Protects Desk】Wire organizer under desk built-in soft rubber pads on the clamp protect your desk from scratches, so you will not damage your furniture or office.
- 【Sturdy and Beautiful】Cord management under desk is made of high-quality carbon steel and holds about 15 lbs. The wire basket has 2 holes on each side of the tray to enable your cables to pass freely. Cable baskets are suitable for desk thickness use from 0.4" to 2.4".
- 【Space Saving & Keep Organized】Desk cable management tray holds all your power cords, outlet strips, USB hubs and computer transformers hidden from your feet, and the data cables will not enter the vacuum cleaner to keep your desk clean and tidy. Desk wire organizer is also suitable for use in the kitchen and outdoors.
- 【Reduce Safety Risks】Cable tray under desk keeps all plugs away from your kids, no more worry about tripping. Using 2 holes wire tray saves extra space, helps you easily cabling and wire protection. The mesh design is not easy to accumulate dust, but avoids safety accidents caused by messy wires.
When the provider has no matching request
A missing provider-side request usually implicates the client, proxy, load balancer, firewall, DNS, TLS negotiation or egress path. Review connection and read timeouts, rejected outbound connections, certificate errors, DNS failure rates and stale pooled connections. Also check whether a request was abandoned locally before it reached the provider.
Use geography as a test
Compare identical traffic from more than one gateway region. A regression isolated to one region is evidence for a local network or egress issue; a simultaneous change everywhere is more consistent with gateway policy, provider capacity or request-shape changes.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.7. Inspect the model work itself
Model workload changes can raise latency without any infrastructure fault.
- Uncached prompt tokens increase the work required before the first token, so compare prompt-token distributions and TTFT together.
- Reasoning effort can increase TTFT even when the visible prompt is unchanged. Track the model’s reasoning setting or requested budget where the provider exposes it.
- More output tokens increase the interval after the first token and therefore total request time.
- Token velocity (tokens per second) distinguishes a longer requested completion from a slower generation rate.
Segment these measurements by model and request class. A change in application prompts, retrieved context or maximum output can create a latency regression that looks like a provider slowdown.
Compare plausible causes with a decision matrix
| Observation | Most likely area | Next check |
|---|---|---|
| P50, P95 and P99 rise for every model and project | Gateway queue, policy layer or shared network path | Queue depth, concurrency, gateway CPU, DNS and connection setup. |
| Only P95/P99 rise; median is stable | Bursts, retries, timeouts or a regional path | Attempt counts, 429s, backoff, tail by region and pooled-connection age. |
| One model or service tier changes | Provider queue or model-specific work | Provider request IDs, tier counters, TTFT and prompt-token distribution. |
| TTFT is stable but total time rises | Output length or generation velocity | Output tokens, token-per-second rate and max-output settings. |
| Gateway log exists but provider log does not | DNS, TCP, TLS, firewall, proxy or local timeout | Connection-phase timings and rejected or abandoned attempts. |
| Cache-hit rate falls while request shape is unchanged | Cache key, header or policy change | Key components, eligibility, streaming mode and request-size limits. |
Instrument the gateway so the next regression is diagnosable
Emit one structured record per logical request and one record per attempt. At minimum include:
- A correlation ID, upstream request ID and attempt number.
- Timestamped durations for queue wait, policy processing, DNS, connect, TLS, upload, TTFT, download and total time.
- Model, service tier, project, region, consumer and route.
- Prompt-token estimate, output-token count, reasoning setting and streaming flag.
- Cache eligibility, cache key version, hit or miss result and response status.
- 429 status, rate-limit headers, queue depth, concurrency and retry reason.
- Timeout phase, configured deadline, cancellation result and final billing status where available.
Keep sensitive prompts out of general logs. Store token counts and a hashed or versioned request-shape identifier when full content is not required for diagnosis.
A narrow mitigation playbook
- Protect the user-visible deadline. Set separate connect, TTFT/read and total-request limits. Return a clear timeout rather than allowing an unbounded wait.
- Stop retry amplification. Cap attempts, use exponential backoff with jitter, and enforce one overall deadline across all layers.
- Control admission. Apply bounded concurrency and smoothing for bursts; alert on queue depth before it becomes a long tail.
- Fix cache eligibility. Stabilize safe cache keys, remove accidental per-request variance, and document that streaming responses are not cached by default.
- Reduce avoidable model work. Trim repeated context, cap output where appropriate, and review reasoning settings for latency-sensitive routes.
- Route around a confirmed provider issue. Use a provider fallback only after matching provider IDs and timing evidence show that the provider is the slow stage. Ensure the original request is cancelled and understand whether an abandoned or retried request can still incur provider charges.
- Re-measure by percentile. After each change, compare the same model, tier, project, region and request-size slice. A lower average with a worse P99 is not a successful fix for interactive users.
Cloudflare AI Gateway limits to keep in the runbook
| Control | Documented value | Qualification |
|---|---|---|
| Unified Billing request limit | 200 requests per 60 seconds per gateway | Cloudflare documentation, 2026; exceeding the limit returns HTTP 429. |
| Cacheable request size | 25 MB maximum | Cloudflare documentation, 2026; verify current account applicability. |
| Maximum cache TTL | One month | Cloudflare documentation, 2026; TTL is an upper limit, not a guarantee of a hit. |
| Automatic retry attempts | Five maximum | Cloudflare documentation, 2026; client-side or upstream retries may add more attempts. |
| Retry delay | 60 seconds maximum | Cloudflare documentation, 2026; long delays can consume a user-visible deadline. |
Runbook for an active incident
- Capture the current P50, P75, P95, P99 and TTFT, then freeze a known-good comparison window.
- Filter to one model, service tier, project, region and request-size band.
- Check 429s, queue depth, concurrency, burst rate, retry counts and timeout phases.
- Compare cache hits and misses, including streaming and request-size eligibility.
- Match gateway correlation IDs to provider request IDs and classify requests with no upstream record.
- Compare prompt tokens, reasoning settings, output tokens and token velocity with the baseline.
- Apply one bounded mitigation, document its expected stage of impact, and recheck the same percentiles and slice.
This sequence prevents a common mistake: changing model, timeout and retry settings at the same time, then losing the evidence needed to identify the actual source of latency.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
