Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Use Grafana Agent Observability with OpenTelemetry to instrument every model generation, tool call, and workflow step. Send metrics to Grafana’s Prometheus-compatible path and traces to a backend such as Tempo, then build panels for operation duration, time to first token (TTFT), token counts, estimated cost, tool-call performance, and errors. Link those panels to exemplars and traces so a latency spike or failed request can be traced to the exact model, tool, or workflow step. Grafana provides SDKs for Go, Python, TypeScript, Java, and .NET, plus integrations for LangChain, LangGraph, OpenAI Agents, LlamaIndex, Google ADK, and Vercel AI SDK.
What does Grafana need to observe an AI agent?
An agent must emit four related kinds of telemetry: metrics for aggregate behavior, traces for execution paths, logs or events for individual failures, and conversation or generation records for model and prompt context. OpenTelemetry provides the transport and context propagation; Agent Observability supplies the AI-specific fields and views.
| Signal | Use it for | Examples of information |
|---|---|---|
| Metrics | Trends, service objectives, and alerts | Latency distributions, TTFT, token consumption, generation counts, tool-call counts, and error rates |
| Traces | Finding where time was spent | The path from a user request through generations, workflow steps, and tools |
| Logs and events | Explaining one failure | Error payloads, exception context, and operational details for a request |
| Conversation and generation records | Analyzing AI-specific behavior | Prompt and model context, token breakdown, cost, tool information, and quality annotations |
A single conversation can contain several model generations. Keep conversation totals and individual generation totals separate when you design panels; otherwise, a multi-step request can look like duplicate traffic.
What architecture should I use?
There are two supported routing patterns: export OTLP directly to Grafana Cloud, or send OTLP through Grafana Alloy or an OpenTelemetry Collector before it reaches your Grafana backends. Metrics use the Prometheus-compatible path, while traces are stored in a trace backend such as Tempo.
#1 Best Overall
- Powerful Turbo Fan:CAGIWIRU Electric Air Duster have reached speed up to 130000RPM,Can efficient remove dust and debris.With LED light and Three Adjustable Speed to meet with different cleaning tasks.
- With Attached USB C Fasting Charging cable.Charing 3 hours,Enjoy up to 240 minutes of use on the lowest setting, For third speed can use for 25mins.with four charging options to meet with your needs.
- Wide Application: attached 5 different nozzles,making it suitable for a variety of scenes, such as car,pet, pc, keyboards, and other electronic devices. It also serves for office and home clean duster.
- Handy Design & Portable:Cordless air duster electric nozzle is Light weight and small size, design and comfortable to hold and space saving, convenient to carry around. You can put it in your room or inside the car without taking up space.
- Economical &Reusable: Built durable materials with a long-lasting battery,our CAGIWIRU air duster it's a sustainable alternative to disposable air cans.
Direct export to Grafana Cloud
This is the shortest path for a managed deployment. The agent’s SDK or OpenTelemetry instrumentation exports generation data, traces, and metrics to the OTLP endpoints provided by your Grafana Cloud stack. Use this when you want Grafana to operate the storage and query services.
Export through Alloy or an OpenTelemetry Collector
A collector gives you a control point for batching, filtering, redaction, routing, and multi-destination export. It is useful when several services share one telemetry pipeline or when policy requires sensitive fields to be removed before they leave your network.
Components to verify before instrumentation
- An Agent Observability SDK or compatible OpenTelemetry instrumentation for the language and framework used by your agent.
- An OTLP route to Grafana Cloud or to Alloy/OpenTelemetry Collector.
- A metrics destination that supports the Prometheus-compatible queries used by Grafana dashboards.
- A trace backend, such as Tempo, with trace-to-metrics or exemplar links enabled in your dashboards.
- Retention, access-control, redaction, and data-residency settings that match the prompts and tool payloads your agent handles.
How do I instrument an AI agent correctly?
Install the Agent Observability SDK for your language, or add compatible OpenTelemetry instrumentation to the agent framework. Then configure both the generation exporter and the OpenTelemetry providers; configuring only one can silently drop traces or metrics.
- Choose an SDK or integration. Grafana documents SDKs for Go, Python, TypeScript, Java, and .NET. Framework integrations cover LangChain, LangGraph, OpenAI Agents, LlamaIndex, Google ADK, and Vercel AI SDK. Use the integration that matches your orchestration layer so generations and tool calls inherit the same trace context.
- Configure generation export. Enable the Agent Observability generation-export path so each model generation records its operation, model, provider, token usage, cost fields, and any quality annotations your application supplies.
- Configure an OpenTelemetry
TracerProvider. This provider carries spans for the incoming request, workflow steps, model calls, and tools. Preserve parent-child relationships so one trace shows the complete execution path. - Configure an OpenTelemetry
MeterProvider. This provider publishes duration, TTFT, token, generation, tool-call, and error instruments. Grafana warns that traces and metrics can be silently lost when the providers are not configured, even if generation records appear to be exporting. - Propagate context through asynchronous work. Agent frameworks often run tools or branches concurrently. Ensure each branch keeps the trace context supplied by the SDK rather than creating unrelated root spans.
- Send a controlled test request. Use a request that performs one generation and, separately, a request that invokes a tool. Confirm that both appear as generations or spans before rolling the instrumentation to all traffic.
Keep model, provider, agent name, application version, tool name, and environment as structured attributes. Do not put unbounded user text, complete prompts, or arbitrary tool arguments into metric labels; those fields belong in protected trace or generation records when your retention policy allows them.
Which Grafana metrics show AI agent latency and throughput?
Start with the documented Agent Observability instruments below. Plot distributions rather than only averages so queueing and long-tail requests are visible.
| Metric | What it measures | Useful breakdowns |
|---|---|---|
gen_ai_client_operation_duration_seconds |
Total duration of a model operation | Agent, model, provider, operation, version, and time range |
gen_ai_client_operation_time_to_first_chunk_seconds |
Time until the first streamed chunk | Model, provider, streaming mode, and agent |
gen_ai_client_time_to_first_token_seconds |
Time until the first token is available | Model, provider, region or environment, and version |
gen_ai_client_token_usage |
Token consumption reported for a generation | Input versus output where available, model, agent, and conversation |
gen_ai_client_tool_calls_per_operation_count |
Number of tool calls associated with an operation | Tool, agent, operation, and outcome |
Put generation count beside duration and TTFT. A rising duration with stable TTFT indicates time spent generating or in downstream work; a rising TTFT points toward queueing, provider startup, network delay, or model-side time before streaming begins. Use p95 or another percentile panel for user-impacting latency and retain a histogram view for troubleshooting.
Rank #2
- Screen Cleaner is Your Screen’s New Bestfriend! - EVEO is proud to present a screen cleaner that’s perfectly suited for your TV. Meet your new screen cleaner, computer screen cleaner, laptop cleaner, iPad cleaner, Macbook,tv cleaner or any other electronic device. Our screen cleaner spray and wipe along with the included plush microfiber cloth, easily removes dust, fingerprints & other smudges on the screen’s surface. Clean effectively for a streak-free screen
- Gentle on your Screens - Our screen cleaner spray and wipes is compatible to be used even for the most sensitive LCD, LED, CRT, and OLED screens in proper use. Be at ease knowing that this product is compatible with all the major brands: LG, SONY, Samsung, Sharp, iPad, iPhone, Android screens, Kindle, PC monitors - we’ve got you covered! Can be used as LCD screen cleaner, tv cleaner, smart tv screen cleaner, and more
- Premium Super Soft Plush Microfiber Cloth - Included in this efficient screen cleaning kit. This screen cleaner spray and wipe has our signature plush microfiber cloth specially designed to comply with screens and monitors and leave them streak-free. Has been shown to effectively absorbs stubborn stains, zero streaks, and no fiber shedding. Always use this together with our screen cleaner for the best results
- Easy-to-Use Screen Cleaner Kit + microfiber cleaning cloth - The EVEO Screen Cleaning Kit will easily be your new favorite tool when it comes to taking care of the things you love most, quick and easy solution for a streak-free screen. Simply screw on the sprayer gun on the screen cleaner bottle. Spray the cleaning liquid onto the microfiber cloth, then wipe the screen until desired cleanliness & shine is achieved.
- Quality Screen Care Kit Worth Purchasing - Enjoy a convenient Screen Cleaner anytime, anywhere you need it. Your satisfaction is our top priority. We stand behind the quality of our products and that’s why, you shouldn’t be worrying at all. We manufacture the Best Screen Cleaner. and can be used for laptop cleaning kit, laptop screen cleaner, tv screen cleaner for smart tv. MacBook screen cleaner, monitor cleaner or screen cleaner spray for electronic devices with microfiber cleaning cloth
How should I build the first Grafana dashboard?
Create one overview dashboard that answers whether the agent is healthy, then link to focused dashboards for cost, tools, and individual traces.
- Health row: show generation count, error rate, p95 operation duration, and p95 TTFT for the selected time range.
- Usage row: show input and output token usage, total generations, estimated cost, and the number of conversations. Keep conversation count distinct from generation count.
- Tool row: show tool-call count and duration, with a table of tools sorted by calls, latency, and failures.
- Breakdown controls: add variables or filters for agent, model, provider, application version, tool, and time range. Start with low-cardinality dimensions and add finer filters only in drill-down panels.
- Trace links: enable exemplars or dashboard links that open the trace, generation, conversation, or workflow-step record represented by a metric point.
- Annotations: mark deployments, model changes, prompt-template changes, and provider incidents so a latency or cost shift can be correlated with an operational event.
When a panel compares models, normalize the time range and traffic slice. A model used for short routing decisions should not be judged against a model handling long-form generations without separating those operations.
How do I investigate a latency spike?
Use metrics to establish that the spike is real, then use exemplars and traces to locate the slow component.
- Confirm the scope. Compare p95 duration and TTFT with the prior period. Filter by agent, model, provider, and version to determine whether all traffic or one slice is affected.
- Separate first-token delay from completion time. A TTFT increase implicates work before streaming starts; normal TTFT with a long total duration points to generation length, downstream tools, or workflow orchestration.
- Open an exemplar or trace. Follow the request from the conversation span to each generation and workflow step. Check whether the slow span is a model call, a tool call, a retry, or waiting between branches.
- Compare tool timing. In the Tools view, sort by duration and error rate. A tool that is called frequently can dominate user latency even when model durations look normal.
- Check version and provider dimensions. A deployment, model version, or provider change confined to one dimension is a stronger lead than a global average.
Do not infer a root cause from an aggregate average. Averages can remain flat while a small group of users experiences severe tail latency.
How can I track token usage and estimated cost?
Use Usage and cost analytics and keep cost tied to each generation. This reveals high-token agents, unnecessary prompt context, expensive model choices, and excessive reasoning tokens.
| Metric | Purpose |
|---|---|
agento11y_generation_cost_usd_total |
Accumulated estimated cost for recorded generations |
agento11y_model_price_usd_per_1k_tokens |
Configured model price used for token-based estimation |
agento11y_model_price_request_usd |
Configured per-request price component |
Read token panels at the right level
Use generation-level panels to find a prompt, model, or tool path that consumes too many tokens. Use conversation-level panels to understand the total cost of a user interaction that invokes several generations. Break token usage into input, output, and reasoning categories when the provider reports those fields.
Rank #3
- 【Large size】：Measuring 800 x 300 mm/ 31. 5×12 inches, which is wider and taller than others. Large mouse pad can perfectly fit your keyboard and mouse of any size, and there is also ample space for peripherals such as phones, tablets, etc.
- 【LED backlight mouse pad】：Blade Hawks RGB mouse pad provides 7 static modes and 7 dynamic modes will give you the options you are looking for. Power-off memory function remembers the last lighting mode you selected. The super glow fiber Chroma will give you a colorful gaming setup that you want.
- 【Anti-slip rubber base】：Made from Natural rubber, this gaming mousepad will firmly grip your desktop, allowing you to enjoy the ultimate precision in the games you are most passionate about. It also comes with a texture that enhances this ability even further.
- 【Smooth and waterproof surface】: Micro-textured cloth surface, for extremely precise mouse control and a smooth movement. When liquid splashes on the gaming mouse pad, it forms water droplets and slides down. Will not delay your work or gameplay.
- 【Plug and play】: This mouse and keyboard pad is powered by USB, plug and play, no driver required. One button control light modes. Press one time to change the lighting mode, press twice to change the brightness, press and hold about 3 seconds turn ON/OFF.
Why Grafana’s estimate may not match a provider invoice
Recorded costs can incorporate configured prices, historical rates, long-context tiers, and per-request fees. Multiplying an aggregate token count by a current public list price therefore may not reproduce the recorded total, and Grafana’s estimate is not a provider invoice. Preserve the price configuration and the generation records used for an estimate so finance or platform teams can reconcile differences.
Grafana documents that Agent Observability generation and evaluator/guard-token metering starts on October 1, 2026. Billing rules and feature names can change, so verify the current Grafana documentation and your workspace’s terms before using these figures for budgeting.
How do I find which tool call or model caused an agent error?
Start with the error-rate panel, then inspect the failed trace and its call_error payload. The payload identifies the failed call and supplies the operational context needed to distinguish a model error from a tool or orchestration error.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware match- Filter the error panel. Select the affected agent, model, provider, version, and time range. Check whether failures are concentrated in one dimension.
- Open a failed trace. Follow the span tree from the user request to workflow steps, generations, and tools. The first failed child span is usually more useful than the final top-level exception.
- Read the
call_errorpayload. Capture the error type, message, and call context without exposing secrets or user content in a dashboard intended for broad access. - Use the Tools view. Compare each tool’s duration and error rate. A model may be healthy while a dependency repeatedly times out, or a tool may be invoked incorrectly by only one agent version.
- Check retries and fan-out. Multiple generations in one conversation can multiply token usage and obscure the original failure. Trace ordering shows whether a retry preceded the final error.
Keep error events correlated with the same trace and generation identifiers used by your application logs. That correlation lets an on-call engineer move from an alert to a concrete request without searching raw text.
How do I alert on high latency, failures, cost, or quality?
Define alerts against your own service objectives and traffic volume; Grafana’s documented alert categories are error-rate thresholds, p95 latency, daily estimated-cost budgets, and evaluation-score drops.
| Alert category | Signal to evaluate | Recommended response |
|---|---|---|
| Error rate | Failed generations or calls divided by the relevant request volume | Open the error-rate panel, filter by model and tool, and inspect a failed trace and call_error payload. |
| Latency | p95 operation duration or TTFT over a defined window | Compare model, provider, version, and tool dimensions; then follow exemplars into traces. |
| Daily cost | Estimated generation cost against a daily budget | Identify high-token agents and model choices before changing traffic or limits. |
| Quality | Evaluation score or guardrail score falling below its objective | Inspect the affected conversation and generation records, then correlate the change with a model, prompt, or version deployment. |
Choose a window long enough to avoid paging on one transient request and a threshold that reflects the SLO for that agent. High-volume agents may need a rate or percentile condition, while low-volume agents need a minimum sample requirement so one failure does not create a misleading percentage.
Rank #4
- 【Softer and More Comfortable】Vaydeer wrist rest has unique diamond pattern, which is the combination of softness and aesthetics. The materials of wrist rest are improved into higher quality memory foam and covered with silky smooth lycra. The computer wrist rest makes you as comfortable and cushiony as like rest your wrists on clouds.
- 【Ergonomic Wrist Saver】The wrist rests for keyboard and mouse comes with a 17.32×3.15×0.83 inch keyboard wrist pad and a 5.94×3.15×0.83 inch mouse wrist support. Based on ergonomic design, the unique concave shape is the perfect fit for your wrist joints. The wrist rest pad fits most computer keyboards and laptops, improve hand and wrist posture, release your wrist and arm stress.
- 【Non-Slip Rubber Bottom】Featuring an anti-skid silicone base on the bottom, this wrist keyboard support stays firmly in place on your desk, preventing the padding from sliding around, ensuring stable and consistent wrist support during extended computer sessions.
- 【Better Experience & Pain Relief】Our keyboard arm rest is beneficial to alleviate the soreness caused by direct contact and friction between your arm and a hard desk surface, reducing the risk of wrist fatigue or carpal tunnel. The soft texture of memory foam can evenly distribute the pressure around your wrists and provide good support with just enough give.
- 【Helpful in Multiple Scenarios】Whether you're working, studying, writing, typing, gaming, this keyboard and mouse rest combo is an essential accessory to add comfort and support to your hands and wrists. It’s also a great gift for men, women, family, friend, coworker, gamer, teacher, etc.
Include agent, model, provider, version, and environment in alert annotations. A notification that says only “latency high” forces the responder to repeat the dashboard filtering work.
Free tools Windows power users keep installed
One-click scans. No signup required.
What should I compare in Grafana Cloud versus a self-managed stack?
The important choice is not a feature checklist alone; it is how each deployment handles instrumentation, storage, query performance, operations, and data governance.
| Decision axis | Grafana Cloud setup | Self-managed Grafana with Alloy or an OpenTelemetry Collector | Other hosted OpenTelemetry stack |
|---|---|---|---|
| Instrumentation coverage | Use the documented Agent Observability SDKs and framework integrations. | Use the same SDKs, with your collector handling routing. | Coverage depends on the vendor’s support for GenAI semantic conventions and integrations; not stated. |
| Metrics and trace retention | Determined by the selected Grafana Cloud services and plan; not stated. | Configured and operated by your team; capacity and retention are not stated. | Vendor-specific; not stated. |
| Cardinality and query performance | Managed service limits and query behavior apply; not stated. | You control sizing, indexing, and limits but own tuning. | Vendor-specific; not stated. |
| Drill-down and alerting | Use Grafana dashboards, exemplars, traces, and documented alert categories. | Reproduce those links and alert rules across your components. | Depends on the platform; not stated. |
| Privacy, residency, and tenant isolation | Review workspace controls and regional terms before exporting prompts or tool payloads. | Keep data inside your chosen network and enforce your own controls. | Review the provider’s controls; not stated. |
| Operating cost | Managed-service usage and retention pricing; current amounts are not stated here. | Infrastructure, storage, upgrades, and on-call labor; amounts are not stated here. | Vendor-specific pricing; not stated. |
A collector-based design can also be a migration path: instrument once with OpenTelemetry, route to a self-managed backend during evaluation, and later change exporters without rewriting agent code. Validate semantic fields and trace links at each destination rather than assuming every backend renders the same GenAI records.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What privacy and reliability pitfalls should I avoid?
Agent telemetry can contain prompts, responses, uploaded documents, tool arguments, and identifiers. Treat it as application data, not harmless infrastructure metadata.
- Redact before export. Remove credentials, authorization headers, payment data, personal identifiers, and confidential documents in the SDK or collector where possible.
- Limit access. Separate permissions for aggregate dashboards from permissions to read full prompts and tool payloads.
- Set retention deliberately. Keep high-cardinality or sensitive generation records only as long as incident response, quality review, and compliance require.
- Control metric cardinality. Never label metrics with raw prompts, conversation IDs, URLs containing user data, or unbounded error strings.
- Watch exporter failure modes. Batch and retry settings can create backpressure. Monitor the telemetry pipeline itself so an agent does not appear healthy merely because its observability exporter stopped sending.
- Version dashboards and rules. Metric names, feature availability, prices, and billing rules change quickly. Record the dashboard and alert version used for an incident.
What is a practical incident workflow?
Use the same sequence for latency, cost, and reliability incidents so responders reach a specific generation quickly.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →- Open the overview dashboard and select the incident time range.
- Filter by agent, model, provider, version, and environment until the affected slice is isolated.
- Compare generation count, p95 duration, TTFT, token usage, tool calls, and error rate.
- Open an exemplar or trace from the abnormal panel.
- Inspect each generation and workflow step, then read the relevant tool details or
call_errorpayload. - Check deployments, model changes, prompt changes, provider status, and collector health.
- Apply the smallest corrective action—rollback, route traffic, fix a tool dependency, or reduce unnecessary context—and verify the same panels after the change.
- Record the affected dimensions and trace identifiers so a later cost or quality review can use the same evidence.
Frequently asked questions
Can one trace represent an agent that calls several models?
Yes. Keep the incoming request as the parent trace and record each model generation as a child span or generation record. This preserves ordering and lets you attribute tokens, cost, and latency to each model while still measuring the conversation as a whole.
Best Value
- 【No Drill Mounted】Cable management tray can be assembled in as little as 3 minutes or less-simply. Side clips easily clampe the edge of desk, pass through the holes on the sides, and secure excess cables with baskets. Desk cable management supports inward or outward installation to meet your needs in different scenarios, and provides you with great convenience when collecting and organizing wires.
- 【Rubber Pad Protects Desk】Wire organizer under desk built-in soft rubber pads on the clamp protect your desk from scratches, so you will not damage your furniture or office.
- 【Sturdy and Beautiful】Cord management under desk is made of high-quality carbon steel and holds about 15 lbs. The wire basket has 2 holes on each side of the tray to enable your cables to pass freely. Cable baskets are suitable for desk thickness use from 0.4" to 2.4".
- 【Space Saving & Keep Organized】Desk cable management tray holds all your power cords, outlet strips, USB hubs and computer transformers hidden from your feet, and the data cables will not enter the vacuum cleaner to keep your desk clean and tidy. Desk wire organizer is also suitable for use in the kitchen and outdoors.
- 【Reduce Safety Risks】Cable tray under desk keeps all plugs away from your kids, no more worry about tripping. Using 2 holes wire tray saves extra space, helps you easily cabling and wire protection. The mesh design is not easy to accumulate dust, but avoids safety accidents caused by messy wires.
Should prompts and tool arguments be stored in every environment?
No. Production dashboards usually need aggregate metrics and trace identifiers, while full payloads may belong only in a restricted environment or a short-retention store. Decide which fields are necessary for debugging, redact secrets before export, and apply access controls to any record that retains user content.
How can I test that telemetry is not silently missing?
Send a known request that performs one generation and another that performs a tool call, then verify a generation record, a metric sample, and a connected trace for each. Repeat the check after changing exporters, collectors, SDK versions, or dashboard queries.
Why can token totals differ between a conversation panel and a model panel?
A conversation can invoke multiple generations, retries, or different models. The conversation total aggregates those calls, whereas a model panel includes only generations attributed to its selected model and filters.
When should I use TTFT instead of total duration for an alert?
Use TTFT when users are sensitive to how quickly streaming begins, such as interactive chat. Use total duration when the complete answer, tool workflow, or background job must finish within an objective; many agents need both signals.
Frequently Asked Questions
Can one trace represent an agent that calls several models?
Yes. Keep the incoming request as the parent trace and record each model generation as a child span or generation record. This preserves ordering and lets you attribute tokens, cost, and latency to each model while still measuring the conversation as a whole.
Should prompts and tool arguments be stored in every environment?
No. Production dashboards usually need aggregate metrics and trace identifiers, while full payloads may belong only in a restricted environment or a short-retention store. Decide which fields are necessary for debugging, redact secrets before export, and apply access controls to any record that retains user content.
How can I test that telemetry is not silently missing?
Send a known request that performs one generation and another that performs a tool call, then verify a generation record, a metric sample, and a connected trace for each. Repeat the check after changing exporters, collectors, SDK versions, or dashboard queries.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsWhy can token totals differ between a conversation panel and a model panel?
A conversation can invoke multiple generations, retries, or different models. The conversation total aggregates those calls, whereas a model panel includes only generations attributed to its selected model and filters.
When should I use TTFT instead of total duration for an alert?
Use TTFT when users are sensitive to how quickly streaming begins, such as interactive chat. Use total duration when the complete answer, tool workflow, or background job must finish within an objective; many agents need both signals.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
