OpenTelemetry GenAI conventions and an LLM observability platform solve different parts of the problem: the conventions describe how to represent AI operations in telemetry, while a platform ingests and interprets that data and provides views and workflows. You can use OpenTelemetry instrumentation with a vendor backend, but sending spans over OTLP does not guarantee the backend understands every GenAI attribute or offers the same LLM-specific features.
What is the difference?
OpenTelemetry is an instrumentation and telemetry layer. Its GenAI semantic conventions provide a shared vocabulary for describing operations such as model calls. A backend is the destination where traces are ingested, mapped, displayed, and analyzed. A vendor-specific LLM observability platform may add workflows for prompts, token usage, costs, scoring, or evaluation.
These are not mutually exclusive choices. A team can instrument with OpenTelemetry and send the resulting data to a vendor platform. The key question is whether the destination supports the specific convention and version emitted, and what it does with those spans after ingestion.
What to compare before choosing
Convention and version support
OTLP is a transport, not a guarantee of semantic compatibility. Confirm the exact GenAI convention and version the backend accepts, whether it also supports conventions such as OpenInference, which attributes it requires, and whether it maps, transforms, or drops spans. OpenTelemetry’s conventions page identifies version 1.44.0 and notes that GenAI conventions have moved to a separate repository; consult that repository for current convention details: OpenTelemetry semantic conventions [c001].
Free tools Windows power users keep installed
One-click scans. No signup required.
#1 Best Overall
Instrumentation coverage and trace context
Check whether instrumentation covers the frameworks, model providers, tools, and retrieval steps in your application. Determine whether custom spans are needed and whether AI operations appear within the ordinary request trace, with agent and tool steps nested in a useful way. This context can make it easier to see where an AI call fits in a service request, rather than viewing it in isolation.
LLM-specific workflows
Trace ingestion is not the same as a complete LLM workflow. If you need token usage, cost tracking, prompt linking, scoring, experimentation, or evaluation, verify those capabilities in the current product documentation. A backend may accept trace data without providing every one of these features.
Privacy and data handling
Prompt and completion bodies can contain personal information, credentials, or regulated data. Review defaults for content capture, filters, attribute-level obfuscation, baggage propagation, retention, hosting region, and compliance needs. Treat baggage carefully: it propagates across service boundaries and may reach third-party APIs.
How documented platform support differs
| Platform | Documented OpenTelemetry or GenAI handling | Notable checks |
|---|---|---|
| Datadog Agent Observability | Datadog documents ingestion of traces using OpenTelemetry GenAI semantic conventions v1.37+ or supported OpenInference conventions. It maps data into its Agent Observability span schema. Datadog documentation [c002] | Use compatible instrumentation or custom spans with required attributes. Datadog says traces can be dropped if no span qualifies using listed GenAI, OpenInference, or Langfuse attributes; spans without any gen_ai.* attribute can also be dropped individually. |
| New Relic AI Monitoring | New Relic documents sending GenAI spans through OTLP and displaying LLM calls, tools, and agent steps in request traces. New Relic AI Monitoring documentation [c003] | Prerequisites include an ingest license key, an instrumented LLM application, and network egress to the account-region endpoint. New Relic states, “Content capture is off by default,” for most instrumentations; review filters or attribute-level obfuscation before enabling capture. |
| Langfuse | Langfuse documents an OTLP endpoint and an OpenTelemetry-native SDK v4 that converts spans into Langfuse observations. Its SDK includes helpers for token usage, cost tracking, prompt linking, and scoring. Langfuse OpenTelemetry documentation [c004] | Other OTEL-instrumented libraries can share OpenTelemetry context. Review deployment and data-location options, and avoid placing sensitive information in baggage. |
| Amazon OpenSearch Service | AWS describes AI observability built on GenAI semantic conventions and natively integrated with OpenTelemetry, including hierarchical traces for agent orchestration, LLM calls, tool invocations, and retrieval. Amazon OpenSearch Service documentation [c006] | AWS documents an instrumentation example using GenAI attributes and an OpenSearch Ingestion pipeline. Check the example against your own instrumentation and trace structure. |
Validate a backend with a representative trace
- List the data you need. Decide whether prompts or completions must be captured, which AI operations must be visible, and whether they need to correlate with ordinary service requests.
- Map your instrumentation. Identify the frameworks and providers in use, their emitted convention and version, and any custom spans or qualifying attributes required by the candidate backend.
- Send a representative trace. Include the model call and, where relevant, tool or retrieval activity. Inspect the backend’s resulting trace to see which spans and attributes it recognizes, maps, transforms, or drops.
- Test required workflows and controls. Check the LLM-specific features your team needs and verify content-capture defaults, filtering, obfuscation, baggage, retention, and hosting-region requirements.
- Recheck current documentation. Conventions, instrumentation packages, and product support can change; confirm the version and behavior you plan to rely on before rollout.
Choose by requirement, not by label
- Consider a general APM backend if correlating AI work with existing service traces is a priority and the backend documents support for your emitted conventions and trace structure.
- Consider an LLM-focused platform if you need workflows such as prompt linking, token and cost tracking, or scoring; confirm those features and their data-handling implications in current documentation.
- Prioritize deployment constraints if hosting model, region, or data location is decisive; verify the options the specific service currently documents.
These are decision criteria, not claims that one category is universally better. A practical choice depends on the traces your application emits, the interpretation and workflows the destination supports, and the data your organization is prepared to send.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →How to interpret older support claims
LangChain’s December 9, 2024 announcement described direct OpenTelemetry trace ingestion in LangSmith using the OpenLLMetry semantic convention. It said support for other conventions, including OpenTelemetry GenAI, was planned at that time. That announcement establishes what was described on that date, not LangSmith’s current support status; check current LangSmith documentation before relying on it. LangChain’s announcement [c005]
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
