OpenTelemetry

OpenTelemetry (OTEL) is the open, vendor-neutral standard for traces, metrics, and logs. Confident AI's LLM tracing is OpenTelemetry-native: every trace arrives as OTLP, whether it came from confident-trace, a third-party instrumentor, or a raw OpenTelemetry SDK.

Overview

Confident AI's OpenTelemetry server (OTEL server) is the OTLP/HTTP ingestion endpoint for all tracing data, hosted at https://(eu.)otel.confident-ai.com. All spans enter through this endpoint, whether produced by the confident-trace SDKs, a third-party integration, a cloud runtime, or an OpenTelemetry Collector.

The OTEL server accepts standard OTLP, so confident-trace is not required. However, confident-trace is still highly recommended even if you are using frameworks that are OTel native such as Google ADK, Pydantic AI, or AgentCore because of confident-trace's native integration with 25+ AI providers, frameworks, gateays, all at once.

The OTEL Server

flowchart LR
    subgraph App["Your application"]
        direction TB
        A[confident-trace<br/>25+ auto-instrumentations]
        B[OpenInference / OpenLLMetry<br/>instrumentors]
        C[Raw OpenTelemetry SDK<br/>Go, Ruby, C#, Java...]
        D[Cloud runtime<br/>AgentCore, Foundry, Gemini Enterprise]
    end
    subgraph Export["Export (OTLP/HTTP)"]
        direction TB
        X[OTLP exporter<br/>in-process]
        Y[OpenTelemetry Collector<br/>optional, fan-out to other backends]
    end
    A --> X
    B --> X
    C --> X
    D --> X
    X -.-> Y
    X --> S
    Y --> S
    S[Confident AI OTEL server<br/>otel.confident-ai.com] --> O[Confident AI]
    O --> E[Online evals]
    O --> T[Trace forwarding]

Spans reach Confident AI in one of two ways:

• Directly from the process. The SDK's OTLP exporter (which confident-trace configures for you) batches spans and POSTs them with your project API key in the x-confident-api-key header.
• Through an OpenTelemetry Collector. Your services export to a Collector you run, and its otlphttp exporter sends a copy to Confident AI alongside your other backends. See trace broadcasting.

On receipt, Confident AI authenticates the request, reads attributes from every namespace it understands, and reconstructs traces from the OpenTelemetry trace_id and parent span IDs. Spans from different services and languages therefore land in the same trace as long as they share W3C trace context — the basis of distributed tracing.

Regions and endpoints

Use the base endpoint for OTEL_EXPORTER_OTLP_ENDPOINT (the SDK appends /v1/traces) and the traces endpoint for OTEL_EXPORTER_OTLP_TRACES_ENDPOINT or CONFIDENT_OTEL_ENDPOINT. See data residency and self-hosting.

Protocol support

Limits and responses

The OTEL server isn't subject to the API's rate limit, so exporters can send as fast as their batching produces spans. What it does enforce is your project's Trace Storage usage limit: the size of every request, after decompression, counts toward it. Once the project reaches the limit, every request returns 429 until the next billing cycle starts or the limit is raised. On the Free plan and during a trial, the limit is 1 GB-month.

How confident-trace Works

confident-trace is the OpenTelemetry SDK pre-configured for Confident AI, not a proprietary SDK with an exporter bolted on. Every span it creates — from an automatic integration, the @span decorator or span() context manager in Python, or withSpan() in TypeScript — is a real OpenTelemetry span carrying the same gen_ai.* and confident.* attributes you would set by hand.

init() does the following once at startup:

Its helpers write confident.* attributes so you never touch attribute names directly:

Because it is plain OpenTelemetry underneath:

• Spans from tracer.start_as_current_span(...) and confident-trace nest in the same trace with no glue code.
• An existing TracerProvider can be reused — init(tracer_provider=provider) in Python, createSpanProcessor() from confident-trace/otel in TypeScript. See existing OpenTelemetry provider.
• OTEL_RESOURCE_ATTRIBUTES, OTEL_SERVICE_NAME, and W3C propagation apply unchanged.

Which Spans Are Exported

By default, confident-trace exports only AI spans, meaning spans with confident.* or gen_ai.* data, plus their parent spans. Other spans, such as HTTP requests or database queries, are exported only when they are a parent of an AI span.

To export every span:

from confident_trace import init

init(export_non_ai_spans=True)

import { init } from "confident-trace";

init({ exportNonAiSpans: true });

GenAI Semantic Conventions

The OpenTelemetry GenAI semantic conventions define the gen_ai.* vocabulary for model calls, tool executions, and agent invocations. Confident AI adopts them as its baseline, so any conforming instrumentor produces typed traces without Confident-specific attributes.

Span types

When no confident.span.type is present, the span type is inferred from gen_ai.operation.name:

Attributes

Message content is accepted in every form the conventions have used, so older instrumentors keep working:

• Span attributes (current) — gen_ai.input.messages / gen_ai.output.messages as JSON arrays
• Span events (legacy) — gen_ai.content.prompt, gen_ai.content.completion, and per-message gen_ai.*.message events
• Indexed attributes (OpenLLMetry-style) — gen_ai.prompt.{n}.* / gen_ai.completion.{n}.*

Supported Attribute Namespaces

Beyond gen_ai.*, the OTEL server recognizes the vocabularies of the most common LLM instrumentation ecosystems. Spans from any of these render as typed spans with input, output, model, and token usage populated.

OpenInference

Setup with confident-trace as the exporter is on the OpenInference integration page.

OpenLLMetry

Point the Traceloop SDK at Confident AI with its standard exporter settings:

export TRACELOOP_BASE_URL="https://otel.confident-ai.com"
export TRACELOOP_HEADERS="x-confident-api-key=confident_us..."

Attribute precedence

When a span carries the same concept in several namespaces, Confident AI resolves it in this order:

1. confident.* — always wins, so you can override anything an instrumentor set
2. gen_ai.* — current names, then deprecated aliases
3. OpenInference (openinference.*, llm.*)
4. OpenLLMetry (traceloop.*, llm.*)
5. Framework-specific namespaces (ai.*, mastra.*, …)
6. Inference from span name and structure

Unconsumed attributes are kept verbatim as span metadata.

Choosing an Approach

Last updated on October 7, 2026