TrueFoundry
Trace LLM Gateway traffic from TrueFoundry AI Gateway using OpenTelemetry
Overview
TrueFoundry AI Gateway is a proxy that sits between your applications and 1000+ LLM providers, giving you a single interface with built-in observability and governance.
Confident AI integrates with TrueFoundry over OpenTelemetry: the gateway exports OTLP traces directly to Confident AI's endpoint, so every LLM call you route through it shows up in the Observatory without instrumenting each application separately.
Before you start, make sure your TrueFoundry AI Gateway is set up and routing traffic, and that you have access to your project in Confident AI.
Tracing Quickstart
Create a Confident AI API key
Log in to Confident AI, open your project, and go to Settings → API Keys to create a new key. Store it securely, you'll pass it as the
x-confident-api-keyheader when configuring TrueFoundry.Configure OTEL export in TrueFoundry
In the TrueFoundry dashboard, go to AI Gateway → Controls → Settings, open OTEL Config via the edit (pencil) icon, and turn on Otel Traces Exporter Configuration.
- Choose HTTP Configuration.
- Set Endpoint to Confident AI's OTLP traces URL:
https://otel.confident-ai.com/v1/traces- Set Encoding to
Json.
TrueFoundry AI Gateway OTEL Config Configure headers and save
Turn on Headers and add:
Header Value Content-Typeapplication/jsonx-confident-api-key<your-confident-api-key>Swap in the key from step 1, then click Save.
Verify traces in the Observatory
Send a few requests through the gateway, then check TrueFoundry's Monitor page to confirm traffic is flowing. Back in Confident AI, open Observability → Traces — you should see new traces with latency, environment, and status for each request.
From here you can search traces by UUID, filter by environment, inspect cost and latency, and run evals on incoming traffic.
Configuration reference
| Setting | Value |
|---|---|
| Traces endpoint (US/default) | https://otel.confident-ai.com/v1/traces |
| Traces endpoint (EU) | https://eu.otel.confident-ai.com/v1/traces |
| Protocol | HTTP (OTLP) |
| Encoding | JSON |
| Authentication Header | x-confident-api-key: <your-confident-api-key> |
| Content-Type | application/json |
Advanced Usage
Online evals
Once TrueFoundry traces are landing in Confident AI, you can run online evals against them directly from the Observatory, no extra configuration on the TrueFoundry side is needed.
Distributed tracing
If your own services also emit OpenTelemetry spans, you can correlate them with the gateway-level traces from TrueFoundry for a full picture end to end. See the OpenTelemetry integration guide for attribute references and multi-service setup.