OpenAI Agents
Use Confident AI for LLM observability and evals for OpenAI Agents
Overview
OpenAI Agents is a lightweight framework for creating agentic workflows using agent swarms, handoffs, and tool use. Confident AI lets you trace and evaluate OpenAI Agents workflows with one line of code — call init() from confident-trace, Confident AI's OpenTelemetry-native tracing SDK, and your agents, tools, handoffs, and guardrails stay exactly as they are.
| Runtime | Requirements | Setup |
|---|---|---|
| Python | Python 3.10+, openai-agents, confident-trace[openai-agents] extra | Call init() before agent runs |
| TypeScript | Node.js 22+, @openai/agents >=0.17.0 <0.18 | Call init() and launch your entry point with the preload |
Auto-Instrument
Install Dependencies
Run the following command to install
confident-tracealongside the OpenAI Agents SDK:pip install 'confident-trace[openai-agents]' openai-agentstsxis only needed if you run TypeScript source directly.npm install confident-trace '@openai/agents@>=0.17.0 <0.18' npm install -D tsxyarn add confident-trace '@openai/agents@>=0.17.0 <0.18' yarn add -D tsxSet Your API Keys
Get your Confident AI Project API key and set it as an environment variable, along with your OpenAI key:
export CONFIDENT_API_KEY="<your-confident-project-key>" export OPENAI_API_KEY="<your-openai-key>"Instrument OpenAI Agents
Call
init()once before running agents. It detects the Agents SDK automatically and hooks its tracing — there's no trace processor to register in your code.main.py from agents import Agent, Runner from confident_trace import init, shutdown init() agent = Agent(name="Assistant", instructions="You are a helpful assistant") try: result = Runner.run_sync(agent, "Write a haiku about recursion in programming.") print(result.final_output) finally: shutdown()src/index.ts import { init } from "confident-trace"; import { Agent, run } from "@openai/agents"; const runtime = init(); const agent = new Agent({ name: "Assistant", instructions: "You are a helpful assistant", }); try { const result = await run(agent, "Write a haiku about recursion in programming."); console.log(result.finalOutput); } finally { await runtime.shutdown(); }TypeScript needs one more thing: launch your entry point with the
confident-trace/registerpreload so the SDK can hook@openai/agentsas Node loads it.init()handles export, the preload handles instrumentation — you need both.Run OpenAI Agents
Run your script to send the trace to Confident AI:
python main.py# Running TypeScript source directly node --import tsx --import confident-trace/register src/index.ts # Running compiled JavaScript node --import confident-trace/register dist/index.jsTo make this your normal startup command, add it to your
package.jsonscripts:package.json { "scripts": { "start": "node --import confident-trace/register dist/index.js", "dev": "node --import tsx --import confident-trace/register src/index.ts" } }Done ✅. Open the Observatory in your Confident AI project to inspect the trace and its workflow, agent, and model spans.
What Gets Captured
The integration converts the Agents SDK's native tracing objects into spans and preserves their parentage, so the trace tree in the Observatory matches the run:
| Span type | Captured data |
|---|---|
| Workflow | The root of each Runner execution |
| Agent | One agent span per agent that participates in the run, including after handoffs |
| Model (LLM) | One LLM span per model request, with messages and token usage |
| Function tool | One tool span per tool execution, with input parameters and output |
| Handoff, guardrail, turn, custom | The remaining SDK span kinds, kept in their original position in the tree |
Regular and streamed Runner executions are supported, including runs that hand off between agents or trigger guardrails.
The Python bridge is an OpenInference instrumentor, so spans are forwarded with their original OpenInference attributes rather than rewritten — see the OpenInference page for how those spans are exported. A few consequences worth knowing:
- Content policy — Confident's content limits and redaction don't apply to these spans. Configure capture with OpenInference's
TraceConfigor environment settings before callinginit()if you need to. - SDK processors — the Agents SDK's own processors, including its default exporter, stay installed.
RunConfig(tracing_disabled=True)still turns framework spans off. - Provider spans — direct
openaiclient calls outside a run, and provider calls inside tools, get their own Confident LLM spans.
- Model spans — normalized text/tool messages and usage; detail depends on the model implementation emitting generation or response callbacks.
- Handoff, guardrail, and turn spans — typed
custom; the native span type stays available on the span. - Omitted — audio payloads, credentials, arbitrary trace metadata, and error text.
Set Trace Span Properties
Use a trace context to add properties you know before the call starts. It creates no extra span; the trace started by Runner.run_sync() / run() inherits the tags, metadata, and user ID.
from confident_trace import init, trace_context
from agents import Agent, Runner
init()
agent = Agent(name="Assistant", instructions="Be concise.")
with trace_context(
tags=["support"],
metadata={"release": "2026-09"},
user_id="user-42",
):
result = Runner.run_sync(agent, "Explain OpenTelemetry in one sentence.")import { init, traceContext } from "confident-trace";
import { Agent, run } from "@openai/agents";
init();
const agent = new Agent({ name: "Assistant", instructions: "Be concise." });
const result = await traceContext(
{ tags: ["support"], metadata: { release: "2026-09" }, userId: "user-42" },
() => run(agent, "Explain OpenTelemetry in one sentence."),
);See trace context for every supported trace property and update behavior.
Instrumenting Multi-Turn
You do not need turn() when one OpenAI Agents entry-point call is already one conversational turn—the integration creates that turn's trace automatically. Use turn() when you want to define the boundary yourself, such as grouping two sequential OpenAI Agents calls into one turn. Reuse the same thread ID on later turns to group them into one conversation.
from confident_trace import init, turn
init()
with turn("support-turn", thread_id="chat-42"):
context = Runner.run_sync(agent, "Find the relevant account details.")
answer = Runner.run_sync(agent, f"Summarize these details: {context.final_output}")import { init, turn } from "confident-trace";
init();
const answer = await turn({ name: "support-turn", threadId: "chat-42" }, async () => {
const context = await run(agent, "Find the relevant account details.");
return run(agent, `Summarize these details: ${context.finalOutput}`);
});See threads for thread I/O, turn IDs, and user IDs.
Troubleshooting
- No trace: make sure
init()runs before any run, and that the process reachesshutdown()so buffered spans are flushed. - Model calls traced but no agent, tool, or handoff spans: install the extra with
pip install 'confident-trace[openai-agents]', and confirmtracing_disabledisn't set on the run. - Duplicate spans: don't attach a second OpenAI Agents instrumentor to the same process.
- Incomplete streams: drain or cancel streamed runs before
shutdown(); otherwise spans end without their final output. - Spans go to the wrong exporter: an instrumentor configured with its own tracer provider before
init()keeps sending there. Use the global provider, or pass the same one toinit(tracer_provider=...). See existing OpenTelemetry provider.
- No trace: make sure
init()runs before any run, that your start command includes--import confident-trace/register, and that the process reachesshutdown()so buffered spans are flushed.runtime.getInstrumentationStatus()tells you whether the hook attached. - Duplicate spans: don't attach a second OpenAI Agents instrumentor to the same process.
- Incomplete streams: drain or cancel streamed runs before
shutdown(); otherwise spans end without their final output.
For general issues, see troubleshooting.
Disable OpenAI Agents Instrumentation
Pass init() a list of integration identifiers to opt in to only those integrations. The identifier for OpenAI Agents is "openai_agents" in Python or "openai-agents" in TypeScript; omit it to disable this integration. An empty list disables all automatic instrumentation:
from confident_trace import init
init(instrumentations=())
# Use ("openai_agents",) to opt in; omit "openai_agents" to disable it.import { init } from "confident-trace";
init({ instrumentations: [] });
// Use ["openai-agents"] to opt in; omit "openai-agents" to disable it.This turns off Confident AI's automatic instrumentation; calls made after initialization are not instrumented by this integration.
Next Steps
Online Evals
Run evaluations on traces and spans in real-time as they're ingested into Confident AI to monitor AI quality in production.
Threads
Group multi-turn agent conversations into threads, set turn I/O, and evaluate entire conversations as a single unit.
Last updated on