Launch Week 02 wrapped — explore all five launches

OpenAI Agents

Use Confident AI for LLM observability and evals for OpenAI Agents

Overview

OpenAI Agents is a lightweight framework for creating agentic workflows using agent swarms, handoffs, and tool use. Confident AI lets you trace and evaluate OpenAI Agents workflows with one line of code — call init() from confident-trace, Confident AI's OpenTelemetry-native tracing SDK, and your agents, tools, handoffs, and guardrails stay exactly as they are.

RuntimeRequirementsSetup
PythonPython 3.10+, openai-agents, confident-trace[openai-agents] extraCall init() before agent runs
TypeScriptNode.js 22+, @openai/agents >=0.17.0 <0.18Call init() and launch your entry point with the preload

Auto-Instrument

  1. Install Dependencies

    Run the following command to install confident-trace alongside the OpenAI Agents SDK:

    pip install 'confident-trace[openai-agents]' openai-agents
  2. Set Your API Keys

    Get your Confident AI Project API key and set it as an environment variable, along with your OpenAI key:

    export CONFIDENT_API_KEY="<your-confident-project-key>"
    export OPENAI_API_KEY="<your-openai-key>"
  3. Instrument OpenAI Agents

    Call init() once before running agents. It detects the Agents SDK automatically and hooks its tracing — there's no trace processor to register in your code.

    main.py
    from agents import Agent, Runner
    from confident_trace import init, shutdown
    
    init()
    agent = Agent(name="Assistant", instructions="You are a helpful assistant")
    
    try:
        result = Runner.run_sync(agent, "Write a haiku about recursion in programming.")
        print(result.final_output)
    finally:
        shutdown()
  4. Run OpenAI Agents

    Run your script to send the trace to Confident AI:

    python main.py

    Done ✅. Open the Observatory in your Confident AI project to inspect the trace and its workflow, agent, and model spans.

What Gets Captured

The integration converts the Agents SDK's native tracing objects into spans and preserves their parentage, so the trace tree in the Observatory matches the run:

Span typeCaptured data
WorkflowThe root of each Runner execution
AgentOne agent span per agent that participates in the run, including after handoffs
Model (LLM)One LLM span per model request, with messages and token usage
Function toolOne tool span per tool execution, with input parameters and output
Handoff, guardrail, turn, customThe remaining SDK span kinds, kept in their original position in the tree

Regular and streamed Runner executions are supported, including runs that hand off between agents or trigger guardrails.

The Python bridge is an OpenInference instrumentor, so spans are forwarded with their original OpenInference attributes rather than rewritten — see the OpenInference page for how those spans are exported. A few consequences worth knowing:

  • Content policy — Confident's content limits and redaction don't apply to these spans. Configure capture with OpenInference's TraceConfig or environment settings before calling init() if you need to.
  • SDK processors — the Agents SDK's own processors, including its default exporter, stay installed. RunConfig(tracing_disabled=True) still turns framework spans off.
  • Provider spans — direct openai client calls outside a run, and provider calls inside tools, get their own Confident LLM spans.

Set Trace Span Properties

Use a trace context to add properties you know before the call starts. It creates no extra span; the trace started by Runner.run_sync() / run() inherits the tags, metadata, and user ID.

main.py
from confident_trace import init, trace_context
from agents import Agent, Runner

init()

agent = Agent(name="Assistant", instructions="Be concise.")

with trace_context(
    tags=["support"],
    metadata={"release": "2026-09"},
    user_id="user-42",
):
    result = Runner.run_sync(agent, "Explain OpenTelemetry in one sentence.")

See trace context for every supported trace property and update behavior.

Instrumenting Multi-Turn

You do not need turn() when one OpenAI Agents entry-point call is already one conversational turn—the integration creates that turn's trace automatically. Use turn() when you want to define the boundary yourself, such as grouping two sequential OpenAI Agents calls into one turn. Reuse the same thread ID on later turns to group them into one conversation.

main.py
from confident_trace import init, turn

init()

with turn("support-turn", thread_id="chat-42"):
    context = Runner.run_sync(agent, "Find the relevant account details.")
    answer = Runner.run_sync(agent, f"Summarize these details: {context.final_output}")

See threads for thread I/O, turn IDs, and user IDs.

Troubleshooting

  • No trace: make sure init() runs before any run, and that the process reaches shutdown() so buffered spans are flushed.
  • Model calls traced but no agent, tool, or handoff spans: install the extra with pip install 'confident-trace[openai-agents]', and confirm tracing_disabled isn't set on the run.
  • Duplicate spans: don't attach a second OpenAI Agents instrumentor to the same process.
  • Incomplete streams: drain or cancel streamed runs before shutdown(); otherwise spans end without their final output.
  • Spans go to the wrong exporter: an instrumentor configured with its own tracer provider before init() keeps sending there. Use the global provider, or pass the same one to init(tracer_provider=...). See existing OpenTelemetry provider.

For general issues, see troubleshooting.

Disable OpenAI Agents Instrumentation

Pass init() a list of integration identifiers to opt in to only those integrations. The identifier for OpenAI Agents is "openai_agents" in Python or "openai-agents" in TypeScript; omit it to disable this integration. An empty list disables all automatic instrumentation:

main.py
from confident_trace import init
init(instrumentations=())
# Use ("openai_agents",) to opt in; omit "openai_agents" to disable it.

This turns off Confident AI's automatic instrumentation; calls made after initialization are not instrumented by this integration.

Next Steps

Need help instrumenting your application?Connect your model calls and agent workflows to Confident AITalk to an expert

Last updated on

Built byConfident AI