Deep Agents
Use Confident AI for LLM observability and evals for Deep Agents
Overview
Deep Agents is LangChain's agent framework for complex tasks, with built-in filesystem tools and subagent delegation. Confident AI traces your Deep Agents application automatically through confident-trace — call init() once to inspect agent runs, model calls, and tools in the Observatory.
The integration captures the following spans from your Deep Agents application:
- Agent execution — graph invocations, nodes, and intermediate runnables, with their parent-child relationships
- Subagent delegation —
tasktool calls and the nested subagent's model and tool execution - LLM spans — model details, token usage, finish reasons, and input/output messages, including requested tool calls
- Tool spans — built-in filesystem tools and custom tools, with their input parameters and output
| Runtime | Requirements | Setup |
|---|---|---|
| Python | Python 3.11+, deepagents 0.7.x and your model integration | Call init() before invoking your agent |
| TypeScript | Not validated by this integration | — |
Auto-Instrument
Install Dependencies
Install
confident-tracealongside Deep Agents and the model integration your agent uses. This example uses OpenAI:pip install confident-trace 'deepagents>=0.7.13,<0.8' 'langchain-openai>=1,<2'Set Your API Keys
Get your Confident AI Project API key and set it as an environment variable, along with your model provider's key:
export CONFIDENT_API_KEY="<your-confident-project-key>" export OPENAI_API_KEY="<your-openai-key>"Instrument Deep Agents
Call
init()once before running the agent. It detects the installed LangChain and LangGraph packages and attaches the shared tracing bridge automatically.main.py from confident_trace import init, shutdown from deepagents import create_deep_agent from langchain_openai import ChatOpenAI init() def get_weather(city: str) -> str: """Return example weather for a city.""" return f"It's sunny in {city}." agent = create_deep_agent( name="weather-assistant", model=ChatOpenAI(model="gpt-4.1-mini"), tools=[get_weather], system_prompt="Use get_weather to answer weather questions.", ) try: result = agent.invoke( {"messages": [{"role": "user", "content": "What is the weather in San Francisco?"}]} ) print(result["messages"][-1].content) finally: shutdown()Run Deep Agents
Run your script to send the trace to Confident AI:
python main.pyDone ✅. Open the Observatory in your Confident AI project to inspect the trace and its graph, model, and tool spans.
Trace Subagents
Deep Agents delegates work through the task tool. The tracing bridge follows that delegation automatically, including parallel subagents and the model and tool calls inside each one.
To add a subagent to the quickstart, replace the agent = create_deep_agent(...) block with the following. Keep the existing init(), invocation, and shutdown code:
model = ChatOpenAI(model="gpt-4.1-mini")
agent = create_deep_agent(
name="weather-coordinator",
model=model,
system_prompt="Delegate weather questions to weather-researcher, then summarize its answer.",
subagents=[
{
"name": "weather-researcher",
"description": "Look up weather for the requested city.",
"system_prompt": "Use get_weather to answer the question.",
"model": model,
"tools": [get_weather],
}
],
)When the coordinator delegates, its trace includes the task tool span, the nested weather-researcher graph, and that subagent's model and get_weather tool spans. The exact nodes and number of model calls depend on the agent's execution.
What Gets Captured
- Graphs and nodes — each agent invocation, nested subagent graph, node, and intermediate runnable reported by LangGraph callbacks.
- Model calls — LLM spans with available model information, token usage, finish reasons, and normalized messages.
- Tool executions — tool spans for delegation, filesystem operations such as
write_file, and your own tools. - Planning tools —
write_todoscalls when your agent hasTodoListMiddlewareenabled. - Custom spans — application spans created inside a node or tool inherit that execution's OpenTelemetry context.
Inputs, outputs, and messages follow the content policy. These traces capture SDK execution; commands running inside a separate sandbox process need their own instrumentation for internal spans.
Streaming and Async Runs
The same setup traces invoke, ainvoke, stream, and astream. For example, replace the invocation inside the quickstart's try block with a streamed call:
for state in agent.stream(
{"messages": [{"role": "user", "content": "What is the weather in San Francisco?"}]},
stream_mode="values",
):
print(state["messages"][-1].content)Consume streams fully, or close them when stopping early, before calling shutdown(). See flush and shutdown.
Set Trace Span Properties
Use a trace context to add properties you know before the call starts. It creates no extra span; the trace started by agent.invoke() inherits the tags, metadata, and user ID.
from confident_trace import trace_context
with trace_context(
tags=["weather"],
metadata={"release": "2026-09"},
user_id="user-42",
):
result = agent.invoke(
{"messages": [{"role": "user", "content": "What is the weather in San Francisco?"}]}
)Use this in your initialized application, before shutdown. See trace context for all supported properties.
Instrumenting Multi-Turn
Create your agent with a checkpointer and reuse configurable.thread_id across invocations to retain conversation state. The Python integration reads this thread ID and associates each invocation's trace with the same conversation.
from langgraph.checkpoint.memory import InMemorySaver
agent = create_deep_agent(
model=ChatOpenAI(model="gpt-4.1-mini"),
tools=[get_weather],
system_prompt="Use get_weather to answer weather questions.",
checkpointer=InMemorySaver(),
)
config = {"configurable": {"thread_id": "weather-chat-42"}}
for prompt in ["What is the weather in San Francisco?", "And in New York?"]:
result = agent.invoke(
{"messages": [{"role": "user", "content": prompt}]},
config,
)
print(result["messages"][-1].content)Use this agent construction and invocation loop in place of the corresponding quickstart code, keeping initialization and shutdown. InMemorySaver keeps state for the current process; use a persistent checkpointer when state must survive restarts.
Human approval interrupts are normal control flow. Resuming with Command(resume=...) creates a new invocation trace and retains the conversation ID. See LangGraph checkpoint and resume behavior and threads.
Disable Deep Agents Instrumentation
Pass init() a list of integration identifiers to enable only those integrations. "deepagents", "langgraph", and "langchain" all enable the same callback bridge, so omit all three to disable framework tracing. An empty tuple disables all automatic instrumentation:
from confident_trace import init
init(instrumentations=())
# Use ("deepagents",) to enable only the shared framework bridge.If you keep a model provider integration enabled, it can still capture direct provider calls independently of the framework bridge.
Next Steps
Now that your agent is traced, dive deeper into:
Online Evals
Run evaluations on traces and spans as they're ingested into Confident AI to monitor your agent's quality.
Threads
Group agent runs from the same conversation into a thread and evaluate the whole conversation as one unit.
Last updated on