Pydantic AI

Overview

Pydantic AI is a Python-native LLM agent framework built on the foundations of Pydantic validation. Confident AI lets you trace and evaluate Pydantic AI agents through confident-trace, Confident AI's OpenTelemetry-native tracing SDK — call init() once and keep constructing your Agent exactly as you do today.

Auto-Instrument

1. Install Dependencies

   Run the following command to install confident-trace alongside Pydantic AI and the model SDK your agent uses (the example uses OpenAI):

   pip install confident-trace 'pydantic-ai-slim[openai]'

2. Set Your API Keys

   Get your Confident AI Project API key and set it as an environment variable, along with your model provider's key:

   export CONFIDENT_API_KEY="<your-confident-project-key>"
   export OPENAI_API_KEY="<your-openai-key>"

3. Instrument Pydantic AI

   Call init() once before running agents. Keep constructing Agent from pydantic_ai as usual — there's nothing to pass to instrument=.

   SynchronousAsynchronousStreaming

   from confident_trace import init, shutdown
   from pydantic_ai import Agent
   
   init()
   agent = Agent("openai:gpt-4.1-mini", system_prompt="Be concise, reply with one sentence.")
   
   try:
       result = agent.run_sync("What are LLMs?")
       print(result.output)
   finally:
       shutdown()

4. Run Pydantic AI

   Run your script to send the trace to Confident AI:

   python main.py

   Done ✅. Open the Observatory in your Confident AI project to inspect the trace and its agent, model, and tool spans.

What Gets Captured

Pydantic AI's native spans are exported with their original IDs, parents, events, and attributes, so what you see in the Observatory is exactly the structure Pydantic AI reports:

• Agent runs, model requests, and tool executions — in the hierarchy Pydantic AI reports.
• Model content — prompts and completions as captured by Pydantic AI, which is on by default in the framework.
• Async runs and streams — agent.run and agent.run_stream are covered with the same init() setup.

Tools and Streaming

Tools defined with Pydantic AI's decorators are captured by its native instrumentation — no Confident decorator is needed. These snippets replace the agent.run_sync call inside the quickstart's try block.

import asyncio

@agent.tool_plain
def get_weather(city: str) -> str:
    return f"{city}: sunny, 22°C"

async def main():
    async with agent.run_stream("What is the weather in Tokyo?") as stream:
        async for chunk in stream.stream_text(delta=True):
            print(chunk, end="", flush=True)

asyncio.run(main())

Set Trace Span Properties

Use a trace context to add properties you know before the call starts. It creates no extra span; the trace started by agent.run_sync() inherits the tags, metadata, user ID, and customer ID.

from confident_trace import init, trace_context
from pydantic_ai import Agent

init()

agent = Agent("openai:gpt-4.1-mini", system_prompt="Be concise.")

with trace_context(
    tags=["support"],
    metadata={"release": "2026-09"},
    user_id="user-42",
    customer_id="customer-7",
):
    result = agent.run_sync("Explain OpenTelemetry in one sentence.")

See users and customers to set an optional display name alongside each ID, and trace context for every supported trace property and update behavior.

Instrumenting Multi-Turn

You do not need turn() when one Pydantic AI entry-point call is already one conversational turn—the integration creates that turn's trace automatically. Use turn() when you want to define the boundary yourself, such as grouping two sequential Pydantic AI calls into one turn. Reuse the same thread ID on later turns to group them into one conversation.

from confident_trace import init, turn

init()

with turn("support-turn", thread_id="chat-42"):
    context = agent.run_sync("Find the relevant account details.")
    answer = agent.run_sync(f"Summarize these details: {context.output}")

See threads for thread I/O, turn IDs, and user IDs.

Troubleshooting

• No trace: make sure init() runs before the agent does, that the process reaches shutdown() so buffered spans are flushed, and that Pydantic AI's instrumentation isn't disabled on the agent (agent.instrument = False).
• Duplicate model spans: you have a second provider instrumentor attached. Pass init(instrumentations=()) when external instrumentation already covers provider calls.
• Missing content: native span content is controlled by Pydantic AI's InstrumentationSettings, not capture_content.
• Spans go elsewhere: if Pydantic AI was configured with its own tracer provider, pass that same provider to init(tracer_provider=...) and register it globally. Register the same provider globally before initialization.

For general issues, see troubleshooting.

Disable Pydantic AI Instrumentation

Pass init() a list of integration identifiers to opt in to only those integrations. The identifier for Pydantic AI is "pydantic_ai" in Python; omit it to disable this integration. An empty list disables all automatic instrumentation:

from confident_trace import init
init(instrumentations=())
# Use ("pydantic_ai",) to opt in; omit "pydantic_ai" to disable it.

This turns off Confident AI's automatic instrumentation; calls made after initialization are not instrumented by this integration.

Next Steps

Last updated on October 23, 2026