Introducing Confident Voice: Watch the Launch Video Now

Thread Traces

Group your traces as threads to evaluate an entire conversation workflow

Overview

A thread is a group of traces that share a thread ID. It lets you view and evaluate a whole conversation as one unit, whether it's a chatbot, a multi-turn agent or a voice agent.

Most apps create one trace per turn. Traces with the same thread ID are then shown in order, as the turns of the conversation.

Create a Thread

The simplest way to create a thread is to wrap each turn of your app in turn(). It starts a new trace for that turn and gives it the thread ID you pass in. Traces with the same thread ID form one thread.

main.py
from openai import OpenAI
from confident_trace import init, turn, update_trace, shutdown

init()
client = OpenAI()

def llm_app(query: str, thread_id: str):
    with turn("llm_app", thread_id=thread_id):
        res = client.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": query}]
        ).choices[0].message.content
        update_trace(input=query, output=res)
        return res

try:
    llm_app("What's the weather in SF?", thread_id="your-thread-id")
    llm_app("What about tomorrow?", thread_id="your-thread-id")
finally:
    shutdown()

A few things to know about turn():

  • The thread ID can be any string, typically a session or conversation ID from your app.
  • It can identify the user and customer too, with user_id / userId and customer_id / customerId.
  • It works with sync and async code.

Add a Thread ID to an Existing Trace

Setting a thread ID on an existing trace only labels that trace. Unlike turn(), it doesn't start a new one.

Use this only when your app already starts a separate trace for every request. Otherwise, use turn() so each turn gets its own trace.

main.py
from openai import OpenAI
from confident_trace import span, update_trace

client = OpenAI()

def llm_app(query: str):
    with span("llm_app", type="agent"):
        res = client.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": query}]
        ).choices[0].message.content
        update_trace(thread_id="your-thread-id", input=query, output=res)
        return res

If your app is fully auto-instrumented and already starts a trace for every request, a trace context can add the thread ID without a wrapper span. It can set the user and customer the same way:

main.py
from confident_trace import trace_context

def llm_app(query: str, thread_id: str):
    with trace_context(thread_id=thread_id):
        return client.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": query}]
        ).choices[0].message.content

A trace context doesn't create a trace or a span. It only fills in defaults for the traces started inside it.

Use turn() instead when you need a new trace for each turn, or want to set thread I/O yourself. For how trace contexts behave in full, see Update Trace Properties.

Voice Calls

Put your voice agent's calls in threads to review and evaluate them call by call:

  • Voice timing. With LiveKit or Pipecat, each turn shows how long the agent took to answer and whether the caller interrupted it.
  • Audio on every turn. Each turn plays its own audio, from its message or its part of the call recording. See Audio in Traces.
  • Voice metrics. Thread evals score the audio of the call, not just its transcript. See Evaluate Threads.

With LiveKit or Pipecat, set the thread ID with a trace context around the call, as in LiveKit or Pipecat.

Customize Threads

Set what each trace in a thread carries, so the conversation reads and evaluates the way it happened.

Set Thread I/O

For each trace, set input to the raw user text and output to the LLM's reply. Confident AI shows them as the turns of the conversation, and thread evaluations read them the same way.

main.py
from openai import OpenAI
from confident_trace import turn, update_trace

client = OpenAI()

def llm_app(query: str):
    with turn("llm_app", thread_id="your-thread-id"):
        messages = [{"role": "user", "content": query}]
        res = client.chat.completions.create(
            model="gpt-4o",
            messages=messages
        ).choices[0].message.content

        # ✅ Do this — query is the raw user input
        update_trace(input=query, output=res)

        # ❌ Don't do this — messages is not the raw user input
        # update_trace(input=messages, output=res)
        return res

You don't have to set both on every trace. If a turn only has a user message or only an LLM reply, set just that one, and Confident AI shows and evaluates the turn accordingly.

example.py
# ✅ Set only input (e.g. user message with no immediate LLM response)
update_trace(thread_id="your-thread-id", input=query)

# ✅ Set only output (e.g. proactive LLM message with no user input)
update_trace(thread_id="your-thread-id", output=res)

# ✅ Omit both (e.g. background processing step in the conversation)
update_trace(thread_id="your-thread-id")

Set Thread Fields

Tags and metadata on a thread label the conversation as a whole, such as its agent version, client or status. You can filter and group by them across the Observatory.

They're separate from the tags and metadata on each trace. To set them, pass a thread object when starting a turn, with the ID, tags (a list of strings) and metadata (any JSON-serializable values):

main.py
from confident_trace import turn

with turn(thread={
    "id": "chat-42",
    "tags": ["support"],
    "metadata": {"channel": "web"},
}):
    agent.invoke(...)

You can identify the thread in either of two ways — use one form or the other:

  • Pass thread_id / threadId when you only need the ID.
  • Pass a thread object when you also want to set thread tags or metadata. The object can contain id, tags, and metadata. turn() still requires an ID in the object.

The same two options are available with a trace context and the trace update helper.

Set Tools Called

If your app calls tools, log which ones each turn used. They're attached to the trace next to the output they helped produce, and each tool is an object with at least a name.

main.py
from confident_trace import turn, update_trace

def llm_app(query: str):
    with turn("llm_app", thread_id="your-thread-id"):
        res, tools = call_agent(query)
        update_trace(
            input=query,
            output=res,
            tools_called=[{"name": "WebSearch"}, {"name": "Calculator"}],
        )
        return res

Set Retrieval Context

For RAG apps, log the retrieval context behind each reply, so Confident AI can evaluate retrieval quality across the conversation.

main.py
from confident_trace import turn, update_trace

def llm_app(query: str):
    with turn("llm_app", thread_id="your-thread-id"):
        chunks = retrieve(query)
        res = generate(query, chunks)
        update_trace(
            input=query,
            output=res,
            retrieval_context=[chunk.text for chunk in chunks],
        )
        return res

Next Steps

With threads set up, evaluate conversation quality or add more context to your traces.

Ready to monitor AI in production?Connect traces, alerts, dashboards, and evals in one production workflowBook a demo

Last updated on

Built byConfident AI