Thread Traces
Group your traces as threads to evaluate an entire conversation workflow
Overview
A thread is a group of traces that share a thread ID. It lets you view and evaluate a whole conversation as one unit, whether it's a chatbot, a multi-turn agent or a voice agent.
Most apps create one trace per turn. Traces with the same thread ID are then shown in order, as the turns of the conversation.
Create a Thread
The simplest way to create a thread is to wrap each turn of your app in turn(). It starts a new trace for that turn and gives it the thread ID you pass in. Traces with the same thread ID form one thread.
from openai import OpenAI
from confident_trace import init, turn, update_trace, shutdown
init()
client = OpenAI()
def llm_app(query: str, thread_id: str):
with turn("llm_app", thread_id=thread_id):
res = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": query}]
).choices[0].message.content
update_trace(input=query, output=res)
return res
try:
llm_app("What's the weather in SF?", thread_id="your-thread-id")
llm_app("What about tomorrow?", thread_id="your-thread-id")
finally:
shutdown()import OpenAI from "openai";
import { init, turn, updateTrace } from "confident-trace";
const runtime = init();
const openai = new OpenAI();
const llmApp = async (query: string, threadId: string) => {
return turn({ threadId }, async () => {
const res = await openai.chat.completions.create({
model: "gpt-4o",
messages: [{ role: "user", content: query }],
});
const data = res.choices[0].message.content;
updateTrace({ input: query, output: data });
return data;
});
};
try {
await llmApp("What's the weather in SF?", "your-thread-id");
await llmApp("What about tomorrow?", "your-thread-id");
} finally {
await runtime.shutdown();
}Remember to run your entry point with the Node preload so auto-instrumented spans (like the OpenAI call above) nest inside the turn.
A few things to know about turn():
- The thread ID can be any string, typically a session or conversation ID from your app.
- It can identify the user and customer too, with
user_id/userIdandcustomer_id/customerId. - It works with sync and async code.
Add a Thread ID to an Existing Trace
Setting a thread ID on an existing trace only labels that trace. Unlike turn(), it doesn't start a new one.
Use this only when your app already starts a separate trace for every request. Otherwise, use turn() so each turn gets its own trace.
from openai import OpenAI
from confident_trace import span, update_trace
client = OpenAI()
def llm_app(query: str):
with span("llm_app", type="agent"):
res = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": query}]
).choices[0].message.content
update_trace(thread_id="your-thread-id", input=query, output=res)
return resimport OpenAI from "openai";
import { withSpan, updateTrace } from "confident-trace";
const openai = new OpenAI();
const llmApp = async (query: string) => {
return withSpan({ name: "llm_app", type: "agent" }, async () => {
const res = await openai.chat.completions.create({
model: "gpt-4o",
messages: [{ role: "user", content: query }],
});
const data = res.choices[0].message.content;
updateTrace({ threadId: "your-thread-id", input: query, output: data });
return data;
});
};If your app is fully auto-instrumented and already starts a trace for every request, a trace context can add the thread ID without a wrapper span. It can set the user and customer the same way:
from confident_trace import trace_context
def llm_app(query: str, thread_id: str):
with trace_context(thread_id=thread_id):
return client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": query}]
).choices[0].message.contentimport { traceContext } from "confident-trace";
const llmApp = async (query: string, threadId: string) => {
const res = await traceContext({ threadId }, () =>
openai.chat.completions.create({
model: "gpt-4o",
messages: [{ role: "user", content: query }],
}),
);
return res.choices[0].message.content;
};A trace context doesn't create a trace or a span. It only fills in defaults for the traces started inside it.
Use turn() instead when you need a new trace for each turn, or want to set thread I/O yourself. For how trace contexts behave in full, see Update Trace Properties.
Voice Calls
Put your voice agent's calls in threads to review and evaluate them call by call:
- Voice timing. With LiveKit or Pipecat, each turn shows how long the agent took to answer and whether the caller interrupted it.
- Audio on every turn. Each turn plays its own audio, from its message or its part of the call recording. See Audio in Traces.
- Voice metrics. Thread evals score the audio of the call, not just its transcript. See Evaluate Threads.
With LiveKit or Pipecat, set the thread ID with a trace context around the call, as in LiveKit or Pipecat.
Customize Threads
Set what each trace in a thread carries, so the conversation reads and evaluates the way it happened.
Set Thread I/O
For each trace, set input to the raw user text and output to the LLM's reply. Confident AI shows them as the turns of the conversation, and thread evaluations read them the same way.
from openai import OpenAI
from confident_trace import turn, update_trace
client = OpenAI()
def llm_app(query: str):
with turn("llm_app", thread_id="your-thread-id"):
messages = [{"role": "user", "content": query}]
res = client.chat.completions.create(
model="gpt-4o",
messages=messages
).choices[0].message.content
# ✅ Do this — query is the raw user input
update_trace(input=query, output=res)
# ❌ Don't do this — messages is not the raw user input
# update_trace(input=messages, output=res)
return resimport OpenAI from "openai";
import { turn, updateTrace } from "confident-trace";
const openai = new OpenAI();
const llmApp = async (query: string) => {
return turn({ threadId: "your-thread-id" }, async () => {
const messages = [{ role: "user" as const, content: query }];
const res = await openai.chat.completions.create({
model: "gpt-4o",
messages,
});
const data = res.choices[0].message.content;
// ✅ Do this — query is the raw user input
updateTrace({ input: query, output: data });
// ❌ Don't do this — messages is not the raw user input
// updateTrace({ input: messages, output: data });
return data;
});
};You don't have to set both on every trace. If a turn only has a user message or only an LLM reply, set just that one, and Confident AI shows and evaluates the turn accordingly.
# ✅ Set only input (e.g. user message with no immediate LLM response)
update_trace(thread_id="your-thread-id", input=query)
# ✅ Set only output (e.g. proactive LLM message with no user input)
update_trace(thread_id="your-thread-id", output=res)
# ✅ Omit both (e.g. background processing step in the conversation)
update_trace(thread_id="your-thread-id")// ✅ Set only input
updateTrace({ threadId: "your-thread-id", input: query });
// ✅ Set only output
updateTrace({ threadId: "your-thread-id", output: data });
// ✅ Omit both
updateTrace({ threadId: "your-thread-id" });Set Thread Fields
Tags and metadata on a thread label the conversation as a whole, such as its agent version, client or status. You can filter and group by them across the Observatory.
They're separate from the tags and metadata on each trace. To set them, pass a thread object when starting a turn, with the ID, tags (a list of strings) and metadata (any JSON-serializable values):
from confident_trace import turn
with turn(thread={
"id": "chat-42",
"tags": ["support"],
"metadata": {"channel": "web"},
}):
agent.invoke(...)import { turn } from "confident-trace";
await turn({
thread: {
id: "chat-42",
tags: ["support"],
metadata: { channel: "web" },
},
}, async () => {
await agent.invoke(...);
});You can identify the thread in either of two ways — use one form or the other:
- Pass
thread_id/threadIdwhen you only need the ID. - Pass a
threadobject when you also want to set thread tags or metadata. The object can containid,tags, andmetadata.turn()still requires an ID in the object.
The same two options are available with a trace context and the trace update helper.
Set Tools Called
If your app calls tools, log which ones each turn used. They're attached to the trace next to the output they helped produce, and each tool is an object with at least a name.
from confident_trace import turn, update_trace
def llm_app(query: str):
with turn("llm_app", thread_id="your-thread-id"):
res, tools = call_agent(query)
update_trace(
input=query,
output=res,
tools_called=[{"name": "WebSearch"}, {"name": "Calculator"}],
)
return resimport { turn, updateTrace } from "confident-trace";
const llmApp = async (query: string) => {
return turn({ threadId: "your-thread-id" }, async () => {
const { res, tools } = await callAgent(query);
updateTrace({
input: query,
output: res,
toolsCalled: [{ name: "WebSearch" }, { name: "Calculator" }],
});
return res;
});
};Set Retrieval Context
For RAG apps, log the retrieval context behind each reply, so Confident AI can evaluate retrieval quality across the conversation.
from confident_trace import turn, update_trace
def llm_app(query: str):
with turn("llm_app", thread_id="your-thread-id"):
chunks = retrieve(query)
res = generate(query, chunks)
update_trace(
input=query,
output=res,
retrieval_context=[chunk.text for chunk in chunks],
)
return resimport { turn, updateTrace } from "confident-trace";
const llmApp = async (query: string) => {
return turn({ threadId: "your-thread-id" }, async () => {
const chunks = await retrieve(query);
const res = await generate(query, chunks);
updateTrace({
input: query,
output: res,
retrievalContext: chunks.map((c) => c.text),
});
return res;
});
};Next Steps
With threads set up, evaluate conversation quality or add more context to your traces.
Evaluate Threads
Run online evaluations on entire conversation threads to monitor multi-turn quality.
Customize Traces
Add tags, metadata, user info, and customer info to your traces.
Last updated on