Thread Traces

Overview

A “thread” on Confident AI is a group of one or more traces linked by a shared thread ID. This is useful for building conversational AI apps — chatbots, multi-turn agents, etc. — where you want to view and evaluate an entire conversation as a single unit.

Each call to your app creates a trace, and traces with the same thread ID are grouped together chronologically as turns in a conversation.

Create a Thread

To create a thread, set a thread_id on your traces using update_current_trace / updateCurrentTrace. Any traces that share the same thread ID will be grouped into a single thread.

1
from deepeval.tracing import observe, update_current_trace
2
from openai import OpenAI
3
4
client = OpenAI()
5
6
@observe()
7
def llm_app(query: str):
8
    res = client.chat.completions.create(
9
        model="gpt-4o",
10
        messages=[{"role": "user", "content": query}]
11
    ).choices[0].message.content
12
13
    update_current_trace(thread_id="your-thread-id", input=query, output=res)
14
    return res
15
16
llm_app("What's the weather in SF?")
17
llm_app("What about tomorrow?")

The thread_id / threadId can be any string — typically a session ID or conversation ID from your app.

Set Thread I/O

Although not strictly enforced, you should set the input to the raw user text and the output to the generated LLM text for each trace. These are used as the conversation turns for display on Confident AI and for thread evaluations.

1
from deepeval.tracing import observe, update_current_trace
2
from openai import OpenAI
3
4
client = OpenAI()
5
6
@observe()
7
def llm_app(query: str):
8
    messages = {"role": "user", "content": query}
9
    res = client.chat.completions.create(
10
        model="gpt-4o",
11
        messages=messages
12
    ).choices[0].message.content
13
14
    # ✅ Do this — query is the raw user input
15
    update_current_trace(thread_id="your-thread-id", input=query, output=res)
16
17
    # ❌ Don't do this — messages is not the raw user input
18
    # update_current_trace(thread_id="your-thread-id", input=messages, output=res)
19
    return res

You don’t have to set both input and output on every trace. If a turn only has a user input or only an LLM output, you can set just one. Confident AI will format the turns accordingly on the UI and for evals.

1
# ✅ Set only input (e.g. user message with no immediate LLM response)
2
update_current_trace(thread_id="your-thread-id", input=query)
3
4
# ✅ Set only output (e.g. proactive LLM message with no user input)
5
update_current_trace(thread_id="your-thread-id", output=res)
6
7
# ✅ Omit both (e.g. background processing step in the conversation)
8
update_current_trace(thread_id="your-thread-id")

Set Thread Fields

You can attach custom metadata and tags to a thread to label production conversations with attributes like DVA version, client, agent ID, or status flags. Both are filterable and groupable across the observatory, which makes it easy to slice production traffic.

Thread fields are set by including a thread object on any trace you ingest into the thread. thread.id is an alternate, idiomatic way to specify the thread — it’s equivalent to top-level threadId and either one is sufficient. Metadata values can be any JSON-serializable type (stringified server-side), and tags are an array of strings.

1
{
2
  "uuid": "<TRACE-UUID>",
3
  "input": "What's the weather in SF?",
4
  "output": "It's 65°F and sunny.",
5
  "startTime": "2025-01-15T10:30:00Z",
6
  "endTime": "2025-01-15T10:30:05Z",
7
  "thread": {
8
    "id": "your-thread-id",
9
    "metadata": {
10
      "dvaVersion": "1.4.2",
11
      "client": "acme-corp",
12
      "agentId": "support-agent"
13
    },
14
    "tags": ["vip", "billing"]
15
  }
16
}

Successive ingestions for the same thread merge metadata keys, so you can build up a thread’s metadata incrementally across turns. Sending the same key again overwrites the previous value. Tags replace any previously stored value, so always send the full set you want on the thread:

1
{
2
  "uuid": "<NEXT-TRACE-UUID>",
3
  "threadId": "your-thread-id",
4
  "startTime": "2025-01-15T10:31:00Z",
5
  "endTime": "2025-01-15T10:31:05Z",
6
  "thread": {
7
    "metadata": {
8
      "agentId": "support-agent-v2",
9
      "escalated": true
10
    }
11
  }
12
}

After the two requests above, the thread’s metadata is:

1
{
2
  "dvaVersion": "1.4.2",
3
  "client": "acme-corp",
4
  "agentId": "support-agent-v2",
5
  "escalated": true
6
}

Set Tools Called

If your LLM app uses tool/function calling, you can log which tools were invoked for a given turn. This is attached to the trace alongside the output it helped generate.

1
from deepeval.tracing import observe, update_current_trace
2
from deepeval.test_case import ToolCall
3
4
@observe()
5
def llm_app(query: str):
6
    res, tools = call_agent(query)
7
    update_current_trace(
8
        thread_id="your-thread-id",
9
        input=query,
10
        output=res,
11
        tools_called=[ToolCall(name="WebSearch"), ToolCall(name="Calculator")]
12
    )
13
    return res

Set Retrieval Context

For RAG-based conversational apps, you can log the retrieval context used to generate a response. This enables Confident AI to evaluate retrieval quality across conversation turns.

1
from deepeval.tracing import observe, update_current_trace
2
3
@observe()
4
def llm_app(query: str):
5
    chunks = retrieve(query)
6
    res = generate(query, chunks)
7
    update_current_trace(
8
        thread_id="your-thread-id",
9
        input=query,
10
        output=res,
11
        retrieval_context=[chunk.text for chunk in chunks]
12
    )
13
    return res

Next Steps

With threads set up, evaluate conversation quality or add more context to your traces.