LLM TracingAdvanced Features

Attributes

Learn how to set attributes while tracing your LLM application

Overview

Attributes are like metadata but specific to different span types.

Setting attributes will make the tracing UI easier to navigate on Confident AI, but is by no means required. You also cannot set attributes for custom span types.

Set Attributes At Runtime

Attributes are set at runtime using update functions specific to each span type:

  • update_llm_span - update_retriever_span

These functions update the attributes for the CURRENT span of the component to which the @observe decorator is applied. For example, update_retriever_span will update the attributes for inner_function.

main.py
1from deepeval.tracing import observe, update_retriever_span
2
3@observe(type="custom")
4def outer_function():
5
6    @observe(type="retriever")
7    def inner_function():
8
9        # Here, update_retriever_span() will update the Retriever span
10        update_retriever_span(
11            embedder="text-embedding-ada-002",
12            chunk_size=10,
13        )

The current span is determined using Python’s context manager, which automatically track the active span based on the execution context. This means you don’t need to manually pass span references around - the system knows which span you’re currently executing within, even in asynchronous contexts.

LLM attributes

LLM attributes track the model, prompt, and token usage and costs of language model calls. It is highly RECOMMENDED that you set the attributes for an LLM span.

main.py
1from deepeval.tracing import observe, update_llm_span
2
3@observe(type="llm", model="gpt-4.1")
4def generate_response(input):
5    prompt = Prompt(alias="My Prompt")
6    prompt.pull("00.00.01")
7    output = "Generated response to: " + input
8    update_llm_span(
9        prompt=prompt,
10        input_token_count=10,
11        output_token_count=25,
12        cost_per_input_token=0.01,
13        cost_per_output_token=0.01,
14    )
15    return output

There are SIX optional parameters for update_llm_span:

  • [Optional] model: The model used, of type str.
  • [Optional] prompt: The prompt of type Prompt, which must be pulled prior to updating the span.
  • [Optional] input_token_count: The number of tokens of type float in the input.
  • [Optional] output_token_count: The number of tokens of type float in the generated response.
  • [Optional] cost_per_input_token: The cost per input token of type float.
  • [Optional] cost_per_output_token: The cost per output token of type float.

The model and per-token costs can be set in the @observe decorator, but will be overridden if set in update_llm_span.

Retriever attributes

Retriever attributes track the embedder, top_k, and chunk_size in RAG pipelines. It is highly RECOMMENDED that you set the attributes for a retriever span.

main.py
1from deepeval.tracing import observe, update_retriever_span
2
3@observe(type="retriever", embedder="text-embedding-ada-002")
4def retrieve_documents(query):
5    fetched_documents = ["doc1", "doc2"]
6    update_retriever_span(
7        embedder="text-embedding-ada-002",
8        chunk_size=10,
9        top_k=5,
10    )
11    return fetched_documents

There are THREE optional parameters for update_retriever_span:

  • [Optional] embedder: The name of the embedding model used of type str.
  • [Optional] chunk_size: The size of the text chunks retrieved of type int from the vector store.
  • [Optional] top_k: The number of text chunks retrieved of type int from the vector store.