Log Prompts

Log prompts to LLM spans for version tracking in production

Overview

When you use prompts managed on Confident AI, you can log the exact prompt version used in each LLM call. Prompt logging works by:

Pulling a prompt from Confident AI
Logging it to the LLM span via update_llm_span / updateLlmSpan

That's it! This lets you monitor what prompts are running in production and which prompts performs best over time.

Log a Prompt

Prompt logging is only available for LLM spans. Make sure your observed function has type="llm" set.

Pull and interpolate your prompt

Pull the prompt version from Confident AI and interpolate any variables.

main.py

from deepeval.prompt import Prompt

prompt = Prompt(alias="YOUR-PROMPT-ALIAS")
prompt.pull()
interpolated_prompt = prompt.interpolate(name="Joe")

index.ts

import { Prompt } from "deepeval";

const prompt = new Prompt({ alias: "YOUR-PROMPT-ALIAS" });
await prompt.pull();
const interpolatedPrompt = prompt.interpolate({ name: "Joe" });

Use the prompt and log it to the span

Inside an observed LLM function, use the interpolated prompt for generation and log the original prompt object to the span.

main.py

from deepeval.tracing import observe, update_llm_span
from deepeval.prompt import Prompt
from openai import OpenAI

@observe(type="llm", model="gpt-4o")
def generate_response(user_input: str) -> str:
    prompt = Prompt(alias="YOUR-PROMPT-ALIAS")
    prompt.pull()
    interpolated_prompt = prompt.interpolate(name="Joe")

    response = OpenAI().chat.completions.create(
        model="gpt-4o",
        messages=interpolated_prompt,
    )
    update_llm_span(prompt=prompt)
    return response.choices[0].message.content

index.ts

import { observe, updateLlmSpan } from "deepeval/tracing";
import { Prompt } from "deepeval";
import OpenAI from "openai";

const generateResponse = async (userInput: string) => {
  const prompt = new Prompt({ alias: "YOUR-PROMPT-ALIAS" });
  await prompt.pull();
  const interpolatedPrompt = prompt.interpolate({ name: "Joe" });

  const openai = new OpenAI();
  const response = await openai.chat.completions.create({
    model: "gpt-4o",
    messages: interpolatedPrompt as any[],
  });
  updateLlmSpan({ prompt });
  return response.choices[0].message.content;
};

const observedGenerateResponse = observe({
  type: "llm",
  model: "gpt-4o",
  fn: generateResponse,
});

Once logged, Confident AI will display the prompt alias and version directly on the LLM span in the trace view, making it easy to see exactly which prompt was used for each LLM call.

Next Steps

With prompts logged, set up cost tracking or refine what data your traces capture.

Track LLM Costs

Track token usage and cost for your LLM spans — manually or automatically.

Set Input/Output

Override the default input and output on traces and spans for better visualization and evaluation.

Ready to monitor AI in production?Connect traces, alerts, dashboards, and evals in one production workflowBook a demo

Last updated on July 27, 2026