For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
Trust CenterStatusSupportGet a demoPlatform
DocumentationEvals API ReferenceIntegrations & OTELPlatform SettingsSelf-HostingChangelog
DocumentationEvals API ReferenceIntegrations & OTELPlatform SettingsSelf-HostingChangelog
  • Get Started
    • Introduction
    • Setup and Installation
  • LLM Evaluation
    • Introduction
    • Experiments
  • Metrics
    • Introduction
    • Metric Collections
    • Custom Metrics
  • LLM Tracing
    • Introduction
      • Quickstart
      • Configure Span Types
      • Log Prompts
      • Track LLM Costs
      • Set Input/Output
      • Thread Traces
    • Signals
    • Troubleshooting
  • Human-in-the-Loop
    • Introduction
    • Collect Feedback
  • Reporting & Analytics
    • Dashboards
    • Executive Insights
  • Red Teaming
    • Introduction
    • Quickstart
    • Frameworks & Policies
    • Risk Profiles
    • Red Team Using DeepTeam
  • Resources
    • Why Confident AI
    • Support
    • Data Handling
    • LLM Use Cases
LogoLogo
Trust CenterStatusSupportGet a demoPlatform
On this page
  • Overview
  • Log a Prompt
  • Next Steps
LLM TracingInstrument Your App

Log Prompts

Log prompts to LLM spans for version tracking in production
Was this page helpful?
Previous

Track LLM Costs

Track the token usage and cost of your LLM calls
Next
Built with

Overview

When you use prompts managed on Confident AI, you can log the exact prompt version used in each LLM call. Prompt logging works by:

  1. Pulling a prompt from Confident AI
  2. Logging it to the LLM span via update_llm_span / updateLlmSpan

That’s it! This lets you monitor what prompts are running in production and which prompts performs best over time.

Prompt Observability & Performance

If you haven’t already, learn how prompt management works on Confident AI here.

Log a Prompt

Prompt logging is only available for LLM spans. Make sure your observed function has type="llm" set.

1

Pull and interpolate your prompt

Pull the prompt version from Confident AI and interpolate any variables.

Python
TypeScript
main.py
1from deepeval.prompt import Prompt
2
3prompt = Prompt(alias="YOUR-PROMPT-ALIAS")
4prompt.pull()
5interpolated_prompt = prompt.interpolate(name="Joe")

If you don’t have any variables, you must still call interpolate() to create a usable copy of your prompt template.

2

Use the prompt and log it to the span

Inside an observed LLM function, use the interpolated prompt for generation and log the original prompt object to the span.

Python
TypeScript
main.py
1from deepeval.tracing import observe, update_llm_span
2from deepeval.prompt import Prompt
3from openai import OpenAI
4
5@observe(type="llm", model="gpt-4o")
6def generate_response(user_input: str) -> str:
7    prompt = Prompt(alias="YOUR-PROMPT-ALIAS")
8    prompt.pull()
9    interpolated_prompt = prompt.interpolate(name="Joe")
10
11    response = OpenAI().chat.completions.create(
12        model="gpt-4o",
13        messages=interpolated_prompt,
14    )
15    update_llm_span(prompt=prompt)
16    return response.choices[0].message.content

Always pass the original pulled prompt object (not the interpolated version) to update_llm_span / updateLlmSpan. Confident AI uses it to link the span back to the versioned prompt — passing the interpolated string would log a raw string instead.

Once logged, Confident AI will display the prompt alias and version directly on the LLM span in the trace view, making it easy to see exactly which prompt was used for each LLM call.

Next Steps

With prompts logged, set up cost tracking or refine what data your traces capture.

Track LLM Costs

Track token usage and cost for your LLM spans — manually or automatically.

Set Input/Output

Override the default input and output on traces and spans for better visualization and evaluation.