Working with Prompts

Overview

You can pull a prompt version from Confident AI like how you would pull a dataset. It works by:

• Providing Confident AI with the alias and optionally version of the prompt you wish to retrieve
• Confident AI will provide the non-interpolated version of the prompt
• You will then interpolate the variables in code

You should pull prompts once and save it in memory instead of pulling it everytime you need to use it.

Using Prompt Versions

1
from deepeval.prompt import Prompt
2
3
prompt = Prompt(alias="YOUR-PROMPT-ALIAS")
4
prompt.pull()

1
{
2
  "role": "system",
3
  "content": "You are a helpful assistant called {{ name }}. Speak normally like a human."
4
}

1
{
2
  "role": "system",
3
  "content": "You are a helpful assistant called Joe. Speak normally like a human."
4
}

1
interpolated_prompt = prompt.interpolate(name="Joe")

1
{
2
  "role": "system",
3
  "content": "You are a helpful assistant called Joe. Speak normally like a human."
4
}

1
from deepeval.prompt import Prompt
2
from openai import OpenAI
3
4
prompt = Prompt(alias="YOUR-PROMPT-ALIAS")
5
prompt.pull()
6
interpolated_prompt = prompt.interpolate() # interpolate prompt
7
8
response = OpenAI().chat.completions.create(
9
    model="gpt-4o-mini",
10
    messages=interpolated_prompt
11
)
12
13
print(response.choices[0].message.content)

Pull Prompts By Label

Previously we saw how we can pull a prompt by supplying the version number. You can also “deploy” a prompt using a label, which allows you to select a specific version without defaulting to the latest one:

1
from deepeval.prompt import Prompt
2
3
prompt = Prompt(alias="YOUR-PROMPT-ALIAS")
4
prompt.pull(label="staging")

You must manually label each prompt version. Click here to learn how to do so.

Logging Prompts in Traces

1
from deepeval.tracing import observe
2
3
@observe(type="llm", model="gpt-4.1")
4
def your_llm_component():
5
    ...

1
from deepeval.tracing import observe
2
from deepeval.prompt import Prompt
3
from openai import OpenAI
4
5
@observe(type="llm", model="gpt-4.1")
6
def your_llm_component():
7
    prompt = Prompt(alias="YOUR-PROMPT-ALIAS")
8
    prompt.pull()
9
    interpolated_prompt = prompt.interpolate(name="Joe")
10
    response = OpenAI().chat.completions.create(model="gpt-4o-mini", messages=interpolated_prompt)
11
    return response.choices[0].message.content

1
from deepeval.tracing import observe, update_llm_span
2
from deepeval.prompt import Prompt
3
from openai import OpenAI
4
5
@observe(type="llm", model="gpt-4.1")
6
def your_llm_component():
7
    prompt = Prompt(alias="YOUR-PROMPT-ALIAS")
8
    prompt.pull()
9
    interpolated_prompt = prompt.interpolate(name="Joe")
10
    response = OpenAI().chat.completions.create(model="gpt-4o-mini", messages=interpolated_prompt)
11
    update_llm_span(prompt=prompt)
12
    return response.choices[0].message.content

Prompt caching

Confident AI automatically caches prompts on the client side to minimize API call latency and ensure prompt availability, which is especially useful in production environments.

Customize refresh rate

By default, the cache is refetched every 60 seconds, where DeepEval will automatically update the cached prompt with the up-to-date version from Confident AI. This can be overridden by setting the refresh parameter to a different value. Fetching is done asynchronously, so it will not block your application.

1
from deepeval.prompt import Prompt
2
3
prompt = Prompt(alias="YOUR-PROMPT-ALIAS")
4
prompt.pull(refresh=60)
5
interpolated_prompt = prompt.interpolate(name="Joe")

Disable Caching

To disable caching, you can set refresh=0. This will force an API call every time you pull the prompt, which is particularly useful for development and testing.

1
from deepeval.prompt import Prompt
2
3
prompt = Prompt(alias="YOUR-PROMPT-ALIAS")
4
prompt.pull(refresh=0)
5
interpolated_prompt = prompt.interpolate(name="Joe")