Prompts & Hyperparameters

Overview

Prompts and hyperparameters are sent to your AI Connection endpoint as part of the payload, and both are logged alongside your test runs and experiments. This lets you trace every evaluation result back to the exact prompt versions and configuration values used to produce it.

Prompts

Associate prompt versions with your AI connection. When running evaluations, these prompts will be attributed to each test run, letting you trace results back to the prompts used.

The prompts variable in your payload is a dictionary where each key maps to an object containing alias and version:

1
{
2
  "system": { "alias": "system-prompt", "version": "1.0.0" },
3
  "assistant": { "alias": "assistant-prompt", "version": "2.1.0" }
4
}

Here’s an example of how your Python endpoint might handle the prompts dictionary:

1
from deepeval.prompt import Prompt
2
3
@app.post("/generate")
4
def generate(request: dict):
5
    # Pull different prompt versions using their keys
6
    system_info = request["prompts"]["system"]
7
    assistant_info = request["prompts"]["assistant"]
8
9
    system_prompt = Prompt(alias=system_info["alias"]).pull(version=system_info["version"])
10
    assistant_prompt = Prompt(alias=assistant_info["alias"]).pull(version=assistant_info["version"])
11
12
    # Use the prompts in your generation
13
    response = llm.generate(
14
        system=system_prompt.text,
15
        assistant=assistant_prompt.text,
16
        user=request["input"]
17
    )
18
19
    return {"output": response}

For more details on working with prompts, see Prompt Versioning.

Hyperparameters

Define optional hyperparameters as string key-value pairs. These are sent to your endpoint as part of the payload and are also logged in test runs and experiments, making it easy to track which configuration was used for each evaluation.

Hyperparameters are useful for passing model configuration values like temperature, model_name, or max_tokens to your AI app without hardcoding them into your endpoint. Since they’re logged alongside test run and experiment results, you can compare how different hyperparameter values affect evaluation outcomes.

The hyperparameters variable in your payload is a dictionary where both keys and values are strings:

1
{
2
  "temperature": "0.7",
3
  "model": "gpt-4o",
4
  "max_tokens": "1024"
5
}

Here’s an example of how your Python endpoint might use hyperparameters:

1
@app.post("/generate")
2
def generate(request: dict):
3
    hyperparameters = request.get("hyperparameters", {})
4
5
    response = llm.generate(
6
        model=hyperparameters.get("model", "gpt-4o"),
7
        temperature=float(hyperparameters.get("temperature", "0.7")),
8
        max_tokens=int(hyperparameters.get("max_tokens", "1024")),
9
        user=request["input"]
10
    )
11
12
    return {"output": response}

Next Steps

With prompts and hyperparameters attached, configure how Confident AI reads your endpoint’s response.