# Version Prompts Create and manage different versions of your prompts ## Overview Prompt versioning allows you to optimize and test different versions of your prompts. Managing prompts on Confident AI allows you to: 1. Collaborate and centralize where prompts are stored and edited, even for non-technical team members 2. Pinpoint which version, or even combination of your prompt versions, performed best 3. Optionally co-locate model settings, output type, and tools with prompts to version them as a single unit There are a million places you can keep your prompts - on GitHub, CSV files, in memory in code, Google Sheets, Notion, or even written in a diary hidden under your table drawer. But only by keeping prompts on Confident AI can you fully leverage Confident AI's evaluation features. > Prompts are a type of hyperparameter on Confident AI. Others include things > like models, embedders, top-K, and max tokens. When you run evals against > prompts kept on Confident AI, we can tell you which version performs best, and > later automatically optimize it for you. > **Prompts vs Prompts + Model Config:** You can use Confident AI purely for > prompt versioning — pull your prompts and use them with whatever model you > configure in your code. Alternatively, if you want to manage prompts and model > configurations together as a single versioned unit, you can attach model > settings, output type, and tools to prompt versions. ## Types of Prompts There are two types of prompts you can create: - **(Single) Text Prompt**: Use this when you need a straightforward, one-off prompt for simple completions. - **Prompt Message List**: Use this when you need to define multiple messages with specific roles (system, user, assistant) in an OpenAI messages format. This format is ideal for few-shot prompting, where you can start with a system message that sets the context. > If you ever see a prompt being mentioned without any mention of "message" or > "list", assume it is a single prompt we're talking about. ## Understanding Prompt Versioning In Confident AI, each prompt is identified by a unique `alias`. This `alias` acts as a unique identifier and refers to a single, specific prompt. Different aliases refer to completely separate prompts. Every change you make to a prompt is tracked as a **commit**. This ensures complete history and traceability of all prompt modifications. When you're ready to mark a commit as a stable release, you can promote it to a **version**. > **Example** > > Suppose you have a prompt with the alias `MyPrompt`. Every edit creates a new commit. You can then promote specific commits to versions. > > ```mermaid > flowchart TD > MyPrompt["Alias: MyPrompt"] > MyPrompt --> C1["Commit 1"] > MyPrompt --> C2["Commit 2"] > MyPrompt --> C3["Commit 3 → Version 00.00.01"] > MyPrompt --> C4["Commit 4"] > MyPrompt --> C5["Commit 5 → Version 00.00.02"] > ``` - **Commit**: Every change to a prompt creates a new commit. Commits are automatically tracked and provide a complete history of all modifications. - **Version**: A promoted commit that represents a stable release. Version numbers are controlled by Confident AI in the format `00.00.0X` (e.g., `00.00.01`, `00.00.02`). - **Label**: Labels (like `staging` or `production`) can only be assigned to versions, not commits. This ensures that only stable, versioned prompts are deployed to different environments. ## Commit a New Prompt You can create a prompt in **Project** > **Prompt Studio** through two simple steps: 1. Create a text or messages prompt 2. Edit and commit your changes in the prompt editor \> A prompt cannot be both a text and message prompt at the same time. #### Messages [Video](https://confident-docs.s3.us-east-1.amazonaws.com/prompts:create-messages-4k.mp4) *Create Prompt Messages* #### Text [Video](https://confident-docs.s3.us-east-1.amazonaws.com/prompts:create-text.mp4) *Create Prompt Text* Don't forget to **commit** your changes after you're done editing. Every commit is tracked, and you can later promote any commit to a version. You can also create commits from [code.](/docs/llm-evaluation/prompt-management/automate-prompt-management) > A new version can only be created for commits made after the most recently > versioned commit. Commits made before an existing version cannot be promoted > to a version. > For more advanced push options including model settings, output type, and > tools, see [Automate Prompt > Management](/docs/llm-evaluation/prompt-management/automate-prompt-management). ## Templating Options ### Dynamic variables You can include variables that can be interpolated dynamically in your LLM application later on. There are five interpolation types available: | Type | Syntax | Example | | --------------------- | ---------------- | --------------------------------- | | `FSTRING` | `{variable}` | `Hello, {name}!` | | `MUSTACHE` | `{{variable}}` | `Hello, {{name}}!` | | `MUSTACHE_WITH_SPACE` | `{{ variable }}` | `Hello, {{ name }}!` | | `DOLLAR_BRACKETS` | `${variable}` | `Hello, ${name}!` | | `JINJA` | `{% ... %}` | `{% if admin %}Hello!{% endif %}` | Variable names must not contain spaces: ```python # ✅ Correct usage: "Hi, my name is {name}." "The temperature is {temperature} degrees." "User input: {user_input}" # ❌ Incorrect usage: "Hi, my name is {variable name}." # Spaces in variable name ``` ### Conditional logic Conditional logic can be added when using JINJA interpolation. JINJA supports [jinja templates](https://realpython.com/primer-on-jinja-templating/), which allows you to render more complex logic such as conditional if/else blocks: ```txt If/Else {% if is_admin %} Welcome back, mighty admin {{ name }}! {% else %} Hello {{ name }}, you have regular access. {% endif %} ``` As well as for loops: ```txt For loop Shopping List: {% for item in items %} - {{ item }} {% endfor %} ``` \> Jinja interpolated prompts are only available for Python users. ### Including images You can also include images simply by dragging and dropping something into the text areas. Product preview: Prompt Editor on main with a reference photo of a thermometer embedded in the support assistant's system message *Prompt with images* ## Model Configs Beyond creating and editing prompts, you can also configure model settings, output type, and tools associated with your prompt. These configurations are included in each commit, allowing you to: - Track not just prompt changes, but also model configuration changes - Use directly in code when [pulling prompts in code](/docs/llm-evaluation/prompt-management/pull-prompts) - Compare the impact of models on the same prompt (and vice versa) when [running experiments on your AI app](/docs/llm-evaluation/experiments) > Keeping model configs for prompts on Confident AI does no harm but it doesn't mean you have to use it - for both in code and on the platform in the [Arena](/docs/llm-evaluation/no-code-evals/arena) or when running experiments. However, if model configs confuses you, feel free to leave them out. ### Model settings You can configure the model provider, model name, and model parameters for each prompt. These settings are tracked with each commit, ensuring that when you pull a prompt in code, you also get the exact model configuration needed to run it. Product preview: Model Configuration dialog with OpenAI gpt-4.1, temperature 0.7, top P 1, penalties at 0 and max tokens 1024 *Configure Model Settings* | Field | Description | Example | | -------------- | ------------------------------------------------------------ | ------------------------------------------ | | **Provider** | The LLM provider (e.g., OpenAI, Anthropic, Azure) | `openai` | | **Model** | The specific model name | `gpt-4.1` | | **Parameters** | Model-specific parameters like temperature, max tokens, etc. | `{"temperature": 0.7, "max_tokens": 1024}` | > **Example** > > For an OpenAI GPT-4.1 configuration with custom temperature and max tokens: > > - **Provider**: `openai` > - **Model**: `gpt-4.1` > - **Parameters**: > ```json > { > "temperature": 0.7, > "max_tokens": 1024 > } > ``` ### Output type You can specify the expected output format for your prompt by selecting an output type. This configuration is tracked with each commit: - **Text Output**: Standard text response (default) - **JSON Output**: Structured JSON response - **Schema Output**: Structured response conforming to a defined schema Product preview: Prompt Editor with the Output Type menu open: Text Output, JSON Output and Schema Output *Configure Output Type* When using **Schema Output**, you can define a custom schema that your LLM response should conform to. This is useful for ensuring structured, predictable outputs. To configure a schema: 1. Click on **Schema Output** in the output type dropdown 2. Enter a **Schema Name** (required) 3. Add **Schema Fields** with their property names and types (String, Number, Boolean, etc.) 4. Click **Save Schema** Product preview: Configure Schema Output dialog for SupportAnswer with answer, order\_id, escalate and sources fields and its Pydantic preview *Configure Schema Output* The schema will be previewed as a Pydantic `BaseModel` class, making it easy to visualize how your structured output will look. ### Attach tools You can attach tools to your prompt for function calling capabilities. This allows your LLM to invoke external tools like web search, APIs, or custom functions. Tools are versioned, and each commit records which version of each tool it uses. To attach tools: 1. Under **Tools** in the prompt editor, click the tools field (it reads **Select tool...** until a tool is attached, then lists the attached tools) 2. Search for a tool, or click **New tool** to create one 3. Check the tools you want to enable for this prompt version. Each shows the version the prompt uses Product preview: Prompt Editor with the Tools menu open, each tool showing its version, lookup\_order and search\_knowledge\_base checked *Attach Tools* Changing a tool's input parameters or tool mode saves it as a new version; renaming it or editing its description doesn't. To use a different version in this prompt, click **Edit tool** beside the tool, choose its **Version**, and click **Use this version**. A prompt on an older version shows both, e.g. **Version 00.00.01 · latest 00.00.02**. ## Assign Prompt Labels Labels can only be assigned to **versions**, not commits. This ensures that only stable, versioned prompts are deployed to different environments. You can assign labels in the **Version History** page so no code changes are required to "deploy" a new version into a certain environment. [Video](https://confident-docs.s3.us-east-1.amazonaws.com/prompts:label-versions.mp4) To assign a label, first promote a commit to a version, then assign the desired label (e.g., `staging`, `production`) to that version. Only users with [sufficient permissions](/docs/settings/project/roles-and-permissions) are able to modify prompt labels. > The next section will dive deeper into this topic but this is how you can pull a prompt by its label with the Confident AI SDK in Python: > > ```python main.py > from confident_ai import ConfidentAI > > client = ConfidentAI() > > prompt = client.prompt(prompt_id="") > prompt.pull(label="staging") > ``` ## Next Steps Now that you know how to version prompts on the platform, put them to work in evaluations. #### [Experiments](/docs/llm-evaluation/experiments) Compare prompt versions side-by-side with statistical rigor. #### [Single-Turn Evals](/docs/llm-evaluation/no-code-evals/single-turn-evals) Run evaluations on your prompts without writing code. #### [Pull Prompts](/docs/llm-evaluation/prompt-management/pull-prompts) Pull prompt versions into your code for use in your LLM app. #### [Automate Prompt Management](/docs/llm-evaluation/prompt-management/automate-prompt-management) Push and manage prompts programmatically via the Confident API.