Transformers

Transformers are Python functions that preprocess and reshape data before it runs through evaluation pipelines. They let you extract the exact fields needed for evaluation, even from nested or non-standard response structures from your AI application.

Create a Transformer

To create a new transformer:

1. Navigate to Project Settings > Transformers
2. Click New Transformer
3. Enter a unique Name for the transformer
4. Optionally add a Description to help you remember what the transformer does
5. Write your Python function in the code editor
6. Click Save

Every transformer must define a transformer function that accepts a data parameter and returns the transformed result:

from typing import Any

def transformer(data: Any):
    # Your logic to process the data here
    return data

The from typing import Any import and the def transformer(data: Any): function signature are fixed and cannot be modified.

For example, a trace's input is often a list of messages in the OpenAI chat format. To use it as a golden's input in a dataset, return the last user message:

from typing import Any

def transformer(data: Any):
    # data: OpenAI chat messages, e.g. [{"role": "user", "content": "..."}]
    user_messages = [m["content"] for m in data if m["role"] == "user"]
    return user_messages[-1] if user_messages else ""

Test a Transformer

The Transformer Debugger next to the code editor lets you verify your transformer before saving:

1. Enter mock input data in the debugger (Python syntax, e.g. {"key": "value"}); Format tidies it up
2. Click Test code to run the transformer against the input
3. View the result below the input

Test data is persisted locally so you can iterate without re-entering it each time.

Use Transformers in AI Connections

Transformers are used in AI Connections as an alternative to JSON key paths for extracting data from AI app responses. You can assign a transformer for each of the following fields:

• Actual Output - Extracts the main response from your AI app
• Retrieval Context - Extracts retrieval context from responses
• Tools Called - Extracts tool call information
• State - Extracts state data from responses

When configuring an AI Connection, select the Transformer tab in any parsing section and choose the transformer you want to use from the dropdown.

Manage Transformers

From the transformers list, use the three-dot menu on any row to Edit or Delete a transformer.

Last updated on October 5, 2026