Introducing Confident Monitor

Managing Tools via Confident AI SDK

Overview

The Confident AI SDK exposes every Tool method on the platform. This page documents how to call these methods in all supported languages. See the introduction to install the SDK and set your API key.

Methods

List Tools

Lists the tools in your Confident AI project one page at a time, newest first. Retrieve a tool by id to read its input schema.

from confident_ai import ConfidentAI

client = ConfidentAI()

result = client.tools.list(page=1, page_size=25, search_term="customer")

For async mode, call a_list and await it as shown below:

result = await client.tools.a_list(...)

Parameters

ParameterTypeDescription
pageOptional[int]The page to return. Defaults to 1.
page_sizeOptional[int]The number of results per page, at most 100. Defaults to 25.
search_termOptional[str]Returns only tools whose name contains this text.

Returns

This method returns an object of type ToolList.

Create Tool

Creates a tool in your Confident AI project and returns its id. Names are unique within a project; reusing one returns a 409.

from confident_ai import ConfidentAI
from confident_ai.common import ToolMode

client = ConfidentAI()

result = client.tools.create(
    name="get_customer_name",
    mode=ToolMode.ALLOW_ADDITIONAL,
    schema={
        "type": "object",
        "properties": {
            "customerId": {
                "type": "string",
                "description": "The id of the customer to look up."
            },
            "nameFormat": {
                "type": "string",
                "enum": ["FULL", "FIRST"],
                "description": "Whether to return the full name or only the first name."
            }
        },
        "required": ["customerId"]
    },
    description="Looks up the customer's name by id.",
)

For async mode, call a_create and await it as shown below:

result = await client.tools.a_create(...)

Parameters

ParameterTypeDescription
namestrRequired. The name of the tool, unique within the project. The model calls the tool by this name.
modeToolModeRequired. See ToolMode.
schemaDict[str, Any]Required. The JSON Schema for the tool's input. List each parameter under properties with a type, and name the mandatory ones in required. A parameter can also have a description, an enum of allowed values, or items for an array. Whether extra properties are allowed is set by mode.
descriptionOptional[str]What the tool does. The model reads it to decide when to call the tool.

Returns

This method returns an object of type ToolRef.

Get Tool

Retrieves a tool by id, including its input schema. Pass version to retrieve that version instead of the latest.

from confident_ai import ConfidentAI

client = ConfidentAI()

result = client.tools.get(tool_id="<TOOL-ID>", version="00.00.01")

For async mode, call a_get and await it as shown below:

result = await client.tools.a_get(...)

Parameters

ParameterTypeDescription
tool_idstrRequired. The id of the tool.
versionOptional[str]The version to retrieve. Omit it for the latest version.

Returns

This method returns an object of type Tool.

Update Tool

Updates a tool and returns it at its latest version. A changed mode or schema creates a new version, and prompt commits keep the version they pinned. Names are unique within a project; taking another tool's name returns a 409.

from confident_ai import ConfidentAI
from confident_ai.common import ToolMode

client = ConfidentAI()

result = client.tools.update(
    tool_id="<TOOL-ID>",
    name="get_customer_name",
    description="Looks up the customer's name by id.",
    mode=ToolMode.ALLOW_ADDITIONAL,
    schema={
        "type": "object",
        "properties": {
            "customerId": {
                "type": "string",
                "description": "The id of the customer to look up."
            },
            "nameFormat": {
                "type": "string",
                "enum": ["FULL", "FIRST"],
                "description": "Whether to return the full name or only the first name."
            }
        },
        "required": ["customerId"]
    },
)

For async mode, call a_update and await it as shown below:

result = await client.tools.a_update(...)

Parameters

ParameterTypeDescription
tool_idstrRequired. The id of the tool.
nameOptional[str]The name of the tool, unique within the project. Omit it to keep the current name.
descriptionOptional[str]What the tool does. Send null to clear it, or omit it to keep the current one.
modeOptional[ToolMode]See ToolMode.
schemaOptional[Dict[str, Any]]The JSON Schema for the tool's input. List each parameter under properties with a type, and name the mandatory ones in required. A parameter can also have a description, an enum of allowed values, or items for an array. Whether extra properties are allowed is set by mode. Omit it to keep the current one.

Returns

This method returns an object of type Tool.

Delete Tool

Permanently deletes a tool and all its versions. Prompt commits that pinned it no longer include it.

from confident_ai import ConfidentAI

client = ConfidentAI()

result = client.tools.delete(tool_id="<TOOL-ID>")

For async mode, call a_delete and await it as shown below:

result = await client.tools.a_delete(...)

Parameters

ParameterTypeDescription
tool_idstrRequired. The id of the tool.

Returns

This method returns an object of type ToolRef.

List Versions

Lists a tool's versions, newest first, so the first is its latest. Changing a tool's mode or schema adds a version.

from confident_ai import ConfidentAI

client = ConfidentAI()

result = client.tools.list_versions(
    tool_id="<TOOL-ID>",
    page=1,
    page_size=25,
)

For async mode, call a_list_versions and await it as shown below:

result = await client.tools.a_list_versions(...)

Parameters

ParameterTypeDescription
tool_idstrRequired. The id of the tool.
pageOptional[int]The page to return. Defaults to 1.
page_sizeOptional[int]The number of versions per page, at most 100. Defaults to 25.

Returns

This method returns an object of type ToolVersionList.

Types

Tool

A tool your prompts can call. Changing its mode or schema creates a new version.

class Tool:
    id: str
    name: str
    description: Optional[str]
    mode: ToolMode
    version: str
    schema_: Dict[str, Any] = Field(alias="schema")

idstrRequired

The id of the tool, generated by Confident AI.

Example: "<TOOL-ID>"

namestrRequired

The name of the tool, unique within the project.

Example: "get_customer_name"

descriptionOptional[str]Required

What the tool does.

Example: "Looks up the customer's name by id."

modeToolModeRequired

versionstrRequired

The version of the tool. A pulled prompt returns the version its commit pinned.

Example: "00.00.01"

schema_Dict[str, Any]Required

The JSON Schema for the tool's input. List each parameter under properties with a type, and name the mandatory ones in required. A parameter can also have a description, an enum of allowed values, or items for an array. Whether extra properties are allowed is set by mode.

Example: {"type":"object","properties":{"customerId":{"type":"string","description":"The id of the customer to look up."},"nameFormat":{"type":"string","enum":["FULL","FIRST"],"description":"Whether to return the full name or only the first name."}},"required":["customerId"]}

ToolList

One page of tools, with the total across all pages.

class ToolList:
    tools: List[ToolSummary]
    total_tools: int = Field(alias="totalTools")
    page: int
    page_size: int = Field(alias="pageSize")

toolsList[ToolSummary]Required

The tools for the current page, newest first.

See ToolSummary.

total_toolsintRequired

The total number of tools in this project.

Example: 3

pageintRequired

The page this response covers.

Example: 1

page_sizeintRequired

The number of tools per page.

Example: 25

ToolMode

This is the mode for your tool input fields, which controls whether fields outside the schema are allowed.

class ToolMode(Enum):
    ALLOW_ADDITIONAL = "ALLOW_ADDITIONAL"
    NO_ADDITIONAL = "NO_ADDITIONAL"
    STRICT = "STRICT"

ALLOW_ADDITIONAL · NO_ADDITIONAL · STRICT

ToolRef

A reference to a tool by its id.

class ToolRef:
    id: str

idstrRequired

The id of the tool, generated by Confident AI.

Example: "<TOOL-ID>"

ToolSummary

A tool as it appears in a list: enough to recognise it and pick its id. Retrieve one by id to read its input schema.

class ToolSummary:
    id: str
    name: str
    version: str

idstrRequired

The id of the tool, generated by Confident AI.

Example: "<TOOL-ID>"

namestrRequired

The name of the tool.

Example: "get_customer_name"

versionstrRequired

The latest version of the tool.

Example: "00.00.02"

ToolVersion

One saved definition of a tool.

class ToolVersion:
    version: str
    mode: ToolMode
    schema_: Dict[str, Any] = Field(alias="schema")
    created_at: str = Field(alias="createdAt")

versionstrRequired

The version of the tool.

Example: "00.00.02"

modeToolModeRequired

schema_Dict[str, Any]Required

The JSON Schema for the tool's input. List each parameter under properties with a type, and name the mandatory ones in required. A parameter can also have a description, an enum of allowed values, or items for an array. Whether extra properties are allowed is set by mode.

Example: {"type":"object","properties":{"customerId":{"type":"string","description":"The id of the customer to look up."},"nameFormat":{"type":"string","enum":["FULL","FIRST"],"description":"Whether to return the full name or only the first name."}},"required":["customerId"]}

created_atstrRequired

When the version was saved.

Example: "2025-01-18T16:45:00+00:00"

ToolVersionList

One page of a tool's versions, with the total across all pages.

class ToolVersionList:
    versions: List[ToolVersion]
    total_tool_versions: int = Field(alias="totalToolVersions")
    page: int
    page_size: int = Field(alias="pageSize")

versionsList[ToolVersion]Required

The tool's versions for the current page, newest first.

See ToolVersion.

total_tool_versionsintRequired

The number of versions this tool has.

Example: 2

pageintRequired

The page this response covers.

Example: 1

page_sizeintRequired

The number of versions per page.

Example: 25

Building a production pipeline?Design a scalable API workflow for evals, datasets, traces, and promptsTalk to an engineer

Last updated on

Built byConfident AI