Model Settings

By default, Confident AI provides the models used for every eval, simulation and AI feature on the platform. Model Settings lets you choose your own instead, and connect the provider credentials they need.

Navigate to Project Settings → Model Settings. The page has four tabs: Credentials, Large-Language Models, Voice Models and Decision Models.

Credentials

Connect a provider here before you pick its models on the other tabs: you can only select a provider that has credentials.

To configure a provider, click the three-dot menu (⋮) on the right of its row and enter your API key or configuration details. Providers are grouped into three tables:

Model Providers — provide your API key to run evaluations with that provider:

• OpenAI
• Anthropic
• Gemini
• xAI
• DeepSeek
• Mistral
• Perplexity
• fal (speech-to-speech voice models)
• TypeSafe (decision models)

Cloud Providers — run evaluations using models hosted on your cloud infrastructure:

• Amazon Bedrock
• Vertex AI
• Azure

LLM Gateway — connect a gateway and manage tag-based routing credentials:

• Portkey
• OpenRouter
• LiteLLM
• TrueFoundry
• Hugging Face

Your organization's model provider policy can hide providers it doesn't allow.

Alternatively, toggle Inherit from Organization to use the model credentials configured at the organization level instead of configuring them per project.

Large-Language Models

The Large-Language Models tab chooses the LLMs your project uses, one row each:

• Evaluation Model — runs every LLM-as-a-judge metric on Confident AI.
• Platform Model — powers platform features for this project, such as categorization, scenario generation and AI summaries.
• Simulation Model — simulates user turns in multi-turn evaluations and red teaming.

To change a model:

1. Select a Model Provider from the dropdown
2. Select the specific Model to use (e.g., gpt-4o)
3. Click Save to apply your changes

The Platform Model and Simulation Model use your organization's models by default, shown greyed out. Switch on Override organization model to choose a different one for this project.

Voice Models

The Voice Models tab chooses the models used in phone simulations of voice agents:

• Text-to-Speech (TTS) — speaks the simulated user's turns to your voice agent.
• Speech-to-Text (STT) — transcribes what your voice agent says.
• Speech-to-Speech (S2S) — switch on Use real-time architecture to use a real-time caller instead of the default cascade (Speech-to-Text, simulation model, Text-to-Speech), for lower latency and natural interruptions. The first call can take about a minute to start.

Decision Models

The Decision Models tab chooses the decision model that answers metric questions with calibrated probabilities.

Last updated on October 5, 2026