Platform Settings
Confident AI provides comprehensive settings to help you configure your workspace exactly how you need it. Settings are organized into two categories based on their scope and impact.
Overview
Settings in Confident AI are divided into two levels:
- Project Settings — Configure settings that apply to a specific project
- Organization Settings — Manage settings that apply across all projects in your organization
Teams focused on org-wide AI adoption—such as Centers of Excellence, innovation teams, or platform teams in large enterprises—will find Organization Settings particularly relevant. These settings help manage resources and governance for multiple AI teams across the company.
Project vs Organization
When you create an account on Confident AI, an organization and a project within that organization are automatically created for you. Each organization can have multiple projects and users.
-
Organization — The top-level container for all your Confident AI resources. Subscription plans and billing are managed at the organization level.
-
Project — A workspace where data and access permissions are isolated. All data (test cases, metrics, datasets, traces, etc.) is separated at the project level.
Project Settings
Project settings allow you to customize configurations for individual projects, including team access, evaluation models, and integrations.
Add team members to your project and manage their access.
Generate and manage API keys for authentication.
Configure the models used for LLM-as-a-judge evaluations.
Set up Slack, Discord, Teams, or email notifications.
Connect your AI provider API keys for evaluations.
Configure alerts for important project events.
Define criteria for human-in-the-loop annotations.
Manage how long your evaluation data is stored.
Control access levels for team members.
Organization Settings
Organization settings control resources and configurations that apply to all projects within your organization.
View and manage all projects in your organization.
Manage users and their access across the organization.
Configure Single Sign-On for secure authentication.
Manage API credentials for AI models shared across projects.
Control which features are enabled for projects.
