# Confident AI Docs

## Instructions for AI Agents

- For clean Markdown of any page, append `.md` to the page URL
- For a complete documentation index, see https://confident-ai.docs.buildwithfern.com/docs/llms.txt
- For full documentation content in one file, see https://confident-ai.docs.buildwithfern.com/docs/llms-full.txt
- For section-specific indexes, append `/llms.txt` to any section URL
- For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://confident-ai.docs.buildwithfern.com/docs/_mcp/server

## Docs

- [Introduction](https://confident-ai.docs.buildwithfern.com/docs.md): Get started with Confident AI for LLM evaluation and observability
- [Setup and Installation](https://confident-ai.docs.buildwithfern.com/docs/setup-and-installation.md): Quick setup guide for Confident AI
- [Introduction to LLM Evaluation](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/introduction.md): Learn how to evaluate AI applications using Confident AI's code-driven or no-code workflows.
- [LLM Evaluation Quickstart](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/quickstart.md): 5 min quickstart guide for a code-driven LLM evaluation workflow
- [Single-Turn, E2E Evals](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/single-turn/end-to-end.md): Learn how to run end-to-end testing for single-turn use cases
- [Single-Turn, Component-Level Evals](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/single-turn/component-level.md): Learn how to run component-level testing for single-turn use cases
- [Multi-Turn Evals](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/code-driven/multi-turn.md): Simulate conversations and run end-to-end testing for multi-turn use cases
- [Unit-Testing in CI/CD](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/unit-testing-cicd.md): Setup an automated pre-deployment workflow in CI/CD
- [No-Code Evals Quickstart](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/no-code-evals/quickstart.md): Run your first evaluation in the platform UI — no code required.
- [Single-Turn Evals (No-Code)](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/no-code-evals/single-turn-evals.md): Evaluate one-shot interactions like Q&A, summarization, and classification.
- [Multi-Turn Evals (No-Code)](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/no-code-evals/multi-turn-evals.md): Evaluate conversational AI where context builds across multiple exchanges.
- [Arena](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/no-code-evals/arena.md): Quickly compare prompts, models, and AI connections side-by-side without running a full evaluation.
- [Experiment with AI Apps](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/experiments.md): Move beyond vibes. Compare multiple versions of your AI app with statistical rigor.
- [Manage Datasets](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/dataset-management/manage-datasets.md): Learn the core functions of a dataset, and ways to manipulate goldens within
- [Introduction](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/dataset-management/synthetic-data/introduction.md): Generate synthetic goldens from your own data sources
- [Data Source Connectors](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/dataset-management/synthetic-data/data-source-connectors.md): Set up credentials and connect external data sources to generate synthetic goldens
- [Automate Dataset Management](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/dataset-management/automate-dataset-management.md): Programmatically push goldens to datasets via the Evals API.
- [Pull Datasets](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/dataset-management/using-datasets.md): Pull datasets locally to use them for evaluation.
- [Version Prompts](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/prompt-management/version-prompts.md): Create and manage different versions of your prompts
- [Automate Prompt Management](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/prompt-management/automate-prompt-management.md): Build automated prompt management pipelines via the Evals API
- [Pull Prompts](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/prompt-management/pull-prompts.md): Learn how to test and use prompts in your LLM app
- [Single vs Multi-Turn Evals](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/core-concepts/single-vs-multi-turn-evals.md): Get started with LLM evaluation in minutes
- [Test Cases, Goldens, and Datasets](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/core-concepts/test-cases-goldens-datasets.md): Get started with LLM evaluation in minutes
- [LLM-as-a-Judge Metrics](https://confident-ai.docs.buildwithfern.com/docs/llm-evaluation/core-concepts/llm-as-a-judge.md): Get started with LLM evaluation in minutes
- [LLM Metrics on Confident AI](https://confident-ai.docs.buildwithfern.com/docs/documentation/metrics/introduction.md): Overview of metrics in Confident AI
- [Metric Collections](https://confident-ai.docs.buildwithfern.com/docs/metrics/metric-collections.md): Metric collections allow you to group together metric runs on Confident AI
- [Custom Metrics](https://confident-ai.docs.buildwithfern.com/docs/documentation/metrics/custom-metrics.md): Create custom metrics for your specific use case
- [G-Eval](https://confident-ai.docs.buildwithfern.com/docs/documentation/metrics/custom-metrics/g-eval.md): Learn how to create a G-Eval metric for custom evaluation algorithms
- [Code-Evals](https://confident-ai.docs.buildwithfern.com/docs/documentation/metrics/custom-metrics/code-evals.md): Create custom metrics using Python code on Confident AI
- [Answer Relevancy](https://confident-ai.docs.buildwithfern.com/docs/metrics/single-turn/answer-relevancy-metric.md): Answer relevancy is a single-turn metric to evaluate RAG generators
- [Bias](https://confident-ai.docs.buildwithfern.com/docs/metrics/single-turn/bias-metric.md): Bias is a single-turn safety metric to determine if your LLM output contains gender, racial, or political bias.
- [Contextual Precision](https://confident-ai.docs.buildwithfern.com/docs/metrics/single-turn/contextual-precision-metric.md): Contextual Precision is a single-turn metric used to evaluate a RAG retriever
- [Contextual Recall](https://confident-ai.docs.buildwithfern.com/docs/metrics/single-turn/contextual-recall-metric.md): Contextual Recall is a single-turn metric used to evaluate a RAG retriever
- [Contextual Relevancy](https://confident-ai.docs.buildwithfern.com/docs/metrics/single-turn/contextual-relevancy-metric.md): Contextual Relevancy is a single-turn metric used to evaluate a RAG retriever
- [Faithfulness](https://confident-ai.docs.buildwithfern.com/docs/metrics/single-turn/faithfulness-metric.md): Faithfulness is a single-turn metric to evaluate RAG generators
- [Hallucination](https://confident-ai.docs.buildwithfern.com/docs/metrics/single-turn/hallucination-metric.md): Halucination is a single-turn metric to determine if your LLM is hallucinating false information.
- [Summarization](https://confident-ai.docs.buildwithfern.com/docs/metrics/single-turn/summarization-metric.md): Summarization is a single-turn metric to determine if your summarizer is generating facutally correct summaries.
- [Task Completion](https://confident-ai.docs.buildwithfern.com/docs/metrics/single-turn/task-completion-metric.md): Task Completion is a single-turn metric to determine an agent's task completion score
- [Tool Correctness](https://confident-ai.docs.buildwithfern.com/docs/metrics/single-turn/tool-correctness-metric.md): Tool Correctness is a single-turn metric to determine an agent's tool calling ability
- [Toxicity](https://confident-ai.docs.buildwithfern.com/docs/metrics/single-turn/toxicity-metric.md): Toxicity is a single-turn safety metric to determine any toxicity in LLM's output
- [Conversation Completeness](https://confident-ai.docs.buildwithfern.com/docs/metrics/multi-turn/conversation-completeness-metric.md): Conversation Completeness is a multi-turn metric to determine if a conversation is complete.
- [Knowledge Retention](https://confident-ai.docs.buildwithfern.com/docs/metrics/single-turn/knowledge-retention-metric.md): Knowledge Retention is a multi-turn metric to determine if your chatbot remembers details well.
- [Role Adherence](https://confident-ai.docs.buildwithfern.com/docs/metrics/multi-turn/role-adherence-metric.md): Role Adherence is a multi-turn metric to determine if your chatbot adheres to a specified role
- [Turn Relevancy](https://confident-ai.docs.buildwithfern.com/docs/metrics/multi-turn/turn-relevancy-metric.md): Turn Relevancy is a multi-turn metric to determine if your chatbot responses are releveant to user input
- [Introduction to LLM Tracing](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/introduction.md): Learn about LLM tracing with Confident AI
- [LLM Tracing Quickstart](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/quickstart.md): Instrument your LLM application for observability in less than 5 minutes
- [Configure Span Types](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/features/span-types.md): Categorize your spans by type and set type-specific attributes
- [Log Prompts](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/features/log-prompts.md): Log prompts to LLM spans for version tracking in production
- [Track LLM Costs](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/features/token-usage-cost.md): Track the token usage and cost of your LLM calls
- [Set Input/Output](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/features/input-output.md): Learn how to supply input and output of your LLM application in a trace
- [Thread Traces](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/features/threads.md): Group your traces as threads to evaluate an entire conversation workflow
- [Evaluate Traces & Spans](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/online-evals.md): Run online and offline evaluations on individual traces and spans on the fly
- [Evaluate Threads](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/evaluate-threads.md): Run evaluations on multi-turn conversations by evaluating entire threads
- [Name](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/features/name.md): Giving names to your traces for better visbility on Confident AI
- [Tags](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/features/tags.md): Adding tags to your traces for better visibility on Confident AI
- [Metadata](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/features/metadata.md): Adding metadata to your traces for additional information
- [Users](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/features/users.md): Tracking user info in your traces for observability
- [Projects](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/features/projects.md): Send traces to different projects on Confident AI
- [Environment](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/features/environment.md): Set your environments during tracing for better debugging
- [Masking](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/features/masking.md): Protect your sensitive information from traces using the masking feature
- [Sampling](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/features/sampling.md): Sending only part of your traces to Confident AI
- [Dropping Traces](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/features/dropping-traces.md): Conditionally dropping traces before they are sent to Confident AI
- [Signals](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/features/signals.md): Explore Signals — the runtime view of classifier output across your project.
- [Troubleshooting](https://confident-ai.docs.buildwithfern.com/docs/llm-tracing/troubleshooting.md): Common issues and fixes when using @observe for tracing
- [Introduction](https://confident-ai.docs.buildwithfern.com/docs/human-in-the-loop/introduction.md): Learn how domain experts can contribute to AI testing
- [Collect Feedback](https://confident-ai.docs.buildwithfern.com/docs/human-in-the-loop/collect-feedback.md): Incorperate real user feedback into your evaluation pipeline
- [Overview](https://confident-ai.docs.buildwithfern.com/docs/human-in-the-loop/annotation-queues.md): Learn about annotation queues, and how to assign items for team members to annotate
- [Eval Alignment](https://confident-ai.docs.buildwithfern.com/docs/human-in-the-loop/eval-alignment.md): Measure agreement between metric evaluations and human annotations on the same items in an annotation queue.
- [Error Analysis](https://confident-ai.docs.buildwithfern.com/docs/human-in-the-loop/error-analysis.md): Run an LLM-driven analysis over annotation queue feedback to surface failure modes and generate metric suggestions.
- [Dashboards](https://confident-ai.docs.buildwithfern.com/docs/reporting-analytics/dashboards.md): Create dashboards with widgets that visualize and break down your project data.
- [Executive Insights](https://confident-ai.docs.buildwithfern.com/docs/reporting-analytics/executive-insights.md): Configure executive insights, read the generated reports, and export them as PDFs.
- [Introduction to Red Teaming](https://confident-ai.docs.buildwithfern.com/docs/red-teaming/introduction.md): Get started with Confident AI's Red Teaming for AI safety and security assessment
- [Red Teaming Quickstart (No-Code)](https://confident-ai.docs.buildwithfern.com/docs/red-teaming/no-code-assessments/quickstart.md): Run your first risk assessment in the platform UI — no code required.
- [Frameworks & Policies](https://confident-ai.docs.buildwithfern.com/docs/red-teaming/framework-policies.md): Learn how to use AI security frameworks managed on Confident AI
- [Risk Profiles](https://confident-ai.docs.buildwithfern.com/docs/red-teaming/risk-profile.md): Risk assessments, top vulnerabilities, incident monitoring, and more.
- [Trace-Level Detections](https://confident-ai.docs.buildwithfern.com/docs/red-teaming/trace-level-detections.md): How Confident AI scans traced application spans for vulnerabilities during risk assessments and where to view the resulting detections.
- [Red Team Using DeepTeam](https://confident-ai.docs.buildwithfern.com/docs/red-teaming/code-driven-assessments.md): Get started with AI red teaming in code via DeepTeam
- [Why Confident AI](https://confident-ai.docs.buildwithfern.com/docs/resources/why-confident-ai.md): Understanding why Confident AI is right for you
- [Support](https://confident-ai.docs.buildwithfern.com/docs/support.md): Our team is here to support you along the way
- [Data Handling](https://confident-ai.docs.buildwithfern.com/docs/resources/data-handling.md): Learn about data handling and privacy
- [LLM Use Cases](https://confident-ai.docs.buildwithfern.com/docs/resources/llm-use-cases.md): Learn about supported LLM use cases
- [Introduction](https://confident-ai.docs.buildwithfern.com/docs/api-reference/introduction.md): Welcome to Confident AI's Evals API reference.
- [Quickstart](https://confident-ai.docs.buildwithfern.com/docs/api-reference/quickstart.md): 5 min quickstart guide for Confident AI's Evals API
- [Authentication](https://confident-ai.docs.buildwithfern.com/docs/api-reference/authentication.md): Understand how auth works for Evals API
- [Data Models](https://confident-ai.docs.buildwithfern.com/docs/api-reference/data-models.md): Understand the data models that you will be manipulating via the Evals API
- [API Conventions](https://confident-ai.docs.buildwithfern.com/docs/api-reference/api-conventions.md): Understand the status codes, error formats, and response structures returned by the Evals API
- [Integrations](https://confident-ai.docs.buildwithfern.com/docs/integrations.md): Instrument your LLM app with one-line integrations or via the OpenTelemetry standard
- [OpenTelemetry](https://confident-ai.docs.buildwithfern.com/docs/integrations/opentelemetry.md)
- [Distributed Tracing](https://confident-ai.docs.buildwithfern.com/docs/integrations/opentelemetry/distributed-tracing.md)
- [Trace Broadcasting](https://confident-ai.docs.buildwithfern.com/docs/integrations/opentelemetry/trace-broadcasting.md)
- [OpenAI](https://confident-ai.docs.buildwithfern.com/docs/integrations/third-party/openai.md): Use Confident AI for LLM observability and evals for OpenAI
- [LangChain](https://confident-ai.docs.buildwithfern.com/docs/integrations/third-party/langchain.md): Use Confident AI for LLM observability and evals for LangChain
- [Pydantic AI](https://confident-ai.docs.buildwithfern.com/docs/integrations/third-party/pydantic-ai.md): Use Confident AI for LLM observability and evals for PydanticAI
- [LangGraph](https://confident-ai.docs.buildwithfern.com/docs/integrations/third-party/langgraph.md): Use Confident AI for LLM observability and evals for LangGraph
- [OpenAI Agents](https://confident-ai.docs.buildwithfern.com/docs/integrations/third-party/openai-agents.md): Use Confident AI for LLM observability and evals for OpenAI Agents
- [LlamaIndex](https://confident-ai.docs.buildwithfern.com/docs/integrations/third-party/llama-index.md): Use Confident AI for LLM observability and evals for LlamaIndex
- [Crew AI](https://confident-ai.docs.buildwithfern.com/docs/integrations/third-party/crew-ai.md): Use Confident AI for LLM observability and evals for CrewAI
- [Vercel AI SDK](https://confident-ai.docs.buildwithfern.com/docs/integrations/third-party/vercel-ai-sdk.md): Use Confident AI for LLM observability and evals for Vercel AI SDK on typescript
- [Agent Core](https://confident-ai.docs.buildwithfern.com/docs/integrations/third-party/agentcore.md): Use Confident AI for LLM observability and evals for Amazon AgentCore
- [Strands Agents](https://confident-ai.docs.buildwithfern.com/docs/integrations/third-party/strands.md): Use Confident AI for LLM observability and evals for Strands Agents
- [Google ADK](https://confident-ai.docs.buildwithfern.com/docs/integrations/third-party/google-adk.md): Use Confident AI for LLM observability and evals for Google ADK
- [Open Inference](https://confident-ai.docs.buildwithfern.com/docs/integrations/third-party/openinference.md): Use Confident AI for LLM observability and evals for OpenInference
- [LiteLLM](https://confident-ai.docs.buildwithfern.com/docs/integrations/third-party/litellm.md): Use Confident AI for LLM observability and evals for LiteLLM
- [Portkey](https://confident-ai.docs.buildwithfern.com/docs/integrations/third-party/portkey.md): Portkey AI serves as a unified interface for interacting LLMs
- [Platform Settings](https://confident-ai.docs.buildwithfern.com/docs/settings.md): Configure and manage your project and organization settings on Confident AI
- [Data Residency](https://confident-ai.docs.buildwithfern.com/docs/settings/data-residency.md): Learn about data residency configuration
- [RBAC](https://confident-ai.docs.buildwithfern.com/docs/settings/rbac.md): Learn about role-based access control in Confident AI
- [API Keys](https://confident-ai.docs.buildwithfern.com/docs/settings/project/api-keys.md): Learn about API key management
- [Team Members](https://confident-ai.docs.buildwithfern.com/docs/settings/project/team-members.md): Learn about inviting and managing team members.
- [Roles & Permissions](https://confident-ai.docs.buildwithfern.com/docs/settings/project/roles-and-permissions.md): Learn about roles and permissions management
- [Transformers](https://confident-ai.docs.buildwithfern.com/docs/settings/project/transformers.md): Learn about data transformers
- [Integrations](https://confident-ai.docs.buildwithfern.com/docs/settings/project/integrations.md): Learn about project integrations
- [Alerts](https://confident-ai.docs.buildwithfern.com/docs/settings/project/alerts.md): Learn about project alerts configuration
- [AI Connections](https://confident-ai.docs.buildwithfern.com/docs/settings/project/ai-connections.md): Learn about setting up AI connections
- [Confident Agent](https://confident-ai.docs.buildwithfern.com/docs/settings/project/confident-agent.md): Learn how to deploy the Confident Agent to bridge evaluations to internal endpoints.
- [Model Costs](https://confident-ai.docs.buildwithfern.com/docs/settings/project/model-costs.md): Learn about setting up custom model costs for cost tracing
- [Data Usage](https://confident-ai.docs.buildwithfern.com/docs/settings/project/data-usage.md): Learn about tracking data usage and cost insights
- [Evaluation Models](https://confident-ai.docs.buildwithfern.com/docs/settings/project/evaluation-models.md): Learn about evaluation models configuration
- [Evaluation Rules](https://confident-ai.docs.buildwithfern.com/docs/settings/project/evaluation-rules.md): Learn about evaluation rules configuration
- [Annotation Options](https://confident-ai.docs.buildwithfern.com/docs/settings/project/annotation-options.md): Learn about annotation option configuration
- [Classifiers](https://confident-ai.docs.buildwithfern.com/docs/settings/project/classifiers.md): Learn how to configure classifiers and labels that automatically tag traces and threads.
- [Data Sources](https://confident-ai.docs.buildwithfern.com/docs/settings/project/data-sources.md): Learn about data sources for dataset generation
- [Data Retention](https://confident-ai.docs.buildwithfern.com/docs/settings/project/data-retention.md): Learn about data retention settings
- [Audit Logs](https://confident-ai.docs.buildwithfern.com/docs/settings/project/audit-logs.md): Learn about project audit logs
- [Projects](https://confident-ai.docs.buildwithfern.com/docs/settings/organization/projects.md): Learn about organization project management
- [Users](https://confident-ai.docs.buildwithfern.com/docs/settings/organization/users.md): Learn about organization user management
- [Roles & Permissions](https://confident-ai.docs.buildwithfern.com/docs/settings/organization/roles-and-permissions.md): Learn about organization roles and permissions management
- [Model Credentials](https://confident-ai.docs.buildwithfern.com/docs/settings/organization/model-credentials.md): Learn about organization-wide model credentials
- [Single-Sign-On (SSO)](https://confident-ai.docs.buildwithfern.com/docs/settings/organization/sso.md): Learn about SSO configuration
- [Data Retention](https://confident-ai.docs.buildwithfern.com/docs/settings/organization/data-retention.md): Learn about organization data retention settings
- [Audit Logs](https://confident-ai.docs.buildwithfern.com/docs/settings/organization/audit-logs.md): Learn about organization audit logs
- [Feature Access](https://confident-ai.docs.buildwithfern.com/docs/settings/organization/feature-access.md): Learn about feature access configuration
- [Introduction to the Admin SDK](https://confident-ai.docs.buildwithfern.com/docs/settings/project/management/introduction.md): Manage organizations, projects, members, roles, policies, and API keys using the Admin SDK for Python and TypeScript.
- [Admin SDK Quickstart](https://confident-ai.docs.buildwithfern.com/docs/settings/project/management/quickstart.md): Install the Admin SDK for Python or TypeScript, configure it with an Organization API Key, and make your first programmatic management call.
- [Organization](https://confident-ai.docs.buildwithfern.com/docs/settings/project/management/organization.md): Manage your organization programmatically with the Admin SDK for Python and TypeScript.
- [Projects](https://confident-ai.docs.buildwithfern.com/docs/settings/project/management/projects.md): Manage Confident AI projects programmatically with the Admin SDK for Python and TypeScript.
- [Members & Invitations](https://confident-ai.docs.buildwithfern.com/docs/settings/project/management/members-and-invitations.md): Manage organization and project members and invitations programmatically with the Admin SDK for Python and TypeScript.
- [Roles, Policies & Permissions](https://confident-ai.docs.buildwithfern.com/docs/settings/project/management/roles-policies-permissions.md): Manage roles, policies, and permissions programmatically with the Admin SDK for Python and TypeScript.
- [API Keys](https://confident-ai.docs.buildwithfern.com/docs/settings/project/management/api-keys.md): Manage organization and project API keys programmatically with the Admin SDK for Python and TypeScript.
- [Self-Hosting](https://confident-ai.docs.buildwithfern.com/docs/self-hosting.md): Deploy Confident AI in your own infrastructure with self-hosted deployment options
- [Security & Compliance](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/security-and-compliance.md): Security, compliance, and data handling for self-hosted Confident AI deployments
- [Overview](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/aws/overview.md): Deploy Confident AI on Amazon Web Services (AWS)
- [Quickstart](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/aws/quickstart.md): Quick setup guide for Confident AI's AWS infrastructure and Kubernetes workloads using Terraform
- [Requirements](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/aws/requirements.md): Technologies and resource requirements for Confident AI on AWS
- [Prerequisites](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/aws/prerequisites.md): Prerequisites for deploying Confident AI on AWS
- [Configuration](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/aws/configuration.md): Configure Terraform variables for Confident AI on AWS
- [Provisioning](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/aws/provisioning.md): Provision AWS infrastructure with Terraform
- [SSL Certificates](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/aws/ssl-certificates.md): Configure SSL/TLS certificates for Confident AI
- [Cluster Access](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/aws/cluster-access.md): Configure kubectl access to the EKS cluster
- [Kubernetes Deployment](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/aws/kubernetes-deployment.md): Deploy Confident AI workloads to EKS
- [Verification](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/aws/verification.md): Verify the Confident AI deployment on AWS
- [Overview](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/azure/overview.md): Deploy Confident AI on Microsoft Azure
- [Quickstart](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/azure/quickstart.md): Quick setup guide for Confident AI's Azure infrastructure and Kubernetes workloads using Terraform
- [Requirements](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/azure/requirements.md): Technologies and resource requirements for Confident AI on Azure
- [Prerequisites](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/azure/prerequisites.md): Prerequisites for deploying Confident AI on Azure
- [Configuration](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/azure/configuration.md): Configure Terraform variables for Confident AI on Azure
- [Provisioning](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/azure/provisioning.md): Provision Azure infrastructure with Terraform
- [TLS Certificates](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/azure/tls-certificates.md): Configure TLS certificates for Confident AI on Azure
- [Cluster Access](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/azure/cluster-access.md): Configure kubectl access to the AKS cluster
- [Kubernetes Deployment](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/azure/kubernetes-deployment.md): Deploy Confident AI workloads to AKS
- [Verification](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/azure/verification.md): Verify the Confident AI deployment on Azure
- [Overview](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/gcp/overview.md): Deploy Confident AI on Google Cloud Platform (GCP)
- [Quickstart](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/gcp/quickstart.md): Quick setup guide for Confident AI's GCP infrastructure and Kubernetes workloads using Terraform
- [Requirements](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/gcp/requirements.md): Technologies and resource requirements for Confident AI on GCP
- [Prerequisites](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/gcp/prerequisites.md): Prerequisites for deploying Confident AI on GCP
- [Configuration](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/gcp/configuration.md): Configure Terraform variables for Confident AI on GCP
- [Provisioning](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/gcp/provisioning.md): Provision GCP infrastructure with Terraform
- [TLS Certificates](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/gcp/tls-certificates.md): Configure TLS certificates for Confident AI on GCP
- [Cluster Access](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/gcp/cluster-access.md): Configure kubectl access to the GKE cluster
- [Kubernetes Deployment](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/gcp/kubernetes-deployment.md): Deploy Confident AI workloads to GKE
- [Verification](https://confident-ai.docs.buildwithfern.com/docs/self-hosting/gcp/verification.md): Verify the Confident AI deployment on GCP
- [Guides](https://confident-ai.docs.buildwithfern.com/docs/guides/introduction.md): End-to-end guides that combine tracing, evaluation, and programmatic management for common Confident AI implementation workflows.
- [Provision Projects for Agents on the Fly](https://confident-ai.docs.buildwithfern.com/docs/guides/multi-tenant-project-isolation.md): Programmatically create a Confident AI project per agent with the Admin SDK, store each returned Project API Key, and route every agent's traces to the right project.

## API Docs

- API Reference > Data Models > Metrics [List Metrics](https://confident-ai.docs.buildwithfern.com/docs/api-reference/metrics/get-custom-metric.md)
- API Reference > Data Models > Metrics [Create Metrics](https://confident-ai.docs.buildwithfern.com/docs/api-reference/metrics/create-custom-metric.md)
- API Reference > Data Models > Metrics [Update Metrics](https://confident-ai.docs.buildwithfern.com/docs/api-reference/metrics/update-custom-metric.md)
- API Reference > Data Models > Metrics [Pull Metric](https://confident-ai.docs.buildwithfern.com/docs/api-reference/metrics/pull-custom-metric.md)
- API Reference > Data Models > Metrics > Batch Metrics [Batch Create](https://confident-ai.docs.buildwithfern.com/docs/api-reference/metrics/batch-metrics/create-metrics-batch.md)
- API Reference > Data Models > Metric Collections [Create Collection](https://confident-ai.docs.buildwithfern.com/docs/api-reference/metric-collections/create-metric-collection.md)
- API Reference > Data Models > Metric Collections [List Metric Collections](https://confident-ai.docs.buildwithfern.com/docs/api-reference/metric-collections/list-metric-collections.md)
- API Reference > Data Models > Metric Collections [Update Collection](https://confident-ai.docs.buildwithfern.com/docs/api-reference/metric-collections/update-metric-collection.md)
- API Reference > Data Models > Metrics Data [List Metrics Data](https://confident-ai.docs.buildwithfern.com/docs/api-reference/metrics-data/list-metrics-data.md)
- API Reference > Data Models > Datasets [List Datasets](https://confident-ai.docs.buildwithfern.com/docs/api-reference/datasets/list-datasets.md)
- API Reference > Data Models > Datasets [Pull Dataset](https://confident-ai.docs.buildwithfern.com/docs/api-reference/datasets/pull-dataset.md)
- API Reference > Data Models > Datasets [Push Dataset](https://confident-ai.docs.buildwithfern.com/docs/api-reference/datasets/push-dataset.md)
- API Reference > Data Models > Datasets [Delete Dataset](https://confident-ai.docs.buildwithfern.com/docs/api-reference/datasets/delete-dataset.md)
- API Reference > Data Models > Datasets > Dataset Versions [List Dataset Versions](https://confident-ai.docs.buildwithfern.com/docs/api-reference/datasets/dataset-versions/get-dataset-versions.md)
- API Reference > Data Models > Datasets > Dataset Versions [Create Dataset Version](https://confident-ai.docs.buildwithfern.com/docs/api-reference/datasets/dataset-versions/create-dataset-version.md)
- API Reference > Data Models > Test Runs [List Test Runs](https://confident-ai.docs.buildwithfern.com/docs/api-reference/test-runs/list-test-runs.md)
- API Reference > Data Models > Test Runs [Retrieve Test Run](https://confident-ai.docs.buildwithfern.com/docs/api-reference/test-runs/get-test-run.md)
- API Reference > Data Models > Traces [List Traces](https://confident-ai.docs.buildwithfern.com/docs/api-reference/traces/get-trace.md)
- API Reference > Data Models > Traces [Trace Ingestion](https://confident-ai.docs.buildwithfern.com/docs/api-reference/traces/create-trace.md)
- API Reference > Data Models > Traces [Retrieve Trace](https://confident-ai.docs.buildwithfern.com/docs/api-reference/traces/fetch-trace.md)
- API Reference > Data Models > Spans [List Spans](https://confident-ai.docs.buildwithfern.com/docs/api-reference/spans/get-span.md)
- API Reference > Data Models > Spans [Retrieve Span](https://confident-ai.docs.buildwithfern.com/docs/api-reference/spans/retrieve-span.md)
- API Reference > Data Models > Threads [List Threads](https://confident-ai.docs.buildwithfern.com/docs/api-reference/threads/list-threads.md)
- API Reference > Data Models > Threads [Retrieve Thread](https://confident-ai.docs.buildwithfern.com/docs/api-reference/threads/retrieve-thread.md)
- API Reference > Data Models > Prompts [List Prompts](https://confident-ai.docs.buildwithfern.com/docs/api-reference/prompts/list-prompts.md)
- API Reference > Data Models > Prompts [Push Prompts](https://confident-ai.docs.buildwithfern.com/docs/api-reference/prompts/push-prompt.md)
- API Reference > Data Models > Prompts > Labels [Pull Prompts By Label](https://confident-ai.docs.buildwithfern.com/docs/api-reference/prompts/labels/get-prompt-by-label.md)
- API Reference > Data Models > Prompts > Versions [List Versions](https://confident-ai.docs.buildwithfern.com/docs/api-reference/prompts/versions/get-prompt-versions.md)
- API Reference > Data Models > Prompts > Versions [Create Version](https://confident-ai.docs.buildwithfern.com/docs/api-reference/prompts/versions/create-version.md)
- API Reference > Data Models > Prompts > Versions [Pull Prompts By Version](https://confident-ai.docs.buildwithfern.com/docs/api-reference/prompts/versions/get-prompt-by-version.md)
- API Reference > Data Models > Prompts > Commits [List Commits](https://confident-ai.docs.buildwithfern.com/docs/api-reference/prompts/commits/get-prompt-commits.md)
- API Reference > Data Models > Prompts > Commits [Pull Prompts By Commit](https://confident-ai.docs.buildwithfern.com/docs/api-reference/prompts/commits/get-prompt-by-commit.md)
- API Reference > Data Models > Prompts > Branches [List Branches](https://confident-ai.docs.buildwithfern.com/docs/api-reference/prompts/branches/get-prompt-branches.md)
- API Reference > Data Models > Prompts > Branches [Create Branch](https://confident-ai.docs.buildwithfern.com/docs/api-reference/prompts/branches/create-prompt-branch.md)
- API Reference > Data Models > Prompts > Branches [Update Branch](https://confident-ai.docs.buildwithfern.com/docs/api-reference/prompts/branches/update-prompt-branch.md)
- API Reference > Data Models > Prompts > Branches [Delete Branch](https://confident-ai.docs.buildwithfern.com/docs/api-reference/prompts/branches/delete-prompt-branch.md)
- API Reference > Data Models > Annotations [List Annotations](https://confident-ai.docs.buildwithfern.com/docs/api-reference/annotations/list-annotations.md)
- API Reference > Data Models > Annotations [Create Annotation](https://confident-ai.docs.buildwithfern.com/docs/api-reference/annotations/create-annotation.md)
- API Reference > Data Models > Annotations [Get Annotation](https://confident-ai.docs.buildwithfern.com/docs/api-reference/annotations/get-annotation.md)
- API Reference > Data Models > Annotations [Update Annotation](https://confident-ai.docs.buildwithfern.com/docs/api-reference/annotations/update-annotation.md)
- API Reference > Data Models > Annotation Queues [List Queues](https://confident-ai.docs.buildwithfern.com/docs/api-reference/annotation-queues/list-annotation-queues.md)
- API Reference > Data Models > Annotation Queues [Create Queue](https://confident-ai.docs.buildwithfern.com/docs/api-reference/annotation-queues/create-annotation-queue.md)
- API Reference > Data Models > Annotation Queues [Get Queue](https://confident-ai.docs.buildwithfern.com/docs/api-reference/annotation-queues/get-annotation-queue.md)
- API Reference > Data Models > Annotation Queues [Delete Queue](https://confident-ai.docs.buildwithfern.com/docs/api-reference/annotation-queues/delete-annotation-queue.md)
- API Reference > Data Models > Annotation Queues > Queue Items [List Queue Items](https://confident-ai.docs.buildwithfern.com/docs/api-reference/annotation-queues/queue-items/list-queue-items.md)
- API Reference > Data Models > Annotation Queues > Queue Items [Annotate Queue Item](https://confident-ai.docs.buildwithfern.com/docs/api-reference/annotation-queues/queue-items/annotate-item.md)
- API Reference > Data Models > Projects [List Projects](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/list-projects.md)
- API Reference > Data Models > Projects [Create Project](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/create-project.md)
- API Reference > Data Models > Projects [Retrieve Project](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/get-project.md)
- API Reference > Data Models > Projects [Update Project](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/update-project.md)
- API Reference > Data Models > Projects [Delete Project](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/delete-project.md)
- API Reference > Data Models > Projects > API Keys [List Project API Keys](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/api-keys/list-project-api-keys.md)
- API Reference > Data Models > Projects > API Keys [Get Project API Key](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/api-keys/get-project-api-key.md)
- API Reference > Data Models > Projects > API Keys [Create Project API Key](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/api-keys/create-project-api-key.md)
- API Reference > Data Models > Projects > API Keys [Update Project API Key](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/api-keys/update-project-api-key.md)
- API Reference > Data Models > Projects > API Keys [Revoke Project API Key](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/api-keys/delete-project-api-key.md)
- API Reference > Data Models > Projects > Members [List Project Members](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/members/list-project-members.md)
- API Reference > Data Models > Projects > Members [Update Project Member Role](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/members/update-project-member-role.md)
- API Reference > Data Models > Projects > Members [Remove Project Member](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/members/remove-project-member.md)
- API Reference > Data Models > Projects > Invitations [List Project Invitations](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/invitations/list-project-invitations.md)
- API Reference > Data Models > Projects > Invitations [Create Project Invitations](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/invitations/create-project-invitations.md)
- API Reference > Data Models > Projects > Invitations [Resend Project Invitation](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/invitations/resend-project-invitation.md)
- API Reference > Data Models > Projects > Invitations [Revoke Project Invitation](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/invitations/delete-project-invitation.md)
- API Reference > Data Models > Projects > Roles [List Project Roles](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/roles/list-project-roles.md)
- API Reference > Data Models > Projects > Roles [Create Project Role](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/roles/create-project-role.md)
- API Reference > Data Models > Projects > Roles [Update Project Role](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/roles/update-project-role.md)
- API Reference > Data Models > Projects > Roles [Delete Project Role](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/roles/delete-project-role.md)
- API Reference > Data Models > Projects > Policies [List Project Policies](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/policies/list-project-policies.md)
- API Reference > Data Models > Projects > Policies [Create Project Policy](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/policies/create-project-policy.md)
- API Reference > Data Models > Projects > Policies [Update Project Policy](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/policies/update-project-policy.md)
- API Reference > Data Models > Projects > Policies [Delete Project Policy](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/policies/delete-project-policy.md)
- API Reference > Data Models > Projects > Permissions [List Project Permissions](https://confident-ai.docs.buildwithfern.com/docs/api-reference/projects/permissions/list-project-permissions.md)
- API Reference > Data Models > Organization [Retrieve Organization](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/get-organization.md)
- API Reference > Data Models > Organization [Update Organization](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/update-organization.md)
- API Reference > Data Models > Organization > API Keys [List Organization API Keys](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/api-keys/list-organization-api-keys.md)
- API Reference > Data Models > Organization > API Keys [Get Organization API Key](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/api-keys/get-organization-api-key.md)
- API Reference > Data Models > Organization > API Keys [Create Organization API Key](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/api-keys/create-organization-api-key.md)
- API Reference > Data Models > Organization > API Keys [Update Organization API Key](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/api-keys/update-organization-api-key.md)
- API Reference > Data Models > Organization > API Keys [Revoke Organization API Key](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/api-keys/delete-organization-api-key.md)
- API Reference > Data Models > Organization > Members [List Organization Members](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/members/list-members.md)
- API Reference > Data Models > Organization > Members [Update Organization Member Role](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/members/update-member-role.md)
- API Reference > Data Models > Organization > Members [Remove Organization Member](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/members/remove-member.md)
- API Reference > Data Models > Organization > Invitations [List Organization Invitations](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/invitations/list-invitations.md)
- API Reference > Data Models > Organization > Invitations [Create Organization Invitations](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/invitations/create-invitations.md)
- API Reference > Data Models > Organization > Invitations [Resend Organization Invitation](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/invitations/resend-invitation.md)
- API Reference > Data Models > Organization > Invitations [Revoke Organization Invitation](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/invitations/delete-invitation.md)
- API Reference > Data Models > Organization > Roles [List Organization Roles](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/roles/list-roles.md)
- API Reference > Data Models > Organization > Roles [Create Organization Role](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/roles/create-role.md)
- API Reference > Data Models > Organization > Roles [Update Organization Role](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/roles/update-role.md)
- API Reference > Data Models > Organization > Roles [Delete Organization Role](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/roles/delete-role.md)
- API Reference > Data Models > Organization > Policies [List Organization Policies](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/policies/list-policies.md)
- API Reference > Data Models > Organization > Policies [Create Organization Policy](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/policies/create-policy.md)
- API Reference > Data Models > Organization > Policies [Update Organization Policy](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/policies/update-policy.md)
- API Reference > Data Models > Organization > Policies [Delete Organization Policy](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/policies/delete-policy.md)
- API Reference > Data Models > Organization > Permissions [List Organization Permissions](https://confident-ai.docs.buildwithfern.com/docs/api-reference/organization/permissions/list-permissions.md)
- API Reference > Evals > Evaluate [Run LLM Evals](https://confident-ai.docs.buildwithfern.com/docs/api-reference/evaluate/evaluate-llm.md)
- API Reference > Evals > Evaluate [Evaluate Span](https://confident-ai.docs.buildwithfern.com/docs/api-reference/evaluate/evaluate-span.md)
- API Reference > Evals > Evaluate [Evaluate Trace](https://confident-ai.docs.buildwithfern.com/docs/api-reference/evaluate/evaluate-trace.md)
- API Reference > Evals > Evaluate [Evaluate Thread](https://confident-ai.docs.buildwithfern.com/docs/api-reference/evaluate/evaluate-thread.md)
- API Reference > Evals > Simulate [Simulate Conversation](https://confident-ai.docs.buildwithfern.com/docs/api-reference/simulate/simulate-conversation.md)
- API Reference > Legacy > Metrics > Batch Metrics [Batch Create](https://confident-ai.docs.buildwithfern.com/docs/api-reference/metrics/batch-metrics/create-batch-metrics.md)
- API Reference > Legacy > Metrics Data [List Metric Data](https://confident-ai.docs.buildwithfern.com/docs/api-reference/metrics-data/list-metric-data.md)

## OpenAPI Specification

The raw OpenAPI 3.1 specification for this API is available at:
- [OpenAPI JSON](https://confident-ai.docs.buildwithfern.com/openapi.json)
- [OpenAPI YAML](https://confident-ai.docs.buildwithfern.com/openapi.yaml)