MCP Server for Coding Agents

Connect Cursor, Claude Code, Codex, and other coding agents to your project over MCP.

Overview

The Confident AI MCP server connects your coding agent to your Confident AI project over the Model Context Protocol, giving it control of your resources without leaving the editor:

Prompt versioning and evaluation datasets
Cloud evaluations, metrics, and metric collections
Production tracing and observability
Human annotations and annotation queues
Analytics dashboards
Risk assessments and governance policies

Everything the MCP server exposes is also available in the web UI — think AWS console versus AWS CLI. Same resources, different interface. If you use DeepEval, this brings the backend that already persists your evaluation results directly into Cursor, Claude Code, and Windsurf.

Want more control?

The MCP server covers the resources you reach for from an editor. For the full breadth of endpoints available, use the Confident API directly.

Prerequisites

A Confident AI account.
An MCP client that supports remote servers and OAuth — Cursor, Claude Code, Claude Desktop, Windsurf, or anything else that speaks the Model Context Protocol.

Server URLs

Confident AI hosts the MCP server for you. Pick the URL for your region:

Region	MCP server URL
US (default)	`https://mcp.confident-ai.com/mcp`
EU	`https://eu.mcp.confident-ai.com/mcp`
Self-hosted	Your deployment's own `/mcp` URL

The examples below use the US URL. Swap in the EU URL if that's your region, or your own URL if you're self-hosting.

Connect Your Client

Authentication is OAuth. The first time your client reaches the server, it opens a browser window where you sign in to Confident AI and approve the connection. Your client stores the resulting token and refreshes it on its own, so you won't have to authenticate again.

Add the following to your .cursor/mcp.json file:

{
  "mcpServers": {
    "Confident AI MCP": {
      "url": "https://mcp.confident-ai.com/mcp"
    }
  }
}

Then open Cursor Settings → MCP and hit Authenticate on the server to finish signing in through your browser.

Run the following in your terminal:

claude mcp add --transport http confident-ai https://mcp.confident-ai.com/mcp

Then run /mcp inside Claude Code and pick the server to authenticate in your browser.

Claude Desktop connects through its UI rather than a config file:

Open Settings → Connectors.
Click Add custom connector.
Give it a name (for example, Confident AI) and paste https://mcp.confident-ai.com/mcp as the URL.
Click Connect, then finish signing in through the browser window that opens.

Add the following to your Windsurf MCP configuration:

{
  "mcpServers": {
    "Confident AI MCP": {
      "serverUrl": "https://mcp.confident-ai.com/mcp"
    }
  }
}

Then refresh the MCP panel and complete the browser sign-in.

Pick a Project

Every tool takes a required project_id — except list_projects, which is how your agent discovers the ids available to it.

In practice you never type an id yourself. Ask your agent to work in a project by name, and it will call list_projects first, match the name, and reuse that id for the rest of the session:

List my Confident AI projects, then pull the latest traces from the production one.

list_projects returns each project's id, name, description, organization, and governance policy — enough for your agent to tell them apart, or to ask you which one you meant when the name is ambiguous.

Available Tools

The server exposes 75 tools across 13 areas.

Projects — 1 tool

Discover which projects this connection can act on. Start here — every other tool needs a project_id.

Tool	Description
`list_projects`	List the projects your account can reach, with ids and metadata

Prompts — 11 tools

Manage prompt templates with full version control — pull, push, version, branch, and interpolate.

Tool	Description
`pull_prompt`	Fetch a prompt by alias, version, label, or commit hash
`push_prompt`	Create or update a prompt template
`interpolate_prompt`	Locally render a prompt template by replacing placeholders with values
`create_prompt_version`	Assign a version string to a specific prompt commit
`list_prompt_versions`	List all formal versions of a prompt
`list_prompt_commits`	List the full commit history of a prompt
`list_prompts`	List all prompts in your project
`list_prompt_branches`	List all branches of a prompt
`create_prompt_branch`	Create a branch diverging from main's head commit
`update_prompt_branch`	Rename a branch (main is protected)
`delete_prompt_branch`	Delete a branch (blocked while it has open pull requests)

Datasets — 11 tools

Pull, edit, and version evaluation datasets — down to individual goldens — with immutable snapshots to pin runs to.

Tool	Description
`pull_dataset`	Fetch a dataset (single-turn or multi-turn) by alias, optionally pinned to a version
`push_dataset`	Create or update datasets by adding new goldens, optionally onto a specific version
`list_datasets`	List all datasets in your project
`delete_dataset`	Permanently delete a dataset and all of its goldens and versions
`create_dataset_version`	Snapshot the current dataset state as a new immutable version
`list_dataset_versions`	List all versions of a dataset, newest first
`create_golden`	Add a single golden to a dataset, optionally onto a specific version
`get_golden`	Fetch a single golden with all fields, custom columns, and tags
`update_golden`	Replace a golden's fields (full replacement)
`delete_golden`	Permanently delete a single golden
`queue_goldens`	Queue unfinalized goldens for annotation, creating the dataset if needed

Evaluate — 2 tools

Trigger cloud evaluations and simulate multi-turn conversations.

Tool	Description
`run_llm_evals`	Run cloud evaluations on a batch of test cases against a metric collection
`simulate_conversation`	Simulate the next turn of a multi-turn conversation from a scenario and expected outcome

Traces, threads, and spans — 9 tools

Browse, inspect, and evaluate production observability data at every level of your LLM pipeline.

Tool	Description
`list_traces`	List traces with filtering by environment, time range, and sort order
`get_trace`	Get full details of a trace, including all spans
`list_threads`	List conversation threads with filtering and pagination
`get_thread`	Get full details of a thread, including all traces and thread-level metrics
`list_spans`	List spans with filtering by type, error state, prompt version, and more
`get_span`	Get full details of a span, including I/O, cost, metrics, and annotations
`evaluate_trace`	Trigger a cloud evaluation on a trace
`evaluate_thread`	Trigger a cloud evaluation on a conversation thread
`evaluate_span`	Trigger a cloud evaluation on a span

Annotations — 4 tools

Create and manage human feedback on traces, spans, and threads.

Tool	Description
`list_annotations`	List annotations with filtering by target, type, and rating range
`get_annotation`	Get full details of an annotation
`create_annotation`	Create a thumbs or star rating on a trace, span, or thread
`update_annotation`	Update an annotation's rating, explanation, or expected output

Annotation queues — 10 tools

Organize human review work: queue traces, spans, or threads for annotation and submit the results.

Tool	Description
`list_annotation_queues`	List queues with completion statistics
`create_annotation_queue`	Create a queue for traces, spans, threads, goldens, or test runs
`get_annotation_queue`	Get a queue's statistics and per-annotator assignment breakdown
`update_annotation_queue`	Rename a queue
`delete_annotation_queue`	Delete a queue and its items (submitted annotations are kept)
`add_items_to_annotation_queue`	Queue traces, spans, or threads by reference (duplicates skipped)
`list_annotation_queue_items`	List a queue's items, oldest first, filterable by completion
`get_next_annotation_queue_item`	Fetch the next pending item with its full underlying data
`annotate_queue_item`	Submit annotations and/or custom form responses for one item
`batch_annotate_queue_items`	Annotate many items in one best-effort call

Test runs — 2 tools

Inspect past evaluation runs and their results.

Tool	Description
`list_test_runs`	List test runs with filtering by status, time range, and multi-turn type
`get_test_run`	Get full details of a test run, including per-test-case metric scores and reasoning

Metrics — 6 tools

Define and manage custom LLM-as-a-judge metrics, and read online evaluation results.

Tool	Description
`list_metrics`	List all custom metrics with their criteria and required parameters
`get_metric`	Fetch a single metric by name
`create_metric`	Create a metric from criteria or evaluation steps, with optional rubric
`batch_create_metrics`	Create multiple metrics in one call (existing names skipped)
`update_metric`	Update a metric's criteria, steps, params, or rubric
`list_metric_data`	List online evaluation results, paginated with time-range filters

Metric collections — 3 tools

Group metrics into collections — the unit a cloud evaluation runs against.

Tool	Description
`list_metric_collections`	List all metric collections, including their metrics and thresholds
`create_metric_collection`	Create a collection from existing metrics with per-metric settings
`update_metric_collection`	Rename a collection or replace its metric settings

Dashboards — 11 tools

Create and query analytics dashboards. The widget-authoring tools carry a full composition guide — chart types, data models, aggregations, dimensions, and filters — so your agent can build meaningful dashboards, and preview_widget lets it check a widget's data before saving it.

Tool	Description
`list_dashboards`	List all dashboards with widget counts
`create_dashboard`	Create a dashboard, optionally with widgets in one call (auto-laid-out)
`get_dashboard`	Fetch a dashboard with all widget definitions
`update_dashboard`	Update a dashboard's name, description, or privacy
`delete_dashboard`	Permanently delete a dashboard
`add_dashboard_widget`	Add a widget to a dashboard (auto-placed)
`update_dashboard_widget`	Replace a widget's definition (full replacement, including lines)
`delete_dashboard_widget`	Remove a widget from a dashboard
`query_dashboard`	Execute a dashboard's widgets and return their data
`query_dashboard_widget`	Execute a single widget and return its data
`preview_widget`	Execute a widget definition without saving it — iterate before committing

Risk assessments — 3 tools

Red-team your LLM application against configured frameworks. Requires the Enterprise plan.

Tool	Description
`list_risk_assessment_frameworks`	List frameworks with their risk categories and attack coverage
`run_risk_assessment_framework`	Dispatch an async red-teaming run against a prompt or AI connection
`create_risk_assessment`	Upload an externally executed red-teaming run with full results

AI connections and governance — 2 tools

Reach the LLM endpoints you've registered, and check a project against its governance policy.

Tool	Description
`list_ai_connections`	List registered LLM app endpoints, used by simulations and risk assessments
`assess_governance`	Re-run the project's governance-policy controls and report each one's status

Self-Hosted Deployments

If you run Confident AI inside your own cloud account, the MCP server ships with your deployment and talks to your instance rather than the hosted endpoint — your traces, prompts, and evaluation data never leave your infrastructure. Point your client at your deployment's /mcp URL in place of the hosted one; everything else on this page is unchanged.

Sign-in follows the same path. The MCP server advertises your own deployment's backend as its authorization server, so the OAuth flow runs entirely against your instance.

See self-hosting for how a self-hosted deployment is architected, and security and compliance for the full security model.

Next Steps

Agent Skills

Pair the MCP server with the official deepeval, confident-tracing, confident-otel, and confident-client skills so your agent knows the workflows, not just the resources.

Browse the skills

Custom Agent Skills

Serve project-specific onboarding instructions to Claude Code, Codex, Cursor, and other coding agents.

Read the guide

Scaling beyond prototype?For teams evaluating Confident AI in productionTalk to us

Last updated on September 14, 2026