MCP Server for Coding Agents

Connect Cursor, Claude Code, Codex, and other coding agents to your project over MCP.

The Confident AI MCP server connects your coding agent to your Confident AI project over the Model Context Protocol, giving it control of your resources without leaving the editor:

Cloud evaluations and metric collections
Evaluation datasets
Prompt versioning and management
Production tracing and observability
Human annotations and feedback

Everything the MCP server exposes is also available in the web UI — think AWS console versus AWS CLI. Same resources, different interface. If you use DeepEval, this brings the backend that already persists your evaluation results directly into Cursor, Claude Code, and Windsurf.

Want more control?

The MCP server covers the resources you reach for from an editor. For the full breadth of endpoints available, use the Evals API directly.

Prerequisites

A Confident AI API key.
An MCP-compatible client — Cursor, Claude, Windsurf, or anything else that speaks the Model Context Protocol.

Server URLs

Confident AI hosts the MCP server for you. Pick the URL for your region:

Region	MCP server URL
US (default)	`https://mcp.confident-ai.com/mcp`
EU	`https://eu.mcp.confident-ai.com/mcp`
Self-hosted	Your deployment's own `/mcp` URL

The examples below use the US URL. Swap in the EU URL if that's your region, or your own URL if you're self-hosting.

Connect your client

Authentication is a Bearer token in the Authorization header — your Confident AI API key.

Add the following to your .cursor/mcp.json file:

{
  "mcpServers": {
    "confident-ai": {
      "url": "https://mcp.confident-ai.com/mcp",
      "headers": {
        "Authorization": "Bearer <YOUR_CONFIDENT_API_KEY>"
      }
    }
  }
}

Run the following in your terminal:

claude mcp add --transport http confident-ai https://mcp.confident-ai.com/mcp --header "Authorization: Bearer <YOUR_CONFIDENT_API_KEY>"

Add the following to your claude_desktop_config.json file:

{
  "mcpServers": {
    "confident-ai": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote",
        "https://mcp.confident-ai.com/mcp",
        "--header",
        "Authorization: Bearer <YOUR_CONFIDENT_API_KEY>"
      ]
    }
  }
}

Add the following to your Windsurf MCP configuration:

{
  "mcpServers": {
    "confident-ai": {
      "serverUrl": "https://mcp.confident-ai.com/mcp",
      "headers": {
        "Authorization": "Bearer <YOUR_CONFIDENT_API_KEY>"
      }
    }
  }
}

Available tools

The server exposes 30 tools across seven areas.

Prompts — 7 tools

Manage prompt templates with full version control — pull, push, version, and interpolate.

Tool	Description
`pull_prompt`	Fetch a prompt by alias, version, label, or commit hash
`push_prompt`	Create or update a prompt template
`interpolate_prompt`	Locally render a prompt template by replacing placeholders with values
`create_prompt_version`	Assign a version string to a specific prompt commit
`list_prompt_versions`	List all formal versions of a prompt
`list_prompt_commits`	List the full commit history of a prompt
`list_prompts`	List all prompts in your project

Datasets — 5 tools

Pull evaluation datasets for local test runs or agent workflows, pinning runs to immutable snapshots of goldens.

Tool	Description
`pull_dataset`	Fetch a dataset (single-turn or multi-turn) by alias, optionally pinned to a version
`push_dataset`	Create or update datasets by adding new goldens, optionally onto a specific version
`list_datasets`	List all datasets in your project
`create_dataset_version`	Snapshot the current dataset state as a new immutable version
`list_dataset_versions`	List all versions of a dataset, newest first

Evaluate — 2 tools

Trigger cloud evaluations and simulate multi-turn conversations.

Tool	Description
`run_llm_evals`	Run cloud evaluations on a batch of test cases against a metric collection
`simulate_conversation`	Simulate the next turn of a multi-turn conversation from a scenario and expected outcome

Traces, threads, and spans — 9 tools

Browse, inspect, and evaluate production observability data at every level of your LLM pipeline.

Tool	Description
`list_traces`	List traces with filtering by environment, time range, and sort order
`get_trace`	Get full details of a trace, including all spans
`list_threads`	List conversation threads with filtering and pagination
`get_thread`	Get full details of a thread, including all traces and thread-level metrics
`list_spans`	List spans with filtering by type, error state, prompt version, and more
`get_span`	Get full details of a span, including I/O, cost, metrics, and annotations
`evaluate_trace`	Trigger a cloud evaluation on a trace
`evaluate_thread`	Trigger a cloud evaluation on a conversation thread
`evaluate_span`	Trigger a cloud evaluation on a span

Annotations — 4 tools

Create and manage human feedback on traces, spans, and threads.

Tool	Description
`list_annotations`	List annotations with filtering by target, type, and rating range
`get_annotation`	Get full details of an annotation
`create_annotation`	Create a thumbs or star rating on a trace, span, or thread
`update_annotation`	Update an annotation's rating, explanation, or expected output

Test runs — 2 tools

Inspect past evaluation runs and their results.

Tool	Description
`list_test_runs`	List test runs with filtering by status, time range, and multi-turn type
`get_test_run`	Get full details of a test run, including per-test-case metric scores and reasoning

Metric collections — 1 tool

Discover available metric collections before triggering evaluations.

Tool	Description
`list_metric_collections`	List all metric collections, including their metrics and thresholds

Self-hosted deployments

If you run Confident AI inside your own cloud account, the MCP server ships with your deployment and talks to your instance rather than the hosted endpoint — your traces, prompts, and evaluation data never leave your infrastructure. Point your client at your deployment's /mcp URL in place of the hosted one; everything else on this page is unchanged.

See self-hosting for how a self-hosted deployment is architected, and security and compliance for the full security model.

Next steps

Vibe Code Your Administration

Create projects, invite members, and provision API keys from your coding agent with the confident-client Agent Skill.

Read the guide

Custom Agent Skills

Serve project-specific onboarding instructions to Claude Code, Codex, Cursor, and other coding agents.

Read the guide

Scaling beyond prototype?For teams evaluating Confident AI in productionTalk to us

Last updated on August 7, 2026