Confident AI Administration Skill

Teach your agent to administer your account with the Admin SDK — projects, members, RBAC, and API keys.

Overview

The confident-client Agent Skill teaches your coding agent how to administer your Confident AI account with the Admin SDK — the confidentai package for Python and TypeScript. Describe what you want ("create a project owned by alice@example.com", "pull the production prompt") and the agent writes and runs the correct SDK call. It ships in the confident-ai/confident-client repository — the same repo as the SDKs, so the guidance always matches the code.

The skill covers 275 operations across 35 resources: your account (organizations, projects, members and invitations, RBAC, governance policies, API keys) and the resources a project holds (prompts, datasets, traces, spans, threads, metrics, test runs, evaluations, dashboards, annotation queues, red teaming).

When It Triggers

The skill activates on prompts like:

Prompts that trigger the skill

Create a Confident AI project called "Customer Support Bot" and make alice@example.com the owner.
Invite bob@example.com to my organization with a read-only Analyst role.
Rotate the API keys on all our staging projects.
Assign our governance policy to every production project.
Pull the prompt labelled production and show me its template.
Add these five goldens to the "capitals" dataset and push it.
List the traces from production that errored in the last hour.

Installation

Works with Cursor, Claude Code, Codex, Windsurf, OpenCode, and any other Skills-compatible assistant:

npx skills add confident-ai/confident-client --skill "confident-client"

/plugin marketplace add confident-ai/confident-client
/plugin install confident-client@confident-ai-plugins
/reload-plugins

Copy the skill folder into your agent's skills directory:

git clone https://github.com/confident-ai/confident-client
cp -r confident-client/skills/confident-client .claude/skills/

Prerequisites

The Admin SDK: pip install confidentai or npm install confidentai
An Organization API Key exported as CONFIDENT_ORG_API_KEY for account work
A Project API Key exported as CONFIDENT_PROJ_API_KEY for everything inside a project

The resource decides which key a call needs, and one client holds both:

Scope	Environment variable	Reaches
Organization	`CONFIDENT_ORG_API_KEY`	`client.organization`, `client.projects`, `client.project(id)`
Project	`CONFIDENT_PROJ_API_KEY`	every other resource

Getting Started

Export your API keys

export CONFIDENT_ORG_API_KEY="confident_us_org_..."
export CONFIDENT_PROJ_API_KEY="confident_us_proj_..."

Describe what you want
The skill detects whether your project is Python or TypeScript — and stops to ask if the codebase has markers from both — then asks about consequential options you didn't specify, like whether the project should have an owner.
Prompt
```
Create a Confident AI project called "Customer Support Bot" and make alice@example.com the owner.
```

The agent writes and runs the SDK call

See what the agent runs

from confidentai import ConfidentAI

client = ConfidentAI()  # reads both key environment variables

created = client.projects.create(
    "Customer Support Bot",
    email="alice@example.com",
)
print(created.project.id)
print(created.api_key.value)  # shown only once — the skill surfaces it immediately

import { ConfidentAI } from "confidentai";

const client = new ConfidentAI(); // reads both key environment variables

const created = await client.projects.create(
  "Customer Support Bot",
  "Production support assistant",
  "alice@example.com",
);
console.log(created.project.id);
console.log(created.apiKey?.value); // shown only once — the skill surfaces it immediately

Keep going with prompts
The same flow covers members, RBAC (composed in order: permissions → policies → roles → members), governance policies, and key rotation — with the skill preferring to disable a key over deleting it when revocation might be temporary.
Prompt
```
Invite bob@example.com with a read-only Analyst role, then assign our governance policy to every production project.
```

FAQs

How is the organization API key different from the project API key?

CONFIDENT_ORG_API_KEY is organization-scoped and reaches the account — the organization itself, projects, members, roles and keys. CONFIDENT_PROJ_API_KEY is project-scoped and reaches everything inside one project: prompts, datasets, traces, metrics, test runs. A third, CONFIDENT_API_KEY, is what tracing and evals use. All three are separate variables, so they can be set at once.

Can the skill delete things? Is that safe?

It can, and deletions are irreversible — deleting a project permanently removes its datasets, prompts, traces, and evaluations. The skill's guardrails help: it asks before consequential mutations, and it prefers disabling an API key (setting valid to false) over deleting it when revocation might be temporary.

Does it work with both Python and TypeScript?

Yes — the confidentai package ships for both, and every reference in the skill carries both examples. The skill infers the language from your project's files and stops to ask when the codebase has markers from both ecosystems rather than guessing.

Can it send traces or run evals too?

The skill reads them, and can pull a trace or start an evaluation over a metric collection, because those are API calls. It does not instrument a running application or assert metrics inside a test suite. For that, use the deepeval, confident-tracing, or confident-otel skills.

Next Steps

Vibe Code Your Administration

The step-by-step walkthrough: install the SDK and skill, create a project, and onboard members entirely through prompts.

Admin SDK Quickstart

See the underlying SDK calls the skill generates, for Python and TypeScript.

Scaling beyond prototype?For teams evaluating Confident AI in productionTalk to us

Last updated on September 22, 2026