Introduction to SDK Reference
Welcome to the Confident AI SDK reference.
What is the Confident AI SDK?
The Confident AI SDK is the official Python and TypeScript client for the Confident API. It wraps every endpoint in typed methods, so you work with objects in your own language instead of building requests by hand. It allows you to:
- Run evaluations, pull datasets, and version prompts from inside your application code
- Ingest LLM traces and spans, and read them back for debugging and monitoring
- Manage metrics, annotations, alerts, and the rest of your project data programmatically
- Administer your organization — projects, members, roles, and API keys
Every SDK method maps onto a documented API endpoint, so anything you can do over HTTP you can do here with autocomplete and type checking.
Prerequisites
- Python 3.9+ or Node.js 18+
- A Confident AI account. If you don't have one, sign up here.
- A project or organization API key
Install the SDK
Install the Confident AI SDK package for your language.
pip install confident-ainpm install confident-aiSet your API key
There are two kinds of API key, and which one you need depends on the resource you are reaching for. Get them from Confident AI, then set them as environment variables in the shell.
export CONFIDENT_PROJ_API_KEY="<PROJECT-API-KEY>" export CONFIDENT_ORG_API_KEY="<ORGANIZATION-API-KEY>"$env:CONFIDENT_PROJ_API_KEY = "<PROJECT-API-KEY>" $env:CONFIDENT_ORG_API_KEY = "<ORGANIZATION-API-KEY>"A project API key authenticates everything scoped to one project — datasets, prompts, traces, metrics, test runs, and the rest. An organization API key authenticates organization administration only:
organizationandprojects. Most code needs the project key alone.Create a client
One client serves every resource. It reads both environment variables when it is constructed, and each method reaches for whichever key it needs.
from confident_ai import ConfidentAI client = ConfidentAI()import { ConfidentAI } from "confident-ai"; const client = new ConfidentAI();
Resources
A resource is one area of the platform, grouped onto the client under its own name — client.datasets, client.metrics, client.traces. Each page in this section documents one of them: the methods it offers, the parameters each takes, and the types they return. Which key a resource needs follows its scope, as set out above.
Resources come in two shapes, and the shape decides how you call them — so it is worth knowing which you are looking at.
Stateless resources
Most resources are stateless. They hang off the client under a plural name, and every method is self-contained: you pass each id it needs as an argument, and nothing carries over between calls.
client.annotations.list(page=1, page_size=25)
client.metrics.get(metric_id="<METRIC-ID>")
client.traces.get(trace_uuid="<TRACE-UUID>")client.annotations.list({ page: 1, pageSize: 25 });
client.metrics.get("<METRIC-ID>");
client.traces.get("<TRACE-UUID>");Stateful resources
Datasets, prompts, and projects are also reachable statefully. Opening one under its singular name returns an object that stands for a single record: it stores that record's fields, and passes its id to every method you call on it, so you never repeat the id.
dataset = client.dataset(alias="my-dataset")
dataset.pull() # fills the object's fields from the platform
dataset.create_golden(...) # no dataset_id argument needed
dataset.push() # sends the fields the object is holdingconst dataset = client.dataset(undefined, { alias: "my-dataset" });
await dataset.pull(); // fills the object's fields from the platform
await dataset.createGolden(...); // no datasetId argument needed
await dataset.push(); // sends the fields the object is holdingThese are the three stateful resources:
| Resource | Object | Open with |
|---|---|---|
| Datasets | Dataset | client.dataset(), by id or alias |
| Prompts | Prompt | client.prompt(), by id or alias |
| Projects | Project | client.project(), by id |
These three keep their stateless clients as well, and the two are for different jobs. Use the plural client to work across records — listing them, or creating one before you have an id. Use the singular object to work within one record.
client.datasets.list() # across datasets
client.dataset(alias="my-dataset").pull() # within one datasetclient.datasets.list(); // across datasets
client.dataset(undefined, { alias: "my-dataset" }).pull(); // within one datasetAsynchronous methods
Every method is available asynchronously. In Python each one has an a_-prefixed twin you await; in TypeScript every method already returns a promise, so there is nothing separate to call.
result = await client.annotations.a_list(page=1)
await dataset.a_pull()const result = await client.annotations.list({ page: 1 });
await dataset.pull();FAQs
How is the SDK different from the Confident API?
The SDK is a typed client over the same API — same authentication, same objects, same behaviour. Reach for the SDK when you are writing Python or TypeScript, and for the API directly when you are in another language or need control over the HTTP request itself.
Which API key do I need?
The project API key, in almost every case. The organization key is only for administering the organization itself — creating and deleting projects, managing members, roles, and API keys. A client can hold both at once, and each method uses the one its resource requires.
When should I use a stateful object instead of the client?
Use an object when you are making several calls against one dataset, prompt, or project, or when a method sends stored fields rather than arguments — push is the clearest case, since it sends what the object is holding. For a single call where you already have the id, the stateless client is more direct.
Last updated on