Items

Overview

The Confident AI SDK exposes every Item method on the platform. This page documents how to call these methods in all supported languages. See the introduction to install the SDK and set your API key.

Methods

List Items

Lists the items in an annotation queue one page at a time, oldest first. Filter by status to see only what is still waiting to be annotated, only what a reviewer has set aside, or only what is done.

from confident_ai import ConfidentAI
from confident_ai.annotation_queues import QueueItemStatus

client = ConfidentAI()

result = client.annotation_queues.list_items(
    annotation_queue_id="<ANNOTATION-QUEUE-ID>",
    page=1,
    page_size=25,
    status=QueueItemStatus.IN_PROGRESS,
)

For async mode, call a_list_items and await it as shown below:

result = await client.annotation_queues.a_list_items(...)

Parameters

Parameter	Type	Description
`annotation_queue_id`	`str`	Required. The id of the annotation queue.
`page`	`Optional[int]`	The page of items to return. Defaults to 1.
`page_size`	`Optional[int]`	The number of items per page, at most 100. Defaults to 25.
`status`	`Optional[QueueItemStatus]`	Returns only items in this state. Omit to return every item whatever its state. See QueueItemStatus.

import { ConfidentAI } from "confident-ai";
import { QueueItemStatus } from "confident-ai/annotation-queues";

const client = new ConfidentAI();

const result = await client.annotationQueues.listItems(
  "<ANNOTATION-QUEUE-ID>",
  { page: 1, pageSize: 25, status: QueueItemStatus.IN_PROGRESS },
);

Parameters

Parameter	Type	Description
`annotationQueueId`	`string`	Required. The id of the annotation queue.
`page`	`number`	The page of items to return. Defaults to 1.
`pageSize`	`number`	The number of items per page, at most 100. Defaults to 25.
`status`	`QueueItemStatus`	Returns only items in this state. Omit to return every item whatever its state. See QueueItemStatus.

Returns

This method returns an object of type AnnotationQueueItemList.

Add Trace Queue Items

Adds production data to an annotation queue and returns the ids of the items created. Send the list that matches the queue's type; every id must already exist in your project, and anything already in the queue is skipped.

Traces to add to a TRACE queue.

from confident_ai import ConfidentAI
from confident_ai.annotation_queues import AddTraceQueueItemsRequest

client = ConfidentAI()

result = client.annotation_queues.add_items(
    annotation_queue_id="<ANNOTATION-QUEUE-ID>",
    items=AddTraceQueueItemsRequest(
        trace_uuids=["3f9c2a1e-5b7d-4c8e-9f01-2a3b4c5d6e7f"]
    ),
)

For async mode, call a_add_items and await it as shown below:

result = await client.annotation_queues.a_add_items(...)

Parameters

Parameter	Type	Description
`annotation_queue_id`	`str`	Required. The id of the annotation queue.
`items`	`AddQueueItemsRequest`	Required. The production data to add to the queue. Send the list that matches the queue's `type` — `traceUuids` for a `TRACE` queue, `spanUuids` for a `SPAN` queue, `threadIds` for a `THREAD` queue — and every id must already exist in your project. Items already in the queue are skipped. Pass one of AddTraceQueueItemsRequest, AddSpanQueueItemsRequest, AddThreadQueueItemsRequest. See AddQueueItemsRequest.

import { ConfidentAI } from "confident-ai";

const client = new ConfidentAI();

const result = await client.annotationQueues.addItems(
  "<ANNOTATION-QUEUE-ID>",
  {
    traceUuids: ["3f9c2a1e-5b7d-4c8e-9f01-2a3b4c5d6e7f"]
  },
);

Parameters

Parameter	Type	Description
`annotationQueueId`	`string`	Required. The id of the annotation queue.
`items`	`AddQueueItemsRequest`	Required. The production data to add to the queue. Send the list that matches the queue's `type` — `traceUuids` for a `TRACE` queue, `spanUuids` for a `SPAN` queue, `threadIds` for a `THREAD` queue — and every id must already exist in your project. Items already in the queue are skipped. Pass one of AddTraceQueueItemsRequest, AddSpanQueueItemsRequest, AddThreadQueueItemsRequest. See AddQueueItemsRequest.

Returns

This method returns an object of type AddedQueueItems.

Add Span Queue Items

Spans to add to a SPAN queue.

from confident_ai import ConfidentAI
from confident_ai.annotation_queues import AddSpanQueueItemsRequest

client = ConfidentAI()

result = client.annotation_queues.add_items(
    annotation_queue_id="<ANNOTATION-QUEUE-ID>",
    items=AddSpanQueueItemsRequest(
        span_uuids=["9d2c6f7e-1a3b-4c5d-8e9f-0a1b2c3d4e5f"]
    ),
)

For async mode, call a_add_items and await it as shown below:

result = await client.annotation_queues.a_add_items(...)

Parameters

Parameter	Type	Description
`annotation_queue_id`	`str`	Required. The id of the annotation queue.
`items`	`AddQueueItemsRequest`	Required. The production data to add to the queue. Send the list that matches the queue's `type` — `traceUuids` for a `TRACE` queue, `spanUuids` for a `SPAN` queue, `threadIds` for a `THREAD` queue — and every id must already exist in your project. Items already in the queue are skipped. Pass one of AddTraceQueueItemsRequest, AddSpanQueueItemsRequest, AddThreadQueueItemsRequest. See AddQueueItemsRequest.

import { ConfidentAI } from "confident-ai";

const client = new ConfidentAI();

const result = await client.annotationQueues.addItems(
  "<ANNOTATION-QUEUE-ID>",
  {
    spanUuids: ["9d2c6f7e-1a3b-4c5d-8e9f-0a1b2c3d4e5f"]
  },
);

Parameters

Parameter	Type	Description
`annotationQueueId`	`string`	Required. The id of the annotation queue.
`items`	`AddQueueItemsRequest`	Required. The production data to add to the queue. Send the list that matches the queue's `type` — `traceUuids` for a `TRACE` queue, `spanUuids` for a `SPAN` queue, `threadIds` for a `THREAD` queue — and every id must already exist in your project. Items already in the queue are skipped. Pass one of AddTraceQueueItemsRequest, AddSpanQueueItemsRequest, AddThreadQueueItemsRequest. See AddQueueItemsRequest.

Returns

This method returns an object of type AddedQueueItems.

Add Thread Queue Items

Threads to add to a THREAD queue.

from confident_ai import ConfidentAI
from confident_ai.annotation_queues import AddThreadQueueItemsRequest

client = ConfidentAI()

result = client.annotation_queues.add_items(
    annotation_queue_id="<ANNOTATION-QUEUE-ID>",
    items=AddThreadQueueItemsRequest(
        thread_ids=["thread-42"]
    ),
)

For async mode, call a_add_items and await it as shown below:

result = await client.annotation_queues.a_add_items(...)

Parameters

Parameter	Type	Description
`annotation_queue_id`	`str`	Required. The id of the annotation queue.
`items`	`AddQueueItemsRequest`	Required. The production data to add to the queue. Send the list that matches the queue's `type` — `traceUuids` for a `TRACE` queue, `spanUuids` for a `SPAN` queue, `threadIds` for a `THREAD` queue — and every id must already exist in your project. Items already in the queue are skipped. Pass one of AddTraceQueueItemsRequest, AddSpanQueueItemsRequest, AddThreadQueueItemsRequest. See AddQueueItemsRequest.

import { ConfidentAI } from "confident-ai";

const client = new ConfidentAI();

const result = await client.annotationQueues.addItems(
  "<ANNOTATION-QUEUE-ID>",
  {
    threadIds: ["thread-42"]
  },
);

Parameters

Parameter	Type	Description
`annotationQueueId`	`string`	Required. The id of the annotation queue.
`items`	`AddQueueItemsRequest`	Required. The production data to add to the queue. Send the list that matches the queue's `type` — `traceUuids` for a `TRACE` queue, `spanUuids` for a `SPAN` queue, `threadIds` for a `THREAD` queue — and every id must already exist in your project. Items already in the queue are skipped. Pass one of AddTraceQueueItemsRequest, AddSpanQueueItemsRequest, AddThreadQueueItemsRequest. See AddQueueItemsRequest.

Returns

This method returns an object of type AddedQueueItems.

Annotate Item

Records your team's annotation of one queue item and marks it complete, returning the ids of what was written. Send annotations for criteria ratings, formResponses for answers to the queue's annotation form, or both; answering the form requires annotatorEmail.

from confident_ai import ConfidentAI
from confident_ai.annotation_queues import QueueItemAnnotation
from confident_ai.annotation_queues import QueueItemFormResponse

client = ConfidentAI()

result = client.annotation_queues.annotate_item(
    annotation_queue_id="<ANNOTATION-QUEUE-ID>",
    queue_item_id="<QUEUE-ITEM-ID>",
    annotations=[
        QueueItemAnnotation(
            field_type="THUMBS_RATING",
            value=True,
            name="Helpfulness",
            explanation="Answered the question and cited the right source.",
            expected_output="Mount Everest is 8,848 metres tall.",
            expected_outcome="The user learns how tall Mount Everest is.",
            images_mapping={}
        )
    ],
    form_responses=[
        QueueItemFormResponse(
            label="How helpful was the answer?",
            value="Very helpful"
        )
    ],
    annotator_email="jane@acme.com",
    flagged=False,
    mark_as_completed=True,
)

For async mode, call a_annotate_item and await it as shown below:

result = await client.annotation_queues.a_annotate_item(...)

Parameters

Parameter	Type	Description
`annotation_queue_id`	`str`	Required. The id of the annotation queue the item belongs to.
`queue_item_id`	`str`	Required. The id of the queue item.
`annotations`	`Optional[List[QueueItemAnnotation]]`	The criteria ratings to record on the item, one entry per criterion. See QueueItemAnnotation.
`form_responses`	`Optional[List[QueueItemFormResponse]]`	The annotations for the fields of the queue's annotation form. Sending them requires `annotatorEmail`. See QueueItemFormResponse.
`annotator_email`	`Optional[str]`	The email address of the project member the work is credited to. Required when `formResponses` are sent, and what makes the annotation visible on the platform.
`flagged`	`Optional[bool]`	Whether to flag the item for a second opinion.
`mark_as_completed`	`Optional[bool]`	Whether to mark the item annotated, taking it out of the pending list. Defaults to true.

import { ConfidentAI } from "confident-ai";

const client = new ConfidentAI();

const result = await client.annotationQueues.annotateItem(
  "<ANNOTATION-QUEUE-ID>",
  "<QUEUE-ITEM-ID>",
  {
    annotations: [
      {
        fieldType: "THUMBS_RATING",
        value: true,
        name: "Helpfulness",
        explanation: "Answered the question and cited the right source.",
        expectedOutput: "Mount Everest is 8,848 metres tall.",
        expectedOutcome: "The user learns how tall Mount Everest is.",
        imagesMapping: {}
      }
    ],
    formResponses: [
      {
        label: "How helpful was the answer?",
        value: "Very helpful"
      }
    ],
    annotatorEmail: "jane@acme.com",
    flagged: false,
    markAsCompleted: true
  },
);

Parameters

Parameter	Type	Description
`annotationQueueId`	`string`	Required. The id of the annotation queue the item belongs to.
`queueItemId`	`string`	Required. The id of the queue item.
`annotations`	`QueueItemAnnotation[]`	The criteria ratings to record on the item, one entry per criterion. See QueueItemAnnotation.
`formResponses`	`QueueItemFormResponse[]`	The annotations for the fields of the queue's annotation form. Sending them requires `annotatorEmail`. See QueueItemFormResponse.
`annotatorEmail`	`string`	The email address of the project member the work is credited to. Required when `formResponses` are sent, and what makes the annotation visible on the platform.
`flagged`	`boolean`	Whether to flag the item for a second opinion.
`markAsCompleted`	`boolean`	Whether to mark the item annotated, taking it out of the pending list. Defaults to true.

Returns

This method returns an object of type AnnotateQueueItemResult.

Types

AddQueueItemsRequest

The production data to add to the queue. Send the list that matches the queue's type — traceUuids for a TRACE queue, spanUuids for a SPAN queue, threadIds for a THREAD queue — and every id must already exist in your project. Items already in the queue are skipped.

AddQueueItemsRequest = Union[
    AddTraceQueueItemsRequest,
    AddSpanQueueItemsRequest,
    AddThreadQueueItemsRequest,
]

type AddQueueItemsRequest =
  | AddTraceQueueItemsRequest
  | AddSpanQueueItemsRequest
  | AddThreadQueueItemsRequest;

A AddQueueItemsRequest is one of the shapes below. Send the fields of one of them, never a mix of both.

Traces to add to a TRACE queue.

class AddTraceQueueItemsRequest:
    trace_uuids: List[str] = Field(alias="traceUuids")

trace_uuidsList[str]Required

The uuids of the traces to add.

Example: ["3f9c2a1e-5b7d-4c8e-9f01-2a3b4c5d6e7f"]

interface AddTraceQueueItemsRequest {
  traceUuids: string[];
}

traceUuidsstring[]Required

The uuids of the traces to add.

Example: ["3f9c2a1e-5b7d-4c8e-9f01-2a3b4c5d6e7f"]

Spans to add to a SPAN queue.

class AddSpanQueueItemsRequest:
    span_uuids: List[str] = Field(alias="spanUuids")

span_uuidsList[str]Required

The uuids of the spans to add.

Example: ["9d2c6f7e-1a3b-4c5d-8e9f-0a1b2c3d4e5f"]

interface AddSpanQueueItemsRequest {
  spanUuids: string[];
}

spanUuidsstring[]Required

The uuids of the spans to add.

Example: ["9d2c6f7e-1a3b-4c5d-8e9f-0a1b2c3d4e5f"]

Threads to add to a THREAD queue.

class AddThreadQueueItemsRequest:
    thread_ids: List[str] = Field(alias="threadIds")

thread_idsList[str]Required

The ids of the threads to add.

Example: ["thread-42"]

interface AddThreadQueueItemsRequest {
  threadIds: string[];
}

threadIdsstring[]Required

The ids of the threads to add.

Example: ["thread-42"]

AddedQueueItems

The items created by adding production data to a queue.

class AddedQueueItems:
    ids: List[str]

idsList[str]Required

The ids of the queue items created, one per item that was not already in the queue.

Example: ["<QUEUE-ITEM-ID>"]

interface AddedQueueItems {
  ids: string[];
}

idsstring[]Required

The ids of the queue items created, one per item that was not already in the queue.

Example: ["<QUEUE-ITEM-ID>"]

AnnotateQueueItemResult

What one item's annotation wrote.

class AnnotateQueueItemResult:
    annotation_ids: List[str] = Field(alias="annotationIds")
    form_response_ids: List[str] = Field(alias="formResponseIds")

annotation_idsList[str]Required

The ids of the annotations recorded, one per criterion rated.

Example: ["<ANNOTATION-ID>"]

form_response_idsList[str]Required

The ids of the form answers recorded, one per field answered.

Example: ["<ANNOTATION-FORM-RESPONSE-ID>"]

interface AnnotateQueueItemResult {
  annotationIds: string[];
  formResponseIds: string[];
}

annotationIdsstring[]Required

The ids of the annotations recorded, one per criterion rated.

Example: ["<ANNOTATION-ID>"]

formResponseIdsstring[]Required

The ids of the form answers recorded, one per field answered.

Example: ["<ANNOTATION-FORM-RESPONSE-ID>"]

AnnotationQueueItem

One piece of production data waiting to be annotated. Retrieve the trace, span or thread it names for the payload to show your annotator.

class AnnotationQueueItem:
    id: str
    trace_uuid: Optional[str] = Field(alias="traceUuid")
    span_uuid: Optional[str] = Field(alias="spanUuid")
    thread_id: Optional[str] = Field(alias="threadId")
    test_case_id: Optional[str] = Field(alias="testCaseId")
    added_at: str = Field(alias="addedAt")
    status: QueueItemStatus
    assigned_to_email: Optional[str] = Field(alias="assignedToEmail")

idstrRequired

The id of the queue item, generated by Confident AI.

Example: "<QUEUE-ITEM-ID>"

trace_uuidOptional[str]Required

The uuid of the trace waiting to be annotated, or null for an item of another kind.

Example: "3f9c2a1e-5b7d-4c8e-9f01-2a3b4c5d6e7f"

span_uuidOptional[str]Required

The uuid of the span waiting to be annotated, or null for an item of another kind.

thread_idOptional[str]Required

The id of the thread waiting to be annotated, or null for an item of another kind.

test_case_idOptional[str]Required

The id of the test case waiting to be annotated, for a queue Confident AI created from a test run.

added_atstrRequired

When the item was added to the queue.

Example: "2025-01-15T10:30:00+00:00"

statusQueueItemStatusRequired

See QueueItemStatus.

assigned_to_emailOptional[str]Required

The email address of the reviewer this item is assigned to, or null when it is open to anyone.

Example: "jane@acme.com"

interface AnnotationQueueItem {
  id: string;
  traceUuid: string | null;
  spanUuid: string | null;
  threadId: string | null;
  testCaseId: string | null;
  addedAt: string;
  status: QueueItemStatus;
  assignedToEmail: string | null;
}

idstringRequired

The id of the queue item, generated by Confident AI.

Example: "<QUEUE-ITEM-ID>"

traceUuidstring | nullRequired

The uuid of the trace waiting to be annotated, or null for an item of another kind.

Example: "3f9c2a1e-5b7d-4c8e-9f01-2a3b4c5d6e7f"

spanUuidstring | nullRequired

The uuid of the span waiting to be annotated, or null for an item of another kind.

threadIdstring | nullRequired

The id of the thread waiting to be annotated, or null for an item of another kind.

testCaseIdstring | nullRequired

The id of the test case waiting to be annotated, for a queue Confident AI created from a test run.

addedAtstringRequired

When the item was added to the queue.

Example: "2025-01-15T10:30:00+00:00"

statusQueueItemStatusRequired

See QueueItemStatus.

assignedToEmailstring | nullRequired

The email address of the reviewer this item is assigned to, or null when it is open to anyone.

Example: "jane@acme.com"

AnnotationQueueItemList

One page of queue items, with the total across all pages.

class AnnotationQueueItemList:
    items: List[AnnotationQueueItem]
    total_annotation_queue_items: int = Field(alias="totalAnnotationQueueItems")
    page: int
    page_size: int = Field(alias="pageSize")

itemsList[AnnotationQueueItem]Required

The items for the current page, oldest first.

See AnnotationQueueItem.

total_annotation_queue_itemsintRequired

The total number of items matching the query.

Example: 40

pageintRequired

The page this response covers.

Example: 1

page_sizeintRequired

The number of items per page.

Example: 25

interface AnnotationQueueItemList {
  items: AnnotationQueueItem[];
  totalAnnotationQueueItems: number;
  page: number;
  pageSize: number;
}

itemsAnnotationQueueItem[]Required

The items for the current page, oldest first.

See AnnotationQueueItem.

totalAnnotationQueueItemsnumberRequired

The total number of items matching the query.

Example: 40

pagenumberRequired

The page this response covers.

Example: 1

pageSizenumberRequired

The number of items per page.

Example: 25

MLLMImage

An image referenced from a text field by a [DEEPEVAL:IMAGE:<key>] marker. Send either a public url or the bytes in base64.

class MLLMImage:
    url: str
    local: bool
    base64: Optional[str] = None
    filename: Optional[str] = None
    mime_type: Optional[str] = Field(default=None, alias="mimeType")
    data_base64: Optional[str] = Field(default=None, alias="dataBase64")

urlstrRequired

This is the URL of the image.

Example: "https://example.com/everest.png"

localboolRequired

This is true when the image is your local file.

Example: false

base64Optional[str]

The base64 data of the image.

Example: "iVBORw0KGgo="

filenameOptional[str]

The original file name.

Example: "everest.png"

mime_typeOptional[str]

The image's MIME type.

Example: "image/png"

data_base64Optional[str]

The image encoded as a base64 data URL.

Example: "data:image/png;base64,iVBORw0KGgo="

interface MLLMImage {
  url: string;
  local: boolean;
  base64?: string;
  filename?: string;
  mimeType?: string;
  dataBase64?: string;
}

urlstringRequired

This is the URL of the image.

Example: "https://example.com/everest.png"

localbooleanRequired

This is true when the image is your local file.

Example: false

base64string

The base64 data of the image.

Example: "iVBORw0KGgo="

filenamestring

The original file name.

Example: "everest.png"

mimeTypestring

The image's MIME type.

Example: "image/png"

dataBase64string

The image encoded as a base64 data URL.

Example: "data:image/png;base64,iVBORw0KGgo="

QueueItemAnnotation

One rating, as an annotator would leave it on the item. fieldType defaults to THUMBS_RATING.

class QueueItemAnnotation:
    field_type: Optional[Literal["THUMBS_RATING", "FIVE_STAR_RATING"]] = Field(default=None, alias="fieldType")
    value: Union[bool, int]
    name: Optional[str] = None
    explanation: Optional[str] = None
    expected_output: Optional[str] = Field(default=None, alias="expectedOutput")
    expected_outcome: Optional[str] = Field(default=None, alias="expectedOutcome")
    images_mapping: Optional[Dict[str, MLLMImage]] = Field(default=None, alias="imagesMapping")

field_typeOptional[Literal["THUMBS_RATING", "FIVE_STAR_RATING"]]

The kind of rating. Defaults to THUMBS_RATING.

valueUnion[bool, int]Required

The rating to record: true or false for a THUMBS_RATING, 1 to 5 for a FIVE_STAR_RATING.

Example: true

nameOptional[str]

A name for what this rating is for. On a queue with a form, it matches a rating field's label; on a queue without one, omit it for the default rating.

Example: "Helpfulness"

explanationOptional[str]

Why the rating was given.

Example: "Answered the question and cited the right source."

expected_outputOptional[str]

The output the target should have produced. Only for an item holding a trace or span.

Example: "Mount Everest is 8,848 metres tall."

expected_outcomeOptional[str]

The outcome the conversation should have reached. Only for an item holding a thread.

Example: "The user learns how tall Mount Everest is."

images_mappingOptional[Dict[str, MLLMImage]]

Images referenced by [DEEPEVAL:IMAGE:<key>] markers in the text fields, keyed by that marker's key.

See MLLMImage.

interface QueueItemAnnotation {
  fieldType?: "THUMBS_RATING" | "FIVE_STAR_RATING";
  value: boolean | number;
  name?: string;
  explanation?: string;
  expectedOutput?: string;
  expectedOutcome?: string;
  imagesMapping?: Record<string, MLLMImage>;
}

fieldType"THUMBS_RATING" | "FIVE_STAR_RATING"

The kind of rating. Defaults to THUMBS_RATING.

valueboolean | numberRequired

The rating to record: true or false for a THUMBS_RATING, 1 to 5 for a FIVE_STAR_RATING.

Example: true

namestring

A name for what this rating is for. On a queue with a form, it matches a rating field's label; on a queue without one, omit it for the default rating.

Example: "Helpfulness"

explanationstring

Why the rating was given.

Example: "Answered the question and cited the right source."

expectedOutputstring

The output the target should have produced. Only for an item holding a trace or span.

Example: "Mount Everest is 8,848 metres tall."

expectedOutcomestring

The outcome the conversation should have reached. Only for an item holding a thread.

Example: "The user learns how tall Mount Everest is."

imagesMappingRecord<string, MLLMImage>

Images referenced by [DEEPEVAL:IMAGE:<key>] markers in the text fields, keyed by that marker's key.

See MLLMImage.

QueueItemFormResponse

One annotation on a field of the queue's annotation form.

class QueueItemFormResponse:
    label: str
    value: Optional[Any] = None

labelstrRequired

The label of the form field being answered, exactly as the form spells it.

Example: "How helpful was the answer?"

valueOptional[Any]

The annotation value, in the shape the field's type expects: a string for TEXT, a number for NUMBER or FLOAT, a boolean for BOOLEAN, one of the options in the field's config for CHOICE, and a list of them for MULTIPLE_CHOICE.

Example: "Very helpful"

interface QueueItemFormResponse {
  label: string;
  value?: unknown;
}

labelstringRequired

The label of the form field being answered, exactly as the form spells it.

Example: "How helpful was the answer?"

valueunknown

Example: "Very helpful"

QueueItemStatus

Where an item stands in the review: IN_PROGRESS before work starts, DRAFT after work is autosaved but before it is submitted, DEFERRED once a reviewer has set it aside to come back to, and COMPLETED once it has been submitted.

class QueueItemStatus(Enum):
    IN_PROGRESS = "IN_PROGRESS"
    DRAFT = "DRAFT"
    DEFERRED = "DEFERRED"
    COMPLETED = "COMPLETED"

enum QueueItemStatus {
  IN_PROGRESS = "IN_PROGRESS",
  DRAFT = "DRAFT",
  DEFERRED = "DEFERRED",
  COMPLETED = "COMPLETED",
}

IN_PROGRESS · DRAFT · DEFERRED · COMPLETED

Building a production pipeline?Design a scalable API workflow for evals, datasets, traces, and promptsTalk to an engineer

Last updated on September 30, 2026

Built byConfident AI