Create Dataset Ingestion Task

POSThttps://api.confident-ai.com/v1/datasets/{alias}/dataset-ingestion-tasks

Creates a standing rule that harvests matching production traces, spans, or threads into the dataset as goldens, starting immediately unless enabled is false.

dataModel must match the dataset: a multi-turn dataset only accepts THREAD tasks, and a single-turn dataset only accepts TRACE or SPAN. Requires the Starter plan or above.

POST/v1/datasets/{alias}/dataset-ingestion-tasks

curl -X POST "https://api.confident-ai.com/v1/datasets/{alias}/dataset-ingestion-tasks" \
  -H "CONFIDENT_API_KEY: <PROJECT-API-KEY>" \
  -H "Content-Type: application/json" \
  -d '{
  "name": "Harvest failed checkouts",
  "dataModel": "TRACE",
  "sampleRate": 0.1,
  "maxGoldens": 500,
  "includeInput": true,
  "includeActualOutput": true
}'

200

{
  "success": true,
  "data": {
    "id": "INGESTION-TASK-ID"
  },
  "link": "https://app.confident-ai.com/project/<PROJECT-ID>/workflows"
}

Headers

CONFIDENT_API_KEYstringRequired
The API key of your Confident AI project.

Path parameters

aliasstringRequired
The unique alias of the dataset to harvest into.

Request body

descriptionstring
A note about what the task harvests.
enabledboolean
Whether the task runs. Disabling it unschedules the job; goldens already created are kept.
sampleRatenumber
The fraction of matching items to ingest. Defaults to 1 (all of them).
filtersobject
A set of filter groups combined by a top-level operator.
Show 2 propertiesHide 2 properties
- operatorenumRequired
  How filters or groups are combined.
  Show 2 enum valuesHide 2 enum values
  - AND
  - OR
- groupslist of objectsRequired
  The filter groups.
  Show 2 propertiesHide 2 properties
  - operatorenumRequired
    How filters or groups are combined.
    Show 2 enum valuesHide 2 enum values
    AND
    OR
  - filterslist of objectsRequired
    The filter rows in this group.
    Show 4 propertiesHide 4 properties
    categorystringRequired
    The property a filter row matches on (e.g. "Name", "User Id", "Model", "Metadata"). The set of valid values depends on the line's dataModel.
    conditionenumRequired
    The comparison a filter row applies. Valid conditions depend on the category.
    Show 18 enum valuesHide 18 enum values
    Is
    Is not
    Is equal to
    Does not equal
    Is less than
    Is equal or less than
    Is greater than
    Is equal or greater than
    Has
    Has not
    Contains
    Contains only
    Does not contain
    Has increased by more than
    Has increased by less than
    Has decreased by more than
    Has decreased by less than
    Has changed from
    valuestring | number | list of stringsRequired
    The value to match against.
    Show 3 variantsHide 3 variants
    string
    OR
    number
    OR
    list of strings
    keystring
    The property key. Auto-populated from category when omitted; required for Metadata, Metric, and Classifier filters.
maxGoldensinteger
The maximum number of goldens this task will ever create. Send null to remove the cap.
inputTransformerIdstring
A transformer that reshapes the harvested input before it is stored. Send null to detach it.
outputTransformerIdstring
A transformer that reshapes the harvested output before it is stored. Send null to detach it.
includeInputboolean
Populate the golden's input from the harvested item. Defaults to true; every other include flag defaults to false.
includeActualOutputboolean
Populate the golden's actualOutput.
includeExpectedOutputboolean
Populate the golden's expectedOutput.
includeRetrievalContextboolean
Populate the golden's retrievalContext.
includeContextboolean
Populate the golden's context.
includeToolsCalledboolean
Populate the golden's toolsCalled.
includeExpectedToolsboolean
Populate the golden's expectedTools.
namestringRequired
A name for the task, unique within the dataset.
dataModelenumRequired
What kind of item to harvest. Must match the dataset — THREAD for multi-turn datasets, TRACE or SPAN for single-turn ones.
Show 3 enum valuesHide 3 enum values
- TRACE
- SPAN
- THREAD

Response

successboolean
Indicates if the ingestion task was successfully created.
dataobject
Show 1 propertyHide 1 property
- idstring
  The unique identifier of the created ingestion task.
linkstring
A link to the workflows page.

Built byConfident AI