Annotate Items

POSThttps://api.confident-ai.com/v2/annotation-queues/{annotationQueueId}/batch-annotate

Records annotations for several items of one queue in a single call. Each entry is applied on its own, so the response carries one result per entry and a failure on one item does not stop the rest. annotatorEmail and markAsCompleted given at the top level apply to every entry that does not set its own.

POST/v2/annotation-queues/{annotationQueueId}/batch-annotate

curl -X POST "https://api.confident-ai.com/v2/annotation-queues/{annotationQueueId}/batch-annotate" \
  -H "CONFIDENT_API_KEY: <PROJECT-API-KEY>" \
  -H "Content-Type: application/json" \
  -d '{
  "annotatorEmail": "jane@acme.com",
  "markAsCompleted": true,
  "items": [
    {
      "queueItemId": "<QUEUE-ITEM-ID>",
      "annotations": [
        {
          "rating": 1,
          "type": "FIVE_STAR_RATING",
          "name": "Helpfulness",
          "explanation": "Answered the question and cited the right source.",
          "expectedOutput": "Mount Everest is 8,848 metres tall.",
          "expectedOutcome": "The user learns how tall Mount Everest is.",
          "imagesMapping": {}
        }
      ],
      "formResponses": [
        {
          "label": "How helpful was the answer?",
          "value": "Very helpful"
        }
      ],
      "annotatorEmail": "jane@acme.com",
      "flagged": false,
      "markAsCompleted": true
    }
  ]
}'

200

{
  "success": true,
  "data": {
    "results": [
      {
        "queueItemId": "<QUEUE-ITEM-ID>",
        "success": true,
        "annotationIds": [
          "<ANNOTATION-ID>"
        ],
        "formResponseIds": [
          "<ANNOTATION-FORM-RESPONSE-ID>"
        ]
      }
    ]
  },
  "link": "https://app.confident-ai.com/project/<PROJECT-ID>/annotation-queues/<ANNOTATION-QUEUE-ID>",
  "deprecated": false
}

Headers

CONFIDENT_API_KEYstringRequired
The API key of your Confident AI project.

Path parameters

annotationQueueIdstringRequired
The id of the annotation queue.

Request body

annotatorEmailstring
The email address credited for every entry that does not name its own annotator.
markAsCompletedboolean
Whether to mark the items annotated, for every entry that does not say otherwise. Defaults to true.
itemslist of objectsRequired
The items to annotate. Each is processed on its own, so one failure does not stop the rest.
Show 6 propertiesHide 6 properties
- queueItemIdstringRequired
  The id of the queue item this entry annotates.
- annotationslist of objects
  The criteria ratings to record on the item, one entry per criterion.
  Show 7 propertiesHide 7 properties
  - ratingintegerRequired
    The rating to record: 0 or 1 for a THUMBS_RATING, 1 to 5 for a FIVE_STAR_RATING.
  - typeenum
    This is the type of annotation, which can be either thumbs rating or five star rating.
    Show 2 enum valuesHide 2 enum values
    FIVE_STAR_RATING
    THUMBS_RATING
  - namestring
    The criterion this rating is for, matching a custom annotation option in this project. Omit it to rate the built-in criterion.
  - explanationstring
    Why the rating was given.
  - expectedOutputstring
    The output the target should have produced. Only for an item holding a trace or span.
  - expectedOutcomestring
    The outcome the conversation should have reached. Only for an item holding a thread.
  - imagesMappingobject
    Images referenced by [DEEPEVAL:IMAGE:<key>] markers in the text fields, keyed by that marker's key.
- formResponseslist of objects
  The answers to the fields of the queue's annotation form. Sending them requires annotatorEmail.
  Show 2 propertiesHide 2 properties
  - labelstringRequired
    The label of the form field being answered, exactly as the form spells it.
  - valueany
    The answer, in the shape the field's type expects: a string for TEXT, a number for NUMBER or FLOAT, a boolean for BOOLEAN, one of selectOptions for SELECT, and a list of them for MULTI_SELECT.
- annotatorEmailstring
  The email address of the project member the work is credited to. Required when formResponses are sent, and what makes the annotation visible on the platform.
- flaggedboolean
  Whether to flag the item for a second opinion.
- markAsCompletedboolean
  Whether to mark the item annotated, taking it out of the pending list. Defaults to true.

Response

Annotate Items succeeded.

successboolean
Indicates if the request was successful.
dataobject
One result per entry sent to a batch annotation.
Show 1 propertyHide 1 property
- resultslist of object | object
  One result per entry sent, in the order they were sent. Read success on each to tell the two shapes apart.
  Show 2 variantsHide 2 variants
  - Batch Annotate Successobject
    An entry that was annotated.
    Show 4 propertiesHide 4 properties
    queueItemIdstring
    The id of the queue item this result is for.
    successenum
    True when the item was annotated.
    Show 1 enum valueHide 1 enum value
    true
    annotationIdslist of strings
    The ids of the annotations recorded for this item.
    formResponseIdslist of strings
    The ids of the form answers recorded for this item.
  - OR
  - Batch Annotate Failureobject
    An entry that could not be annotated.
    Show 3 propertiesHide 3 properties
    queueItemIdstring
    The id of the queue item this result is for.
    successenum
    False when the entry could not be annotated.
    Show 1 enum valueHide 1 enum value
    false
    errorstring
    Why this entry failed. The rest of the batch still applied.
linkstring
This is the URL of the resource on the Confident AI platform.
deprecatedboolean
Indicates if this endpoint is deprecated.

Built byConfident AI