Create Collection

(v1)

POST

Creates a metric collection from the name and metricSettings you specify, and returns its id. A metric that does not exist in the project, or does not match multiTurn, rejects the whole request.

POST/v1/metric-collections

curl -X POST "https://api.confident-ai.com/v1/metric-collections" \
  -H "CONFIDENT_API_KEY: <PROJECT-API-KEY>" \
  -H "Content-Type: application/json" \
  -d '{
  "name": "Collection Name",
  "multiTurn": false,
  "metricSettings": [
    {
      "metric": {
        "name": "Answer Relevancy"
      },
      "threshold": 0.8
    }
  ]
}'

200

{
  "success": true,
  "data": {
    "id": "COLLECTION-ID"
  },
  "deprecated": false
}

Headers

CONFIDENT_API_KEYstringRequired
The API key of your Confident AI project.

Request body

namestringRequired
The name of the metric collection, which must be unique within your project.
multiTurnboolean
This is true if your metric collection is multi-turn, which contains only multi-turn metrics. It cannot be changed once the collection exists.
metricSettingslist of objects
This is the list of metric settings for the collection.
Show 8 propertiesHide 8 properties
- metricobjectRequired
  This is a metric object, which contains the metric name.
  Show 1 propertyHide 1 property
  - namestringRequired
    This is the name of the metric.
- activatedboolean
  This determines if the metric is activated. Only activated metrics are used for evaluations. Non-activated metrics are skipped.
- thresholdnumber
  This determines the threshold for the metric which determines if the metric passes or fails depending on if the metric score is equal or greater than the threshold.
- includeReasonboolean
  This determines if the reason for the metric score should be generated during evaluations.
- strictModeboolean
  This determines if the metric is in strict mode. Metrics in strict mode output a binary score of 0 or 1, indicating pass or fail, as opposed to a continuous score from 0 to 1.
- sampleRatenumber
  This determines the probability of the metric being ran for evaluation.
- evaluationModelProviderenum
  Evaluates this metric with your own model instead of the project default.
  Show 18 enum valuesHide 18 enum values
  - OPEN_AI
  - CUSTOM
  - CONFIDENT_AI
  - BEDROCK
  - ANTHROPIC
  - GEMINI
  - X_AI
  - DEEPSEEK
  - MOONSHOT_AI
  - VERTEX_AI
  - AZURE
  - MISTRAL
  - PERPLEXITY
  - OPEN_ROUTER
  - PORTKEY
  - LITE_LLM
  - TRUE_FOUNDRY
  - HUGGING_FACE
- evaluationModelNamestring
  The model to use, required whenever evaluationModelProvider is set to anything other than CONFIDENT_AI.
sampleRatenumber
The share of eligible entities the whole collection is run against, between 0 and 1. Applied on top of each metric's own sampleRate.
inputTransformerIdstring
A transformer that reshapes the payload before evaluation. Ids come from the transformers endpoint. Send null to unset it.
outputTransformerIdstring
A transformer that reshapes the result after evaluation. Send null to unset it.

Response

The id of the created metric collection.

successboolean
Indicates if the request was successful.
dataobject
The id of the affected metric collection.
Show 1 propertyHide 1 property
- idstring
  This is the id of the metric collection.
deprecatedboolean
Indicates if this endpoint is deprecated.

Built byConfident AI