Retrieve Evaluation Rule

GEThttps://api.confident-ai.com/v1/evaluation-rules/{evaluationRuleId}

Retrieves a single evaluation rule with its full configuration and the metric collection it runs. Requires the Starter plan or above.

GET/v1/evaluation-rules/{evaluationRuleId}

curl -X GET "https://api.confident-ai.com/v1/evaluation-rules/{evaluationRuleId}" \
  -H "CONFIDENT_API_KEY: <PROJECT-API-KEY>"

200

{
  "success": true,
  "data": {
    "evaluationRule": {
      "id": "EVALUATION-RULE-ID",
      "name": "Score production answers",
      "enabled": true,
      "sampleRate": 0.2,
      "dataModel": "TRACE",
      "metricCollection": {
        "id": "METRIC-COLLECTION-ID",
        "name": "Answer Quality",
        "multiTurn": false
      }
    }
  },
  "link": "https://app.confident-ai.com/project/<PROJECT-ID>/workflows"
}

Headers

CONFIDENT_API_KEYstringRequired
The API key of your Confident AI project.

Path parameters

evaluationRuleIdstringRequired
The unique identifier of the evaluation rule.

Response

successboolean
Indicates if the request was successful.
dataobject
Show 1 propertyHide 1 property
- evaluationRuleobject
  Show 11 propertiesHide 11 properties
  - idstring
    The unique identifier of the evaluation rule.
  - namestring
    The name of the evaluation rule.
  - dataModelenum
    What kind of item the rule evaluates.
    Show 3 enum valuesHide 3 enum values
    TRACE
    SPAN
    THREAD
  - metricCollectionobject
    The metric collection the rule runs, with any transformers attached to it.
    Show 5 propertiesHide 5 properties
    idstring
    The unique identifier of the metric collection.
    namestring
    The name of the metric collection.
    multiTurnboolean
    Whether the collection holds multi-turn metrics.
    inputTransformerobject
    The transformer applied to inputs before evaluation.
    Show 2 propertiesHide 2 properties
    idstring
    namestring
    outputTransformerobject
    The transformer applied to outputs before evaluation.
    Show 2 propertiesHide 2 properties
    idstring
    namestring
  - descriptionstring
    A note about what the rule checks.
  - enabledboolean
    Whether the rule runs. Defaults to true on create.
  - sampleRatenumber
    The fraction of matching items to evaluate. Defaults to 1 (all of them).
  - spanTypeenum
    Only evaluate spans of this type. Allowed only when dataModel is SPAN, and cleared automatically if the rule moves off SPAN.
    Show 5 enum valuesHide 5 enum values
    SPAN
    AGENT
    TOOL
    RETRIEVER
    LLM
  - filtersobject
    A set of filter groups combined by a top-level operator.
    Show 2 propertiesHide 2 properties
    operatorenum
    How filters or groups are combined.
    Show 2 enum valuesHide 2 enum values
    AND
    OR
    groupslist of objects
    The filter groups.
    Show 2 propertiesHide 2 properties
    operatorenum
    How filters or groups are combined.
    Show 2 enum valuesHide 2 enum values
    AND
    OR
    filterslist of objects
    The filter rows in this group.
    Show 4 propertiesHide 4 properties
    categorystring
    The property a filter row matches on (e.g. "Name", "User Id", "Model", "Metadata"). The set of valid values depends on the line's dataModel.
    conditionenum
    The comparison a filter row applies. Valid conditions depend on the category.
    Show 18 enum valuesHide 18 enum values
    Is
    Is not
    Is equal to
    Does not equal
    Is less than
    Is equal or less than
    Is greater than
    Is equal or greater than
    Has
    Has not
    Contains
    Contains only
    Does not contain
    Has increased by more than
    Has increased by less than
    Has decreased by more than
    Has decreased by less than
    Has changed from
    valuestring | number | list of strings
    The value to match against.
    keystring
    The property key. Auto-populated from category when omitted; required for Metadata, Metric, and Classifier filters.
  - threadTimelimitinteger
    For THREAD rules, the seconds of inactivity to wait before evaluating a thread, so an in-progress conversation is not scored halfway. Defaults to 300.
  - overwriteEvalsboolean
    Re-evaluate items that already have results for this metric collection instead of skipping them. Defaults to false.
linkstring
A link to the workflows page.

Built byConfident AI