List Metric Versions

(v2)

GET

Lists a custom metric's definition history, newest first, so the first entry is the definition new evaluations run. Versions are append-only: updating a metric with a changed definition adds one, and an update that changes nothing adds none.

GET/v2/metrics/{metricId}/versions

curl -X GET "https://api.confident-ai.com/v2/metrics/{metricId}/versions" \
  -H "CONFIDENT_API_KEY: <PROJECT-API-KEY>"

200

{
  "success": true,
  "data": {
    "versions": [
      {
        "id": "<METRIC-VERSION-ID>",
        "version": "00.00.02",
        "sequence": 2,
        "createdAt": "2025-01-18T16:45:00.000Z",
        "algorithm": "DEFAULT",
        "criteria": "Determine if the actual output is correct based on the expected output.",
        "evaluationSteps": null,
        "rubric": [
          {
            "scoreRange": [
              8,
              10
            ],
            "expectedOutcome": "The answer is factually correct and complete."
          }
        ],
        "dag": {
          "nodes": {
            "root": {
              "type": "BinaryJudgementNode",
              "criteria": "Does the actual output answer the input?",
              "evaluation_params": [
                "input",
                "actual_output"
              ],
              "children": [
                "pass",
                "fail"
              ]
            },
            "pass": {
              "type": "VerdictNode",
              "verdict": true,
              "score": 10
            },
            "fail": {
              "type": "VerdictNode",
              "verdict": false,
              "score": 0
            }
          }
        },
        "questions": [
          {
            "type": "noul",
            "question": "Does the actual output answer the input?",
            "weight": 2
          }
        ],
        "promptConfig": {
          "messages": [
            {
              "role": "system",
              "content": "You grade support answers."
            },
            {
              "role": "user",
              "content": "QUESTION: {{question}}\nANSWER: {{answer}}"
            }
          ],
          "variables": {
            "question": {
              "field": "input",
              "path": [
                "messages",
                0,
                "content"
              ]
            },
            "answer": {
              "field": "output",
              "path": [
                "messages",
                -1,
                "content"
              ]
            }
          }
        },
        "requireInput": false,
        "requireActualOutput": false,
        "requireExpectedOutput": false,
        "requireContext": false,
        "requireExpectedTools": false,
        "requireContent": false,
        "requireRole": false,
        "requireScenario": false,
        "requireExpectedOutcome": false,
        "requireAudio": false,
        "requireRetrievalContext": false,
        "requireToolsCalled": false,
        "requireMetadata": false,
        "requireTags": false,
        "codeDefinition": {
          "language": "PYTHON",
          "code": "def measure(test_case): return 1.0"
        }
      }
    ],
    "totalMetricVersions": 2,
    "page": 1,
    "pageSize": 25
  },
  "deprecated": false
}

Headers

CONFIDENT_API_KEYstringRequired
The API key of your Confident AI project.

Path parameters

metricIdstringRequired
The unique id of the metric.

Query parameters

pageintegerdefault: 1
The page to return. Defaults to 1.
pageSizeintegerdefault: 25
The number of versions per page, at most 100. Defaults to 25.

Response

List Metric Versions succeeded.

successboolean
Indicates if the request was successful.
dataobject
One page of a metric's definition history, with the total across all pages.
Show 4 propertiesHide 4 properties
- versionslist of objects
  The metric's definition history for the current page, newest version first, so the first entry of the first page is the current definition.
  Show 26 propertiesHide 26 properties
  - idstring
    This is the unique id of the version.
  - versionstring
    This is the human-readable label for sequence, written as three two-digit groups that roll over at 100, so sequence 2 is 00.00.02 and sequence 100 is 00.01.00.
  - sequenceinteger
    This is the position of the version in the metric's history, counting from 1. The highest sequence is the definition new evaluations run.
  - createdAtstring
    This is when the version was saved.
  - algorithmenum | null
    The algorithm the metric is evaluated with. GEVAL scores against criteria or evaluation steps, DAG walks a decision graph, JEVAL asks a decision model bounded questions, PROMPT sends your own judge prompt to the evaluation model, CODE runs your own code, and DEFAULT is a built-in metric.
    Show 6 enum valuesHide 6 enum values
    DEFAULT
    DAG
    GEVAL
    JEVAL
    CODE
    PROMPT
  - criteriastring | null
    This is the criteria the version scores against, or null when it uses evaluation steps.
  - evaluationStepsarray | null
    These are the steps the version follows to score, or null when it uses criteria.
  - rubricarray | null
    These are the score ranges that anchor how the version scores, or null.
    Show 2 propertiesHide 2 properties
    scoreRangelist of any
    The inclusive start and end of the score range this outcome describes, each between 0 and 10, with the start no greater than the end.
    expectedOutcomestring
    What a response scoring in this range looks like.
  - dagobject | null
    The decision graph of a DAG metric. It is validated on write, and metric references are resolved against the metrics in your project.
    Show 1 propertyHide 1 property
    nodesobject
    The graph's nodes keyed by node id, as serialized by deepeval's DAG metric. Judgement nodes list their children by id; verdict nodes carry a verdict and a score, or point at another metric by metric_name.
  - questionslist of object | object | object | null
    Show 3 variantsHide 3 variants
    Jev Noul Questionobject
    A question the decision model answers true or false. trueDescription and falseDescription tell it what each answer means.
    Show 5 propertiesHide 5 properties
    typeenum
    Marks this as a noul question.
    Show 1 enum valueHide 1 enum value
    noul
    questionstring
    The question the decision model answers.
    weightnumber
    How heavily this question counts towards the metric's score, relative to the metric's other questions.
    trueDescriptionstring
    What a true answer to this question means.
    falseDescriptionstring
    What a false answer to this question means.
    OR
    Jev Score Questionobject
    A question the decision model answers by picking one of levels, which are ordered worst first.
    Show 4 propertiesHide 4 properties
    typeenum
    Marks this as a score question.
    Show 1 enum valueHide 1 enum value
    score
    questionstring
    The question the decision model answers.
    weightnumber
    How heavily this question counts towards the metric's score, relative to the metric's other questions.
    levelslist of strings
    Between 2 and 10 ordered levels, worst first.
    OR
    Jev Choice Questionobject
    A question the decision model answers by picking one of options, each carrying the credit that answer earns.
    Show 4 propertiesHide 4 properties
    typeenum
    Marks this as a choice question.
    Show 1 enum valueHide 1 enum value
    choice
    questionstring
    The question the decision model answers.
    weightnumber
    How heavily this question counts towards the metric's score, relative to the metric's other questions.
    optionsobject
    Each option maps to a credit between 0 and 1, or null to mark the question not applicable when that option is likely.
  - promptConfigobject | null
    The judge prompt of a PROMPT metric, and how its variables are filled. The judge scores from 0 to 10 and the score is normalized to 0 to 1.
    Show 2 propertiesHide 2 properties
    messageslist of objects
    The judge prompt, as one or more messages.
    Show 2 propertiesHide 2 properties
    roleenum
    Who the message is from. A system message is only allowed first.
    Show 3 enum valuesHide 3 enum values
    system
    user
    assistant
    contentstring
    The message text. Use {{name}} to insert a mapped variable.
    variablesobject
    Maps every {{name}} used in the messages to the data it is filled with.
  - requireInputboolean
    This is true when the metric needs the test case's input.
  - requireActualOutputboolean
    This is true when the metric needs the test case's actualOutput.
  - requireExpectedOutputboolean
    This is true when the metric needs the test case's expectedOutput.
  - requireContextboolean
    This is true when the metric needs the test case's context.
  - requireExpectedToolsboolean
    This is true when the metric needs the test case's expectedTools.
  - requireContentboolean
    This is true when the metric needs the test case's content.
  - requireRoleboolean
    This is true when the metric needs the test case's role.
  - requireScenarioboolean
    This is true when the metric needs the test case's scenario.
  - requireExpectedOutcomeboolean
    This is true when the metric needs the test case's expectedOutcome.
  - requireAudioboolean
    This is true when the metric needs the test case's audio.
  - requireRetrievalContextboolean
    This is true when the metric needs the test case's retrievalContext.
  - requireToolsCalledboolean
    This is true when the metric needs the test case's toolsCalled.
  - requireMetadataboolean
    This is true when the metric needs the test case's metadata.
  - requireTagsboolean
    This is true when the metric needs the test case's tags.
  - codeDefinitionobject | null
    The code a CODE metric runs to score a test case.
    Show 2 propertiesHide 2 properties
    languageenum
    This is the language the code is written in.
    Show 1 enum valueHide 1 enum value
    PYTHON
    codestring
    This is the source code the metric runs.
- totalMetricVersionsinteger
  The number of versions this metric has, across every page.
- pageinteger
  The page this response covers.
- pageSizeinteger
  The number of versions per page.
deprecatedboolean
Indicates if this endpoint is deprecated.

Built byConfident AI