Pull Metric

(v2)

GET

Retrieves a custom metric by id so it can be run locally. The metric must have criteria or evaluation steps and at least one evaluation parameter, or be a valid DAG.

GET/v2/metrics/{metricId}

curl -X GET "https://api.confident-ai.com/v2/metrics/{metricId}" \
  -H "CONFIDENT_API_KEY: <PROJECT-API-KEY>"

200

{
  "success": true,
  "data": {
    "id": "<METRIC-ID>",
    "name": "Correctness",
    "algorithm": "DEFAULT",
    "criteria": "Determine if the actual output is correct based on the expected output.",
    "evaluationSteps": null,
    "rubric": [
      {
        "scoreRange": [
          8,
          10
        ],
        "expectedOutcome": "The answer is factually correct and complete."
      }
    ],
    "dag": {
      "nodes": {
        "root": {
          "type": "BinaryJudgementNode",
          "criteria": "Does the actual output answer the input?",
          "evaluation_params": [
            "input",
            "actual_output"
          ],
          "children": [
            "pass",
            "fail"
          ]
        },
        "pass": {
          "type": "VerdictNode",
          "verdict": true,
          "score": 10
        },
        "fail": {
          "type": "VerdictNode",
          "verdict": false,
          "score": 0
        }
      }
    },
    "questions": [
      {
        "type": "noul",
        "question": "Does the actual output answer the input?",
        "weight": 2
      }
    ],
    "promptConfig": {
      "messages": [
        {
          "role": "system",
          "content": "You grade support answers."
        },
        {
          "role": "user",
          "content": "QUESTION: {{question}}\nANSWER: {{answer}}"
        }
      ],
      "variables": {
        "question": {
          "field": "input",
          "path": [
            "messages",
            0,
            "content"
          ]
        },
        "answer": {
          "field": "output",
          "path": [
            "messages",
            -1,
            "content"
          ]
        }
      }
    },
    "multiTurn": false,
    "requiredParameters": [
      "actualOutput",
      "expectedOutput"
    ]
  },
  "deprecated": false
}

Headers

CONFIDENT_API_KEYstringRequired
The API key of your Confident AI project.

Path parameters

metricIdstringRequired
The unique id of the metric.

Response

Pull Metric succeeded.

successboolean
Indicates if the request was successful.
dataobject
A custom metric: how it scores, and which test case fields it needs.
Show 11 propertiesHide 11 properties
- idstring
  This is the unique id of the metric.
- namestring
  This is the name of the metric, unique within your project.
- algorithmenum | null
  The algorithm the metric is evaluated with. GEVAL scores against criteria or evaluation steps, DAG walks a decision graph, JEVAL asks a decision model bounded questions, PROMPT sends your own judge prompt to the evaluation model, CODE runs your own code, and DEFAULT is a built-in metric.
  Show 6 enum valuesHide 6 enum values
  - DEFAULT
  - DAG
  - GEVAL
  - JEVAL
  - CODE
  - PROMPT
- criteriastring | null
  This is the criteria the metric scores against, or null when it uses evaluation steps.
- evaluationStepsarray | null
  These are the steps the metric follows to score, or null when it uses criteria.
- rubricarray | null
  These are the score ranges that anchor how the metric scores, or null.
  Show 2 propertiesHide 2 properties
  - scoreRangelist of any
    The inclusive start and end of the score range this outcome describes, each between 0 and 10, with the start no greater than the end.
  - expectedOutcomestring
    What a response scoring in this range looks like.
- dagobject | null
  The decision graph of a DAG metric. It is validated on write, and metric references are resolved against the metrics in your project.
  Show 1 propertyHide 1 property
  - nodesobject
    The graph's nodes keyed by node id, as serialized by deepeval's DAG metric. Judgement nodes list their children by id; verdict nodes carry a verdict and a score, or point at another metric by metric_name.
- questionslist of object | object | object | null
  Show 3 variantsHide 3 variants
  - object
    Show 5 propertiesHide 5 properties
    typeenum
    Show 1 enum valueHide 1 enum value
    noul
    questionstring
    weightnumber
    trueDescriptionstring
    falseDescriptionstring
  - OR
  - object
    Show 4 propertiesHide 4 properties
    typeenum
    Show 1 enum valueHide 1 enum value
    score
    questionstring
    weightnumber
    levelslist of strings
    Between 2 and 10 ordered levels, worst first.
  - OR
  - object
    Show 4 propertiesHide 4 properties
    typeenum
    Show 1 enum valueHide 1 enum value
    choice
    questionstring
    weightnumber
    optionsobject
    Each option maps to a credit between 0 and 1, or null to mark the question not applicable when that option is likely.
- promptConfigobject | null
  The judge prompt of a PROMPT metric, and how its variables are filled. The judge scores from 0 to 10 and the score is normalized to 0 to 1.
  Show 2 propertiesHide 2 properties
  - messageslist of objects
    The judge prompt, as one or more messages.
    Show 2 propertiesHide 2 properties
    roleenum
    Who the message is from. A system message is only allowed first.
    Show 3 enum valuesHide 3 enum values
    system
    user
    assistant
    contentstring
    The message text. Use {{name}} to insert a mapped variable.
  - variablesobject
    Maps every {{name}} used in the messages to the data it is filled with.
- multiTurnboolean
  This is true when the metric evaluates conversations rather than single test cases.
- requiredParameterslist of enums
  The test case fields the metric needs to run.
  Show 14 enum valuesHide 14 enum values
  - input
  - actualOutput
  - expectedOutput
  - context
  - expectedTools
  - content
  - role
  - scenario
  - expectedOutcome
  - turns
  - toolsCalled
  - retrievalContext
  - metadata
  - tags
deprecatedboolean
Indicates if this endpoint is deprecated.

Built byConfident AI