List Runs

GEThttps://api.confident-ai.com/v2/test-runs

Lists the test runs in your Confident AI project, newest first by default. Filter by status, multiTurn and the start/end window, sort with sortBy and ascending, and page with page and pageSize. Requires an active trial or paid plan.

GET/v2/test-runs

curl -X GET "https://api.confident-ai.com/v2/test-runs" \
  -H "CONFIDENT_API_KEY: <PROJECT-API-KEY>"

200

{
  "success": true,
  "data": {
    "testRuns": [
      {
        "id": "<TEST-RUN-ID>",
        "createdAt": "2025-01-01T12:00:00.000Z",
        "identifier": "run-399-102",
        "status": "IN_PROGRESS",
        "multiTurn": false,
        "testsPassed": 8,
        "testsFailed": 2,
        "totalTests": 10,
        "metricsScores": [
          {
            "metric": "Answer Correctness",
            "scores": [
              0.9,
              1
            ],
            "passes": 8,
            "fails": 2,
            "errors": 0,
            "errorType": "AI_CONNECTION_ERROR"
          }
        ],
        "runDuration": 15.2,
        "evaluationCost": 0.254,
        "datasetAlias": "geography-goldens",
        "testFile": "test_geography.py",
        "summary": {
          "summaryOverview": {
            "summary": [
              "8 of 10 test cases passed."
            ],
            "actionItems": [
              "Add goldens for lesser-known peaks."
            ]
          },
          "topicSummaries": [
            {
              "topic": "Mountain heights",
              "summaryPoints": [
                {
                  "content": "Answers about mountain heights were correct and cited the retrieved context.",
                  "testCaseIds": [
                    "<TEST-CASE-ID>"
                  ],
                  "grade": 0.9
                }
              ],
              "testCaseIds": [
                "<TEST-CASE-ID>"
              ]
            }
          ]
        }
      }
    ],
    "totalTestRuns": 113,
    "page": 1,
    "pageSize": 25
  },
  "deprecated": false
}

Headers

CONFIDENT_API_KEYstringRequired
The API key of your Confident AI project.

Query parameters

pageintegerdefault: 1
The page of test runs to return. Defaults to 1.
pageSizeintegerdefault: 25
The number of test runs per page, at most 100. Defaults to 25.
startstring
Returns only test runs created at or after this ISO 8601 datetime. Defaults to 60 days ago.
endstring
Returns only test runs created at or before this ISO 8601 datetime. Defaults to now.
sortByenumdefault: createdAt
The field to sort by. Defaults to createdAt.
ascendingbooleandefault: false
This determines if the field specified in sortBy should be in ascending order. Defaults to false.
statusenum
Returns only test runs with this status.
multiTurnboolean
When true, returns only multi-turn test runs; when false, only single-turn test runs. Omit to return both.

Response

List Runs succeeded.

successboolean
Indicates if the request was successful.
dataobject
Show 4 propertiesHide 4 properties
- testRunslist of objects
  This is the page of test runs.
  Show 14 propertiesHide 14 properties
  - idstring
    This is the unique ID for the test run, generated by Confident AI and not to be confused with the identifier provided by the user.
  - createdAtstring
    The time the test run was created.
  - identifierstring | null
    The human-readable identifier you gave the test run, if any.
  - statusenum
    The status of the test run: IN_PROGRESS while test cases are still being evaluated, then COMPLETED, ERRORED or CANCELLED.
    Show 4 enum valuesHide 4 enum values
    IN_PROGRESS
    COMPLETED
    ERRORED
    CANCELLED
  - multiTurnboolean
    Whether this test run contains multi-turn test cases.
  - testsPassedinteger
    The number of test cases that passed.
  - testsFailedinteger
    The number of test cases that failed.
  - totalTestsinteger
    The total number of test cases in this test run.
  - metricsScoreslist of objects
    The aggregated metric scores across all test cases.
    Show 6 propertiesHide 6 properties
    metricstring
    This is the name of the metric.
    scoreslist of numbers
    This is an array of scores for the metric across test cases, one per test case that produced a score.
    passesinteger
    This is the number of times this metric passed the threshold.
    failsinteger
    This is the number of times this metric failed to pass the threshold.
    errorsinteger
    This is the number of times this metric errored during evaluation.
    errorTypeenum | null
    Why an evaluation errored: the AI connection or a transformer failed, the evaluation model failed, the test case lacked the parameters the metric needs, or an internal error occurred.
    Show 5 enum valuesHide 5 enum values
    AI_CONNECTION_ERROR
    TRANSFORMER_ERROR
    EVALUATION_MODEL_ERROR
    INVALID_TEST_CASE_PARAMETERS
    INTERNAL_ERROR
  - runDurationnumber
    The total duration of the test run in seconds.
  - evaluationCostnumber | null
    The cost of evaluating every test case in the test run.
  - datasetAliasstring | null
    The alias of the dataset the test run was evaluated on, if any.
  - testFilestring | null
    The test file the test run was started from, if any.
  - summaryobject | null
    The AI-generated summary of a test run, produced on the Confident AI platform. Null until it has been generated.
    Show 2 propertiesHide 2 properties
    summaryOverviewobject
    The headline findings and action items of a test run.
    Show 2 propertiesHide 2 properties
    summarylist of strings
    The headline findings across every topic.
    actionItemslist of strings
    What to change to improve the next test run.
    topicSummarieslist of objects
    The findings for each topic the test cases were grouped into.
    Show 3 propertiesHide 3 properties
    topicstring
    The topic the test cases were grouped under.
    summaryPointslist of objects
    The findings for this topic.
    Show 3 propertiesHide 3 properties
    contentstring
    One finding about the test cases in this topic.
    testCaseIdslist of strings
    The ids of the test cases this finding is drawn from.
    gradenumber
    How well the test cases behind this finding performed, from 0 to 1.
    testCaseIdslist of strings
    The ids of the test cases grouped under this topic.
- totalTestRunsinteger
  Total number of test runs matching the filters.
- pageinteger
  The page this response covers.
- pageSizeinteger
  The number of test runs per page.
deprecatedboolean
Indicates if this endpoint is deprecated.

Built byConfident AI