Submit Test Case Result

POSThttps://api.confident-ai.com/v1/test-runs/evaluate/{testCaseId}

Submit the result for a single test case in a long-running agent evaluation. Confident AI automatically evaluates the test case and finalizes the test run once every result has been received.

Long-running mode is available for single-turn AI connection evaluations only.

POST/v1/test-runs/evaluate/{testCaseId}

curl -X POST "https://api.confident-ai.com/v1/test-runs/evaluate/{testCaseId}" \
  -H "CONFIDENT_API_KEY: <PROJECT-API-KEY>" \
  -H "Content-Type: application/json" \
  -d '{
  "actualOutput": "The capital of France is Paris."
}'

200

{
  "success": true,
  "data": {
    "testCaseId": "<TEST-CASE-ID>",
    "status": "accepted"
  }
}

Headers

CONFIDENT_API_KEYstringRequired
The API key of your Confident AI project.

Path parameters

testCaseIdstringRequired
The test case id Confident AI sent to your AI connection as confident.testCaseId when it dispatched this golden.

Request body

actualOutputstring
The actual output produced by your agent.
retrievalContextlist of strings
The retrieval context your agent used, if any.
toolsCalledlist of objects
The tools your agent called while producing the output.
Show 5 propertiesHide 5 properties
- namestringRequired
  This is the name of the tool.
- descriptionstringRequired
  This is the description of the tool.
- inputParametersobject
  This is the input parameters that are passed to the tool.
- outputstring
  This is the output of the tool.
- reasoningstring
  This is the reasoning your LLM provided for the tool call.
expectedToolslist of objects
The tools you expected to be called for this test case.
Show 5 propertiesHide 5 properties
- namestringRequired
  This is the name of the tool.
- descriptionstringRequired
  This is the description of the tool.
- inputParametersobject
  This is the input parameters that are passed to the tool.
- outputstring
  This is the output of the tool.
- reasoningstring
  This is the reasoning your LLM provided for the tool call.
metadataobject
Optional additional metadata to attach to the test case.

Response

The result was accepted for evaluation.

successboolean
A boolean indicating the success or failure of the API call.
dataobject
The recorded test case id and its status.
Show 2 propertiesHide 2 properties
- testCaseIdstring
  The test case id the result was recorded for.
- statusenum
  accepted when queued for evaluation; already_received on an idempotent retry.
  Show 2 enum valuesHide 2 enum values
  - accepted
  - already_received

Built byConfident AI