Submit Result

POSThttps://api.confident-ai.com/v2/test-runs/evaluate/{testCaseId}

Submits the result for a single test case in a long-running agent evaluation. Confident AI evaluates the test case and finalizes the test run once every result has been received; a repeated submission for the same test case is ignored and reported as already_received. Long-running mode is available for single-turn AI connection evaluations only. Responds 410 when the testCaseId is unknown or its result window has closed, 404 when its test run is not in this project, 409 when the test run is no longer accepting results, and 400 when the test run is multi-turn or has no metric collection.

POST/v2/test-runs/evaluate/{testCaseId}

curl -X POST "https://api.confident-ai.com/v2/test-runs/evaluate/{testCaseId}" \
  -H "CONFIDENT_API_KEY: <PROJECT-API-KEY>" \
  -H "Content-Type: application/json" \
  -d '{
  "actualOutput": "Mount Everest is 8,848 metres tall.",
  "retrievalContext": [
    "Everest is 8,848 metres tall."
  ],
  "toolsCalled": [
    {
      "name": "get_landmark_info",
      "type": "FUNCTION",
      "description": "This tool gives information about a mountain.",
      "inputParameters": {
        "mountain": "Everest"
      },
      "output": "8,848 metres",
      "reasoning": "The user asked for the height of a mountain."
    }
  ],
  "expectedTools": [
    {
      "name": "get_landmark_info",
      "type": "FUNCTION",
      "description": "This tool gives information about a mountain.",
      "inputParameters": {
        "mountain": "Everest"
      },
      "output": "8,848 metres",
      "reasoning": "The user asked for the height of a mountain."
    }
  ],
  "tokenCost": 0.002,
  "inputTokenCount": 24,
  "outputTokenCount": 12,
  "metadata": {
    "region": "Nepal"
  }
}'

200

{
  "success": true,
  "data": {
    "testCaseId": "<TEST-CASE-ID>",
    "status": "accepted"
  },
  "deprecated": false
}

Headers

CONFIDENT_API_KEYstringRequired
The API key of your Confident AI project.

Path parameters

testCaseIdstringRequired
The test case id Confident AI sent to your AI connection as confident.testCaseId when it dispatched this golden.

Request body

actualOutputstring
The actual output produced by your agent.
retrievalContextlist of strings
The retrieval context your agent used, if any.
toolsCalledlist of objects
The tools your agent called while producing the output.
Show 6 propertiesHide 6 properties
- namestringRequired
  This is the name of the tool.
- typeenum
  The type of the tool call, either a function or an MCP tool.
  Show 2 enum valuesHide 2 enum values
  - FUNCTION
  - MCP
- descriptionstring
  This is the description of the tool.
- inputParametersobject | null
  This is the input parameters that are passed to the tool.
- outputany
  This is the output of the tool.
- reasoningstring
  This is the reasoning your LLM provided for the tool call.
expectedToolslist of objects
The tools you expected to be called for this test case.
Show 6 propertiesHide 6 properties
- namestringRequired
  This is the name of the tool.
- typeenum
  The type of the tool call, either a function or an MCP tool.
  Show 2 enum valuesHide 2 enum values
  - FUNCTION
  - MCP
- descriptionstring
  This is the description of the tool.
- inputParametersobject | null
  This is the input parameters that are passed to the tool.
- outputany
  This is the output of the tool.
- reasoningstring
  This is the reasoning your LLM provided for the tool call.
tokenCostnumber
This is the cost of the tokens used to produce the output.
inputTokenCountinteger
This is the number of input tokens passed to the LLM model.
outputTokenCountinteger
This is the number of output tokens generated by the LLM model.
metadataobject
Optional additional metadata to attach to the test case.

Response

Submit Result succeeded.

successboolean
Indicates if the request was successful.
dataobject
The recorded test case id and whether its result was accepted.
Show 2 propertiesHide 2 properties
- testCaseIdstring
  The test case id the result was recorded for.
- statusenum
  accepted when the result was queued for evaluation; already_received when a result for this test case had already been recorded and this retry was ignored.
  Show 2 enum valuesHide 2 enum values
  - accepted
  - already_received
deprecatedboolean
Indicates if this endpoint is deprecated.

Built byConfident AI