Launch Week 3: Five days of launches
Confident AI
Docs
Pricing
Products
LLM Evaluation
LLM Evaluation

Benchmark LLM systems with research-backed metrics.

LLM Observability
LLM Observability

Trace, monitor, and alert on production LLM systems.

AI Red Teaming
AI Red Teaming

Stress-test LLM apps against adversarial attacks.

AI Governance
AI Governance

Enforce AI standards and controls across teams.

Book a DemoSign Up
Book a DemoSign Up

Exact Match

Exact Match is a single-turn metric to determine if your LLM output matches the expected output exactly

Overview

The exact match metric is a single-turn metric that checks whether your LLM application's output is identical to the expected output. Unlike other metrics, it does not use an LLM for evaluation.

Required Parameters

These are the parameters you must supply in your test case to run evaluations for exact match metric:

inputstringRequired

The input you supplied to your LLM application.

actual_outputstringRequired

The final output your LLM application generates.

expected_outputstringRequired

The expected output your LLM application has to generate for a given input.

How Is It Calculated?

The exact match metric uses a deterministic approach to calculate the score by comparing the actual output against the expected output for strict equality.


Exact Match={1if actual output = expected output0otherwise\text{Exact Match} = \begin{cases} 1 & \text{if actual output = expected output} \\ 0 & \text{otherwise} \end{cases}

The final score is binary, the actual output either matches the expected output exactly or it does not.

Usage

To run the exact match metric on Confident AI, add it to a single-turn metric collection. This will allow you to use exact match metric for:

  • Single-turn E2E testing
  • Single-turn component-level testing
  • Online and offline evals for traces and spans

Last updated on

Built byConfident AI