Introducing Confident Monitor
Confident AI
Docs
Pricing
Products
LLM Evaluation
LLM Evaluation

Benchmark LLM systems with research-backed metrics.

LLM Observability
LLM Observability

Trace, monitor, and alert on production LLM systems.

AI Red Teaming
AI Red Teaming

Stress-test LLM apps against adversarial attacks.

AI Governance
AI Governance

Enforce AI standards and controls across teams.

Contact salesSign Up
Contact salesSign Up

Hallucination

Halucination is a single-turn metric to determine if your LLM is hallucinating false information.

Overview

The hallucination metric is a single-turn safety metric that uses LLM-as-a-judge to assess whether your LLM's output is truthful and free from false or hallucinated information.

Required Parameters

These are the parameters you must supply in your test case to run evaluations for hallucination metric:

inputstringRequired

The input you supplied to your LLM application.

actual_outputstringRequired

The final output your LLM application generates.

contextlist of stringRequired

A list of strings containing context that can be used to answer the input. Usually strings of documents.

How Is It Calculated?

The hallucination metric uses an LLM to identify contradictions between the actual output and the provided context, treating the context as ground truth.


Hallucination=Number of Contradicted ContextsNumber of Contexts\text{Hallucination} = \frac{\text{Number of Contradicted Contexts}}{\text{Number of Contexts}}

The final score is the proportion of contradicted contexts found in the actual output.

Usage

To run the hallucination metric on Confident AI, add it to a single-turn metric collection. This will allow you to use hallucination metric for:

  • Single-turn E2E testing
  • Single-turn component-level testing
  • Online and offline evals for traces and spans

Last updated on

Built byConfident AI