Image Coherence
Image Coherence is a single-turn image metric to determine if images complement the text around them
Overview
The image coherence metric is a single-turn image metric that uses MLLM-as-a-judge to assess whether each image in your LLM application's output complements and enhances the text it sits alongside.
Required Parameters
These are the parameters you must supply in your test case to run evaluations for image coherence metric:
inputlist of string | MLLMImageRequired
The input you supplied to your LLM application.
actual_outputlist of string | MLLMImageRequired
The final output your LLM application generates, as the text and images it returned in order.
Metric Parameters
These are the parameters you can configure for image coherence metric when adding it to a metric collection:
max_context_sizenumber
An integer specifying the maximum number of characters of the text above and
below the image to consider. For example, 100.
How Is It Calculated?
The image coherence metric scores each image in the actual output against the text directly above and below it using an MLLM, then averages those scores across all of the images.
The final score is the average coherence score of the images found in the actual output.
Usage
To run the image coherence metric on Confident AI, add it to a single-turn metric collection. This will allow you to use image coherence metric for:
- Single-turn E2E testing
- Single-turn component-level testing
- Online and offline evals for traces and spans
Last updated on