Launch Week 3: Five days of launches
Confident AI
Docs
Pricing
Products
LLM Evaluation
LLM Evaluation

Benchmark LLM systems with research-backed metrics.

LLM Observability
LLM Observability

Trace, monitor, and alert on production LLM systems.

AI Red Teaming
AI Red Teaming

Stress-test LLM apps against adversarial attacks.

AI Governance
AI Governance

Enforce AI standards and controls across teams.

Book a DemoSign Up
Book a DemoSign Up

Role Violation

Role Violation is a single-turn safety metric to determine if your LLM output breaks its assigned character

Overview

The role violation metric is a single-turn safety metric that uses LLM-as-a-judge to assess whether your LLM application's output breaks the character or persona it was assigned.

Required Parameters

These are the parameters you must supply in your test case to run evaluations for role violation metric:

inputstringRequired

The input you supplied to your LLM application.

actual_outputstringRequired

The final output your LLM application generates.

Metric Parameters

These are the parameters you can configure for role violation metric when adding it to a metric collection:

rolestringRequired

A string specifying the expected role or character of the specialized chatbot. For example, helpful customer service agent.

How Is It Calculated?

The role violation metric first extracts the statements and behaviours in the actual output that could break the specified role using an LLM, then uses the same LLM to classify whether each one is an actual violation of that role.


Role Violation={1if no role violations are found0if any role violation is detected\text{Role Violation} = \begin{cases} 1 & \text{if no role violations are found} \\ 0 & \text{if any role violation is detected} \end{cases}

The final score is binary, a single violation anywhere in the actual output scores 0.

Usage

To run the role violation metric on Confident AI, add it to a single-turn metric collection. This will allow you to use role violation metric for:

  • Single-turn E2E testing
  • Single-turn component-level testing
  • Online and offline evals for traces and spans

Last updated on

Built byConfident AI