Trace-Level Detections

Per-span vulnerability findings generated during risk assessments on traced applications.

Included on the Enterprise plan. Book a demo, opens in a new tab. Not included on the Team plan. Not included on the Starter plan. Not included on the Free plan.

Overview

When your AI application is traced and linked to a risk assessment via test_case_id (or turn_id for multi-turn), Confident AI scans each span in the trace for vulnerability findings after the assessment completes. These per-span findings are called Detections.

How it works

sequenceDiagram
    participant User as You
    participant Platform as Confident AI
    participant Framework as Security Framework
    participant AI as Your AI App

    User->>Platform: Start Risk Assessment

    Platform->>Framework: Load vulnerabilities & attacks

    loop For each vulnerability and attack
        Framework-->>Platform: Generate adversarial input
        Platform->>AI: Send attack (with test_case_id)
        AI-->>Platform: Return response
        Note over AI,Platform: AI emits trace with test_case_id
        Platform->>Framework: Evaluate response (pass/fail)
    end

    Note over Platform: Scan each linked trace span for vulnerability findings
    Platform-->>User: Risk Assessment Report + Detections in trace view

The trace scan runs once the assessment finalizes. No extra configuration is required beyond linking your traces to test cases.

Prerequisites

Your AI application instrumented for tracing on Confident AI — see LLM Tracing Introduction
Each trace linked to its test case using test_case_id (single-turn) or turn_id (multi-turn) — see setup instructions below

Linking traces to test cases

When Confident AI sends an attack to your AI Connection, it includes a test_case_id in the request payload. Pass that ID into your tracing implementation so Confident AI can match the trace back to the correct test case.

Confident AI sends testCaseId (and turnId for multi-turn) automatically in the request payload. Forward it to your tracing setup.

Setup instructions and code examples:

Set the confident.trace.test_case_id attribute on your root span to link the trace.

Attribute reference and examples:

Test Case Id attribute (single-turn)
Turn Id attribute (multi-turn)

Detections

A detection is a vulnerability finding attributed to a specific span. The assessment's configured evaluation model analyzes each span's input and output, together with its position in the execution tree, to determine whether a vulnerability was introduced.

Outcomes

Outcome	Description
`materialized`	The span produced violating content and it reached the user — no downstream span caught it.
`mitigated`	The span produced violating content but a downstream span sanitized, blocked, or replaced it before the final output.
`attempted`	A clear attempt to introduce the vulnerability, but no breach occurred.

Viewing detections

Spans with detections show a shield icon in the trace tree. Click any span and open the Detections tab to see the full list of findings for that span — including outcome, vulnerability type, attack vector, and reason.

Shield icons in the trace tree and the Detections tab in the span detail panel

Span attribution

Detections are attributed to the span that introduced the vulnerability, not to parent or wrapper spans. For example, if a child LLM span generates harmful content and a parent guardrail span blocks it before output:

The child LLM span gets a mitigated detection
The parent span gets no detection

This means detections in multi-span pipelines reflect where the issue originated, not which spans happened to pass the output along.

Notes

Trace scanning runs alongside the standard pass/fail evaluation on the test case's final output. Both appear in the assessment view.
The trace scan uses the vulnerability definitions from your security framework, including any custom vulnerabilities.
Detections are generated for any traced application with traces linked via test_case_id or turn_id.

Next steps

Risk Profiles

View CVSS scores, vulnerability coverage, and exploitability breakdowns across your assessments.

LLM Tracing Introduction

Instrument your AI application for tracing on Confident AI.

Securing AI for production?Find safety gaps before users, auditors, or attackers doBook a security briefing

Last updated on July 27, 2026