Classifiers

Overview

A classifier assigns labels to traces, spans and threads, based on a description and a set of labels you define. Where a metric scores how well your AI did, a classifier sorts what happened: which topic a conversation was about, how the user felt, or which kind of issue occurred.

Classifiers run in three places:

• Workflows — a classifier task labels traces, spans and threads automatically as they're ingested.
• Classify — label a trace, span or thread, or a selection of them, on demand from the Observatory.
• Alerts — Signal spike alerts watch a classifier's labels for sudden changes.

The labels they produce surface as Signals, as filterable dimensions across the Observatory and Dashboards, and as the problem labels Failure Modes are generated from. To run several classifiers together, group them in a classifier collection.

Seeded Classifiers

Every new project comes with classifiers already set up and running, so labels start appearing as soon as your traces do:

Each one is added to your project's default workflow, so you don't need to set up a classifier task for them. You can edit, disable or delete them like any classifier you create.

Recommended Labels

Topics, Use Cases and Issues start without labels, because the right ones depend on your app. Instead, Confident AI recommends them from your own traffic:

• Generate labels from traces runs the classifier over a sample of your recent traces or threads and suggests a label for each new pattern it finds (see Generate Labels).
• Auto classify, on by default, lets a classifier suggest a new label whenever an item doesn't fit any it has.

Recommended labels show up as Suggested on the classifier's page. Accept the ones you want to keep and decline the rest.

How a Classifier Thinks

When a classifier runs, the underlying LLM receives:

1. The classifier's description: what is this classifier looking for?
2. The list of labels with each label's description: when should this label be assigned?
3. The trace, span or thread payload: input, output, metadata, error, tags, and (for threads) the conversation turns

The model picks one label or returns "no match." There is no rule engine, no metadata-based pre-filtering, and no regex: everything depends on how the descriptions read against the data.

Labels

Each classifier has one or more labels. A label has:

• Name and Description — what the label represents, and when the model should assign it.
• Which is better? — More is better, Less is better or Neither is better. Signals use this to flag whether a spike is a concern, and Failure Modes are generated from Less is better labels. Labels on an issue classifier default to Less is better; others default to Neither is better.
• Applies to — traces, spans and threads, unless you narrow the label to fewer.

Each label has its own toggle: switched-off labels are not given to new items but stay in the classifier's history.

Next Steps

Last updated on November 1, 2026