Classifier Collections

Overview

A classifier collection on Confident AI is a named group of classifiers that you run together. Wherever you'd pick classifiers one by one, you can pick a collection instead:

• A classifier task in a workflow, which classifies traces, spans or threads as they're ingested
• Classify on traces, spans and threads, which classifies existing data on demand
• A Signal spike alert, which watches the collection's classifiers for spikes in their labels

A collection holds references to classifiers, not copies. Each classifier is still created and edited from the Classifiers page, and a change to it applies everywhere it runs, in every collection that includes it.

Why Classifier Collection?

Classifier collections solve key challenges when classifying data at scale:

• Reusable configurations — Define a set of classifiers once and reuse it across workflows, on-demand classification, and alerts
• One place to change — Add or remove a classifier from the collection, and every classifier task and alert that uses it picks up the change
• No-code management — Create and update collections entirely through the UI without touching any code
• Consistent classification — Ensure all workflows label data with the same classifiers

Create a Classifier Collection

Classifier collections are listed under Evals > Classifier collections in the left navigation. Click + next to the heading (or Create collection when there are none yet) to open New collection:

1. Enter a Collection name, which must be unique within the project
2. Pick the Classifiers it runs
3. Click Create collection

The collection opens on its own page.

You can also start from the Classifiers page: select the classifiers you want, then click Create collection in the bar that appears. New collection opens with those classifiers already picked.

Edit a Classifier Collection

A collection's page shows its name as the heading, the Classifiers it runs, and its Sample Rate. Each field saves on its own:

• Name — click the heading to rename the collection
• Add classifiers — opens a dialog where you pick classifiers to add or unpick ones to drop, then Save
• Remove — in a classifier's ⋯ menu, takes it out of the collection. The classifier itself stays in the library
• Edit classifier — in the same menu, opens the classifier on the Classifiers page

Expand a classifier's row to see its first few labels. The row also shows how many labels it has, and whether it Includes audio or is Disabled.

Use a Classifier Collection

Anywhere you choose classifiers, the selector has a Classifiers tab and a Collections tab. Pick a collection from Collections instead of individual classifiers; View collection opens it for a look (or an edit) without leaving the page.

• Workflows — in a classifier task, choose the collection under Classifiers. The task runs the collection's classifiers as they are at the time it runs, so it follows any later edits to the collection
• Classify — on a trace, span or thread (or a selection of them in the Observatory), choose the collection under Select classifiers to classify them now
• Alerts — in a Signal spike alert, choose the collection under Classifiers. The alert watches the collection's current classifiers, so editing the collection changes what it watches

Sample Rate

The Sample Rate is the probability (0–1) that the collection runs on each trace, span or thread when a classifier task runs it. It defaults to 1, so every item the task selects is classified. The decision is deterministic, so the same trace, span or thread always makes the same choice.

It compounds with the classifier task's own sample rate: the collection's classifiers run at task rate × collection rate. For example, a task at 0.5 running a collection at 0.4 classifies roughly 20% of matching items.

Delete a Classifier Collection

Delete a collection from the ⋯ menu on its row under Classifier collections in the left navigation. The classifier tasks that run it are deleted along with it, so those workflows stop classifying with it. Its classifiers stay in the library. This can't be undone.

Understanding Classifier Collections

Classifiers and collections are connected many-to-many: a collection runs several classifiers, and a classifier can sit in several collections. Classifier tasks and alerts point at the collection, never at a copy of its classifiers.

graph TD
    A[Classifier Task] --> C[Classifier Collection 1]
    B[Signal Spike Alert] --> C
    E[Classifier Task] --> D[Classifier Collection 2]

    C --> F[Classifier]
    C --> G[Classifier]
    D --> G
    D --> H[Classifier]

    style A fill:#fff3e0,color:#1e293b
    style B fill:#fff3e0,color:#1e293b
    style E fill:#fff3e0,color:#1e293b
    style C fill:#e1f5fe,color:#1e293b
    style D fill:#e1f5fe,color:#1e293b
    style F fill:#f3e5f5,color:#1e293b
    style G fill:#f3e5f5,color:#1e293b
    style H fill:#f3e5f5,color:#1e293b

When a classifier task or alert runs, Confident AI looks up the collection's classifiers at that moment and runs (or watches) those, each with its own labels and settings.

Last updated on November 1, 2026