Classifiers

A classifier is an LLM-driven rule that reads each ingested trace (or idle thread) and tags it with one of the labels you define — or with no label, if nothing fits. The labels it produces show up as Signals in the platform and as filterable dimensions across the Observatory and Dashboards.

The Classifiers page has two tabs: Trace Classifiers and Thread Classifiers. They behave the same way conceptually, but differ in when they run and what data they see.

How a Classifier Thinks

When a classifier runs, the underlying LLM receives:

1. The classifier’s description — what is this classifier looking for?
2. The list of labels with each label’s description — when do I assign this label?
3. The trace or thread payload — input, output, metadata, error, tags, and (for threads) the conversation turns.

The model picks one label or returns “no match.” There is no rule engine, no metadata-based pre-filtering, and no regex. Everything depends on how the descriptions read against the data.

Trace Classifiers

Trace classifiers run immediately on each ingested trace, subject to the Sample Rate.

Create a Trace Classifier

Sample Rate

The trace Sample Rate controls the probability that an incoming trace is classified. Pick a value between 0.0 and 1.0 — 0.5 means roughly half of incoming traces get classified. Sampling is shared across all enabled trace classifiers, so this is the right knob for controlling cost and overhead at high volume.

Thread Classifiers

Thread classifiers behave the same as trace classifiers, except they run on a multi-turn conversation after a period of inactivity instead of on every ingest. This lets the conversation settle before you grade things like sentiment, resolution, or escalation.

Create a Thread Classifier

The flow is identical to trace classifiers — pick Thread Classifiers in the tab switcher, then New Classifier and follow the same steps.

Sample Rate

The thread Sample Rate controls the probability that an idle thread will be classified. The same 0.0 — 1.0 semantics apply.

Time Limit

The Time Limit (in seconds) defines how long a thread must be idle before it is eligible for classification. When the next trace arrives after this idle window, the thread is classified.

For example, a time limit of 300 means a thread must sit idle for five minutes before it is eligible. Set this long enough that follow-up turns from the user have stopped arriving, but not so long that you miss the conversation entirely.

The Classifier Detail Page

Each classifier has a dedicated detail page where you edit its Name, Description, Labels, and runtime Settings. Open it by clicking a classifier from the list (or right after creating one).

Labels can be added one at a time with New Label (Name + Description) or auto-suggested in bulk via Generate Labels. Each row has its own enable toggle and an edit / delete kebab.

Generate Labels

If you don’t yet know what labels you need, Generate Labels proposes a set from your recent traces or threads. Click Configure Generation first to set the prompt and clustering parameters, then Generate Labels to run the pipeline:

1. Summarizing — the configured model summarizes a sample of your recent traces or threads using the Summary prompt.
2. Clustering — summaries are grouped into the configured Number of clusters with K-means.
3. Labeling — each cluster is turned into a candidate label (name + description) and shown on the row as Recommended.

Recommended rows show Accept (✓) and Decline (✕) actions instead of the regular kebab menu. Accepted labels become regular labels and start running on the next ingestion tick; declined labels are deleted. Re-running generation while recommendations are still pending discards the old ones first.

Auto Classify

Inside Settings, the Auto Classify toggle is separate from Enabled:

• Enabled — turns the classifier on or off entirely.
• Auto Classify — when on, the classifier may propose new labels (saved as Recommended on the labels list) when none of your existing labels fit a trace or thread. When off, the classifier can only choose from the labels you’ve already defined or return no match.

Leave Auto Classify on if you want to keep discovering edge cases, and off if you want a fixed taxonomy.

Manage Classifiers

Each row in the classifier list has a toggle to enable or disable the classifier without deleting it. A disabled classifier stops running on incoming data but keeps its configuration and any historical labels it produced.

Use the three-dot menu (⋮) on a row to Edit or Delete a classifier. Deleting a classifier removes it and all of its label definitions, but historical labels already attached to traces and threads remain on those entities for filtering.

Cost & Billing

Each classification logs usage that rolls into your project’s metered billing. See Project Settings → Data Usage under the Signals line for live usage and projected cost.