Launch Week 02 wrapped — explore all five launches
Confident AI
Docs
Pricing
Products
LLM Evaluation
LLM Evaluation

Benchmark LLM systems with research-backed metrics.

LLM Observability
LLM Observability

Trace, monitor, and alert on production LLM systems.

AI Red Teaming
AI Red Teaming

Stress-test LLM apps against adversarial attacks.

AI Governance
AI Governance

Enforce AI standards and controls across teams.

Book a DemoSign Up
Book a DemoSign Up

September 2, 2026

  • Enterprise
  • Self-Hosting

v2.6.0

Release v2.6.0 adds a substantial validation-set and validation-run system, including new backend storage, APIs, and UI flows for classifier and metric validation. It also moves parts of validation state into PostgreSQL and introduces several operator-relevant schema and migration changes.

Highlights

  • Validation sets can now include multiple classifiers or metrics, with per-target runs and results.
  • Validation runs and validation items were moved into PostgreSQL-backed storage and new migrations were added.
  • Validation UI was expanded with set creation, item selection, run progress, and results views.
  • Metric validation now supports threshold tuning, pass/fail labeling, and per-metric reporting.

New Features

  • Validation Sets — Added ValidationSet support, including CRUD routes, item routes, set detail pages, and project-settings entry points for creating sets.
  • Validation Runs — Added validation run lifecycle support, run routes, SSE progress streaming, run selectors, and run results views.
  • Metric Validation — Added metric validation flows for pass/fail labeling, threshold tuning, per-metric results, and promoting tuned metrics into a collection.
  • Validation Labeling UI — Added validation labeling views, item transitions, confusion-matrix filtering, and target-specific validation panels in the classifier and observatory UI.

Improvements

  • PostgreSQL Backing — Validation runs, run metadata, items, and related set data were reorganized to use PostgreSQL-backed tables and relations.
  • Target Grouping — Validation set creation and results now support selecting and displaying multiple classifiers or metrics per set.
  • Results Presentation — Validation results were reorganized around verdicts, confusion matrices, target leaderboards, and agreement history.
  • Thread Token Aggregates — Thread span aggregates now include input and output token totals, and the thread drawer overview shows total tokens.

Fixes

  • Validation Write Precision — Validation writes now stamp millisecond-precision versions.
  • Validation Labeler Stability — Fixed validation labeler and trace-loading flows to avoid stale Redux state, aborted fetches, and remount-related trace loss.
  • Run and Item Handling — Validation runs now tolerate deleted runs when jobs report back, and validation item rendering restores its page data effects reliably.
  • Thread Aggregation Accuracy — Thread span aggregation now deduplicates span versions and correctly counts directly supplied span cost.

Upgrade Notes

Apply the new validation migrations, including the consolidated per-store migration set, before starting the upgraded services. Verify any scripts or integrations that read validation tables, run metadata, or target assumptions, because those schemas and relations changed in this release.

Built byConfident AI