Launch Week 02 wrapped — explore all five launches
Confident AI
Docs
Pricing
Products
LLM Evaluation
LLM Evaluation

Benchmark LLM systems with research-backed metrics.

LLM Observability
LLM Observability

Trace, monitor, and alert on production LLM systems.

AI Red Teaming
AI Red Teaming

Stress-test LLM apps against adversarial attacks.

AI Governance
AI Governance

Enforce AI standards and controls across teams.

Book a DemoSign Up
Book a DemoSign Up

July 24, 2026

  • Observability
  • Evals
  • Quality of Life

Every Version Everywhere All at Once

TGIF! Thank god it's features, here's what we shipped this week:

Your agent isn't one agent—it's dozens of variants living double lives in production. Automatic agent versioning now discovers every single one straight from your traces—no tags, no config, no archaeology. Add cost insights that name names, invites that finally exist, and filters you can actually share, and it's a big week in the agent-verse.

Changelog July 24, 2026

Added

  • Automatic Agent Versioning - We hash the LLMs and metadata in your traces to discover and version every deployment automatically, then let you compare dozens of variations on the fly. You ship, we catalog. Zero configuration, full receipts. A hash made in heaven.
  • Organization-Wide Cost Insights - One page, every project, all the spend. Finally find out which project has been eating the budget. (You know the one.) Follow the money.
  • Test Run Cost Insights - Test runs with traces now come with the bill attached. Quality on one axis, cost on the other, decisions suddenly defensible. Run the numbers.
  • Onboarding Invitations - Invite teammates during onboarding with token-based invites and an actual invitation accept page. Which, yes, we didn't have before. We're not proud either. Consider yourself invited.
  • Starter-to-Teams Upgrades - Upgrade from Starter to Teams on the fly, with your costs visible before you commit. No billing jumpscares. Dream big, Teams bigger.
  • Per-Metric Evaluation Models - Every metric can now pick its own judge. Match the model to the job, not the other way around. Here comes the judge.
  • Metric Collection Sample Rates - Evaluate a slice, not the firehose. Keep the signal, skip the bill. Please sample responsibly.
  • Shareable Filtered Views - Filters now live in the URL. Build a view, copy the link, and your teammate lands exactly where you wanted them. No more "okay now set these six filters and scroll down." URL in good hands.
  • Signals from Trace One - New projects turn on classifications and online evals automatically. Send traces, get signals. That's it. That's the setup. Love at first trace.

That's the drop for this week—see you next Friday.

Built byConfident AI