Launch Week 02 wrapped — explore all five launches
Confident AI
Docs
Pricing
Products
LLM Evaluation
LLM Evaluation

Benchmark LLM systems with research-backed metrics.

LLM Observability
LLM Observability

Trace, monitor, and alert on production LLM systems.

AI Red Teaming
AI Red Teaming

Stress-test LLM apps against adversarial attacks.

AI Governance
AI Governance

Enforce AI standards and controls across teams.

Book a DemoSign Up
Book a DemoSign Up

August 28, 2026

  • Evals
  • Prompts
  • Red Teaming
  • Integrations

Reporting for Duty

TGIF! Thank god it's features, here's what we shipped this week:

Reports got a brain transplant. Instead of one prompt doing its best in one shot, report generation now runs a full agent loop with tools—it goes and fetches the data it needs, checks its own work, and streams the document to you in real time as it writes. Elsewhere: the Eval Gate learned to gate on risk assessments, AI connections picked up the entire prompt data model (versions, commits, branches, labels), simulation models got reliability benchmarks so you can pick the less flaky one—and now get logged on red teaming test cases—and the MCP server went from 75 tools to 219. Your agent is reporting for duty.

Changelog August 28, 2026

Added

  • Agentic Report Generation - Reports are now written by an agent instead of a single prompt. It runs a proper agent loop, calls tools to go pull the numbers it needs rather than guessing at them, and streams the document to you in real time as it composes. Better sourcing, sharper analysis, and no more staring at a spinner wondering whether it died. Reporting for duty.
  • Risk Assessments in the Eval Gate - The Eval Gate now gates on red teaming risk assessments, not just eval test runs. "Is it safe" blocks a merge the same way "is it good" already did, so a vulnerability regression gets stopped at the door instead of discovered in production. Two questions, one gate. Risk it and it won't ship.
  • Full Prompt Support for AI Connections - AI connections now speak the whole prompt data model: versions, commits, branches, and labels. Point a connection at a label and your evals follow the prompt wherever it moves, instead of pinning a version by hand every time someone edits a system prompt. Fully committed.
  • Simulation Model Reliability Benchmarks - The model settings page now shows reliability benchmarks for every simulation model, so picking one stops being a coin flip. Choose the model that actually finishes the conversation instead of the one that wanders off three turns in. Flakiness, quantified. Simulate responsibly.
  • Simulation Models on Red Teaming Test Cases - Red teaming test cases now record which simulation model ran the attack. When results move between assessments, you can finally tell whether your app changed or the attacker did. Know who was asking the questions. Prime model suspect.
  • From 75 to 219 MCP Tools - The MCP server and public API nearly tripled. New coverage spans reports and report templates, AI connections and metric collections, kicking off dataset evals, workflows, alerts, personas, governance, and most of the rest of the platform. The rule of thumb is now simple: if you can click it, your agent can call it. Every tool in the shed.

That's the drop for this week—see you next Friday.

Built byConfident AI