September 11, 2026

Bye Bye DeepEval, Hello OTel

TGIF! Thank god it's features, here's what we shipped this week:

Is DeepEval going away? NO! But did we just release something that lets you trace 25+ integrations in two lines of code, OTel-native from the very first span? YES! Meet Confident Trace: two lines in, every span checks in. Case traced.

Added

• Confident Trace - Meet Confident Trace, our new open-source, OTel-first tracing SDK for AI systems. Two lines of code unlock automatic instrumentation across 25+ agent frameworks, model providers, and LLM gateways in Python and TypeScript, covering agent runs, model calls, tools, retrieval, and application code. It exports over standard OTLP to Confident AI or your own OpenTelemetry Collector, so your traces are yours from the first span. Open standards in, portable telemetry out.
• Native GenAI & GCP OpenTelemetry Ingestion - A brand-new OTel server now ingests the OpenTelemetry GenAI namespace and GCP semantic convention namespaces natively. Send standards-based telemetry directly from your existing instrumentation without translating every attribute into a proprietary schema first. Speak OTel, and Confident speaks it back.
• Confident Tracing & Confident OTel Skills - Coding agents can now use dedicated confident-tracing and confident-otel skills to instrument applications, configure exporters, and troubleshoot telemetry with the right Confident and OpenTelemetry conventions built in. Less tab-hopping through docs, more traces arriving where they should. Your agent has the skills to pay the telemetry bills.
• Code Scanning for GitHub & GitLab Eval Gates - Eval Gates now scan code changes in GitHub and GitLab as part of the evaluation workflow. Catch AI quality regressions where they start—the diff—before they clear the gate and reach production. Review the code, run the evals, block the merge. Scan first, ship second.
• Flaky Metric Detection - Metrics can now be identified as flaky, making inconsistent evaluation behavior visible instead of quietly turning your test run into a coin toss. Separate real regressions from unstable judges and focus fixes where they actually belong. Flakes belong in breakfast, not your metrics.
• Multi-Reviewer Custom Form Responses - Annotation queue custom forms now accept responses from multiple users on the same item. Collect independent judgments without reviewers overwriting one another, and keep every response attached to the work it evaluated. More reviewers, more signal, one queue.
• Metric Alignment in Annotate - Metric alignment is now a first-class workflow under Annotate. Compare automated metric judgments with human feedback where annotation already happens, so calibrating an evaluator no longer requires stitching together separate tools and exports. Humans and metrics, finally on the same page.
• Error Analysis in Annotate - Error analysis is now a first-class workflow under Annotate, complete with a history view of every previous run. Revisit earlier analyses, compare findings over time, and keep the investigation trail alongside the examples your team is reviewing. Every error has a past; now you can see it.
• Bring Your Own SMTP for On-Prem - Enterprise self-hosted deployments can now connect their own SMTP server for platform email. Route notifications through infrastructure your security and operations teams already control, without depending on an external mail provider. Your deployment, your network, your mail.

That's the drop for this week—see you next Friday.