MC…What?!!
TGIF! Thank god it’s features, here’s what we shipped this week:
Headline first: Confident AI now has an MCP server (open-sourced on github)—plug your evals, datasets, and traces into any MCP-compatible client. Also shipping this week: automatic dataset curation from production traces, a wave of Observatory upgrades (custom column variable mapping, annotation tabs, category filters, metric columns), and PagerDuty for alerts.
Added
- MCP Server - Plug Confident AI into any MCP-compatible client. Your evals, datasets, and traces—accessible from wherever you already work. The model context protocol is served.
- Automatic Dataset Curation from Traces & Spans - The big one. Turn production traces and spans into curated datasets automatically. Your best (and worst) real-world examples, ready for eval—no manual curation required. Let your data curate itself.
- Annotation Tabs in Observatory - Annotations now live in their own tabs, so you can flip between views without losing context. We’re keeping tabs on your feedback.
- PagerDuty Integration for Alerts - Route alerts straight to PagerDuty so the right people get paged at the right time. On-call never looked so connected.
- Custom Column Variable Mapping - Map variables directly to custom columns in Observatory. Your data, your layout—no more squinting at mismatched fields. Finally, everything maps out.
- Category Filters & Metric/Annotation Column Options - Filter by category and toggle metric or annotation columns on and off. Observatory now lets you see exactly what matters—no more, no less. Filter out the noise.
Changed
- General Stability & Performance Improvements - Faster loads, fewer hiccups, smoother everything. The kind of changes you feel more than you see.
