Shared evaluation dashboards
Every DeepEval test run is automatically synced to a shared dashboard. No more exporting CSVs or pasting results in Slack — your whole team sees the same metrics in real time.
Bring your DeepEval workflows into Confident AI
DeepEval is the open-source LLM evaluation framework built and maintained by Confident AI. Run 50+ research-backed metrics in Python or TypeScript, then scale them org-wide with Confident AI's observability and collaboration.From the creators of DeepEval.
TRUSTED BY 500+ LEADING AI COMPANIES
Evals ran to date[ 0+ ]
COLLABORATION
Bye bye CSVs. Hello collaboration.
Comment & annotate results
Leave comments on individual test cases and evaluation runs. Tag teammates, flag regressions, and resolve issues without switching tools.
Version datasets
Every dataset change is tracked with full version history. Roll back bad edits, compare test case coverage across versions, and know exactly what changed between evaluation runs.
Align metrics with humans
Compare metric scores against human annotations to surface false positives and negatives. Know exactly where your evals agree with your team — and where they don't.
Regression testing
Catch quality drops before they ship. Automatically compare new runs against your last known-good baseline and surface the exact test cases that regressed.
HIPAA & SOC 2 TYPE II COMPLIANT
Our compliance standards meets the requirements of even the most regulated healthcare, insurance, and financial industries.
MULTI-DATA RESIDENCY
Store and process data in the United States of America (North Carolina) or the European Union (Frankfurt).
RBAC AND DATA MASKING
Our flexible infrastructure allows data separation between projects, custom permissions control, and masking for LLM traces.
99.9% UPTIME SLA
We offer enterprise-level guarantees for our services to ensure mission critical workflows are always accessible.
ON-PREM HOSTING
Optionally deploy Confident AI in your cloud premises, may it be AWS, Azure, or GCP, with tailored hands-on support.
INTEGRATIONSStay in your stack.
Stay in your stack.
We'll meet you there.
OTel-native, 30+ integrations, and SDKs in Python and TypeScript that feed straight into traces and evals.
COMMUNITY
The future of quality AI depends on you.
Join the largest and fastest growing community on AI evaluation.
FAQ
Have a Question?
Checkout our FAQs below, or talk to a human. They won't hallucinate.
DeepEval is an open-source evaluation framework that lets you write and run LLM evaluation tests locally in Python or TypeScript. Confident AI is the cloud platform built on top of DeepEval that adds centralized test management, observability, collaboration, and analytics so teams can scale their evaluation workflows organization-wide.
Yes. Confident AI created DeepEval and maintains it today. Confident AI open-sourced DeepEval to give the community a best-in-class LLM evaluation framework, and extends it with the enterprise features teams need to operationalize evaluations at scale.
No. Confident AI builds both — DeepEval ships with native Confident AI support, and the platform works standalone if you never write a line of DeepEval. Confident AI is also a full LLM observability platform, so you can use it to trace, monitor, and evaluate your LLM applications in one place — no more siloing evals and tracing across different tools.
Almost certainly. Observability is trivial, evals are the real challenge. Confident AI's observability is one of the best solutions for quality-driven monitoring of AI apps, and one of the cheapest on the market. [See pricing →]
Yes. DeepEval is fully open-source under the Apache 2.0 license and free to use for any purpose. Confident AI offers a free tier as well, along with paid plans for teams that need advanced features like role-based access, custom dashboards, and dedicated support.
DeepEval ships with 50+ research-backed metrics including faithfulness, answer relevancy, contextual recall, contextual precision, hallucination, bias, toxicity, and more. You can also define fully custom metrics using Python or LLM-as-a-judge approaches.
Yes. While Confident AI has first-class support for DeepEval, it also integrates with other popular tools and frameworks through its REST API and SDKs, so you can centralize results regardless of how you run your evaluations.
Install DeepEval with pip install deepeval, write your first evaluation test, and optionally connect to Confident AI by running deepeval login. You can also sign up for Confident AI directly and start using the platform without DeepEval.