Introducing confident-trace: Production tracing without the evaluation framework

A smaller dependency with a clearer job

DeepEval grew around the work of evaluating AI systems. Its tracing support became a way to connect application behavior to those evaluations, and eventually to Confident AI's production monitoring too.

Those uses overlap, but they don't need to share an installation. The application handling customer requests may have a very different set of dependencies from the environment where you build datasets and run experiments.

I think our packages should reflect that. You should be able to adopt production monitoring on its own, even if you haven't written your first eval yet.

confident-trace handles instrumentation and trace export. DeepEval remains the evaluation framework. You can use both in your workflow without making your production service depend on both.

Built around OpenTelemetry

We chose OpenTelemetry as the foundation for confident-trace. It captures agent activity, model calls, tools, and your own application code as spans, then exports them over standard OTLP.

You can send those traces to Confident AI or through an OpenTelemetry Collector. If your team already uses OTel, the tracing setup fits into that infrastructure.

Both the Python and TypeScript SDKs support automatic instrumentation for supported integrations, along with ways to instrument custom code. Provider and framework dependencies are optional, so you install the ones your application needs.

There are 25 integrations across the two SDKs, including LangChain, LangGraph, Pydantic AI, Vercel AI SDK, and Mastra. The exact coverage differs by language and framework; the integration table in the repo is the best place to check your stack.

Quickstart

For Python, install confident-trace with pip install confident-trace and set CONFIDENT_API_KEY to your Confident AI project API key. Then add this at application startup, before making calls through a supported SDK:

from confident_trace import init
init()

init() instruments supported SDKs you already have installed and exports their spans to Confident AI. Keep using your existing clients. For TypeScript, see the setup instructions.

From a production issue to an eval

Here's the workflow this is meant to support. A customer reports that your agent gave an unhelpful answer. You open the trace and find that it retrieved the right information, then ignored it when composing the response.

Now you have something concrete to work with: the input, the retrieval step, and the answer the customer received. You can review the interaction in Confident AI, turn that failure into a test case, and check whether your next change fixes it.

That connection between production behavior and evaluation is still central to how we build Confident AI. Separating the SDKs makes it easier to get the production side connected without deciding how you'll run every part of the evaluation workflow first.

Which package should you use?

For a new production tracing setup, use confident-trace. Follow the Python or TypeScript setup instructions for your application and integration.

If you're using DeepEval to write and run evals, there's nothing to replace. It continues to serve that role, and its existing tracing functionality is still available. Our recommendation is specifically changing for production instrumentation.

The confident-trace repository has the SDKs, examples, and integration details. Try it with your application and let us know how it goes.