Open Inference

Overview

OpenInference is an open standard for capturing and storing AI model inferences. Confident AI allows you to trace and evaluate any application instrumented with OpenInference in just a few lines of code.

Tracing Quickstart

1. Install Dependencies

   Run the following command to install the required packages:

   pip install -U deepeval opentelemetry-sdk opentelemetry-exporter-otlp-proto-http

   npm install deepeval

2. Instrument OpenInference

   Call instrument_openinference once at startup, before your agent runs. It attaches to the active OpenTelemetry TracerProvider and begins forwarding spans to Confident AI.

   PythonTypeScript

   import os
   from openinference.instrumentation.langchain import LangChainInstrumentor
   from langchain_openai import ChatOpenAI
   from deepeval.integrations.openinference import instrument_openinference
   
   LangChainInstrumentor().instrument()
   
   instrument_openinference()
   
   llm = ChatOpenAI(model="gpt-4o-mini")
   result = llm.invoke("What are LLMs?")

3. Run your code

   Get your traces by running your code as shown here:

   python main.py

   npx tsx file-name.ts

   You can directly view the traces on Confident AI's observatory page

Advanced Usage

Logging prompts

If you are managing prompts on Confident AI and wish to log them, pass your Prompt object to instrument_openinference.

import os
from deepeval.prompt import Prompt
from langchain_openai import ChatOpenAI
from openinference.instrumentation.langchain import LangChainInstrumentor
from deepeval.integrations.openinference import instrument_openinference

prompt = Prompt(alias="my-prompt")
prompt.pull(version="00.00.01")

system_prompt = prompt.interpolate()

LangChainInstrumentor().instrument()

instrument_openinference()

llm = ChatOpenAI(model="gpt-4o-mini")
result = llm.invoke(system_prompt)

import { instrumentOpenInference } from "deepeval/integrations/openinference";
import { Prompt } from "deepeval";

const prompt = new Prompt({ alias: "my-prompt" });
await prompt.pull();

instrumentOpenInference({
  prompt: prompt
});

Logging threads

Threads are used to group related traces together, and are useful for chat apps, agents, or any multi-turn interactions. You can learn more about threads here. Pass the thread_id to instrument_openinference.

import os
from openinference.instrumentation.langchain import LangChainInstrumentor
from langchain_openai import ChatOpenAI
from deepeval.integrations.openinference import instrument_openinference

LangChainInstrumentor().instrument()

instrument_openinference(
    thread_id="thread_1",
    user_id="user_1"
)

llm = ChatOpenAI(model="gpt-4o-mini")
result = llm.invoke("What are LLMs?")

import { instrumentOpenInference } from "deepeval/integrations/openinference";

instrumentOpenInference({
  threadId: "thread_123",
  userId: "user_456",
});

Trace attributes

Other trace attributes can also be passed to instrument_openinference.

import os
from openinference.instrumentation.langchain import LangChainInstrumentor
from langchain_openai import ChatOpenAI
from deepeval.integrations.openinference import instrument_openinference

LangChainInstrumentor().instrument()

instrument_openinference(
    name="Name of Trace",
    tags=["Tag 1", "Tag 2"],
    metadata={"Key": "Value"},
    user_id="user_1",
    thread_id="conversation-abc123",
    test_case_id="test-case-001",
    turn_id="turn-1",
    environment="production",
)

llm = ChatOpenAI(model="gpt-4o-mini")
result = llm.invoke("What are LLMs?")

import { instrumentOpenInference } from "deepeval/integrations/openinference";

instrumentOpenInference({
  name: "Custom Trace Name",
  threadId: "thread_123",
  userId: "user_456",
  testCaseId: "test-case-001",
  turnId: "turn-1",
  tags: ["production", "v1"],
  metadata: { "internal_version": "1.0.2" },
  environment: "production"
});

Evals Usage

Online evals

You can run online evals on your OpenInference instrumentation, which will run evaluations on all incoming traces on Confident AI's servers.

1. Create metric collection

   Create a metric collection on Confident AI with the metrics you wish to use to evaluate your traces.

2. Run evals

   Pass the metric_collection name (Python) or metricCollection / traceMetricCollection name (TypeScript) to instrument_openinference to enable online evaluations at the trace level.

   TraceLLM SpanTool Span

   import os
   from openinference.instrumentation.langchain import LangChainInstrumentor
   from langchain_openai import ChatOpenAI
   from deepeval.integrations.openinference import instrument_openinference
   
   LangChainInstrumentor().instrument()
   
   instrument_openinference(
       name="openinference_application",
       metric_collection="TRACE-METRIC-COLLECTION"
   )
   
   llm = ChatOpenAI(model="gpt-4o-mini")
   result = llm.invoke("What are LLMs?")

   import { instrumentOpenInference } from "deepeval/integrations/openinference";
   
   instrumentOpenInference({
     name: "Custom Trace Name",
     traceMetricCollection: "TRACE-METRIC-COLLECTION"
   });

You can view evals on Confident AI by clicking on the link in the output printed in the console.

Last updated on July 27, 2026