Open Inference

Overview

OpenInference is an open standard for capturing AI model inferences as OpenTelemetry spans, with instrumentors for dozens of frameworks and providers. Confident AI lets you trace and evaluate any application instrumented with OpenInference in just a few lines of code, using confident-trace, Confident AI's OpenTelemetry-native tracing SDK.

The division of labour is simple: OpenInference creates the spans, and confident-trace exports them. The instrumentor for your framework patches the SDK and emits spans through the global OpenTelemetry provider; confident-trace installs that provider and the export pipeline, and forwards the spans unchanged to the Observatory.

Auto-Instrument

1. Install Dependencies

   Install confident-trace plus the OpenInference instrumentor for your framework or provider.

   PythonTypeScript

   This example uses the LangChain instrumentor:

   pip install confident-trace openinference-instrumentation-langchain 'langchain>=1,<2' 'langchain-openai>=1,<2'

2. Set Your API Keys

   Get your project API key from Confident AI and set it as an environment variable, along with the provider key your app uses:

   export CONFIDENT_API_KEY="<your-confident-project-key>"
   export OPENAI_API_KEY="<your-openai-key>"

3. Instrument OpenInference

   Call init() once at startup with an empty instrumentation list, then enable your OpenInference instrumentor. The empty list matters: confident-trace would otherwise also instrument OpenAI, LangChain, and friends itself, and every model call would show up twice — once from OpenInference and once from Confident's own integration.

   PythonTypeScript

   init() installs the global provider and Confident's export pipeline; the instrumentor you enable afterwards picks up that provider automatically.

   from confident_trace import init, shutdown
   from openinference.instrumentation.langchain import LangChainInstrumentor
   from langchain_openai import ChatOpenAI
   
   init(instrumentations=())
   LangChainInstrumentor().instrument()
   
   try:
       llm = ChatOpenAI(model="gpt-4.1-mini")
       print(llm.invoke("What are LLMs?").content)
   finally:
       shutdown()

4. Run Your Code

   PythonTypeScript

   python main.py

   Done ✅. You can view the traces on Confident AI's traces page inside the Observatory.

What Gets Captured

Whatever the OpenInference instrumentor emits. confident-trace acts as a pass-through exporter:

• Preserved as-is — the instrumentor's span names, attributes, and events are exported exactly as produced.
• Hierarchy — spans keep the parent/child structure the instrumentor reports; a span without a parent starts a new trace.
• No rewriting — OpenInference attributes are not converted to the GenAI conventions that Confident's own integrations use. Confident AI's platform-side OpenInference mapping is what decides how each span is displayed as an LLM, tool, or agent span and how its content is extracted.

Available Instrumentors

Any OpenInference instrumentor that emits through the global OpenTelemetry provider works the same way. Common packages include:

Install and enable each one as documented by OpenInference.

Set Trace Span Properties

Use a trace context to add properties you know before the call starts. It creates no extra span; the trace started by the instrumented framework or provider call inherits the tags, metadata, and user ID.

from confident_trace import init, trace_context
from langchain_openai import ChatOpenAI

init(instrumentations=())

llm = ChatOpenAI(model="gpt-4.1-mini")

with trace_context(
    tags=["support"],
    metadata={"instrumentor": "openinference"},
    user_id="user-42",
):
    result = llm.invoke("Explain OpenTelemetry in one sentence.")

import { init, traceContext } from "confident-trace";

init({ instrumentations: [] });

const result = await traceContext(
  { tags: ["support"], metadata: { instrumentor: "openinference" }, userId: "user-42" },
  () => client.responses.create({
    model: "gpt-4.1-mini",
    input: "Explain OpenTelemetry in one sentence.",
  }),
);

See trace context for every supported trace property and update behavior.

Instrumenting Multi-Turn

You do not need turn() when one OpenInference entry-point call is already one conversational turn—the integration creates that turn's trace automatically. Use turn() when you want to define the boundary yourself, such as grouping two sequential OpenInference calls into one turn. Reuse the same thread ID on later turns to group them into one conversation.

from confident_trace import init, turn

init(instrumentations=())

with turn("support-turn", thread_id="chat-42"):
    context = llm.invoke("Find the relevant account details.")
    answer = llm.invoke(f"Summarize these details: {context.content}")

import { init, turn } from "confident-trace";

init({ instrumentations: [] });

const answer = await turn({ name: "support-turn", threadId: "chat-42" }, async () => {
  const context = await client.responses.create({ model: "gpt-4.1-mini", input: "Find the relevant account details." });
  return client.responses.create({ model: "gpt-4.1-mini", input: `Summarize these details: ${context.output_text}` });
});

See threads for thread I/O, turn IDs, and user IDs.

Disable OpenInference Instrumentation

Pass init() a list of identifiers to opt in to Confident AI integrations. OpenInference has no init() identifier because you register its instrumentor separately; use an empty list and do not register the OpenInference instrumentor to keep it disabled:

from confident_trace import init
init(instrumentations=())
# OpenInference has no identifier; do not register its instrumentor.

import { init } from "confident-trace";
init({ instrumentations: [] });
// OpenInference has no identifier; do not register its instrumentor.

This disables Confident AI's automatic instrumentors. Do not register an OpenInference instrumentor afterward.

Next Steps

Now that your OpenInference spans are landing on Confident AI, dive deeper into:

Last updated on September 10, 2026