LLM Tracing Quickstart

Overview

This guide shows you how to instrument your LLM app with confident-trace, Confident AI's OpenTelemetry-native tracing SDK for Python and TypeScript. You'll install the package, call init() once, and see your first trace in the Observatory — usually in under five minutes.

How it works

Tracing works through instrumentation, which is either automatic (via confident-trace's built-in integrations) or manual (via the span decorator/wrapper):

1. Call init() once when your app starts — it detects every supported SDK and framework you have installed and instruments them for you
2. Each LLM call, retrieval, tool execution, or span-wrapped function becomes a span
3. The outermost span becomes the trace — all nested spans roll up into it (see troubleshooting if spans are creating separate traces instead of nesting)
4. Spans are exported to Confident AI asynchronously in batches, with zero latency impact on your app
5. Once ingested, traces can be evaluated automatically using your configured metrics

You should also understand the terminology for tracing:

Vibe Code Your Tracing

Let your coding agent instrument your app for you — it picks the right path (confident-trace auto-instrumentation, a custom span, or OpenTelemetry) based on what your project uses and what you ask. Choose the install method for your agent below.

/plugin marketplace add confident-ai/confident-trace
/plugin install confident-trace@confident-trace-plugins
/reload-plugins
/plugins

npx skills add confident-ai/confident-trace --skill confident-tracing

Once installed, open the project you want to trace and tell your agent what you need. Example prompts:

• "Instrument this app with confident-trace and send traces to Confident AI."
• "Add tracing so my OpenAI calls show up on Confident AI."
• "I'm using LangGraph — wire up Confident AI tracing for it."

Your agent will read the codebase, choose between a native integration and manual spans, and confirm traces land in the Observatory.

Instrument Your AI App

1. Install confident-trace

   Instrumentation is done in code, so first install confident-trace. The examples below also use the OpenAI SDK, so install that too if you want to follow along:

   PythonTypeScript

   pip install confident-trace

2. Set Your API Key

   Get your Confident AI Project API key and set it as an environment variable:

   export CONFIDENT_API_KEY=YOUR-API-KEY

   Your API key alone does not decide where traces go. If you're on the EU region or a self-hosted deployment, also set the OTEL endpoint, or your traces will be sent to our US servers:

   export CONFIDENT_OTEL_ENDPOINT="https://eu.otel.confident-ai.com/v1/traces"

3. Instrument Your App

   Call init() once at the entry point of your application:

   PythonTypeScript

   from openai import OpenAI
   from confident_trace import init, shutdown
   
   init()
   client = OpenAI()
   
   def llm_app(query: str) -> str:
       return client.chat.completions.create(
           model="gpt-4o",
           messages=[{"role": "user", "content": query}]
       ).choices[0].message.content
   
   # Call app to send trace to Confident AI
   try:
       llm_app("Write me a poem.")
   finally:
       shutdown()

   Done ✅. You just created a trace with an LLM span inside it. Go to the Observatory to see your traces there.

   If you don't see the trace, it is 99.99% because your program exited before the traces had a chance to get posted. Make sure you're calling shutdown() (or flush() in long-running processes) before exit:

   from confident_trace import flush
   
   flush()  # blocks until the local queue is drained

   See the troubleshooting page for more details.

Configure init()

init() is the single place tracing is configured. With no arguments it reads everything from the environment, which is why the example above only needed CONFIDENT_API_KEY. Each core setting can be supplied as an argument or as a CONFIDENT_* environment variable:

Explicit arguments always win over environment variables. For deployments, environment variables are usually the way to go:

export CONFIDENT_API_KEY="<your-confident-project-key>"
export CONFIDENT_OTEL_ENDPOINT="https://eu.otel.confident-ai.com/v1/traces"  # EU or self-hosted only
export CONFIDENT_SAMPLE_RATE="0.5"
export CONFIDENT_ENVIRONMENT="production"
export CONFIDENT_OTEL_TLS_SKIP_VERIFY="false"  # true only for temporary POC troubleshooting

Or pass them explicitly when the values come from your own config system:

from confident_trace import init

init(
    api_key=settings.confident_api_key,
    endpoint="https://eu.otel.confident-ai.com/v1/traces",
    sample_rate=0.5,
    environment="production",
    tls_skip_verify=False,
)

import { init } from "confident-trace";

const runtime = init({
  apiKey: settings.confidentApiKey,
  endpoint: "https://eu.otel.confident-ai.com/v1/traces",
  sampleRate: 0.5,
  environment: "production",
  tlsSkipVerify: false,
});

• Sample rate is a head-sampling ratio between 0 and 1, decided once at the root of each trace. Child spans follow their parent, so a trace is either fully exported or not at all — you'll never see half a trace. See sampling.
• Environment stamps every trace from this process (e.g. production, staging, development) so you can filter the Observatory by deployment. Override it for a single trace with update_trace(environment=...) / updateTrace({ environment }). See environment.
• Endpoint only needs changing for the EU region, a self-hosted deployment, or your own OpenTelemetry collector. The default US endpoint works out of the box.

Flush and shutdown

Spans are exported in batches from a background worker, so if your process exits before the queue drains you'll lose the tail end of your traces. Two helpers handle this:

from confident_trace import flush, shutdown

flush(timeout_millis=30000)   # drain the queue, keep running
shutdown(timeout_millis=5000) # at process exit

await runtime.flush(30000);   // drain the queue, keep running
await runtime.shutdown(5000); // at process exit

• flush() is for long-running processes and serverless functions — call it at the end of a request handler (after any streams have finished) so traces are sent before the environment freezes. Your app keeps running afterwards.
• shutdown() is for process exit — it flushes, then tears down the exporter. Call it once, not per request.

SSL/TLS certificate verification

Sometimes your organization's network routes HTTPS traffic through a corporate proxy that uses an internal certificate authority, or your self-hosted POC uses a self-signed certificate. If your app doesn't trust that certificate, trace exports can fail with an SSL/TLS certificate verification error. To temporarily bypass certificate verification and verify that tracing works, set this environment variable before starting your app:

export CONFIDENT_OTEL_TLS_SKIP_VERIFY="true"

Or pass the option directly to your existing init() call:

from confident_trace import init

init(tls_skip_verify=True)

import { init } from "confident-trace";

const runtime = init({ tlsSkipVerify: true });

Explicit arguments override the environment variable, including False in Python or false in TypeScript to keep verification enabled. Only the exact environment value "true" enables the bypass. Configure this before initialization, then restart your app to apply changes.

This applies to confident-trace's built-in HTTP/protobuf trace exports across integrations and LiveKit recording uploads. It does not change gRPC, custom exporters, or your app's connections to model providers.

Instrument Multi-Turn Apps

If your app handles conversations or multi-turn interactions, you can group traces into a thread by providing a thread ID. Each turn creates its own trace, and traces with the same thread ID are grouped together as a conversation.

Suppose each request to your server is a turn in your conversational agent. Wrap the handler's work in turn() with the thread ID from the request — it starts a fresh trace for that turn, stamps the thread ID on it, and anything you call inside (including auto-instrumented LLM calls) nests under it:

from fastapi import FastAPI
from openai import OpenAI
from confident_trace import init, turn, update_trace

init()
app = FastAPI()
client = OpenAI()

@app.post("/chat")
def chat(thread_id: str, query: str):
    with turn("chat", thread_id=thread_id):
        res = client.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": query}],
        ).choices[0].message.content
        update_trace(input=query, output=res)
        return {"output": res}

import express from "express";
import OpenAI from "openai";
import { init, turn, updateTrace } from "confident-trace";

init();
const app = express().use(express.json());
const openai = new OpenAI();

app.post("/chat", async (req, res) => {
  const { threadId, query } = req.body;
  const output = await turn({ name: "chat", threadId }, async () => {
    const completion = await openai.chat.completions.create({
      model: "gpt-4o",
      messages: [{ role: "user", content: query }],
    });
    const output = completion.choices[0].message.content;
    updateTrace({ input: query, output });
    return output;
  });
  res.json({ output });
});

app.listen(3000);

Every request that comes in with the same thread_id / threadId lands in the same thread, so two calls to /chat for "What's the weather in SF?" and "What about tomorrow?" show up as one conversation with two turns. The thread ID can be any string — typically a session or conversation ID your app already has.

Next Steps

Now that you've learnt the very basics of instrumenting your AI app, dive deeper into:

Last updated on October 9, 2026