Bifrost

Overview

Bifrost is an AI gateway that routes model requests through OpenAI- and Anthropic-compatible endpoints. Confident AI traces and evaluates your Bifrost calls through confident-trace, Confident AI's OpenTelemetry-native tracing SDK for Python and TypeScript.

The integration captures the following data from supported calls:

• LLM spans — model names, timing, status, and token usage
• Messages — input/output messages and tool-call data returned by the model
• Streaming output — response content as your application consumes it

Auto-Instrument

1. Install Dependencies

   Install confident-trace alongside the client used in the examples:

   PythonTypeScript

   pip install confident-trace

2. Set Your API Keys

   Get your project API key from Confident AI and set the credentials for your model client:

   export CONFIDENT_API_KEY="<your-confident-project-key>"
   export BIFROST_API_KEY="<your-gateway-key>"
   export BIFROST_BASE_URL="http://localhost:8080/openai"

3. Instrument Bifrost

   Call init() once and register the exact gateway base URL used by your client. Requests keep their normal SDK shape.

   PythonTypeScript

   import os
   from confident_trace import init, shutdown
   from openai import OpenAI
   
   base_url = os.environ["BIFROST_BASE_URL"]
   init(bifrost_proxy_urls=[base_url])
   client = OpenAI(base_url=base_url, api_key=os.environ["BIFROST_API_KEY"])
   
   try:
       response = client.chat.completions.create(
           model="openai/gpt-4o-mini",
           messages=[
               {"role": "user", "content": "Explain OpenTelemetry in one sentence."}
           ],
       )
       print(response.choices[0].message.content)
   finally:
       shutdown()

4. Run Bifrost

   PythonTypeScript

   python main.py

   Done ✅. Open the Observatory in your Confident AI project to inspect the trace and its model-call spans.

Anthropic-Compatible Clients

The integration also supports Anthropic Messages create and stream. Install anthropic>=0.69,<2 for Python or @anthropic-ai/sdk>=0.124.0 <0.125 for TypeScript alongside confident-trace. These snippets replace the quickstart’s client setup; keep the same startup and shutdown lifecycle.

export BIFROST_ANTHROPIC_BASE_URL="http://localhost:8080/anthropic"

import os
from anthropic import Anthropic
from confident_trace import init

base_url = os.environ["BIFROST_ANTHROPIC_BASE_URL"]
api_key = os.environ["BIFROST_API_KEY"]
init(bifrost_proxy_urls=[base_url])
client = Anthropic(
    base_url=base_url,
    api_key=api_key,
)

import Anthropic from "@anthropic-ai/sdk";
import { init } from "confident-trace";

const baseURL = process.env.BIFROST_ANTHROPIC_BASE_URL!;
const apiKey = process.env.BIFROST_API_KEY!;
const runtime = init({ bifrostProxyUrls: [baseURL] });
const client = new Anthropic({
  baseURL,
  apiKey,
});

Call client.messages.create with model, max_tokens, and messages, or use client.messages.stream. Use a model configured for this endpoint. If you use both SDKs, list both client base URLs in the same init() call. Matching uses exact base URLs; there is no automatic localhost detection.

For manual TypeScript setup, use instrumentOpenAI or instrumentAnthropic from the matching confident-trace subpath, passing { bifrostProxyUrls: [baseURL] }. Initialize export with init(); manual adapters do not require the preload.

What Gets Captured

Each supported application call creates a model-call span. It nests under the active span; without a parent, it starts a new trace.

• Supported APIs — OpenAI Chat Completions and Responses create, plus Anthropic Messages create and stream, including Python sync/async calls. Google GenAI/Bedrock gateway detection, embeddings, and background inference polling lifecycles are outside this integration.
• Model and usage — the requested model or alias, the returned model when available, response ID, finish reasons, and token counts supplied by the gateway
• Messages and tools — normalized input/output messages and tool-call data; executing a tool is separate work and is not traced by this gateway integration
• Gateway identity — Spans retain their OpenAI or Anthropic integration label and record gateway/provider identity as bifrost.

Messages follow the content policy, including capture opt-out, redaction, and configured size limits. Binary multimodal payloads are omitted.

Streaming

Streaming uses the same setup. Each example includes initialization and shutdown:

import os
from confident_trace import init, shutdown
from openai import OpenAI

base_url = os.environ["BIFROST_BASE_URL"]
init(bifrost_proxy_urls=[base_url])
client = OpenAI(base_url=base_url, api_key=os.environ["BIFROST_API_KEY"])

try:
    stream = client.chat.completions.create(
        model="openai/gpt-4o-mini",
        messages=[{"role": "user", "content": "Tell me a short story."}],
        stream=True,
    )
    try:
        for chunk in stream:
            print(chunk)
    finally:
        stream.close()
finally:
    shutdown()

import OpenAI from "openai";
import { init } from "confident-trace";

const baseURL = process.env.BIFROST_BASE_URL!;
const runtime = init({ bifrostProxyUrls: [baseURL] });
const client = new OpenAI({ baseURL, apiKey: process.env.BIFROST_API_KEY! });

try {
  const stream = await client.chat.completions.create({
    model: "openai/gpt-4o-mini",
    messages: [{ role: "user", content: "Tell me a short story." }],
    stream: true,
  });
  for await (const chunk of stream) {
    console.log(chunk);
  }
} finally {
  await runtime.shutdown();
}

For Python async code, use AsyncOpenAI or AsyncAnthropic and await the corresponding method; consume streams with async for.

Set Trace Span Properties

Use a trace context to add properties you know before the call starts. It creates no extra span. Each example initializes tracing and creates its client before making the call.

import os
from confident_trace import init, shutdown, trace_context
from openai import OpenAI

base_url = os.environ["BIFROST_BASE_URL"]
init(bifrost_proxy_urls=[base_url])
client = OpenAI(base_url=base_url, api_key=os.environ["BIFROST_API_KEY"])

try:
    with trace_context(
        tags=["support"],
        metadata={"gateway": "bifrost"},
        user_id="user-42",
    ):
        response = client.chat.completions.create(
            model="openai/gpt-4o-mini",
            messages=[
                {"role": "user", "content": "Explain OpenTelemetry in one sentence."}
            ],
        )
finally:
    shutdown()

import OpenAI from "openai";
import { init, traceContext } from "confident-trace";

const baseURL = process.env.BIFROST_BASE_URL!;
const runtime = init({ bifrostProxyUrls: [baseURL] });
const client = new OpenAI({ baseURL, apiKey: process.env.BIFROST_API_KEY! });

try {
  const response = await traceContext(
    { tags: ["support"], metadata: { gateway: "bifrost" }, userId: "user-42" },
    () =>
      client.chat.completions.create({
        model: "openai/gpt-4o-mini",
        messages: [{ role: "user", content: "Explain OpenTelemetry in one sentence." }],
      }),
  );
} finally {
  await runtime.shutdown();
}

See trace context for every supported property.

Instrumenting Multi-Turn

You do not need turn() just to trace a single model call. Use it when you want to define a conversational boundary, such as grouping two calls into one turn. Reuse the same thread ID on later turns to group them into one conversation.

Each example includes initialization and shutdown. Use the same environment variables as the quickstart.

import os
from confident_trace import init, shutdown, turn
from openai import OpenAI

base_url = os.environ["BIFROST_BASE_URL"]
init(bifrost_proxy_urls=[base_url])
client = OpenAI(base_url=base_url, api_key=os.environ["BIFROST_API_KEY"])

try:
    with turn("support-turn", thread_id="chat-42"):
        context = client.chat.completions.create(
            model="openai/gpt-4o-mini",
            messages=[
                {
                    "role": "user",
                    "content": "List two useful facts about OpenTelemetry.",
                }
            ],
        )
        answer = client.chat.completions.create(
            model="openai/gpt-4o-mini",
            messages=[
                {
                    "role": "user",
                    "content": f"Summarize these facts: {context.choices[0].message.content}",
                }
            ],
        )
finally:
    shutdown()

import OpenAI from "openai";
import { init, turn } from "confident-trace";

const baseURL = process.env.BIFROST_BASE_URL!;
const runtime = init({ bifrostProxyUrls: [baseURL] });
const client = new OpenAI({ baseURL, apiKey: process.env.BIFROST_API_KEY! });

try {
  const answer = await turn({ name: "support-turn", threadId: "chat-42" }, async () => {
    const context = await client.chat.completions.create({
      model: "openai/gpt-4o-mini",
      messages: [
        { role: "user", content: "List two useful facts about OpenTelemetry." },
      ],
    });
    return client.chat.completions.create({
      model: "openai/gpt-4o-mini",
      messages: [
        { role: "user", content: `Summarize these facts: ${JSON.stringify(context)}` },
      ],
    });
  });
} finally {
  await runtime.shutdown();
}

See threads for conversation grouping and turn properties.

Troubleshooting

• Missing content: check capture settings and the supported API list above. Gateway-side exporters have separate content controls.
• Duplicate records: avoid combining multiple instrumentors for the same client call. Client instrumentation and gateway-side export can also report separate views of one request.

For general setup issues, see troubleshooting.

Disable Bifrost Instrumentation

Pass init() a list of integration identifiers to opt in to only those integrations. The quickstart uses "openai" in Python and "openai" in TypeScript. Anthropic clients use "anthropic". There is no separate "bifrost" integration selector. An empty list disables all automatic instrumentation:

from confident_trace import init

init(instrumentations=())
# To enable only the quickstart integration: instrumentations=("openai",)

import { init } from "confident-trace";

init({ instrumentations: [] });
// To enable only the quickstart integration: instrumentations: ["openai"]

Apply the selection to your startup init() call. It controls SDK hooks, not individual gateway hosts, and does not change a gateway’s own export settings. Manually installed TypeScript adapters have their own restoration function.

Next Steps

Last updated on September 11, 2026